2024-09-19T08:26:20.983 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-19T08:26:21.019 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T08:26:21.117 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911859 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/iogen}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7911859' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1747 sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 sleep_before_teardown: 0 subset: 176/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi164.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNcwapRZgKShZSH9AXovbg8xjyyN+U8o1gZ/GM0jxA8zvOFgMuAeYHDSYiwg9nGem4LfXgrsVrFULIiDSArJDEs= smithi170.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIBneZyibae61eofp11Kn9hQv1nvTn6y09h/0ClRhnYgtaBNtpEXFZ1jei7otzQhEKj9nKetTTVhMOAPO2yQ1ZI= smithi173.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAs9JVDLlHFu29BV7TlvKgIEBqqDFG7sFL/xGHFS7TETct/y4WCoqrYBcYuiIJIZTF16t/1rFZ2igeGoL6V1qwA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - quiescer: null - workunit: clients: all: - suites/iogen.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-18_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-19T08:26:21.117 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa; will attempt to use it 2024-09-19T08:26:21.119 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks 2024-09-19T08:26:21.119 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-19T08:26:21.120 INFO:teuthology.task.internal:Checking packages... 2024-09-19T08:26:21.148 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '2f599a46ad10e37c929e5c43da2f6907ddcd1961' 2024-09-19T08:26:21.148 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-19T08:26:21.148 INFO:teuthology.packaging:ref: None 2024-09-19T08:26:21.148 INFO:teuthology.packaging:tag: None 2024-09-19T08:26:21.148 INFO:teuthology.packaging:branch: squid 2024-09-19T08:26:21.148 INFO:teuthology.packaging:sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:26:21.148 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-19T08:26:21.357 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-363.g6fad5dd7 2024-09-19T08:26:21.359 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-19T08:26:21.365 INFO:teuthology.task.internal:no buildpackages task found 2024-09-19T08:26:21.366 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-19T08:26:21.373 INFO:teuthology.task.internal:Saving configuration 2024-09-19T08:26:21.393 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-19T08:26:21.400 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-19T08:26:21.425 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911859', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 08:21:16.018995', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNcwapRZgKShZSH9AXovbg8xjyyN+U8o1gZ/GM0jxA8zvOFgMuAeYHDSYiwg9nGem4LfXgrsVrFULIiDSArJDEs='} 2024-09-19T08:26:21.449 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi170.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911859', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 08:21:16.022709', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIBneZyibae61eofp11Kn9hQv1nvTn6y09h/0ClRhnYgtaBNtpEXFZ1jei7otzQhEKj9nKetTTVhMOAPO2yQ1ZI='} 2024-09-19T08:26:21.474 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi173.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911859', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 08:21:16.020969', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAs9JVDLlHFu29BV7TlvKgIEBqqDFG7sFL/xGHFS7TETct/y4WCoqrYBcYuiIJIZTF16t/1rFZ2igeGoL6V1qwA='} 2024-09-19T08:26:21.474 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-19T08:26:21.481 INFO:teuthology.task.internal:roles: ubuntu@smithi164.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-19T08:26:21.482 INFO:teuthology.task.internal:roles: ubuntu@smithi170.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-19T08:26:21.482 INFO:teuthology.task.internal:roles: ubuntu@smithi173.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-19T08:26:21.482 INFO:teuthology.run_tasks:Running task console_log... 2024-09-19T08:26:21.560 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8f56397be0>, signals=[15]) 2024-09-19T08:26:21.560 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-19T08:26:21.566 INFO:teuthology.task.internal:Opening connections... 2024-09-19T08:26:21.566 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi164.front.sepia.ceph.com 2024-09-19T08:26:21.567 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T08:26:21.645 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi170.front.sepia.ceph.com 2024-09-19T08:26:21.646 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T08:26:21.723 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi173.front.sepia.ceph.com 2024-09-19T08:26:21.724 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T08:26:21.797 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-19T08:26:21.804 DEBUG:teuthology.orchestra.run.smithi164:> uname -m 2024-09-19T08:26:21.823 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2024-09-19T08:26:21.823 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release 2024-09-19T08:26:21.882 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Stream" 2024-09-19T08:26:21.882 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="9" 2024-09-19T08:26:21.882 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos" 2024-09-19T08:26:21.882 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora" 2024-09-19T08:26:21.882 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="9" 2024-09-19T08:26:21.882 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el9" 2024-09-19T08:26:21.882 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T08:26:21.882 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31" 2024-09-19T08:26:21.882 INFO:teuthology.orchestra.run.smithi164.stdout:LOGO="fedora-logo-icon" 2024-09-19T08:26:21.882 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T08:26:21.882 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://centos.org/" 2024-09-19T08:26:21.883 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T08:26:21.883 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T08:26:21.883 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T08:26:21.883 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2024-09-19T08:26:21.907 DEBUG:teuthology.orchestra.run.smithi170:> uname -m 2024-09-19T08:26:21.927 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64 2024-09-19T08:26:21.927 DEBUG:teuthology.orchestra.run.smithi170:> cat /etc/os-release 2024-09-19T08:26:21.985 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="CentOS Stream" 2024-09-19T08:26:21.986 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="9" 2024-09-19T08:26:21.986 INFO:teuthology.orchestra.run.smithi170.stdout:ID="centos" 2024-09-19T08:26:21.986 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE="rhel fedora" 2024-09-19T08:26:21.986 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="9" 2024-09-19T08:26:21.986 INFO:teuthology.orchestra.run.smithi170.stdout:PLATFORM_ID="platform:el9" 2024-09-19T08:26:21.986 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T08:26:21.986 INFO:teuthology.orchestra.run.smithi170.stdout:ANSI_COLOR="0;31" 2024-09-19T08:26:21.986 INFO:teuthology.orchestra.run.smithi170.stdout:LOGO="fedora-logo-icon" 2024-09-19T08:26:21.986 INFO:teuthology.orchestra.run.smithi170.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T08:26:21.987 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://centos.org/" 2024-09-19T08:26:21.987 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T08:26:21.987 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T08:26:21.987 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T08:26:21.987 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2024-09-19T08:26:22.017 DEBUG:teuthology.orchestra.run.smithi173:> uname -m 2024-09-19T08:26:22.040 INFO:teuthology.orchestra.run.smithi173.stdout:x86_64 2024-09-19T08:26:22.042 DEBUG:teuthology.orchestra.run.smithi173:> cat /etc/os-release 2024-09-19T08:26:22.104 INFO:teuthology.orchestra.run.smithi173.stdout:NAME="CentOS Stream" 2024-09-19T08:26:22.104 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION="9" 2024-09-19T08:26:22.104 INFO:teuthology.orchestra.run.smithi173.stdout:ID="centos" 2024-09-19T08:26:22.104 INFO:teuthology.orchestra.run.smithi173.stdout:ID_LIKE="rhel fedora" 2024-09-19T08:26:22.104 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_ID="9" 2024-09-19T08:26:22.104 INFO:teuthology.orchestra.run.smithi173.stdout:PLATFORM_ID="platform:el9" 2024-09-19T08:26:22.104 INFO:teuthology.orchestra.run.smithi173.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T08:26:22.104 INFO:teuthology.orchestra.run.smithi173.stdout:ANSI_COLOR="0;31" 2024-09-19T08:26:22.104 INFO:teuthology.orchestra.run.smithi173.stdout:LOGO="fedora-logo-icon" 2024-09-19T08:26:22.105 INFO:teuthology.orchestra.run.smithi173.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T08:26:22.105 INFO:teuthology.orchestra.run.smithi173.stdout:HOME_URL="https://centos.org/" 2024-09-19T08:26:22.105 INFO:teuthology.orchestra.run.smithi173.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T08:26:22.105 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T08:26:22.105 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T08:26:22.105 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server 2024-09-19T08:26:22.154 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-19T08:26:22.257 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-19T08:26:22.262 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-19T08:26:22.262 DEBUG:teuthology.orchestra.run.smithi164:> test '!' -e /home/ubuntu/cephtest 2024-09-19T08:26:22.264 DEBUG:teuthology.orchestra.run.smithi170:> test '!' -e /home/ubuntu/cephtest 2024-09-19T08:26:22.267 DEBUG:teuthology.orchestra.run.smithi173:> test '!' -e /home/ubuntu/cephtest 2024-09-19T08:26:22.284 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-19T08:26:22.290 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-19T08:26:22.291 DEBUG:teuthology.orchestra.run.smithi164:> test -z $(ls -A /var/lib/ceph) 2024-09-19T08:26:22.322 DEBUG:teuthology.orchestra.run.smithi170:> test -z $(ls -A /var/lib/ceph) 2024-09-19T08:26:22.324 DEBUG:teuthology.orchestra.run.smithi173:> test -z $(ls -A /var/lib/ceph) 2024-09-19T08:26:22.431 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-19T08:26:22.508 INFO:teuthology.run_tasks:Running task kernel... 2024-09-19T08:26:22.523 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-19T08:26:22.523 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-19T08:26:22.523 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-19T08:26:22.523 DEBUG:teuthology.orchestra.run.smithi164:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T08:26:22.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:26:22.541 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2024-09-19T08:26:22.599 INFO:teuthology.orchestra.run.smithi164.stdout:5.14.0-503.el9.x86_64 2024-09-19T08:26:22.599 INFO:teuthology.task.kernel:Running kernel on smithi164: 5.14.0-503.el9.x86_64 2024-09-19T08:26:22.599 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum install -y kernel 2024-09-19T08:26:27.318 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - BaseOS 2.8 MB/s | 8.3 MB 00:02 2024-09-19T08:26:29.110 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - AppStream 28 MB/s | 20 MB 00:00 2024-09-19T08:26:34.594 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.5 MB 00:01 2024-09-19T08:26:36.261 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 19 kB 00:00 2024-09-19T08:26:37.415 INFO:teuthology.orchestra.run.smithi164.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-19T08:26:42.779 INFO:teuthology.orchestra.run.smithi164.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-19T08:26:44.744 INFO:teuthology.orchestra.run.smithi164.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-19T08:26:44.744 INFO:teuthology.orchestra.run.smithi164.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-19T08:26:44.846 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-19T08:26:44.854 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-19T08:26:44.854 INFO:teuthology.orchestra.run.smithi164.stdout: Package Architecture Version Repository Size 2024-09-19T08:26:44.855 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-19T08:26:44.855 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-19T08:26:44.855 INFO:teuthology.orchestra.run.smithi164.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-19T08:26:44.855 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-19T08:26:44.855 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-19T08:26:44.855 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-19T08:26:44.855 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-19T08:26:44.855 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:26:44.855 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-19T08:26:44.855 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-19T08:26:44.855 INFO:teuthology.orchestra.run.smithi164.stdout:Install 4 Packages 2024-09-19T08:26:44.856 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:26:44.856 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 79 M 2024-09-19T08:26:44.856 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 126 M 2024-09-19T08:26:44.856 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-19T08:26:45.239 INFO:teuthology.orchestra.run.smithi164.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 277 kB/s | 36 kB 00:00 2024-09-19T08:26:46.314 INFO:teuthology.orchestra.run.smithi164.stdout:(2/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 29 MB/s | 35 MB 00:01 2024-09-19T08:26:46.856 INFO:teuthology.orchestra.run.smithi164.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 18 MB/s | 29 MB 00:01 2024-09-19T08:26:47.098 INFO:teuthology.orchestra.run.smithi164.stdout:(4/4): kernel-core-5.14.0-508.el9.x86_64.rpm 7.9 MB/s | 16 MB 00:01 2024-09-19T08:26:47.098 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-19T08:26:47.098 INFO:teuthology.orchestra.run.smithi164.stdout:Total 35 MB/s | 79 MB 00:02 2024-09-19T08:26:47.777 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-19T08:26:48.138 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-19T08:26:48.139 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-19T08:26:49.102 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-19T08:26:49.102 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-19T08:26:51.532 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-19T08:26:52.206 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-19T08:26:52.289 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T08:26:55.068 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T08:26:55.209 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T08:27:00.036 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T08:27:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T08:27:05.046 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T08:27:43.459 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T08:27:43.499 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-19T08:27:43.796 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T08:27:43.796 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-19T08:27:43.796 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T08:27:43.797 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T08:27:44.674 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T08:27:44.674 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:27:44.674 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-19T08:27:44.674 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-19T08:27:44.674 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-19T08:27:44.675 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-19T08:27:44.675 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-19T08:27:44.675 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:27:44.675 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-19T08:27:45.063 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T08:27:45.149 INFO:teuthology.orchestra.run.smithi164.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T08:27:45.150 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T08:27:45.150 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-19T08:27:45.150 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-19T08:27:45.150 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-19T08:27:45.150 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-19T08:27:45.150 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi164.front.sepia.ceph.com, path=None, version=distro) 2024-09-19T08:27:45.150 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T08:27:45.209 INFO:teuthology.orchestra.run.smithi164.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T08:27:45.209 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T08:27:45.209 DEBUG:teuthology.orchestra.run.smithi164:> sudo rpm -qi grub2-tools 2024-09-19T08:27:45.247 INFO:teuthology.orchestra.run.smithi164.stdout:Name : grub2-tools 2024-09-19T08:27:45.247 INFO:teuthology.orchestra.run.smithi164.stdout:Epoch : 1 2024-09-19T08:27:45.247 INFO:teuthology.orchestra.run.smithi164.stdout:Version : 2.06 2024-09-19T08:27:45.247 INFO:teuthology.orchestra.run.smithi164.stdout:Release : 82.el9 2024-09-19T08:27:45.247 INFO:teuthology.orchestra.run.smithi164.stdout:Architecture: x86_64 2024-09-19T08:27:45.247 INFO:teuthology.orchestra.run.smithi164.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-19T08:27:45.248 INFO:teuthology.orchestra.run.smithi164.stdout:Group : Unspecified 2024-09-19T08:27:45.248 INFO:teuthology.orchestra.run.smithi164.stdout:Size : 8274670 2024-09-19T08:27:45.248 INFO:teuthology.orchestra.run.smithi164.stdout:License : GPLv3+ 2024-09-19T08:27:45.248 INFO:teuthology.orchestra.run.smithi164.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-19T08:27:45.248 INFO:teuthology.orchestra.run.smithi164.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-19T08:27:45.248 INFO:teuthology.orchestra.run.smithi164.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-19T08:27:45.248 INFO:teuthology.orchestra.run.smithi164.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-19T08:27:45.248 INFO:teuthology.orchestra.run.smithi164.stdout:Packager : builder@centos.org 2024-09-19T08:27:45.248 INFO:teuthology.orchestra.run.smithi164.stdout:Vendor : CentOS 2024-09-19T08:27:45.248 INFO:teuthology.orchestra.run.smithi164.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-19T08:27:45.248 INFO:teuthology.orchestra.run.smithi164.stdout:Summary : Support tools for GRUB. 2024-09-19T08:27:45.248 INFO:teuthology.orchestra.run.smithi164.stdout:Description : 2024-09-19T08:27:45.249 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:27:45.249 INFO:teuthology.orchestra.run.smithi164.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-19T08:27:45.249 INFO:teuthology.orchestra.run.smithi164.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-19T08:27:45.249 INFO:teuthology.orchestra.run.smithi164.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-19T08:27:45.249 INFO:teuthology.orchestra.run.smithi164.stdout:hardware devices. 2024-09-19T08:27:45.249 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:27:45.249 INFO:teuthology.orchestra.run.smithi164.stdout:This subpackage provides tools for support of all platforms. 2024-09-19T08:27:45.250 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-19T08:27:45.250 INFO:teuthology.task.kernel:Updating grub on smithi164 to boot 5.14.0-508.el9.x86_64 2024-09-19T08:27:45.250 DEBUG:teuthology.orchestra.run.smithi164:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-19T08:27:45.907 INFO:teuthology.orchestra.run.smithi164.stderr:Generating grub configuration file ... 2024-09-19T08:27:47.265 INFO:teuthology.orchestra.run.smithi164.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-19T08:27:47.300 INFO:teuthology.orchestra.run.smithi164.stderr:done 2024-09-19T08:27:47.303 DEBUG:teuthology.orchestra.run.smithi164:> mktemp 2024-09-19T08:27:47.318 INFO:teuthology.orchestra.run.smithi164.stdout:/tmp/tmp.D5HD41Tao7 2024-09-19T08:27:47.318 DEBUG:teuthology.orchestra.run.smithi164:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.D5HD41Tao7 2024-09-19T08:27:47.384 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0666 /tmp/tmp.D5HD41Tao7 2024-09-19T08:27:47.524 DEBUG:teuthology.orchestra.remote:smithi164:/tmp/tmp.D5HD41Tao7 is 6KB 2024-09-19T08:27:47.573 DEBUG:teuthology.orchestra.run.smithi164:> rm -fr /tmp/tmp.D5HD41Tao7 2024-09-19T08:27:47.589 DEBUG:teuthology.orchestra.run.smithi164:> sudo /bin/ls /boot/loader/entries || true 2024-09-19T08:27:47.651 INFO:teuthology.orchestra.run.smithi164.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-19T08:27:47.651 INFO:teuthology.orchestra.run.smithi164.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-19T08:27:47.651 INFO:teuthology.orchestra.run.smithi164.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-19T08:27:47.651 INFO:teuthology.orchestra.run.smithi164.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-19T08:27:47.653 DEBUG:teuthology.orchestra.run.smithi164:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-19T08:27:47.792 DEBUG:teuthology.orchestra.run.smithi164:> sudo shutdown -r now 2024-09-19T08:28:17.795 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-19T08:28:17.796 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-19T08:28:17.797 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T08:28:52.413 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-09-19T08:29:01.415 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-19T08:29:01.416 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T08:29:19.805 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-09-19T08:29:31.806 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-19T08:29:31.807 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T08:29:32.197 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-09-19T08:29:32.566 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-19T08:29:32.566 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-508.el9.x86_64"... 2024-09-19T08:29:32.566 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2024-09-19T08:29:32.581 INFO:teuthology.orchestra.run.smithi164.stdout:5.14.0-508.el9.x86_64 2024-09-19T08:29:32.581 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-19T08:29:32.582 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T08:29:32.582 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-19T08:29:33.583 DEBUG:teuthology.parallel:result is None 2024-09-19T08:29:33.583 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-19T08:29:33.591 INFO:teuthology.task.internal:Creating test directory... 2024-09-19T08:29:33.591 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T08:29:33.594 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T08:29:33.596 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T08:29:33.621 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-19T08:29:33.628 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-19T08:29:33.636 INFO:teuthology.task.internal:Creating archive directory... 2024-09-19T08:29:33.636 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T08:29:33.651 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T08:29:33.655 DEBUG:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T08:29:33.787 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-19T08:29:33.795 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-19T08:29:33.795 DEBUG:teuthology.orchestra.run.smithi170:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T08:29:33.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:29:33.813 DEBUG:teuthology.orchestra.run.smithi173:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T08:29:33.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:29:33.831 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T08:29:33.833 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T08:29:33.856 DEBUG:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T08:29:33.899 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T08:29:33.915 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T08:29:33.930 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T08:29:33.939 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T08:29:33.961 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T08:29:33.968 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T08:29:33.970 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-19T08:29:33.978 INFO:teuthology.task.internal:Configuring sudo... 2024-09-19T08:29:33.978 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T08:29:33.980 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T08:29:33.983 DEBUG:teuthology.orchestra.run.smithi173:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T08:29:34.011 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-19T08:29:34.021 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-19T08:29:34.021 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T08:29:34.046 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T08:29:34.053 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T08:29:34.070 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T08:29:34.143 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T08:29:34.193 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T08:29:34.273 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T08:29:34.315 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-19T08:29:34.315 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T08:29:34.379 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T08:29:34.423 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T08:29:34.517 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T08:29:34.564 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T08:29:34.600 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-19T08:29:34.600 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T08:29:34.666 DEBUG:teuthology.orchestra.run.smithi173:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T08:29:34.709 DEBUG:teuthology.orchestra.run.smithi173:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T08:29:34.768 DEBUG:teuthology.orchestra.run.smithi173:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T08:29:34.854 DEBUG:teuthology.orchestra.run.smithi173:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T08:29:34.896 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-19T08:29:34.897 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T08:29:34.964 DEBUG:teuthology.orchestra.run.smithi164:> sudo service rsyslog restart 2024-09-19T08:29:34.966 DEBUG:teuthology.orchestra.run.smithi170:> sudo service rsyslog restart 2024-09-19T08:29:34.968 DEBUG:teuthology.orchestra.run.smithi173:> sudo service rsyslog restart 2024-09-19T08:29:34.999 INFO:teuthology.orchestra.run.smithi170.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T08:29:35.009 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T08:29:35.044 INFO:teuthology.orchestra.run.smithi173.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T08:29:35.447 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-19T08:29:35.454 INFO:teuthology.task.internal:Starting timer... 2024-09-19T08:29:35.454 INFO:teuthology.run_tasks:Running task pcp... 2024-09-19T08:29:35.494 INFO:teuthology.run_tasks:Running task selinux... 2024-09-19T08:29:35.503 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-19T08:29:35.503 DEBUG:teuthology.orchestra.run.smithi164:> sudo service auditd rotate 2024-09-19T08:29:35.569 INFO:teuthology.orchestra.run.smithi164.stdout:Rotating logs: 2024-09-19T08:29:35.571 DEBUG:teuthology.orchestra.run.smithi170:> sudo service auditd rotate 2024-09-19T08:29:35.626 INFO:teuthology.orchestra.run.smithi170.stdout:Rotating logs: 2024-09-19T08:29:35.627 DEBUG:teuthology.orchestra.run.smithi173:> sudo service auditd rotate 2024-09-19T08:29:35.694 INFO:teuthology.orchestra.run.smithi173.stdout:Rotating logs: 2024-09-19T08:29:35.696 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-19T08:29:35.696 DEBUG:teuthology.orchestra.run.smithi164:> /usr/sbin/getenforce 2024-09-19T08:29:35.726 INFO:teuthology.orchestra.run.smithi164.stdout:Permissive 2024-09-19T08:29:35.726 DEBUG:teuthology.orchestra.run.smithi170:> /usr/sbin/getenforce 2024-09-19T08:29:35.753 INFO:teuthology.orchestra.run.smithi170.stdout:Permissive 2024-09-19T08:29:35.753 DEBUG:teuthology.orchestra.run.smithi173:> /usr/sbin/getenforce 2024-09-19T08:29:35.779 INFO:teuthology.orchestra.run.smithi173.stdout:Permissive 2024-09-19T08:29:35.780 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi164.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi170.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi173.front.sepia.ceph.com': 'permissive'} 2024-09-19T08:29:35.780 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T08:29:35.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:29:35.805 DEBUG:teuthology.orchestra.run.smithi170:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T08:29:35.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:29:35.834 DEBUG:teuthology.orchestra.run.smithi173:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T08:29:35.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:29:35.862 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-19T08:29:35.862 DEBUG:teuthology.orchestra.run.smithi164:> sudo /usr/sbin/setenforce permissive 2024-09-19T08:29:35.893 DEBUG:teuthology.orchestra.run.smithi170:> sudo /usr/sbin/setenforce permissive 2024-09-19T08:29:35.928 DEBUG:teuthology.orchestra.run.smithi173:> sudo /usr/sbin/setenforce permissive 2024-09-19T08:29:35.960 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-19T08:29:35.969 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-19T08:29:35.981 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-19T08:29:35.982 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi164.front.sepia.ceph.com,smithi170.front.sepia.ceph.com,smithi173.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-19T08:33:38.731 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi164.front.sepia.ceph.com'), Remote(name='ubuntu@smithi170.front.sepia.ceph.com'), Remote(name='ubuntu@smithi173.front.sepia.ceph.com')] 2024-09-19T08:33:38.733 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-19T08:33:38.734 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T08:33:38.811 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-09-19T08:33:38.893 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-19T08:33:38.894 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-09-19T08:33:38.894 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T08:33:39.020 DEBUG:teuthology.orchestra.run.smithi170:> true 2024-09-19T08:33:39.098 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-09-19T08:33:39.098 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-09-19T08:33:39.099 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T08:33:39.199 DEBUG:teuthology.orchestra.run.smithi173:> true 2024-09-19T08:33:39.277 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-09-19T08:33:39.278 INFO:teuthology.run_tasks:Running task clock... 2024-09-19T08:33:39.534 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-19T08:33:39.534 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T08:33:39.535 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T08:33:39.537 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T08:33:39.538 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T08:33:39.540 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T08:33:39.541 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T08:33:39.570 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T08:33:39.571 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T08:33:39.572 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T08:33:39.585 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T08:33:39.586 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T08:33:39.588 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T08:33:39.610 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: ntpd: command not found 2024-09-19T08:33:39.613 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: ntpd: command not found 2024-09-19T08:33:39.615 INFO:teuthology.orchestra.run.smithi170.stderr:sudo: ntpd: command not found 2024-09-19T08:33:39.622 INFO:teuthology.orchestra.run.smithi173.stdout:506 Cannot talk to daemon 2024-09-19T08:33:39.624 INFO:teuthology.orchestra.run.smithi164.stdout:506 Cannot talk to daemon 2024-09-19T08:33:39.627 INFO:teuthology.orchestra.run.smithi170.stdout:506 Cannot talk to daemon 2024-09-19T08:33:39.637 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T08:33:39.639 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T08:33:39.642 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T08:33:39.651 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T08:33:39.654 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T08:33:39.656 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T08:33:39.708 INFO:teuthology.orchestra.run.smithi170.stderr:bash: line 1: ntpq: command not found 2024-09-19T08:33:39.710 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-09-19T08:33:39.711 INFO:teuthology.orchestra.run.smithi170.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T08:33:39.711 INFO:teuthology.orchestra.run.smithi170.stdout:=============================================================================== 2024-09-19T08:33:39.711 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:33:39.711 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:33:39.712 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:33:39.712 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:33:39.713 INFO:teuthology.orchestra.run.smithi173.stderr:bash: line 1: ntpq: command not found 2024-09-19T08:33:39.714 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T08:33:39.714 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-09-19T08:33:39.714 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:33:39.714 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:33:39.715 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:33:39.715 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:33:39.716 INFO:teuthology.orchestra.run.smithi173.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T08:33:39.716 INFO:teuthology.orchestra.run.smithi173.stdout:=============================================================================== 2024-09-19T08:33:39.717 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:33:39.717 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:33:39.717 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:33:39.717 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:33:39.718 INFO:teuthology.run_tasks:Running task install... 2024-09-19T08:33:39.726 DEBUG:teuthology.task.install:project ceph 2024-09-19T08:33:39.727 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}} 2024-09-19T08:33:39.727 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'} 2024-09-19T08:33:39.727 INFO:teuthology.task.install:Using flavor: default 2024-09-19T08:33:39.735 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-19T08:33:39.735 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-19T08:33:39.735 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T08:33:39.736 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:33:39.738 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T08:33:39.739 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:33:39.740 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T08:33:39.740 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:33:39.892 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T08:33:39.893 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T08:33:39.897 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T08:33:39.898 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T08:33:39.903 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T08:33:39.904 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T08:33:40.013 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T08:33:40.013 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-19T08:33:40.013 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T08:33:40.024 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T08:33:40.025 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-19T08:33:40.025 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T08:33:40.035 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T08:33:40.035 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-19T08:33:40.035 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T08:33:40.039 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T08:33:40.039 DEBUG:teuthology.orchestra.run.smithi164:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T08:33:40.054 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T08:33:40.054 DEBUG:teuthology.orchestra.run.smithi170:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T08:33:40.061 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T08:33:40.061 DEBUG:teuthology.orchestra.run.smithi173:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T08:33:40.106 DEBUG:teuthology.orchestra.run.smithi164:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T08:33:40.121 DEBUG:teuthology.orchestra.run.smithi170:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T08:33:40.127 DEBUG:teuthology.orchestra.run.smithi173:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T08:33:40.182 DEBUG:teuthology.orchestra.run.smithi164:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T08:33:40.196 DEBUG:teuthology.orchestra.run.smithi170:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T08:33:40.202 DEBUG:teuthology.orchestra.run.smithi173:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T08:33:40.247 INFO:teuthology.orchestra.run.smithi164.stdout:check_obsoletes = 1 2024-09-19T08:33:40.248 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum clean all 2024-09-19T08:33:40.261 INFO:teuthology.orchestra.run.smithi170.stdout:check_obsoletes = 1 2024-09-19T08:33:40.262 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum clean all 2024-09-19T08:33:40.266 INFO:teuthology.orchestra.run.smithi173.stdout:check_obsoletes = 1 2024-09-19T08:33:40.267 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum clean all 2024-09-19T08:33:40.482 INFO:teuthology.orchestra.run.smithi164.stdout:45 files removed 2024-09-19T08:33:40.486 INFO:teuthology.orchestra.run.smithi170.stdout:45 files removed 2024-09-19T08:33:40.498 INFO:teuthology.orchestra.run.smithi173.stdout:45 files removed 2024-09-19T08:33:40.509 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T08:33:40.513 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T08:33:40.527 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T08:33:41.057 INFO:teuthology.orchestra.run.smithi170.stdout:ceph packages for x86_64 311 kB/s | 84 kB 00:00 2024-09-19T08:33:41.080 INFO:teuthology.orchestra.run.smithi173.stdout:ceph packages for x86_64 305 kB/s | 84 kB 00:00 2024-09-19T08:33:41.081 INFO:teuthology.orchestra.run.smithi164.stdout:ceph packages for x86_64 292 kB/s | 84 kB 00:00 2024-09-19T08:33:41.341 INFO:teuthology.orchestra.run.smithi170.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-19T08:33:41.366 INFO:teuthology.orchestra.run.smithi164.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-19T08:33:41.372 INFO:teuthology.orchestra.run.smithi173.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-19T08:33:41.617 INFO:teuthology.orchestra.run.smithi170.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-19T08:33:41.641 INFO:teuthology.orchestra.run.smithi164.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-19T08:33:41.647 INFO:teuthology.orchestra.run.smithi173.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-19T08:33:42.208 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.3 MB 00:00 2024-09-19T08:33:43.592 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.3 MB 00:01 2024-09-19T08:33:44.422 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - BaseOS 3.0 MB/s | 8.3 MB 00:02 2024-09-19T08:33:45.493 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - AppStream 24 MB/s | 20 MB 00:00 2024-09-19T08:33:47.583 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - AppStream 9.7 MB/s | 20 MB 00:02 2024-09-19T08:33:49.861 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-19T08:33:50.492 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - AppStream 2.8 MB/s | 20 MB 00:07 2024-09-19T08:33:51.561 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 19 kB 00:00 2024-09-19T08:33:51.824 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.5 MB 00:00 2024-09-19T08:33:52.390 INFO:teuthology.orchestra.run.smithi164.stdout:Extra Packages for Enterprise Linux 32 MB/s | 23 MB 00:00 2024-09-19T08:33:53.317 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - Extras packages 70 kB/s | 19 kB 00:00 2024-09-19T08:33:54.196 INFO:teuthology.orchestra.run.smithi173.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-19T08:33:56.510 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 6.5 MB 00:02 2024-09-19T08:33:57.805 INFO:teuthology.orchestra.run.smithi164.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-19T08:33:58.018 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - Extras packages 64 kB/s | 19 kB 00:00 2024-09-19T08:33:59.201 INFO:teuthology.orchestra.run.smithi170.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-19T08:33:59.510 INFO:teuthology.orchestra.run.smithi173.stdout:lab-extras 42 kB/s | 1.7 kB 00:00 2024-09-19T08:34:00.040 INFO:teuthology.orchestra.run.smithi164.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T08:34:00.040 INFO:teuthology.orchestra.run.smithi164.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T08:34:00.052 INFO:teuthology.orchestra.run.smithi164.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T08:34:00.052 INFO:teuthology.orchestra.run.smithi164.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T08:34:00.057 INFO:teuthology.orchestra.run.smithi164.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T08:34:00.057 INFO:teuthology.orchestra.run.smithi164.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T08:34:00.057 INFO:teuthology.orchestra.run.smithi164.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T08:34:00.058 INFO:teuthology.orchestra.run.smithi164.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T08:34:00.059 INFO:teuthology.orchestra.run.smithi164.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T08:34:00.121 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-19T08:34:00.138 INFO:teuthology.orchestra.run.smithi164.stdout:====================================================================================== 2024-09-19T08:34:00.139 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-19T08:34:00.139 INFO:teuthology.orchestra.run.smithi164.stdout:====================================================================================== 2024-09-19T08:34:00.139 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-19T08:34:00.139 INFO:teuthology.orchestra.run.smithi164.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T08:34:00.139 INFO:teuthology.orchestra.run.smithi164.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T08:34:00.139 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T08:34:00.139 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T08:34:00.139 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T08:34:00.139 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T08:34:00.139 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T08:34:00.139 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T08:34:00.139 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T08:34:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T08:34:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T08:34:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T08:34:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T08:34:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T08:34:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T08:34:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T08:34:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T08:34:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T08:34:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T08:34:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T08:34:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T08:34:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T08:34:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T08:34:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T08:34:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T08:34:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T08:34:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T08:34:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T08:34:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T08:34:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T08:34:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T08:34:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T08:34:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T08:34:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T08:34:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T08:34:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T08:34:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T08:34:00.142 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrading: 2024-09-19T08:34:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T08:34:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T08:34:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T08:34:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T08:34:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T08:34:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T08:34:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T08:34:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T08:34:00.143 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-19T08:34:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T08:34:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T08:34:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T08:34:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T08:34:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T08:34:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T08:34:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T08:34:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T08:34:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T08:34:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T08:34:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T08:34:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T08:34:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T08:34:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T08:34:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T08:34:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T08:34:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T08:34:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T08:34:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T08:34:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T08:34:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T08:34:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T08:34:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T08:34:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T08:34:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T08:34:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T08:34:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T08:34:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T08:34:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T08:34:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T08:34:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T08:34:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T08:34:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T08:34:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T08:34:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T08:34:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T08:34:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T08:34:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T08:34:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T08:34:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T08:34:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T08:34:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T08:34:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T08:34:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T08:34:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T08:34:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T08:34:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T08:34:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T08:34:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T08:34:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T08:34:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T08:34:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T08:34:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T08:34:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T08:34:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T08:34:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T08:34:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T08:34:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T08:34:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T08:34:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T08:34:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T08:34:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T08:34:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T08:34:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T08:34:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T08:34:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T08:34:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T08:34:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T08:34:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T08:34:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T08:34:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T08:34:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T08:34:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T08:34:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T08:34:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T08:34:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T08:34:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T08:34:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T08:34:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T08:34:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T08:34:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T08:34:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T08:34:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T08:34:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T08:34:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T08:34:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T08:34:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T08:34:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T08:34:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T08:34:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T08:34:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T08:34:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T08:34:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T08:34:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T08:34:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T08:34:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T08:34:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T08:34:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T08:34:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T08:34:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T08:34:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T08:34:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T08:34:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T08:34:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T08:34:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T08:34:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T08:34:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T08:34:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T08:34:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T08:34:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T08:34:00.153 INFO:teuthology.orchestra.run.smithi164.stdout:Installing weak dependencies: 2024-09-19T08:34:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T08:34:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:34:00.153 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-19T08:34:00.153 INFO:teuthology.orchestra.run.smithi164.stdout:====================================================================================== 2024-09-19T08:34:00.153 INFO:teuthology.orchestra.run.smithi164.stdout:Install 148 Packages 2024-09-19T08:34:00.153 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrade 8 Packages 2024-09-19T08:34:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:34:00.154 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 226 M 2024-09-19T08:34:00.154 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-19T08:34:00.831 INFO:teuthology.orchestra.run.smithi164.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 49 kB/s | 6.6 kB 00:00 2024-09-19T08:34:01.056 INFO:teuthology.orchestra.run.smithi164.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 3.9 MB/s | 896 kB 00:00 2024-09-19T08:34:01.148 INFO:teuthology.orchestra.run.smithi164.stdout:(3/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 12 MB/s | 5.4 MB 00:00 2024-09-19T08:34:01.174 INFO:teuthology.orchestra.run.smithi164.stdout:(4/156): ceph-immutable-object-cache-19.1.1-361 1.2 MB/s | 145 kB 00:00 2024-09-19T08:34:01.291 INFO:teuthology.orchestra.run.smithi164.stdout:(5/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 17 MB/s | 2.4 MB 00:00 2024-09-19T08:34:01.424 INFO:teuthology.orchestra.run.smithi164.stdout:(6/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 6.1 MB/s | 1.5 MB 00:00 2024-09-19T08:34:01.583 INFO:teuthology.orchestra.run.smithi164.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 16 MB/s | 4.7 MB 00:00 2024-09-19T08:34:01.655 INFO:teuthology.orchestra.run.smithi173.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T08:34:01.656 INFO:teuthology.orchestra.run.smithi173.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T08:34:01.669 INFO:teuthology.orchestra.run.smithi173.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T08:34:01.670 INFO:teuthology.orchestra.run.smithi173.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T08:34:01.671 INFO:teuthology.orchestra.run.smithi173.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T08:34:01.672 INFO:teuthology.orchestra.run.smithi173.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T08:34:01.674 INFO:teuthology.orchestra.run.smithi173.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T08:34:01.677 INFO:teuthology.orchestra.run.smithi173.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T08:34:01.678 INFO:teuthology.orchestra.run.smithi173.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T08:34:01.740 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-19T08:34:01.765 INFO:teuthology.orchestra.run.smithi173.stdout:====================================================================================== 2024-09-19T08:34:01.765 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repository Size 2024-09-19T08:34:01.766 INFO:teuthology.orchestra.run.smithi173.stdout:====================================================================================== 2024-09-19T08:34:01.766 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-09-19T08:34:01.766 INFO:teuthology.orchestra.run.smithi173.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T08:34:01.766 INFO:teuthology.orchestra.run.smithi173.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T08:34:01.766 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T08:34:01.766 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T08:34:01.766 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T08:34:01.766 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T08:34:01.766 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T08:34:01.766 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T08:34:01.766 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T08:34:01.766 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T08:34:01.767 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T08:34:01.767 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T08:34:01.767 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T08:34:01.767 INFO:teuthology.orchestra.run.smithi173.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T08:34:01.767 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T08:34:01.767 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T08:34:01.767 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T08:34:01.767 INFO:teuthology.orchestra.run.smithi173.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T08:34:01.767 INFO:teuthology.orchestra.run.smithi173.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T08:34:01.767 INFO:teuthology.orchestra.run.smithi173.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T08:34:01.767 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T08:34:01.768 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T08:34:01.768 INFO:teuthology.orchestra.run.smithi173.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T08:34:01.768 INFO:teuthology.orchestra.run.smithi173.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T08:34:01.768 INFO:teuthology.orchestra.run.smithi173.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T08:34:01.768 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T08:34:01.768 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T08:34:01.768 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T08:34:01.768 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T08:34:01.769 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T08:34:01.769 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T08:34:01.769 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T08:34:01.769 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T08:34:01.769 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T08:34:01.769 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T08:34:01.769 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T08:34:01.769 INFO:teuthology.orchestra.run.smithi173.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T08:34:01.769 INFO:teuthology.orchestra.run.smithi173.stdout:Upgrading: 2024-09-19T08:34:01.769 INFO:teuthology.orchestra.run.smithi173.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T08:34:01.769 INFO:teuthology.orchestra.run.smithi173.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T08:34:01.769 INFO:teuthology.orchestra.run.smithi173.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T08:34:01.769 INFO:teuthology.orchestra.run.smithi173.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T08:34:01.770 INFO:teuthology.orchestra.run.smithi173.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T08:34:01.770 INFO:teuthology.orchestra.run.smithi173.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T08:34:01.770 INFO:teuthology.orchestra.run.smithi173.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T08:34:01.770 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T08:34:01.770 INFO:teuthology.orchestra.run.smithi173.stdout:Installing dependencies: 2024-09-19T08:34:01.770 INFO:teuthology.orchestra.run.smithi173.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T08:34:01.770 INFO:teuthology.orchestra.run.smithi173.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T08:34:01.770 INFO:teuthology.orchestra.run.smithi173.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T08:34:01.770 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T08:34:01.770 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T08:34:01.770 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T08:34:01.770 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T08:34:01.771 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T08:34:01.771 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T08:34:01.771 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T08:34:01.771 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T08:34:01.771 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T08:34:01.771 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T08:34:01.771 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T08:34:01.771 INFO:teuthology.orchestra.run.smithi173.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T08:34:01.771 INFO:teuthology.orchestra.run.smithi173.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T08:34:01.771 INFO:teuthology.orchestra.run.smithi173.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T08:34:01.771 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T08:34:01.771 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T08:34:01.772 INFO:teuthology.orchestra.run.smithi173.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T08:34:01.772 INFO:teuthology.orchestra.run.smithi173.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T08:34:01.772 INFO:teuthology.orchestra.run.smithi173.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T08:34:01.772 INFO:teuthology.orchestra.run.smithi173.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T08:34:01.772 INFO:teuthology.orchestra.run.smithi173.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T08:34:01.772 INFO:teuthology.orchestra.run.smithi173.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T08:34:01.772 INFO:teuthology.orchestra.run.smithi173.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T08:34:01.772 INFO:teuthology.orchestra.run.smithi173.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T08:34:01.772 INFO:teuthology.orchestra.run.smithi173.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T08:34:01.772 INFO:teuthology.orchestra.run.smithi173.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T08:34:01.772 INFO:teuthology.orchestra.run.smithi173.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T08:34:01.772 INFO:teuthology.orchestra.run.smithi173.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T08:34:01.773 INFO:teuthology.orchestra.run.smithi173.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T08:34:01.773 INFO:teuthology.orchestra.run.smithi173.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T08:34:01.773 INFO:teuthology.orchestra.run.smithi173.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T08:34:01.773 INFO:teuthology.orchestra.run.smithi173.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T08:34:01.773 INFO:teuthology.orchestra.run.smithi173.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T08:34:01.773 INFO:teuthology.orchestra.run.smithi173.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T08:34:01.773 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T08:34:01.773 INFO:teuthology.orchestra.run.smithi173.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T08:34:01.773 INFO:teuthology.orchestra.run.smithi173.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T08:34:01.773 INFO:teuthology.orchestra.run.smithi173.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T08:34:01.773 INFO:teuthology.orchestra.run.smithi173.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T08:34:01.773 INFO:teuthology.orchestra.run.smithi173.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T08:34:01.773 INFO:teuthology.orchestra.run.smithi173.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T08:34:01.774 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T08:34:01.774 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T08:34:01.774 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T08:34:01.774 INFO:teuthology.orchestra.run.smithi173.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T08:34:01.774 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T08:34:01.774 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T08:34:01.774 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T08:34:01.774 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T08:34:01.774 INFO:teuthology.orchestra.run.smithi173.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T08:34:01.775 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T08:34:01.775 INFO:teuthology.orchestra.run.smithi173.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T08:34:01.775 INFO:teuthology.orchestra.run.smithi173.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T08:34:01.775 INFO:teuthology.orchestra.run.smithi173.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T08:34:01.775 INFO:teuthology.orchestra.run.smithi173.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T08:34:01.775 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T08:34:01.775 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T08:34:01.775 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T08:34:01.775 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T08:34:01.775 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T08:34:01.775 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T08:34:01.775 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T08:34:01.775 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T08:34:01.776 INFO:teuthology.orchestra.run.smithi173.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T08:34:01.776 INFO:teuthology.orchestra.run.smithi173.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T08:34:01.776 INFO:teuthology.orchestra.run.smithi173.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T08:34:01.776 INFO:teuthology.orchestra.run.smithi173.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T08:34:01.776 INFO:teuthology.orchestra.run.smithi173.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T08:34:01.776 INFO:teuthology.orchestra.run.smithi173.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T08:34:01.776 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T08:34:01.776 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T08:34:01.776 INFO:teuthology.orchestra.run.smithi173.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T08:34:01.776 INFO:teuthology.orchestra.run.smithi173.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T08:34:01.777 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T08:34:01.777 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T08:34:01.777 INFO:teuthology.orchestra.run.smithi173.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T08:34:01.777 INFO:teuthology.orchestra.run.smithi173.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T08:34:01.777 INFO:teuthology.orchestra.run.smithi173.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T08:34:01.777 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T08:34:01.777 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T08:34:01.777 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T08:34:01.777 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T08:34:01.777 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T08:34:01.777 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T08:34:01.777 INFO:teuthology.orchestra.run.smithi173.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T08:34:01.778 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T08:34:01.778 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T08:34:01.778 INFO:teuthology.orchestra.run.smithi173.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T08:34:01.778 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T08:34:01.778 INFO:teuthology.orchestra.run.smithi173.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T08:34:01.778 INFO:teuthology.orchestra.run.smithi173.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T08:34:01.778 INFO:teuthology.orchestra.run.smithi173.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T08:34:01.779 INFO:teuthology.orchestra.run.smithi173.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T08:34:01.779 INFO:teuthology.orchestra.run.smithi173.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T08:34:01.779 INFO:teuthology.orchestra.run.smithi173.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T08:34:01.779 INFO:teuthology.orchestra.run.smithi173.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T08:34:01.779 INFO:teuthology.orchestra.run.smithi173.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T08:34:01.779 INFO:teuthology.orchestra.run.smithi173.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T08:34:01.779 INFO:teuthology.orchestra.run.smithi173.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T08:34:01.779 INFO:teuthology.orchestra.run.smithi173.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T08:34:01.779 INFO:teuthology.orchestra.run.smithi173.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T08:34:01.779 INFO:teuthology.orchestra.run.smithi173.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T08:34:01.779 INFO:teuthology.orchestra.run.smithi173.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T08:34:01.780 INFO:teuthology.orchestra.run.smithi173.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T08:34:01.780 INFO:teuthology.orchestra.run.smithi173.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T08:34:01.780 INFO:teuthology.orchestra.run.smithi173.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T08:34:01.780 INFO:teuthology.orchestra.run.smithi173.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T08:34:01.780 INFO:teuthology.orchestra.run.smithi173.stdout:Installing weak dependencies: 2024-09-19T08:34:01.780 INFO:teuthology.orchestra.run.smithi173.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T08:34:01.780 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-19T08:34:01.780 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-09-19T08:34:01.780 INFO:teuthology.orchestra.run.smithi173.stdout:====================================================================================== 2024-09-19T08:34:01.780 INFO:teuthology.orchestra.run.smithi173.stdout:Install 148 Packages 2024-09-19T08:34:01.780 INFO:teuthology.orchestra.run.smithi173.stdout:Upgrade 8 Packages 2024-09-19T08:34:01.780 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-19T08:34:01.781 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 226 M 2024-09-19T08:34:01.781 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-09-19T08:34:01.808 INFO:teuthology.orchestra.run.smithi164.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 19 MB/s | 21 MB 00:01 2024-09-19T08:34:01.858 INFO:teuthology.orchestra.run.smithi164.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 505 kB/s | 25 kB 00:00 2024-09-19T08:34:02.209 INFO:teuthology.orchestra.run.smithi164.stdout:(10/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 17 MB/s | 11 MB 00:00 2024-09-19T08:34:02.211 INFO:teuthology.orchestra.run.smithi173.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 40 kB/s | 6.6 kB 00:00 2024-09-19T08:34:02.275 INFO:teuthology.orchestra.run.smithi164.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 3.4 MB/s | 228 kB 00:00 2024-09-19T08:34:02.342 INFO:teuthology.orchestra.run.smithi164.stdout:(12/156): cephfs-shell-19.1.1-361.g2f599a46.el9 405 kB/s | 27 kB 00:00 2024-09-19T08:34:02.409 INFO:teuthology.orchestra.run.smithi164.stdout:(13/156): libcephfs-devel-19.1.1-361.g2f599a46. 476 kB/s | 32 kB 00:00 2024-09-19T08:34:02.501 INFO:teuthology.orchestra.run.smithi164.stdout:(14/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 8.0 MB/s | 751 kB 00:00 2024-09-19T08:34:02.653 INFO:teuthology.orchestra.run.smithi173.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 2.0 MB/s | 896 kB 00:00 2024-09-19T08:34:02.710 INFO:teuthology.orchestra.run.smithi164.stdout:(15/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 13 MB/s | 17 MB 00:01 2024-09-19T08:34:02.720 INFO:teuthology.orchestra.run.smithi173.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 2.1 MB/s | 145 kB 00:00 2024-09-19T08:34:02.735 INFO:teuthology.orchestra.run.smithi164.stdout:(16/156): libcephsqlite-19.1.1-361.g2f599a46.el 699 kB/s | 163 kB 00:00 2024-09-19T08:34:02.761 INFO:teuthology.orchestra.run.smithi164.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 2.5 MB/s | 127 kB 00:00 2024-09-19T08:34:02.803 INFO:teuthology.orchestra.run.smithi164.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 7.4 MB/s | 506 kB 00:00 2024-09-19T08:34:02.878 INFO:teuthology.orchestra.run.smithi164.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 601 kB/s | 45 kB 00:00 2024-09-19T08:34:02.970 INFO:teuthology.orchestra.run.smithi164.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 1.5 MB/s | 136 kB 00:00 2024-09-19T08:34:03.062 INFO:teuthology.orchestra.run.smithi164.stdout:(21/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 18 MB/s | 5.3 MB 00:00 2024-09-19T08:34:03.087 INFO:teuthology.orchestra.run.smithi164.stdout:(22/156): python3-cephfs-19.1.1-361.g2f599a46.e 1.3 MB/s | 161 kB 00:00 2024-09-19T08:34:03.121 INFO:teuthology.orchestra.run.smithi164.stdout:(23/156): python3-rados-19.1.1-361.g2f599a46.el 5.4 MB/s | 320 kB 00:00 2024-09-19T08:34:03.129 INFO:teuthology.orchestra.run.smithi173.stdout:(4/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 5.0 MB/s | 5.4 MB 00:01 2024-09-19T08:34:03.146 INFO:teuthology.orchestra.run.smithi164.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 5.0 MB/s | 300 kB 00:00 2024-09-19T08:34:03.172 INFO:teuthology.orchestra.run.smithi164.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.9 MB/s | 99 kB 00:00 2024-09-19T08:34:03.197 INFO:teuthology.orchestra.run.smithi164.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.6 MB/s | 85 kB 00:00 2024-09-19T08:34:03.298 INFO:teuthology.orchestra.run.smithi164.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 1.7 MB/s | 171 kB 00:00 2024-09-19T08:34:03.364 INFO:teuthology.orchestra.run.smithi164.stdout:(28/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 16 MB/s | 3.1 MB 00:00 2024-09-19T08:34:03.371 INFO:teuthology.orchestra.run.smithi173.stdout:(5/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 6.3 MB/s | 1.5 MB 00:00 2024-09-19T08:34:03.390 INFO:teuthology.orchestra.run.smithi164.stdout:(29/156): ceph-grafana-dashboards-19.1.1-361.g2 346 kB/s | 32 kB 00:00 2024-09-19T08:34:03.423 INFO:teuthology.orchestra.run.smithi164.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.4 MB/s | 146 kB 00:00 2024-09-19T08:34:03.446 INFO:teuthology.orchestra.run.smithi173.stdout:(6/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 3.3 MB/s | 2.4 MB 00:00 2024-09-19T08:34:03.865 INFO:teuthology.orchestra.run.smithi164.stdout:(31/156): ceph-mgr-diskprediction-local-19.1.1- 17 MB/s | 7.4 MB 00:00 2024-09-19T08:34:03.932 INFO:teuthology.orchestra.run.smithi164.stdout:(32/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 6.6 MB/s | 3.6 MB 00:00 2024-09-19T08:34:03.983 INFO:teuthology.orchestra.run.smithi164.stdout:(33/156): ceph-mgr-modules-core-19.1.1-361.g2f5 2.1 MB/s | 249 kB 00:00 2024-09-19T08:34:04.008 INFO:teuthology.orchestra.run.smithi164.stdout:(34/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 655 kB/s | 49 kB 00:00 2024-09-19T08:34:04.012 INFO:teuthology.orchestra.run.smithi173.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 7.3 MB/s | 4.7 MB 00:00 2024-09-19T08:34:04.033 INFO:teuthology.orchestra.run.smithi164.stdout:(35/156): ceph-prometheus-alerts-19.1.1-361.g2f 328 kB/s | 16 kB 00:00 2024-09-19T08:34:04.058 INFO:teuthology.orchestra.run.smithi164.stdout:(36/156): ceph-volume-19.1.1-361.g2f599a46.el9. 5.6 MB/s | 285 kB 00:00 2024-09-19T08:34:04.100 INFO:teuthology.orchestra.run.smithi164.stdout:(37/156): cephadm-19.1.1-361.g2f599a46.el9.noar 11 MB/s | 765 kB 00:00 2024-09-19T08:34:04.125 INFO:teuthology.orchestra.run.smithi164.stdout:(38/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 272 kB/s | 18 kB 00:00 2024-09-19T08:34:04.226 INFO:teuthology.orchestra.run.smithi164.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 637 kB/s | 80 kB 00:00 2024-09-19T08:34:04.285 INFO:teuthology.orchestra.run.smithi164.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-19T08:34:04.318 INFO:teuthology.orchestra.run.smithi164.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 4.1 MB/s | 801 kB 00:00 2024-09-19T08:34:04.360 INFO:teuthology.orchestra.run.smithi164.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-09-19T08:34:04.402 INFO:teuthology.orchestra.run.smithi164.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-19T08:34:04.654 INFO:teuthology.orchestra.run.smithi170.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-19T08:34:04.730 INFO:teuthology.orchestra.run.smithi173.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 7.9 MB/s | 21 MB 00:02 2024-09-19T08:34:04.780 INFO:teuthology.orchestra.run.smithi173.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 506 kB/s | 25 kB 00:00 2024-09-19T08:34:04.836 INFO:teuthology.orchestra.run.smithi164.stdout:(44/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 16 MB/s | 49 MB 00:02 2024-09-19T08:34:04.861 INFO:teuthology.orchestra.run.smithi164.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 295 kB/s | 135 kB 00:00 2024-09-19T08:34:04.903 INFO:teuthology.orchestra.run.smithi164.stdout:(46/156): python3-cryptography-36.0.1-4.el9.x86 2.1 MB/s | 1.2 MB 00:00 2024-09-19T08:34:04.929 INFO:teuthology.orchestra.run.smithi164.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-09-19T08:34:04.970 INFO:teuthology.orchestra.run.smithi164.stdout:(48/156): python3-requests-2.25.1-8.el9.noarch. 931 kB/s | 125 kB 00:00 2024-09-19T08:34:05.038 INFO:teuthology.orchestra.run.smithi173.stdout:(10/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 10 MB/s | 11 MB 00:01 2024-09-19T08:34:05.054 INFO:teuthology.orchestra.run.smithi164.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 1.3 MB/s | 107 kB 00:00 2024-09-19T08:34:05.079 INFO:teuthology.orchestra.run.smithi164.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 3.8 MB/s | 689 kB 00:00 2024-09-19T08:34:05.088 INFO:teuthology.orchestra.run.smithi173.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 4.5 MB/s | 228 kB 00:00 2024-09-19T08:34:05.113 INFO:teuthology.orchestra.run.smithi164.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 5.0 MB/s | 943 kB 00:00 2024-09-19T08:34:05.130 INFO:teuthology.orchestra.run.smithi164.stdout:(52/156): elfutils-libelf-devel-0.191-4.el9.x86 286 kB/s | 22 kB 00:00 2024-09-19T08:34:05.139 INFO:teuthology.orchestra.run.smithi173.stdout:(12/156): cephfs-shell-19.1.1-361.g2f599a46.el9 541 kB/s | 27 kB 00:00 2024-09-19T08:34:05.180 INFO:teuthology.orchestra.run.smithi164.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 489 kB/s | 33 kB 00:00 2024-09-19T08:34:05.189 INFO:teuthology.orchestra.run.smithi173.stdout:(13/156): libcephfs-devel-19.1.1-361.g2f599a46. 633 kB/s | 32 kB 00:00 2024-09-19T08:34:05.214 INFO:teuthology.orchestra.run.smithi164.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-09-19T08:34:05.247 INFO:teuthology.orchestra.run.smithi164.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 272 kB/s | 18 kB 00:00 2024-09-19T08:34:05.265 INFO:teuthology.orchestra.run.smithi164.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-09-19T08:34:05.273 INFO:teuthology.orchestra.run.smithi173.stdout:(14/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 8.8 MB/s | 751 kB 00:00 2024-09-19T08:34:05.323 INFO:teuthology.orchestra.run.smithi173.stdout:(15/156): libcephsqlite-19.1.1-361.g2f599a46.el 3.2 MB/s | 163 kB 00:00 2024-09-19T08:34:05.324 INFO:teuthology.orchestra.run.smithi164.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-09-19T08:34:05.365 INFO:teuthology.orchestra.run.smithi164.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-09-19T08:34:05.373 INFO:teuthology.orchestra.run.smithi173.stdout:(16/156): librados-devel-19.1.1-361.g2f599a46.e 2.5 MB/s | 127 kB 00:00 2024-09-19T08:34:05.390 INFO:teuthology.orchestra.run.smithi164.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 360 kB/s | 45 kB 00:00 2024-09-19T08:34:05.449 INFO:teuthology.orchestra.run.smithi164.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 420 kB/s | 24 kB 00:00 2024-09-19T08:34:05.468 INFO:teuthology.orchestra.run.smithi173.stdout:(17/156): libradosstriper1-19.1.1-361.g2f599a46 5.3 MB/s | 506 kB 00:00 2024-09-19T08:34:05.474 INFO:teuthology.orchestra.run.smithi164.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-09-19T08:34:05.516 INFO:teuthology.orchestra.run.smithi164.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 709 kB/s | 47 kB 00:00 2024-09-19T08:34:05.550 INFO:teuthology.orchestra.run.smithi164.stdout:(63/156): libtool-2.4.6-46.el9.x86_64.rpm 3.1 MB/s | 581 kB 00:00 2024-09-19T08:34:05.576 INFO:teuthology.orchestra.run.smithi164.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-09-19T08:34:05.625 INFO:teuthology.orchestra.run.smithi164.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-19T08:34:05.651 INFO:teuthology.orchestra.run.smithi164.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 383 kB/s | 38 kB 00:00 2024-09-19T08:34:05.816 INFO:teuthology.orchestra.run.smithi173.stdout:(18/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 15 MB/s | 5.3 MB 00:00 2024-09-19T08:34:05.851 INFO:teuthology.orchestra.run.smithi164.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 18 MB/s | 4.9 MB 00:00 2024-09-19T08:34:05.866 INFO:teuthology.orchestra.run.smithi173.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 911 kB/s | 45 kB 00:00 2024-09-19T08:34:05.893 INFO:teuthology.orchestra.run.smithi164.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 6.6 MB/s | 1.6 MB 00:00 2024-09-19T08:34:05.916 INFO:teuthology.orchestra.run.smithi173.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 2.7 MB/s | 136 kB 00:00 2024-09-19T08:34:05.966 INFO:teuthology.orchestra.run.smithi173.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 3.2 MB/s | 161 kB 00:00 2024-09-19T08:34:05.976 INFO:teuthology.orchestra.run.smithi164.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 1.6 MB/s | 137 kB 00:00 2024-09-19T08:34:06.010 INFO:teuthology.orchestra.run.smithi164.stdout:(70/156): postgresql-contrib-13.16-1.el9.x86_64 5.6 MB/s | 910 kB 00:00 2024-09-19T08:34:06.017 INFO:teuthology.orchestra.run.smithi173.stdout:(22/156): python3-rados-19.1.1-361.g2f599a46.el 6.3 MB/s | 320 kB 00:00 2024-09-19T08:34:06.067 INFO:teuthology.orchestra.run.smithi173.stdout:(23/156): python3-rbd-19.1.1-361.g2f599a46.el9. 5.9 MB/s | 300 kB 00:00 2024-09-19T08:34:06.117 INFO:teuthology.orchestra.run.smithi173.stdout:(24/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.9 MB/s | 99 kB 00:00 2024-09-19T08:34:06.152 INFO:teuthology.orchestra.run.smithi164.stdout:(71/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 8.4 MB/s | 4.4 MB 00:00 2024-09-19T08:34:06.167 INFO:teuthology.orchestra.run.smithi173.stdout:(25/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-19T08:34:06.188 INFO:teuthology.orchestra.run.smithi164.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 5.8 MB/s | 1.0 MB 00:00 2024-09-19T08:34:06.253 INFO:teuthology.orchestra.run.smithi164.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 875 kB/s | 87 kB 00:00 2024-09-19T08:34:06.328 INFO:teuthology.orchestra.run.smithi164.stdout:(74/156): postgresql-server-13.16-1.el9.x86_64. 16 MB/s | 5.8 MB 00:00 2024-09-19T08:34:06.386 INFO:teuthology.orchestra.run.smithi164.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 459 kB/s | 61 kB 00:00 2024-09-19T08:34:06.429 INFO:teuthology.orchestra.run.smithi164.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-19T08:34:06.479 INFO:teuthology.orchestra.run.smithi164.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-09-19T08:34:06.504 INFO:teuthology.orchestra.run.smithi164.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 340 kB/s | 26 kB 00:00 2024-09-19T08:34:06.509 INFO:teuthology.orchestra.run.smithi173.stdout:(26/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 5.5 MB/s | 17 MB 00:03 2024-09-19T08:34:06.554 INFO:teuthology.orchestra.run.smithi164.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 255 kB/s | 19 kB 00:00 2024-09-19T08:34:06.568 INFO:teuthology.orchestra.run.smithi173.stdout:(27/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 7.8 MB/s | 3.1 MB 00:00 2024-09-19T08:34:06.593 INFO:teuthology.orchestra.run.smithi173.stdout:(28/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 2.0 MB/s | 171 kB 00:00 2024-09-19T08:34:06.619 INFO:teuthology.orchestra.run.smithi173.stdout:(29/156): ceph-grafana-dashboards-19.1.1-361.g2 637 kB/s | 32 kB 00:00 2024-09-19T08:34:06.622 INFO:teuthology.orchestra.run.smithi164.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 518 kB/s | 35 kB 00:00 2024-09-19T08:34:06.644 INFO:teuthology.orchestra.run.smithi173.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.9 MB/s | 146 kB 00:00 2024-09-19T08:34:06.647 INFO:teuthology.orchestra.run.smithi164.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-09-19T08:34:06.722 INFO:teuthology.orchestra.run.smithi164.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-09-19T08:34:06.781 INFO:teuthology.orchestra.run.smithi170.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T08:34:06.782 INFO:teuthology.orchestra.run.smithi170.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T08:34:06.793 INFO:teuthology.orchestra.run.smithi170.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T08:34:06.794 INFO:teuthology.orchestra.run.smithi170.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T08:34:06.795 INFO:teuthology.orchestra.run.smithi170.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T08:34:06.796 INFO:teuthology.orchestra.run.smithi170.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T08:34:06.797 INFO:teuthology.orchestra.run.smithi170.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T08:34:06.798 INFO:teuthology.orchestra.run.smithi164.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-09-19T08:34:06.800 INFO:teuthology.orchestra.run.smithi170.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T08:34:06.801 INFO:teuthology.orchestra.run.smithi170.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T08:34:06.864 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-09-19T08:34:06.869 INFO:teuthology.orchestra.run.smithi173.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 14 MB/s | 3.6 MB 00:00 2024-09-19T08:34:06.873 INFO:teuthology.orchestra.run.smithi164.stdout:(84/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 250 kB 00:00 2024-09-19T08:34:06.891 INFO:teuthology.orchestra.run.smithi170.stdout:====================================================================================== 2024-09-19T08:34:06.891 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2024-09-19T08:34:06.891 INFO:teuthology.orchestra.run.smithi170.stdout:====================================================================================== 2024-09-19T08:34:06.892 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2024-09-19T08:34:06.892 INFO:teuthology.orchestra.run.smithi170.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T08:34:06.892 INFO:teuthology.orchestra.run.smithi170.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T08:34:06.892 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T08:34:06.892 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T08:34:06.892 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T08:34:06.892 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T08:34:06.892 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T08:34:06.892 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T08:34:06.892 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T08:34:06.892 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T08:34:06.892 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T08:34:06.893 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T08:34:06.893 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T08:34:06.893 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T08:34:06.893 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T08:34:06.893 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T08:34:06.893 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T08:34:06.893 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T08:34:06.893 INFO:teuthology.orchestra.run.smithi170.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T08:34:06.893 INFO:teuthology.orchestra.run.smithi170.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T08:34:06.893 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T08:34:06.893 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T08:34:06.893 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T08:34:06.894 INFO:teuthology.orchestra.run.smithi170.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T08:34:06.894 INFO:teuthology.orchestra.run.smithi170.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T08:34:06.894 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T08:34:06.894 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T08:34:06.894 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T08:34:06.894 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T08:34:06.894 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T08:34:06.894 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T08:34:06.894 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T08:34:06.894 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T08:34:06.894 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T08:34:06.894 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T08:34:06.895 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T08:34:06.895 INFO:teuthology.orchestra.run.smithi170.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T08:34:06.895 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrading: 2024-09-19T08:34:06.895 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T08:34:06.895 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T08:34:06.895 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T08:34:06.895 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T08:34:06.895 INFO:teuthology.orchestra.run.smithi170.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T08:34:06.895 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T08:34:06.895 INFO:teuthology.orchestra.run.smithi170.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T08:34:06.895 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T08:34:06.895 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2024-09-19T08:34:06.895 INFO:teuthology.orchestra.run.smithi170.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T08:34:06.896 INFO:teuthology.orchestra.run.smithi170.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T08:34:06.896 INFO:teuthology.orchestra.run.smithi170.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T08:34:06.896 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T08:34:06.896 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T08:34:06.896 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T08:34:06.896 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T08:34:06.896 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T08:34:06.896 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T08:34:06.896 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T08:34:06.896 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T08:34:06.896 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T08:34:06.896 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T08:34:06.897 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T08:34:06.897 INFO:teuthology.orchestra.run.smithi170.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T08:34:06.897 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T08:34:06.897 INFO:teuthology.orchestra.run.smithi170.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T08:34:06.897 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T08:34:06.897 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T08:34:06.897 INFO:teuthology.orchestra.run.smithi170.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T08:34:06.897 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T08:34:06.897 INFO:teuthology.orchestra.run.smithi170.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T08:34:06.897 INFO:teuthology.orchestra.run.smithi170.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T08:34:06.897 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T08:34:06.897 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T08:34:06.897 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T08:34:06.898 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T08:34:06.898 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T08:34:06.898 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T08:34:06.898 INFO:teuthology.orchestra.run.smithi170.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T08:34:06.898 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T08:34:06.898 INFO:teuthology.orchestra.run.smithi170.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T08:34:06.898 INFO:teuthology.orchestra.run.smithi170.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T08:34:06.898 INFO:teuthology.orchestra.run.smithi170.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T08:34:06.898 INFO:teuthology.orchestra.run.smithi170.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T08:34:06.898 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T08:34:06.898 INFO:teuthology.orchestra.run.smithi170.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T08:34:06.898 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T08:34:06.898 INFO:teuthology.orchestra.run.smithi170.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T08:34:06.898 INFO:teuthology.orchestra.run.smithi170.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T08:34:06.899 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T08:34:06.899 INFO:teuthology.orchestra.run.smithi170.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T08:34:06.899 INFO:teuthology.orchestra.run.smithi170.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T08:34:06.899 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T08:34:06.899 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T08:34:06.899 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T08:34:06.899 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T08:34:06.899 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T08:34:06.899 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T08:34:06.899 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T08:34:06.899 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T08:34:06.899 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T08:34:06.900 INFO:teuthology.orchestra.run.smithi170.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T08:34:06.900 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T08:34:06.900 INFO:teuthology.orchestra.run.smithi170.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T08:34:06.900 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T08:34:06.900 INFO:teuthology.orchestra.run.smithi170.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T08:34:06.900 INFO:teuthology.orchestra.run.smithi170.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T08:34:06.900 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T08:34:06.900 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T08:34:06.900 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T08:34:06.900 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T08:34:06.900 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T08:34:06.900 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T08:34:06.900 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T08:34:06.901 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T08:34:06.901 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T08:34:06.901 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T08:34:06.901 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T08:34:06.901 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T08:34:06.901 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T08:34:06.901 INFO:teuthology.orchestra.run.smithi170.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T08:34:06.901 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T08:34:06.901 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T08:34:06.901 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T08:34:06.901 INFO:teuthology.orchestra.run.smithi170.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T08:34:06.901 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T08:34:06.901 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T08:34:06.902 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T08:34:06.902 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T08:34:06.902 INFO:teuthology.orchestra.run.smithi170.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T08:34:06.902 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T08:34:06.902 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T08:34:06.902 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T08:34:06.902 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T08:34:06.902 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T08:34:06.902 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T08:34:06.902 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T08:34:06.902 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T08:34:06.902 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T08:34:06.902 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T08:34:06.903 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T08:34:06.903 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T08:34:06.903 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T08:34:06.903 INFO:teuthology.orchestra.run.smithi170.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T08:34:06.903 INFO:teuthology.orchestra.run.smithi170.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T08:34:06.903 INFO:teuthology.orchestra.run.smithi170.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T08:34:06.903 INFO:teuthology.orchestra.run.smithi170.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T08:34:06.903 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T08:34:06.903 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T08:34:06.903 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T08:34:06.903 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T08:34:06.903 INFO:teuthology.orchestra.run.smithi170.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T08:34:06.904 INFO:teuthology.orchestra.run.smithi170.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T08:34:06.904 INFO:teuthology.orchestra.run.smithi170.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T08:34:06.904 INFO:teuthology.orchestra.run.smithi170.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T08:34:06.904 INFO:teuthology.orchestra.run.smithi170.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T08:34:06.904 INFO:teuthology.orchestra.run.smithi170.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T08:34:06.904 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T08:34:06.904 INFO:teuthology.orchestra.run.smithi170.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T08:34:06.904 INFO:teuthology.orchestra.run.smithi170.stdout:Installing weak dependencies: 2024-09-19T08:34:06.904 INFO:teuthology.orchestra.run.smithi170.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T08:34:06.904 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-19T08:34:06.904 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-09-19T08:34:06.904 INFO:teuthology.orchestra.run.smithi170.stdout:====================================================================================== 2024-09-19T08:34:06.904 INFO:teuthology.orchestra.run.smithi170.stdout:Install 148 Packages 2024-09-19T08:34:06.904 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrade 8 Packages 2024-09-19T08:34:06.905 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-19T08:34:06.905 INFO:teuthology.orchestra.run.smithi164.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 771 kB/s | 77 kB 00:00 2024-09-19T08:34:06.905 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 226 M 2024-09-19T08:34:06.906 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2024-09-19T08:34:06.919 INFO:teuthology.orchestra.run.smithi173.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 4.9 MB/s | 249 kB 00:00 2024-09-19T08:34:06.957 INFO:teuthology.orchestra.run.smithi164.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 506 kB/s | 42 kB 00:00 2024-09-19T08:34:06.970 INFO:teuthology.orchestra.run.smithi173.stdout:(33/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 988 kB/s | 49 kB 00:00 2024-09-19T08:34:07.007 INFO:teuthology.orchestra.run.smithi164.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-09-19T08:34:07.020 INFO:teuthology.orchestra.run.smithi173.stdout:(34/156): ceph-prometheus-alerts-19.1.1-361.g2f 330 kB/s | 16 kB 00:00 2024-09-19T08:34:07.066 INFO:teuthology.orchestra.run.smithi164.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-19T08:34:07.070 INFO:teuthology.orchestra.run.smithi173.stdout:(35/156): ceph-volume-19.1.1-361.g2f599a46.el9. 5.6 MB/s | 285 kB 00:00 2024-09-19T08:34:07.149 INFO:teuthology.orchestra.run.smithi164.stdout:(89/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.7 MB/s | 5.1 MB 00:00 2024-09-19T08:34:07.154 INFO:teuthology.orchestra.run.smithi173.stdout:(36/156): cephadm-19.1.1-361.g2f599a46.el9.noar 9.0 MB/s | 765 kB 00:00 2024-09-19T08:34:07.174 INFO:teuthology.orchestra.run.smithi164.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 470 kB/s | 51 kB 00:00 2024-09-19T08:34:07.200 INFO:teuthology.orchestra.run.smithi164.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.4 MB/s | 279 kB 00:00 2024-09-19T08:34:07.204 INFO:teuthology.orchestra.run.smithi173.stdout:(37/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 366 kB/s | 18 kB 00:00 2024-09-19T08:34:07.233 INFO:teuthology.orchestra.run.smithi164.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 641 kB/s | 54 kB 00:00 2024-09-19T08:34:07.309 INFO:teuthology.orchestra.run.smithi164.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 383 kB/s | 42 kB 00:00 2024-09-19T08:34:07.334 INFO:teuthology.orchestra.run.smithi164.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 441 kB/s | 44 kB 00:00 2024-09-19T08:34:07.371 INFO:teuthology.orchestra.run.smithi173.stdout:(38/156): fuse-2.9.9-16.el9.x86_64.rpm 479 kB/s | 80 kB 00:00 2024-09-19T08:34:07.392 INFO:teuthology.orchestra.run.smithi164.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 618 kB/s | 36 kB 00:00 2024-09-19T08:34:07.418 INFO:teuthology.orchestra.run.smithi164.stdout:(96/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-09-19T08:34:07.476 INFO:teuthology.orchestra.run.smithi164.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 567 kB/s | 47 kB 00:00 2024-09-19T08:34:07.535 INFO:teuthology.orchestra.run.smithi164.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.5 MB/s | 304 kB 00:00 2024-09-19T08:34:07.560 INFO:teuthology.orchestra.run.smithi164.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 693 kB/s | 58 kB 00:00 2024-09-19T08:34:07.613 INFO:teuthology.orchestra.run.smithi170.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 48 kB/s | 6.6 kB 00:00 2024-09-19T08:34:07.627 INFO:teuthology.orchestra.run.smithi164.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.0 MB/s | 96 kB 00:00 2024-09-19T08:34:07.644 INFO:teuthology.orchestra.run.smithi164.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 763 kB/s | 64 kB 00:00 2024-09-19T08:34:07.695 INFO:teuthology.orchestra.run.smithi164.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 671 kB/s | 45 kB 00:00 2024-09-19T08:34:07.728 INFO:teuthology.orchestra.run.smithi164.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-09-19T08:34:07.787 INFO:teuthology.orchestra.run.smithi164.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-19T08:34:07.813 INFO:teuthology.orchestra.run.smithi173.stdout:(39/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 16 MB/s | 49 MB 00:03 2024-09-19T08:34:07.820 INFO:teuthology.orchestra.run.smithi164.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 9.0 MB/s | 308 kB 00:00 2024-09-19T08:34:07.837 INFO:teuthology.orchestra.run.smithi164.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 1.2 MB/s | 19 kB 00:00 2024-09-19T08:34:07.905 INFO:teuthology.orchestra.run.smithi173.stdout:(40/156): ceph-mgr-diskprediction-local-19.1.1- 5.9 MB/s | 7.4 MB 00:01 2024-09-19T08:34:07.954 INFO:teuthology.orchestra.run.smithi164.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2024-09-19T08:34:07.979 INFO:teuthology.orchestra.run.smithi164.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-19T08:34:07.997 INFO:teuthology.orchestra.run.smithi173.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 801 kB 00:00 2024-09-19T08:34:07.997 INFO:teuthology.orchestra.run.smithi170.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 2.3 MB/s | 896 kB 00:00 2024-09-19T08:34:08.005 INFO:teuthology.orchestra.run.smithi164.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-19T08:34:08.047 INFO:teuthology.orchestra.run.smithi164.stdout:(110/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.4 MB/s | 863 kB 00:00 2024-09-19T08:34:08.055 INFO:teuthology.orchestra.run.smithi170.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 2.4 MB/s | 145 kB 00:00 2024-09-19T08:34:08.072 INFO:teuthology.orchestra.run.smithi164.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-09-19T08:34:08.114 INFO:teuthology.orchestra.run.smithi173.stdout:(42/156): libquadmath-11.5.0-2.el9.x86_64.rpm 636 kB/s | 191 kB 00:00 2024-09-19T08:34:08.131 INFO:teuthology.orchestra.run.smithi173.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-09-19T08:34:08.139 INFO:teuthology.orchestra.run.smithi164.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-09-19T08:34:08.165 INFO:teuthology.orchestra.run.smithi164.stdout:(113/156): luarocks-3.9.2-5.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-09-19T08:34:08.181 INFO:teuthology.orchestra.run.smithi173.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 6.7 MB/s | 1.2 MB 00:00 2024-09-19T08:34:08.206 INFO:teuthology.orchestra.run.smithi173.stdout:(45/156): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-09-19T08:34:08.232 INFO:teuthology.orchestra.run.smithi173.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-09-19T08:34:08.257 INFO:teuthology.orchestra.run.smithi164.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 489 kB/s | 43 kB 00:00 2024-09-19T08:34:08.265 INFO:teuthology.orchestra.run.smithi173.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-19T08:34:08.282 INFO:teuthology.orchestra.run.smithi164.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-19T08:34:08.290 INFO:teuthology.orchestra.run.smithi173.stdout:(48/156): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-09-19T08:34:08.315 INFO:teuthology.orchestra.run.smithi164.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 425 kB/s | 14 kB 00:00 2024-09-19T08:34:08.349 INFO:teuthology.orchestra.run.smithi164.stdout:(117/156): python3-asyncssh-2.13.2-5.el9.noarch 2.6 MB/s | 548 kB 00:00 2024-09-19T08:34:08.375 INFO:teuthology.orchestra.run.smithi164.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-19T08:34:08.481 INFO:teuthology.orchestra.run.smithi170.stdout:(4/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 5.6 MB/s | 2.4 MB 00:00 2024-09-19T08:34:08.499 INFO:teuthology.orchestra.run.smithi173.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 513 kB/s | 107 kB 00:00 2024-09-19T08:34:08.542 INFO:teuthology.orchestra.run.smithi164.stdout:(119/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 12 MB/s | 16 MB 00:01 2024-09-19T08:34:08.558 INFO:teuthology.orchestra.run.smithi173.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 372 kB/s | 22 kB 00:00 2024-09-19T08:34:08.567 INFO:teuthology.orchestra.run.smithi164.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.5 MB/s | 303 kB 00:00 2024-09-19T08:34:08.581 INFO:teuthology.orchestra.run.smithi170.stdout:(5/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 4.9 MB/s | 5.4 MB 00:01 2024-09-19T08:34:08.601 INFO:teuthology.orchestra.run.smithi164.stdout:(121/156): python3-cherrypy-18.6.1-2.el9.noarch 1.4 MB/s | 358 kB 00:00 2024-09-19T08:34:08.625 INFO:teuthology.orchestra.run.smithi173.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 2.6 MB/s | 943 kB 00:00 2024-09-19T08:34:08.634 INFO:teuthology.orchestra.run.smithi164.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.5 MB/s | 144 kB 00:00 2024-09-19T08:34:08.658 INFO:teuthology.orchestra.run.smithi173.stdout:(52/156): automake-1.16.2-8.el9.noarch.rpm 1.6 MB/s | 689 kB 00:00 2024-09-19T08:34:08.659 INFO:teuthology.orchestra.run.smithi164.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-19T08:34:08.675 INFO:teuthology.orchestra.run.smithi173.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 654 kB/s | 33 kB 00:00 2024-09-19T08:34:08.684 INFO:teuthology.orchestra.run.smithi164.stdout:(124/156): python3-google-auth-2.34.0-1.el9.noa 1.5 MB/s | 220 kB 00:00 2024-09-19T08:34:08.700 INFO:teuthology.orchestra.run.smithi173.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 313 kB 00:00 2024-09-19T08:34:08.706 INFO:teuthology.orchestra.run.smithi170.stdout:(6/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 6.8 MB/s | 1.5 MB 00:00 2024-09-19T08:34:08.710 INFO:teuthology.orchestra.run.smithi164.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-09-19T08:34:08.735 INFO:teuthology.orchestra.run.smithi173.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 307 kB/s | 18 kB 00:00 2024-09-19T08:34:08.735 INFO:teuthology.orchestra.run.smithi164.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-19T08:34:08.760 INFO:teuthology.orchestra.run.smithi164.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-09-19T08:34:08.768 INFO:teuthology.orchestra.run.smithi173.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 164 kB/s | 11 kB 00:00 2024-09-19T08:34:08.786 INFO:teuthology.orchestra.run.smithi164.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-19T08:34:08.802 INFO:teuthology.orchestra.run.smithi173.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-09-19T08:34:08.811 INFO:teuthology.orchestra.run.smithi164.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T08:34:08.845 INFO:teuthology.orchestra.run.smithi164.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-19T08:34:08.861 INFO:teuthology.orchestra.run.smithi173.stdout:(58/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2024-09-19T08:34:08.870 INFO:teuthology.orchestra.run.smithi164.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-19T08:34:08.885 INFO:teuthology.orchestra.run.smithi173.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 388 kB/s | 45 kB 00:00 2024-09-19T08:34:08.904 INFO:teuthology.orchestra.run.smithi164.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-09-19T08:34:08.919 INFO:teuthology.orchestra.run.smithi173.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-09-19T08:34:08.923 INFO:teuthology.orchestra.run.smithi170.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 14 MB/s | 4.7 MB 00:00 2024-09-19T08:34:08.929 INFO:teuthology.orchestra.run.smithi164.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 665 kB/s | 16 kB 00:00 2024-09-19T08:34:08.945 INFO:teuthology.orchestra.run.smithi173.stdout:(61/156): libtool-2.4.6-46.el9.x86_64.rpm 6.8 MB/s | 581 kB 00:00 2024-09-19T08:34:08.954 INFO:teuthology.orchestra.run.smithi164.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-19T08:34:08.970 INFO:teuthology.orchestra.run.smithi173.stdout:(62/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 291 kB/s | 24 kB 00:00 2024-09-19T08:34:08.979 INFO:teuthology.orchestra.run.smithi164.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-19T08:34:08.995 INFO:teuthology.orchestra.run.smithi173.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 630 kB/s | 47 kB 00:00 2024-09-19T08:34:09.005 INFO:teuthology.orchestra.run.smithi164.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 558 kB/s | 28 kB 00:00 2024-09-19T08:34:09.022 INFO:teuthology.orchestra.run.smithi164.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 730 kB/s | 31 kB 00:00 2024-09-19T08:34:09.029 INFO:teuthology.orchestra.run.smithi173.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-09-19T08:34:09.054 INFO:teuthology.orchestra.run.smithi173.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-19T08:34:09.064 INFO:teuthology.orchestra.run.smithi164.stdout:(138/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.2 MB/s | 2.0 MB 00:00 2024-09-19T08:34:09.080 INFO:teuthology.orchestra.run.smithi173.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 458 kB/s | 38 kB 00:00 2024-09-19T08:34:09.081 INFO:teuthology.orchestra.run.smithi164.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-09-19T08:34:09.106 INFO:teuthology.orchestra.run.smithi164.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-09-19T08:34:09.123 INFO:teuthology.orchestra.run.smithi164.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 858 kB/s | 36 kB 00:00 2024-09-19T08:34:09.148 INFO:teuthology.orchestra.run.smithi164.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 1.1 MB/s | 74 kB 00:00 2024-09-19T08:34:09.174 INFO:teuthology.orchestra.run.smithi164.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-19T08:34:09.188 INFO:teuthology.orchestra.run.smithi173.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-19T08:34:09.199 INFO:teuthology.orchestra.run.smithi164.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-09-19T08:34:09.227 INFO:teuthology.orchestra.run.smithi164.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 379 kB/s | 20 kB 00:00 2024-09-19T08:34:09.252 INFO:teuthology.orchestra.run.smithi164.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.5 MB/s | 191 kB 00:00 2024-09-19T08:34:09.263 INFO:teuthology.orchestra.run.smithi173.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 21 MB/s | 4.9 MB 00:00 2024-09-19T08:34:09.277 INFO:teuthology.orchestra.run.smithi164.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.2 MB/s | 427 kB 00:00 2024-09-19T08:34:09.344 INFO:teuthology.orchestra.run.smithi164.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-19T08:34:09.347 INFO:teuthology.orchestra.run.smithi173.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-09-19T08:34:09.372 INFO:teuthology.orchestra.run.smithi173.stdout:(70/156): postgresql-contrib-13.16-1.el9.x86_64 4.8 MB/s | 910 kB 00:00 2024-09-19T08:34:09.397 INFO:teuthology.orchestra.run.smithi173.stdout:(71/156): postgresql-private-libs-13.16-1.el9.x 1.0 MB/s | 137 kB 00:00 2024-09-19T08:34:09.497 INFO:teuthology.orchestra.run.smithi173.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 878 kB/s | 87 kB 00:00 2024-09-19T08:34:09.531 INFO:teuthology.orchestra.run.smithi173.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 7.6 MB/s | 1.0 MB 00:00 2024-09-19T08:34:09.590 INFO:teuthology.orchestra.run.smithi170.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 10 MB/s | 21 MB 00:02 2024-09-19T08:34:09.611 INFO:teuthology.orchestra.run.smithi164.stdout:(149/156): librados2-19.1.1-361.g2f599a46.el9.x 9.5 MB/s | 3.4 MB 00:00 2024-09-19T08:34:09.614 INFO:teuthology.orchestra.run.smithi173.stdout:(74/156): postgresql-server-13.16-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-09-19T08:34:09.640 INFO:teuthology.orchestra.run.smithi173.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 564 kB/s | 61 kB 00:00 2024-09-19T08:34:09.682 INFO:teuthology.orchestra.run.smithi173.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 3.6 MB/s | 245 kB 00:00 2024-09-19T08:34:09.686 INFO:teuthology.orchestra.run.smithi164.stdout:(150/156): librbd1-19.1.1-361.g2f599a46.el9.x86 7.7 MB/s | 3.2 MB 00:00 2024-09-19T08:34:09.707 INFO:teuthology.orchestra.run.smithi170.stdout:(9/156): ceph-radosgw-19.1.1-361.g2f599a46.el9. 14 MB/s | 11 MB 00:00 2024-09-19T08:34:09.724 INFO:teuthology.orchestra.run.smithi173.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-09-19T08:34:09.732 INFO:teuthology.orchestra.run.smithi170.stdout:(10/156): ceph-selinux-19.1.1-361.g2f599a46.el9 177 kB/s | 25 kB 00:00 2024-09-19T08:34:09.778 INFO:teuthology.orchestra.run.smithi164.stdout:(151/156): NetworkManager-team-1.51.0-1.el9.x86 447 kB/s | 41 kB 00:00 2024-09-19T08:34:09.782 INFO:teuthology.orchestra.run.smithi170.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 4.5 MB/s | 228 kB 00:00 2024-09-19T08:34:09.791 INFO:teuthology.orchestra.run.smithi173.stdout:(78/156): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 6.0 MB 00:00 2024-09-19T08:34:09.816 INFO:teuthology.orchestra.run.smithi173.stdout:(79/156): python3-jsonpatch-1.21-16.el9.noarch. 192 kB/s | 26 kB 00:00 2024-09-19T08:34:09.837 INFO:teuthology.orchestra.run.smithi164.stdout:(152/156): NetworkManager-1.51.0-1.el9.x86_64.r 4.7 MB/s | 2.3 MB 00:00 2024-09-19T08:34:09.841 INFO:teuthology.orchestra.run.smithi173.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 163 kB/s | 19 kB 00:00 2024-09-19T08:34:09.862 INFO:teuthology.orchestra.run.smithi164.stdout:(153/156): NetworkManager-tui-1.51.0-1.el9.x86_ 2.9 MB/s | 249 kB 00:00 2024-09-19T08:34:09.866 INFO:teuthology.orchestra.run.smithi173.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-09-19T08:34:09.933 INFO:teuthology.orchestra.run.smithi173.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-09-19T08:34:09.959 INFO:teuthology.orchestra.run.smithi173.stdout:(83/156): python3-markupsafe-1.1.1-12.el9.x86_6 297 kB/s | 35 kB 00:00 2024-09-19T08:34:09.975 INFO:teuthology.orchestra.run.smithi170.stdout:(12/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 13 MB/s | 17 MB 00:01 2024-09-19T08:34:10.000 INFO:teuthology.orchestra.run.smithi170.stdout:(13/156): cephfs-shell-19.1.1-361.g2f599a46.el9 125 kB/s | 27 kB 00:00 2024-09-19T08:34:10.013 INFO:teuthology.orchestra.run.smithi164.stdout:(154/156): NetworkManager-libnm-1.51.0-1.el9.x8 4.5 MB/s | 1.8 MB 00:00 2024-09-19T08:34:10.025 INFO:teuthology.orchestra.run.smithi170.stdout:(14/156): libcephfs-devel-19.1.1-361.g2f599a46. 635 kB/s | 32 kB 00:00 2024-09-19T08:34:10.034 INFO:teuthology.orchestra.run.smithi173.stdout:(84/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 27 MB/s | 5.1 MB 00:00 2024-09-19T08:34:10.059 INFO:teuthology.orchestra.run.smithi173.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 768 kB/s | 77 kB 00:00 2024-09-19T08:34:10.075 INFO:teuthology.orchestra.run.smithi170.stdout:(15/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 9.8 MB/s | 751 kB 00:00 2024-09-19T08:34:10.085 INFO:teuthology.orchestra.run.smithi173.stdout:(86/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 222 kB 00:00 2024-09-19T08:34:10.101 INFO:teuthology.orchestra.run.smithi170.stdout:(16/156): libcephsqlite-19.1.1-361.g2f599a46.el 2.1 MB/s | 163 kB 00:00 2024-09-19T08:34:10.110 INFO:teuthology.orchestra.run.smithi173.stdout:(87/156): python3-prettytable-0.7.2-27.el9.noar 560 kB/s | 42 kB 00:00 2024-09-19T08:34:10.134 INFO:teuthology.orchestra.run.smithi170.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 2.1 MB/s | 127 kB 00:00 2024-09-19T08:34:10.258 INFO:teuthology.orchestra.run.smithi170.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 8.4 MB/s | 506 kB 00:00 2024-09-19T08:34:10.258 INFO:teuthology.orchestra.run.smithi170.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 772 kB/s | 45 kB 00:00 2024-09-19T08:34:10.259 INFO:teuthology.orchestra.run.smithi173.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-09-19T08:34:10.259 INFO:teuthology.orchestra.run.smithi173.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-09-19T08:34:10.259 INFO:teuthology.orchestra.run.smithi173.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-09-19T08:34:10.259 INFO:teuthology.orchestra.run.smithi173.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 760 kB/s | 51 kB 00:00 2024-09-19T08:34:10.259 INFO:teuthology.orchestra.run.smithi173.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 915 kB/s | 54 kB 00:00 2024-09-19T08:34:10.264 INFO:teuthology.orchestra.run.smithi173.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 709 kB/s | 42 kB 00:00 2024-09-19T08:34:10.279 INFO:teuthology.orchestra.run.smithi173.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 752 kB/s | 44 kB 00:00 2024-09-19T08:34:10.280 INFO:teuthology.orchestra.run.smithi164.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 3.1 MB/s | 1.4 MB 00:00 2024-09-19T08:34:10.285 INFO:teuthology.orchestra.run.smithi170.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 2.0 MB/s | 136 kB 00:00 2024-09-19T08:34:10.333 INFO:teuthology.orchestra.run.smithi173.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-09-19T08:34:10.346 INFO:teuthology.orchestra.run.smithi173.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 537 kB/s | 36 kB 00:00 2024-09-19T08:34:10.360 INFO:teuthology.orchestra.run.smithi170.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 2.1 MB/s | 161 kB 00:00 2024-09-19T08:34:10.389 INFO:teuthology.orchestra.run.smithi173.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 811 kB/s | 47 kB 00:00 2024-09-19T08:34:10.430 INFO:teuthology.orchestra.run.smithi173.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 3.6 MB/s | 304 kB 00:00 2024-09-19T08:34:10.447 INFO:teuthology.orchestra.run.smithi173.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 984 kB/s | 58 kB 00:00 2024-09-19T08:34:10.471 INFO:teuthology.orchestra.run.smithi164.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 3.9 MB/s | 2.4 MB 00:00 2024-09-19T08:34:10.481 INFO:teuthology.orchestra.run.smithi170.stdout:(22/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 16 MB/s | 5.3 MB 00:00 2024-09-19T08:34:10.481 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-19T08:34:10.481 INFO:teuthology.orchestra.run.smithi164.stdout:Total 22 MB/s | 226 MB 00:10 2024-09-19T08:34:10.497 INFO:teuthology.orchestra.run.smithi173.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.4 MB/s | 96 kB 00:00 2024-09-19T08:34:10.503 INFO:teuthology.orchestra.run.smithi170.stdout:(23/156): python3-rados-19.1.1-361.g2f599a46.el 2.2 MB/s | 320 kB 00:00 2024-09-19T08:34:10.528 INFO:teuthology.orchestra.run.smithi170.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 5.9 MB/s | 300 kB 00:00 2024-09-19T08:34:10.554 INFO:teuthology.orchestra.run.smithi170.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.9 MB/s | 99 kB 00:00 2024-09-19T08:34:10.579 INFO:teuthology.orchestra.run.smithi170.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-19T08:34:10.654 INFO:teuthology.orchestra.run.smithi170.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 2.2 MB/s | 171 kB 00:00 2024-09-19T08:34:10.655 INFO:teuthology.orchestra.run.smithi173.stdout:(101/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-09-19T08:34:10.672 INFO:teuthology.orchestra.run.smithi173.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 255 kB/s | 45 kB 00:00 2024-09-19T08:34:10.690 INFO:teuthology.orchestra.run.smithi173.stdout:(103/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 263 kB/s | 64 kB 00:00 2024-09-19T08:34:10.738 INFO:teuthology.orchestra.run.smithi170.stdout:(28/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 17 MB/s | 3.1 MB 00:00 2024-09-19T08:34:10.748 INFO:teuthology.orchestra.run.smithi173.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-19T08:34:10.763 INFO:teuthology.orchestra.run.smithi170.stdout:(29/156): ceph-grafana-dashboards-19.1.1-361.g2 294 kB/s | 32 kB 00:00 2024-09-19T08:34:10.764 INFO:teuthology.orchestra.run.smithi173.stdout:(105/156): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-09-19T08:34:10.788 INFO:teuthology.orchestra.run.smithi170.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.9 MB/s | 146 kB 00:00 2024-09-19T08:34:10.815 INFO:teuthology.orchestra.run.smithi173.stdout:(106/156): gperftools-libs-2.9.1-3.el9.x86_64.r 4.5 MB/s | 308 kB 00:00 2024-09-19T08:34:10.840 INFO:teuthology.orchestra.run.smithi173.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 258 kB/s | 19 kB 00:00 2024-09-19T08:34:10.879 INFO:teuthology.orchestra.run.smithi173.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 675 kB/s | 26 kB 00:00 2024-09-19T08:34:10.916 INFO:teuthology.orchestra.run.smithi173.stdout:(109/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.5 MB/s | 863 kB 00:00 2024-09-19T08:34:10.941 INFO:teuthology.orchestra.run.smithi173.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 786 kB/s | 49 kB 00:00 2024-09-19T08:34:11.000 INFO:teuthology.orchestra.run.smithi173.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-09-19T08:34:11.025 INFO:teuthology.orchestra.run.smithi173.stdout:(112/156): libunwind-1.6.2-1.el9.x86_64.rpm 618 kB/s | 67 kB 00:00 2024-09-19T08:34:11.039 INFO:teuthology.orchestra.run.smithi170.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 13 MB/s | 3.6 MB 00:00 2024-09-19T08:34:11.084 INFO:teuthology.orchestra.run.smithi173.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 839 kB 00:00 2024-09-19T08:34:11.109 INFO:teuthology.orchestra.run.smithi173.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-09-19T08:34:11.114 INFO:teuthology.orchestra.run.smithi170.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 3.3 MB/s | 249 kB 00:00 2024-09-19T08:34:11.134 INFO:teuthology.orchestra.run.smithi173.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 865 kB/s | 43 kB 00:00 2024-09-19T08:34:11.159 INFO:teuthology.orchestra.run.smithi173.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-09-19T08:34:11.185 INFO:teuthology.orchestra.run.smithi173.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 282 kB/s | 14 kB 00:00 2024-09-19T08:34:11.189 INFO:teuthology.orchestra.run.smithi170.stdout:(33/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 659 kB/s | 49 kB 00:00 2024-09-19T08:34:11.210 INFO:teuthology.orchestra.run.smithi173.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-09-19T08:34:11.244 INFO:teuthology.orchestra.run.smithi173.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-19T08:34:11.269 INFO:teuthology.orchestra.run.smithi173.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.0 MB/s | 303 kB 00:00 2024-09-19T08:34:11.281 INFO:teuthology.orchestra.run.smithi170.stdout:(34/156): ceph-mgr-diskprediction-local-19.1.1- 15 MB/s | 7.4 MB 00:00 2024-09-19T08:34:11.294 INFO:teuthology.orchestra.run.smithi173.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-19T08:34:11.306 INFO:teuthology.orchestra.run.smithi170.stdout:(35/156): ceph-prometheus-alerts-19.1.1-361.g2f 140 kB/s | 16 kB 00:00 2024-09-19T08:34:11.328 INFO:teuthology.orchestra.run.smithi173.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-19T08:34:11.340 INFO:teuthology.orchestra.run.smithi170.stdout:(36/156): ceph-volume-19.1.1-361.g2f599a46.el9. 4.8 MB/s | 285 kB 00:00 2024-09-19T08:34:11.353 INFO:teuthology.orchestra.run.smithi173.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-19T08:34:11.373 INFO:teuthology.orchestra.run.smithi170.stdout:(37/156): cephadm-19.1.1-361.g2f599a46.el9.noar 11 MB/s | 765 kB 00:00 2024-09-19T08:34:11.378 INFO:teuthology.orchestra.run.smithi173.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 715 kB/s | 18 kB 00:00 2024-09-19T08:34:11.399 INFO:teuthology.orchestra.run.smithi170.stdout:(38/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 312 kB/s | 18 kB 00:00 2024-09-19T08:34:11.403 INFO:teuthology.orchestra.run.smithi173.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 922 kB/s | 23 kB 00:00 2024-09-19T08:34:11.429 INFO:teuthology.orchestra.run.smithi173.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 782 kB/s | 19 kB 00:00 2024-09-19T08:34:11.488 INFO:teuthology.orchestra.run.smithi173.stdout:(127/156): libarrow-9.0.0-11.el9.x86_64.rpm 6.6 MB/s | 4.4 MB 00:00 2024-09-19T08:34:11.513 INFO:teuthology.orchestra.run.smithi173.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 235 kB/s | 20 kB 00:00 2024-09-19T08:34:11.563 INFO:teuthology.orchestra.run.smithi173.stdout:(129/156): python3-grpcio-1.46.7-10.el9.x86_64. 6.9 MB/s | 2.0 MB 00:00 2024-09-19T08:34:11.588 INFO:teuthology.orchestra.run.smithi173.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 615 kB/s | 46 kB 00:00 2024-09-19T08:34:11.613 INFO:teuthology.orchestra.run.smithi173.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-09-19T08:34:11.640 INFO:teuthology.orchestra.run.smithi173.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-19T08:34:11.664 INFO:teuthology.orchestra.run.smithi173.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 662 kB/s | 16 kB 00:00 2024-09-19T08:34:11.689 INFO:teuthology.orchestra.run.smithi173.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-19T08:34:11.706 INFO:teuthology.orchestra.run.smithi173.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.1 MB/s | 90 kB 00:00 2024-09-19T08:34:11.726 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-19T08:34:11.731 INFO:teuthology.orchestra.run.smithi173.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 674 kB/s | 28 kB 00:00 2024-09-19T08:34:11.757 INFO:teuthology.orchestra.run.smithi173.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-09-19T08:34:11.791 INFO:teuthology.orchestra.run.smithi173.stdout:(138/156): python3-kubernetes-26.1.0-2.el9.noar 3.4 MB/s | 1.0 MB 00:00 2024-09-19T08:34:11.816 INFO:teuthology.orchestra.run.smithi173.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-09-19T08:34:11.841 INFO:teuthology.orchestra.run.smithi173.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 710 kB/s | 36 kB 00:00 2024-09-19T08:34:11.867 INFO:teuthology.orchestra.run.smithi173.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-09-19T08:34:11.880 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-19T08:34:11.880 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-19T08:34:11.891 INFO:teuthology.orchestra.run.smithi173.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 987 kB/s | 74 kB 00:00 2024-09-19T08:34:11.917 INFO:teuthology.orchestra.run.smithi173.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-19T08:34:11.924 INFO:teuthology.orchestra.run.smithi170.stdout:(39/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 22 MB/s | 49 MB 00:02 2024-09-19T08:34:11.942 INFO:teuthology.orchestra.run.smithi173.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-19T08:34:11.949 INFO:teuthology.orchestra.run.smithi170.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 139 kB/s | 80 kB 00:00 2024-09-19T08:34:11.967 INFO:teuthology.orchestra.run.smithi173.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-09-19T08:34:11.993 INFO:teuthology.orchestra.run.smithi173.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-19T08:34:12.027 INFO:teuthology.orchestra.run.smithi173.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-19T08:34:12.058 INFO:teuthology.orchestra.run.smithi170.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-09-19T08:34:12.076 INFO:teuthology.orchestra.run.smithi173.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-19T08:34:12.083 INFO:teuthology.orchestra.run.smithi170.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 801 kB 00:00 2024-09-19T08:34:12.109 INFO:teuthology.orchestra.run.smithi170.stdout:(43/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 191 kB 00:00 2024-09-19T08:34:12.125 INFO:teuthology.orchestra.run.smithi170.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-19T08:34:12.159 INFO:teuthology.orchestra.run.smithi170.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-19T08:34:12.184 INFO:teuthology.orchestra.run.smithi170.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-19T08:34:12.243 INFO:teuthology.orchestra.run.smithi170.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-19T08:34:12.377 INFO:teuthology.orchestra.run.smithi170.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 3.9 MB/s | 1.2 MB 00:00 2024-09-19T08:34:12.660 INFO:teuthology.orchestra.run.smithi173.stdout:(149/156): librados2-19.1.1-361.g2f599a46.el9.x 5.1 MB/s | 3.4 MB 00:00 2024-09-19T08:34:12.761 INFO:teuthology.orchestra.run.smithi173.stdout:(150/156): NetworkManager-1.51.0-1.el9.x86_64.r 3.4 MB/s | 2.3 MB 00:00 2024-09-19T08:34:12.819 INFO:teuthology.orchestra.run.smithi173.stdout:(151/156): NetworkManager-team-1.51.0-1.el9.x86 704 kB/s | 41 kB 00:00 2024-09-19T08:34:12.852 INFO:teuthology.orchestra.run.smithi170.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 225 kB/s | 107 kB 00:00 2024-09-19T08:34:12.972 INFO:teuthology.orchestra.run.smithi170.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 182 kB/s | 22 kB 00:00 2024-09-19T08:34:12.986 INFO:teuthology.orchestra.run.smithi173.stdout:(152/156): librbd1-19.1.1-361.g2f599a46.el9.x86 3.3 MB/s | 3.2 MB 00:00 2024-09-19T08:34:13.011 INFO:teuthology.orchestra.run.smithi173.stdout:(153/156): NetworkManager-tui-1.51.0-1.el9.x86_ 1.3 MB/s | 249 kB 00:00 2024-09-19T08:34:13.030 INFO:teuthology.orchestra.run.smithi170.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 815 kB/s | 689 kB 00:00 2024-09-19T08:34:13.136 INFO:teuthology.orchestra.run.smithi173.stdout:(154/156): NetworkManager-libnm-1.51.0-1.el9.x8 3.8 MB/s | 1.8 MB 00:00 2024-09-19T08:34:13.239 INFO:teuthology.orchestra.run.smithi170.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 947 kB/s | 943 kB 00:00 2024-09-19T08:34:13.264 INFO:teuthology.orchestra.run.smithi170.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 140 kB/s | 33 kB 00:00 2024-09-19T08:34:13.345 INFO:teuthology.orchestra.run.smithi173.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 3.8 MB/s | 1.4 MB 00:00 2024-09-19T08:34:13.373 INFO:teuthology.orchestra.run.smithi170.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-09-19T08:34:13.398 INFO:teuthology.orchestra.run.smithi170.stdout:(55/156): flex-2.6.4-9.el9.x86_64.rpm 736 kB/s | 313 kB 00:00 2024-09-19T08:34:13.481 INFO:teuthology.orchestra.run.smithi170.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 11 kB 00:00 2024-09-19T08:34:13.598 INFO:teuthology.orchestra.run.smithi170.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 387 kB/s | 45 kB 00:00 2024-09-19T08:34:13.624 INFO:teuthology.orchestra.run.smithi170.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 710 kB/s | 160 kB 00:00 2024-09-19T08:34:13.907 INFO:teuthology.orchestra.run.smithi170.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 2.1 MB/s | 662 kB 00:00 2024-09-19T08:34:13.932 INFO:teuthology.orchestra.run.smithi170.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 1.8 MB/s | 581 kB 00:00 2024-09-19T08:34:14.033 INFO:teuthology.orchestra.run.smithi170.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 196 kB/s | 24 kB 00:00 2024-09-19T08:34:14.058 INFO:teuthology.orchestra.run.smithi170.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 378 kB/s | 47 kB 00:00 2024-09-19T08:34:14.158 INFO:teuthology.orchestra.run.smithi170.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-19T08:34:14.161 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-19T08:34:14.162 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-19T08:34:14.233 INFO:teuthology.orchestra.run.smithi170.stdout:(64/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.0 MB/s | 3.0 MB 00:00 2024-09-19T08:34:14.259 INFO:teuthology.orchestra.run.smithi170.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 940 kB/s | 188 kB 00:00 2024-09-19T08:34:14.284 INFO:teuthology.orchestra.run.smithi170.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 307 kB/s | 38 kB 00:00 2024-09-19T08:34:14.295 INFO:teuthology.orchestra.run.smithi173.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 1.9 MB/s | 2.4 MB 00:01 2024-09-19T08:34:14.301 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-09-19T08:34:14.301 INFO:teuthology.orchestra.run.smithi173.stdout:Total 18 MB/s | 226 MB 00:12 2024-09-19T08:34:14.517 INFO:teuthology.orchestra.run.smithi170.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 6.8 MB/s | 1.6 MB 00:00 2024-09-19T08:34:14.643 INFO:teuthology.orchestra.run.smithi170.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 7.1 MB/s | 910 kB 00:00 2024-09-19T08:34:14.768 INFO:teuthology.orchestra.run.smithi170.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-19T08:34:15.524 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-09-19T08:34:15.673 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-09-19T08:34:15.673 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-09-19T08:34:15.843 INFO:teuthology.orchestra.run.smithi170.stdout:(70/156): openblas-openmp-0.3.26-2.el9.x86_64.r 3.1 MB/s | 4.9 MB 00:01 2024-09-19T08:34:16.210 INFO:teuthology.orchestra.run.smithi170.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 2.8 MB/s | 1.0 MB 00:00 2024-09-19T08:34:16.327 INFO:teuthology.orchestra.run.smithi170.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 751 kB/s | 87 kB 00:00 2024-09-19T08:34:16.686 INFO:teuthology.orchestra.run.smithi170.stdout:(73/156): postgresql-server-13.16-1.el9.x86_64. 3.0 MB/s | 5.8 MB 00:01 2024-09-19T08:34:16.794 INFO:teuthology.orchestra.run.smithi170.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 566 kB/s | 61 kB 00:00 2024-09-19T08:34:16.911 INFO:teuthology.orchestra.run.smithi170.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.0 MB/s | 245 kB 00:00 2024-09-19T08:34:17.029 INFO:teuthology.orchestra.run.smithi170.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-09-19T08:34:17.145 INFO:teuthology.orchestra.run.smithi170.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 220 kB/s | 26 kB 00:00 2024-09-19T08:34:17.271 INFO:teuthology.orchestra.run.smithi170.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 153 kB/s | 19 kB 00:00 2024-09-19T08:34:17.396 INFO:teuthology.orchestra.run.smithi170.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-19T08:34:17.513 INFO:teuthology.orchestra.run.smithi170.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 298 kB/s | 35 kB 00:00 2024-09-19T08:34:17.551 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-09-19T08:34:17.552 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-09-19T08:34:17.971 INFO:teuthology.orchestra.run.smithi170.stdout:(81/156): python3-babel-2.9.1-2.el9.noarch.rpm 3.6 MB/s | 6.0 MB 00:01 2024-09-19T08:34:18.030 INFO:teuthology.orchestra.run.smithi170.stdout:(82/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 1.2 MB/s | 4.4 MB 00:03 2024-09-19T08:34:18.089 INFO:teuthology.orchestra.run.smithi170.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-09-19T08:34:18.147 INFO:teuthology.orchestra.run.smithi170.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.9 MB/s | 222 kB 00:00 2024-09-19T08:34:18.189 INFO:teuthology.orchestra.run.smithi170.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 770 kB/s | 77 kB 00:00 2024-09-19T08:34:18.256 INFO:teuthology.orchestra.run.smithi170.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 389 kB/s | 42 kB 00:00 2024-09-19T08:34:18.306 INFO:teuthology.orchestra.run.smithi170.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 269 kB 00:00 2024-09-19T08:34:18.373 INFO:teuthology.orchestra.run.smithi170.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-19T08:34:18.415 INFO:teuthology.orchestra.run.smithi170.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-09-19T08:34:18.490 INFO:teuthology.orchestra.run.smithi170.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 436 kB/s | 51 kB 00:00 2024-09-19T08:34:18.532 INFO:teuthology.orchestra.run.smithi170.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 459 kB/s | 54 kB 00:00 2024-09-19T08:34:18.649 INFO:teuthology.orchestra.run.smithi170.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 357 kB/s | 42 kB 00:00 2024-09-19T08:34:18.766 INFO:teuthology.orchestra.run.smithi170.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 378 kB/s | 44 kB 00:00 2024-09-19T08:34:18.883 INFO:teuthology.orchestra.run.smithi170.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.8 MB/s | 221 kB 00:00 2024-09-19T08:34:19.000 INFO:teuthology.orchestra.run.smithi170.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 308 kB/s | 36 kB 00:00 2024-09-19T08:34:19.134 INFO:teuthology.orchestra.run.smithi170.stdout:(96/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.1 MB/s | 5.1 MB 00:01 2024-09-19T08:34:19.159 INFO:teuthology.orchestra.run.smithi170.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 299 kB/s | 47 kB 00:00 2024-09-19T08:34:19.259 INFO:teuthology.orchestra.run.smithi170.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.4 MB/s | 304 kB 00:00 2024-09-19T08:34:19.285 INFO:teuthology.orchestra.run.smithi170.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 461 kB/s | 58 kB 00:00 2024-09-19T08:34:19.368 INFO:teuthology.orchestra.run.smithi170.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 883 kB/s | 96 kB 00:00 2024-09-19T08:34:19.402 INFO:teuthology.orchestra.run.smithi170.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 544 kB/s | 64 kB 00:00 2024-09-19T08:34:19.460 INFO:teuthology.orchestra.run.smithi170.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 383 kB/s | 22 kB 00:00 2024-09-19T08:34:19.485 INFO:teuthology.orchestra.run.smithi170.stdout:(103/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 382 kB/s | 45 kB 00:00 2024-09-19T08:34:19.569 INFO:teuthology.orchestra.run.smithi170.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 6.4 MB/s | 548 kB 00:00 2024-09-19T08:34:19.603 INFO:teuthology.orchestra.run.smithi170.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 9.0 MB/s | 308 kB 00:00 2024-09-19T08:34:19.637 INFO:teuthology.orchestra.run.smithi170.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 585 kB/s | 19 kB 00:00 2024-09-19T08:34:19.703 INFO:teuthology.orchestra.run.smithi170.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.5 MB/s | 863 kB 00:00 2024-09-19T08:34:19.812 INFO:teuthology.orchestra.run.smithi170.stdout:(108/156): libarrow-9.0.0-11.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-09-19T08:34:19.837 INFO:teuthology.orchestra.run.smithi170.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 194 kB/s | 26 kB 00:00 2024-09-19T08:34:19.863 INFO:teuthology.orchestra.run.smithi170.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 970 kB/s | 49 kB 00:00 2024-09-19T08:34:19.888 INFO:teuthology.orchestra.run.smithi170.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-19T08:34:19.913 INFO:teuthology.orchestra.run.smithi170.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-09-19T08:34:19.955 INFO:teuthology.orchestra.run.smithi170.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-19T08:34:19.997 INFO:teuthology.orchestra.run.smithi170.stdout:(114/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-09-19T08:34:20.014 INFO:teuthology.orchestra.run.smithi170.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 744 kB/s | 43 kB 00:00 2024-09-19T08:34:20.032 INFO:teuthology.orchestra.run.smithi170.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 962 kB/s | 32 kB 00:00 2024-09-19T08:34:20.048 INFO:teuthology.orchestra.run.smithi170.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 420 kB/s | 14 kB 00:00 2024-09-19T08:34:20.073 INFO:teuthology.orchestra.run.smithi170.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 4.0 MB/s | 172 kB 00:00 2024-09-19T08:34:20.099 INFO:teuthology.orchestra.run.smithi170.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-09-19T08:34:20.124 INFO:teuthology.orchestra.run.smithi170.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-19T08:34:20.149 INFO:teuthology.orchestra.run.smithi170.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-19T08:34:20.183 INFO:teuthology.orchestra.run.smithi170.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-19T08:34:20.208 INFO:teuthology.orchestra.run.smithi170.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-19T08:34:20.250 INFO:teuthology.orchestra.run.smithi170.stdout:(124/156): python3-grpcio-1.46.7-10.el9.x86_64. 16 MB/s | 2.0 MB 00:00 2024-09-19T08:34:20.275 INFO:teuthology.orchestra.run.smithi170.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 264 kB/s | 18 kB 00:00 2024-09-19T08:34:20.301 INFO:teuthology.orchestra.run.smithi170.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-19T08:34:20.326 INFO:teuthology.orchestra.run.smithi170.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-09-19T08:34:20.351 INFO:teuthology.orchestra.run.smithi170.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-19T08:34:20.377 INFO:teuthology.orchestra.run.smithi170.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T08:34:20.402 INFO:teuthology.orchestra.run.smithi170.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-19T08:34:20.427 INFO:teuthology.orchestra.run.smithi170.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-19T08:34:20.461 INFO:teuthology.orchestra.run.smithi170.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-09-19T08:34:20.462 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-19T08:34:20.486 INFO:teuthology.orchestra.run.smithi170.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-09-19T08:34:20.520 INFO:teuthology.orchestra.run.smithi170.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 279 kB/s | 16 kB 00:00 2024-09-19T08:34:20.545 INFO:teuthology.orchestra.run.smithi170.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-19T08:34:20.570 INFO:teuthology.orchestra.run.smithi170.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-19T08:34:20.588 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T08:34:20.596 INFO:teuthology.orchestra.run.smithi170.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-19T08:34:20.621 INFO:teuthology.orchestra.run.smithi170.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-19T08:34:20.638 INFO:teuthology.orchestra.run.smithi170.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.7 MB/s | 188 kB 00:00 2024-09-19T08:34:20.663 INFO:teuthology.orchestra.run.smithi170.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 854 kB/s | 36 kB 00:00 2024-09-19T08:34:20.688 INFO:teuthology.orchestra.run.smithi170.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-19T08:34:20.714 INFO:teuthology.orchestra.run.smithi170.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-19T08:34:20.733 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T08:34:20.739 INFO:teuthology.orchestra.run.smithi170.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-19T08:34:20.764 INFO:teuthology.orchestra.run.smithi170.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 802 kB/s | 20 kB 00:00 2024-09-19T08:34:20.790 INFO:teuthology.orchestra.run.smithi170.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-09-19T08:34:20.815 INFO:teuthology.orchestra.run.smithi170.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-19T08:34:20.865 INFO:teuthology.orchestra.run.smithi170.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-19T08:34:20.908 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T08:34:21.009 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T08:34:21.035 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T08:34:21.090 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T08:34:21.141 INFO:teuthology.orchestra.run.smithi170.stdout:(148/156): librbd1-19.1.1-361.g2f599a46.el9.x86 11 MB/s | 3.2 MB 00:00 2024-09-19T08:34:21.349 INFO:teuthology.orchestra.run.smithi170.stdout:(149/156): librados2-19.1.1-361.g2f599a46.el9.x 6.4 MB/s | 3.4 MB 00:00 2024-09-19T08:34:21.419 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T08:34:21.536 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T08:34:21.641 INFO:teuthology.orchestra.run.smithi170.stdout:(150/156): NetworkManager-1.51.0-1.el9.x86_64.r 4.6 MB/s | 2.3 MB 00:00 2024-09-19T08:34:21.641 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T08:34:21.670 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T08:34:21.708 INFO:teuthology.orchestra.run.smithi170.stdout:(151/156): NetworkManager-team-1.51.0-1.el9.x86 614 kB/s | 41 kB 00:00 2024-09-19T08:34:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T08:34:21.766 INFO:teuthology.orchestra.run.smithi170.stdout:(152/156): NetworkManager-libnm-1.51.0-1.el9.x8 4.3 MB/s | 1.8 MB 00:00 2024-09-19T08:34:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T08:34:21.834 INFO:teuthology.orchestra.run.smithi170.stdout:(153/156): NetworkManager-tui-1.51.0-1.el9.x86_ 1.9 MB/s | 249 kB 00:00 2024-09-19T08:34:21.972 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T08:34:22.031 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T08:34:22.058 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T08:34:22.154 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T08:34:22.184 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T08:34:22.230 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T08:34:22.260 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T08:34:22.281 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T08:34:22.359 INFO:teuthology.orchestra.run.smithi170.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 2.3 MB/s | 1.4 MB 00:00 2024-09-19T08:34:22.463 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T08:34:22.526 INFO:teuthology.orchestra.run.smithi170.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 3.5 MB/s | 2.4 MB 00:00 2024-09-19T08:34:22.636 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T08:34:22.742 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T08:34:22.855 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T08:34:22.935 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T08:34:23.155 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T08:34:23.271 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T08:34:23.368 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T08:34:23.403 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T08:34:23.442 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T08:34:23.505 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T08:34:23.551 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T08:34:23.583 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T08:34:23.635 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T08:34:23.676 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T08:34:23.704 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T08:34:23.771 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T08:34:23.825 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T08:34:23.831 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-09-19T08:34:23.857 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T08:34:23.901 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T08:34:23.932 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T08:34:24.003 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T08:34:24.036 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T08:34:24.062 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T08:34:24.080 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T08:34:24.109 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T08:34:24.178 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T08:34:24.232 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T08:34:24.250 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T08:34:24.292 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T08:34:24.306 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T08:34:24.317 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T08:34:24.360 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T08:34:24.445 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T08:34:24.510 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T08:34:24.617 INFO:teuthology.orchestra.run.smithi170.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 2.7 MB/s | 16 MB 00:06 2024-09-19T08:34:24.623 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2024-09-19T08:34:24.623 INFO:teuthology.orchestra.run.smithi170.stdout:Total 13 MB/s | 226 MB 00:17 2024-09-19T08:34:24.699 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T08:34:24.817 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T08:34:24.931 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T08:34:24.968 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T08:34:25.030 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T08:34:25.059 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T08:34:25.089 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T08:34:25.113 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T08:34:25.142 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T08:34:25.186 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T08:34:25.223 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T08:34:25.235 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T08:34:25.281 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T08:34:25.285 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T08:34:25.314 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T08:34:25.417 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T08:34:25.438 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T08:34:25.440 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T08:34:25.486 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T08:34:25.493 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T08:34:25.508 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T08:34:25.534 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T08:34:25.556 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T08:34:25.637 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T08:34:25.669 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T08:34:25.705 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T08:34:25.751 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T08:34:25.752 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T08:34:25.788 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T08:34:25.831 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T08:34:25.880 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T08:34:25.888 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T08:34:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T08:34:26.017 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T08:34:26.020 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2024-09-19T08:34:26.020 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2024-09-19T08:34:26.020 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2024-09-19T08:34:26.050 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T08:34:26.091 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T08:34:26.106 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T08:34:26.182 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T08:34:26.345 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T08:34:26.384 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T08:34:26.400 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T08:34:26.429 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T08:34:26.457 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T08:34:26.486 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T08:34:26.508 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T08:34:26.582 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T08:34:26.600 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T08:34:26.637 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T08:34:26.673 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T08:34:26.710 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T08:34:26.743 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T08:34:26.762 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T08:34:26.781 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T08:34:26.814 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T08:34:26.832 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T08:34:26.853 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T08:34:26.863 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T08:34:26.884 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T08:34:26.906 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T08:34:26.923 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T08:34:26.939 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T08:34:26.957 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T08:34:26.967 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T08:34:26.991 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T08:34:27.032 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T08:34:27.043 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T08:34:27.067 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T08:34:27.097 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T08:34:27.128 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T08:34:27.152 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T08:34:27.163 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T08:34:27.197 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T08:34:27.231 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T08:34:27.237 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T08:34:27.266 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T08:34:27.310 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T08:34:27.338 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T08:34:27.355 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T08:34:27.407 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T08:34:27.413 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T08:34:27.465 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T08:34:27.539 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T08:34:27.672 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T08:34:27.725 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T08:34:28.070 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2024-09-19T08:34:28.070 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2024-09-19T08:34:28.121 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T08:34:28.177 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T08:34:28.213 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T08:34:28.243 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T08:34:28.319 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T08:34:28.320 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T08:34:28.349 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T08:34:28.368 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T08:34:28.384 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T08:34:28.413 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T08:34:28.422 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T08:34:28.464 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T08:34:28.507 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T08:34:28.541 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T08:34:28.621 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T08:34:28.692 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T08:34:28.724 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T08:34:28.747 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T08:34:28.755 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T08:34:28.820 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T08:34:28.833 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T08:34:28.859 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T08:34:28.896 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T08:34:28.942 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T08:34:28.978 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T08:34:29.022 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T08:34:29.063 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T08:34:29.199 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T08:34:29.258 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T08:34:29.299 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T08:34:29.474 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T08:34:29.568 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T08:34:29.595 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T08:34:29.668 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T08:34:29.706 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T08:34:29.732 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T08:34:29.756 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T08:34:29.831 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T08:34:29.958 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T08:34:30.005 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T08:34:30.033 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T08:34:30.067 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T08:34:30.106 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T08:34:30.138 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T08:34:30.168 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T08:34:30.202 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T08:34:30.235 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T08:34:30.277 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T08:34:30.311 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T08:34:30.397 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T08:34:30.442 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T08:34:30.482 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T08:34:30.583 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T08:34:30.641 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T08:34:30.840 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T08:34:30.935 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T08:34:30.962 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T08:34:30.995 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T08:34:31.306 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T08:34:31.352 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T08:34:31.359 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T08:34:31.404 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T08:34:31.438 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T08:34:31.443 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T08:34:31.474 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T08:34:31.510 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T08:34:31.536 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T08:34:31.595 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T08:34:31.640 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T08:34:31.728 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T08:34:31.767 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T08:34:31.952 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T08:34:31.982 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T08:34:31.992 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T08:34:32.030 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T08:34:32.059 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T08:34:32.102 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T08:34:32.140 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T08:34:32.696 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T08:34:32.805 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T08:34:34.021 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T08:34:34.111 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T08:34:34.138 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T08:34:34.171 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T08:34:34.386 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2024-09-19T08:34:34.435 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T08:34:34.471 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T08:34:34.471 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T08:34:34.479 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T08:34:34.518 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T08:34:34.581 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T08:34:34.617 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T08:34:34.648 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T08:34:34.683 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T08:34:34.853 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T08:34:34.889 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T08:34:34.914 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T08:34:34.967 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T08:34:35.026 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T08:34:35.074 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T08:34:35.166 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T08:34:35.178 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T08:34:35.178 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T08:34:35.178 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T08:34:35.178 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:34:35.196 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T08:34:35.217 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T08:34:35.267 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T08:34:35.296 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T08:34:35.297 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T08:34:35.398 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T08:34:35.520 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T08:34:35.558 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T08:34:35.641 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T08:34:35.670 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T08:34:35.822 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T08:34:35.876 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T08:34:35.904 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T08:34:36.008 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T08:34:36.033 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T08:34:36.093 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T08:34:36.123 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T08:34:36.149 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T08:34:36.307 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T08:34:36.437 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T08:34:36.555 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T08:34:36.649 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T08:34:36.723 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T08:34:36.955 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T08:34:37.067 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T08:34:37.158 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T08:34:37.186 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T08:34:37.221 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T08:34:37.295 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T08:34:37.347 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T08:34:37.379 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T08:34:37.423 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T08:34:37.455 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T08:34:37.483 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T08:34:37.552 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T08:34:37.606 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T08:34:37.640 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T08:34:37.675 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T08:34:37.752 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T08:34:37.794 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T08:34:37.837 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T08:34:37.866 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T08:34:37.867 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T08:34:37.919 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T08:34:37.938 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T08:34:37.989 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T08:34:38.064 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T08:34:38.198 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T08:34:38.250 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T08:34:38.461 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T08:34:38.505 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T08:34:38.584 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T08:34:38.584 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T08:34:38.584 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T08:34:38.584 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-19T08:34:38.615 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T08:34:38.787 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T08:34:38.863 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T08:34:38.892 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T08:34:38.934 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T08:34:38.972 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T08:34:39.014 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T08:34:39.172 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T08:34:39.234 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T08:34:39.289 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T08:34:39.362 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T08:34:39.393 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T08:34:39.429 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T08:34:39.476 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T08:34:39.521 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T08:34:39.563 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T08:34:39.609 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T08:34:39.752 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T08:34:39.801 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T08:34:39.843 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T08:34:39.955 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T08:34:39.993 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T08:34:40.042 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T08:34:40.067 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T08:34:40.092 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T08:34:40.166 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T08:34:40.302 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T08:34:40.366 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T08:34:40.394 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T08:34:40.428 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T08:34:40.467 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T08:34:40.499 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T08:34:40.529 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T08:34:40.563 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T08:34:40.597 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T08:34:40.638 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T08:34:40.673 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T08:34:40.759 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T08:34:40.803 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T08:34:40.834 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T08:34:40.937 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T08:34:40.986 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T08:34:41.708 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T08:34:41.783 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T08:34:41.816 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T08:34:41.918 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T08:34:41.963 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T08:34:42.008 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T08:34:42.105 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T08:34:42.153 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T08:34:42.353 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T08:34:42.383 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T08:34:42.460 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T08:34:43.099 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T08:34:43.215 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T08:34:44.455 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T08:34:44.539 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T08:34:44.567 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T08:34:44.600 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T08:34:44.910 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T08:34:44.955 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T08:34:45.054 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T08:34:45.085 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T08:34:45.121 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T08:34:45.603 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T08:34:45.650 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T08:34:45.705 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T08:34:45.735 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T08:34:47.104 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T08:34:47.104 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /sys 2024-09-19T08:34:47.104 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /proc 2024-09-19T08:34:47.104 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /mnt 2024-09-19T08:34:47.104 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /var/tmp 2024-09-19T08:34:47.104 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /home 2024-09-19T08:34:47.104 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /root 2024-09-19T08:34:47.104 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /tmp 2024-09-19T08:34:47.105 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:34:47.290 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T08:34:47.306 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T08:34:47.306 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:34:47.306 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T08:34:47.306 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:34:48.027 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T08:34:48.065 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T08:34:48.609 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T08:34:48.643 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T08:34:48.722 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T08:34:48.722 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T08:34:48.723 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T08:34:48.723 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-19T08:34:48.753 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T08:34:48.773 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T08:34:48.800 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T08:34:48.800 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:34:48.801 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T08:34:48.801 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T08:34:48.801 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T08:34:48.801 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:34:49.034 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T08:34:49.058 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T08:34:49.058 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:34:49.058 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T08:34:49.058 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T08:34:49.058 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T08:34:49.058 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:34:49.501 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T08:34:49.525 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T08:34:49.525 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:34:49.525 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T08:34:49.525 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T08:34:49.525 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T08:34:49.525 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:34:49.632 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T08:34:49.680 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T08:34:49.736 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T08:34:49.920 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T08:34:49.977 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T08:34:50.055 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T08:34:50.143 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T08:34:50.299 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T08:34:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T08:34:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T08:34:50.412 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T08:34:50.492 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T08:34:50.492 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /sys 2024-09-19T08:34:50.492 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /proc 2024-09-19T08:34:50.492 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /mnt 2024-09-19T08:34:50.492 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /var/tmp 2024-09-19T08:34:50.492 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /home 2024-09-19T08:34:50.492 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /root 2024-09-19T08:34:50.493 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /tmp 2024-09-19T08:34:50.493 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-19T08:34:50.668 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T08:34:50.687 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T08:34:50.687 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:34:50.688 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T08:34:50.688 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-19T08:34:50.850 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T08:34:50.878 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T08:34:50.909 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T08:34:50.972 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T08:34:51.001 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T08:34:52.059 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T08:34:52.094 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T08:34:52.179 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T08:34:52.207 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T08:34:52.207 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:34:52.207 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T08:34:52.207 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T08:34:52.207 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T08:34:52.207 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-19T08:34:52.430 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T08:34:52.453 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T08:34:52.453 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:34:52.454 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T08:34:52.454 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T08:34:52.454 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T08:34:52.454 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-19T08:34:52.889 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T08:34:52.912 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T08:34:52.913 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:34:52.913 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T08:34:52.913 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T08:34:52.913 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T08:34:52.913 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-19T08:34:53.019 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T08:34:53.053 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T08:34:53.068 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T08:34:53.086 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T08:34:53.122 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T08:34:53.188 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T08:34:53.316 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T08:34:53.348 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T08:34:53.368 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T08:34:53.380 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T08:34:53.404 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T08:34:53.404 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:34:53.405 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T08:34:53.405 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T08:34:53.405 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T08:34:53.405 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:34:53.442 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T08:34:53.456 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T08:34:53.468 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T08:34:53.532 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T08:34:53.572 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T08:34:53.630 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T08:34:53.662 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T08:34:53.706 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T08:34:53.738 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T08:34:53.766 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T08:34:54.244 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T08:34:54.273 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T08:34:54.304 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T08:34:54.366 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T08:34:54.396 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T08:34:54.553 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T08:34:54.592 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T08:34:54.616 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T08:34:54.616 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:34:54.617 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T08:34:54.617 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T08:34:54.617 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T08:34:54.617 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:34:54.659 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T08:34:54.683 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T08:34:54.683 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:34:54.683 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T08:34:54.683 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:34:54.727 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T08:34:54.750 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T08:34:54.750 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:34:54.750 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T08:34:54.751 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T08:34:54.751 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T08:34:54.751 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:34:55.061 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T08:34:55.084 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T08:34:55.085 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:34:55.085 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T08:34:55.085 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T08:34:55.085 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T08:34:55.085 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:34:55.448 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T08:34:55.481 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T08:34:56.447 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T08:34:56.473 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T08:34:56.575 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T08:34:56.735 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T08:34:56.759 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T08:34:56.783 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T08:34:56.783 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:34:56.784 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T08:34:56.784 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T08:34:56.784 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T08:34:56.784 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-19T08:34:56.846 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T08:34:56.848 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T08:34:56.944 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T08:34:56.990 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T08:34:57.911 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T08:34:57.937 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T08:34:57.961 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T08:34:57.961 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:34:57.961 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T08:34:57.961 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T08:34:57.962 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T08:34:57.962 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-19T08:34:58.004 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T08:34:58.028 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T08:34:58.028 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:34:58.028 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T08:34:58.028 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-19T08:34:58.088 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T08:34:58.111 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T08:34:58.111 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:34:58.111 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T08:34:58.111 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T08:34:58.112 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T08:34:58.112 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-19T08:34:58.422 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T08:34:58.446 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T08:34:58.446 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:34:58.446 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T08:34:58.447 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T08:34:58.447 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T08:34:58.447 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-19T08:34:59.631 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T08:34:59.672 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T08:34:59.713 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T08:34:59.745 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T08:34:59.781 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T08:34:59.986 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T08:35:00.036 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T08:35:00.067 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T08:35:00.103 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T08:35:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T08:35:00.402 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T08:35:00.402 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /sys 2024-09-19T08:35:00.402 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /proc 2024-09-19T08:35:00.402 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /mnt 2024-09-19T08:35:00.403 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /var/tmp 2024-09-19T08:35:00.403 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /home 2024-09-19T08:35:00.403 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /root 2024-09-19T08:35:00.403 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /tmp 2024-09-19T08:35:00.403 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-19T08:35:00.583 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T08:35:00.599 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T08:35:00.600 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:35:00.600 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T08:35:00.600 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-19T08:35:00.775 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T08:35:00.815 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T08:35:00.836 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T08:35:01.679 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T08:35:01.771 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T08:35:01.822 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T08:35:01.924 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T08:35:01.957 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T08:35:01.983 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T08:35:02.017 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T08:35:02.017 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T08:35:02.077 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T08:35:02.101 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T08:35:02.101 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T08:35:02.104 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T08:35:02.104 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:35:02.104 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T08:35:02.104 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T08:35:02.105 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T08:35:02.105 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-19T08:35:02.185 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T08:35:02.218 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T08:35:02.224 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T08:35:02.224 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T08:35:02.345 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T08:35:02.368 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T08:35:02.368 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:35:02.368 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T08:35:02.368 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T08:35:02.368 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T08:35:02.368 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-19T08:35:02.420 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T08:35:02.421 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T08:35:02.504 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T08:35:02.571 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T08:35:02.812 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T08:35:02.836 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T08:35:02.836 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:35:02.836 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T08:35:02.836 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T08:35:02.836 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T08:35:02.836 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-19T08:35:02.942 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T08:35:02.999 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T08:35:03.001 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T08:35:03.042 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T08:35:03.054 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T08:35:03.083 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T08:35:03.115 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T08:35:03.142 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T08:35:03.242 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T08:35:03.297 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T08:35:03.324 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T08:35:03.372 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T08:35:03.375 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T08:35:03.411 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T08:35:03.449 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T08:35:03.463 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T08:35:03.487 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T08:35:03.597 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T08:35:03.646 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T08:35:03.679 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T08:35:03.706 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T08:35:04.059 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T08:35:04.060 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T08:35:04.088 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T08:35:04.101 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T08:35:04.119 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T08:35:04.122 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T08:35:04.183 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T08:35:04.236 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T08:35:04.950 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T08:35:05.039 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T08:35:05.079 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T08:35:05.159 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T08:35:05.192 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T08:35:05.217 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T08:35:05.252 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T08:35:05.252 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T08:35:05.285 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T08:35:05.322 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T08:35:05.344 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T08:35:05.345 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T08:35:05.437 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T08:35:05.462 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T08:35:05.467 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T08:35:05.467 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T08:35:05.647 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T08:35:05.647 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T08:35:05.731 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T08:35:05.802 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T08:35:06.286 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T08:35:06.314 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T08:35:06.419 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T08:35:06.576 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T08:35:06.607 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T08:35:06.631 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T08:35:06.631 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:35:06.632 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T08:35:06.632 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T08:35:06.632 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T08:35:06.632 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-19T08:35:06.683 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T08:35:06.695 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T08:35:06.793 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T08:35:06.848 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T08:35:07.774 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T08:35:07.810 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T08:35:07.835 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T08:35:07.835 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:35:07.835 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T08:35:07.835 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T08:35:07.835 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T08:35:07.835 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-19T08:35:07.878 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T08:35:07.901 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T08:35:07.901 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:35:07.901 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T08:35:07.901 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-19T08:35:07.953 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T08:35:07.977 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T08:35:07.977 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:35:07.977 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T08:35:07.978 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T08:35:07.978 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T08:35:07.978 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-19T08:35:08.279 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T08:35:08.303 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T08:35:08.303 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T08:35:08.303 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T08:35:08.303 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T08:35:08.304 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T08:35:08.304 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-19T08:35:08.360 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T08:35:08.360 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T08:35:08.360 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T08:35:08.360 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T08:35:08.361 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T08:35:08.361 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T08:35:08.361 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T08:35:08.361 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T08:35:08.361 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T08:35:08.361 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T08:35:08.361 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T08:35:08.361 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T08:35:08.361 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T08:35:08.361 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T08:35:08.361 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T08:35:08.362 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T08:35:08.362 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T08:35:08.362 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T08:35:08.362 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T08:35:08.362 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T08:35:08.362 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T08:35:08.362 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T08:35:08.362 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T08:35:08.362 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T08:35:08.362 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T08:35:08.362 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T08:35:08.362 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T08:35:08.362 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T08:35:08.363 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T08:35:08.363 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T08:35:08.363 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T08:35:08.363 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T08:35:08.363 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T08:35:08.363 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T08:35:08.363 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T08:35:08.363 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T08:35:08.363 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T08:35:08.363 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T08:35:08.363 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T08:35:08.363 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T08:35:08.363 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T08:35:08.364 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T08:35:08.364 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T08:35:08.364 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T08:35:08.364 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T08:35:08.364 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T08:35:08.364 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T08:35:08.364 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T08:35:08.364 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T08:35:08.364 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T08:35:08.364 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T08:35:08.364 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T08:35:08.364 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T08:35:08.365 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T08:35:08.365 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T08:35:08.365 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T08:35:08.365 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T08:35:08.365 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T08:35:08.365 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T08:35:08.365 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T08:35:08.365 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T08:35:08.366 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T08:35:08.366 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T08:35:08.366 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T08:35:08.366 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T08:35:08.366 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T08:35:08.366 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T08:35:08.366 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T08:35:08.366 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T08:35:08.366 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T08:35:08.366 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T08:35:08.366 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T08:35:08.367 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T08:35:08.367 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T08:35:08.367 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T08:35:08.367 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T08:35:08.367 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T08:35:08.367 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T08:35:08.367 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T08:35:08.367 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T08:35:08.367 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T08:35:08.367 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T08:35:08.367 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T08:35:08.367 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T08:35:08.367 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T08:35:08.368 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T08:35:08.368 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T08:35:08.368 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T08:35:08.368 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T08:35:08.368 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T08:35:08.368 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T08:35:08.368 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T08:35:08.368 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T08:35:08.368 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T08:35:08.368 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T08:35:08.368 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T08:35:08.368 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T08:35:08.368 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T08:35:08.369 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T08:35:08.369 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T08:35:08.369 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T08:35:08.369 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T08:35:08.369 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T08:35:08.369 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T08:35:08.369 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T08:35:08.369 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T08:35:08.369 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T08:35:08.369 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T08:35:08.369 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T08:35:08.369 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T08:35:08.369 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T08:35:08.370 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T08:35:08.370 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T08:35:08.370 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T08:35:08.370 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T08:35:08.370 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T08:35:08.370 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T08:35:08.370 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T08:35:08.370 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T08:35:08.370 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T08:35:08.370 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T08:35:08.370 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T08:35:08.370 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T08:35:08.370 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T08:35:08.370 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T08:35:08.371 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T08:35:08.371 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T08:35:08.371 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T08:35:08.371 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T08:35:08.371 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T08:35:08.371 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T08:35:08.371 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T08:35:08.371 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T08:35:08.371 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T08:35:08.372 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T08:35:08.372 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T08:35:08.372 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T08:35:08.372 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T08:35:08.372 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T08:35:08.372 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T08:35:08.372 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T08:35:08.372 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T08:35:08.372 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T08:35:08.372 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T08:35:08.372 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T08:35:08.372 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T08:35:08.372 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T08:35:08.372 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T08:35:08.373 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T08:35:08.373 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T08:35:08.373 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T08:35:08.373 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T08:35:08.373 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T08:35:08.373 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T08:35:08.373 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T08:35:08.373 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T08:35:08.373 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T08:35:08.373 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T08:35:08.373 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T08:35:08.373 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T08:35:08.374 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T08:35:08.374 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T08:35:08.374 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T08:35:08.374 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T08:35:11.156 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T08:35:11.156 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:35:11.156 INFO:teuthology.orchestra.run.smithi164.stdout:Upgraded: 2024-09-19T08:35:11.156 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T08:35:11.156 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T08:35:11.157 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T08:35:11.157 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T08:35:11.157 INFO:teuthology.orchestra.run.smithi164.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:11.157 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:11.157 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T08:35:11.157 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T08:35:11.157 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-19T08:35:11.157 INFO:teuthology.orchestra.run.smithi164.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T08:35:11.157 INFO:teuthology.orchestra.run.smithi164.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T08:35:11.157 INFO:teuthology.orchestra.run.smithi164.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T08:35:11.157 INFO:teuthology.orchestra.run.smithi164.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T08:35:11.158 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:11.158 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:11.158 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:11.158 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:11.158 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:35:11.158 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:11.158 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:11.158 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:11.158 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:35:11.158 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:35:11.158 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:35:11.158 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:35:11.159 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:35:11.159 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:11.159 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:11.159 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:35:11.159 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:11.159 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:11.159 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:11.159 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:35:11.159 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:35:11.159 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:11.160 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:11.160 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:35:11.160 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T08:35:11.160 INFO:teuthology.orchestra.run.smithi164.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T08:35:11.160 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T08:35:11.160 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T08:35:11.160 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T08:35:11.160 INFO:teuthology.orchestra.run.smithi164.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T08:35:11.160 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T08:35:11.160 INFO:teuthology.orchestra.run.smithi164.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T08:35:11.161 INFO:teuthology.orchestra.run.smithi164.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T08:35:11.161 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T08:35:11.161 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T08:35:11.161 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:11.161 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:11.161 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:11.161 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T08:35:11.161 INFO:teuthology.orchestra.run.smithi164.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T08:35:11.161 INFO:teuthology.orchestra.run.smithi164.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T08:35:11.162 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T08:35:11.162 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T08:35:11.162 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:11.162 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:11.162 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T08:35:11.162 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:11.162 INFO:teuthology.orchestra.run.smithi164.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T08:35:11.163 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T08:35:11.163 INFO:teuthology.orchestra.run.smithi164.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T08:35:11.163 INFO:teuthology.orchestra.run.smithi164.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T08:35:11.163 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T08:35:11.163 INFO:teuthology.orchestra.run.smithi164.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T08:35:11.163 INFO:teuthology.orchestra.run.smithi164.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T08:35:11.163 INFO:teuthology.orchestra.run.smithi164.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T08:35:11.163 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T08:35:11.163 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T08:35:11.164 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T08:35:11.164 INFO:teuthology.orchestra.run.smithi164.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T08:35:11.164 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T08:35:11.164 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T08:35:11.164 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T08:35:11.164 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T08:35:11.164 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T08:35:11.164 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T08:35:11.164 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T08:35:11.164 INFO:teuthology.orchestra.run.smithi164.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T08:35:11.164 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T08:35:11.165 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T08:35:11.165 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T08:35:11.165 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:11.165 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:11.165 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:11.165 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T08:35:11.165 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T08:35:11.165 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T08:35:11.165 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T08:35:11.165 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T08:35:11.166 INFO:teuthology.orchestra.run.smithi164.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T08:35:11.166 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T08:35:11.166 INFO:teuthology.orchestra.run.smithi164.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T08:35:11.166 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T08:35:11.166 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T08:35:11.166 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T08:35:11.166 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T08:35:11.166 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T08:35:11.166 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T08:35:11.166 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T08:35:11.166 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T08:35:11.167 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T08:35:11.167 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T08:35:11.167 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T08:35:11.167 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T08:35:11.167 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T08:35:11.167 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T08:35:11.167 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T08:35:11.167 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T08:35:11.167 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T08:35:11.167 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T08:35:11.167 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T08:35:11.167 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T08:35:11.168 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T08:35:11.168 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T08:35:11.168 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T08:35:11.168 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T08:35:11.168 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T08:35:11.168 INFO:teuthology.orchestra.run.smithi164.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T08:35:11.168 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T08:35:11.169 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T08:35:11.169 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T08:35:11.169 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T08:35:11.169 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T08:35:11.169 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T08:35:11.169 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:11.169 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:11.169 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T08:35:11.169 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T08:35:11.169 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T08:35:11.169 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:11.170 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T08:35:11.170 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T08:35:11.170 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T08:35:11.170 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T08:35:11.170 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T08:35:11.170 INFO:teuthology.orchestra.run.smithi164.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T08:35:11.170 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T08:35:11.170 INFO:teuthology.orchestra.run.smithi164.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T08:35:11.170 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T08:35:11.170 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T08:35:11.170 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T08:35:11.171 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T08:35:11.171 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T08:35:11.171 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T08:35:11.171 INFO:teuthology.orchestra.run.smithi164.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T08:35:11.171 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:11.171 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:11.171 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:11.171 INFO:teuthology.orchestra.run.smithi164.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T08:35:11.171 INFO:teuthology.orchestra.run.smithi164.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T08:35:11.171 INFO:teuthology.orchestra.run.smithi164.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T08:35:11.171 INFO:teuthology.orchestra.run.smithi164.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T08:35:11.171 INFO:teuthology.orchestra.run.smithi164.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T08:35:11.172 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T08:35:11.172 INFO:teuthology.orchestra.run.smithi164.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T08:35:11.172 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:35:11.172 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-19T08:35:11.459 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T08:35:11.459 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T08:35:11.459 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T08:35:11.459 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T08:35:11.459 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T08:35:11.459 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T08:35:11.460 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T08:35:11.460 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T08:35:11.460 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T08:35:11.460 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T08:35:11.460 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T08:35:11.460 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T08:35:11.460 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T08:35:11.460 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T08:35:11.460 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T08:35:11.460 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T08:35:11.460 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T08:35:11.460 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T08:35:11.461 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T08:35:11.461 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T08:35:11.461 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T08:35:11.461 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T08:35:11.461 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T08:35:11.461 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T08:35:11.461 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T08:35:11.461 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T08:35:11.461 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T08:35:11.461 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T08:35:11.461 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T08:35:11.473 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T08:35:11.473 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T08:35:11.473 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T08:35:11.473 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T08:35:11.473 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T08:35:11.473 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T08:35:11.474 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T08:35:11.474 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T08:35:11.474 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T08:35:11.474 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T08:35:11.474 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T08:35:11.474 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T08:35:11.474 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T08:35:11.474 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T08:35:11.474 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T08:35:11.474 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T08:35:11.474 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T08:35:11.474 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T08:35:11.475 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T08:35:11.475 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T08:35:11.475 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T08:35:11.475 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T08:35:11.475 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T08:35:11.475 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T08:35:11.475 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T08:35:11.475 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T08:35:11.475 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T08:35:11.475 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T08:35:11.475 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T08:35:11.475 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T08:35:11.476 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T08:35:11.476 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T08:35:11.476 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T08:35:11.476 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T08:35:11.476 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T08:35:11.476 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T08:35:11.476 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T08:35:11.476 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T08:35:11.476 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T08:35:11.476 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T08:35:11.476 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T08:35:11.476 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T08:35:11.476 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T08:35:11.477 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T08:35:11.477 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T08:35:11.477 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T08:35:11.477 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T08:35:11.477 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T08:35:11.477 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T08:35:11.477 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T08:35:11.477 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T08:35:11.477 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T08:35:11.477 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T08:35:11.477 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T08:35:11.477 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T08:35:11.477 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T08:35:11.478 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T08:35:11.478 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T08:35:11.478 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T08:35:11.478 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T08:35:11.478 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T08:35:11.478 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T08:35:11.478 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T08:35:11.478 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T08:35:11.478 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T08:35:11.478 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T08:35:11.478 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T08:35:11.478 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T08:35:11.478 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T08:35:11.479 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T08:35:11.479 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T08:35:11.479 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T08:35:11.479 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T08:35:11.479 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T08:35:11.479 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T08:35:11.479 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T08:35:11.479 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T08:35:11.479 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T08:35:11.479 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T08:35:11.479 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T08:35:11.479 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T08:35:11.479 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T08:35:11.480 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T08:35:11.480 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T08:35:11.480 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T08:35:11.480 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T08:35:11.480 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T08:35:11.480 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T08:35:11.480 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T08:35:11.480 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T08:35:11.480 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T08:35:11.480 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T08:35:11.480 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T08:35:11.480 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T08:35:11.481 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T08:35:11.481 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T08:35:11.481 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T08:35:11.481 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T08:35:11.481 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T08:35:11.481 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T08:35:11.481 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T08:35:11.481 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T08:35:11.481 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T08:35:11.481 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T08:35:11.481 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T08:35:11.481 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T08:35:11.481 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T08:35:11.482 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T08:35:11.482 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T08:35:11.482 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T08:35:11.482 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T08:35:11.482 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T08:35:11.482 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T08:35:11.482 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T08:35:11.482 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T08:35:11.482 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T08:35:11.482 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T08:35:11.482 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T08:35:11.482 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T08:35:11.483 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T08:35:11.483 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T08:35:11.483 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T08:35:11.483 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T08:35:11.483 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T08:35:11.483 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T08:35:11.483 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T08:35:11.483 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T08:35:11.483 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T08:35:11.483 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T08:35:11.483 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T08:35:11.483 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T08:35:11.483 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T08:35:11.484 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T08:35:11.484 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T08:35:11.484 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T08:35:11.569 DEBUG:teuthology.parallel:result is None 2024-09-19T08:35:12.866 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T08:35:12.907 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T08:35:12.949 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T08:35:12.980 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T08:35:13.008 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T08:35:13.198 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T08:35:13.246 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T08:35:13.268 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T08:35:13.298 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T08:35:13.344 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T08:35:13.878 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T08:35:13.917 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T08:35:13.938 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T08:35:14.339 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T08:35:14.339 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-19T08:35:14.339 INFO:teuthology.orchestra.run.smithi173.stdout:Upgraded: 2024-09-19T08:35:14.339 INFO:teuthology.orchestra.run.smithi173.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T08:35:14.339 INFO:teuthology.orchestra.run.smithi173.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T08:35:14.340 INFO:teuthology.orchestra.run.smithi173.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T08:35:14.340 INFO:teuthology.orchestra.run.smithi173.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T08:35:14.340 INFO:teuthology.orchestra.run.smithi173.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:14.340 INFO:teuthology.orchestra.run.smithi173.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:14.340 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T08:35:14.340 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T08:35:14.340 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-09-19T08:35:14.340 INFO:teuthology.orchestra.run.smithi173.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T08:35:14.340 INFO:teuthology.orchestra.run.smithi173.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T08:35:14.340 INFO:teuthology.orchestra.run.smithi173.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T08:35:14.340 INFO:teuthology.orchestra.run.smithi173.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T08:35:14.340 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:14.341 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:14.341 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:14.341 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:14.341 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:35:14.341 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:14.341 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:14.341 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:14.341 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:35:14.341 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:35:14.341 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:35:14.341 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:35:14.341 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:35:14.342 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:14.342 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:14.342 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:35:14.342 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:14.342 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:14.342 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:14.342 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:35:14.342 INFO:teuthology.orchestra.run.smithi173.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:35:14.342 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:14.342 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:14.342 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:35:14.342 INFO:teuthology.orchestra.run.smithi173.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T08:35:14.343 INFO:teuthology.orchestra.run.smithi173.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T08:35:14.343 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T08:35:14.343 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T08:35:14.343 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T08:35:14.343 INFO:teuthology.orchestra.run.smithi173.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T08:35:14.343 INFO:teuthology.orchestra.run.smithi173.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T08:35:14.343 INFO:teuthology.orchestra.run.smithi173.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T08:35:14.343 INFO:teuthology.orchestra.run.smithi173.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T08:35:14.343 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T08:35:14.343 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T08:35:14.343 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:14.343 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:14.343 INFO:teuthology.orchestra.run.smithi173.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:14.344 INFO:teuthology.orchestra.run.smithi173.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T08:35:14.344 INFO:teuthology.orchestra.run.smithi173.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T08:35:14.344 INFO:teuthology.orchestra.run.smithi173.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T08:35:14.344 INFO:teuthology.orchestra.run.smithi173.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T08:35:14.344 INFO:teuthology.orchestra.run.smithi173.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T08:35:14.344 INFO:teuthology.orchestra.run.smithi173.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:14.344 INFO:teuthology.orchestra.run.smithi173.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:14.344 INFO:teuthology.orchestra.run.smithi173.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T08:35:14.344 INFO:teuthology.orchestra.run.smithi173.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:14.344 INFO:teuthology.orchestra.run.smithi173.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T08:35:14.344 INFO:teuthology.orchestra.run.smithi173.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T08:35:14.345 INFO:teuthology.orchestra.run.smithi173.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T08:35:14.345 INFO:teuthology.orchestra.run.smithi173.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T08:35:14.345 INFO:teuthology.orchestra.run.smithi173.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T08:35:14.345 INFO:teuthology.orchestra.run.smithi173.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T08:35:14.345 INFO:teuthology.orchestra.run.smithi173.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T08:35:14.345 INFO:teuthology.orchestra.run.smithi173.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T08:35:14.345 INFO:teuthology.orchestra.run.smithi173.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T08:35:14.345 INFO:teuthology.orchestra.run.smithi173.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T08:35:14.345 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T08:35:14.345 INFO:teuthology.orchestra.run.smithi173.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T08:35:14.345 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T08:35:14.345 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T08:35:14.345 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T08:35:14.346 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T08:35:14.346 INFO:teuthology.orchestra.run.smithi173.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T08:35:14.346 INFO:teuthology.orchestra.run.smithi173.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T08:35:14.346 INFO:teuthology.orchestra.run.smithi173.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T08:35:14.346 INFO:teuthology.orchestra.run.smithi173.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T08:35:14.346 INFO:teuthology.orchestra.run.smithi173.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T08:35:14.346 INFO:teuthology.orchestra.run.smithi173.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T08:35:14.346 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T08:35:14.346 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:14.346 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:14.346 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:14.346 INFO:teuthology.orchestra.run.smithi173.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T08:35:14.346 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T08:35:14.347 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T08:35:14.347 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T08:35:14.347 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T08:35:14.347 INFO:teuthology.orchestra.run.smithi173.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T08:35:14.347 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T08:35:14.347 INFO:teuthology.orchestra.run.smithi173.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T08:35:14.347 INFO:teuthology.orchestra.run.smithi173.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T08:35:14.347 INFO:teuthology.orchestra.run.smithi173.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T08:35:14.347 INFO:teuthology.orchestra.run.smithi173.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T08:35:14.347 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T08:35:14.347 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T08:35:14.347 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T08:35:14.347 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T08:35:14.348 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T08:35:14.348 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T08:35:14.348 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T08:35:14.348 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T08:35:14.348 INFO:teuthology.orchestra.run.smithi173.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T08:35:14.348 INFO:teuthology.orchestra.run.smithi173.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T08:35:14.348 INFO:teuthology.orchestra.run.smithi173.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T08:35:14.348 INFO:teuthology.orchestra.run.smithi173.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T08:35:14.348 INFO:teuthology.orchestra.run.smithi173.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T08:35:14.348 INFO:teuthology.orchestra.run.smithi173.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T08:35:14.349 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T08:35:14.349 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T08:35:14.349 INFO:teuthology.orchestra.run.smithi173.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T08:35:14.349 INFO:teuthology.orchestra.run.smithi173.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T08:35:14.349 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T08:35:14.349 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T08:35:14.349 INFO:teuthology.orchestra.run.smithi173.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T08:35:14.349 INFO:teuthology.orchestra.run.smithi173.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T08:35:14.349 INFO:teuthology.orchestra.run.smithi173.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T08:35:14.349 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T08:35:14.350 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T08:35:14.350 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T08:35:14.350 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T08:35:14.350 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T08:35:14.350 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T08:35:14.350 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:14.350 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:14.350 INFO:teuthology.orchestra.run.smithi173.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T08:35:14.352 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T08:35:14.352 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T08:35:14.353 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:14.353 INFO:teuthology.orchestra.run.smithi173.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T08:35:14.353 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T08:35:14.353 INFO:teuthology.orchestra.run.smithi173.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T08:35:14.353 INFO:teuthology.orchestra.run.smithi173.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T08:35:14.353 INFO:teuthology.orchestra.run.smithi173.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T08:35:14.353 INFO:teuthology.orchestra.run.smithi173.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T08:35:14.354 INFO:teuthology.orchestra.run.smithi173.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T08:35:14.354 INFO:teuthology.orchestra.run.smithi173.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T08:35:14.354 INFO:teuthology.orchestra.run.smithi173.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T08:35:14.354 INFO:teuthology.orchestra.run.smithi173.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T08:35:14.354 INFO:teuthology.orchestra.run.smithi173.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T08:35:14.354 INFO:teuthology.orchestra.run.smithi173.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T08:35:14.354 INFO:teuthology.orchestra.run.smithi173.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T08:35:14.355 INFO:teuthology.orchestra.run.smithi173.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T08:35:14.355 INFO:teuthology.orchestra.run.smithi173.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T08:35:14.355 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:14.355 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:14.355 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:14.355 INFO:teuthology.orchestra.run.smithi173.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T08:35:14.355 INFO:teuthology.orchestra.run.smithi173.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T08:35:14.356 INFO:teuthology.orchestra.run.smithi173.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T08:35:14.356 INFO:teuthology.orchestra.run.smithi173.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T08:35:14.356 INFO:teuthology.orchestra.run.smithi173.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T08:35:14.356 INFO:teuthology.orchestra.run.smithi173.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T08:35:14.357 INFO:teuthology.orchestra.run.smithi173.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T08:35:14.357 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-19T08:35:14.357 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-19T08:35:14.745 DEBUG:teuthology.parallel:result is None 2024-09-19T08:35:14.784 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T08:35:14.864 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T08:35:14.904 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T08:35:14.986 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T08:35:15.025 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T08:35:15.051 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T08:35:15.077 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T08:35:15.077 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T08:35:15.169 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T08:35:15.169 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T08:35:15.253 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T08:35:15.278 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T08:35:15.284 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T08:35:15.284 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T08:35:15.472 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T08:35:15.472 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T08:35:15.572 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T08:35:15.636 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T08:35:21.216 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T08:35:21.217 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T08:35:21.217 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T08:35:21.217 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T08:35:21.217 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T08:35:21.217 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T08:35:21.217 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T08:35:21.217 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T08:35:21.217 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T08:35:21.217 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T08:35:21.218 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T08:35:21.218 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T08:35:21.218 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T08:35:21.218 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T08:35:21.218 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T08:35:21.218 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T08:35:21.218 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T08:35:21.218 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T08:35:21.218 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T08:35:21.218 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T08:35:21.218 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T08:35:21.219 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T08:35:21.219 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T08:35:21.219 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T08:35:21.219 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T08:35:21.219 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T08:35:21.219 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T08:35:21.219 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T08:35:21.219 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T08:35:21.219 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T08:35:21.219 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T08:35:21.219 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T08:35:21.220 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T08:35:21.220 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T08:35:21.220 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T08:35:21.220 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T08:35:21.220 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T08:35:21.220 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T08:35:21.220 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T08:35:21.220 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T08:35:21.220 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T08:35:21.220 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T08:35:21.220 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T08:35:21.220 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T08:35:21.221 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T08:35:21.221 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T08:35:21.221 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T08:35:21.221 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T08:35:21.221 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T08:35:21.221 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T08:35:21.221 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T08:35:21.221 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T08:35:21.221 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T08:35:21.221 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T08:35:21.221 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T08:35:21.221 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T08:35:21.222 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T08:35:21.222 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T08:35:21.222 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T08:35:21.222 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T08:35:21.222 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T08:35:21.222 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T08:35:21.222 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T08:35:21.222 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T08:35:21.222 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T08:35:21.222 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T08:35:21.222 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T08:35:21.223 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T08:35:21.223 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T08:35:21.223 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T08:35:21.223 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T08:35:21.223 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T08:35:21.223 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T08:35:21.223 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T08:35:21.223 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T08:35:21.223 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T08:35:21.223 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T08:35:21.223 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T08:35:21.223 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T08:35:21.223 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T08:35:21.224 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T08:35:21.224 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T08:35:21.224 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T08:35:21.224 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T08:35:21.224 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T08:35:21.224 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T08:35:21.224 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T08:35:21.231 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T08:35:21.232 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T08:35:21.232 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T08:35:21.232 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T08:35:21.232 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T08:35:21.232 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T08:35:21.232 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T08:35:21.232 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T08:35:21.233 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T08:35:21.233 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T08:35:21.233 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T08:35:21.233 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T08:35:21.233 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T08:35:21.233 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T08:35:21.233 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T08:35:21.233 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T08:35:21.233 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T08:35:21.233 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T08:35:21.233 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T08:35:21.234 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T08:35:21.234 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T08:35:21.234 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T08:35:21.234 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T08:35:21.234 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T08:35:21.234 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T08:35:21.234 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T08:35:21.236 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T08:35:21.236 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T08:35:21.236 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T08:35:21.236 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T08:35:21.237 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T08:35:21.237 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T08:35:21.237 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T08:35:21.237 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T08:35:21.237 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T08:35:21.237 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T08:35:21.237 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T08:35:21.237 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T08:35:21.237 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T08:35:21.238 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T08:35:21.238 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T08:35:21.238 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T08:35:21.238 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T08:35:21.238 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T08:35:21.238 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T08:35:21.238 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T08:35:21.238 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T08:35:21.238 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T08:35:21.238 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T08:35:21.238 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T08:35:21.238 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T08:35:21.238 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T08:35:21.239 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T08:35:21.239 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T08:35:21.239 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T08:35:21.239 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T08:35:21.239 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T08:35:21.239 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T08:35:21.239 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T08:35:21.239 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T08:35:21.239 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T08:35:21.239 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T08:35:21.239 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T08:35:21.239 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T08:35:21.239 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T08:35:21.240 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T08:35:21.240 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T08:35:21.240 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T08:35:21.240 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T08:35:21.240 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T08:35:21.240 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T08:35:21.240 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T08:35:21.240 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T08:35:21.240 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T08:35:21.240 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T08:35:21.240 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T08:35:21.240 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T08:35:23.868 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T08:35:23.868 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-19T08:35:23.868 INFO:teuthology.orchestra.run.smithi170.stdout:Upgraded: 2024-09-19T08:35:23.868 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T08:35:23.868 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T08:35:23.868 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T08:35:23.868 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T08:35:23.868 INFO:teuthology.orchestra.run.smithi170.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:23.870 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:23.872 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T08:35:23.872 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T08:35:23.872 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2024-09-19T08:35:23.872 INFO:teuthology.orchestra.run.smithi170.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T08:35:23.872 INFO:teuthology.orchestra.run.smithi170.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T08:35:23.872 INFO:teuthology.orchestra.run.smithi170.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T08:35:23.872 INFO:teuthology.orchestra.run.smithi170.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T08:35:23.872 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:23.872 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:23.873 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:23.873 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:23.873 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:35:23.873 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:23.873 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:23.873 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:23.873 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:35:23.873 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:35:23.873 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:35:23.873 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:35:23.874 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:35:23.874 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:23.874 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:23.874 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:35:23.874 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:23.874 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:23.874 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:23.874 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:35:23.874 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:35:23.874 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:23.874 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:23.875 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T08:35:23.875 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T08:35:23.875 INFO:teuthology.orchestra.run.smithi170.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T08:35:23.875 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T08:35:23.875 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T08:35:23.875 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T08:35:23.876 INFO:teuthology.orchestra.run.smithi170.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T08:35:23.877 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T08:35:23.877 INFO:teuthology.orchestra.run.smithi170.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T08:35:23.877 INFO:teuthology.orchestra.run.smithi170.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T08:35:23.877 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T08:35:23.877 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T08:35:23.877 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:23.877 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:23.877 INFO:teuthology.orchestra.run.smithi170.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:23.877 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T08:35:23.877 INFO:teuthology.orchestra.run.smithi170.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T08:35:23.877 INFO:teuthology.orchestra.run.smithi170.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T08:35:23.877 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T08:35:23.878 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T08:35:23.878 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:23.878 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:23.878 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T08:35:23.878 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:23.878 INFO:teuthology.orchestra.run.smithi170.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T08:35:23.878 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T08:35:23.878 INFO:teuthology.orchestra.run.smithi170.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T08:35:23.878 INFO:teuthology.orchestra.run.smithi170.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T08:35:23.878 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T08:35:23.879 INFO:teuthology.orchestra.run.smithi170.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T08:35:23.879 INFO:teuthology.orchestra.run.smithi170.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T08:35:23.879 INFO:teuthology.orchestra.run.smithi170.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T08:35:23.879 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T08:35:23.879 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T08:35:23.879 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T08:35:23.879 INFO:teuthology.orchestra.run.smithi170.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T08:35:23.879 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T08:35:23.879 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T08:35:23.879 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T08:35:23.879 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T08:35:23.879 INFO:teuthology.orchestra.run.smithi170.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T08:35:23.879 INFO:teuthology.orchestra.run.smithi170.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T08:35:23.880 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T08:35:23.880 INFO:teuthology.orchestra.run.smithi170.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T08:35:23.880 INFO:teuthology.orchestra.run.smithi170.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T08:35:23.880 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T08:35:23.880 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T08:35:23.880 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:23.880 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:23.880 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:23.880 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T08:35:23.880 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T08:35:23.881 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T08:35:23.881 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T08:35:23.881 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T08:35:23.881 INFO:teuthology.orchestra.run.smithi170.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T08:35:23.881 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T08:35:23.881 INFO:teuthology.orchestra.run.smithi170.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T08:35:23.881 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T08:35:23.881 INFO:teuthology.orchestra.run.smithi170.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T08:35:23.881 INFO:teuthology.orchestra.run.smithi170.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T08:35:23.881 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T08:35:23.881 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T08:35:23.881 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T08:35:23.881 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T08:35:23.882 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T08:35:23.882 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T08:35:23.882 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T08:35:23.882 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T08:35:23.882 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T08:35:23.882 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T08:35:23.882 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T08:35:23.882 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T08:35:23.882 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T08:35:23.882 INFO:teuthology.orchestra.run.smithi170.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T08:35:23.882 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T08:35:23.883 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T08:35:23.883 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T08:35:23.883 INFO:teuthology.orchestra.run.smithi170.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T08:35:23.883 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T08:35:23.883 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T08:35:23.883 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T08:35:23.883 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T08:35:23.883 INFO:teuthology.orchestra.run.smithi170.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T08:35:23.883 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T08:35:23.883 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T08:35:23.883 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T08:35:23.883 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T08:35:23.883 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T08:35:23.884 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T08:35:23.884 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:23.884 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:23.884 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T08:35:23.884 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T08:35:23.884 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T08:35:23.884 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:23.884 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T08:35:23.884 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T08:35:23.884 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T08:35:23.884 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T08:35:23.884 INFO:teuthology.orchestra.run.smithi170.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T08:35:23.885 INFO:teuthology.orchestra.run.smithi170.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T08:35:23.885 INFO:teuthology.orchestra.run.smithi170.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T08:35:23.885 INFO:teuthology.orchestra.run.smithi170.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T08:35:23.885 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T08:35:23.885 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T08:35:23.885 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T08:35:23.886 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T08:35:23.886 INFO:teuthology.orchestra.run.smithi170.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T08:35:23.886 INFO:teuthology.orchestra.run.smithi170.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T08:35:23.886 INFO:teuthology.orchestra.run.smithi170.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T08:35:23.886 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:23.886 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:23.886 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T08:35:23.887 INFO:teuthology.orchestra.run.smithi170.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T08:35:23.887 INFO:teuthology.orchestra.run.smithi170.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T08:35:23.887 INFO:teuthology.orchestra.run.smithi170.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T08:35:23.887 INFO:teuthology.orchestra.run.smithi170.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T08:35:23.887 INFO:teuthology.orchestra.run.smithi170.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T08:35:23.888 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T08:35:23.888 INFO:teuthology.orchestra.run.smithi170.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T08:35:23.888 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-19T08:35:23.889 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-09-19T08:35:24.304 DEBUG:teuthology.parallel:result is None 2024-09-19T08:35:24.308 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:35:24.457 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T08:35:24.484 INFO:teuthology.orchestra.run.smithi164.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T08:35:24.484 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T08:35:24.485 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T08:35:24.487 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:35:24.635 DEBUG:teuthology.orchestra.run.smithi170:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T08:35:24.658 INFO:teuthology.orchestra.run.smithi170.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T08:35:24.658 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T08:35:24.658 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T08:35:24.661 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:35:24.799 DEBUG:teuthology.orchestra.run.smithi173:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T08:35:24.830 INFO:teuthology.orchestra.run.smithi173.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T08:35:24.830 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T08:35:24.831 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T08:35:24.832 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-19T08:35:24.833 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-19T08:35:24.833 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T08:35:24.866 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-19T08:35:24.866 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T08:35:24.895 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-19T08:35:24.895 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T08:35:24.928 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-19T08:35:24.929 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-19T08:35:24.929 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T08:35:24.958 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T08:35:25.028 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-19T08:35:25.028 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T08:35:25.054 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T08:35:25.120 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-19T08:35:25.120 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T08:35:25.150 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T08:35:25.218 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-19T08:35:25.218 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-19T08:35:25.218 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T08:35:25.248 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T08:35:25.315 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-19T08:35:25.316 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T08:35:25.341 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T08:35:25.407 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-19T08:35:25.407 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T08:35:25.437 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T08:35:25.506 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-19T08:35:25.506 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-19T08:35:25.506 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T08:35:25.532 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T08:35:25.597 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-19T08:35:25.597 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T08:35:25.624 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T08:35:25.690 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-19T08:35:25.691 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T08:35:25.720 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T08:35:25.788 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-19T08:35:25.884 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-19T08:35:25.884 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:35:25.888 INFO:tasks.cephadm:Cluster fsid is 1e4b0d74-7662-11ef-bceb-c7b262605968 2024-09-19T08:35:25.888 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-19T08:35:25.888 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.164', 'mon.b': '172.21.15.170', 'mon.c': '172.21.15.173'} 2024-09-19T08:35:25.888 INFO:tasks.cephadm:First mon is mon.a on smithi164 2024-09-19T08:35:25.889 INFO:tasks.cephadm:First mgr is x 2024-09-19T08:35:25.889 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-19T08:35:25.889 DEBUG:teuthology.orchestra.run.smithi164:> sudo hostname $(hostname -s) 2024-09-19T08:35:25.916 DEBUG:teuthology.orchestra.run.smithi170:> sudo hostname $(hostname -s) 2024-09-19T08:35:25.943 DEBUG:teuthology.orchestra.run.smithi173:> sudo hostname $(hostname -s) 2024-09-19T08:35:25.972 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-19T08:35:25.972 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:35:26.110 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56106/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-361-g2f599a46', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-361.g2f599a46'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-18 17:52:24.287651', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-19T08:35:26.265 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref squid, sha1 2f599a46ad10e37c929e5c43da2f6907ddcd1961 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:35:26.267 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T08:35:26.267 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T08:35:26.267 DEBUG:teuthology.orchestra.run.smithi164:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T08:35:26.668 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 08:35 /home/ubuntu/cephtest/cephadm 2024-09-19T08:35:26.669 DEBUG:teuthology.orchestra.run.smithi170:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T08:35:27.061 INFO:teuthology.orchestra.run.smithi170.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 08:35 /home/ubuntu/cephtest/cephadm 2024-09-19T08:35:27.062 DEBUG:teuthology.orchestra.run.smithi173:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T08:35:27.440 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 08:35 /home/ubuntu/cephtest/cephadm 2024-09-19T08:35:27.441 DEBUG:teuthology.orchestra.run.smithi164:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T08:35:27.460 DEBUG:teuthology.orchestra.run.smithi170:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T08:35:27.480 DEBUG:teuthology.orchestra.run.smithi173:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T08:35:27.513 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 on all hosts... 2024-09-19T08:35:27.513 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T08:35:27.516 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T08:35:27.523 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T08:35:29.030 INFO:teuthology.orchestra.run.smithi173.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T08:35:29.060 INFO:teuthology.orchestra.run.smithi164.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T08:35:29.091 INFO:teuthology.orchestra.run.smithi170.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T08:35:58.411 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-19T08:35:58.411 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:35:58.411 INFO:teuthology.orchestra.run.smithi170.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T08:35:58.411 INFO:teuthology.orchestra.run.smithi170.stdout: "repo_digests": [ 2024-09-19T08:35:58.411 INFO:teuthology.orchestra.run.smithi170.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T08:35:58.411 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-19T08:35:58.412 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-19T08:35:58.929 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-19T08:35:58.930 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:35:58.930 INFO:teuthology.orchestra.run.smithi173.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T08:35:58.930 INFO:teuthology.orchestra.run.smithi173.stdout: "repo_digests": [ 2024-09-19T08:35:58.930 INFO:teuthology.orchestra.run.smithi173.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T08:35:58.930 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-19T08:35:58.930 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-19T08:36:00.339 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:36:00.339 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:36:00.339 INFO:teuthology.orchestra.run.smithi164.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T08:36:00.340 INFO:teuthology.orchestra.run.smithi164.stdout: "repo_digests": [ 2024-09-19T08:36:00.340 INFO:teuthology.orchestra.run.smithi164.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T08:36:00.340 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-19T08:36:00.340 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:36:00.364 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /etc/ceph 2024-09-19T08:36:00.418 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /etc/ceph 2024-09-19T08:36:00.452 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph 2024-09-19T08:36:00.485 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 777 /etc/ceph 2024-09-19T08:36:00.510 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 777 /etc/ceph 2024-09-19T08:36:00.540 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 777 /etc/ceph 2024-09-19T08:36:00.569 INFO:tasks.cephadm:Writing seed config... 2024-09-19T08:36:00.570 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-19T08:36:00.570 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-19T08:36:00.570 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-19T08:36:00.570 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-19T08:36:00.571 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-19T08:36:00.571 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-19T08:36:00.571 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-19T08:36:00.571 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-19T08:36:00.571 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-19T08:36:00.571 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-19T08:36:00.571 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-19T08:36:00.571 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-19T08:36:00.571 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-19T08:36:00.571 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-19T08:36:00.571 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-19T08:36:00.571 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-19T08:36:00.571 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-19T08:36:00.571 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-19T08:36:00.572 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-19T08:36:00.572 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-19T08:36:00.572 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-19T08:36:00.572 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-19T08:36:00.572 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-19T08:36:00.572 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-19T08:36:00.572 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-19T08:36:00.572 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-19T08:36:00.572 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-19T08:36:00.572 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-19T08:36:00.572 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-19T08:36:00.572 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-19T08:36:00.572 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-19T08:36:00.573 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-19T08:36:00.573 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-19T08:36:00.573 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-19T08:36:00.573 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-19T08:36:00.573 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-19T08:36:00.573 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-19T08:36:00.573 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-19T08:36:00.573 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-19T08:36:00.573 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-19T08:36:00.573 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-19T08:36:00.573 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-19T08:36:00.573 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-19T08:36:00.573 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-19T08:36:00.573 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-19T08:36:00.573 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-19T08:36:00.574 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-19T08:36:00.574 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-19T08:36:00.574 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-19T08:36:00.575 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-19T08:36:00.590 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 1e4b0d74-7662-11ef-bceb-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-19T08:36:00.590 DEBUG:teuthology.orchestra.run.smithi164:mon.a> sudo journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mon.a.service 2024-09-19T08:36:00.633 DEBUG:teuthology.orchestra.run.smithi164:mgr.x> sudo journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mgr.x.service 2024-09-19T08:36:00.676 INFO:tasks.cephadm:Bootstrapping... 2024-09-19T08:36:00.676 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 -v bootstrap --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.164 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-19T08:36:00.877 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-19T08:36:00.877 INFO:teuthology.orchestra.run.smithi164.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961', '-v', 'bootstrap', '--fsid', '1e4b0d74-7662-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.164', '--skip-admin-label'] 2024-09-19T08:36:00.878 INFO:teuthology.orchestra.run.smithi164.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-19T08:36:00.878 INFO:teuthology.orchestra.run.smithi164.stdout:Verifying podman|docker is present... 2024-09-19T08:36:00.900 INFO:teuthology.orchestra.run.smithi164.stdout:/bin/podman: stdout 5.2.2 2024-09-19T08:36:00.900 INFO:teuthology.orchestra.run.smithi164.stdout:Verifying lvm2 is present... 2024-09-19T08:36:00.900 INFO:teuthology.orchestra.run.smithi164.stdout:Verifying time synchronization is in place... 2024-09-19T08:36:00.908 INFO:teuthology.orchestra.run.smithi164.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T08:36:00.908 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T08:36:00.914 INFO:teuthology.orchestra.run.smithi164.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T08:36:00.914 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stdout inactive 2024-09-19T08:36:00.921 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stdout enabled 2024-09-19T08:36:00.927 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stdout active 2024-09-19T08:36:00.927 INFO:teuthology.orchestra.run.smithi164.stdout:Unit chronyd.service is enabled and running 2024-09-19T08:36:00.927 INFO:teuthology.orchestra.run.smithi164.stdout:Repeating the final host check... 2024-09-19T08:36:00.949 INFO:teuthology.orchestra.run.smithi164.stdout:/bin/podman: stdout 5.2.2 2024-09-19T08:36:00.949 INFO:teuthology.orchestra.run.smithi164.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-19T08:36:00.950 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl is present 2024-09-19T08:36:00.950 INFO:teuthology.orchestra.run.smithi164.stdout:lvcreate is present 2024-09-19T08:36:00.959 INFO:teuthology.orchestra.run.smithi164.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T08:36:00.959 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T08:36:00.962 INFO:teuthology.orchestra.run.smithi164.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T08:36:00.962 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stdout inactive 2024-09-19T08:36:00.969 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stdout enabled 2024-09-19T08:36:00.976 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stdout active 2024-09-19T08:36:00.976 INFO:teuthology.orchestra.run.smithi164.stdout:Unit chronyd.service is enabled and running 2024-09-19T08:36:00.976 INFO:teuthology.orchestra.run.smithi164.stdout:Host looks OK 2024-09-19T08:36:00.976 INFO:teuthology.orchestra.run.smithi164.stdout:Cluster fsid: 1e4b0d74-7662-11ef-bceb-c7b262605968 2024-09-19T08:36:00.977 INFO:teuthology.orchestra.run.smithi164.stdout:Acquiring lock 139654667285504 on /run/cephadm/1e4b0d74-7662-11ef-bceb-c7b262605968.lock 2024-09-19T08:36:00.977 INFO:teuthology.orchestra.run.smithi164.stdout:Lock 139654667285504 acquired on /run/cephadm/1e4b0d74-7662-11ef-bceb-c7b262605968.lock 2024-09-19T08:36:00.977 INFO:teuthology.orchestra.run.smithi164.stdout:Verifying IP 172.21.15.164 port 3300 ... 2024-09-19T08:36:00.977 INFO:teuthology.orchestra.run.smithi164.stdout:Verifying IP 172.21.15.164 port 6789 ... 2024-09-19T08:36:00.978 INFO:teuthology.orchestra.run.smithi164.stdout:Base mon IP(s) is [172.21.15.164:3300, 172.21.15.164:6789], mon addrv is [v2:172.21.15.164:3300,v1:172.21.15.164:6789] 2024-09-19T08:36:00.980 INFO:teuthology.orchestra.run.smithi164.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.164 metric 100 2024-09-19T08:36:00.980 INFO:teuthology.orchestra.run.smithi164.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.164 metric 100 2024-09-19T08:36:00.982 INFO:teuthology.orchestra.run.smithi164.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-19T08:36:00.983 INFO:teuthology.orchestra.run.smithi164.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-19T08:36:00.983 INFO:teuthology.orchestra.run.smithi164.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-19T08:36:00.985 INFO:teuthology.orchestra.run.smithi164.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-19T08:36:00.985 INFO:teuthology.orchestra.run.smithi164.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-19T08:36:00.985 INFO:teuthology.orchestra.run.smithi164.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T08:36:00.985 INFO:teuthology.orchestra.run.smithi164.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-19T08:36:00.985 INFO:teuthology.orchestra.run.smithi164.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:73f3/64 scope link noprefixroute 2024-09-19T08:36:00.985 INFO:teuthology.orchestra.run.smithi164.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T08:36:00.986 INFO:teuthology.orchestra.run.smithi164.stdout:Mon IP `172.21.15.164` is in CIDR network `172.21.0.0/20` 2024-09-19T08:36:00.986 INFO:teuthology.orchestra.run.smithi164.stdout:Mon IP `172.21.15.164` is in CIDR network `172.21.0.0/20` 2024-09-19T08:36:00.986 INFO:teuthology.orchestra.run.smithi164.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-19T08:36:00.986 INFO:teuthology.orchestra.run.smithi164.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-19T08:36:00.987 INFO:teuthology.orchestra.run.smithi164.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T08:36:01.458 INFO:teuthology.orchestra.run.smithi164.stdout:/bin/podman: stdout d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T08:36:01.458 INFO:teuthology.orchestra.run.smithi164.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T08:36:01.458 INFO:teuthology.orchestra.run.smithi164.stdout:/bin/podman: stderr Getting image source signatures 2024-09-19T08:36:01.459 INFO:teuthology.orchestra.run.smithi164.stdout:/bin/podman: stderr Copying blob sha256:c4221751d16d26d2a4b7031ebd71ecffffff13c8cf7401cf95ce80dab3b6d67a 2024-09-19T08:36:01.459 INFO:teuthology.orchestra.run.smithi164.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-19T08:36:01.459 INFO:teuthology.orchestra.run.smithi164.stdout:/bin/podman: stderr Copying config sha256:d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T08:36:01.459 INFO:teuthology.orchestra.run.smithi164.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-19T08:36:02.413 INFO:teuthology.orchestra.run.smithi164.stdout:ceph: stdout ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T08:36:02.413 INFO:teuthology.orchestra.run.smithi164.stdout:Ceph version: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T08:36:02.413 INFO:teuthology.orchestra.run.smithi164.stdout:Extracting ceph user uid/gid from container image... 2024-09-19T08:36:03.368 INFO:teuthology.orchestra.run.smithi164.stdout:stat: stdout 167 167 2024-09-19T08:36:03.368 INFO:teuthology.orchestra.run.smithi164.stdout:Creating initial keys... 2024-09-19T08:36:04.291 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph-authtool: stdout AQBz4utmBUjmKRAAQ+YrwrW0eNONVzgi03gWJw== 2024-09-19T08:36:05.204 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph-authtool: stdout AQB04utmehEwJRAA+lO9n9N9QLxUn+pQWnvH2g== 2024-09-19T08:36:06.277 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph-authtool: stdout AQB14utmVJ4SIBAApKj7PCdyXVGpqwwj0LfBwA== 2024-09-19T08:36:06.277 INFO:teuthology.orchestra.run.smithi164.stdout:Creating initial monmap... 2024-09-19T08:36:07.174 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T08:36:07.174 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-19T08:36:07.174 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 1e4b0d74-7662-11ef-bceb-c7b262605968 2024-09-19T08:36:07.174 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T08:36:07.174 INFO:teuthology.orchestra.run.smithi164.stdout:monmaptool for a [v2:172.21.15.164:3300,v1:172.21.15.164:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T08:36:07.174 INFO:teuthology.orchestra.run.smithi164.stdout:setting min_mon_release = quincy 2024-09-19T08:36:07.174 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/monmaptool: set fsid to 1e4b0d74-7662-11ef-bceb-c7b262605968 2024-09-19T08:36:07.174 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T08:36:07.174 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:36:07.174 INFO:teuthology.orchestra.run.smithi164.stdout:Creating mon... 2024-09-19T08:36:08.145 INFO:teuthology.orchestra.run.smithi164.stdout:create mon.a on 2024-09-19T08:36:08.352 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-19T08:36:08.498 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-19T08:36:08.680 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-1e4b0d74-7662-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-1e4b0d74-7662-11ef-bceb-c7b262605968.target. 2024-09-19T08:36:08.680 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-1e4b0d74-7662-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-1e4b0d74-7662-11ef-bceb-c7b262605968.target. 2024-09-19T08:36:08.908 INFO:teuthology.orchestra.run.smithi164.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mon.a 2024-09-19T08:36:08.908 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-19T08:36:09.073 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1e4b0d74-7662-11ef-bceb-c7b262605968.target.wants/ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@.service. 2024-09-19T08:36:09.607 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:09 smithi164 ceph-mon[30565]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T08:36:09.607 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:09 smithi164 bash[30546]: 850bea03f6a27ce67fa493eb80f9de7e36b4970750a01e4c45bfd00b7686e26f 2024-09-19T08:36:09.620 INFO:teuthology.orchestra.run.smithi164.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T08:36:09.620 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stdout disabled 2024-09-19T08:36:09.628 INFO:teuthology.orchestra.run.smithi164.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T08:36:09.628 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stdout inactive 2024-09-19T08:36:09.628 INFO:teuthology.orchestra.run.smithi164.stdout:firewalld.service is not enabled 2024-09-19T08:36:09.628 INFO:teuthology.orchestra.run.smithi164.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T08:36:09.628 INFO:teuthology.orchestra.run.smithi164.stdout:Waiting for mon to start... 2024-09-19T08:36:09.628 INFO:teuthology.orchestra.run.smithi164.stdout:Waiting for mon... 2024-09-19T08:36:09.869 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:09 smithi164 systemd[1]: Started Ceph mon.a for 1e4b0d74-7662-11ef-bceb-c7b262605968. 2024-09-19T08:36:10.671 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:10 smithi164 ceph-mon[30565]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T08:36:10.671 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:10 smithi164 ceph-mon[30565]: monmap epoch 1 2024-09-19T08:36:10.671 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:10 smithi164 ceph-mon[30565]: fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 2024-09-19T08:36:10.671 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:10 smithi164 ceph-mon[30565]: last_changed 2024-09-19T08:36:06.626290+0000 2024-09-19T08:36:10.671 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:10 smithi164 ceph-mon[30565]: created 2024-09-19T08:36:06.626290+0000 2024-09-19T08:36:10.671 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:10 smithi164 ceph-mon[30565]: min_mon_release 19 (squid) 2024-09-19T08:36:10.671 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:10 smithi164 ceph-mon[30565]: election_strategy: 1 2024-09-19T08:36:10.672 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:10 smithi164 ceph-mon[30565]: 0: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.a 2024-09-19T08:36:10.672 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:10 smithi164 ceph-mon[30565]: fsmap 2024-09-19T08:36:10.672 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:10 smithi164 ceph-mon[30565]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T08:36:10.672 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:10 smithi164 ceph-mon[30565]: mgrmap e1: no daemons active 2024-09-19T08:36:10.672 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:10 smithi164 ceph-mon[30565]: from='client.? 172.21.15.164:0/3668333480' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-19T08:36:10.679 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout cluster: 2024-09-19T08:36:10.680 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout id: 1e4b0d74-7662-11ef-bceb-c7b262605968 2024-09-19T08:36:10.680 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-19T08:36:10.680 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout 2024-09-19T08:36:10.680 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout services: 2024-09-19T08:36:10.680 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.542477s) 2024-09-19T08:36:10.680 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-19T08:36:10.680 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-19T08:36:10.680 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout 2024-09-19T08:36:10.680 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout data: 2024-09-19T08:36:10.680 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-19T08:36:10.680 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-19T08:36:10.680 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-19T08:36:10.681 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout pgs: 2024-09-19T08:36:10.681 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout 2024-09-19T08:36:10.681 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.037+0000 7fd3a74f7640 1 Processor -- start 2024-09-19T08:36:10.681 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.037+0000 7fd3a74f7640 1 -- start start 2024-09-19T08:36:10.681 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.037+0000 7fd3a74f7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3a007de70 0x7fd3a007c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:10.681 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.037+0000 7fd3a74f7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a007c8f0 con 0x7fd3a007de70 2024-09-19T08:36:10.681 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.038+0000 7fd3a5293640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3a007de70 0x7fd3a007c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:10.681 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.038+0000 7fd3a5293640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3a007de70 0x7fd3a007c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52972/0 (socket says 172.21.15.164:52972) 2024-09-19T08:36:10.681 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.038+0000 7fd3a5293640 1 -- 172.21.15.164:0/3228321881 learned_addr learned my addr 172.21.15.164:0/3228321881 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:10.681 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.038+0000 7fd3a5293640 1 -- 172.21.15.164:0/3228321881 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a007ca70 con 0x7fd3a007de70 2024-09-19T08:36:10.681 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.040+0000 7fd3a5293640 1 --2- 172.21.15.164:0/3228321881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3a007de70 0x7fd3a007c320 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd398009820 tx=0x7fd39802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2165c70e3c48599a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:10.681 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.040+0000 7fd39ffff640 1 -- 172.21.15.164:0/3228321881 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd398037070 con 0x7fd3a007de70 2024-09-19T08:36:10.681 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.041+0000 7fd39ffff640 1 -- 172.21.15.164:0/3228321881 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd398032440 con 0x7fd3a007de70 2024-09-19T08:36:10.682 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.041+0000 7fd39ffff640 1 -- 172.21.15.164:0/3228321881 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd398032740 con 0x7fd3a007de70 2024-09-19T08:36:10.682 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.041+0000 7fd3a74f7640 1 -- 172.21.15.164:0/3228321881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3a007de70 msgr2=0x7fd3a007c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:10.682 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.041+0000 7fd3a74f7640 1 --2- 172.21.15.164:0/3228321881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3a007de70 0x7fd3a007c320 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd398009820 tx=0x7fd39802a270 comp rx=0 tx=0).stop 2024-09-19T08:36:10.682 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.041+0000 7fd3a74f7640 1 -- 172.21.15.164:0/3228321881 shutdown_connections 2024-09-19T08:36:10.682 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.041+0000 7fd3a74f7640 1 --2- 172.21.15.164:0/3228321881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3a007de70 0x7fd3a007c320 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:10.682 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.041+0000 7fd3a74f7640 1 -- 172.21.15.164:0/3228321881 >> 172.21.15.164:0/3228321881 conn(0x7fd3a0102ef0 msgr2=0x7fd3a0105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:10.682 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.041+0000 7fd3a74f7640 1 -- 172.21.15.164:0/3228321881 shutdown_connections 2024-09-19T08:36:10.683 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.041+0000 7fd3a74f7640 1 -- 172.21.15.164:0/3228321881 wait complete. 2024-09-19T08:36:10.683 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.042+0000 7fd3a74f7640 1 Processor -- start 2024-09-19T08:36:10.683 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.042+0000 7fd3a74f7640 1 -- start start 2024-09-19T08:36:10.683 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.042+0000 7fd3a74f7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3a007de70 0x7fd3a019d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:10.683 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.042+0000 7fd3a74f7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a007d010 con 0x7fd3a007de70 2024-09-19T08:36:10.683 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.042+0000 7fd3a5293640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3a007de70 0x7fd3a019d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:10.684 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.042+0000 7fd3a5293640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3a007de70 0x7fd3a019d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52980/0 (socket says 172.21.15.164:52980) 2024-09-19T08:36:10.684 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.042+0000 7fd3a5293640 1 -- 172.21.15.164:0/3668333480 learned_addr learned my addr 172.21.15.164:0/3668333480 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:10.684 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.042+0000 7fd3a5293640 1 -- 172.21.15.164:0/3668333480 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a019d5a0 con 0x7fd3a007de70 2024-09-19T08:36:10.684 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.043+0000 7fd3a5293640 1 --2- 172.21.15.164:0/3668333480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3a007de70 0x7fd3a019d060 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd3980090b0 tx=0x7fd3980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:10.684 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.043+0000 7fd39e7fc640 1 -- 172.21.15.164:0/3668333480 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd398041070 con 0x7fd3a007de70 2024-09-19T08:36:10.684 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.043+0000 7fd39e7fc640 1 -- 172.21.15.164:0/3668333480 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd39803c7a0 con 0x7fd3a007de70 2024-09-19T08:36:10.684 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.043+0000 7fd3a74f7640 1 -- 172.21.15.164:0/3668333480 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3a019d830 con 0x7fd3a007de70 2024-09-19T08:36:10.684 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.043+0000 7fd39e7fc640 1 -- 172.21.15.164:0/3668333480 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd398037040 con 0x7fd3a007de70 2024-09-19T08:36:10.684 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.043+0000 7fd3a74f7640 1 -- 172.21.15.164:0/3668333480 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a01a0520 con 0x7fd3a007de70 2024-09-19T08:36:10.685 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.043+0000 7fd39e7fc640 1 -- 172.21.15.164:0/3668333480 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd398044070 con 0x7fd3a007de70 2024-09-19T08:36:10.685 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.043+0000 7fd39e7fc640 1 -- 172.21.15.164:0/3668333480 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd39803d3f0 con 0x7fd3a007de70 2024-09-19T08:36:10.685 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.044+0000 7fd3a74f7640 1 -- 172.21.15.164:0/3668333480 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3a007c3a0 con 0x7fd3a007de70 2024-09-19T08:36:10.685 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.046+0000 7fd39e7fc640 1 -- 172.21.15.164:0/3668333480 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fd398046450 con 0x7fd3a007de70 2024-09-19T08:36:10.685 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.105+0000 7fd3a74f7640 1 -- 172.21.15.164:0/3668333480 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fd3a002d940 con 0x7fd3a007de70 2024-09-19T08:36:10.685 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.105+0000 7fd39e7fc640 1 -- 172.21.15.164:0/3668333480 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fd398046630 con 0x7fd3a007de70 2024-09-19T08:36:10.685 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.106+0000 7fd3a74f7640 1 -- 172.21.15.164:0/3668333480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3a007de70 msgr2=0x7fd3a019d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:10.685 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.106+0000 7fd3a74f7640 1 --2- 172.21.15.164:0/3668333480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3a007de70 0x7fd3a019d060 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd3980090b0 tx=0x7fd3980046c0 comp rx=0 tx=0).stop 2024-09-19T08:36:10.685 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.106+0000 7fd3a74f7640 1 -- 172.21.15.164:0/3668333480 shutdown_connections 2024-09-19T08:36:10.685 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.106+0000 7fd3a74f7640 1 --2- 172.21.15.164:0/3668333480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3a007de70 0x7fd3a019d060 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:10.685 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.106+0000 7fd3a74f7640 1 -- 172.21.15.164:0/3668333480 >> 172.21.15.164:0/3668333480 conn(0x7fd3a0102ef0 msgr2=0x7fd3a0103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:10.685 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.106+0000 7fd3a74f7640 1 -- 172.21.15.164:0/3668333480 shutdown_connections 2024-09-19T08:36:10.686 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:10.106+0000 7fd3a74f7640 1 -- 172.21.15.164:0/3668333480 wait complete. 2024-09-19T08:36:10.686 INFO:teuthology.orchestra.run.smithi164.stdout:mon is available 2024-09-19T08:36:10.686 INFO:teuthology.orchestra.run.smithi164.stdout:Assimilating anything we can from ceph.conf... 2024-09-19T08:36:11.744 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout 2024-09-19T08:36:11.744 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T08:36:11.744 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout fsid = 1e4b0d74-7662-11ef-bceb-c7b262605968 2024-09-19T08:36:11.744 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.164:3300,v1:172.21.15.164:6789] 2024-09-19T08:36:11.744 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T08:36:11.745 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T08:36:11.745 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T08:36:11.745 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T08:36:11.745 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout 2024-09-19T08:36:11.745 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T08:36:11.745 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T08:36:11.745 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout 2024-09-19T08:36:11.745 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T08:36:11.745 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T08:36:11.745 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T08:36:11.745 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T08:36:11.745 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T08:36:11.746 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T08:36:11.746 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.085+0000 7fd3b7753640 1 Processor -- start 2024-09-19T08:36:11.746 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.085+0000 7fd3b7753640 1 -- start start 2024-09-19T08:36:11.746 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.085+0000 7fd3b7753640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3b0105ff0 0x7fd3b01063f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:11.746 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.085+0000 7fd3b7753640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b007be30 con 0x7fd3b0105ff0 2024-09-19T08:36:11.746 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.085+0000 7fd3b54ef640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3b0105ff0 0x7fd3b01063f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:11.746 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.085+0000 7fd3b54ef640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3b0105ff0 0x7fd3b01063f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52986/0 (socket says 172.21.15.164:52986) 2024-09-19T08:36:11.746 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.085+0000 7fd3b54ef640 1 -- 172.21.15.164:0/3565591924 learned_addr learned my addr 172.21.15.164:0/3565591924 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:11.746 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.086+0000 7fd3b54ef640 1 -- 172.21.15.164:0/3565591924 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b0106930 con 0x7fd3b0105ff0 2024-09-19T08:36:11.746 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.086+0000 7fd3b54ef640 1 --2- 172.21.15.164:0/3565591924 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3b0105ff0 0x7fd3b01063f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009820 tx=0x7fd3ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c1a9ad23cf4d72 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:11.746 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.086+0000 7fd3a7fff640 1 -- 172.21.15.164:0/3565591924 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3ac037070 con 0x7fd3b0105ff0 2024-09-19T08:36:11.746 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.086+0000 7fd3a7fff640 1 -- 172.21.15.164:0/3565591924 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3ac032440 con 0x7fd3b0105ff0 2024-09-19T08:36:11.747 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.086+0000 7fd3a7fff640 1 -- 172.21.15.164:0/3565591924 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3ac032740 con 0x7fd3b0105ff0 2024-09-19T08:36:11.747 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.087+0000 7fd3b7753640 1 -- 172.21.15.164:0/3565591924 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3b0105ff0 msgr2=0x7fd3b01063f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:11.747 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.087+0000 7fd3b7753640 1 --2- 172.21.15.164:0/3565591924 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3b0105ff0 0x7fd3b01063f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009820 tx=0x7fd3ac02a270 comp rx=0 tx=0).stop 2024-09-19T08:36:11.747 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.087+0000 7fd3b7753640 1 -- 172.21.15.164:0/3565591924 shutdown_connections 2024-09-19T08:36:11.747 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.087+0000 7fd3b7753640 1 --2- 172.21.15.164:0/3565591924 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3b0105ff0 0x7fd3b01063f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:11.747 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.087+0000 7fd3b7753640 1 -- 172.21.15.164:0/3565591924 >> 172.21.15.164:0/3565591924 conn(0x7fd3b0101da0 msgr2=0x7fd3b01041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:11.747 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.087+0000 7fd3b7753640 1 -- 172.21.15.164:0/3565591924 shutdown_connections 2024-09-19T08:36:11.747 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.087+0000 7fd3b7753640 1 -- 172.21.15.164:0/3565591924 wait complete. 2024-09-19T08:36:11.747 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.088+0000 7fd3b7753640 1 Processor -- start 2024-09-19T08:36:11.747 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.088+0000 7fd3b7753640 1 -- start start 2024-09-19T08:36:11.747 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.088+0000 7fd3b7753640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3b0105ff0 0x7fd3b019d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:11.747 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.088+0000 7fd3b7753640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b0106ed0 con 0x7fd3b0105ff0 2024-09-19T08:36:11.748 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.088+0000 7fd3b54ef640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3b0105ff0 0x7fd3b019d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:11.748 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.088+0000 7fd3b54ef640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3b0105ff0 0x7fd3b019d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52992/0 (socket says 172.21.15.164:52992) 2024-09-19T08:36:11.748 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.088+0000 7fd3b54ef640 1 -- 172.21.15.164:0/463746085 learned_addr learned my addr 172.21.15.164:0/463746085 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:11.748 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.088+0000 7fd3b54ef640 1 -- 172.21.15.164:0/463746085 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b019d5a0 con 0x7fd3b0105ff0 2024-09-19T08:36:11.748 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.088+0000 7fd3b54ef640 1 --2- 172.21.15.164:0/463746085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3b0105ff0 0x7fd3b019d060 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac0090b0 tx=0x7fd3ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:11.748 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.089+0000 7fd3a67fc640 1 -- 172.21.15.164:0/463746085 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3ac041070 con 0x7fd3b0105ff0 2024-09-19T08:36:11.748 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.089+0000 7fd3a67fc640 1 -- 172.21.15.164:0/463746085 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3ac03c7a0 con 0x7fd3b0105ff0 2024-09-19T08:36:11.748 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.089+0000 7fd3a67fc640 1 -- 172.21.15.164:0/463746085 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3ac037040 con 0x7fd3b0105ff0 2024-09-19T08:36:11.748 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.089+0000 7fd3b7753640 1 -- 172.21.15.164:0/463746085 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3b019d830 con 0x7fd3b0105ff0 2024-09-19T08:36:11.748 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.089+0000 7fd3b7753640 1 -- 172.21.15.164:0/463746085 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3b01a0520 con 0x7fd3b0105ff0 2024-09-19T08:36:11.749 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.089+0000 7fd3a67fc640 1 -- 172.21.15.164:0/463746085 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd3ac044070 con 0x7fd3b0105ff0 2024-09-19T08:36:11.749 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.089+0000 7fd3a67fc640 1 -- 172.21.15.164:0/463746085 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd3ac03d3f0 con 0x7fd3b0105ff0 2024-09-19T08:36:11.749 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.090+0000 7fd3b7753640 1 -- 172.21.15.164:0/463746085 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3b01063f0 con 0x7fd3b0105ff0 2024-09-19T08:36:11.749 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.092+0000 7fd3a67fc640 1 -- 172.21.15.164:0/463746085 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fd3ac046e10 con 0x7fd3b0105ff0 2024-09-19T08:36:11.749 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.149+0000 7fd3b7753640 1 -- 172.21.15.164:0/463746085 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fd3b01a1bf0 con 0x7fd3b0105ff0 2024-09-19T08:36:11.749 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.152+0000 7fd3a67fc640 1 -- 172.21.15.164:0/463746085 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+431 (secure 0 0 0) 0x7fd3ac046450 con 0x7fd3b0105ff0 2024-09-19T08:36:11.749 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.153+0000 7fd3a67fc640 1 -- 172.21.15.164:0/463746085 <== mon.0 v2:172.21.15.164:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd3ac046630 con 0x7fd3b0105ff0 2024-09-19T08:36:11.749 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.154+0000 7fd3b7753640 1 -- 172.21.15.164:0/463746085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3b0105ff0 msgr2=0x7fd3b019d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:11.749 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.154+0000 7fd3b7753640 1 --2- 172.21.15.164:0/463746085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3b0105ff0 0x7fd3b019d060 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac0090b0 tx=0x7fd3ac0046c0 comp rx=0 tx=0).stop 2024-09-19T08:36:11.749 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.154+0000 7fd3b7753640 1 -- 172.21.15.164:0/463746085 shutdown_connections 2024-09-19T08:36:11.749 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.154+0000 7fd3b7753640 1 --2- 172.21.15.164:0/463746085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3b0105ff0 0x7fd3b019d060 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:11.749 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.154+0000 7fd3b7753640 1 -- 172.21.15.164:0/463746085 >> 172.21.15.164:0/463746085 conn(0x7fd3b0101da0 msgr2=0x7fd3b0193670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:11.750 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.154+0000 7fd3b7753640 1 -- 172.21.15.164:0/463746085 shutdown_connections 2024-09-19T08:36:11.750 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:11.154+0000 7fd3b7753640 1 -- 172.21.15.164:0/463746085 wait complete. 2024-09-19T08:36:11.750 INFO:teuthology.orchestra.run.smithi164.stdout:Generating new minimal ceph.conf... 2024-09-19T08:36:11.781 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:11 smithi164 ceph-mon[30565]: from='client.? 172.21.15.164:0/463746085' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T08:36:11.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:11 smithi164 ceph-mon[30565]: from='client.? 172.21.15.164:0/463746085' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-19T08:36:12.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:12 smithi164 ceph-mon[30565]: from='client.? 172.21.15.164:0/3896981160' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:36:12.808 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.165+0000 7fd4e13a0640 1 Processor -- start 2024-09-19T08:36:12.808 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.165+0000 7fd4e13a0640 1 -- start start 2024-09-19T08:36:12.808 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.166+0000 7fd4e13a0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4dc1081c0 0x7fd4dc10a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:12.808 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.166+0000 7fd4e13a0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc07be50 con 0x7fd4dc1081c0 2024-09-19T08:36:12.808 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.166+0000 7fd4daffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4dc1081c0 0x7fd4dc10a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:12.808 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.166+0000 7fd4daffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4dc1081c0 0x7fd4dc10a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:53008/0 (socket says 172.21.15.164:53008) 2024-09-19T08:36:12.808 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.166+0000 7fd4daffd640 1 -- 172.21.15.164:0/2420691956 learned_addr learned my addr 172.21.15.164:0/2420691956 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:12.808 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.166+0000 7fd4daffd640 1 -- 172.21.15.164:0/2420691956 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4dc10aaf0 con 0x7fd4dc1081c0 2024-09-19T08:36:12.808 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.167+0000 7fd4daffd640 1 --2- 172.21.15.164:0/2420691956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4dc1081c0 0x7fd4dc10a5b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009820 tx=0x7fd4cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d907daf1c15939f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:12.808 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.167+0000 7fd4d9ffb640 1 -- 172.21.15.164:0/2420691956 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4cc037070 con 0x7fd4dc1081c0 2024-09-19T08:36:12.809 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.167+0000 7fd4d9ffb640 1 -- 172.21.15.164:0/2420691956 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd4cc032900 con 0x7fd4dc1081c0 2024-09-19T08:36:12.809 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.167+0000 7fd4d9ffb640 1 -- 172.21.15.164:0/2420691956 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4cc030850 con 0x7fd4dc1081c0 2024-09-19T08:36:12.809 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.167+0000 7fd4e13a0640 1 -- 172.21.15.164:0/2420691956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4dc1081c0 msgr2=0x7fd4dc10a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:12.809 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.167+0000 7fd4e13a0640 1 --2- 172.21.15.164:0/2420691956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4dc1081c0 0x7fd4dc10a5b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009820 tx=0x7fd4cc02a270 comp rx=0 tx=0).stop 2024-09-19T08:36:12.809 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.167+0000 7fd4e13a0640 1 -- 172.21.15.164:0/2420691956 shutdown_connections 2024-09-19T08:36:12.809 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.167+0000 7fd4e13a0640 1 --2- 172.21.15.164:0/2420691956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4dc1081c0 0x7fd4dc10a5b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:12.809 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.167+0000 7fd4e13a0640 1 -- 172.21.15.164:0/2420691956 >> 172.21.15.164:0/2420691956 conn(0x7fd4dc101e20 msgr2=0x7fd4dc104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:12.809 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.168+0000 7fd4e13a0640 1 -- 172.21.15.164:0/2420691956 shutdown_connections 2024-09-19T08:36:12.809 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.168+0000 7fd4e13a0640 1 -- 172.21.15.164:0/2420691956 wait complete. 2024-09-19T08:36:12.809 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.168+0000 7fd4e13a0640 1 Processor -- start 2024-09-19T08:36:12.809 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.168+0000 7fd4e13a0640 1 -- start start 2024-09-19T08:36:12.809 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.168+0000 7fd4e13a0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4dc1081c0 0x7fd4dc19b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:12.809 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.168+0000 7fd4e13a0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc10bb30 con 0x7fd4dc1081c0 2024-09-19T08:36:12.810 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.169+0000 7fd4daffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4dc1081c0 0x7fd4dc19b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:12.810 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.169+0000 7fd4daffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4dc1081c0 0x7fd4dc19b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:53014/0 (socket says 172.21.15.164:53014) 2024-09-19T08:36:12.810 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.169+0000 7fd4daffd640 1 -- 172.21.15.164:0/3896981160 learned_addr learned my addr 172.21.15.164:0/3896981160 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:12.810 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.169+0000 7fd4daffd640 1 -- 172.21.15.164:0/3896981160 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4dc19bc20 con 0x7fd4dc1081c0 2024-09-19T08:36:12.810 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.169+0000 7fd4daffd640 1 --2- 172.21.15.164:0/3896981160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4dc1081c0 0x7fd4dc19b6e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0097f0 tx=0x7fd4cc030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:12.810 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.169+0000 7fd4d3fff640 1 -- 172.21.15.164:0/3896981160 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4cc042070 con 0x7fd4dc1081c0 2024-09-19T08:36:12.810 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.169+0000 7fd4d3fff640 1 -- 172.21.15.164:0/3896981160 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd4cc032ca0 con 0x7fd4dc1081c0 2024-09-19T08:36:12.810 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.169+0000 7fd4e13a0640 1 -- 172.21.15.164:0/3896981160 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4dc19beb0 con 0x7fd4dc1081c0 2024-09-19T08:36:12.810 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.169+0000 7fd4d3fff640 1 -- 172.21.15.164:0/3896981160 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4cc037040 con 0x7fd4dc1081c0 2024-09-19T08:36:12.810 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.169+0000 7fd4e13a0640 1 -- 172.21.15.164:0/3896981160 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4dc19eba0 con 0x7fd4dc1081c0 2024-09-19T08:36:12.810 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.169+0000 7fd4d3fff640 1 -- 172.21.15.164:0/3896981160 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd4cc045070 con 0x7fd4dc1081c0 2024-09-19T08:36:12.810 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.169+0000 7fd4d3fff640 1 -- 172.21.15.164:0/3896981160 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd4cc047a90 con 0x7fd4dc1081c0 2024-09-19T08:36:12.810 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.170+0000 7fd4e13a0640 1 -- 172.21.15.164:0/3896981160 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4dc108620 con 0x7fd4dc1081c0 2024-09-19T08:36:12.811 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.172+0000 7fd4d3fff640 1 -- 172.21.15.164:0/3896981160 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fd4cc03db10 con 0x7fd4dc1081c0 2024-09-19T08:36:12.811 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.229+0000 7fd4e13a0640 1 -- 172.21.15.164:0/3896981160 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fd4dc19f120 con 0x7fd4dc1081c0 2024-09-19T08:36:12.811 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.230+0000 7fd4d3fff640 1 -- 172.21.15.164:0/3896981160 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7fd4cc03d430 con 0x7fd4dc1081c0 2024-09-19T08:36:12.811 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.230+0000 7fd4e13a0640 1 -- 172.21.15.164:0/3896981160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4dc1081c0 msgr2=0x7fd4dc19b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:12.811 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.230+0000 7fd4e13a0640 1 --2- 172.21.15.164:0/3896981160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4dc1081c0 0x7fd4dc19b6e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0097f0 tx=0x7fd4cc030ec0 comp rx=0 tx=0).stop 2024-09-19T08:36:12.811 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.231+0000 7fd4e13a0640 1 -- 172.21.15.164:0/3896981160 shutdown_connections 2024-09-19T08:36:12.811 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.231+0000 7fd4e13a0640 1 --2- 172.21.15.164:0/3896981160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4dc1081c0 0x7fd4dc19b6e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:12.811 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.231+0000 7fd4e13a0640 1 -- 172.21.15.164:0/3896981160 >> 172.21.15.164:0/3896981160 conn(0x7fd4dc101e20 msgr2=0x7fd4dc102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:12.811 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.231+0000 7fd4e13a0640 1 -- 172.21.15.164:0/3896981160 shutdown_connections 2024-09-19T08:36:12.811 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:12.231+0000 7fd4e13a0640 1 -- 172.21.15.164:0/3896981160 wait complete. 2024-09-19T08:36:12.811 INFO:teuthology.orchestra.run.smithi164.stdout:Restarting the monitor... 2024-09-19T08:36:13.087 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:12 smithi164 systemd[1]: Stopping Ceph mon.a for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T08:36:13.342 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:13 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mon-a[30561]: 2024-09-19T08:36:13.087+0000 7f9648b96640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:36:13.342 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:13 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mon-a[30561]: 2024-09-19T08:36:13.087+0000 7f9648b96640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-19T08:36:13.620 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:13 smithi164 podman[30805]: 2024-09-19 08:36:13.342783932 +0000 UTC m=+0.372397480 container died 850bea03f6a27ce67fa493eb80f9de7e36b4970750a01e4c45bfd00b7686e26f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mon-a, ceph=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-19T08:36:13.621 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:13 smithi164 podman[30805]: 2024-09-19 08:36:13.621511083 +0000 UTC m=+0.651124699 container cleanup 850bea03f6a27ce67fa493eb80f9de7e36b4970750a01e4c45bfd00b7686e26f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-19T08:36:13.972 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:13 smithi164 bash[30805]: ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mon-a 2024-09-19T08:36:13.972 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:13 smithi164 podman[30817]: 2024-09-19 08:36:13.863945205 +0000 UTC m=+0.519759400 container remove 850bea03f6a27ce67fa493eb80f9de7e36b4970750a01e4c45bfd00b7686e26f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-19T08:36:14.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:13 smithi164 systemd[1]: ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-19T08:36:14.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:13 smithi164 systemd[1]: Stopped Ceph mon.a for 1e4b0d74-7662-11ef-bceb-c7b262605968. 2024-09-19T08:36:14.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:13 smithi164 systemd[1]: Starting Ceph mon.a for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T08:36:14.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 podman[30910]: 2024-09-19 08:36:14.094438056 +0000 UTC m=+0.015877264 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:36:14.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 podman[30910]: 2024-09-19 08:36:14.223093805 +0000 UTC m=+0.144533013 container create 3264bf1a15c86784721d1d56060d27fad20b2b359fc13c3f3db50fb14a7cbd12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mon-a, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=) 2024-09-19T08:36:14.520 INFO:teuthology.orchestra.run.smithi164.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-19T08:36:14.626 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 podman[30910]: 2024-09-19 08:36:14.433469803 +0000 UTC m=+0.354909011 container init 3264bf1a15c86784721d1d56060d27fad20b2b359fc13c3f3db50fb14a7cbd12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-19T08:36:14.626 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 podman[30910]: 2024-09-19 08:36:14.436525982 +0000 UTC m=+0.357965184 container start 3264bf1a15c86784721d1d56060d27fad20b2b359fc13c3f3db50fb14a7cbd12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mon-a, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-19T08:36:14.626 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T08:36:14.626 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T08:36:14.627 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: pidfile_write: ignore empty --pid-file 2024-09-19T08:36:14.627 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: load: jerasure load: lrc 2024-09-19T08:36:14.627 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: RocksDB version: 7.9.2 2024-09-19T08:36:14.627 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Git sha 0 2024-09-19T08:36:14.627 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T08:36:14.627 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: DB SUMMARY 2024-09-19T08:36:14.627 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: DB Session ID: 1OZQ5UAIO9M2R0PWGIY2 2024-09-19T08:36:14.627 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: CURRENT file: CURRENT 2024-09-19T08:36:14.627 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T08:36:14.627 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-19T08:36:14.627 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-19T08:36:14.627 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105850 ; 2024-09-19T08:36:14.627 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.error_if_exists: 0 2024-09-19T08:36:14.628 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.create_if_missing: 0 2024-09-19T08:36:14.628 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.paranoid_checks: 1 2024-09-19T08:36:14.628 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T08:36:14.628 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T08:36:14.628 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T08:36:14.628 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.env: 0x55cacec16940 2024-09-19T08:36:14.628 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T08:36:14.628 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.info_log: 0x55cad13696e0 2024-09-19T08:36:14.628 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T08:36:14.628 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.statistics: (nil) 2024-09-19T08:36:14.628 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.use_fsync: 0 2024-09-19T08:36:14.628 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.max_log_file_size: 0 2024-09-19T08:36:14.629 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T08:36:14.629 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T08:36:14.629 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T08:36:14.629 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T08:36:14.629 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.allow_fallocate: 1 2024-09-19T08:36:14.629 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T08:36:14.629 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T08:36:14.629 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.use_direct_reads: 0 2024-09-19T08:36:14.629 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T08:36:14.629 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T08:36:14.629 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.db_log_dir: 2024-09-19T08:36:14.629 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.wal_dir: 2024-09-19T08:36:14.629 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T08:36:14.630 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T08:36:14.630 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T08:36:14.630 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T08:36:14.630 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T08:36:14.630 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T08:36:14.630 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T08:36:14.630 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T08:36:14.630 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.write_buffer_manager: 0x55cad1378dc0 2024-09-19T08:36:14.630 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T08:36:14.630 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T08:36:14.630 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T08:36:14.630 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T08:36:14.630 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T08:36:14.630 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T08:36:14.631 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T08:36:14.631 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T08:36:14.631 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.unordered_write: 0 2024-09-19T08:36:14.631 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T08:36:14.631 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T08:36:14.631 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T08:36:14.631 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T08:36:14.631 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.row_cache: None 2024-09-19T08:36:14.631 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.wal_filter: None 2024-09-19T08:36:14.631 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T08:36:14.632 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T08:36:14.632 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.two_write_queues: 0 2024-09-19T08:36:14.633 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T08:36:14.633 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.wal_compression: 0 2024-09-19T08:36:14.633 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.atomic_flush: 0 2024-09-19T08:36:14.633 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T08:36:14.633 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T08:36:14.633 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T08:36:14.633 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.log_readahead_size: 0 2024-09-19T08:36:14.633 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T08:36:14.633 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T08:36:14.633 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T08:36:14.633 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T08:36:14.633 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T08:36:14.633 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T08:36:14.634 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T08:36:14.634 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.max_background_jobs: 2 2024-09-19T08:36:14.634 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.max_background_compactions: -1 2024-09-19T08:36:14.634 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.max_subcompactions: 1 2024-09-19T08:36:14.634 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T08:36:14.634 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T08:36:14.634 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T08:36:14.634 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T08:36:14.634 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T08:36:14.634 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T08:36:14.634 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T08:36:14.634 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T08:36:14.634 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.max_open_files: -1 2024-09-19T08:36:14.635 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T08:36:14.635 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T08:36:14.635 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T08:36:14.635 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T08:36:14.635 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.max_background_flushes: -1 2024-09-19T08:36:14.635 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Compression algorithms supported: 2024-09-19T08:36:14.635 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: kZSTD supported: 0 2024-09-19T08:36:14.635 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: kXpressCompression supported: 0 2024-09-19T08:36:14.635 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: kBZip2Compression supported: 0 2024-09-19T08:36:14.635 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T08:36:14.635 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: kLZ4Compression supported: 1 2024-09-19T08:36:14.635 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: kZlibCompression supported: 1 2024-09-19T08:36:14.635 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T08:36:14.635 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: kSnappyCompression supported: 1 2024-09-19T08:36:14.636 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T08:36:14.636 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T08:36:14.636 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-19T08:36:14.636 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T08:36:14.636 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T08:36:14.636 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.merge_operator: 2024-09-19T08:36:14.636 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.compaction_filter: None 2024-09-19T08:36:14.636 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T08:36:14.636 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T08:36:14.636 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T08:36:14.636 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T08:36:14.636 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55cad1369320) 2024-09-19T08:36:14.636 INFO:journalctl@ceph.mon.a.smithi164.stdout: cache_index_and_filter_blocks: 1 2024-09-19T08:36:14.637 INFO:journalctl@ceph.mon.a.smithi164.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T08:36:14.637 INFO:journalctl@ceph.mon.a.smithi164.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T08:36:14.637 INFO:journalctl@ceph.mon.a.smithi164.stdout: pin_top_level_index_and_filter: 1 2024-09-19T08:36:14.637 INFO:journalctl@ceph.mon.a.smithi164.stdout: index_type: 0 2024-09-19T08:36:14.637 INFO:journalctl@ceph.mon.a.smithi164.stdout: data_block_index_type: 0 2024-09-19T08:36:14.637 INFO:journalctl@ceph.mon.a.smithi164.stdout: index_shortening: 1 2024-09-19T08:36:14.637 INFO:journalctl@ceph.mon.a.smithi164.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T08:36:14.637 INFO:journalctl@ceph.mon.a.smithi164.stdout: checksum: 4 2024-09-19T08:36:14.637 INFO:journalctl@ceph.mon.a.smithi164.stdout: no_block_cache: 0 2024-09-19T08:36:14.637 INFO:journalctl@ceph.mon.a.smithi164.stdout: block_cache: 0x55cad135d1f0 2024-09-19T08:36:14.637 INFO:journalctl@ceph.mon.a.smithi164.stdout: block_cache_name: BinnedLRUCache 2024-09-19T08:36:14.637 INFO:journalctl@ceph.mon.a.smithi164.stdout: block_cache_options: 2024-09-19T08:36:14.637 INFO:journalctl@ceph.mon.a.smithi164.stdout: capacity : 536870912 2024-09-19T08:36:14.637 INFO:journalctl@ceph.mon.a.smithi164.stdout: num_shard_bits : 4 2024-09-19T08:36:14.638 INFO:journalctl@ceph.mon.a.smithi164.stdout: strict_capacity_limit : 0 2024-09-19T08:36:14.638 INFO:journalctl@ceph.mon.a.smithi164.stdout: high_pri_pool_ratio: 0.000 2024-09-19T08:36:14.638 INFO:journalctl@ceph.mon.a.smithi164.stdout: block_cache_compressed: (nil) 2024-09-19T08:36:14.638 INFO:journalctl@ceph.mon.a.smithi164.stdout: persistent_cache: (nil) 2024-09-19T08:36:14.638 INFO:journalctl@ceph.mon.a.smithi164.stdout: block_size: 4096 2024-09-19T08:36:14.638 INFO:journalctl@ceph.mon.a.smithi164.stdout: block_size_deviation: 10 2024-09-19T08:36:14.638 INFO:journalctl@ceph.mon.a.smithi164.stdout: block_restart_interval: 16 2024-09-19T08:36:14.638 INFO:journalctl@ceph.mon.a.smithi164.stdout: index_block_restart_interval: 1 2024-09-19T08:36:14.638 INFO:journalctl@ceph.mon.a.smithi164.stdout: metadata_block_size: 4096 2024-09-19T08:36:14.638 INFO:journalctl@ceph.mon.a.smithi164.stdout: partition_filters: 0 2024-09-19T08:36:14.638 INFO:journalctl@ceph.mon.a.smithi164.stdout: use_delta_encoding: 1 2024-09-19T08:36:14.638 INFO:journalctl@ceph.mon.a.smithi164.stdout: filter_policy: bloomfilter 2024-09-19T08:36:14.638 INFO:journalctl@ceph.mon.a.smithi164.stdout: whole_key_filtering: 1 2024-09-19T08:36:14.638 INFO:journalctl@ceph.mon.a.smithi164.stdout: verify_compression: 0 2024-09-19T08:36:14.639 INFO:journalctl@ceph.mon.a.smithi164.stdout: read_amp_bytes_per_bit: 0 2024-09-19T08:36:14.639 INFO:journalctl@ceph.mon.a.smithi164.stdout: format_version: 5 2024-09-19T08:36:14.639 INFO:journalctl@ceph.mon.a.smithi164.stdout: enable_index_compression: 1 2024-09-19T08:36:14.639 INFO:journalctl@ceph.mon.a.smithi164.stdout: block_align: 0 2024-09-19T08:36:14.639 INFO:journalctl@ceph.mon.a.smithi164.stdout: max_auto_readahead_size: 262144 2024-09-19T08:36:14.639 INFO:journalctl@ceph.mon.a.smithi164.stdout: prepopulate_block_cache: 0 2024-09-19T08:36:14.639 INFO:journalctl@ceph.mon.a.smithi164.stdout: initial_auto_readahead_size: 8192 2024-09-19T08:36:14.639 INFO:journalctl@ceph.mon.a.smithi164.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T08:36:14.639 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T08:36:14.639 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T08:36:14.639 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.compression: NoCompression 2024-09-19T08:36:14.640 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T08:36:14.640 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T08:36:14.640 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T08:36:14.640 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.num_levels: 7 2024-09-19T08:36:14.640 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T08:36:14.640 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T08:36:14.640 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T08:36:14.640 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T08:36:14.640 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T08:36:14.640 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T08:36:14.640 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T08:36:14.640 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T08:36:14.640 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T08:36:14.641 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T08:36:14.641 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T08:36:14.641 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T08:36:14.641 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T08:36:14.641 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T08:36:14.641 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T08:36:14.641 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T08:36:14.641 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T08:36:14.641 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T08:36:14.641 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T08:36:14.641 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T08:36:14.641 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T08:36:14.642 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T08:36:14.642 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T08:36:14.642 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T08:36:14.642 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T08:36:14.642 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T08:36:14.642 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T08:36:14.642 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T08:36:14.642 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T08:36:14.642 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T08:36:14.642 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T08:36:14.642 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T08:36:14.642 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T08:36:14.642 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T08:36:14.643 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T08:36:14.643 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T08:36:14.643 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T08:36:14.643 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T08:36:14.643 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T08:36:14.643 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T08:36:14.643 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T08:36:14.643 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T08:36:14.643 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T08:36:14.643 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T08:36:14.644 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T08:36:14.644 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T08:36:14.644 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T08:36:14.644 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T08:36:14.644 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T08:36:14.644 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T08:36:14.644 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T08:36:14.644 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T08:36:14.644 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T08:36:14.644 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T08:36:14.644 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.inplace_update_support: 0 2024-09-19T08:36:14.644 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T08:36:14.644 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T08:36:14.645 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T08:36:14.645 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T08:36:14.645 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.bloom_locality: 0 2024-09-19T08:36:14.645 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.max_successive_merges: 0 2024-09-19T08:36:14.645 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T08:36:14.645 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T08:36:14.645 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T08:36:14.645 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T08:36:14.645 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.ttl: 2592000 2024-09-19T08:36:14.645 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T08:36:14.645 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T08:36:14.645 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T08:36:14.645 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.enable_blob_files: false 2024-09-19T08:36:14.645 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.min_blob_size: 0 2024-09-19T08:36:14.646 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T08:36:14.646 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T08:36:14.646 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T08:36:14.646 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T08:36:14.646 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T08:36:14.646 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T08:36:14.646 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T08:36:14.646 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T08:36:14.646 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-19T08:36:14.646 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-19T08:36:14.646 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2173f493-ccd6-45a0-8955-e93195cf80f2 2024-09-19T08:36:14.646 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726734974462396, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-19T08:36:14.646 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-19T08:36:14.647 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726734974463445, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100861, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 98746, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14423, "raw_average_key_size": 49, "raw_value_size": 90952, "raw_average_value_size": 314, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726734974, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2173f493-ccd6-45a0-8955-e93195cf80f2", "db_session_id": "1OZQ5UAIO9M2R0PWGIY2", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-19T08:36:14.647 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726734974463556, "job": 1, "event": "recovery_finished"} 2024-09-19T08:36:14.647 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-19T08:36:14.647 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T08:36:14.647 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55cad138ae00 2024-09-19T08:36:14.647 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: DB pointer 0x55cad1486000 2024-09-19T08:36:14.647 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T08:36:14.647 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T08:36:14.647 INFO:journalctl@ceph.mon.a.smithi164.stdout: ** DB Stats ** 2024-09-19T08:36:14.647 INFO:journalctl@ceph.mon.a.smithi164.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T08:36:14.647 INFO:journalctl@ceph.mon.a.smithi164.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T08:36:14.647 INFO:journalctl@ceph.mon.a.smithi164.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T08:36:14.647 INFO:journalctl@ceph.mon.a.smithi164.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T08:36:14.647 INFO:journalctl@ceph.mon.a.smithi164.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T08:36:14.648 INFO:journalctl@ceph.mon.a.smithi164.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T08:36:14.648 INFO:journalctl@ceph.mon.a.smithi164.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T08:36:14.648 INFO:journalctl@ceph.mon.a.smithi164.stdout: 2024-09-19T08:36:14.648 INFO:journalctl@ceph.mon.a.smithi164.stdout: ** Compaction Stats [default] ** 2024-09-19T08:36:14.648 INFO:journalctl@ceph.mon.a.smithi164.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T08:36:14.648 INFO:journalctl@ceph.mon.a.smithi164.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T08:36:14.648 INFO:journalctl@ceph.mon.a.smithi164.stdout: L0 2/0 100.36 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 143.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T08:36:14.648 INFO:journalctl@ceph.mon.a.smithi164.stdout: Sum 2/0 100.36 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 143.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T08:36:14.648 INFO:journalctl@ceph.mon.a.smithi164.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 143.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T08:36:14.648 INFO:journalctl@ceph.mon.a.smithi164.stdout: 2024-09-19T08:36:14.648 INFO:journalctl@ceph.mon.a.smithi164.stdout: ** Compaction Stats [default] ** 2024-09-19T08:36:14.648 INFO:journalctl@ceph.mon.a.smithi164.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T08:36:14.648 INFO:journalctl@ceph.mon.a.smithi164.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T08:36:14.649 INFO:journalctl@ceph.mon.a.smithi164.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 143.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T08:36:14.649 INFO:journalctl@ceph.mon.a.smithi164.stdout: 2024-09-19T08:36:14.649 INFO:journalctl@ceph.mon.a.smithi164.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T08:36:14.649 INFO:journalctl@ceph.mon.a.smithi164.stdout: 2024-09-19T08:36:14.649 INFO:journalctl@ceph.mon.a.smithi164.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T08:36:14.649 INFO:journalctl@ceph.mon.a.smithi164.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T08:36:14.649 INFO:journalctl@ceph.mon.a.smithi164.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T08:36:14.649 INFO:journalctl@ceph.mon.a.smithi164.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T08:36:14.649 INFO:journalctl@ceph.mon.a.smithi164.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T08:36:14.649 INFO:journalctl@ceph.mon.a.smithi164.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T08:36:14.649 INFO:journalctl@ceph.mon.a.smithi164.stdout: Cumulative compaction: 0.00 GB write, 30.30 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T08:36:14.649 INFO:journalctl@ceph.mon.a.smithi164.stdout: Interval compaction: 0.00 GB write, 30.30 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T08:36:14.649 INFO:journalctl@ceph.mon.a.smithi164.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T08:36:14.649 INFO:journalctl@ceph.mon.a.smithi164.stdout: Block cache BinnedLRUCache@0x55cad135d1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-19T08:36:14.650 INFO:journalctl@ceph.mon.a.smithi164.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-19T08:36:14.650 INFO:journalctl@ceph.mon.a.smithi164.stdout: 2024-09-19T08:36:14.650 INFO:journalctl@ceph.mon.a.smithi164.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T08:36:14.650 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: starting mon.a rank 0 at public addrs [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] at bind addrs [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 2024-09-19T08:36:14.650 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: mon.a@-1(???) e1 preinit fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 2024-09-19T08:36:14.650 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: mon.a@-1(???).mds e1 new map 2024-09-19T08:36:14.650 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: mon.a@-1(???).mds e1 print_map 2024-09-19T08:36:14.650 INFO:journalctl@ceph.mon.a.smithi164.stdout: e1 2024-09-19T08:36:14.650 INFO:journalctl@ceph.mon.a.smithi164.stdout: btime 2024-09-19T08:36:09:564314+0000 2024-09-19T08:36:14.650 INFO:journalctl@ceph.mon.a.smithi164.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T08:36:14.650 INFO:journalctl@ceph.mon.a.smithi164.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T08:36:14.650 INFO:journalctl@ceph.mon.a.smithi164.stdout: legacy client fscid: -1 2024-09-19T08:36:14.650 INFO:journalctl@ceph.mon.a.smithi164.stdout: 2024-09-19T08:36:14.651 INFO:journalctl@ceph.mon.a.smithi164.stdout: No filesystems configured 2024-09-19T08:36:14.651 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T08:36:14.651 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T08:36:14.651 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T08:36:14.651 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T08:36:14.651 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-19T08:36:14.651 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 bash[30910]: 3264bf1a15c86784721d1d56060d27fad20b2b359fc13c3f3db50fb14a7cbd12 2024-09-19T08:36:14.651 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 systemd[1]: Started Ceph mon.a for 1e4b0d74-7662-11ef-bceb-c7b262605968. 2024-09-19T08:36:14.651 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T08:36:14.651 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: monmap epoch 1 2024-09-19T08:36:14.651 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 2024-09-19T08:36:14.651 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: last_changed 2024-09-19T08:36:06.626290+0000 2024-09-19T08:36:14.652 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: created 2024-09-19T08:36:06.626290+0000 2024-09-19T08:36:14.652 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: min_mon_release 19 (squid) 2024-09-19T08:36:14.652 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: election_strategy: 1 2024-09-19T08:36:14.652 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: 0: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.a 2024-09-19T08:36:14.652 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: fsmap 2024-09-19T08:36:14.652 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T08:36:14.652 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:14 smithi164 ceph-mon[30928]: mgrmap e1: no daemons active 2024-09-19T08:36:15.553 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.961+0000 7f98abcc3640 1 Processor -- start 2024-09-19T08:36:15.554 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.961+0000 7f98abcc3640 1 -- start start 2024-09-19T08:36:15.554 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.961+0000 7f98abcc3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98a407de50 0x7f98a407c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:15.554 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.961+0000 7f98abcc3640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f98a407c840 con 0x7f98a407de50 2024-09-19T08:36:15.554 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.962+0000 7f98a9a5f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98a407de50 0x7f98a407c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:15.554 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.962+0000 7f98a9a5f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98a407de50 0x7f98a407c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43270/0 (socket says 172.21.15.164:43270) 2024-09-19T08:36:15.554 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.962+0000 7f98a9a5f640 1 -- 172.21.15.164:0/1893802468 learned_addr learned my addr 172.21.15.164:0/1893802468 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:15.554 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.962+0000 7f98a9a5f640 1 -- 172.21.15.164:0/1893802468 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a407c9c0 con 0x7f98a407de50 2024-09-19T08:36:15.554 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.964+0000 7f98a9a5f640 1 --2- 172.21.15.164:0/1893802468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98a407de50 0x7f98a407c300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f98a0009820 tx=0x7f98a002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8890c4cb8ec63dc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:15.554 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.964+0000 7f98a8a5d640 1 -- 172.21.15.164:0/1893802468 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98a0037070 con 0x7f98a407de50 2024-09-19T08:36:15.554 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.964+0000 7f98a8a5d640 1 -- 172.21.15.164:0/1893802468 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f98a0032900 con 0x7f98a407de50 2024-09-19T08:36:15.554 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.964+0000 7f98a8a5d640 1 -- 172.21.15.164:0/1893802468 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98a00308e0 con 0x7f98a407de50 2024-09-19T08:36:15.555 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.965+0000 7f98abcc3640 1 -- 172.21.15.164:0/1893802468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98a407de50 msgr2=0x7f98a407c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:15.555 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.965+0000 7f98abcc3640 1 --2- 172.21.15.164:0/1893802468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98a407de50 0x7f98a407c300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f98a0009820 tx=0x7f98a002a300 comp rx=0 tx=0).stop 2024-09-19T08:36:15.555 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.965+0000 7f98abcc3640 1 -- 172.21.15.164:0/1893802468 shutdown_connections 2024-09-19T08:36:15.555 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.965+0000 7f98abcc3640 1 --2- 172.21.15.164:0/1893802468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98a407de50 0x7f98a407c300 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:15.555 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.965+0000 7f98abcc3640 1 -- 172.21.15.164:0/1893802468 >> 172.21.15.164:0/1893802468 conn(0x7f98a4102ef0 msgr2=0x7f98a4105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:15.555 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.965+0000 7f98abcc3640 1 -- 172.21.15.164:0/1893802468 shutdown_connections 2024-09-19T08:36:15.555 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.965+0000 7f98abcc3640 1 -- 172.21.15.164:0/1893802468 wait complete. 2024-09-19T08:36:15.555 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.965+0000 7f98abcc3640 1 Processor -- start 2024-09-19T08:36:15.555 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.965+0000 7f98abcc3640 1 -- start start 2024-09-19T08:36:15.555 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.966+0000 7f98abcc3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98a407de50 0x7f98a419d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:15.555 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.966+0000 7f98abcc3640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f98a410dd30 con 0x7f98a407de50 2024-09-19T08:36:15.555 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.966+0000 7f98a9a5f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98a407de50 0x7f98a419d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:15.556 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.966+0000 7f98a9a5f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98a407de50 0x7f98a419d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43286/0 (socket says 172.21.15.164:43286) 2024-09-19T08:36:15.556 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.966+0000 7f98a9a5f640 1 -- 172.21.15.164:0/3736778613 learned_addr learned my addr 172.21.15.164:0/3736778613 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:15.556 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.966+0000 7f98a9a5f640 1 -- 172.21.15.164:0/3736778613 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a419dde0 con 0x7f98a407de50 2024-09-19T08:36:15.556 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.966+0000 7f98a9a5f640 1 --2- 172.21.15.164:0/3736778613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98a407de50 0x7f98a419d8a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f98a0030610 tx=0x7f98a0032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:15.556 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.966+0000 7f989affd640 1 -- 172.21.15.164:0/3736778613 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98a0037040 con 0x7f98a407de50 2024-09-19T08:36:15.556 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.966+0000 7f989affd640 1 -- 172.21.15.164:0/3736778613 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f98a0040070 con 0x7f98a407de50 2024-09-19T08:36:15.556 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.966+0000 7f98abcc3640 1 -- 172.21.15.164:0/3736778613 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a419e070 con 0x7f98a407de50 2024-09-19T08:36:15.556 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.966+0000 7f989affd640 1 -- 172.21.15.164:0/3736778613 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98a003a390 con 0x7f98a407de50 2024-09-19T08:36:15.556 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.966+0000 7f98abcc3640 1 -- 172.21.15.164:0/3736778613 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a41a0d60 con 0x7f98a407de50 2024-09-19T08:36:15.556 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.966+0000 7f989affd640 1 -- 172.21.15.164:0/3736778613 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f98a003a640 con 0x7f98a407de50 2024-09-19T08:36:15.556 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.967+0000 7f989affd640 1 -- 172.21.15.164:0/3736778613 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f98a0044420 con 0x7f98a407de50 2024-09-19T08:36:15.556 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.967+0000 7f98abcc3640 1 -- 172.21.15.164:0/3736778613 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a407c380 con 0x7f98a407de50 2024-09-19T08:36:15.556 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:14.970+0000 7f989affd640 1 -- 172.21.15.164:0/3736778613 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f98a004a020 con 0x7f98a407de50 2024-09-19T08:36:15.557 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:15.027+0000 7f98abcc3640 1 -- 172.21.15.164:0/3736778613 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f98a41a0f30 con 0x7f98a407de50 2024-09-19T08:36:15.557 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:15.031+0000 7f989affd640 1 -- 172.21.15.164:0/3736778613 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f98a003b430 con 0x7f98a407de50 2024-09-19T08:36:15.557 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:15.032+0000 7f98abcc3640 1 -- 172.21.15.164:0/3736778613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98a407de50 msgr2=0x7f98a419d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:15.557 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:15.032+0000 7f98abcc3640 1 --2- 172.21.15.164:0/3736778613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98a407de50 0x7f98a419d8a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f98a0030610 tx=0x7f98a0032670 comp rx=0 tx=0).stop 2024-09-19T08:36:15.557 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:15.032+0000 7f98abcc3640 1 -- 172.21.15.164:0/3736778613 shutdown_connections 2024-09-19T08:36:15.557 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:15.032+0000 7f98abcc3640 1 --2- 172.21.15.164:0/3736778613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98a407de50 0x7f98a419d8a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:15.557 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:15.032+0000 7f98abcc3640 1 -- 172.21.15.164:0/3736778613 >> 172.21.15.164:0/3736778613 conn(0x7f98a4102ef0 msgr2=0x7f98a4103a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:15.557 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:15.032+0000 7f98abcc3640 1 -- 172.21.15.164:0/3736778613 shutdown_connections 2024-09-19T08:36:15.557 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:15.032+0000 7f98abcc3640 1 -- 172.21.15.164:0/3736778613 wait complete. 2024-09-19T08:36:15.557 INFO:teuthology.orchestra.run.smithi164.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-19T08:36:15.572 INFO:teuthology.orchestra.run.smithi164.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-19T08:36:15.572 INFO:teuthology.orchestra.run.smithi164.stdout:Creating mgr... 2024-09-19T08:36:15.572 INFO:teuthology.orchestra.run.smithi164.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-19T08:36:15.573 INFO:teuthology.orchestra.run.smithi164.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-19T08:36:15.832 INFO:teuthology.orchestra.run.smithi164.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mgr.x 2024-09-19T08:36:15.833 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mgr.x.service: Unit ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mgr.x.service not loaded. 2024-09-19T08:36:15.986 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1e4b0d74-7662-11ef-bceb-c7b262605968.target.wants/ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@.service. 2024-09-19T08:36:16.208 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:16 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3736778613' entity='client.admin' 2024-09-19T08:36:16.468 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:16 smithi164 podman[31138]: 2024-09-19 08:36:16.405850313 +0000 UTC m=+0.299717691 container start 3287a3ff4ddf6c06dadb892475c84695b5f03768a0b8b72ef2e1d4ebbee6fb44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True) 2024-09-19T08:36:16.469 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:16 smithi164 bash[31138]: 3287a3ff4ddf6c06dadb892475c84695b5f03768a0b8b72ef2e1d4ebbee6fb44 2024-09-19T08:36:16.482 INFO:teuthology.orchestra.run.smithi164.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T08:36:16.482 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stdout disabled 2024-09-19T08:36:16.489 INFO:teuthology.orchestra.run.smithi164.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T08:36:16.489 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stdout inactive 2024-09-19T08:36:16.489 INFO:teuthology.orchestra.run.smithi164.stdout:firewalld.service is not enabled 2024-09-19T08:36:16.489 INFO:teuthology.orchestra.run.smithi164.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T08:36:16.498 INFO:teuthology.orchestra.run.smithi164.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T08:36:16.498 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stdout disabled 2024-09-19T08:36:16.505 INFO:teuthology.orchestra.run.smithi164.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T08:36:16.506 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stdout inactive 2024-09-19T08:36:16.506 INFO:teuthology.orchestra.run.smithi164.stdout:firewalld.service is not enabled 2024-09-19T08:36:16.506 INFO:teuthology.orchestra.run.smithi164.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-19T08:36:16.506 INFO:teuthology.orchestra.run.smithi164.stdout:Waiting for mgr to start... 2024-09-19T08:36:16.506 INFO:teuthology.orchestra.run.smithi164.stdout:Waiting for mgr... 2024-09-19T08:36:16.727 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:16 smithi164 systemd[1]: Started Ceph mgr.x for 1e4b0d74-7662-11ef-bceb-c7b262605968. 2024-09-19T08:36:16.727 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:16 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:16.529+0000 7fc759131040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T08:36:16.727 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:16 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:16.586+0000 7fc759131040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T08:36:16.977 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:16 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:16.740+0000 7fc759131040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T08:36:16.977 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:16 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:16.806+0000 7fc759131040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T08:36:16.977 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:16 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:16.961+0000 7fc759131040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:36:17.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:17 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2461422006' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T08:36:17.282 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:17 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:17.108+0000 7fc759131040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T08:36:17.613 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout 2024-09-19T08:36:17.613 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout { 2024-09-19T08:36:17.614 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "fsid": "1e4b0d74-7662-11ef-bceb-c7b262605968", 2024-09-19T08:36:17.614 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T08:36:17.614 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T08:36:17.614 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T08:36:17.614 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T08:36:17.614 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:36:17.614 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T08:36:17.614 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T08:36:17.614 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout 0 2024-09-19T08:36:17.614 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout ], 2024-09-19T08:36:17.614 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T08:36:17.614 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T08:36:17.614 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout ], 2024-09-19T08:36:17.615 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-19T08:36:17.615 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T08:36:17.615 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:36:17.615 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T08:36:17.615 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T08:36:17.615 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:36:17.615 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T08:36:17.615 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:36:17.615 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T08:36:17.615 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T08:36:17.615 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T08:36:17.623 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T08:36:17.623 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T08:36:17.623 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T08:36:17.623 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:36:17.623 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T08:36:17.623 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T08:36:17.623 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T08:36:17.623 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T08:36:17.624 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T08:36:17.624 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T08:36:17.624 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T08:36:17.624 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T08:36:17.624 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T08:36:17.624 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:36:17.624 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T08:36:17.624 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:36:17.624 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T08:36:09:564314+0000", 2024-09-19T08:36:17.624 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T08:36:17.624 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T08:36:17.624 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:36:17.624 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T08:36:17.625 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-19T08:36:17.625 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T08:36:17.625 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T08:36:17.625 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T08:36:17.625 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T08:36:17.625 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T08:36:17.625 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout ], 2024-09-19T08:36:17.625 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T08:36:17.625 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:36:17.625 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T08:36:17.625 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:36:17.625 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T08:36:09.564961+0000", 2024-09-19T08:36:17.626 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T08:36:17.626 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:36:17.626 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T08:36:17.626 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout } 2024-09-19T08:36:17.626 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.908+0000 7f880e75a640 1 Processor -- start 2024-09-19T08:36:17.626 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.908+0000 7f880e75a640 1 -- start start 2024-09-19T08:36:17.626 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.909+0000 7f880e75a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88081065f0 0x7f88081069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:17.626 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.909+0000 7f880e75a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8808106fc0 con 0x7f88081065f0 2024-09-19T08:36:17.626 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.909+0000 7f8807fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88081065f0 0x7f88081069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:17.626 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.909+0000 7f8807fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88081065f0 0x7f88081069f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43330/0 (socket says 172.21.15.164:43330) 2024-09-19T08:36:17.626 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.909+0000 7f8807fff640 1 -- 172.21.15.164:0/1295493717 learned_addr learned my addr 172.21.15.164:0/1295493717 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:17.626 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.909+0000 7f8807fff640 1 -- 172.21.15.164:0/1295493717 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88081077f0 con 0x7f88081065f0 2024-09-19T08:36:17.627 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.910+0000 7f8807fff640 1 --2- 172.21.15.164:0/1295493717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88081065f0 0x7f88081069f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009820 tx=0x7f87f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb6bc84974a8e2e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:17.627 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.910+0000 7f88077fe640 1 -- 172.21.15.164:0/1295493717 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87f8037070 con 0x7f88081065f0 2024-09-19T08:36:17.627 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.910+0000 7f88077fe640 1 -- 172.21.15.164:0/1295493717 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f87f8032900 con 0x7f88081065f0 2024-09-19T08:36:17.627 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.910+0000 7f88077fe640 1 -- 172.21.15.164:0/1295493717 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87f8030850 con 0x7f88081065f0 2024-09-19T08:36:17.627 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.911+0000 7f880e75a640 1 -- 172.21.15.164:0/1295493717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88081065f0 msgr2=0x7f88081069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:17.627 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.911+0000 7f880e75a640 1 --2- 172.21.15.164:0/1295493717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88081065f0 0x7f88081069f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009820 tx=0x7f87f802a270 comp rx=0 tx=0).stop 2024-09-19T08:36:17.627 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.911+0000 7f880e75a640 1 -- 172.21.15.164:0/1295493717 shutdown_connections 2024-09-19T08:36:17.627 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.911+0000 7f880e75a640 1 --2- 172.21.15.164:0/1295493717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88081065f0 0x7f88081069f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:17.627 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.911+0000 7f880e75a640 1 -- 172.21.15.164:0/1295493717 >> 172.21.15.164:0/1295493717 conn(0x7f8808101e20 msgr2=0x7f8808104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:17.627 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.911+0000 7f880e75a640 1 -- 172.21.15.164:0/1295493717 shutdown_connections 2024-09-19T08:36:17.627 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.911+0000 7f880e75a640 1 -- 172.21.15.164:0/1295493717 wait complete. 2024-09-19T08:36:17.627 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.912+0000 7f880e75a640 1 Processor -- start 2024-09-19T08:36:17.627 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.912+0000 7f880e75a640 1 -- start start 2024-09-19T08:36:17.628 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.912+0000 7f880e75a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88081065f0 0x7f880807ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:17.628 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.912+0000 7f880e75a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f880810aba0 con 0x7f88081065f0 2024-09-19T08:36:17.628 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.912+0000 7f8807fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88081065f0 0x7f880807ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:17.628 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.912+0000 7f8807fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88081065f0 0x7f880807ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43334/0 (socket says 172.21.15.164:43334) 2024-09-19T08:36:17.628 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.912+0000 7f8807fff640 1 -- 172.21.15.164:0/2461422006 learned_addr learned my addr 172.21.15.164:0/2461422006 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:17.628 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.913+0000 7f8807fff640 1 -- 172.21.15.164:0/2461422006 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f880819e470 con 0x7f88081065f0 2024-09-19T08:36:17.628 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.913+0000 7f8807fff640 1 --2- 172.21.15.164:0/2461422006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88081065f0 0x7f880807ba50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009940 tx=0x7f87f80326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:17.628 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.913+0000 7f8805ffb640 1 -- 172.21.15.164:0/2461422006 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87f8042070 con 0x7f88081065f0 2024-09-19T08:36:17.628 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.913+0000 7f8805ffb640 1 -- 172.21.15.164:0/2461422006 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f87f8032be0 con 0x7f88081065f0 2024-09-19T08:36:17.628 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.913+0000 7f880e75a640 1 -- 172.21.15.164:0/2461422006 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f880819e640 con 0x7f88081065f0 2024-09-19T08:36:17.628 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.913+0000 7f8805ffb640 1 -- 172.21.15.164:0/2461422006 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87f8037040 con 0x7f88081065f0 2024-09-19T08:36:17.628 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.913+0000 7f880e75a640 1 -- 172.21.15.164:0/2461422006 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f880819e9e0 con 0x7f88081065f0 2024-09-19T08:36:17.629 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.913+0000 7f8805ffb640 1 -- 172.21.15.164:0/2461422006 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f87f8045070 con 0x7f88081065f0 2024-09-19T08:36:17.629 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.913+0000 7f8805ffb640 1 -- 172.21.15.164:0/2461422006 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f87f8047da0 con 0x7f88081065f0 2024-09-19T08:36:17.629 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.914+0000 7f880e75a640 1 -- 172.21.15.164:0/2461422006 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f880810ad20 con 0x7f88081065f0 2024-09-19T08:36:17.629 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.916+0000 7f8805ffb640 1 -- 172.21.15.164:0/2461422006 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f87f8032d90 con 0x7f88081065f0 2024-09-19T08:36:17.629 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.977+0000 7f880e75a640 1 -- 172.21.15.164:0/2461422006 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f880819ee40 con 0x7f88081065f0 2024-09-19T08:36:17.629 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.977+0000 7f8805ffb640 1 -- 172.21.15.164:0/2461422006 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f87f8047440 con 0x7f88081065f0 2024-09-19T08:36:17.629 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.979+0000 7f880e75a640 1 -- 172.21.15.164:0/2461422006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88081065f0 msgr2=0x7f880807ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:17.629 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.979+0000 7f880e75a640 1 --2- 172.21.15.164:0/2461422006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88081065f0 0x7f880807ba50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009940 tx=0x7f87f80326e0 comp rx=0 tx=0).stop 2024-09-19T08:36:17.629 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.979+0000 7f880e75a640 1 -- 172.21.15.164:0/2461422006 shutdown_connections 2024-09-19T08:36:17.629 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.979+0000 7f880e75a640 1 --2- 172.21.15.164:0/2461422006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88081065f0 0x7f880807ba50 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:17.629 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.979+0000 7f880e75a640 1 -- 172.21.15.164:0/2461422006 >> 172.21.15.164:0/2461422006 conn(0x7f8808101e20 msgr2=0x7f8808102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:17.629 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.979+0000 7f880e75a640 1 -- 172.21.15.164:0/2461422006 shutdown_connections 2024-09-19T08:36:17.629 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:16.979+0000 7f880e75a640 1 -- 172.21.15.164:0/2461422006 wait complete. 2024-09-19T08:36:17.630 INFO:teuthology.orchestra.run.smithi164.stdout:mgr not available, waiting (1/15)... 2024-09-19T08:36:17.782 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:17 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:17.426+0000 7fc759131040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T08:36:17.782 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:17 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:17.487+0000 7fc759131040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T08:36:17.782 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:17 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:17.675+0000 7fc759131040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T08:36:19.032 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:18 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:18.548+0000 7fc759131040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T08:36:19.032 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:18 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:18.614+0000 7fc759131040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T08:36:19.577 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:19 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:19.322+0000 7fc759131040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T08:36:19.577 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:19 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:19.436+0000 7fc759131040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T08:36:19.577 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:19 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:19.577+0000 7fc759131040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:36:19.862 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:19 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:19.641+0000 7fc759131040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T08:36:19.862 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:19 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:19.793+0000 7fc759131040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T08:36:19.862 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:19 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:19.862+0000 7fc759131040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T08:36:20.147 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:19 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:19.925+0000 7fc759131040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T08:36:20.147 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:20 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:20.089+0000 7fc759131040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T08:36:20.431 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:20 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2363175428' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T08:36:20.432 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:20 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:20.159+0000 7fc759131040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T08:36:20.432 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:20 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:20.227+0000 7fc759131040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T08:36:20.432 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:20 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:20.309+0000 7fc759131040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T08:36:20.432 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:20 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:20.373+0000 7fc759131040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T08:36:20.432 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:20 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:20.431+0000 7fc759131040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T08:36:20.693 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout 2024-09-19T08:36:20.694 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout { 2024-09-19T08:36:20.694 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "fsid": "1e4b0d74-7662-11ef-bceb-c7b262605968", 2024-09-19T08:36:20.694 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T08:36:20.694 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T08:36:20.694 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T08:36:20.694 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T08:36:20.694 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:36:20.694 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T08:36:20.694 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T08:36:20.694 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout 0 2024-09-19T08:36:20.694 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout ], 2024-09-19T08:36:20.694 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T08:36:20.694 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T08:36:20.695 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout ], 2024-09-19T08:36:20.695 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-19T08:36:20.695 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T08:36:20.695 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:36:20.695 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T08:36:20.695 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T08:36:20.695 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:36:20.695 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T08:36:20.695 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:36:20.695 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T08:36:20.695 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T08:36:20.695 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T08:36:20.695 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T08:36:20.696 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T08:36:20.696 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T08:36:20.696 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:36:20.696 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T08:36:20.696 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T08:36:20.696 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T08:36:20.696 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T08:36:20.696 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T08:36:20.696 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T08:36:20.696 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T08:36:20.696 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T08:36:20.696 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T08:36:20.696 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:36:20.697 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T08:36:20.697 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:36:20.697 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T08:36:09:564314+0000", 2024-09-19T08:36:20.697 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T08:36:20.697 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T08:36:20.697 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:36:20.697 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T08:36:20.697 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-19T08:36:20.697 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T08:36:20.697 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T08:36:20.697 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T08:36:20.697 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T08:36:20.697 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T08:36:20.697 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout ], 2024-09-19T08:36:20.698 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T08:36:20.698 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:36:20.698 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T08:36:20.698 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:36:20.698 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T08:36:09.564961+0000", 2024-09-19T08:36:20.698 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T08:36:20.698 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:36:20.698 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T08:36:20.698 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout } 2024-09-19T08:36:20.698 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.029+0000 7f0cf8c13640 1 Processor -- start 2024-09-19T08:36:20.698 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.029+0000 7f0cf8c13640 1 -- start start 2024-09-19T08:36:20.698 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.029+0000 7f0cf8c13640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cf41065f0 0x7f0cf41069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:20.698 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.029+0000 7f0cf8c13640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf4106fc0 con 0x7f0cf41065f0 2024-09-19T08:36:20.699 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.029+0000 7f0cf259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cf41065f0 0x7f0cf41069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:20.699 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.029+0000 7f0cf259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cf41065f0 0x7f0cf41069f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43340/0 (socket says 172.21.15.164:43340) 2024-09-19T08:36:20.699 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.029+0000 7f0cf259c640 1 -- 172.21.15.164:0/4237508311 learned_addr learned my addr 172.21.15.164:0/4237508311 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:20.699 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.030+0000 7f0cf259c640 1 -- 172.21.15.164:0/4237508311 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf41077f0 con 0x7f0cf41065f0 2024-09-19T08:36:20.699 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.030+0000 7f0cf259c640 1 --2- 172.21.15.164:0/4237508311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cf41065f0 0x7f0cf41069f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009820 tx=0x7f0ce402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc3030ac4ef9d7d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:20.699 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.031+0000 7f0cf159a640 1 -- 172.21.15.164:0/4237508311 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ce4037070 con 0x7f0cf41065f0 2024-09-19T08:36:20.699 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.031+0000 7f0cf159a640 1 -- 172.21.15.164:0/4237508311 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0ce4032900 con 0x7f0cf41065f0 2024-09-19T08:36:20.699 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.031+0000 7f0cf159a640 1 -- 172.21.15.164:0/4237508311 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ce4030850 con 0x7f0cf41065f0 2024-09-19T08:36:20.699 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.031+0000 7f0cf8c13640 1 -- 172.21.15.164:0/4237508311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cf41065f0 msgr2=0x7f0cf41069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:20.699 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.031+0000 7f0cf8c13640 1 --2- 172.21.15.164:0/4237508311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cf41065f0 0x7f0cf41069f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009820 tx=0x7f0ce402a270 comp rx=0 tx=0).stop 2024-09-19T08:36:20.699 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.031+0000 7f0cf8c13640 1 -- 172.21.15.164:0/4237508311 shutdown_connections 2024-09-19T08:36:20.699 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.031+0000 7f0cf8c13640 1 --2- 172.21.15.164:0/4237508311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cf41065f0 0x7f0cf41069f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:20.700 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.031+0000 7f0cf8c13640 1 -- 172.21.15.164:0/4237508311 >> 172.21.15.164:0/4237508311 conn(0x7f0cf4101e20 msgr2=0x7f0cf4104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:20.700 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.031+0000 7f0cf8c13640 1 -- 172.21.15.164:0/4237508311 shutdown_connections 2024-09-19T08:36:20.700 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.031+0000 7f0cf8c13640 1 -- 172.21.15.164:0/4237508311 wait complete. 2024-09-19T08:36:20.700 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.032+0000 7f0cf8c13640 1 Processor -- start 2024-09-19T08:36:20.700 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.032+0000 7f0cf8c13640 1 -- start start 2024-09-19T08:36:20.700 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.032+0000 7f0cf8c13640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cf41065f0 0x7f0cf419b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:20.700 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.032+0000 7f0cf8c13640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf410aba0 con 0x7f0cf41065f0 2024-09-19T08:36:20.700 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.032+0000 7f0cf259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cf41065f0 0x7f0cf419b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:20.700 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.032+0000 7f0cf259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cf41065f0 0x7f0cf419b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43348/0 (socket says 172.21.15.164:43348) 2024-09-19T08:36:20.700 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.032+0000 7f0cf259c640 1 -- 172.21.15.164:0/2363175428 learned_addr learned my addr 172.21.15.164:0/2363175428 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:20.700 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.033+0000 7f0cf259c640 1 -- 172.21.15.164:0/2363175428 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf419bca0 con 0x7f0cf41065f0 2024-09-19T08:36:20.700 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.033+0000 7f0cf259c640 1 --2- 172.21.15.164:0/2363175428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cf41065f0 0x7f0cf419b760 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009940 tx=0x7f0ce40326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:20.701 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.033+0000 7f0ce37fe640 1 -- 172.21.15.164:0/2363175428 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ce4042070 con 0x7f0cf41065f0 2024-09-19T08:36:20.701 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.033+0000 7f0ce37fe640 1 -- 172.21.15.164:0/2363175428 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0ce4032be0 con 0x7f0cf41065f0 2024-09-19T08:36:20.701 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.033+0000 7f0cf8c13640 1 -- 172.21.15.164:0/2363175428 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf419bf30 con 0x7f0cf41065f0 2024-09-19T08:36:20.701 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.033+0000 7f0ce37fe640 1 -- 172.21.15.164:0/2363175428 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ce4037040 con 0x7f0cf41065f0 2024-09-19T08:36:20.701 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.033+0000 7f0cf8c13640 1 -- 172.21.15.164:0/2363175428 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf419ec20 con 0x7f0cf41065f0 2024-09-19T08:36:20.701 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.033+0000 7f0ce37fe640 1 -- 172.21.15.164:0/2363175428 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0ce4045070 con 0x7f0cf41065f0 2024-09-19T08:36:20.701 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.033+0000 7f0ce37fe640 1 -- 172.21.15.164:0/2363175428 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0ce4047a30 con 0x7f0cf41065f0 2024-09-19T08:36:20.701 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.034+0000 7f0cf8c13640 1 -- 172.21.15.164:0/2363175428 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cf410ad20 con 0x7f0cf41065f0 2024-09-19T08:36:20.701 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.036+0000 7f0ce37fe640 1 -- 172.21.15.164:0/2363175428 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f0ce4032d90 con 0x7f0cf41065f0 2024-09-19T08:36:20.701 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.096+0000 7f0cf8c13640 1 -- 172.21.15.164:0/2363175428 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0cf41956c0 con 0x7f0cf41065f0 2024-09-19T08:36:20.701 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.097+0000 7f0ce37fe640 1 -- 172.21.15.164:0/2363175428 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f0ce403d430 con 0x7f0cf41065f0 2024-09-19T08:36:20.701 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.097+0000 7f0cf8c13640 1 -- 172.21.15.164:0/2363175428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cf41065f0 msgr2=0x7f0cf419b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:20.701 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.097+0000 7f0cf8c13640 1 --2- 172.21.15.164:0/2363175428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cf41065f0 0x7f0cf419b760 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009940 tx=0x7f0ce40326e0 comp rx=0 tx=0).stop 2024-09-19T08:36:20.701 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.098+0000 7f0cf8c13640 1 -- 172.21.15.164:0/2363175428 shutdown_connections 2024-09-19T08:36:20.702 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.098+0000 7f0cf8c13640 1 --2- 172.21.15.164:0/2363175428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cf41065f0 0x7f0cf419b760 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:20.702 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.098+0000 7f0cf8c13640 1 -- 172.21.15.164:0/2363175428 >> 172.21.15.164:0/2363175428 conn(0x7f0cf4101e20 msgr2=0x7f0cf4102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:20.702 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.098+0000 7f0cf8c13640 1 -- 172.21.15.164:0/2363175428 shutdown_connections 2024-09-19T08:36:20.702 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:20.098+0000 7f0cf8c13640 1 -- 172.21.15.164:0/2363175428 wait complete. 2024-09-19T08:36:20.702 INFO:teuthology.orchestra.run.smithi164.stdout:mgr not available, waiting (2/15)... 2024-09-19T08:36:21.532 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:21 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:21.223+0000 7fc759131040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T08:36:21.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:21 smithi164 ceph-mon[30928]: Activating manager daemon x 2024-09-19T08:36:21.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:21 smithi164 ceph-mon[30928]: mgrmap e2: x(active, starting, since 0.00329759s) 2024-09-19T08:36:21.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:21 smithi164 ceph-mon[30928]: from='mgr.14100 172.21.15.164:0/2274202032' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T08:36:21.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:21 smithi164 ceph-mon[30928]: from='mgr.14100 172.21.15.164:0/2274202032' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T08:36:21.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:21 smithi164 ceph-mon[30928]: from='mgr.14100 172.21.15.164:0/2274202032' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T08:36:21.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:21 smithi164 ceph-mon[30928]: from='mgr.14100 172.21.15.164:0/2274202032' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:36:21.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:21 smithi164 ceph-mon[30928]: from='mgr.14100 172.21.15.164:0/2274202032' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T08:36:21.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:21 smithi164 ceph-mon[30928]: Manager daemon x is now available 2024-09-19T08:36:21.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:21 smithi164 ceph-mon[30928]: from='mgr.14100 172.21.15.164:0/2274202032' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T08:36:21.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:21 smithi164 ceph-mon[30928]: from='mgr.14100 172.21.15.164:0/2274202032' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T08:36:21.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:21 smithi164 ceph-mon[30928]: from='mgr.14100 172.21.15.164:0/2274202032' entity='mgr.x' 2024-09-19T08:36:21.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:21 smithi164 ceph-mon[30928]: from='mgr.14100 172.21.15.164:0/2274202032' entity='mgr.x' 2024-09-19T08:36:21.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:21 smithi164 ceph-mon[30928]: from='mgr.14100 172.21.15.164:0/2274202032' entity='mgr.x' 2024-09-19T08:36:23.342 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:23 smithi164 ceph-mon[30928]: mgrmap e3: x(active, since 1.00733s) 2024-09-19T08:36:23.859 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout 2024-09-19T08:36:23.859 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout { 2024-09-19T08:36:23.859 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "fsid": "1e4b0d74-7662-11ef-bceb-c7b262605968", 2024-09-19T08:36:23.859 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T08:36:23.859 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T08:36:23.860 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T08:36:23.860 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T08:36:23.860 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:36:23.860 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T08:36:23.860 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T08:36:23.860 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout 0 2024-09-19T08:36:23.861 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout ], 2024-09-19T08:36:23.861 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T08:36:23.861 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T08:36:23.861 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout ], 2024-09-19T08:36:23.861 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-09-19T08:36:23.861 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T08:36:23.861 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:36:23.861 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T08:36:23.861 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T08:36:23.861 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:36:23.861 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T08:36:23.861 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:36:23.862 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T08:36:23.862 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T08:36:23.862 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T08:36:23.872 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T08:36:23.872 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T08:36:23.872 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T08:36:23.872 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:36:23.873 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T08:36:23.873 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T08:36:23.873 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T08:36:23.873 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T08:36:23.873 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T08:36:23.873 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T08:36:23.873 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T08:36:23.873 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T08:36:23.873 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T08:36:23.873 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:36:23.873 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T08:36:23.874 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:36:23.874 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T08:36:09:564314+0000", 2024-09-19T08:36:23.874 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T08:36:23.874 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T08:36:23.874 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:36:23.874 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T08:36:23.874 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T08:36:23.874 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T08:36:23.874 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T08:36:23.874 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T08:36:23.874 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T08:36:23.874 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T08:36:23.874 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout ], 2024-09-19T08:36:23.875 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T08:36:23.875 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:36:23.875 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T08:36:23.875 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T08:36:23.875 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T08:36:09.564961+0000", 2024-09-19T08:36:23.875 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T08:36:23.875 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout }, 2024-09-19T08:36:23.875 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T08:36:23.875 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout } 2024-09-19T08:36:23.875 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.090+0000 7f50db803640 1 Processor -- start 2024-09-19T08:36:23.875 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.090+0000 7f50db803640 1 -- start start 2024-09-19T08:36:23.875 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.091+0000 7f50db803640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50d41081c0 0x7f50d410a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:23.875 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.091+0000 7f50db803640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f50d407be50 con 0x7f50d41081c0 2024-09-19T08:36:23.876 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.091+0000 7f50d959f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50d41081c0 0x7f50d410a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:23.876 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.091+0000 7f50d959f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50d41081c0 0x7f50d410a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43412/0 (socket says 172.21.15.164:43412) 2024-09-19T08:36:23.876 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.091+0000 7f50d959f640 1 -- 172.21.15.164:0/1545988198 learned_addr learned my addr 172.21.15.164:0/1545988198 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:23.876 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.091+0000 7f50d959f640 1 -- 172.21.15.164:0/1545988198 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d410aaf0 con 0x7f50d41081c0 2024-09-19T08:36:23.876 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.092+0000 7f50d959f640 1 --2- 172.21.15.164:0/1545988198 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50d41081c0 0x7f50d410a5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f50cc009e80 tx=0x7f50cc02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b6aa85de12a765 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:23.876 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.092+0000 7f50d3fff640 1 -- 172.21.15.164:0/1545988198 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50cc036070 con 0x7f50d41081c0 2024-09-19T08:36:23.876 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.092+0000 7f50d3fff640 1 -- 172.21.15.164:0/1545988198 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f50cc0318c0 con 0x7f50d41081c0 2024-09-19T08:36:23.876 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.092+0000 7f50d3fff640 1 -- 172.21.15.164:0/1545988198 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50cc007ab0 con 0x7f50d41081c0 2024-09-19T08:36:23.876 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.092+0000 7f50db803640 1 -- 172.21.15.164:0/1545988198 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50d41081c0 msgr2=0x7f50d410a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:23.876 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.092+0000 7f50db803640 1 --2- 172.21.15.164:0/1545988198 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50d41081c0 0x7f50d410a5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f50cc009e80 tx=0x7f50cc02a7a0 comp rx=0 tx=0).stop 2024-09-19T08:36:23.876 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.093+0000 7f50db803640 1 -- 172.21.15.164:0/1545988198 shutdown_connections 2024-09-19T08:36:23.876 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.093+0000 7f50db803640 1 --2- 172.21.15.164:0/1545988198 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50d41081c0 0x7f50d410a5b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:23.876 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.093+0000 7f50db803640 1 -- 172.21.15.164:0/1545988198 >> 172.21.15.164:0/1545988198 conn(0x7f50d4101e20 msgr2=0x7f50d4104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:23.877 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.093+0000 7f50db803640 1 -- 172.21.15.164:0/1545988198 shutdown_connections 2024-09-19T08:36:23.877 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.093+0000 7f50db803640 1 -- 172.21.15.164:0/1545988198 wait complete. 2024-09-19T08:36:23.877 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.093+0000 7f50db803640 1 Processor -- start 2024-09-19T08:36:23.877 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.093+0000 7f50db803640 1 -- start start 2024-09-19T08:36:23.877 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.093+0000 7f50db803640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50d41081c0 0x7f50d4107a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:23.877 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.093+0000 7f50db803640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f50d410bb30 con 0x7f50d41081c0 2024-09-19T08:36:23.877 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.094+0000 7f50d959f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50d41081c0 0x7f50d4107a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:23.877 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.094+0000 7f50d959f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50d41081c0 0x7f50d4107a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43428/0 (socket says 172.21.15.164:43428) 2024-09-19T08:36:23.877 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.094+0000 7f50d959f640 1 -- 172.21.15.164:0/2400291179 learned_addr learned my addr 172.21.15.164:0/2400291179 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:23.877 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.094+0000 7f50d959f640 1 -- 172.21.15.164:0/2400291179 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d4107f80 con 0x7f50d41081c0 2024-09-19T08:36:23.877 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.094+0000 7f50d959f640 1 --2- 172.21.15.164:0/2400291179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50d41081c0 0x7f50d4107a40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f50cc031ca0 tx=0x7f50cc031cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:23.877 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.094+0000 7f50d27fc640 1 -- 172.21.15.164:0/2400291179 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50cc036050 con 0x7f50d41081c0 2024-09-19T08:36:23.878 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.094+0000 7f50d27fc640 1 -- 172.21.15.164:0/2400291179 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f50cc042070 con 0x7f50d41081c0 2024-09-19T08:36:23.878 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.094+0000 7f50db803640 1 -- 172.21.15.164:0/2400291179 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50d4106170 con 0x7f50d41081c0 2024-09-19T08:36:23.878 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.094+0000 7f50d27fc640 1 -- 172.21.15.164:0/2400291179 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50cc03d940 con 0x7f50d41081c0 2024-09-19T08:36:23.878 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.094+0000 7f50db803640 1 -- 172.21.15.164:0/2400291179 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50d41066b0 con 0x7f50d41081c0 2024-09-19T08:36:23.878 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.095+0000 7f50d27fc640 1 -- 172.21.15.164:0/2400291179 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 3) ==== 46344+0+0 (secure 0 0 0) 0x7f50cc03dc40 con 0x7f50d41081c0 2024-09-19T08:36:23.878 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.095+0000 7f50db803640 1 -- 172.21.15.164:0/2400291179 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50d4108620 con 0x7f50d41081c0 2024-09-19T08:36:23.878 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.095+0000 7f50d27fc640 1 --2- 172.21.15.164:0/2400291179 >> [v2:172.21.15.164:6800/2805828009,v1:172.21.15.164:6801/2805828009] conn(0x7f50bc039050 0x7f50bc03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:23.878 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.095+0000 7f50d27fc640 1 -- 172.21.15.164:0/2400291179 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f50cc007e00 con 0x7f50d41081c0 2024-09-19T08:36:23.878 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.095+0000 7f50d8d9e640 1 --2- 172.21.15.164:0/2400291179 >> [v2:172.21.15.164:6800/2805828009,v1:172.21.15.164:6801/2805828009] conn(0x7f50bc039050 0x7f50bc03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:23.878 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.096+0000 7f50d8d9e640 1 --2- 172.21.15.164:0/2400291179 >> [v2:172.21.15.164:6800/2805828009,v1:172.21.15.164:6801/2805828009] conn(0x7f50bc039050 0x7f50bc03b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f50c80098c0 tx=0x7f50c8006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:23.878 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.100+0000 7f50d27fc640 1 -- 172.21.15.164:0/2400291179 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f50cc044310 con 0x7f50d41081c0 2024-09-19T08:36:23.878 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.233+0000 7f50d27fc640 1 -- 172.21.15.164:0/2400291179 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f50cc071070 con 0x7f50d41081c0 2024-09-19T08:36:23.878 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.331+0000 7f50db803640 1 -- 172.21.15.164:0/2400291179 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f50d4106f60 con 0x7f50d41081c0 2024-09-19T08:36:23.879 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.331+0000 7f50d27fc640 1 -- 172.21.15.164:0/2400291179 <== mon.0 v2:172.21.15.164:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7f50cc007310 con 0x7f50d41081c0 2024-09-19T08:36:23.879 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.333+0000 7f50db803640 1 -- 172.21.15.164:0/2400291179 >> [v2:172.21.15.164:6800/2805828009,v1:172.21.15.164:6801/2805828009] conn(0x7f50bc039050 msgr2=0x7f50bc03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:23.879 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.333+0000 7f50db803640 1 --2- 172.21.15.164:0/2400291179 >> [v2:172.21.15.164:6800/2805828009,v1:172.21.15.164:6801/2805828009] conn(0x7f50bc039050 0x7f50bc03b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f50c80098c0 tx=0x7f50c8006db0 comp rx=0 tx=0).stop 2024-09-19T08:36:23.879 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.333+0000 7f50db803640 1 -- 172.21.15.164:0/2400291179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50d41081c0 msgr2=0x7f50d4107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:23.879 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.333+0000 7f50db803640 1 --2- 172.21.15.164:0/2400291179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50d41081c0 0x7f50d4107a40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f50cc031ca0 tx=0x7f50cc031cd0 comp rx=0 tx=0).stop 2024-09-19T08:36:23.879 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.334+0000 7f50db803640 1 -- 172.21.15.164:0/2400291179 shutdown_connections 2024-09-19T08:36:23.879 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.334+0000 7f50db803640 1 --2- 172.21.15.164:0/2400291179 >> [v2:172.21.15.164:6800/2805828009,v1:172.21.15.164:6801/2805828009] conn(0x7f50bc039050 0x7f50bc03b510 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:23.879 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.334+0000 7f50db803640 1 --2- 172.21.15.164:0/2400291179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50d41081c0 0x7f50d4107a40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:23.879 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.334+0000 7f50db803640 1 -- 172.21.15.164:0/2400291179 >> 172.21.15.164:0/2400291179 conn(0x7f50d4101e20 msgr2=0x7f50d4102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:23.879 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.334+0000 7f50db803640 1 -- 172.21.15.164:0/2400291179 shutdown_connections 2024-09-19T08:36:23.879 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:23.334+0000 7f50db803640 1 -- 172.21.15.164:0/2400291179 wait complete. 2024-09-19T08:36:23.879 INFO:teuthology.orchestra.run.smithi164.stdout:mgr is available 2024-09-19T08:36:24.451 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:24 smithi164 ceph-mon[30928]: mgrmap e4: x(active, since 2s) 2024-09-19T08:36:24.451 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:24 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2400291179' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T08:36:25.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:25 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/1353857705' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T08:36:25.863 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout 2024-09-19T08:36:25.863 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T08:36:25.863 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout fsid = 1e4b0d74-7662-11ef-bceb-c7b262605968 2024-09-19T08:36:25.863 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.164:3300,v1:172.21.15.164:6789] 2024-09-19T08:36:25.863 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T08:36:25.864 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T08:36:25.864 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T08:36:25.864 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T08:36:25.864 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout 2024-09-19T08:36:25.864 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T08:36:25.864 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T08:36:25.864 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout 2024-09-19T08:36:25.864 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T08:36:25.864 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T08:36:25.864 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T08:36:25.864 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T08:36:25.864 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T08:36:25.865 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T08:36:25.865 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.276+0000 7f4355389640 1 Processor -- start 2024-09-19T08:36:25.865 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.276+0000 7f4355389640 1 -- start start 2024-09-19T08:36:25.865 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.276+0000 7f4355389640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43501081e0 0x7f435010a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:25.865 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.276+0000 7f4355389640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f435007be50 con 0x7f43501081e0 2024-09-19T08:36:25.865 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.276+0000 7f434effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43501081e0 0x7f435010a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:25.865 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.276+0000 7f434effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43501081e0 0x7f435010a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51590/0 (socket says 172.21.15.164:51590) 2024-09-19T08:36:25.865 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.276+0000 7f434effd640 1 -- 172.21.15.164:0/3558943393 learned_addr learned my addr 172.21.15.164:0/3558943393 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:25.866 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.277+0000 7f434effd640 1 -- 172.21.15.164:0/3558943393 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f435010ab10 con 0x7f43501081e0 2024-09-19T08:36:25.866 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.277+0000 7f434effd640 1 --2- 172.21.15.164:0/3558943393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43501081e0 0x7f435010a5d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4340009820 tx=0x7f434002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac0ae81b5021665 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:25.866 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.277+0000 7f434dffb640 1 -- 172.21.15.164:0/3558943393 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4340037070 con 0x7f43501081e0 2024-09-19T08:36:25.866 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.278+0000 7f434dffb640 1 -- 172.21.15.164:0/3558943393 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4340032900 con 0x7f43501081e0 2024-09-19T08:36:25.866 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.278+0000 7f434dffb640 1 -- 172.21.15.164:0/3558943393 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4340030850 con 0x7f43501081e0 2024-09-19T08:36:25.866 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.278+0000 7f4355389640 1 -- 172.21.15.164:0/3558943393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43501081e0 msgr2=0x7f435010a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:25.866 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.278+0000 7f4355389640 1 --2- 172.21.15.164:0/3558943393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43501081e0 0x7f435010a5d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4340009820 tx=0x7f434002a270 comp rx=0 tx=0).stop 2024-09-19T08:36:25.866 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.278+0000 7f4355389640 1 -- 172.21.15.164:0/3558943393 shutdown_connections 2024-09-19T08:36:25.866 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.278+0000 7f4355389640 1 --2- 172.21.15.164:0/3558943393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43501081e0 0x7f435010a5d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:25.866 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.278+0000 7f4355389640 1 -- 172.21.15.164:0/3558943393 >> 172.21.15.164:0/3558943393 conn(0x7f4350101da0 msgr2=0x7f43501041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:25.867 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.278+0000 7f4355389640 1 -- 172.21.15.164:0/3558943393 shutdown_connections 2024-09-19T08:36:25.867 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.278+0000 7f4355389640 1 -- 172.21.15.164:0/3558943393 wait complete. 2024-09-19T08:36:25.867 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.279+0000 7f4355389640 1 Processor -- start 2024-09-19T08:36:25.867 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.279+0000 7f4355389640 1 -- start start 2024-09-19T08:36:25.867 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.279+0000 7f4355389640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43501081e0 0x7f435019b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:25.867 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.279+0000 7f4355389640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f435010bb50 con 0x7f43501081e0 2024-09-19T08:36:25.867 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.279+0000 7f434effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43501081e0 0x7f435019b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:25.867 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.279+0000 7f434effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43501081e0 0x7f435019b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51598/0 (socket says 172.21.15.164:51598) 2024-09-19T08:36:25.868 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.279+0000 7f434effd640 1 -- 172.21.15.164:0/1353857705 learned_addr learned my addr 172.21.15.164:0/1353857705 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:25.868 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.279+0000 7f434effd640 1 -- 172.21.15.164:0/1353857705 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f435019bc50 con 0x7f43501081e0 2024-09-19T08:36:25.868 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.280+0000 7f434effd640 1 --2- 172.21.15.164:0/1353857705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43501081e0 0x7f435019b710 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4340031000 tx=0x7f43400326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:25.868 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.280+0000 7f4337fff640 1 -- 172.21.15.164:0/1353857705 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4340042070 con 0x7f43501081e0 2024-09-19T08:36:25.868 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.280+0000 7f4355389640 1 -- 172.21.15.164:0/1353857705 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f435019bee0 con 0x7f43501081e0 2024-09-19T08:36:25.868 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.280+0000 7f4355389640 1 -- 172.21.15.164:0/1353857705 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f435019ebd0 con 0x7f43501081e0 2024-09-19T08:36:25.868 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.280+0000 7f4337fff640 1 -- 172.21.15.164:0/1353857705 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f43400328a0 con 0x7f43501081e0 2024-09-19T08:36:25.868 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.280+0000 7f4337fff640 1 -- 172.21.15.164:0/1353857705 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4340037040 con 0x7f43501081e0 2024-09-19T08:36:25.869 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.281+0000 7f4337fff640 1 -- 172.21.15.164:0/1353857705 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f4340045070 con 0x7f43501081e0 2024-09-19T08:36:25.869 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.281+0000 7f4355389640 1 -- 172.21.15.164:0/1353857705 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f433c0052c0 con 0x7f43501081e0 2024-09-19T08:36:25.869 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.281+0000 7f4337fff640 1 --2- 172.21.15.164:0/1353857705 >> [v2:172.21.15.164:6800/2805828009,v1:172.21.15.164:6801/2805828009] conn(0x7f4338039160 0x7f433803b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:25.869 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.281+0000 7f4337fff640 1 -- 172.21.15.164:0/1353857705 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f434006c1e0 con 0x7f43501081e0 2024-09-19T08:36:25.869 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.281+0000 7f434e7fc640 1 --2- 172.21.15.164:0/1353857705 >> [v2:172.21.15.164:6800/2805828009,v1:172.21.15.164:6801/2805828009] conn(0x7f4338039160 0x7f433803b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:25.869 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.281+0000 7f434e7fc640 1 --2- 172.21.15.164:0/1353857705 >> [v2:172.21.15.164:6800/2805828009,v1:172.21.15.164:6801/2805828009] conn(0x7f4338039160 0x7f433803b620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f43440098c0 tx=0x7f4344006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:25.869 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.286+0000 7f4337fff640 1 -- 172.21.15.164:0/1353857705 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f434003be90 con 0x7f43501081e0 2024-09-19T08:36:25.870 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.451+0000 7f4355389640 1 -- 172.21.15.164:0/1353857705 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f433c002420 con 0x7f43501081e0 2024-09-19T08:36:25.870 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.452+0000 7f4337fff640 1 -- 172.21.15.164:0/1353857705 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+431 (secure 0 0 0) 0x7f4340039580 con 0x7f43501081e0 2024-09-19T08:36:25.870 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.454+0000 7f4355389640 1 -- 172.21.15.164:0/1353857705 >> [v2:172.21.15.164:6800/2805828009,v1:172.21.15.164:6801/2805828009] conn(0x7f4338039160 msgr2=0x7f433803b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:25.870 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.454+0000 7f4355389640 1 --2- 172.21.15.164:0/1353857705 >> [v2:172.21.15.164:6800/2805828009,v1:172.21.15.164:6801/2805828009] conn(0x7f4338039160 0x7f433803b620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f43440098c0 tx=0x7f4344006db0 comp rx=0 tx=0).stop 2024-09-19T08:36:25.870 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.454+0000 7f4355389640 1 -- 172.21.15.164:0/1353857705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43501081e0 msgr2=0x7f435019b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:25.870 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.454+0000 7f4355389640 1 --2- 172.21.15.164:0/1353857705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43501081e0 0x7f435019b710 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4340031000 tx=0x7f43400326e0 comp rx=0 tx=0).stop 2024-09-19T08:36:25.870 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.454+0000 7f4355389640 1 -- 172.21.15.164:0/1353857705 shutdown_connections 2024-09-19T08:36:25.870 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.455+0000 7f4355389640 1 --2- 172.21.15.164:0/1353857705 >> [v2:172.21.15.164:6800/2805828009,v1:172.21.15.164:6801/2805828009] conn(0x7f4338039160 0x7f433803b620 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:25.870 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.455+0000 7f4355389640 1 --2- 172.21.15.164:0/1353857705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43501081e0 0x7f435019b710 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:25.870 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.455+0000 7f4355389640 1 -- 172.21.15.164:0/1353857705 >> 172.21.15.164:0/1353857705 conn(0x7f4350101da0 msgr2=0x7f4350102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:25.871 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.455+0000 7f4355389640 1 -- 172.21.15.164:0/1353857705 shutdown_connections 2024-09-19T08:36:25.871 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:24.455+0000 7f4355389640 1 -- 172.21.15.164:0/1353857705 wait complete. 2024-09-19T08:36:25.871 INFO:teuthology.orchestra.run.smithi164.stdout:Enabling cephadm module... 2024-09-19T08:36:29.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:29 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/826158496' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-19T08:36:29.282 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:29 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: ignoring --setuser ceph since I am not root 2024-09-19T08:36:29.282 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:29 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: ignoring --setgroup ceph since I am not root 2024-09-19T08:36:29.282 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:29 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:29.140+0000 7fdb8e820040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T08:36:29.282 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:29 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:29.196+0000 7fdb8e820040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T08:36:29.697 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:29 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:29.344+0000 7fdb8e820040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T08:36:29.697 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:29 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:29.410+0000 7fdb8e820040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T08:36:29.697 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:29 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:29.556+0000 7fdb8e820040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:36:29.697 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:29 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:29.697+0000 7fdb8e820040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T08:36:30.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:30 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/826158496' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-19T08:36:30.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:30 smithi164 ceph-mon[30928]: mgrmap e5: x(active, since 7s) 2024-09-19T08:36:30.282 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:30 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:30.009+0000 7fdb8e820040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T08:36:30.282 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:30 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:30.070+0000 7fdb8e820040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T08:36:30.282 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:30 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:30.255+0000 7fdb8e820040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T08:36:31.308 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:31 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:31.143+0000 7fdb8e820040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T08:36:31.308 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:31 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:31.213+0000 7fdb8e820040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T08:36:32.223 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:31 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:31.956+0000 7fdb8e820040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T08:36:32.223 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:32 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:32.076+0000 7fdb8e820040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T08:36:32.507 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:32 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:32.223+0000 7fdb8e820040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:36:32.508 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:32 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:32.286+0000 7fdb8e820040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T08:36:32.508 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:32 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:32.439+0000 7fdb8e820040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T08:36:32.508 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:32 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:32.508+0000 7fdb8e820040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T08:36:32.782 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:32 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:32.568+0000 7fdb8e820040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T08:36:32.782 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:32 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:32.727+0000 7fdb8e820040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T08:36:33.061 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:32 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:32.791+0000 7fdb8e820040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T08:36:33.061 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:32 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:32.859+0000 7fdb8e820040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T08:36:33.061 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:32 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:32.942+0000 7fdb8e820040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T08:36:33.061 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:33 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:33.001+0000 7fdb8e820040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T08:36:33.061 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:33 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:33.061+0000 7fdb8e820040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T08:36:34.282 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:33 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:33.855+0000 7fdb8e820040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T08:36:34.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:33 smithi164 ceph-mon[30928]: Active manager daemon x restarted 2024-09-19T08:36:34.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:33 smithi164 ceph-mon[30928]: Activating manager daemon x 2024-09-19T08:36:34.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:33 smithi164 ceph-mon[30928]: osdmap e2: 0 total, 0 up, 0 in 2024-09-19T08:36:34.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:33 smithi164 ceph-mon[30928]: mgrmap e6: x(active, starting, since 0.00328927s) 2024-09-19T08:36:34.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:33 smithi164 ceph-mon[30928]: from='mgr.14118 172.21.15.164:0/2031629681' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:36:34.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:33 smithi164 ceph-mon[30928]: from='mgr.14118 172.21.15.164:0/2031629681' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T08:36:34.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:33 smithi164 ceph-mon[30928]: from='mgr.14118 172.21.15.164:0/2031629681' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T08:36:34.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:33 smithi164 ceph-mon[30928]: from='mgr.14118 172.21.15.164:0/2031629681' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T08:36:34.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:33 smithi164 ceph-mon[30928]: from='mgr.14118 172.21.15.164:0/2031629681' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T08:36:34.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:33 smithi164 ceph-mon[30928]: Manager daemon x is now available 2024-09-19T08:36:34.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:33 smithi164 ceph-mon[30928]: from='mgr.14118 172.21.15.164:0/2031629681' entity='mgr.x' 2024-09-19T08:36:34.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:33 smithi164 ceph-mon[30928]: from='mgr.14118 172.21.15.164:0/2031629681' entity='mgr.x' 2024-09-19T08:36:34.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:33 smithi164 ceph-mon[30928]: from='mgr.14118 172.21.15.164:0/2031629681' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:36:34.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:33 smithi164 ceph-mon[30928]: from='mgr.14118 172.21.15.164:0/2031629681' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:36:34.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:33 smithi164 ceph-mon[30928]: from='mgr.14118 172.21.15.164:0/2031629681' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T08:36:34.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:33 smithi164 ceph-mon[30928]: from='mgr.14118 172.21.15.164:0/2031629681' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T08:36:35.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:34 smithi164 ceph-mon[30928]: Found migration_current of "None". Setting to last migration. 2024-09-19T08:36:35.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:34 smithi164 ceph-mon[30928]: from='mgr.14118 172.21.15.164:0/2031629681' entity='mgr.x' 2024-09-19T08:36:35.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:34 smithi164 ceph-mon[30928]: from='mgr.14118 172.21.15.164:0/2031629681' entity='mgr.x' 2024-09-19T08:36:35.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:34 smithi164 ceph-mon[30928]: mgrmap e7: x(active, since 1.00508s) 2024-09-19T08:36:35.494 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.728+0000 7f74882e8640 1 Processor -- start 2024-09-19T08:36:35.494 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.728+0000 7f74882e8640 1 -- start start 2024-09-19T08:36:35.494 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.728+0000 7f74882e8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f748007de70 0x7f748007c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:35.494 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.728+0000 7f74882e8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f748007c8f0 con 0x7f748007de70 2024-09-19T08:36:35.495 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.728+0000 7f7486084640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f748007de70 0x7f748007c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:35.495 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.728+0000 7f7486084640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f748007de70 0x7f748007c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51606/0 (socket says 172.21.15.164:51606) 2024-09-19T08:36:35.495 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.728+0000 7f7486084640 1 -- 172.21.15.164:0/615125290 learned_addr learned my addr 172.21.15.164:0/615125290 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:35.495 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.729+0000 7f7486084640 1 -- 172.21.15.164:0/615125290 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f748007ca70 con 0x7f748007de70 2024-09-19T08:36:35.495 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.729+0000 7f7486084640 1 --2- 172.21.15.164:0/615125290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f748007de70 0x7f748007c320 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7474009820 tx=0x7f747402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e623b3e2cb68ccc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:35.495 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.730+0000 7f7485082640 1 -- 172.21.15.164:0/615125290 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7474037070 con 0x7f748007de70 2024-09-19T08:36:35.495 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.730+0000 7f7485082640 1 -- 172.21.15.164:0/615125290 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7474032900 con 0x7f748007de70 2024-09-19T08:36:35.495 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.730+0000 7f7485082640 1 -- 172.21.15.164:0/615125290 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7474030850 con 0x7f748007de70 2024-09-19T08:36:35.495 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.730+0000 7f74882e8640 1 -- 172.21.15.164:0/615125290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f748007de70 msgr2=0x7f748007c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:35.495 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.730+0000 7f74882e8640 1 --2- 172.21.15.164:0/615125290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f748007de70 0x7f748007c320 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7474009820 tx=0x7f747402a270 comp rx=0 tx=0).stop 2024-09-19T08:36:35.495 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.730+0000 7f74882e8640 1 -- 172.21.15.164:0/615125290 shutdown_connections 2024-09-19T08:36:35.496 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.730+0000 7f74882e8640 1 --2- 172.21.15.164:0/615125290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f748007de70 0x7f748007c320 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:35.496 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.730+0000 7f74882e8640 1 -- 172.21.15.164:0/615125290 >> 172.21.15.164:0/615125290 conn(0x7f7480102ef0 msgr2=0x7f7480105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:35.496 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.730+0000 7f74882e8640 1 -- 172.21.15.164:0/615125290 shutdown_connections 2024-09-19T08:36:35.496 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.730+0000 7f74882e8640 1 -- 172.21.15.164:0/615125290 wait complete. 2024-09-19T08:36:35.496 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.731+0000 7f74882e8640 1 Processor -- start 2024-09-19T08:36:35.496 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.731+0000 7f74882e8640 1 -- start start 2024-09-19T08:36:35.496 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.731+0000 7f74882e8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f748007de70 0x7f748019d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:35.496 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.731+0000 7f74882e8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f748010dd10 con 0x7f748007de70 2024-09-19T08:36:35.496 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.731+0000 7f7486084640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f748007de70 0x7f748019d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:35.496 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.731+0000 7f7486084640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f748007de70 0x7f748019d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51618/0 (socket says 172.21.15.164:51618) 2024-09-19T08:36:35.496 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.731+0000 7f7486084640 1 -- 172.21.15.164:0/826158496 learned_addr learned my addr 172.21.15.164:0/826158496 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:35.496 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.732+0000 7f7486084640 1 -- 172.21.15.164:0/826158496 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f748019dd90 con 0x7f748007de70 2024-09-19T08:36:35.497 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.732+0000 7f7486084640 1 --2- 172.21.15.164:0/826158496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f748007de70 0x7f748019d850 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7474032ce0 tx=0x7f7474032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:35.497 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.732+0000 7f74737fe640 1 -- 172.21.15.164:0/826158496 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7474042070 con 0x7f748007de70 2024-09-19T08:36:35.497 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.732+0000 7f74737fe640 1 -- 172.21.15.164:0/826158496 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f74740308c0 con 0x7f748007de70 2024-09-19T08:36:35.497 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.732+0000 7f74737fe640 1 -- 172.21.15.164:0/826158496 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7474037040 con 0x7f748007de70 2024-09-19T08:36:35.497 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.732+0000 7f74882e8640 1 -- 172.21.15.164:0/826158496 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f748019e020 con 0x7f748007de70 2024-09-19T08:36:35.497 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.732+0000 7f74882e8640 1 -- 172.21.15.164:0/826158496 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74801a0d10 con 0x7f748007de70 2024-09-19T08:36:35.497 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.733+0000 7f74737fe640 1 -- 172.21.15.164:0/826158496 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f7474045070 con 0x7f748007de70 2024-09-19T08:36:35.497 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.733+0000 7f74737fe640 1 --2- 172.21.15.164:0/826158496 >> [v2:172.21.15.164:6800/2805828009,v1:172.21.15.164:6801/2805828009] conn(0x7f746c0391b0 0x7f746c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:35.497 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.733+0000 7f74882e8640 1 -- 172.21.15.164:0/826158496 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f748007c3a0 con 0x7f748007de70 2024-09-19T08:36:35.497 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.733+0000 7f74737fe640 1 -- 172.21.15.164:0/826158496 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f747406c1d0 con 0x7f748007de70 2024-09-19T08:36:35.497 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.733+0000 7f7485883640 1 --2- 172.21.15.164:0/826158496 >> [v2:172.21.15.164:6800/2805828009,v1:172.21.15.164:6801/2805828009] conn(0x7f746c0391b0 0x7f746c03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:35.498 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.734+0000 7f7485883640 1 --2- 172.21.15.164:0/826158496 >> [v2:172.21.15.164:6800/2805828009,v1:172.21.15.164:6801/2805828009] conn(0x7f746c0391b0 0x7f746c03b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f747c0098c0 tx=0x7f747c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:35.498 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.738+0000 7f74737fe640 1 -- 172.21.15.164:0/826158496 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f747404b170 con 0x7f748007de70 2024-09-19T08:36:35.498 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:28.950+0000 7f74882e8640 1 -- 172.21.15.164:0/826158496 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f74801a1360 con 0x7f748007de70 2024-09-19T08:36:35.498 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:29.001+0000 7f74737fe640 1 -- 172.21.15.164:0/826158496 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f747403ccc0 con 0x7f748007de70 2024-09-19T08:36:35.498 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:29.003+0000 7f74737fe640 1 -- 172.21.15.164:0/826158496 <== mon.0 v2:172.21.15.164:3300/0 8 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f747403c740 con 0x7f748007de70 2024-09-19T08:36:35.498 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:29.004+0000 7f74882e8640 1 -- 172.21.15.164:0/826158496 >> [v2:172.21.15.164:6800/2805828009,v1:172.21.15.164:6801/2805828009] conn(0x7f746c0391b0 msgr2=0x7f746c03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:35.498 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:29.004+0000 7f74882e8640 1 --2- 172.21.15.164:0/826158496 >> [v2:172.21.15.164:6800/2805828009,v1:172.21.15.164:6801/2805828009] conn(0x7f746c0391b0 0x7f746c03b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f747c0098c0 tx=0x7f747c006db0 comp rx=0 tx=0).stop 2024-09-19T08:36:35.498 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:29.004+0000 7f74882e8640 1 -- 172.21.15.164:0/826158496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f748007de70 msgr2=0x7f748019d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:35.498 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:29.004+0000 7f74882e8640 1 --2- 172.21.15.164:0/826158496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f748007de70 0x7f748019d850 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7474032ce0 tx=0x7f7474032d10 comp rx=0 tx=0).stop 2024-09-19T08:36:35.498 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:29.004+0000 7f74882e8640 1 -- 172.21.15.164:0/826158496 shutdown_connections 2024-09-19T08:36:35.498 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:29.004+0000 7f74882e8640 1 --2- 172.21.15.164:0/826158496 >> [v2:172.21.15.164:6800/2805828009,v1:172.21.15.164:6801/2805828009] conn(0x7f746c0391b0 0x7f746c03b670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:35.498 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:29.004+0000 7f74882e8640 1 --2- 172.21.15.164:0/826158496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f748007de70 0x7f748019d850 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:35.499 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:29.004+0000 7f74882e8640 1 -- 172.21.15.164:0/826158496 >> 172.21.15.164:0/826158496 conn(0x7f7480102ef0 msgr2=0x7f7480103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:35.499 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:29.004+0000 7f74882e8640 1 -- 172.21.15.164:0/826158496 shutdown_connections 2024-09-19T08:36:35.499 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:29.004+0000 7f74882e8640 1 -- 172.21.15.164:0/826158496 wait complete. 2024-09-19T08:36:35.782 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:35 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:35.644+0000 7fdaf5b7a640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:08:36:35] ENGINE Error in HTTPServer.serve 2024-09-19T08:36:35.783 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:35 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: Traceback (most recent call last): 2024-09-19T08:36:35.783 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:35 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T08:36:35.783 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:35 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: self._connections.run(self.expiration_interval) 2024-09-19T08:36:35.783 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:35 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T08:36:35.783 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:35 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: self._run(expiration_interval) 2024-09-19T08:36:35.783 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:35 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T08:36:35.783 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:35 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T08:36:35.783 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:35 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T08:36:35.783 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:35 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T08:36:35.783 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:35 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T08:36:35.784 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:35 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: s = self.context.wrap_socket( 2024-09-19T08:36:35.784 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:35 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T08:36:35.784 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:35 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: return self.sslsocket_class._create( 2024-09-19T08:36:35.784 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:35 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T08:36:35.784 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:35 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: self.do_handshake() 2024-09-19T08:36:35.784 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:35 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T08:36:35.784 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:35 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: self._sslobj.do_handshake() 2024-09-19T08:36:35.784 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:35 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T08:36:35.784 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:35 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:36.281 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:35 smithi164 ceph-mon[30928]: from='mgr.14118 172.21.15.164:0/2031629681' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:36:37.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:36 smithi164 ceph-mon[30928]: [19/Sep/2024:08:36:35] ENGINE Bus STARTING 2024-09-19T08:36:37.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:36 smithi164 ceph-mon[30928]: [19/Sep/2024:08:36:35] ENGINE Serving on http://172.21.15.164:8765 2024-09-19T08:36:37.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:36 smithi164 ceph-mon[30928]: [19/Sep/2024:08:36:35] ENGINE Serving on https://172.21.15.164:7150 2024-09-19T08:36:37.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:36 smithi164 ceph-mon[30928]: [19/Sep/2024:08:36:35] ENGINE Bus STARTED 2024-09-19T08:36:37.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:36 smithi164 ceph-mon[30928]: [19/Sep/2024:08:36:35] ENGINE Error in HTTPServer.serve 2024-09-19T08:36:37.282 INFO:journalctl@ceph.mon.a.smithi164.stdout: Traceback (most recent call last): 2024-09-19T08:36:37.283 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T08:36:37.283 INFO:journalctl@ceph.mon.a.smithi164.stdout: self._connections.run(self.expiration_interval) 2024-09-19T08:36:37.283 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T08:36:37.283 INFO:journalctl@ceph.mon.a.smithi164.stdout: self._run(expiration_interval) 2024-09-19T08:36:37.283 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T08:36:37.283 INFO:journalctl@ceph.mon.a.smithi164.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T08:36:37.283 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T08:36:37.283 INFO:journalctl@ceph.mon.a.smithi164.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T08:36:37.284 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T08:36:37.284 INFO:journalctl@ceph.mon.a.smithi164.stdout: s = self.context.wrap_socket( 2024-09-19T08:36:37.284 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T08:36:37.284 INFO:journalctl@ceph.mon.a.smithi164.stdout: return self.sslsocket_class._create( 2024-09-19T08:36:37.284 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T08:36:37.284 INFO:journalctl@ceph.mon.a.smithi164.stdout: self.do_handshake() 2024-09-19T08:36:37.284 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T08:36:37.284 INFO:journalctl@ceph.mon.a.smithi164.stdout: self._sslobj.do_handshake() 2024-09-19T08:36:37.284 INFO:journalctl@ceph.mon.a.smithi164.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T08:36:37.284 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:36 smithi164 ceph-mon[30928]: mgrmap e8: x(active, since 2s) 2024-09-19T08:36:38.107 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:37 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3016145919' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T08:36:38.400 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout { 2024-09-19T08:36:38.400 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "epoch": 8, 2024-09-19T08:36:38.400 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T08:36:38.400 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T08:36:38.400 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T08:36:38.400 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout } 2024-09-19T08:36:38.400 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.609+0000 7ff640e8f640 1 Processor -- start 2024-09-19T08:36:38.400 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.609+0000 7ff640e8f640 1 -- start start 2024-09-19T08:36:38.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.609+0000 7ff640e8f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff63c107720 0x7ff63c107b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:38.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.609+0000 7ff640e8f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c1080f0 con 0x7ff63c107720 2024-09-19T08:36:38.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.610+0000 7ff63a59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff63c107720 0x7ff63c107b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:38.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.610+0000 7ff63a59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff63c107720 0x7ff63c107b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39612/0 (socket says 172.21.15.164:39612) 2024-09-19T08:36:38.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.610+0000 7ff63a59c640 1 -- 172.21.15.164:0/2980173643 learned_addr learned my addr 172.21.15.164:0/2980173643 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:38.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.610+0000 7ff63a59c640 1 -- 172.21.15.164:0/2980173643 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63c1088d0 con 0x7ff63c107720 2024-09-19T08:36:38.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.611+0000 7ff63a59c640 1 --2- 172.21.15.164:0/2980173643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff63c107720 0x7ff63c107b20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009820 tx=0x7ff62c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1903f5a24471bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:38.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.611+0000 7ff639d9b640 1 -- 172.21.15.164:0/2980173643 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff62c037070 con 0x7ff63c107720 2024-09-19T08:36:38.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.611+0000 7ff639d9b640 1 -- 172.21.15.164:0/2980173643 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff62c032900 con 0x7ff63c107720 2024-09-19T08:36:38.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.611+0000 7ff639d9b640 1 -- 172.21.15.164:0/2980173643 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff62c030850 con 0x7ff63c107720 2024-09-19T08:36:38.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.612+0000 7ff640e8f640 1 -- 172.21.15.164:0/2980173643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff63c107720 msgr2=0x7ff63c107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:38.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.612+0000 7ff640e8f640 1 --2- 172.21.15.164:0/2980173643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff63c107720 0x7ff63c107b20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009820 tx=0x7ff62c02a270 comp rx=0 tx=0).stop 2024-09-19T08:36:38.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.612+0000 7ff640e8f640 1 -- 172.21.15.164:0/2980173643 shutdown_connections 2024-09-19T08:36:38.402 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.612+0000 7ff640e8f640 1 --2- 172.21.15.164:0/2980173643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff63c107720 0x7ff63c107b20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:38.402 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.612+0000 7ff640e8f640 1 -- 172.21.15.164:0/2980173643 >> 172.21.15.164:0/2980173643 conn(0x7ff63c102ef0 msgr2=0x7ff63c105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:38.402 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.612+0000 7ff640e8f640 1 -- 172.21.15.164:0/2980173643 shutdown_connections 2024-09-19T08:36:38.402 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.612+0000 7ff640e8f640 1 -- 172.21.15.164:0/2980173643 wait complete. 2024-09-19T08:36:38.402 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.613+0000 7ff640e8f640 1 Processor -- start 2024-09-19T08:36:38.402 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.613+0000 7ff640e8f640 1 -- start start 2024-09-19T08:36:38.402 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.613+0000 7ff640e8f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff63c107720 0x7ff63c19b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:38.402 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.613+0000 7ff640e8f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c10bc80 con 0x7ff63c107720 2024-09-19T08:36:38.402 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.613+0000 7ff63a59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff63c107720 0x7ff63c19b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:38.402 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.613+0000 7ff63a59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff63c107720 0x7ff63c19b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39624/0 (socket says 172.21.15.164:39624) 2024-09-19T08:36:38.402 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.613+0000 7ff63a59c640 1 -- 172.21.15.164:0/3016145919 learned_addr learned my addr 172.21.15.164:0/3016145919 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:38.402 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.614+0000 7ff63a59c640 1 -- 172.21.15.164:0/3016145919 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63c19bc00 con 0x7ff63c107720 2024-09-19T08:36:38.403 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.614+0000 7ff63a59c640 1 --2- 172.21.15.164:0/3016145919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff63c107720 0x7ff63c19b6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009940 tx=0x7ff62c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:38.403 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.614+0000 7ff633fff640 1 -- 172.21.15.164:0/3016145919 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff62c042070 con 0x7ff63c107720 2024-09-19T08:36:38.403 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.614+0000 7ff633fff640 1 -- 172.21.15.164:0/3016145919 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff62c0308c0 con 0x7ff63c107720 2024-09-19T08:36:38.403 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.614+0000 7ff640e8f640 1 -- 172.21.15.164:0/3016145919 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff63c19be90 con 0x7ff63c107720 2024-09-19T08:36:38.403 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.614+0000 7ff640e8f640 1 -- 172.21.15.164:0/3016145919 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff63c19eb80 con 0x7ff63c107720 2024-09-19T08:36:38.403 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.614+0000 7ff633fff640 1 -- 172.21.15.164:0/3016145919 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff62c037040 con 0x7ff63c107720 2024-09-19T08:36:38.403 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.615+0000 7ff633fff640 1 -- 172.21.15.164:0/3016145919 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7ff62c045070 con 0x7ff63c107720 2024-09-19T08:36:38.403 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.615+0000 7ff640e8f640 1 -- 172.21.15.164:0/3016145919 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff61c0052c0 con 0x7ff63c107720 2024-09-19T08:36:38.403 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.615+0000 7ff633fff640 1 --2- 172.21.15.164:0/3016145919 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7ff6280391b0 0x7ff62803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:38.403 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.615+0000 7ff633fff640 1 -- 172.21.15.164:0/3016145919 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff62c06c2f0 con 0x7ff63c107720 2024-09-19T08:36:38.403 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.615+0000 7ff631d9b640 1 --2- 172.21.15.164:0/3016145919 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7ff6280391b0 0x7ff62803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:38.403 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.616+0000 7ff631d9b640 1 --2- 172.21.15.164:0/3016145919 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7ff6280391b0 0x7ff62803b670 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff6240098c0 tx=0x7ff624006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:38.404 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.620+0000 7ff633fff640 1 -- 172.21.15.164:0/3016145919 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff62c03cdb0 con 0x7ff63c107720 2024-09-19T08:36:38.404 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.835+0000 7ff640e8f640 1 -- 172.21.15.164:0/3016145919 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7ff61c005dc0 con 0x7ff63c107720 2024-09-19T08:36:38.404 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.835+0000 7ff633fff640 1 -- 172.21.15.164:0/3016145919 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v8) ==== 56+0+88 (secure 0 0 0) 0x7ff62c03c820 con 0x7ff63c107720 2024-09-19T08:36:38.404 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.837+0000 7ff640e8f640 1 -- 172.21.15.164:0/3016145919 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7ff6280391b0 msgr2=0x7ff62803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:38.404 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.837+0000 7ff640e8f640 1 --2- 172.21.15.164:0/3016145919 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7ff6280391b0 0x7ff62803b670 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff6240098c0 tx=0x7ff624006db0 comp rx=0 tx=0).stop 2024-09-19T08:36:38.404 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.837+0000 7ff640e8f640 1 -- 172.21.15.164:0/3016145919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff63c107720 msgr2=0x7ff63c19b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:38.404 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.837+0000 7ff640e8f640 1 --2- 172.21.15.164:0/3016145919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff63c107720 0x7ff63c19b6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009940 tx=0x7ff62c032ce0 comp rx=0 tx=0).stop 2024-09-19T08:36:38.404 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.837+0000 7ff640e8f640 1 -- 172.21.15.164:0/3016145919 shutdown_connections 2024-09-19T08:36:38.404 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.837+0000 7ff640e8f640 1 --2- 172.21.15.164:0/3016145919 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7ff6280391b0 0x7ff62803b670 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:38.404 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.837+0000 7ff640e8f640 1 --2- 172.21.15.164:0/3016145919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff63c107720 0x7ff63c19b6c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:38.404 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.837+0000 7ff640e8f640 1 -- 172.21.15.164:0/3016145919 >> 172.21.15.164:0/3016145919 conn(0x7ff63c102ef0 msgr2=0x7ff63c103a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:38.404 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.837+0000 7ff640e8f640 1 -- 172.21.15.164:0/3016145919 shutdown_connections 2024-09-19T08:36:38.405 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:37.837+0000 7ff640e8f640 1 -- 172.21.15.164:0/3016145919 wait complete. 2024-09-19T08:36:38.405 INFO:teuthology.orchestra.run.smithi164.stdout:Waiting for the mgr to restart... 2024-09-19T08:36:38.405 INFO:teuthology.orchestra.run.smithi164.stdout:Waiting for mgr epoch 8... 2024-09-19T08:36:39.323 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:39 smithi164 ceph-mon[30928]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-19T08:36:39.323 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:39 smithi164 ceph-mon[30928]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-19T08:36:39.330 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout { 2024-09-19T08:36:39.330 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 8, 2024-09-19T08:36:39.330 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T08:36:39.330 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout } 2024-09-19T08:36:39.330 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.754+0000 7f52c7259640 1 Processor -- start 2024-09-19T08:36:39.330 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.754+0000 7f52c7259640 1 -- start start 2024-09-19T08:36:39.330 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.754+0000 7f52c7259640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52c0107740 0x7f52c0107b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:39.331 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.755+0000 7f52c7259640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f52c0108080 con 0x7f52c0107740 2024-09-19T08:36:39.331 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.755+0000 7f52c4ff5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52c0107740 0x7f52c0107b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:39.331 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.755+0000 7f52c4ff5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52c0107740 0x7f52c0107b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39638/0 (socket says 172.21.15.164:39638) 2024-09-19T08:36:39.331 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.755+0000 7f52c4ff5640 1 -- 172.21.15.164:0/3460223536 learned_addr learned my addr 172.21.15.164:0/3460223536 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:39.331 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.755+0000 7f52c4ff5640 1 -- 172.21.15.164:0/3460223536 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c0108200 con 0x7f52c0107740 2024-09-19T08:36:39.331 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.756+0000 7f52c4ff5640 1 --2- 172.21.15.164:0/3460223536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52c0107740 0x7f52c0107b40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009820 tx=0x7f52b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6873575bb70cc655 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:39.331 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.756+0000 7f52bf7fe640 1 -- 172.21.15.164:0/3460223536 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52b0037070 con 0x7f52c0107740 2024-09-19T08:36:39.331 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.756+0000 7f52bf7fe640 1 -- 172.21.15.164:0/3460223536 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f52b0032900 con 0x7f52c0107740 2024-09-19T08:36:39.331 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.756+0000 7f52bf7fe640 1 -- 172.21.15.164:0/3460223536 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52b0030850 con 0x7f52c0107740 2024-09-19T08:36:39.331 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.756+0000 7f52c7259640 1 -- 172.21.15.164:0/3460223536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52c0107740 msgr2=0x7f52c0107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:39.331 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.756+0000 7f52c7259640 1 --2- 172.21.15.164:0/3460223536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52c0107740 0x7f52c0107b40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009820 tx=0x7f52b002a270 comp rx=0 tx=0).stop 2024-09-19T08:36:39.331 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.756+0000 7f52c7259640 1 -- 172.21.15.164:0/3460223536 shutdown_connections 2024-09-19T08:36:39.331 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.756+0000 7f52c7259640 1 --2- 172.21.15.164:0/3460223536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52c0107740 0x7f52c0107b40 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:39.332 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.756+0000 7f52c7259640 1 -- 172.21.15.164:0/3460223536 >> 172.21.15.164:0/3460223536 conn(0x7f52c0102f50 msgr2=0x7f52c01053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:39.332 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.756+0000 7f52c7259640 1 -- 172.21.15.164:0/3460223536 shutdown_connections 2024-09-19T08:36:39.332 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.756+0000 7f52c7259640 1 -- 172.21.15.164:0/3460223536 wait complete. 2024-09-19T08:36:39.332 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.757+0000 7f52c7259640 1 Processor -- start 2024-09-19T08:36:39.332 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.757+0000 7f52c7259640 1 -- start start 2024-09-19T08:36:39.332 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.757+0000 7f52c7259640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52c0107740 0x7f52c007db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:39.332 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.757+0000 7f52c7259640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f52c010bbb0 con 0x7f52c0107740 2024-09-19T08:36:39.332 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.757+0000 7f52c4ff5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52c0107740 0x7f52c007db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:39.332 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.757+0000 7f52c4ff5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52c0107740 0x7f52c007db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39650/0 (socket says 172.21.15.164:39650) 2024-09-19T08:36:39.332 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.757+0000 7f52c4ff5640 1 -- 172.21.15.164:0/3691209340 learned_addr learned my addr 172.21.15.164:0/3691209340 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:39.332 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.758+0000 7f52c4ff5640 1 -- 172.21.15.164:0/3691209340 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c007e0b0 con 0x7f52c0107740 2024-09-19T08:36:39.332 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.758+0000 7f52c4ff5640 1 --2- 172.21.15.164:0/3691209340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52c0107740 0x7f52c007db70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f52b0031000 tx=0x7f52b00326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:39.333 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.758+0000 7f52bdffb640 1 -- 172.21.15.164:0/3691209340 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52b0042070 con 0x7f52c0107740 2024-09-19T08:36:39.333 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.758+0000 7f52bdffb640 1 -- 172.21.15.164:0/3691209340 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f52b0032be0 con 0x7f52c0107740 2024-09-19T08:36:39.333 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.758+0000 7f52c7259640 1 -- 172.21.15.164:0/3691209340 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52c007c2a0 con 0x7f52c0107740 2024-09-19T08:36:39.333 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.758+0000 7f52bdffb640 1 -- 172.21.15.164:0/3691209340 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52b0037040 con 0x7f52c0107740 2024-09-19T08:36:39.333 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.758+0000 7f52c7259640 1 -- 172.21.15.164:0/3691209340 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52c007c7e0 con 0x7f52c0107740 2024-09-19T08:36:39.333 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.759+0000 7f52bdffb640 1 -- 172.21.15.164:0/3691209340 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f52b0045070 con 0x7f52c0107740 2024-09-19T08:36:39.333 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.759+0000 7f52bdffb640 1 --2- 172.21.15.164:0/3691209340 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f52a8039200 0x7f52a803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:39.333 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.759+0000 7f52c7259640 1 -- 172.21.15.164:0/3691209340 --> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f52c0107bc0 con 0x7f52a8039200 2024-09-19T08:36:39.333 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.759+0000 7f52bdffb640 1 -- 172.21.15.164:0/3691209340 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f52b006c3e0 con 0x7f52c0107740 2024-09-19T08:36:39.333 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.759+0000 7f52bffff640 1 --2- 172.21.15.164:0/3691209340 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f52a8039200 0x7f52a803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:39.333 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.760+0000 7f52bffff640 1 --2- 172.21.15.164:0/3691209340 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f52a8039200 0x7f52a803b6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f52b40098c0 tx=0x7f52b4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:39.333 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.760+0000 7f52bdffb640 1 -- 172.21.15.164:0/3691209340 <== mgr.14118 v2:172.21.15.164:6800/1173822158 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f52c0107bc0 con 0x7f52a8039200 2024-09-19T08:36:39.333 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.766+0000 7f52c7259640 1 -- 172.21.15.164:0/3691209340 --> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f52c0104230 con 0x7f52a8039200 2024-09-19T08:36:39.334 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.767+0000 7f52bdffb640 1 -- 172.21.15.164:0/3691209340 <== mgr.14118 v2:172.21.15.164:6800/1173822158 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f52c0104230 con 0x7f52a8039200 2024-09-19T08:36:39.334 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.767+0000 7f52c7259640 1 -- 172.21.15.164:0/3691209340 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f52a8039200 msgr2=0x7f52a803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:39.334 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.767+0000 7f52c7259640 1 --2- 172.21.15.164:0/3691209340 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f52a8039200 0x7f52a803b6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f52b40098c0 tx=0x7f52b4006db0 comp rx=0 tx=0).stop 2024-09-19T08:36:39.334 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.767+0000 7f52c7259640 1 -- 172.21.15.164:0/3691209340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52c0107740 msgr2=0x7f52c007db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:39.334 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.767+0000 7f52c7259640 1 --2- 172.21.15.164:0/3691209340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52c0107740 0x7f52c007db70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f52b0031000 tx=0x7f52b00326e0 comp rx=0 tx=0).stop 2024-09-19T08:36:39.334 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.767+0000 7f52c7259640 1 -- 172.21.15.164:0/3691209340 shutdown_connections 2024-09-19T08:36:39.334 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.767+0000 7f52c7259640 1 --2- 172.21.15.164:0/3691209340 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f52a8039200 0x7f52a803b6c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:39.334 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.767+0000 7f52c7259640 1 --2- 172.21.15.164:0/3691209340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52c0107740 0x7f52c007db70 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:39.334 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.767+0000 7f52c7259640 1 -- 172.21.15.164:0/3691209340 >> 172.21.15.164:0/3691209340 conn(0x7f52c0102f50 msgr2=0x7f52c0103b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:39.334 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.767+0000 7f52c7259640 1 -- 172.21.15.164:0/3691209340 shutdown_connections 2024-09-19T08:36:39.334 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:38.767+0000 7f52c7259640 1 -- 172.21.15.164:0/3691209340 wait complete. 2024-09-19T08:36:39.334 INFO:teuthology.orchestra.run.smithi164.stdout:mgr epoch 8 is available 2024-09-19T08:36:39.334 INFO:teuthology.orchestra.run.smithi164.stdout:Setting orchestrator backend to cephadm... 2024-09-19T08:36:40.513 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.711+0000 7f579e15d640 1 Processor -- start 2024-09-19T08:36:40.514 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.711+0000 7f579e15d640 1 -- start start 2024-09-19T08:36:40.514 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.711+0000 7f579e15d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57981076f0 0x7f5798107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:40.514 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.711+0000 7f579e15d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f57981080c0 con 0x7f57981076f0 2024-09-19T08:36:40.514 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.711+0000 7f57977fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57981076f0 0x7f5798107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:40.514 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.711+0000 7f57977fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57981076f0 0x7f5798107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39654/0 (socket says 172.21.15.164:39654) 2024-09-19T08:36:40.515 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.711+0000 7f57977fe640 1 -- 172.21.15.164:0/2641794591 learned_addr learned my addr 172.21.15.164:0/2641794591 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:40.515 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.711+0000 7f57977fe640 1 -- 172.21.15.164:0/2641794591 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57981088f0 con 0x7f57981076f0 2024-09-19T08:36:40.515 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.712+0000 7f57977fe640 1 --2- 172.21.15.164:0/2641794591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57981076f0 0x7f5798107af0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5788009820 tx=0x7f578802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36fa56cc7ec8784e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:40.515 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.712+0000 7f57967fc640 1 -- 172.21.15.164:0/2641794591 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5788037070 con 0x7f57981076f0 2024-09-19T08:36:40.515 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.712+0000 7f57967fc640 1 -- 172.21.15.164:0/2641794591 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5788032900 con 0x7f57981076f0 2024-09-19T08:36:40.515 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.712+0000 7f57967fc640 1 -- 172.21.15.164:0/2641794591 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5788030850 con 0x7f57981076f0 2024-09-19T08:36:40.516 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.713+0000 7f579e15d640 1 -- 172.21.15.164:0/2641794591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57981076f0 msgr2=0x7f5798107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:40.516 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.713+0000 7f579e15d640 1 --2- 172.21.15.164:0/2641794591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57981076f0 0x7f5798107af0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5788009820 tx=0x7f578802a270 comp rx=0 tx=0).stop 2024-09-19T08:36:40.516 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.713+0000 7f579e15d640 1 -- 172.21.15.164:0/2641794591 shutdown_connections 2024-09-19T08:36:40.516 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.713+0000 7f579e15d640 1 --2- 172.21.15.164:0/2641794591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57981076f0 0x7f5798107af0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:40.516 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.713+0000 7f579e15d640 1 -- 172.21.15.164:0/2641794591 >> 172.21.15.164:0/2641794591 conn(0x7f5798102ec0 msgr2=0x7f57981052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:40.517 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.713+0000 7f579e15d640 1 -- 172.21.15.164:0/2641794591 shutdown_connections 2024-09-19T08:36:40.517 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.713+0000 7f579e15d640 1 -- 172.21.15.164:0/2641794591 wait complete. 2024-09-19T08:36:40.517 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.713+0000 7f579e15d640 1 Processor -- start 2024-09-19T08:36:40.517 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.714+0000 7f579e15d640 1 -- start start 2024-09-19T08:36:40.517 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.714+0000 7f579e15d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57981076f0 0x7f579819b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:40.517 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.714+0000 7f579e15d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f579810bca0 con 0x7f57981076f0 2024-09-19T08:36:40.518 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.714+0000 7f57977fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57981076f0 0x7f579819b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:40.518 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.714+0000 7f57977fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57981076f0 0x7f579819b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39670/0 (socket says 172.21.15.164:39670) 2024-09-19T08:36:40.518 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.714+0000 7f57977fe640 1 -- 172.21.15.164:0/3900718885 learned_addr learned my addr 172.21.15.164:0/3900718885 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:40.518 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.714+0000 7f57977fe640 1 -- 172.21.15.164:0/3900718885 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f579819bc20 con 0x7f57981076f0 2024-09-19T08:36:40.518 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.714+0000 7f57977fe640 1 --2- 172.21.15.164:0/3900718885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57981076f0 0x7f579819b6e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f57880097f0 tx=0x7f5788030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:40.518 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.715+0000 7f5794ff9640 1 -- 172.21.15.164:0/3900718885 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5788037040 con 0x7f57981076f0 2024-09-19T08:36:40.519 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.715+0000 7f5794ff9640 1 -- 172.21.15.164:0/3900718885 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5788040070 con 0x7f57981076f0 2024-09-19T08:36:40.519 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.715+0000 7f5794ff9640 1 -- 172.21.15.164:0/3900718885 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f578803a390 con 0x7f57981076f0 2024-09-19T08:36:40.519 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.715+0000 7f579e15d640 1 -- 172.21.15.164:0/3900718885 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f579819beb0 con 0x7f57981076f0 2024-09-19T08:36:40.519 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.715+0000 7f579e15d640 1 -- 172.21.15.164:0/3900718885 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f579819eba0 con 0x7f57981076f0 2024-09-19T08:36:40.519 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.715+0000 7f5794ff9640 1 -- 172.21.15.164:0/3900718885 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f578803b430 con 0x7f57981076f0 2024-09-19T08:36:40.519 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.716+0000 7f5794ff9640 1 --2- 172.21.15.164:0/3900718885 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f57840391b0 0x7f578403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:40.520 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.716+0000 7f579e15d640 1 -- 172.21.15.164:0/3900718885 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5798107b70 con 0x7f57981076f0 2024-09-19T08:36:40.520 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.716+0000 7f5794ff9640 1 -- 172.21.15.164:0/3900718885 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f578806c7a0 con 0x7f57981076f0 2024-09-19T08:36:40.520 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.716+0000 7f5796ffd640 1 --2- 172.21.15.164:0/3900718885 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f57840391b0 0x7f578403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:40.520 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.716+0000 7f5796ffd640 1 --2- 172.21.15.164:0/3900718885 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f57840391b0 0x7f578403b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f57800098c0 tx=0x7f5780006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:40.520 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.721+0000 7f5794ff9640 1 -- 172.21.15.164:0/3900718885 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f578803a5e0 con 0x7f57981076f0 2024-09-19T08:36:40.521 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.899+0000 7f579e15d640 1 -- 172.21.15.164:0/3900718885 --> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f57981040e0 con 0x7f57840391b0 2024-09-19T08:36:40.521 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.904+0000 7f5794ff9640 1 -- 172.21.15.164:0/3900718885 <== mgr.14118 v2:172.21.15.164:6800/1173822158 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f57981040e0 con 0x7f57840391b0 2024-09-19T08:36:40.521 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.907+0000 7f579e15d640 1 -- 172.21.15.164:0/3900718885 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f57840391b0 msgr2=0x7f578403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:40.521 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.907+0000 7f579e15d640 1 --2- 172.21.15.164:0/3900718885 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f57840391b0 0x7f578403b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f57800098c0 tx=0x7f5780006db0 comp rx=0 tx=0).stop 2024-09-19T08:36:40.521 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.907+0000 7f579e15d640 1 -- 172.21.15.164:0/3900718885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57981076f0 msgr2=0x7f579819b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:40.522 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.907+0000 7f579e15d640 1 --2- 172.21.15.164:0/3900718885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57981076f0 0x7f579819b6e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f57880097f0 tx=0x7f5788030d80 comp rx=0 tx=0).stop 2024-09-19T08:36:40.522 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.907+0000 7f579e15d640 1 -- 172.21.15.164:0/3900718885 shutdown_connections 2024-09-19T08:36:40.522 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.907+0000 7f579e15d640 1 --2- 172.21.15.164:0/3900718885 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f57840391b0 0x7f578403b670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:40.522 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.907+0000 7f579e15d640 1 --2- 172.21.15.164:0/3900718885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57981076f0 0x7f579819b6e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:40.522 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.907+0000 7f579e15d640 1 -- 172.21.15.164:0/3900718885 >> 172.21.15.164:0/3900718885 conn(0x7f5798102ec0 msgr2=0x7f57981039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:40.522 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.907+0000 7f579e15d640 1 -- 172.21.15.164:0/3900718885 shutdown_connections 2024-09-19T08:36:40.523 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:39.907+0000 7f579e15d640 1 -- 172.21.15.164:0/3900718885 wait complete. 2024-09-19T08:36:41.055 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:40 smithi164 ceph-mon[30928]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:36:41.056 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:40 smithi164 ceph-mon[30928]: from='mgr.14118 172.21.15.164:0/2031629681' entity='mgr.x' 2024-09-19T08:36:41.056 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:40 smithi164 ceph-mon[30928]: from='mgr.14118 172.21.15.164:0/2031629681' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:36:41.902 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-19T08:36:41.902 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.874+0000 7f9755eed640 1 Processor -- start 2024-09-19T08:36:41.902 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.874+0000 7f9755eed640 1 -- start start 2024-09-19T08:36:41.902 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.874+0000 7f9755eed640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9750106620 0x7f9750106a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:41.902 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.874+0000 7f9755eed640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9750106ff0 con 0x7f9750106620 2024-09-19T08:36:41.902 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.874+0000 7f974f7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9750106620 0x7f9750106a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:41.902 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.874+0000 7f974f7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9750106620 0x7f9750106a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39682/0 (socket says 172.21.15.164:39682) 2024-09-19T08:36:41.902 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.874+0000 7f974f7fe640 1 -- 172.21.15.164:0/3128676352 learned_addr learned my addr 172.21.15.164:0/3128676352 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:41.902 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.875+0000 7f974f7fe640 1 -- 172.21.15.164:0/3128676352 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97501077d0 con 0x7f9750106620 2024-09-19T08:36:41.903 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.875+0000 7f974f7fe640 1 --2- 172.21.15.164:0/3128676352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9750106620 0x7f9750106a20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9740009820 tx=0x7f974002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=499911859a37b32e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:41.903 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.875+0000 7f974e7fc640 1 -- 172.21.15.164:0/3128676352 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9740037070 con 0x7f9750106620 2024-09-19T08:36:41.903 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.876+0000 7f974e7fc640 1 -- 172.21.15.164:0/3128676352 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9740032900 con 0x7f9750106620 2024-09-19T08:36:41.903 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.876+0000 7f974e7fc640 1 -- 172.21.15.164:0/3128676352 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9740030850 con 0x7f9750106620 2024-09-19T08:36:41.903 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.876+0000 7f9755eed640 1 -- 172.21.15.164:0/3128676352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9750106620 msgr2=0x7f9750106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:41.903 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.876+0000 7f9755eed640 1 --2- 172.21.15.164:0/3128676352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9750106620 0x7f9750106a20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9740009820 tx=0x7f974002a270 comp rx=0 tx=0).stop 2024-09-19T08:36:41.903 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.876+0000 7f9755eed640 1 -- 172.21.15.164:0/3128676352 shutdown_connections 2024-09-19T08:36:41.903 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.876+0000 7f9755eed640 1 --2- 172.21.15.164:0/3128676352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9750106620 0x7f9750106a20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:41.903 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.876+0000 7f9755eed640 1 -- 172.21.15.164:0/3128676352 >> 172.21.15.164:0/3128676352 conn(0x7f9750101db0 msgr2=0x7f97501041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:41.903 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.876+0000 7f9755eed640 1 -- 172.21.15.164:0/3128676352 shutdown_connections 2024-09-19T08:36:41.903 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.876+0000 7f9755eed640 1 -- 172.21.15.164:0/3128676352 wait complete. 2024-09-19T08:36:41.903 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.877+0000 7f9755eed640 1 Processor -- start 2024-09-19T08:36:41.903 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.877+0000 7f9755eed640 1 -- start start 2024-09-19T08:36:41.904 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.877+0000 7f9755eed640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9750106620 0x7f975007ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:41.904 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.877+0000 7f9755eed640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f975010ab80 con 0x7f9750106620 2024-09-19T08:36:41.904 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.877+0000 7f974f7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9750106620 0x7f975007ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:41.904 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.877+0000 7f974f7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9750106620 0x7f975007ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39692/0 (socket says 172.21.15.164:39692) 2024-09-19T08:36:41.904 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.877+0000 7f974f7fe640 1 -- 172.21.15.164:0/2044616215 learned_addr learned my addr 172.21.15.164:0/2044616215 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:41.904 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.877+0000 7f974f7fe640 1 -- 172.21.15.164:0/2044616215 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f975019e440 con 0x7f9750106620 2024-09-19T08:36:41.904 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.878+0000 7f974f7fe640 1 --2- 172.21.15.164:0/2044616215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9750106620 0x7f975007ba50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9740009940 tx=0x7f97400326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:41.904 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.878+0000 7f974cff9640 1 -- 172.21.15.164:0/2044616215 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9740042070 con 0x7f9750106620 2024-09-19T08:36:41.904 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.878+0000 7f974cff9640 1 -- 172.21.15.164:0/2044616215 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9740032be0 con 0x7f9750106620 2024-09-19T08:36:41.904 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.878+0000 7f9755eed640 1 -- 172.21.15.164:0/2044616215 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f975019e610 con 0x7f9750106620 2024-09-19T08:36:41.904 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.878+0000 7f974cff9640 1 -- 172.21.15.164:0/2044616215 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9740037040 con 0x7f9750106620 2024-09-19T08:36:41.904 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.878+0000 7f9755eed640 1 -- 172.21.15.164:0/2044616215 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f975019e9b0 con 0x7f9750106620 2024-09-19T08:36:41.905 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.879+0000 7f974cff9640 1 -- 172.21.15.164:0/2044616215 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f9740045070 con 0x7f9750106620 2024-09-19T08:36:41.905 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.879+0000 7f974cff9640 1 --2- 172.21.15.164:0/2044616215 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f97380391b0 0x7f973803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:41.905 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.879+0000 7f974cff9640 1 -- 172.21.15.164:0/2044616215 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f974006c350 con 0x7f9750106620 2024-09-19T08:36:41.905 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.879+0000 7f9755eed640 1 -- 172.21.15.164:0/2044616215 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9750106aa0 con 0x7f9750106620 2024-09-19T08:36:41.905 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.879+0000 7f974effd640 1 --2- 172.21.15.164:0/2044616215 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f97380391b0 0x7f973803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:41.905 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.879+0000 7f974effd640 1 --2- 172.21.15.164:0/2044616215 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f97380391b0 0x7f973803b670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f974400ad30 tx=0x7f97440093f0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:41.905 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:40.884+0000 7f974cff9640 1 -- 172.21.15.164:0/2044616215 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f974003bde0 con 0x7f9750106620 2024-09-19T08:36:41.905 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:41.055+0000 7f9755eed640 1 -- 172.21.15.164:0/2044616215 --> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f9750103060 con 0x7f97380391b0 2024-09-19T08:36:41.905 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:41.055+0000 7f974cff9640 1 -- 172.21.15.164:0/2044616215 <== mgr.14118 v2:172.21.15.164:6800/1173822158 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f9750103060 con 0x7f97380391b0 2024-09-19T08:36:41.905 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:41.058+0000 7f9755eed640 1 -- 172.21.15.164:0/2044616215 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f97380391b0 msgr2=0x7f973803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:41.905 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:41.058+0000 7f9755eed640 1 --2- 172.21.15.164:0/2044616215 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f97380391b0 0x7f973803b670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f974400ad30 tx=0x7f97440093f0 comp rx=0 tx=0).stop 2024-09-19T08:36:41.905 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:41.058+0000 7f9755eed640 1 -- 172.21.15.164:0/2044616215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9750106620 msgr2=0x7f975007ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:41.905 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:41.058+0000 7f9755eed640 1 --2- 172.21.15.164:0/2044616215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9750106620 0x7f975007ba50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9740009940 tx=0x7f97400326e0 comp rx=0 tx=0).stop 2024-09-19T08:36:41.906 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:41.058+0000 7f9755eed640 1 -- 172.21.15.164:0/2044616215 shutdown_connections 2024-09-19T08:36:41.906 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:41.058+0000 7f9755eed640 1 --2- 172.21.15.164:0/2044616215 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f97380391b0 0x7f973803b670 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:41.906 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:41.058+0000 7f9755eed640 1 --2- 172.21.15.164:0/2044616215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9750106620 0x7f975007ba50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:41.906 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:41.058+0000 7f9755eed640 1 -- 172.21.15.164:0/2044616215 >> 172.21.15.164:0/2044616215 conn(0x7f9750101db0 msgr2=0x7f9750102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:41.906 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:41.058+0000 7f9755eed640 1 -- 172.21.15.164:0/2044616215 shutdown_connections 2024-09-19T08:36:41.906 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:41.058+0000 7f9755eed640 1 -- 172.21.15.164:0/2044616215 wait complete. 2024-09-19T08:36:41.906 INFO:teuthology.orchestra.run.smithi164.stdout:Generating ssh key... 2024-09-19T08:36:42.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:42 smithi164 ceph-mon[30928]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:36:43.000 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:42 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: Generating public/private rsa key pair. 2024-09-19T08:36:43.000 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:42 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: Your identification has been saved in /tmp/tmp81px724q/key 2024-09-19T08:36:43.000 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:42 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: Your public key has been saved in /tmp/tmp81px724q/key.pub 2024-09-19T08:36:43.000 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:42 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: The key fingerprint is: 2024-09-19T08:36:43.000 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:42 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: SHA256:X+K15Ow88QPO4boCy69xAvFcn2eoJCWRYAt6gTWmUdw ceph-1e4b0d74-7662-11ef-bceb-c7b262605968 2024-09-19T08:36:43.000 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:42 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: The key's randomart image is: 2024-09-19T08:36:43.000 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:42 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: +---[RSA 3072]----+ 2024-09-19T08:36:43.001 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:42 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: |.=B.o... | 2024-09-19T08:36:43.001 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:42 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: |.=.=E... | 2024-09-19T08:36:43.001 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:42 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: |o . o . o | 2024-09-19T08:36:43.001 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:42 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: | . + + . o | 2024-09-19T08:36:43.001 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:42 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: | . + S = * | 2024-09-19T08:36:43.001 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:42 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: | ..o + X+. | 2024-09-19T08:36:43.001 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:42 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: | .ooo o++= | 2024-09-19T08:36:43.001 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:42 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: | o+. o= o | 2024-09-19T08:36:43.001 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:42 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: | .o..ooo. . | 2024-09-19T08:36:43.001 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:42 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: +----[SHA256]-----+ 2024-09-19T08:36:43.241 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.319+0000 7f164f96e640 1 Processor -- start 2024-09-19T08:36:43.241 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.319+0000 7f164f96e640 1 -- start start 2024-09-19T08:36:43.241 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.320+0000 7f164f96e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1648108160 0x7f164810a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:43.241 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.320+0000 7f164f96e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f164807beb0 con 0x7f1648108160 2024-09-19T08:36:43.241 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.320+0000 7f164d70a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1648108160 0x7f164810a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:43.241 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.320+0000 7f164d70a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1648108160 0x7f164810a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39698/0 (socket says 172.21.15.164:39698) 2024-09-19T08:36:43.242 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.320+0000 7f164d70a640 1 -- 172.21.15.164:0/1494873186 learned_addr learned my addr 172.21.15.164:0/1494873186 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:43.242 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.320+0000 7f164d70a640 1 -- 172.21.15.164:0/1494873186 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f164810aa90 con 0x7f1648108160 2024-09-19T08:36:43.242 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.321+0000 7f164d70a640 1 --2- 172.21.15.164:0/1494873186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1648108160 0x7f164810a550 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1644009820 tx=0x7f164402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958a0d30deaa4819 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:43.242 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.321+0000 7f163bfff640 1 -- 172.21.15.164:0/1494873186 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1644037070 con 0x7f1648108160 2024-09-19T08:36:43.242 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.321+0000 7f163bfff640 1 -- 172.21.15.164:0/1494873186 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1644032900 con 0x7f1648108160 2024-09-19T08:36:43.242 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.321+0000 7f163bfff640 1 -- 172.21.15.164:0/1494873186 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1644030850 con 0x7f1648108160 2024-09-19T08:36:43.242 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.322+0000 7f164f96e640 1 -- 172.21.15.164:0/1494873186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1648108160 msgr2=0x7f164810a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:43.242 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.322+0000 7f164f96e640 1 --2- 172.21.15.164:0/1494873186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1648108160 0x7f164810a550 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1644009820 tx=0x7f164402a270 comp rx=0 tx=0).stop 2024-09-19T08:36:43.242 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.322+0000 7f164f96e640 1 -- 172.21.15.164:0/1494873186 shutdown_connections 2024-09-19T08:36:43.242 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.322+0000 7f164f96e640 1 --2- 172.21.15.164:0/1494873186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1648108160 0x7f164810a550 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:43.242 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.322+0000 7f164f96e640 1 -- 172.21.15.164:0/1494873186 >> 172.21.15.164:0/1494873186 conn(0x7f1648101e20 msgr2=0x7f1648104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:43.242 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.322+0000 7f164f96e640 1 -- 172.21.15.164:0/1494873186 shutdown_connections 2024-09-19T08:36:43.242 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.322+0000 7f164f96e640 1 -- 172.21.15.164:0/1494873186 wait complete. 2024-09-19T08:36:43.243 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.323+0000 7f164f96e640 1 Processor -- start 2024-09-19T08:36:43.243 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.323+0000 7f164f96e640 1 -- start start 2024-09-19T08:36:43.243 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.323+0000 7f164f96e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1648108160 0x7f16481079e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:43.243 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.323+0000 7f164f96e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f164810bb30 con 0x7f1648108160 2024-09-19T08:36:43.243 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.323+0000 7f164d70a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1648108160 0x7f16481079e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:43.243 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.323+0000 7f164d70a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1648108160 0x7f16481079e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39712/0 (socket says 172.21.15.164:39712) 2024-09-19T08:36:43.243 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.323+0000 7f164d70a640 1 -- 172.21.15.164:0/30421269 learned_addr learned my addr 172.21.15.164:0/30421269 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:43.243 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.323+0000 7f164d70a640 1 -- 172.21.15.164:0/30421269 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1648107f20 con 0x7f1648108160 2024-09-19T08:36:43.243 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.324+0000 7f164d70a640 1 --2- 172.21.15.164:0/30421269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1648108160 0x7f16481079e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1644031000 tx=0x7f16440326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:43.243 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.324+0000 7f163a7fc640 1 -- 172.21.15.164:0/30421269 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1644042070 con 0x7f1648108160 2024-09-19T08:36:43.243 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.324+0000 7f163a7fc640 1 -- 172.21.15.164:0/30421269 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1644032be0 con 0x7f1648108160 2024-09-19T08:36:43.243 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.324+0000 7f164f96e640 1 -- 172.21.15.164:0/30421269 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1648106110 con 0x7f1648108160 2024-09-19T08:36:43.243 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.324+0000 7f163a7fc640 1 -- 172.21.15.164:0/30421269 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1644037040 con 0x7f1648108160 2024-09-19T08:36:43.244 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.324+0000 7f164f96e640 1 -- 172.21.15.164:0/30421269 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1648106650 con 0x7f1648108160 2024-09-19T08:36:43.244 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.325+0000 7f164f96e640 1 -- 172.21.15.164:0/30421269 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16481085c0 con 0x7f1648108160 2024-09-19T08:36:43.244 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.325+0000 7f163a7fc640 1 -- 172.21.15.164:0/30421269 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f1644045070 con 0x7f1648108160 2024-09-19T08:36:43.244 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.325+0000 7f163a7fc640 1 --2- 172.21.15.164:0/30421269 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f16340391b0 0x7f163403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:43.244 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.325+0000 7f163a7fc640 1 -- 172.21.15.164:0/30421269 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f164406c2f0 con 0x7f1648108160 2024-09-19T08:36:43.244 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.325+0000 7f164cf09640 1 --2- 172.21.15.164:0/30421269 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f16340391b0 0x7f163403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:43.244 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.326+0000 7f164cf09640 1 --2- 172.21.15.164:0/30421269 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f16340391b0 0x7f163403b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f163c0098c0 tx=0x7f163c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:43.244 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.330+0000 7f163a7fc640 1 -- 172.21.15.164:0/30421269 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1644042210 con 0x7f1648108160 2024-09-19T08:36:43.244 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.500+0000 7f164f96e640 1 -- 172.21.15.164:0/30421269 --> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f1648106f00 con 0x7f16340391b0 2024-09-19T08:36:43.245 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.703+0000 7f163a7fc640 1 -- 172.21.15.164:0/30421269 <== mgr.14118 v2:172.21.15.164:6800/1173822158 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1648106f00 con 0x7f16340391b0 2024-09-19T08:36:43.245 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.705+0000 7f164f96e640 1 -- 172.21.15.164:0/30421269 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f16340391b0 msgr2=0x7f163403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:43.245 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.705+0000 7f164f96e640 1 --2- 172.21.15.164:0/30421269 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f16340391b0 0x7f163403b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f163c0098c0 tx=0x7f163c006db0 comp rx=0 tx=0).stop 2024-09-19T08:36:43.245 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.705+0000 7f164f96e640 1 -- 172.21.15.164:0/30421269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1648108160 msgr2=0x7f16481079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:43.245 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.705+0000 7f164f96e640 1 --2- 172.21.15.164:0/30421269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1648108160 0x7f16481079e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1644031000 tx=0x7f16440326e0 comp rx=0 tx=0).stop 2024-09-19T08:36:43.245 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.705+0000 7f164f96e640 1 -- 172.21.15.164:0/30421269 shutdown_connections 2024-09-19T08:36:43.245 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.705+0000 7f164f96e640 1 --2- 172.21.15.164:0/30421269 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f16340391b0 0x7f163403b670 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:43.245 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.705+0000 7f164f96e640 1 --2- 172.21.15.164:0/30421269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1648108160 0x7f16481079e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:43.245 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.705+0000 7f164f96e640 1 -- 172.21.15.164:0/30421269 >> 172.21.15.164:0/30421269 conn(0x7f1648101e20 msgr2=0x7f16481029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:43.245 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.706+0000 7f164f96e640 1 -- 172.21.15.164:0/30421269 shutdown_connections 2024-09-19T08:36:43.246 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:42.706+0000 7f164f96e640 1 -- 172.21.15.164:0/30421269 wait complete. 2024-09-19T08:36:44.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:43 smithi164 ceph-mon[30928]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:36:44.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:43 smithi164 ceph-mon[30928]: Generating ssh key... 2024-09-19T08:36:44.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:43 smithi164 ceph-mon[30928]: from='mgr.14118 172.21.15.164:0/2031629681' entity='mgr.x' 2024-09-19T08:36:44.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:43 smithi164 ceph-mon[30928]: from='mgr.14118 172.21.15.164:0/2031629681' entity='mgr.x' 2024-09-19T08:36:44.397 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwRZTs4bFdIT7Ia5SP6k7MgjHFWAtGy4C3fBsqsbIeiH6bUO1UHUIQlbvqN0KE02PHSDv6Tj74pBEqTKB4yyK2Hn4HYy4oE1ju4bMqrf+OOrkg2yR1Sf6vlxOHEILE7s5mi1S/WfJshFRD3JdG6HfYWkvF1Fe2umpF4ESUmnc//8dNT6CK/tA55XHShkxuB9hoFtq9hxeBpujxRRoQNXrQTzewYtVdVIGlgEjnMKM7iOsy4MS/LxKo1vxVxUqxcpfBWiyyXTfsNKvK4al8rLK5If5yvUhlkJb82HEGC7FZpnYL+s9oikobpRDWe+4VPfD1wqQm8I+dYDJZ1jhZG2j6vWkxQDmOD1bOz/PHfd2lAHfTH3rkjje7ftPkw8yXnk78oQ4cqT/4xxml8TYcH6Hez8PlCcKm/5xSx5JUcd6i1QBJhgwZJEVh5ftQpasGz1gAg2ghKd2N0Wnw+sIYXx+yqqxw2QFJO2lw6VpdTn0LcIxo5f58em+0e3RhR0ndZLM= ceph-1e4b0d74-7662-11ef-bceb-c7b262605968 2024-09-19T08:36:44.397 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.646+0000 7f505a8cc640 1 Processor -- start 2024-09-19T08:36:44.397 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.646+0000 7f505a8cc640 1 -- start start 2024-09-19T08:36:44.397 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.646+0000 7f505a8cc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5054106610 0x7f5054106a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:44.397 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.646+0000 7f505a8cc640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5054106fe0 con 0x7f5054106610 2024-09-19T08:36:44.397 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.646+0000 7f5053fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5054106610 0x7f5054106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:44.397 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.646+0000 7f5053fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5054106610 0x7f5054106a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40700/0 (socket says 172.21.15.164:40700) 2024-09-19T08:36:44.398 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.646+0000 7f5053fff640 1 -- 172.21.15.164:0/2919996790 learned_addr learned my addr 172.21.15.164:0/2919996790 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:44.398 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.646+0000 7f5053fff640 1 -- 172.21.15.164:0/2919996790 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5054107810 con 0x7f5054106610 2024-09-19T08:36:44.398 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.647+0000 7f5053fff640 1 --2- 172.21.15.164:0/2919996790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5054106610 0x7f5054106a10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5044009820 tx=0x7f504402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84ee3970547cf13 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:44.398 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.647+0000 7f5052ffd640 1 -- 172.21.15.164:0/2919996790 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5044037070 con 0x7f5054106610 2024-09-19T08:36:44.398 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.647+0000 7f5052ffd640 1 -- 172.21.15.164:0/2919996790 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5044032900 con 0x7f5054106610 2024-09-19T08:36:44.398 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.647+0000 7f5052ffd640 1 -- 172.21.15.164:0/2919996790 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5044030850 con 0x7f5054106610 2024-09-19T08:36:44.398 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.648+0000 7f505a8cc640 1 -- 172.21.15.164:0/2919996790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5054106610 msgr2=0x7f5054106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:44.398 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.648+0000 7f505a8cc640 1 --2- 172.21.15.164:0/2919996790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5054106610 0x7f5054106a10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5044009820 tx=0x7f504402a270 comp rx=0 tx=0).stop 2024-09-19T08:36:44.398 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.648+0000 7f505a8cc640 1 -- 172.21.15.164:0/2919996790 shutdown_connections 2024-09-19T08:36:44.398 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.648+0000 7f505a8cc640 1 --2- 172.21.15.164:0/2919996790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5054106610 0x7f5054106a10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:44.398 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.648+0000 7f505a8cc640 1 -- 172.21.15.164:0/2919996790 >> 172.21.15.164:0/2919996790 conn(0x7f5054101da0 msgr2=0x7f50541041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:44.398 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.648+0000 7f505a8cc640 1 -- 172.21.15.164:0/2919996790 shutdown_connections 2024-09-19T08:36:44.399 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.648+0000 7f505a8cc640 1 -- 172.21.15.164:0/2919996790 wait complete. 2024-09-19T08:36:44.399 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.648+0000 7f505a8cc640 1 Processor -- start 2024-09-19T08:36:44.399 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.649+0000 7f505a8cc640 1 -- start start 2024-09-19T08:36:44.399 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.649+0000 7f505a8cc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5054106610 0x7f505407ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:44.399 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.649+0000 7f505a8cc640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f505410abc0 con 0x7f5054106610 2024-09-19T08:36:44.399 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.649+0000 7f5053fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5054106610 0x7f505407ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:44.399 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.649+0000 7f5053fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5054106610 0x7f505407ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40712/0 (socket says 172.21.15.164:40712) 2024-09-19T08:36:44.399 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.649+0000 7f5053fff640 1 -- 172.21.15.164:0/10209643 learned_addr learned my addr 172.21.15.164:0/10209643 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:44.399 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.649+0000 7f5053fff640 1 -- 172.21.15.164:0/10209643 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f505419e480 con 0x7f5054106610 2024-09-19T08:36:44.399 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.649+0000 7f5053fff640 1 --2- 172.21.15.164:0/10209643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5054106610 0x7f505407ba50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5044009940 tx=0x7f50440326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:44.399 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.649+0000 7f50517fa640 1 -- 172.21.15.164:0/10209643 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5044042070 con 0x7f5054106610 2024-09-19T08:36:44.399 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.650+0000 7f505a8cc640 1 -- 172.21.15.164:0/10209643 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f505419e650 con 0x7f5054106610 2024-09-19T08:36:44.399 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.650+0000 7f50517fa640 1 -- 172.21.15.164:0/10209643 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5044032be0 con 0x7f5054106610 2024-09-19T08:36:44.400 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.650+0000 7f50517fa640 1 -- 172.21.15.164:0/10209643 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5044037040 con 0x7f5054106610 2024-09-19T08:36:44.400 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.650+0000 7f505a8cc640 1 -- 172.21.15.164:0/10209643 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f505419e9f0 con 0x7f5054106610 2024-09-19T08:36:44.400 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.650+0000 7f50517fa640 1 -- 172.21.15.164:0/10209643 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f5044045070 con 0x7f5054106610 2024-09-19T08:36:44.400 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.651+0000 7f50517fa640 1 --2- 172.21.15.164:0/10209643 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f503c0391b0 0x7f503c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:44.400 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.651+0000 7f50517fa640 1 -- 172.21.15.164:0/10209643 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f504406c350 con 0x7f5054106610 2024-09-19T08:36:44.400 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.651+0000 7f505a8cc640 1 -- 172.21.15.164:0/10209643 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5054106a90 con 0x7f5054106610 2024-09-19T08:36:44.400 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.651+0000 7f50537fe640 1 --2- 172.21.15.164:0/10209643 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f503c0391b0 0x7f503c03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:44.400 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.651+0000 7f50537fe640 1 --2- 172.21.15.164:0/10209643 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f503c0391b0 0x7f503c03b670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f50480098c0 tx=0x7f5048006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:44.400 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.655+0000 7f50517fa640 1 -- 172.21.15.164:0/10209643 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f504403be10 con 0x7f5054106610 2024-09-19T08:36:44.400 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.823+0000 7f505a8cc640 1 -- 172.21.15.164:0/10209643 --> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f5054103070 con 0x7f503c0391b0 2024-09-19T08:36:44.400 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.824+0000 7f50517fa640 1 -- 172.21.15.164:0/10209643 <== mgr.14118 v2:172.21.15.164:6800/1173822158 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f5054103070 con 0x7f503c0391b0 2024-09-19T08:36:44.400 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.826+0000 7f505a8cc640 1 -- 172.21.15.164:0/10209643 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f503c0391b0 msgr2=0x7f503c03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:44.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.826+0000 7f505a8cc640 1 --2- 172.21.15.164:0/10209643 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f503c0391b0 0x7f503c03b670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f50480098c0 tx=0x7f5048006db0 comp rx=0 tx=0).stop 2024-09-19T08:36:44.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.826+0000 7f505a8cc640 1 -- 172.21.15.164:0/10209643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5054106610 msgr2=0x7f505407ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:44.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.826+0000 7f505a8cc640 1 --2- 172.21.15.164:0/10209643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5054106610 0x7f505407ba50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5044009940 tx=0x7f50440326e0 comp rx=0 tx=0).stop 2024-09-19T08:36:44.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.826+0000 7f505a8cc640 1 -- 172.21.15.164:0/10209643 shutdown_connections 2024-09-19T08:36:44.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.826+0000 7f505a8cc640 1 --2- 172.21.15.164:0/10209643 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f503c0391b0 0x7f503c03b670 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:44.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.826+0000 7f505a8cc640 1 --2- 172.21.15.164:0/10209643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5054106610 0x7f505407ba50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:44.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.826+0000 7f505a8cc640 1 -- 172.21.15.164:0/10209643 >> 172.21.15.164:0/10209643 conn(0x7f5054101da0 msgr2=0x7f5054102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:44.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.826+0000 7f505a8cc640 1 -- 172.21.15.164:0/10209643 shutdown_connections 2024-09-19T08:36:44.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:43.826+0000 7f505a8cc640 1 -- 172.21.15.164:0/10209643 wait complete. 2024-09-19T08:36:44.401 INFO:teuthology.orchestra.run.smithi164.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-19T08:36:44.401 INFO:teuthology.orchestra.run.smithi164.stdout:Adding key to root@localhost authorized_keys... 2024-09-19T08:36:44.406 INFO:teuthology.orchestra.run.smithi164.stdout:Adding host smithi164... 2024-09-19T08:36:44.778 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:44 smithi164 ceph-mon[30928]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:36:45.771 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:45 smithi164 ceph-mon[30928]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi164", "addr": "172.21.15.164", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:36:47.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:46 smithi164 ceph-mon[30928]: Deploying cephadm binary to smithi164 2024-09-19T08:36:48.774 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout Added host 'smithi164' with addr '172.21.15.164' 2024-09-19T08:36:48.774 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.785+0000 7fec1e5ed640 1 Processor -- start 2024-09-19T08:36:48.774 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.785+0000 7fec1e5ed640 1 -- start start 2024-09-19T08:36:48.774 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.785+0000 7fec1e5ed640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec181065f0 0x7fec181069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:48.774 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.785+0000 7fec1e5ed640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fec18106fc0 con 0x7fec181065f0 2024-09-19T08:36:48.774 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.785+0000 7fec17fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec181065f0 0x7fec181069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:48.774 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.785+0000 7fec17fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec181065f0 0x7fec181069f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40726/0 (socket says 172.21.15.164:40726) 2024-09-19T08:36:48.774 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.785+0000 7fec17fff640 1 -- 172.21.15.164:0/803344918 learned_addr learned my addr 172.21.15.164:0/803344918 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:48.774 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.785+0000 7fec17fff640 1 -- 172.21.15.164:0/803344918 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec181077f0 con 0x7fec181065f0 2024-09-19T08:36:48.775 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.786+0000 7fec17fff640 1 --2- 172.21.15.164:0/803344918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec181065f0 0x7fec181069f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fec08009820 tx=0x7fec0802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df578e83dd4033bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:48.775 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.786+0000 7fec177fe640 1 -- 172.21.15.164:0/803344918 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec08037070 con 0x7fec181065f0 2024-09-19T08:36:48.775 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.786+0000 7fec177fe640 1 -- 172.21.15.164:0/803344918 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fec08032900 con 0x7fec181065f0 2024-09-19T08:36:48.775 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.786+0000 7fec177fe640 1 -- 172.21.15.164:0/803344918 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec08030850 con 0x7fec181065f0 2024-09-19T08:36:48.775 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.787+0000 7fec1e5ed640 1 -- 172.21.15.164:0/803344918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec181065f0 msgr2=0x7fec181069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:48.775 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.787+0000 7fec1e5ed640 1 --2- 172.21.15.164:0/803344918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec181065f0 0x7fec181069f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fec08009820 tx=0x7fec0802a270 comp rx=0 tx=0).stop 2024-09-19T08:36:48.775 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.787+0000 7fec1e5ed640 1 -- 172.21.15.164:0/803344918 shutdown_connections 2024-09-19T08:36:48.775 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.787+0000 7fec1e5ed640 1 --2- 172.21.15.164:0/803344918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec181065f0 0x7fec181069f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:48.775 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.787+0000 7fec1e5ed640 1 -- 172.21.15.164:0/803344918 >> 172.21.15.164:0/803344918 conn(0x7fec18101e20 msgr2=0x7fec18104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:48.775 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.787+0000 7fec1e5ed640 1 -- 172.21.15.164:0/803344918 shutdown_connections 2024-09-19T08:36:48.775 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.787+0000 7fec1e5ed640 1 -- 172.21.15.164:0/803344918 wait complete. 2024-09-19T08:36:48.775 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.787+0000 7fec1e5ed640 1 Processor -- start 2024-09-19T08:36:48.775 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.787+0000 7fec1e5ed640 1 -- start start 2024-09-19T08:36:48.776 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.788+0000 7fec1e5ed640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec181065f0 0x7fec1807ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:48.776 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.788+0000 7fec1e5ed640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fec1810aba0 con 0x7fec181065f0 2024-09-19T08:36:48.776 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.788+0000 7fec17fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec181065f0 0x7fec1807ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:48.776 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.788+0000 7fec17fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec181065f0 0x7fec1807ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40734/0 (socket says 172.21.15.164:40734) 2024-09-19T08:36:48.776 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.788+0000 7fec17fff640 1 -- 172.21.15.164:0/1811460928 learned_addr learned my addr 172.21.15.164:0/1811460928 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:48.776 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.788+0000 7fec17fff640 1 -- 172.21.15.164:0/1811460928 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec1819c260 con 0x7fec181065f0 2024-09-19T08:36:48.776 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.788+0000 7fec17fff640 1 --2- 172.21.15.164:0/1811460928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec181065f0 0x7fec1807ba50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fec080097f0 tx=0x7fec08030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:48.776 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.788+0000 7fec15ffb640 1 -- 172.21.15.164:0/1811460928 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec08042070 con 0x7fec181065f0 2024-09-19T08:36:48.776 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.788+0000 7fec1e5ed640 1 -- 172.21.15.164:0/1811460928 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec181a0870 con 0x7fec181065f0 2024-09-19T08:36:48.776 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.788+0000 7fec15ffb640 1 -- 172.21.15.164:0/1811460928 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fec08032ca0 con 0x7fec181065f0 2024-09-19T08:36:48.776 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.788+0000 7fec15ffb640 1 -- 172.21.15.164:0/1811460928 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec08037040 con 0x7fec181065f0 2024-09-19T08:36:48.776 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.788+0000 7fec1e5ed640 1 -- 172.21.15.164:0/1811460928 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec181a0c10 con 0x7fec181065f0 2024-09-19T08:36:48.777 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.789+0000 7fec15ffb640 1 -- 172.21.15.164:0/1811460928 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fec08045070 con 0x7fec181065f0 2024-09-19T08:36:48.777 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.789+0000 7fec15ffb640 1 --2- 172.21.15.164:0/1811460928 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7fec040391b0 0x7fec0403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:48.777 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.789+0000 7fec15ffb640 1 -- 172.21.15.164:0/1811460928 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fec0806c1a0 con 0x7fec181065f0 2024-09-19T08:36:48.777 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.789+0000 7fec1e5ed640 1 -- 172.21.15.164:0/1811460928 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec18106a70 con 0x7fec181065f0 2024-09-19T08:36:48.777 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.790+0000 7fec0ffff640 1 --2- 172.21.15.164:0/1811460928 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7fec040391b0 0x7fec0403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:48.777 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.790+0000 7fec0ffff640 1 --2- 172.21.15.164:0/1811460928 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7fec040391b0 0x7fec0403b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fec000098c0 tx=0x7fec00006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:48.777 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.794+0000 7fec15ffb640 1 -- 172.21.15.164:0/1811460928 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fec0804b170 con 0x7fec181065f0 2024-09-19T08:36:48.777 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:44.965+0000 7fec1e5ed640 1 -- 172.21.15.164:0/1811460928 --> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi164", "addr": "172.21.15.164", "target": ["mon-mgr", ""]}) -- 0x7fec18103090 con 0x7fec040391b0 2024-09-19T08:36:48.777 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:48.216+0000 7fec15ffb640 1 -- 172.21.15.164:0/1811460928 <== mgr.14118 v2:172.21.15.164:6800/1173822158 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fec18103090 con 0x7fec040391b0 2024-09-19T08:36:48.777 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:48.219+0000 7fec1e5ed640 1 -- 172.21.15.164:0/1811460928 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7fec040391b0 msgr2=0x7fec0403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:48.777 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:48.219+0000 7fec1e5ed640 1 --2- 172.21.15.164:0/1811460928 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7fec040391b0 0x7fec0403b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fec000098c0 tx=0x7fec00006db0 comp rx=0 tx=0).stop 2024-09-19T08:36:48.777 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:48.219+0000 7fec1e5ed640 1 -- 172.21.15.164:0/1811460928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec181065f0 msgr2=0x7fec1807ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:48.777 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:48.219+0000 7fec1e5ed640 1 --2- 172.21.15.164:0/1811460928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec181065f0 0x7fec1807ba50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fec080097f0 tx=0x7fec08030ec0 comp rx=0 tx=0).stop 2024-09-19T08:36:48.778 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:48.219+0000 7fec1e5ed640 1 -- 172.21.15.164:0/1811460928 shutdown_connections 2024-09-19T08:36:48.778 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:48.219+0000 7fec1e5ed640 1 --2- 172.21.15.164:0/1811460928 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7fec040391b0 0x7fec0403b670 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:48.778 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:48.219+0000 7fec1e5ed640 1 --2- 172.21.15.164:0/1811460928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec181065f0 0x7fec1807ba50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:48.778 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:48.219+0000 7fec1e5ed640 1 -- 172.21.15.164:0/1811460928 >> 172.21.15.164:0/1811460928 conn(0x7fec18101e20 msgr2=0x7fec18102980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:48.778 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:48.219+0000 7fec1e5ed640 1 -- 172.21.15.164:0/1811460928 shutdown_connections 2024-09-19T08:36:48.778 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:48.219+0000 7fec1e5ed640 1 -- 172.21.15.164:0/1811460928 wait complete. 2024-09-19T08:36:48.778 INFO:teuthology.orchestra.run.smithi164.stdout:Deploying unmanaged mon service... 2024-09-19T08:36:49.338 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:49 smithi164 ceph-mon[30928]: from='mgr.14118 172.21.15.164:0/2031629681' entity='mgr.x' 2024-09-19T08:36:49.339 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:49 smithi164 ceph-mon[30928]: Added host smithi164 2024-09-19T08:36:49.339 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:49 smithi164 ceph-mon[30928]: from='mgr.14118 172.21.15.164:0/2031629681' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:36:50.723 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:50 smithi164 ceph-mon[30928]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:36:50.723 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:50 smithi164 ceph-mon[30928]: Saving service mon spec with placement count:5 2024-09-19T08:36:50.724 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:50 smithi164 ceph-mon[30928]: from='mgr.14118 172.21.15.164:0/2031629681' entity='mgr.x' 2024-09-19T08:36:51.006 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-19T08:36:51.006 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.259+0000 7ff8a854c640 1 Processor -- start 2024-09-19T08:36:51.006 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.259+0000 7ff8a854c640 1 -- start start 2024-09-19T08:36:51.006 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.260+0000 7ff8a854c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8a01065f0 0x7ff8a01069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:51.006 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.260+0000 7ff8a854c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a0106fc0 con 0x7ff8a01065f0 2024-09-19T08:36:51.007 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.260+0000 7ff8a62e8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8a01065f0 0x7ff8a01069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:51.007 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.260+0000 7ff8a62e8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8a01065f0 0x7ff8a01069f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40742/0 (socket says 172.21.15.164:40742) 2024-09-19T08:36:51.007 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.260+0000 7ff8a62e8640 1 -- 172.21.15.164:0/654235631 learned_addr learned my addr 172.21.15.164:0/654235631 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:51.007 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.260+0000 7ff8a62e8640 1 -- 172.21.15.164:0/654235631 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a01077f0 con 0x7ff8a01065f0 2024-09-19T08:36:51.007 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.261+0000 7ff8a62e8640 1 --2- 172.21.15.164:0/654235631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8a01065f0 0x7ff8a01069f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff89c009820 tx=0x7ff89c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=966bc547211cc15c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:51.007 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.261+0000 7ff8a52e6640 1 -- 172.21.15.164:0/654235631 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff89c037070 con 0x7ff8a01065f0 2024-09-19T08:36:51.007 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.261+0000 7ff8a52e6640 1 -- 172.21.15.164:0/654235631 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff89c032900 con 0x7ff8a01065f0 2024-09-19T08:36:51.007 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.261+0000 7ff8a52e6640 1 -- 172.21.15.164:0/654235631 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff89c030850 con 0x7ff8a01065f0 2024-09-19T08:36:51.007 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.261+0000 7ff8a854c640 1 -- 172.21.15.164:0/654235631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8a01065f0 msgr2=0x7ff8a01069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:51.007 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.261+0000 7ff8a854c640 1 --2- 172.21.15.164:0/654235631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8a01065f0 0x7ff8a01069f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff89c009820 tx=0x7ff89c02a270 comp rx=0 tx=0).stop 2024-09-19T08:36:51.007 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.262+0000 7ff8a854c640 1 -- 172.21.15.164:0/654235631 shutdown_connections 2024-09-19T08:36:51.007 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.262+0000 7ff8a854c640 1 --2- 172.21.15.164:0/654235631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8a01065f0 0x7ff8a01069f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:51.007 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.262+0000 7ff8a854c640 1 -- 172.21.15.164:0/654235631 >> 172.21.15.164:0/654235631 conn(0x7ff8a0101e20 msgr2=0x7ff8a0104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:51.008 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.262+0000 7ff8a854c640 1 -- 172.21.15.164:0/654235631 shutdown_connections 2024-09-19T08:36:51.008 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.262+0000 7ff8a854c640 1 -- 172.21.15.164:0/654235631 wait complete. 2024-09-19T08:36:51.008 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.262+0000 7ff8a854c640 1 Processor -- start 2024-09-19T08:36:51.008 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.262+0000 7ff8a854c640 1 -- start start 2024-09-19T08:36:51.008 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.262+0000 7ff8a854c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8a01065f0 0x7ff8a019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:51.008 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.262+0000 7ff8a854c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a010aba0 con 0x7ff8a01065f0 2024-09-19T08:36:51.008 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.262+0000 7ff8a62e8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8a01065f0 0x7ff8a019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:51.008 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.262+0000 7ff8a62e8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8a01065f0 0x7ff8a019a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40744/0 (socket says 172.21.15.164:40744) 2024-09-19T08:36:51.008 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.262+0000 7ff8a62e8640 1 -- 172.21.15.164:0/471574391 learned_addr learned my addr 172.21.15.164:0/471574391 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:51.008 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.263+0000 7ff8a62e8640 1 -- 172.21.15.164:0/471574391 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a019ab60 con 0x7ff8a01065f0 2024-09-19T08:36:51.008 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.263+0000 7ff8a62e8640 1 --2- 172.21.15.164:0/471574391 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8a01065f0 0x7ff8a019a620 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff89c0097f0 tx=0x7ff89c030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:51.008 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.263+0000 7ff8977fe640 1 -- 172.21.15.164:0/471574391 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff89c042070 con 0x7ff8a01065f0 2024-09-19T08:36:51.008 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.263+0000 7ff8977fe640 1 -- 172.21.15.164:0/471574391 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff89c032ca0 con 0x7ff8a01065f0 2024-09-19T08:36:51.009 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.263+0000 7ff8a854c640 1 -- 172.21.15.164:0/471574391 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8a019adf0 con 0x7ff8a01065f0 2024-09-19T08:36:51.009 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.263+0000 7ff8977fe640 1 -- 172.21.15.164:0/471574391 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff89c037040 con 0x7ff8a01065f0 2024-09-19T08:36:51.009 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.263+0000 7ff8a854c640 1 -- 172.21.15.164:0/471574391 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8a019dae0 con 0x7ff8a01065f0 2024-09-19T08:36:51.009 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.264+0000 7ff8977fe640 1 -- 172.21.15.164:0/471574391 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7ff89c045070 con 0x7ff8a01065f0 2024-09-19T08:36:51.009 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.264+0000 7ff8977fe640 1 --2- 172.21.15.164:0/471574391 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7ff88c039200 0x7ff88c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:51.009 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.264+0000 7ff8977fe640 1 -- 172.21.15.164:0/471574391 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff89c06c1e0 con 0x7ff8a01065f0 2024-09-19T08:36:51.009 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.264+0000 7ff8a854c640 1 -- 172.21.15.164:0/471574391 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8a010ad20 con 0x7ff8a01065f0 2024-09-19T08:36:51.009 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.264+0000 7ff8a5ae7640 1 --2- 172.21.15.164:0/471574391 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7ff88c039200 0x7ff88c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:51.009 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.264+0000 7ff8a5ae7640 1 --2- 172.21.15.164:0/471574391 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7ff88c039200 0x7ff88c03b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff8900098c0 tx=0x7ff890006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:51.010 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.269+0000 7ff8977fe640 1 -- 172.21.15.164:0/471574391 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff89c045360 con 0x7ff8a01065f0 2024-09-19T08:36:51.010 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.452+0000 7ff8a854c640 1 -- 172.21.15.164:0/471574391 --> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7ff8a0103080 con 0x7ff88c039200 2024-09-19T08:36:51.010 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.460+0000 7ff8977fe640 1 -- 172.21.15.164:0/471574391 <== mgr.14118 v2:172.21.15.164:6800/1173822158 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff8a0103080 con 0x7ff88c039200 2024-09-19T08:36:51.010 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.462+0000 7ff8a854c640 1 -- 172.21.15.164:0/471574391 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7ff88c039200 msgr2=0x7ff88c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:51.010 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.462+0000 7ff8a854c640 1 --2- 172.21.15.164:0/471574391 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7ff88c039200 0x7ff88c03b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff8900098c0 tx=0x7ff890006db0 comp rx=0 tx=0).stop 2024-09-19T08:36:51.010 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.462+0000 7ff8a854c640 1 -- 172.21.15.164:0/471574391 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8a01065f0 msgr2=0x7ff8a019a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:51.010 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.462+0000 7ff8a854c640 1 --2- 172.21.15.164:0/471574391 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8a01065f0 0x7ff8a019a620 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff89c0097f0 tx=0x7ff89c030ec0 comp rx=0 tx=0).stop 2024-09-19T08:36:51.010 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.462+0000 7ff8a854c640 1 -- 172.21.15.164:0/471574391 shutdown_connections 2024-09-19T08:36:51.010 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.462+0000 7ff8a854c640 1 --2- 172.21.15.164:0/471574391 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7ff88c039200 0x7ff88c03b6c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:51.010 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.462+0000 7ff8a854c640 1 --2- 172.21.15.164:0/471574391 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8a01065f0 0x7ff8a019a620 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:51.010 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.462+0000 7ff8a854c640 1 -- 172.21.15.164:0/471574391 >> 172.21.15.164:0/471574391 conn(0x7ff8a0101e20 msgr2=0x7ff8a0102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:51.010 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.462+0000 7ff8a854c640 1 -- 172.21.15.164:0/471574391 shutdown_connections 2024-09-19T08:36:51.010 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:49.462+0000 7ff8a854c640 1 -- 172.21.15.164:0/471574391 wait complete. 2024-09-19T08:36:51.011 INFO:teuthology.orchestra.run.smithi164.stdout:Deploying unmanaged mgr service... 2024-09-19T08:36:52.013 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:51 smithi164 ceph-mon[30928]: from='mgr.14118 172.21.15.164:0/2031629681' entity='mgr.x' 2024-09-19T08:36:52.013 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:51 smithi164 ceph-mon[30928]: from='mgr.14118 172.21.15.164:0/2031629681' entity='mgr.x' 2024-09-19T08:36:52.874 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:52 smithi164 ceph-mon[30928]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:36:52.874 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:52 smithi164 ceph-mon[30928]: Saving service mgr spec with placement count:2 2024-09-19T08:36:52.875 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:52 smithi164 ceph-mon[30928]: from='mgr.14118 172.21.15.164:0/2031629681' entity='mgr.x' 2024-09-19T08:36:52.875 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:52 smithi164 ceph-mon[30928]: from='mgr.14118 172.21.15.164:0/2031629681' entity='mgr.x' 2024-09-19T08:36:52.882 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-19T08:36:52.883 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.388+0000 7f21331be640 1 Processor -- start 2024-09-19T08:36:52.883 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.388+0000 7f21331be640 1 -- start start 2024-09-19T08:36:52.883 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.388+0000 7f21331be640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f212c1065f0 0x7f212c1069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:52.883 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.388+0000 7f21331be640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f212c106fc0 con 0x7f212c1065f0 2024-09-19T08:36:52.883 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.388+0000 7f2130f5a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f212c1065f0 0x7f212c1069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:52.883 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.389+0000 7f2130f5a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f212c1065f0 0x7f212c1069f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40760/0 (socket says 172.21.15.164:40760) 2024-09-19T08:36:52.884 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.389+0000 7f2130f5a640 1 -- 172.21.15.164:0/3535826527 learned_addr learned my addr 172.21.15.164:0/3535826527 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:52.884 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.389+0000 7f2130f5a640 1 -- 172.21.15.164:0/3535826527 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f212c1077f0 con 0x7f212c1065f0 2024-09-19T08:36:52.884 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.390+0000 7f2130f5a640 1 --2- 172.21.15.164:0/3535826527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f212c1065f0 0x7f212c1069f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2120009820 tx=0x7f212002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a156c5a90a3d5089 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:52.884 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.390+0000 7f212b7fe640 1 -- 172.21.15.164:0/3535826527 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2120037070 con 0x7f212c1065f0 2024-09-19T08:36:52.885 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.390+0000 7f212b7fe640 1 -- 172.21.15.164:0/3535826527 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2120032920 con 0x7f212c1065f0 2024-09-19T08:36:52.885 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.390+0000 7f212b7fe640 1 -- 172.21.15.164:0/3535826527 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2120030810 con 0x7f212c1065f0 2024-09-19T08:36:52.885 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.391+0000 7f21331be640 1 -- 172.21.15.164:0/3535826527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f212c1065f0 msgr2=0x7f212c1069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:52.885 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.391+0000 7f21331be640 1 --2- 172.21.15.164:0/3535826527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f212c1065f0 0x7f212c1069f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2120009820 tx=0x7f212002a270 comp rx=0 tx=0).stop 2024-09-19T08:36:52.885 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.391+0000 7f21331be640 1 -- 172.21.15.164:0/3535826527 shutdown_connections 2024-09-19T08:36:52.885 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.391+0000 7f21331be640 1 --2- 172.21.15.164:0/3535826527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f212c1065f0 0x7f212c1069f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:52.885 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.391+0000 7f21331be640 1 -- 172.21.15.164:0/3535826527 >> 172.21.15.164:0/3535826527 conn(0x7f212c101e20 msgr2=0x7f212c104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:52.886 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.391+0000 7f21331be640 1 -- 172.21.15.164:0/3535826527 shutdown_connections 2024-09-19T08:36:52.886 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.391+0000 7f21331be640 1 -- 172.21.15.164:0/3535826527 wait complete. 2024-09-19T08:36:52.886 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.391+0000 7f21331be640 1 Processor -- start 2024-09-19T08:36:52.886 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.391+0000 7f21331be640 1 -- start start 2024-09-19T08:36:52.886 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.392+0000 7f21331be640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f212c1065f0 0x7f212c19b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:52.886 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.392+0000 7f21331be640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f212c10aba0 con 0x7f212c1065f0 2024-09-19T08:36:52.886 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.392+0000 7f2130f5a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f212c1065f0 0x7f212c19b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:52.886 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.392+0000 7f2130f5a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f212c1065f0 0x7f212c19b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40770/0 (socket says 172.21.15.164:40770) 2024-09-19T08:36:52.887 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.392+0000 7f2130f5a640 1 -- 172.21.15.164:0/2345731806 learned_addr learned my addr 172.21.15.164:0/2345731806 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:52.887 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.392+0000 7f2130f5a640 1 -- 172.21.15.164:0/2345731806 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f212c19bcc0 con 0x7f212c1065f0 2024-09-19T08:36:52.887 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.392+0000 7f2130f5a640 1 --2- 172.21.15.164:0/2345731806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f212c1065f0 0x7f212c19b780 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f21200097f0 tx=0x7f21200327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:52.887 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.392+0000 7f2129ffb640 1 -- 172.21.15.164:0/2345731806 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2120037040 con 0x7f212c1065f0 2024-09-19T08:36:52.887 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.392+0000 7f21331be640 1 -- 172.21.15.164:0/2345731806 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f212c19bf50 con 0x7f212c1065f0 2024-09-19T08:36:52.887 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.392+0000 7f2129ffb640 1 -- 172.21.15.164:0/2345731806 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2120030c40 con 0x7f212c1065f0 2024-09-19T08:36:52.887 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.392+0000 7f2129ffb640 1 -- 172.21.15.164:0/2345731806 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f212003a5b0 con 0x7f212c1065f0 2024-09-19T08:36:52.887 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.393+0000 7f21331be640 1 -- 172.21.15.164:0/2345731806 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f212c19ec40 con 0x7f212c1065f0 2024-09-19T08:36:52.888 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.393+0000 7f2129ffb640 1 -- 172.21.15.164:0/2345731806 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f212003a7f0 con 0x7f212c1065f0 2024-09-19T08:36:52.888 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.393+0000 7f2129ffb640 1 --2- 172.21.15.164:0/2345731806 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f211c0391b0 0x7f211c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:52.888 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.393+0000 7f2129ffb640 1 -- 172.21.15.164:0/2345731806 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f212006db30 con 0x7f212c1065f0 2024-09-19T08:36:52.888 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.394+0000 7f212bfff640 1 --2- 172.21.15.164:0/2345731806 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f211c0391b0 0x7f211c03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:52.888 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.394+0000 7f21331be640 1 -- 172.21.15.164:0/2345731806 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f212c10ad20 con 0x7f212c1065f0 2024-09-19T08:36:52.888 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.394+0000 7f212bfff640 1 --2- 172.21.15.164:0/2345731806 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f211c0391b0 0x7f211c03b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f21180098c0 tx=0x7f2118006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:52.888 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.399+0000 7f2129ffb640 1 -- 172.21.15.164:0/2345731806 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f21200371e0 con 0x7f212c1065f0 2024-09-19T08:36:52.889 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.620+0000 7f21331be640 1 -- 172.21.15.164:0/2345731806 --> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f212c02d8d0 con 0x7f211c0391b0 2024-09-19T08:36:52.889 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.631+0000 7f2129ffb640 1 -- 172.21.15.164:0/2345731806 <== mgr.14118 v2:172.21.15.164:6800/1173822158 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f212c02d8d0 con 0x7f211c0391b0 2024-09-19T08:36:52.889 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.634+0000 7f21331be640 1 -- 172.21.15.164:0/2345731806 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f211c0391b0 msgr2=0x7f211c03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:52.889 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.634+0000 7f21331be640 1 --2- 172.21.15.164:0/2345731806 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f211c0391b0 0x7f211c03b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f21180098c0 tx=0x7f2118006db0 comp rx=0 tx=0).stop 2024-09-19T08:36:52.889 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.634+0000 7f21331be640 1 -- 172.21.15.164:0/2345731806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f212c1065f0 msgr2=0x7f212c19b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:52.889 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.634+0000 7f21331be640 1 --2- 172.21.15.164:0/2345731806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f212c1065f0 0x7f212c19b780 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f21200097f0 tx=0x7f21200327b0 comp rx=0 tx=0).stop 2024-09-19T08:36:52.889 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.634+0000 7f21331be640 1 -- 172.21.15.164:0/2345731806 shutdown_connections 2024-09-19T08:36:52.889 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.634+0000 7f21331be640 1 --2- 172.21.15.164:0/2345731806 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f211c0391b0 0x7f211c03b670 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:52.889 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.634+0000 7f21331be640 1 --2- 172.21.15.164:0/2345731806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f212c1065f0 0x7f212c19b780 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:52.889 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.634+0000 7f21331be640 1 -- 172.21.15.164:0/2345731806 >> 172.21.15.164:0/2345731806 conn(0x7f212c101e20 msgr2=0x7f212c102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:52.890 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.634+0000 7f21331be640 1 -- 172.21.15.164:0/2345731806 shutdown_connections 2024-09-19T08:36:52.890 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:51.634+0000 7f21331be640 1 -- 172.21.15.164:0/2345731806 wait complete. 2024-09-19T08:36:54.082 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.336+0000 7fbd0d0ad640 1 Processor -- start 2024-09-19T08:36:54.083 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.336+0000 7fbd0d0ad640 1 -- start start 2024-09-19T08:36:54.083 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.336+0000 7fbd0d0ad640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd081081c0 0x7fbd0810a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:54.083 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.336+0000 7fbd0d0ad640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0807be50 con 0x7fbd081081c0 2024-09-19T08:36:54.083 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.336+0000 7fbd06d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd081081c0 0x7fbd0810a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:54.083 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.336+0000 7fbd06d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd081081c0 0x7fbd0810a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40782/0 (socket says 172.21.15.164:40782) 2024-09-19T08:36:54.083 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.336+0000 7fbd06d9d640 1 -- 172.21.15.164:0/1280655310 learned_addr learned my addr 172.21.15.164:0/1280655310 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:54.083 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.337+0000 7fbd06d9d640 1 -- 172.21.15.164:0/1280655310 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd0810aaf0 con 0x7fbd081081c0 2024-09-19T08:36:54.083 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.337+0000 7fbd06d9d640 1 --2- 172.21.15.164:0/1280655310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd081081c0 0x7fbd0810a5b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8009820 tx=0x7fbcf802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788f7152c075fee7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:54.083 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.337+0000 7fbd0659c640 1 -- 172.21.15.164:0/1280655310 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcf8037070 con 0x7fbd081081c0 2024-09-19T08:36:54.083 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.337+0000 7fbd0659c640 1 -- 172.21.15.164:0/1280655310 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcf8032920 con 0x7fbd081081c0 2024-09-19T08:36:54.084 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.337+0000 7fbd0659c640 1 -- 172.21.15.164:0/1280655310 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcf8030810 con 0x7fbd081081c0 2024-09-19T08:36:54.084 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.338+0000 7fbd0d0ad640 1 -- 172.21.15.164:0/1280655310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd081081c0 msgr2=0x7fbd0810a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:54.084 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.338+0000 7fbd0d0ad640 1 --2- 172.21.15.164:0/1280655310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd081081c0 0x7fbd0810a5b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8009820 tx=0x7fbcf802a270 comp rx=0 tx=0).stop 2024-09-19T08:36:54.084 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.338+0000 7fbd0d0ad640 1 -- 172.21.15.164:0/1280655310 shutdown_connections 2024-09-19T08:36:54.084 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.338+0000 7fbd0d0ad640 1 --2- 172.21.15.164:0/1280655310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd081081c0 0x7fbd0810a5b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:54.084 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.338+0000 7fbd0d0ad640 1 -- 172.21.15.164:0/1280655310 >> 172.21.15.164:0/1280655310 conn(0x7fbd08101e20 msgr2=0x7fbd08104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:54.084 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.338+0000 7fbd0d0ad640 1 -- 172.21.15.164:0/1280655310 shutdown_connections 2024-09-19T08:36:54.084 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.338+0000 7fbd0d0ad640 1 -- 172.21.15.164:0/1280655310 wait complete. 2024-09-19T08:36:54.084 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.339+0000 7fbd0d0ad640 1 Processor -- start 2024-09-19T08:36:54.084 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.339+0000 7fbd0d0ad640 1 -- start start 2024-09-19T08:36:54.084 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.339+0000 7fbd0d0ad640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd081081c0 0x7fbd08107a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:54.085 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.339+0000 7fbd0d0ad640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0810bb70 con 0x7fbd081081c0 2024-09-19T08:36:54.085 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.339+0000 7fbd06d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd081081c0 0x7fbd08107a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:54.085 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.339+0000 7fbd06d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd081081c0 0x7fbd08107a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40792/0 (socket says 172.21.15.164:40792) 2024-09-19T08:36:54.085 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.339+0000 7fbd06d9d640 1 -- 172.21.15.164:0/3483475143 learned_addr learned my addr 172.21.15.164:0/3483475143 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:54.085 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.340+0000 7fbd06d9d640 1 -- 172.21.15.164:0/3483475143 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd08107f80 con 0x7fbd081081c0 2024-09-19T08:36:54.085 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.340+0000 7fbd06d9d640 1 --2- 172.21.15.164:0/3483475143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd081081c0 0x7fbd08107a40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbcf80097f0 tx=0x7fbcf80327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:54.085 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.340+0000 7fbd04d99640 1 -- 172.21.15.164:0/3483475143 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcf8037040 con 0x7fbd081081c0 2024-09-19T08:36:54.085 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.340+0000 7fbd04d99640 1 -- 172.21.15.164:0/3483475143 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcf8030c40 con 0x7fbd081081c0 2024-09-19T08:36:54.085 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.340+0000 7fbd0d0ad640 1 -- 172.21.15.164:0/3483475143 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd08106170 con 0x7fbd081081c0 2024-09-19T08:36:54.086 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.340+0000 7fbd04d99640 1 -- 172.21.15.164:0/3483475143 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcf803a5d0 con 0x7fbd081081c0 2024-09-19T08:36:54.086 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.340+0000 7fbd0d0ad640 1 -- 172.21.15.164:0/3483475143 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd081066b0 con 0x7fbd081081c0 2024-09-19T08:36:54.086 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.341+0000 7fbd04d99640 1 -- 172.21.15.164:0/3483475143 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fbcf803b3f0 con 0x7fbd081081c0 2024-09-19T08:36:54.086 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.341+0000 7fbd04d99640 1 --2- 172.21.15.164:0/3483475143 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7fbcf4039200 0x7fbcf403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:54.086 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.341+0000 7fbd0d0ad640 1 -- 172.21.15.164:0/3483475143 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd08108620 con 0x7fbd081081c0 2024-09-19T08:36:54.086 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.341+0000 7fbd04d99640 1 -- 172.21.15.164:0/3483475143 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbcf806c840 con 0x7fbd081081c0 2024-09-19T08:36:54.086 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.341+0000 7fbcfffff640 1 --2- 172.21.15.164:0/3483475143 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7fbcf4039200 0x7fbcf403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:54.086 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.342+0000 7fbcfffff640 1 --2- 172.21.15.164:0/3483475143 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7fbcf4039200 0x7fbcf403b6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbcf00098c0 tx=0x7fbcf0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:54.086 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.346+0000 7fbd04d99640 1 -- 172.21.15.164:0/3483475143 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbcf80371e0 con 0x7fbd081081c0 2024-09-19T08:36:54.086 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.522+0000 7fbd0d0ad640 1 -- 172.21.15.164:0/3483475143 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fbd08106930 con 0x7fbd081081c0 2024-09-19T08:36:54.086 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.526+0000 7fbd04d99640 1 -- 172.21.15.164:0/3483475143 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fbcf8044df0 con 0x7fbd081081c0 2024-09-19T08:36:54.087 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.528+0000 7fbd0d0ad640 1 -- 172.21.15.164:0/3483475143 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7fbcf4039200 msgr2=0x7fbcf403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:54.087 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.528+0000 7fbd0d0ad640 1 --2- 172.21.15.164:0/3483475143 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7fbcf4039200 0x7fbcf403b6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbcf00098c0 tx=0x7fbcf0006db0 comp rx=0 tx=0).stop 2024-09-19T08:36:54.087 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.528+0000 7fbd0d0ad640 1 -- 172.21.15.164:0/3483475143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd081081c0 msgr2=0x7fbd08107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:54.087 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.528+0000 7fbd0d0ad640 1 --2- 172.21.15.164:0/3483475143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd081081c0 0x7fbd08107a40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbcf80097f0 tx=0x7fbcf80327b0 comp rx=0 tx=0).stop 2024-09-19T08:36:54.087 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.528+0000 7fbd0d0ad640 1 -- 172.21.15.164:0/3483475143 shutdown_connections 2024-09-19T08:36:54.087 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.528+0000 7fbd0d0ad640 1 --2- 172.21.15.164:0/3483475143 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7fbcf4039200 0x7fbcf403b6c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:54.087 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.528+0000 7fbd0d0ad640 1 --2- 172.21.15.164:0/3483475143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd081081c0 0x7fbd08107a40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:54.087 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.528+0000 7fbd0d0ad640 1 -- 172.21.15.164:0/3483475143 >> 172.21.15.164:0/3483475143 conn(0x7fbd08101e20 msgr2=0x7fbd08102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:54.087 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.528+0000 7fbd0d0ad640 1 -- 172.21.15.164:0/3483475143 shutdown_connections 2024-09-19T08:36:54.087 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:53.528+0000 7fbd0d0ad640 1 -- 172.21.15.164:0/3483475143 wait complete. 2024-09-19T08:36:54.453 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:54 smithi164 ceph-mon[30928]: from='mgr.14118 172.21.15.164:0/2031629681' entity='mgr.x' 2024-09-19T08:36:54.454 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:54 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3483475143' entity='client.admin' 2024-09-19T08:36:54.454 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:54 smithi164 ceph-mon[30928]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:36:55.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:55 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2941827278' entity='client.admin' 2024-09-19T08:36:56.072 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.531+0000 7f4243f45640 1 Processor -- start 2024-09-19T08:36:56.072 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.532+0000 7f4243f45640 1 -- start start 2024-09-19T08:36:56.072 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.532+0000 7f4243f45640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f423c1076f0 0x7f423c107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:56.072 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.532+0000 7f4243f45640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f423c1080c0 con 0x7f423c1076f0 2024-09-19T08:36:56.072 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.532+0000 7f4241ce1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f423c1076f0 0x7f423c107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:56.073 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.532+0000 7f4241ce1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f423c1076f0 0x7f423c107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50668/0 (socket says 172.21.15.164:50668) 2024-09-19T08:36:56.073 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.532+0000 7f4241ce1640 1 -- 172.21.15.164:0/1075624913 learned_addr learned my addr 172.21.15.164:0/1075624913 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:56.073 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.532+0000 7f4241ce1640 1 -- 172.21.15.164:0/1075624913 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f423c1088f0 con 0x7f423c1076f0 2024-09-19T08:36:56.073 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.533+0000 7f4241ce1640 1 --2- 172.21.15.164:0/1075624913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f423c1076f0 0x7f423c107af0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4230009820 tx=0x7f423002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c5973b1182373f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:56.073 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.533+0000 7f4240cdf640 1 -- 172.21.15.164:0/1075624913 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4230037070 con 0x7f423c1076f0 2024-09-19T08:36:56.073 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.533+0000 7f4240cdf640 1 -- 172.21.15.164:0/1075624913 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4230032920 con 0x7f423c1076f0 2024-09-19T08:36:56.073 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.533+0000 7f4240cdf640 1 -- 172.21.15.164:0/1075624913 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4230030810 con 0x7f423c1076f0 2024-09-19T08:36:56.074 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.534+0000 7f4243f45640 1 -- 172.21.15.164:0/1075624913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f423c1076f0 msgr2=0x7f423c107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:56.074 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.534+0000 7f4243f45640 1 --2- 172.21.15.164:0/1075624913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f423c1076f0 0x7f423c107af0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4230009820 tx=0x7f423002a270 comp rx=0 tx=0).stop 2024-09-19T08:36:56.074 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.534+0000 7f4243f45640 1 -- 172.21.15.164:0/1075624913 shutdown_connections 2024-09-19T08:36:56.074 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.534+0000 7f4243f45640 1 --2- 172.21.15.164:0/1075624913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f423c1076f0 0x7f423c107af0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:56.074 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.534+0000 7f4243f45640 1 -- 172.21.15.164:0/1075624913 >> 172.21.15.164:0/1075624913 conn(0x7f423c102ec0 msgr2=0x7f423c1052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:56.074 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.534+0000 7f4243f45640 1 -- 172.21.15.164:0/1075624913 shutdown_connections 2024-09-19T08:36:56.074 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.534+0000 7f4243f45640 1 -- 172.21.15.164:0/1075624913 wait complete. 2024-09-19T08:36:56.075 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.534+0000 7f4243f45640 1 Processor -- start 2024-09-19T08:36:56.075 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.534+0000 7f4243f45640 1 -- start start 2024-09-19T08:36:56.075 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.535+0000 7f4243f45640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f423c1076f0 0x7f423c19b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:56.075 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.535+0000 7f4243f45640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f423c10bca0 con 0x7f423c1076f0 2024-09-19T08:36:56.075 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.535+0000 7f4241ce1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f423c1076f0 0x7f423c19b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:56.075 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.535+0000 7f4241ce1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f423c1076f0 0x7f423c19b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50678/0 (socket says 172.21.15.164:50678) 2024-09-19T08:36:56.075 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.535+0000 7f4241ce1640 1 -- 172.21.15.164:0/2941827278 learned_addr learned my addr 172.21.15.164:0/2941827278 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:56.076 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.535+0000 7f4241ce1640 1 -- 172.21.15.164:0/2941827278 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f423c19bc30 con 0x7f423c1076f0 2024-09-19T08:36:56.076 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.535+0000 7f4241ce1640 1 --2- 172.21.15.164:0/2941827278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f423c1076f0 0x7f423c19b6f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4230030560 tx=0x7f4230032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:56.076 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.535+0000 7f422effd640 1 -- 172.21.15.164:0/2941827278 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4230037040 con 0x7f423c1076f0 2024-09-19T08:36:56.076 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.535+0000 7f422effd640 1 -- 172.21.15.164:0/2941827278 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4230040070 con 0x7f423c1076f0 2024-09-19T08:36:56.076 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.535+0000 7f4243f45640 1 -- 172.21.15.164:0/2941827278 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f423c19bec0 con 0x7f423c1076f0 2024-09-19T08:36:56.076 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.535+0000 7f422effd640 1 -- 172.21.15.164:0/2941827278 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f423003a740 con 0x7f423c1076f0 2024-09-19T08:36:56.077 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.535+0000 7f4243f45640 1 -- 172.21.15.164:0/2941827278 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f423c19ebb0 con 0x7f423c1076f0 2024-09-19T08:36:56.077 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.536+0000 7f422effd640 1 -- 172.21.15.164:0/2941827278 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f423003b420 con 0x7f423c1076f0 2024-09-19T08:36:56.077 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.536+0000 7f4243f45640 1 -- 172.21.15.164:0/2941827278 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f423c10be20 con 0x7f423c1076f0 2024-09-19T08:36:56.077 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.536+0000 7f422effd640 1 --2- 172.21.15.164:0/2941827278 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f42280391b0 0x7f422803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:56.077 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.536+0000 7f422effd640 1 -- 172.21.15.164:0/2941827278 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f423006c830 con 0x7f423c1076f0 2024-09-19T08:36:56.077 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.536+0000 7f42414e0640 1 --2- 172.21.15.164:0/2941827278 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f42280391b0 0x7f422803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:56.077 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.537+0000 7f42414e0640 1 --2- 172.21.15.164:0/2941827278 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f42280391b0 0x7f422803b670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f42380098c0 tx=0x7f4238006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:56.078 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.541+0000 7f422effd640 1 -- 172.21.15.164:0/2941827278 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f423004a200 con 0x7f423c1076f0 2024-09-19T08:36:56.078 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.715+0000 7f4243f45640 1 -- 172.21.15.164:0/2941827278 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f423c107b70 con 0x7f423c1076f0 2024-09-19T08:36:56.078 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.717+0000 7f422effd640 1 -- 172.21.15.164:0/2941827278 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f4230031e80 con 0x7f423c1076f0 2024-09-19T08:36:56.078 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.719+0000 7f4243f45640 1 -- 172.21.15.164:0/2941827278 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f42280391b0 msgr2=0x7f422803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:56.078 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.719+0000 7f4243f45640 1 --2- 172.21.15.164:0/2941827278 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f42280391b0 0x7f422803b670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f42380098c0 tx=0x7f4238006db0 comp rx=0 tx=0).stop 2024-09-19T08:36:56.078 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.719+0000 7f4243f45640 1 -- 172.21.15.164:0/2941827278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f423c1076f0 msgr2=0x7f423c19b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:56.079 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.719+0000 7f4243f45640 1 --2- 172.21.15.164:0/2941827278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f423c1076f0 0x7f423c19b6f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4230030560 tx=0x7f4230032670 comp rx=0 tx=0).stop 2024-09-19T08:36:56.079 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.719+0000 7f4243f45640 1 -- 172.21.15.164:0/2941827278 shutdown_connections 2024-09-19T08:36:56.079 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.719+0000 7f4243f45640 1 --2- 172.21.15.164:0/2941827278 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f42280391b0 0x7f422803b670 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:56.079 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.719+0000 7f4243f45640 1 --2- 172.21.15.164:0/2941827278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f423c1076f0 0x7f423c19b6f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:56.079 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.719+0000 7f4243f45640 1 -- 172.21.15.164:0/2941827278 >> 172.21.15.164:0/2941827278 conn(0x7f423c102ec0 msgr2=0x7f423c1039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:56.079 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.719+0000 7f4243f45640 1 -- 172.21.15.164:0/2941827278 shutdown_connections 2024-09-19T08:36:56.079 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:54.719+0000 7f4243f45640 1 -- 172.21.15.164:0/2941827278 wait complete. 2024-09-19T08:36:56.079 INFO:teuthology.orchestra.run.smithi164.stdout:Enabling the dashboard module... 2024-09-19T08:36:57.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:56 smithi164 ceph-mon[30928]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:36:57.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:56 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/52669797' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-19T08:36:57.032 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:56 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: ignoring --setuser ceph since I am not root 2024-09-19T08:36:57.032 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:56 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: ignoring --setgroup ceph since I am not root 2024-09-19T08:36:57.032 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:56 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:56.872+0000 7f02dc1f7040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T08:36:57.032 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:56 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:56.930+0000 7f02dc1f7040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T08:36:57.306 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.460+0000 7f15b76e3640 1 Processor -- start 2024-09-19T08:36:57.307 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.460+0000 7f15b76e3640 1 -- start start 2024-09-19T08:36:57.307 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.460+0000 7f15b76e3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15b0105d50 0x7f15b0106150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:57.307 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.460+0000 7f15b76e3640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f15b0106720 con 0x7f15b0105d50 2024-09-19T08:36:57.307 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.460+0000 7f15b547f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15b0105d50 0x7f15b0106150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:57.307 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.460+0000 7f15b547f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15b0105d50 0x7f15b0106150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50682/0 (socket says 172.21.15.164:50682) 2024-09-19T08:36:57.307 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.460+0000 7f15b547f640 1 -- 172.21.15.164:0/106043871 learned_addr learned my addr 172.21.15.164:0/106043871 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:57.308 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.461+0000 7f15b547f640 1 -- 172.21.15.164:0/106043871 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b0106f50 con 0x7f15b0105d50 2024-09-19T08:36:57.308 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.461+0000 7f15b547f640 1 --2- 172.21.15.164:0/106043871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15b0105d50 0x7f15b0106150 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00a9c0 tx=0x7f15ac02b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a21d775ff0a58a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:57.308 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.461+0000 7f15a7fff640 1 -- 172.21.15.164:0/106043871 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15ac00b040 con 0x7f15b0105d50 2024-09-19T08:36:57.308 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.461+0000 7f15a7fff640 1 -- 172.21.15.164:0/106043871 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15ac030b20 con 0x7f15b0105d50 2024-09-19T08:36:57.308 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.461+0000 7f15a7fff640 1 -- 172.21.15.164:0/106043871 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15ac032cb0 con 0x7f15b0105d50 2024-09-19T08:36:57.308 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.462+0000 7f15b76e3640 1 -- 172.21.15.164:0/106043871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15b0105d50 msgr2=0x7f15b0106150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:57.308 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.462+0000 7f15b76e3640 1 --2- 172.21.15.164:0/106043871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15b0105d50 0x7f15b0106150 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00a9c0 tx=0x7f15ac02b610 comp rx=0 tx=0).stop 2024-09-19T08:36:57.308 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.462+0000 7f15b76e3640 1 -- 172.21.15.164:0/106043871 shutdown_connections 2024-09-19T08:36:57.308 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.462+0000 7f15b76e3640 1 --2- 172.21.15.164:0/106043871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15b0105d50 0x7f15b0106150 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:57.308 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.462+0000 7f15b76e3640 1 -- 172.21.15.164:0/106043871 >> 172.21.15.164:0/106043871 conn(0x7f15b0101580 msgr2=0x7f15b01039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:57.308 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.462+0000 7f15b76e3640 1 -- 172.21.15.164:0/106043871 shutdown_connections 2024-09-19T08:36:57.308 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.462+0000 7f15b76e3640 1 -- 172.21.15.164:0/106043871 wait complete. 2024-09-19T08:36:57.309 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.463+0000 7f15b76e3640 1 Processor -- start 2024-09-19T08:36:57.309 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.463+0000 7f15b76e3640 1 -- start start 2024-09-19T08:36:57.309 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.463+0000 7f15b76e3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15b0105d50 0x7f15b019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:57.309 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.463+0000 7f15b76e3640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f15b0107790 con 0x7f15b0105d50 2024-09-19T08:36:57.309 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.463+0000 7f15b547f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15b0105d50 0x7f15b019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:57.309 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.463+0000 7f15b547f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15b0105d50 0x7f15b019abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50690/0 (socket says 172.21.15.164:50690) 2024-09-19T08:36:57.309 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.463+0000 7f15b547f640 1 -- 172.21.15.164:0/52669797 learned_addr learned my addr 172.21.15.164:0/52669797 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:57.309 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.463+0000 7f15b547f640 1 -- 172.21.15.164:0/52669797 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b019b130 con 0x7f15b0105d50 2024-09-19T08:36:57.309 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.463+0000 7f15b547f640 1 --2- 172.21.15.164:0/52669797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15b0105d50 0x7f15b019abf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f15ac0329c0 tx=0x7f15ac030510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:57.309 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.463+0000 7f15a67fc640 1 -- 172.21.15.164:0/52669797 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15ac03f070 con 0x7f15b0105d50 2024-09-19T08:36:57.309 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.464+0000 7f15a67fc640 1 -- 172.21.15.164:0/52669797 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15ac03b910 con 0x7f15b0105d50 2024-09-19T08:36:57.309 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.464+0000 7f15b76e3640 1 -- 172.21.15.164:0/52669797 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b0199320 con 0x7f15b0105d50 2024-09-19T08:36:57.309 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.464+0000 7f15a67fc640 1 -- 172.21.15.164:0/52669797 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15ac03a970 con 0x7f15b0105d50 2024-09-19T08:36:57.310 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.464+0000 7f15b76e3640 1 -- 172.21.15.164:0/52669797 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b0199860 con 0x7f15b0105d50 2024-09-19T08:36:57.310 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.464+0000 7f15a67fc640 1 -- 172.21.15.164:0/52669797 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f15ac042070 con 0x7f15b0105d50 2024-09-19T08:36:57.310 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.465+0000 7f15b76e3640 1 -- 172.21.15.164:0/52669797 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15b0106150 con 0x7f15b0105d50 2024-09-19T08:36:57.310 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.465+0000 7f15a67fc640 1 --2- 172.21.15.164:0/52669797 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f159c039200 0x7f159c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:57.310 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.465+0000 7f15a67fc640 1 -- 172.21.15.164:0/52669797 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f15ac06ce70 con 0x7f15b0105d50 2024-09-19T08:36:57.310 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.465+0000 7f15b4c7e640 1 --2- 172.21.15.164:0/52669797 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f159c039200 0x7f159c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:57.310 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.465+0000 7f15b4c7e640 1 --2- 172.21.15.164:0/52669797 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f159c039200 0x7f159c03b6c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f15a00098c0 tx=0x7f15a0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:57.310 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.469+0000 7f15a67fc640 1 -- 172.21.15.164:0/52669797 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f15ac040260 con 0x7f15b0105d50 2024-09-19T08:36:57.310 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.680+0000 7f15b76e3640 1 -- 172.21.15.164:0/52669797 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f15b0192a90 con 0x7f15b0105d50 2024-09-19T08:36:57.311 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.720+0000 7f15a67fc640 1 -- 172.21.15.164:0/52669797 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f15b0192a90 con 0x7f15b0105d50 2024-09-19T08:36:57.311 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.722+0000 7f15a67fc640 1 -- 172.21.15.164:0/52669797 <== mon.0 v2:172.21.15.164:3300/0 8 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f15ac03b430 con 0x7f15b0105d50 2024-09-19T08:36:57.311 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.724+0000 7f15b76e3640 1 -- 172.21.15.164:0/52669797 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f159c039200 msgr2=0x7f159c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:57.311 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.724+0000 7f15b76e3640 1 --2- 172.21.15.164:0/52669797 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f159c039200 0x7f159c03b6c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f15a00098c0 tx=0x7f15a0006db0 comp rx=0 tx=0).stop 2024-09-19T08:36:57.311 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.724+0000 7f15b76e3640 1 -- 172.21.15.164:0/52669797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15b0105d50 msgr2=0x7f15b019abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:57.311 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.724+0000 7f15b76e3640 1 --2- 172.21.15.164:0/52669797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15b0105d50 0x7f15b019abf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f15ac0329c0 tx=0x7f15ac030510 comp rx=0 tx=0).stop 2024-09-19T08:36:57.311 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.724+0000 7f15b76e3640 1 -- 172.21.15.164:0/52669797 shutdown_connections 2024-09-19T08:36:57.311 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.724+0000 7f15b76e3640 1 --2- 172.21.15.164:0/52669797 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f159c039200 0x7f159c03b6c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:57.311 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.724+0000 7f15b76e3640 1 --2- 172.21.15.164:0/52669797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15b0105d50 0x7f15b019abf0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:57.311 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.724+0000 7f15b76e3640 1 -- 172.21.15.164:0/52669797 >> 172.21.15.164:0/52669797 conn(0x7f15b0101580 msgr2=0x7f15b01901f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:57.311 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.724+0000 7f15b76e3640 1 -- 172.21.15.164:0/52669797 shutdown_connections 2024-09-19T08:36:57.311 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:56.724+0000 7f15b76e3640 1 -- 172.21.15.164:0/52669797 wait complete. 2024-09-19T08:36:57.425 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:57 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:57.081+0000 7f02dc1f7040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T08:36:57.425 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:57 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:57.147+0000 7f02dc1f7040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T08:36:57.426 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:57 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:57.296+0000 7f02dc1f7040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:36:57.694 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:57 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:57.443+0000 7f02dc1f7040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T08:36:57.979 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:57 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/52669797' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-19T08:36:57.980 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:57 smithi164 ceph-mon[30928]: mgrmap e9: x(active, since 22s) 2024-09-19T08:36:57.980 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:57 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:57.768+0000 7f02dc1f7040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T08:36:57.980 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:57 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:57.832+0000 7f02dc1f7040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T08:36:58.282 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:58 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:58.026+0000 7f02dc1f7040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T08:36:58.558 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout { 2024-09-19T08:36:58.566 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-19T08:36:58.566 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T08:36:58.566 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T08:36:58.566 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T08:36:58.566 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout } 2024-09-19T08:36:58.566 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.772+0000 7f55f68b3640 1 Processor -- start 2024-09-19T08:36:58.566 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.772+0000 7f55f68b3640 1 -- start start 2024-09-19T08:36:58.566 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.772+0000 7f55f68b3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f01081a0 0x7f55f010a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:58.566 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.772+0000 7f55f68b3640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f55f007be70 con 0x7f55f01081a0 2024-09-19T08:36:58.567 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.773+0000 7f55effff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f01081a0 0x7f55f010a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:58.567 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.773+0000 7f55effff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f01081a0 0x7f55f010a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50722/0 (socket says 172.21.15.164:50722) 2024-09-19T08:36:58.567 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.773+0000 7f55effff640 1 -- 172.21.15.164:0/1603548836 learned_addr learned my addr 172.21.15.164:0/1603548836 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:58.567 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.773+0000 7f55effff640 1 -- 172.21.15.164:0/1603548836 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f010aad0 con 0x7f55f01081a0 2024-09-19T08:36:58.567 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.774+0000 7f55effff640 1 --2- 172.21.15.164:0/1603548836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f01081a0 0x7f55f010a590 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f55e0009820 tx=0x7f55e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8b4d0652b20b1e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:58.567 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.774+0000 7f55ef7fe640 1 -- 172.21.15.164:0/1603548836 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55e0037070 con 0x7f55f01081a0 2024-09-19T08:36:58.567 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.774+0000 7f55ef7fe640 1 -- 172.21.15.164:0/1603548836 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55e0032920 con 0x7f55f01081a0 2024-09-19T08:36:58.567 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.774+0000 7f55ef7fe640 1 -- 172.21.15.164:0/1603548836 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55e0030810 con 0x7f55f01081a0 2024-09-19T08:36:58.567 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.775+0000 7f55f68b3640 1 -- 172.21.15.164:0/1603548836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f01081a0 msgr2=0x7f55f010a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:58.567 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.775+0000 7f55f68b3640 1 --2- 172.21.15.164:0/1603548836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f01081a0 0x7f55f010a590 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f55e0009820 tx=0x7f55e002a270 comp rx=0 tx=0).stop 2024-09-19T08:36:58.567 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.775+0000 7f55f68b3640 1 -- 172.21.15.164:0/1603548836 shutdown_connections 2024-09-19T08:36:58.567 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.775+0000 7f55f68b3640 1 --2- 172.21.15.164:0/1603548836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f01081a0 0x7f55f010a590 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:58.567 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.775+0000 7f55f68b3640 1 -- 172.21.15.164:0/1603548836 >> 172.21.15.164:0/1603548836 conn(0x7f55f0101e20 msgr2=0x7f55f0104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:58.568 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.775+0000 7f55f68b3640 1 -- 172.21.15.164:0/1603548836 shutdown_connections 2024-09-19T08:36:58.568 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.775+0000 7f55f68b3640 1 -- 172.21.15.164:0/1603548836 wait complete. 2024-09-19T08:36:58.568 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.775+0000 7f55f68b3640 1 Processor -- start 2024-09-19T08:36:58.568 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.775+0000 7f55f68b3640 1 -- start start 2024-09-19T08:36:58.569 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.776+0000 7f55f68b3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f01081a0 0x7f55f019b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:58.569 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.776+0000 7f55f68b3640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f55f010bb50 con 0x7f55f01081a0 2024-09-19T08:36:58.569 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.776+0000 7f55effff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f01081a0 0x7f55f019b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:36:58.569 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.776+0000 7f55effff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f01081a0 0x7f55f019b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50728/0 (socket says 172.21.15.164:50728) 2024-09-19T08:36:58.569 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.776+0000 7f55effff640 1 -- 172.21.15.164:0/2942138185 learned_addr learned my addr 172.21.15.164:0/2942138185 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:36:58.569 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.776+0000 7f55effff640 1 -- 172.21.15.164:0/2942138185 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f019bc50 con 0x7f55f01081a0 2024-09-19T08:36:58.569 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.776+0000 7f55effff640 1 --2- 172.21.15.164:0/2942138185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f01081a0 0x7f55f019b710 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f55e00097f0 tx=0x7f55e00327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:36:58.569 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.776+0000 7f55edffb640 1 -- 172.21.15.164:0/2942138185 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55e0037040 con 0x7f55f01081a0 2024-09-19T08:36:58.569 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.776+0000 7f55edffb640 1 -- 172.21.15.164:0/2942138185 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55e0030c40 con 0x7f55f01081a0 2024-09-19T08:36:58.569 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.776+0000 7f55f68b3640 1 -- 172.21.15.164:0/2942138185 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f019bee0 con 0x7f55f01081a0 2024-09-19T08:36:58.570 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.777+0000 7f55edffb640 1 -- 172.21.15.164:0/2942138185 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55e003a5d0 con 0x7f55f01081a0 2024-09-19T08:36:58.570 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.777+0000 7f55f68b3640 1 -- 172.21.15.164:0/2942138185 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f019ebd0 con 0x7f55f01081a0 2024-09-19T08:36:58.570 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.777+0000 7f55edffb640 1 -- 172.21.15.164:0/2942138185 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f55e003a870 con 0x7f55f01081a0 2024-09-19T08:36:58.570 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.778+0000 7f55f68b3640 1 -- 172.21.15.164:0/2942138185 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55f0108600 con 0x7f55f01081a0 2024-09-19T08:36:58.570 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.778+0000 7f55edffb640 1 --2- 172.21.15.164:0/2942138185 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f55dc039200 0x7f55dc03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:36:58.570 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.778+0000 7f55e7fff640 1 -- 172.21.15.164:0/2942138185 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f55dc039200 msgr2=0x7f55dc03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.164:6800/1173822158 2024-09-19T08:36:58.570 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.778+0000 7f55edffb640 1 -- 172.21.15.164:0/2942138185 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f55e006c910 con 0x7f55f01081a0 2024-09-19T08:36:58.570 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.778+0000 7f55e7fff640 1 --2- 172.21.15.164:0/2942138185 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f55dc039200 0x7f55dc03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T08:36:58.570 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.782+0000 7f55edffb640 1 -- 172.21.15.164:0/2942138185 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f55e00319f0 con 0x7f55f01081a0 2024-09-19T08:36:58.570 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.979+0000 7f55e7fff640 1 -- 172.21.15.164:0/2942138185 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f55dc039200 msgr2=0x7f55dc03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.164:6800/1173822158 2024-09-19T08:36:58.570 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:57.979+0000 7f55e7fff640 1 --2- 172.21.15.164:0/2942138185 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f55dc039200 0x7f55dc03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T08:36:58.570 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.004+0000 7f55f68b3640 1 -- 172.21.15.164:0/2942138185 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f55f019f1c0 con 0x7f55f01081a0 2024-09-19T08:36:58.571 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.005+0000 7f55edffb640 1 -- 172.21.15.164:0/2942138185 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f55e0031c50 con 0x7f55f01081a0 2024-09-19T08:36:58.571 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.007+0000 7f55f68b3640 1 -- 172.21.15.164:0/2942138185 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f55dc039200 msgr2=0x7f55dc03b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:36:58.571 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.007+0000 7f55f68b3640 1 --2- 172.21.15.164:0/2942138185 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f55dc039200 0x7f55dc03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:58.571 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.007+0000 7f55f68b3640 1 -- 172.21.15.164:0/2942138185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f01081a0 msgr2=0x7f55f019b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:36:58.571 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.007+0000 7f55f68b3640 1 --2- 172.21.15.164:0/2942138185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f01081a0 0x7f55f019b710 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f55e00097f0 tx=0x7f55e00327b0 comp rx=0 tx=0).stop 2024-09-19T08:36:58.571 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.007+0000 7f55f68b3640 1 -- 172.21.15.164:0/2942138185 shutdown_connections 2024-09-19T08:36:58.571 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.007+0000 7f55f68b3640 1 --2- 172.21.15.164:0/2942138185 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f55dc039200 0x7f55dc03b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:58.571 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.007+0000 7f55f68b3640 1 --2- 172.21.15.164:0/2942138185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f01081a0 0x7f55f019b710 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:36:58.572 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.007+0000 7f55f68b3640 1 -- 172.21.15.164:0/2942138185 >> 172.21.15.164:0/2942138185 conn(0x7f55f0101e20 msgr2=0x7f55f0102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:36:58.572 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.007+0000 7f55f68b3640 1 -- 172.21.15.164:0/2942138185 shutdown_connections 2024-09-19T08:36:58.572 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.007+0000 7f55f68b3640 1 -- 172.21.15.164:0/2942138185 wait complete. 2024-09-19T08:36:58.572 INFO:teuthology.orchestra.run.smithi164.stdout:Waiting for the mgr to restart... 2024-09-19T08:36:58.572 INFO:teuthology.orchestra.run.smithi164.stdout:Waiting for mgr epoch 9... 2024-09-19T08:36:58.883 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:36:58 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2942138185' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T08:36:59.169 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:58 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:58.940+0000 7f02dc1f7040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T08:36:59.170 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:59 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:59.011+0000 7f02dc1f7040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T08:36:59.839 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:59 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:59.724+0000 7f02dc1f7040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T08:37:00.188 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:59 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:59.838+0000 7f02dc1f7040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T08:37:00.189 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:36:59 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:36:59.982+0000 7f02dc1f7040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:37:00.189 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:37:00 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:37:00.043+0000 7f02dc1f7040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T08:37:00.446 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:37:00 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:37:00.194+0000 7f02dc1f7040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T08:37:00.446 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:37:00 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:37:00.266+0000 7f02dc1f7040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T08:37:00.446 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:37:00 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:37:00.329+0000 7f02dc1f7040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T08:37:00.722 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:37:00 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:37:00.496+0000 7f02dc1f7040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T08:37:00.723 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:37:00 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:37:00.563+0000 7f02dc1f7040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T08:37:00.723 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:37:00 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:37:00.634+0000 7f02dc1f7040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T08:37:00.723 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:37:00 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:37:00.721+0000 7f02dc1f7040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T08:37:01.032 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:37:00 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:37:00.784+0000 7f02dc1f7040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T08:37:01.032 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:37:00 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:37:00.846+0000 7f02dc1f7040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T08:37:01.714 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:37:01 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:37:01.660+0000 7f02dc1f7040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T08:37:02.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:01 smithi164 ceph-mon[30928]: Active manager daemon x restarted 2024-09-19T08:37:02.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:01 smithi164 ceph-mon[30928]: Activating manager daemon x 2024-09-19T08:37:02.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:01 smithi164 ceph-mon[30928]: osdmap e3: 0 total, 0 up, 0 in 2024-09-19T08:37:02.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:01 smithi164 ceph-mon[30928]: mgrmap e10: x(active, starting, since 0.00335807s) 2024-09-19T08:37:02.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:01 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:37:02.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:01 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T08:37:02.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:01 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T08:37:02.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:01 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T08:37:02.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:01 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T08:37:02.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:01 smithi164 ceph-mon[30928]: Manager daemon x is now available 2024-09-19T08:37:02.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:01 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:37:02.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:01 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T08:37:02.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:01 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T08:37:03.399 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout { 2024-09-19T08:37:03.399 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-19T08:37:03.399 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T08:37:03.399 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout } 2024-09-19T08:37:03.400 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.962+0000 7f9dd2fde640 1 Processor -- start 2024-09-19T08:37:03.400 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.962+0000 7f9dd2fde640 1 -- start start 2024-09-19T08:37:03.400 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.962+0000 7f9dd2fde640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9dcc106620 0x7f9dcc106a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:03.400 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.962+0000 7f9dd2fde640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc106ff0 con 0x7f9dcc106620 2024-09-19T08:37:03.400 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.962+0000 7f9dd0d7a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9dcc106620 0x7f9dcc106a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:03.400 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.962+0000 7f9dd0d7a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9dcc106620 0x7f9dcc106a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50740/0 (socket says 172.21.15.164:50740) 2024-09-19T08:37:03.400 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.963+0000 7f9dd0d7a640 1 -- 172.21.15.164:0/4036008479 learned_addr learned my addr 172.21.15.164:0/4036008479 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:37:03.400 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.963+0000 7f9dd0d7a640 1 -- 172.21.15.164:0/4036008479 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dcc1077d0 con 0x7f9dcc106620 2024-09-19T08:37:03.400 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.963+0000 7f9dd0d7a640 1 --2- 172.21.15.164:0/4036008479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9dcc106620 0x7f9dcc106a20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0009820 tx=0x7f9dc002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99da012507c886f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:03.400 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.964+0000 7f9dcb7fe640 1 -- 172.21.15.164:0/4036008479 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dc0037070 con 0x7f9dcc106620 2024-09-19T08:37:03.400 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.964+0000 7f9dcb7fe640 1 -- 172.21.15.164:0/4036008479 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dc0032920 con 0x7f9dcc106620 2024-09-19T08:37:03.400 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.964+0000 7f9dcb7fe640 1 -- 172.21.15.164:0/4036008479 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dc0030810 con 0x7f9dcc106620 2024-09-19T08:37:03.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.964+0000 7f9dd2fde640 1 -- 172.21.15.164:0/4036008479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9dcc106620 msgr2=0x7f9dcc106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:03.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.964+0000 7f9dd2fde640 1 --2- 172.21.15.164:0/4036008479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9dcc106620 0x7f9dcc106a20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0009820 tx=0x7f9dc002a270 comp rx=0 tx=0).stop 2024-09-19T08:37:03.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.964+0000 7f9dd2fde640 1 -- 172.21.15.164:0/4036008479 shutdown_connections 2024-09-19T08:37:03.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.964+0000 7f9dd2fde640 1 --2- 172.21.15.164:0/4036008479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9dcc106620 0x7f9dcc106a20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:03.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.964+0000 7f9dd2fde640 1 -- 172.21.15.164:0/4036008479 >> 172.21.15.164:0/4036008479 conn(0x7f9dcc101db0 msgr2=0x7f9dcc1041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:03.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.964+0000 7f9dd2fde640 1 -- 172.21.15.164:0/4036008479 shutdown_connections 2024-09-19T08:37:03.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.964+0000 7f9dd2fde640 1 -- 172.21.15.164:0/4036008479 wait complete. 2024-09-19T08:37:03.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.965+0000 7f9dd2fde640 1 Processor -- start 2024-09-19T08:37:03.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.965+0000 7f9dd2fde640 1 -- start start 2024-09-19T08:37:03.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.965+0000 7f9dd2fde640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9dcc106620 0x7f9dcc07ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:03.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.965+0000 7f9dd2fde640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc10ab80 con 0x7f9dcc106620 2024-09-19T08:37:03.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.965+0000 7f9dd0d7a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9dcc106620 0x7f9dcc07ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:03.401 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.965+0000 7f9dd0d7a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9dcc106620 0x7f9dcc07ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50746/0 (socket says 172.21.15.164:50746) 2024-09-19T08:37:03.402 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.965+0000 7f9dd0d7a640 1 -- 172.21.15.164:0/2291356131 learned_addr learned my addr 172.21.15.164:0/2291356131 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:37:03.402 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.966+0000 7f9dd0d7a640 1 -- 172.21.15.164:0/2291356131 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dcc19e4c0 con 0x7f9dcc106620 2024-09-19T08:37:03.402 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.966+0000 7f9dd0d7a640 1 --2- 172.21.15.164:0/2291356131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9dcc106620 0x7f9dcc07ba50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0030560 tx=0x7f9dc0032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:03.402 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.966+0000 7f9dc9ffb640 1 -- 172.21.15.164:0/2291356131 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dc0037040 con 0x7f9dcc106620 2024-09-19T08:37:03.402 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.966+0000 7f9dc9ffb640 1 -- 172.21.15.164:0/2291356131 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dc0040070 con 0x7f9dcc106620 2024-09-19T08:37:03.402 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.966+0000 7f9dd2fde640 1 -- 172.21.15.164:0/2291356131 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dcc19e690 con 0x7f9dcc106620 2024-09-19T08:37:03.402 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.966+0000 7f9dc9ffb640 1 -- 172.21.15.164:0/2291356131 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dc003a6d0 con 0x7f9dcc106620 2024-09-19T08:37:03.402 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.966+0000 7f9dd2fde640 1 -- 172.21.15.164:0/2291356131 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dcc19ea30 con 0x7f9dcc106620 2024-09-19T08:37:03.402 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.967+0000 7f9dc9ffb640 1 -- 172.21.15.164:0/2291356131 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f9dc003a940 con 0x7f9dcc106620 2024-09-19T08:37:03.402 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.968+0000 7f9dc9ffb640 1 --2- 172.21.15.164:0/2291356131 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f9dbc039200 0x7f9dbc03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:03.402 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.968+0000 7f9dc9ffb640 1 -- 172.21.15.164:0/2291356131 --> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9dbc03bdd0 con 0x7f9dbc039200 2024-09-19T08:37:03.402 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.968+0000 7f9dc9ffb640 1 -- 172.21.15.164:0/2291356131 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9dc006c860 con 0x7f9dcc106620 2024-09-19T08:37:03.403 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.968+0000 7f9dcbfff640 1 -- 172.21.15.164:0/2291356131 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f9dbc039200 msgr2=0x7f9dbc03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.164:6800/1173822158 2024-09-19T08:37:03.403 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:58.968+0000 7f9dcbfff640 1 --2- 172.21.15.164:0/2291356131 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f9dbc039200 0x7f9dbc03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T08:37:03.403 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:59.169+0000 7f9dcbfff640 1 -- 172.21.15.164:0/2291356131 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f9dbc039200 msgr2=0x7f9dbc03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.164:6800/1173822158 2024-09-19T08:37:03.403 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:59.169+0000 7f9dcbfff640 1 --2- 172.21.15.164:0/2291356131 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f9dbc039200 0x7f9dbc03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T08:37:03.403 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:59.570+0000 7f9dcbfff640 1 -- 172.21.15.164:0/2291356131 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f9dbc039200 msgr2=0x7f9dbc03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.164:6800/1173822158 2024-09-19T08:37:03.403 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:36:59.570+0000 7f9dcbfff640 1 --2- 172.21.15.164:0/2291356131 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f9dbc039200 0x7f9dbc03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T08:37:03.403 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:00.371+0000 7f9dcbfff640 1 -- 172.21.15.164:0/2291356131 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f9dbc039200 msgr2=0x7f9dbc03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.164:6800/1173822158 2024-09-19T08:37:03.403 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:00.371+0000 7f9dcbfff640 1 --2- 172.21.15.164:0/2291356131 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f9dbc039200 0x7f9dbc03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T08:37:03.403 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:01.663+0000 7f9dc9ffb640 1 -- 172.21.15.164:0/2291356131 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mgrmap(e 10) ==== 46241+0+0 (secure 0 0 0) 0x7f9dc006b310 con 0x7f9dcc106620 2024-09-19T08:37:03.403 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:01.663+0000 7f9dc9ffb640 1 -- 172.21.15.164:0/2291356131 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f9dbc039200 msgr2=0x7f9dbc03b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:37:03.403 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:01.663+0000 7f9dc9ffb640 1 --2- 172.21.15.164:0/2291356131 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f9dbc039200 0x7f9dbc03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:03.403 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:02.665+0000 7f9dc9ffb640 1 -- 172.21.15.164:0/2291356131 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7f9dc0045460 con 0x7f9dcc106620 2024-09-19T08:37:03.404 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:02.665+0000 7f9dc9ffb640 1 --2- 172.21.15.164:0/2291356131 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f9dbc03ccc0 0x7f9dbc03f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:03.404 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:02.665+0000 7f9dc9ffb640 1 -- 172.21.15.164:0/2291356131 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9dbc03bdd0 con 0x7f9dbc03ccc0 2024-09-19T08:37:03.404 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:02.665+0000 7f9dcbfff640 1 --2- 172.21.15.164:0/2291356131 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f9dbc03ccc0 0x7f9dbc03f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:03.404 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:02.666+0000 7f9dcbfff640 1 --2- 172.21.15.164:0/2291356131 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f9dbc03ccc0 0x7f9dbc03f0b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9db8003980 tx=0x7f9db8006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:03.404 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:02.667+0000 7f9dc9ffb640 1 -- 172.21.15.164:0/2291356131 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f9dbc03bdd0 con 0x7f9dbc03ccc0 2024-09-19T08:37:03.404 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:02.675+0000 7f9dd2fde640 1 -- 172.21.15.164:0/2291356131 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f9dcc10ad00 con 0x7f9dbc03ccc0 2024-09-19T08:37:03.404 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:02.675+0000 7f9dc9ffb640 1 -- 172.21.15.164:0/2291356131 <== mgr.14150 v2:172.21.15.164:6800/1094864383 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f9dcc10ad00 con 0x7f9dbc03ccc0 2024-09-19T08:37:03.404 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:02.676+0000 7f9dd2fde640 1 -- 172.21.15.164:0/2291356131 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f9dbc03ccc0 msgr2=0x7f9dbc03f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:03.404 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:02.676+0000 7f9dd2fde640 1 --2- 172.21.15.164:0/2291356131 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f9dbc03ccc0 0x7f9dbc03f0b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9db8003980 tx=0x7f9db8006e10 comp rx=0 tx=0).stop 2024-09-19T08:37:03.404 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:02.676+0000 7f9dd2fde640 1 -- 172.21.15.164:0/2291356131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9dcc106620 msgr2=0x7f9dcc07ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:03.404 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:02.676+0000 7f9dd2fde640 1 --2- 172.21.15.164:0/2291356131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9dcc106620 0x7f9dcc07ba50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0030560 tx=0x7f9dc0032670 comp rx=0 tx=0).stop 2024-09-19T08:37:03.404 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:02.676+0000 7f9dd2fde640 1 -- 172.21.15.164:0/2291356131 shutdown_connections 2024-09-19T08:37:03.405 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:02.676+0000 7f9dd2fde640 1 --2- 172.21.15.164:0/2291356131 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f9dbc03ccc0 0x7f9dbc03f0b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:03.405 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:02.676+0000 7f9dd2fde640 1 --2- 172.21.15.164:0/2291356131 >> [v2:172.21.15.164:6800/1173822158,v1:172.21.15.164:6801/1173822158] conn(0x7f9dbc039200 0x7f9dbc03b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:03.405 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:02.676+0000 7f9dd2fde640 1 --2- 172.21.15.164:0/2291356131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9dcc106620 0x7f9dcc07ba50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:03.405 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:02.676+0000 7f9dd2fde640 1 -- 172.21.15.164:0/2291356131 >> 172.21.15.164:0/2291356131 conn(0x7f9dcc101db0 msgr2=0x7f9dcc102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:03.405 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:02.676+0000 7f9dd2fde640 1 -- 172.21.15.164:0/2291356131 shutdown_connections 2024-09-19T08:37:03.405 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:02.676+0000 7f9dd2fde640 1 -- 172.21.15.164:0/2291356131 wait complete. 2024-09-19T08:37:03.405 INFO:teuthology.orchestra.run.smithi164.stdout:mgr epoch 9 is available 2024-09-19T08:37:03.405 INFO:teuthology.orchestra.run.smithi164.stdout:Generating a dashboard self-signed certificate... 2024-09-19T08:37:03.666 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:37:03 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:37:03.584+0000 7f0241cc0640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:08:37:03] ENGINE Error in HTTPServer.serve 2024-09-19T08:37:03.666 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:37:03 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: Traceback (most recent call last): 2024-09-19T08:37:03.666 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:37:03 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T08:37:03.666 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:37:03 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: self._connections.run(self.expiration_interval) 2024-09-19T08:37:03.666 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:37:03 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T08:37:03.667 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:37:03 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: self._run(expiration_interval) 2024-09-19T08:37:03.667 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:37:03 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T08:37:03.667 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:37:03 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T08:37:03.667 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:37:03 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T08:37:03.667 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:37:03 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T08:37:03.667 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:37:03 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T08:37:03.667 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:37:03 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: s = self.context.wrap_socket( 2024-09-19T08:37:03.667 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:37:03 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T08:37:03.668 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:37:03 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: return self.sslsocket_class._create( 2024-09-19T08:37:03.668 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:37:03 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T08:37:03.668 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:37:03 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: self.do_handshake() 2024-09-19T08:37:03.668 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:37:03 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T08:37:03.668 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:37:03 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: self._sslobj.do_handshake() 2024-09-19T08:37:03.668 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:37:03 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T08:37:03.668 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:37:03 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:37:04.027 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:03 smithi164 ceph-mon[30928]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-19T08:37:04.027 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:03 smithi164 ceph-mon[30928]: mgrmap e11: x(active, since 1.006s) 2024-09-19T08:37:04.027 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:03 smithi164 ceph-mon[30928]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-19T08:37:04.663 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-19T08:37:04.663 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.840+0000 7f9c6ecca640 1 Processor -- start 2024-09-19T08:37:04.663 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.840+0000 7f9c6ecca640 1 -- start start 2024-09-19T08:37:04.663 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.840+0000 7f9c6ecca640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c681081c0 0x7f9c6810a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:04.663 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.840+0000 7f9c6ecca640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6807be50 con 0x7f9c681081c0 2024-09-19T08:37:04.663 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.840+0000 7f9c6ca66640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c681081c0 0x7f9c6810a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:04.663 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.840+0000 7f9c6ca66640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c681081c0 0x7f9c6810a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40728/0 (socket says 172.21.15.164:40728) 2024-09-19T08:37:04.663 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.840+0000 7f9c6ca66640 1 -- 172.21.15.164:0/929082860 learned_addr learned my addr 172.21.15.164:0/929082860 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:37:04.663 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.840+0000 7f9c6ca66640 1 -- 172.21.15.164:0/929082860 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c6810aaf0 con 0x7f9c681081c0 2024-09-19T08:37:04.664 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.841+0000 7f9c6ca66640 1 --2- 172.21.15.164:0/929082860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c681081c0 0x7f9c6810a5b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c009820 tx=0x7f9c5c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ba4dbaffa4f42b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:04.664 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.841+0000 7f9c677fe640 1 -- 172.21.15.164:0/929082860 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c5c037070 con 0x7f9c681081c0 2024-09-19T08:37:04.664 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.841+0000 7f9c677fe640 1 -- 172.21.15.164:0/929082860 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c5c032920 con 0x7f9c681081c0 2024-09-19T08:37:04.664 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.841+0000 7f9c677fe640 1 -- 172.21.15.164:0/929082860 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c5c030810 con 0x7f9c681081c0 2024-09-19T08:37:04.664 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.841+0000 7f9c6ecca640 1 -- 172.21.15.164:0/929082860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c681081c0 msgr2=0x7f9c6810a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:04.664 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.841+0000 7f9c6ecca640 1 --2- 172.21.15.164:0/929082860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c681081c0 0x7f9c6810a5b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c009820 tx=0x7f9c5c02a270 comp rx=0 tx=0).stop 2024-09-19T08:37:04.664 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.842+0000 7f9c6ecca640 1 -- 172.21.15.164:0/929082860 shutdown_connections 2024-09-19T08:37:04.664 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.842+0000 7f9c6ecca640 1 --2- 172.21.15.164:0/929082860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c681081c0 0x7f9c6810a5b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:04.664 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.842+0000 7f9c6ecca640 1 -- 172.21.15.164:0/929082860 >> 172.21.15.164:0/929082860 conn(0x7f9c68101e20 msgr2=0x7f9c68104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:04.664 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.842+0000 7f9c6ecca640 1 -- 172.21.15.164:0/929082860 shutdown_connections 2024-09-19T08:37:04.664 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.842+0000 7f9c6ecca640 1 -- 172.21.15.164:0/929082860 wait complete. 2024-09-19T08:37:04.664 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.842+0000 7f9c6ecca640 1 Processor -- start 2024-09-19T08:37:04.665 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.842+0000 7f9c6ecca640 1 -- start start 2024-09-19T08:37:04.665 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.842+0000 7f9c6ecca640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c681081c0 0x7f9c6819b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:04.665 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.842+0000 7f9c6ecca640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6810bb70 con 0x7f9c681081c0 2024-09-19T08:37:04.665 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.842+0000 7f9c6ca66640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c681081c0 0x7f9c6819b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:04.665 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.842+0000 7f9c6ca66640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c681081c0 0x7f9c6819b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40736/0 (socket says 172.21.15.164:40736) 2024-09-19T08:37:04.665 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.842+0000 7f9c6ca66640 1 -- 172.21.15.164:0/424575642 learned_addr learned my addr 172.21.15.164:0/424575642 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:37:04.665 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.843+0000 7f9c6ca66640 1 -- 172.21.15.164:0/424575642 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c6819bc40 con 0x7f9c681081c0 2024-09-19T08:37:04.665 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.843+0000 7f9c6ca66640 1 --2- 172.21.15.164:0/424575642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c681081c0 0x7f9c6819b700 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c030420 tx=0x7f9c5c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:04.665 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.843+0000 7f9c65ffb640 1 -- 172.21.15.164:0/424575642 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c5c037040 con 0x7f9c681081c0 2024-09-19T08:37:04.665 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.843+0000 7f9c65ffb640 1 -- 172.21.15.164:0/424575642 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c5c040070 con 0x7f9c681081c0 2024-09-19T08:37:04.665 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.843+0000 7f9c65ffb640 1 -- 172.21.15.164:0/424575642 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c5c03a6e0 con 0x7f9c681081c0 2024-09-19T08:37:04.665 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.843+0000 7f9c6ecca640 1 -- 172.21.15.164:0/424575642 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c6819bed0 con 0x7f9c681081c0 2024-09-19T08:37:04.665 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.843+0000 7f9c6ecca640 1 -- 172.21.15.164:0/424575642 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c6819ebc0 con 0x7f9c681081c0 2024-09-19T08:37:04.666 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.844+0000 7f9c65ffb640 1 -- 172.21.15.164:0/424575642 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f9c5c03bb20 con 0x7f9c681081c0 2024-09-19T08:37:04.666 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.844+0000 7f9c65ffb640 1 --2- 172.21.15.164:0/424575642 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f9c58039260 0x7f9c5803b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:04.666 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.844+0000 7f9c6ecca640 1 -- 172.21.15.164:0/424575642 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c68108620 con 0x7f9c681081c0 2024-09-19T08:37:04.666 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.844+0000 7f9c65ffb640 1 -- 172.21.15.164:0/424575642 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9c5c06ca50 con 0x7f9c681081c0 2024-09-19T08:37:04.666 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.844+0000 7f9c67fff640 1 --2- 172.21.15.164:0/424575642 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f9c58039260 0x7f9c5803b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:04.666 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.845+0000 7f9c67fff640 1 --2- 172.21.15.164:0/424575642 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f9c58039260 0x7f9c5803b720 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9c540098c0 tx=0x7f9c54006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:04.666 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:03.849+0000 7f9c65ffb640 1 -- 172.21.15.164:0/424575642 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9c5c043360 con 0x7f9c681081c0 2024-09-19T08:37:04.666 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:04.026+0000 7f9c6ecca640 1 -- 172.21.15.164:0/424575642 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f9c68103060 con 0x7f9c58039260 2024-09-19T08:37:04.666 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:04.091+0000 7f9c65ffb640 1 -- 172.21.15.164:0/424575642 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f9c68103060 con 0x7f9c58039260 2024-09-19T08:37:04.666 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:04.093+0000 7f9c6ecca640 1 -- 172.21.15.164:0/424575642 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f9c58039260 msgr2=0x7f9c5803b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:04.666 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:04.093+0000 7f9c6ecca640 1 --2- 172.21.15.164:0/424575642 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f9c58039260 0x7f9c5803b720 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9c540098c0 tx=0x7f9c54006db0 comp rx=0 tx=0).stop 2024-09-19T08:37:04.666 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:04.093+0000 7f9c6ecca640 1 -- 172.21.15.164:0/424575642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c681081c0 msgr2=0x7f9c6819b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:04.667 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:04.093+0000 7f9c6ecca640 1 --2- 172.21.15.164:0/424575642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c681081c0 0x7f9c6819b700 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c030420 tx=0x7f9c5c032670 comp rx=0 tx=0).stop 2024-09-19T08:37:04.667 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:04.093+0000 7f9c6ecca640 1 -- 172.21.15.164:0/424575642 shutdown_connections 2024-09-19T08:37:04.667 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:04.093+0000 7f9c6ecca640 1 --2- 172.21.15.164:0/424575642 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f9c58039260 0x7f9c5803b720 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:04.667 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:04.093+0000 7f9c6ecca640 1 --2- 172.21.15.164:0/424575642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c681081c0 0x7f9c6819b700 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:04.667 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:04.093+0000 7f9c6ecca640 1 -- 172.21.15.164:0/424575642 >> 172.21.15.164:0/424575642 conn(0x7f9c68101e20 msgr2=0x7f9c68102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:04.667 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:04.093+0000 7f9c6ecca640 1 -- 172.21.15.164:0/424575642 shutdown_connections 2024-09-19T08:37:04.667 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:04.093+0000 7f9c6ecca640 1 -- 172.21.15.164:0/424575642 wait complete. 2024-09-19T08:37:04.667 INFO:teuthology.orchestra.run.smithi164.stdout:Creating initial admin user... 2024-09-19T08:37:04.814 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:04 smithi164 ceph-mon[30928]: [19/Sep/2024:08:37:03] ENGINE Bus STARTING 2024-09-19T08:37:04.814 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:04 smithi164 ceph-mon[30928]: [19/Sep/2024:08:37:03] ENGINE Serving on http://172.21.15.164:8765 2024-09-19T08:37:04.815 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:04 smithi164 ceph-mon[30928]: [19/Sep/2024:08:37:03] ENGINE Serving on https://172.21.15.164:7150 2024-09-19T08:37:04.815 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:04 smithi164 ceph-mon[30928]: [19/Sep/2024:08:37:03] ENGINE Bus STARTED 2024-09-19T08:37:04.815 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:04 smithi164 ceph-mon[30928]: [19/Sep/2024:08:37:03] ENGINE Error in HTTPServer.serve 2024-09-19T08:37:04.815 INFO:journalctl@ceph.mon.a.smithi164.stdout: Traceback (most recent call last): 2024-09-19T08:37:04.815 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T08:37:04.815 INFO:journalctl@ceph.mon.a.smithi164.stdout: self._connections.run(self.expiration_interval) 2024-09-19T08:37:04.815 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T08:37:04.815 INFO:journalctl@ceph.mon.a.smithi164.stdout: self._run(expiration_interval) 2024-09-19T08:37:04.815 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T08:37:04.815 INFO:journalctl@ceph.mon.a.smithi164.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T08:37:04.815 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T08:37:04.815 INFO:journalctl@ceph.mon.a.smithi164.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T08:37:04.816 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T08:37:04.816 INFO:journalctl@ceph.mon.a.smithi164.stdout: s = self.context.wrap_socket( 2024-09-19T08:37:04.816 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T08:37:04.816 INFO:journalctl@ceph.mon.a.smithi164.stdout: return self.sslsocket_class._create( 2024-09-19T08:37:04.816 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T08:37:04.816 INFO:journalctl@ceph.mon.a.smithi164.stdout: self.do_handshake() 2024-09-19T08:37:04.816 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T08:37:04.816 INFO:journalctl@ceph.mon.a.smithi164.stdout: self._sslobj.do_handshake() 2024-09-19T08:37:04.816 INFO:journalctl@ceph.mon.a.smithi164.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T08:37:04.816 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:04 smithi164 ceph-mon[30928]: mgrmap e12: x(active, since 2s) 2024-09-19T08:37:04.816 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:04 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:04.816 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:04 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:04.817 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:04 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:06.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:05 smithi164 ceph-mon[30928]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:37:06.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:05 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:06.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:05 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:06.348 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$N.6/0ksynzv4X2efKhzu5uBCE5F1agxpOH5uzikXXTZJpZ/FnasjK", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726735025, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-19T08:37:06.348 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.097+0000 7fb81d00c640 1 Processor -- start 2024-09-19T08:37:06.348 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.098+0000 7fb81d00c640 1 -- start start 2024-09-19T08:37:06.348 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.098+0000 7fb81d00c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8181081a0 0x7fb81810a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:06.348 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.098+0000 7fb81d00c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb81807be70 con 0x7fb8181081a0 2024-09-19T08:37:06.348 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.098+0000 7fb816d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8181081a0 0x7fb81810a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:06.348 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.098+0000 7fb816d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8181081a0 0x7fb81810a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40746/0 (socket says 172.21.15.164:40746) 2024-09-19T08:37:06.348 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.098+0000 7fb816d9d640 1 -- 172.21.15.164:0/3485329325 learned_addr learned my addr 172.21.15.164:0/3485329325 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:37:06.349 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.098+0000 7fb816d9d640 1 -- 172.21.15.164:0/3485329325 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb81810aad0 con 0x7fb8181081a0 2024-09-19T08:37:06.349 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.099+0000 7fb816d9d640 1 --2- 172.21.15.164:0/3485329325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8181081a0 0x7fb81810a590 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb808009820 tx=0x7fb80802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24793987937ba061 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:06.349 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.099+0000 7fb81659c640 1 -- 172.21.15.164:0/3485329325 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb808037070 con 0x7fb8181081a0 2024-09-19T08:37:06.349 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.099+0000 7fb81659c640 1 -- 172.21.15.164:0/3485329325 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb808032920 con 0x7fb8181081a0 2024-09-19T08:37:06.349 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.099+0000 7fb81659c640 1 -- 172.21.15.164:0/3485329325 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb808030810 con 0x7fb8181081a0 2024-09-19T08:37:06.349 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.100+0000 7fb81d00c640 1 -- 172.21.15.164:0/3485329325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8181081a0 msgr2=0x7fb81810a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:06.349 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.100+0000 7fb81d00c640 1 --2- 172.21.15.164:0/3485329325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8181081a0 0x7fb81810a590 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb808009820 tx=0x7fb80802a270 comp rx=0 tx=0).stop 2024-09-19T08:37:06.349 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.100+0000 7fb81d00c640 1 -- 172.21.15.164:0/3485329325 shutdown_connections 2024-09-19T08:37:06.349 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.100+0000 7fb81d00c640 1 --2- 172.21.15.164:0/3485329325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8181081a0 0x7fb81810a590 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:06.349 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.100+0000 7fb81d00c640 1 -- 172.21.15.164:0/3485329325 >> 172.21.15.164:0/3485329325 conn(0x7fb818101e20 msgr2=0x7fb818104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:06.349 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.100+0000 7fb81d00c640 1 -- 172.21.15.164:0/3485329325 shutdown_connections 2024-09-19T08:37:06.349 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.100+0000 7fb81d00c640 1 -- 172.21.15.164:0/3485329325 wait complete. 2024-09-19T08:37:06.349 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.101+0000 7fb81d00c640 1 Processor -- start 2024-09-19T08:37:06.350 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.101+0000 7fb81d00c640 1 -- start start 2024-09-19T08:37:06.350 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.101+0000 7fb81d00c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8181081a0 0x7fb818107a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:06.350 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.101+0000 7fb81d00c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb81810bb50 con 0x7fb8181081a0 2024-09-19T08:37:06.350 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.101+0000 7fb816d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8181081a0 0x7fb818107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:06.350 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.101+0000 7fb816d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8181081a0 0x7fb818107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40754/0 (socket says 172.21.15.164:40754) 2024-09-19T08:37:06.350 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.101+0000 7fb816d9d640 1 -- 172.21.15.164:0/656285417 learned_addr learned my addr 172.21.15.164:0/656285417 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:37:06.350 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.101+0000 7fb816d9d640 1 -- 172.21.15.164:0/656285417 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb818107f60 con 0x7fb8181081a0 2024-09-19T08:37:06.350 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.101+0000 7fb816d9d640 1 --2- 172.21.15.164:0/656285417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8181081a0 0x7fb818107a20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb808030420 tx=0x7fb808032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:06.350 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.101+0000 7fb814d99640 1 -- 172.21.15.164:0/656285417 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb808037040 con 0x7fb8181081a0 2024-09-19T08:37:06.350 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.101+0000 7fb814d99640 1 -- 172.21.15.164:0/656285417 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb808040070 con 0x7fb8181081a0 2024-09-19T08:37:06.350 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.101+0000 7fb814d99640 1 -- 172.21.15.164:0/656285417 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb80803a6e0 con 0x7fb8181081a0 2024-09-19T08:37:06.350 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.102+0000 7fb81d00c640 1 -- 172.21.15.164:0/656285417 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb818106150 con 0x7fb8181081a0 2024-09-19T08:37:06.350 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.102+0000 7fb81d00c640 1 -- 172.21.15.164:0/656285417 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb818106690 con 0x7fb8181081a0 2024-09-19T08:37:06.351 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.102+0000 7fb814d99640 1 -- 172.21.15.164:0/656285417 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fb80803a990 con 0x7fb8181081a0 2024-09-19T08:37:06.351 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.103+0000 7fb814d99640 1 --2- 172.21.15.164:0/656285417 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb804039260 0x7fb80403b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:06.351 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.103+0000 7fb81d00c640 1 -- 172.21.15.164:0/656285417 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb818108600 con 0x7fb8181081a0 2024-09-19T08:37:06.351 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.103+0000 7fb814d99640 1 -- 172.21.15.164:0/656285417 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb80806cad0 con 0x7fb8181081a0 2024-09-19T08:37:06.351 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.103+0000 7fb80ffff640 1 --2- 172.21.15.164:0/656285417 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb804039260 0x7fb80403b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:06.351 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.103+0000 7fb80ffff640 1 --2- 172.21.15.164:0/656285417 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb804039260 0x7fb80403b720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb8000098c0 tx=0x7fb800006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:06.351 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.107+0000 7fb814d99640 1 -- 172.21.15.164:0/656285417 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb80803ad20 con 0x7fb8181081a0 2024-09-19T08:37:06.351 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.304+0000 7fb81d00c640 1 -- 172.21.15.164:0/656285417 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fb818106f40 con 0x7fb804039260 2024-09-19T08:37:06.351 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.531+0000 7fb814d99640 1 -- 172.21.15.164:0/656285417 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fb818106f40 con 0x7fb804039260 2024-09-19T08:37:06.352 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.533+0000 7fb81d00c640 1 -- 172.21.15.164:0/656285417 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb804039260 msgr2=0x7fb80403b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:06.352 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.533+0000 7fb81d00c640 1 --2- 172.21.15.164:0/656285417 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb804039260 0x7fb80403b720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb8000098c0 tx=0x7fb800006db0 comp rx=0 tx=0).stop 2024-09-19T08:37:06.352 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.533+0000 7fb81d00c640 1 -- 172.21.15.164:0/656285417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8181081a0 msgr2=0x7fb818107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:06.352 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.533+0000 7fb81d00c640 1 --2- 172.21.15.164:0/656285417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8181081a0 0x7fb818107a20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb808030420 tx=0x7fb808032670 comp rx=0 tx=0).stop 2024-09-19T08:37:06.352 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.533+0000 7fb81d00c640 1 -- 172.21.15.164:0/656285417 shutdown_connections 2024-09-19T08:37:06.352 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.533+0000 7fb81d00c640 1 --2- 172.21.15.164:0/656285417 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb804039260 0x7fb80403b720 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:06.352 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.533+0000 7fb81d00c640 1 --2- 172.21.15.164:0/656285417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8181081a0 0x7fb818107a20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:06.352 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.533+0000 7fb81d00c640 1 -- 172.21.15.164:0/656285417 >> 172.21.15.164:0/656285417 conn(0x7fb818101e20 msgr2=0x7fb818102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:06.352 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.533+0000 7fb81d00c640 1 -- 172.21.15.164:0/656285417 shutdown_connections 2024-09-19T08:37:06.353 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:05.533+0000 7fb81d00c640 1 -- 172.21.15.164:0/656285417 wait complete. 2024-09-19T08:37:06.353 INFO:teuthology.orchestra.run.smithi164.stdout:Fetching dashboard port number... 2024-09-19T08:37:06.911 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:06 smithi164 ceph-mon[30928]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:37:07.832 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:07 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2648535123' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-19T08:37:07.839 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stdout 8443 2024-09-19T08:37:07.840 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.725+0000 7fcb1465d640 1 Processor -- start 2024-09-19T08:37:07.840 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.725+0000 7fcb1465d640 1 -- start start 2024-09-19T08:37:07.840 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.726+0000 7fcb1465d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb0c1065f0 0x7fcb0c1069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:07.840 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.726+0000 7fcb1465d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0c106fc0 con 0x7fcb0c1065f0 2024-09-19T08:37:07.840 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.726+0000 7fcb123f9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb0c1065f0 0x7fcb0c1069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:07.840 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.726+0000 7fcb123f9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb0c1065f0 0x7fcb0c1069f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40766/0 (socket says 172.21.15.164:40766) 2024-09-19T08:37:07.840 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.726+0000 7fcb123f9640 1 -- 172.21.15.164:0/1707270203 learned_addr learned my addr 172.21.15.164:0/1707270203 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:37:07.840 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.726+0000 7fcb123f9640 1 -- 172.21.15.164:0/1707270203 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb0c1077f0 con 0x7fcb0c1065f0 2024-09-19T08:37:07.840 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.727+0000 7fcb123f9640 1 --2- 172.21.15.164:0/1707270203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb0c1065f0 0x7fcb0c1069f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009820 tx=0x7fcb0802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8e07a6d8fb6a7d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:07.840 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.727+0000 7fcb113f7640 1 -- 172.21.15.164:0/1707270203 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb08037070 con 0x7fcb0c1065f0 2024-09-19T08:37:07.840 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.727+0000 7fcb113f7640 1 -- 172.21.15.164:0/1707270203 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb08032920 con 0x7fcb0c1065f0 2024-09-19T08:37:07.841 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.727+0000 7fcb113f7640 1 -- 172.21.15.164:0/1707270203 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb08030810 con 0x7fcb0c1065f0 2024-09-19T08:37:07.841 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.727+0000 7fcb1465d640 1 -- 172.21.15.164:0/1707270203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb0c1065f0 msgr2=0x7fcb0c1069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:07.841 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.727+0000 7fcb1465d640 1 --2- 172.21.15.164:0/1707270203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb0c1065f0 0x7fcb0c1069f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009820 tx=0x7fcb0802a270 comp rx=0 tx=0).stop 2024-09-19T08:37:07.841 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.727+0000 7fcb1465d640 1 -- 172.21.15.164:0/1707270203 shutdown_connections 2024-09-19T08:37:07.841 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.727+0000 7fcb1465d640 1 --2- 172.21.15.164:0/1707270203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb0c1065f0 0x7fcb0c1069f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:07.841 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.727+0000 7fcb1465d640 1 -- 172.21.15.164:0/1707270203 >> 172.21.15.164:0/1707270203 conn(0x7fcb0c101e20 msgr2=0x7fcb0c104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:07.841 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.728+0000 7fcb1465d640 1 -- 172.21.15.164:0/1707270203 shutdown_connections 2024-09-19T08:37:07.841 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.728+0000 7fcb1465d640 1 -- 172.21.15.164:0/1707270203 wait complete. 2024-09-19T08:37:07.841 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.728+0000 7fcb1465d640 1 Processor -- start 2024-09-19T08:37:07.841 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.728+0000 7fcb1465d640 1 -- start start 2024-09-19T08:37:07.841 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.728+0000 7fcb1465d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb0c1065f0 0x7fcb0c19b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:07.841 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.728+0000 7fcb1465d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0c10aba0 con 0x7fcb0c1065f0 2024-09-19T08:37:07.841 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.728+0000 7fcb123f9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb0c1065f0 0x7fcb0c19b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:07.841 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.728+0000 7fcb123f9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb0c1065f0 0x7fcb0c19b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40784/0 (socket says 172.21.15.164:40784) 2024-09-19T08:37:07.842 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.728+0000 7fcb123f9640 1 -- 172.21.15.164:0/2648535123 learned_addr learned my addr 172.21.15.164:0/2648535123 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:37:07.842 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.729+0000 7fcb123f9640 1 -- 172.21.15.164:0/2648535123 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb0c19bcc0 con 0x7fcb0c1065f0 2024-09-19T08:37:07.842 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.729+0000 7fcb123f9640 1 --2- 172.21.15.164:0/2648535123 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb0c1065f0 0x7fcb0c19b780 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcb080097f0 tx=0x7fcb080327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:07.842 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.729+0000 7fcaff7fe640 1 -- 172.21.15.164:0/2648535123 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb08037040 con 0x7fcb0c1065f0 2024-09-19T08:37:07.842 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.729+0000 7fcaff7fe640 1 -- 172.21.15.164:0/2648535123 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb08030c40 con 0x7fcb0c1065f0 2024-09-19T08:37:07.842 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.729+0000 7fcaff7fe640 1 -- 172.21.15.164:0/2648535123 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb0803a5b0 con 0x7fcb0c1065f0 2024-09-19T08:37:07.842 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.729+0000 7fcb1465d640 1 -- 172.21.15.164:0/2648535123 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb0c19bf50 con 0x7fcb0c1065f0 2024-09-19T08:37:07.842 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.729+0000 7fcb1465d640 1 -- 172.21.15.164:0/2648535123 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb0c19ec40 con 0x7fcb0c1065f0 2024-09-19T08:37:07.842 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.730+0000 7fcaff7fe640 1 -- 172.21.15.164:0/2648535123 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fcb0803a7a0 con 0x7fcb0c1065f0 2024-09-19T08:37:07.842 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.730+0000 7fcb1465d640 1 -- 172.21.15.164:0/2648535123 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb0c10ad20 con 0x7fcb0c1065f0 2024-09-19T08:37:07.842 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.730+0000 7fcaff7fe640 1 --2- 172.21.15.164:0/2648535123 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fcaf8039260 0x7fcaf803b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:07.842 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.730+0000 7fcaff7fe640 1 -- 172.21.15.164:0/2648535123 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcb0806c890 con 0x7fcb0c1065f0 2024-09-19T08:37:07.842 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.730+0000 7fcb11bf8640 1 --2- 172.21.15.164:0/2648535123 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fcaf8039260 0x7fcaf803b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:07.843 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.731+0000 7fcb11bf8640 1 --2- 172.21.15.164:0/2648535123 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fcaf8039260 0x7fcaf803b720 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcb000098c0 tx=0x7fcb00006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:07.843 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.735+0000 7fcaff7fe640 1 -- 172.21.15.164:0/2648535123 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcb080371e0 con 0x7fcb0c1065f0 2024-09-19T08:37:07.843 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.911+0000 7fcb1465d640 1 -- 172.21.15.164:0/2648535123 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fcb0c106a70 con 0x7fcb0c1065f0 2024-09-19T08:37:07.843 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.912+0000 7fcaff7fe640 1 -- 172.21.15.164:0/2648535123 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fcb0803b3f0 con 0x7fcb0c1065f0 2024-09-19T08:37:07.843 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.914+0000 7fcb1465d640 1 -- 172.21.15.164:0/2648535123 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fcaf8039260 msgr2=0x7fcaf803b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:07.843 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.914+0000 7fcb1465d640 1 --2- 172.21.15.164:0/2648535123 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fcaf8039260 0x7fcaf803b720 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcb000098c0 tx=0x7fcb00006db0 comp rx=0 tx=0).stop 2024-09-19T08:37:07.843 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.914+0000 7fcb1465d640 1 -- 172.21.15.164:0/2648535123 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb0c1065f0 msgr2=0x7fcb0c19b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:07.843 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.914+0000 7fcb1465d640 1 --2- 172.21.15.164:0/2648535123 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb0c1065f0 0x7fcb0c19b780 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcb080097f0 tx=0x7fcb080327b0 comp rx=0 tx=0).stop 2024-09-19T08:37:07.843 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.914+0000 7fcb1465d640 1 -- 172.21.15.164:0/2648535123 shutdown_connections 2024-09-19T08:37:07.843 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.914+0000 7fcb1465d640 1 --2- 172.21.15.164:0/2648535123 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fcaf8039260 0x7fcaf803b720 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:07.843 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.914+0000 7fcb1465d640 1 --2- 172.21.15.164:0/2648535123 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb0c1065f0 0x7fcb0c19b780 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:07.843 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.914+0000 7fcb1465d640 1 -- 172.21.15.164:0/2648535123 >> 172.21.15.164:0/2648535123 conn(0x7fcb0c101e20 msgr2=0x7fcb0c102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:07.843 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.914+0000 7fcb1465d640 1 -- 172.21.15.164:0/2648535123 shutdown_connections 2024-09-19T08:37:07.844 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:06.914+0000 7fcb1465d640 1 -- 172.21.15.164:0/2648535123 wait complete. 2024-09-19T08:37:07.848 INFO:teuthology.orchestra.run.smithi164.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T08:37:07.848 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stdout disabled 2024-09-19T08:37:07.856 INFO:teuthology.orchestra.run.smithi164.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T08:37:07.856 INFO:teuthology.orchestra.run.smithi164.stdout:systemctl: stdout inactive 2024-09-19T08:37:07.856 INFO:teuthology.orchestra.run.smithi164.stdout:firewalld.service is not enabled 2024-09-19T08:37:07.856 INFO:teuthology.orchestra.run.smithi164.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-19T08:37:07.858 INFO:teuthology.orchestra.run.smithi164.stdout:Ceph Dashboard is now available at: 2024-09-19T08:37:07.858 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:37:07.858 INFO:teuthology.orchestra.run.smithi164.stdout: URL: https://smithi164.front.sepia.ceph.com:8443/ 2024-09-19T08:37:07.858 INFO:teuthology.orchestra.run.smithi164.stdout: User: admin 2024-09-19T08:37:07.858 INFO:teuthology.orchestra.run.smithi164.stdout: Password: w646q3z6q0 2024-09-19T08:37:07.858 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:37:07.858 INFO:teuthology.orchestra.run.smithi164.stdout:Saving cluster configuration to /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config directory 2024-09-19T08:37:08.938 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:08 smithi164 ceph-mon[30928]: mgrmap e13: x(active, since 6s) 2024-09-19T08:37:08.938 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:08 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2369773584' entity='client.admin' 2024-09-19T08:37:09.191 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.374+0000 7feaf7185640 1 Processor -- start 2024-09-19T08:37:09.192 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.374+0000 7feaf7185640 1 -- start start 2024-09-19T08:37:09.192 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.375+0000 7feaf7185640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feaf01081e0 0x7feaf010a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:09.192 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.375+0000 7feaf7185640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feaf007be30 con 0x7feaf01081e0 2024-09-19T08:37:09.192 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.375+0000 7feaf4f21640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feaf01081e0 0x7feaf010a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:09.192 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.375+0000 7feaf4f21640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feaf01081e0 0x7feaf010a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40786/0 (socket says 172.21.15.164:40786) 2024-09-19T08:37:09.192 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.375+0000 7feaf4f21640 1 -- 172.21.15.164:0/579629100 learned_addr learned my addr 172.21.15.164:0/579629100 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:37:09.192 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.375+0000 7feaf4f21640 1 -- 172.21.15.164:0/579629100 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf010ab10 con 0x7feaf01081e0 2024-09-19T08:37:09.193 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.376+0000 7feaf4f21640 1 --2- 172.21.15.164:0/579629100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feaf01081e0 0x7feaf010a5d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7feae0009e80 tx=0x7feae002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621582e9dc3700d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:09.193 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.376+0000 7feaef7fe640 1 -- 172.21.15.164:0/579629100 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feae0036070 con 0x7feaf01081e0 2024-09-19T08:37:09.193 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.376+0000 7feaef7fe640 1 -- 172.21.15.164:0/579629100 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feae00318e0 con 0x7feaf01081e0 2024-09-19T08:37:09.193 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.376+0000 7feaef7fe640 1 -- 172.21.15.164:0/579629100 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feae0007a00 con 0x7feaf01081e0 2024-09-19T08:37:09.193 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.377+0000 7feaf7185640 1 -- 172.21.15.164:0/579629100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feaf01081e0 msgr2=0x7feaf010a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:09.193 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.377+0000 7feaf7185640 1 --2- 172.21.15.164:0/579629100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feaf01081e0 0x7feaf010a5d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7feae0009e80 tx=0x7feae002a7a0 comp rx=0 tx=0).stop 2024-09-19T08:37:09.193 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.377+0000 7feaf7185640 1 -- 172.21.15.164:0/579629100 shutdown_connections 2024-09-19T08:37:09.193 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.377+0000 7feaf7185640 1 --2- 172.21.15.164:0/579629100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feaf01081e0 0x7feaf010a5d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:09.194 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.377+0000 7feaf7185640 1 -- 172.21.15.164:0/579629100 >> 172.21.15.164:0/579629100 conn(0x7feaf0101e20 msgr2=0x7feaf0104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:09.194 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.377+0000 7feaf7185640 1 -- 172.21.15.164:0/579629100 shutdown_connections 2024-09-19T08:37:09.194 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.377+0000 7feaf7185640 1 -- 172.21.15.164:0/579629100 wait complete. 2024-09-19T08:37:09.194 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.377+0000 7feaf7185640 1 Processor -- start 2024-09-19T08:37:09.194 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.378+0000 7feaf7185640 1 -- start start 2024-09-19T08:37:09.194 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.378+0000 7feaf7185640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feaf01081e0 0x7feaf019b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:09.194 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.378+0000 7feaf7185640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feaf010bb30 con 0x7feaf01081e0 2024-09-19T08:37:09.194 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.378+0000 7feaf4f21640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feaf01081e0 0x7feaf019b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:09.194 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.378+0000 7feaf4f21640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feaf01081e0 0x7feaf019b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40802/0 (socket says 172.21.15.164:40802) 2024-09-19T08:37:09.194 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.378+0000 7feaf4f21640 1 -- 172.21.15.164:0/2369773584 learned_addr learned my addr 172.21.15.164:0/2369773584 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:37:09.195 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.378+0000 7feaf4f21640 1 -- 172.21.15.164:0/2369773584 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf019bc50 con 0x7feaf01081e0 2024-09-19T08:37:09.195 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.378+0000 7feaf4f21640 1 --2- 172.21.15.164:0/2369773584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feaf01081e0 0x7feaf019b710 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7feae0031cf0 tx=0x7feae0031d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:09.195 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.378+0000 7feaedffb640 1 -- 172.21.15.164:0/2369773584 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feae0036050 con 0x7feaf01081e0 2024-09-19T08:37:09.195 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.378+0000 7feaedffb640 1 -- 172.21.15.164:0/2369773584 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feae0042070 con 0x7feaf01081e0 2024-09-19T08:37:09.195 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.378+0000 7feaedffb640 1 -- 172.21.15.164:0/2369773584 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feae003d9e0 con 0x7feaf01081e0 2024-09-19T08:37:09.195 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.378+0000 7feaf7185640 1 -- 172.21.15.164:0/2369773584 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf019bee0 con 0x7feaf01081e0 2024-09-19T08:37:09.195 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.379+0000 7feaf7185640 1 -- 172.21.15.164:0/2369773584 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf019ebd0 con 0x7feaf01081e0 2024-09-19T08:37:09.195 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.380+0000 7feaedffb640 1 -- 172.21.15.164:0/2369773584 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7feae003bd60 con 0x7feaf01081e0 2024-09-19T08:37:09.195 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.380+0000 7feaedffb640 1 --2- 172.21.15.164:0/2369773584 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fead8039300 0x7fead803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:09.195 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.380+0000 7feaf7185640 1 -- 172.21.15.164:0/2369773584 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaf0108640 con 0x7feaf01081e0 2024-09-19T08:37:09.195 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.380+0000 7feaedffb640 1 -- 172.21.15.164:0/2369773584 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7feae006d8d0 con 0x7feaf01081e0 2024-09-19T08:37:09.195 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.380+0000 7feaeffff640 1 --2- 172.21.15.164:0/2369773584 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fead8039300 0x7fead803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:09.195 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.380+0000 7feaeffff640 1 --2- 172.21.15.164:0/2369773584 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fead8039300 0x7fead803b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7feae40098c0 tx=0x7feae4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:09.196 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.385+0000 7feaedffb640 1 -- 172.21.15.164:0/2369773584 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7feae004c200 con 0x7feaf01081e0 2024-09-19T08:37:09.196 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.637+0000 7feaf7185640 1 -- 172.21.15.164:0/2369773584 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7feaf019f190 con 0x7feaf01081e0 2024-09-19T08:37:09.196 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.640+0000 7feaedffb640 1 -- 172.21.15.164:0/2369773584 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7feae0036c30 con 0x7feaf01081e0 2024-09-19T08:37:09.196 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-19T08:37:09.196 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.642+0000 7feaf7185640 1 -- 172.21.15.164:0/2369773584 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fead8039300 msgr2=0x7fead803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:09.196 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.643+0000 7feaf7185640 1 --2- 172.21.15.164:0/2369773584 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fead8039300 0x7fead803b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7feae40098c0 tx=0x7feae4006db0 comp rx=0 tx=0).stop 2024-09-19T08:37:09.196 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.643+0000 7feaf7185640 1 -- 172.21.15.164:0/2369773584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feaf01081e0 msgr2=0x7feaf019b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:09.196 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.643+0000 7feaf7185640 1 --2- 172.21.15.164:0/2369773584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feaf01081e0 0x7feaf019b710 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7feae0031cf0 tx=0x7feae0031d20 comp rx=0 tx=0).stop 2024-09-19T08:37:09.196 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.643+0000 7feaf7185640 1 -- 172.21.15.164:0/2369773584 shutdown_connections 2024-09-19T08:37:09.196 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.643+0000 7feaf7185640 1 --2- 172.21.15.164:0/2369773584 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fead8039300 0x7fead803b7c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:09.196 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.643+0000 7feaf7185640 1 --2- 172.21.15.164:0/2369773584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feaf01081e0 0x7feaf019b710 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:09.196 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.643+0000 7feaf7185640 1 -- 172.21.15.164:0/2369773584 >> 172.21.15.164:0/2369773584 conn(0x7feaf0101e20 msgr2=0x7feaf0102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:09.197 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.643+0000 7feaf7185640 1 -- 172.21.15.164:0/2369773584 shutdown_connections 2024-09-19T08:37:09.197 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/bin/ceph: stderr 2024-09-19T08:37:08.643+0000 7feaf7185640 1 -- 172.21.15.164:0/2369773584 wait complete. 2024-09-19T08:37:09.197 INFO:teuthology.orchestra.run.smithi164.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-19T08:37:09.197 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:37:09.197 INFO:teuthology.orchestra.run.smithi164.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-19T08:37:09.197 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:37:09.197 INFO:teuthology.orchestra.run.smithi164.stdout:Or, if you are only running a single cluster on this host: 2024-09-19T08:37:09.197 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:37:09.197 INFO:teuthology.orchestra.run.smithi164.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-19T08:37:09.197 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:37:09.197 INFO:teuthology.orchestra.run.smithi164.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-19T08:37:09.197 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:37:09.198 INFO:teuthology.orchestra.run.smithi164.stdout: ceph telemetry on 2024-09-19T08:37:09.198 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:37:09.198 INFO:teuthology.orchestra.run.smithi164.stdout:For more information see: 2024-09-19T08:37:09.198 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:37:09.198 INFO:teuthology.orchestra.run.smithi164.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-19T08:37:09.198 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:37:09.198 INFO:teuthology.orchestra.run.smithi164.stdout:Bootstrap complete. 2024-09-19T08:37:09.226 INFO:tasks.cephadm:Fetching config... 2024-09-19T08:37:09.227 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-19T08:37:09.227 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-19T08:37:09.243 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-19T08:37:09.243 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-19T08:37:09.243 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-19T08:37:09.301 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-19T08:37:09.301 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-19T08:37:09.302 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-19T08:37:09.367 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-19T08:37:09.367 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-19T08:37:09.367 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-19T08:37:09.425 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-19T08:37:09.425 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwRZTs4bFdIT7Ia5SP6k7MgjHFWAtGy4C3fBsqsbIeiH6bUO1UHUIQlbvqN0KE02PHSDv6Tj74pBEqTKB4yyK2Hn4HYy4oE1ju4bMqrf+OOrkg2yR1Sf6vlxOHEILE7s5mi1S/WfJshFRD3JdG6HfYWkvF1Fe2umpF4ESUmnc//8dNT6CK/tA55XHShkxuB9hoFtq9hxeBpujxRRoQNXrQTzewYtVdVIGlgEjnMKM7iOsy4MS/LxKo1vxVxUqxcpfBWiyyXTfsNKvK4al8rLK5If5yvUhlkJb82HEGC7FZpnYL+s9oikobpRDWe+4VPfD1wqQm8I+dYDJZ1jhZG2j6vWkxQDmOD1bOz/PHfd2lAHfTH3rkjje7ftPkw8yXnk78oQ4cqT/4xxml8TYcH6Hez8PlCcKm/5xSx5JUcd6i1QBJhgwZJEVh5ftQpasGz1gAg2ghKd2N0Wnw+sIYXx+yqqxw2QFJO2lw6VpdTn0LcIxo5f58em+0e3RhR0ndZLM= ceph-1e4b0d74-7662-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T08:37:09.526 INFO:teuthology.orchestra.run.smithi164.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwRZTs4bFdIT7Ia5SP6k7MgjHFWAtGy4C3fBsqsbIeiH6bUO1UHUIQlbvqN0KE02PHSDv6Tj74pBEqTKB4yyK2Hn4HYy4oE1ju4bMqrf+OOrkg2yR1Sf6vlxOHEILE7s5mi1S/WfJshFRD3JdG6HfYWkvF1Fe2umpF4ESUmnc//8dNT6CK/tA55XHShkxuB9hoFtq9hxeBpujxRRoQNXrQTzewYtVdVIGlgEjnMKM7iOsy4MS/LxKo1vxVxUqxcpfBWiyyXTfsNKvK4al8rLK5If5yvUhlkJb82HEGC7FZpnYL+s9oikobpRDWe+4VPfD1wqQm8I+dYDJZ1jhZG2j6vWkxQDmOD1bOz/PHfd2lAHfTH3rkjje7ftPkw8yXnk78oQ4cqT/4xxml8TYcH6Hez8PlCcKm/5xSx5JUcd6i1QBJhgwZJEVh5ftQpasGz1gAg2ghKd2N0Wnw+sIYXx+yqqxw2QFJO2lw6VpdTn0LcIxo5f58em+0e3RhR0ndZLM= ceph-1e4b0d74-7662-11ef-bceb-c7b262605968 2024-09-19T08:37:09.537 DEBUG:teuthology.orchestra.run.smithi170:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwRZTs4bFdIT7Ia5SP6k7MgjHFWAtGy4C3fBsqsbIeiH6bUO1UHUIQlbvqN0KE02PHSDv6Tj74pBEqTKB4yyK2Hn4HYy4oE1ju4bMqrf+OOrkg2yR1Sf6vlxOHEILE7s5mi1S/WfJshFRD3JdG6HfYWkvF1Fe2umpF4ESUmnc//8dNT6CK/tA55XHShkxuB9hoFtq9hxeBpujxRRoQNXrQTzewYtVdVIGlgEjnMKM7iOsy4MS/LxKo1vxVxUqxcpfBWiyyXTfsNKvK4al8rLK5If5yvUhlkJb82HEGC7FZpnYL+s9oikobpRDWe+4VPfD1wqQm8I+dYDJZ1jhZG2j6vWkxQDmOD1bOz/PHfd2lAHfTH3rkjje7ftPkw8yXnk78oQ4cqT/4xxml8TYcH6Hez8PlCcKm/5xSx5JUcd6i1QBJhgwZJEVh5ftQpasGz1gAg2ghKd2N0Wnw+sIYXx+yqqxw2QFJO2lw6VpdTn0LcIxo5f58em+0e3RhR0ndZLM= ceph-1e4b0d74-7662-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T08:37:09.578 INFO:teuthology.orchestra.run.smithi170.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwRZTs4bFdIT7Ia5SP6k7MgjHFWAtGy4C3fBsqsbIeiH6bUO1UHUIQlbvqN0KE02PHSDv6Tj74pBEqTKB4yyK2Hn4HYy4oE1ju4bMqrf+OOrkg2yR1Sf6vlxOHEILE7s5mi1S/WfJshFRD3JdG6HfYWkvF1Fe2umpF4ESUmnc//8dNT6CK/tA55XHShkxuB9hoFtq9hxeBpujxRRoQNXrQTzewYtVdVIGlgEjnMKM7iOsy4MS/LxKo1vxVxUqxcpfBWiyyXTfsNKvK4al8rLK5If5yvUhlkJb82HEGC7FZpnYL+s9oikobpRDWe+4VPfD1wqQm8I+dYDJZ1jhZG2j6vWkxQDmOD1bOz/PHfd2lAHfTH3rkjje7ftPkw8yXnk78oQ4cqT/4xxml8TYcH6Hez8PlCcKm/5xSx5JUcd6i1QBJhgwZJEVh5ftQpasGz1gAg2ghKd2N0Wnw+sIYXx+yqqxw2QFJO2lw6VpdTn0LcIxo5f58em+0e3RhR0ndZLM= ceph-1e4b0d74-7662-11ef-bceb-c7b262605968 2024-09-19T08:37:09.591 DEBUG:teuthology.orchestra.run.smithi173:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwRZTs4bFdIT7Ia5SP6k7MgjHFWAtGy4C3fBsqsbIeiH6bUO1UHUIQlbvqN0KE02PHSDv6Tj74pBEqTKB4yyK2Hn4HYy4oE1ju4bMqrf+OOrkg2yR1Sf6vlxOHEILE7s5mi1S/WfJshFRD3JdG6HfYWkvF1Fe2umpF4ESUmnc//8dNT6CK/tA55XHShkxuB9hoFtq9hxeBpujxRRoQNXrQTzewYtVdVIGlgEjnMKM7iOsy4MS/LxKo1vxVxUqxcpfBWiyyXTfsNKvK4al8rLK5If5yvUhlkJb82HEGC7FZpnYL+s9oikobpRDWe+4VPfD1wqQm8I+dYDJZ1jhZG2j6vWkxQDmOD1bOz/PHfd2lAHfTH3rkjje7ftPkw8yXnk78oQ4cqT/4xxml8TYcH6Hez8PlCcKm/5xSx5JUcd6i1QBJhgwZJEVh5ftQpasGz1gAg2ghKd2N0Wnw+sIYXx+yqqxw2QFJO2lw6VpdTn0LcIxo5f58em+0e3RhR0ndZLM= ceph-1e4b0d74-7662-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T08:37:09.633 INFO:teuthology.orchestra.run.smithi173.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwRZTs4bFdIT7Ia5SP6k7MgjHFWAtGy4C3fBsqsbIeiH6bUO1UHUIQlbvqN0KE02PHSDv6Tj74pBEqTKB4yyK2Hn4HYy4oE1ju4bMqrf+OOrkg2yR1Sf6vlxOHEILE7s5mi1S/WfJshFRD3JdG6HfYWkvF1Fe2umpF4ESUmnc//8dNT6CK/tA55XHShkxuB9hoFtq9hxeBpujxRRoQNXrQTzewYtVdVIGlgEjnMKM7iOsy4MS/LxKo1vxVxUqxcpfBWiyyXTfsNKvK4al8rLK5If5yvUhlkJb82HEGC7FZpnYL+s9oikobpRDWe+4VPfD1wqQm8I+dYDJZ1jhZG2j6vWkxQDmOD1bOz/PHfd2lAHfTH3rkjje7ftPkw8yXnk78oQ4cqT/4xxml8TYcH6Hez8PlCcKm/5xSx5JUcd6i1QBJhgwZJEVh5ftQpasGz1gAg2ghKd2N0Wnw+sIYXx+yqqxw2QFJO2lw6VpdTn0LcIxo5f58em+0e3RhR0ndZLM= ceph-1e4b0d74-7662-11ef-bceb-c7b262605968 2024-09-19T08:37:09.645 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-19T08:37:09.895 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:37:10.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.389+0000 7f650a791640 1 -- 172.21.15.164:0/3319002781 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6504100520 msgr2=0x7f6504100920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:10.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.389+0000 7f650a791640 1 --2- 172.21.15.164:0/3319002781 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6504100520 0x7f6504100920 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f64f4009880 tx=0x7f64f402a5d0 comp rx=0 tx=0).stop 2024-09-19T08:37:10.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.389+0000 7f650a791640 1 -- 172.21.15.164:0/3319002781 shutdown_connections 2024-09-19T08:37:10.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.389+0000 7f650a791640 1 --2- 172.21.15.164:0/3319002781 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6504100520 0x7f6504100920 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:10.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.389+0000 7f650a791640 1 -- 172.21.15.164:0/3319002781 >> 172.21.15.164:0/3319002781 conn(0x7f65040fbcb0 msgr2=0x7f65040fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:10.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.389+0000 7f650a791640 1 -- 172.21.15.164:0/3319002781 shutdown_connections 2024-09-19T08:37:10.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.389+0000 7f650a791640 1 -- 172.21.15.164:0/3319002781 wait complete. 2024-09-19T08:37:10.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.390+0000 7f650a791640 1 Processor -- start 2024-09-19T08:37:10.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.390+0000 7f650a791640 1 -- start start 2024-09-19T08:37:10.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.390+0000 7f650a791640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6504100520 0x7f65040fdb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:10.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.390+0000 7f650a791640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6504106630 con 0x7f6504100520 2024-09-19T08:37:10.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.390+0000 7f6503fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6504100520 0x7f65040fdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:10.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.390+0000 7f6503fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6504100520 0x7f65040fdb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40828/0 (socket says 172.21.15.164:40828) 2024-09-19T08:37:10.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.390+0000 7f6503fff640 1 -- 172.21.15.164:0/2941271938 learned_addr learned my addr 172.21.15.164:0/2941271938 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:37:10.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.390+0000 7f6503fff640 1 -- 172.21.15.164:0/2941271938 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65040fe0b0 con 0x7f6504100520 2024-09-19T08:37:10.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.391+0000 7f6503fff640 1 --2- 172.21.15.164:0/2941271938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6504100520 0x7f65040fdb70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f64f4004f60 tx=0x7f64f40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:10.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.391+0000 7f65017fa640 1 -- 172.21.15.164:0/2941271938 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64f403f070 con 0x7f6504100520 2024-09-19T08:37:10.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.391+0000 7f65017fa640 1 -- 172.21.15.164:0/2941271938 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64f4004d40 con 0x7f6504100520 2024-09-19T08:37:10.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.391+0000 7f650a791640 1 -- 172.21.15.164:0/2941271938 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65040fc2a0 con 0x7f6504100520 2024-09-19T08:37:10.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.391+0000 7f65017fa640 1 -- 172.21.15.164:0/2941271938 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64f4035030 con 0x7f6504100520 2024-09-19T08:37:10.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.391+0000 7f650a791640 1 -- 172.21.15.164:0/2941271938 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65040fc7e0 con 0x7f6504100520 2024-09-19T08:37:10.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.392+0000 7f65017fa640 1 -- 172.21.15.164:0/2941271938 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f64f4042070 con 0x7f6504100520 2024-09-19T08:37:10.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.392+0000 7f65017fa640 1 --2- 172.21.15.164:0/2941271938 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f64f00392a0 0x7f64f003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:10.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.392+0000 7f65017fa640 1 -- 172.21.15.164:0/2941271938 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f64f406d020 con 0x7f6504100520 2024-09-19T08:37:10.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.392+0000 7f650a791640 1 -- 172.21.15.164:0/2941271938 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6504106db0 con 0x7f6504100520 2024-09-19T08:37:10.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.392+0000 7f65037fe640 1 --2- 172.21.15.164:0/2941271938 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f64f00392a0 0x7f64f003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:10.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.392+0000 7f65037fe640 1 --2- 172.21.15.164:0/2941271938 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f64f00392a0 0x7f64f003b760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f64ec0098c0 tx=0x7f64ec006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:10.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.397+0000 7f65017fa640 1 -- 172.21.15.164:0/2941271938 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f64f4039350 con 0x7f6504100520 2024-09-19T08:37:10.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.566+0000 7f650a791640 1 -- 172.21.15.164:0/2941271938 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f6504190210 con 0x7f6504100520 2024-09-19T08:37:10.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.568+0000 7f65017fa640 1 -- 172.21.15.164:0/2941271938 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f64f402f300 con 0x7f6504100520 2024-09-19T08:37:10.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.571+0000 7f650a791640 1 -- 172.21.15.164:0/2941271938 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f64f00392a0 msgr2=0x7f64f003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:10.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.571+0000 7f650a791640 1 --2- 172.21.15.164:0/2941271938 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f64f00392a0 0x7f64f003b760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f64ec0098c0 tx=0x7f64ec006db0 comp rx=0 tx=0).stop 2024-09-19T08:37:10.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.571+0000 7f650a791640 1 -- 172.21.15.164:0/2941271938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6504100520 msgr2=0x7f65040fdb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:10.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.571+0000 7f650a791640 1 --2- 172.21.15.164:0/2941271938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6504100520 0x7f65040fdb70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f64f4004f60 tx=0x7f64f40048f0 comp rx=0 tx=0).stop 2024-09-19T08:37:10.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.571+0000 7f650a791640 1 -- 172.21.15.164:0/2941271938 shutdown_connections 2024-09-19T08:37:10.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.571+0000 7f650a791640 1 --2- 172.21.15.164:0/2941271938 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f64f00392a0 0x7f64f003b760 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:10.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.571+0000 7f650a791640 1 --2- 172.21.15.164:0/2941271938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6504100520 0x7f65040fdb70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:10.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.571+0000 7f650a791640 1 -- 172.21.15.164:0/2941271938 >> 172.21.15.164:0/2941271938 conn(0x7f65040fbcb0 msgr2=0x7f650406a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:10.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.571+0000 7f650a791640 1 -- 172.21.15.164:0/2941271938 shutdown_connections 2024-09-19T08:37:10.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:10.571+0000 7f650a791640 1 -- 172.21.15.164:0/2941271938 wait complete. 2024-09-19T08:37:11.856 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:11 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2941271938' entity='client.admin' 2024-09-19T08:37:12.029 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-19T08:37:12.030 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-19T08:37:12.712 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:37:13.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.149+0000 7f01a2046640 1 -- 172.21.15.164:0/901328215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f019c100530 msgr2=0x7f019c100930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:13.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.149+0000 7f01a2046640 1 --2- 172.21.15.164:0/901328215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f019c100530 0x7f019c100930 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f018c009c20 tx=0x7f018c02aa90 comp rx=0 tx=0).stop 2024-09-19T08:37:13.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.149+0000 7f01a2046640 1 -- 172.21.15.164:0/901328215 shutdown_connections 2024-09-19T08:37:13.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.149+0000 7f01a2046640 1 --2- 172.21.15.164:0/901328215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f019c100530 0x7f019c100930 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:13.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.149+0000 7f01a2046640 1 -- 172.21.15.164:0/901328215 >> 172.21.15.164:0/901328215 conn(0x7f019c0fbca0 msgr2=0x7f019c0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:13.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.149+0000 7f01a2046640 1 -- 172.21.15.164:0/901328215 shutdown_connections 2024-09-19T08:37:13.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.149+0000 7f01a2046640 1 -- 172.21.15.164:0/901328215 wait complete. 2024-09-19T08:37:13.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.150+0000 7f01a2046640 1 Processor -- start 2024-09-19T08:37:13.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.150+0000 7f01a2046640 1 -- start start 2024-09-19T08:37:13.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.150+0000 7f01a2046640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f019c100530 0x7f019c0fdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:13.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.150+0000 7f01a2046640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f019c106640 con 0x7f019c100530 2024-09-19T08:37:13.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.150+0000 7f019b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f019c100530 0x7f019c0fdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:13.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.150+0000 7f019b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f019c100530 0x7f019c0fdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40844/0 (socket says 172.21.15.164:40844) 2024-09-19T08:37:13.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.150+0000 7f019b7fe640 1 -- 172.21.15.164:0/19997208 learned_addr learned my addr 172.21.15.164:0/19997208 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:37:13.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.150+0000 7f019b7fe640 1 -- 172.21.15.164:0/19997208 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f019c0fe0c0 con 0x7f019c100530 2024-09-19T08:37:13.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.151+0000 7f019b7fe640 1 --2- 172.21.15.164:0/19997208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f019c100530 0x7f019c0fdb80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f018c005cc0 tx=0x7f018c02fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:13.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.151+0000 7f0198ff9640 1 -- 172.21.15.164:0/19997208 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f018c047070 con 0x7f019c100530 2024-09-19T08:37:13.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.151+0000 7f01a2046640 1 -- 172.21.15.164:0/19997208 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f019c0fc2b0 con 0x7f019c100530 2024-09-19T08:37:13.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.151+0000 7f0198ff9640 1 -- 172.21.15.164:0/19997208 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f018c003e50 con 0x7f019c100530 2024-09-19T08:37:13.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.151+0000 7f0198ff9640 1 -- 172.21.15.164:0/19997208 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f018c033050 con 0x7f019c100530 2024-09-19T08:37:13.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.151+0000 7f01a2046640 1 -- 172.21.15.164:0/19997208 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f019c0fc7f0 con 0x7f019c100530 2024-09-19T08:37:13.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.152+0000 7f0198ff9640 1 -- 172.21.15.164:0/19997208 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f018c04a070 con 0x7f019c100530 2024-09-19T08:37:13.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.152+0000 7f0198ff9640 1 --2- 172.21.15.164:0/19997208 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f01840392a0 0x7f018403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:13.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.152+0000 7f01a2046640 1 -- 172.21.15.164:0/19997208 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f019c106dc0 con 0x7f019c100530 2024-09-19T08:37:13.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.152+0000 7f0198ff9640 1 -- 172.21.15.164:0/19997208 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f018c03e030 con 0x7f019c100530 2024-09-19T08:37:13.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.152+0000 7f019affd640 1 --2- 172.21.15.164:0/19997208 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f01840392a0 0x7f018403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:13.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.152+0000 7f019affd640 1 --2- 172.21.15.164:0/19997208 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f01840392a0 0x7f018403b760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f01900098c0 tx=0x7f0190006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:13.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.157+0000 7f0198ff9640 1 -- 172.21.15.164:0/19997208 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f018c0373b0 con 0x7f019c100530 2024-09-19T08:37:13.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.326+0000 7f01a2046640 1 -- 172.21.15.164:0/19997208 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f019c1903a0 con 0x7f01840392a0 2024-09-19T08:37:13.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.329+0000 7f0198ff9640 1 -- 172.21.15.164:0/19997208 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f019c1903a0 con 0x7f01840392a0 2024-09-19T08:37:13.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.332+0000 7f01a2046640 1 -- 172.21.15.164:0/19997208 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f01840392a0 msgr2=0x7f018403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:13.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.332+0000 7f01a2046640 1 --2- 172.21.15.164:0/19997208 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f01840392a0 0x7f018403b760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f01900098c0 tx=0x7f0190006db0 comp rx=0 tx=0).stop 2024-09-19T08:37:13.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.332+0000 7f01a2046640 1 -- 172.21.15.164:0/19997208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f019c100530 msgr2=0x7f019c0fdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:13.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.332+0000 7f01a2046640 1 --2- 172.21.15.164:0/19997208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f019c100530 0x7f019c0fdb80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f018c005cc0 tx=0x7f018c02fc10 comp rx=0 tx=0).stop 2024-09-19T08:37:13.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.332+0000 7f01a2046640 1 -- 172.21.15.164:0/19997208 shutdown_connections 2024-09-19T08:37:13.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.332+0000 7f01a2046640 1 --2- 172.21.15.164:0/19997208 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f01840392a0 0x7f018403b760 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:13.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.332+0000 7f01a2046640 1 --2- 172.21.15.164:0/19997208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f019c100530 0x7f019c0fdb80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:13.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.332+0000 7f01a2046640 1 -- 172.21.15.164:0/19997208 >> 172.21.15.164:0/19997208 conn(0x7f019c0fbca0 msgr2=0x7f019c069ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:13.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.332+0000 7f01a2046640 1 -- 172.21.15.164:0/19997208 shutdown_connections 2024-09-19T08:37:13.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:13.332+0000 7f01a2046640 1 -- 172.21.15.164:0/19997208 wait complete. 2024-09-19T08:37:14.027 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:13 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:14.028 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:13 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:14.028 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:13 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi164", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:37:14.028 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:13 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:14.028 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:13 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:37:14.028 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:13 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:14.028 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:13 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:14.028 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:13 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:37:14.028 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:13 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:14.028 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:13 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:37:14.058 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi170 2024-09-19T08:37:14.058 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-19T08:37:14.058 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/etc/ceph/ceph.conf 2024-09-19T08:37:14.078 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-19T08:37:14.078 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:37:14.136 INFO:tasks.cephadm:Adding host smithi170 to orchestrator... 2024-09-19T08:37:14.136 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph orch host add smithi170 2024-09-19T08:37:14.382 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:37:14.781 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:14 smithi164 ceph-mon[30928]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:37:14.781 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:14 smithi164 ceph-mon[30928]: Updating smithi164:/etc/ceph/ceph.conf 2024-09-19T08:37:14.781 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:14 smithi164 ceph-mon[30928]: Updating smithi164:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:37:14.781 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:14 smithi164 ceph-mon[30928]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:37:14.781 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:14 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:14.781 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:14 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:14.781 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:14 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:14.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:14.786+0000 7f2fd0ff1640 1 -- 172.21.15.164:0/3824144654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fcc107690 msgr2=0x7f2fcc107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:14.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:14.786+0000 7f2fd0ff1640 1 --2- 172.21.15.164:0/3824144654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fcc107690 0x7f2fcc107a70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc009c20 tx=0x7f2fbc02aa90 comp rx=0 tx=0).stop 2024-09-19T08:37:14.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:14.787+0000 7f2fd0ff1640 1 -- 172.21.15.164:0/3824144654 shutdown_connections 2024-09-19T08:37:14.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:14.787+0000 7f2fd0ff1640 1 --2- 172.21.15.164:0/3824144654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fcc107690 0x7f2fcc107a70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:14.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:14.787+0000 7f2fd0ff1640 1 -- 172.21.15.164:0/3824144654 >> 172.21.15.164:0/3824144654 conn(0x7f2fcc076250 msgr2=0x7f2fcc076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:14.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:14.787+0000 7f2fd0ff1640 1 -- 172.21.15.164:0/3824144654 shutdown_connections 2024-09-19T08:37:14.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:14.787+0000 7f2fd0ff1640 1 -- 172.21.15.164:0/3824144654 wait complete. 2024-09-19T08:37:14.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:14.787+0000 7f2fd0ff1640 1 Processor -- start 2024-09-19T08:37:14.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:14.787+0000 7f2fd0ff1640 1 -- start start 2024-09-19T08:37:14.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:14.788+0000 7f2fd0ff1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fcc107690 0x7f2fcc100800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:14.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:14.788+0000 7f2fd0ff1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc10bce0 con 0x7f2fcc107690 2024-09-19T08:37:14.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:14.788+0000 7f2fca59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fcc107690 0x7f2fcc100800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:14.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:14.788+0000 7f2fca59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fcc107690 0x7f2fcc100800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:55542/0 (socket says 172.21.15.164:55542) 2024-09-19T08:37:14.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:14.788+0000 7f2fca59c640 1 -- 172.21.15.164:0/299075535 learned_addr learned my addr 172.21.15.164:0/299075535 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:37:14.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:14.788+0000 7f2fca59c640 1 -- 172.21.15.164:0/299075535 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fcc1026f0 con 0x7f2fcc107690 2024-09-19T08:37:14.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:14.788+0000 7f2fca59c640 1 --2- 172.21.15.164:0/299075535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fcc107690 0x7f2fcc100800 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc005e30 tx=0x7f2fbc02fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:14.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:14.788+0000 7f2fbb7fe640 1 -- 172.21.15.164:0/299075535 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fbc047070 con 0x7f2fcc107690 2024-09-19T08:37:14.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:14.788+0000 7f2fd0ff1640 1 -- 172.21.15.164:0/299075535 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fcc100d40 con 0x7f2fcc107690 2024-09-19T08:37:14.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:14.788+0000 7f2fbb7fe640 1 -- 172.21.15.164:0/299075535 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fbc003df0 con 0x7f2fcc107690 2024-09-19T08:37:14.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:14.788+0000 7f2fbb7fe640 1 -- 172.21.15.164:0/299075535 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fbc033050 con 0x7f2fcc107690 2024-09-19T08:37:14.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:14.788+0000 7f2fd0ff1640 1 -- 172.21.15.164:0/299075535 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fcc101200 con 0x7f2fcc107690 2024-09-19T08:37:14.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:14.789+0000 7f2fbb7fe640 1 -- 172.21.15.164:0/299075535 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f2fbc04a070 con 0x7f2fcc107690 2024-09-19T08:37:14.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:14.789+0000 7f2fbb7fe640 1 --2- 172.21.15.164:0/299075535 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f2fb40392a0 0x7f2fb403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:14.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:14.789+0000 7f2fbb7fe640 1 -- 172.21.15.164:0/299075535 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2fbc03e030 con 0x7f2fcc107690 2024-09-19T08:37:14.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:14.790+0000 7f2fc9d9b640 1 --2- 172.21.15.164:0/299075535 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f2fb40392a0 0x7f2fb403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:14.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:14.790+0000 7f2fd0ff1640 1 -- 172.21.15.164:0/299075535 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fcc102d30 con 0x7f2fcc107690 2024-09-19T08:37:14.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:14.790+0000 7f2fc9d9b640 1 --2- 172.21.15.164:0/299075535 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f2fb40392a0 0x7f2fb403b760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2fc00098c0 tx=0x7f2fc0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:14.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:14.795+0000 7f2fbb7fe640 1 -- 172.21.15.164:0/299075535 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2fbc051080 con 0x7f2fcc107690 2024-09-19T08:37:14.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:14.963+0000 7f2fd0ff1640 1 -- 172.21.15.164:0/299075535 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi170", "target": ["mon-mgr", ""]}) -- 0x7f2fcc101da0 con 0x7f2fb40392a0 2024-09-19T08:37:16.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:15 smithi164 ceph-mon[30928]: Updating smithi164:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T08:37:17.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:16 smithi164 ceph-mon[30928]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi170", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:37:18.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:17 smithi164 ceph-mon[30928]: Deploying cephadm binary to smithi170 2024-09-19T08:37:18.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:18.083+0000 7f2fbb7fe640 1 -- 172.21.15.164:0/299075535 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f2fcc101da0 con 0x7f2fb40392a0 2024-09-19T08:37:18.082 INFO:teuthology.orchestra.run.smithi164.stdout:Added host 'smithi170' with addr '172.21.15.170' 2024-09-19T08:37:18.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:18.086+0000 7f2fd0ff1640 1 -- 172.21.15.164:0/299075535 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f2fb40392a0 msgr2=0x7f2fb403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:18.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:18.086+0000 7f2fd0ff1640 1 --2- 172.21.15.164:0/299075535 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f2fb40392a0 0x7f2fb403b760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2fc00098c0 tx=0x7f2fc0006db0 comp rx=0 tx=0).stop 2024-09-19T08:37:18.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:18.086+0000 7f2fd0ff1640 1 -- 172.21.15.164:0/299075535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fcc107690 msgr2=0x7f2fcc100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:18.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:18.086+0000 7f2fd0ff1640 1 --2- 172.21.15.164:0/299075535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fcc107690 0x7f2fcc100800 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc005e30 tx=0x7f2fbc02fc10 comp rx=0 tx=0).stop 2024-09-19T08:37:18.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:18.086+0000 7f2fd0ff1640 1 -- 172.21.15.164:0/299075535 shutdown_connections 2024-09-19T08:37:18.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:18.086+0000 7f2fd0ff1640 1 --2- 172.21.15.164:0/299075535 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f2fb40392a0 0x7f2fb403b760 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:18.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:18.086+0000 7f2fd0ff1640 1 --2- 172.21.15.164:0/299075535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fcc107690 0x7f2fcc100800 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:18.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:18.086+0000 7f2fd0ff1640 1 -- 172.21.15.164:0/299075535 >> 172.21.15.164:0/299075535 conn(0x7f2fcc076250 msgr2=0x7f2fcc0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:18.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:18.086+0000 7f2fd0ff1640 1 -- 172.21.15.164:0/299075535 shutdown_connections 2024-09-19T08:37:18.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:18.086+0000 7f2fd0ff1640 1 -- 172.21.15.164:0/299075535 wait complete. 2024-09-19T08:37:18.671 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T08:37:18.910 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:37:19.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:19 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:19.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:19 smithi164 ceph-mon[30928]: Added host smithi170 2024-09-19T08:37:19.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:19 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:37:19.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:19 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:19.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:19 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:19.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.385+0000 7f9dbaf7c640 1 -- 172.21.15.164:0/427824752 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9db4106590 msgr2=0x7f9db4106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:19.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.385+0000 7f9dbaf7c640 1 --2- 172.21.15.164:0/427824752 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9db4106590 0x7f9db4106970 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009880 tx=0x7f9db002a5d0 comp rx=0 tx=0).stop 2024-09-19T08:37:19.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.385+0000 7f9dbaf7c640 1 -- 172.21.15.164:0/427824752 shutdown_connections 2024-09-19T08:37:19.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.385+0000 7f9dbaf7c640 1 --2- 172.21.15.164:0/427824752 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9db4106590 0x7f9db4106970 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:19.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.385+0000 7f9dbaf7c640 1 -- 172.21.15.164:0/427824752 >> 172.21.15.164:0/427824752 conn(0x7f9db40fc1a0 msgr2=0x7f9db40fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:19.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.385+0000 7f9dbaf7c640 1 -- 172.21.15.164:0/427824752 shutdown_connections 2024-09-19T08:37:19.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.385+0000 7f9dbaf7c640 1 -- 172.21.15.164:0/427824752 wait complete. 2024-09-19T08:37:19.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.386+0000 7f9dbaf7c640 1 Processor -- start 2024-09-19T08:37:19.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.386+0000 7f9dbaf7c640 1 -- start start 2024-09-19T08:37:19.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.386+0000 7f9dbaf7c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9db4106590 0x7f9db4197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:19.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.386+0000 7f9dbaf7c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9db410abc0 con 0x7f9db4106590 2024-09-19T08:37:19.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.386+0000 7f9db8d18640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9db4106590 0x7f9db4197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:19.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.386+0000 7f9db8d18640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9db4106590 0x7f9db4197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:55562/0 (socket says 172.21.15.164:55562) 2024-09-19T08:37:19.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.386+0000 7f9db8d18640 1 -- 172.21.15.164:0/2815682260 learned_addr learned my addr 172.21.15.164:0/2815682260 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:37:19.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.387+0000 7f9db8d18640 1 -- 172.21.15.164:0/2815682260 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db41984a0 con 0x7f9db4106590 2024-09-19T08:37:19.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.387+0000 7f9db8d18640 1 --2- 172.21.15.164:0/2815682260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9db4106590 0x7f9db4197f60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009230 tx=0x7f9db0003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:19.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.387+0000 7f9da9ffb640 1 -- 172.21.15.164:0/2815682260 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9db003f070 con 0x7f9db4106590 2024-09-19T08:37:19.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.387+0000 7f9da9ffb640 1 -- 172.21.15.164:0/2815682260 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9db003bc80 con 0x7f9db4106590 2024-09-19T08:37:19.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.387+0000 7f9dbaf7c640 1 -- 172.21.15.164:0/2815682260 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9db4198730 con 0x7f9db4106590 2024-09-19T08:37:19.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.387+0000 7f9da9ffb640 1 -- 172.21.15.164:0/2815682260 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9db0035030 con 0x7f9db4106590 2024-09-19T08:37:19.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.387+0000 7f9dbaf7c640 1 -- 172.21.15.164:0/2815682260 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9db4198b90 con 0x7f9db4106590 2024-09-19T08:37:19.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.388+0000 7f9da9ffb640 1 -- 172.21.15.164:0/2815682260 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f9db0042070 con 0x7f9db4106590 2024-09-19T08:37:19.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.388+0000 7f9da9ffb640 1 --2- 172.21.15.164:0/2815682260 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f9d9c0392a0 0x7f9d9c03b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:19.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.388+0000 7f9da9ffb640 1 -- 172.21.15.164:0/2815682260 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9db006cfa0 con 0x7f9db4106590 2024-09-19T08:37:19.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.388+0000 7f9dbaf7c640 1 -- 172.21.15.164:0/2815682260 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9db4101c00 con 0x7f9db4106590 2024-09-19T08:37:19.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.388+0000 7f9dabfff640 1 --2- 172.21.15.164:0/2815682260 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f9d9c0392a0 0x7f9d9c03b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:19.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.389+0000 7f9dabfff640 1 --2- 172.21.15.164:0/2815682260 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f9d9c0392a0 0x7f9d9c03b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9da40098c0 tx=0x7f9da4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:19.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.393+0000 7f9da9ffb640 1 -- 172.21.15.164:0/2815682260 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9db004c350 con 0x7f9db4106590 2024-09-19T08:37:19.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.566+0000 7f9dbaf7c640 1 -- 172.21.15.164:0/2815682260 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9db41069f0 con 0x7f9d9c0392a0 2024-09-19T08:37:19.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.567+0000 7f9da9ffb640 1 -- 172.21.15.164:0/2815682260 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7f9db41069f0 con 0x7f9d9c0392a0 2024-09-19T08:37:19.566 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:37:19.567 INFO:teuthology.orchestra.run.smithi164.stdout:[{"addr": "172.21.15.164", "hostname": "smithi164", "labels": [], "status": ""}, {"addr": "172.21.15.170", "hostname": "smithi170", "labels": [], "status": ""}] 2024-09-19T08:37:19.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.569+0000 7f9dbaf7c640 1 -- 172.21.15.164:0/2815682260 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f9d9c0392a0 msgr2=0x7f9d9c03b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:19.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.569+0000 7f9dbaf7c640 1 --2- 172.21.15.164:0/2815682260 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f9d9c0392a0 0x7f9d9c03b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9da40098c0 tx=0x7f9da4006db0 comp rx=0 tx=0).stop 2024-09-19T08:37:19.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.569+0000 7f9dbaf7c640 1 -- 172.21.15.164:0/2815682260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9db4106590 msgr2=0x7f9db4197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:19.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.569+0000 7f9dbaf7c640 1 --2- 172.21.15.164:0/2815682260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9db4106590 0x7f9db4197f60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009230 tx=0x7f9db0003d40 comp rx=0 tx=0).stop 2024-09-19T08:37:19.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.569+0000 7f9dbaf7c640 1 -- 172.21.15.164:0/2815682260 shutdown_connections 2024-09-19T08:37:19.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.569+0000 7f9dbaf7c640 1 --2- 172.21.15.164:0/2815682260 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f9d9c0392a0 0x7f9d9c03b760 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:19.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.569+0000 7f9dbaf7c640 1 --2- 172.21.15.164:0/2815682260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9db4106590 0x7f9db4197f60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:19.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.569+0000 7f9dbaf7c640 1 -- 172.21.15.164:0/2815682260 >> 172.21.15.164:0/2815682260 conn(0x7f9db40fc1a0 msgr2=0x7f9db40fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:19.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.569+0000 7f9dbaf7c640 1 -- 172.21.15.164:0/2815682260 shutdown_connections 2024-09-19T08:37:19.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:19.569+0000 7f9dbaf7c640 1 -- 172.21.15.164:0/2815682260 wait complete. 2024-09-19T08:37:20.152 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi173 2024-09-19T08:37:20.152 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-19T08:37:20.152 DEBUG:teuthology.orchestra.run.smithi173:> dd of=/etc/ceph/ceph.conf 2024-09-19T08:37:20.172 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-19T08:37:20.172 DEBUG:teuthology.orchestra.run.smithi173:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:37:20.229 INFO:tasks.cephadm:Adding host smithi173 to orchestrator... 2024-09-19T08:37:20.229 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph orch host add smithi173 2024-09-19T08:37:20.469 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:37:20.675 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:20 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:20.675 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:20 smithi164 ceph-mon[30928]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:37:20.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:20.893+0000 7f3a4408b640 1 -- 172.21.15.164:0/1018242150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a3c074590 msgr2=0x7f3a3c074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:20.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:20.893+0000 7f3a4408b640 1 --2- 172.21.15.164:0/1018242150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a3c074590 0x7f3a3c074970 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3a38009880 tx=0x7f3a3802a670 comp rx=0 tx=0).stop 2024-09-19T08:37:20.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:20.893+0000 7f3a4408b640 1 -- 172.21.15.164:0/1018242150 shutdown_connections 2024-09-19T08:37:20.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:20.893+0000 7f3a4408b640 1 --2- 172.21.15.164:0/1018242150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a3c074590 0x7f3a3c074970 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:20.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:20.894+0000 7f3a4408b640 1 -- 172.21.15.164:0/1018242150 >> 172.21.15.164:0/1018242150 conn(0x7f3a3c0fd2b0 msgr2=0x7f3a3c0ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:20.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:20.894+0000 7f3a4408b640 1 -- 172.21.15.164:0/1018242150 shutdown_connections 2024-09-19T08:37:20.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:20.894+0000 7f3a4408b640 1 -- 172.21.15.164:0/1018242150 wait complete. 2024-09-19T08:37:20.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:20.894+0000 7f3a4408b640 1 Processor -- start 2024-09-19T08:37:20.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:20.894+0000 7f3a4408b640 1 -- start start 2024-09-19T08:37:20.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:20.894+0000 7f3a4408b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a3c074590 0x7f3a3c1a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:20.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:20.894+0000 7f3a4408b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3c1122f0 con 0x7f3a3c074590 2024-09-19T08:37:20.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:20.894+0000 7f3a41e27640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a3c074590 0x7f3a3c1a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:20.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:20.894+0000 7f3a41e27640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a3c074590 0x7f3a3c1a33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:55574/0 (socket says 172.21.15.164:55574) 2024-09-19T08:37:20.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:20.894+0000 7f3a41e27640 1 -- 172.21.15.164:0/2196586848 learned_addr learned my addr 172.21.15.164:0/2196586848 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:37:20.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:20.895+0000 7f3a41e27640 1 -- 172.21.15.164:0/2196586848 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a3c1a3920 con 0x7f3a3c074590 2024-09-19T08:37:20.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:20.895+0000 7f3a41e27640 1 --2- 172.21.15.164:0/2196586848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a3c074590 0x7f3a3c1a33e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3a38009230 tx=0x7f3a38003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:20.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:20.895+0000 7f3a32ffd640 1 -- 172.21.15.164:0/2196586848 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a3803f070 con 0x7f3a3c074590 2024-09-19T08:37:20.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:20.895+0000 7f3a32ffd640 1 -- 172.21.15.164:0/2196586848 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a3803bc80 con 0x7f3a3c074590 2024-09-19T08:37:20.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:20.895+0000 7f3a4408b640 1 -- 172.21.15.164:0/2196586848 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a3c1a3bb0 con 0x7f3a3c074590 2024-09-19T08:37:20.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:20.895+0000 7f3a32ffd640 1 -- 172.21.15.164:0/2196586848 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a38035030 con 0x7f3a3c074590 2024-09-19T08:37:20.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:20.895+0000 7f3a4408b640 1 -- 172.21.15.164:0/2196586848 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a3c19d5a0 con 0x7f3a3c074590 2024-09-19T08:37:20.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:20.896+0000 7f3a32ffd640 1 -- 172.21.15.164:0/2196586848 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f3a38042070 con 0x7f3a3c074590 2024-09-19T08:37:20.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:20.896+0000 7f3a32ffd640 1 --2- 172.21.15.164:0/2196586848 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f3a28039300 0x7f3a2803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:20.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:20.896+0000 7f3a4408b640 1 -- 172.21.15.164:0/2196586848 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a3c075cd0 con 0x7f3a3c074590 2024-09-19T08:37:20.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:20.896+0000 7f3a32ffd640 1 -- 172.21.15.164:0/2196586848 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3a3806cf80 con 0x7f3a3c074590 2024-09-19T08:37:20.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:20.896+0000 7f3a41626640 1 --2- 172.21.15.164:0/2196586848 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f3a28039300 0x7f3a2803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:20.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:20.897+0000 7f3a41626640 1 --2- 172.21.15.164:0/2196586848 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f3a28039300 0x7f3a2803b7c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c0098c0 tx=0x7f3a2c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:20.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:20.901+0000 7f3a32ffd640 1 -- 172.21.15.164:0/2196586848 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3a380372d0 con 0x7f3a3c074590 2024-09-19T08:37:21.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:21.072+0000 7f3a4408b640 1 -- 172.21.15.164:0/2196586848 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi173", "target": ["mon-mgr", ""]}) -- 0x7f3a3c0749f0 con 0x7f3a28039300 2024-09-19T08:37:23.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:22 smithi164 ceph-mon[30928]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi173", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:37:23.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:22 smithi164 ceph-mon[30928]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:23.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:22 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:23.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:22 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:23.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:22 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:23.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:22 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:23.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:22 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi170", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:37:23.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:22 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:23.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:22 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:37:24.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:23 smithi164 ceph-mon[30928]: Deploying cephadm binary to smithi173 2024-09-19T08:37:24.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:23 smithi164 ceph-mon[30928]: Updating smithi170:/etc/ceph/ceph.conf 2024-09-19T08:37:24.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:23 smithi164 ceph-mon[30928]: Updating smithi170:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:37:24.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:23 smithi164 ceph-mon[30928]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:37:24.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:23 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:24.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:23 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:24.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:23 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:24.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:24.240+0000 7f3a32ffd640 1 -- 172.21.15.164:0/2196586848 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f3a3c0749f0 con 0x7f3a28039300 2024-09-19T08:37:24.239 INFO:teuthology.orchestra.run.smithi164.stdout:Added host 'smithi173' with addr '172.21.15.173' 2024-09-19T08:37:24.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:24.242+0000 7f3a4408b640 1 -- 172.21.15.164:0/2196586848 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f3a28039300 msgr2=0x7f3a2803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:24.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:24.242+0000 7f3a4408b640 1 --2- 172.21.15.164:0/2196586848 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f3a28039300 0x7f3a2803b7c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c0098c0 tx=0x7f3a2c006db0 comp rx=0 tx=0).stop 2024-09-19T08:37:24.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:24.242+0000 7f3a4408b640 1 -- 172.21.15.164:0/2196586848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a3c074590 msgr2=0x7f3a3c1a33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:24.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:24.242+0000 7f3a4408b640 1 --2- 172.21.15.164:0/2196586848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a3c074590 0x7f3a3c1a33e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3a38009230 tx=0x7f3a38003d40 comp rx=0 tx=0).stop 2024-09-19T08:37:24.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:24.242+0000 7f3a4408b640 1 -- 172.21.15.164:0/2196586848 shutdown_connections 2024-09-19T08:37:24.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:24.242+0000 7f3a4408b640 1 --2- 172.21.15.164:0/2196586848 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f3a28039300 0x7f3a2803b7c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:24.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:24.242+0000 7f3a4408b640 1 --2- 172.21.15.164:0/2196586848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a3c074590 0x7f3a3c1a33e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:24.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:24.242+0000 7f3a4408b640 1 -- 172.21.15.164:0/2196586848 >> 172.21.15.164:0/2196586848 conn(0x7f3a3c0fd2b0 msgr2=0x7f3a3c0fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:24.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:24.242+0000 7f3a4408b640 1 -- 172.21.15.164:0/2196586848 shutdown_connections 2024-09-19T08:37:24.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:24.242+0000 7f3a4408b640 1 -- 172.21.15.164:0/2196586848 wait complete. 2024-09-19T08:37:24.814 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:24 smithi164 ceph-mon[30928]: Updating smithi170:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T08:37:24.814 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:24 smithi164 ceph-mon[30928]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:24.814 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:24 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:24.814 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:24 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:37:24.814 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:24 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:24.842 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T08:37:25.080 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:37:25.498 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.498+0000 7fefb5b5f640 1 -- 172.21.15.164:0/2515173503 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefb0106590 msgr2=0x7fefb0106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:25.498 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.498+0000 7fefb5b5f640 1 --2- 172.21.15.164:0/2515173503 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefb0106590 0x7fefb0106970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009880 tx=0x7fefa002a5d0 comp rx=0 tx=0).stop 2024-09-19T08:37:25.498 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.498+0000 7fefb5b5f640 1 -- 172.21.15.164:0/2515173503 shutdown_connections 2024-09-19T08:37:25.498 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.498+0000 7fefb5b5f640 1 --2- 172.21.15.164:0/2515173503 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefb0106590 0x7fefb0106970 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:25.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.498+0000 7fefb5b5f640 1 -- 172.21.15.164:0/2515173503 >> 172.21.15.164:0/2515173503 conn(0x7fefb00fc1a0 msgr2=0x7fefb00fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:25.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.498+0000 7fefb5b5f640 1 -- 172.21.15.164:0/2515173503 shutdown_connections 2024-09-19T08:37:25.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.498+0000 7fefb5b5f640 1 -- 172.21.15.164:0/2515173503 wait complete. 2024-09-19T08:37:25.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.499+0000 7fefb5b5f640 1 Processor -- start 2024-09-19T08:37:25.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.499+0000 7fefb5b5f640 1 -- start start 2024-09-19T08:37:25.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.499+0000 7fefb5b5f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefb0106590 0x7fefb0194ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:25.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.499+0000 7fefb5b5f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fefb0109ad0 con 0x7fefb0106590 2024-09-19T08:37:25.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.499+0000 7fefaf7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefb0106590 0x7fefb0194ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:25.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.499+0000 7fefaf7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefb0106590 0x7fefb0194ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57170/0 (socket says 172.21.15.164:57170) 2024-09-19T08:37:25.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.499+0000 7fefaf7fe640 1 -- 172.21.15.164:0/604135503 learned_addr learned my addr 172.21.15.164:0/604135503 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:37:25.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.500+0000 7fefaf7fe640 1 -- 172.21.15.164:0/604135503 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb0195420 con 0x7fefb0106590 2024-09-19T08:37:25.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.500+0000 7fefaf7fe640 1 --2- 172.21.15.164:0/604135503 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefb0106590 0x7fefb0194ee0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009230 tx=0x7fefa0003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:25.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.500+0000 7fefacff9640 1 -- 172.21.15.164:0/604135503 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefa003f070 con 0x7fefb0106590 2024-09-19T08:37:25.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.500+0000 7fefb5b5f640 1 -- 172.21.15.164:0/604135503 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefb01956b0 con 0x7fefb0106590 2024-09-19T08:37:25.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.500+0000 7fefacff9640 1 -- 172.21.15.164:0/604135503 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa003bc80 con 0x7fefb0106590 2024-09-19T08:37:25.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.500+0000 7fefacff9640 1 -- 172.21.15.164:0/604135503 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefa0035030 con 0x7fefb0106590 2024-09-19T08:37:25.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.500+0000 7fefb5b5f640 1 -- 172.21.15.164:0/604135503 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefb01997b0 con 0x7fefb0106590 2024-09-19T08:37:25.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.501+0000 7fefacff9640 1 -- 172.21.15.164:0/604135503 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fefa0042070 con 0x7fefb0106590 2024-09-19T08:37:25.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.501+0000 7fefb5b5f640 1 -- 172.21.15.164:0/604135503 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefb0101c00 con 0x7fefb0106590 2024-09-19T08:37:25.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.501+0000 7fefacff9640 1 --2- 172.21.15.164:0/604135503 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fef9c03b550 0x7fef9c03da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:25.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.501+0000 7fefacff9640 1 -- 172.21.15.164:0/604135503 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fefa006d050 con 0x7fefb0106590 2024-09-19T08:37:25.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.501+0000 7fefaeffd640 1 --2- 172.21.15.164:0/604135503 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fef9c03b550 0x7fef9c03da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:25.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.502+0000 7fefaeffd640 1 --2- 172.21.15.164:0/604135503 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fef9c03b550 0x7fef9c03da10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fef980098c0 tx=0x7fef98006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:25.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.506+0000 7fefacff9640 1 -- 172.21.15.164:0/604135503 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fefa003fde0 con 0x7fefb0106590 2024-09-19T08:37:25.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.678+0000 7fefb5b5f640 1 -- 172.21.15.164:0/604135503 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fefb0196240 con 0x7fef9c03b550 2024-09-19T08:37:25.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.680+0000 7fefacff9640 1 -- 172.21.15.164:0/604135503 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7fefb0196240 con 0x7fef9c03b550 2024-09-19T08:37:25.682 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:37:25.683 INFO:teuthology.orchestra.run.smithi164.stdout:[{"addr": "172.21.15.164", "hostname": "smithi164", "labels": [], "status": ""}, {"addr": "172.21.15.170", "hostname": "smithi170", "labels": [], "status": ""}, {"addr": "172.21.15.173", "hostname": "smithi173", "labels": [], "status": ""}] 2024-09-19T08:37:25.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.682+0000 7fefb5b5f640 1 -- 172.21.15.164:0/604135503 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fef9c03b550 msgr2=0x7fef9c03da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:25.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.682+0000 7fefb5b5f640 1 --2- 172.21.15.164:0/604135503 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fef9c03b550 0x7fef9c03da10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fef980098c0 tx=0x7fef98006db0 comp rx=0 tx=0).stop 2024-09-19T08:37:25.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.682+0000 7fefb5b5f640 1 -- 172.21.15.164:0/604135503 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefb0106590 msgr2=0x7fefb0194ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:25.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.682+0000 7fefb5b5f640 1 --2- 172.21.15.164:0/604135503 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefb0106590 0x7fefb0194ee0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009230 tx=0x7fefa0003d40 comp rx=0 tx=0).stop 2024-09-19T08:37:25.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.682+0000 7fefb5b5f640 1 -- 172.21.15.164:0/604135503 shutdown_connections 2024-09-19T08:37:25.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.682+0000 7fefb5b5f640 1 --2- 172.21.15.164:0/604135503 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fef9c03b550 0x7fef9c03da10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:25.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.682+0000 7fefb5b5f640 1 --2- 172.21.15.164:0/604135503 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefb0106590 0x7fefb0194ee0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:25.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.682+0000 7fefb5b5f640 1 -- 172.21.15.164:0/604135503 >> 172.21.15.164:0/604135503 conn(0x7fefb00fc1a0 msgr2=0x7fefb00fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:25.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.682+0000 7fefb5b5f640 1 -- 172.21.15.164:0/604135503 shutdown_connections 2024-09-19T08:37:25.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:25.682+0000 7fefb5b5f640 1 -- 172.21.15.164:0/604135503 wait complete. 2024-09-19T08:37:25.781 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:25 smithi164 ceph-mon[30928]: Added host smithi173 2024-09-19T08:37:25.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:25 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:25.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:25 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:26.292 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-19T08:37:26.292 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-19T08:37:26.535 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:37:26.923 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:26 smithi164 ceph-mon[30928]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:26.923 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:26 smithi164 ceph-mon[30928]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:37:27.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.008+0000 7fcf57223640 1 -- 172.21.15.164:0/2097616340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf50106590 msgr2=0x7fcf50106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:27.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.008+0000 7fcf57223640 1 --2- 172.21.15.164:0/2097616340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf50106590 0x7fcf50106970 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009880 tx=0x7fcf4002a5d0 comp rx=0 tx=0).stop 2024-09-19T08:37:27.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.008+0000 7fcf57223640 1 -- 172.21.15.164:0/2097616340 shutdown_connections 2024-09-19T08:37:27.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.008+0000 7fcf57223640 1 --2- 172.21.15.164:0/2097616340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf50106590 0x7fcf50106970 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:27.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.008+0000 7fcf57223640 1 -- 172.21.15.164:0/2097616340 >> 172.21.15.164:0/2097616340 conn(0x7fcf500fc1a0 msgr2=0x7fcf500fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:27.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.008+0000 7fcf57223640 1 -- 172.21.15.164:0/2097616340 shutdown_connections 2024-09-19T08:37:27.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.008+0000 7fcf57223640 1 -- 172.21.15.164:0/2097616340 wait complete. 2024-09-19T08:37:27.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.008+0000 7fcf57223640 1 Processor -- start 2024-09-19T08:37:27.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.008+0000 7fcf57223640 1 -- start start 2024-09-19T08:37:27.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.008+0000 7fcf57223640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf50106590 0x7fcf500fc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:27.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.008+0000 7fcf57223640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5010abc0 con 0x7fcf50106590 2024-09-19T08:37:27.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.009+0000 7fcf54fbf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf50106590 0x7fcf500fc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:27.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.009+0000 7fcf54fbf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf50106590 0x7fcf500fc650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57188/0 (socket says 172.21.15.164:57188) 2024-09-19T08:37:27.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.009+0000 7fcf54fbf640 1 -- 172.21.15.164:0/200116557 learned_addr learned my addr 172.21.15.164:0/200116557 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:37:27.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.009+0000 7fcf54fbf640 1 -- 172.21.15.164:0/200116557 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf500fe540 con 0x7fcf50106590 2024-09-19T08:37:27.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.009+0000 7fcf54fbf640 1 --2- 172.21.15.164:0/200116557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf50106590 0x7fcf500fc650 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009230 tx=0x7fcf40003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:27.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.009+0000 7fcf4dffb640 1 -- 172.21.15.164:0/200116557 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf4003f070 con 0x7fcf50106590 2024-09-19T08:37:27.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.009+0000 7fcf4dffb640 1 -- 172.21.15.164:0/200116557 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf4003bc80 con 0x7fcf50106590 2024-09-19T08:37:27.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.009+0000 7fcf4dffb640 1 -- 172.21.15.164:0/200116557 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf40035030 con 0x7fcf50106590 2024-09-19T08:37:27.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.009+0000 7fcf57223640 1 -- 172.21.15.164:0/200116557 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf500fcb90 con 0x7fcf50106590 2024-09-19T08:37:27.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.009+0000 7fcf57223640 1 -- 172.21.15.164:0/200116557 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf500fd050 con 0x7fcf50106590 2024-09-19T08:37:27.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.010+0000 7fcf4dffb640 1 -- 172.21.15.164:0/200116557 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fcf40042070 con 0x7fcf50106590 2024-09-19T08:37:27.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.010+0000 7fcf4dffb640 1 --2- 172.21.15.164:0/200116557 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fcf38039300 0x7fcf3803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:27.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.010+0000 7fcf57223640 1 -- 172.21.15.164:0/200116557 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf50101c00 con 0x7fcf50106590 2024-09-19T08:37:27.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.010+0000 7fcf4dffb640 1 -- 172.21.15.164:0/200116557 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcf4006cfb0 con 0x7fcf50106590 2024-09-19T08:37:27.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.010+0000 7fcf4ffff640 1 --2- 172.21.15.164:0/200116557 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fcf38039300 0x7fcf3803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:27.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.011+0000 7fcf4ffff640 1 --2- 172.21.15.164:0/200116557 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fcf38039300 0x7fcf3803b7c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcf440098c0 tx=0x7fcf44006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:27.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.015+0000 7fcf4dffb640 1 -- 172.21.15.164:0/200116557 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcf400433b0 con 0x7fcf50106590 2024-09-19T08:37:27.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.185+0000 7fcf57223640 1 -- 172.21.15.164:0/200116557 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fcf500fdb80 con 0x7fcf50106590 2024-09-19T08:37:27.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.728+0000 7fcf4dffb640 1 -- 172.21.15.164:0/200116557 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fcf400372d0 con 0x7fcf50106590 2024-09-19T08:37:27.728 INFO:teuthology.orchestra.run.smithi164.stderr:adjusted tunables profile to default 2024-09-19T08:37:27.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.730+0000 7fcf57223640 1 -- 172.21.15.164:0/200116557 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fcf38039300 msgr2=0x7fcf3803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:27.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.730+0000 7fcf57223640 1 --2- 172.21.15.164:0/200116557 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fcf38039300 0x7fcf3803b7c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcf440098c0 tx=0x7fcf44006db0 comp rx=0 tx=0).stop 2024-09-19T08:37:27.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.730+0000 7fcf57223640 1 -- 172.21.15.164:0/200116557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf50106590 msgr2=0x7fcf500fc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:27.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.730+0000 7fcf57223640 1 --2- 172.21.15.164:0/200116557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf50106590 0x7fcf500fc650 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009230 tx=0x7fcf40003d40 comp rx=0 tx=0).stop 2024-09-19T08:37:27.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.730+0000 7fcf57223640 1 -- 172.21.15.164:0/200116557 shutdown_connections 2024-09-19T08:37:27.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.730+0000 7fcf57223640 1 --2- 172.21.15.164:0/200116557 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fcf38039300 0x7fcf3803b7c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:27.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.730+0000 7fcf57223640 1 --2- 172.21.15.164:0/200116557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf50106590 0x7fcf500fc650 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:27.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.730+0000 7fcf57223640 1 -- 172.21.15.164:0/200116557 >> 172.21.15.164:0/200116557 conn(0x7fcf500fc1a0 msgr2=0x7fcf50069ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:27.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.731+0000 7fcf57223640 1 -- 172.21.15.164:0/200116557 shutdown_connections 2024-09-19T08:37:27.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:27.731+0000 7fcf57223640 1 -- 172.21.15.164:0/200116557 wait complete. 2024-09-19T08:37:28.026 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:27 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/200116557' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T08:37:28.346 INFO:tasks.cephadm:Adding mon.a on smithi164 2024-09-19T08:37:28.346 INFO:tasks.cephadm:Adding mon.b on smithi170 2024-09-19T08:37:28.347 INFO:tasks.cephadm:Adding mon.c on smithi173 2024-09-19T08:37:28.347 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi164:172.21.15.164=a;smithi170:172.21.15.170=b;smithi173:172.21.15.173=c' 2024-09-19T08:37:28.591 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T08:37:28.636 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:37:29.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:28 smithi164 ceph-mon[30928]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:29.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:28 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/200116557' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T08:37:29.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:28 smithi164 ceph-mon[30928]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T08:37:29.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:28 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:29.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:28 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:29.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:28 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:29.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:28 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:29.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:28 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi173", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:37:29.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:28 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:29.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:28 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:37:29.107 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.076+0000 7f4e5a503640 1 -- 172.21.15.173:0/3432648929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e54107660 msgr2=0x7f4e54107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:29.107 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.076+0000 7f4e5a503640 1 --2- 172.21.15.173:0/3432648929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e54107660 0x7f4e54107a40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009880 tx=0x7f4e4402a5d0 comp rx=0 tx=0).stop 2024-09-19T08:37:29.107 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.076+0000 7f4e5a503640 1 -- 172.21.15.173:0/3432648929 shutdown_connections 2024-09-19T08:37:29.108 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.076+0000 7f4e5a503640 1 --2- 172.21.15.173:0/3432648929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e54107660 0x7f4e54107a40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:29.108 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.076+0000 7f4e5a503640 1 -- 172.21.15.173:0/3432648929 >> 172.21.15.173:0/3432648929 conn(0x7f4e540fd2b0 msgr2=0x7f4e540ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:29.108 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.076+0000 7f4e5a503640 1 -- 172.21.15.173:0/3432648929 shutdown_connections 2024-09-19T08:37:29.108 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.076+0000 7f4e5a503640 1 -- 172.21.15.173:0/3432648929 wait complete. 2024-09-19T08:37:29.108 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.077+0000 7f4e5a503640 1 Processor -- start 2024-09-19T08:37:29.108 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.077+0000 7f4e5a503640 1 -- start start 2024-09-19T08:37:29.108 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.077+0000 7f4e5a503640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e54107660 0x7f4e540744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:29.108 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.077+0000 7f4e5a503640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5410bc70 con 0x7f4e54107660 2024-09-19T08:37:29.108 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.077+0000 7f4e53fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e54107660 0x7f4e540744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:29.108 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.077+0000 7f4e53fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e54107660 0x7f4e540744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.173:46802/0 (socket says 172.21.15.173:46802) 2024-09-19T08:37:29.109 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.077+0000 7f4e53fff640 1 -- 172.21.15.173:0/3581620895 learned_addr learned my addr 172.21.15.173:0/3581620895 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-09-19T08:37:29.109 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.078+0000 7f4e53fff640 1 -- 172.21.15.173:0/3581620895 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e540763e0 con 0x7f4e54107660 2024-09-19T08:37:29.109 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.078+0000 7f4e53fff640 1 --2- 172.21.15.173:0/3581620895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e54107660 0x7f4e540744f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009230 tx=0x7f4e44003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:29.109 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.078+0000 7f4e517fa640 1 -- 172.21.15.173:0/3581620895 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e4403f070 con 0x7f4e54107660 2024-09-19T08:37:29.109 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.078+0000 7f4e517fa640 1 -- 172.21.15.173:0/3581620895 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e4403bc80 con 0x7f4e54107660 2024-09-19T08:37:29.109 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.078+0000 7f4e5a503640 1 -- 172.21.15.173:0/3581620895 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e54074a90 con 0x7f4e54107660 2024-09-19T08:37:29.109 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.078+0000 7f4e517fa640 1 -- 172.21.15.173:0/3581620895 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e44035030 con 0x7f4e54107660 2024-09-19T08:37:29.109 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.078+0000 7f4e5a503640 1 -- 172.21.15.173:0/3581620895 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e54074f50 con 0x7f4e54107660 2024-09-19T08:37:29.109 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.079+0000 7f4e517fa640 1 -- 172.21.15.173:0/3581620895 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f4e44042070 con 0x7f4e54107660 2024-09-19T08:37:29.109 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.079+0000 7f4e5a503640 1 -- 172.21.15.173:0/3581620895 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e54102d10 con 0x7f4e54107660 2024-09-19T08:37:29.109 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.079+0000 7f4e517fa640 1 --2- 172.21.15.173:0/3581620895 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f4e3c0392a0 0x7f4e3c03b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:29.109 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.079+0000 7f4e517fa640 1 -- 172.21.15.173:0/3581620895 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4e4406d0c0 con 0x7f4e54107660 2024-09-19T08:37:29.109 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.079+0000 7f4e537fe640 1 --2- 172.21.15.173:0/3581620895 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f4e3c0392a0 0x7f4e3c03b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:29.110 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.080+0000 7f4e537fe640 1 --2- 172.21.15.173:0/3581620895 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f4e3c0392a0 0x7f4e3c03b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4e480098c0 tx=0x7f4e48006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:29.110 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.084+0000 7f4e517fa640 1 -- 172.21.15.173:0/3581620895 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4e44031e20 con 0x7f4e54107660 2024-09-19T08:37:29.272 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.271+0000 7f4e5a503640 1 -- 172.21.15.173:0/3581620895 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi164:172.21.15.164=a;smithi170:172.21.15.170=b;smithi173:172.21.15.173=c", "target": ["mon-mgr", ""]}) -- 0x7f4e540751e0 con 0x7f4e3c0392a0 2024-09-19T08:37:29.277 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.277+0000 7f4e517fa640 1 -- 172.21.15.173:0/3581620895 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4e540751e0 con 0x7f4e3c0392a0 2024-09-19T08:37:29.278 INFO:teuthology.orchestra.run.smithi173.stdout:Scheduled mon update... 2024-09-19T08:37:29.280 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.279+0000 7f4e5a503640 1 -- 172.21.15.173:0/3581620895 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f4e3c0392a0 msgr2=0x7f4e3c03b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:29.280 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.279+0000 7f4e5a503640 1 --2- 172.21.15.173:0/3581620895 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f4e3c0392a0 0x7f4e3c03b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4e480098c0 tx=0x7f4e48006db0 comp rx=0 tx=0).stop 2024-09-19T08:37:29.281 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.279+0000 7f4e5a503640 1 -- 172.21.15.173:0/3581620895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e54107660 msgr2=0x7f4e540744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:29.281 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.279+0000 7f4e5a503640 1 --2- 172.21.15.173:0/3581620895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e54107660 0x7f4e540744f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009230 tx=0x7f4e44003d40 comp rx=0 tx=0).stop 2024-09-19T08:37:29.281 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.279+0000 7f4e5a503640 1 -- 172.21.15.173:0/3581620895 shutdown_connections 2024-09-19T08:37:29.281 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.279+0000 7f4e5a503640 1 --2- 172.21.15.173:0/3581620895 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f4e3c0392a0 0x7f4e3c03b760 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:29.281 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.279+0000 7f4e5a503640 1 --2- 172.21.15.173:0/3581620895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e54107660 0x7f4e540744f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:29.281 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.279+0000 7f4e5a503640 1 -- 172.21.15.173:0/3581620895 >> 172.21.15.173:0/3581620895 conn(0x7f4e540fd2b0 msgr2=0x7f4e540feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:29.281 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.279+0000 7f4e5a503640 1 -- 172.21.15.173:0/3581620895 shutdown_connections 2024-09-19T08:37:29.281 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:29.279+0000 7f4e5a503640 1 -- 172.21.15.173:0/3581620895 wait complete. 2024-09-19T08:37:29.883 DEBUG:teuthology.orchestra.run.smithi170:mon.b> sudo journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mon.b.service 2024-09-19T08:37:29.886 DEBUG:teuthology.orchestra.run.smithi173:mon.c> sudo journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mon.c.service 2024-09-19T08:37:29.889 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T08:37:29.889 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T08:37:30.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:29 smithi164 ceph-mon[30928]: Updating smithi173:/etc/ceph/ceph.conf 2024-09-19T08:37:30.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:29 smithi164 ceph-mon[30928]: Updating smithi173:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:37:30.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:29 smithi164 ceph-mon[30928]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:37:30.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:29 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:30.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:29 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:30.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:29 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:30.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:29 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:30.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:29 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:37:30.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:29 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:30.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:29 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:37:30.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:29 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:30.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:29 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:37:30.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:29 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:30.527 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:37:31.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:30 smithi164 ceph-mon[30928]: Updating smithi173:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T08:37:31.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:30 smithi164 ceph-mon[30928]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi164:172.21.15.164=a;smithi170:172.21.15.170=b;smithi173:172.21.15.173=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:37:31.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:30 smithi164 ceph-mon[30928]: Saving service mon spec with placement smithi164:172.21.15.164=a;smithi170:172.21.15.170=b;smithi173:172.21.15.173=c;count:3 2024-09-19T08:37:31.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:30 smithi164 ceph-mon[30928]: Deploying daemon mon.c on smithi173 2024-09-19T08:37:31.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:30 smithi164 ceph-mon[30928]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:32.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:31 smithi164 ceph-mon[30928]: from='client.? 172.21.15.173:0/3510386455' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T08:37:32.237 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:30.939+0000 7fc97ea73640 1 -- 172.21.15.173:0/4168240087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc978106590 msgr2=0x7fc978106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:32.237 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:30.939+0000 7fc97ea73640 1 --2- 172.21.15.173:0/4168240087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc978106590 0x7fc978106970 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc96c009880 tx=0x7fc96c02a5d0 comp rx=0 tx=0).stop 2024-09-19T08:37:32.238 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:30.939+0000 7fc97ea73640 1 -- 172.21.15.173:0/4168240087 shutdown_connections 2024-09-19T08:37:32.238 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:30.939+0000 7fc97ea73640 1 --2- 172.21.15.173:0/4168240087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc978106590 0x7fc978106970 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:32.238 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:30.939+0000 7fc97ea73640 1 -- 172.21.15.173:0/4168240087 >> 172.21.15.173:0/4168240087 conn(0x7fc9780fc1a0 msgr2=0x7fc9780fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:32.238 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:30.939+0000 7fc97ea73640 1 -- 172.21.15.173:0/4168240087 shutdown_connections 2024-09-19T08:37:32.238 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:30.939+0000 7fc97ea73640 1 -- 172.21.15.173:0/4168240087 wait complete. 2024-09-19T08:37:32.238 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:30.940+0000 7fc97ea73640 1 Processor -- start 2024-09-19T08:37:32.238 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:30.940+0000 7fc97ea73640 1 -- start start 2024-09-19T08:37:32.238 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:30.940+0000 7fc97ea73640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc978106590 0x7fc97810b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:32.238 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:30.940+0000 7fc97ea73640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc97810abc0 con 0x7fc978106590 2024-09-19T08:37:32.238 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:30.940+0000 7fc97c80f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc978106590 0x7fc97810b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:32.238 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:30.940+0000 7fc97c80f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc978106590 0x7fc97810b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.173:46810/0 (socket says 172.21.15.173:46810) 2024-09-19T08:37:32.239 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:30.940+0000 7fc97c80f640 1 -- 172.21.15.173:0/3510386455 learned_addr learned my addr 172.21.15.173:0/3510386455 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-09-19T08:37:32.239 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:30.941+0000 7fc97c80f640 1 -- 172.21.15.173:0/3510386455 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc97810d300 con 0x7fc978106590 2024-09-19T08:37:32.239 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:30.941+0000 7fc97c80f640 1 --2- 172.21.15.173:0/3510386455 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc978106590 0x7fc97810b410 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc96c009230 tx=0x7fc96c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:32.239 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:30.941+0000 7fc975ffb640 1 -- 172.21.15.173:0/3510386455 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc96c03f070 con 0x7fc978106590 2024-09-19T08:37:32.239 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:30.941+0000 7fc975ffb640 1 -- 172.21.15.173:0/3510386455 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc96c03bc80 con 0x7fc978106590 2024-09-19T08:37:32.239 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:30.941+0000 7fc97ea73640 1 -- 172.21.15.173:0/3510386455 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc97810b950 con 0x7fc978106590 2024-09-19T08:37:32.239 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:30.941+0000 7fc975ffb640 1 -- 172.21.15.173:0/3510386455 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc96c035030 con 0x7fc978106590 2024-09-19T08:37:32.239 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:30.941+0000 7fc97ea73640 1 -- 172.21.15.173:0/3510386455 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc97810be10 con 0x7fc978106590 2024-09-19T08:37:32.239 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:30.942+0000 7fc975ffb640 1 -- 172.21.15.173:0/3510386455 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fc96c042070 con 0x7fc978106590 2024-09-19T08:37:32.239 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:30.942+0000 7fc97ea73640 1 -- 172.21.15.173:0/3510386455 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc978101c00 con 0x7fc978106590 2024-09-19T08:37:32.239 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:30.942+0000 7fc975ffb640 1 --2- 172.21.15.173:0/3510386455 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fc9680392a0 0x7fc96803b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:32.239 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:30.942+0000 7fc975ffb640 1 -- 172.21.15.173:0/3510386455 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc96c06d090 con 0x7fc978106590 2024-09-19T08:37:32.239 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:30.942+0000 7fc977fff640 1 --2- 172.21.15.173:0/3510386455 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fc9680392a0 0x7fc96803b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:32.240 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:30.943+0000 7fc977fff640 1 --2- 172.21.15.173:0/3510386455 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fc9680392a0 0x7fc96803b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc9640098c0 tx=0x7fc964006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:32.240 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:30.947+0000 7fc975ffb640 1 -- 172.21.15.173:0/3510386455 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc96c031e20 con 0x7fc978106590 2024-09-19T08:37:32.240 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:31.181+0000 7fc97ea73640 1 -- 172.21.15.173:0/3510386455 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc97810c940 con 0x7fc978106590 2024-09-19T08:37:32.240 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:31.182+0000 7fc975ffb640 1 -- 172.21.15.173:0/3510386455 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7fc96c0372d0 con 0x7fc978106590 2024-09-19T08:37:32.240 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 1 2024-09-19T08:37:32.240 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:31.184+0000 7fc97ea73640 1 -- 172.21.15.173:0/3510386455 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fc9680392a0 msgr2=0x7fc96803b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:32.240 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:31.184+0000 7fc97ea73640 1 --2- 172.21.15.173:0/3510386455 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fc9680392a0 0x7fc96803b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc9640098c0 tx=0x7fc964006db0 comp rx=0 tx=0).stop 2024-09-19T08:37:32.240 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:31.184+0000 7fc97ea73640 1 -- 172.21.15.173:0/3510386455 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc978106590 msgr2=0x7fc97810b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:32.240 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:31.184+0000 7fc97ea73640 1 --2- 172.21.15.173:0/3510386455 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc978106590 0x7fc97810b410 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc96c009230 tx=0x7fc96c003d40 comp rx=0 tx=0).stop 2024-09-19T08:37:32.240 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:31.184+0000 7fc97ea73640 1 -- 172.21.15.173:0/3510386455 shutdown_connections 2024-09-19T08:37:32.240 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:31.184+0000 7fc97ea73640 1 --2- 172.21.15.173:0/3510386455 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fc9680392a0 0x7fc96803b760 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:32.240 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:31.184+0000 7fc97ea73640 1 --2- 172.21.15.173:0/3510386455 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc978106590 0x7fc97810b410 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:32.241 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:31.184+0000 7fc97ea73640 1 -- 172.21.15.173:0/3510386455 >> 172.21.15.173:0/3510386455 conn(0x7fc9780fc1a0 msgr2=0x7fc9780fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:32.241 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:31.184+0000 7fc97ea73640 1 -- 172.21.15.173:0/3510386455 shutdown_connections 2024-09-19T08:37:32.241 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:31.184+0000 7fc97ea73640 1 -- 172.21.15.173:0/3510386455 wait complete. 2024-09-19T08:37:32.241 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-19T08:37:32.241 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":1,"fsid":"1e4b0d74-7662-11ef-bceb-c7b262605968","modified":"2024-09-19T08:36:06.626290Z","created":"2024-09-19T08:36:06.626290Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:3300","nonce":0},{"type":"v1","addr":"172.21.15.164:6789","nonce":0}]},"addr":"172.21.15.164:6789/0","public_addr":"172.21.15.164:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T08:37:33.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:32 smithi164 ceph-mon[30928]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:33.679 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-19T08:37:33.903 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T08:37:33.903 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T08:37:33.929 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-19T08:37:33.930 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-19T08:37:33.930 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-19T08:37:33.930 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-19T08:37:33.930 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-19T08:37:33.930 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T08:37:33.930 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T08:37:33.930 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T08:37:33.930 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T08:37:33.930 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi173", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:37:33.930 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:33.930 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:33.930 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:33.931 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:33.931 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:33.931 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi170", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:37:33.931 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:33.931 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:37:33.931 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: Deploying cephadm binary to smithi173 2024-09-19T08:37:33.931 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: Updating smithi170:/etc/ceph/ceph.conf 2024-09-19T08:37:33.931 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: Updating smithi170:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:37:33.931 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:37:33.931 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:33.931 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:33.931 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:33.931 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: Updating smithi170:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T08:37:33.932 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:33.932 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:33.932 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:37:33.932 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:33.932 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: Added host smithi173 2024-09-19T08:37:33.932 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:33.932 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:33.932 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:33.933 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:37:33.933 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/200116557' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T08:37:33.933 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:33.933 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/200116557' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T08:37:33.933 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T08:37:33.933 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:33.933 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:33.933 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:33.933 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:33.933 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi173", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:37:33.933 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:33.933 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:37:33.933 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: Updating smithi173:/etc/ceph/ceph.conf 2024-09-19T08:37:33.934 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: Updating smithi173:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:37:33.934 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:37:33.934 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:33.934 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:33.934 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:33.934 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:33.934 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:37:33.934 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:33.934 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:37:33.934 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:33.934 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:37:33.934 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:33.934 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: Updating smithi173:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T08:37:33.934 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi164:172.21.15.164=a;smithi170:172.21.15.170=b;smithi173:172.21.15.173=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:37:33.935 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: Saving service mon spec with placement smithi164:172.21.15.164=a;smithi170:172.21.15.170=b;smithi173:172.21.15.173=c;count:3 2024-09-19T08:37:33.935 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: Deploying daemon mon.c on smithi173 2024-09-19T08:37:33.935 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:33.935 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: from='client.? 172.21.15.173:0/3510386455' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T08:37:33.935 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:33.935 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 ceph-mon[35306]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-19T08:37:33.935 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 bash[35287]: e5ca074777f9e05eca868f38677adcb8fe5d452cfc5a7171b45fc38a08928a2e 2024-09-19T08:37:33.935 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:33 smithi173 systemd[1]: Started Ceph mon.c for 1e4b0d74-7662-11ef-bceb-c7b262605968. 2024-09-19T08:37:34.157 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:37:38.968 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.968+0000 7fd977ff8640 1 -- 172.21.15.173:0/1537628607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd95c003530 msgr2=0x7fd95c0059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:38.968 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.968+0000 7fd977ff8640 1 --2- 172.21.15.173:0/1537628607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd95c003530 0x7fd95c0059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd9701016c0 tx=0x7fd96c02c490 comp rx=0 tx=0).stop 2024-09-19T08:37:38.969 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.968+0000 7fd977ff8640 1 -- 172.21.15.173:0/1537628607 shutdown_connections 2024-09-19T08:37:38.969 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.968+0000 7fd977ff8640 1 --2- 172.21.15.173:0/1537628607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd95c003530 0x7fd95c0059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:38.969 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.968+0000 7fd977ff8640 1 -- 172.21.15.173:0/1537628607 >> 172.21.15.173:0/1537628607 conn(0x7fd9700fc280 msgr2=0x7fd9700fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:38.969 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.968+0000 7fd977ff8640 1 -- 172.21.15.173:0/1537628607 shutdown_connections 2024-09-19T08:37:38.969 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.968+0000 7fd977ff8640 1 -- 172.21.15.173:0/1537628607 wait complete. 2024-09-19T08:37:38.970 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.969+0000 7fd977ff8640 1 Processor -- start 2024-09-19T08:37:38.971 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.969+0000 7fd977ff8640 1 -- start start 2024-09-19T08:37:38.971 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.969+0000 7fd977ff8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd95c003530 0x7fd97018e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:38.971 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.969+0000 7fd977ff8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd970192790 0x7fd97018ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:38.971 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.969+0000 7fd977ff8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd970108690 con 0x7fd970192790 2024-09-19T08:37:38.971 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.969+0000 7fd977ff8640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd970108510 con 0x7fd95c003530 2024-09-19T08:37:38.971 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.969+0000 7fd975d94640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd95c003530 0x7fd97018e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:38.971 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.970+0000 7fd975d94640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd95c003530 0x7fd97018e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:55094/0 (socket says 172.21.15.173:55094) 2024-09-19T08:37:38.971 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.970+0000 7fd975d94640 1 -- 172.21.15.173:0/2594347568 learned_addr learned my addr 172.21.15.173:0/2594347568 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-09-19T08:37:38.971 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.970+0000 7fd975d94640 1 -- 172.21.15.173:0/2594347568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd95c003530 msgr2=0x7fd97018e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-19T08:37:38.971 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.970+0000 7fd975d94640 1 -- 172.21.15.173:0/2594347568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd95c003530 msgr2=0x7fd97018e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T08:37:38.971 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.970+0000 7fd975d94640 1 --2- 172.21.15.173:0/2594347568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd95c003530 0x7fd97018e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T08:37:38.971 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.970+0000 7fd975d94640 1 --2- 172.21.15.173:0/2594347568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd95c003530 0x7fd97018e830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T08:37:38.972 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.970+0000 7fd975593640 1 --2- 172.21.15.173:0/2594347568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd970192790 0x7fd97018ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:38.972 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.970+0000 7fd975593640 1 -- 172.21.15.173:0/2594347568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd95c003530 msgr2=0x7fd97018e830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:37:38.972 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.970+0000 7fd975593640 1 --2- 172.21.15.173:0/2594347568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd95c003530 0x7fd97018e830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:38.972 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.970+0000 7fd975593640 1 -- 172.21.15.173:0/2594347568 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd97018f470 con 0x7fd970192790 2024-09-19T08:37:38.972 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.970+0000 7fd975593640 1 --2- 172.21.15.173:0/2594347568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd970192790 0x7fd97018ed70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd96000b3d0 tx=0x7fd96000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:38.972 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.971+0000 7fd966ffd640 1 -- 172.21.15.173:0/2594347568 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd960011630 con 0x7fd970192790 2024-09-19T08:37:38.972 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.971+0000 7fd977ff8640 1 -- 172.21.15.173:0/2594347568 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9701a7a90 con 0x7fd970192790 2024-09-19T08:37:38.972 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.971+0000 7fd966ffd640 1 -- 172.21.15.173:0/2594347568 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd960011cb0 con 0x7fd970192790 2024-09-19T08:37:38.972 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.971+0000 7fd966ffd640 1 -- 172.21.15.173:0/2594347568 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd960010df0 con 0x7fd970192790 2024-09-19T08:37:38.972 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.971+0000 7fd977ff8640 1 -- 172.21.15.173:0/2594347568 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9701a7f10 con 0x7fd970192790 2024-09-19T08:37:38.973 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.972+0000 7fd966ffd640 1 -- 172.21.15.173:0/2594347568 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fd960018020 con 0x7fd970192790 2024-09-19T08:37:38.973 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.972+0000 7fd966ffd640 1 --2- 172.21.15.173:0/2594347568 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fd954039040 0x7fd95403b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:38.973 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.972+0000 7fd966ffd640 1 -- 172.21.15.173:0/2594347568 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd960011e60 con 0x7fd970192790 2024-09-19T08:37:38.973 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.972+0000 7fd977ff8640 1 -- 172.21.15.173:0/2594347568 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd970101cc0 con 0x7fd970192790 2024-09-19T08:37:38.973 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.972+0000 7fd975d94640 1 --2- 172.21.15.173:0/2594347568 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fd954039040 0x7fd95403b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:38.974 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.973+0000 7fd975d94640 1 --2- 172.21.15.173:0/2594347568 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fd954039040 0x7fd95403b500 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd96c012c10 tx=0x7fd96c0323a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:38.977 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:38.977+0000 7fd966ffd640 1 -- 172.21.15.173:0/2594347568 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd96001f020 con 0x7fd970192790 2024-09-19T08:37:39.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:38 smithi164 ceph-mon[30928]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:39.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:38 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:37:39.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:38 smithi164 ceph-mon[30928]: mon.a calling monitor election 2024-09-19T08:37:39.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:38 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:37:39.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:38 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:37:39.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:38 smithi164 ceph-mon[30928]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:39.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:38 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:37:39.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:38 smithi164 ceph-mon[30928]: mon.c calling monitor election 2024-09-19T08:37:39.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:38 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:37:39.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:38 smithi164 ceph-mon[30928]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:39.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:38 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:37:39.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:38 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:37:39.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:38 smithi164 ceph-mon[30928]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T08:37:39.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:38 smithi164 ceph-mon[30928]: monmap epoch 2 2024-09-19T08:37:39.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:38 smithi164 ceph-mon[30928]: fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 2024-09-19T08:37:39.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:38 smithi164 ceph-mon[30928]: last_changed 2024-09-19T08:37:33.685168+0000 2024-09-19T08:37:39.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:38 smithi164 ceph-mon[30928]: created 2024-09-19T08:36:06.626290+0000 2024-09-19T08:37:39.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:38 smithi164 ceph-mon[30928]: min_mon_release 19 (squid) 2024-09-19T08:37:39.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:38 smithi164 ceph-mon[30928]: election_strategy: 1 2024-09-19T08:37:39.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:38 smithi164 ceph-mon[30928]: 0: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.a 2024-09-19T08:37:39.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:38 smithi164 ceph-mon[30928]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.c 2024-09-19T08:37:39.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:38 smithi164 ceph-mon[30928]: fsmap 2024-09-19T08:37:39.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:38 smithi164 ceph-mon[30928]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T08:37:39.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:38 smithi164 ceph-mon[30928]: mgrmap e13: x(active, since 37s) 2024-09-19T08:37:39.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:38 smithi164 ceph-mon[30928]: overall HEALTH_OK 2024-09-19T08:37:39.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:38 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:39.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:38 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:39.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:38 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:39.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:38 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:37:39.035 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:38 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:39.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:38 smithi173 ceph-mon[35306]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:39.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:38 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:37:39.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:38 smithi173 ceph-mon[35306]: mon.a calling monitor election 2024-09-19T08:37:39.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:38 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:37:39.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:38 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:37:39.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:38 smithi173 ceph-mon[35306]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:39.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:38 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:37:39.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:38 smithi173 ceph-mon[35306]: mon.c calling monitor election 2024-09-19T08:37:39.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:38 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:37:39.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:38 smithi173 ceph-mon[35306]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:39.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:38 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:37:39.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:38 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:37:39.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:38 smithi173 ceph-mon[35306]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T08:37:39.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:38 smithi173 ceph-mon[35306]: monmap epoch 2 2024-09-19T08:37:39.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:38 smithi173 ceph-mon[35306]: fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 2024-09-19T08:37:39.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:38 smithi173 ceph-mon[35306]: last_changed 2024-09-19T08:37:33.685168+0000 2024-09-19T08:37:39.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:38 smithi173 ceph-mon[35306]: created 2024-09-19T08:36:06.626290+0000 2024-09-19T08:37:39.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:38 smithi173 ceph-mon[35306]: min_mon_release 19 (squid) 2024-09-19T08:37:39.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:38 smithi173 ceph-mon[35306]: election_strategy: 1 2024-09-19T08:37:39.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:38 smithi173 ceph-mon[35306]: 0: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.a 2024-09-19T08:37:39.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:38 smithi173 ceph-mon[35306]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.c 2024-09-19T08:37:39.163 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:38 smithi173 ceph-mon[35306]: fsmap 2024-09-19T08:37:39.163 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:38 smithi173 ceph-mon[35306]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T08:37:39.163 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:38 smithi173 ceph-mon[35306]: mgrmap e13: x(active, since 37s) 2024-09-19T08:37:39.163 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:38 smithi173 ceph-mon[35306]: overall HEALTH_OK 2024-09-19T08:37:39.163 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:38 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:39.163 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:38 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:39.163 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:38 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:39.163 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:38 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:37:39.163 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:38 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:39.211 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:39.211+0000 7fd977ff8640 1 -- 172.21.15.173:0/2594347568 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd9701a8280 con 0x7fd970192790 2024-09-19T08:37:39.212 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:39.212+0000 7fd966ffd640 1 -- 172.21.15.173:0/2594347568 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7fd960018310 con 0x7fd970192790 2024-09-19T08:37:39.212 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-19T08:37:39.212 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":2,"fsid":"1e4b0d74-7662-11ef-bceb-c7b262605968","modified":"2024-09-19T08:37:33.685168Z","created":"2024-09-19T08:36:06.626290Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:3300","nonce":0},{"type":"v1","addr":"172.21.15.164:6789","nonce":0}]},"addr":"172.21.15.164:6789/0","public_addr":"172.21.15.164:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:3300","nonce":0},{"type":"v1","addr":"172.21.15.173:6789","nonce":0}]},"addr":"172.21.15.173:6789/0","public_addr":"172.21.15.173:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T08:37:39.213 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 2 2024-09-19T08:37:39.214 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:39.214+0000 7fd977ff8640 1 -- 172.21.15.173:0/2594347568 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fd954039040 msgr2=0x7fd95403b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:39.214 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:39.214+0000 7fd977ff8640 1 --2- 172.21.15.173:0/2594347568 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fd954039040 0x7fd95403b500 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd96c012c10 tx=0x7fd96c0323a0 comp rx=0 tx=0).stop 2024-09-19T08:37:39.214 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:39.214+0000 7fd977ff8640 1 -- 172.21.15.173:0/2594347568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd970192790 msgr2=0x7fd97018ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:39.214 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:39.214+0000 7fd977ff8640 1 --2- 172.21.15.173:0/2594347568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd970192790 0x7fd97018ed70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd96000b3d0 tx=0x7fd96000b890 comp rx=0 tx=0).stop 2024-09-19T08:37:39.214 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:39.214+0000 7fd977ff8640 1 -- 172.21.15.173:0/2594347568 shutdown_connections 2024-09-19T08:37:39.214 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:39.214+0000 7fd977ff8640 1 --2- 172.21.15.173:0/2594347568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd970192790 0x7fd97018ed70 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:39.214 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:39.214+0000 7fd977ff8640 1 --2- 172.21.15.173:0/2594347568 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fd954039040 0x7fd95403b500 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:39.214 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:39.214+0000 7fd977ff8640 1 --2- 172.21.15.173:0/2594347568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd95c003530 0x7fd97018e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:39.214 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:39.214+0000 7fd977ff8640 1 -- 172.21.15.173:0/2594347568 >> 172.21.15.173:0/2594347568 conn(0x7fd9700fc280 msgr2=0x7fd97006a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:39.215 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:39.214+0000 7fd977ff8640 1 -- 172.21.15.173:0/2594347568 shutdown_connections 2024-09-19T08:37:39.215 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:39.214+0000 7fd977ff8640 1 -- 172.21.15.173:0/2594347568 wait complete. 2024-09-19T08:37:40.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:39 smithi164 ceph-mon[30928]: Deploying daemon mon.b on smithi170 2024-09-19T08:37:40.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:39 smithi164 ceph-mon[30928]: from='client.? 172.21.15.173:0/2594347568' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T08:37:40.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:39 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:37:40.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:39 smithi173 ceph-mon[35306]: Deploying daemon mon.b on smithi170 2024-09-19T08:37:40.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:39 smithi173 ceph-mon[35306]: from='client.? 172.21.15.173:0/2594347568' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T08:37:40.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:39 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:37:40.814 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T08:37:40.815 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T08:37:41.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:40 smithi164 ceph-mon[30928]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:41.057 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:37:41.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:40 smithi173 ceph-mon[35306]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:41.480 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.478+0000 7feb3d87a640 1 -- 172.21.15.173:0/2119529229 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb38106660 msgr2=0x7feb38106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:41.481 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.478+0000 7feb3d87a640 1 --2- 172.21.15.173:0/2119529229 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb38106660 0x7feb38106a40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7feb28009880 tx=0x7feb2802a5d0 comp rx=0 tx=0).stop 2024-09-19T08:37:41.481 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.478+0000 7feb3d87a640 1 -- 172.21.15.173:0/2119529229 shutdown_connections 2024-09-19T08:37:41.481 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.478+0000 7feb3d87a640 1 --2- 172.21.15.173:0/2119529229 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb38106660 0x7feb38106a40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:41.481 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.478+0000 7feb3d87a640 1 -- 172.21.15.173:0/2119529229 >> 172.21.15.173:0/2119529229 conn(0x7feb380fc250 msgr2=0x7feb380fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:41.481 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.478+0000 7feb3d87a640 1 -- 172.21.15.173:0/2119529229 shutdown_connections 2024-09-19T08:37:41.481 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.478+0000 7feb3d87a640 1 -- 172.21.15.173:0/2119529229 wait complete. 2024-09-19T08:37:41.481 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.479+0000 7feb3d87a640 1 Processor -- start 2024-09-19T08:37:41.481 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.479+0000 7feb3d87a640 1 -- start start 2024-09-19T08:37:41.481 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.479+0000 7feb3d87a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb38106660 0x7feb381973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:41.481 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.479+0000 7feb3d87a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb381978e0 0x7feb3819bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:41.481 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.479+0000 7feb3d87a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feb3810ac90 con 0x7feb38106660 2024-09-19T08:37:41.482 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.479+0000 7feb3d87a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feb3810ab10 con 0x7feb381978e0 2024-09-19T08:37:41.482 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.479+0000 7feb2ffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb381978e0 0x7feb3819bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:41.482 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.479+0000 7feb36ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb38106660 0x7feb381973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:41.482 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.479+0000 7feb2ffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb381978e0 0x7feb3819bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:55104/0 (socket says 172.21.15.173:55104) 2024-09-19T08:37:41.482 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.479+0000 7feb2ffff640 1 -- 172.21.15.173:0/1484121889 learned_addr learned my addr 172.21.15.173:0/1484121889 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-09-19T08:37:41.482 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.479+0000 7feb2ffff640 1 -- 172.21.15.173:0/1484121889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb381978e0 msgr2=0x7feb3819bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-19T08:37:41.482 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.479+0000 7feb2ffff640 1 -- 172.21.15.173:0/1484121889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb381978e0 msgr2=0x7feb3819bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T08:37:41.482 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.479+0000 7feb2ffff640 1 --2- 172.21.15.173:0/1484121889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb381978e0 0x7feb3819bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T08:37:41.482 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.479+0000 7feb2ffff640 1 --2- 172.21.15.173:0/1484121889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb381978e0 0x7feb3819bc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T08:37:41.482 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.479+0000 7feb36ffd640 1 -- 172.21.15.173:0/1484121889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb381978e0 msgr2=0x7feb3819bc70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:37:41.483 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.479+0000 7feb36ffd640 1 --2- 172.21.15.173:0/1484121889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb381978e0 0x7feb3819bc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:41.483 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.479+0000 7feb36ffd640 1 -- 172.21.15.173:0/1484121889 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb38197f00 con 0x7feb38106660 2024-09-19T08:37:41.483 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.480+0000 7feb36ffd640 1 --2- 172.21.15.173:0/1484121889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb38106660 0x7feb381973a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7feb28005ab0 tx=0x7feb28032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:41.483 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.480+0000 7feb3c878640 1 -- 172.21.15.173:0/1484121889 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7feb28035070 con 0x7feb38106660 2024-09-19T08:37:41.483 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.480+0000 7feb3d87a640 1 -- 172.21.15.173:0/1484121889 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb3819c1b0 con 0x7feb38106660 2024-09-19T08:37:41.483 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.480+0000 7feb3c878640 1 -- 172.21.15.173:0/1484121889 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb28032e40 con 0x7feb38106660 2024-09-19T08:37:41.483 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.480+0000 7feb3c878640 1 -- 172.21.15.173:0/1484121889 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7feb2803bc10 con 0x7feb38106660 2024-09-19T08:37:41.483 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.480+0000 7feb3d87a640 1 -- 172.21.15.173:0/1484121889 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb3819c710 con 0x7feb38106660 2024-09-19T08:37:41.483 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.481+0000 7feb3d87a640 1 -- 172.21.15.173:0/1484121889 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb38101cd0 con 0x7feb38106660 2024-09-19T08:37:41.483 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.481+0000 7feb3c878640 1 -- 172.21.15.173:0/1484121889 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7feb28042030 con 0x7feb38106660 2024-09-19T08:37:41.484 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.481+0000 7feb3c878640 1 --2- 172.21.15.173:0/1484121889 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7feb24039040 0x7feb2403b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:41.484 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.481+0000 7feb3c878640 1 -- 172.21.15.173:0/1484121889 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7feb28040070 con 0x7feb38106660 2024-09-19T08:37:41.485 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.481+0000 7feb2ffff640 1 --2- 172.21.15.173:0/1484121889 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7feb24039040 0x7feb2403b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:41.485 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.482+0000 7feb2ffff640 1 --2- 172.21.15.173:0/1484121889 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7feb24039040 0x7feb2403b500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7feb20009100 tx=0x7feb20008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:41.486 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.486+0000 7feb3c878640 1 -- 172.21.15.173:0/1484121889 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7feb2804a260 con 0x7feb38106660 2024-09-19T08:37:41.718 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.718+0000 7feb3d87a640 1 -- 172.21.15.173:0/1484121889 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7feb380008d0 con 0x7feb38106660 2024-09-19T08:37:41.718 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.719+0000 7feb3c878640 1 -- 172.21.15.173:0/1484121889 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7feb2803bdb0 con 0x7feb38106660 2024-09-19T08:37:41.719 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-19T08:37:41.719 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":2,"fsid":"1e4b0d74-7662-11ef-bceb-c7b262605968","modified":"2024-09-19T08:37:33.685168Z","created":"2024-09-19T08:36:06.626290Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:3300","nonce":0},{"type":"v1","addr":"172.21.15.164:6789","nonce":0}]},"addr":"172.21.15.164:6789/0","public_addr":"172.21.15.164:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:3300","nonce":0},{"type":"v1","addr":"172.21.15.173:6789","nonce":0}]},"addr":"172.21.15.173:6789/0","public_addr":"172.21.15.173:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T08:37:41.719 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 2 2024-09-19T08:37:41.721 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.721+0000 7feb3d87a640 1 -- 172.21.15.173:0/1484121889 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7feb24039040 msgr2=0x7feb2403b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:41.721 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.721+0000 7feb3d87a640 1 --2- 172.21.15.173:0/1484121889 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7feb24039040 0x7feb2403b500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7feb20009100 tx=0x7feb20008040 comp rx=0 tx=0).stop 2024-09-19T08:37:41.721 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.721+0000 7feb3d87a640 1 -- 172.21.15.173:0/1484121889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb38106660 msgr2=0x7feb381973a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:41.721 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.721+0000 7feb3d87a640 1 --2- 172.21.15.173:0/1484121889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb38106660 0x7feb381973a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7feb28005ab0 tx=0x7feb28032c50 comp rx=0 tx=0).stop 2024-09-19T08:37:41.721 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.721+0000 7feb3d87a640 1 -- 172.21.15.173:0/1484121889 shutdown_connections 2024-09-19T08:37:41.721 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.721+0000 7feb3d87a640 1 --2- 172.21.15.173:0/1484121889 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7feb24039040 0x7feb2403b500 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:41.721 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.721+0000 7feb3d87a640 1 --2- 172.21.15.173:0/1484121889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb381978e0 0x7feb3819bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:41.721 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.721+0000 7feb3d87a640 1 --2- 172.21.15.173:0/1484121889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb38106660 0x7feb381973a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:41.721 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.721+0000 7feb3d87a640 1 -- 172.21.15.173:0/1484121889 >> 172.21.15.173:0/1484121889 conn(0x7feb380fc250 msgr2=0x7feb38104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:41.721 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.721+0000 7feb3d87a640 1 -- 172.21.15.173:0/1484121889 shutdown_connections 2024-09-19T08:37:41.721 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:41.721+0000 7feb3d87a640 1 -- 172.21.15.173:0/1484121889 wait complete. 2024-09-19T08:37:42.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:41 smithi164 ceph-mon[30928]: from='client.? 172.21.15.173:0/1484121889' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T08:37:42.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:41 smithi173 ceph-mon[35306]: from='client.? 172.21.15.173:0/1484121889' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T08:37:42.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:42 smithi170 systemd[1]: Started Ceph mon.b for 1e4b0d74-7662-11ef-bceb-c7b262605968. 2024-09-19T08:37:43.322 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T08:37:43.322 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T08:37:43.572 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:37:47.622 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.620+0000 7fce6c75f640 1 -- 172.21.15.173:0/20654776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce50003530 msgr2=0x7fce500059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:47.622 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.620+0000 7fce6c75f640 1 --2- 172.21.15.173:0/20654776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce50003530 0x7fce500059c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fce641016a0 tx=0x7fce6002c3f0 comp rx=0 tx=0).stop 2024-09-19T08:37:47.623 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.620+0000 7fce6c75f640 1 -- 172.21.15.173:0/20654776 shutdown_connections 2024-09-19T08:37:47.623 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.620+0000 7fce6c75f640 1 --2- 172.21.15.173:0/20654776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce50003530 0x7fce500059c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:47.623 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.620+0000 7fce6c75f640 1 -- 172.21.15.173:0/20654776 >> 172.21.15.173:0/20654776 conn(0x7fce640fc220 msgr2=0x7fce640fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:47.623 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.620+0000 7fce6c75f640 1 -- 172.21.15.173:0/20654776 shutdown_connections 2024-09-19T08:37:47.623 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.620+0000 7fce6c75f640 1 -- 172.21.15.173:0/20654776 wait complete. 2024-09-19T08:37:47.623 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.621+0000 7fce6c75f640 1 Processor -- start 2024-09-19T08:37:47.623 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.621+0000 7fce6c75f640 1 -- start start 2024-09-19T08:37:47.623 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.621+0000 7fce6c75f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce50003530 0x7fce641972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:47.623 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.621+0000 7fce6c75f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce64197830 0x7fce6419bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:47.623 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.621+0000 7fce6c75f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce6419c100 0x7fce6419c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:47.623 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.621+0000 7fce6c75f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fce64108670 con 0x7fce6419c100 2024-09-19T08:37:47.623 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.621+0000 7fce6c75f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fce641084f0 con 0x7fce64197830 2024-09-19T08:37:47.624 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.621+0000 7fce6c75f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fce641087f0 con 0x7fce50003530 2024-09-19T08:37:47.624 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.621+0000 7fce6a4fb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce50003530 0x7fce641972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:47.624 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.621+0000 7fce6a4fb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce50003530 0x7fce641972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:55112/0 (socket says 172.21.15.173:55112) 2024-09-19T08:37:47.624 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.621+0000 7fce6a4fb640 1 -- 172.21.15.173:0/2497814673 learned_addr learned my addr 172.21.15.173:0/2497814673 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-09-19T08:37:47.624 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.621+0000 7fce6acfc640 1 --2- 172.21.15.173:0/2497814673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce6419c100 0x7fce6419c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:47.624 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.621+0000 7fce69cfa640 1 --2- 172.21.15.173:0/2497814673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce64197830 0x7fce6419bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:47.624 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.621+0000 7fce6a4fb640 1 -- 172.21.15.173:0/2497814673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce50003530 msgr2=0x7fce641972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-19T08:37:47.624 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.621+0000 7fce6a4fb640 1 -- 172.21.15.173:0/2497814673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce50003530 msgr2=0x7fce641972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T08:37:47.624 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.622+0000 7fce6a4fb640 1 --2- 172.21.15.173:0/2497814673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce50003530 0x7fce641972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T08:37:47.624 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.622+0000 7fce6a4fb640 1 --2- 172.21.15.173:0/2497814673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce50003530 0x7fce641972f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T08:37:47.624 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.622+0000 7fce69cfa640 1 -- 172.21.15.173:0/2497814673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce64197830 msgr2=0x7fce6419bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-19T08:37:47.624 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.622+0000 7fce69cfa640 1 -- 172.21.15.173:0/2497814673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce64197830 msgr2=0x7fce6419bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T08:37:47.625 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.622+0000 7fce69cfa640 1 --2- 172.21.15.173:0/2497814673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce64197830 0x7fce6419bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T08:37:47.625 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.622+0000 7fce69cfa640 1 --2- 172.21.15.173:0/2497814673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce64197830 0x7fce6419bbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T08:37:47.625 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.622+0000 7fce6acfc640 1 -- 172.21.15.173:0/2497814673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce50003530 msgr2=0x7fce641972f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:37:47.625 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.622+0000 7fce6acfc640 1 --2- 172.21.15.173:0/2497814673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce50003530 0x7fce641972f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:47.625 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.622+0000 7fce6acfc640 1 -- 172.21.15.173:0/2497814673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce64197830 msgr2=0x7fce6419bbc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:37:47.625 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.622+0000 7fce6acfc640 1 --2- 172.21.15.173:0/2497814673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce64197830 0x7fce6419bbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:47.625 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.622+0000 7fce6acfc640 1 -- 172.21.15.173:0/2497814673 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce641a3e70 con 0x7fce6419c100 2024-09-19T08:37:47.625 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.622+0000 7fce6acfc640 1 --2- 172.21.15.173:0/2497814673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce6419c100 0x7fce6419c5b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fce5c00bd50 tx=0x7fce5c00be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:47.625 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.622+0000 7fce5b7fe640 1 -- 172.21.15.173:0/2497814673 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c019070 con 0x7fce6419c100 2024-09-19T08:37:47.625 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.622+0000 7fce5b7fe640 1 -- 172.21.15.173:0/2497814673 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce5c014920 con 0x7fce6419c100 2024-09-19T08:37:47.625 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.622+0000 7fce6c75f640 1 -- 172.21.15.173:0/2497814673 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce641a40a0 con 0x7fce6419c100 2024-09-19T08:37:47.625 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.622+0000 7fce5b7fe640 1 -- 172.21.15.173:0/2497814673 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c012990 con 0x7fce6419c100 2024-09-19T08:37:47.626 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.622+0000 7fce6c75f640 1 -- 172.21.15.173:0/2497814673 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce641a45e0 con 0x7fce6419c100 2024-09-19T08:37:47.626 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.623+0000 7fce5b7fe640 1 -- 172.21.15.173:0/2497814673 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fce5c01c070 con 0x7fce6419c100 2024-09-19T08:37:47.626 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.624+0000 7fce5b7fe640 1 --2- 172.21.15.173:0/2497814673 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fce48039730 0x7fce4803bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:47.627 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.624+0000 7fce5b7fe640 1 -- 172.21.15.173:0/2497814673 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fce5c04d0e0 con 0x7fce6419c100 2024-09-19T08:37:47.627 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.624+0000 7fce6a4fb640 1 --2- 172.21.15.173:0/2497814673 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fce48039730 0x7fce4803bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:47.627 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.624+0000 7fce6c75f640 1 -- 172.21.15.173:0/2497814673 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce641097a0 con 0x7fce6419c100 2024-09-19T08:37:47.627 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.625+0000 7fce6a4fb640 1 --2- 172.21.15.173:0/2497814673 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fce48039730 0x7fce4803bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fce60005fd0 tx=0x7fce60005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:47.629 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.629+0000 7fce5b7fe640 1 -- 172.21.15.173:0/2497814673 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fce5c00e0a0 con 0x7fce6419c100 2024-09-19T08:37:47.871 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.871+0000 7fce6c75f640 1 -- 172.21.15.173:0/2497814673 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fce641a4870 con 0x7fce6419c100 2024-09-19T08:37:47.872 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.872+0000 7fce5b7fe640 1 -- 172.21.15.173:0/2497814673 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1289 (secure 0 0 0) 0x7fce5c0139a0 con 0x7fce6419c100 2024-09-19T08:37:47.872 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:47 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:37:47.872 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:47 smithi173 ceph-mon[35306]: mon.a calling monitor election 2024-09-19T08:37:47.873 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:47 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:37:47.873 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:47 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:37:47.873 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:47 smithi173 ceph-mon[35306]: mon.c calling monitor election 2024-09-19T08:37:47.873 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:47 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:37:47.873 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:47 smithi173 ceph-mon[35306]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:47.873 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:47 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:37:47.873 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:47 smithi173 ceph-mon[35306]: mon.b calling monitor election 2024-09-19T08:37:47.874 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:47 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:37:47.874 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:47 smithi173 ceph-mon[35306]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:47.874 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:47 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:37:47.874 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:47 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:37:47.874 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:47 smithi173 ceph-mon[35306]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T08:37:47.874 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:47 smithi173 ceph-mon[35306]: monmap epoch 3 2024-09-19T08:37:47.874 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:47 smithi173 ceph-mon[35306]: fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 2024-09-19T08:37:47.874 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:47 smithi173 ceph-mon[35306]: last_changed 2024-09-19T08:37:42.489722+0000 2024-09-19T08:37:47.875 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:47 smithi173 ceph-mon[35306]: created 2024-09-19T08:36:06.626290+0000 2024-09-19T08:37:47.875 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:47 smithi173 ceph-mon[35306]: min_mon_release 19 (squid) 2024-09-19T08:37:47.875 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:47 smithi173 ceph-mon[35306]: election_strategy: 1 2024-09-19T08:37:47.875 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:47 smithi173 ceph-mon[35306]: 0: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.a 2024-09-19T08:37:47.875 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:47 smithi173 ceph-mon[35306]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.c 2024-09-19T08:37:47.875 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:47 smithi173 ceph-mon[35306]: 2: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.b 2024-09-19T08:37:47.875 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:47 smithi173 ceph-mon[35306]: fsmap 2024-09-19T08:37:47.875 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:47 smithi173 ceph-mon[35306]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T08:37:47.875 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:47 smithi173 ceph-mon[35306]: mgrmap e13: x(active, since 45s) 2024-09-19T08:37:47.876 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:47 smithi173 ceph-mon[35306]: overall HEALTH_OK 2024-09-19T08:37:47.876 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:47 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:47.876 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:47 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:47.876 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:47 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:47.876 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:47 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:47.876 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-19T08:37:47.877 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":3,"fsid":"1e4b0d74-7662-11ef-bceb-c7b262605968","modified":"2024-09-19T08:37:42.489722Z","created":"2024-09-19T08:36:06.626290Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:3300","nonce":0},{"type":"v1","addr":"172.21.15.164:6789","nonce":0}]},"addr":"172.21.15.164:6789/0","public_addr":"172.21.15.164:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:3300","nonce":0},{"type":"v1","addr":"172.21.15.173:6789","nonce":0}]},"addr":"172.21.15.173:6789/0","public_addr":"172.21.15.173:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:3300","nonce":0},{"type":"v1","addr":"172.21.15.170:6789","nonce":0}]},"addr":"172.21.15.170:6789/0","public_addr":"172.21.15.170:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-19T08:37:47.877 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 3 2024-09-19T08:37:47.877 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.874+0000 7fce6c75f640 1 -- 172.21.15.173:0/2497814673 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fce48039730 msgr2=0x7fce4803bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:47.877 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.874+0000 7fce6c75f640 1 --2- 172.21.15.173:0/2497814673 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fce48039730 0x7fce4803bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fce60005fd0 tx=0x7fce60005f60 comp rx=0 tx=0).stop 2024-09-19T08:37:47.877 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.874+0000 7fce6c75f640 1 -- 172.21.15.173:0/2497814673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce6419c100 msgr2=0x7fce6419c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:47.877 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.874+0000 7fce6c75f640 1 --2- 172.21.15.173:0/2497814673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce6419c100 0x7fce6419c5b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fce5c00bd50 tx=0x7fce5c00be40 comp rx=0 tx=0).stop 2024-09-19T08:37:47.878 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.874+0000 7fce6c75f640 1 -- 172.21.15.173:0/2497814673 shutdown_connections 2024-09-19T08:37:47.878 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.874+0000 7fce6c75f640 1 --2- 172.21.15.173:0/2497814673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce6419c100 0x7fce6419c5b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:47.878 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.874+0000 7fce6c75f640 1 --2- 172.21.15.173:0/2497814673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce64197830 0x7fce6419bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:47.878 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.874+0000 7fce6c75f640 1 --2- 172.21.15.173:0/2497814673 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fce48039730 0x7fce4803bbf0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:47.878 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.874+0000 7fce6c75f640 1 --2- 172.21.15.173:0/2497814673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce50003530 0x7fce641972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:47.878 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.874+0000 7fce6c75f640 1 -- 172.21.15.173:0/2497814673 >> 172.21.15.173:0/2497814673 conn(0x7fce640fc220 msgr2=0x7fce640fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:47.878 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.874+0000 7fce6c75f640 1 -- 172.21.15.173:0/2497814673 shutdown_connections 2024-09-19T08:37:47.878 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:47.874+0000 7fce6c75f640 1 -- 172.21.15.173:0/2497814673 wait complete. 2024-09-19T08:37:47.879 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:47 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:37:47.879 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:47 smithi170 ceph-mon[35072]: mon.a calling monitor election 2024-09-19T08:37:47.879 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:47 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:37:47.879 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:47 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:37:47.879 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:47 smithi170 ceph-mon[35072]: mon.c calling monitor election 2024-09-19T08:37:47.879 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:47 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:37:47.879 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:47 smithi170 ceph-mon[35072]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:47.880 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:47 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:37:47.880 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:47 smithi170 ceph-mon[35072]: mon.b calling monitor election 2024-09-19T08:37:47.880 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:47 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:37:47.880 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:47 smithi170 ceph-mon[35072]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:47.880 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:47 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:37:47.880 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:47 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:37:47.880 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:47 smithi170 ceph-mon[35072]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T08:37:47.880 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:47 smithi170 ceph-mon[35072]: monmap epoch 3 2024-09-19T08:37:47.880 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:47 smithi170 ceph-mon[35072]: fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 2024-09-19T08:37:47.880 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:47 smithi170 ceph-mon[35072]: last_changed 2024-09-19T08:37:42.489722+0000 2024-09-19T08:37:47.880 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:47 smithi170 ceph-mon[35072]: created 2024-09-19T08:36:06.626290+0000 2024-09-19T08:37:47.880 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:47 smithi170 ceph-mon[35072]: min_mon_release 19 (squid) 2024-09-19T08:37:47.881 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:47 smithi170 ceph-mon[35072]: election_strategy: 1 2024-09-19T08:37:47.881 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:47 smithi170 ceph-mon[35072]: 0: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.a 2024-09-19T08:37:47.881 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:47 smithi170 ceph-mon[35072]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.c 2024-09-19T08:37:47.881 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:47 smithi170 ceph-mon[35072]: 2: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.b 2024-09-19T08:37:47.881 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:47 smithi170 ceph-mon[35072]: fsmap 2024-09-19T08:37:47.881 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:47 smithi170 ceph-mon[35072]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T08:37:47.881 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:47 smithi170 ceph-mon[35072]: mgrmap e13: x(active, since 45s) 2024-09-19T08:37:47.881 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:47 smithi170 ceph-mon[35072]: overall HEALTH_OK 2024-09-19T08:37:47.881 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:47 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:47.881 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:47 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:47.881 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:47 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:47.881 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:47 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:48.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:47 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:37:48.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:47 smithi164 ceph-mon[30928]: mon.a calling monitor election 2024-09-19T08:37:48.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:47 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:37:48.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:47 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:37:48.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:47 smithi164 ceph-mon[30928]: mon.c calling monitor election 2024-09-19T08:37:48.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:47 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:37:48.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:47 smithi164 ceph-mon[30928]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:48.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:47 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:37:48.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:47 smithi164 ceph-mon[30928]: mon.b calling monitor election 2024-09-19T08:37:48.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:47 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:37:48.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:47 smithi164 ceph-mon[30928]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:48.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:47 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:37:48.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:47 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:37:48.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:47 smithi164 ceph-mon[30928]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T08:37:48.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:47 smithi164 ceph-mon[30928]: monmap epoch 3 2024-09-19T08:37:48.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:47 smithi164 ceph-mon[30928]: fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 2024-09-19T08:37:48.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:47 smithi164 ceph-mon[30928]: last_changed 2024-09-19T08:37:42.489722+0000 2024-09-19T08:37:48.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:47 smithi164 ceph-mon[30928]: created 2024-09-19T08:36:06.626290+0000 2024-09-19T08:37:48.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:47 smithi164 ceph-mon[30928]: min_mon_release 19 (squid) 2024-09-19T08:37:48.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:47 smithi164 ceph-mon[30928]: election_strategy: 1 2024-09-19T08:37:48.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:47 smithi164 ceph-mon[30928]: 0: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.a 2024-09-19T08:37:48.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:47 smithi164 ceph-mon[30928]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.c 2024-09-19T08:37:48.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:47 smithi164 ceph-mon[30928]: 2: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.b 2024-09-19T08:37:48.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:47 smithi164 ceph-mon[30928]: fsmap 2024-09-19T08:37:48.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:47 smithi164 ceph-mon[30928]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T08:37:48.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:47 smithi164 ceph-mon[30928]: mgrmap e13: x(active, since 45s) 2024-09-19T08:37:48.035 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:47 smithi164 ceph-mon[30928]: overall HEALTH_OK 2024-09-19T08:37:48.035 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:47 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:48.035 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:47 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:48.035 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:47 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:48.035 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:47 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:48.459 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-19T08:37:48.460 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-19T08:37:48.717 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:37:48.753 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:48 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:37:48.753 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:48 smithi173 ceph-mon[35306]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:48.753 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:48 smithi173 ceph-mon[35306]: from='client.? 172.21.15.173:0/2497814673' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T08:37:48.754 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:48 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:37:48.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:48 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:37:48.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:48 smithi164 ceph-mon[30928]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:48.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:48 smithi164 ceph-mon[30928]: from='client.? 172.21.15.173:0/2497814673' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T08:37:48.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:48 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:37:48.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:48 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:37:48.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:48 smithi170 ceph-mon[35072]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:48.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:48 smithi170 ceph-mon[35072]: from='client.? 172.21.15.173:0/2497814673' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T08:37:48.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:48 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:37:49.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.172+0000 7efe4aab2640 1 -- 172.21.15.164:0/2746826429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe44100990 msgr2=0x7efe44100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:49.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.172+0000 7efe4aab2640 1 --2- 172.21.15.164:0/2746826429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe44100990 0x7efe44100d70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efe38009880 tx=0x7efe3802a5d0 comp rx=0 tx=0).stop 2024-09-19T08:37:49.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.172+0000 7efe4aab2640 1 -- 172.21.15.164:0/2746826429 shutdown_connections 2024-09-19T08:37:49.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.172+0000 7efe4aab2640 1 --2- 172.21.15.164:0/2746826429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe44100990 0x7efe44100d70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:49.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.172+0000 7efe4aab2640 1 -- 172.21.15.164:0/2746826429 >> 172.21.15.164:0/2746826429 conn(0x7efe44076250 msgr2=0x7efe44076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:49.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.172+0000 7efe4aab2640 1 -- 172.21.15.164:0/2746826429 shutdown_connections 2024-09-19T08:37:49.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.172+0000 7efe4aab2640 1 -- 172.21.15.164:0/2746826429 wait complete. 2024-09-19T08:37:49.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.173+0000 7efe4aab2640 1 Processor -- start 2024-09-19T08:37:49.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.173+0000 7efe4aab2640 1 -- start start 2024-09-19T08:37:49.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.173+0000 7efe4aab2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe44100990 0x7efe44197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:49.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.173+0000 7efe4aab2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe44197850 0x7efe4419bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:49.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.173+0000 7efe4aab2640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe4419c120 0x7efe4419c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:49.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.173+0000 7efe4aab2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efe4410ab40 con 0x7efe44100990 2024-09-19T08:37:49.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.173+0000 7efe4aab2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7efe4410a9c0 con 0x7efe44197850 2024-09-19T08:37:49.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.173+0000 7efe4aab2640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efe4410acc0 con 0x7efe4419c120 2024-09-19T08:37:49.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.173+0000 7efe4884e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe44100990 0x7efe44197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:49.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.173+0000 7efe4884e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe44100990 0x7efe44197310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60184/0 (socket says 172.21.15.164:60184) 2024-09-19T08:37:49.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.173+0000 7efe4884e640 1 -- 172.21.15.164:0/2882619574 learned_addr learned my addr 172.21.15.164:0/2882619574 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:37:49.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.173+0000 7efe4904f640 1 --2- 172.21.15.164:0/2882619574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe4419c120 0x7efe4419c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:49.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.173+0000 7efe43fff640 1 --2- 172.21.15.164:0/2882619574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe44197850 0x7efe4419bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:49.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.174+0000 7efe4884e640 1 -- 172.21.15.164:0/2882619574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe4419c120 msgr2=0x7efe4419c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:49.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.174+0000 7efe4884e640 1 --2- 172.21.15.164:0/2882619574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe4419c120 0x7efe4419c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:49.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.174+0000 7efe4884e640 1 -- 172.21.15.164:0/2882619574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe44197850 msgr2=0x7efe4419bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:49.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.174+0000 7efe4884e640 1 --2- 172.21.15.164:0/2882619574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe44197850 0x7efe4419bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:49.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.174+0000 7efe4884e640 1 -- 172.21.15.164:0/2882619574 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe440fd710 con 0x7efe44100990 2024-09-19T08:37:49.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.174+0000 7efe4884e640 1 --2- 172.21.15.164:0/2882619574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe44100990 0x7efe44197310 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efe38031550 tx=0x7efe380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:49.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.174+0000 7efe41ffb640 1 -- 172.21.15.164:0/2882619574 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3803f030 con 0x7efe44100990 2024-09-19T08:37:49.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.174+0000 7efe41ffb640 1 -- 172.21.15.164:0/2882619574 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe38003ce0 con 0x7efe44100990 2024-09-19T08:37:49.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.174+0000 7efe4aab2640 1 -- 172.21.15.164:0/2882619574 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe440fd9a0 con 0x7efe44100990 2024-09-19T08:37:49.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.174+0000 7efe41ffb640 1 -- 172.21.15.164:0/2882619574 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe38035070 con 0x7efe44100990 2024-09-19T08:37:49.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.174+0000 7efe4aab2640 1 -- 172.21.15.164:0/2882619574 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe440fdfb0 con 0x7efe44100990 2024-09-19T08:37:49.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.175+0000 7efe4aab2640 1 -- 172.21.15.164:0/2882619574 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe44100df0 con 0x7efe44100990 2024-09-19T08:37:49.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.176+0000 7efe41ffb640 1 -- 172.21.15.164:0/2882619574 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7efe38042030 con 0x7efe44100990 2024-09-19T08:37:49.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.176+0000 7efe41ffb640 1 --2- 172.21.15.164:0/2882619574 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7efe34039730 0x7efe3403bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:49.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.176+0000 7efe41ffb640 1 -- 172.21.15.164:0/2882619574 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7efe3806cdb0 con 0x7efe44100990 2024-09-19T08:37:49.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.176+0000 7efe43fff640 1 --2- 172.21.15.164:0/2882619574 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7efe34039730 0x7efe3403bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:49.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.177+0000 7efe43fff640 1 --2- 172.21.15.164:0/2882619574 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7efe34039730 0x7efe3403bbf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efe30009a30 tx=0x7efe30009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:49.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.181+0000 7efe41ffb640 1 -- 172.21.15.164:0/2882619574 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efe3803bce0 con 0x7efe44100990 2024-09-19T08:37:49.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.364+0000 7efe4aab2640 1 -- 172.21.15.164:0/2882619574 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7efe440fe2d0 con 0x7efe44100990 2024-09-19T08:37:49.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.364+0000 7efe41ffb640 1 -- 172.21.15.164:0/2882619574 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7efe38044410 con 0x7efe44100990 2024-09-19T08:37:49.365 INFO:teuthology.orchestra.run.smithi164.stdout:# minimal ceph.conf for 1e4b0d74-7662-11ef-bceb-c7b262605968 2024-09-19T08:37:49.365 INFO:teuthology.orchestra.run.smithi164.stdout:[global] 2024-09-19T08:37:49.365 INFO:teuthology.orchestra.run.smithi164.stdout: fsid = 1e4b0d74-7662-11ef-bceb-c7b262605968 2024-09-19T08:37:49.365 INFO:teuthology.orchestra.run.smithi164.stdout: mon_host = [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-09-19T08:37:49.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.368+0000 7efe4aab2640 1 -- 172.21.15.164:0/2882619574 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7efe34039730 msgr2=0x7efe3403bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:49.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.368+0000 7efe4aab2640 1 --2- 172.21.15.164:0/2882619574 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7efe34039730 0x7efe3403bbf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efe30009a30 tx=0x7efe30009190 comp rx=0 tx=0).stop 2024-09-19T08:37:49.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.368+0000 7efe4aab2640 1 -- 172.21.15.164:0/2882619574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe44100990 msgr2=0x7efe44197310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:49.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.368+0000 7efe4aab2640 1 --2- 172.21.15.164:0/2882619574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe44100990 0x7efe44197310 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efe38031550 tx=0x7efe380042f0 comp rx=0 tx=0).stop 2024-09-19T08:37:49.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.368+0000 7efe4aab2640 1 -- 172.21.15.164:0/2882619574 shutdown_connections 2024-09-19T08:37:49.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.368+0000 7efe4aab2640 1 --2- 172.21.15.164:0/2882619574 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7efe34039730 0x7efe3403bbf0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:49.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.368+0000 7efe4aab2640 1 --2- 172.21.15.164:0/2882619574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe4419c120 0x7efe4419c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:49.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.368+0000 7efe4aab2640 1 --2- 172.21.15.164:0/2882619574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe44197850 0x7efe4419bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:49.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.368+0000 7efe4aab2640 1 --2- 172.21.15.164:0/2882619574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe44100990 0x7efe44197310 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:49.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.368+0000 7efe4aab2640 1 -- 172.21.15.164:0/2882619574 >> 172.21.15.164:0/2882619574 conn(0x7efe44076250 msgr2=0x7efe440696c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:49.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.368+0000 7efe4aab2640 1 -- 172.21.15.164:0/2882619574 shutdown_connections 2024-09-19T08:37:49.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:49.368+0000 7efe4aab2640 1 -- 172.21.15.164:0/2882619574 wait complete. 2024-09-19T08:37:50.002 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:49 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:50.002 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:49 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:50.002 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:49 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:50.002 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:49 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:50.002 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:49 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:50.002 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:49 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:37:50.002 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:49 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2882619574' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:50.003 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:49 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:50.003 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:49 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:50.003 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:49 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:50.003 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:49 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:50.003 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:49 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:50.003 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:49 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:50.003 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:49 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:50.033 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-19T08:37:50.033 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-19T08:37:50.033 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T08:37:50.062 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-19T08:37:50.062 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:37:50.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:49 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:50.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:49 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:50.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:49 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:50.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:49 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:50.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:49 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:50.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:49 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:37:50.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:49 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2882619574' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:50.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:49 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:50.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:49 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:50.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:49 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:50.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:49 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:50.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:49 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:50.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:49 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:50.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:49 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:50.127 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-19T08:37:50.127 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T08:37:50.155 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-19T08:37:50.155 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:37:50.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:49 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:50.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:49 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:50.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:49 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:50.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:49 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:50.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:49 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:50.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:49 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:37:50.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:49 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2882619574' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:50.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:49 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:50.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:49 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:50.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:49 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:50.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:49 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:50.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:49 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:50.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:49 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:50.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:49 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:50.220 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-19T08:37:50.220 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T08:37:50.247 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-19T08:37:50.247 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:37:50.312 INFO:tasks.cephadm:Adding mgr.x on smithi164 2024-09-19T08:37:50.312 INFO:tasks.cephadm:Adding mgr.y on smithi170 2024-09-19T08:37:50.312 INFO:tasks.cephadm:Adding mgr.z on smithi173 2024-09-19T08:37:50.313 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi164=x;smithi170=y;smithi173=z' 2024-09-19T08:37:50.588 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:37:51.016 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:50 smithi173 ceph-mon[35306]: Updating smithi164:/etc/ceph/ceph.conf 2024-09-19T08:37:51.016 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:50 smithi173 ceph-mon[35306]: Updating smithi170:/etc/ceph/ceph.conf 2024-09-19T08:37:51.016 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:50 smithi173 ceph-mon[35306]: Updating smithi173:/etc/ceph/ceph.conf 2024-09-19T08:37:51.016 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:50 smithi173 ceph-mon[35306]: Updating smithi173:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:37:51.016 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:50 smithi173 ceph-mon[35306]: Updating smithi170:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:37:51.016 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:50 smithi173 ceph-mon[35306]: Updating smithi164:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:37:51.016 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:50 smithi173 ceph-mon[35306]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:51.017 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:50 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:51.017 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:50 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:51.017 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:50 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:51.017 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:50 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:51.017 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:50 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:37:51.017 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:50 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T08:37:51.017 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:50 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:51.019 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.015+0000 7f5b0c5df640 1 -- 172.21.15.173:0/1667344303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b04102170 msgr2=0x7f5b04104560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:51.019 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.015+0000 7f5b0c5df640 1 --2- 172.21.15.173:0/1667344303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b04102170 0x7f5b04104560 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009880 tx=0x7f5b0002a5d0 comp rx=0 tx=0).stop 2024-09-19T08:37:51.019 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.015+0000 7f5b0c5df640 1 -- 172.21.15.173:0/1667344303 shutdown_connections 2024-09-19T08:37:51.020 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.015+0000 7f5b0c5df640 1 --2- 172.21.15.173:0/1667344303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b04102170 0x7f5b04104560 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:51.020 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.015+0000 7f5b0c5df640 1 -- 172.21.15.173:0/1667344303 >> 172.21.15.173:0/1667344303 conn(0x7f5b040fbdb0 msgr2=0x7f5b040fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:51.020 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.015+0000 7f5b0c5df640 1 -- 172.21.15.173:0/1667344303 shutdown_connections 2024-09-19T08:37:51.020 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.015+0000 7f5b0c5df640 1 -- 172.21.15.173:0/1667344303 wait complete. 2024-09-19T08:37:51.020 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.016+0000 7f5b0c5df640 1 Processor -- start 2024-09-19T08:37:51.020 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.016+0000 7f5b0c5df640 1 -- start start 2024-09-19T08:37:51.020 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.016+0000 7f5b0c5df640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b04102170 0x7f5b04196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:51.020 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.016+0000 7f5b0c5df640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b04197460 0x7f5b0419c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:51.020 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.017+0000 7f5b0c5df640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b041978e0 0x7f5b0419ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:51.020 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.017+0000 7f5b0c5df640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5b04107730 con 0x7f5b04197460 2024-09-19T08:37:51.020 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.017+0000 7f5b0c5df640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5b041075b0 con 0x7f5b041978e0 2024-09-19T08:37:51.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.017+0000 7f5b0c5df640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5b041078b0 con 0x7f5b04102170 2024-09-19T08:37:51.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.017+0000 7f5b0a37b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b04102170 0x7f5b04196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:51.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.017+0000 7f5b0a37b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b04102170 0x7f5b04196f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:50638/0 (socket says 172.21.15.173:50638) 2024-09-19T08:37:51.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.017+0000 7f5b0a37b640 1 -- 172.21.15.173:0/1270154070 learned_addr learned my addr 172.21.15.173:0/1270154070 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-09-19T08:37:51.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.017+0000 7f5b09b7a640 1 --2- 172.21.15.173:0/1270154070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b04197460 0x7f5b0419c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:51.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.017+0000 7f5b0ab7c640 1 --2- 172.21.15.173:0/1270154070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b041978e0 0x7f5b0419ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:51.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.017+0000 7f5b0a37b640 1 -- 172.21.15.173:0/1270154070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b041978e0 msgr2=0x7f5b0419ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:51.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.017+0000 7f5b0a37b640 1 --2- 172.21.15.173:0/1270154070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b041978e0 0x7f5b0419ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:51.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.017+0000 7f5b0a37b640 1 -- 172.21.15.173:0/1270154070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b04197460 msgr2=0x7f5b0419c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:51.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.017+0000 7f5b0a37b640 1 --2- 172.21.15.173:0/1270154070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b04197460 0x7f5b0419c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:51.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.017+0000 7f5b0a37b640 1 -- 172.21.15.173:0/1270154070 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b0419f320 con 0x7f5b04102170 2024-09-19T08:37:51.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.017+0000 7f5b0a37b640 1 --2- 172.21.15.173:0/1270154070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b04102170 0x7f5b04196f20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5b00000c00 tx=0x7f5b00004430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:51.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.017+0000 7f5afb7fe640 1 -- 172.21.15.173:0/1270154070 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0003f030 con 0x7f5b04102170 2024-09-19T08:37:51.022 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.017+0000 7f5afb7fe640 1 -- 172.21.15.173:0/1270154070 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b00003ce0 con 0x7f5b04102170 2024-09-19T08:37:51.022 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.018+0000 7f5afb7fe640 1 -- 172.21.15.173:0/1270154070 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00035070 con 0x7f5b04102170 2024-09-19T08:37:51.022 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.018+0000 7f5b0c5df640 1 -- 172.21.15.173:0/1270154070 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b0419f5b0 con 0x7f5b04102170 2024-09-19T08:37:51.022 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.018+0000 7f5afb7fe640 1 -- 172.21.15.173:0/1270154070 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f5b00042030 con 0x7f5b04102170 2024-09-19T08:37:51.022 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.018+0000 7f5b0c5df640 1 -- 172.21.15.173:0/1270154070 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b0419fb70 con 0x7f5b04102170 2024-09-19T08:37:51.022 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.019+0000 7f5afb7fe640 1 --2- 172.21.15.173:0/1270154070 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f5af0039730 0x7f5af003bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:51.022 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.019+0000 7f5afb7fe640 1 -- 172.21.15.173:0/1270154070 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5b00071930 con 0x7f5b04102170 2024-09-19T08:37:51.022 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.019+0000 7f5b09b7a640 1 --2- 172.21.15.173:0/1270154070 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f5af0039730 0x7f5af003bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:51.022 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.019+0000 7f5b09b7a640 1 --2- 172.21.15.173:0/1270154070 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f5af0039730 0x7f5af003bbf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5b04198440 tx=0x7f5af4009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:51.022 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.019+0000 7f5b0c5df640 1 -- 172.21.15.173:0/1270154070 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b041025d0 con 0x7f5b04102170 2024-09-19T08:37:51.024 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.024+0000 7f5afb7fe640 1 -- 172.21.15.173:0/1270154070 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5b00043c60 con 0x7f5b04102170 2024-09-19T08:37:51.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:50 smithi164 ceph-mon[30928]: Updating smithi164:/etc/ceph/ceph.conf 2024-09-19T08:37:51.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:50 smithi164 ceph-mon[30928]: Updating smithi170:/etc/ceph/ceph.conf 2024-09-19T08:37:51.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:50 smithi164 ceph-mon[30928]: Updating smithi173:/etc/ceph/ceph.conf 2024-09-19T08:37:51.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:50 smithi164 ceph-mon[30928]: Updating smithi173:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:37:51.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:50 smithi164 ceph-mon[30928]: Updating smithi170:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:37:51.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:50 smithi164 ceph-mon[30928]: Updating smithi164:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:37:51.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:50 smithi164 ceph-mon[30928]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:51.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:50 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:51.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:50 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:51.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:50 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:51.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:50 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:51.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:50 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:37:51.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:50 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T08:37:51.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:50 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:51.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:50 smithi170 ceph-mon[35072]: Updating smithi164:/etc/ceph/ceph.conf 2024-09-19T08:37:51.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:50 smithi170 ceph-mon[35072]: Updating smithi170:/etc/ceph/ceph.conf 2024-09-19T08:37:51.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:50 smithi170 ceph-mon[35072]: Updating smithi173:/etc/ceph/ceph.conf 2024-09-19T08:37:51.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:50 smithi170 ceph-mon[35072]: Updating smithi173:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:37:51.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:50 smithi170 ceph-mon[35072]: Updating smithi170:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:37:51.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:50 smithi170 ceph-mon[35072]: Updating smithi164:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:37:51.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:50 smithi170 ceph-mon[35072]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:51.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:50 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:51.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:50 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:51.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:50 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:51.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:50 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:51.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:50 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:37:51.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:50 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T08:37:51.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:50 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:51.202 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.202+0000 7f5b0c5df640 1 -- 172.21.15.173:0/1270154070 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi164=x;smithi170=y;smithi173=z", "target": ["mon-mgr", ""]}) -- 0x7f5b040626c0 con 0x7f5af0039730 2024-09-19T08:37:51.206 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.207+0000 7f5afb7fe640 1 -- 172.21.15.173:0/1270154070 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5b040626c0 con 0x7f5af0039730 2024-09-19T08:37:51.207 INFO:teuthology.orchestra.run.smithi173.stdout:Scheduled mgr update... 2024-09-19T08:37:51.209 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.209+0000 7f5b0c5df640 1 -- 172.21.15.173:0/1270154070 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f5af0039730 msgr2=0x7f5af003bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:51.209 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.209+0000 7f5b0c5df640 1 --2- 172.21.15.173:0/1270154070 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f5af0039730 0x7f5af003bbf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5b04198440 tx=0x7f5af4009190 comp rx=0 tx=0).stop 2024-09-19T08:37:51.209 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.209+0000 7f5b0c5df640 1 -- 172.21.15.173:0/1270154070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b04102170 msgr2=0x7f5b04196f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:51.209 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.209+0000 7f5b0c5df640 1 --2- 172.21.15.173:0/1270154070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b04102170 0x7f5b04196f20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5b00000c00 tx=0x7f5b00004430 comp rx=0 tx=0).stop 2024-09-19T08:37:51.209 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.209+0000 7f5b0c5df640 1 -- 172.21.15.173:0/1270154070 shutdown_connections 2024-09-19T08:37:51.209 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.209+0000 7f5b0c5df640 1 --2- 172.21.15.173:0/1270154070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b041978e0 0x7f5b0419ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:51.209 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.209+0000 7f5b0c5df640 1 --2- 172.21.15.173:0/1270154070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b04197460 0x7f5b0419c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:51.209 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.209+0000 7f5b0c5df640 1 --2- 172.21.15.173:0/1270154070 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f5af0039730 0x7f5af003bbf0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:51.209 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.209+0000 7f5b0c5df640 1 --2- 172.21.15.173:0/1270154070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b04102170 0x7f5b04196f20 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:51.209 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.209+0000 7f5b0c5df640 1 -- 172.21.15.173:0/1270154070 >> 172.21.15.173:0/1270154070 conn(0x7f5b040fbdb0 msgr2=0x7f5b040fc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:51.210 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.209+0000 7f5b0c5df640 1 -- 172.21.15.173:0/1270154070 shutdown_connections 2024-09-19T08:37:51.210 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:37:51.209+0000 7f5b0c5df640 1 -- 172.21.15.173:0/1270154070 wait complete. 2024-09-19T08:37:51.775 DEBUG:teuthology.orchestra.run.smithi170:mgr.y> sudo journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mgr.y.service 2024-09-19T08:37:51.777 DEBUG:teuthology.orchestra.run.smithi173:mgr.z> sudo journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mgr.z.service 2024-09-19T08:37:51.780 INFO:tasks.cephadm:Deploying OSDs... 2024-09-19T08:37:51.781 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-19T08:37:51.781 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T08:37:51.801 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T08:37:51.801 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_1 2024-09-19T08:37:51.859 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T08:37:51.859 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:37:51.859 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-19T08:37:51.860 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:37:51.860 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:37:51.860 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-09-19 08:37:11.925380455 +0000 2024-09-19T08:37:51.860 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-09-19 08:33:02.164223935 +0000 2024-09-19T08:37:51.860 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-09-19 08:33:02.164223935 +0000 2024-09-19T08:37:51.860 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-09-19 08:33:02.164223935 +0000 2024-09-19T08:37:51.860 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T08:37:51.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:51 smithi170 ceph-mon[35072]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T08:37:51.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:51 smithi170 ceph-mon[35072]: Reconfiguring daemon mon.a on smithi164 2024-09-19T08:37:51.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:51 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:51.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:51 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:51.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:51 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:37:51.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:51 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T08:37:51.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:51 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:51.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:51 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:51.928 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-09-19T08:37:51.928 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-09-19T08:37:51.928 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000233539 s, 2.2 MB/s 2024-09-19T08:37:51.929 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T08:37:51.988 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_2 2024-09-19T08:37:52.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:51 smithi164 ceph-mon[30928]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T08:37:52.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:51 smithi164 ceph-mon[30928]: Reconfiguring daemon mon.a on smithi164 2024-09-19T08:37:52.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:51 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:52.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:51 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:52.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:51 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:37:52.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:51 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T08:37:52.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:51 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:52.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:51 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:52.046 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T08:37:52.046 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:37:52.046 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-19T08:37:52.046 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:37:52.046 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:37:52.046 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-09-19 08:37:11.958379910 +0000 2024-09-19T08:37:52.046 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-09-19 08:33:02.505218349 +0000 2024-09-19T08:37:52.046 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-09-19 08:33:02.505218349 +0000 2024-09-19T08:37:52.046 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-09-19 08:33:02.505218349 +0000 2024-09-19T08:37:52.047 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T08:37:52.112 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-09-19T08:37:52.113 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-09-19T08:37:52.113 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000210569 s, 2.4 MB/s 2024-09-19T08:37:52.114 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T08:37:52.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:51 smithi173 ceph-mon[35306]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T08:37:52.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:51 smithi173 ceph-mon[35306]: Reconfiguring daemon mon.a on smithi164 2024-09-19T08:37:52.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:51 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:52.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:51 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:52.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:51 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:37:52.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:51 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T08:37:52.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:51 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:52.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:51 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:52.172 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_3 2024-09-19T08:37:52.229 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T08:37:52.230 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:37:52.230 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-19T08:37:52.230 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:37:52.230 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:37:52.230 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-09-19 08:37:11.991379365 +0000 2024-09-19T08:37:52.230 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-09-19 08:33:02.869212386 +0000 2024-09-19T08:37:52.230 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-09-19 08:33:02.869212386 +0000 2024-09-19T08:37:52.230 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-09-19 08:33:02.869212386 +0000 2024-09-19T08:37:52.230 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T08:37:52.295 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-09-19T08:37:52.295 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-09-19T08:37:52.296 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000210727 s, 2.4 MB/s 2024-09-19T08:37:52.297 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T08:37:52.355 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_4 2024-09-19T08:37:52.412 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T08:37:52.412 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:37:52.412 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-19T08:37:52.412 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:37:52.412 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:37:52.412 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-09-19 08:37:12.028378754 +0000 2024-09-19T08:37:52.412 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-09-19 08:33:03.220206637 +0000 2024-09-19T08:37:52.412 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-09-19 08:33:03.220206637 +0000 2024-09-19T08:37:52.412 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-09-19 08:33:03.220206637 +0000 2024-09-19T08:37:52.412 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T08:37:52.478 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-09-19T08:37:52.478 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-09-19T08:37:52.478 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000185071 s, 2.8 MB/s 2024-09-19T08:37:52.479 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T08:37:52.536 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-19T08:37:52.537 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T08:37:52.553 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T08:37:52.553 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_1 2024-09-19T08:37:52.609 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T08:37:52.609 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:37:52.609 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-19T08:37:52.610 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:37:52.610 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:37:52.610 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-09-19 08:37:41.945333265 +0000 2024-09-19T08:37:52.610 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-09-19 08:33:20.071985528 +0000 2024-09-19T08:37:52.610 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-09-19 08:33:20.071985528 +0000 2024-09-19T08:37:52.610 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-09-19 08:33:20.071985528 +0000 2024-09-19T08:37:52.610 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T08:37:52.674 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-09-19T08:37:52.674 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-09-19T08:37:52.674 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000114251 s, 4.5 MB/s 2024-09-19T08:37:52.675 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T08:37:52.733 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_2 2024-09-19T08:37:52.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:52 smithi164 ceph-mon[30928]: Reconfiguring mon.b (monmap changed)... 2024-09-19T08:37:52.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:52 smithi164 ceph-mon[30928]: Reconfiguring daemon mon.b on smithi170 2024-09-19T08:37:52.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:52 smithi164 ceph-mon[30928]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi164=x;smithi170=y;smithi173=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:37:52.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:52 smithi164 ceph-mon[30928]: Saving service mgr spec with placement smithi164=x;smithi170=y;smithi173=z;count:3 2024-09-19T08:37:52.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:52 smithi164 ceph-mon[30928]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:52.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:52 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:52.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:52 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:52.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:52 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:52.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:52 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:37:52.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:52 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T08:37:52.783 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:52 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:52.788 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T08:37:52.788 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:37:52.788 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-19T08:37:52.789 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:37:52.789 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:37:52.789 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-09-19 08:37:41.945333265 +0000 2024-09-19T08:37:52.789 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-09-19 08:33:20.393977972 +0000 2024-09-19T08:37:52.789 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-09-19 08:33:20.393977972 +0000 2024-09-19T08:37:52.789 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-09-19 08:33:20.393977972 +0000 2024-09-19T08:37:52.789 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T08:37:52.852 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-09-19T08:37:52.852 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-09-19T08:37:52.852 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000115546 s, 4.4 MB/s 2024-09-19T08:37:52.854 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T08:37:52.911 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_3 2024-09-19T08:37:52.966 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T08:37:52.966 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:37:52.966 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-19T08:37:52.966 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:37:52.966 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:37:52.966 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-09-19 08:37:41.946333242 +0000 2024-09-19T08:37:52.967 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-09-19 08:33:20.700970769 +0000 2024-09-19T08:37:52.967 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-09-19 08:33:20.700970769 +0000 2024-09-19T08:37:52.967 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-09-19 08:33:20.700970769 +0000 2024-09-19T08:37:52.967 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T08:37:53.028 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:52 smithi170 ceph-mon[35072]: Reconfiguring mon.b (monmap changed)... 2024-09-19T08:37:53.028 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:52 smithi170 ceph-mon[35072]: Reconfiguring daemon mon.b on smithi170 2024-09-19T08:37:53.028 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:52 smithi170 ceph-mon[35072]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi164=x;smithi170=y;smithi173=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:37:53.028 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:52 smithi170 ceph-mon[35072]: Saving service mgr spec with placement smithi164=x;smithi170=y;smithi173=z;count:3 2024-09-19T08:37:53.028 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:52 smithi170 ceph-mon[35072]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:53.028 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:52 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:53.028 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:52 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:53.028 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:52 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:53.028 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:52 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:37:53.028 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:52 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T08:37:53.029 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:52 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:53.030 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-09-19T08:37:53.030 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-09-19T08:37:53.030 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000104208 s, 4.9 MB/s 2024-09-19T08:37:53.031 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T08:37:53.032 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:52 smithi173 ceph-mon[35306]: Reconfiguring mon.b (monmap changed)... 2024-09-19T08:37:53.032 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:52 smithi173 ceph-mon[35306]: Reconfiguring daemon mon.b on smithi170 2024-09-19T08:37:53.032 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:52 smithi173 ceph-mon[35306]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi164=x;smithi170=y;smithi173=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:37:53.032 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:52 smithi173 ceph-mon[35306]: Saving service mgr spec with placement smithi164=x;smithi170=y;smithi173=z;count:3 2024-09-19T08:37:53.032 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:52 smithi173 ceph-mon[35306]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:53.032 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:52 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:53.032 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:52 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:53.032 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:52 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:53.032 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:52 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T08:37:53.032 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:52 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T08:37:53.032 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:52 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:53.088 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_4 2024-09-19T08:37:53.142 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T08:37:53.143 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:37:53.143 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-19T08:37:53.143 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:37:53.143 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:37:53.143 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-09-19 08:37:41.946333242 +0000 2024-09-19T08:37:53.143 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-09-19 08:33:21.020963260 +0000 2024-09-19T08:37:53.143 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-09-19 08:33:21.020963260 +0000 2024-09-19T08:37:53.143 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-09-19 08:33:21.020963260 +0000 2024-09-19T08:37:53.143 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T08:37:53.205 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-09-19T08:37:53.206 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-09-19T08:37:53.206 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000131091 s, 3.9 MB/s 2024-09-19T08:37:53.207 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T08:37:53.264 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-19T08:37:53.264 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T08:37:53.281 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T08:37:53.281 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_1 2024-09-19T08:37:53.338 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T08:37:53.338 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:37:53.338 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-19T08:37:53.338 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:37:53.338 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:37:53.338 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-09-19 08:37:33.139730812 +0000 2024-09-19T08:37:53.338 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-09-19 08:33:14.386436408 +0000 2024-09-19T08:37:53.339 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-09-19 08:33:14.386436408 +0000 2024-09-19T08:37:53.339 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-09-19 08:33:14.386436408 +0000 2024-09-19T08:37:53.339 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T08:37:53.405 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-09-19T08:37:53.405 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-09-19T08:37:53.406 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000257161 s, 2.0 MB/s 2024-09-19T08:37:53.407 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T08:37:53.465 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_2 2024-09-19T08:37:53.521 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T08:37:53.522 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:37:53.522 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-19T08:37:53.522 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:37:53.522 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:37:53.522 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-09-19 08:37:33.139730812 +0000 2024-09-19T08:37:53.522 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-09-19 08:33:14.692429074 +0000 2024-09-19T08:37:53.522 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-09-19 08:33:14.692429074 +0000 2024-09-19T08:37:53.522 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-09-19 08:33:14.692429074 +0000 2024-09-19T08:37:53.523 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T08:37:53.585 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-09-19T08:37:53.586 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-09-19T08:37:53.586 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000169593 s, 3.0 MB/s 2024-09-19T08:37:53.587 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T08:37:53.645 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_3 2024-09-19T08:37:53.701 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T08:37:53.701 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:37:53.701 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-09-19T08:37:53.701 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:37:53.701 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:37:53.701 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-09-19 08:37:33.139730812 +0000 2024-09-19T08:37:53.701 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-09-19 08:33:15.027421044 +0000 2024-09-19T08:37:53.701 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-09-19 08:33:15.027421044 +0000 2024-09-19T08:37:53.701 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-09-19 08:33:15.027421044 +0000 2024-09-19T08:37:53.701 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T08:37:53.764 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-09-19T08:37:53.765 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-09-19T08:37:53.765 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000200947 s, 2.5 MB/s 2024-09-19T08:37:53.766 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T08:37:53.823 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_4 2024-09-19T08:37:53.880 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T08:37:53.881 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:37:53.881 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-09-19T08:37:53.881 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:37:53.881 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T08:37:53.881 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-09-19 08:37:33.140730789 +0000 2024-09-19T08:37:53.881 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-09-19 08:33:15.344413446 +0000 2024-09-19T08:37:53.881 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-09-19 08:33:15.344413446 +0000 2024-09-19T08:37:53.882 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-09-19 08:33:15.344413446 +0000 2024-09-19T08:37:53.882 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T08:37:53.945 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-09-19T08:37:53.945 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-09-19T08:37:53.945 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000178117 s, 2.9 MB/s 2024-09-19T08:37:53.947 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T08:37:54.002 INFO:tasks.cephadm:Deploying osd.0 on smithi164 with /dev/vg_nvme/lv_4... 2024-09-19T08:37:54.003 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T08:37:54.030 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:53 smithi164 ceph-mon[30928]: Reconfiguring mon.c (monmap changed)... 2024-09-19T08:37:54.030 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:53 smithi164 ceph-mon[30928]: Reconfiguring daemon mon.c on smithi173 2024-09-19T08:37:54.030 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:53 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:54.030 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:53 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:54.030 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:53 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:37:54.030 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:53 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:54.030 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:53 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:37:54.030 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:53 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:54.030 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:53 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T08:37:54.030 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:53 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T08:37:54.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:53 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T08:37:54.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:53 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:54.071 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:53 smithi173 ceph-mon[35306]: Reconfiguring mon.c (monmap changed)... 2024-09-19T08:37:54.071 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:53 smithi173 ceph-mon[35306]: Reconfiguring daemon mon.c on smithi173 2024-09-19T08:37:54.071 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:53 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:54.072 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:53 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:54.072 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:53 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:37:54.072 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:53 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:54.072 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:53 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:37:54.072 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:53 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:54.072 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:53 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T08:37:54.072 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:53 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T08:37:54.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:53 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T08:37:54.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:53 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:54.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:53 smithi170 ceph-mon[35072]: Reconfiguring mon.c (monmap changed)... 2024-09-19T08:37:54.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:53 smithi170 ceph-mon[35072]: Reconfiguring daemon mon.c on smithi173 2024-09-19T08:37:54.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:53 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:54.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:53 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:54.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:53 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:37:54.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:53 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:54.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:53 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:37:54.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:53 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:54.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:53 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T08:37:54.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:53 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T08:37:54.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:53 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T08:37:54.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:53 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:54.251 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:37:54.999 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:54 smithi173 ceph-mon[35306]: Deploying daemon mgr.z on smithi173 2024-09-19T08:37:55.000 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:54 smithi173 ceph-mon[35306]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:55.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:54 smithi164 ceph-mon[30928]: Deploying daemon mgr.z on smithi173 2024-09-19T08:37:55.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:54 smithi164 ceph-mon[30928]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:55.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:54 smithi170 ceph-mon[35072]: Deploying daemon mgr.z on smithi173 2024-09-19T08:37:55.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:54 smithi170 ceph-mon[35072]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:55.661 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:37:55 smithi173 podman[37095]: 2024-09-19 08:37:55.464297563 +0000 UTC m=+0.015344834 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:37:55.914 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:37:55 smithi173 podman[37095]: 2024-09-19 08:37:55.789301885 +0000 UTC m=+0.340349158 container init a9e0ddc90b8e785ffaed7cb2c29960990abe6aace148f2b497292ba5996d239e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4) 2024-09-19T08:37:55.914 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:37:55 smithi173 podman[37095]: 2024-09-19 08:37:55.79286117 +0000 UTC m=+0.343908437 container start a9e0ddc90b8e785ffaed7cb2c29960990abe6aace148f2b497292ba5996d239e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True) 2024-09-19T08:37:55.914 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:37:55 smithi173 ceph-mgr[37113]: -- 172.21.15.173:0/2989566420 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55df6d91f0e0 con 0x55df6d8fd400 2024-09-19T08:37:55.914 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:37:55 smithi173 bash[37095]: a9e0ddc90b8e785ffaed7cb2c29960990abe6aace148f2b497292ba5996d239e 2024-09-19T08:37:55.914 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:37:55 smithi173 systemd[1]: Started Ceph mgr.z for 1e4b0d74-7662-11ef-bceb-c7b262605968. 2024-09-19T08:37:56.183 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:37:55 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:37:55.913+0000 7f599b5c2040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T08:37:56.183 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:37:55 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:37:55.973+0000 7f599b5c2040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T08:37:56.183 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:37:56 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:37:56.118+0000 7f599b5c2040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T08:37:56.184 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:37:56 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:37:56.183+0000 7f599b5c2040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T08:37:56.465 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:37:56 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:37:56.328+0000 7f599b5c2040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:37:56.465 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:37:56 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:37:56.465+0000 7f599b5c2040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T08:37:56.598 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:37:56.623 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi164:vg_nvme/lv_4 2024-09-19T08:37:56.906 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:37:57.019 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:37:56 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:37:56.776+0000 7f599b5c2040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T08:37:57.019 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:37:56 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:37:56.836+0000 7f599b5c2040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T08:37:57.019 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:56 smithi173 ceph-mon[35306]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:57.019 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:56 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:57.019 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:56 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:57.019 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:56 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:57.019 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:56 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T08:37:57.019 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:56 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T08:37:57.020 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:56 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T08:37:57.020 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:56 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:57.027 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:56 smithi164 ceph-mon[30928]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:57.027 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:56 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:57.027 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:56 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:57.027 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:56 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:57.027 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:56 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T08:37:57.027 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:56 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T08:37:57.027 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:56 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T08:37:57.027 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:56 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:57.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:56 smithi170 ceph-mon[35072]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:57.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:56 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:57.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:56 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:57.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:56 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:57.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:56 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T08:37:57.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:56 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T08:37:57.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:56 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T08:37:57.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:56 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:57.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.390+0000 7f5d415c1640 1 -- 172.21.15.164:0/2184034747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d3c100b10 msgr2=0x7f5d3c100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:57.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.390+0000 7f5d415c1640 1 --2- 172.21.15.164:0/2184034747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d3c100b10 0x7f5d3c100f10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c009900 tx=0x7f5d2c02a410 comp rx=0 tx=0).stop 2024-09-19T08:37:57.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.390+0000 7f5d415c1640 1 -- 172.21.15.164:0/2184034747 shutdown_connections 2024-09-19T08:37:57.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.390+0000 7f5d415c1640 1 --2- 172.21.15.164:0/2184034747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d3c1026d0 0x7f5d3c108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.390+0000 7f5d415c1640 1 --2- 172.21.15.164:0/2184034747 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d3c101d10 0x7f5d3c102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.390+0000 7f5d415c1640 1 --2- 172.21.15.164:0/2184034747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d3c100b10 0x7f5d3c100f10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.390+0000 7f5d415c1640 1 -- 172.21.15.164:0/2184034747 >> 172.21.15.164:0/2184034747 conn(0x7f5d3c0fc2a0 msgr2=0x7f5d3c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:57.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.390+0000 7f5d415c1640 1 -- 172.21.15.164:0/2184034747 shutdown_connections 2024-09-19T08:37:57.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.390+0000 7f5d415c1640 1 -- 172.21.15.164:0/2184034747 wait complete. 2024-09-19T08:37:57.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.390+0000 7f5d415c1640 1 Processor -- start 2024-09-19T08:37:57.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.391+0000 7f5d415c1640 1 -- start start 2024-09-19T08:37:57.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.391+0000 7f5d415c1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d3c100b10 0x7f5d3c06b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:57.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.391+0000 7f5d415c1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d3c101d10 0x7f5d3c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:57.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.391+0000 7f5d415c1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d3c1026d0 0x7f5d3c19f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:57.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.391+0000 7f5d415c1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c10c790 con 0x7f5d3c101d10 2024-09-19T08:37:57.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.391+0000 7f5d415c1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c10c610 con 0x7f5d3c100b10 2024-09-19T08:37:57.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.391+0000 7f5d415c1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c10c910 con 0x7f5d3c1026d0 2024-09-19T08:37:57.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.391+0000 7f5d3a7fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d3c101d10 0x7f5d3c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:57.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.391+0000 7f5d3a7fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d3c101d10 0x7f5d3c19ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:55576/0 (socket says 172.21.15.164:55576) 2024-09-19T08:37:57.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.391+0000 7f5d3a7fc640 1 -- 172.21.15.164:0/2535903824 learned_addr learned my addr 172.21.15.164:0/2535903824 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:37:57.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.391+0000 7f5d3affd640 1 --2- 172.21.15.164:0/2535903824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d3c100b10 0x7f5d3c06b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:57.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.391+0000 7f5d3b7fe640 1 --2- 172.21.15.164:0/2535903824 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d3c1026d0 0x7f5d3c19f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:57.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.391+0000 7f5d3a7fc640 1 -- 172.21.15.164:0/2535903824 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d3c1026d0 msgr2=0x7f5d3c19f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:57.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.391+0000 7f5d3a7fc640 1 --2- 172.21.15.164:0/2535903824 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d3c1026d0 0x7f5d3c19f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.391+0000 7f5d3a7fc640 1 -- 172.21.15.164:0/2535903824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d3c100b10 msgr2=0x7f5d3c06b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:57.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.391+0000 7f5d3a7fc640 1 --2- 172.21.15.164:0/2535903824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d3c100b10 0x7f5d3c06b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.391+0000 7f5d3a7fc640 1 -- 172.21.15.164:0/2535903824 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3c19fbe0 con 0x7f5d3c101d10 2024-09-19T08:37:57.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.391+0000 7f5d3a7fc640 1 --2- 172.21.15.164:0/2535903824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d3c101d10 0x7f5d3c19ccb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000d770 tx=0x7f5d3000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:57.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.391+0000 7f5d23fff640 1 -- 172.21.15.164:0/2535903824 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d30017070 con 0x7f5d3c101d10 2024-09-19T08:37:57.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.392+0000 7f5d415c1640 1 -- 172.21.15.164:0/2535903824 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d3c19fe70 con 0x7f5d3c101d10 2024-09-19T08:37:57.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.392+0000 7f5d23fff640 1 -- 172.21.15.164:0/2535903824 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d30005500 con 0x7f5d3c101d10 2024-09-19T08:37:57.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.392+0000 7f5d23fff640 1 -- 172.21.15.164:0/2535903824 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d30004e00 con 0x7f5d3c101d10 2024-09-19T08:37:57.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.392+0000 7f5d415c1640 1 -- 172.21.15.164:0/2535903824 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d3c1a02d0 con 0x7f5d3c101d10 2024-09-19T08:37:57.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.392+0000 7f5d23fff640 1 -- 172.21.15.164:0/2535903824 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f5d30010460 con 0x7f5d3c101d10 2024-09-19T08:37:57.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.393+0000 7f5d415c1640 1 -- 172.21.15.164:0/2535903824 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d3c107f80 con 0x7f5d3c101d10 2024-09-19T08:37:57.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.393+0000 7f5d23fff640 1 --2- 172.21.15.164:0/2535903824 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f5d240394d0 0x7f5d2403b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:57.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.393+0000 7f5d23fff640 1 -- 172.21.15.164:0/2535903824 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5d3004c120 con 0x7f5d3c101d10 2024-09-19T08:37:57.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.393+0000 7f5d3affd640 1 --2- 172.21.15.164:0/2535903824 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f5d240394d0 0x7f5d2403b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:57.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.393+0000 7f5d3affd640 1 --2- 172.21.15.164:0/2535903824 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f5d240394d0 0x7f5d2403b990 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c005ab0 tx=0x7f5d2c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:57.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.397+0000 7f5d23fff640 1 -- 172.21.15.164:0/2535903824 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5d3001a360 con 0x7f5d3c101d10 2024-09-19T08:37:57.410 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:37:57 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:37:57.019+0000 7f599b5c2040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T08:37:57.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:37:57.574+0000 7f5d415c1640 1 -- 172.21.15.164:0/2535903824 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f5d3c062630 con 0x7f5d240394d0 2024-09-19T08:37:57.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:57 smithi170 ceph-mon[35072]: Deploying daemon mgr.y on smithi170 2024-09-19T08:37:57.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:57 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:37:57.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:57 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:37:57.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:57 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:57.876 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:37:57 smithi170 systemd[1]: Starting Ceph mgr.y for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T08:37:57.974 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:57 smithi164 ceph-mon[30928]: Deploying daemon mgr.y on smithi170 2024-09-19T08:37:57.974 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:57 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:37:57.974 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:57 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:37:57.974 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:57 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:58.160 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:37:57 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:37:57.895+0000 7f599b5c2040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T08:37:58.161 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:37:57 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:37:57.962+0000 7f599b5c2040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T08:37:58.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:57 smithi173 ceph-mon[35306]: Deploying daemon mgr.y on smithi170 2024-09-19T08:37:58.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:57 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:37:58.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:57 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:37:58.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:57 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:37:58.210 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:37:57 smithi170 podman[36423]: 2024-09-19 08:37:57.862634785 +0000 UTC m=+0.016069310 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:37:58.211 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:37:57 smithi170 podman[36423]: 2024-09-19 08:37:57.975896054 +0000 UTC m=+0.129330569 container create 5b8dfecde34d1590f2aa36d0bdf618296d34299caea028440196db3201f5d674 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-19T08:37:58.211 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:37:58 smithi170 podman[36423]: 2024-09-19 08:37:58.211492407 +0000 UTC m=+0.364926922 container init 5b8dfecde34d1590f2aa36d0bdf618296d34299caea028440196db3201f5d674 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0) 2024-09-19T08:37:58.561 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:37:58 smithi170 podman[36423]: 2024-09-19 08:37:58.214434181 +0000 UTC m=+0.367868714 container start 5b8dfecde34d1590f2aa36d0bdf618296d34299caea028440196db3201f5d674 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-19T08:37:58.561 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:37:58 smithi170 ceph-mgr[36442]: -- 172.21.15.170:0/2429703444 <== mon.2 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558a404030e0 con 0x558a403e0800 2024-09-19T08:37:58.561 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:37:58 smithi170 bash[36423]: 5b8dfecde34d1590f2aa36d0bdf618296d34299caea028440196db3201f5d674 2024-09-19T08:37:58.561 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:37:58 smithi170 systemd[1]: Started Ceph mgr.y for 1e4b0d74-7662-11ef-bceb-c7b262605968. 2024-09-19T08:37:58.561 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:37:58 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:37:58.341+0000 7f0b98e5c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T08:37:58.561 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:37:58 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:37:58.401+0000 7f0b98e5c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T08:37:58.561 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:37:58 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:37:58.559+0000 7f0b98e5c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T08:37:58.840 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:58 smithi164 ceph-mon[30928]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:37:58.840 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:58 smithi164 ceph-mon[30928]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:58.840 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:58 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:58.840 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:58 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:58.840 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:58 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:58.840 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:58 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:58.840 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:37:58 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:37:58.875 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:37:58 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:37:58.628+0000 7f0b98e5c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T08:37:58.875 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:37:58 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:37:58.789+0000 7f0b98e5c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:37:58.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:58 smithi170 ceph-mon[35072]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:37:58.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:58 smithi170 ceph-mon[35072]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:58.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:58 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:58.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:58 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:58.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:58 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:58.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:58 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:58.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:37:58 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:37:58.984 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:37:58 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:37:58.692+0000 7f599b5c2040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T08:37:58.984 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:37:58 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:37:58.825+0000 7f599b5c2040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T08:37:58.984 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:37:58 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:37:58.984+0000 7f599b5c2040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:37:58.985 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:58 smithi173 ceph-mon[35306]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:37:58.985 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:58 smithi173 ceph-mon[35306]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:37:58.985 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:58 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:58.985 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:58 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:58.985 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:58 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:58.985 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:58 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:37:58.985 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:37:58 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:37:59.212 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:37:58 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:37:58.947+0000 7f0b98e5c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T08:37:59.277 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:37:59 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:37:59.050+0000 7f599b5c2040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T08:37:59.277 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:37:59 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:37:59.206+0000 7f599b5c2040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T08:37:59.277 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:37:59 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:37:59.276+0000 7f599b5c2040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T08:37:59.528 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:37:59 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:37:59.272+0000 7f0b98e5c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T08:37:59.529 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:37:59 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:37:59.335+0000 7f0b98e5c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T08:37:59.571 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:37:59 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:37:59.336+0000 7f599b5c2040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T08:37:59.571 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:37:59 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:37:59.502+0000 7f599b5c2040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T08:37:59.571 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:37:59 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:37:59.571+0000 7f599b5c2040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T08:37:59.843 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:37:59 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:37:59.639+0000 7f599b5c2040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T08:37:59.843 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:37:59 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:37:59.724+0000 7f599b5c2040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T08:37:59.844 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:37:59 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:37:59.783+0000 7f599b5c2040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T08:37:59.844 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:37:59 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:37:59.843+0000 7f599b5c2040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T08:37:59.874 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:37:59 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:37:59.527+0000 7f0b98e5c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T08:38:00.748 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:00 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:00.748 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:00 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:00.748 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:00 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:00.748 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:00 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:00.748 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:00 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:00.748 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:00 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:38:00.748 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:00 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:00.749 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:00 smithi164 ceph-mon[30928]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T08:38:00.749 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:00 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T08:38:00.749 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:00 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T08:38:00.749 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:00 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:00.749 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:00 smithi164 ceph-mon[30928]: Reconfiguring daemon mgr.x on smithi164 2024-09-19T08:38:00.749 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:00 smithi164 ceph-mon[30928]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:00.749 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:00 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2811138352' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33b11f37-6ac5-41fc-8cfd-1b08c7eef688"}]: dispatch 2024-09-19T08:38:00.749 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:00 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2811138352' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33b11f37-6ac5-41fc-8cfd-1b08c7eef688"}]': finished 2024-09-19T08:38:00.749 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:00 smithi164 ceph-mon[30928]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T08:38:00.749 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:00 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:38:00.749 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:00 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/492653701' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:38:00.875 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:38:00 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:38:00.414+0000 7f0b98e5c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T08:38:00.875 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:38:00 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:38:00.481+0000 7f0b98e5c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T08:38:00.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:00 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:00.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:00 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:00.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:00 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:00.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:00 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:00.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:00 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:00.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:00 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:38:00.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:00 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:00.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:00 smithi170 ceph-mon[35072]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T08:38:00.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:00 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T08:38:00.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:00 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T08:38:00.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:00 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:00.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:00 smithi170 ceph-mon[35072]: Reconfiguring daemon mgr.x on smithi164 2024-09-19T08:38:00.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:00 smithi170 ceph-mon[35072]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:00.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:00 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2811138352' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33b11f37-6ac5-41fc-8cfd-1b08c7eef688"}]: dispatch 2024-09-19T08:38:00.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:00 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2811138352' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33b11f37-6ac5-41fc-8cfd-1b08c7eef688"}]': finished 2024-09-19T08:38:00.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:00 smithi170 ceph-mon[35072]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T08:38:00.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:00 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:38:00.877 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:00 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/492653701' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:38:00.910 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:38:00 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:38:00.629+0000 7f599b5c2040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T08:38:00.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:00 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:00.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:00 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:00.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:00 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:00.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:00 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:00.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:00 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:00.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:00 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:38:00.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:00 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:00.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:00 smithi173 ceph-mon[35306]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T08:38:00.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:00 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T08:38:00.912 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:00 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T08:38:00.912 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:00 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:00.912 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:00 smithi173 ceph-mon[35306]: Reconfiguring daemon mgr.x on smithi164 2024-09-19T08:38:00.912 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:00 smithi173 ceph-mon[35306]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:00.912 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:00 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2811138352' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33b11f37-6ac5-41fc-8cfd-1b08c7eef688"}]: dispatch 2024-09-19T08:38:00.912 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:00 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2811138352' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33b11f37-6ac5-41fc-8cfd-1b08c7eef688"}]': finished 2024-09-19T08:38:00.912 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:00 smithi173 ceph-mon[35306]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T08:38:00.912 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:00 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:38:00.912 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:00 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/492653701' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:38:01.448 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:38:01 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:38:01.193+0000 7f0b98e5c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T08:38:01.448 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:38:01 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:38:01.307+0000 7f0b98e5c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T08:38:01.448 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:38:01 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:38:01.446+0000 7f0b98e5c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:38:01.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:38:01.551+0000 7f5d23fff640 1 -- 172.21.15.164:0/2535903824 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mgrmap(e 14) ==== 92409+0+0 (secure 0 0 0) 0x7f5d30010750 con 0x7f5d3c101d10 2024-09-19T08:38:01.726 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:01 smithi170 ceph-mon[35072]: Standby manager daemon z started 2024-09-19T08:38:01.726 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:01 smithi170 ceph-mon[35072]: from='mgr.? 172.21.15.173:0/1494419880' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T08:38:01.726 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:01 smithi170 ceph-mon[35072]: from='mgr.? 172.21.15.173:0/1494419880' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T08:38:01.726 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:01 smithi170 ceph-mon[35072]: from='mgr.? 172.21.15.173:0/1494419880' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T08:38:01.726 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:01 smithi170 ceph-mon[35072]: from='mgr.? 172.21.15.173:0/1494419880' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T08:38:01.727 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:38:01 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:38:01.507+0000 7f0b98e5c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T08:38:01.727 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:38:01 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:38:01.657+0000 7f0b98e5c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T08:38:01.727 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:38:01 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:38:01.725+0000 7f0b98e5c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T08:38:01.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:01 smithi173 ceph-mon[35306]: Standby manager daemon z started 2024-09-19T08:38:01.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:01 smithi173 ceph-mon[35306]: from='mgr.? 172.21.15.173:0/1494419880' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T08:38:01.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:01 smithi173 ceph-mon[35306]: from='mgr.? 172.21.15.173:0/1494419880' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T08:38:01.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:01 smithi173 ceph-mon[35306]: from='mgr.? 172.21.15.173:0/1494419880' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T08:38:01.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:01 smithi173 ceph-mon[35306]: from='mgr.? 172.21.15.173:0/1494419880' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T08:38:02.005 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:38:01 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:38:01.785+0000 7f0b98e5c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T08:38:02.005 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:38:01 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:38:01.940+0000 7f0b98e5c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T08:38:02.005 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:38:02 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:38:02.004+0000 7f0b98e5c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T08:38:02.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:01 smithi164 ceph-mon[30928]: Standby manager daemon z started 2024-09-19T08:38:02.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:01 smithi164 ceph-mon[30928]: from='mgr.? 172.21.15.173:0/1494419880' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T08:38:02.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:01 smithi164 ceph-mon[30928]: from='mgr.? 172.21.15.173:0/1494419880' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T08:38:02.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:01 smithi164 ceph-mon[30928]: from='mgr.? 172.21.15.173:0/1494419880' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T08:38:02.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:01 smithi164 ceph-mon[30928]: from='mgr.? 172.21.15.173:0/1494419880' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T08:38:02.269 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:38:02 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:38:02.070+0000 7f0b98e5c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T08:38:02.269 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:38:02 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:38:02.151+0000 7f0b98e5c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T08:38:02.269 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:38:02 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:38:02.209+0000 7f0b98e5c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T08:38:02.269 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:38:02 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:38:02.268+0000 7f0b98e5c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T08:38:02.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:02 smithi170 ceph-mon[35072]: mgrmap e14: x(active, since 59s), standbys: z 2024-09-19T08:38:02.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:02 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T08:38:02.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:02 smithi170 ceph-mon[35072]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:02.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:02 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:02.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:02 smithi173 ceph-mon[35306]: mgrmap e14: x(active, since 59s), standbys: z 2024-09-19T08:38:02.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:02 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T08:38:02.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:02 smithi173 ceph-mon[35306]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:02.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:02 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:03.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:02 smithi164 ceph-mon[30928]: mgrmap e14: x(active, since 59s), standbys: z 2024-09-19T08:38:03.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:02 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T08:38:03.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:02 smithi164 ceph-mon[30928]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:03.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:02 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:03.375 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:38:03 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:38:03.051+0000 7f0b98e5c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T08:38:03.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:38:03.561+0000 7f5d23fff640 1 -- 172.21.15.164:0/2535903824 <== mon.0 v2:172.21.15.164:3300/0 8 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7f5d30010b70 con 0x7f5d3c101d10 2024-09-19T08:38:03.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:03 smithi170 ceph-mon[35072]: Standby manager daemon y started 2024-09-19T08:38:03.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:03 smithi170 ceph-mon[35072]: from='mgr.? 172.21.15.170:0/1286782178' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T08:38:03.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:03 smithi170 ceph-mon[35072]: from='mgr.? 172.21.15.170:0/1286782178' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T08:38:03.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:03 smithi170 ceph-mon[35072]: from='mgr.? 172.21.15.170:0/1286782178' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T08:38:03.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:03 smithi170 ceph-mon[35072]: from='mgr.? 172.21.15.170:0/1286782178' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T08:38:03.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:03 smithi173 ceph-mon[35306]: Standby manager daemon y started 2024-09-19T08:38:03.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:03 smithi173 ceph-mon[35306]: from='mgr.? 172.21.15.170:0/1286782178' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T08:38:03.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:03 smithi173 ceph-mon[35306]: from='mgr.? 172.21.15.170:0/1286782178' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T08:38:03.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:03 smithi173 ceph-mon[35306]: from='mgr.? 172.21.15.170:0/1286782178' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T08:38:03.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:03 smithi173 ceph-mon[35306]: from='mgr.? 172.21.15.170:0/1286782178' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T08:38:04.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:03 smithi164 ceph-mon[30928]: Standby manager daemon y started 2024-09-19T08:38:04.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:03 smithi164 ceph-mon[30928]: from='mgr.? 172.21.15.170:0/1286782178' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T08:38:04.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:03 smithi164 ceph-mon[30928]: from='mgr.? 172.21.15.170:0/1286782178' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T08:38:04.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:03 smithi164 ceph-mon[30928]: from='mgr.? 172.21.15.170:0/1286782178' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T08:38:04.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:03 smithi164 ceph-mon[30928]: from='mgr.? 172.21.15.170:0/1286782178' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T08:38:04.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:04 smithi170 ceph-mon[35072]: mgrmap e15: x(active, since 61s), standbys: z, y 2024-09-19T08:38:04.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:04 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T08:38:04.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:04 smithi170 ceph-mon[35072]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:04.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:04 smithi173 ceph-mon[35306]: mgrmap e15: x(active, since 61s), standbys: z, y 2024-09-19T08:38:04.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:04 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T08:38:04.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:04 smithi173 ceph-mon[35306]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:04.968 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:04 smithi164 ceph-mon[30928]: mgrmap e15: x(active, since 61s), standbys: z, y 2024-09-19T08:38:04.968 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:04 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T08:38:04.968 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:04 smithi164 ceph-mon[30928]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:07.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:06 smithi164 ceph-mon[30928]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:07.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:06 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:07.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:06 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:07.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:06 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:38:07.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:06 smithi170 ceph-mon[35072]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:07.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:06 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:07.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:06 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:07.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:06 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:38:07.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:06 smithi173 ceph-mon[35306]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:07.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:06 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:07.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:06 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:07.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:06 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:38:07.949 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:07 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:07.950 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:07 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:38:07.950 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:07 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:08.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:07 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:08.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:07 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:38:08.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:07 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:08.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:07 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:08.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:07 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:38:08.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:07 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:09.026 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:08 smithi170 ceph-mon[35072]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:09.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:08 smithi164 ceph-mon[30928]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:09.062 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:08 smithi173 ceph-mon[35306]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:10.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:09 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T08:38:10.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:09 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:10.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:09 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T08:38:10.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:09 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:10.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:09 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T08:38:10.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:09 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:11.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:10 smithi164 ceph-mon[30928]: Deploying daemon osd.0 on smithi164 2024-09-19T08:38:11.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:10 smithi164 ceph-mon[30928]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:11.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:10 smithi170 ceph-mon[35072]: Deploying daemon osd.0 on smithi164 2024-09-19T08:38:11.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:10 smithi170 ceph-mon[35072]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:11.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:10 smithi173 ceph-mon[35306]: Deploying daemon osd.0 on smithi164 2024-09-19T08:38:11.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:10 smithi173 ceph-mon[35306]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:13.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:12 smithi164 ceph-mon[30928]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:13.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:12 smithi170 ceph-mon[35072]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:13.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:12 smithi173 ceph-mon[35306]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:14.982 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:14 smithi164 ceph-mon[30928]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:15.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:14 smithi170 ceph-mon[35072]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:15.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:14 smithi173 ceph-mon[35306]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:16.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:15 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:38:16.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:15 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:16.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:15 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:16.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:15 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:38:16.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:15 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:16.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:15 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:16.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:15 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:38:16.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:15 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:16.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:15 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:16.960 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:16 smithi164 ceph-mon[30928]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:17.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:16 smithi170 ceph-mon[35072]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:17.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:16 smithi173 ceph-mon[35306]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:18.371 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 0 on host 'smithi164' 2024-09-19T08:38:18.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:38:18.371+0000 7f5d23fff640 1 -- 172.21.15.164:0/2535903824 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5d3c062630 con 0x7f5d240394d0 2024-09-19T08:38:18.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:38:18.374+0000 7f5d415c1640 1 -- 172.21.15.164:0/2535903824 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f5d240394d0 msgr2=0x7f5d2403b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:18.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:38:18.374+0000 7f5d415c1640 1 --2- 172.21.15.164:0/2535903824 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f5d240394d0 0x7f5d2403b990 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c005ab0 tx=0x7f5d2c000bc0 comp rx=0 tx=0).stop 2024-09-19T08:38:18.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:38:18.374+0000 7f5d415c1640 1 -- 172.21.15.164:0/2535903824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d3c101d10 msgr2=0x7f5d3c19ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:18.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:38:18.374+0000 7f5d415c1640 1 --2- 172.21.15.164:0/2535903824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d3c101d10 0x7f5d3c19ccb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000d770 tx=0x7f5d3000dc30 comp rx=0 tx=0).stop 2024-09-19T08:38:18.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:38:18.374+0000 7f5d415c1640 1 -- 172.21.15.164:0/2535903824 shutdown_connections 2024-09-19T08:38:18.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:38:18.374+0000 7f5d415c1640 1 --2- 172.21.15.164:0/2535903824 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f5d240394d0 0x7f5d2403b990 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:18.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:38:18.374+0000 7f5d415c1640 1 --2- 172.21.15.164:0/2535903824 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d3c1026d0 0x7f5d3c19f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:18.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:38:18.374+0000 7f5d415c1640 1 --2- 172.21.15.164:0/2535903824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d3c101d10 0x7f5d3c19ccb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:18.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:38:18.374+0000 7f5d415c1640 1 --2- 172.21.15.164:0/2535903824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d3c100b10 0x7f5d3c06b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:18.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:38:18.374+0000 7f5d415c1640 1 -- 172.21.15.164:0/2535903824 >> 172.21.15.164:0/2535903824 conn(0x7f5d3c0fc2a0 msgr2=0x7f5d3c105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:18.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:38:18.374+0000 7f5d415c1640 1 -- 172.21.15.164:0/2535903824 shutdown_connections 2024-09-19T08:38:18.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:38:18.374+0000 7f5d415c1640 1 -- 172.21.15.164:0/2535903824 wait complete. 2024-09-19T08:38:18.668 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:18 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:18.668 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:18 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:18.668 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:18 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:18.668 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:18 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:38:18.669 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:18 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:18.669 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:18 smithi164 ceph-mon[30928]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:18.669 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:18 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:38:18.669 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:18 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:18.669 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:18 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:18.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:18 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:18.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:18 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:18.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:18 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:18.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:18 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:38:18.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:18 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:18.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:18 smithi170 ceph-mon[35072]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:18.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:18 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:38:18.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:18 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:18.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:18 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:18.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:18 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:18.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:18 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:18.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:18 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:18.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:18 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:38:18.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:18 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:18.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:18 smithi173 ceph-mon[35306]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:18.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:18 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:38:18.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:18 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:18.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:18 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:18.978 DEBUG:teuthology.orchestra.run.smithi164:osd.0> sudo journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.0.service 2024-09-19T08:38:18.981 INFO:tasks.cephadm:Deploying osd.1 on smithi170 with /dev/vg_nvme/lv_4... 2024-09-19T08:38:18.981 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T08:38:19.234 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T08:38:20.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:20 smithi170 ceph-mon[35072]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:21.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:20 smithi164 ceph-mon[30928]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:21.032 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 19 08:38:20 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-0[44856]: 2024-09-19T08:38:20.827+0000 7f401bfdd640 -1 osd.0 0 log_to_monitors true 2024-09-19T08:38:21.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:20 smithi173 ceph-mon[35306]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:21.669 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-19T08:38:21.695 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi170:vg_nvme/lv_4 2024-09-19T08:38:21.943 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T08:38:22.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:21 smithi164 ceph-mon[30928]: from='osd.0 [v2:172.21.15.164:6802/663038243,v1:172.21.15.164:6803/663038243]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T08:38:22.051 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:21 smithi170 ceph-mon[35072]: from='osd.0 [v2:172.21.15.164:6802/663038243,v1:172.21.15.164:6803/663038243]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T08:38:22.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:21 smithi173 ceph-mon[35306]: from='osd.0 [v2:172.21.15.164:6802/663038243,v1:172.21.15.164:6803/663038243]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T08:38:22.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.379+0000 7f3935c84640 1 -- 172.21.15.170:0/1992444489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3930102eb0 msgr2=0x7f3930103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:22.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.379+0000 7f3935c84640 1 --2- 172.21.15.170:0/1992444489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3930102eb0 0x7f3930103330 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3924009900 tx=0x7f392402a410 comp rx=0 tx=0).stop 2024-09-19T08:38:22.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.380+0000 7f3935c84640 1 -- 172.21.15.170:0/1992444489 shutdown_connections 2024-09-19T08:38:22.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.380+0000 7f3935c84640 1 --2- 172.21.15.170:0/1992444489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3930103870 0x7f393010a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:22.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.380+0000 7f3935c84640 1 --2- 172.21.15.170:0/1992444489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3930102eb0 0x7f3930103330 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:22.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.380+0000 7f3935c84640 1 --2- 172.21.15.170:0/1992444489 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39300ffed0 0x7f39301002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:22.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.380+0000 7f3935c84640 1 -- 172.21.15.170:0/1992444489 >> 172.21.15.170:0/1992444489 conn(0x7f39300fba40 msgr2=0x7f39300fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:22.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.380+0000 7f3935c84640 1 -- 172.21.15.170:0/1992444489 shutdown_connections 2024-09-19T08:38:22.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.380+0000 7f3935c84640 1 -- 172.21.15.170:0/1992444489 wait complete. 2024-09-19T08:38:22.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.380+0000 7f3935c84640 1 Processor -- start 2024-09-19T08:38:22.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.380+0000 7f3935c84640 1 -- start start 2024-09-19T08:38:22.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.381+0000 7f3935c84640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39300ffed0 0x7f3930197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:22.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.381+0000 7f3935c84640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3930102eb0 0x7f39301984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:22.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.381+0000 7f3935c84640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3930103870 0x7f393019f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:22.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.381+0000 7f3935c84640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f393010d890 con 0x7f3930102eb0 2024-09-19T08:38:22.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.381+0000 7f3935c84640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f393010d710 con 0x7f3930103870 2024-09-19T08:38:22.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.381+0000 7f3935c84640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f393010da10 con 0x7f39300ffed0 2024-09-19T08:38:22.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.381+0000 7f392ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3930103870 0x7f393019f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:22.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.381+0000 7f392f7fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39300ffed0 0x7f3930197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:22.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.381+0000 7f392ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3930103870 0x7f393019f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39250/0 (socket says 172.21.15.170:39250) 2024-09-19T08:38:22.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.381+0000 7f392ffff640 1 -- 172.21.15.170:0/3768629360 learned_addr learned my addr 172.21.15.170:0/3768629360 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-19T08:38:22.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.381+0000 7f392effd640 1 --2- 172.21.15.170:0/3768629360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3930102eb0 0x7f39301984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:22.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.381+0000 7f392ffff640 1 -- 172.21.15.170:0/3768629360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39300ffed0 msgr2=0x7f3930197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:22.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.381+0000 7f392ffff640 1 --2- 172.21.15.170:0/3768629360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39300ffed0 0x7f3930197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:22.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.381+0000 7f392ffff640 1 -- 172.21.15.170:0/3768629360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3930102eb0 msgr2=0x7f39301984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:22.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.381+0000 7f392ffff640 1 --2- 172.21.15.170:0/3768629360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3930102eb0 0x7f39301984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:22.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.381+0000 7f392ffff640 1 -- 172.21.15.170:0/3768629360 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f393019fbb0 con 0x7f3930103870 2024-09-19T08:38:22.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.381+0000 7f392ffff640 1 --2- 172.21.15.170:0/3768629360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3930103870 0x7f393019f540 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f392800eaa0 tx=0x7f392800ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:22.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.381+0000 7f392cff9640 1 -- 172.21.15.170:0/3768629360 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392800cdb0 con 0x7f3930103870 2024-09-19T08:38:22.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.381+0000 7f392cff9640 1 -- 172.21.15.170:0/3768629360 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3928005020 con 0x7f3930103870 2024-09-19T08:38:22.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.382+0000 7f3935c84640 1 -- 172.21.15.170:0/3768629360 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f393019fea0 con 0x7f3930103870 2024-09-19T08:38:22.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.382+0000 7f392cff9640 1 -- 172.21.15.170:0/3768629360 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39280116b0 con 0x7f3930103870 2024-09-19T08:38:22.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.382+0000 7f3935c84640 1 -- 172.21.15.170:0/3768629360 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39301a0300 con 0x7f3930103870 2024-09-19T08:38:22.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.382+0000 7f3935c84640 1 -- 172.21.15.170:0/3768629360 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39301091a0 con 0x7f3930103870 2024-09-19T08:38:22.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.383+0000 7f392cff9640 1 -- 172.21.15.170:0/3768629360 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7f3928005560 con 0x7f3930103870 2024-09-19T08:38:22.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.384+0000 7f392cff9640 1 --2- 172.21.15.170:0/3768629360 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f39180a31e0 0x7f39180a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:22.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.384+0000 7f392cff9640 1 -- 172.21.15.170:0/3768629360 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1361+0+0 (secure 0 0 0) 0x7f39280cde40 con 0x7f3930103870 2024-09-19T08:38:22.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.384+0000 7f392f7fe640 1 --2- 172.21.15.170:0/3768629360 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f39180a31e0 0x7f39180a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:22.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.385+0000 7f392f7fe640 1 --2- 172.21.15.170:0/3768629360 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f39180a31e0 0x7f39180a56a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f393006a590 tx=0x7f392000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:22.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.389+0000 7f392cff9640 1 -- 172.21.15.170:0/3768629360 <== mon.2 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3928016030 con 0x7f3930103870 2024-09-19T08:38:22.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:22.562+0000 7f3935c84640 1 -- 172.21.15.170:0/3768629360 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f3930062630 con 0x7f39180a31e0 2024-09-19T08:38:23.013 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:22 smithi170 ceph-mon[35072]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:23.013 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:22 smithi170 ceph-mon[35072]: Detected new or changed devices on smithi164 2024-09-19T08:38:23.013 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:22 smithi170 ceph-mon[35072]: from='osd.0 [v2:172.21.15.164:6802/663038243,v1:172.21.15.164:6803/663038243]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T08:38:23.013 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:22 smithi170 ceph-mon[35072]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T08:38:23.013 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:22 smithi170 ceph-mon[35072]: from='osd.0 [v2:172.21.15.164:6802/663038243,v1:172.21.15.164:6803/663038243]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-19T08:38:23.013 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:22 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:38:23.013 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:22 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:23.013 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:22 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:23.013 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:22 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:38:23.014 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:22 smithi170 ceph-mon[35072]: Adjusting osd_memory_target on smithi164 to 17088M 2024-09-19T08:38:23.014 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:22 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:23.014 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:22 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:23.014 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:22 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:38:23.014 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:22 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:23.014 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:22 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:38:23.014 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:22 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:38:23.014 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:22 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:23.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:22 smithi164 ceph-mon[30928]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:23.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:22 smithi164 ceph-mon[30928]: Detected new or changed devices on smithi164 2024-09-19T08:38:23.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:22 smithi164 ceph-mon[30928]: from='osd.0 [v2:172.21.15.164:6802/663038243,v1:172.21.15.164:6803/663038243]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T08:38:23.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:22 smithi164 ceph-mon[30928]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T08:38:23.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:22 smithi164 ceph-mon[30928]: from='osd.0 [v2:172.21.15.164:6802/663038243,v1:172.21.15.164:6803/663038243]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-19T08:38:23.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:22 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:38:23.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:22 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:23.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:22 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:23.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:22 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:38:23.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:22 smithi164 ceph-mon[30928]: Adjusting osd_memory_target on smithi164 to 17088M 2024-09-19T08:38:23.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:22 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:23.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:22 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:23.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:22 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:38:23.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:22 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:23.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:22 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:38:23.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:22 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:38:23.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:22 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:23.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:22 smithi173 ceph-mon[35306]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:23.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:22 smithi173 ceph-mon[35306]: Detected new or changed devices on smithi164 2024-09-19T08:38:23.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:22 smithi173 ceph-mon[35306]: from='osd.0 [v2:172.21.15.164:6802/663038243,v1:172.21.15.164:6803/663038243]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T08:38:23.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:22 smithi173 ceph-mon[35306]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T08:38:23.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:22 smithi173 ceph-mon[35306]: from='osd.0 [v2:172.21.15.164:6802/663038243,v1:172.21.15.164:6803/663038243]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-19T08:38:23.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:22 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:38:23.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:22 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:23.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:22 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:23.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:22 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:38:23.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:22 smithi173 ceph-mon[35306]: Adjusting osd_memory_target on smithi164 to 17088M 2024-09-19T08:38:23.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:22 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:23.163 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:22 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:23.163 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:22 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:38:23.163 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:22 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:23.163 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:22 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:38:23.163 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:22 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:38:23.163 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:22 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:23.532 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 19 08:38:23 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-0[44856]: 2024-09-19T08:38:23.143+0000 7f4017f60640 -1 osd.0 0 waiting for initial osdmap 2024-09-19T08:38:23.532 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 19 08:38:23 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-0[44856]: 2024-09-19T08:38:23.146+0000 7f400fd80640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:38:24.027 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:23 smithi170 ceph-mon[35072]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:38:24.027 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:23 smithi170 ceph-mon[35072]: from='osd.0 [v2:172.21.15.164:6802/663038243,v1:172.21.15.164:6803/663038243]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-19T08:38:24.027 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:23 smithi170 ceph-mon[35072]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T08:38:24.027 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:23 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:38:24.027 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:23 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:38:24.027 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:23 smithi170 ceph-mon[35072]: from='osd.0 [v2:172.21.15.164:6802/663038243,v1:172.21.15.164:6803/663038243]' entity='osd.0' 2024-09-19T08:38:24.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:23 smithi164 ceph-mon[30928]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:38:24.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:23 smithi164 ceph-mon[30928]: from='osd.0 [v2:172.21.15.164:6802/663038243,v1:172.21.15.164:6803/663038243]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-19T08:38:24.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:23 smithi164 ceph-mon[30928]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T08:38:24.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:23 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:38:24.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:23 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:38:24.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:23 smithi164 ceph-mon[30928]: from='osd.0 [v2:172.21.15.164:6802/663038243,v1:172.21.15.164:6803/663038243]' entity='osd.0' 2024-09-19T08:38:24.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:23 smithi173 ceph-mon[35306]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:38:24.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:23 smithi173 ceph-mon[35306]: from='osd.0 [v2:172.21.15.164:6802/663038243,v1:172.21.15.164:6803/663038243]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-19T08:38:24.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:23 smithi173 ceph-mon[35306]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T08:38:24.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:23 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:38:24.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:23 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:38:24.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:23 smithi173 ceph-mon[35306]: from='osd.0 [v2:172.21.15.164:6802/663038243,v1:172.21.15.164:6803/663038243]' entity='osd.0' 2024-09-19T08:38:25.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:24 smithi164 ceph-mon[30928]: purged_snaps scrub starts 2024-09-19T08:38:25.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:24 smithi164 ceph-mon[30928]: purged_snaps scrub ok 2024-09-19T08:38:25.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:24 smithi164 ceph-mon[30928]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:25.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:24 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:38:25.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:24 smithi164 ceph-mon[30928]: osd.0 [v2:172.21.15.164:6802/663038243,v1:172.21.15.164:6803/663038243] boot 2024-09-19T08:38:25.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:24 smithi164 ceph-mon[30928]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T08:38:25.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:24 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:38:25.038 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:24 smithi170 ceph-mon[35072]: purged_snaps scrub starts 2024-09-19T08:38:25.038 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:24 smithi170 ceph-mon[35072]: purged_snaps scrub ok 2024-09-19T08:38:25.038 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:24 smithi170 ceph-mon[35072]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:25.038 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:24 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:38:25.038 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:24 smithi170 ceph-mon[35072]: osd.0 [v2:172.21.15.164:6802/663038243,v1:172.21.15.164:6803/663038243] boot 2024-09-19T08:38:25.038 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:24 smithi170 ceph-mon[35072]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T08:38:25.038 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:24 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:38:25.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:24 smithi173 ceph-mon[35306]: purged_snaps scrub starts 2024-09-19T08:38:25.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:24 smithi173 ceph-mon[35306]: purged_snaps scrub ok 2024-09-19T08:38:25.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:24 smithi173 ceph-mon[35306]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T08:38:25.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:24 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:38:25.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:24 smithi173 ceph-mon[35306]: osd.0 [v2:172.21.15.164:6802/663038243,v1:172.21.15.164:6803/663038243] boot 2024-09-19T08:38:25.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:24 smithi173 ceph-mon[35306]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T08:38:25.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:24 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:38:26.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:25 smithi164 ceph-mon[30928]: from='client.? 172.21.15.170:0/4176384588' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6de64b0a-46e2-4a98-8005-a326a64bc87d"}]: dispatch 2024-09-19T08:38:26.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:25 smithi164 ceph-mon[30928]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6de64b0a-46e2-4a98-8005-a326a64bc87d"}]: dispatch 2024-09-19T08:38:26.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:25 smithi164 ceph-mon[30928]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6de64b0a-46e2-4a98-8005-a326a64bc87d"}]': finished 2024-09-19T08:38:26.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:25 smithi164 ceph-mon[30928]: osdmap e9: 2 total, 1 up, 2 in 2024-09-19T08:38:26.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:25 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:38:26.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:25 smithi164 ceph-mon[30928]: from='client.? 172.21.15.170:0/1493553953' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:38:26.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:25 smithi170 ceph-mon[35072]: from='client.? 172.21.15.170:0/4176384588' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6de64b0a-46e2-4a98-8005-a326a64bc87d"}]: dispatch 2024-09-19T08:38:26.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:25 smithi170 ceph-mon[35072]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6de64b0a-46e2-4a98-8005-a326a64bc87d"}]: dispatch 2024-09-19T08:38:26.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:25 smithi170 ceph-mon[35072]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6de64b0a-46e2-4a98-8005-a326a64bc87d"}]': finished 2024-09-19T08:38:26.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:25 smithi170 ceph-mon[35072]: osdmap e9: 2 total, 1 up, 2 in 2024-09-19T08:38:26.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:25 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:38:26.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:25 smithi170 ceph-mon[35072]: from='client.? 172.21.15.170:0/1493553953' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:38:26.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:25 smithi173 ceph-mon[35306]: from='client.? 172.21.15.170:0/4176384588' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6de64b0a-46e2-4a98-8005-a326a64bc87d"}]: dispatch 2024-09-19T08:38:26.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:25 smithi173 ceph-mon[35306]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6de64b0a-46e2-4a98-8005-a326a64bc87d"}]: dispatch 2024-09-19T08:38:26.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:25 smithi173 ceph-mon[35306]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6de64b0a-46e2-4a98-8005-a326a64bc87d"}]': finished 2024-09-19T08:38:26.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:25 smithi173 ceph-mon[35306]: osdmap e9: 2 total, 1 up, 2 in 2024-09-19T08:38:26.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:25 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:38:26.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:25 smithi173 ceph-mon[35306]: from='client.? 172.21.15.170:0/1493553953' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:38:27.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:26 smithi164 ceph-mon[30928]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:27.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:26 smithi164 ceph-mon[30928]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T08:38:27.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:26 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:38:27.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:26 smithi170 ceph-mon[35072]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:27.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:26 smithi170 ceph-mon[35072]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T08:38:27.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:26 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:38:27.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:26 smithi173 ceph-mon[35306]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:27.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:26 smithi173 ceph-mon[35306]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T08:38:27.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:26 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:38:29.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:28 smithi164 ceph-mon[30928]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:29.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:28 smithi170 ceph-mon[35072]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:29.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:28 smithi173 ceph-mon[35306]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:30.945 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:30 smithi170 ceph-mon[35072]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:31.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:30 smithi164 ceph-mon[30928]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:31.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:30 smithi173 ceph-mon[35306]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:33.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:32 smithi164 ceph-mon[30928]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:33.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:32 smithi170 ceph-mon[35072]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:33.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:32 smithi173 ceph-mon[35306]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:33.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:33 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T08:38:33.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:33 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:34.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:33 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T08:38:34.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:33 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:34.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T08:38:34.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:34.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:34 smithi170 ceph-mon[35072]: Deploying daemon osd.1 on smithi170 2024-09-19T08:38:34.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:34 smithi170 ceph-mon[35072]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:35.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:34 smithi164 ceph-mon[30928]: Deploying daemon osd.1 on smithi170 2024-09-19T08:38:35.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:34 smithi164 ceph-mon[30928]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:35.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:34 smithi173 ceph-mon[35306]: Deploying daemon osd.1 on smithi170 2024-09-19T08:38:35.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:34 smithi173 ceph-mon[35306]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:37.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:36 smithi164 ceph-mon[30928]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:37.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:36 smithi170 ceph-mon[35072]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:37.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:36 smithi173 ceph-mon[35306]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:38.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:38 smithi170 ceph-mon[35072]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:39.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:38 smithi164 ceph-mon[30928]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:39.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:38 smithi173 ceph-mon[35306]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:40.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:39 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:38:40.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:39 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:40.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:39 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:40.062 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:39 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:38:40.062 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:39 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:40.062 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:39 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:40.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:39 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:38:40.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:39 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:40.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:39 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:41.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:40 smithi164 ceph-mon[30928]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:41.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:40 smithi170 ceph-mon[35072]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:41.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:40 smithi173 ceph-mon[35306]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:42.743 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:42 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:42.743 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:42 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:42.743 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:42 smithi170 ceph-mon[35072]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:42.743 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:42 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:42.743 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:42 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:38:42.743 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:42 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:42.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:42 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:42.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:42 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:42.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:42 smithi173 ceph-mon[35306]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:42.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:42 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:42.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:42 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:38:42.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:42 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:43.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:42 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:43.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:42 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:43.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:42 smithi164 ceph-mon[30928]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:43.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:42 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:43.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:42 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:38:43.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:42 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:43.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:43.059+0000 7f392cff9640 1 -- 172.21.15.170:0/3768629360 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3930062630 con 0x7f39180a31e0 2024-09-19T08:38:43.060 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 1 on host 'smithi170' 2024-09-19T08:38:43.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:43.061+0000 7f3935c84640 1 -- 172.21.15.170:0/3768629360 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f39180a31e0 msgr2=0x7f39180a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:43.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:43.061+0000 7f3935c84640 1 --2- 172.21.15.170:0/3768629360 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f39180a31e0 0x7f39180a56a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f393006a590 tx=0x7f392000a2d0 comp rx=0 tx=0).stop 2024-09-19T08:38:43.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:43.061+0000 7f3935c84640 1 -- 172.21.15.170:0/3768629360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3930103870 msgr2=0x7f393019f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:43.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:43.061+0000 7f3935c84640 1 --2- 172.21.15.170:0/3768629360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3930103870 0x7f393019f540 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f392800eaa0 tx=0x7f392800ef60 comp rx=0 tx=0).stop 2024-09-19T08:38:43.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:43.061+0000 7f3935c84640 1 -- 172.21.15.170:0/3768629360 shutdown_connections 2024-09-19T08:38:43.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:43.061+0000 7f3935c84640 1 --2- 172.21.15.170:0/3768629360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3930103870 0x7f393019f540 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:43.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:43.061+0000 7f3935c84640 1 --2- 172.21.15.170:0/3768629360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3930102eb0 0x7f39301984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:43.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:43.061+0000 7f3935c84640 1 --2- 172.21.15.170:0/3768629360 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f39180a31e0 0x7f39180a56a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:43.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:43.061+0000 7f3935c84640 1 --2- 172.21.15.170:0/3768629360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39300ffed0 0x7f3930197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:43.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:43.061+0000 7f3935c84640 1 -- 172.21.15.170:0/3768629360 >> 172.21.15.170:0/3768629360 conn(0x7f39300fba40 msgr2=0x7f3930106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:43.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:43.061+0000 7f3935c84640 1 -- 172.21.15.170:0/3768629360 shutdown_connections 2024-09-19T08:38:43.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:38:43.061+0000 7f3935c84640 1 -- 172.21.15.170:0/3768629360 wait complete. 2024-09-19T08:38:43.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:43 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:38:43.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:43 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:43.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:43 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:43.692 DEBUG:teuthology.orchestra.run.smithi170:osd.1> sudo journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.1.service 2024-09-19T08:38:43.699 INFO:tasks.cephadm:Deploying osd.2 on smithi173 with /dev/vg_nvme/lv_4... 2024-09-19T08:38:43.699 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T08:38:43.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:43 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:38:43.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:43 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:43.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:43 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:43.951 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:38:44.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:43 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:38:44.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:43 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:44.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:43 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:44.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:44 smithi173 ceph-mon[35306]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:44.803 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:44 smithi170 ceph-mon[35072]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:45.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:44 smithi164 ceph-mon[30928]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:45.092 INFO:journalctl@ceph.osd.1.smithi170.stdout:Sep 19 08:38:44 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-1[39460]: 2024-09-19T08:38:44.976+0000 7f4b81488640 -1 osd.1 0 log_to_monitors true 2024-09-19T08:38:45.640 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:45 smithi173 ceph-mon[35306]: from='osd.1 [v2:172.21.15.170:6800/2167066371,v1:172.21.15.170:6801/2167066371]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T08:38:45.640 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:45 smithi173 ceph-mon[35306]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T08:38:45.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:45 smithi170 ceph-mon[35072]: from='osd.1 [v2:172.21.15.170:6800/2167066371,v1:172.21.15.170:6801/2167066371]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T08:38:45.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:45 smithi170 ceph-mon[35072]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T08:38:46.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:45 smithi164 ceph-mon[30928]: from='osd.1 [v2:172.21.15.170:6800/2167066371,v1:172.21.15.170:6801/2167066371]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T08:38:46.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:45 smithi164 ceph-mon[30928]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T08:38:46.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:46 smithi173 ceph-mon[35306]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T08:38:46.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:46 smithi173 ceph-mon[35306]: from='osd.1 [v2:172.21.15.170:6800/2167066371,v1:172.21.15.170:6801/2167066371]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-19T08:38:46.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:46 smithi173 ceph-mon[35306]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T08:38:46.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:46 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:38:46.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:46 smithi173 ceph-mon[35306]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-19T08:38:46.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:46 smithi173 ceph-mon[35306]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:46.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:46 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:46.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:46 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:46.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:46 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:38:46.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:46 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:46.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:46 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:46.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:46 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:38:46.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:46 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:46.765 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-19T08:38:46.791 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi173:vg_nvme/lv_4 2024-09-19T08:38:46.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:46 smithi170 ceph-mon[35072]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T08:38:46.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:46 smithi170 ceph-mon[35072]: from='osd.1 [v2:172.21.15.170:6800/2167066371,v1:172.21.15.170:6801/2167066371]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-19T08:38:46.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:46 smithi170 ceph-mon[35072]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T08:38:46.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:46 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:38:46.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:46 smithi170 ceph-mon[35072]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-19T08:38:46.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:46 smithi170 ceph-mon[35072]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:46.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:46 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:46.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:46 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:46.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:46 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:38:46.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:46 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:46.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:46 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:46.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:46 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:38:46.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:46 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:47.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:46 smithi164 ceph-mon[30928]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T08:38:47.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:46 smithi164 ceph-mon[30928]: from='osd.1 [v2:172.21.15.170:6800/2167066371,v1:172.21.15.170:6801/2167066371]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-19T08:38:47.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:46 smithi164 ceph-mon[30928]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T08:38:47.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:46 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:38:47.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:46 smithi164 ceph-mon[30928]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-19T08:38:47.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:46 smithi164 ceph-mon[30928]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:47.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:46 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:47.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:46 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:47.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:46 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:38:47.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:46 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:47.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:46 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:47.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:46 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:38:47.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:46 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:38:47.035 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:38:47.375 INFO:journalctl@ceph.osd.1.smithi170.stdout:Sep 19 08:38:46 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-1[39460]: 2024-09-19T08:38:46.987+0000 7f4b7d40b640 -1 osd.1 0 waiting for initial osdmap 2024-09-19T08:38:47.375 INFO:journalctl@ceph.osd.1.smithi170.stdout:Sep 19 08:38:46 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-1[39460]: 2024-09-19T08:38:46.990+0000 7f4b7522b640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:38:47.499 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.465+0000 7f892e7e2640 1 -- 172.21.15.173:0/1383198824 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89281037c0 msgr2=0x7f892810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:47.499 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.465+0000 7f892e7e2640 1 --2- 172.21.15.173:0/1383198824 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89281037c0 0x7f892810a050 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8920009ca0 tx=0x7f892002a8c0 comp rx=0 tx=0).stop 2024-09-19T08:38:47.499 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.465+0000 7f892e7e2640 1 -- 172.21.15.173:0/1383198824 shutdown_connections 2024-09-19T08:38:47.499 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.465+0000 7f892e7e2640 1 --2- 172.21.15.173:0/1383198824 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89281037c0 0x7f892810a050 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:47.500 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.465+0000 7f892e7e2640 1 --2- 172.21.15.173:0/1383198824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8928102e00 0x7f8928103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:47.500 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.465+0000 7f892e7e2640 1 --2- 172.21.15.173:0/1383198824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8928101c00 0x7f8928102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:47.500 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.465+0000 7f892e7e2640 1 -- 172.21.15.173:0/1383198824 >> 172.21.15.173:0/1383198824 conn(0x7f89280fd390 msgr2=0x7f89280ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:47.500 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.465+0000 7f892e7e2640 1 -- 172.21.15.173:0/1383198824 shutdown_connections 2024-09-19T08:38:47.500 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.466+0000 7f892e7e2640 1 -- 172.21.15.173:0/1383198824 wait complete. 2024-09-19T08:38:47.500 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.466+0000 7f892e7e2640 1 Processor -- start 2024-09-19T08:38:47.500 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.466+0000 7f892e7e2640 1 -- start start 2024-09-19T08:38:47.500 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.466+0000 7f892e7e2640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8928101c00 0x7f8928198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:47.500 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.466+0000 7f892e7e2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8928102e00 0x7f8928198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:47.500 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.467+0000 7f892e7e2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89281037c0 0x7f892819f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:47.500 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.467+0000 7f892e7e2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f892810d980 con 0x7f89281037c0 2024-09-19T08:38:47.500 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.467+0000 7f892e7e2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f892810d800 con 0x7f8928102e00 2024-09-19T08:38:47.501 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.467+0000 7f892e7e2640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f892810db00 con 0x7f8928101c00 2024-09-19T08:38:47.501 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.467+0000 7f8927fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8928101c00 0x7f8928198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:47.501 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.467+0000 7f8927fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8928101c00 0x7f8928198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:44568/0 (socket says 172.21.15.173:44568) 2024-09-19T08:38:47.501 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.467+0000 7f8927fff640 1 -- 172.21.15.173:0/1408619475 learned_addr learned my addr 172.21.15.173:0/1408619475 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-09-19T08:38:47.501 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.467+0000 7f89277fe640 1 --2- 172.21.15.173:0/1408619475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8928102e00 0x7f8928198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:47.501 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.467+0000 7f892cd7f640 1 --2- 172.21.15.173:0/1408619475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89281037c0 0x7f892819f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:47.501 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.467+0000 7f8927fff640 1 -- 172.21.15.173:0/1408619475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8928102e00 msgr2=0x7f8928198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:47.501 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.467+0000 7f8927fff640 1 --2- 172.21.15.173:0/1408619475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8928102e00 0x7f8928198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:47.501 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.467+0000 7f8927fff640 1 -- 172.21.15.173:0/1408619475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89281037c0 msgr2=0x7f892819f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:47.501 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.467+0000 7f8927fff640 1 --2- 172.21.15.173:0/1408619475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89281037c0 0x7f892819f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:47.501 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.467+0000 7f8927fff640 1 -- 172.21.15.173:0/1408619475 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f892819fc30 con 0x7f8928101c00 2024-09-19T08:38:47.501 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.467+0000 7f8927fff640 1 --2- 172.21.15.173:0/1408619475 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8928101c00 0x7f8928198000 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f891800b450 tx=0x7f891800b850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:47.502 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.467+0000 7f89257fa640 1 -- 172.21.15.173:0/1408619475 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8918015020 con 0x7f8928101c00 2024-09-19T08:38:47.502 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.467+0000 7f89257fa640 1 -- 172.21.15.173:0/1408619475 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8918011d10 con 0x7f8928101c00 2024-09-19T08:38:47.502 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.467+0000 7f892e7e2640 1 -- 172.21.15.173:0/1408619475 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f892819ff20 con 0x7f8928101c00 2024-09-19T08:38:47.502 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.467+0000 7f89257fa640 1 -- 172.21.15.173:0/1408619475 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891800f2e0 con 0x7f8928101c00 2024-09-19T08:38:47.502 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.467+0000 7f892e7e2640 1 -- 172.21.15.173:0/1408619475 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89281a03f0 con 0x7f8928101c00 2024-09-19T08:38:47.502 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.469+0000 7f892e7e2640 1 -- 172.21.15.173:0/1408619475 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8928109050 con 0x7f8928101c00 2024-09-19T08:38:47.502 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.470+0000 7f89257fa640 1 -- 172.21.15.173:0/1408619475 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7f8918018020 con 0x7f8928101c00 2024-09-19T08:38:47.502 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.470+0000 7f89257fa640 1 --2- 172.21.15.173:0/1408619475 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f89100a31e0 0x7f89100a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:47.502 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.470+0000 7f89257fa640 1 -- 172.21.15.173:0/1408619475 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7f89180cdc60 con 0x7f8928101c00 2024-09-19T08:38:47.502 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.471+0000 7f89277fe640 1 --2- 172.21.15.173:0/1408619475 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f89100a31e0 0x7f89100a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:47.502 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.471+0000 7f89277fe640 1 --2- 172.21.15.173:0/1408619475 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f89100a31e0 0x7f89100a56a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8928199520 tx=0x7f891c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:47.502 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.474+0000 7f89257fa640 1 -- 172.21.15.173:0/1408619475 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f891801f020 con 0x7f8928101c00 2024-09-19T08:38:47.649 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:38:47.648+0000 7f892e7e2640 1 -- 172.21.15.173:0/1408619475 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f8928062630 con 0x7f89100a31e0 2024-09-19T08:38:47.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:47 smithi170 ceph-mon[35072]: Detected new or changed devices on smithi170 2024-09-19T08:38:47.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:47 smithi170 ceph-mon[35072]: Adjusting osd_memory_target on smithi170 to 17088M 2024-09-19T08:38:47.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:47 smithi170 ceph-mon[35072]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-09-19T08:38:47.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:47 smithi170 ceph-mon[35072]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T08:38:47.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:47 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:38:47.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:47 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:38:47.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:47 smithi170 ceph-mon[35072]: from='osd.1 ' entity='osd.1' 2024-09-19T08:38:47.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:47 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:38:47.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:47 smithi173 ceph-mon[35306]: Detected new or changed devices on smithi170 2024-09-19T08:38:47.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:47 smithi173 ceph-mon[35306]: Adjusting osd_memory_target on smithi170 to 17088M 2024-09-19T08:38:47.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:47 smithi173 ceph-mon[35306]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-09-19T08:38:47.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:47 smithi173 ceph-mon[35306]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T08:38:47.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:47 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:38:47.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:47 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:38:47.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:47 smithi173 ceph-mon[35306]: from='osd.1 ' entity='osd.1' 2024-09-19T08:38:47.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:47 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:38:48.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:47 smithi164 ceph-mon[30928]: Detected new or changed devices on smithi170 2024-09-19T08:38:48.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:47 smithi164 ceph-mon[30928]: Adjusting osd_memory_target on smithi170 to 17088M 2024-09-19T08:38:48.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:47 smithi164 ceph-mon[30928]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-09-19T08:38:48.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:47 smithi164 ceph-mon[30928]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T08:38:48.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:47 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:38:48.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:47 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:38:48.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:47 smithi164 ceph-mon[30928]: from='osd.1 ' entity='osd.1' 2024-09-19T08:38:48.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:47 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:38:48.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:48 smithi170 ceph-mon[35072]: purged_snaps scrub starts 2024-09-19T08:38:48.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:48 smithi170 ceph-mon[35072]: purged_snaps scrub ok 2024-09-19T08:38:48.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:48 smithi170 ceph-mon[35072]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:38:48.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:48 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:38:48.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:48 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:38:48.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:48 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:48.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:48 smithi170 ceph-mon[35072]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:48.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:48 smithi170 ceph-mon[35072]: osd.1 [v2:172.21.15.170:6800/2167066371,v1:172.21.15.170:6801/2167066371] boot 2024-09-19T08:38:48.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:48 smithi170 ceph-mon[35072]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T08:38:48.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:48 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:38:48.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:48 smithi173 ceph-mon[35306]: purged_snaps scrub starts 2024-09-19T08:38:48.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:48 smithi173 ceph-mon[35306]: purged_snaps scrub ok 2024-09-19T08:38:48.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:48 smithi173 ceph-mon[35306]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:38:48.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:48 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:38:48.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:48 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:38:48.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:48 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:48.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:48 smithi173 ceph-mon[35306]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:48.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:48 smithi173 ceph-mon[35306]: osd.1 [v2:172.21.15.170:6800/2167066371,v1:172.21.15.170:6801/2167066371] boot 2024-09-19T08:38:48.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:48 smithi173 ceph-mon[35306]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T08:38:48.912 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:48 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:38:49.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:48 smithi164 ceph-mon[30928]: purged_snaps scrub starts 2024-09-19T08:38:49.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:48 smithi164 ceph-mon[30928]: purged_snaps scrub ok 2024-09-19T08:38:49.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:48 smithi164 ceph-mon[30928]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:38:49.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:48 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:38:49.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:48 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:38:49.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:48 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:49.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:48 smithi164 ceph-mon[30928]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T08:38:49.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:48 smithi164 ceph-mon[30928]: osd.1 [v2:172.21.15.170:6800/2167066371,v1:172.21.15.170:6801/2167066371] boot 2024-09-19T08:38:49.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:48 smithi164 ceph-mon[30928]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T08:38:49.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:48 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:38:50.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:50 smithi173 ceph-mon[35306]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T08:38:50.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:50 smithi173 ceph-mon[35306]: pgmap v58: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:38:50.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:50 smithi173 ceph-mon[35306]: from='client.? 172.21.15.173:0/1439811595' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4158cc7-9822-42a7-9601-d65089907584"}]: dispatch 2024-09-19T08:38:50.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:50 smithi173 ceph-mon[35306]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4158cc7-9822-42a7-9601-d65089907584"}]: dispatch 2024-09-19T08:38:50.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:50 smithi173 ceph-mon[35306]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4158cc7-9822-42a7-9601-d65089907584"}]': finished 2024-09-19T08:38:50.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:50 smithi173 ceph-mon[35306]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T08:38:50.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:50 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:38:50.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:50 smithi173 ceph-mon[35306]: from='client.? 172.21.15.173:0/321416324' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:38:51.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:50 smithi164 ceph-mon[30928]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T08:38:51.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:50 smithi164 ceph-mon[30928]: pgmap v58: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:38:51.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:50 smithi164 ceph-mon[30928]: from='client.? 172.21.15.173:0/1439811595' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4158cc7-9822-42a7-9601-d65089907584"}]: dispatch 2024-09-19T08:38:51.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:50 smithi164 ceph-mon[30928]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4158cc7-9822-42a7-9601-d65089907584"}]: dispatch 2024-09-19T08:38:51.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:50 smithi164 ceph-mon[30928]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4158cc7-9822-42a7-9601-d65089907584"}]': finished 2024-09-19T08:38:51.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:50 smithi164 ceph-mon[30928]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T08:38:51.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:50 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:38:51.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:50 smithi164 ceph-mon[30928]: from='client.? 172.21.15.173:0/321416324' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:38:51.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:50 smithi170 ceph-mon[35072]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T08:38:51.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:50 smithi170 ceph-mon[35072]: pgmap v58: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:38:51.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:50 smithi170 ceph-mon[35072]: from='client.? 172.21.15.173:0/1439811595' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4158cc7-9822-42a7-9601-d65089907584"}]: dispatch 2024-09-19T08:38:51.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:50 smithi170 ceph-mon[35072]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4158cc7-9822-42a7-9601-d65089907584"}]: dispatch 2024-09-19T08:38:51.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:50 smithi170 ceph-mon[35072]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4158cc7-9822-42a7-9601-d65089907584"}]': finished 2024-09-19T08:38:51.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:50 smithi170 ceph-mon[35072]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T08:38:51.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:50 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:38:51.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:50 smithi170 ceph-mon[35072]: from='client.? 172.21.15.173:0/321416324' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:38:53.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:52 smithi164 ceph-mon[30928]: pgmap v60: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:38:53.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:52 smithi170 ceph-mon[35072]: pgmap v60: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:38:53.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:52 smithi173 ceph-mon[35306]: pgmap v60: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:38:55.003 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:54 smithi173 ceph-mon[35306]: pgmap v61: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:38:55.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:54 smithi164 ceph-mon[30928]: pgmap v61: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:38:55.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:54 smithi170 ceph-mon[35072]: pgmap v61: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:38:56.930 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:56 smithi173 ceph-mon[35306]: pgmap v62: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:38:57.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:56 smithi164 ceph-mon[30928]: pgmap v62: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:38:57.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:56 smithi170 ceph-mon[35072]: pgmap v62: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:38:58.850 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:58 smithi173 ceph-mon[35306]: pgmap v63: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:38:58.850 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:58 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T08:38:58.850 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:58 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:59.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:58 smithi164 ceph-mon[30928]: pgmap v63: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:38:59.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:58 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T08:38:59.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:58 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:59.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:58 smithi170 ceph-mon[35072]: pgmap v63: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:38:59.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:58 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T08:38:59.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:58 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:38:59.994 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:38:59 smithi173 ceph-mon[35306]: Deploying daemon osd.2 on smithi173 2024-09-19T08:39:00.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:38:59 smithi164 ceph-mon[30928]: Deploying daemon osd.2 on smithi173 2024-09-19T08:39:00.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:38:59 smithi170 ceph-mon[35072]: Deploying daemon osd.2 on smithi173 2024-09-19T08:39:01.014 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:00 smithi173 ceph-mon[35306]: pgmap v64: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:39:01.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:00 smithi164 ceph-mon[30928]: pgmap v64: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:39:01.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:00 smithi170 ceph-mon[35072]: pgmap v64: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:39:03.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:02 smithi164 ceph-mon[30928]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:39:03.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:02 smithi170 ceph-mon[35072]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:39:03.131 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:02 smithi173 ceph-mon[35306]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:39:05.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:04 smithi164 ceph-mon[30928]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:39:05.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:04 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:39:05.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:04 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:05.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:04 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:05.039 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:04 smithi173 ceph-mon[35306]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:39:05.039 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:04 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:39:05.039 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:04 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:05.040 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:04 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:05.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:04 smithi170 ceph-mon[35072]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:39:05.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:04 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:39:05.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:04 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:05.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:04 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:07.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:06 smithi164 ceph-mon[30928]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:39:07.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:06 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:07.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:06 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:07.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:06 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:39:07.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:06 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:39:07.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:06 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:07.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:06 smithi170 ceph-mon[35072]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:39:07.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:06 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:07.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:06 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:07.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:06 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:39:07.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:06 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:39:07.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:06 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:07.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:06 smithi173 ceph-mon[35306]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:39:07.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:06 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:07.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:06 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:07.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:06 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:39:07.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:06 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:39:07.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:06 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:08.418 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:39:08.417+0000 7f89257fa640 1 -- 172.21.15.173:0/1408619475 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8928062630 con 0x7f89100a31e0 2024-09-19T08:39:08.418 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 2 on host 'smithi173' 2024-09-19T08:39:08.420 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:39:08.420+0000 7f892e7e2640 1 -- 172.21.15.173:0/1408619475 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f89100a31e0 msgr2=0x7f89100a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:08.420 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:39:08.420+0000 7f892e7e2640 1 --2- 172.21.15.173:0/1408619475 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f89100a31e0 0x7f89100a56a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8928199520 tx=0x7f891c009110 comp rx=0 tx=0).stop 2024-09-19T08:39:08.420 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:39:08.420+0000 7f892e7e2640 1 -- 172.21.15.173:0/1408619475 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8928101c00 msgr2=0x7f8928198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:08.420 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:39:08.420+0000 7f892e7e2640 1 --2- 172.21.15.173:0/1408619475 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8928101c00 0x7f8928198000 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f891800b450 tx=0x7f891800b850 comp rx=0 tx=0).stop 2024-09-19T08:39:08.420 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:39:08.420+0000 7f892e7e2640 1 -- 172.21.15.173:0/1408619475 shutdown_connections 2024-09-19T08:39:08.420 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:39:08.420+0000 7f892e7e2640 1 --2- 172.21.15.173:0/1408619475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89281037c0 0x7f892819f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:08.420 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:39:08.420+0000 7f892e7e2640 1 --2- 172.21.15.173:0/1408619475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8928102e00 0x7f8928198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:08.420 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:39:08.420+0000 7f892e7e2640 1 --2- 172.21.15.173:0/1408619475 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f89100a31e0 0x7f89100a56a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:08.421 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:39:08.420+0000 7f892e7e2640 1 --2- 172.21.15.173:0/1408619475 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8928101c00 0x7f8928198000 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:08.421 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:39:08.420+0000 7f892e7e2640 1 -- 172.21.15.173:0/1408619475 >> 172.21.15.173:0/1408619475 conn(0x7f89280fd390 msgr2=0x7f8928106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:08.421 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:39:08.420+0000 7f892e7e2640 1 -- 172.21.15.173:0/1408619475 shutdown_connections 2024-09-19T08:39:08.421 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:39:08.420+0000 7f892e7e2640 1 -- 172.21.15.173:0/1408619475 wait complete. 2024-09-19T08:39:09.022 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:08 smithi173 ceph-mon[35306]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:39:09.022 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:08 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:39:09.022 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:08 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:09.022 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:08 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:09.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:08 smithi164 ceph-mon[30928]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:39:09.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:08 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:39:09.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:08 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:09.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:08 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:09.057 DEBUG:teuthology.orchestra.run.smithi173:osd.2> sudo journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.2.service 2024-09-19T08:39:09.060 INFO:tasks.cephadm:Deploying osd.3 on smithi164 with /dev/vg_nvme/lv_3... 2024-09-19T08:39:09.061 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T08:39:09.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:08 smithi170 ceph-mon[35072]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:39:09.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:08 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:39:09.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:08 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:09.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:08 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:09.301 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:39:10.397 INFO:journalctl@ceph.osd.2.smithi173.stdout:Sep 19 08:39:10 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-2[40262]: 2024-09-19T08:39:10.348+0000 7f43c1515640 -1 osd.2 0 log_to_monitors true 2024-09-19T08:39:11.010 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:10 smithi164 ceph-mon[30928]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:39:11.011 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:10 smithi164 ceph-mon[30928]: from='osd.2 [v2:172.21.15.173:6800/1646561048,v1:172.21.15.173:6801/1646561048]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T08:39:11.011 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:10 smithi164 ceph-mon[30928]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T08:39:11.084 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:10 smithi173 ceph-mon[35306]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:39:11.084 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:10 smithi173 ceph-mon[35306]: from='osd.2 [v2:172.21.15.173:6800/1646561048,v1:172.21.15.173:6801/1646561048]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T08:39:11.084 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:10 smithi173 ceph-mon[35306]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T08:39:11.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:10 smithi170 ceph-mon[35072]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:39:11.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:10 smithi170 ceph-mon[35072]: from='osd.2 [v2:172.21.15.173:6800/1646561048,v1:172.21.15.173:6801/1646561048]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T08:39:11.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:10 smithi170 ceph-mon[35072]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T08:39:11.763 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:39:11.796 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi164:vg_nvme/lv_3 2024-09-19T08:39:12.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:11 smithi164 ceph-mon[30928]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T08:39:12.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:11 smithi164 ceph-mon[30928]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T08:39:12.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:11 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:39:12.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:11 smithi164 ceph-mon[30928]: from='osd.2 [v2:172.21.15.173:6800/1646561048,v1:172.21.15.173:6801/1646561048]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-19T08:39:12.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:11 smithi164 ceph-mon[30928]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-19T08:39:12.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:11 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:12.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:11 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:12.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:11 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:39:12.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:11 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:12.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:11 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:39:12.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:11 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:39:12.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:11 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:12.050 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:39:12.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:11 smithi170 ceph-mon[35072]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T08:39:12.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:11 smithi170 ceph-mon[35072]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T08:39:12.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:11 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:39:12.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:11 smithi170 ceph-mon[35072]: from='osd.2 [v2:172.21.15.173:6800/1646561048,v1:172.21.15.173:6801/1646561048]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-19T08:39:12.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:11 smithi170 ceph-mon[35072]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-19T08:39:12.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:11 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:12.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:11 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:12.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:11 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:39:12.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:11 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:12.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:11 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:39:12.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:11 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:39:12.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:11 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:12.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:11 smithi173 ceph-mon[35306]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T08:39:12.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:11 smithi173 ceph-mon[35306]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T08:39:12.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:11 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:39:12.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:11 smithi173 ceph-mon[35306]: from='osd.2 [v2:172.21.15.173:6800/1646561048,v1:172.21.15.173:6801/1646561048]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-19T08:39:12.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:11 smithi173 ceph-mon[35306]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-19T08:39:12.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:11 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:12.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:11 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:12.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:11 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:39:12.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:11 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:12.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:11 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:39:12.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:11 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:39:12.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:11 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:12.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.481+0000 7f1654c3d640 1 -- 172.21.15.164:0/1748938894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1650101c00 msgr2=0x7f1650102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:12.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.481+0000 7f1654c3d640 1 --2- 172.21.15.164:0/1748938894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1650101c00 0x7f1650102000 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1644009900 tx=0x7f164402a410 comp rx=0 tx=0).stop 2024-09-19T08:39:12.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.481+0000 7f1654c3d640 1 -- 172.21.15.164:0/1748938894 shutdown_connections 2024-09-19T08:39:12.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.481+0000 7f1654c3d640 1 --2- 172.21.15.164:0/1748938894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16501037c0 0x7f165010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:12.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.482+0000 7f1654c3d640 1 --2- 172.21.15.164:0/1748938894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1650102e00 0x7f1650103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:12.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.482+0000 7f1654c3d640 1 --2- 172.21.15.164:0/1748938894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1650101c00 0x7f1650102000 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:12.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.482+0000 7f1654c3d640 1 -- 172.21.15.164:0/1748938894 >> 172.21.15.164:0/1748938894 conn(0x7f16500fd390 msgr2=0x7f16500ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:12.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.482+0000 7f1654c3d640 1 -- 172.21.15.164:0/1748938894 shutdown_connections 2024-09-19T08:39:12.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.482+0000 7f1654c3d640 1 -- 172.21.15.164:0/1748938894 wait complete. 2024-09-19T08:39:12.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.482+0000 7f1654c3d640 1 Processor -- start 2024-09-19T08:39:12.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.482+0000 7f1654c3d640 1 -- start start 2024-09-19T08:39:12.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.482+0000 7f1654c3d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1650101c00 0x7f1650197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:12.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.482+0000 7f1654c3d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1650102e00 0x7f1650198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:12.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.483+0000 7f1654c3d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16501037c0 0x7f165019f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:12.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.483+0000 7f1654c3d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f165010d860 con 0x7f1650102e00 2024-09-19T08:39:12.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.483+0000 7f1654c3d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f165010d6e0 con 0x7f1650101c00 2024-09-19T08:39:12.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.483+0000 7f1654c3d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f165010d9e0 con 0x7f16501037c0 2024-09-19T08:39:12.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.483+0000 7f164dd9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1650102e00 0x7f1650198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:12.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.483+0000 7f164dd9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1650102e00 0x7f1650198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49962/0 (socket says 172.21.15.164:49962) 2024-09-19T08:39:12.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.483+0000 7f164dd9b640 1 -- 172.21.15.164:0/915488222 learned_addr learned my addr 172.21.15.164:0/915488222 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:39:12.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.483+0000 7f164dd9b640 1 -- 172.21.15.164:0/915488222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16501037c0 msgr2=0x7f165019f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:12.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.483+0000 7f164ed9d640 1 --2- 172.21.15.164:0/915488222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16501037c0 0x7f165019f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:12.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.483+0000 7f164e59c640 1 --2- 172.21.15.164:0/915488222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1650101c00 0x7f1650197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:12.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.483+0000 7f164dd9b640 1 --2- 172.21.15.164:0/915488222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16501037c0 0x7f165019f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:12.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.483+0000 7f164dd9b640 1 -- 172.21.15.164:0/915488222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1650101c00 msgr2=0x7f1650197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:12.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.483+0000 7f164dd9b640 1 --2- 172.21.15.164:0/915488222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1650101c00 0x7f1650197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:12.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.483+0000 7f164dd9b640 1 -- 172.21.15.164:0/915488222 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f165019fb30 con 0x7f1650102e00 2024-09-19T08:39:12.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.483+0000 7f164ed9d640 1 --2- 172.21.15.164:0/915488222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16501037c0 0x7f165019f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:12.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.483+0000 7f164dd9b640 1 --2- 172.21.15.164:0/915488222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1650102e00 0x7f1650198440 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f164000d950 tx=0x7f164000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:12.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.483+0000 7f163f7fe640 1 -- 172.21.15.164:0/915488222 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1640017030 con 0x7f1650102e00 2024-09-19T08:39:12.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.483+0000 7f1654c3d640 1 -- 172.21.15.164:0/915488222 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f165019fe20 con 0x7f1650102e00 2024-09-19T08:39:12.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.483+0000 7f163f7fe640 1 -- 172.21.15.164:0/915488222 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1640004020 con 0x7f1650102e00 2024-09-19T08:39:12.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.483+0000 7f163f7fe640 1 -- 172.21.15.164:0/915488222 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16400047a0 con 0x7f1650102e00 2024-09-19T08:39:12.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.483+0000 7f1654c3d640 1 -- 172.21.15.164:0/915488222 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16501a0320 con 0x7f1650102e00 2024-09-19T08:39:12.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.484+0000 7f1654c3d640 1 -- 172.21.15.164:0/915488222 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1650109000 con 0x7f1650102e00 2024-09-19T08:39:12.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.486+0000 7f163f7fe640 1 -- 172.21.15.164:0/915488222 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7f16400041d0 con 0x7f1650102e00 2024-09-19T08:39:12.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.486+0000 7f163f7fe640 1 --2- 172.21.15.164:0/915488222 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f16380a31e0 0x7f16380a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:12.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.486+0000 7f164e59c640 1 --2- 172.21.15.164:0/915488222 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f16380a31e0 0x7f16380a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:12.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.486+0000 7f163f7fe640 1 -- 172.21.15.164:0/915488222 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7f16400cdc40 con 0x7f1650102e00 2024-09-19T08:39:12.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.487+0000 7f164e59c640 1 --2- 172.21.15.164:0/915488222 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f16380a31e0 0x7f16380a56a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1644031610 tx=0x7f1644000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:12.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.491+0000 7f163f7fe640 1 -- 172.21.15.164:0/915488222 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f16400d4080 con 0x7f1650102e00 2024-09-19T08:39:12.661 INFO:journalctl@ceph.osd.2.smithi173.stdout:Sep 19 08:39:12 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-2[40262]: 2024-09-19T08:39:12.281+0000 7f43bdcac640 -1 osd.2 0 waiting for initial osdmap 2024-09-19T08:39:12.661 INFO:journalctl@ceph.osd.2.smithi173.stdout:Sep 19 08:39:12 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-2[40262]: 2024-09-19T08:39:12.287+0000 7f43b52b8640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:39:12.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:12.664+0000 7f1654c3d640 1 -- 172.21.15.164:0/915488222 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f1650062630 con 0x7f16380a31e0 2024-09-19T08:39:13.070 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:12 smithi164 ceph-mon[30928]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:39:13.070 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:12 smithi164 ceph-mon[30928]: Detected new or changed devices on smithi173 2024-09-19T08:39:13.070 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:12 smithi164 ceph-mon[30928]: Adjusting osd_memory_target on smithi173 to 17088M 2024-09-19T08:39:13.070 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:12 smithi164 ceph-mon[30928]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-19T08:39:13.070 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:12 smithi164 ceph-mon[30928]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T08:39:13.070 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:12 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:39:13.070 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:12 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:39:13.071 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:12 smithi164 ceph-mon[30928]: from='osd.2 ' entity='osd.2' 2024-09-19T08:39:13.071 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:12 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:39:13.071 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:12 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:39:13.071 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:12 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:39:13.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:12 smithi170 ceph-mon[35072]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:39:13.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:12 smithi170 ceph-mon[35072]: Detected new or changed devices on smithi173 2024-09-19T08:39:13.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:12 smithi170 ceph-mon[35072]: Adjusting osd_memory_target on smithi173 to 17088M 2024-09-19T08:39:13.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:12 smithi170 ceph-mon[35072]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-19T08:39:13.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:12 smithi170 ceph-mon[35072]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T08:39:13.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:12 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:39:13.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:12 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:39:13.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:12 smithi170 ceph-mon[35072]: from='osd.2 ' entity='osd.2' 2024-09-19T08:39:13.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:12 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:39:13.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:12 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:39:13.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:12 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:39:13.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:12 smithi173 ceph-mon[35306]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:39:13.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:12 smithi173 ceph-mon[35306]: Detected new or changed devices on smithi173 2024-09-19T08:39:13.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:12 smithi173 ceph-mon[35306]: Adjusting osd_memory_target on smithi173 to 17088M 2024-09-19T08:39:13.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:12 smithi173 ceph-mon[35306]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-19T08:39:13.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:12 smithi173 ceph-mon[35306]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T08:39:13.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:12 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:39:13.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:12 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:39:13.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:12 smithi173 ceph-mon[35306]: from='osd.2 ' entity='osd.2' 2024-09-19T08:39:13.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:12 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:39:13.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:12 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:39:13.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:12 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:39:14.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:13 smithi164 ceph-mon[30928]: purged_snaps scrub starts 2024-09-19T08:39:14.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:13 smithi164 ceph-mon[30928]: purged_snaps scrub ok 2024-09-19T08:39:14.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:13 smithi164 ceph-mon[30928]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:39:14.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:13 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:39:14.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:13 smithi164 ceph-mon[30928]: osd.2 [v2:172.21.15.173:6800/1646561048,v1:172.21.15.173:6801/1646561048] boot 2024-09-19T08:39:14.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:13 smithi164 ceph-mon[30928]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T08:39:14.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:13 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:39:14.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:13 smithi170 ceph-mon[35072]: purged_snaps scrub starts 2024-09-19T08:39:14.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:13 smithi170 ceph-mon[35072]: purged_snaps scrub ok 2024-09-19T08:39:14.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:13 smithi170 ceph-mon[35072]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:39:14.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:13 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:39:14.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:13 smithi170 ceph-mon[35072]: osd.2 [v2:172.21.15.173:6800/1646561048,v1:172.21.15.173:6801/1646561048] boot 2024-09-19T08:39:14.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:13 smithi170 ceph-mon[35072]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T08:39:14.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:13 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:39:14.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:13 smithi173 ceph-mon[35306]: purged_snaps scrub starts 2024-09-19T08:39:14.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:13 smithi173 ceph-mon[35306]: purged_snaps scrub ok 2024-09-19T08:39:14.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:13 smithi173 ceph-mon[35306]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:39:14.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:13 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:39:14.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:13 smithi173 ceph-mon[35306]: osd.2 [v2:172.21.15.173:6800/1646561048,v1:172.21.15.173:6801/1646561048] boot 2024-09-19T08:39:14.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:13 smithi173 ceph-mon[35306]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T08:39:14.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:13 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:39:15.281 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:14 smithi164 ceph-mon[30928]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:39:15.281 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:14 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T08:39:15.281 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:14 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T08:39:15.281 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:14 smithi164 ceph-mon[30928]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T08:39:15.281 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:14 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T08:39:15.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:14 smithi170 ceph-mon[35072]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:39:15.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:14 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T08:39:15.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:14 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T08:39:15.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:14 smithi170 ceph-mon[35072]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T08:39:15.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:14 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T08:39:15.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:14 smithi173 ceph-mon[35306]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T08:39:15.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:14 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T08:39:15.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:14 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T08:39:15.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:14 smithi173 ceph-mon[35306]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T08:39:15.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:14 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T08:39:15.532 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 19 08:39:15 smithi164 sudo[49552]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T08:39:15.532 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 19 08:39:15 smithi164 sudo[49552]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:39:15.532 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 19 08:39:15 smithi164 sudo[49552]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:39:15.532 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 19 08:39:15 smithi164 sudo[49552]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:39:15.964 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 19 08:39:15 smithi164 sudo[49558]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T08:39:15.964 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 19 08:39:15 smithi164 sudo[49558]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:39:15.965 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 19 08:39:15 smithi164 sudo[49558]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:39:15.965 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 19 08:39:15 smithi164 sudo[49558]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:39:16.169 INFO:journalctl@ceph.osd.1.smithi170.stdout:Sep 19 08:39:15 smithi170 sudo[43550]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T08:39:16.169 INFO:journalctl@ceph.osd.1.smithi170.stdout:Sep 19 08:39:15 smithi170 sudo[43550]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:39:16.169 INFO:journalctl@ceph.osd.1.smithi170.stdout:Sep 19 08:39:15 smithi170 sudo[43550]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:39:16.169 INFO:journalctl@ceph.osd.1.smithi170.stdout:Sep 19 08:39:15 smithi170 sudo[43550]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:39:16.169 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:15 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3217726139' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cf3f7f1-3a8c-42ce-88a9-9bea52e02221"}]: dispatch 2024-09-19T08:39:16.169 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:15 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T08:39:16.169 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:15 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3217726139' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0cf3f7f1-3a8c-42ce-88a9-9bea52e02221"}]': finished 2024-09-19T08:39:16.170 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:15 smithi170 ceph-mon[35072]: osdmap e20: 4 total, 3 up, 4 in 2024-09-19T08:39:16.170 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:15 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:39:16.170 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:15 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/743368014' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:39:16.170 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:15 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:16.170 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:15 smithi170 ceph-mon[35072]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:16.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:15 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3217726139' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cf3f7f1-3a8c-42ce-88a9-9bea52e02221"}]: dispatch 2024-09-19T08:39:16.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:15 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T08:39:16.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:15 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3217726139' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0cf3f7f1-3a8c-42ce-88a9-9bea52e02221"}]': finished 2024-09-19T08:39:16.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:15 smithi164 ceph-mon[30928]: osdmap e20: 4 total, 3 up, 4 in 2024-09-19T08:39:16.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:15 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:39:16.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:15 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/743368014' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:39:16.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:15 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:16.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:15 smithi164 ceph-mon[30928]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:16.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:15 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3217726139' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cf3f7f1-3a8c-42ce-88a9-9bea52e02221"}]: dispatch 2024-09-19T08:39:16.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:15 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T08:39:16.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:15 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3217726139' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0cf3f7f1-3a8c-42ce-88a9-9bea52e02221"}]': finished 2024-09-19T08:39:16.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:15 smithi173 ceph-mon[35306]: osdmap e20: 4 total, 3 up, 4 in 2024-09-19T08:39:16.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:15 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:39:16.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:15 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/743368014' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:39:16.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:15 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:16.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:15 smithi173 ceph-mon[35306]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:16.625 INFO:journalctl@ceph.osd.1.smithi170.stdout:Sep 19 08:39:16 smithi170 sudo[43553]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T08:39:16.625 INFO:journalctl@ceph.osd.1.smithi170.stdout:Sep 19 08:39:16 smithi170 sudo[43553]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:39:16.625 INFO:journalctl@ceph.osd.1.smithi170.stdout:Sep 19 08:39:16 smithi170 sudo[43553]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:39:16.625 INFO:journalctl@ceph.osd.1.smithi170.stdout:Sep 19 08:39:16 smithi170 sudo[43553]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:39:16.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:16.682+0000 7f163f7fe640 1 -- 172.21.15.164:0/915488222 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f1640098020 con 0x7f1650102e00 2024-09-19T08:39:16.697 INFO:journalctl@ceph.osd.2.smithi173.stdout:Sep 19 08:39:16 smithi173 sudo[43902]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T08:39:16.697 INFO:journalctl@ceph.osd.2.smithi173.stdout:Sep 19 08:39:16 smithi173 sudo[43902]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:39:16.697 INFO:journalctl@ceph.osd.2.smithi173.stdout:Sep 19 08:39:16 smithi173 sudo[43902]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:39:16.697 INFO:journalctl@ceph.osd.2.smithi173.stdout:Sep 19 08:39:16 smithi173 sudo[43902]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:39:17.160 INFO:journalctl@ceph.osd.2.smithi173.stdout:Sep 19 08:39:16 smithi173 sudo[43905]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T08:39:17.160 INFO:journalctl@ceph.osd.2.smithi173.stdout:Sep 19 08:39:16 smithi173 sudo[43905]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:39:17.161 INFO:journalctl@ceph.osd.2.smithi173.stdout:Sep 19 08:39:16 smithi173 sudo[43905]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:39:17.161 INFO:journalctl@ceph.osd.2.smithi173.stdout:Sep 19 08:39:16 smithi173 sudo[43905]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:39:17.222 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:16 smithi164 sudo[49790]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T08:39:17.222 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:16 smithi164 sudo[49790]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:39:17.223 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:16 smithi164 sudo[49790]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:39:17.223 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:16 smithi164 sudo[49790]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:39:17.485 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:17 smithi173 ceph-mon[35306]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T08:39:17.485 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:17 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:39:17.485 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:17 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:17.485 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:17 smithi173 ceph-mon[35306]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T08:39:17.485 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:17 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:17.485 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:17 smithi173 ceph-mon[35306]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T08:39:17.485 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:17 smithi173 sudo[43908]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T08:39:17.485 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:17 smithi173 sudo[43908]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:39:17.485 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:17 smithi173 sudo[43908]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:39:17.485 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:17 smithi173 sudo[43908]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:39:17.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:17 smithi164 sudo[49976]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T08:39:17.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:17 smithi164 sudo[49976]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:39:17.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:17 smithi164 sudo[49976]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:39:17.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:17 smithi164 sudo[49976]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:39:17.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:17 smithi164 ceph-mon[30928]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T08:39:17.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:17 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:39:17.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:17 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:17.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:17 smithi164 ceph-mon[30928]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T08:39:17.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:17 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:17.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:17 smithi164 ceph-mon[30928]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T08:39:17.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:17 smithi170 ceph-mon[35072]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T08:39:17.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:17 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:39:17.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:17 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:17.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:17 smithi170 ceph-mon[35072]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T08:39:17.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:17 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:17.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:17 smithi170 ceph-mon[35072]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T08:39:17.913 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:17 smithi173 sudo[43911]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T08:39:17.913 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:17 smithi173 sudo[43911]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:39:17.913 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:17 smithi173 sudo[43911]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:39:17.913 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:17 smithi173 sudo[43911]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:39:18.219 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:17 smithi170 sudo[43556]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T08:39:18.219 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:17 smithi170 sudo[43556]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:39:18.219 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:17 smithi170 sudo[43556]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:39:18.220 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:17 smithi170 sudo[43556]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:39:18.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:18 smithi164 ceph-mon[30928]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T08:39:18.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:18 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:39:18.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:18 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:39:18.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:18 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:39:18.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:18 smithi164 ceph-mon[30928]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T08:39:18.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:18 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:39:18.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:18 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:39:18.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:18 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:39:18.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:18 smithi164 ceph-mon[30928]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:18.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:18 smithi164 ceph-mon[30928]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T08:39:18.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:18 smithi164 ceph-mon[30928]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T08:39:18.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:18 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:39:18.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:18 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:39:18.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:18 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:39:18.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:18 smithi164 ceph-mon[30928]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T08:39:18.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:18 smithi170 sudo[43559]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T08:39:18.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:18 smithi170 sudo[43559]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T08:39:18.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:18 smithi170 sudo[43559]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T08:39:18.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:18 smithi170 sudo[43559]: pam_unix(sudo:session): session closed for user root 2024-09-19T08:39:18.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:18 smithi170 ceph-mon[35072]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T08:39:18.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:18 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:39:18.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:18 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:39:18.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:18 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:39:18.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:18 smithi170 ceph-mon[35072]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T08:39:18.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:18 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:39:18.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:18 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:39:18.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:18 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:39:18.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:18 smithi170 ceph-mon[35072]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:18.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:18 smithi170 ceph-mon[35072]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T08:39:18.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:18 smithi170 ceph-mon[35072]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T08:39:18.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:18 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:39:18.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:18 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:39:18.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:18 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:39:18.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:18 smithi170 ceph-mon[35072]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T08:39:18.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:18 smithi173 ceph-mon[35306]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T08:39:18.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:18 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:39:18.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:18 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:39:18.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:18 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:39:18.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:18 smithi173 ceph-mon[35306]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T08:39:18.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:18 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:39:18.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:18 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:39:18.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:18 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:39:18.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:18 smithi173 ceph-mon[35306]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:18.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:18 smithi173 ceph-mon[35306]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T08:39:18.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:18 smithi173 ceph-mon[35306]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T08:39:18.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:18 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:39:18.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:18 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:39:18.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:18 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:39:18.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:18 smithi173 ceph-mon[35306]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T08:39:20.862 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:20 smithi164 ceph-mon[30928]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:21.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:20 smithi170 ceph-mon[35072]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:21.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:20 smithi173 ceph-mon[35306]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:23.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:22 smithi164 ceph-mon[30928]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:23.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:22 smithi170 ceph-mon[35072]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:23.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:22 smithi173 ceph-mon[35306]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:24.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:23 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T08:39:24.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:23 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:39:24.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:23 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T08:39:24.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:23 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:39:24.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:23 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T08:39:24.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:23 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:39:24.869 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:24 smithi164 ceph-mon[30928]: Deploying daemon osd.3 on smithi164 2024-09-19T08:39:24.869 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:24 smithi164 ceph-mon[30928]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:25.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:24 smithi170 ceph-mon[35072]: Deploying daemon osd.3 on smithi164 2024-09-19T08:39:25.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:24 smithi170 ceph-mon[35072]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:25.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:24 smithi173 ceph-mon[35306]: Deploying daemon osd.3 on smithi164 2024-09-19T08:39:25.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:24 smithi173 ceph-mon[35306]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:26.890 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:26 smithi164 ceph-mon[30928]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:27.128 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:26 smithi170 ceph-mon[35072]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:27.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:26 smithi173 ceph-mon[35306]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:29.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:28 smithi164 ceph-mon[30928]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:29.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:28 smithi170 ceph-mon[35072]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:29.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:28 smithi173 ceph-mon[35306]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:30.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:29 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:39:30.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:29 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:30.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:29 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:30.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:29 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:39:30.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:29 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:30.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:29 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:30.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:29 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:39:30.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:29 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:30.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:29 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:31.017 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:30 smithi164 ceph-mon[30928]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:31.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:30 smithi170 ceph-mon[35072]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:31.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:30 smithi173 ceph-mon[35306]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:33.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:32 smithi164 ceph-mon[30928]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:33.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:32 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:33.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:32 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:33.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:32 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:39:33.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:32 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:39:33.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:32 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:33.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:32 smithi170 ceph-mon[35072]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:33.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:32 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:33.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:32 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:33.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:32 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:39:33.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:32 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:39:33.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:32 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:33.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:32 smithi173 ceph-mon[35306]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:33.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:32 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:33.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:32 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:33.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:32 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:39:33.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:32 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:39:33.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:32 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:33.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:33.548+0000 7f163f7fe640 1 -- 172.21.15.164:0/915488222 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1650062630 con 0x7f16380a31e0 2024-09-19T08:39:33.548 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 3 on host 'smithi164' 2024-09-19T08:39:33.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:33.550+0000 7f1654c3d640 1 -- 172.21.15.164:0/915488222 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f16380a31e0 msgr2=0x7f16380a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:33.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:33.550+0000 7f1654c3d640 1 --2- 172.21.15.164:0/915488222 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f16380a31e0 0x7f16380a56a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1644031610 tx=0x7f1644000bc0 comp rx=0 tx=0).stop 2024-09-19T08:39:33.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:33.550+0000 7f1654c3d640 1 -- 172.21.15.164:0/915488222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1650102e00 msgr2=0x7f1650198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:33.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:33.550+0000 7f1654c3d640 1 --2- 172.21.15.164:0/915488222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1650102e00 0x7f1650198440 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f164000d950 tx=0x7f164000de10 comp rx=0 tx=0).stop 2024-09-19T08:39:33.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:33.551+0000 7f1654c3d640 1 -- 172.21.15.164:0/915488222 shutdown_connections 2024-09-19T08:39:33.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:33.551+0000 7f1654c3d640 1 --2- 172.21.15.164:0/915488222 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f16380a31e0 0x7f16380a56a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:33.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:33.551+0000 7f1654c3d640 1 --2- 172.21.15.164:0/915488222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16501037c0 0x7f165019f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:33.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:33.551+0000 7f1654c3d640 1 --2- 172.21.15.164:0/915488222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1650102e00 0x7f1650198440 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:33.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:33.551+0000 7f1654c3d640 1 --2- 172.21.15.164:0/915488222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1650101c00 0x7f1650197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:33.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:33.551+0000 7f1654c3d640 1 -- 172.21.15.164:0/915488222 >> 172.21.15.164:0/915488222 conn(0x7f16500fd390 msgr2=0x7f16501061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:33.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:33.551+0000 7f1654c3d640 1 -- 172.21.15.164:0/915488222 shutdown_connections 2024-09-19T08:39:33.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:39:33.551+0000 7f1654c3d640 1 -- 172.21.15.164:0/915488222 wait complete. 2024-09-19T08:39:34.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:33 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:39:34.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:33 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:34.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:33 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:34.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:33 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:39:34.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:33 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:34.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:33 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:34.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:39:34.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:34.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:34.250 DEBUG:teuthology.orchestra.run.smithi164:osd.3> sudo journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.3.service 2024-09-19T08:39:34.253 INFO:tasks.cephadm:Deploying osd.4 on smithi170 with /dev/vg_nvme/lv_3... 2024-09-19T08:39:34.253 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T08:39:34.510 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T08:39:34.954 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:34 smithi170 ceph-mon[35072]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:35.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:34 smithi164 ceph-mon[30928]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:35.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:34 smithi173 ceph-mon[35306]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:36.282 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 19 08:39:36 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-3[51564]: 2024-09-19T08:39:36.003+0000 7f076f82f640 -1 osd.3 0 log_to_monitors true 2024-09-19T08:39:36.988 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:36 smithi164 ceph-mon[30928]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:36.988 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:36 smithi164 ceph-mon[30928]: from='osd.3 [v2:172.21.15.164:6810/406795560,v1:172.21.15.164:6811/406795560]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T08:39:37.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:36 smithi170 ceph-mon[35072]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:37.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:36 smithi170 ceph-mon[35072]: from='osd.3 [v2:172.21.15.164:6810/406795560,v1:172.21.15.164:6811/406795560]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T08:39:37.136 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-19T08:39:37.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:36 smithi173 ceph-mon[35306]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:37.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:36 smithi173 ceph-mon[35306]: from='osd.3 [v2:172.21.15.164:6810/406795560,v1:172.21.15.164:6811/406795560]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T08:39:37.162 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi170:vg_nvme/lv_3 2024-09-19T08:39:37.410 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T08:39:37.811 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:37 smithi170 ceph-mon[35072]: from='osd.3 [v2:172.21.15.164:6810/406795560,v1:172.21.15.164:6811/406795560]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T08:39:37.811 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:37 smithi170 ceph-mon[35072]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T08:39:37.811 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:37 smithi170 ceph-mon[35072]: from='osd.3 [v2:172.21.15.164:6810/406795560,v1:172.21.15.164:6811/406795560]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-19T08:39:37.811 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:37 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:39:37.812 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:37 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:37.812 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:37 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:37.812 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:37 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:39:37.812 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:37 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:39:37.812 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:37 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:37.812 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:37 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:39:37.812 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:37 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:39:37.812 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:37 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:37.812 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:37 smithi170 ceph-mon[35072]: from='osd.3 [v2:172.21.15.164:6810/406795560,v1:172.21.15.164:6811/406795560]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-19T08:39:37.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.893+0000 7f84ef3d9640 1 -- 172.21.15.170:0/2829229846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84e80fff30 msgr2=0x7f84e8100330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:37.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.893+0000 7f84ef3d9640 1 --2- 172.21.15.170:0/2829229846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84e80fff30 0x7f84e8100330 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f84d8009cd0 tx=0x7f84d802a8f0 comp rx=0 tx=0).stop 2024-09-19T08:39:37.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.894+0000 7f84ef3d9640 1 -- 172.21.15.170:0/2829229846 shutdown_connections 2024-09-19T08:39:37.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.894+0000 7f84ef3d9640 1 --2- 172.21.15.170:0/2829229846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84e8103970 0x7f84e810a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:37.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.894+0000 7f84ef3d9640 1 --2- 172.21.15.170:0/2829229846 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84e8102fb0 0x7f84e8103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:37.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.894+0000 7f84ef3d9640 1 --2- 172.21.15.170:0/2829229846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84e80fff30 0x7f84e8100330 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:37.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.894+0000 7f84ef3d9640 1 -- 172.21.15.170:0/2829229846 >> 172.21.15.170:0/2829229846 conn(0x7f84e80fba80 msgr2=0x7f84e80fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:37.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.894+0000 7f84ef3d9640 1 -- 172.21.15.170:0/2829229846 shutdown_connections 2024-09-19T08:39:37.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.894+0000 7f84ef3d9640 1 -- 172.21.15.170:0/2829229846 wait complete. 2024-09-19T08:39:37.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.894+0000 7f84ef3d9640 1 Processor -- start 2024-09-19T08:39:37.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.894+0000 7f84ef3d9640 1 -- start start 2024-09-19T08:39:37.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.894+0000 7f84ef3d9640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84e80fff30 0x7f84e8102910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:37.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.894+0000 7f84ef3d9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84e8102fb0 0x7f84e8100f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:37.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.894+0000 7f84ef3d9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84e8103970 0x7f84e81014a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:37.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.894+0000 7f84ef3d9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f84e810d880 con 0x7f84e8103970 2024-09-19T08:39:37.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.894+0000 7f84ef3d9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f84e810d700 con 0x7f84e8102fb0 2024-09-19T08:39:37.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.894+0000 7f84ef3d9640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f84e810da00 con 0x7f84e80fff30 2024-09-19T08:39:37.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.895+0000 7f84ec974640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84e8102fb0 0x7f84e8100f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:37.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.895+0000 7f84ec974640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84e8102fb0 0x7f84e8100f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59328/0 (socket says 172.21.15.170:59328) 2024-09-19T08:39:37.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.895+0000 7f84ec974640 1 -- 172.21.15.170:0/3805684052 learned_addr learned my addr 172.21.15.170:0/3805684052 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-19T08:39:37.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.895+0000 7f84ed175640 1 --2- 172.21.15.170:0/3805684052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84e80fff30 0x7f84e8102910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:37.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.895+0000 7f84ed976640 1 --2- 172.21.15.170:0/3805684052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84e8103970 0x7f84e81014a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:37.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.895+0000 7f84ec974640 1 -- 172.21.15.170:0/3805684052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84e80fff30 msgr2=0x7f84e8102910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:37.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.895+0000 7f84ec974640 1 --2- 172.21.15.170:0/3805684052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84e80fff30 0x7f84e8102910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:37.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.895+0000 7f84ec974640 1 -- 172.21.15.170:0/3805684052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84e8103970 msgr2=0x7f84e81014a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:37.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.895+0000 7f84ec974640 1 --2- 172.21.15.170:0/3805684052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84e8103970 0x7f84e81014a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:37.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.895+0000 7f84ec974640 1 -- 172.21.15.170:0/3805684052 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e8101d60 con 0x7f84e8102fb0 2024-09-19T08:39:37.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.895+0000 7f84ec974640 1 --2- 172.21.15.170:0/3805684052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84e8102fb0 0x7f84e8100f60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f84dc00d770 tx=0x7f84dc00dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:37.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.895+0000 7f84e67fc640 1 -- 172.21.15.170:0/3805684052 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84dc017070 con 0x7f84e8102fb0 2024-09-19T08:39:37.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.895+0000 7f84e67fc640 1 -- 172.21.15.170:0/3805684052 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84dc005500 con 0x7f84e8102fb0 2024-09-19T08:39:37.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.895+0000 7f84ef3d9640 1 -- 172.21.15.170:0/3805684052 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e81a20d0 con 0x7f84e8102fb0 2024-09-19T08:39:37.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.895+0000 7f84e67fc640 1 -- 172.21.15.170:0/3805684052 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84dc004e00 con 0x7f84e8102fb0 2024-09-19T08:39:37.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.895+0000 7f84ef3d9640 1 -- 172.21.15.170:0/3805684052 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e81a2590 con 0x7f84e8102fb0 2024-09-19T08:39:37.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.896+0000 7f84ef3d9640 1 -- 172.21.15.170:0/3805684052 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84e81091d0 con 0x7f84e8102fb0 2024-09-19T08:39:37.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.897+0000 7f84e67fc640 1 -- 172.21.15.170:0/3805684052 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f84dc004fa0 con 0x7f84e8102fb0 2024-09-19T08:39:37.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.898+0000 7f84e67fc640 1 --2- 172.21.15.170:0/3805684052 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f84d00a3240 0x7f84d00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:37.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.898+0000 7f84e67fc640 1 -- 172.21.15.170:0/3805684052 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 3389+0+0 (secure 0 0 0) 0x7f84dc0ce030 con 0x7f84e8102fb0 2024-09-19T08:39:37.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.898+0000 7f84ed175640 1 --2- 172.21.15.170:0/3805684052 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f84d00a3240 0x7f84d00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:37.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.899+0000 7f84ed175640 1 --2- 172.21.15.170:0/3805684052 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f84d00a3240 0x7f84d00a5700 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f84d8009ca0 tx=0x7f84d8031000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:37.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:37.903+0000 7f84e67fc640 1 -- 172.21.15.170:0/3805684052 <== mon.2 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f84dc09c090 con 0x7f84e8102fb0 2024-09-19T08:39:38.032 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 19 08:39:37 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-3[51564]: 2024-09-19T08:39:37.776+0000 7f076b7b2640 -1 osd.3 0 waiting for initial osdmap 2024-09-19T08:39:38.032 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 19 08:39:37 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-3[51564]: 2024-09-19T08:39:37.782+0000 7f07675da640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:39:38.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:37 smithi164 ceph-mon[30928]: from='osd.3 [v2:172.21.15.164:6810/406795560,v1:172.21.15.164:6811/406795560]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T08:39:38.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:37 smithi164 ceph-mon[30928]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T08:39:38.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:37 smithi164 ceph-mon[30928]: from='osd.3 [v2:172.21.15.164:6810/406795560,v1:172.21.15.164:6811/406795560]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-19T08:39:38.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:37 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:39:38.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:37 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:38.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:37 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:38.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:37 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:39:38.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:37 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:39:38.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:37 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:38.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:37 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:39:38.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:37 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:39:38.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:37 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:38.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:37 smithi164 ceph-mon[30928]: from='osd.3 [v2:172.21.15.164:6810/406795560,v1:172.21.15.164:6811/406795560]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-19T08:39:38.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:38.078+0000 7f84ef3d9640 1 -- 172.21.15.170:0/3805684052 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f84e8062630 con 0x7f84d00a3240 2024-09-19T08:39:38.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:37 smithi173 ceph-mon[35306]: from='osd.3 [v2:172.21.15.164:6810/406795560,v1:172.21.15.164:6811/406795560]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T08:39:38.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:37 smithi173 ceph-mon[35306]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T08:39:38.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:37 smithi173 ceph-mon[35306]: from='osd.3 [v2:172.21.15.164:6810/406795560,v1:172.21.15.164:6811/406795560]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-19T08:39:38.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:37 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:39:38.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:37 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:38.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:37 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:38.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:37 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:39:38.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:37 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:39:38.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:37 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:38.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:37 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:39:38.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:37 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:39:38.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:37 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:38.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:37 smithi173 ceph-mon[35306]: from='osd.3 [v2:172.21.15.164:6810/406795560,v1:172.21.15.164:6811/406795560]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-19T08:39:39.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:38 smithi164 ceph-mon[30928]: Detected new or changed devices on smithi164 2024-09-19T08:39:39.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:38 smithi164 ceph-mon[30928]: Adjusting osd_memory_target on smithi164 to 8544M 2024-09-19T08:39:39.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:38 smithi164 ceph-mon[30928]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:39.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:38 smithi164 ceph-mon[30928]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T08:39:39.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:38 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:39:39.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:38 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:39:39.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:38 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:39:39.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:38 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:39:39.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:38 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:39:39.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:38 smithi164 ceph-mon[30928]: osd.3 [v2:172.21.15.164:6810/406795560,v1:172.21.15.164:6811/406795560] boot 2024-09-19T08:39:39.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:38 smithi164 ceph-mon[30928]: osdmap e24: 4 total, 4 up, 4 in 2024-09-19T08:39:39.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:38 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:39:39.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:38 smithi170 ceph-mon[35072]: Detected new or changed devices on smithi164 2024-09-19T08:39:39.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:38 smithi170 ceph-mon[35072]: Adjusting osd_memory_target on smithi164 to 8544M 2024-09-19T08:39:39.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:38 smithi170 ceph-mon[35072]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:39.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:38 smithi170 ceph-mon[35072]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T08:39:39.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:38 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:39:39.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:38 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:39:39.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:38 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:39:39.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:38 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:39:39.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:38 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:39:39.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:38 smithi170 ceph-mon[35072]: osd.3 [v2:172.21.15.164:6810/406795560,v1:172.21.15.164:6811/406795560] boot 2024-09-19T08:39:39.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:38 smithi170 ceph-mon[35072]: osdmap e24: 4 total, 4 up, 4 in 2024-09-19T08:39:39.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:38 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:39:39.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:38 smithi173 ceph-mon[35306]: Detected new or changed devices on smithi164 2024-09-19T08:39:39.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:38 smithi173 ceph-mon[35306]: Adjusting osd_memory_target on smithi164 to 8544M 2024-09-19T08:39:39.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:38 smithi173 ceph-mon[35306]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-19T08:39:39.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:38 smithi173 ceph-mon[35306]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T08:39:39.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:38 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:39:39.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:38 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:39:39.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:38 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:39:39.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:38 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:39:39.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:38 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:39:39.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:38 smithi173 ceph-mon[35306]: osd.3 [v2:172.21.15.164:6810/406795560,v1:172.21.15.164:6811/406795560] boot 2024-09-19T08:39:39.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:38 smithi173 ceph-mon[35306]: osdmap e24: 4 total, 4 up, 4 in 2024-09-19T08:39:39.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:38 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:39:40.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:39 smithi164 ceph-mon[30928]: purged_snaps scrub starts 2024-09-19T08:39:40.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:39 smithi164 ceph-mon[30928]: purged_snaps scrub ok 2024-09-19T08:39:40.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:39 smithi164 ceph-mon[30928]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:39:40.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:39 smithi164 ceph-mon[30928]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T08:39:40.062 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:39 smithi173 ceph-mon[35306]: purged_snaps scrub starts 2024-09-19T08:39:40.062 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:39 smithi173 ceph-mon[35306]: purged_snaps scrub ok 2024-09-19T08:39:40.062 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:39 smithi173 ceph-mon[35306]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:39:40.062 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:39 smithi173 ceph-mon[35306]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T08:39:40.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:39 smithi170 ceph-mon[35072]: purged_snaps scrub starts 2024-09-19T08:39:40.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:39 smithi170 ceph-mon[35072]: purged_snaps scrub ok 2024-09-19T08:39:40.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:39 smithi170 ceph-mon[35072]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:39:40.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:39 smithi170 ceph-mon[35072]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T08:39:41.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:40 smithi164 ceph-mon[30928]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:39:41.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:40 smithi164 ceph-mon[30928]: from='client.? 172.21.15.170:0/1826336400' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6dca048a-2860-4a02-8d8e-450b30bc1a3c"}]: dispatch 2024-09-19T08:39:41.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:40 smithi164 ceph-mon[30928]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6dca048a-2860-4a02-8d8e-450b30bc1a3c"}]: dispatch 2024-09-19T08:39:41.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:40 smithi164 ceph-mon[30928]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6dca048a-2860-4a02-8d8e-450b30bc1a3c"}]': finished 2024-09-19T08:39:41.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:40 smithi164 ceph-mon[30928]: osdmap e26: 5 total, 4 up, 5 in 2024-09-19T08:39:41.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:40 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:39:41.081 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:40 smithi170 ceph-mon[35072]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:39:41.082 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:40 smithi170 ceph-mon[35072]: from='client.? 172.21.15.170:0/1826336400' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6dca048a-2860-4a02-8d8e-450b30bc1a3c"}]: dispatch 2024-09-19T08:39:41.082 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:40 smithi170 ceph-mon[35072]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6dca048a-2860-4a02-8d8e-450b30bc1a3c"}]: dispatch 2024-09-19T08:39:41.082 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:40 smithi170 ceph-mon[35072]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6dca048a-2860-4a02-8d8e-450b30bc1a3c"}]': finished 2024-09-19T08:39:41.082 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:40 smithi170 ceph-mon[35072]: osdmap e26: 5 total, 4 up, 5 in 2024-09-19T08:39:41.082 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:40 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:39:41.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:40 smithi173 ceph-mon[35306]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:39:41.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:40 smithi173 ceph-mon[35306]: from='client.? 172.21.15.170:0/1826336400' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6dca048a-2860-4a02-8d8e-450b30bc1a3c"}]: dispatch 2024-09-19T08:39:41.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:40 smithi173 ceph-mon[35306]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6dca048a-2860-4a02-8d8e-450b30bc1a3c"}]: dispatch 2024-09-19T08:39:41.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:40 smithi173 ceph-mon[35306]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6dca048a-2860-4a02-8d8e-450b30bc1a3c"}]': finished 2024-09-19T08:39:41.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:40 smithi173 ceph-mon[35306]: osdmap e26: 5 total, 4 up, 5 in 2024-09-19T08:39:41.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:40 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:39:42.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:41 smithi164 ceph-mon[30928]: from='client.? 172.21.15.170:0/2623058415' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:39:42.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:41 smithi164 ceph-mon[30928]: osdmap e27: 5 total, 4 up, 5 in 2024-09-19T08:39:42.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:41 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:39:42.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:41 smithi170 ceph-mon[35072]: from='client.? 172.21.15.170:0/2623058415' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:39:42.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:41 smithi170 ceph-mon[35072]: osdmap e27: 5 total, 4 up, 5 in 2024-09-19T08:39:42.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:41 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:39:42.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:41 smithi173 ceph-mon[35306]: from='client.? 172.21.15.170:0/2623058415' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:39:42.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:41 smithi173 ceph-mon[35306]: osdmap e27: 5 total, 4 up, 5 in 2024-09-19T08:39:42.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:41 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:39:43.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:42 smithi170 ceph-mon[35072]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:39:43.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:42 smithi170 ceph-mon[35072]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T08:39:43.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:42 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:39:43.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:42 smithi173 ceph-mon[35306]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:39:43.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:42 smithi173 ceph-mon[35306]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T08:39:43.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:42 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:39:43.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:42 smithi164 ceph-mon[30928]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:39:43.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:42 smithi164 ceph-mon[30928]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T08:39:43.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:42 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:39:45.072 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:44 smithi170 ceph-mon[35072]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-09-19T08:39:45.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:44 smithi173 ceph-mon[35306]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-09-19T08:39:45.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:44 smithi164 ceph-mon[30928]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-09-19T08:39:47.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:46 smithi170 ceph-mon[35072]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T08:39:47.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:46 smithi173 ceph-mon[35306]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T08:39:47.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:46 smithi164 ceph-mon[30928]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T08:39:49.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:48 smithi170 ceph-mon[35072]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-19T08:39:49.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:48 smithi173 ceph-mon[35306]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-19T08:39:49.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:48 smithi164 ceph-mon[30928]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-19T08:39:49.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:49 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T08:39:49.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:49 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:39:50.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:49 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T08:39:50.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:49 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:39:50.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:49 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T08:39:50.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:49 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:39:51.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:50 smithi170 ceph-mon[35072]: Deploying daemon osd.4 on smithi170 2024-09-19T08:39:51.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:50 smithi170 ceph-mon[35072]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-19T08:39:51.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:50 smithi173 ceph-mon[35306]: Deploying daemon osd.4 on smithi170 2024-09-19T08:39:51.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:50 smithi173 ceph-mon[35306]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-19T08:39:51.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:50 smithi164 ceph-mon[30928]: Deploying daemon osd.4 on smithi170 2024-09-19T08:39:51.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:50 smithi164 ceph-mon[30928]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-19T08:39:53.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:52 smithi170 ceph-mon[35072]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T08:39:53.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:52 smithi173 ceph-mon[35306]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T08:39:53.281 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:52 smithi164 ceph-mon[30928]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T08:39:55.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:54 smithi170 ceph-mon[35072]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-19T08:39:55.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:54 smithi173 ceph-mon[35306]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-19T08:39:55.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:54 smithi164 ceph-mon[30928]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-19T08:39:56.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:55 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:39:56.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:55 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:56.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:55 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:56.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:55 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:39:56.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:55 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:56.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:55 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:56.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:55 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:39:56.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:55 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:56.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:55 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:57.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:56 smithi170 ceph-mon[35072]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:39:57.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:56 smithi173 ceph-mon[35306]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:39:57.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:56 smithi164 ceph-mon[30928]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:39:58.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:57 smithi170 ceph-mon[35072]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:39:58.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:57 smithi173 ceph-mon[35306]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:39:58.281 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:57 smithi164 ceph-mon[30928]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:39:59.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:58 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:59.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:58 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:59.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:58 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:39:59.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:58 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:39:59.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:58 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:59.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:59.201+0000 7f84e67fc640 1 -- 172.21.15.170:0/3805684052 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f84e8062630 con 0x7f84d00a3240 2024-09-19T08:39:59.201 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 4 on host 'smithi170' 2024-09-19T08:39:59.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:59.203+0000 7f84ef3d9640 1 -- 172.21.15.170:0/3805684052 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f84d00a3240 msgr2=0x7f84d00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:59.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:59.203+0000 7f84ef3d9640 1 --2- 172.21.15.170:0/3805684052 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f84d00a3240 0x7f84d00a5700 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f84d8009ca0 tx=0x7f84d8031000 comp rx=0 tx=0).stop 2024-09-19T08:39:59.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:59.203+0000 7f84ef3d9640 1 -- 172.21.15.170:0/3805684052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84e8102fb0 msgr2=0x7f84e8100f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:59.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:59.203+0000 7f84ef3d9640 1 --2- 172.21.15.170:0/3805684052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84e8102fb0 0x7f84e8100f60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f84dc00d770 tx=0x7f84dc00dc30 comp rx=0 tx=0).stop 2024-09-19T08:39:59.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:59.203+0000 7f84ef3d9640 1 -- 172.21.15.170:0/3805684052 shutdown_connections 2024-09-19T08:39:59.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:59.203+0000 7f84ef3d9640 1 --2- 172.21.15.170:0/3805684052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84e8103970 0x7f84e81014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:59.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:59.203+0000 7f84ef3d9640 1 --2- 172.21.15.170:0/3805684052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84e8102fb0 0x7f84e8100f60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:59.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:59.203+0000 7f84ef3d9640 1 --2- 172.21.15.170:0/3805684052 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f84d00a3240 0x7f84d00a5700 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:59.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:59.203+0000 7f84ef3d9640 1 --2- 172.21.15.170:0/3805684052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84e80fff30 0x7f84e8102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:59.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:59.203+0000 7f84ef3d9640 1 -- 172.21.15.170:0/3805684052 >> 172.21.15.170:0/3805684052 conn(0x7f84e80fba80 msgr2=0x7f84e81063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:59.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:59.203+0000 7f84ef3d9640 1 -- 172.21.15.170:0/3805684052 shutdown_connections 2024-09-19T08:39:59.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:39:59.203+0000 7f84ef3d9640 1 -- 172.21.15.170:0/3805684052 wait complete. 2024-09-19T08:39:59.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:58 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:59.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:58 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:59.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:58 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:39:59.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:58 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:39:59.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:58 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:59.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:58 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:59.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:58 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:59.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:58 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:39:59.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:58 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:39:59.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:58 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:39:59.832 DEBUG:teuthology.orchestra.run.smithi170:osd.4> sudo journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.4.service 2024-09-19T08:39:59.835 INFO:tasks.cephadm:Deploying osd.5 on smithi173 with /dev/vg_nvme/lv_3... 2024-09-19T08:39:59.835 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T08:40:00.055 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:59 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:40:00.055 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:59 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:00.056 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:59 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:00.056 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:39:59 smithi170 ceph-mon[35072]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:40:00.088 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:40:00.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:59 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:40:00.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:59 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:00.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:59 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:00.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:39:59 smithi173 ceph-mon[35306]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:40:00.203 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:59 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:40:00.203 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:59 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:00.204 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:59 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:00.204 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:39:59 smithi164 ceph-mon[30928]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:40:01.230 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:00 smithi173 ceph-mon[35306]: overall HEALTH_OK 2024-09-19T08:40:01.237 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:00 smithi170 ceph-mon[35072]: overall HEALTH_OK 2024-09-19T08:40:01.238 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 19 08:40:01 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-4[46203]: 2024-09-19T08:40:01.102+0000 7f12e5ade640 -1 osd.4 0 log_to_monitors true 2024-09-19T08:40:01.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:00 smithi164 ceph-mon[30928]: overall HEALTH_OK 2024-09-19T08:40:02.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:01 smithi173 ceph-mon[35306]: from='osd.4 [v2:172.21.15.170:6808/4120832165,v1:172.21.15.170:6809/4120832165]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T08:40:02.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:01 smithi173 ceph-mon[35306]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T08:40:02.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:01 smithi173 ceph-mon[35306]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:40:02.208 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:01 smithi170 ceph-mon[35072]: from='osd.4 [v2:172.21.15.170:6808/4120832165,v1:172.21.15.170:6809/4120832165]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T08:40:02.209 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:01 smithi170 ceph-mon[35072]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T08:40:02.209 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:01 smithi170 ceph-mon[35072]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:40:02.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:01 smithi164 ceph-mon[30928]: from='osd.4 [v2:172.21.15.170:6808/4120832165,v1:172.21.15.170:6809/4120832165]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T08:40:02.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:01 smithi164 ceph-mon[30928]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T08:40:02.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:01 smithi164 ceph-mon[30928]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:40:02.624 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-19T08:40:02.651 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi173:vg_nvme/lv_3 2024-09-19T08:40:02.899 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:40:03.253 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:02 smithi173 ceph-mon[35306]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T08:40:03.254 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:02 smithi173 ceph-mon[35306]: from='osd.4 [v2:172.21.15.170:6808/4120832165,v1:172.21.15.170:6809/4120832165]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-19T08:40:03.254 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:02 smithi173 ceph-mon[35306]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T08:40:03.254 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:02 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:40:03.254 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:02 smithi173 ceph-mon[35306]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-19T08:40:03.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:02 smithi164 ceph-mon[30928]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T08:40:03.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:02 smithi164 ceph-mon[30928]: from='osd.4 [v2:172.21.15.170:6808/4120832165,v1:172.21.15.170:6809/4120832165]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-19T08:40:03.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:02 smithi164 ceph-mon[30928]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T08:40:03.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:02 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:40:03.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:02 smithi164 ceph-mon[30928]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-19T08:40:03.375 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 19 08:40:02 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-4[46203]: 2024-09-19T08:40:02.965+0000 7f12e1a61640 -1 osd.4 0 waiting for initial osdmap 2024-09-19T08:40:03.375 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 19 08:40:02 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-4[46203]: 2024-09-19T08:40:02.973+0000 7f12dd889640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:40:03.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:02 smithi170 ceph-mon[35072]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T08:40:03.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:02 smithi170 ceph-mon[35072]: from='osd.4 [v2:172.21.15.170:6808/4120832165,v1:172.21.15.170:6809/4120832165]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-19T08:40:03.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:02 smithi170 ceph-mon[35072]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T08:40:03.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:02 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:40:03.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:02 smithi170 ceph-mon[35072]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-19T08:40:03.382 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.374+0000 7f4defa65640 1 -- 172.21.15.173:0/272078230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4de8102690 msgr2=0x7f4de8108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:03.382 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.374+0000 7f4defa65640 1 --2- 172.21.15.173:0/272078230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4de8102690 0x7f4de8108f20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009ca0 tx=0x7f4de002a8c0 comp rx=0 tx=0).stop 2024-09-19T08:40:03.382 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.375+0000 7f4defa65640 1 -- 172.21.15.173:0/272078230 shutdown_connections 2024-09-19T08:40:03.382 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.375+0000 7f4defa65640 1 --2- 172.21.15.173:0/272078230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4de8102690 0x7f4de8108f20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:03.382 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.375+0000 7f4defa65640 1 --2- 172.21.15.173:0/272078230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4de8101cd0 0x7f4de8102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:03.382 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.375+0000 7f4defa65640 1 --2- 172.21.15.173:0/272078230 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4de8100ad0 0x7f4de8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:03.382 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.375+0000 7f4defa65640 1 -- 172.21.15.173:0/272078230 >> 172.21.15.173:0/272078230 conn(0x7f4de80fc280 msgr2=0x7f4de80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:40:03.382 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.375+0000 7f4defa65640 1 -- 172.21.15.173:0/272078230 shutdown_connections 2024-09-19T08:40:03.383 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.375+0000 7f4defa65640 1 -- 172.21.15.173:0/272078230 wait complete. 2024-09-19T08:40:03.383 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.375+0000 7f4defa65640 1 Processor -- start 2024-09-19T08:40:03.383 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.375+0000 7f4defa65640 1 -- start start 2024-09-19T08:40:03.383 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.375+0000 7f4defa65640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4de8100ad0 0x7f4de8196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:03.383 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.375+0000 7f4defa65640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4de8101cd0 0x7f4de81972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:03.383 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.376+0000 7f4defa65640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4de8102690 0x7f4de819e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:03.383 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.376+0000 7f4defa65640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4de810c730 con 0x7f4de8100ad0 2024-09-19T08:40:03.383 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.376+0000 7f4defa65640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4de810c5b0 con 0x7f4de8101cd0 2024-09-19T08:40:03.383 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.376+0000 7f4defa65640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4de810c8b0 con 0x7f4de8102690 2024-09-19T08:40:03.383 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.376+0000 7f4dee002640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4de8102690 0x7f4de819e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:03.383 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.376+0000 7f4dee002640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4de8102690 0x7f4de819e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:55980/0 (socket says 172.21.15.173:55980) 2024-09-19T08:40:03.383 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.376+0000 7f4ded000640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4de8101cd0 0x7f4de81972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:03.383 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.376+0000 7f4dee002640 1 -- 172.21.15.173:0/231439527 learned_addr learned my addr 172.21.15.173:0/231439527 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-09-19T08:40:03.384 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.376+0000 7f4ded801640 1 --2- 172.21.15.173:0/231439527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4de8100ad0 0x7f4de8196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:03.384 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.376+0000 7f4dee002640 1 -- 172.21.15.173:0/231439527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4de8101cd0 msgr2=0x7f4de81972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:03.384 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.376+0000 7f4dee002640 1 --2- 172.21.15.173:0/231439527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4de8101cd0 0x7f4de81972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:03.384 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.376+0000 7f4dee002640 1 -- 172.21.15.173:0/231439527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4de8100ad0 msgr2=0x7f4de8196da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:03.384 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.376+0000 7f4dee002640 1 --2- 172.21.15.173:0/231439527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4de8100ad0 0x7f4de8196da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:03.384 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.376+0000 7f4dee002640 1 -- 172.21.15.173:0/231439527 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de819e9c0 con 0x7f4de8102690 2024-09-19T08:40:03.384 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.376+0000 7f4dee002640 1 --2- 172.21.15.173:0/231439527 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4de8102690 0x7f4de819e320 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4de0031ae0 tx=0x7f4de0031ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:03.384 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.376+0000 7f4dda7fc640 1 -- 172.21.15.173:0/231439527 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de0049070 con 0x7f4de8102690 2024-09-19T08:40:03.384 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.376+0000 7f4defa65640 1 -- 172.21.15.173:0/231439527 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4de8069f70 con 0x7f4de8102690 2024-09-19T08:40:03.384 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.376+0000 7f4dda7fc640 1 -- 172.21.15.173:0/231439527 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4de0003b80 con 0x7f4de8102690 2024-09-19T08:40:03.385 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.376+0000 7f4dda7fc640 1 -- 172.21.15.173:0/231439527 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de002fad0 con 0x7f4de8102690 2024-09-19T08:40:03.385 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.376+0000 7f4defa65640 1 -- 172.21.15.173:0/231439527 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4de806a500 con 0x7f4de8102690 2024-09-19T08:40:03.385 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.377+0000 7f4defa65640 1 -- 172.21.15.173:0/231439527 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4de81078e0 con 0x7f4de8102690 2024-09-19T08:40:03.385 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.378+0000 7f4dda7fc640 1 -- 172.21.15.173:0/231439527 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f4de004c070 con 0x7f4de8102690 2024-09-19T08:40:03.385 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.379+0000 7f4dda7fc640 1 --2- 172.21.15.173:0/231439527 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f4dd40a3240 0x7f4dd40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:03.385 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.379+0000 7f4dda7fc640 1 -- 172.21.15.173:0/231439527 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 3821+0+0 (secure 0 0 0) 0x7f4de00f0f00 con 0x7f4de8102690 2024-09-19T08:40:03.385 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.379+0000 7f4ded801640 1 --2- 172.21.15.173:0/231439527 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f4dd40a3240 0x7f4dd40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:03.385 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.380+0000 7f4ded801640 1 --2- 172.21.15.173:0/231439527 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f4dd40a3240 0x7f4dd40a5700 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4de8198280 tx=0x7f4de4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:03.386 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.383+0000 7f4dda7fc640 1 -- 172.21.15.173:0/231439527 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4de00bedb0 con 0x7f4de8102690 2024-09-19T08:40:03.556 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:03.556+0000 7f4defa65640 1 -- 172.21.15.173:0/231439527 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4de806a820 con 0x7f4dd40a3240 2024-09-19T08:40:04.215 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:03 smithi173 ceph-mon[35306]: Detected new or changed devices on smithi170 2024-09-19T08:40:04.216 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:03 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:04.216 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:03 smithi173 ceph-mon[35306]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-09-19T08:40:04.216 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:03 smithi173 ceph-mon[35306]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T08:40:04.216 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:03 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:40:04.216 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:03 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:04.216 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:03 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:40:04.216 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:03 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:40:04.216 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:03 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:40:04.216 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:03 smithi173 ceph-mon[35306]: Adjusting osd_memory_target on smithi170 to 8544M 2024-09-19T08:40:04.216 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:03 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:04.216 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:03 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:40:04.217 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:03 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:40:04.217 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:03 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:04.217 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:03 smithi173 ceph-mon[35306]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:40:04.217 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:03 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:40:04.217 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:03 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:40:04.217 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:03 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:40:04.217 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:03 smithi173 ceph-mon[35306]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:40:04.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:03 smithi164 ceph-mon[30928]: Detected new or changed devices on smithi170 2024-09-19T08:40:04.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:03 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:04.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:03 smithi164 ceph-mon[30928]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-09-19T08:40:04.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:03 smithi164 ceph-mon[30928]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T08:40:04.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:03 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:40:04.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:03 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:04.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:03 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:40:04.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:03 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:40:04.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:03 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:40:04.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:03 smithi164 ceph-mon[30928]: Adjusting osd_memory_target on smithi170 to 8544M 2024-09-19T08:40:04.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:03 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:04.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:03 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:40:04.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:03 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:40:04.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:03 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:04.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:03 smithi164 ceph-mon[30928]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:40:04.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:03 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:40:04.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:03 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:40:04.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:03 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:40:04.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:03 smithi164 ceph-mon[30928]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:40:04.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:03 smithi170 ceph-mon[35072]: Detected new or changed devices on smithi170 2024-09-19T08:40:04.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:03 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:04.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:03 smithi170 ceph-mon[35072]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-09-19T08:40:04.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:03 smithi170 ceph-mon[35072]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T08:40:04.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:03 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:40:04.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:03 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:04.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:03 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:40:04.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:03 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:40:04.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:03 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:40:04.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:03 smithi170 ceph-mon[35072]: Adjusting osd_memory_target on smithi170 to 8544M 2024-09-19T08:40:04.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:03 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:04.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:03 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:40:04.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:03 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:40:04.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:03 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:04.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:03 smithi170 ceph-mon[35072]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:40:04.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:03 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:40:04.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:03 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:40:04.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:03 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:40:04.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:03 smithi170 ceph-mon[35072]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-19T08:40:05.243 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:04 smithi173 ceph-mon[35306]: purged_snaps scrub starts 2024-09-19T08:40:05.243 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:04 smithi173 ceph-mon[35306]: purged_snaps scrub ok 2024-09-19T08:40:05.243 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:04 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:40:05.243 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:04 smithi173 ceph-mon[35306]: osd.4 [v2:172.21.15.170:6808/4120832165,v1:172.21.15.170:6809/4120832165] boot 2024-09-19T08:40:05.243 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:04 smithi173 ceph-mon[35306]: osdmap e31: 5 total, 5 up, 5 in 2024-09-19T08:40:05.244 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:04 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:40:05.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:04 smithi164 ceph-mon[30928]: purged_snaps scrub starts 2024-09-19T08:40:05.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:04 smithi164 ceph-mon[30928]: purged_snaps scrub ok 2024-09-19T08:40:05.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:04 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:40:05.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:04 smithi164 ceph-mon[30928]: osd.4 [v2:172.21.15.170:6808/4120832165,v1:172.21.15.170:6809/4120832165] boot 2024-09-19T08:40:05.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:04 smithi164 ceph-mon[30928]: osdmap e31: 5 total, 5 up, 5 in 2024-09-19T08:40:05.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:04 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:40:05.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:04 smithi170 ceph-mon[35072]: purged_snaps scrub starts 2024-09-19T08:40:05.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:04 smithi170 ceph-mon[35072]: purged_snaps scrub ok 2024-09-19T08:40:05.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:04 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:40:05.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:04 smithi170 ceph-mon[35072]: osd.4 [v2:172.21.15.170:6808/4120832165,v1:172.21.15.170:6809/4120832165] boot 2024-09-19T08:40:05.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:04 smithi170 ceph-mon[35072]: osdmap e31: 5 total, 5 up, 5 in 2024-09-19T08:40:05.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:04 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:40:06.236 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:05 smithi173 ceph-mon[35306]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T08:40:06.236 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:05 smithi173 ceph-mon[35306]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:06.236 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:05 smithi173 ceph-mon[35306]: from='client.? 172.21.15.173:0/2962430750' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3822521f-69d6-4141-a83f-5fc9cecc7b3e"}]: dispatch 2024-09-19T08:40:06.236 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:05 smithi173 ceph-mon[35306]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3822521f-69d6-4141-a83f-5fc9cecc7b3e"}]: dispatch 2024-09-19T08:40:06.236 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:05 smithi173 ceph-mon[35306]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3822521f-69d6-4141-a83f-5fc9cecc7b3e"}]': finished 2024-09-19T08:40:06.236 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:05 smithi173 ceph-mon[35306]: osdmap e33: 6 total, 5 up, 6 in 2024-09-19T08:40:06.236 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:05 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:40:06.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:05 smithi164 ceph-mon[30928]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T08:40:06.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:05 smithi164 ceph-mon[30928]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:06.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:05 smithi164 ceph-mon[30928]: from='client.? 172.21.15.173:0/2962430750' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3822521f-69d6-4141-a83f-5fc9cecc7b3e"}]: dispatch 2024-09-19T08:40:06.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:05 smithi164 ceph-mon[30928]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3822521f-69d6-4141-a83f-5fc9cecc7b3e"}]: dispatch 2024-09-19T08:40:06.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:05 smithi164 ceph-mon[30928]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3822521f-69d6-4141-a83f-5fc9cecc7b3e"}]': finished 2024-09-19T08:40:06.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:05 smithi164 ceph-mon[30928]: osdmap e33: 6 total, 5 up, 6 in 2024-09-19T08:40:06.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:05 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:40:06.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:05 smithi170 ceph-mon[35072]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T08:40:06.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:05 smithi170 ceph-mon[35072]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:06.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:05 smithi170 ceph-mon[35072]: from='client.? 172.21.15.173:0/2962430750' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3822521f-69d6-4141-a83f-5fc9cecc7b3e"}]: dispatch 2024-09-19T08:40:06.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:05 smithi170 ceph-mon[35072]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3822521f-69d6-4141-a83f-5fc9cecc7b3e"}]: dispatch 2024-09-19T08:40:06.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:05 smithi170 ceph-mon[35072]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3822521f-69d6-4141-a83f-5fc9cecc7b3e"}]': finished 2024-09-19T08:40:06.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:05 smithi170 ceph-mon[35072]: osdmap e33: 6 total, 5 up, 6 in 2024-09-19T08:40:06.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:05 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:40:07.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:06 smithi164 ceph-mon[30928]: from='client.? 172.21.15.173:0/1945189374' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:40:07.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:06 smithi170 ceph-mon[35072]: from='client.? 172.21.15.173:0/1945189374' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:40:07.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:06 smithi173 ceph-mon[35306]: from='client.? 172.21.15.173:0/1945189374' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:40:08.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:07 smithi164 ceph-mon[30928]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:08.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:07 smithi170 ceph-mon[35072]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:08.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:07 smithi173 ceph-mon[35306]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:11.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:10 smithi164 ceph-mon[30928]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:11.078 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:10 smithi170 ceph-mon[35072]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:11.114 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:10 smithi173 ceph-mon[35306]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:12.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:12 smithi173 ceph-mon[35306]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:13.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:12 smithi164 ceph-mon[30928]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:13.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:12 smithi170 ceph-mon[35072]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:14.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:14 smithi173 ceph-mon[35306]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:14.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:14 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T08:40:14.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:14 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:40:15.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:14 smithi164 ceph-mon[30928]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:15.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:14 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T08:40:15.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:14 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:40:15.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:14 smithi170 ceph-mon[35072]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:15.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:14 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T08:40:15.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:14 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:40:16.020 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:15 smithi173 ceph-mon[35306]: Deploying daemon osd.5 on smithi173 2024-09-19T08:40:16.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:15 smithi164 ceph-mon[30928]: Deploying daemon osd.5 on smithi173 2024-09-19T08:40:16.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:15 smithi170 ceph-mon[35072]: Deploying daemon osd.5 on smithi173 2024-09-19T08:40:16.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:16 smithi173 ceph-mon[35306]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:17.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:16 smithi164 ceph-mon[30928]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:17.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:16 smithi170 ceph-mon[35072]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:19.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:18 smithi164 ceph-mon[30928]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:19.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:18 smithi170 ceph-mon[35072]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:19.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:18 smithi173 ceph-mon[35306]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:20.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:20 smithi173 ceph-mon[35306]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:21.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:20 smithi164 ceph-mon[30928]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:21.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:20 smithi170 ceph-mon[35072]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:22.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:21 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:40:22.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:21 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:22.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:21 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:22.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:21 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:40:22.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:21 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:22.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:21 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:22.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:21 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:40:22.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:21 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:22.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:21 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:23.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:22 smithi164 ceph-mon[30928]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:23.069 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:22 smithi173 ceph-mon[35306]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:23.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:22 smithi170 ceph-mon[35072]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:24.980 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:24.980+0000 7f4dda7fc640 1 -- 172.21.15.173:0/231439527 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4de806a820 con 0x7f4dd40a3240 2024-09-19T08:40:24.980 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 5 on host 'smithi173' 2024-09-19T08:40:24.980 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:24 smithi173 ceph-mon[35306]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:24.980 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:24 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:24.980 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:24 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:24.981 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:24 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:40:24.981 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:24 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:40:24.981 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:24 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:24.982 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:24.982+0000 7f4defa65640 1 -- 172.21.15.173:0/231439527 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f4dd40a3240 msgr2=0x7f4dd40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:24.982 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:24.982+0000 7f4defa65640 1 --2- 172.21.15.173:0/231439527 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f4dd40a3240 0x7f4dd40a5700 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4de8198280 tx=0x7f4de4005f90 comp rx=0 tx=0).stop 2024-09-19T08:40:24.982 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:24.982+0000 7f4defa65640 1 -- 172.21.15.173:0/231439527 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4de8102690 msgr2=0x7f4de819e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:24.982 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:24.982+0000 7f4defa65640 1 --2- 172.21.15.173:0/231439527 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4de8102690 0x7f4de819e320 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4de0031ae0 tx=0x7f4de0031ee0 comp rx=0 tx=0).stop 2024-09-19T08:40:24.982 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:24.982+0000 7f4defa65640 1 -- 172.21.15.173:0/231439527 shutdown_connections 2024-09-19T08:40:24.982 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:24.982+0000 7f4defa65640 1 --2- 172.21.15.173:0/231439527 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f4dd40a3240 0x7f4dd40a5700 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:24.982 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:24.982+0000 7f4defa65640 1 --2- 172.21.15.173:0/231439527 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4de8102690 0x7f4de819e320 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:24.982 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:24.982+0000 7f4defa65640 1 --2- 172.21.15.173:0/231439527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4de8101cd0 0x7f4de81972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:24.982 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:24.982+0000 7f4defa65640 1 --2- 172.21.15.173:0/231439527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4de8100ad0 0x7f4de8196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:24.982 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:24.982+0000 7f4defa65640 1 -- 172.21.15.173:0/231439527 >> 172.21.15.173:0/231439527 conn(0x7f4de80fc280 msgr2=0x7f4de8105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:40:24.983 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:24.982+0000 7f4defa65640 1 -- 172.21.15.173:0/231439527 shutdown_connections 2024-09-19T08:40:24.983 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:40:24.982+0000 7f4defa65640 1 -- 172.21.15.173:0/231439527 wait complete. 2024-09-19T08:40:25.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:24 smithi164 ceph-mon[30928]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:25.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:24 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:25.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:24 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:25.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:24 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:40:25.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:24 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:40:25.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:24 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:25.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:24 smithi170 ceph-mon[35072]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:25.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:24 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:25.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:24 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:25.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:24 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:40:25.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:24 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:40:25.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:24 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:25.631 DEBUG:teuthology.orchestra.run.smithi173:osd.5> sudo journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.5.service 2024-09-19T08:40:25.634 INFO:tasks.cephadm:Deploying osd.6 on smithi164 with /dev/vg_nvme/lv_2... 2024-09-19T08:40:25.634 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T08:40:25.892 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:40:25.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:25 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:40:25.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:25 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:25.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:25 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:26.013 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:25 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:40:26.013 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:25 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:26.013 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:25 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:26.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:25 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:40:26.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:25 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:26.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:25 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:26.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:26 smithi173 ceph-mon[35306]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:27.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:26 smithi164 ceph-mon[30928]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:27.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:26 smithi170 ceph-mon[35072]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:27.177 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:40:26 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:40:26.916+0000 7fea008bc640 -1 osd.5 0 log_to_monitors true 2024-09-19T08:40:28.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:27 smithi164 ceph-mon[30928]: from='osd.5 [v2:172.21.15.173:6808/2914653552,v1:172.21.15.173:6809/2914653552]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T08:40:28.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:27 smithi164 ceph-mon[30928]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T08:40:28.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:27 smithi170 ceph-mon[35072]: from='osd.5 [v2:172.21.15.173:6808/2914653552,v1:172.21.15.173:6809/2914653552]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T08:40:28.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:27 smithi170 ceph-mon[35072]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T08:40:28.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:27 smithi173 ceph-mon[35306]: from='osd.5 [v2:172.21.15.173:6808/2914653552,v1:172.21.15.173:6809/2914653552]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T08:40:28.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:27 smithi173 ceph-mon[35306]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T08:40:28.470 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:40:28.497 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi164:vg_nvme/lv_2 2024-09-19T08:40:28.760 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:40:28.913 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:28 smithi173 ceph-mon[35306]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:28.913 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:28 smithi173 ceph-mon[35306]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T08:40:28.913 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:28 smithi173 ceph-mon[35306]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T08:40:28.913 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:28 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:40:28.913 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:28 smithi173 ceph-mon[35306]: from='osd.5 [v2:172.21.15.173:6808/2914653552,v1:172.21.15.173:6809/2914653552]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-19T08:40:28.913 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:28 smithi173 ceph-mon[35306]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-19T08:40:28.913 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:28 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:28.913 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:28 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:28.913 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:28 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:40:28.913 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:28 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:40:28.913 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:28 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:28.914 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:28 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:40:28.914 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:28 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:40:28.914 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:28 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:29.030 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:28 smithi164 ceph-mon[30928]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:29.030 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:28 smithi164 ceph-mon[30928]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T08:40:29.030 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:28 smithi164 ceph-mon[30928]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T08:40:29.030 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:28 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:40:29.030 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:28 smithi164 ceph-mon[30928]: from='osd.5 [v2:172.21.15.173:6808/2914653552,v1:172.21.15.173:6809/2914653552]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-19T08:40:29.030 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:28 smithi164 ceph-mon[30928]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-19T08:40:29.030 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:28 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:29.030 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:28 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:29.030 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:28 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:40:29.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:28 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:40:29.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:28 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:29.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:28 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:40:29.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:28 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:40:29.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:28 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:29.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:28 smithi170 ceph-mon[35072]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:29.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:28 smithi170 ceph-mon[35072]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T08:40:29.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:28 smithi170 ceph-mon[35072]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T08:40:29.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:28 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:40:29.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:28 smithi170 ceph-mon[35072]: from='osd.5 [v2:172.21.15.173:6808/2914653552,v1:172.21.15.173:6809/2914653552]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-19T08:40:29.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:28 smithi170 ceph-mon[35072]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-19T08:40:29.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:28 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:29.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:28 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:29.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:28 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:40:29.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:28 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:40:29.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:28 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:29.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:28 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:40:29.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:28 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:40:29.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:28 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:29.410 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:40:29 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:40:29.136+0000 7fe9fc83f640 -1 osd.5 0 waiting for initial osdmap 2024-09-19T08:40:29.411 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:40:29 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:40:29.144+0000 7fe9f465f640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:40:29.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.400+0000 7f667a4a0640 1 -- 172.21.15.164:0/4190142124 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6674102690 msgr2=0x7f6674108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:29.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.400+0000 7f667a4a0640 1 --2- 172.21.15.164:0/4190142124 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6674102690 0x7f6674108f20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f666c009ca0 tx=0x7f666c02a8c0 comp rx=0 tx=0).stop 2024-09-19T08:40:29.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.400+0000 7f667a4a0640 1 -- 172.21.15.164:0/4190142124 shutdown_connections 2024-09-19T08:40:29.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.400+0000 7f667a4a0640 1 --2- 172.21.15.164:0/4190142124 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6674102690 0x7f6674108f20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:29.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.400+0000 7f667a4a0640 1 --2- 172.21.15.164:0/4190142124 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6674101cd0 0x7f6674102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:29.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.400+0000 7f667a4a0640 1 --2- 172.21.15.164:0/4190142124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6674100ad0 0x7f6674100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:29.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.400+0000 7f667a4a0640 1 -- 172.21.15.164:0/4190142124 >> 172.21.15.164:0/4190142124 conn(0x7f66740fc280 msgr2=0x7f66740fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:40:29.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.401+0000 7f667a4a0640 1 -- 172.21.15.164:0/4190142124 shutdown_connections 2024-09-19T08:40:29.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.401+0000 7f667a4a0640 1 -- 172.21.15.164:0/4190142124 wait complete. 2024-09-19T08:40:29.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.401+0000 7f667a4a0640 1 Processor -- start 2024-09-19T08:40:29.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.401+0000 7f667a4a0640 1 -- start start 2024-09-19T08:40:29.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.401+0000 7f667a4a0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6674100ad0 0x7f667406aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:29.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.401+0000 7f667a4a0640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6674101cd0 0x7f667419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:29.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.401+0000 7f667a4a0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6674102690 0x7f667419f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:29.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.401+0000 7f667a4a0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f667410c750 con 0x7f6674100ad0 2024-09-19T08:40:29.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.401+0000 7f667a4a0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f667410c5d0 con 0x7f6674102690 2024-09-19T08:40:29.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.401+0000 7f667a4a0640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f667410c8d0 con 0x7f6674101cd0 2024-09-19T08:40:29.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.402+0000 7f6673fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6674100ad0 0x7f667406aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:29.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.402+0000 7f6673fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6674100ad0 0x7f667406aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47320/0 (socket says 172.21.15.164:47320) 2024-09-19T08:40:29.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.402+0000 7f6673fff640 1 -- 172.21.15.164:0/2821623953 learned_addr learned my addr 172.21.15.164:0/2821623953 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:40:29.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.402+0000 7f66737fe640 1 --2- 172.21.15.164:0/2821623953 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6674101cd0 0x7f667419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:29.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.402+0000 7f6678a3d640 1 --2- 172.21.15.164:0/2821623953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6674102690 0x7f667419f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:29.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.402+0000 7f6673fff640 1 -- 172.21.15.164:0/2821623953 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6674101cd0 msgr2=0x7f667419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:29.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.402+0000 7f6673fff640 1 --2- 172.21.15.164:0/2821623953 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6674101cd0 0x7f667419cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:29.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.402+0000 7f6673fff640 1 -- 172.21.15.164:0/2821623953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6674102690 msgr2=0x7f667419f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:29.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.402+0000 7f6673fff640 1 --2- 172.21.15.164:0/2821623953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6674102690 0x7f667419f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:29.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.402+0000 7f6673fff640 1 -- 172.21.15.164:0/2821623953 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667419fb00 con 0x7f6674100ad0 2024-09-19T08:40:29.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.402+0000 7f6678a3d640 1 --2- 172.21.15.164:0/2821623953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6674102690 0x7f667419f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:40:29.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.402+0000 7f6673fff640 1 --2- 172.21.15.164:0/2821623953 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6674100ad0 0x7f667406aad0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f666400d770 tx=0x7f666400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:29.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.402+0000 7f66717fa640 1 -- 172.21.15.164:0/2821623953 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664017070 con 0x7f6674100ad0 2024-09-19T08:40:29.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.402+0000 7f667a4a0640 1 -- 172.21.15.164:0/2821623953 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f667419fd90 con 0x7f6674100ad0 2024-09-19T08:40:29.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.402+0000 7f66717fa640 1 -- 172.21.15.164:0/2821623953 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6664009c30 con 0x7f6674100ad0 2024-09-19T08:40:29.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.402+0000 7f66717fa640 1 -- 172.21.15.164:0/2821623953 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664003dd0 con 0x7f6674100ad0 2024-09-19T08:40:29.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.402+0000 7f667a4a0640 1 -- 172.21.15.164:0/2821623953 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66741a0250 con 0x7f6674100ad0 2024-09-19T08:40:29.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.403+0000 7f667a4a0640 1 -- 172.21.15.164:0/2821623953 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6674107e50 con 0x7f6674100ad0 2024-09-19T08:40:29.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.405+0000 7f66717fa640 1 -- 172.21.15.164:0/2821623953 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f6664004000 con 0x7f6674100ad0 2024-09-19T08:40:29.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.406+0000 7f66717fa640 1 --2- 172.21.15.164:0/2821623953 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f665c0a3240 0x7f665c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:29.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.406+0000 7f66737fe640 1 --2- 172.21.15.164:0/2821623953 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f665c0a3240 0x7f665c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:29.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.406+0000 7f66717fa640 1 -- 172.21.15.164:0/2821623953 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4253+0+0 (secure 0 0 0) 0x7f66640ce2a0 con 0x7f6674100ad0 2024-09-19T08:40:29.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.407+0000 7f66737fe640 1 --2- 172.21.15.164:0/2821623953 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f665c0a3240 0x7f665c0a5700 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f667406b050 tx=0x7f6668005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:29.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.411+0000 7f66717fa640 1 -- 172.21.15.164:0/2821623953 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f666409bfa0 con 0x7f6674100ad0 2024-09-19T08:40:29.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:29.594+0000 7f667a4a0640 1 -- 172.21.15.164:0/2821623953 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f6674062630 con 0x7f665c0a3240 2024-09-19T08:40:30.006 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:29 smithi164 ceph-mon[30928]: Detected new or changed devices on smithi173 2024-09-19T08:40:30.006 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:29 smithi164 ceph-mon[30928]: Adjusting osd_memory_target on smithi173 to 8544M 2024-09-19T08:40:30.007 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:29 smithi164 ceph-mon[30928]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-19T08:40:30.007 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:29 smithi164 ceph-mon[30928]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T08:40:30.007 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:29 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:40:30.007 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:29 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:40:30.007 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:29 smithi164 ceph-mon[30928]: from='osd.5 ' entity='osd.5' 2024-09-19T08:40:30.007 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:29 smithi164 ceph-mon[30928]: osd.5 [v2:172.21.15.173:6808/2914653552,v1:172.21.15.173:6809/2914653552] boot 2024-09-19T08:40:30.007 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:29 smithi164 ceph-mon[30928]: osdmap e36: 6 total, 6 up, 6 in 2024-09-19T08:40:30.007 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:29 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:40:30.007 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:29 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:40:30.007 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:29 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:40:30.007 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:29 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:40:30.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:29 smithi170 ceph-mon[35072]: Detected new or changed devices on smithi173 2024-09-19T08:40:30.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:29 smithi170 ceph-mon[35072]: Adjusting osd_memory_target on smithi173 to 8544M 2024-09-19T08:40:30.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:29 smithi170 ceph-mon[35072]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-19T08:40:30.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:29 smithi170 ceph-mon[35072]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T08:40:30.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:29 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:40:30.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:29 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:40:30.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:29 smithi170 ceph-mon[35072]: from='osd.5 ' entity='osd.5' 2024-09-19T08:40:30.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:29 smithi170 ceph-mon[35072]: osd.5 [v2:172.21.15.173:6808/2914653552,v1:172.21.15.173:6809/2914653552] boot 2024-09-19T08:40:30.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:29 smithi170 ceph-mon[35072]: osdmap e36: 6 total, 6 up, 6 in 2024-09-19T08:40:30.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:29 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:40:30.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:29 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:40:30.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:29 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:40:30.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:29 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:40:30.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:29 smithi173 ceph-mon[35306]: Detected new or changed devices on smithi173 2024-09-19T08:40:30.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:29 smithi173 ceph-mon[35306]: Adjusting osd_memory_target on smithi173 to 8544M 2024-09-19T08:40:30.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:29 smithi173 ceph-mon[35306]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-19T08:40:30.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:29 smithi173 ceph-mon[35306]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T08:40:30.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:29 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:40:30.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:29 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:40:30.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:29 smithi173 ceph-mon[35306]: from='osd.5 ' entity='osd.5' 2024-09-19T08:40:30.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:29 smithi173 ceph-mon[35306]: osd.5 [v2:172.21.15.173:6808/2914653552,v1:172.21.15.173:6809/2914653552] boot 2024-09-19T08:40:30.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:29 smithi173 ceph-mon[35306]: osdmap e36: 6 total, 6 up, 6 in 2024-09-19T08:40:30.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:29 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:40:30.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:29 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:40:30.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:29 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:40:30.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:29 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:40:30.952 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:30 smithi164 ceph-mon[30928]: purged_snaps scrub starts 2024-09-19T08:40:30.953 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:30 smithi164 ceph-mon[30928]: purged_snaps scrub ok 2024-09-19T08:40:30.953 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:30 smithi164 ceph-mon[30928]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:40:30.953 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:30 smithi164 ceph-mon[30928]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:30.953 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:30 smithi164 ceph-mon[30928]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T08:40:31.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:30 smithi170 ceph-mon[35072]: purged_snaps scrub starts 2024-09-19T08:40:31.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:30 smithi170 ceph-mon[35072]: purged_snaps scrub ok 2024-09-19T08:40:31.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:30 smithi170 ceph-mon[35072]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:40:31.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:30 smithi170 ceph-mon[35072]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:31.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:30 smithi170 ceph-mon[35072]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T08:40:31.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:30 smithi173 ceph-mon[35306]: purged_snaps scrub starts 2024-09-19T08:40:31.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:30 smithi173 ceph-mon[35306]: purged_snaps scrub ok 2024-09-19T08:40:31.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:30 smithi173 ceph-mon[35306]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:40:31.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:30 smithi173 ceph-mon[35306]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-19T08:40:31.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:30 smithi173 ceph-mon[35306]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T08:40:32.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:32 smithi164 ceph-mon[30928]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T08:40:32.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:32 smithi164 ceph-mon[30928]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 207 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:40:32.783 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:32 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2217594758' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f5a965e-2eb2-4c24-b089-577e0639c160"}]: dispatch 2024-09-19T08:40:32.783 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:32 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2217594758' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f5a965e-2eb2-4c24-b089-577e0639c160"}]': finished 2024-09-19T08:40:32.783 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:32 smithi164 ceph-mon[30928]: osdmap e39: 7 total, 6 up, 7 in 2024-09-19T08:40:32.783 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:32 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:40:32.783 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:32 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2485212985' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:40:32.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:32 smithi170 ceph-mon[35072]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T08:40:32.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:32 smithi170 ceph-mon[35072]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 207 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:40:32.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:32 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2217594758' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f5a965e-2eb2-4c24-b089-577e0639c160"}]: dispatch 2024-09-19T08:40:32.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:32 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2217594758' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f5a965e-2eb2-4c24-b089-577e0639c160"}]': finished 2024-09-19T08:40:32.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:32 smithi170 ceph-mon[35072]: osdmap e39: 7 total, 6 up, 7 in 2024-09-19T08:40:32.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:32 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:40:32.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:32 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2485212985' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:40:32.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:32 smithi173 ceph-mon[35306]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T08:40:32.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:32 smithi173 ceph-mon[35306]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 207 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:40:32.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:32 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2217594758' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f5a965e-2eb2-4c24-b089-577e0639c160"}]: dispatch 2024-09-19T08:40:32.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:32 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2217594758' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f5a965e-2eb2-4c24-b089-577e0639c160"}]': finished 2024-09-19T08:40:32.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:32 smithi173 ceph-mon[35306]: osdmap e39: 7 total, 6 up, 7 in 2024-09-19T08:40:32.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:32 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:40:32.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:32 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2485212985' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:40:34.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:33 smithi164 ceph-mon[30928]: osdmap e40: 7 total, 6 up, 7 in 2024-09-19T08:40:34.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:33 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:40:34.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:33 smithi164 ceph-mon[30928]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 207 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T08:40:34.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:33 smithi170 ceph-mon[35072]: osdmap e40: 7 total, 6 up, 7 in 2024-09-19T08:40:34.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:33 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:40:34.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:33 smithi170 ceph-mon[35072]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 207 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T08:40:34.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:33 smithi173 ceph-mon[35306]: osdmap e40: 7 total, 6 up, 7 in 2024-09-19T08:40:34.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:33 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:40:34.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:33 smithi173 ceph-mon[35306]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 207 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T08:40:37.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:36 smithi164 ceph-mon[30928]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-09-19T08:40:37.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:36 smithi170 ceph-mon[35072]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-09-19T08:40:37.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:36 smithi173 ceph-mon[35306]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-09-19T08:40:38.876 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:38 smithi164 ceph-mon[30928]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-19T08:40:39.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:38 smithi170 ceph-mon[35072]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-19T08:40:39.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:38 smithi173 ceph-mon[35306]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-19T08:40:40.861 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:40 smithi164 ceph-mon[30928]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T08:40:40.862 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:40 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T08:40:40.862 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:40 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:40:41.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:40 smithi170 ceph-mon[35072]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T08:40:41.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:40 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T08:40:41.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:40 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:40:41.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:40 smithi173 ceph-mon[35306]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T08:40:41.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:40 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T08:40:41.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:40 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:40:42.026 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:41 smithi170 ceph-mon[35072]: Deploying daemon osd.6 on smithi164 2024-09-19T08:40:42.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:41 smithi164 ceph-mon[30928]: Deploying daemon osd.6 on smithi164 2024-09-19T08:40:42.061 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:41 smithi173 ceph-mon[35306]: Deploying daemon osd.6 on smithi164 2024-09-19T08:40:43.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:42 smithi164 ceph-mon[30928]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-19T08:40:43.043 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:42 smithi170 ceph-mon[35072]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-19T08:40:43.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:42 smithi173 ceph-mon[35306]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-19T08:40:44.901 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:44 smithi164 ceph-mon[30928]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-19T08:40:45.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:44 smithi170 ceph-mon[35072]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-19T08:40:45.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:44 smithi173 ceph-mon[35306]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-19T08:40:46.772 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:46 smithi164 ceph-mon[30928]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:40:47.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:46 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:40:47.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:46 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:47.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:46 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:47.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:46 smithi170 ceph-mon[35072]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:40:47.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:46 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:40:47.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:46 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:47.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:46 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:47.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:46 smithi173 ceph-mon[35306]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:40:47.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:46 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:40:47.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:46 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:47.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:46 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:49.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:48 smithi164 ceph-mon[30928]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:40:49.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:48 smithi170 ceph-mon[35072]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:40:49.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:48 smithi173 ceph-mon[35306]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:40:49.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:49.964+0000 7f66717fa640 1 -- 172.21.15.164:0/2821623953 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6674062630 con 0x7f665c0a3240 2024-09-19T08:40:49.965 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 6 on host 'smithi164' 2024-09-19T08:40:49.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:49.967+0000 7f667a4a0640 1 -- 172.21.15.164:0/2821623953 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f665c0a3240 msgr2=0x7f665c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:49.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:49.967+0000 7f667a4a0640 1 --2- 172.21.15.164:0/2821623953 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f665c0a3240 0x7f665c0a5700 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f667406b050 tx=0x7f6668005f90 comp rx=0 tx=0).stop 2024-09-19T08:40:49.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:49.967+0000 7f667a4a0640 1 -- 172.21.15.164:0/2821623953 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6674100ad0 msgr2=0x7f667406aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:49.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:49.967+0000 7f667a4a0640 1 --2- 172.21.15.164:0/2821623953 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6674100ad0 0x7f667406aad0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f666400d770 tx=0x7f666400dc30 comp rx=0 tx=0).stop 2024-09-19T08:40:49.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:49.967+0000 7f667a4a0640 1 -- 172.21.15.164:0/2821623953 shutdown_connections 2024-09-19T08:40:49.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:49.967+0000 7f667a4a0640 1 --2- 172.21.15.164:0/2821623953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6674102690 0x7f667419f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:49.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:49.967+0000 7f667a4a0640 1 --2- 172.21.15.164:0/2821623953 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f665c0a3240 0x7f665c0a5700 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:49.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:49.967+0000 7f667a4a0640 1 --2- 172.21.15.164:0/2821623953 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6674101cd0 0x7f667419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:49.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:49.967+0000 7f667a4a0640 1 --2- 172.21.15.164:0/2821623953 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6674100ad0 0x7f667406aad0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:49.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:49.967+0000 7f667a4a0640 1 -- 172.21.15.164:0/2821623953 >> 172.21.15.164:0/2821623953 conn(0x7f66740fc280 msgr2=0x7f6674105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:40:49.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:49.967+0000 7f667a4a0640 1 -- 172.21.15.164:0/2821623953 shutdown_connections 2024-09-19T08:40:49.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:40:49.967+0000 7f667a4a0640 1 -- 172.21.15.164:0/2821623953 wait complete. 2024-09-19T08:40:50.184 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:49 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:50.184 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:49 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:50.185 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:49 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:40:50.185 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:49 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:40:50.185 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:49 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:50.185 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:49 smithi164 ceph-mon[30928]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:40:50.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:49 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:50.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:49 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:50.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:49 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:40:50.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:49 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:40:50.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:49 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:50.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:49 smithi170 ceph-mon[35072]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:40:50.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:49 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:50.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:49 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:50.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:49 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:40:50.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:49 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:40:50.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:49 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:50.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:49 smithi173 ceph-mon[35306]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:40:50.621 DEBUG:teuthology.orchestra.run.smithi164:osd.6> sudo journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.6.service 2024-09-19T08:40:50.624 INFO:tasks.cephadm:Deploying osd.7 on smithi170 with /dev/vg_nvme/lv_2... 2024-09-19T08:40:50.624 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T08:40:50.883 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T08:40:51.262 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:50 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:40:51.262 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:50 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:51.262 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:50 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:51.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:50 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:40:51.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:50 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:51.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:50 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:51.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:50 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:40:51.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:50 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:51.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:50 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:52.204 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:51 smithi164 ceph-mon[30928]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:40:52.217 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:51 smithi170 ceph-mon[35072]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:40:52.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:51 smithi173 ceph-mon[35306]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:40:52.532 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:40:52 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:40:52.363+0000 7fbd4710c640 -1 osd.6 0 log_to_monitors true 2024-09-19T08:40:53.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:52 smithi170 ceph-mon[35072]: from='osd.6 [v2:172.21.15.164:6818/1138341393,v1:172.21.15.164:6819/1138341393]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T08:40:53.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:52 smithi173 ceph-mon[35306]: from='osd.6 [v2:172.21.15.164:6818/1138341393,v1:172.21.15.164:6819/1138341393]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T08:40:53.258 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:52 smithi164 ceph-mon[30928]: from='osd.6 [v2:172.21.15.164:6818/1138341393,v1:172.21.15.164:6819/1138341393]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T08:40:53.527 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-19T08:40:53.552 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi170:vg_nvme/lv_2 2024-09-19T08:40:53.795 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T08:40:54.244 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:53 smithi170 ceph-mon[35072]: from='osd.6 [v2:172.21.15.164:6818/1138341393,v1:172.21.15.164:6819/1138341393]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T08:40:54.245 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:53 smithi170 ceph-mon[35072]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T08:40:54.245 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:53 smithi170 ceph-mon[35072]: from='osd.6 [v2:172.21.15.164:6818/1138341393,v1:172.21.15.164:6819/1138341393]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-19T08:40:54.245 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:53 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:40:54.245 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:53 smithi170 ceph-mon[35072]: Detected new or changed devices on smithi164 2024-09-19T08:40:54.245 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:53 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:54.245 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:53 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:54.245 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:53 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:40:54.245 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:53 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:40:54.246 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:53 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:40:54.246 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:53 smithi170 ceph-mon[35072]: Adjusting osd_memory_target on smithi164 to 5696M 2024-09-19T08:40:54.246 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:53 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:54.246 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:53 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:40:54.246 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:53 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:40:54.246 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:53 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:54.246 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:53 smithi170 ceph-mon[35072]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:40:54.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:53 smithi164 ceph-mon[30928]: from='osd.6 [v2:172.21.15.164:6818/1138341393,v1:172.21.15.164:6819/1138341393]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T08:40:54.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:53 smithi164 ceph-mon[30928]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T08:40:54.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:53 smithi164 ceph-mon[30928]: from='osd.6 [v2:172.21.15.164:6818/1138341393,v1:172.21.15.164:6819/1138341393]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-19T08:40:54.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:53 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:40:54.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:53 smithi164 ceph-mon[30928]: Detected new or changed devices on smithi164 2024-09-19T08:40:54.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:53 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:54.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:53 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:54.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:53 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:40:54.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:53 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:40:54.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:53 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:40:54.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:53 smithi164 ceph-mon[30928]: Adjusting osd_memory_target on smithi164 to 5696M 2024-09-19T08:40:54.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:53 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:54.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:53 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:40:54.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:53 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:40:54.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:53 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:54.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:53 smithi164 ceph-mon[30928]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:40:54.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.327+0000 7fa639abe640 1 -- 172.21.15.170:0/3112196401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa634101cf0 msgr2=0x7fa634102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:54.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.327+0000 7fa639abe640 1 --2- 172.21.15.170:0/3112196401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa634101cf0 0x7fa634102170 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa628009900 tx=0x7fa62802a410 comp rx=0 tx=0).stop 2024-09-19T08:40:54.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.327+0000 7fa639abe640 1 -- 172.21.15.170:0/3112196401 shutdown_connections 2024-09-19T08:40:54.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.327+0000 7fa639abe640 1 --2- 172.21.15.170:0/3112196401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6341026b0 0x7fa634108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:54.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.327+0000 7fa639abe640 1 --2- 172.21.15.170:0/3112196401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa634101cf0 0x7fa634102170 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:54.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.327+0000 7fa639abe640 1 --2- 172.21.15.170:0/3112196401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa634100af0 0x7fa634100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:54.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.327+0000 7fa639abe640 1 -- 172.21.15.170:0/3112196401 >> 172.21.15.170:0/3112196401 conn(0x7fa6340fc2a0 msgr2=0x7fa6340fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:40:54.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.327+0000 7fa639abe640 1 -- 172.21.15.170:0/3112196401 shutdown_connections 2024-09-19T08:40:54.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.327+0000 7fa639abe640 1 -- 172.21.15.170:0/3112196401 wait complete. 2024-09-19T08:40:54.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.327+0000 7fa639abe640 1 Processor -- start 2024-09-19T08:40:54.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.328+0000 7fa639abe640 1 -- start start 2024-09-19T08:40:54.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.328+0000 7fa639abe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa634100af0 0x7fa63419a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:54.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.328+0000 7fa639abe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa634101cf0 0x7fa63419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:54.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.328+0000 7fa639abe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6341026b0 0x7fa6341a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:54.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.328+0000 7fa639abe640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa63410d8e0 con 0x7fa634101cf0 2024-09-19T08:40:54.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.328+0000 7fa639abe640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa63410d760 con 0x7fa634100af0 2024-09-19T08:40:54.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.328+0000 7fa639abe640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa63410da60 con 0x7fa6341026b0 2024-09-19T08:40:54.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.328+0000 7fa6337fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa634100af0 0x7fa63419a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:54.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.328+0000 7fa6337fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa634100af0 0x7fa63419a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40564/0 (socket says 172.21.15.170:40564) 2024-09-19T08:40:54.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.328+0000 7fa6337fe640 1 -- 172.21.15.170:0/4247181484 learned_addr learned my addr 172.21.15.170:0/4247181484 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-19T08:40:54.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.328+0000 7fa632ffd640 1 --2- 172.21.15.170:0/4247181484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa634101cf0 0x7fa63419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:54.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.328+0000 7fa633fff640 1 --2- 172.21.15.170:0/4247181484 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6341026b0 0x7fa6341a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:54.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.328+0000 7fa6337fe640 1 -- 172.21.15.170:0/4247181484 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6341026b0 msgr2=0x7fa6341a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:54.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.328+0000 7fa6337fe640 1 --2- 172.21.15.170:0/4247181484 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6341026b0 0x7fa6341a1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:54.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.328+0000 7fa6337fe640 1 -- 172.21.15.170:0/4247181484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa634101cf0 msgr2=0x7fa63419a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:54.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.328+0000 7fa6337fe640 1 --2- 172.21.15.170:0/4247181484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa634101cf0 0x7fa63419a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:54.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.328+0000 7fa6337fe640 1 -- 172.21.15.170:0/4247181484 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6341a1db0 con 0x7fa634100af0 2024-09-19T08:40:54.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.328+0000 7fa6337fe640 1 --2- 172.21.15.170:0/4247181484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa634100af0 0x7fa63419a180 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa624009fd0 tx=0x7fa62400c270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:54.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.329+0000 7fa630ff9640 1 -- 172.21.15.170:0/4247181484 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa624011470 con 0x7fa634100af0 2024-09-19T08:40:54.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.329+0000 7fa639abe640 1 -- 172.21.15.170:0/4247181484 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6341a20a0 con 0x7fa634100af0 2024-09-19T08:40:54.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.329+0000 7fa630ff9640 1 -- 172.21.15.170:0/4247181484 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa624011af0 con 0x7fa634100af0 2024-09-19T08:40:54.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.329+0000 7fa630ff9640 1 -- 172.21.15.170:0/4247181484 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62401ab90 con 0x7fa634100af0 2024-09-19T08:40:54.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.329+0000 7fa639abe640 1 -- 172.21.15.170:0/4247181484 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6341a25b0 con 0x7fa634100af0 2024-09-19T08:40:54.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.330+0000 7fa639abe640 1 -- 172.21.15.170:0/4247181484 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6341a2870 con 0x7fa634100af0 2024-09-19T08:40:54.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.331+0000 7fa630ff9640 1 -- 172.21.15.170:0/4247181484 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fa624011610 con 0x7fa634100af0 2024-09-19T08:40:54.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.331+0000 7fa630ff9640 1 --2- 172.21.15.170:0/4247181484 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fa61c0a3270 0x7fa61c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:54.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.332+0000 7fa630ff9640 1 -- 172.21.15.170:0/4247181484 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4685+0+0 (secure 0 0 0) 0x7fa6240cf2e0 con 0x7fa634100af0 2024-09-19T08:40:54.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.332+0000 7fa632ffd640 1 --2- 172.21.15.170:0/4247181484 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fa61c0a3270 0x7fa61c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:54.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.332+0000 7fa632ffd640 1 --2- 172.21.15.170:0/4247181484 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fa61c0a3270 0x7fa61c0a5730 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa6280314e0 tx=0x7fa628000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:54.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.335+0000 7fa630ff9640 1 -- 172.21.15.170:0/4247181484 <== mon.2 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa624016030 con 0x7fa634100af0 2024-09-19T08:40:54.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:53 smithi173 ceph-mon[35306]: from='osd.6 [v2:172.21.15.164:6818/1138341393,v1:172.21.15.164:6819/1138341393]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T08:40:54.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:53 smithi173 ceph-mon[35306]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T08:40:54.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:53 smithi173 ceph-mon[35306]: from='osd.6 [v2:172.21.15.164:6818/1138341393,v1:172.21.15.164:6819/1138341393]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-19T08:40:54.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:53 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:40:54.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:53 smithi173 ceph-mon[35306]: Detected new or changed devices on smithi164 2024-09-19T08:40:54.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:53 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:54.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:53 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:54.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:53 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:40:54.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:53 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:40:54.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:53 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:40:54.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:53 smithi173 ceph-mon[35306]: Adjusting osd_memory_target on smithi164 to 5696M 2024-09-19T08:40:54.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:53 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:54.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:53 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:40:54.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:53 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:40:54.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:53 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:40:54.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:53 smithi173 ceph-mon[35306]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:40:54.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:40:54.511+0000 7fa639abe640 1 -- 172.21.15.170:0/4247181484 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fa634108bd0 con 0x7fa61c0a3270 2024-09-19T08:40:54.782 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:40:54 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:40:54.357+0000 7fbd4308f640 -1 osd.6 0 waiting for initial osdmap 2024-09-19T08:40:54.782 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:40:54 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:40:54.367+0000 7fbd3aeaf640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:40:55.256 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:54 smithi170 ceph-mon[35072]: from='osd.6 [v2:172.21.15.164:6818/1138341393,v1:172.21.15.164:6819/1138341393]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-19T08:40:55.257 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:54 smithi170 ceph-mon[35072]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T08:40:55.257 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:54 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:40:55.257 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:54 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:40:55.257 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:54 smithi170 ceph-mon[35072]: from='osd.6 [v2:172.21.15.164:6818/1138341393,v1:172.21.15.164:6819/1138341393]' entity='osd.6' 2024-09-19T08:40:55.257 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:54 smithi170 ceph-mon[35072]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:40:55.257 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:54 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:40:55.257 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:54 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:40:55.257 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:54 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:40:55.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:54 smithi164 ceph-mon[30928]: from='osd.6 [v2:172.21.15.164:6818/1138341393,v1:172.21.15.164:6819/1138341393]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-19T08:40:55.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:54 smithi164 ceph-mon[30928]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T08:40:55.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:54 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:40:55.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:54 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:40:55.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:54 smithi164 ceph-mon[30928]: from='osd.6 [v2:172.21.15.164:6818/1138341393,v1:172.21.15.164:6819/1138341393]' entity='osd.6' 2024-09-19T08:40:55.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:54 smithi164 ceph-mon[30928]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:40:55.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:54 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:40:55.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:54 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:40:55.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:54 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:40:55.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:54 smithi173 ceph-mon[35306]: from='osd.6 [v2:172.21.15.164:6818/1138341393,v1:172.21.15.164:6819/1138341393]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-19T08:40:55.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:54 smithi173 ceph-mon[35306]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T08:40:55.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:54 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:40:55.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:54 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:40:55.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:54 smithi173 ceph-mon[35306]: from='osd.6 [v2:172.21.15.164:6818/1138341393,v1:172.21.15.164:6819/1138341393]' entity='osd.6' 2024-09-19T08:40:55.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:54 smithi173 ceph-mon[35306]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:40:55.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:54 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:40:55.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:54 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:40:55.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:54 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:40:56.155 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:55 smithi170 ceph-mon[35072]: purged_snaps scrub starts 2024-09-19T08:40:56.155 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:55 smithi170 ceph-mon[35072]: purged_snaps scrub ok 2024-09-19T08:40:56.155 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:55 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:40:56.155 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:55 smithi170 ceph-mon[35072]: osd.6 [v2:172.21.15.164:6818/1138341393,v1:172.21.15.164:6819/1138341393] boot 2024-09-19T08:40:56.155 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:55 smithi170 ceph-mon[35072]: osdmap e43: 7 total, 7 up, 7 in 2024-09-19T08:40:56.155 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:55 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:40:56.155 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:55 smithi170 ceph-mon[35072]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:40:56.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:55 smithi164 ceph-mon[30928]: purged_snaps scrub starts 2024-09-19T08:40:56.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:55 smithi164 ceph-mon[30928]: purged_snaps scrub ok 2024-09-19T08:40:56.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:55 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:40:56.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:55 smithi164 ceph-mon[30928]: osd.6 [v2:172.21.15.164:6818/1138341393,v1:172.21.15.164:6819/1138341393] boot 2024-09-19T08:40:56.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:55 smithi164 ceph-mon[30928]: osdmap e43: 7 total, 7 up, 7 in 2024-09-19T08:40:56.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:55 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:40:56.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:55 smithi164 ceph-mon[30928]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:40:56.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:55 smithi173 ceph-mon[35306]: purged_snaps scrub starts 2024-09-19T08:40:56.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:55 smithi173 ceph-mon[35306]: purged_snaps scrub ok 2024-09-19T08:40:56.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:55 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:40:56.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:55 smithi173 ceph-mon[35306]: osd.6 [v2:172.21.15.164:6818/1138341393,v1:172.21.15.164:6819/1138341393] boot 2024-09-19T08:40:56.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:55 smithi173 ceph-mon[35306]: osdmap e43: 7 total, 7 up, 7 in 2024-09-19T08:40:56.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:55 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:40:56.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:55 smithi173 ceph-mon[35306]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-19T08:40:57.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:57 smithi170 ceph-mon[35072]: from='client.? 172.21.15.170:0/4111358509' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f16cb2c3-639b-46ae-a740-1bd28ca1863c"}]: dispatch 2024-09-19T08:40:57.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:57 smithi170 ceph-mon[35072]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f16cb2c3-639b-46ae-a740-1bd28ca1863c"}]: dispatch 2024-09-19T08:40:57.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:57 smithi170 ceph-mon[35072]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f16cb2c3-639b-46ae-a740-1bd28ca1863c"}]': finished 2024-09-19T08:40:57.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:57 smithi170 ceph-mon[35072]: osdmap e44: 8 total, 7 up, 8 in 2024-09-19T08:40:57.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:57 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:40:57.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:57 smithi173 ceph-mon[35306]: from='client.? 172.21.15.170:0/4111358509' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f16cb2c3-639b-46ae-a740-1bd28ca1863c"}]: dispatch 2024-09-19T08:40:57.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:57 smithi173 ceph-mon[35306]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f16cb2c3-639b-46ae-a740-1bd28ca1863c"}]: dispatch 2024-09-19T08:40:57.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:57 smithi173 ceph-mon[35306]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f16cb2c3-639b-46ae-a740-1bd28ca1863c"}]': finished 2024-09-19T08:40:57.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:57 smithi173 ceph-mon[35306]: osdmap e44: 8 total, 7 up, 8 in 2024-09-19T08:40:57.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:57 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:40:57.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:57 smithi164 ceph-mon[30928]: from='client.? 172.21.15.170:0/4111358509' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f16cb2c3-639b-46ae-a740-1bd28ca1863c"}]: dispatch 2024-09-19T08:40:57.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:57 smithi164 ceph-mon[30928]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f16cb2c3-639b-46ae-a740-1bd28ca1863c"}]: dispatch 2024-09-19T08:40:57.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:57 smithi164 ceph-mon[30928]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f16cb2c3-639b-46ae-a740-1bd28ca1863c"}]': finished 2024-09-19T08:40:57.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:57 smithi164 ceph-mon[30928]: osdmap e44: 8 total, 7 up, 8 in 2024-09-19T08:40:57.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:57 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:40:58.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:58 smithi170 ceph-mon[35072]: from='client.? 172.21.15.170:0/1220435425' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:40:58.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:40:58 smithi170 ceph-mon[35072]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:40:58.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:58 smithi173 ceph-mon[35306]: from='client.? 172.21.15.170:0/1220435425' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:40:58.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:40:58 smithi173 ceph-mon[35306]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:40:58.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:58 smithi164 ceph-mon[30928]: from='client.? 172.21.15.170:0/1220435425' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:40:58.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:40:58 smithi164 ceph-mon[30928]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:01.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:00 smithi164 ceph-mon[30928]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:01.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:00 smithi170 ceph-mon[35072]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:01.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:00 smithi173 ceph-mon[35306]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:03.014 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:02 smithi170 ceph-mon[35072]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:03.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:02 smithi164 ceph-mon[30928]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:03.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:02 smithi173 ceph-mon[35306]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:05.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:04 smithi164 ceph-mon[30928]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:05.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:04 smithi170 ceph-mon[35072]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:05.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:04 smithi173 ceph-mon[35306]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:05.874 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:05 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T08:41:05.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:05 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:41:06.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:05 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T08:41:06.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:05 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:41:06.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:05 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T08:41:06.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:05 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:41:06.903 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:06 smithi170 ceph-mon[35072]: Deploying daemon osd.7 on smithi170 2024-09-19T08:41:06.903 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:06 smithi170 ceph-mon[35072]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:07.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:06 smithi164 ceph-mon[30928]: Deploying daemon osd.7 on smithi170 2024-09-19T08:41:07.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:06 smithi164 ceph-mon[30928]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:07.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:06 smithi173 ceph-mon[35306]: Deploying daemon osd.7 on smithi170 2024-09-19T08:41:07.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:06 smithi173 ceph-mon[35306]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:09.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:08 smithi164 ceph-mon[30928]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:09.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:08 smithi170 ceph-mon[35072]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:09.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:08 smithi173 ceph-mon[35306]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:10.995 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:10 smithi170 ceph-mon[35072]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:11.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:10 smithi164 ceph-mon[30928]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:11.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:10 smithi173 ceph-mon[35306]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:12.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:11 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:41:12.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:11 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:12.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:11 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:12.062 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:11 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:41:12.062 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:11 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:12.062 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:11 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:12.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:11 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:41:12.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:11 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:12.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:11 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:13.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:12 smithi164 ceph-mon[30928]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:13.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:12 smithi170 ceph-mon[35072]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:13.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:12 smithi173 ceph-mon[35306]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:14.948 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:14 smithi170 ceph-mon[35072]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:14.948 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:14 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:14.948 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:14 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:14.948 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:14 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:41:14.948 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:14 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:41:14.948 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:14 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:14.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:41:14.988+0000 7fa630ff9640 1 -- 172.21.15.170:0/4247181484 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa634108bd0 con 0x7fa61c0a3270 2024-09-19T08:41:14.989 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 7 on host 'smithi170' 2024-09-19T08:41:14.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:41:14.990+0000 7fa639abe640 1 -- 172.21.15.170:0/4247181484 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fa61c0a3270 msgr2=0x7fa61c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:14.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:41:14.990+0000 7fa639abe640 1 --2- 172.21.15.170:0/4247181484 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fa61c0a3270 0x7fa61c0a5730 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa6280314e0 tx=0x7fa628000bc0 comp rx=0 tx=0).stop 2024-09-19T08:41:14.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:41:14.990+0000 7fa639abe640 1 -- 172.21.15.170:0/4247181484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa634100af0 msgr2=0x7fa63419a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:14.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:41:14.990+0000 7fa639abe640 1 --2- 172.21.15.170:0/4247181484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa634100af0 0x7fa63419a180 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa624009fd0 tx=0x7fa62400c270 comp rx=0 tx=0).stop 2024-09-19T08:41:14.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:41:14.990+0000 7fa639abe640 1 -- 172.21.15.170:0/4247181484 shutdown_connections 2024-09-19T08:41:14.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:41:14.990+0000 7fa639abe640 1 --2- 172.21.15.170:0/4247181484 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fa61c0a3270 0x7fa61c0a5730 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:14.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:41:14.990+0000 7fa639abe640 1 --2- 172.21.15.170:0/4247181484 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6341026b0 0x7fa6341a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:14.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:41:14.990+0000 7fa639abe640 1 --2- 172.21.15.170:0/4247181484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa634101cf0 0x7fa63419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:14.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:41:14.990+0000 7fa639abe640 1 --2- 172.21.15.170:0/4247181484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa634100af0 0x7fa63419a180 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:14.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:41:14.990+0000 7fa639abe640 1 -- 172.21.15.170:0/4247181484 >> 172.21.15.170:0/4247181484 conn(0x7fa6340fc2a0 msgr2=0x7fa6341074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:14.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:41:14.991+0000 7fa639abe640 1 -- 172.21.15.170:0/4247181484 shutdown_connections 2024-09-19T08:41:14.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:41:14.991+0000 7fa639abe640 1 -- 172.21.15.170:0/4247181484 wait complete. 2024-09-19T08:41:15.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:14 smithi173 ceph-mon[35306]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:15.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:14 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:15.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:14 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:15.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:14 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:41:15.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:14 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:41:15.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:14 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:15.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:14 smithi164 ceph-mon[30928]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:15.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:14 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:15.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:14 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:15.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:14 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:41:15.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:14 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:41:15.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:14 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:15.663 DEBUG:teuthology.orchestra.run.smithi170:osd.7> sudo journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.7.service 2024-09-19T08:41:15.665 INFO:tasks.cephadm:Deploying osd.8 on smithi173 with /dev/vg_nvme/lv_2... 2024-09-19T08:41:15.665 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T08:41:15.921 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:41:15.979 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:15 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:41:15.979 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:15 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:15.979 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:15 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:16.041 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:15 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:41:16.041 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:15 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:16.041 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:15 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:16.281 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:15 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:41:16.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:15 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:16.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:15 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:17.054 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:16 smithi173 ceph-mon[35306]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:17.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:16 smithi170 ceph-mon[35072]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:17.125 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:41:16 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:41:16.865+0000 7f209a95e640 -1 osd.7 0 log_to_monitors true 2024-09-19T08:41:17.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:16 smithi164 ceph-mon[30928]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:17.938 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:17 smithi173 ceph-mon[35306]: from='osd.7 [v2:172.21.15.170:6816/2124160360,v1:172.21.15.170:6817/2124160360]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T08:41:17.938 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:17 smithi173 ceph-mon[35306]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T08:41:18.038 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:17 smithi170 ceph-mon[35072]: from='osd.7 [v2:172.21.15.170:6816/2124160360,v1:172.21.15.170:6817/2124160360]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T08:41:18.038 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:17 smithi170 ceph-mon[35072]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T08:41:18.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:17 smithi164 ceph-mon[30928]: from='osd.7 [v2:172.21.15.170:6816/2124160360,v1:172.21.15.170:6817/2124160360]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T08:41:18.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:17 smithi164 ceph-mon[30928]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T08:41:18.736 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-19T08:41:18.760 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi173:vg_nvme/lv_2 2024-09-19T08:41:19.006 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:41:19.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:18 smithi170 ceph-mon[35072]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:19.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:18 smithi170 ceph-mon[35072]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T08:41:19.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:18 smithi170 ceph-mon[35072]: osdmap e45: 8 total, 7 up, 8 in 2024-09-19T08:41:19.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:18 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:41:19.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:18 smithi170 ceph-mon[35072]: from='osd.7 [v2:172.21.15.170:6816/2124160360,v1:172.21.15.170:6817/2124160360]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-19T08:41:19.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:18 smithi170 ceph-mon[35072]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-19T08:41:19.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:18 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:19.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:18 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:19.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:18 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:41:19.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:18 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:41:19.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:18 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:41:19.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:18 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:19.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:18 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:41:19.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:18 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:41:19.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:18 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:19.127 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:18 smithi173 ceph-mon[35306]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:19.127 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:18 smithi173 ceph-mon[35306]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T08:41:19.127 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:18 smithi173 ceph-mon[35306]: osdmap e45: 8 total, 7 up, 8 in 2024-09-19T08:41:19.127 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:18 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:41:19.127 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:18 smithi173 ceph-mon[35306]: from='osd.7 [v2:172.21.15.170:6816/2124160360,v1:172.21.15.170:6817/2124160360]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-19T08:41:19.127 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:18 smithi173 ceph-mon[35306]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-19T08:41:19.127 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:18 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:19.127 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:18 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:19.128 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:18 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:41:19.128 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:18 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:41:19.128 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:18 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:41:19.128 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:18 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:19.128 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:18 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:41:19.128 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:18 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:41:19.128 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:18 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:19.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:18 smithi164 ceph-mon[30928]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:19.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:18 smithi164 ceph-mon[30928]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T08:41:19.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:18 smithi164 ceph-mon[30928]: osdmap e45: 8 total, 7 up, 8 in 2024-09-19T08:41:19.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:18 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:41:19.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:18 smithi164 ceph-mon[30928]: from='osd.7 [v2:172.21.15.170:6816/2124160360,v1:172.21.15.170:6817/2124160360]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-19T08:41:19.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:18 smithi164 ceph-mon[30928]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-19T08:41:19.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:18 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:19.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:18 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:19.284 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:18 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:41:19.284 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:18 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:41:19.284 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:18 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:41:19.284 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:18 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:19.284 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:18 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:41:19.284 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:18 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:41:19.284 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:18 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:19.478 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.461+0000 7f798f4c0640 1 -- 172.21.15.173:0/1361507241 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7988101c10 msgr2=0x7f7988102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:19.478 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.461+0000 7f798f4c0640 1 --2- 172.21.15.173:0/1361507241 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7988101c10 0x7f7988102010 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7978009ca0 tx=0x7f797802a8c0 comp rx=0 tx=0).stop 2024-09-19T08:41:19.478 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.461+0000 7f798f4c0640 1 -- 172.21.15.173:0/1361507241 shutdown_connections 2024-09-19T08:41:19.478 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.461+0000 7f798f4c0640 1 --2- 172.21.15.173:0/1361507241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79881037d0 0x7f798810a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:19.478 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.461+0000 7f798f4c0640 1 --2- 172.21.15.173:0/1361507241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7988102e10 0x7f7988103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:19.479 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.461+0000 7f798f4c0640 1 --2- 172.21.15.173:0/1361507241 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7988101c10 0x7f7988102010 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:19.479 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.461+0000 7f798f4c0640 1 -- 172.21.15.173:0/1361507241 >> 172.21.15.173:0/1361507241 conn(0x7f79880fd3c0 msgr2=0x7f79880ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:19.479 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.461+0000 7f798f4c0640 1 -- 172.21.15.173:0/1361507241 shutdown_connections 2024-09-19T08:41:19.479 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.461+0000 7f798f4c0640 1 -- 172.21.15.173:0/1361507241 wait complete. 2024-09-19T08:41:19.479 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.461+0000 7f798f4c0640 1 Processor -- start 2024-09-19T08:41:19.479 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.461+0000 7f798f4c0640 1 -- start start 2024-09-19T08:41:19.479 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.462+0000 7f798f4c0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7988101c10 0x7f7988197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:19.479 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.462+0000 7f798f4c0640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7988102e10 0x7f7988198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:19.479 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.462+0000 7f798f4c0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79881037d0 0x7f798819f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:19.479 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.462+0000 7f798f4c0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f798810d8a0 con 0x7f79881037d0 2024-09-19T08:41:19.479 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.462+0000 7f798f4c0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f798810d720 con 0x7f7988101c10 2024-09-19T08:41:19.479 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.462+0000 7f798f4c0640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f798810da20 con 0x7f7988102e10 2024-09-19T08:41:19.480 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.462+0000 7f798ca5b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7988102e10 0x7f7988198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:19.480 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.462+0000 7f798d25c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7988101c10 0x7f7988197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:19.480 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.462+0000 7f798ca5b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7988102e10 0x7f7988198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:43132/0 (socket says 172.21.15.173:43132) 2024-09-19T08:41:19.480 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.462+0000 7f798ca5b640 1 -- 172.21.15.173:0/1993297004 learned_addr learned my addr 172.21.15.173:0/1993297004 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-09-19T08:41:19.480 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.462+0000 7f798da5d640 1 --2- 172.21.15.173:0/1993297004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79881037d0 0x7f798819f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:19.480 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.462+0000 7f798ca5b640 1 -- 172.21.15.173:0/1993297004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7988101c10 msgr2=0x7f7988197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:19.480 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.462+0000 7f798ca5b640 1 --2- 172.21.15.173:0/1993297004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7988101c10 0x7f7988197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:19.480 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.462+0000 7f798ca5b640 1 -- 172.21.15.173:0/1993297004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79881037d0 msgr2=0x7f798819f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:19.480 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.462+0000 7f798ca5b640 1 --2- 172.21.15.173:0/1993297004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79881037d0 0x7f798819f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:19.480 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.462+0000 7f798ca5b640 1 -- 172.21.15.173:0/1993297004 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f798819fb60 con 0x7f7988102e10 2024-09-19T08:41:19.480 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.462+0000 7f798d25c640 1 --2- 172.21.15.173:0/1993297004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7988101c10 0x7f7988197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:41:19.481 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.462+0000 7f798ca5b640 1 --2- 172.21.15.173:0/1993297004 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7988102e10 0x7f7988198470 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f797c00c890 tx=0x7f797c00cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:19.481 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.462+0000 7f79867fc640 1 -- 172.21.15.173:0/1993297004 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797c004220 con 0x7f7988102e10 2024-09-19T08:41:19.481 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.462+0000 7f79867fc640 1 -- 172.21.15.173:0/1993297004 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f797c0043c0 con 0x7f7988102e10 2024-09-19T08:41:19.481 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.462+0000 7f798f4c0640 1 -- 172.21.15.173:0/1993297004 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f798819fe50 con 0x7f7988102e10 2024-09-19T08:41:19.481 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.462+0000 7f79867fc640 1 -- 172.21.15.173:0/1993297004 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797c004a00 con 0x7f7988102e10 2024-09-19T08:41:19.481 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.463+0000 7f798f4c0640 1 -- 172.21.15.173:0/1993297004 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79881a0300 con 0x7f7988102e10 2024-09-19T08:41:19.481 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.464+0000 7f798f4c0640 1 -- 172.21.15.173:0/1993297004 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7988109060 con 0x7f7988102e10 2024-09-19T08:41:19.481 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.465+0000 7f79867fc640 1 -- 172.21.15.173:0/1993297004 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f797c0079b0 con 0x7f7988102e10 2024-09-19T08:41:19.481 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.465+0000 7f79867fc640 1 --2- 172.21.15.173:0/1993297004 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f79700a3270 0x7f79700a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:19.481 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.465+0000 7f79867fc640 1 -- 172.21.15.173:0/1993297004 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5117+0+0 (secure 0 0 0) 0x7f797c0cebd0 con 0x7f7988102e10 2024-09-19T08:41:19.481 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.466+0000 7f798d25c640 1 --2- 172.21.15.173:0/1993297004 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f79700a3270 0x7f79700a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:19.482 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.466+0000 7f798d25c640 1 --2- 172.21.15.173:0/1993297004 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f79700a3270 0x7f79700a5730 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7978006010 tx=0x7f7978000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:19.482 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.469+0000 7f79867fc640 1 -- 172.21.15.173:0/1993297004 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f797c014030 con 0x7f7988102e10 2024-09-19T08:41:19.625 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:41:19 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:41:19.196+0000 7f20968e1640 -1 osd.7 0 waiting for initial osdmap 2024-09-19T08:41:19.625 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:41:19 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:41:19.205+0000 7f208e701640 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:41:19.648 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:19.649+0000 7f798f4c0640 1 -- 172.21.15.173:0/1993297004 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f7988062630 con 0x7f79700a3270 2024-09-19T08:41:19.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:19 smithi173 ceph-mon[35306]: Detected new or changed devices on smithi170 2024-09-19T08:41:19.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:19 smithi173 ceph-mon[35306]: Adjusting osd_memory_target on smithi170 to 5696M 2024-09-19T08:41:19.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:19 smithi173 ceph-mon[35306]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-09-19T08:41:19.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:19 smithi173 ceph-mon[35306]: osdmap e46: 8 total, 7 up, 8 in 2024-09-19T08:41:19.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:19 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:41:19.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:19 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:41:19.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:19 smithi173 ceph-mon[35306]: from='osd.7 ' entity='osd.7' 2024-09-19T08:41:19.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:19 smithi173 ceph-mon[35306]: osd.7 [v2:172.21.15.170:6816/2124160360,v1:172.21.15.170:6817/2124160360] boot 2024-09-19T08:41:19.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:19 smithi173 ceph-mon[35306]: osdmap e47: 8 total, 8 up, 8 in 2024-09-19T08:41:19.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:19 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:41:19.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:19 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:41:19.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:19 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:41:19.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:19 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:41:20.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:19 smithi170 ceph-mon[35072]: Detected new or changed devices on smithi170 2024-09-19T08:41:20.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:19 smithi170 ceph-mon[35072]: Adjusting osd_memory_target on smithi170 to 5696M 2024-09-19T08:41:20.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:19 smithi170 ceph-mon[35072]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-09-19T08:41:20.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:19 smithi170 ceph-mon[35072]: osdmap e46: 8 total, 7 up, 8 in 2024-09-19T08:41:20.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:19 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:41:20.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:19 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:41:20.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:19 smithi170 ceph-mon[35072]: from='osd.7 ' entity='osd.7' 2024-09-19T08:41:20.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:19 smithi170 ceph-mon[35072]: osd.7 [v2:172.21.15.170:6816/2124160360,v1:172.21.15.170:6817/2124160360] boot 2024-09-19T08:41:20.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:19 smithi170 ceph-mon[35072]: osdmap e47: 8 total, 8 up, 8 in 2024-09-19T08:41:20.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:19 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:41:20.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:19 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:41:20.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:19 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:41:20.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:19 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:41:20.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:19 smithi164 ceph-mon[30928]: Detected new or changed devices on smithi170 2024-09-19T08:41:20.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:19 smithi164 ceph-mon[30928]: Adjusting osd_memory_target on smithi170 to 5696M 2024-09-19T08:41:20.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:19 smithi164 ceph-mon[30928]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-09-19T08:41:20.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:19 smithi164 ceph-mon[30928]: osdmap e46: 8 total, 7 up, 8 in 2024-09-19T08:41:20.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:19 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:41:20.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:19 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:41:20.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:19 smithi164 ceph-mon[30928]: from='osd.7 ' entity='osd.7' 2024-09-19T08:41:20.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:19 smithi164 ceph-mon[30928]: osd.7 [v2:172.21.15.170:6816/2124160360,v1:172.21.15.170:6817/2124160360] boot 2024-09-19T08:41:20.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:19 smithi164 ceph-mon[30928]: osdmap e47: 8 total, 8 up, 8 in 2024-09-19T08:41:20.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:19 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:41:20.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:19 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:41:20.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:19 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:41:20.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:19 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:41:20.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:20 smithi173 ceph-mon[35306]: purged_snaps scrub starts 2024-09-19T08:41:20.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:20 smithi173 ceph-mon[35306]: purged_snaps scrub ok 2024-09-19T08:41:20.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:20 smithi173 ceph-mon[35306]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:41:20.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:20 smithi173 ceph-mon[35306]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:20.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:20 smithi173 ceph-mon[35306]: osdmap e48: 8 total, 8 up, 8 in 2024-09-19T08:41:21.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:20 smithi170 ceph-mon[35072]: purged_snaps scrub starts 2024-09-19T08:41:21.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:20 smithi170 ceph-mon[35072]: purged_snaps scrub ok 2024-09-19T08:41:21.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:20 smithi170 ceph-mon[35072]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:41:21.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:20 smithi170 ceph-mon[35072]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:21.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:20 smithi170 ceph-mon[35072]: osdmap e48: 8 total, 8 up, 8 in 2024-09-19T08:41:21.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:20 smithi164 ceph-mon[30928]: purged_snaps scrub starts 2024-09-19T08:41:21.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:20 smithi164 ceph-mon[30928]: purged_snaps scrub ok 2024-09-19T08:41:21.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:20 smithi164 ceph-mon[30928]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:41:21.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:20 smithi164 ceph-mon[30928]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-09-19T08:41:21.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:20 smithi164 ceph-mon[30928]: osdmap e48: 8 total, 8 up, 8 in 2024-09-19T08:41:22.781 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:22 smithi173 ceph-mon[35306]: osdmap e49: 8 total, 8 up, 8 in 2024-09-19T08:41:22.781 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:22 smithi173 ceph-mon[35306]: pgmap v169: 1 pgs: 1 remapped; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:22.781 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:22 smithi173 ceph-mon[35306]: from='client.? 172.21.15.173:0/1911783993' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "161cff01-46ad-48e3-b999-a5473a2ad618"}]: dispatch 2024-09-19T08:41:22.781 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:22 smithi173 ceph-mon[35306]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "161cff01-46ad-48e3-b999-a5473a2ad618"}]: dispatch 2024-09-19T08:41:22.781 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:22 smithi173 ceph-mon[35306]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "161cff01-46ad-48e3-b999-a5473a2ad618"}]': finished 2024-09-19T08:41:22.781 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:22 smithi173 ceph-mon[35306]: osdmap e50: 9 total, 8 up, 9 in 2024-09-19T08:41:22.782 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:22 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:41:22.782 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:22 smithi173 ceph-mon[35306]: from='client.? 172.21.15.173:0/3267694447' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:41:22.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:22 smithi170 ceph-mon[35072]: osdmap e49: 8 total, 8 up, 8 in 2024-09-19T08:41:22.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:22 smithi170 ceph-mon[35072]: pgmap v169: 1 pgs: 1 remapped; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:22.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:22 smithi170 ceph-mon[35072]: from='client.? 172.21.15.173:0/1911783993' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "161cff01-46ad-48e3-b999-a5473a2ad618"}]: dispatch 2024-09-19T08:41:22.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:22 smithi170 ceph-mon[35072]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "161cff01-46ad-48e3-b999-a5473a2ad618"}]: dispatch 2024-09-19T08:41:22.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:22 smithi170 ceph-mon[35072]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "161cff01-46ad-48e3-b999-a5473a2ad618"}]': finished 2024-09-19T08:41:22.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:22 smithi170 ceph-mon[35072]: osdmap e50: 9 total, 8 up, 9 in 2024-09-19T08:41:22.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:22 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:41:22.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:22 smithi170 ceph-mon[35072]: from='client.? 172.21.15.173:0/3267694447' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:41:23.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:22 smithi164 ceph-mon[30928]: osdmap e49: 8 total, 8 up, 8 in 2024-09-19T08:41:23.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:22 smithi164 ceph-mon[30928]: pgmap v169: 1 pgs: 1 remapped; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:23.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:22 smithi164 ceph-mon[30928]: from='client.? 172.21.15.173:0/1911783993' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "161cff01-46ad-48e3-b999-a5473a2ad618"}]: dispatch 2024-09-19T08:41:23.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:22 smithi164 ceph-mon[30928]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "161cff01-46ad-48e3-b999-a5473a2ad618"}]: dispatch 2024-09-19T08:41:23.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:22 smithi164 ceph-mon[30928]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "161cff01-46ad-48e3-b999-a5473a2ad618"}]': finished 2024-09-19T08:41:23.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:22 smithi164 ceph-mon[30928]: osdmap e50: 9 total, 8 up, 9 in 2024-09-19T08:41:23.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:22 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:41:23.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:22 smithi164 ceph-mon[30928]: from='client.? 172.21.15.173:0/3267694447' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:41:24.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:23 smithi164 ceph-mon[30928]: osdmap e51: 9 total, 8 up, 9 in 2024-09-19T08:41:24.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:23 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:41:24.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:23 smithi164 ceph-mon[30928]: pgmap v172: 1 pgs: 1 remapped; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:24.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:23 smithi170 ceph-mon[35072]: osdmap e51: 9 total, 8 up, 9 in 2024-09-19T08:41:24.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:23 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:41:24.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:23 smithi170 ceph-mon[35072]: pgmap v172: 1 pgs: 1 remapped; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:24.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:23 smithi173 ceph-mon[35306]: osdmap e51: 9 total, 8 up, 9 in 2024-09-19T08:41:24.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:23 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:41:24.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:23 smithi173 ceph-mon[35306]: pgmap v172: 1 pgs: 1 remapped; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:27.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:26 smithi164 ceph-mon[30928]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:27.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:26 smithi173 ceph-mon[35306]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:27.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:26 smithi170 ceph-mon[35072]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:28.874 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:28 smithi173 ceph-mon[35306]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:29.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:28 smithi164 ceph-mon[30928]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:29.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:28 smithi170 ceph-mon[35072]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:31.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:30 smithi164 ceph-mon[30928]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:31.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:30 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T08:41:31.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:30 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:41:31.042 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:30 smithi173 ceph-mon[35306]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:31.043 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:30 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T08:41:31.043 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:30 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:41:31.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:30 smithi170 ceph-mon[35072]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:31.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:30 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T08:41:31.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:30 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:41:31.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:31 smithi173 ceph-mon[35306]: Deploying daemon osd.8 on smithi173 2024-09-19T08:41:32.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:31 smithi164 ceph-mon[30928]: Deploying daemon osd.8 on smithi173 2024-09-19T08:41:32.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:31 smithi170 ceph-mon[35072]: Deploying daemon osd.8 on smithi173 2024-09-19T08:41:33.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:32 smithi170 ceph-mon[35072]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:33.133 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:32 smithi173 ceph-mon[35306]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:33.170 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:32 smithi164 ceph-mon[30928]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:35.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:34 smithi170 ceph-mon[35072]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:35.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:34 smithi173 ceph-mon[35306]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:35.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:34 smithi164 ceph-mon[30928]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:37.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:36 smithi173 ceph-mon[35306]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:37.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:36 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:41:37.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:36 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:37.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:36 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:37.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:36 smithi170 ceph-mon[35072]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:37.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:36 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:41:37.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:36 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:37.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:36 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:37.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:36 smithi164 ceph-mon[30928]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:37.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:36 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:41:37.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:36 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:37.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:36 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:38.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:38 smithi173 ceph-mon[35306]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:39.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:38 smithi170 ceph-mon[35072]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:39.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:38 smithi164 ceph-mon[30928]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:39.961 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:39.960+0000 7f79867fc640 1 -- 172.21.15.173:0/1993297004 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7988062630 con 0x7f79700a3270 2024-09-19T08:41:39.961 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 8 on host 'smithi173' 2024-09-19T08:41:39.963 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:39.962+0000 7f798f4c0640 1 -- 172.21.15.173:0/1993297004 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f79700a3270 msgr2=0x7f79700a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:39.964 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:39.962+0000 7f798f4c0640 1 --2- 172.21.15.173:0/1993297004 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f79700a3270 0x7f79700a5730 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7978006010 tx=0x7f7978000f40 comp rx=0 tx=0).stop 2024-09-19T08:41:39.964 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:39.962+0000 7f798f4c0640 1 -- 172.21.15.173:0/1993297004 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7988102e10 msgr2=0x7f7988198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:39.964 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:39.962+0000 7f798f4c0640 1 --2- 172.21.15.173:0/1993297004 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7988102e10 0x7f7988198470 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f797c00c890 tx=0x7f797c00cd50 comp rx=0 tx=0).stop 2024-09-19T08:41:39.964 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:39.962+0000 7f798f4c0640 1 -- 172.21.15.173:0/1993297004 shutdown_connections 2024-09-19T08:41:39.964 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:39.962+0000 7f798f4c0640 1 --2- 172.21.15.173:0/1993297004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79881037d0 0x7f798819f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:39.964 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:39.962+0000 7f798f4c0640 1 --2- 172.21.15.173:0/1993297004 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f79700a3270 0x7f79700a5730 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:39.964 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:39.962+0000 7f798f4c0640 1 --2- 172.21.15.173:0/1993297004 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7988102e10 0x7f7988198470 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:39.964 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:39.962+0000 7f798f4c0640 1 --2- 172.21.15.173:0/1993297004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7988101c10 0x7f7988197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:39.964 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:39.963+0000 7f798f4c0640 1 -- 172.21.15.173:0/1993297004 >> 172.21.15.173:0/1993297004 conn(0x7f79880fd3c0 msgr2=0x7f7988106230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:39.964 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:39.963+0000 7f798f4c0640 1 -- 172.21.15.173:0/1993297004 shutdown_connections 2024-09-19T08:41:39.964 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:41:39.963+0000 7f798f4c0640 1 -- 172.21.15.173:0/1993297004 wait complete. 2024-09-19T08:41:40.616 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:40 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:40.616 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:40 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:40.617 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:40 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:41:40.617 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:40 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:41:40.617 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:40 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:40.617 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:40 smithi173 ceph-mon[35306]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:40.617 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:40 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:41:40.617 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:40 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:40.617 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:40 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:40.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:40 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:40.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:40 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:40.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:40 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:41:40.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:40 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:41:40.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:40 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:40.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:40 smithi170 ceph-mon[35072]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:40.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:40 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:41:40.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:40 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:40.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:40 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:40.642 DEBUG:teuthology.orchestra.run.smithi173:osd.8> sudo journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.8.service 2024-09-19T08:41:40.644 INFO:tasks.cephadm:Deploying osd.9 on smithi164 with /dev/vg_nvme/lv_1... 2024-09-19T08:41:40.644 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T08:41:40.673 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:40 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:40.673 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:40 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:40.674 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:40 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:41:40.674 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:40 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:41:40.674 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:40 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:40.674 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:40 smithi164 ceph-mon[30928]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:40.674 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:40 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:41:40.674 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:40 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:40.674 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:40 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:40.899 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:41:42.661 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:41:42 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:41:42.389+0000 7f67eff2e640 -1 osd.8 0 log_to_monitors true 2024-09-19T08:41:43.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:42 smithi164 ceph-mon[30928]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:43.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:42 smithi164 ceph-mon[30928]: from='osd.8 [v2:172.21.15.173:6816/2285175061,v1:172.21.15.173:6817/2285175061]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T08:41:43.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:42 smithi164 ceph-mon[30928]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T08:41:43.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:42 smithi170 ceph-mon[35072]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:43.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:42 smithi170 ceph-mon[35072]: from='osd.8 [v2:172.21.15.173:6816/2285175061,v1:172.21.15.173:6817/2285175061]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T08:41:43.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:42 smithi170 ceph-mon[35072]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T08:41:43.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:42 smithi173 ceph-mon[35306]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:43.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:42 smithi173 ceph-mon[35306]: from='osd.8 [v2:172.21.15.173:6816/2285175061,v1:172.21.15.173:6817/2285175061]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T08:41:43.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:42 smithi173 ceph-mon[35306]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T08:41:43.616 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:41:43.643 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi164:vg_nvme/lv_1 2024-09-19T08:41:43.966 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:41:43.987 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:43 smithi173 ceph-mon[35306]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T08:41:43.987 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:43 smithi173 ceph-mon[35306]: osdmap e52: 9 total, 8 up, 9 in 2024-09-19T08:41:43.988 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:43 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:41:43.988 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:43 smithi173 ceph-mon[35306]: from='osd.8 [v2:172.21.15.173:6816/2285175061,v1:172.21.15.173:6817/2285175061]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-19T08:41:43.988 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:43 smithi173 ceph-mon[35306]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-19T08:41:43.988 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:43 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:43.988 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:43 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:43.988 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:43 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:41:43.988 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:43 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:41:43.988 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:43 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:41:43.988 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:43 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:43.988 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:43 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:41:43.988 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:43 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:41:43.989 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:43 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:44.034 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:43 smithi170 ceph-mon[35072]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T08:41:44.034 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:43 smithi170 ceph-mon[35072]: osdmap e52: 9 total, 8 up, 9 in 2024-09-19T08:41:44.035 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:43 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:41:44.035 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:43 smithi170 ceph-mon[35072]: from='osd.8 [v2:172.21.15.173:6816/2285175061,v1:172.21.15.173:6817/2285175061]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-19T08:41:44.035 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:43 smithi170 ceph-mon[35072]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-19T08:41:44.035 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:43 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:44.035 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:43 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:44.035 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:43 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:41:44.035 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:43 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:41:44.035 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:43 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:41:44.035 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:43 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:44.035 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:43 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:41:44.035 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:43 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:41:44.035 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:43 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:44.093 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:43 smithi164 ceph-mon[30928]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T08:41:44.093 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:43 smithi164 ceph-mon[30928]: osdmap e52: 9 total, 8 up, 9 in 2024-09-19T08:41:44.093 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:43 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:41:44.093 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:43 smithi164 ceph-mon[30928]: from='osd.8 [v2:172.21.15.173:6816/2285175061,v1:172.21.15.173:6817/2285175061]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-19T08:41:44.093 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:43 smithi164 ceph-mon[30928]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-19T08:41:44.093 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:43 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:44.094 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:43 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:44.094 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:43 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:41:44.094 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:43 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:41:44.094 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:43 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:41:44.094 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:43 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:44.094 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:43 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:41:44.094 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:43 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:41:44.094 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:43 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:41:44.410 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:41:44 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:41:44.223+0000 7f67ec6c5640 -1 osd.8 0 waiting for initial osdmap 2024-09-19T08:41:44.410 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:41:44 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:41:44.234+0000 7f67e3cd1640 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:41:44.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.444+0000 7fc844a77640 1 -- 172.21.15.164:0/2437082249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc840100b00 msgr2=0x7fc840100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:44.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.444+0000 7fc844a77640 1 --2- 172.21.15.164:0/2437082249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc840100b00 0x7fc840100f00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc834009900 tx=0x7fc83402a410 comp rx=0 tx=0).stop 2024-09-19T08:41:44.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.444+0000 7fc844a77640 1 -- 172.21.15.164:0/2437082249 shutdown_connections 2024-09-19T08:41:44.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.444+0000 7fc844a77640 1 --2- 172.21.15.164:0/2437082249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8401026c0 0x7fc840108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:44.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.444+0000 7fc844a77640 1 --2- 172.21.15.164:0/2437082249 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc840101d00 0x7fc840102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:44.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.444+0000 7fc844a77640 1 --2- 172.21.15.164:0/2437082249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc840100b00 0x7fc840100f00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:44.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.444+0000 7fc844a77640 1 -- 172.21.15.164:0/2437082249 >> 172.21.15.164:0/2437082249 conn(0x7fc8400fc290 msgr2=0x7fc8400fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:44.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.444+0000 7fc844a77640 1 -- 172.21.15.164:0/2437082249 shutdown_connections 2024-09-19T08:41:44.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.444+0000 7fc844a77640 1 -- 172.21.15.164:0/2437082249 wait complete. 2024-09-19T08:41:44.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.445+0000 7fc844a77640 1 Processor -- start 2024-09-19T08:41:44.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.445+0000 7fc844a77640 1 -- start start 2024-09-19T08:41:44.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.445+0000 7fc844a77640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc840100b00 0x7fc84006b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:44.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.445+0000 7fc844a77640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc840101d00 0x7fc84019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:44.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.445+0000 7fc844a77640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8401026c0 0x7fc84019f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:44.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.445+0000 7fc844a77640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc84010c780 con 0x7fc840100b00 2024-09-19T08:41:44.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.445+0000 7fc844a77640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc84010c600 con 0x7fc8401026c0 2024-09-19T08:41:44.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.445+0000 7fc844a77640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc84010c900 con 0x7fc840101d00 2024-09-19T08:41:44.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.445+0000 7fc83e59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc840100b00 0x7fc84006b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:44.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.445+0000 7fc83e59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc840100b00 0x7fc84006b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35640/0 (socket says 172.21.15.164:35640) 2024-09-19T08:41:44.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.445+0000 7fc83e59c640 1 -- 172.21.15.164:0/1465021808 learned_addr learned my addr 172.21.15.164:0/1465021808 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:41:44.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.445+0000 7fc83dd9b640 1 --2- 172.21.15.164:0/1465021808 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc840101d00 0x7fc84019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:44.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.445+0000 7fc83ed9d640 1 --2- 172.21.15.164:0/1465021808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8401026c0 0x7fc84019f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:44.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.445+0000 7fc83e59c640 1 -- 172.21.15.164:0/1465021808 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc840101d00 msgr2=0x7fc84019cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:44.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.445+0000 7fc83e59c640 1 --2- 172.21.15.164:0/1465021808 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc840101d00 0x7fc84019cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:44.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.445+0000 7fc83e59c640 1 -- 172.21.15.164:0/1465021808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8401026c0 msgr2=0x7fc84019f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:44.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.445+0000 7fc83e59c640 1 --2- 172.21.15.164:0/1465021808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8401026c0 0x7fc84019f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:44.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.445+0000 7fc83e59c640 1 -- 172.21.15.164:0/1465021808 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc84019fba0 con 0x7fc840100b00 2024-09-19T08:41:44.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.445+0000 7fc83ed9d640 1 --2- 172.21.15.164:0/1465021808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8401026c0 0x7fc84019f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:41:44.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.446+0000 7fc83e59c640 1 --2- 172.21.15.164:0/1465021808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc840100b00 0x7fc84006b2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc834031610 tx=0x7fc834004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:44.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.446+0000 7fc82f7fe640 1 -- 172.21.15.164:0/1465021808 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83403f030 con 0x7fc840100b00 2024-09-19T08:41:44.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.446+0000 7fc844a77640 1 -- 172.21.15.164:0/1465021808 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc84019fdd0 con 0x7fc840100b00 2024-09-19T08:41:44.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.446+0000 7fc82f7fe640 1 -- 172.21.15.164:0/1465021808 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc834003ad0 con 0x7fc840100b00 2024-09-19T08:41:44.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.446+0000 7fc82f7fe640 1 -- 172.21.15.164:0/1465021808 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc834035070 con 0x7fc840100b00 2024-09-19T08:41:44.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.446+0000 7fc844a77640 1 -- 172.21.15.164:0/1465021808 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8401a0220 con 0x7fc840100b00 2024-09-19T08:41:44.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.447+0000 7fc844a77640 1 -- 172.21.15.164:0/1465021808 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc840107f70 con 0x7fc840100b00 2024-09-19T08:41:44.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.448+0000 7fc82f7fe640 1 -- 172.21.15.164:0/1465021808 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fc834042030 con 0x7fc840100b00 2024-09-19T08:41:44.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.449+0000 7fc82f7fe640 1 --2- 172.21.15.164:0/1465021808 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fc8280a3240 0x7fc8280a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:44.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.449+0000 7fc83dd9b640 1 --2- 172.21.15.164:0/1465021808 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fc8280a3240 0x7fc8280a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:44.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.449+0000 7fc82f7fe640 1 -- 172.21.15.164:0/1465021808 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5549+0+0 (secure 0 0 0) 0x7fc8340ef630 con 0x7fc840100b00 2024-09-19T08:41:44.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.449+0000 7fc83dd9b640 1 --2- 172.21.15.164:0/1465021808 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fc8280a3240 0x7fc8280a5700 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc84006b870 tx=0x7fc830005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:44.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.452+0000 7fc82f7fe640 1 -- 172.21.15.164:0/1465021808 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc8340bc7d0 con 0x7fc840100b00 2024-09-19T08:41:44.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:41:44.636+0000 7fc844a77640 1 -- 172.21.15.164:0/1465021808 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fc840062630 con 0x7fc8280a3240 2024-09-19T08:41:45.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:44 smithi164 ceph-mon[30928]: Detected new or changed devices on smithi173 2024-09-19T08:41:45.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:44 smithi164 ceph-mon[30928]: Adjusting osd_memory_target on smithi173 to 5696M 2024-09-19T08:41:45.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:44 smithi164 ceph-mon[30928]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:45.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:44 smithi164 ceph-mon[30928]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-19T08:41:45.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:44 smithi164 ceph-mon[30928]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T08:41:45.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:44 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:41:45.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:44 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:41:45.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:44 smithi164 ceph-mon[30928]: from='osd.8 ' entity='osd.8' 2024-09-19T08:41:45.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:44 smithi164 ceph-mon[30928]: osd.8 [v2:172.21.15.173:6816/2285175061,v1:172.21.15.173:6817/2285175061] boot 2024-09-19T08:41:45.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:44 smithi164 ceph-mon[30928]: osdmap e54: 9 total, 9 up, 9 in 2024-09-19T08:41:45.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:44 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:41:45.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:44 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:41:45.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:44 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:41:45.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:44 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:41:45.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:44 smithi170 ceph-mon[35072]: Detected new or changed devices on smithi173 2024-09-19T08:41:45.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:44 smithi170 ceph-mon[35072]: Adjusting osd_memory_target on smithi173 to 5696M 2024-09-19T08:41:45.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:44 smithi170 ceph-mon[35072]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:45.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:44 smithi170 ceph-mon[35072]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-19T08:41:45.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:44 smithi170 ceph-mon[35072]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T08:41:45.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:44 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:41:45.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:44 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:41:45.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:44 smithi170 ceph-mon[35072]: from='osd.8 ' entity='osd.8' 2024-09-19T08:41:45.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:44 smithi170 ceph-mon[35072]: osd.8 [v2:172.21.15.173:6816/2285175061,v1:172.21.15.173:6817/2285175061] boot 2024-09-19T08:41:45.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:44 smithi170 ceph-mon[35072]: osdmap e54: 9 total, 9 up, 9 in 2024-09-19T08:41:45.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:44 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:41:45.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:44 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:41:45.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:44 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:41:45.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:44 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:41:45.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:44 smithi173 ceph-mon[35306]: Detected new or changed devices on smithi173 2024-09-19T08:41:45.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:44 smithi173 ceph-mon[35306]: Adjusting osd_memory_target on smithi173 to 5696M 2024-09-19T08:41:45.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:44 smithi173 ceph-mon[35306]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-09-19T08:41:45.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:44 smithi173 ceph-mon[35306]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-19T08:41:45.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:44 smithi173 ceph-mon[35306]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T08:41:45.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:44 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:41:45.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:44 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:41:45.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:44 smithi173 ceph-mon[35306]: from='osd.8 ' entity='osd.8' 2024-09-19T08:41:45.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:44 smithi173 ceph-mon[35306]: osd.8 [v2:172.21.15.173:6816/2285175061,v1:172.21.15.173:6817/2285175061] boot 2024-09-19T08:41:45.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:44 smithi173 ceph-mon[35306]: osdmap e54: 9 total, 9 up, 9 in 2024-09-19T08:41:45.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:44 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:41:45.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:44 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:41:45.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:44 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:41:45.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:44 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:41:46.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:45 smithi164 ceph-mon[30928]: purged_snaps scrub starts 2024-09-19T08:41:46.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:45 smithi164 ceph-mon[30928]: purged_snaps scrub ok 2024-09-19T08:41:46.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:45 smithi164 ceph-mon[30928]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:41:46.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:45 smithi164 ceph-mon[30928]: osdmap e55: 9 total, 9 up, 9 in 2024-09-19T08:41:46.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:45 smithi170 ceph-mon[35072]: purged_snaps scrub starts 2024-09-19T08:41:46.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:45 smithi170 ceph-mon[35072]: purged_snaps scrub ok 2024-09-19T08:41:46.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:45 smithi170 ceph-mon[35072]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:41:46.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:45 smithi170 ceph-mon[35072]: osdmap e55: 9 total, 9 up, 9 in 2024-09-19T08:41:46.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:45 smithi173 ceph-mon[35306]: purged_snaps scrub starts 2024-09-19T08:41:46.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:45 smithi173 ceph-mon[35306]: purged_snaps scrub ok 2024-09-19T08:41:46.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:45 smithi173 ceph-mon[35306]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:41:46.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:45 smithi173 ceph-mon[35306]: osdmap e55: 9 total, 9 up, 9 in 2024-09-19T08:41:46.840 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:46 smithi164 ceph-mon[30928]: pgmap v187: 1 pgs: 1 remapped+peering; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:41:46.841 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:46 smithi164 ceph-mon[30928]: osdmap e56: 9 total, 9 up, 9 in 2024-09-19T08:41:47.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:46 smithi170 ceph-mon[35072]: pgmap v187: 1 pgs: 1 remapped+peering; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:41:47.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:46 smithi170 ceph-mon[35072]: osdmap e56: 9 total, 9 up, 9 in 2024-09-19T08:41:47.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:46 smithi173 ceph-mon[35306]: pgmap v187: 1 pgs: 1 remapped+peering; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:41:47.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:46 smithi173 ceph-mon[35306]: osdmap e56: 9 total, 9 up, 9 in 2024-09-19T08:41:48.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:47 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/627709990' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "994666e8-8fbd-4231-ac67-81a79aaf7c2a"}]: dispatch 2024-09-19T08:41:48.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:47 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/627709990' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "994666e8-8fbd-4231-ac67-81a79aaf7c2a"}]': finished 2024-09-19T08:41:48.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:47 smithi164 ceph-mon[30928]: osdmap e57: 10 total, 9 up, 10 in 2024-09-19T08:41:48.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:47 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:41:48.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:47 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3499240454' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:41:48.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:47 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/627709990' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "994666e8-8fbd-4231-ac67-81a79aaf7c2a"}]: dispatch 2024-09-19T08:41:48.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:47 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/627709990' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "994666e8-8fbd-4231-ac67-81a79aaf7c2a"}]': finished 2024-09-19T08:41:48.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:47 smithi170 ceph-mon[35072]: osdmap e57: 10 total, 9 up, 10 in 2024-09-19T08:41:48.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:47 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:41:48.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:47 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3499240454' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:41:48.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:47 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/627709990' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "994666e8-8fbd-4231-ac67-81a79aaf7c2a"}]: dispatch 2024-09-19T08:41:48.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:47 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/627709990' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "994666e8-8fbd-4231-ac67-81a79aaf7c2a"}]': finished 2024-09-19T08:41:48.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:47 smithi173 ceph-mon[35306]: osdmap e57: 10 total, 9 up, 10 in 2024-09-19T08:41:48.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:47 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:41:48.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:47 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3499240454' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:41:49.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:48 smithi170 ceph-mon[35072]: pgmap v190: 1 pgs: 1 remapped+peering; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:41:49.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:48 smithi170 ceph-mon[35072]: osdmap e58: 10 total, 9 up, 10 in 2024-09-19T08:41:49.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:48 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:41:49.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:48 smithi173 ceph-mon[35306]: pgmap v190: 1 pgs: 1 remapped+peering; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:41:49.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:48 smithi173 ceph-mon[35306]: osdmap e58: 10 total, 9 up, 10 in 2024-09-19T08:41:49.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:48 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:41:49.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:48 smithi164 ceph-mon[30928]: pgmap v190: 1 pgs: 1 remapped+peering; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:41:49.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:48 smithi164 ceph-mon[30928]: osdmap e58: 10 total, 9 up, 10 in 2024-09-19T08:41:49.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:48 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:41:51.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:50 smithi170 ceph-mon[35072]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:41:51.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:50 smithi173 ceph-mon[35306]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:41:51.281 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:50 smithi164 ceph-mon[30928]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:41:53.001 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:52 smithi164 ceph-mon[30928]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T08:41:53.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:52 smithi170 ceph-mon[35072]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T08:41:53.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:52 smithi173 ceph-mon[35306]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T08:41:55.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:54 smithi164 ceph-mon[30928]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-19T08:41:55.054 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:54 smithi170 ceph-mon[35072]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-19T08:41:55.054 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:54 smithi173 ceph-mon[35306]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-19T08:41:56.993 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:56 smithi164 ceph-mon[30928]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-19T08:41:56.993 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:56 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T08:41:56.994 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:56 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:41:57.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:56 smithi170 ceph-mon[35072]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-19T08:41:57.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:56 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T08:41:57.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:56 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:41:57.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:56 smithi173 ceph-mon[35306]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-19T08:41:57.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:56 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T08:41:57.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:56 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:41:58.021 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:57 smithi164 ceph-mon[30928]: Deploying daemon osd.9 on smithi164 2024-09-19T08:41:58.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:57 smithi170 ceph-mon[35072]: Deploying daemon osd.9 on smithi164 2024-09-19T08:41:58.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:57 smithi173 ceph-mon[35306]: Deploying daemon osd.9 on smithi164 2024-09-19T08:41:58.883 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:41:58 smithi164 ceph-mon[30928]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T08:41:59.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:41:58 smithi170 ceph-mon[35072]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T08:41:59.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:41:58 smithi173 ceph-mon[35306]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T08:42:01.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:00 smithi170 ceph-mon[35072]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-19T08:42:01.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:00 smithi173 ceph-mon[35306]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-19T08:42:01.236 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:00 smithi164 ceph-mon[30928]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-19T08:42:02.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:01 smithi164 ceph-mon[30928]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T08:42:02.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:01 smithi170 ceph-mon[35072]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T08:42:02.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:01 smithi173 ceph-mon[35306]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T08:42:02.906 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:02 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:42:02.907 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:02 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:02.907 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:02 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:03.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:02 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:42:03.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:02 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:03.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:02 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:03.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:02 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:42:03.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:02 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:03.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:02 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:03.918 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:03 smithi164 ceph-mon[30928]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:42:04.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:03 smithi170 ceph-mon[35072]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:42:04.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:03 smithi173 ceph-mon[35306]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:42:04.920 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 9 on host 'smithi164' 2024-09-19T08:42:04.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:04.920+0000 7fc82f7fe640 1 -- 172.21.15.164:0/1465021808 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc840062630 con 0x7fc8280a3240 2024-09-19T08:42:04.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:04.922+0000 7fc844a77640 1 -- 172.21.15.164:0/1465021808 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fc8280a3240 msgr2=0x7fc8280a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:04.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:04.922+0000 7fc844a77640 1 --2- 172.21.15.164:0/1465021808 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fc8280a3240 0x7fc8280a5700 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc84006b870 tx=0x7fc830005f90 comp rx=0 tx=0).stop 2024-09-19T08:42:04.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:04.922+0000 7fc844a77640 1 -- 172.21.15.164:0/1465021808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc840100b00 msgr2=0x7fc84006b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:04.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:04.922+0000 7fc844a77640 1 --2- 172.21.15.164:0/1465021808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc840100b00 0x7fc84006b2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc834031610 tx=0x7fc834004270 comp rx=0 tx=0).stop 2024-09-19T08:42:04.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:04.922+0000 7fc844a77640 1 -- 172.21.15.164:0/1465021808 shutdown_connections 2024-09-19T08:42:04.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:04.922+0000 7fc844a77640 1 --2- 172.21.15.164:0/1465021808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8401026c0 0x7fc84019f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:04.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:04.922+0000 7fc844a77640 1 --2- 172.21.15.164:0/1465021808 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fc8280a3240 0x7fc8280a5700 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:04.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:04.922+0000 7fc844a77640 1 --2- 172.21.15.164:0/1465021808 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc840101d00 0x7fc84019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:04.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:04.922+0000 7fc844a77640 1 --2- 172.21.15.164:0/1465021808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc840100b00 0x7fc84006b2f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:04.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:04.922+0000 7fc844a77640 1 -- 172.21.15.164:0/1465021808 >> 172.21.15.164:0/1465021808 conn(0x7fc8400fc290 msgr2=0x7fc840105140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:04.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:04.922+0000 7fc844a77640 1 -- 172.21.15.164:0/1465021808 shutdown_connections 2024-09-19T08:42:04.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:04.922+0000 7fc844a77640 1 -- 172.21.15.164:0/1465021808 wait complete. 2024-09-19T08:42:05.960 DEBUG:teuthology.orchestra.run.smithi164:osd.9> sudo journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.9.service 2024-09-19T08:42:05.962 INFO:tasks.cephadm:Deploying osd.10 on smithi170 with /dev/vg_nvme/lv_1... 2024-09-19T08:42:05.962 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T08:42:06.222 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T08:42:06.281 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:05 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:06.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:05 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:06.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:05 smithi164 ceph-mon[30928]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:42:06.343 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:05 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:06.343 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:05 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:06.343 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:05 smithi170 ceph-mon[35072]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:42:06.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:05 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:06.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:05 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:06.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:05 smithi173 ceph-mon[35306]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:42:07.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:07 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:07.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:07 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:07.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:07 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:07.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:07 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:07.549 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:07 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:07.550 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:07 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:08.151 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:42:07 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:42:07.959+0000 7f428a420640 -1 osd.9 0 log_to_monitors true 2024-09-19T08:42:08.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:08 smithi173 ceph-mon[35306]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:42:08.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:08 smithi173 ceph-mon[35306]: from='osd.9 [v2:172.21.15.164:6826/3918847952,v1:172.21.15.164:6827/3918847952]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T08:42:08.467 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:08 smithi170 ceph-mon[35072]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:42:08.467 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:08 smithi170 ceph-mon[35072]: from='osd.9 [v2:172.21.15.164:6826/3918847952,v1:172.21.15.164:6827/3918847952]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T08:42:08.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:08 smithi164 ceph-mon[30928]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:42:08.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:08 smithi164 ceph-mon[30928]: from='osd.9 [v2:172.21.15.164:6826/3918847952,v1:172.21.15.164:6827/3918847952]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T08:42:08.877 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-19T08:42:08.910 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi170:vg_nvme/lv_1 2024-09-19T08:42:09.159 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T08:42:09.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:09 smithi173 ceph-mon[35306]: from='osd.9 [v2:172.21.15.164:6826/3918847952,v1:172.21.15.164:6827/3918847952]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T08:42:09.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:09 smithi173 ceph-mon[35306]: osdmap e59: 10 total, 9 up, 10 in 2024-09-19T08:42:09.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:09 smithi173 ceph-mon[35306]: from='osd.9 [v2:172.21.15.164:6826/3918847952,v1:172.21.15.164:6827/3918847952]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-19T08:42:09.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:09 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:42:09.461 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:09 smithi170 ceph-mon[35072]: from='osd.9 [v2:172.21.15.164:6826/3918847952,v1:172.21.15.164:6827/3918847952]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T08:42:09.461 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:09 smithi170 ceph-mon[35072]: osdmap e59: 10 total, 9 up, 10 in 2024-09-19T08:42:09.461 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:09 smithi170 ceph-mon[35072]: from='osd.9 [v2:172.21.15.164:6826/3918847952,v1:172.21.15.164:6827/3918847952]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-19T08:42:09.461 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:09 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:42:09.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:09 smithi164 ceph-mon[30928]: from='osd.9 [v2:172.21.15.164:6826/3918847952,v1:172.21.15.164:6827/3918847952]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T08:42:09.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:09 smithi164 ceph-mon[30928]: osdmap e59: 10 total, 9 up, 10 in 2024-09-19T08:42:09.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:09 smithi164 ceph-mon[30928]: from='osd.9 [v2:172.21.15.164:6826/3918847952,v1:172.21.15.164:6827/3918847952]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-09-19T08:42:09.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:09 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:42:09.532 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:42:09 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:42:09.157+0000 7f42863a3640 -1 osd.9 0 waiting for initial osdmap 2024-09-19T08:42:09.532 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:42:09 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:42:09.167+0000 7f42821cb640 -1 osd.9 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:42:09.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.701+0000 7f776dcb7640 1 -- 172.21.15.170:0/2022796095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7768107900 msgr2=0x7f7768109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:09.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.701+0000 7f776dcb7640 1 --2- 172.21.15.170:0/2022796095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7768107900 0x7f7768109d50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7760009ca0 tx=0x7f776002a8c0 comp rx=0 tx=0).stop 2024-09-19T08:42:09.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.701+0000 7f776dcb7640 1 -- 172.21.15.170:0/2022796095 shutdown_connections 2024-09-19T08:42:09.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.701+0000 7f776dcb7640 1 --2- 172.21.15.170:0/2022796095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7768107900 0x7f7768109d50 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:09.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.701+0000 7f776dcb7640 1 --2- 172.21.15.170:0/2022796095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7768104fd0 0x7f77681073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:09.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.701+0000 7f776dcb7640 1 --2- 172.21.15.170:0/2022796095 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f77681026a0 0x7f7768104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:09.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.701+0000 7f776dcb7640 1 -- 172.21.15.170:0/2022796095 >> 172.21.15.170:0/2022796095 conn(0x7f77680fc2a0 msgr2=0x7f77680fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:09.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.701+0000 7f776dcb7640 1 -- 172.21.15.170:0/2022796095 shutdown_connections 2024-09-19T08:42:09.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.701+0000 7f776dcb7640 1 -- 172.21.15.170:0/2022796095 wait complete. 2024-09-19T08:42:09.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.702+0000 7f776dcb7640 1 Processor -- start 2024-09-19T08:42:09.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.702+0000 7f776dcb7640 1 -- start start 2024-09-19T08:42:09.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.702+0000 7f776dcb7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77681026a0 0x7f7768198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:09.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.702+0000 7f776dcb7640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7768104fd0 0x7f7768198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:09.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.702+0000 7f776dcb7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7768107900 0x7f776819f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:09.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.702+0000 7f776dcb7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f776810d990 con 0x7f7768107900 2024-09-19T08:42:09.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.702+0000 7f776dcb7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f776810d810 con 0x7f77681026a0 2024-09-19T08:42:09.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.702+0000 7f776dcb7640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f776810db10 con 0x7f7768104fd0 2024-09-19T08:42:09.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.702+0000 7f77677fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77681026a0 0x7f7768198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:09.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.702+0000 7f77677fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77681026a0 0x7f7768198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51854/0 (socket says 172.21.15.170:51854) 2024-09-19T08:42:09.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.702+0000 7f77677fe640 1 -- 172.21.15.170:0/1114233775 learned_addr learned my addr 172.21.15.170:0/1114233775 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-19T08:42:09.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.702+0000 7f7767fff640 1 --2- 172.21.15.170:0/1114233775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7768107900 0x7f776819f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:09.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.703+0000 7f77677fe640 1 -- 172.21.15.170:0/1114233775 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7768104fd0 msgr2=0x7f7768198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:09.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.703+0000 7f7766ffd640 1 --2- 172.21.15.170:0/1114233775 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7768104fd0 0x7f7768198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:09.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.703+0000 7f77677fe640 1 --2- 172.21.15.170:0/1114233775 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7768104fd0 0x7f7768198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:09.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.703+0000 7f77677fe640 1 -- 172.21.15.170:0/1114233775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7768107900 msgr2=0x7f776819f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:09.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.703+0000 7f77677fe640 1 --2- 172.21.15.170:0/1114233775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7768107900 0x7f776819f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:09.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.703+0000 7f77677fe640 1 -- 172.21.15.170:0/1114233775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776819fc70 con 0x7f77681026a0 2024-09-19T08:42:09.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.703+0000 7f7767fff640 1 --2- 172.21.15.170:0/1114233775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7768107900 0x7f776819f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:09.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.703+0000 7f7766ffd640 1 --2- 172.21.15.170:0/1114233775 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7768104fd0 0x7f7768198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:09.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.703+0000 7f77677fe640 1 --2- 172.21.15.170:0/1114233775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77681026a0 0x7f7768198040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f775800b4a0 tx=0x7f775800b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:09.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.703+0000 7f7764ff9640 1 -- 172.21.15.170:0/1114233775 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7758014070 con 0x7f77681026a0 2024-09-19T08:42:09.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.703+0000 7f7764ff9640 1 -- 172.21.15.170:0/1114233775 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7758010af0 con 0x7f77681026a0 2024-09-19T08:42:09.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.703+0000 7f7764ff9640 1 -- 172.21.15.170:0/1114233775 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7758017d10 con 0x7f77681026a0 2024-09-19T08:42:09.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.703+0000 7f776dcb7640 1 -- 172.21.15.170:0/1114233775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f776819ff60 con 0x7f77681026a0 2024-09-19T08:42:09.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.703+0000 7f776dcb7640 1 -- 172.21.15.170:0/1114233775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77681a03e0 con 0x7f77681026a0 2024-09-19T08:42:09.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.704+0000 7f776dcb7640 1 -- 172.21.15.170:0/1114233775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77680fdd90 con 0x7f77681026a0 2024-09-19T08:42:09.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.705+0000 7f7764ff9640 1 -- 172.21.15.170:0/1114233775 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f7758017070 con 0x7f77681026a0 2024-09-19T08:42:09.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.706+0000 7f7764ff9640 1 --2- 172.21.15.170:0/1114233775 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f77500a3270 0x7f77500a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:09.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.706+0000 7f7764ff9640 1 -- 172.21.15.170:0/1114233775 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5981+0+0 (secure 0 0 0) 0x7f77580ce480 con 0x7f77681026a0 2024-09-19T08:42:09.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.706+0000 7f7766ffd640 1 --2- 172.21.15.170:0/1114233775 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f77500a3270 0x7f77500a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:09.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.707+0000 7f7766ffd640 1 --2- 172.21.15.170:0/1114233775 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f77500a3270 0x7f77500a5730 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f775c006fd0 tx=0x7f775c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:09.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.710+0000 7f7764ff9640 1 -- 172.21.15.170:0/1114233775 <== mon.2 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f775809bac0 con 0x7f77681026a0 2024-09-19T08:42:09.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:09.887+0000 7f776dcb7640 1 -- 172.21.15.170:0/1114233775 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f7768062630 con 0x7f77500a3270 2024-09-19T08:42:10.444 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:10 smithi170 ceph-mon[35072]: from='osd.9 [v2:172.21.15.164:6826/3918847952,v1:172.21.15.164:6827/3918847952]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-19T08:42:10.444 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:10 smithi170 ceph-mon[35072]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T08:42:10.444 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:10 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:42:10.444 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:10 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:42:10.444 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:10 smithi170 ceph-mon[35072]: Detected new or changed devices on smithi164 2024-09-19T08:42:10.444 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:10 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:10.445 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:10 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:10.445 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:10 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:42:10.445 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:10 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:42:10.445 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:10 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:42:10.445 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:10 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:42:10.445 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:10 smithi170 ceph-mon[35072]: Adjusting osd_memory_target on smithi164 to 4272M 2024-09-19T08:42:10.445 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:10 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:10.445 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:10 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:42:10.445 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:10 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:42:10.445 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:10 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:10.445 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:10 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:42:10.445 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:10 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:42:10.445 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:10 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:42:10.446 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:10 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:10.446 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:10 smithi170 ceph-mon[35072]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:42:10.446 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:10 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:42:10.446 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:10 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:42:10.446 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:10 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:42:10.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:10 smithi164 ceph-mon[30928]: from='osd.9 [v2:172.21.15.164:6826/3918847952,v1:172.21.15.164:6827/3918847952]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-19T08:42:10.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:10 smithi164 ceph-mon[30928]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T08:42:10.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:10 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:42:10.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:10 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:42:10.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:10 smithi164 ceph-mon[30928]: Detected new or changed devices on smithi164 2024-09-19T08:42:10.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:10 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:10.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:10 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:10.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:10 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:42:10.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:10 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:42:10.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:10 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:42:10.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:10 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:42:10.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:10 smithi164 ceph-mon[30928]: Adjusting osd_memory_target on smithi164 to 4272M 2024-09-19T08:42:10.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:10 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:10.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:10 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:42:10.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:10 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:42:10.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:10 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:10.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:10 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:42:10.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:10 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:42:10.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:10 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:42:10.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:10 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:10.534 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:10 smithi164 ceph-mon[30928]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:42:10.534 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:10 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:42:10.534 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:10 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:42:10.534 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:10 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:42:10.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:10 smithi173 ceph-mon[35306]: from='osd.9 [v2:172.21.15.164:6826/3918847952,v1:172.21.15.164:6827/3918847952]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-09-19T08:42:10.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:10 smithi173 ceph-mon[35306]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T08:42:10.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:10 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:42:10.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:10 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:42:10.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:10 smithi173 ceph-mon[35306]: Detected new or changed devices on smithi164 2024-09-19T08:42:10.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:10 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:10.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:10 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:10.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:10 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:42:10.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:10 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:42:10.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:10 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:42:10.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:10 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:42:10.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:10 smithi173 ceph-mon[35306]: Adjusting osd_memory_target on smithi164 to 4272M 2024-09-19T08:42:10.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:10 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:10.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:10 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:42:10.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:10 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:42:10.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:10 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:10.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:10 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:42:10.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:10 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:42:10.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:10 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:42:10.663 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:10 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:10.663 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:10 smithi173 ceph-mon[35306]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-19T08:42:10.663 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:10 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:42:10.663 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:10 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:42:10.663 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:10 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:42:11.356 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:11 smithi170 ceph-mon[35072]: purged_snaps scrub starts 2024-09-19T08:42:11.356 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:11 smithi170 ceph-mon[35072]: purged_snaps scrub ok 2024-09-19T08:42:11.357 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:11 smithi170 ceph-mon[35072]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:42:11.357 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:11 smithi170 ceph-mon[35072]: osd.9 [v2:172.21.15.164:6826/3918847952,v1:172.21.15.164:6827/3918847952] boot 2024-09-19T08:42:11.357 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:11 smithi170 ceph-mon[35072]: osdmap e61: 10 total, 10 up, 10 in 2024-09-19T08:42:11.357 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:11 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:42:11.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:11 smithi164 ceph-mon[30928]: purged_snaps scrub starts 2024-09-19T08:42:11.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:11 smithi164 ceph-mon[30928]: purged_snaps scrub ok 2024-09-19T08:42:11.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:11 smithi164 ceph-mon[30928]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:42:11.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:11 smithi164 ceph-mon[30928]: osd.9 [v2:172.21.15.164:6826/3918847952,v1:172.21.15.164:6827/3918847952] boot 2024-09-19T08:42:11.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:11 smithi164 ceph-mon[30928]: osdmap e61: 10 total, 10 up, 10 in 2024-09-19T08:42:11.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:11 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:42:11.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:11 smithi173 ceph-mon[35306]: purged_snaps scrub starts 2024-09-19T08:42:11.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:11 smithi173 ceph-mon[35306]: purged_snaps scrub ok 2024-09-19T08:42:11.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:11 smithi173 ceph-mon[35306]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:42:11.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:11 smithi173 ceph-mon[35306]: osd.9 [v2:172.21.15.164:6826/3918847952,v1:172.21.15.164:6827/3918847952] boot 2024-09-19T08:42:11.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:11 smithi173 ceph-mon[35306]: osdmap e61: 10 total, 10 up, 10 in 2024-09-19T08:42:11.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:11 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:42:12.711 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:12 smithi170 ceph-mon[35072]: osdmap e62: 10 total, 10 up, 10 in 2024-09-19T08:42:12.711 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:12 smithi170 ceph-mon[35072]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:12.711 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:12 smithi170 ceph-mon[35072]: from='client.? 172.21.15.170:0/2973994940' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff5e9099-cdb4-4905-b888-5b67320ef76b"}]: dispatch 2024-09-19T08:42:12.711 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:12 smithi170 ceph-mon[35072]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff5e9099-cdb4-4905-b888-5b67320ef76b"}]: dispatch 2024-09-19T08:42:12.711 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:12 smithi170 ceph-mon[35072]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff5e9099-cdb4-4905-b888-5b67320ef76b"}]': finished 2024-09-19T08:42:12.711 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:12 smithi170 ceph-mon[35072]: osdmap e63: 11 total, 10 up, 11 in 2024-09-19T08:42:12.711 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:12 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:42:12.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:12 smithi164 ceph-mon[30928]: osdmap e62: 10 total, 10 up, 10 in 2024-09-19T08:42:12.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:12 smithi164 ceph-mon[30928]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:12.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:12 smithi164 ceph-mon[30928]: from='client.? 172.21.15.170:0/2973994940' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff5e9099-cdb4-4905-b888-5b67320ef76b"}]: dispatch 2024-09-19T08:42:12.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:12 smithi164 ceph-mon[30928]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff5e9099-cdb4-4905-b888-5b67320ef76b"}]: dispatch 2024-09-19T08:42:12.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:12 smithi164 ceph-mon[30928]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff5e9099-cdb4-4905-b888-5b67320ef76b"}]': finished 2024-09-19T08:42:12.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:12 smithi164 ceph-mon[30928]: osdmap e63: 11 total, 10 up, 11 in 2024-09-19T08:42:12.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:12 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:42:12.828 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:12 smithi173 ceph-mon[35306]: osdmap e62: 10 total, 10 up, 10 in 2024-09-19T08:42:12.828 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:12 smithi173 ceph-mon[35306]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:12.828 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:12 smithi173 ceph-mon[35306]: from='client.? 172.21.15.170:0/2973994940' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff5e9099-cdb4-4905-b888-5b67320ef76b"}]: dispatch 2024-09-19T08:42:12.828 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:12 smithi173 ceph-mon[35306]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff5e9099-cdb4-4905-b888-5b67320ef76b"}]: dispatch 2024-09-19T08:42:12.828 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:12 smithi173 ceph-mon[35306]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff5e9099-cdb4-4905-b888-5b67320ef76b"}]': finished 2024-09-19T08:42:12.829 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:12 smithi173 ceph-mon[35306]: osdmap e63: 11 total, 10 up, 11 in 2024-09-19T08:42:12.829 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:12 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:42:13.770 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:13 smithi164 ceph-mon[30928]: from='client.? 172.21.15.170:0/3763951976' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:42:13.784 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:13 smithi173 ceph-mon[35306]: from='client.? 172.21.15.170:0/3763951976' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:42:13.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:13 smithi170 ceph-mon[35072]: from='client.? 172.21.15.170:0/3763951976' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:42:14.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:14 smithi164 ceph-mon[30928]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:14.874 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:14 smithi170 ceph-mon[35072]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:14.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:14 smithi173 ceph-mon[35306]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:17.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:16 smithi164 ceph-mon[30928]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:17.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:16 smithi170 ceph-mon[35072]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:17.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:16 smithi173 ceph-mon[35306]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:19.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:18 smithi164 ceph-mon[30928]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:19.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:18 smithi170 ceph-mon[35072]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:19.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:18 smithi173 ceph-mon[35306]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:20.855 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:20 smithi170 ceph-mon[35072]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:21.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:20 smithi164 ceph-mon[30928]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:21.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:20 smithi173 ceph-mon[35306]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:21.995 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:21 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T08:42:21.995 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:21 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:42:22.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:21 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T08:42:22.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:21 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:42:22.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:21 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T08:42:22.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:21 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:42:23.075 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:22 smithi170 ceph-mon[35072]: Deploying daemon osd.10 on smithi170 2024-09-19T08:42:23.076 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:22 smithi170 ceph-mon[35072]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:23.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:22 smithi173 ceph-mon[35306]: Deploying daemon osd.10 on smithi170 2024-09-19T08:42:23.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:22 smithi173 ceph-mon[35306]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:23.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:22 smithi164 ceph-mon[30928]: Deploying daemon osd.10 on smithi170 2024-09-19T08:42:23.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:22 smithi164 ceph-mon[30928]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:25.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:24 smithi170 ceph-mon[35072]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:25.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:24 smithi173 ceph-mon[35306]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:25.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:24 smithi164 ceph-mon[30928]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:27.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:26 smithi170 ceph-mon[35072]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:27.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:26 smithi173 ceph-mon[35306]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:27.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:26 smithi164 ceph-mon[30928]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:28.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:27 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:42:28.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:27 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:28.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:27 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:28.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:27 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:42:28.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:27 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:28.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:27 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:28.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:27 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:42:28.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:27 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:28.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:27 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:29.050 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:28 smithi170 ceph-mon[35072]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:29.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:28 smithi173 ceph-mon[35306]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:29.281 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:28 smithi164 ceph-mon[30928]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:30.383 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 10 on host 'smithi170' 2024-09-19T08:42:30.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:30.382+0000 7f7764ff9640 1 -- 172.21.15.170:0/1114233775 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f7768062630 con 0x7f77500a3270 2024-09-19T08:42:30.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:30.385+0000 7f776dcb7640 1 -- 172.21.15.170:0/1114233775 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f77500a3270 msgr2=0x7f77500a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:30.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:30.385+0000 7f776dcb7640 1 --2- 172.21.15.170:0/1114233775 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f77500a3270 0x7f77500a5730 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f775c006fd0 tx=0x7f775c008040 comp rx=0 tx=0).stop 2024-09-19T08:42:30.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:30.385+0000 7f776dcb7640 1 -- 172.21.15.170:0/1114233775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77681026a0 msgr2=0x7f7768198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:30.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:30.385+0000 7f776dcb7640 1 --2- 172.21.15.170:0/1114233775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77681026a0 0x7f7768198040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f775800b4a0 tx=0x7f775800b960 comp rx=0 tx=0).stop 2024-09-19T08:42:30.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:30.385+0000 7f776dcb7640 1 -- 172.21.15.170:0/1114233775 shutdown_connections 2024-09-19T08:42:30.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:30.385+0000 7f776dcb7640 1 --2- 172.21.15.170:0/1114233775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7768107900 0x7f776819f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:30.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:30.385+0000 7f776dcb7640 1 --2- 172.21.15.170:0/1114233775 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f77500a3270 0x7f77500a5730 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:30.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:30.385+0000 7f776dcb7640 1 --2- 172.21.15.170:0/1114233775 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7768104fd0 0x7f7768198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:30.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:30.385+0000 7f776dcb7640 1 --2- 172.21.15.170:0/1114233775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77681026a0 0x7f7768198040 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:30.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:30.385+0000 7f776dcb7640 1 -- 172.21.15.170:0/1114233775 >> 172.21.15.170:0/1114233775 conn(0x7f77680fc2a0 msgr2=0x7f7768103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:30.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:30.385+0000 7f776dcb7640 1 -- 172.21.15.170:0/1114233775 shutdown_connections 2024-09-19T08:42:30.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-19T08:42:30.385+0000 7f776dcb7640 1 -- 172.21.15.170:0/1114233775 wait complete. 2024-09-19T08:42:31.003 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:30 smithi170 ceph-mon[35072]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:31.003 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:30 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:31.003 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:30 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:31.030 DEBUG:teuthology.orchestra.run.smithi170:osd.10> sudo journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.10.service 2024-09-19T08:42:31.033 INFO:tasks.cephadm:Deploying osd.11 on smithi173 with /dev/vg_nvme/lv_1... 2024-09-19T08:42:31.033 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T08:42:31.061 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:30 smithi173 ceph-mon[35306]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:31.062 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:30 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:31.062 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:30 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:31.084 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:30 smithi164 ceph-mon[30928]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:31.084 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:30 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:31.085 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:30 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:31.290 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:42:32.223 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:32 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:32.223 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:32 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:32.224 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:32 smithi170 ceph-mon[35072]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:32.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:32 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:32.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:32 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:32.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:32 smithi164 ceph-mon[30928]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:32.613 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:32 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:32.613 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:32 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:32.613 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:32 smithi173 ceph-mon[35306]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:33.255 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:33 smithi170 ceph-mon[35072]: from='osd.10 [v2:172.21.15.170:6824/2707848227,v1:172.21.15.170:6825/2707848227]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T08:42:33.256 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:33 smithi170 ceph-mon[35072]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T08:42:33.256 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:42:33 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:42:33.049+0000 7f1752a8a640 -1 osd.10 0 log_to_monitors true 2024-09-19T08:42:33.434 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:33 smithi173 ceph-mon[35306]: from='osd.10 [v2:172.21.15.170:6824/2707848227,v1:172.21.15.170:6825/2707848227]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T08:42:33.434 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:33 smithi173 ceph-mon[35306]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T08:42:33.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:33 smithi164 ceph-mon[30928]: from='osd.10 [v2:172.21.15.170:6824/2707848227,v1:172.21.15.170:6825/2707848227]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T08:42:33.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:33 smithi164 ceph-mon[30928]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T08:42:34.081 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-19T08:42:34.108 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi173:vg_nvme/lv_1 2024-09-19T08:42:34.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:34 smithi170 ceph-mon[35072]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T08:42:34.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:34 smithi170 ceph-mon[35072]: from='osd.10 [v2:172.21.15.170:6824/2707848227,v1:172.21.15.170:6825/2707848227]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-19T08:42:34.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:34 smithi170 ceph-mon[35072]: osdmap e64: 11 total, 10 up, 11 in 2024-09-19T08:42:34.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:34 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:42:34.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:34 smithi170 ceph-mon[35072]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-19T08:42:34.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:34 smithi170 ceph-mon[35072]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:34.407 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:42:34.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:34 smithi173 ceph-mon[35306]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T08:42:34.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:34 smithi173 ceph-mon[35306]: from='osd.10 [v2:172.21.15.170:6824/2707848227,v1:172.21.15.170:6825/2707848227]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-19T08:42:34.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:34 smithi173 ceph-mon[35306]: osdmap e64: 11 total, 10 up, 11 in 2024-09-19T08:42:34.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:34 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:42:34.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:34 smithi173 ceph-mon[35306]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-19T08:42:34.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:34 smithi173 ceph-mon[35306]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:34.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:34 smithi164 ceph-mon[30928]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T08:42:34.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:34 smithi164 ceph-mon[30928]: from='osd.10 [v2:172.21.15.170:6824/2707848227,v1:172.21.15.170:6825/2707848227]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-19T08:42:34.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:34 smithi164 ceph-mon[30928]: osdmap e64: 11 total, 10 up, 11 in 2024-09-19T08:42:34.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:34 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:42:34.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:34 smithi164 ceph-mon[30928]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-09-19T08:42:34.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:34 smithi164 ceph-mon[30928]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:34.812 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:42:34 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:42:34.613+0000 7f174ea0d640 -1 osd.10 0 waiting for initial osdmap 2024-09-19T08:42:34.812 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:42:34 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:42:34.622+0000 7f174682d640 -1 osd.10 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:42:34.868 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.868+0000 7fc9a4bbe640 1 -- 172.21.15.173:0/2452234568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9a01037c0 msgr2=0x7fc9a010a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:34.868 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.868+0000 7fc9a4bbe640 1 --2- 172.21.15.173:0/2452234568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9a01037c0 0x7fc9a010a050 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc998009ca0 tx=0x7fc99802a8c0 comp rx=0 tx=0).stop 2024-09-19T08:42:34.868 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.868+0000 7fc9a4bbe640 1 -- 172.21.15.173:0/2452234568 shutdown_connections 2024-09-19T08:42:34.868 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.868+0000 7fc9a4bbe640 1 --2- 172.21.15.173:0/2452234568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9a01037c0 0x7fc9a010a050 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:34.868 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.868+0000 7fc9a4bbe640 1 --2- 172.21.15.173:0/2452234568 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9a0102e00 0x7fc9a0103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:34.868 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.868+0000 7fc9a4bbe640 1 --2- 172.21.15.173:0/2452234568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9a0101c00 0x7fc9a0102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:34.868 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.868+0000 7fc9a4bbe640 1 -- 172.21.15.173:0/2452234568 >> 172.21.15.173:0/2452234568 conn(0x7fc9a00fd390 msgr2=0x7fc9a00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:34.868 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.868+0000 7fc9a4bbe640 1 -- 172.21.15.173:0/2452234568 shutdown_connections 2024-09-19T08:42:34.869 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.868+0000 7fc9a4bbe640 1 -- 172.21.15.173:0/2452234568 wait complete. 2024-09-19T08:42:34.870 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.869+0000 7fc9a4bbe640 1 Processor -- start 2024-09-19T08:42:34.870 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.869+0000 7fc9a4bbe640 1 -- start start 2024-09-19T08:42:34.870 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.869+0000 7fc9a4bbe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9a0101c00 0x7fc9a0197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:34.871 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.869+0000 7fc9a4bbe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9a0102e00 0x7fc9a01983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:34.871 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.869+0000 7fc9a4bbe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9a01037c0 0x7fc9a019f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:34.871 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.869+0000 7fc9a4bbe640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a010d8d0 con 0x7fc9a0101c00 2024-09-19T08:42:34.871 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.869+0000 7fc9a4bbe640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a010d750 con 0x7fc9a01037c0 2024-09-19T08:42:34.871 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.869+0000 7fc9a4bbe640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a010da50 con 0x7fc9a0102e00 2024-09-19T08:42:34.871 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.869+0000 7fc99dd9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9a0102e00 0x7fc9a01983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:34.871 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.869+0000 7fc99dd9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9a0102e00 0x7fc9a01983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:57830/0 (socket says 172.21.15.173:57830) 2024-09-19T08:42:34.871 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.869+0000 7fc99dd9b640 1 -- 172.21.15.173:0/2878116586 learned_addr learned my addr 172.21.15.173:0/2878116586 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-09-19T08:42:34.871 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.869+0000 7fc99e59c640 1 --2- 172.21.15.173:0/2878116586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9a0101c00 0x7fc9a0197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:34.871 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.869+0000 7fc99ed9d640 1 --2- 172.21.15.173:0/2878116586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9a01037c0 0x7fc9a019f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:34.871 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.869+0000 7fc99dd9b640 1 -- 172.21.15.173:0/2878116586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9a01037c0 msgr2=0x7fc9a019f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:34.871 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.869+0000 7fc99dd9b640 1 --2- 172.21.15.173:0/2878116586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9a01037c0 0x7fc9a019f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:34.872 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.869+0000 7fc99dd9b640 1 -- 172.21.15.173:0/2878116586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9a0101c00 msgr2=0x7fc9a0197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:34.872 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.869+0000 7fc99dd9b640 1 --2- 172.21.15.173:0/2878116586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9a0101c00 0x7fc9a0197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:34.872 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.869+0000 7fc99dd9b640 1 -- 172.21.15.173:0/2878116586 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a019fac0 con 0x7fc9a0102e00 2024-09-19T08:42:34.872 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.869+0000 7fc99ed9d640 1 --2- 172.21.15.173:0/2878116586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9a01037c0 0x7fc9a019f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:34.872 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.869+0000 7fc99dd9b640 1 --2- 172.21.15.173:0/2878116586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9a0102e00 0x7fc9a01983d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc99400e9f0 tx=0x7fc99400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:34.872 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.870+0000 7fc98f7fe640 1 -- 172.21.15.173:0/2878116586 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99400cd80 con 0x7fc9a0102e00 2024-09-19T08:42:34.872 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.870+0000 7fc98f7fe640 1 -- 172.21.15.173:0/2878116586 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc994005500 con 0x7fc9a0102e00 2024-09-19T08:42:34.872 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.870+0000 7fc98f7fe640 1 -- 172.21.15.173:0/2878116586 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc994010690 con 0x7fc9a0102e00 2024-09-19T08:42:34.872 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.870+0000 7fc9a4bbe640 1 -- 172.21.15.173:0/2878116586 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9a019fdb0 con 0x7fc9a0102e00 2024-09-19T08:42:34.872 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.870+0000 7fc9a4bbe640 1 -- 172.21.15.173:0/2878116586 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9a01a0260 con 0x7fc9a0102e00 2024-09-19T08:42:34.872 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.871+0000 7fc9a4bbe640 1 -- 172.21.15.173:0/2878116586 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9a0108f30 con 0x7fc9a0102e00 2024-09-19T08:42:34.873 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.872+0000 7fc98f7fe640 1 -- 172.21.15.173:0/2878116586 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fc9940046c0 con 0x7fc9a0102e00 2024-09-19T08:42:34.873 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.873+0000 7fc98f7fe640 1 --2- 172.21.15.173:0/2878116586 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fc9880a32f0 0x7fc9880a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:34.873 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.873+0000 7fc98f7fe640 1 -- 172.21.15.173:0/2878116586 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6447+0+0 (secure 0 0 0) 0x7fc9940cf510 con 0x7fc9a0102e00 2024-09-19T08:42:34.874 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.873+0000 7fc99e59c640 1 --2- 172.21.15.173:0/2878116586 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fc9880a32f0 0x7fc9880a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:34.874 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.873+0000 7fc99e59c640 1 --2- 172.21.15.173:0/2878116586 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fc9880a32f0 0x7fc9880a57b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc990005fd0 tx=0x7fc99000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:34.877 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:34.877+0000 7fc98f7fe640 1 -- 172.21.15.173:0/2878116586 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc99409c980 con 0x7fc9a0102e00 2024-09-19T08:42:35.055 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:35.055+0000 7fc9a4bbe640 1 -- 172.21.15.173:0/2878116586 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fc9a0062630 con 0x7fc9880a32f0 2024-09-19T08:42:35.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:35 smithi173 ceph-mon[35306]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-09-19T08:42:35.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:35 smithi173 ceph-mon[35306]: osdmap e65: 11 total, 10 up, 11 in 2024-09-19T08:42:35.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:35 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:42:35.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:35 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:42:35.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:35 smithi173 ceph-mon[35306]: osdmap e66: 11 total, 10 up, 11 in 2024-09-19T08:42:35.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:35 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:42:35.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:35 smithi173 ceph-mon[35306]: from='osd.10 ' entity='osd.10' 2024-09-19T08:42:35.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:35 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:35.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:35 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:35.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:35 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:42:35.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:35 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:42:35.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:35 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:42:35.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:35 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:42:35.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:35 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:35.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:35 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:42:35.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:35 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:42:35.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:35 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:35.413 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:35 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:42:35.413 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:35 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:42:35.413 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:35 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:42:35.413 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:35 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:35.413 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:35 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:42:35.413 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:35 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:42:35.413 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:35 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:42:35.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:35 smithi164 ceph-mon[30928]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-09-19T08:42:35.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:35 smithi164 ceph-mon[30928]: osdmap e65: 11 total, 10 up, 11 in 2024-09-19T08:42:35.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:35 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:42:35.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:35 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:42:35.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:35 smithi164 ceph-mon[30928]: osdmap e66: 11 total, 10 up, 11 in 2024-09-19T08:42:35.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:35 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:42:35.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:35 smithi164 ceph-mon[30928]: from='osd.10 ' entity='osd.10' 2024-09-19T08:42:35.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:35 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:35.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:35 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:35.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:35 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:42:35.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:35 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:42:35.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:35 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:42:35.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:35 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:42:35.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:35 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:35.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:35 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:42:35.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:35 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:42:35.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:35 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:35.534 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:35 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:42:35.534 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:35 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:42:35.534 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:35 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:42:35.534 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:35 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:35.534 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:35 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:42:35.534 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:35 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:42:35.534 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:35 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:42:35.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:35 smithi170 ceph-mon[35072]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-09-19T08:42:35.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:35 smithi170 ceph-mon[35072]: osdmap e65: 11 total, 10 up, 11 in 2024-09-19T08:42:35.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:35 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:42:35.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:35 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:42:35.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:35 smithi170 ceph-mon[35072]: osdmap e66: 11 total, 10 up, 11 in 2024-09-19T08:42:35.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:35 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:42:35.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:35 smithi170 ceph-mon[35072]: from='osd.10 ' entity='osd.10' 2024-09-19T08:42:35.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:35 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:35.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:35 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:35.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:35 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:42:35.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:35 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:42:35.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:35 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:42:35.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:35 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:42:35.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:35 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:35.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:35 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:42:35.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:35 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:42:35.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:35 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:35.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:35 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:42:35.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:35 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:42:35.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:35 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:42:35.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:35 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:35.627 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:35 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T08:42:35.627 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:35 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T08:42:35.627 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:35 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:42:36.506 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:36 smithi173 ceph-mon[35306]: purged_snaps scrub starts 2024-09-19T08:42:36.506 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:36 smithi173 ceph-mon[35306]: purged_snaps scrub ok 2024-09-19T08:42:36.506 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:36 smithi173 ceph-mon[35306]: Detected new or changed devices on smithi170 2024-09-19T08:42:36.506 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:36 smithi173 ceph-mon[35306]: Adjusting osd_memory_target on smithi170 to 4272M 2024-09-19T08:42:36.507 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:36 smithi173 ceph-mon[35306]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:42:36.507 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:36 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:42:36.507 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:36 smithi173 ceph-mon[35306]: osd.10 [v2:172.21.15.170:6824/2707848227,v1:172.21.15.170:6825/2707848227] boot 2024-09-19T08:42:36.507 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:36 smithi173 ceph-mon[35306]: osdmap e67: 11 total, 11 up, 11 in 2024-09-19T08:42:36.507 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:36 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:42:36.507 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:36 smithi173 ceph-mon[35306]: pgmap v224: 1 pgs: 1 remapped+peering; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:36.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:36 smithi164 ceph-mon[30928]: purged_snaps scrub starts 2024-09-19T08:42:36.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:36 smithi164 ceph-mon[30928]: purged_snaps scrub ok 2024-09-19T08:42:36.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:36 smithi164 ceph-mon[30928]: Detected new or changed devices on smithi170 2024-09-19T08:42:36.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:36 smithi164 ceph-mon[30928]: Adjusting osd_memory_target on smithi170 to 4272M 2024-09-19T08:42:36.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:36 smithi164 ceph-mon[30928]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:42:36.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:36 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:42:36.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:36 smithi164 ceph-mon[30928]: osd.10 [v2:172.21.15.170:6824/2707848227,v1:172.21.15.170:6825/2707848227] boot 2024-09-19T08:42:36.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:36 smithi164 ceph-mon[30928]: osdmap e67: 11 total, 11 up, 11 in 2024-09-19T08:42:36.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:36 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:42:36.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:36 smithi164 ceph-mon[30928]: pgmap v224: 1 pgs: 1 remapped+peering; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:36.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:36 smithi170 ceph-mon[35072]: purged_snaps scrub starts 2024-09-19T08:42:36.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:36 smithi170 ceph-mon[35072]: purged_snaps scrub ok 2024-09-19T08:42:36.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:36 smithi170 ceph-mon[35072]: Detected new or changed devices on smithi170 2024-09-19T08:42:36.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:36 smithi170 ceph-mon[35072]: Adjusting osd_memory_target on smithi170 to 4272M 2024-09-19T08:42:36.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:36 smithi170 ceph-mon[35072]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:42:36.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:36 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:42:36.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:36 smithi170 ceph-mon[35072]: osd.10 [v2:172.21.15.170:6824/2707848227,v1:172.21.15.170:6825/2707848227] boot 2024-09-19T08:42:36.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:36 smithi170 ceph-mon[35072]: osdmap e67: 11 total, 11 up, 11 in 2024-09-19T08:42:36.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:36 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:42:36.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:36 smithi170 ceph-mon[35072]: pgmap v224: 1 pgs: 1 remapped+peering; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-19T08:42:37.818 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:37 smithi173 ceph-mon[35306]: osdmap e68: 11 total, 11 up, 11 in 2024-09-19T08:42:37.818 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:37 smithi173 ceph-mon[35306]: from='client.? 172.21.15.173:0/3657628574' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d93088f-e791-4938-a207-12f15b56abd8"}]: dispatch 2024-09-19T08:42:37.818 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:37 smithi173 ceph-mon[35306]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d93088f-e791-4938-a207-12f15b56abd8"}]: dispatch 2024-09-19T08:42:37.818 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:37 smithi173 ceph-mon[35306]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d93088f-e791-4938-a207-12f15b56abd8"}]': finished 2024-09-19T08:42:37.818 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:37 smithi173 ceph-mon[35306]: osdmap e69: 12 total, 11 up, 12 in 2024-09-19T08:42:37.818 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:37 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:42:37.874 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:37 smithi170 ceph-mon[35072]: osdmap e68: 11 total, 11 up, 11 in 2024-09-19T08:42:37.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:37 smithi170 ceph-mon[35072]: from='client.? 172.21.15.173:0/3657628574' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d93088f-e791-4938-a207-12f15b56abd8"}]: dispatch 2024-09-19T08:42:37.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:37 smithi170 ceph-mon[35072]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d93088f-e791-4938-a207-12f15b56abd8"}]: dispatch 2024-09-19T08:42:37.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:37 smithi170 ceph-mon[35072]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d93088f-e791-4938-a207-12f15b56abd8"}]': finished 2024-09-19T08:42:37.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:37 smithi170 ceph-mon[35072]: osdmap e69: 12 total, 11 up, 12 in 2024-09-19T08:42:37.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:37 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:42:38.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:37 smithi164 ceph-mon[30928]: osdmap e68: 11 total, 11 up, 11 in 2024-09-19T08:42:38.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:37 smithi164 ceph-mon[30928]: from='client.? 172.21.15.173:0/3657628574' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d93088f-e791-4938-a207-12f15b56abd8"}]: dispatch 2024-09-19T08:42:38.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:37 smithi164 ceph-mon[30928]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d93088f-e791-4938-a207-12f15b56abd8"}]: dispatch 2024-09-19T08:42:38.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:37 smithi164 ceph-mon[30928]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d93088f-e791-4938-a207-12f15b56abd8"}]': finished 2024-09-19T08:42:38.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:37 smithi164 ceph-mon[30928]: osdmap e69: 12 total, 11 up, 12 in 2024-09-19T08:42:38.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:37 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:42:38.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:38 smithi170 ceph-mon[35072]: pgmap v227: 1 pgs: 1 remapped+peering; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:42:38.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:38 smithi170 ceph-mon[35072]: from='client.? 172.21.15.173:0/859398438' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:42:38.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:38 smithi173 ceph-mon[35306]: pgmap v227: 1 pgs: 1 remapped+peering; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:42:38.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:38 smithi173 ceph-mon[35306]: from='client.? 172.21.15.173:0/859398438' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:42:39.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:38 smithi164 ceph-mon[30928]: pgmap v227: 1 pgs: 1 remapped+peering; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:42:39.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:38 smithi164 ceph-mon[30928]: from='client.? 172.21.15.173:0/859398438' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T08:42:39.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:39 smithi170 ceph-mon[35072]: osdmap e70: 12 total, 11 up, 12 in 2024-09-19T08:42:39.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:39 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:42:39.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:39 smithi173 ceph-mon[35306]: osdmap e70: 12 total, 11 up, 12 in 2024-09-19T08:42:39.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:39 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:42:40.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:39 smithi164 ceph-mon[30928]: osdmap e70: 12 total, 11 up, 12 in 2024-09-19T08:42:40.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:39 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:42:40.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:40 smithi170 ceph-mon[35072]: osdmap e71: 12 total, 11 up, 12 in 2024-09-19T08:42:40.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:40 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:42:40.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:40 smithi170 ceph-mon[35072]: pgmap v230: 1 pgs: 1 remapped+peering; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:42:40.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:40 smithi173 ceph-mon[35306]: osdmap e71: 12 total, 11 up, 12 in 2024-09-19T08:42:40.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:40 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:42:40.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:40 smithi173 ceph-mon[35306]: pgmap v230: 1 pgs: 1 remapped+peering; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:42:41.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:40 smithi164 ceph-mon[30928]: osdmap e71: 12 total, 11 up, 12 in 2024-09-19T08:42:41.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:40 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:42:41.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:40 smithi164 ceph-mon[30928]: pgmap v230: 1 pgs: 1 remapped+peering; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:42:42.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:42 smithi173 ceph-mon[35306]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-09-19T08:42:43.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:42 smithi164 ceph-mon[30928]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-09-19T08:42:43.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:42 smithi170 ceph-mon[35072]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-09-19T08:42:44.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:44 smithi173 ceph-mon[35306]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-19T08:42:45.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:44 smithi170 ceph-mon[35072]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-19T08:42:45.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:44 smithi164 ceph-mon[30928]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-19T08:42:47.010 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:46 smithi173 ceph-mon[35306]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T08:42:47.010 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:46 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T08:42:47.010 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:46 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:42:47.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:46 smithi170 ceph-mon[35072]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T08:42:47.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:46 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T08:42:47.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:46 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:42:47.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:46 smithi164 ceph-mon[30928]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T08:42:47.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:46 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T08:42:47.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:46 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:42:48.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:47 smithi170 ceph-mon[35072]: Deploying daemon osd.11 on smithi173 2024-09-19T08:42:48.140 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:47 smithi173 ceph-mon[35306]: Deploying daemon osd.11 on smithi173 2024-09-19T08:42:48.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:47 smithi164 ceph-mon[30928]: Deploying daemon osd.11 on smithi173 2024-09-19T08:42:49.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:48 smithi170 ceph-mon[35072]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-19T08:42:49.139 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:48 smithi173 ceph-mon[35306]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-19T08:42:49.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:48 smithi164 ceph-mon[30928]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-19T08:42:51.057 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:50 smithi173 ceph-mon[35306]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-19T08:42:51.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:50 smithi170 ceph-mon[35072]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-19T08:42:51.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:50 smithi164 ceph-mon[30928]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-19T08:42:53.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:52 smithi173 ceph-mon[35306]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T08:42:53.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:52 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:42:53.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:52 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:53.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:52 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:53.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:52 smithi170 ceph-mon[35072]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T08:42:53.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:52 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:42:53.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:52 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:53.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:52 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:53.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:52 smithi164 ceph-mon[30928]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T08:42:53.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:52 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:42:53.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:52 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:53.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:52 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:55.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:54 smithi170 ceph-mon[35072]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:42:55.128 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:54 smithi173 ceph-mon[35306]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:42:55.281 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:54 smithi164 ceph-mon[30928]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:42:55.853 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:55.853+0000 7fc98f7fe640 1 -- 172.21.15.173:0/2878116586 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fc9a0062630 con 0x7fc9880a32f0 2024-09-19T08:42:55.853 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 11 on host 'smithi173' 2024-09-19T08:42:55.856 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:55.855+0000 7fc9a4bbe640 1 -- 172.21.15.173:0/2878116586 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fc9880a32f0 msgr2=0x7fc9880a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:55.856 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:55.855+0000 7fc9a4bbe640 1 --2- 172.21.15.173:0/2878116586 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fc9880a32f0 0x7fc9880a57b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc990005fd0 tx=0x7fc99000b410 comp rx=0 tx=0).stop 2024-09-19T08:42:55.856 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:55.855+0000 7fc9a4bbe640 1 -- 172.21.15.173:0/2878116586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9a0102e00 msgr2=0x7fc9a01983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:55.856 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:55.855+0000 7fc9a4bbe640 1 --2- 172.21.15.173:0/2878116586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9a0102e00 0x7fc9a01983d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc99400e9f0 tx=0x7fc99400edf0 comp rx=0 tx=0).stop 2024-09-19T08:42:55.856 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:55.856+0000 7fc9a4bbe640 1 -- 172.21.15.173:0/2878116586 shutdown_connections 2024-09-19T08:42:55.856 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:55.856+0000 7fc9a4bbe640 1 --2- 172.21.15.173:0/2878116586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9a01037c0 0x7fc9a019f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:55.856 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:55.856+0000 7fc9a4bbe640 1 --2- 172.21.15.173:0/2878116586 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fc9880a32f0 0x7fc9880a57b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:55.856 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:55.856+0000 7fc9a4bbe640 1 --2- 172.21.15.173:0/2878116586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9a0102e00 0x7fc9a01983d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:55.856 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:55.856+0000 7fc9a4bbe640 1 --2- 172.21.15.173:0/2878116586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9a0101c00 0x7fc9a0197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:55.856 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:55.856+0000 7fc9a4bbe640 1 -- 172.21.15.173:0/2878116586 >> 172.21.15.173:0/2878116586 conn(0x7fc9a00fd390 msgr2=0x7fc9a0106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:55.857 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:55.856+0000 7fc9a4bbe640 1 -- 172.21.15.173:0/2878116586 shutdown_connections 2024-09-19T08:42:55.857 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:42:55.856+0000 7fc9a4bbe640 1 -- 172.21.15.173:0/2878116586 wait complete. 2024-09-19T08:42:56.505 DEBUG:teuthology.orchestra.run.smithi173:osd.11> sudo journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.11.service 2024-09-19T08:42:56.507 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-19T08:42:56.508 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T08:42:56.776 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:42:57.060 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:56 smithi164 ceph-mon[30928]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:42:57.060 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:56 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:57.060 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:56 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:57.060 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:56 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:57.060 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:56 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:57.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:56 smithi173 ceph-mon[35306]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:42:57.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:56 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:57.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:56 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:57.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:56 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:57.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:56 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:57.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:56 smithi170 ceph-mon[35072]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:42:57.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:56 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:57.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:56 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:57.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:56 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:57.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:56 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:42:57.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.253+0000 7f74f505e640 1 -- 172.21.15.164:0/1055892024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74f0101c10 msgr2=0x7f74f0102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:57.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.253+0000 7f74f505e640 1 --2- 172.21.15.164:0/1055892024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74f0101c10 0x7f74f0102010 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f74e0009ca0 tx=0x7f74e002a8c0 comp rx=0 tx=0).stop 2024-09-19T08:42:57.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.253+0000 7f74f505e640 1 -- 172.21.15.164:0/1055892024 shutdown_connections 2024-09-19T08:42:57.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.253+0000 7f74f505e640 1 --2- 172.21.15.164:0/1055892024 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74f01037d0 0x7f74f010a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:57.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.253+0000 7f74f505e640 1 --2- 172.21.15.164:0/1055892024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74f0102e10 0x7f74f0103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:57.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.253+0000 7f74f505e640 1 --2- 172.21.15.164:0/1055892024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74f0101c10 0x7f74f0102010 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:57.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.253+0000 7f74f505e640 1 -- 172.21.15.164:0/1055892024 >> 172.21.15.164:0/1055892024 conn(0x7f74f00fd3c0 msgr2=0x7f74f00ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:57.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.253+0000 7f74f505e640 1 -- 172.21.15.164:0/1055892024 shutdown_connections 2024-09-19T08:42:57.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.253+0000 7f74f505e640 1 -- 172.21.15.164:0/1055892024 wait complete. 2024-09-19T08:42:57.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.253+0000 7f74f505e640 1 Processor -- start 2024-09-19T08:42:57.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.254+0000 7f74f505e640 1 -- start start 2024-09-19T08:42:57.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.254+0000 7f74f505e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74f0101c10 0x7f74f0075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:57.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.254+0000 7f74f505e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74f0102e10 0x7f74f00745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:57.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.254+0000 7f74f505e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74f01037d0 0x7f74f0074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:57.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.254+0000 7f74f505e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f74f010d910 con 0x7f74f01037d0 2024-09-19T08:42:57.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.254+0000 7f74f505e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f74f010d790 con 0x7f74f0101c10 2024-09-19T08:42:57.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.254+0000 7f74f505e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f74f010da90 con 0x7f74f0102e10 2024-09-19T08:42:57.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.254+0000 7f74ef59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74f01037d0 0x7f74f0074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:57.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.254+0000 7f74ef59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74f01037d0 0x7f74f0074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46186/0 (socket says 172.21.15.164:46186) 2024-09-19T08:42:57.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.254+0000 7f74ef59e640 1 -- 172.21.15.164:0/3095882309 learned_addr learned my addr 172.21.15.164:0/3095882309 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:42:57.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.254+0000 7f74ee59c640 1 --2- 172.21.15.164:0/3095882309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74f0102e10 0x7f74f00745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:57.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.254+0000 7f74eed9d640 1 --2- 172.21.15.164:0/3095882309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74f0101c10 0x7f74f0075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:57.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.254+0000 7f74ef59e640 1 -- 172.21.15.164:0/3095882309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74f0102e10 msgr2=0x7f74f00745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:57.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.254+0000 7f74ef59e640 1 --2- 172.21.15.164:0/3095882309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74f0102e10 0x7f74f00745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:57.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.254+0000 7f74ef59e640 1 -- 172.21.15.164:0/3095882309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74f0101c10 msgr2=0x7f74f0075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:57.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.254+0000 7f74ef59e640 1 --2- 172.21.15.164:0/3095882309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74f0101c10 0x7f74f0075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:57.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.254+0000 7f74ef59e640 1 -- 172.21.15.164:0/3095882309 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f00753e0 con 0x7f74f01037d0 2024-09-19T08:42:57.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.254+0000 7f74eed9d640 1 --2- 172.21.15.164:0/3095882309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74f0101c10 0x7f74f0075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:42:57.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.254+0000 7f74ef59e640 1 --2- 172.21.15.164:0/3095882309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74f01037d0 0x7f74f0074b20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f74e800d750 tx=0x7f74e800db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:57.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.255+0000 7f74d7fff640 1 -- 172.21.15.164:0/3095882309 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e8017070 con 0x7f74f01037d0 2024-09-19T08:42:57.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.255+0000 7f74d7fff640 1 -- 172.21.15.164:0/3095882309 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74e80097c0 con 0x7f74f01037d0 2024-09-19T08:42:57.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.255+0000 7f74f505e640 1 -- 172.21.15.164:0/3095882309 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74f01a2120 con 0x7f74f01037d0 2024-09-19T08:42:57.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.255+0000 7f74d7fff640 1 -- 172.21.15.164:0/3095882309 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e8004d00 con 0x7f74f01037d0 2024-09-19T08:42:57.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.255+0000 7f74f505e640 1 -- 172.21.15.164:0/3095882309 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74f01a26b0 con 0x7f74f01037d0 2024-09-19T08:42:57.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.256+0000 7f74f505e640 1 -- 172.21.15.164:0/3095882309 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74f0109030 con 0x7f74f01037d0 2024-09-19T08:42:57.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.257+0000 7f74d7fff640 1 -- 172.21.15.164:0/3095882309 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f74e8009970 con 0x7f74f01037d0 2024-09-19T08:42:57.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.258+0000 7f74d7fff640 1 --2- 172.21.15.164:0/3095882309 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f74d80a3240 0x7f74d80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:57.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.258+0000 7f74eed9d640 1 --2- 172.21.15.164:0/3095882309 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f74d80a3240 0x7f74d80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:57.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.258+0000 7f74d7fff640 1 -- 172.21.15.164:0/3095882309 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6809+0+0 (secure 0 0 0) 0x7f74e80cecd0 con 0x7f74f01037d0 2024-09-19T08:42:57.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.258+0000 7f74eed9d640 1 --2- 172.21.15.164:0/3095882309 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f74d80a3240 0x7f74d80a5700 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f74e0006010 tx=0x7f74e0031bb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:57.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.263+0000 7f74d7fff640 1 -- 172.21.15.164:0/3095882309 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f74e809bfd0 con 0x7f74f01037d0 2024-09-19T08:42:57.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.436+0000 7f74f505e640 1 -- 172.21.15.164:0/3095882309 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f74f0062df0 con 0x7f74f01037d0 2024-09-19T08:42:57.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.437+0000 7f74d7fff640 1 -- 172.21.15.164:0/3095882309 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7f74e809b8d0 con 0x7f74f01037d0 2024-09-19T08:42:57.438 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:42:57.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.439+0000 7f74f505e640 1 -- 172.21.15.164:0/3095882309 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f74d80a3240 msgr2=0x7f74d80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:57.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.439+0000 7f74f505e640 1 --2- 172.21.15.164:0/3095882309 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f74d80a3240 0x7f74d80a5700 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f74e0006010 tx=0x7f74e0031bb0 comp rx=0 tx=0).stop 2024-09-19T08:42:57.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.439+0000 7f74f505e640 1 -- 172.21.15.164:0/3095882309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74f01037d0 msgr2=0x7f74f0074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:57.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.439+0000 7f74f505e640 1 --2- 172.21.15.164:0/3095882309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74f01037d0 0x7f74f0074b20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f74e800d750 tx=0x7f74e800db50 comp rx=0 tx=0).stop 2024-09-19T08:42:57.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.439+0000 7f74f505e640 1 -- 172.21.15.164:0/3095882309 shutdown_connections 2024-09-19T08:42:57.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.439+0000 7f74f505e640 1 --2- 172.21.15.164:0/3095882309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74f01037d0 0x7f74f0074b20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:57.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.439+0000 7f74f505e640 1 --2- 172.21.15.164:0/3095882309 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f74d80a3240 0x7f74d80a5700 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:57.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.439+0000 7f74f505e640 1 --2- 172.21.15.164:0/3095882309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74f0102e10 0x7f74f00745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:57.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.439+0000 7f74f505e640 1 --2- 172.21.15.164:0/3095882309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74f0101c10 0x7f74f0075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:57.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.439+0000 7f74f505e640 1 -- 172.21.15.164:0/3095882309 >> 172.21.15.164:0/3095882309 conn(0x7f74f00fd3c0 msgr2=0x7f74f0106200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:57.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.439+0000 7f74f505e640 1 -- 172.21.15.164:0/3095882309 shutdown_connections 2024-09-19T08:42:57.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:42:57.439+0000 7f74f505e640 1 -- 172.21.15.164:0/3095882309 wait complete. 2024-09-19T08:42:58.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:57 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3095882309' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T08:42:58.081 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:57 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3095882309' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T08:42:58.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:57 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3095882309' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T08:42:58.362 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:42:58 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:42:58.273+0000 7fcceab29640 -1 osd.11 0 log_to_monitors true 2024-09-19T08:42:58.397 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":71,"num_osds":12,"num_up_osds":11,"osd_up_since":1726735355,"num_in_osds":12,"osd_in_since":1726735357,"num_remapped_pgs":0} 2024-09-19T08:42:59.084 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:58 smithi173 ceph-mon[35306]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:42:59.084 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:58 smithi173 ceph-mon[35306]: from='osd.11 [v2:172.21.15.173:6824/2831924738,v1:172.21.15.173:6825/2831924738]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T08:42:59.084 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:58 smithi173 ceph-mon[35306]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T08:42:59.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:58 smithi170 ceph-mon[35072]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:42:59.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:58 smithi170 ceph-mon[35072]: from='osd.11 [v2:172.21.15.173:6824/2831924738,v1:172.21.15.173:6825/2831924738]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T08:42:59.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:58 smithi170 ceph-mon[35072]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T08:42:59.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:58 smithi164 ceph-mon[30928]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:42:59.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:58 smithi164 ceph-mon[30928]: from='osd.11 [v2:172.21.15.173:6824/2831924738,v1:172.21.15.173:6825/2831924738]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T08:42:59.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:58 smithi164 ceph-mon[30928]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T08:42:59.398 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T08:42:59.645 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:43:00.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:59 smithi164 ceph-mon[30928]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T08:43:00.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:59 smithi164 ceph-mon[30928]: from='osd.11 [v2:172.21.15.173:6824/2831924738,v1:172.21.15.173:6825/2831924738]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-19T08:43:00.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:59 smithi164 ceph-mon[30928]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T08:43:00.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:59 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:43:00.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:59 smithi164 ceph-mon[30928]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-19T08:43:00.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:59 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:00.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:59 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:00.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:59 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:43:00.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:59 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:43:00.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:59 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:43:00.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:59 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:43:00.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:59 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:00.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:59 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:00.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:59 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:43:00.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:42:59 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:00.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:59 smithi170 ceph-mon[35072]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T08:43:00.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:59 smithi170 ceph-mon[35072]: from='osd.11 [v2:172.21.15.173:6824/2831924738,v1:172.21.15.173:6825/2831924738]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-19T08:43:00.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:59 smithi170 ceph-mon[35072]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T08:43:00.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:59 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:43:00.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:59 smithi170 ceph-mon[35072]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-19T08:43:00.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:59 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:00.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:59 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:00.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:59 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:43:00.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:59 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:43:00.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:59 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:43:00.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:59 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:43:00.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:59 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:00.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:59 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:00.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:59 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:43:00.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:42:59 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:00.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.120+0000 7fc349b4d640 1 -- 172.21.15.164:0/618100600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc3440ffed0 msgr2=0x7fc3441002d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:00.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.120+0000 7fc349b4d640 1 --2- 172.21.15.164:0/618100600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc3440ffed0 0x7fc3441002d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc334009930 tx=0x7fc33402a420 comp rx=0 tx=0).stop 2024-09-19T08:43:00.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.120+0000 7fc349b4d640 1 -- 172.21.15.164:0/618100600 shutdown_connections 2024-09-19T08:43:00.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.120+0000 7fc349b4d640 1 --2- 172.21.15.164:0/618100600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc344103870 0x7fc34410a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:00.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.120+0000 7fc349b4d640 1 --2- 172.21.15.164:0/618100600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc344102eb0 0x7fc344103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:00.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.120+0000 7fc349b4d640 1 --2- 172.21.15.164:0/618100600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc3440ffed0 0x7fc3441002d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:00.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.120+0000 7fc349b4d640 1 -- 172.21.15.164:0/618100600 >> 172.21.15.164:0/618100600 conn(0x7fc3440fba40 msgr2=0x7fc3440fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:00.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.120+0000 7fc349b4d640 1 -- 172.21.15.164:0/618100600 shutdown_connections 2024-09-19T08:43:00.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.120+0000 7fc349b4d640 1 -- 172.21.15.164:0/618100600 wait complete. 2024-09-19T08:43:00.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.120+0000 7fc349b4d640 1 Processor -- start 2024-09-19T08:43:00.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.121+0000 7fc349b4d640 1 -- start start 2024-09-19T08:43:00.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.121+0000 7fc349b4d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3440ffed0 0x7fc344198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:00.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.121+0000 7fc349b4d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc344102eb0 0x7fc344198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:00.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.121+0000 7fc349b4d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc344103870 0x7fc34419f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:00.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.121+0000 7fc349b4d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc34410d8b0 con 0x7fc344102eb0 2024-09-19T08:43:00.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.121+0000 7fc349b4d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc34410d730 con 0x7fc3440ffed0 2024-09-19T08:43:00.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.121+0000 7fc349b4d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc34410da30 con 0x7fc344103870 2024-09-19T08:43:00.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.121+0000 7fc342ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc344102eb0 0x7fc344198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:00.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.121+0000 7fc342ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc344102eb0 0x7fc344198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46194/0 (socket says 172.21.15.164:46194) 2024-09-19T08:43:00.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.121+0000 7fc342ffd640 1 -- 172.21.15.164:0/930867267 learned_addr learned my addr 172.21.15.164:0/930867267 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:43:00.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.121+0000 7fc343fff640 1 --2- 172.21.15.164:0/930867267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc344103870 0x7fc34419f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:00.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.121+0000 7fc3437fe640 1 --2- 172.21.15.164:0/930867267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3440ffed0 0x7fc344198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:00.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.121+0000 7fc342ffd640 1 -- 172.21.15.164:0/930867267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc344103870 msgr2=0x7fc34419f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:00.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.121+0000 7fc342ffd640 1 --2- 172.21.15.164:0/930867267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc344103870 0x7fc34419f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:00.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.121+0000 7fc342ffd640 1 -- 172.21.15.164:0/930867267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3440ffed0 msgr2=0x7fc344198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:00.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.121+0000 7fc342ffd640 1 --2- 172.21.15.164:0/930867267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3440ffed0 0x7fc344198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:00.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.121+0000 7fc342ffd640 1 -- 172.21.15.164:0/930867267 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc34419fc50 con 0x7fc344102eb0 2024-09-19T08:43:00.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.121+0000 7fc342ffd640 1 --2- 172.21.15.164:0/930867267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc344102eb0 0x7fc344198560 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc33800d770 tx=0x7fc33800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:00.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.122+0000 7fc340ff9640 1 -- 172.21.15.164:0/930867267 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc338017070 con 0x7fc344102eb0 2024-09-19T08:43:00.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.122+0000 7fc340ff9640 1 -- 172.21.15.164:0/930867267 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc338005500 con 0x7fc344102eb0 2024-09-19T08:43:00.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.122+0000 7fc349b4d640 1 -- 172.21.15.164:0/930867267 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc34419ff40 con 0x7fc344102eb0 2024-09-19T08:43:00.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.122+0000 7fc340ff9640 1 -- 172.21.15.164:0/930867267 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc338004e00 con 0x7fc344102eb0 2024-09-19T08:43:00.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.122+0000 7fc349b4d640 1 -- 172.21.15.164:0/930867267 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3441a03a0 con 0x7fc344102eb0 2024-09-19T08:43:00.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.123+0000 7fc349b4d640 1 -- 172.21.15.164:0/930867267 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc344109140 con 0x7fc344102eb0 2024-09-19T08:43:00.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.124+0000 7fc340ff9640 1 -- 172.21.15.164:0/930867267 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fc338004fa0 con 0x7fc344102eb0 2024-09-19T08:43:00.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.124+0000 7fc340ff9640 1 --2- 172.21.15.164:0/930867267 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fc32c0a3270 0x7fc32c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:00.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.125+0000 7fc3437fe640 1 --2- 172.21.15.164:0/930867267 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fc32c0a3270 0x7fc32c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:00.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.125+0000 7fc340ff9640 1 -- 172.21.15.164:0/930867267 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6847+0+0 (secure 0 0 0) 0x7fc3380cec70 con 0x7fc344102eb0 2024-09-19T08:43:00.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.125+0000 7fc3437fe640 1 --2- 172.21.15.164:0/930867267 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fc32c0a3270 0x7fc32c0a5730 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc334009900 tx=0x7fc3340315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:00.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.129+0000 7fc340ff9640 1 -- 172.21.15.164:0/930867267 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc33809b900 con 0x7fc344102eb0 2024-09-19T08:43:00.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:59 smithi173 ceph-mon[35306]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T08:43:00.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:59 smithi173 ceph-mon[35306]: from='osd.11 [v2:172.21.15.173:6824/2831924738,v1:172.21.15.173:6825/2831924738]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-19T08:43:00.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:59 smithi173 ceph-mon[35306]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T08:43:00.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:59 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:43:00.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:59 smithi173 ceph-mon[35306]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-09-19T08:43:00.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:59 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:00.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:59 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:00.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:59 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:43:00.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:59 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:43:00.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:59 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:43:00.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:59 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:43:00.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:59 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:00.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:59 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:00.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:59 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:43:00.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:42:59 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:00.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.295+0000 7fc349b4d640 1 -- 172.21.15.164:0/930867267 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fc344062e90 con 0x7fc344102eb0 2024-09-19T08:43:00.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.296+0000 7fc340ff9640 1 -- 172.21.15.164:0/930867267 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7fc33809fc00 con 0x7fc344102eb0 2024-09-19T08:43:00.296 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:43:00.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.298+0000 7fc349b4d640 1 -- 172.21.15.164:0/930867267 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fc32c0a3270 msgr2=0x7fc32c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:00.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.298+0000 7fc349b4d640 1 --2- 172.21.15.164:0/930867267 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fc32c0a3270 0x7fc32c0a5730 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc334009900 tx=0x7fc3340315b0 comp rx=0 tx=0).stop 2024-09-19T08:43:00.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.298+0000 7fc349b4d640 1 -- 172.21.15.164:0/930867267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc344102eb0 msgr2=0x7fc344198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:00.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.298+0000 7fc349b4d640 1 --2- 172.21.15.164:0/930867267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc344102eb0 0x7fc344198560 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc33800d770 tx=0x7fc33800dc30 comp rx=0 tx=0).stop 2024-09-19T08:43:00.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.298+0000 7fc349b4d640 1 -- 172.21.15.164:0/930867267 shutdown_connections 2024-09-19T08:43:00.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.298+0000 7fc349b4d640 1 --2- 172.21.15.164:0/930867267 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fc32c0a3270 0x7fc32c0a5730 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:00.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.298+0000 7fc349b4d640 1 --2- 172.21.15.164:0/930867267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc344103870 0x7fc34419f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:00.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.298+0000 7fc349b4d640 1 --2- 172.21.15.164:0/930867267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc344102eb0 0x7fc344198560 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:00.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.298+0000 7fc349b4d640 1 --2- 172.21.15.164:0/930867267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3440ffed0 0x7fc344198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:00.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.298+0000 7fc349b4d640 1 -- 172.21.15.164:0/930867267 >> 172.21.15.164:0/930867267 conn(0x7fc3440fba40 msgr2=0x7fc344106310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:00.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.298+0000 7fc349b4d640 1 -- 172.21.15.164:0/930867267 shutdown_connections 2024-09-19T08:43:00.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:00.298+0000 7fc349b4d640 1 -- 172.21.15.164:0/930867267 wait complete. 2024-09-19T08:43:00.660 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:43:00 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:43:00.258+0000 7fcce6aac640 -1 osd.11 0 waiting for initial osdmap 2024-09-19T08:43:00.661 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:43:00 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:43:00.271+0000 7fccde8cc640 -1 osd.11 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:43:01.078 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:00 smithi164 ceph-mon[30928]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:43:01.078 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:00 smithi164 ceph-mon[30928]: Detected new or changed devices on smithi173 2024-09-19T08:43:01.078 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:00 smithi164 ceph-mon[30928]: Adjusting osd_memory_target on smithi173 to 4272M 2024-09-19T08:43:01.078 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:00 smithi164 ceph-mon[30928]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-19T08:43:01.078 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:00 smithi164 ceph-mon[30928]: osdmap e73: 12 total, 11 up, 12 in 2024-09-19T08:43:01.078 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:00 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:43:01.078 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:00 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:43:01.079 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:00 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:43:01.079 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:00 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:01.079 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:00 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:43:01.079 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:00 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:01.079 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:00 smithi164 ceph-mon[30928]: from='osd.11 ' entity='osd.11' 2024-09-19T08:43:01.079 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:00 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/930867267' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T08:43:01.105 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":73,"num_osds":12,"num_up_osds":11,"osd_up_since":1726735355,"num_in_osds":12,"osd_in_since":1726735357,"num_remapped_pgs":0} 2024-09-19T08:43:01.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:00 smithi170 ceph-mon[35072]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:43:01.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:00 smithi170 ceph-mon[35072]: Detected new or changed devices on smithi173 2024-09-19T08:43:01.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:00 smithi170 ceph-mon[35072]: Adjusting osd_memory_target on smithi173 to 4272M 2024-09-19T08:43:01.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:00 smithi170 ceph-mon[35072]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-19T08:43:01.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:00 smithi170 ceph-mon[35072]: osdmap e73: 12 total, 11 up, 12 in 2024-09-19T08:43:01.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:00 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:43:01.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:00 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:43:01.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:00 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:43:01.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:00 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:01.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:00 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:43:01.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:00 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:01.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:00 smithi170 ceph-mon[35072]: from='osd.11 ' entity='osd.11' 2024-09-19T08:43:01.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:00 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/930867267' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T08:43:01.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:00 smithi173 ceph-mon[35306]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:43:01.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:00 smithi173 ceph-mon[35306]: Detected new or changed devices on smithi173 2024-09-19T08:43:01.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:00 smithi173 ceph-mon[35306]: Adjusting osd_memory_target on smithi173 to 4272M 2024-09-19T08:43:01.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:00 smithi173 ceph-mon[35306]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-19T08:43:01.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:00 smithi173 ceph-mon[35306]: osdmap e73: 12 total, 11 up, 12 in 2024-09-19T08:43:01.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:00 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:43:01.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:00 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:43:01.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:00 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:43:01.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:00 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:01.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:00 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:43:01.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:00 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:01.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:00 smithi173 ceph-mon[35306]: from='osd.11 ' entity='osd.11' 2024-09-19T08:43:01.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:00 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/930867267' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T08:43:02.106 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T08:43:02.118 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:01 smithi164 ceph-mon[30928]: purged_snaps scrub starts 2024-09-19T08:43:02.119 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:01 smithi164 ceph-mon[30928]: purged_snaps scrub ok 2024-09-19T08:43:02.119 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:01 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:43:02.119 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:01 smithi164 ceph-mon[30928]: osd.11 [v2:172.21.15.173:6824/2831924738,v1:172.21.15.173:6825/2831924738] boot 2024-09-19T08:43:02.119 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:01 smithi164 ceph-mon[30928]: osdmap e74: 12 total, 12 up, 12 in 2024-09-19T08:43:02.119 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:01 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:43:02.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:01 smithi170 ceph-mon[35072]: purged_snaps scrub starts 2024-09-19T08:43:02.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:01 smithi170 ceph-mon[35072]: purged_snaps scrub ok 2024-09-19T08:43:02.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:01 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:43:02.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:01 smithi170 ceph-mon[35072]: osd.11 [v2:172.21.15.173:6824/2831924738,v1:172.21.15.173:6825/2831924738] boot 2024-09-19T08:43:02.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:01 smithi170 ceph-mon[35072]: osdmap e74: 12 total, 12 up, 12 in 2024-09-19T08:43:02.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:01 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:43:02.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:01 smithi173 ceph-mon[35306]: purged_snaps scrub starts 2024-09-19T08:43:02.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:01 smithi173 ceph-mon[35306]: purged_snaps scrub ok 2024-09-19T08:43:02.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:01 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:43:02.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:01 smithi173 ceph-mon[35306]: osd.11 [v2:172.21.15.173:6824/2831924738,v1:172.21.15.173:6825/2831924738] boot 2024-09-19T08:43:02.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:01 smithi173 ceph-mon[35306]: osdmap e74: 12 total, 12 up, 12 in 2024-09-19T08:43:02.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:01 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:43:02.364 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:43:02.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.812+0000 7f4fdee54640 1 -- 172.21.15.164:0/3289924804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fd81037c0 msgr2=0x7f4fd810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:02.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.812+0000 7f4fdee54640 1 --2- 172.21.15.164:0/3289924804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fd81037c0 0x7f4fd810a050 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0009ca0 tx=0x7f4fd002a8c0 comp rx=0 tx=0).stop 2024-09-19T08:43:02.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.812+0000 7f4fdee54640 1 -- 172.21.15.164:0/3289924804 shutdown_connections 2024-09-19T08:43:02.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.812+0000 7f4fdee54640 1 --2- 172.21.15.164:0/3289924804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fd81037c0 0x7f4fd810a050 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:02.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.812+0000 7f4fdee54640 1 --2- 172.21.15.164:0/3289924804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fd8102e00 0x7f4fd8103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:02.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.812+0000 7f4fdee54640 1 --2- 172.21.15.164:0/3289924804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fd8101c00 0x7f4fd8102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:02.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.812+0000 7f4fdee54640 1 -- 172.21.15.164:0/3289924804 >> 172.21.15.164:0/3289924804 conn(0x7f4fd80fd390 msgr2=0x7f4fd80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:02.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.812+0000 7f4fdee54640 1 -- 172.21.15.164:0/3289924804 shutdown_connections 2024-09-19T08:43:02.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.812+0000 7f4fdee54640 1 -- 172.21.15.164:0/3289924804 wait complete. 2024-09-19T08:43:02.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.812+0000 7f4fdee54640 1 Processor -- start 2024-09-19T08:43:02.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.812+0000 7f4fdee54640 1 -- start start 2024-09-19T08:43:02.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.813+0000 7f4fdee54640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fd8101c00 0x7f4fd8197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:02.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.813+0000 7f4fdee54640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fd8102e00 0x7f4fd81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:02.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.813+0000 7f4fdee54640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fd81037c0 0x7f4fd819f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:02.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.813+0000 7f4fdee54640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd810d880 con 0x7f4fd8101c00 2024-09-19T08:43:02.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.813+0000 7f4fdee54640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd810d700 con 0x7f4fd81037c0 2024-09-19T08:43:02.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.813+0000 7f4fdee54640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd810da00 con 0x7f4fd8102e00 2024-09-19T08:43:02.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.813+0000 7f4fdcbf0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fd8101c00 0x7f4fd8197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:02.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.813+0000 7f4fdcbf0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fd8101c00 0x7f4fd8197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46208/0 (socket says 172.21.15.164:46208) 2024-09-19T08:43:02.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.813+0000 7f4fdcbf0640 1 -- 172.21.15.164:0/932365224 learned_addr learned my addr 172.21.15.164:0/932365224 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:43:02.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.813+0000 7f4fd7fff640 1 --2- 172.21.15.164:0/932365224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fd8102e00 0x7f4fd81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:02.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.813+0000 7f4fdd3f1640 1 --2- 172.21.15.164:0/932365224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fd81037c0 0x7f4fd819f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:02.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.813+0000 7f4fdcbf0640 1 -- 172.21.15.164:0/932365224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fd8102e00 msgr2=0x7f4fd81983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:02.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.813+0000 7f4fdcbf0640 1 --2- 172.21.15.164:0/932365224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fd8102e00 0x7f4fd81983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:02.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.813+0000 7f4fdcbf0640 1 -- 172.21.15.164:0/932365224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fd81037c0 msgr2=0x7f4fd819f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:02.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.813+0000 7f4fdcbf0640 1 --2- 172.21.15.164:0/932365224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fd81037c0 0x7f4fd819f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:02.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.813+0000 7f4fdcbf0640 1 -- 172.21.15.164:0/932365224 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd819fad0 con 0x7f4fd8101c00 2024-09-19T08:43:02.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.813+0000 7f4fd7fff640 1 --2- 172.21.15.164:0/932365224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fd8102e00 0x7f4fd81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:43:02.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.813+0000 7f4fdd3f1640 1 --2- 172.21.15.164:0/932365224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fd81037c0 0x7f4fd819f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:02.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.813+0000 7f4fdcbf0640 1 --2- 172.21.15.164:0/932365224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fd8101c00 0x7f4fd8197ea0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc00c970 tx=0x7f4fcc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:02.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.813+0000 7f4fd5ffb640 1 -- 172.21.15.164:0/932365224 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc004220 con 0x7f4fd8101c00 2024-09-19T08:43:02.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.814+0000 7f4fdee54640 1 -- 172.21.15.164:0/932365224 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd819fdc0 con 0x7f4fd8101c00 2024-09-19T08:43:02.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.814+0000 7f4fd5ffb640 1 -- 172.21.15.164:0/932365224 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fcc0043c0 con 0x7f4fd8101c00 2024-09-19T08:43:02.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.814+0000 7f4fd5ffb640 1 -- 172.21.15.164:0/932365224 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc00f450 con 0x7f4fd8101c00 2024-09-19T08:43:02.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.814+0000 7f4fdee54640 1 -- 172.21.15.164:0/932365224 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd81a0270 con 0x7f4fd8101c00 2024-09-19T08:43:02.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.815+0000 7f4fdee54640 1 -- 172.21.15.164:0/932365224 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fd8108f80 con 0x7f4fd8101c00 2024-09-19T08:43:02.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.816+0000 7f4fd5ffb640 1 -- 172.21.15.164:0/932365224 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f4fcc0079b0 con 0x7f4fd8101c00 2024-09-19T08:43:02.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.816+0000 7f4fd5ffb640 1 --2- 172.21.15.164:0/932365224 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f4fc80a32f0 0x7f4fc80a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:02.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.817+0000 7f4fd7fff640 1 --2- 172.21.15.164:0/932365224 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f4fc80a32f0 0x7f4fc80a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:02.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.817+0000 7f4fd5ffb640 1 -- 172.21.15.164:0/932365224 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f4fcc0cf1f0 con 0x7f4fd8101c00 2024-09-19T08:43:02.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.817+0000 7f4fd7fff640 1 --2- 172.21.15.164:0/932365224 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f4fc80a32f0 0x7f4fc80a57b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4fc40058c0 tx=0x7f4fc4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:02.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.821+0000 7f4fd5ffb640 1 -- 172.21.15.164:0/932365224 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4fcc014030 con 0x7f4fd8101c00 2024-09-19T08:43:02.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.988+0000 7f4fdee54640 1 -- 172.21.15.164:0/932365224 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f4fd802da70 con 0x7f4fd8101c00 2024-09-19T08:43:02.989 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:02 smithi164 ceph-mon[30928]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:43:02.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.989+0000 7f4fd5ffb640 1 -- 172.21.15.164:0/932365224 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) ==== 74+0+133 (secure 0 0 0) 0x7f4fcc09c3b0 con 0x7f4fd8101c00 2024-09-19T08:43:02.990 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:43:02.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.991+0000 7f4fdee54640 1 -- 172.21.15.164:0/932365224 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f4fc80a32f0 msgr2=0x7f4fc80a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:02.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.991+0000 7f4fdee54640 1 --2- 172.21.15.164:0/932365224 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f4fc80a32f0 0x7f4fc80a57b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4fc40058c0 tx=0x7f4fc4005850 comp rx=0 tx=0).stop 2024-09-19T08:43:02.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.991+0000 7f4fdee54640 1 -- 172.21.15.164:0/932365224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fd8101c00 msgr2=0x7f4fd8197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:02.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.991+0000 7f4fdee54640 1 --2- 172.21.15.164:0/932365224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fd8101c00 0x7f4fd8197ea0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc00c970 tx=0x7f4fcc00ce30 comp rx=0 tx=0).stop 2024-09-19T08:43:02.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.991+0000 7f4fdee54640 1 -- 172.21.15.164:0/932365224 shutdown_connections 2024-09-19T08:43:02.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.991+0000 7f4fdee54640 1 --2- 172.21.15.164:0/932365224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fd81037c0 0x7f4fd819f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:02.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.991+0000 7f4fdee54640 1 --2- 172.21.15.164:0/932365224 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f4fc80a32f0 0x7f4fc80a57b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:02.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.991+0000 7f4fdee54640 1 --2- 172.21.15.164:0/932365224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fd8102e00 0x7f4fd81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:02.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.991+0000 7f4fdee54640 1 --2- 172.21.15.164:0/932365224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fd8101c00 0x7f4fd8197ea0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:02.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.991+0000 7f4fdee54640 1 -- 172.21.15.164:0/932365224 >> 172.21.15.164:0/932365224 conn(0x7f4fd80fd390 msgr2=0x7f4fd8106150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:02.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.991+0000 7f4fdee54640 1 -- 172.21.15.164:0/932365224 shutdown_connections 2024-09-19T08:43:02.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:02.991+0000 7f4fdee54640 1 -- 172.21.15.164:0/932365224 wait complete. 2024-09-19T08:43:03.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:02 smithi170 ceph-mon[35072]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:43:03.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:02 smithi173 ceph-mon[35306]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-09-19T08:43:03.791 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":75,"num_osds":12,"num_up_osds":12,"osd_up_since":1726735381,"num_in_osds":12,"osd_in_since":1726735357,"num_remapped_pgs":0} 2024-09-19T08:43:03.791 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T08:43:04.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:03 smithi164 ceph-mon[30928]: osdmap e75: 12 total, 12 up, 12 in 2024-09-19T08:43:04.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:03 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/932365224' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T08:43:04.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:03 smithi164 ceph-mon[30928]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:04.073 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:43:04.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:03 smithi170 ceph-mon[35072]: osdmap e75: 12 total, 12 up, 12 in 2024-09-19T08:43:04.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:03 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/932365224' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T08:43:04.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:03 smithi170 ceph-mon[35072]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:04.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:03 smithi173 ceph-mon[35306]: osdmap e75: 12 total, 12 up, 12 in 2024-09-19T08:43:04.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:03 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/932365224' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T08:43:04.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:03 smithi173 ceph-mon[35306]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:04.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.526+0000 7f90c734b640 1 -- 172.21.15.164:0/2516791728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90c0100e20 msgr2=0x7f90c0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:04.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.526+0000 7f90c734b640 1 --2- 172.21.15.164:0/2516791728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90c0100e20 0x7f90c0101200 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f90bc009900 tx=0x7f90bc02a480 comp rx=0 tx=0).stop 2024-09-19T08:43:04.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.527+0000 7f90c734b640 1 -- 172.21.15.164:0/2516791728 shutdown_connections 2024-09-19T08:43:04.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.527+0000 7f90c734b640 1 --2- 172.21.15.164:0/2516791728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90c0102ef0 0x7f90c010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.527+0000 7f90c734b640 1 --2- 172.21.15.164:0/2516791728 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90c0102570 0x7f90c01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.527+0000 7f90c734b640 1 --2- 172.21.15.164:0/2516791728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90c0100e20 0x7f90c0101200 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.527+0000 7f90c734b640 1 -- 172.21.15.164:0/2516791728 >> 172.21.15.164:0/2516791728 conn(0x7f90c00fc7d0 msgr2=0x7f90c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:04.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.527+0000 7f90c734b640 1 -- 172.21.15.164:0/2516791728 shutdown_connections 2024-09-19T08:43:04.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.527+0000 7f90c734b640 1 -- 172.21.15.164:0/2516791728 wait complete. 2024-09-19T08:43:04.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.527+0000 7f90c734b640 1 Processor -- start 2024-09-19T08:43:04.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.527+0000 7f90c734b640 1 -- start start 2024-09-19T08:43:04.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.527+0000 7f90c734b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90c0100e20 0x7f90c0198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:04.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.527+0000 7f90c734b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90c0102570 0x7f90c0198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:04.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.527+0000 7f90c734b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90c0102ef0 0x7f90c019cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:04.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.527+0000 7f90c734b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f90c0111020 con 0x7f90c0102ef0 2024-09-19T08:43:04.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.527+0000 7f90c734b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f90c0110ea0 con 0x7f90c0100e20 2024-09-19T08:43:04.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.527+0000 7f90c734b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f90c01111a0 con 0x7f90c0102570 2024-09-19T08:43:04.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.528+0000 7f90c58e8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90c0102ef0 0x7f90c019cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:04.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.528+0000 7f90c58e8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90c0102ef0 0x7f90c019cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38010/0 (socket says 172.21.15.164:38010) 2024-09-19T08:43:04.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.528+0000 7f90c58e8640 1 -- 172.21.15.164:0/475830604 learned_addr learned my addr 172.21.15.164:0/475830604 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:43:04.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.528+0000 7f90c48e6640 1 --2- 172.21.15.164:0/475830604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90c0102570 0x7f90c0198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:04.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.528+0000 7f90c50e7640 1 --2- 172.21.15.164:0/475830604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90c0100e20 0x7f90c0198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:04.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.528+0000 7f90c58e8640 1 -- 172.21.15.164:0/475830604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90c0102570 msgr2=0x7f90c0198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:04.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.528+0000 7f90c58e8640 1 --2- 172.21.15.164:0/475830604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90c0102570 0x7f90c0198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.528+0000 7f90c58e8640 1 -- 172.21.15.164:0/475830604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90c0100e20 msgr2=0x7f90c0198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:04.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.528+0000 7f90c58e8640 1 --2- 172.21.15.164:0/475830604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90c0100e20 0x7f90c0198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.528+0000 7f90c58e8640 1 -- 172.21.15.164:0/475830604 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c019d5f0 con 0x7f90c0102ef0 2024-09-19T08:43:04.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.528+0000 7f90c58e8640 1 --2- 172.21.15.164:0/475830604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90c0102ef0 0x7f90c019cf10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f90b800b970 tx=0x7f90b800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:04.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.528+0000 7f90b67fc640 1 -- 172.21.15.164:0/475830604 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b8019070 con 0x7f90c0102ef0 2024-09-19T08:43:04.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.528+0000 7f90b67fc640 1 -- 172.21.15.164:0/475830604 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90b800cb40 con 0x7f90c0102ef0 2024-09-19T08:43:04.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.528+0000 7f90b67fc640 1 -- 172.21.15.164:0/475830604 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b8007650 con 0x7f90c0102ef0 2024-09-19T08:43:04.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.528+0000 7f90c734b640 1 -- 172.21.15.164:0/475830604 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90c019d8e0 con 0x7f90c0102ef0 2024-09-19T08:43:04.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.528+0000 7f90c734b640 1 -- 172.21.15.164:0/475830604 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90c01a51c0 con 0x7f90c0102ef0 2024-09-19T08:43:04.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.529+0000 7f90c734b640 1 -- 172.21.15.164:0/475830604 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90c0101280 con 0x7f90c0102ef0 2024-09-19T08:43:04.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.530+0000 7f90b67fc640 1 -- 172.21.15.164:0/475830604 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f90b801c070 con 0x7f90c0102ef0 2024-09-19T08:43:04.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.531+0000 7f90b67fc640 1 --2- 172.21.15.164:0/475830604 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f90ac0a32c0 0x7f90ac0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:04.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.531+0000 7f90c50e7640 1 --2- 172.21.15.164:0/475830604 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f90ac0a32c0 0x7f90ac0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:04.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.531+0000 7f90b67fc640 1 -- 172.21.15.164:0/475830604 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f90b80ceaa0 con 0x7f90c0102ef0 2024-09-19T08:43:04.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.532+0000 7f90c50e7640 1 --2- 172.21.15.164:0/475830604 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f90ac0a32c0 0x7f90ac0a5780 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f90bc005ab0 tx=0x7f90bc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:04.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.535+0000 7f90b67fc640 1 -- 172.21.15.164:0/475830604 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f90b809bc60 con 0x7f90c0102ef0 2024-09-19T08:43:04.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.706+0000 7f90c734b640 1 -- 172.21.15.164:0/475830604 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f90c010a0a0 con 0x7f90c0102ef0 2024-09-19T08:43:04.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.707+0000 7f90b67fc640 1 -- 172.21.15.164:0/475830604 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+18551 (secure 0 0 0) 0x7f90b809b560 con 0x7f90c0102ef0 2024-09-19T08:43:04.708 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:43:04.708 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":75,"fsid":"1e4b0d74-7662-11ef-bceb-c7b262605968","created":"2024-09-19T08:36:09.564509+0000","modified":"2024-09-19T08:43:02.831839+0000","last_up_change":"2024-09-19T08:43:01.260852+0000","last_in_change":"2024-09-19T08:42:37.492464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:39:13.961749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33b11f37-6ac5-41fc-8cfd-1b08c7eef688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6803","nonce":663038243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6805","nonce":663038243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6809","nonce":663038243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6807","nonce":663038243}]},"public_addr":"172.21.15.164:6803/663038243","cluster_addr":"172.21.15.164:6805/663038243","heartbeat_back_addr":"172.21.15.164:6809/663038243","heartbeat_front_addr":"172.21.15.164:6807/663038243","state":["exists","up"]},{"osd":1,"uuid":"6de64b0a-46e2-4a98-8005-a326a64bc87d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6801","nonce":2167066371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6803","nonce":2167066371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6807","nonce":2167066371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6805","nonce":2167066371}]},"public_addr":"172.21.15.170:6801/2167066371","cluster_addr":"172.21.15.170:6803/2167066371","heartbeat_back_addr":"172.21.15.170:6807/2167066371","heartbeat_front_addr":"172.21.15.170:6805/2167066371","state":["exists","up"]},{"osd":2,"uuid":"d4158cc7-9822-42a7-9601-d65089907584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6801","nonce":1646561048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6803","nonce":1646561048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6807","nonce":1646561048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6805","nonce":1646561048}]},"public_addr":"172.21.15.173:6801/1646561048","cluster_addr":"172.21.15.173:6803/1646561048","heartbeat_back_addr":"172.21.15.173:6807/1646561048","heartbeat_front_addr":"172.21.15.173:6805/1646561048","state":["exists","up"]},{"osd":3,"uuid":"0cf3f7f1-3a8c-42ce-88a9-9bea52e02221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6811","nonce":406795560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6813","nonce":406795560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6817","nonce":406795560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6815","nonce":406795560}]},"public_addr":"172.21.15.164:6811/406795560","cluster_addr":"172.21.15.164:6813/406795560","heartbeat_back_addr":"172.21.15.164:6817/406795560","heartbeat_front_addr":"172.21.15.164:6815/406795560","state":["exists","up"]},{"osd":4,"uuid":"6dca048a-2860-4a02-8d8e-450b30bc1a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6809","nonce":4120832165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6811","nonce":4120832165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6815","nonce":4120832165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6813","nonce":4120832165}]},"public_addr":"172.21.15.170:6809/4120832165","cluster_addr":"172.21.15.170:6811/4120832165","heartbeat_back_addr":"172.21.15.170:6815/4120832165","heartbeat_front_addr":"172.21.15.170:6813/4120832165","state":["exists","up"]},{"osd":5,"uuid":"3822521f-69d6-4141-a83f-5fc9cecc7b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6809","nonce":2914653552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6811","nonce":2914653552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6815","nonce":2914653552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6813","nonce":2914653552}]},"public_addr":"172.21.15.173:6809/2914653552","cluster_addr":"172.21.15.173:6811/2914653552","heartbeat_back_addr":"172.21.15.173:6815/2914653552","heartbeat_front_addr":"172.21.15.173:6813/2914653552","state":["exists","up"]},{"osd":6,"uuid":"0f5a965e-2eb2-4c24-b089-577e0639c160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6819","nonce":1138341393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6821","nonce":1138341393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6825","nonce":1138341393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6823","nonce":1138341393}]},"public_addr":"172.21.15.164:6819/1138341393","cluster_addr":"172.21.15.164:6821/1138341393","heartbeat_back_addr":"172.21.15.164:6825/1138341393","heartbeat_front_addr":"172.21.15.164:6823/1138341393","state":["exists","up"]},{"osd":7,"uuid":"f16cb2c3-639b-46ae-a740-1bd28ca1863c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6817","nonce":2124160360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6819","nonce":2124160360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6823","nonce":2124160360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6821","nonce":2124160360}]},"public_addr":"172.21.15.170:6817/2124160360","cluster_addr":"172.21.15.170:6819/2124160360","heartbeat_back_addr":"172.21.15.170:6823/2124160360","heartbeat_front_addr":"172.21.15.170:6821/2124160360","state":["exists","up"]},{"osd":8,"uuid":"161cff01-46ad-48e3-b999-a5473a2ad618","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6817","nonce":2285175061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6819","nonce":2285175061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6823","nonce":2285175061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6821","nonce":2285175061}]},"public_addr":"172.21.15.173:6817/2285175061","cluster_addr":"172.21.15.173:6819/2285175061","heartbeat_back_addr":"172.21.15.173:6823/2285175061","heartbeat_front_addr":"172.21.15.173:6821/2285175061","state":["exists","up"]},{"osd":9,"uuid":"994666e8-8fbd-4231-ac67-81a79aaf7c2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6827","nonce":3918847952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6829","nonce":3918847952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6833","nonce":3918847952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6831","nonce":3918847952}]},"public_addr":"172.21.15.164:6827/3918847952","cluster_addr":"172.21.15.164:6829/3918847952","heartbeat_back_addr":"172.21.15.164:6833/3918847952","heartbeat_front_addr":"172.21.15.164:6831/3918847952","state":["exists","up"]},{"osd":10,"uuid":"ff5e9099-cdb4-4905-b888-5b67320ef76b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6825","nonce":2707848227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6827","nonce":2707848227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6831","nonce":2707848227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6829","nonce":2707848227}]},"public_addr":"172.21.15.170:6825/2707848227","cluster_addr":"172.21.15.170:6827/2707848227","heartbeat_back_addr":"172.21.15.170:6831/2707848227","heartbeat_front_addr":"172.21.15.170:6829/2707848227","state":["exists","up"]},{"osd":11,"uuid":"4d93088f-e791-4938-a207-12f15b56abd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6825","nonce":2831924738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6827","nonce":2831924738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6831","nonce":2831924738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6829","nonce":2831924738}]},"public_addr":"172.21.15.173:6825/2831924738","cluster_addr":"172.21.15.173:6827/2831924738","heartbeat_back_addr":"172.21.15.173:6831/2831924738","heartbeat_front_addr":"172.21.15.173:6829/2831924738","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:21.787352+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:46.012323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:39:11.353040+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:39:37.015124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:02.063379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:27.892865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:53.372041+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:41:17.855284+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:41:43.389882+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:08.956895+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:34.069779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:59.303897+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.164:0/339032808":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6801/1173822158":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6800/1173822158":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6801/2805828009":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1837164919":"2024-09-20T08:37:01.663065+0000","172.21.15.164:0/4018459143":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6800/2805828009":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/2770797546":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1905394497":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1369265436":"2024-09-20T08:36:33.856642+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:43:04.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.709+0000 7f90c734b640 1 -- 172.21.15.164:0/475830604 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f90ac0a32c0 msgr2=0x7f90ac0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:04.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.709+0000 7f90c734b640 1 --2- 172.21.15.164:0/475830604 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f90ac0a32c0 0x7f90ac0a5780 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f90bc005ab0 tx=0x7f90bc005a20 comp rx=0 tx=0).stop 2024-09-19T08:43:04.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.709+0000 7f90c734b640 1 -- 172.21.15.164:0/475830604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90c0102ef0 msgr2=0x7f90c019cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:04.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.709+0000 7f90c734b640 1 --2- 172.21.15.164:0/475830604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90c0102ef0 0x7f90c019cf10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f90b800b970 tx=0x7f90b800be30 comp rx=0 tx=0).stop 2024-09-19T08:43:04.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.709+0000 7f90c734b640 1 -- 172.21.15.164:0/475830604 shutdown_connections 2024-09-19T08:43:04.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.709+0000 7f90c734b640 1 --2- 172.21.15.164:0/475830604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90c0102ef0 0x7f90c019cf10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.709+0000 7f90c734b640 1 --2- 172.21.15.164:0/475830604 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f90ac0a32c0 0x7f90ac0a5780 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.709+0000 7f90c734b640 1 --2- 172.21.15.164:0/475830604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90c0102570 0x7f90c0198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.709+0000 7f90c734b640 1 --2- 172.21.15.164:0/475830604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90c0100e20 0x7f90c0198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.709+0000 7f90c734b640 1 -- 172.21.15.164:0/475830604 >> 172.21.15.164:0/475830604 conn(0x7f90c00fc7d0 msgr2=0x7f90c006a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:04.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.709+0000 7f90c734b640 1 -- 172.21.15.164:0/475830604 shutdown_connections 2024-09-19T08:43:04.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:04.710+0000 7f90c734b640 1 -- 172.21.15.164:0/475830604 wait complete. 2024-09-19T08:43:05.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:04 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/475830604' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:43:05.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:04 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/475830604' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:43:05.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:04 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/475830604' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:43:05.524 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T08:39:13.961749+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T08:43:05.525 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-19T08:43:05.767 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:43:06.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:05 smithi170 ceph-mon[35072]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:06.130 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:05 smithi164 ceph-mon[30928]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:06.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:05 smithi173 ceph-mon[35306]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:06.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.211+0000 7f324608c640 1 -- 172.21.15.164:0/2005353849 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3240103660 msgr2=0x7f324010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:06.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.211+0000 7f324608c640 1 --2- 172.21.15.164:0/2005353849 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3240103660 0x7f324010bb50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3234009900 tx=0x7f323402a480 comp rx=0 tx=0).stop 2024-09-19T08:43:06.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.211+0000 7f324608c640 1 -- 172.21.15.164:0/2005353849 shutdown_connections 2024-09-19T08:43:06.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.211+0000 7f324608c640 1 --2- 172.21.15.164:0/2005353849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f324010c170 0x7f324010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:06.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.211+0000 7f324608c640 1 --2- 172.21.15.164:0/2005353849 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3240103660 0x7f324010bb50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:06.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.211+0000 7f324608c640 1 --2- 172.21.15.164:0/2005353849 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3240102d40 0x7f3240103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:06.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.211+0000 7f324608c640 1 -- 172.21.15.164:0/2005353849 >> 172.21.15.164:0/2005353849 conn(0x7f32400fc7a0 msgr2=0x7f32400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:06.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.211+0000 7f324608c640 1 -- 172.21.15.164:0/2005353849 shutdown_connections 2024-09-19T08:43:06.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.211+0000 7f324608c640 1 -- 172.21.15.164:0/2005353849 wait complete. 2024-09-19T08:43:06.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.212+0000 7f324608c640 1 Processor -- start 2024-09-19T08:43:06.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.212+0000 7f324608c640 1 -- start start 2024-09-19T08:43:06.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.212+0000 7f324608c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3240102d40 0x7f32401984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:06.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.212+0000 7f324608c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3240103660 0x7f32401989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:06.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.212+0000 7f324608c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f324010c170 0x7f324019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:06.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.212+0000 7f324608c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3240111fd0 con 0x7f3240103660 2024-09-19T08:43:06.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.212+0000 7f324608c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3240111e50 con 0x7f3240102d40 2024-09-19T08:43:06.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.212+0000 7f324608c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3240112150 con 0x7f324010c170 2024-09-19T08:43:06.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.212+0000 7f323effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3240103660 0x7f32401989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:06.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.212+0000 7f323effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3240103660 0x7f32401989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38026/0 (socket says 172.21.15.164:38026) 2024-09-19T08:43:06.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.212+0000 7f323effd640 1 -- 172.21.15.164:0/4030905128 learned_addr learned my addr 172.21.15.164:0/4030905128 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:43:06.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.213+0000 7f323f7fe640 1 --2- 172.21.15.164:0/4030905128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3240102d40 0x7f32401984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:06.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.213+0000 7f323ffff640 1 --2- 172.21.15.164:0/4030905128 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f324010c170 0x7f324019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:06.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.213+0000 7f323effd640 1 -- 172.21.15.164:0/4030905128 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f324010c170 msgr2=0x7f324019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:06.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.213+0000 7f323effd640 1 --2- 172.21.15.164:0/4030905128 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f324010c170 0x7f324019cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:06.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.213+0000 7f323effd640 1 -- 172.21.15.164:0/4030905128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3240102d40 msgr2=0x7f32401984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:06.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.213+0000 7f323effd640 1 --2- 172.21.15.164:0/4030905128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3240102d40 0x7f32401984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:06.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.213+0000 7f323effd640 1 -- 172.21.15.164:0/4030905128 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f324019d460 con 0x7f3240103660 2024-09-19T08:43:06.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.213+0000 7f323ffff640 1 --2- 172.21.15.164:0/4030905128 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f324010c170 0x7f324019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:06.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.213+0000 7f323effd640 1 --2- 172.21.15.164:0/4030905128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3240103660 0x7f32401989f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3234005ab0 tx=0x7f32340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:06.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.213+0000 7f323cff9640 1 -- 172.21.15.164:0/4030905128 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323403f030 con 0x7f3240103660 2024-09-19T08:43:06.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.213+0000 7f324608c640 1 -- 172.21.15.164:0/4030905128 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f324019d6f0 con 0x7f3240103660 2024-09-19T08:43:06.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.213+0000 7f323cff9640 1 -- 172.21.15.164:0/4030905128 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3234003ce0 con 0x7f3240103660 2024-09-19T08:43:06.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.213+0000 7f323cff9640 1 -- 172.21.15.164:0/4030905128 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3234035070 con 0x7f3240103660 2024-09-19T08:43:06.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.213+0000 7f324608c640 1 -- 172.21.15.164:0/4030905128 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32401a5030 con 0x7f3240103660 2024-09-19T08:43:06.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.214+0000 7f324608c640 1 -- 172.21.15.164:0/4030905128 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f324006b070 con 0x7f3240103660 2024-09-19T08:43:06.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.215+0000 7f323cff9640 1 -- 172.21.15.164:0/4030905128 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f3234042030 con 0x7f3240103660 2024-09-19T08:43:06.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.216+0000 7f323cff9640 1 --2- 172.21.15.164:0/4030905128 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f32280a3240 0x7f32280a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:06.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.216+0000 7f323f7fe640 1 --2- 172.21.15.164:0/4030905128 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f32280a3240 0x7f32280a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:06.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.216+0000 7f323cff9640 1 -- 172.21.15.164:0/4030905128 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f32340efff0 con 0x7f3240103660 2024-09-19T08:43:06.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.216+0000 7f323f7fe640 1 --2- 172.21.15.164:0/4030905128 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f32280a3240 0x7f32280a5700 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3230005c40 tx=0x7f323000a390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:06.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.219+0000 7f323cff9640 1 -- 172.21.15.164:0/4030905128 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f32340bd1b0 con 0x7f3240103660 2024-09-19T08:43:06.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.385+0000 7f324608c640 1 -- 172.21.15.164:0/4030905128 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f32401071e0 con 0x7f3240103660 2024-09-19T08:43:06.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.385+0000 7f323cff9640 1 -- 172.21.15.164:0/4030905128 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) ==== 93+0+10 (secure 0 0 0) 0x7f32340bcab0 con 0x7f3240103660 2024-09-19T08:43:06.386 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-09-19T08:43:06.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.388+0000 7f324608c640 1 -- 172.21.15.164:0/4030905128 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f32280a3240 msgr2=0x7f32280a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:06.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.388+0000 7f324608c640 1 --2- 172.21.15.164:0/4030905128 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f32280a3240 0x7f32280a5700 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3230005c40 tx=0x7f323000a390 comp rx=0 tx=0).stop 2024-09-19T08:43:06.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.388+0000 7f324608c640 1 -- 172.21.15.164:0/4030905128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3240103660 msgr2=0x7f32401989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:06.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.388+0000 7f324608c640 1 --2- 172.21.15.164:0/4030905128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3240103660 0x7f32401989f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3234005ab0 tx=0x7f32340042f0 comp rx=0 tx=0).stop 2024-09-19T08:43:06.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.388+0000 7f324608c640 1 -- 172.21.15.164:0/4030905128 shutdown_connections 2024-09-19T08:43:06.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.388+0000 7f324608c640 1 --2- 172.21.15.164:0/4030905128 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f32280a3240 0x7f32280a5700 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:06.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.388+0000 7f324608c640 1 --2- 172.21.15.164:0/4030905128 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f324010c170 0x7f324019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:06.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.388+0000 7f324608c640 1 --2- 172.21.15.164:0/4030905128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3240103660 0x7f32401989f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:06.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.388+0000 7f324608c640 1 --2- 172.21.15.164:0/4030905128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3240102d40 0x7f32401984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:06.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.388+0000 7f324608c640 1 -- 172.21.15.164:0/4030905128 >> 172.21.15.164:0/4030905128 conn(0x7f32400fc7a0 msgr2=0x7f32400fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:06.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.388+0000 7f324608c640 1 -- 172.21.15.164:0/4030905128 shutdown_connections 2024-09-19T08:43:06.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:06.388+0000 7f324608c640 1 -- 172.21.15.164:0/4030905128 wait complete. 2024-09-19T08:43:07.102 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:06 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/4030905128' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T08:43:07.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:06 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/4030905128' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T08:43:07.128 INFO:tasks.cephadm:Adding mds.a on smithi164 2024-09-19T08:43:07.129 INFO:tasks.cephadm:Adding mds.d on smithi164 2024-09-19T08:43:07.129 INFO:tasks.cephadm:Adding mds.g on smithi164 2024-09-19T08:43:07.129 INFO:tasks.cephadm:Adding mds.j on smithi164 2024-09-19T08:43:07.129 INFO:tasks.cephadm:Adding mds.b on smithi170 2024-09-19T08:43:07.129 INFO:tasks.cephadm:Adding mds.e on smithi170 2024-09-19T08:43:07.129 INFO:tasks.cephadm:Adding mds.h on smithi170 2024-09-19T08:43:07.129 INFO:tasks.cephadm:Adding mds.k on smithi170 2024-09-19T08:43:07.129 INFO:tasks.cephadm:Adding mds.c on smithi173 2024-09-19T08:43:07.129 INFO:tasks.cephadm:Adding mds.f on smithi173 2024-09-19T08:43:07.129 INFO:tasks.cephadm:Adding mds.i on smithi173 2024-09-19T08:43:07.129 INFO:tasks.cephadm:Adding mds.l on smithi173 2024-09-19T08:43:07.129 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph orch apply mds all '12;smithi164=a;smithi164=d;smithi164=g;smithi164=j;smithi170=b;smithi170=e;smithi170=h;smithi170=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l' 2024-09-19T08:43:07.158 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:06 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/4030905128' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T08:43:07.387 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:43:07.872 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.857+0000 7f6286c80640 1 -- 172.21.15.173:0/284302882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6280100ad0 msgr2=0x7f6280100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:07.872 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.857+0000 7f6286c80640 1 --2- 172.21.15.173:0/284302882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6280100ad0 0x7f6280100ed0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6274009900 tx=0x7f627402a410 comp rx=0 tx=0).stop 2024-09-19T08:43:07.873 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.857+0000 7f6286c80640 1 -- 172.21.15.173:0/284302882 shutdown_connections 2024-09-19T08:43:07.873 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.857+0000 7f6286c80640 1 --2- 172.21.15.173:0/284302882 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6280102690 0x7f6280108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:07.873 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.857+0000 7f6286c80640 1 --2- 172.21.15.173:0/284302882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6280101cd0 0x7f6280102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:07.873 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.857+0000 7f6286c80640 1 --2- 172.21.15.173:0/284302882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6280100ad0 0x7f6280100ed0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:07.873 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.857+0000 7f6286c80640 1 -- 172.21.15.173:0/284302882 >> 172.21.15.173:0/284302882 conn(0x7f62800fc280 msgr2=0x7f62800fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:07.873 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.857+0000 7f6286c80640 1 -- 172.21.15.173:0/284302882 shutdown_connections 2024-09-19T08:43:07.873 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.857+0000 7f6286c80640 1 -- 172.21.15.173:0/284302882 wait complete. 2024-09-19T08:43:07.873 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.857+0000 7f6286c80640 1 Processor -- start 2024-09-19T08:43:07.873 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.857+0000 7f6286c80640 1 -- start start 2024-09-19T08:43:07.873 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.858+0000 7f6286c80640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6280100ad0 0x7f6280197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:07.873 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.858+0000 7f6286c80640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6280101cd0 0x7f6280198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:07.873 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.858+0000 7f6286c80640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6280102690 0x7f628019f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:07.874 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.858+0000 7f6286c80640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f628010c730 con 0x7f6280102690 2024-09-19T08:43:07.874 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.858+0000 7f6286c80640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f628010c5b0 con 0x7f6280101cd0 2024-09-19T08:43:07.874 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.858+0000 7f6286c80640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f628010c8b0 con 0x7f6280100ad0 2024-09-19T08:43:07.874 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.858+0000 7f6284a1c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6280100ad0 0x7f6280197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:07.874 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.858+0000 7f6284a1c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6280100ad0 0x7f6280197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:37732/0 (socket says 172.21.15.173:37732) 2024-09-19T08:43:07.874 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.858+0000 7f6284a1c640 1 -- 172.21.15.173:0/2670129346 learned_addr learned my addr 172.21.15.173:0/2670129346 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-09-19T08:43:07.874 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.858+0000 7f627ffff640 1 --2- 172.21.15.173:0/2670129346 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6280101cd0 0x7f6280198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:07.874 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.858+0000 7f628521d640 1 --2- 172.21.15.173:0/2670129346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6280102690 0x7f628019f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:07.874 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.858+0000 7f6284a1c640 1 -- 172.21.15.173:0/2670129346 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6280101cd0 msgr2=0x7f6280198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:07.874 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.858+0000 7f6284a1c640 1 --2- 172.21.15.173:0/2670129346 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6280101cd0 0x7f6280198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:07.874 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.858+0000 7f6284a1c640 1 -- 172.21.15.173:0/2670129346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6280102690 msgr2=0x7f628019f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:07.874 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.858+0000 7f6284a1c640 1 --2- 172.21.15.173:0/2670129346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6280102690 0x7f628019f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:07.875 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.858+0000 7f6284a1c640 1 -- 172.21.15.173:0/2670129346 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f628019fb50 con 0x7f6280100ad0 2024-09-19T08:43:07.875 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.858+0000 7f628521d640 1 --2- 172.21.15.173:0/2670129346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6280102690 0x7f628019f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:43:07.875 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.859+0000 7f6284a1c640 1 --2- 172.21.15.173:0/2670129346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6280100ad0 0x7f6280197f20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6274031610 tx=0x7f6274004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:07.875 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.861+0000 7f627dffb640 1 -- 172.21.15.173:0/2670129346 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627403f030 con 0x7f6280100ad0 2024-09-19T08:43:07.875 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.861+0000 7f627dffb640 1 -- 172.21.15.173:0/2670129346 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6274003ad0 con 0x7f6280100ad0 2024-09-19T08:43:07.875 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.861+0000 7f6286c80640 1 -- 172.21.15.173:0/2670129346 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f628019fde0 con 0x7f6280100ad0 2024-09-19T08:43:07.875 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.861+0000 7f627dffb640 1 -- 172.21.15.173:0/2670129346 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6274035070 con 0x7f6280100ad0 2024-09-19T08:43:07.875 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.861+0000 7f6286c80640 1 -- 172.21.15.173:0/2670129346 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62801a0230 con 0x7f6280100ad0 2024-09-19T08:43:07.875 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.862+0000 7f6286c80640 1 -- 172.21.15.173:0/2670129346 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62801a0460 con 0x7f6280100ad0 2024-09-19T08:43:07.875 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.863+0000 7f627dffb640 1 -- 172.21.15.173:0/2670129346 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f6274042030 con 0x7f6280100ad0 2024-09-19T08:43:07.875 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.864+0000 7f627dffb640 1 --2- 172.21.15.173:0/2670129346 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f62700a32f0 0x7f62700a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:07.875 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.864+0000 7f627dffb640 1 -- 172.21.15.173:0/2670129346 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f62740efa20 con 0x7f6280100ad0 2024-09-19T08:43:07.875 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.864+0000 7f627ffff640 1 --2- 172.21.15.173:0/2670129346 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f62700a32f0 0x7f62700a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:07.876 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.864+0000 7f627ffff640 1 --2- 172.21.15.173:0/2670129346 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f62700a32f0 0x7f62700a57b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6280199440 tx=0x7f626c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:07.876 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:07.868+0000 7f627dffb640 1 -- 172.21.15.173:0/2670129346 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f62740bc590 con 0x7f6280100ad0 2024-09-19T08:43:08.039 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:07 smithi173 ceph-mon[35306]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:08.053 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:08.052+0000 7f6286c80640 1 -- 172.21.15.173:0/2670129346 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi164=a;smithi164=d;smithi164=g;smithi164=j;smithi170=b;smithi170=e;smithi170=h;smithi170=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l", "target": ["mon-mgr", ""]}) -- 0x7f628006a850 con 0x7f62700a32f0 2024-09-19T08:43:08.063 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:08.062+0000 7f627dffb640 1 -- 172.21.15.173:0/2670129346 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f628006a850 con 0x7f62700a32f0 2024-09-19T08:43:08.063 INFO:teuthology.orchestra.run.smithi173.stdout:Scheduled mds.all update... 2024-09-19T08:43:08.065 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:08.065+0000 7f6286c80640 1 -- 172.21.15.173:0/2670129346 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f62700a32f0 msgr2=0x7f62700a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:08.065 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:08.065+0000 7f6286c80640 1 --2- 172.21.15.173:0/2670129346 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f62700a32f0 0x7f62700a57b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6280199440 tx=0x7f626c005f90 comp rx=0 tx=0).stop 2024-09-19T08:43:08.066 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:08.065+0000 7f6286c80640 1 -- 172.21.15.173:0/2670129346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6280100ad0 msgr2=0x7f6280197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:08.066 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:08.065+0000 7f6286c80640 1 --2- 172.21.15.173:0/2670129346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6280100ad0 0x7f6280197f20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6274031610 tx=0x7f6274004270 comp rx=0 tx=0).stop 2024-09-19T08:43:08.066 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:08.065+0000 7f6286c80640 1 -- 172.21.15.173:0/2670129346 shutdown_connections 2024-09-19T08:43:08.066 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:08.065+0000 7f6286c80640 1 --2- 172.21.15.173:0/2670129346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6280102690 0x7f628019f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:08.066 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:08.065+0000 7f6286c80640 1 --2- 172.21.15.173:0/2670129346 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6280101cd0 0x7f6280198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:08.066 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:08.065+0000 7f6286c80640 1 --2- 172.21.15.173:0/2670129346 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f62700a32f0 0x7f62700a57b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:08.066 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:08.065+0000 7f6286c80640 1 --2- 172.21.15.173:0/2670129346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6280100ad0 0x7f6280197f20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:08.066 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:08.065+0000 7f6286c80640 1 -- 172.21.15.173:0/2670129346 >> 172.21.15.173:0/2670129346 conn(0x7f62800fc280 msgr2=0x7f628006a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:08.066 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:08.065+0000 7f6286c80640 1 -- 172.21.15.173:0/2670129346 shutdown_connections 2024-09-19T08:43:08.066 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-19T08:43:08.065+0000 7f6286c80640 1 -- 172.21.15.173:0/2670129346 wait complete. 2024-09-19T08:43:08.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:07 smithi170 ceph-mon[35072]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:08.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:07 smithi164 ceph-mon[30928]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:08.780 DEBUG:teuthology.orchestra.run.smithi164:mds.a> sudo journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.a.service 2024-09-19T08:43:08.783 DEBUG:teuthology.orchestra.run.smithi164:mds.d> sudo journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.d.service 2024-09-19T08:43:08.826 DEBUG:teuthology.orchestra.run.smithi164:mds.g> sudo journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.g.service 2024-09-19T08:43:08.869 DEBUG:teuthology.orchestra.run.smithi164:mds.j> sudo journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.j.service 2024-09-19T08:43:08.912 DEBUG:teuthology.orchestra.run.smithi170:mds.b> sudo journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.b.service 2024-09-19T08:43:08.916 DEBUG:teuthology.orchestra.run.smithi170:mds.e> sudo journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.e.service 2024-09-19T08:43:08.959 DEBUG:teuthology.orchestra.run.smithi170:mds.h> sudo journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.h.service 2024-09-19T08:43:09.004 DEBUG:teuthology.orchestra.run.smithi170:mds.k> sudo journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.k.service 2024-09-19T08:43:09.047 DEBUG:teuthology.orchestra.run.smithi173:mds.c> sudo journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.c.service 2024-09-19T08:43:09.050 DEBUG:teuthology.orchestra.run.smithi173:mds.f> sudo journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.f.service 2024-09-19T08:43:09.094 DEBUG:teuthology.orchestra.run.smithi173:mds.i> sudo journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.i.service 2024-09-19T08:43:09.137 DEBUG:teuthology.orchestra.run.smithi173:mds.l> sudo journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.l.service 2024-09-19T08:43:09.180 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-19T08:43:09.181 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:43:09.196 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:09 smithi173 ceph-mon[35306]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi164=a;smithi164=d;smithi164=g;smithi164=j;smithi170=b;smithi170=e;smithi170=h;smithi170=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:43:09.196 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:09 smithi173 ceph-mon[35306]: Saving service mds.all spec with placement smithi164=a;smithi164=d;smithi164=g;smithi164=j;smithi170=b;smithi170=e;smithi170=h;smithi170=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l;count:12 2024-09-19T08:43:09.196 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:09 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:09.196 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:09 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:43:09.196 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:09 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:09.196 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:09 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:43:09.196 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:09 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:09.196 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:09 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:09.196 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:09 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:09.197 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:09 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:09.197 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:09 smithi173 ceph-mon[35306]: Deploying daemon mds.l on smithi173 2024-09-19T08:43:09.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:09 smithi170 ceph-mon[35072]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi164=a;smithi164=d;smithi164=g;smithi164=j;smithi170=b;smithi170=e;smithi170=h;smithi170=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:43:09.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:09 smithi170 ceph-mon[35072]: Saving service mds.all spec with placement smithi164=a;smithi164=d;smithi164=g;smithi164=j;smithi170=b;smithi170=e;smithi170=h;smithi170=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l;count:12 2024-09-19T08:43:09.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:09 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:09.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:09 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:43:09.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:09 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:09.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:09 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:43:09.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:09 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:09.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:09 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:09.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:09 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:09.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:09 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:09.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:09 smithi170 ceph-mon[35072]: Deploying daemon mds.l on smithi173 2024-09-19T08:43:09.427 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:43:09.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:09 smithi164 ceph-mon[30928]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi164=a;smithi164=d;smithi164=g;smithi164=j;smithi170=b;smithi170=e;smithi170=h;smithi170=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:43:09.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:09 smithi164 ceph-mon[30928]: Saving service mds.all spec with placement smithi164=a;smithi164=d;smithi164=g;smithi164=j;smithi170=b;smithi170=e;smithi170=h;smithi170=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l;count:12 2024-09-19T08:43:09.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:09 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:09.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:09 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:43:09.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:09 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:09.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:09 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:43:09.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:09 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:09.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:09 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:09.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:09 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:09.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:09 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:09.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:09 smithi164 ceph-mon[30928]: Deploying daemon mds.l on smithi173 2024-09-19T08:43:09.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.850+0000 7f8a05249640 1 -- 172.21.15.164:0/2127910397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a00100e20 msgr2=0x7f8a00101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:09.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.850+0000 7f8a05249640 1 --2- 172.21.15.164:0/2127910397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a00100e20 0x7f8a00101200 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009900 tx=0x7f89f002a480 comp rx=0 tx=0).stop 2024-09-19T08:43:09.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.850+0000 7f8a05249640 1 -- 172.21.15.164:0/2127910397 shutdown_connections 2024-09-19T08:43:09.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.850+0000 7f8a05249640 1 --2- 172.21.15.164:0/2127910397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a00102ef0 0x7f8a0010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:09.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.850+0000 7f8a05249640 1 --2- 172.21.15.164:0/2127910397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a00102570 0x7f8a001029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:09.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.850+0000 7f8a05249640 1 --2- 172.21.15.164:0/2127910397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a00100e20 0x7f8a00101200 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:09.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.850+0000 7f8a05249640 1 -- 172.21.15.164:0/2127910397 >> 172.21.15.164:0/2127910397 conn(0x7f8a000fc7d0 msgr2=0x7f8a000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:09.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.850+0000 7f8a05249640 1 -- 172.21.15.164:0/2127910397 shutdown_connections 2024-09-19T08:43:09.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.850+0000 7f8a05249640 1 -- 172.21.15.164:0/2127910397 wait complete. 2024-09-19T08:43:09.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.851+0000 7f8a05249640 1 Processor -- start 2024-09-19T08:43:09.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.851+0000 7f8a05249640 1 -- start start 2024-09-19T08:43:09.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.851+0000 7f8a05249640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a00100e20 0x7f8a00198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:09.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.851+0000 7f8a05249640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a00102570 0x7f8a00198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:09.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.851+0000 7f8a05249640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a00102ef0 0x7f8a0019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:09.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.851+0000 7f8a05249640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8a001120a0 con 0x7f8a00102ef0 2024-09-19T08:43:09.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.851+0000 7f8a05249640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8a00111f20 con 0x7f8a00100e20 2024-09-19T08:43:09.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.851+0000 7f8a05249640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8a00112220 con 0x7f8a00102570 2024-09-19T08:43:09.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.851+0000 7f89ff59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a00102ef0 0x7f8a0019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:09.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.851+0000 7f89fe59c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a00102570 0x7f8a00198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:09.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.851+0000 7f89ff59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a00102ef0 0x7f8a0019ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38050/0 (socket says 172.21.15.164:38050) 2024-09-19T08:43:09.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.851+0000 7f89fe59c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a00102570 0x7f8a00198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:42792/0 (socket says 172.21.15.164:42792) 2024-09-19T08:43:09.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.851+0000 7f89ff59e640 1 -- 172.21.15.164:0/570942294 learned_addr learned my addr 172.21.15.164:0/570942294 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:43:09.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.852+0000 7f89fed9d640 1 --2- 172.21.15.164:0/570942294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a00100e20 0x7f8a00198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:09.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.852+0000 7f89fe59c640 1 -- 172.21.15.164:0/570942294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a00100e20 msgr2=0x7f8a00198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:09.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.852+0000 7f89fe59c640 1 --2- 172.21.15.164:0/570942294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a00100e20 0x7f8a00198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:09.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.852+0000 7f89fe59c640 1 -- 172.21.15.164:0/570942294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a00102ef0 msgr2=0x7f8a0019ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:09.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.852+0000 7f89fe59c640 1 --2- 172.21.15.164:0/570942294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a00102ef0 0x7f8a0019ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:09.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.852+0000 7f89fe59c640 1 -- 172.21.15.164:0/570942294 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0019d4f0 con 0x7f8a00102570 2024-09-19T08:43:09.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.852+0000 7f89ff59e640 1 --2- 172.21.15.164:0/570942294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a00102ef0 0x7f8a0019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:43:09.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.852+0000 7f89fed9d640 1 --2- 172.21.15.164:0/570942294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a00100e20 0x7f8a00198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:43:09.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.852+0000 7f89fe59c640 1 --2- 172.21.15.164:0/570942294 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a00102570 0x7f8a00198a80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f89f4005fd0 tx=0x7f89f400dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:09.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.852+0000 7f89e7fff640 1 -- 172.21.15.164:0/570942294 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f4009c30 con 0x7f8a00102570 2024-09-19T08:43:09.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.852+0000 7f89e7fff640 1 -- 172.21.15.164:0/570942294 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89f4009dd0 con 0x7f8a00102570 2024-09-19T08:43:09.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.852+0000 7f89e7fff640 1 -- 172.21.15.164:0/570942294 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f4010460 con 0x7f8a00102570 2024-09-19T08:43:09.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.852+0000 7f8a05249640 1 -- 172.21.15.164:0/570942294 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a0019d7e0 con 0x7f8a00102570 2024-09-19T08:43:09.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.852+0000 7f8a05249640 1 -- 172.21.15.164:0/570942294 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a001a50c0 con 0x7f8a00102570 2024-09-19T08:43:09.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.853+0000 7f8a05249640 1 -- 172.21.15.164:0/570942294 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a00101200 con 0x7f8a00102570 2024-09-19T08:43:09.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.855+0000 7f89e7fff640 1 -- 172.21.15.164:0/570942294 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f89f4003ef0 con 0x7f8a00102570 2024-09-19T08:43:09.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.855+0000 7f89e7fff640 1 --2- 172.21.15.164:0/570942294 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f89e80a32c0 0x7f89e80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:09.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.855+0000 7f89fed9d640 1 --2- 172.21.15.164:0/570942294 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f89e80a32c0 0x7f89e80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:09.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.855+0000 7f89e7fff640 1 -- 172.21.15.164:0/570942294 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f89f40cf060 con 0x7f8a00102570 2024-09-19T08:43:09.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.856+0000 7f89fed9d640 1 --2- 172.21.15.164:0/570942294 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f89e80a32c0 0x7f89e80a5780 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f89f0005ab0 tx=0x7f89f0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:09.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:09.858+0000 7f89e7fff640 1 -- 172.21.15.164:0/570942294 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f89f4017020 con 0x7f8a00102570 2024-09-19T08:43:10.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:10.060+0000 7f8a05249640 1 -- 172.21.15.164:0/570942294 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a001998a0 con 0x7f8a00102570 2024-09-19T08:43:10.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:10.061+0000 7f89e7fff640 1 -- 172.21.15.164:0/570942294 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f8a001998a0 con 0x7f8a00102570 2024-09-19T08:43:10.061 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:43:10.061 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":1,"btime":"2024-09-19T08:36:09:564314+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-19T08:43:10.062 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 1 2024-09-19T08:43:10.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:10.063+0000 7f8a05249640 1 -- 172.21.15.164:0/570942294 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f89e80a32c0 msgr2=0x7f89e80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:10.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:10.063+0000 7f8a05249640 1 --2- 172.21.15.164:0/570942294 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f89e80a32c0 0x7f89e80a5780 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f89f0005ab0 tx=0x7f89f0005a20 comp rx=0 tx=0).stop 2024-09-19T08:43:10.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:10.063+0000 7f8a05249640 1 -- 172.21.15.164:0/570942294 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a00102570 msgr2=0x7f8a00198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:10.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:10.063+0000 7f8a05249640 1 --2- 172.21.15.164:0/570942294 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a00102570 0x7f8a00198a80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f89f4005fd0 tx=0x7f89f400dde0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:10.063+0000 7f8a05249640 1 -- 172.21.15.164:0/570942294 shutdown_connections 2024-09-19T08:43:10.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:10.063+0000 7f8a05249640 1 --2- 172.21.15.164:0/570942294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a00102ef0 0x7f8a0019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:10.063+0000 7f8a05249640 1 --2- 172.21.15.164:0/570942294 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f89e80a32c0 0x7f89e80a5780 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:10.063+0000 7f8a05249640 1 --2- 172.21.15.164:0/570942294 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a00102570 0x7f8a00198a80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:10.063+0000 7f8a05249640 1 --2- 172.21.15.164:0/570942294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a00100e20 0x7f8a00198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:10.063+0000 7f8a05249640 1 -- 172.21.15.164:0/570942294 >> 172.21.15.164:0/570942294 conn(0x7f8a000fc7d0 msgr2=0x7f8a000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:10.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:10.063+0000 7f8a05249640 1 -- 172.21.15.164:0/570942294 shutdown_connections 2024-09-19T08:43:10.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:10.063+0000 7f8a05249640 1 -- 172.21.15.164:0/570942294 wait complete. 2024-09-19T08:43:10.281 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:10 smithi164 ceph-mon[30928]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:10.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:10 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/570942294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:43:10.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:10 smithi173 ceph-mon[35306]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:10.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:10 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/570942294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:43:10.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:10 smithi170 ceph-mon[35072]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:10.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:10 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/570942294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:43:10.660 INFO:journalctl@ceph.mds.l.smithi173.stdout:Sep 19 08:43:10 smithi173 podman[64746]: 2024-09-19 08:43:10.588752879 +0000 UTC m=+0.134562703 container create d7e3e84b85558175d090c64b097bc54d943bd8d7024cb496ba8ce8d34537b244 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-l, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-19T08:43:11.160 INFO:journalctl@ceph.mds.l.smithi173.stdout:Sep 19 08:43:10 smithi173 podman[64746]: 2024-09-19 08:43:10.824841867 +0000 UTC m=+0.370651686 container init d7e3e84b85558175d090c64b097bc54d943bd8d7024cb496ba8ce8d34537b244 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T08:43:11.161 INFO:journalctl@ceph.mds.l.smithi173.stdout:Sep 19 08:43:10 smithi173 podman[64746]: 2024-09-19 08:43:10.828434122 +0000 UTC m=+0.374243944 container start d7e3e84b85558175d090c64b097bc54d943bd8d7024cb496ba8ce8d34537b244 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-l, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-19T08:43:11.161 INFO:journalctl@ceph.mds.l.smithi173.stdout:Sep 19 08:43:10 smithi173 ceph-mds[64766]: -- 172.21.15.173:0/3509631406 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5615f4891860 con 0x5615f486e800 2024-09-19T08:43:11.161 INFO:journalctl@ceph.mds.l.smithi173.stdout:Sep 19 08:43:10 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-l[64762]: starting mds.l at 2024-09-19T08:43:11.161 INFO:journalctl@ceph.mds.l.smithi173.stdout:Sep 19 08:43:10 smithi173 bash[64746]: d7e3e84b85558175d090c64b097bc54d943bd8d7024cb496ba8ce8d34537b244 2024-09-19T08:43:11.161 INFO:journalctl@ceph.mds.l.smithi173.stdout:Sep 19 08:43:10 smithi173 systemd[1]: Started Ceph mds.l for 1e4b0d74-7662-11ef-bceb-c7b262605968. 2024-09-19T08:43:12.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:11 smithi170 ceph-mon[35072]: mds.? [v2:172.21.15.173:6832/486946900,v1:172.21.15.173:6833/486946900] up:boot 2024-09-19T08:43:12.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:11 smithi170 ceph-mon[35072]: fsmap 1 up:standby 2024-09-19T08:43:12.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:11 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T08:43:12.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:11 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:12.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:11 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:12.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:11 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:12.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:11 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:12.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:11 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:12.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:11 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:12.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:11 smithi170 ceph-mon[35072]: Deploying daemon mds.k on smithi170 2024-09-19T08:43:12.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:11 smithi170 ceph-mon[35072]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:12.158 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:11 smithi164 ceph-mon[30928]: mds.? [v2:172.21.15.173:6832/486946900,v1:172.21.15.173:6833/486946900] up:boot 2024-09-19T08:43:12.158 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:11 smithi164 ceph-mon[30928]: fsmap 1 up:standby 2024-09-19T08:43:12.158 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:11 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T08:43:12.159 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:11 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:12.159 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:11 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:12.159 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:11 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:12.159 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:11 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:12.159 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:11 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:12.160 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:11 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:12.160 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:11 smithi164 ceph-mon[30928]: Deploying daemon mds.k on smithi170 2024-09-19T08:43:12.160 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:11 smithi164 ceph-mon[30928]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:12.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:11 smithi173 ceph-mon[35306]: mds.? [v2:172.21.15.173:6832/486946900,v1:172.21.15.173:6833/486946900] up:boot 2024-09-19T08:43:12.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:11 smithi173 ceph-mon[35306]: fsmap 1 up:standby 2024-09-19T08:43:12.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:11 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T08:43:12.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:11 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:12.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:11 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:12.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:11 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:12.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:11 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:12.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:11 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:12.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:11 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:12.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:11 smithi173 ceph-mon[35306]: Deploying daemon mds.k on smithi170 2024-09-19T08:43:12.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:11 smithi173 ceph-mon[35306]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:12.876 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:43:13.125 INFO:journalctl@ceph.mds.k.smithi170.stdout:Sep 19 08:43:12 smithi170 systemd[1]: Starting Ceph mds.k for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T08:43:13.131 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:43:13.450 INFO:journalctl@ceph.mds.k.smithi170.stdout:Sep 19 08:43:13 smithi170 podman[64089]: 2024-09-19 08:43:13.065422162 +0000 UTC m=+0.015914197 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:43:13.450 INFO:journalctl@ceph.mds.k.smithi170.stdout:Sep 19 08:43:13 smithi170 podman[64089]: 2024-09-19 08:43:13.181067542 +0000 UTC m=+0.131559621 container create 23dd6748303302218cada725d87c8efa82bf66e7c1a4a5203fef6e4ea66856e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T08:43:13.450 INFO:journalctl@ceph.mds.k.smithi170.stdout:Sep 19 08:43:13 smithi170 podman[64089]: 2024-09-19 08:43:13.450790062 +0000 UTC m=+0.401282095 container init 23dd6748303302218cada725d87c8efa82bf66e7c1a4a5203fef6e4ea66856e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-k, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-19T08:43:13.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.577+0000 7f2f6820e640 1 -- 172.21.15.164:0/3423322030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f60106f10 msgr2=0x7f2f6010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:13.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.577+0000 7f2f6820e640 1 --2- 172.21.15.164:0/3423322030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f60106f10 0x7f2f6010ea90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2f58009ca0 tx=0x7f2f5802a8c0 comp rx=0 tx=0).stop 2024-09-19T08:43:13.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.577+0000 7f2f6820e640 1 -- 172.21.15.164:0/3423322030 shutdown_connections 2024-09-19T08:43:13.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.577+0000 7f2f6820e640 1 --2- 172.21.15.164:0/3423322030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f60106f10 0x7f2f6010ea90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:13.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.577+0000 7f2f6820e640 1 --2- 172.21.15.164:0/3423322030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f60102790 0x7f2f601067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:13.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.577+0000 7f2f6820e640 1 --2- 172.21.15.164:0/3423322030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f60101de0 0x7f2f601021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:13.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.577+0000 7f2f6820e640 1 -- 172.21.15.164:0/3423322030 >> 172.21.15.164:0/3423322030 conn(0x7f2f600fd8c0 msgr2=0x7f2f600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:13.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.577+0000 7f2f6820e640 1 -- 172.21.15.164:0/3423322030 shutdown_connections 2024-09-19T08:43:13.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.577+0000 7f2f6820e640 1 -- 172.21.15.164:0/3423322030 wait complete. 2024-09-19T08:43:13.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.578+0000 7f2f6820e640 1 Processor -- start 2024-09-19T08:43:13.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.578+0000 7f2f6820e640 1 -- start start 2024-09-19T08:43:13.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.578+0000 7f2f6820e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f60101de0 0x7f2f600744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:13.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.578+0000 7f2f6820e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f60102790 0x7f2f60074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:13.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.578+0000 7f2f6820e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f60106f10 0x7f2f60074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:13.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.578+0000 7f2f6820e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2f60111f70 con 0x7f2f60102790 2024-09-19T08:43:13.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.578+0000 7f2f6820e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2f60111df0 con 0x7f2f60101de0 2024-09-19T08:43:13.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.578+0000 7f2f6820e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2f601120f0 con 0x7f2f60106f10 2024-09-19T08:43:13.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.578+0000 7f2f657a9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f60102790 0x7f2f60074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:13.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.578+0000 7f2f657a9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f60102790 0x7f2f60074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49494/0 (socket says 172.21.15.164:49494) 2024-09-19T08:43:13.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.578+0000 7f2f657a9640 1 -- 172.21.15.164:0/357057705 learned_addr learned my addr 172.21.15.164:0/357057705 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:43:13.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.578+0000 7f2f65faa640 1 --2- 172.21.15.164:0/357057705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f60101de0 0x7f2f600744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:13.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.578+0000 7f2f667ab640 1 --2- 172.21.15.164:0/357057705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f60106f10 0x7f2f60074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:13.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.578+0000 7f2f657a9640 1 -- 172.21.15.164:0/357057705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f60106f10 msgr2=0x7f2f60074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:13.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.578+0000 7f2f657a9640 1 --2- 172.21.15.164:0/357057705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f60106f10 0x7f2f60074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:13.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.578+0000 7f2f657a9640 1 -- 172.21.15.164:0/357057705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f60101de0 msgr2=0x7f2f600744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:13.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.578+0000 7f2f657a9640 1 --2- 172.21.15.164:0/357057705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f60101de0 0x7f2f600744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:13.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.578+0000 7f2f657a9640 1 -- 172.21.15.164:0/357057705 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f601a6870 con 0x7f2f60102790 2024-09-19T08:43:13.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.578+0000 7f2f65faa640 1 --2- 172.21.15.164:0/357057705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f60101de0 0x7f2f600744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:43:13.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.579+0000 7f2f657a9640 1 --2- 172.21.15.164:0/357057705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f60102790 0x7f2f60074a30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2f5000e9f0 tx=0x7f2f5000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:13.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.579+0000 7f2f56ffd640 1 -- 172.21.15.164:0/357057705 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5000cd80 con 0x7f2f60102790 2024-09-19T08:43:13.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.579+0000 7f2f56ffd640 1 -- 172.21.15.164:0/357057705 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f50010910 con 0x7f2f60102790 2024-09-19T08:43:13.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.579+0000 7f2f56ffd640 1 -- 172.21.15.164:0/357057705 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f50018990 con 0x7f2f60102790 2024-09-19T08:43:13.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.579+0000 7f2f6820e640 1 -- 172.21.15.164:0/357057705 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f601a6b60 con 0x7f2f60102790 2024-09-19T08:43:13.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.579+0000 7f2f6820e640 1 -- 172.21.15.164:0/357057705 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f601a70f0 con 0x7f2f60102790 2024-09-19T08:43:13.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.580+0000 7f2f6820e640 1 -- 172.21.15.164:0/357057705 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f601021c0 con 0x7f2f60102790 2024-09-19T08:43:13.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.581+0000 7f2f56ffd640 1 -- 172.21.15.164:0/357057705 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f2f50010ac0 con 0x7f2f60102790 2024-09-19T08:43:13.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.582+0000 7f2f56ffd640 1 --2- 172.21.15.164:0/357057705 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f2f4c0a32c0 0x7f2f4c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:13.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.582+0000 7f2f65faa640 1 --2- 172.21.15.164:0/357057705 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f2f4c0a32c0 0x7f2f4c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:13.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.582+0000 7f2f56ffd640 1 -- 172.21.15.164:0/357057705 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f2f500cfab0 con 0x7f2f60102790 2024-09-19T08:43:13.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.583+0000 7f2f65faa640 1 --2- 172.21.15.164:0/357057705 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f2f4c0a32c0 0x7f2f4c0a5780 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c0058c0 tx=0x7f2f5c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:13.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.585+0000 7f2f56ffd640 1 -- 172.21.15.164:0/357057705 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2f5009c600 con 0x7f2f60102790 2024-09-19T08:43:13.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.806+0000 7f2f6820e640 1 -- 172.21.15.164:0/357057705 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f60075a20 con 0x7f2f60102790 2024-09-19T08:43:13.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.807+0000 7f2f56ffd640 1 -- 172.21.15.164:0/357057705 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2263 (secure 0 0 0) 0x7f2f60075a20 con 0x7f2f60102790 2024-09-19T08:43:13.807 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:43:13.807 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":3,"btime":"2024-09-19T08:43:13:532503+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T08:43:13.807 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 3 2024-09-19T08:43:13.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.809+0000 7f2f6820e640 1 -- 172.21.15.164:0/357057705 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f2f4c0a32c0 msgr2=0x7f2f4c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:13.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.809+0000 7f2f6820e640 1 --2- 172.21.15.164:0/357057705 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f2f4c0a32c0 0x7f2f4c0a5780 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c0058c0 tx=0x7f2f5c00a2d0 comp rx=0 tx=0).stop 2024-09-19T08:43:13.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.809+0000 7f2f6820e640 1 -- 172.21.15.164:0/357057705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f60102790 msgr2=0x7f2f60074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:13.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.809+0000 7f2f6820e640 1 --2- 172.21.15.164:0/357057705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f60102790 0x7f2f60074a30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2f5000e9f0 tx=0x7f2f5000eeb0 comp rx=0 tx=0).stop 2024-09-19T08:43:13.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.809+0000 7f2f6820e640 1 -- 172.21.15.164:0/357057705 shutdown_connections 2024-09-19T08:43:13.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.809+0000 7f2f6820e640 1 --2- 172.21.15.164:0/357057705 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f2f4c0a32c0 0x7f2f4c0a5780 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:13.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.809+0000 7f2f6820e640 1 --2- 172.21.15.164:0/357057705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f60106f10 0x7f2f60074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:13.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.809+0000 7f2f6820e640 1 --2- 172.21.15.164:0/357057705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f60102790 0x7f2f60074a30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:13.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.809+0000 7f2f6820e640 1 --2- 172.21.15.164:0/357057705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f60101de0 0x7f2f600744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:13.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.809+0000 7f2f6820e640 1 -- 172.21.15.164:0/357057705 >> 172.21.15.164:0/357057705 conn(0x7f2f600fd8c0 msgr2=0x7f2f600fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:13.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.809+0000 7f2f6820e640 1 -- 172.21.15.164:0/357057705 shutdown_connections 2024-09-19T08:43:13.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:13.809+0000 7f2f6820e640 1 -- 172.21.15.164:0/357057705 wait complete. 2024-09-19T08:43:13.875 INFO:journalctl@ceph.mds.k.smithi170.stdout:Sep 19 08:43:13 smithi170 podman[64089]: 2024-09-19 08:43:13.454596836 +0000 UTC m=+0.405088860 container start 23dd6748303302218cada725d87c8efa82bf66e7c1a4a5203fef6e4ea66856e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-k, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T08:43:13.875 INFO:journalctl@ceph.mds.k.smithi170.stdout:Sep 19 08:43:13 smithi170 ceph-mds[64108]: -- 172.21.15.170:0/4213195525 <== mon.2 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55adb13d5860 con 0x55adb13b3400 2024-09-19T08:43:13.875 INFO:journalctl@ceph.mds.k.smithi170.stdout:Sep 19 08:43:13 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-k[64104]: starting mds.k at 2024-09-19T08:43:13.875 INFO:journalctl@ceph.mds.k.smithi170.stdout:Sep 19 08:43:13 smithi170 bash[64089]: 23dd6748303302218cada725d87c8efa82bf66e7c1a4a5203fef6e4ea66856e8 2024-09-19T08:43:13.875 INFO:journalctl@ceph.mds.k.smithi170.stdout:Sep 19 08:43:13 smithi170 systemd[1]: Started Ceph mds.k for 1e4b0d74-7662-11ef-bceb-c7b262605968. 2024-09-19T08:43:14.776 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:14 smithi164 ceph-mon[30928]: mds.? [v2:172.21.15.170:6832/2726071671,v1:172.21.15.170:6833/2726071671] up:boot 2024-09-19T08:43:14.776 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:14 smithi164 ceph-mon[30928]: fsmap 2 up:standby 2024-09-19T08:43:14.777 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:14 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T08:43:14.777 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:14 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:14.777 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:14 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:14.777 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:14 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:14.777 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:14 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:14.777 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:14 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:14.777 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:14 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:14.777 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:14 smithi164 ceph-mon[30928]: Deploying daemon mds.j on smithi164 2024-09-19T08:43:14.777 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:14 smithi164 ceph-mon[30928]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:14.777 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:14 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/357057705' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:43:14.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:14 smithi170 ceph-mon[35072]: mds.? [v2:172.21.15.170:6832/2726071671,v1:172.21.15.170:6833/2726071671] up:boot 2024-09-19T08:43:14.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:14 smithi170 ceph-mon[35072]: fsmap 2 up:standby 2024-09-19T08:43:14.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:14 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T08:43:14.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:14 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:14.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:14 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:14.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:14 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:14.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:14 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:14.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:14 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:14.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:14 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:14.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:14 smithi170 ceph-mon[35072]: Deploying daemon mds.j on smithi164 2024-09-19T08:43:14.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:14 smithi170 ceph-mon[35072]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:14.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:14 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/357057705' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:43:14.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:14 smithi173 ceph-mon[35306]: mds.? [v2:172.21.15.170:6832/2726071671,v1:172.21.15.170:6833/2726071671] up:boot 2024-09-19T08:43:14.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:14 smithi173 ceph-mon[35306]: fsmap 2 up:standby 2024-09-19T08:43:14.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:14 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T08:43:14.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:14 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:14.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:14 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:14.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:14 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:14.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:14 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:14.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:14 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:14.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:14 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:14.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:14 smithi173 ceph-mon[35306]: Deploying daemon mds.j on smithi164 2024-09-19T08:43:14.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:14 smithi173 ceph-mon[35306]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:14.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:14 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/357057705' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:43:16.532 INFO:journalctl@ceph.mds.j.smithi164.stdout:Sep 19 08:43:16 smithi164 systemd[1]: Starting Ceph mds.j for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T08:43:16.557 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:43:16.840 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:16 smithi164 ceph-mon[30928]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:16.840 INFO:journalctl@ceph.mds.j.smithi164.stdout:Sep 19 08:43:16 smithi164 podman[69968]: 2024-09-19 08:43:16.510206483 +0000 UTC m=+0.016124133 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:43:16.840 INFO:journalctl@ceph.mds.j.smithi164.stdout:Sep 19 08:43:16 smithi164 podman[69968]: 2024-09-19 08:43:16.61980395 +0000 UTC m=+0.125721580 container create d5aaa7f97ba043430262cc7e49e68603850fcf0da05b7fdf2240ab198d5a8e72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-j, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-19T08:43:16.840 INFO:journalctl@ceph.mds.j.smithi164.stdout:Sep 19 08:43:16 smithi164 podman[69968]: 2024-09-19 08:43:16.814126645 +0000 UTC m=+0.320044294 container init d5aaa7f97ba043430262cc7e49e68603850fcf0da05b7fdf2240ab198d5a8e72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-19T08:43:16.840 INFO:journalctl@ceph.mds.j.smithi164.stdout:Sep 19 08:43:16 smithi164 podman[69968]: 2024-09-19 08:43:16.817417311 +0000 UTC m=+0.323334942 container start d5aaa7f97ba043430262cc7e49e68603850fcf0da05b7fdf2240ab198d5a8e72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-j, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, ceph=True) 2024-09-19T08:43:16.840 INFO:journalctl@ceph.mds.j.smithi164.stdout:Sep 19 08:43:16 smithi164 ceph-mds[70017]: -- 172.21.15.164:0/2840709943 <== mon.0 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564b98b45860 con 0x564b98b23400 2024-09-19T08:43:16.993 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:43:17.116 INFO:journalctl@ceph.mds.j.smithi164.stdout:Sep 19 08:43:16 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-j[70001]: starting mds.j at 2024-09-19T08:43:17.116 INFO:journalctl@ceph.mds.j.smithi164.stdout:Sep 19 08:43:16 smithi164 bash[69968]: d5aaa7f97ba043430262cc7e49e68603850fcf0da05b7fdf2240ab198d5a8e72 2024-09-19T08:43:17.116 INFO:journalctl@ceph.mds.j.smithi164.stdout:Sep 19 08:43:16 smithi164 systemd[1]: Started Ceph mds.j for 1e4b0d74-7662-11ef-bceb-c7b262605968. 2024-09-19T08:43:17.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:16 smithi170 ceph-mon[35072]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:17.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:16 smithi173 ceph-mon[35306]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:17.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.516+0000 7f89bb673640 1 -- 172.21.15.164:0/2089753507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89b4102ef0 msgr2=0x7f89b410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:17.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.516+0000 7f89bb673640 1 --2- 172.21.15.164:0/2089753507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89b4102ef0 0x7f89b410da40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f89ac009900 tx=0x7f89ac02a480 comp rx=0 tx=0).stop 2024-09-19T08:43:17.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.516+0000 7f89bb673640 1 -- 172.21.15.164:0/2089753507 shutdown_connections 2024-09-19T08:43:17.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.516+0000 7f89bb673640 1 --2- 172.21.15.164:0/2089753507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89b4102ef0 0x7f89b410da40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:17.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.516+0000 7f89bb673640 1 --2- 172.21.15.164:0/2089753507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89b4102570 0x7f89b41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:17.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.516+0000 7f89bb673640 1 --2- 172.21.15.164:0/2089753507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89b4100e20 0x7f89b4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:17.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.516+0000 7f89bb673640 1 -- 172.21.15.164:0/2089753507 >> 172.21.15.164:0/2089753507 conn(0x7f89b40fc7d0 msgr2=0x7f89b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:17.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.516+0000 7f89bb673640 1 -- 172.21.15.164:0/2089753507 shutdown_connections 2024-09-19T08:43:17.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.516+0000 7f89bb673640 1 -- 172.21.15.164:0/2089753507 wait complete. 2024-09-19T08:43:17.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.517+0000 7f89bb673640 1 Processor -- start 2024-09-19T08:43:17.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.517+0000 7f89bb673640 1 -- start start 2024-09-19T08:43:17.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.517+0000 7f89bb673640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89b4100e20 0x7f89b4198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:17.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.517+0000 7f89bb673640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89b4102570 0x7f89b4198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:17.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.517+0000 7f89bb673640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89b4102ef0 0x7f89b419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:17.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.517+0000 7f89bb673640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f89b41120a0 con 0x7f89b4102ef0 2024-09-19T08:43:17.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.517+0000 7f89bb673640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f89b4111f20 con 0x7f89b4102570 2024-09-19T08:43:17.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.517+0000 7f89bb673640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f89b4112220 con 0x7f89b4100e20 2024-09-19T08:43:17.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.517+0000 7f89b9c10640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89b4102ef0 0x7f89b419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:17.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.517+0000 7f89b9c10640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89b4102ef0 0x7f89b419ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49532/0 (socket says 172.21.15.164:49532) 2024-09-19T08:43:17.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.517+0000 7f89b9c10640 1 -- 172.21.15.164:0/1007405765 learned_addr learned my addr 172.21.15.164:0/1007405765 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:43:17.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.517+0000 7f89b940f640 1 --2- 172.21.15.164:0/1007405765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89b4100e20 0x7f89b4198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:17.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.517+0000 7f89b8c0e640 1 --2- 172.21.15.164:0/1007405765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89b4102570 0x7f89b4198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:17.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.517+0000 7f89b9c10640 1 -- 172.21.15.164:0/1007405765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89b4100e20 msgr2=0x7f89b4198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:17.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.517+0000 7f89b9c10640 1 --2- 172.21.15.164:0/1007405765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89b4100e20 0x7f89b4198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:17.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.517+0000 7f89b9c10640 1 -- 172.21.15.164:0/1007405765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89b4102570 msgr2=0x7f89b4198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:17.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.517+0000 7f89b9c10640 1 --2- 172.21.15.164:0/1007405765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89b4102570 0x7f89b4198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:17.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.517+0000 7f89b9c10640 1 -- 172.21.15.164:0/1007405765 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b419d3d0 con 0x7f89b4102ef0 2024-09-19T08:43:17.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.517+0000 7f89b8c0e640 1 --2- 172.21.15.164:0/1007405765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89b4102570 0x7f89b4198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:17.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.517+0000 7f89b9c10640 1 --2- 172.21.15.164:0/1007405765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89b4102ef0 0x7f89b419ccf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f89ac004770 tx=0x7f89ac003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:17.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.518+0000 7f89b27fc640 1 -- 172.21.15.164:0/1007405765 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac03f070 con 0x7f89b4102ef0 2024-09-19T08:43:17.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.518+0000 7f89b27fc640 1 -- 172.21.15.164:0/1007405765 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89ac031c30 con 0x7f89b4102ef0 2024-09-19T08:43:17.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.518+0000 7f89b27fc640 1 -- 172.21.15.164:0/1007405765 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac035030 con 0x7f89b4102ef0 2024-09-19T08:43:17.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.518+0000 7f89bb673640 1 -- 172.21.15.164:0/1007405765 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89b419d660 con 0x7f89b4102ef0 2024-09-19T08:43:17.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.518+0000 7f89bb673640 1 -- 172.21.15.164:0/1007405765 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89b41a4fa0 con 0x7f89b4102ef0 2024-09-19T08:43:17.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.519+0000 7f89bb673640 1 -- 172.21.15.164:0/1007405765 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89b41021b0 con 0x7f89b4102ef0 2024-09-19T08:43:17.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.520+0000 7f89b27fc640 1 -- 172.21.15.164:0/1007405765 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f89ac042070 con 0x7f89b4102ef0 2024-09-19T08:43:17.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.520+0000 7f89b27fc640 1 --2- 172.21.15.164:0/1007405765 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f899c0a3240 0x7f899c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:17.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.521+0000 7f89b940f640 1 --2- 172.21.15.164:0/1007405765 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f899c0a3240 0x7f899c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:17.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.521+0000 7f89b27fc640 1 -- 172.21.15.164:0/1007405765 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f89ac0efd80 con 0x7f89b4102ef0 2024-09-19T08:43:17.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.521+0000 7f89b940f640 1 --2- 172.21.15.164:0/1007405765 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f899c0a3240 0x7f899c0a5700 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f89a4005fd0 tx=0x7f89a4009870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:17.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.525+0000 7f89b27fc640 1 -- 172.21.15.164:0/1007405765 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f89ac0bcf40 con 0x7f89b4102ef0 2024-09-19T08:43:17.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.738+0000 7f89bb673640 1 -- 172.21.15.164:0/1007405765 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89b41a5170 con 0x7f89b4102ef0 2024-09-19T08:43:17.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.739+0000 7f89b27fc640 1 -- 172.21.15.164:0/1007405765 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2263 (secure 0 0 0) 0x7f89ac0bc840 con 0x7f89b4102ef0 2024-09-19T08:43:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:43:17.739 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":3,"btime":"2024-09-19T08:43:13:532503+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T08:43:17.739 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 3 2024-09-19T08:43:17.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.741+0000 7f89bb673640 1 -- 172.21.15.164:0/1007405765 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f899c0a3240 msgr2=0x7f899c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:17.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.741+0000 7f89bb673640 1 --2- 172.21.15.164:0/1007405765 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f899c0a3240 0x7f899c0a5700 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f89a4005fd0 tx=0x7f89a4009870 comp rx=0 tx=0).stop 2024-09-19T08:43:17.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.741+0000 7f89bb673640 1 -- 172.21.15.164:0/1007405765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89b4102ef0 msgr2=0x7f89b419ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:17.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.741+0000 7f89bb673640 1 --2- 172.21.15.164:0/1007405765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89b4102ef0 0x7f89b419ccf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f89ac004770 tx=0x7f89ac003c10 comp rx=0 tx=0).stop 2024-09-19T08:43:17.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.741+0000 7f89bb673640 1 -- 172.21.15.164:0/1007405765 shutdown_connections 2024-09-19T08:43:17.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.741+0000 7f89bb673640 1 --2- 172.21.15.164:0/1007405765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89b4102ef0 0x7f89b419ccf0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:17.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.741+0000 7f89bb673640 1 --2- 172.21.15.164:0/1007405765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89b4102570 0x7f89b4198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:17.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.741+0000 7f89bb673640 1 --2- 172.21.15.164:0/1007405765 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f899c0a3240 0x7f899c0a5700 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:17.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.741+0000 7f89bb673640 1 --2- 172.21.15.164:0/1007405765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89b4100e20 0x7f89b4198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:17.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.741+0000 7f89bb673640 1 -- 172.21.15.164:0/1007405765 >> 172.21.15.164:0/1007405765 conn(0x7f89b40fc7d0 msgr2=0x7f89b41067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:17.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.741+0000 7f89bb673640 1 -- 172.21.15.164:0/1007405765 shutdown_connections 2024-09-19T08:43:17.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:17.741+0000 7f89bb673640 1 -- 172.21.15.164:0/1007405765 wait complete. 2024-09-19T08:43:18.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:17 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:18.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:17 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:18.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:17 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:18.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:17 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:18.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:17 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:18.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:17 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:18.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:17 smithi164 ceph-mon[30928]: Deploying daemon mds.d on smithi164 2024-09-19T08:43:18.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:17 smithi164 ceph-mon[30928]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:18.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:17 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/1007405765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:43:18.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:17 smithi164 ceph-mon[30928]: mds.? [v2:172.21.15.164:6834/3695235648,v1:172.21.15.164:6835/3695235648] up:boot 2024-09-19T08:43:18.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:17 smithi164 ceph-mon[30928]: fsmap 3 up:standby 2024-09-19T08:43:18.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:17 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T08:43:18.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:17 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:18.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:17 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:18.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:17 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:18.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:17 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:18.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:17 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:18.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:17 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:18.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:17 smithi170 ceph-mon[35072]: Deploying daemon mds.d on smithi164 2024-09-19T08:43:18.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:17 smithi170 ceph-mon[35072]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:18.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:17 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/1007405765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:43:18.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:17 smithi170 ceph-mon[35072]: mds.? [v2:172.21.15.164:6834/3695235648,v1:172.21.15.164:6835/3695235648] up:boot 2024-09-19T08:43:18.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:17 smithi170 ceph-mon[35072]: fsmap 3 up:standby 2024-09-19T08:43:18.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:17 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T08:43:18.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:17 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:18.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:17 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:18.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:17 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:18.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:17 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:18.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:17 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:18.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:17 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:18.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:17 smithi173 ceph-mon[35306]: Deploying daemon mds.d on smithi164 2024-09-19T08:43:18.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:17 smithi173 ceph-mon[35306]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:18.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:17 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/1007405765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:43:18.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:17 smithi173 ceph-mon[35306]: mds.? [v2:172.21.15.164:6834/3695235648,v1:172.21.15.164:6835/3695235648] up:boot 2024-09-19T08:43:18.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:17 smithi173 ceph-mon[35306]: fsmap 3 up:standby 2024-09-19T08:43:18.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:17 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T08:43:20.231 INFO:journalctl@ceph.mds.d.smithi164.stdout:Sep 19 08:43:20 smithi164 systemd[1]: Starting Ceph mds.d for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T08:43:20.503 INFO:journalctl@ceph.mds.d.smithi164.stdout:Sep 19 08:43:20 smithi164 podman[70383]: 2024-09-19 08:43:20.134578109 +0000 UTC m=+0.016002826 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:43:20.503 INFO:journalctl@ceph.mds.d.smithi164.stdout:Sep 19 08:43:20 smithi164 podman[70383]: 2024-09-19 08:43:20.23314462 +0000 UTC m=+0.114569323 container create fcf4f129a2b0e002b4b5d2f7937927378f062eab7770de20e71273f1944e86dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-19T08:43:20.503 INFO:journalctl@ceph.mds.d.smithi164.stdout:Sep 19 08:43:20 smithi164 podman[70383]: 2024-09-19 08:43:20.477428824 +0000 UTC m=+0.358853540 container init fcf4f129a2b0e002b4b5d2f7937927378f062eab7770de20e71273f1944e86dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-d, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T08:43:20.503 INFO:journalctl@ceph.mds.d.smithi164.stdout:Sep 19 08:43:20 smithi164 podman[70383]: 2024-09-19 08:43:20.480163585 +0000 UTC m=+0.361588289 container start fcf4f129a2b0e002b4b5d2f7937927378f062eab7770de20e71273f1944e86dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-19T08:43:20.782 INFO:journalctl@ceph.mds.d.smithi164.stdout:Sep 19 08:43:20 smithi164 ceph-mds[70403]: -- 172.21.15.164:0/109836346 <== mon.0 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561694b5d860 con 0x561694b3a800 2024-09-19T08:43:20.782 INFO:journalctl@ceph.mds.d.smithi164.stdout:Sep 19 08:43:20 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-d[70399]: starting mds.d at 2024-09-19T08:43:20.782 INFO:journalctl@ceph.mds.d.smithi164.stdout:Sep 19 08:43:20 smithi164 bash[70383]: fcf4f129a2b0e002b4b5d2f7937927378f062eab7770de20e71273f1944e86dc 2024-09-19T08:43:20.782 INFO:journalctl@ceph.mds.d.smithi164.stdout:Sep 19 08:43:20 smithi164 systemd[1]: Started Ceph mds.d for 1e4b0d74-7662-11ef-bceb-c7b262605968. 2024-09-19T08:43:21.020 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:43:21.044 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:20 smithi164 ceph-mon[30928]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:21.045 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:20 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:21.045 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:20 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:21.045 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:20 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:21.045 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:20 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:21.045 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:20 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:21.045 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:20 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:21.052 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:20 smithi170 ceph-mon[35072]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:21.053 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:20 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:21.053 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:20 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:21.053 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:20 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:21.053 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:20 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:21.053 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:20 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:21.053 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:20 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:21.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:20 smithi173 ceph-mon[35306]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:21.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:20 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:21.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:20 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:21.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:20 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:21.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:20 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:21.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:20 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:21.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:20 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:21.268 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:43:21.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.744+0000 7f5454c2c640 1 -- 172.21.15.164:0/3469486968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5450106a80 msgr2=0x7f5450106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:21.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.744+0000 7f5454c2c640 1 --2- 172.21.15.164:0/3469486968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5450106a80 0x7f5450106e60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5440009930 tx=0x7f544002a420 comp rx=0 tx=0).stop 2024-09-19T08:43:21.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.744+0000 7f5454c2c640 1 -- 172.21.15.164:0/3469486968 shutdown_connections 2024-09-19T08:43:21.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.744+0000 7f5454c2c640 1 --2- 172.21.15.164:0/3469486968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5450101420 0x7f545010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:21.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.744+0000 7f5454c2c640 1 --2- 172.21.15.164:0/3469486968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5450100a80 0x7f5450100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:21.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.744+0000 7f5454c2c640 1 --2- 172.21.15.164:0/3469486968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5450106a80 0x7f5450106e60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:21.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.744+0000 7f5454c2c640 1 -- 172.21.15.164:0/3469486968 >> 172.21.15.164:0/3469486968 conn(0x7f54500fc7a0 msgr2=0x7f54500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:21.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.744+0000 7f5454c2c640 1 -- 172.21.15.164:0/3469486968 shutdown_connections 2024-09-19T08:43:21.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.744+0000 7f5454c2c640 1 -- 172.21.15.164:0/3469486968 wait complete. 2024-09-19T08:43:21.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.745+0000 7f5454c2c640 1 Processor -- start 2024-09-19T08:43:21.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.745+0000 7f5454c2c640 1 -- start start 2024-09-19T08:43:21.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.745+0000 7f5454c2c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5450100a80 0x7f54501974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:21.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.745+0000 7f5454c2c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5450101420 0x7f5450197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:21.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.746+0000 7f5454c2c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5450106a80 0x7f545019bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:21.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.746+0000 7f5454c2c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5450110f20 con 0x7f5450106a80 2024-09-19T08:43:21.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.746+0000 7f5454c2c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5450110da0 con 0x7f5450100a80 2024-09-19T08:43:21.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.746+0000 7f5454c2c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f54501110a0 con 0x7f5450101420 2024-09-19T08:43:21.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.746+0000 7f544ed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5450106a80 0x7f545019bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:21.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.746+0000 7f544ed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5450106a80 0x7f545019bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49556/0 (socket says 172.21.15.164:49556) 2024-09-19T08:43:21.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.746+0000 7f544ed9d640 1 -- 172.21.15.164:0/2137713448 learned_addr learned my addr 172.21.15.164:0/2137713448 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:43:21.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.746+0000 7f544e59c640 1 --2- 172.21.15.164:0/2137713448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5450100a80 0x7f54501974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:21.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.746+0000 7f5447fff640 1 --2- 172.21.15.164:0/2137713448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5450101420 0x7f5450197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:21.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.746+0000 7f544ed9d640 1 -- 172.21.15.164:0/2137713448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5450101420 msgr2=0x7f5450197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:21.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.746+0000 7f544ed9d640 1 --2- 172.21.15.164:0/2137713448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5450101420 0x7f5450197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:21.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.746+0000 7f544ed9d640 1 -- 172.21.15.164:0/2137713448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5450100a80 msgr2=0x7f54501974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:21.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.746+0000 7f544ed9d640 1 --2- 172.21.15.164:0/2137713448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5450100a80 0x7f54501974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:21.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.746+0000 7f544ed9d640 1 -- 172.21.15.164:0/2137713448 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545019c490 con 0x7f5450106a80 2024-09-19T08:43:21.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.746+0000 7f544ed9d640 1 --2- 172.21.15.164:0/2137713448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5450106a80 0x7f545019bdb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f544800b970 tx=0x7f544800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:21.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.746+0000 7f54477fe640 1 -- 172.21.15.164:0/2137713448 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448019070 con 0x7f5450106a80 2024-09-19T08:43:21.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.746+0000 7f54477fe640 1 -- 172.21.15.164:0/2137713448 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f544800cb40 con 0x7f5450106a80 2024-09-19T08:43:21.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.746+0000 7f5454c2c640 1 -- 172.21.15.164:0/2137713448 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f545019c780 con 0x7f5450106a80 2024-09-19T08:43:21.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.746+0000 7f54477fe640 1 -- 172.21.15.164:0/2137713448 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448007650 con 0x7f5450106a80 2024-09-19T08:43:21.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.746+0000 7f5454c2c640 1 -- 172.21.15.164:0/2137713448 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54501a4060 con 0x7f5450106a80 2024-09-19T08:43:21.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.747+0000 7f5454c2c640 1 -- 172.21.15.164:0/2137713448 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5450100f60 con 0x7f5450106a80 2024-09-19T08:43:21.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.748+0000 7f54477fe640 1 -- 172.21.15.164:0/2137713448 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f544801c070 con 0x7f5450106a80 2024-09-19T08:43:21.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.749+0000 7f54477fe640 1 --2- 172.21.15.164:0/2137713448 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f543c0a3240 0x7f543c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:21.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.749+0000 7f544e59c640 1 --2- 172.21.15.164:0/2137713448 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f543c0a3240 0x7f543c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:21.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.749+0000 7f54477fe640 1 -- 172.21.15.164:0/2137713448 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f54480cea60 con 0x7f5450106a80 2024-09-19T08:43:21.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.750+0000 7f544e59c640 1 --2- 172.21.15.164:0/2137713448 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f543c0a3240 0x7f543c0a5700 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5440009900 tx=0x7f54400058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:21.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.753+0000 7f54477fe640 1 -- 172.21.15.164:0/2137713448 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f544809b5d0 con 0x7f5450106a80 2024-09-19T08:43:21.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.956+0000 7f5454c2c640 1 -- 172.21.15.164:0/2137713448 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f545002d7c0 con 0x7f5450106a80 2024-09-19T08:43:21.957 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:21 smithi164 ceph-mon[30928]: Deploying daemon mds.e on smithi170 2024-09-19T08:43:21.957 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:21 smithi164 ceph-mon[30928]: mds.? [v2:172.21.15.164:6836/199930574,v1:172.21.15.164:6837/199930574] up:boot 2024-09-19T08:43:21.957 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:21 smithi164 ceph-mon[30928]: fsmap 4 up:standby 2024-09-19T08:43:21.957 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:21 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T08:43:21.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.957+0000 7f54477fe640 1 -- 172.21.15.164:0/2137713448 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3916 (secure 0 0 0) 0x7f54480cce20 con 0x7f5450106a80 2024-09-19T08:43:21.957 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:43:21.958 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":5,"btime":"2024-09-19T08:43:20:787291+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T08:43:21.958 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 5 2024-09-19T08:43:21.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.959+0000 7f5454c2c640 1 -- 172.21.15.164:0/2137713448 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f543c0a3240 msgr2=0x7f543c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:21.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.959+0000 7f5454c2c640 1 --2- 172.21.15.164:0/2137713448 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f543c0a3240 0x7f543c0a5700 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5440009900 tx=0x7f54400058d0 comp rx=0 tx=0).stop 2024-09-19T08:43:21.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.959+0000 7f5454c2c640 1 -- 172.21.15.164:0/2137713448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5450106a80 msgr2=0x7f545019bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:21.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.959+0000 7f5454c2c640 1 --2- 172.21.15.164:0/2137713448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5450106a80 0x7f545019bdb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f544800b970 tx=0x7f544800be30 comp rx=0 tx=0).stop 2024-09-19T08:43:21.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.959+0000 7f5454c2c640 1 -- 172.21.15.164:0/2137713448 shutdown_connections 2024-09-19T08:43:21.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.959+0000 7f5454c2c640 1 --2- 172.21.15.164:0/2137713448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5450106a80 0x7f545019bdb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:21.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.959+0000 7f5454c2c640 1 --2- 172.21.15.164:0/2137713448 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f543c0a3240 0x7f543c0a5700 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:21.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.959+0000 7f5454c2c640 1 --2- 172.21.15.164:0/2137713448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5450101420 0x7f5450197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:21.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.959+0000 7f5454c2c640 1 --2- 172.21.15.164:0/2137713448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5450100a80 0x7f54501974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:21.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.959+0000 7f5454c2c640 1 -- 172.21.15.164:0/2137713448 >> 172.21.15.164:0/2137713448 conn(0x7f54500fc7a0 msgr2=0x7f54500fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:21.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.959+0000 7f5454c2c640 1 -- 172.21.15.164:0/2137713448 shutdown_connections 2024-09-19T08:43:21.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:21.959+0000 7f5454c2c640 1 -- 172.21.15.164:0/2137713448 wait complete. 2024-09-19T08:43:22.079 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:21 smithi170 ceph-mon[35072]: Deploying daemon mds.e on smithi170 2024-09-19T08:43:22.079 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:21 smithi170 ceph-mon[35072]: mds.? [v2:172.21.15.164:6836/199930574,v1:172.21.15.164:6837/199930574] up:boot 2024-09-19T08:43:22.079 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:21 smithi170 ceph-mon[35072]: fsmap 4 up:standby 2024-09-19T08:43:22.079 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:21 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T08:43:22.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:21 smithi173 ceph-mon[35306]: Deploying daemon mds.e on smithi170 2024-09-19T08:43:22.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:21 smithi173 ceph-mon[35306]: mds.? [v2:172.21.15.164:6836/199930574,v1:172.21.15.164:6837/199930574] up:boot 2024-09-19T08:43:22.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:21 smithi173 ceph-mon[35306]: fsmap 4 up:standby 2024-09-19T08:43:22.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:21 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T08:43:22.798 INFO:journalctl@ceph.mds.e.smithi170.stdout:Sep 19 08:43:22 smithi170 systemd[1]: Starting Ceph mds.e for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T08:43:23.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:22 smithi164 ceph-mon[30928]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:23.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:22 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2137713448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:43:23.121 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:22 smithi173 ceph-mon[35306]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:23.122 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:22 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2137713448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:43:23.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:22 smithi170 ceph-mon[35072]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:23.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:22 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2137713448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:43:23.125 INFO:journalctl@ceph.mds.e.smithi170.stdout:Sep 19 08:43:22 smithi170 podman[64401]: 2024-09-19 08:43:22.754522379 +0000 UTC m=+0.015841550 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:43:23.125 INFO:journalctl@ceph.mds.e.smithi170.stdout:Sep 19 08:43:22 smithi170 podman[64401]: 2024-09-19 08:43:22.871183981 +0000 UTC m=+0.132503152 container create 68372a737eac2538bbf161b2e5fdb8de896082ec6b1cbf7512c545fb4ac6559a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-e, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T08:43:23.625 INFO:journalctl@ceph.mds.e.smithi170.stdout:Sep 19 08:43:23 smithi170 podman[64401]: 2024-09-19 08:43:23.137899979 +0000 UTC m=+0.399219150 container init 68372a737eac2538bbf161b2e5fdb8de896082ec6b1cbf7512c545fb4ac6559a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-e, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4) 2024-09-19T08:43:23.625 INFO:journalctl@ceph.mds.e.smithi170.stdout:Sep 19 08:43:23 smithi170 podman[64401]: 2024-09-19 08:43:23.140787131 +0000 UTC m=+0.402106305 container start 68372a737eac2538bbf161b2e5fdb8de896082ec6b1cbf7512c545fb4ac6559a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-e, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-19T08:43:23.625 INFO:journalctl@ceph.mds.e.smithi170.stdout:Sep 19 08:43:23 smithi170 ceph-mds[64420]: -- 172.21.15.170:0/492501168 <== mon.2 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559f29fd1860 con 0x559f29fae800 2024-09-19T08:43:23.625 INFO:journalctl@ceph.mds.e.smithi170.stdout:Sep 19 08:43:23 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-e[64416]: starting mds.e at 2024-09-19T08:43:23.625 INFO:journalctl@ceph.mds.e.smithi170.stdout:Sep 19 08:43:23 smithi170 bash[64401]: 68372a737eac2538bbf161b2e5fdb8de896082ec6b1cbf7512c545fb4ac6559a 2024-09-19T08:43:23.625 INFO:journalctl@ceph.mds.e.smithi170.stdout:Sep 19 08:43:23 smithi170 systemd[1]: Started Ceph mds.e for 1e4b0d74-7662-11ef-bceb-c7b262605968. 2024-09-19T08:43:24.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:24 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:24.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:24 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:24.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:24 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:24.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:24 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:24.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:24 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:24.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:24 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:24.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:24 smithi173 ceph-mon[35306]: Deploying daemon mds.f on smithi173 2024-09-19T08:43:24.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:24 smithi173 ceph-mon[35306]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:24.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:24 smithi173 ceph-mon[35306]: mds.? [v2:172.21.15.170:6834/321254273,v1:172.21.15.170:6835/321254273] up:boot 2024-09-19T08:43:24.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:24 smithi173 ceph-mon[35306]: fsmap 5 up:standby 2024-09-19T08:43:24.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:24 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T08:43:24.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:24 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:24.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:24 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:24.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:24 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:24.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:24 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:24.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:24 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:24.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:24 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:24.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:24 smithi170 ceph-mon[35072]: Deploying daemon mds.f on smithi173 2024-09-19T08:43:24.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:24 smithi170 ceph-mon[35072]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:24.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:24 smithi170 ceph-mon[35072]: mds.? [v2:172.21.15.170:6834/321254273,v1:172.21.15.170:6835/321254273] up:boot 2024-09-19T08:43:24.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:24 smithi170 ceph-mon[35072]: fsmap 5 up:standby 2024-09-19T08:43:24.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:24 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T08:43:24.724 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:43:24.752 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:24 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:24.753 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:24 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:24.753 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:24 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:24.753 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:24 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:24.753 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:24 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:24.753 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:24 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:24.753 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:24 smithi164 ceph-mon[30928]: Deploying daemon mds.f on smithi173 2024-09-19T08:43:24.753 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:24 smithi164 ceph-mon[30928]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:24.753 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:24 smithi164 ceph-mon[30928]: mds.? [v2:172.21.15.170:6834/321254273,v1:172.21.15.170:6835/321254273] up:boot 2024-09-19T08:43:24.753 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:24 smithi164 ceph-mon[30928]: fsmap 5 up:standby 2024-09-19T08:43:24.754 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:24 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T08:43:24.982 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:43:25.411 INFO:journalctl@ceph.mds.f.smithi173.stdout:Sep 19 08:43:25 smithi173 systemd[1]: Starting Ceph mds.f for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T08:43:25.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.488+0000 7f0cf075f640 1 -- 172.21.15.164:0/649172939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ce81037a0 msgr2=0x7f0ce810bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:25.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.488+0000 7f0cf075f640 1 --2- 172.21.15.164:0/649172939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ce81037a0 0x7f0ce810bc00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8009900 tx=0x7f0cd802a410 comp rx=0 tx=0).stop 2024-09-19T08:43:25.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.488+0000 7f0cf075f640 1 -- 172.21.15.164:0/649172939 shutdown_connections 2024-09-19T08:43:25.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.488+0000 7f0cf075f640 1 --2- 172.21.15.164:0/649172939 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ce810c2f0 0x7f0ce810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:25.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.488+0000 7f0cf075f640 1 --2- 172.21.15.164:0/649172939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ce81037a0 0x7f0ce810bc00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:25.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.488+0000 7f0cf075f640 1 --2- 172.21.15.164:0/649172939 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ce8102df0 0x7f0ce81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:25.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.488+0000 7f0cf075f640 1 -- 172.21.15.164:0/649172939 >> 172.21.15.164:0/649172939 conn(0x7f0ce80fc7d0 msgr2=0x7f0ce80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:25.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.489+0000 7f0cf075f640 1 -- 172.21.15.164:0/649172939 shutdown_connections 2024-09-19T08:43:25.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.489+0000 7f0cf075f640 1 -- 172.21.15.164:0/649172939 wait complete. 2024-09-19T08:43:25.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.489+0000 7f0cf075f640 1 Processor -- start 2024-09-19T08:43:25.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.489+0000 7f0cf075f640 1 -- start start 2024-09-19T08:43:25.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.489+0000 7f0cf075f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ce8102df0 0x7f0ce8198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:25.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.489+0000 7f0cf075f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ce81037a0 0x7f0ce8198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:25.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.489+0000 7f0cf075f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ce810c2f0 0x7f0ce819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:25.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.489+0000 7f0cf075f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce8112130 con 0x7f0ce81037a0 2024-09-19T08:43:25.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.489+0000 7f0cf075f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce8111fb0 con 0x7f0ce810c2f0 2024-09-19T08:43:25.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.489+0000 7f0cf075f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce81122b0 con 0x7f0ce8102df0 2024-09-19T08:43:25.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.489+0000 7f0cedcfa640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ce81037a0 0x7f0ce8198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:25.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.489+0000 7f0cedcfa640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ce81037a0 0x7f0ce8198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51900/0 (socket says 172.21.15.164:51900) 2024-09-19T08:43:25.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.489+0000 7f0cedcfa640 1 -- 172.21.15.164:0/2399094053 learned_addr learned my addr 172.21.15.164:0/2399094053 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:43:25.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.490+0000 7f0ceecfc640 1 --2- 172.21.15.164:0/2399094053 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ce810c2f0 0x7f0ce819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:25.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.490+0000 7f0cee4fb640 1 --2- 172.21.15.164:0/2399094053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ce8102df0 0x7f0ce8198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:25.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.490+0000 7f0cedcfa640 1 -- 172.21.15.164:0/2399094053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ce8102df0 msgr2=0x7f0ce8198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:25.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.490+0000 7f0cedcfa640 1 --2- 172.21.15.164:0/2399094053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ce8102df0 0x7f0ce8198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:25.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.490+0000 7f0cedcfa640 1 -- 172.21.15.164:0/2399094053 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ce810c2f0 msgr2=0x7f0ce819cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:25.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.490+0000 7f0cedcfa640 1 --2- 172.21.15.164:0/2399094053 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ce810c2f0 0x7f0ce819cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:25.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.490+0000 7f0cedcfa640 1 -- 172.21.15.164:0/2399094053 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce819d5f0 con 0x7f0ce81037a0 2024-09-19T08:43:25.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.490+0000 7f0cee4fb640 1 --2- 172.21.15.164:0/2399094053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ce8102df0 0x7f0ce8198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:25.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.490+0000 7f0cedcfa640 1 --2- 172.21.15.164:0/2399094053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ce81037a0 0x7f0ce8198b80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8005940 tx=0x7f0cd80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:25.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.490+0000 7f0cdf7fe640 1 -- 172.21.15.164:0/2399094053 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd803f030 con 0x7f0ce81037a0 2024-09-19T08:43:25.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.490+0000 7f0cdf7fe640 1 -- 172.21.15.164:0/2399094053 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cd8003ce0 con 0x7f0ce81037a0 2024-09-19T08:43:25.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.490+0000 7f0cf075f640 1 -- 172.21.15.164:0/2399094053 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ce819d880 con 0x7f0ce81037a0 2024-09-19T08:43:25.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.490+0000 7f0cdf7fe640 1 -- 172.21.15.164:0/2399094053 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8035070 con 0x7f0ce81037a0 2024-09-19T08:43:25.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.490+0000 7f0cf075f640 1 -- 172.21.15.164:0/2399094053 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ce81a51c0 con 0x7f0ce81037a0 2024-09-19T08:43:25.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.491+0000 7f0cf075f640 1 -- 172.21.15.164:0/2399094053 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ce81031d0 con 0x7f0ce81037a0 2024-09-19T08:43:25.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.492+0000 7f0cdf7fe640 1 -- 172.21.15.164:0/2399094053 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f0cd8042030 con 0x7f0ce81037a0 2024-09-19T08:43:25.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.493+0000 7f0cdf7fe640 1 --2- 172.21.15.164:0/2399094053 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f0cd40a3240 0x7f0cd40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:25.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.493+0000 7f0cee4fb640 1 --2- 172.21.15.164:0/2399094053 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f0cd40a3240 0x7f0cd40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:25.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.493+0000 7f0cdf7fe640 1 -- 172.21.15.164:0/2399094053 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f0cd80efcf0 con 0x7f0ce81037a0 2024-09-19T08:43:25.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.493+0000 7f0cee4fb640 1 --2- 172.21.15.164:0/2399094053 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f0cd40a3240 0x7f0cd40a5700 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009b40 tx=0x7f0ce4005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:25.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.496+0000 7f0cdf7fe640 1 -- 172.21.15.164:0/2399094053 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0cd80bceb0 con 0x7f0ce81037a0 2024-09-19T08:43:25.685 INFO:journalctl@ceph.mds.f.smithi173.stdout:Sep 19 08:43:25 smithi173 podman[65050]: 2024-09-19 08:43:25.335988391 +0000 UTC m=+0.015616545 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:43:25.685 INFO:journalctl@ceph.mds.f.smithi173.stdout:Sep 19 08:43:25 smithi173 podman[65050]: 2024-09-19 08:43:25.458955688 +0000 UTC m=+0.138583834 container create d799c13ea99f151a589f00b2b87100d061c188e6a3c2276b9fc264d4d0d3b959 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-f, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-19T08:43:25.686 INFO:journalctl@ceph.mds.f.smithi173.stdout:Sep 19 08:43:25 smithi173 podman[65050]: 2024-09-19 08:43:25.686302652 +0000 UTC m=+0.365930791 container init d799c13ea99f151a589f00b2b87100d061c188e6a3c2276b9fc264d4d0d3b959 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-f, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-19T08:43:25.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.700+0000 7f0cf075f640 1 -- 172.21.15.164:0/2399094053 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ce81072c0 con 0x7f0ce81037a0 2024-09-19T08:43:25.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.700+0000 7f0cdf7fe640 1 -- 172.21.15.164:0/2399094053 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4741 (secure 0 0 0) 0x7f0cd80bc7b0 con 0x7f0ce81037a0 2024-09-19T08:43:25.700 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:43:25.700 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":6,"btime":"2024-09-19T08:43:23:798694+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-19T08:43:25.700 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 6 2024-09-19T08:43:25.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.703+0000 7f0cf075f640 1 -- 172.21.15.164:0/2399094053 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f0cd40a3240 msgr2=0x7f0cd40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:25.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.703+0000 7f0cf075f640 1 --2- 172.21.15.164:0/2399094053 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f0cd40a3240 0x7f0cd40a5700 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009b40 tx=0x7f0ce4005e50 comp rx=0 tx=0).stop 2024-09-19T08:43:25.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.703+0000 7f0cf075f640 1 -- 172.21.15.164:0/2399094053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ce81037a0 msgr2=0x7f0ce8198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:25.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.703+0000 7f0cf075f640 1 --2- 172.21.15.164:0/2399094053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ce81037a0 0x7f0ce8198b80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8005940 tx=0x7f0cd80042f0 comp rx=0 tx=0).stop 2024-09-19T08:43:25.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.703+0000 7f0cf075f640 1 -- 172.21.15.164:0/2399094053 shutdown_connections 2024-09-19T08:43:25.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.703+0000 7f0cf075f640 1 --2- 172.21.15.164:0/2399094053 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ce810c2f0 0x7f0ce819cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:25.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.703+0000 7f0cf075f640 1 --2- 172.21.15.164:0/2399094053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ce81037a0 0x7f0ce8198b80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:25.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.703+0000 7f0cf075f640 1 --2- 172.21.15.164:0/2399094053 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f0cd40a3240 0x7f0cd40a5700 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:25.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.703+0000 7f0cf075f640 1 --2- 172.21.15.164:0/2399094053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ce8102df0 0x7f0ce8198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:25.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.703+0000 7f0cf075f640 1 -- 172.21.15.164:0/2399094053 >> 172.21.15.164:0/2399094053 conn(0x7f0ce80fc7d0 msgr2=0x7f0ce80fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:25.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.703+0000 7f0cf075f640 1 -- 172.21.15.164:0/2399094053 shutdown_connections 2024-09-19T08:43:25.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:25.703+0000 7f0cf075f640 1 -- 172.21.15.164:0/2399094053 wait complete. 2024-09-19T08:43:26.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:25 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2399094053' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:43:26.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:25 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2399094053' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:43:26.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:25 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2399094053' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:43:26.161 INFO:journalctl@ceph.mds.f.smithi173.stdout:Sep 19 08:43:25 smithi173 podman[65050]: 2024-09-19 08:43:25.689048555 +0000 UTC m=+0.368676701 container start d799c13ea99f151a589f00b2b87100d061c188e6a3c2276b9fc264d4d0d3b959 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_CLEAN=True) 2024-09-19T08:43:26.161 INFO:journalctl@ceph.mds.f.smithi173.stdout:Sep 19 08:43:25 smithi173 ceph-mds[65069]: -- 172.21.15.173:0/3705075869 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5563bfa37860 con 0x5563bfa14800 2024-09-19T08:43:26.161 INFO:journalctl@ceph.mds.f.smithi173.stdout:Sep 19 08:43:25 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-f[65065]: starting mds.f at 2024-09-19T08:43:26.161 INFO:journalctl@ceph.mds.f.smithi173.stdout:Sep 19 08:43:25 smithi173 bash[65050]: d799c13ea99f151a589f00b2b87100d061c188e6a3c2276b9fc264d4d0d3b959 2024-09-19T08:43:26.161 INFO:journalctl@ceph.mds.f.smithi173.stdout:Sep 19 08:43:25 smithi173 systemd[1]: Started Ceph mds.f for 1e4b0d74-7662-11ef-bceb-c7b262605968. 2024-09-19T08:43:26.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:26 smithi173 ceph-mon[35306]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:26.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:26 smithi173 ceph-mon[35306]: mds.? [v2:172.21.15.173:6834/947317705,v1:172.21.15.173:6835/947317705] up:boot 2024-09-19T08:43:26.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:26 smithi173 ceph-mon[35306]: fsmap 6 up:standby 2024-09-19T08:43:26.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:26 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T08:43:26.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:26 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:26.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:26 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:26.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:26 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:26.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:26 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:26.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:26 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:26.912 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:26 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:27.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:26 smithi164 ceph-mon[30928]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:27.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:26 smithi164 ceph-mon[30928]: mds.? [v2:172.21.15.173:6834/947317705,v1:172.21.15.173:6835/947317705] up:boot 2024-09-19T08:43:27.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:26 smithi164 ceph-mon[30928]: fsmap 6 up:standby 2024-09-19T08:43:27.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:26 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T08:43:27.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:26 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:27.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:26 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:27.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:26 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:27.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:26 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:27.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:26 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:27.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:26 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:27.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:26 smithi170 ceph-mon[35072]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:27.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:26 smithi170 ceph-mon[35072]: mds.? [v2:172.21.15.173:6834/947317705,v1:172.21.15.173:6835/947317705] up:boot 2024-09-19T08:43:27.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:26 smithi170 ceph-mon[35072]: fsmap 6 up:standby 2024-09-19T08:43:27.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:26 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T08:43:27.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:26 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:27.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:26 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:27.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:26 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:27.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:26 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:27.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:26 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:27.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:26 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:27.777 INFO:journalctl@ceph.mds.c.smithi173.stdout:Sep 19 08:43:27 smithi173 systemd[1]: Starting Ceph mds.c for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T08:43:28.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:27 smithi164 ceph-mon[30928]: Deploying daemon mds.c on smithi173 2024-09-19T08:43:28.041 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:27 smithi173 ceph-mon[35306]: Deploying daemon mds.c on smithi173 2024-09-19T08:43:28.041 INFO:journalctl@ceph.mds.c.smithi173.stdout:Sep 19 08:43:27 smithi173 podman[65352]: 2024-09-19 08:43:27.783810754 +0000 UTC m=+0.016457187 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:43:28.041 INFO:journalctl@ceph.mds.c.smithi173.stdout:Sep 19 08:43:27 smithi173 podman[65352]: 2024-09-19 08:43:27.897060371 +0000 UTC m=+0.129706786 container create 8658563522b083f049fc01a5e198fb37d73d3cc933ecf6b79e3aee6184410a76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-19T08:43:28.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:27 smithi170 ceph-mon[35072]: Deploying daemon mds.c on smithi173 2024-09-19T08:43:28.329 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:43:28.410 INFO:journalctl@ceph.mds.c.smithi173.stdout:Sep 19 08:43:28 smithi173 podman[65352]: 2024-09-19 08:43:28.135506808 +0000 UTC m=+0.368153225 container init 8658563522b083f049fc01a5e198fb37d73d3cc933ecf6b79e3aee6184410a76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-c, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-19T08:43:28.411 INFO:journalctl@ceph.mds.c.smithi173.stdout:Sep 19 08:43:28 smithi173 podman[65352]: 2024-09-19 08:43:28.138363246 +0000 UTC m=+0.371009669 container start 8658563522b083f049fc01a5e198fb37d73d3cc933ecf6b79e3aee6184410a76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-c, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-19T08:43:28.411 INFO:journalctl@ceph.mds.c.smithi173.stdout:Sep 19 08:43:28 smithi173 ceph-mds[65371]: -- 172.21.15.173:0/4279762530 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555918f8f860 con 0x555918f6c800 2024-09-19T08:43:28.411 INFO:journalctl@ceph.mds.c.smithi173.stdout:Sep 19 08:43:28 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-c[65367]: starting mds.c at 2024-09-19T08:43:28.411 INFO:journalctl@ceph.mds.c.smithi173.stdout:Sep 19 08:43:28 smithi173 bash[65352]: 8658563522b083f049fc01a5e198fb37d73d3cc933ecf6b79e3aee6184410a76 2024-09-19T08:43:28.411 INFO:journalctl@ceph.mds.c.smithi173.stdout:Sep 19 08:43:28 smithi173 systemd[1]: Started Ceph mds.c for 1e4b0d74-7662-11ef-bceb-c7b262605968. 2024-09-19T08:43:28.585 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:43:28.972 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:28 smithi164 ceph-mon[30928]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:28.972 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:28 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:28.973 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:28 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:28.973 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:28 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:28.973 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:28 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:28.973 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:28 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:28.973 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:28 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:29.027 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:28 smithi170 ceph-mon[35072]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:29.027 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:28 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:29.027 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:28 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:29.027 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:28 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:29.027 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:28 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:29.027 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:28 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:29.027 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:28 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:29.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.057+0000 7f86c6a2b640 1 -- 172.21.15.164:0/4073943976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86c0102570 msgr2=0x7f86c01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:29.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.057+0000 7f86c6a2b640 1 --2- 172.21.15.164:0/4073943976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86c0102570 0x7f86c01029b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f86a8009900 tx=0x7f86a802a480 comp rx=0 tx=0).stop 2024-09-19T08:43:29.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.057+0000 7f86c6a2b640 1 -- 172.21.15.164:0/4073943976 shutdown_connections 2024-09-19T08:43:29.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.057+0000 7f86c6a2b640 1 --2- 172.21.15.164:0/4073943976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86c0102ef0 0x7f86c010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:29.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.057+0000 7f86c6a2b640 1 --2- 172.21.15.164:0/4073943976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86c0102570 0x7f86c01029b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:29.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.057+0000 7f86c6a2b640 1 --2- 172.21.15.164:0/4073943976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86c0100e20 0x7f86c0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:29.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.057+0000 7f86c6a2b640 1 -- 172.21.15.164:0/4073943976 >> 172.21.15.164:0/4073943976 conn(0x7f86c00fc7d0 msgr2=0x7f86c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:29.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.057+0000 7f86c6a2b640 1 -- 172.21.15.164:0/4073943976 shutdown_connections 2024-09-19T08:43:29.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.057+0000 7f86c6a2b640 1 -- 172.21.15.164:0/4073943976 wait complete. 2024-09-19T08:43:29.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.057+0000 7f86c6a2b640 1 Processor -- start 2024-09-19T08:43:29.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.057+0000 7f86c6a2b640 1 -- start start 2024-09-19T08:43:29.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.058+0000 7f86c6a2b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86c0100e20 0x7f86c00fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:29.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.058+0000 7f86c6a2b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86c0102570 0x7f86c00fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:29.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.058+0000 7f86c6a2b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86c0102ef0 0x7f86c00fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:29.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.058+0000 7f86c6a2b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f86c0110f80 con 0x7f86c0102ef0 2024-09-19T08:43:29.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.058+0000 7f86c6a2b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f86c0110e00 con 0x7f86c0102570 2024-09-19T08:43:29.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.058+0000 7f86c6a2b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f86c0111100 con 0x7f86c0100e20 2024-09-19T08:43:29.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.058+0000 7f86c4fc8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86c0102ef0 0x7f86c00fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:29.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.058+0000 7f86b7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86c0102570 0x7f86c00fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:29.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.058+0000 7f86c4fc8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86c0102ef0 0x7f86c00fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51918/0 (socket says 172.21.15.164:51918) 2024-09-19T08:43:29.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.058+0000 7f86c4fc8640 1 -- 172.21.15.164:0/2745206021 learned_addr learned my addr 172.21.15.164:0/2745206021 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:43:29.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.058+0000 7f86bffff640 1 --2- 172.21.15.164:0/2745206021 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86c0100e20 0x7f86c00fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:29.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.058+0000 7f86c4fc8640 1 -- 172.21.15.164:0/2745206021 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86c0100e20 msgr2=0x7f86c00fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:29.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.058+0000 7f86c4fc8640 1 --2- 172.21.15.164:0/2745206021 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86c0100e20 0x7f86c00fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:29.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.058+0000 7f86c4fc8640 1 -- 172.21.15.164:0/2745206021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86c0102570 msgr2=0x7f86c00fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:29.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.058+0000 7f86c4fc8640 1 --2- 172.21.15.164:0/2745206021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86c0102570 0x7f86c00fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:29.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.058+0000 7f86c4fc8640 1 -- 172.21.15.164:0/2745206021 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c01a6a20 con 0x7f86c0102ef0 2024-09-19T08:43:29.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.058+0000 7f86bffff640 1 --2- 172.21.15.164:0/2745206021 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86c0100e20 0x7f86c00fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:43:29.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.058+0000 7f86c4fc8640 1 --2- 172.21.15.164:0/2745206021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86c0102ef0 0x7f86c00fd700 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f86b800eb10 tx=0x7f86b800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:29.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.058+0000 7f86bdffb640 1 -- 172.21.15.164:0/2745206021 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b8004b20 con 0x7f86c0102ef0 2024-09-19T08:43:29.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.059+0000 7f86c6a2b640 1 -- 172.21.15.164:0/2745206021 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86c01a6d10 con 0x7f86c0102ef0 2024-09-19T08:43:29.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.059+0000 7f86bdffb640 1 -- 172.21.15.164:0/2745206021 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86b8004cc0 con 0x7f86c0102ef0 2024-09-19T08:43:29.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.059+0000 7f86bdffb640 1 -- 172.21.15.164:0/2745206021 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b8010760 con 0x7f86c0102ef0 2024-09-19T08:43:29.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.059+0000 7f86c6a2b640 1 -- 172.21.15.164:0/2745206021 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86c01a71d0 con 0x7f86c0102ef0 2024-09-19T08:43:29.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.060+0000 7f86c6a2b640 1 -- 172.21.15.164:0/2745206021 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86c0101200 con 0x7f86c0102ef0 2024-09-19T08:43:29.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.061+0000 7f86bdffb640 1 -- 172.21.15.164:0/2745206021 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f86b8010900 con 0x7f86c0102ef0 2024-09-19T08:43:29.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.061+0000 7f86bdffb640 1 --2- 172.21.15.164:0/2745206021 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f86ac0a3240 0x7f86ac0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:29.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.062+0000 7f86bffff640 1 --2- 172.21.15.164:0/2745206021 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f86ac0a3240 0x7f86ac0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:29.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.062+0000 7f86bdffb640 1 -- 172.21.15.164:0/2745206021 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f86b80cfc40 con 0x7f86c0102ef0 2024-09-19T08:43:29.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.062+0000 7f86bffff640 1 --2- 172.21.15.164:0/2745206021 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f86ac0a3240 0x7f86ac0a5700 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f86b0005fd0 tx=0x7f86b0005ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:29.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.069+0000 7f86bdffb640 1 -- 172.21.15.164:0/2745206021 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f86b8054210 con 0x7f86c0102ef0 2024-09-19T08:43:29.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:28 smithi173 ceph-mon[35306]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:29.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:28 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:29.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:28 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:29.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:28 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:29.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:28 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:29.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:28 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:29.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:28 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:29.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.277+0000 7f86c6a2b640 1 -- 172.21.15.164:0/2745206021 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86c00fe1b0 con 0x7f86c0102ef0 2024-09-19T08:43:29.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.277+0000 7f86bdffb640 1 -- 172.21.15.164:0/2745206021 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6394 (secure 0 0 0) 0x7f86c00fe1b0 con 0x7f86c0102ef0 2024-09-19T08:43:29.277 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:43:29.277 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":8,"btime":"2024-09-19T08:43:28:777201+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24418,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-19T08:43:29.277 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 8 2024-09-19T08:43:29.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.280+0000 7f86c6a2b640 1 -- 172.21.15.164:0/2745206021 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f86ac0a3240 msgr2=0x7f86ac0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:29.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.280+0000 7f86c6a2b640 1 --2- 172.21.15.164:0/2745206021 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f86ac0a3240 0x7f86ac0a5700 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f86b0005fd0 tx=0x7f86b0005ea0 comp rx=0 tx=0).stop 2024-09-19T08:43:29.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.280+0000 7f86c6a2b640 1 -- 172.21.15.164:0/2745206021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86c0102ef0 msgr2=0x7f86c00fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:29.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.280+0000 7f86c6a2b640 1 --2- 172.21.15.164:0/2745206021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86c0102ef0 0x7f86c00fd700 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f86b800eb10 tx=0x7f86b800c2d0 comp rx=0 tx=0).stop 2024-09-19T08:43:29.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.280+0000 7f86c6a2b640 1 -- 172.21.15.164:0/2745206021 shutdown_connections 2024-09-19T08:43:29.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.280+0000 7f86c6a2b640 1 --2- 172.21.15.164:0/2745206021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86c0102ef0 0x7f86c00fd700 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:29.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.280+0000 7f86c6a2b640 1 --2- 172.21.15.164:0/2745206021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86c0102570 0x7f86c00fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:29.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.280+0000 7f86c6a2b640 1 --2- 172.21.15.164:0/2745206021 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f86ac0a3240 0x7f86ac0a5700 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:29.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.280+0000 7f86c6a2b640 1 --2- 172.21.15.164:0/2745206021 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86c0100e20 0x7f86c00fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:29.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.280+0000 7f86c6a2b640 1 -- 172.21.15.164:0/2745206021 >> 172.21.15.164:0/2745206021 conn(0x7f86c00fc7d0 msgr2=0x7f86c006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:29.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.280+0000 7f86c6a2b640 1 -- 172.21.15.164:0/2745206021 shutdown_connections 2024-09-19T08:43:29.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:29.280+0000 7f86c6a2b640 1 -- 172.21.15.164:0/2745206021 wait complete. 2024-09-19T08:43:30.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:29 smithi164 ceph-mon[30928]: Deploying daemon mds.h on smithi170 2024-09-19T08:43:30.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:29 smithi164 ceph-mon[30928]: mds.? [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] up:boot 2024-09-19T08:43:30.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:29 smithi164 ceph-mon[30928]: fsmap 7 up:standby 2024-09-19T08:43:30.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:29 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T08:43:30.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:29 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2745206021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:43:30.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:29 smithi170 ceph-mon[35072]: Deploying daemon mds.h on smithi170 2024-09-19T08:43:30.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:29 smithi170 ceph-mon[35072]: mds.? [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] up:boot 2024-09-19T08:43:30.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:29 smithi170 ceph-mon[35072]: fsmap 7 up:standby 2024-09-19T08:43:30.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:29 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T08:43:30.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:29 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2745206021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:43:30.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:29 smithi173 ceph-mon[35306]: Deploying daemon mds.h on smithi170 2024-09-19T08:43:30.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:29 smithi173 ceph-mon[35306]: mds.? [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] up:boot 2024-09-19T08:43:30.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:29 smithi173 ceph-mon[35306]: fsmap 7 up:standby 2024-09-19T08:43:30.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:29 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T08:43:30.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:29 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2745206021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:43:30.514 INFO:journalctl@ceph.mds.h.smithi170.stdout:Sep 19 08:43:30 smithi170 systemd[1]: Starting Ceph mds.h for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T08:43:30.792 INFO:journalctl@ceph.mds.h.smithi170.stdout:Sep 19 08:43:30 smithi170 podman[64700]: 2024-09-19 08:43:30.417409391 +0000 UTC m=+0.015894106 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:43:30.793 INFO:journalctl@ceph.mds.h.smithi170.stdout:Sep 19 08:43:30 smithi170 podman[64700]: 2024-09-19 08:43:30.535504507 +0000 UTC m=+0.133989213 container create e9a4945d27ce27383cb5e0d53608dabcf59f2244a210fe744851953d9a72ab80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-19T08:43:31.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:30 smithi170 ceph-mon[35072]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:31.125 INFO:journalctl@ceph.mds.h.smithi170.stdout:Sep 19 08:43:30 smithi170 podman[64700]: 2024-09-19 08:43:30.796532274 +0000 UTC m=+0.395016980 container init e9a4945d27ce27383cb5e0d53608dabcf59f2244a210fe744851953d9a72ab80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-19T08:43:31.125 INFO:journalctl@ceph.mds.h.smithi170.stdout:Sep 19 08:43:30 smithi170 podman[64700]: 2024-09-19 08:43:30.799495975 +0000 UTC m=+0.397980691 container start e9a4945d27ce27383cb5e0d53608dabcf59f2244a210fe744851953d9a72ab80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-h, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-19T08:43:31.125 INFO:journalctl@ceph.mds.h.smithi170.stdout:Sep 19 08:43:30 smithi170 ceph-mds[64719]: -- 172.21.15.170:0/2780496386 <== mon.2 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a6d672b860 con 0x55a6d6709400 2024-09-19T08:43:31.125 INFO:journalctl@ceph.mds.h.smithi170.stdout:Sep 19 08:43:30 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-h[64715]: starting mds.h at 2024-09-19T08:43:31.125 INFO:journalctl@ceph.mds.h.smithi170.stdout:Sep 19 08:43:30 smithi170 bash[64700]: e9a4945d27ce27383cb5e0d53608dabcf59f2244a210fe744851953d9a72ab80 2024-09-19T08:43:31.125 INFO:journalctl@ceph.mds.h.smithi170.stdout:Sep 19 08:43:30 smithi170 systemd[1]: Started Ceph mds.h for 1e4b0d74-7662-11ef-bceb-c7b262605968. 2024-09-19T08:43:31.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:30 smithi173 ceph-mon[35306]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:31.285 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:30 smithi164 ceph-mon[30928]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:31.915 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:43:32.179 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:43:32.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:31 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:32.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:31 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:32.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:31 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:32.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:31 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:32.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:31 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:32.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:31 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:32.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:31 smithi164 ceph-mon[30928]: Deploying daemon mds.i on smithi173 2024-09-19T08:43:32.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:31 smithi164 ceph-mon[30928]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:32.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:31 smithi164 ceph-mon[30928]: mds.? [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] up:boot 2024-09-19T08:43:32.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:31 smithi164 ceph-mon[30928]: fsmap 8 up:standby 2024-09-19T08:43:32.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:31 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T08:43:32.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:31 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:32.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:31 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:32.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:31 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:32.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:31 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:32.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:31 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:32.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:31 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:32.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:31 smithi170 ceph-mon[35072]: Deploying daemon mds.i on smithi173 2024-09-19T08:43:32.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:31 smithi170 ceph-mon[35072]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:32.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:31 smithi170 ceph-mon[35072]: mds.? [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] up:boot 2024-09-19T08:43:32.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:31 smithi170 ceph-mon[35072]: fsmap 8 up:standby 2024-09-19T08:43:32.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:31 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T08:43:32.398 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:31 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:32.398 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:31 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:32.398 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:31 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:32.398 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:31 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:32.398 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:31 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:32.398 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:31 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:32.398 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:31 smithi173 ceph-mon[35306]: Deploying daemon mds.i on smithi173 2024-09-19T08:43:32.398 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:31 smithi173 ceph-mon[35306]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:32.398 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:31 smithi173 ceph-mon[35306]: mds.? [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] up:boot 2024-09-19T08:43:32.399 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:31 smithi173 ceph-mon[35306]: fsmap 8 up:standby 2024-09-19T08:43:32.399 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:31 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T08:43:32.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.629+0000 7fb37a3f0640 1 -- 172.21.15.164:0/1256400446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb374101420 msgr2=0x7fb37410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:32.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.629+0000 7fb37a3f0640 1 --2- 172.21.15.164:0/1256400446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb374101420 0x7fb37410d950 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb36c009cd0 tx=0x7fb36c02a8e0 comp rx=0 tx=0).stop 2024-09-19T08:43:32.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.629+0000 7fb37a3f0640 1 -- 172.21.15.164:0/1256400446 shutdown_connections 2024-09-19T08:43:32.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.629+0000 7fb37a3f0640 1 --2- 172.21.15.164:0/1256400446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb374101420 0x7fb37410d950 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:32.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.629+0000 7fb37a3f0640 1 --2- 172.21.15.164:0/1256400446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb374100a80 0x7fb374100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:32.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.629+0000 7fb37a3f0640 1 --2- 172.21.15.164:0/1256400446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb374106a80 0x7fb374106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:32.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.629+0000 7fb37a3f0640 1 -- 172.21.15.164:0/1256400446 >> 172.21.15.164:0/1256400446 conn(0x7fb3740fc7a0 msgr2=0x7fb3740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:32.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.629+0000 7fb37a3f0640 1 -- 172.21.15.164:0/1256400446 shutdown_connections 2024-09-19T08:43:32.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.629+0000 7fb37a3f0640 1 -- 172.21.15.164:0/1256400446 wait complete. 2024-09-19T08:43:32.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.630+0000 7fb37a3f0640 1 Processor -- start 2024-09-19T08:43:32.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.630+0000 7fb37a3f0640 1 -- start start 2024-09-19T08:43:32.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.630+0000 7fb37a3f0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb374100a80 0x7fb374196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:32.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.630+0000 7fb37a3f0640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb374101420 0x7fb374196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:32.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.630+0000 7fb37a3f0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb374106a80 0x7fb37419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:32.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.630+0000 7fb37a3f0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb37410fd70 con 0x7fb374106a80 2024-09-19T08:43:32.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.630+0000 7fb37a3f0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb37410fbf0 con 0x7fb374100a80 2024-09-19T08:43:32.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.630+0000 7fb37a3f0640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb37410fef0 con 0x7fb374101420 2024-09-19T08:43:32.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.630+0000 7fb37898d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb374106a80 0x7fb37419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:32.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.630+0000 7fb37898d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb374106a80 0x7fb37419aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51934/0 (socket says 172.21.15.164:51934) 2024-09-19T08:43:32.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.630+0000 7fb37898d640 1 -- 172.21.15.164:0/290242245 learned_addr learned my addr 172.21.15.164:0/290242245 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:43:32.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.630+0000 7fb36bfff640 1 --2- 172.21.15.164:0/290242245 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb374101420 0x7fb374196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:32.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.630+0000 7fb373fff640 1 --2- 172.21.15.164:0/290242245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb374100a80 0x7fb374196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:32.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.631+0000 7fb37898d640 1 -- 172.21.15.164:0/290242245 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb374101420 msgr2=0x7fb374196750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:32.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.631+0000 7fb37898d640 1 --2- 172.21.15.164:0/290242245 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb374101420 0x7fb374196750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:32.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.631+0000 7fb37898d640 1 -- 172.21.15.164:0/290242245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb374100a80 msgr2=0x7fb374196210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:32.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.631+0000 7fb37898d640 1 --2- 172.21.15.164:0/290242245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb374100a80 0x7fb374196210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:32.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.631+0000 7fb37898d640 1 -- 172.21.15.164:0/290242245 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37419b1c0 con 0x7fb374106a80 2024-09-19T08:43:32.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.631+0000 7fb36bfff640 1 --2- 172.21.15.164:0/290242245 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb374101420 0x7fb374196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:43:32.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.631+0000 7fb37898d640 1 --2- 172.21.15.164:0/290242245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb374106a80 0x7fb37419aae0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb36c005cc0 tx=0x7fb36c032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:32.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.631+0000 7fb371ffb640 1 -- 172.21.15.164:0/290242245 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c004040 con 0x7fb374106a80 2024-09-19T08:43:32.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.631+0000 7fb371ffb640 1 -- 172.21.15.164:0/290242245 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb36c0046c0 con 0x7fb374106a80 2024-09-19T08:43:32.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.631+0000 7fb37a3f0640 1 -- 172.21.15.164:0/290242245 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb37419b450 con 0x7fb374106a80 2024-09-19T08:43:32.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.631+0000 7fb371ffb640 1 -- 172.21.15.164:0/290242245 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c030e00 con 0x7fb374106a80 2024-09-19T08:43:32.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.631+0000 7fb37a3f0640 1 -- 172.21.15.164:0/290242245 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3741a2d90 con 0x7fb374106a80 2024-09-19T08:43:32.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.632+0000 7fb37a3f0640 1 -- 172.21.15.164:0/290242245 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb374100ee0 con 0x7fb374106a80 2024-09-19T08:43:32.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.633+0000 7fb371ffb640 1 -- 172.21.15.164:0/290242245 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fb36c0305a0 con 0x7fb374106a80 2024-09-19T08:43:32.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.634+0000 7fb371ffb640 1 --2- 172.21.15.164:0/290242245 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb3600a5450 0x7fb3600a7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:32.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.634+0000 7fb373fff640 1 --2- 172.21.15.164:0/290242245 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb3600a5450 0x7fb3600a7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:32.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.634+0000 7fb371ffb640 1 -- 172.21.15.164:0/290242245 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fb36c02e030 con 0x7fb374106a80 2024-09-19T08:43:32.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.634+0000 7fb373fff640 1 --2- 172.21.15.164:0/290242245 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb3600a5450 0x7fb3600a7910 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb364009b50 tx=0x7fb364005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:32.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.638+0000 7fb371ffb640 1 -- 172.21.15.164:0/290242245 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb36c0f2e20 con 0x7fb374106a80 2024-09-19T08:43:32.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.839+0000 7fb37a3f0640 1 -- 172.21.15.164:0/290242245 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb374109050 con 0x7fb374106a80 2024-09-19T08:43:32.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.840+0000 7fb371ffb640 1 -- 172.21.15.164:0/290242245 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7222 (secure 0 0 0) 0x7fb36c04b060 con 0x7fb374106a80 2024-09-19T08:43:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:43:32.840 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":9,"btime":"2024-09-19T08:43:31:793152+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24418,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-19T08:43:32.840 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 9 2024-09-19T08:43:32.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.842+0000 7fb37a3f0640 1 -- 172.21.15.164:0/290242245 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb3600a5450 msgr2=0x7fb3600a7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:32.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.842+0000 7fb37a3f0640 1 --2- 172.21.15.164:0/290242245 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb3600a5450 0x7fb3600a7910 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb364009b50 tx=0x7fb364005f90 comp rx=0 tx=0).stop 2024-09-19T08:43:32.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.842+0000 7fb37a3f0640 1 -- 172.21.15.164:0/290242245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb374106a80 msgr2=0x7fb37419aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:32.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.842+0000 7fb37a3f0640 1 --2- 172.21.15.164:0/290242245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb374106a80 0x7fb37419aae0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb36c005cc0 tx=0x7fb36c032f40 comp rx=0 tx=0).stop 2024-09-19T08:43:32.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.842+0000 7fb37a3f0640 1 -- 172.21.15.164:0/290242245 shutdown_connections 2024-09-19T08:43:32.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.842+0000 7fb37a3f0640 1 --2- 172.21.15.164:0/290242245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb374106a80 0x7fb37419aae0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:32.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.842+0000 7fb37a3f0640 1 --2- 172.21.15.164:0/290242245 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb3600a5450 0x7fb3600a7910 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:32.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.842+0000 7fb37a3f0640 1 --2- 172.21.15.164:0/290242245 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb374101420 0x7fb374196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:32.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.842+0000 7fb37a3f0640 1 --2- 172.21.15.164:0/290242245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb374100a80 0x7fb374196210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:32.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.842+0000 7fb37a3f0640 1 -- 172.21.15.164:0/290242245 >> 172.21.15.164:0/290242245 conn(0x7fb3740fc7a0 msgr2=0x7fb3740fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:32.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.842+0000 7fb37a3f0640 1 -- 172.21.15.164:0/290242245 shutdown_connections 2024-09-19T08:43:32.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:32.842+0000 7fb37a3f0640 1 -- 172.21.15.164:0/290242245 wait complete. 2024-09-19T08:43:32.974 INFO:journalctl@ceph.mds.i.smithi173.stdout:Sep 19 08:43:32 smithi173 systemd[1]: Starting Ceph mds.i for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T08:43:33.253 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:32 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/290242245' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:43:33.253 INFO:journalctl@ceph.mds.i.smithi173.stdout:Sep 19 08:43:33 smithi173 podman[65652]: 2024-09-19 08:43:33.006913222 +0000 UTC m=+0.015777859 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:43:33.253 INFO:journalctl@ceph.mds.i.smithi173.stdout:Sep 19 08:43:33 smithi173 podman[65652]: 2024-09-19 08:43:33.14793293 +0000 UTC m=+0.156797558 container create 06eaf68a19e0dc7df99e4cbae06eb7766ee33b21ae8fab0d665672c64f62bf96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-i, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T08:43:33.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:32 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/290242245' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:43:33.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:32 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/290242245' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:43:33.660 INFO:journalctl@ceph.mds.i.smithi173.stdout:Sep 19 08:43:33 smithi173 podman[65652]: 2024-09-19 08:43:33.359571567 +0000 UTC m=+0.368436196 container init 06eaf68a19e0dc7df99e4cbae06eb7766ee33b21ae8fab0d665672c64f62bf96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-i, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-19T08:43:33.661 INFO:journalctl@ceph.mds.i.smithi173.stdout:Sep 19 08:43:33 smithi173 podman[65652]: 2024-09-19 08:43:33.362596827 +0000 UTC m=+0.371461468 container start 06eaf68a19e0dc7df99e4cbae06eb7766ee33b21ae8fab0d665672c64f62bf96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-i, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True) 2024-09-19T08:43:33.661 INFO:journalctl@ceph.mds.i.smithi173.stdout:Sep 19 08:43:33 smithi173 ceph-mds[65671]: -- 172.21.15.173:0/2685968360 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5555835b9c20 con 0x555583597000 2024-09-19T08:43:33.661 INFO:journalctl@ceph.mds.i.smithi173.stdout:Sep 19 08:43:33 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-i[65667]: starting mds.i at 2024-09-19T08:43:33.661 INFO:journalctl@ceph.mds.i.smithi173.stdout:Sep 19 08:43:33 smithi173 bash[65652]: 06eaf68a19e0dc7df99e4cbae06eb7766ee33b21ae8fab0d665672c64f62bf96 2024-09-19T08:43:33.661 INFO:journalctl@ceph.mds.i.smithi173.stdout:Sep 19 08:43:33 smithi173 systemd[1]: Started Ceph mds.i for 1e4b0d74-7662-11ef-bceb-c7b262605968. 2024-09-19T08:43:34.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:34 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:34.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:34 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:34.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:34 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:34.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:34 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:34.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:34 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:34.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:34 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:34.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:34 smithi164 ceph-mon[30928]: Deploying daemon mds.g on smithi164 2024-09-19T08:43:34.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:34 smithi164 ceph-mon[30928]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:34.783 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:34 smithi164 ceph-mon[30928]: mds.? [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] up:boot 2024-09-19T08:43:34.783 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:34 smithi164 ceph-mon[30928]: fsmap 9 up:standby 2024-09-19T08:43:34.783 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:34 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T08:43:34.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:34 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:34.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:34 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:34.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:34 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:34.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:34 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:34.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:34 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:34.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:34 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:34.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:34 smithi170 ceph-mon[35072]: Deploying daemon mds.g on smithi164 2024-09-19T08:43:34.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:34 smithi170 ceph-mon[35072]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:34.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:34 smithi170 ceph-mon[35072]: mds.? [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] up:boot 2024-09-19T08:43:34.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:34 smithi170 ceph-mon[35072]: fsmap 9 up:standby 2024-09-19T08:43:34.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:34 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T08:43:34.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:34 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:34.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:34 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:34.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:34 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:34.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:34 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:34.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:34 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:34.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:34 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:34.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:34 smithi173 ceph-mon[35306]: Deploying daemon mds.g on smithi164 2024-09-19T08:43:34.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:34 smithi173 ceph-mon[35306]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:34.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:34 smithi173 ceph-mon[35306]: mds.? [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] up:boot 2024-09-19T08:43:34.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:34 smithi173 ceph-mon[35306]: fsmap 9 up:standby 2024-09-19T08:43:34.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:34 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T08:43:35.467 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:43:35.762 INFO:journalctl@ceph.mds.g.smithi164.stdout:Sep 19 08:43:35 smithi164 systemd[1]: Starting Ceph mds.g for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T08:43:36.015 INFO:journalctl@ceph.mds.g.smithi164.stdout:Sep 19 08:43:35 smithi164 podman[71171]: 2024-09-19 08:43:35.667116015 +0000 UTC m=+0.017746107 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:43:36.015 INFO:journalctl@ceph.mds.g.smithi164.stdout:Sep 19 08:43:35 smithi164 podman[71171]: 2024-09-19 08:43:35.763296311 +0000 UTC m=+0.113926389 container create 085df45388ee07c82fa8ec731cc4304013419c0403a75d9dfdf6150ecb6b3c0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-g, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T08:43:36.015 INFO:journalctl@ceph.mds.g.smithi164.stdout:Sep 19 08:43:36 smithi164 podman[71171]: 2024-09-19 08:43:36.016108893 +0000 UTC m=+0.366738968 container init 085df45388ee07c82fa8ec731cc4304013419c0403a75d9dfdf6150ecb6b3c0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-g, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T08:43:36.143 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:43:36.265 INFO:journalctl@ceph.mds.g.smithi164.stdout:Sep 19 08:43:36 smithi164 podman[71171]: 2024-09-19 08:43:36.019112417 +0000 UTC m=+0.369742502 container start 085df45388ee07c82fa8ec731cc4304013419c0403a75d9dfdf6150ecb6b3c0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-g, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-19T08:43:36.266 INFO:journalctl@ceph.mds.g.smithi164.stdout:Sep 19 08:43:36 smithi164 ceph-mds[71218]: -- 172.21.15.164:0/3644125819 <== mon.0 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555f28f67860 con 0x555f28f45000 2024-09-19T08:43:36.266 INFO:journalctl@ceph.mds.g.smithi164.stdout:Sep 19 08:43:36 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-g[71206]: starting mds.g at 2024-09-19T08:43:36.266 INFO:journalctl@ceph.mds.g.smithi164.stdout:Sep 19 08:43:36 smithi164 bash[71171]: 085df45388ee07c82fa8ec731cc4304013419c0403a75d9dfdf6150ecb6b3c0e 2024-09-19T08:43:36.266 INFO:journalctl@ceph.mds.g.smithi164.stdout:Sep 19 08:43:36 smithi164 systemd[1]: Started Ceph mds.g for 1e4b0d74-7662-11ef-bceb-c7b262605968. 2024-09-19T08:43:36.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.685+0000 7fe443442640 1 -- 172.21.15.164:0/2140726250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe43c100e20 msgr2=0x7fe43c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:36.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.685+0000 7fe443442640 1 --2- 172.21.15.164:0/2140726250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe43c100e20 0x7fe43c101200 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe434009ca0 tx=0x7fe43402a950 comp rx=0 tx=0).stop 2024-09-19T08:43:36.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.685+0000 7fe443442640 1 -- 172.21.15.164:0/2140726250 shutdown_connections 2024-09-19T08:43:36.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.685+0000 7fe443442640 1 --2- 172.21.15.164:0/2140726250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe43c102ef0 0x7fe43c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:36.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.685+0000 7fe443442640 1 --2- 172.21.15.164:0/2140726250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe43c102570 0x7fe43c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:36.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.685+0000 7fe443442640 1 --2- 172.21.15.164:0/2140726250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe43c100e20 0x7fe43c101200 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:36.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.685+0000 7fe443442640 1 -- 172.21.15.164:0/2140726250 >> 172.21.15.164:0/2140726250 conn(0x7fe43c0fc7d0 msgr2=0x7fe43c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:36.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.685+0000 7fe443442640 1 -- 172.21.15.164:0/2140726250 shutdown_connections 2024-09-19T08:43:36.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.685+0000 7fe443442640 1 -- 172.21.15.164:0/2140726250 wait complete. 2024-09-19T08:43:36.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.686+0000 7fe443442640 1 Processor -- start 2024-09-19T08:43:36.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.686+0000 7fe443442640 1 -- start start 2024-09-19T08:43:36.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.686+0000 7fe443442640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe43c100e20 0x7fe43c110900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:36.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.686+0000 7fe443442640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe43c102570 0x7fe43c110e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:36.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.686+0000 7fe443442640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe43c102ef0 0x7fe43c111380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:36.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.686+0000 7fe443442640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c06a690 con 0x7fe43c102570 2024-09-19T08:43:36.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.686+0000 7fe443442640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c06a510 con 0x7fe43c100e20 2024-09-19T08:43:36.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.686+0000 7fe443442640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c06a810 con 0x7fe43c102ef0 2024-09-19T08:43:36.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.686+0000 7fe4409dd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe43c102570 0x7fe43c110e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:36.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.686+0000 7fe4409dd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe43c102570 0x7fe43c110e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44332/0 (socket says 172.21.15.164:44332) 2024-09-19T08:43:36.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.686+0000 7fe4409dd640 1 -- 172.21.15.164:0/2582731866 learned_addr learned my addr 172.21.15.164:0/2582731866 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:43:36.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.686+0000 7fe4411de640 1 --2- 172.21.15.164:0/2582731866 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe43c100e20 0x7fe43c110900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:36.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.686+0000 7fe4419df640 1 --2- 172.21.15.164:0/2582731866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe43c102ef0 0x7fe43c111380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:36.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.687+0000 7fe4409dd640 1 -- 172.21.15.164:0/2582731866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe43c102ef0 msgr2=0x7fe43c111380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:36.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.687+0000 7fe4409dd640 1 --2- 172.21.15.164:0/2582731866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe43c102ef0 0x7fe43c111380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:36.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.687+0000 7fe4409dd640 1 -- 172.21.15.164:0/2582731866 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe43c100e20 msgr2=0x7fe43c110900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:36.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.687+0000 7fe4409dd640 1 --2- 172.21.15.164:0/2582731866 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe43c100e20 0x7fe43c110900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:36.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.687+0000 7fe4409dd640 1 -- 172.21.15.164:0/2582731866 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe43c1a6920 con 0x7fe43c102570 2024-09-19T08:43:36.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.687+0000 7fe4409dd640 1 --2- 172.21.15.164:0/2582731866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe43c102570 0x7fe43c110e40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe43000c920 tx=0x7fe43000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:36.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.687+0000 7fe43a7fc640 1 -- 172.21.15.164:0/2582731866 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe430007c70 con 0x7fe43c102570 2024-09-19T08:43:36.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.687+0000 7fe43a7fc640 1 -- 172.21.15.164:0/2582731866 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe430007e10 con 0x7fe43c102570 2024-09-19T08:43:36.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.687+0000 7fe443442640 1 -- 172.21.15.164:0/2582731866 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe43c1a6c10 con 0x7fe43c102570 2024-09-19T08:43:36.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.687+0000 7fe43a7fc640 1 -- 172.21.15.164:0/2582731866 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe430018890 con 0x7fe43c102570 2024-09-19T08:43:36.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.687+0000 7fe443442640 1 -- 172.21.15.164:0/2582731866 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe43c1a7150 con 0x7fe43c102570 2024-09-19T08:43:36.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.688+0000 7fe443442640 1 -- 172.21.15.164:0/2582731866 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe43c101200 con 0x7fe43c102570 2024-09-19T08:43:36.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.689+0000 7fe43a7fc640 1 -- 172.21.15.164:0/2582731866 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fe430018a70 con 0x7fe43c102570 2024-09-19T08:43:36.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.690+0000 7fe43a7fc640 1 --2- 172.21.15.164:0/2582731866 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fe4240a3240 0x7fe4240a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:36.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.690+0000 7fe4411de640 1 --2- 172.21.15.164:0/2582731866 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fe4240a3240 0x7fe4240a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:36.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.690+0000 7fe43a7fc640 1 -- 172.21.15.164:0/2582731866 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fe4300cf5c0 con 0x7fe43c102570 2024-09-19T08:43:36.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.691+0000 7fe4411de640 1 --2- 172.21.15.164:0/2582731866 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fe4240a3240 0x7fe4240a5700 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe4340093d0 tx=0x7fe434009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:36.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.695+0000 7fe43a7fc640 1 -- 172.21.15.164:0/2582731866 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe430014030 con 0x7fe43c102570 2024-09-19T08:43:36.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.917+0000 7fe443442640 1 -- 172.21.15.164:0/2582731866 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe43c1a73b0 con 0x7fe43c102570 2024-09-19T08:43:36.917 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:36 smithi164 ceph-mon[30928]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:36.917 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:36 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:36.918 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:36 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:36.918 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:36 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:36.918 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:36 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:36.918 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:36 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:36.918 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:36 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:36.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.918+0000 7fe43a7fc640 1 -- 172.21.15.164:0/2582731866 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8878 (secure 0 0 0) 0x7fe43009c780 con 0x7fe43c102570 2024-09-19T08:43:36.919 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:43:36.919 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":11,"btime":"2024-09-19T08:43:36:791503+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24418,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24442,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-19T08:43:36.919 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 11 2024-09-19T08:43:36.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.920+0000 7fe443442640 1 -- 172.21.15.164:0/2582731866 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fe4240a3240 msgr2=0x7fe4240a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:36.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.920+0000 7fe443442640 1 --2- 172.21.15.164:0/2582731866 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fe4240a3240 0x7fe4240a5700 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe4340093d0 tx=0x7fe434009360 comp rx=0 tx=0).stop 2024-09-19T08:43:36.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.920+0000 7fe443442640 1 -- 172.21.15.164:0/2582731866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe43c102570 msgr2=0x7fe43c110e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:36.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.920+0000 7fe443442640 1 --2- 172.21.15.164:0/2582731866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe43c102570 0x7fe43c110e40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe43000c920 tx=0x7fe43000cde0 comp rx=0 tx=0).stop 2024-09-19T08:43:36.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.921+0000 7fe443442640 1 -- 172.21.15.164:0/2582731866 shutdown_connections 2024-09-19T08:43:36.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.921+0000 7fe443442640 1 --2- 172.21.15.164:0/2582731866 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fe4240a3240 0x7fe4240a5700 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:36.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.921+0000 7fe443442640 1 --2- 172.21.15.164:0/2582731866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe43c102ef0 0x7fe43c111380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:36.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.921+0000 7fe443442640 1 --2- 172.21.15.164:0/2582731866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe43c102570 0x7fe43c110e40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:36.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.921+0000 7fe443442640 1 --2- 172.21.15.164:0/2582731866 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe43c100e20 0x7fe43c110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:36.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.921+0000 7fe443442640 1 -- 172.21.15.164:0/2582731866 >> 172.21.15.164:0/2582731866 conn(0x7fe43c0fc7d0 msgr2=0x7fe43c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:36.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.921+0000 7fe443442640 1 -- 172.21.15.164:0/2582731866 shutdown_connections 2024-09-19T08:43:36.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:36.921+0000 7fe443442640 1 -- 172.21.15.164:0/2582731866 wait complete. 2024-09-19T08:43:37.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:36 smithi170 ceph-mon[35072]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:37.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:36 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:37.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:36 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:37.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:36 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:37.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:36 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:37.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:36 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:37.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:36 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:37.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:36 smithi173 ceph-mon[35306]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:37.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:36 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:37.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:36 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:37.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:36 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:37.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:36 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:37.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:36 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:37.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:36 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:38.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:37 smithi170 ceph-mon[35072]: Deploying daemon mds.a on smithi164 2024-09-19T08:43:38.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:37 smithi170 ceph-mon[35072]: mds.? [v2:172.21.15.164:6838/538276454,v1:172.21.15.164:6839/538276454] up:boot 2024-09-19T08:43:38.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:37 smithi170 ceph-mon[35072]: fsmap 10 up:standby 2024-09-19T08:43:38.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:37 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T08:43:38.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:37 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2582731866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:43:38.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:37 smithi173 ceph-mon[35306]: Deploying daemon mds.a on smithi164 2024-09-19T08:43:38.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:37 smithi173 ceph-mon[35306]: mds.? [v2:172.21.15.164:6838/538276454,v1:172.21.15.164:6839/538276454] up:boot 2024-09-19T08:43:38.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:37 smithi173 ceph-mon[35306]: fsmap 10 up:standby 2024-09-19T08:43:38.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:37 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T08:43:38.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:37 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2582731866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:43:38.180 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:37 smithi164 ceph-mon[30928]: Deploying daemon mds.a on smithi164 2024-09-19T08:43:38.180 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:37 smithi164 ceph-mon[30928]: mds.? [v2:172.21.15.164:6838/538276454,v1:172.21.15.164:6839/538276454] up:boot 2024-09-19T08:43:38.180 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:37 smithi164 ceph-mon[30928]: fsmap 10 up:standby 2024-09-19T08:43:38.180 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:37 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T08:43:38.180 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:37 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2582731866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:43:38.782 INFO:journalctl@ceph.mds.a.smithi164.stdout:Sep 19 08:43:38 smithi164 systemd[1]: Starting Ceph mds.a for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T08:43:39.060 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:38 smithi164 ceph-mon[30928]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:39.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:38 smithi170 ceph-mon[35072]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:39.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:38 smithi173 ceph-mon[35306]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:39.338 INFO:journalctl@ceph.mds.a.smithi164.stdout:Sep 19 08:43:39 smithi164 podman[71569]: 2024-09-19 08:43:38.964261893 +0000 UTC m=+0.016055350 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:43:39.338 INFO:journalctl@ceph.mds.a.smithi164.stdout:Sep 19 08:43:39 smithi164 podman[71569]: 2024-09-19 08:43:39.086107602 +0000 UTC m=+0.137901084 container create 29a373f4a0e40c7e1175ddeaf4ca590d6b0a554a44ecc0c1bfdff9ee2e73d1ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-a, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4) 2024-09-19T08:43:39.724 INFO:journalctl@ceph.mds.a.smithi164.stdout:Sep 19 08:43:39 smithi164 podman[71569]: 2024-09-19 08:43:39.338801604 +0000 UTC m=+0.390595045 container init 29a373f4a0e40c7e1175ddeaf4ca590d6b0a554a44ecc0c1bfdff9ee2e73d1ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-19T08:43:39.724 INFO:journalctl@ceph.mds.a.smithi164.stdout:Sep 19 08:43:39 smithi164 podman[71569]: 2024-09-19 08:43:39.341775965 +0000 UTC m=+0.393569416 container start 29a373f4a0e40c7e1175ddeaf4ca590d6b0a554a44ecc0c1bfdff9ee2e73d1ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-a, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T08:43:39.724 INFO:journalctl@ceph.mds.a.smithi164.stdout:Sep 19 08:43:39 smithi164 ceph-mds[71588]: -- 172.21.15.164:0/1498237084 <== mon.0 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556eec481860 con 0x556eec45f400 2024-09-19T08:43:39.724 INFO:journalctl@ceph.mds.a.smithi164.stdout:Sep 19 08:43:39 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-a[71584]: starting mds.a at 2024-09-19T08:43:39.724 INFO:journalctl@ceph.mds.a.smithi164.stdout:Sep 19 08:43:39 smithi164 bash[71569]: 29a373f4a0e40c7e1175ddeaf4ca590d6b0a554a44ecc0c1bfdff9ee2e73d1ff 2024-09-19T08:43:39.724 INFO:journalctl@ceph.mds.a.smithi164.stdout:Sep 19 08:43:39 smithi164 systemd[1]: Started Ceph mds.a for 1e4b0d74-7662-11ef-bceb-c7b262605968. 2024-09-19T08:43:40.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:40 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:40.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:40 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:40.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:40 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:40.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:40 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:40.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:40 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:40.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:40 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:40.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:40 smithi164 ceph-mon[30928]: Deploying daemon mds.b on smithi170 2024-09-19T08:43:40.783 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:40 smithi164 ceph-mon[30928]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:40.783 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:40 smithi164 ceph-mon[30928]: mds.? [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] up:boot 2024-09-19T08:43:40.783 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:40 smithi164 ceph-mon[30928]: fsmap 11 up:standby 2024-09-19T08:43:40.783 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:40 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T08:43:40.871 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:43:40.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:40 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:40.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:40 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:40.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:40 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:40.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:40 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:40.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:40 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:40.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:40 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:40.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:40 smithi170 ceph-mon[35072]: Deploying daemon mds.b on smithi170 2024-09-19T08:43:40.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:40 smithi170 ceph-mon[35072]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:40.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:40 smithi170 ceph-mon[35072]: mds.? [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] up:boot 2024-09-19T08:43:40.877 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:40 smithi170 ceph-mon[35072]: fsmap 11 up:standby 2024-09-19T08:43:40.877 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:40 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T08:43:40.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:40 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:40.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:40 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:40.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:40 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:40.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:40 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T08:43:40.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:40 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T08:43:40.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:40 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:40.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:40 smithi173 ceph-mon[35306]: Deploying daemon mds.b on smithi170 2024-09-19T08:43:40.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:40 smithi173 ceph-mon[35306]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:40.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:40 smithi173 ceph-mon[35306]: mds.? [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] up:boot 2024-09-19T08:43:40.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:40 smithi173 ceph-mon[35306]: fsmap 11 up:standby 2024-09-19T08:43:40.912 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:40 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T08:43:41.128 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:43:41.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.588+0000 7f1af8ba5640 1 -- 172.21.15.164:0/4075783049 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1af4101420 msgr2=0x7f1af410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:41.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.588+0000 7f1af8ba5640 1 --2- 172.21.15.164:0/4075783049 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1af4101420 0x7f1af410d950 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1aec009cd0 tx=0x7f1aec02a8e0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.588+0000 7f1af8ba5640 1 -- 172.21.15.164:0/4075783049 shutdown_connections 2024-09-19T08:43:41.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.588+0000 7f1af8ba5640 1 --2- 172.21.15.164:0/4075783049 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1af4101420 0x7f1af410d950 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.588+0000 7f1af8ba5640 1 --2- 172.21.15.164:0/4075783049 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1af4100a80 0x7f1af4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.588+0000 7f1af8ba5640 1 --2- 172.21.15.164:0/4075783049 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1af4106a80 0x7f1af4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.588+0000 7f1af8ba5640 1 -- 172.21.15.164:0/4075783049 >> 172.21.15.164:0/4075783049 conn(0x7f1af40fc7a0 msgr2=0x7f1af40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:41.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.588+0000 7f1af8ba5640 1 -- 172.21.15.164:0/4075783049 shutdown_connections 2024-09-19T08:43:41.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.588+0000 7f1af8ba5640 1 -- 172.21.15.164:0/4075783049 wait complete. 2024-09-19T08:43:41.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.589+0000 7f1af8ba5640 1 Processor -- start 2024-09-19T08:43:41.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.589+0000 7f1af8ba5640 1 -- start start 2024-09-19T08:43:41.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.589+0000 7f1af8ba5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1af4100a80 0x7f1af4102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:41.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.589+0000 7f1af8ba5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1af4101420 0x7f1af4102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:41.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.589+0000 7f1af8ba5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1af4106a80 0x7f1af41030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:41.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.589+0000 7f1af8ba5640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1af4110e80 con 0x7f1af4100a80 2024-09-19T08:43:41.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.589+0000 7f1af8ba5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1af4110d00 con 0x7f1af4106a80 2024-09-19T08:43:41.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.589+0000 7f1af8ba5640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1af4111000 con 0x7f1af4101420 2024-09-19T08:43:41.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.590+0000 7f1af259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1af4100a80 0x7f1af4102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:41.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.590+0000 7f1af259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1af4100a80 0x7f1af4102650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44364/0 (socket says 172.21.15.164:44364) 2024-09-19T08:43:41.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.590+0000 7f1af259c640 1 -- 172.21.15.164:0/3537899098 learned_addr learned my addr 172.21.15.164:0/3537899098 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:43:41.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.590+0000 7f1af2d9d640 1 --2- 172.21.15.164:0/3537899098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1af4106a80 0x7f1af41030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:41.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.590+0000 7f1af1d9b640 1 --2- 172.21.15.164:0/3537899098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1af4101420 0x7f1af4102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:41.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.590+0000 7f1af259c640 1 -- 172.21.15.164:0/3537899098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1af4101420 msgr2=0x7f1af4102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:41.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.590+0000 7f1af259c640 1 --2- 172.21.15.164:0/3537899098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1af4101420 0x7f1af4102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.590+0000 7f1af259c640 1 -- 172.21.15.164:0/3537899098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1af4106a80 msgr2=0x7f1af41030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:41.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.590+0000 7f1af259c640 1 --2- 172.21.15.164:0/3537899098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1af4106a80 0x7f1af41030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.590+0000 7f1af259c640 1 -- 172.21.15.164:0/3537899098 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af41a68f0 con 0x7f1af4100a80 2024-09-19T08:43:41.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.590+0000 7f1af1d9b640 1 --2- 172.21.15.164:0/3537899098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1af4101420 0x7f1af4102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:41.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.590+0000 7f1af259c640 1 --2- 172.21.15.164:0/3537899098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1af4100a80 0x7f1af4102650 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400c970 tx=0x7f1ae400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:41.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.590+0000 7f1aeb7fe640 1 -- 172.21.15.164:0/3537899098 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae4007bb0 con 0x7f1af4100a80 2024-09-19T08:43:41.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.590+0000 7f1aeb7fe640 1 -- 172.21.15.164:0/3537899098 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ae4007d50 con 0x7f1af4100a80 2024-09-19T08:43:41.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.590+0000 7f1af8ba5640 1 -- 172.21.15.164:0/3537899098 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1af41a6be0 con 0x7f1af4100a80 2024-09-19T08:43:41.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.590+0000 7f1aeb7fe640 1 -- 172.21.15.164:0/3537899098 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae400f450 con 0x7f1af4100a80 2024-09-19T08:43:41.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.590+0000 7f1af8ba5640 1 -- 172.21.15.164:0/3537899098 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1af41a70d0 con 0x7f1af4100a80 2024-09-19T08:43:41.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.591+0000 7f1af8ba5640 1 -- 172.21.15.164:0/3537899098 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1af4100ee0 con 0x7f1af4100a80 2024-09-19T08:43:41.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.593+0000 7f1aeb7fe640 1 -- 172.21.15.164:0/3537899098 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f1ae4004fe0 con 0x7f1af4100a80 2024-09-19T08:43:41.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.593+0000 7f1aeb7fe640 1 --2- 172.21.15.164:0/3537899098 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1ae00a3240 0x7f1ae00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:41.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.593+0000 7f1af1d9b640 1 --2- 172.21.15.164:0/3537899098 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1ae00a3240 0x7f1ae00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:41.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.593+0000 7f1aeb7fe640 1 -- 172.21.15.164:0/3537899098 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f1ae40cf580 con 0x7f1af4100a80 2024-09-19T08:43:41.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.594+0000 7f1af1d9b640 1 --2- 172.21.15.164:0/3537899098 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1ae00a3240 0x7f1ae00a5700 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1af41040e0 tx=0x7f1adc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:41.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.596+0000 7f1aeb7fe640 1 -- 172.21.15.164:0/3537899098 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1ae4014030 con 0x7f1af4100a80 2024-09-19T08:43:41.624 INFO:journalctl@ceph.mds.b.smithi170.stdout:Sep 19 08:43:41 smithi170 systemd[1]: Starting Ceph mds.b for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T08:43:41.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.799+0000 7f1af8ba5640 1 -- 172.21.15.164:0/3537899098 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1af4103b80 con 0x7f1af4100a80 2024-09-19T08:43:41.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.800+0000 7f1aeb7fe640 1 -- 172.21.15.164:0/3537899098 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9707 (secure 0 0 0) 0x7f1ae409c740 con 0x7f1af4100a80 2024-09-19T08:43:41.801 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:43:41.801 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":12,"btime":"2024-09-19T08:43:39:803205+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24418,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24442,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-19T08:43:41.801 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 12 2024-09-19T08:43:41.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.802+0000 7f1af8ba5640 1 -- 172.21.15.164:0/3537899098 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1ae00a3240 msgr2=0x7f1ae00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:41.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.802+0000 7f1af8ba5640 1 --2- 172.21.15.164:0/3537899098 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1ae00a3240 0x7f1ae00a5700 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1af41040e0 tx=0x7f1adc005f90 comp rx=0 tx=0).stop 2024-09-19T08:43:41.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.802+0000 7f1af8ba5640 1 -- 172.21.15.164:0/3537899098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1af4100a80 msgr2=0x7f1af4102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:41.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.802+0000 7f1af8ba5640 1 --2- 172.21.15.164:0/3537899098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1af4100a80 0x7f1af4102650 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400c970 tx=0x7f1ae400ce30 comp rx=0 tx=0).stop 2024-09-19T08:43:41.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.802+0000 7f1af8ba5640 1 -- 172.21.15.164:0/3537899098 shutdown_connections 2024-09-19T08:43:41.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.802+0000 7f1af8ba5640 1 --2- 172.21.15.164:0/3537899098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1af4106a80 0x7f1af41030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.802+0000 7f1af8ba5640 1 --2- 172.21.15.164:0/3537899098 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1ae00a3240 0x7f1ae00a5700 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.802+0000 7f1af8ba5640 1 --2- 172.21.15.164:0/3537899098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1af4101420 0x7f1af4102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.802+0000 7f1af8ba5640 1 --2- 172.21.15.164:0/3537899098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1af4100a80 0x7f1af4102650 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.802+0000 7f1af8ba5640 1 -- 172.21.15.164:0/3537899098 >> 172.21.15.164:0/3537899098 conn(0x7f1af40fc7a0 msgr2=0x7f1af406a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:41.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.802+0000 7f1af8ba5640 1 -- 172.21.15.164:0/3537899098 shutdown_connections 2024-09-19T08:43:41.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:41.802+0000 7f1af8ba5640 1 -- 172.21.15.164:0/3537899098 wait complete. 2024-09-19T08:43:42.045 INFO:journalctl@ceph.mds.b.smithi170.stdout:Sep 19 08:43:41 smithi170 podman[65003]: 2024-09-19 08:43:41.775154193 +0000 UTC m=+0.106665439 container create c4e1796a1740e1592f87b3b525c52b515d8d102befb2965f6c2244687a954fdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-19T08:43:42.045 INFO:journalctl@ceph.mds.b.smithi170.stdout:Sep 19 08:43:41 smithi170 podman[65003]: 2024-09-19 08:43:41.684339726 +0000 UTC m=+0.015850985 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:43:42.045 INFO:journalctl@ceph.mds.b.smithi170.stdout:Sep 19 08:43:42 smithi170 podman[65003]: 2024-09-19 08:43:42.019591015 +0000 UTC m=+0.351102261 container init c4e1796a1740e1592f87b3b525c52b515d8d102befb2965f6c2244687a954fdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-19T08:43:42.045 INFO:journalctl@ceph.mds.b.smithi170.stdout:Sep 19 08:43:42 smithi170 podman[65003]: 2024-09-19 08:43:42.022478908 +0000 UTC m=+0.353990155 container start c4e1796a1740e1592f87b3b525c52b515d8d102befb2965f6c2244687a954fdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4) 2024-09-19T08:43:42.375 INFO:journalctl@ceph.mds.b.smithi170.stdout:Sep 19 08:43:42 smithi170 ceph-mds[65022]: -- 172.21.15.170:0/1939840257 <== mon.2 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55644619f860 con 0x55644617c800 2024-09-19T08:43:42.375 INFO:journalctl@ceph.mds.b.smithi170.stdout:Sep 19 08:43:42 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-b[65018]: starting mds.b at 2024-09-19T08:43:42.375 INFO:journalctl@ceph.mds.b.smithi170.stdout:Sep 19 08:43:42 smithi170 bash[65003]: c4e1796a1740e1592f87b3b525c52b515d8d102befb2965f6c2244687a954fdc 2024-09-19T08:43:42.375 INFO:journalctl@ceph.mds.b.smithi170.stdout:Sep 19 08:43:42 smithi170 systemd[1]: Started Ceph mds.b for 1e4b0d74-7662-11ef-bceb-c7b262605968. 2024-09-19T08:43:43.047 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:42 smithi170 ceph-mon[35072]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:43.047 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:42 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3537899098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:43:43.047 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:42 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:43.048 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:42 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:43.048 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:42 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:43.048 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:42 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:43.048 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:42 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:43.048 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:42 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:43:43.129 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:42 smithi173 ceph-mon[35306]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:43.129 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:42 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3537899098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:43:43.129 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:42 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:43.129 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:42 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:43.129 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:42 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:43.129 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:42 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:43.130 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:42 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:43.130 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:42 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:43:43.144 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:42 smithi164 ceph-mon[30928]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:43.144 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:42 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3537899098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:43:43.144 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:42 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:43.144 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:42 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:43.144 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:42 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:43.144 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:42 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:43.144 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:42 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:43.145 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:42 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:43:44.011 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:43 smithi173 ceph-mon[35306]: mds.? [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] up:boot 2024-09-19T08:43:44.011 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:43 smithi173 ceph-mon[35306]: fsmap 12 up:standby 2024-09-19T08:43:44.011 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:43 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T08:43:44.011 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:43 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:44.011 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:43 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:44.011 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:43 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:44.011 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:43 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:44.012 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:43 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:44.012 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:43 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:44.026 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:43 smithi170 ceph-mon[35072]: mds.? [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] up:boot 2024-09-19T08:43:44.026 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:43 smithi170 ceph-mon[35072]: fsmap 12 up:standby 2024-09-19T08:43:44.026 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:43 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T08:43:44.026 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:43 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:44.026 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:43 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:44.026 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:43 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:44.026 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:43 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:44.027 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:43 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:44.027 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:43 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:44.281 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:43 smithi164 ceph-mon[30928]: mds.? [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] up:boot 2024-09-19T08:43:44.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:43 smithi164 ceph-mon[30928]: fsmap 12 up:standby 2024-09-19T08:43:44.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:43 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T08:43:44.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:43 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:44.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:43 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:44.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:43 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:44.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:43 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:44.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:43 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:44.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:43 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:44.485 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:43:45.069 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:44 smithi170 ceph-mon[35072]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:45.070 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:44 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:45.070 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:44 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:43:45.070 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:44 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:45.070 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:44 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:43:45.102 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:44 smithi173 ceph-mon[35306]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:45.102 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:44 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:45.102 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:44 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:43:45.102 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:44 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:45.102 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:44 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:43:45.104 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:44 smithi164 ceph-mon[30928]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:45.104 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:44 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:45.104 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:44 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:43:45.105 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:44 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:45.105 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:44 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:43:45.405 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:43:45.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.862+0000 7f4e35f44640 1 -- 172.21.15.164:0/3784663366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e30106a80 msgr2=0x7f4e30106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:45.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.862+0000 7f4e35f44640 1 --2- 172.21.15.164:0/3784663366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e30106a80 0x7f4e30106e60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4e20009930 tx=0x7f4e2002a420 comp rx=0 tx=0).stop 2024-09-19T08:43:45.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.862+0000 7f4e35f44640 1 -- 172.21.15.164:0/3784663366 shutdown_connections 2024-09-19T08:43:45.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.862+0000 7f4e35f44640 1 --2- 172.21.15.164:0/3784663366 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e30101420 0x7f4e3010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:45.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.862+0000 7f4e35f44640 1 --2- 172.21.15.164:0/3784663366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e30100a80 0x7f4e30100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:45.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.862+0000 7f4e35f44640 1 --2- 172.21.15.164:0/3784663366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e30106a80 0x7f4e30106e60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:45.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.862+0000 7f4e35f44640 1 -- 172.21.15.164:0/3784663366 >> 172.21.15.164:0/3784663366 conn(0x7f4e300fc7a0 msgr2=0x7f4e300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:45.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.862+0000 7f4e35f44640 1 -- 172.21.15.164:0/3784663366 shutdown_connections 2024-09-19T08:43:45.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.862+0000 7f4e35f44640 1 -- 172.21.15.164:0/3784663366 wait complete. 2024-09-19T08:43:45.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.863+0000 7f4e35f44640 1 Processor -- start 2024-09-19T08:43:45.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.863+0000 7f4e35f44640 1 -- start start 2024-09-19T08:43:45.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.863+0000 7f4e35f44640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e30100a80 0x7f4e301992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:45.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.863+0000 7f4e35f44640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e30101420 0x7f4e30199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:45.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.863+0000 7f4e35f44640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e30106a80 0x7f4e301a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:45.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.863+0000 7f4e35f44640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4e30110e80 con 0x7f4e30106a80 2024-09-19T08:43:45.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.863+0000 7f4e35f44640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4e30110d00 con 0x7f4e30101420 2024-09-19T08:43:45.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.863+0000 7f4e35f44640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4e30111000 con 0x7f4e30100a80 2024-09-19T08:43:45.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.863+0000 7f4e2ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e30106a80 0x7f4e301a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:45.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.863+0000 7f4e2ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e30106a80 0x7f4e301a4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51870/0 (socket says 172.21.15.164:51870) 2024-09-19T08:43:45.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.863+0000 7f4e2ffff640 1 -- 172.21.15.164:0/4244466665 learned_addr learned my addr 172.21.15.164:0/4244466665 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:43:45.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.863+0000 7f4e2f7fe640 1 --2- 172.21.15.164:0/4244466665 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e30100a80 0x7f4e301992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:45.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.863+0000 7f4e27fff640 1 --2- 172.21.15.164:0/4244466665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e30101420 0x7f4e30199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:45.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.863+0000 7f4e2ffff640 1 -- 172.21.15.164:0/4244466665 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e30100a80 msgr2=0x7f4e301992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:45.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.863+0000 7f4e2ffff640 1 --2- 172.21.15.164:0/4244466665 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e30100a80 0x7f4e301992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:45.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.863+0000 7f4e2ffff640 1 -- 172.21.15.164:0/4244466665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e30101420 msgr2=0x7f4e30199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:45.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.863+0000 7f4e2ffff640 1 --2- 172.21.15.164:0/4244466665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e30101420 0x7f4e30199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:45.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.863+0000 7f4e2ffff640 1 -- 172.21.15.164:0/4244466665 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e301a48b0 con 0x7f4e30106a80 2024-09-19T08:43:45.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.864+0000 7f4e2ffff640 1 --2- 172.21.15.164:0/4244466665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e30106a80 0x7f4e301a4370 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009fd0 tx=0x7f4e2800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:45.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.864+0000 7f4e2d7fa640 1 -- 172.21.15.164:0/4244466665 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e28017070 con 0x7f4e30106a80 2024-09-19T08:43:45.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.864+0000 7f4e2d7fa640 1 -- 172.21.15.164:0/4244466665 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e28003e50 con 0x7f4e30106a80 2024-09-19T08:43:45.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.864+0000 7f4e35f44640 1 -- 172.21.15.164:0/4244466665 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e301a4a80 con 0x7f4e30106a80 2024-09-19T08:43:45.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.864+0000 7f4e2d7fa640 1 -- 172.21.15.164:0/4244466665 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e280106c0 con 0x7f4e30106a80 2024-09-19T08:43:45.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.864+0000 7f4e35f44640 1 -- 172.21.15.164:0/4244466665 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e301a4f40 con 0x7f4e30106a80 2024-09-19T08:43:45.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.865+0000 7f4e35f44640 1 -- 172.21.15.164:0/4244466665 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e30100ee0 con 0x7f4e30106a80 2024-09-19T08:43:45.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.866+0000 7f4e2d7fa640 1 -- 172.21.15.164:0/4244466665 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f4e28003970 con 0x7f4e30106a80 2024-09-19T08:43:45.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.866+0000 7f4e2d7fa640 1 --2- 172.21.15.164:0/4244466665 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f4e1c0a3240 0x7f4e1c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:45.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.867+0000 7f4e2f7fe640 1 --2- 172.21.15.164:0/4244466665 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f4e1c0a3240 0x7f4e1c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:45.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.867+0000 7f4e2d7fa640 1 -- 172.21.15.164:0/4244466665 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f4e280cf130 con 0x7f4e30106a80 2024-09-19T08:43:45.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.867+0000 7f4e2f7fe640 1 --2- 172.21.15.164:0/4244466665 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f4e1c0a3240 0x7f4e1c0a5700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4e20009900 tx=0x7f4e200058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:45.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:45.871+0000 7f4e2d7fa640 1 -- 172.21.15.164:0/4244466665 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4e2809bca0 con 0x7f4e30106a80 2024-09-19T08:43:46.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:46.105+0000 7f4e35f44640 1 -- 172.21.15.164:0/4244466665 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e30108ff0 con 0x7f4e30106a80 2024-09-19T08:43:46.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:46.106+0000 7f4e2d7fa640 1 -- 172.21.15.164:0/4244466665 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10536 (secure 0 0 0) 0x7f4e2809ffa0 con 0x7f4e30106a80 2024-09-19T08:43:46.107 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:43:46.107 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":13,"btime":"2024-09-19T08:43:42:792134+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24418,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24442,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-19T08:43:46.107 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 13 2024-09-19T08:43:46.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:46.108+0000 7f4e35f44640 1 -- 172.21.15.164:0/4244466665 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f4e1c0a3240 msgr2=0x7f4e1c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:46.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:46.108+0000 7f4e35f44640 1 --2- 172.21.15.164:0/4244466665 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f4e1c0a3240 0x7f4e1c0a5700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4e20009900 tx=0x7f4e200058d0 comp rx=0 tx=0).stop 2024-09-19T08:43:46.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:46.108+0000 7f4e35f44640 1 -- 172.21.15.164:0/4244466665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e30106a80 msgr2=0x7f4e301a4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:46.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:46.108+0000 7f4e35f44640 1 --2- 172.21.15.164:0/4244466665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e30106a80 0x7f4e301a4370 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009fd0 tx=0x7f4e2800b260 comp rx=0 tx=0).stop 2024-09-19T08:43:46.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:46.108+0000 7f4e35f44640 1 -- 172.21.15.164:0/4244466665 shutdown_connections 2024-09-19T08:43:46.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:46.108+0000 7f4e35f44640 1 --2- 172.21.15.164:0/4244466665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e30106a80 0x7f4e301a4370 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:46.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:46.108+0000 7f4e35f44640 1 --2- 172.21.15.164:0/4244466665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e30101420 0x7f4e30199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:46.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:46.108+0000 7f4e35f44640 1 --2- 172.21.15.164:0/4244466665 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f4e1c0a3240 0x7f4e1c0a5700 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:46.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:46.108+0000 7f4e35f44640 1 --2- 172.21.15.164:0/4244466665 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e30100a80 0x7f4e301992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:46.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:46.108+0000 7f4e35f44640 1 -- 172.21.15.164:0/4244466665 >> 172.21.15.164:0/4244466665 conn(0x7f4e300fc7a0 msgr2=0x7f4e300fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:46.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:46.108+0000 7f4e35f44640 1 -- 172.21.15.164:0/4244466665 shutdown_connections 2024-09-19T08:43:46.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:46.109+0000 7f4e35f44640 1 -- 172.21.15.164:0/4244466665 wait complete. 2024-09-19T08:43:46.897 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:46 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:46.897 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:46 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:46.897 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:46 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:46.897 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:46 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:46.897 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:46 smithi164 ceph-mon[30928]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:46.897 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:46 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/4244466665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:43:46.897 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:46 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:46.898 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:46 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:46.898 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:46 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:46.898 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:46 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:43:46.898 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:46 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:46.924 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-19T08:43:47.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:46 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:47.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:46 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:47.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:46 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:47.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:46 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:47.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:46 smithi170 ceph-mon[35072]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:47.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:46 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/4244466665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:43:47.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:46 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:47.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:46 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:47.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:46 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:47.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:46 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:43:47.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:46 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:47.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:46 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:47.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:46 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:47.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:46 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:47.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:46 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:47.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:46 smithi173 ceph-mon[35306]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:47.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:46 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/4244466665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:43:47.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:46 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:47.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:46 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:47.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:46 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:43:47.163 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:46 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:43:47.163 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:46 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:47.167 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:43:47.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.639+0000 7fd665bc0640 1 -- 172.21.15.164:0/323669438 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd660101df0 msgr2=0x7fd6601021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:47.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.639+0000 7fd665bc0640 1 --2- 172.21.15.164:0/323669438 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd660101df0 0x7fd6601021d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd650009900 tx=0x7fd65002a410 comp rx=0 tx=0).stop 2024-09-19T08:43:47.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.639+0000 7fd665bc0640 1 -- 172.21.15.164:0/323669438 shutdown_connections 2024-09-19T08:43:47.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.639+0000 7fd665bc0640 1 --2- 172.21.15.164:0/323669438 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd660106f20 0x7fd66010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.639+0000 7fd665bc0640 1 --2- 172.21.15.164:0/323669438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6601027a0 0x7fd6601067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.639+0000 7fd665bc0640 1 --2- 172.21.15.164:0/323669438 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd660101df0 0x7fd6601021d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.639+0000 7fd665bc0640 1 -- 172.21.15.164:0/323669438 >> 172.21.15.164:0/323669438 conn(0x7fd6600fd8f0 msgr2=0x7fd6600ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:47.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.640+0000 7fd665bc0640 1 -- 172.21.15.164:0/323669438 shutdown_connections 2024-09-19T08:43:47.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.640+0000 7fd665bc0640 1 -- 172.21.15.164:0/323669438 wait complete. 2024-09-19T08:43:47.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.640+0000 7fd665bc0640 1 Processor -- start 2024-09-19T08:43:47.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.640+0000 7fd665bc0640 1 -- start start 2024-09-19T08:43:47.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.640+0000 7fd665bc0640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd660101df0 0x7fd660198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:47.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.640+0000 7fd665bc0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6601027a0 0x7fd660198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:47.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.640+0000 7fd665bc0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd660106f20 0x7fd66019ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:47.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.640+0000 7fd665bc0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd660112020 con 0x7fd660106f20 2024-09-19T08:43:47.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.640+0000 7fd665bc0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd660111ea0 con 0x7fd6601027a0 2024-09-19T08:43:47.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.640+0000 7fd665bc0640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd6601121a0 con 0x7fd660101df0 2024-09-19T08:43:47.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.641+0000 7fd65ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd660106f20 0x7fd66019ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:47.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.641+0000 7fd65ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd660106f20 0x7fd66019ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51892/0 (socket says 172.21.15.164:51892) 2024-09-19T08:43:47.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.641+0000 7fd65ffff640 1 -- 172.21.15.164:0/2543109045 learned_addr learned my addr 172.21.15.164:0/2543109045 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:43:47.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.641+0000 7fd65f7fe640 1 --2- 172.21.15.164:0/2543109045 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd660101df0 0x7fd660198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:47.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.641+0000 7fd65effd640 1 --2- 172.21.15.164:0/2543109045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6601027a0 0x7fd660198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:47.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.641+0000 7fd65ffff640 1 -- 172.21.15.164:0/2543109045 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd660101df0 msgr2=0x7fd660198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:47.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.641+0000 7fd65ffff640 1 --2- 172.21.15.164:0/2543109045 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd660101df0 0x7fd660198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.641+0000 7fd65ffff640 1 -- 172.21.15.164:0/2543109045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6601027a0 msgr2=0x7fd660198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:47.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.641+0000 7fd65ffff640 1 --2- 172.21.15.164:0/2543109045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6601027a0 0x7fd660198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.641+0000 7fd65ffff640 1 -- 172.21.15.164:0/2543109045 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd66019d530 con 0x7fd660106f20 2024-09-19T08:43:47.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.641+0000 7fd65ffff640 1 --2- 172.21.15.164:0/2543109045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd660106f20 0x7fd66019ce50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd65800b970 tx=0x7fd65800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:47.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.641+0000 7fd65cff9640 1 -- 172.21.15.164:0/2543109045 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd658019070 con 0x7fd660106f20 2024-09-19T08:43:47.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.641+0000 7fd65cff9640 1 -- 172.21.15.164:0/2543109045 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd65800cb40 con 0x7fd660106f20 2024-09-19T08:43:47.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.641+0000 7fd665bc0640 1 -- 172.21.15.164:0/2543109045 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd66019d820 con 0x7fd660106f20 2024-09-19T08:43:47.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.641+0000 7fd65cff9640 1 -- 172.21.15.164:0/2543109045 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd658007650 con 0x7fd660106f20 2024-09-19T08:43:47.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.641+0000 7fd665bc0640 1 -- 172.21.15.164:0/2543109045 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6601a5100 con 0x7fd660106f20 2024-09-19T08:43:47.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.642+0000 7fd665bc0640 1 -- 172.21.15.164:0/2543109045 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd660102250 con 0x7fd660106f20 2024-09-19T08:43:47.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.643+0000 7fd65cff9640 1 -- 172.21.15.164:0/2543109045 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fd65801c070 con 0x7fd660106f20 2024-09-19T08:43:47.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.644+0000 7fd65cff9640 1 --2- 172.21.15.164:0/2543109045 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fd6480a3240 0x7fd6480a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:47.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.644+0000 7fd65f7fe640 1 --2- 172.21.15.164:0/2543109045 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fd6480a3240 0x7fd6480a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:47.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.644+0000 7fd65cff9640 1 -- 172.21.15.164:0/2543109045 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fd6580ceb10 con 0x7fd660106f20 2024-09-19T08:43:47.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.644+0000 7fd65f7fe640 1 --2- 172.21.15.164:0/2543109045 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fd6480a3240 0x7fd6480a5700 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd6500096c0 tx=0x7fd650031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:47.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.648+0000 7fd65cff9640 1 -- 172.21.15.164:0/2543109045 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd65809bcd0 con 0x7fd660106f20 2024-09-19T08:43:47.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.819+0000 7fd665bc0640 1 -- 172.21.15.164:0/2543109045 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd660109b70 con 0x7fd660106f20 2024-09-19T08:43:47.821 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:47 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:47.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.820+0000 7fd65cff9640 1 -- 172.21.15.164:0/2543109045 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v75) ==== 84+0+65 (secure 0 0 0) 0x7fd65809b5d0 con 0x7fd660106f20 2024-09-19T08:43:47.822 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:43:47.822 INFO:teuthology.orchestra.run.smithi164.stdout:[ 2024-09-19T08:43:47.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:43:47.823 INFO:teuthology.orchestra.run.smithi164.stdout: "poolnum": 1, 2024-09-19T08:43:47.823 INFO:teuthology.orchestra.run.smithi164.stdout: "poolname": ".mgr" 2024-09-19T08:43:47.823 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:43:47.823 INFO:teuthology.orchestra.run.smithi164.stdout:] 2024-09-19T08:43:47.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.822+0000 7fd665bc0640 1 -- 172.21.15.164:0/2543109045 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fd6480a3240 msgr2=0x7fd6480a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:47.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.822+0000 7fd665bc0640 1 --2- 172.21.15.164:0/2543109045 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fd6480a3240 0x7fd6480a5700 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd6500096c0 tx=0x7fd650031490 comp rx=0 tx=0).stop 2024-09-19T08:43:47.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.822+0000 7fd665bc0640 1 -- 172.21.15.164:0/2543109045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd660106f20 msgr2=0x7fd66019ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:47.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.822+0000 7fd665bc0640 1 --2- 172.21.15.164:0/2543109045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd660106f20 0x7fd66019ce50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd65800b970 tx=0x7fd65800be30 comp rx=0 tx=0).stop 2024-09-19T08:43:47.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.822+0000 7fd665bc0640 1 -- 172.21.15.164:0/2543109045 shutdown_connections 2024-09-19T08:43:47.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.822+0000 7fd665bc0640 1 --2- 172.21.15.164:0/2543109045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd660106f20 0x7fd66019ce50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.822+0000 7fd665bc0640 1 --2- 172.21.15.164:0/2543109045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6601027a0 0x7fd660198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.822+0000 7fd665bc0640 1 --2- 172.21.15.164:0/2543109045 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fd6480a3240 0x7fd6480a5700 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.822+0000 7fd665bc0640 1 --2- 172.21.15.164:0/2543109045 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd660101df0 0x7fd660198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.822+0000 7fd665bc0640 1 -- 172.21.15.164:0/2543109045 >> 172.21.15.164:0/2543109045 conn(0x7fd6600fd8f0 msgr2=0x7fd6600fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:47.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.822+0000 7fd665bc0640 1 -- 172.21.15.164:0/2543109045 shutdown_connections 2024-09-19T08:43:47.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:47.822+0000 7fd665bc0640 1 -- 172.21.15.164:0/2543109045 wait complete. 2024-09-19T08:43:48.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:47 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:48.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:47 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:43:48.500 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T08:43:48.500 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T08:43:48.747 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:43:48.989 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:48 smithi164 ceph-mon[30928]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:48.989 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:48 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2543109045' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T08:43:49.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:48 smithi170 ceph-mon[35072]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:49.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:48 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2543109045' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T08:43:49.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:48 smithi173 ceph-mon[35306]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:49.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:48 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2543109045' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T08:43:49.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.202+0000 7fccd6934640 1 -- 172.21.15.164:0/1707524750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccd0102790 msgr2=0x7fccd01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:49.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.202+0000 7fccd6934640 1 --2- 172.21.15.164:0/1707524750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccd0102790 0x7fccd01067e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009900 tx=0x7fccb802a410 comp rx=0 tx=0).stop 2024-09-19T08:43:49.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.202+0000 7fccd6934640 1 -- 172.21.15.164:0/1707524750 shutdown_connections 2024-09-19T08:43:49.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.202+0000 7fccd6934640 1 --2- 172.21.15.164:0/1707524750 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fccd0106f10 0x7fccd010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:49.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.202+0000 7fccd6934640 1 --2- 172.21.15.164:0/1707524750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccd0102790 0x7fccd01067e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:49.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.202+0000 7fccd6934640 1 --2- 172.21.15.164:0/1707524750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccd0101de0 0x7fccd01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:49.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.202+0000 7fccd6934640 1 -- 172.21.15.164:0/1707524750 >> 172.21.15.164:0/1707524750 conn(0x7fccd00fd8c0 msgr2=0x7fccd00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:49.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.202+0000 7fccd6934640 1 -- 172.21.15.164:0/1707524750 shutdown_connections 2024-09-19T08:43:49.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.202+0000 7fccd6934640 1 -- 172.21.15.164:0/1707524750 wait complete. 2024-09-19T08:43:49.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.202+0000 7fccd6934640 1 Processor -- start 2024-09-19T08:43:49.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.202+0000 7fccd6934640 1 -- start start 2024-09-19T08:43:49.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.203+0000 7fccd6934640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccd0101de0 0x7fccd0198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:49.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.203+0000 7fccd6934640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccd0102790 0x7fccd01988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:49.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.203+0000 7fccd6934640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fccd0106f10 0x7fccd019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:49.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.203+0000 7fccd6934640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fccd0111f70 con 0x7fccd0101de0 2024-09-19T08:43:49.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.203+0000 7fccd6934640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fccd0111df0 con 0x7fccd0102790 2024-09-19T08:43:49.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.203+0000 7fccd6934640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fccd01120f0 con 0x7fccd0106f10 2024-09-19T08:43:49.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.203+0000 7fcccffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccd0101de0 0x7fccd0198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:49.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.203+0000 7fcccffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccd0101de0 0x7fccd0198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51914/0 (socket says 172.21.15.164:51914) 2024-09-19T08:43:49.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.203+0000 7fcccffff640 1 -- 172.21.15.164:0/21400286 learned_addr learned my addr 172.21.15.164:0/21400286 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:43:49.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.203+0000 7fccc7fff640 1 --2- 172.21.15.164:0/21400286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccd0102790 0x7fccd01988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:49.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.203+0000 7fccd4ed1640 1 --2- 172.21.15.164:0/21400286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fccd0106f10 0x7fccd019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:49.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.203+0000 7fcccffff640 1 -- 172.21.15.164:0/21400286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fccd0106f10 msgr2=0x7fccd019cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:49.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.203+0000 7fcccffff640 1 --2- 172.21.15.164:0/21400286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fccd0106f10 0x7fccd019cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:49.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.203+0000 7fcccffff640 1 -- 172.21.15.164:0/21400286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccd0102790 msgr2=0x7fccd01988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:49.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.203+0000 7fcccffff640 1 --2- 172.21.15.164:0/21400286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccd0102790 0x7fccd01988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:49.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.203+0000 7fcccffff640 1 -- 172.21.15.164:0/21400286 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd019d310 con 0x7fccd0101de0 2024-09-19T08:43:49.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.203+0000 7fcccffff640 1 --2- 172.21.15.164:0/21400286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccd0101de0 0x7fccd0198360 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fccc000c920 tx=0x7fccc000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:49.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.203+0000 7fcccdffb640 1 -- 172.21.15.164:0/21400286 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0007c70 con 0x7fccd0101de0 2024-09-19T08:43:49.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.203+0000 7fcccdffb640 1 -- 172.21.15.164:0/21400286 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccc0007e10 con 0x7fccd0101de0 2024-09-19T08:43:49.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.203+0000 7fcccdffb640 1 -- 172.21.15.164:0/21400286 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0018890 con 0x7fccd0101de0 2024-09-19T08:43:49.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.203+0000 7fccd6934640 1 -- 172.21.15.164:0/21400286 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd019d600 con 0x7fccd0101de0 2024-09-19T08:43:49.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.204+0000 7fccd6934640 1 -- 172.21.15.164:0/21400286 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd01a4ee0 con 0x7fccd0101de0 2024-09-19T08:43:49.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.204+0000 7fccd6934640 1 -- 172.21.15.164:0/21400286 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccd01021c0 con 0x7fccd0101de0 2024-09-19T08:43:49.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.205+0000 7fcccdffb640 1 -- 172.21.15.164:0/21400286 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fccc0018a70 con 0x7fccd0101de0 2024-09-19T08:43:49.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.206+0000 7fcccdffb640 1 --2- 172.21.15.164:0/21400286 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fccbc0a3240 0x7fccbc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:49.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.206+0000 7fccc7fff640 1 --2- 172.21.15.164:0/21400286 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fccbc0a3240 0x7fccbc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:49.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.206+0000 7fcccdffb640 1 -- 172.21.15.164:0/21400286 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fccc00cf670 con 0x7fccd0101de0 2024-09-19T08:43:49.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.207+0000 7fccc7fff640 1 --2- 172.21.15.164:0/21400286 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fccbc0a3240 0x7fccbc0a5700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fccd00fdce0 tx=0x7fccb8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:49.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.210+0000 7fcccdffb640 1 -- 172.21.15.164:0/21400286 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fccc0014030 con 0x7fccd0101de0 2024-09-19T08:43:49.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.379+0000 7fccd6934640 1 -- 172.21.15.164:0/21400286 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fccd0109b60 con 0x7fccd0101de0 2024-09-19T08:43:49.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.730+0000 7fcccdffb640 1 -- 172.21.15.164:0/21400286 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v76) ==== 138+0+0 (secure 0 0 0) 0x7fccc000de20 con 0x7fccd0101de0 2024-09-19T08:43:49.731 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_metadata' created 2024-09-19T08:43:49.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.732+0000 7fccd6934640 1 -- 172.21.15.164:0/21400286 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fccbc0a3240 msgr2=0x7fccbc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:49.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.732+0000 7fccd6934640 1 --2- 172.21.15.164:0/21400286 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fccbc0a3240 0x7fccbc0a5700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fccd00fdce0 tx=0x7fccb8005a20 comp rx=0 tx=0).stop 2024-09-19T08:43:49.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.732+0000 7fccd6934640 1 -- 172.21.15.164:0/21400286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccd0101de0 msgr2=0x7fccd0198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:49.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.732+0000 7fccd6934640 1 --2- 172.21.15.164:0/21400286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccd0101de0 0x7fccd0198360 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fccc000c920 tx=0x7fccc000cde0 comp rx=0 tx=0).stop 2024-09-19T08:43:49.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.733+0000 7fccd6934640 1 -- 172.21.15.164:0/21400286 shutdown_connections 2024-09-19T08:43:49.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.733+0000 7fccd6934640 1 --2- 172.21.15.164:0/21400286 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fccbc0a3240 0x7fccbc0a5700 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:49.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.733+0000 7fccd6934640 1 --2- 172.21.15.164:0/21400286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fccd0106f10 0x7fccd019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:49.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.733+0000 7fccd6934640 1 --2- 172.21.15.164:0/21400286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccd0102790 0x7fccd01988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:49.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.733+0000 7fccd6934640 1 --2- 172.21.15.164:0/21400286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccd0101de0 0x7fccd0198360 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:49.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.733+0000 7fccd6934640 1 -- 172.21.15.164:0/21400286 >> 172.21.15.164:0/21400286 conn(0x7fccd00fd8c0 msgr2=0x7fccd00fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:49.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.733+0000 7fccd6934640 1 -- 172.21.15.164:0/21400286 shutdown_connections 2024-09-19T08:43:49.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:49.733+0000 7fccd6934640 1 -- 172.21.15.164:0/21400286 wait complete. 2024-09-19T08:43:50.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:49 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/21400286' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T08:43:50.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:49 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/21400286' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T08:43:50.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:49 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/21400286' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T08:43:50.425 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T08:43:50.669 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:43:51.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:50 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/21400286' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T08:43:51.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:50 smithi164 ceph-mon[30928]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T08:43:51.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:50 smithi164 ceph-mon[30928]: pgmap v270: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:51.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:50 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/21400286' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T08:43:51.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:50 smithi170 ceph-mon[35072]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T08:43:51.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:50 smithi170 ceph-mon[35072]: pgmap v270: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:51.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:50 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/21400286' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T08:43:51.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:50 smithi173 ceph-mon[35306]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T08:43:51.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:50 smithi173 ceph-mon[35306]: pgmap v270: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:51.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.161+0000 7fa3bc6af640 1 -- 172.21.15.164:0/886991473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3b4100a80 msgr2=0x7fa3b4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:51.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.161+0000 7fa3bc6af640 1 --2- 172.21.15.164:0/886991473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3b4100a80 0x7fa3b4100ee0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4009930 tx=0x7fa3a402a420 comp rx=0 tx=0).stop 2024-09-19T08:43:51.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.162+0000 7fa3bc6af640 1 -- 172.21.15.164:0/886991473 shutdown_connections 2024-09-19T08:43:51.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.162+0000 7fa3bc6af640 1 --2- 172.21.15.164:0/886991473 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3b4101420 0x7fa3b410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.162+0000 7fa3bc6af640 1 --2- 172.21.15.164:0/886991473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3b4100a80 0x7fa3b4100ee0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.162+0000 7fa3bc6af640 1 --2- 172.21.15.164:0/886991473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3b4106a80 0x7fa3b4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.162+0000 7fa3bc6af640 1 -- 172.21.15.164:0/886991473 >> 172.21.15.164:0/886991473 conn(0x7fa3b40fc7a0 msgr2=0x7fa3b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:51.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.162+0000 7fa3bc6af640 1 -- 172.21.15.164:0/886991473 shutdown_connections 2024-09-19T08:43:51.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.162+0000 7fa3bc6af640 1 -- 172.21.15.164:0/886991473 wait complete. 2024-09-19T08:43:51.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.162+0000 7fa3bc6af640 1 Processor -- start 2024-09-19T08:43:51.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.162+0000 7fa3bc6af640 1 -- start start 2024-09-19T08:43:51.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.162+0000 7fa3bc6af640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3b4100a80 0x7fa3b4199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:51.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.162+0000 7fa3bc6af640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3b4101420 0x7fa3b41998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:51.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.162+0000 7fa3bc6af640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3b4106a80 0x7fa3b41a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:51.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.162+0000 7fa3bc6af640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b4110ea0 con 0x7fa3b4101420 2024-09-19T08:43:51.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.162+0000 7fa3bc6af640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b4110d20 con 0x7fa3b4100a80 2024-09-19T08:43:51.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.162+0000 7fa3bc6af640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b4111020 con 0x7fa3b4106a80 2024-09-19T08:43:51.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.162+0000 7fa3b9c4a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3b4101420 0x7fa3b41998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:51.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.162+0000 7fa3b9c4a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3b4101420 0x7fa3b41998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51938/0 (socket says 172.21.15.164:51938) 2024-09-19T08:43:51.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.162+0000 7fa3b9c4a640 1 -- 172.21.15.164:0/2460143201 learned_addr learned my addr 172.21.15.164:0/2460143201 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:43:51.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.163+0000 7fa3ba44b640 1 --2- 172.21.15.164:0/2460143201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3b4100a80 0x7fa3b4199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:51.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.163+0000 7fa3b9c4a640 1 -- 172.21.15.164:0/2460143201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3b4106a80 msgr2=0x7fa3b41a43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:51.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.163+0000 7fa3bac4c640 1 --2- 172.21.15.164:0/2460143201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3b4106a80 0x7fa3b41a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:51.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.163+0000 7fa3b9c4a640 1 --2- 172.21.15.164:0/2460143201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3b4106a80 0x7fa3b41a43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.163+0000 7fa3b9c4a640 1 -- 172.21.15.164:0/2460143201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3b4100a80 msgr2=0x7fa3b4199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:51.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.163+0000 7fa3b9c4a640 1 --2- 172.21.15.164:0/2460143201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3b4100a80 0x7fa3b4199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.163+0000 7fa3b9c4a640 1 -- 172.21.15.164:0/2460143201 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3b41a4930 con 0x7fa3b4101420 2024-09-19T08:43:51.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.163+0000 7fa3ba44b640 1 --2- 172.21.15.164:0/2460143201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3b4100a80 0x7fa3b4199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:43:51.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.163+0000 7fa3b9c4a640 1 --2- 172.21.15.164:0/2460143201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3b4101420 0x7fa3b41998b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4009a50 tx=0x7fa3a4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:51.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.163+0000 7fa3ab7fe640 1 -- 172.21.15.164:0/2460143201 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a403f030 con 0x7fa3b4101420 2024-09-19T08:43:51.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.163+0000 7fa3bc6af640 1 -- 172.21.15.164:0/2460143201 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3b41a4b00 con 0x7fa3b4101420 2024-09-19T08:43:51.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.163+0000 7fa3ab7fe640 1 -- 172.21.15.164:0/2460143201 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3a4003ce0 con 0x7fa3b4101420 2024-09-19T08:43:51.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.163+0000 7fa3ab7fe640 1 -- 172.21.15.164:0/2460143201 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4035070 con 0x7fa3b4101420 2024-09-19T08:43:51.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.163+0000 7fa3bc6af640 1 -- 172.21.15.164:0/2460143201 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3b41a4fb0 con 0x7fa3b4101420 2024-09-19T08:43:51.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.164+0000 7fa3bc6af640 1 -- 172.21.15.164:0/2460143201 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3b4100ee0 con 0x7fa3b4101420 2024-09-19T08:43:51.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.166+0000 7fa3ab7fe640 1 -- 172.21.15.164:0/2460143201 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fa3a4042030 con 0x7fa3b4101420 2024-09-19T08:43:51.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.166+0000 7fa3ab7fe640 1 --2- 172.21.15.164:0/2460143201 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fa3a00a3240 0x7fa3a00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:51.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.166+0000 7fa3ba44b640 1 --2- 172.21.15.164:0/2460143201 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fa3a00a3240 0x7fa3a00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:51.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.167+0000 7fa3ab7fe640 1 -- 172.21.15.164:0/2460143201 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7497+0+0 (secure 0 0 0) 0x7fa3a40f0080 con 0x7fa3b4101420 2024-09-19T08:43:51.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.167+0000 7fa3ba44b640 1 --2- 172.21.15.164:0/2460143201 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fa3a00a3240 0x7fa3a00a5700 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00058c0 tx=0x7fa3b0005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:51.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.169+0000 7fa3ab7fe640 1 -- 172.21.15.164:0/2460143201 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa3a40bd0d0 con 0x7fa3b4101420 2024-09-19T08:43:51.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.338+0000 7fa3bc6af640 1 -- 172.21.15.164:0/2460143201 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa3b4109010 con 0x7fa3b4101420 2024-09-19T08:43:51.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.741+0000 7fa3ab7fe640 1 -- 172.21.15.164:0/2460143201 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v78) ==== 170+0+0 (secure 0 0 0) 0x7fa3a40bc9d0 con 0x7fa3b4101420 2024-09-19T08:43:51.742 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_data' created 2024-09-19T08:43:51.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.745+0000 7fa3bc6af640 1 -- 172.21.15.164:0/2460143201 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fa3a00a3240 msgr2=0x7fa3a00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:51.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.745+0000 7fa3bc6af640 1 --2- 172.21.15.164:0/2460143201 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fa3a00a3240 0x7fa3a00a5700 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00058c0 tx=0x7fa3b0005ec0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.745+0000 7fa3bc6af640 1 -- 172.21.15.164:0/2460143201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3b4101420 msgr2=0x7fa3b41998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:51.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.745+0000 7fa3bc6af640 1 --2- 172.21.15.164:0/2460143201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3b4101420 0x7fa3b41998b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4009a50 tx=0x7fa3a4004270 comp rx=0 tx=0).stop 2024-09-19T08:43:51.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.745+0000 7fa3bc6af640 1 -- 172.21.15.164:0/2460143201 shutdown_connections 2024-09-19T08:43:51.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.745+0000 7fa3bc6af640 1 --2- 172.21.15.164:0/2460143201 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fa3a00a3240 0x7fa3a00a5700 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.745+0000 7fa3bc6af640 1 --2- 172.21.15.164:0/2460143201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3b4106a80 0x7fa3b41a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.745+0000 7fa3bc6af640 1 --2- 172.21.15.164:0/2460143201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3b4101420 0x7fa3b41998b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.745+0000 7fa3bc6af640 1 --2- 172.21.15.164:0/2460143201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3b4100a80 0x7fa3b4199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.745+0000 7fa3bc6af640 1 -- 172.21.15.164:0/2460143201 >> 172.21.15.164:0/2460143201 conn(0x7fa3b40fc7a0 msgr2=0x7fa3b40fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:51.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.745+0000 7fa3bc6af640 1 -- 172.21.15.164:0/2460143201 shutdown_connections 2024-09-19T08:43:51.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:51.745+0000 7fa3bc6af640 1 -- 172.21.15.164:0/2460143201 wait complete. 2024-09-19T08:43:52.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:51 smithi164 ceph-mon[30928]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T08:43:52.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:51 smithi164 ceph-mon[30928]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T08:43:52.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:51 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2460143201' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T08:43:52.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:51 smithi170 ceph-mon[35072]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T08:43:52.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:51 smithi170 ceph-mon[35072]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T08:43:52.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:51 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2460143201' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T08:43:52.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:51 smithi173 ceph-mon[35306]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T08:43:52.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:51 smithi173 ceph-mon[35306]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T08:43:52.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:51 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2460143201' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T08:43:52.459 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T08:43:52.702 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:43:52.756 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:52 smithi164 ceph-mon[30928]: pgmap v272: 33 pgs: 1 creating+activating, 17 creating+peering, 9 active+clean, 6 unknown; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:52.756 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:52 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2460143201' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T08:43:52.756 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:52 smithi164 ceph-mon[30928]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T08:43:53.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:52 smithi170 ceph-mon[35072]: pgmap v272: 33 pgs: 1 creating+activating, 17 creating+peering, 9 active+clean, 6 unknown; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:53.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:52 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2460143201' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T08:43:53.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:52 smithi170 ceph-mon[35072]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T08:43:53.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:52 smithi173 ceph-mon[35306]: pgmap v272: 33 pgs: 1 creating+activating, 17 creating+peering, 9 active+clean, 6 unknown; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:53.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:52 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2460143201' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T08:43:53.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:52 smithi173 ceph-mon[35306]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T08:43:53.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.175+0000 7fef45022640 1 -- 172.21.15.164:0/1713147029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef40101420 msgr2=0x7fef4010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:53.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.175+0000 7fef45022640 1 --2- 172.21.15.164:0/1713147029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef40101420 0x7fef4010d950 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fef30009930 tx=0x7fef3002a420 comp rx=0 tx=0).stop 2024-09-19T08:43:53.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.175+0000 7fef45022640 1 -- 172.21.15.164:0/1713147029 shutdown_connections 2024-09-19T08:43:53.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.175+0000 7fef45022640 1 --2- 172.21.15.164:0/1713147029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef40101420 0x7fef4010d950 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:53.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.175+0000 7fef45022640 1 --2- 172.21.15.164:0/1713147029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef40100a80 0x7fef40100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:53.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.175+0000 7fef45022640 1 --2- 172.21.15.164:0/1713147029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef40106a80 0x7fef40106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:53.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.175+0000 7fef45022640 1 -- 172.21.15.164:0/1713147029 >> 172.21.15.164:0/1713147029 conn(0x7fef400fc7a0 msgr2=0x7fef400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:53.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.175+0000 7fef45022640 1 -- 172.21.15.164:0/1713147029 shutdown_connections 2024-09-19T08:43:53.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.175+0000 7fef45022640 1 -- 172.21.15.164:0/1713147029 wait complete. 2024-09-19T08:43:53.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.175+0000 7fef45022640 1 Processor -- start 2024-09-19T08:43:53.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.176+0000 7fef45022640 1 -- start start 2024-09-19T08:43:53.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.176+0000 7fef45022640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef40100a80 0x7fef40197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:53.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.176+0000 7fef45022640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef40101420 0x7fef40197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:53.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.176+0000 7fef45022640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef40106a80 0x7fef4019bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:53.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.176+0000 7fef45022640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fef40110e80 con 0x7fef40100a80 2024-09-19T08:43:53.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.176+0000 7fef45022640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fef40110d00 con 0x7fef40106a80 2024-09-19T08:43:53.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.176+0000 7fef45022640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fef40111000 con 0x7fef40101420 2024-09-19T08:43:53.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.176+0000 7fef3ed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef40100a80 0x7fef40197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:53.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.176+0000 7fef3ed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef40100a80 0x7fef40197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51964/0 (socket says 172.21.15.164:51964) 2024-09-19T08:43:53.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.176+0000 7fef3ed9d640 1 -- 172.21.15.164:0/556103167 learned_addr learned my addr 172.21.15.164:0/556103167 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:43:53.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.176+0000 7fef3e59c640 1 --2- 172.21.15.164:0/556103167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef40101420 0x7fef40197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:53.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.176+0000 7fef3f59e640 1 --2- 172.21.15.164:0/556103167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef40106a80 0x7fef4019bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:53.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.176+0000 7fef3ed9d640 1 -- 172.21.15.164:0/556103167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef40101420 msgr2=0x7fef40197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:53.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.176+0000 7fef3ed9d640 1 --2- 172.21.15.164:0/556103167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef40101420 0x7fef40197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:53.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.176+0000 7fef3ed9d640 1 -- 172.21.15.164:0/556103167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef40106a80 msgr2=0x7fef4019bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:53.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.176+0000 7fef3ed9d640 1 --2- 172.21.15.164:0/556103167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef40106a80 0x7fef4019bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:53.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.176+0000 7fef3ed9d640 1 -- 172.21.15.164:0/556103167 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef4019c2d0 con 0x7fef40100a80 2024-09-19T08:43:53.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.177+0000 7fef3ed9d640 1 --2- 172.21.15.164:0/556103167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef40100a80 0x7fef40197320 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fef3800d770 tx=0x7fef3800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:53.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.177+0000 7fef37fff640 1 -- 172.21.15.164:0/556103167 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef38017070 con 0x7fef40100a80 2024-09-19T08:43:53.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.177+0000 7fef45022640 1 -- 172.21.15.164:0/556103167 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef4019c5c0 con 0x7fef40100a80 2024-09-19T08:43:53.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.177+0000 7fef37fff640 1 -- 172.21.15.164:0/556103167 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef38005500 con 0x7fef40100a80 2024-09-19T08:43:53.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.177+0000 7fef37fff640 1 -- 172.21.15.164:0/556103167 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef38004e00 con 0x7fef40100a80 2024-09-19T08:43:53.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.177+0000 7fef45022640 1 -- 172.21.15.164:0/556103167 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef401a3ea0 con 0x7fef40100a80 2024-09-19T08:43:53.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.178+0000 7fef45022640 1 -- 172.21.15.164:0/556103167 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef40100ee0 con 0x7fef40100a80 2024-09-19T08:43:53.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.179+0000 7fef37fff640 1 -- 172.21.15.164:0/556103167 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fef38004fa0 con 0x7fef40100a80 2024-09-19T08:43:53.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.180+0000 7fef37fff640 1 --2- 172.21.15.164:0/556103167 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fef240a3240 0x7fef240a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:53.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.180+0000 7fef3e59c640 1 --2- 172.21.15.164:0/556103167 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fef240a3240 0x7fef240a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:53.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.180+0000 7fef37fff640 1 -- 172.21.15.164:0/556103167 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7879+0+0 (secure 0 0 0) 0x7fef380cf4f0 con 0x7fef40100a80 2024-09-19T08:43:53.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.180+0000 7fef3e59c640 1 --2- 172.21.15.164:0/556103167 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fef240a3240 0x7fef240a5700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fef400fed60 tx=0x7fef2c00a590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:53.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.183+0000 7fef37fff640 1 -- 172.21.15.164:0/556103167 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fef3809c3c0 con 0x7fef40100a80 2024-09-19T08:43:53.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.383+0000 7fef45022640 1 -- 172.21.15.164:0/556103167 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fef40198680 con 0x7fef40100a80 2024-09-19T08:43:53.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.759+0000 7fef37fff640 1 -- 172.21.15.164:0/556103167 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fef40198680 con 0x7fef40100a80 2024-09-19T08:43:53.760 INFO:teuthology.orchestra.run.smithi164.stderr:new fs with metadata pool 2 and data pool 3 2024-09-19T08:43:53.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.762+0000 7fef45022640 1 -- 172.21.15.164:0/556103167 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fef240a3240 msgr2=0x7fef240a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:53.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.762+0000 7fef45022640 1 --2- 172.21.15.164:0/556103167 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fef240a3240 0x7fef240a5700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fef400fed60 tx=0x7fef2c00a590 comp rx=0 tx=0).stop 2024-09-19T08:43:53.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.762+0000 7fef45022640 1 -- 172.21.15.164:0/556103167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef40100a80 msgr2=0x7fef40197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:53.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.762+0000 7fef45022640 1 --2- 172.21.15.164:0/556103167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef40100a80 0x7fef40197320 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fef3800d770 tx=0x7fef3800dc30 comp rx=0 tx=0).stop 2024-09-19T08:43:53.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.762+0000 7fef45022640 1 -- 172.21.15.164:0/556103167 shutdown_connections 2024-09-19T08:43:53.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.762+0000 7fef45022640 1 --2- 172.21.15.164:0/556103167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef40106a80 0x7fef4019bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:53.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.762+0000 7fef45022640 1 --2- 172.21.15.164:0/556103167 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fef240a3240 0x7fef240a5700 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:53.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.762+0000 7fef45022640 1 --2- 172.21.15.164:0/556103167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef40101420 0x7fef40197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:53.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.762+0000 7fef45022640 1 --2- 172.21.15.164:0/556103167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef40100a80 0x7fef40197320 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:53.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.762+0000 7fef45022640 1 -- 172.21.15.164:0/556103167 >> 172.21.15.164:0/556103167 conn(0x7fef400fc7a0 msgr2=0x7fef400fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:53.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.762+0000 7fef45022640 1 -- 172.21.15.164:0/556103167 shutdown_connections 2024-09-19T08:43:53.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:53.762+0000 7fef45022640 1 -- 172.21.15.164:0/556103167 wait complete. 2024-09-19T08:43:54.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:53 smithi164 ceph-mon[30928]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T08:43:54.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:53 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/556103167' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T08:43:54.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:53 smithi164 ceph-mon[30928]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T08:43:54.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:53 smithi170 ceph-mon[35072]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T08:43:54.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:53 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/556103167' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T08:43:54.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:53 smithi170 ceph-mon[35072]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T08:43:54.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:53 smithi173 ceph-mon[35306]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T08:43:54.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:53 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/556103167' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T08:43:54.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:53 smithi173 ceph-mon[35306]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T08:43:54.442 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T08:43:54.689 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:43:55.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:54 smithi164 ceph-mon[30928]: pgmap v275: 97 pgs: 1 creating+activating, 24 creating+peering, 16 active+clean, 56 unknown; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:55.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:54 smithi164 ceph-mon[30928]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T08:43:55.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:54 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/556103167' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T08:43:55.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:54 smithi164 ceph-mon[30928]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T08:43:55.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:54 smithi164 ceph-mon[30928]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T08:43:55.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:54 smithi164 ceph-mon[30928]: mds.? [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] up:active 2024-09-19T08:43:55.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:54 smithi164 ceph-mon[30928]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T08:43:55.081 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:54 smithi173 ceph-mon[35306]: pgmap v275: 97 pgs: 1 creating+activating, 24 creating+peering, 16 active+clean, 56 unknown; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:55.082 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:54 smithi173 ceph-mon[35306]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T08:43:55.082 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:54 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/556103167' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T08:43:55.082 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:54 smithi173 ceph-mon[35306]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T08:43:55.082 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:54 smithi173 ceph-mon[35306]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T08:43:55.082 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:54 smithi173 ceph-mon[35306]: mds.? [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] up:active 2024-09-19T08:43:55.082 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:54 smithi173 ceph-mon[35306]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T08:43:55.090 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:54 smithi170 ceph-mon[35072]: pgmap v275: 97 pgs: 1 creating+activating, 24 creating+peering, 16 active+clean, 56 unknown; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:43:55.090 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:54 smithi170 ceph-mon[35072]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T08:43:55.090 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:54 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/556103167' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T08:43:55.090 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:54 smithi170 ceph-mon[35072]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T08:43:55.090 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:54 smithi170 ceph-mon[35072]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T08:43:55.090 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:54 smithi170 ceph-mon[35072]: mds.? [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] up:active 2024-09-19T08:43:55.090 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:54 smithi170 ceph-mon[35072]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T08:43:55.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.148+0000 7f857350c640 1 -- 172.21.15.164:0/2998658147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f856c101420 msgr2=0x7f856c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:55.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.148+0000 7f857350c640 1 --2- 172.21.15.164:0/2998658147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f856c101420 0x7f856c10d950 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8564009cd0 tx=0x7f856402a8e0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.148+0000 7f857350c640 1 -- 172.21.15.164:0/2998658147 shutdown_connections 2024-09-19T08:43:55.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.148+0000 7f857350c640 1 --2- 172.21.15.164:0/2998658147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f856c101420 0x7f856c10d950 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.148+0000 7f857350c640 1 --2- 172.21.15.164:0/2998658147 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f856c100a80 0x7f856c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.148+0000 7f857350c640 1 --2- 172.21.15.164:0/2998658147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f856c106a80 0x7f856c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.148+0000 7f857350c640 1 -- 172.21.15.164:0/2998658147 >> 172.21.15.164:0/2998658147 conn(0x7f856c0fc7a0 msgr2=0x7f856c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:55.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.148+0000 7f857350c640 1 -- 172.21.15.164:0/2998658147 shutdown_connections 2024-09-19T08:43:55.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.148+0000 7f857350c640 1 -- 172.21.15.164:0/2998658147 wait complete. 2024-09-19T08:43:55.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.148+0000 7f857350c640 1 Processor -- start 2024-09-19T08:43:55.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.148+0000 7f857350c640 1 -- start start 2024-09-19T08:43:55.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.149+0000 7f857350c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f856c100a80 0x7f856c199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:55.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.149+0000 7f857350c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f856c101420 0x7f856c1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:55.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.149+0000 7f857350c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f856c106a80 0x7f856c1a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:55.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.149+0000 7f857350c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f856c110e80 con 0x7f856c101420 2024-09-19T08:43:55.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.149+0000 7f857350c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f856c110d00 con 0x7f856c100a80 2024-09-19T08:43:55.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.149+0000 7f857350c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f856c111000 con 0x7f856c106a80 2024-09-19T08:43:55.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.149+0000 7f8570aa7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f856c101420 0x7f856c1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:55.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.149+0000 7f8570aa7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f856c101420 0x7f856c1997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47216/0 (socket says 172.21.15.164:47216) 2024-09-19T08:43:55.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.149+0000 7f8570aa7640 1 -- 172.21.15.164:0/107117231 learned_addr learned my addr 172.21.15.164:0/107117231 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:43:55.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.149+0000 7f85712a8640 1 --2- 172.21.15.164:0/107117231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f856c100a80 0x7f856c199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:55.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.149+0000 7f8571aa9640 1 --2- 172.21.15.164:0/107117231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f856c106a80 0x7f856c1a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:55.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.149+0000 7f8570aa7640 1 -- 172.21.15.164:0/107117231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f856c106a80 msgr2=0x7f856c1a4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:55.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.149+0000 7f8570aa7640 1 --2- 172.21.15.164:0/107117231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f856c106a80 0x7f856c1a4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.149+0000 7f8570aa7640 1 -- 172.21.15.164:0/107117231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f856c100a80 msgr2=0x7f856c199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:55.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.149+0000 7f8570aa7640 1 --2- 172.21.15.164:0/107117231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f856c100a80 0x7f856c199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.149+0000 7f8570aa7640 1 -- 172.21.15.164:0/107117231 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f856c1a4850 con 0x7f856c101420 2024-09-19T08:43:55.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.149+0000 7f8570aa7640 1 --2- 172.21.15.164:0/107117231 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f856c101420 0x7f856c1997d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f856800d950 tx=0x7f856800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:55.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.149+0000 7f855e7fc640 1 -- 172.21.15.164:0/107117231 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8568017070 con 0x7f856c101420 2024-09-19T08:43:55.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.149+0000 7f855e7fc640 1 -- 172.21.15.164:0/107117231 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8568009c30 con 0x7f856c101420 2024-09-19T08:43:55.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.149+0000 7f855e7fc640 1 -- 172.21.15.164:0/107117231 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856800f6b0 con 0x7f856c101420 2024-09-19T08:43:55.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.149+0000 7f857350c640 1 -- 172.21.15.164:0/107117231 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f856c1a4a20 con 0x7f856c101420 2024-09-19T08:43:55.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.150+0000 7f857350c640 1 -- 172.21.15.164:0/107117231 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f856c1a4f60 con 0x7f856c101420 2024-09-19T08:43:55.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.151+0000 7f857350c640 1 -- 172.21.15.164:0/107117231 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f856c100ee0 con 0x7f856c101420 2024-09-19T08:43:55.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.151+0000 7f855e7fc640 1 -- 172.21.15.164:0/107117231 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f85680109d0 con 0x7f856c101420 2024-09-19T08:43:55.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.152+0000 7f855e7fc640 1 --2- 172.21.15.164:0/107117231 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f85580a32c0 0x7f85580a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:55.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.152+0000 7f85712a8640 1 --2- 172.21.15.164:0/107117231 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f85580a32c0 0x7f85580a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:55.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.152+0000 7f855e7fc640 1 -- 172.21.15.164:0/107117231 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f85680cf320 con 0x7f856c101420 2024-09-19T08:43:55.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.153+0000 7f85712a8640 1 --2- 172.21.15.164:0/107117231 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f85580a32c0 0x7f85580a5780 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8560005fd0 tx=0x7f856000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:55.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.156+0000 7f855e7fc640 1 -- 172.21.15.164:0/107117231 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8568009de0 con 0x7f856c101420 2024-09-19T08:43:55.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.326+0000 7f857350c640 1 -- 172.21.15.164:0/107117231 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f856c108ff0 con 0x7f856c101420 2024-09-19T08:43:55.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.328+0000 7f855e7fc640 1 -- 172.21.15.164:0/107117231 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22362 (secure 0 0 0) 0x7f856809bb20 con 0x7f856c101420 2024-09-19T08:43:55.329 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:43:55.330 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":80,"fsid":"1e4b0d74-7662-11ef-bceb-c7b262605968","created":"2024-09-19T08:36:09.564509+0000","modified":"2024-09-19T08:43:53.385842+0000","last_up_change":"2024-09-19T08:43:01.260852+0000","last_in_change":"2024-09-19T08:42:37.492464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:39:13.961749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:43:49.613471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T08:43:51.570675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33b11f37-6ac5-41fc-8cfd-1b08c7eef688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6803","nonce":663038243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6805","nonce":663038243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6809","nonce":663038243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6807","nonce":663038243}]},"public_addr":"172.21.15.164:6803/663038243","cluster_addr":"172.21.15.164:6805/663038243","heartbeat_back_addr":"172.21.15.164:6809/663038243","heartbeat_front_addr":"172.21.15.164:6807/663038243","state":["exists","up"]},{"osd":1,"uuid":"6de64b0a-46e2-4a98-8005-a326a64bc87d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6801","nonce":2167066371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6803","nonce":2167066371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6807","nonce":2167066371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6805","nonce":2167066371}]},"public_addr":"172.21.15.170:6801/2167066371","cluster_addr":"172.21.15.170:6803/2167066371","heartbeat_back_addr":"172.21.15.170:6807/2167066371","heartbeat_front_addr":"172.21.15.170:6805/2167066371","state":["exists","up"]},{"osd":2,"uuid":"d4158cc7-9822-42a7-9601-d65089907584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6801","nonce":1646561048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6803","nonce":1646561048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6807","nonce":1646561048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6805","nonce":1646561048}]},"public_addr":"172.21.15.173:6801/1646561048","cluster_addr":"172.21.15.173:6803/1646561048","heartbeat_back_addr":"172.21.15.173:6807/1646561048","heartbeat_front_addr":"172.21.15.173:6805/1646561048","state":["exists","up"]},{"osd":3,"uuid":"0cf3f7f1-3a8c-42ce-88a9-9bea52e02221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6811","nonce":406795560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6813","nonce":406795560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6817","nonce":406795560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6815","nonce":406795560}]},"public_addr":"172.21.15.164:6811/406795560","cluster_addr":"172.21.15.164:6813/406795560","heartbeat_back_addr":"172.21.15.164:6817/406795560","heartbeat_front_addr":"172.21.15.164:6815/406795560","state":["exists","up"]},{"osd":4,"uuid":"6dca048a-2860-4a02-8d8e-450b30bc1a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6809","nonce":4120832165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6811","nonce":4120832165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6815","nonce":4120832165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6813","nonce":4120832165}]},"public_addr":"172.21.15.170:6809/4120832165","cluster_addr":"172.21.15.170:6811/4120832165","heartbeat_back_addr":"172.21.15.170:6815/4120832165","heartbeat_front_addr":"172.21.15.170:6813/4120832165","state":["exists","up"]},{"osd":5,"uuid":"3822521f-69d6-4141-a83f-5fc9cecc7b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6809","nonce":2914653552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6811","nonce":2914653552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6815","nonce":2914653552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6813","nonce":2914653552}]},"public_addr":"172.21.15.173:6809/2914653552","cluster_addr":"172.21.15.173:6811/2914653552","heartbeat_back_addr":"172.21.15.173:6815/2914653552","heartbeat_front_addr":"172.21.15.173:6813/2914653552","state":["exists","up"]},{"osd":6,"uuid":"0f5a965e-2eb2-4c24-b089-577e0639c160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6819","nonce":1138341393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6821","nonce":1138341393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6825","nonce":1138341393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6823","nonce":1138341393}]},"public_addr":"172.21.15.164:6819/1138341393","cluster_addr":"172.21.15.164:6821/1138341393","heartbeat_back_addr":"172.21.15.164:6825/1138341393","heartbeat_front_addr":"172.21.15.164:6823/1138341393","state":["exists","up"]},{"osd":7,"uuid":"f16cb2c3-639b-46ae-a740-1bd28ca1863c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6817","nonce":2124160360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6819","nonce":2124160360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6823","nonce":2124160360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6821","nonce":2124160360}]},"public_addr":"172.21.15.170:6817/2124160360","cluster_addr":"172.21.15.170:6819/2124160360","heartbeat_back_addr":"172.21.15.170:6823/2124160360","heartbeat_front_addr":"172.21.15.170:6821/2124160360","state":["exists","up"]},{"osd":8,"uuid":"161cff01-46ad-48e3-b999-a5473a2ad618","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6817","nonce":2285175061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6819","nonce":2285175061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6823","nonce":2285175061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6821","nonce":2285175061}]},"public_addr":"172.21.15.173:6817/2285175061","cluster_addr":"172.21.15.173:6819/2285175061","heartbeat_back_addr":"172.21.15.173:6823/2285175061","heartbeat_front_addr":"172.21.15.173:6821/2285175061","state":["exists","up"]},{"osd":9,"uuid":"994666e8-8fbd-4231-ac67-81a79aaf7c2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6827","nonce":3918847952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6829","nonce":3918847952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6833","nonce":3918847952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6831","nonce":3918847952}]},"public_addr":"172.21.15.164:6827/3918847952","cluster_addr":"172.21.15.164:6829/3918847952","heartbeat_back_addr":"172.21.15.164:6833/3918847952","heartbeat_front_addr":"172.21.15.164:6831/3918847952","state":["exists","up"]},{"osd":10,"uuid":"ff5e9099-cdb4-4905-b888-5b67320ef76b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6825","nonce":2707848227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6827","nonce":2707848227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6831","nonce":2707848227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6829","nonce":2707848227}]},"public_addr":"172.21.15.170:6825/2707848227","cluster_addr":"172.21.15.170:6827/2707848227","heartbeat_back_addr":"172.21.15.170:6831/2707848227","heartbeat_front_addr":"172.21.15.170:6829/2707848227","state":["exists","up"]},{"osd":11,"uuid":"4d93088f-e791-4938-a207-12f15b56abd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6825","nonce":2831924738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6827","nonce":2831924738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6831","nonce":2831924738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6829","nonce":2831924738}]},"public_addr":"172.21.15.173:6825/2831924738","cluster_addr":"172.21.15.173:6827/2831924738","heartbeat_back_addr":"172.21.15.173:6831/2831924738","heartbeat_front_addr":"172.21.15.173:6829/2831924738","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:21.787352+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:46.012323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:39:11.353040+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:39:37.015124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:02.063379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:27.892865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:53.372041+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:41:17.855284+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:41:43.389882+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:08.956895+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:34.069779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:59.303897+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.164:0/339032808":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6801/1173822158":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6800/1173822158":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6801/2805828009":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1837164919":"2024-09-20T08:37:01.663065+0000","172.21.15.164:0/4018459143":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6800/2805828009":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/2770797546":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1905394497":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1369265436":"2024-09-20T08:36:33.856642+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:43:55.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.330+0000 7f857350c640 1 -- 172.21.15.164:0/107117231 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f85580a32c0 msgr2=0x7f85580a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:55.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.330+0000 7f857350c640 1 --2- 172.21.15.164:0/107117231 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f85580a32c0 0x7f85580a5780 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8560005fd0 tx=0x7f856000b390 comp rx=0 tx=0).stop 2024-09-19T08:43:55.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.330+0000 7f857350c640 1 -- 172.21.15.164:0/107117231 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f856c101420 msgr2=0x7f856c1997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:55.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.330+0000 7f857350c640 1 --2- 172.21.15.164:0/107117231 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f856c101420 0x7f856c1997d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f856800d950 tx=0x7f856800de10 comp rx=0 tx=0).stop 2024-09-19T08:43:55.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.330+0000 7f857350c640 1 -- 172.21.15.164:0/107117231 shutdown_connections 2024-09-19T08:43:55.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.330+0000 7f857350c640 1 --2- 172.21.15.164:0/107117231 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f85580a32c0 0x7f85580a5780 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.330+0000 7f857350c640 1 --2- 172.21.15.164:0/107117231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f856c106a80 0x7f856c1a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.330+0000 7f857350c640 1 --2- 172.21.15.164:0/107117231 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f856c101420 0x7f856c1997d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.330+0000 7f857350c640 1 --2- 172.21.15.164:0/107117231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f856c100a80 0x7f856c199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.330+0000 7f857350c640 1 -- 172.21.15.164:0/107117231 >> 172.21.15.164:0/107117231 conn(0x7f856c0fc7a0 msgr2=0x7f856c0fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:55.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.330+0000 7f857350c640 1 -- 172.21.15.164:0/107117231 shutdown_connections 2024-09-19T08:43:55.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:55.330+0000 7f857350c640 1 -- 172.21.15.164:0/107117231 wait complete. 2024-09-19T08:43:55.990 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:55 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/107117231' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:43:56.019 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T08:43:56.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:55 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/107117231' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:43:56.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:55 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/107117231' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:43:56.266 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:43:56.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.790+0000 7f9893bd3640 1 -- 172.21.15.164:0/3290230292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f988c106f10 msgr2=0x7f988c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:56.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.790+0000 7f9893bd3640 1 --2- 172.21.15.164:0/3290230292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f988c106f10 0x7f988c10ea90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9884009ca0 tx=0x7f988402a8c0 comp rx=0 tx=0).stop 2024-09-19T08:43:56.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.791+0000 7f9893bd3640 1 -- 172.21.15.164:0/3290230292 shutdown_connections 2024-09-19T08:43:56.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.791+0000 7f9893bd3640 1 --2- 172.21.15.164:0/3290230292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f988c106f10 0x7f988c10ea90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:56.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.791+0000 7f9893bd3640 1 --2- 172.21.15.164:0/3290230292 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f988c102790 0x7f988c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:56.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.791+0000 7f9893bd3640 1 --2- 172.21.15.164:0/3290230292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f988c101de0 0x7f988c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:56.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.791+0000 7f9893bd3640 1 -- 172.21.15.164:0/3290230292 >> 172.21.15.164:0/3290230292 conn(0x7f988c0fd8c0 msgr2=0x7f988c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:56.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.791+0000 7f9893bd3640 1 -- 172.21.15.164:0/3290230292 shutdown_connections 2024-09-19T08:43:56.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.791+0000 7f9893bd3640 1 -- 172.21.15.164:0/3290230292 wait complete. 2024-09-19T08:43:56.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.791+0000 7f9893bd3640 1 Processor -- start 2024-09-19T08:43:56.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.791+0000 7f9893bd3640 1 -- start start 2024-09-19T08:43:56.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.791+0000 7f9893bd3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f988c101de0 0x7f988c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:56.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.791+0000 7f9893bd3640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f988c102790 0x7f988c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:56.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.791+0000 7f9893bd3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f988c106f10 0x7f988c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:56.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.791+0000 7f9893bd3640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f988c111f90 con 0x7f988c106f10 2024-09-19T08:43:56.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.791+0000 7f9893bd3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f988c111e10 con 0x7f988c101de0 2024-09-19T08:43:56.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.791+0000 7f9893bd3640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f988c112110 con 0x7f988c102790 2024-09-19T08:43:56.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.792+0000 7f9892170640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f988c106f10 0x7f988c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:56.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.792+0000 7f9892170640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f988c106f10 0x7f988c19cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47238/0 (socket says 172.21.15.164:47238) 2024-09-19T08:43:56.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.792+0000 7f9892170640 1 -- 172.21.15.164:0/1274061254 learned_addr learned my addr 172.21.15.164:0/1274061254 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:43:56.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.792+0000 7f989196f640 1 --2- 172.21.15.164:0/1274061254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f988c101de0 0x7f988c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:56.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.792+0000 7f989116e640 1 --2- 172.21.15.164:0/1274061254 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f988c102790 0x7f988c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:56.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.792+0000 7f9892170640 1 -- 172.21.15.164:0/1274061254 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f988c102790 msgr2=0x7f988c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:56.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.792+0000 7f9892170640 1 --2- 172.21.15.164:0/1274061254 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f988c102790 0x7f988c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:56.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.792+0000 7f9892170640 1 -- 172.21.15.164:0/1274061254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f988c101de0 msgr2=0x7f988c198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:56.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.792+0000 7f9892170640 1 --2- 172.21.15.164:0/1274061254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f988c101de0 0x7f988c198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:56.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.792+0000 7f9892170640 1 -- 172.21.15.164:0/1274061254 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f988c19d420 con 0x7f988c106f10 2024-09-19T08:43:56.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.792+0000 7f989196f640 1 --2- 172.21.15.164:0/1274061254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f988c101de0 0x7f988c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:43:56.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.792+0000 7f9892170640 1 --2- 172.21.15.164:0/1274061254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f988c106f10 0x7f988c19cd40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9884031a60 tx=0x7f9884031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:56.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.792+0000 7f987effd640 1 -- 172.21.15.164:0/1274061254 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9884003c30 con 0x7f988c106f10 2024-09-19T08:43:56.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.792+0000 7f987effd640 1 -- 172.21.15.164:0/1274061254 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9884003dd0 con 0x7f988c106f10 2024-09-19T08:43:56.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.792+0000 7f9893bd3640 1 -- 172.21.15.164:0/1274061254 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f988c19d6b0 con 0x7f988c106f10 2024-09-19T08:43:56.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.792+0000 7f987effd640 1 -- 172.21.15.164:0/1274061254 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98840047e0 con 0x7f988c106f10 2024-09-19T08:43:56.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.792+0000 7f9893bd3640 1 -- 172.21.15.164:0/1274061254 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f988c1a4ff0 con 0x7f988c106f10 2024-09-19T08:43:56.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.793+0000 7f9893bd3640 1 -- 172.21.15.164:0/1274061254 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f988c1021c0 con 0x7f988c106f10 2024-09-19T08:43:56.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.795+0000 7f987effd640 1 -- 172.21.15.164:0/1274061254 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f9884004b40 con 0x7f988c106f10 2024-09-19T08:43:56.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.795+0000 7f987effd640 1 --2- 172.21.15.164:0/1274061254 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f98780a32c0 0x7f98780a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:56.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.795+0000 7f989196f640 1 --2- 172.21.15.164:0/1274061254 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f98780a32c0 0x7f98780a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:56.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.795+0000 7f987effd640 1 -- 172.21.15.164:0/1274061254 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f98840f0da0 con 0x7f988c106f10 2024-09-19T08:43:56.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.796+0000 7f989196f640 1 --2- 172.21.15.164:0/1274061254 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f98780a32c0 0x7f98780a5780 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f98880058c0 tx=0x7f9888005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:56.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.799+0000 7f987effd640 1 -- 172.21.15.164:0/1274061254 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f98840bdca0 con 0x7f988c106f10 2024-09-19T08:43:56.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.968+0000 7f9893bd3640 1 -- 172.21.15.164:0/1274061254 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f988c1997d0 con 0x7f988c106f10 2024-09-19T08:43:56.969 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:56 smithi164 ceph-mon[30928]: pgmap v277: 97 pgs: 1 creating+activating, 7 creating+peering, 77 active+clean, 12 unknown; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-19T08:43:56.969 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:56 smithi164 ceph-mon[30928]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T08:43:56.969 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:56 smithi164 ceph-mon[30928]: Cluster is now healthy 2024-09-19T08:43:56.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.969+0000 7f987effd640 1 -- 172.21.15.164:0/1274061254 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22362 (secure 0 0 0) 0x7f988c1997d0 con 0x7f988c106f10 2024-09-19T08:43:56.970 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:43:56.970 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":80,"fsid":"1e4b0d74-7662-11ef-bceb-c7b262605968","created":"2024-09-19T08:36:09.564509+0000","modified":"2024-09-19T08:43:53.385842+0000","last_up_change":"2024-09-19T08:43:01.260852+0000","last_in_change":"2024-09-19T08:42:37.492464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:39:13.961749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:43:49.613471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T08:43:51.570675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33b11f37-6ac5-41fc-8cfd-1b08c7eef688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6803","nonce":663038243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6805","nonce":663038243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6809","nonce":663038243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6807","nonce":663038243}]},"public_addr":"172.21.15.164:6803/663038243","cluster_addr":"172.21.15.164:6805/663038243","heartbeat_back_addr":"172.21.15.164:6809/663038243","heartbeat_front_addr":"172.21.15.164:6807/663038243","state":["exists","up"]},{"osd":1,"uuid":"6de64b0a-46e2-4a98-8005-a326a64bc87d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6801","nonce":2167066371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6803","nonce":2167066371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6807","nonce":2167066371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6805","nonce":2167066371}]},"public_addr":"172.21.15.170:6801/2167066371","cluster_addr":"172.21.15.170:6803/2167066371","heartbeat_back_addr":"172.21.15.170:6807/2167066371","heartbeat_front_addr":"172.21.15.170:6805/2167066371","state":["exists","up"]},{"osd":2,"uuid":"d4158cc7-9822-42a7-9601-d65089907584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6801","nonce":1646561048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6803","nonce":1646561048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6807","nonce":1646561048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6805","nonce":1646561048}]},"public_addr":"172.21.15.173:6801/1646561048","cluster_addr":"172.21.15.173:6803/1646561048","heartbeat_back_addr":"172.21.15.173:6807/1646561048","heartbeat_front_addr":"172.21.15.173:6805/1646561048","state":["exists","up"]},{"osd":3,"uuid":"0cf3f7f1-3a8c-42ce-88a9-9bea52e02221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6811","nonce":406795560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6813","nonce":406795560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6817","nonce":406795560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6815","nonce":406795560}]},"public_addr":"172.21.15.164:6811/406795560","cluster_addr":"172.21.15.164:6813/406795560","heartbeat_back_addr":"172.21.15.164:6817/406795560","heartbeat_front_addr":"172.21.15.164:6815/406795560","state":["exists","up"]},{"osd":4,"uuid":"6dca048a-2860-4a02-8d8e-450b30bc1a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6809","nonce":4120832165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6811","nonce":4120832165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6815","nonce":4120832165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6813","nonce":4120832165}]},"public_addr":"172.21.15.170:6809/4120832165","cluster_addr":"172.21.15.170:6811/4120832165","heartbeat_back_addr":"172.21.15.170:6815/4120832165","heartbeat_front_addr":"172.21.15.170:6813/4120832165","state":["exists","up"]},{"osd":5,"uuid":"3822521f-69d6-4141-a83f-5fc9cecc7b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6809","nonce":2914653552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6811","nonce":2914653552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6815","nonce":2914653552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6813","nonce":2914653552}]},"public_addr":"172.21.15.173:6809/2914653552","cluster_addr":"172.21.15.173:6811/2914653552","heartbeat_back_addr":"172.21.15.173:6815/2914653552","heartbeat_front_addr":"172.21.15.173:6813/2914653552","state":["exists","up"]},{"osd":6,"uuid":"0f5a965e-2eb2-4c24-b089-577e0639c160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6819","nonce":1138341393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6821","nonce":1138341393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6825","nonce":1138341393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6823","nonce":1138341393}]},"public_addr":"172.21.15.164:6819/1138341393","cluster_addr":"172.21.15.164:6821/1138341393","heartbeat_back_addr":"172.21.15.164:6825/1138341393","heartbeat_front_addr":"172.21.15.164:6823/1138341393","state":["exists","up"]},{"osd":7,"uuid":"f16cb2c3-639b-46ae-a740-1bd28ca1863c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6817","nonce":2124160360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6819","nonce":2124160360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6823","nonce":2124160360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6821","nonce":2124160360}]},"public_addr":"172.21.15.170:6817/2124160360","cluster_addr":"172.21.15.170:6819/2124160360","heartbeat_back_addr":"172.21.15.170:6823/2124160360","heartbeat_front_addr":"172.21.15.170:6821/2124160360","state":["exists","up"]},{"osd":8,"uuid":"161cff01-46ad-48e3-b999-a5473a2ad618","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6817","nonce":2285175061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6819","nonce":2285175061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6823","nonce":2285175061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6821","nonce":2285175061}]},"public_addr":"172.21.15.173:6817/2285175061","cluster_addr":"172.21.15.173:6819/2285175061","heartbeat_back_addr":"172.21.15.173:6823/2285175061","heartbeat_front_addr":"172.21.15.173:6821/2285175061","state":["exists","up"]},{"osd":9,"uuid":"994666e8-8fbd-4231-ac67-81a79aaf7c2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6827","nonce":3918847952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6829","nonce":3918847952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6833","nonce":3918847952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6831","nonce":3918847952}]},"public_addr":"172.21.15.164:6827/3918847952","cluster_addr":"172.21.15.164:6829/3918847952","heartbeat_back_addr":"172.21.15.164:6833/3918847952","heartbeat_front_addr":"172.21.15.164:6831/3918847952","state":["exists","up"]},{"osd":10,"uuid":"ff5e9099-cdb4-4905-b888-5b67320ef76b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6825","nonce":2707848227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6827","nonce":2707848227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6831","nonce":2707848227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6829","nonce":2707848227}]},"public_addr":"172.21.15.170:6825/2707848227","cluster_addr":"172.21.15.170:6827/2707848227","heartbeat_back_addr":"172.21.15.170:6831/2707848227","heartbeat_front_addr":"172.21.15.170:6829/2707848227","state":["exists","up"]},{"osd":11,"uuid":"4d93088f-e791-4938-a207-12f15b56abd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6825","nonce":2831924738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6827","nonce":2831924738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6831","nonce":2831924738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6829","nonce":2831924738}]},"public_addr":"172.21.15.173:6825/2831924738","cluster_addr":"172.21.15.173:6827/2831924738","heartbeat_back_addr":"172.21.15.173:6831/2831924738","heartbeat_front_addr":"172.21.15.173:6829/2831924738","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:21.787352+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:46.012323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:39:11.353040+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:39:37.015124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:02.063379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:27.892865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:53.372041+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:41:17.855284+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:41:43.389882+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:08.956895+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:34.069779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:59.303897+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.164:0/339032808":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6801/1173822158":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6800/1173822158":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6801/2805828009":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1837164919":"2024-09-20T08:37:01.663065+0000","172.21.15.164:0/4018459143":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6800/2805828009":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/2770797546":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1905394497":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1369265436":"2024-09-20T08:36:33.856642+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:43:56.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.972+0000 7f9893bd3640 1 -- 172.21.15.164:0/1274061254 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f98780a32c0 msgr2=0x7f98780a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:56.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.972+0000 7f9893bd3640 1 --2- 172.21.15.164:0/1274061254 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f98780a32c0 0x7f98780a5780 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f98880058c0 tx=0x7f9888005ec0 comp rx=0 tx=0).stop 2024-09-19T08:43:56.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.972+0000 7f9893bd3640 1 -- 172.21.15.164:0/1274061254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f988c106f10 msgr2=0x7f988c19cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:56.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.972+0000 7f9893bd3640 1 --2- 172.21.15.164:0/1274061254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f988c106f10 0x7f988c19cd40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9884031a60 tx=0x7f9884031a90 comp rx=0 tx=0).stop 2024-09-19T08:43:56.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.972+0000 7f9893bd3640 1 -- 172.21.15.164:0/1274061254 shutdown_connections 2024-09-19T08:43:56.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.972+0000 7f9893bd3640 1 --2- 172.21.15.164:0/1274061254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f988c106f10 0x7f988c19cd40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:56.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.972+0000 7f9893bd3640 1 --2- 172.21.15.164:0/1274061254 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f98780a32c0 0x7f98780a5780 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:56.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.972+0000 7f9893bd3640 1 --2- 172.21.15.164:0/1274061254 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f988c102790 0x7f988c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:56.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.972+0000 7f9893bd3640 1 --2- 172.21.15.164:0/1274061254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f988c101de0 0x7f988c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:56.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.972+0000 7f9893bd3640 1 -- 172.21.15.164:0/1274061254 >> 172.21.15.164:0/1274061254 conn(0x7f988c0fd8c0 msgr2=0x7f988c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:56.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.973+0000 7f9893bd3640 1 -- 172.21.15.164:0/1274061254 shutdown_connections 2024-09-19T08:43:56.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:56.973+0000 7f9893bd3640 1 -- 172.21.15.164:0/1274061254 wait complete. 2024-09-19T08:43:57.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:56 smithi170 ceph-mon[35072]: pgmap v277: 97 pgs: 1 creating+activating, 7 creating+peering, 77 active+clean, 12 unknown; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-19T08:43:57.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:56 smithi170 ceph-mon[35072]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T08:43:57.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:56 smithi170 ceph-mon[35072]: Cluster is now healthy 2024-09-19T08:43:57.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:56 smithi173 ceph-mon[35306]: pgmap v277: 97 pgs: 1 creating+activating, 7 creating+peering, 77 active+clean, 12 unknown; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-19T08:43:57.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:56 smithi173 ceph-mon[35306]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T08:43:57.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:56 smithi173 ceph-mon[35306]: Cluster is now healthy 2024-09-19T08:43:57.815 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-19T08:43:58.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:57 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/1274061254' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:43:58.062 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:43:58.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:57 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/1274061254' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:43:58.125 INFO:journalctl@ceph.mds.b.smithi170.stdout:Sep 19 08:43:57 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-b[65018]: 2024-09-19T08:43:57.803+0000 7f2f742fb640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T08:43:58.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:57 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/1274061254' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:43:58.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.516+0000 7f4dea122640 1 -- 172.21.15.164:0/3899371874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4de4101de0 msgr2=0x7f4de41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:58.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.516+0000 7f4dea122640 1 --2- 172.21.15.164:0/3899371874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4de4101de0 0x7f4de41021c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4009900 tx=0x7f4dd402a410 comp rx=0 tx=0).stop 2024-09-19T08:43:58.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.516+0000 7f4dea122640 1 -- 172.21.15.164:0/3899371874 shutdown_connections 2024-09-19T08:43:58.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.516+0000 7f4dea122640 1 --2- 172.21.15.164:0/3899371874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4de4106f10 0x7f4de410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.516+0000 7f4dea122640 1 --2- 172.21.15.164:0/3899371874 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4de4102790 0x7f4de41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.516+0000 7f4dea122640 1 --2- 172.21.15.164:0/3899371874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4de4101de0 0x7f4de41021c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.516+0000 7f4dea122640 1 -- 172.21.15.164:0/3899371874 >> 172.21.15.164:0/3899371874 conn(0x7f4de40fd8c0 msgr2=0x7f4de40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:58.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.516+0000 7f4dea122640 1 -- 172.21.15.164:0/3899371874 shutdown_connections 2024-09-19T08:43:58.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.516+0000 7f4dea122640 1 -- 172.21.15.164:0/3899371874 wait complete. 2024-09-19T08:43:58.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.517+0000 7f4dea122640 1 Processor -- start 2024-09-19T08:43:58.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.517+0000 7f4dea122640 1 -- start start 2024-09-19T08:43:58.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.517+0000 7f4dea122640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4de4101de0 0x7f4de41984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:58.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.517+0000 7f4dea122640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4de4102790 0x7f4de41989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:58.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.517+0000 7f4dea122640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4de4106f10 0x7f4de419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:58.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.517+0000 7f4dea122640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4de4111f90 con 0x7f4de4102790 2024-09-19T08:43:58.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.517+0000 7f4dea122640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4de4111e10 con 0x7f4de4106f10 2024-09-19T08:43:58.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.517+0000 7f4dea122640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4de4112110 con 0x7f4de4101de0 2024-09-19T08:43:58.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.517+0000 7f4de2ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4de4102790 0x7f4de41989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:58.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.517+0000 7f4de2ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4de4102790 0x7f4de41989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47250/0 (socket says 172.21.15.164:47250) 2024-09-19T08:43:58.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.517+0000 7f4de2ffd640 1 -- 172.21.15.164:0/1933512596 learned_addr learned my addr 172.21.15.164:0/1933512596 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:43:58.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.517+0000 7f4de3fff640 1 --2- 172.21.15.164:0/1933512596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4de4106f10 0x7f4de419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:58.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.517+0000 7f4de37fe640 1 --2- 172.21.15.164:0/1933512596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4de4101de0 0x7f4de41984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:58.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.518+0000 7f4de2ffd640 1 -- 172.21.15.164:0/1933512596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4de4101de0 msgr2=0x7f4de41984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:58.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.518+0000 7f4de2ffd640 1 --2- 172.21.15.164:0/1933512596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4de4101de0 0x7f4de41984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.518+0000 7f4de2ffd640 1 -- 172.21.15.164:0/1933512596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4de4106f10 msgr2=0x7f4de419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:58.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.518+0000 7f4de2ffd640 1 --2- 172.21.15.164:0/1933512596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4de4106f10 0x7f4de419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.518+0000 7f4de2ffd640 1 -- 172.21.15.164:0/1933512596 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de419d460 con 0x7f4de4102790 2024-09-19T08:43:58.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.518+0000 7f4de2ffd640 1 --2- 172.21.15.164:0/1933512596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4de4102790 0x7f4de41989f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800b4a0 tx=0x7f4dd800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:58.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.518+0000 7f4de0ff9640 1 -- 172.21.15.164:0/1933512596 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8014070 con 0x7f4de4102790 2024-09-19T08:43:58.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.518+0000 7f4de0ff9640 1 -- 172.21.15.164:0/1933512596 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dd8010af0 con 0x7f4de4102790 2024-09-19T08:43:58.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.518+0000 7f4dea122640 1 -- 172.21.15.164:0/1933512596 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4de419d750 con 0x7f4de4102790 2024-09-19T08:43:58.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.518+0000 7f4de0ff9640 1 -- 172.21.15.164:0/1933512596 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8017d10 con 0x7f4de4102790 2024-09-19T08:43:58.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.518+0000 7f4dea122640 1 -- 172.21.15.164:0/1933512596 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4de41a5030 con 0x7f4de4102790 2024-09-19T08:43:58.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.519+0000 7f4dea122640 1 -- 172.21.15.164:0/1933512596 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4de41021c0 con 0x7f4de4102790 2024-09-19T08:43:58.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.520+0000 7f4de0ff9640 1 -- 172.21.15.164:0/1933512596 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f4dd8017070 con 0x7f4de4102790 2024-09-19T08:43:58.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.521+0000 7f4de0ff9640 1 --2- 172.21.15.164:0/1933512596 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f4dcc0a3240 0x7f4dcc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:58.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.521+0000 7f4de37fe640 1 --2- 172.21.15.164:0/1933512596 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f4dcc0a3240 0x7f4dcc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:58.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.521+0000 7f4de0ff9640 1 -- 172.21.15.164:0/1933512596 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f4dd80ceb50 con 0x7f4de4102790 2024-09-19T08:43:58.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.521+0000 7f4de37fe640 1 --2- 172.21.15.164:0/1933512596 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f4dcc0a3240 0x7f4dcc0a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4000c00 tx=0x7f4dd40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:58.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.525+0000 7f4de0ff9640 1 -- 172.21.15.164:0/1933512596 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4dd809b370 con 0x7f4de4102790 2024-09-19T08:43:58.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.728+0000 7f4dea122640 1 -- 172.21.15.164:0/1933512596 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f4de410a150 con 0x7f4de4102790 2024-09-19T08:43:58.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.774+0000 7f4de0ff9640 1 -- 172.21.15.164:0/1933512596 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f4dd809f670 con 0x7f4de4102790 2024-09-19T08:43:58.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.777+0000 7f4dea122640 1 -- 172.21.15.164:0/1933512596 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f4dcc0a3240 msgr2=0x7f4dcc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:58.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.777+0000 7f4dea122640 1 --2- 172.21.15.164:0/1933512596 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f4dcc0a3240 0x7f4dcc0a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4000c00 tx=0x7f4dd40058d0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.777+0000 7f4dea122640 1 -- 172.21.15.164:0/1933512596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4de4102790 msgr2=0x7f4de41989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:58.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.777+0000 7f4dea122640 1 --2- 172.21.15.164:0/1933512596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4de4102790 0x7f4de41989f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800b4a0 tx=0x7f4dd800b960 comp rx=0 tx=0).stop 2024-09-19T08:43:58.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.777+0000 7f4dea122640 1 -- 172.21.15.164:0/1933512596 shutdown_connections 2024-09-19T08:43:58.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.777+0000 7f4dea122640 1 --2- 172.21.15.164:0/1933512596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4de4106f10 0x7f4de419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.777+0000 7f4dea122640 1 --2- 172.21.15.164:0/1933512596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4de4102790 0x7f4de41989f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.777+0000 7f4dea122640 1 --2- 172.21.15.164:0/1933512596 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f4dcc0a3240 0x7f4dcc0a5700 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.777+0000 7f4dea122640 1 --2- 172.21.15.164:0/1933512596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4de4101de0 0x7f4de41984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.777+0000 7f4dea122640 1 -- 172.21.15.164:0/1933512596 >> 172.21.15.164:0/1933512596 conn(0x7f4de40fd8c0 msgr2=0x7f4de40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:58.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.777+0000 7f4dea122640 1 -- 172.21.15.164:0/1933512596 shutdown_connections 2024-09-19T08:43:58.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:43:58.777+0000 7f4dea122640 1 -- 172.21.15.164:0/1933512596 wait complete. 2024-09-19T08:43:59.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:58 smithi164 ceph-mon[30928]: pgmap v278: 97 pgs: 97 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-19T08:43:59.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:58 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/1933512596' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T08:43:59.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:58 smithi170 ceph-mon[35072]: pgmap v278: 97 pgs: 97 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-19T08:43:59.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:58 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/1933512596' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T08:43:59.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:58 smithi173 ceph-mon[35306]: pgmap v278: 97 pgs: 97 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-19T08:43:59.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:58 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/1933512596' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T08:43:59.417 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-19T08:43:59.418 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-19T08:43:59.676 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:00.074 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:59 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/1933512596' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T08:44:00.074 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:43:59 smithi164 ceph-mon[30928]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T08:44:00.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:59 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/1933512596' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T08:44:00.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:43:59 smithi170 ceph-mon[35072]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T08:44:00.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:59 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/1933512596' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T08:44:00.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:43:59 smithi173 ceph-mon[35306]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T08:44:00.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.222+0000 7f21b905c640 1 -- 172.21.15.164:0/1009557246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21b4102570 msgr2=0x7f21b41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:00.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.222+0000 7f21b905c640 1 --2- 172.21.15.164:0/1009557246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21b4102570 0x7f21b41029b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f219c009900 tx=0x7f219c02a480 comp rx=0 tx=0).stop 2024-09-19T08:44:00.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.222+0000 7f21b905c640 1 -- 172.21.15.164:0/1009557246 shutdown_connections 2024-09-19T08:44:00.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.222+0000 7f21b905c640 1 --2- 172.21.15.164:0/1009557246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21b4102ef0 0x7f21b410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:00.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.222+0000 7f21b905c640 1 --2- 172.21.15.164:0/1009557246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21b4102570 0x7f21b41029b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:00.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.222+0000 7f21b905c640 1 --2- 172.21.15.164:0/1009557246 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21b4100e20 0x7f21b4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:00.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.222+0000 7f21b905c640 1 -- 172.21.15.164:0/1009557246 >> 172.21.15.164:0/1009557246 conn(0x7f21b40fc7d0 msgr2=0x7f21b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:00.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.223+0000 7f21b905c640 1 -- 172.21.15.164:0/1009557246 shutdown_connections 2024-09-19T08:44:00.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.223+0000 7f21b905c640 1 -- 172.21.15.164:0/1009557246 wait complete. 2024-09-19T08:44:00.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.223+0000 7f21b905c640 1 Processor -- start 2024-09-19T08:44:00.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.223+0000 7f21b905c640 1 -- start start 2024-09-19T08:44:00.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.224+0000 7f21b905c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21b4100e20 0x7f21b4197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:00.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.224+0000 7f21b905c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21b4102570 0x7f21b4197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:00.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.224+0000 7f21b905c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21b4102ef0 0x7f21b419bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:00.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.224+0000 7f21b905c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f21b4110fa0 con 0x7f21b4100e20 2024-09-19T08:44:00.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.224+0000 7f21b905c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f21b4110e20 con 0x7f21b4102ef0 2024-09-19T08:44:00.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.224+0000 7f21b905c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f21b4111120 con 0x7f21b4102570 2024-09-19T08:44:00.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.224+0000 7f21b2d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21b4100e20 0x7f21b4197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:00.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.224+0000 7f21b2d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21b4100e20 0x7f21b4197400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47256/0 (socket says 172.21.15.164:47256) 2024-09-19T08:44:00.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.224+0000 7f21b2d9d640 1 -- 172.21.15.164:0/919317335 learned_addr learned my addr 172.21.15.164:0/919317335 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:00.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.224+0000 7f21b259c640 1 --2- 172.21.15.164:0/919317335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21b4102570 0x7f21b4197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:00.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.224+0000 7f21b359e640 1 --2- 172.21.15.164:0/919317335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21b4102ef0 0x7f21b419bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:00.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.224+0000 7f21b2d9d640 1 -- 172.21.15.164:0/919317335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21b4102570 msgr2=0x7f21b4197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:00.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.224+0000 7f21b2d9d640 1 --2- 172.21.15.164:0/919317335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21b4102570 0x7f21b4197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:00.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.224+0000 7f21b2d9d640 1 -- 172.21.15.164:0/919317335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21b4102ef0 msgr2=0x7f21b419bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:00.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.224+0000 7f21b2d9d640 1 --2- 172.21.15.164:0/919317335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21b4102ef0 0x7f21b419bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:00.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.224+0000 7f21b2d9d640 1 -- 172.21.15.164:0/919317335 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b419c3b0 con 0x7f21b4100e20 2024-09-19T08:44:00.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.224+0000 7f21b359e640 1 --2- 172.21.15.164:0/919317335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21b4102ef0 0x7f21b419bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:00.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.224+0000 7f21b2d9d640 1 --2- 172.21.15.164:0/919317335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21b4100e20 0x7f21b4197400 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f21a400ba50 tx=0x7f21a400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:00.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.225+0000 7f21abfff640 1 -- 172.21.15.164:0/919317335 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a4019070 con 0x7f21b4100e20 2024-09-19T08:44:00.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.225+0000 7f21b905c640 1 -- 172.21.15.164:0/919317335 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21b419c6a0 con 0x7f21b4100e20 2024-09-19T08:44:00.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.225+0000 7f21abfff640 1 -- 172.21.15.164:0/919317335 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21a400cbe0 con 0x7f21b4100e20 2024-09-19T08:44:00.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.225+0000 7f21abfff640 1 -- 172.21.15.164:0/919317335 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a40076e0 con 0x7f21b4100e20 2024-09-19T08:44:00.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.225+0000 7f21b905c640 1 -- 172.21.15.164:0/919317335 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21b41a3f80 con 0x7f21b4100e20 2024-09-19T08:44:00.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.226+0000 7f21b905c640 1 -- 172.21.15.164:0/919317335 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21b4101200 con 0x7f21b4100e20 2024-09-19T08:44:00.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.227+0000 7f21abfff640 1 -- 172.21.15.164:0/919317335 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f21a401c070 con 0x7f21b4100e20 2024-09-19T08:44:00.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.227+0000 7f21abfff640 1 --2- 172.21.15.164:0/919317335 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f21a00a3240 0x7f21a00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:00.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.228+0000 7f21b259c640 1 --2- 172.21.15.164:0/919317335 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f21a00a3240 0x7f21a00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:00.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.228+0000 7f21abfff640 1 -- 172.21.15.164:0/919317335 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f21a40cee70 con 0x7f21b4100e20 2024-09-19T08:44:00.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.228+0000 7f21b259c640 1 --2- 172.21.15.164:0/919317335 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f21a00a3240 0x7f21a00a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f21b40fec10 tx=0x7f219c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:00.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.231+0000 7f21abfff640 1 -- 172.21.15.164:0/919317335 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f21a409bce0 con 0x7f21b4100e20 2024-09-19T08:44:00.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.433+0000 7f21b905c640 1 -- 172.21.15.164:0/919317335 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f21b41a4150 con 0x7f21b4100e20 2024-09-19T08:44:00.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.790+0000 7f21abfff640 1 -- 172.21.15.164:0/919317335 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v17) ==== 105+0+0 (secure 0 0 0) 0x7f21a409b5e0 con 0x7f21b4100e20 2024-09-19T08:44:00.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.793+0000 7f21b905c640 1 -- 172.21.15.164:0/919317335 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f21a00a3240 msgr2=0x7f21a00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:00.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.793+0000 7f21b905c640 1 --2- 172.21.15.164:0/919317335 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f21a00a3240 0x7f21a00a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f21b40fec10 tx=0x7f219c005a20 comp rx=0 tx=0).stop 2024-09-19T08:44:00.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.793+0000 7f21b905c640 1 -- 172.21.15.164:0/919317335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21b4100e20 msgr2=0x7f21b4197400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:00.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.793+0000 7f21b905c640 1 --2- 172.21.15.164:0/919317335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21b4100e20 0x7f21b4197400 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f21a400ba50 tx=0x7f21a400bf10 comp rx=0 tx=0).stop 2024-09-19T08:44:00.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.793+0000 7f21b905c640 1 -- 172.21.15.164:0/919317335 shutdown_connections 2024-09-19T08:44:00.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.793+0000 7f21b905c640 1 --2- 172.21.15.164:0/919317335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21b4102ef0 0x7f21b419bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:00.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.793+0000 7f21b905c640 1 --2- 172.21.15.164:0/919317335 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f21a00a3240 0x7f21a00a5700 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:00.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.793+0000 7f21b905c640 1 --2- 172.21.15.164:0/919317335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21b4102570 0x7f21b4197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:00.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.793+0000 7f21b905c640 1 --2- 172.21.15.164:0/919317335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21b4100e20 0x7f21b4197400 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:00.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.793+0000 7f21b905c640 1 -- 172.21.15.164:0/919317335 >> 172.21.15.164:0/919317335 conn(0x7f21b40fc7d0 msgr2=0x7f21b41068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:00.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.793+0000 7f21b905c640 1 -- 172.21.15.164:0/919317335 shutdown_connections 2024-09-19T08:44:00.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:00.793+0000 7f21b905c640 1 -- 172.21.15.164:0/919317335 wait complete. 2024-09-19T08:44:01.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:00 smithi170 ceph-mon[35072]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-19T08:44:01.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:00 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/919317335' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T08:44:01.148 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:00 smithi164 ceph-mon[30928]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-19T08:44:01.148 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:00 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/919317335' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T08:44:01.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:00 smithi173 ceph-mon[35306]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-19T08:44:01.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:00 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/919317335' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T08:44:01.484 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-19T08:44:01.733 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:02.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:01 smithi170 ceph-mon[35072]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T08:44:02.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:01 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/919317335' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T08:44:02.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:01 smithi170 ceph-mon[35072]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T08:44:02.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:01 smithi170 ceph-mon[35072]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T08:44:02.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:01 smithi170 ceph-mon[35072]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-19T08:44:02.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:01 smithi170 ceph-mon[35072]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-19T08:44:02.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:01 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-09-19T08:44:02.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:01 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-09-19T08:44:02.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:01 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-09-19T08:44:02.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:01 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-09-19T08:44:02.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:01 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-09-19T08:44:02.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:01 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-19T08:44:02.152 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:01 smithi164 ceph-mon[30928]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T08:44:02.153 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:01 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/919317335' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T08:44:02.153 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:01 smithi164 ceph-mon[30928]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T08:44:02.153 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:01 smithi164 ceph-mon[30928]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T08:44:02.153 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:01 smithi164 ceph-mon[30928]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-19T08:44:02.153 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:01 smithi164 ceph-mon[30928]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-19T08:44:02.153 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:01 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-09-19T08:44:02.153 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:01 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-09-19T08:44:02.153 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:01 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-09-19T08:44:02.153 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:01 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-09-19T08:44:02.153 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:01 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-09-19T08:44:02.154 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:01 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-19T08:44:02.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:01 smithi173 ceph-mon[35306]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T08:44:02.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:01 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/919317335' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T08:44:02.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:01 smithi173 ceph-mon[35306]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T08:44:02.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:01 smithi173 ceph-mon[35306]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T08:44:02.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:01 smithi173 ceph-mon[35306]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-19T08:44:02.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:01 smithi173 ceph-mon[35306]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-19T08:44:02.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:01 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-09-19T08:44:02.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:01 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-09-19T08:44:02.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:01 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-09-19T08:44:02.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:01 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-09-19T08:44:02.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:01 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-09-19T08:44:02.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:01 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-19T08:44:02.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.236+0000 7fd1c9a0b640 1 -- 172.21.15.164:0/4278168525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1c4100e20 msgr2=0x7fd1c4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:02.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.236+0000 7fd1c9a0b640 1 --2- 172.21.15.164:0/4278168525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1c4100e20 0x7fd1c4101200 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4009900 tx=0x7fd1b402a480 comp rx=0 tx=0).stop 2024-09-19T08:44:02.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.236+0000 7fd1c9a0b640 1 -- 172.21.15.164:0/4278168525 shutdown_connections 2024-09-19T08:44:02.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.236+0000 7fd1c9a0b640 1 --2- 172.21.15.164:0/4278168525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1c4102ef0 0x7fd1c410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:02.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.236+0000 7fd1c9a0b640 1 --2- 172.21.15.164:0/4278168525 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1c4102570 0x7fd1c41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:02.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.236+0000 7fd1c9a0b640 1 --2- 172.21.15.164:0/4278168525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1c4100e20 0x7fd1c4101200 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:02.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.236+0000 7fd1c9a0b640 1 -- 172.21.15.164:0/4278168525 >> 172.21.15.164:0/4278168525 conn(0x7fd1c40fc7d0 msgr2=0x7fd1c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:02.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.236+0000 7fd1c9a0b640 1 -- 172.21.15.164:0/4278168525 shutdown_connections 2024-09-19T08:44:02.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.236+0000 7fd1c9a0b640 1 -- 172.21.15.164:0/4278168525 wait complete. 2024-09-19T08:44:02.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.237+0000 7fd1c9a0b640 1 Processor -- start 2024-09-19T08:44:02.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.237+0000 7fd1c9a0b640 1 -- start start 2024-09-19T08:44:02.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.237+0000 7fd1c9a0b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1c4100e20 0x7fd1c40fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:02.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.237+0000 7fd1c9a0b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1c4102570 0x7fd1c40fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:02.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.237+0000 7fd1c9a0b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1c4102ef0 0x7fd1c40fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:02.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.237+0000 7fd1c9a0b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c4110fa0 con 0x7fd1c4102ef0 2024-09-19T08:44:02.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.237+0000 7fd1c9a0b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c4110e20 con 0x7fd1c4102570 2024-09-19T08:44:02.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.237+0000 7fd1c9a0b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c4111120 con 0x7fd1c4100e20 2024-09-19T08:44:02.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.237+0000 7fd1c37fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1c4102ef0 0x7fd1c40fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:02.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.237+0000 7fd1c37fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1c4102ef0 0x7fd1c40fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47276/0 (socket says 172.21.15.164:47276) 2024-09-19T08:44:02.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.237+0000 7fd1c37fe640 1 -- 172.21.15.164:0/1438838092 learned_addr learned my addr 172.21.15.164:0/1438838092 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:02.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.237+0000 7fd1c2ffd640 1 --2- 172.21.15.164:0/1438838092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1c4100e20 0x7fd1c40fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:02.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.237+0000 7fd1bbfff640 1 --2- 172.21.15.164:0/1438838092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1c4102570 0x7fd1c40fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:02.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.237+0000 7fd1c37fe640 1 -- 172.21.15.164:0/1438838092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1c4100e20 msgr2=0x7fd1c40fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:02.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.237+0000 7fd1c37fe640 1 --2- 172.21.15.164:0/1438838092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1c4100e20 0x7fd1c40fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:02.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.237+0000 7fd1c37fe640 1 -- 172.21.15.164:0/1438838092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1c4102570 msgr2=0x7fd1c40fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:02.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.237+0000 7fd1c37fe640 1 --2- 172.21.15.164:0/1438838092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1c4102570 0x7fd1c40fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:02.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.237+0000 7fd1c37fe640 1 -- 172.21.15.164:0/1438838092 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c41a6ab0 con 0x7fd1c4102ef0 2024-09-19T08:44:02.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.238+0000 7fd1c37fe640 1 --2- 172.21.15.164:0/1438838092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1c4102ef0 0x7fd1c40fd700 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00b970 tx=0x7fd1bc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:02.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.238+0000 7fd1c8a09640 1 -- 172.21.15.164:0/1438838092 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc019070 con 0x7fd1c4102ef0 2024-09-19T08:44:02.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.238+0000 7fd1c8a09640 1 -- 172.21.15.164:0/1438838092 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1bc00cb40 con 0x7fd1c4102ef0 2024-09-19T08:44:02.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.238+0000 7fd1c9a0b640 1 -- 172.21.15.164:0/1438838092 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1c41a6da0 con 0x7fd1c4102ef0 2024-09-19T08:44:02.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.238+0000 7fd1c8a09640 1 -- 172.21.15.164:0/1438838092 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc007650 con 0x7fd1c4102ef0 2024-09-19T08:44:02.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.238+0000 7fd1c9a0b640 1 -- 172.21.15.164:0/1438838092 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1c41a72b0 con 0x7fd1c4102ef0 2024-09-19T08:44:02.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.239+0000 7fd1c9a0b640 1 -- 172.21.15.164:0/1438838092 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1c4101200 con 0x7fd1c4102ef0 2024-09-19T08:44:02.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.240+0000 7fd1c8a09640 1 -- 172.21.15.164:0/1438838092 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fd1bc01c070 con 0x7fd1c4102ef0 2024-09-19T08:44:02.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.241+0000 7fd1c8a09640 1 --2- 172.21.15.164:0/1438838092 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fd1b00a3240 0x7fd1b00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:02.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.241+0000 7fd1c2ffd640 1 --2- 172.21.15.164:0/1438838092 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fd1b00a3240 0x7fd1b00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:02.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.241+0000 7fd1c8a09640 1 -- 172.21.15.164:0/1438838092 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8153+0+0 (secure 0 0 0) 0x7fd1bc0cee70 con 0x7fd1c4102ef0 2024-09-19T08:44:02.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.241+0000 7fd1c2ffd640 1 --2- 172.21.15.164:0/1438838092 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fd1b00a3240 0x7fd1b00a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4005ab0 tx=0x7fd1b4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:02.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.245+0000 7fd1c8a09640 1 -- 172.21.15.164:0/1438838092 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd1bc09bc30 con 0x7fd1c4102ef0 2024-09-19T08:44:02.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.444+0000 7fd1c9a0b640 1 -- 172.21.15.164:0/1438838092 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fd1c41a7540 con 0x7fd1c4102ef0 2024-09-19T08:44:02.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.821+0000 7fd1c8a09640 1 -- 172.21.15.164:0/1438838092 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7fd1bc09b530 con 0x7fd1c4102ef0 2024-09-19T08:44:02.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.824+0000 7fd1c9a0b640 1 -- 172.21.15.164:0/1438838092 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fd1b00a3240 msgr2=0x7fd1b00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:02.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.824+0000 7fd1c9a0b640 1 --2- 172.21.15.164:0/1438838092 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fd1b00a3240 0x7fd1b00a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4005ab0 tx=0x7fd1b4005a20 comp rx=0 tx=0).stop 2024-09-19T08:44:02.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.824+0000 7fd1c9a0b640 1 -- 172.21.15.164:0/1438838092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1c4102ef0 msgr2=0x7fd1c40fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:02.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.824+0000 7fd1c9a0b640 1 --2- 172.21.15.164:0/1438838092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1c4102ef0 0x7fd1c40fd700 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00b970 tx=0x7fd1bc00be30 comp rx=0 tx=0).stop 2024-09-19T08:44:02.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.824+0000 7fd1c9a0b640 1 -- 172.21.15.164:0/1438838092 shutdown_connections 2024-09-19T08:44:02.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.824+0000 7fd1c9a0b640 1 --2- 172.21.15.164:0/1438838092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1c4102ef0 0x7fd1c40fd700 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:02.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.824+0000 7fd1c9a0b640 1 --2- 172.21.15.164:0/1438838092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1c4102570 0x7fd1c40fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:02.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.824+0000 7fd1c9a0b640 1 --2- 172.21.15.164:0/1438838092 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fd1b00a3240 0x7fd1b00a5700 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:02.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.824+0000 7fd1c9a0b640 1 --2- 172.21.15.164:0/1438838092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1c4100e20 0x7fd1c40fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:02.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.824+0000 7fd1c9a0b640 1 -- 172.21.15.164:0/1438838092 >> 172.21.15.164:0/1438838092 conn(0x7fd1c40fc7d0 msgr2=0x7fd1c4069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:02.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.824+0000 7fd1c9a0b640 1 -- 172.21.15.164:0/1438838092 shutdown_connections 2024-09-19T08:44:02.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:02.824+0000 7fd1c9a0b640 1 -- 172.21.15.164:0/1438838092 wait complete. 2024-09-19T08:44:03.125 INFO:journalctl@ceph.mds.b.smithi170.stdout:Sep 19 08:44:02 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-b[65018]: 2024-09-19T08:44:02.803+0000 7f2f742fb640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-19T08:44:03.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:02 smithi170 ceph-mon[35072]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T08:44:03.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:02 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-19T08:44:03.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:02 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-19T08:44:03.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:02 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-19T08:44:03.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:02 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-09-19T08:44:03.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:02 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-19T08:44:03.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:02 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-19T08:44:03.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:02 smithi170 ceph-mon[35072]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T08:44:03.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:02 smithi170 ceph-mon[35072]: mds.? [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] up:active 2024-09-19T08:44:03.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:02 smithi170 ceph-mon[35072]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T08:44:03.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:02 smithi170 ceph-mon[35072]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-19T08:44:03.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:02 smithi170 ceph-mon[35072]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-19T08:44:03.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:02 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/1438838092' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T08:44:03.148 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:02 smithi164 ceph-mon[30928]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T08:44:03.148 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:02 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-19T08:44:03.148 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:02 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-19T08:44:03.148 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:02 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-19T08:44:03.148 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:02 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-09-19T08:44:03.148 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:02 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-19T08:44:03.149 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:02 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-19T08:44:03.149 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:02 smithi164 ceph-mon[30928]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T08:44:03.149 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:02 smithi164 ceph-mon[30928]: mds.? [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] up:active 2024-09-19T08:44:03.149 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:02 smithi164 ceph-mon[30928]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T08:44:03.149 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:02 smithi164 ceph-mon[30928]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-19T08:44:03.149 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:02 smithi164 ceph-mon[30928]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-19T08:44:03.149 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:02 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/1438838092' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T08:44:03.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:02 smithi173 ceph-mon[35306]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T08:44:03.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:02 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-19T08:44:03.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:02 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-19T08:44:03.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:02 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-19T08:44:03.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:02 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-09-19T08:44:03.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:02 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-19T08:44:03.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:02 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-19T08:44:03.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:02 smithi173 ceph-mon[35306]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T08:44:03.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:02 smithi173 ceph-mon[35306]: mds.? [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] up:active 2024-09-19T08:44:03.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:02 smithi173 ceph-mon[35306]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T08:44:03.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:02 smithi173 ceph-mon[35306]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-19T08:44:03.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:02 smithi173 ceph-mon[35306]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-19T08:44:03.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:02 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/1438838092' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T08:44:03.534 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-19T08:44:03.783 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:04.068 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:03 smithi164 ceph-mon[30928]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T08:44:04.068 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:03 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/1438838092' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T08:44:04.068 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:03 smithi164 ceph-mon[30928]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-19T08:44:04.068 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:03 smithi164 ceph-mon[30928]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-19T08:44:04.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:03 smithi170 ceph-mon[35072]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T08:44:04.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:03 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/1438838092' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T08:44:04.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:03 smithi170 ceph-mon[35072]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-19T08:44:04.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:03 smithi170 ceph-mon[35072]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-19T08:44:04.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:03 smithi173 ceph-mon[35306]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T08:44:04.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:03 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/1438838092' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T08:44:04.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:03 smithi173 ceph-mon[35306]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-19T08:44:04.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:03 smithi173 ceph-mon[35306]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-19T08:44:04.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.268+0000 7f2986e73640 1 -- 172.21.15.164:0/2069561412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2980100a80 msgr2=0x7f2980100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:04.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.268+0000 7f2986e73640 1 --2- 172.21.15.164:0/2069561412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2980100a80 0x7f2980100ee0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f296c009930 tx=0x7f296c02a420 comp rx=0 tx=0).stop 2024-09-19T08:44:04.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.268+0000 7f2986e73640 1 -- 172.21.15.164:0/2069561412 shutdown_connections 2024-09-19T08:44:04.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.268+0000 7f2986e73640 1 --2- 172.21.15.164:0/2069561412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2980101420 0x7f298010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.268+0000 7f2986e73640 1 --2- 172.21.15.164:0/2069561412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2980100a80 0x7f2980100ee0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.268+0000 7f2986e73640 1 --2- 172.21.15.164:0/2069561412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2980106a80 0x7f2980106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.268+0000 7f2986e73640 1 -- 172.21.15.164:0/2069561412 >> 172.21.15.164:0/2069561412 conn(0x7f29800fc7a0 msgr2=0x7f29800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:04.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.268+0000 7f2986e73640 1 -- 172.21.15.164:0/2069561412 shutdown_connections 2024-09-19T08:44:04.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.268+0000 7f2986e73640 1 -- 172.21.15.164:0/2069561412 wait complete. 2024-09-19T08:44:04.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.269+0000 7f2986e73640 1 Processor -- start 2024-09-19T08:44:04.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.269+0000 7f2986e73640 1 -- start start 2024-09-19T08:44:04.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.270+0000 7f2986e73640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2980100a80 0x7f29801985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:04.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.270+0000 7f2986e73640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2980101420 0x7f2980198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:04.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.270+0000 7f2986e73640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2980106a80 0x7f298019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:04.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.270+0000 7f2986e73640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2980110f20 con 0x7f2980106a80 2024-09-19T08:44:04.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.270+0000 7f2986e73640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2980110da0 con 0x7f2980100a80 2024-09-19T08:44:04.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.270+0000 7f2986e73640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f29801110a0 con 0x7f2980101420 2024-09-19T08:44:04.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.270+0000 7f2985410640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2980106a80 0x7f298019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:04.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.270+0000 7f2985410640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2980106a80 0x7f298019ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59952/0 (socket says 172.21.15.164:59952) 2024-09-19T08:44:04.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.270+0000 7f2985410640 1 -- 172.21.15.164:0/11719387 learned_addr learned my addr 172.21.15.164:0/11719387 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:04.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.270+0000 7f297ffff640 1 --2- 172.21.15.164:0/11719387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2980101420 0x7f2980198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:04.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.270+0000 7f2984c0f640 1 --2- 172.21.15.164:0/11719387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2980100a80 0x7f29801985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:04.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.270+0000 7f2985410640 1 -- 172.21.15.164:0/11719387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2980101420 msgr2=0x7f2980198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:04.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.270+0000 7f2985410640 1 --2- 172.21.15.164:0/11719387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2980101420 0x7f2980198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.270+0000 7f2985410640 1 -- 172.21.15.164:0/11719387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2980100a80 msgr2=0x7f29801985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:04.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.270+0000 7f2985410640 1 --2- 172.21.15.164:0/11719387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2980100a80 0x7f29801985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.270+0000 7f2985410640 1 -- 172.21.15.164:0/11719387 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f298019d560 con 0x7f2980106a80 2024-09-19T08:44:04.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.270+0000 7f2984c0f640 1 --2- 172.21.15.164:0/11719387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2980100a80 0x7f29801985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:04.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.270+0000 7f2985410640 1 --2- 172.21.15.164:0/11719387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2980106a80 0x7f298019ce80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f297800b970 tx=0x7f297800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:04.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.270+0000 7f297dffb640 1 -- 172.21.15.164:0/11719387 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2978019070 con 0x7f2980106a80 2024-09-19T08:44:04.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.271+0000 7f2986e73640 1 -- 172.21.15.164:0/11719387 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f298019d850 con 0x7f2980106a80 2024-09-19T08:44:04.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.271+0000 7f297dffb640 1 -- 172.21.15.164:0/11719387 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f297800cb40 con 0x7f2980106a80 2024-09-19T08:44:04.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.271+0000 7f297dffb640 1 -- 172.21.15.164:0/11719387 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2978007650 con 0x7f2980106a80 2024-09-19T08:44:04.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.271+0000 7f2986e73640 1 -- 172.21.15.164:0/11719387 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29801a5130 con 0x7f2980106a80 2024-09-19T08:44:04.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.272+0000 7f2986e73640 1 -- 172.21.15.164:0/11719387 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2980100f60 con 0x7f2980106a80 2024-09-19T08:44:04.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.273+0000 7f297dffb640 1 -- 172.21.15.164:0/11719387 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f297801c070 con 0x7f2980106a80 2024-09-19T08:44:04.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.273+0000 7f297dffb640 1 --2- 172.21.15.164:0/11719387 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f29700a3240 0x7f29700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:04.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.273+0000 7f2984c0f640 1 --2- 172.21.15.164:0/11719387 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f29700a3240 0x7f29700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:04.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.273+0000 7f297dffb640 1 -- 172.21.15.164:0/11719387 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f29780cee60 con 0x7f2980106a80 2024-09-19T08:44:04.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.274+0000 7f2984c0f640 1 --2- 172.21.15.164:0/11719387 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f29700a3240 0x7f29700a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2974006fd0 tx=0x7f2974008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:04.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.277+0000 7f297dffb640 1 -- 172.21.15.164:0/11719387 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f297809b5d0 con 0x7f2980106a80 2024-09-19T08:44:04.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.479+0000 7f2986e73640 1 -- 172.21.15.164:0/11719387 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f298010a080 con 0x7f2980106a80 2024-09-19T08:44:04.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.621+0000 7f297dffb640 1 -- 172.21.15.164:0/11719387 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v24) ==== 115+0+0 (secure 0 0 0) 0x7f297809f8d0 con 0x7f2980106a80 2024-09-19T08:44:04.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.623+0000 7f2986e73640 1 -- 172.21.15.164:0/11719387 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f29700a3240 msgr2=0x7f29700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:04.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.623+0000 7f2986e73640 1 --2- 172.21.15.164:0/11719387 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f29700a3240 0x7f29700a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2974006fd0 tx=0x7f2974008040 comp rx=0 tx=0).stop 2024-09-19T08:44:04.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.623+0000 7f2986e73640 1 -- 172.21.15.164:0/11719387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2980106a80 msgr2=0x7f298019ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:04.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.623+0000 7f2986e73640 1 --2- 172.21.15.164:0/11719387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2980106a80 0x7f298019ce80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f297800b970 tx=0x7f297800be30 comp rx=0 tx=0).stop 2024-09-19T08:44:04.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.623+0000 7f2986e73640 1 -- 172.21.15.164:0/11719387 shutdown_connections 2024-09-19T08:44:04.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.623+0000 7f2986e73640 1 --2- 172.21.15.164:0/11719387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2980106a80 0x7f298019ce80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.623+0000 7f2986e73640 1 --2- 172.21.15.164:0/11719387 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f29700a3240 0x7f29700a5700 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.623+0000 7f2986e73640 1 --2- 172.21.15.164:0/11719387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2980101420 0x7f2980198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.623+0000 7f2986e73640 1 --2- 172.21.15.164:0/11719387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2980100a80 0x7f29801985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.623+0000 7f2986e73640 1 -- 172.21.15.164:0/11719387 >> 172.21.15.164:0/11719387 conn(0x7f29800fc7a0 msgr2=0x7f298006aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:04.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.623+0000 7f2986e73640 1 -- 172.21.15.164:0/11719387 shutdown_connections 2024-09-19T08:44:04.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:04.623+0000 7f2986e73640 1 -- 172.21.15.164:0/11719387 wait complete. 2024-09-19T08:44:05.099 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:04 smithi164 ceph-mon[30928]: pgmap v283: 97 pgs: 2 peering, 95 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 3 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T08:44:05.099 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:04 smithi164 ceph-mon[30928]: mds.? [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] up:active 2024-09-19T08:44:05.099 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:04 smithi164 ceph-mon[30928]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T08:44:05.099 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:04 smithi164 ceph-mon[30928]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-19T08:44:05.099 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:04 smithi164 ceph-mon[30928]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-09-19T08:44:05.099 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:04 smithi164 ceph-mon[30928]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-19T08:44:05.099 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:04 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/11719387' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T08:44:05.099 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:04 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/11719387' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T08:44:05.100 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:04 smithi164 ceph-mon[30928]: mds.? [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] up:active 2024-09-19T08:44:05.100 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:04 smithi164 ceph-mon[30928]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T08:44:05.100 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:04 smithi164 ceph-mon[30928]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T08:44:05.100 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:04 smithi164 ceph-mon[30928]: Cluster is now healthy 2024-09-19T08:44:05.100 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:04 smithi164 ceph-mon[30928]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 4 up:standby-replay 4 up:standby 2024-09-19T08:44:05.100 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:04 smithi164 ceph-mon[30928]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-09-19T08:44:05.100 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:04 smithi164 ceph-mon[30928]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-09-19T08:44:05.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:04 smithi170 ceph-mon[35072]: pgmap v283: 97 pgs: 2 peering, 95 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 3 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T08:44:05.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:04 smithi170 ceph-mon[35072]: mds.? [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] up:active 2024-09-19T08:44:05.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:04 smithi170 ceph-mon[35072]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T08:44:05.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:04 smithi170 ceph-mon[35072]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-19T08:44:05.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:04 smithi170 ceph-mon[35072]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-09-19T08:44:05.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:04 smithi170 ceph-mon[35072]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-19T08:44:05.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:04 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/11719387' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T08:44:05.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:04 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/11719387' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T08:44:05.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:04 smithi170 ceph-mon[35072]: mds.? [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] up:active 2024-09-19T08:44:05.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:04 smithi170 ceph-mon[35072]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T08:44:05.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:04 smithi170 ceph-mon[35072]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T08:44:05.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:04 smithi170 ceph-mon[35072]: Cluster is now healthy 2024-09-19T08:44:05.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:04 smithi170 ceph-mon[35072]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 4 up:standby-replay 4 up:standby 2024-09-19T08:44:05.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:04 smithi170 ceph-mon[35072]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-09-19T08:44:05.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:04 smithi170 ceph-mon[35072]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-09-19T08:44:05.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:04 smithi173 ceph-mon[35306]: pgmap v283: 97 pgs: 2 peering, 95 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 3 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T08:44:05.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:04 smithi173 ceph-mon[35306]: mds.? [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] up:active 2024-09-19T08:44:05.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:04 smithi173 ceph-mon[35306]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T08:44:05.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:04 smithi173 ceph-mon[35306]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-19T08:44:05.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:04 smithi173 ceph-mon[35306]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-09-19T08:44:05.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:04 smithi173 ceph-mon[35306]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-19T08:44:05.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:04 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/11719387' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T08:44:05.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:04 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/11719387' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T08:44:05.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:04 smithi173 ceph-mon[35306]: mds.? [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] up:active 2024-09-19T08:44:05.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:04 smithi173 ceph-mon[35306]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T08:44:05.163 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:04 smithi173 ceph-mon[35306]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T08:44:05.163 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:04 smithi173 ceph-mon[35306]: Cluster is now healthy 2024-09-19T08:44:05.163 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:04 smithi173 ceph-mon[35306]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 4 up:standby-replay 4 up:standby 2024-09-19T08:44:05.163 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:04 smithi173 ceph-mon[35306]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-09-19T08:44:05.163 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:04 smithi173 ceph-mon[35306]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-09-19T08:44:05.444 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-09-19T08:44:05.444 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-19T08:44:05.693 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:06.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.142+0000 7fede3183640 1 -- 172.21.15.164:0/4049024017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feddc106a80 msgr2=0x7feddc106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:06.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.142+0000 7fede3183640 1 --2- 172.21.15.164:0/4049024017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feddc106a80 0x7feddc106e60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fedd0009930 tx=0x7fedd002a420 comp rx=0 tx=0).stop 2024-09-19T08:44:06.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.142+0000 7fede3183640 1 -- 172.21.15.164:0/4049024017 shutdown_connections 2024-09-19T08:44:06.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.142+0000 7fede3183640 1 --2- 172.21.15.164:0/4049024017 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feddc101420 0x7feddc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:06.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.142+0000 7fede3183640 1 --2- 172.21.15.164:0/4049024017 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feddc100a80 0x7feddc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:06.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.142+0000 7fede3183640 1 --2- 172.21.15.164:0/4049024017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feddc106a80 0x7feddc106e60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:06.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.142+0000 7fede3183640 1 -- 172.21.15.164:0/4049024017 >> 172.21.15.164:0/4049024017 conn(0x7feddc0fc7a0 msgr2=0x7feddc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:06.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.142+0000 7fede3183640 1 -- 172.21.15.164:0/4049024017 shutdown_connections 2024-09-19T08:44:06.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.142+0000 7fede3183640 1 -- 172.21.15.164:0/4049024017 wait complete. 2024-09-19T08:44:06.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.143+0000 7fede3183640 1 Processor -- start 2024-09-19T08:44:06.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.143+0000 7fede3183640 1 -- start start 2024-09-19T08:44:06.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.143+0000 7fede3183640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feddc100a80 0x7feddc199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:06.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.143+0000 7fede3183640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feddc101420 0x7feddc199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:06.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.143+0000 7fede3183640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feddc106a80 0x7feddc1a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:06.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.143+0000 7fede3183640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feddc110e80 con 0x7feddc106a80 2024-09-19T08:44:06.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.143+0000 7fede3183640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7feddc110d00 con 0x7feddc100a80 2024-09-19T08:44:06.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.143+0000 7fede3183640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feddc111000 con 0x7feddc101420 2024-09-19T08:44:06.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.143+0000 7fede1720640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feddc106a80 0x7feddc1a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:06.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.143+0000 7fede1720640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feddc106a80 0x7feddc1a4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59978/0 (socket says 172.21.15.164:59978) 2024-09-19T08:44:06.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.143+0000 7fede1720640 1 -- 172.21.15.164:0/631086987 learned_addr learned my addr 172.21.15.164:0/631086987 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:06.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.143+0000 7fede0f1f640 1 --2- 172.21.15.164:0/631086987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feddc100a80 0x7feddc199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:06.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.143+0000 7feddbfff640 1 --2- 172.21.15.164:0/631086987 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feddc101420 0x7feddc199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:06.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.144+0000 7fede1720640 1 -- 172.21.15.164:0/631086987 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feddc101420 msgr2=0x7feddc199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:06.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.144+0000 7fede1720640 1 --2- 172.21.15.164:0/631086987 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feddc101420 0x7feddc199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:06.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.144+0000 7fede1720640 1 -- 172.21.15.164:0/631086987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feddc100a80 msgr2=0x7feddc199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:06.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.144+0000 7fede1720640 1 --2- 172.21.15.164:0/631086987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feddc100a80 0x7feddc199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:06.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.144+0000 7fede1720640 1 -- 172.21.15.164:0/631086987 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feddc1a48d0 con 0x7feddc106a80 2024-09-19T08:44:06.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.144+0000 7fede1720640 1 --2- 172.21.15.164:0/631086987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feddc106a80 0x7feddc1a4390 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fedd400eaa0 tx=0x7fedd400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:06.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.144+0000 7fedd9ffb640 1 -- 172.21.15.164:0/631086987 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd400cdb0 con 0x7feddc106a80 2024-09-19T08:44:06.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.144+0000 7fedd9ffb640 1 -- 172.21.15.164:0/631086987 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedd4005020 con 0x7feddc106a80 2024-09-19T08:44:06.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.144+0000 7fede3183640 1 -- 172.21.15.164:0/631086987 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feddc1a4aa0 con 0x7feddc106a80 2024-09-19T08:44:06.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.144+0000 7fedd9ffb640 1 -- 172.21.15.164:0/631086987 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd40116b0 con 0x7feddc106a80 2024-09-19T08:44:06.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.144+0000 7fede3183640 1 -- 172.21.15.164:0/631086987 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feddc1a4fb0 con 0x7feddc106a80 2024-09-19T08:44:06.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.145+0000 7fede3183640 1 -- 172.21.15.164:0/631086987 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feddc100ee0 con 0x7feddc106a80 2024-09-19T08:44:06.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.146+0000 7fedd9ffb640 1 -- 172.21.15.164:0/631086987 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fedd4005560 con 0x7feddc106a80 2024-09-19T08:44:06.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.147+0000 7fedd9ffb640 1 --2- 172.21.15.164:0/631086987 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fedcc0a32c0 0x7fedcc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:06.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.147+0000 7fede0f1f640 1 --2- 172.21.15.164:0/631086987 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fedcc0a32c0 0x7fedcc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:06.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.147+0000 7fedd9ffb640 1 -- 172.21.15.164:0/631086987 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fedd40cfa00 con 0x7feddc106a80 2024-09-19T08:44:06.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.147+0000 7fede0f1f640 1 --2- 172.21.15.164:0/631086987 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fedcc0a32c0 0x7fedcc0a5780 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fedd0009900 tx=0x7fedd00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:06.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.151+0000 7fedd9ffb640 1 -- 172.21.15.164:0/631086987 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fedd4016030 con 0x7feddc106a80 2024-09-19T08:44:06.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.344+0000 7fede3183640 1 -- 172.21.15.164:0/631086987 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7feddc062630 con 0x7fedcc0a32c0 2024-09-19T08:44:06.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.370+0000 7fedd9ffb640 1 -- 172.21.15.164:0/631086987 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7feddc062630 con 0x7fedcc0a32c0 2024-09-19T08:44:06.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.372+0000 7fede3183640 1 -- 172.21.15.164:0/631086987 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fedcc0a32c0 msgr2=0x7fedcc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:06.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.372+0000 7fede3183640 1 --2- 172.21.15.164:0/631086987 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fedcc0a32c0 0x7fedcc0a5780 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fedd0009900 tx=0x7fedd00058d0 comp rx=0 tx=0).stop 2024-09-19T08:44:06.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.372+0000 7fede3183640 1 -- 172.21.15.164:0/631086987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feddc106a80 msgr2=0x7feddc1a4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:06.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.372+0000 7fede3183640 1 --2- 172.21.15.164:0/631086987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feddc106a80 0x7feddc1a4390 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fedd400eaa0 tx=0x7fedd400ef60 comp rx=0 tx=0).stop 2024-09-19T08:44:06.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.373+0000 7fede3183640 1 -- 172.21.15.164:0/631086987 shutdown_connections 2024-09-19T08:44:06.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.373+0000 7fede3183640 1 --2- 172.21.15.164:0/631086987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feddc106a80 0x7feddc1a4390 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:06.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.373+0000 7fede3183640 1 --2- 172.21.15.164:0/631086987 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fedcc0a32c0 0x7fedcc0a5780 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:06.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.373+0000 7fede3183640 1 --2- 172.21.15.164:0/631086987 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feddc101420 0x7feddc199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:06.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.373+0000 7fede3183640 1 --2- 172.21.15.164:0/631086987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feddc100a80 0x7feddc199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:06.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.373+0000 7fede3183640 1 -- 172.21.15.164:0/631086987 >> 172.21.15.164:0/631086987 conn(0x7feddc0fc7a0 msgr2=0x7feddc0fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:06.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.373+0000 7fede3183640 1 -- 172.21.15.164:0/631086987 shutdown_connections 2024-09-19T08:44:06.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:06.373+0000 7fede3183640 1 -- 172.21.15.164:0/631086987 wait complete. 2024-09-19T08:44:06.629 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:44:06 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:44:06.349+0000 7f0247d8c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:44:06.630 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:44:06 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:44:06.349+0000 7f0247d8c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:44:06.630 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:44:06 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:44:06.349+0000 7f0247d8c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:44:06.630 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:44:06 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:44:06.349+0000 7f0247d8c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:44:06.630 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:44:06 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:44:06.349+0000 7f0247d8c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:44:06.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:06 smithi173 ceph-mon[35306]: mds.? [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] up:active 2024-09-19T08:44:06.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:06 smithi173 ceph-mon[35306]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-19T08:44:06.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:06 smithi173 ceph-mon[35306]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T08:44:06.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:06 smithi173 ceph-mon[35306]: pgmap v284: 97 pgs: 2 peering, 95 active+clean; 583 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4.2 KiB/s wr, 14 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T08:44:07.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:06 smithi164 ceph-mon[30928]: mds.? [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] up:active 2024-09-19T08:44:07.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:06 smithi164 ceph-mon[30928]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-19T08:44:07.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:06 smithi164 ceph-mon[30928]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T08:44:07.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:06 smithi164 ceph-mon[30928]: pgmap v284: 97 pgs: 2 peering, 95 active+clean; 583 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4.2 KiB/s wr, 14 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T08:44:07.103 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-19T08:44:07.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:06 smithi170 ceph-mon[35072]: mds.? [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] up:active 2024-09-19T08:44:07.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:06 smithi170 ceph-mon[35072]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-19T08:44:07.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:06 smithi170 ceph-mon[35072]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T08:44:07.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:06 smithi170 ceph-mon[35072]: pgmap v284: 97 pgs: 2 peering, 95 active+clean; 583 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4.2 KiB/s wr, 14 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T08:44:07.351 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:07.781 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:07 smithi164 ceph-mon[30928]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:44:07.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.789+0000 7fdd8b207640 1 -- 172.21.15.164:0/2546759437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd84101420 msgr2=0x7fdd8410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:07.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.789+0000 7fdd8b207640 1 --2- 172.21.15.164:0/2546759437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd84101420 0x7fdd8410d950 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c009cd0 tx=0x7fdd7c02a8e0 comp rx=0 tx=0).stop 2024-09-19T08:44:07.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.789+0000 7fdd8b207640 1 -- 172.21.15.164:0/2546759437 shutdown_connections 2024-09-19T08:44:07.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.789+0000 7fdd8b207640 1 --2- 172.21.15.164:0/2546759437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd84101420 0x7fdd8410d950 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:07.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.789+0000 7fdd8b207640 1 --2- 172.21.15.164:0/2546759437 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd84100a80 0x7fdd84100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:07.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.789+0000 7fdd8b207640 1 --2- 172.21.15.164:0/2546759437 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd84106a80 0x7fdd84106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:07.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.789+0000 7fdd8b207640 1 -- 172.21.15.164:0/2546759437 >> 172.21.15.164:0/2546759437 conn(0x7fdd840fc7a0 msgr2=0x7fdd840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:07.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.789+0000 7fdd8b207640 1 -- 172.21.15.164:0/2546759437 shutdown_connections 2024-09-19T08:44:07.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.789+0000 7fdd8b207640 1 -- 172.21.15.164:0/2546759437 wait complete. 2024-09-19T08:44:07.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.790+0000 7fdd8b207640 1 Processor -- start 2024-09-19T08:44:07.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.790+0000 7fdd8b207640 1 -- start start 2024-09-19T08:44:07.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.790+0000 7fdd8b207640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd84100a80 0x7fdd841985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:07.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.790+0000 7fdd8b207640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd84101420 0x7fdd84198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:07.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.790+0000 7fdd8b207640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd84106a80 0x7fdd8419ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:07.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.790+0000 7fdd8b207640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdd84112020 con 0x7fdd84100a80 2024-09-19T08:44:07.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.790+0000 7fdd8b207640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdd84111ea0 con 0x7fdd84101420 2024-09-19T08:44:07.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.790+0000 7fdd8b207640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdd841121a0 con 0x7fdd84106a80 2024-09-19T08:44:07.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.790+0000 7fdd88fa3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd84100a80 0x7fdd841985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:07.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.790+0000 7fdd88fa3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd84100a80 0x7fdd841985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60022/0 (socket says 172.21.15.164:60022) 2024-09-19T08:44:07.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.790+0000 7fdd88fa3640 1 -- 172.21.15.164:0/2740595112 learned_addr learned my addr 172.21.15.164:0/2740595112 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:07.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.790+0000 7fdd83fff640 1 --2- 172.21.15.164:0/2740595112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd84101420 0x7fdd84198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:07.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.790+0000 7fdd88fa3640 1 -- 172.21.15.164:0/2740595112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd84106a80 msgr2=0x7fdd8419ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:07.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.790+0000 7fdd897a4640 1 --2- 172.21.15.164:0/2740595112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd84106a80 0x7fdd8419ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:07.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.790+0000 7fdd88fa3640 1 --2- 172.21.15.164:0/2740595112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd84106a80 0x7fdd8419ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:07.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.790+0000 7fdd88fa3640 1 -- 172.21.15.164:0/2740595112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd84101420 msgr2=0x7fdd84198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:07.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.790+0000 7fdd88fa3640 1 --2- 172.21.15.164:0/2740595112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd84101420 0x7fdd84198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:07.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.790+0000 7fdd88fa3640 1 -- 172.21.15.164:0/2740595112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd8419d550 con 0x7fdd84100a80 2024-09-19T08:44:07.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.790+0000 7fdd897a4640 1 --2- 172.21.15.164:0/2740595112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd84106a80 0x7fdd8419ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:07.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.790+0000 7fdd83fff640 1 --2- 172.21.15.164:0/2740595112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd84101420 0x7fdd84198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:44:07.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.790+0000 7fdd88fa3640 1 --2- 172.21.15.164:0/2740595112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd84100a80 0x7fdd841985a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdd7800b4a0 tx=0x7fdd7800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:07.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.791+0000 7fdd81ffb640 1 -- 172.21.15.164:0/2740595112 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd78014070 con 0x7fdd84100a80 2024-09-19T08:44:07.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.791+0000 7fdd8b207640 1 -- 172.21.15.164:0/2740595112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd8419d840 con 0x7fdd84100a80 2024-09-19T08:44:07.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.791+0000 7fdd81ffb640 1 -- 172.21.15.164:0/2740595112 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd78010af0 con 0x7fdd84100a80 2024-09-19T08:44:07.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.791+0000 7fdd81ffb640 1 -- 172.21.15.164:0/2740595112 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd78017d10 con 0x7fdd84100a80 2024-09-19T08:44:07.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.791+0000 7fdd8b207640 1 -- 172.21.15.164:0/2740595112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd841a5120 con 0x7fdd84100a80 2024-09-19T08:44:07.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.792+0000 7fdd8b207640 1 -- 172.21.15.164:0/2740595112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd84100f60 con 0x7fdd84100a80 2024-09-19T08:44:07.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.793+0000 7fdd81ffb640 1 -- 172.21.15.164:0/2740595112 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fdd78017070 con 0x7fdd84100a80 2024-09-19T08:44:07.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.793+0000 7fdd81ffb640 1 --2- 172.21.15.164:0/2740595112 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fdd740a3240 0x7fdd740a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:07.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.793+0000 7fdd83fff640 1 --2- 172.21.15.164:0/2740595112 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fdd740a3240 0x7fdd740a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:07.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.793+0000 7fdd81ffb640 1 -- 172.21.15.164:0/2740595112 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fdd780ced90 con 0x7fdd84100a80 2024-09-19T08:44:07.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.794+0000 7fdd83fff640 1 --2- 172.21.15.164:0/2740595112 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fdd740a3240 0x7fdd740a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdd840fede0 tx=0x7fdd70008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:07.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.798+0000 7fdd81ffb640 1 -- 172.21.15.164:0/2740595112 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdd7809bab0 con 0x7fdd84100a80 2024-09-19T08:44:07.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:07 smithi173 ceph-mon[35306]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:44:07.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:07.989+0000 7fdd8b207640 1 -- 172.21.15.164:0/2740595112 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fdd84062630 con 0x7fdd740a3240 2024-09-19T08:44:08.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:08.013+0000 7fdd81ffb640 1 -- 172.21.15.164:0/2740595112 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdd84062630 con 0x7fdd740a3240 2024-09-19T08:44:08.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:08.015+0000 7fdd8b207640 1 -- 172.21.15.164:0/2740595112 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fdd740a3240 msgr2=0x7fdd740a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:08.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:08.015+0000 7fdd8b207640 1 --2- 172.21.15.164:0/2740595112 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fdd740a3240 0x7fdd740a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdd840fede0 tx=0x7fdd70008040 comp rx=0 tx=0).stop 2024-09-19T08:44:08.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:08.015+0000 7fdd8b207640 1 -- 172.21.15.164:0/2740595112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd84100a80 msgr2=0x7fdd841985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:08.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:08.015+0000 7fdd8b207640 1 --2- 172.21.15.164:0/2740595112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd84100a80 0x7fdd841985a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdd7800b4a0 tx=0x7fdd7800b960 comp rx=0 tx=0).stop 2024-09-19T08:44:08.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:08.016+0000 7fdd8b207640 1 -- 172.21.15.164:0/2740595112 shutdown_connections 2024-09-19T08:44:08.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:08.016+0000 7fdd8b207640 1 --2- 172.21.15.164:0/2740595112 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fdd740a3240 0x7fdd740a5700 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:08.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:08.016+0000 7fdd8b207640 1 --2- 172.21.15.164:0/2740595112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd84106a80 0x7fdd8419ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:08.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:08.016+0000 7fdd8b207640 1 --2- 172.21.15.164:0/2740595112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd84101420 0x7fdd84198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:08.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:08.016+0000 7fdd8b207640 1 --2- 172.21.15.164:0/2740595112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd84100a80 0x7fdd841985a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:08.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:08.016+0000 7fdd8b207640 1 -- 172.21.15.164:0/2740595112 >> 172.21.15.164:0/2740595112 conn(0x7fdd840fc7a0 msgr2=0x7fdd840fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:08.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:08.016+0000 7fdd8b207640 1 -- 172.21.15.164:0/2740595112 shutdown_connections 2024-09-19T08:44:08.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:08.016+0000 7fdd8b207640 1 -- 172.21.15.164:0/2740595112 wait complete. 2024-09-19T08:44:08.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:07 smithi170 ceph-mon[35072]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:44:08.125 INFO:journalctl@ceph.mds.b.smithi170.stdout:Sep 19 08:44:07 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-b[65018]: 2024-09-19T08:44:07.803+0000 7f2f742fb640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-09-19T08:44:08.125 INFO:journalctl@ceph.mds.b.smithi170.stdout:Sep 19 08:44:07 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-b[65018]: 2024-09-19T08:44:07.803+0000 7f2f742fb640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-19T08:44:08.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:08 smithi164 ceph-mon[30928]: pgmap v285: 97 pgs: 97 active+clean; 584 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 5.7 KiB/s wr, 22 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T08:44:08.811 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-19T08:44:08.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:08 smithi173 ceph-mon[35306]: pgmap v285: 97 pgs: 97 active+clean; 584 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 5.7 KiB/s wr, 22 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T08:44:09.055 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:09.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:08 smithi170 ceph-mon[35072]: pgmap v285: 97 pgs: 97 active+clean; 584 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 5.7 KiB/s wr, 22 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T08:44:09.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.505+0000 7ff0a3647640 1 -- 172.21.15.164:0/1738512211 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff09c102570 msgr2=0x7ff09c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:09.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.505+0000 7ff0a3647640 1 --2- 172.21.15.164:0/1738512211 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff09c102570 0x7ff09c1029b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009900 tx=0x7ff08c02a480 comp rx=0 tx=0).stop 2024-09-19T08:44:09.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.505+0000 7ff0a3647640 1 -- 172.21.15.164:0/1738512211 shutdown_connections 2024-09-19T08:44:09.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.505+0000 7ff0a3647640 1 --2- 172.21.15.164:0/1738512211 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff09c102ef0 0x7ff09c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:09.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.505+0000 7ff0a3647640 1 --2- 172.21.15.164:0/1738512211 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff09c102570 0x7ff09c1029b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:09.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.505+0000 7ff0a3647640 1 --2- 172.21.15.164:0/1738512211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff09c100e20 0x7ff09c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:09.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.505+0000 7ff0a3647640 1 -- 172.21.15.164:0/1738512211 >> 172.21.15.164:0/1738512211 conn(0x7ff09c0fc7d0 msgr2=0x7ff09c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:09.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.505+0000 7ff0a3647640 1 -- 172.21.15.164:0/1738512211 shutdown_connections 2024-09-19T08:44:09.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.505+0000 7ff0a3647640 1 -- 172.21.15.164:0/1738512211 wait complete. 2024-09-19T08:44:09.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.506+0000 7ff0a3647640 1 Processor -- start 2024-09-19T08:44:09.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.506+0000 7ff0a3647640 1 -- start start 2024-09-19T08:44:09.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.506+0000 7ff0a3647640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff09c100e20 0x7ff09c199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:09.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.506+0000 7ff0a3647640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff09c102570 0x7ff09c199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:09.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.506+0000 7ff0a3647640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff09c102ef0 0x7ff09c1a44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:09.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.506+0000 7ff0a3647640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c110fa0 con 0x7ff09c102570 2024-09-19T08:44:09.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.506+0000 7ff0a3647640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c110e20 con 0x7ff09c102ef0 2024-09-19T08:44:09.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.506+0000 7ff0a3647640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c111120 con 0x7ff09c100e20 2024-09-19T08:44:09.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.506+0000 7ff0a0be2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff09c102570 0x7ff09c199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:09.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.506+0000 7ff0a0be2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff09c102570 0x7ff09c199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60040/0 (socket says 172.21.15.164:60040) 2024-09-19T08:44:09.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.506+0000 7ff0a0be2640 1 -- 172.21.15.164:0/45088677 learned_addr learned my addr 172.21.15.164:0/45088677 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:09.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.506+0000 7ff0a13e3640 1 --2- 172.21.15.164:0/45088677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff09c100e20 0x7ff09c199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:09.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.506+0000 7ff0a1be4640 1 --2- 172.21.15.164:0/45088677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff09c102ef0 0x7ff09c1a44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:09.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.507+0000 7ff0a0be2640 1 -- 172.21.15.164:0/45088677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff09c100e20 msgr2=0x7ff09c199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:09.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.507+0000 7ff0a0be2640 1 --2- 172.21.15.164:0/45088677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff09c100e20 0x7ff09c199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:09.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.507+0000 7ff0a0be2640 1 -- 172.21.15.164:0/45088677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff09c102ef0 msgr2=0x7ff09c1a44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:09.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.507+0000 7ff0a0be2640 1 --2- 172.21.15.164:0/45088677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff09c102ef0 0x7ff09c1a44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:09.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.507+0000 7ff0a0be2640 1 -- 172.21.15.164:0/45088677 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff09c1a4a00 con 0x7ff09c102570 2024-09-19T08:44:09.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.507+0000 7ff0a1be4640 1 --2- 172.21.15.164:0/45088677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff09c102ef0 0x7ff09c1a44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:44:09.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.507+0000 7ff0a0be2640 1 --2- 172.21.15.164:0/45088677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff09c102570 0x7ff09c199980 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff08c006010 tx=0x7ff08c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:09.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.507+0000 7ff0927fc640 1 -- 172.21.15.164:0/45088677 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c03f070 con 0x7ff09c102570 2024-09-19T08:44:09.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.507+0000 7ff0927fc640 1 -- 172.21.15.164:0/45088677 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff08c031e30 con 0x7ff09c102570 2024-09-19T08:44:09.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.507+0000 7ff0a3647640 1 -- 172.21.15.164:0/45088677 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff09c1a4bd0 con 0x7ff09c102570 2024-09-19T08:44:09.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.507+0000 7ff0927fc640 1 -- 172.21.15.164:0/45088677 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c035030 con 0x7ff09c102570 2024-09-19T08:44:09.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.507+0000 7ff0a3647640 1 -- 172.21.15.164:0/45088677 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff09c1a5080 con 0x7ff09c102570 2024-09-19T08:44:09.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.508+0000 7ff0a3647640 1 -- 172.21.15.164:0/45088677 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff09c101200 con 0x7ff09c102570 2024-09-19T08:44:09.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.509+0000 7ff0927fc640 1 -- 172.21.15.164:0/45088677 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7ff08c042070 con 0x7ff09c102570 2024-09-19T08:44:09.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.510+0000 7ff0927fc640 1 --2- 172.21.15.164:0/45088677 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7ff0880a3350 0x7ff0880a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:09.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.510+0000 7ff0a13e3640 1 --2- 172.21.15.164:0/45088677 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7ff0880a3350 0x7ff0880a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:09.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.510+0000 7ff0927fc640 1 -- 172.21.15.164:0/45088677 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff08c0f0230 con 0x7ff09c102570 2024-09-19T08:44:09.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.510+0000 7ff0a13e3640 1 --2- 172.21.15.164:0/45088677 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7ff0880a3350 0x7ff0880a5810 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff098009990 tx=0x7ff098009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:09.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.514+0000 7ff0927fc640 1 -- 172.21.15.164:0/45088677 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff08c0bcef0 con 0x7ff09c102570 2024-09-19T08:44:09.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.716+0000 7ff0a3647640 1 -- 172.21.15.164:0/45088677 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7ff09c062630 con 0x7ff0880a3350 2024-09-19T08:44:09.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.720+0000 7ff0927fc640 1 -- 172.21.15.164:0/45088677 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7ff09c062630 con 0x7ff0880a3350 2024-09-19T08:44:09.721 INFO:teuthology.orchestra.run.smithi164.stdout:/volumes/qa/sv_0/639cb283-95e6-4acb-a56a-1b7b6163ef1e 2024-09-19T08:44:09.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.723+0000 7ff0a3647640 1 -- 172.21.15.164:0/45088677 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7ff0880a3350 msgr2=0x7ff0880a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:09.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.723+0000 7ff0a3647640 1 --2- 172.21.15.164:0/45088677 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7ff0880a3350 0x7ff0880a5810 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff098009990 tx=0x7ff098009110 comp rx=0 tx=0).stop 2024-09-19T08:44:09.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.723+0000 7ff0a3647640 1 -- 172.21.15.164:0/45088677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff09c102570 msgr2=0x7ff09c199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:09.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.723+0000 7ff0a3647640 1 --2- 172.21.15.164:0/45088677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff09c102570 0x7ff09c199980 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff08c006010 tx=0x7ff08c003c80 comp rx=0 tx=0).stop 2024-09-19T08:44:09.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.723+0000 7ff0a3647640 1 -- 172.21.15.164:0/45088677 shutdown_connections 2024-09-19T08:44:09.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.723+0000 7ff0a3647640 1 --2- 172.21.15.164:0/45088677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff09c102ef0 0x7ff09c1a44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:09.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.723+0000 7ff0a3647640 1 --2- 172.21.15.164:0/45088677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff09c102570 0x7ff09c199980 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:09.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.723+0000 7ff0a3647640 1 --2- 172.21.15.164:0/45088677 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7ff0880a3350 0x7ff0880a5810 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:09.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.723+0000 7ff0a3647640 1 --2- 172.21.15.164:0/45088677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff09c100e20 0x7ff09c199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:09.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.723+0000 7ff0a3647640 1 -- 172.21.15.164:0/45088677 >> 172.21.15.164:0/45088677 conn(0x7ff09c0fc7d0 msgr2=0x7ff09c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:09.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.723+0000 7ff0a3647640 1 -- 172.21.15.164:0/45088677 shutdown_connections 2024-09-19T08:44:09.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:09.723+0000 7ff0a3647640 1 -- 172.21.15.164:0/45088677 wait complete. 2024-09-19T08:44:09.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:09 smithi164 ceph-mon[30928]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:44:09.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:09 smithi164 ceph-mon[30928]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T08:44:09.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:09 smithi173 ceph-mon[35306]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:44:09.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:09 smithi173 ceph-mon[35306]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T08:44:10.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:09 smithi170 ceph-mon[35072]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:44:10.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:09 smithi170 ceph-mon[35072]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T08:44:10.491 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-19T08:44:10.740 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:10.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:10 smithi164 ceph-mon[30928]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:44:10.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:10 smithi164 ceph-mon[30928]: pgmap v286: 97 pgs: 97 active+clean; 585 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 6.0 KiB/s wr, 24 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T08:44:10.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:10 smithi173 ceph-mon[35306]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:44:10.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:10 smithi173 ceph-mon[35306]: pgmap v286: 97 pgs: 97 active+clean; 585 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 6.0 KiB/s wr, 24 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T08:44:11.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:10 smithi170 ceph-mon[35072]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:44:11.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:10 smithi170 ceph-mon[35072]: pgmap v286: 97 pgs: 97 active+clean; 585 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 6.0 KiB/s wr, 24 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T08:44:11.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.236+0000 7f1da910c640 1 -- 172.21.15.164:0/3172823699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1da4100e20 msgr2=0x7f1da4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:11.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.236+0000 7f1da910c640 1 --2- 172.21.15.164:0/3172823699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1da4100e20 0x7f1da4101200 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009900 tx=0x7f1d9402a480 comp rx=0 tx=0).stop 2024-09-19T08:44:11.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.237+0000 7f1da910c640 1 -- 172.21.15.164:0/3172823699 shutdown_connections 2024-09-19T08:44:11.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.237+0000 7f1da910c640 1 --2- 172.21.15.164:0/3172823699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1da4102ef0 0x7f1da410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:11.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.237+0000 7f1da910c640 1 --2- 172.21.15.164:0/3172823699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1da4102570 0x7f1da41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:11.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.237+0000 7f1da910c640 1 --2- 172.21.15.164:0/3172823699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1da4100e20 0x7f1da4101200 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:11.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.237+0000 7f1da910c640 1 -- 172.21.15.164:0/3172823699 >> 172.21.15.164:0/3172823699 conn(0x7f1da40fc7d0 msgr2=0x7f1da40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:11.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.237+0000 7f1da910c640 1 -- 172.21.15.164:0/3172823699 shutdown_connections 2024-09-19T08:44:11.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.237+0000 7f1da910c640 1 -- 172.21.15.164:0/3172823699 wait complete. 2024-09-19T08:44:11.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.237+0000 7f1da910c640 1 Processor -- start 2024-09-19T08:44:11.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.237+0000 7f1da910c640 1 -- start start 2024-09-19T08:44:11.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.238+0000 7f1da910c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1da4100e20 0x7f1da40fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:11.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.238+0000 7f1da910c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1da4102570 0x7f1da40fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:11.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.238+0000 7f1da910c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1da4102ef0 0x7f1da40fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:11.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.238+0000 7f1da910c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1da4110fc0 con 0x7f1da4100e20 2024-09-19T08:44:11.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.238+0000 7f1da910c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1da4110e40 con 0x7f1da4102ef0 2024-09-19T08:44:11.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.238+0000 7f1da910c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1da4111140 con 0x7f1da4102570 2024-09-19T08:44:11.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.238+0000 7f1da2d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1da4100e20 0x7f1da40fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:11.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.238+0000 7f1da2d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1da4100e20 0x7f1da40fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60058/0 (socket says 172.21.15.164:60058) 2024-09-19T08:44:11.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.238+0000 7f1da2d9d640 1 -- 172.21.15.164:0/1976933454 learned_addr learned my addr 172.21.15.164:0/1976933454 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:11.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.238+0000 7f1d9bfff640 1 --2- 172.21.15.164:0/1976933454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1da4102570 0x7f1da40fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:11.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.238+0000 7f1da359e640 1 --2- 172.21.15.164:0/1976933454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1da4102ef0 0x7f1da40fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:11.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.238+0000 7f1da2d9d640 1 -- 172.21.15.164:0/1976933454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1da4102570 msgr2=0x7f1da40fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:11.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.238+0000 7f1da2d9d640 1 --2- 172.21.15.164:0/1976933454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1da4102570 0x7f1da40fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:11.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.238+0000 7f1da2d9d640 1 -- 172.21.15.164:0/1976933454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1da4102ef0 msgr2=0x7f1da40fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:11.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.238+0000 7f1da2d9d640 1 --2- 172.21.15.164:0/1976933454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1da4102ef0 0x7f1da40fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:11.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.238+0000 7f1da2d9d640 1 -- 172.21.15.164:0/1976933454 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da41a6b00 con 0x7f1da4100e20 2024-09-19T08:44:11.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.238+0000 7f1da359e640 1 --2- 172.21.15.164:0/1976933454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1da4102ef0 0x7f1da40fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:11.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.238+0000 7f1da2d9d640 1 --2- 172.21.15.164:0/1976933454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1da4100e20 0x7f1da40fcc80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1d94006010 tx=0x7f1d94003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:11.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.239+0000 7f1da0d99640 1 -- 172.21.15.164:0/1976933454 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9403f070 con 0x7f1da4100e20 2024-09-19T08:44:11.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.239+0000 7f1da0d99640 1 -- 172.21.15.164:0/1976933454 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d94031c30 con 0x7f1da4100e20 2024-09-19T08:44:11.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.239+0000 7f1da910c640 1 -- 172.21.15.164:0/1976933454 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da41a6d90 con 0x7f1da4100e20 2024-09-19T08:44:11.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.239+0000 7f1da0d99640 1 -- 172.21.15.164:0/1976933454 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d94035030 con 0x7f1da4100e20 2024-09-19T08:44:11.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.239+0000 7f1da910c640 1 -- 172.21.15.164:0/1976933454 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da41a7210 con 0x7f1da4100e20 2024-09-19T08:44:11.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.240+0000 7f1da910c640 1 -- 172.21.15.164:0/1976933454 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1da4101200 con 0x7f1da4100e20 2024-09-19T08:44:11.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.241+0000 7f1da0d99640 1 -- 172.21.15.164:0/1976933454 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f1d94042070 con 0x7f1da4100e20 2024-09-19T08:44:11.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.241+0000 7f1da0d99640 1 --2- 172.21.15.164:0/1976933454 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1d900a32d0 0x7f1d900a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:11.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.241+0000 7f1d9bfff640 1 --2- 172.21.15.164:0/1976933454 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1d900a32d0 0x7f1d900a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:11.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.241+0000 7f1da0d99640 1 -- 172.21.15.164:0/1976933454 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1d940f0200 con 0x7f1da4100e20 2024-09-19T08:44:11.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.242+0000 7f1d9bfff640 1 --2- 172.21.15.164:0/1976933454 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1d900a32d0 0x7f1d900a5790 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1da40fe710 tx=0x7f1d8c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:11.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.245+0000 7f1da0d99640 1 -- 172.21.15.164:0/1976933454 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1d94031de0 con 0x7f1da4100e20 2024-09-19T08:44:11.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.436+0000 7f1da910c640 1 -- 172.21.15.164:0/1976933454 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f1da4062630 con 0x7f1d900a32d0 2024-09-19T08:44:11.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.462+0000 7f1da0d99640 1 -- 172.21.15.164:0/1976933454 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1da4062630 con 0x7f1d900a32d0 2024-09-19T08:44:11.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.464+0000 7f1da910c640 1 -- 172.21.15.164:0/1976933454 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1d900a32d0 msgr2=0x7f1d900a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:11.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.464+0000 7f1da910c640 1 --2- 172.21.15.164:0/1976933454 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1d900a32d0 0x7f1d900a5790 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1da40fe710 tx=0x7f1d8c00b390 comp rx=0 tx=0).stop 2024-09-19T08:44:11.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.464+0000 7f1da910c640 1 -- 172.21.15.164:0/1976933454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1da4100e20 msgr2=0x7f1da40fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:11.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.464+0000 7f1da910c640 1 --2- 172.21.15.164:0/1976933454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1da4100e20 0x7f1da40fcc80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1d94006010 tx=0x7f1d94003c10 comp rx=0 tx=0).stop 2024-09-19T08:44:11.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.464+0000 7f1da910c640 1 -- 172.21.15.164:0/1976933454 shutdown_connections 2024-09-19T08:44:11.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.464+0000 7f1da910c640 1 --2- 172.21.15.164:0/1976933454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1da4102ef0 0x7f1da40fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:11.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.464+0000 7f1da910c640 1 --2- 172.21.15.164:0/1976933454 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1d900a32d0 0x7f1d900a5790 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:11.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.464+0000 7f1da910c640 1 --2- 172.21.15.164:0/1976933454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1da4102570 0x7f1da40fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:11.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.464+0000 7f1da910c640 1 --2- 172.21.15.164:0/1976933454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1da4100e20 0x7f1da40fcc80 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:11.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.464+0000 7f1da910c640 1 -- 172.21.15.164:0/1976933454 >> 172.21.15.164:0/1976933454 conn(0x7f1da40fc7d0 msgr2=0x7f1da406a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:11.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.464+0000 7f1da910c640 1 -- 172.21.15.164:0/1976933454 shutdown_connections 2024-09-19T08:44:11.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:11.464+0000 7f1da910c640 1 -- 172.21.15.164:0/1976933454 wait complete. 2024-09-19T08:44:12.209 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-19T08:44:12.455 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:12.883 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:12 smithi164 ceph-mon[30928]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:44:12.884 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:12 smithi164 ceph-mon[30928]: pgmap v287: 97 pgs: 97 active+clean; 605 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.4 KiB/s wr, 26 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T08:44:12.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.969+0000 7fd76ff48640 1 -- 172.21.15.164:0/2060968173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd768102fc0 msgr2=0x7fd7681005d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:12.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.969+0000 7fd76ff48640 1 --2- 172.21.15.164:0/2060968173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd768102fc0 0x7fd7681005d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd758009900 tx=0x7fd75802a400 comp rx=0 tx=0).stop 2024-09-19T08:44:12.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.969+0000 7fd76ff48640 1 -- 172.21.15.164:0/2060968173 shutdown_connections 2024-09-19T08:44:12.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.969+0000 7fd76ff48640 1 --2- 172.21.15.164:0/2060968173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd76810e810 0x7fd768110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.969+0000 7fd76ff48640 1 --2- 172.21.15.164:0/2060968173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd768102fc0 0x7fd7681005d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.969+0000 7fd76ff48640 1 --2- 172.21.15.164:0/2060968173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd768102b50 0x7fd768100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.969+0000 7fd76ff48640 1 -- 172.21.15.164:0/2060968173 >> 172.21.15.164:0/2060968173 conn(0x7fd768077210 msgr2=0x7fd7680fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:12.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.969+0000 7fd76ff48640 1 -- 172.21.15.164:0/2060968173 shutdown_connections 2024-09-19T08:44:12.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.969+0000 7fd76ff48640 1 -- 172.21.15.164:0/2060968173 wait complete. 2024-09-19T08:44:12.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.970+0000 7fd76ff48640 1 Processor -- start 2024-09-19T08:44:12.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.970+0000 7fd76ff48640 1 -- start start 2024-09-19T08:44:12.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.970+0000 7fd76ff48640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd768102b50 0x7fd7681a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:12.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.970+0000 7fd76ff48640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd76810e810 0x7fd7681a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:12.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.970+0000 7fd76ff48640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd76819a590 0x7fd76819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:12.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.970+0000 7fd76ff48640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd7681142c0 con 0x7fd768102b50 2024-09-19T08:44:12.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.970+0000 7fd76ff48640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd768114140 con 0x7fd76810e810 2024-09-19T08:44:12.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.970+0000 7fd76ff48640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd768114440 con 0x7fd76819a590 2024-09-19T08:44:12.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.970+0000 7fd76dce4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd768102b50 0x7fd7681a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:12.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.970+0000 7fd76dce4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd768102b50 0x7fd7681a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60078/0 (socket says 172.21.15.164:60078) 2024-09-19T08:44:12.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.970+0000 7fd76dce4640 1 -- 172.21.15.164:0/1846281993 learned_addr learned my addr 172.21.15.164:0/1846281993 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:12.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.970+0000 7fd76d4e3640 1 --2- 172.21.15.164:0/1846281993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd76810e810 0x7fd7681a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:12.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.970+0000 7fd76e4e5640 1 --2- 172.21.15.164:0/1846281993 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd76819a590 0x7fd76819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:12.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.970+0000 7fd76dce4640 1 -- 172.21.15.164:0/1846281993 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd76819a590 msgr2=0x7fd76819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:12.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.971+0000 7fd76dce4640 1 --2- 172.21.15.164:0/1846281993 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd76819a590 0x7fd76819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.971+0000 7fd76dce4640 1 -- 172.21.15.164:0/1846281993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd76810e810 msgr2=0x7fd7681a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:12.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.971+0000 7fd76dce4640 1 --2- 172.21.15.164:0/1846281993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd76810e810 0x7fd7681a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.971+0000 7fd76dce4640 1 -- 172.21.15.164:0/1846281993 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd76819b0b0 con 0x7fd768102b50 2024-09-19T08:44:12.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.971+0000 7fd76d4e3640 1 --2- 172.21.15.164:0/1846281993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd76810e810 0x7fd7681a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:44:12.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.971+0000 7fd76e4e5640 1 --2- 172.21.15.164:0/1846281993 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd76819a590 0x7fd76819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:12.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.971+0000 7fd76dce4640 1 --2- 172.21.15.164:0/1846281993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd768102b50 0x7fd7681a03d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd76400b4a0 tx=0x7fd76400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:12.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.971+0000 7fd75effd640 1 -- 172.21.15.164:0/1846281993 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd764014070 con 0x7fd768102b50 2024-09-19T08:44:12.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.971+0000 7fd75effd640 1 -- 172.21.15.164:0/1846281993 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd764010af0 con 0x7fd768102b50 2024-09-19T08:44:12.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.971+0000 7fd76ff48640 1 -- 172.21.15.164:0/1846281993 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd76819b3a0 con 0x7fd768102b50 2024-09-19T08:44:12.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.971+0000 7fd75effd640 1 -- 172.21.15.164:0/1846281993 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd764017d10 con 0x7fd768102b50 2024-09-19T08:44:12.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.971+0000 7fd76ff48640 1 -- 172.21.15.164:0/1846281993 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7681a7660 con 0x7fd768102b50 2024-09-19T08:44:12.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.972+0000 7fd76ff48640 1 -- 172.21.15.164:0/1846281993 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd768100110 con 0x7fd768102b50 2024-09-19T08:44:12.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.973+0000 7fd75effd640 1 -- 172.21.15.164:0/1846281993 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fd764017070 con 0x7fd768102b50 2024-09-19T08:44:12.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.973+0000 7fd75effd640 1 --2- 172.21.15.164:0/1846281993 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fd7540a3300 0x7fd7540a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:12.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.974+0000 7fd76d4e3640 1 --2- 172.21.15.164:0/1846281993 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fd7540a3300 0x7fd7540a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:12.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.974+0000 7fd75effd640 1 -- 172.21.15.164:0/1846281993 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fd7640cecf0 con 0x7fd768102b50 2024-09-19T08:44:12.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.974+0000 7fd76d4e3640 1 --2- 172.21.15.164:0/1846281993 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fd7540a3300 0x7fd7540a57c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd75802a9f0 tx=0x7fd758000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:12.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:12.978+0000 7fd75effd640 1 -- 172.21.15.164:0/1846281993 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd76409bac0 con 0x7fd768102b50 2024-09-19T08:44:13.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:12 smithi170 ceph-mon[35072]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:44:13.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:12 smithi170 ceph-mon[35072]: pgmap v287: 97 pgs: 97 active+clean; 605 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.4 KiB/s wr, 26 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T08:44:13.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:12 smithi173 ceph-mon[35306]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:44:13.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:12 smithi173 ceph-mon[35306]: pgmap v287: 97 pgs: 97 active+clean; 605 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.4 KiB/s wr, 26 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T08:44:13.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:13.170+0000 7fd76ff48640 1 -- 172.21.15.164:0/1846281993 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fd7681030c0 con 0x7fd7540a3300 2024-09-19T08:44:13.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:13.175+0000 7fd75effd640 1 -- 172.21.15.164:0/1846281993 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fd7681030c0 con 0x7fd7540a3300 2024-09-19T08:44:13.175 INFO:teuthology.orchestra.run.smithi164.stdout:/volumes/qa/sv_1/0902124e-0b89-4d29-848b-19de8f45d8f9 2024-09-19T08:44:13.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:13.177+0000 7fd76ff48640 1 -- 172.21.15.164:0/1846281993 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fd7540a3300 msgr2=0x7fd7540a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:13.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:13.177+0000 7fd76ff48640 1 --2- 172.21.15.164:0/1846281993 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fd7540a3300 0x7fd7540a57c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd75802a9f0 tx=0x7fd758000bc0 comp rx=0 tx=0).stop 2024-09-19T08:44:13.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:13.177+0000 7fd76ff48640 1 -- 172.21.15.164:0/1846281993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd768102b50 msgr2=0x7fd7681a03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:13.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:13.177+0000 7fd76ff48640 1 --2- 172.21.15.164:0/1846281993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd768102b50 0x7fd7681a03d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd76400b4a0 tx=0x7fd76400b960 comp rx=0 tx=0).stop 2024-09-19T08:44:13.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:13.177+0000 7fd76ff48640 1 -- 172.21.15.164:0/1846281993 shutdown_connections 2024-09-19T08:44:13.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:13.177+0000 7fd76ff48640 1 --2- 172.21.15.164:0/1846281993 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fd7540a3300 0x7fd7540a57c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:13.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:13.177+0000 7fd76ff48640 1 --2- 172.21.15.164:0/1846281993 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd76819a590 0x7fd76819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:13.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:13.177+0000 7fd76ff48640 1 --2- 172.21.15.164:0/1846281993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd76810e810 0x7fd7681a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:13.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:13.177+0000 7fd76ff48640 1 --2- 172.21.15.164:0/1846281993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd768102b50 0x7fd7681a03d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:13.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:13.177+0000 7fd76ff48640 1 -- 172.21.15.164:0/1846281993 >> 172.21.15.164:0/1846281993 conn(0x7fd768077210 msgr2=0x7fd76810fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:13.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:13.178+0000 7fd76ff48640 1 -- 172.21.15.164:0/1846281993 shutdown_connections 2024-09-19T08:44:13.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:13.178+0000 7fd76ff48640 1 -- 172.21.15.164:0/1846281993 wait complete. 2024-09-19T08:44:14.117 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:44:14.368 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:14.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.843+0000 7faaf8c2f640 1 -- 172.21.15.164:0/3617017608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faaf41027a0 msgr2=0x7faaf41067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:14.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.843+0000 7faaf8c2f640 1 --2- 172.21.15.164:0/3617017608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faaf41027a0 0x7faaf41067f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7faadc009900 tx=0x7faadc02a410 comp rx=0 tx=0).stop 2024-09-19T08:44:14.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.844+0000 7faaf8c2f640 1 -- 172.21.15.164:0/3617017608 shutdown_connections 2024-09-19T08:44:14.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.844+0000 7faaf8c2f640 1 --2- 172.21.15.164:0/3617017608 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faaf4106f20 0x7faaf410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:14.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.844+0000 7faaf8c2f640 1 --2- 172.21.15.164:0/3617017608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faaf41027a0 0x7faaf41067f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:14.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.844+0000 7faaf8c2f640 1 --2- 172.21.15.164:0/3617017608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faaf4101df0 0x7faaf41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:14.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.844+0000 7faaf8c2f640 1 -- 172.21.15.164:0/3617017608 >> 172.21.15.164:0/3617017608 conn(0x7faaf40fd8f0 msgr2=0x7faaf40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:14.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.844+0000 7faaf8c2f640 1 -- 172.21.15.164:0/3617017608 shutdown_connections 2024-09-19T08:44:14.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.844+0000 7faaf8c2f640 1 -- 172.21.15.164:0/3617017608 wait complete. 2024-09-19T08:44:14.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.844+0000 7faaf8c2f640 1 Processor -- start 2024-09-19T08:44:14.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.844+0000 7faaf8c2f640 1 -- start start 2024-09-19T08:44:14.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.845+0000 7faaf8c2f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faaf4101df0 0x7faaf4198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:14.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.845+0000 7faaf8c2f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faaf41027a0 0x7faaf41989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:14.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.845+0000 7faaf8c2f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faaf4106f20 0x7faaf419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:14.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.845+0000 7faaf8c2f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faaf4111f80 con 0x7faaf4106f20 2024-09-19T08:44:14.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.845+0000 7faaf8c2f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7faaf4111e00 con 0x7faaf4101df0 2024-09-19T08:44:14.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.845+0000 7faaf8c2f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faaf4112100 con 0x7faaf41027a0 2024-09-19T08:44:14.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.845+0000 7faaf2d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faaf4106f20 0x7faaf419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:14.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.845+0000 7faaf2d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faaf4106f20 0x7faaf419cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35618/0 (socket says 172.21.15.164:35618) 2024-09-19T08:44:14.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.845+0000 7faaf2d9d640 1 -- 172.21.15.164:0/3364543706 learned_addr learned my addr 172.21.15.164:0/3364543706 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:14.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.845+0000 7faaf1d9b640 1 --2- 172.21.15.164:0/3364543706 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faaf41027a0 0x7faaf41989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:14.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.845+0000 7faaf259c640 1 --2- 172.21.15.164:0/3364543706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faaf4101df0 0x7faaf4198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:14.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.845+0000 7faaf2d9d640 1 -- 172.21.15.164:0/3364543706 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faaf41027a0 msgr2=0x7faaf41989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:14.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.845+0000 7faaf2d9d640 1 --2- 172.21.15.164:0/3364543706 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faaf41027a0 0x7faaf41989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:14.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.845+0000 7faaf2d9d640 1 -- 172.21.15.164:0/3364543706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faaf4101df0 msgr2=0x7faaf4198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:14.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.845+0000 7faaf2d9d640 1 --2- 172.21.15.164:0/3364543706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faaf4101df0 0x7faaf4198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:14.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.845+0000 7faaf2d9d640 1 -- 172.21.15.164:0/3364543706 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf419d410 con 0x7faaf4106f20 2024-09-19T08:44:14.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.845+0000 7faaf259c640 1 --2- 172.21.15.164:0/3364543706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faaf4101df0 0x7faaf4198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:44:14.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.845+0000 7faaf2d9d640 1 --2- 172.21.15.164:0/3364543706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faaf4106f20 0x7faaf419cd30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7faaec00d750 tx=0x7faaec00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:14.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.845+0000 7faaeb7fe640 1 -- 172.21.15.164:0/3364543706 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec017070 con 0x7faaf4106f20 2024-09-19T08:44:14.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.845+0000 7faaeb7fe640 1 -- 172.21.15.164:0/3364543706 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaec005500 con 0x7faaf4106f20 2024-09-19T08:44:14.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.845+0000 7faaf8c2f640 1 -- 172.21.15.164:0/3364543706 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaf419d700 con 0x7faaf4106f20 2024-09-19T08:44:14.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.846+0000 7faaeb7fe640 1 -- 172.21.15.164:0/3364543706 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec004d90 con 0x7faaf4106f20 2024-09-19T08:44:14.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.846+0000 7faaf8c2f640 1 -- 172.21.15.164:0/3364543706 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaf41a4fe0 con 0x7faaf4106f20 2024-09-19T08:44:14.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.847+0000 7faaf8c2f640 1 -- 172.21.15.164:0/3364543706 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaf41032d0 con 0x7faaf4106f20 2024-09-19T08:44:14.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.848+0000 7faaeb7fe640 1 -- 172.21.15.164:0/3364543706 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7faaec004f30 con 0x7faaf4106f20 2024-09-19T08:44:14.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.849+0000 7faaeb7fe640 1 --2- 172.21.15.164:0/3364543706 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7faae00a32d0 0x7faae00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:14.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.849+0000 7faaf259c640 1 --2- 172.21.15.164:0/3364543706 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7faae00a32d0 0x7faae00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:14.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.849+0000 7faaeb7fe640 1 -- 172.21.15.164:0/3364543706 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7faaec0cf6e0 con 0x7faaf4106f20 2024-09-19T08:44:14.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.849+0000 7faaf259c640 1 --2- 172.21.15.164:0/3364543706 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7faae00a32d0 0x7faae00a5790 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faae4005c40 tx=0x7faae400a430 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:14.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:14.852+0000 7faaeb7fe640 1 -- 172.21.15.164:0/3364543706 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7faaec09c3a0 con 0x7faaf4106f20 2024-09-19T08:44:15.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:14 smithi164 ceph-mon[30928]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:44:15.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:14 smithi164 ceph-mon[30928]: pgmap v288: 97 pgs: 97 active+clean; 605 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.8 KiB/s wr, 24 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T08:44:15.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:15.053+0000 7faaf8c2f640 1 -- 172.21.15.164:0/3364543706 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faaf41021d0 con 0x7faaf4106f20 2024-09-19T08:44:15.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:15.054+0000 7faaeb7fe640 1 -- 172.21.15.164:0/3364543706 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11984 (secure 0 0 0) 0x7faaf41021d0 con 0x7faaf4106f20 2024-09-19T08:44:15.054 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:44:15.054 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":27,"btime":"2024-09-19T08:44:05:632372+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:44:05.632353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24442,24424,24449,14595]},"id":1}]} 2024-09-19T08:44:15.054 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 27 2024-09-19T08:44:15.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:15.056+0000 7faaf8c2f640 1 -- 172.21.15.164:0/3364543706 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7faae00a32d0 msgr2=0x7faae00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:15.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:15.056+0000 7faaf8c2f640 1 --2- 172.21.15.164:0/3364543706 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7faae00a32d0 0x7faae00a5790 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faae4005c40 tx=0x7faae400a430 comp rx=0 tx=0).stop 2024-09-19T08:44:15.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:15.056+0000 7faaf8c2f640 1 -- 172.21.15.164:0/3364543706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faaf4106f20 msgr2=0x7faaf419cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:15.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:15.056+0000 7faaf8c2f640 1 --2- 172.21.15.164:0/3364543706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faaf4106f20 0x7faaf419cd30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7faaec00d750 tx=0x7faaec00dc10 comp rx=0 tx=0).stop 2024-09-19T08:44:15.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:15.056+0000 7faaf8c2f640 1 -- 172.21.15.164:0/3364543706 shutdown_connections 2024-09-19T08:44:15.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:15.056+0000 7faaf8c2f640 1 --2- 172.21.15.164:0/3364543706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faaf4106f20 0x7faaf419cd30 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:15.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:15.056+0000 7faaf8c2f640 1 --2- 172.21.15.164:0/3364543706 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7faae00a32d0 0x7faae00a5790 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:15.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:15.056+0000 7faaf8c2f640 1 --2- 172.21.15.164:0/3364543706 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faaf41027a0 0x7faaf41989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:15.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:15.056+0000 7faaf8c2f640 1 --2- 172.21.15.164:0/3364543706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faaf4101df0 0x7faaf4198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:15.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:15.057+0000 7faaf8c2f640 1 -- 172.21.15.164:0/3364543706 >> 172.21.15.164:0/3364543706 conn(0x7faaf40fd8f0 msgr2=0x7faaf40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:15.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:15.057+0000 7faaf8c2f640 1 -- 172.21.15.164:0/3364543706 shutdown_connections 2024-09-19T08:44:15.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:15.057+0000 7faaf8c2f640 1 -- 172.21.15.164:0/3364543706 wait complete. 2024-09-19T08:44:15.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:14 smithi170 ceph-mon[35072]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:44:15.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:14 smithi170 ceph-mon[35072]: pgmap v288: 97 pgs: 97 active+clean; 605 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.8 KiB/s wr, 24 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T08:44:15.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:14 smithi173 ceph-mon[35306]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:44:15.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:14 smithi173 ceph-mon[35306]: pgmap v288: 97 pgs: 97 active+clean; 605 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.8 KiB/s wr, 24 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T08:44:15.689 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T08:44:15.940 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:16.061 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:15 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3364543706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:44:16.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:15 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3364543706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:44:16.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:15 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3364543706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:44:16.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.390+0000 7f4101a19640 1 -- 172.21.15.164:0/8986309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40fc1002f0 msgr2=0x7f40fc1006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:16.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.390+0000 7f4101a19640 1 --2- 172.21.15.164:0/8986309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40fc1002f0 0x7f40fc1006d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f40e4009930 tx=0x7f40e402a420 comp rx=0 tx=0).stop 2024-09-19T08:44:16.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.390+0000 7f4101a19640 1 -- 172.21.15.164:0/8986309 shutdown_connections 2024-09-19T08:44:16.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.390+0000 7f4101a19640 1 --2- 172.21.15.164:0/8986309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40fc106f50 0x7f40fc10eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:16.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.390+0000 7f4101a19640 1 --2- 172.21.15.164:0/8986309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40fc100ca0 0x7f40fc106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:16.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.390+0000 7f4101a19640 1 --2- 172.21.15.164:0/8986309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40fc1002f0 0x7f40fc1006d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:16.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.390+0000 7f4101a19640 1 -- 172.21.15.164:0/8986309 >> 172.21.15.164:0/8986309 conn(0x7f40fc077210 msgr2=0x7f40fc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:16.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.390+0000 7f4101a19640 1 -- 172.21.15.164:0/8986309 shutdown_connections 2024-09-19T08:44:16.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.390+0000 7f4101a19640 1 -- 172.21.15.164:0/8986309 wait complete. 2024-09-19T08:44:16.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.391+0000 7f4101a19640 1 Processor -- start 2024-09-19T08:44:16.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.391+0000 7f4101a19640 1 -- start start 2024-09-19T08:44:16.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.391+0000 7f4101a19640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40fc1002f0 0x7f40fc198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:16.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.391+0000 7f4101a19640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40fc100ca0 0x7f40fc198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:16.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.391+0000 7f4101a19640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40fc106f50 0x7f40fc19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:16.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.391+0000 7f4101a19640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc112000 con 0x7f40fc100ca0 2024-09-19T08:44:16.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.391+0000 7f4101a19640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc111e80 con 0x7f40fc106f50 2024-09-19T08:44:16.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.391+0000 7f4101a19640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc112180 con 0x7f40fc1002f0 2024-09-19T08:44:16.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.391+0000 7f40fa7fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40fc100ca0 0x7f40fc198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:16.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.391+0000 7f40fa7fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40fc100ca0 0x7f40fc198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35644/0 (socket says 172.21.15.164:35644) 2024-09-19T08:44:16.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.391+0000 7f40fa7fc640 1 -- 172.21.15.164:0/2473222148 learned_addr learned my addr 172.21.15.164:0/2473222148 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:16.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.392+0000 7f40fb7fe640 1 --2- 172.21.15.164:0/2473222148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40fc106f50 0x7f40fc19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:16.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.392+0000 7f40faffd640 1 --2- 172.21.15.164:0/2473222148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40fc1002f0 0x7f40fc198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:16.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.392+0000 7f40fa7fc640 1 -- 172.21.15.164:0/2473222148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40fc1002f0 msgr2=0x7f40fc198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:16.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.392+0000 7f40fa7fc640 1 --2- 172.21.15.164:0/2473222148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40fc1002f0 0x7f40fc198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:16.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.392+0000 7f40fa7fc640 1 -- 172.21.15.164:0/2473222148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40fc106f50 msgr2=0x7f40fc19cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:16.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.392+0000 7f40fa7fc640 1 --2- 172.21.15.164:0/2473222148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40fc106f50 0x7f40fc19cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:16.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.392+0000 7f40fa7fc640 1 -- 172.21.15.164:0/2473222148 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40fc19d4d0 con 0x7f40fc100ca0 2024-09-19T08:44:16.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.392+0000 7f40fa7fc640 1 --2- 172.21.15.164:0/2473222148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40fc100ca0 0x7f40fc198a60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f40ec00c970 tx=0x7f40ec00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:16.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.392+0000 7f4100a17640 1 -- 172.21.15.164:0/2473222148 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec014070 con 0x7f40fc100ca0 2024-09-19T08:44:16.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.392+0000 7f4100a17640 1 -- 172.21.15.164:0/2473222148 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40ec004440 con 0x7f40fc100ca0 2024-09-19T08:44:16.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.392+0000 7f4101a19640 1 -- 172.21.15.164:0/2473222148 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40fc19d7c0 con 0x7f40fc100ca0 2024-09-19T08:44:16.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.392+0000 7f4100a17640 1 -- 172.21.15.164:0/2473222148 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec00f450 con 0x7f40fc100ca0 2024-09-19T08:44:16.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.392+0000 7f4101a19640 1 -- 172.21.15.164:0/2473222148 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40fc1a50a0 con 0x7f40fc100ca0 2024-09-19T08:44:16.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.393+0000 7f4101a19640 1 -- 172.21.15.164:0/2473222148 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40fc1006d0 con 0x7f40fc100ca0 2024-09-19T08:44:16.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.394+0000 7f4100a17640 1 -- 172.21.15.164:0/2473222148 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f40ec017070 con 0x7f40fc100ca0 2024-09-19T08:44:16.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.395+0000 7f4100a17640 1 --2- 172.21.15.164:0/2473222148 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f40e80a32d0 0x7f40e80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:16.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.395+0000 7f40faffd640 1 --2- 172.21.15.164:0/2473222148 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f40e80a32d0 0x7f40e80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:16.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.395+0000 7f4100a17640 1 -- 172.21.15.164:0/2473222148 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f40ec0cf8b0 con 0x7f40fc100ca0 2024-09-19T08:44:16.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.395+0000 7f40faffd640 1 --2- 172.21.15.164:0/2473222148 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f40e80a32d0 0x7f40e80a5790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f40e4000c00 tx=0x7f40e4031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:16.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.399+0000 7f4100a17640 1 -- 172.21.15.164:0/2473222148 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f40ec01e020 con 0x7f40fc100ca0 2024-09-19T08:44:16.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.566+0000 7f4101a19640 1 -- 172.21.15.164:0/2473222148 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f40fc109bb0 con 0x7f40fc100ca0 2024-09-19T08:44:16.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.568+0000 7f4100a17640 1 -- 172.21.15.164:0/2473222148 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22702 (secure 0 0 0) 0x7f40ec09c570 con 0x7f40fc100ca0 2024-09-19T08:44:16.568 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:44:16.568 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":82,"fsid":"1e4b0d74-7662-11ef-bceb-c7b262605968","created":"2024-09-19T08:36:09.564509+0000","modified":"2024-09-19T08:44:02.811236+0000","last_up_change":"2024-09-19T08:43:01.260852+0000","last_in_change":"2024-09-19T08:42:37.492464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:39:13.961749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:43:49.613471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T08:43:51.570675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33b11f37-6ac5-41fc-8cfd-1b08c7eef688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6803","nonce":663038243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6805","nonce":663038243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6809","nonce":663038243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6807","nonce":663038243}]},"public_addr":"172.21.15.164:6803/663038243","cluster_addr":"172.21.15.164:6805/663038243","heartbeat_back_addr":"172.21.15.164:6809/663038243","heartbeat_front_addr":"172.21.15.164:6807/663038243","state":["exists","up"]},{"osd":1,"uuid":"6de64b0a-46e2-4a98-8005-a326a64bc87d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6801","nonce":2167066371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6803","nonce":2167066371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6807","nonce":2167066371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6805","nonce":2167066371}]},"public_addr":"172.21.15.170:6801/2167066371","cluster_addr":"172.21.15.170:6803/2167066371","heartbeat_back_addr":"172.21.15.170:6807/2167066371","heartbeat_front_addr":"172.21.15.170:6805/2167066371","state":["exists","up"]},{"osd":2,"uuid":"d4158cc7-9822-42a7-9601-d65089907584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6801","nonce":1646561048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6803","nonce":1646561048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6807","nonce":1646561048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6805","nonce":1646561048}]},"public_addr":"172.21.15.173:6801/1646561048","cluster_addr":"172.21.15.173:6803/1646561048","heartbeat_back_addr":"172.21.15.173:6807/1646561048","heartbeat_front_addr":"172.21.15.173:6805/1646561048","state":["exists","up"]},{"osd":3,"uuid":"0cf3f7f1-3a8c-42ce-88a9-9bea52e02221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6811","nonce":406795560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6813","nonce":406795560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6817","nonce":406795560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6815","nonce":406795560}]},"public_addr":"172.21.15.164:6811/406795560","cluster_addr":"172.21.15.164:6813/406795560","heartbeat_back_addr":"172.21.15.164:6817/406795560","heartbeat_front_addr":"172.21.15.164:6815/406795560","state":["exists","up"]},{"osd":4,"uuid":"6dca048a-2860-4a02-8d8e-450b30bc1a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6809","nonce":4120832165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6811","nonce":4120832165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6815","nonce":4120832165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6813","nonce":4120832165}]},"public_addr":"172.21.15.170:6809/4120832165","cluster_addr":"172.21.15.170:6811/4120832165","heartbeat_back_addr":"172.21.15.170:6815/4120832165","heartbeat_front_addr":"172.21.15.170:6813/4120832165","state":["exists","up"]},{"osd":5,"uuid":"3822521f-69d6-4141-a83f-5fc9cecc7b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6809","nonce":2914653552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6811","nonce":2914653552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6815","nonce":2914653552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6813","nonce":2914653552}]},"public_addr":"172.21.15.173:6809/2914653552","cluster_addr":"172.21.15.173:6811/2914653552","heartbeat_back_addr":"172.21.15.173:6815/2914653552","heartbeat_front_addr":"172.21.15.173:6813/2914653552","state":["exists","up"]},{"osd":6,"uuid":"0f5a965e-2eb2-4c24-b089-577e0639c160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6819","nonce":1138341393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6821","nonce":1138341393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6825","nonce":1138341393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6823","nonce":1138341393}]},"public_addr":"172.21.15.164:6819/1138341393","cluster_addr":"172.21.15.164:6821/1138341393","heartbeat_back_addr":"172.21.15.164:6825/1138341393","heartbeat_front_addr":"172.21.15.164:6823/1138341393","state":["exists","up"]},{"osd":7,"uuid":"f16cb2c3-639b-46ae-a740-1bd28ca1863c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6817","nonce":2124160360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6819","nonce":2124160360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6823","nonce":2124160360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6821","nonce":2124160360}]},"public_addr":"172.21.15.170:6817/2124160360","cluster_addr":"172.21.15.170:6819/2124160360","heartbeat_back_addr":"172.21.15.170:6823/2124160360","heartbeat_front_addr":"172.21.15.170:6821/2124160360","state":["exists","up"]},{"osd":8,"uuid":"161cff01-46ad-48e3-b999-a5473a2ad618","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6817","nonce":2285175061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6819","nonce":2285175061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6823","nonce":2285175061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6821","nonce":2285175061}]},"public_addr":"172.21.15.173:6817/2285175061","cluster_addr":"172.21.15.173:6819/2285175061","heartbeat_back_addr":"172.21.15.173:6823/2285175061","heartbeat_front_addr":"172.21.15.173:6821/2285175061","state":["exists","up"]},{"osd":9,"uuid":"994666e8-8fbd-4231-ac67-81a79aaf7c2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6827","nonce":3918847952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6829","nonce":3918847952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6833","nonce":3918847952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6831","nonce":3918847952}]},"public_addr":"172.21.15.164:6827/3918847952","cluster_addr":"172.21.15.164:6829/3918847952","heartbeat_back_addr":"172.21.15.164:6833/3918847952","heartbeat_front_addr":"172.21.15.164:6831/3918847952","state":["exists","up"]},{"osd":10,"uuid":"ff5e9099-cdb4-4905-b888-5b67320ef76b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6825","nonce":2707848227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6827","nonce":2707848227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6831","nonce":2707848227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6829","nonce":2707848227}]},"public_addr":"172.21.15.170:6825/2707848227","cluster_addr":"172.21.15.170:6827/2707848227","heartbeat_back_addr":"172.21.15.170:6831/2707848227","heartbeat_front_addr":"172.21.15.170:6829/2707848227","state":["exists","up"]},{"osd":11,"uuid":"4d93088f-e791-4938-a207-12f15b56abd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6825","nonce":2831924738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6827","nonce":2831924738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6831","nonce":2831924738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6829","nonce":2831924738}]},"public_addr":"172.21.15.173:6825/2831924738","cluster_addr":"172.21.15.173:6827/2831924738","heartbeat_back_addr":"172.21.15.173:6831/2831924738","heartbeat_front_addr":"172.21.15.173:6829/2831924738","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:21.787352+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:46.012323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:39:11.353040+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:39:37.015124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:02.063379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:27.892865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:53.372041+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:41:17.855284+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:41:43.389882+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:08.956895+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:34.069779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:59.303897+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.164:0/339032808":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6801/1173822158":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6800/1173822158":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6801/2805828009":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1837164919":"2024-09-20T08:37:01.663065+0000","172.21.15.164:0/4018459143":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6800/2805828009":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/2770797546":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1905394497":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1369265436":"2024-09-20T08:36:33.856642+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:44:16.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.572+0000 7f4101a19640 1 -- 172.21.15.164:0/2473222148 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f40e80a32d0 msgr2=0x7f40e80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:16.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.572+0000 7f4101a19640 1 --2- 172.21.15.164:0/2473222148 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f40e80a32d0 0x7f40e80a5790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f40e4000c00 tx=0x7f40e4031490 comp rx=0 tx=0).stop 2024-09-19T08:44:16.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.572+0000 7f4101a19640 1 -- 172.21.15.164:0/2473222148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40fc100ca0 msgr2=0x7f40fc198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:16.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.572+0000 7f4101a19640 1 --2- 172.21.15.164:0/2473222148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40fc100ca0 0x7f40fc198a60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f40ec00c970 tx=0x7f40ec00ce30 comp rx=0 tx=0).stop 2024-09-19T08:44:16.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.572+0000 7f4101a19640 1 -- 172.21.15.164:0/2473222148 shutdown_connections 2024-09-19T08:44:16.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.572+0000 7f4101a19640 1 --2- 172.21.15.164:0/2473222148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40fc106f50 0x7f40fc19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:16.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.572+0000 7f4101a19640 1 --2- 172.21.15.164:0/2473222148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40fc100ca0 0x7f40fc198a60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:16.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.572+0000 7f4101a19640 1 --2- 172.21.15.164:0/2473222148 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f40e80a32d0 0x7f40e80a5790 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:16.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.572+0000 7f4101a19640 1 --2- 172.21.15.164:0/2473222148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40fc1002f0 0x7f40fc198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:16.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.572+0000 7f4101a19640 1 -- 172.21.15.164:0/2473222148 >> 172.21.15.164:0/2473222148 conn(0x7f40fc077210 msgr2=0x7f40fc105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:16.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.572+0000 7f4101a19640 1 -- 172.21.15.164:0/2473222148 shutdown_connections 2024-09-19T08:44:16.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:16.572+0000 7f4101a19640 1 -- 172.21.15.164:0/2473222148 wait complete. 2024-09-19T08:44:16.965 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:16 smithi164 ceph-mon[30928]: pgmap v289: 97 pgs: 97 active+clean; 606 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.2 KiB/s wr, 23 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T08:44:16.965 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:16 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2473222148' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:44:17.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:16 smithi170 ceph-mon[35072]: pgmap v289: 97 pgs: 97 active+clean; 606 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.2 KiB/s wr, 23 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T08:44:17.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:16 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2473222148' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:44:17.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:16 smithi173 ceph-mon[35306]: pgmap v289: 97 pgs: 97 active+clean; 606 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.2 KiB/s wr, 23 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T08:44:17.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:16 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2473222148' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:44:17.276 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T08:44:17.276 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T08:44:17.522 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:18.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.007+0000 7f1b81d41640 1 -- 172.21.15.164:0/205773808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b7c102570 msgr2=0x7f1b7c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:18.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.007+0000 7f1b81d41640 1 --2- 172.21.15.164:0/205773808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b7c102570 0x7f1b7c1029b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1b64009900 tx=0x7f1b6402a480 comp rx=0 tx=0).stop 2024-09-19T08:44:18.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.007+0000 7f1b81d41640 1 -- 172.21.15.164:0/205773808 shutdown_connections 2024-09-19T08:44:18.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.007+0000 7f1b81d41640 1 --2- 172.21.15.164:0/205773808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b7c102ef0 0x7f1b7c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:18.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.007+0000 7f1b81d41640 1 --2- 172.21.15.164:0/205773808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b7c102570 0x7f1b7c1029b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:18.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.007+0000 7f1b81d41640 1 --2- 172.21.15.164:0/205773808 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b7c100e20 0x7f1b7c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:18.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.007+0000 7f1b81d41640 1 -- 172.21.15.164:0/205773808 >> 172.21.15.164:0/205773808 conn(0x7f1b7c0fc7d0 msgr2=0x7f1b7c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:18.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.007+0000 7f1b81d41640 1 -- 172.21.15.164:0/205773808 shutdown_connections 2024-09-19T08:44:18.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.007+0000 7f1b81d41640 1 -- 172.21.15.164:0/205773808 wait complete. 2024-09-19T08:44:18.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.008+0000 7f1b81d41640 1 Processor -- start 2024-09-19T08:44:18.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.008+0000 7f1b81d41640 1 -- start start 2024-09-19T08:44:18.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.008+0000 7f1b81d41640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b7c100e20 0x7f1b7c1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:18.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.008+0000 7f1b81d41640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b7c102570 0x7f1b7c198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:18.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.008+0000 7f1b81d41640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b7c102ef0 0x7f1b7c19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:18.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.008+0000 7f1b81d41640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c1120a0 con 0x7f1b7c102570 2024-09-19T08:44:18.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.008+0000 7f1b81d41640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c111f20 con 0x7f1b7c102ef0 2024-09-19T08:44:18.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.008+0000 7f1b81d41640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c112220 con 0x7f1b7c100e20 2024-09-19T08:44:18.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.008+0000 7f1b73fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b7c102570 0x7f1b7c198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:18.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.008+0000 7f1b73fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b7c102570 0x7f1b7c198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35672/0 (socket says 172.21.15.164:35672) 2024-09-19T08:44:18.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.008+0000 7f1b73fff640 1 -- 172.21.15.164:0/1916134707 learned_addr learned my addr 172.21.15.164:0/1916134707 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:18.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.008+0000 7f1b7bfff640 1 --2- 172.21.15.164:0/1916134707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b7c102ef0 0x7f1b7c19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:18.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.008+0000 7f1b7b7fe640 1 --2- 172.21.15.164:0/1916134707 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b7c100e20 0x7f1b7c1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:18.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.008+0000 7f1b73fff640 1 -- 172.21.15.164:0/1916134707 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b7c100e20 msgr2=0x7f1b7c1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:18.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.008+0000 7f1b73fff640 1 --2- 172.21.15.164:0/1916134707 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b7c100e20 0x7f1b7c1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:18.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.008+0000 7f1b73fff640 1 -- 172.21.15.164:0/1916134707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b7c102ef0 msgr2=0x7f1b7c19ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:18.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.008+0000 7f1b73fff640 1 --2- 172.21.15.164:0/1916134707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b7c102ef0 0x7f1b7c19ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:18.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.008+0000 7f1b73fff640 1 -- 172.21.15.164:0/1916134707 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7c19d570 con 0x7f1b7c102570 2024-09-19T08:44:18.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.008+0000 7f1b7b7fe640 1 --2- 172.21.15.164:0/1916134707 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b7c100e20 0x7f1b7c1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:44:18.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.008+0000 7f1b73fff640 1 --2- 172.21.15.164:0/1916134707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b7c102570 0x7f1b7c198b00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1b64006010 tx=0x7f1b64003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:18.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.009+0000 7f1b797fa640 1 -- 172.21.15.164:0/1916134707 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6403f070 con 0x7f1b7c102570 2024-09-19T08:44:18.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.009+0000 7f1b797fa640 1 -- 172.21.15.164:0/1916134707 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b64031c30 con 0x7f1b7c102570 2024-09-19T08:44:18.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.009+0000 7f1b81d41640 1 -- 172.21.15.164:0/1916134707 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b7c19d800 con 0x7f1b7c102570 2024-09-19T08:44:18.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.009+0000 7f1b797fa640 1 -- 172.21.15.164:0/1916134707 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64035030 con 0x7f1b7c102570 2024-09-19T08:44:18.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.009+0000 7f1b81d41640 1 -- 172.21.15.164:0/1916134707 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b7c1a5140 con 0x7f1b7c102570 2024-09-19T08:44:18.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.010+0000 7f1b81d41640 1 -- 172.21.15.164:0/1916134707 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b7c1021b0 con 0x7f1b7c102570 2024-09-19T08:44:18.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.011+0000 7f1b797fa640 1 -- 172.21.15.164:0/1916134707 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f1b64042070 con 0x7f1b7c102570 2024-09-19T08:44:18.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.011+0000 7f1b797fa640 1 --2- 172.21.15.164:0/1916134707 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1b680a32d0 0x7f1b680a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:18.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.012+0000 7f1b797fa640 1 -- 172.21.15.164:0/1916134707 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1b640f0720 con 0x7f1b7c102570 2024-09-19T08:44:18.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.012+0000 7f1b7b7fe640 1 --2- 172.21.15.164:0/1916134707 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1b680a32d0 0x7f1b680a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:18.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.012+0000 7f1b7b7fe640 1 --2- 172.21.15.164:0/1916134707 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1b680a32d0 0x7f1b680a5790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c009930 tx=0x7f1b6c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:18.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.015+0000 7f1b797fa640 1 -- 172.21.15.164:0/1916134707 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1b64031de0 con 0x7f1b7c102570 2024-09-19T08:44:18.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.176+0000 7f1b81d41640 1 -- 172.21.15.164:0/1916134707 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1b7c062630 con 0x7f1b680a32d0 2024-09-19T08:44:18.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.183+0000 7f1b797fa640 1 -- 172.21.15.164:0/1916134707 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258905 (secure 0 0 0) 0x7f1b7c062630 con 0x7f1b680a32d0 2024-09-19T08:44:18.185 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:44:18.186 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-09-19T08:44:18.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.186+0000 7f1b81d41640 1 -- 172.21.15.164:0/1916134707 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1b680a32d0 msgr2=0x7f1b680a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:18.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.186+0000 7f1b81d41640 1 --2- 172.21.15.164:0/1916134707 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1b680a32d0 0x7f1b680a5790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c009930 tx=0x7f1b6c008040 comp rx=0 tx=0).stop 2024-09-19T08:44:18.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.186+0000 7f1b81d41640 1 -- 172.21.15.164:0/1916134707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b7c102570 msgr2=0x7f1b7c198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:18.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.186+0000 7f1b81d41640 1 --2- 172.21.15.164:0/1916134707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b7c102570 0x7f1b7c198b00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1b64006010 tx=0x7f1b64003c10 comp rx=0 tx=0).stop 2024-09-19T08:44:18.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.186+0000 7f1b81d41640 1 -- 172.21.15.164:0/1916134707 shutdown_connections 2024-09-19T08:44:18.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.186+0000 7f1b81d41640 1 --2- 172.21.15.164:0/1916134707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b7c102ef0 0x7f1b7c19ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:18.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.186+0000 7f1b81d41640 1 --2- 172.21.15.164:0/1916134707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b7c102570 0x7f1b7c198b00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:18.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.186+0000 7f1b81d41640 1 --2- 172.21.15.164:0/1916134707 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1b680a32d0 0x7f1b680a5790 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:18.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.186+0000 7f1b81d41640 1 --2- 172.21.15.164:0/1916134707 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b7c100e20 0x7f1b7c1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:18.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.186+0000 7f1b81d41640 1 -- 172.21.15.164:0/1916134707 >> 172.21.15.164:0/1916134707 conn(0x7f1b7c0fc7d0 msgr2=0x7f1b7c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:18.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.186+0000 7f1b81d41640 1 -- 172.21.15.164:0/1916134707 shutdown_connections 2024-09-19T08:44:18.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:18.186+0000 7f1b81d41640 1 -- 172.21.15.164:0/1916134707 wait complete. 2024-09-19T08:44:18.795 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-09-19T08:44:17.748001+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":644612,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":287,"num_read_kb":258,"num_write":416,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":340,"ondisk_log_size":340,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":389432,"kb_used_data":11052,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124552392,"statfs":{"total":1151940427776,"available":1151541649408,"internally_reserved":0,"allocated":11317248,"data_stored":7153308,"data_compressed":109335,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":47236,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":-5,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":131,"num_read_kb":121,"num_write":94,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003034"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.817273+0000","last_change":"2024-09-19T08:43:52.766709+0000","last_active":"2024-09-19T08:44:02.817273+0000","last_peered":"2024-09-19T08:44:02.817273+0000","last_clean":"2024-09-19T08:44:02.817273+0000","last_became_active":"2024-09-19T08:43:52.765970+0000","last_became_peered":"2024-09-19T08:43:52.765970+0000","last_unstale":"2024-09-19T08:44:02.817273+0000","last_undegraded":"2024-09-19T08:44:02.817273+0000","last_fullsized":"2024-09-19T08:44:02.817273+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:12.176870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819309+0000","last_change":"2024-09-19T08:43:52.766590+0000","last_active":"2024-09-19T08:44:02.819309+0000","last_peered":"2024-09-19T08:44:02.819309+0000","last_clean":"2024-09-19T08:44:02.819309+0000","last_became_active":"2024-09-19T08:43:52.766255+0000","last_became_peered":"2024-09-19T08:43:52.766255+0000","last_unstale":"2024-09-19T08:44:02.819309+0000","last_undegraded":"2024-09-19T08:44:02.819309+0000","last_fullsized":"2024-09-19T08:44:02.819309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:15:41.359806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818510+0000","last_change":"2024-09-19T08:43:52.761956+0000","last_active":"2024-09-19T08:44:02.818510+0000","last_peered":"2024-09-19T08:44:02.818510+0000","last_clean":"2024-09-19T08:44:02.818510+0000","last_became_active":"2024-09-19T08:43:52.761644+0000","last_became_peered":"2024-09-19T08:43:52.761644+0000","last_unstale":"2024-09-19T08:44:02.818510+0000","last_undegraded":"2024-09-19T08:44:02.818510+0000","last_fullsized":"2024-09-19T08:44:02.818510+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:13.770361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.079353+0000","last_change":"2024-09-19T08:43:52.764131+0000","last_active":"2024-09-19T08:44:03.079353+0000","last_peered":"2024-09-19T08:44:03.079353+0000","last_clean":"2024-09-19T08:44:03.079353+0000","last_became_active":"2024-09-19T08:43:52.763812+0000","last_became_peered":"2024-09-19T08:43:52.763812+0000","last_unstale":"2024-09-19T08:44:03.079353+0000","last_undegraded":"2024-09-19T08:44:03.079353+0000","last_fullsized":"2024-09-19T08:44:03.079353+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:32.272157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818711+0000","last_change":"2024-09-19T08:43:52.754673+0000","last_active":"2024-09-19T08:44:02.818711+0000","last_peered":"2024-09-19T08:44:02.818711+0000","last_clean":"2024-09-19T08:44:02.818711+0000","last_became_active":"2024-09-19T08:43:52.754144+0000","last_became_peered":"2024-09-19T08:43:52.754144+0000","last_unstale":"2024-09-19T08:44:02.818711+0000","last_undegraded":"2024-09-19T08:44:02.818711+0000","last_fullsized":"2024-09-19T08:44:02.818711+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:16.069260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.817142+0000","last_change":"2024-09-19T08:43:52.758939+0000","last_active":"2024-09-19T08:44:02.817142+0000","last_peered":"2024-09-19T08:44:02.817142+0000","last_clean":"2024-09-19T08:44:02.817142+0000","last_became_active":"2024-09-19T08:43:52.758611+0000","last_became_peered":"2024-09-19T08:43:52.758611+0000","last_unstale":"2024-09-19T08:44:02.817142+0000","last_undegraded":"2024-09-19T08:44:02.817142+0000","last_fullsized":"2024-09-19T08:44:02.817142+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:35:56.463545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821786+0000","last_change":"2024-09-19T08:43:52.770261+0000","last_active":"2024-09-19T08:44:02.821786+0000","last_peered":"2024-09-19T08:44:02.821786+0000","last_clean":"2024-09-19T08:44:02.821786+0000","last_became_active":"2024-09-19T08:43:52.768253+0000","last_became_peered":"2024-09-19T08:43:52.768253+0000","last_unstale":"2024-09-19T08:44:02.821786+0000","last_undegraded":"2024-09-19T08:44:02.821786+0000","last_fullsized":"2024-09-19T08:44:02.821786+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:40.756988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.493456+0000","last_change":"2024-09-19T08:43:52.768564+0000","last_active":"2024-09-19T08:44:03.493456+0000","last_peered":"2024-09-19T08:44:03.493456+0000","last_clean":"2024-09-19T08:44:03.493456+0000","last_became_active":"2024-09-19T08:43:52.768260+0000","last_became_peered":"2024-09-19T08:43:52.768260+0000","last_unstale":"2024-09-19T08:44:03.493456+0000","last_undegraded":"2024-09-19T08:44:03.493456+0000","last_fullsized":"2024-09-19T08:44:03.493456+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:16.528115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.820687+0000","last_change":"2024-09-19T08:43:52.769804+0000","last_active":"2024-09-19T08:44:02.820687+0000","last_peered":"2024-09-19T08:44:02.820687+0000","last_clean":"2024-09-19T08:44:02.820687+0000","last_became_active":"2024-09-19T08:43:52.769504+0000","last_became_peered":"2024-09-19T08:43:52.769504+0000","last_unstale":"2024-09-19T08:44:02.820687+0000","last_undegraded":"2024-09-19T08:44:02.820687+0000","last_fullsized":"2024-09-19T08:44:02.820687+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:22:38.514410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.817329+0000","last_change":"2024-09-19T08:43:52.763278+0000","last_active":"2024-09-19T08:44:02.817329+0000","last_peered":"2024-09-19T08:44:02.817329+0000","last_clean":"2024-09-19T08:44:02.817329+0000","last_became_active":"2024-09-19T08:43:52.762985+0000","last_became_peered":"2024-09-19T08:43:52.762985+0000","last_unstale":"2024-09-19T08:44:02.817329+0000","last_undegraded":"2024-09-19T08:44:02.817329+0000","last_fullsized":"2024-09-19T08:44:02.817329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:30.048782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.822068+0000","last_change":"2024-09-19T08:43:52.770864+0000","last_active":"2024-09-19T08:44:02.822068+0000","last_peered":"2024-09-19T08:44:02.822068+0000","last_clean":"2024-09-19T08:44:02.822068+0000","last_became_active":"2024-09-19T08:43:52.770518+0000","last_became_peered":"2024-09-19T08:43:52.770518+0000","last_unstale":"2024-09-19T08:44:02.822068+0000","last_undegraded":"2024-09-19T08:44:02.822068+0000","last_fullsized":"2024-09-19T08:44:02.822068+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:08.931452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.079430+0000","last_change":"2024-09-19T08:43:52.757464+0000","last_active":"2024-09-19T08:44:03.079430+0000","last_peered":"2024-09-19T08:44:03.079430+0000","last_clean":"2024-09-19T08:44:03.079430+0000","last_became_active":"2024-09-19T08:43:52.757101+0000","last_became_peered":"2024-09-19T08:43:52.757101+0000","last_unstale":"2024-09-19T08:44:03.079430+0000","last_undegraded":"2024-09-19T08:44:03.079430+0000","last_fullsized":"2024-09-19T08:44:03.079430+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:17.395735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818784+0000","last_change":"2024-09-19T08:43:52.767063+0000","last_active":"2024-09-19T08:44:02.818784+0000","last_peered":"2024-09-19T08:44:02.818784+0000","last_clean":"2024-09-19T08:44:02.818784+0000","last_became_active":"2024-09-19T08:43:52.766755+0000","last_became_peered":"2024-09-19T08:43:52.766755+0000","last_unstale":"2024-09-19T08:44:02.818784+0000","last_undegraded":"2024-09-19T08:44:02.818784+0000","last_fullsized":"2024-09-19T08:44:02.818784+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:30.530937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818799+0000","last_change":"2024-09-19T08:43:52.766515+0000","last_active":"2024-09-19T08:44:02.818799+0000","last_peered":"2024-09-19T08:44:02.818799+0000","last_clean":"2024-09-19T08:44:02.818799+0000","last_became_active":"2024-09-19T08:43:52.765847+0000","last_became_peered":"2024-09-19T08:43:52.765847+0000","last_unstale":"2024-09-19T08:44:02.818799+0000","last_undegraded":"2024-09-19T08:44:02.818799+0000","last_fullsized":"2024-09-19T08:44:02.818799+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:50:35.948663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818391+0000","last_change":"2024-09-19T08:43:52.773702+0000","last_active":"2024-09-19T08:44:02.818391+0000","last_peered":"2024-09-19T08:44:02.818391+0000","last_clean":"2024-09-19T08:44:02.818391+0000","last_became_active":"2024-09-19T08:43:52.773324+0000","last_became_peered":"2024-09-19T08:43:52.773324+0000","last_unstale":"2024-09-19T08:44:02.818391+0000","last_undegraded":"2024-09-19T08:44:02.818391+0000","last_fullsized":"2024-09-19T08:44:02.818391+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:34.368014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.820811+0000","last_change":"2024-09-19T08:43:52.762745+0000","last_active":"2024-09-19T08:44:02.820811+0000","last_peered":"2024-09-19T08:44:02.820811+0000","last_clean":"2024-09-19T08:44:02.820811+0000","last_became_active":"2024-09-19T08:43:52.762172+0000","last_became_peered":"2024-09-19T08:43:52.762172+0000","last_unstale":"2024-09-19T08:44:02.820811+0000","last_undegraded":"2024-09-19T08:44:02.820811+0000","last_fullsized":"2024-09-19T08:44:02.820811+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:57.369775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.817462+0000","last_change":"2024-09-19T08:43:52.767118+0000","last_active":"2024-09-19T08:44:02.817462+0000","last_peered":"2024-09-19T08:44:02.817462+0000","last_clean":"2024-09-19T08:44:02.817462+0000","last_became_active":"2024-09-19T08:43:52.766388+0000","last_became_peered":"2024-09-19T08:43:52.766388+0000","last_unstale":"2024-09-19T08:44:02.817462+0000","last_undegraded":"2024-09-19T08:44:02.817462+0000","last_fullsized":"2024-09-19T08:44:02.817462+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:34.182923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821529+0000","last_change":"2024-09-19T08:43:52.765008+0000","last_active":"2024-09-19T08:44:02.821529+0000","last_peered":"2024-09-19T08:44:02.821529+0000","last_clean":"2024-09-19T08:44:02.821529+0000","last_became_active":"2024-09-19T08:43:52.764558+0000","last_became_peered":"2024-09-19T08:43:52.764558+0000","last_unstale":"2024-09-19T08:44:02.821529+0000","last_undegraded":"2024-09-19T08:44:02.821529+0000","last_fullsized":"2024-09-19T08:44:02.821529+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:52:42.756421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819875+0000","last_change":"2024-09-19T08:43:52.764376+0000","last_active":"2024-09-19T08:44:02.819875+0000","last_peered":"2024-09-19T08:44:02.819875+0000","last_clean":"2024-09-19T08:44:02.819875+0000","last_became_active":"2024-09-19T08:43:52.764048+0000","last_became_peered":"2024-09-19T08:43:52.764048+0000","last_unstale":"2024-09-19T08:44:02.819875+0000","last_undegraded":"2024-09-19T08:44:02.819875+0000","last_fullsized":"2024-09-19T08:44:02.819875+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:56.732094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.827925+0000","last_change":"2024-09-19T08:44:02.827925+0000","last_active":"2024-09-19T08:44:02.827925+0000","last_peered":"2024-09-19T08:44:02.827925+0000","last_clean":"2024-09-19T08:44:02.827925+0000","last_became_active":"2024-09-19T08:44:02.827269+0000","last_became_peered":"2024-09-19T08:44:02.827269+0000","last_unstale":"2024-09-19T08:44:02.827925+0000","last_undegraded":"2024-09-19T08:44:02.827925+0000","last_fullsized":"2024-09-19T08:44:02.827925+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:31:54.556786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818875+0000","last_change":"2024-09-19T08:43:52.762190+0000","last_active":"2024-09-19T08:44:02.818875+0000","last_peered":"2024-09-19T08:44:02.818875+0000","last_clean":"2024-09-19T08:44:02.818875+0000","last_became_active":"2024-09-19T08:43:52.761818+0000","last_became_peered":"2024-09-19T08:43:52.761818+0000","last_unstale":"2024-09-19T08:44:02.818875+0000","last_undegraded":"2024-09-19T08:44:02.818875+0000","last_fullsized":"2024-09-19T08:44:02.818875+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:19:55.246779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.820265+0000","last_change":"2024-09-19T08:43:52.761069+0000","last_active":"2024-09-19T08:44:02.820265+0000","last_peered":"2024-09-19T08:44:02.820265+0000","last_clean":"2024-09-19T08:44:02.820265+0000","last_became_active":"2024-09-19T08:43:52.760705+0000","last_became_peered":"2024-09-19T08:43:52.760705+0000","last_unstale":"2024-09-19T08:44:02.820265+0000","last_undegraded":"2024-09-19T08:44:02.820265+0000","last_fullsized":"2024-09-19T08:44:02.820265+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:46:27.059026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.493512+0000","last_change":"2024-09-19T08:43:52.773449+0000","last_active":"2024-09-19T08:44:03.493512+0000","last_peered":"2024-09-19T08:44:03.493512+0000","last_clean":"2024-09-19T08:44:03.493512+0000","last_became_active":"2024-09-19T08:43:52.772929+0000","last_became_peered":"2024-09-19T08:43:52.772929+0000","last_unstale":"2024-09-19T08:44:03.493512+0000","last_undegraded":"2024-09-19T08:44:03.493512+0000","last_fullsized":"2024-09-19T08:44:03.493512+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:00:38.929199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821652+0000","last_change":"2024-09-19T08:43:52.773885+0000","last_active":"2024-09-19T08:44:02.821652+0000","last_peered":"2024-09-19T08:44:02.821652+0000","last_clean":"2024-09-19T08:44:02.821652+0000","last_became_active":"2024-09-19T08:43:52.772856+0000","last_became_peered":"2024-09-19T08:43:52.772856+0000","last_unstale":"2024-09-19T08:44:02.821652+0000","last_undegraded":"2024-09-19T08:44:02.821652+0000","last_fullsized":"2024-09-19T08:44:02.821652+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:13:22.581335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.493984+0000","last_change":"2024-09-19T08:43:52.762909+0000","last_active":"2024-09-19T08:44:03.493984+0000","last_peered":"2024-09-19T08:44:03.493984+0000","last_clean":"2024-09-19T08:44:03.493984+0000","last_became_active":"2024-09-19T08:43:52.762600+0000","last_became_peered":"2024-09-19T08:43:52.762600+0000","last_unstale":"2024-09-19T08:44:03.493984+0000","last_undegraded":"2024-09-19T08:44:03.493984+0000","last_fullsized":"2024-09-19T08:44:03.493984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:46.098294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.820300+0000","last_change":"2024-09-19T08:43:52.762861+0000","last_active":"2024-09-19T08:44:02.820300+0000","last_peered":"2024-09-19T08:44:02.820300+0000","last_clean":"2024-09-19T08:44:02.820300+0000","last_became_active":"2024-09-19T08:43:52.762525+0000","last_became_peered":"2024-09-19T08:43:52.762525+0000","last_unstale":"2024-09-19T08:44:02.820300+0000","last_undegraded":"2024-09-19T08:44:02.820300+0000","last_fullsized":"2024-09-19T08:44:02.820300+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:54.717249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.816836+0000","last_change":"2024-09-19T08:43:52.768370+0000","last_active":"2024-09-19T08:44:02.816836+0000","last_peered":"2024-09-19T08:44:02.816836+0000","last_clean":"2024-09-19T08:44:02.816836+0000","last_became_active":"2024-09-19T08:43:52.768055+0000","last_became_peered":"2024-09-19T08:43:52.768055+0000","last_unstale":"2024-09-19T08:44:02.816836+0000","last_undegraded":"2024-09-19T08:44:02.816836+0000","last_fullsized":"2024-09-19T08:44:02.816836+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:48.212862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819012+0000","last_change":"2024-09-19T08:43:52.758541+0000","last_active":"2024-09-19T08:44:02.819012+0000","last_peered":"2024-09-19T08:44:02.819012+0000","last_clean":"2024-09-19T08:44:02.819012+0000","last_became_active":"2024-09-19T08:43:52.758122+0000","last_became_peered":"2024-09-19T08:43:52.758122+0000","last_unstale":"2024-09-19T08:44:02.819012+0000","last_undegraded":"2024-09-19T08:44:02.819012+0000","last_fullsized":"2024-09-19T08:44:02.819012+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:54:40.681880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818709+0000","last_change":"2024-09-19T08:43:52.758109+0000","last_active":"2024-09-19T08:44:02.818709+0000","last_peered":"2024-09-19T08:44:02.818709+0000","last_clean":"2024-09-19T08:44:02.818709+0000","last_became_active":"2024-09-19T08:43:52.757808+0000","last_became_peered":"2024-09-19T08:43:52.757808+0000","last_unstale":"2024-09-19T08:44:02.818709+0000","last_undegraded":"2024-09-19T08:44:02.818709+0000","last_fullsized":"2024-09-19T08:44:02.818709+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:02:47.614562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819377+0000","last_change":"2024-09-19T08:43:52.772501+0000","last_active":"2024-09-19T08:44:02.819377+0000","last_peered":"2024-09-19T08:44:02.819377+0000","last_clean":"2024-09-19T08:44:02.819377+0000","last_became_active":"2024-09-19T08:43:52.771611+0000","last_became_peered":"2024-09-19T08:43:52.771611+0000","last_unstale":"2024-09-19T08:44:02.819377+0000","last_undegraded":"2024-09-19T08:44:02.819377+0000","last_fullsized":"2024-09-19T08:44:02.819377+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:49.537237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819339+0000","last_change":"2024-09-19T08:43:52.773664+0000","last_active":"2024-09-19T08:44:02.819339+0000","last_peered":"2024-09-19T08:44:02.819339+0000","last_clean":"2024-09-19T08:44:02.819339+0000","last_became_active":"2024-09-19T08:43:52.773014+0000","last_became_peered":"2024-09-19T08:43:52.773014+0000","last_unstale":"2024-09-19T08:44:02.819339+0000","last_undegraded":"2024-09-19T08:44:02.819339+0000","last_fullsized":"2024-09-19T08:44:02.819339+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:03.448825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.816727+0000","last_change":"2024-09-19T08:43:52.769578+0000","last_active":"2024-09-19T08:44:02.816727+0000","last_peered":"2024-09-19T08:44:02.816727+0000","last_clean":"2024-09-19T08:44:02.816727+0000","last_became_active":"2024-09-19T08:43:52.769252+0000","last_became_peered":"2024-09-19T08:43:52.769252+0000","last_unstale":"2024-09-19T08:44:02.816727+0000","last_undegraded":"2024-09-19T08:44:02.816727+0000","last_fullsized":"2024-09-19T08:44:02.816727+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:27.936118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818612+0000","last_change":"2024-09-19T08:43:52.770453+0000","last_active":"2024-09-19T08:44:02.818612+0000","last_peered":"2024-09-19T08:44:02.818612+0000","last_clean":"2024-09-19T08:44:02.818612+0000","last_became_active":"2024-09-19T08:43:52.770155+0000","last_became_peered":"2024-09-19T08:43:52.770155+0000","last_unstale":"2024-09-19T08:44:02.818612+0000","last_undegraded":"2024-09-19T08:44:02.818612+0000","last_fullsized":"2024-09-19T08:44:02.818612+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:12.893572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:13.645840+0000","last_change":"2024-09-19T08:43:50.751316+0000","last_active":"2024-09-19T08:44:13.645840+0000","last_peered":"2024-09-19T08:44:13.645840+0000","last_clean":"2024-09-19T08:44:13.645840+0000","last_became_active":"2024-09-19T08:43:50.750972+0000","last_became_peered":"2024-09-19T08:43:50.750972+0000","last_unstale":"2024-09-19T08:44:13.645840+0000","last_undegraded":"2024-09-19T08:44:13.645840+0000","last_fullsized":"2024-09-19T08:44:13.645840+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:40.654729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.822390+0000","last_change":"2024-09-19T08:43:52.773979+0000","last_active":"2024-09-19T08:44:02.822390+0000","last_peered":"2024-09-19T08:44:02.822390+0000","last_clean":"2024-09-19T08:44:02.822390+0000","last_became_active":"2024-09-19T08:43:52.773648+0000","last_became_peered":"2024-09-19T08:43:52.773648+0000","last_unstale":"2024-09-19T08:44:02.822390+0000","last_undegraded":"2024-09-19T08:44:02.822390+0000","last_fullsized":"2024-09-19T08:44:02.822390+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:36.061209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.839445+0000","last_change":"2024-09-19T08:43:50.750704+0000","last_active":"2024-09-19T08:44:03.839445+0000","last_peered":"2024-09-19T08:44:03.839445+0000","last_clean":"2024-09-19T08:44:03.839445+0000","last_became_active":"2024-09-19T08:43:50.750399+0000","last_became_peered":"2024-09-19T08:43:50.750399+0000","last_unstale":"2024-09-19T08:44:03.839445+0000","last_undegraded":"2024-09-19T08:44:03.839445+0000","last_fullsized":"2024-09-19T08:44:03.839445+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:25.447320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818514+0000","last_change":"2024-09-19T08:43:50.744919+0000","last_active":"2024-09-19T08:44:02.818514+0000","last_peered":"2024-09-19T08:44:02.818514+0000","last_clean":"2024-09-19T08:44:02.818514+0000","last_became_active":"2024-09-19T08:43:50.744604+0000","last_became_peered":"2024-09-19T08:43:50.744604+0000","last_unstale":"2024-09-19T08:44:02.818514+0000","last_undegraded":"2024-09-19T08:44:02.818514+0000","last_fullsized":"2024-09-19T08:44:02.818514+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:25:06.410093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.493518+0000","last_change":"2024-09-19T08:43:52.766779+0000","last_active":"2024-09-19T08:44:03.493518+0000","last_peered":"2024-09-19T08:44:03.493518+0000","last_clean":"2024-09-19T08:44:03.493518+0000","last_became_active":"2024-09-19T08:43:52.766484+0000","last_became_peered":"2024-09-19T08:43:52.766484+0000","last_unstale":"2024-09-19T08:44:03.493518+0000","last_undegraded":"2024-09-19T08:44:03.493518+0000","last_fullsized":"2024-09-19T08:44:03.493518+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:17:29.350778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.631853+0000","last_change":"2024-09-19T08:43:50.751282+0000","last_active":"2024-09-19T08:44:04.631853+0000","last_peered":"2024-09-19T08:44:04.631853+0000","last_clean":"2024-09-19T08:44:04.631853+0000","last_became_active":"2024-09-19T08:43:50.750754+0000","last_became_peered":"2024-09-19T08:43:50.750754+0000","last_unstale":"2024-09-19T08:44:04.631853+0000","last_undegraded":"2024-09-19T08:44:04.631853+0000","last_fullsized":"2024-09-19T08:44:04.631853+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:11.661626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821354+0000","last_change":"2024-09-19T08:43:52.756442+0000","last_active":"2024-09-19T08:44:02.821354+0000","last_peered":"2024-09-19T08:44:02.821354+0000","last_clean":"2024-09-19T08:44:02.821354+0000","last_became_active":"2024-09-19T08:43:52.756116+0000","last_became_peered":"2024-09-19T08:43:52.756116+0000","last_unstale":"2024-09-19T08:44:02.821354+0000","last_undegraded":"2024-09-19T08:44:02.821354+0000","last_fullsized":"2024-09-19T08:44:02.821354+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:40.029272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:15.647597+0000","last_change":"2024-09-19T08:43:50.749637+0000","last_active":"2024-09-19T08:44:15.647597+0000","last_peered":"2024-09-19T08:44:15.647597+0000","last_clean":"2024-09-19T08:44:15.647597+0000","last_became_active":"2024-09-19T08:43:50.749307+0000","last_became_peered":"2024-09-19T08:43:50.749307+0000","last_unstale":"2024-09-19T08:44:15.647597+0000","last_undegraded":"2024-09-19T08:44:15.647597+0000","last_fullsized":"2024-09-19T08:44:15.647597+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:23:33.409671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.820558+0000","last_change":"2024-09-19T08:43:52.763592+0000","last_active":"2024-09-19T08:44:02.820558+0000","last_peered":"2024-09-19T08:44:02.820558+0000","last_clean":"2024-09-19T08:44:02.820558+0000","last_became_active":"2024-09-19T08:43:52.763146+0000","last_became_peered":"2024-09-19T08:43:52.763146+0000","last_unstale":"2024-09-19T08:44:02.820558+0000","last_undegraded":"2024-09-19T08:44:02.820558+0000","last_fullsized":"2024-09-19T08:44:02.820558+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:32.194465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.638566+0000","last_change":"2024-09-19T08:43:50.749184+0000","last_active":"2024-09-19T08:44:04.638566+0000","last_peered":"2024-09-19T08:44:04.638566+0000","last_clean":"2024-09-19T08:44:04.638566+0000","last_became_active":"2024-09-19T08:43:50.748844+0000","last_became_peered":"2024-09-19T08:43:50.748844+0000","last_unstale":"2024-09-19T08:44:04.638566+0000","last_undegraded":"2024-09-19T08:44:04.638566+0000","last_fullsized":"2024-09-19T08:44:04.638566+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:15.334759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819256+0000","last_change":"2024-09-19T08:43:52.768696+0000","last_active":"2024-09-19T08:44:02.819256+0000","last_peered":"2024-09-19T08:44:02.819256+0000","last_clean":"2024-09-19T08:44:02.819256+0000","last_became_active":"2024-09-19T08:43:52.768374+0000","last_became_peered":"2024-09-19T08:43:52.768374+0000","last_unstale":"2024-09-19T08:44:02.819256+0000","last_undegraded":"2024-09-19T08:44:02.819256+0000","last_fullsized":"2024-09-19T08:44:02.819256+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:34:39.343041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.838113+0000","last_change":"2024-09-19T08:43:51.026270+0000","last_active":"2024-09-19T08:44:03.838113+0000","last_peered":"2024-09-19T08:44:03.838113+0000","last_clean":"2024-09-19T08:44:03.838113+0000","last_became_active":"2024-09-19T08:43:51.025823+0000","last_became_peered":"2024-09-19T08:43:51.025823+0000","last_unstale":"2024-09-19T08:44:03.838113+0000","last_undegraded":"2024-09-19T08:44:03.838113+0000","last_fullsized":"2024-09-19T08:44:03.838113+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:00.480438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.820391+0000","last_change":"2024-09-19T08:43:52.759555+0000","last_active":"2024-09-19T08:44:02.820391+0000","last_peered":"2024-09-19T08:44:02.820391+0000","last_clean":"2024-09-19T08:44:02.820391+0000","last_became_active":"2024-09-19T08:43:52.759223+0000","last_became_peered":"2024-09-19T08:43:52.759223+0000","last_unstale":"2024-09-19T08:44:02.820391+0000","last_undegraded":"2024-09-19T08:44:02.820391+0000","last_fullsized":"2024-09-19T08:44:02.820391+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:57:06.451323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'24","reported_seq":76,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:15.648388+0000","last_change":"2024-09-19T08:43:50.751924+0000","last_active":"2024-09-19T08:44:15.648388+0000","last_peered":"2024-09-19T08:44:15.648388+0000","last_clean":"2024-09-19T08:44:15.648388+0000","last_became_active":"2024-09-19T08:43:50.751117+0000","last_became_peered":"2024-09-19T08:43:50.751117+0000","last_unstale":"2024-09-19T08:44:15.648388+0000","last_undegraded":"2024-09-19T08:44:15.648388+0000","last_fullsized":"2024-09-19T08:44:15.648388+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:33:57.196283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46802,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":30,"num_read_kb":60,"num_write":26,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818871+0000","last_change":"2024-09-19T08:43:52.762260+0000","last_active":"2024-09-19T08:44:02.818871+0000","last_peered":"2024-09-19T08:44:02.818871+0000","last_clean":"2024-09-19T08:44:02.818871+0000","last_became_active":"2024-09-19T08:43:52.761923+0000","last_became_peered":"2024-09-19T08:43:52.761923+0000","last_unstale":"2024-09-19T08:44:02.818871+0000","last_undegraded":"2024-09-19T08:44:02.818871+0000","last_fullsized":"2024-09-19T08:44:02.818871+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:39:39.503332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.638455+0000","last_change":"2024-09-19T08:43:50.749745+0000","last_active":"2024-09-19T08:44:04.638455+0000","last_peered":"2024-09-19T08:44:04.638455+0000","last_clean":"2024-09-19T08:44:04.638455+0000","last_became_active":"2024-09-19T08:43:50.749073+0000","last_became_peered":"2024-09-19T08:43:50.749073+0000","last_unstale":"2024-09-19T08:44:04.638455+0000","last_undegraded":"2024-09-19T08:44:04.638455+0000","last_fullsized":"2024-09-19T08:44:04.638455+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:48:11.965841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818633+0000","last_change":"2024-09-19T08:43:52.761495+0000","last_active":"2024-09-19T08:44:02.818633+0000","last_peered":"2024-09-19T08:44:02.818633+0000","last_clean":"2024-09-19T08:44:02.818633+0000","last_became_active":"2024-09-19T08:43:52.760425+0000","last_became_peered":"2024-09-19T08:43:52.760425+0000","last_unstale":"2024-09-19T08:44:02.818633+0000","last_undegraded":"2024-09-19T08:44:02.818633+0000","last_fullsized":"2024-09-19T08:44:02.818633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:57.689284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.639600+0000","last_change":"2024-09-19T08:43:50.749563+0000","last_active":"2024-09-19T08:44:04.639600+0000","last_peered":"2024-09-19T08:44:04.639600+0000","last_clean":"2024-09-19T08:44:04.639600+0000","last_became_active":"2024-09-19T08:43:50.749244+0000","last_became_peered":"2024-09-19T08:43:50.749244+0000","last_unstale":"2024-09-19T08:44:04.639600+0000","last_undegraded":"2024-09-19T08:44:04.639600+0000","last_fullsized":"2024-09-19T08:44:04.639600+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:59:02.312883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821915+0000","last_change":"2024-09-19T08:43:52.767087+0000","last_active":"2024-09-19T08:44:02.821915+0000","last_peered":"2024-09-19T08:44:02.821915+0000","last_clean":"2024-09-19T08:44:02.821915+0000","last_became_active":"2024-09-19T08:43:52.766730+0000","last_became_peered":"2024-09-19T08:43:52.766730+0000","last_unstale":"2024-09-19T08:44:02.821915+0000","last_undegraded":"2024-09-19T08:44:02.821915+0000","last_fullsized":"2024-09-19T08:44:02.821915+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:04.040896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:13.649953+0000","last_change":"2024-09-19T08:43:50.749659+0000","last_active":"2024-09-19T08:44:13.649953+0000","last_peered":"2024-09-19T08:44:13.649953+0000","last_clean":"2024-09-19T08:44:13.649953+0000","last_became_active":"2024-09-19T08:43:50.749314+0000","last_became_peered":"2024-09-19T08:43:50.749314+0000","last_unstale":"2024-09-19T08:44:13.649953+0000","last_undegraded":"2024-09-19T08:44:13.649953+0000","last_fullsized":"2024-09-19T08:44:13.649953+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:08.576209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821732+0000","last_change":"2024-09-19T08:43:52.767928+0000","last_active":"2024-09-19T08:44:02.821732+0000","last_peered":"2024-09-19T08:44:02.821732+0000","last_clean":"2024-09-19T08:44:02.821732+0000","last_became_active":"2024-09-19T08:43:52.767580+0000","last_became_peered":"2024-09-19T08:43:52.767580+0000","last_unstale":"2024-09-19T08:44:02.821732+0000","last_undegraded":"2024-09-19T08:44:02.821732+0000","last_fullsized":"2024-09-19T08:44:02.821732+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:45.344693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.630057+0000","last_change":"2024-09-19T08:44:02.833887+0000","last_active":"2024-09-19T08:44:04.630057+0000","last_peered":"2024-09-19T08:44:04.630057+0000","last_clean":"2024-09-19T08:44:04.630057+0000","last_became_active":"2024-09-19T08:44:02.821624+0000","last_became_peered":"2024-09-19T08:44:02.821624+0000","last_unstale":"2024-09-19T08:44:04.630057+0000","last_undegraded":"2024-09-19T08:44:04.630057+0000","last_fullsized":"2024-09-19T08:44:04.630057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:54:36.131949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.080821+0000","last_change":"2024-09-19T08:44:03.080503+0000","last_active":"2024-09-19T08:44:03.080821+0000","last_peered":"2024-09-19T08:44:03.080821+0000","last_clean":"2024-09-19T08:44:03.080821+0000","last_became_active":"2024-09-19T08:44:03.080129+0000","last_became_peered":"2024-09-19T08:44:03.080129+0000","last_unstale":"2024-09-19T08:44:03.080821+0000","last_undegraded":"2024-09-19T08:44:03.080821+0000","last_fullsized":"2024-09-19T08:44:03.080821+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:01:37.034795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.629282+0000","last_change":"2024-09-19T08:43:50.747624+0000","last_active":"2024-09-19T08:44:04.629282+0000","last_peered":"2024-09-19T08:44:04.629282+0000","last_clean":"2024-09-19T08:44:04.629282+0000","last_became_active":"2024-09-19T08:43:50.747263+0000","last_became_peered":"2024-09-19T08:43:50.747263+0000","last_unstale":"2024-09-19T08:44:04.629282+0000","last_undegraded":"2024-09-19T08:44:04.629282+0000","last_fullsized":"2024-09-19T08:44:04.629282+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:18.580106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.816903+0000","last_change":"2024-09-19T08:43:52.770051+0000","last_active":"2024-09-19T08:44:02.816903+0000","last_peered":"2024-09-19T08:44:02.816903+0000","last_clean":"2024-09-19T08:44:02.816903+0000","last_became_active":"2024-09-19T08:43:52.769727+0000","last_became_peered":"2024-09-19T08:43:52.769727+0000","last_unstale":"2024-09-19T08:44:02.816903+0000","last_undegraded":"2024-09-19T08:44:02.816903+0000","last_fullsized":"2024-09-19T08:44:02.816903+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:34.954897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.638777+0000","last_change":"2024-09-19T08:43:50.746696+0000","last_active":"2024-09-19T08:44:04.638777+0000","last_peered":"2024-09-19T08:44:04.638777+0000","last_clean":"2024-09-19T08:44:04.638777+0000","last_became_active":"2024-09-19T08:43:50.746246+0000","last_became_peered":"2024-09-19T08:43:50.746246+0000","last_unstale":"2024-09-19T08:44:04.638777+0000","last_undegraded":"2024-09-19T08:44:04.638777+0000","last_fullsized":"2024-09-19T08:44:04.638777+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:38:39.935348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.822242+0000","last_change":"2024-09-19T08:43:52.765199+0000","last_active":"2024-09-19T08:44:02.822242+0000","last_peered":"2024-09-19T08:44:02.822242+0000","last_clean":"2024-09-19T08:44:02.822242+0000","last_became_active":"2024-09-19T08:43:52.764896+0000","last_became_peered":"2024-09-19T08:43:52.764896+0000","last_unstale":"2024-09-19T08:44:02.822242+0000","last_undegraded":"2024-09-19T08:44:02.822242+0000","last_fullsized":"2024-09-19T08:44:02.822242+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:02:59.570018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.628862+0000","last_change":"2024-09-19T08:43:50.747242+0000","last_active":"2024-09-19T08:44:04.628862+0000","last_peered":"2024-09-19T08:44:04.628862+0000","last_clean":"2024-09-19T08:44:04.628862+0000","last_became_active":"2024-09-19T08:43:50.746853+0000","last_became_peered":"2024-09-19T08:43:50.746853+0000","last_unstale":"2024-09-19T08:44:04.628862+0000","last_undegraded":"2024-09-19T08:44:04.628862+0000","last_fullsized":"2024-09-19T08:44:04.628862+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:21.545635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.827031+0000","last_change":"2024-09-19T08:44:02.826535+0000","last_active":"2024-09-19T08:44:02.827031+0000","last_peered":"2024-09-19T08:44:02.827031+0000","last_clean":"2024-09-19T08:44:02.827031+0000","last_became_active":"2024-09-19T08:44:02.826113+0000","last_became_peered":"2024-09-19T08:44:02.826113+0000","last_unstale":"2024-09-19T08:44:02.827031+0000","last_undegraded":"2024-09-19T08:44:02.827031+0000","last_fullsized":"2024-09-19T08:44:02.827031+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:20.247732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'6","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:12.644260+0000","last_change":"2024-09-19T08:43:51.131540+0000","last_active":"2024-09-19T08:44:12.644260+0000","last_peered":"2024-09-19T08:44:12.644260+0000","last_clean":"2024-09-19T08:44:12.644260+0000","last_became_active":"2024-09-19T08:43:51.131170+0000","last_became_peered":"2024-09-19T08:43:51.131170+0000","last_unstale":"2024-09-19T08:44:12.644260+0000","last_undegraded":"2024-09-19T08:44:12.644260+0000","last_fullsized":"2024-09-19T08:44:12.644260+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:43.635612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.822208+0000","last_change":"2024-09-19T08:43:52.763308+0000","last_active":"2024-09-19T08:44:02.822208+0000","last_peered":"2024-09-19T08:44:02.822208+0000","last_clean":"2024-09-19T08:44:02.822208+0000","last_became_active":"2024-09-19T08:43:52.762859+0000","last_became_peered":"2024-09-19T08:43:52.762859+0000","last_unstale":"2024-09-19T08:44:02.822208+0000","last_undegraded":"2024-09-19T08:44:02.822208+0000","last_fullsized":"2024-09-19T08:44:02.822208+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:32:35.083361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":347,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821905+0000","last_change":"2024-09-19T08:42:39.567408+0000","last_active":"2024-09-19T08:44:02.821905+0000","last_peered":"2024-09-19T08:44:02.821905+0000","last_clean":"2024-09-19T08:44:02.821905+0000","last_became_active":"2024-09-19T08:42:39.566967+0000","last_became_peered":"2024-09-19T08:42:39.566967+0000","last_unstale":"2024-09-19T08:44:02.821905+0000","last_undegraded":"2024-09-19T08:44:02.821905+0000","last_fullsized":"2024-09-19T08:44:02.821905+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:14.287944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:14.287944+0000","last_clean_scrub_stamp":"2024-09-19T08:39:14.287944+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:31:03.245386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.638441+0000","last_change":"2024-09-19T08:43:50.751097+0000","last_active":"2024-09-19T08:44:04.638441+0000","last_peered":"2024-09-19T08:44:04.638441+0000","last_clean":"2024-09-19T08:44:04.638441+0000","last_became_active":"2024-09-19T08:43:50.750341+0000","last_became_peered":"2024-09-19T08:43:50.750341+0000","last_unstale":"2024-09-19T08:44:04.638441+0000","last_undegraded":"2024-09-19T08:44:04.638441+0000","last_fullsized":"2024-09-19T08:44:04.638441+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:25.694535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.493670+0000","last_change":"2024-09-19T08:43:52.767905+0000","last_active":"2024-09-19T08:44:03.493670+0000","last_peered":"2024-09-19T08:44:03.493670+0000","last_clean":"2024-09-19T08:44:03.493670+0000","last_became_active":"2024-09-19T08:43:52.767601+0000","last_became_peered":"2024-09-19T08:43:52.767601+0000","last_unstale":"2024-09-19T08:44:03.493670+0000","last_undegraded":"2024-09-19T08:44:03.493670+0000","last_fullsized":"2024-09-19T08:44:03.493670+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:14.519879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.822161+0000","last_change":"2024-09-19T08:43:50.751397+0000","last_active":"2024-09-19T08:44:02.822161+0000","last_peered":"2024-09-19T08:44:02.822161+0000","last_clean":"2024-09-19T08:44:02.822161+0000","last_became_active":"2024-09-19T08:43:50.751079+0000","last_became_peered":"2024-09-19T08:43:50.751079+0000","last_unstale":"2024-09-19T08:44:02.822161+0000","last_undegraded":"2024-09-19T08:44:02.822161+0000","last_fullsized":"2024-09-19T08:44:02.822161+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:29.904786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818371+0000","last_change":"2024-09-19T08:43:52.773348+0000","last_active":"2024-09-19T08:44:02.818371+0000","last_peered":"2024-09-19T08:44:02.818371+0000","last_clean":"2024-09-19T08:44:02.818371+0000","last_became_active":"2024-09-19T08:43:52.773011+0000","last_became_peered":"2024-09-19T08:43:52.773011+0000","last_unstale":"2024-09-19T08:44:02.818371+0000","last_undegraded":"2024-09-19T08:44:02.818371+0000","last_fullsized":"2024-09-19T08:44:02.818371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:10.806578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:14.650708+0000","last_change":"2024-09-19T08:43:51.025300+0000","last_active":"2024-09-19T08:44:14.650708+0000","last_peered":"2024-09-19T08:44:14.650708+0000","last_clean":"2024-09-19T08:44:14.650708+0000","last_became_active":"2024-09-19T08:43:51.024936+0000","last_became_peered":"2024-09-19T08:43:51.024936+0000","last_unstale":"2024-09-19T08:44:14.650708+0000","last_undegraded":"2024-09-19T08:44:14.650708+0000","last_fullsized":"2024-09-19T08:44:14.650708+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:03.186066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821758+0000","last_change":"2024-09-19T08:43:52.774962+0000","last_active":"2024-09-19T08:44:02.821758+0000","last_peered":"2024-09-19T08:44:02.821758+0000","last_clean":"2024-09-19T08:44:02.821758+0000","last_became_active":"2024-09-19T08:43:52.774598+0000","last_became_peered":"2024-09-19T08:43:52.774598+0000","last_unstale":"2024-09-19T08:44:02.821758+0000","last_undegraded":"2024-09-19T08:44:02.821758+0000","last_fullsized":"2024-09-19T08:44:02.821758+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:02.687136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'5","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:13.650866+0000","last_change":"2024-09-19T08:43:50.748180+0000","last_active":"2024-09-19T08:44:13.650866+0000","last_peered":"2024-09-19T08:44:13.650866+0000","last_clean":"2024-09-19T08:44:13.650866+0000","last_became_active":"2024-09-19T08:43:50.747853+0000","last_became_peered":"2024-09-19T08:43:50.747853+0000","last_unstale":"2024-09-19T08:44:13.650866+0000","last_undegraded":"2024-09-19T08:44:13.650866+0000","last_fullsized":"2024-09-19T08:44:13.650866+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:56.047259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.817252+0000","last_change":"2024-09-19T08:43:52.770253+0000","last_active":"2024-09-19T08:44:02.817252+0000","last_peered":"2024-09-19T08:44:02.817252+0000","last_clean":"2024-09-19T08:44:02.817252+0000","last_became_active":"2024-09-19T08:43:52.769951+0000","last_became_peered":"2024-09-19T08:43:52.769951+0000","last_unstale":"2024-09-19T08:44:02.817252+0000","last_undegraded":"2024-09-19T08:44:02.817252+0000","last_fullsized":"2024-09-19T08:44:02.817252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:02.353144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821619+0000","last_change":"2024-09-19T08:43:51.026453+0000","last_active":"2024-09-19T08:44:02.821619+0000","last_peered":"2024-09-19T08:44:02.821619+0000","last_clean":"2024-09-19T08:44:02.821619+0000","last_became_active":"2024-09-19T08:43:51.025873+0000","last_became_peered":"2024-09-19T08:43:51.025873+0000","last_unstale":"2024-09-19T08:44:02.821619+0000","last_undegraded":"2024-09-19T08:44:02.821619+0000","last_fullsized":"2024-09-19T08:44:02.821619+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:25:17.475074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.817186+0000","last_change":"2024-09-19T08:43:52.761700+0000","last_active":"2024-09-19T08:44:02.817186+0000","last_peered":"2024-09-19T08:44:02.817186+0000","last_clean":"2024-09-19T08:44:02.817186+0000","last_became_active":"2024-09-19T08:43:52.755527+0000","last_became_peered":"2024-09-19T08:43:52.755527+0000","last_unstale":"2024-09-19T08:44:02.817186+0000","last_undegraded":"2024-09-19T08:44:02.817186+0000","last_fullsized":"2024-09-19T08:44:02.817186+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:18:43.171284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.630369+0000","last_change":"2024-09-19T08:43:51.026136+0000","last_active":"2024-09-19T08:44:04.630369+0000","last_peered":"2024-09-19T08:44:04.630369+0000","last_clean":"2024-09-19T08:44:04.630369+0000","last_became_active":"2024-09-19T08:43:51.025740+0000","last_became_peered":"2024-09-19T08:43:51.025740+0000","last_unstale":"2024-09-19T08:44:04.630369+0000","last_undegraded":"2024-09-19T08:44:04.630369+0000","last_fullsized":"2024-09-19T08:44:04.630369+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:17:32.741083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819140+0000","last_change":"2024-09-19T08:43:52.763808+0000","last_active":"2024-09-19T08:44:02.819140+0000","last_peered":"2024-09-19T08:44:02.819140+0000","last_clean":"2024-09-19T08:44:02.819140+0000","last_became_active":"2024-09-19T08:43:52.763509+0000","last_became_peered":"2024-09-19T08:43:52.763509+0000","last_unstale":"2024-09-19T08:44:02.819140+0000","last_undegraded":"2024-09-19T08:44:02.819140+0000","last_fullsized":"2024-09-19T08:44:02.819140+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:45:08.385419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'13","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:13.771746+0000","last_change":"2024-09-19T08:43:50.744213+0000","last_active":"2024-09-19T08:44:13.771746+0000","last_peered":"2024-09-19T08:44:13.771746+0000","last_clean":"2024-09-19T08:44:13.771746+0000","last_became_active":"2024-09-19T08:43:50.743848+0000","last_became_peered":"2024-09-19T08:43:50.743848+0000","last_unstale":"2024-09-19T08:44:13.771746+0000","last_undegraded":"2024-09-19T08:44:13.771746+0000","last_fullsized":"2024-09-19T08:44:13.771746+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:26.613055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.494010+0000","last_change":"2024-09-19T08:43:52.767102+0000","last_active":"2024-09-19T08:44:03.494010+0000","last_peered":"2024-09-19T08:44:03.494010+0000","last_clean":"2024-09-19T08:44:03.494010+0000","last_became_active":"2024-09-19T08:43:52.766262+0000","last_became_peered":"2024-09-19T08:43:52.766262+0000","last_unstale":"2024-09-19T08:44:03.494010+0000","last_undegraded":"2024-09-19T08:44:03.494010+0000","last_fullsized":"2024-09-19T08:44:03.494010+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:35.740462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.638198+0000","last_change":"2024-09-19T08:43:51.131742+0000","last_active":"2024-09-19T08:44:04.638198+0000","last_peered":"2024-09-19T08:44:04.638198+0000","last_clean":"2024-09-19T08:44:04.638198+0000","last_became_active":"2024-09-19T08:43:51.131388+0000","last_became_peered":"2024-09-19T08:43:51.131388+0000","last_unstale":"2024-09-19T08:44:04.638198+0000","last_undegraded":"2024-09-19T08:44:04.638198+0000","last_fullsized":"2024-09-19T08:44:04.638198+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:26.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819357+0000","last_change":"2024-09-19T08:43:52.756004+0000","last_active":"2024-09-19T08:44:02.819357+0000","last_peered":"2024-09-19T08:44:02.819357+0000","last_clean":"2024-09-19T08:44:02.819357+0000","last_became_active":"2024-09-19T08:43:52.755596+0000","last_became_peered":"2024-09-19T08:43:52.755596+0000","last_unstale":"2024-09-19T08:44:02.819357+0000","last_undegraded":"2024-09-19T08:44:02.819357+0000","last_fullsized":"2024-09-19T08:44:02.819357+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:35.942375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.835986+0000","last_change":"2024-09-19T08:44:02.835986+0000","last_active":"2024-09-19T08:44:02.835986+0000","last_peered":"2024-09-19T08:44:02.835986+0000","last_clean":"2024-09-19T08:44:02.835986+0000","last_became_active":"2024-09-19T08:44:02.826314+0000","last_became_peered":"2024-09-19T08:44:02.826314+0000","last_unstale":"2024-09-19T08:44:02.835986+0000","last_undegraded":"2024-09-19T08:44:02.835986+0000","last_fullsized":"2024-09-19T08:44:02.835986+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:16.943989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818539+0000","last_change":"2024-09-19T08:43:52.766582+0000","last_active":"2024-09-19T08:44:02.818539+0000","last_peered":"2024-09-19T08:44:02.818539+0000","last_clean":"2024-09-19T08:44:02.818539+0000","last_became_active":"2024-09-19T08:43:52.766232+0000","last_became_peered":"2024-09-19T08:43:52.766232+0000","last_unstale":"2024-09-19T08:44:02.818539+0000","last_undegraded":"2024-09-19T08:44:02.818539+0000","last_fullsized":"2024-09-19T08:44:02.818539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:43:42.319422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.635810+0000","last_change":"2024-09-19T08:43:50.747611+0000","last_active":"2024-09-19T08:44:04.635810+0000","last_peered":"2024-09-19T08:44:04.635810+0000","last_clean":"2024-09-19T08:44:04.635810+0000","last_became_active":"2024-09-19T08:43:50.747263+0000","last_became_peered":"2024-09-19T08:43:50.747263+0000","last_unstale":"2024-09-19T08:44:04.635810+0000","last_undegraded":"2024-09-19T08:44:04.635810+0000","last_fullsized":"2024-09-19T08:44:04.635810+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:14.717820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.816851+0000","last_change":"2024-09-19T08:43:52.766493+0000","last_active":"2024-09-19T08:44:02.816851+0000","last_peered":"2024-09-19T08:44:02.816851+0000","last_clean":"2024-09-19T08:44:02.816851+0000","last_became_active":"2024-09-19T08:43:52.766158+0000","last_became_peered":"2024-09-19T08:43:52.766158+0000","last_unstale":"2024-09-19T08:44:02.816851+0000","last_undegraded":"2024-09-19T08:44:02.816851+0000","last_fullsized":"2024-09-19T08:44:02.816851+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:59.175949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.838733+0000","last_change":"2024-09-19T08:43:50.751368+0000","last_active":"2024-09-19T08:44:03.838733+0000","last_peered":"2024-09-19T08:44:03.838733+0000","last_clean":"2024-09-19T08:44:03.838733+0000","last_became_active":"2024-09-19T08:43:50.751038+0000","last_became_peered":"2024-09-19T08:43:50.751038+0000","last_unstale":"2024-09-19T08:44:03.838733+0000","last_undegraded":"2024-09-19T08:44:03.838733+0000","last_fullsized":"2024-09-19T08:44:03.838733+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:46.302996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.493641+0000","last_change":"2024-09-19T08:43:52.756998+0000","last_active":"2024-09-19T08:44:03.493641+0000","last_peered":"2024-09-19T08:44:03.493641+0000","last_clean":"2024-09-19T08:44:03.493641+0000","last_became_active":"2024-09-19T08:43:52.756661+0000","last_became_peered":"2024-09-19T08:43:52.756661+0000","last_unstale":"2024-09-19T08:44:03.493641+0000","last_undegraded":"2024-09-19T08:44:03.493641+0000","last_fullsized":"2024-09-19T08:44:03.493641+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:15.182574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818827+0000","last_change":"2024-09-19T08:43:50.750095+0000","last_active":"2024-09-19T08:44:02.818827+0000","last_peered":"2024-09-19T08:44:02.818827+0000","last_clean":"2024-09-19T08:44:02.818827+0000","last_became_active":"2024-09-19T08:43:50.749672+0000","last_became_peered":"2024-09-19T08:43:50.749672+0000","last_unstale":"2024-09-19T08:44:02.818827+0000","last_undegraded":"2024-09-19T08:44:02.818827+0000","last_fullsized":"2024-09-19T08:44:02.818827+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:25:34.978094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821907+0000","last_change":"2024-09-19T08:43:52.764097+0000","last_active":"2024-09-19T08:44:02.821907+0000","last_peered":"2024-09-19T08:44:02.821907+0000","last_clean":"2024-09-19T08:44:02.821907+0000","last_became_active":"2024-09-19T08:43:52.763773+0000","last_became_peered":"2024-09-19T08:43:52.763773+0000","last_unstale":"2024-09-19T08:44:02.821907+0000","last_undegraded":"2024-09-19T08:44:02.821907+0000","last_fullsized":"2024-09-19T08:44:02.821907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:01:42.098460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.825238+0000","last_change":"2024-09-19T08:44:02.825238+0000","last_active":"2024-09-19T08:44:02.825238+0000","last_peered":"2024-09-19T08:44:02.825238+0000","last_clean":"2024-09-19T08:44:02.825238+0000","last_became_active":"2024-09-19T08:44:02.824873+0000","last_became_peered":"2024-09-19T08:44:02.824873+0000","last_unstale":"2024-09-19T08:44:02.825238+0000","last_undegraded":"2024-09-19T08:44:02.825238+0000","last_fullsized":"2024-09-19T08:44:02.825238+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:38.896082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.639612+0000","last_change":"2024-09-19T08:43:50.748789+0000","last_active":"2024-09-19T08:44:04.639612+0000","last_peered":"2024-09-19T08:44:04.639612+0000","last_clean":"2024-09-19T08:44:04.639612+0000","last_became_active":"2024-09-19T08:43:50.748464+0000","last_became_peered":"2024-09-19T08:43:50.748464+0000","last_unstale":"2024-09-19T08:44:04.639612+0000","last_undegraded":"2024-09-19T08:44:04.639612+0000","last_fullsized":"2024-09-19T08:44:04.639612+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:36:45.905639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.820329+0000","last_change":"2024-09-19T08:43:52.756608+0000","last_active":"2024-09-19T08:44:02.820329+0000","last_peered":"2024-09-19T08:44:02.820329+0000","last_clean":"2024-09-19T08:44:02.820329+0000","last_became_active":"2024-09-19T08:43:52.756227+0000","last_became_peered":"2024-09-19T08:43:52.756227+0000","last_unstale":"2024-09-19T08:44:02.820329+0000","last_undegraded":"2024-09-19T08:44:02.820329+0000","last_fullsized":"2024-09-19T08:44:02.820329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:29:58.085584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'12","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.639102+0000","last_change":"2024-09-19T08:43:50.749135+0000","last_active":"2024-09-19T08:44:04.639102+0000","last_peered":"2024-09-19T08:44:04.639102+0000","last_clean":"2024-09-19T08:44:04.639102+0000","last_became_active":"2024-09-19T08:43:50.748755+0000","last_became_peered":"2024-09-19T08:43:50.748755+0000","last_unstale":"2024-09-19T08:44:04.639102+0000","last_undegraded":"2024-09-19T08:44:04.639102+0000","last_fullsized":"2024-09-19T08:44:04.639102+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:55.079780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819170+0000","last_change":"2024-09-19T08:43:52.763277+0000","last_active":"2024-09-19T08:44:02.819170+0000","last_peered":"2024-09-19T08:44:02.819170+0000","last_clean":"2024-09-19T08:44:02.819170+0000","last_became_active":"2024-09-19T08:43:52.762941+0000","last_became_peered":"2024-09-19T08:43:52.762941+0000","last_unstale":"2024-09-19T08:44:02.819170+0000","last_undegraded":"2024-09-19T08:44:02.819170+0000","last_fullsized":"2024-09-19T08:44:02.819170+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:35.531946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.494005+0000","last_change":"2024-09-19T08:43:51.025350+0000","last_active":"2024-09-19T08:44:03.494005+0000","last_peered":"2024-09-19T08:44:03.494005+0000","last_clean":"2024-09-19T08:44:03.494005+0000","last_became_active":"2024-09-19T08:43:51.025046+0000","last_became_peered":"2024-09-19T08:43:51.025046+0000","last_unstale":"2024-09-19T08:44:03.494005+0000","last_undegraded":"2024-09-19T08:44:03.494005+0000","last_fullsized":"2024-09-19T08:44:03.494005+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:27.595545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.493915+0000","last_change":"2024-09-19T08:43:52.767917+0000","last_active":"2024-09-19T08:44:03.493915+0000","last_peered":"2024-09-19T08:44:03.493915+0000","last_clean":"2024-09-19T08:44:03.493915+0000","last_became_active":"2024-09-19T08:43:52.767575+0000","last_became_peered":"2024-09-19T08:43:52.767575+0000","last_unstale":"2024-09-19T08:44:03.493915+0000","last_undegraded":"2024-09-19T08:44:03.493915+0000","last_fullsized":"2024-09-19T08:44:03.493915+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:05.950399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'13","reported_seq":47,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:14.651442+0000","last_change":"2024-09-19T08:43:50.746400+0000","last_active":"2024-09-19T08:44:14.651442+0000","last_peered":"2024-09-19T08:44:14.651442+0000","last_clean":"2024-09-19T08:44:14.651442+0000","last_became_active":"2024-09-19T08:43:50.745891+0000","last_became_peered":"2024-09-19T08:43:50.745891+0000","last_unstale":"2024-09-19T08:44:14.651442+0000","last_undegraded":"2024-09-19T08:44:14.651442+0000","last_fullsized":"2024-09-19T08:44:14.651442+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:03.098202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":54010,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":151,"num_read_kb":139,"num_write":233,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":162030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":201,"ondisk_log_size":201,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":250201,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579922,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":436653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":1073979,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005083,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":436195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234016,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":436344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462949,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":970752,"data_stored":1028976,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593771,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":435795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822703,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":983040,"data_stored":1028654,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986228,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":905216,"data_stored":435656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215161,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":435503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":974848,"data_stored":484516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":954368,"data_stored":483510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":925696,"data_stored":437527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34807,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34807,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34807,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":48554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":49560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":48538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:44:18.800 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T08:44:19.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:18 smithi164 ceph-mon[30928]: pgmap v290: 97 pgs: 97 active+clean; 630 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 18 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T08:44:19.050 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:19.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:18 smithi170 ceph-mon[35072]: pgmap v290: 97 pgs: 97 active+clean; 630 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 18 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T08:44:19.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:18 smithi173 ceph-mon[35306]: pgmap v290: 97 pgs: 97 active+clean; 630 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 18 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T08:44:19.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.573+0000 7f0be6551640 1 -- 172.21.15.164:0/4121872395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0be0100e20 msgr2=0x7f0be0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:19.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.573+0000 7f0be6551640 1 --2- 172.21.15.164:0/4121872395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0be0100e20 0x7f0be0101200 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0009900 tx=0x7f0bd002a480 comp rx=0 tx=0).stop 2024-09-19T08:44:19.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.573+0000 7f0be6551640 1 -- 172.21.15.164:0/4121872395 shutdown_connections 2024-09-19T08:44:19.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.573+0000 7f0be6551640 1 --2- 172.21.15.164:0/4121872395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0be0102ef0 0x7f0be010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:19.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.573+0000 7f0be6551640 1 --2- 172.21.15.164:0/4121872395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0be0102570 0x7f0be01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:19.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.573+0000 7f0be6551640 1 --2- 172.21.15.164:0/4121872395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0be0100e20 0x7f0be0101200 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:19.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.573+0000 7f0be6551640 1 -- 172.21.15.164:0/4121872395 >> 172.21.15.164:0/4121872395 conn(0x7f0be00fc7d0 msgr2=0x7f0be00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:19.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.573+0000 7f0be6551640 1 -- 172.21.15.164:0/4121872395 shutdown_connections 2024-09-19T08:44:19.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.573+0000 7f0be6551640 1 -- 172.21.15.164:0/4121872395 wait complete. 2024-09-19T08:44:19.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.574+0000 7f0be6551640 1 Processor -- start 2024-09-19T08:44:19.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.574+0000 7f0be6551640 1 -- start start 2024-09-19T08:44:19.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.574+0000 7f0be6551640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0be0100e20 0x7f0be01994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:19.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.574+0000 7f0be6551640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0be0102570 0x7f0be0199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:19.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.574+0000 7f0be6551640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0be0102ef0 0x7f0be01a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:19.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.574+0000 7f0be6551640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0be0111020 con 0x7f0be0102570 2024-09-19T08:44:19.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.574+0000 7f0be6551640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0be0110ea0 con 0x7f0be0100e20 2024-09-19T08:44:19.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.574+0000 7f0be6551640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0be01111a0 con 0x7f0be0102ef0 2024-09-19T08:44:19.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.574+0000 7f0bdf7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0be0102570 0x7f0be0199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:19.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.574+0000 7f0bdf7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0be0102570 0x7f0be0199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35692/0 (socket says 172.21.15.164:35692) 2024-09-19T08:44:19.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.574+0000 7f0bdf7fe640 1 -- 172.21.15.164:0/2155052243 learned_addr learned my addr 172.21.15.164:0/2155052243 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:19.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.575+0000 7f0bdffff640 1 --2- 172.21.15.164:0/2155052243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0be0100e20 0x7f0be01994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:19.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.575+0000 7f0bdf7fe640 1 -- 172.21.15.164:0/2155052243 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0be0102ef0 msgr2=0x7f0be01a4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:19.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.575+0000 7f0bdf7fe640 1 --2- 172.21.15.164:0/2155052243 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0be0102ef0 0x7f0be01a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:19.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.575+0000 7f0bdf7fe640 1 -- 172.21.15.164:0/2155052243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0be0100e20 msgr2=0x7f0be01994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:19.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.575+0000 7f0bdf7fe640 1 --2- 172.21.15.164:0/2155052243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0be0100e20 0x7f0be01994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:19.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.575+0000 7f0bdf7fe640 1 -- 172.21.15.164:0/2155052243 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be01a4ab0 con 0x7f0be0102570 2024-09-19T08:44:19.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.575+0000 7f0bdf7fe640 1 --2- 172.21.15.164:0/2155052243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0be0102570 0x7f0be0199a30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0bd400b4a0 tx=0x7f0bd400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:19.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.575+0000 7f0bdd7fa640 1 -- 172.21.15.164:0/2155052243 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4014070 con 0x7f0be0102570 2024-09-19T08:44:19.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.575+0000 7f0be6551640 1 -- 172.21.15.164:0/2155052243 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be01a4c80 con 0x7f0be0102570 2024-09-19T08:44:19.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.575+0000 7f0bdd7fa640 1 -- 172.21.15.164:0/2155052243 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bd4010af0 con 0x7f0be0102570 2024-09-19T08:44:19.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.575+0000 7f0bdd7fa640 1 -- 172.21.15.164:0/2155052243 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4017d10 con 0x7f0be0102570 2024-09-19T08:44:19.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.575+0000 7f0be6551640 1 -- 172.21.15.164:0/2155052243 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be01a5140 con 0x7f0be0102570 2024-09-19T08:44:19.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.576+0000 7f0be6551640 1 -- 172.21.15.164:0/2155052243 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0be0101280 con 0x7f0be0102570 2024-09-19T08:44:19.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.577+0000 7f0bdd7fa640 1 -- 172.21.15.164:0/2155052243 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f0bd4017070 con 0x7f0be0102570 2024-09-19T08:44:19.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.578+0000 7f0bdd7fa640 1 --2- 172.21.15.164:0/2155052243 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f0bc80a32d0 0x7f0bc80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:19.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.578+0000 7f0bdffff640 1 --2- 172.21.15.164:0/2155052243 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f0bc80a32d0 0x7f0bc80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:19.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.578+0000 7f0bdd7fa640 1 -- 172.21.15.164:0/2155052243 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f0bd40ced10 con 0x7f0be0102570 2024-09-19T08:44:19.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.578+0000 7f0bdffff640 1 --2- 172.21.15.164:0/2155052243 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f0bc80a32d0 0x7f0bc80a5790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0005ab0 tx=0x7f0bd0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:19.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.582+0000 7f0bdd7fa640 1 -- 172.21.15.164:0/2155052243 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0bd409bae0 con 0x7f0be0102570 2024-09-19T08:44:19.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.747+0000 7f0be6551640 1 -- 172.21.15.164:0/2155052243 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0be0062630 con 0x7f0bc80a32d0 2024-09-19T08:44:19.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.754+0000 7f0bdd7fa640 1 -- 172.21.15.164:0/2155052243 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258903 (secure 0 0 0) 0x7f0be0062630 con 0x7f0bc80a32d0 2024-09-19T08:44:19.756 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:44:19.757 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-09-19T08:44:19.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.757+0000 7f0be6551640 1 -- 172.21.15.164:0/2155052243 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f0bc80a32d0 msgr2=0x7f0bc80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:19.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.757+0000 7f0be6551640 1 --2- 172.21.15.164:0/2155052243 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f0bc80a32d0 0x7f0bc80a5790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0005ab0 tx=0x7f0bd0005a20 comp rx=0 tx=0).stop 2024-09-19T08:44:19.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.757+0000 7f0be6551640 1 -- 172.21.15.164:0/2155052243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0be0102570 msgr2=0x7f0be0199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:19.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.757+0000 7f0be6551640 1 --2- 172.21.15.164:0/2155052243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0be0102570 0x7f0be0199a30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0bd400b4a0 tx=0x7f0bd400b960 comp rx=0 tx=0).stop 2024-09-19T08:44:19.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.757+0000 7f0be6551640 1 -- 172.21.15.164:0/2155052243 shutdown_connections 2024-09-19T08:44:19.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.757+0000 7f0be6551640 1 --2- 172.21.15.164:0/2155052243 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f0bc80a32d0 0x7f0bc80a5790 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:19.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.757+0000 7f0be6551640 1 --2- 172.21.15.164:0/2155052243 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0be0102ef0 0x7f0be01a4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:19.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.757+0000 7f0be6551640 1 --2- 172.21.15.164:0/2155052243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0be0102570 0x7f0be0199a30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:19.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.757+0000 7f0be6551640 1 --2- 172.21.15.164:0/2155052243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0be0100e20 0x7f0be01994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:19.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.757+0000 7f0be6551640 1 -- 172.21.15.164:0/2155052243 >> 172.21.15.164:0/2155052243 conn(0x7f0be00fc7d0 msgr2=0x7f0be00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:19.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.757+0000 7f0be6551640 1 -- 172.21.15.164:0/2155052243 shutdown_connections 2024-09-19T08:44:19.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:19.757+0000 7f0be6551640 1 -- 172.21.15.164:0/2155052243 wait complete. 2024-09-19T08:44:20.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:19 smithi164 ceph-mon[30928]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:44:20.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:19 smithi170 ceph-mon[35072]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:44:20.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:19 smithi173 ceph-mon[35306]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:44:20.628 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-09-19T08:44:19.748391+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":644612,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":292,"num_read_kb":258,"num_write":416,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":340,"ondisk_log_size":340,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":389448,"kb_used_data":11068,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124552376,"statfs":{"total":1151940427776,"available":1151541633024,"internally_reserved":0,"allocated":11333632,"data_stored":7169612,"data_compressed":109335,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":46832,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":120,"num_read_kb":107,"num_write":44,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002919"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.817273+0000","last_change":"2024-09-19T08:43:52.766709+0000","last_active":"2024-09-19T08:44:02.817273+0000","last_peered":"2024-09-19T08:44:02.817273+0000","last_clean":"2024-09-19T08:44:02.817273+0000","last_became_active":"2024-09-19T08:43:52.765970+0000","last_became_peered":"2024-09-19T08:43:52.765970+0000","last_unstale":"2024-09-19T08:44:02.817273+0000","last_undegraded":"2024-09-19T08:44:02.817273+0000","last_fullsized":"2024-09-19T08:44:02.817273+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:12.176870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819309+0000","last_change":"2024-09-19T08:43:52.766590+0000","last_active":"2024-09-19T08:44:02.819309+0000","last_peered":"2024-09-19T08:44:02.819309+0000","last_clean":"2024-09-19T08:44:02.819309+0000","last_became_active":"2024-09-19T08:43:52.766255+0000","last_became_peered":"2024-09-19T08:43:52.766255+0000","last_unstale":"2024-09-19T08:44:02.819309+0000","last_undegraded":"2024-09-19T08:44:02.819309+0000","last_fullsized":"2024-09-19T08:44:02.819309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:15:41.359806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818510+0000","last_change":"2024-09-19T08:43:52.761956+0000","last_active":"2024-09-19T08:44:02.818510+0000","last_peered":"2024-09-19T08:44:02.818510+0000","last_clean":"2024-09-19T08:44:02.818510+0000","last_became_active":"2024-09-19T08:43:52.761644+0000","last_became_peered":"2024-09-19T08:43:52.761644+0000","last_unstale":"2024-09-19T08:44:02.818510+0000","last_undegraded":"2024-09-19T08:44:02.818510+0000","last_fullsized":"2024-09-19T08:44:02.818510+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:13.770361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.079353+0000","last_change":"2024-09-19T08:43:52.764131+0000","last_active":"2024-09-19T08:44:03.079353+0000","last_peered":"2024-09-19T08:44:03.079353+0000","last_clean":"2024-09-19T08:44:03.079353+0000","last_became_active":"2024-09-19T08:43:52.763812+0000","last_became_peered":"2024-09-19T08:43:52.763812+0000","last_unstale":"2024-09-19T08:44:03.079353+0000","last_undegraded":"2024-09-19T08:44:03.079353+0000","last_fullsized":"2024-09-19T08:44:03.079353+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:32.272157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818711+0000","last_change":"2024-09-19T08:43:52.754673+0000","last_active":"2024-09-19T08:44:02.818711+0000","last_peered":"2024-09-19T08:44:02.818711+0000","last_clean":"2024-09-19T08:44:02.818711+0000","last_became_active":"2024-09-19T08:43:52.754144+0000","last_became_peered":"2024-09-19T08:43:52.754144+0000","last_unstale":"2024-09-19T08:44:02.818711+0000","last_undegraded":"2024-09-19T08:44:02.818711+0000","last_fullsized":"2024-09-19T08:44:02.818711+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:16.069260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.817142+0000","last_change":"2024-09-19T08:43:52.758939+0000","last_active":"2024-09-19T08:44:02.817142+0000","last_peered":"2024-09-19T08:44:02.817142+0000","last_clean":"2024-09-19T08:44:02.817142+0000","last_became_active":"2024-09-19T08:43:52.758611+0000","last_became_peered":"2024-09-19T08:43:52.758611+0000","last_unstale":"2024-09-19T08:44:02.817142+0000","last_undegraded":"2024-09-19T08:44:02.817142+0000","last_fullsized":"2024-09-19T08:44:02.817142+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:35:56.463545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821786+0000","last_change":"2024-09-19T08:43:52.770261+0000","last_active":"2024-09-19T08:44:02.821786+0000","last_peered":"2024-09-19T08:44:02.821786+0000","last_clean":"2024-09-19T08:44:02.821786+0000","last_became_active":"2024-09-19T08:43:52.768253+0000","last_became_peered":"2024-09-19T08:43:52.768253+0000","last_unstale":"2024-09-19T08:44:02.821786+0000","last_undegraded":"2024-09-19T08:44:02.821786+0000","last_fullsized":"2024-09-19T08:44:02.821786+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:40.756988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.493456+0000","last_change":"2024-09-19T08:43:52.768564+0000","last_active":"2024-09-19T08:44:03.493456+0000","last_peered":"2024-09-19T08:44:03.493456+0000","last_clean":"2024-09-19T08:44:03.493456+0000","last_became_active":"2024-09-19T08:43:52.768260+0000","last_became_peered":"2024-09-19T08:43:52.768260+0000","last_unstale":"2024-09-19T08:44:03.493456+0000","last_undegraded":"2024-09-19T08:44:03.493456+0000","last_fullsized":"2024-09-19T08:44:03.493456+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:16.528115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.820687+0000","last_change":"2024-09-19T08:43:52.769804+0000","last_active":"2024-09-19T08:44:02.820687+0000","last_peered":"2024-09-19T08:44:02.820687+0000","last_clean":"2024-09-19T08:44:02.820687+0000","last_became_active":"2024-09-19T08:43:52.769504+0000","last_became_peered":"2024-09-19T08:43:52.769504+0000","last_unstale":"2024-09-19T08:44:02.820687+0000","last_undegraded":"2024-09-19T08:44:02.820687+0000","last_fullsized":"2024-09-19T08:44:02.820687+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:22:38.514410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.817329+0000","last_change":"2024-09-19T08:43:52.763278+0000","last_active":"2024-09-19T08:44:02.817329+0000","last_peered":"2024-09-19T08:44:02.817329+0000","last_clean":"2024-09-19T08:44:02.817329+0000","last_became_active":"2024-09-19T08:43:52.762985+0000","last_became_peered":"2024-09-19T08:43:52.762985+0000","last_unstale":"2024-09-19T08:44:02.817329+0000","last_undegraded":"2024-09-19T08:44:02.817329+0000","last_fullsized":"2024-09-19T08:44:02.817329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:30.048782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.822068+0000","last_change":"2024-09-19T08:43:52.770864+0000","last_active":"2024-09-19T08:44:02.822068+0000","last_peered":"2024-09-19T08:44:02.822068+0000","last_clean":"2024-09-19T08:44:02.822068+0000","last_became_active":"2024-09-19T08:43:52.770518+0000","last_became_peered":"2024-09-19T08:43:52.770518+0000","last_unstale":"2024-09-19T08:44:02.822068+0000","last_undegraded":"2024-09-19T08:44:02.822068+0000","last_fullsized":"2024-09-19T08:44:02.822068+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:08.931452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.079430+0000","last_change":"2024-09-19T08:43:52.757464+0000","last_active":"2024-09-19T08:44:03.079430+0000","last_peered":"2024-09-19T08:44:03.079430+0000","last_clean":"2024-09-19T08:44:03.079430+0000","last_became_active":"2024-09-19T08:43:52.757101+0000","last_became_peered":"2024-09-19T08:43:52.757101+0000","last_unstale":"2024-09-19T08:44:03.079430+0000","last_undegraded":"2024-09-19T08:44:03.079430+0000","last_fullsized":"2024-09-19T08:44:03.079430+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:17.395735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818784+0000","last_change":"2024-09-19T08:43:52.767063+0000","last_active":"2024-09-19T08:44:02.818784+0000","last_peered":"2024-09-19T08:44:02.818784+0000","last_clean":"2024-09-19T08:44:02.818784+0000","last_became_active":"2024-09-19T08:43:52.766755+0000","last_became_peered":"2024-09-19T08:43:52.766755+0000","last_unstale":"2024-09-19T08:44:02.818784+0000","last_undegraded":"2024-09-19T08:44:02.818784+0000","last_fullsized":"2024-09-19T08:44:02.818784+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:30.530937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818799+0000","last_change":"2024-09-19T08:43:52.766515+0000","last_active":"2024-09-19T08:44:02.818799+0000","last_peered":"2024-09-19T08:44:02.818799+0000","last_clean":"2024-09-19T08:44:02.818799+0000","last_became_active":"2024-09-19T08:43:52.765847+0000","last_became_peered":"2024-09-19T08:43:52.765847+0000","last_unstale":"2024-09-19T08:44:02.818799+0000","last_undegraded":"2024-09-19T08:44:02.818799+0000","last_fullsized":"2024-09-19T08:44:02.818799+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:50:35.948663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818391+0000","last_change":"2024-09-19T08:43:52.773702+0000","last_active":"2024-09-19T08:44:02.818391+0000","last_peered":"2024-09-19T08:44:02.818391+0000","last_clean":"2024-09-19T08:44:02.818391+0000","last_became_active":"2024-09-19T08:43:52.773324+0000","last_became_peered":"2024-09-19T08:43:52.773324+0000","last_unstale":"2024-09-19T08:44:02.818391+0000","last_undegraded":"2024-09-19T08:44:02.818391+0000","last_fullsized":"2024-09-19T08:44:02.818391+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:34.368014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.820811+0000","last_change":"2024-09-19T08:43:52.762745+0000","last_active":"2024-09-19T08:44:02.820811+0000","last_peered":"2024-09-19T08:44:02.820811+0000","last_clean":"2024-09-19T08:44:02.820811+0000","last_became_active":"2024-09-19T08:43:52.762172+0000","last_became_peered":"2024-09-19T08:43:52.762172+0000","last_unstale":"2024-09-19T08:44:02.820811+0000","last_undegraded":"2024-09-19T08:44:02.820811+0000","last_fullsized":"2024-09-19T08:44:02.820811+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:57.369775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.817462+0000","last_change":"2024-09-19T08:43:52.767118+0000","last_active":"2024-09-19T08:44:02.817462+0000","last_peered":"2024-09-19T08:44:02.817462+0000","last_clean":"2024-09-19T08:44:02.817462+0000","last_became_active":"2024-09-19T08:43:52.766388+0000","last_became_peered":"2024-09-19T08:43:52.766388+0000","last_unstale":"2024-09-19T08:44:02.817462+0000","last_undegraded":"2024-09-19T08:44:02.817462+0000","last_fullsized":"2024-09-19T08:44:02.817462+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:34.182923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821529+0000","last_change":"2024-09-19T08:43:52.765008+0000","last_active":"2024-09-19T08:44:02.821529+0000","last_peered":"2024-09-19T08:44:02.821529+0000","last_clean":"2024-09-19T08:44:02.821529+0000","last_became_active":"2024-09-19T08:43:52.764558+0000","last_became_peered":"2024-09-19T08:43:52.764558+0000","last_unstale":"2024-09-19T08:44:02.821529+0000","last_undegraded":"2024-09-19T08:44:02.821529+0000","last_fullsized":"2024-09-19T08:44:02.821529+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:52:42.756421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819875+0000","last_change":"2024-09-19T08:43:52.764376+0000","last_active":"2024-09-19T08:44:02.819875+0000","last_peered":"2024-09-19T08:44:02.819875+0000","last_clean":"2024-09-19T08:44:02.819875+0000","last_became_active":"2024-09-19T08:43:52.764048+0000","last_became_peered":"2024-09-19T08:43:52.764048+0000","last_unstale":"2024-09-19T08:44:02.819875+0000","last_undegraded":"2024-09-19T08:44:02.819875+0000","last_fullsized":"2024-09-19T08:44:02.819875+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:56.732094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.827925+0000","last_change":"2024-09-19T08:44:02.827925+0000","last_active":"2024-09-19T08:44:02.827925+0000","last_peered":"2024-09-19T08:44:02.827925+0000","last_clean":"2024-09-19T08:44:02.827925+0000","last_became_active":"2024-09-19T08:44:02.827269+0000","last_became_peered":"2024-09-19T08:44:02.827269+0000","last_unstale":"2024-09-19T08:44:02.827925+0000","last_undegraded":"2024-09-19T08:44:02.827925+0000","last_fullsized":"2024-09-19T08:44:02.827925+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:31:54.556786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818875+0000","last_change":"2024-09-19T08:43:52.762190+0000","last_active":"2024-09-19T08:44:02.818875+0000","last_peered":"2024-09-19T08:44:02.818875+0000","last_clean":"2024-09-19T08:44:02.818875+0000","last_became_active":"2024-09-19T08:43:52.761818+0000","last_became_peered":"2024-09-19T08:43:52.761818+0000","last_unstale":"2024-09-19T08:44:02.818875+0000","last_undegraded":"2024-09-19T08:44:02.818875+0000","last_fullsized":"2024-09-19T08:44:02.818875+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:19:55.246779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.820265+0000","last_change":"2024-09-19T08:43:52.761069+0000","last_active":"2024-09-19T08:44:02.820265+0000","last_peered":"2024-09-19T08:44:02.820265+0000","last_clean":"2024-09-19T08:44:02.820265+0000","last_became_active":"2024-09-19T08:43:52.760705+0000","last_became_peered":"2024-09-19T08:43:52.760705+0000","last_unstale":"2024-09-19T08:44:02.820265+0000","last_undegraded":"2024-09-19T08:44:02.820265+0000","last_fullsized":"2024-09-19T08:44:02.820265+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:46:27.059026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.493512+0000","last_change":"2024-09-19T08:43:52.773449+0000","last_active":"2024-09-19T08:44:03.493512+0000","last_peered":"2024-09-19T08:44:03.493512+0000","last_clean":"2024-09-19T08:44:03.493512+0000","last_became_active":"2024-09-19T08:43:52.772929+0000","last_became_peered":"2024-09-19T08:43:52.772929+0000","last_unstale":"2024-09-19T08:44:03.493512+0000","last_undegraded":"2024-09-19T08:44:03.493512+0000","last_fullsized":"2024-09-19T08:44:03.493512+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:00:38.929199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821652+0000","last_change":"2024-09-19T08:43:52.773885+0000","last_active":"2024-09-19T08:44:02.821652+0000","last_peered":"2024-09-19T08:44:02.821652+0000","last_clean":"2024-09-19T08:44:02.821652+0000","last_became_active":"2024-09-19T08:43:52.772856+0000","last_became_peered":"2024-09-19T08:43:52.772856+0000","last_unstale":"2024-09-19T08:44:02.821652+0000","last_undegraded":"2024-09-19T08:44:02.821652+0000","last_fullsized":"2024-09-19T08:44:02.821652+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:13:22.581335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.493984+0000","last_change":"2024-09-19T08:43:52.762909+0000","last_active":"2024-09-19T08:44:03.493984+0000","last_peered":"2024-09-19T08:44:03.493984+0000","last_clean":"2024-09-19T08:44:03.493984+0000","last_became_active":"2024-09-19T08:43:52.762600+0000","last_became_peered":"2024-09-19T08:43:52.762600+0000","last_unstale":"2024-09-19T08:44:03.493984+0000","last_undegraded":"2024-09-19T08:44:03.493984+0000","last_fullsized":"2024-09-19T08:44:03.493984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:46.098294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.820300+0000","last_change":"2024-09-19T08:43:52.762861+0000","last_active":"2024-09-19T08:44:02.820300+0000","last_peered":"2024-09-19T08:44:02.820300+0000","last_clean":"2024-09-19T08:44:02.820300+0000","last_became_active":"2024-09-19T08:43:52.762525+0000","last_became_peered":"2024-09-19T08:43:52.762525+0000","last_unstale":"2024-09-19T08:44:02.820300+0000","last_undegraded":"2024-09-19T08:44:02.820300+0000","last_fullsized":"2024-09-19T08:44:02.820300+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:54.717249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.816836+0000","last_change":"2024-09-19T08:43:52.768370+0000","last_active":"2024-09-19T08:44:02.816836+0000","last_peered":"2024-09-19T08:44:02.816836+0000","last_clean":"2024-09-19T08:44:02.816836+0000","last_became_active":"2024-09-19T08:43:52.768055+0000","last_became_peered":"2024-09-19T08:43:52.768055+0000","last_unstale":"2024-09-19T08:44:02.816836+0000","last_undegraded":"2024-09-19T08:44:02.816836+0000","last_fullsized":"2024-09-19T08:44:02.816836+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:48.212862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819012+0000","last_change":"2024-09-19T08:43:52.758541+0000","last_active":"2024-09-19T08:44:02.819012+0000","last_peered":"2024-09-19T08:44:02.819012+0000","last_clean":"2024-09-19T08:44:02.819012+0000","last_became_active":"2024-09-19T08:43:52.758122+0000","last_became_peered":"2024-09-19T08:43:52.758122+0000","last_unstale":"2024-09-19T08:44:02.819012+0000","last_undegraded":"2024-09-19T08:44:02.819012+0000","last_fullsized":"2024-09-19T08:44:02.819012+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:54:40.681880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818709+0000","last_change":"2024-09-19T08:43:52.758109+0000","last_active":"2024-09-19T08:44:02.818709+0000","last_peered":"2024-09-19T08:44:02.818709+0000","last_clean":"2024-09-19T08:44:02.818709+0000","last_became_active":"2024-09-19T08:43:52.757808+0000","last_became_peered":"2024-09-19T08:43:52.757808+0000","last_unstale":"2024-09-19T08:44:02.818709+0000","last_undegraded":"2024-09-19T08:44:02.818709+0000","last_fullsized":"2024-09-19T08:44:02.818709+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:02:47.614562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819377+0000","last_change":"2024-09-19T08:43:52.772501+0000","last_active":"2024-09-19T08:44:02.819377+0000","last_peered":"2024-09-19T08:44:02.819377+0000","last_clean":"2024-09-19T08:44:02.819377+0000","last_became_active":"2024-09-19T08:43:52.771611+0000","last_became_peered":"2024-09-19T08:43:52.771611+0000","last_unstale":"2024-09-19T08:44:02.819377+0000","last_undegraded":"2024-09-19T08:44:02.819377+0000","last_fullsized":"2024-09-19T08:44:02.819377+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:49.537237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819339+0000","last_change":"2024-09-19T08:43:52.773664+0000","last_active":"2024-09-19T08:44:02.819339+0000","last_peered":"2024-09-19T08:44:02.819339+0000","last_clean":"2024-09-19T08:44:02.819339+0000","last_became_active":"2024-09-19T08:43:52.773014+0000","last_became_peered":"2024-09-19T08:43:52.773014+0000","last_unstale":"2024-09-19T08:44:02.819339+0000","last_undegraded":"2024-09-19T08:44:02.819339+0000","last_fullsized":"2024-09-19T08:44:02.819339+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:03.448825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.816727+0000","last_change":"2024-09-19T08:43:52.769578+0000","last_active":"2024-09-19T08:44:02.816727+0000","last_peered":"2024-09-19T08:44:02.816727+0000","last_clean":"2024-09-19T08:44:02.816727+0000","last_became_active":"2024-09-19T08:43:52.769252+0000","last_became_peered":"2024-09-19T08:43:52.769252+0000","last_unstale":"2024-09-19T08:44:02.816727+0000","last_undegraded":"2024-09-19T08:44:02.816727+0000","last_fullsized":"2024-09-19T08:44:02.816727+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:27.936118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818612+0000","last_change":"2024-09-19T08:43:52.770453+0000","last_active":"2024-09-19T08:44:02.818612+0000","last_peered":"2024-09-19T08:44:02.818612+0000","last_clean":"2024-09-19T08:44:02.818612+0000","last_became_active":"2024-09-19T08:43:52.770155+0000","last_became_peered":"2024-09-19T08:43:52.770155+0000","last_unstale":"2024-09-19T08:44:02.818612+0000","last_undegraded":"2024-09-19T08:44:02.818612+0000","last_fullsized":"2024-09-19T08:44:02.818612+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:12.893572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:13.645840+0000","last_change":"2024-09-19T08:43:50.751316+0000","last_active":"2024-09-19T08:44:13.645840+0000","last_peered":"2024-09-19T08:44:13.645840+0000","last_clean":"2024-09-19T08:44:13.645840+0000","last_became_active":"2024-09-19T08:43:50.750972+0000","last_became_peered":"2024-09-19T08:43:50.750972+0000","last_unstale":"2024-09-19T08:44:13.645840+0000","last_undegraded":"2024-09-19T08:44:13.645840+0000","last_fullsized":"2024-09-19T08:44:13.645840+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:40.654729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.822390+0000","last_change":"2024-09-19T08:43:52.773979+0000","last_active":"2024-09-19T08:44:02.822390+0000","last_peered":"2024-09-19T08:44:02.822390+0000","last_clean":"2024-09-19T08:44:02.822390+0000","last_became_active":"2024-09-19T08:43:52.773648+0000","last_became_peered":"2024-09-19T08:43:52.773648+0000","last_unstale":"2024-09-19T08:44:02.822390+0000","last_undegraded":"2024-09-19T08:44:02.822390+0000","last_fullsized":"2024-09-19T08:44:02.822390+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:36.061209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.839445+0000","last_change":"2024-09-19T08:43:50.750704+0000","last_active":"2024-09-19T08:44:03.839445+0000","last_peered":"2024-09-19T08:44:03.839445+0000","last_clean":"2024-09-19T08:44:03.839445+0000","last_became_active":"2024-09-19T08:43:50.750399+0000","last_became_peered":"2024-09-19T08:43:50.750399+0000","last_unstale":"2024-09-19T08:44:03.839445+0000","last_undegraded":"2024-09-19T08:44:03.839445+0000","last_fullsized":"2024-09-19T08:44:03.839445+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:25.447320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818514+0000","last_change":"2024-09-19T08:43:50.744919+0000","last_active":"2024-09-19T08:44:02.818514+0000","last_peered":"2024-09-19T08:44:02.818514+0000","last_clean":"2024-09-19T08:44:02.818514+0000","last_became_active":"2024-09-19T08:43:50.744604+0000","last_became_peered":"2024-09-19T08:43:50.744604+0000","last_unstale":"2024-09-19T08:44:02.818514+0000","last_undegraded":"2024-09-19T08:44:02.818514+0000","last_fullsized":"2024-09-19T08:44:02.818514+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:25:06.410093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.493518+0000","last_change":"2024-09-19T08:43:52.766779+0000","last_active":"2024-09-19T08:44:03.493518+0000","last_peered":"2024-09-19T08:44:03.493518+0000","last_clean":"2024-09-19T08:44:03.493518+0000","last_became_active":"2024-09-19T08:43:52.766484+0000","last_became_peered":"2024-09-19T08:43:52.766484+0000","last_unstale":"2024-09-19T08:44:03.493518+0000","last_undegraded":"2024-09-19T08:44:03.493518+0000","last_fullsized":"2024-09-19T08:44:03.493518+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:17:29.350778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.631853+0000","last_change":"2024-09-19T08:43:50.751282+0000","last_active":"2024-09-19T08:44:04.631853+0000","last_peered":"2024-09-19T08:44:04.631853+0000","last_clean":"2024-09-19T08:44:04.631853+0000","last_became_active":"2024-09-19T08:43:50.750754+0000","last_became_peered":"2024-09-19T08:43:50.750754+0000","last_unstale":"2024-09-19T08:44:04.631853+0000","last_undegraded":"2024-09-19T08:44:04.631853+0000","last_fullsized":"2024-09-19T08:44:04.631853+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:11.661626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821354+0000","last_change":"2024-09-19T08:43:52.756442+0000","last_active":"2024-09-19T08:44:02.821354+0000","last_peered":"2024-09-19T08:44:02.821354+0000","last_clean":"2024-09-19T08:44:02.821354+0000","last_became_active":"2024-09-19T08:43:52.756116+0000","last_became_peered":"2024-09-19T08:43:52.756116+0000","last_unstale":"2024-09-19T08:44:02.821354+0000","last_undegraded":"2024-09-19T08:44:02.821354+0000","last_fullsized":"2024-09-19T08:44:02.821354+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:40.029272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:15.647597+0000","last_change":"2024-09-19T08:43:50.749637+0000","last_active":"2024-09-19T08:44:15.647597+0000","last_peered":"2024-09-19T08:44:15.647597+0000","last_clean":"2024-09-19T08:44:15.647597+0000","last_became_active":"2024-09-19T08:43:50.749307+0000","last_became_peered":"2024-09-19T08:43:50.749307+0000","last_unstale":"2024-09-19T08:44:15.647597+0000","last_undegraded":"2024-09-19T08:44:15.647597+0000","last_fullsized":"2024-09-19T08:44:15.647597+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:23:33.409671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.820558+0000","last_change":"2024-09-19T08:43:52.763592+0000","last_active":"2024-09-19T08:44:02.820558+0000","last_peered":"2024-09-19T08:44:02.820558+0000","last_clean":"2024-09-19T08:44:02.820558+0000","last_became_active":"2024-09-19T08:43:52.763146+0000","last_became_peered":"2024-09-19T08:43:52.763146+0000","last_unstale":"2024-09-19T08:44:02.820558+0000","last_undegraded":"2024-09-19T08:44:02.820558+0000","last_fullsized":"2024-09-19T08:44:02.820558+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:32.194465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.638566+0000","last_change":"2024-09-19T08:43:50.749184+0000","last_active":"2024-09-19T08:44:04.638566+0000","last_peered":"2024-09-19T08:44:04.638566+0000","last_clean":"2024-09-19T08:44:04.638566+0000","last_became_active":"2024-09-19T08:43:50.748844+0000","last_became_peered":"2024-09-19T08:43:50.748844+0000","last_unstale":"2024-09-19T08:44:04.638566+0000","last_undegraded":"2024-09-19T08:44:04.638566+0000","last_fullsized":"2024-09-19T08:44:04.638566+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:15.334759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819256+0000","last_change":"2024-09-19T08:43:52.768696+0000","last_active":"2024-09-19T08:44:02.819256+0000","last_peered":"2024-09-19T08:44:02.819256+0000","last_clean":"2024-09-19T08:44:02.819256+0000","last_became_active":"2024-09-19T08:43:52.768374+0000","last_became_peered":"2024-09-19T08:43:52.768374+0000","last_unstale":"2024-09-19T08:44:02.819256+0000","last_undegraded":"2024-09-19T08:44:02.819256+0000","last_fullsized":"2024-09-19T08:44:02.819256+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:34:39.343041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.838113+0000","last_change":"2024-09-19T08:43:51.026270+0000","last_active":"2024-09-19T08:44:03.838113+0000","last_peered":"2024-09-19T08:44:03.838113+0000","last_clean":"2024-09-19T08:44:03.838113+0000","last_became_active":"2024-09-19T08:43:51.025823+0000","last_became_peered":"2024-09-19T08:43:51.025823+0000","last_unstale":"2024-09-19T08:44:03.838113+0000","last_undegraded":"2024-09-19T08:44:03.838113+0000","last_fullsized":"2024-09-19T08:44:03.838113+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:00.480438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.820391+0000","last_change":"2024-09-19T08:43:52.759555+0000","last_active":"2024-09-19T08:44:02.820391+0000","last_peered":"2024-09-19T08:44:02.820391+0000","last_clean":"2024-09-19T08:44:02.820391+0000","last_became_active":"2024-09-19T08:43:52.759223+0000","last_became_peered":"2024-09-19T08:43:52.759223+0000","last_unstale":"2024-09-19T08:44:02.820391+0000","last_undegraded":"2024-09-19T08:44:02.820391+0000","last_fullsized":"2024-09-19T08:44:02.820391+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:57:06.451323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'24","reported_seq":76,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:15.648388+0000","last_change":"2024-09-19T08:43:50.751924+0000","last_active":"2024-09-19T08:44:15.648388+0000","last_peered":"2024-09-19T08:44:15.648388+0000","last_clean":"2024-09-19T08:44:15.648388+0000","last_became_active":"2024-09-19T08:43:50.751117+0000","last_became_peered":"2024-09-19T08:43:50.751117+0000","last_unstale":"2024-09-19T08:44:15.648388+0000","last_undegraded":"2024-09-19T08:44:15.648388+0000","last_fullsized":"2024-09-19T08:44:15.648388+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:33:57.196283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46802,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":30,"num_read_kb":60,"num_write":26,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818871+0000","last_change":"2024-09-19T08:43:52.762260+0000","last_active":"2024-09-19T08:44:02.818871+0000","last_peered":"2024-09-19T08:44:02.818871+0000","last_clean":"2024-09-19T08:44:02.818871+0000","last_became_active":"2024-09-19T08:43:52.761923+0000","last_became_peered":"2024-09-19T08:43:52.761923+0000","last_unstale":"2024-09-19T08:44:02.818871+0000","last_undegraded":"2024-09-19T08:44:02.818871+0000","last_fullsized":"2024-09-19T08:44:02.818871+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:39:39.503332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.638455+0000","last_change":"2024-09-19T08:43:50.749745+0000","last_active":"2024-09-19T08:44:04.638455+0000","last_peered":"2024-09-19T08:44:04.638455+0000","last_clean":"2024-09-19T08:44:04.638455+0000","last_became_active":"2024-09-19T08:43:50.749073+0000","last_became_peered":"2024-09-19T08:43:50.749073+0000","last_unstale":"2024-09-19T08:44:04.638455+0000","last_undegraded":"2024-09-19T08:44:04.638455+0000","last_fullsized":"2024-09-19T08:44:04.638455+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:48:11.965841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818633+0000","last_change":"2024-09-19T08:43:52.761495+0000","last_active":"2024-09-19T08:44:02.818633+0000","last_peered":"2024-09-19T08:44:02.818633+0000","last_clean":"2024-09-19T08:44:02.818633+0000","last_became_active":"2024-09-19T08:43:52.760425+0000","last_became_peered":"2024-09-19T08:43:52.760425+0000","last_unstale":"2024-09-19T08:44:02.818633+0000","last_undegraded":"2024-09-19T08:44:02.818633+0000","last_fullsized":"2024-09-19T08:44:02.818633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:57.689284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.639600+0000","last_change":"2024-09-19T08:43:50.749563+0000","last_active":"2024-09-19T08:44:04.639600+0000","last_peered":"2024-09-19T08:44:04.639600+0000","last_clean":"2024-09-19T08:44:04.639600+0000","last_became_active":"2024-09-19T08:43:50.749244+0000","last_became_peered":"2024-09-19T08:43:50.749244+0000","last_unstale":"2024-09-19T08:44:04.639600+0000","last_undegraded":"2024-09-19T08:44:04.639600+0000","last_fullsized":"2024-09-19T08:44:04.639600+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:59:02.312883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821915+0000","last_change":"2024-09-19T08:43:52.767087+0000","last_active":"2024-09-19T08:44:02.821915+0000","last_peered":"2024-09-19T08:44:02.821915+0000","last_clean":"2024-09-19T08:44:02.821915+0000","last_became_active":"2024-09-19T08:43:52.766730+0000","last_became_peered":"2024-09-19T08:43:52.766730+0000","last_unstale":"2024-09-19T08:44:02.821915+0000","last_undegraded":"2024-09-19T08:44:02.821915+0000","last_fullsized":"2024-09-19T08:44:02.821915+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:04.040896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:13.649953+0000","last_change":"2024-09-19T08:43:50.749659+0000","last_active":"2024-09-19T08:44:13.649953+0000","last_peered":"2024-09-19T08:44:13.649953+0000","last_clean":"2024-09-19T08:44:13.649953+0000","last_became_active":"2024-09-19T08:43:50.749314+0000","last_became_peered":"2024-09-19T08:43:50.749314+0000","last_unstale":"2024-09-19T08:44:13.649953+0000","last_undegraded":"2024-09-19T08:44:13.649953+0000","last_fullsized":"2024-09-19T08:44:13.649953+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:08.576209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821732+0000","last_change":"2024-09-19T08:43:52.767928+0000","last_active":"2024-09-19T08:44:02.821732+0000","last_peered":"2024-09-19T08:44:02.821732+0000","last_clean":"2024-09-19T08:44:02.821732+0000","last_became_active":"2024-09-19T08:43:52.767580+0000","last_became_peered":"2024-09-19T08:43:52.767580+0000","last_unstale":"2024-09-19T08:44:02.821732+0000","last_undegraded":"2024-09-19T08:44:02.821732+0000","last_fullsized":"2024-09-19T08:44:02.821732+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:45.344693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.630057+0000","last_change":"2024-09-19T08:44:02.833887+0000","last_active":"2024-09-19T08:44:04.630057+0000","last_peered":"2024-09-19T08:44:04.630057+0000","last_clean":"2024-09-19T08:44:04.630057+0000","last_became_active":"2024-09-19T08:44:02.821624+0000","last_became_peered":"2024-09-19T08:44:02.821624+0000","last_unstale":"2024-09-19T08:44:04.630057+0000","last_undegraded":"2024-09-19T08:44:04.630057+0000","last_fullsized":"2024-09-19T08:44:04.630057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:54:36.131949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.080821+0000","last_change":"2024-09-19T08:44:03.080503+0000","last_active":"2024-09-19T08:44:03.080821+0000","last_peered":"2024-09-19T08:44:03.080821+0000","last_clean":"2024-09-19T08:44:03.080821+0000","last_became_active":"2024-09-19T08:44:03.080129+0000","last_became_peered":"2024-09-19T08:44:03.080129+0000","last_unstale":"2024-09-19T08:44:03.080821+0000","last_undegraded":"2024-09-19T08:44:03.080821+0000","last_fullsized":"2024-09-19T08:44:03.080821+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:01:37.034795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.629282+0000","last_change":"2024-09-19T08:43:50.747624+0000","last_active":"2024-09-19T08:44:04.629282+0000","last_peered":"2024-09-19T08:44:04.629282+0000","last_clean":"2024-09-19T08:44:04.629282+0000","last_became_active":"2024-09-19T08:43:50.747263+0000","last_became_peered":"2024-09-19T08:43:50.747263+0000","last_unstale":"2024-09-19T08:44:04.629282+0000","last_undegraded":"2024-09-19T08:44:04.629282+0000","last_fullsized":"2024-09-19T08:44:04.629282+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:18.580106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.816903+0000","last_change":"2024-09-19T08:43:52.770051+0000","last_active":"2024-09-19T08:44:02.816903+0000","last_peered":"2024-09-19T08:44:02.816903+0000","last_clean":"2024-09-19T08:44:02.816903+0000","last_became_active":"2024-09-19T08:43:52.769727+0000","last_became_peered":"2024-09-19T08:43:52.769727+0000","last_unstale":"2024-09-19T08:44:02.816903+0000","last_undegraded":"2024-09-19T08:44:02.816903+0000","last_fullsized":"2024-09-19T08:44:02.816903+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:34.954897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.638777+0000","last_change":"2024-09-19T08:43:50.746696+0000","last_active":"2024-09-19T08:44:04.638777+0000","last_peered":"2024-09-19T08:44:04.638777+0000","last_clean":"2024-09-19T08:44:04.638777+0000","last_became_active":"2024-09-19T08:43:50.746246+0000","last_became_peered":"2024-09-19T08:43:50.746246+0000","last_unstale":"2024-09-19T08:44:04.638777+0000","last_undegraded":"2024-09-19T08:44:04.638777+0000","last_fullsized":"2024-09-19T08:44:04.638777+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:38:39.935348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.822242+0000","last_change":"2024-09-19T08:43:52.765199+0000","last_active":"2024-09-19T08:44:02.822242+0000","last_peered":"2024-09-19T08:44:02.822242+0000","last_clean":"2024-09-19T08:44:02.822242+0000","last_became_active":"2024-09-19T08:43:52.764896+0000","last_became_peered":"2024-09-19T08:43:52.764896+0000","last_unstale":"2024-09-19T08:44:02.822242+0000","last_undegraded":"2024-09-19T08:44:02.822242+0000","last_fullsized":"2024-09-19T08:44:02.822242+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:02:59.570018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.628862+0000","last_change":"2024-09-19T08:43:50.747242+0000","last_active":"2024-09-19T08:44:04.628862+0000","last_peered":"2024-09-19T08:44:04.628862+0000","last_clean":"2024-09-19T08:44:04.628862+0000","last_became_active":"2024-09-19T08:43:50.746853+0000","last_became_peered":"2024-09-19T08:43:50.746853+0000","last_unstale":"2024-09-19T08:44:04.628862+0000","last_undegraded":"2024-09-19T08:44:04.628862+0000","last_fullsized":"2024-09-19T08:44:04.628862+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:21.545635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.827031+0000","last_change":"2024-09-19T08:44:02.826535+0000","last_active":"2024-09-19T08:44:02.827031+0000","last_peered":"2024-09-19T08:44:02.827031+0000","last_clean":"2024-09-19T08:44:02.827031+0000","last_became_active":"2024-09-19T08:44:02.826113+0000","last_became_peered":"2024-09-19T08:44:02.826113+0000","last_unstale":"2024-09-19T08:44:02.827031+0000","last_undegraded":"2024-09-19T08:44:02.827031+0000","last_fullsized":"2024-09-19T08:44:02.827031+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:20.247732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'6","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:17.651305+0000","last_change":"2024-09-19T08:43:51.131540+0000","last_active":"2024-09-19T08:44:17.651305+0000","last_peered":"2024-09-19T08:44:17.651305+0000","last_clean":"2024-09-19T08:44:17.651305+0000","last_became_active":"2024-09-19T08:43:51.131170+0000","last_became_peered":"2024-09-19T08:43:51.131170+0000","last_unstale":"2024-09-19T08:44:17.651305+0000","last_undegraded":"2024-09-19T08:44:17.651305+0000","last_fullsized":"2024-09-19T08:44:17.651305+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:43.635612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.822208+0000","last_change":"2024-09-19T08:43:52.763308+0000","last_active":"2024-09-19T08:44:02.822208+0000","last_peered":"2024-09-19T08:44:02.822208+0000","last_clean":"2024-09-19T08:44:02.822208+0000","last_became_active":"2024-09-19T08:43:52.762859+0000","last_became_peered":"2024-09-19T08:43:52.762859+0000","last_unstale":"2024-09-19T08:44:02.822208+0000","last_undegraded":"2024-09-19T08:44:02.822208+0000","last_fullsized":"2024-09-19T08:44:02.822208+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:32:35.083361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":347,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821905+0000","last_change":"2024-09-19T08:42:39.567408+0000","last_active":"2024-09-19T08:44:02.821905+0000","last_peered":"2024-09-19T08:44:02.821905+0000","last_clean":"2024-09-19T08:44:02.821905+0000","last_became_active":"2024-09-19T08:42:39.566967+0000","last_became_peered":"2024-09-19T08:42:39.566967+0000","last_unstale":"2024-09-19T08:44:02.821905+0000","last_undegraded":"2024-09-19T08:44:02.821905+0000","last_fullsized":"2024-09-19T08:44:02.821905+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:14.287944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:14.287944+0000","last_clean_scrub_stamp":"2024-09-19T08:39:14.287944+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:31:03.245386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.638441+0000","last_change":"2024-09-19T08:43:50.751097+0000","last_active":"2024-09-19T08:44:04.638441+0000","last_peered":"2024-09-19T08:44:04.638441+0000","last_clean":"2024-09-19T08:44:04.638441+0000","last_became_active":"2024-09-19T08:43:50.750341+0000","last_became_peered":"2024-09-19T08:43:50.750341+0000","last_unstale":"2024-09-19T08:44:04.638441+0000","last_undegraded":"2024-09-19T08:44:04.638441+0000","last_fullsized":"2024-09-19T08:44:04.638441+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:25.694535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.493670+0000","last_change":"2024-09-19T08:43:52.767905+0000","last_active":"2024-09-19T08:44:03.493670+0000","last_peered":"2024-09-19T08:44:03.493670+0000","last_clean":"2024-09-19T08:44:03.493670+0000","last_became_active":"2024-09-19T08:43:52.767601+0000","last_became_peered":"2024-09-19T08:43:52.767601+0000","last_unstale":"2024-09-19T08:44:03.493670+0000","last_undegraded":"2024-09-19T08:44:03.493670+0000","last_fullsized":"2024-09-19T08:44:03.493670+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:14.519879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.822161+0000","last_change":"2024-09-19T08:43:50.751397+0000","last_active":"2024-09-19T08:44:02.822161+0000","last_peered":"2024-09-19T08:44:02.822161+0000","last_clean":"2024-09-19T08:44:02.822161+0000","last_became_active":"2024-09-19T08:43:50.751079+0000","last_became_peered":"2024-09-19T08:43:50.751079+0000","last_unstale":"2024-09-19T08:44:02.822161+0000","last_undegraded":"2024-09-19T08:44:02.822161+0000","last_fullsized":"2024-09-19T08:44:02.822161+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:29.904786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818371+0000","last_change":"2024-09-19T08:43:52.773348+0000","last_active":"2024-09-19T08:44:02.818371+0000","last_peered":"2024-09-19T08:44:02.818371+0000","last_clean":"2024-09-19T08:44:02.818371+0000","last_became_active":"2024-09-19T08:43:52.773011+0000","last_became_peered":"2024-09-19T08:43:52.773011+0000","last_unstale":"2024-09-19T08:44:02.818371+0000","last_undegraded":"2024-09-19T08:44:02.818371+0000","last_fullsized":"2024-09-19T08:44:02.818371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:10.806578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:14.650708+0000","last_change":"2024-09-19T08:43:51.025300+0000","last_active":"2024-09-19T08:44:14.650708+0000","last_peered":"2024-09-19T08:44:14.650708+0000","last_clean":"2024-09-19T08:44:14.650708+0000","last_became_active":"2024-09-19T08:43:51.024936+0000","last_became_peered":"2024-09-19T08:43:51.024936+0000","last_unstale":"2024-09-19T08:44:14.650708+0000","last_undegraded":"2024-09-19T08:44:14.650708+0000","last_fullsized":"2024-09-19T08:44:14.650708+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:03.186066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821758+0000","last_change":"2024-09-19T08:43:52.774962+0000","last_active":"2024-09-19T08:44:02.821758+0000","last_peered":"2024-09-19T08:44:02.821758+0000","last_clean":"2024-09-19T08:44:02.821758+0000","last_became_active":"2024-09-19T08:43:52.774598+0000","last_became_peered":"2024-09-19T08:43:52.774598+0000","last_unstale":"2024-09-19T08:44:02.821758+0000","last_undegraded":"2024-09-19T08:44:02.821758+0000","last_fullsized":"2024-09-19T08:44:02.821758+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:02.687136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'5","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:13.650866+0000","last_change":"2024-09-19T08:43:50.748180+0000","last_active":"2024-09-19T08:44:13.650866+0000","last_peered":"2024-09-19T08:44:13.650866+0000","last_clean":"2024-09-19T08:44:13.650866+0000","last_became_active":"2024-09-19T08:43:50.747853+0000","last_became_peered":"2024-09-19T08:43:50.747853+0000","last_unstale":"2024-09-19T08:44:13.650866+0000","last_undegraded":"2024-09-19T08:44:13.650866+0000","last_fullsized":"2024-09-19T08:44:13.650866+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:56.047259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.817252+0000","last_change":"2024-09-19T08:43:52.770253+0000","last_active":"2024-09-19T08:44:02.817252+0000","last_peered":"2024-09-19T08:44:02.817252+0000","last_clean":"2024-09-19T08:44:02.817252+0000","last_became_active":"2024-09-19T08:43:52.769951+0000","last_became_peered":"2024-09-19T08:43:52.769951+0000","last_unstale":"2024-09-19T08:44:02.817252+0000","last_undegraded":"2024-09-19T08:44:02.817252+0000","last_fullsized":"2024-09-19T08:44:02.817252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:02.353144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821619+0000","last_change":"2024-09-19T08:43:51.026453+0000","last_active":"2024-09-19T08:44:02.821619+0000","last_peered":"2024-09-19T08:44:02.821619+0000","last_clean":"2024-09-19T08:44:02.821619+0000","last_became_active":"2024-09-19T08:43:51.025873+0000","last_became_peered":"2024-09-19T08:43:51.025873+0000","last_unstale":"2024-09-19T08:44:02.821619+0000","last_undegraded":"2024-09-19T08:44:02.821619+0000","last_fullsized":"2024-09-19T08:44:02.821619+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:25:17.475074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.817186+0000","last_change":"2024-09-19T08:43:52.761700+0000","last_active":"2024-09-19T08:44:02.817186+0000","last_peered":"2024-09-19T08:44:02.817186+0000","last_clean":"2024-09-19T08:44:02.817186+0000","last_became_active":"2024-09-19T08:43:52.755527+0000","last_became_peered":"2024-09-19T08:43:52.755527+0000","last_unstale":"2024-09-19T08:44:02.817186+0000","last_undegraded":"2024-09-19T08:44:02.817186+0000","last_fullsized":"2024-09-19T08:44:02.817186+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:18:43.171284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.630369+0000","last_change":"2024-09-19T08:43:51.026136+0000","last_active":"2024-09-19T08:44:04.630369+0000","last_peered":"2024-09-19T08:44:04.630369+0000","last_clean":"2024-09-19T08:44:04.630369+0000","last_became_active":"2024-09-19T08:43:51.025740+0000","last_became_peered":"2024-09-19T08:43:51.025740+0000","last_unstale":"2024-09-19T08:44:04.630369+0000","last_undegraded":"2024-09-19T08:44:04.630369+0000","last_fullsized":"2024-09-19T08:44:04.630369+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:17:32.741083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819140+0000","last_change":"2024-09-19T08:43:52.763808+0000","last_active":"2024-09-19T08:44:02.819140+0000","last_peered":"2024-09-19T08:44:02.819140+0000","last_clean":"2024-09-19T08:44:02.819140+0000","last_became_active":"2024-09-19T08:43:52.763509+0000","last_became_peered":"2024-09-19T08:43:52.763509+0000","last_unstale":"2024-09-19T08:44:02.819140+0000","last_undegraded":"2024-09-19T08:44:02.819140+0000","last_fullsized":"2024-09-19T08:44:02.819140+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:45:08.385419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'13","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:13.771746+0000","last_change":"2024-09-19T08:43:50.744213+0000","last_active":"2024-09-19T08:44:13.771746+0000","last_peered":"2024-09-19T08:44:13.771746+0000","last_clean":"2024-09-19T08:44:13.771746+0000","last_became_active":"2024-09-19T08:43:50.743848+0000","last_became_peered":"2024-09-19T08:43:50.743848+0000","last_unstale":"2024-09-19T08:44:13.771746+0000","last_undegraded":"2024-09-19T08:44:13.771746+0000","last_fullsized":"2024-09-19T08:44:13.771746+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:26.613055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.494010+0000","last_change":"2024-09-19T08:43:52.767102+0000","last_active":"2024-09-19T08:44:03.494010+0000","last_peered":"2024-09-19T08:44:03.494010+0000","last_clean":"2024-09-19T08:44:03.494010+0000","last_became_active":"2024-09-19T08:43:52.766262+0000","last_became_peered":"2024-09-19T08:43:52.766262+0000","last_unstale":"2024-09-19T08:44:03.494010+0000","last_undegraded":"2024-09-19T08:44:03.494010+0000","last_fullsized":"2024-09-19T08:44:03.494010+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:35.740462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.638198+0000","last_change":"2024-09-19T08:43:51.131742+0000","last_active":"2024-09-19T08:44:04.638198+0000","last_peered":"2024-09-19T08:44:04.638198+0000","last_clean":"2024-09-19T08:44:04.638198+0000","last_became_active":"2024-09-19T08:43:51.131388+0000","last_became_peered":"2024-09-19T08:43:51.131388+0000","last_unstale":"2024-09-19T08:44:04.638198+0000","last_undegraded":"2024-09-19T08:44:04.638198+0000","last_fullsized":"2024-09-19T08:44:04.638198+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:26.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819357+0000","last_change":"2024-09-19T08:43:52.756004+0000","last_active":"2024-09-19T08:44:02.819357+0000","last_peered":"2024-09-19T08:44:02.819357+0000","last_clean":"2024-09-19T08:44:02.819357+0000","last_became_active":"2024-09-19T08:43:52.755596+0000","last_became_peered":"2024-09-19T08:43:52.755596+0000","last_unstale":"2024-09-19T08:44:02.819357+0000","last_undegraded":"2024-09-19T08:44:02.819357+0000","last_fullsized":"2024-09-19T08:44:02.819357+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:35.942375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.835986+0000","last_change":"2024-09-19T08:44:02.835986+0000","last_active":"2024-09-19T08:44:02.835986+0000","last_peered":"2024-09-19T08:44:02.835986+0000","last_clean":"2024-09-19T08:44:02.835986+0000","last_became_active":"2024-09-19T08:44:02.826314+0000","last_became_peered":"2024-09-19T08:44:02.826314+0000","last_unstale":"2024-09-19T08:44:02.835986+0000","last_undegraded":"2024-09-19T08:44:02.835986+0000","last_fullsized":"2024-09-19T08:44:02.835986+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:16.943989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818539+0000","last_change":"2024-09-19T08:43:52.766582+0000","last_active":"2024-09-19T08:44:02.818539+0000","last_peered":"2024-09-19T08:44:02.818539+0000","last_clean":"2024-09-19T08:44:02.818539+0000","last_became_active":"2024-09-19T08:43:52.766232+0000","last_became_peered":"2024-09-19T08:43:52.766232+0000","last_unstale":"2024-09-19T08:44:02.818539+0000","last_undegraded":"2024-09-19T08:44:02.818539+0000","last_fullsized":"2024-09-19T08:44:02.818539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:43:42.319422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.635810+0000","last_change":"2024-09-19T08:43:50.747611+0000","last_active":"2024-09-19T08:44:04.635810+0000","last_peered":"2024-09-19T08:44:04.635810+0000","last_clean":"2024-09-19T08:44:04.635810+0000","last_became_active":"2024-09-19T08:43:50.747263+0000","last_became_peered":"2024-09-19T08:43:50.747263+0000","last_unstale":"2024-09-19T08:44:04.635810+0000","last_undegraded":"2024-09-19T08:44:04.635810+0000","last_fullsized":"2024-09-19T08:44:04.635810+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:14.717820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.816851+0000","last_change":"2024-09-19T08:43:52.766493+0000","last_active":"2024-09-19T08:44:02.816851+0000","last_peered":"2024-09-19T08:44:02.816851+0000","last_clean":"2024-09-19T08:44:02.816851+0000","last_became_active":"2024-09-19T08:43:52.766158+0000","last_became_peered":"2024-09-19T08:43:52.766158+0000","last_unstale":"2024-09-19T08:44:02.816851+0000","last_undegraded":"2024-09-19T08:44:02.816851+0000","last_fullsized":"2024-09-19T08:44:02.816851+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:59.175949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.838733+0000","last_change":"2024-09-19T08:43:50.751368+0000","last_active":"2024-09-19T08:44:03.838733+0000","last_peered":"2024-09-19T08:44:03.838733+0000","last_clean":"2024-09-19T08:44:03.838733+0000","last_became_active":"2024-09-19T08:43:50.751038+0000","last_became_peered":"2024-09-19T08:43:50.751038+0000","last_unstale":"2024-09-19T08:44:03.838733+0000","last_undegraded":"2024-09-19T08:44:03.838733+0000","last_fullsized":"2024-09-19T08:44:03.838733+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:46.302996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.493641+0000","last_change":"2024-09-19T08:43:52.756998+0000","last_active":"2024-09-19T08:44:03.493641+0000","last_peered":"2024-09-19T08:44:03.493641+0000","last_clean":"2024-09-19T08:44:03.493641+0000","last_became_active":"2024-09-19T08:43:52.756661+0000","last_became_peered":"2024-09-19T08:43:52.756661+0000","last_unstale":"2024-09-19T08:44:03.493641+0000","last_undegraded":"2024-09-19T08:44:03.493641+0000","last_fullsized":"2024-09-19T08:44:03.493641+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:15.182574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818827+0000","last_change":"2024-09-19T08:43:50.750095+0000","last_active":"2024-09-19T08:44:02.818827+0000","last_peered":"2024-09-19T08:44:02.818827+0000","last_clean":"2024-09-19T08:44:02.818827+0000","last_became_active":"2024-09-19T08:43:50.749672+0000","last_became_peered":"2024-09-19T08:43:50.749672+0000","last_unstale":"2024-09-19T08:44:02.818827+0000","last_undegraded":"2024-09-19T08:44:02.818827+0000","last_fullsized":"2024-09-19T08:44:02.818827+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:25:34.978094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821907+0000","last_change":"2024-09-19T08:43:52.764097+0000","last_active":"2024-09-19T08:44:02.821907+0000","last_peered":"2024-09-19T08:44:02.821907+0000","last_clean":"2024-09-19T08:44:02.821907+0000","last_became_active":"2024-09-19T08:43:52.763773+0000","last_became_peered":"2024-09-19T08:43:52.763773+0000","last_unstale":"2024-09-19T08:44:02.821907+0000","last_undegraded":"2024-09-19T08:44:02.821907+0000","last_fullsized":"2024-09-19T08:44:02.821907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:01:42.098460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.825238+0000","last_change":"2024-09-19T08:44:02.825238+0000","last_active":"2024-09-19T08:44:02.825238+0000","last_peered":"2024-09-19T08:44:02.825238+0000","last_clean":"2024-09-19T08:44:02.825238+0000","last_became_active":"2024-09-19T08:44:02.824873+0000","last_became_peered":"2024-09-19T08:44:02.824873+0000","last_unstale":"2024-09-19T08:44:02.825238+0000","last_undegraded":"2024-09-19T08:44:02.825238+0000","last_fullsized":"2024-09-19T08:44:02.825238+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:38.896082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.639612+0000","last_change":"2024-09-19T08:43:50.748789+0000","last_active":"2024-09-19T08:44:04.639612+0000","last_peered":"2024-09-19T08:44:04.639612+0000","last_clean":"2024-09-19T08:44:04.639612+0000","last_became_active":"2024-09-19T08:43:50.748464+0000","last_became_peered":"2024-09-19T08:43:50.748464+0000","last_unstale":"2024-09-19T08:44:04.639612+0000","last_undegraded":"2024-09-19T08:44:04.639612+0000","last_fullsized":"2024-09-19T08:44:04.639612+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:36:45.905639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.820329+0000","last_change":"2024-09-19T08:43:52.756608+0000","last_active":"2024-09-19T08:44:02.820329+0000","last_peered":"2024-09-19T08:44:02.820329+0000","last_clean":"2024-09-19T08:44:02.820329+0000","last_became_active":"2024-09-19T08:43:52.756227+0000","last_became_peered":"2024-09-19T08:43:52.756227+0000","last_unstale":"2024-09-19T08:44:02.820329+0000","last_undegraded":"2024-09-19T08:44:02.820329+0000","last_fullsized":"2024-09-19T08:44:02.820329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:29:58.085584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'12","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.639102+0000","last_change":"2024-09-19T08:43:50.749135+0000","last_active":"2024-09-19T08:44:04.639102+0000","last_peered":"2024-09-19T08:44:04.639102+0000","last_clean":"2024-09-19T08:44:04.639102+0000","last_became_active":"2024-09-19T08:43:50.748755+0000","last_became_peered":"2024-09-19T08:43:50.748755+0000","last_unstale":"2024-09-19T08:44:04.639102+0000","last_undegraded":"2024-09-19T08:44:04.639102+0000","last_fullsized":"2024-09-19T08:44:04.639102+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:55.079780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819170+0000","last_change":"2024-09-19T08:43:52.763277+0000","last_active":"2024-09-19T08:44:02.819170+0000","last_peered":"2024-09-19T08:44:02.819170+0000","last_clean":"2024-09-19T08:44:02.819170+0000","last_became_active":"2024-09-19T08:43:52.762941+0000","last_became_peered":"2024-09-19T08:43:52.762941+0000","last_unstale":"2024-09-19T08:44:02.819170+0000","last_undegraded":"2024-09-19T08:44:02.819170+0000","last_fullsized":"2024-09-19T08:44:02.819170+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:35.531946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.494005+0000","last_change":"2024-09-19T08:43:51.025350+0000","last_active":"2024-09-19T08:44:03.494005+0000","last_peered":"2024-09-19T08:44:03.494005+0000","last_clean":"2024-09-19T08:44:03.494005+0000","last_became_active":"2024-09-19T08:43:51.025046+0000","last_became_peered":"2024-09-19T08:43:51.025046+0000","last_unstale":"2024-09-19T08:44:03.494005+0000","last_undegraded":"2024-09-19T08:44:03.494005+0000","last_fullsized":"2024-09-19T08:44:03.494005+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:27.595545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.493915+0000","last_change":"2024-09-19T08:43:52.767917+0000","last_active":"2024-09-19T08:44:03.493915+0000","last_peered":"2024-09-19T08:44:03.493915+0000","last_clean":"2024-09-19T08:44:03.493915+0000","last_became_active":"2024-09-19T08:43:52.767575+0000","last_became_peered":"2024-09-19T08:43:52.767575+0000","last_unstale":"2024-09-19T08:44:03.493915+0000","last_undegraded":"2024-09-19T08:44:03.493915+0000","last_fullsized":"2024-09-19T08:44:03.493915+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:05.950399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'13","reported_seq":47,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:14.651442+0000","last_change":"2024-09-19T08:43:50.746400+0000","last_active":"2024-09-19T08:44:14.651442+0000","last_peered":"2024-09-19T08:44:14.651442+0000","last_clean":"2024-09-19T08:44:14.651442+0000","last_became_active":"2024-09-19T08:43:50.745891+0000","last_became_peered":"2024-09-19T08:43:50.745891+0000","last_unstale":"2024-09-19T08:44:14.651442+0000","last_undegraded":"2024-09-19T08:44:14.651442+0000","last_fullsized":"2024-09-19T08:44:14.651442+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:03.098202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":54010,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":156,"num_read_kb":139,"num_write":233,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":178334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":201,"ondisk_log_size":201,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":250201,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579922,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":436653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":1073979,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005083,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":436195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234016,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":436344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462949,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":970752,"data_stored":1028976,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593771,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":435795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822703,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":983040,"data_stored":1028654,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986228,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":905216,"data_stored":435656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215162,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":435503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":983040,"data_stored":492668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":962560,"data_stored":491662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":925696,"data_stored":437527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34807,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34807,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34807,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":56706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":57712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":48538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:44:20.632 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T08:44:20.633 INFO:tasks.cephadm:Setting up client nodes... 2024-09-19T08:44:20.633 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-19T08:44:20.879 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:21.000 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:20 smithi164 ceph-mon[30928]: pgmap v291: 97 pgs: 97 active+clean; 630 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.9 KiB/s wr, 13 op/s 2024-09-19T08:44:21.000 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:20 smithi164 ceph-mon[30928]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:44:21.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:20 smithi170 ceph-mon[35072]: pgmap v291: 97 pgs: 97 active+clean; 630 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.9 KiB/s wr, 13 op/s 2024-09-19T08:44:21.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:20 smithi170 ceph-mon[35072]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:44:21.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:20 smithi173 ceph-mon[35306]: pgmap v291: 97 pgs: 97 active+clean; 630 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.9 KiB/s wr, 13 op/s 2024-09-19T08:44:21.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:20 smithi173 ceph-mon[35306]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:44:21.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.351+0000 7f927e103640 1 -- 172.21.15.164:0/1280682474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9278105030 msgr2=0x7f9278107420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:21.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.351+0000 7f927e103640 1 --2- 172.21.15.164:0/1280682474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9278105030 0x7f9278107420 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f926c009900 tx=0x7f926c02a410 comp rx=0 tx=0).stop 2024-09-19T08:44:21.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.352+0000 7f927e103640 1 -- 172.21.15.164:0/1280682474 shutdown_connections 2024-09-19T08:44:21.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.352+0000 7f927e103640 1 --2- 172.21.15.164:0/1280682474 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9278107960 0x7f9278109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:21.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.352+0000 7f927e103640 1 --2- 172.21.15.164:0/1280682474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9278105030 0x7f9278107420 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:21.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.352+0000 7f927e103640 1 --2- 172.21.15.164:0/1280682474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9278102700 0x7f9278104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:21.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.352+0000 7f927e103640 1 -- 172.21.15.164:0/1280682474 >> 172.21.15.164:0/1280682474 conn(0x7f92780fc2a0 msgr2=0x7f92780fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:21.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.352+0000 7f927e103640 1 -- 172.21.15.164:0/1280682474 shutdown_connections 2024-09-19T08:44:21.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.352+0000 7f927e103640 1 -- 172.21.15.164:0/1280682474 wait complete. 2024-09-19T08:44:21.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.352+0000 7f927e103640 1 Processor -- start 2024-09-19T08:44:21.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.352+0000 7f927e103640 1 -- start start 2024-09-19T08:44:21.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.352+0000 7f927e103640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9278102700 0x7f9278102060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:21.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.352+0000 7f927e103640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9278105030 0x7f92781006b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:21.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.352+0000 7f927e103640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9278107960 0x7f9278100bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:21.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.353+0000 7f927e103640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f927810d890 con 0x7f9278105030 2024-09-19T08:44:21.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.353+0000 7f927e103640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f927810d710 con 0x7f9278107960 2024-09-19T08:44:21.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.353+0000 7f927e103640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f927810da10 con 0x7f9278102700 2024-09-19T08:44:21.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.353+0000 7f9276ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9278105030 0x7f92781006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:21.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.353+0000 7f9276ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9278105030 0x7f92781006b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35714/0 (socket says 172.21.15.164:35714) 2024-09-19T08:44:21.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.353+0000 7f9276ffd640 1 -- 172.21.15.164:0/2031071952 learned_addr learned my addr 172.21.15.164:0/2031071952 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:21.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.353+0000 7f92777fe640 1 --2- 172.21.15.164:0/2031071952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9278102700 0x7f9278102060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:21.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.353+0000 7f9276ffd640 1 -- 172.21.15.164:0/2031071952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9278102700 msgr2=0x7f9278102060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:21.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.353+0000 7f9276ffd640 1 --2- 172.21.15.164:0/2031071952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9278102700 0x7f9278102060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:21.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.353+0000 7f9276ffd640 1 -- 172.21.15.164:0/2031071952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9278107960 msgr2=0x7f9278100bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:21.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.353+0000 7f9276ffd640 1 --2- 172.21.15.164:0/2031071952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9278107960 0x7f9278100bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:21.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.353+0000 7f9276ffd640 1 -- 172.21.15.164:0/2031071952 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92781014b0 con 0x7f9278105030 2024-09-19T08:44:21.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.353+0000 7f9276ffd640 1 --2- 172.21.15.164:0/2031071952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9278105030 0x7f92781006b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f926c031610 tx=0x7f926c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:21.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.353+0000 7f9274ff9640 1 -- 172.21.15.164:0/2031071952 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c03f030 con 0x7f9278105030 2024-09-19T08:44:21.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.353+0000 7f927e103640 1 -- 172.21.15.164:0/2031071952 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92781a2020 con 0x7f9278105030 2024-09-19T08:44:21.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.353+0000 7f9274ff9640 1 -- 172.21.15.164:0/2031071952 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f926c003ce0 con 0x7f9278105030 2024-09-19T08:44:21.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.353+0000 7f9274ff9640 1 -- 172.21.15.164:0/2031071952 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c035070 con 0x7f9278105030 2024-09-19T08:44:21.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.353+0000 7f927e103640 1 -- 172.21.15.164:0/2031071952 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92781a24d0 con 0x7f9278105030 2024-09-19T08:44:21.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.354+0000 7f927e103640 1 -- 172.21.15.164:0/2031071952 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92780fdd90 con 0x7f9278105030 2024-09-19T08:44:21.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.355+0000 7f9274ff9640 1 -- 172.21.15.164:0/2031071952 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f926c042030 con 0x7f9278105030 2024-09-19T08:44:21.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.356+0000 7f9274ff9640 1 --2- 172.21.15.164:0/2031071952 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f92600a32d0 0x7f92600a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:21.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.356+0000 7f92777fe640 1 --2- 172.21.15.164:0/2031071952 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f92600a32d0 0x7f92600a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:21.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.356+0000 7f9274ff9640 1 -- 172.21.15.164:0/2031071952 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f926c0efff0 con 0x7f9278105030 2024-09-19T08:44:21.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.356+0000 7f92777fe640 1 --2- 172.21.15.164:0/2031071952 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f92600a32d0 0x7f92600a5790 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f92680058c0 tx=0x7f926800a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:21.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.360+0000 7f9274ff9640 1 -- 172.21.15.164:0/2031071952 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f926c0bccb0 con 0x7f9278105030 2024-09-19T08:44:21.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.590+0000 7f927e103640 1 -- 172.21.15.164:0/2031071952 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f9278062df0 con 0x7f9278105030 2024-09-19T08:44:21.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.592+0000 7f9274ff9640 1 -- 172.21.15.164:0/2031071952 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f926c0bc5b0 con 0x7f9278105030 2024-09-19T08:44:21.592 INFO:teuthology.orchestra.run.smithi164.stdout:[client.0] 2024-09-19T08:44:21.592 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQBl5OtmoiRBIxAARP5vPAZFLdzqLn+D5yBJXw== 2024-09-19T08:44:21.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.595+0000 7f927e103640 1 -- 172.21.15.164:0/2031071952 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f92600a32d0 msgr2=0x7f92600a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:21.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.595+0000 7f927e103640 1 --2- 172.21.15.164:0/2031071952 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f92600a32d0 0x7f92600a5790 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f92680058c0 tx=0x7f926800a4c0 comp rx=0 tx=0).stop 2024-09-19T08:44:21.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.595+0000 7f927e103640 1 -- 172.21.15.164:0/2031071952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9278105030 msgr2=0x7f92781006b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:21.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.595+0000 7f927e103640 1 --2- 172.21.15.164:0/2031071952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9278105030 0x7f92781006b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f926c031610 tx=0x7f926c0043b0 comp rx=0 tx=0).stop 2024-09-19T08:44:21.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.596+0000 7f927e103640 1 -- 172.21.15.164:0/2031071952 shutdown_connections 2024-09-19T08:44:21.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.596+0000 7f927e103640 1 --2- 172.21.15.164:0/2031071952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9278107960 0x7f9278100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:21.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.596+0000 7f927e103640 1 --2- 172.21.15.164:0/2031071952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9278105030 0x7f92781006b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:21.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.596+0000 7f927e103640 1 --2- 172.21.15.164:0/2031071952 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f92600a32d0 0x7f92600a5790 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:21.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.596+0000 7f927e103640 1 --2- 172.21.15.164:0/2031071952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9278102700 0x7f9278102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:21.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.596+0000 7f927e103640 1 -- 172.21.15.164:0/2031071952 >> 172.21.15.164:0/2031071952 conn(0x7f92780fc2a0 msgr2=0x7f9278103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:21.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.596+0000 7f927e103640 1 -- 172.21.15.164:0/2031071952 shutdown_connections 2024-09-19T08:44:21.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:21.596+0000 7f927e103640 1 -- 172.21.15.164:0/2031071952 wait complete. 2024-09-19T08:44:21.940 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:21 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2031071952' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T08:44:21.941 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:21 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2031071952' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T08:44:22.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:21 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2031071952' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T08:44:22.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:21 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2031071952' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T08:44:22.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:21 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2031071952' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T08:44:22.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:21 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2031071952' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T08:44:22.363 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-19T08:44:22.363 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-19T08:44:22.364 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-19T08:44:22.401 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-19T08:44:22.402 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-19T08:44:22.402 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-19T08:44:22.685 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:23.087 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:22 smithi164 ceph-mon[30928]: pgmap v292: 97 pgs: 97 active+clean; 637 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 KiB/s wr, 16 op/s 2024-09-19T08:44:23.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:22 smithi170 ceph-mon[35072]: pgmap v292: 97 pgs: 97 active+clean; 637 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 KiB/s wr, 16 op/s 2024-09-19T08:44:23.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:22 smithi173 ceph-mon[35306]: pgmap v292: 97 pgs: 97 active+clean; 637 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 KiB/s wr, 16 op/s 2024-09-19T08:44:23.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.169+0000 7f0406a1e640 1 -- 172.21.15.164:0/1844466679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0400106a80 msgr2=0x7f0400106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:23.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.169+0000 7f0406a1e640 1 --2- 172.21.15.164:0/1844466679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0400106a80 0x7f0400106e60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009930 tx=0x7f03f002a420 comp rx=0 tx=0).stop 2024-09-19T08:44:23.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.169+0000 7f0406a1e640 1 -- 172.21.15.164:0/1844466679 shutdown_connections 2024-09-19T08:44:23.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.169+0000 7f0406a1e640 1 --2- 172.21.15.164:0/1844466679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0400101420 0x7f040010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:23.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.169+0000 7f0406a1e640 1 --2- 172.21.15.164:0/1844466679 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0400100a80 0x7f0400100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:23.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.169+0000 7f0406a1e640 1 --2- 172.21.15.164:0/1844466679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0400106a80 0x7f0400106e60 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:23.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.169+0000 7f0406a1e640 1 -- 172.21.15.164:0/1844466679 >> 172.21.15.164:0/1844466679 conn(0x7f04000fc7a0 msgr2=0x7f04000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:23.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.169+0000 7f0406a1e640 1 -- 172.21.15.164:0/1844466679 shutdown_connections 2024-09-19T08:44:23.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.169+0000 7f0406a1e640 1 -- 172.21.15.164:0/1844466679 wait complete. 2024-09-19T08:44:23.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.170+0000 7f0406a1e640 1 Processor -- start 2024-09-19T08:44:23.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.170+0000 7f0406a1e640 1 -- start start 2024-09-19T08:44:23.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.170+0000 7f0406a1e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0400100a80 0x7f0400198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:23.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.170+0000 7f0406a1e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0400101420 0x7f0400198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:23.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.170+0000 7f0406a1e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0400106a80 0x7f040019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:23.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.170+0000 7f0406a1e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0400111f50 con 0x7f0400101420 2024-09-19T08:44:23.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.170+0000 7f0406a1e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0400111dd0 con 0x7f0400106a80 2024-09-19T08:44:23.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.170+0000 7f0406a1e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f04001120d0 con 0x7f0400100a80 2024-09-19T08:44:23.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.170+0000 7f03f7fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0400101420 0x7f0400198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:23.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.170+0000 7f03f7fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0400101420 0x7f0400198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35730/0 (socket says 172.21.15.164:35730) 2024-09-19T08:44:23.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.170+0000 7f03f7fff640 1 -- 172.21.15.164:0/4240336756 learned_addr learned my addr 172.21.15.164:0/4240336756 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:23.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.170+0000 7f03fffff640 1 --2- 172.21.15.164:0/4240336756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0400100a80 0x7f0400198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:23.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.170+0000 7f03f7fff640 1 -- 172.21.15.164:0/4240336756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0400100a80 msgr2=0x7f0400198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:23.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.170+0000 7f03f7fff640 1 --2- 172.21.15.164:0/4240336756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0400100a80 0x7f0400198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:23.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.170+0000 7f0404fbb640 1 --2- 172.21.15.164:0/4240336756 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0400106a80 0x7f040019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:23.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.170+0000 7f03f7fff640 1 -- 172.21.15.164:0/4240336756 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0400106a80 msgr2=0x7f040019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:23.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.170+0000 7f03f7fff640 1 --2- 172.21.15.164:0/4240336756 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0400106a80 0x7f040019cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:23.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.170+0000 7f03f7fff640 1 -- 172.21.15.164:0/4240336756 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f040019d3c0 con 0x7f0400101420 2024-09-19T08:44:23.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.170+0000 7f03fffff640 1 --2- 172.21.15.164:0/4240336756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0400100a80 0x7f0400198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:44:23.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.170+0000 7f03f7fff640 1 --2- 172.21.15.164:0/4240336756 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0400101420 0x7f0400198950 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f03e800c920 tx=0x7f03e800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:23.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.171+0000 7f03fdffb640 1 -- 172.21.15.164:0/4240336756 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8007c70 con 0x7f0400101420 2024-09-19T08:44:23.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.171+0000 7f0406a1e640 1 -- 172.21.15.164:0/4240336756 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f040019d6b0 con 0x7f0400101420 2024-09-19T08:44:23.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.171+0000 7f03fdffb640 1 -- 172.21.15.164:0/4240336756 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03e8007e10 con 0x7f0400101420 2024-09-19T08:44:23.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.171+0000 7f03fdffb640 1 -- 172.21.15.164:0/4240336756 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8018890 con 0x7f0400101420 2024-09-19T08:44:23.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.171+0000 7f0406a1e640 1 -- 172.21.15.164:0/4240336756 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04001a4f90 con 0x7f0400101420 2024-09-19T08:44:23.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.172+0000 7f0406a1e640 1 -- 172.21.15.164:0/4240336756 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0400100ee0 con 0x7f0400101420 2024-09-19T08:44:23.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.173+0000 7f03fdffb640 1 -- 172.21.15.164:0/4240336756 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f03e8018a50 con 0x7f0400101420 2024-09-19T08:44:23.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.173+0000 7f03fdffb640 1 --2- 172.21.15.164:0/4240336756 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f03ec0a32d0 0x7f03ec0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:23.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.173+0000 7f03fffff640 1 --2- 172.21.15.164:0/4240336756 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f03ec0a32d0 0x7f03ec0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:23.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.173+0000 7f03fdffb640 1 -- 172.21.15.164:0/4240336756 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f03e80cfe40 con 0x7f0400101420 2024-09-19T08:44:23.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.174+0000 7f03fffff640 1 --2- 172.21.15.164:0/4240336756 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f03ec0a32d0 0x7f03ec0a5790 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f03f0000c00 tx=0x7f03f0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:23.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.176+0000 7f03fdffb640 1 -- 172.21.15.164:0/4240336756 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f03e8014030 con 0x7f0400101420 2024-09-19T08:44:23.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.387+0000 7f0406a1e640 1 -- 172.21.15.164:0/4240336756 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f040002d7c0 con 0x7f0400101420 2024-09-19T08:44:23.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.391+0000 7f03fdffb640 1 -- 172.21.15.164:0/4240336756 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264059 (secure 0 0 0) 0x7f03e809cb90 con 0x7f0400101420 2024-09-19T08:44:23.394 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:44:23.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.394+0000 7f0406a1e640 1 -- 172.21.15.164:0/4240336756 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f03ec0a32d0 msgr2=0x7f03ec0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:23.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.394+0000 7f0406a1e640 1 --2- 172.21.15.164:0/4240336756 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f03ec0a32d0 0x7f03ec0a5790 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f03f0000c00 tx=0x7f03f0005a20 comp rx=0 tx=0).stop 2024-09-19T08:44:23.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.394+0000 7f0406a1e640 1 -- 172.21.15.164:0/4240336756 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0400101420 msgr2=0x7f0400198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:23.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.394+0000 7f0406a1e640 1 --2- 172.21.15.164:0/4240336756 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0400101420 0x7f0400198950 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f03e800c920 tx=0x7f03e800cde0 comp rx=0 tx=0).stop 2024-09-19T08:44:23.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.394+0000 7f0406a1e640 1 -- 172.21.15.164:0/4240336756 shutdown_connections 2024-09-19T08:44:23.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.394+0000 7f0406a1e640 1 --2- 172.21.15.164:0/4240336756 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0400106a80 0x7f040019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:23.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.394+0000 7f0406a1e640 1 --2- 172.21.15.164:0/4240336756 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0400101420 0x7f0400198950 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:23.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.394+0000 7f0406a1e640 1 --2- 172.21.15.164:0/4240336756 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f03ec0a32d0 0x7f03ec0a5790 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:23.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.394+0000 7f0406a1e640 1 --2- 172.21.15.164:0/4240336756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0400100a80 0x7f0400198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:23.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.394+0000 7f0406a1e640 1 -- 172.21.15.164:0/4240336756 >> 172.21.15.164:0/4240336756 conn(0x7f04000fc7a0 msgr2=0x7f04000fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:23.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.394+0000 7f0406a1e640 1 -- 172.21.15.164:0/4240336756 shutdown_connections 2024-09-19T08:44:23.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:23.394+0000 7f0406a1e640 1 -- 172.21.15.164:0/4240336756 wait complete. 2024-09-19T08:44:24.084 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:23 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/4240336756' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T08:44:24.084 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:23 smithi164 ceph-mon[30928]: pgmap v293: 97 pgs: 97 active+clean; 637 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-09-19T08:44:24.110 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":1094864383},{"type":"v1","addr":"172.21.15.164:6801","nonce":1094864383}]},"active_addr":"172.21.15.164:6801/1094864383","active_change":"2024-09-19T08:37:01.663175+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.164:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.164:0","nonce":1807120560}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.164:0","nonce":3424843982}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.164:0","nonce":2087067799}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.164:0","nonce":2727626049}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.164:0","nonce":2843507036}]}]} 2024-09-19T08:44:24.116 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-19T08:44:24.116 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-19T08:44:24.116 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T08:44:24.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:23 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/4240336756' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T08:44:24.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:23 smithi170 ceph-mon[35072]: pgmap v293: 97 pgs: 97 active+clean; 637 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-09-19T08:44:24.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:23 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/4240336756' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T08:44:24.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:23 smithi173 ceph-mon[35306]: pgmap v293: 97 pgs: 97 active+clean; 637 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-09-19T08:44:24.361 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:24.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.861+0000 7fcce079c640 1 -- 172.21.15.164:0/455907033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccd8101420 msgr2=0x7fccd810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:24.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.861+0000 7fcce079c640 1 --2- 172.21.15.164:0/455907033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccd8101420 0x7fccd810d950 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fccd0009cd0 tx=0x7fccd002a8e0 comp rx=0 tx=0).stop 2024-09-19T08:44:24.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.861+0000 7fcce079c640 1 -- 172.21.15.164:0/455907033 shutdown_connections 2024-09-19T08:44:24.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.861+0000 7fcce079c640 1 --2- 172.21.15.164:0/455907033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccd8101420 0x7fccd810d950 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:24.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.861+0000 7fcce079c640 1 --2- 172.21.15.164:0/455907033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccd8100a80 0x7fccd8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:24.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.861+0000 7fcce079c640 1 --2- 172.21.15.164:0/455907033 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fccd8106a80 0x7fccd8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:24.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.861+0000 7fcce079c640 1 -- 172.21.15.164:0/455907033 >> 172.21.15.164:0/455907033 conn(0x7fccd80fc7a0 msgr2=0x7fccd80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:24.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.861+0000 7fcce079c640 1 -- 172.21.15.164:0/455907033 shutdown_connections 2024-09-19T08:44:24.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.861+0000 7fcce079c640 1 -- 172.21.15.164:0/455907033 wait complete. 2024-09-19T08:44:24.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.862+0000 7fcce079c640 1 Processor -- start 2024-09-19T08:44:24.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.862+0000 7fcce079c640 1 -- start start 2024-09-19T08:44:24.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.862+0000 7fcce079c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccd8100a80 0x7fccd8196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:24.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.862+0000 7fcce079c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccd8101420 0x7fccd8197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:24.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.862+0000 7fcce079c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fccd8106a80 0x7fccd8197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:24.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.862+0000 7fcce079c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fccd8110e80 con 0x7fccd8100a80 2024-09-19T08:44:24.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.862+0000 7fcce079c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fccd8110d00 con 0x7fccd8101420 2024-09-19T08:44:24.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.862+0000 7fcce079c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fccd8111000 con 0x7fccd8106a80 2024-09-19T08:44:24.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.862+0000 7fccde538640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccd8100a80 0x7fccd8196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:24.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.862+0000 7fccde538640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccd8100a80 0x7fccd8196ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:32990/0 (socket says 172.21.15.164:32990) 2024-09-19T08:44:24.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.862+0000 7fccde538640 1 -- 172.21.15.164:0/3780642280 learned_addr learned my addr 172.21.15.164:0/3780642280 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:24.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.862+0000 7fccded39640 1 --2- 172.21.15.164:0/3780642280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fccd8106a80 0x7fccd8197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:24.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.862+0000 7fccddd37640 1 --2- 172.21.15.164:0/3780642280 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccd8101420 0x7fccd8197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:24.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.862+0000 7fccde538640 1 -- 172.21.15.164:0/3780642280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fccd8106a80 msgr2=0x7fccd8197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:24.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.862+0000 7fccde538640 1 --2- 172.21.15.164:0/3780642280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fccd8106a80 0x7fccd8197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:24.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.862+0000 7fccde538640 1 -- 172.21.15.164:0/3780642280 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccd8101420 msgr2=0x7fccd8197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:24.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.862+0000 7fccde538640 1 --2- 172.21.15.164:0/3780642280 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccd8101420 0x7fccd8197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:24.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.862+0000 7fccde538640 1 -- 172.21.15.164:0/3780642280 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd81a6860 con 0x7fccd8100a80 2024-09-19T08:44:24.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.863+0000 7fccde538640 1 --2- 172.21.15.164:0/3780642280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccd8100a80 0x7fccd8196ff0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcccc00c920 tx=0x7fcccc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:24.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.863+0000 7fcccb7fe640 1 -- 172.21.15.164:0/3780642280 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc007c70 con 0x7fccd8100a80 2024-09-19T08:44:24.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.863+0000 7fcccb7fe640 1 -- 172.21.15.164:0/3780642280 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcccc007e10 con 0x7fccd8100a80 2024-09-19T08:44:24.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.863+0000 7fcce079c640 1 -- 172.21.15.164:0/3780642280 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd81a6b50 con 0x7fccd8100a80 2024-09-19T08:44:24.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.863+0000 7fcccb7fe640 1 -- 172.21.15.164:0/3780642280 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc018890 con 0x7fccd8100a80 2024-09-19T08:44:24.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.863+0000 7fcce079c640 1 -- 172.21.15.164:0/3780642280 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd81a70e0 con 0x7fccd8100a80 2024-09-19T08:44:24.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.864+0000 7fcce079c640 1 -- 172.21.15.164:0/3780642280 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccd8100ee0 con 0x7fccd8100a80 2024-09-19T08:44:24.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.866+0000 7fcccb7fe640 1 -- 172.21.15.164:0/3780642280 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fcccc018a50 con 0x7fccd8100a80 2024-09-19T08:44:24.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.866+0000 7fcccb7fe640 1 --2- 172.21.15.164:0/3780642280 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fccc40a3350 0x7fccc40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:24.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.866+0000 7fccddd37640 1 --2- 172.21.15.164:0/3780642280 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fccc40a3350 0x7fccc40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:24.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.867+0000 7fcccb7fe640 1 -- 172.21.15.164:0/3780642280 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fcccc0cfc30 con 0x7fccd8100a80 2024-09-19T08:44:24.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.867+0000 7fccddd37640 1 --2- 172.21.15.164:0/3780642280 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fccc40a3350 0x7fccc40a5810 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fccd8199150 tx=0x7fccd400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:24.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:24.870+0000 7fcccb7fe640 1 -- 172.21.15.164:0/3780642280 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcccc014030 con 0x7fccd8100a80 2024-09-19T08:44:25.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:25.040+0000 7fcce079c640 1 -- 172.21.15.164:0/3780642280 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fccd806ad80 con 0x7fccd8100a80 2024-09-19T08:44:25.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:25.041+0000 7fcccb7fe640 1 -- 172.21.15.164:0/3780642280 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22702 (secure 0 0 0) 0x7fccd806ad80 con 0x7fccd8100a80 2024-09-19T08:44:25.041 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:44:25.041 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":82,"fsid":"1e4b0d74-7662-11ef-bceb-c7b262605968","created":"2024-09-19T08:36:09.564509+0000","modified":"2024-09-19T08:44:02.811236+0000","last_up_change":"2024-09-19T08:43:01.260852+0000","last_in_change":"2024-09-19T08:42:37.492464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:39:13.961749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:43:49.613471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T08:43:51.570675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33b11f37-6ac5-41fc-8cfd-1b08c7eef688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6803","nonce":663038243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6805","nonce":663038243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6809","nonce":663038243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6807","nonce":663038243}]},"public_addr":"172.21.15.164:6803/663038243","cluster_addr":"172.21.15.164:6805/663038243","heartbeat_back_addr":"172.21.15.164:6809/663038243","heartbeat_front_addr":"172.21.15.164:6807/663038243","state":["exists","up"]},{"osd":1,"uuid":"6de64b0a-46e2-4a98-8005-a326a64bc87d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6801","nonce":2167066371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6803","nonce":2167066371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6807","nonce":2167066371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6805","nonce":2167066371}]},"public_addr":"172.21.15.170:6801/2167066371","cluster_addr":"172.21.15.170:6803/2167066371","heartbeat_back_addr":"172.21.15.170:6807/2167066371","heartbeat_front_addr":"172.21.15.170:6805/2167066371","state":["exists","up"]},{"osd":2,"uuid":"d4158cc7-9822-42a7-9601-d65089907584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6801","nonce":1646561048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6803","nonce":1646561048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6807","nonce":1646561048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6805","nonce":1646561048}]},"public_addr":"172.21.15.173:6801/1646561048","cluster_addr":"172.21.15.173:6803/1646561048","heartbeat_back_addr":"172.21.15.173:6807/1646561048","heartbeat_front_addr":"172.21.15.173:6805/1646561048","state":["exists","up"]},{"osd":3,"uuid":"0cf3f7f1-3a8c-42ce-88a9-9bea52e02221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6811","nonce":406795560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6813","nonce":406795560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6817","nonce":406795560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6815","nonce":406795560}]},"public_addr":"172.21.15.164:6811/406795560","cluster_addr":"172.21.15.164:6813/406795560","heartbeat_back_addr":"172.21.15.164:6817/406795560","heartbeat_front_addr":"172.21.15.164:6815/406795560","state":["exists","up"]},{"osd":4,"uuid":"6dca048a-2860-4a02-8d8e-450b30bc1a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6809","nonce":4120832165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6811","nonce":4120832165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6815","nonce":4120832165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6813","nonce":4120832165}]},"public_addr":"172.21.15.170:6809/4120832165","cluster_addr":"172.21.15.170:6811/4120832165","heartbeat_back_addr":"172.21.15.170:6815/4120832165","heartbeat_front_addr":"172.21.15.170:6813/4120832165","state":["exists","up"]},{"osd":5,"uuid":"3822521f-69d6-4141-a83f-5fc9cecc7b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6809","nonce":2914653552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6811","nonce":2914653552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6815","nonce":2914653552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6813","nonce":2914653552}]},"public_addr":"172.21.15.173:6809/2914653552","cluster_addr":"172.21.15.173:6811/2914653552","heartbeat_back_addr":"172.21.15.173:6815/2914653552","heartbeat_front_addr":"172.21.15.173:6813/2914653552","state":["exists","up"]},{"osd":6,"uuid":"0f5a965e-2eb2-4c24-b089-577e0639c160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6819","nonce":1138341393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6821","nonce":1138341393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6825","nonce":1138341393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6823","nonce":1138341393}]},"public_addr":"172.21.15.164:6819/1138341393","cluster_addr":"172.21.15.164:6821/1138341393","heartbeat_back_addr":"172.21.15.164:6825/1138341393","heartbeat_front_addr":"172.21.15.164:6823/1138341393","state":["exists","up"]},{"osd":7,"uuid":"f16cb2c3-639b-46ae-a740-1bd28ca1863c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6817","nonce":2124160360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6819","nonce":2124160360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6823","nonce":2124160360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6821","nonce":2124160360}]},"public_addr":"172.21.15.170:6817/2124160360","cluster_addr":"172.21.15.170:6819/2124160360","heartbeat_back_addr":"172.21.15.170:6823/2124160360","heartbeat_front_addr":"172.21.15.170:6821/2124160360","state":["exists","up"]},{"osd":8,"uuid":"161cff01-46ad-48e3-b999-a5473a2ad618","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6817","nonce":2285175061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6819","nonce":2285175061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6823","nonce":2285175061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6821","nonce":2285175061}]},"public_addr":"172.21.15.173:6817/2285175061","cluster_addr":"172.21.15.173:6819/2285175061","heartbeat_back_addr":"172.21.15.173:6823/2285175061","heartbeat_front_addr":"172.21.15.173:6821/2285175061","state":["exists","up"]},{"osd":9,"uuid":"994666e8-8fbd-4231-ac67-81a79aaf7c2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6827","nonce":3918847952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6829","nonce":3918847952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6833","nonce":3918847952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6831","nonce":3918847952}]},"public_addr":"172.21.15.164:6827/3918847952","cluster_addr":"172.21.15.164:6829/3918847952","heartbeat_back_addr":"172.21.15.164:6833/3918847952","heartbeat_front_addr":"172.21.15.164:6831/3918847952","state":["exists","up"]},{"osd":10,"uuid":"ff5e9099-cdb4-4905-b888-5b67320ef76b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6825","nonce":2707848227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6827","nonce":2707848227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6831","nonce":2707848227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6829","nonce":2707848227}]},"public_addr":"172.21.15.170:6825/2707848227","cluster_addr":"172.21.15.170:6827/2707848227","heartbeat_back_addr":"172.21.15.170:6831/2707848227","heartbeat_front_addr":"172.21.15.170:6829/2707848227","state":["exists","up"]},{"osd":11,"uuid":"4d93088f-e791-4938-a207-12f15b56abd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6825","nonce":2831924738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6827","nonce":2831924738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6831","nonce":2831924738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6829","nonce":2831924738}]},"public_addr":"172.21.15.173:6825/2831924738","cluster_addr":"172.21.15.173:6827/2831924738","heartbeat_back_addr":"172.21.15.173:6831/2831924738","heartbeat_front_addr":"172.21.15.173:6829/2831924738","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:21.787352+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:46.012323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:39:11.353040+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:39:37.015124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:02.063379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:27.892865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:53.372041+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:41:17.855284+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:41:43.389882+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:08.956895+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:34.069779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:59.303897+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.164:0/339032808":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6801/1173822158":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6800/1173822158":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6801/2805828009":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1837164919":"2024-09-20T08:37:01.663065+0000","172.21.15.164:0/4018459143":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6800/2805828009":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/2770797546":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1905394497":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1369265436":"2024-09-20T08:36:33.856642+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:44:25.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:25.044+0000 7fcce079c640 1 -- 172.21.15.164:0/3780642280 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fccc40a3350 msgr2=0x7fccc40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:25.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:25.044+0000 7fcce079c640 1 --2- 172.21.15.164:0/3780642280 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fccc40a3350 0x7fccc40a5810 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fccd8199150 tx=0x7fccd400b410 comp rx=0 tx=0).stop 2024-09-19T08:44:25.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:25.044+0000 7fcce079c640 1 -- 172.21.15.164:0/3780642280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccd8100a80 msgr2=0x7fccd8196ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:25.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:25.044+0000 7fcce079c640 1 --2- 172.21.15.164:0/3780642280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccd8100a80 0x7fccd8196ff0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcccc00c920 tx=0x7fcccc00cde0 comp rx=0 tx=0).stop 2024-09-19T08:44:25.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:25.044+0000 7fcce079c640 1 -- 172.21.15.164:0/3780642280 shutdown_connections 2024-09-19T08:44:25.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:25.044+0000 7fcce079c640 1 --2- 172.21.15.164:0/3780642280 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fccc40a3350 0x7fccc40a5810 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:25.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:25.044+0000 7fcce079c640 1 --2- 172.21.15.164:0/3780642280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fccd8106a80 0x7fccd8197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:25.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:25.044+0000 7fcce079c640 1 --2- 172.21.15.164:0/3780642280 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccd8101420 0x7fccd8197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:25.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:25.044+0000 7fcce079c640 1 --2- 172.21.15.164:0/3780642280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccd8100a80 0x7fccd8196ff0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:25.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:25.044+0000 7fcce079c640 1 -- 172.21.15.164:0/3780642280 >> 172.21.15.164:0/3780642280 conn(0x7fccd80fc7a0 msgr2=0x7fccd80fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:25.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:25.044+0000 7fcce079c640 1 -- 172.21.15.164:0/3780642280 shutdown_connections 2024-09-19T08:44:25.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:25.044+0000 7fcce079c640 1 -- 172.21.15.164:0/3780642280 wait complete. 2024-09-19T08:44:25.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:25 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3780642280' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:44:25.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:25 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3780642280' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:44:25.509 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:25 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3780642280' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:44:25.983 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-19T08:44:25.983 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T08:44:26.233 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:26.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:26 smithi164 ceph-mon[30928]: pgmap v294: 97 pgs: 97 active+clean; 637 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-09-19T08:44:26.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:26 smithi170 ceph-mon[35072]: pgmap v294: 97 pgs: 97 active+clean; 637 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-09-19T08:44:26.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:26 smithi173 ceph-mon[35306]: pgmap v294: 97 pgs: 97 active+clean; 637 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-09-19T08:44:26.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.683+0000 7fe0a2a6e640 1 -- 172.21.15.164:0/2652487940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe09c100a80 msgr2=0x7fe09c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:26.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.683+0000 7fe0a2a6e640 1 --2- 172.21.15.164:0/2652487940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe09c100a80 0x7fe09c100ee0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe088009930 tx=0x7fe08802a420 comp rx=0 tx=0).stop 2024-09-19T08:44:26.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.683+0000 7fe0a2a6e640 1 -- 172.21.15.164:0/2652487940 shutdown_connections 2024-09-19T08:44:26.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.683+0000 7fe0a2a6e640 1 --2- 172.21.15.164:0/2652487940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe09c101420 0x7fe09c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:26.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.683+0000 7fe0a2a6e640 1 --2- 172.21.15.164:0/2652487940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe09c100a80 0x7fe09c100ee0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:26.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.683+0000 7fe0a2a6e640 1 --2- 172.21.15.164:0/2652487940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe09c106a80 0x7fe09c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:26.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.683+0000 7fe0a2a6e640 1 -- 172.21.15.164:0/2652487940 >> 172.21.15.164:0/2652487940 conn(0x7fe09c0fc7a0 msgr2=0x7fe09c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:26.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.683+0000 7fe0a2a6e640 1 -- 172.21.15.164:0/2652487940 shutdown_connections 2024-09-19T08:44:26.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.683+0000 7fe0a2a6e640 1 -- 172.21.15.164:0/2652487940 wait complete. 2024-09-19T08:44:26.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.684+0000 7fe0a2a6e640 1 Processor -- start 2024-09-19T08:44:26.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.684+0000 7fe0a2a6e640 1 -- start start 2024-09-19T08:44:26.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.684+0000 7fe0a2a6e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe09c100a80 0x7fe09c197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:26.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.684+0000 7fe0a2a6e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe09c101420 0x7fe09c197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:26.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.684+0000 7fe0a2a6e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe09c106a80 0x7fe09c19bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:26.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.684+0000 7fe0a2a6e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c110e80 con 0x7fe09c101420 2024-09-19T08:44:26.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.684+0000 7fe0a2a6e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c110d00 con 0x7fe09c106a80 2024-09-19T08:44:26.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.684+0000 7fe0a2a6e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c111000 con 0x7fe09c100a80 2024-09-19T08:44:26.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.684+0000 7fe09bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe09c101420 0x7fe09c197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:26.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.684+0000 7fe09bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe09c101420 0x7fe09c197880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33012/0 (socket says 172.21.15.164:33012) 2024-09-19T08:44:26.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.684+0000 7fe09bfff640 1 -- 172.21.15.164:0/3726363648 learned_addr learned my addr 172.21.15.164:0/3726363648 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:26.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.684+0000 7fe0a080a640 1 --2- 172.21.15.164:0/3726363648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe09c100a80 0x7fe09c197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:26.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.684+0000 7fe0a100b640 1 --2- 172.21.15.164:0/3726363648 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe09c106a80 0x7fe09c19bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:26.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.685+0000 7fe09bfff640 1 -- 172.21.15.164:0/3726363648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe09c100a80 msgr2=0x7fe09c197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:26.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.685+0000 7fe09bfff640 1 --2- 172.21.15.164:0/3726363648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe09c100a80 0x7fe09c197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:26.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.685+0000 7fe09bfff640 1 -- 172.21.15.164:0/3726363648 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe09c106a80 msgr2=0x7fe09c19bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:26.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.685+0000 7fe09bfff640 1 --2- 172.21.15.164:0/3726363648 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe09c106a80 0x7fe09c19bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:26.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.685+0000 7fe09bfff640 1 -- 172.21.15.164:0/3726363648 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe09c19c2f0 con 0x7fe09c101420 2024-09-19T08:44:26.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.685+0000 7fe09bfff640 1 --2- 172.21.15.164:0/3726363648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe09c101420 0x7fe09c197880 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe088009230 tx=0x7fe088004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:26.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.685+0000 7fe099ffb640 1 -- 172.21.15.164:0/3726363648 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08803f030 con 0x7fe09c101420 2024-09-19T08:44:26.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.685+0000 7fe099ffb640 1 -- 172.21.15.164:0/3726363648 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe088003ad0 con 0x7fe09c101420 2024-09-19T08:44:26.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.685+0000 7fe0a2a6e640 1 -- 172.21.15.164:0/3726363648 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe09c19c580 con 0x7fe09c101420 2024-09-19T08:44:26.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.685+0000 7fe099ffb640 1 -- 172.21.15.164:0/3726363648 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe088035070 con 0x7fe09c101420 2024-09-19T08:44:26.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.685+0000 7fe0a2a6e640 1 -- 172.21.15.164:0/3726363648 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe09c1a3ec0 con 0x7fe09c101420 2024-09-19T08:44:26.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.686+0000 7fe0a2a6e640 1 -- 172.21.15.164:0/3726363648 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe09c100ee0 con 0x7fe09c101420 2024-09-19T08:44:26.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.687+0000 7fe099ffb640 1 -- 172.21.15.164:0/3726363648 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fe088042030 con 0x7fe09c101420 2024-09-19T08:44:26.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.688+0000 7fe099ffb640 1 --2- 172.21.15.164:0/3726363648 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fe08c0a32d0 0x7fe08c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:26.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.688+0000 7fe0a080a640 1 --2- 172.21.15.164:0/3726363648 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fe08c0a32d0 0x7fe08c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:26.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.688+0000 7fe099ffb640 1 -- 172.21.15.164:0/3726363648 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe0880f02d0 con 0x7fe09c101420 2024-09-19T08:44:26.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.688+0000 7fe0a080a640 1 --2- 172.21.15.164:0/3726363648 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fe08c0a32d0 0x7fe08c0a5790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe09000bb60 tx=0x7fe09000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:26.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.691+0000 7fe099ffb640 1 -- 172.21.15.164:0/3726363648 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe0880bcf90 con 0x7fe09c101420 2024-09-19T08:44:26.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.862+0000 7fe0a2a6e640 1 -- 172.21.15.164:0/3726363648 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe09c109010 con 0x7fe09c101420 2024-09-19T08:44:26.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.863+0000 7fe099ffb640 1 -- 172.21.15.164:0/3726363648 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22702 (secure 0 0 0) 0x7fe0880bc890 con 0x7fe09c101420 2024-09-19T08:44:26.864 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:44:26.865 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":82,"fsid":"1e4b0d74-7662-11ef-bceb-c7b262605968","created":"2024-09-19T08:36:09.564509+0000","modified":"2024-09-19T08:44:02.811236+0000","last_up_change":"2024-09-19T08:43:01.260852+0000","last_in_change":"2024-09-19T08:42:37.492464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:39:13.961749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:43:49.613471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T08:43:51.570675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33b11f37-6ac5-41fc-8cfd-1b08c7eef688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6803","nonce":663038243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6805","nonce":663038243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6809","nonce":663038243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6807","nonce":663038243}]},"public_addr":"172.21.15.164:6803/663038243","cluster_addr":"172.21.15.164:6805/663038243","heartbeat_back_addr":"172.21.15.164:6809/663038243","heartbeat_front_addr":"172.21.15.164:6807/663038243","state":["exists","up"]},{"osd":1,"uuid":"6de64b0a-46e2-4a98-8005-a326a64bc87d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6801","nonce":2167066371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6803","nonce":2167066371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6807","nonce":2167066371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6805","nonce":2167066371}]},"public_addr":"172.21.15.170:6801/2167066371","cluster_addr":"172.21.15.170:6803/2167066371","heartbeat_back_addr":"172.21.15.170:6807/2167066371","heartbeat_front_addr":"172.21.15.170:6805/2167066371","state":["exists","up"]},{"osd":2,"uuid":"d4158cc7-9822-42a7-9601-d65089907584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6801","nonce":1646561048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6803","nonce":1646561048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6807","nonce":1646561048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6805","nonce":1646561048}]},"public_addr":"172.21.15.173:6801/1646561048","cluster_addr":"172.21.15.173:6803/1646561048","heartbeat_back_addr":"172.21.15.173:6807/1646561048","heartbeat_front_addr":"172.21.15.173:6805/1646561048","state":["exists","up"]},{"osd":3,"uuid":"0cf3f7f1-3a8c-42ce-88a9-9bea52e02221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6811","nonce":406795560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6813","nonce":406795560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6817","nonce":406795560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6815","nonce":406795560}]},"public_addr":"172.21.15.164:6811/406795560","cluster_addr":"172.21.15.164:6813/406795560","heartbeat_back_addr":"172.21.15.164:6817/406795560","heartbeat_front_addr":"172.21.15.164:6815/406795560","state":["exists","up"]},{"osd":4,"uuid":"6dca048a-2860-4a02-8d8e-450b30bc1a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6809","nonce":4120832165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6811","nonce":4120832165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6815","nonce":4120832165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6813","nonce":4120832165}]},"public_addr":"172.21.15.170:6809/4120832165","cluster_addr":"172.21.15.170:6811/4120832165","heartbeat_back_addr":"172.21.15.170:6815/4120832165","heartbeat_front_addr":"172.21.15.170:6813/4120832165","state":["exists","up"]},{"osd":5,"uuid":"3822521f-69d6-4141-a83f-5fc9cecc7b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6809","nonce":2914653552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6811","nonce":2914653552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6815","nonce":2914653552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6813","nonce":2914653552}]},"public_addr":"172.21.15.173:6809/2914653552","cluster_addr":"172.21.15.173:6811/2914653552","heartbeat_back_addr":"172.21.15.173:6815/2914653552","heartbeat_front_addr":"172.21.15.173:6813/2914653552","state":["exists","up"]},{"osd":6,"uuid":"0f5a965e-2eb2-4c24-b089-577e0639c160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6819","nonce":1138341393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6821","nonce":1138341393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6825","nonce":1138341393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6823","nonce":1138341393}]},"public_addr":"172.21.15.164:6819/1138341393","cluster_addr":"172.21.15.164:6821/1138341393","heartbeat_back_addr":"172.21.15.164:6825/1138341393","heartbeat_front_addr":"172.21.15.164:6823/1138341393","state":["exists","up"]},{"osd":7,"uuid":"f16cb2c3-639b-46ae-a740-1bd28ca1863c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6817","nonce":2124160360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6819","nonce":2124160360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6823","nonce":2124160360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6821","nonce":2124160360}]},"public_addr":"172.21.15.170:6817/2124160360","cluster_addr":"172.21.15.170:6819/2124160360","heartbeat_back_addr":"172.21.15.170:6823/2124160360","heartbeat_front_addr":"172.21.15.170:6821/2124160360","state":["exists","up"]},{"osd":8,"uuid":"161cff01-46ad-48e3-b999-a5473a2ad618","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6817","nonce":2285175061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6819","nonce":2285175061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6823","nonce":2285175061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6821","nonce":2285175061}]},"public_addr":"172.21.15.173:6817/2285175061","cluster_addr":"172.21.15.173:6819/2285175061","heartbeat_back_addr":"172.21.15.173:6823/2285175061","heartbeat_front_addr":"172.21.15.173:6821/2285175061","state":["exists","up"]},{"osd":9,"uuid":"994666e8-8fbd-4231-ac67-81a79aaf7c2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6827","nonce":3918847952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6829","nonce":3918847952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6833","nonce":3918847952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6831","nonce":3918847952}]},"public_addr":"172.21.15.164:6827/3918847952","cluster_addr":"172.21.15.164:6829/3918847952","heartbeat_back_addr":"172.21.15.164:6833/3918847952","heartbeat_front_addr":"172.21.15.164:6831/3918847952","state":["exists","up"]},{"osd":10,"uuid":"ff5e9099-cdb4-4905-b888-5b67320ef76b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6825","nonce":2707848227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6827","nonce":2707848227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6831","nonce":2707848227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6829","nonce":2707848227}]},"public_addr":"172.21.15.170:6825/2707848227","cluster_addr":"172.21.15.170:6827/2707848227","heartbeat_back_addr":"172.21.15.170:6831/2707848227","heartbeat_front_addr":"172.21.15.170:6829/2707848227","state":["exists","up"]},{"osd":11,"uuid":"4d93088f-e791-4938-a207-12f15b56abd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6825","nonce":2831924738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6827","nonce":2831924738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6831","nonce":2831924738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6829","nonce":2831924738}]},"public_addr":"172.21.15.173:6825/2831924738","cluster_addr":"172.21.15.173:6827/2831924738","heartbeat_back_addr":"172.21.15.173:6831/2831924738","heartbeat_front_addr":"172.21.15.173:6829/2831924738","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:21.787352+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:46.012323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:39:11.353040+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:39:37.015124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:02.063379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:27.892865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:53.372041+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:41:17.855284+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:41:43.389882+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:08.956895+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:34.069779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:59.303897+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.164:0/339032808":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6801/1173822158":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6800/1173822158":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6801/2805828009":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1837164919":"2024-09-20T08:37:01.663065+0000","172.21.15.164:0/4018459143":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6800/2805828009":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/2770797546":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1905394497":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1369265436":"2024-09-20T08:36:33.856642+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:44:26.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.865+0000 7fe0a2a6e640 1 -- 172.21.15.164:0/3726363648 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fe08c0a32d0 msgr2=0x7fe08c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:26.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.865+0000 7fe0a2a6e640 1 --2- 172.21.15.164:0/3726363648 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fe08c0a32d0 0x7fe08c0a5790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe09000bb60 tx=0x7fe09000b390 comp rx=0 tx=0).stop 2024-09-19T08:44:26.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.865+0000 7fe0a2a6e640 1 -- 172.21.15.164:0/3726363648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe09c101420 msgr2=0x7fe09c197880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:26.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.865+0000 7fe0a2a6e640 1 --2- 172.21.15.164:0/3726363648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe09c101420 0x7fe09c197880 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe088009230 tx=0x7fe088004270 comp rx=0 tx=0).stop 2024-09-19T08:44:26.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.865+0000 7fe0a2a6e640 1 -- 172.21.15.164:0/3726363648 shutdown_connections 2024-09-19T08:44:26.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.865+0000 7fe0a2a6e640 1 --2- 172.21.15.164:0/3726363648 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe09c106a80 0x7fe09c19bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:26.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.865+0000 7fe0a2a6e640 1 --2- 172.21.15.164:0/3726363648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe09c101420 0x7fe09c197880 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:26.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.865+0000 7fe0a2a6e640 1 --2- 172.21.15.164:0/3726363648 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fe08c0a32d0 0x7fe08c0a5790 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:26.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.865+0000 7fe0a2a6e640 1 --2- 172.21.15.164:0/3726363648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe09c100a80 0x7fe09c197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:26.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.865+0000 7fe0a2a6e640 1 -- 172.21.15.164:0/3726363648 >> 172.21.15.164:0/3726363648 conn(0x7fe09c0fc7a0 msgr2=0x7fe09c0fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:26.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.866+0000 7fe0a2a6e640 1 -- 172.21.15.164:0/3726363648 shutdown_connections 2024-09-19T08:44:26.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:26.866+0000 7fe0a2a6e640 1 -- 172.21.15.164:0/3726363648 wait complete. 2024-09-19T08:44:27.265 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:27 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3726363648' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:44:27.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:27 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3726363648' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:44:27.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:27 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3726363648' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:44:27.605 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-19T08:44:27.606 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-19T08:44:27.606 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-19T08:44:27.607 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-19T08:44:27.607 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-19T08:44:27.608 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-19T08:44:27.608 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-19T08:44:27.608 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-19T08:44:27.609 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-19T08:44:27.610 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-19T08:44:27.610 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-19T08:44:27.610 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-19T08:44:28.304 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:28.356 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:28.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:28 smithi170 ceph-mon[35072]: pgmap v295: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-09-19T08:44:28.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:28 smithi173 ceph-mon[35306]: pgmap v295: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-09-19T08:44:28.473 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:28 smithi164 ceph-mon[30928]: pgmap v295: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-09-19T08:44:28.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.843+0000 7f7eee7a8640 1 -- 172.21.15.164:0/1585890909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ee81027a0 msgr2=0x7f7ee81067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:28.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.843+0000 7f7eee7a8640 1 --2- 172.21.15.164:0/1585890909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ee81027a0 0x7f7ee81067f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7edc009900 tx=0x7f7edc02a410 comp rx=0 tx=0).stop 2024-09-19T08:44:28.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.843+0000 7f7eee7a8640 1 -- 172.21.15.164:0/1585890909 shutdown_connections 2024-09-19T08:44:28.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.843+0000 7f7eee7a8640 1 --2- 172.21.15.164:0/1585890909 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ee8106f20 0x7f7ee810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:28.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.843+0000 7f7eee7a8640 1 --2- 172.21.15.164:0/1585890909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ee81027a0 0x7f7ee81067f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:28.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.843+0000 7f7eee7a8640 1 --2- 172.21.15.164:0/1585890909 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ee8101df0 0x7f7ee81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:28.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.843+0000 7f7eee7a8640 1 -- 172.21.15.164:0/1585890909 >> 172.21.15.164:0/1585890909 conn(0x7f7ee80fd8b0 msgr2=0x7f7ee80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:28.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.843+0000 7f7eee7a8640 1 -- 172.21.15.164:0/1585890909 shutdown_connections 2024-09-19T08:44:28.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.843+0000 7f7eee7a8640 1 -- 172.21.15.164:0/1585890909 wait complete. 2024-09-19T08:44:28.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.844+0000 7f7eee7a8640 1 Processor -- start 2024-09-19T08:44:28.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.844+0000 7f7eee7a8640 1 -- start start 2024-09-19T08:44:28.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.844+0000 7f7eee7a8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ee8101df0 0x7f7ee80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:28.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.844+0000 7f7eee7a8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ee81027a0 0x7f7ee8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:28.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.844+0000 7f7eee7a8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ee8106f20 0x7f7ee8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:28.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.844+0000 7f7eee7a8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee8111fa0 con 0x7f7ee8101df0 2024-09-19T08:44:28.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.844+0000 7f7eee7a8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee8111e20 con 0x7f7ee81027a0 2024-09-19T08:44:28.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.844+0000 7f7eee7a8640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee8112120 con 0x7f7ee8106f20 2024-09-19T08:44:28.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.844+0000 7f7ee7fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ee8101df0 0x7f7ee80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:28.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.844+0000 7f7ee7fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ee8101df0 0x7f7ee80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33026/0 (socket says 172.21.15.164:33026) 2024-09-19T08:44:28.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.844+0000 7f7ee7fff640 1 -- 172.21.15.164:0/1465968148 learned_addr learned my addr 172.21.15.164:0/1465968148 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:28.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.844+0000 7f7ee77fe640 1 --2- 172.21.15.164:0/1465968148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ee81027a0 0x7f7ee8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:28.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.844+0000 7f7eecd45640 1 --2- 172.21.15.164:0/1465968148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ee8106f20 0x7f7ee8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:28.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.845+0000 7f7ee7fff640 1 -- 172.21.15.164:0/1465968148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ee8106f20 msgr2=0x7f7ee8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:28.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.845+0000 7f7ee7fff640 1 --2- 172.21.15.164:0/1465968148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ee8106f20 0x7f7ee8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:28.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.845+0000 7f7ee7fff640 1 -- 172.21.15.164:0/1465968148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ee81027a0 msgr2=0x7f7ee8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:28.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.845+0000 7f7ee7fff640 1 --2- 172.21.15.164:0/1465968148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ee81027a0 0x7f7ee8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:28.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.845+0000 7f7ee7fff640 1 -- 172.21.15.164:0/1465968148 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee81a68a0 con 0x7f7ee8101df0 2024-09-19T08:44:28.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.845+0000 7f7ee77fe640 1 --2- 172.21.15.164:0/1465968148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ee81027a0 0x7f7ee8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:44:28.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.845+0000 7f7ee7fff640 1 --2- 172.21.15.164:0/1465968148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ee8101df0 0x7f7ee80744f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800c920 tx=0x7f7ed800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:28.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.845+0000 7f7ee57fa640 1 -- 172.21.15.164:0/1465968148 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8007c70 con 0x7f7ee8101df0 2024-09-19T08:44:28.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.845+0000 7f7ee57fa640 1 -- 172.21.15.164:0/1465968148 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ed8007e10 con 0x7f7ee8101df0 2024-09-19T08:44:28.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.845+0000 7f7ee57fa640 1 -- 172.21.15.164:0/1465968148 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8018890 con 0x7f7ee8101df0 2024-09-19T08:44:28.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.845+0000 7f7eee7a8640 1 -- 172.21.15.164:0/1465968148 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ee81a6b90 con 0x7f7ee8101df0 2024-09-19T08:44:28.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.845+0000 7f7eee7a8640 1 -- 172.21.15.164:0/1465968148 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ee81a7080 con 0x7f7ee8101df0 2024-09-19T08:44:28.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.846+0000 7f7eee7a8640 1 -- 172.21.15.164:0/1465968148 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7ee80628c0 con 0x7f7ee8101df0 2024-09-19T08:44:28.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.847+0000 7f7ee57fa640 1 -- 172.21.15.164:0/1465968148 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f7ed8018a30 con 0x7f7ee8101df0 2024-09-19T08:44:28.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.848+0000 7f7ee57fa640 1 --2- 172.21.15.164:0/1465968148 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f7ed00a32d0 0x7f7ed00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:28.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.848+0000 7f7ee77fe640 1 --2- 172.21.15.164:0/1465968148 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f7ed00a32d0 0x7f7ed00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:28.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.848+0000 7f7ee57fa640 1 -- 172.21.15.164:0/1465968148 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7ed80cffb0 con 0x7f7ee8101df0 2024-09-19T08:44:28.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.848+0000 7f7ee57fa640 1 --2- 172.21.15.164:0/1465968148 >> [v2:172.21.15.173:6808/2914653552,v1:172.21.15.173:6809/2914653552] conn(0x7f7ed00acd80 0x7f7ed00af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:28.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.848+0000 7f7ee57fa640 1 -- 172.21.15.164:0/1465968148 --> [v2:172.21.15.173:6808/2914653552,v1:172.21.15.173:6809/2914653552] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7ed00af8f0 con 0x7f7ed00acd80 2024-09-19T08:44:28.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.848+0000 7f7ee57fa640 1 -- 172.21.15.164:0/1465968148 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f7ed80d0490 con 0x7f7ee8101df0 2024-09-19T08:44:28.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.848+0000 7f7ee77fe640 1 --2- 172.21.15.164:0/1465968148 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f7ed00a32d0 0x7f7ed00a5790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8075f80 tx=0x7f7edc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:28.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.849+0000 7f7eecd45640 1 --2- 172.21.15.164:0/1465968148 >> [v2:172.21.15.173:6808/2914653552,v1:172.21.15.173:6809/2914653552] conn(0x7f7ed00acd80 0x7f7ed00af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:28.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.849+0000 7f7eecd45640 1 --2- 172.21.15.164:0/1465968148 >> [v2:172.21.15.173:6808/2914653552,v1:172.21.15.173:6809/2914653552] conn(0x7f7ed00acd80 0x7f7ed00af220 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:28.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.850+0000 7f7ee57fa640 1 -- 172.21.15.164:0/1465968148 <== osd.5 v2:172.21.15.173:6808/2914653552 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ed00af8f0 con 0x7f7ed00acd80 2024-09-19T08:44:28.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.864+0000 7f7eee7a8640 1 -- 172.21.15.164:0/1465968148 --> [v2:172.21.15.173:6808/2914653552,v1:172.21.15.173:6809/2914653552] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7ee8062d40 con 0x7f7ed00acd80 2024-09-19T08:44:28.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.864+0000 7f7ee57fa640 1 -- 172.21.15.164:0/1465968148 <== osd.5 v2:172.21.15.173:6808/2914653552 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7ee8062d40 con 0x7f7ed00acd80 2024-09-19T08:44:28.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.864+0000 7f7eee7a8640 1 -- 172.21.15.164:0/1465968148 >> [v2:172.21.15.173:6808/2914653552,v1:172.21.15.173:6809/2914653552] conn(0x7f7ed00acd80 msgr2=0x7f7ed00af220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:28.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.865+0000 7f7eee7a8640 1 --2- 172.21.15.164:0/1465968148 >> [v2:172.21.15.173:6808/2914653552,v1:172.21.15.173:6809/2914653552] conn(0x7f7ed00acd80 0x7f7ed00af220 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:28.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.865+0000 7f7eee7a8640 1 -- 172.21.15.164:0/1465968148 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f7ed00a32d0 msgr2=0x7f7ed00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:28.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.865+0000 7f7eee7a8640 1 --2- 172.21.15.164:0/1465968148 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f7ed00a32d0 0x7f7ed00a5790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8075f80 tx=0x7f7edc005a20 comp rx=0 tx=0).stop 2024-09-19T08:44:28.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.865+0000 7f7eee7a8640 1 -- 172.21.15.164:0/1465968148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ee8101df0 msgr2=0x7f7ee80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:28.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.865+0000 7f7eee7a8640 1 --2- 172.21.15.164:0/1465968148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ee8101df0 0x7f7ee80744f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800c920 tx=0x7f7ed800cde0 comp rx=0 tx=0).stop 2024-09-19T08:44:28.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.865+0000 7f7eecd45640 1 -- 172.21.15.164:0/1465968148 reap_dead start 2024-09-19T08:44:28.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.865+0000 7f7eee7a8640 1 -- 172.21.15.164:0/1465968148 shutdown_connections 2024-09-19T08:44:28.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.865+0000 7f7eee7a8640 1 -- 172.21.15.164:0/1465968148 >> 172.21.15.164:0/1465968148 conn(0x7f7ee80fd8b0 msgr2=0x7f7ee80feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:28.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.865+0000 7f7eee7a8640 1 -- 172.21.15.164:0/1465968148 shutdown_connections 2024-09-19T08:44:28.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:28.865+0000 7f7eee7a8640 1 -- 172.21.15.164:0/1465968148 wait complete. 2024-09-19T08:44:29.115 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:29.116 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:29.118 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:29.119 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:29.120 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:29.126 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:29.131 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:29.143 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:29.691 INFO:teuthology.orchestra.run.smithi164.stdout:154618822706 2024-09-19T08:44:29.692 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-19T08:44:30.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.479+0000 7f355936b640 1 -- 172.21.15.164:0/3717485529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35541036c0 msgr2=0x7f355410bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:30.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.479+0000 7f355936b640 1 --2- 172.21.15.164:0/3717485529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35541036c0 0x7f355410bbb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3548009900 tx=0x7f354802a480 comp rx=0 tx=0).stop 2024-09-19T08:44:30.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.479+0000 7f355936b640 1 -- 172.21.15.164:0/3717485529 shutdown_connections 2024-09-19T08:44:30.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.479+0000 7f355936b640 1 --2- 172.21.15.164:0/3717485529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f355410c180 0x7f355410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:30.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.479+0000 7f355936b640 1 --2- 172.21.15.164:0/3717485529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35541036c0 0x7f355410bbb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:30.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.479+0000 7f355936b640 1 --2- 172.21.15.164:0/3717485529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3554102da0 0x7f3554103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:30.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.479+0000 7f355936b640 1 -- 172.21.15.164:0/3717485529 >> 172.21.15.164:0/3717485529 conn(0x7f35540fc7a0 msgr2=0x7f35540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:30.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.479+0000 7f355936b640 1 -- 172.21.15.164:0/3717485529 shutdown_connections 2024-09-19T08:44:30.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.479+0000 7f355936b640 1 -- 172.21.15.164:0/3717485529 wait complete. 2024-09-19T08:44:30.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.480+0000 7f355936b640 1 Processor -- start 2024-09-19T08:44:30.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.480+0000 7f355936b640 1 -- start start 2024-09-19T08:44:30.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.480+0000 7f355936b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3554102da0 0x7f3554100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:30.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.480+0000 7f355936b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35541036c0 0x7f3554100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:30.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.480+0000 7f355936b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f355410c180 0x7f3554101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:30.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.480+0000 7f355936b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3554111f60 con 0x7f355410c180 2024-09-19T08:44:30.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.480+0000 7f355936b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3554111de0 con 0x7f35541036c0 2024-09-19T08:44:30.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.480+0000 7f355936b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f35541120e0 con 0x7f3554102da0 2024-09-19T08:44:30.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.480+0000 7f35537fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f355410c180 0x7f3554101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:30.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.480+0000 7f35537fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f355410c180 0x7f3554101480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33044/0 (socket says 172.21.15.164:33044) 2024-09-19T08:44:30.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.480+0000 7f35537fe640 1 -- 172.21.15.164:0/2032174867 learned_addr learned my addr 172.21.15.164:0/2032174867 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:30.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.480+0000 7f35527fc640 1 --2- 172.21.15.164:0/2032174867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35541036c0 0x7f3554100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:30.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.481+0000 7f3552ffd640 1 --2- 172.21.15.164:0/2032174867 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3554102da0 0x7f3554100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:30.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.481+0000 7f35537fe640 1 -- 172.21.15.164:0/2032174867 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3554102da0 msgr2=0x7f3554100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:30.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.481+0000 7f35537fe640 1 --2- 172.21.15.164:0/2032174867 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3554102da0 0x7f3554100a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:30.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.481+0000 7f35537fe640 1 -- 172.21.15.164:0/2032174867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35541036c0 msgr2=0x7f3554100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:30.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.481+0000 7f35537fe640 1 --2- 172.21.15.164:0/2032174867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35541036c0 0x7f3554100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:30.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.481+0000 7f35537fe640 1 -- 172.21.15.164:0/2032174867 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35541a68e0 con 0x7f355410c180 2024-09-19T08:44:30.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.481+0000 7f35527fc640 1 --2- 172.21.15.164:0/2032174867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35541036c0 0x7f3554100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:44:30.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.481+0000 7f3552ffd640 1 --2- 172.21.15.164:0/2032174867 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3554102da0 0x7f3554100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:30.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.481+0000 7f35537fe640 1 --2- 172.21.15.164:0/2032174867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f355410c180 0x7f3554101480 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f354c00d750 tx=0x7f354c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:30.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.481+0000 7f353bfff640 1 -- 172.21.15.164:0/2032174867 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354c017070 con 0x7f355410c180 2024-09-19T08:44:30.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.481+0000 7f355936b640 1 -- 172.21.15.164:0/2032174867 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35541a6bd0 con 0x7f355410c180 2024-09-19T08:44:30.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.481+0000 7f353bfff640 1 -- 172.21.15.164:0/2032174867 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f354c005500 con 0x7f355410c180 2024-09-19T08:44:30.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.481+0000 7f353bfff640 1 -- 172.21.15.164:0/2032174867 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354c004d90 con 0x7f355410c180 2024-09-19T08:44:30.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.481+0000 7f355936b640 1 -- 172.21.15.164:0/2032174867 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35541a7110 con 0x7f355410c180 2024-09-19T08:44:30.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.482+0000 7f355936b640 1 -- 172.21.15.164:0/2032174867 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f35540628c0 con 0x7f355410c180 2024-09-19T08:44:30.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.483+0000 7f353bfff640 1 -- 172.21.15.164:0/2032174867 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f354c004f30 con 0x7f355410c180 2024-09-19T08:44:30.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.484+0000 7f353bfff640 1 --2- 172.21.15.164:0/2032174867 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f353c0a32d0 0x7f353c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:30.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.484+0000 7f3552ffd640 1 --2- 172.21.15.164:0/2032174867 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f353c0a32d0 0x7f353c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:30.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.484+0000 7f353bfff640 1 -- 172.21.15.164:0/2032174867 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f354c0cf7a0 con 0x7f355410c180 2024-09-19T08:44:30.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.484+0000 7f353bfff640 1 --2- 172.21.15.164:0/2032174867 >> [v2:172.21.15.170:6800/2167066371,v1:172.21.15.170:6801/2167066371] conn(0x7f353c0acd80 0x7f353c0af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:30.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.484+0000 7f353bfff640 1 -- 172.21.15.164:0/2032174867 --> [v2:172.21.15.170:6800/2167066371,v1:172.21.15.170:6801/2167066371] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f354c001d30 con 0x7f353c0acd80 2024-09-19T08:44:30.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.484+0000 7f353bfff640 1 -- 172.21.15.164:0/2032174867 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f354c0cfc80 con 0x7f355410c180 2024-09-19T08:44:30.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.484+0000 7f3552ffd640 1 --2- 172.21.15.164:0/2032174867 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f353c0a32d0 0x7f353c0a5790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3544005c40 tx=0x7f35440082f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:30.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.484+0000 7f35527fc640 1 --2- 172.21.15.164:0/2032174867 >> [v2:172.21.15.170:6800/2167066371,v1:172.21.15.170:6801/2167066371] conn(0x7f353c0acd80 0x7f353c0af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:30.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.485+0000 7f35527fc640 1 --2- 172.21.15.164:0/2032174867 >> [v2:172.21.15.170:6800/2167066371,v1:172.21.15.170:6801/2167066371] conn(0x7f353c0acd80 0x7f353c0af220 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:30.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.486+0000 7f353bfff640 1 -- 172.21.15.164:0/2032174867 <== osd.1 v2:172.21.15.170:6800/2167066371 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f354c001d30 con 0x7f353c0acd80 2024-09-19T08:44:30.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.500+0000 7f355936b640 1 -- 172.21.15.164:0/2032174867 --> [v2:172.21.15.170:6800/2167066371,v1:172.21.15.170:6801/2167066371] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3554062d40 con 0x7f353c0acd80 2024-09-19T08:44:30.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.501+0000 7f353bfff640 1 -- 172.21.15.164:0/2032174867 <== osd.1 v2:172.21.15.170:6800/2167066371 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3554062d40 con 0x7f353c0acd80 2024-09-19T08:44:30.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.501+0000 7f355936b640 1 -- 172.21.15.164:0/2032174867 >> [v2:172.21.15.170:6800/2167066371,v1:172.21.15.170:6801/2167066371] conn(0x7f353c0acd80 msgr2=0x7f353c0af220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:30.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.501+0000 7f355936b640 1 --2- 172.21.15.164:0/2032174867 >> [v2:172.21.15.170:6800/2167066371,v1:172.21.15.170:6801/2167066371] conn(0x7f353c0acd80 0x7f353c0af220 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:30.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.501+0000 7f355936b640 1 -- 172.21.15.164:0/2032174867 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f353c0a32d0 msgr2=0x7f353c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:30.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.501+0000 7f355936b640 1 --2- 172.21.15.164:0/2032174867 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f353c0a32d0 0x7f353c0a5790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3544005c40 tx=0x7f35440082f0 comp rx=0 tx=0).stop 2024-09-19T08:44:30.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.501+0000 7f355936b640 1 -- 172.21.15.164:0/2032174867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f355410c180 msgr2=0x7f3554101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:30.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.501+0000 7f355936b640 1 --2- 172.21.15.164:0/2032174867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f355410c180 0x7f3554101480 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f354c00d750 tx=0x7f354c00dc10 comp rx=0 tx=0).stop 2024-09-19T08:44:30.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.501+0000 7f35537fe640 1 -- 172.21.15.164:0/2032174867 reap_dead start 2024-09-19T08:44:30.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.501+0000 7f355936b640 1 -- 172.21.15.164:0/2032174867 shutdown_connections 2024-09-19T08:44:30.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.501+0000 7f355936b640 1 -- 172.21.15.164:0/2032174867 >> 172.21.15.164:0/2032174867 conn(0x7f35540fc7a0 msgr2=0x7f35540fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:30.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.501+0000 7f355936b640 1 -- 172.21.15.164:0/2032174867 shutdown_connections 2024-09-19T08:44:30.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:30.501+0000 7f355936b640 1 -- 172.21.15.164:0/2032174867 wait complete. 2024-09-19T08:44:30.943 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:31.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:30 smithi164 ceph-mon[30928]: pgmap v296: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-19T08:44:31.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:30 smithi170 ceph-mon[35072]: pgmap v296: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-19T08:44:31.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:30 smithi173 ceph-mon[35306]: pgmap v296: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-19T08:44:31.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.623+0000 7fea26baf640 1 -- 172.21.15.164:0/1108935831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea20101420 msgr2=0x7fea2010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:31.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.623+0000 7fea26baf640 1 --2- 172.21.15.164:0/1108935831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea20101420 0x7fea2010d950 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fea18009cd0 tx=0x7fea1802a8e0 comp rx=0 tx=0).stop 2024-09-19T08:44:31.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.623+0000 7fea26baf640 1 -- 172.21.15.164:0/1108935831 shutdown_connections 2024-09-19T08:44:31.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.623+0000 7fea26baf640 1 --2- 172.21.15.164:0/1108935831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea20101420 0x7fea2010d950 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:31.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.623+0000 7fea26baf640 1 --2- 172.21.15.164:0/1108935831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea20100a80 0x7fea20100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:31.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.623+0000 7fea26baf640 1 --2- 172.21.15.164:0/1108935831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea20106a80 0x7fea20106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:31.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.623+0000 7fea26baf640 1 -- 172.21.15.164:0/1108935831 >> 172.21.15.164:0/1108935831 conn(0x7fea200fc7a0 msgr2=0x7fea200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:31.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.623+0000 7fea26baf640 1 -- 172.21.15.164:0/1108935831 shutdown_connections 2024-09-19T08:44:31.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.623+0000 7fea26baf640 1 -- 172.21.15.164:0/1108935831 wait complete. 2024-09-19T08:44:31.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.624+0000 7fea26baf640 1 Processor -- start 2024-09-19T08:44:31.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.624+0000 7fea26baf640 1 -- start start 2024-09-19T08:44:31.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.624+0000 7fea26baf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea20100a80 0x7fea20198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:31.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.624+0000 7fea26baf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea20101420 0x7fea20198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:31.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.624+0000 7fea26baf640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea20106a80 0x7fea2019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:31.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.624+0000 7fea26baf640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fea20110f20 con 0x7fea20100a80 2024-09-19T08:44:31.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.624+0000 7fea26baf640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fea20110da0 con 0x7fea20101420 2024-09-19T08:44:31.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.624+0000 7fea26baf640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fea201110a0 con 0x7fea20106a80 2024-09-19T08:44:31.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.624+0000 7fea2494b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea20100a80 0x7fea20198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:31.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.624+0000 7fea2494b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea20100a80 0x7fea20198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33072/0 (socket says 172.21.15.164:33072) 2024-09-19T08:44:31.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.624+0000 7fea2494b640 1 -- 172.21.15.164:0/687879163 learned_addr learned my addr 172.21.15.164:0/687879163 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:31.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.625+0000 7fea2494b640 1 -- 172.21.15.164:0/687879163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea20106a80 msgr2=0x7fea2019ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:31.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.625+0000 7fea2494b640 1 --2- 172.21.15.164:0/687879163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea20106a80 0x7fea2019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:31.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.625+0000 7fea1ffff640 1 --2- 172.21.15.164:0/687879163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea20101420 0x7fea20198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:31.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.625+0000 7fea2494b640 1 -- 172.21.15.164:0/687879163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea20101420 msgr2=0x7fea20198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:31.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.625+0000 7fea2494b640 1 --2- 172.21.15.164:0/687879163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea20101420 0x7fea20198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:31.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.625+0000 7fea2494b640 1 -- 172.21.15.164:0/687879163 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea2019d4f0 con 0x7fea20100a80 2024-09-19T08:44:31.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.625+0000 7fea2494b640 1 --2- 172.21.15.164:0/687879163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea20100a80 0x7fea20198540 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fea1000b4a0 tx=0x7fea1000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:31.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.625+0000 7fea1dffb640 1 -- 172.21.15.164:0/687879163 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea10014070 con 0x7fea20100a80 2024-09-19T08:44:31.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.625+0000 7fea26baf640 1 -- 172.21.15.164:0/687879163 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea2019d7e0 con 0x7fea20100a80 2024-09-19T08:44:31.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.625+0000 7fea1dffb640 1 -- 172.21.15.164:0/687879163 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea10010af0 con 0x7fea20100a80 2024-09-19T08:44:31.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.625+0000 7fea1dffb640 1 -- 172.21.15.164:0/687879163 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea10017d10 con 0x7fea20100a80 2024-09-19T08:44:31.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.625+0000 7fea26baf640 1 -- 172.21.15.164:0/687879163 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea201a50c0 con 0x7fea20100a80 2024-09-19T08:44:31.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.626+0000 7fea26baf640 1 -- 172.21.15.164:0/687879163 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fea20062980 con 0x7fea20100a80 2024-09-19T08:44:31.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.627+0000 7fea1dffb640 1 -- 172.21.15.164:0/687879163 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fea10017070 con 0x7fea20100a80 2024-09-19T08:44:31.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.628+0000 7fea1dffb640 1 --2- 172.21.15.164:0/687879163 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fea080a32d0 0x7fea080a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:31.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.628+0000 7fea1ffff640 1 --2- 172.21.15.164:0/687879163 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fea080a32d0 0x7fea080a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:31.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.628+0000 7fea1dffb640 1 -- 172.21.15.164:0/687879163 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fea100cf0e0 con 0x7fea20100a80 2024-09-19T08:44:31.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.628+0000 7fea1dffb640 1 --2- 172.21.15.164:0/687879163 >> [v2:172.21.15.164:6810/406795560,v1:172.21.15.164:6811/406795560] conn(0x7fea080acd80 0x7fea080af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:31.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.628+0000 7fea1dffb640 1 -- 172.21.15.164:0/687879163 --> [v2:172.21.15.164:6810/406795560,v1:172.21.15.164:6811/406795560] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fea080af8f0 con 0x7fea080acd80 2024-09-19T08:44:31.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.628+0000 7fea1dffb640 1 -- 172.21.15.164:0/687879163 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fea100cf5c0 con 0x7fea20100a80 2024-09-19T08:44:31.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.628+0000 7fea2514c640 1 --2- 172.21.15.164:0/687879163 >> [v2:172.21.15.164:6810/406795560,v1:172.21.15.164:6811/406795560] conn(0x7fea080acd80 0x7fea080af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:31.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.628+0000 7fea1ffff640 1 --2- 172.21.15.164:0/687879163 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fea080a32d0 0x7fea080a5790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fea200ff200 tx=0x7fea14008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:31.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.628+0000 7fea2514c640 1 --2- 172.21.15.164:0/687879163 >> [v2:172.21.15.164:6810/406795560,v1:172.21.15.164:6811/406795560] conn(0x7fea080acd80 0x7fea080af220 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:31.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.629+0000 7fea1dffb640 1 -- 172.21.15.164:0/687879163 <== osd.3 v2:172.21.15.164:6810/406795560 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fea080af8f0 con 0x7fea080acd80 2024-09-19T08:44:31.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.642+0000 7fea26baf640 1 -- 172.21.15.164:0/687879163 --> [v2:172.21.15.164:6810/406795560,v1:172.21.15.164:6811/406795560] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fea20062e00 con 0x7fea080acd80 2024-09-19T08:44:31.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.643+0000 7fea1dffb640 1 -- 172.21.15.164:0/687879163 <== osd.3 v2:172.21.15.164:6810/406795560 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fea20062e00 con 0x7fea080acd80 2024-09-19T08:44:31.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.643+0000 7fea26baf640 1 -- 172.21.15.164:0/687879163 >> [v2:172.21.15.164:6810/406795560,v1:172.21.15.164:6811/406795560] conn(0x7fea080acd80 msgr2=0x7fea080af220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:31.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.643+0000 7fea26baf640 1 --2- 172.21.15.164:0/687879163 >> [v2:172.21.15.164:6810/406795560,v1:172.21.15.164:6811/406795560] conn(0x7fea080acd80 0x7fea080af220 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:31.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.643+0000 7fea26baf640 1 -- 172.21.15.164:0/687879163 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fea080a32d0 msgr2=0x7fea080a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:31.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.643+0000 7fea26baf640 1 --2- 172.21.15.164:0/687879163 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fea080a32d0 0x7fea080a5790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fea200ff200 tx=0x7fea14008040 comp rx=0 tx=0).stop 2024-09-19T08:44:31.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.643+0000 7fea26baf640 1 -- 172.21.15.164:0/687879163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea20100a80 msgr2=0x7fea20198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:31.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.643+0000 7fea26baf640 1 --2- 172.21.15.164:0/687879163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea20100a80 0x7fea20198540 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fea1000b4a0 tx=0x7fea1000b960 comp rx=0 tx=0).stop 2024-09-19T08:44:31.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.643+0000 7fea2514c640 1 -- 172.21.15.164:0/687879163 reap_dead start 2024-09-19T08:44:31.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.643+0000 7fea26baf640 1 -- 172.21.15.164:0/687879163 shutdown_connections 2024-09-19T08:44:31.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.643+0000 7fea26baf640 1 -- 172.21.15.164:0/687879163 >> 172.21.15.164:0/687879163 conn(0x7fea200fc7a0 msgr2=0x7fea2006aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:31.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.643+0000 7fea26baf640 1 -- 172.21.15.164:0/687879163 shutdown_connections 2024-09-19T08:44:31.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:31.643+0000 7fea26baf640 1 -- 172.21.15.164:0/687879163 wait complete. 2024-09-19T08:44:31.850 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:32.384 INFO:teuthology.orchestra.run.smithi164.stdout:103079215165 2024-09-19T08:44:32.384 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-19T08:44:32.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.542+0000 7f7e1e22b640 1 -- 172.21.15.164:0/1451089716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e18102ef0 msgr2=0x7f7e1810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:32.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.542+0000 7f7e1e22b640 1 --2- 172.21.15.164:0/1451089716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e18102ef0 0x7f7e1810da40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009ca0 tx=0x7f7e1002a950 comp rx=0 tx=0).stop 2024-09-19T08:44:32.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.542+0000 7f7e1e22b640 1 -- 172.21.15.164:0/1451089716 shutdown_connections 2024-09-19T08:44:32.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.542+0000 7f7e1e22b640 1 --2- 172.21.15.164:0/1451089716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e18102ef0 0x7f7e1810da40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:32.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.542+0000 7f7e1e22b640 1 --2- 172.21.15.164:0/1451089716 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e18102570 0x7f7e181029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:32.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.542+0000 7f7e1e22b640 1 --2- 172.21.15.164:0/1451089716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e18100e20 0x7f7e18101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:32.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.542+0000 7f7e1e22b640 1 -- 172.21.15.164:0/1451089716 >> 172.21.15.164:0/1451089716 conn(0x7f7e180fc7d0 msgr2=0x7f7e180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:32.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.542+0000 7f7e1e22b640 1 -- 172.21.15.164:0/1451089716 shutdown_connections 2024-09-19T08:44:32.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.542+0000 7f7e1e22b640 1 -- 172.21.15.164:0/1451089716 wait complete. 2024-09-19T08:44:32.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.543+0000 7f7e1e22b640 1 Processor -- start 2024-09-19T08:44:32.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.543+0000 7f7e1e22b640 1 -- start start 2024-09-19T08:44:32.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.543+0000 7f7e1e22b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e18100e20 0x7f7e181970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:32.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.543+0000 7f7e1e22b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e18102570 0x7f7e18197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:32.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.543+0000 7f7e1e22b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e18102ef0 0x7f7e18197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:32.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.543+0000 7f7e1e22b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7e18110f80 con 0x7f7e18102570 2024-09-19T08:44:32.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.543+0000 7f7e1e22b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7e18110e00 con 0x7f7e18100e20 2024-09-19T08:44:32.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.543+0000 7f7e1e22b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7e18111100 con 0x7f7e18102ef0 2024-09-19T08:44:32.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.543+0000 7f7e16ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e18102570 0x7f7e18197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:32.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.543+0000 7f7e16ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e18102570 0x7f7e18197620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33094/0 (socket says 172.21.15.164:33094) 2024-09-19T08:44:32.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.543+0000 7f7e16ffd640 1 -- 172.21.15.164:0/3514301039 learned_addr learned my addr 172.21.15.164:0/3514301039 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:32.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.544+0000 7f7e177fe640 1 --2- 172.21.15.164:0/3514301039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e18100e20 0x7f7e181970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:32.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.544+0000 7f7e17fff640 1 --2- 172.21.15.164:0/3514301039 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e18102ef0 0x7f7e18197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:32.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.544+0000 7f7e16ffd640 1 -- 172.21.15.164:0/3514301039 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e18102ef0 msgr2=0x7f7e18197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:32.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.544+0000 7f7e16ffd640 1 --2- 172.21.15.164:0/3514301039 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e18102ef0 0x7f7e18197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:32.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.544+0000 7f7e16ffd640 1 -- 172.21.15.164:0/3514301039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e18100e20 msgr2=0x7f7e181970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:32.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.544+0000 7f7e16ffd640 1 --2- 172.21.15.164:0/3514301039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e18100e20 0x7f7e181970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:32.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.544+0000 7f7e16ffd640 1 -- 172.21.15.164:0/3514301039 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e181a6950 con 0x7f7e18102570 2024-09-19T08:44:32.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.544+0000 7f7e16ffd640 1 --2- 172.21.15.164:0/3514301039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e18102570 0x7f7e18197620 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c00c970 tx=0x7f7e0c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:32.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.544+0000 7f7e14ff9640 1 -- 172.21.15.164:0/3514301039 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c007bb0 con 0x7f7e18102570 2024-09-19T08:44:32.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.544+0000 7f7e14ff9640 1 -- 172.21.15.164:0/3514301039 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e0c007d50 con 0x7f7e18102570 2024-09-19T08:44:32.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.544+0000 7f7e1e22b640 1 -- 172.21.15.164:0/3514301039 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e181a6c40 con 0x7f7e18102570 2024-09-19T08:44:32.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.544+0000 7f7e14ff9640 1 -- 172.21.15.164:0/3514301039 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c00f450 con 0x7f7e18102570 2024-09-19T08:44:32.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.544+0000 7f7e1e22b640 1 -- 172.21.15.164:0/3514301039 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e181a7180 con 0x7f7e18102570 2024-09-19T08:44:32.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.545+0000 7f7e1e22b640 1 -- 172.21.15.164:0/3514301039 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7e18107260 con 0x7f7e18102570 2024-09-19T08:44:32.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.547+0000 7f7e14ff9640 1 -- 172.21.15.164:0/3514301039 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f7e0c00f5f0 con 0x7f7e18102570 2024-09-19T08:44:32.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.548+0000 7f7e14ff9640 1 --2- 172.21.15.164:0/3514301039 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f7e000a3350 0x7f7e000a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:32.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.548+0000 7f7e177fe640 1 --2- 172.21.15.164:0/3514301039 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f7e000a3350 0x7f7e000a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:32.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.548+0000 7f7e14ff9640 1 -- 172.21.15.164:0/3514301039 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7e0c0cfbe0 con 0x7f7e18102570 2024-09-19T08:44:32.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.548+0000 7f7e14ff9640 1 --2- 172.21.15.164:0/3514301039 >> [v2:172.21.15.173:6800/1646561048,v1:172.21.15.173:6801/1646561048] conn(0x7f7e000ace00 0x7f7e000af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:32.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.548+0000 7f7e14ff9640 1 -- 172.21.15.164:0/3514301039 --> [v2:172.21.15.173:6800/1646561048,v1:172.21.15.173:6801/1646561048] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7e0c01de50 con 0x7f7e000ace00 2024-09-19T08:44:32.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.548+0000 7f7e14ff9640 1 -- 172.21.15.164:0/3514301039 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f7e0c0d00c0 con 0x7f7e18102570 2024-09-19T08:44:32.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.548+0000 7f7e177fe640 1 --2- 172.21.15.164:0/3514301039 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f7e000a3350 0x7f7e000a5810 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7e080058c0 tx=0x7f7e0800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:32.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.549+0000 7f7e17fff640 1 --2- 172.21.15.164:0/3514301039 >> [v2:172.21.15.173:6800/1646561048,v1:172.21.15.173:6801/1646561048] conn(0x7f7e000ace00 0x7f7e000af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:32.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.549+0000 7f7e17fff640 1 --2- 172.21.15.164:0/3514301039 >> [v2:172.21.15.173:6800/1646561048,v1:172.21.15.173:6801/1646561048] conn(0x7f7e000ace00 0x7f7e000af2a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:32.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.550+0000 7f7e14ff9640 1 -- 172.21.15.164:0/3514301039 <== osd.2 v2:172.21.15.173:6800/1646561048 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7e0c01de50 con 0x7f7e000ace00 2024-09-19T08:44:32.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.563+0000 7f7e1e22b640 1 -- 172.21.15.164:0/3514301039 --> [v2:172.21.15.173:6800/1646561048,v1:172.21.15.173:6801/1646561048] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7e181021b0 con 0x7f7e000ace00 2024-09-19T08:44:32.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.564+0000 7f7e14ff9640 1 -- 172.21.15.164:0/3514301039 <== osd.2 v2:172.21.15.173:6800/1646561048 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7e181021b0 con 0x7f7e000ace00 2024-09-19T08:44:32.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.564+0000 7f7e1e22b640 1 -- 172.21.15.164:0/3514301039 >> [v2:172.21.15.173:6800/1646561048,v1:172.21.15.173:6801/1646561048] conn(0x7f7e000ace00 msgr2=0x7f7e000af2a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:32.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.564+0000 7f7e1e22b640 1 --2- 172.21.15.164:0/3514301039 >> [v2:172.21.15.173:6800/1646561048,v1:172.21.15.173:6801/1646561048] conn(0x7f7e000ace00 0x7f7e000af2a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:32.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.564+0000 7f7e1e22b640 1 -- 172.21.15.164:0/3514301039 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f7e000a3350 msgr2=0x7f7e000a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:32.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.564+0000 7f7e1e22b640 1 --2- 172.21.15.164:0/3514301039 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f7e000a3350 0x7f7e000a5810 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7e080058c0 tx=0x7f7e0800a2d0 comp rx=0 tx=0).stop 2024-09-19T08:44:32.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.564+0000 7f7e1e22b640 1 -- 172.21.15.164:0/3514301039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e18102570 msgr2=0x7f7e18197620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:32.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.564+0000 7f7e1e22b640 1 --2- 172.21.15.164:0/3514301039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e18102570 0x7f7e18197620 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c00c970 tx=0x7f7e0c00ce30 comp rx=0 tx=0).stop 2024-09-19T08:44:32.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.564+0000 7f7e17fff640 1 -- 172.21.15.164:0/3514301039 reap_dead start 2024-09-19T08:44:32.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.564+0000 7f7e1e22b640 1 -- 172.21.15.164:0/3514301039 shutdown_connections 2024-09-19T08:44:32.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.564+0000 7f7e1e22b640 1 -- 172.21.15.164:0/3514301039 >> 172.21.15.164:0/3514301039 conn(0x7f7e180fc7d0 msgr2=0x7f7e18106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:32.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.565+0000 7f7e1e22b640 1 -- 172.21.15.164:0/3514301039 shutdown_connections 2024-09-19T08:44:32.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.565+0000 7f7e1e22b640 1 -- 172.21.15.164:0/3514301039 wait complete. 2024-09-19T08:44:32.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.752+0000 7f20db9b4640 1 -- 172.21.15.164:0/609413219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20d4108290 msgr2=0x7f20d410ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:32.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.752+0000 7f20db9b4640 1 --2- 172.21.15.164:0/609413219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20d4108290 0x7f20d410ec30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f20cc009930 tx=0x7f20cc02a420 comp rx=0 tx=0).stop 2024-09-19T08:44:32.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.752+0000 7f20db9b4640 1 -- 172.21.15.164:0/609413219 shutdown_connections 2024-09-19T08:44:32.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.752+0000 7f20db9b4640 1 --2- 172.21.15.164:0/609413219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20d4108290 0x7f20d410ec30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:32.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.752+0000 7f20db9b4640 1 --2- 172.21.15.164:0/609413219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20d4103740 0x7f20d4107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:32.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.752+0000 7f20db9b4640 1 --2- 172.21.15.164:0/609413219 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20d4102d90 0x7f20d4103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:32.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.752+0000 7f20db9b4640 1 -- 172.21.15.164:0/609413219 >> 172.21.15.164:0/609413219 conn(0x7f20d40fc7d0 msgr2=0x7f20d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:32.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.752+0000 7f20db9b4640 1 -- 172.21.15.164:0/609413219 shutdown_connections 2024-09-19T08:44:32.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.752+0000 7f20db9b4640 1 -- 172.21.15.164:0/609413219 wait complete. 2024-09-19T08:44:32.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.753+0000 7f20db9b4640 1 Processor -- start 2024-09-19T08:44:32.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.753+0000 7f20db9b4640 1 -- start start 2024-09-19T08:44:32.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.753+0000 7f20db9b4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20d4102d90 0x7f20d41009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:32.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.753+0000 7f20db9b4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20d4103740 0x7f20d4100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:32.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.753+0000 7f20db9b4640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20d4108290 0x7f20d4101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:32.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.753+0000 7f20db9b4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f20d4112090 con 0x7f20d4102d90 2024-09-19T08:44:32.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.753+0000 7f20db9b4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f20d4111f10 con 0x7f20d4103740 2024-09-19T08:44:32.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.753+0000 7f20db9b4640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f20d4112210 con 0x7f20d4108290 2024-09-19T08:44:32.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.753+0000 7f20d9750640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20d4102d90 0x7f20d41009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:32.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.753+0000 7f20d9750640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20d4102d90 0x7f20d41009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33120/0 (socket says 172.21.15.164:33120) 2024-09-19T08:44:32.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.753+0000 7f20d9750640 1 -- 172.21.15.164:0/1246588575 learned_addr learned my addr 172.21.15.164:0/1246588575 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:32.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.754+0000 7f20d9f51640 1 --2- 172.21.15.164:0/1246588575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20d4108290 0x7f20d4101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:32.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.754+0000 7f20d8f4f640 1 --2- 172.21.15.164:0/1246588575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20d4103740 0x7f20d4100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:32.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.754+0000 7f20d9750640 1 -- 172.21.15.164:0/1246588575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20d4108290 msgr2=0x7f20d4101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:32.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.754+0000 7f20d9750640 1 --2- 172.21.15.164:0/1246588575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20d4108290 0x7f20d4101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:32.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.754+0000 7f20d9750640 1 -- 172.21.15.164:0/1246588575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20d4103740 msgr2=0x7f20d4100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:32.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.754+0000 7f20d9750640 1 --2- 172.21.15.164:0/1246588575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20d4103740 0x7f20d4100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:32.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.754+0000 7f20d9750640 1 -- 172.21.15.164:0/1246588575 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d41a6a30 con 0x7f20d4102d90 2024-09-19T08:44:32.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.754+0000 7f20d9f51640 1 --2- 172.21.15.164:0/1246588575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20d4108290 0x7f20d4101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:44:32.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.754+0000 7f20d8f4f640 1 --2- 172.21.15.164:0/1246588575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20d4103740 0x7f20d4100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:44:32.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.754+0000 7f20d9750640 1 --2- 172.21.15.164:0/1246588575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20d4102d90 0x7f20d41009f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f20c400cce0 tx=0x7f20c400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:32.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.754+0000 7f20d27fc640 1 -- 172.21.15.164:0/1246588575 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c400f650 con 0x7f20d4102d90 2024-09-19T08:44:32.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.754+0000 7f20d27fc640 1 -- 172.21.15.164:0/1246588575 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20c400fcd0 con 0x7f20d4102d90 2024-09-19T08:44:32.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.754+0000 7f20db9b4640 1 -- 172.21.15.164:0/1246588575 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d41a6d20 con 0x7f20d4102d90 2024-09-19T08:44:32.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.754+0000 7f20d27fc640 1 -- 172.21.15.164:0/1246588575 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4018c70 con 0x7f20d4102d90 2024-09-19T08:44:32.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.754+0000 7f20db9b4640 1 -- 172.21.15.164:0/1246588575 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d41a7200 con 0x7f20d4102d90 2024-09-19T08:44:32.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.755+0000 7f20db9b4640 1 -- 172.21.15.164:0/1246588575 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f20d4062880 con 0x7f20d4102d90 2024-09-19T08:44:32.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.756+0000 7f20d27fc640 1 -- 172.21.15.164:0/1246588575 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f20c400f7f0 con 0x7f20d4102d90 2024-09-19T08:44:32.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.757+0000 7f20d27fc640 1 --2- 172.21.15.164:0/1246588575 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f20bc0a32d0 0x7f20bc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:32.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.757+0000 7f20d8f4f640 1 --2- 172.21.15.164:0/1246588575 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f20bc0a32d0 0x7f20bc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:32.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.757+0000 7f20d27fc640 1 -- 172.21.15.164:0/1246588575 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f20c40d0320 con 0x7f20d4102d90 2024-09-19T08:44:32.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.757+0000 7f20d27fc640 1 --2- 172.21.15.164:0/1246588575 >> [v2:172.21.15.164:6818/1138341393,v1:172.21.15.164:6819/1138341393] conn(0x7f20bc0acd80 0x7f20bc0af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:32.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.757+0000 7f20d27fc640 1 -- 172.21.15.164:0/1246588575 --> [v2:172.21.15.164:6818/1138341393,v1:172.21.15.164:6819/1138341393] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f20bc0af8f0 con 0x7f20bc0acd80 2024-09-19T08:44:32.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.757+0000 7f20d27fc640 1 -- 172.21.15.164:0/1246588575 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f20c40d0800 con 0x7f20d4102d90 2024-09-19T08:44:32.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.757+0000 7f20d9f51640 1 --2- 172.21.15.164:0/1246588575 >> [v2:172.21.15.164:6818/1138341393,v1:172.21.15.164:6819/1138341393] conn(0x7f20bc0acd80 0x7f20bc0af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:32.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.758+0000 7f20d8f4f640 1 --2- 172.21.15.164:0/1246588575 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f20bc0a32d0 0x7f20bc0a5790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f20c8005fd0 tx=0x7f20c800b5d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:32.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.758+0000 7f20d9f51640 1 --2- 172.21.15.164:0/1246588575 >> [v2:172.21.15.164:6818/1138341393,v1:172.21.15.164:6819/1138341393] conn(0x7f20bc0acd80 0x7f20bc0af220 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:32.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.759+0000 7f20d27fc640 1 -- 172.21.15.164:0/1246588575 <== osd.6 v2:172.21.15.164:6818/1138341393 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f20bc0af8f0 con 0x7f20bc0acd80 2024-09-19T08:44:32.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.774+0000 7f20db9b4640 1 -- 172.21.15.164:0/1246588575 --> [v2:172.21.15.164:6818/1138341393,v1:172.21.15.164:6819/1138341393] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f20d4062c80 con 0x7f20bc0acd80 2024-09-19T08:44:32.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.774+0000 7f20d27fc640 1 -- 172.21.15.164:0/1246588575 <== osd.6 v2:172.21.15.164:6818/1138341393 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f20d4062c80 con 0x7f20bc0acd80 2024-09-19T08:44:32.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.775+0000 7f20db9b4640 1 -- 172.21.15.164:0/1246588575 >> [v2:172.21.15.164:6818/1138341393,v1:172.21.15.164:6819/1138341393] conn(0x7f20bc0acd80 msgr2=0x7f20bc0af220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:32.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.775+0000 7f20db9b4640 1 --2- 172.21.15.164:0/1246588575 >> [v2:172.21.15.164:6818/1138341393,v1:172.21.15.164:6819/1138341393] conn(0x7f20bc0acd80 0x7f20bc0af220 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:32.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.775+0000 7f20db9b4640 1 -- 172.21.15.164:0/1246588575 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f20bc0a32d0 msgr2=0x7f20bc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:32.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.775+0000 7f20db9b4640 1 --2- 172.21.15.164:0/1246588575 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f20bc0a32d0 0x7f20bc0a5790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f20c8005fd0 tx=0x7f20c800b5d0 comp rx=0 tx=0).stop 2024-09-19T08:44:32.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.775+0000 7f20db9b4640 1 -- 172.21.15.164:0/1246588575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20d4102d90 msgr2=0x7f20d41009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:32.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.775+0000 7f20db9b4640 1 --2- 172.21.15.164:0/1246588575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f20d4102d90 0x7f20d41009f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f20c400cce0 tx=0x7f20c400a620 comp rx=0 tx=0).stop 2024-09-19T08:44:32.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.775+0000 7f20d9f51640 1 -- 172.21.15.164:0/1246588575 reap_dead start 2024-09-19T08:44:32.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.775+0000 7f20db9b4640 1 -- 172.21.15.164:0/1246588575 shutdown_connections 2024-09-19T08:44:32.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.775+0000 7f20db9b4640 1 -- 172.21.15.164:0/1246588575 >> 172.21.15.164:0/1246588575 conn(0x7f20d40fc7d0 msgr2=0x7f20d4106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:32.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.775+0000 7f20db9b4640 1 -- 172.21.15.164:0/1246588575 shutdown_connections 2024-09-19T08:44:32.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:32.775+0000 7f20db9b4640 1 -- 172.21.15.164:0/1246588575 wait complete. 2024-09-19T08:44:33.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:32 smithi164 ceph-mon[30928]: pgmap v297: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-19T08:44:33.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:32 smithi170 ceph-mon[35072]: pgmap v297: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-19T08:44:33.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:32 smithi173 ceph-mon[35306]: pgmap v297: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-19T08:44:33.839 INFO:teuthology.orchestra.run.smithi164.stdout:184683593775 2024-09-19T08:44:33.840 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-19T08:44:34.292 INFO:teuthology.orchestra.run.smithi164.stdout:55834574920 2024-09-19T08:44:34.292 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-19T08:44:34.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.828+0000 7f86c9715640 1 -- 172.21.15.164:0/486291556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86c4102570 msgr2=0x7f86c41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:34.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.828+0000 7f86c9715640 1 --2- 172.21.15.164:0/486291556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86c4102570 0x7f86c41029b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f86ac009900 tx=0x7f86ac02a480 comp rx=0 tx=0).stop 2024-09-19T08:44:34.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.829+0000 7f86c9715640 1 -- 172.21.15.164:0/486291556 shutdown_connections 2024-09-19T08:44:34.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.829+0000 7f86c9715640 1 --2- 172.21.15.164:0/486291556 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86c4102ef0 0x7f86c410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:34.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.829+0000 7f86c9715640 1 --2- 172.21.15.164:0/486291556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86c4102570 0x7f86c41029b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:34.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.829+0000 7f86c9715640 1 --2- 172.21.15.164:0/486291556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86c4100e20 0x7f86c4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:34.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.829+0000 7f86c9715640 1 -- 172.21.15.164:0/486291556 >> 172.21.15.164:0/486291556 conn(0x7f86c40fc7d0 msgr2=0x7f86c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:34.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.829+0000 7f86c9715640 1 -- 172.21.15.164:0/486291556 shutdown_connections 2024-09-19T08:44:34.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.829+0000 7f86c9715640 1 -- 172.21.15.164:0/486291556 wait complete. 2024-09-19T08:44:34.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.829+0000 7f86c9715640 1 Processor -- start 2024-09-19T08:44:34.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.829+0000 7f86c9715640 1 -- start start 2024-09-19T08:44:34.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.830+0000 7f86c9715640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86c4100e20 0x7f86c41973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:34.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.830+0000 7f86c9715640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86c4102570 0x7f86c4197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:34.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.830+0000 7f86c9715640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86c4102ef0 0x7f86c419bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:34.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.830+0000 7f86c9715640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f86c4110f80 con 0x7f86c4100e20 2024-09-19T08:44:34.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.830+0000 7f86c9715640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f86c4110e00 con 0x7f86c4102ef0 2024-09-19T08:44:34.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.830+0000 7f86c9715640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f86c4111100 con 0x7f86c4102570 2024-09-19T08:44:34.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.830+0000 7f86c2ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86c4100e20 0x7f86c41973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:34.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.830+0000 7f86c2ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86c4100e20 0x7f86c41973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36714/0 (socket says 172.21.15.164:36714) 2024-09-19T08:44:34.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.830+0000 7f86c2ffd640 1 -- 172.21.15.164:0/732063007 learned_addr learned my addr 172.21.15.164:0/732063007 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:34.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.830+0000 7f86ba7fc640 1 --2- 172.21.15.164:0/732063007 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86c4102570 0x7f86c4197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:34.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.830+0000 7f86c37fe640 1 --2- 172.21.15.164:0/732063007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86c4102ef0 0x7f86c419bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:34.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.830+0000 7f86c2ffd640 1 -- 172.21.15.164:0/732063007 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86c4102570 msgr2=0x7f86c4197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:34.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.830+0000 7f86c2ffd640 1 --2- 172.21.15.164:0/732063007 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86c4102570 0x7f86c4197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:34.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.830+0000 7f86c2ffd640 1 -- 172.21.15.164:0/732063007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86c4102ef0 msgr2=0x7f86c419bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:34.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.830+0000 7f86c2ffd640 1 --2- 172.21.15.164:0/732063007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86c4102ef0 0x7f86c419bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:34.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.830+0000 7f86c2ffd640 1 -- 172.21.15.164:0/732063007 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c419c370 con 0x7f86c4100e20 2024-09-19T08:44:34.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.830+0000 7f86c2ffd640 1 --2- 172.21.15.164:0/732063007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86c4100e20 0x7f86c41973c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f86b400e9f0 tx=0x7f86b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:34.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.830+0000 7f86c0ff9640 1 -- 172.21.15.164:0/732063007 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b400cd80 con 0x7f86c4100e20 2024-09-19T08:44:34.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.830+0000 7f86c0ff9640 1 -- 172.21.15.164:0/732063007 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86b4010910 con 0x7f86c4100e20 2024-09-19T08:44:34.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.830+0000 7f86c9715640 1 -- 172.21.15.164:0/732063007 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86c419c660 con 0x7f86c4100e20 2024-09-19T08:44:34.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.830+0000 7f86c0ff9640 1 -- 172.21.15.164:0/732063007 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b4018990 con 0x7f86c4100e20 2024-09-19T08:44:34.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.830+0000 7f86c9715640 1 -- 172.21.15.164:0/732063007 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86c41a3f40 con 0x7f86c4100e20 2024-09-19T08:44:34.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.831+0000 7f86c9715640 1 -- 172.21.15.164:0/732063007 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f86c4107300 con 0x7f86c4100e20 2024-09-19T08:44:34.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.833+0000 7f86c0ff9640 1 -- 172.21.15.164:0/732063007 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f86b4010ac0 con 0x7f86c4100e20 2024-09-19T08:44:34.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.833+0000 7f86c0ff9640 1 --2- 172.21.15.164:0/732063007 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f86b00a32d0 0x7f86b00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:34.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.833+0000 7f86ba7fc640 1 --2- 172.21.15.164:0/732063007 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f86b00a32d0 0x7f86b00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:34.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.833+0000 7f86c0ff9640 1 -- 172.21.15.164:0/732063007 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f86b40d0320 con 0x7f86c4100e20 2024-09-19T08:44:34.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.833+0000 7f86c0ff9640 1 --2- 172.21.15.164:0/732063007 >> [v2:172.21.15.173:6824/2831924738,v1:172.21.15.173:6825/2831924738] conn(0x7f86b00acd80 0x7f86b00af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:34.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.833+0000 7f86c0ff9640 1 -- 172.21.15.164:0/732063007 --> [v2:172.21.15.173:6824/2831924738,v1:172.21.15.173:6825/2831924738] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f86b4022e50 con 0x7f86b00acd80 2024-09-19T08:44:34.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.833+0000 7f86c0ff9640 1 -- 172.21.15.164:0/732063007 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f86b40d0800 con 0x7f86c4100e20 2024-09-19T08:44:34.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.834+0000 7f86ba7fc640 1 --2- 172.21.15.164:0/732063007 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f86b00a32d0 0x7f86b00a5790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f86ac009230 tx=0x7f86ac0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:34.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.834+0000 7f86c37fe640 1 --2- 172.21.15.164:0/732063007 >> [v2:172.21.15.173:6824/2831924738,v1:172.21.15.173:6825/2831924738] conn(0x7f86b00acd80 0x7f86b00af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:34.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.834+0000 7f86c37fe640 1 --2- 172.21.15.164:0/732063007 >> [v2:172.21.15.173:6824/2831924738,v1:172.21.15.173:6825/2831924738] conn(0x7f86b00acd80 0x7f86b00af220 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:34.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.835+0000 7f86c0ff9640 1 -- 172.21.15.164:0/732063007 <== osd.11 v2:172.21.15.173:6824/2831924738 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f86b4022e50 con 0x7f86b00acd80 2024-09-19T08:44:34.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.849+0000 7f86c9715640 1 -- 172.21.15.164:0/732063007 --> [v2:172.21.15.173:6824/2831924738,v1:172.21.15.173:6825/2831924738] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f86c41021b0 con 0x7f86b00acd80 2024-09-19T08:44:34.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.849+0000 7f86c0ff9640 1 -- 172.21.15.164:0/732063007 <== osd.11 v2:172.21.15.173:6824/2831924738 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f86c41021b0 con 0x7f86b00acd80 2024-09-19T08:44:34.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.849+0000 7f86c9715640 1 -- 172.21.15.164:0/732063007 >> [v2:172.21.15.173:6824/2831924738,v1:172.21.15.173:6825/2831924738] conn(0x7f86b00acd80 msgr2=0x7f86b00af220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:34.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.849+0000 7f86c9715640 1 --2- 172.21.15.164:0/732063007 >> [v2:172.21.15.173:6824/2831924738,v1:172.21.15.173:6825/2831924738] conn(0x7f86b00acd80 0x7f86b00af220 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:34.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.850+0000 7f86c9715640 1 -- 172.21.15.164:0/732063007 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f86b00a32d0 msgr2=0x7f86b00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:34.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.850+0000 7f86c9715640 1 --2- 172.21.15.164:0/732063007 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f86b00a32d0 0x7f86b00a5790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f86ac009230 tx=0x7f86ac0058d0 comp rx=0 tx=0).stop 2024-09-19T08:44:34.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.850+0000 7f86c9715640 1 -- 172.21.15.164:0/732063007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86c4100e20 msgr2=0x7f86c41973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:34.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.850+0000 7f86c9715640 1 --2- 172.21.15.164:0/732063007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86c4100e20 0x7f86c41973c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f86b400e9f0 tx=0x7f86b400eeb0 comp rx=0 tx=0).stop 2024-09-19T08:44:34.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.850+0000 7f86c37fe640 1 -- 172.21.15.164:0/732063007 reap_dead start 2024-09-19T08:44:34.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.850+0000 7f86c9715640 1 -- 172.21.15.164:0/732063007 shutdown_connections 2024-09-19T08:44:34.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.850+0000 7f86c9715640 1 -- 172.21.15.164:0/732063007 >> 172.21.15.164:0/732063007 conn(0x7f86c40fc7d0 msgr2=0x7f86c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:34.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.850+0000 7f86c9715640 1 -- 172.21.15.164:0/732063007 shutdown_connections 2024-09-19T08:44:34.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:34.850+0000 7f86c9715640 1 -- 172.21.15.164:0/732063007 wait complete. 2024-09-19T08:44:35.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:34 smithi170 ceph-mon[35072]: pgmap v298: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 8 op/s 2024-09-19T08:44:35.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:34 smithi173 ceph-mon[35306]: pgmap v298: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 8 op/s 2024-09-19T08:44:35.189 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:34 smithi164 ceph-mon[30928]: pgmap v298: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 8 op/s 2024-09-19T08:44:35.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.271+0000 7f2a39e43640 1 -- 172.21.15.164:0/1564540024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a34100a80 msgr2=0x7f2a34100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:35.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.271+0000 7f2a39e43640 1 --2- 172.21.15.164:0/1564540024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a34100a80 0x7f2a34100ee0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c009930 tx=0x7f2a1c02a420 comp rx=0 tx=0).stop 2024-09-19T08:44:35.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.271+0000 7f2a39e43640 1 -- 172.21.15.164:0/1564540024 shutdown_connections 2024-09-19T08:44:35.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.271+0000 7f2a39e43640 1 --2- 172.21.15.164:0/1564540024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a34101420 0x7f2a3410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.271+0000 7f2a39e43640 1 --2- 172.21.15.164:0/1564540024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a34100a80 0x7f2a34100ee0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.271+0000 7f2a39e43640 1 --2- 172.21.15.164:0/1564540024 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a34106a80 0x7f2a34106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.271+0000 7f2a39e43640 1 -- 172.21.15.164:0/1564540024 >> 172.21.15.164:0/1564540024 conn(0x7f2a340fc7a0 msgr2=0x7f2a340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:35.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.271+0000 7f2a39e43640 1 -- 172.21.15.164:0/1564540024 shutdown_connections 2024-09-19T08:44:35.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.271+0000 7f2a39e43640 1 -- 172.21.15.164:0/1564540024 wait complete. 2024-09-19T08:44:35.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.272+0000 7f2a39e43640 1 Processor -- start 2024-09-19T08:44:35.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.272+0000 7f2a39e43640 1 -- start start 2024-09-19T08:44:35.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.272+0000 7f2a39e43640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a34100a80 0x7f2a341984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:35.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.272+0000 7f2a39e43640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a34101420 0x7f2a34198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:35.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.272+0000 7f2a39e43640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a34106a80 0x7f2a3419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:35.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.272+0000 7f2a39e43640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2a34112070 con 0x7f2a34106a80 2024-09-19T08:44:35.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.272+0000 7f2a39e43640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2a34111ef0 con 0x7f2a34100a80 2024-09-19T08:44:35.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.272+0000 7f2a39e43640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2a341121f0 con 0x7f2a34101420 2024-09-19T08:44:35.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.272+0000 7f2a33fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a34106a80 0x7f2a3419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:35.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.272+0000 7f2a33fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a34106a80 0x7f2a3419cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36732/0 (socket says 172.21.15.164:36732) 2024-09-19T08:44:35.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.272+0000 7f2a33fff640 1 -- 172.21.15.164:0/1309721232 learned_addr learned my addr 172.21.15.164:0/1309721232 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:35.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.272+0000 7f2a337fe640 1 --2- 172.21.15.164:0/1309721232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a34100a80 0x7f2a341984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:35.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.272+0000 7f2a32ffd640 1 --2- 172.21.15.164:0/1309721232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a34101420 0x7f2a34198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:35.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.272+0000 7f2a33fff640 1 -- 172.21.15.164:0/1309721232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a34101420 msgr2=0x7f2a34198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:35.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.272+0000 7f2a33fff640 1 --2- 172.21.15.164:0/1309721232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a34101420 0x7f2a34198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.272+0000 7f2a33fff640 1 -- 172.21.15.164:0/1309721232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a34100a80 msgr2=0x7f2a341984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:35.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.272+0000 7f2a33fff640 1 --2- 172.21.15.164:0/1309721232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a34100a80 0x7f2a341984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.272+0000 7f2a33fff640 1 -- 172.21.15.164:0/1309721232 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a3419d470 con 0x7f2a34106a80 2024-09-19T08:44:35.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.273+0000 7f2a33fff640 1 --2- 172.21.15.164:0/1309721232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a34106a80 0x7f2a3419cd90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c00eaa0 tx=0x7f2a2c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:35.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.273+0000 7f2a30ff9640 1 -- 172.21.15.164:0/1309721232 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c004d20 con 0x7f2a34106a80 2024-09-19T08:44:35.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.273+0000 7f2a30ff9640 1 -- 172.21.15.164:0/1309721232 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a2c005020 con 0x7f2a34106a80 2024-09-19T08:44:35.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.273+0000 7f2a39e43640 1 -- 172.21.15.164:0/1309721232 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a3419d760 con 0x7f2a34106a80 2024-09-19T08:44:35.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.273+0000 7f2a30ff9640 1 -- 172.21.15.164:0/1309721232 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c0116d0 con 0x7f2a34106a80 2024-09-19T08:44:35.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.273+0000 7f2a39e43640 1 -- 172.21.15.164:0/1309721232 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a341a5040 con 0x7f2a34106a80 2024-09-19T08:44:35.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.274+0000 7f2a39e43640 1 -- 172.21.15.164:0/1309721232 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2a34062980 con 0x7f2a34106a80 2024-09-19T08:44:35.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.275+0000 7f2a30ff9640 1 -- 172.21.15.164:0/1309721232 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f2a2c0051d0 con 0x7f2a34106a80 2024-09-19T08:44:35.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.276+0000 7f2a30ff9640 1 --2- 172.21.15.164:0/1309721232 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f2a200a32d0 0x7f2a200a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:35.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.276+0000 7f2a337fe640 1 --2- 172.21.15.164:0/1309721232 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f2a200a32d0 0x7f2a200a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:35.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.276+0000 7f2a30ff9640 1 -- 172.21.15.164:0/1309721232 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2a2c0cff70 con 0x7f2a34106a80 2024-09-19T08:44:35.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.276+0000 7f2a30ff9640 1 --2- 172.21.15.164:0/1309721232 >> [v2:172.21.15.170:6808/4120832165,v1:172.21.15.170:6809/4120832165] conn(0x7f2a200acd80 0x7f2a200af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:35.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.276+0000 7f2a30ff9640 1 -- 172.21.15.164:0/1309721232 --> [v2:172.21.15.170:6808/4120832165,v1:172.21.15.170:6809/4120832165] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2a2c014ec0 con 0x7f2a200acd80 2024-09-19T08:44:35.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.276+0000 7f2a30ff9640 1 -- 172.21.15.164:0/1309721232 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f2a2c0d0450 con 0x7f2a34106a80 2024-09-19T08:44:35.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.276+0000 7f2a337fe640 1 --2- 172.21.15.164:0/1309721232 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f2a200a32d0 0x7f2a200a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009990 tx=0x7f2a24009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:35.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.276+0000 7f2a32ffd640 1 --2- 172.21.15.164:0/1309721232 >> [v2:172.21.15.170:6808/4120832165,v1:172.21.15.170:6809/4120832165] conn(0x7f2a200acd80 0x7f2a200af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:35.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.277+0000 7f2a32ffd640 1 --2- 172.21.15.164:0/1309721232 >> [v2:172.21.15.170:6808/4120832165,v1:172.21.15.170:6809/4120832165] conn(0x7f2a200acd80 0x7f2a200af220 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:35.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.278+0000 7f2a30ff9640 1 -- 172.21.15.164:0/1309721232 <== osd.4 v2:172.21.15.170:6808/4120832165 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2a2c014ec0 con 0x7f2a200acd80 2024-09-19T08:44:35.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.291+0000 7f2a39e43640 1 -- 172.21.15.164:0/1309721232 --> [v2:172.21.15.170:6808/4120832165,v1:172.21.15.170:6809/4120832165] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2a34062e00 con 0x7f2a200acd80 2024-09-19T08:44:35.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.292+0000 7f2a30ff9640 1 -- 172.21.15.164:0/1309721232 <== osd.4 v2:172.21.15.170:6808/4120832165 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2a34062e00 con 0x7f2a200acd80 2024-09-19T08:44:35.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.292+0000 7f2a39e43640 1 -- 172.21.15.164:0/1309721232 >> [v2:172.21.15.170:6808/4120832165,v1:172.21.15.170:6809/4120832165] conn(0x7f2a200acd80 msgr2=0x7f2a200af220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:35.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.292+0000 7f2a39e43640 1 --2- 172.21.15.164:0/1309721232 >> [v2:172.21.15.170:6808/4120832165,v1:172.21.15.170:6809/4120832165] conn(0x7f2a200acd80 0x7f2a200af220 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.292+0000 7f2a39e43640 1 -- 172.21.15.164:0/1309721232 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f2a200a32d0 msgr2=0x7f2a200a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:35.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.292+0000 7f2a39e43640 1 --2- 172.21.15.164:0/1309721232 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f2a200a32d0 0x7f2a200a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009990 tx=0x7f2a24009110 comp rx=0 tx=0).stop 2024-09-19T08:44:35.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.292+0000 7f2a39e43640 1 -- 172.21.15.164:0/1309721232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a34106a80 msgr2=0x7f2a3419cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:35.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.292+0000 7f2a39e43640 1 --2- 172.21.15.164:0/1309721232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a34106a80 0x7f2a3419cd90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c00eaa0 tx=0x7f2a2c00eea0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.292+0000 7f2a33fff640 1 -- 172.21.15.164:0/1309721232 reap_dead start 2024-09-19T08:44:35.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.292+0000 7f2a39e43640 1 -- 172.21.15.164:0/1309721232 shutdown_connections 2024-09-19T08:44:35.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.292+0000 7f2a39e43640 1 -- 172.21.15.164:0/1309721232 >> 172.21.15.164:0/1309721232 conn(0x7f2a340fc7a0 msgr2=0x7f2a340fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:35.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.292+0000 7f2a39e43640 1 -- 172.21.15.164:0/1309721232 shutdown_connections 2024-09-19T08:44:35.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.292+0000 7f2a39e43640 1 -- 172.21.15.164:0/1309721232 wait complete. 2024-09-19T08:44:35.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.798+0000 7f7999935640 1 -- 172.21.15.164:0/3038911960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7994100a80 msgr2=0x7f7994100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:35.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.798+0000 7f7999935640 1 --2- 172.21.15.164:0/3038911960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7994100a80 0x7f7994100ee0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7984009930 tx=0x7f798402a420 comp rx=0 tx=0).stop 2024-09-19T08:44:35.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.798+0000 7f7999935640 1 -- 172.21.15.164:0/3038911960 shutdown_connections 2024-09-19T08:44:35.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.798+0000 7f7999935640 1 --2- 172.21.15.164:0/3038911960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7994101420 0x7f799410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.798+0000 7f7999935640 1 --2- 172.21.15.164:0/3038911960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7994100a80 0x7f7994100ee0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.798+0000 7f7999935640 1 --2- 172.21.15.164:0/3038911960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7994106a80 0x7f7994106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.798+0000 7f7999935640 1 -- 172.21.15.164:0/3038911960 >> 172.21.15.164:0/3038911960 conn(0x7f79940fc7a0 msgr2=0x7f79940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:35.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.798+0000 7f7999935640 1 -- 172.21.15.164:0/3038911960 shutdown_connections 2024-09-19T08:44:35.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.798+0000 7f7999935640 1 -- 172.21.15.164:0/3038911960 wait complete. 2024-09-19T08:44:35.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.798+0000 7f7999935640 1 Processor -- start 2024-09-19T08:44:35.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.798+0000 7f7999935640 1 -- start start 2024-09-19T08:44:35.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.799+0000 7f7999935640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7994100a80 0x7f7994197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:35.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.799+0000 7f7999935640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7994101420 0x7f7994197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:35.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.799+0000 7f7999935640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7994106a80 0x7f799419bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:35.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.799+0000 7f7999935640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7994110ea0 con 0x7f7994100a80 2024-09-19T08:44:35.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.799+0000 7f7999935640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7994110d20 con 0x7f7994106a80 2024-09-19T08:44:35.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.799+0000 7f7999935640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7994111020 con 0x7f7994101420 2024-09-19T08:44:35.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.799+0000 7f7992ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7994100a80 0x7f7994197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:35.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.799+0000 7f7992ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7994100a80 0x7f7994197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36742/0 (socket says 172.21.15.164:36742) 2024-09-19T08:44:35.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.799+0000 7f7992ffd640 1 -- 172.21.15.164:0/840361807 learned_addr learned my addr 172.21.15.164:0/840361807 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:35.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.799+0000 7f7992ffd640 1 -- 172.21.15.164:0/840361807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7994101420 msgr2=0x7f7994197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:35.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.799+0000 7f7992ffd640 1 --2- 172.21.15.164:0/840361807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7994101420 0x7f7994197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.799+0000 7f7992ffd640 1 -- 172.21.15.164:0/840361807 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7994106a80 msgr2=0x7f799419bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:35.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.799+0000 7f7992ffd640 1 --2- 172.21.15.164:0/840361807 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7994106a80 0x7f799419bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.799+0000 7f7992ffd640 1 -- 172.21.15.164:0/840361807 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799419c2f0 con 0x7f7994100a80 2024-09-19T08:44:35.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.799+0000 7f7992ffd640 1 --2- 172.21.15.164:0/840361807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7994100a80 0x7f7994197340 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f797c00c920 tx=0x7f797c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:35.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.799+0000 7f7998933640 1 -- 172.21.15.164:0/840361807 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797c007c70 con 0x7f7994100a80 2024-09-19T08:44:35.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.799+0000 7f7999935640 1 -- 172.21.15.164:0/840361807 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f799419c5e0 con 0x7f7994100a80 2024-09-19T08:44:35.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.799+0000 7f7998933640 1 -- 172.21.15.164:0/840361807 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f797c007e10 con 0x7f7994100a80 2024-09-19T08:44:35.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.799+0000 7f7998933640 1 -- 172.21.15.164:0/840361807 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797c018890 con 0x7f7994100a80 2024-09-19T08:44:35.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.799+0000 7f7999935640 1 -- 172.21.15.164:0/840361807 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79941a3ec0 con 0x7f7994100a80 2024-09-19T08:44:35.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.800+0000 7f7999935640 1 -- 172.21.15.164:0/840361807 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f79940fd880 con 0x7f7994100a80 2024-09-19T08:44:35.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.802+0000 7f7998933640 1 -- 172.21.15.164:0/840361807 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f797c018a30 con 0x7f7994100a80 2024-09-19T08:44:35.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.802+0000 7f7998933640 1 --2- 172.21.15.164:0/840361807 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f79800a32d0 0x7f79800a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:35.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.802+0000 7f79927fc640 1 --2- 172.21.15.164:0/840361807 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f79800a32d0 0x7f79800a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:35.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.802+0000 7f7998933640 1 -- 172.21.15.164:0/840361807 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f797c0cffe0 con 0x7f7994100a80 2024-09-19T08:44:35.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.803+0000 7f7998933640 1 --2- 172.21.15.164:0/840361807 >> [v2:172.21.15.170:6824/2707848227,v1:172.21.15.170:6825/2707848227] conn(0x7f79800acd80 0x7f79800af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:35.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.803+0000 7f7998933640 1 -- 172.21.15.164:0/840361807 --> [v2:172.21.15.170:6824/2707848227,v1:172.21.15.170:6825/2707848227] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f79800af8f0 con 0x7f79800acd80 2024-09-19T08:44:35.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.803+0000 7f7998933640 1 -- 172.21.15.164:0/840361807 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f797c0d04c0 con 0x7f7994100a80 2024-09-19T08:44:35.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.803+0000 7f79927fc640 1 --2- 172.21.15.164:0/840361807 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f79800a32d0 0x7f79800a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f798402a940 tx=0x7f7984000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:35.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.803+0000 7f79937fe640 1 --2- 172.21.15.164:0/840361807 >> [v2:172.21.15.170:6824/2707848227,v1:172.21.15.170:6825/2707848227] conn(0x7f79800acd80 0x7f79800af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:35.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.803+0000 7f79937fe640 1 --2- 172.21.15.164:0/840361807 >> [v2:172.21.15.170:6824/2707848227,v1:172.21.15.170:6825/2707848227] conn(0x7f79800acd80 0x7f79800af220 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:35.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.804+0000 7f7998933640 1 -- 172.21.15.164:0/840361807 <== osd.10 v2:172.21.15.170:6824/2707848227 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f79800af8f0 con 0x7f79800acd80 2024-09-19T08:44:35.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.820+0000 7f7999935640 1 -- 172.21.15.164:0/840361807 --> [v2:172.21.15.170:6824/2707848227,v1:172.21.15.170:6825/2707848227] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f79941021c0 con 0x7f79800acd80 2024-09-19T08:44:35.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.821+0000 7f7998933640 1 -- 172.21.15.164:0/840361807 <== osd.10 v2:172.21.15.170:6824/2707848227 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f79941021c0 con 0x7f79800acd80 2024-09-19T08:44:35.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.821+0000 7f7999935640 1 -- 172.21.15.164:0/840361807 >> [v2:172.21.15.170:6824/2707848227,v1:172.21.15.170:6825/2707848227] conn(0x7f79800acd80 msgr2=0x7f79800af220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:35.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.821+0000 7f7999935640 1 --2- 172.21.15.164:0/840361807 >> [v2:172.21.15.170:6824/2707848227,v1:172.21.15.170:6825/2707848227] conn(0x7f79800acd80 0x7f79800af220 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.821+0000 7f7999935640 1 -- 172.21.15.164:0/840361807 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f79800a32d0 msgr2=0x7f79800a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:35.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.821+0000 7f7999935640 1 --2- 172.21.15.164:0/840361807 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f79800a32d0 0x7f79800a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f798402a940 tx=0x7f7984000bc0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.821+0000 7f7999935640 1 -- 172.21.15.164:0/840361807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7994100a80 msgr2=0x7f7994197340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:35.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.821+0000 7f7999935640 1 --2- 172.21.15.164:0/840361807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7994100a80 0x7f7994197340 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f797c00c920 tx=0x7f797c00cde0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.821+0000 7f79937fe640 1 -- 172.21.15.164:0/840361807 reap_dead start 2024-09-19T08:44:35.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.821+0000 7f7999935640 1 -- 172.21.15.164:0/840361807 shutdown_connections 2024-09-19T08:44:35.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.821+0000 7f7999935640 1 -- 172.21.15.164:0/840361807 >> 172.21.15.164:0/840361807 conn(0x7f79940fc7a0 msgr2=0x7f79940fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:35.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.821+0000 7f7999935640 1 -- 172.21.15.164:0/840361807 shutdown_connections 2024-09-19T08:44:35.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:35.821+0000 7f7999935640 1 -- 172.21.15.164:0/840361807 wait complete. 2024-09-19T08:44:36.717 INFO:teuthology.orchestra.run.smithi164.stdout:287762808859 2024-09-19T08:44:36.718 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-19T08:44:37.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:36 smithi164 ceph-mon[30928]: pgmap v299: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-19T08:44:37.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:36 smithi170 ceph-mon[35072]: pgmap v299: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-19T08:44:37.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:36 smithi173 ceph-mon[35306]: pgmap v299: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-19T08:44:37.397 INFO:teuthology.orchestra.run.smithi164.stdout:77309411395 2024-09-19T08:44:37.397 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-19T08:44:37.831 INFO:teuthology.orchestra.run.smithi164.stdout:317827579926 2024-09-19T08:44:37.832 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-19T08:44:38.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.446+0000 7f5cb0c5b640 1 -- 172.21.15.164:0/517824481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5cac100a80 msgr2=0x7f5cac100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:38.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.446+0000 7f5cb0c5b640 1 --2- 172.21.15.164:0/517824481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5cac100a80 0x7f5cac100ee0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0009930 tx=0x7f5ca002a420 comp rx=0 tx=0).stop 2024-09-19T08:44:38.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.446+0000 7f5cb0c5b640 1 -- 172.21.15.164:0/517824481 shutdown_connections 2024-09-19T08:44:38.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.446+0000 7f5cb0c5b640 1 --2- 172.21.15.164:0/517824481 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cac101420 0x7f5cac10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:38.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.446+0000 7f5cb0c5b640 1 --2- 172.21.15.164:0/517824481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5cac100a80 0x7f5cac100ee0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:38.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.446+0000 7f5cb0c5b640 1 --2- 172.21.15.164:0/517824481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cac106a80 0x7f5cac106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:38.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.446+0000 7f5cb0c5b640 1 -- 172.21.15.164:0/517824481 >> 172.21.15.164:0/517824481 conn(0x7f5cac0fc7a0 msgr2=0x7f5cac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:38.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.446+0000 7f5cb0c5b640 1 -- 172.21.15.164:0/517824481 shutdown_connections 2024-09-19T08:44:38.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.446+0000 7f5cb0c5b640 1 -- 172.21.15.164:0/517824481 wait complete. 2024-09-19T08:44:38.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.447+0000 7f5cb0c5b640 1 Processor -- start 2024-09-19T08:44:38.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.447+0000 7f5cb0c5b640 1 -- start start 2024-09-19T08:44:38.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.447+0000 7f5cb0c5b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5cac100a80 0x7f5cac198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:38.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.447+0000 7f5cb0c5b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cac101420 0x7f5cac198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:38.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.447+0000 7f5cb0c5b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cac106a80 0x7f5cac19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:38.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.447+0000 7f5cb0c5b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac110e80 con 0x7f5cac100a80 2024-09-19T08:44:38.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.447+0000 7f5cb0c5b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac110d00 con 0x7f5cac106a80 2024-09-19T08:44:38.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.447+0000 7f5cb0c5b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac111000 con 0x7f5cac101420 2024-09-19T08:44:38.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.448+0000 7f5caa59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5cac100a80 0x7f5cac198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:38.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.448+0000 7f5caa59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5cac100a80 0x7f5cac198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36772/0 (socket says 172.21.15.164:36772) 2024-09-19T08:44:38.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.448+0000 7f5caa59c640 1 -- 172.21.15.164:0/4039462336 learned_addr learned my addr 172.21.15.164:0/4039462336 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:38.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.448+0000 7f5caad9d640 1 --2- 172.21.15.164:0/4039462336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cac106a80 0x7f5cac19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:38.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.448+0000 7f5caa59c640 1 -- 172.21.15.164:0/4039462336 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cac101420 msgr2=0x7f5cac198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:38.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.448+0000 7f5ca9d9b640 1 --2- 172.21.15.164:0/4039462336 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cac101420 0x7f5cac198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:38.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.448+0000 7f5caa59c640 1 --2- 172.21.15.164:0/4039462336 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cac101420 0x7f5cac198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:38.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.448+0000 7f5caa59c640 1 -- 172.21.15.164:0/4039462336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cac106a80 msgr2=0x7f5cac19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:38.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.448+0000 7f5caa59c640 1 --2- 172.21.15.164:0/4039462336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cac106a80 0x7f5cac19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:38.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.448+0000 7f5caa59c640 1 -- 172.21.15.164:0/4039462336 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cac19d3b0 con 0x7f5cac100a80 2024-09-19T08:44:38.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.448+0000 7f5caad9d640 1 --2- 172.21.15.164:0/4039462336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cac106a80 0x7f5cac19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:38.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.448+0000 7f5ca9d9b640 1 --2- 172.21.15.164:0/4039462336 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cac101420 0x7f5cac198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:38.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.448+0000 7f5caa59c640 1 --2- 172.21.15.164:0/4039462336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5cac100a80 0x7f5cac198400 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00d950 tx=0x7f5c9c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:38.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.448+0000 7f5c9b7fe640 1 -- 172.21.15.164:0/4039462336 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c017070 con 0x7f5cac100a80 2024-09-19T08:44:38.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.448+0000 7f5cb0c5b640 1 -- 172.21.15.164:0/4039462336 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cac19d6a0 con 0x7f5cac100a80 2024-09-19T08:44:38.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.448+0000 7f5c9b7fe640 1 -- 172.21.15.164:0/4039462336 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c9c009c30 con 0x7f5cac100a80 2024-09-19T08:44:38.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.448+0000 7f5c9b7fe640 1 -- 172.21.15.164:0/4039462336 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c00f6b0 con 0x7f5cac100a80 2024-09-19T08:44:38.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.448+0000 7f5cb0c5b640 1 -- 172.21.15.164:0/4039462336 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cac1a4f80 con 0x7f5cac100a80 2024-09-19T08:44:38.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.449+0000 7f5cb0c5b640 1 -- 172.21.15.164:0/4039462336 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5cac06aae0 con 0x7f5cac100a80 2024-09-19T08:44:38.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.450+0000 7f5c9b7fe640 1 -- 172.21.15.164:0/4039462336 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f5c9c0109d0 con 0x7f5cac100a80 2024-09-19T08:44:38.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.451+0000 7f5c9b7fe640 1 --2- 172.21.15.164:0/4039462336 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f5c940a32d0 0x7f5c940a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:38.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.451+0000 7f5ca9d9b640 1 --2- 172.21.15.164:0/4039462336 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f5c940a32d0 0x7f5c940a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:38.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.451+0000 7f5c9b7fe640 1 -- 172.21.15.164:0/4039462336 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5c9c0cf5e0 con 0x7f5cac100a80 2024-09-19T08:44:38.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.451+0000 7f5c9b7fe640 1 --2- 172.21.15.164:0/4039462336 >> [v2:172.21.15.170:6816/2124160360,v1:172.21.15.170:6817/2124160360] conn(0x7f5c940acd80 0x7f5c940af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:38.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.451+0000 7f5c9b7fe640 1 -- 172.21.15.164:0/4039462336 --> [v2:172.21.15.170:6816/2124160360,v1:172.21.15.170:6817/2124160360] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5c940af8f0 con 0x7f5c940acd80 2024-09-19T08:44:38.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.451+0000 7f5c9b7fe640 1 -- 172.21.15.164:0/4039462336 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f5c9c0d6050 con 0x7f5cac100a80 2024-09-19T08:44:38.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.452+0000 7f5ca9d9b640 1 --2- 172.21.15.164:0/4039462336 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f5c940a32d0 0x7f5c940a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0000c00 tx=0x7f5ca0032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:38.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.452+0000 7f5caad9d640 1 --2- 172.21.15.164:0/4039462336 >> [v2:172.21.15.170:6816/2124160360,v1:172.21.15.170:6817/2124160360] conn(0x7f5c940acd80 0x7f5c940af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:38.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.452+0000 7f5caad9d640 1 --2- 172.21.15.164:0/4039462336 >> [v2:172.21.15.170:6816/2124160360,v1:172.21.15.170:6817/2124160360] conn(0x7f5c940acd80 0x7f5c940af220 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:38.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.453+0000 7f5c9b7fe640 1 -- 172.21.15.164:0/4039462336 <== osd.7 v2:172.21.15.170:6816/2124160360 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5c940af8f0 con 0x7f5c940acd80 2024-09-19T08:44:38.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.466+0000 7f5cb0c5b640 1 -- 172.21.15.164:0/4039462336 --> [v2:172.21.15.170:6816/2124160360,v1:172.21.15.170:6817/2124160360] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5cac1021c0 con 0x7f5c940acd80 2024-09-19T08:44:38.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.467+0000 7f5c9b7fe640 1 -- 172.21.15.164:0/4039462336 <== osd.7 v2:172.21.15.170:6816/2124160360 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5cac1021c0 con 0x7f5c940acd80 2024-09-19T08:44:38.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.467+0000 7f5cb0c5b640 1 -- 172.21.15.164:0/4039462336 >> [v2:172.21.15.170:6816/2124160360,v1:172.21.15.170:6817/2124160360] conn(0x7f5c940acd80 msgr2=0x7f5c940af220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:38.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.467+0000 7f5cb0c5b640 1 --2- 172.21.15.164:0/4039462336 >> [v2:172.21.15.170:6816/2124160360,v1:172.21.15.170:6817/2124160360] conn(0x7f5c940acd80 0x7f5c940af220 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:38.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.467+0000 7f5cb0c5b640 1 -- 172.21.15.164:0/4039462336 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f5c940a32d0 msgr2=0x7f5c940a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:38.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.467+0000 7f5cb0c5b640 1 --2- 172.21.15.164:0/4039462336 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f5c940a32d0 0x7f5c940a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0000c00 tx=0x7f5ca0032c70 comp rx=0 tx=0).stop 2024-09-19T08:44:38.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.467+0000 7f5cb0c5b640 1 -- 172.21.15.164:0/4039462336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5cac100a80 msgr2=0x7f5cac198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:38.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.467+0000 7f5cb0c5b640 1 --2- 172.21.15.164:0/4039462336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5cac100a80 0x7f5cac198400 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00d950 tx=0x7f5c9c00de10 comp rx=0 tx=0).stop 2024-09-19T08:44:38.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.467+0000 7f5caad9d640 1 -- 172.21.15.164:0/4039462336 reap_dead start 2024-09-19T08:44:38.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.467+0000 7f5cb0c5b640 1 -- 172.21.15.164:0/4039462336 shutdown_connections 2024-09-19T08:44:38.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.467+0000 7f5cb0c5b640 1 -- 172.21.15.164:0/4039462336 >> 172.21.15.164:0/4039462336 conn(0x7f5cac0fc7a0 msgr2=0x7f5cac104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:38.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.467+0000 7f5cb0c5b640 1 -- 172.21.15.164:0/4039462336 shutdown_connections 2024-09-19T08:44:38.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:38.467+0000 7f5cb0c5b640 1 -- 172.21.15.164:0/4039462336 wait complete. 2024-09-19T08:44:39.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:38 smithi164 ceph-mon[30928]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-19T08:44:39.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:38 smithi170 ceph-mon[35072]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-19T08:44:39.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:38 smithi173 ceph-mon[35306]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-19T08:44:39.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.292+0000 7f855cc86640 1 -- 172.21.15.164:0/4253036630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f855810c170 msgr2=0x7f855810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:39.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.292+0000 7f855cc86640 1 --2- 172.21.15.164:0/4253036630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f855810c170 0x7f855810e610 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8550009ca0 tx=0x7f855002a950 comp rx=0 tx=0).stop 2024-09-19T08:44:39.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.292+0000 7f855cc86640 1 -- 172.21.15.164:0/4253036630 shutdown_connections 2024-09-19T08:44:39.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.292+0000 7f855cc86640 1 --2- 172.21.15.164:0/4253036630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f855810c170 0x7f855810e610 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:39.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.292+0000 7f855cc86640 1 --2- 172.21.15.164:0/4253036630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8558103660 0x7f855810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:39.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.292+0000 7f855cc86640 1 --2- 172.21.15.164:0/4253036630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8558102d40 0x7f8558103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:39.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.292+0000 7f855cc86640 1 -- 172.21.15.164:0/4253036630 >> 172.21.15.164:0/4253036630 conn(0x7f85580fc7a0 msgr2=0x7f85580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:39.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.292+0000 7f855cc86640 1 -- 172.21.15.164:0/4253036630 shutdown_connections 2024-09-19T08:44:39.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.292+0000 7f855cc86640 1 -- 172.21.15.164:0/4253036630 wait complete. 2024-09-19T08:44:39.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.293+0000 7f855cc86640 1 Processor -- start 2024-09-19T08:44:39.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.293+0000 7f855cc86640 1 -- start start 2024-09-19T08:44:39.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.293+0000 7f855cc86640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8558102d40 0x7f8558198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:39.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.293+0000 7f855cc86640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8558103660 0x7f8558198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:39.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.293+0000 7f855cc86640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f855810c170 0x7f855819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:39.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.293+0000 7f855cc86640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8558111f70 con 0x7f8558102d40 2024-09-19T08:44:39.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.293+0000 7f855cc86640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8558111df0 con 0x7f855810c170 2024-09-19T08:44:39.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.293+0000 7f855cc86640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f85581120f0 con 0x7f8558103660 2024-09-19T08:44:39.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.293+0000 7f855659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8558102d40 0x7f8558198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:39.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.293+0000 7f855659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8558102d40 0x7f8558198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36800/0 (socket says 172.21.15.164:36800) 2024-09-19T08:44:39.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.293+0000 7f855659c640 1 -- 172.21.15.164:0/206173530 learned_addr learned my addr 172.21.15.164:0/206173530 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:39.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.294+0000 7f8555d9b640 1 --2- 172.21.15.164:0/206173530 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8558103660 0x7f8558198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:39.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.294+0000 7f855659c640 1 -- 172.21.15.164:0/206173530 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8558103660 msgr2=0x7f8558198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:39.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.294+0000 7f855659c640 1 --2- 172.21.15.164:0/206173530 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8558103660 0x7f8558198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:39.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.294+0000 7f855659c640 1 -- 172.21.15.164:0/206173530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f855810c170 msgr2=0x7f855819cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:39.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.294+0000 7f8556d9d640 1 --2- 172.21.15.164:0/206173530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f855810c170 0x7f855819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:39.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.294+0000 7f855659c640 1 --2- 172.21.15.164:0/206173530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f855810c170 0x7f855819cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:39.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.294+0000 7f855659c640 1 -- 172.21.15.164:0/206173530 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f855819d2e0 con 0x7f8558102d40 2024-09-19T08:44:39.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.294+0000 7f8555d9b640 1 --2- 172.21.15.164:0/206173530 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8558103660 0x7f8558198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:39.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.294+0000 7f855659c640 1 --2- 172.21.15.164:0/206173530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8558102d40 0x7f8558198330 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f854800c890 tx=0x7f854800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:39.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.294+0000 7f85477fe640 1 -- 172.21.15.164:0/206173530 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8548004220 con 0x7f8558102d40 2024-09-19T08:44:39.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.294+0000 7f85477fe640 1 -- 172.21.15.164:0/206173530 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85480043c0 con 0x7f8558102d40 2024-09-19T08:44:39.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.294+0000 7f855cc86640 1 -- 172.21.15.164:0/206173530 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f855819d5d0 con 0x7f8558102d40 2024-09-19T08:44:39.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.294+0000 7f85477fe640 1 -- 172.21.15.164:0/206173530 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85480049e0 con 0x7f8558102d40 2024-09-19T08:44:39.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.294+0000 7f855cc86640 1 -- 172.21.15.164:0/206173530 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85581a4eb0 con 0x7f8558102d40 2024-09-19T08:44:39.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.295+0000 7f855cc86640 1 -- 172.21.15.164:0/206173530 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f85580fe790 con 0x7f8558102d40 2024-09-19T08:44:39.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.296+0000 7f85477fe640 1 -- 172.21.15.164:0/206173530 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f85480079b0 con 0x7f8558102d40 2024-09-19T08:44:39.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.297+0000 7f85477fe640 1 --2- 172.21.15.164:0/206173530 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f85400a32d0 0x7f85400a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:39.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.297+0000 7f8555d9b640 1 --2- 172.21.15.164:0/206173530 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f85400a32d0 0x7f85400a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:39.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.297+0000 7f85477fe640 1 -- 172.21.15.164:0/206173530 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f85480cfc90 con 0x7f8558102d40 2024-09-19T08:44:39.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.297+0000 7f85477fe640 1 --2- 172.21.15.164:0/206173530 >> [v2:172.21.15.164:6826/3918847952,v1:172.21.15.164:6827/3918847952] conn(0x7f85400acd80 0x7f85400af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:39.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.297+0000 7f85477fe640 1 -- 172.21.15.164:0/206173530 --> [v2:172.21.15.164:6826/3918847952,v1:172.21.15.164:6827/3918847952] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f85400af8f0 con 0x7f85400acd80 2024-09-19T08:44:39.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.297+0000 7f85477fe640 1 -- 172.21.15.164:0/206173530 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f85480d0170 con 0x7f8558102d40 2024-09-19T08:44:39.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.297+0000 7f8556d9d640 1 --2- 172.21.15.164:0/206173530 >> [v2:172.21.15.164:6826/3918847952,v1:172.21.15.164:6827/3918847952] conn(0x7f85400acd80 0x7f85400af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:39.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.297+0000 7f8555d9b640 1 --2- 172.21.15.164:0/206173530 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f85400a32d0 0x7f85400a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f854c005fd0 tx=0x7f854c005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:39.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.298+0000 7f8556d9d640 1 --2- 172.21.15.164:0/206173530 >> [v2:172.21.15.164:6826/3918847952,v1:172.21.15.164:6827/3918847952] conn(0x7f85400acd80 0x7f85400af220 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:39.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.298+0000 7f85477fe640 1 -- 172.21.15.164:0/206173530 <== osd.9 v2:172.21.15.164:6826/3918847952 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f85400af8f0 con 0x7f85400acd80 2024-09-19T08:44:39.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.312+0000 7f855cc86640 1 -- 172.21.15.164:0/206173530 --> [v2:172.21.15.164:6826/3918847952,v1:172.21.15.164:6827/3918847952] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f855806b070 con 0x7f85400acd80 2024-09-19T08:44:39.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.312+0000 7f85477fe640 1 -- 172.21.15.164:0/206173530 <== osd.9 v2:172.21.15.164:6826/3918847952 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f855806b070 con 0x7f85400acd80 2024-09-19T08:44:39.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.312+0000 7f855cc86640 1 -- 172.21.15.164:0/206173530 >> [v2:172.21.15.164:6826/3918847952,v1:172.21.15.164:6827/3918847952] conn(0x7f85400acd80 msgr2=0x7f85400af220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:39.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.312+0000 7f855cc86640 1 --2- 172.21.15.164:0/206173530 >> [v2:172.21.15.164:6826/3918847952,v1:172.21.15.164:6827/3918847952] conn(0x7f85400acd80 0x7f85400af220 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:39.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.312+0000 7f855cc86640 1 -- 172.21.15.164:0/206173530 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f85400a32d0 msgr2=0x7f85400a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:39.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.312+0000 7f855cc86640 1 --2- 172.21.15.164:0/206173530 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f85400a32d0 0x7f85400a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f854c005fd0 tx=0x7f854c005dc0 comp rx=0 tx=0).stop 2024-09-19T08:44:39.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.312+0000 7f855cc86640 1 -- 172.21.15.164:0/206173530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8558102d40 msgr2=0x7f8558198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:39.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.312+0000 7f855cc86640 1 --2- 172.21.15.164:0/206173530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8558102d40 0x7f8558198330 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f854800c890 tx=0x7f854800cd50 comp rx=0 tx=0).stop 2024-09-19T08:44:39.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.312+0000 7f8556d9d640 1 -- 172.21.15.164:0/206173530 reap_dead start 2024-09-19T08:44:39.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.312+0000 7f855cc86640 1 -- 172.21.15.164:0/206173530 shutdown_connections 2024-09-19T08:44:39.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.312+0000 7f855cc86640 1 -- 172.21.15.164:0/206173530 >> 172.21.15.164:0/206173530 conn(0x7f85580fc7a0 msgr2=0x7f85580fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:39.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.312+0000 7f855cc86640 1 -- 172.21.15.164:0/206173530 shutdown_connections 2024-09-19T08:44:39.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.312+0000 7f855cc86640 1 -- 172.21.15.164:0/206173530 wait complete. 2024-09-19T08:44:39.665 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:40.243 INFO:teuthology.orchestra.run.smithi164.stdout:133143986233 2024-09-19T08:44:40.243 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-19T08:44:40.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.475+0000 7f3526aa9640 1 -- 172.21.15.164:0/3061233624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3520074590 msgr2=0x7f3520074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:40.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.475+0000 7f3526aa9640 1 --2- 172.21.15.164:0/3061233624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3520074590 0x7f3520074970 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3514009900 tx=0x7f351402a480 comp rx=0 tx=0).stop 2024-09-19T08:44:40.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.476+0000 7f3526aa9640 1 -- 172.21.15.164:0/3061233624 shutdown_connections 2024-09-19T08:44:40.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.476+0000 7f3526aa9640 1 --2- 172.21.15.164:0/3061233624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3520112ba0 0x7f3520114f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:40.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.476+0000 7f3526aa9640 1 --2- 172.21.15.164:0/3061233624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3520074eb0 0x7f3520112660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:40.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.476+0000 7f3526aa9640 1 --2- 172.21.15.164:0/3061233624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3520074590 0x7f3520074970 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:40.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.476+0000 7f3526aa9640 1 -- 172.21.15.164:0/3061233624 >> 172.21.15.164:0/3061233624 conn(0x7f35200fd8f0 msgr2=0x7f35200ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:40.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.476+0000 7f3526aa9640 1 -- 172.21.15.164:0/3061233624 shutdown_connections 2024-09-19T08:44:40.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.476+0000 7f3526aa9640 1 -- 172.21.15.164:0/3061233624 wait complete. 2024-09-19T08:44:40.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.477+0000 7f3526aa9640 1 Processor -- start 2024-09-19T08:44:40.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.477+0000 7f3526aa9640 1 -- start start 2024-09-19T08:44:40.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.477+0000 7f3526aa9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3520074590 0x7f35201a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:40.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.477+0000 7f3526aa9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3520074eb0 0x7f35201a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:40.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.477+0000 7f3526aa9640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3520112ba0 0x7f352019ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:40.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.477+0000 7f3526aa9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3520118660 con 0x7f3520074590 2024-09-19T08:44:40.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.477+0000 7f3526aa9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f35201184e0 con 0x7f3520074eb0 2024-09-19T08:44:40.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.477+0000 7f3526aa9640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f35201187e0 con 0x7f3520112ba0 2024-09-19T08:44:40.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.477+0000 7f3524845640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3520074590 0x7f35201a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:40.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.477+0000 7f3524845640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3520074590 0x7f35201a4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36826/0 (socket says 172.21.15.164:36826) 2024-09-19T08:44:40.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.477+0000 7f3524845640 1 -- 172.21.15.164:0/3242642839 learned_addr learned my addr 172.21.15.164:0/3242642839 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:40.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.477+0000 7f3525046640 1 --2- 172.21.15.164:0/3242642839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3520112ba0 0x7f352019ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:40.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.477+0000 7f351ffff640 1 --2- 172.21.15.164:0/3242642839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3520074eb0 0x7f35201a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:40.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.477+0000 7f3524845640 1 -- 172.21.15.164:0/3242642839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3520112ba0 msgr2=0x7f352019ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:40.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.477+0000 7f3524845640 1 --2- 172.21.15.164:0/3242642839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3520112ba0 0x7f352019ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:40.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.477+0000 7f3524845640 1 -- 172.21.15.164:0/3242642839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3520074eb0 msgr2=0x7f35201a4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:40.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.477+0000 7f3524845640 1 --2- 172.21.15.164:0/3242642839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3520074eb0 0x7f35201a4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:40.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.477+0000 7f3524845640 1 -- 172.21.15.164:0/3242642839 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f352019f200 con 0x7f3520074590 2024-09-19T08:44:40.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.477+0000 7f351ffff640 1 --2- 172.21.15.164:0/3242642839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3520074eb0 0x7f35201a4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:40.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.478+0000 7f3524845640 1 --2- 172.21.15.164:0/3242642839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3520074590 0x7f35201a4910 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3514005ab0 tx=0x7f35140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:40.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.478+0000 7f351dffb640 1 -- 172.21.15.164:0/3242642839 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351403f030 con 0x7f3520074590 2024-09-19T08:44:40.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.478+0000 7f3526aa9640 1 -- 172.21.15.164:0/3242642839 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f352019f490 con 0x7f3520074590 2024-09-19T08:44:40.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.478+0000 7f351dffb640 1 -- 172.21.15.164:0/3242642839 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3514003ce0 con 0x7f3520074590 2024-09-19T08:44:40.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.478+0000 7f351dffb640 1 -- 172.21.15.164:0/3242642839 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3514035070 con 0x7f3520074590 2024-09-19T08:44:40.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.478+0000 7f3526aa9640 1 -- 172.21.15.164:0/3242642839 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35201ab7e0 con 0x7f3520074590 2024-09-19T08:44:40.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.479+0000 7f3526aa9640 1 -- 172.21.15.164:0/3242642839 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3520062900 con 0x7f3520074590 2024-09-19T08:44:40.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.480+0000 7f351dffb640 1 -- 172.21.15.164:0/3242642839 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f3514042030 con 0x7f3520074590 2024-09-19T08:44:40.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.481+0000 7f351dffb640 1 --2- 172.21.15.164:0/3242642839 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f35100a32d0 0x7f35100a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:40.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.481+0000 7f351ffff640 1 --2- 172.21.15.164:0/3242642839 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f35100a32d0 0x7f35100a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:40.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.481+0000 7f351dffb640 1 -- 172.21.15.164:0/3242642839 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f35140f06e0 con 0x7f3520074590 2024-09-19T08:44:40.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.481+0000 7f351dffb640 1 --2- 172.21.15.164:0/3242642839 >> [v2:172.21.15.164:6802/663038243,v1:172.21.15.164:6803/663038243] conn(0x7f35100acd80 0x7f35100af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:40.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.481+0000 7f351dffb640 1 -- 172.21.15.164:0/3242642839 --> [v2:172.21.15.164:6802/663038243,v1:172.21.15.164:6803/663038243] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3514001d30 con 0x7f35100acd80 2024-09-19T08:44:40.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.481+0000 7f351dffb640 1 -- 172.21.15.164:0/3242642839 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f35140f0a90 con 0x7f3520074590 2024-09-19T08:44:40.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.481+0000 7f3525046640 1 --2- 172.21.15.164:0/3242642839 >> [v2:172.21.15.164:6802/663038243,v1:172.21.15.164:6803/663038243] conn(0x7f35100acd80 0x7f35100af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:40.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.482+0000 7f351ffff640 1 --2- 172.21.15.164:0/3242642839 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f35100a32d0 0x7f35100a5790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f350c006fd0 tx=0x7f350c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:40.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.482+0000 7f3525046640 1 --2- 172.21.15.164:0/3242642839 >> [v2:172.21.15.164:6802/663038243,v1:172.21.15.164:6803/663038243] conn(0x7f35100acd80 0x7f35100af220 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:40.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.482+0000 7f351dffb640 1 -- 172.21.15.164:0/3242642839 <== osd.0 v2:172.21.15.164:6802/663038243 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3514001d30 con 0x7f35100acd80 2024-09-19T08:44:40.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.496+0000 7f3526aa9640 1 -- 172.21.15.164:0/3242642839 --> [v2:172.21.15.164:6802/663038243,v1:172.21.15.164:6803/663038243] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3520062d00 con 0x7f35100acd80 2024-09-19T08:44:40.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.497+0000 7f351dffb640 1 -- 172.21.15.164:0/3242642839 <== osd.0 v2:172.21.15.164:6802/663038243 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3520062d00 con 0x7f35100acd80 2024-09-19T08:44:40.498 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.497+0000 7f3526aa9640 1 -- 172.21.15.164:0/3242642839 >> [v2:172.21.15.164:6802/663038243,v1:172.21.15.164:6803/663038243] conn(0x7f35100acd80 msgr2=0x7f35100af220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:40.498 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.497+0000 7f3526aa9640 1 --2- 172.21.15.164:0/3242642839 >> [v2:172.21.15.164:6802/663038243,v1:172.21.15.164:6803/663038243] conn(0x7f35100acd80 0x7f35100af220 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:40.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.498+0000 7f3526aa9640 1 -- 172.21.15.164:0/3242642839 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f35100a32d0 msgr2=0x7f35100a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:40.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.498+0000 7f3526aa9640 1 --2- 172.21.15.164:0/3242642839 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f35100a32d0 0x7f35100a5790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f350c006fd0 tx=0x7f350c008040 comp rx=0 tx=0).stop 2024-09-19T08:44:40.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.498+0000 7f3526aa9640 1 -- 172.21.15.164:0/3242642839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3520074590 msgr2=0x7f35201a4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:40.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.498+0000 7f3526aa9640 1 --2- 172.21.15.164:0/3242642839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3520074590 0x7f35201a4910 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3514005ab0 tx=0x7f35140042f0 comp rx=0 tx=0).stop 2024-09-19T08:44:40.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.498+0000 7f3525046640 1 -- 172.21.15.164:0/3242642839 reap_dead start 2024-09-19T08:44:40.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.498+0000 7f3526aa9640 1 -- 172.21.15.164:0/3242642839 shutdown_connections 2024-09-19T08:44:40.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.498+0000 7f3526aa9640 1 -- 172.21.15.164:0/3242642839 >> 172.21.15.164:0/3242642839 conn(0x7f35200fd8f0 msgr2=0x7f35200ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:40.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.498+0000 7f3526aa9640 1 -- 172.21.15.164:0/3242642839 shutdown_connections 2024-09-19T08:44:40.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:40.498+0000 7f3526aa9640 1 -- 172.21.15.164:0/3242642839 wait complete. 2024-09-19T08:44:40.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.848+0000 7fddc7670640 1 -- 172.21.15.164:0/628882929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fddc0102570 msgr2=0x7fddc01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:40.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.848+0000 7fddc7670640 1 --2- 172.21.15.164:0/628882929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fddc0102570 0x7fddc01029b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fddb0009900 tx=0x7fddb002a480 comp rx=0 tx=0).stop 2024-09-19T08:44:40.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.848+0000 7fddc7670640 1 -- 172.21.15.164:0/628882929 shutdown_connections 2024-09-19T08:44:40.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.848+0000 7fddc7670640 1 --2- 172.21.15.164:0/628882929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddc0102ef0 0x7fddc010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:40.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.848+0000 7fddc7670640 1 --2- 172.21.15.164:0/628882929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fddc0102570 0x7fddc01029b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:40.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.848+0000 7fddc7670640 1 --2- 172.21.15.164:0/628882929 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fddc0100e20 0x7fddc0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:40.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.848+0000 7fddc7670640 1 -- 172.21.15.164:0/628882929 >> 172.21.15.164:0/628882929 conn(0x7fddc00fc7d0 msgr2=0x7fddc00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:40.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.848+0000 7fddc7670640 1 -- 172.21.15.164:0/628882929 shutdown_connections 2024-09-19T08:44:40.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.848+0000 7fddc7670640 1 -- 172.21.15.164:0/628882929 wait complete. 2024-09-19T08:44:40.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.849+0000 7fddc7670640 1 Processor -- start 2024-09-19T08:44:40.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.849+0000 7fddc7670640 1 -- start start 2024-09-19T08:44:40.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.849+0000 7fddc7670640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fddc0100e20 0x7fddc0198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:40.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.849+0000 7fddc7670640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fddc0102570 0x7fddc0198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:40.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.849+0000 7fddc7670640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddc0102ef0 0x7fddc019cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:40.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.849+0000 7fddc7670640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fddc0111020 con 0x7fddc0100e20 2024-09-19T08:44:40.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.849+0000 7fddc7670640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fddc0110ea0 con 0x7fddc0102ef0 2024-09-19T08:44:40.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.849+0000 7fddc7670640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fddc01111a0 con 0x7fddc0102570 2024-09-19T08:44:40.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.849+0000 7fddc540c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fddc0100e20 0x7fddc0198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:40.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.850+0000 7fddc540c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fddc0100e20 0x7fddc0198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36812/0 (socket says 172.21.15.164:36812) 2024-09-19T08:44:40.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.850+0000 7fddc540c640 1 -- 172.21.15.164:0/3893450724 learned_addr learned my addr 172.21.15.164:0/3893450724 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:40.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.850+0000 7fddc5c0d640 1 --2- 172.21.15.164:0/3893450724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddc0102ef0 0x7fddc019cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:40.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.850+0000 7fddc540c640 1 -- 172.21.15.164:0/3893450724 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fddc0102570 msgr2=0x7fddc0198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:40.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.850+0000 7fddc4c0b640 1 --2- 172.21.15.164:0/3893450724 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fddc0102570 0x7fddc0198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:40.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.850+0000 7fddc540c640 1 --2- 172.21.15.164:0/3893450724 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fddc0102570 0x7fddc0198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:40.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.850+0000 7fddc540c640 1 -- 172.21.15.164:0/3893450724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddc0102ef0 msgr2=0x7fddc019cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:40.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.850+0000 7fddc540c640 1 --2- 172.21.15.164:0/3893450724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddc0102ef0 0x7fddc019cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:40.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.850+0000 7fddc540c640 1 -- 172.21.15.164:0/3893450724 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc019d6b0 con 0x7fddc0100e20 2024-09-19T08:44:40.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.850+0000 7fddc540c640 1 --2- 172.21.15.164:0/3893450724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fddc0100e20 0x7fddc0198700 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fddbc00b4a0 tx=0x7fddbc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:40.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.850+0000 7fddb67fc640 1 -- 172.21.15.164:0/3893450724 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddbc014070 con 0x7fddc0100e20 2024-09-19T08:44:40.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.850+0000 7fddb67fc640 1 -- 172.21.15.164:0/3893450724 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddbc010af0 con 0x7fddc0100e20 2024-09-19T08:44:40.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.850+0000 7fddc7670640 1 -- 172.21.15.164:0/3893450724 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddc019d9a0 con 0x7fddc0100e20 2024-09-19T08:44:40.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.850+0000 7fddb67fc640 1 -- 172.21.15.164:0/3893450724 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddbc017d10 con 0x7fddc0100e20 2024-09-19T08:44:40.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.850+0000 7fddc7670640 1 -- 172.21.15.164:0/3893450724 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddc01a5280 con 0x7fddc0100e20 2024-09-19T08:44:40.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.851+0000 7fddc7670640 1 -- 172.21.15.164:0/3893450724 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fddc0062980 con 0x7fddc0100e20 2024-09-19T08:44:40.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.853+0000 7fddb67fc640 1 -- 172.21.15.164:0/3893450724 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fddbc017070 con 0x7fddc0100e20 2024-09-19T08:44:40.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.853+0000 7fddb67fc640 1 --2- 172.21.15.164:0/3893450724 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fddac0a32d0 0x7fddac0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:40.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.853+0000 7fddc4c0b640 1 --2- 172.21.15.164:0/3893450724 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fddac0a32d0 0x7fddac0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:40.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.854+0000 7fddb67fc640 1 -- 172.21.15.164:0/3893450724 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fddbc0cf0e0 con 0x7fddc0100e20 2024-09-19T08:44:40.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.854+0000 7fddb67fc640 1 --2- 172.21.15.164:0/3893450724 >> [v2:172.21.15.173:6816/2285175061,v1:172.21.15.173:6817/2285175061] conn(0x7fddac0acd80 0x7fddac0af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:40.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.854+0000 7fddb67fc640 1 -- 172.21.15.164:0/3893450724 --> [v2:172.21.15.173:6816/2285175061,v1:172.21.15.173:6817/2285175061] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fddac0af8f0 con 0x7fddac0acd80 2024-09-19T08:44:40.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.854+0000 7fddb67fc640 1 -- 172.21.15.164:0/3893450724 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fddbc0cf5c0 con 0x7fddc0100e20 2024-09-19T08:44:40.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.854+0000 7fddc4c0b640 1 --2- 172.21.15.164:0/3893450724 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fddac0a32d0 0x7fddac0a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fddb00096c0 tx=0x7fddb0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:40.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.854+0000 7fddc5c0d640 1 --2- 172.21.15.164:0/3893450724 >> [v2:172.21.15.173:6816/2285175061,v1:172.21.15.173:6817/2285175061] conn(0x7fddac0acd80 0x7fddac0af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:40.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.854+0000 7fddc5c0d640 1 --2- 172.21.15.164:0/3893450724 >> [v2:172.21.15.173:6816/2285175061,v1:172.21.15.173:6817/2285175061] conn(0x7fddac0acd80 0x7fddac0af220 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:40.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.855+0000 7fddb67fc640 1 -- 172.21.15.164:0/3893450724 <== osd.8 v2:172.21.15.173:6816/2285175061 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fddac0af8f0 con 0x7fddac0acd80 2024-09-19T08:44:40.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.868+0000 7fddc7670640 1 -- 172.21.15.164:0/3893450724 --> [v2:172.21.15.173:6816/2285175061,v1:172.21.15.173:6817/2285175061] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fddc0062e00 con 0x7fddac0acd80 2024-09-19T08:44:40.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.869+0000 7fddb67fc640 1 -- 172.21.15.164:0/3893450724 <== osd.8 v2:172.21.15.173:6816/2285175061 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fddc0062e00 con 0x7fddac0acd80 2024-09-19T08:44:40.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.869+0000 7fddc7670640 1 -- 172.21.15.164:0/3893450724 >> [v2:172.21.15.173:6816/2285175061,v1:172.21.15.173:6817/2285175061] conn(0x7fddac0acd80 msgr2=0x7fddac0af220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:40.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.869+0000 7fddc7670640 1 --2- 172.21.15.164:0/3893450724 >> [v2:172.21.15.173:6816/2285175061,v1:172.21.15.173:6817/2285175061] conn(0x7fddac0acd80 0x7fddac0af220 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:40.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.869+0000 7fddc7670640 1 -- 172.21.15.164:0/3893450724 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fddac0a32d0 msgr2=0x7fddac0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:40.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.869+0000 7fddc7670640 1 --2- 172.21.15.164:0/3893450724 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fddac0a32d0 0x7fddac0a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fddb00096c0 tx=0x7fddb0000bc0 comp rx=0 tx=0).stop 2024-09-19T08:44:40.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.869+0000 7fddc7670640 1 -- 172.21.15.164:0/3893450724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fddc0100e20 msgr2=0x7fddc0198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:40.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.869+0000 7fddc7670640 1 --2- 172.21.15.164:0/3893450724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fddc0100e20 0x7fddc0198700 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fddbc00b4a0 tx=0x7fddbc00b960 comp rx=0 tx=0).stop 2024-09-19T08:44:40.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.869+0000 7fddc5c0d640 1 -- 172.21.15.164:0/3893450724 reap_dead start 2024-09-19T08:44:40.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.869+0000 7fddc7670640 1 -- 172.21.15.164:0/3893450724 shutdown_connections 2024-09-19T08:44:40.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.869+0000 7fddc7670640 1 -- 172.21.15.164:0/3893450724 >> 172.21.15.164:0/3893450724 conn(0x7fddc00fc7d0 msgr2=0x7fddc006a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:40.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.869+0000 7fddc7670640 1 -- 172.21.15.164:0/3893450724 shutdown_connections 2024-09-19T08:44:40.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:39.870+0000 7fddc7670640 1 -- 172.21.15.164:0/3893450724 wait complete. 2024-09-19T08:44:41.086 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:40 smithi164 ceph-mon[30928]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-09-19T08:44:41.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:40 smithi170 ceph-mon[35072]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-09-19T08:44:41.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:40 smithi173 ceph-mon[35306]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-09-19T08:44:41.791 INFO:teuthology.orchestra.run.smithi164.stdout:231928234022 2024-09-19T08:44:41.792 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-19T08:44:42.187 INFO:teuthology.orchestra.run.smithi164.stdout:201863462954 2024-09-19T08:44:42.187 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-19T08:44:42.634 INFO:teuthology.orchestra.run.smithi164.stdout:34359738446 2024-09-19T08:44:42.634 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-19T08:44:43.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:42 smithi164 ceph-mon[30928]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T08:44:43.048 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:43.049 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:43.049 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:43.049 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:43.053 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:43.107 INFO:teuthology.orchestra.run.smithi164.stdout:261993005088 2024-09-19T08:44:43.108 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-19T08:44:43.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:42 smithi170 ceph-mon[35072]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T08:44:43.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:42 smithi173 ceph-mon[35306]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T08:44:43.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.935+0000 7f15ff59e640 1 -- 172.21.15.164:0/578663063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16001008d0 msgr2=0x7f1600100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:43.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.935+0000 7f15ff59e640 1 --2- 172.21.15.164:0/578663063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16001008d0 0x7f1600100d30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009930 tx=0x7f15e802a420 comp rx=0 tx=0).stop 2024-09-19T08:44:43.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.935+0000 7f15ff59e640 1 -- 172.21.15.164:0/578663063 shutdown_connections 2024-09-19T08:44:43.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.935+0000 7f15ff59e640 1 --2- 172.21.15.164:0/578663063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1600101270 0x7f160010d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.935+0000 7f15ff59e640 1 --2- 172.21.15.164:0/578663063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16001008d0 0x7f1600100d30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.935+0000 7f15ff59e640 1 --2- 172.21.15.164:0/578663063 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16001068d0 0x7f1600106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.935+0000 7f15ff59e640 1 -- 172.21.15.164:0/578663063 >> 172.21.15.164:0/578663063 conn(0x7f16000fc610 msgr2=0x7f16000fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:43.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.936+0000 7f15ff59e640 1 -- 172.21.15.164:0/578663063 shutdown_connections 2024-09-19T08:44:43.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.936+0000 7f15ff59e640 1 -- 172.21.15.164:0/578663063 wait complete. 2024-09-19T08:44:43.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.936+0000 7f15ff59e640 1 Processor -- start 2024-09-19T08:44:43.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.937+0000 7f15ff59e640 1 -- start start 2024-09-19T08:44:43.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.937+0000 7f15ff59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16001008d0 0x7f1600199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:43.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.937+0000 7f15ff59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1600101270 0x7f1600199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:43.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.937+0000 7f15ff59e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16001068d0 0x7f16001a41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:43.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.937+0000 7f15ff59e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1600110cd0 con 0x7f16001008d0 2024-09-19T08:44:43.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.937+0000 7f15ff59e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1600110b50 con 0x7f1600101270 2024-09-19T08:44:43.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.937+0000 7f15ff59e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1600110e50 con 0x7f16001068d0 2024-09-19T08:44:43.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.937+0000 7f15fe59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16001008d0 0x7f1600199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:43.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.937+0000 7f15fe59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16001008d0 0x7f1600199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58638/0 (socket says 172.21.15.164:58638) 2024-09-19T08:44:43.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.937+0000 7f15fe59c640 1 -- 172.21.15.164:0/3541778566 learned_addr learned my addr 172.21.15.164:0/3541778566 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:43.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.937+0000 7f15fed9d640 1 --2- 172.21.15.164:0/3541778566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16001068d0 0x7f16001a41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:43.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.937+0000 7f15f5d9b640 1 --2- 172.21.15.164:0/3541778566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1600101270 0x7f1600199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:43.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.937+0000 7f15fe59c640 1 -- 172.21.15.164:0/3541778566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16001068d0 msgr2=0x7f16001a41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:43.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.937+0000 7f15fe59c640 1 --2- 172.21.15.164:0/3541778566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16001068d0 0x7f16001a41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.937+0000 7f15fe59c640 1 -- 172.21.15.164:0/3541778566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1600101270 msgr2=0x7f1600199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:43.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.937+0000 7f15fe59c640 1 --2- 172.21.15.164:0/3541778566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1600101270 0x7f1600199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.937+0000 7f15fe59c640 1 -- 172.21.15.164:0/3541778566 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16001a46e0 con 0x7f16001008d0 2024-09-19T08:44:43.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.937+0000 7f15fe59c640 1 --2- 172.21.15.164:0/3541778566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16001008d0 0x7f1600199120 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f15f000d770 tx=0x7f15f000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:43.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.938+0000 7f15f7fff640 1 -- 172.21.15.164:0/3541778566 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f0017070 con 0x7f16001008d0 2024-09-19T08:44:43.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.938+0000 7f15ff59e640 1 -- 172.21.15.164:0/3541778566 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16001a48b0 con 0x7f16001008d0 2024-09-19T08:44:43.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.938+0000 7f15f7fff640 1 -- 172.21.15.164:0/3541778566 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15f0005500 con 0x7f16001008d0 2024-09-19T08:44:43.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.938+0000 7f15f7fff640 1 -- 172.21.15.164:0/3541778566 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f0004e00 con 0x7f16001008d0 2024-09-19T08:44:43.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.938+0000 7f15ff59e640 1 -- 172.21.15.164:0/3541778566 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16001a4d70 con 0x7f16001008d0 2024-09-19T08:44:43.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.939+0000 7f15ff59e640 1 -- 172.21.15.164:0/3541778566 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1600100d30 con 0x7f16001008d0 2024-09-19T08:44:43.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.940+0000 7f15f7fff640 1 -- 172.21.15.164:0/3541778566 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f15f0004fa0 con 0x7f16001008d0 2024-09-19T08:44:43.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.940+0000 7f15f7fff640 1 --2- 172.21.15.164:0/3541778566 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f15ec0a32d0 0x7f15ec0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:43.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.940+0000 7f15f5d9b640 1 --2- 172.21.15.164:0/3541778566 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f15ec0a32d0 0x7f15ec0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:43.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.941+0000 7f15f7fff640 1 -- 172.21.15.164:0/3541778566 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f15f00cf2b0 con 0x7f16001008d0 2024-09-19T08:44:43.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.941+0000 7f15f5d9b640 1 --2- 172.21.15.164:0/3541778566 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f15ec0a32d0 0x7f15ec0a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009900 tx=0x7f15e80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:43.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:43.945+0000 7f15f7fff640 1 -- 172.21.15.164:0/3541778566 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f15f009b920 con 0x7f16001008d0 2024-09-19T08:44:44.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.117+0000 7f15ff59e640 1 -- 172.21.15.164:0/3541778566 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f1600108e40 con 0x7f16001008d0 2024-09-19T08:44:44.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.118+0000 7f15f7fff640 1 -- 172.21.15.164:0/3541778566 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f15f009fc20 con 0x7f16001008d0 2024-09-19T08:44:44.118 INFO:teuthology.orchestra.run.smithi164.stdout:287762808861 2024-09-19T08:44:44.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.121+0000 7f15ff59e640 1 -- 172.21.15.164:0/3541778566 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f15ec0a32d0 msgr2=0x7f15ec0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:44.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.121+0000 7f15ff59e640 1 --2- 172.21.15.164:0/3541778566 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f15ec0a32d0 0x7f15ec0a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009900 tx=0x7f15e80058d0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.121+0000 7f15ff59e640 1 -- 172.21.15.164:0/3541778566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16001008d0 msgr2=0x7f1600199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:44.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.121+0000 7f15ff59e640 1 --2- 172.21.15.164:0/3541778566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16001008d0 0x7f1600199120 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f15f000d770 tx=0x7f15f000dc30 comp rx=0 tx=0).stop 2024-09-19T08:44:44.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.121+0000 7f15ff59e640 1 -- 172.21.15.164:0/3541778566 shutdown_connections 2024-09-19T08:44:44.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.121+0000 7f15ff59e640 1 --2- 172.21.15.164:0/3541778566 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f15ec0a32d0 0x7f15ec0a5790 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.121+0000 7f15ff59e640 1 --2- 172.21.15.164:0/3541778566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16001068d0 0x7f16001a41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.121+0000 7f15ff59e640 1 --2- 172.21.15.164:0/3541778566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1600101270 0x7f1600199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.121+0000 7f15ff59e640 1 --2- 172.21.15.164:0/3541778566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16001008d0 0x7f1600199120 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.121+0000 7f15ff59e640 1 -- 172.21.15.164:0/3541778566 >> 172.21.15.164:0/3541778566 conn(0x7f16000fc610 msgr2=0x7f16000fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:44.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.121+0000 7f15ff59e640 1 -- 172.21.15.164:0/3541778566 shutdown_connections 2024-09-19T08:44:44.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.121+0000 7f15ff59e640 1 -- 172.21.15.164:0/3541778566 wait complete. 2024-09-19T08:44:44.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.386+0000 7fe1b927a640 1 -- 172.21.15.164:0/1191913801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1b4101de0 msgr2=0x7fe1b41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:44.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.386+0000 7fe1b927a640 1 --2- 172.21.15.164:0/1191913801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1b4101de0 0x7fe1b41021c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4009900 tx=0x7fe1a402a410 comp rx=0 tx=0).stop 2024-09-19T08:44:44.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.386+0000 7fe1b927a640 1 -- 172.21.15.164:0/1191913801 shutdown_connections 2024-09-19T08:44:44.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.386+0000 7fe1b927a640 1 --2- 172.21.15.164:0/1191913801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1b4106f10 0x7fe1b410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.386+0000 7fe1b927a640 1 --2- 172.21.15.164:0/1191913801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1b4102790 0x7fe1b41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.386+0000 7fe1b927a640 1 --2- 172.21.15.164:0/1191913801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1b4101de0 0x7fe1b41021c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.386+0000 7fe1b927a640 1 -- 172.21.15.164:0/1191913801 >> 172.21.15.164:0/1191913801 conn(0x7fe1b40fd8c0 msgr2=0x7fe1b40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:44.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.386+0000 7fe1b927a640 1 -- 172.21.15.164:0/1191913801 shutdown_connections 2024-09-19T08:44:44.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.386+0000 7fe1b927a640 1 -- 172.21.15.164:0/1191913801 wait complete. 2024-09-19T08:44:44.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.387+0000 7fe1b927a640 1 Processor -- start 2024-09-19T08:44:44.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.387+0000 7fe1b927a640 1 -- start start 2024-09-19T08:44:44.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.387+0000 7fe1b927a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1b4101de0 0x7fe1b4198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:44.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.387+0000 7fe1b927a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1b4102790 0x7fe1b41989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:44.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.387+0000 7fe1b927a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1b4106f10 0x7fe1b419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:44.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.387+0000 7fe1b927a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b4111f30 con 0x7fe1b4101de0 2024-09-19T08:44:44.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.387+0000 7fe1b927a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b4111db0 con 0x7fe1b4106f10 2024-09-19T08:44:44.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.387+0000 7fe1b927a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b41120b0 con 0x7fe1b4102790 2024-09-19T08:44:44.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.387+0000 7fe1b2ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1b4101de0 0x7fe1b4198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:44.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.387+0000 7fe1b2ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1b4101de0 0x7fe1b4198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58650/0 (socket says 172.21.15.164:58650) 2024-09-19T08:44:44.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.388+0000 7fe1b2ffd640 1 -- 172.21.15.164:0/2361544356 learned_addr learned my addr 172.21.15.164:0/2361544356 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:44.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.388+0000 7fe1b27fc640 1 --2- 172.21.15.164:0/2361544356 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1b4102790 0x7fe1b41989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:44.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.388+0000 7fe1b37fe640 1 --2- 172.21.15.164:0/2361544356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1b4106f10 0x7fe1b419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:44.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.388+0000 7fe1b2ffd640 1 -- 172.21.15.164:0/2361544356 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1b4102790 msgr2=0x7fe1b41989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:44.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.388+0000 7fe1b2ffd640 1 --2- 172.21.15.164:0/2361544356 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1b4102790 0x7fe1b41989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.388+0000 7fe1b2ffd640 1 -- 172.21.15.164:0/2361544356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1b4106f10 msgr2=0x7fe1b419cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:44.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.388+0000 7fe1b2ffd640 1 --2- 172.21.15.164:0/2361544356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1b4106f10 0x7fe1b419cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.388+0000 7fe1b2ffd640 1 -- 172.21.15.164:0/2361544356 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b419d410 con 0x7fe1b4101de0 2024-09-19T08:44:44.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.388+0000 7fe1b37fe640 1 --2- 172.21.15.164:0/2361544356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1b4106f10 0x7fe1b419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:44:44.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.388+0000 7fe1b2ffd640 1 --2- 172.21.15.164:0/2361544356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1b4101de0 0x7fe1b4198460 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4006010 tx=0x7fe1a40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:44.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.388+0000 7fe19bfff640 1 -- 172.21.15.164:0/2361544356 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a403f030 con 0x7fe1b4101de0 2024-09-19T08:44:44.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.388+0000 7fe19bfff640 1 -- 172.21.15.164:0/2361544356 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1a4003ce0 con 0x7fe1b4101de0 2024-09-19T08:44:44.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.388+0000 7fe1b927a640 1 -- 172.21.15.164:0/2361544356 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1b419d6a0 con 0x7fe1b4101de0 2024-09-19T08:44:44.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.388+0000 7fe19bfff640 1 -- 172.21.15.164:0/2361544356 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a4035070 con 0x7fe1b4101de0 2024-09-19T08:44:44.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.388+0000 7fe1b927a640 1 -- 172.21.15.164:0/2361544356 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1b41a4fe0 con 0x7fe1b4101de0 2024-09-19T08:44:44.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.389+0000 7fe1b927a640 1 -- 172.21.15.164:0/2361544356 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1b41032c0 con 0x7fe1b4101de0 2024-09-19T08:44:44.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.391+0000 7fe19bfff640 1 -- 172.21.15.164:0/2361544356 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fe1a4042030 con 0x7fe1b4101de0 2024-09-19T08:44:44.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.391+0000 7fe19bfff640 1 --2- 172.21.15.164:0/2361544356 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fe19c0a32d0 0x7fe19c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:44.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.391+0000 7fe1b27fc640 1 --2- 172.21.15.164:0/2361544356 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fe19c0a32d0 0x7fe19c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:44.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.391+0000 7fe19bfff640 1 -- 172.21.15.164:0/2361544356 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe1a40f01c0 con 0x7fe1b4101de0 2024-09-19T08:44:44.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.392+0000 7fe1b27fc640 1 --2- 172.21.15.164:0/2361544356 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fe19c0a32d0 0x7fe19c0a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40fdce0 tx=0x7fe1a800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:44.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.394+0000 7fe19bfff640 1 -- 172.21.15.164:0/2361544356 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe1a40bcf40 con 0x7fe1b4101de0 2024-09-19T08:44:44.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.564+0000 7fe1b927a640 1 -- 172.21.15.164:0/2361544356 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fe1b410a0f0 con 0x7fe1b4101de0 2024-09-19T08:44:44.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.565+0000 7fe19bfff640 1 -- 172.21.15.164:0/2361544356 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe1a40bc840 con 0x7fe1b4101de0 2024-09-19T08:44:44.565 INFO:teuthology.orchestra.run.smithi164.stdout:55834574923 2024-09-19T08:44:44.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.567+0000 7fe1b927a640 1 -- 172.21.15.164:0/2361544356 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fe19c0a32d0 msgr2=0x7fe19c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:44.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.567+0000 7fe1b927a640 1 --2- 172.21.15.164:0/2361544356 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fe19c0a32d0 0x7fe19c0a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40fdce0 tx=0x7fe1a800a2d0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.567+0000 7fe1b927a640 1 -- 172.21.15.164:0/2361544356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1b4101de0 msgr2=0x7fe1b4198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:44.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.567+0000 7fe1b927a640 1 --2- 172.21.15.164:0/2361544356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1b4101de0 0x7fe1b4198460 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4006010 tx=0x7fe1a40042f0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.567+0000 7fe1b927a640 1 -- 172.21.15.164:0/2361544356 shutdown_connections 2024-09-19T08:44:44.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.567+0000 7fe1b927a640 1 --2- 172.21.15.164:0/2361544356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1b4106f10 0x7fe1b419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.567+0000 7fe1b927a640 1 --2- 172.21.15.164:0/2361544356 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fe19c0a32d0 0x7fe19c0a5790 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.567+0000 7fe1b927a640 1 --2- 172.21.15.164:0/2361544356 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1b4102790 0x7fe1b41989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.567+0000 7fe1b927a640 1 --2- 172.21.15.164:0/2361544356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1b4101de0 0x7fe1b4198460 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.567+0000 7fe1b927a640 1 -- 172.21.15.164:0/2361544356 >> 172.21.15.164:0/2361544356 conn(0x7fe1b40fd8c0 msgr2=0x7fe1b40fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:44.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.567+0000 7fe1b927a640 1 -- 172.21.15.164:0/2361544356 shutdown_connections 2024-09-19T08:44:44.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.567+0000 7fe1b927a640 1 -- 172.21.15.164:0/2361544356 wait complete. 2024-09-19T08:44:44.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.613+0000 7f3c514aa640 1 -- 172.21.15.164:0/724686587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c4c101dd0 msgr2=0x7f3c4c1021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:44.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.613+0000 7f3c514aa640 1 --2- 172.21.15.164:0/724686587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c4c101dd0 0x7f3c4c1021b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c009900 tx=0x7f3c3c02a410 comp rx=0 tx=0).stop 2024-09-19T08:44:44.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.613+0000 7f3c514aa640 1 -- 172.21.15.164:0/724686587 shutdown_connections 2024-09-19T08:44:44.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.613+0000 7f3c514aa640 1 --2- 172.21.15.164:0/724686587 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c4c106f00 0x7f3c4c10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.613+0000 7f3c514aa640 1 --2- 172.21.15.164:0/724686587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c4c102780 0x7f3c4c1067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.613+0000 7f3c514aa640 1 --2- 172.21.15.164:0/724686587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c4c101dd0 0x7f3c4c1021b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.613+0000 7f3c514aa640 1 -- 172.21.15.164:0/724686587 >> 172.21.15.164:0/724686587 conn(0x7f3c4c0fd8b0 msgr2=0x7f3c4c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:44.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.613+0000 7f3c514aa640 1 -- 172.21.15.164:0/724686587 shutdown_connections 2024-09-19T08:44:44.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.613+0000 7f3c514aa640 1 -- 172.21.15.164:0/724686587 wait complete. 2024-09-19T08:44:44.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.614+0000 7f3c514aa640 1 Processor -- start 2024-09-19T08:44:44.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.614+0000 7f3c514aa640 1 -- start start 2024-09-19T08:44:44.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.614+0000 7f3c514aa640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c4c101dd0 0x7f3c4c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:44.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.614+0000 7f3c514aa640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c4c102780 0x7f3c4c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:44.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.614+0000 7f3c514aa640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c4c106f00 0x7f3c4c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:44.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.614+0000 7f3c514aa640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c111f80 con 0x7f3c4c106f00 2024-09-19T08:44:44.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.614+0000 7f3c514aa640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c111e00 con 0x7f3c4c101dd0 2024-09-19T08:44:44.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.614+0000 7f3c514aa640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c112100 con 0x7f3c4c102780 2024-09-19T08:44:44.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.614+0000 7f3c4b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c4c106f00 0x7f3c4c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:44.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.614+0000 7f3c4b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c4c106f00 0x7f3c4c19cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58664/0 (socket says 172.21.15.164:58664) 2024-09-19T08:44:44.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.614+0000 7f3c4b7fe640 1 -- 172.21.15.164:0/1131556488 learned_addr learned my addr 172.21.15.164:0/1131556488 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:44.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.614+0000 7f3c43fff640 1 --2- 172.21.15.164:0/1131556488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c4c102780 0x7f3c4c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:44.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.615+0000 7f3c4affd640 1 --2- 172.21.15.164:0/1131556488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c4c101dd0 0x7f3c4c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:44.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.615+0000 7f3c4b7fe640 1 -- 172.21.15.164:0/1131556488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c4c102780 msgr2=0x7f3c4c1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:44.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.615+0000 7f3c4b7fe640 1 --2- 172.21.15.164:0/1131556488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c4c102780 0x7f3c4c1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.615+0000 7f3c4b7fe640 1 -- 172.21.15.164:0/1131556488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c4c101dd0 msgr2=0x7f3c4c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:44.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.615+0000 7f3c4b7fe640 1 --2- 172.21.15.164:0/1131556488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c4c101dd0 0x7f3c4c198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.615+0000 7f3c4b7fe640 1 -- 172.21.15.164:0/1131556488 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c4c19d410 con 0x7f3c4c106f00 2024-09-19T08:44:44.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.615+0000 7f3c4b7fe640 1 --2- 172.21.15.164:0/1131556488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c4c106f00 0x7f3c4c19cd30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400cce0 tx=0x7f3c4400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:44.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.615+0000 7f3c48ff9640 1 -- 172.21.15.164:0/1131556488 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4400f450 con 0x7f3c4c106f00 2024-09-19T08:44:44.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.615+0000 7f3c514aa640 1 -- 172.21.15.164:0/1131556488 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c4c19d700 con 0x7f3c4c106f00 2024-09-19T08:44:44.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.615+0000 7f3c48ff9640 1 -- 172.21.15.164:0/1131556488 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c4400fad0 con 0x7f3c4c106f00 2024-09-19T08:44:44.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.615+0000 7f3c48ff9640 1 -- 172.21.15.164:0/1131556488 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c44018b60 con 0x7f3c4c106f00 2024-09-19T08:44:44.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.615+0000 7f3c514aa640 1 -- 172.21.15.164:0/1131556488 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c4c1a4fe0 con 0x7f3c4c106f00 2024-09-19T08:44:44.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.616+0000 7f3c514aa640 1 -- 172.21.15.164:0/1131556488 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c4c1021b0 con 0x7f3c4c106f00 2024-09-19T08:44:44.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.617+0000 7f3c48ff9640 1 -- 172.21.15.164:0/1131556488 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f3c4400fc80 con 0x7f3c4c106f00 2024-09-19T08:44:44.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.618+0000 7f3c48ff9640 1 --2- 172.21.15.164:0/1131556488 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f3c380a32d0 0x7f3c380a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:44.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.618+0000 7f3c4affd640 1 --2- 172.21.15.164:0/1131556488 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f3c380a32d0 0x7f3c380a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:44.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.619+0000 7f3c48ff9640 1 -- 172.21.15.164:0/1131556488 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f3c440d0100 con 0x7f3c4c106f00 2024-09-19T08:44:44.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.619+0000 7f3c4affd640 1 --2- 172.21.15.164:0/1131556488 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f3c380a32d0 0x7f3c380a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0096c0 tx=0x7f3c3c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:44.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.622+0000 7f3c48ff9640 1 -- 172.21.15.164:0/1131556488 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3c44014030 con 0x7f3c4c106f00 2024-09-19T08:44:44.797 INFO:teuthology.orchestra.run.smithi164.stdout:77309411397 2024-09-19T08:44:44.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.796+0000 7f3c514aa640 1 -- 172.21.15.164:0/1131556488 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f3c4c109b50 con 0x7f3c4c106f00 2024-09-19T08:44:44.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.796+0000 7f3c48ff9640 1 -- 172.21.15.164:0/1131556488 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3c4409cdc0 con 0x7f3c4c106f00 2024-09-19T08:44:44.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.798+0000 7f3c514aa640 1 -- 172.21.15.164:0/1131556488 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f3c380a32d0 msgr2=0x7f3c380a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:44.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.798+0000 7f3c514aa640 1 --2- 172.21.15.164:0/1131556488 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f3c380a32d0 0x7f3c380a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0096c0 tx=0x7f3c3c000bc0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.798+0000 7f3c514aa640 1 -- 172.21.15.164:0/1131556488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c4c106f00 msgr2=0x7f3c4c19cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:44.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.798+0000 7f3c514aa640 1 --2- 172.21.15.164:0/1131556488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c4c106f00 0x7f3c4c19cd30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400cce0 tx=0x7f3c4400a620 comp rx=0 tx=0).stop 2024-09-19T08:44:44.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.798+0000 7f3c514aa640 1 -- 172.21.15.164:0/1131556488 shutdown_connections 2024-09-19T08:44:44.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.798+0000 7f3c514aa640 1 --2- 172.21.15.164:0/1131556488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c4c106f00 0x7f3c4c19cd30 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.798+0000 7f3c514aa640 1 --2- 172.21.15.164:0/1131556488 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f3c380a32d0 0x7f3c380a5790 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.798+0000 7f3c514aa640 1 --2- 172.21.15.164:0/1131556488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c4c102780 0x7f3c4c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.798+0000 7f3c514aa640 1 --2- 172.21.15.164:0/1131556488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c4c101dd0 0x7f3c4c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.798+0000 7f3c514aa640 1 -- 172.21.15.164:0/1131556488 >> 172.21.15.164:0/1131556488 conn(0x7f3c4c0fd8b0 msgr2=0x7f3c4c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:44.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.798+0000 7f3c514aa640 1 -- 172.21.15.164:0/1131556488 shutdown_connections 2024-09-19T08:44:44.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:44.798+0000 7f3c514aa640 1 -- 172.21.15.164:0/1131556488 wait complete. 2024-09-19T08:44:45.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:44 smithi170 ceph-mon[35072]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:44:45.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:44 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3541778566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T08:44:45.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:44 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2361544356' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T08:44:45.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:44 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/1131556488' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T08:44:45.160 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:44 smithi164 ceph-mon[30928]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:44:45.160 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:44 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3541778566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T08:44:45.161 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:44 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2361544356' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T08:44:45.161 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:44 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/1131556488' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T08:44:45.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:44 smithi173 ceph-mon[35306]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:44:45.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:44 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3541778566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T08:44:45.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:44 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2361544356' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T08:44:45.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:44 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/1131556488' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T08:44:45.592 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:45.593 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:46.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.285+0000 7f2b31522640 1 -- 172.21.15.164:0/3117975363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b2c102d30 msgr2=0x7f2b2c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:46.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.285+0000 7f2b31522640 1 --2- 172.21.15.164:0/3117975363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b2c102d30 0x7f2b2c103110 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c009930 tx=0x7f2b1c02a420 comp rx=0 tx=0).stop 2024-09-19T08:44:46.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.285+0000 7f2b31522640 1 -- 172.21.15.164:0/3117975363 shutdown_connections 2024-09-19T08:44:46.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.285+0000 7f2b31522640 1 --2- 172.21.15.164:0/3117975363 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b2c1082a0 0x7f2b2c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:46.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.285+0000 7f2b31522640 1 --2- 172.21.15.164:0/3117975363 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b2c1036e0 0x7f2b2c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:46.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.285+0000 7f2b31522640 1 --2- 172.21.15.164:0/3117975363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b2c102d30 0x7f2b2c103110 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:46.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.285+0000 7f2b31522640 1 -- 172.21.15.164:0/3117975363 >> 172.21.15.164:0/3117975363 conn(0x7f2b2c0fc7d0 msgr2=0x7f2b2c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:46.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.285+0000 7f2b31522640 1 -- 172.21.15.164:0/3117975363 shutdown_connections 2024-09-19T08:44:46.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.285+0000 7f2b31522640 1 -- 172.21.15.164:0/3117975363 wait complete. 2024-09-19T08:44:46.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.286+0000 7f2b31522640 1 Processor -- start 2024-09-19T08:44:46.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.286+0000 7f2b31522640 1 -- start start 2024-09-19T08:44:46.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.286+0000 7f2b31522640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b2c102d30 0x7f2b2c1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:46.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.286+0000 7f2b31522640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b2c1036e0 0x7f2b2c198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:46.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.286+0000 7f2b31522640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b2c1082a0 0x7f2b2c19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:46.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.286+0000 7f2b31522640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c1120a0 con 0x7f2b2c102d30 2024-09-19T08:44:46.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.286+0000 7f2b31522640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c111f20 con 0x7f2b2c1036e0 2024-09-19T08:44:46.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.286+0000 7f2b31522640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c112220 con 0x7f2b2c1082a0 2024-09-19T08:44:46.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.287+0000 7f2b2affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b2c102d30 0x7f2b2c1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:46.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.287+0000 7f2b2affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b2c102d30 0x7f2b2c1985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58686/0 (socket says 172.21.15.164:58686) 2024-09-19T08:44:46.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.287+0000 7f2b2affd640 1 -- 172.21.15.164:0/1430450742 learned_addr learned my addr 172.21.15.164:0/1430450742 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:46.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.287+0000 7f2b2b7fe640 1 --2- 172.21.15.164:0/1430450742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b2c1082a0 0x7f2b2c19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:46.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.287+0000 7f2b2a7fc640 1 --2- 172.21.15.164:0/1430450742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b2c1036e0 0x7f2b2c198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:46.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.287+0000 7f2b2affd640 1 -- 172.21.15.164:0/1430450742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b2c1082a0 msgr2=0x7f2b2c19ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:46.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.287+0000 7f2b2affd640 1 --2- 172.21.15.164:0/1430450742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b2c1082a0 0x7f2b2c19ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:46.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.287+0000 7f2b2affd640 1 -- 172.21.15.164:0/1430450742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b2c1036e0 msgr2=0x7f2b2c198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:46.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.287+0000 7f2b2affd640 1 --2- 172.21.15.164:0/1430450742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b2c1036e0 0x7f2b2c198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:46.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.287+0000 7f2b2affd640 1 -- 172.21.15.164:0/1430450742 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b2c19d590 con 0x7f2b2c102d30 2024-09-19T08:44:46.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.287+0000 7f2b2affd640 1 --2- 172.21.15.164:0/1430450742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b2c102d30 0x7f2b2c1985e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c009230 tx=0x7f2b1c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:46.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.287+0000 7f2b13fff640 1 -- 172.21.15.164:0/1430450742 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c03f030 con 0x7f2b2c102d30 2024-09-19T08:44:46.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.287+0000 7f2b31522640 1 -- 172.21.15.164:0/1430450742 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b2c19d820 con 0x7f2b2c102d30 2024-09-19T08:44:46.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.287+0000 7f2b13fff640 1 -- 172.21.15.164:0/1430450742 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b1c003ad0 con 0x7f2b2c102d30 2024-09-19T08:44:46.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.287+0000 7f2b13fff640 1 -- 172.21.15.164:0/1430450742 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c035070 con 0x7f2b2c102d30 2024-09-19T08:44:46.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.288+0000 7f2b31522640 1 -- 172.21.15.164:0/1430450742 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b2c1008c0 con 0x7f2b2c102d30 2024-09-19T08:44:46.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.289+0000 7f2b31522640 1 -- 172.21.15.164:0/1430450742 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b2c06a850 con 0x7f2b2c102d30 2024-09-19T08:44:46.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.290+0000 7f2b13fff640 1 -- 172.21.15.164:0/1430450742 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f2b1c042030 con 0x7f2b2c102d30 2024-09-19T08:44:46.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.291+0000 7f2b13fff640 1 --2- 172.21.15.164:0/1430450742 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f2b140a32d0 0x7f2b140a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:46.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.291+0000 7f2b2a7fc640 1 --2- 172.21.15.164:0/1430450742 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f2b140a32d0 0x7f2b140a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:46.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.291+0000 7f2b13fff640 1 -- 172.21.15.164:0/1430450742 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f2b1c0eff70 con 0x7f2b2c102d30 2024-09-19T08:44:46.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.291+0000 7f2b2a7fc640 1 --2- 172.21.15.164:0/1430450742 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f2b140a32d0 0x7f2b140a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c0fee70 tx=0x7f2b20005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:46.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.295+0000 7f2b13fff640 1 -- 172.21.15.164:0/1430450742 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2b1c0bc5e0 con 0x7f2b2c102d30 2024-09-19T08:44:46.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.477+0000 7f2b31522640 1 -- 172.21.15.164:0/1430450742 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f2b2c199940 con 0x7f2b2c102d30 2024-09-19T08:44:46.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.477+0000 7f2b13fff640 1 -- 172.21.15.164:0/1430450742 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f2b1c0c08e0 con 0x7f2b2c102d30 2024-09-19T08:44:46.478 INFO:teuthology.orchestra.run.smithi164.stdout:317827579928 2024-09-19T08:44:46.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.479+0000 7f2b31522640 1 -- 172.21.15.164:0/1430450742 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f2b140a32d0 msgr2=0x7f2b140a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:46.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.479+0000 7f2b31522640 1 --2- 172.21.15.164:0/1430450742 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f2b140a32d0 0x7f2b140a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c0fee70 tx=0x7f2b20005f90 comp rx=0 tx=0).stop 2024-09-19T08:44:46.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.479+0000 7f2b31522640 1 -- 172.21.15.164:0/1430450742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b2c102d30 msgr2=0x7f2b2c1985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:46.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.479+0000 7f2b31522640 1 --2- 172.21.15.164:0/1430450742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b2c102d30 0x7f2b2c1985e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c009230 tx=0x7f2b1c004270 comp rx=0 tx=0).stop 2024-09-19T08:44:46.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.479+0000 7f2b31522640 1 -- 172.21.15.164:0/1430450742 shutdown_connections 2024-09-19T08:44:46.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.479+0000 7f2b31522640 1 --2- 172.21.15.164:0/1430450742 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f2b140a32d0 0x7f2b140a5790 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:46.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.479+0000 7f2b31522640 1 --2- 172.21.15.164:0/1430450742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b2c1082a0 0x7f2b2c19ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:46.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.479+0000 7f2b31522640 1 --2- 172.21.15.164:0/1430450742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b2c1036e0 0x7f2b2c198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:46.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.479+0000 7f2b31522640 1 --2- 172.21.15.164:0/1430450742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b2c102d30 0x7f2b2c1985e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:46.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.479+0000 7f2b31522640 1 -- 172.21.15.164:0/1430450742 >> 172.21.15.164:0/1430450742 conn(0x7f2b2c0fc7d0 msgr2=0x7f2b2c105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:46.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.480+0000 7f2b31522640 1 -- 172.21.15.164:0/1430450742 shutdown_connections 2024-09-19T08:44:46.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:46.480+0000 7f2b31522640 1 -- 172.21.15.164:0/1430450742 wait complete. 2024-09-19T08:44:46.918 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411395 got 77309411397 for osd.2 2024-09-19T08:44:46.918 DEBUG:teuthology.parallel:result is None 2024-09-19T08:44:47.014 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:47.042 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574920 got 55834574923 for osd.1 2024-09-19T08:44:47.042 DEBUG:teuthology.parallel:result is None 2024-09-19T08:44:47.096 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:46 smithi173 ceph-mon[35306]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 9 op/s 2024-09-19T08:44:47.096 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:46 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:44:47.096 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:46 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/1430450742' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T08:44:47.111 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:46 smithi170 ceph-mon[35072]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 9 op/s 2024-09-19T08:44:47.111 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:46 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:44:47.111 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:46 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/1430450742' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T08:44:47.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:46 smithi164 ceph-mon[30928]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 9 op/s 2024-09-19T08:44:47.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:46 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:44:47.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:46 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/1430450742' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T08:44:47.695 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579926 got 317827579928 for osd.11 2024-09-19T08:44:47.696 DEBUG:teuthology.parallel:result is None 2024-09-19T08:44:47.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.842+0000 7f4255f23640 1 -- 172.21.15.164:0/2764769726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4250100e20 msgr2=0x7f4250101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:47.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.842+0000 7f4255f23640 1 --2- 172.21.15.164:0/2764769726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4250100e20 0x7f4250101200 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4244009900 tx=0x7f424402a480 comp rx=0 tx=0).stop 2024-09-19T08:44:47.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.842+0000 7f4255f23640 1 -- 172.21.15.164:0/2764769726 shutdown_connections 2024-09-19T08:44:47.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.842+0000 7f4255f23640 1 --2- 172.21.15.164:0/2764769726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4250102ef0 0x7f425010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:47.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.842+0000 7f4255f23640 1 --2- 172.21.15.164:0/2764769726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4250102570 0x7f42501029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:47.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.842+0000 7f4255f23640 1 --2- 172.21.15.164:0/2764769726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4250100e20 0x7f4250101200 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:47.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.842+0000 7f4255f23640 1 -- 172.21.15.164:0/2764769726 >> 172.21.15.164:0/2764769726 conn(0x7f42500fc7d0 msgr2=0x7f42500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:47.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.842+0000 7f4255f23640 1 -- 172.21.15.164:0/2764769726 shutdown_connections 2024-09-19T08:44:47.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.842+0000 7f4255f23640 1 -- 172.21.15.164:0/2764769726 wait complete. 2024-09-19T08:44:47.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.843+0000 7f4255f23640 1 Processor -- start 2024-09-19T08:44:47.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.843+0000 7f4255f23640 1 -- start start 2024-09-19T08:44:47.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.843+0000 7f4255f23640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4250100e20 0x7f4250199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:47.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.843+0000 7f4255f23640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4250102570 0x7f42501998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:47.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.843+0000 7f4255f23640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4250102ef0 0x7f42501a4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:47.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.843+0000 7f4255f23640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4250110f80 con 0x7f4250102570 2024-09-19T08:44:47.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.843+0000 7f4255f23640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4250110e00 con 0x7f4250100e20 2024-09-19T08:44:47.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.843+0000 7f4255f23640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4250111100 con 0x7f4250102ef0 2024-09-19T08:44:47.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.843+0000 7f424effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4250102570 0x7f42501998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:47.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.843+0000 7f424effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4250102570 0x7f42501998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58706/0 (socket says 172.21.15.164:58706) 2024-09-19T08:44:47.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.843+0000 7f424effd640 1 -- 172.21.15.164:0/210653312 learned_addr learned my addr 172.21.15.164:0/210653312 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:47.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.843+0000 7f424f7fe640 1 --2- 172.21.15.164:0/210653312 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4250100e20 0x7f4250199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:47.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.843+0000 7f424ffff640 1 --2- 172.21.15.164:0/210653312 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4250102ef0 0x7f42501a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:47.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.843+0000 7f424effd640 1 -- 172.21.15.164:0/210653312 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4250102ef0 msgr2=0x7f42501a4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:47.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.843+0000 7f424effd640 1 --2- 172.21.15.164:0/210653312 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4250102ef0 0x7f42501a4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:47.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.843+0000 7f424effd640 1 -- 172.21.15.164:0/210653312 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4250100e20 msgr2=0x7f4250199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:47.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.843+0000 7f424effd640 1 --2- 172.21.15.164:0/210653312 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4250100e20 0x7f4250199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:47.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.843+0000 7f424effd640 1 -- 172.21.15.164:0/210653312 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42501a4940 con 0x7f4250102570 2024-09-19T08:44:47.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.843+0000 7f424f7fe640 1 --2- 172.21.15.164:0/210653312 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4250100e20 0x7f4250199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:44:47.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.844+0000 7f424effd640 1 --2- 172.21.15.164:0/210653312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4250102570 0x7f42501998c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f424000e9f0 tx=0x7f424000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:47.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.844+0000 7f424cff9640 1 -- 172.21.15.164:0/210653312 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424000cd80 con 0x7f4250102570 2024-09-19T08:44:47.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.844+0000 7f4255f23640 1 -- 172.21.15.164:0/210653312 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42501a4b10 con 0x7f4250102570 2024-09-19T08:44:47.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.844+0000 7f424cff9640 1 -- 172.21.15.164:0/210653312 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4240005500 con 0x7f4250102570 2024-09-19T08:44:47.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.844+0000 7f424cff9640 1 -- 172.21.15.164:0/210653312 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4240010690 con 0x7f4250102570 2024-09-19T08:44:47.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.844+0000 7f4255f23640 1 -- 172.21.15.164:0/210653312 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42501a4fd0 con 0x7f4250102570 2024-09-19T08:44:47.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.845+0000 7f4255f23640 1 -- 172.21.15.164:0/210653312 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4250101200 con 0x7f4250102570 2024-09-19T08:44:47.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.846+0000 7f424cff9640 1 -- 172.21.15.164:0/210653312 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f42400046c0 con 0x7f4250102570 2024-09-19T08:44:47.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.846+0000 7f424cff9640 1 --2- 172.21.15.164:0/210653312 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f42380a32d0 0x7f42380a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:47.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.847+0000 7f424f7fe640 1 --2- 172.21.15.164:0/210653312 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f42380a32d0 0x7f42380a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:47.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.847+0000 7f424cff9640 1 -- 172.21.15.164:0/210653312 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f42400cfe00 con 0x7f4250102570 2024-09-19T08:44:47.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.847+0000 7f424f7fe640 1 --2- 172.21.15.164:0/210653312 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f42380a32d0 0x7f42380a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4244009230 tx=0x7f42440058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:47.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:47.851+0000 7f424cff9640 1 -- 172.21.15.164:0/210653312 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f424009ca40 con 0x7f4250102570 2024-09-19T08:44:48.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:48.022+0000 7f4255f23640 1 -- 172.21.15.164:0/210653312 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f4250109090 con 0x7f4250102570 2024-09-19T08:44:48.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:48.022+0000 7f424cff9640 1 -- 172.21.15.164:0/210653312 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f424009c340 con 0x7f4250102570 2024-09-19T08:44:48.023 INFO:teuthology.orchestra.run.smithi164.stdout:184683593777 2024-09-19T08:44:48.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:48.024+0000 7f4255f23640 1 -- 172.21.15.164:0/210653312 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f42380a32d0 msgr2=0x7f42380a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:48.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:48.024+0000 7f4255f23640 1 --2- 172.21.15.164:0/210653312 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f42380a32d0 0x7f42380a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4244009230 tx=0x7f42440058d0 comp rx=0 tx=0).stop 2024-09-19T08:44:48.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:48.024+0000 7f4255f23640 1 -- 172.21.15.164:0/210653312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4250102570 msgr2=0x7f42501998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:48.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:48.024+0000 7f4255f23640 1 --2- 172.21.15.164:0/210653312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4250102570 0x7f42501998c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f424000e9f0 tx=0x7f424000edf0 comp rx=0 tx=0).stop 2024-09-19T08:44:48.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:48.024+0000 7f4255f23640 1 -- 172.21.15.164:0/210653312 shutdown_connections 2024-09-19T08:44:48.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:48.024+0000 7f4255f23640 1 --2- 172.21.15.164:0/210653312 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f42380a32d0 0x7f42380a5790 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:48.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:48.024+0000 7f4255f23640 1 --2- 172.21.15.164:0/210653312 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4250102ef0 0x7f42501a4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:48.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:48.024+0000 7f4255f23640 1 --2- 172.21.15.164:0/210653312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4250102570 0x7f42501998c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:48.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:48.024+0000 7f4255f23640 1 --2- 172.21.15.164:0/210653312 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4250100e20 0x7f4250199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:48.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:48.024+0000 7f4255f23640 1 -- 172.21.15.164:0/210653312 >> 172.21.15.164:0/210653312 conn(0x7f42500fc7d0 msgr2=0x7f42500fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:48.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:48.025+0000 7f4255f23640 1 -- 172.21.15.164:0/210653312 shutdown_connections 2024-09-19T08:44:48.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:48.025+0000 7f4255f23640 1 -- 172.21.15.164:0/210653312 wait complete. 2024-09-19T08:44:49.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:48 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:44:49.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:48 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:44:49.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:48 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:44:49.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:48 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:44:49.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:48 smithi164 ceph-mon[30928]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s 2024-09-19T08:44:49.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:48 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/210653312' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T08:44:49.109 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808859 got 287762808861 for osd.10 2024-09-19T08:44:49.109 DEBUG:teuthology.parallel:result is None 2024-09-19T08:44:49.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:48 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:44:49.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:48 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:44:49.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:48 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:44:49.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:48 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:44:49.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:48 smithi170 ceph-mon[35072]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s 2024-09-19T08:44:49.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:48 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/210653312' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T08:44:49.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:48 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:44:49.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:48 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:44:49.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:48 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:44:49.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:48 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:44:49.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:48 smithi173 ceph-mon[35306]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s 2024-09-19T08:44:49.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:48 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/210653312' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T08:44:49.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.272+0000 7f7c2d3bb640 1 -- 172.21.15.164:0/2119520448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c28101de0 msgr2=0x7f7c281021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:49.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.272+0000 7f7c2d3bb640 1 --2- 172.21.15.164:0/2119520448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c28101de0 0x7f7c281021c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7c18009900 tx=0x7f7c1802a410 comp rx=0 tx=0).stop 2024-09-19T08:44:49.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.272+0000 7f7c2d3bb640 1 -- 172.21.15.164:0/2119520448 shutdown_connections 2024-09-19T08:44:49.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.272+0000 7f7c2d3bb640 1 --2- 172.21.15.164:0/2119520448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c28106f10 0x7f7c2810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.272+0000 7f7c2d3bb640 1 --2- 172.21.15.164:0/2119520448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c28102790 0x7f7c281067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.272+0000 7f7c2d3bb640 1 --2- 172.21.15.164:0/2119520448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c28101de0 0x7f7c281021c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.272+0000 7f7c2d3bb640 1 -- 172.21.15.164:0/2119520448 >> 172.21.15.164:0/2119520448 conn(0x7f7c280fd8c0 msgr2=0x7f7c280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:49.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.272+0000 7f7c2d3bb640 1 -- 172.21.15.164:0/2119520448 shutdown_connections 2024-09-19T08:44:49.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.272+0000 7f7c2d3bb640 1 -- 172.21.15.164:0/2119520448 wait complete. 2024-09-19T08:44:49.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.273+0000 7f7c2d3bb640 1 Processor -- start 2024-09-19T08:44:49.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.273+0000 7f7c2d3bb640 1 -- start start 2024-09-19T08:44:49.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.273+0000 7f7c2d3bb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c28101de0 0x7f7c281985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:49.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.273+0000 7f7c2d3bb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c28102790 0x7f7c28198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:49.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.273+0000 7f7c2d3bb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c28106f10 0x7f7c2819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:49.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.273+0000 7f7c2d3bb640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7c28112010 con 0x7f7c28106f10 2024-09-19T08:44:49.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.273+0000 7f7c2d3bb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7c28111e90 con 0x7f7c28102790 2024-09-19T08:44:49.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.273+0000 7f7c2d3bb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7c28112190 con 0x7f7c28101de0 2024-09-19T08:44:49.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.273+0000 7f7c277fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c28106f10 0x7f7c2819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:49.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.273+0000 7f7c277fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c28106f10 0x7f7c2819ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58724/0 (socket says 172.21.15.164:58724) 2024-09-19T08:44:49.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.273+0000 7f7c277fe640 1 -- 172.21.15.164:0/1451530030 learned_addr learned my addr 172.21.15.164:0/1451530030 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:49.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.273+0000 7f7c26ffd640 1 --2- 172.21.15.164:0/1451530030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c28101de0 0x7f7c281985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:49.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.273+0000 7f7c1ffff640 1 --2- 172.21.15.164:0/1451530030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c28102790 0x7f7c28198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:49.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.273+0000 7f7c277fe640 1 -- 172.21.15.164:0/1451530030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c28101de0 msgr2=0x7f7c281985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:49.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.273+0000 7f7c277fe640 1 --2- 172.21.15.164:0/1451530030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c28101de0 0x7f7c281985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.273+0000 7f7c277fe640 1 -- 172.21.15.164:0/1451530030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c28102790 msgr2=0x7f7c28198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:49.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.273+0000 7f7c277fe640 1 --2- 172.21.15.164:0/1451530030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c28102790 0x7f7c28198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.274+0000 7f7c277fe640 1 -- 172.21.15.164:0/1451530030 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c2819d560 con 0x7f7c28106f10 2024-09-19T08:44:49.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.274+0000 7f7c26ffd640 1 --2- 172.21.15.164:0/1451530030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c28101de0 0x7f7c281985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:44:49.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.274+0000 7f7c277fe640 1 --2- 172.21.15.164:0/1451530030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c28106f10 0x7f7c2819ce80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000b970 tx=0x7f7c2000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:49.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.274+0000 7f7c24ff9640 1 -- 172.21.15.164:0/1451530030 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c20019070 con 0x7f7c28106f10 2024-09-19T08:44:49.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.274+0000 7f7c2d3bb640 1 -- 172.21.15.164:0/1451530030 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c2819d850 con 0x7f7c28106f10 2024-09-19T08:44:49.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.274+0000 7f7c24ff9640 1 -- 172.21.15.164:0/1451530030 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c2000cb40 con 0x7f7c28106f10 2024-09-19T08:44:49.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.274+0000 7f7c24ff9640 1 -- 172.21.15.164:0/1451530030 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c20007650 con 0x7f7c28106f10 2024-09-19T08:44:49.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.274+0000 7f7c2d3bb640 1 -- 172.21.15.164:0/1451530030 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c281a5130 con 0x7f7c28106f10 2024-09-19T08:44:49.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.275+0000 7f7c2d3bb640 1 -- 172.21.15.164:0/1451530030 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c28102240 con 0x7f7c28106f10 2024-09-19T08:44:49.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.276+0000 7f7c24ff9640 1 -- 172.21.15.164:0/1451530030 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f7c2001c070 con 0x7f7c28106f10 2024-09-19T08:44:49.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.277+0000 7f7c24ff9640 1 --2- 172.21.15.164:0/1451530030 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f7c140a32d0 0x7f7c140a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:49.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.277+0000 7f7c26ffd640 1 --2- 172.21.15.164:0/1451530030 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f7c140a32d0 0x7f7c140a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:49.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.277+0000 7f7c24ff9640 1 -- 172.21.15.164:0/1451530030 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f7c200cf3e0 con 0x7f7c28106f10 2024-09-19T08:44:49.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.277+0000 7f7c26ffd640 1 --2- 172.21.15.164:0/1451530030 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f7c140a32d0 0x7f7c140a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7c18005ab0 tx=0x7f7c18005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:49.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.280+0000 7f7c24ff9640 1 -- 172.21.15.164:0/1451530030 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7c2009c1b0 con 0x7f7c28106f10 2024-09-19T08:44:49.314 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:49.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.461+0000 7f7c2d3bb640 1 -- 172.21.15.164:0/1451530030 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f7c28109b60 con 0x7f7c28106f10 2024-09-19T08:44:49.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.461+0000 7f7c24ff9640 1 -- 172.21.15.164:0/1451530030 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7c2009bab0 con 0x7f7c28106f10 2024-09-19T08:44:49.462 INFO:teuthology.orchestra.run.smithi164.stdout:154618822710 2024-09-19T08:44:49.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.464+0000 7f7c2d3bb640 1 -- 172.21.15.164:0/1451530030 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f7c140a32d0 msgr2=0x7f7c140a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:49.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.465+0000 7f7c2d3bb640 1 --2- 172.21.15.164:0/1451530030 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f7c140a32d0 0x7f7c140a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7c18005ab0 tx=0x7f7c18005a20 comp rx=0 tx=0).stop 2024-09-19T08:44:49.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.465+0000 7f7c2d3bb640 1 -- 172.21.15.164:0/1451530030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c28106f10 msgr2=0x7f7c2819ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:49.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.465+0000 7f7c2d3bb640 1 --2- 172.21.15.164:0/1451530030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c28106f10 0x7f7c2819ce80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000b970 tx=0x7f7c2000be30 comp rx=0 tx=0).stop 2024-09-19T08:44:49.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.465+0000 7f7c2d3bb640 1 -- 172.21.15.164:0/1451530030 shutdown_connections 2024-09-19T08:44:49.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.465+0000 7f7c2d3bb640 1 --2- 172.21.15.164:0/1451530030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c28106f10 0x7f7c2819ce80 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.465+0000 7f7c2d3bb640 1 --2- 172.21.15.164:0/1451530030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c28102790 0x7f7c28198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.465+0000 7f7c2d3bb640 1 --2- 172.21.15.164:0/1451530030 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f7c140a32d0 0x7f7c140a5790 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.465+0000 7f7c2d3bb640 1 --2- 172.21.15.164:0/1451530030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c28101de0 0x7f7c281985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.465+0000 7f7c2d3bb640 1 -- 172.21.15.164:0/1451530030 >> 172.21.15.164:0/1451530030 conn(0x7f7c280fd8c0 msgr2=0x7f7c280fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:49.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.465+0000 7f7c2d3bb640 1 -- 172.21.15.164:0/1451530030 shutdown_connections 2024-09-19T08:44:49.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.465+0000 7f7c2d3bb640 1 -- 172.21.15.164:0/1451530030 wait complete. 2024-09-19T08:44:49.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.509+0000 7f1b78f85640 1 -- 172.21.15.164:0/2354320242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b741082e0 msgr2=0x7f1b7410ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:49.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.509+0000 7f1b78f85640 1 --2- 172.21.15.164:0/2354320242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b741082e0 0x7f1b7410ec50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c009930 tx=0x7f1b6c02a420 comp rx=0 tx=0).stop 2024-09-19T08:44:49.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.510+0000 7f1b78f85640 1 -- 172.21.15.164:0/2354320242 shutdown_connections 2024-09-19T08:44:49.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.510+0000 7f1b78f85640 1 --2- 172.21.15.164:0/2354320242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b741082e0 0x7f1b7410ec50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.510+0000 7f1b78f85640 1 --2- 172.21.15.164:0/2354320242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b74103760 0x7f1b74107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.510+0000 7f1b78f85640 1 --2- 172.21.15.164:0/2354320242 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b74102db0 0x7f1b74103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.510+0000 7f1b78f85640 1 -- 172.21.15.164:0/2354320242 >> 172.21.15.164:0/2354320242 conn(0x7f1b740fc7d0 msgr2=0x7f1b740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:49.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.510+0000 7f1b78f85640 1 -- 172.21.15.164:0/2354320242 shutdown_connections 2024-09-19T08:44:49.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.510+0000 7f1b78f85640 1 -- 172.21.15.164:0/2354320242 wait complete. 2024-09-19T08:44:49.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.510+0000 7f1b78f85640 1 Processor -- start 2024-09-19T08:44:49.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.510+0000 7f1b78f85640 1 -- start start 2024-09-19T08:44:49.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.511+0000 7f1b78f85640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b74102db0 0x7f1b74198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:49.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.511+0000 7f1b78f85640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b74103760 0x7f1b74198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:49.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.511+0000 7f1b78f85640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b741082e0 0x7f1b7419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:49.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.511+0000 7f1b78f85640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1b741120b0 con 0x7f1b74103760 2024-09-19T08:44:49.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.511+0000 7f1b78f85640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1b74111f30 con 0x7f1b74102db0 2024-09-19T08:44:49.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.511+0000 7f1b78f85640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1b74112230 con 0x7f1b741082e0 2024-09-19T08:44:49.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.511+0000 7f1b71d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b74103760 0x7f1b74198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:49.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.511+0000 7f1b71d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b74103760 0x7f1b74198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58744/0 (socket says 172.21.15.164:58744) 2024-09-19T08:44:49.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.511+0000 7f1b71d9b640 1 -- 172.21.15.164:0/3163386583 learned_addr learned my addr 172.21.15.164:0/3163386583 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:49.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.511+0000 7f1b72d9d640 1 --2- 172.21.15.164:0/3163386583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b741082e0 0x7f1b7419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:49.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.511+0000 7f1b7259c640 1 --2- 172.21.15.164:0/3163386583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b74102db0 0x7f1b74198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:49.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.511+0000 7f1b71d9b640 1 -- 172.21.15.164:0/3163386583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b741082e0 msgr2=0x7f1b7419ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:49.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.511+0000 7f1b71d9b640 1 --2- 172.21.15.164:0/3163386583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b741082e0 0x7f1b7419ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.511+0000 7f1b71d9b640 1 -- 172.21.15.164:0/3163386583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b74102db0 msgr2=0x7f1b74198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:49.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.511+0000 7f1b71d9b640 1 --2- 172.21.15.164:0/3163386583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b74102db0 0x7f1b74198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.511+0000 7f1b71d9b640 1 -- 172.21.15.164:0/3163386583 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7419d530 con 0x7f1b74103760 2024-09-19T08:44:49.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.511+0000 7f1b7259c640 1 --2- 172.21.15.164:0/3163386583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b74102db0 0x7f1b74198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:44:49.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.511+0000 7f1b72d9d640 1 --2- 172.21.15.164:0/3163386583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b741082e0 0x7f1b7419ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:44:49.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.511+0000 7f1b71d9b640 1 --2- 172.21.15.164:0/3163386583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b74103760 0x7f1b74198ac0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1b6800c970 tx=0x7f1b6800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:49.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.511+0000 7f1b637fe640 1 -- 172.21.15.164:0/3163386583 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68007bb0 con 0x7f1b74103760 2024-09-19T08:44:49.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.511+0000 7f1b637fe640 1 -- 172.21.15.164:0/3163386583 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b68007d50 con 0x7f1b74103760 2024-09-19T08:44:49.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.511+0000 7f1b78f85640 1 -- 172.21.15.164:0/3163386583 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b7419d820 con 0x7f1b74103760 2024-09-19T08:44:49.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.511+0000 7f1b637fe640 1 -- 172.21.15.164:0/3163386583 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6800f450 con 0x7f1b74103760 2024-09-19T08:44:49.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.511+0000 7f1b78f85640 1 -- 172.21.15.164:0/3163386583 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b741a5100 con 0x7f1b74103760 2024-09-19T08:44:49.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.512+0000 7f1b78f85640 1 -- 172.21.15.164:0/3163386583 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b74103190 con 0x7f1b74103760 2024-09-19T08:44:49.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.514+0000 7f1b637fe640 1 -- 172.21.15.164:0/3163386583 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f1b68004f40 con 0x7f1b74103760 2024-09-19T08:44:49.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.514+0000 7f1b637fe640 1 --2- 172.21.15.164:0/3163386583 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1b5c0a32d0 0x7f1b5c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:49.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.514+0000 7f1b7259c640 1 --2- 172.21.15.164:0/3163386583 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1b5c0a32d0 0x7f1b5c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:49.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.514+0000 7f1b637fe640 1 -- 172.21.15.164:0/3163386583 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1b680cfaf0 con 0x7f1b74103760 2024-09-19T08:44:49.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.515+0000 7f1b7259c640 1 --2- 172.21.15.164:0/3163386583 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1b5c0a32d0 0x7f1b5c0a5790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1b64005c40 tx=0x7f1b6400a410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:49.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.517+0000 7f1b637fe640 1 -- 172.21.15.164:0/3163386583 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1b68014030 con 0x7f1b74103760 2024-09-19T08:44:49.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.687+0000 7f1b78f85640 1 -- 172.21.15.164:0/3163386583 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f1b741998e0 con 0x7f1b74103760 2024-09-19T08:44:49.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.688+0000 7f1b637fe640 1 -- 172.21.15.164:0/3163386583 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1b741998e0 con 0x7f1b74103760 2024-09-19T08:44:49.689 INFO:teuthology.orchestra.run.smithi164.stdout:201863462956 2024-09-19T08:44:49.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.690+0000 7f1b78f85640 1 -- 172.21.15.164:0/3163386583 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1b5c0a32d0 msgr2=0x7f1b5c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:49.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.690+0000 7f1b78f85640 1 --2- 172.21.15.164:0/3163386583 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1b5c0a32d0 0x7f1b5c0a5790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1b64005c40 tx=0x7f1b6400a410 comp rx=0 tx=0).stop 2024-09-19T08:44:49.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.690+0000 7f1b78f85640 1 -- 172.21.15.164:0/3163386583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b74103760 msgr2=0x7f1b74198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:49.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.690+0000 7f1b78f85640 1 --2- 172.21.15.164:0/3163386583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b74103760 0x7f1b74198ac0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1b6800c970 tx=0x7f1b6800ce30 comp rx=0 tx=0).stop 2024-09-19T08:44:49.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.690+0000 7f1b78f85640 1 -- 172.21.15.164:0/3163386583 shutdown_connections 2024-09-19T08:44:49.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.690+0000 7f1b78f85640 1 --2- 172.21.15.164:0/3163386583 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1b5c0a32d0 0x7f1b5c0a5790 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.690+0000 7f1b78f85640 1 --2- 172.21.15.164:0/3163386583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b741082e0 0x7f1b7419ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.690+0000 7f1b78f85640 1 --2- 172.21.15.164:0/3163386583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b74103760 0x7f1b74198ac0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.690+0000 7f1b78f85640 1 --2- 172.21.15.164:0/3163386583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b74102db0 0x7f1b74198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.690+0000 7f1b78f85640 1 -- 172.21.15.164:0/3163386583 >> 172.21.15.164:0/3163386583 conn(0x7f1b740fc7d0 msgr2=0x7f1b74105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:49.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.690+0000 7f1b78f85640 1 -- 172.21.15.164:0/3163386583 shutdown_connections 2024-09-19T08:44:49.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:49.690+0000 7f1b78f85640 1 -- 172.21.15.164:0/3163386583 wait complete. 2024-09-19T08:44:50.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:49 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/1451530030' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T08:44:50.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:49 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3163386583' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T08:44:50.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:49 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/1451530030' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T08:44:50.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:49 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3163386583' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T08:44:50.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:49 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/1451530030' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T08:44:50.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:49 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3163386583' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T08:44:50.428 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462954 got 201863462956 for osd.7 2024-09-19T08:44:50.428 DEBUG:teuthology.parallel:result is None 2024-09-19T08:44:51.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:50 smithi164 ceph-mon[30928]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:44:51.080 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822706 got 154618822710 for osd.5 2024-09-19T08:44:51.081 DEBUG:teuthology.parallel:result is None 2024-09-19T08:44:51.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:50 smithi170 ceph-mon[35072]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:44:51.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:50 smithi173 ceph-mon[35306]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:44:51.611 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593775 got 184683593777 for osd.6 2024-09-19T08:44:51.611 DEBUG:teuthology.parallel:result is None 2024-09-19T08:44:52.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.183+0000 7fb92750c640 1 -- 172.21.15.164:0/707922388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb920102ef0 msgr2=0x7fb92010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:52.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.183+0000 7fb92750c640 1 --2- 172.21.15.164:0/707922388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb920102ef0 0x7fb92010da40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb918009900 tx=0x7fb91802a480 comp rx=0 tx=0).stop 2024-09-19T08:44:52.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.183+0000 7fb92750c640 1 -- 172.21.15.164:0/707922388 shutdown_connections 2024-09-19T08:44:52.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.183+0000 7fb92750c640 1 --2- 172.21.15.164:0/707922388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb920102ef0 0x7fb92010da40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.183+0000 7fb92750c640 1 --2- 172.21.15.164:0/707922388 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb920102570 0x7fb9201029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.183+0000 7fb92750c640 1 --2- 172.21.15.164:0/707922388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb920100e20 0x7fb920101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.183+0000 7fb92750c640 1 -- 172.21.15.164:0/707922388 >> 172.21.15.164:0/707922388 conn(0x7fb9200fc7d0 msgr2=0x7fb9200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:52.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.183+0000 7fb92750c640 1 -- 172.21.15.164:0/707922388 shutdown_connections 2024-09-19T08:44:52.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.183+0000 7fb92750c640 1 -- 172.21.15.164:0/707922388 wait complete. 2024-09-19T08:44:52.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.184+0000 7fb92750c640 1 Processor -- start 2024-09-19T08:44:52.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.184+0000 7fb92750c640 1 -- start start 2024-09-19T08:44:52.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.184+0000 7fb92750c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb920100e20 0x7fb9201974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:52.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.184+0000 7fb92750c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb920102570 0x7fb920197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:52.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.184+0000 7fb92750c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb920102ef0 0x7fb92019bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:52.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.184+0000 7fb92750c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb920111020 con 0x7fb920102ef0 2024-09-19T08:44:52.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.184+0000 7fb92750c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb920110ea0 con 0x7fb920100e20 2024-09-19T08:44:52.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.184+0000 7fb92750c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb9201111a0 con 0x7fb920102570 2024-09-19T08:44:52.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.184+0000 7fb925aa9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb920102ef0 0x7fb92019bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:52.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.184+0000 7fb925aa9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb920102ef0 0x7fb92019bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58802/0 (socket says 172.21.15.164:58802) 2024-09-19T08:44:52.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.184+0000 7fb925aa9640 1 -- 172.21.15.164:0/770268173 learned_addr learned my addr 172.21.15.164:0/770268173 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:52.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.185+0000 7fb9252a8640 1 --2- 172.21.15.164:0/770268173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb920100e20 0x7fb9201974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:52.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.185+0000 7fb924aa7640 1 --2- 172.21.15.164:0/770268173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb920102570 0x7fb920197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:52.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.185+0000 7fb925aa9640 1 -- 172.21.15.164:0/770268173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb920102570 msgr2=0x7fb920197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:52.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.185+0000 7fb925aa9640 1 --2- 172.21.15.164:0/770268173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb920102570 0x7fb920197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.185+0000 7fb925aa9640 1 -- 172.21.15.164:0/770268173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb920100e20 msgr2=0x7fb9201974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:52.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.185+0000 7fb925aa9640 1 --2- 172.21.15.164:0/770268173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb920100e20 0x7fb9201974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.185+0000 7fb925aa9640 1 -- 172.21.15.164:0/770268173 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92019c490 con 0x7fb920102ef0 2024-09-19T08:44:52.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.185+0000 7fb925aa9640 1 --2- 172.21.15.164:0/770268173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb920102ef0 0x7fb92019bdb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb918005ab0 tx=0x7fb918003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:52.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.185+0000 7fb91e7fc640 1 -- 172.21.15.164:0/770268173 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91803f070 con 0x7fb920102ef0 2024-09-19T08:44:52.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.185+0000 7fb91e7fc640 1 -- 172.21.15.164:0/770268173 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb918031e30 con 0x7fb920102ef0 2024-09-19T08:44:52.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.185+0000 7fb92750c640 1 -- 172.21.15.164:0/770268173 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb92019c720 con 0x7fb920102ef0 2024-09-19T08:44:52.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.185+0000 7fb91e7fc640 1 -- 172.21.15.164:0/770268173 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb918035030 con 0x7fb920102ef0 2024-09-19T08:44:52.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.185+0000 7fb92750c640 1 -- 172.21.15.164:0/770268173 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9201a4060 con 0x7fb920102ef0 2024-09-19T08:44:52.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.186+0000 7fb92750c640 1 -- 172.21.15.164:0/770268173 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb920101280 con 0x7fb920102ef0 2024-09-19T08:44:52.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.187+0000 7fb91e7fc640 1 -- 172.21.15.164:0/770268173 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fb918042070 con 0x7fb920102ef0 2024-09-19T08:44:52.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.188+0000 7fb91e7fc640 1 --2- 172.21.15.164:0/770268173 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb9080a3350 0x7fb9080a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:52.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.188+0000 7fb9252a8640 1 --2- 172.21.15.164:0/770268173 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb9080a3350 0x7fb9080a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:52.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.188+0000 7fb91e7fc640 1 -- 172.21.15.164:0/770268173 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb9180f09e0 con 0x7fb920102ef0 2024-09-19T08:44:52.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.189+0000 7fb9252a8640 1 --2- 172.21.15.164:0/770268173 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb9080a3350 0x7fb9080a5810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb910009f20 tx=0x7fb9100094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:52.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.192+0000 7fb91e7fc640 1 -- 172.21.15.164:0/770268173 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb9180bd6a0 con 0x7fb920102ef0 2024-09-19T08:44:52.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.365+0000 7fb92750c640 1 -- 172.21.15.164:0/770268173 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fb920198840 con 0x7fb920102ef0 2024-09-19T08:44:52.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.366+0000 7fb91e7fc640 1 -- 172.21.15.164:0/770268173 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb920198840 con 0x7fb920102ef0 2024-09-19T08:44:52.366 INFO:teuthology.orchestra.run.smithi164.stdout:231928234024 2024-09-19T08:44:52.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.368+0000 7fb92750c640 1 -- 172.21.15.164:0/770268173 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb9080a3350 msgr2=0x7fb9080a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:52.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.368+0000 7fb92750c640 1 --2- 172.21.15.164:0/770268173 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb9080a3350 0x7fb9080a5810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb910009f20 tx=0x7fb9100094f0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.368+0000 7fb92750c640 1 -- 172.21.15.164:0/770268173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb920102ef0 msgr2=0x7fb92019bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:52.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.368+0000 7fb92750c640 1 --2- 172.21.15.164:0/770268173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb920102ef0 0x7fb92019bdb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb918005ab0 tx=0x7fb918003c30 comp rx=0 tx=0).stop 2024-09-19T08:44:52.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.368+0000 7fb92750c640 1 -- 172.21.15.164:0/770268173 shutdown_connections 2024-09-19T08:44:52.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.368+0000 7fb92750c640 1 --2- 172.21.15.164:0/770268173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb920102ef0 0x7fb92019bdb0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.368+0000 7fb92750c640 1 --2- 172.21.15.164:0/770268173 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb9080a3350 0x7fb9080a5810 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.368+0000 7fb92750c640 1 --2- 172.21.15.164:0/770268173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb920102570 0x7fb920197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.368+0000 7fb92750c640 1 --2- 172.21.15.164:0/770268173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb920100e20 0x7fb9201974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.368+0000 7fb92750c640 1 -- 172.21.15.164:0/770268173 >> 172.21.15.164:0/770268173 conn(0x7fb9200fc7d0 msgr2=0x7fb9200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:52.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.368+0000 7fb92750c640 1 -- 172.21.15.164:0/770268173 shutdown_connections 2024-09-19T08:44:52.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.368+0000 7fb92750c640 1 -- 172.21.15.164:0/770268173 wait complete. 2024-09-19T08:44:52.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.415+0000 7f16bb73e640 1 -- 172.21.15.164:0/3149666234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16b4100b30 msgr2=0x7f16b4100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:52.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.415+0000 7f16bb73e640 1 --2- 172.21.15.164:0/3149666234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16b4100b30 0x7f16b4100f10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009ca0 tx=0x7f16b002a940 comp rx=0 tx=0).stop 2024-09-19T08:44:52.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.415+0000 7f16bb73e640 1 -- 172.21.15.164:0/3149666234 shutdown_connections 2024-09-19T08:44:52.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.415+0000 7f16bb73e640 1 --2- 172.21.15.164:0/3149666234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16b4105f40 0x7f16b41063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.415+0000 7f16bb73e640 1 --2- 172.21.15.164:0/3149666234 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16b4101450 0x7f16b4105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.415+0000 7f16bb73e640 1 --2- 172.21.15.164:0/3149666234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16b4100b30 0x7f16b4100f10 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.415+0000 7f16bb73e640 1 -- 172.21.15.164:0/3149666234 >> 172.21.15.164:0/3149666234 conn(0x7f16b40fc7a0 msgr2=0x7f16b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:52.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.415+0000 7f16bb73e640 1 -- 172.21.15.164:0/3149666234 shutdown_connections 2024-09-19T08:44:52.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.415+0000 7f16bb73e640 1 -- 172.21.15.164:0/3149666234 wait complete. 2024-09-19T08:44:52.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.416+0000 7f16bb73e640 1 Processor -- start 2024-09-19T08:44:52.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.416+0000 7f16bb73e640 1 -- start start 2024-09-19T08:44:52.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.416+0000 7f16bb73e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16b4100b30 0x7f16b41a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:52.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.416+0000 7f16bb73e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16b4101450 0x7f16b41a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:52.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.416+0000 7f16bb73e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16b4105f40 0x7f16b419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:52.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.416+0000 7f16bb73e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f16b4118550 con 0x7f16b4105f40 2024-09-19T08:44:52.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.416+0000 7f16bb73e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f16b41183d0 con 0x7f16b4100b30 2024-09-19T08:44:52.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.416+0000 7f16bb73e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f16b41186d0 con 0x7f16b4101450 2024-09-19T08:44:52.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.416+0000 7f16b9cdb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16b4105f40 0x7f16b419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:52.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.416+0000 7f16b9cdb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16b4105f40 0x7f16b419e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58824/0 (socket says 172.21.15.164:58824) 2024-09-19T08:44:52.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.416+0000 7f16b9cdb640 1 -- 172.21.15.164:0/3283932590 learned_addr learned my addr 172.21.15.164:0/3283932590 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:52.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.416+0000 7f16b8cd9640 1 --2- 172.21.15.164:0/3283932590 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16b4101450 0x7f16b41a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:52.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.416+0000 7f16b94da640 1 --2- 172.21.15.164:0/3283932590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16b4100b30 0x7f16b41a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:52.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.416+0000 7f16b9cdb640 1 -- 172.21.15.164:0/3283932590 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16b4101450 msgr2=0x7f16b41a4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:52.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.416+0000 7f16b9cdb640 1 --2- 172.21.15.164:0/3283932590 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16b4101450 0x7f16b41a4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.416+0000 7f16b9cdb640 1 -- 172.21.15.164:0/3283932590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16b4100b30 msgr2=0x7f16b41a4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:52.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.416+0000 7f16b9cdb640 1 --2- 172.21.15.164:0/3283932590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16b4100b30 0x7f16b41a4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.416+0000 7f16b9cdb640 1 -- 172.21.15.164:0/3283932590 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b419f070 con 0x7f16b4105f40 2024-09-19T08:44:52.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.417+0000 7f16b9cdb640 1 --2- 172.21.15.164:0/3283932590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16b4105f40 0x7f16b419e940 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f16ac00c970 tx=0x7f16ac00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:52.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.417+0000 7f16aa7fc640 1 -- 172.21.15.164:0/3283932590 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac007bb0 con 0x7f16b4105f40 2024-09-19T08:44:52.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.417+0000 7f16aa7fc640 1 -- 172.21.15.164:0/3283932590 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16ac007d50 con 0x7f16b4105f40 2024-09-19T08:44:52.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.417+0000 7f16bb73e640 1 -- 172.21.15.164:0/3283932590 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b419f360 con 0x7f16b4105f40 2024-09-19T08:44:52.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.417+0000 7f16aa7fc640 1 -- 172.21.15.164:0/3283932590 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac00f450 con 0x7f16b4105f40 2024-09-19T08:44:52.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.417+0000 7f16bb73e640 1 -- 172.21.15.164:0/3283932590 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b41ab650 con 0x7f16b4105f40 2024-09-19T08:44:52.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.418+0000 7f16bb73e640 1 -- 172.21.15.164:0/3283932590 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16b4105920 con 0x7f16b4105f40 2024-09-19T08:44:52.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.419+0000 7f16aa7fc640 1 -- 172.21.15.164:0/3283932590 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f16ac004f60 con 0x7f16b4105f40 2024-09-19T08:44:52.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.419+0000 7f16aa7fc640 1 --2- 172.21.15.164:0/3283932590 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f16a00a3350 0x7f16a00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:52.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.420+0000 7f16b94da640 1 --2- 172.21.15.164:0/3283932590 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f16a00a3350 0x7f16a00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:52.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.420+0000 7f16aa7fc640 1 -- 172.21.15.164:0/3283932590 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f16ac0cf520 con 0x7f16b4105f40 2024-09-19T08:44:52.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.420+0000 7f16b94da640 1 --2- 172.21.15.164:0/3283932590 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f16a00a3350 0x7f16a00a5810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f16b0000f80 tx=0x7f16b0005f30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:52.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.423+0000 7f16aa7fc640 1 -- 172.21.15.164:0/3283932590 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f16ac014030 con 0x7f16b4105f40 2024-09-19T08:44:52.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.799+0000 7f1ab47bb640 1 -- 172.21.15.164:0/376595177 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1aac106a80 msgr2=0x7f1aac106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:52.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.799+0000 7f1ab47bb640 1 --2- 172.21.15.164:0/376595177 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1aac106a80 0x7f1aac106e60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8009930 tx=0x7f1aa802a420 comp rx=0 tx=0).stop 2024-09-19T08:44:52.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.799+0000 7f1ab47bb640 1 -- 172.21.15.164:0/376595177 shutdown_connections 2024-09-19T08:44:52.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.799+0000 7f1ab47bb640 1 --2- 172.21.15.164:0/376595177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1aac101420 0x7f1aac10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.799+0000 7f1ab47bb640 1 --2- 172.21.15.164:0/376595177 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1aac100a80 0x7f1aac100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.799+0000 7f1ab47bb640 1 --2- 172.21.15.164:0/376595177 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1aac106a80 0x7f1aac106e60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.799+0000 7f1ab47bb640 1 -- 172.21.15.164:0/376595177 >> 172.21.15.164:0/376595177 conn(0x7f1aac0fc7a0 msgr2=0x7f1aac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:52.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.799+0000 7f1ab47bb640 1 -- 172.21.15.164:0/376595177 shutdown_connections 2024-09-19T08:44:52.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.799+0000 7f1ab47bb640 1 -- 172.21.15.164:0/376595177 wait complete. 2024-09-19T08:44:52.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.800+0000 7f1ab47bb640 1 Processor -- start 2024-09-19T08:44:52.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.800+0000 7f1ab47bb640 1 -- start start 2024-09-19T08:44:52.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.800+0000 7f1ab47bb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1aac100a80 0x7f1aac198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:52.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.800+0000 7f1ab47bb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1aac101420 0x7f1aac198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:52.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.800+0000 7f1ab47bb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1aac106a80 0x7f1aac19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:52.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.800+0000 7f1ab47bb640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1aac110e40 con 0x7f1aac101420 2024-09-19T08:44:52.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.800+0000 7f1ab47bb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1aac110cc0 con 0x7f1aac100a80 2024-09-19T08:44:52.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.800+0000 7f1ab47bb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1aac110fc0 con 0x7f1aac106a80 2024-09-19T08:44:52.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.800+0000 7f1ab1d56640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1aac101420 0x7f1aac198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:52.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.800+0000 7f1ab1d56640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1aac101420 0x7f1aac198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58774/0 (socket says 172.21.15.164:58774) 2024-09-19T08:44:52.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.800+0000 7f1ab1d56640 1 -- 172.21.15.164:0/3466673332 learned_addr learned my addr 172.21.15.164:0/3466673332 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:52.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.800+0000 7f1ab2557640 1 --2- 172.21.15.164:0/3466673332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1aac100a80 0x7f1aac198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:52.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.800+0000 7f1ab2d58640 1 --2- 172.21.15.164:0/3466673332 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1aac106a80 0x7f1aac19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:52.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.800+0000 7f1ab1d56640 1 -- 172.21.15.164:0/3466673332 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1aac106a80 msgr2=0x7f1aac19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:52.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.800+0000 7f1ab1d56640 1 --2- 172.21.15.164:0/3466673332 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1aac106a80 0x7f1aac19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.800+0000 7f1ab1d56640 1 -- 172.21.15.164:0/3466673332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1aac100a80 msgr2=0x7f1aac198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:52.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.800+0000 7f1ab1d56640 1 --2- 172.21.15.164:0/3466673332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1aac100a80 0x7f1aac198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.800+0000 7f1ab1d56640 1 -- 172.21.15.164:0/3466673332 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aac19d3d0 con 0x7f1aac101420 2024-09-19T08:44:52.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.800+0000 7f1ab2d58640 1 --2- 172.21.15.164:0/3466673332 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1aac106a80 0x7f1aac19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:44:52.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.801+0000 7f1ab1d56640 1 --2- 172.21.15.164:0/3466673332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1aac101420 0x7f1aac198960 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00c920 tx=0x7f1a9c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:52.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.801+0000 7f1aa37fe640 1 -- 172.21.15.164:0/3466673332 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c007c70 con 0x7f1aac101420 2024-09-19T08:44:52.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.801+0000 7f1aa37fe640 1 -- 172.21.15.164:0/3466673332 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a9c007e10 con 0x7f1aac101420 2024-09-19T08:44:52.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.801+0000 7f1ab47bb640 1 -- 172.21.15.164:0/3466673332 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aac19d6c0 con 0x7f1aac101420 2024-09-19T08:44:52.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.801+0000 7f1aa37fe640 1 -- 172.21.15.164:0/3466673332 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c018890 con 0x7f1aac101420 2024-09-19T08:44:52.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.801+0000 7f1ab47bb640 1 -- 172.21.15.164:0/3466673332 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aac1a4fa0 con 0x7f1aac101420 2024-09-19T08:44:52.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.802+0000 7f1ab47bb640 1 -- 172.21.15.164:0/3466673332 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aac100ee0 con 0x7f1aac101420 2024-09-19T08:44:52.551 INFO:teuthology.orchestra.run.smithi164.stdout:133143986236 2024-09-19T08:44:52.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.803+0000 7f1aa37fe640 1 -- 172.21.15.164:0/3466673332 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f1a9c018a70 con 0x7f1aac101420 2024-09-19T08:44:52.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.803+0000 7f1aa37fe640 1 --2- 172.21.15.164:0/3466673332 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1a980a32d0 0x7f1a980a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:52.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.804+0000 7f1ab2557640 1 --2- 172.21.15.164:0/3466673332 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1a980a32d0 0x7f1a980a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:52.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.804+0000 7f1aa37fe640 1 -- 172.21.15.164:0/3466673332 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1a9c0cfaf0 con 0x7f1aac101420 2024-09-19T08:44:52.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.804+0000 7f1ab2557640 1 --2- 172.21.15.164:0/3466673332 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1a980a32d0 0x7f1a980a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8009900 tx=0x7f1aa80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:52.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.808+0000 7f1aa37fe640 1 -- 172.21.15.164:0/3466673332 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1a9c014030 con 0x7f1aac101420 2024-09-19T08:44:52.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.980+0000 7f1ab47bb640 1 -- 172.21.15.164:0/3466673332 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f1aac199780 con 0x7f1aac101420 2024-09-19T08:44:52.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.980+0000 7f1aa37fe640 1 -- 172.21.15.164:0/3466673332 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1aac199780 con 0x7f1aac101420 2024-09-19T08:44:52.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.983+0000 7f1ab47bb640 1 -- 172.21.15.164:0/3466673332 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1a980a32d0 msgr2=0x7f1a980a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:52.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.983+0000 7f1ab47bb640 1 --2- 172.21.15.164:0/3466673332 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1a980a32d0 0x7f1a980a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8009900 tx=0x7f1aa80058d0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.983+0000 7f1ab47bb640 1 -- 172.21.15.164:0/3466673332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1aac101420 msgr2=0x7f1aac198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:52.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.983+0000 7f1ab47bb640 1 --2- 172.21.15.164:0/3466673332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1aac101420 0x7f1aac198960 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00c920 tx=0x7f1a9c00cde0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.983+0000 7f1ab47bb640 1 -- 172.21.15.164:0/3466673332 shutdown_connections 2024-09-19T08:44:52.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.983+0000 7f1ab47bb640 1 --2- 172.21.15.164:0/3466673332 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1a980a32d0 0x7f1a980a5790 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.983+0000 7f1ab47bb640 1 --2- 172.21.15.164:0/3466673332 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1aac106a80 0x7f1aac19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.983+0000 7f1ab47bb640 1 --2- 172.21.15.164:0/3466673332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1aac101420 0x7f1aac198960 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.983+0000 7f1ab47bb640 1 --2- 172.21.15.164:0/3466673332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1aac100a80 0x7f1aac198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.983+0000 7f1ab47bb640 1 -- 172.21.15.164:0/3466673332 >> 172.21.15.164:0/3466673332 conn(0x7f1aac0fc7a0 msgr2=0x7f1aac1048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:52.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.983+0000 7f1ab47bb640 1 -- 172.21.15.164:0/3466673332 shutdown_connections 2024-09-19T08:44:52.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:51.983+0000 7f1ab47bb640 1 -- 172.21.15.164:0/3466673332 wait complete. 2024-09-19T08:44:52.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.593+0000 7f16bb73e640 1 -- 172.21.15.164:0/3283932590 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f16b41ab990 con 0x7f16b4105f40 2024-09-19T08:44:52.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.593+0000 7f16aa7fc640 1 -- 172.21.15.164:0/3283932590 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f16ac09bb90 con 0x7f16b4105f40 2024-09-19T08:44:52.594 INFO:teuthology.orchestra.run.smithi164.stdout:103079215169 2024-09-19T08:44:52.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.595+0000 7f16bb73e640 1 -- 172.21.15.164:0/3283932590 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f16a00a3350 msgr2=0x7f16a00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:52.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.595+0000 7f16bb73e640 1 --2- 172.21.15.164:0/3283932590 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f16a00a3350 0x7f16a00a5810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f16b0000f80 tx=0x7f16b0005f30 comp rx=0 tx=0).stop 2024-09-19T08:44:52.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.595+0000 7f16bb73e640 1 -- 172.21.15.164:0/3283932590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16b4105f40 msgr2=0x7f16b419e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:52.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.595+0000 7f16bb73e640 1 --2- 172.21.15.164:0/3283932590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16b4105f40 0x7f16b419e940 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f16ac00c970 tx=0x7f16ac00ce30 comp rx=0 tx=0).stop 2024-09-19T08:44:52.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.595+0000 7f16bb73e640 1 -- 172.21.15.164:0/3283932590 shutdown_connections 2024-09-19T08:44:52.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.595+0000 7f16bb73e640 1 --2- 172.21.15.164:0/3283932590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16b4105f40 0x7f16b419e940 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.595+0000 7f16bb73e640 1 --2- 172.21.15.164:0/3283932590 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f16a00a3350 0x7f16a00a5810 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.595+0000 7f16bb73e640 1 --2- 172.21.15.164:0/3283932590 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16b4101450 0x7f16b41a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.595+0000 7f16bb73e640 1 --2- 172.21.15.164:0/3283932590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16b4100b30 0x7f16b41a4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.595+0000 7f16bb73e640 1 -- 172.21.15.164:0/3283932590 >> 172.21.15.164:0/3283932590 conn(0x7f16b40fc7a0 msgr2=0x7f16b410f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:52.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.595+0000 7f16bb73e640 1 -- 172.21.15.164:0/3283932590 shutdown_connections 2024-09-19T08:44:52.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:52.595+0000 7f16bb73e640 1 -- 172.21.15.164:0/3283932590 wait complete. 2024-09-19T08:44:52.757 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:53.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:52 smithi170 ceph-mon[35072]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-09-19T08:44:53.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:52 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3466673332' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T08:44:53.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:52 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/770268173' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T08:44:53.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:52 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3283932590' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T08:44:53.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:52 smithi173 ceph-mon[35306]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-09-19T08:44:53.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:52 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3466673332' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T08:44:53.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:52 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/770268173' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T08:44:53.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:52 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3283932590' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T08:44:53.178 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:52 smithi164 ceph-mon[30928]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-09-19T08:44:53.178 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:52 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3466673332' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T08:44:53.178 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:52 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/770268173' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T08:44:53.178 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:52 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3283932590' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T08:44:53.206 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986233 got 133143986236 for osd.4 2024-09-19T08:44:53.206 DEBUG:teuthology.parallel:result is None 2024-09-19T08:44:53.646 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:54.075 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215165 got 103079215169 for osd.3 2024-09-19T08:44:54.076 DEBUG:teuthology.parallel:result is None 2024-09-19T08:44:54.560 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234022 got 231928234024 for osd.8 2024-09-19T08:44:54.560 DEBUG:teuthology.parallel:result is None 2024-09-19T08:44:55.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:54 smithi164 ceph-mon[30928]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:44:55.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:54 smithi170 ceph-mon[35072]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:44:55.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:54 smithi173 ceph-mon[35306]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:44:55.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.281+0000 7f5e027bc640 1 -- 172.21.15.164:0/3580201488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5dfc100e20 msgr2=0x7f5dfc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:55.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.281+0000 7f5e027bc640 1 --2- 172.21.15.164:0/3580201488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5dfc100e20 0x7f5dfc101200 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5dec009900 tx=0x7f5dec02a480 comp rx=0 tx=0).stop 2024-09-19T08:44:55.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.282+0000 7f5e027bc640 1 -- 172.21.15.164:0/3580201488 shutdown_connections 2024-09-19T08:44:55.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.282+0000 7f5e027bc640 1 --2- 172.21.15.164:0/3580201488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5dfc102ef0 0x7f5dfc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:55.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.282+0000 7f5e027bc640 1 --2- 172.21.15.164:0/3580201488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dfc102570 0x7f5dfc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:55.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.282+0000 7f5e027bc640 1 --2- 172.21.15.164:0/3580201488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5dfc100e20 0x7f5dfc101200 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:55.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.282+0000 7f5e027bc640 1 -- 172.21.15.164:0/3580201488 >> 172.21.15.164:0/3580201488 conn(0x7f5dfc0fc7d0 msgr2=0x7f5dfc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:55.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.282+0000 7f5e027bc640 1 -- 172.21.15.164:0/3580201488 shutdown_connections 2024-09-19T08:44:55.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.282+0000 7f5e027bc640 1 -- 172.21.15.164:0/3580201488 wait complete. 2024-09-19T08:44:55.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.282+0000 7f5e027bc640 1 Processor -- start 2024-09-19T08:44:55.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.282+0000 7f5e027bc640 1 -- start start 2024-09-19T08:44:55.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.282+0000 7f5e027bc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5dfc100e20 0x7f5dfc197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:55.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.282+0000 7f5e027bc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5dfc102570 0x7f5dfc197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:55.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.282+0000 7f5e027bc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dfc102ef0 0x7f5dfc19be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:55.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.282+0000 7f5e027bc640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc111020 con 0x7f5dfc102570 2024-09-19T08:44:55.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.282+0000 7f5e027bc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc110ea0 con 0x7f5dfc102ef0 2024-09-19T08:44:55.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.282+0000 7f5e027bc640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc1111a0 con 0x7f5dfc100e20 2024-09-19T08:44:55.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.282+0000 7f5df3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5dfc102570 0x7f5dfc197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:55.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.283+0000 7f5df3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5dfc102570 0x7f5dfc197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48766/0 (socket says 172.21.15.164:48766) 2024-09-19T08:44:55.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.283+0000 7f5df3fff640 1 -- 172.21.15.164:0/4035987051 learned_addr learned my addr 172.21.15.164:0/4035987051 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:55.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.283+0000 7f5dfbfff640 1 --2- 172.21.15.164:0/4035987051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5dfc100e20 0x7f5dfc197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:55.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.283+0000 7f5e00d59640 1 --2- 172.21.15.164:0/4035987051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dfc102ef0 0x7f5dfc19be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:55.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.283+0000 7f5df3fff640 1 -- 172.21.15.164:0/4035987051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5dfc100e20 msgr2=0x7f5dfc197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:55.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.283+0000 7f5df3fff640 1 --2- 172.21.15.164:0/4035987051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5dfc100e20 0x7f5dfc197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:55.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.283+0000 7f5df3fff640 1 -- 172.21.15.164:0/4035987051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dfc102ef0 msgr2=0x7f5dfc19be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:55.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.283+0000 7f5df3fff640 1 --2- 172.21.15.164:0/4035987051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dfc102ef0 0x7f5dfc19be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:55.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.283+0000 7f5df3fff640 1 -- 172.21.15.164:0/4035987051 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dfc19c530 con 0x7f5dfc102570 2024-09-19T08:44:55.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.283+0000 7f5dfbfff640 1 --2- 172.21.15.164:0/4035987051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5dfc100e20 0x7f5dfc197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:44:55.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.283+0000 7f5df3fff640 1 --2- 172.21.15.164:0/4035987051 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5dfc102570 0x7f5dfc197ac0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5de400b4a0 tx=0x7f5de400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:55.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.283+0000 7f5df9ffb640 1 -- 172.21.15.164:0/4035987051 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4014070 con 0x7f5dfc102570 2024-09-19T08:44:55.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.283+0000 7f5df9ffb640 1 -- 172.21.15.164:0/4035987051 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5de4010af0 con 0x7f5dfc102570 2024-09-19T08:44:55.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.283+0000 7f5df9ffb640 1 -- 172.21.15.164:0/4035987051 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4017d10 con 0x7f5dfc102570 2024-09-19T08:44:55.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.283+0000 7f5e027bc640 1 -- 172.21.15.164:0/4035987051 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dfc19c820 con 0x7f5dfc102570 2024-09-19T08:44:55.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.283+0000 7f5e027bc640 1 -- 172.21.15.164:0/4035987051 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dfc1a4100 con 0x7f5dfc102570 2024-09-19T08:44:55.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.284+0000 7f5e027bc640 1 -- 172.21.15.164:0/4035987051 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dfc101280 con 0x7f5dfc102570 2024-09-19T08:44:55.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.285+0000 7f5df9ffb640 1 -- 172.21.15.164:0/4035987051 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f5de4017070 con 0x7f5dfc102570 2024-09-19T08:44:55.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.286+0000 7f5df9ffb640 1 --2- 172.21.15.164:0/4035987051 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f5de80a3350 0x7f5de80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:55.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.286+0000 7f5dfbfff640 1 --2- 172.21.15.164:0/4035987051 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f5de80a3350 0x7f5de80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:55.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.286+0000 7f5df9ffb640 1 -- 172.21.15.164:0/4035987051 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5de40cec30 con 0x7f5dfc102570 2024-09-19T08:44:55.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.286+0000 7f5dfbfff640 1 --2- 172.21.15.164:0/4035987051 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f5de80a3350 0x7f5de80a5810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5dec005ab0 tx=0x7f5dec005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:55.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.290+0000 7f5df9ffb640 1 -- 172.21.15.164:0/4035987051 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5de409ba00 con 0x7f5dfc102570 2024-09-19T08:44:55.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.462+0000 7f5e027bc640 1 -- 172.21.15.164:0/4035987051 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f5dfc1090c0 con 0x7f5dfc102570 2024-09-19T08:44:55.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.462+0000 7f5df9ffb640 1 -- 172.21.15.164:0/4035987051 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5de409b300 con 0x7f5dfc102570 2024-09-19T08:44:55.463 INFO:teuthology.orchestra.run.smithi164.stdout:261993005091 2024-09-19T08:44:55.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.464+0000 7f5e027bc640 1 -- 172.21.15.164:0/4035987051 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f5de80a3350 msgr2=0x7f5de80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:55.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.464+0000 7f5e027bc640 1 --2- 172.21.15.164:0/4035987051 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f5de80a3350 0x7f5de80a5810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5dec005ab0 tx=0x7f5dec005a20 comp rx=0 tx=0).stop 2024-09-19T08:44:55.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.464+0000 7f5e027bc640 1 -- 172.21.15.164:0/4035987051 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5dfc102570 msgr2=0x7f5dfc197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:55.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.464+0000 7f5e027bc640 1 --2- 172.21.15.164:0/4035987051 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5dfc102570 0x7f5dfc197ac0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5de400b4a0 tx=0x7f5de400b960 comp rx=0 tx=0).stop 2024-09-19T08:44:55.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.464+0000 7f5e027bc640 1 -- 172.21.15.164:0/4035987051 shutdown_connections 2024-09-19T08:44:55.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.465+0000 7f5e027bc640 1 --2- 172.21.15.164:0/4035987051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dfc102ef0 0x7f5dfc19be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:55.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.465+0000 7f5e027bc640 1 --2- 172.21.15.164:0/4035987051 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5dfc102570 0x7f5dfc197ac0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:55.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.465+0000 7f5e027bc640 1 --2- 172.21.15.164:0/4035987051 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f5de80a3350 0x7f5de80a5810 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:55.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.465+0000 7f5e027bc640 1 --2- 172.21.15.164:0/4035987051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5dfc100e20 0x7f5dfc197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:55.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.465+0000 7f5e027bc640 1 -- 172.21.15.164:0/4035987051 >> 172.21.15.164:0/4035987051 conn(0x7f5dfc0fc7d0 msgr2=0x7f5dfc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:55.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.465+0000 7f5e027bc640 1 -- 172.21.15.164:0/4035987051 shutdown_connections 2024-09-19T08:44:55.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.465+0000 7f5e027bc640 1 -- 172.21.15.164:0/4035987051 wait complete. 2024-09-19T08:44:55.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.524+0000 7f2820115640 1 -- 172.21.15.164:0/3234740534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2818106f10 msgr2=0x7f281810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:55.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.524+0000 7f2820115640 1 --2- 172.21.15.164:0/3234740534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2818106f10 0x7f281810ea90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2810009900 tx=0x7f281002a410 comp rx=0 tx=0).stop 2024-09-19T08:44:55.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.524+0000 7f2820115640 1 -- 172.21.15.164:0/3234740534 shutdown_connections 2024-09-19T08:44:55.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.524+0000 7f2820115640 1 --2- 172.21.15.164:0/3234740534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2818106f10 0x7f281810ea90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:55.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.524+0000 7f2820115640 1 --2- 172.21.15.164:0/3234740534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2818102790 0x7f28181067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:55.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.524+0000 7f2820115640 1 --2- 172.21.15.164:0/3234740534 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2818101de0 0x7f28181021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:55.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.524+0000 7f2820115640 1 -- 172.21.15.164:0/3234740534 >> 172.21.15.164:0/3234740534 conn(0x7f28180fd8c0 msgr2=0x7f28180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:55.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.524+0000 7f2820115640 1 -- 172.21.15.164:0/3234740534 shutdown_connections 2024-09-19T08:44:55.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.524+0000 7f2820115640 1 -- 172.21.15.164:0/3234740534 wait complete. 2024-09-19T08:44:55.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.525+0000 7f2820115640 1 Processor -- start 2024-09-19T08:44:55.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.525+0000 7f2820115640 1 -- start start 2024-09-19T08:44:55.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.525+0000 7f2820115640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2818101de0 0x7f28181983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:55.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.525+0000 7f2820115640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2818102790 0x7f2818198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:55.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.525+0000 7f2820115640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2818106f10 0x7f281819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:55.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.525+0000 7f2820115640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2818111f70 con 0x7f2818101de0 2024-09-19T08:44:55.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.525+0000 7f2820115640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2818111df0 con 0x7f2818106f10 2024-09-19T08:44:55.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.525+0000 7f2820115640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f28181120f0 con 0x7f2818102790 2024-09-19T08:44:55.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.525+0000 7f281deb1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2818101de0 0x7f28181983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:55.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.525+0000 7f281deb1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2818101de0 0x7f28181983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48790/0 (socket says 172.21.15.164:48790) 2024-09-19T08:44:55.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.525+0000 7f281deb1640 1 -- 172.21.15.164:0/2980669828 learned_addr learned my addr 172.21.15.164:0/2980669828 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:55.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.526+0000 7f281e6b2640 1 --2- 172.21.15.164:0/2980669828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2818106f10 0x7f281819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:55.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.526+0000 7f281d6b0640 1 --2- 172.21.15.164:0/2980669828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2818102790 0x7f2818198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:55.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.526+0000 7f281deb1640 1 -- 172.21.15.164:0/2980669828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2818102790 msgr2=0x7f2818198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:55.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.526+0000 7f281deb1640 1 --2- 172.21.15.164:0/2980669828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2818102790 0x7f2818198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:55.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.526+0000 7f281deb1640 1 -- 172.21.15.164:0/2980669828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2818106f10 msgr2=0x7f281819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:55.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.526+0000 7f281deb1640 1 --2- 172.21.15.164:0/2980669828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2818106f10 0x7f281819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:55.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.526+0000 7f281deb1640 1 -- 172.21.15.164:0/2980669828 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281819d380 con 0x7f2818101de0 2024-09-19T08:44:55.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.526+0000 7f281d6b0640 1 --2- 172.21.15.164:0/2980669828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2818102790 0x7f2818198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:44:55.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.526+0000 7f281e6b2640 1 --2- 172.21.15.164:0/2980669828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2818106f10 0x7f281819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:44:55.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.526+0000 7f281deb1640 1 --2- 172.21.15.164:0/2980669828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2818101de0 0x7f28181983d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f281400eb90 tx=0x7f281400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:55.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.526+0000 7f280effd640 1 -- 172.21.15.164:0/2980669828 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2814004d20 con 0x7f2818101de0 2024-09-19T08:44:55.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.526+0000 7f280effd640 1 -- 172.21.15.164:0/2980669828 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f281400ee90 con 0x7f2818101de0 2024-09-19T08:44:55.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.526+0000 7f2820115640 1 -- 172.21.15.164:0/2980669828 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f281819d670 con 0x7f2818101de0 2024-09-19T08:44:55.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.526+0000 7f280effd640 1 -- 172.21.15.164:0/2980669828 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2814010830 con 0x7f2818101de0 2024-09-19T08:44:55.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.526+0000 7f2820115640 1 -- 172.21.15.164:0/2980669828 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28181a4f50 con 0x7f2818101de0 2024-09-19T08:44:55.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.527+0000 7f2820115640 1 -- 172.21.15.164:0/2980669828 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28181021c0 con 0x7f2818101de0 2024-09-19T08:44:55.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.528+0000 7f280effd640 1 -- 172.21.15.164:0/2980669828 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f2814005020 con 0x7f2818101de0 2024-09-19T08:44:55.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.529+0000 7f280effd640 1 --2- 172.21.15.164:0/2980669828 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f28040a32d0 0x7f28040a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:55.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.529+0000 7f281d6b0640 1 --2- 172.21.15.164:0/2980669828 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f28040a32d0 0x7f28040a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:55.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.529+0000 7f280effd640 1 -- 172.21.15.164:0/2980669828 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f28140d0150 con 0x7f2818101de0 2024-09-19T08:44:55.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.529+0000 7f281d6b0640 1 --2- 172.21.15.164:0/2980669828 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f28040a32d0 0x7f28040a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f28181088e0 tx=0x7f2808005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:55.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.533+0000 7f280effd640 1 -- 172.21.15.164:0/2980669828 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f281409ce10 con 0x7f2818101de0 2024-09-19T08:44:55.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.702+0000 7f2820115640 1 -- 172.21.15.164:0/2980669828 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f281810a180 con 0x7f2818101de0 2024-09-19T08:44:55.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.703+0000 7f280effd640 1 -- 172.21.15.164:0/2980669828 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f281409c710 con 0x7f2818101de0 2024-09-19T08:44:55.703 INFO:teuthology.orchestra.run.smithi164.stdout:34359738448 2024-09-19T08:44:55.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.705+0000 7f2820115640 1 -- 172.21.15.164:0/2980669828 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f28040a32d0 msgr2=0x7f28040a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:55.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.705+0000 7f2820115640 1 --2- 172.21.15.164:0/2980669828 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f28040a32d0 0x7f28040a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f28181088e0 tx=0x7f2808005f90 comp rx=0 tx=0).stop 2024-09-19T08:44:55.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.705+0000 7f2820115640 1 -- 172.21.15.164:0/2980669828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2818101de0 msgr2=0x7f28181983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:55.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.705+0000 7f2820115640 1 --2- 172.21.15.164:0/2980669828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2818101de0 0x7f28181983d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f281400eb90 tx=0x7f281400c6a0 comp rx=0 tx=0).stop 2024-09-19T08:44:55.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.705+0000 7f2820115640 1 -- 172.21.15.164:0/2980669828 shutdown_connections 2024-09-19T08:44:55.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.705+0000 7f2820115640 1 --2- 172.21.15.164:0/2980669828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2818106f10 0x7f281819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:55.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.705+0000 7f2820115640 1 --2- 172.21.15.164:0/2980669828 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f28040a32d0 0x7f28040a5790 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:55.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.705+0000 7f2820115640 1 --2- 172.21.15.164:0/2980669828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2818102790 0x7f2818198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:55.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.705+0000 7f2820115640 1 --2- 172.21.15.164:0/2980669828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2818101de0 0x7f28181983d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:55.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.705+0000 7f2820115640 1 -- 172.21.15.164:0/2980669828 >> 172.21.15.164:0/2980669828 conn(0x7f28180fd8c0 msgr2=0x7f28180fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:55.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.705+0000 7f2820115640 1 -- 172.21.15.164:0/2980669828 shutdown_connections 2024-09-19T08:44:55.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:55.705+0000 7f2820115640 1 -- 172.21.15.164:0/2980669828 wait complete. 2024-09-19T08:44:56.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:55 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/4035987051' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T08:44:56.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:55 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2980669828' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T08:44:56.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:55 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/4035987051' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T08:44:56.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:55 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2980669828' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T08:44:56.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:55 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/4035987051' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T08:44:56.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:55 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2980669828' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T08:44:56.427 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005088 got 261993005091 for osd.9 2024-09-19T08:44:56.428 DEBUG:teuthology.parallel:result is None 2024-09-19T08:44:57.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:56 smithi164 ceph-mon[30928]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T08:44:57.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:56 smithi170 ceph-mon[35072]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T08:44:57.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:56 smithi173 ceph-mon[35306]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T08:44:57.758 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738446 got 34359738448 for osd.0 2024-09-19T08:44:57.759 DEBUG:teuthology.parallel:result is None 2024-09-19T08:44:57.759 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T08:44:57.759 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T08:44:58.055 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:44:58.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.556+0000 7f542af72640 1 -- 172.21.15.164:0/2539662194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5424102ef0 msgr2=0x7f542410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:58.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.556+0000 7f542af72640 1 --2- 172.21.15.164:0/2539662194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5424102ef0 0x7f542410da40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f541c009ca0 tx=0x7f541c02a950 comp rx=0 tx=0).stop 2024-09-19T08:44:58.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.556+0000 7f542af72640 1 -- 172.21.15.164:0/2539662194 shutdown_connections 2024-09-19T08:44:58.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.556+0000 7f542af72640 1 --2- 172.21.15.164:0/2539662194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5424102ef0 0x7f542410da40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:58.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.556+0000 7f542af72640 1 --2- 172.21.15.164:0/2539662194 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5424102570 0x7f54241029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:58.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.556+0000 7f542af72640 1 --2- 172.21.15.164:0/2539662194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5424100e20 0x7f5424101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:58.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.556+0000 7f542af72640 1 -- 172.21.15.164:0/2539662194 >> 172.21.15.164:0/2539662194 conn(0x7f54240fc7d0 msgr2=0x7f54240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:58.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.556+0000 7f542af72640 1 -- 172.21.15.164:0/2539662194 shutdown_connections 2024-09-19T08:44:58.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.556+0000 7f542af72640 1 -- 172.21.15.164:0/2539662194 wait complete. 2024-09-19T08:44:58.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.557+0000 7f542af72640 1 Processor -- start 2024-09-19T08:44:58.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.557+0000 7f542af72640 1 -- start start 2024-09-19T08:44:58.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.557+0000 7f542af72640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5424100e20 0x7f54241992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:58.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.557+0000 7f542af72640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5424102570 0x7f5424199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:58.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.557+0000 7f542af72640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5424102ef0 0x7f54241a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:58.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.557+0000 7f542af72640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5424110f80 con 0x7f5424100e20 2024-09-19T08:44:58.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.557+0000 7f542af72640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5424110e00 con 0x7f5424102570 2024-09-19T08:44:58.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.557+0000 7f542af72640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5424111100 con 0x7f5424102ef0 2024-09-19T08:44:58.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.557+0000 7f5428d0e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5424100e20 0x7f54241992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:58.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.557+0000 7f5428d0e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5424100e20 0x7f54241992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48808/0 (socket says 172.21.15.164:48808) 2024-09-19T08:44:58.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.557+0000 7f5428d0e640 1 -- 172.21.15.164:0/2736387731 learned_addr learned my addr 172.21.15.164:0/2736387731 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:44:58.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.557+0000 7f5423fff640 1 --2- 172.21.15.164:0/2736387731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5424102570 0x7f5424199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:58.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.557+0000 7f542950f640 1 --2- 172.21.15.164:0/2736387731 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5424102ef0 0x7f54241a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:58.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.558+0000 7f5428d0e640 1 -- 172.21.15.164:0/2736387731 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5424102ef0 msgr2=0x7f54241a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:58.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.558+0000 7f5428d0e640 1 --2- 172.21.15.164:0/2736387731 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5424102ef0 0x7f54241a4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:58.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.558+0000 7f5428d0e640 1 -- 172.21.15.164:0/2736387731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5424102570 msgr2=0x7f5424199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:58.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.558+0000 7f5428d0e640 1 --2- 172.21.15.164:0/2736387731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5424102570 0x7f5424199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:58.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.558+0000 7f5428d0e640 1 -- 172.21.15.164:0/2736387731 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54241a4890 con 0x7f5424100e20 2024-09-19T08:44:58.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.558+0000 7f5423fff640 1 --2- 172.21.15.164:0/2736387731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5424102570 0x7f5424199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:44:58.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.558+0000 7f5428d0e640 1 --2- 172.21.15.164:0/2736387731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5424100e20 0x7f54241992d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f541800e9f0 tx=0x7f541800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:58.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.558+0000 7f5421ffb640 1 -- 172.21.15.164:0/2736387731 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541800cd80 con 0x7f5424100e20 2024-09-19T08:44:58.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.558+0000 7f5421ffb640 1 -- 172.21.15.164:0/2736387731 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5418010910 con 0x7f5424100e20 2024-09-19T08:44:58.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.558+0000 7f5421ffb640 1 -- 172.21.15.164:0/2736387731 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5418018990 con 0x7f5424100e20 2024-09-19T08:44:58.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.558+0000 7f542af72640 1 -- 172.21.15.164:0/2736387731 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54241970c0 con 0x7f5424100e20 2024-09-19T08:44:58.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.558+0000 7f542af72640 1 -- 172.21.15.164:0/2736387731 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54241975b0 con 0x7f5424100e20 2024-09-19T08:44:58.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.559+0000 7f542af72640 1 -- 172.21.15.164:0/2736387731 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5424101200 con 0x7f5424100e20 2024-09-19T08:44:58.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.561+0000 7f5421ffb640 1 -- 172.21.15.164:0/2736387731 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f5418010ac0 con 0x7f5424100e20 2024-09-19T08:44:58.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.561+0000 7f5421ffb640 1 --2- 172.21.15.164:0/2736387731 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f54140a32d0 0x7f54140a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:58.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.561+0000 7f5423fff640 1 --2- 172.21.15.164:0/2736387731 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f54140a32d0 0x7f54140a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:58.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.561+0000 7f5421ffb640 1 -- 172.21.15.164:0/2736387731 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f54180d01a0 con 0x7f5424100e20 2024-09-19T08:44:58.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.562+0000 7f5423fff640 1 --2- 172.21.15.164:0/2736387731 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f54140a32d0 0x7f54140a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f542406aac0 tx=0x7f5410005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:58.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.564+0000 7f5421ffb640 1 -- 172.21.15.164:0/2736387731 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f541809ce60 con 0x7f5424100e20 2024-09-19T08:44:58.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.741+0000 7f542af72640 1 -- 172.21.15.164:0/2736387731 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5424106f90 con 0x7f54140a32d0 2024-09-19T08:44:58.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.748+0000 7f5421ffb640 1 -- 172.21.15.164:0/2736387731 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258900 (secure 0 0 0) 0x7f5424106f90 con 0x7f54140a32d0 2024-09-19T08:44:58.751 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:44:58.752 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-09-19T08:44:58.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.751+0000 7f542af72640 1 -- 172.21.15.164:0/2736387731 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f54140a32d0 msgr2=0x7f54140a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:58.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.751+0000 7f542af72640 1 --2- 172.21.15.164:0/2736387731 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f54140a32d0 0x7f54140a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f542406aac0 tx=0x7f5410005f90 comp rx=0 tx=0).stop 2024-09-19T08:44:58.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.751+0000 7f542af72640 1 -- 172.21.15.164:0/2736387731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5424100e20 msgr2=0x7f54241992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:58.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.751+0000 7f542af72640 1 --2- 172.21.15.164:0/2736387731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5424100e20 0x7f54241992d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f541800e9f0 tx=0x7f541800eeb0 comp rx=0 tx=0).stop 2024-09-19T08:44:58.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.751+0000 7f542af72640 1 -- 172.21.15.164:0/2736387731 shutdown_connections 2024-09-19T08:44:58.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.751+0000 7f542af72640 1 --2- 172.21.15.164:0/2736387731 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f54140a32d0 0x7f54140a5790 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:58.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.751+0000 7f542af72640 1 --2- 172.21.15.164:0/2736387731 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5424102ef0 0x7f54241a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:58.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.751+0000 7f542af72640 1 --2- 172.21.15.164:0/2736387731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5424102570 0x7f5424199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:58.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.751+0000 7f542af72640 1 --2- 172.21.15.164:0/2736387731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5424100e20 0x7f54241992d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:58.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.751+0000 7f542af72640 1 -- 172.21.15.164:0/2736387731 >> 172.21.15.164:0/2736387731 conn(0x7f54240fc7d0 msgr2=0x7f5424106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:58.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.751+0000 7f542af72640 1 -- 172.21.15.164:0/2736387731 shutdown_connections 2024-09-19T08:44:58.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:44:58.751+0000 7f542af72640 1 -- 172.21.15.164:0/2736387731 wait complete. 2024-09-19T08:44:59.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:58 smithi164 ceph-mon[30928]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-19T08:44:59.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:58 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:44:59.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:58 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:44:59.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:58 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:44:59.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:58 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:44:59.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:58 smithi164 ceph-mon[30928]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:44:59.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:58 smithi170 ceph-mon[35072]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-19T08:44:59.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:58 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:44:59.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:58 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:44:59.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:58 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:44:59.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:58 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:44:59.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:58 smithi170 ceph-mon[35072]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:44:59.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:58 smithi173 ceph-mon[35306]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-19T08:44:59.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:58 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:44:59.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:58 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:44:59.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:58 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:44:59.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:58 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:44:59.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:58 smithi173 ceph-mon[35306]: from='mgr.14150 172.21.15.164:0/3718195962' entity='mgr.x' 2024-09-19T08:44:59.512 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-19T08:44:57.758695+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657021,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":689,"num_read_kb":471,"num_write":424,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":346,"ondisk_log_size":346,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":389468,"kb_used_data":11088,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124552356,"statfs":{"total":1151940427776,"available":1151541612544,"internally_reserved":0,"allocated":11354112,"data_stored":7190535,"data_compressed":109335,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":75,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002991"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.817273+0000","last_change":"2024-09-19T08:43:52.766709+0000","last_active":"2024-09-19T08:44:02.817273+0000","last_peered":"2024-09-19T08:44:02.817273+0000","last_clean":"2024-09-19T08:44:02.817273+0000","last_became_active":"2024-09-19T08:43:52.765970+0000","last_became_peered":"2024-09-19T08:43:52.765970+0000","last_unstale":"2024-09-19T08:44:02.817273+0000","last_undegraded":"2024-09-19T08:44:02.817273+0000","last_fullsized":"2024-09-19T08:44:02.817273+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:12.176870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819309+0000","last_change":"2024-09-19T08:43:52.766590+0000","last_active":"2024-09-19T08:44:02.819309+0000","last_peered":"2024-09-19T08:44:02.819309+0000","last_clean":"2024-09-19T08:44:02.819309+0000","last_became_active":"2024-09-19T08:43:52.766255+0000","last_became_peered":"2024-09-19T08:43:52.766255+0000","last_unstale":"2024-09-19T08:44:02.819309+0000","last_undegraded":"2024-09-19T08:44:02.819309+0000","last_fullsized":"2024-09-19T08:44:02.819309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:15:41.359806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818510+0000","last_change":"2024-09-19T08:43:52.761956+0000","last_active":"2024-09-19T08:44:02.818510+0000","last_peered":"2024-09-19T08:44:02.818510+0000","last_clean":"2024-09-19T08:44:02.818510+0000","last_became_active":"2024-09-19T08:43:52.761644+0000","last_became_peered":"2024-09-19T08:43:52.761644+0000","last_unstale":"2024-09-19T08:44:02.818510+0000","last_undegraded":"2024-09-19T08:44:02.818510+0000","last_fullsized":"2024-09-19T08:44:02.818510+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:13.770361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.079353+0000","last_change":"2024-09-19T08:43:52.764131+0000","last_active":"2024-09-19T08:44:03.079353+0000","last_peered":"2024-09-19T08:44:03.079353+0000","last_clean":"2024-09-19T08:44:03.079353+0000","last_became_active":"2024-09-19T08:43:52.763812+0000","last_became_peered":"2024-09-19T08:43:52.763812+0000","last_unstale":"2024-09-19T08:44:03.079353+0000","last_undegraded":"2024-09-19T08:44:03.079353+0000","last_fullsized":"2024-09-19T08:44:03.079353+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:32.272157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818711+0000","last_change":"2024-09-19T08:43:52.754673+0000","last_active":"2024-09-19T08:44:02.818711+0000","last_peered":"2024-09-19T08:44:02.818711+0000","last_clean":"2024-09-19T08:44:02.818711+0000","last_became_active":"2024-09-19T08:43:52.754144+0000","last_became_peered":"2024-09-19T08:43:52.754144+0000","last_unstale":"2024-09-19T08:44:02.818711+0000","last_undegraded":"2024-09-19T08:44:02.818711+0000","last_fullsized":"2024-09-19T08:44:02.818711+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:16.069260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.817142+0000","last_change":"2024-09-19T08:43:52.758939+0000","last_active":"2024-09-19T08:44:02.817142+0000","last_peered":"2024-09-19T08:44:02.817142+0000","last_clean":"2024-09-19T08:44:02.817142+0000","last_became_active":"2024-09-19T08:43:52.758611+0000","last_became_peered":"2024-09-19T08:43:52.758611+0000","last_unstale":"2024-09-19T08:44:02.817142+0000","last_undegraded":"2024-09-19T08:44:02.817142+0000","last_fullsized":"2024-09-19T08:44:02.817142+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:35:56.463545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821786+0000","last_change":"2024-09-19T08:43:52.770261+0000","last_active":"2024-09-19T08:44:02.821786+0000","last_peered":"2024-09-19T08:44:02.821786+0000","last_clean":"2024-09-19T08:44:02.821786+0000","last_became_active":"2024-09-19T08:43:52.768253+0000","last_became_peered":"2024-09-19T08:43:52.768253+0000","last_unstale":"2024-09-19T08:44:02.821786+0000","last_undegraded":"2024-09-19T08:44:02.821786+0000","last_fullsized":"2024-09-19T08:44:02.821786+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:40.756988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.493456+0000","last_change":"2024-09-19T08:43:52.768564+0000","last_active":"2024-09-19T08:44:03.493456+0000","last_peered":"2024-09-19T08:44:03.493456+0000","last_clean":"2024-09-19T08:44:03.493456+0000","last_became_active":"2024-09-19T08:43:52.768260+0000","last_became_peered":"2024-09-19T08:43:52.768260+0000","last_unstale":"2024-09-19T08:44:03.493456+0000","last_undegraded":"2024-09-19T08:44:03.493456+0000","last_fullsized":"2024-09-19T08:44:03.493456+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:16.528115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.820687+0000","last_change":"2024-09-19T08:43:52.769804+0000","last_active":"2024-09-19T08:44:02.820687+0000","last_peered":"2024-09-19T08:44:02.820687+0000","last_clean":"2024-09-19T08:44:02.820687+0000","last_became_active":"2024-09-19T08:43:52.769504+0000","last_became_peered":"2024-09-19T08:43:52.769504+0000","last_unstale":"2024-09-19T08:44:02.820687+0000","last_undegraded":"2024-09-19T08:44:02.820687+0000","last_fullsized":"2024-09-19T08:44:02.820687+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:22:38.514410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.817329+0000","last_change":"2024-09-19T08:43:52.763278+0000","last_active":"2024-09-19T08:44:02.817329+0000","last_peered":"2024-09-19T08:44:02.817329+0000","last_clean":"2024-09-19T08:44:02.817329+0000","last_became_active":"2024-09-19T08:43:52.762985+0000","last_became_peered":"2024-09-19T08:43:52.762985+0000","last_unstale":"2024-09-19T08:44:02.817329+0000","last_undegraded":"2024-09-19T08:44:02.817329+0000","last_fullsized":"2024-09-19T08:44:02.817329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:30.048782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.822068+0000","last_change":"2024-09-19T08:43:52.770864+0000","last_active":"2024-09-19T08:44:02.822068+0000","last_peered":"2024-09-19T08:44:02.822068+0000","last_clean":"2024-09-19T08:44:02.822068+0000","last_became_active":"2024-09-19T08:43:52.770518+0000","last_became_peered":"2024-09-19T08:43:52.770518+0000","last_unstale":"2024-09-19T08:44:02.822068+0000","last_undegraded":"2024-09-19T08:44:02.822068+0000","last_fullsized":"2024-09-19T08:44:02.822068+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:08.931452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.079430+0000","last_change":"2024-09-19T08:43:52.757464+0000","last_active":"2024-09-19T08:44:03.079430+0000","last_peered":"2024-09-19T08:44:03.079430+0000","last_clean":"2024-09-19T08:44:03.079430+0000","last_became_active":"2024-09-19T08:43:52.757101+0000","last_became_peered":"2024-09-19T08:43:52.757101+0000","last_unstale":"2024-09-19T08:44:03.079430+0000","last_undegraded":"2024-09-19T08:44:03.079430+0000","last_fullsized":"2024-09-19T08:44:03.079430+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:17.395735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818784+0000","last_change":"2024-09-19T08:43:52.767063+0000","last_active":"2024-09-19T08:44:02.818784+0000","last_peered":"2024-09-19T08:44:02.818784+0000","last_clean":"2024-09-19T08:44:02.818784+0000","last_became_active":"2024-09-19T08:43:52.766755+0000","last_became_peered":"2024-09-19T08:43:52.766755+0000","last_unstale":"2024-09-19T08:44:02.818784+0000","last_undegraded":"2024-09-19T08:44:02.818784+0000","last_fullsized":"2024-09-19T08:44:02.818784+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:30.530937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818799+0000","last_change":"2024-09-19T08:43:52.766515+0000","last_active":"2024-09-19T08:44:02.818799+0000","last_peered":"2024-09-19T08:44:02.818799+0000","last_clean":"2024-09-19T08:44:02.818799+0000","last_became_active":"2024-09-19T08:43:52.765847+0000","last_became_peered":"2024-09-19T08:43:52.765847+0000","last_unstale":"2024-09-19T08:44:02.818799+0000","last_undegraded":"2024-09-19T08:44:02.818799+0000","last_fullsized":"2024-09-19T08:44:02.818799+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:50:35.948663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818391+0000","last_change":"2024-09-19T08:43:52.773702+0000","last_active":"2024-09-19T08:44:02.818391+0000","last_peered":"2024-09-19T08:44:02.818391+0000","last_clean":"2024-09-19T08:44:02.818391+0000","last_became_active":"2024-09-19T08:43:52.773324+0000","last_became_peered":"2024-09-19T08:43:52.773324+0000","last_unstale":"2024-09-19T08:44:02.818391+0000","last_undegraded":"2024-09-19T08:44:02.818391+0000","last_fullsized":"2024-09-19T08:44:02.818391+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:34.368014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.820811+0000","last_change":"2024-09-19T08:43:52.762745+0000","last_active":"2024-09-19T08:44:02.820811+0000","last_peered":"2024-09-19T08:44:02.820811+0000","last_clean":"2024-09-19T08:44:02.820811+0000","last_became_active":"2024-09-19T08:43:52.762172+0000","last_became_peered":"2024-09-19T08:43:52.762172+0000","last_unstale":"2024-09-19T08:44:02.820811+0000","last_undegraded":"2024-09-19T08:44:02.820811+0000","last_fullsized":"2024-09-19T08:44:02.820811+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:57.369775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.817462+0000","last_change":"2024-09-19T08:43:52.767118+0000","last_active":"2024-09-19T08:44:02.817462+0000","last_peered":"2024-09-19T08:44:02.817462+0000","last_clean":"2024-09-19T08:44:02.817462+0000","last_became_active":"2024-09-19T08:43:52.766388+0000","last_became_peered":"2024-09-19T08:43:52.766388+0000","last_unstale":"2024-09-19T08:44:02.817462+0000","last_undegraded":"2024-09-19T08:44:02.817462+0000","last_fullsized":"2024-09-19T08:44:02.817462+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:34.182923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821529+0000","last_change":"2024-09-19T08:43:52.765008+0000","last_active":"2024-09-19T08:44:02.821529+0000","last_peered":"2024-09-19T08:44:02.821529+0000","last_clean":"2024-09-19T08:44:02.821529+0000","last_became_active":"2024-09-19T08:43:52.764558+0000","last_became_peered":"2024-09-19T08:43:52.764558+0000","last_unstale":"2024-09-19T08:44:02.821529+0000","last_undegraded":"2024-09-19T08:44:02.821529+0000","last_fullsized":"2024-09-19T08:44:02.821529+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:52:42.756421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819875+0000","last_change":"2024-09-19T08:43:52.764376+0000","last_active":"2024-09-19T08:44:02.819875+0000","last_peered":"2024-09-19T08:44:02.819875+0000","last_clean":"2024-09-19T08:44:02.819875+0000","last_became_active":"2024-09-19T08:43:52.764048+0000","last_became_peered":"2024-09-19T08:43:52.764048+0000","last_unstale":"2024-09-19T08:44:02.819875+0000","last_undegraded":"2024-09-19T08:44:02.819875+0000","last_fullsized":"2024-09-19T08:44:02.819875+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:56.732094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.827925+0000","last_change":"2024-09-19T08:44:02.827925+0000","last_active":"2024-09-19T08:44:02.827925+0000","last_peered":"2024-09-19T08:44:02.827925+0000","last_clean":"2024-09-19T08:44:02.827925+0000","last_became_active":"2024-09-19T08:44:02.827269+0000","last_became_peered":"2024-09-19T08:44:02.827269+0000","last_unstale":"2024-09-19T08:44:02.827925+0000","last_undegraded":"2024-09-19T08:44:02.827925+0000","last_fullsized":"2024-09-19T08:44:02.827925+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:31:54.556786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818875+0000","last_change":"2024-09-19T08:43:52.762190+0000","last_active":"2024-09-19T08:44:02.818875+0000","last_peered":"2024-09-19T08:44:02.818875+0000","last_clean":"2024-09-19T08:44:02.818875+0000","last_became_active":"2024-09-19T08:43:52.761818+0000","last_became_peered":"2024-09-19T08:43:52.761818+0000","last_unstale":"2024-09-19T08:44:02.818875+0000","last_undegraded":"2024-09-19T08:44:02.818875+0000","last_fullsized":"2024-09-19T08:44:02.818875+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:19:55.246779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.820265+0000","last_change":"2024-09-19T08:43:52.761069+0000","last_active":"2024-09-19T08:44:02.820265+0000","last_peered":"2024-09-19T08:44:02.820265+0000","last_clean":"2024-09-19T08:44:02.820265+0000","last_became_active":"2024-09-19T08:43:52.760705+0000","last_became_peered":"2024-09-19T08:43:52.760705+0000","last_unstale":"2024-09-19T08:44:02.820265+0000","last_undegraded":"2024-09-19T08:44:02.820265+0000","last_fullsized":"2024-09-19T08:44:02.820265+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:46:27.059026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.493512+0000","last_change":"2024-09-19T08:43:52.773449+0000","last_active":"2024-09-19T08:44:03.493512+0000","last_peered":"2024-09-19T08:44:03.493512+0000","last_clean":"2024-09-19T08:44:03.493512+0000","last_became_active":"2024-09-19T08:43:52.772929+0000","last_became_peered":"2024-09-19T08:43:52.772929+0000","last_unstale":"2024-09-19T08:44:03.493512+0000","last_undegraded":"2024-09-19T08:44:03.493512+0000","last_fullsized":"2024-09-19T08:44:03.493512+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:00:38.929199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821652+0000","last_change":"2024-09-19T08:43:52.773885+0000","last_active":"2024-09-19T08:44:02.821652+0000","last_peered":"2024-09-19T08:44:02.821652+0000","last_clean":"2024-09-19T08:44:02.821652+0000","last_became_active":"2024-09-19T08:43:52.772856+0000","last_became_peered":"2024-09-19T08:43:52.772856+0000","last_unstale":"2024-09-19T08:44:02.821652+0000","last_undegraded":"2024-09-19T08:44:02.821652+0000","last_fullsized":"2024-09-19T08:44:02.821652+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:13:22.581335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.493984+0000","last_change":"2024-09-19T08:43:52.762909+0000","last_active":"2024-09-19T08:44:03.493984+0000","last_peered":"2024-09-19T08:44:03.493984+0000","last_clean":"2024-09-19T08:44:03.493984+0000","last_became_active":"2024-09-19T08:43:52.762600+0000","last_became_peered":"2024-09-19T08:43:52.762600+0000","last_unstale":"2024-09-19T08:44:03.493984+0000","last_undegraded":"2024-09-19T08:44:03.493984+0000","last_fullsized":"2024-09-19T08:44:03.493984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:46.098294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.820300+0000","last_change":"2024-09-19T08:43:52.762861+0000","last_active":"2024-09-19T08:44:02.820300+0000","last_peered":"2024-09-19T08:44:02.820300+0000","last_clean":"2024-09-19T08:44:02.820300+0000","last_became_active":"2024-09-19T08:43:52.762525+0000","last_became_peered":"2024-09-19T08:43:52.762525+0000","last_unstale":"2024-09-19T08:44:02.820300+0000","last_undegraded":"2024-09-19T08:44:02.820300+0000","last_fullsized":"2024-09-19T08:44:02.820300+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:54.717249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.816836+0000","last_change":"2024-09-19T08:43:52.768370+0000","last_active":"2024-09-19T08:44:02.816836+0000","last_peered":"2024-09-19T08:44:02.816836+0000","last_clean":"2024-09-19T08:44:02.816836+0000","last_became_active":"2024-09-19T08:43:52.768055+0000","last_became_peered":"2024-09-19T08:43:52.768055+0000","last_unstale":"2024-09-19T08:44:02.816836+0000","last_undegraded":"2024-09-19T08:44:02.816836+0000","last_fullsized":"2024-09-19T08:44:02.816836+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:48.212862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819012+0000","last_change":"2024-09-19T08:43:52.758541+0000","last_active":"2024-09-19T08:44:02.819012+0000","last_peered":"2024-09-19T08:44:02.819012+0000","last_clean":"2024-09-19T08:44:02.819012+0000","last_became_active":"2024-09-19T08:43:52.758122+0000","last_became_peered":"2024-09-19T08:43:52.758122+0000","last_unstale":"2024-09-19T08:44:02.819012+0000","last_undegraded":"2024-09-19T08:44:02.819012+0000","last_fullsized":"2024-09-19T08:44:02.819012+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:54:40.681880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818709+0000","last_change":"2024-09-19T08:43:52.758109+0000","last_active":"2024-09-19T08:44:02.818709+0000","last_peered":"2024-09-19T08:44:02.818709+0000","last_clean":"2024-09-19T08:44:02.818709+0000","last_became_active":"2024-09-19T08:43:52.757808+0000","last_became_peered":"2024-09-19T08:43:52.757808+0000","last_unstale":"2024-09-19T08:44:02.818709+0000","last_undegraded":"2024-09-19T08:44:02.818709+0000","last_fullsized":"2024-09-19T08:44:02.818709+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:02:47.614562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819377+0000","last_change":"2024-09-19T08:43:52.772501+0000","last_active":"2024-09-19T08:44:02.819377+0000","last_peered":"2024-09-19T08:44:02.819377+0000","last_clean":"2024-09-19T08:44:02.819377+0000","last_became_active":"2024-09-19T08:43:52.771611+0000","last_became_peered":"2024-09-19T08:43:52.771611+0000","last_unstale":"2024-09-19T08:44:02.819377+0000","last_undegraded":"2024-09-19T08:44:02.819377+0000","last_fullsized":"2024-09-19T08:44:02.819377+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:49.537237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819339+0000","last_change":"2024-09-19T08:43:52.773664+0000","last_active":"2024-09-19T08:44:02.819339+0000","last_peered":"2024-09-19T08:44:02.819339+0000","last_clean":"2024-09-19T08:44:02.819339+0000","last_became_active":"2024-09-19T08:43:52.773014+0000","last_became_peered":"2024-09-19T08:43:52.773014+0000","last_unstale":"2024-09-19T08:44:02.819339+0000","last_undegraded":"2024-09-19T08:44:02.819339+0000","last_fullsized":"2024-09-19T08:44:02.819339+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:03.448825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.816727+0000","last_change":"2024-09-19T08:43:52.769578+0000","last_active":"2024-09-19T08:44:02.816727+0000","last_peered":"2024-09-19T08:44:02.816727+0000","last_clean":"2024-09-19T08:44:02.816727+0000","last_became_active":"2024-09-19T08:43:52.769252+0000","last_became_peered":"2024-09-19T08:43:52.769252+0000","last_unstale":"2024-09-19T08:44:02.816727+0000","last_undegraded":"2024-09-19T08:44:02.816727+0000","last_fullsized":"2024-09-19T08:44:02.816727+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:27.936118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818612+0000","last_change":"2024-09-19T08:43:52.770453+0000","last_active":"2024-09-19T08:44:02.818612+0000","last_peered":"2024-09-19T08:44:02.818612+0000","last_clean":"2024-09-19T08:44:02.818612+0000","last_became_active":"2024-09-19T08:43:52.770155+0000","last_became_peered":"2024-09-19T08:43:52.770155+0000","last_unstale":"2024-09-19T08:44:02.818612+0000","last_undegraded":"2024-09-19T08:44:02.818612+0000","last_fullsized":"2024-09-19T08:44:02.818612+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:12.893572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":82,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:53.703154+0000","last_change":"2024-09-19T08:43:50.751316+0000","last_active":"2024-09-19T08:44:53.703154+0000","last_peered":"2024-09-19T08:44:53.703154+0000","last_clean":"2024-09-19T08:44:53.703154+0000","last_became_active":"2024-09-19T08:43:50.750972+0000","last_became_peered":"2024-09-19T08:43:50.750972+0000","last_unstale":"2024-09-19T08:44:53.703154+0000","last_undegraded":"2024-09-19T08:44:53.703154+0000","last_fullsized":"2024-09-19T08:44:53.703154+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:40.654729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.822390+0000","last_change":"2024-09-19T08:43:52.773979+0000","last_active":"2024-09-19T08:44:02.822390+0000","last_peered":"2024-09-19T08:44:02.822390+0000","last_clean":"2024-09-19T08:44:02.822390+0000","last_became_active":"2024-09-19T08:43:52.773648+0000","last_became_peered":"2024-09-19T08:43:52.773648+0000","last_unstale":"2024-09-19T08:44:02.822390+0000","last_undegraded":"2024-09-19T08:44:02.822390+0000","last_fullsized":"2024-09-19T08:44:02.822390+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:36.061209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.839445+0000","last_change":"2024-09-19T08:43:50.750704+0000","last_active":"2024-09-19T08:44:03.839445+0000","last_peered":"2024-09-19T08:44:03.839445+0000","last_clean":"2024-09-19T08:44:03.839445+0000","last_became_active":"2024-09-19T08:43:50.750399+0000","last_became_peered":"2024-09-19T08:43:50.750399+0000","last_unstale":"2024-09-19T08:44:03.839445+0000","last_undegraded":"2024-09-19T08:44:03.839445+0000","last_fullsized":"2024-09-19T08:44:03.839445+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:25.447320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818514+0000","last_change":"2024-09-19T08:43:50.744919+0000","last_active":"2024-09-19T08:44:02.818514+0000","last_peered":"2024-09-19T08:44:02.818514+0000","last_clean":"2024-09-19T08:44:02.818514+0000","last_became_active":"2024-09-19T08:43:50.744604+0000","last_became_peered":"2024-09-19T08:43:50.744604+0000","last_unstale":"2024-09-19T08:44:02.818514+0000","last_undegraded":"2024-09-19T08:44:02.818514+0000","last_fullsized":"2024-09-19T08:44:02.818514+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:25:06.410093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.493518+0000","last_change":"2024-09-19T08:43:52.766779+0000","last_active":"2024-09-19T08:44:03.493518+0000","last_peered":"2024-09-19T08:44:03.493518+0000","last_clean":"2024-09-19T08:44:03.493518+0000","last_became_active":"2024-09-19T08:43:52.766484+0000","last_became_peered":"2024-09-19T08:43:52.766484+0000","last_unstale":"2024-09-19T08:44:03.493518+0000","last_undegraded":"2024-09-19T08:44:03.493518+0000","last_fullsized":"2024-09-19T08:44:03.493518+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:17:29.350778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.631853+0000","last_change":"2024-09-19T08:43:50.751282+0000","last_active":"2024-09-19T08:44:04.631853+0000","last_peered":"2024-09-19T08:44:04.631853+0000","last_clean":"2024-09-19T08:44:04.631853+0000","last_became_active":"2024-09-19T08:43:50.750754+0000","last_became_peered":"2024-09-19T08:43:50.750754+0000","last_unstale":"2024-09-19T08:44:04.631853+0000","last_undegraded":"2024-09-19T08:44:04.631853+0000","last_fullsized":"2024-09-19T08:44:04.631853+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:11.661626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821354+0000","last_change":"2024-09-19T08:43:52.756442+0000","last_active":"2024-09-19T08:44:02.821354+0000","last_peered":"2024-09-19T08:44:02.821354+0000","last_clean":"2024-09-19T08:44:02.821354+0000","last_became_active":"2024-09-19T08:43:52.756116+0000","last_became_peered":"2024-09-19T08:43:52.756116+0000","last_unstale":"2024-09-19T08:44:02.821354+0000","last_undegraded":"2024-09-19T08:44:02.821354+0000","last_fullsized":"2024-09-19T08:44:02.821354+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:40.029272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":130,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:55.705106+0000","last_change":"2024-09-19T08:43:50.749637+0000","last_active":"2024-09-19T08:44:55.705106+0000","last_peered":"2024-09-19T08:44:55.705106+0000","last_clean":"2024-09-19T08:44:55.705106+0000","last_became_active":"2024-09-19T08:43:50.749307+0000","last_became_peered":"2024-09-19T08:43:50.749307+0000","last_unstale":"2024-09-19T08:44:55.705106+0000","last_undegraded":"2024-09-19T08:44:55.705106+0000","last_fullsized":"2024-09-19T08:44:55.705106+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:23:33.409671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.820558+0000","last_change":"2024-09-19T08:43:52.763592+0000","last_active":"2024-09-19T08:44:02.820558+0000","last_peered":"2024-09-19T08:44:02.820558+0000","last_clean":"2024-09-19T08:44:02.820558+0000","last_became_active":"2024-09-19T08:43:52.763146+0000","last_became_peered":"2024-09-19T08:43:52.763146+0000","last_unstale":"2024-09-19T08:44:02.820558+0000","last_undegraded":"2024-09-19T08:44:02.820558+0000","last_fullsized":"2024-09-19T08:44:02.820558+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:32.194465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.638566+0000","last_change":"2024-09-19T08:43:50.749184+0000","last_active":"2024-09-19T08:44:04.638566+0000","last_peered":"2024-09-19T08:44:04.638566+0000","last_clean":"2024-09-19T08:44:04.638566+0000","last_became_active":"2024-09-19T08:43:50.748844+0000","last_became_peered":"2024-09-19T08:43:50.748844+0000","last_unstale":"2024-09-19T08:44:04.638566+0000","last_undegraded":"2024-09-19T08:44:04.638566+0000","last_fullsized":"2024-09-19T08:44:04.638566+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:15.334759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819256+0000","last_change":"2024-09-19T08:43:52.768696+0000","last_active":"2024-09-19T08:44:02.819256+0000","last_peered":"2024-09-19T08:44:02.819256+0000","last_clean":"2024-09-19T08:44:02.819256+0000","last_became_active":"2024-09-19T08:43:52.768374+0000","last_became_peered":"2024-09-19T08:43:52.768374+0000","last_unstale":"2024-09-19T08:44:02.819256+0000","last_undegraded":"2024-09-19T08:44:02.819256+0000","last_fullsized":"2024-09-19T08:44:02.819256+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:34:39.343041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.838113+0000","last_change":"2024-09-19T08:43:51.026270+0000","last_active":"2024-09-19T08:44:03.838113+0000","last_peered":"2024-09-19T08:44:03.838113+0000","last_clean":"2024-09-19T08:44:03.838113+0000","last_became_active":"2024-09-19T08:43:51.025823+0000","last_became_peered":"2024-09-19T08:43:51.025823+0000","last_unstale":"2024-09-19T08:44:03.838113+0000","last_undegraded":"2024-09-19T08:44:03.838113+0000","last_fullsized":"2024-09-19T08:44:03.838113+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:00.480438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.820391+0000","last_change":"2024-09-19T08:43:52.759555+0000","last_active":"2024-09-19T08:44:02.820391+0000","last_peered":"2024-09-19T08:44:02.820391+0000","last_clean":"2024-09-19T08:44:02.820391+0000","last_became_active":"2024-09-19T08:43:52.759223+0000","last_became_peered":"2024-09-19T08:43:52.759223+0000","last_unstale":"2024-09-19T08:44:02.820391+0000","last_undegraded":"2024-09-19T08:44:02.820391+0000","last_fullsized":"2024-09-19T08:44:02.820391+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:57:06.451323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'28","reported_seq":162,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:55.705107+0000","last_change":"2024-09-19T08:43:50.751924+0000","last_active":"2024-09-19T08:44:55.705107+0000","last_peered":"2024-09-19T08:44:55.705107+0000","last_clean":"2024-09-19T08:44:55.705107+0000","last_became_active":"2024-09-19T08:43:50.751117+0000","last_became_peered":"2024-09-19T08:43:50.751117+0000","last_unstale":"2024-09-19T08:44:55.705107+0000","last_undegraded":"2024-09-19T08:44:55.705107+0000","last_fullsized":"2024-09-19T08:44:55.705107+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:33:57.196283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59211,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":112,"num_read_kb":113,"num_write":30,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818871+0000","last_change":"2024-09-19T08:43:52.762260+0000","last_active":"2024-09-19T08:44:02.818871+0000","last_peered":"2024-09-19T08:44:02.818871+0000","last_clean":"2024-09-19T08:44:02.818871+0000","last_became_active":"2024-09-19T08:43:52.761923+0000","last_became_peered":"2024-09-19T08:43:52.761923+0000","last_unstale":"2024-09-19T08:44:02.818871+0000","last_undegraded":"2024-09-19T08:44:02.818871+0000","last_fullsized":"2024-09-19T08:44:02.818871+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:39:39.503332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.638455+0000","last_change":"2024-09-19T08:43:50.749745+0000","last_active":"2024-09-19T08:44:04.638455+0000","last_peered":"2024-09-19T08:44:04.638455+0000","last_clean":"2024-09-19T08:44:04.638455+0000","last_became_active":"2024-09-19T08:43:50.749073+0000","last_became_peered":"2024-09-19T08:43:50.749073+0000","last_unstale":"2024-09-19T08:44:04.638455+0000","last_undegraded":"2024-09-19T08:44:04.638455+0000","last_fullsized":"2024-09-19T08:44:04.638455+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:48:11.965841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818633+0000","last_change":"2024-09-19T08:43:52.761495+0000","last_active":"2024-09-19T08:44:02.818633+0000","last_peered":"2024-09-19T08:44:02.818633+0000","last_clean":"2024-09-19T08:44:02.818633+0000","last_became_active":"2024-09-19T08:43:52.760425+0000","last_became_peered":"2024-09-19T08:43:52.760425+0000","last_unstale":"2024-09-19T08:44:02.818633+0000","last_undegraded":"2024-09-19T08:44:02.818633+0000","last_fullsized":"2024-09-19T08:44:02.818633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:57.689284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.639600+0000","last_change":"2024-09-19T08:43:50.749563+0000","last_active":"2024-09-19T08:44:04.639600+0000","last_peered":"2024-09-19T08:44:04.639600+0000","last_clean":"2024-09-19T08:44:04.639600+0000","last_became_active":"2024-09-19T08:43:50.749244+0000","last_became_peered":"2024-09-19T08:43:50.749244+0000","last_unstale":"2024-09-19T08:44:04.639600+0000","last_undegraded":"2024-09-19T08:44:04.639600+0000","last_fullsized":"2024-09-19T08:44:04.639600+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:59:02.312883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821915+0000","last_change":"2024-09-19T08:43:52.767087+0000","last_active":"2024-09-19T08:44:02.821915+0000","last_peered":"2024-09-19T08:44:02.821915+0000","last_clean":"2024-09-19T08:44:02.821915+0000","last_became_active":"2024-09-19T08:43:52.766730+0000","last_became_peered":"2024-09-19T08:43:52.766730+0000","last_unstale":"2024-09-19T08:44:02.821915+0000","last_undegraded":"2024-09-19T08:44:02.821915+0000","last_fullsized":"2024-09-19T08:44:02.821915+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:04.040896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":80,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:53.709332+0000","last_change":"2024-09-19T08:43:50.749659+0000","last_active":"2024-09-19T08:44:53.709332+0000","last_peered":"2024-09-19T08:44:53.709332+0000","last_clean":"2024-09-19T08:44:53.709332+0000","last_became_active":"2024-09-19T08:43:50.749314+0000","last_became_peered":"2024-09-19T08:43:50.749314+0000","last_unstale":"2024-09-19T08:44:53.709332+0000","last_undegraded":"2024-09-19T08:44:53.709332+0000","last_fullsized":"2024-09-19T08:44:53.709332+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:08.576209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821732+0000","last_change":"2024-09-19T08:43:52.767928+0000","last_active":"2024-09-19T08:44:02.821732+0000","last_peered":"2024-09-19T08:44:02.821732+0000","last_clean":"2024-09-19T08:44:02.821732+0000","last_became_active":"2024-09-19T08:43:52.767580+0000","last_became_peered":"2024-09-19T08:43:52.767580+0000","last_unstale":"2024-09-19T08:44:02.821732+0000","last_undegraded":"2024-09-19T08:44:02.821732+0000","last_fullsized":"2024-09-19T08:44:02.821732+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:45.344693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.630057+0000","last_change":"2024-09-19T08:44:02.833887+0000","last_active":"2024-09-19T08:44:04.630057+0000","last_peered":"2024-09-19T08:44:04.630057+0000","last_clean":"2024-09-19T08:44:04.630057+0000","last_became_active":"2024-09-19T08:44:02.821624+0000","last_became_peered":"2024-09-19T08:44:02.821624+0000","last_unstale":"2024-09-19T08:44:04.630057+0000","last_undegraded":"2024-09-19T08:44:04.630057+0000","last_fullsized":"2024-09-19T08:44:04.630057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:54:36.131949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.080821+0000","last_change":"2024-09-19T08:44:03.080503+0000","last_active":"2024-09-19T08:44:03.080821+0000","last_peered":"2024-09-19T08:44:03.080821+0000","last_clean":"2024-09-19T08:44:03.080821+0000","last_became_active":"2024-09-19T08:44:03.080129+0000","last_became_peered":"2024-09-19T08:44:03.080129+0000","last_unstale":"2024-09-19T08:44:03.080821+0000","last_undegraded":"2024-09-19T08:44:03.080821+0000","last_fullsized":"2024-09-19T08:44:03.080821+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:01:37.034795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.629282+0000","last_change":"2024-09-19T08:43:50.747624+0000","last_active":"2024-09-19T08:44:04.629282+0000","last_peered":"2024-09-19T08:44:04.629282+0000","last_clean":"2024-09-19T08:44:04.629282+0000","last_became_active":"2024-09-19T08:43:50.747263+0000","last_became_peered":"2024-09-19T08:43:50.747263+0000","last_unstale":"2024-09-19T08:44:04.629282+0000","last_undegraded":"2024-09-19T08:44:04.629282+0000","last_fullsized":"2024-09-19T08:44:04.629282+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:18.580106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.816903+0000","last_change":"2024-09-19T08:43:52.770051+0000","last_active":"2024-09-19T08:44:02.816903+0000","last_peered":"2024-09-19T08:44:02.816903+0000","last_clean":"2024-09-19T08:44:02.816903+0000","last_became_active":"2024-09-19T08:43:52.769727+0000","last_became_peered":"2024-09-19T08:43:52.769727+0000","last_unstale":"2024-09-19T08:44:02.816903+0000","last_undegraded":"2024-09-19T08:44:02.816903+0000","last_fullsized":"2024-09-19T08:44:02.816903+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:34.954897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.638777+0000","last_change":"2024-09-19T08:43:50.746696+0000","last_active":"2024-09-19T08:44:04.638777+0000","last_peered":"2024-09-19T08:44:04.638777+0000","last_clean":"2024-09-19T08:44:04.638777+0000","last_became_active":"2024-09-19T08:43:50.746246+0000","last_became_peered":"2024-09-19T08:43:50.746246+0000","last_unstale":"2024-09-19T08:44:04.638777+0000","last_undegraded":"2024-09-19T08:44:04.638777+0000","last_fullsized":"2024-09-19T08:44:04.638777+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:38:39.935348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.822242+0000","last_change":"2024-09-19T08:43:52.765199+0000","last_active":"2024-09-19T08:44:02.822242+0000","last_peered":"2024-09-19T08:44:02.822242+0000","last_clean":"2024-09-19T08:44:02.822242+0000","last_became_active":"2024-09-19T08:43:52.764896+0000","last_became_peered":"2024-09-19T08:43:52.764896+0000","last_unstale":"2024-09-19T08:44:02.822242+0000","last_undegraded":"2024-09-19T08:44:02.822242+0000","last_fullsized":"2024-09-19T08:44:02.822242+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:02:59.570018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.628862+0000","last_change":"2024-09-19T08:43:50.747242+0000","last_active":"2024-09-19T08:44:04.628862+0000","last_peered":"2024-09-19T08:44:04.628862+0000","last_clean":"2024-09-19T08:44:04.628862+0000","last_became_active":"2024-09-19T08:43:50.746853+0000","last_became_peered":"2024-09-19T08:43:50.746853+0000","last_unstale":"2024-09-19T08:44:04.628862+0000","last_undegraded":"2024-09-19T08:44:04.628862+0000","last_fullsized":"2024-09-19T08:44:04.628862+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:21.545635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.827031+0000","last_change":"2024-09-19T08:44:02.826535+0000","last_active":"2024-09-19T08:44:02.827031+0000","last_peered":"2024-09-19T08:44:02.827031+0000","last_clean":"2024-09-19T08:44:02.827031+0000","last_became_active":"2024-09-19T08:44:02.826113+0000","last_became_peered":"2024-09-19T08:44:02.826113+0000","last_unstale":"2024-09-19T08:44:02.827031+0000","last_undegraded":"2024-09-19T08:44:02.827031+0000","last_fullsized":"2024-09-19T08:44:02.827031+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:20.247732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'6","reported_seq":81,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:52.701668+0000","last_change":"2024-09-19T08:43:51.131540+0000","last_active":"2024-09-19T08:44:52.701668+0000","last_peered":"2024-09-19T08:44:52.701668+0000","last_clean":"2024-09-19T08:44:52.701668+0000","last_became_active":"2024-09-19T08:43:51.131170+0000","last_became_peered":"2024-09-19T08:43:51.131170+0000","last_unstale":"2024-09-19T08:44:52.701668+0000","last_undegraded":"2024-09-19T08:44:52.701668+0000","last_fullsized":"2024-09-19T08:44:52.701668+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:43.635612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.822208+0000","last_change":"2024-09-19T08:43:52.763308+0000","last_active":"2024-09-19T08:44:02.822208+0000","last_peered":"2024-09-19T08:44:02.822208+0000","last_clean":"2024-09-19T08:44:02.822208+0000","last_became_active":"2024-09-19T08:43:52.762859+0000","last_became_peered":"2024-09-19T08:43:52.762859+0000","last_unstale":"2024-09-19T08:44:02.822208+0000","last_undegraded":"2024-09-19T08:44:02.822208+0000","last_fullsized":"2024-09-19T08:44:02.822208+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:32:35.083361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":347,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821905+0000","last_change":"2024-09-19T08:42:39.567408+0000","last_active":"2024-09-19T08:44:02.821905+0000","last_peered":"2024-09-19T08:44:02.821905+0000","last_clean":"2024-09-19T08:44:02.821905+0000","last_became_active":"2024-09-19T08:42:39.566967+0000","last_became_peered":"2024-09-19T08:42:39.566967+0000","last_unstale":"2024-09-19T08:44:02.821905+0000","last_undegraded":"2024-09-19T08:44:02.821905+0000","last_fullsized":"2024-09-19T08:44:02.821905+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:14.287944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:14.287944+0000","last_clean_scrub_stamp":"2024-09-19T08:39:14.287944+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:31:03.245386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.638441+0000","last_change":"2024-09-19T08:43:50.751097+0000","last_active":"2024-09-19T08:44:04.638441+0000","last_peered":"2024-09-19T08:44:04.638441+0000","last_clean":"2024-09-19T08:44:04.638441+0000","last_became_active":"2024-09-19T08:43:50.750341+0000","last_became_peered":"2024-09-19T08:43:50.750341+0000","last_unstale":"2024-09-19T08:44:04.638441+0000","last_undegraded":"2024-09-19T08:44:04.638441+0000","last_fullsized":"2024-09-19T08:44:04.638441+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:25.694535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.493670+0000","last_change":"2024-09-19T08:43:52.767905+0000","last_active":"2024-09-19T08:44:03.493670+0000","last_peered":"2024-09-19T08:44:03.493670+0000","last_clean":"2024-09-19T08:44:03.493670+0000","last_became_active":"2024-09-19T08:43:52.767601+0000","last_became_peered":"2024-09-19T08:43:52.767601+0000","last_unstale":"2024-09-19T08:44:03.493670+0000","last_undegraded":"2024-09-19T08:44:03.493670+0000","last_fullsized":"2024-09-19T08:44:03.493670+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:14.519879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.822161+0000","last_change":"2024-09-19T08:43:50.751397+0000","last_active":"2024-09-19T08:44:02.822161+0000","last_peered":"2024-09-19T08:44:02.822161+0000","last_clean":"2024-09-19T08:44:02.822161+0000","last_became_active":"2024-09-19T08:43:50.751079+0000","last_became_peered":"2024-09-19T08:43:50.751079+0000","last_unstale":"2024-09-19T08:44:02.822161+0000","last_undegraded":"2024-09-19T08:44:02.822161+0000","last_fullsized":"2024-09-19T08:44:02.822161+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:29.904786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818371+0000","last_change":"2024-09-19T08:43:52.773348+0000","last_active":"2024-09-19T08:44:02.818371+0000","last_peered":"2024-09-19T08:44:02.818371+0000","last_clean":"2024-09-19T08:44:02.818371+0000","last_became_active":"2024-09-19T08:43:52.773011+0000","last_became_peered":"2024-09-19T08:43:52.773011+0000","last_unstale":"2024-09-19T08:44:02.818371+0000","last_undegraded":"2024-09-19T08:44:02.818371+0000","last_fullsized":"2024-09-19T08:44:02.818371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:10.806578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":80,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:54.710341+0000","last_change":"2024-09-19T08:43:51.025300+0000","last_active":"2024-09-19T08:44:54.710341+0000","last_peered":"2024-09-19T08:44:54.710341+0000","last_clean":"2024-09-19T08:44:54.710341+0000","last_became_active":"2024-09-19T08:43:51.024936+0000","last_became_peered":"2024-09-19T08:43:51.024936+0000","last_unstale":"2024-09-19T08:44:54.710341+0000","last_undegraded":"2024-09-19T08:44:54.710341+0000","last_fullsized":"2024-09-19T08:44:54.710341+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:03.186066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821758+0000","last_change":"2024-09-19T08:43:52.774962+0000","last_active":"2024-09-19T08:44:02.821758+0000","last_peered":"2024-09-19T08:44:02.821758+0000","last_clean":"2024-09-19T08:44:02.821758+0000","last_became_active":"2024-09-19T08:43:52.774598+0000","last_became_peered":"2024-09-19T08:43:52.774598+0000","last_unstale":"2024-09-19T08:44:02.821758+0000","last_undegraded":"2024-09-19T08:44:02.821758+0000","last_fullsized":"2024-09-19T08:44:02.821758+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:02.687136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'5","reported_seq":78,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:53.717897+0000","last_change":"2024-09-19T08:43:50.748180+0000","last_active":"2024-09-19T08:44:53.717897+0000","last_peered":"2024-09-19T08:44:53.717897+0000","last_clean":"2024-09-19T08:44:53.717897+0000","last_became_active":"2024-09-19T08:43:50.747853+0000","last_became_peered":"2024-09-19T08:43:50.747853+0000","last_unstale":"2024-09-19T08:44:53.717897+0000","last_undegraded":"2024-09-19T08:44:53.717897+0000","last_fullsized":"2024-09-19T08:44:53.717897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:56.047259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.817252+0000","last_change":"2024-09-19T08:43:52.770253+0000","last_active":"2024-09-19T08:44:02.817252+0000","last_peered":"2024-09-19T08:44:02.817252+0000","last_clean":"2024-09-19T08:44:02.817252+0000","last_became_active":"2024-09-19T08:43:52.769951+0000","last_became_peered":"2024-09-19T08:43:52.769951+0000","last_unstale":"2024-09-19T08:44:02.817252+0000","last_undegraded":"2024-09-19T08:44:02.817252+0000","last_fullsized":"2024-09-19T08:44:02.817252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:02.353144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821619+0000","last_change":"2024-09-19T08:43:51.026453+0000","last_active":"2024-09-19T08:44:02.821619+0000","last_peered":"2024-09-19T08:44:02.821619+0000","last_clean":"2024-09-19T08:44:02.821619+0000","last_became_active":"2024-09-19T08:43:51.025873+0000","last_became_peered":"2024-09-19T08:43:51.025873+0000","last_unstale":"2024-09-19T08:44:02.821619+0000","last_undegraded":"2024-09-19T08:44:02.821619+0000","last_fullsized":"2024-09-19T08:44:02.821619+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:25:17.475074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.817186+0000","last_change":"2024-09-19T08:43:52.761700+0000","last_active":"2024-09-19T08:44:02.817186+0000","last_peered":"2024-09-19T08:44:02.817186+0000","last_clean":"2024-09-19T08:44:02.817186+0000","last_became_active":"2024-09-19T08:43:52.755527+0000","last_became_peered":"2024-09-19T08:43:52.755527+0000","last_unstale":"2024-09-19T08:44:02.817186+0000","last_undegraded":"2024-09-19T08:44:02.817186+0000","last_fullsized":"2024-09-19T08:44:02.817186+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:18:43.171284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.630369+0000","last_change":"2024-09-19T08:43:51.026136+0000","last_active":"2024-09-19T08:44:04.630369+0000","last_peered":"2024-09-19T08:44:04.630369+0000","last_clean":"2024-09-19T08:44:04.630369+0000","last_became_active":"2024-09-19T08:43:51.025740+0000","last_became_peered":"2024-09-19T08:43:51.025740+0000","last_unstale":"2024-09-19T08:44:04.630369+0000","last_undegraded":"2024-09-19T08:44:04.630369+0000","last_fullsized":"2024-09-19T08:44:04.630369+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:17:32.741083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819140+0000","last_change":"2024-09-19T08:43:52.763808+0000","last_active":"2024-09-19T08:44:02.819140+0000","last_peered":"2024-09-19T08:44:02.819140+0000","last_clean":"2024-09-19T08:44:02.819140+0000","last_became_active":"2024-09-19T08:43:52.763509+0000","last_became_peered":"2024-09-19T08:43:52.763509+0000","last_unstale":"2024-09-19T08:44:02.819140+0000","last_undegraded":"2024-09-19T08:44:02.819140+0000","last_fullsized":"2024-09-19T08:44:02.819140+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:45:08.385419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'15","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:17.772532+0000","last_change":"2024-09-19T08:43:50.744213+0000","last_active":"2024-09-19T08:44:17.772532+0000","last_peered":"2024-09-19T08:44:17.772532+0000","last_clean":"2024-09-19T08:44:17.772532+0000","last_became_active":"2024-09-19T08:43:50.743848+0000","last_became_peered":"2024-09-19T08:43:50.743848+0000","last_unstale":"2024-09-19T08:44:17.772532+0000","last_undegraded":"2024-09-19T08:44:17.772532+0000","last_fullsized":"2024-09-19T08:44:17.772532+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:26.613055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.494010+0000","last_change":"2024-09-19T08:43:52.767102+0000","last_active":"2024-09-19T08:44:03.494010+0000","last_peered":"2024-09-19T08:44:03.494010+0000","last_clean":"2024-09-19T08:44:03.494010+0000","last_became_active":"2024-09-19T08:43:52.766262+0000","last_became_peered":"2024-09-19T08:43:52.766262+0000","last_unstale":"2024-09-19T08:44:03.494010+0000","last_undegraded":"2024-09-19T08:44:03.494010+0000","last_fullsized":"2024-09-19T08:44:03.494010+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:35.740462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.638198+0000","last_change":"2024-09-19T08:43:51.131742+0000","last_active":"2024-09-19T08:44:04.638198+0000","last_peered":"2024-09-19T08:44:04.638198+0000","last_clean":"2024-09-19T08:44:04.638198+0000","last_became_active":"2024-09-19T08:43:51.131388+0000","last_became_peered":"2024-09-19T08:43:51.131388+0000","last_unstale":"2024-09-19T08:44:04.638198+0000","last_undegraded":"2024-09-19T08:44:04.638198+0000","last_fullsized":"2024-09-19T08:44:04.638198+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:26.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819357+0000","last_change":"2024-09-19T08:43:52.756004+0000","last_active":"2024-09-19T08:44:02.819357+0000","last_peered":"2024-09-19T08:44:02.819357+0000","last_clean":"2024-09-19T08:44:02.819357+0000","last_became_active":"2024-09-19T08:43:52.755596+0000","last_became_peered":"2024-09-19T08:43:52.755596+0000","last_unstale":"2024-09-19T08:44:02.819357+0000","last_undegraded":"2024-09-19T08:44:02.819357+0000","last_fullsized":"2024-09-19T08:44:02.819357+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:35.942375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.835986+0000","last_change":"2024-09-19T08:44:02.835986+0000","last_active":"2024-09-19T08:44:02.835986+0000","last_peered":"2024-09-19T08:44:02.835986+0000","last_clean":"2024-09-19T08:44:02.835986+0000","last_became_active":"2024-09-19T08:44:02.826314+0000","last_became_peered":"2024-09-19T08:44:02.826314+0000","last_unstale":"2024-09-19T08:44:02.835986+0000","last_undegraded":"2024-09-19T08:44:02.835986+0000","last_fullsized":"2024-09-19T08:44:02.835986+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:16.943989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818539+0000","last_change":"2024-09-19T08:43:52.766582+0000","last_active":"2024-09-19T08:44:02.818539+0000","last_peered":"2024-09-19T08:44:02.818539+0000","last_clean":"2024-09-19T08:44:02.818539+0000","last_became_active":"2024-09-19T08:43:52.766232+0000","last_became_peered":"2024-09-19T08:43:52.766232+0000","last_unstale":"2024-09-19T08:44:02.818539+0000","last_undegraded":"2024-09-19T08:44:02.818539+0000","last_fullsized":"2024-09-19T08:44:02.818539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:43:42.319422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.635810+0000","last_change":"2024-09-19T08:43:50.747611+0000","last_active":"2024-09-19T08:44:04.635810+0000","last_peered":"2024-09-19T08:44:04.635810+0000","last_clean":"2024-09-19T08:44:04.635810+0000","last_became_active":"2024-09-19T08:43:50.747263+0000","last_became_peered":"2024-09-19T08:43:50.747263+0000","last_unstale":"2024-09-19T08:44:04.635810+0000","last_undegraded":"2024-09-19T08:44:04.635810+0000","last_fullsized":"2024-09-19T08:44:04.635810+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:14.717820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.816851+0000","last_change":"2024-09-19T08:43:52.766493+0000","last_active":"2024-09-19T08:44:02.816851+0000","last_peered":"2024-09-19T08:44:02.816851+0000","last_clean":"2024-09-19T08:44:02.816851+0000","last_became_active":"2024-09-19T08:43:52.766158+0000","last_became_peered":"2024-09-19T08:43:52.766158+0000","last_unstale":"2024-09-19T08:44:02.816851+0000","last_undegraded":"2024-09-19T08:44:02.816851+0000","last_fullsized":"2024-09-19T08:44:02.816851+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:59.175949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.838733+0000","last_change":"2024-09-19T08:43:50.751368+0000","last_active":"2024-09-19T08:44:03.838733+0000","last_peered":"2024-09-19T08:44:03.838733+0000","last_clean":"2024-09-19T08:44:03.838733+0000","last_became_active":"2024-09-19T08:43:50.751038+0000","last_became_peered":"2024-09-19T08:43:50.751038+0000","last_unstale":"2024-09-19T08:44:03.838733+0000","last_undegraded":"2024-09-19T08:44:03.838733+0000","last_fullsized":"2024-09-19T08:44:03.838733+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:46.302996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.493641+0000","last_change":"2024-09-19T08:43:52.756998+0000","last_active":"2024-09-19T08:44:03.493641+0000","last_peered":"2024-09-19T08:44:03.493641+0000","last_clean":"2024-09-19T08:44:03.493641+0000","last_became_active":"2024-09-19T08:43:52.756661+0000","last_became_peered":"2024-09-19T08:43:52.756661+0000","last_unstale":"2024-09-19T08:44:03.493641+0000","last_undegraded":"2024-09-19T08:44:03.493641+0000","last_fullsized":"2024-09-19T08:44:03.493641+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:15.182574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818827+0000","last_change":"2024-09-19T08:43:50.750095+0000","last_active":"2024-09-19T08:44:02.818827+0000","last_peered":"2024-09-19T08:44:02.818827+0000","last_clean":"2024-09-19T08:44:02.818827+0000","last_became_active":"2024-09-19T08:43:50.749672+0000","last_became_peered":"2024-09-19T08:43:50.749672+0000","last_unstale":"2024-09-19T08:44:02.818827+0000","last_undegraded":"2024-09-19T08:44:02.818827+0000","last_fullsized":"2024-09-19T08:44:02.818827+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:25:34.978094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821907+0000","last_change":"2024-09-19T08:43:52.764097+0000","last_active":"2024-09-19T08:44:02.821907+0000","last_peered":"2024-09-19T08:44:02.821907+0000","last_clean":"2024-09-19T08:44:02.821907+0000","last_became_active":"2024-09-19T08:43:52.763773+0000","last_became_peered":"2024-09-19T08:43:52.763773+0000","last_unstale":"2024-09-19T08:44:02.821907+0000","last_undegraded":"2024-09-19T08:44:02.821907+0000","last_fullsized":"2024-09-19T08:44:02.821907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:01:42.098460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.825238+0000","last_change":"2024-09-19T08:44:02.825238+0000","last_active":"2024-09-19T08:44:02.825238+0000","last_peered":"2024-09-19T08:44:02.825238+0000","last_clean":"2024-09-19T08:44:02.825238+0000","last_became_active":"2024-09-19T08:44:02.824873+0000","last_became_peered":"2024-09-19T08:44:02.824873+0000","last_unstale":"2024-09-19T08:44:02.825238+0000","last_undegraded":"2024-09-19T08:44:02.825238+0000","last_fullsized":"2024-09-19T08:44:02.825238+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:38.896082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.639612+0000","last_change":"2024-09-19T08:43:50.748789+0000","last_active":"2024-09-19T08:44:04.639612+0000","last_peered":"2024-09-19T08:44:04.639612+0000","last_clean":"2024-09-19T08:44:04.639612+0000","last_became_active":"2024-09-19T08:43:50.748464+0000","last_became_peered":"2024-09-19T08:43:50.748464+0000","last_unstale":"2024-09-19T08:44:04.639612+0000","last_undegraded":"2024-09-19T08:44:04.639612+0000","last_fullsized":"2024-09-19T08:44:04.639612+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:36:45.905639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.820329+0000","last_change":"2024-09-19T08:43:52.756608+0000","last_active":"2024-09-19T08:44:02.820329+0000","last_peered":"2024-09-19T08:44:02.820329+0000","last_clean":"2024-09-19T08:44:02.820329+0000","last_became_active":"2024-09-19T08:43:52.756227+0000","last_became_peered":"2024-09-19T08:43:52.756227+0000","last_unstale":"2024-09-19T08:44:02.820329+0000","last_undegraded":"2024-09-19T08:44:02.820329+0000","last_fullsized":"2024-09-19T08:44:02.820329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:29:58.085584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'12","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.639102+0000","last_change":"2024-09-19T08:43:50.749135+0000","last_active":"2024-09-19T08:44:04.639102+0000","last_peered":"2024-09-19T08:44:04.639102+0000","last_clean":"2024-09-19T08:44:04.639102+0000","last_became_active":"2024-09-19T08:43:50.748755+0000","last_became_peered":"2024-09-19T08:43:50.748755+0000","last_unstale":"2024-09-19T08:44:04.639102+0000","last_undegraded":"2024-09-19T08:44:04.639102+0000","last_fullsized":"2024-09-19T08:44:04.639102+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:55.079780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819170+0000","last_change":"2024-09-19T08:43:52.763277+0000","last_active":"2024-09-19T08:44:02.819170+0000","last_peered":"2024-09-19T08:44:02.819170+0000","last_clean":"2024-09-19T08:44:02.819170+0000","last_became_active":"2024-09-19T08:43:52.762941+0000","last_became_peered":"2024-09-19T08:43:52.762941+0000","last_unstale":"2024-09-19T08:44:02.819170+0000","last_undegraded":"2024-09-19T08:44:02.819170+0000","last_fullsized":"2024-09-19T08:44:02.819170+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:35.531946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.494005+0000","last_change":"2024-09-19T08:43:51.025350+0000","last_active":"2024-09-19T08:44:03.494005+0000","last_peered":"2024-09-19T08:44:03.494005+0000","last_clean":"2024-09-19T08:44:03.494005+0000","last_became_active":"2024-09-19T08:43:51.025046+0000","last_became_peered":"2024-09-19T08:43:51.025046+0000","last_unstale":"2024-09-19T08:44:03.494005+0000","last_undegraded":"2024-09-19T08:44:03.494005+0000","last_fullsized":"2024-09-19T08:44:03.494005+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:27.595545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.493915+0000","last_change":"2024-09-19T08:43:52.767917+0000","last_active":"2024-09-19T08:44:03.493915+0000","last_peered":"2024-09-19T08:44:03.493915+0000","last_clean":"2024-09-19T08:44:03.493915+0000","last_became_active":"2024-09-19T08:43:52.767575+0000","last_became_peered":"2024-09-19T08:43:52.767575+0000","last_unstale":"2024-09-19T08:44:03.493915+0000","last_undegraded":"2024-09-19T08:44:03.493915+0000","last_fullsized":"2024-09-19T08:44:03.493915+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:05.950399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'13","reported_seq":87,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:54.718778+0000","last_change":"2024-09-19T08:43:50.746400+0000","last_active":"2024-09-19T08:44:54.718778+0000","last_peered":"2024-09-19T08:44:54.718778+0000","last_clean":"2024-09-19T08:44:54.718778+0000","last_became_active":"2024-09-19T08:43:50.745891+0000","last_became_peered":"2024-09-19T08:43:50.745891+0000","last_unstale":"2024-09-19T08:44:54.718778+0000","last_undegraded":"2024-09-19T08:44:54.718778+0000","last_fullsized":"2024-09-19T08:44:54.718778+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:03.098202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66419,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":553,"num_read_kb":352,"num_write":241,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":199257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":250201,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579931,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":436653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808864,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":1086388,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005092,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":436195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234025,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":436344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462958,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":970752,"data_stored":1028976,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593780,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":435795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822712,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":983040,"data_stored":1028654,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986237,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":905216,"data_stored":435656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215170,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":435503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":987136,"data_stored":496925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":966656,"data_stored":495919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":925696,"data_stored":437527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34807,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34807,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34807,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:44:59.517 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T08:44:59.777 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:45:00.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:44:59 smithi164 ceph-mon[30928]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:45:00.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:44:59 smithi170 ceph-mon[35072]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:45:00.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:44:59 smithi173 ceph-mon[35306]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:45:00.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.274+0000 7fb1e88ed640 1 -- 172.21.15.164:0/1314072431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1e4101bc0 msgr2=0x7fb1e410d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:00.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.274+0000 7fb1e88ed640 1 --2- 172.21.15.164:0/1314072431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1e4101bc0 0x7fb1e410d910 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc009ca0 tx=0x7fb1dc02a8c0 comp rx=0 tx=0).stop 2024-09-19T08:45:00.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.275+0000 7fb1e88ed640 1 -- 172.21.15.164:0/1314072431 shutdown_connections 2024-09-19T08:45:00.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.275+0000 7fb1e88ed640 1 --2- 172.21.15.164:0/1314072431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1e4101bc0 0x7fb1e410d910 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:00.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.275+0000 7fb1e88ed640 1 --2- 172.21.15.164:0/1314072431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1e4101240 0x7fb1e4101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:00.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.275+0000 7fb1e88ed640 1 --2- 172.21.15.164:0/1314072431 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1e4105a90 0x7fb1e4105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:00.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.275+0000 7fb1e88ed640 1 -- 172.21.15.164:0/1314072431 >> 172.21.15.164:0/1314072431 conn(0x7fb1e4077210 msgr2=0x7fb1e40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:00.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.275+0000 7fb1e88ed640 1 -- 172.21.15.164:0/1314072431 shutdown_connections 2024-09-19T08:45:00.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.275+0000 7fb1e88ed640 1 -- 172.21.15.164:0/1314072431 wait complete. 2024-09-19T08:45:00.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.275+0000 7fb1e88ed640 1 Processor -- start 2024-09-19T08:45:00.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.275+0000 7fb1e88ed640 1 -- start start 2024-09-19T08:45:00.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.276+0000 7fb1e88ed640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1e4101240 0x7fb1e4197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:00.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.276+0000 7fb1e88ed640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1e4101bc0 0x7fb1e4197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:00.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.276+0000 7fb1e88ed640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1e4105a90 0x7fb1e419be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:00.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.276+0000 7fb1e88ed640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e4111030 con 0x7fb1e4101bc0 2024-09-19T08:45:00.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.276+0000 7fb1e88ed640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e4110eb0 con 0x7fb1e4101240 2024-09-19T08:45:00.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.276+0000 7fb1e88ed640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e41111b0 con 0x7fb1e4105a90 2024-09-19T08:45:00.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.276+0000 7fb1e1d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1e4101bc0 0x7fb1e4197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:00.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.276+0000 7fb1e1d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1e4101bc0 0x7fb1e4197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48820/0 (socket says 172.21.15.164:48820) 2024-09-19T08:45:00.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.276+0000 7fb1e1d9b640 1 -- 172.21.15.164:0/219376043 learned_addr learned my addr 172.21.15.164:0/219376043 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:00.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.276+0000 7fb1e2d9d640 1 --2- 172.21.15.164:0/219376043 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1e4105a90 0x7fb1e419be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:00.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.276+0000 7fb1e1d9b640 1 -- 172.21.15.164:0/219376043 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1e4105a90 msgr2=0x7fb1e419be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:00.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.276+0000 7fb1e259c640 1 --2- 172.21.15.164:0/219376043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1e4101240 0x7fb1e4197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:00.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.276+0000 7fb1e1d9b640 1 --2- 172.21.15.164:0/219376043 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1e4105a90 0x7fb1e419be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:00.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.276+0000 7fb1e1d9b640 1 -- 172.21.15.164:0/219376043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1e4101240 msgr2=0x7fb1e4197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:00.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.276+0000 7fb1e1d9b640 1 --2- 172.21.15.164:0/219376043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1e4101240 0x7fb1e4197570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:00.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.276+0000 7fb1e1d9b640 1 -- 172.21.15.164:0/219376043 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e419c520 con 0x7fb1e4101bc0 2024-09-19T08:45:00.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.276+0000 7fb1e259c640 1 --2- 172.21.15.164:0/219376043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1e4101240 0x7fb1e4197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:00.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.276+0000 7fb1e1d9b640 1 --2- 172.21.15.164:0/219376043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1e4101bc0 0x7fb1e4197ab0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400b4a0 tx=0x7fb1d400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:00.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.276+0000 7fb1db7fe640 1 -- 172.21.15.164:0/219376043 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4014070 con 0x7fb1e4101bc0 2024-09-19T08:45:00.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.277+0000 7fb1db7fe640 1 -- 172.21.15.164:0/219376043 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1d4010af0 con 0x7fb1e4101bc0 2024-09-19T08:45:00.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.277+0000 7fb1e88ed640 1 -- 172.21.15.164:0/219376043 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1e419c810 con 0x7fb1e4101bc0 2024-09-19T08:45:00.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.277+0000 7fb1db7fe640 1 -- 172.21.15.164:0/219376043 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4017d10 con 0x7fb1e4101bc0 2024-09-19T08:45:00.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.277+0000 7fb1e88ed640 1 -- 172.21.15.164:0/219376043 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1e41a40f0 con 0x7fb1e4101bc0 2024-09-19T08:45:00.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.278+0000 7fb1e88ed640 1 -- 172.21.15.164:0/219376043 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1e4101680 con 0x7fb1e4101bc0 2024-09-19T08:45:00.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.279+0000 7fb1db7fe640 1 -- 172.21.15.164:0/219376043 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fb1d4017070 con 0x7fb1e4101bc0 2024-09-19T08:45:00.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.279+0000 7fb1db7fe640 1 --2- 172.21.15.164:0/219376043 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb1d00a32d0 0x7fb1d00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:00.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.280+0000 7fb1e259c640 1 --2- 172.21.15.164:0/219376043 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb1d00a32d0 0x7fb1d00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:00.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.280+0000 7fb1db7fe640 1 -- 172.21.15.164:0/219376043 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb1d40ced10 con 0x7fb1e4101bc0 2024-09-19T08:45:00.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.280+0000 7fb1e259c640 1 --2- 172.21.15.164:0/219376043 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb1d00a32d0 0x7fb1d00a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc006fd0 tx=0x7fb1cc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:00.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.284+0000 7fb1db7fe640 1 -- 172.21.15.164:0/219376043 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb1d409bae0 con 0x7fb1e4101bc0 2024-09-19T08:45:00.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.448+0000 7fb1e88ed640 1 -- 172.21.15.164:0/219376043 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb1e4108fe0 con 0x7fb1d00a32d0 2024-09-19T08:45:00.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.456+0000 7fb1db7fe640 1 -- 172.21.15.164:0/219376043 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258900 (secure 0 0 0) 0x7fb1e4108fe0 con 0x7fb1d00a32d0 2024-09-19T08:45:00.459 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:45:00.460 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-09-19T08:45:00.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.460+0000 7fb1e88ed640 1 -- 172.21.15.164:0/219376043 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb1d00a32d0 msgr2=0x7fb1d00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:00.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.460+0000 7fb1e88ed640 1 --2- 172.21.15.164:0/219376043 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb1d00a32d0 0x7fb1d00a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc006fd0 tx=0x7fb1cc008040 comp rx=0 tx=0).stop 2024-09-19T08:45:00.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.460+0000 7fb1e88ed640 1 -- 172.21.15.164:0/219376043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1e4101bc0 msgr2=0x7fb1e4197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:00.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.460+0000 7fb1e88ed640 1 --2- 172.21.15.164:0/219376043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1e4101bc0 0x7fb1e4197ab0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400b4a0 tx=0x7fb1d400b960 comp rx=0 tx=0).stop 2024-09-19T08:45:00.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.460+0000 7fb1e88ed640 1 -- 172.21.15.164:0/219376043 shutdown_connections 2024-09-19T08:45:00.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.460+0000 7fb1e88ed640 1 --2- 172.21.15.164:0/219376043 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb1d00a32d0 0x7fb1d00a5790 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:00.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.460+0000 7fb1e88ed640 1 --2- 172.21.15.164:0/219376043 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1e4105a90 0x7fb1e419be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:00.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.460+0000 7fb1e88ed640 1 --2- 172.21.15.164:0/219376043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1e4101bc0 0x7fb1e4197ab0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:00.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.460+0000 7fb1e88ed640 1 --2- 172.21.15.164:0/219376043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1e4101240 0x7fb1e4197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:00.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.460+0000 7fb1e88ed640 1 -- 172.21.15.164:0/219376043 >> 172.21.15.164:0/219376043 conn(0x7fb1e4077210 msgr2=0x7fb1e4103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:00.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.460+0000 7fb1e88ed640 1 -- 172.21.15.164:0/219376043 shutdown_connections 2024-09-19T08:45:00.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:00.460+0000 7fb1e88ed640 1 -- 172.21.15.164:0/219376043 wait complete. 2024-09-19T08:45:01.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:00 smithi170 ceph-mon[35072]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:45:01.150 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:00 smithi164 ceph-mon[30928]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:45:01.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:00 smithi173 ceph-mon[35306]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:45:01.180 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-09-19T08:44:59.759144+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657021,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":694,"num_read_kb":471,"num_write":424,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":346,"ondisk_log_size":346,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":389468,"kb_used_data":11088,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124552356,"statfs":{"total":1151940427776,"available":1151541612544,"internally_reserved":0,"allocated":11354112,"data_stored":7190535,"data_compressed":109335,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002880"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.817273+0000","last_change":"2024-09-19T08:43:52.766709+0000","last_active":"2024-09-19T08:44:02.817273+0000","last_peered":"2024-09-19T08:44:02.817273+0000","last_clean":"2024-09-19T08:44:02.817273+0000","last_became_active":"2024-09-19T08:43:52.765970+0000","last_became_peered":"2024-09-19T08:43:52.765970+0000","last_unstale":"2024-09-19T08:44:02.817273+0000","last_undegraded":"2024-09-19T08:44:02.817273+0000","last_fullsized":"2024-09-19T08:44:02.817273+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:12.176870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819309+0000","last_change":"2024-09-19T08:43:52.766590+0000","last_active":"2024-09-19T08:44:02.819309+0000","last_peered":"2024-09-19T08:44:02.819309+0000","last_clean":"2024-09-19T08:44:02.819309+0000","last_became_active":"2024-09-19T08:43:52.766255+0000","last_became_peered":"2024-09-19T08:43:52.766255+0000","last_unstale":"2024-09-19T08:44:02.819309+0000","last_undegraded":"2024-09-19T08:44:02.819309+0000","last_fullsized":"2024-09-19T08:44:02.819309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:15:41.359806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818510+0000","last_change":"2024-09-19T08:43:52.761956+0000","last_active":"2024-09-19T08:44:02.818510+0000","last_peered":"2024-09-19T08:44:02.818510+0000","last_clean":"2024-09-19T08:44:02.818510+0000","last_became_active":"2024-09-19T08:43:52.761644+0000","last_became_peered":"2024-09-19T08:43:52.761644+0000","last_unstale":"2024-09-19T08:44:02.818510+0000","last_undegraded":"2024-09-19T08:44:02.818510+0000","last_fullsized":"2024-09-19T08:44:02.818510+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:13.770361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.079353+0000","last_change":"2024-09-19T08:43:52.764131+0000","last_active":"2024-09-19T08:44:03.079353+0000","last_peered":"2024-09-19T08:44:03.079353+0000","last_clean":"2024-09-19T08:44:03.079353+0000","last_became_active":"2024-09-19T08:43:52.763812+0000","last_became_peered":"2024-09-19T08:43:52.763812+0000","last_unstale":"2024-09-19T08:44:03.079353+0000","last_undegraded":"2024-09-19T08:44:03.079353+0000","last_fullsized":"2024-09-19T08:44:03.079353+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:32.272157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818711+0000","last_change":"2024-09-19T08:43:52.754673+0000","last_active":"2024-09-19T08:44:02.818711+0000","last_peered":"2024-09-19T08:44:02.818711+0000","last_clean":"2024-09-19T08:44:02.818711+0000","last_became_active":"2024-09-19T08:43:52.754144+0000","last_became_peered":"2024-09-19T08:43:52.754144+0000","last_unstale":"2024-09-19T08:44:02.818711+0000","last_undegraded":"2024-09-19T08:44:02.818711+0000","last_fullsized":"2024-09-19T08:44:02.818711+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:16.069260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.817142+0000","last_change":"2024-09-19T08:43:52.758939+0000","last_active":"2024-09-19T08:44:02.817142+0000","last_peered":"2024-09-19T08:44:02.817142+0000","last_clean":"2024-09-19T08:44:02.817142+0000","last_became_active":"2024-09-19T08:43:52.758611+0000","last_became_peered":"2024-09-19T08:43:52.758611+0000","last_unstale":"2024-09-19T08:44:02.817142+0000","last_undegraded":"2024-09-19T08:44:02.817142+0000","last_fullsized":"2024-09-19T08:44:02.817142+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:35:56.463545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821786+0000","last_change":"2024-09-19T08:43:52.770261+0000","last_active":"2024-09-19T08:44:02.821786+0000","last_peered":"2024-09-19T08:44:02.821786+0000","last_clean":"2024-09-19T08:44:02.821786+0000","last_became_active":"2024-09-19T08:43:52.768253+0000","last_became_peered":"2024-09-19T08:43:52.768253+0000","last_unstale":"2024-09-19T08:44:02.821786+0000","last_undegraded":"2024-09-19T08:44:02.821786+0000","last_fullsized":"2024-09-19T08:44:02.821786+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:40.756988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.493456+0000","last_change":"2024-09-19T08:43:52.768564+0000","last_active":"2024-09-19T08:44:03.493456+0000","last_peered":"2024-09-19T08:44:03.493456+0000","last_clean":"2024-09-19T08:44:03.493456+0000","last_became_active":"2024-09-19T08:43:52.768260+0000","last_became_peered":"2024-09-19T08:43:52.768260+0000","last_unstale":"2024-09-19T08:44:03.493456+0000","last_undegraded":"2024-09-19T08:44:03.493456+0000","last_fullsized":"2024-09-19T08:44:03.493456+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:16.528115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.820687+0000","last_change":"2024-09-19T08:43:52.769804+0000","last_active":"2024-09-19T08:44:02.820687+0000","last_peered":"2024-09-19T08:44:02.820687+0000","last_clean":"2024-09-19T08:44:02.820687+0000","last_became_active":"2024-09-19T08:43:52.769504+0000","last_became_peered":"2024-09-19T08:43:52.769504+0000","last_unstale":"2024-09-19T08:44:02.820687+0000","last_undegraded":"2024-09-19T08:44:02.820687+0000","last_fullsized":"2024-09-19T08:44:02.820687+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:22:38.514410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.817329+0000","last_change":"2024-09-19T08:43:52.763278+0000","last_active":"2024-09-19T08:44:02.817329+0000","last_peered":"2024-09-19T08:44:02.817329+0000","last_clean":"2024-09-19T08:44:02.817329+0000","last_became_active":"2024-09-19T08:43:52.762985+0000","last_became_peered":"2024-09-19T08:43:52.762985+0000","last_unstale":"2024-09-19T08:44:02.817329+0000","last_undegraded":"2024-09-19T08:44:02.817329+0000","last_fullsized":"2024-09-19T08:44:02.817329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:30.048782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.822068+0000","last_change":"2024-09-19T08:43:52.770864+0000","last_active":"2024-09-19T08:44:02.822068+0000","last_peered":"2024-09-19T08:44:02.822068+0000","last_clean":"2024-09-19T08:44:02.822068+0000","last_became_active":"2024-09-19T08:43:52.770518+0000","last_became_peered":"2024-09-19T08:43:52.770518+0000","last_unstale":"2024-09-19T08:44:02.822068+0000","last_undegraded":"2024-09-19T08:44:02.822068+0000","last_fullsized":"2024-09-19T08:44:02.822068+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:08.931452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.079430+0000","last_change":"2024-09-19T08:43:52.757464+0000","last_active":"2024-09-19T08:44:03.079430+0000","last_peered":"2024-09-19T08:44:03.079430+0000","last_clean":"2024-09-19T08:44:03.079430+0000","last_became_active":"2024-09-19T08:43:52.757101+0000","last_became_peered":"2024-09-19T08:43:52.757101+0000","last_unstale":"2024-09-19T08:44:03.079430+0000","last_undegraded":"2024-09-19T08:44:03.079430+0000","last_fullsized":"2024-09-19T08:44:03.079430+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:17.395735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818784+0000","last_change":"2024-09-19T08:43:52.767063+0000","last_active":"2024-09-19T08:44:02.818784+0000","last_peered":"2024-09-19T08:44:02.818784+0000","last_clean":"2024-09-19T08:44:02.818784+0000","last_became_active":"2024-09-19T08:43:52.766755+0000","last_became_peered":"2024-09-19T08:43:52.766755+0000","last_unstale":"2024-09-19T08:44:02.818784+0000","last_undegraded":"2024-09-19T08:44:02.818784+0000","last_fullsized":"2024-09-19T08:44:02.818784+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:30.530937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818799+0000","last_change":"2024-09-19T08:43:52.766515+0000","last_active":"2024-09-19T08:44:02.818799+0000","last_peered":"2024-09-19T08:44:02.818799+0000","last_clean":"2024-09-19T08:44:02.818799+0000","last_became_active":"2024-09-19T08:43:52.765847+0000","last_became_peered":"2024-09-19T08:43:52.765847+0000","last_unstale":"2024-09-19T08:44:02.818799+0000","last_undegraded":"2024-09-19T08:44:02.818799+0000","last_fullsized":"2024-09-19T08:44:02.818799+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:50:35.948663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818391+0000","last_change":"2024-09-19T08:43:52.773702+0000","last_active":"2024-09-19T08:44:02.818391+0000","last_peered":"2024-09-19T08:44:02.818391+0000","last_clean":"2024-09-19T08:44:02.818391+0000","last_became_active":"2024-09-19T08:43:52.773324+0000","last_became_peered":"2024-09-19T08:43:52.773324+0000","last_unstale":"2024-09-19T08:44:02.818391+0000","last_undegraded":"2024-09-19T08:44:02.818391+0000","last_fullsized":"2024-09-19T08:44:02.818391+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:34.368014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.820811+0000","last_change":"2024-09-19T08:43:52.762745+0000","last_active":"2024-09-19T08:44:02.820811+0000","last_peered":"2024-09-19T08:44:02.820811+0000","last_clean":"2024-09-19T08:44:02.820811+0000","last_became_active":"2024-09-19T08:43:52.762172+0000","last_became_peered":"2024-09-19T08:43:52.762172+0000","last_unstale":"2024-09-19T08:44:02.820811+0000","last_undegraded":"2024-09-19T08:44:02.820811+0000","last_fullsized":"2024-09-19T08:44:02.820811+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:57.369775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.817462+0000","last_change":"2024-09-19T08:43:52.767118+0000","last_active":"2024-09-19T08:44:02.817462+0000","last_peered":"2024-09-19T08:44:02.817462+0000","last_clean":"2024-09-19T08:44:02.817462+0000","last_became_active":"2024-09-19T08:43:52.766388+0000","last_became_peered":"2024-09-19T08:43:52.766388+0000","last_unstale":"2024-09-19T08:44:02.817462+0000","last_undegraded":"2024-09-19T08:44:02.817462+0000","last_fullsized":"2024-09-19T08:44:02.817462+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:34.182923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821529+0000","last_change":"2024-09-19T08:43:52.765008+0000","last_active":"2024-09-19T08:44:02.821529+0000","last_peered":"2024-09-19T08:44:02.821529+0000","last_clean":"2024-09-19T08:44:02.821529+0000","last_became_active":"2024-09-19T08:43:52.764558+0000","last_became_peered":"2024-09-19T08:43:52.764558+0000","last_unstale":"2024-09-19T08:44:02.821529+0000","last_undegraded":"2024-09-19T08:44:02.821529+0000","last_fullsized":"2024-09-19T08:44:02.821529+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:52:42.756421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819875+0000","last_change":"2024-09-19T08:43:52.764376+0000","last_active":"2024-09-19T08:44:02.819875+0000","last_peered":"2024-09-19T08:44:02.819875+0000","last_clean":"2024-09-19T08:44:02.819875+0000","last_became_active":"2024-09-19T08:43:52.764048+0000","last_became_peered":"2024-09-19T08:43:52.764048+0000","last_unstale":"2024-09-19T08:44:02.819875+0000","last_undegraded":"2024-09-19T08:44:02.819875+0000","last_fullsized":"2024-09-19T08:44:02.819875+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:56.732094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.827925+0000","last_change":"2024-09-19T08:44:02.827925+0000","last_active":"2024-09-19T08:44:02.827925+0000","last_peered":"2024-09-19T08:44:02.827925+0000","last_clean":"2024-09-19T08:44:02.827925+0000","last_became_active":"2024-09-19T08:44:02.827269+0000","last_became_peered":"2024-09-19T08:44:02.827269+0000","last_unstale":"2024-09-19T08:44:02.827925+0000","last_undegraded":"2024-09-19T08:44:02.827925+0000","last_fullsized":"2024-09-19T08:44:02.827925+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:31:54.556786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818875+0000","last_change":"2024-09-19T08:43:52.762190+0000","last_active":"2024-09-19T08:44:02.818875+0000","last_peered":"2024-09-19T08:44:02.818875+0000","last_clean":"2024-09-19T08:44:02.818875+0000","last_became_active":"2024-09-19T08:43:52.761818+0000","last_became_peered":"2024-09-19T08:43:52.761818+0000","last_unstale":"2024-09-19T08:44:02.818875+0000","last_undegraded":"2024-09-19T08:44:02.818875+0000","last_fullsized":"2024-09-19T08:44:02.818875+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:19:55.246779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.820265+0000","last_change":"2024-09-19T08:43:52.761069+0000","last_active":"2024-09-19T08:44:02.820265+0000","last_peered":"2024-09-19T08:44:02.820265+0000","last_clean":"2024-09-19T08:44:02.820265+0000","last_became_active":"2024-09-19T08:43:52.760705+0000","last_became_peered":"2024-09-19T08:43:52.760705+0000","last_unstale":"2024-09-19T08:44:02.820265+0000","last_undegraded":"2024-09-19T08:44:02.820265+0000","last_fullsized":"2024-09-19T08:44:02.820265+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:46:27.059026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.493512+0000","last_change":"2024-09-19T08:43:52.773449+0000","last_active":"2024-09-19T08:44:03.493512+0000","last_peered":"2024-09-19T08:44:03.493512+0000","last_clean":"2024-09-19T08:44:03.493512+0000","last_became_active":"2024-09-19T08:43:52.772929+0000","last_became_peered":"2024-09-19T08:43:52.772929+0000","last_unstale":"2024-09-19T08:44:03.493512+0000","last_undegraded":"2024-09-19T08:44:03.493512+0000","last_fullsized":"2024-09-19T08:44:03.493512+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:00:38.929199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821652+0000","last_change":"2024-09-19T08:43:52.773885+0000","last_active":"2024-09-19T08:44:02.821652+0000","last_peered":"2024-09-19T08:44:02.821652+0000","last_clean":"2024-09-19T08:44:02.821652+0000","last_became_active":"2024-09-19T08:43:52.772856+0000","last_became_peered":"2024-09-19T08:43:52.772856+0000","last_unstale":"2024-09-19T08:44:02.821652+0000","last_undegraded":"2024-09-19T08:44:02.821652+0000","last_fullsized":"2024-09-19T08:44:02.821652+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:13:22.581335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.493984+0000","last_change":"2024-09-19T08:43:52.762909+0000","last_active":"2024-09-19T08:44:03.493984+0000","last_peered":"2024-09-19T08:44:03.493984+0000","last_clean":"2024-09-19T08:44:03.493984+0000","last_became_active":"2024-09-19T08:43:52.762600+0000","last_became_peered":"2024-09-19T08:43:52.762600+0000","last_unstale":"2024-09-19T08:44:03.493984+0000","last_undegraded":"2024-09-19T08:44:03.493984+0000","last_fullsized":"2024-09-19T08:44:03.493984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:46.098294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.820300+0000","last_change":"2024-09-19T08:43:52.762861+0000","last_active":"2024-09-19T08:44:02.820300+0000","last_peered":"2024-09-19T08:44:02.820300+0000","last_clean":"2024-09-19T08:44:02.820300+0000","last_became_active":"2024-09-19T08:43:52.762525+0000","last_became_peered":"2024-09-19T08:43:52.762525+0000","last_unstale":"2024-09-19T08:44:02.820300+0000","last_undegraded":"2024-09-19T08:44:02.820300+0000","last_fullsized":"2024-09-19T08:44:02.820300+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:54.717249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.816836+0000","last_change":"2024-09-19T08:43:52.768370+0000","last_active":"2024-09-19T08:44:02.816836+0000","last_peered":"2024-09-19T08:44:02.816836+0000","last_clean":"2024-09-19T08:44:02.816836+0000","last_became_active":"2024-09-19T08:43:52.768055+0000","last_became_peered":"2024-09-19T08:43:52.768055+0000","last_unstale":"2024-09-19T08:44:02.816836+0000","last_undegraded":"2024-09-19T08:44:02.816836+0000","last_fullsized":"2024-09-19T08:44:02.816836+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:48.212862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819012+0000","last_change":"2024-09-19T08:43:52.758541+0000","last_active":"2024-09-19T08:44:02.819012+0000","last_peered":"2024-09-19T08:44:02.819012+0000","last_clean":"2024-09-19T08:44:02.819012+0000","last_became_active":"2024-09-19T08:43:52.758122+0000","last_became_peered":"2024-09-19T08:43:52.758122+0000","last_unstale":"2024-09-19T08:44:02.819012+0000","last_undegraded":"2024-09-19T08:44:02.819012+0000","last_fullsized":"2024-09-19T08:44:02.819012+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:54:40.681880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818709+0000","last_change":"2024-09-19T08:43:52.758109+0000","last_active":"2024-09-19T08:44:02.818709+0000","last_peered":"2024-09-19T08:44:02.818709+0000","last_clean":"2024-09-19T08:44:02.818709+0000","last_became_active":"2024-09-19T08:43:52.757808+0000","last_became_peered":"2024-09-19T08:43:52.757808+0000","last_unstale":"2024-09-19T08:44:02.818709+0000","last_undegraded":"2024-09-19T08:44:02.818709+0000","last_fullsized":"2024-09-19T08:44:02.818709+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:02:47.614562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819377+0000","last_change":"2024-09-19T08:43:52.772501+0000","last_active":"2024-09-19T08:44:02.819377+0000","last_peered":"2024-09-19T08:44:02.819377+0000","last_clean":"2024-09-19T08:44:02.819377+0000","last_became_active":"2024-09-19T08:43:52.771611+0000","last_became_peered":"2024-09-19T08:43:52.771611+0000","last_unstale":"2024-09-19T08:44:02.819377+0000","last_undegraded":"2024-09-19T08:44:02.819377+0000","last_fullsized":"2024-09-19T08:44:02.819377+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:49.537237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819339+0000","last_change":"2024-09-19T08:43:52.773664+0000","last_active":"2024-09-19T08:44:02.819339+0000","last_peered":"2024-09-19T08:44:02.819339+0000","last_clean":"2024-09-19T08:44:02.819339+0000","last_became_active":"2024-09-19T08:43:52.773014+0000","last_became_peered":"2024-09-19T08:43:52.773014+0000","last_unstale":"2024-09-19T08:44:02.819339+0000","last_undegraded":"2024-09-19T08:44:02.819339+0000","last_fullsized":"2024-09-19T08:44:02.819339+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:03.448825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.816727+0000","last_change":"2024-09-19T08:43:52.769578+0000","last_active":"2024-09-19T08:44:02.816727+0000","last_peered":"2024-09-19T08:44:02.816727+0000","last_clean":"2024-09-19T08:44:02.816727+0000","last_became_active":"2024-09-19T08:43:52.769252+0000","last_became_peered":"2024-09-19T08:43:52.769252+0000","last_unstale":"2024-09-19T08:44:02.816727+0000","last_undegraded":"2024-09-19T08:44:02.816727+0000","last_fullsized":"2024-09-19T08:44:02.816727+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:27.936118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818612+0000","last_change":"2024-09-19T08:43:52.770453+0000","last_active":"2024-09-19T08:44:02.818612+0000","last_peered":"2024-09-19T08:44:02.818612+0000","last_clean":"2024-09-19T08:44:02.818612+0000","last_became_active":"2024-09-19T08:43:52.770155+0000","last_became_peered":"2024-09-19T08:43:52.770155+0000","last_unstale":"2024-09-19T08:44:02.818612+0000","last_undegraded":"2024-09-19T08:44:02.818612+0000","last_fullsized":"2024-09-19T08:44:02.818612+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:12.893572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":82,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:53.703154+0000","last_change":"2024-09-19T08:43:50.751316+0000","last_active":"2024-09-19T08:44:53.703154+0000","last_peered":"2024-09-19T08:44:53.703154+0000","last_clean":"2024-09-19T08:44:53.703154+0000","last_became_active":"2024-09-19T08:43:50.750972+0000","last_became_peered":"2024-09-19T08:43:50.750972+0000","last_unstale":"2024-09-19T08:44:53.703154+0000","last_undegraded":"2024-09-19T08:44:53.703154+0000","last_fullsized":"2024-09-19T08:44:53.703154+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:40.654729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.822390+0000","last_change":"2024-09-19T08:43:52.773979+0000","last_active":"2024-09-19T08:44:02.822390+0000","last_peered":"2024-09-19T08:44:02.822390+0000","last_clean":"2024-09-19T08:44:02.822390+0000","last_became_active":"2024-09-19T08:43:52.773648+0000","last_became_peered":"2024-09-19T08:43:52.773648+0000","last_unstale":"2024-09-19T08:44:02.822390+0000","last_undegraded":"2024-09-19T08:44:02.822390+0000","last_fullsized":"2024-09-19T08:44:02.822390+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:36.061209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.839445+0000","last_change":"2024-09-19T08:43:50.750704+0000","last_active":"2024-09-19T08:44:03.839445+0000","last_peered":"2024-09-19T08:44:03.839445+0000","last_clean":"2024-09-19T08:44:03.839445+0000","last_became_active":"2024-09-19T08:43:50.750399+0000","last_became_peered":"2024-09-19T08:43:50.750399+0000","last_unstale":"2024-09-19T08:44:03.839445+0000","last_undegraded":"2024-09-19T08:44:03.839445+0000","last_fullsized":"2024-09-19T08:44:03.839445+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:25.447320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818514+0000","last_change":"2024-09-19T08:43:50.744919+0000","last_active":"2024-09-19T08:44:02.818514+0000","last_peered":"2024-09-19T08:44:02.818514+0000","last_clean":"2024-09-19T08:44:02.818514+0000","last_became_active":"2024-09-19T08:43:50.744604+0000","last_became_peered":"2024-09-19T08:43:50.744604+0000","last_unstale":"2024-09-19T08:44:02.818514+0000","last_undegraded":"2024-09-19T08:44:02.818514+0000","last_fullsized":"2024-09-19T08:44:02.818514+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:25:06.410093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.493518+0000","last_change":"2024-09-19T08:43:52.766779+0000","last_active":"2024-09-19T08:44:03.493518+0000","last_peered":"2024-09-19T08:44:03.493518+0000","last_clean":"2024-09-19T08:44:03.493518+0000","last_became_active":"2024-09-19T08:43:52.766484+0000","last_became_peered":"2024-09-19T08:43:52.766484+0000","last_unstale":"2024-09-19T08:44:03.493518+0000","last_undegraded":"2024-09-19T08:44:03.493518+0000","last_fullsized":"2024-09-19T08:44:03.493518+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:17:29.350778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.631853+0000","last_change":"2024-09-19T08:43:50.751282+0000","last_active":"2024-09-19T08:44:04.631853+0000","last_peered":"2024-09-19T08:44:04.631853+0000","last_clean":"2024-09-19T08:44:04.631853+0000","last_became_active":"2024-09-19T08:43:50.750754+0000","last_became_peered":"2024-09-19T08:43:50.750754+0000","last_unstale":"2024-09-19T08:44:04.631853+0000","last_undegraded":"2024-09-19T08:44:04.631853+0000","last_fullsized":"2024-09-19T08:44:04.631853+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:11.661626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821354+0000","last_change":"2024-09-19T08:43:52.756442+0000","last_active":"2024-09-19T08:44:02.821354+0000","last_peered":"2024-09-19T08:44:02.821354+0000","last_clean":"2024-09-19T08:44:02.821354+0000","last_became_active":"2024-09-19T08:43:52.756116+0000","last_became_peered":"2024-09-19T08:43:52.756116+0000","last_unstale":"2024-09-19T08:44:02.821354+0000","last_undegraded":"2024-09-19T08:44:02.821354+0000","last_fullsized":"2024-09-19T08:44:02.821354+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:40.029272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":130,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:55.705106+0000","last_change":"2024-09-19T08:43:50.749637+0000","last_active":"2024-09-19T08:44:55.705106+0000","last_peered":"2024-09-19T08:44:55.705106+0000","last_clean":"2024-09-19T08:44:55.705106+0000","last_became_active":"2024-09-19T08:43:50.749307+0000","last_became_peered":"2024-09-19T08:43:50.749307+0000","last_unstale":"2024-09-19T08:44:55.705106+0000","last_undegraded":"2024-09-19T08:44:55.705106+0000","last_fullsized":"2024-09-19T08:44:55.705106+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:23:33.409671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.820558+0000","last_change":"2024-09-19T08:43:52.763592+0000","last_active":"2024-09-19T08:44:02.820558+0000","last_peered":"2024-09-19T08:44:02.820558+0000","last_clean":"2024-09-19T08:44:02.820558+0000","last_became_active":"2024-09-19T08:43:52.763146+0000","last_became_peered":"2024-09-19T08:43:52.763146+0000","last_unstale":"2024-09-19T08:44:02.820558+0000","last_undegraded":"2024-09-19T08:44:02.820558+0000","last_fullsized":"2024-09-19T08:44:02.820558+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:32.194465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.638566+0000","last_change":"2024-09-19T08:43:50.749184+0000","last_active":"2024-09-19T08:44:04.638566+0000","last_peered":"2024-09-19T08:44:04.638566+0000","last_clean":"2024-09-19T08:44:04.638566+0000","last_became_active":"2024-09-19T08:43:50.748844+0000","last_became_peered":"2024-09-19T08:43:50.748844+0000","last_unstale":"2024-09-19T08:44:04.638566+0000","last_undegraded":"2024-09-19T08:44:04.638566+0000","last_fullsized":"2024-09-19T08:44:04.638566+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:15.334759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819256+0000","last_change":"2024-09-19T08:43:52.768696+0000","last_active":"2024-09-19T08:44:02.819256+0000","last_peered":"2024-09-19T08:44:02.819256+0000","last_clean":"2024-09-19T08:44:02.819256+0000","last_became_active":"2024-09-19T08:43:52.768374+0000","last_became_peered":"2024-09-19T08:43:52.768374+0000","last_unstale":"2024-09-19T08:44:02.819256+0000","last_undegraded":"2024-09-19T08:44:02.819256+0000","last_fullsized":"2024-09-19T08:44:02.819256+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:34:39.343041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.838113+0000","last_change":"2024-09-19T08:43:51.026270+0000","last_active":"2024-09-19T08:44:03.838113+0000","last_peered":"2024-09-19T08:44:03.838113+0000","last_clean":"2024-09-19T08:44:03.838113+0000","last_became_active":"2024-09-19T08:43:51.025823+0000","last_became_peered":"2024-09-19T08:43:51.025823+0000","last_unstale":"2024-09-19T08:44:03.838113+0000","last_undegraded":"2024-09-19T08:44:03.838113+0000","last_fullsized":"2024-09-19T08:44:03.838113+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:00.480438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.820391+0000","last_change":"2024-09-19T08:43:52.759555+0000","last_active":"2024-09-19T08:44:02.820391+0000","last_peered":"2024-09-19T08:44:02.820391+0000","last_clean":"2024-09-19T08:44:02.820391+0000","last_became_active":"2024-09-19T08:43:52.759223+0000","last_became_peered":"2024-09-19T08:43:52.759223+0000","last_unstale":"2024-09-19T08:44:02.820391+0000","last_undegraded":"2024-09-19T08:44:02.820391+0000","last_fullsized":"2024-09-19T08:44:02.820391+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:57:06.451323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'28","reported_seq":162,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:55.705107+0000","last_change":"2024-09-19T08:43:50.751924+0000","last_active":"2024-09-19T08:44:55.705107+0000","last_peered":"2024-09-19T08:44:55.705107+0000","last_clean":"2024-09-19T08:44:55.705107+0000","last_became_active":"2024-09-19T08:43:50.751117+0000","last_became_peered":"2024-09-19T08:43:50.751117+0000","last_unstale":"2024-09-19T08:44:55.705107+0000","last_undegraded":"2024-09-19T08:44:55.705107+0000","last_fullsized":"2024-09-19T08:44:55.705107+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:33:57.196283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59211,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":112,"num_read_kb":113,"num_write":30,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818871+0000","last_change":"2024-09-19T08:43:52.762260+0000","last_active":"2024-09-19T08:44:02.818871+0000","last_peered":"2024-09-19T08:44:02.818871+0000","last_clean":"2024-09-19T08:44:02.818871+0000","last_became_active":"2024-09-19T08:43:52.761923+0000","last_became_peered":"2024-09-19T08:43:52.761923+0000","last_unstale":"2024-09-19T08:44:02.818871+0000","last_undegraded":"2024-09-19T08:44:02.818871+0000","last_fullsized":"2024-09-19T08:44:02.818871+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:39:39.503332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.638455+0000","last_change":"2024-09-19T08:43:50.749745+0000","last_active":"2024-09-19T08:44:04.638455+0000","last_peered":"2024-09-19T08:44:04.638455+0000","last_clean":"2024-09-19T08:44:04.638455+0000","last_became_active":"2024-09-19T08:43:50.749073+0000","last_became_peered":"2024-09-19T08:43:50.749073+0000","last_unstale":"2024-09-19T08:44:04.638455+0000","last_undegraded":"2024-09-19T08:44:04.638455+0000","last_fullsized":"2024-09-19T08:44:04.638455+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:48:11.965841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818633+0000","last_change":"2024-09-19T08:43:52.761495+0000","last_active":"2024-09-19T08:44:02.818633+0000","last_peered":"2024-09-19T08:44:02.818633+0000","last_clean":"2024-09-19T08:44:02.818633+0000","last_became_active":"2024-09-19T08:43:52.760425+0000","last_became_peered":"2024-09-19T08:43:52.760425+0000","last_unstale":"2024-09-19T08:44:02.818633+0000","last_undegraded":"2024-09-19T08:44:02.818633+0000","last_fullsized":"2024-09-19T08:44:02.818633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:57.689284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.639600+0000","last_change":"2024-09-19T08:43:50.749563+0000","last_active":"2024-09-19T08:44:04.639600+0000","last_peered":"2024-09-19T08:44:04.639600+0000","last_clean":"2024-09-19T08:44:04.639600+0000","last_became_active":"2024-09-19T08:43:50.749244+0000","last_became_peered":"2024-09-19T08:43:50.749244+0000","last_unstale":"2024-09-19T08:44:04.639600+0000","last_undegraded":"2024-09-19T08:44:04.639600+0000","last_fullsized":"2024-09-19T08:44:04.639600+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:59:02.312883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821915+0000","last_change":"2024-09-19T08:43:52.767087+0000","last_active":"2024-09-19T08:44:02.821915+0000","last_peered":"2024-09-19T08:44:02.821915+0000","last_clean":"2024-09-19T08:44:02.821915+0000","last_became_active":"2024-09-19T08:43:52.766730+0000","last_became_peered":"2024-09-19T08:43:52.766730+0000","last_unstale":"2024-09-19T08:44:02.821915+0000","last_undegraded":"2024-09-19T08:44:02.821915+0000","last_fullsized":"2024-09-19T08:44:02.821915+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:04.040896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":80,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:53.709332+0000","last_change":"2024-09-19T08:43:50.749659+0000","last_active":"2024-09-19T08:44:53.709332+0000","last_peered":"2024-09-19T08:44:53.709332+0000","last_clean":"2024-09-19T08:44:53.709332+0000","last_became_active":"2024-09-19T08:43:50.749314+0000","last_became_peered":"2024-09-19T08:43:50.749314+0000","last_unstale":"2024-09-19T08:44:53.709332+0000","last_undegraded":"2024-09-19T08:44:53.709332+0000","last_fullsized":"2024-09-19T08:44:53.709332+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:08.576209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821732+0000","last_change":"2024-09-19T08:43:52.767928+0000","last_active":"2024-09-19T08:44:02.821732+0000","last_peered":"2024-09-19T08:44:02.821732+0000","last_clean":"2024-09-19T08:44:02.821732+0000","last_became_active":"2024-09-19T08:43:52.767580+0000","last_became_peered":"2024-09-19T08:43:52.767580+0000","last_unstale":"2024-09-19T08:44:02.821732+0000","last_undegraded":"2024-09-19T08:44:02.821732+0000","last_fullsized":"2024-09-19T08:44:02.821732+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:45.344693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.630057+0000","last_change":"2024-09-19T08:44:02.833887+0000","last_active":"2024-09-19T08:44:04.630057+0000","last_peered":"2024-09-19T08:44:04.630057+0000","last_clean":"2024-09-19T08:44:04.630057+0000","last_became_active":"2024-09-19T08:44:02.821624+0000","last_became_peered":"2024-09-19T08:44:02.821624+0000","last_unstale":"2024-09-19T08:44:04.630057+0000","last_undegraded":"2024-09-19T08:44:04.630057+0000","last_fullsized":"2024-09-19T08:44:04.630057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:54:36.131949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.080821+0000","last_change":"2024-09-19T08:44:03.080503+0000","last_active":"2024-09-19T08:44:03.080821+0000","last_peered":"2024-09-19T08:44:03.080821+0000","last_clean":"2024-09-19T08:44:03.080821+0000","last_became_active":"2024-09-19T08:44:03.080129+0000","last_became_peered":"2024-09-19T08:44:03.080129+0000","last_unstale":"2024-09-19T08:44:03.080821+0000","last_undegraded":"2024-09-19T08:44:03.080821+0000","last_fullsized":"2024-09-19T08:44:03.080821+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:01:37.034795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.629282+0000","last_change":"2024-09-19T08:43:50.747624+0000","last_active":"2024-09-19T08:44:04.629282+0000","last_peered":"2024-09-19T08:44:04.629282+0000","last_clean":"2024-09-19T08:44:04.629282+0000","last_became_active":"2024-09-19T08:43:50.747263+0000","last_became_peered":"2024-09-19T08:43:50.747263+0000","last_unstale":"2024-09-19T08:44:04.629282+0000","last_undegraded":"2024-09-19T08:44:04.629282+0000","last_fullsized":"2024-09-19T08:44:04.629282+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:18.580106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.816903+0000","last_change":"2024-09-19T08:43:52.770051+0000","last_active":"2024-09-19T08:44:02.816903+0000","last_peered":"2024-09-19T08:44:02.816903+0000","last_clean":"2024-09-19T08:44:02.816903+0000","last_became_active":"2024-09-19T08:43:52.769727+0000","last_became_peered":"2024-09-19T08:43:52.769727+0000","last_unstale":"2024-09-19T08:44:02.816903+0000","last_undegraded":"2024-09-19T08:44:02.816903+0000","last_fullsized":"2024-09-19T08:44:02.816903+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:34.954897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.638777+0000","last_change":"2024-09-19T08:43:50.746696+0000","last_active":"2024-09-19T08:44:04.638777+0000","last_peered":"2024-09-19T08:44:04.638777+0000","last_clean":"2024-09-19T08:44:04.638777+0000","last_became_active":"2024-09-19T08:43:50.746246+0000","last_became_peered":"2024-09-19T08:43:50.746246+0000","last_unstale":"2024-09-19T08:44:04.638777+0000","last_undegraded":"2024-09-19T08:44:04.638777+0000","last_fullsized":"2024-09-19T08:44:04.638777+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:38:39.935348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.822242+0000","last_change":"2024-09-19T08:43:52.765199+0000","last_active":"2024-09-19T08:44:02.822242+0000","last_peered":"2024-09-19T08:44:02.822242+0000","last_clean":"2024-09-19T08:44:02.822242+0000","last_became_active":"2024-09-19T08:43:52.764896+0000","last_became_peered":"2024-09-19T08:43:52.764896+0000","last_unstale":"2024-09-19T08:44:02.822242+0000","last_undegraded":"2024-09-19T08:44:02.822242+0000","last_fullsized":"2024-09-19T08:44:02.822242+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:02:59.570018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.628862+0000","last_change":"2024-09-19T08:43:50.747242+0000","last_active":"2024-09-19T08:44:04.628862+0000","last_peered":"2024-09-19T08:44:04.628862+0000","last_clean":"2024-09-19T08:44:04.628862+0000","last_became_active":"2024-09-19T08:43:50.746853+0000","last_became_peered":"2024-09-19T08:43:50.746853+0000","last_unstale":"2024-09-19T08:44:04.628862+0000","last_undegraded":"2024-09-19T08:44:04.628862+0000","last_fullsized":"2024-09-19T08:44:04.628862+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:21.545635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.827031+0000","last_change":"2024-09-19T08:44:02.826535+0000","last_active":"2024-09-19T08:44:02.827031+0000","last_peered":"2024-09-19T08:44:02.827031+0000","last_clean":"2024-09-19T08:44:02.827031+0000","last_became_active":"2024-09-19T08:44:02.826113+0000","last_became_peered":"2024-09-19T08:44:02.826113+0000","last_unstale":"2024-09-19T08:44:02.827031+0000","last_undegraded":"2024-09-19T08:44:02.827031+0000","last_fullsized":"2024-09-19T08:44:02.827031+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:20.247732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'6","reported_seq":86,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.708607+0000","last_change":"2024-09-19T08:43:51.131540+0000","last_active":"2024-09-19T08:44:57.708607+0000","last_peered":"2024-09-19T08:44:57.708607+0000","last_clean":"2024-09-19T08:44:57.708607+0000","last_became_active":"2024-09-19T08:43:51.131170+0000","last_became_peered":"2024-09-19T08:43:51.131170+0000","last_unstale":"2024-09-19T08:44:57.708607+0000","last_undegraded":"2024-09-19T08:44:57.708607+0000","last_fullsized":"2024-09-19T08:44:57.708607+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:43.635612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":58,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.822208+0000","last_change":"2024-09-19T08:43:52.763308+0000","last_active":"2024-09-19T08:44:02.822208+0000","last_peered":"2024-09-19T08:44:02.822208+0000","last_clean":"2024-09-19T08:44:02.822208+0000","last_became_active":"2024-09-19T08:43:52.762859+0000","last_became_peered":"2024-09-19T08:43:52.762859+0000","last_unstale":"2024-09-19T08:44:02.822208+0000","last_undegraded":"2024-09-19T08:44:02.822208+0000","last_fullsized":"2024-09-19T08:44:02.822208+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:32:35.083361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":347,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821905+0000","last_change":"2024-09-19T08:42:39.567408+0000","last_active":"2024-09-19T08:44:02.821905+0000","last_peered":"2024-09-19T08:44:02.821905+0000","last_clean":"2024-09-19T08:44:02.821905+0000","last_became_active":"2024-09-19T08:42:39.566967+0000","last_became_peered":"2024-09-19T08:42:39.566967+0000","last_unstale":"2024-09-19T08:44:02.821905+0000","last_undegraded":"2024-09-19T08:44:02.821905+0000","last_fullsized":"2024-09-19T08:44:02.821905+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:14.287944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:14.287944+0000","last_clean_scrub_stamp":"2024-09-19T08:39:14.287944+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:31:03.245386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.638441+0000","last_change":"2024-09-19T08:43:50.751097+0000","last_active":"2024-09-19T08:44:04.638441+0000","last_peered":"2024-09-19T08:44:04.638441+0000","last_clean":"2024-09-19T08:44:04.638441+0000","last_became_active":"2024-09-19T08:43:50.750341+0000","last_became_peered":"2024-09-19T08:43:50.750341+0000","last_unstale":"2024-09-19T08:44:04.638441+0000","last_undegraded":"2024-09-19T08:44:04.638441+0000","last_fullsized":"2024-09-19T08:44:04.638441+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:25.694535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.493670+0000","last_change":"2024-09-19T08:43:52.767905+0000","last_active":"2024-09-19T08:44:03.493670+0000","last_peered":"2024-09-19T08:44:03.493670+0000","last_clean":"2024-09-19T08:44:03.493670+0000","last_became_active":"2024-09-19T08:43:52.767601+0000","last_became_peered":"2024-09-19T08:43:52.767601+0000","last_unstale":"2024-09-19T08:44:03.493670+0000","last_undegraded":"2024-09-19T08:44:03.493670+0000","last_fullsized":"2024-09-19T08:44:03.493670+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:14.519879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.822161+0000","last_change":"2024-09-19T08:43:50.751397+0000","last_active":"2024-09-19T08:44:02.822161+0000","last_peered":"2024-09-19T08:44:02.822161+0000","last_clean":"2024-09-19T08:44:02.822161+0000","last_became_active":"2024-09-19T08:43:50.751079+0000","last_became_peered":"2024-09-19T08:43:50.751079+0000","last_unstale":"2024-09-19T08:44:02.822161+0000","last_undegraded":"2024-09-19T08:44:02.822161+0000","last_fullsized":"2024-09-19T08:44:02.822161+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:29.904786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818371+0000","last_change":"2024-09-19T08:43:52.773348+0000","last_active":"2024-09-19T08:44:02.818371+0000","last_peered":"2024-09-19T08:44:02.818371+0000","last_clean":"2024-09-19T08:44:02.818371+0000","last_became_active":"2024-09-19T08:43:52.773011+0000","last_became_peered":"2024-09-19T08:43:52.773011+0000","last_unstale":"2024-09-19T08:44:02.818371+0000","last_undegraded":"2024-09-19T08:44:02.818371+0000","last_fullsized":"2024-09-19T08:44:02.818371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:10.806578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":80,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:54.710341+0000","last_change":"2024-09-19T08:43:51.025300+0000","last_active":"2024-09-19T08:44:54.710341+0000","last_peered":"2024-09-19T08:44:54.710341+0000","last_clean":"2024-09-19T08:44:54.710341+0000","last_became_active":"2024-09-19T08:43:51.024936+0000","last_became_peered":"2024-09-19T08:43:51.024936+0000","last_unstale":"2024-09-19T08:44:54.710341+0000","last_undegraded":"2024-09-19T08:44:54.710341+0000","last_fullsized":"2024-09-19T08:44:54.710341+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:03.186066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821758+0000","last_change":"2024-09-19T08:43:52.774962+0000","last_active":"2024-09-19T08:44:02.821758+0000","last_peered":"2024-09-19T08:44:02.821758+0000","last_clean":"2024-09-19T08:44:02.821758+0000","last_became_active":"2024-09-19T08:43:52.774598+0000","last_became_peered":"2024-09-19T08:43:52.774598+0000","last_unstale":"2024-09-19T08:44:02.821758+0000","last_undegraded":"2024-09-19T08:44:02.821758+0000","last_fullsized":"2024-09-19T08:44:02.821758+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:02.687136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'5","reported_seq":78,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:53.717897+0000","last_change":"2024-09-19T08:43:50.748180+0000","last_active":"2024-09-19T08:44:53.717897+0000","last_peered":"2024-09-19T08:44:53.717897+0000","last_clean":"2024-09-19T08:44:53.717897+0000","last_became_active":"2024-09-19T08:43:50.747853+0000","last_became_peered":"2024-09-19T08:43:50.747853+0000","last_unstale":"2024-09-19T08:44:53.717897+0000","last_undegraded":"2024-09-19T08:44:53.717897+0000","last_fullsized":"2024-09-19T08:44:53.717897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:56.047259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.817252+0000","last_change":"2024-09-19T08:43:52.770253+0000","last_active":"2024-09-19T08:44:02.817252+0000","last_peered":"2024-09-19T08:44:02.817252+0000","last_clean":"2024-09-19T08:44:02.817252+0000","last_became_active":"2024-09-19T08:43:52.769951+0000","last_became_peered":"2024-09-19T08:43:52.769951+0000","last_unstale":"2024-09-19T08:44:02.817252+0000","last_undegraded":"2024-09-19T08:44:02.817252+0000","last_fullsized":"2024-09-19T08:44:02.817252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:02.353144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821619+0000","last_change":"2024-09-19T08:43:51.026453+0000","last_active":"2024-09-19T08:44:02.821619+0000","last_peered":"2024-09-19T08:44:02.821619+0000","last_clean":"2024-09-19T08:44:02.821619+0000","last_became_active":"2024-09-19T08:43:51.025873+0000","last_became_peered":"2024-09-19T08:43:51.025873+0000","last_unstale":"2024-09-19T08:44:02.821619+0000","last_undegraded":"2024-09-19T08:44:02.821619+0000","last_fullsized":"2024-09-19T08:44:02.821619+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:25:17.475074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.817186+0000","last_change":"2024-09-19T08:43:52.761700+0000","last_active":"2024-09-19T08:44:02.817186+0000","last_peered":"2024-09-19T08:44:02.817186+0000","last_clean":"2024-09-19T08:44:02.817186+0000","last_became_active":"2024-09-19T08:43:52.755527+0000","last_became_peered":"2024-09-19T08:43:52.755527+0000","last_unstale":"2024-09-19T08:44:02.817186+0000","last_undegraded":"2024-09-19T08:44:02.817186+0000","last_fullsized":"2024-09-19T08:44:02.817186+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:18:43.171284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.630369+0000","last_change":"2024-09-19T08:43:51.026136+0000","last_active":"2024-09-19T08:44:04.630369+0000","last_peered":"2024-09-19T08:44:04.630369+0000","last_clean":"2024-09-19T08:44:04.630369+0000","last_became_active":"2024-09-19T08:43:51.025740+0000","last_became_peered":"2024-09-19T08:43:51.025740+0000","last_unstale":"2024-09-19T08:44:04.630369+0000","last_undegraded":"2024-09-19T08:44:04.630369+0000","last_fullsized":"2024-09-19T08:44:04.630369+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:17:32.741083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819140+0000","last_change":"2024-09-19T08:43:52.763808+0000","last_active":"2024-09-19T08:44:02.819140+0000","last_peered":"2024-09-19T08:44:02.819140+0000","last_clean":"2024-09-19T08:44:02.819140+0000","last_became_active":"2024-09-19T08:43:52.763509+0000","last_became_peered":"2024-09-19T08:43:52.763509+0000","last_unstale":"2024-09-19T08:44:02.819140+0000","last_undegraded":"2024-09-19T08:44:02.819140+0000","last_fullsized":"2024-09-19T08:44:02.819140+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:45:08.385419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'15","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:17.772532+0000","last_change":"2024-09-19T08:43:50.744213+0000","last_active":"2024-09-19T08:44:17.772532+0000","last_peered":"2024-09-19T08:44:17.772532+0000","last_clean":"2024-09-19T08:44:17.772532+0000","last_became_active":"2024-09-19T08:43:50.743848+0000","last_became_peered":"2024-09-19T08:43:50.743848+0000","last_unstale":"2024-09-19T08:44:17.772532+0000","last_undegraded":"2024-09-19T08:44:17.772532+0000","last_fullsized":"2024-09-19T08:44:17.772532+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:26.613055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.494010+0000","last_change":"2024-09-19T08:43:52.767102+0000","last_active":"2024-09-19T08:44:03.494010+0000","last_peered":"2024-09-19T08:44:03.494010+0000","last_clean":"2024-09-19T08:44:03.494010+0000","last_became_active":"2024-09-19T08:43:52.766262+0000","last_became_peered":"2024-09-19T08:43:52.766262+0000","last_unstale":"2024-09-19T08:44:03.494010+0000","last_undegraded":"2024-09-19T08:44:03.494010+0000","last_fullsized":"2024-09-19T08:44:03.494010+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:35.740462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.638198+0000","last_change":"2024-09-19T08:43:51.131742+0000","last_active":"2024-09-19T08:44:04.638198+0000","last_peered":"2024-09-19T08:44:04.638198+0000","last_clean":"2024-09-19T08:44:04.638198+0000","last_became_active":"2024-09-19T08:43:51.131388+0000","last_became_peered":"2024-09-19T08:43:51.131388+0000","last_unstale":"2024-09-19T08:44:04.638198+0000","last_undegraded":"2024-09-19T08:44:04.638198+0000","last_fullsized":"2024-09-19T08:44:04.638198+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:26.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819357+0000","last_change":"2024-09-19T08:43:52.756004+0000","last_active":"2024-09-19T08:44:02.819357+0000","last_peered":"2024-09-19T08:44:02.819357+0000","last_clean":"2024-09-19T08:44:02.819357+0000","last_became_active":"2024-09-19T08:43:52.755596+0000","last_became_peered":"2024-09-19T08:43:52.755596+0000","last_unstale":"2024-09-19T08:44:02.819357+0000","last_undegraded":"2024-09-19T08:44:02.819357+0000","last_fullsized":"2024-09-19T08:44:02.819357+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:35.942375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.835986+0000","last_change":"2024-09-19T08:44:02.835986+0000","last_active":"2024-09-19T08:44:02.835986+0000","last_peered":"2024-09-19T08:44:02.835986+0000","last_clean":"2024-09-19T08:44:02.835986+0000","last_became_active":"2024-09-19T08:44:02.826314+0000","last_became_peered":"2024-09-19T08:44:02.826314+0000","last_unstale":"2024-09-19T08:44:02.835986+0000","last_undegraded":"2024-09-19T08:44:02.835986+0000","last_fullsized":"2024-09-19T08:44:02.835986+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:16.943989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818539+0000","last_change":"2024-09-19T08:43:52.766582+0000","last_active":"2024-09-19T08:44:02.818539+0000","last_peered":"2024-09-19T08:44:02.818539+0000","last_clean":"2024-09-19T08:44:02.818539+0000","last_became_active":"2024-09-19T08:43:52.766232+0000","last_became_peered":"2024-09-19T08:43:52.766232+0000","last_unstale":"2024-09-19T08:44:02.818539+0000","last_undegraded":"2024-09-19T08:44:02.818539+0000","last_fullsized":"2024-09-19T08:44:02.818539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:43:42.319422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.635810+0000","last_change":"2024-09-19T08:43:50.747611+0000","last_active":"2024-09-19T08:44:04.635810+0000","last_peered":"2024-09-19T08:44:04.635810+0000","last_clean":"2024-09-19T08:44:04.635810+0000","last_became_active":"2024-09-19T08:43:50.747263+0000","last_became_peered":"2024-09-19T08:43:50.747263+0000","last_unstale":"2024-09-19T08:44:04.635810+0000","last_undegraded":"2024-09-19T08:44:04.635810+0000","last_fullsized":"2024-09-19T08:44:04.635810+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:14.717820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.816851+0000","last_change":"2024-09-19T08:43:52.766493+0000","last_active":"2024-09-19T08:44:02.816851+0000","last_peered":"2024-09-19T08:44:02.816851+0000","last_clean":"2024-09-19T08:44:02.816851+0000","last_became_active":"2024-09-19T08:43:52.766158+0000","last_became_peered":"2024-09-19T08:43:52.766158+0000","last_unstale":"2024-09-19T08:44:02.816851+0000","last_undegraded":"2024-09-19T08:44:02.816851+0000","last_fullsized":"2024-09-19T08:44:02.816851+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:59.175949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.838733+0000","last_change":"2024-09-19T08:43:50.751368+0000","last_active":"2024-09-19T08:44:03.838733+0000","last_peered":"2024-09-19T08:44:03.838733+0000","last_clean":"2024-09-19T08:44:03.838733+0000","last_became_active":"2024-09-19T08:43:50.751038+0000","last_became_peered":"2024-09-19T08:43:50.751038+0000","last_unstale":"2024-09-19T08:44:03.838733+0000","last_undegraded":"2024-09-19T08:44:03.838733+0000","last_fullsized":"2024-09-19T08:44:03.838733+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:46.302996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.493641+0000","last_change":"2024-09-19T08:43:52.756998+0000","last_active":"2024-09-19T08:44:03.493641+0000","last_peered":"2024-09-19T08:44:03.493641+0000","last_clean":"2024-09-19T08:44:03.493641+0000","last_became_active":"2024-09-19T08:43:52.756661+0000","last_became_peered":"2024-09-19T08:43:52.756661+0000","last_unstale":"2024-09-19T08:44:03.493641+0000","last_undegraded":"2024-09-19T08:44:03.493641+0000","last_fullsized":"2024-09-19T08:44:03.493641+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:15.182574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.818827+0000","last_change":"2024-09-19T08:43:50.750095+0000","last_active":"2024-09-19T08:44:02.818827+0000","last_peered":"2024-09-19T08:44:02.818827+0000","last_clean":"2024-09-19T08:44:02.818827+0000","last_became_active":"2024-09-19T08:43:50.749672+0000","last_became_peered":"2024-09-19T08:43:50.749672+0000","last_unstale":"2024-09-19T08:44:02.818827+0000","last_undegraded":"2024-09-19T08:44:02.818827+0000","last_fullsized":"2024-09-19T08:44:02.818827+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:25:34.978094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.821907+0000","last_change":"2024-09-19T08:43:52.764097+0000","last_active":"2024-09-19T08:44:02.821907+0000","last_peered":"2024-09-19T08:44:02.821907+0000","last_clean":"2024-09-19T08:44:02.821907+0000","last_became_active":"2024-09-19T08:43:52.763773+0000","last_became_peered":"2024-09-19T08:43:52.763773+0000","last_unstale":"2024-09-19T08:44:02.821907+0000","last_undegraded":"2024-09-19T08:44:02.821907+0000","last_fullsized":"2024-09-19T08:44:02.821907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:01:42.098460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.825238+0000","last_change":"2024-09-19T08:44:02.825238+0000","last_active":"2024-09-19T08:44:02.825238+0000","last_peered":"2024-09-19T08:44:02.825238+0000","last_clean":"2024-09-19T08:44:02.825238+0000","last_became_active":"2024-09-19T08:44:02.824873+0000","last_became_peered":"2024-09-19T08:44:02.824873+0000","last_unstale":"2024-09-19T08:44:02.825238+0000","last_undegraded":"2024-09-19T08:44:02.825238+0000","last_fullsized":"2024-09-19T08:44:02.825238+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:38.896082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.639612+0000","last_change":"2024-09-19T08:43:50.748789+0000","last_active":"2024-09-19T08:44:04.639612+0000","last_peered":"2024-09-19T08:44:04.639612+0000","last_clean":"2024-09-19T08:44:04.639612+0000","last_became_active":"2024-09-19T08:43:50.748464+0000","last_became_peered":"2024-09-19T08:43:50.748464+0000","last_unstale":"2024-09-19T08:44:04.639612+0000","last_undegraded":"2024-09-19T08:44:04.639612+0000","last_fullsized":"2024-09-19T08:44:04.639612+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:36:45.905639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.820329+0000","last_change":"2024-09-19T08:43:52.756608+0000","last_active":"2024-09-19T08:44:02.820329+0000","last_peered":"2024-09-19T08:44:02.820329+0000","last_clean":"2024-09-19T08:44:02.820329+0000","last_became_active":"2024-09-19T08:43:52.756227+0000","last_became_peered":"2024-09-19T08:43:52.756227+0000","last_unstale":"2024-09-19T08:44:02.820329+0000","last_undegraded":"2024-09-19T08:44:02.820329+0000","last_fullsized":"2024-09-19T08:44:02.820329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:29:58.085584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'12","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:04.639102+0000","last_change":"2024-09-19T08:43:50.749135+0000","last_active":"2024-09-19T08:44:04.639102+0000","last_peered":"2024-09-19T08:44:04.639102+0000","last_clean":"2024-09-19T08:44:04.639102+0000","last_became_active":"2024-09-19T08:43:50.748755+0000","last_became_peered":"2024-09-19T08:43:50.748755+0000","last_unstale":"2024-09-19T08:44:04.639102+0000","last_undegraded":"2024-09-19T08:44:04.639102+0000","last_fullsized":"2024-09-19T08:44:04.639102+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:55.079780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:02.819170+0000","last_change":"2024-09-19T08:43:52.763277+0000","last_active":"2024-09-19T08:44:02.819170+0000","last_peered":"2024-09-19T08:44:02.819170+0000","last_clean":"2024-09-19T08:44:02.819170+0000","last_became_active":"2024-09-19T08:43:52.762941+0000","last_became_peered":"2024-09-19T08:43:52.762941+0000","last_unstale":"2024-09-19T08:44:02.819170+0000","last_undegraded":"2024-09-19T08:44:02.819170+0000","last_fullsized":"2024-09-19T08:44:02.819170+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:35.531946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.494005+0000","last_change":"2024-09-19T08:43:51.025350+0000","last_active":"2024-09-19T08:44:03.494005+0000","last_peered":"2024-09-19T08:44:03.494005+0000","last_clean":"2024-09-19T08:44:03.494005+0000","last_became_active":"2024-09-19T08:43:51.025046+0000","last_became_peered":"2024-09-19T08:43:51.025046+0000","last_unstale":"2024-09-19T08:44:03.494005+0000","last_undegraded":"2024-09-19T08:44:03.494005+0000","last_fullsized":"2024-09-19T08:44:03.494005+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:27.595545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:03.493915+0000","last_change":"2024-09-19T08:43:52.767917+0000","last_active":"2024-09-19T08:44:03.493915+0000","last_peered":"2024-09-19T08:44:03.493915+0000","last_clean":"2024-09-19T08:44:03.493915+0000","last_became_active":"2024-09-19T08:43:52.767575+0000","last_became_peered":"2024-09-19T08:43:52.767575+0000","last_unstale":"2024-09-19T08:44:03.493915+0000","last_undegraded":"2024-09-19T08:44:03.493915+0000","last_fullsized":"2024-09-19T08:44:03.493915+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:05.950399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'13","reported_seq":87,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:44:54.718778+0000","last_change":"2024-09-19T08:43:50.746400+0000","last_active":"2024-09-19T08:44:54.718778+0000","last_peered":"2024-09-19T08:44:54.718778+0000","last_clean":"2024-09-19T08:44:54.718778+0000","last_became_active":"2024-09-19T08:43:50.745891+0000","last_became_peered":"2024-09-19T08:43:50.745891+0000","last_unstale":"2024-09-19T08:44:54.718778+0000","last_undegraded":"2024-09-19T08:44:54.718778+0000","last_fullsized":"2024-09-19T08:44:54.718778+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:03.098202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66419,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":558,"num_read_kb":352,"num_write":241,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":199257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":250201,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579931,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":436653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808864,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":1086388,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005092,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":436195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234025,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":436344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462958,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":970752,"data_stored":1028976,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593780,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":435795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822712,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":983040,"data_stored":1028654,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986237,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":905216,"data_stored":435656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215171,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":435503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":987136,"data_stored":496925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":966656,"data_stored":495919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":925696,"data_stored":437527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34807,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34807,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34807,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:45:01.184 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T08:45:01.185 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-19T08:45:01.185 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-19T08:45:01.185 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-19T08:45:01.434 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:45:01.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.911+0000 7f1095b59640 1 -- 172.21.15.164:0/3908720456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1090101de0 msgr2=0x7f10901021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:01.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.911+0000 7f1095b59640 1 --2- 172.21.15.164:0/3908720456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1090101de0 0x7f10901021c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1080009900 tx=0x7f108002a410 comp rx=0 tx=0).stop 2024-09-19T08:45:01.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.911+0000 7f1095b59640 1 -- 172.21.15.164:0/3908720456 shutdown_connections 2024-09-19T08:45:01.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.911+0000 7f1095b59640 1 --2- 172.21.15.164:0/3908720456 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1090106f10 0x7f109010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.911+0000 7f1095b59640 1 --2- 172.21.15.164:0/3908720456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1090102790 0x7f10901067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.911+0000 7f1095b59640 1 --2- 172.21.15.164:0/3908720456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1090101de0 0x7f10901021c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.911+0000 7f1095b59640 1 -- 172.21.15.164:0/3908720456 >> 172.21.15.164:0/3908720456 conn(0x7f10900fd8c0 msgr2=0x7f10900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:01.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.911+0000 7f1095b59640 1 -- 172.21.15.164:0/3908720456 shutdown_connections 2024-09-19T08:45:01.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.911+0000 7f1095b59640 1 -- 172.21.15.164:0/3908720456 wait complete. 2024-09-19T08:45:01.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.912+0000 7f1095b59640 1 Processor -- start 2024-09-19T08:45:01.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.912+0000 7f1095b59640 1 -- start start 2024-09-19T08:45:01.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.912+0000 7f1095b59640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1090101de0 0x7f10900744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:01.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.912+0000 7f1095b59640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1090102790 0x7f1090074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:01.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.912+0000 7f1095b59640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1090106f10 0x7f1090074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:01.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.912+0000 7f1095b59640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1090111f30 con 0x7f1090106f10 2024-09-19T08:45:01.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.912+0000 7f1095b59640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1090111db0 con 0x7f1090101de0 2024-09-19T08:45:01.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.912+0000 7f1095b59640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f10901120b0 con 0x7f1090102790 2024-09-19T08:45:01.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.912+0000 7f108ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1090106f10 0x7f1090074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:01.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.912+0000 7f108ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1090106f10 0x7f1090074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48844/0 (socket says 172.21.15.164:48844) 2024-09-19T08:45:01.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.912+0000 7f108ffff640 1 -- 172.21.15.164:0/3071296068 learned_addr learned my addr 172.21.15.164:0/3071296068 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:01.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.912+0000 7f1087fff640 1 --2- 172.21.15.164:0/3071296068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1090102790 0x7f1090074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:01.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.912+0000 7f108f7fe640 1 --2- 172.21.15.164:0/3071296068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1090101de0 0x7f10900744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:01.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.912+0000 7f108ffff640 1 -- 172.21.15.164:0/3071296068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1090102790 msgr2=0x7f1090074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:01.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.912+0000 7f108ffff640 1 --2- 172.21.15.164:0/3071296068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1090102790 0x7f1090074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.912+0000 7f108ffff640 1 -- 172.21.15.164:0/3071296068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1090101de0 msgr2=0x7f10900744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:01.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.912+0000 7f108ffff640 1 --2- 172.21.15.164:0/3071296068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1090101de0 0x7f10900744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.912+0000 7f108ffff640 1 -- 172.21.15.164:0/3071296068 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10901a6980 con 0x7f1090106f10 2024-09-19T08:45:01.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.912+0000 7f108f7fe640 1 --2- 172.21.15.164:0/3071296068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1090101de0 0x7f10900744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:01.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.913+0000 7f108ffff640 1 --2- 172.21.15.164:0/3071296068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1090106f10 0x7f1090074f70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f108800eaa0 tx=0x7f108800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:01.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.913+0000 7f108d7fa640 1 -- 172.21.15.164:0/3071296068 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108800cdb0 con 0x7f1090106f10 2024-09-19T08:45:01.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.913+0000 7f108d7fa640 1 -- 172.21.15.164:0/3071296068 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1088005020 con 0x7f1090106f10 2024-09-19T08:45:01.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.913+0000 7f1095b59640 1 -- 172.21.15.164:0/3071296068 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10901a6c70 con 0x7f1090106f10 2024-09-19T08:45:01.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.913+0000 7f108d7fa640 1 -- 172.21.15.164:0/3071296068 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10880116b0 con 0x7f1090106f10 2024-09-19T08:45:01.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.913+0000 7f1095b59640 1 -- 172.21.15.164:0/3071296068 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10901a7160 con 0x7f1090106f10 2024-09-19T08:45:01.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.914+0000 7f1095b59640 1 -- 172.21.15.164:0/3071296068 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10901021c0 con 0x7f1090106f10 2024-09-19T08:45:01.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.915+0000 7f108d7fa640 1 -- 172.21.15.164:0/3071296068 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f1088005560 con 0x7f1090106f10 2024-09-19T08:45:01.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.915+0000 7f108d7fa640 1 --2- 172.21.15.164:0/3071296068 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f107c0a32d0 0x7f107c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:01.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.915+0000 7f108d7fa640 1 -- 172.21.15.164:0/3071296068 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f10880cfbb0 con 0x7f1090106f10 2024-09-19T08:45:01.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.915+0000 7f108f7fe640 1 --2- 172.21.15.164:0/3071296068 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f107c0a32d0 0x7f107c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:01.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.916+0000 7f108f7fe640 1 --2- 172.21.15.164:0/3071296068 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f107c0a32d0 0x7f107c0a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1080005ab0 tx=0x7f1080005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:01.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:01.920+0000 7f108d7fa640 1 -- 172.21.15.164:0/3071296068 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f10880d6080 con 0x7f1090106f10 2024-09-19T08:45:02.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:02.123+0000 7f1095b59640 1 -- 172.21.15.164:0/3071296068 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f109010a0f0 con 0x7f1090106f10 2024-09-19T08:45:02.123 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:01 smithi164 ceph-mon[30928]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:45:02.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:02.123+0000 7f108d7fa640 1 -- 172.21.15.164:0/3071296068 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f1088016030 con 0x7f1090106f10 2024-09-19T08:45:02.124 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:45:02.124 INFO:teuthology.orchestra.run.smithi164.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T08:45:02.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:01 smithi170 ceph-mon[35072]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:45:02.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:02.125+0000 7f1095b59640 1 -- 172.21.15.164:0/3071296068 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f107c0a32d0 msgr2=0x7f107c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:02.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:02.125+0000 7f1095b59640 1 --2- 172.21.15.164:0/3071296068 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f107c0a32d0 0x7f107c0a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1080005ab0 tx=0x7f1080005a20 comp rx=0 tx=0).stop 2024-09-19T08:45:02.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:02.125+0000 7f1095b59640 1 -- 172.21.15.164:0/3071296068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1090106f10 msgr2=0x7f1090074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:02.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:02.125+0000 7f1095b59640 1 --2- 172.21.15.164:0/3071296068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1090106f10 0x7f1090074f70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f108800eaa0 tx=0x7f108800ef60 comp rx=0 tx=0).stop 2024-09-19T08:45:02.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:02.125+0000 7f1095b59640 1 -- 172.21.15.164:0/3071296068 shutdown_connections 2024-09-19T08:45:02.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:02.125+0000 7f1095b59640 1 --2- 172.21.15.164:0/3071296068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1090106f10 0x7f1090074f70 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:02.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:02.125+0000 7f1095b59640 1 --2- 172.21.15.164:0/3071296068 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f107c0a32d0 0x7f107c0a5790 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:02.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:02.125+0000 7f1095b59640 1 --2- 172.21.15.164:0/3071296068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1090102790 0x7f1090074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:02.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:02.125+0000 7f1095b59640 1 --2- 172.21.15.164:0/3071296068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1090101de0 0x7f10900744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:02.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:02.125+0000 7f1095b59640 1 -- 172.21.15.164:0/3071296068 >> 172.21.15.164:0/3071296068 conn(0x7f10900fd8c0 msgr2=0x7f10900fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:02.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:02.125+0000 7f1095b59640 1 -- 172.21.15.164:0/3071296068 shutdown_connections 2024-09-19T08:45:02.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:02.126+0000 7f1095b59640 1 -- 172.21.15.164:0/3071296068 wait complete. 2024-09-19T08:45:02.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:01 smithi173 ceph-mon[35306]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:45:02.834 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-19T08:45:02.834 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:45:03.083 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:45:03.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:02 smithi170 ceph-mon[35072]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T08:45:03.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:02 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3071296068' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T08:45:03.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:02 smithi173 ceph-mon[35306]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T08:45:03.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:02 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3071296068' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T08:45:03.205 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:02 smithi164 ceph-mon[30928]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T08:45:03.607 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:02 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3071296068' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T08:45:03.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.575+0000 7f1ed9a59640 1 -- 172.21.15.164:0/397585465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ed4106f10 msgr2=0x7f1ed410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:03.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.575+0000 7f1ed9a59640 1 --2- 172.21.15.164:0/397585465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ed4106f10 0x7f1ed410ea90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc009900 tx=0x7f1ecc02a410 comp rx=0 tx=0).stop 2024-09-19T08:45:03.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.575+0000 7f1ed9a59640 1 -- 172.21.15.164:0/397585465 shutdown_connections 2024-09-19T08:45:03.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.575+0000 7f1ed9a59640 1 --2- 172.21.15.164:0/397585465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ed4106f10 0x7f1ed410ea90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:03.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.575+0000 7f1ed9a59640 1 --2- 172.21.15.164:0/397585465 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ed4102790 0x7f1ed41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:03.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.575+0000 7f1ed9a59640 1 --2- 172.21.15.164:0/397585465 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ed4101de0 0x7f1ed41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:03.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.575+0000 7f1ed9a59640 1 -- 172.21.15.164:0/397585465 >> 172.21.15.164:0/397585465 conn(0x7f1ed40fd880 msgr2=0x7f1ed40ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:03.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.575+0000 7f1ed9a59640 1 -- 172.21.15.164:0/397585465 shutdown_connections 2024-09-19T08:45:03.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.575+0000 7f1ed9a59640 1 -- 172.21.15.164:0/397585465 wait complete. 2024-09-19T08:45:03.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.575+0000 7f1ed9a59640 1 Processor -- start 2024-09-19T08:45:03.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.576+0000 7f1ed9a59640 1 -- start start 2024-09-19T08:45:03.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.576+0000 7f1ed9a59640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ed4101de0 0x7f1ed41983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:03.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.576+0000 7f1ed9a59640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ed4102790 0x7f1ed4198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:03.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.576+0000 7f1ed9a59640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ed4106f10 0x7f1ed419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:03.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.576+0000 7f1ed9a59640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed4111f90 con 0x7f1ed4106f10 2024-09-19T08:45:03.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.576+0000 7f1ed9a59640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed4111e10 con 0x7f1ed4101de0 2024-09-19T08:45:03.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.576+0000 7f1ed9a59640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed4112110 con 0x7f1ed4102790 2024-09-19T08:45:03.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.576+0000 7f1ed37fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ed4106f10 0x7f1ed419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:03.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.576+0000 7f1ed37fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ed4106f10 0x7f1ed419ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51782/0 (socket says 172.21.15.164:51782) 2024-09-19T08:45:03.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.576+0000 7f1ed37fe640 1 -- 172.21.15.164:0/3395041096 learned_addr learned my addr 172.21.15.164:0/3395041096 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:03.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.576+0000 7f1ed2ffd640 1 --2- 172.21.15.164:0/3395041096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ed4101de0 0x7f1ed41983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:03.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.576+0000 7f1ed27fc640 1 --2- 172.21.15.164:0/3395041096 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ed4102790 0x7f1ed4198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:03.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.576+0000 7f1ed37fe640 1 -- 172.21.15.164:0/3395041096 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ed4102790 msgr2=0x7f1ed4198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:03.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.576+0000 7f1ed37fe640 1 --2- 172.21.15.164:0/3395041096 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ed4102790 0x7f1ed4198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:03.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.576+0000 7f1ed37fe640 1 -- 172.21.15.164:0/3395041096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ed4101de0 msgr2=0x7f1ed41983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:03.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.576+0000 7f1ed37fe640 1 --2- 172.21.15.164:0/3395041096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ed4101de0 0x7f1ed41983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:03.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.576+0000 7f1ed37fe640 1 -- 172.21.15.164:0/3395041096 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed419d3a0 con 0x7f1ed4106f10 2024-09-19T08:45:03.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.576+0000 7f1ed37fe640 1 --2- 172.21.15.164:0/3395041096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ed4106f10 0x7f1ed419ccc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc005ab0 tx=0x7f1ecc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:03.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.577+0000 7f1ed8a57640 1 -- 172.21.15.164:0/3395041096 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc03f030 con 0x7f1ed4106f10 2024-09-19T08:45:03.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.577+0000 7f1ed9a59640 1 -- 172.21.15.164:0/3395041096 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ed419d630 con 0x7f1ed4106f10 2024-09-19T08:45:03.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.577+0000 7f1ed8a57640 1 -- 172.21.15.164:0/3395041096 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ecc003ce0 con 0x7f1ed4106f10 2024-09-19T08:45:03.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.577+0000 7f1ed8a57640 1 -- 172.21.15.164:0/3395041096 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc035070 con 0x7f1ed4106f10 2024-09-19T08:45:03.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.577+0000 7f1ed9a59640 1 -- 172.21.15.164:0/3395041096 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ed41a4f70 con 0x7f1ed4106f10 2024-09-19T08:45:03.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.578+0000 7f1ed9a59640 1 -- 172.21.15.164:0/3395041096 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ed41032c0 con 0x7f1ed4106f10 2024-09-19T08:45:03.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.579+0000 7f1ed8a57640 1 -- 172.21.15.164:0/3395041096 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f1ecc042030 con 0x7f1ed4106f10 2024-09-19T08:45:03.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.579+0000 7f1ed8a57640 1 --2- 172.21.15.164:0/3395041096 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1ebc0a32d0 0x7f1ebc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:03.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.580+0000 7f1ed8a57640 1 -- 172.21.15.164:0/3395041096 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1ecc0f03b0 con 0x7f1ed4106f10 2024-09-19T08:45:03.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.580+0000 7f1ed2ffd640 1 --2- 172.21.15.164:0/3395041096 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1ebc0a32d0 0x7f1ebc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:03.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.580+0000 7f1ed2ffd640 1 --2- 172.21.15.164:0/3395041096 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1ebc0a32d0 0x7f1ebc0a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1ec400a400 tx=0x7f1ec400d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:03.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.583+0000 7f1ed8a57640 1 -- 172.21.15.164:0/3395041096 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1ecc0bd070 con 0x7f1ed4106f10 2024-09-19T08:45:03.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.787+0000 7f1ed9a59640 1 -- 172.21.15.164:0/3395041096 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ed410a150 con 0x7f1ed4106f10 2024-09-19T08:45:03.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.788+0000 7f1ed8a57640 1 -- 172.21.15.164:0/3395041096 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11984 (secure 0 0 0) 0x7f1ecc0bc970 con 0x7f1ed4106f10 2024-09-19T08:45:03.788 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:45:03.789 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":27,"btime":"2024-09-19T08:44:05:632372+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:44:05.632353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24442,24424,24449,14595]},"id":1}]} 2024-09-19T08:45:03.789 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 27 2024-09-19T08:45:03.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.790+0000 7f1ed9a59640 1 -- 172.21.15.164:0/3395041096 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1ebc0a32d0 msgr2=0x7f1ebc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:03.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.790+0000 7f1ed9a59640 1 --2- 172.21.15.164:0/3395041096 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1ebc0a32d0 0x7f1ebc0a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1ec400a400 tx=0x7f1ec400d040 comp rx=0 tx=0).stop 2024-09-19T08:45:03.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.790+0000 7f1ed9a59640 1 -- 172.21.15.164:0/3395041096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ed4106f10 msgr2=0x7f1ed419ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:03.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.790+0000 7f1ed9a59640 1 --2- 172.21.15.164:0/3395041096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ed4106f10 0x7f1ed419ccc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc005ab0 tx=0x7f1ecc0042f0 comp rx=0 tx=0).stop 2024-09-19T08:45:03.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.790+0000 7f1ed9a59640 1 -- 172.21.15.164:0/3395041096 shutdown_connections 2024-09-19T08:45:03.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.790+0000 7f1ed9a59640 1 --2- 172.21.15.164:0/3395041096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ed4106f10 0x7f1ed419ccc0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:03.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.790+0000 7f1ed9a59640 1 --2- 172.21.15.164:0/3395041096 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1ebc0a32d0 0x7f1ebc0a5790 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:03.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.790+0000 7f1ed9a59640 1 --2- 172.21.15.164:0/3395041096 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ed4102790 0x7f1ed4198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:03.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.790+0000 7f1ed9a59640 1 --2- 172.21.15.164:0/3395041096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ed4101de0 0x7f1ed41983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:03.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.790+0000 7f1ed9a59640 1 -- 172.21.15.164:0/3395041096 >> 172.21.15.164:0/3395041096 conn(0x7f1ed40fd880 msgr2=0x7f1ed40feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:03.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.790+0000 7f1ed9a59640 1 -- 172.21.15.164:0/3395041096 shutdown_connections 2024-09-19T08:45:03.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:03.790+0000 7f1ed9a59640 1 -- 172.21.15.164:0/3395041096 wait complete. 2024-09-19T08:45:04.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:03 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3395041096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:45:04.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:03 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3395041096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:45:04.236 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:03 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3395041096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:45:04.474 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:45:04.723 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:45:05.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:04 smithi170 ceph-mon[35072]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:45:05.130 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:04 smithi164 ceph-mon[30928]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:45:05.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:04 smithi173 ceph-mon[35306]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:45:05.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.212+0000 7f57b4677640 1 -- 172.21.15.164:0/1607960078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57ac106f10 msgr2=0x7f57ac10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:05.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.212+0000 7f57b4677640 1 --2- 172.21.15.164:0/1607960078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57ac106f10 0x7f57ac10ea90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f57a8009ca0 tx=0x7f57a802a8c0 comp rx=0 tx=0).stop 2024-09-19T08:45:05.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.212+0000 7f57b4677640 1 -- 172.21.15.164:0/1607960078 shutdown_connections 2024-09-19T08:45:05.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.212+0000 7f57b4677640 1 --2- 172.21.15.164:0/1607960078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57ac106f10 0x7f57ac10ea90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:05.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.212+0000 7f57b4677640 1 --2- 172.21.15.164:0/1607960078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57ac102790 0x7f57ac1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:05.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.212+0000 7f57b4677640 1 --2- 172.21.15.164:0/1607960078 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57ac101de0 0x7f57ac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:05.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.212+0000 7f57b4677640 1 -- 172.21.15.164:0/1607960078 >> 172.21.15.164:0/1607960078 conn(0x7f57ac0fd8c0 msgr2=0x7f57ac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:05.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.212+0000 7f57b4677640 1 -- 172.21.15.164:0/1607960078 shutdown_connections 2024-09-19T08:45:05.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.212+0000 7f57b4677640 1 -- 172.21.15.164:0/1607960078 wait complete. 2024-09-19T08:45:05.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.213+0000 7f57b4677640 1 Processor -- start 2024-09-19T08:45:05.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.213+0000 7f57b4677640 1 -- start start 2024-09-19T08:45:05.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.213+0000 7f57b4677640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57ac101de0 0x7f57ac198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:05.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.213+0000 7f57b4677640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57ac102790 0x7f57ac1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:05.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.213+0000 7f57b4677640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57ac106f10 0x7f57ac19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:05.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.213+0000 7f57b4677640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f57ac111f90 con 0x7f57ac101de0 2024-09-19T08:45:05.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.213+0000 7f57b4677640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f57ac111e10 con 0x7f57ac106f10 2024-09-19T08:45:05.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.213+0000 7f57b4677640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f57ac112110 con 0x7f57ac102790 2024-09-19T08:45:05.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.213+0000 7f57b2413640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57ac101de0 0x7f57ac198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:05.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.213+0000 7f57b2413640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57ac101de0 0x7f57ac198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51804/0 (socket says 172.21.15.164:51804) 2024-09-19T08:45:05.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.213+0000 7f57b2413640 1 -- 172.21.15.164:0/650107984 learned_addr learned my addr 172.21.15.164:0/650107984 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:05.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.213+0000 7f57b2c14640 1 --2- 172.21.15.164:0/650107984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57ac106f10 0x7f57ac19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:05.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.213+0000 7f57b1c12640 1 --2- 172.21.15.164:0/650107984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57ac102790 0x7f57ac1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:05.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.213+0000 7f57b2413640 1 -- 172.21.15.164:0/650107984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57ac102790 msgr2=0x7f57ac1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:05.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.213+0000 7f57b2413640 1 --2- 172.21.15.164:0/650107984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57ac102790 0x7f57ac1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:05.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.213+0000 7f57b2413640 1 -- 172.21.15.164:0/650107984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57ac106f10 msgr2=0x7f57ac19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:05.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.213+0000 7f57b2413640 1 --2- 172.21.15.164:0/650107984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57ac106f10 0x7f57ac19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:05.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.213+0000 7f57b2413640 1 -- 172.21.15.164:0/650107984 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57ac19d420 con 0x7f57ac101de0 2024-09-19T08:45:05.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.213+0000 7f57b2c14640 1 --2- 172.21.15.164:0/650107984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57ac106f10 0x7f57ac19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:05.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.213+0000 7f57b1c12640 1 --2- 172.21.15.164:0/650107984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57ac102790 0x7f57ac1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:05.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.213+0000 7f57b2413640 1 --2- 172.21.15.164:0/650107984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57ac101de0 0x7f57ac198470 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f57a400b4a0 tx=0x7f57a400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:05.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.214+0000 7f57a37fe640 1 -- 172.21.15.164:0/650107984 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a4014070 con 0x7f57ac101de0 2024-09-19T08:45:05.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.214+0000 7f57a37fe640 1 -- 172.21.15.164:0/650107984 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57a4010af0 con 0x7f57ac101de0 2024-09-19T08:45:05.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.214+0000 7f57b4677640 1 -- 172.21.15.164:0/650107984 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57ac19d710 con 0x7f57ac101de0 2024-09-19T08:45:05.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.214+0000 7f57a37fe640 1 -- 172.21.15.164:0/650107984 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a4017d10 con 0x7f57ac101de0 2024-09-19T08:45:05.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.214+0000 7f57b4677640 1 -- 172.21.15.164:0/650107984 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57ac1a4ff0 con 0x7f57ac101de0 2024-09-19T08:45:05.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.215+0000 7f57b4677640 1 -- 172.21.15.164:0/650107984 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57ac1021c0 con 0x7f57ac101de0 2024-09-19T08:45:05.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.216+0000 7f57a37fe640 1 -- 172.21.15.164:0/650107984 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f57a4017070 con 0x7f57ac101de0 2024-09-19T08:45:05.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.216+0000 7f57a37fe640 1 --2- 172.21.15.164:0/650107984 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f57980a32d0 0x7f57980a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:05.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.216+0000 7f57b1c12640 1 --2- 172.21.15.164:0/650107984 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f57980a32d0 0x7f57980a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:05.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.216+0000 7f57a37fe640 1 -- 172.21.15.164:0/650107984 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f57a40ced10 con 0x7f57ac101de0 2024-09-19T08:45:05.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.217+0000 7f57b1c12640 1 --2- 172.21.15.164:0/650107984 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f57980a32d0 0x7f57980a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f57ac1088e0 tx=0x7f579c005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:05.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.221+0000 7f57a37fe640 1 -- 172.21.15.164:0/650107984 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f57a409bae0 con 0x7f57ac101de0 2024-09-19T08:45:05.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.423+0000 7f57b4677640 1 -- 172.21.15.164:0/650107984 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57ac02d7c0 con 0x7f57ac101de0 2024-09-19T08:45:05.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.424+0000 7f57a37fe640 1 -- 172.21.15.164:0/650107984 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11984 (secure 0 0 0) 0x7f57a409b3e0 con 0x7f57ac101de0 2024-09-19T08:45:05.424 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:45:05.424 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":27,"btime":"2024-09-19T08:44:05:632372+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:44:05.632353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24442,24424,24449,14595]},"id":1}]} 2024-09-19T08:45:05.425 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 27 2024-09-19T08:45:05.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.426+0000 7f57b4677640 1 -- 172.21.15.164:0/650107984 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f57980a32d0 msgr2=0x7f57980a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:05.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.426+0000 7f57b4677640 1 --2- 172.21.15.164:0/650107984 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f57980a32d0 0x7f57980a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f57ac1088e0 tx=0x7f579c005d30 comp rx=0 tx=0).stop 2024-09-19T08:45:05.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.426+0000 7f57b4677640 1 -- 172.21.15.164:0/650107984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57ac101de0 msgr2=0x7f57ac198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:05.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.426+0000 7f57b4677640 1 --2- 172.21.15.164:0/650107984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57ac101de0 0x7f57ac198470 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f57a400b4a0 tx=0x7f57a400b960 comp rx=0 tx=0).stop 2024-09-19T08:45:05.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.426+0000 7f57b4677640 1 -- 172.21.15.164:0/650107984 shutdown_connections 2024-09-19T08:45:05.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.426+0000 7f57b4677640 1 --2- 172.21.15.164:0/650107984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57ac106f10 0x7f57ac19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:05.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.426+0000 7f57b4677640 1 --2- 172.21.15.164:0/650107984 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f57980a32d0 0x7f57980a5790 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:05.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.426+0000 7f57b4677640 1 --2- 172.21.15.164:0/650107984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57ac102790 0x7f57ac1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:05.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.426+0000 7f57b4677640 1 --2- 172.21.15.164:0/650107984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57ac101de0 0x7f57ac198470 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:05.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.426+0000 7f57b4677640 1 -- 172.21.15.164:0/650107984 >> 172.21.15.164:0/650107984 conn(0x7f57ac0fd8c0 msgr2=0x7f57ac0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:05.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.426+0000 7f57b4677640 1 -- 172.21.15.164:0/650107984 shutdown_connections 2024-09-19T08:45:05.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:05.426+0000 7f57b4677640 1 -- 172.21.15.164:0/650107984 wait complete. 2024-09-19T08:45:06.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:05 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/650107984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:45:06.127 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:05 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/650107984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:45:06.155 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T08:45:06.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:05 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/650107984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:45:06.399 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:45:06.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.928+0000 7f21e56cc640 1 -- 172.21.15.164:0/3209489870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21e0102d40 msgr2=0x7f21e0103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:06.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.928+0000 7f21e56cc640 1 --2- 172.21.15.164:0/3209489870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21e0102d40 0x7f21e0103120 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f21c8009900 tx=0x7f21c802a480 comp rx=0 tx=0).stop 2024-09-19T08:45:06.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.928+0000 7f21e56cc640 1 -- 172.21.15.164:0/3209489870 shutdown_connections 2024-09-19T08:45:06.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.928+0000 7f21e56cc640 1 --2- 172.21.15.164:0/3209489870 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21e010c170 0x7f21e010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:06.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.928+0000 7f21e56cc640 1 --2- 172.21.15.164:0/3209489870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21e0103660 0x7f21e010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:06.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.928+0000 7f21e56cc640 1 --2- 172.21.15.164:0/3209489870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21e0102d40 0x7f21e0103120 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:06.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.928+0000 7f21e56cc640 1 -- 172.21.15.164:0/3209489870 >> 172.21.15.164:0/3209489870 conn(0x7f21e00fc7a0 msgr2=0x7f21e00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:06.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.928+0000 7f21e56cc640 1 -- 172.21.15.164:0/3209489870 shutdown_connections 2024-09-19T08:45:06.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.928+0000 7f21e56cc640 1 -- 172.21.15.164:0/3209489870 wait complete. 2024-09-19T08:45:06.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.929+0000 7f21e56cc640 1 Processor -- start 2024-09-19T08:45:06.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.929+0000 7f21e56cc640 1 -- start start 2024-09-19T08:45:06.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.929+0000 7f21e56cc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21e0102d40 0x7f21e01009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:06.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.929+0000 7f21e56cc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21e0103660 0x7f21e0100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:06.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.929+0000 7f21e56cc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21e010c170 0x7f21e0101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:06.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.929+0000 7f21e56cc640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f21e0111f70 con 0x7f21e0103660 2024-09-19T08:45:06.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.929+0000 7f21e56cc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f21e0111df0 con 0x7f21e0102d40 2024-09-19T08:45:06.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.929+0000 7f21e56cc640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f21e01120f0 con 0x7f21e010c170 2024-09-19T08:45:06.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.929+0000 7f21de7fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21e0103660 0x7f21e0100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:06.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.930+0000 7f21de7fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21e0103660 0x7f21e0100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51824/0 (socket says 172.21.15.164:51824) 2024-09-19T08:45:06.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.930+0000 7f21de7fc640 1 -- 172.21.15.164:0/4276470520 learned_addr learned my addr 172.21.15.164:0/4276470520 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:06.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.930+0000 7f21deffd640 1 --2- 172.21.15.164:0/4276470520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21e0102d40 0x7f21e01009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:06.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.930+0000 7f21df7fe640 1 --2- 172.21.15.164:0/4276470520 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21e010c170 0x7f21e0101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:06.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.930+0000 7f21de7fc640 1 -- 172.21.15.164:0/4276470520 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21e010c170 msgr2=0x7f21e0101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:06.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.930+0000 7f21de7fc640 1 --2- 172.21.15.164:0/4276470520 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21e010c170 0x7f21e0101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:06.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.930+0000 7f21de7fc640 1 -- 172.21.15.164:0/4276470520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21e0102d40 msgr2=0x7f21e01009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:06.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.930+0000 7f21de7fc640 1 --2- 172.21.15.164:0/4276470520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21e0102d40 0x7f21e01009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:06.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.930+0000 7f21de7fc640 1 -- 172.21.15.164:0/4276470520 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e01a68e0 con 0x7f21e0103660 2024-09-19T08:45:06.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.930+0000 7f21df7fe640 1 --2- 172.21.15.164:0/4276470520 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21e010c170 0x7f21e0101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:06.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.930+0000 7f21de7fc640 1 --2- 172.21.15.164:0/4276470520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21e0103660 0x7f21e0100ee0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f21d000c970 tx=0x7f21d000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:06.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.930+0000 7f21d7fff640 1 -- 172.21.15.164:0/4276470520 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d0007bb0 con 0x7f21e0103660 2024-09-19T08:45:06.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.930+0000 7f21d7fff640 1 -- 172.21.15.164:0/4276470520 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21d0007d50 con 0x7f21e0103660 2024-09-19T08:45:06.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.930+0000 7f21d7fff640 1 -- 172.21.15.164:0/4276470520 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d0018820 con 0x7f21e0103660 2024-09-19T08:45:06.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.930+0000 7f21e56cc640 1 -- 172.21.15.164:0/4276470520 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21e01a6bd0 con 0x7f21e0103660 2024-09-19T08:45:06.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.930+0000 7f21e56cc640 1 -- 172.21.15.164:0/4276470520 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21e01a7160 con 0x7f21e0103660 2024-09-19T08:45:06.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.931+0000 7f21e56cc640 1 -- 172.21.15.164:0/4276470520 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21e0103120 con 0x7f21e0103660 2024-09-19T08:45:06.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.932+0000 7f21d7fff640 1 -- 172.21.15.164:0/4276470520 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f21d00189c0 con 0x7f21e0103660 2024-09-19T08:45:06.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.933+0000 7f21d7fff640 1 --2- 172.21.15.164:0/4276470520 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f21cc0a3350 0x7f21cc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:06.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.933+0000 7f21deffd640 1 --2- 172.21.15.164:0/4276470520 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f21cc0a3350 0x7f21cc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:06.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.933+0000 7f21d7fff640 1 -- 172.21.15.164:0/4276470520 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f21d00cf9b0 con 0x7f21e0103660 2024-09-19T08:45:06.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.934+0000 7f21deffd640 1 --2- 172.21.15.164:0/4276470520 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f21cc0a3350 0x7f21cc0a5810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f21c8009230 tx=0x7f21c80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:06.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:06.937+0000 7f21d7fff640 1 -- 172.21.15.164:0/4276470520 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f21d0014030 con 0x7f21e0103660 2024-09-19T08:45:07.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:07.105+0000 7f21e56cc640 1 -- 172.21.15.164:0/4276470520 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f21e0107240 con 0x7f21e0103660 2024-09-19T08:45:07.106 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:06 smithi164 ceph-mon[30928]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T08:45:07.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:07.106+0000 7f21d7fff640 1 -- 172.21.15.164:0/4276470520 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22702 (secure 0 0 0) 0x7f21d009c670 con 0x7f21e0103660 2024-09-19T08:45:07.107 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:45:07.107 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":82,"fsid":"1e4b0d74-7662-11ef-bceb-c7b262605968","created":"2024-09-19T08:36:09.564509+0000","modified":"2024-09-19T08:44:02.811236+0000","last_up_change":"2024-09-19T08:43:01.260852+0000","last_in_change":"2024-09-19T08:42:37.492464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:39:13.961749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:43:49.613471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T08:43:51.570675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33b11f37-6ac5-41fc-8cfd-1b08c7eef688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6803","nonce":663038243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6805","nonce":663038243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6809","nonce":663038243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6807","nonce":663038243}]},"public_addr":"172.21.15.164:6803/663038243","cluster_addr":"172.21.15.164:6805/663038243","heartbeat_back_addr":"172.21.15.164:6809/663038243","heartbeat_front_addr":"172.21.15.164:6807/663038243","state":["exists","up"]},{"osd":1,"uuid":"6de64b0a-46e2-4a98-8005-a326a64bc87d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6801","nonce":2167066371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6803","nonce":2167066371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6807","nonce":2167066371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6805","nonce":2167066371}]},"public_addr":"172.21.15.170:6801/2167066371","cluster_addr":"172.21.15.170:6803/2167066371","heartbeat_back_addr":"172.21.15.170:6807/2167066371","heartbeat_front_addr":"172.21.15.170:6805/2167066371","state":["exists","up"]},{"osd":2,"uuid":"d4158cc7-9822-42a7-9601-d65089907584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6801","nonce":1646561048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6803","nonce":1646561048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6807","nonce":1646561048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6805","nonce":1646561048}]},"public_addr":"172.21.15.173:6801/1646561048","cluster_addr":"172.21.15.173:6803/1646561048","heartbeat_back_addr":"172.21.15.173:6807/1646561048","heartbeat_front_addr":"172.21.15.173:6805/1646561048","state":["exists","up"]},{"osd":3,"uuid":"0cf3f7f1-3a8c-42ce-88a9-9bea52e02221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6811","nonce":406795560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6813","nonce":406795560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6817","nonce":406795560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6815","nonce":406795560}]},"public_addr":"172.21.15.164:6811/406795560","cluster_addr":"172.21.15.164:6813/406795560","heartbeat_back_addr":"172.21.15.164:6817/406795560","heartbeat_front_addr":"172.21.15.164:6815/406795560","state":["exists","up"]},{"osd":4,"uuid":"6dca048a-2860-4a02-8d8e-450b30bc1a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6809","nonce":4120832165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6811","nonce":4120832165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6815","nonce":4120832165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6813","nonce":4120832165}]},"public_addr":"172.21.15.170:6809/4120832165","cluster_addr":"172.21.15.170:6811/4120832165","heartbeat_back_addr":"172.21.15.170:6815/4120832165","heartbeat_front_addr":"172.21.15.170:6813/4120832165","state":["exists","up"]},{"osd":5,"uuid":"3822521f-69d6-4141-a83f-5fc9cecc7b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6809","nonce":2914653552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6811","nonce":2914653552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6815","nonce":2914653552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6813","nonce":2914653552}]},"public_addr":"172.21.15.173:6809/2914653552","cluster_addr":"172.21.15.173:6811/2914653552","heartbeat_back_addr":"172.21.15.173:6815/2914653552","heartbeat_front_addr":"172.21.15.173:6813/2914653552","state":["exists","up"]},{"osd":6,"uuid":"0f5a965e-2eb2-4c24-b089-577e0639c160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6819","nonce":1138341393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6821","nonce":1138341393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6825","nonce":1138341393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6823","nonce":1138341393}]},"public_addr":"172.21.15.164:6819/1138341393","cluster_addr":"172.21.15.164:6821/1138341393","heartbeat_back_addr":"172.21.15.164:6825/1138341393","heartbeat_front_addr":"172.21.15.164:6823/1138341393","state":["exists","up"]},{"osd":7,"uuid":"f16cb2c3-639b-46ae-a740-1bd28ca1863c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6817","nonce":2124160360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6819","nonce":2124160360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6823","nonce":2124160360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6821","nonce":2124160360}]},"public_addr":"172.21.15.170:6817/2124160360","cluster_addr":"172.21.15.170:6819/2124160360","heartbeat_back_addr":"172.21.15.170:6823/2124160360","heartbeat_front_addr":"172.21.15.170:6821/2124160360","state":["exists","up"]},{"osd":8,"uuid":"161cff01-46ad-48e3-b999-a5473a2ad618","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6817","nonce":2285175061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6819","nonce":2285175061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6823","nonce":2285175061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6821","nonce":2285175061}]},"public_addr":"172.21.15.173:6817/2285175061","cluster_addr":"172.21.15.173:6819/2285175061","heartbeat_back_addr":"172.21.15.173:6823/2285175061","heartbeat_front_addr":"172.21.15.173:6821/2285175061","state":["exists","up"]},{"osd":9,"uuid":"994666e8-8fbd-4231-ac67-81a79aaf7c2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6827","nonce":3918847952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6829","nonce":3918847952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6833","nonce":3918847952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6831","nonce":3918847952}]},"public_addr":"172.21.15.164:6827/3918847952","cluster_addr":"172.21.15.164:6829/3918847952","heartbeat_back_addr":"172.21.15.164:6833/3918847952","heartbeat_front_addr":"172.21.15.164:6831/3918847952","state":["exists","up"]},{"osd":10,"uuid":"ff5e9099-cdb4-4905-b888-5b67320ef76b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6825","nonce":2707848227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6827","nonce":2707848227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6831","nonce":2707848227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6829","nonce":2707848227}]},"public_addr":"172.21.15.170:6825/2707848227","cluster_addr":"172.21.15.170:6827/2707848227","heartbeat_back_addr":"172.21.15.170:6831/2707848227","heartbeat_front_addr":"172.21.15.170:6829/2707848227","state":["exists","up"]},{"osd":11,"uuid":"4d93088f-e791-4938-a207-12f15b56abd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6825","nonce":2831924738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6827","nonce":2831924738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6831","nonce":2831924738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6829","nonce":2831924738}]},"public_addr":"172.21.15.173:6825/2831924738","cluster_addr":"172.21.15.173:6827/2831924738","heartbeat_back_addr":"172.21.15.173:6831/2831924738","heartbeat_front_addr":"172.21.15.173:6829/2831924738","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:21.787352+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:46.012323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:39:11.353040+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:39:37.015124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:02.063379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:27.892865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:53.372041+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:41:17.855284+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:41:43.389882+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:08.956895+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:34.069779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:59.303897+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.164:0/339032808":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6801/1173822158":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6800/1173822158":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6801/2805828009":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1837164919":"2024-09-20T08:37:01.663065+0000","172.21.15.164:0/4018459143":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6800/2805828009":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/2770797546":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1905394497":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1369265436":"2024-09-20T08:36:33.856642+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:45:07.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:07.109+0000 7f21e56cc640 1 -- 172.21.15.164:0/4276470520 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f21cc0a3350 msgr2=0x7f21cc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:07.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:07.109+0000 7f21e56cc640 1 --2- 172.21.15.164:0/4276470520 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f21cc0a3350 0x7f21cc0a5810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f21c8009230 tx=0x7f21c80058d0 comp rx=0 tx=0).stop 2024-09-19T08:45:07.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:07.109+0000 7f21e56cc640 1 -- 172.21.15.164:0/4276470520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21e0103660 msgr2=0x7f21e0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:07.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:07.109+0000 7f21e56cc640 1 --2- 172.21.15.164:0/4276470520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21e0103660 0x7f21e0100ee0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f21d000c970 tx=0x7f21d000ce30 comp rx=0 tx=0).stop 2024-09-19T08:45:07.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:07.109+0000 7f21e56cc640 1 -- 172.21.15.164:0/4276470520 shutdown_connections 2024-09-19T08:45:07.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:07.109+0000 7f21e56cc640 1 --2- 172.21.15.164:0/4276470520 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f21cc0a3350 0x7f21cc0a5810 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:07.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:07.109+0000 7f21e56cc640 1 --2- 172.21.15.164:0/4276470520 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21e010c170 0x7f21e0101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:07.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:07.109+0000 7f21e56cc640 1 --2- 172.21.15.164:0/4276470520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21e0103660 0x7f21e0100ee0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:07.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:07.109+0000 7f21e56cc640 1 --2- 172.21.15.164:0/4276470520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21e0102d40 0x7f21e01009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:07.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:07.109+0000 7f21e56cc640 1 -- 172.21.15.164:0/4276470520 >> 172.21.15.164:0/4276470520 conn(0x7f21e00fc7a0 msgr2=0x7f21e00fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:07.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:07.109+0000 7f21e56cc640 1 -- 172.21.15.164:0/4276470520 shutdown_connections 2024-09-19T08:45:07.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:07.109+0000 7f21e56cc640 1 -- 172.21.15.164:0/4276470520 wait complete. 2024-09-19T08:45:07.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:06 smithi170 ceph-mon[35072]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T08:45:07.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:06 smithi173 ceph-mon[35306]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T08:45:07.943 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:45:08.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:07 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/4276470520' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:45:08.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:07 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/4276470520' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:45:08.192 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:45:08.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:07 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/4276470520' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:45:08.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.661+0000 7f3c76706640 1 -- 172.21.15.164:0/1479222264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c70102570 msgr2=0x7f3c701029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:08.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.661+0000 7f3c76706640 1 --2- 172.21.15.164:0/1479222264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c70102570 0x7f3c701029b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3c58009900 tx=0x7f3c5802a480 comp rx=0 tx=0).stop 2024-09-19T08:45:08.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.661+0000 7f3c76706640 1 -- 172.21.15.164:0/1479222264 shutdown_connections 2024-09-19T08:45:08.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.661+0000 7f3c76706640 1 --2- 172.21.15.164:0/1479222264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c70102ef0 0x7f3c7010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:08.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.661+0000 7f3c76706640 1 --2- 172.21.15.164:0/1479222264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c70102570 0x7f3c701029b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:08.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.661+0000 7f3c76706640 1 --2- 172.21.15.164:0/1479222264 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c70100e20 0x7f3c70101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:08.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.661+0000 7f3c76706640 1 -- 172.21.15.164:0/1479222264 >> 172.21.15.164:0/1479222264 conn(0x7f3c700fc7d0 msgr2=0x7f3c700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:08.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.661+0000 7f3c76706640 1 -- 172.21.15.164:0/1479222264 shutdown_connections 2024-09-19T08:45:08.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.661+0000 7f3c76706640 1 -- 172.21.15.164:0/1479222264 wait complete. 2024-09-19T08:45:08.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.661+0000 7f3c76706640 1 Processor -- start 2024-09-19T08:45:08.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.661+0000 7f3c76706640 1 -- start start 2024-09-19T08:45:08.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.662+0000 7f3c76706640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c70100e20 0x7f3c70198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:08.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.662+0000 7f3c76706640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c70102570 0x7f3c70198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:08.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.662+0000 7f3c76706640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c70102ef0 0x7f3c7019ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:08.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.662+0000 7f3c76706640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3c70110fa0 con 0x7f3c70100e20 2024-09-19T08:45:08.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.662+0000 7f3c76706640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3c70110e20 con 0x7f3c70102570 2024-09-19T08:45:08.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.662+0000 7f3c76706640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3c70111120 con 0x7f3c70102ef0 2024-09-19T08:45:08.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.662+0000 7f3c6ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c70100e20 0x7f3c70198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:08.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.662+0000 7f3c6ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c70100e20 0x7f3c70198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51842/0 (socket says 172.21.15.164:51842) 2024-09-19T08:45:08.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.662+0000 7f3c6ffff640 1 -- 172.21.15.164:0/1213150697 learned_addr learned my addr 172.21.15.164:0/1213150697 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:08.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.662+0000 7f3c67fff640 1 --2- 172.21.15.164:0/1213150697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c70102570 0x7f3c70198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:08.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.662+0000 7f3c74ca3640 1 --2- 172.21.15.164:0/1213150697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c70102ef0 0x7f3c7019ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:08.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.662+0000 7f3c6ffff640 1 -- 172.21.15.164:0/1213150697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c70102ef0 msgr2=0x7f3c7019ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:08.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.662+0000 7f3c6ffff640 1 --2- 172.21.15.164:0/1213150697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c70102ef0 0x7f3c7019ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:08.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.662+0000 7f3c6ffff640 1 -- 172.21.15.164:0/1213150697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c70102570 msgr2=0x7f3c70198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:08.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.662+0000 7f3c6ffff640 1 --2- 172.21.15.164:0/1213150697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c70102570 0x7f3c70198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:08.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.662+0000 7f3c6ffff640 1 -- 172.21.15.164:0/1213150697 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c7019d5b0 con 0x7f3c70100e20 2024-09-19T08:45:08.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.662+0000 7f3c74ca3640 1 --2- 172.21.15.164:0/1213150697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c70102ef0 0x7f3c7019ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:08.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.662+0000 7f3c67fff640 1 --2- 172.21.15.164:0/1213150697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c70102570 0x7f3c70198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:08.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.662+0000 7f3c6ffff640 1 --2- 172.21.15.164:0/1213150697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c70100e20 0x7f3c70198600 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3c6000c890 tx=0x7f3c6000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:08.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.662+0000 7f3c6dffb640 1 -- 172.21.15.164:0/1213150697 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c60004220 con 0x7f3c70100e20 2024-09-19T08:45:08.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.662+0000 7f3c6dffb640 1 -- 172.21.15.164:0/1213150697 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c600043c0 con 0x7f3c70100e20 2024-09-19T08:45:08.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.662+0000 7f3c76706640 1 -- 172.21.15.164:0/1213150697 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c7019d8a0 con 0x7f3c70100e20 2024-09-19T08:45:08.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.662+0000 7f3c6dffb640 1 -- 172.21.15.164:0/1213150697 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c600049e0 con 0x7f3c70100e20 2024-09-19T08:45:08.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.662+0000 7f3c76706640 1 -- 172.21.15.164:0/1213150697 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c701a5180 con 0x7f3c70100e20 2024-09-19T08:45:08.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.663+0000 7f3c76706640 1 -- 172.21.15.164:0/1213150697 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c70101200 con 0x7f3c70100e20 2024-09-19T08:45:08.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.665+0000 7f3c6dffb640 1 -- 172.21.15.164:0/1213150697 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f3c600079b0 con 0x7f3c70100e20 2024-09-19T08:45:08.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.665+0000 7f3c6dffb640 1 --2- 172.21.15.164:0/1213150697 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f3c5c0a32d0 0x7f3c5c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:08.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.665+0000 7f3c67fff640 1 --2- 172.21.15.164:0/1213150697 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f3c5c0a32d0 0x7f3c5c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:08.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.665+0000 7f3c6dffb640 1 -- 172.21.15.164:0/1213150697 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f3c600cfb00 con 0x7f3c70100e20 2024-09-19T08:45:08.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.666+0000 7f3c67fff640 1 --2- 172.21.15.164:0/1213150697 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f3c5c0a32d0 0x7f3c5c0a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3c700fef00 tx=0x7f3c58005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:08.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.668+0000 7f3c6dffb640 1 -- 172.21.15.164:0/1213150697 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3c60014030 con 0x7f3c70100e20 2024-09-19T08:45:08.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.876+0000 7f3c76706640 1 -- 172.21.15.164:0/1213150697 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c70199960 con 0x7f3c70100e20 2024-09-19T08:45:08.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.877+0000 7f3c6dffb640 1 -- 172.21.15.164:0/1213150697 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11984 (secure 0 0 0) 0x7f3c6009c7c0 con 0x7f3c70100e20 2024-09-19T08:45:08.878 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:45:08.878 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":27,"btime":"2024-09-19T08:44:05:632372+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:44:05.632353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24442,24424,24449,14595]},"id":1}]} 2024-09-19T08:45:08.879 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 27 2024-09-19T08:45:08.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.879+0000 7f3c76706640 1 -- 172.21.15.164:0/1213150697 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f3c5c0a32d0 msgr2=0x7f3c5c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:08.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.879+0000 7f3c76706640 1 --2- 172.21.15.164:0/1213150697 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f3c5c0a32d0 0x7f3c5c0a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3c700fef00 tx=0x7f3c58005a20 comp rx=0 tx=0).stop 2024-09-19T08:45:08.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.879+0000 7f3c76706640 1 -- 172.21.15.164:0/1213150697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c70100e20 msgr2=0x7f3c70198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:08.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.879+0000 7f3c76706640 1 --2- 172.21.15.164:0/1213150697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c70100e20 0x7f3c70198600 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3c6000c890 tx=0x7f3c6000cd50 comp rx=0 tx=0).stop 2024-09-19T08:45:08.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.879+0000 7f3c76706640 1 -- 172.21.15.164:0/1213150697 shutdown_connections 2024-09-19T08:45:08.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.879+0000 7f3c76706640 1 --2- 172.21.15.164:0/1213150697 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f3c5c0a32d0 0x7f3c5c0a5790 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:08.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.879+0000 7f3c76706640 1 --2- 172.21.15.164:0/1213150697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c70102ef0 0x7f3c7019ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:08.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.879+0000 7f3c76706640 1 --2- 172.21.15.164:0/1213150697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c70102570 0x7f3c70198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:08.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.879+0000 7f3c76706640 1 --2- 172.21.15.164:0/1213150697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c70100e20 0x7f3c70198600 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:08.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.879+0000 7f3c76706640 1 -- 172.21.15.164:0/1213150697 >> 172.21.15.164:0/1213150697 conn(0x7f3c700fc7d0 msgr2=0x7f3c70069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:08.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.879+0000 7f3c76706640 1 -- 172.21.15.164:0/1213150697 shutdown_connections 2024-09-19T08:45:08.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:08.879+0000 7f3c76706640 1 -- 172.21.15.164:0/1213150697 wait complete. 2024-09-19T08:45:09.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:08 smithi170 ceph-mon[35072]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-19T08:45:09.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:08 smithi173 ceph-mon[35306]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-19T08:45:09.197 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:08 smithi164 ceph-mon[30928]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-19T08:45:09.545 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T08:43:53.386044+0000', 'modified': '2024-09-19T08:44:05.632353+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24479, 'mds_1': 24449, 'mds_2': 24442, 'mds_3': 24424, 'mds_4': 14595}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14583': {'gid': 14583, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6839/538276454', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6838', 'nonce': 538276454}, {'type': 'v1', 'addr': '172.21.15.164:6839', 'nonce': 538276454}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14595': {'gid': 14595, 'name': 'a', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.164:6841/2221659234', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6840', 'nonce': 2221659234}, {'type': 'v1', 'addr': '172.21.15.164:6841', 'nonce': 2221659234}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24401': {'gid': 24401, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6833/2726071671', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6832', 'nonce': 2726071671}, {'type': 'v1', 'addr': '172.21.15.170:6833', 'nonce': 2726071671}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24403': {'gid': 24403, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.173:6833/486946900', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6832', 'nonce': 486946900}, {'type': 'v1', 'addr': '172.21.15.173:6833', 'nonce': 486946900}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24418': {'gid': 24418, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.173:6835/947317705', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6834', 'nonce': 947317705}, {'type': 'v1', 'addr': '172.21.15.173:6835', 'nonce': 947317705}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6835/321254273', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6834', 'nonce': 321254273}, {'type': 'v1', 'addr': '172.21.15.170:6835', 'nonce': 321254273}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24424': {'gid': 24424, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.173:6837/2431090203', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6836', 'nonce': 2431090203}, {'type': 'v1', 'addr': '172.21.15.173:6837', 'nonce': 2431090203}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'i', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.173:6839/1089658493', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6838', 'nonce': 1089658493}, {'type': 'v1', 'addr': '172.21.15.173:6839', 'nonce': 1089658493}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'h', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.170:6837/2350302392', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6836', 'nonce': 2350302392}, {'type': 'v1', 'addr': '172.21.15.170:6837', 'nonce': 2350302392}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.170:6839/1854887138', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6838', 'nonce': 1854887138}, {'type': 'v1', 'addr': '172.21.15.170:6839', 'nonce': 1854887138}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24479, 'qdb_cluster': [24479, 24442, 24424, 24449, 14595]} 2024-09-19T08:45:09.545 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-19T08:45:09.546 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.a status 2024-09-19T08:45:09.791 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:45:10.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:09 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/1213150697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:45:10.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:09 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/1213150697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:45:10.215 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:09 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/1213150697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:45:10.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.296+0000 7f68fa17b640 1 -- 172.21.15.164:0/1984083058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68f4102d40 msgr2=0x7f68f4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:10.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.296+0000 7f68fa17b640 1 --2- 172.21.15.164:0/1984083058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68f4102d40 0x7f68f4103120 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f68e4009900 tx=0x7f68e402a480 comp rx=0 tx=0).stop 2024-09-19T08:45:10.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.296+0000 7f68fa17b640 1 -- 172.21.15.164:0/1984083058 shutdown_connections 2024-09-19T08:45:10.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.296+0000 7f68fa17b640 1 --2- 172.21.15.164:0/1984083058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68f410c170 0x7f68f410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.296+0000 7f68fa17b640 1 --2- 172.21.15.164:0/1984083058 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68f4103660 0x7f68f410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.296+0000 7f68fa17b640 1 --2- 172.21.15.164:0/1984083058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68f4102d40 0x7f68f4103120 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.296+0000 7f68fa17b640 1 -- 172.21.15.164:0/1984083058 >> 172.21.15.164:0/1984083058 conn(0x7f68f40fc7a0 msgr2=0x7f68f40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:10.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.296+0000 7f68fa17b640 1 -- 172.21.15.164:0/1984083058 shutdown_connections 2024-09-19T08:45:10.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.296+0000 7f68fa17b640 1 -- 172.21.15.164:0/1984083058 wait complete. 2024-09-19T08:45:10.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.297+0000 7f68fa17b640 1 Processor -- start 2024-09-19T08:45:10.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.297+0000 7f68fa17b640 1 -- start start 2024-09-19T08:45:10.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.297+0000 7f68fa17b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68f4102d40 0x7f68f4198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.297+0000 7f68fa17b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68f4103660 0x7f68f4198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.297+0000 7f68fa17b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68f410c170 0x7f68f419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.297+0000 7f68fa17b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f68f4111f70 con 0x7f68f4103660 2024-09-19T08:45:10.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.297+0000 7f68fa17b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f68f4111df0 con 0x7f68f410c170 2024-09-19T08:45:10.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.297+0000 7f68fa17b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f68f41120f0 con 0x7f68f4102d40 2024-09-19T08:45:10.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.297+0000 7f68f2ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68f4103660 0x7f68f4198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.297+0000 7f68f2ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68f4103660 0x7f68f4198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51850/0 (socket says 172.21.15.164:51850) 2024-09-19T08:45:10.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.297+0000 7f68f2ffd640 1 -- 172.21.15.164:0/1553322790 learned_addr learned my addr 172.21.15.164:0/1553322790 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:10.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.297+0000 7f68f3fff640 1 --2- 172.21.15.164:0/1553322790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68f410c170 0x7f68f419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.297+0000 7f68f37fe640 1 --2- 172.21.15.164:0/1553322790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68f4102d40 0x7f68f4198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.298+0000 7f68f2ffd640 1 -- 172.21.15.164:0/1553322790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68f4102d40 msgr2=0x7f68f4198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:10.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.298+0000 7f68f2ffd640 1 --2- 172.21.15.164:0/1553322790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68f4102d40 0x7f68f4198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.298+0000 7f68f2ffd640 1 -- 172.21.15.164:0/1553322790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68f410c170 msgr2=0x7f68f419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:10.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.298+0000 7f68f2ffd640 1 --2- 172.21.15.164:0/1553322790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68f410c170 0x7f68f419cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.298+0000 7f68f2ffd640 1 -- 172.21.15.164:0/1553322790 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68f419d300 con 0x7f68f4103660 2024-09-19T08:45:10.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.298+0000 7f68f3fff640 1 --2- 172.21.15.164:0/1553322790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68f410c170 0x7f68f419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:10.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.298+0000 7f68f2ffd640 1 --2- 172.21.15.164:0/1553322790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68f4103660 0x7f68f4198890 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f68dc00c970 tx=0x7f68dc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:10.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.298+0000 7f68f0ff9640 1 -- 172.21.15.164:0/1553322790 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68dc007bb0 con 0x7f68f4103660 2024-09-19T08:45:10.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.298+0000 7f68f0ff9640 1 -- 172.21.15.164:0/1553322790 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68dc007d50 con 0x7f68f4103660 2024-09-19T08:45:10.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.298+0000 7f68fa17b640 1 -- 172.21.15.164:0/1553322790 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68f419d5f0 con 0x7f68f4103660 2024-09-19T08:45:10.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.298+0000 7f68f0ff9640 1 -- 172.21.15.164:0/1553322790 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68dc018820 con 0x7f68f4103660 2024-09-19T08:45:10.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.298+0000 7f68fa17b640 1 -- 172.21.15.164:0/1553322790 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68f41a4ed0 con 0x7f68f4103660 2024-09-19T08:45:10.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.300+0000 7f68f0ff9640 1 -- 172.21.15.164:0/1553322790 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f68dc0189c0 con 0x7f68f4103660 2024-09-19T08:45:10.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.300+0000 7f68fc554740 1 Processor -- start 2024-09-19T08:45:10.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.300+0000 7f68fc554740 1 -- start start 2024-09-19T08:45:10.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.301+0000 7f68fc554740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555ef16249d0 0x555ef1624db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.301+0000 7f68fc554740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ef16252f0 0x555ef162f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.301+0000 7f68f0ff9640 1 --2- 172.21.15.164:0/1553322790 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f68e00a32d0 0x7f68e00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.301+0000 7f68fc554740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ef162fbb0 0x555ef1631fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.301+0000 7f68fc554740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555ef158b090 con 0x555ef16249d0 2024-09-19T08:45:10.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.301+0000 7f68fc554740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555ef1532a70 con 0x555ef16252f0 2024-09-19T08:45:10.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.301+0000 7f68fc554740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555ef1587050 con 0x555ef162fbb0 2024-09-19T08:45:10.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.301+0000 7f68f3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555ef16249d0 0x555ef1624db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.301+0000 7f68f3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555ef16249d0 0x555ef1624db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51864/0 (socket says 172.21.15.164:51864) 2024-09-19T08:45:10.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.301+0000 7f68f3fff640 1 -- 172.21.15.164:0/3607493774 learned_addr learned my addr 172.21.15.164:0/3607493774 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:10.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.301+0000 7f68f2ffd640 1 --2- 172.21.15.164:0/1553322790 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f68e00a32d0 0x7f68e00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.301+0000 7f68f0ff9640 1 -- 172.21.15.164:0/1553322790 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f68dc0cfb30 con 0x7f68f4103660 2024-09-19T08:45:10.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.301+0000 7f68f37fe640 1 --2- 172.21.15.164:0/3607493774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ef16252f0 0x555ef162f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.301+0000 7f68f3fff640 1 -- 172.21.15.164:0/3607493774 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ef162fbb0 msgr2=0x555ef1631fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:10.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.301+0000 7f68f3fff640 1 --2- 172.21.15.164:0/3607493774 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ef162fbb0 0x555ef1631fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.301+0000 7f68f3fff640 1 -- 172.21.15.164:0/3607493774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ef16252f0 msgr2=0x555ef162f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:10.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.301+0000 7f68f3fff640 1 --2- 172.21.15.164:0/3607493774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ef16252f0 0x555ef162f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.301+0000 7f68f3fff640 1 -- 172.21.15.164:0/3607493774 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ef153f130 con 0x555ef16249d0 2024-09-19T08:45:10.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.301+0000 7f68f37fe640 1 --2- 172.21.15.164:0/3607493774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ef16252f0 0x555ef162f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:10.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.301+0000 7f68f3fff640 1 --2- 172.21.15.164:0/3607493774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555ef16249d0 0x555ef1624db0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f68ec005cc0 tx=0x7f68ec00e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910f7e8cdc3f1c19 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:10.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.301+0000 7f68f2ffd640 1 --2- 172.21.15.164:0/1553322790 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f68e00a32d0 0x7f68e00a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f68dc098550 tx=0x7f68dc06dec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:10.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.301+0000 7f68e9ffb640 1 -- 172.21.15.164:0/3607493774 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ec014070 con 0x555ef16249d0 2024-09-19T08:45:10.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.302+0000 7f68e9ffb640 1 -- 172.21.15.164:0/3607493774 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68ec003e50 con 0x555ef16249d0 2024-09-19T08:45:10.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.302+0000 7f68e9ffb640 1 -- 172.21.15.164:0/3607493774 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ec004030 con 0x555ef16249d0 2024-09-19T08:45:10.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.302+0000 7f68fc554740 1 -- 172.21.15.164:0/3607493774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555ef16249d0 msgr2=0x555ef1624db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:10.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.302+0000 7f68fc554740 1 --2- 172.21.15.164:0/3607493774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555ef16249d0 0x555ef1624db0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f68ec005cc0 tx=0x7f68ec00e7e0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.302+0000 7f68fc554740 1 -- 172.21.15.164:0/3607493774 shutdown_connections 2024-09-19T08:45:10.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.302+0000 7f68fc554740 1 --2- 172.21.15.164:0/3607493774 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ef162fbb0 0x555ef1631fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.302+0000 7f68fc554740 1 --2- 172.21.15.164:0/3607493774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ef16252f0 0x555ef162f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.302+0000 7f68fc554740 1 --2- 172.21.15.164:0/3607493774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555ef16249d0 0x555ef1624db0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.302+0000 7f68fc554740 1 -- 172.21.15.164:0/3607493774 >> 172.21.15.164:0/3607493774 conn(0x555ef1620f60 msgr2=0x555ef1621340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:10.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.302+0000 7f68fc554740 1 -- 172.21.15.164:0/3607493774 shutdown_connections 2024-09-19T08:45:10.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.302+0000 7f68fc554740 1 -- 172.21.15.164:0/3607493774 wait complete. 2024-09-19T08:45:10.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.303+0000 7f68fc554740 1 -- start start 2024-09-19T08:45:10.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.303+0000 7f68fc554740 1 Processor -- start 2024-09-19T08:45:10.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.303+0000 7f68fc554740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555ef16249d0 0x555ef162f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.303+0000 7f68fc554740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ef16252f0 0x555ef162c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.303+0000 7f68fc554740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ef162fbb0 0x555ef162ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.303+0000 7f68fc554740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555ef1632780 con 0x555ef16249d0 2024-09-19T08:45:10.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.303+0000 7f68fc554740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555ef1545f20 con 0x555ef162fbb0 2024-09-19T08:45:10.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.303+0000 7f68fc554740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555ef1632900 con 0x555ef16252f0 2024-09-19T08:45:10.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.303+0000 7f68fc554740 10 client.0 _ll_register_callbacks cb 0x555ef150f030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:45:10.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.303+0000 7f68f3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555ef16249d0 0x555ef162f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.303+0000 7f68f3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555ef16249d0 0x555ef162f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51868/0 (socket says 172.21.15.164:51868) 2024-09-19T08:45:10.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.303+0000 7f68f3fff640 1 -- 172.21.15.164:0/1951280388 learned_addr learned my addr 172.21.15.164:0/1951280388 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:10.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.303+0000 7f68f37fe640 1 --2- 172.21.15.164:0/1951280388 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ef16252f0 0x555ef162c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.303+0000 7f68f3fff640 1 --2- 172.21.15.164:0/1951280388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ef162fbb0 0x555ef162ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.304+0000 7f68f3fff640 1 -- 172.21.15.164:0/1951280388 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ef16252f0 msgr2=0x555ef162c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:10.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.304+0000 7f68f3fff640 1 --2- 172.21.15.164:0/1951280388 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ef16252f0 0x555ef162c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.304+0000 7f68f3fff640 1 -- 172.21.15.164:0/1951280388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ef162fbb0 msgr2=0x555ef162ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:10.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.304+0000 7f68f3fff640 1 --2- 172.21.15.164:0/1951280388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ef162fbb0 0x555ef162ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.304+0000 7f68f3fff640 1 -- 172.21.15.164:0/1951280388 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ef12fc2b0 con 0x555ef16249d0 2024-09-19T08:45:10.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.304+0000 7f68f3fff640 1 --2- 172.21.15.164:0/1951280388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555ef16249d0 0x555ef162f430 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f68ec010b50 tx=0x7f68ec0075c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d8f21ef49bfcae4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:10.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.304+0000 7f68d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:45:10.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.304+0000 7f68d2ffd640 1 -- 172.21.15.164:0/1951280388 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ec014040 con 0x555ef16249d0 2024-09-19T08:45:10.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.304+0000 7f68d2ffd640 1 -- 172.21.15.164:0/1951280388 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f68ec004d00 con 0x555ef16249d0 2024-09-19T08:45:10.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.304+0000 7f68fc554740 1 -- 172.21.15.164:0/1951280388 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68ec003e50 con 0x555ef16249d0 2024-09-19T08:45:10.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.304+0000 7f68d2ffd640 1 -- 172.21.15.164:0/1951280388 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f68ec030050 con 0x555ef16249d0 2024-09-19T08:45:10.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.304+0000 7f68fc554740 10 client.14949 fetch_fsmap learned FSMap version 27 2024-09-19T08:45:10.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.304+0000 7f68fc554740 1 -- 172.21.15.164:0/1951280388 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555ef163e000 con 0x555ef16249d0 2024-09-19T08:45:10.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.304+0000 7f68d2ffd640 1 -- 172.21.15.164:0/1951280388 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68ec003e50 con 0x555ef16249d0 2024-09-19T08:45:10.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.304+0000 7f68d2ffd640 1 -- 172.21.15.164:0/1951280388 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f68ec005340 con 0x555ef16249d0 2024-09-19T08:45:10.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.304+0000 7f68fc554740 10 client.14949 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T08:45:10.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.304+0000 7f68fc554740 10 client.14949 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T08:45:10.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.305+0000 7f68fc554740 20 client.14949 populate_metadata read hostname 'smithi164' 2024-09-19T08:45:10.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.305+0000 7f68fc554740 1 --2- 172.21.15.164:0/1951280388 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x555ef16401c0 0x555ef1641630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.305+0000 7f68fc554740 4 client.14949 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:45:10.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.305+0000 7f68fc554740 1 -- 172.21.15.164:0/1951280388 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ef1598600 con 0x555ef16401c0 2024-09-19T08:45:10.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.305+0000 7f68f37fe640 1 --2- 172.21.15.164:0/1951280388 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x555ef16401c0 0x555ef1641630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.305+0000 7f68d2ffd640 10 client.14949 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:45:10.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.305+0000 7f68f37fe640 1 --2- 172.21.15.164:0/1951280388 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x555ef16401c0 0x555ef1641630 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c6e798ea0235d18 server_cookie=b8b73063a2b9f392 in_seq=0 out_seq=0 2024-09-19T08:45:10.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.306+0000 7f68d2ffd640 1 -- 172.21.15.164:0/1951280388 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555ef1598600 con 0x555ef16401c0 2024-09-19T08:45:10.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.306+0000 7f68d2ffd640 10 client.14949 handle_command_reply: tid=0 2024-09-19T08:45:10.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.306+0000 7f68fc554740 1 client.14949 shutdown 2024-09-19T08:45:10.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.306+0000 7f68fc554740 1 -- 172.21.15.164:0/1951280388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555ef16249d0 msgr2=0x555ef162f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:10.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.306+0000 7f68fc554740 1 --2- 172.21.15.164:0/1951280388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555ef16249d0 0x555ef162f430 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f68ec010b50 tx=0x7f68ec0075c0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.306+0000 7f68fc554740 1 -- 172.21.15.164:0/1951280388 shutdown_connections 2024-09-19T08:45:10.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.306+0000 7f68fc554740 1 --2- 172.21.15.164:0/1951280388 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x555ef16401c0 0x555ef1641630 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.306+0000 7f68fc554740 1 --2- 172.21.15.164:0/1951280388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ef162fbb0 0x555ef162ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.306+0000 7f68fc554740 1 --2- 172.21.15.164:0/1951280388 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ef16252f0 0x555ef162c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.306+0000 7f68fc554740 1 --2- 172.21.15.164:0/1951280388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555ef16249d0 0x555ef162f430 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.306+0000 7f68fc554740 1 -- 172.21.15.164:0/1951280388 >> 172.21.15.164:0/1951280388 conn(0x555ef1620f60 msgr2=0x555ef1626c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:10.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.306+0000 7f68d2ffd640 0 client.14949 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:45:10.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.306+0000 7f68fc554740 1 -- 172.21.15.164:0/1951280388 shutdown_connections 2024-09-19T08:45:10.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.306+0000 7f68fc554740 1 -- 172.21.15.164:0/1951280388 wait complete. 2024-09-19T08:45:10.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.306+0000 7f68fc554740 20 client.14949 trim_cache size 0 max 16384 2024-09-19T08:45:10.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.324+0000 7f68fc554740 1 Processor -- start 2024-09-19T08:45:10.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.324+0000 7f68fc554740 1 -- start start 2024-09-19T08:45:10.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.324+0000 7f68fc554740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ef16249d0 0x555ef163c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.324+0000 7f68fc554740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555ef16252f0 0x555ef162b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.324+0000 7f68fc554740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ef162fbb0 0x555ef162b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.324+0000 7f68fc554740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555ef163f970 con 0x555ef16252f0 2024-09-19T08:45:10.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.324+0000 7f68fc554740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555ef1640040 con 0x555ef16249d0 2024-09-19T08:45:10.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.324+0000 7f68fc554740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555ef1640710 con 0x555ef162fbb0 2024-09-19T08:45:10.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.324+0000 7f68f37fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555ef16252f0 0x555ef162b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.324+0000 7f68f37fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555ef16252f0 0x555ef162b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51882/0 (socket says 172.21.15.164:51882) 2024-09-19T08:45:10.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.324+0000 7f68f37fe640 1 -- 172.21.15.164:0/2149144695 learned_addr learned my addr 172.21.15.164:0/2149144695 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:10.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.324+0000 7f68f3fff640 1 --2- 172.21.15.164:0/2149144695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ef16249d0 0x555ef163c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.324+0000 7f68f37fe640 1 -- 172.21.15.164:0/2149144695 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ef162fbb0 msgr2=0x555ef162b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:10.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.324+0000 7f68f37fe640 1 --2- 172.21.15.164:0/2149144695 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ef162fbb0 0x555ef162b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.324+0000 7f68f37fe640 1 -- 172.21.15.164:0/2149144695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ef16249d0 msgr2=0x555ef163c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:10.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.324+0000 7f68f37fe640 1 --2- 172.21.15.164:0/2149144695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ef16249d0 0x555ef163c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.324+0000 7f68f37fe640 1 -- 172.21.15.164:0/2149144695 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ef16209e0 con 0x555ef16252f0 2024-09-19T08:45:10.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.324+0000 7f68f37fe640 1 --2- 172.21.15.164:0/2149144695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555ef16252f0 0x555ef162b050 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f68e4005ab0 tx=0x7f68e40058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e592ba5672ea29 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:10.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.325+0000 7f68e97fa640 1 -- 172.21.15.164:0/2149144695 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e4048070 con 0x555ef16252f0 2024-09-19T08:45:10.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.325+0000 7f68e97fa640 1 -- 172.21.15.164:0/2149144695 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68e4005500 con 0x555ef16252f0 2024-09-19T08:45:10.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.325+0000 7f68e97fa640 1 -- 172.21.15.164:0/2149144695 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e4035040 con 0x555ef16252f0 2024-09-19T08:45:10.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.325+0000 7f68fc554740 1 -- 172.21.15.164:0/2149144695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555ef16252f0 msgr2=0x555ef162b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:10.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.325+0000 7f68fc554740 1 --2- 172.21.15.164:0/2149144695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555ef16252f0 0x555ef162b050 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f68e4005ab0 tx=0x7f68e40058b0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.325+0000 7f68fc554740 1 -- 172.21.15.164:0/2149144695 shutdown_connections 2024-09-19T08:45:10.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.325+0000 7f68fc554740 1 --2- 172.21.15.164:0/2149144695 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ef162fbb0 0x555ef162b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.325+0000 7f68fc554740 1 --2- 172.21.15.164:0/2149144695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555ef16252f0 0x555ef162b050 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.325+0000 7f68fc554740 1 --2- 172.21.15.164:0/2149144695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ef16249d0 0x555ef163c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.325+0000 7f68fc554740 1 -- 172.21.15.164:0/2149144695 >> 172.21.15.164:0/2149144695 conn(0x555ef1620f60 msgr2=0x555ef162c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:10.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.325+0000 7f68fc554740 1 -- 172.21.15.164:0/2149144695 shutdown_connections 2024-09-19T08:45:10.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.325+0000 7f68fc554740 1 -- 172.21.15.164:0/2149144695 wait complete. 2024-09-19T08:45:10.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.326+0000 7f68fc554740 1 -- start start 2024-09-19T08:45:10.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.326+0000 7f68fc554740 1 Processor -- start 2024-09-19T08:45:10.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.326+0000 7f68fc554740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ef16249d0 0x555ef16446f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.326+0000 7f68fc554740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555ef16252f0 0x555ef1644c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.326+0000 7f68fc554740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ef162fbb0 0x555ef1645170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.326+0000 7f68fc554740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555ef16414b0 con 0x555ef16252f0 2024-09-19T08:45:10.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.326+0000 7f68fc554740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555ef1640de0 con 0x555ef162fbb0 2024-09-19T08:45:10.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.326+0000 7f68fc554740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555ef1641b80 con 0x555ef16249d0 2024-09-19T08:45:10.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.326+0000 7f68fc554740 10 client.0 _ll_register_callbacks cb 0x7f68ec007e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:45:10.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.326+0000 7f68f37fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555ef16252f0 0x555ef1644c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.326+0000 7f68f37fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555ef16252f0 0x555ef1644c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51884/0 (socket says 172.21.15.164:51884) 2024-09-19T08:45:10.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.326+0000 7f68f37fe640 1 -- 172.21.15.164:0/3968942917 learned_addr learned my addr 172.21.15.164:0/3968942917 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:10.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.326+0000 7f68f3fff640 1 --2- 172.21.15.164:0/3968942917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ef162fbb0 0x555ef1645170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.326+0000 7f68f3fff640 1 --2- 172.21.15.164:0/3968942917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ef16249d0 0x555ef16446f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.327+0000 7f68f37fe640 1 -- 172.21.15.164:0/3968942917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ef16249d0 msgr2=0x555ef16446f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:10.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.327+0000 7f68f37fe640 1 --2- 172.21.15.164:0/3968942917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ef16249d0 0x555ef16446f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.327+0000 7f68f37fe640 1 -- 172.21.15.164:0/3968942917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ef162fbb0 msgr2=0x555ef1645170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:10.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.327+0000 7f68f37fe640 1 --2- 172.21.15.164:0/3968942917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ef162fbb0 0x555ef1645170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.327+0000 7f68f37fe640 1 -- 172.21.15.164:0/3968942917 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ef16a89c0 con 0x555ef16252f0 2024-09-19T08:45:10.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.327+0000 7f68f3fff640 1 --2- 172.21.15.164:0/3968942917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ef16249d0 0x555ef16446f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:10.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.327+0000 7f68f37fe640 1 --2- 172.21.15.164:0/3968942917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555ef16252f0 0x555ef1644c30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f68e4000c00 tx=0x7f68e40056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae4e80848e87dbb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:10.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.327+0000 7f68d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:45:10.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.327+0000 7f68d2ffd640 1 -- 172.21.15.164:0/3968942917 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e4035050 con 0x555ef16252f0 2024-09-19T08:45:10.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.327+0000 7f68d2ffd640 1 -- 172.21.15.164:0/3968942917 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f68e4004a50 con 0x555ef16252f0 2024-09-19T08:45:10.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.327+0000 7f68fc554740 1 -- 172.21.15.164:0/3968942917 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68e4005500 con 0x555ef16252f0 2024-09-19T08:45:10.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.327+0000 7f68d2ffd640 1 -- 172.21.15.164:0/3968942917 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f68e4048040 con 0x555ef16252f0 2024-09-19T08:45:10.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.327+0000 7f68fc554740 10 client.14955 fetch_fsmap learned FSMap version 27 2024-09-19T08:45:10.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.327+0000 7f68fc554740 1 -- 172.21.15.164:0/3968942917 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555ef16a9cc0 con 0x555ef16252f0 2024-09-19T08:45:10.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.327+0000 7f68d2ffd640 1 -- 172.21.15.164:0/3968942917 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68e4005500 con 0x555ef16252f0 2024-09-19T08:45:10.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.327+0000 7f68d2ffd640 1 -- 172.21.15.164:0/3968942917 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f68e404c400 con 0x555ef16252f0 2024-09-19T08:45:10.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.327+0000 7f68fc554740 10 client.14955 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T08:45:10.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.327+0000 7f68fc554740 10 client.14955 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T08:45:10.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.327+0000 7f68fc554740 20 client.14955 populate_metadata read hostname 'smithi164' 2024-09-19T08:45:10.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.327+0000 7f68fc554740 1 --2- 172.21.15.164:0/3968942917 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x555ef16ae400 0x555ef16ad110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.327+0000 7f68fc554740 4 client.14955 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:45:10.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.327+0000 7f68fc554740 1 -- 172.21.15.164:0/3968942917 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "status"}) -- 0x555ef1592b10 con 0x555ef16ae400 2024-09-19T08:45:10.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.328+0000 7f68f3fff640 1 --2- 172.21.15.164:0/3968942917 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x555ef16ae400 0x555ef16ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.328+0000 7f68d2ffd640 10 client.14955 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:45:10.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.328+0000 7f68f3fff640 1 --2- 172.21.15.164:0/3968942917 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x555ef16ae400 0x555ef16ad110 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d4e57bd178b569ed server_cookie=6f1fd79d6a8ba65 in_seq=0 out_seq=0 2024-09-19T08:45:10.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.328+0000 7f68d2ffd640 1 -- 172.21.15.164:0/3968942917 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x555ef1592b10 con 0x555ef16ae400 2024-09-19T08:45:10.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.328+0000 7f68d2ffd640 10 client.14955 handle_command_reply: tid=0 2024-09-19T08:45:10.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.328+0000 7f68fc554740 1 client.14955 shutdown 2024-09-19T08:45:10.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.328+0000 7f68fc554740 1 -- 172.21.15.164:0/3968942917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555ef16252f0 msgr2=0x555ef1644c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:10.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.328+0000 7f68fc554740 1 --2- 172.21.15.164:0/3968942917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555ef16252f0 0x555ef1644c30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f68e4000c00 tx=0x7f68e40056d0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.328+0000 7f68fc554740 1 -- 172.21.15.164:0/3968942917 shutdown_connections 2024-09-19T08:45:10.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.328+0000 7f68fc554740 1 --2- 172.21.15.164:0/3968942917 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x555ef16ae400 0x555ef16ad110 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.328+0000 7f68fc554740 1 --2- 172.21.15.164:0/3968942917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ef162fbb0 0x555ef1645170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.328+0000 7f68fc554740 1 --2- 172.21.15.164:0/3968942917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555ef16252f0 0x555ef1644c30 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.328+0000 7f68fc554740 1 --2- 172.21.15.164:0/3968942917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ef16249d0 0x555ef16446f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.328+0000 7f68fc554740 1 -- 172.21.15.164:0/3968942917 >> 172.21.15.164:0/3968942917 conn(0x555ef1620f60 msgr2=0x555ef162b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:10.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.328+0000 7f68d2ffd640 0 client.14955 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:45:10.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.329+0000 7f68fc554740 1 -- 172.21.15.164:0/3968942917 shutdown_connections 2024-09-19T08:45:10.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.329+0000 7f68fc554740 1 -- 172.21.15.164:0/3968942917 wait complete. 2024-09-19T08:45:10.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.329+0000 7f68fc554740 20 client.14955 trim_cache size 0 max 16384 2024-09-19T08:45:10.345 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:45:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: "cluster_fsid": "1e4b0d74-7662-11ef-bceb-c7b262605968", 2024-09-19T08:45:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: "whoami": 4, 2024-09-19T08:45:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 14595, 2024-09-19T08:45:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: "want_state": "up:active", 2024-09-19T08:45:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: "state": "up:active", 2024-09-19T08:45:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs", 2024-09-19T08:45:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: "rank_uptime": 65.699880299, 2024-09-19T08:45:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: "mdsmap_epoch": 27, 2024-09-19T08:45:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch": 82, 2024-09-19T08:45:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch_barrier": 82, 2024-09-19T08:45:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: "uptime": 90.960696951000003 2024-09-19T08:45:10.346 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:45:10.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.329+0000 7f68fa17b640 1 -- 172.21.15.164:0/1553322790 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f68e00a32d0 msgr2=0x7f68e00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:10.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.329+0000 7f68fa17b640 1 --2- 172.21.15.164:0/1553322790 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f68e00a32d0 0x7f68e00a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f68dc098550 tx=0x7f68dc06dec0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.329+0000 7f68fa17b640 1 -- 172.21.15.164:0/1553322790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68f4103660 msgr2=0x7f68f4198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:10.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.329+0000 7f68fa17b640 1 --2- 172.21.15.164:0/1553322790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68f4103660 0x7f68f4198890 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f68dc00c970 tx=0x7f68dc00ce30 comp rx=0 tx=0).stop 2024-09-19T08:45:10.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.329+0000 7f68fa17b640 1 -- 172.21.15.164:0/1553322790 shutdown_connections 2024-09-19T08:45:10.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.329+0000 7f68fa17b640 1 --2- 172.21.15.164:0/1553322790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68f410c170 0x7f68f419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.329+0000 7f68fa17b640 1 --2- 172.21.15.164:0/1553322790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68f4103660 0x7f68f4198890 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.329+0000 7f68fa17b640 1 --2- 172.21.15.164:0/1553322790 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f68e00a32d0 0x7f68e00a5790 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.329+0000 7f68fa17b640 1 --2- 172.21.15.164:0/1553322790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68f4102d40 0x7f68f4198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.329+0000 7f68fa17b640 1 -- 172.21.15.164:0/1553322790 >> 172.21.15.164:0/1553322790 conn(0x7f68f40fc7a0 msgr2=0x7f68f40fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:10.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.329+0000 7f68fa17b640 1 -- 172.21.15.164:0/1553322790 shutdown_connections 2024-09-19T08:45:10.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:10.329+0000 7f68fa17b640 1 -- 172.21.15.164:0/1553322790 wait complete. 2024-09-19T08:45:11.014 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T08:45:11.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:10 smithi170 ceph-mon[35072]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:45:11.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:10 smithi173 ceph-mon[35306]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:45:11.261 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:45:11.281 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:10 smithi164 ceph-mon[30928]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:45:11.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.707+0000 7ff55174b640 1 -- 172.21.15.164:0/997920054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff54c1082a0 msgr2=0x7ff54c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:11.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.707+0000 7ff55174b640 1 --2- 172.21.15.164:0/997920054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff54c1082a0 0x7ff54c10ec40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff544009930 tx=0x7ff54402a440 comp rx=0 tx=0).stop 2024-09-19T08:45:11.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.707+0000 7ff55174b640 1 -- 172.21.15.164:0/997920054 shutdown_connections 2024-09-19T08:45:11.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.707+0000 7ff55174b640 1 --2- 172.21.15.164:0/997920054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff54c1082a0 0x7ff54c10ec40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.707+0000 7ff55174b640 1 --2- 172.21.15.164:0/997920054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff54c1036e0 0x7ff54c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.707+0000 7ff55174b640 1 --2- 172.21.15.164:0/997920054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff54c102d30 0x7ff54c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.707+0000 7ff55174b640 1 -- 172.21.15.164:0/997920054 >> 172.21.15.164:0/997920054 conn(0x7ff54c0fc7d0 msgr2=0x7ff54c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:11.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.707+0000 7ff55174b640 1 -- 172.21.15.164:0/997920054 shutdown_connections 2024-09-19T08:45:11.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.707+0000 7ff55174b640 1 -- 172.21.15.164:0/997920054 wait complete. 2024-09-19T08:45:11.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.707+0000 7ff55174b640 1 Processor -- start 2024-09-19T08:45:11.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.708+0000 7ff55174b640 1 -- start start 2024-09-19T08:45:11.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.708+0000 7ff55174b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff54c102d30 0x7ff54c198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:11.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.708+0000 7ff55174b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff54c1036e0 0x7ff54c198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:11.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.708+0000 7ff55174b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff54c1082a0 0x7ff54c19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:11.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.708+0000 7ff55174b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff54c1120e0 con 0x7ff54c1036e0 2024-09-19T08:45:11.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.708+0000 7ff55174b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff54c111f60 con 0x7ff54c102d30 2024-09-19T08:45:11.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.708+0000 7ff55174b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff54c112260 con 0x7ff54c1082a0 2024-09-19T08:45:11.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.708+0000 7ff54a7fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff54c1036e0 0x7ff54c198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:11.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.708+0000 7ff54b7fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff54c1082a0 0x7ff54c19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:11.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.708+0000 7ff54a7fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff54c1036e0 0x7ff54c198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51900/0 (socket says 172.21.15.164:51900) 2024-09-19T08:45:11.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.708+0000 7ff54a7fc640 1 -- 172.21.15.164:0/3470111129 learned_addr learned my addr 172.21.15.164:0/3470111129 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:11.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.708+0000 7ff54affd640 1 --2- 172.21.15.164:0/3470111129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff54c102d30 0x7ff54c198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:11.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.708+0000 7ff54a7fc640 1 -- 172.21.15.164:0/3470111129 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff54c1082a0 msgr2=0x7ff54c19ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:11.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.708+0000 7ff54a7fc640 1 --2- 172.21.15.164:0/3470111129 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff54c1082a0 0x7ff54c19ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.708+0000 7ff54a7fc640 1 -- 172.21.15.164:0/3470111129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff54c102d30 msgr2=0x7ff54c198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:11.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.708+0000 7ff54a7fc640 1 --2- 172.21.15.164:0/3470111129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff54c102d30 0x7ff54c198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.708+0000 7ff54a7fc640 1 -- 172.21.15.164:0/3470111129 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff54c19d4e0 con 0x7ff54c1036e0 2024-09-19T08:45:11.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.708+0000 7ff54a7fc640 1 --2- 172.21.15.164:0/3470111129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff54c1036e0 0x7ff54c198a70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff540006fd0 tx=0x7ff54000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:11.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.709+0000 7ff533fff640 1 -- 172.21.15.164:0/3470111129 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff540011830 con 0x7ff54c1036e0 2024-09-19T08:45:11.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.709+0000 7ff533fff640 1 -- 172.21.15.164:0/3470111129 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff540016070 con 0x7ff54c1036e0 2024-09-19T08:45:11.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.709+0000 7ff55174b640 1 -- 172.21.15.164:0/3470111129 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff54c19d7d0 con 0x7ff54c1036e0 2024-09-19T08:45:11.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.709+0000 7ff533fff640 1 -- 172.21.15.164:0/3470111129 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54000f2e0 con 0x7ff54c1036e0 2024-09-19T08:45:11.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.709+0000 7ff55174b640 1 -- 172.21.15.164:0/3470111129 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff54c1a50b0 con 0x7ff54c1036e0 2024-09-19T08:45:11.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.711+0000 7ff533fff640 1 -- 172.21.15.164:0/3470111129 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7ff54001b460 con 0x7ff54c1036e0 2024-09-19T08:45:11.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.711+0000 7ff553b24740 1 Processor -- start 2024-09-19T08:45:11.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.711+0000 7ff553b24740 1 -- start start 2024-09-19T08:45:11.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.711+0000 7ff553b24740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edd5b029d0 0x55edd5b02db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:11.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.711+0000 7ff553b24740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edd5b032f0 0x55edd5b0d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:11.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.711+0000 7ff553b24740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55edd5b0dbb0 0x55edd5b0ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:11.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.711+0000 7ff553b24740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55edd5a69090 con 0x55edd5b032f0 2024-09-19T08:45:11.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.711+0000 7ff553b24740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55edd5a10a70 con 0x55edd5b029d0 2024-09-19T08:45:11.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.711+0000 7ff553b24740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55edd5a65050 con 0x55edd5b0dbb0 2024-09-19T08:45:11.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.712+0000 7ff54affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edd5b032f0 0x55edd5b0d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:11.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.712+0000 7ff533fff640 1 --2- 172.21.15.164:0/3470111129 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7ff5340a32d0 0x7ff5340a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:11.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.712+0000 7ff54b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edd5b029d0 0x55edd5b02db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:11.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.712+0000 7ff54affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edd5b032f0 0x55edd5b0d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51916/0 (socket says 172.21.15.164:51916) 2024-09-19T08:45:11.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.712+0000 7ff54affd640 1 -- 172.21.15.164:0/1293490326 learned_addr learned my addr 172.21.15.164:0/1293490326 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:11.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.712+0000 7ff54b7fe640 1 --2- 172.21.15.164:0/3470111129 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7ff5340a32d0 0x7ff5340a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:11.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.712+0000 7ff533fff640 1 -- 172.21.15.164:0/3470111129 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff5400cef30 con 0x7ff54c1036e0 2024-09-19T08:45:11.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.712+0000 7ff54a7fc640 1 --2- 172.21.15.164:0/1293490326 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55edd5b0dbb0 0x55edd5b0ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:11.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.712+0000 7ff54affd640 1 -- 172.21.15.164:0/1293490326 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55edd5b0dbb0 msgr2=0x55edd5b0ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:11.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.712+0000 7ff54affd640 1 --2- 172.21.15.164:0/1293490326 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55edd5b0dbb0 0x55edd5b0ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.712+0000 7ff54affd640 1 -- 172.21.15.164:0/1293490326 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edd5b029d0 msgr2=0x55edd5b02db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:11.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.712+0000 7ff54affd640 1 --2- 172.21.15.164:0/1293490326 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edd5b029d0 0x55edd5b02db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.712+0000 7ff54affd640 1 -- 172.21.15.164:0/1293490326 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edd5a1d130 con 0x55edd5b032f0 2024-09-19T08:45:11.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.712+0000 7ff54a7fc640 1 --2- 172.21.15.164:0/1293490326 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55edd5b0dbb0 0x55edd5b0ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:11.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.712+0000 7ff54b7fe640 1 --2- 172.21.15.164:0/1293490326 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edd5b029d0 0x55edd5b02db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:45:11.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.712+0000 7ff54affd640 1 --2- 172.21.15.164:0/1293490326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edd5b032f0 0x55edd5b0d670 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff53c007ed0 tx=0x7ff53c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb8af0a24f66da1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:11.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.712+0000 7ff54b7fe640 1 --2- 172.21.15.164:0/3470111129 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7ff5340a32d0 0x7ff5340a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff544031550 tx=0x7ff54402a440 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:11.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.712+0000 7ff5317fa640 1 -- 172.21.15.164:0/1293490326 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c014070 con 0x55edd5b032f0 2024-09-19T08:45:11.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.712+0000 7ff5317fa640 1 -- 172.21.15.164:0/1293490326 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff53c004d20 con 0x55edd5b032f0 2024-09-19T08:45:11.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.712+0000 7ff5317fa640 1 -- 172.21.15.164:0/1293490326 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c0054a0 con 0x55edd5b032f0 2024-09-19T08:45:11.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.713+0000 7ff553b24740 1 -- 172.21.15.164:0/1293490326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edd5b032f0 msgr2=0x55edd5b0d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:11.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.713+0000 7ff553b24740 1 --2- 172.21.15.164:0/1293490326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edd5b032f0 0x55edd5b0d670 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff53c007ed0 tx=0x7ff53c00e720 comp rx=0 tx=0).stop 2024-09-19T08:45:11.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.713+0000 7ff553b24740 1 -- 172.21.15.164:0/1293490326 shutdown_connections 2024-09-19T08:45:11.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.713+0000 7ff553b24740 1 --2- 172.21.15.164:0/1293490326 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55edd5b0dbb0 0x55edd5b0ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.713+0000 7ff553b24740 1 --2- 172.21.15.164:0/1293490326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edd5b032f0 0x55edd5b0d670 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.713+0000 7ff553b24740 1 --2- 172.21.15.164:0/1293490326 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edd5b029d0 0x55edd5b02db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.713+0000 7ff553b24740 1 -- 172.21.15.164:0/1293490326 >> 172.21.15.164:0/1293490326 conn(0x55edd5afef60 msgr2=0x55edd5aff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:11.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.713+0000 7ff553b24740 1 -- 172.21.15.164:0/1293490326 shutdown_connections 2024-09-19T08:45:11.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.713+0000 7ff553b24740 1 -- 172.21.15.164:0/1293490326 wait complete. 2024-09-19T08:45:11.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.713+0000 7ff553b24740 1 -- start start 2024-09-19T08:45:11.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.714+0000 7ff553b24740 1 Processor -- start 2024-09-19T08:45:11.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.714+0000 7ff553b24740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edd5b029d0 0x55edd5b0d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:11.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.714+0000 7ff553b24740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edd5b032f0 0x55edd5b0a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:11.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.714+0000 7ff553b24740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55edd5b0dbb0 0x55edd5b0ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:11.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.714+0000 7ff553b24740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55edd5b10780 con 0x55edd5b029d0 2024-09-19T08:45:11.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.714+0000 7ff553b24740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55edd5a23f20 con 0x55edd5b032f0 2024-09-19T08:45:11.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.714+0000 7ff553b24740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55edd5b10900 con 0x55edd5b0dbb0 2024-09-19T08:45:11.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.714+0000 7ff553b24740 10 client.0 _ll_register_callbacks cb 0x55edd59ed030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:45:11.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.714+0000 7ff54affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edd5b029d0 0x55edd5b0d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:11.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.714+0000 7ff54affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edd5b029d0 0x55edd5b0d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51922/0 (socket says 172.21.15.164:51922) 2024-09-19T08:45:11.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.714+0000 7ff54affd640 1 -- 172.21.15.164:0/3018895454 learned_addr learned my addr 172.21.15.164:0/3018895454 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:11.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.714+0000 7ff54b7fe640 1 --2- 172.21.15.164:0/3018895454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55edd5b0dbb0 0x55edd5b0ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:11.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.714+0000 7ff54affd640 1 -- 172.21.15.164:0/3018895454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55edd5b0dbb0 msgr2=0x55edd5b0ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:11.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.714+0000 7ff54a7fc640 1 --2- 172.21.15.164:0/3018895454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edd5b032f0 0x55edd5b0a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:11.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.714+0000 7ff54affd640 1 --2- 172.21.15.164:0/3018895454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55edd5b0dbb0 0x55edd5b0ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.714+0000 7ff54affd640 1 -- 172.21.15.164:0/3018895454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edd5b032f0 msgr2=0x55edd5b0a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:11.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.714+0000 7ff54affd640 1 --2- 172.21.15.164:0/3018895454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edd5b032f0 0x55edd5b0a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.714+0000 7ff54affd640 1 -- 172.21.15.164:0/3018895454 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edd57da2b0 con 0x55edd5b029d0 2024-09-19T08:45:11.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.714+0000 7ff54b7fe640 1 --2- 172.21.15.164:0/3018895454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55edd5b0dbb0 0x55edd5b0ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:11.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.714+0000 7ff54affd640 1 --2- 172.21.15.164:0/3018895454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edd5b029d0 0x55edd5b0d470 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff53c005100 tx=0x7ff53c007c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3a91d42b8ea708 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:11.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.714+0000 7ff52eff5640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:45:11.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.714+0000 7ff52eff5640 1 -- 172.21.15.164:0/3018895454 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c014030 con 0x55edd5b029d0 2024-09-19T08:45:11.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.715+0000 7ff52eff5640 1 -- 172.21.15.164:0/3018895454 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff53c003970 con 0x55edd5b029d0 2024-09-19T08:45:11.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.715+0000 7ff553b24740 1 -- 172.21.15.164:0/3018895454 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff53c004d20 con 0x55edd5b029d0 2024-09-19T08:45:11.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.715+0000 7ff52eff5640 1 -- 172.21.15.164:0/3018895454 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff53c010bb0 con 0x55edd5b029d0 2024-09-19T08:45:11.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.715+0000 7ff553b24740 10 client.14967 fetch_fsmap learned FSMap version 27 2024-09-19T08:45:11.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.715+0000 7ff553b24740 1 -- 172.21.15.164:0/3018895454 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55edd5b1c040 con 0x55edd5b029d0 2024-09-19T08:45:11.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.715+0000 7ff52eff5640 1 -- 172.21.15.164:0/3018895454 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff53c004d20 con 0x55edd5b029d0 2024-09-19T08:45:11.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.715+0000 7ff52eff5640 1 -- 172.21.15.164:0/3018895454 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff53c004050 con 0x55edd5b029d0 2024-09-19T08:45:11.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.715+0000 7ff553b24740 10 client.14967 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T08:45:11.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.715+0000 7ff553b24740 10 client.14967 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T08:45:11.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.715+0000 7ff553b24740 20 client.14967 populate_metadata read hostname 'smithi164' 2024-09-19T08:45:11.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.715+0000 7ff553b24740 1 --2- 172.21.15.164:0/3018895454 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55edd5b1e480 0x55edd5b1f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:11.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.715+0000 7ff553b24740 4 client.14967 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:45:11.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.715+0000 7ff553b24740 1 -- 172.21.15.164:0/3018895454 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edd5a76600 con 0x55edd5b1e480 2024-09-19T08:45:11.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.715+0000 7ff54a7fc640 1 --2- 172.21.15.164:0/3018895454 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55edd5b1e480 0x55edd5b1f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:11.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.716+0000 7ff52eff5640 10 client.14967 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:45:11.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.716+0000 7ff54a7fc640 1 --2- 172.21.15.164:0/3018895454 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55edd5b1e480 0x55edd5b1f8f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=abb34850dfd717ba server_cookie=d50527023676f938 in_seq=0 out_seq=0 2024-09-19T08:45:11.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.717+0000 7ff52eff5640 1 -- 172.21.15.164:0/3018895454 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55edd5a76600 con 0x55edd5b1e480 2024-09-19T08:45:11.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.717+0000 7ff52eff5640 10 client.14967 handle_command_reply: tid=0 2024-09-19T08:45:11.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.717+0000 7ff553b24740 1 client.14967 shutdown 2024-09-19T08:45:11.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.717+0000 7ff553b24740 1 -- 172.21.15.164:0/3018895454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edd5b029d0 msgr2=0x55edd5b0d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:11.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.717+0000 7ff553b24740 1 --2- 172.21.15.164:0/3018895454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edd5b029d0 0x55edd5b0d470 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff53c005100 tx=0x7ff53c007c10 comp rx=0 tx=0).stop 2024-09-19T08:45:11.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.717+0000 7ff553b24740 1 -- 172.21.15.164:0/3018895454 shutdown_connections 2024-09-19T08:45:11.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.717+0000 7ff553b24740 1 --2- 172.21.15.164:0/3018895454 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55edd5b1e480 0x55edd5b1f8f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.717+0000 7ff553b24740 1 --2- 172.21.15.164:0/3018895454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55edd5b0dbb0 0x55edd5b0ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.717+0000 7ff553b24740 1 --2- 172.21.15.164:0/3018895454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edd5b032f0 0x55edd5b0a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.717+0000 7ff553b24740 1 --2- 172.21.15.164:0/3018895454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edd5b029d0 0x55edd5b0d470 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.717+0000 7ff553b24740 1 -- 172.21.15.164:0/3018895454 >> 172.21.15.164:0/3018895454 conn(0x55edd5afef60 msgr2=0x55edd5b04c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:11.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.717+0000 7ff52eff5640 0 client.14967 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:45:11.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.717+0000 7ff553b24740 1 -- 172.21.15.164:0/3018895454 shutdown_connections 2024-09-19T08:45:11.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.717+0000 7ff553b24740 1 -- 172.21.15.164:0/3018895454 wait complete. 2024-09-19T08:45:11.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.717+0000 7ff553b24740 20 client.14967 trim_cache size 0 max 16384 2024-09-19T08:45:11.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.736+0000 7ff553b24740 1 Processor -- start 2024-09-19T08:45:11.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.736+0000 7ff553b24740 1 -- start start 2024-09-19T08:45:11.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.736+0000 7ff553b24740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55edd5b029d0 0x55edd5b1ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:11.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.736+0000 7ff553b24740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edd5b032f0 0x55edd5b1f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:11.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.736+0000 7ff553b24740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edd5b0dbb0 0x55edd5b05cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:11.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.736+0000 7ff553b24740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55edd5b1dc30 con 0x55edd5b032f0 2024-09-19T08:45:11.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.736+0000 7ff553b24740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55edd5b1e300 con 0x55edd5b0dbb0 2024-09-19T08:45:11.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.736+0000 7ff553b24740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55edd5b1e9d0 con 0x55edd5b029d0 2024-09-19T08:45:11.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.736+0000 7ff54a7fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edd5b032f0 0x55edd5b1f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:11.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.736+0000 7ff54affd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55edd5b029d0 0x55edd5b1ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:11.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.736+0000 7ff54affd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55edd5b029d0 0x55edd5b1ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:46072/0 (socket says 172.21.15.164:46072) 2024-09-19T08:45:11.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.736+0000 7ff54affd640 1 -- 172.21.15.164:0/3991552090 learned_addr learned my addr 172.21.15.164:0/3991552090 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:11.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.736+0000 7ff54b7fe640 1 --2- 172.21.15.164:0/3991552090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edd5b0dbb0 0x55edd5b05cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:11.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.737+0000 7ff54affd640 1 -- 172.21.15.164:0/3991552090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edd5b0dbb0 msgr2=0x55edd5b05cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:11.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.737+0000 7ff54affd640 1 --2- 172.21.15.164:0/3991552090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edd5b0dbb0 0x55edd5b05cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.737+0000 7ff54affd640 1 -- 172.21.15.164:0/3991552090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edd5b032f0 msgr2=0x55edd5b1f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:11.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.737+0000 7ff54affd640 1 --2- 172.21.15.164:0/3991552090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edd5b032f0 0x55edd5b1f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.737+0000 7ff54affd640 1 -- 172.21.15.164:0/3991552090 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edd5b0d7e0 con 0x55edd5b029d0 2024-09-19T08:45:11.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.737+0000 7ff54a7fc640 1 --2- 172.21.15.164:0/3991552090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edd5b032f0 0x55edd5b1f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:45:11.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.737+0000 7ff54affd640 1 --2- 172.21.15.164:0/3991552090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55edd5b029d0 0x55edd5b1ddb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff53c010e60 tx=0x7ff53c004920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25c99ce0178ea2cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:11.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.737+0000 7ff530ff9640 1 -- 172.21.15.164:0/3991552090 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c014040 con 0x55edd5b029d0 2024-09-19T08:45:11.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.737+0000 7ff530ff9640 1 -- 172.21.15.164:0/3991552090 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff53c032020 con 0x55edd5b029d0 2024-09-19T08:45:11.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.737+0000 7ff530ff9640 1 -- 172.21.15.164:0/3991552090 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c005720 con 0x55edd5b029d0 2024-09-19T08:45:11.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.738+0000 7ff553b24740 1 -- 172.21.15.164:0/3991552090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55edd5b029d0 msgr2=0x55edd5b1ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:11.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.738+0000 7ff553b24740 1 --2- 172.21.15.164:0/3991552090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55edd5b029d0 0x55edd5b1ddb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff53c010e60 tx=0x7ff53c004920 comp rx=0 tx=0).stop 2024-09-19T08:45:11.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.738+0000 7ff553b24740 1 -- 172.21.15.164:0/3991552090 shutdown_connections 2024-09-19T08:45:11.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.738+0000 7ff553b24740 1 --2- 172.21.15.164:0/3991552090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edd5b0dbb0 0x55edd5b05cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.738+0000 7ff553b24740 1 --2- 172.21.15.164:0/3991552090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edd5b032f0 0x55edd5b1f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.738+0000 7ff553b24740 1 --2- 172.21.15.164:0/3991552090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55edd5b029d0 0x55edd5b1ddb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.738+0000 7ff553b24740 1 -- 172.21.15.164:0/3991552090 >> 172.21.15.164:0/3991552090 conn(0x55edd5afef60 msgr2=0x55edd5b055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:11.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.738+0000 7ff553b24740 1 -- 172.21.15.164:0/3991552090 shutdown_connections 2024-09-19T08:45:11.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.738+0000 7ff553b24740 1 -- 172.21.15.164:0/3991552090 wait complete. 2024-09-19T08:45:11.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.738+0000 7ff553b24740 1 -- start start 2024-09-19T08:45:11.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.739+0000 7ff553b24740 1 Processor -- start 2024-09-19T08:45:11.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.739+0000 7ff553b24740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edd5b029d0 0x55edd5b07c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:11.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.739+0000 7ff553b24740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55edd5b032f0 0x55edd5b08160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:11.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.739+0000 7ff553b24740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edd5b0dbb0 0x55edd5b26fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:11.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.739+0000 7ff553b24740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55edd5b1f770 con 0x55edd5b0dbb0 2024-09-19T08:45:11.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.739+0000 7ff553b24740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55edd5b1f0a0 con 0x55edd5b029d0 2024-09-19T08:45:11.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.739+0000 7ff553b24740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55edd5b1fe40 con 0x55edd5b032f0 2024-09-19T08:45:11.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.739+0000 7ff54b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edd5b0dbb0 0x55edd5b26fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:11.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.739+0000 7ff553b24740 10 client.0 _ll_register_callbacks cb 0x7ff544009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:45:11.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.739+0000 7ff54b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edd5b0dbb0 0x55edd5b26fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51940/0 (socket says 172.21.15.164:51940) 2024-09-19T08:45:11.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.739+0000 7ff54b7fe640 1 -- 172.21.15.164:0/2003651886 learned_addr learned my addr 172.21.15.164:0/2003651886 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:11.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.739+0000 7ff54a7fc640 1 --2- 172.21.15.164:0/2003651886 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55edd5b032f0 0x55edd5b08160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:11.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.739+0000 7ff54affd640 1 --2- 172.21.15.164:0/2003651886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edd5b029d0 0x55edd5b07c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:11.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.739+0000 7ff54b7fe640 1 -- 172.21.15.164:0/2003651886 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55edd5b032f0 msgr2=0x55edd5b08160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:11.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.739+0000 7ff54b7fe640 1 --2- 172.21.15.164:0/2003651886 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55edd5b032f0 0x55edd5b08160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.739+0000 7ff54b7fe640 1 -- 172.21.15.164:0/2003651886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edd5b029d0 msgr2=0x55edd5b07c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:11.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.739+0000 7ff54b7fe640 1 --2- 172.21.15.164:0/2003651886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edd5b029d0 0x55edd5b07c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.739+0000 7ff54b7fe640 1 -- 172.21.15.164:0/2003651886 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edd5b275a0 con 0x55edd5b0dbb0 2024-09-19T08:45:11.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.739+0000 7ff54affd640 1 --2- 172.21.15.164:0/2003651886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edd5b029d0 0x55edd5b07c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:45:11.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.740+0000 7ff54b7fe640 1 --2- 172.21.15.164:0/2003651886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edd5b0dbb0 0x55edd5b26fb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff544009930 tx=0x7ff544009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ef0b7981fbe573 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:11.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.740+0000 7ff52eff5640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:45:11.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.740+0000 7ff52eff5640 1 -- 172.21.15.164:0/2003651886 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff544044070 con 0x55edd5b0dbb0 2024-09-19T08:45:11.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.740+0000 7ff553b24740 1 -- 172.21.15.164:0/2003651886 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff53c032020 con 0x55edd5b0dbb0 2024-09-19T08:45:11.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.740+0000 7ff52eff5640 1 -- 172.21.15.164:0/2003651886 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7ff544003d40 con 0x55edd5b0dbb0 2024-09-19T08:45:11.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.740+0000 7ff52eff5640 1 -- 172.21.15.164:0/2003651886 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff544035040 con 0x55edd5b0dbb0 2024-09-19T08:45:11.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.740+0000 7ff553b24740 10 client.14973 fetch_fsmap learned FSMap version 27 2024-09-19T08:45:11.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.740+0000 7ff553b24740 1 -- 172.21.15.164:0/2003651886 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55edd5b8c480 con 0x55edd5b0dbb0 2024-09-19T08:45:11.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.740+0000 7ff52eff5640 1 -- 172.21.15.164:0/2003651886 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff53c032020 con 0x55edd5b0dbb0 2024-09-19T08:45:11.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.740+0000 7ff52eff5640 1 -- 172.21.15.164:0/2003651886 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff544052070 con 0x55edd5b0dbb0 2024-09-19T08:45:11.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.741+0000 7ff553b24740 10 client.14973 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T08:45:11.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.741+0000 7ff553b24740 10 client.14973 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T08:45:11.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.741+0000 7ff553b24740 20 client.14973 populate_metadata read hostname 'smithi164' 2024-09-19T08:45:11.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.741+0000 7ff553b24740 1 --2- 172.21.15.164:0/2003651886 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55edd5b910e0 0x55edd5b8fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:11.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.741+0000 7ff553b24740 4 client.14973 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:45:11.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.741+0000 7ff553b24740 1 -- 172.21.15.164:0/2003651886 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "status"}) -- 0x55edd5a70b10 con 0x55edd5b910e0 2024-09-19T08:45:11.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.741+0000 7ff54affd640 1 --2- 172.21.15.164:0/2003651886 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55edd5b910e0 0x55edd5b8fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:11.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.741+0000 7ff54affd640 1 --2- 172.21.15.164:0/2003651886 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55edd5b910e0 0x55edd5b8fdf0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5979a3481bb8aed6 server_cookie=dee929a13fb72253 in_seq=0 out_seq=0 2024-09-19T08:45:11.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.741+0000 7ff52eff5640 10 client.14973 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:45:11.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.742+0000 7ff52eff5640 1 -- 172.21.15.164:0/2003651886 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55edd5a70b10 con 0x55edd5b910e0 2024-09-19T08:45:11.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.742+0000 7ff52eff5640 10 client.14973 handle_command_reply: tid=0 2024-09-19T08:45:11.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.742+0000 7ff553b24740 1 client.14973 shutdown 2024-09-19T08:45:11.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.742+0000 7ff553b24740 1 -- 172.21.15.164:0/2003651886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edd5b0dbb0 msgr2=0x55edd5b26fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:11.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.742+0000 7ff553b24740 1 --2- 172.21.15.164:0/2003651886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edd5b0dbb0 0x55edd5b26fb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff544009930 tx=0x7ff544009260 comp rx=0 tx=0).stop 2024-09-19T08:45:11.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.742+0000 7ff553b24740 1 -- 172.21.15.164:0/2003651886 shutdown_connections 2024-09-19T08:45:11.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.742+0000 7ff553b24740 1 --2- 172.21.15.164:0/2003651886 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55edd5b910e0 0x55edd5b8fdf0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.742+0000 7ff553b24740 1 --2- 172.21.15.164:0/2003651886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edd5b0dbb0 0x55edd5b26fb0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.742+0000 7ff553b24740 1 --2- 172.21.15.164:0/2003651886 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55edd5b032f0 0x55edd5b08160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.742+0000 7ff553b24740 1 --2- 172.21.15.164:0/2003651886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edd5b029d0 0x55edd5b07c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.742+0000 7ff52eff5640 0 client.14973 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:45:11.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.742+0000 7ff553b24740 1 -- 172.21.15.164:0/2003651886 >> 172.21.15.164:0/2003651886 conn(0x55edd5afef60 msgr2=0x55edd5b05da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:11.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.742+0000 7ff553b24740 1 -- 172.21.15.164:0/2003651886 shutdown_connections 2024-09-19T08:45:11.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.742+0000 7ff553b24740 1 -- 172.21.15.164:0/2003651886 wait complete. 2024-09-19T08:45:11.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.742+0000 7ff553b24740 20 client.14973 trim_cache size 0 max 16384 2024-09-19T08:45:11.755 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:45:11.755 INFO:teuthology.orchestra.run.smithi164.stdout: "cluster_fsid": "1e4b0d74-7662-11ef-bceb-c7b262605968", 2024-09-19T08:45:11.755 INFO:teuthology.orchestra.run.smithi164.stdout: "whoami": 3, 2024-09-19T08:45:11.755 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 24424, 2024-09-19T08:45:11.755 INFO:teuthology.orchestra.run.smithi164.stdout: "want_state": "up:active", 2024-09-19T08:45:11.755 INFO:teuthology.orchestra.run.smithi164.stdout: "state": "up:active", 2024-09-19T08:45:11.756 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs", 2024-09-19T08:45:11.756 INFO:teuthology.orchestra.run.smithi164.stdout: "rank_uptime": 67.911556798999996, 2024-09-19T08:45:11.756 INFO:teuthology.orchestra.run.smithi164.stdout: "mdsmap_epoch": 27, 2024-09-19T08:45:11.756 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch": 82, 2024-09-19T08:45:11.756 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch_barrier": 82, 2024-09-19T08:45:11.756 INFO:teuthology.orchestra.run.smithi164.stdout: "uptime": 103.578615836 2024-09-19T08:45:11.756 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:45:11.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.743+0000 7ff55174b640 1 -- 172.21.15.164:0/3470111129 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7ff5340a32d0 msgr2=0x7ff5340a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:11.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.743+0000 7ff55174b640 1 --2- 172.21.15.164:0/3470111129 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7ff5340a32d0 0x7ff5340a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff544031550 tx=0x7ff54402a440 comp rx=0 tx=0).stop 2024-09-19T08:45:11.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.743+0000 7ff55174b640 1 -- 172.21.15.164:0/3470111129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff54c1036e0 msgr2=0x7ff54c198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:11.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.743+0000 7ff55174b640 1 --2- 172.21.15.164:0/3470111129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff54c1036e0 0x7ff54c198a70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff540006fd0 tx=0x7ff54000b8e0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.743+0000 7ff55174b640 1 -- 172.21.15.164:0/3470111129 shutdown_connections 2024-09-19T08:45:11.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.743+0000 7ff55174b640 1 --2- 172.21.15.164:0/3470111129 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7ff5340a32d0 0x7ff5340a5790 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.743+0000 7ff55174b640 1 --2- 172.21.15.164:0/3470111129 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff54c1082a0 0x7ff54c19ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.743+0000 7ff55174b640 1 --2- 172.21.15.164:0/3470111129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff54c1036e0 0x7ff54c198a70 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.743+0000 7ff55174b640 1 --2- 172.21.15.164:0/3470111129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff54c102d30 0x7ff54c198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.743+0000 7ff55174b640 1 -- 172.21.15.164:0/3470111129 >> 172.21.15.164:0/3470111129 conn(0x7ff54c0fc7d0 msgr2=0x7ff54c106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:11.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.743+0000 7ff55174b640 1 -- 172.21.15.164:0/3470111129 shutdown_connections 2024-09-19T08:45:11.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:11.743+0000 7ff55174b640 1 -- 172.21.15.164:0/3470111129 wait complete. 2024-09-19T08:45:12.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:11 smithi164 ceph-mon[30928]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T08:45:12.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:11 smithi170 ceph-mon[35072]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T08:45:12.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:11 smithi173 ceph-mon[35306]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T08:45:12.456 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T08:45:12.701 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:45:13.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.271+0000 7fbcaafa8640 1 -- 172.21.15.164:0/784513415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbca4100100 msgr2=0x7fbca4100520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:13.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.271+0000 7fbcaafa8640 1 --2- 172.21.15.164:0/784513415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbca4100100 0x7fbca4100520 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009900 tx=0x7fbc9002a410 comp rx=0 tx=0).stop 2024-09-19T08:45:13.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.271+0000 7fbcaafa8640 1 -- 172.21.15.164:0/784513415 shutdown_connections 2024-09-19T08:45:13.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.271+0000 7fbcaafa8640 1 --2- 172.21.15.164:0/784513415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbca4100c80 0x7fbca4110890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.271+0000 7fbcaafa8640 1 --2- 172.21.15.164:0/784513415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbca4100100 0x7fbca4100520 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.271+0000 7fbcaafa8640 1 --2- 172.21.15.164:0/784513415 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbca4102a50 0x7fbca4102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.271+0000 7fbcaafa8640 1 -- 172.21.15.164:0/784513415 >> 172.21.15.164:0/784513415 conn(0x7fbca4077230 msgr2=0x7fbca40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:13.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.272+0000 7fbcaafa8640 1 -- 172.21.15.164:0/784513415 shutdown_connections 2024-09-19T08:45:13.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.272+0000 7fbcaafa8640 1 -- 172.21.15.164:0/784513415 wait complete. 2024-09-19T08:45:13.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.272+0000 7fbcaafa8640 1 Processor -- start 2024-09-19T08:45:13.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.272+0000 7fbcaafa8640 1 -- start start 2024-09-19T08:45:13.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.272+0000 7fbcaafa8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbca4100100 0x7fbca419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:13.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.272+0000 7fbcaafa8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbca4100c80 0x7fbca419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:13.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.272+0000 7fbcaafa8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbca4102a50 0x7fbca419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:13.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.272+0000 7fbcaafa8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbca41141a0 con 0x7fbca4100100 2024-09-19T08:45:13.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.272+0000 7fbcaafa8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbca4114020 con 0x7fbca4100c80 2024-09-19T08:45:13.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.272+0000 7fbcaafa8640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbca4114320 con 0x7fbca4102a50 2024-09-19T08:45:13.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.272+0000 7fbca8d44640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbca4100100 0x7fbca419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:13.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.273+0000 7fbca8d44640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbca4100100 0x7fbca419a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51968/0 (socket says 172.21.15.164:51968) 2024-09-19T08:45:13.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.273+0000 7fbca8d44640 1 -- 172.21.15.164:0/325016491 learned_addr learned my addr 172.21.15.164:0/325016491 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:13.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.273+0000 7fbca3fff640 1 --2- 172.21.15.164:0/325016491 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbca4100c80 0x7fbca419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:13.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.273+0000 7fbca9545640 1 --2- 172.21.15.164:0/325016491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbca4102a50 0x7fbca419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:13.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.273+0000 7fbca8d44640 1 -- 172.21.15.164:0/325016491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbca4102a50 msgr2=0x7fbca419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:13.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.273+0000 7fbca8d44640 1 --2- 172.21.15.164:0/325016491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbca4102a50 0x7fbca419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.273+0000 7fbca8d44640 1 -- 172.21.15.164:0/325016491 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbca4100c80 msgr2=0x7fbca419abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:13.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.273+0000 7fbca8d44640 1 --2- 172.21.15.164:0/325016491 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbca4100c80 0x7fbca419abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.273+0000 7fbca8d44640 1 -- 172.21.15.164:0/325016491 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca419f660 con 0x7fbca4100100 2024-09-19T08:45:13.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.273+0000 7fbca3fff640 1 --2- 172.21.15.164:0/325016491 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbca4100c80 0x7fbca419abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:13.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.273+0000 7fbca8d44640 1 --2- 172.21.15.164:0/325016491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbca4100100 0x7fbca419a6b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbc9800b4a0 tx=0x7fbc9800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:13.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.273+0000 7fbca1ffb640 1 -- 172.21.15.164:0/325016491 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc98014070 con 0x7fbca4100100 2024-09-19T08:45:13.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.273+0000 7fbcaafa8640 1 -- 172.21.15.164:0/325016491 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbca419f950 con 0x7fbca4100100 2024-09-19T08:45:13.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.273+0000 7fbca1ffb640 1 -- 172.21.15.164:0/325016491 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc98010af0 con 0x7fbca4100100 2024-09-19T08:45:13.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.273+0000 7fbca1ffb640 1 -- 172.21.15.164:0/325016491 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc98017d10 con 0x7fbca4100100 2024-09-19T08:45:13.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.273+0000 7fbcaafa8640 1 -- 172.21.15.164:0/325016491 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbca41a7230 con 0x7fbca4100100 2024-09-19T08:45:13.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.276+0000 7fbca1ffb640 1 -- 172.21.15.164:0/325016491 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fbc98017070 con 0x7fbca4100100 2024-09-19T08:45:13.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.276+0000 7fbcad381740 1 Processor -- start 2024-09-19T08:45:13.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.276+0000 7fbcad381740 1 -- start start 2024-09-19T08:45:13.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.276+0000 7fbca1ffb640 1 --2- 172.21.15.164:0/325016491 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fbc940a32d0 0x7fbc940a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:13.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.276+0000 7fbcad381740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be7c4829d0 0x55be7c482db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:13.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.276+0000 7fbcad381740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be7c4832f0 0x55be7c48d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:13.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.276+0000 7fbcad381740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be7c48dbb0 0x55be7c48ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:13.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.276+0000 7fbcad381740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55be7c3e9090 con 0x55be7c4829d0 2024-09-19T08:45:13.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.276+0000 7fbcad381740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55be7c390a70 con 0x55be7c48dbb0 2024-09-19T08:45:13.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.276+0000 7fbcad381740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55be7c3e5050 con 0x55be7c4832f0 2024-09-19T08:45:13.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.276+0000 7fbca8d44640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be7c4829d0 0x55be7c482db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:13.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.276+0000 7fbca1ffb640 1 -- 172.21.15.164:0/325016491 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fbc980ced10 con 0x7fbca4100100 2024-09-19T08:45:13.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.276+0000 7fbca9545640 1 --2- 172.21.15.164:0/325016491 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fbc940a32d0 0x7fbc940a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:13.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.276+0000 7fbca8d44640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be7c4829d0 0x55be7c482db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51978/0 (socket says 172.21.15.164:51978) 2024-09-19T08:45:13.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.276+0000 7fbca8d44640 1 -- 172.21.15.164:0/1223418774 learned_addr learned my addr 172.21.15.164:0/1223418774 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:13.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.277+0000 7fbca3fff640 1 --2- 172.21.15.164:0/1223418774 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be7c4832f0 0x55be7c48d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:13.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.277+0000 7fbca9545640 1 --2- 172.21.15.164:0/1223418774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be7c48dbb0 0x55be7c48ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:13.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.277+0000 7fbca8d44640 1 -- 172.21.15.164:0/1223418774 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be7c4832f0 msgr2=0x55be7c48d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:13.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.277+0000 7fbca8d44640 1 --2- 172.21.15.164:0/1223418774 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be7c4832f0 0x55be7c48d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.277+0000 7fbca8d44640 1 -- 172.21.15.164:0/1223418774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be7c48dbb0 msgr2=0x55be7c48ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:13.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.277+0000 7fbca8d44640 1 --2- 172.21.15.164:0/1223418774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be7c48dbb0 0x55be7c48ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.277+0000 7fbca8d44640 1 -- 172.21.15.164:0/1223418774 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be7c39d130 con 0x55be7c4829d0 2024-09-19T08:45:13.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.277+0000 7fbca9545640 1 --2- 172.21.15.164:0/325016491 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fbc940a32d0 0x7fbc940a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c005fd0 tx=0x7fbc9c0147e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:13.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.277+0000 7fbca8d44640 1 --2- 172.21.15.164:0/1223418774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be7c4829d0 0x55be7c482db0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbc980cc900 tx=0x7fbc98009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e5fdff7b77f11a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:13.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.277+0000 7fbc8effd640 1 -- 172.21.15.164:0/1223418774 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9801e070 con 0x55be7c4829d0 2024-09-19T08:45:13.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.277+0000 7fbc8effd640 1 -- 172.21.15.164:0/1223418774 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc9809f790 con 0x55be7c4829d0 2024-09-19T08:45:13.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.277+0000 7fbc8effd640 1 -- 172.21.15.164:0/1223418774 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc980bb320 con 0x55be7c4829d0 2024-09-19T08:45:13.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.278+0000 7fbcad381740 1 -- 172.21.15.164:0/1223418774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be7c4829d0 msgr2=0x55be7c482db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:13.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.278+0000 7fbcad381740 1 --2- 172.21.15.164:0/1223418774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be7c4829d0 0x55be7c482db0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbc980cc900 tx=0x7fbc98009390 comp rx=0 tx=0).stop 2024-09-19T08:45:13.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.278+0000 7fbcad381740 1 -- 172.21.15.164:0/1223418774 shutdown_connections 2024-09-19T08:45:13.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.278+0000 7fbcad381740 1 --2- 172.21.15.164:0/1223418774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be7c48dbb0 0x55be7c48ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.278+0000 7fbcad381740 1 --2- 172.21.15.164:0/1223418774 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be7c4832f0 0x55be7c48d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.278+0000 7fbcad381740 1 --2- 172.21.15.164:0/1223418774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be7c4829d0 0x55be7c482db0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.278+0000 7fbcad381740 1 -- 172.21.15.164:0/1223418774 >> 172.21.15.164:0/1223418774 conn(0x55be7c47ef60 msgr2=0x55be7c47f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:13.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.278+0000 7fbcad381740 1 -- 172.21.15.164:0/1223418774 shutdown_connections 2024-09-19T08:45:13.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.278+0000 7fbcad381740 1 -- 172.21.15.164:0/1223418774 wait complete. 2024-09-19T08:45:13.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.278+0000 7fbcad381740 1 -- start start 2024-09-19T08:45:13.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.278+0000 7fbcad381740 1 Processor -- start 2024-09-19T08:45:13.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.279+0000 7fbcad381740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be7c4829d0 0x55be7c48d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:13.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.279+0000 7fbcad381740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be7c4832f0 0x55be7c48a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:13.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.279+0000 7fbcad381740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be7c48dbb0 0x55be7c48ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:13.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.279+0000 7fbcad381740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55be7c490780 con 0x55be7c48dbb0 2024-09-19T08:45:13.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.279+0000 7fbcad381740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55be7c3a3f20 con 0x55be7c4832f0 2024-09-19T08:45:13.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.279+0000 7fbcad381740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55be7c490900 con 0x55be7c4829d0 2024-09-19T08:45:13.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.279+0000 7fbcad381740 10 client.0 _ll_register_callbacks cb 0x55be7c36d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:45:13.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.279+0000 7fbca9545640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be7c48dbb0 0x55be7c48ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:13.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.279+0000 7fbca9545640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be7c48dbb0 0x55be7c48ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51984/0 (socket says 172.21.15.164:51984) 2024-09-19T08:45:13.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.279+0000 7fbca9545640 1 -- 172.21.15.164:0/2950837114 learned_addr learned my addr 172.21.15.164:0/2950837114 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:13.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.279+0000 7fbca3fff640 1 --2- 172.21.15.164:0/2950837114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be7c4832f0 0x55be7c48a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:13.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.279+0000 7fbca9545640 1 -- 172.21.15.164:0/2950837114 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be7c4829d0 msgr2=0x55be7c48d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:13.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.279+0000 7fbca9545640 1 --2- 172.21.15.164:0/2950837114 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be7c4829d0 0x55be7c48d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.279+0000 7fbca9545640 1 -- 172.21.15.164:0/2950837114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be7c4832f0 msgr2=0x55be7c48a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:13.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.279+0000 7fbca9545640 1 --2- 172.21.15.164:0/2950837114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be7c4832f0 0x55be7c48a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.279+0000 7fbca9545640 1 -- 172.21.15.164:0/2950837114 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be7c15a2b0 con 0x55be7c48dbb0 2024-09-19T08:45:13.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.279+0000 7fbca3fff640 1 --2- 172.21.15.164:0/2950837114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be7c4832f0 0x55be7c48a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:13.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.279+0000 7fbca9545640 1 --2- 172.21.15.164:0/2950837114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be7c48dbb0 0x55be7c48ad60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c012d40 tx=0x7fbc9c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac99fd4d31245d7e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:13.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.279+0000 7fbc83fff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:45:13.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.279+0000 7fbc83fff640 1 -- 172.21.15.164:0/2950837114 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c01e5f0 con 0x55be7c48dbb0 2024-09-19T08:45:13.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.279+0000 7fbcad381740 1 -- 172.21.15.164:0/2950837114 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc9809f790 con 0x55be7c48dbb0 2024-09-19T08:45:13.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.279+0000 7fbc83fff640 1 -- 172.21.15.164:0/2950837114 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fbc9c01e790 con 0x55be7c48dbb0 2024-09-19T08:45:13.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.280+0000 7fbc83fff640 1 -- 172.21.15.164:0/2950837114 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbc9c030050 con 0x55be7c48dbb0 2024-09-19T08:45:13.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.280+0000 7fbcad381740 10 client.14985 fetch_fsmap learned FSMap version 27 2024-09-19T08:45:13.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.280+0000 7fbcad381740 1 -- 172.21.15.164:0/2950837114 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55be7c49c100 con 0x55be7c48dbb0 2024-09-19T08:45:13.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.280+0000 7fbc83fff640 1 -- 172.21.15.164:0/2950837114 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc9809f790 con 0x55be7c48dbb0 2024-09-19T08:45:13.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.280+0000 7fbc83fff640 1 -- 172.21.15.164:0/2950837114 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbc9c0299e0 con 0x55be7c48dbb0 2024-09-19T08:45:13.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.280+0000 7fbcad381740 10 client.14985 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T08:45:13.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.280+0000 7fbcad381740 10 client.14985 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T08:45:13.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.280+0000 7fbcad381740 20 client.14985 populate_metadata read hostname 'smithi164' 2024-09-19T08:45:13.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.280+0000 7fbcad381740 1 --2- 172.21.15.164:0/2950837114 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55be7c49e7d0 0x55be7c49fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:13.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.280+0000 7fbcad381740 4 client.14985 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:45:13.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.280+0000 7fbcad381740 1 -- 172.21.15.164:0/2950837114 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be7c3f6600 con 0x55be7c49e7d0 2024-09-19T08:45:13.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.280+0000 7fbca8d44640 1 --2- 172.21.15.164:0/2950837114 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55be7c49e7d0 0x55be7c49fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:13.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.281+0000 7fbc83fff640 10 client.14985 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:45:13.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.281+0000 7fbca8d44640 1 --2- 172.21.15.164:0/2950837114 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55be7c49e7d0 0x55be7c49fc40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4acf3e91aceddde4 server_cookie=a77548c36c25cbf9 in_seq=0 out_seq=0 2024-09-19T08:45:13.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.282+0000 7fbc83fff640 1 -- 172.21.15.164:0/2950837114 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55be7c3f6600 con 0x55be7c49e7d0 2024-09-19T08:45:13.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.282+0000 7fbc83fff640 10 client.14985 handle_command_reply: tid=0 2024-09-19T08:45:13.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.282+0000 7fbcad381740 1 client.14985 shutdown 2024-09-19T08:45:13.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.282+0000 7fbcad381740 1 -- 172.21.15.164:0/2950837114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be7c48dbb0 msgr2=0x55be7c48ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:13.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.282+0000 7fbcad381740 1 --2- 172.21.15.164:0/2950837114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be7c48dbb0 0x55be7c48ad60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c012d40 tx=0x7fbc9c004ab0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.282+0000 7fbcad381740 1 -- 172.21.15.164:0/2950837114 shutdown_connections 2024-09-19T08:45:13.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.282+0000 7fbcad381740 1 --2- 172.21.15.164:0/2950837114 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55be7c49e7d0 0x55be7c49fc40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.282+0000 7fbcad381740 1 --2- 172.21.15.164:0/2950837114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be7c48dbb0 0x55be7c48ad60 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.282+0000 7fbc83fff640 0 client.14985 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:45:13.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.282+0000 7fbcad381740 1 --2- 172.21.15.164:0/2950837114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be7c4832f0 0x55be7c48a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.282+0000 7fbcad381740 1 --2- 172.21.15.164:0/2950837114 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be7c4829d0 0x55be7c48d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.282+0000 7fbcad381740 1 -- 172.21.15.164:0/2950837114 >> 172.21.15.164:0/2950837114 conn(0x55be7c47ef60 msgr2=0x55be7c484c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:13.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.282+0000 7fbcad381740 1 -- 172.21.15.164:0/2950837114 shutdown_connections 2024-09-19T08:45:13.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.282+0000 7fbcad381740 1 -- 172.21.15.164:0/2950837114 wait complete. 2024-09-19T08:45:13.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.282+0000 7fbcad381740 20 client.14985 trim_cache size 0 max 16384 2024-09-19T08:45:13.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.300+0000 7fbcad381740 1 Processor -- start 2024-09-19T08:45:13.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.300+0000 7fbcad381740 1 -- start start 2024-09-19T08:45:13.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.300+0000 7fbcad381740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be7c4829d0 0x55be7c49fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:13.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.300+0000 7fbcad381740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be7c4832f0 0x55be7c48a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:13.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.300+0000 7fbcad381740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be7c48dbb0 0x55be7c48ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:13.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.300+0000 7fbcad381740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55be7c49df80 con 0x55be7c4829d0 2024-09-19T08:45:13.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.300+0000 7fbcad381740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55be7c49e650 con 0x55be7c4832f0 2024-09-19T08:45:13.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.300+0000 7fbcad381740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55be7c49ed20 con 0x55be7c48dbb0 2024-09-19T08:45:13.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.300+0000 7fbca8d44640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be7c4829d0 0x55be7c49fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:13.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.300+0000 7fbca8d44640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be7c4829d0 0x55be7c49fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51994/0 (socket says 172.21.15.164:51994) 2024-09-19T08:45:13.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.300+0000 7fbca8d44640 1 -- 172.21.15.164:0/2104093191 learned_addr learned my addr 172.21.15.164:0/2104093191 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:13.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.300+0000 7fbca9545640 1 --2- 172.21.15.164:0/2104093191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be7c48dbb0 0x55be7c48ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:13.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.301+0000 7fbca3fff640 1 --2- 172.21.15.164:0/2104093191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be7c4832f0 0x55be7c48a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:13.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.301+0000 7fbca8d44640 1 -- 172.21.15.164:0/2104093191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be7c48dbb0 msgr2=0x55be7c48ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:13.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.301+0000 7fbca8d44640 1 --2- 172.21.15.164:0/2104093191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be7c48dbb0 0x55be7c48ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.301+0000 7fbca8d44640 1 -- 172.21.15.164:0/2104093191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be7c4832f0 msgr2=0x55be7c48a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:13.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.301+0000 7fbca8d44640 1 --2- 172.21.15.164:0/2104093191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be7c4832f0 0x55be7c48a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.301+0000 7fbca8d44640 1 -- 172.21.15.164:0/2104093191 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be7c48bba0 con 0x55be7c4829d0 2024-09-19T08:45:13.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.301+0000 7fbca8d44640 1 --2- 172.21.15.164:0/2104093191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be7c4829d0 0x55be7c49fc40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbc980cc950 tx=0x7fbc98000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d708d57a2dced1f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:13.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.301+0000 7fbc8e7fc640 1 -- 172.21.15.164:0/2104093191 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc980b9070 con 0x55be7c4829d0 2024-09-19T08:45:13.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.301+0000 7fbc8e7fc640 1 -- 172.21.15.164:0/2104093191 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc98099480 con 0x55be7c4829d0 2024-09-19T08:45:13.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.301+0000 7fbc8e7fc640 1 -- 172.21.15.164:0/2104093191 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc980b3030 con 0x55be7c4829d0 2024-09-19T08:45:13.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.302+0000 7fbcad381740 1 -- 172.21.15.164:0/2104093191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be7c4829d0 msgr2=0x55be7c49fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:13.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.302+0000 7fbcad381740 1 --2- 172.21.15.164:0/2104093191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be7c4829d0 0x55be7c49fc40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbc980cc950 tx=0x7fbc98000bc0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.302+0000 7fbcad381740 1 -- 172.21.15.164:0/2104093191 shutdown_connections 2024-09-19T08:45:13.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.302+0000 7fbcad381740 1 --2- 172.21.15.164:0/2104093191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be7c48dbb0 0x55be7c48ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.302+0000 7fbcad381740 1 --2- 172.21.15.164:0/2104093191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be7c4832f0 0x55be7c48a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.302+0000 7fbcad381740 1 --2- 172.21.15.164:0/2104093191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be7c4829d0 0x55be7c49fc40 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.302+0000 7fbcad381740 1 -- 172.21.15.164:0/2104093191 >> 172.21.15.164:0/2104093191 conn(0x55be7c47ef60 msgr2=0x55be7c489d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:13.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.302+0000 7fbcad381740 1 -- 172.21.15.164:0/2104093191 shutdown_connections 2024-09-19T08:45:13.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.302+0000 7fbcad381740 1 -- 172.21.15.164:0/2104093191 wait complete. 2024-09-19T08:45:13.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.302+0000 7fbcad381740 1 -- start start 2024-09-19T08:45:13.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.302+0000 7fbcad381740 1 Processor -- start 2024-09-19T08:45:13.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.302+0000 7fbcad381740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be7c4829d0 0x55be7c4a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:13.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.302+0000 7fbcad381740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be7c4832f0 0x55be7c4a3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:13.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.302+0000 7fbcad381740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be7c48dbb0 0x55be7c4a3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:13.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.302+0000 7fbcad381740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55be7c49fac0 con 0x55be7c4832f0 2024-09-19T08:45:13.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.302+0000 7fbcad381740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55be7c49f3f0 con 0x55be7c48dbb0 2024-09-19T08:45:13.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.302+0000 7fbcad381740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55be7c4a0190 con 0x55be7c4829d0 2024-09-19T08:45:13.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.302+0000 7fbcad381740 10 client.0 _ll_register_callbacks cb 0x7fbc9c009690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:45:13.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.303+0000 7fbca3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be7c4832f0 0x55be7c4a3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:13.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.303+0000 7fbca3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be7c4832f0 0x55be7c4a3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52002/0 (socket says 172.21.15.164:52002) 2024-09-19T08:45:13.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.303+0000 7fbca3fff640 1 -- 172.21.15.164:0/2008876630 learned_addr learned my addr 172.21.15.164:0/2008876630 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:13.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.303+0000 7fbca8d44640 1 --2- 172.21.15.164:0/2008876630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be7c4829d0 0x55be7c4a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:13.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.303+0000 7fbca3fff640 1 -- 172.21.15.164:0/2008876630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be7c4829d0 msgr2=0x55be7c4a2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:13.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.303+0000 7fbca3fff640 1 --2- 172.21.15.164:0/2008876630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be7c4829d0 0x55be7c4a2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.303+0000 7fbca3fff640 1 -- 172.21.15.164:0/2008876630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be7c48dbb0 msgr2=0x55be7c4a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:13.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.303+0000 7fbca3fff640 1 --2- 172.21.15.164:0/2008876630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be7c48dbb0 0x55be7c4a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.303+0000 7fbca3fff640 1 -- 172.21.15.164:0/2008876630 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be7c5070a0 con 0x55be7c4832f0 2024-09-19T08:45:13.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.303+0000 7fbca8d44640 1 --2- 172.21.15.164:0/2008876630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be7c4829d0 0x55be7c4a2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:13.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.303+0000 7fbca3fff640 1 --2- 172.21.15.164:0/2008876630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be7c4832f0 0x55be7c4a3340 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009900 tx=0x7fbc90031a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2693dcf6d32f29 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:13.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.303+0000 7fbc83fff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:45:13.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.303+0000 7fbc83fff640 1 -- 172.21.15.164:0/2008876630 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc90035040 con 0x55be7c4832f0 2024-09-19T08:45:13.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.303+0000 7fbc83fff640 1 -- 172.21.15.164:0/2008876630 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fbc90004250 con 0x55be7c4832f0 2024-09-19T08:45:13.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.303+0000 7fbcad381740 1 -- 172.21.15.164:0/2008876630 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc98099480 con 0x55be7c4832f0 2024-09-19T08:45:13.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.303+0000 7fbc83fff640 1 -- 172.21.15.164:0/2008876630 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbc90004450 con 0x55be7c4832f0 2024-09-19T08:45:13.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.303+0000 7fbcad381740 10 client.14991 fetch_fsmap learned FSMap version 27 2024-09-19T08:45:13.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.303+0000 7fbcad381740 1 -- 172.21.15.164:0/2008876630 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55be7c508370 con 0x55be7c4832f0 2024-09-19T08:45:13.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.304+0000 7fbc83fff640 1 -- 172.21.15.164:0/2008876630 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc98099480 con 0x55be7c4832f0 2024-09-19T08:45:13.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.304+0000 7fbc83fff640 1 -- 172.21.15.164:0/2008876630 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbc90004c40 con 0x55be7c4832f0 2024-09-19T08:45:13.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.304+0000 7fbcad381740 10 client.14991 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T08:45:13.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.304+0000 7fbcad381740 10 client.14991 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T08:45:13.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.304+0000 7fbcad381740 20 client.14991 populate_metadata read hostname 'smithi164' 2024-09-19T08:45:13.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.304+0000 7fbcad381740 1 --2- 172.21.15.164:0/2008876630 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55be7c50d050 0x55be7c50bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:13.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.304+0000 7fbcad381740 4 client.14991 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:45:13.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.304+0000 7fbcad381740 1 -- 172.21.15.164:0/2008876630 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "status"}) -- 0x55be7c3f0b10 con 0x55be7c50d050 2024-09-19T08:45:13.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.304+0000 7fbca8d44640 1 --2- 172.21.15.164:0/2008876630 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55be7c50d050 0x55be7c50bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:13.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.304+0000 7fbc83fff640 10 client.14991 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:45:13.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.304+0000 7fbca8d44640 1 --2- 172.21.15.164:0/2008876630 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55be7c50d050 0x55be7c50bd60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f112464f09468940 server_cookie=94757d7d092fa0f5 in_seq=0 out_seq=0 2024-09-19T08:45:13.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.305+0000 7fbc83fff640 1 -- 172.21.15.164:0/2008876630 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55be7c3f0b10 con 0x55be7c50d050 2024-09-19T08:45:13.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.305+0000 7fbc83fff640 10 client.14991 handle_command_reply: tid=0 2024-09-19T08:45:13.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.305+0000 7fbcad381740 1 client.14991 shutdown 2024-09-19T08:45:13.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.305+0000 7fbcad381740 1 -- 172.21.15.164:0/2008876630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be7c4832f0 msgr2=0x55be7c4a3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:13.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.305+0000 7fbcad381740 1 --2- 172.21.15.164:0/2008876630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be7c4832f0 0x55be7c4a3340 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009900 tx=0x7fbc90031a60 comp rx=0 tx=0).stop 2024-09-19T08:45:13.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.305+0000 7fbcad381740 1 -- 172.21.15.164:0/2008876630 shutdown_connections 2024-09-19T08:45:13.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.305+0000 7fbcad381740 1 --2- 172.21.15.164:0/2008876630 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55be7c50d050 0x55be7c50bd60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.305+0000 7fbcad381740 1 --2- 172.21.15.164:0/2008876630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be7c48dbb0 0x55be7c4a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.305+0000 7fbcad381740 1 --2- 172.21.15.164:0/2008876630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be7c4832f0 0x55be7c4a3340 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.305+0000 7fbc83fff640 0 client.14991 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:45:13.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.305+0000 7fbcad381740 1 --2- 172.21.15.164:0/2008876630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be7c4829d0 0x55be7c4a2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.305+0000 7fbcad381740 1 -- 172.21.15.164:0/2008876630 >> 172.21.15.164:0/2008876630 conn(0x55be7c47ef60 msgr2=0x55be7c48a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:13.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.305+0000 7fbcad381740 1 -- 172.21.15.164:0/2008876630 shutdown_connections 2024-09-19T08:45:13.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.305+0000 7fbcad381740 1 -- 172.21.15.164:0/2008876630 wait complete. 2024-09-19T08:45:13.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.305+0000 7fbcad381740 20 client.14991 trim_cache size 0 max 16384 2024-09-19T08:45:13.314 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:45:13.315 INFO:teuthology.orchestra.run.smithi164.stdout: "cluster_fsid": "1e4b0d74-7662-11ef-bceb-c7b262605968", 2024-09-19T08:45:13.315 INFO:teuthology.orchestra.run.smithi164.stdout: "whoami": 2, 2024-09-19T08:45:13.315 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 24442, 2024-09-19T08:45:13.315 INFO:teuthology.orchestra.run.smithi164.stdout: "want_state": "up:active", 2024-09-19T08:45:13.315 INFO:teuthology.orchestra.run.smithi164.stdout: "state": "up:active", 2024-09-19T08:45:13.315 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs", 2024-09-19T08:45:13.315 INFO:teuthology.orchestra.run.smithi164.stdout: "rank_uptime": 71.489225993999995, 2024-09-19T08:45:13.315 INFO:teuthology.orchestra.run.smithi164.stdout: "mdsmap_epoch": 27, 2024-09-19T08:45:13.315 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch": 82, 2024-09-19T08:45:13.315 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch_barrier": 82, 2024-09-19T08:45:13.315 INFO:teuthology.orchestra.run.smithi164.stdout: "uptime": 99.917448226000005 2024-09-19T08:45:13.315 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:45:13.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.306+0000 7fbcaafa8640 1 -- 172.21.15.164:0/325016491 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fbc940a32d0 msgr2=0x7fbc940a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:13.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.306+0000 7fbcaafa8640 1 --2- 172.21.15.164:0/325016491 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fbc940a32d0 0x7fbc940a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c005fd0 tx=0x7fbc9c0147e0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.306+0000 7fbcaafa8640 1 -- 172.21.15.164:0/325016491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbca4100100 msgr2=0x7fbca419a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:13.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.306+0000 7fbcaafa8640 1 --2- 172.21.15.164:0/325016491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbca4100100 0x7fbca419a6b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbc9800b4a0 tx=0x7fbc9800b960 comp rx=0 tx=0).stop 2024-09-19T08:45:13.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.306+0000 7fbcaafa8640 1 -- 172.21.15.164:0/325016491 shutdown_connections 2024-09-19T08:45:13.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.306+0000 7fbcaafa8640 1 --2- 172.21.15.164:0/325016491 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fbc940a32d0 0x7fbc940a5790 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.306+0000 7fbcaafa8640 1 --2- 172.21.15.164:0/325016491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbca4102a50 0x7fbca419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.306+0000 7fbcaafa8640 1 --2- 172.21.15.164:0/325016491 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbca4100c80 0x7fbca419abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.306+0000 7fbcaafa8640 1 --2- 172.21.15.164:0/325016491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbca4100100 0x7fbca419a6b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.306+0000 7fbcaafa8640 1 -- 172.21.15.164:0/325016491 >> 172.21.15.164:0/325016491 conn(0x7fbca4077230 msgr2=0x7fbca40fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:13.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.306+0000 7fbcaafa8640 1 -- 172.21.15.164:0/325016491 shutdown_connections 2024-09-19T08:45:13.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:13.306+0000 7fbcaafa8640 1 -- 172.21.15.164:0/325016491 wait complete. 2024-09-19T08:45:14.025 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T08:45:14.268 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:45:14.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.775+0000 7f690bf81640 1 -- 172.21.15.164:0/1930216267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6904103740 msgr2=0x7f6904107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:14.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.775+0000 7f690bf81640 1 --2- 172.21.15.164:0/1930216267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6904103740 0x7f6904107b80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f68f8009930 tx=0x7f68f802a420 comp rx=0 tx=0).stop 2024-09-19T08:45:14.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.775+0000 7f690bf81640 1 -- 172.21.15.164:0/1930216267 shutdown_connections 2024-09-19T08:45:14.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.775+0000 7f690bf81640 1 --2- 172.21.15.164:0/1930216267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6904108290 0x7f690410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.775+0000 7f690bf81640 1 --2- 172.21.15.164:0/1930216267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6904103740 0x7f6904107b80 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.775+0000 7f690bf81640 1 --2- 172.21.15.164:0/1930216267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6904102d90 0x7f6904103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.775+0000 7f690bf81640 1 -- 172.21.15.164:0/1930216267 >> 172.21.15.164:0/1930216267 conn(0x7f69040fc7d0 msgr2=0x7f69040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:14.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.775+0000 7f690bf81640 1 -- 172.21.15.164:0/1930216267 shutdown_connections 2024-09-19T08:45:14.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.775+0000 7f690bf81640 1 -- 172.21.15.164:0/1930216267 wait complete. 2024-09-19T08:45:14.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.776+0000 7f690bf81640 1 Processor -- start 2024-09-19T08:45:14.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.776+0000 7f690bf81640 1 -- start start 2024-09-19T08:45:14.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.776+0000 7f690bf81640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6904102d90 0x7f6904198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:14.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.776+0000 7f690bf81640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6904103740 0x7f6904198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:14.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.776+0000 7f690bf81640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6904108290 0x7f690419cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:14.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.776+0000 7f690bf81640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6904112090 con 0x7f6904103740 2024-09-19T08:45:14.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.776+0000 7f690bf81640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6904111f10 con 0x7f6904102d90 2024-09-19T08:45:14.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.776+0000 7f690bf81640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6904112210 con 0x7f6904108290 2024-09-19T08:45:14.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.776+0000 7f690951c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6904103740 0x7f6904198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:14.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.776+0000 7f690951c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6904103740 0x7f6904198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52700/0 (socket says 172.21.15.164:52700) 2024-09-19T08:45:14.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.776+0000 7f690951c640 1 -- 172.21.15.164:0/3863317401 learned_addr learned my addr 172.21.15.164:0/3863317401 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:14.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.776+0000 7f6909d1d640 1 --2- 172.21.15.164:0/3863317401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6904102d90 0x7f6904198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:14.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.776+0000 7f690a51e640 1 --2- 172.21.15.164:0/3863317401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6904108290 0x7f690419cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:14.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.776+0000 7f690951c640 1 -- 172.21.15.164:0/3863317401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6904108290 msgr2=0x7f690419cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:14.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.776+0000 7f690951c640 1 --2- 172.21.15.164:0/3863317401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6904108290 0x7f690419cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.776+0000 7f690951c640 1 -- 172.21.15.164:0/3863317401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6904102d90 msgr2=0x7f6904198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:14.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.776+0000 7f690951c640 1 --2- 172.21.15.164:0/3863317401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6904102d90 0x7f6904198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.776+0000 7f690951c640 1 -- 172.21.15.164:0/3863317401 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f690419d600 con 0x7f6904103740 2024-09-19T08:45:14.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.777+0000 7f690951c640 1 --2- 172.21.15.164:0/3863317401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6904103740 0x7f6904198b90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f68f8005940 tx=0x7f68f80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:14.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.777+0000 7f6902ffd640 1 -- 172.21.15.164:0/3863317401 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f803f030 con 0x7f6904103740 2024-09-19T08:45:14.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.777+0000 7f690bf81640 1 -- 172.21.15.164:0/3863317401 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f690419d890 con 0x7f6904103740 2024-09-19T08:45:14.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.777+0000 7f6902ffd640 1 -- 172.21.15.164:0/3863317401 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68f8003ce0 con 0x7f6904103740 2024-09-19T08:45:14.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.777+0000 7f690bf81640 1 -- 172.21.15.164:0/3863317401 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69041a51d0 con 0x7f6904103740 2024-09-19T08:45:14.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.777+0000 7f6902ffd640 1 -- 172.21.15.164:0/3863317401 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8035070 con 0x7f6904103740 2024-09-19T08:45:14.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.779+0000 7f6902ffd640 1 -- 172.21.15.164:0/3863317401 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f68f8042030 con 0x7f6904103740 2024-09-19T08:45:14.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.779+0000 7f690e35a740 1 Processor -- start 2024-09-19T08:45:14.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.780+0000 7f690e35a740 1 -- start start 2024-09-19T08:45:14.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.780+0000 7f690e35a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55726c3308d0 0x55726c330cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:14.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.780+0000 7f690e35a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55726c3311f0 0x55726c33fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:14.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.780+0000 7f690e35a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55726c33ffc0 0x55726c3423b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:14.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.780+0000 7f690e35a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55726c297090 con 0x55726c3311f0 2024-09-19T08:45:14.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.780+0000 7f690e35a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55726c23ea70 con 0x55726c3308d0 2024-09-19T08:45:14.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.780+0000 7f690e35a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55726c293050 con 0x55726c33ffc0 2024-09-19T08:45:14.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.780+0000 7f6909d1d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55726c3311f0 0x55726c33fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:14.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.780+0000 7f6909d1d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55726c3311f0 0x55726c33fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52706/0 (socket says 172.21.15.164:52706) 2024-09-19T08:45:14.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.780+0000 7f690a51e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55726c3308d0 0x55726c330cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:14.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.780+0000 7f6909d1d640 1 -- 172.21.15.164:0/3494699677 learned_addr learned my addr 172.21.15.164:0/3494699677 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:14.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.780+0000 7f6902ffd640 1 --2- 172.21.15.164:0/3863317401 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f68ec0a3350 0x7f68ec0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:14.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.780+0000 7f690951c640 1 --2- 172.21.15.164:0/3494699677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55726c33ffc0 0x55726c3423b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:14.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.780+0000 7f6909d1d640 1 -- 172.21.15.164:0/3494699677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55726c33ffc0 msgr2=0x55726c3423b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:14.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.780+0000 7f6909d1d640 1 --2- 172.21.15.164:0/3494699677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55726c33ffc0 0x55726c3423b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.780+0000 7f6909d1d640 1 -- 172.21.15.164:0/3494699677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55726c3308d0 msgr2=0x55726c330cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:14.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.780+0000 7f6909d1d640 1 --2- 172.21.15.164:0/3494699677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55726c3308d0 0x55726c330cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.780+0000 7f6909d1d640 1 -- 172.21.15.164:0/3494699677 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55726c24b130 con 0x55726c3311f0 2024-09-19T08:45:14.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.780+0000 7f690a51e640 1 --2- 172.21.15.164:0/3863317401 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f68ec0a3350 0x7f68ec0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:14.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.780+0000 7f6902ffd640 1 -- 172.21.15.164:0/3863317401 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f68f80efea0 con 0x7f6904103740 2024-09-19T08:45:14.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.781+0000 7f6909d1d640 1 --2- 172.21.15.164:0/3494699677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55726c3311f0 0x55726c33fa80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f68fc00bd10 tx=0x7f68fc0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac7c4286f1f90e24 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:14.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.781+0000 7f68ebfff640 1 -- 172.21.15.164:0/3494699677 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc004030 con 0x55726c3311f0 2024-09-19T08:45:14.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.781+0000 7f690a51e640 1 --2- 172.21.15.164:0/3863317401 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f68ec0a3350 0x7f68ec0a5810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f68f4005fd0 tx=0x7f68f4004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:14.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.781+0000 7f68ebfff640 1 -- 172.21.15.164:0/3494699677 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68fc0041d0 con 0x55726c3311f0 2024-09-19T08:45:14.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.781+0000 7f68ebfff640 1 -- 172.21.15.164:0/3494699677 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc011840 con 0x55726c3311f0 2024-09-19T08:45:14.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.781+0000 7f690e35a740 1 -- 172.21.15.164:0/3494699677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55726c3311f0 msgr2=0x55726c33fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:14.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.781+0000 7f690e35a740 1 --2- 172.21.15.164:0/3494699677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55726c3311f0 0x55726c33fa80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f68fc00bd10 tx=0x7f68fc0107a0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.781+0000 7f690e35a740 1 -- 172.21.15.164:0/3494699677 shutdown_connections 2024-09-19T08:45:14.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.782+0000 7f690e35a740 1 --2- 172.21.15.164:0/3494699677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55726c33ffc0 0x55726c3423b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.782+0000 7f690e35a740 1 --2- 172.21.15.164:0/3494699677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55726c3311f0 0x55726c33fa80 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.782+0000 7f690e35a740 1 --2- 172.21.15.164:0/3494699677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55726c3308d0 0x55726c330cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.782+0000 7f690e35a740 1 -- 172.21.15.164:0/3494699677 >> 172.21.15.164:0/3494699677 conn(0x55726c32cf60 msgr2=0x55726c32d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:14.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.782+0000 7f690e35a740 1 -- 172.21.15.164:0/3494699677 shutdown_connections 2024-09-19T08:45:14.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.782+0000 7f690e35a740 1 -- 172.21.15.164:0/3494699677 wait complete. 2024-09-19T08:45:14.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.782+0000 7f690e35a740 1 -- start start 2024-09-19T08:45:14.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.782+0000 7f690e35a740 1 Processor -- start 2024-09-19T08:45:14.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.782+0000 7f690e35a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55726c3308d0 0x55726c33f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:14.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.782+0000 7f690e35a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55726c3311f0 0x55726c33cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:14.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.782+0000 7f690e35a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55726c33ffc0 0x55726c33d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:14.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.782+0000 7f690e35a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55726c342bf0 con 0x55726c3308d0 2024-09-19T08:45:14.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.782+0000 7f690e35a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55726c251f20 con 0x55726c33ffc0 2024-09-19T08:45:14.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.782+0000 7f690e35a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55726c342d70 con 0x55726c3311f0 2024-09-19T08:45:14.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.783+0000 7f690e35a740 10 client.0 _ll_register_callbacks cb 0x55726c21b030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:45:14.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.783+0000 7f6909d1d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55726c3308d0 0x55726c33f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:14.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.783+0000 7f6909d1d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55726c3308d0 0x55726c33f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52714/0 (socket says 172.21.15.164:52714) 2024-09-19T08:45:14.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.783+0000 7f6909d1d640 1 -- 172.21.15.164:0/268921421 learned_addr learned my addr 172.21.15.164:0/268921421 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:14.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.783+0000 7f690951c640 1 --2- 172.21.15.164:0/268921421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55726c3311f0 0x55726c33cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:14.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.783+0000 7f6909d1d640 1 -- 172.21.15.164:0/268921421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55726c3311f0 msgr2=0x55726c33cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:14.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.783+0000 7f6909d1d640 1 --2- 172.21.15.164:0/268921421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55726c3311f0 0x55726c33cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.783+0000 7f6909d1d640 1 -- 172.21.15.164:0/268921421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55726c33ffc0 msgr2=0x55726c33d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:14.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.783+0000 7f6909d1d640 1 --2- 172.21.15.164:0/268921421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55726c33ffc0 0x55726c33d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.783+0000 7f6909d1d640 1 -- 172.21.15.164:0/268921421 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55726c0082b0 con 0x55726c3308d0 2024-09-19T08:45:14.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.783+0000 7f6909d1d640 1 --2- 172.21.15.164:0/268921421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55726c3308d0 0x55726c33f800 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f68fc0114a0 tx=0x7f68fc003e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac4c5c32e180860 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:14.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.783+0000 7f68e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:45:14.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.783+0000 7f68e97fa640 1 -- 172.21.15.164:0/268921421 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc0035f0 con 0x55726c3308d0 2024-09-19T08:45:14.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.783+0000 7f68e97fa640 1 -- 172.21.15.164:0/268921421 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f68fc003790 con 0x55726c3308d0 2024-09-19T08:45:14.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.783+0000 7f690e35a740 1 -- 172.21.15.164:0/268921421 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68fc0041d0 con 0x55726c3308d0 2024-09-19T08:45:14.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.783+0000 7f68e97fa640 1 -- 172.21.15.164:0/268921421 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f68fc0039b0 con 0x55726c3308d0 2024-09-19T08:45:14.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.783+0000 7f690e35a740 10 client.15003 fetch_fsmap learned FSMap version 27 2024-09-19T08:45:14.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.783+0000 7f690e35a740 1 -- 172.21.15.164:0/268921421 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55726c34e550 con 0x55726c3308d0 2024-09-19T08:45:14.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.783+0000 7f68e97fa640 1 -- 172.21.15.164:0/268921421 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68fc0041d0 con 0x55726c3308d0 2024-09-19T08:45:14.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.784+0000 7f68e97fa640 1 -- 172.21.15.164:0/268921421 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f68fc004a60 con 0x55726c3308d0 2024-09-19T08:45:14.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.784+0000 7f690e35a740 10 client.15003 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T08:45:14.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.784+0000 7f690e35a740 10 client.15003 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T08:45:14.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.784+0000 7f690e35a740 20 client.15003 populate_metadata read hostname 'smithi164' 2024-09-19T08:45:14.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.784+0000 7f690e35a740 1 --2- 172.21.15.164:0/268921421 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55726c350940 0x55726c351db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:14.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.784+0000 7f690e35a740 4 client.15003 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:45:14.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.784+0000 7f690e35a740 1 -- 172.21.15.164:0/268921421 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55726c2a4600 con 0x55726c350940 2024-09-19T08:45:14.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.784+0000 7f690951c640 1 --2- 172.21.15.164:0/268921421 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55726c350940 0x55726c351db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:14.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.784+0000 7f690951c640 1 --2- 172.21.15.164:0/268921421 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55726c350940 0x55726c351db0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bf50d7adabbf33bb server_cookie=4dec807b8c1dacba in_seq=0 out_seq=0 2024-09-19T08:45:14.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.784+0000 7f68e97fa640 10 client.15003 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:45:14.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.785+0000 7f68e97fa640 1 -- 172.21.15.164:0/268921421 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55726c2a4600 con 0x55726c350940 2024-09-19T08:45:14.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.785+0000 7f68e97fa640 10 client.15003 handle_command_reply: tid=0 2024-09-19T08:45:14.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.785+0000 7f690e35a740 1 client.15003 shutdown 2024-09-19T08:45:14.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.785+0000 7f690e35a740 1 -- 172.21.15.164:0/268921421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55726c3308d0 msgr2=0x55726c33f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:14.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.785+0000 7f690e35a740 1 --2- 172.21.15.164:0/268921421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55726c3308d0 0x55726c33f800 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f68fc0114a0 tx=0x7f68fc003e20 comp rx=0 tx=0).stop 2024-09-19T08:45:14.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.785+0000 7f690e35a740 1 -- 172.21.15.164:0/268921421 shutdown_connections 2024-09-19T08:45:14.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.785+0000 7f690e35a740 1 --2- 172.21.15.164:0/268921421 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55726c350940 0x55726c351db0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.785+0000 7f690e35a740 1 --2- 172.21.15.164:0/268921421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55726c33ffc0 0x55726c33d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.785+0000 7f690e35a740 1 --2- 172.21.15.164:0/268921421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55726c3311f0 0x55726c33cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.785+0000 7f68e97fa640 0 client.15003 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:45:14.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.785+0000 7f690e35a740 1 --2- 172.21.15.164:0/268921421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55726c3308d0 0x55726c33f800 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.785+0000 7f690e35a740 1 -- 172.21.15.164:0/268921421 >> 172.21.15.164:0/268921421 conn(0x55726c32cf60 msgr2=0x55726c332cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:14.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.785+0000 7f690e35a740 1 -- 172.21.15.164:0/268921421 shutdown_connections 2024-09-19T08:45:14.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.785+0000 7f690e35a740 1 -- 172.21.15.164:0/268921421 wait complete. 2024-09-19T08:45:14.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.786+0000 7f690e35a740 20 client.15003 trim_cache size 0 max 16384 2024-09-19T08:45:14.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.803+0000 7f690e35a740 1 Processor -- start 2024-09-19T08:45:14.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.803+0000 7f690e35a740 1 -- start start 2024-09-19T08:45:14.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.803+0000 7f690e35a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55726c3308d0 0x55726c350270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:14.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.803+0000 7f690e35a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55726c3311f0 0x55726c338240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:14.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.803+0000 7f690e35a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55726c33ffc0 0x55726c339db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:14.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.803+0000 7f690e35a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55726c3500f0 con 0x55726c3308d0 2024-09-19T08:45:14.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.803+0000 7f690e35a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55726c3507c0 con 0x55726c33ffc0 2024-09-19T08:45:14.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.803+0000 7f690e35a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55726c350e90 con 0x55726c3311f0 2024-09-19T08:45:14.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.803+0000 7f6909d1d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55726c3308d0 0x55726c350270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:14.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.804+0000 7f6909d1d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55726c3308d0 0x55726c350270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52726/0 (socket says 172.21.15.164:52726) 2024-09-19T08:45:14.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.804+0000 7f6909d1d640 1 -- 172.21.15.164:0/2586098286 learned_addr learned my addr 172.21.15.164:0/2586098286 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:14.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.804+0000 7f690951c640 1 --2- 172.21.15.164:0/2586098286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55726c3311f0 0x55726c338240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:14.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.804+0000 7f690a51e640 1 --2- 172.21.15.164:0/2586098286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55726c33ffc0 0x55726c339db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:14.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.804+0000 7f6909d1d640 1 -- 172.21.15.164:0/2586098286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55726c3311f0 msgr2=0x55726c338240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:14.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.804+0000 7f6909d1d640 1 --2- 172.21.15.164:0/2586098286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55726c3311f0 0x55726c338240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.804+0000 7f6909d1d640 1 -- 172.21.15.164:0/2586098286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55726c33ffc0 msgr2=0x55726c339db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:14.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.804+0000 7f6909d1d640 1 --2- 172.21.15.164:0/2586098286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55726c33ffc0 0x55726c339db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.804+0000 7f6909d1d640 1 -- 172.21.15.164:0/2586098286 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55726c331020 con 0x55726c3308d0 2024-09-19T08:45:14.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.804+0000 7f6909d1d640 1 --2- 172.21.15.164:0/2586098286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55726c3308d0 0x55726c350270 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f68fc011010 tx=0x7f68fc0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc62823f5d2583d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:14.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.804+0000 7f68eb7fe640 1 -- 172.21.15.164:0/2586098286 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc003b50 con 0x55726c3308d0 2024-09-19T08:45:14.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.804+0000 7f68eb7fe640 1 -- 172.21.15.164:0/2586098286 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68fc033070 con 0x55726c3308d0 2024-09-19T08:45:14.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.804+0000 7f68eb7fe640 1 -- 172.21.15.164:0/2586098286 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc004680 con 0x55726c3308d0 2024-09-19T08:45:14.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.805+0000 7f690e35a740 1 -- 172.21.15.164:0/2586098286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55726c3308d0 msgr2=0x55726c350270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:14.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.805+0000 7f690e35a740 1 --2- 172.21.15.164:0/2586098286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55726c3308d0 0x55726c350270 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f68fc011010 tx=0x7f68fc0107a0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.805+0000 7f690e35a740 1 -- 172.21.15.164:0/2586098286 shutdown_connections 2024-09-19T08:45:14.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.805+0000 7f690e35a740 1 --2- 172.21.15.164:0/2586098286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55726c33ffc0 0x55726c339db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.805+0000 7f690e35a740 1 --2- 172.21.15.164:0/2586098286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55726c3311f0 0x55726c338240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.805+0000 7f690e35a740 1 --2- 172.21.15.164:0/2586098286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55726c3308d0 0x55726c350270 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.805+0000 7f690e35a740 1 -- 172.21.15.164:0/2586098286 >> 172.21.15.164:0/2586098286 conn(0x55726c32cf60 msgr2=0x55726c34f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:14.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.805+0000 7f690e35a740 1 -- 172.21.15.164:0/2586098286 shutdown_connections 2024-09-19T08:45:14.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.805+0000 7f690e35a740 1 -- 172.21.15.164:0/2586098286 wait complete. 2024-09-19T08:45:14.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.805+0000 7f690e35a740 1 -- start start 2024-09-19T08:45:14.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.805+0000 7f690e35a740 1 Processor -- start 2024-09-19T08:45:14.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.806+0000 7f690e35a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55726c3308d0 0x55726c35a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:14.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.806+0000 7f690e35a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55726c3311f0 0x55726c3539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:14.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.806+0000 7f690e35a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55726c33ffc0 0x55726c353f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:14.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.806+0000 7f690e35a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55726c351c30 con 0x55726c3311f0 2024-09-19T08:45:14.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.806+0000 7f690e35a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55726c351560 con 0x55726c33ffc0 2024-09-19T08:45:14.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.806+0000 7f690e35a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55726c352300 con 0x55726c3308d0 2024-09-19T08:45:14.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.806+0000 7f690e35a740 10 client.0 _ll_register_callbacks cb 0x7f68f4009580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:45:14.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.806+0000 7f690951c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55726c3311f0 0x55726c3539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:14.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.806+0000 7f690951c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55726c3311f0 0x55726c3539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52742/0 (socket says 172.21.15.164:52742) 2024-09-19T08:45:14.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.806+0000 7f690951c640 1 -- 172.21.15.164:0/3539324256 learned_addr learned my addr 172.21.15.164:0/3539324256 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:14.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.806+0000 7f6909d1d640 1 --2- 172.21.15.164:0/3539324256 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55726c3308d0 0x55726c35a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:14.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.806+0000 7f690a51e640 1 --2- 172.21.15.164:0/3539324256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55726c33ffc0 0x55726c353f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:14.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.806+0000 7f690951c640 1 -- 172.21.15.164:0/3539324256 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55726c3308d0 msgr2=0x55726c35a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:14.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.806+0000 7f690951c640 1 --2- 172.21.15.164:0/3539324256 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55726c3308d0 0x55726c35a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.806+0000 7f690951c640 1 -- 172.21.15.164:0/3539324256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55726c33ffc0 msgr2=0x55726c353f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:14.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.806+0000 7f690951c640 1 --2- 172.21.15.164:0/3539324256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55726c33ffc0 0x55726c353f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.806+0000 7f690951c640 1 -- 172.21.15.164:0/3539324256 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55726c3545f0 con 0x55726c3311f0 2024-09-19T08:45:14.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.807+0000 7f690951c640 1 --2- 172.21.15.164:0/3539324256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55726c3311f0 0x55726c3539c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f68f80ed880 tx=0x7f68f80f22f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcbc7a33810782c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:14.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.807+0000 7f68e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:45:14.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.807+0000 7f68e97fa640 1 -- 172.21.15.164:0/3539324256 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8049070 con 0x55726c3311f0 2024-09-19T08:45:14.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.807+0000 7f68e97fa640 1 -- 172.21.15.164:0/3539324256 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f68f80b89d0 con 0x55726c3311f0 2024-09-19T08:45:14.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.807+0000 7f690e35a740 1 -- 172.21.15.164:0/3539324256 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68fc033070 con 0x55726c3311f0 2024-09-19T08:45:14.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.807+0000 7f68e97fa640 1 -- 172.21.15.164:0/3539324256 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f68f80bca60 con 0x55726c3311f0 2024-09-19T08:45:14.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.807+0000 7f690e35a740 10 client.15009 fetch_fsmap learned FSMap version 27 2024-09-19T08:45:14.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.807+0000 7f690e35a740 1 -- 172.21.15.164:0/3539324256 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55726c3bb050 con 0x55726c3311f0 2024-09-19T08:45:14.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.807+0000 7f68e97fa640 1 -- 172.21.15.164:0/3539324256 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68fc033070 con 0x55726c3311f0 2024-09-19T08:45:14.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.807+0000 7f68e97fa640 1 -- 172.21.15.164:0/3539324256 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f68f803b460 con 0x55726c3311f0 2024-09-19T08:45:14.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.807+0000 7f690e35a740 10 client.15009 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T08:45:14.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.807+0000 7f690e35a740 10 client.15009 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T08:45:14.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.807+0000 7f690e35a740 20 client.15009 populate_metadata read hostname 'smithi164' 2024-09-19T08:45:14.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.807+0000 7f690e35a740 1 --2- 172.21.15.164:0/3539324256 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55726c3bfba0 0x55726c3be8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:14.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.807+0000 7f690e35a740 4 client.15009 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:45:14.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.807+0000 7f690e35a740 1 -- 172.21.15.164:0/3539324256 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "status"}) -- 0x55726c29eb10 con 0x55726c3bfba0 2024-09-19T08:45:14.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.807+0000 7f6909d1d640 1 --2- 172.21.15.164:0/3539324256 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55726c3bfba0 0x55726c3be8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:14.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.808+0000 7f68e97fa640 10 client.15009 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:45:14.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.808+0000 7f6909d1d640 1 --2- 172.21.15.164:0/3539324256 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55726c3bfba0 0x55726c3be8b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=434da289ae9ea3e1 server_cookie=397c5ebcc26ed7e6 in_seq=0 out_seq=0 2024-09-19T08:45:14.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.808+0000 7f68e97fa640 1 -- 172.21.15.164:0/3539324256 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55726c29eb10 con 0x55726c3bfba0 2024-09-19T08:45:14.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.808+0000 7f68e97fa640 10 client.15009 handle_command_reply: tid=0 2024-09-19T08:45:14.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.808+0000 7f690e35a740 1 client.15009 shutdown 2024-09-19T08:45:14.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.808+0000 7f690e35a740 1 -- 172.21.15.164:0/3539324256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55726c3311f0 msgr2=0x55726c3539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:14.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.808+0000 7f690e35a740 1 --2- 172.21.15.164:0/3539324256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55726c3311f0 0x55726c3539c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f68f80ed880 tx=0x7f68f80f22f0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.808+0000 7f690e35a740 1 -- 172.21.15.164:0/3539324256 shutdown_connections 2024-09-19T08:45:14.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.808+0000 7f690e35a740 1 --2- 172.21.15.164:0/3539324256 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55726c3bfba0 0x55726c3be8b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.808+0000 7f690e35a740 1 --2- 172.21.15.164:0/3539324256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55726c33ffc0 0x55726c353f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.808+0000 7f690e35a740 1 --2- 172.21.15.164:0/3539324256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55726c3311f0 0x55726c3539c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.808+0000 7f690e35a740 1 --2- 172.21.15.164:0/3539324256 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55726c3308d0 0x55726c35a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.808+0000 7f68e97fa640 0 client.15009 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:45:14.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.808+0000 7f690e35a740 1 -- 172.21.15.164:0/3539324256 >> 172.21.15.164:0/3539324256 conn(0x55726c32cf60 msgr2=0x55726c32b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:14.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.808+0000 7f690e35a740 1 -- 172.21.15.164:0/3539324256 shutdown_connections 2024-09-19T08:45:14.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.808+0000 7f690e35a740 1 -- 172.21.15.164:0/3539324256 wait complete. 2024-09-19T08:45:14.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.808+0000 7f690e35a740 20 client.15009 trim_cache size 0 max 16384 2024-09-19T08:45:14.814 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:45:14.814 INFO:teuthology.orchestra.run.smithi164.stdout: "cluster_fsid": "1e4b0d74-7662-11ef-bceb-c7b262605968", 2024-09-19T08:45:14.815 INFO:teuthology.orchestra.run.smithi164.stdout: "whoami": 1, 2024-09-19T08:45:14.815 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 24449, 2024-09-19T08:45:14.815 INFO:teuthology.orchestra.run.smithi164.stdout: "want_state": "up:active", 2024-09-19T08:45:14.815 INFO:teuthology.orchestra.run.smithi164.stdout: "state": "up:active", 2024-09-19T08:45:14.815 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs", 2024-09-19T08:45:14.815 INFO:teuthology.orchestra.run.smithi164.stdout: "rank_uptime": 74.010911023999995, 2024-09-19T08:45:14.815 INFO:teuthology.orchestra.run.smithi164.stdout: "mdsmap_epoch": 27, 2024-09-19T08:45:14.815 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch": 82, 2024-09-19T08:45:14.815 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch_barrier": 82, 2024-09-19T08:45:14.815 INFO:teuthology.orchestra.run.smithi164.stdout: "uptime": 103.983507929 2024-09-19T08:45:14.815 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:45:14.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.809+0000 7f690bf81640 1 -- 172.21.15.164:0/3863317401 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f68ec0a3350 msgr2=0x7f68ec0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:14.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.809+0000 7f690bf81640 1 --2- 172.21.15.164:0/3863317401 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f68ec0a3350 0x7f68ec0a5810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f68f4005fd0 tx=0x7f68f4004ab0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.809+0000 7f690bf81640 1 -- 172.21.15.164:0/3863317401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6904103740 msgr2=0x7f6904198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:14.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.809+0000 7f690bf81640 1 --2- 172.21.15.164:0/3863317401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6904103740 0x7f6904198b90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f68f8005940 tx=0x7f68f80042f0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.809+0000 7f690bf81640 1 -- 172.21.15.164:0/3863317401 shutdown_connections 2024-09-19T08:45:14.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.809+0000 7f690bf81640 1 --2- 172.21.15.164:0/3863317401 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f68ec0a3350 0x7f68ec0a5810 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.809+0000 7f690bf81640 1 --2- 172.21.15.164:0/3863317401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6904108290 0x7f690419cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.809+0000 7f690bf81640 1 --2- 172.21.15.164:0/3863317401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6904103740 0x7f6904198b90 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.809+0000 7f690bf81640 1 --2- 172.21.15.164:0/3863317401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6904102d90 0x7f6904198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.809+0000 7f690bf81640 1 -- 172.21.15.164:0/3863317401 >> 172.21.15.164:0/3863317401 conn(0x7f69040fc7d0 msgr2=0x7f6904105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:14.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.809+0000 7f690bf81640 1 -- 172.21.15.164:0/3863317401 shutdown_connections 2024-09-19T08:45:14.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:14.809+0000 7f690bf81640 1 -- 172.21.15.164:0/3863317401 wait complete. 2024-09-19T08:45:15.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:14 smithi164 ceph-mon[30928]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:45:15.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:14 smithi170 ceph-mon[35072]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:45:15.138 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:14 smithi173 ceph-mon[35306]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:45:15.529 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T08:45:15.776 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:45:16.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.262+0000 7f69d27c0640 1 -- 172.21.15.164:0/2446141800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69cc100e20 msgr2=0x7f69cc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:16.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.262+0000 7f69d27c0640 1 --2- 172.21.15.164:0/2446141800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69cc100e20 0x7f69cc101200 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f69bc009900 tx=0x7f69bc02a480 comp rx=0 tx=0).stop 2024-09-19T08:45:16.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.262+0000 7f69d27c0640 1 -- 172.21.15.164:0/2446141800 shutdown_connections 2024-09-19T08:45:16.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.262+0000 7f69d27c0640 1 --2- 172.21.15.164:0/2446141800 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69cc102ef0 0x7f69cc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.262+0000 7f69d27c0640 1 --2- 172.21.15.164:0/2446141800 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69cc102570 0x7f69cc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.262+0000 7f69d27c0640 1 --2- 172.21.15.164:0/2446141800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69cc100e20 0x7f69cc101200 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.262+0000 7f69d27c0640 1 -- 172.21.15.164:0/2446141800 >> 172.21.15.164:0/2446141800 conn(0x7f69cc0fc7d0 msgr2=0x7f69cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:16.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.262+0000 7f69d27c0640 1 -- 172.21.15.164:0/2446141800 shutdown_connections 2024-09-19T08:45:16.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.262+0000 7f69d27c0640 1 -- 172.21.15.164:0/2446141800 wait complete. 2024-09-19T08:45:16.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.263+0000 7f69d27c0640 1 Processor -- start 2024-09-19T08:45:16.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.263+0000 7f69d27c0640 1 -- start start 2024-09-19T08:45:16.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.263+0000 7f69d27c0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69cc100e20 0x7f69cc197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.263+0000 7f69d27c0640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69cc102570 0x7f69cc197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.263+0000 7f69d27c0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69cc102ef0 0x7f69cc19bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.263+0000 7f69d27c0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc110f50 con 0x7f69cc102ef0 2024-09-19T08:45:16.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.263+0000 7f69d27c0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc110dd0 con 0x7f69cc100e20 2024-09-19T08:45:16.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.263+0000 7f69d27c0640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc1110d0 con 0x7f69cc102570 2024-09-19T08:45:16.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.263+0000 7f69d0d5d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69cc102ef0 0x7f69cc19bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.263+0000 7f69cbfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69cc100e20 0x7f69cc197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.263+0000 7f69d0d5d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69cc102ef0 0x7f69cc19bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52758/0 (socket says 172.21.15.164:52758) 2024-09-19T08:45:16.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.263+0000 7f69d0d5d640 1 -- 172.21.15.164:0/2985231190 learned_addr learned my addr 172.21.15.164:0/2985231190 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:16.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.263+0000 7f69c3fff640 1 --2- 172.21.15.164:0/2985231190 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69cc102570 0x7f69cc197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.264+0000 7f69d0d5d640 1 -- 172.21.15.164:0/2985231190 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69cc102570 msgr2=0x7f69cc197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:16.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.264+0000 7f69d0d5d640 1 --2- 172.21.15.164:0/2985231190 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69cc102570 0x7f69cc197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.264+0000 7f69d0d5d640 1 -- 172.21.15.164:0/2985231190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69cc100e20 msgr2=0x7f69cc197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:16.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.264+0000 7f69d0d5d640 1 --2- 172.21.15.164:0/2985231190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69cc100e20 0x7f69cc197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.264+0000 7f69d0d5d640 1 -- 172.21.15.164:0/2985231190 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69cc19c2c0 con 0x7f69cc102ef0 2024-09-19T08:45:16.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.264+0000 7f69d0d5d640 1 --2- 172.21.15.164:0/2985231190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69cc102ef0 0x7f69cc19bbe0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f69c400ebb0 tx=0x7f69c4009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:16.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.264+0000 7f69c9ffb640 1 -- 172.21.15.164:0/2985231190 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c400e2e0 con 0x7f69cc102ef0 2024-09-19T08:45:16.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.264+0000 7f69c9ffb640 1 -- 172.21.15.164:0/2985231190 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69c4007950 con 0x7f69cc102ef0 2024-09-19T08:45:16.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.264+0000 7f69d27c0640 1 -- 172.21.15.164:0/2985231190 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69cc19c550 con 0x7f69cc102ef0 2024-09-19T08:45:16.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.264+0000 7f69c9ffb640 1 -- 172.21.15.164:0/2985231190 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c4005790 con 0x7f69cc102ef0 2024-09-19T08:45:16.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.264+0000 7f69d27c0640 1 -- 172.21.15.164:0/2985231190 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69cc069f70 con 0x7f69cc102ef0 2024-09-19T08:45:16.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.266+0000 7f69c9ffb640 1 -- 172.21.15.164:0/2985231190 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f69c4007b00 con 0x7f69cc102ef0 2024-09-19T08:45:16.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.267+0000 7f69d4b99740 1 Processor -- start 2024-09-19T08:45:16.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.267+0000 7f69d4b99740 1 -- start start 2024-09-19T08:45:16.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.267+0000 7f69d4b99740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56187aa059d0 0x56187aa05db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.267+0000 7f69d4b99740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56187aa062f0 0x56187aa10670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.267+0000 7f69d4b99740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56187aa10bb0 0x56187aa12fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.267+0000 7f69d4b99740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56187a96c090 con 0x56187aa10bb0 2024-09-19T08:45:16.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.267+0000 7f69d4b99740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56187a913a70 con 0x56187aa062f0 2024-09-19T08:45:16.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.267+0000 7f69d4b99740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56187a968050 con 0x56187aa059d0 2024-09-19T08:45:16.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.267+0000 7f69c9ffb640 1 --2- 172.21.15.164:0/2985231190 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f69b80a32d0 0x7f69b80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.267+0000 7f69c3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56187aa10bb0 0x56187aa12fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.267+0000 7f69c3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56187aa10bb0 0x56187aa12fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52760/0 (socket says 172.21.15.164:52760) 2024-09-19T08:45:16.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.267+0000 7f69c3fff640 1 -- 172.21.15.164:0/552827102 learned_addr learned my addr 172.21.15.164:0/552827102 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:16.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.267+0000 7f69cbfff640 1 --2- 172.21.15.164:0/552827102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56187aa062f0 0x56187aa10670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.267+0000 7f69d0d5d640 1 --2- 172.21.15.164:0/2985231190 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f69b80a32d0 0x7f69b80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.267+0000 7f69c3fff640 1 --2- 172.21.15.164:0/552827102 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56187aa059d0 0x56187aa05db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.267+0000 7f69c3fff640 1 -- 172.21.15.164:0/552827102 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56187aa059d0 msgr2=0x56187aa05db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:16.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.267+0000 7f69c3fff640 1 --2- 172.21.15.164:0/552827102 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56187aa059d0 0x56187aa05db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.267+0000 7f69c3fff640 1 -- 172.21.15.164:0/552827102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56187aa062f0 msgr2=0x56187aa10670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:16.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.267+0000 7f69c3fff640 1 --2- 172.21.15.164:0/552827102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56187aa062f0 0x56187aa10670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.267+0000 7f69c3fff640 1 -- 172.21.15.164:0/552827102 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56187a920130 con 0x56187aa10bb0 2024-09-19T08:45:16.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.267+0000 7f69c9ffb640 1 -- 172.21.15.164:0/2985231190 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f69c40d0ef0 con 0x7f69cc102ef0 2024-09-19T08:45:16.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.268+0000 7f69c3fff640 1 --2- 172.21.15.164:0/552827102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56187aa10bb0 0x56187aa12fa0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f69b4011560 tx=0x7f69b400a810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a47fdeacfc9fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:16.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.268+0000 7f69d0d5d640 1 --2- 172.21.15.164:0/2985231190 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f69b80a32d0 0x7f69b80a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f69c4070300 tx=0x7f69c4070330 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:16.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.268+0000 7f69c27fc640 1 -- 172.21.15.164:0/552827102 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b4005220 con 0x56187aa10bb0 2024-09-19T08:45:16.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.268+0000 7f69c27fc640 1 -- 172.21.15.164:0/552827102 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69b4004c80 con 0x56187aa10bb0 2024-09-19T08:45:16.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.268+0000 7f69c27fc640 1 -- 172.21.15.164:0/552827102 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b400fc50 con 0x56187aa10bb0 2024-09-19T08:45:16.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.268+0000 7f69d4b99740 1 -- 172.21.15.164:0/552827102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56187aa10bb0 msgr2=0x56187aa12fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:16.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.268+0000 7f69d4b99740 1 --2- 172.21.15.164:0/552827102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56187aa10bb0 0x56187aa12fa0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f69b4011560 tx=0x7f69b400a810 comp rx=0 tx=0).stop 2024-09-19T08:45:16.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.268+0000 7f69d4b99740 1 -- 172.21.15.164:0/552827102 shutdown_connections 2024-09-19T08:45:16.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.268+0000 7f69d4b99740 1 --2- 172.21.15.164:0/552827102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56187aa10bb0 0x56187aa12fa0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.269+0000 7f69d4b99740 1 --2- 172.21.15.164:0/552827102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56187aa062f0 0x56187aa10670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.269+0000 7f69d4b99740 1 --2- 172.21.15.164:0/552827102 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56187aa059d0 0x56187aa05db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.269+0000 7f69d4b99740 1 -- 172.21.15.164:0/552827102 >> 172.21.15.164:0/552827102 conn(0x56187aa01f60 msgr2=0x56187aa02340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:16.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.269+0000 7f69d4b99740 1 -- 172.21.15.164:0/552827102 shutdown_connections 2024-09-19T08:45:16.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.269+0000 7f69d4b99740 1 -- 172.21.15.164:0/552827102 wait complete. 2024-09-19T08:45:16.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.269+0000 7f69d4b99740 1 -- start start 2024-09-19T08:45:16.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.269+0000 7f69d4b99740 1 Processor -- start 2024-09-19T08:45:16.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.269+0000 7f69d4b99740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56187aa059d0 0x56187aa10450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.269+0000 7f69d4b99740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56187aa062f0 0x56187aa0d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.269+0000 7f69d4b99740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56187aa10bb0 0x56187aa0dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.269+0000 7f69d4b99740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56187aa17b90 con 0x56187aa10bb0 2024-09-19T08:45:16.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.269+0000 7f69d4b99740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56187a926f20 con 0x56187aa059d0 2024-09-19T08:45:16.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.269+0000 7f69d4b99740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56187aa17d10 con 0x56187aa062f0 2024-09-19T08:45:16.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.269+0000 7f69d4b99740 10 client.0 _ll_register_callbacks cb 0x56187a8f0030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:45:16.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.270+0000 7f69c3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56187aa059d0 0x56187aa10450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.270+0000 7f69c3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56187aa059d0 0x56187aa10450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:37254/0 (socket says 172.21.15.164:37254) 2024-09-19T08:45:16.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.270+0000 7f69c3fff640 1 -- 172.21.15.164:0/3799437970 learned_addr learned my addr 172.21.15.164:0/3799437970 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:16.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.270+0000 7f69c3fff640 1 --2- 172.21.15.164:0/3799437970 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56187aa10bb0 0x56187aa0dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.270+0000 7f69cbfff640 1 --2- 172.21.15.164:0/3799437970 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56187aa062f0 0x56187aa0d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.270+0000 7f69c3fff640 1 -- 172.21.15.164:0/3799437970 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56187aa062f0 msgr2=0x56187aa0d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:16.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.270+0000 7f69c3fff640 1 --2- 172.21.15.164:0/3799437970 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56187aa062f0 0x56187aa0d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.270+0000 7f69c3fff640 1 -- 172.21.15.164:0/3799437970 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56187aa10bb0 msgr2=0x56187aa0dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:16.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.270+0000 7f69c3fff640 1 --2- 172.21.15.164:0/3799437970 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56187aa10bb0 0x56187aa0dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.270+0000 7f69c3fff640 1 -- 172.21.15.164:0/3799437970 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56187a6dd2b0 con 0x56187aa059d0 2024-09-19T08:45:16.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.270+0000 7f69c3fff640 1 --2- 172.21.15.164:0/3799437970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56187aa059d0 0x56187aa10450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f69b40057c0 tx=0x7f69b400f850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1e0688ac0601cd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:16.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.270+0000 7f69ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-19T08:45:16.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.270+0000 7f69ab7fe640 1 -- 172.21.15.164:0/3799437970 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b402f060 con 0x56187aa059d0 2024-09-19T08:45:16.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.270+0000 7f69d4b99740 1 -- 172.21.15.164:0/3799437970 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69b4004c80 con 0x56187aa059d0 2024-09-19T08:45:16.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.270+0000 7f69ab7fe640 1 -- 172.21.15.164:0/3799437970 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f69b40048f0 con 0x56187aa059d0 2024-09-19T08:45:16.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.270+0000 7f69ab7fe640 1 -- 172.21.15.164:0/3799437970 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f69b4021040 con 0x56187aa059d0 2024-09-19T08:45:16.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.270+0000 7f69d4b99740 10 client.24713 fetch_fsmap learned FSMap version 27 2024-09-19T08:45:16.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.271+0000 7f69d4b99740 1 -- 172.21.15.164:0/3799437970 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56187aa234b0 con 0x56187aa059d0 2024-09-19T08:45:16.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.271+0000 7f69ab7fe640 1 -- 172.21.15.164:0/3799437970 <== mon.2 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69b4004c80 con 0x56187aa059d0 2024-09-19T08:45:16.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.271+0000 7f69ab7fe640 1 -- 172.21.15.164:0/3799437970 <== mon.2 v2:172.21.15.170:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f69b4027070 con 0x56187aa059d0 2024-09-19T08:45:16.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.271+0000 7f69d4b99740 10 client.24713 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T08:45:16.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.271+0000 7f69d4b99740 10 client.24713 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T08:45:16.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.271+0000 7f69d4b99740 20 client.24713 populate_metadata read hostname 'smithi164' 2024-09-19T08:45:16.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.271+0000 7f69d4b99740 1 --2- 172.21.15.164:0/3799437970 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56187aa25880 0x56187aa26cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.271+0000 7f69d4b99740 4 client.24713 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:45:16.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.271+0000 7f69d4b99740 1 -- 172.21.15.164:0/3799437970 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56187a979600 con 0x56187aa25880 2024-09-19T08:45:16.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.271+0000 7f69cbfff640 1 --2- 172.21.15.164:0/3799437970 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56187aa25880 0x56187aa26cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.272+0000 7f69cbfff640 1 --2- 172.21.15.164:0/3799437970 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56187aa25880 0x56187aa26cf0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c089653df68be47 server_cookie=2ac3a1e2f864a3c8 in_seq=0 out_seq=0 2024-09-19T08:45:16.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.272+0000 7f69ab7fe640 10 client.24713 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:45:16.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.272+0000 7f69ab7fe640 1 -- 172.21.15.164:0/3799437970 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56187a979600 con 0x56187aa25880 2024-09-19T08:45:16.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.272+0000 7f69ab7fe640 10 client.24713 handle_command_reply: tid=0 2024-09-19T08:45:16.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.272+0000 7f69d4b99740 1 client.24713 shutdown 2024-09-19T08:45:16.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.272+0000 7f69d4b99740 1 -- 172.21.15.164:0/3799437970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56187aa059d0 msgr2=0x56187aa10450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:16.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.272+0000 7f69d4b99740 1 --2- 172.21.15.164:0/3799437970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56187aa059d0 0x56187aa10450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f69b40057c0 tx=0x7f69b400f850 comp rx=0 tx=0).stop 2024-09-19T08:45:16.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.272+0000 7f69d4b99740 1 -- 172.21.15.164:0/3799437970 shutdown_connections 2024-09-19T08:45:16.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.272+0000 7f69d4b99740 1 --2- 172.21.15.164:0/3799437970 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56187aa10bb0 0x56187aa0dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.272+0000 7f69d4b99740 1 --2- 172.21.15.164:0/3799437970 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56187aa062f0 0x56187aa0d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.272+0000 7f69d4b99740 1 --2- 172.21.15.164:0/3799437970 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56187aa25880 0x56187aa26cf0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.272+0000 7f69d4b99740 1 --2- 172.21.15.164:0/3799437970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56187aa059d0 0x56187aa10450 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.273+0000 7f69ab7fe640 0 client.24713 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:45:16.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.273+0000 7f69d4b99740 1 -- 172.21.15.164:0/3799437970 >> 172.21.15.164:0/3799437970 conn(0x56187aa01f60 msgr2=0x56187aa07c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:16.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.273+0000 7f69d4b99740 1 -- 172.21.15.164:0/3799437970 shutdown_connections 2024-09-19T08:45:16.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.273+0000 7f69d4b99740 1 -- 172.21.15.164:0/3799437970 wait complete. 2024-09-19T08:45:16.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.273+0000 7f69d4b99740 20 client.24713 trim_cache size 0 max 16384 2024-09-19T08:45:16.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.290+0000 7f69d4b99740 1 Processor -- start 2024-09-19T08:45:16.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.290+0000 7f69d4b99740 1 -- start start 2024-09-19T08:45:16.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.290+0000 7f69d4b99740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56187aa059d0 0x56187aa26cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.290+0000 7f69d4b99740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56187aa062f0 0x56187aa0c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.290+0000 7f69d4b99740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56187aa10bb0 0x56187aa0c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.290+0000 7f69d4b99740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56187aa25030 con 0x56187aa062f0 2024-09-19T08:45:16.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.290+0000 7f69d4b99740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56187aa25700 con 0x56187aa059d0 2024-09-19T08:45:16.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.290+0000 7f69d4b99740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56187aa25dd0 con 0x56187aa10bb0 2024-09-19T08:45:16.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.290+0000 7f69cbfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56187aa062f0 0x56187aa0c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.290+0000 7f69cbfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56187aa062f0 0x56187aa0c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52778/0 (socket says 172.21.15.164:52778) 2024-09-19T08:45:16.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.291+0000 7f69cbfff640 1 -- 172.21.15.164:0/2609358038 learned_addr learned my addr 172.21.15.164:0/2609358038 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:16.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.291+0000 7f69c3fff640 1 --2- 172.21.15.164:0/2609358038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56187aa059d0 0x56187aa26cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.291+0000 7f69c3fff640 1 --2- 172.21.15.164:0/2609358038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56187aa10bb0 0x56187aa0c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.291+0000 7f69cbfff640 1 -- 172.21.15.164:0/2609358038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56187aa10bb0 msgr2=0x56187aa0c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:16.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.291+0000 7f69cbfff640 1 --2- 172.21.15.164:0/2609358038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56187aa10bb0 0x56187aa0c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.291+0000 7f69cbfff640 1 -- 172.21.15.164:0/2609358038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56187aa059d0 msgr2=0x56187aa26cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:16.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.291+0000 7f69cbfff640 1 --2- 172.21.15.164:0/2609358038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56187aa059d0 0x56187aa26cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.291+0000 7f69cbfff640 1 -- 172.21.15.164:0/2609358038 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56187aa06120 con 0x56187aa062f0 2024-09-19T08:45:16.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.291+0000 7f69c3fff640 1 --2- 172.21.15.164:0/2609358038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56187aa10bb0 0x56187aa0c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:16.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.291+0000 7f69cbfff640 1 --2- 172.21.15.164:0/2609358038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56187aa062f0 0x56187aa0c070 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f69bc0314e0 tx=0x7f69bc004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628feda22436442c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:16.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.291+0000 7f69c1ffb640 1 -- 172.21.15.164:0/2609358038 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc04a020 con 0x56187aa062f0 2024-09-19T08:45:16.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.291+0000 7f69c1ffb640 1 -- 172.21.15.164:0/2609358038 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69bc0047a0 con 0x56187aa062f0 2024-09-19T08:45:16.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.291+0000 7f69c1ffb640 1 -- 172.21.15.164:0/2609358038 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc035040 con 0x56187aa062f0 2024-09-19T08:45:16.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.292+0000 7f69d4b99740 1 -- 172.21.15.164:0/2609358038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56187aa062f0 msgr2=0x56187aa0c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:16.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.292+0000 7f69d4b99740 1 --2- 172.21.15.164:0/2609358038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56187aa062f0 0x56187aa0c070 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f69bc0314e0 tx=0x7f69bc004530 comp rx=0 tx=0).stop 2024-09-19T08:45:16.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.292+0000 7f69d4b99740 1 -- 172.21.15.164:0/2609358038 shutdown_connections 2024-09-19T08:45:16.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.292+0000 7f69d4b99740 1 --2- 172.21.15.164:0/2609358038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56187aa10bb0 0x56187aa0c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.292+0000 7f69d4b99740 1 --2- 172.21.15.164:0/2609358038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56187aa062f0 0x56187aa0c070 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.292+0000 7f69d4b99740 1 --2- 172.21.15.164:0/2609358038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56187aa059d0 0x56187aa26cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.292+0000 7f69d4b99740 1 -- 172.21.15.164:0/2609358038 >> 172.21.15.164:0/2609358038 conn(0x56187aa01f60 msgr2=0x56187aa0d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:16.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.292+0000 7f69d4b99740 1 -- 172.21.15.164:0/2609358038 shutdown_connections 2024-09-19T08:45:16.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.292+0000 7f69d4b99740 1 -- 172.21.15.164:0/2609358038 wait complete. 2024-09-19T08:45:16.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.292+0000 7f69d4b99740 1 -- start start 2024-09-19T08:45:16.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.292+0000 7f69d4b99740 1 Processor -- start 2024-09-19T08:45:16.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.293+0000 7f69d4b99740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56187aa059d0 0x56187aa2a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.293+0000 7f69d4b99740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56187aa062f0 0x56187aa2a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.293+0000 7f69d4b99740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56187aa10bb0 0x56187aa8df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.293+0000 7f69d4b99740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56187aa26b70 con 0x56187aa059d0 2024-09-19T08:45:16.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.293+0000 7f69d4b99740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56187aa264a0 con 0x56187aa10bb0 2024-09-19T08:45:16.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.293+0000 7f69d4b99740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56187aa27240 con 0x56187aa062f0 2024-09-19T08:45:16.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.293+0000 7f69d4b99740 10 client.0 _ll_register_callbacks cb 0x7f69bc009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:45:16.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.293+0000 7f69c3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56187aa059d0 0x56187aa2a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.293+0000 7f69c3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56187aa059d0 0x56187aa2a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52782/0 (socket says 172.21.15.164:52782) 2024-09-19T08:45:16.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.293+0000 7f69c3fff640 1 -- 172.21.15.164:0/3776078972 learned_addr learned my addr 172.21.15.164:0/3776078972 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:16.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.293+0000 7f69cbfff640 1 --2- 172.21.15.164:0/3776078972 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56187aa062f0 0x56187aa2a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.293+0000 7f69c3fff640 1 --2- 172.21.15.164:0/3776078972 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56187aa10bb0 0x56187aa8df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.293+0000 7f69c3fff640 1 -- 172.21.15.164:0/3776078972 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56187aa062f0 msgr2=0x56187aa2a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:16.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.293+0000 7f69c3fff640 1 --2- 172.21.15.164:0/3776078972 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56187aa062f0 0x56187aa2a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.293+0000 7f69c3fff640 1 -- 172.21.15.164:0/3776078972 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56187aa10bb0 msgr2=0x56187aa8df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:16.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.293+0000 7f69c3fff640 1 --2- 172.21.15.164:0/3776078972 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56187aa10bb0 0x56187aa8df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.293+0000 7f69c3fff640 1 -- 172.21.15.164:0/3776078972 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56187aa2ae80 con 0x56187aa059d0 2024-09-19T08:45:16.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.293+0000 7f69c3fff640 1 --2- 172.21.15.164:0/3776078972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56187aa059d0 0x56187aa2a060 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f69b400aeb0 tx=0x7f69b400f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16f5d9687187852 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:16.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.293+0000 7f69ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:45:16.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.293+0000 7f69ab7fe640 1 -- 172.21.15.164:0/3776078972 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b4039070 con 0x56187aa059d0 2024-09-19T08:45:16.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.293+0000 7f69ab7fe640 1 -- 172.21.15.164:0/3776078972 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f69b401c5f0 con 0x56187aa059d0 2024-09-19T08:45:16.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.293+0000 7f69d4b99740 1 -- 172.21.15.164:0/3776078972 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69bc0047a0 con 0x56187aa059d0 2024-09-19T08:45:16.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.293+0000 7f69ab7fe640 1 -- 172.21.15.164:0/3776078972 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f69b4047050 con 0x56187aa059d0 2024-09-19T08:45:16.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.294+0000 7f69d4b99740 10 client.15027 fetch_fsmap learned FSMap version 27 2024-09-19T08:45:16.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.294+0000 7f69d4b99740 1 -- 172.21.15.164:0/3776078972 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56187aa8f540 con 0x56187aa059d0 2024-09-19T08:45:16.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.294+0000 7f69ab7fe640 1 -- 172.21.15.164:0/3776078972 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69bc0047a0 con 0x56187aa059d0 2024-09-19T08:45:16.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.294+0000 7f69ab7fe640 1 -- 172.21.15.164:0/3776078972 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f69b4027070 con 0x56187aa059d0 2024-09-19T08:45:16.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.294+0000 7f69d4b99740 10 client.15027 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T08:45:16.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.294+0000 7f69d4b99740 10 client.15027 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T08:45:16.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.294+0000 7f69d4b99740 20 client.15027 populate_metadata read hostname 'smithi164' 2024-09-19T08:45:16.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.294+0000 7f69d4b99740 1 --2- 172.21.15.164:0/3776078972 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56187aa940a0 0x56187aa92db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.294+0000 7f69d4b99740 4 client.15027 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:45:16.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.294+0000 7f69d4b99740 1 -- 172.21.15.164:0/3776078972 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "status"}) -- 0x56187a973b10 con 0x56187aa940a0 2024-09-19T08:45:16.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.294+0000 7f69cbfff640 1 --2- 172.21.15.164:0/3776078972 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56187aa940a0 0x56187aa92db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.295+0000 7f69ab7fe640 10 client.15027 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:45:16.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.295+0000 7f69cbfff640 1 --2- 172.21.15.164:0/3776078972 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56187aa940a0 0x56187aa92db0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=763f80b2ad388e48 server_cookie=cae58119976abe7b in_seq=0 out_seq=0 2024-09-19T08:45:16.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.295+0000 7f69ab7fe640 1 -- 172.21.15.164:0/3776078972 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56187a973b10 con 0x56187aa940a0 2024-09-19T08:45:16.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.295+0000 7f69ab7fe640 10 client.15027 handle_command_reply: tid=0 2024-09-19T08:45:16.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.295+0000 7f69d4b99740 1 client.15027 shutdown 2024-09-19T08:45:16.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.295+0000 7f69d4b99740 1 -- 172.21.15.164:0/3776078972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56187aa059d0 msgr2=0x56187aa2a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:16.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.295+0000 7f69d4b99740 1 --2- 172.21.15.164:0/3776078972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56187aa059d0 0x56187aa2a060 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f69b400aeb0 tx=0x7f69b400f7b0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.295+0000 7f69d4b99740 1 -- 172.21.15.164:0/3776078972 shutdown_connections 2024-09-19T08:45:16.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.295+0000 7f69d4b99740 1 --2- 172.21.15.164:0/3776078972 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56187aa940a0 0x56187aa92db0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.295+0000 7f69d4b99740 1 --2- 172.21.15.164:0/3776078972 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56187aa10bb0 0x56187aa8df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.295+0000 7f69d4b99740 1 --2- 172.21.15.164:0/3776078972 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56187aa062f0 0x56187aa2a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.295+0000 7f69ab7fe640 0 client.15027 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:45:16.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.295+0000 7f69d4b99740 1 --2- 172.21.15.164:0/3776078972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56187aa059d0 0x56187aa2a060 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.295+0000 7f69d4b99740 1 -- 172.21.15.164:0/3776078972 >> 172.21.15.164:0/3776078972 conn(0x56187aa01f60 msgr2=0x56187aa0c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:16.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.295+0000 7f69d4b99740 1 -- 172.21.15.164:0/3776078972 shutdown_connections 2024-09-19T08:45:16.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.295+0000 7f69d4b99740 1 -- 172.21.15.164:0/3776078972 wait complete. 2024-09-19T08:45:16.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.295+0000 7f69d4b99740 20 client.15027 trim_cache size 0 max 16384 2024-09-19T08:45:16.303 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:45:16.303 INFO:teuthology.orchestra.run.smithi164.stdout: "cluster_fsid": "1e4b0d74-7662-11ef-bceb-c7b262605968", 2024-09-19T08:45:16.303 INFO:teuthology.orchestra.run.smithi164.stdout: "whoami": 0, 2024-09-19T08:45:16.303 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 24479, 2024-09-19T08:45:16.303 INFO:teuthology.orchestra.run.smithi164.stdout: "want_state": "up:active", 2024-09-19T08:45:16.303 INFO:teuthology.orchestra.run.smithi164.stdout: "state": "up:active", 2024-09-19T08:45:16.303 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs", 2024-09-19T08:45:16.303 INFO:teuthology.orchestra.run.smithi164.stdout: "rank_uptime": 82.532935194000004, 2024-09-19T08:45:16.303 INFO:teuthology.orchestra.run.smithi164.stdout: "mdsmap_epoch": 27, 2024-09-19T08:45:16.304 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch": 82, 2024-09-19T08:45:16.304 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch_barrier": 82, 2024-09-19T08:45:16.304 INFO:teuthology.orchestra.run.smithi164.stdout: "uptime": 94.247190899000003 2024-09-19T08:45:16.304 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:45:16.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.296+0000 7f69d27c0640 1 -- 172.21.15.164:0/2985231190 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f69b80a32d0 msgr2=0x7f69b80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:16.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.296+0000 7f69d27c0640 1 --2- 172.21.15.164:0/2985231190 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f69b80a32d0 0x7f69b80a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f69c4070300 tx=0x7f69c4070330 comp rx=0 tx=0).stop 2024-09-19T08:45:16.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.296+0000 7f69d27c0640 1 -- 172.21.15.164:0/2985231190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69cc102ef0 msgr2=0x7f69cc19bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:16.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.296+0000 7f69d27c0640 1 --2- 172.21.15.164:0/2985231190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69cc102ef0 0x7f69cc19bbe0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f69c400ebb0 tx=0x7f69c4009f60 comp rx=0 tx=0).stop 2024-09-19T08:45:16.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.296+0000 7f69d27c0640 1 -- 172.21.15.164:0/2985231190 shutdown_connections 2024-09-19T08:45:16.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.296+0000 7f69d27c0640 1 --2- 172.21.15.164:0/2985231190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69cc102ef0 0x7f69cc19bbe0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.296+0000 7f69d27c0640 1 --2- 172.21.15.164:0/2985231190 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f69b80a32d0 0x7f69b80a5790 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.296+0000 7f69d27c0640 1 --2- 172.21.15.164:0/2985231190 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69cc102570 0x7f69cc197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.296+0000 7f69d27c0640 1 --2- 172.21.15.164:0/2985231190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69cc100e20 0x7f69cc197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.296+0000 7f69d27c0640 1 -- 172.21.15.164:0/2985231190 >> 172.21.15.164:0/2985231190 conn(0x7f69cc0fc7d0 msgr2=0x7f69cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:16.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.296+0000 7f69d27c0640 1 -- 172.21.15.164:0/2985231190 shutdown_connections 2024-09-19T08:45:16.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:16.296+0000 7f69d27c0640 1 -- 172.21.15.164:0/2985231190 wait complete. 2024-09-19T08:45:17.012 INFO:tasks.cephadm:Setup complete, yielding 2024-09-19T08:45:17.013 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T08:45:17.023 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi164.front.sepia.ceph.com 2024-09-19T08:45:17.023 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-19T08:45:17.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:16 smithi170 ceph-mon[35072]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T08:45:17.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:16 smithi173 ceph-mon[35306]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T08:45:17.269 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:45:17.281 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:16 smithi164 ceph-mon[30928]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T08:45:17.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.721+0000 7f71e3651640 1 -- 172.21.15.164:0/977004616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71dc1037c0 msgr2=0x7f71dc10a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:17.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.721+0000 7f71e3651640 1 --2- 172.21.15.164:0/977004616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71dc1037c0 0x7f71dc10a050 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f71d4009900 tx=0x7f71d402a410 comp rx=0 tx=0).stop 2024-09-19T08:45:17.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.721+0000 7f71e3651640 1 -- 172.21.15.164:0/977004616 shutdown_connections 2024-09-19T08:45:17.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.721+0000 7f71e3651640 1 --2- 172.21.15.164:0/977004616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71dc1037c0 0x7f71dc10a050 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:17.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.721+0000 7f71e3651640 1 --2- 172.21.15.164:0/977004616 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71dc102e00 0x7f71dc103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:17.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.721+0000 7f71e3651640 1 --2- 172.21.15.164:0/977004616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71dc101c00 0x7f71dc102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:17.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.721+0000 7f71e3651640 1 -- 172.21.15.164:0/977004616 >> 172.21.15.164:0/977004616 conn(0x7f71dc0fd390 msgr2=0x7f71dc0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:17.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.721+0000 7f71e3651640 1 -- 172.21.15.164:0/977004616 shutdown_connections 2024-09-19T08:45:17.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.721+0000 7f71e3651640 1 -- 172.21.15.164:0/977004616 wait complete. 2024-09-19T08:45:17.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.722+0000 7f71e3651640 1 Processor -- start 2024-09-19T08:45:17.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.722+0000 7f71e3651640 1 -- start start 2024-09-19T08:45:17.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.722+0000 7f71e3651640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71dc101c00 0x7f71dc075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:17.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.722+0000 7f71e3651640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71dc102e00 0x7f71dc0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:17.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.722+0000 7f71e3651640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71dc1037c0 0x7f71dc074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:17.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.722+0000 7f71e3651640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc10d880 con 0x7f71dc1037c0 2024-09-19T08:45:17.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.722+0000 7f71e3651640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc10d700 con 0x7f71dc101c00 2024-09-19T08:45:17.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.722+0000 7f71e3651640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc10da00 con 0x7f71dc102e00 2024-09-19T08:45:17.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.722+0000 7f71e1bee640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71dc1037c0 0x7f71dc074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:17.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.722+0000 7f71e1bee640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71dc1037c0 0x7f71dc074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52808/0 (socket says 172.21.15.164:52808) 2024-09-19T08:45:17.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.722+0000 7f71e1bee640 1 -- 172.21.15.164:0/697148115 learned_addr learned my addr 172.21.15.164:0/697148115 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:17.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.722+0000 7f71e0bec640 1 --2- 172.21.15.164:0/697148115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71dc102e00 0x7f71dc0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:17.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.722+0000 7f71e13ed640 1 --2- 172.21.15.164:0/697148115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71dc101c00 0x7f71dc075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:17.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.722+0000 7f71e1bee640 1 -- 172.21.15.164:0/697148115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71dc102e00 msgr2=0x7f71dc0745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:17.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.722+0000 7f71e1bee640 1 --2- 172.21.15.164:0/697148115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71dc102e00 0x7f71dc0745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:17.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.722+0000 7f71e1bee640 1 -- 172.21.15.164:0/697148115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71dc101c00 msgr2=0x7f71dc075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:17.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.722+0000 7f71e1bee640 1 --2- 172.21.15.164:0/697148115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71dc101c00 0x7f71dc075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:17.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.722+0000 7f71e1bee640 1 -- 172.21.15.164:0/697148115 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71dc0753e0 con 0x7f71dc1037c0 2024-09-19T08:45:17.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.723+0000 7f71e1bee640 1 --2- 172.21.15.164:0/697148115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71dc1037c0 0x7f71dc074b20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f71d40314e0 tx=0x7f71d4004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:17.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.723+0000 7f71da7fc640 1 -- 172.21.15.164:0/697148115 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d403f030 con 0x7f71dc1037c0 2024-09-19T08:45:17.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.723+0000 7f71da7fc640 1 -- 172.21.15.164:0/697148115 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71d4003ce0 con 0x7f71dc1037c0 2024-09-19T08:45:17.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.723+0000 7f71e3651640 1 -- 172.21.15.164:0/697148115 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71dc1a1fd0 con 0x7f71dc1037c0 2024-09-19T08:45:17.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.723+0000 7f71da7fc640 1 -- 172.21.15.164:0/697148115 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d4035070 con 0x7f71dc1037c0 2024-09-19T08:45:17.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.723+0000 7f71e3651640 1 -- 172.21.15.164:0/697148115 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71dc1a2420 con 0x7f71dc1037c0 2024-09-19T08:45:17.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.724+0000 7f71e3651640 1 -- 172.21.15.164:0/697148115 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71dc108f80 con 0x7f71dc1037c0 2024-09-19T08:45:17.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.725+0000 7f71da7fc640 1 -- 172.21.15.164:0/697148115 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f71d4042030 con 0x7f71dc1037c0 2024-09-19T08:45:17.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.726+0000 7f71da7fc640 1 --2- 172.21.15.164:0/697148115 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f71c40a32d0 0x7f71c40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:17.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.726+0000 7f71e13ed640 1 --2- 172.21.15.164:0/697148115 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f71c40a32d0 0x7f71c40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:17.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.726+0000 7f71da7fc640 1 -- 172.21.15.164:0/697148115 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f71d40f0320 con 0x7f71dc1037c0 2024-09-19T08:45:17.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.726+0000 7f71e13ed640 1 --2- 172.21.15.164:0/697148115 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f71c40a32d0 0x7f71c40a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00a300 tx=0x7f71cc00a290 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:17.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.730+0000 7f71da7fc640 1 -- 172.21.15.164:0/697148115 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f71d40bcfe0 con 0x7f71dc1037c0 2024-09-19T08:45:17.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.915+0000 7f71e3651640 1 -- 172.21.15.164:0/697148115 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f71dc062630 con 0x7f71c40a32d0 2024-09-19T08:45:17.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.916+0000 7f71da7fc640 1 -- 172.21.15.164:0/697148115 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f71dc062630 con 0x7f71c40a32d0 2024-09-19T08:45:17.915 INFO:teuthology.orchestra.run.smithi164.stdout:Backend: cephadm 2024-09-19T08:45:17.915 INFO:teuthology.orchestra.run.smithi164.stdout:Available: Yes 2024-09-19T08:45:17.915 INFO:teuthology.orchestra.run.smithi164.stdout:Paused: No 2024-09-19T08:45:17.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.918+0000 7f71e3651640 1 -- 172.21.15.164:0/697148115 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f71c40a32d0 msgr2=0x7f71c40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:17.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.918+0000 7f71e3651640 1 --2- 172.21.15.164:0/697148115 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f71c40a32d0 0x7f71c40a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00a300 tx=0x7f71cc00a290 comp rx=0 tx=0).stop 2024-09-19T08:45:17.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.918+0000 7f71e3651640 1 -- 172.21.15.164:0/697148115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71dc1037c0 msgr2=0x7f71dc074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:17.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.918+0000 7f71e3651640 1 --2- 172.21.15.164:0/697148115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71dc1037c0 0x7f71dc074b20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f71d40314e0 tx=0x7f71d4004360 comp rx=0 tx=0).stop 2024-09-19T08:45:17.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.918+0000 7f71e3651640 1 -- 172.21.15.164:0/697148115 shutdown_connections 2024-09-19T08:45:17.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.918+0000 7f71e3651640 1 --2- 172.21.15.164:0/697148115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71dc1037c0 0x7f71dc074b20 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:17.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.918+0000 7f71e3651640 1 --2- 172.21.15.164:0/697148115 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f71c40a32d0 0x7f71c40a5790 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:17.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.918+0000 7f71e3651640 1 --2- 172.21.15.164:0/697148115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71dc102e00 0x7f71dc0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:17.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.918+0000 7f71e3651640 1 --2- 172.21.15.164:0/697148115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71dc101c00 0x7f71dc075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:17.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.918+0000 7f71e3651640 1 -- 172.21.15.164:0/697148115 >> 172.21.15.164:0/697148115 conn(0x7f71dc0fd390 msgr2=0x7f71dc106150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:17.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.918+0000 7f71e3651640 1 -- 172.21.15.164:0/697148115 shutdown_connections 2024-09-19T08:45:17.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:17.918+0000 7f71e3651640 1 -- 172.21.15.164:0/697148115 wait complete. 2024-09-19T08:45:18.688 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-19T08:45:18.938 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:45:19.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:18 smithi164 ceph-mon[30928]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-19T08:45:19.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:18 smithi170 ceph-mon[35072]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-19T08:45:19.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:18 smithi173 ceph-mon[35306]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-19T08:45:19.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.412+0000 7fe4a4025640 1 -- 172.21.15.164:0/905005518 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe49c105030 msgr2=0x7fe49c107420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:19.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.412+0000 7fe4a4025640 1 --2- 172.21.15.164:0/905005518 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe49c105030 0x7fe49c107420 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe48c009900 tx=0x7fe48c02a410 comp rx=0 tx=0).stop 2024-09-19T08:45:19.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.412+0000 7fe4a4025640 1 -- 172.21.15.164:0/905005518 shutdown_connections 2024-09-19T08:45:19.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.412+0000 7fe4a4025640 1 --2- 172.21.15.164:0/905005518 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe49c107960 0x7fe49c109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:19.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.412+0000 7fe4a4025640 1 --2- 172.21.15.164:0/905005518 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe49c105030 0x7fe49c107420 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:19.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.412+0000 7fe4a4025640 1 --2- 172.21.15.164:0/905005518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe49c102700 0x7fe49c104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:19.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.412+0000 7fe4a4025640 1 -- 172.21.15.164:0/905005518 >> 172.21.15.164:0/905005518 conn(0x7fe49c0fc280 msgr2=0x7fe49c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:19.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.412+0000 7fe4a4025640 1 -- 172.21.15.164:0/905005518 shutdown_connections 2024-09-19T08:45:19.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.412+0000 7fe4a4025640 1 -- 172.21.15.164:0/905005518 wait complete. 2024-09-19T08:45:19.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.413+0000 7fe4a4025640 1 Processor -- start 2024-09-19T08:45:19.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.413+0000 7fe4a4025640 1 -- start start 2024-09-19T08:45:19.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.413+0000 7fe4a4025640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe49c102700 0x7fe49c102060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:19.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.413+0000 7fe4a4025640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe49c105030 0x7fe49c1006b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:19.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.413+0000 7fe4a4025640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe49c107960 0x7fe49c100bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:19.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.413+0000 7fe4a4025640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c10d930 con 0x7fe49c102700 2024-09-19T08:45:19.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.413+0000 7fe4a4025640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c10d7b0 con 0x7fe49c105030 2024-09-19T08:45:19.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.413+0000 7fe4a4025640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c10dab0 con 0x7fe49c107960 2024-09-19T08:45:19.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.413+0000 7fe4a1dc1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe49c102700 0x7fe49c102060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:19.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.413+0000 7fe4a1dc1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe49c102700 0x7fe49c102060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52822/0 (socket says 172.21.15.164:52822) 2024-09-19T08:45:19.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.413+0000 7fe4a1dc1640 1 -- 172.21.15.164:0/4021644281 learned_addr learned my addr 172.21.15.164:0/4021644281 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:19.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.413+0000 7fe4a25c2640 1 --2- 172.21.15.164:0/4021644281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe49c107960 0x7fe49c100bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:19.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.413+0000 7fe4a15c0640 1 --2- 172.21.15.164:0/4021644281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe49c105030 0x7fe49c1006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:19.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.413+0000 7fe4a1dc1640 1 -- 172.21.15.164:0/4021644281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe49c107960 msgr2=0x7fe49c100bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:19.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.413+0000 7fe4a1dc1640 1 --2- 172.21.15.164:0/4021644281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe49c107960 0x7fe49c100bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:19.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.413+0000 7fe4a1dc1640 1 -- 172.21.15.164:0/4021644281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe49c105030 msgr2=0x7fe49c1006b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:19.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.413+0000 7fe4a1dc1640 1 --2- 172.21.15.164:0/4021644281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe49c105030 0x7fe49c1006b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:19.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.413+0000 7fe4a1dc1640 1 -- 172.21.15.164:0/4021644281 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49c1014b0 con 0x7fe49c102700 2024-09-19T08:45:19.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.413+0000 7fe4a1dc1640 1 --2- 172.21.15.164:0/4021644281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe49c102700 0x7fe49c102060 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe49800b4a0 tx=0x7fe49800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:19.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.414+0000 7fe492ffd640 1 -- 172.21.15.164:0/4021644281 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe498014070 con 0x7fe49c102700 2024-09-19T08:45:19.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.414+0000 7fe4a4025640 1 -- 172.21.15.164:0/4021644281 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe49c1a21e0 con 0x7fe49c102700 2024-09-19T08:45:19.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.414+0000 7fe492ffd640 1 -- 172.21.15.164:0/4021644281 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe498010af0 con 0x7fe49c102700 2024-09-19T08:45:19.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.414+0000 7fe492ffd640 1 -- 172.21.15.164:0/4021644281 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe498017d10 con 0x7fe49c102700 2024-09-19T08:45:19.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.414+0000 7fe4a4025640 1 -- 172.21.15.164:0/4021644281 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe49c1a2770 con 0x7fe49c102700 2024-09-19T08:45:19.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.415+0000 7fe4a4025640 1 -- 172.21.15.164:0/4021644281 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe49c0fdd70 con 0x7fe49c102700 2024-09-19T08:45:19.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.416+0000 7fe492ffd640 1 -- 172.21.15.164:0/4021644281 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fe498017070 con 0x7fe49c102700 2024-09-19T08:45:19.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.416+0000 7fe492ffd640 1 --2- 172.21.15.164:0/4021644281 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fe4880a32d0 0x7fe4880a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:19.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.417+0000 7fe4a15c0640 1 --2- 172.21.15.164:0/4021644281 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fe4880a32d0 0x7fe4880a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:19.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.417+0000 7fe492ffd640 1 -- 172.21.15.164:0/4021644281 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe4980ceb50 con 0x7fe49c102700 2024-09-19T08:45:19.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.417+0000 7fe4a15c0640 1 --2- 172.21.15.164:0/4021644281 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fe4880a32d0 0x7fe4880a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe48c005ab0 tx=0x7fe48c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:19.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.421+0000 7fe492ffd640 1 -- 172.21.15.164:0/4021644281 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe49809b2d0 con 0x7fe49c102700 2024-09-19T08:45:19.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.597+0000 7fe4a4025640 1 -- 172.21.15.164:0/4021644281 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fe49c062630 con 0x7fe4880a32d0 2024-09-19T08:45:19.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.609+0000 7fe492ffd640 1 -- 172.21.15.164:0/4021644281 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7fe49c062630 con 0x7fe4880a32d0 2024-09-19T08:45:19.609 INFO:teuthology.orchestra.run.smithi164.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-19T08:45:19.609 INFO:teuthology.orchestra.run.smithi164.stdout:mds.a smithi164 running (100s) 21s ago 100s 19.7M - 19.1.1-361-g2f599a46 d2baa855c067 29a373f4a0e4 2024-09-19T08:45:19.609 INFO:teuthology.orchestra.run.smithi164.stdout:mds.b smithi170 running (97s) 31s ago 97s 26.6M - 19.1.1-361-g2f599a46 d2baa855c067 c4e1796a1740 2024-09-19T08:45:19.609 INFO:teuthology.orchestra.run.smithi164.stdout:mds.c smithi173 running (111s) 31s ago 111s 18.0M - 19.1.1-361-g2f599a46 d2baa855c067 8658563522b0 2024-09-19T08:45:19.609 INFO:teuthology.orchestra.run.smithi164.stdout:mds.d smithi164 running (119s) 21s ago 119s 14.4M - 19.1.1-361-g2f599a46 d2baa855c067 fcf4f129a2b0 2024-09-19T08:45:19.609 INFO:teuthology.orchestra.run.smithi164.stdout:mds.e smithi170 running (116s) 31s ago 116s 19.9M - 19.1.1-361-g2f599a46 d2baa855c067 68372a737eac 2024-09-19T08:45:19.610 INFO:teuthology.orchestra.run.smithi164.stdout:mds.f smithi173 running (114s) 31s ago 113s 16.7M - 19.1.1-361-g2f599a46 d2baa855c067 d799c13ea99f 2024-09-19T08:45:19.610 INFO:teuthology.orchestra.run.smithi164.stdout:mds.g smithi164 running (103s) 21s ago 103s 17.0M - 19.1.1-361-g2f599a46 d2baa855c067 085df45388ee 2024-09-19T08:45:19.610 INFO:teuthology.orchestra.run.smithi164.stdout:mds.h smithi170 running (109s) 31s ago 108s 18.4M - 19.1.1-361-g2f599a46 d2baa855c067 e9a4945d27ce 2024-09-19T08:45:19.610 INFO:teuthology.orchestra.run.smithi164.stdout:mds.i smithi173 running (106s) 31s ago 106s 17.9M - 19.1.1-361-g2f599a46 d2baa855c067 06eaf68a19e0 2024-09-19T08:45:19.610 INFO:teuthology.orchestra.run.smithi164.stdout:mds.j smithi164 running (2m) 21s ago 2m 14.5M - 19.1.1-361-g2f599a46 d2baa855c067 d5aaa7f97ba0 2024-09-19T08:45:19.610 INFO:teuthology.orchestra.run.smithi164.stdout:mds.k smithi170 running (2m) 31s ago 2m 19.0M - 19.1.1-361-g2f599a46 d2baa855c067 23dd67483033 2024-09-19T08:45:19.610 INFO:teuthology.orchestra.run.smithi164.stdout:mds.l smithi173 running (2m) 31s ago 2m 18.3M - 19.1.1-361-g2f599a46 d2baa855c067 d7e3e84b8555 2024-09-19T08:45:19.610 INFO:teuthology.orchestra.run.smithi164.stdout:mgr.x smithi164 *:9283,8765 running (9m) 21s ago 9m 510M - 19.1.1-361-g2f599a46 d2baa855c067 3287a3ff4ddf 2024-09-19T08:45:19.610 INFO:teuthology.orchestra.run.smithi164.stdout:mgr.y smithi170 *:8443,8765 running (7m) 31s ago 7m 444M - 19.1.1-361-g2f599a46 d2baa855c067 5b8dfecde34d 2024-09-19T08:45:19.610 INFO:teuthology.orchestra.run.smithi164.stdout:mgr.z smithi173 *:8443,8765 running (7m) 31s ago 7m 441M - 19.1.1-361-g2f599a46 d2baa855c067 a9e0ddc90b8e 2024-09-19T08:45:19.610 INFO:teuthology.orchestra.run.smithi164.stdout:mon.a smithi164 running (9m) 21s ago 9m 50.8M 2048M 19.1.1-361-g2f599a46 d2baa855c067 3264bf1a15c8 2024-09-19T08:45:19.610 INFO:teuthology.orchestra.run.smithi164.stdout:mon.b smithi170 running (7m) 31s ago 7m 39.9M 2048M 19.1.1-361-g2f599a46 d2baa855c067 746ea7b73d55 2024-09-19T08:45:19.610 INFO:teuthology.orchestra.run.smithi164.stdout:mon.c smithi173 running (7m) 31s ago 7m 42.0M 2048M 19.1.1-361-g2f599a46 d2baa855c067 e5ca074777f9 2024-09-19T08:45:19.611 INFO:teuthology.orchestra.run.smithi164.stdout:osd.0 smithi164 running (7m) 21s ago 7m 57.6M 4272M 19.1.1-361-g2f599a46 d2baa855c067 8bea16b48f18 2024-09-19T08:45:19.611 INFO:teuthology.orchestra.run.smithi164.stdout:osd.1 smithi170 running (6m) 31s ago 6m 56.0M 4272M 19.1.1-361-g2f599a46 d2baa855c067 333615f94397 2024-09-19T08:45:19.611 INFO:teuthology.orchestra.run.smithi164.stdout:osd.2 smithi173 running (6m) 31s ago 6m 60.3M 4272M 19.1.1-361-g2f599a46 d2baa855c067 be62044fbceb 2024-09-19T08:45:19.611 INFO:teuthology.orchestra.run.smithi164.stdout:osd.3 smithi164 running (5m) 21s ago 5m 43.6M 4272M 19.1.1-361-g2f599a46 d2baa855c067 1dae88379a9c 2024-09-19T08:45:19.611 INFO:teuthology.orchestra.run.smithi164.stdout:osd.4 smithi170 running (5m) 31s ago 5m 43.7M 4272M 19.1.1-361-g2f599a46 d2baa855c067 4879674e2cb2 2024-09-19T08:45:19.611 INFO:teuthology.orchestra.run.smithi164.stdout:osd.5 smithi173 running (4m) 31s ago 4m 58.0M 4272M 19.1.1-361-g2f599a46 d2baa855c067 7d2b7b7911e8 2024-09-19T08:45:19.611 INFO:teuthology.orchestra.run.smithi164.stdout:osd.6 smithi164 running (4m) 21s ago 4m 54.0M 4272M 19.1.1-361-g2f599a46 d2baa855c067 39dc4d6a1c02 2024-09-19T08:45:19.611 INFO:teuthology.orchestra.run.smithi164.stdout:osd.7 smithi170 running (4m) 31s ago 4m 55.2M 4272M 19.1.1-361-g2f599a46 d2baa855c067 cb4500d4e572 2024-09-19T08:45:19.611 INFO:teuthology.orchestra.run.smithi164.stdout:osd.8 smithi173 running (3m) 31s ago 3m 53.6M 4272M 19.1.1-361-g2f599a46 d2baa855c067 adb4883fa29d 2024-09-19T08:45:19.611 INFO:teuthology.orchestra.run.smithi164.stdout:osd.9 smithi164 running (3m) 21s ago 3m 42.0M 4272M 19.1.1-361-g2f599a46 d2baa855c067 787e37fd935b 2024-09-19T08:45:19.612 INFO:teuthology.orchestra.run.smithi164.stdout:osd.10 smithi170 running (2m) 31s ago 2m 55.1M 4272M 19.1.1-361-g2f599a46 d2baa855c067 b960d27b5343 2024-09-19T08:45:19.612 INFO:teuthology.orchestra.run.smithi164.stdout:osd.11 smithi173 running (2m) 31s ago 2m 51.6M 4272M 19.1.1-361-g2f599a46 d2baa855c067 7b0299d2d0bd 2024-09-19T08:45:19.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.611+0000 7fe4a4025640 1 -- 172.21.15.164:0/4021644281 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fe4880a32d0 msgr2=0x7fe4880a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:19.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.611+0000 7fe4a4025640 1 --2- 172.21.15.164:0/4021644281 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fe4880a32d0 0x7fe4880a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe48c005ab0 tx=0x7fe48c000bc0 comp rx=0 tx=0).stop 2024-09-19T08:45:19.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.611+0000 7fe4a4025640 1 -- 172.21.15.164:0/4021644281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe49c102700 msgr2=0x7fe49c102060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:19.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.611+0000 7fe4a4025640 1 --2- 172.21.15.164:0/4021644281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe49c102700 0x7fe49c102060 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe49800b4a0 tx=0x7fe49800b960 comp rx=0 tx=0).stop 2024-09-19T08:45:19.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.612+0000 7fe4a4025640 1 -- 172.21.15.164:0/4021644281 shutdown_connections 2024-09-19T08:45:19.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.612+0000 7fe4a4025640 1 --2- 172.21.15.164:0/4021644281 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fe4880a32d0 0x7fe4880a5790 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:19.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.612+0000 7fe4a4025640 1 --2- 172.21.15.164:0/4021644281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe49c107960 0x7fe49c100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:19.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.612+0000 7fe4a4025640 1 --2- 172.21.15.164:0/4021644281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe49c105030 0x7fe49c1006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:19.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.612+0000 7fe4a4025640 1 --2- 172.21.15.164:0/4021644281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe49c102700 0x7fe49c102060 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:19.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.612+0000 7fe4a4025640 1 -- 172.21.15.164:0/4021644281 >> 172.21.15.164:0/4021644281 conn(0x7fe49c0fc280 msgr2=0x7fe49c103200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:19.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.612+0000 7fe4a4025640 1 -- 172.21.15.164:0/4021644281 shutdown_connections 2024-09-19T08:45:19.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:19.612+0000 7fe4a4025640 1 -- 172.21.15.164:0/4021644281 wait complete. 2024-09-19T08:45:19.919 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:19 smithi164 ceph-mon[30928]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:45:20.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:19 smithi170 ceph-mon[35072]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:45:20.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:19 smithi173 ceph-mon[35306]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:45:20.297 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-19T08:45:20.541 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:45:20.940 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:20 smithi164 ceph-mon[30928]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:45:20.941 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:20 smithi164 ceph-mon[30928]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:45:21.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.021+0000 7f6c6e8ec640 1 -- 172.21.15.164:0/154242405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c681037b0 msgr2=0x7f6c6810a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:21.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.021+0000 7f6c6e8ec640 1 --2- 172.21.15.164:0/154242405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c681037b0 0x7f6c6810a040 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6c60009ca0 tx=0x7f6c6002a8c0 comp rx=0 tx=0).stop 2024-09-19T08:45:21.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.022+0000 7f6c6e8ec640 1 -- 172.21.15.164:0/154242405 shutdown_connections 2024-09-19T08:45:21.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.022+0000 7f6c6e8ec640 1 --2- 172.21.15.164:0/154242405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c681037b0 0x7f6c6810a040 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:21.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.022+0000 7f6c6e8ec640 1 --2- 172.21.15.164:0/154242405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c68102df0 0x7f6c68103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:21.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.022+0000 7f6c6e8ec640 1 --2- 172.21.15.164:0/154242405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c68101bf0 0x7f6c68101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:21.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.022+0000 7f6c6e8ec640 1 -- 172.21.15.164:0/154242405 >> 172.21.15.164:0/154242405 conn(0x7f6c680fd3c0 msgr2=0x7f6c680ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:21.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.022+0000 7f6c6e8ec640 1 -- 172.21.15.164:0/154242405 shutdown_connections 2024-09-19T08:45:21.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.022+0000 7f6c6e8ec640 1 -- 172.21.15.164:0/154242405 wait complete. 2024-09-19T08:45:21.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.022+0000 7f6c6e8ec640 1 Processor -- start 2024-09-19T08:45:21.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.022+0000 7f6c6e8ec640 1 -- start start 2024-09-19T08:45:21.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.022+0000 7f6c6e8ec640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c68101bf0 0x7f6c68197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:21.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.022+0000 7f6c6e8ec640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c68102df0 0x7f6c68198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:21.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.022+0000 7f6c6e8ec640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c681037b0 0x7f6c6819f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:21.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.022+0000 7f6c6e8ec640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6810d9c0 con 0x7f6c68101bf0 2024-09-19T08:45:21.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.022+0000 7f6c6e8ec640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6810d840 con 0x7f6c681037b0 2024-09-19T08:45:21.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.022+0000 7f6c6e8ec640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6810db40 con 0x7f6c68102df0 2024-09-19T08:45:21.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.022+0000 7f6c67fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c68101bf0 0x7f6c68197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:21.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.023+0000 7f6c67fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c68101bf0 0x7f6c68197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52846/0 (socket says 172.21.15.164:52846) 2024-09-19T08:45:21.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.023+0000 7f6c67fff640 1 -- 172.21.15.164:0/2846948936 learned_addr learned my addr 172.21.15.164:0/2846948936 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:21.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.023+0000 7f6c6ce89640 1 --2- 172.21.15.164:0/2846948936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c681037b0 0x7f6c6819f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:21.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.023+0000 7f6c5ffff640 1 --2- 172.21.15.164:0/2846948936 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c68102df0 0x7f6c68198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:21.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.023+0000 7f6c67fff640 1 -- 172.21.15.164:0/2846948936 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c68102df0 msgr2=0x7f6c68198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:21.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.023+0000 7f6c67fff640 1 --2- 172.21.15.164:0/2846948936 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c68102df0 0x7f6c68198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:21.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.023+0000 7f6c67fff640 1 -- 172.21.15.164:0/2846948936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c681037b0 msgr2=0x7f6c6819f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:21.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.023+0000 7f6c67fff640 1 --2- 172.21.15.164:0/2846948936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c681037b0 0x7f6c6819f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:21.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.023+0000 7f6c67fff640 1 -- 172.21.15.164:0/2846948936 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c6819fc20 con 0x7f6c68101bf0 2024-09-19T08:45:21.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.023+0000 7f6c5ffff640 1 --2- 172.21.15.164:0/2846948936 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c68102df0 0x7f6c68198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:21.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.023+0000 7f6c67fff640 1 --2- 172.21.15.164:0/2846948936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c68101bf0 0x7f6c68197ff0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6c5800b450 tx=0x7f6c5800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:21.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.023+0000 7f6c65ffb640 1 -- 172.21.15.164:0/2846948936 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c58015020 con 0x7f6c68101bf0 2024-09-19T08:45:21.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.023+0000 7f6c65ffb640 1 -- 172.21.15.164:0/2846948936 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c58011d70 con 0x7f6c68101bf0 2024-09-19T08:45:21.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.023+0000 7f6c6e8ec640 1 -- 172.21.15.164:0/2846948936 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c6819ff10 con 0x7f6c68101bf0 2024-09-19T08:45:21.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.023+0000 7f6c65ffb640 1 -- 172.21.15.164:0/2846948936 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5800f340 con 0x7f6c68101bf0 2024-09-19T08:45:21.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.023+0000 7f6c6e8ec640 1 -- 172.21.15.164:0/2846948936 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c681a0390 con 0x7f6c68101bf0 2024-09-19T08:45:21.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.025+0000 7f6c6e8ec640 1 -- 172.21.15.164:0/2846948936 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c68108fa0 con 0x7f6c68101bf0 2024-09-19T08:45:21.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.026+0000 7f6c65ffb640 1 -- 172.21.15.164:0/2846948936 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f6c58018020 con 0x7f6c68101bf0 2024-09-19T08:45:21.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.026+0000 7f6c65ffb640 1 --2- 172.21.15.164:0/2846948936 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f6c540a3300 0x7f6c540a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:21.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.026+0000 7f6c5ffff640 1 --2- 172.21.15.164:0/2846948936 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f6c540a3300 0x7f6c540a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:21.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.026+0000 7f6c65ffb640 1 -- 172.21.15.164:0/2846948936 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f6c580cf920 con 0x7f6c68101bf0 2024-09-19T08:45:21.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.027+0000 7f6c5ffff640 1 --2- 172.21.15.164:0/2846948936 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f6c540a3300 0x7f6c540a57c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6c50006fd0 tx=0x7f6c50008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:21.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.030+0000 7f6c65ffb640 1 -- 172.21.15.164:0/2846948936 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6c5801f020 con 0x7f6c68101bf0 2024-09-19T08:45:21.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:20 smithi170 ceph-mon[35072]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:45:21.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:20 smithi170 ceph-mon[35072]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:45:21.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:20 smithi173 ceph-mon[35306]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:45:21.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:20 smithi173 ceph-mon[35306]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:45:21.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.204+0000 7f6c6e8ec640 1 -- 172.21.15.164:0/2846948936 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f6c68062630 con 0x7f6c540a3300 2024-09-19T08:45:21.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.210+0000 7f6c65ffb640 1 -- 172.21.15.164:0/2846948936 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f6c68062630 con 0x7f6c540a3300 2024-09-19T08:45:21.209 INFO:teuthology.orchestra.run.smithi164.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-19T08:45:21.209 INFO:teuthology.orchestra.run.smithi164.stdout:mds.all 12/12 33s ago 2m smithi164=a;smithi164=d;smithi164=g;smithi164=j;smithi170=b;smithi170=e;smithi170=h;smithi170=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l;count:12 2024-09-19T08:45:21.209 INFO:teuthology.orchestra.run.smithi164.stdout:mgr 3/3 33s ago 7m smithi164=x;smithi170=y;smithi173=z;count:3 2024-09-19T08:45:21.210 INFO:teuthology.orchestra.run.smithi164.stdout:mon 3/3 33s ago 7m smithi164:172.21.15.164=a;smithi170:172.21.15.170=b;smithi173:172.21.15.173=c;count:3 2024-09-19T08:45:21.210 INFO:teuthology.orchestra.run.smithi164.stdout:osd 12 33s ago - 2024-09-19T08:45:21.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.212+0000 7f6c6e8ec640 1 -- 172.21.15.164:0/2846948936 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f6c540a3300 msgr2=0x7f6c540a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:21.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.212+0000 7f6c6e8ec640 1 --2- 172.21.15.164:0/2846948936 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f6c540a3300 0x7f6c540a57c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6c50006fd0 tx=0x7f6c50008040 comp rx=0 tx=0).stop 2024-09-19T08:45:21.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.212+0000 7f6c6e8ec640 1 -- 172.21.15.164:0/2846948936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c68101bf0 msgr2=0x7f6c68197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:21.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.212+0000 7f6c6e8ec640 1 --2- 172.21.15.164:0/2846948936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c68101bf0 0x7f6c68197ff0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6c5800b450 tx=0x7f6c5800b910 comp rx=0 tx=0).stop 2024-09-19T08:45:21.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.212+0000 7f6c6e8ec640 1 -- 172.21.15.164:0/2846948936 shutdown_connections 2024-09-19T08:45:21.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.212+0000 7f6c6e8ec640 1 --2- 172.21.15.164:0/2846948936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c681037b0 0x7f6c6819f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:21.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.212+0000 7f6c6e8ec640 1 --2- 172.21.15.164:0/2846948936 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f6c540a3300 0x7f6c540a57c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:21.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.212+0000 7f6c6e8ec640 1 --2- 172.21.15.164:0/2846948936 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c68102df0 0x7f6c68198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:21.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.212+0000 7f6c6e8ec640 1 --2- 172.21.15.164:0/2846948936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c68101bf0 0x7f6c68197ff0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:21.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.212+0000 7f6c6e8ec640 1 -- 172.21.15.164:0/2846948936 >> 172.21.15.164:0/2846948936 conn(0x7f6c680fd3c0 msgr2=0x7f6c68106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:21.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.212+0000 7f6c6e8ec640 1 -- 172.21.15.164:0/2846948936 shutdown_connections 2024-09-19T08:45:21.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:21.212+0000 7f6c6e8ec640 1 -- 172.21.15.164:0/2846948936 wait complete. 2024-09-19T08:45:21.902 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-19T08:45:22.150 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:45:22.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.640+0000 7f5bbca4a640 1 -- 172.21.15.164:0/1535811428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bb8104ff0 msgr2=0x7f5bb81073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:22.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.640+0000 7f5bbca4a640 1 --2- 172.21.15.164:0/1535811428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bb8104ff0 0x7f5bb81073e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009900 tx=0x7f5bac02a410 comp rx=0 tx=0).stop 2024-09-19T08:45:22.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.640+0000 7f5bbca4a640 1 -- 172.21.15.164:0/1535811428 shutdown_connections 2024-09-19T08:45:22.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.640+0000 7f5bbca4a640 1 --2- 172.21.15.164:0/1535811428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bb8107920 0x7f5bb8109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:22.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.640+0000 7f5bbca4a640 1 --2- 172.21.15.164:0/1535811428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bb8104ff0 0x7f5bb81073e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:22.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.640+0000 7f5bbca4a640 1 --2- 172.21.15.164:0/1535811428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5bb81026c0 0x7f5bb8104ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:22.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.640+0000 7f5bbca4a640 1 -- 172.21.15.164:0/1535811428 >> 172.21.15.164:0/1535811428 conn(0x7f5bb80fc280 msgr2=0x7f5bb80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:22.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.640+0000 7f5bbca4a640 1 -- 172.21.15.164:0/1535811428 shutdown_connections 2024-09-19T08:45:22.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.640+0000 7f5bbca4a640 1 -- 172.21.15.164:0/1535811428 wait complete. 2024-09-19T08:45:22.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.641+0000 7f5bbca4a640 1 Processor -- start 2024-09-19T08:45:22.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.641+0000 7f5bbca4a640 1 -- start start 2024-09-19T08:45:22.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.641+0000 7f5bbca4a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5bb81026c0 0x7f5bb81980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:22.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.641+0000 7f5bbca4a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bb8104ff0 0x7f5bb81985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:22.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.641+0000 7f5bbca4a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bb8107920 0x7f5bb819f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:22.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.641+0000 7f5bbca4a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb810d930 con 0x7f5bb8107920 2024-09-19T08:45:22.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.641+0000 7f5bbca4a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb810d7b0 con 0x7f5bb81026c0 2024-09-19T08:45:22.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.641+0000 7f5bbca4a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb810dab0 con 0x7f5bb8104ff0 2024-09-19T08:45:22.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.641+0000 7f5bb6d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bb8107920 0x7f5bb819f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:22.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.641+0000 7f5bb6d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bb8107920 0x7f5bb819f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52866/0 (socket says 172.21.15.164:52866) 2024-09-19T08:45:22.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.641+0000 7f5bb6d9d640 1 -- 172.21.15.164:0/705879708 learned_addr learned my addr 172.21.15.164:0/705879708 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:22.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.641+0000 7f5bb659c640 1 --2- 172.21.15.164:0/705879708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5bb81026c0 0x7f5bb81980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:22.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.641+0000 7f5bb5d9b640 1 --2- 172.21.15.164:0/705879708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bb8104ff0 0x7f5bb81985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:22.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.641+0000 7f5bb6d9d640 1 -- 172.21.15.164:0/705879708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bb8104ff0 msgr2=0x7f5bb81985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:22.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.641+0000 7f5bb6d9d640 1 --2- 172.21.15.164:0/705879708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bb8104ff0 0x7f5bb81985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:22.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.641+0000 7f5bb6d9d640 1 -- 172.21.15.164:0/705879708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5bb81026c0 msgr2=0x7f5bb81980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:22.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.641+0000 7f5bb6d9d640 1 --2- 172.21.15.164:0/705879708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5bb81026c0 0x7f5bb81980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:22.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.641+0000 7f5bb6d9d640 1 -- 172.21.15.164:0/705879708 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb819fcd0 con 0x7f5bb8107920 2024-09-19T08:45:22.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.642+0000 7f5bb6d9d640 1 --2- 172.21.15.164:0/705879708 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bb8107920 0x7f5bb819f660 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000b970 tx=0x7f5bb000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:22.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.642+0000 7f5ba77fe640 1 -- 172.21.15.164:0/705879708 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0019070 con 0x7f5bb8107920 2024-09-19T08:45:22.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.642+0000 7f5ba77fe640 1 -- 172.21.15.164:0/705879708 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bb0014440 con 0x7f5bb8107920 2024-09-19T08:45:22.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.642+0000 7f5ba77fe640 1 -- 172.21.15.164:0/705879708 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb00075d0 con 0x7f5bb8107920 2024-09-19T08:45:22.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.642+0000 7f5bbca4a640 1 -- 172.21.15.164:0/705879708 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bb819ffc0 con 0x7f5bb8107920 2024-09-19T08:45:22.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.642+0000 7f5bbca4a640 1 -- 172.21.15.164:0/705879708 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bb81a0540 con 0x7f5bb8107920 2024-09-19T08:45:22.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.643+0000 7f5bbca4a640 1 -- 172.21.15.164:0/705879708 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bb80fdd70 con 0x7f5bb8107920 2024-09-19T08:45:22.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.644+0000 7f5ba77fe640 1 -- 172.21.15.164:0/705879708 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f5bb001c070 con 0x7f5bb8107920 2024-09-19T08:45:22.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.644+0000 7f5ba77fe640 1 --2- 172.21.15.164:0/705879708 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f5ba00a3380 0x7f5ba00a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:22.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.644+0000 7f5bb659c640 1 --2- 172.21.15.164:0/705879708 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f5ba00a3380 0x7f5ba00a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:22.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.645+0000 7f5ba77fe640 1 -- 172.21.15.164:0/705879708 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f5bb00cedc0 con 0x7f5bb8107920 2024-09-19T08:45:22.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.645+0000 7f5bb659c640 1 --2- 172.21.15.164:0/705879708 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f5ba00a3380 0x7f5ba00a5840 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8009990 tx=0x7f5ba8009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:22.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.648+0000 7f5ba77fe640 1 -- 172.21.15.164:0/705879708 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5bb009bb90 con 0x7f5bb8107920 2024-09-19T08:45:22.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.823+0000 7f5bbca4a640 1 -- 172.21.15.164:0/705879708 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f5bb8062630 con 0x7f5ba00a3380 2024-09-19T08:45:22.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.824+0000 7f5ba77fe640 1 -- 172.21.15.164:0/705879708 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f5bb8062630 con 0x7f5ba00a3380 2024-09-19T08:45:22.824 INFO:teuthology.orchestra.run.smithi164.stdout:HOST ADDR LABELS STATUS 2024-09-19T08:45:22.824 INFO:teuthology.orchestra.run.smithi164.stdout:smithi164 172.21.15.164 2024-09-19T08:45:22.824 INFO:teuthology.orchestra.run.smithi164.stdout:smithi170 172.21.15.170 2024-09-19T08:45:22.824 INFO:teuthology.orchestra.run.smithi164.stdout:smithi173 172.21.15.173 2024-09-19T08:45:22.824 INFO:teuthology.orchestra.run.smithi164.stdout:3 hosts in cluster 2024-09-19T08:45:22.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.826+0000 7f5bbca4a640 1 -- 172.21.15.164:0/705879708 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f5ba00a3380 msgr2=0x7f5ba00a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:22.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.826+0000 7f5bbca4a640 1 --2- 172.21.15.164:0/705879708 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f5ba00a3380 0x7f5ba00a5840 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8009990 tx=0x7f5ba8009110 comp rx=0 tx=0).stop 2024-09-19T08:45:22.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.826+0000 7f5bbca4a640 1 -- 172.21.15.164:0/705879708 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bb8107920 msgr2=0x7f5bb819f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:22.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.826+0000 7f5bbca4a640 1 --2- 172.21.15.164:0/705879708 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bb8107920 0x7f5bb819f660 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000b970 tx=0x7f5bb000bd70 comp rx=0 tx=0).stop 2024-09-19T08:45:22.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.827+0000 7f5bbca4a640 1 -- 172.21.15.164:0/705879708 shutdown_connections 2024-09-19T08:45:22.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.827+0000 7f5bbca4a640 1 --2- 172.21.15.164:0/705879708 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bb8107920 0x7f5bb819f660 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:22.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.827+0000 7f5bbca4a640 1 --2- 172.21.15.164:0/705879708 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f5ba00a3380 0x7f5ba00a5840 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:22.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.827+0000 7f5bbca4a640 1 --2- 172.21.15.164:0/705879708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bb8104ff0 0x7f5bb81985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:22.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.827+0000 7f5bbca4a640 1 --2- 172.21.15.164:0/705879708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5bb81026c0 0x7f5bb81980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:22.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.827+0000 7f5bbca4a640 1 -- 172.21.15.164:0/705879708 >> 172.21.15.164:0/705879708 conn(0x7f5bb80fc280 msgr2=0x7f5bb81030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:22.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.827+0000 7f5bbca4a640 1 -- 172.21.15.164:0/705879708 shutdown_connections 2024-09-19T08:45:22.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:22.827+0000 7f5bbca4a640 1 -- 172.21.15.164:0/705879708 wait complete. 2024-09-19T08:45:23.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:22 smithi170 ceph-mon[35072]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:45:23.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:22 smithi170 ceph-mon[35072]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T08:45:23.147 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:22 smithi164 ceph-mon[30928]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:45:23.147 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:22 smithi164 ceph-mon[30928]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T08:45:23.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:22 smithi173 ceph-mon[35306]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:45:23.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:22 smithi173 ceph-mon[35306]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T08:45:23.500 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-19T08:45:23.746 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:45:24.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:23 smithi170 ceph-mon[35072]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:45:24.151 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:23 smithi164 ceph-mon[30928]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:45:24.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:23 smithi173 ceph-mon[35306]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:45:24.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.233+0000 7fb0ed97a640 1 -- 172.21.15.164:0/3140301533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0e8107900 msgr2=0x7fb0e8109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:24.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.233+0000 7fb0ed97a640 1 --2- 172.21.15.164:0/3140301533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0e8107900 0x7fb0e8109d50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0009900 tx=0x7fb0e002a410 comp rx=0 tx=0).stop 2024-09-19T08:45:24.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.233+0000 7fb0ed97a640 1 -- 172.21.15.164:0/3140301533 shutdown_connections 2024-09-19T08:45:24.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.233+0000 7fb0ed97a640 1 --2- 172.21.15.164:0/3140301533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0e8107900 0x7fb0e8109d50 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:24.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.234+0000 7fb0ed97a640 1 --2- 172.21.15.164:0/3140301533 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0e8104fd0 0x7fb0e81073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:24.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.234+0000 7fb0ed97a640 1 --2- 172.21.15.164:0/3140301533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0e81026a0 0x7fb0e8104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:24.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.234+0000 7fb0ed97a640 1 -- 172.21.15.164:0/3140301533 >> 172.21.15.164:0/3140301533 conn(0x7fb0e80fc2a0 msgr2=0x7fb0e80fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:24.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.234+0000 7fb0ed97a640 1 -- 172.21.15.164:0/3140301533 shutdown_connections 2024-09-19T08:45:24.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.234+0000 7fb0ed97a640 1 -- 172.21.15.164:0/3140301533 wait complete. 2024-09-19T08:45:24.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.234+0000 7fb0ed97a640 1 Processor -- start 2024-09-19T08:45:24.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.234+0000 7fb0ed97a640 1 -- start start 2024-09-19T08:45:24.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.234+0000 7fb0ed97a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0e81026a0 0x7fb0e8197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:24.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.234+0000 7fb0ed97a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0e8104fd0 0x7fb0e8198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:24.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.234+0000 7fb0ed97a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0e8107900 0x7fb0e819f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:24.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.234+0000 7fb0ed97a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e810d8c0 con 0x7fb0e8107900 2024-09-19T08:45:24.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.234+0000 7fb0ed97a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e810d740 con 0x7fb0e81026a0 2024-09-19T08:45:24.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.235+0000 7fb0ed97a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e810da40 con 0x7fb0e8104fd0 2024-09-19T08:45:24.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.235+0000 7fb0e77fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0e8107900 0x7fb0e819f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:24.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.235+0000 7fb0e77fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0e8107900 0x7fb0e819f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59784/0 (socket says 172.21.15.164:59784) 2024-09-19T08:45:24.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.235+0000 7fb0e77fe640 1 -- 172.21.15.164:0/2883607787 learned_addr learned my addr 172.21.15.164:0/2883607787 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:24.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.235+0000 7fb0e6ffd640 1 --2- 172.21.15.164:0/2883607787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0e81026a0 0x7fb0e8197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:24.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.235+0000 7fb0e67fc640 1 --2- 172.21.15.164:0/2883607787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0e8104fd0 0x7fb0e8198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:24.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.235+0000 7fb0e77fe640 1 -- 172.21.15.164:0/2883607787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0e8104fd0 msgr2=0x7fb0e8198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:24.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.235+0000 7fb0e77fe640 1 --2- 172.21.15.164:0/2883607787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0e8104fd0 0x7fb0e8198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:24.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.235+0000 7fb0e77fe640 1 -- 172.21.15.164:0/2883607787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0e81026a0 msgr2=0x7fb0e8197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:24.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.235+0000 7fb0e77fe640 1 --2- 172.21.15.164:0/2883607787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0e81026a0 0x7fb0e8197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:24.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.235+0000 7fb0e77fe640 1 -- 172.21.15.164:0/2883607787 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0e819faf0 con 0x7fb0e8107900 2024-09-19T08:45:24.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.235+0000 7fb0e77fe640 1 --2- 172.21.15.164:0/2883607787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0e8107900 0x7fb0e819f480 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb0e00314e0 tx=0x7fb0e0004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:24.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.235+0000 7fb0ec978640 1 -- 172.21.15.164:0/2883607787 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e003f030 con 0x7fb0e8107900 2024-09-19T08:45:24.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.235+0000 7fb0ed97a640 1 -- 172.21.15.164:0/2883607787 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0e819fd80 con 0x7fb0e8107900 2024-09-19T08:45:24.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.235+0000 7fb0ec978640 1 -- 172.21.15.164:0/2883607787 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0e0003ce0 con 0x7fb0e8107900 2024-09-19T08:45:24.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.235+0000 7fb0ec978640 1 -- 172.21.15.164:0/2883607787 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0035070 con 0x7fb0e8107900 2024-09-19T08:45:24.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.235+0000 7fb0ed97a640 1 -- 172.21.15.164:0/2883607787 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0e81a0200 con 0x7fb0e8107900 2024-09-19T08:45:24.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.236+0000 7fb0ed97a640 1 -- 172.21.15.164:0/2883607787 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0e80fdd90 con 0x7fb0e8107900 2024-09-19T08:45:24.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.237+0000 7fb0ec978640 1 -- 172.21.15.164:0/2883607787 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fb0e0042030 con 0x7fb0e8107900 2024-09-19T08:45:24.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.238+0000 7fb0ec978640 1 --2- 172.21.15.164:0/2883607787 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb0d00a3300 0x7fb0d00a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:24.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.238+0000 7fb0e6ffd640 1 --2- 172.21.15.164:0/2883607787 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb0d00a3300 0x7fb0d00a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:24.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.238+0000 7fb0ec978640 1 -- 172.21.15.164:0/2883607787 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb0e00effa0 con 0x7fb0e8107900 2024-09-19T08:45:24.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.238+0000 7fb0e6ffd640 1 --2- 172.21.15.164:0/2883607787 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb0d00a3300 0x7fb0d00a57c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8005fd0 tx=0x7fb0d8009760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:24.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.242+0000 7fb0ec978640 1 -- 172.21.15.164:0/2883607787 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb0e00bcc60 con 0x7fb0e8107900 2024-09-19T08:45:24.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.416+0000 7fb0ed97a640 1 -- 172.21.15.164:0/2883607787 --> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fb0e8062630 con 0x7fb0d00a3300 2024-09-19T08:45:24.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.417+0000 7fb0ec978640 1 -- 172.21.15.164:0/2883607787 <== mgr.14150 v2:172.21.15.164:6800/1094864383 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fb0e8062630 con 0x7fb0d00a3300 2024-09-19T08:45:24.417 INFO:teuthology.orchestra.run.smithi164.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-19T08:45:24.417 INFO:teuthology.orchestra.run.smithi164.stdout:smithi164 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300H0400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-19T08:45:24.417 INFO:teuthology.orchestra.run.smithi164.stdout:smithi170 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476002R480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-09-19T08:45:24.417 INFO:teuthology.orchestra.run.smithi164.stdout:smithi173 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204019S400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-19T08:45:24.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.419+0000 7fb0ed97a640 1 -- 172.21.15.164:0/2883607787 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb0d00a3300 msgr2=0x7fb0d00a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:24.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.419+0000 7fb0ed97a640 1 --2- 172.21.15.164:0/2883607787 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb0d00a3300 0x7fb0d00a57c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8005fd0 tx=0x7fb0d8009760 comp rx=0 tx=0).stop 2024-09-19T08:45:24.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.419+0000 7fb0ed97a640 1 -- 172.21.15.164:0/2883607787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0e8107900 msgr2=0x7fb0e819f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:24.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.419+0000 7fb0ed97a640 1 --2- 172.21.15.164:0/2883607787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0e8107900 0x7fb0e819f480 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb0e00314e0 tx=0x7fb0e0004360 comp rx=0 tx=0).stop 2024-09-19T08:45:24.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.420+0000 7fb0ed97a640 1 -- 172.21.15.164:0/2883607787 shutdown_connections 2024-09-19T08:45:24.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.420+0000 7fb0ed97a640 1 --2- 172.21.15.164:0/2883607787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0e8107900 0x7fb0e819f480 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:24.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.420+0000 7fb0ed97a640 1 --2- 172.21.15.164:0/2883607787 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb0d00a3300 0x7fb0d00a57c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:24.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.420+0000 7fb0ed97a640 1 --2- 172.21.15.164:0/2883607787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0e8104fd0 0x7fb0e8198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:24.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.420+0000 7fb0ed97a640 1 --2- 172.21.15.164:0/2883607787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0e81026a0 0x7fb0e8197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:24.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.420+0000 7fb0ed97a640 1 -- 172.21.15.164:0/2883607787 >> 172.21.15.164:0/2883607787 conn(0x7fb0e80fc2a0 msgr2=0x7fb0e8102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:24.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.420+0000 7fb0ed97a640 1 -- 172.21.15.164:0/2883607787 shutdown_connections 2024-09-19T08:45:24.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:24.420+0000 7fb0ed97a640 1 -- 172.21.15.164:0/2883607787 wait complete. 2024-09-19T08:45:25.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:24 smithi170 ceph-mon[35072]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:45:25.131 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:24 smithi164 ceph-mon[30928]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:45:25.159 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T08:45:25.168 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi164.front.sepia.ceph.com 2024-09-19T08:45:25.168 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-19T08:45:25.169 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:24 smithi173 ceph-mon[35306]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:45:25.413 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:45:25.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:25 smithi173 ceph-mon[35306]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:45:25.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.901+0000 7f1509f13640 1 -- 172.21.15.164:0/4021316089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1504101d10 msgr2=0x7f1504102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:25.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.901+0000 7f1509f13640 1 --2- 172.21.15.164:0/4021316089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1504101d10 0x7f1504102190 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009900 tx=0x7f14f802a410 comp rx=0 tx=0).stop 2024-09-19T08:45:25.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.901+0000 7f1509f13640 1 -- 172.21.15.164:0/4021316089 shutdown_connections 2024-09-19T08:45:25.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.901+0000 7f1509f13640 1 --2- 172.21.15.164:0/4021316089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15041026d0 0x7f1504108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:25.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.901+0000 7f1509f13640 1 --2- 172.21.15.164:0/4021316089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1504101d10 0x7f1504102190 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:25.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.901+0000 7f1509f13640 1 --2- 172.21.15.164:0/4021316089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1504100b10 0x7f1504100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:25.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.901+0000 7f1509f13640 1 -- 172.21.15.164:0/4021316089 >> 172.21.15.164:0/4021316089 conn(0x7f15040fc2a0 msgr2=0x7f15040fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:25.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.901+0000 7f1509f13640 1 -- 172.21.15.164:0/4021316089 shutdown_connections 2024-09-19T08:45:25.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.901+0000 7f1509f13640 1 -- 172.21.15.164:0/4021316089 wait complete. 2024-09-19T08:45:25.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.902+0000 7f1509f13640 1 Processor -- start 2024-09-19T08:45:25.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.902+0000 7f1509f13640 1 -- start start 2024-09-19T08:45:25.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.902+0000 7f1509f13640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1504100b10 0x7f1504197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:25.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.902+0000 7f1509f13640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1504101d10 0x7f15041984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:25.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.902+0000 7f1509f13640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15041026d0 0x7f150419f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:25.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.902+0000 7f1509f13640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f150410c810 con 0x7f15041026d0 2024-09-19T08:45:25.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.902+0000 7f1509f13640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f150410c690 con 0x7f1504101d10 2024-09-19T08:45:25.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.902+0000 7f1509f13640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f150410c990 con 0x7f1504100b10 2024-09-19T08:45:25.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.902+0000 7f1503fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15041026d0 0x7f150419f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:25.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.902+0000 7f1503fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15041026d0 0x7f150419f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59788/0 (socket says 172.21.15.164:59788) 2024-09-19T08:45:25.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.902+0000 7f1503fff640 1 -- 172.21.15.164:0/4267545561 learned_addr learned my addr 172.21.15.164:0/4267545561 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:25.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.903+0000 7f1502ffd640 1 --2- 172.21.15.164:0/4267545561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1504101d10 0x7f15041984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:25.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.903+0000 7f1503fff640 1 -- 172.21.15.164:0/4267545561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1504100b10 msgr2=0x7f1504197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:25.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.903+0000 7f1503fff640 1 --2- 172.21.15.164:0/4267545561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1504100b10 0x7f1504197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:25.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.903+0000 7f1503fff640 1 -- 172.21.15.164:0/4267545561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1504101d10 msgr2=0x7f15041984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:25.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.903+0000 7f1503fff640 1 --2- 172.21.15.164:0/4267545561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1504101d10 0x7f15041984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:25.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.903+0000 7f1503fff640 1 -- 172.21.15.164:0/4267545561 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150419fbc0 con 0x7f15041026d0 2024-09-19T08:45:25.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.903+0000 7f1503fff640 1 --2- 172.21.15.164:0/4267545561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15041026d0 0x7f150419f550 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f14fc00eaa0 tx=0x7f14fc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:25.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.903+0000 7f1500ff9640 1 -- 172.21.15.164:0/4267545561 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc00cdb0 con 0x7f15041026d0 2024-09-19T08:45:25.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.903+0000 7f1509f13640 1 -- 172.21.15.164:0/4267545561 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f150419feb0 con 0x7f15041026d0 2024-09-19T08:45:25.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.903+0000 7f1500ff9640 1 -- 172.21.15.164:0/4267545561 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14fc005020 con 0x7f15041026d0 2024-09-19T08:45:25.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.903+0000 7f1500ff9640 1 -- 172.21.15.164:0/4267545561 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc0116b0 con 0x7f15041026d0 2024-09-19T08:45:25.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.903+0000 7f1509f13640 1 -- 172.21.15.164:0/4267545561 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15041a03e0 con 0x7f15041026d0 2024-09-19T08:45:25.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.904+0000 7f1509f13640 1 -- 172.21.15.164:0/4267545561 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15041a0670 con 0x7f15041026d0 2024-09-19T08:45:25.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.905+0000 7f1500ff9640 1 -- 172.21.15.164:0/4267545561 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f14fc005560 con 0x7f15041026d0 2024-09-19T08:45:25.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.906+0000 7f1500ff9640 1 --2- 172.21.15.164:0/4267545561 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f14ec0a3300 0x7f14ec0a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:25.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.906+0000 7f15037fe640 1 --2- 172.21.15.164:0/4267545561 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f14ec0a3300 0x7f14ec0a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:25.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.906+0000 7f1500ff9640 1 -- 172.21.15.164:0/4267545561 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f14fc0cff50 con 0x7f15041026d0 2024-09-19T08:45:25.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.907+0000 7f15037fe640 1 --2- 172.21.15.164:0/4267545561 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f14ec0a3300 0x7f14ec0a57c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1504101b70 tx=0x7f14f4006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:25.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:25.910+0000 7f1500ff9640 1 -- 172.21.15.164:0/4267545561 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f14fc016030 con 0x7f15041026d0 2024-09-19T08:45:26.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:26.110+0000 7f1509f13640 1 -- 172.21.15.164:0/4267545561 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f15041a0840 con 0x7f15041026d0 2024-09-19T08:45:26.110 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:25 smithi164 ceph-mon[30928]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:45:26.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:26.111+0000 7f1500ff9640 1 -- 172.21.15.164:0/4267545561 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3135 (secure 0 0 0) 0x7f14fc09cc10 con 0x7f15041026d0 2024-09-19T08:45:26.111 INFO:teuthology.orchestra.run.smithi164.stdout:e27 2024-09-19T08:45:26.111 INFO:teuthology.orchestra.run.smithi164.stdout:btime 2024-09-19T08:44:05:632372+0000 2024-09-19T08:45:26.111 INFO:teuthology.orchestra.run.smithi164.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T08:45:26.111 INFO:teuthology.orchestra.run.smithi164.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T08:45:26.111 INFO:teuthology.orchestra.run.smithi164.stdout:legacy client fscid: 1 2024-09-19T08:45:26.111 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:45:26.111 INFO:teuthology.orchestra.run.smithi164.stdout:Filesystem 'cephfs' (1) 2024-09-19T08:45:26.111 INFO:teuthology.orchestra.run.smithi164.stdout:fs_name cephfs 2024-09-19T08:45:26.111 INFO:teuthology.orchestra.run.smithi164.stdout:epoch 27 2024-09-19T08:45:26.111 INFO:teuthology.orchestra.run.smithi164.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-19T08:45:26.112 INFO:teuthology.orchestra.run.smithi164.stdout:created 2024-09-19T08:43:53.386044+0000 2024-09-19T08:45:26.112 INFO:teuthology.orchestra.run.smithi164.stdout:modified 2024-09-19T08:44:05.632353+0000 2024-09-19T08:45:26.112 INFO:teuthology.orchestra.run.smithi164.stdout:tableserver 0 2024-09-19T08:45:26.112 INFO:teuthology.orchestra.run.smithi164.stdout:root 0 2024-09-19T08:45:26.112 INFO:teuthology.orchestra.run.smithi164.stdout:session_timeout 300 2024-09-19T08:45:26.112 INFO:teuthology.orchestra.run.smithi164.stdout:session_autoclose 300 2024-09-19T08:45:26.112 INFO:teuthology.orchestra.run.smithi164.stdout:max_file_size 1099511627776 2024-09-19T08:45:26.112 INFO:teuthology.orchestra.run.smithi164.stdout:max_xattr_size 65536 2024-09-19T08:45:26.112 INFO:teuthology.orchestra.run.smithi164.stdout:required_client_features {} 2024-09-19T08:45:26.112 INFO:teuthology.orchestra.run.smithi164.stdout:last_failure 0 2024-09-19T08:45:26.112 INFO:teuthology.orchestra.run.smithi164.stdout:last_failure_osd_epoch 0 2024-09-19T08:45:26.112 INFO:teuthology.orchestra.run.smithi164.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T08:45:26.112 INFO:teuthology.orchestra.run.smithi164.stdout:max_mds 5 2024-09-19T08:45:26.113 INFO:teuthology.orchestra.run.smithi164.stdout:in 0,1,2,3,4 2024-09-19T08:45:26.113 INFO:teuthology.orchestra.run.smithi164.stdout:up {0=24479,1=24449,2=24442,3=24424,4=14595} 2024-09-19T08:45:26.113 INFO:teuthology.orchestra.run.smithi164.stdout:failed 2024-09-19T08:45:26.113 INFO:teuthology.orchestra.run.smithi164.stdout:damaged 2024-09-19T08:45:26.113 INFO:teuthology.orchestra.run.smithi164.stdout:stopped 2024-09-19T08:45:26.113 INFO:teuthology.orchestra.run.smithi164.stdout:data_pools [3] 2024-09-19T08:45:26.113 INFO:teuthology.orchestra.run.smithi164.stdout:metadata_pool 2 2024-09-19T08:45:26.113 INFO:teuthology.orchestra.run.smithi164.stdout:inline_data disabled 2024-09-19T08:45:26.113 INFO:teuthology.orchestra.run.smithi164.stdout:balancer 2024-09-19T08:45:26.113 INFO:teuthology.orchestra.run.smithi164.stdout:bal_rank_mask -1 2024-09-19T08:45:26.113 INFO:teuthology.orchestra.run.smithi164.stdout:standby_count_wanted 0 2024-09-19T08:45:26.113 INFO:teuthology.orchestra.run.smithi164.stdout:qdb_cluster leader: 24479 members: 24479,24442,24424,24449,14595 2024-09-19T08:45:26.113 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.b{0:24479} state up:active seq 4 addr [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T08:45:26.113 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.e{0:24422} state up:standby-replay seq 1 addr [v2:172.21.15.170:6834/321254273,v1:172.21.15.170:6835/321254273] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T08:45:26.114 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.h{1:24449} state up:active seq 9 addr [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T08:45:26.114 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.f{1:24418} state up:standby-replay seq 1 addr [v2:172.21.15.173:6834/947317705,v1:172.21.15.173:6835/947317705] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T08:45:26.114 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.i{2:24442} state up:active seq 9 addr [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T08:45:26.114 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.l{2:24403} state up:standby-replay seq 1 addr [v2:172.21.15.173:6832/486946900,v1:172.21.15.173:6833/486946900] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T08:45:26.114 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.c{3:24424} state up:active seq 10 addr [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T08:45:26.114 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.k{3:24401} state up:standby-replay seq 1 addr [v2:172.21.15.170:6832/2726071671,v1:172.21.15.170:6833/2726071671] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T08:45:26.114 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.a{4:14595} state up:active seq 8 addr [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T08:45:26.114 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.g{4:14583} state up:standby-replay seq 1 addr [v2:172.21.15.164:6838/538276454,v1:172.21.15.164:6839/538276454] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T08:45:26.114 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:45:26.114 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:45:26.114 INFO:teuthology.orchestra.run.smithi164.stdout:Standby daemons: 2024-09-19T08:45:26.114 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:45:26.114 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.j{-1:14526} state up:standby seq 1 addr [v2:172.21.15.164:6834/3695235648,v1:172.21.15.164:6835/3695235648] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T08:45:26.114 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.d{-1:14538} state up:standby seq 1 addr [v2:172.21.15.164:6836/199930574,v1:172.21.15.164:6837/199930574] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T08:45:26.115 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 27 2024-09-19T08:45:26.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:26.113+0000 7f1509f13640 1 -- 172.21.15.164:0/4267545561 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f14ec0a3300 msgr2=0x7f14ec0a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:26.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:26.113+0000 7f1509f13640 1 --2- 172.21.15.164:0/4267545561 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f14ec0a3300 0x7f14ec0a57c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1504101b70 tx=0x7f14f4006c90 comp rx=0 tx=0).stop 2024-09-19T08:45:26.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:26.113+0000 7f1509f13640 1 -- 172.21.15.164:0/4267545561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15041026d0 msgr2=0x7f150419f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:26.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:26.113+0000 7f1509f13640 1 --2- 172.21.15.164:0/4267545561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15041026d0 0x7f150419f550 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f14fc00eaa0 tx=0x7f14fc00ef60 comp rx=0 tx=0).stop 2024-09-19T08:45:26.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:26.113+0000 7f1509f13640 1 -- 172.21.15.164:0/4267545561 shutdown_connections 2024-09-19T08:45:26.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:26.113+0000 7f1509f13640 1 --2- 172.21.15.164:0/4267545561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15041026d0 0x7f150419f550 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:26.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:26.113+0000 7f1509f13640 1 --2- 172.21.15.164:0/4267545561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1504101d10 0x7f15041984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:26.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:26.113+0000 7f1509f13640 1 --2- 172.21.15.164:0/4267545561 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f14ec0a3300 0x7f14ec0a57c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:26.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:26.113+0000 7f1509f13640 1 --2- 172.21.15.164:0/4267545561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1504100b10 0x7f1504197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:26.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:26.113+0000 7f1509f13640 1 -- 172.21.15.164:0/4267545561 >> 172.21.15.164:0/4267545561 conn(0x7f15040fc2a0 msgr2=0x7f150406a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:26.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:26.113+0000 7f1509f13640 1 -- 172.21.15.164:0/4267545561 shutdown_connections 2024-09-19T08:45:26.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:26.113+0000 7f1509f13640 1 -- 172.21.15.164:0/4267545561 wait complete. 2024-09-19T08:45:26.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:25 smithi170 ceph-mon[35072]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:45:26.933 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-19T08:45:27.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:26 smithi170 ceph-mon[35072]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T08:45:27.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:26 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/4267545561' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T08:45:27.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:26 smithi173 ceph-mon[35306]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T08:45:27.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:26 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/4267545561' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T08:45:27.184 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:45:27.281 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:26 smithi164 ceph-mon[30928]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T08:45:27.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:26 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/4267545561' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T08:45:27.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.658+0000 7fcbdd39c640 1 -- 172.21.15.164:0/635611092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbd8107740 msgr2=0x7fcbd8109b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:27.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.658+0000 7fcbdd39c640 1 --2- 172.21.15.164:0/635611092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbd8107740 0x7fcbd8109b90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009ca0 tx=0x7fcbd002a8c0 comp rx=0 tx=0).stop 2024-09-19T08:45:27.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.658+0000 7fcbdd39c640 1 -- 172.21.15.164:0/635611092 shutdown_connections 2024-09-19T08:45:27.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.658+0000 7fcbdd39c640 1 --2- 172.21.15.164:0/635611092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbd8107740 0x7fcbd8109b90 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:27.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.658+0000 7fcbdd39c640 1 --2- 172.21.15.164:0/635611092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbd8104e10 0x7fcbd8107200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:27.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.658+0000 7fcbdd39c640 1 --2- 172.21.15.164:0/635611092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbd81024e0 0x7fcbd81048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:27.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.658+0000 7fcbdd39c640 1 -- 172.21.15.164:0/635611092 >> 172.21.15.164:0/635611092 conn(0x7fcbd80fc0e0 msgr2=0x7fcbd80fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:27.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.658+0000 7fcbdd39c640 1 -- 172.21.15.164:0/635611092 shutdown_connections 2024-09-19T08:45:27.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.658+0000 7fcbdd39c640 1 -- 172.21.15.164:0/635611092 wait complete. 2024-09-19T08:45:27.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.659+0000 7fcbdd39c640 1 Processor -- start 2024-09-19T08:45:27.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.659+0000 7fcbdd39c640 1 -- start start 2024-09-19T08:45:27.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.659+0000 7fcbdd39c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbd81024e0 0x7fcbd8197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:27.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.659+0000 7fcbdd39c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbd8104e10 0x7fcbd8198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:27.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.659+0000 7fcbdd39c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbd8107740 0x7fcbd819f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:27.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.659+0000 7fcbdd39c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd810d780 con 0x7fcbd81024e0 2024-09-19T08:45:27.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.659+0000 7fcbdd39c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd810d600 con 0x7fcbd8107740 2024-09-19T08:45:27.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.659+0000 7fcbdd39c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd810d900 con 0x7fcbd8104e10 2024-09-19T08:45:27.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.660+0000 7fcbd7fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbd81024e0 0x7fcbd8197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:27.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.660+0000 7fcbd7fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbd81024e0 0x7fcbd8197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59806/0 (socket says 172.21.15.164:59806) 2024-09-19T08:45:27.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.660+0000 7fcbd7fff640 1 -- 172.21.15.164:0/441934068 learned_addr learned my addr 172.21.15.164:0/441934068 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:27.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.660+0000 7fcbcffff640 1 --2- 172.21.15.164:0/441934068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbd8104e10 0x7fcbd8198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:27.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.660+0000 7fcbd7fff640 1 -- 172.21.15.164:0/441934068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbd8104e10 msgr2=0x7fcbd8198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:27.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.660+0000 7fcbd7fff640 1 --2- 172.21.15.164:0/441934068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbd8104e10 0x7fcbd8198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:27.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.660+0000 7fcbdcb9b640 1 --2- 172.21.15.164:0/441934068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbd8107740 0x7fcbd819f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:27.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.660+0000 7fcbd7fff640 1 -- 172.21.15.164:0/441934068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbd8107740 msgr2=0x7fcbd819f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:27.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.660+0000 7fcbd7fff640 1 --2- 172.21.15.164:0/441934068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbd8107740 0x7fcbd819f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:27.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.660+0000 7fcbd7fff640 1 -- 172.21.15.164:0/441934068 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd819fa10 con 0x7fcbd81024e0 2024-09-19T08:45:27.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.660+0000 7fcbcffff640 1 --2- 172.21.15.164:0/441934068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbd8104e10 0x7fcbd8198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:27.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.660+0000 7fcbd7fff640 1 --2- 172.21.15.164:0/441934068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbd81024e0 0x7fcbd8197de0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fcbc800b4a0 tx=0x7fcbc800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:27.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.660+0000 7fcbd5ffb640 1 -- 172.21.15.164:0/441934068 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8014070 con 0x7fcbd81024e0 2024-09-19T08:45:27.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.660+0000 7fcbd5ffb640 1 -- 172.21.15.164:0/441934068 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbc8010af0 con 0x7fcbd81024e0 2024-09-19T08:45:27.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.660+0000 7fcbdd39c640 1 -- 172.21.15.164:0/441934068 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbd819fd00 con 0x7fcbd81024e0 2024-09-19T08:45:27.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.660+0000 7fcbd5ffb640 1 -- 172.21.15.164:0/441934068 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8017d10 con 0x7fcbd81024e0 2024-09-19T08:45:27.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.660+0000 7fcbdd39c640 1 -- 172.21.15.164:0/441934068 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbd81a0210 con 0x7fcbd81024e0 2024-09-19T08:45:27.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.661+0000 7fcbdd39c640 1 -- 172.21.15.164:0/441934068 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbd80fdbd0 con 0x7fcbd81024e0 2024-09-19T08:45:27.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.662+0000 7fcbd5ffb640 1 -- 172.21.15.164:0/441934068 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fcbc8017070 con 0x7fcbd81024e0 2024-09-19T08:45:27.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.663+0000 7fcbd5ffb640 1 --2- 172.21.15.164:0/441934068 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fcbc40a3300 0x7fcbc40a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:27.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.663+0000 7fcbcffff640 1 --2- 172.21.15.164:0/441934068 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fcbc40a3300 0x7fcbc40a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:27.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.663+0000 7fcbd5ffb640 1 -- 172.21.15.164:0/441934068 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fcbc80cecf0 con 0x7fcbd81024e0 2024-09-19T08:45:27.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.664+0000 7fcbcffff640 1 --2- 172.21.15.164:0/441934068 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fcbc40a3300 0x7fcbc40a57c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcbc00058c0 tx=0x7fcbc0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:27.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.667+0000 7fcbd5ffb640 1 -- 172.21.15.164:0/441934068 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcbc809bac0 con 0x7fcbd81024e0 2024-09-19T08:45:27.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.837+0000 7fcbdd39c640 1 -- 172.21.15.164:0/441934068 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fcbd802d7c0 con 0x7fcbd81024e0 2024-09-19T08:45:27.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.838+0000 7fcbd5ffb640 1 -- 172.21.15.164:0/441934068 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v82) ==== 56+0+5263 (secure 0 0 0) 0x7fcbc809b3c0 con 0x7fcbd81024e0 2024-09-19T08:45:27.838 INFO:teuthology.orchestra.run.smithi164.stdout:epoch 82 2024-09-19T08:45:27.838 INFO:teuthology.orchestra.run.smithi164.stdout:fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 2024-09-19T08:45:27.839 INFO:teuthology.orchestra.run.smithi164.stdout:created 2024-09-19T08:36:09.564509+0000 2024-09-19T08:45:27.839 INFO:teuthology.orchestra.run.smithi164.stdout:modified 2024-09-19T08:44:02.811236+0000 2024-09-19T08:45:27.839 INFO:teuthology.orchestra.run.smithi164.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-19T08:45:27.839 INFO:teuthology.orchestra.run.smithi164.stdout:crush_version 26 2024-09-19T08:45:27.839 INFO:teuthology.orchestra.run.smithi164.stdout:full_ratio 0.95 2024-09-19T08:45:27.839 INFO:teuthology.orchestra.run.smithi164.stdout:backfillfull_ratio 0.9 2024-09-19T08:45:27.839 INFO:teuthology.orchestra.run.smithi164.stdout:nearfull_ratio 0.85 2024-09-19T08:45:27.839 INFO:teuthology.orchestra.run.smithi164.stdout:require_min_compat_client luminous 2024-09-19T08:45:27.839 INFO:teuthology.orchestra.run.smithi164.stdout:min_compat_client luminous 2024-09-19T08:45:27.839 INFO:teuthology.orchestra.run.smithi164.stdout:require_osd_release squid 2024-09-19T08:45:27.839 INFO:teuthology.orchestra.run.smithi164.stdout:stretch_mode_enabled false 2024-09-19T08:45:27.839 INFO:teuthology.orchestra.run.smithi164.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-19T08:45:27.839 INFO:teuthology.orchestra.run.smithi164.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-19T08:45:27.840 INFO:teuthology.orchestra.run.smithi164.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-19T08:45:27.840 INFO:teuthology.orchestra.run.smithi164.stdout:max_osd 12 2024-09-19T08:45:27.840 INFO:teuthology.orchestra.run.smithi164.stdout:osd.0 up in weight 1 up_from 8 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6802/663038243,v1:172.21.15.164:6803/663038243] [v2:172.21.15.164:6804/663038243,v1:172.21.15.164:6805/663038243] exists,up 33b11f37-6ac5-41fc-8cfd-1b08c7eef688 2024-09-19T08:45:27.840 INFO:teuthology.orchestra.run.smithi164.stdout:osd.1 up in weight 1 up_from 13 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6800/2167066371,v1:172.21.15.170:6801/2167066371] [v2:172.21.15.170:6802/2167066371,v1:172.21.15.170:6803/2167066371] exists,up 6de64b0a-46e2-4a98-8005-a326a64bc87d 2024-09-19T08:45:27.840 INFO:teuthology.orchestra.run.smithi164.stdout:osd.2 up in weight 1 up_from 18 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6800/1646561048,v1:172.21.15.173:6801/1646561048] [v2:172.21.15.173:6802/1646561048,v1:172.21.15.173:6803/1646561048] exists,up d4158cc7-9822-42a7-9601-d65089907584 2024-09-19T08:45:27.840 INFO:teuthology.orchestra.run.smithi164.stdout:osd.3 up in weight 1 up_from 24 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6810/406795560,v1:172.21.15.164:6811/406795560] [v2:172.21.15.164:6812/406795560,v1:172.21.15.164:6813/406795560] exists,up 0cf3f7f1-3a8c-42ce-88a9-9bea52e02221 2024-09-19T08:45:27.840 INFO:teuthology.orchestra.run.smithi164.stdout:osd.4 up in weight 1 up_from 31 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6808/4120832165,v1:172.21.15.170:6809/4120832165] [v2:172.21.15.170:6810/4120832165,v1:172.21.15.170:6811/4120832165] exists,up 6dca048a-2860-4a02-8d8e-450b30bc1a3c 2024-09-19T08:45:27.840 INFO:teuthology.orchestra.run.smithi164.stdout:osd.5 up in weight 1 up_from 36 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6808/2914653552,v1:172.21.15.173:6809/2914653552] [v2:172.21.15.173:6810/2914653552,v1:172.21.15.173:6811/2914653552] exists,up 3822521f-69d6-4141-a83f-5fc9cecc7b3e 2024-09-19T08:45:27.840 INFO:teuthology.orchestra.run.smithi164.stdout:osd.6 up in weight 1 up_from 43 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6818/1138341393,v1:172.21.15.164:6819/1138341393] [v2:172.21.15.164:6820/1138341393,v1:172.21.15.164:6821/1138341393] exists,up 0f5a965e-2eb2-4c24-b089-577e0639c160 2024-09-19T08:45:27.840 INFO:teuthology.orchestra.run.smithi164.stdout:osd.7 up in weight 1 up_from 47 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6816/2124160360,v1:172.21.15.170:6817/2124160360] [v2:172.21.15.170:6818/2124160360,v1:172.21.15.170:6819/2124160360] exists,up f16cb2c3-639b-46ae-a740-1bd28ca1863c 2024-09-19T08:45:27.841 INFO:teuthology.orchestra.run.smithi164.stdout:osd.8 up in weight 1 up_from 54 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6816/2285175061,v1:172.21.15.173:6817/2285175061] [v2:172.21.15.173:6818/2285175061,v1:172.21.15.173:6819/2285175061] exists,up 161cff01-46ad-48e3-b999-a5473a2ad618 2024-09-19T08:45:27.841 INFO:teuthology.orchestra.run.smithi164.stdout:osd.9 up in weight 1 up_from 61 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6826/3918847952,v1:172.21.15.164:6827/3918847952] [v2:172.21.15.164:6828/3918847952,v1:172.21.15.164:6829/3918847952] exists,up 994666e8-8fbd-4231-ac67-81a79aaf7c2a 2024-09-19T08:45:27.841 INFO:teuthology.orchestra.run.smithi164.stdout:osd.10 up in weight 1 up_from 67 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6824/2707848227,v1:172.21.15.170:6825/2707848227] [v2:172.21.15.170:6826/2707848227,v1:172.21.15.170:6827/2707848227] exists,up ff5e9099-cdb4-4905-b888-5b67320ef76b 2024-09-19T08:45:27.841 INFO:teuthology.orchestra.run.smithi164.stdout:osd.11 up in weight 1 up_from 74 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6824/2831924738,v1:172.21.15.173:6825/2831924738] [v2:172.21.15.173:6826/2831924738,v1:172.21.15.173:6827/2831924738] exists,up 4d93088f-e791-4938-a207-12f15b56abd8 2024-09-19T08:45:27.841 INFO:teuthology.orchestra.run.smithi164.stdout:pg_upmap_items 2.8 [2,11] 2024-09-19T08:45:27.841 INFO:teuthology.orchestra.run.smithi164.stdout:pg_upmap_items 2.10 [8,11] 2024-09-19T08:45:27.841 INFO:teuthology.orchestra.run.smithi164.stdout:pg_upmap_items 3.c [3,9] 2024-09-19T08:45:27.841 INFO:teuthology.orchestra.run.smithi164.stdout:pg_upmap_items 3.11 [5,9] 2024-09-19T08:45:27.841 INFO:teuthology.orchestra.run.smithi164.stdout:pg_upmap_items 3.1d [4,8] 2024-09-19T08:45:27.841 INFO:teuthology.orchestra.run.smithi164.stdout:pg_upmap_items 3.2c [4,9] 2024-09-19T08:45:27.841 INFO:teuthology.orchestra.run.smithi164.stdout:blocklist 172.21.15.164:0/339032808 expires 2024-09-20T08:37:01.663065+0000 2024-09-19T08:45:27.841 INFO:teuthology.orchestra.run.smithi164.stdout:blocklist 172.21.15.164:6801/1173822158 expires 2024-09-20T08:37:01.663065+0000 2024-09-19T08:45:27.842 INFO:teuthology.orchestra.run.smithi164.stdout:blocklist 172.21.15.164:6800/1173822158 expires 2024-09-20T08:37:01.663065+0000 2024-09-19T08:45:27.842 INFO:teuthology.orchestra.run.smithi164.stdout:blocklist 172.21.15.164:6801/2805828009 expires 2024-09-20T08:36:33.856642+0000 2024-09-19T08:45:27.842 INFO:teuthology.orchestra.run.smithi164.stdout:blocklist 172.21.15.164:0/1837164919 expires 2024-09-20T08:37:01.663065+0000 2024-09-19T08:45:27.842 INFO:teuthology.orchestra.run.smithi164.stdout:blocklist 172.21.15.164:0/4018459143 expires 2024-09-20T08:37:01.663065+0000 2024-09-19T08:45:27.842 INFO:teuthology.orchestra.run.smithi164.stdout:blocklist 172.21.15.164:6800/2805828009 expires 2024-09-20T08:36:33.856642+0000 2024-09-19T08:45:27.842 INFO:teuthology.orchestra.run.smithi164.stdout:blocklist 172.21.15.164:0/2770797546 expires 2024-09-20T08:36:33.856642+0000 2024-09-19T08:45:27.842 INFO:teuthology.orchestra.run.smithi164.stdout:blocklist 172.21.15.164:0/1905394497 expires 2024-09-20T08:36:33.856642+0000 2024-09-19T08:45:27.842 INFO:teuthology.orchestra.run.smithi164.stdout:blocklist 172.21.15.164:0/1369265436 expires 2024-09-20T08:36:33.856642+0000 2024-09-19T08:45:27.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.840+0000 7fcbdd39c640 1 -- 172.21.15.164:0/441934068 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fcbc40a3300 msgr2=0x7fcbc40a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:27.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.840+0000 7fcbdd39c640 1 --2- 172.21.15.164:0/441934068 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fcbc40a3300 0x7fcbc40a57c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcbc00058c0 tx=0x7fcbc0005850 comp rx=0 tx=0).stop 2024-09-19T08:45:27.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.841+0000 7fcbdd39c640 1 -- 172.21.15.164:0/441934068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbd81024e0 msgr2=0x7fcbd8197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:27.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.841+0000 7fcbdd39c640 1 --2- 172.21.15.164:0/441934068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbd81024e0 0x7fcbd8197de0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fcbc800b4a0 tx=0x7fcbc800b960 comp rx=0 tx=0).stop 2024-09-19T08:45:27.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.841+0000 7fcbdd39c640 1 -- 172.21.15.164:0/441934068 shutdown_connections 2024-09-19T08:45:27.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.841+0000 7fcbdd39c640 1 --2- 172.21.15.164:0/441934068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbd8107740 0x7fcbd819f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:27.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.841+0000 7fcbdd39c640 1 --2- 172.21.15.164:0/441934068 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fcbc40a3300 0x7fcbc40a57c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:27.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.841+0000 7fcbdd39c640 1 --2- 172.21.15.164:0/441934068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbd8104e10 0x7fcbd8198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:27.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.841+0000 7fcbdd39c640 1 --2- 172.21.15.164:0/441934068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbd81024e0 0x7fcbd8197de0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:27.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.841+0000 7fcbdd39c640 1 -- 172.21.15.164:0/441934068 >> 172.21.15.164:0/441934068 conn(0x7fcbd80fc0e0 msgr2=0x7fcbd8102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:27.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.841+0000 7fcbdd39c640 1 -- 172.21.15.164:0/441934068 shutdown_connections 2024-09-19T08:45:27.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:27.841+0000 7fcbdd39c640 1 -- 172.21.15.164:0/441934068 wait complete. 2024-09-19T08:45:28.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:27 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/441934068' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T08:45:28.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:27 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/441934068' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T08:45:28.214 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:27 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/441934068' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T08:45:28.616 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-19T08:45:28.628 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:45:28.877 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:45:29.110 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:28 smithi164 ceph-mon[30928]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-19T08:45:29.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:28 smithi170 ceph-mon[35072]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-19T08:45:29.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:28 smithi173 ceph-mon[35306]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-19T08:45:29.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.425+0000 7fe8a92bf640 1 -- 172.21.15.164:0/1826237858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8a4101de0 msgr2=0x7fe8a41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:29.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.425+0000 7fe8a92bf640 1 --2- 172.21.15.164:0/1826237858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8a4101de0 0x7fe8a41021c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe894009900 tx=0x7fe89402a410 comp rx=0 tx=0).stop 2024-09-19T08:45:29.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.425+0000 7fe8a92bf640 1 -- 172.21.15.164:0/1826237858 shutdown_connections 2024-09-19T08:45:29.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.425+0000 7fe8a92bf640 1 --2- 172.21.15.164:0/1826237858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8a4106f10 0x7fe8a410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.425+0000 7fe8a92bf640 1 --2- 172.21.15.164:0/1826237858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8a4102790 0x7fe8a41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.425+0000 7fe8a92bf640 1 --2- 172.21.15.164:0/1826237858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8a4101de0 0x7fe8a41021c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.425+0000 7fe8a92bf640 1 -- 172.21.15.164:0/1826237858 >> 172.21.15.164:0/1826237858 conn(0x7fe8a40fd8c0 msgr2=0x7fe8a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:29.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.425+0000 7fe8a92bf640 1 -- 172.21.15.164:0/1826237858 shutdown_connections 2024-09-19T08:45:29.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.425+0000 7fe8a92bf640 1 -- 172.21.15.164:0/1826237858 wait complete. 2024-09-19T08:45:29.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.426+0000 7fe8a92bf640 1 Processor -- start 2024-09-19T08:45:29.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.426+0000 7fe8a92bf640 1 -- start start 2024-09-19T08:45:29.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.426+0000 7fe8a92bf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8a4101de0 0x7fe8a419f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:29.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.426+0000 7fe8a92bf640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8a4102790 0x7fe8a419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:29.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.426+0000 7fe8a92bf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8a4106f10 0x7fe8a410dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:29.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.426+0000 7fe8a92bf640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a4111f90 con 0x7fe8a4106f10 2024-09-19T08:45:29.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.426+0000 7fe8a92bf640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a4111e10 con 0x7fe8a4101de0 2024-09-19T08:45:29.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.426+0000 7fe8a92bf640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a4112110 con 0x7fe8a4102790 2024-09-19T08:45:29.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.426+0000 7fe8a37fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8a4106f10 0x7fe8a410dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:29.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.426+0000 7fe8a37fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8a4106f10 0x7fe8a410dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59820/0 (socket says 172.21.15.164:59820) 2024-09-19T08:45:29.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.426+0000 7fe8a37fe640 1 -- 172.21.15.164:0/3886129545 learned_addr learned my addr 172.21.15.164:0/3886129545 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:29.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.426+0000 7fe8a2ffd640 1 --2- 172.21.15.164:0/3886129545 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8a4101de0 0x7fe8a419f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:29.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.427+0000 7fe89bfff640 1 --2- 172.21.15.164:0/3886129545 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8a4102790 0x7fe8a419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:29.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.427+0000 7fe8a37fe640 1 -- 172.21.15.164:0/3886129545 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8a4102790 msgr2=0x7fe8a419f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:29.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.427+0000 7fe8a37fe640 1 --2- 172.21.15.164:0/3886129545 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8a4102790 0x7fe8a419f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.427+0000 7fe8a37fe640 1 -- 172.21.15.164:0/3886129545 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8a4101de0 msgr2=0x7fe8a419f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:29.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.427+0000 7fe8a37fe640 1 --2- 172.21.15.164:0/3886129545 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8a4101de0 0x7fe8a419f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.427+0000 7fe8a37fe640 1 -- 172.21.15.164:0/3886129545 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a410e610 con 0x7fe8a4106f10 2024-09-19T08:45:29.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.427+0000 7fe8a37fe640 1 --2- 172.21.15.164:0/3886129545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8a4106f10 0x7fe8a410dd50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe89c00b970 tx=0x7fe89c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:29.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.427+0000 7fe8a0ff9640 1 -- 172.21.15.164:0/3886129545 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89c019070 con 0x7fe8a4106f10 2024-09-19T08:45:29.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.427+0000 7fe8a92bf640 1 -- 172.21.15.164:0/3886129545 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8a410e900 con 0x7fe8a4106f10 2024-09-19T08:45:29.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.427+0000 7fe8a0ff9640 1 -- 172.21.15.164:0/3886129545 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe89c00cb40 con 0x7fe8a4106f10 2024-09-19T08:45:29.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.427+0000 7fe8a0ff9640 1 -- 172.21.15.164:0/3886129545 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89c007650 con 0x7fe8a4106f10 2024-09-19T08:45:29.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.427+0000 7fe8a92bf640 1 -- 172.21.15.164:0/3886129545 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8a419ad90 con 0x7fe8a4106f10 2024-09-19T08:45:29.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.428+0000 7fe8a92bf640 1 -- 172.21.15.164:0/3886129545 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8a41032c0 con 0x7fe8a4106f10 2024-09-19T08:45:29.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.430+0000 7fe8a0ff9640 1 -- 172.21.15.164:0/3886129545 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fe89c01c070 con 0x7fe8a4106f10 2024-09-19T08:45:29.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.430+0000 7fe8a0ff9640 1 --2- 172.21.15.164:0/3886129545 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fe8900a32d0 0x7fe8900a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:29.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.430+0000 7fe8a2ffd640 1 --2- 172.21.15.164:0/3886129545 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fe8900a32d0 0x7fe8900a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:29.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.430+0000 7fe8a0ff9640 1 -- 172.21.15.164:0/3886129545 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fe89c0ceef0 con 0x7fe8a4106f10 2024-09-19T08:45:29.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.431+0000 7fe8a2ffd640 1 --2- 172.21.15.164:0/3886129545 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fe8900a32d0 0x7fe8900a5790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe894009230 tx=0x7fe8940058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:29.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.434+0000 7fe8a0ff9640 1 -- 172.21.15.164:0/3886129545 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe89c09b570 con 0x7fe8a4106f10 2024-09-19T08:45:29.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.642+0000 7fe8a92bf640 1 -- 172.21.15.164:0/3886129545 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8a419b840 con 0x7fe8a4106f10 2024-09-19T08:45:29.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.643+0000 7fe8a0ff9640 1 -- 172.21.15.164:0/3886129545 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11984 (secure 0 0 0) 0x7fe89c09f870 con 0x7fe8a4106f10 2024-09-19T08:45:29.643 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:45:29.643 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":27,"btime":"2024-09-19T08:44:05:632372+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:44:05.632353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24442,24424,24449,14595]},"id":1}]} 2024-09-19T08:45:29.643 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 27 2024-09-19T08:45:29.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.645+0000 7fe8a92bf640 1 -- 172.21.15.164:0/3886129545 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fe8900a32d0 msgr2=0x7fe8900a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:29.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.645+0000 7fe8a92bf640 1 --2- 172.21.15.164:0/3886129545 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fe8900a32d0 0x7fe8900a5790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe894009230 tx=0x7fe8940058d0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.645+0000 7fe8a92bf640 1 -- 172.21.15.164:0/3886129545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8a4106f10 msgr2=0x7fe8a410dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:29.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.645+0000 7fe8a92bf640 1 --2- 172.21.15.164:0/3886129545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8a4106f10 0x7fe8a410dd50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe89c00b970 tx=0x7fe89c00be30 comp rx=0 tx=0).stop 2024-09-19T08:45:29.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.645+0000 7fe8a92bf640 1 -- 172.21.15.164:0/3886129545 shutdown_connections 2024-09-19T08:45:29.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.645+0000 7fe8a92bf640 1 --2- 172.21.15.164:0/3886129545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8a4106f10 0x7fe8a410dd50 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.645+0000 7fe8a92bf640 1 --2- 172.21.15.164:0/3886129545 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fe8900a32d0 0x7fe8900a5790 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.645+0000 7fe8a92bf640 1 --2- 172.21.15.164:0/3886129545 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8a4102790 0x7fe8a419f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.645+0000 7fe8a92bf640 1 --2- 172.21.15.164:0/3886129545 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8a4101de0 0x7fe8a419f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.645+0000 7fe8a92bf640 1 -- 172.21.15.164:0/3886129545 >> 172.21.15.164:0/3886129545 conn(0x7fe8a40fd8c0 msgr2=0x7fe8a40fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:29.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.645+0000 7fe8a92bf640 1 -- 172.21.15.164:0/3886129545 shutdown_connections 2024-09-19T08:45:29.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:29.645+0000 7fe8a92bf640 1 -- 172.21.15.164:0/3886129545 wait complete. 2024-09-19T08:45:30.020 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:29 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3886129545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:45:30.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:29 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3886129545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:45:30.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:29 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3886129545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:45:30.428 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:45:30.684 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:45:30.957 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:30 smithi164 ceph-mon[30928]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:45:31.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:30 smithi170 ceph-mon[35072]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:45:31.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:30 smithi173 ceph-mon[35306]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:45:31.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.168+0000 7f76ee789640 1 -- 172.21.15.164:0/4185493814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76e8100a80 msgr2=0x7f76e8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:31.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.168+0000 7f76ee789640 1 --2- 172.21.15.164:0/4185493814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76e8100a80 0x7f76e8100ee0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f76d0009930 tx=0x7f76d002a420 comp rx=0 tx=0).stop 2024-09-19T08:45:31.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.168+0000 7f76ee789640 1 -- 172.21.15.164:0/4185493814 shutdown_connections 2024-09-19T08:45:31.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.168+0000 7f76ee789640 1 --2- 172.21.15.164:0/4185493814 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76e8101420 0x7f76e810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.168+0000 7f76ee789640 1 --2- 172.21.15.164:0/4185493814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76e8100a80 0x7f76e8100ee0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.168+0000 7f76ee789640 1 --2- 172.21.15.164:0/4185493814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76e8106a80 0x7f76e8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.168+0000 7f76ee789640 1 -- 172.21.15.164:0/4185493814 >> 172.21.15.164:0/4185493814 conn(0x7f76e80fc7a0 msgr2=0x7f76e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:31.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.168+0000 7f76ee789640 1 -- 172.21.15.164:0/4185493814 shutdown_connections 2024-09-19T08:45:31.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.168+0000 7f76ee789640 1 -- 172.21.15.164:0/4185493814 wait complete. 2024-09-19T08:45:31.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.168+0000 7f76ee789640 1 Processor -- start 2024-09-19T08:45:31.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.168+0000 7f76ee789640 1 -- start start 2024-09-19T08:45:31.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.169+0000 7f76ee789640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76e8100a80 0x7f76e81993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:31.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.169+0000 7f76ee789640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76e8101420 0x7f76e8199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:31.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.169+0000 7f76ee789640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76e8106a80 0x7f76e81a4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:31.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.169+0000 7f76ee789640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f76e8110f20 con 0x7f76e8106a80 2024-09-19T08:45:31.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.169+0000 7f76ee789640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f76e8110da0 con 0x7f76e8100a80 2024-09-19T08:45:31.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.169+0000 7f76ee789640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f76e81110a0 con 0x7f76e8101420 2024-09-19T08:45:31.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.169+0000 7f76ecd26640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76e8106a80 0x7f76e81a4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:31.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.169+0000 7f76ecd26640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76e8106a80 0x7f76e81a4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59834/0 (socket says 172.21.15.164:59834) 2024-09-19T08:45:31.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.169+0000 7f76ecd26640 1 -- 172.21.15.164:0/2161661191 learned_addr learned my addr 172.21.15.164:0/2161661191 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:31.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.169+0000 7f76e7fff640 1 --2- 172.21.15.164:0/2161661191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76e8100a80 0x7f76e81993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:31.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.169+0000 7f76ecd26640 1 -- 172.21.15.164:0/2161661191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76e8101420 msgr2=0x7f76e8199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:31.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.169+0000 7f76ecd26640 1 --2- 172.21.15.164:0/2161661191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76e8101420 0x7f76e8199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.169+0000 7f76ecd26640 1 -- 172.21.15.164:0/2161661191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76e8100a80 msgr2=0x7f76e81993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:31.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.169+0000 7f76ecd26640 1 --2- 172.21.15.164:0/2161661191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76e8100a80 0x7f76e81993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.169+0000 7f76ecd26640 1 -- 172.21.15.164:0/2161661191 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e81a49b0 con 0x7f76e8106a80 2024-09-19T08:45:31.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.169+0000 7f76ecd26640 1 --2- 172.21.15.164:0/2161661191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76e8106a80 0x7f76e81a4470 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f76e000b970 tx=0x7f76e000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:31.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.169+0000 7f76e5ffb640 1 -- 172.21.15.164:0/2161661191 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e0019070 con 0x7f76e8106a80 2024-09-19T08:45:31.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.169+0000 7f76e5ffb640 1 -- 172.21.15.164:0/2161661191 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76e000cb40 con 0x7f76e8106a80 2024-09-19T08:45:31.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.169+0000 7f76ee789640 1 -- 172.21.15.164:0/2161661191 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e81a4b80 con 0x7f76e8106a80 2024-09-19T08:45:31.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.169+0000 7f76e5ffb640 1 -- 172.21.15.164:0/2161661191 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e0007650 con 0x7f76e8106a80 2024-09-19T08:45:31.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.169+0000 7f76ee789640 1 -- 172.21.15.164:0/2161661191 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e81a5110 con 0x7f76e8106a80 2024-09-19T08:45:31.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.170+0000 7f76ee789640 1 -- 172.21.15.164:0/2161661191 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76e8100f60 con 0x7f76e8106a80 2024-09-19T08:45:31.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.172+0000 7f76e5ffb640 1 -- 172.21.15.164:0/2161661191 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f76e001c070 con 0x7f76e8106a80 2024-09-19T08:45:31.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.173+0000 7f76e5ffb640 1 --2- 172.21.15.164:0/2161661191 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f76d40a3350 0x7f76d40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:31.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.173+0000 7f76e7fff640 1 --2- 172.21.15.164:0/2161661191 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f76d40a3350 0x7f76d40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:31.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.173+0000 7f76e5ffb640 1 -- 172.21.15.164:0/2161661191 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f76e00cf2b0 con 0x7f76e8106a80 2024-09-19T08:45:31.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.173+0000 7f76e7fff640 1 --2- 172.21.15.164:0/2161661191 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f76d40a3350 0x7f76d40a5810 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f76d8009890 tx=0x7f76d8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:31.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.176+0000 7f76e5ffb640 1 -- 172.21.15.164:0/2161661191 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f76e009c080 con 0x7f76e8106a80 2024-09-19T08:45:31.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.380+0000 7f76ee789640 1 -- 172.21.15.164:0/2161661191 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76e802d7c0 con 0x7f76e8106a80 2024-09-19T08:45:31.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.380+0000 7f76e5ffb640 1 -- 172.21.15.164:0/2161661191 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11984 (secure 0 0 0) 0x7f76e009b980 con 0x7f76e8106a80 2024-09-19T08:45:31.380 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:45:31.380 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":27,"btime":"2024-09-19T08:44:05:632372+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:44:05.632353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24442,24424,24449,14595]},"id":1}]} 2024-09-19T08:45:31.380 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 27 2024-09-19T08:45:31.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.382+0000 7f76ee789640 1 -- 172.21.15.164:0/2161661191 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f76d40a3350 msgr2=0x7f76d40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:31.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.382+0000 7f76ee789640 1 --2- 172.21.15.164:0/2161661191 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f76d40a3350 0x7f76d40a5810 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f76d8009890 tx=0x7f76d8008040 comp rx=0 tx=0).stop 2024-09-19T08:45:31.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.382+0000 7f76ee789640 1 -- 172.21.15.164:0/2161661191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76e8106a80 msgr2=0x7f76e81a4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:31.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.382+0000 7f76ee789640 1 --2- 172.21.15.164:0/2161661191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76e8106a80 0x7f76e81a4470 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f76e000b970 tx=0x7f76e000be30 comp rx=0 tx=0).stop 2024-09-19T08:45:31.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.383+0000 7f76ee789640 1 -- 172.21.15.164:0/2161661191 shutdown_connections 2024-09-19T08:45:31.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.383+0000 7f76ee789640 1 --2- 172.21.15.164:0/2161661191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76e8106a80 0x7f76e81a4470 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.383+0000 7f76ee789640 1 --2- 172.21.15.164:0/2161661191 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f76d40a3350 0x7f76d40a5810 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.383+0000 7f76ee789640 1 --2- 172.21.15.164:0/2161661191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76e8101420 0x7f76e8199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.383+0000 7f76ee789640 1 --2- 172.21.15.164:0/2161661191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76e8100a80 0x7f76e81993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.383+0000 7f76ee789640 1 -- 172.21.15.164:0/2161661191 >> 172.21.15.164:0/2161661191 conn(0x7f76e80fc7a0 msgr2=0x7f76e80fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:31.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.383+0000 7f76ee789640 1 -- 172.21.15.164:0/2161661191 shutdown_connections 2024-09-19T08:45:31.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:31.383+0000 7f76ee789640 1 -- 172.21.15.164:0/2161661191 wait complete. 2024-09-19T08:45:32.016 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:31 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2161661191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:45:32.043 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T08:45:32.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:31 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2161661191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:45:32.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:31 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2161661191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:45:32.289 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:45:32.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.821+0000 7fac1301e640 1 -- 172.21.15.164:0/2019402090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac0c1082a0 msgr2=0x7fac0c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:32.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.821+0000 7fac1301e640 1 --2- 172.21.15.164:0/2019402090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac0c1082a0 0x7fac0c10ec40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fac04009cd0 tx=0x7fac0402a910 comp rx=0 tx=0).stop 2024-09-19T08:45:32.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.821+0000 7fac1301e640 1 -- 172.21.15.164:0/2019402090 shutdown_connections 2024-09-19T08:45:32.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.821+0000 7fac1301e640 1 --2- 172.21.15.164:0/2019402090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac0c1082a0 0x7fac0c10ec40 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:32.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.821+0000 7fac1301e640 1 --2- 172.21.15.164:0/2019402090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac0c1036e0 0x7fac0c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:32.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.821+0000 7fac1301e640 1 --2- 172.21.15.164:0/2019402090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac0c102d30 0x7fac0c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:32.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.821+0000 7fac1301e640 1 -- 172.21.15.164:0/2019402090 >> 172.21.15.164:0/2019402090 conn(0x7fac0c0fc7d0 msgr2=0x7fac0c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:32.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.821+0000 7fac1301e640 1 -- 172.21.15.164:0/2019402090 shutdown_connections 2024-09-19T08:45:32.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.821+0000 7fac1301e640 1 -- 172.21.15.164:0/2019402090 wait complete. 2024-09-19T08:45:32.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.821+0000 7fac1301e640 1 Processor -- start 2024-09-19T08:45:32.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.821+0000 7fac1301e640 1 -- start start 2024-09-19T08:45:32.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.822+0000 7fac1301e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac0c102d30 0x7fac0c198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:32.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.822+0000 7fac1301e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac0c1036e0 0x7fac0c198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:32.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.822+0000 7fac1301e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac0c1082a0 0x7fac0c19d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:32.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.822+0000 7fac1301e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fac0c112160 con 0x7fac0c102d30 2024-09-19T08:45:32.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.822+0000 7fac1301e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fac0c111fe0 con 0x7fac0c1036e0 2024-09-19T08:45:32.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.822+0000 7fac1301e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fac0c1122e0 con 0x7fac0c1082a0 2024-09-19T08:45:32.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.822+0000 7fac10dba640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac0c102d30 0x7fac0c198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:32.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.822+0000 7fac10dba640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac0c102d30 0x7fac0c198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59852/0 (socket says 172.21.15.164:59852) 2024-09-19T08:45:32.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.822+0000 7fac10dba640 1 -- 172.21.15.164:0/3412260293 learned_addr learned my addr 172.21.15.164:0/3412260293 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:32.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.822+0000 7fac10dba640 1 -- 172.21.15.164:0/3412260293 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac0c1082a0 msgr2=0x7fac0c19d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:32.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.822+0000 7fac10dba640 1 --2- 172.21.15.164:0/3412260293 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac0c1082a0 0x7fac0c19d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:32.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.822+0000 7fac10dba640 1 -- 172.21.15.164:0/3412260293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac0c1036e0 msgr2=0x7fac0c198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:32.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.822+0000 7fac0bfff640 1 --2- 172.21.15.164:0/3412260293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac0c1036e0 0x7fac0c198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:32.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.822+0000 7fac10dba640 1 --2- 172.21.15.164:0/3412260293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac0c1036e0 0x7fac0c198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:32.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.822+0000 7fac10dba640 1 -- 172.21.15.164:0/3412260293 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac0c19d6f0 con 0x7fac0c102d30 2024-09-19T08:45:32.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.822+0000 7fac10dba640 1 --2- 172.21.15.164:0/3412260293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac0c102d30 0x7fac0c198740 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fabfc00b4a0 tx=0x7fabfc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:32.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.822+0000 7fac09ffb640 1 -- 172.21.15.164:0/3412260293 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabfc014070 con 0x7fac0c102d30 2024-09-19T08:45:32.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.822+0000 7fac09ffb640 1 -- 172.21.15.164:0/3412260293 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabfc010af0 con 0x7fac0c102d30 2024-09-19T08:45:32.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.822+0000 7fac1301e640 1 -- 172.21.15.164:0/3412260293 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac0c19d9e0 con 0x7fac0c102d30 2024-09-19T08:45:32.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.822+0000 7fac09ffb640 1 -- 172.21.15.164:0/3412260293 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabfc017d10 con 0x7fac0c102d30 2024-09-19T08:45:32.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.823+0000 7fac1301e640 1 -- 172.21.15.164:0/3412260293 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac0c1a52c0 con 0x7fac0c102d30 2024-09-19T08:45:32.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.824+0000 7fac1301e640 1 -- 172.21.15.164:0/3412260293 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac0c06a850 con 0x7fac0c102d30 2024-09-19T08:45:32.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.825+0000 7fac09ffb640 1 -- 172.21.15.164:0/3412260293 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fabfc017070 con 0x7fac0c102d30 2024-09-19T08:45:32.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.825+0000 7fac09ffb640 1 --2- 172.21.15.164:0/3412260293 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fabf80a3240 0x7fabf80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:32.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.825+0000 7fac0bfff640 1 --2- 172.21.15.164:0/3412260293 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fabf80a3240 0x7fabf80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:32.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.825+0000 7fac09ffb640 1 -- 172.21.15.164:0/3412260293 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fabfc0cecf0 con 0x7fac0c102d30 2024-09-19T08:45:32.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.826+0000 7fac0bfff640 1 --2- 172.21.15.164:0/3412260293 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fabf80a3240 0x7fabf80a5700 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fac0c0fedc0 tx=0x7fabf4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:32.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.830+0000 7fac09ffb640 1 -- 172.21.15.164:0/3412260293 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fabfc0d3070 con 0x7fac0c102d30 2024-09-19T08:45:32.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:32.998+0000 7fac1301e640 1 -- 172.21.15.164:0/3412260293 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac0c10a2b0 con 0x7fac0c102d30 2024-09-19T08:45:32.998 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:32 smithi164 ceph-mon[30928]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T08:45:32.998 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:32 smithi164 ceph-mon[30928]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-19T08:45:32.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:33.000+0000 7fac09ffb640 1 -- 172.21.15.164:0/3412260293 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22702 (secure 0 0 0) 0x7fabfc09bab0 con 0x7fac0c102d30 2024-09-19T08:45:33.000 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:45:33.000 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":82,"fsid":"1e4b0d74-7662-11ef-bceb-c7b262605968","created":"2024-09-19T08:36:09.564509+0000","modified":"2024-09-19T08:44:02.811236+0000","last_up_change":"2024-09-19T08:43:01.260852+0000","last_in_change":"2024-09-19T08:42:37.492464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:39:13.961749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:43:49.613471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T08:43:51.570675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33b11f37-6ac5-41fc-8cfd-1b08c7eef688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6803","nonce":663038243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6805","nonce":663038243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6809","nonce":663038243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6807","nonce":663038243}]},"public_addr":"172.21.15.164:6803/663038243","cluster_addr":"172.21.15.164:6805/663038243","heartbeat_back_addr":"172.21.15.164:6809/663038243","heartbeat_front_addr":"172.21.15.164:6807/663038243","state":["exists","up"]},{"osd":1,"uuid":"6de64b0a-46e2-4a98-8005-a326a64bc87d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6801","nonce":2167066371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6803","nonce":2167066371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6807","nonce":2167066371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6805","nonce":2167066371}]},"public_addr":"172.21.15.170:6801/2167066371","cluster_addr":"172.21.15.170:6803/2167066371","heartbeat_back_addr":"172.21.15.170:6807/2167066371","heartbeat_front_addr":"172.21.15.170:6805/2167066371","state":["exists","up"]},{"osd":2,"uuid":"d4158cc7-9822-42a7-9601-d65089907584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6801","nonce":1646561048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6803","nonce":1646561048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6807","nonce":1646561048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6805","nonce":1646561048}]},"public_addr":"172.21.15.173:6801/1646561048","cluster_addr":"172.21.15.173:6803/1646561048","heartbeat_back_addr":"172.21.15.173:6807/1646561048","heartbeat_front_addr":"172.21.15.173:6805/1646561048","state":["exists","up"]},{"osd":3,"uuid":"0cf3f7f1-3a8c-42ce-88a9-9bea52e02221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6811","nonce":406795560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6813","nonce":406795560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6817","nonce":406795560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6815","nonce":406795560}]},"public_addr":"172.21.15.164:6811/406795560","cluster_addr":"172.21.15.164:6813/406795560","heartbeat_back_addr":"172.21.15.164:6817/406795560","heartbeat_front_addr":"172.21.15.164:6815/406795560","state":["exists","up"]},{"osd":4,"uuid":"6dca048a-2860-4a02-8d8e-450b30bc1a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6809","nonce":4120832165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6811","nonce":4120832165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6815","nonce":4120832165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6813","nonce":4120832165}]},"public_addr":"172.21.15.170:6809/4120832165","cluster_addr":"172.21.15.170:6811/4120832165","heartbeat_back_addr":"172.21.15.170:6815/4120832165","heartbeat_front_addr":"172.21.15.170:6813/4120832165","state":["exists","up"]},{"osd":5,"uuid":"3822521f-69d6-4141-a83f-5fc9cecc7b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6809","nonce":2914653552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6811","nonce":2914653552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6815","nonce":2914653552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6813","nonce":2914653552}]},"public_addr":"172.21.15.173:6809/2914653552","cluster_addr":"172.21.15.173:6811/2914653552","heartbeat_back_addr":"172.21.15.173:6815/2914653552","heartbeat_front_addr":"172.21.15.173:6813/2914653552","state":["exists","up"]},{"osd":6,"uuid":"0f5a965e-2eb2-4c24-b089-577e0639c160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6819","nonce":1138341393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6821","nonce":1138341393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6825","nonce":1138341393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6823","nonce":1138341393}]},"public_addr":"172.21.15.164:6819/1138341393","cluster_addr":"172.21.15.164:6821/1138341393","heartbeat_back_addr":"172.21.15.164:6825/1138341393","heartbeat_front_addr":"172.21.15.164:6823/1138341393","state":["exists","up"]},{"osd":7,"uuid":"f16cb2c3-639b-46ae-a740-1bd28ca1863c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6817","nonce":2124160360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6819","nonce":2124160360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6823","nonce":2124160360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6821","nonce":2124160360}]},"public_addr":"172.21.15.170:6817/2124160360","cluster_addr":"172.21.15.170:6819/2124160360","heartbeat_back_addr":"172.21.15.170:6823/2124160360","heartbeat_front_addr":"172.21.15.170:6821/2124160360","state":["exists","up"]},{"osd":8,"uuid":"161cff01-46ad-48e3-b999-a5473a2ad618","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6817","nonce":2285175061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6819","nonce":2285175061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6823","nonce":2285175061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6821","nonce":2285175061}]},"public_addr":"172.21.15.173:6817/2285175061","cluster_addr":"172.21.15.173:6819/2285175061","heartbeat_back_addr":"172.21.15.173:6823/2285175061","heartbeat_front_addr":"172.21.15.173:6821/2285175061","state":["exists","up"]},{"osd":9,"uuid":"994666e8-8fbd-4231-ac67-81a79aaf7c2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6827","nonce":3918847952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6829","nonce":3918847952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6833","nonce":3918847952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6831","nonce":3918847952}]},"public_addr":"172.21.15.164:6827/3918847952","cluster_addr":"172.21.15.164:6829/3918847952","heartbeat_back_addr":"172.21.15.164:6833/3918847952","heartbeat_front_addr":"172.21.15.164:6831/3918847952","state":["exists","up"]},{"osd":10,"uuid":"ff5e9099-cdb4-4905-b888-5b67320ef76b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6825","nonce":2707848227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6827","nonce":2707848227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6831","nonce":2707848227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6829","nonce":2707848227}]},"public_addr":"172.21.15.170:6825/2707848227","cluster_addr":"172.21.15.170:6827/2707848227","heartbeat_back_addr":"172.21.15.170:6831/2707848227","heartbeat_front_addr":"172.21.15.170:6829/2707848227","state":["exists","up"]},{"osd":11,"uuid":"4d93088f-e791-4938-a207-12f15b56abd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6825","nonce":2831924738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6827","nonce":2831924738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6831","nonce":2831924738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6829","nonce":2831924738}]},"public_addr":"172.21.15.173:6825/2831924738","cluster_addr":"172.21.15.173:6827/2831924738","heartbeat_back_addr":"172.21.15.173:6831/2831924738","heartbeat_front_addr":"172.21.15.173:6829/2831924738","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:21.787352+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:46.012323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:39:11.353040+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:39:37.015124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:02.063379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:27.892865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:53.372041+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:41:17.855284+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:41:43.389882+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:08.956895+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:34.069779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:59.303897+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.164:0/339032808":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6801/1173822158":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6800/1173822158":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6801/2805828009":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1837164919":"2024-09-20T08:37:01.663065+0000","172.21.15.164:0/4018459143":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6800/2805828009":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/2770797546":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1905394497":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1369265436":"2024-09-20T08:36:33.856642+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:45:33.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:33.002+0000 7fac1301e640 1 -- 172.21.15.164:0/3412260293 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fabf80a3240 msgr2=0x7fabf80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:33.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:33.002+0000 7fac1301e640 1 --2- 172.21.15.164:0/3412260293 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fabf80a3240 0x7fabf80a5700 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fac0c0fedc0 tx=0x7fabf4008040 comp rx=0 tx=0).stop 2024-09-19T08:45:33.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:33.002+0000 7fac1301e640 1 -- 172.21.15.164:0/3412260293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac0c102d30 msgr2=0x7fac0c198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:33.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:33.002+0000 7fac1301e640 1 --2- 172.21.15.164:0/3412260293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac0c102d30 0x7fac0c198740 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fabfc00b4a0 tx=0x7fabfc00b960 comp rx=0 tx=0).stop 2024-09-19T08:45:33.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:33.002+0000 7fac1301e640 1 -- 172.21.15.164:0/3412260293 shutdown_connections 2024-09-19T08:45:33.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:33.002+0000 7fac1301e640 1 --2- 172.21.15.164:0/3412260293 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fabf80a3240 0x7fabf80a5700 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:33.002+0000 7fac1301e640 1 --2- 172.21.15.164:0/3412260293 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac0c1082a0 0x7fac0c19d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:33.002+0000 7fac1301e640 1 --2- 172.21.15.164:0/3412260293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac0c1036e0 0x7fac0c198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:33.002+0000 7fac1301e640 1 --2- 172.21.15.164:0/3412260293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac0c102d30 0x7fac0c198740 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:33.002+0000 7fac1301e640 1 -- 172.21.15.164:0/3412260293 >> 172.21.15.164:0/3412260293 conn(0x7fac0c0fc7d0 msgr2=0x7fac0c106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:33.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:33.002+0000 7fac1301e640 1 -- 172.21.15.164:0/3412260293 shutdown_connections 2024-09-19T08:45:33.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:33.002+0000 7fac1301e640 1 -- 172.21.15.164:0/3412260293 wait complete. 2024-09-19T08:45:33.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:32 smithi170 ceph-mon[35072]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T08:45:33.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:32 smithi170 ceph-mon[35072]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-19T08:45:33.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:32 smithi173 ceph-mon[35306]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T08:45:33.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:32 smithi173 ceph-mon[35306]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-19T08:45:33.704 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T08:43:53.386044+0000', 'modified': '2024-09-19T08:44:05.632353+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24479, 'mds_1': 24449, 'mds_2': 24442, 'mds_3': 24424, 'mds_4': 14595}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14583': {'gid': 14583, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6839/538276454', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6838', 'nonce': 538276454}, {'type': 'v1', 'addr': '172.21.15.164:6839', 'nonce': 538276454}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14595': {'gid': 14595, 'name': 'a', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.164:6841/2221659234', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6840', 'nonce': 2221659234}, {'type': 'v1', 'addr': '172.21.15.164:6841', 'nonce': 2221659234}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24401': {'gid': 24401, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6833/2726071671', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6832', 'nonce': 2726071671}, {'type': 'v1', 'addr': '172.21.15.170:6833', 'nonce': 2726071671}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24403': {'gid': 24403, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.173:6833/486946900', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6832', 'nonce': 486946900}, {'type': 'v1', 'addr': '172.21.15.173:6833', 'nonce': 486946900}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24418': {'gid': 24418, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.173:6835/947317705', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6834', 'nonce': 947317705}, {'type': 'v1', 'addr': '172.21.15.173:6835', 'nonce': 947317705}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6835/321254273', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6834', 'nonce': 321254273}, {'type': 'v1', 'addr': '172.21.15.170:6835', 'nonce': 321254273}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24424': {'gid': 24424, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.173:6837/2431090203', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6836', 'nonce': 2431090203}, {'type': 'v1', 'addr': '172.21.15.173:6837', 'nonce': 2431090203}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'i', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.173:6839/1089658493', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6838', 'nonce': 1089658493}, {'type': 'v1', 'addr': '172.21.15.173:6839', 'nonce': 1089658493}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'h', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.170:6837/2350302392', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6836', 'nonce': 2350302392}, {'type': 'v1', 'addr': '172.21.15.170:6837', 'nonce': 2350302392}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.170:6839/1854887138', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6838', 'nonce': 1854887138}, {'type': 'v1', 'addr': '172.21.15.170:6839', 'nonce': 1854887138}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24479, 'qdb_cluster': [24479, 24442, 24424, 24449, 14595]} 2024-09-19T08:45:33.704 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-19T08:45:33.704 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.a status 2024-09-19T08:45:33.953 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:45:34.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:33 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3412260293' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:45:34.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:33 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3412260293' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:45:34.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:33 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3412260293' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:45:34.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.434+0000 7fbd54884640 1 -- 172.21.15.164:0/1271925196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd50102ef0 msgr2=0x7fbd5010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:34.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.434+0000 7fbd54884640 1 --2- 172.21.15.164:0/1271925196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd50102ef0 0x7fbd5010da40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbd48009900 tx=0x7fbd4802a480 comp rx=0 tx=0).stop 2024-09-19T08:45:34.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.434+0000 7fbd54884640 1 -- 172.21.15.164:0/1271925196 shutdown_connections 2024-09-19T08:45:34.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.434+0000 7fbd54884640 1 --2- 172.21.15.164:0/1271925196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd50102ef0 0x7fbd5010da40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:34.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.434+0000 7fbd54884640 1 --2- 172.21.15.164:0/1271925196 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd50102570 0x7fbd501029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:34.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.434+0000 7fbd54884640 1 --2- 172.21.15.164:0/1271925196 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd50100e20 0x7fbd50101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:34.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.434+0000 7fbd54884640 1 -- 172.21.15.164:0/1271925196 >> 172.21.15.164:0/1271925196 conn(0x7fbd500fc7d0 msgr2=0x7fbd500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:34.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.435+0000 7fbd54884640 1 -- 172.21.15.164:0/1271925196 shutdown_connections 2024-09-19T08:45:34.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.435+0000 7fbd54884640 1 -- 172.21.15.164:0/1271925196 wait complete. 2024-09-19T08:45:34.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.435+0000 7fbd54884640 1 Processor -- start 2024-09-19T08:45:34.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.435+0000 7fbd54884640 1 -- start start 2024-09-19T08:45:34.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.435+0000 7fbd54884640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd50100e20 0x7fbd501971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:34.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.435+0000 7fbd54884640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd50102570 0x7fbd50197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:34.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.436+0000 7fbd54884640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd50102ef0 0x7fbd50197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:34.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.436+0000 7fbd54884640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbd50110f80 con 0x7fbd50100e20 2024-09-19T08:45:34.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.436+0000 7fbd54884640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbd50110e00 con 0x7fbd50102ef0 2024-09-19T08:45:34.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.436+0000 7fbd54884640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbd50111100 con 0x7fbd50102570 2024-09-19T08:45:34.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.436+0000 7fbd4e59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd50100e20 0x7fbd501971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:34.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.436+0000 7fbd4e59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd50100e20 0x7fbd501971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43340/0 (socket says 172.21.15.164:43340) 2024-09-19T08:45:34.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.436+0000 7fbd4e59c640 1 -- 172.21.15.164:0/2619720130 learned_addr learned my addr 172.21.15.164:0/2619720130 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:34.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.436+0000 7fbd4ed9d640 1 --2- 172.21.15.164:0/2619720130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd50102ef0 0x7fbd50197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:34.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.436+0000 7fbd4dd9b640 1 --2- 172.21.15.164:0/2619720130 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd50102570 0x7fbd50197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:34.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.436+0000 7fbd4e59c640 1 -- 172.21.15.164:0/2619720130 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd50102570 msgr2=0x7fbd50197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:34.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.436+0000 7fbd4e59c640 1 --2- 172.21.15.164:0/2619720130 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd50102570 0x7fbd50197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:34.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.436+0000 7fbd4e59c640 1 -- 172.21.15.164:0/2619720130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd50102ef0 msgr2=0x7fbd50197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:34.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.436+0000 7fbd4e59c640 1 --2- 172.21.15.164:0/2619720130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd50102ef0 0x7fbd50197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:34.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.436+0000 7fbd4e59c640 1 -- 172.21.15.164:0/2619720130 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd501a6a30 con 0x7fbd50100e20 2024-09-19T08:45:34.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.436+0000 7fbd4e59c640 1 --2- 172.21.15.164:0/2619720130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd50100e20 0x7fbd501971c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbd4000ea80 tx=0x7fbd4000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:34.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.436+0000 7fbd3f7fe640 1 -- 172.21.15.164:0/2619720130 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4000ce40 con 0x7fbd50100e20 2024-09-19T08:45:34.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.436+0000 7fbd54884640 1 -- 172.21.15.164:0/2619720130 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd501a6d20 con 0x7fbd50100e20 2024-09-19T08:45:34.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.436+0000 7fbd3f7fe640 1 -- 172.21.15.164:0/2619720130 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd40005020 con 0x7fbd50100e20 2024-09-19T08:45:34.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.436+0000 7fbd3f7fe640 1 -- 172.21.15.164:0/2619720130 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd40011760 con 0x7fbd50100e20 2024-09-19T08:45:34.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.436+0000 7fbd54884640 1 -- 172.21.15.164:0/2619720130 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd501a71b0 con 0x7fbd50100e20 2024-09-19T08:45:34.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.438+0000 7fbd3f7fe640 1 -- 172.21.15.164:0/2619720130 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fbd400051d0 con 0x7fbd50100e20 2024-09-19T08:45:34.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.439+0000 7fbd56c5d740 1 Processor -- start 2024-09-19T08:45:34.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.439+0000 7fbd56c5d740 1 -- start start 2024-09-19T08:45:34.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.439+0000 7fbd56c5d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9c7fb29d0 0x55c9c7fb2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:34.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.439+0000 7fbd56c5d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c7fb32f0 0x55c9c7fbd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:34.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.439+0000 7fbd56c5d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c9c7fbdbb0 0x55c9c7fbffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:34.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.439+0000 7fbd56c5d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c9c7f19090 con 0x55c9c7fb29d0 2024-09-19T08:45:34.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.439+0000 7fbd56c5d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c9c7ec0a70 con 0x55c9c7fb32f0 2024-09-19T08:45:34.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.439+0000 7fbd56c5d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c9c7f15050 con 0x55c9c7fbdbb0 2024-09-19T08:45:34.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.440+0000 7fbd4ed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9c7fb29d0 0x55c9c7fb2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:34.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.440+0000 7fbd4ed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9c7fb29d0 0x55c9c7fb2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43354/0 (socket says 172.21.15.164:43354) 2024-09-19T08:45:34.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.440+0000 7fbd4ed9d640 1 -- 172.21.15.164:0/3008943064 learned_addr learned my addr 172.21.15.164:0/3008943064 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:34.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.440+0000 7fbd3f7fe640 1 --2- 172.21.15.164:0/2619720130 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fbd380a3240 0x7fbd380a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:34.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.440+0000 7fbd4e59c640 1 --2- 172.21.15.164:0/3008943064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c7fb32f0 0x55c9c7fbd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:34.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.440+0000 7fbd4ed9d640 1 --2- 172.21.15.164:0/2619720130 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fbd380a3240 0x7fbd380a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:34.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.440+0000 7fbd4dd9b640 1 --2- 172.21.15.164:0/3008943064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c9c7fbdbb0 0x55c9c7fbffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:34.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.440+0000 7fbd3f7fe640 1 -- 172.21.15.164:0/2619720130 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fbd400cfe80 con 0x7fbd50100e20 2024-09-19T08:45:34.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.440+0000 7fbd4ed9d640 1 -- 172.21.15.164:0/3008943064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c9c7fbdbb0 msgr2=0x55c9c7fbffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:34.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.440+0000 7fbd4ed9d640 1 --2- 172.21.15.164:0/3008943064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c9c7fbdbb0 0x55c9c7fbffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:34.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.440+0000 7fbd4ed9d640 1 -- 172.21.15.164:0/3008943064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c7fb32f0 msgr2=0x55c9c7fbd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:34.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.440+0000 7fbd4ed9d640 1 --2- 172.21.15.164:0/3008943064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c7fb32f0 0x55c9c7fbd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:34.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.440+0000 7fbd4ed9d640 1 -- 172.21.15.164:0/3008943064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9c7ecd130 con 0x55c9c7fb29d0 2024-09-19T08:45:34.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.440+0000 7fbd4dd9b640 1 --2- 172.21.15.164:0/3008943064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c9c7fbdbb0 0x55c9c7fbffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:34.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.440+0000 7fbd4ed9d640 1 --2- 172.21.15.164:0/3008943064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9c7fb29d0 0x55c9c7fb2db0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbd48009900 tx=0x7fbd480045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a237b6d76a2d93 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:34.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.440+0000 7fbd3cff9640 1 -- 172.21.15.164:0/3008943064 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd48035040 con 0x55c9c7fb29d0 2024-09-19T08:45:34.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.440+0000 7fbd3cff9640 1 -- 172.21.15.164:0/3008943064 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd4804d070 con 0x55c9c7fb29d0 2024-09-19T08:45:34.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.440+0000 7fbd3cff9640 1 -- 172.21.15.164:0/3008943064 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4803d6c0 con 0x55c9c7fb29d0 2024-09-19T08:45:34.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.440+0000 7fbd4ed9d640 1 --2- 172.21.15.164:0/2619720130 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fbd380a3240 0x7fbd380a5700 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbd4804c000 tx=0x7fbd4804a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:34.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.441+0000 7fbd56c5d740 1 -- 172.21.15.164:0/3008943064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9c7fb29d0 msgr2=0x55c9c7fb2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:34.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.441+0000 7fbd56c5d740 1 --2- 172.21.15.164:0/3008943064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9c7fb29d0 0x55c9c7fb2db0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbd48009900 tx=0x7fbd480045b0 comp rx=0 tx=0).stop 2024-09-19T08:45:34.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.441+0000 7fbd56c5d740 1 -- 172.21.15.164:0/3008943064 shutdown_connections 2024-09-19T08:45:34.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.441+0000 7fbd56c5d740 1 --2- 172.21.15.164:0/3008943064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c9c7fbdbb0 0x55c9c7fbffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:34.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.441+0000 7fbd56c5d740 1 --2- 172.21.15.164:0/3008943064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c7fb32f0 0x55c9c7fbd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:34.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.441+0000 7fbd56c5d740 1 --2- 172.21.15.164:0/3008943064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9c7fb29d0 0x55c9c7fb2db0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:34.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.441+0000 7fbd56c5d740 1 -- 172.21.15.164:0/3008943064 >> 172.21.15.164:0/3008943064 conn(0x55c9c7faef60 msgr2=0x55c9c7faf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:34.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.441+0000 7fbd56c5d740 1 -- 172.21.15.164:0/3008943064 shutdown_connections 2024-09-19T08:45:34.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.441+0000 7fbd56c5d740 1 -- 172.21.15.164:0/3008943064 wait complete. 2024-09-19T08:45:34.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.442+0000 7fbd56c5d740 1 -- start start 2024-09-19T08:45:34.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.442+0000 7fbd56c5d740 1 Processor -- start 2024-09-19T08:45:34.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.442+0000 7fbd56c5d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c9c7fb29d0 0x55c9c7fbd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:34.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.442+0000 7fbd56c5d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c7fb32f0 0x55c9c7fba7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:34.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.442+0000 7fbd56c5d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9c7fbdbb0 0x55c9c7fbad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:34.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.442+0000 7fbd56c5d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c9c7fc0780 con 0x55c9c7fbdbb0 2024-09-19T08:45:34.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.442+0000 7fbd56c5d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c9c7ed3f20 con 0x55c9c7fb32f0 2024-09-19T08:45:34.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.442+0000 7fbd56c5d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c9c7fc0900 con 0x55c9c7fb29d0 2024-09-19T08:45:34.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.442+0000 7fbd56c5d740 10 client.0 _ll_register_callbacks cb 0x55c9c7e9d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:45:34.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.442+0000 7fbd4ed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9c7fbdbb0 0x55c9c7fbad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:34.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.442+0000 7fbd4ed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9c7fbdbb0 0x55c9c7fbad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43366/0 (socket says 172.21.15.164:43366) 2024-09-19T08:45:34.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.442+0000 7fbd4e59c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c9c7fb29d0 0x55c9c7fbd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:34.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.442+0000 7fbd4ed9d640 1 -- 172.21.15.164:0/157656489 learned_addr learned my addr 172.21.15.164:0/157656489 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:34.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.442+0000 7fbd4dd9b640 1 --2- 172.21.15.164:0/157656489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c7fb32f0 0x55c9c7fba7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:34.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.442+0000 7fbd4e59c640 1 -- 172.21.15.164:0/157656489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c7fb32f0 msgr2=0x55c9c7fba7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:34.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.442+0000 7fbd4e59c640 1 --2- 172.21.15.164:0/157656489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c7fb32f0 0x55c9c7fba7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:34.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.442+0000 7fbd4e59c640 1 -- 172.21.15.164:0/157656489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9c7fbdbb0 msgr2=0x55c9c7fbad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:34.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.442+0000 7fbd4e59c640 1 --2- 172.21.15.164:0/157656489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9c7fbdbb0 0x55c9c7fbad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:34.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.442+0000 7fbd4e59c640 1 -- 172.21.15.164:0/157656489 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9c7c8a2b0 con 0x55c9c7fb29d0 2024-09-19T08:45:34.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.442+0000 7fbd4ed9d640 1 --2- 172.21.15.164:0/157656489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9c7fbdbb0 0x55c9c7fbad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:45:34.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.442+0000 7fbd4e59c640 1 --2- 172.21.15.164:0/157656489 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c9c7fb29d0 0x55c9c7fbd480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbd400cd980 tx=0x7fbd4006b290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9fa09859529d33a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:34.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.442+0000 7fbd35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-09-19T08:45:34.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.443+0000 7fbd35ffb640 1 -- 172.21.15.164:0/157656489 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4001f070 con 0x55c9c7fb29d0 2024-09-19T08:45:34.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.443+0000 7fbd35ffb640 1 -- 172.21.15.164:0/157656489 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fbd400a4c80 con 0x55c9c7fb29d0 2024-09-19T08:45:34.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.443+0000 7fbd56c5d740 1 -- 172.21.15.164:0/157656489 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd4804d070 con 0x55c9c7fb29d0 2024-09-19T08:45:34.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.443+0000 7fbd35ffb640 1 -- 172.21.15.164:0/157656489 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbd400a5110 con 0x55c9c7fb29d0 2024-09-19T08:45:34.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.443+0000 7fbd56c5d740 10 client.24730 fetch_fsmap learned FSMap version 27 2024-09-19T08:45:34.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.443+0000 7fbd56c5d740 1 -- 172.21.15.164:0/157656489 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c9c7fcc0c0 con 0x55c9c7fb29d0 2024-09-19T08:45:34.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.443+0000 7fbd35ffb640 1 -- 172.21.15.164:0/157656489 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd4804d070 con 0x55c9c7fb29d0 2024-09-19T08:45:34.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.443+0000 7fbd35ffb640 1 -- 172.21.15.164:0/157656489 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbd4000f040 con 0x55c9c7fb29d0 2024-09-19T08:45:34.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.443+0000 7fbd56c5d740 10 client.24730 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T08:45:34.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.443+0000 7fbd56c5d740 10 client.24730 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T08:45:34.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.443+0000 7fbd56c5d740 20 client.24730 populate_metadata read hostname 'smithi164' 2024-09-19T08:45:34.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.443+0000 7fbd56c5d740 1 --2- 172.21.15.164:0/157656489 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55c9c7fce650 0x55c9c7fcfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:34.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.443+0000 7fbd56c5d740 4 client.24730 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:45:34.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.443+0000 7fbd56c5d740 1 -- 172.21.15.164:0/157656489 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9c7f26600 con 0x55c9c7fce650 2024-09-19T08:45:34.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.443+0000 7fbd4dd9b640 1 --2- 172.21.15.164:0/157656489 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55c9c7fce650 0x55c9c7fcfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:34.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.444+0000 7fbd35ffb640 10 client.24730 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:45:34.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.444+0000 7fbd4dd9b640 1 --2- 172.21.15.164:0/157656489 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55c9c7fce650 0x55c9c7fcfac0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4fe5555a72f95436 server_cookie=b0e349d970806801 in_seq=0 out_seq=0 2024-09-19T08:45:34.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.444+0000 7fbd35ffb640 1 -- 172.21.15.164:0/157656489 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c9c7f26600 con 0x55c9c7fce650 2024-09-19T08:45:34.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.444+0000 7fbd35ffb640 10 client.24730 handle_command_reply: tid=0 2024-09-19T08:45:34.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.444+0000 7fbd56c5d740 1 client.24730 shutdown 2024-09-19T08:45:34.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.444+0000 7fbd56c5d740 1 -- 172.21.15.164:0/157656489 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c9c7fb29d0 msgr2=0x55c9c7fbd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:34.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.444+0000 7fbd56c5d740 1 --2- 172.21.15.164:0/157656489 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c9c7fb29d0 0x55c9c7fbd480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbd400cd980 tx=0x7fbd4006b290 comp rx=0 tx=0).stop 2024-09-19T08:45:34.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.445+0000 7fbd56c5d740 1 -- 172.21.15.164:0/157656489 shutdown_connections 2024-09-19T08:45:34.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.445+0000 7fbd56c5d740 1 --2- 172.21.15.164:0/157656489 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55c9c7fce650 0x55c9c7fcfac0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:34.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.445+0000 7fbd56c5d740 1 --2- 172.21.15.164:0/157656489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9c7fbdbb0 0x55c9c7fbad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:34.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.445+0000 7fbd56c5d740 1 --2- 172.21.15.164:0/157656489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c7fb32f0 0x55c9c7fba7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:34.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.445+0000 7fbd56c5d740 1 --2- 172.21.15.164:0/157656489 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c9c7fb29d0 0x55c9c7fbd480 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:34.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.445+0000 7fbd56c5d740 1 -- 172.21.15.164:0/157656489 >> 172.21.15.164:0/157656489 conn(0x55c9c7faef60 msgr2=0x55c9c7fb4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:34.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.445+0000 7fbd35ffb640 0 client.24730 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:45:34.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.445+0000 7fbd56c5d740 1 -- 172.21.15.164:0/157656489 shutdown_connections 2024-09-19T08:45:34.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.445+0000 7fbd56c5d740 1 -- 172.21.15.164:0/157656489 wait complete. 2024-09-19T08:45:34.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.445+0000 7fbd56c5d740 20 client.24730 trim_cache size 0 max 16384 2024-09-19T08:45:34.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.462+0000 7fbd56c5d740 1 Processor -- start 2024-09-19T08:45:34.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.462+0000 7fbd56c5d740 1 -- start start 2024-09-19T08:45:34.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.462+0000 7fbd56c5d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9c7fb29d0 0x55c9c7fce650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:34.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.462+0000 7fbd56c5d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c7fb32f0 0x55c9c7fced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:34.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.462+0000 7fbd56c5d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c9c7fbdbb0 0x55c9c7fcdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:34.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.462+0000 7fbd56c5d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c9c7fcde00 con 0x55c9c7fb29d0 2024-09-19T08:45:34.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.462+0000 7fbd56c5d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c9c7fce4d0 con 0x55c9c7fb32f0 2024-09-19T08:45:34.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.462+0000 7fbd56c5d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c9c7fceba0 con 0x55c9c7fbdbb0 2024-09-19T08:45:34.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.462+0000 7fbd4e59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9c7fb29d0 0x55c9c7fce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:34.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.462+0000 7fbd4e59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9c7fb29d0 0x55c9c7fce650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43368/0 (socket says 172.21.15.164:43368) 2024-09-19T08:45:34.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.462+0000 7fbd4e59c640 1 -- 172.21.15.164:0/2531148695 learned_addr learned my addr 172.21.15.164:0/2531148695 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:34.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.463+0000 7fbd4ed9d640 1 --2- 172.21.15.164:0/2531148695 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c9c7fbdbb0 0x55c9c7fcdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:34.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.463+0000 7fbd4dd9b640 1 --2- 172.21.15.164:0/2531148695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c7fb32f0 0x55c9c7fced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:34.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.463+0000 7fbd4e59c640 1 -- 172.21.15.164:0/2531148695 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c9c7fbdbb0 msgr2=0x55c9c7fcdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:34.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.463+0000 7fbd4e59c640 1 --2- 172.21.15.164:0/2531148695 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c9c7fbdbb0 0x55c9c7fcdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:34.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.463+0000 7fbd4e59c640 1 -- 172.21.15.164:0/2531148695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c7fb32f0 msgr2=0x55c9c7fced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:34.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.463+0000 7fbd4e59c640 1 --2- 172.21.15.164:0/2531148695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c7fb32f0 0x55c9c7fced20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:34.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.463+0000 7fbd4e59c640 1 -- 172.21.15.164:0/2531148695 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9c7fb8d20 con 0x55c9c7fb29d0 2024-09-19T08:45:34.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.463+0000 7fbd4ed9d640 1 --2- 172.21.15.164:0/2531148695 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c9c7fbdbb0 0x55c9c7fcdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:34.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.463+0000 7fbd4e59c640 1 --2- 172.21.15.164:0/2531148695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9c7fb29d0 0x55c9c7fce650 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbd400989b0 tx=0x7fbd40005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9904eaddc1000c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:34.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.463+0000 7fbd37fff640 1 -- 172.21.15.164:0/2531148695 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4001f040 con 0x55c9c7fb29d0 2024-09-19T08:45:34.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.463+0000 7fbd37fff640 1 -- 172.21.15.164:0/2531148695 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd4007c030 con 0x55c9c7fb29d0 2024-09-19T08:45:34.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.463+0000 7fbd37fff640 1 -- 172.21.15.164:0/2531148695 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd400a4f00 con 0x55c9c7fb29d0 2024-09-19T08:45:34.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.464+0000 7fbd56c5d740 1 -- 172.21.15.164:0/2531148695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9c7fb29d0 msgr2=0x55c9c7fce650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:34.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.464+0000 7fbd56c5d740 1 --2- 172.21.15.164:0/2531148695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9c7fb29d0 0x55c9c7fce650 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbd400989b0 tx=0x7fbd40005710 comp rx=0 tx=0).stop 2024-09-19T08:45:34.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.464+0000 7fbd56c5d740 1 -- 172.21.15.164:0/2531148695 shutdown_connections 2024-09-19T08:45:34.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.464+0000 7fbd56c5d740 1 --2- 172.21.15.164:0/2531148695 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c9c7fbdbb0 0x55c9c7fcdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:34.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.464+0000 7fbd56c5d740 1 --2- 172.21.15.164:0/2531148695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c7fb32f0 0x55c9c7fced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:34.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.464+0000 7fbd56c5d740 1 --2- 172.21.15.164:0/2531148695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9c7fb29d0 0x55c9c7fce650 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:34.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.464+0000 7fbd56c5d740 1 -- 172.21.15.164:0/2531148695 >> 172.21.15.164:0/2531148695 conn(0x55c9c7faef60 msgr2=0x55c9c7fa9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:34.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.464+0000 7fbd56c5d740 1 -- 172.21.15.164:0/2531148695 shutdown_connections 2024-09-19T08:45:34.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.464+0000 7fbd56c5d740 1 -- 172.21.15.164:0/2531148695 wait complete. 2024-09-19T08:45:34.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.464+0000 7fbd56c5d740 1 -- start start 2024-09-19T08:45:34.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.464+0000 7fbd56c5d740 1 Processor -- start 2024-09-19T08:45:34.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.464+0000 7fbd56c5d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c7fb29d0 0x55c9c7fd2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:34.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.464+0000 7fbd56c5d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c9c7fb32f0 0x55c9c7fd3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:34.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.464+0000 7fbd56c5d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9c7fbdbb0 0x55c9c7fd3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:34.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.464+0000 7fbd56c5d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c9c7fcf940 con 0x55c9c7fbdbb0 2024-09-19T08:45:34.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.464+0000 7fbd56c5d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c9c7fcf270 con 0x55c9c7fb29d0 2024-09-19T08:45:34.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.464+0000 7fbd56c5d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c9c7fd0010 con 0x55c9c7fb32f0 2024-09-19T08:45:34.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.465+0000 7fbd56c5d740 10 client.0 _ll_register_callbacks cb 0x7fbd40009270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:45:34.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.465+0000 7fbd4ed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9c7fbdbb0 0x55c9c7fd3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:34.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.465+0000 7fbd4ed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9c7fbdbb0 0x55c9c7fd3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43380/0 (socket says 172.21.15.164:43380) 2024-09-19T08:45:34.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.465+0000 7fbd4ed9d640 1 -- 172.21.15.164:0/4170836140 learned_addr learned my addr 172.21.15.164:0/4170836140 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:34.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.465+0000 7fbd4e59c640 1 --2- 172.21.15.164:0/4170836140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c7fb29d0 0x55c9c7fd2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:34.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.465+0000 7fbd4dd9b640 1 --2- 172.21.15.164:0/4170836140 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c9c7fb32f0 0x55c9c7fd3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:34.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.465+0000 7fbd4ed9d640 1 -- 172.21.15.164:0/4170836140 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c9c7fb32f0 msgr2=0x55c9c7fd3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:34.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.465+0000 7fbd4ed9d640 1 --2- 172.21.15.164:0/4170836140 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c9c7fb32f0 0x55c9c7fd3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:34.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.465+0000 7fbd4ed9d640 1 -- 172.21.15.164:0/4170836140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c7fb29d0 msgr2=0x55c9c7fd2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:34.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.465+0000 7fbd4ed9d640 1 --2- 172.21.15.164:0/4170836140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c7fb29d0 0x55c9c7fd2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:34.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.465+0000 7fbd4ed9d640 1 -- 172.21.15.164:0/4170836140 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9c8036d50 con 0x55c9c7fbdbb0 2024-09-19T08:45:34.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.465+0000 7fbd4dd9b640 1 --2- 172.21.15.164:0/4170836140 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c9c7fb32f0 0x55c9c7fd3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:34.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.465+0000 7fbd4ed9d640 1 --2- 172.21.15.164:0/4170836140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9c7fbdbb0 0x55c9c7fd3750 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbd48005d00 tx=0x7fbd48004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1cd4133edba9b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:34.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.465+0000 7fbd35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:45:34.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.465+0000 7fbd35ffb640 1 -- 172.21.15.164:0/4170836140 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd48059060 con 0x55c9c7fbdbb0 2024-09-19T08:45:34.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.465+0000 7fbd35ffb640 1 -- 172.21.15.164:0/4170836140 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fbd4803ddf0 con 0x55c9c7fbdbb0 2024-09-19T08:45:34.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.465+0000 7fbd56c5d740 1 -- 172.21.15.164:0/4170836140 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd4007c030 con 0x55c9c7fbdbb0 2024-09-19T08:45:34.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.465+0000 7fbd35ffb640 1 -- 172.21.15.164:0/4170836140 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbd48035040 con 0x55c9c7fbdbb0 2024-09-19T08:45:34.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.465+0000 7fbd56c5d740 10 client.15105 fetch_fsmap learned FSMap version 27 2024-09-19T08:45:34.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.465+0000 7fbd56c5d740 1 -- 172.21.15.164:0/4170836140 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c9c8037e80 con 0x55c9c7fbdbb0 2024-09-19T08:45:34.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.466+0000 7fbd35ffb640 1 -- 172.21.15.164:0/4170836140 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd4007c030 con 0x55c9c7fbdbb0 2024-09-19T08:45:34.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.466+0000 7fbd35ffb640 1 -- 172.21.15.164:0/4170836140 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbd4805e070 con 0x55c9c7fbdbb0 2024-09-19T08:45:34.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.466+0000 7fbd56c5d740 10 client.15105 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T08:45:34.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.466+0000 7fbd56c5d740 10 client.15105 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T08:45:34.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.466+0000 7fbd56c5d740 20 client.15105 populate_metadata read hostname 'smithi164' 2024-09-19T08:45:34.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.466+0000 7fbd56c5d740 1 --2- 172.21.15.164:0/4170836140 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55c9c803c590 0x55c9c803b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:34.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.466+0000 7fbd56c5d740 4 client.15105 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:45:34.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.466+0000 7fbd56c5d740 1 -- 172.21.15.164:0/4170836140 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "status"}) -- 0x55c9c7f20b10 con 0x55c9c803c590 2024-09-19T08:45:34.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.466+0000 7fbd4e59c640 1 --2- 172.21.15.164:0/4170836140 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55c9c803c590 0x55c9c803b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:34.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.466+0000 7fbd4e59c640 1 --2- 172.21.15.164:0/4170836140 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55c9c803c590 0x55c9c803b2a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6fa122e63971833b server_cookie=c94939f327d57833 in_seq=0 out_seq=0 2024-09-19T08:45:34.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.466+0000 7fbd35ffb640 10 client.15105 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:45:34.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.466+0000 7fbd35ffb640 1 -- 172.21.15.164:0/4170836140 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c9c7f20b10 con 0x55c9c803c590 2024-09-19T08:45:34.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.466+0000 7fbd35ffb640 10 client.15105 handle_command_reply: tid=0 2024-09-19T08:45:34.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.466+0000 7fbd56c5d740 1 client.15105 shutdown 2024-09-19T08:45:34.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.467+0000 7fbd56c5d740 1 -- 172.21.15.164:0/4170836140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9c7fbdbb0 msgr2=0x55c9c7fd3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:34.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.467+0000 7fbd56c5d740 1 --2- 172.21.15.164:0/4170836140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9c7fbdbb0 0x55c9c7fd3750 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbd48005d00 tx=0x7fbd48004170 comp rx=0 tx=0).stop 2024-09-19T08:45:34.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.467+0000 7fbd56c5d740 1 -- 172.21.15.164:0/4170836140 shutdown_connections 2024-09-19T08:45:34.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.467+0000 7fbd56c5d740 1 --2- 172.21.15.164:0/4170836140 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55c9c803c590 0x55c9c803b2a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:34.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.467+0000 7fbd56c5d740 1 --2- 172.21.15.164:0/4170836140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9c7fbdbb0 0x55c9c7fd3750 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:34.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.467+0000 7fbd56c5d740 1 --2- 172.21.15.164:0/4170836140 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c9c7fb32f0 0x55c9c7fd3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:34.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.467+0000 7fbd56c5d740 1 --2- 172.21.15.164:0/4170836140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c7fb29d0 0x55c9c7fd2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:34.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.467+0000 7fbd56c5d740 1 -- 172.21.15.164:0/4170836140 >> 172.21.15.164:0/4170836140 conn(0x55c9c7faef60 msgr2=0x55c9c7fb8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:34.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.467+0000 7fbd35ffb640 0 client.15105 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:45:34.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.467+0000 7fbd56c5d740 1 -- 172.21.15.164:0/4170836140 shutdown_connections 2024-09-19T08:45:34.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.467+0000 7fbd56c5d740 1 -- 172.21.15.164:0/4170836140 wait complete. 2024-09-19T08:45:34.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.467+0000 7fbd56c5d740 20 client.15105 trim_cache size 0 max 16384 2024-09-19T08:45:34.474 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:45:34.474 INFO:teuthology.orchestra.run.smithi164.stdout: "cluster_fsid": "1e4b0d74-7662-11ef-bceb-c7b262605968", 2024-09-19T08:45:34.474 INFO:teuthology.orchestra.run.smithi164.stdout: "whoami": 4, 2024-09-19T08:45:34.474 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 14595, 2024-09-19T08:45:34.474 INFO:teuthology.orchestra.run.smithi164.stdout: "want_state": "up:active", 2024-09-19T08:45:34.474 INFO:teuthology.orchestra.run.smithi164.stdout: "state": "up:active", 2024-09-19T08:45:34.474 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs", 2024-09-19T08:45:34.474 INFO:teuthology.orchestra.run.smithi164.stdout: "rank_uptime": 89.838273048999994, 2024-09-19T08:45:34.474 INFO:teuthology.orchestra.run.smithi164.stdout: "mdsmap_epoch": 27, 2024-09-19T08:45:34.475 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch": 82, 2024-09-19T08:45:34.475 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch_barrier": 82, 2024-09-19T08:45:34.475 INFO:teuthology.orchestra.run.smithi164.stdout: "uptime": 115.099090108 2024-09-19T08:45:34.475 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:45:34.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.467+0000 7fbd54884640 1 -- 172.21.15.164:0/2619720130 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fbd380a3240 msgr2=0x7fbd380a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:34.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.467+0000 7fbd54884640 1 --2- 172.21.15.164:0/2619720130 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fbd380a3240 0x7fbd380a5700 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbd4804c000 tx=0x7fbd4804a000 comp rx=0 tx=0).stop 2024-09-19T08:45:34.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.467+0000 7fbd54884640 1 -- 172.21.15.164:0/2619720130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd50100e20 msgr2=0x7fbd501971c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:34.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.467+0000 7fbd54884640 1 --2- 172.21.15.164:0/2619720130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd50100e20 0x7fbd501971c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbd4000ea80 tx=0x7fbd4000ef40 comp rx=0 tx=0).stop 2024-09-19T08:45:34.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.467+0000 7fbd54884640 1 -- 172.21.15.164:0/2619720130 shutdown_connections 2024-09-19T08:45:34.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.467+0000 7fbd54884640 1 --2- 172.21.15.164:0/2619720130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd50102ef0 0x7fbd50197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:34.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.467+0000 7fbd54884640 1 --2- 172.21.15.164:0/2619720130 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fbd380a3240 0x7fbd380a5700 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:34.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.467+0000 7fbd54884640 1 --2- 172.21.15.164:0/2619720130 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd50102570 0x7fbd50197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:34.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.467+0000 7fbd54884640 1 --2- 172.21.15.164:0/2619720130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd50100e20 0x7fbd501971c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:34.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.467+0000 7fbd54884640 1 -- 172.21.15.164:0/2619720130 >> 172.21.15.164:0/2619720130 conn(0x7fbd500fc7d0 msgr2=0x7fbd501068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:34.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.467+0000 7fbd54884640 1 -- 172.21.15.164:0/2619720130 shutdown_connections 2024-09-19T08:45:34.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:34.468+0000 7fbd54884640 1 -- 172.21.15.164:0/2619720130 wait complete. 2024-09-19T08:45:35.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:34 smithi170 ceph-mon[35072]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:45:35.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:34 smithi173 ceph-mon[35306]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:45:35.281 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:34 smithi164 ceph-mon[30928]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:45:35.339 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T08:45:35.589 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:45:36.020 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:35 smithi164 ceph-mon[30928]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-19T08:45:36.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.102+0000 7fb33e1b5640 1 -- 172.21.15.164:0/3240182714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb338102d40 msgr2=0x7fb338103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:36.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.102+0000 7fb33e1b5640 1 --2- 172.21.15.164:0/3240182714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb338102d40 0x7fb338103120 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb328009900 tx=0x7fb32802a480 comp rx=0 tx=0).stop 2024-09-19T08:45:36.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.102+0000 7fb33e1b5640 1 -- 172.21.15.164:0/3240182714 shutdown_connections 2024-09-19T08:45:36.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.102+0000 7fb33e1b5640 1 --2- 172.21.15.164:0/3240182714 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb33810c170 0x7fb33810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.102+0000 7fb33e1b5640 1 --2- 172.21.15.164:0/3240182714 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb338103660 0x7fb33810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.102+0000 7fb33e1b5640 1 --2- 172.21.15.164:0/3240182714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb338102d40 0x7fb338103120 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.102+0000 7fb33e1b5640 1 -- 172.21.15.164:0/3240182714 >> 172.21.15.164:0/3240182714 conn(0x7fb3380fc7a0 msgr2=0x7fb3380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:36.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.102+0000 7fb33e1b5640 1 -- 172.21.15.164:0/3240182714 shutdown_connections 2024-09-19T08:45:36.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.102+0000 7fb33e1b5640 1 -- 172.21.15.164:0/3240182714 wait complete. 2024-09-19T08:45:36.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.103+0000 7fb33e1b5640 1 Processor -- start 2024-09-19T08:45:36.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.103+0000 7fb33e1b5640 1 -- start start 2024-09-19T08:45:36.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.103+0000 7fb33e1b5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb338102d40 0x7fb3381009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:36.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.103+0000 7fb33e1b5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb338103660 0x7fb338100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:36.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.103+0000 7fb33e1b5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb33810c170 0x7fb338101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:36.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.103+0000 7fb33e1b5640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb338111f70 con 0x7fb338102d40 2024-09-19T08:45:36.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.103+0000 7fb33e1b5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb338111df0 con 0x7fb33810c170 2024-09-19T08:45:36.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.103+0000 7fb33e1b5640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb3381120f0 con 0x7fb338103660 2024-09-19T08:45:36.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.103+0000 7fb3377fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb338102d40 0x7fb3381009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:36.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.103+0000 7fb3377fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb338102d40 0x7fb3381009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43394/0 (socket says 172.21.15.164:43394) 2024-09-19T08:45:36.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.103+0000 7fb3377fe640 1 -- 172.21.15.164:0/2681492765 learned_addr learned my addr 172.21.15.164:0/2681492765 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:36.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.103+0000 7fb336ffd640 1 --2- 172.21.15.164:0/2681492765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb338103660 0x7fb338100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:36.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.104+0000 7fb3377fe640 1 -- 172.21.15.164:0/2681492765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb338103660 msgr2=0x7fb338100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:36.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.104+0000 7fb3377fe640 1 --2- 172.21.15.164:0/2681492765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb338103660 0x7fb338100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.104+0000 7fb337fff640 1 --2- 172.21.15.164:0/2681492765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb33810c170 0x7fb338101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:36.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.104+0000 7fb3377fe640 1 -- 172.21.15.164:0/2681492765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb33810c170 msgr2=0x7fb338101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:36.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.104+0000 7fb3377fe640 1 --2- 172.21.15.164:0/2681492765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb33810c170 0x7fb338101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.104+0000 7fb3377fe640 1 -- 172.21.15.164:0/2681492765 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3381a6930 con 0x7fb338102d40 2024-09-19T08:45:36.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.104+0000 7fb336ffd640 1 --2- 172.21.15.164:0/2681492765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb338103660 0x7fb338100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:36.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.104+0000 7fb337fff640 1 --2- 172.21.15.164:0/2681492765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb33810c170 0x7fb338101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:36.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.104+0000 7fb3377fe640 1 --2- 172.21.15.164:0/2681492765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb338102d40 0x7fb3381009a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb328006010 tx=0x7fb3280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:36.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.104+0000 7fb334ff9640 1 -- 172.21.15.164:0/2681492765 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32803f030 con 0x7fb338102d40 2024-09-19T08:45:36.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.104+0000 7fb334ff9640 1 -- 172.21.15.164:0/2681492765 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb328003ce0 con 0x7fb338102d40 2024-09-19T08:45:36.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.104+0000 7fb33e1b5640 1 -- 172.21.15.164:0/2681492765 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3381a6bc0 con 0x7fb338102d40 2024-09-19T08:45:36.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.104+0000 7fb334ff9640 1 -- 172.21.15.164:0/2681492765 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb328035070 con 0x7fb338102d40 2024-09-19T08:45:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.104+0000 7fb33e1b5640 1 -- 172.21.15.164:0/2681492765 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3381a70a0 con 0x7fb338102d40 2024-09-19T08:45:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.106+0000 7fb334ff9640 1 -- 172.21.15.164:0/2681492765 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fb328042030 con 0x7fb338102d40 2024-09-19T08:45:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.107+0000 7fb34058e740 1 Processor -- start 2024-09-19T08:45:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.107+0000 7fb34058e740 1 -- start start 2024-09-19T08:45:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.107+0000 7fb34058e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4c4e1d9d0 0x55f4c4e1ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.107+0000 7fb34058e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4c4e1e2f0 0x55f4c4e28670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.107+0000 7fb334ff9640 1 --2- 172.21.15.164:0/2681492765 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb3200a3240 0x7fb3200a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.107+0000 7fb34058e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4c4e28bb0 0x55f4c4e2afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.107+0000 7fb34058e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f4c4d84090 con 0x55f4c4e1e2f0 2024-09-19T08:45:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.107+0000 7fb34058e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f4c4d2ba70 con 0x55f4c4e28bb0 2024-09-19T08:45:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.107+0000 7fb34058e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f4c4d80050 con 0x55f4c4e1d9d0 2024-09-19T08:45:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.107+0000 7fb3377fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4c4e1e2f0 0x55f4c4e28670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:36.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.107+0000 7fb336ffd640 1 --2- 172.21.15.164:0/2681492765 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb3200a3240 0x7fb3200a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:36.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.107+0000 7fb337fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4c4e1d9d0 0x55f4c4e1ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:36.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.107+0000 7fb3377fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4c4e1e2f0 0x55f4c4e28670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43408/0 (socket says 172.21.15.164:43408) 2024-09-19T08:45:36.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.107+0000 7fb3377fe640 1 -- 172.21.15.164:0/4248753172 learned_addr learned my addr 172.21.15.164:0/4248753172 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:36.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.107+0000 7fb337fff640 1 --2- 172.21.15.164:0/4248753172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4c4e28bb0 0x55f4c4e2afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:36.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.107+0000 7fb334ff9640 1 -- 172.21.15.164:0/2681492765 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb3280efe30 con 0x7fb338102d40 2024-09-19T08:45:36.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.107+0000 7fb337fff640 1 -- 172.21.15.164:0/4248753172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4c4e28bb0 msgr2=0x55f4c4e2afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:36.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.107+0000 7fb337fff640 1 --2- 172.21.15.164:0/4248753172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4c4e28bb0 0x55f4c4e2afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.107+0000 7fb337fff640 1 -- 172.21.15.164:0/4248753172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4c4e1e2f0 msgr2=0x55f4c4e28670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:36.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.107+0000 7fb337fff640 1 --2- 172.21.15.164:0/4248753172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4c4e1e2f0 0x55f4c4e28670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.107+0000 7fb337fff640 1 -- 172.21.15.164:0/4248753172 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4c4d38130 con 0x55f4c4e1d9d0 2024-09-19T08:45:36.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.108+0000 7fb3377fe640 1 --2- 172.21.15.164:0/4248753172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4c4e1e2f0 0x55f4c4e28670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:45:36.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.108+0000 7fb336ffd640 1 --2- 172.21.15.164:0/2681492765 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb3200a3240 0x7fb3200a5700 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb32c005fd0 tx=0x7fb32c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:36.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.108+0000 7fb337fff640 1 --2- 172.21.15.164:0/4248753172 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4c4e1d9d0 0x55f4c4e1ddb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb330012940 tx=0x7fb33000a510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5ff1585a3c3578e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:36.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.108+0000 7fb31dffb640 1 -- 172.21.15.164:0/4248753172 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33000daf0 con 0x55f4c4e1d9d0 2024-09-19T08:45:36.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.108+0000 7fb31dffb640 1 -- 172.21.15.164:0/4248753172 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb33000dc90 con 0x55f4c4e1d9d0 2024-09-19T08:45:36.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.108+0000 7fb31dffb640 1 -- 172.21.15.164:0/4248753172 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33000f620 con 0x55f4c4e1d9d0 2024-09-19T08:45:36.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.109+0000 7fb34058e740 1 -- 172.21.15.164:0/4248753172 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4c4e1d9d0 msgr2=0x55f4c4e1ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:36.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.109+0000 7fb34058e740 1 --2- 172.21.15.164:0/4248753172 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4c4e1d9d0 0x55f4c4e1ddb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb330012940 tx=0x7fb33000a510 comp rx=0 tx=0).stop 2024-09-19T08:45:36.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.109+0000 7fb34058e740 1 -- 172.21.15.164:0/4248753172 shutdown_connections 2024-09-19T08:45:36.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.109+0000 7fb34058e740 1 --2- 172.21.15.164:0/4248753172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4c4e28bb0 0x55f4c4e2afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.109+0000 7fb34058e740 1 --2- 172.21.15.164:0/4248753172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4c4e1e2f0 0x55f4c4e28670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.109+0000 7fb34058e740 1 --2- 172.21.15.164:0/4248753172 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4c4e1d9d0 0x55f4c4e1ddb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.109+0000 7fb34058e740 1 -- 172.21.15.164:0/4248753172 >> 172.21.15.164:0/4248753172 conn(0x55f4c4e19f60 msgr2=0x55f4c4e1a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:36.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.109+0000 7fb34058e740 1 -- 172.21.15.164:0/4248753172 shutdown_connections 2024-09-19T08:45:36.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.109+0000 7fb34058e740 1 -- 172.21.15.164:0/4248753172 wait complete. 2024-09-19T08:45:36.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.109+0000 7fb34058e740 1 -- start start 2024-09-19T08:45:36.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.109+0000 7fb34058e740 1 Processor -- start 2024-09-19T08:45:36.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.110+0000 7fb34058e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4c4e1d9d0 0x55f4c4e28480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:36.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.110+0000 7fb34058e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4c4e1e2f0 0x55f4c4e257d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:36.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.110+0000 7fb34058e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4c4e28bb0 0x55f4c4e25d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:36.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.110+0000 7fb34058e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f4c4e2b780 con 0x55f4c4e1e2f0 2024-09-19T08:45:36.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.110+0000 7fb34058e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f4c4d3ef20 con 0x55f4c4e1d9d0 2024-09-19T08:45:36.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.110+0000 7fb34058e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f4c4e2b900 con 0x55f4c4e28bb0 2024-09-19T08:45:36.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.110+0000 7fb34058e740 10 client.0 _ll_register_callbacks cb 0x55f4c4d08030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:45:36.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.110+0000 7fb336ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4c4e1e2f0 0x55f4c4e257d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:36.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.110+0000 7fb336ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4c4e1e2f0 0x55f4c4e257d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43412/0 (socket says 172.21.15.164:43412) 2024-09-19T08:45:36.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.110+0000 7fb336ffd640 1 -- 172.21.15.164:0/2982780841 learned_addr learned my addr 172.21.15.164:0/2982780841 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:36.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.110+0000 7fb337fff640 1 --2- 172.21.15.164:0/2982780841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4c4e28bb0 0x55f4c4e25d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:36.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.110+0000 7fb3377fe640 1 --2- 172.21.15.164:0/2982780841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4c4e1d9d0 0x55f4c4e28480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:36.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.110+0000 7fb336ffd640 1 -- 172.21.15.164:0/2982780841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4c4e28bb0 msgr2=0x55f4c4e25d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:36.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.110+0000 7fb336ffd640 1 --2- 172.21.15.164:0/2982780841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4c4e28bb0 0x55f4c4e25d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.110+0000 7fb336ffd640 1 -- 172.21.15.164:0/2982780841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4c4e1d9d0 msgr2=0x55f4c4e28480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:36.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.110+0000 7fb336ffd640 1 --2- 172.21.15.164:0/2982780841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4c4e1d9d0 0x55f4c4e28480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.110+0000 7fb336ffd640 1 -- 172.21.15.164:0/2982780841 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4c4af52b0 con 0x55f4c4e1e2f0 2024-09-19T08:45:36.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.110+0000 7fb337fff640 1 --2- 172.21.15.164:0/2982780841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4c4e28bb0 0x55f4c4e25d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:36.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.110+0000 7fb336ffd640 1 --2- 172.21.15.164:0/2982780841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4c4e1e2f0 0x55f4c4e257d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb32c011440 tx=0x7fb32c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8bd471082176b14 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:36.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.110+0000 7fb316ffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:45:36.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.110+0000 7fb316ffd640 1 -- 172.21.15.164:0/2982780841 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c022070 con 0x55f4c4e1e2f0 2024-09-19T08:45:36.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.110+0000 7fb316ffd640 1 -- 172.21.15.164:0/2982780841 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb32c01e860 con 0x55f4c4e1e2f0 2024-09-19T08:45:36.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.110+0000 7fb34058e740 1 -- 172.21.15.164:0/2982780841 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb33000dc90 con 0x55f4c4e1e2f0 2024-09-19T08:45:36.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.110+0000 7fb316ffd640 1 -- 172.21.15.164:0/2982780841 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb32c02e050 con 0x55f4c4e1e2f0 2024-09-19T08:45:36.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.110+0000 7fb34058e740 10 client.15117 fetch_fsmap learned FSMap version 27 2024-09-19T08:45:36.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.110+0000 7fb34058e740 1 -- 172.21.15.164:0/2982780841 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f4c4e37160 con 0x55f4c4e1e2f0 2024-09-19T08:45:36.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.111+0000 7fb316ffd640 1 -- 172.21.15.164:0/2982780841 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb33000dc90 con 0x55f4c4e1e2f0 2024-09-19T08:45:36.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.111+0000 7fb316ffd640 1 -- 172.21.15.164:0/2982780841 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb32c010c20 con 0x55f4c4e1e2f0 2024-09-19T08:45:36.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.111+0000 7fb34058e740 10 client.15117 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T08:45:36.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.111+0000 7fb34058e740 10 client.15117 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T08:45:36.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.111+0000 7fb34058e740 20 client.15117 populate_metadata read hostname 'smithi164' 2024-09-19T08:45:36.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.111+0000 7fb34058e740 1 --2- 172.21.15.164:0/2982780841 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55f4c4e397d0 0x55f4c4e3ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:36.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.111+0000 7fb34058e740 4 client.15117 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:45:36.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.111+0000 7fb34058e740 1 -- 172.21.15.164:0/2982780841 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4c4d91600 con 0x55f4c4e397d0 2024-09-19T08:45:36.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.111+0000 7fb3377fe640 1 --2- 172.21.15.164:0/2982780841 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55f4c4e397d0 0x55f4c4e3ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:36.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.112+0000 7fb316ffd640 10 client.15117 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:45:36.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.112+0000 7fb3377fe640 1 --2- 172.21.15.164:0/2982780841 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55f4c4e397d0 0x55f4c4e3ac40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=eae668ccce3eb3b2 server_cookie=e08287b731319853 in_seq=0 out_seq=0 2024-09-19T08:45:36.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.112+0000 7fb316ffd640 1 -- 172.21.15.164:0/2982780841 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f4c4d91600 con 0x55f4c4e397d0 2024-09-19T08:45:36.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.112+0000 7fb316ffd640 10 client.15117 handle_command_reply: tid=0 2024-09-19T08:45:36.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.112+0000 7fb34058e740 1 client.15117 shutdown 2024-09-19T08:45:36.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.112+0000 7fb34058e740 1 -- 172.21.15.164:0/2982780841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4c4e1e2f0 msgr2=0x55f4c4e257d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:36.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.112+0000 7fb34058e740 1 --2- 172.21.15.164:0/2982780841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4c4e1e2f0 0x55f4c4e257d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb32c011440 tx=0x7fb32c0046c0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.113+0000 7fb34058e740 1 -- 172.21.15.164:0/2982780841 shutdown_connections 2024-09-19T08:45:36.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.113+0000 7fb34058e740 1 --2- 172.21.15.164:0/2982780841 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55f4c4e397d0 0x55f4c4e3ac40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.113+0000 7fb34058e740 1 --2- 172.21.15.164:0/2982780841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4c4e28bb0 0x55f4c4e25d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.113+0000 7fb34058e740 1 --2- 172.21.15.164:0/2982780841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4c4e1e2f0 0x55f4c4e257d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.113+0000 7fb34058e740 1 --2- 172.21.15.164:0/2982780841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4c4e1d9d0 0x55f4c4e28480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.113+0000 7fb316ffd640 0 client.15117 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:45:36.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.113+0000 7fb34058e740 1 -- 172.21.15.164:0/2982780841 >> 172.21.15.164:0/2982780841 conn(0x55f4c4e19f60 msgr2=0x55f4c4e1fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:36.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.113+0000 7fb34058e740 1 -- 172.21.15.164:0/2982780841 shutdown_connections 2024-09-19T08:45:36.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.113+0000 7fb34058e740 1 -- 172.21.15.164:0/2982780841 wait complete. 2024-09-19T08:45:36.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.114+0000 7fb34058e740 20 client.15117 trim_cache size 0 max 16384 2024-09-19T08:45:36.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:35 smithi170 ceph-mon[35072]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-19T08:45:36.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.131+0000 7fb34058e740 1 Processor -- start 2024-09-19T08:45:36.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.131+0000 7fb34058e740 1 -- start start 2024-09-19T08:45:36.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.131+0000 7fb34058e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4c4e1d9d0 0x55f4c4e39ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:36.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.131+0000 7fb34058e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4c4e1e2f0 0x55f4c4e240a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:36.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.131+0000 7fb34058e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4c4e28bb0 0x55f4c4e22a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:36.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.131+0000 7fb34058e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f4c4e38f80 con 0x55f4c4e1e2f0 2024-09-19T08:45:36.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.131+0000 7fb34058e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f4c4e39650 con 0x55f4c4e28bb0 2024-09-19T08:45:36.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.131+0000 7fb34058e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f4c4e39d20 con 0x55f4c4e1d9d0 2024-09-19T08:45:36.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.131+0000 7fb336ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4c4e1e2f0 0x55f4c4e240a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:36.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.131+0000 7fb336ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4c4e1e2f0 0x55f4c4e240a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43420/0 (socket says 172.21.15.164:43420) 2024-09-19T08:45:36.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.131+0000 7fb336ffd640 1 -- 172.21.15.164:0/3913794675 learned_addr learned my addr 172.21.15.164:0/3913794675 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:36.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.131+0000 7fb3377fe640 1 --2- 172.21.15.164:0/3913794675 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4c4e1d9d0 0x55f4c4e39ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:36.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.132+0000 7fb337fff640 1 --2- 172.21.15.164:0/3913794675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4c4e28bb0 0x55f4c4e22a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:36.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.132+0000 7fb336ffd640 1 -- 172.21.15.164:0/3913794675 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4c4e1d9d0 msgr2=0x55f4c4e39ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:36.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.132+0000 7fb336ffd640 1 --2- 172.21.15.164:0/3913794675 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4c4e1d9d0 0x55f4c4e39ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.132+0000 7fb336ffd640 1 -- 172.21.15.164:0/3913794675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4c4e28bb0 msgr2=0x55f4c4e22a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:36.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.132+0000 7fb336ffd640 1 --2- 172.21.15.164:0/3913794675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4c4e28bb0 0x55f4c4e22a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.132+0000 7fb336ffd640 1 -- 172.21.15.164:0/3913794675 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4c4e24780 con 0x55f4c4e1e2f0 2024-09-19T08:45:36.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.132+0000 7fb336ffd640 1 --2- 172.21.15.164:0/3913794675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4c4e1e2f0 0x55f4c4e240a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00fda0 tx=0x7fb32c00bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65ddaa7be07f4d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:36.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.132+0000 7fb31d7fa640 1 -- 172.21.15.164:0/3913794675 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c022070 con 0x55f4c4e1e2f0 2024-09-19T08:45:36.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.132+0000 7fb31d7fa640 1 -- 172.21.15.164:0/3913794675 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb32c037070 con 0x55f4c4e1e2f0 2024-09-19T08:45:36.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.132+0000 7fb31d7fa640 1 -- 172.21.15.164:0/3913794675 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c01ed40 con 0x55f4c4e1e2f0 2024-09-19T08:45:36.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.133+0000 7fb34058e740 1 -- 172.21.15.164:0/3913794675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4c4e1e2f0 msgr2=0x55f4c4e240a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:36.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.133+0000 7fb34058e740 1 --2- 172.21.15.164:0/3913794675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4c4e1e2f0 0x55f4c4e240a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00fda0 tx=0x7fb32c00bb10 comp rx=0 tx=0).stop 2024-09-19T08:45:36.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.133+0000 7fb34058e740 1 -- 172.21.15.164:0/3913794675 shutdown_connections 2024-09-19T08:45:36.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.133+0000 7fb34058e740 1 --2- 172.21.15.164:0/3913794675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4c4e28bb0 0x55f4c4e22a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.133+0000 7fb34058e740 1 --2- 172.21.15.164:0/3913794675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4c4e1e2f0 0x55f4c4e240a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.133+0000 7fb34058e740 1 --2- 172.21.15.164:0/3913794675 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4c4e1d9d0 0x55f4c4e39ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.133+0000 7fb34058e740 1 -- 172.21.15.164:0/3913794675 >> 172.21.15.164:0/3913794675 conn(0x55f4c4e19f60 msgr2=0x55f4c4e19020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:36.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.133+0000 7fb34058e740 1 -- 172.21.15.164:0/3913794675 shutdown_connections 2024-09-19T08:45:36.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.133+0000 7fb34058e740 1 -- 172.21.15.164:0/3913794675 wait complete. 2024-09-19T08:45:36.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.133+0000 7fb34058e740 1 -- start start 2024-09-19T08:45:36.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.133+0000 7fb34058e740 1 Processor -- start 2024-09-19T08:45:36.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.134+0000 7fb34058e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4c4e1d9d0 0x55f4c4e3ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:36.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.134+0000 7fb34058e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4c4e1e2f0 0x55f4c4e3f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:36.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.134+0000 7fb34058e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4c4e28bb0 0x55f4c4e3f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:36.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.134+0000 7fb34058e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f4c4e3aac0 con 0x55f4c4e1e2f0 2024-09-19T08:45:36.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.134+0000 7fb34058e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f4c4e3a3f0 con 0x55f4c4e1d9d0 2024-09-19T08:45:36.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.134+0000 7fb34058e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f4c4e3b190 con 0x55f4c4e28bb0 2024-09-19T08:45:36.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.134+0000 7fb34058e740 10 client.0 _ll_register_callbacks cb 0x7fb32c015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:45:36.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.134+0000 7fb336ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4c4e1e2f0 0x55f4c4e3f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:36.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.134+0000 7fb336ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4c4e1e2f0 0x55f4c4e3f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43424/0 (socket says 172.21.15.164:43424) 2024-09-19T08:45:36.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.134+0000 7fb336ffd640 1 -- 172.21.15.164:0/1878937444 learned_addr learned my addr 172.21.15.164:0/1878937444 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:36.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.134+0000 7fb3377fe640 1 --2- 172.21.15.164:0/1878937444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4c4e1d9d0 0x55f4c4e3ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:36.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.134+0000 7fb337fff640 1 --2- 172.21.15.164:0/1878937444 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4c4e28bb0 0x55f4c4e3f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:36.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.134+0000 7fb336ffd640 1 -- 172.21.15.164:0/1878937444 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4c4e28bb0 msgr2=0x55f4c4e3f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:36.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.134+0000 7fb336ffd640 1 --2- 172.21.15.164:0/1878937444 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4c4e28bb0 0x55f4c4e3f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.134+0000 7fb336ffd640 1 -- 172.21.15.164:0/1878937444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4c4e1d9d0 msgr2=0x55f4c4e3ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:36.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.134+0000 7fb336ffd640 1 --2- 172.21.15.164:0/1878937444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4c4e1d9d0 0x55f4c4e3ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.134+0000 7fb336ffd640 1 -- 172.21.15.164:0/1878937444 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4c4eab770 con 0x55f4c4e1e2f0 2024-09-19T08:45:36.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.134+0000 7fb336ffd640 1 --2- 172.21.15.164:0/1878937444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4c4e1e2f0 0x55f4c4e3f250 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb32c011440 tx=0x7fb32c004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7f9a4b12da0be5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:36.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.134+0000 7fb316ffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:45:36.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.135+0000 7fb316ffd640 1 -- 172.21.15.164:0/1878937444 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c041020 con 0x55f4c4e1e2f0 2024-09-19T08:45:36.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.135+0000 7fb316ffd640 1 -- 172.21.15.164:0/1878937444 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fb32c01ed20 con 0x55f4c4e1e2f0 2024-09-19T08:45:36.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.135+0000 7fb34058e740 1 -- 172.21.15.164:0/1878937444 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb32c037070 con 0x55f4c4e1e2f0 2024-09-19T08:45:36.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.135+0000 7fb316ffd640 1 -- 172.21.15.164:0/1878937444 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb32c022040 con 0x55f4c4e1e2f0 2024-09-19T08:45:36.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.135+0000 7fb34058e740 10 client.15123 fetch_fsmap learned FSMap version 27 2024-09-19T08:45:36.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.135+0000 7fb34058e740 1 -- 172.21.15.164:0/1878937444 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f4c4eaca30 con 0x55f4c4e1e2f0 2024-09-19T08:45:36.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.135+0000 7fb316ffd640 1 -- 172.21.15.164:0/1878937444 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb32c037070 con 0x55f4c4e1e2f0 2024-09-19T08:45:36.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.135+0000 7fb316ffd640 1 -- 172.21.15.164:0/1878937444 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb32c03d8c0 con 0x55f4c4e1e2f0 2024-09-19T08:45:36.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.135+0000 7fb34058e740 10 client.15123 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T08:45:36.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.135+0000 7fb34058e740 10 client.15123 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T08:45:36.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.135+0000 7fb34058e740 20 client.15123 populate_metadata read hostname 'smithi164' 2024-09-19T08:45:36.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.135+0000 7fb34058e740 1 --2- 172.21.15.164:0/1878937444 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55f4c4eb1210 0x55f4c4eaff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:36.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.135+0000 7fb34058e740 4 client.15123 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:45:36.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.135+0000 7fb34058e740 1 -- 172.21.15.164:0/1878937444 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "status"}) -- 0x55f4c4d8bb10 con 0x55f4c4eb1210 2024-09-19T08:45:36.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.135+0000 7fb3377fe640 1 --2- 172.21.15.164:0/1878937444 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55f4c4eb1210 0x55f4c4eaff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:36.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.136+0000 7fb316ffd640 10 client.15123 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:45:36.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.136+0000 7fb3377fe640 1 --2- 172.21.15.164:0/1878937444 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55f4c4eb1210 0x55f4c4eaff20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=952c986ef0e741a5 server_cookie=233b4460031f8b30 in_seq=0 out_seq=0 2024-09-19T08:45:36.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.136+0000 7fb316ffd640 1 -- 172.21.15.164:0/1878937444 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f4c4d8bb10 con 0x55f4c4eb1210 2024-09-19T08:45:36.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.136+0000 7fb316ffd640 10 client.15123 handle_command_reply: tid=0 2024-09-19T08:45:36.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.136+0000 7fb34058e740 1 client.15123 shutdown 2024-09-19T08:45:36.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.136+0000 7fb34058e740 1 -- 172.21.15.164:0/1878937444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4c4e1e2f0 msgr2=0x55f4c4e3f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:36.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.136+0000 7fb34058e740 1 --2- 172.21.15.164:0/1878937444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4c4e1e2f0 0x55f4c4e3f250 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb32c011440 tx=0x7fb32c004700 comp rx=0 tx=0).stop 2024-09-19T08:45:36.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.136+0000 7fb34058e740 1 -- 172.21.15.164:0/1878937444 shutdown_connections 2024-09-19T08:45:36.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.136+0000 7fb34058e740 1 --2- 172.21.15.164:0/1878937444 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55f4c4eb1210 0x55f4c4eaff20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.136+0000 7fb34058e740 1 --2- 172.21.15.164:0/1878937444 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4c4e28bb0 0x55f4c4e3f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.136+0000 7fb34058e740 1 --2- 172.21.15.164:0/1878937444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f4c4e1e2f0 0x55f4c4e3f250 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.136+0000 7fb34058e740 1 --2- 172.21.15.164:0/1878937444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4c4e1d9d0 0x55f4c4e3ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.136+0000 7fb34058e740 1 -- 172.21.15.164:0/1878937444 >> 172.21.15.164:0/1878937444 conn(0x55f4c4e19f60 msgr2=0x55f4c4e25880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:36.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.136+0000 7fb316ffd640 0 client.15123 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:45:36.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.136+0000 7fb34058e740 1 -- 172.21.15.164:0/1878937444 shutdown_connections 2024-09-19T08:45:36.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.136+0000 7fb34058e740 1 -- 172.21.15.164:0/1878937444 wait complete. 2024-09-19T08:45:36.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.136+0000 7fb34058e740 20 client.15123 trim_cache size 0 max 16384 2024-09-19T08:45:36.142 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:45:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: "cluster_fsid": "1e4b0d74-7662-11ef-bceb-c7b262605968", 2024-09-19T08:45:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: "whoami": 3, 2024-09-19T08:45:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 24424, 2024-09-19T08:45:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: "want_state": "up:active", 2024-09-19T08:45:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: "state": "up:active", 2024-09-19T08:45:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs", 2024-09-19T08:45:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: "rank_uptime": 92.306022178999996, 2024-09-19T08:45:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: "mdsmap_epoch": 27, 2024-09-19T08:45:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch": 82, 2024-09-19T08:45:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch_barrier": 82, 2024-09-19T08:45:36.143 INFO:teuthology.orchestra.run.smithi164.stdout: "uptime": 127.973081292 2024-09-19T08:45:36.143 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:45:36.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.137+0000 7fb33e1b5640 1 -- 172.21.15.164:0/2681492765 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb3200a3240 msgr2=0x7fb3200a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:36.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.137+0000 7fb33e1b5640 1 --2- 172.21.15.164:0/2681492765 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb3200a3240 0x7fb3200a5700 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb32c005fd0 tx=0x7fb32c005850 comp rx=0 tx=0).stop 2024-09-19T08:45:36.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.137+0000 7fb33e1b5640 1 -- 172.21.15.164:0/2681492765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb338102d40 msgr2=0x7fb3381009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:36.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.137+0000 7fb33e1b5640 1 --2- 172.21.15.164:0/2681492765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb338102d40 0x7fb3381009a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb328006010 tx=0x7fb3280042f0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.137+0000 7fb33e1b5640 1 -- 172.21.15.164:0/2681492765 shutdown_connections 2024-09-19T08:45:36.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.137+0000 7fb33e1b5640 1 --2- 172.21.15.164:0/2681492765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb33810c170 0x7fb338101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.137+0000 7fb33e1b5640 1 --2- 172.21.15.164:0/2681492765 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fb3200a3240 0x7fb3200a5700 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.137+0000 7fb33e1b5640 1 --2- 172.21.15.164:0/2681492765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb338103660 0x7fb338100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.137+0000 7fb33e1b5640 1 --2- 172.21.15.164:0/2681492765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb338102d40 0x7fb3381009a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.137+0000 7fb33e1b5640 1 -- 172.21.15.164:0/2681492765 >> 172.21.15.164:0/2681492765 conn(0x7fb3380fc7a0 msgr2=0x7fb3380fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:36.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.137+0000 7fb33e1b5640 1 -- 172.21.15.164:0/2681492765 shutdown_connections 2024-09-19T08:45:36.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:36.137+0000 7fb33e1b5640 1 -- 172.21.15.164:0/2681492765 wait complete. 2024-09-19T08:45:36.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:35 smithi173 ceph-mon[35306]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-19T08:45:36.936 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T08:45:37.180 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:45:37.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.668+0000 7f9f4bac7640 1 -- 172.21.15.164:0/2165595957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f44106f00 msgr2=0x7f9f4410ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:37.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.668+0000 7f9f4bac7640 1 --2- 172.21.15.164:0/2165595957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f44106f00 0x7f9f4410ea80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009900 tx=0x7f9f3c02a410 comp rx=0 tx=0).stop 2024-09-19T08:45:37.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.669+0000 7f9f4bac7640 1 -- 172.21.15.164:0/2165595957 shutdown_connections 2024-09-19T08:45:37.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.669+0000 7f9f4bac7640 1 --2- 172.21.15.164:0/2165595957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f44106f00 0x7f9f4410ea80 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:37.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.669+0000 7f9f4bac7640 1 --2- 172.21.15.164:0/2165595957 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f44102780 0x7f9f441067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:37.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.669+0000 7f9f4bac7640 1 --2- 172.21.15.164:0/2165595957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f44101dd0 0x7f9f441021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:37.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.669+0000 7f9f4bac7640 1 -- 172.21.15.164:0/2165595957 >> 172.21.15.164:0/2165595957 conn(0x7f9f440fd8b0 msgr2=0x7f9f440ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:37.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.669+0000 7f9f4bac7640 1 -- 172.21.15.164:0/2165595957 shutdown_connections 2024-09-19T08:45:37.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.669+0000 7f9f4bac7640 1 -- 172.21.15.164:0/2165595957 wait complete. 2024-09-19T08:45:37.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.669+0000 7f9f4bac7640 1 Processor -- start 2024-09-19T08:45:37.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.669+0000 7f9f4bac7640 1 -- start start 2024-09-19T08:45:37.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.670+0000 7f9f4bac7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f44101dd0 0x7f9f44198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:37.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.670+0000 7f9f4bac7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f44102780 0x7f9f44198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:37.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.670+0000 7f9f4bac7640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f44106f00 0x7f9f4419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:37.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.670+0000 7f9f4bac7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9f44112050 con 0x7f9f44101dd0 2024-09-19T08:45:37.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.670+0000 7f9f4bac7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9f44111ed0 con 0x7f9f44102780 2024-09-19T08:45:37.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.670+0000 7f9f4bac7640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9f441121d0 con 0x7f9f44106f00 2024-09-19T08:45:37.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.670+0000 7f9f49863640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f44101dd0 0x7f9f44198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:37.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.670+0000 7f9f49863640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f44101dd0 0x7f9f44198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43442/0 (socket says 172.21.15.164:43442) 2024-09-19T08:45:37.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.670+0000 7f9f49863640 1 -- 172.21.15.164:0/595267663 learned_addr learned my addr 172.21.15.164:0/595267663 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:37.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.670+0000 7f9f49062640 1 --2- 172.21.15.164:0/595267663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f44102780 0x7f9f44198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:37.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.670+0000 7f9f4a064640 1 --2- 172.21.15.164:0/595267663 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f44106f00 0x7f9f4419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:37.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.670+0000 7f9f49863640 1 -- 172.21.15.164:0/595267663 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f44106f00 msgr2=0x7f9f4419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:37.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.670+0000 7f9f49863640 1 --2- 172.21.15.164:0/595267663 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f44106f00 0x7f9f4419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:37.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.670+0000 7f9f49863640 1 -- 172.21.15.164:0/595267663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f44102780 msgr2=0x7f9f44198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:37.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.670+0000 7f9f49863640 1 --2- 172.21.15.164:0/595267663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f44102780 0x7f9f44198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:37.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.670+0000 7f9f49863640 1 -- 172.21.15.164:0/595267663 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4419d520 con 0x7f9f44101dd0 2024-09-19T08:45:37.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.670+0000 7f9f49863640 1 --2- 172.21.15.164:0/595267663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f44101dd0 0x7f9f44198570 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400ba50 tx=0x7f9f3400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:37.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.670+0000 7f9f42ffd640 1 -- 172.21.15.164:0/595267663 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f34019070 con 0x7f9f44101dd0 2024-09-19T08:45:37.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.671+0000 7f9f42ffd640 1 -- 172.21.15.164:0/595267663 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f34014440 con 0x7f9f44101dd0 2024-09-19T08:45:37.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.671+0000 7f9f4bac7640 1 -- 172.21.15.164:0/595267663 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f4419d810 con 0x7f9f44101dd0 2024-09-19T08:45:37.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.671+0000 7f9f42ffd640 1 -- 172.21.15.164:0/595267663 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f340075d0 con 0x7f9f44101dd0 2024-09-19T08:45:37.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.671+0000 7f9f4bac7640 1 -- 172.21.15.164:0/595267663 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f441a50f0 con 0x7f9f44101dd0 2024-09-19T08:45:37.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.673+0000 7f9f42ffd640 1 -- 172.21.15.164:0/595267663 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f9f3401c070 con 0x7f9f44101dd0 2024-09-19T08:45:37.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.673+0000 7f9f4dea0740 1 Processor -- start 2024-09-19T08:45:37.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.674+0000 7f9f4dea0740 1 -- start start 2024-09-19T08:45:37.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.674+0000 7f9f4dea0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608fd8918b0 0x5608fd891c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:37.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.674+0000 7f9f4dea0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608fd8921d0 0x5608fd8a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:37.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.674+0000 7f9f4dea0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608fd8a0fc0 0x5608fd8a33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:37.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.674+0000 7f9f4dea0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5608fd7f8090 con 0x5608fd8918b0 2024-09-19T08:45:37.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.674+0000 7f9f4dea0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5608fd79fa70 con 0x5608fd8921d0 2024-09-19T08:45:37.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.674+0000 7f9f4dea0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5608fd7f4050 con 0x5608fd8a0fc0 2024-09-19T08:45:37.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.674+0000 7f9f4a064640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608fd8918b0 0x5608fd891c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:37.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.674+0000 7f9f4a064640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608fd8918b0 0x5608fd891c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43456/0 (socket says 172.21.15.164:43456) 2024-09-19T08:45:37.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.674+0000 7f9f4a064640 1 -- 172.21.15.164:0/417622976 learned_addr learned my addr 172.21.15.164:0/417622976 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:37.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.674+0000 7f9f49062640 1 --2- 172.21.15.164:0/417622976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608fd8a0fc0 0x5608fd8a33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:37.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.674+0000 7f9f49863640 1 --2- 172.21.15.164:0/417622976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608fd8921d0 0x5608fd8a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:37.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.674+0000 7f9f4a064640 1 -- 172.21.15.164:0/417622976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608fd8a0fc0 msgr2=0x5608fd8a33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:37.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.674+0000 7f9f4a064640 1 --2- 172.21.15.164:0/417622976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608fd8a0fc0 0x5608fd8a33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:37.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.674+0000 7f9f4a064640 1 -- 172.21.15.164:0/417622976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608fd8921d0 msgr2=0x5608fd8a0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:37.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.674+0000 7f9f4a064640 1 --2- 172.21.15.164:0/417622976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608fd8921d0 0x5608fd8a0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:37.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.674+0000 7f9f4a064640 1 -- 172.21.15.164:0/417622976 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608fd7ac130 con 0x5608fd8918b0 2024-09-19T08:45:37.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.674+0000 7f9f42ffd640 1 --2- 172.21.15.164:0/595267663 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f9f2c0a3240 0x7f9f2c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:37.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.675+0000 7f9f49863640 1 --2- 172.21.15.164:0/595267663 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f9f2c0a3240 0x7f9f2c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:37.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.675+0000 7f9f4a064640 1 --2- 172.21.15.164:0/417622976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608fd8918b0 0x5608fd891c90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c033000 tx=0x7f9f3c0040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a5df0331146660 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:37.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.675+0000 7f9f42ffd640 1 -- 172.21.15.164:0/595267663 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f9f340cee70 con 0x7f9f44101dd0 2024-09-19T08:45:37.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.675+0000 7f9f2bfff640 1 -- 172.21.15.164:0/417622976 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c03f070 con 0x5608fd8918b0 2024-09-19T08:45:37.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.675+0000 7f9f2bfff640 1 -- 172.21.15.164:0/417622976 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f3c004490 con 0x5608fd8918b0 2024-09-19T08:45:37.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.675+0000 7f9f2bfff640 1 -- 172.21.15.164:0/417622976 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c035070 con 0x5608fd8918b0 2024-09-19T08:45:37.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.676+0000 7f9f4dea0740 1 -- 172.21.15.164:0/417622976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608fd8918b0 msgr2=0x5608fd891c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:37.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.676+0000 7f9f49863640 1 --2- 172.21.15.164:0/595267663 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f9f2c0a3240 0x7f9f2c0a5700 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9f34000f80 tx=0x7f9f3400e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:37.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.676+0000 7f9f4dea0740 1 --2- 172.21.15.164:0/417622976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608fd8918b0 0x5608fd891c90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c033000 tx=0x7f9f3c0040f0 comp rx=0 tx=0).stop 2024-09-19T08:45:37.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.676+0000 7f9f4dea0740 1 -- 172.21.15.164:0/417622976 shutdown_connections 2024-09-19T08:45:37.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.676+0000 7f9f4dea0740 1 --2- 172.21.15.164:0/417622976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608fd8a0fc0 0x5608fd8a33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:37.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.676+0000 7f9f4dea0740 1 --2- 172.21.15.164:0/417622976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608fd8921d0 0x5608fd8a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:37.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.676+0000 7f9f4dea0740 1 --2- 172.21.15.164:0/417622976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608fd8918b0 0x5608fd891c90 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:37.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.676+0000 7f9f4dea0740 1 -- 172.21.15.164:0/417622976 >> 172.21.15.164:0/417622976 conn(0x5608fd88df60 msgr2=0x5608fd88e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:37.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.676+0000 7f9f4dea0740 1 -- 172.21.15.164:0/417622976 shutdown_connections 2024-09-19T08:45:37.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.676+0000 7f9f4dea0740 1 -- 172.21.15.164:0/417622976 wait complete. 2024-09-19T08:45:37.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.676+0000 7f9f4dea0740 1 -- start start 2024-09-19T08:45:37.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.676+0000 7f9f4dea0740 1 Processor -- start 2024-09-19T08:45:37.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.677+0000 7f9f4dea0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608fd8918b0 0x5608fd8a0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:37.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.677+0000 7f9f4dea0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608fd8921d0 0x5608fd89dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:37.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.677+0000 7f9f4dea0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608fd8a0fc0 0x5608fd89e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:37.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.677+0000 7f9f4dea0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5608fd8a3b90 con 0x5608fd8a0fc0 2024-09-19T08:45:37.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.677+0000 7f9f4dea0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5608fd7b2f20 con 0x5608fd8921d0 2024-09-19T08:45:37.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.677+0000 7f9f4dea0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5608fd8a3d10 con 0x5608fd8918b0 2024-09-19T08:45:37.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.677+0000 7f9f4dea0740 10 client.0 _ll_register_callbacks cb 0x5608fd77c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:45:37.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.677+0000 7f9f4a064640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608fd8a0fc0 0x5608fd89e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:37.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.677+0000 7f9f4a064640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608fd8a0fc0 0x5608fd89e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43464/0 (socket says 172.21.15.164:43464) 2024-09-19T08:45:37.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.677+0000 7f9f4a064640 1 -- 172.21.15.164:0/3250558897 learned_addr learned my addr 172.21.15.164:0/3250558897 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:37.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.677+0000 7f9f4a064640 1 --2- 172.21.15.164:0/3250558897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608fd8918b0 0x5608fd8a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:37.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.677+0000 7f9f49062640 1 --2- 172.21.15.164:0/3250558897 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608fd8921d0 0x5608fd89dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:37.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.677+0000 7f9f4a064640 1 -- 172.21.15.164:0/3250558897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608fd8918b0 msgr2=0x5608fd8a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:37.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.677+0000 7f9f4a064640 1 --2- 172.21.15.164:0/3250558897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608fd8918b0 0x5608fd8a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:37.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.677+0000 7f9f4a064640 1 -- 172.21.15.164:0/3250558897 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608fd8921d0 msgr2=0x5608fd89dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:37.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.677+0000 7f9f4a064640 1 --2- 172.21.15.164:0/3250558897 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608fd8921d0 0x5608fd89dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:37.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.677+0000 7f9f4a064640 1 -- 172.21.15.164:0/3250558897 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608fd5692b0 con 0x5608fd8a0fc0 2024-09-19T08:45:37.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.677+0000 7f9f4a064640 1 --2- 172.21.15.164:0/3250558897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608fd8a0fc0 0x5608fd89e0e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c004800 tx=0x7f9f3c004d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41afb0dfe44cc914 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:37.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.677+0000 7f9f297fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:45:37.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.677+0000 7f9f297fa640 1 -- 172.21.15.164:0/3250558897 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c03f070 con 0x5608fd8a0fc0 2024-09-19T08:45:37.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.677+0000 7f9f297fa640 1 -- 172.21.15.164:0/3250558897 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f9f3c004290 con 0x5608fd8a0fc0 2024-09-19T08:45:37.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.677+0000 7f9f4dea0740 1 -- 172.21.15.164:0/3250558897 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f3c004490 con 0x5608fd8a0fc0 2024-09-19T08:45:37.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.678+0000 7f9f297fa640 1 -- 172.21.15.164:0/3250558897 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9f3c059050 con 0x5608fd8a0fc0 2024-09-19T08:45:37.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.678+0000 7f9f4dea0740 10 client.15135 fetch_fsmap learned FSMap version 27 2024-09-19T08:45:37.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.678+0000 7f9f4dea0740 1 -- 172.21.15.164:0/3250558897 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5608fd8af4c0 con 0x5608fd8a0fc0 2024-09-19T08:45:37.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.678+0000 7f9f297fa640 1 -- 172.21.15.164:0/3250558897 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f3c004490 con 0x5608fd8a0fc0 2024-09-19T08:45:37.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.678+0000 7f9f297fa640 1 -- 172.21.15.164:0/3250558897 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9f3c051020 con 0x5608fd8a0fc0 2024-09-19T08:45:37.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.678+0000 7f9f4dea0740 10 client.15135 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T08:45:37.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.678+0000 7f9f4dea0740 10 client.15135 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T08:45:37.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.678+0000 7f9f4dea0740 20 client.15135 populate_metadata read hostname 'smithi164' 2024-09-19T08:45:37.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.678+0000 7f9f4dea0740 1 --2- 172.21.15.164:0/3250558897 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x5608fd8b0810 0x5608fd8b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:37.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.678+0000 7f9f4dea0740 4 client.15135 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:45:37.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.678+0000 7f9f4dea0740 1 -- 172.21.15.164:0/3250558897 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608fd805600 con 0x5608fd8b0810 2024-09-19T08:45:37.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.678+0000 7f9f49062640 1 --2- 172.21.15.164:0/3250558897 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x5608fd8b0810 0x5608fd8b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:37.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.679+0000 7f9f49062640 1 --2- 172.21.15.164:0/3250558897 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x5608fd8b0810 0x5608fd8b2a00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=59c44751633cb4bc server_cookie=9ef25e7bf86b326d in_seq=0 out_seq=0 2024-09-19T08:45:37.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.679+0000 7f9f297fa640 10 client.15135 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:45:37.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.679+0000 7f9f297fa640 1 -- 172.21.15.164:0/3250558897 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5608fd805600 con 0x5608fd8b0810 2024-09-19T08:45:37.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.679+0000 7f9f297fa640 10 client.15135 handle_command_reply: tid=0 2024-09-19T08:45:37.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.679+0000 7f9f4dea0740 1 client.15135 shutdown 2024-09-19T08:45:37.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.680+0000 7f9f4dea0740 1 -- 172.21.15.164:0/3250558897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608fd8a0fc0 msgr2=0x5608fd89e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:37.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.680+0000 7f9f4dea0740 1 --2- 172.21.15.164:0/3250558897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608fd8a0fc0 0x5608fd89e0e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c004800 tx=0x7f9f3c004d60 comp rx=0 tx=0).stop 2024-09-19T08:45:37.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.680+0000 7f9f4dea0740 1 -- 172.21.15.164:0/3250558897 shutdown_connections 2024-09-19T08:45:37.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.680+0000 7f9f4dea0740 1 --2- 172.21.15.164:0/3250558897 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x5608fd8b0810 0x5608fd8b2a00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:37.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.680+0000 7f9f4dea0740 1 --2- 172.21.15.164:0/3250558897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608fd8a0fc0 0x5608fd89e0e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:37.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.680+0000 7f9f4dea0740 1 --2- 172.21.15.164:0/3250558897 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608fd8921d0 0x5608fd89dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:37.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.680+0000 7f9f297fa640 0 client.15135 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:45:37.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.680+0000 7f9f4dea0740 1 --2- 172.21.15.164:0/3250558897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608fd8918b0 0x5608fd8a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:37.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.680+0000 7f9f4dea0740 1 -- 172.21.15.164:0/3250558897 >> 172.21.15.164:0/3250558897 conn(0x5608fd88df60 msgr2=0x5608fd893cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:37.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.680+0000 7f9f4dea0740 1 -- 172.21.15.164:0/3250558897 shutdown_connections 2024-09-19T08:45:37.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.680+0000 7f9f4dea0740 1 -- 172.21.15.164:0/3250558897 wait complete. 2024-09-19T08:45:37.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.680+0000 7f9f4dea0740 20 client.15135 trim_cache size 0 max 16384 2024-09-19T08:45:37.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.698+0000 7f9f4dea0740 1 Processor -- start 2024-09-19T08:45:37.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.699+0000 7f9f4dea0740 1 -- start start 2024-09-19T08:45:37.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.699+0000 7f9f4dea0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608fd8918b0 0x5608fd8b1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:37.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.699+0000 7f9f4dea0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608fd8921d0 0x5608fd8b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:37.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.699+0000 7f9f4dea0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608fd8a0fc0 0x5608fd8b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:37.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.699+0000 7f9f4dea0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5608fd8b0d40 con 0x5608fd8a0fc0 2024-09-19T08:45:37.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.699+0000 7f9f4dea0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5608fd8b1410 con 0x5608fd8918b0 2024-09-19T08:45:37.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.699+0000 7f9f4dea0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5608fd8b1ae0 con 0x5608fd8921d0 2024-09-19T08:45:37.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.699+0000 7f9f4a064640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608fd8a0fc0 0x5608fd8b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:37.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.699+0000 7f9f4a064640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608fd8a0fc0 0x5608fd8b0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43478/0 (socket says 172.21.15.164:43478) 2024-09-19T08:45:37.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.699+0000 7f9f4a064640 1 -- 172.21.15.164:0/365944497 learned_addr learned my addr 172.21.15.164:0/365944497 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:37.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.699+0000 7f9f4a064640 1 --2- 172.21.15.164:0/365944497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608fd8918b0 0x5608fd8b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:37.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.699+0000 7f9f49062640 1 --2- 172.21.15.164:0/365944497 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608fd8921d0 0x5608fd8b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:37.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.699+0000 7f9f4a064640 1 -- 172.21.15.164:0/365944497 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608fd8921d0 msgr2=0x5608fd8b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:37.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.699+0000 7f9f4a064640 1 --2- 172.21.15.164:0/365944497 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608fd8921d0 0x5608fd8b1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:37.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.699+0000 7f9f4a064640 1 -- 172.21.15.164:0/365944497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608fd8918b0 msgr2=0x5608fd8b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:37.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.699+0000 7f9f4a064640 1 --2- 172.21.15.164:0/365944497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608fd8918b0 0x5608fd8b1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:37.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.699+0000 7f9f4a064640 1 -- 172.21.15.164:0/365944497 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608fd892000 con 0x5608fd8a0fc0 2024-09-19T08:45:37.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.699+0000 7f9f49062640 1 --2- 172.21.15.164:0/365944497 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608fd8921d0 0x5608fd8b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:37.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.699+0000 7f9f4a064640 1 --2- 172.21.15.164:0/365944497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608fd8a0fc0 0x5608fd8b0ec0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c004d30 tx=0x7f9f3c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b8d2a5d137d29fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:37.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.700+0000 7f9f2b7fe640 1 -- 172.21.15.164:0/365944497 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c03f020 con 0x5608fd8a0fc0 2024-09-19T08:45:37.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.700+0000 7f9f2b7fe640 1 -- 172.21.15.164:0/365944497 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f3c004a70 con 0x5608fd8a0fc0 2024-09-19T08:45:37.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.700+0000 7f9f2b7fe640 1 -- 172.21.15.164:0/365944497 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c035040 con 0x5608fd8a0fc0 2024-09-19T08:45:37.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.700+0000 7f9f4dea0740 1 -- 172.21.15.164:0/365944497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608fd8a0fc0 msgr2=0x5608fd8b0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:37.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.700+0000 7f9f4dea0740 1 --2- 172.21.15.164:0/365944497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608fd8a0fc0 0x5608fd8b0ec0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c004d30 tx=0x7f9f3c005fd0 comp rx=0 tx=0).stop 2024-09-19T08:45:37.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.700+0000 7f9f4dea0740 1 -- 172.21.15.164:0/365944497 shutdown_connections 2024-09-19T08:45:37.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.700+0000 7f9f4dea0740 1 --2- 172.21.15.164:0/365944497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608fd8a0fc0 0x5608fd8b0ec0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:37.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.700+0000 7f9f4dea0740 1 --2- 172.21.15.164:0/365944497 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608fd8921d0 0x5608fd8b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:37.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.700+0000 7f9f4dea0740 1 --2- 172.21.15.164:0/365944497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608fd8918b0 0x5608fd8b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:37.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.700+0000 7f9f4dea0740 1 -- 172.21.15.164:0/365944497 >> 172.21.15.164:0/365944497 conn(0x5608fd88df60 msgr2=0x5608fd88e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:37.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.700+0000 7f9f4dea0740 1 -- 172.21.15.164:0/365944497 shutdown_connections 2024-09-19T08:45:37.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.700+0000 7f9f4dea0740 1 -- 172.21.15.164:0/365944497 wait complete. 2024-09-19T08:45:37.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.701+0000 7f9f4dea0740 1 -- start start 2024-09-19T08:45:37.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.701+0000 7f9f4dea0740 1 Processor -- start 2024-09-19T08:45:37.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.701+0000 7f9f4dea0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608fd8918b0 0x5608fd899e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:37.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.701+0000 7f9f4dea0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608fd8921d0 0x5608fd89a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:37.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.701+0000 7f9f4dea0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608fd8a0fc0 0x5608fd89a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:37.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.701+0000 7f9f4dea0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5608fd8b2880 con 0x5608fd8921d0 2024-09-19T08:45:37.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.701+0000 7f9f4dea0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5608fd8b21b0 con 0x5608fd8918b0 2024-09-19T08:45:37.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.701+0000 7f9f4dea0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5608fd8b2f50 con 0x5608fd8a0fc0 2024-09-19T08:45:37.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.701+0000 7f9f4dea0740 10 client.0 _ll_register_callbacks cb 0x7f9f340d2860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:45:37.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.701+0000 7f9f49062640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608fd8921d0 0x5608fd89a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:37.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.701+0000 7f9f49062640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608fd8921d0 0x5608fd89a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43484/0 (socket says 172.21.15.164:43484) 2024-09-19T08:45:37.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.701+0000 7f9f49062640 1 -- 172.21.15.164:0/1633669145 learned_addr learned my addr 172.21.15.164:0/1633669145 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:37.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.701+0000 7f9f4a064640 1 --2- 172.21.15.164:0/1633669145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608fd8a0fc0 0x5608fd89a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:37.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.702+0000 7f9f4a064640 1 --2- 172.21.15.164:0/1633669145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608fd8918b0 0x5608fd899e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:37.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.702+0000 7f9f49062640 1 -- 172.21.15.164:0/1633669145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608fd8a0fc0 msgr2=0x5608fd89a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:37.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.702+0000 7f9f49062640 1 --2- 172.21.15.164:0/1633669145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608fd8a0fc0 0x5608fd89a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:37.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.702+0000 7f9f49062640 1 -- 172.21.15.164:0/1633669145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608fd8918b0 msgr2=0x5608fd899e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:37.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.702+0000 7f9f49062640 1 --2- 172.21.15.164:0/1633669145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608fd8918b0 0x5608fd899e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:37.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.702+0000 7f9f49062640 1 -- 172.21.15.164:0/1633669145 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608fd8b9cf0 con 0x5608fd8921d0 2024-09-19T08:45:37.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.702+0000 7f9f4a064640 1 --2- 172.21.15.164:0/1633669145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608fd8a0fc0 0x5608fd89a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:37.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.702+0000 7f9f49062640 1 --2- 172.21.15.164:0/1633669145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608fd8921d0 0x5608fd89a390 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9f38009bb0 tx=0x7f9f38006c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b192715747e775 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:37.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.702+0000 7f9f297fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:45:37.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.702+0000 7f9f297fa640 1 -- 172.21.15.164:0/1633669145 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f38017af0 con 0x5608fd8921d0 2024-09-19T08:45:37.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.702+0000 7f9f297fa640 1 -- 172.21.15.164:0/1633669145 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f9f38017c90 con 0x5608fd8921d0 2024-09-19T08:45:37.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.702+0000 7f9f4dea0740 1 -- 172.21.15.164:0/1633669145 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f3c004a70 con 0x5608fd8921d0 2024-09-19T08:45:37.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.702+0000 7f9f297fa640 1 -- 172.21.15.164:0/1633669145 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9f38017e90 con 0x5608fd8921d0 2024-09-19T08:45:37.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.702+0000 7f9f4dea0740 10 client.15141 fetch_fsmap learned FSMap version 27 2024-09-19T08:45:37.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.702+0000 7f9f4dea0740 1 -- 172.21.15.164:0/1633669145 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5608fd8bafd0 con 0x5608fd8921d0 2024-09-19T08:45:37.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.702+0000 7f9f297fa640 1 -- 172.21.15.164:0/1633669145 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f3c004a70 con 0x5608fd8921d0 2024-09-19T08:45:37.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.702+0000 7f9f297fa640 1 -- 172.21.15.164:0/1633669145 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9f380297e0 con 0x5608fd8921d0 2024-09-19T08:45:37.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.703+0000 7f9f4dea0740 10 client.15141 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T08:45:37.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.703+0000 7f9f4dea0740 10 client.15141 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T08:45:37.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.703+0000 7f9f4dea0740 20 client.15141 populate_metadata read hostname 'smithi164' 2024-09-19T08:45:37.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.703+0000 7f9f4dea0740 1 --2- 172.21.15.164:0/1633669145 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x5608fd91fd30 0x5608fd91ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:37.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.703+0000 7f9f4dea0740 4 client.15141 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:45:37.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.703+0000 7f9f4dea0740 1 -- 172.21.15.164:0/1633669145 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "status"}) -- 0x5608fd7ffb10 con 0x5608fd91fd30 2024-09-19T08:45:37.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.703+0000 7f9f4a064640 1 --2- 172.21.15.164:0/1633669145 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x5608fd91fd30 0x5608fd91ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:37.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.703+0000 7f9f4a064640 1 --2- 172.21.15.164:0/1633669145 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x5608fd91fd30 0x5608fd91ea40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6200826a96e5690 server_cookie=682e201737c2bec1 in_seq=0 out_seq=0 2024-09-19T08:45:37.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.703+0000 7f9f297fa640 10 client.15141 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:45:37.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.703+0000 7f9f297fa640 1 -- 172.21.15.164:0/1633669145 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5608fd7ffb10 con 0x5608fd91fd30 2024-09-19T08:45:37.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.703+0000 7f9f297fa640 10 client.15141 handle_command_reply: tid=0 2024-09-19T08:45:37.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.703+0000 7f9f4dea0740 1 client.15141 shutdown 2024-09-19T08:45:37.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.703+0000 7f9f4dea0740 1 -- 172.21.15.164:0/1633669145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608fd8921d0 msgr2=0x5608fd89a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:37.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.703+0000 7f9f4dea0740 1 --2- 172.21.15.164:0/1633669145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608fd8921d0 0x5608fd89a390 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9f38009bb0 tx=0x7f9f38006c30 comp rx=0 tx=0).stop 2024-09-19T08:45:37.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.704+0000 7f9f4dea0740 1 -- 172.21.15.164:0/1633669145 shutdown_connections 2024-09-19T08:45:37.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.704+0000 7f9f4dea0740 1 --2- 172.21.15.164:0/1633669145 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x5608fd91fd30 0x5608fd91ea40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:37.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.704+0000 7f9f4dea0740 1 --2- 172.21.15.164:0/1633669145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608fd8a0fc0 0x5608fd89a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:37.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.704+0000 7f9f4dea0740 1 --2- 172.21.15.164:0/1633669145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608fd8921d0 0x5608fd89a390 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:37.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.704+0000 7f9f297fa640 0 client.15141 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:45:37.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.704+0000 7f9f4dea0740 1 --2- 172.21.15.164:0/1633669145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608fd8918b0 0x5608fd899e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:37.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.704+0000 7f9f4dea0740 1 -- 172.21.15.164:0/1633669145 >> 172.21.15.164:0/1633669145 conn(0x5608fd88df60 msgr2=0x5608fd899240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:37.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.704+0000 7f9f4dea0740 1 -- 172.21.15.164:0/1633669145 shutdown_connections 2024-09-19T08:45:37.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.704+0000 7f9f4dea0740 1 -- 172.21.15.164:0/1633669145 wait complete. 2024-09-19T08:45:37.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.704+0000 7f9f4dea0740 20 client.15141 trim_cache size 0 max 16384 2024-09-19T08:45:37.713 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:45:37.713 INFO:teuthology.orchestra.run.smithi164.stdout: "cluster_fsid": "1e4b0d74-7662-11ef-bceb-c7b262605968", 2024-09-19T08:45:37.713 INFO:teuthology.orchestra.run.smithi164.stdout: "whoami": 2, 2024-09-19T08:45:37.713 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 24442, 2024-09-19T08:45:37.714 INFO:teuthology.orchestra.run.smithi164.stdout: "want_state": "up:active", 2024-09-19T08:45:37.714 INFO:teuthology.orchestra.run.smithi164.stdout: "state": "up:active", 2024-09-19T08:45:37.714 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs", 2024-09-19T08:45:37.714 INFO:teuthology.orchestra.run.smithi164.stdout: "rank_uptime": 95.887989535000003, 2024-09-19T08:45:37.714 INFO:teuthology.orchestra.run.smithi164.stdout: "mdsmap_epoch": 27, 2024-09-19T08:45:37.714 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch": 82, 2024-09-19T08:45:37.714 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch_barrier": 82, 2024-09-19T08:45:37.714 INFO:teuthology.orchestra.run.smithi164.stdout: "uptime": 124.316213301 2024-09-19T08:45:37.714 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:45:37.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.704+0000 7f9f4bac7640 1 -- 172.21.15.164:0/595267663 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f9f2c0a3240 msgr2=0x7f9f2c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:37.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.704+0000 7f9f4bac7640 1 --2- 172.21.15.164:0/595267663 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f9f2c0a3240 0x7f9f2c0a5700 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9f34000f80 tx=0x7f9f3400e040 comp rx=0 tx=0).stop 2024-09-19T08:45:37.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.704+0000 7f9f4bac7640 1 -- 172.21.15.164:0/595267663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f44101dd0 msgr2=0x7f9f44198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:37.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.704+0000 7f9f4bac7640 1 --2- 172.21.15.164:0/595267663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f44101dd0 0x7f9f44198570 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400ba50 tx=0x7f9f3400be50 comp rx=0 tx=0).stop 2024-09-19T08:45:37.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.704+0000 7f9f4bac7640 1 -- 172.21.15.164:0/595267663 shutdown_connections 2024-09-19T08:45:37.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.704+0000 7f9f4bac7640 1 --2- 172.21.15.164:0/595267663 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f9f2c0a3240 0x7f9f2c0a5700 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:37.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.704+0000 7f9f4bac7640 1 --2- 172.21.15.164:0/595267663 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f44106f00 0x7f9f4419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:37.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.704+0000 7f9f4bac7640 1 --2- 172.21.15.164:0/595267663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f44102780 0x7f9f44198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:37.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.704+0000 7f9f4bac7640 1 --2- 172.21.15.164:0/595267663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f44101dd0 0x7f9f44198570 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:37.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.704+0000 7f9f4bac7640 1 -- 172.21.15.164:0/595267663 >> 172.21.15.164:0/595267663 conn(0x7f9f440fd8b0 msgr2=0x7f9f440fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:37.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.704+0000 7f9f4bac7640 1 -- 172.21.15.164:0/595267663 shutdown_connections 2024-09-19T08:45:37.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:37.704+0000 7f9f4bac7640 1 -- 172.21.15.164:0/595267663 wait complete. 2024-09-19T08:45:38.579 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T08:45:38.825 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:45:38.947 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:38 smithi164 ceph-mon[30928]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T08:45:39.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:38 smithi170 ceph-mon[35072]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T08:45:39.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:38 smithi173 ceph-mon[35306]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T08:45:39.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.309+0000 7fa670dac640 1 -- 172.21.15.164:0/1010644043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa66c106a80 msgr2=0x7fa66c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:39.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.309+0000 7fa670dac640 1 --2- 172.21.15.164:0/1010644043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa66c106a80 0x7fa66c106e60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa65c009930 tx=0x7fa65c02a420 comp rx=0 tx=0).stop 2024-09-19T08:45:39.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.309+0000 7fa670dac640 1 -- 172.21.15.164:0/1010644043 shutdown_connections 2024-09-19T08:45:39.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.309+0000 7fa670dac640 1 --2- 172.21.15.164:0/1010644043 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa66c101420 0x7fa66c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.309+0000 7fa670dac640 1 --2- 172.21.15.164:0/1010644043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa66c100a80 0x7fa66c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.309+0000 7fa670dac640 1 --2- 172.21.15.164:0/1010644043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa66c106a80 0x7fa66c106e60 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.309+0000 7fa670dac640 1 -- 172.21.15.164:0/1010644043 >> 172.21.15.164:0/1010644043 conn(0x7fa66c0fc7a0 msgr2=0x7fa66c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:39.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.309+0000 7fa670dac640 1 -- 172.21.15.164:0/1010644043 shutdown_connections 2024-09-19T08:45:39.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.309+0000 7fa670dac640 1 -- 172.21.15.164:0/1010644043 wait complete. 2024-09-19T08:45:39.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.310+0000 7fa670dac640 1 Processor -- start 2024-09-19T08:45:39.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.310+0000 7fa670dac640 1 -- start start 2024-09-19T08:45:39.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.310+0000 7fa670dac640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa66c100a80 0x7fa66c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:39.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.310+0000 7fa670dac640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa66c101420 0x7fa66c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:39.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.310+0000 7fa670dac640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa66c106a80 0x7fa66c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:39.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.310+0000 7fa670dac640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c110e80 con 0x7fa66c100a80 2024-09-19T08:45:39.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.310+0000 7fa670dac640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c110d00 con 0x7fa66c101420 2024-09-19T08:45:39.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.310+0000 7fa670dac640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c111000 con 0x7fa66c106a80 2024-09-19T08:45:39.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.310+0000 7fa66a59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa66c100a80 0x7fa66c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:39.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.310+0000 7fa66a59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa66c100a80 0x7fa66c198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43502/0 (socket says 172.21.15.164:43502) 2024-09-19T08:45:39.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.310+0000 7fa66a59c640 1 -- 172.21.15.164:0/2104320720 learned_addr learned my addr 172.21.15.164:0/2104320720 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:39.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.310+0000 7fa663fff640 1 --2- 172.21.15.164:0/2104320720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa66c101420 0x7fa66c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:39.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.310+0000 7fa66ad9d640 1 --2- 172.21.15.164:0/2104320720 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa66c106a80 0x7fa66c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:39.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.310+0000 7fa66a59c640 1 -- 172.21.15.164:0/2104320720 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa66c106a80 msgr2=0x7fa66c19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:39.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.310+0000 7fa66a59c640 1 --2- 172.21.15.164:0/2104320720 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa66c106a80 0x7fa66c19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.310+0000 7fa66a59c640 1 -- 172.21.15.164:0/2104320720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa66c101420 msgr2=0x7fa66c1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:39.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.310+0000 7fa66a59c640 1 --2- 172.21.15.164:0/2104320720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa66c101420 0x7fa66c1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.310+0000 7fa66a59c640 1 -- 172.21.15.164:0/2104320720 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa66c19d340 con 0x7fa66c100a80 2024-09-19T08:45:39.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.311+0000 7fa66a59c640 1 --2- 172.21.15.164:0/2104320720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa66c100a80 0x7fa66c198390 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa65c009230 tx=0x7fa65c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:39.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.311+0000 7fa6637fe640 1 -- 172.21.15.164:0/2104320720 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65c03f030 con 0x7fa66c100a80 2024-09-19T08:45:39.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.311+0000 7fa6637fe640 1 -- 172.21.15.164:0/2104320720 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa65c003ad0 con 0x7fa66c100a80 2024-09-19T08:45:39.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.311+0000 7fa6637fe640 1 -- 172.21.15.164:0/2104320720 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65c035070 con 0x7fa66c100a80 2024-09-19T08:45:39.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.311+0000 7fa670dac640 1 -- 172.21.15.164:0/2104320720 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa66c19d5d0 con 0x7fa66c100a80 2024-09-19T08:45:39.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.311+0000 7fa670dac640 1 -- 172.21.15.164:0/2104320720 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa66c1a4f10 con 0x7fa66c100a80 2024-09-19T08:45:39.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.313+0000 7fa6637fe640 1 -- 172.21.15.164:0/2104320720 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fa65c042030 con 0x7fa66c100a80 2024-09-19T08:45:39.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.313+0000 7fa673185740 1 Processor -- start 2024-09-19T08:45:39.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.314+0000 7fa673185740 1 -- start start 2024-09-19T08:45:39.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.314+0000 7fa673185740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558277ba69d0 0x558277ba6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:39.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.314+0000 7fa673185740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558277ba72f0 0x558277bb1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:39.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.314+0000 7fa6637fe640 1 --2- 172.21.15.164:0/2104320720 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fa6580a32c0 0x7fa6580a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:39.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.314+0000 7fa673185740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558277bb1bb0 0x558277bb3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:39.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.314+0000 7fa673185740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558277b0d090 con 0x558277bb1bb0 2024-09-19T08:45:39.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.314+0000 7fa673185740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558277ab4a70 con 0x558277ba69d0 2024-09-19T08:45:39.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.314+0000 7fa673185740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558277b09050 con 0x558277ba72f0 2024-09-19T08:45:39.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.314+0000 7fa66ad9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558277bb1bb0 0x558277bb3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:39.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.314+0000 7fa663fff640 1 --2- 172.21.15.164:0/2104320720 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fa6580a32c0 0x7fa6580a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:39.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.314+0000 7fa66ad9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558277bb1bb0 0x558277bb3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43504/0 (socket says 172.21.15.164:43504) 2024-09-19T08:45:39.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.314+0000 7fa66ad9d640 1 -- 172.21.15.164:0/2520850980 learned_addr learned my addr 172.21.15.164:0/2520850980 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:39.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.314+0000 7fa66a59c640 1 --2- 172.21.15.164:0/2520850980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558277ba72f0 0x558277bb1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:39.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.314+0000 7fa6637fe640 1 -- 172.21.15.164:0/2104320720 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa65c0efda0 con 0x7fa66c100a80 2024-09-19T08:45:39.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.314+0000 7fa66ad9d640 1 --2- 172.21.15.164:0/2520850980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558277ba69d0 0x558277ba6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:39.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.314+0000 7fa66ad9d640 1 -- 172.21.15.164:0/2520850980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558277ba72f0 msgr2=0x558277bb1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:39.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.314+0000 7fa66ad9d640 1 --2- 172.21.15.164:0/2520850980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558277ba72f0 0x558277bb1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.314+0000 7fa66ad9d640 1 -- 172.21.15.164:0/2520850980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558277ba69d0 msgr2=0x558277ba6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:39.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.314+0000 7fa66ad9d640 1 --2- 172.21.15.164:0/2520850980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558277ba69d0 0x558277ba6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.314+0000 7fa66ad9d640 1 -- 172.21.15.164:0/2520850980 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558277ac1130 con 0x558277bb1bb0 2024-09-19T08:45:39.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.314+0000 7fa663fff640 1 --2- 172.21.15.164:0/2104320720 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fa6580a32c0 0x7fa6580a5780 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa66c06b870 tx=0x7fa65400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:39.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.315+0000 7fa66ad9d640 1 --2- 172.21.15.164:0/2520850980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558277bb1bb0 0x558277bb3fa0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa664012a70 tx=0x7fa6640077c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c276fced1a862a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:39.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.315+0000 7fa660ff9640 1 -- 172.21.15.164:0/2520850980 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa664004d20 con 0x558277bb1bb0 2024-09-19T08:45:39.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.315+0000 7fa660ff9640 1 -- 172.21.15.164:0/2520850980 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa664005020 con 0x558277bb1bb0 2024-09-19T08:45:39.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.315+0000 7fa660ff9640 1 -- 172.21.15.164:0/2520850980 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66400e640 con 0x558277bb1bb0 2024-09-19T08:45:39.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.315+0000 7fa673185740 1 -- 172.21.15.164:0/2520850980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558277bb1bb0 msgr2=0x558277bb3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:39.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.315+0000 7fa673185740 1 --2- 172.21.15.164:0/2520850980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558277bb1bb0 0x558277bb3fa0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa664012a70 tx=0x7fa6640077c0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.316+0000 7fa673185740 1 -- 172.21.15.164:0/2520850980 shutdown_connections 2024-09-19T08:45:39.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.316+0000 7fa673185740 1 --2- 172.21.15.164:0/2520850980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558277bb1bb0 0x558277bb3fa0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.316+0000 7fa673185740 1 --2- 172.21.15.164:0/2520850980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558277ba72f0 0x558277bb1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.316+0000 7fa673185740 1 --2- 172.21.15.164:0/2520850980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558277ba69d0 0x558277ba6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.316+0000 7fa673185740 1 -- 172.21.15.164:0/2520850980 >> 172.21.15.164:0/2520850980 conn(0x558277ba2f60 msgr2=0x558277ba3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:39.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.316+0000 7fa673185740 1 -- 172.21.15.164:0/2520850980 shutdown_connections 2024-09-19T08:45:39.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.316+0000 7fa673185740 1 -- 172.21.15.164:0/2520850980 wait complete. 2024-09-19T08:45:39.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.316+0000 7fa673185740 1 -- start start 2024-09-19T08:45:39.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.316+0000 7fa673185740 1 Processor -- start 2024-09-19T08:45:39.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.316+0000 7fa673185740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558277ba69d0 0x558277bb1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:39.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.316+0000 7fa673185740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558277ba72f0 0x558277bae7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:39.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.316+0000 7fa673185740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558277bb1bb0 0x558277baece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:39.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.316+0000 7fa673185740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558277bb4780 con 0x558277ba72f0 2024-09-19T08:45:39.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.316+0000 7fa673185740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558277ac7f20 con 0x558277ba69d0 2024-09-19T08:45:39.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.316+0000 7fa673185740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558277bb4900 con 0x558277bb1bb0 2024-09-19T08:45:39.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.316+0000 7fa673185740 10 client.0 _ll_register_callbacks cb 0x558277a91030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:45:39.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.316+0000 7fa663fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558277ba72f0 0x558277bae7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:39.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.317+0000 7fa663fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558277ba72f0 0x558277bae7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43520/0 (socket says 172.21.15.164:43520) 2024-09-19T08:45:39.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.317+0000 7fa663fff640 1 -- 172.21.15.164:0/2942247498 learned_addr learned my addr 172.21.15.164:0/2942247498 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:39.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.317+0000 7fa66a59c640 1 --2- 172.21.15.164:0/2942247498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558277ba69d0 0x558277bb1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:39.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.317+0000 7fa66ad9d640 1 --2- 172.21.15.164:0/2942247498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558277bb1bb0 0x558277baece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:39.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.317+0000 7fa663fff640 1 -- 172.21.15.164:0/2942247498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558277bb1bb0 msgr2=0x558277baece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:39.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.317+0000 7fa663fff640 1 --2- 172.21.15.164:0/2942247498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558277bb1bb0 0x558277baece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.317+0000 7fa663fff640 1 -- 172.21.15.164:0/2942247498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558277ba69d0 msgr2=0x558277bb1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:39.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.317+0000 7fa663fff640 1 --2- 172.21.15.164:0/2942247498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558277ba69d0 0x558277bb1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.317+0000 7fa663fff640 1 -- 172.21.15.164:0/2942247498 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55827787e2b0 con 0x558277ba72f0 2024-09-19T08:45:39.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.317+0000 7fa663fff640 1 --2- 172.21.15.164:0/2942247498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558277ba72f0 0x558277bae7a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa654011010 tx=0x7fa654004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d752beed98bcdc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:39.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.317+0000 7fa651ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:45:39.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.317+0000 7fa651ffb640 1 -- 172.21.15.164:0/2942247498 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65401e8f0 con 0x558277ba72f0 2024-09-19T08:45:39.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.317+0000 7fa651ffb640 1 -- 172.21.15.164:0/2942247498 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa65401ea90 con 0x558277ba72f0 2024-09-19T08:45:39.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.317+0000 7fa673185740 1 -- 172.21.15.164:0/2942247498 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa664005020 con 0x558277ba72f0 2024-09-19T08:45:39.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.317+0000 7fa651ffb640 1 -- 172.21.15.164:0/2942247498 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa65401ec90 con 0x558277ba72f0 2024-09-19T08:45:39.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.317+0000 7fa673185740 10 client.15153 fetch_fsmap learned FSMap version 27 2024-09-19T08:45:39.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.317+0000 7fa673185740 1 -- 172.21.15.164:0/2942247498 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558277bc0100 con 0x558277ba72f0 2024-09-19T08:45:39.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.317+0000 7fa651ffb640 1 -- 172.21.15.164:0/2942247498 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa664005020 con 0x558277ba72f0 2024-09-19T08:45:39.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.318+0000 7fa651ffb640 1 -- 172.21.15.164:0/2942247498 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa65401c3b0 con 0x558277ba72f0 2024-09-19T08:45:39.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.318+0000 7fa673185740 10 client.15153 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T08:45:39.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.318+0000 7fa673185740 10 client.15153 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T08:45:39.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.318+0000 7fa673185740 20 client.15153 populate_metadata read hostname 'smithi164' 2024-09-19T08:45:39.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.318+0000 7fa673185740 1 --2- 172.21.15.164:0/2942247498 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x558277bc2760 0x558277bc3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:39.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.318+0000 7fa673185740 4 client.15153 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:45:39.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.318+0000 7fa673185740 1 -- 172.21.15.164:0/2942247498 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558277b1a600 con 0x558277bc2760 2024-09-19T08:45:39.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.318+0000 7fa66a59c640 1 --2- 172.21.15.164:0/2942247498 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x558277bc2760 0x558277bc3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:39.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.318+0000 7fa651ffb640 10 client.15153 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:45:39.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.318+0000 7fa66a59c640 1 --2- 172.21.15.164:0/2942247498 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x558277bc2760 0x558277bc3bd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=834520d251c944e5 server_cookie=f2cc30ababf19f42 in_seq=0 out_seq=0 2024-09-19T08:45:39.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.319+0000 7fa651ffb640 1 -- 172.21.15.164:0/2942247498 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558277b1a600 con 0x558277bc2760 2024-09-19T08:45:39.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.319+0000 7fa651ffb640 10 client.15153 handle_command_reply: tid=0 2024-09-19T08:45:39.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.319+0000 7fa673185740 1 client.15153 shutdown 2024-09-19T08:45:39.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.319+0000 7fa673185740 1 -- 172.21.15.164:0/2942247498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558277ba72f0 msgr2=0x558277bae7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:39.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.319+0000 7fa673185740 1 --2- 172.21.15.164:0/2942247498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558277ba72f0 0x558277bae7a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa654011010 tx=0x7fa654004730 comp rx=0 tx=0).stop 2024-09-19T08:45:39.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.319+0000 7fa673185740 1 -- 172.21.15.164:0/2942247498 shutdown_connections 2024-09-19T08:45:39.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.319+0000 7fa673185740 1 --2- 172.21.15.164:0/2942247498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558277bb1bb0 0x558277baece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.319+0000 7fa673185740 1 --2- 172.21.15.164:0/2942247498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558277ba72f0 0x558277bae7a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.319+0000 7fa673185740 1 --2- 172.21.15.164:0/2942247498 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x558277bc2760 0x558277bc3bd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.319+0000 7fa673185740 1 --2- 172.21.15.164:0/2942247498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558277ba69d0 0x558277bb1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.319+0000 7fa651ffb640 0 client.15153 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:45:39.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.319+0000 7fa673185740 1 -- 172.21.15.164:0/2942247498 >> 172.21.15.164:0/2942247498 conn(0x558277ba2f60 msgr2=0x558277ba8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:39.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.319+0000 7fa673185740 1 -- 172.21.15.164:0/2942247498 shutdown_connections 2024-09-19T08:45:39.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.320+0000 7fa673185740 1 -- 172.21.15.164:0/2942247498 wait complete. 2024-09-19T08:45:39.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.320+0000 7fa673185740 20 client.15153 trim_cache size 0 max 16384 2024-09-19T08:45:39.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.337+0000 7fa673185740 1 Processor -- start 2024-09-19T08:45:39.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.337+0000 7fa673185740 1 -- start start 2024-09-19T08:45:39.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.338+0000 7fa673185740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558277ba69d0 0x558277bad780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:39.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.338+0000 7fa673185740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558277ba72f0 0x558277baba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:39.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.338+0000 7fa673185740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558277bb1bb0 0x558277babf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:39.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.338+0000 7fa673185740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558277bc1f10 con 0x558277ba72f0 2024-09-19T08:45:39.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.338+0000 7fa673185740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558277bc25e0 con 0x558277ba69d0 2024-09-19T08:45:39.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.338+0000 7fa673185740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558277bc2cb0 con 0x558277bb1bb0 2024-09-19T08:45:39.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.338+0000 7fa663fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558277ba72f0 0x558277baba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:39.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.338+0000 7fa663fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558277ba72f0 0x558277baba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43530/0 (socket says 172.21.15.164:43530) 2024-09-19T08:45:39.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.338+0000 7fa663fff640 1 -- 172.21.15.164:0/794011781 learned_addr learned my addr 172.21.15.164:0/794011781 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:39.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.338+0000 7fa66a59c640 1 --2- 172.21.15.164:0/794011781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558277ba69d0 0x558277bad780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:39.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.338+0000 7fa663fff640 1 -- 172.21.15.164:0/794011781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558277bb1bb0 msgr2=0x558277babf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:39.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.338+0000 7fa663fff640 1 --2- 172.21.15.164:0/794011781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558277bb1bb0 0x558277babf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.338+0000 7fa663fff640 1 -- 172.21.15.164:0/794011781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558277ba69d0 msgr2=0x558277bad780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:39.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.338+0000 7fa663fff640 1 --2- 172.21.15.164:0/794011781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558277ba69d0 0x558277bad780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.338+0000 7fa663fff640 1 -- 172.21.15.164:0/794011781 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558277bb1070 con 0x558277ba72f0 2024-09-19T08:45:39.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.338+0000 7fa66a59c640 1 --2- 172.21.15.164:0/794011781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558277ba69d0 0x558277bad780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:39.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.338+0000 7fa663fff640 1 --2- 172.21.15.164:0/794011781 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558277ba72f0 0x558277baba20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa654011fd0 tx=0x7fa654010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181918abcee34bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:39.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.338+0000 7fa653fff640 1 -- 172.21.15.164:0/794011781 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65401ca40 con 0x558277ba72f0 2024-09-19T08:45:39.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.338+0000 7fa653fff640 1 -- 172.21.15.164:0/794011781 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa65401cbe0 con 0x558277ba72f0 2024-09-19T08:45:39.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.338+0000 7fa653fff640 1 -- 172.21.15.164:0/794011781 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa654031a50 con 0x558277ba72f0 2024-09-19T08:45:39.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.339+0000 7fa673185740 1 -- 172.21.15.164:0/794011781 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558277ba72f0 msgr2=0x558277baba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:39.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.339+0000 7fa673185740 1 --2- 172.21.15.164:0/794011781 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558277ba72f0 0x558277baba20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa654011fd0 tx=0x7fa654010d00 comp rx=0 tx=0).stop 2024-09-19T08:45:39.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.339+0000 7fa673185740 1 -- 172.21.15.164:0/794011781 shutdown_connections 2024-09-19T08:45:39.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.339+0000 7fa673185740 1 --2- 172.21.15.164:0/794011781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558277bb1bb0 0x558277babf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.339+0000 7fa673185740 1 --2- 172.21.15.164:0/794011781 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558277ba72f0 0x558277baba20 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.339+0000 7fa673185740 1 --2- 172.21.15.164:0/794011781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558277ba69d0 0x558277bad780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.339+0000 7fa673185740 1 -- 172.21.15.164:0/794011781 >> 172.21.15.164:0/794011781 conn(0x558277ba2f60 msgr2=0x558277bad070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:39.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.339+0000 7fa673185740 1 -- 172.21.15.164:0/794011781 shutdown_connections 2024-09-19T08:45:39.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.339+0000 7fa673185740 1 -- 172.21.15.164:0/794011781 wait complete. 2024-09-19T08:45:39.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.339+0000 7fa673185740 1 -- start start 2024-09-19T08:45:39.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.340+0000 7fa673185740 1 Processor -- start 2024-09-19T08:45:39.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.340+0000 7fa673185740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558277ba69d0 0x558277bc5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:39.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.340+0000 7fa673185740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558277ba72f0 0x558277bc58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:39.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.340+0000 7fa673185740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558277bb1bb0 0x558277c2ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:39.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.340+0000 7fa673185740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558277bc3a50 con 0x558277bb1bb0 2024-09-19T08:45:39.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.340+0000 7fa673185740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558277bc3380 con 0x558277ba72f0 2024-09-19T08:45:39.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.340+0000 7fa673185740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558277bc4120 con 0x558277ba69d0 2024-09-19T08:45:39.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.340+0000 7fa673185740 10 client.0 _ll_register_callbacks cb 0x7fa65400b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:45:39.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.340+0000 7fa66ad9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558277bb1bb0 0x558277c2ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:39.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.340+0000 7fa66ad9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558277bb1bb0 0x558277c2ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43540/0 (socket says 172.21.15.164:43540) 2024-09-19T08:45:39.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.340+0000 7fa66ad9d640 1 -- 172.21.15.164:0/1080869499 learned_addr learned my addr 172.21.15.164:0/1080869499 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:39.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.340+0000 7fa66a59c640 1 --2- 172.21.15.164:0/1080869499 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558277ba69d0 0x558277bc5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:39.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.340+0000 7fa663fff640 1 --2- 172.21.15.164:0/1080869499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558277ba72f0 0x558277bc58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:39.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.340+0000 7fa66ad9d640 1 -- 172.21.15.164:0/1080869499 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558277ba69d0 msgr2=0x558277bc5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:39.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.340+0000 7fa66ad9d640 1 --2- 172.21.15.164:0/1080869499 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558277ba69d0 0x558277bc5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.340+0000 7fa66ad9d640 1 -- 172.21.15.164:0/1080869499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558277ba72f0 msgr2=0x558277bc58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:39.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.340+0000 7fa66ad9d640 1 --2- 172.21.15.164:0/1080869499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558277ba72f0 0x558277bc58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.340+0000 7fa66ad9d640 1 -- 172.21.15.164:0/1080869499 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558277c2b3a0 con 0x558277bb1bb0 2024-09-19T08:45:39.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.340+0000 7fa66ad9d640 1 --2- 172.21.15.164:0/1080869499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558277bb1bb0 0x558277c2ae60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa664012a70 tx=0x7fa664012840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf936d13d177bd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:39.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.340+0000 7fa651ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:45:39.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.341+0000 7fa651ffb640 1 -- 172.21.15.164:0/1080869499 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66402c060 con 0x558277bb1bb0 2024-09-19T08:45:39.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.341+0000 7fa651ffb640 1 -- 172.21.15.164:0/1080869499 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fa664003b70 con 0x558277bb1bb0 2024-09-19T08:45:39.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.341+0000 7fa673185740 1 -- 172.21.15.164:0/1080869499 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa65401cbe0 con 0x558277bb1bb0 2024-09-19T08:45:39.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.341+0000 7fa651ffb640 1 -- 172.21.15.164:0/1080869499 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa664007530 con 0x558277bb1bb0 2024-09-19T08:45:39.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.341+0000 7fa673185740 10 client.15159 fetch_fsmap learned FSMap version 27 2024-09-19T08:45:39.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.341+0000 7fa673185740 1 -- 172.21.15.164:0/1080869499 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558277c2c620 con 0x558277bb1bb0 2024-09-19T08:45:39.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.341+0000 7fa651ffb640 1 -- 172.21.15.164:0/1080869499 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa65401cbe0 con 0x558277bb1bb0 2024-09-19T08:45:39.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.341+0000 7fa651ffb640 1 -- 172.21.15.164:0/1080869499 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa664031070 con 0x558277bb1bb0 2024-09-19T08:45:39.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.341+0000 7fa673185740 10 client.15159 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T08:45:39.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.341+0000 7fa673185740 10 client.15159 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T08:45:39.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.341+0000 7fa673185740 20 client.15159 populate_metadata read hostname 'smithi164' 2024-09-19T08:45:39.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.341+0000 7fa673185740 1 --2- 172.21.15.164:0/1080869499 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x558277bc2760 0x558277c2fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:39.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.341+0000 7fa673185740 4 client.15159 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:45:39.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.341+0000 7fa673185740 1 -- 172.21.15.164:0/1080869499 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "status"}) -- 0x558277b14b10 con 0x558277bc2760 2024-09-19T08:45:39.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.342+0000 7fa66a59c640 1 --2- 172.21.15.164:0/1080869499 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x558277bc2760 0x558277c2fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:39.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.342+0000 7fa651ffb640 10 client.15159 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:45:39.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.342+0000 7fa66a59c640 1 --2- 172.21.15.164:0/1080869499 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x558277bc2760 0x558277c2fbf0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=67b69c93eceab169 server_cookie=99bb08fd25acbb03 in_seq=0 out_seq=0 2024-09-19T08:45:39.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.342+0000 7fa651ffb640 1 -- 172.21.15.164:0/1080869499 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x558277b14b10 con 0x558277bc2760 2024-09-19T08:45:39.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.342+0000 7fa651ffb640 10 client.15159 handle_command_reply: tid=0 2024-09-19T08:45:39.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.342+0000 7fa673185740 1 client.15159 shutdown 2024-09-19T08:45:39.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.342+0000 7fa673185740 1 -- 172.21.15.164:0/1080869499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558277bb1bb0 msgr2=0x558277c2ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:39.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.342+0000 7fa673185740 1 --2- 172.21.15.164:0/1080869499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558277bb1bb0 0x558277c2ae60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa664012a70 tx=0x7fa664012840 comp rx=0 tx=0).stop 2024-09-19T08:45:39.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.342+0000 7fa673185740 1 -- 172.21.15.164:0/1080869499 shutdown_connections 2024-09-19T08:45:39.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.342+0000 7fa673185740 1 --2- 172.21.15.164:0/1080869499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558277bb1bb0 0x558277c2ae60 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.342+0000 7fa673185740 1 --2- 172.21.15.164:0/1080869499 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x558277bc2760 0x558277c2fbf0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.342+0000 7fa673185740 1 --2- 172.21.15.164:0/1080869499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558277ba72f0 0x558277bc58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.342+0000 7fa673185740 1 --2- 172.21.15.164:0/1080869499 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558277ba69d0 0x558277bc5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.342+0000 7fa651ffb640 0 client.15159 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:45:39.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.342+0000 7fa673185740 1 -- 172.21.15.164:0/1080869499 >> 172.21.15.164:0/1080869499 conn(0x558277ba2f60 msgr2=0x558277ba9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:39.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.342+0000 7fa673185740 1 -- 172.21.15.164:0/1080869499 shutdown_connections 2024-09-19T08:45:39.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.342+0000 7fa673185740 1 -- 172.21.15.164:0/1080869499 wait complete. 2024-09-19T08:45:39.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.343+0000 7fa673185740 20 client.15159 trim_cache size 0 max 16384 2024-09-19T08:45:39.352 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:45:39.352 INFO:teuthology.orchestra.run.smithi164.stdout: "cluster_fsid": "1e4b0d74-7662-11ef-bceb-c7b262605968", 2024-09-19T08:45:39.352 INFO:teuthology.orchestra.run.smithi164.stdout: "whoami": 1, 2024-09-19T08:45:39.352 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 24449, 2024-09-19T08:45:39.352 INFO:teuthology.orchestra.run.smithi164.stdout: "want_state": "up:active", 2024-09-19T08:45:39.353 INFO:teuthology.orchestra.run.smithi164.stdout: "state": "up:active", 2024-09-19T08:45:39.353 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs", 2024-09-19T08:45:39.353 INFO:teuthology.orchestra.run.smithi164.stdout: "rank_uptime": 98.545989106999997, 2024-09-19T08:45:39.353 INFO:teuthology.orchestra.run.smithi164.stdout: "mdsmap_epoch": 27, 2024-09-19T08:45:39.353 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch": 82, 2024-09-19T08:45:39.353 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch_barrier": 82, 2024-09-19T08:45:39.353 INFO:teuthology.orchestra.run.smithi164.stdout: "uptime": 128.51858287900001 2024-09-19T08:45:39.353 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:45:39.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.343+0000 7fa670dac640 1 -- 172.21.15.164:0/2104320720 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fa6580a32c0 msgr2=0x7fa6580a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:39.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.343+0000 7fa670dac640 1 --2- 172.21.15.164:0/2104320720 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fa6580a32c0 0x7fa6580a5780 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa66c06b870 tx=0x7fa65400b410 comp rx=0 tx=0).stop 2024-09-19T08:45:39.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.343+0000 7fa670dac640 1 -- 172.21.15.164:0/2104320720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa66c100a80 msgr2=0x7fa66c198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:39.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.343+0000 7fa670dac640 1 --2- 172.21.15.164:0/2104320720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa66c100a80 0x7fa66c198390 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa65c009230 tx=0x7fa65c004270 comp rx=0 tx=0).stop 2024-09-19T08:45:39.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.343+0000 7fa670dac640 1 -- 172.21.15.164:0/2104320720 shutdown_connections 2024-09-19T08:45:39.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.343+0000 7fa670dac640 1 --2- 172.21.15.164:0/2104320720 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fa6580a32c0 0x7fa6580a5780 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.343+0000 7fa670dac640 1 --2- 172.21.15.164:0/2104320720 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa66c106a80 0x7fa66c19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.343+0000 7fa670dac640 1 --2- 172.21.15.164:0/2104320720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa66c101420 0x7fa66c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.343+0000 7fa670dac640 1 --2- 172.21.15.164:0/2104320720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa66c100a80 0x7fa66c198390 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.343+0000 7fa670dac640 1 -- 172.21.15.164:0/2104320720 >> 172.21.15.164:0/2104320720 conn(0x7fa66c0fc7a0 msgr2=0x7fa66c104b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:39.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.343+0000 7fa670dac640 1 -- 172.21.15.164:0/2104320720 shutdown_connections 2024-09-19T08:45:39.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:39.343+0000 7fa670dac640 1 -- 172.21.15.164:0/2104320720 wait complete. 2024-09-19T08:45:40.406 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T08:45:40.652 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:45:41.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:40 smithi164 ceph-mon[30928]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-19T08:45:41.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:40 smithi170 ceph-mon[35072]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-19T08:45:41.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.137+0000 7f93059d9640 1 -- 172.21.15.164:0/3665269311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f930010c170 msgr2=0x7f930010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:41.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.137+0000 7f93059d9640 1 --2- 172.21.15.164:0/3665269311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f930010c170 0x7f930010e610 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f92f8009ca0 tx=0x7f92f802a950 comp rx=0 tx=0).stop 2024-09-19T08:45:41.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.137+0000 7f93059d9640 1 -- 172.21.15.164:0/3665269311 shutdown_connections 2024-09-19T08:45:41.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.137+0000 7f93059d9640 1 --2- 172.21.15.164:0/3665269311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f930010c170 0x7f930010e610 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.137+0000 7f93059d9640 1 --2- 172.21.15.164:0/3665269311 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9300103660 0x7f930010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.137+0000 7f93059d9640 1 --2- 172.21.15.164:0/3665269311 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9300102d40 0x7f9300103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.137+0000 7f93059d9640 1 -- 172.21.15.164:0/3665269311 >> 172.21.15.164:0/3665269311 conn(0x7f93000fc7a0 msgr2=0x7f93000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:41.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.137+0000 7f93059d9640 1 -- 172.21.15.164:0/3665269311 shutdown_connections 2024-09-19T08:45:41.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.137+0000 7f93059d9640 1 -- 172.21.15.164:0/3665269311 wait complete. 2024-09-19T08:45:41.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.137+0000 7f93059d9640 1 Processor -- start 2024-09-19T08:45:41.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.137+0000 7f93059d9640 1 -- start start 2024-09-19T08:45:41.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.138+0000 7f93059d9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9300102d40 0x7f9300198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.138+0000 7f93059d9640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9300103660 0x7f93001989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.138+0000 7f93059d9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f930010c170 0x7f930019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.138+0000 7f93059d9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9300111fb0 con 0x7f930010c170 2024-09-19T08:45:41.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.138+0000 7f93059d9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9300111e30 con 0x7f9300102d40 2024-09-19T08:45:41.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.138+0000 7f93059d9640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9300112130 con 0x7f9300103660 2024-09-19T08:45:41.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.138+0000 7f92ff7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f930010c170 0x7f930019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.138+0000 7f92ff7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f930010c170 0x7f930019cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43570/0 (socket says 172.21.15.164:43570) 2024-09-19T08:45:41.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.138+0000 7f92ff7fe640 1 -- 172.21.15.164:0/2964569371 learned_addr learned my addr 172.21.15.164:0/2964569371 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:41.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.138+0000 7f92feffd640 1 --2- 172.21.15.164:0/2964569371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9300102d40 0x7f9300198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.138+0000 7f92fe7fc640 1 --2- 172.21.15.164:0/2964569371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9300103660 0x7f93001989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.138+0000 7f92ff7fe640 1 -- 172.21.15.164:0/2964569371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9300103660 msgr2=0x7f93001989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:41.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.138+0000 7f92ff7fe640 1 --2- 172.21.15.164:0/2964569371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9300103660 0x7f93001989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.138+0000 7f92ff7fe640 1 -- 172.21.15.164:0/2964569371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9300102d40 msgr2=0x7f9300198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:41.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.138+0000 7f92ff7fe640 1 --2- 172.21.15.164:0/2964569371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9300102d40 0x7f9300198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.138+0000 7f92ff7fe640 1 -- 172.21.15.164:0/2964569371 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f930019d420 con 0x7f930010c170 2024-09-19T08:45:41.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.138+0000 7f92ff7fe640 1 --2- 172.21.15.164:0/2964569371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f930010c170 0x7f930019cd40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f92f8006010 tx=0x7f92f802f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:41.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.139+0000 7f93049d7640 1 -- 172.21.15.164:0/2964569371 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f8034040 con 0x7f930010c170 2024-09-19T08:45:41.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.139+0000 7f93059d9640 1 -- 172.21.15.164:0/2964569371 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f930019d6b0 con 0x7f930010c170 2024-09-19T08:45:41.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.139+0000 7f93049d7640 1 -- 172.21.15.164:0/2964569371 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92f8003e00 con 0x7f930010c170 2024-09-19T08:45:41.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.139+0000 7f93049d7640 1 -- 172.21.15.164:0/2964569371 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f80456e0 con 0x7f930010c170 2024-09-19T08:45:41.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.139+0000 7f93059d9640 1 -- 172.21.15.164:0/2964569371 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93001a4ff0 con 0x7f930010c170 2024-09-19T08:45:41.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.141+0000 7f93049d7640 1 -- 172.21.15.164:0/2964569371 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f92f8037020 con 0x7f930010c170 2024-09-19T08:45:41.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.141+0000 7f9307db2740 1 Processor -- start 2024-09-19T08:45:41.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.141+0000 7f9307db2740 1 -- start start 2024-09-19T08:45:41.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.141+0000 7f93049d7640 1 --2- 172.21.15.164:0/2964569371 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f92ec0a3240 0x7f92ec0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.142+0000 7f92fe7fc640 1 --2- 172.21.15.164:0/2964569371 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f92ec0a3240 0x7f92ec0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.142+0000 7f9307db2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55993b7c9870 0x55993b7c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.142+0000 7f9307db2740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993b7ca190 0x55993b7d8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.142+0000 7f9307db2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55993b7d8fc0 0x55993b7db3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.142+0000 7f93049d7640 1 -- 172.21.15.164:0/2964569371 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f92f80f0930 con 0x7f930010c170 2024-09-19T08:45:41.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.142+0000 7f9307db2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55993b730090 con 0x55993b7c9870 2024-09-19T08:45:41.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.142+0000 7f9307db2740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55993b6d7a70 con 0x55993b7ca190 2024-09-19T08:45:41.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.142+0000 7f9307db2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55993b72c050 con 0x55993b7d8fc0 2024-09-19T08:45:41.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.142+0000 7f92feffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55993b7c9870 0x55993b7c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.142+0000 7f92feffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55993b7c9870 0x55993b7c9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43580/0 (socket says 172.21.15.164:43580) 2024-09-19T08:45:41.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.142+0000 7f92feffd640 1 -- 172.21.15.164:0/901828127 learned_addr learned my addr 172.21.15.164:0/901828127 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:41.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.142+0000 7f92feffd640 1 -- 172.21.15.164:0/901828127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55993b7d8fc0 msgr2=0x55993b7db3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:41.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.142+0000 7f92fe7fc640 1 --2- 172.21.15.164:0/901828127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993b7ca190 0x55993b7d8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.142+0000 7f92feffd640 1 --2- 172.21.15.164:0/901828127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55993b7d8fc0 0x55993b7db3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.142+0000 7f92feffd640 1 -- 172.21.15.164:0/901828127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993b7ca190 msgr2=0x55993b7d8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:41.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.142+0000 7f92feffd640 1 --2- 172.21.15.164:0/901828127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993b7ca190 0x55993b7d8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.142+0000 7f92feffd640 1 -- 172.21.15.164:0/901828127 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55993b6e4130 con 0x55993b7c9870 2024-09-19T08:45:41.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.142+0000 7f92fe7fc640 1 --2- 172.21.15.164:0/2964569371 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f92ec0a3240 0x7f92ec0a5700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f93000feda0 tx=0x7f92f0009de0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:41.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.142+0000 7f92feffd640 1 --2- 172.21.15.164:0/901828127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55993b7c9870 0x55993b7c9c50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f92e800ac20 tx=0x7f92e800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b4ed9ecfe5005a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:41.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.142+0000 7f92f5ffb640 1 -- 172.21.15.164:0/901828127 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e801d070 con 0x55993b7c9870 2024-09-19T08:45:41.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.142+0000 7f92f5ffb640 1 -- 172.21.15.164:0/901828127 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92e8018b20 con 0x55993b7c9870 2024-09-19T08:45:41.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.142+0000 7f92f5ffb640 1 -- 172.21.15.164:0/901828127 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e8010ba0 con 0x55993b7c9870 2024-09-19T08:45:41.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.143+0000 7f9307db2740 1 -- 172.21.15.164:0/901828127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55993b7c9870 msgr2=0x55993b7c9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:41.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.143+0000 7f9307db2740 1 --2- 172.21.15.164:0/901828127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55993b7c9870 0x55993b7c9c50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f92e800ac20 tx=0x7f92e800f790 comp rx=0 tx=0).stop 2024-09-19T08:45:41.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.143+0000 7f9307db2740 1 -- 172.21.15.164:0/901828127 shutdown_connections 2024-09-19T08:45:41.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.143+0000 7f9307db2740 1 --2- 172.21.15.164:0/901828127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55993b7d8fc0 0x55993b7db3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.143+0000 7f9307db2740 1 --2- 172.21.15.164:0/901828127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993b7ca190 0x55993b7d8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.143+0000 7f9307db2740 1 --2- 172.21.15.164:0/901828127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55993b7c9870 0x55993b7c9c50 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.143+0000 7f9307db2740 1 -- 172.21.15.164:0/901828127 >> 172.21.15.164:0/901828127 conn(0x55993b7c5f60 msgr2=0x55993b7c6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:41.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.143+0000 7f9307db2740 1 -- 172.21.15.164:0/901828127 shutdown_connections 2024-09-19T08:45:41.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.143+0000 7f9307db2740 1 -- 172.21.15.164:0/901828127 wait complete. 2024-09-19T08:45:41.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.143+0000 7f9307db2740 1 -- start start 2024-09-19T08:45:41.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.144+0000 7f9307db2740 1 Processor -- start 2024-09-19T08:45:41.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.144+0000 7f9307db2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55993b7c9870 0x55993b7d8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.144+0000 7f9307db2740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993b7ca190 0x55993b7d5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.144+0000 7f9307db2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55993b7d8fc0 0x55993b7d6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.144+0000 7f9307db2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55993b7dbb90 con 0x55993b7c9870 2024-09-19T08:45:41.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.144+0000 7f9307db2740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55993b6eaf20 con 0x55993b7ca190 2024-09-19T08:45:41.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.144+0000 7f9307db2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55993b7dbd10 con 0x55993b7d8fc0 2024-09-19T08:45:41.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.144+0000 7f9307db2740 10 client.0 _ll_register_callbacks cb 0x55993b6b4030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:45:41.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.144+0000 7f92feffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55993b7c9870 0x55993b7d8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.144+0000 7f92feffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55993b7c9870 0x55993b7d8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43582/0 (socket says 172.21.15.164:43582) 2024-09-19T08:45:41.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.144+0000 7f92feffd640 1 -- 172.21.15.164:0/284108903 learned_addr learned my addr 172.21.15.164:0/284108903 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:41.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.144+0000 7f92fe7fc640 1 --2- 172.21.15.164:0/284108903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993b7ca190 0x55993b7d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.144+0000 7f92ff7fe640 1 --2- 172.21.15.164:0/284108903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55993b7d8fc0 0x55993b7d6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.144+0000 7f92feffd640 1 -- 172.21.15.164:0/284108903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55993b7d8fc0 msgr2=0x55993b7d6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:41.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.144+0000 7f92feffd640 1 --2- 172.21.15.164:0/284108903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55993b7d8fc0 0x55993b7d6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.144+0000 7f92feffd640 1 -- 172.21.15.164:0/284108903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993b7ca190 msgr2=0x55993b7d5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:41.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.144+0000 7f92feffd640 1 --2- 172.21.15.164:0/284108903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993b7ca190 0x55993b7d5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.144+0000 7f92feffd640 1 -- 172.21.15.164:0/284108903 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55993b4a12b0 con 0x55993b7c9870 2024-09-19T08:45:41.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.144+0000 7f92ff7fe640 1 --2- 172.21.15.164:0/284108903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55993b7d8fc0 0x55993b7d6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:41.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.145+0000 7f92feffd640 1 --2- 172.21.15.164:0/284108903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55993b7c9870 0x55993b7d8870 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f92e8000c00 tx=0x7f92e80186f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bbf5aac10b6e9e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:41.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.145+0000 7f92deffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:45:41.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.145+0000 7f92deffd640 1 -- 172.21.15.164:0/284108903 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e801d050 con 0x55993b7c9870 2024-09-19T08:45:41.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.145+0000 7f92deffd640 1 -- 172.21.15.164:0/284108903 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f92e8010ae0 con 0x55993b7c9870 2024-09-19T08:45:41.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.145+0000 7f9307db2740 1 -- 172.21.15.164:0/284108903 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92e8018b20 con 0x55993b7c9870 2024-09-19T08:45:41.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.145+0000 7f92deffd640 1 -- 172.21.15.164:0/284108903 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f92e800a830 con 0x55993b7c9870 2024-09-19T08:45:41.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.145+0000 7f9307db2740 10 client.15171 fetch_fsmap learned FSMap version 27 2024-09-19T08:45:41.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.145+0000 7f9307db2740 1 -- 172.21.15.164:0/284108903 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55993b7e7540 con 0x55993b7c9870 2024-09-19T08:45:41.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.145+0000 7f92deffd640 1 -- 172.21.15.164:0/284108903 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92e8018b20 con 0x55993b7c9870 2024-09-19T08:45:41.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.145+0000 7f92deffd640 1 -- 172.21.15.164:0/284108903 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f92e80049a0 con 0x55993b7c9870 2024-09-19T08:45:41.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.145+0000 7f9307db2740 10 client.15171 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T08:45:41.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.145+0000 7f9307db2740 10 client.15171 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T08:45:41.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.145+0000 7f9307db2740 20 client.15171 populate_metadata read hostname 'smithi164' 2024-09-19T08:45:41.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.145+0000 7f9307db2740 1 --2- 172.21.15.164:0/284108903 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55993b7e9800 0x55993b7eac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.145+0000 7f9307db2740 4 client.15171 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:45:41.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.145+0000 7f9307db2740 1 -- 172.21.15.164:0/284108903 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55993b73d600 con 0x55993b7e9800 2024-09-19T08:45:41.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.146+0000 7f92fe7fc640 1 --2- 172.21.15.164:0/284108903 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55993b7e9800 0x55993b7eac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.146+0000 7f92deffd640 10 client.15171 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:45:41.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.146+0000 7f92fe7fc640 1 --2- 172.21.15.164:0/284108903 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55993b7e9800 0x55993b7eac70 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63bcdefafab4941b server_cookie=f8835ee68f8d332d in_seq=0 out_seq=0 2024-09-19T08:45:41.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.147+0000 7f92deffd640 1 -- 172.21.15.164:0/284108903 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55993b73d600 con 0x55993b7e9800 2024-09-19T08:45:41.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.147+0000 7f92deffd640 10 client.15171 handle_command_reply: tid=0 2024-09-19T08:45:41.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.147+0000 7f9307db2740 1 client.15171 shutdown 2024-09-19T08:45:41.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.147+0000 7f9307db2740 1 -- 172.21.15.164:0/284108903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55993b7c9870 msgr2=0x55993b7d8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:41.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.147+0000 7f9307db2740 1 --2- 172.21.15.164:0/284108903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55993b7c9870 0x55993b7d8870 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f92e8000c00 tx=0x7f92e80186f0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.147+0000 7f9307db2740 1 -- 172.21.15.164:0/284108903 shutdown_connections 2024-09-19T08:45:41.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.147+0000 7f9307db2740 1 --2- 172.21.15.164:0/284108903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55993b7d8fc0 0x55993b7d6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.147+0000 7f9307db2740 1 --2- 172.21.15.164:0/284108903 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55993b7e9800 0x55993b7eac70 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.147+0000 7f92deffd640 0 client.15171 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:45:41.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.147+0000 7f9307db2740 1 --2- 172.21.15.164:0/284108903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993b7ca190 0x55993b7d5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.147+0000 7f9307db2740 1 --2- 172.21.15.164:0/284108903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55993b7c9870 0x55993b7d8870 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.147+0000 7f9307db2740 1 -- 172.21.15.164:0/284108903 >> 172.21.15.164:0/284108903 conn(0x55993b7c5f60 msgr2=0x55993b7cbcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:41.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.147+0000 7f9307db2740 1 -- 172.21.15.164:0/284108903 shutdown_connections 2024-09-19T08:45:41.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.147+0000 7f9307db2740 1 -- 172.21.15.164:0/284108903 wait complete. 2024-09-19T08:45:41.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.147+0000 7f9307db2740 20 client.15171 trim_cache size 0 max 16384 2024-09-19T08:45:41.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:40 smithi173 ceph-mon[35306]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-19T08:45:41.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.164+0000 7f9307db2740 1 Processor -- start 2024-09-19T08:45:41.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.164+0000 7f9307db2740 1 -- start start 2024-09-19T08:45:41.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.165+0000 7f9307db2740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993b7ca190 0x55993b7c0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.165+0000 7f9307db2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55993b7d8fc0 0x55993b7d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.165+0000 7f9307db2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55993b7e9800 0x55993b7d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.165+0000 7f9307db2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55993b7e8fb0 con 0x55993b7e9800 2024-09-19T08:45:41.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.165+0000 7f9307db2740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55993b7e9680 con 0x55993b7ca190 2024-09-19T08:45:41.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.165+0000 7f9307db2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55993b7e9d50 con 0x55993b7d8fc0 2024-09-19T08:45:41.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.165+0000 7f92ff7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55993b7e9800 0x55993b7d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.165+0000 7f92ff7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55993b7e9800 0x55993b7d33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43586/0 (socket says 172.21.15.164:43586) 2024-09-19T08:45:41.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.165+0000 7f92ff7fe640 1 -- 172.21.15.164:0/3460278923 learned_addr learned my addr 172.21.15.164:0/3460278923 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:41.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.165+0000 7f92ff7fe640 1 -- 172.21.15.164:0/3460278923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55993b7d8fc0 msgr2=0x55993b7d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:41.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.165+0000 7f92ff7fe640 1 --2- 172.21.15.164:0/3460278923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55993b7d8fc0 0x55993b7d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.165+0000 7f92ff7fe640 1 -- 172.21.15.164:0/3460278923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993b7ca190 msgr2=0x55993b7c0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:41.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.165+0000 7f92ff7fe640 1 --2- 172.21.15.164:0/3460278923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993b7ca190 0x55993b7c0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.165+0000 7f92ff7fe640 1 -- 172.21.15.164:0/3460278923 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55993b7c57f0 con 0x55993b7e9800 2024-09-19T08:45:41.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.165+0000 7f92ff7fe640 1 --2- 172.21.15.164:0/3460278923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55993b7e9800 0x55993b7d33d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f92f80bcc50 tx=0x7f92f80bc970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31e80939a9cbcd9c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:41.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.165+0000 7f92f57fa640 1 -- 172.21.15.164:0/3460278923 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f802c040 con 0x55993b7e9800 2024-09-19T08:45:41.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.165+0000 7f92f57fa640 1 -- 172.21.15.164:0/3460278923 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92f80bcf50 con 0x55993b7e9800 2024-09-19T08:45:41.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.165+0000 7f92f57fa640 1 -- 172.21.15.164:0/3460278923 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f80b95c0 con 0x55993b7e9800 2024-09-19T08:45:41.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.166+0000 7f9307db2740 1 -- 172.21.15.164:0/3460278923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55993b7e9800 msgr2=0x55993b7d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:41.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.166+0000 7f9307db2740 1 --2- 172.21.15.164:0/3460278923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55993b7e9800 0x55993b7d33d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f92f80bcc50 tx=0x7f92f80bc970 comp rx=0 tx=0).stop 2024-09-19T08:45:41.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.166+0000 7f9307db2740 1 -- 172.21.15.164:0/3460278923 shutdown_connections 2024-09-19T08:45:41.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.166+0000 7f9307db2740 1 --2- 172.21.15.164:0/3460278923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55993b7e9800 0x55993b7d33d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.166+0000 7f9307db2740 1 --2- 172.21.15.164:0/3460278923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55993b7d8fc0 0x55993b7d2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.166+0000 7f9307db2740 1 --2- 172.21.15.164:0/3460278923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993b7ca190 0x55993b7c0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.166+0000 7f9307db2740 1 -- 172.21.15.164:0/3460278923 >> 172.21.15.164:0/3460278923 conn(0x55993b7c5f60 msgr2=0x55993b7d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:41.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.166+0000 7f9307db2740 1 -- 172.21.15.164:0/3460278923 shutdown_connections 2024-09-19T08:45:41.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.166+0000 7f9307db2740 1 -- 172.21.15.164:0/3460278923 wait complete. 2024-09-19T08:45:41.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.166+0000 7f9307db2740 1 -- start start 2024-09-19T08:45:41.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.167+0000 7f9307db2740 1 Processor -- start 2024-09-19T08:45:41.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.167+0000 7f9307db2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55993b7ca190 0x55993b7edb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.167+0000 7f9307db2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55993b7d8fc0 0x55993b7ee080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.167+0000 7f9307db2740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993b7e9800 0x55993b7ee5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.167+0000 7f9307db2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55993b7eaaf0 con 0x55993b7d8fc0 2024-09-19T08:45:41.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.167+0000 7f9307db2740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55993b7ea420 con 0x55993b7e9800 2024-09-19T08:45:41.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.167+0000 7f9307db2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55993b7eb1c0 con 0x55993b7ca190 2024-09-19T08:45:41.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.167+0000 7f9307db2740 10 client.0 _ll_register_callbacks cb 0x7f92e801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:45:41.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.167+0000 7f92fe7fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55993b7d8fc0 0x55993b7ee080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.167+0000 7f92fe7fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55993b7d8fc0 0x55993b7ee080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43600/0 (socket says 172.21.15.164:43600) 2024-09-19T08:45:41.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.167+0000 7f92fe7fc640 1 -- 172.21.15.164:0/352225420 learned_addr learned my addr 172.21.15.164:0/352225420 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:41.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.167+0000 7f92feffd640 1 --2- 172.21.15.164:0/352225420 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55993b7ca190 0x55993b7edb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.167+0000 7f92ff7fe640 1 --2- 172.21.15.164:0/352225420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993b7e9800 0x55993b7ee5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.167+0000 7f92fe7fc640 1 -- 172.21.15.164:0/352225420 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55993b7ca190 msgr2=0x55993b7edb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:41.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.167+0000 7f92fe7fc640 1 --2- 172.21.15.164:0/352225420 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55993b7ca190 0x55993b7edb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.167+0000 7f92fe7fc640 1 -- 172.21.15.164:0/352225420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993b7e9800 msgr2=0x55993b7ee5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:41.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.167+0000 7f92fe7fc640 1 --2- 172.21.15.164:0/352225420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993b7e9800 0x55993b7ee5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.167+0000 7f92fe7fc640 1 -- 172.21.15.164:0/352225420 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55993b7f2dd0 con 0x55993b7d8fc0 2024-09-19T08:45:41.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.167+0000 7f92fe7fc640 1 --2- 172.21.15.164:0/352225420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55993b7d8fc0 0x55993b7ee080 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f92f0009480 tx=0x7f92f00047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816a6ce35e7df2e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:41.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.167+0000 7f92deffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:45:41.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.168+0000 7f92deffd640 1 -- 172.21.15.164:0/352225420 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f00227e0 con 0x55993b7d8fc0 2024-09-19T08:45:41.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.168+0000 7f92deffd640 1 -- 172.21.15.164:0/352225420 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f92f000f040 con 0x55993b7d8fc0 2024-09-19T08:45:41.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.168+0000 7f9307db2740 1 -- 172.21.15.164:0/352225420 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92f80bcf50 con 0x55993b7d8fc0 2024-09-19T08:45:41.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.168+0000 7f92deffd640 1 -- 172.21.15.164:0/352225420 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f92f0022980 con 0x55993b7d8fc0 2024-09-19T08:45:41.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.168+0000 7f9307db2740 10 client.15177 fetch_fsmap learned FSMap version 27 2024-09-19T08:45:41.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.168+0000 7f9307db2740 1 -- 172.21.15.164:0/352225420 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55993b854420 con 0x55993b7d8fc0 2024-09-19T08:45:41.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.168+0000 7f92deffd640 1 -- 172.21.15.164:0/352225420 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92f80bcf50 con 0x55993b7d8fc0 2024-09-19T08:45:41.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.168+0000 7f92deffd640 1 -- 172.21.15.164:0/352225420 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f92e8019ce0 con 0x55993b7d8fc0 2024-09-19T08:45:41.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.168+0000 7f9307db2740 10 client.15177 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T08:45:41.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.168+0000 7f9307db2740 10 client.15177 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T08:45:41.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.168+0000 7f9307db2740 20 client.15177 populate_metadata read hostname 'smithi164' 2024-09-19T08:45:41.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.168+0000 7f9307db2740 1 --2- 172.21.15.164:0/352225420 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55993b858f90 0x55993b857ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.168+0000 7f9307db2740 4 client.15177 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:45:41.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.168+0000 7f9307db2740 1 -- 172.21.15.164:0/352225420 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "status"}) -- 0x55993b737b10 con 0x55993b858f90 2024-09-19T08:45:41.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.169+0000 7f92feffd640 1 --2- 172.21.15.164:0/352225420 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55993b858f90 0x55993b857ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.169+0000 7f92deffd640 10 client.15177 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:45:41.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.169+0000 7f92feffd640 1 --2- 172.21.15.164:0/352225420 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55993b858f90 0x55993b857ca0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8d963462ca79746 server_cookie=c19d289fe33ab3b1 in_seq=0 out_seq=0 2024-09-19T08:45:41.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.169+0000 7f92deffd640 1 -- 172.21.15.164:0/352225420 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55993b737b10 con 0x55993b858f90 2024-09-19T08:45:41.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.169+0000 7f92deffd640 10 client.15177 handle_command_reply: tid=0 2024-09-19T08:45:41.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.169+0000 7f9307db2740 1 client.15177 shutdown 2024-09-19T08:45:41.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.169+0000 7f9307db2740 1 -- 172.21.15.164:0/352225420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55993b7d8fc0 msgr2=0x55993b7ee080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:41.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.169+0000 7f9307db2740 1 --2- 172.21.15.164:0/352225420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55993b7d8fc0 0x55993b7ee080 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f92f0009480 tx=0x7f92f00047d0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.169+0000 7f9307db2740 1 -- 172.21.15.164:0/352225420 shutdown_connections 2024-09-19T08:45:41.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.169+0000 7f9307db2740 1 --2- 172.21.15.164:0/352225420 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55993b858f90 0x55993b857ca0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.170+0000 7f9307db2740 1 --2- 172.21.15.164:0/352225420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993b7e9800 0x55993b7ee5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.170+0000 7f9307db2740 1 --2- 172.21.15.164:0/352225420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55993b7d8fc0 0x55993b7ee080 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.170+0000 7f9307db2740 1 --2- 172.21.15.164:0/352225420 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55993b7ca190 0x55993b7edb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.170+0000 7f9307db2740 1 -- 172.21.15.164:0/352225420 >> 172.21.15.164:0/352225420 conn(0x55993b7c5f60 msgr2=0x55993b7d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:41.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.170+0000 7f92deffd640 0 client.15177 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:45:41.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.170+0000 7f9307db2740 1 -- 172.21.15.164:0/352225420 shutdown_connections 2024-09-19T08:45:41.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.170+0000 7f9307db2740 1 -- 172.21.15.164:0/352225420 wait complete. 2024-09-19T08:45:41.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.170+0000 7f9307db2740 20 client.15177 trim_cache size 0 max 16384 2024-09-19T08:45:41.177 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:45:41.177 INFO:teuthology.orchestra.run.smithi164.stdout: "cluster_fsid": "1e4b0d74-7662-11ef-bceb-c7b262605968", 2024-09-19T08:45:41.177 INFO:teuthology.orchestra.run.smithi164.stdout: "whoami": 0, 2024-09-19T08:45:41.177 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 24479, 2024-09-19T08:45:41.177 INFO:teuthology.orchestra.run.smithi164.stdout: "want_state": "up:active", 2024-09-19T08:45:41.177 INFO:teuthology.orchestra.run.smithi164.stdout: "state": "up:active", 2024-09-19T08:45:41.177 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs", 2024-09-19T08:45:41.177 INFO:teuthology.orchestra.run.smithi164.stdout: "rank_uptime": 107.40827157299999, 2024-09-19T08:45:41.178 INFO:teuthology.orchestra.run.smithi164.stdout: "mdsmap_epoch": 27, 2024-09-19T08:45:41.178 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch": 82, 2024-09-19T08:45:41.178 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch_barrier": 82, 2024-09-19T08:45:41.178 INFO:teuthology.orchestra.run.smithi164.stdout: "uptime": 119.12253668 2024-09-19T08:45:41.178 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:45:41.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.170+0000 7f93059d9640 1 -- 172.21.15.164:0/2964569371 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f92ec0a3240 msgr2=0x7f92ec0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:41.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.170+0000 7f93059d9640 1 --2- 172.21.15.164:0/2964569371 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f92ec0a3240 0x7f92ec0a5700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f93000feda0 tx=0x7f92f0009de0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.170+0000 7f93059d9640 1 -- 172.21.15.164:0/2964569371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f930010c170 msgr2=0x7f930019cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:41.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.170+0000 7f93059d9640 1 --2- 172.21.15.164:0/2964569371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f930010c170 0x7f930019cd40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f92f8006010 tx=0x7f92f802f710 comp rx=0 tx=0).stop 2024-09-19T08:45:41.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.170+0000 7f93059d9640 1 -- 172.21.15.164:0/2964569371 shutdown_connections 2024-09-19T08:45:41.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.170+0000 7f93059d9640 1 --2- 172.21.15.164:0/2964569371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f930010c170 0x7f930019cd40 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.170+0000 7f93059d9640 1 --2- 172.21.15.164:0/2964569371 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f92ec0a3240 0x7f92ec0a5700 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.170+0000 7f93059d9640 1 --2- 172.21.15.164:0/2964569371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9300103660 0x7f93001989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.170+0000 7f93059d9640 1 --2- 172.21.15.164:0/2964569371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9300102d40 0x7f9300198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.170+0000 7f93059d9640 1 -- 172.21.15.164:0/2964569371 >> 172.21.15.164:0/2964569371 conn(0x7f93000fc7a0 msgr2=0x7f93000fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:41.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.170+0000 7f93059d9640 1 -- 172.21.15.164:0/2964569371 shutdown_connections 2024-09-19T08:45:41.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:41.171+0000 7f93059d9640 1 -- 172.21.15.164:0/2964569371 wait complete. 2024-09-19T08:45:42.083 INFO:teuthology.run_tasks:Running task sequential... 2024-09-19T08:45:42.093 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-19T08:45:42.093 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-19T08:45:42.093 INFO:teuthology.task.print:Enabling mgr modules 2024-09-19T08:45:42.094 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-19T08:45:42.094 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T08:45:42.095 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi164.front.sepia.ceph.com 2024-09-19T08:45:42.095 DEBUG:teuthology.orchestra.run.smithi164:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-19T08:45:42.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.197+0000 7efd98df1640 1 -- 172.21.15.164:0/1832461266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd941013c0 msgr2=0x7efd94101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:42.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.197+0000 7efd98df1640 1 --2- 172.21.15.164:0/1832461266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd941013c0 0x7efd94101840 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00aa10 tx=0x7efd7c02b720 comp rx=0 tx=0).stop 2024-09-19T08:45:42.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.197+0000 7efd98df1640 1 -- 172.21.15.164:0/1832461266 shutdown_connections 2024-09-19T08:45:42.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.197+0000 7efd98df1640 1 --2- 172.21.15.164:0/1832461266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd94101d80 0x7efd94108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:42.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.197+0000 7efd98df1640 1 --2- 172.21.15.164:0/1832461266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd941013c0 0x7efd94101840 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:42.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.197+0000 7efd98df1640 1 --2- 172.21.15.164:0/1832461266 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd941001d0 0x7efd941005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:42.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.197+0000 7efd98df1640 1 -- 172.21.15.164:0/1832461266 >> 172.21.15.164:0/1832461266 conn(0x7efd940fb970 msgr2=0x7efd940fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:42.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.197+0000 7efd98df1640 1 -- 172.21.15.164:0/1832461266 shutdown_connections 2024-09-19T08:45:42.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.197+0000 7efd98df1640 1 -- 172.21.15.164:0/1832461266 wait complete. 2024-09-19T08:45:42.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.198+0000 7efd98df1640 1 Processor -- start 2024-09-19T08:45:42.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.198+0000 7efd98df1640 1 -- start start 2024-09-19T08:45:42.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.198+0000 7efd98df1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd941001d0 0x7efd94197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:42.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.198+0000 7efd98df1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd941013c0 0x7efd94198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:42.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.198+0000 7efd98df1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd94101d80 0x7efd9419f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:42.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.198+0000 7efd98df1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efd9410b640 con 0x7efd941013c0 2024-09-19T08:45:42.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.198+0000 7efd98df1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7efd9410b4c0 con 0x7efd941001d0 2024-09-19T08:45:42.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.198+0000 7efd98df1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efd9410b7c0 con 0x7efd94101d80 2024-09-19T08:45:42.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.198+0000 7efd91d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd941013c0 0x7efd94198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:42.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.198+0000 7efd91d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd941013c0 0x7efd94198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43622/0 (socket says 172.21.15.164:43622) 2024-09-19T08:45:42.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.198+0000 7efd91d9b640 1 -- 172.21.15.164:0/83821704 learned_addr learned my addr 172.21.15.164:0/83821704 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:42.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.199+0000 7efd9259c640 1 --2- 172.21.15.164:0/83821704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd941001d0 0x7efd94197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:42.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.199+0000 7efd92d9d640 1 --2- 172.21.15.164:0/83821704 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd94101d80 0x7efd9419f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:42.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.199+0000 7efd91d9b640 1 -- 172.21.15.164:0/83821704 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd94101d80 msgr2=0x7efd9419f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:42.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.199+0000 7efd91d9b640 1 --2- 172.21.15.164:0/83821704 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd94101d80 0x7efd9419f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:42.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.199+0000 7efd91d9b640 1 -- 172.21.15.164:0/83821704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd941001d0 msgr2=0x7efd94197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:42.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.199+0000 7efd91d9b640 1 --2- 172.21.15.164:0/83821704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd941001d0 0x7efd94197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:42.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.199+0000 7efd91d9b640 1 -- 172.21.15.164:0/83821704 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd9419fa30 con 0x7efd941013c0 2024-09-19T08:45:42.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.199+0000 7efd92d9d640 1 --2- 172.21.15.164:0/83821704 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd94101d80 0x7efd9419f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:42.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.199+0000 7efd9259c640 1 --2- 172.21.15.164:0/83821704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd941001d0 0x7efd94197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:42.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.199+0000 7efd91d9b640 1 --2- 172.21.15.164:0/83821704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd941013c0 0x7efd94198350 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00a340 tx=0x7efd7c030e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:42.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.199+0000 7efd8b7fe640 1 -- 172.21.15.164:0/83821704 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c03f070 con 0x7efd941013c0 2024-09-19T08:45:42.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.199+0000 7efd8b7fe640 1 -- 172.21.15.164:0/83821704 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd7c032400 con 0x7efd941013c0 2024-09-19T08:45:42.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.199+0000 7efd98df1640 1 -- 172.21.15.164:0/83821704 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd9419fcc0 con 0x7efd941013c0 2024-09-19T08:45:42.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.199+0000 7efd8b7fe640 1 -- 172.21.15.164:0/83821704 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c036040 con 0x7efd941013c0 2024-09-19T08:45:42.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.199+0000 7efd98df1640 1 -- 172.21.15.164:0/83821704 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd941a0230 con 0x7efd941013c0 2024-09-19T08:45:42.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.200+0000 7efd98df1640 1 -- 172.21.15.164:0/83821704 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd94107570 con 0x7efd941013c0 2024-09-19T08:45:42.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.201+0000 7efd8b7fe640 1 -- 172.21.15.164:0/83821704 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7efd7c042070 con 0x7efd941013c0 2024-09-19T08:45:42.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.202+0000 7efd8b7fe640 1 --2- 172.21.15.164:0/83821704 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7efd800a3370 0x7efd800a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:42.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.202+0000 7efd9259c640 1 --2- 172.21.15.164:0/83821704 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7efd800a3370 0x7efd800a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:42.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.202+0000 7efd8b7fe640 1 -- 172.21.15.164:0/83821704 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7efd7c0f00c0 con 0x7efd941013c0 2024-09-19T08:45:42.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.202+0000 7efd9259c640 1 --2- 172.21.15.164:0/83821704 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7efd800a3370 0x7efd800a5830 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efd94101220 tx=0x7efd8c009570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:42.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.206+0000 7efd8b7fe640 1 -- 172.21.15.164:0/83821704 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efd7c0bcf10 con 0x7efd941013c0 2024-09-19T08:45:42.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.417+0000 7efd98df1640 1 -- 172.21.15.164:0/83821704 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7efd94062df0 con 0x7efd941013c0 2024-09-19T08:45:42.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.831+0000 7efd8b7fe640 1 -- 172.21.15.164:0/83821704 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7efd7c0bc810 con 0x7efd941013c0 2024-09-19T08:45:42.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.834+0000 7efd98df1640 1 -- 172.21.15.164:0/83821704 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7efd800a3370 msgr2=0x7efd800a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:42.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.834+0000 7efd98df1640 1 --2- 172.21.15.164:0/83821704 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7efd800a3370 0x7efd800a5830 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efd94101220 tx=0x7efd8c009570 comp rx=0 tx=0).stop 2024-09-19T08:45:42.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.834+0000 7efd98df1640 1 -- 172.21.15.164:0/83821704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd941013c0 msgr2=0x7efd94198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:42.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.834+0000 7efd98df1640 1 --2- 172.21.15.164:0/83821704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd941013c0 0x7efd94198350 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00a340 tx=0x7efd7c030e70 comp rx=0 tx=0).stop 2024-09-19T08:45:42.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.834+0000 7efd98df1640 1 -- 172.21.15.164:0/83821704 shutdown_connections 2024-09-19T08:45:42.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.834+0000 7efd98df1640 1 --2- 172.21.15.164:0/83821704 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7efd800a3370 0x7efd800a5830 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:42.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.834+0000 7efd98df1640 1 --2- 172.21.15.164:0/83821704 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd94101d80 0x7efd9419f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:42.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.834+0000 7efd98df1640 1 --2- 172.21.15.164:0/83821704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd941013c0 0x7efd94198350 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:42.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.834+0000 7efd98df1640 1 --2- 172.21.15.164:0/83821704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd941001d0 0x7efd94197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:42.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.834+0000 7efd98df1640 1 -- 172.21.15.164:0/83821704 >> 172.21.15.164:0/83821704 conn(0x7efd940fb970 msgr2=0x7efd94104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:42.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.836+0000 7efd98df1640 1 -- 172.21.15.164:0/83821704 shutdown_connections 2024-09-19T08:45:42.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:42.836+0000 7efd98df1640 1 -- 172.21.15.164:0/83821704 wait complete. 2024-09-19T08:45:42.856 INFO:teuthology.run_tasks:Running task kclient... 2024-09-19T08:45:42.883 INFO:tasks.kclient:Mounting kernel clients... 2024-09-19T08:45:42.883 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-19T08:45:42.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:42.883 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-09-19T08:45:42.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:42.900 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link delete ceph-brx 2024-09-19T08:45:42.969 INFO:teuthology.orchestra.run.smithi164.stderr:Cannot find device "ceph-brx" 2024-09-19T08:45:42.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:45:42.970 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-19T08:45:42.970 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T08:45:42.970 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T08:45:42.971 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-19T08:45:43.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:42 smithi170 ceph-mon[35072]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 597 B/s wr, 13 op/s 2024-09-19T08:45:43.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:42 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/83821704' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T08:45:43.125 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:45:42 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: ignoring --setuser ceph since I am not root 2024-09-19T08:45:43.125 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:45:42 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: ignoring --setgroup ceph since I am not root 2024-09-19T08:45:43.125 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:45:42 smithi170 ceph-mgr[36442]: -- 172.21.15.170:0/3705213849 <== mon.0 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562eeae2d0e0 con 0x562eeae0b000 2024-09-19T08:45:43.125 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:45:42 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:45:42.989+0000 7f1dbdede040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T08:45:43.126 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:45:43 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:45:43.046+0000 7f1dbdede040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T08:45:43.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:42 smithi173 ceph-mon[35306]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 597 B/s wr, 13 op/s 2024-09-19T08:45:43.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:42 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/83821704' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T08:45:43.161 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:45:42 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: ignoring --setuser ceph since I am not root 2024-09-19T08:45:43.161 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:45:42 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: ignoring --setgroup ceph since I am not root 2024-09-19T08:45:43.161 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:45:42 smithi173 ceph-mgr[37113]: -- 172.21.15.173:0/2641628495 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fff85990e0 con 0x55fff8577400 2024-09-19T08:45:43.161 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:45:42 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:45:42.989+0000 7f2842019040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T08:45:43.161 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:45:43 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:45:43.046+0000 7f2842019040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T08:45:43.211 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:42 smithi164 ceph-mon[30928]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 597 B/s wr, 13 op/s 2024-09-19T08:45:43.211 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:42 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/83821704' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T08:45:43.212 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:42 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: ignoring --setuser ceph since I am not root 2024-09-19T08:45:43.212 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:42 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: ignoring --setgroup ceph since I am not root 2024-09-19T08:45:43.212 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:42 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:42.993+0000 7fb0a0f40040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T08:45:43.212 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:43 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:43.054+0000 7fb0a0f40040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T08:45:43.212 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:43 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:43.210+0000 7fb0a0f40040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T08:45:43.264 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:45:43.531 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:43 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:43.286+0000 7fb0a0f40040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T08:45:43.532 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:43 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:43.438+0000 7fb0a0f40040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:45:43.545 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:45:43 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:45:43.195+0000 7f2842019040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T08:45:43.545 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:45:43 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:45:43.260+0000 7f2842019040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T08:45:43.545 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:45:43 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:45:43.406+0000 7f2842019040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:45:43.545 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:45:43 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:45:43.545+0000 7f2842019040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T08:45:43.546 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:45:43 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:45:43.193+0000 7f1dbdede040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T08:45:43.546 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:45:43 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:45:43.259+0000 7f1dbdede040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T08:45:43.546 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:45:43 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:45:43.405+0000 7f1dbdede040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:45:43.546 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:45:43 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:45:43.545+0000 7f1dbdede040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T08:45:43.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.763+0000 7f8b4e1fe640 1 -- 172.21.15.164:0/2976018050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b48100e20 msgr2=0x7f8b48101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:43.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.763+0000 7f8b4e1fe640 1 --2- 172.21.15.164:0/2976018050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b48100e20 0x7f8b48101200 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009900 tx=0x7f8b3802a480 comp rx=0 tx=0).stop 2024-09-19T08:45:43.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.763+0000 7f8b4e1fe640 1 -- 172.21.15.164:0/2976018050 shutdown_connections 2024-09-19T08:45:43.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.763+0000 7f8b4e1fe640 1 --2- 172.21.15.164:0/2976018050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b48102ef0 0x7f8b4810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:43.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.763+0000 7f8b4e1fe640 1 --2- 172.21.15.164:0/2976018050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b48102570 0x7f8b481029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:43.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.763+0000 7f8b4e1fe640 1 --2- 172.21.15.164:0/2976018050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b48100e20 0x7f8b48101200 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:43.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.763+0000 7f8b4e1fe640 1 -- 172.21.15.164:0/2976018050 >> 172.21.15.164:0/2976018050 conn(0x7f8b480fc7d0 msgr2=0x7f8b480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:43.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.763+0000 7f8b4e1fe640 1 -- 172.21.15.164:0/2976018050 shutdown_connections 2024-09-19T08:45:43.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.763+0000 7f8b4e1fe640 1 -- 172.21.15.164:0/2976018050 wait complete. 2024-09-19T08:45:43.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.764+0000 7f8b4e1fe640 1 Processor -- start 2024-09-19T08:45:43.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.764+0000 7f8b4e1fe640 1 -- start start 2024-09-19T08:45:43.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.764+0000 7f8b4e1fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b48100e20 0x7f8b48199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:43.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.764+0000 7f8b4e1fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b48102570 0x7f8b48199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:43.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.764+0000 7f8b4e1fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b48102ef0 0x7f8b481a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:43.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.764+0000 7f8b4e1fe640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8b48110f80 con 0x7f8b48100e20 2024-09-19T08:45:43.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.764+0000 7f8b4e1fe640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8b48110e00 con 0x7f8b48102ef0 2024-09-19T08:45:43.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.764+0000 7f8b4e1fe640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8b48111100 con 0x7f8b48102570 2024-09-19T08:45:43.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.764+0000 7f8b477fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b48100e20 0x7f8b48199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:43.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.764+0000 7f8b477fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b48100e20 0x7f8b48199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46294/0 (socket says 172.21.15.164:46294) 2024-09-19T08:45:43.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.764+0000 7f8b477fe640 1 -- 172.21.15.164:0/2419808809 learned_addr learned my addr 172.21.15.164:0/2419808809 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:43.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.764+0000 7f8b3ffff640 1 --2- 172.21.15.164:0/2419808809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b48102570 0x7f8b48199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:43.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.764+0000 7f8b47fff640 1 --2- 172.21.15.164:0/2419808809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b48102ef0 0x7f8b481a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:43.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.765+0000 7f8b477fe640 1 -- 172.21.15.164:0/2419808809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b48102570 msgr2=0x7f8b48199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:43.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.765+0000 7f8b477fe640 1 --2- 172.21.15.164:0/2419808809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b48102570 0x7f8b48199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:43.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.765+0000 7f8b477fe640 1 -- 172.21.15.164:0/2419808809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b48102ef0 msgr2=0x7f8b481a43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:43.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.765+0000 7f8b477fe640 1 --2- 172.21.15.164:0/2419808809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b48102ef0 0x7f8b481a43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:43.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.765+0000 7f8b477fe640 1 -- 172.21.15.164:0/2419808809 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b481a4910 con 0x7f8b48100e20 2024-09-19T08:45:43.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.765+0000 7f8b477fe640 1 --2- 172.21.15.164:0/2419808809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b48100e20 0x7f8b48199350 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8b38006010 tx=0x7f8b38003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:43.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.765+0000 7f8b457fa640 1 -- 172.21.15.164:0/2419808809 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3803f070 con 0x7f8b48100e20 2024-09-19T08:45:43.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.765+0000 7f8b4e1fe640 1 -- 172.21.15.164:0/2419808809 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b481a4ae0 con 0x7f8b48100e20 2024-09-19T08:45:43.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.765+0000 7f8b457fa640 1 -- 172.21.15.164:0/2419808809 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b38031c30 con 0x7f8b48100e20 2024-09-19T08:45:43.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.765+0000 7f8b457fa640 1 -- 172.21.15.164:0/2419808809 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b38035030 con 0x7f8b48100e20 2024-09-19T08:45:43.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.765+0000 7f8b4e1fe640 1 -- 172.21.15.164:0/2419808809 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b481a4f40 con 0x7f8b48100e20 2024-09-19T08:45:43.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.766+0000 7f8b4e1fe640 1 -- 172.21.15.164:0/2419808809 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b481021b0 con 0x7f8b48100e20 2024-09-19T08:45:43.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.767+0000 7f8b457fa640 1 -- 172.21.15.164:0/2419808809 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 19) ==== 138373+0+0 (secure 0 0 0) 0x7f8b38042070 con 0x7f8b48100e20 2024-09-19T08:45:43.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.768+0000 7f8b457fa640 1 --2- 172.21.15.164:0/2419808809 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f8b340a3290 0x7f8b340a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:43.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.768+0000 7f8b3ffff640 1 -- 172.21.15.164:0/2419808809 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f8b340a3290 msgr2=0x7f8b340a5750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.164:6800/1094864383 2024-09-19T08:45:43.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.768+0000 7f8b457fa640 1 -- 172.21.15.164:0/2419808809 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f8b380effc0 con 0x7f8b48100e20 2024-09-19T08:45:43.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.768+0000 7f8b3ffff640 1 --2- 172.21.15.164:0/2419808809 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f8b340a3290 0x7f8b340a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T08:45:43.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.772+0000 7f8b457fa640 1 -- 172.21.15.164:0/2419808809 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8b380bc690 con 0x7f8b48100e20 2024-09-19T08:45:43.833 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:43 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:43.585+0000 7fb0a0f40040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T08:45:43.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.969+0000 7f8b3ffff640 1 -- 172.21.15.164:0/2419808809 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f8b340a3290 msgr2=0x7f8b340a5750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.164:6800/1094864383 2024-09-19T08:45:43.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.969+0000 7f8b3ffff640 1 --2- 172.21.15.164:0/2419808809 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f8b340a3290 0x7f8b340a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T08:45:43.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.984+0000 7f8b4e1fe640 1 -- 172.21.15.164:0/2419808809 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8b48109090 con 0x7f8b48100e20 2024-09-19T08:45:43.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.984+0000 7f8b457fa640 1 -- 172.21.15.164:0/2419808809 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f8b380c0990 con 0x7f8b48100e20 2024-09-19T08:45:43.985 INFO:teuthology.orchestra.run.smithi164.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T08:45:43.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.986+0000 7f8b4e1fe640 1 -- 172.21.15.164:0/2419808809 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f8b340a3290 msgr2=0x7f8b340a5750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:45:43.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.986+0000 7f8b4e1fe640 1 --2- 172.21.15.164:0/2419808809 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f8b340a3290 0x7f8b340a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:43.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.986+0000 7f8b4e1fe640 1 -- 172.21.15.164:0/2419808809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b48100e20 msgr2=0x7f8b48199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:43.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.986+0000 7f8b4e1fe640 1 --2- 172.21.15.164:0/2419808809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b48100e20 0x7f8b48199350 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8b38006010 tx=0x7f8b38003c10 comp rx=0 tx=0).stop 2024-09-19T08:45:43.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.986+0000 7f8b4e1fe640 1 -- 172.21.15.164:0/2419808809 shutdown_connections 2024-09-19T08:45:43.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.986+0000 7f8b4e1fe640 1 --2- 172.21.15.164:0/2419808809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b48102ef0 0x7f8b481a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:43.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.986+0000 7f8b4e1fe640 1 --2- 172.21.15.164:0/2419808809 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f8b340a3290 0x7f8b340a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:43.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.986+0000 7f8b4e1fe640 1 --2- 172.21.15.164:0/2419808809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b48102570 0x7f8b48199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:43.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.986+0000 7f8b4e1fe640 1 --2- 172.21.15.164:0/2419808809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b48100e20 0x7f8b48199350 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:43.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.986+0000 7f8b4e1fe640 1 -- 172.21.15.164:0/2419808809 >> 172.21.15.164:0/2419808809 conn(0x7f8b480fc7d0 msgr2=0x7f8b480fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:43.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.987+0000 7f8b4e1fe640 1 -- 172.21.15.164:0/2419808809 shutdown_connections 2024-09-19T08:45:43.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:43.987+0000 7f8b4e1fe640 1 -- 172.21.15.164:0/2419808809 wait complete. 2024-09-19T08:45:44.099 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:45:43 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:45:43.855+0000 7f2842019040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T08:45:44.099 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:45:43 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:45:43.917+0000 7f2842019040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T08:45:44.099 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:43 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/83821704' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-19T08:45:44.100 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:43 smithi173 ceph-mon[35306]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-19T08:45:44.106 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:43 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/83821704' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-19T08:45:44.106 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:43 smithi170 ceph-mon[35072]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-19T08:45:44.106 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:45:43 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:45:43.862+0000 7f1dbdede040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T08:45:44.107 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:45:43 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:45:43.923+0000 7f1dbdede040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T08:45:44.176 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:43 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/83821704' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-19T08:45:44.176 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:43 smithi164 ceph-mon[30928]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-19T08:45:44.176 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:43 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:43.915+0000 7fb0a0f40040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T08:45:44.177 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:43 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:43.978+0000 7fb0a0f40040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T08:45:44.177 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:44 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:44.175+0000 7fb0a0f40040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T08:45:44.375 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:45:44 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:45:44.106+0000 7f1dbdede040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T08:45:44.410 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:45:44 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:45:44.099+0000 7f2842019040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T08:45:44.794 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T08:45:44.794 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T08:45:44.794 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi164.front.sepia.ceph.com 2024-09-19T08:45:44.794 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T08:45:44.794 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T08:45:44.794 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T08:45:44.794 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T08:45:44.795 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T08:45:44.795 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:45:44.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:44.795 DEBUG:teuthology.orchestra.run.smithi164:> ip addr 2024-09-19T08:45:44.812 INFO:teuthology.orchestra.run.smithi164.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T08:45:44.812 INFO:teuthology.orchestra.run.smithi164.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T08:45:44.812 INFO:teuthology.orchestra.run.smithi164.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T08:45:44.812 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:44.812 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 ::1/128 scope host 2024-09-19T08:45:44.812 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:44.812 INFO:teuthology.orchestra.run.smithi164.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T08:45:44.812 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f4 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:45:44.813 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f0 2024-09-19T08:45:44.813 INFO:teuthology.orchestra.run.smithi164.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T08:45:44.813 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f5 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:45:44.813 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f1 2024-09-19T08:45:44.813 INFO:teuthology.orchestra.run.smithi164.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T08:45:44.813 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f2 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:45:44.813 INFO:teuthology.orchestra.run.smithi164.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T08:45:44.813 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f3 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:45:44.813 INFO:teuthology.orchestra.run.smithi164.stdout: inet 172.21.15.164/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-19T08:45:44.813 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft 42225sec preferred_lft 42225sec 2024-09-19T08:45:44.814 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::ec4:7aff:fe88:73f3/64 scope link noprefixroute 2024-09-19T08:45:44.814 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:44.814 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-19T08:45:44.815 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:45:44.815 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-09-19T08:45:44.815 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link add name ceph-brx type bridge 2024-09-19T08:45:44.815 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip addr flush dev ceph-brx 2024-09-19T08:45:44.815 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set ceph-brx up 2024-09-19T08:45:44.815 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-19T08:45:44.815 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-09-19T08:45:44.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:45:45.012 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:44 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2419808809' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T08:45:45.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:45:45.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:45.030 DEBUG:teuthology.orchestra.run.smithi164:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T08:45:45.055 INFO:teuthology.orchestra.run.smithi164.stdout:1 2024-09-19T08:45:45.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:45.057 DEBUG:teuthology.orchestra.run.smithi164:> route 2024-09-19T08:45:45.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:44 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2419808809' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T08:45:45.125 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:45:44 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:45:44.991+0000 7f1dbdede040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T08:45:45.125 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:45:45 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:45:45.058+0000 7f1dbdede040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T08:45:45.153 INFO:teuthology.orchestra.run.smithi164.stdout:Kernel IP routing table 2024-09-19T08:45:45.153 INFO:teuthology.orchestra.run.smithi164.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T08:45:45.153 INFO:teuthology.orchestra.run.smithi164.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T08:45:45.153 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T08:45:45.153 INFO:teuthology.orchestra.run.smithi164.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T08:45:45.154 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:45:45.154 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-09-19T08:45:45.154 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T08:45:45.154 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T08:45:45.154 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T08:45:45.154 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-09-19T08:45:45.160 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:45:44 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:45:44.976+0000 7f2842019040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T08:45:45.160 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:45:45 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:45:45.043+0000 7f2842019040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T08:45:45.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:44 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2419808809' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T08:45:45.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:45:45.282 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:45 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:45.105+0000 7fb0a0f40040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T08:45:45.282 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:45 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:45.176+0000 7fb0a0f40040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T08:45:45.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:45:45.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:45.437 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-09-19T08:45:45.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:45.453 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list-id 2024-09-19T08:45:45.508 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:45:45.508 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-09-19T08:45:45.508 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:45:45.508 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-19T08:45:45.508 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-09-19T08:45:45.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:45:45.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:45:45.628 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-19T08:45:45.628 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:45:45.629 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-09-19T08:45:45.629 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-19T08:45:45.629 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-19T08:45:45.629 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-19T08:45:45.629 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-19T08:45:45.629 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-19T08:45:45.629 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-09-19T08:45:45.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:45:45.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:45:45.757 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:45:45.757 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-09-19T08:45:45.757 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set brx.0 up 2024-09-19T08:45:45.758 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set dev brx.0 master ceph-brx 2024-09-19T08:45:45.758 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-09-19T08:45:45.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:45:45.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:45:45.838 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:45:45.838 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T08:45:45.854 INFO:teuthology.orchestra.run.smithi164.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:45:45.854 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:45:45.855 DEBUG:teuthology.orchestra.run.smithi164:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T08:45:45.910 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-19T08:45:45.911 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-19T08:45:45.911 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/0902124e-0b89-4d29-848b-19de8f45d8f9 2024-09-19T08:45:45.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:45.911 DEBUG:teuthology.orchestra.run.smithi164:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/0902124e-0b89-4d29-848b-19de8f45d8f9 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-09-19T08:45:45.997 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:45:45 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:45:45.740+0000 7f2842019040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T08:45:45.998 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:45:45 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:45:45.855+0000 7f2842019040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T08:45:46.017 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:45:45 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:45:45.759+0000 7f1dbdede040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T08:45:46.017 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:45:45 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:45:45.876+0000 7f1dbdede040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T08:45:46.018 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-09-19T08:45:46.019 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-09-19T08:45:46.019 INFO:teuthology.orchestra.run.smithi164.stdout:invalid new device string format 2024-09-19T08:45:46.157 INFO:teuthology.orchestra.run.smithi164.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-09-19T08:45:46.157 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-09-19T08:45:46.157 INFO:teuthology.orchestra.run.smithi164.stdout:invalid new device string format 2024-09-19T08:45:46.158 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: resolved to: "172.21.15.164:3300,172.21.15.170:3300,172.21.15.173:3300" 2024-09-19T08:45:46.158 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.164:3300,172.21.15.170:3300,172.21.15.173:3300:/volumes/qa/sv_1/0902124e-0b89-4d29-848b-19de8f45d8f9 2024-09-19T08:45:46.158 INFO:teuthology.orchestra.run.smithi164.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,key=0,fsid=1e4b0d74-7662-11ef-bceb-c7b262605968" will pass to kernel 2024-09-19T08:45:46.158 INFO:teuthology.orchestra.run.smithi164.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-19T08:45:46.159 INFO:teuthology.orchestra.run.smithi164.stdout: You just mounted a file system that supports labels which does not 2024-09-19T08:45:46.159 INFO:teuthology.orchestra.run.smithi164.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-19T08:45:46.159 INFO:teuthology.orchestra.run.smithi164.stdout: applications will generate AVC messages and not be allowed access to 2024-09-19T08:45:46.159 INFO:teuthology.orchestra.run.smithi164.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-19T08:45:46.161 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-19T08:45:46.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:46.161 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T08:45:46.187 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T08:45:46.216 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:45 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:45.939+0000 7fb0a0f40040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T08:45:46.216 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:46 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:46.064+0000 7fb0a0f40040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T08:45:46.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T08:45:46.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:45:46.274 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:45:45 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:45:45.997+0000 7f2842019040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:45:46.274 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:45:46 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:45:46.059+0000 7f2842019040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T08:45:46.274 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:45:46 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:45:46.205+0000 7f2842019040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T08:45:46.275 INFO:teuthology.orchestra.run.smithi164.stdout:client15210 2024-09-19T08:45:46.275 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-19T08:45:46.275 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:45:46.292 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T08:45:46.296 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:45:46 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:45:46.017+0000 7f1dbdede040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:45:46.296 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:45:46 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:45:46.078+0000 7f1dbdede040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T08:45:46.296 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:45:46 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:45:46.228+0000 7f1dbdede040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T08:45:46.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T08:45:46.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:45:46.390 INFO:teuthology.orchestra.run.smithi164.stdout:client15210 2024-09-19T08:45:46.390 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T08:45:46.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T08:45:46.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:45:46.447 INFO:teuthology.orchestra.run.smithi164.stdout:client15210 2024-09-19T08:45:46.448 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968.client15210/status') 2024-09-19T08:45:46.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:45:46.505 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:46 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:46.215+0000 7fb0a0f40040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:45:46.505 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:46 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:46.281+0000 7fb0a0f40040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T08:45:46.505 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:46 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:46.439+0000 7fb0a0f40040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T08:45:46.507 INFO:teuthology.orchestra.run.smithi164.stdout:instance: client.15210 (3)192.168.0.1:0/2382830793 2024-09-19T08:45:46.507 INFO:teuthology.orchestra.run.smithi164.stdout:blocklisted: false 2024-09-19T08:45:46.507 INFO:teuthology.orchestra.run.smithi164.stderr:0+1 records in 2024-09-19T08:45:46.507 INFO:teuthology.orchestra.run.smithi164.stderr:0+1 records out 2024-09-19T08:45:46.507 INFO:teuthology.orchestra.run.smithi164.stderr:70 bytes copied, 2.2829e-05 s, 3.1 MB/s 2024-09-19T08:45:46.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:45:46.515 INFO:teuthology.run_tasks:Running task exec... 2024-09-19T08:45:46.524 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T08:45:46.525 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi164.front.sepia.ceph.com 2024-09-19T08:45:46.525 DEBUG:teuthology.orchestra.run.smithi164:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-19T08:45:46.551 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:45:46 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:45:46.272+0000 7f2842019040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T08:45:46.551 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:45:46 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:45:46.331+0000 7f2842019040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T08:45:46.551 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:45:46 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:45:46.486+0000 7f2842019040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T08:45:46.576 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:45:46 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:45:46.296+0000 7f1dbdede040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T08:45:46.577 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:45:46 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:45:46.355+0000 7f1dbdede040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T08:45:46.577 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:45:46 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:45:46.512+0000 7f1dbdede040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T08:45:46.577 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:45:46 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:45:46.576+0000 7f1dbdede040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T08:45:46.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.628+0000 7f70a3ad9640 1 -- 172.21.15.164:0/48401189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f709c100a40 msgr2=0x7f709c100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:46.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.628+0000 7f70a3ad9640 1 --2- 172.21.15.164:0/48401189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f709c100a40 0x7f709c100e40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7098009900 tx=0x7f709802a410 comp rx=0 tx=0).stop 2024-09-19T08:45:46.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.628+0000 7f70a3ad9640 1 -- 172.21.15.164:0/48401189 shutdown_connections 2024-09-19T08:45:46.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.628+0000 7f70a3ad9640 1 --2- 172.21.15.164:0/48401189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f709c1025f0 0x7f709c108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:46.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.628+0000 7f70a3ad9640 1 --2- 172.21.15.164:0/48401189 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f709c101c30 0x7f709c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:46.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.628+0000 7f70a3ad9640 1 --2- 172.21.15.164:0/48401189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f709c100a40 0x7f709c100e40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:46.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.628+0000 7f70a3ad9640 1 -- 172.21.15.164:0/48401189 >> 172.21.15.164:0/48401189 conn(0x7f709c0fc1e0 msgr2=0x7f709c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:46.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.628+0000 7f70a3ad9640 1 -- 172.21.15.164:0/48401189 shutdown_connections 2024-09-19T08:45:46.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.628+0000 7f70a3ad9640 1 -- 172.21.15.164:0/48401189 wait complete. 2024-09-19T08:45:46.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.629+0000 7f70a3ad9640 1 Processor -- start 2024-09-19T08:45:46.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.629+0000 7f70a3ad9640 1 -- start start 2024-09-19T08:45:46.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.629+0000 7f70a3ad9640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f709c100a40 0x7f709c06a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:46.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.629+0000 7f70a3ad9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f709c101c30 0x7f709c19cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:46.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.629+0000 7f70a3ad9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f709c1025f0 0x7f709c19f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:46.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.629+0000 7f70a3ad9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f709c10c6c0 con 0x7f709c101c30 2024-09-19T08:45:46.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.629+0000 7f70a3ad9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f709c10c540 con 0x7f709c1025f0 2024-09-19T08:45:46.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.629+0000 7f70a3ad9640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f709c10c840 con 0x7f709c100a40 2024-09-19T08:45:46.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.629+0000 7f70a1074640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f709c101c30 0x7f709c19cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:46.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.629+0000 7f70a1875640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f709c100a40 0x7f709c06a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:46.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.629+0000 7f70a1074640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f709c101c30 0x7f709c19cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46314/0 (socket says 172.21.15.164:46314) 2024-09-19T08:45:46.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.629+0000 7f70a1074640 1 -- 172.21.15.164:0/3071960738 learned_addr learned my addr 172.21.15.164:0/3071960738 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:46.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.629+0000 7f70a2076640 1 --2- 172.21.15.164:0/3071960738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f709c1025f0 0x7f709c19f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:46.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.629+0000 7f70a1875640 1 -- 172.21.15.164:0/3071960738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f709c1025f0 msgr2=0x7f709c19f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:46.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.629+0000 7f70a1875640 1 --2- 172.21.15.164:0/3071960738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f709c1025f0 0x7f709c19f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:46.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.629+0000 7f70a1875640 1 -- 172.21.15.164:0/3071960738 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f709c101c30 msgr2=0x7f709c19cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:46.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.629+0000 7f70a1875640 1 --2- 172.21.15.164:0/3071960738 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f709c101c30 0x7f709c19cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:46.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.629+0000 7f70a1875640 1 -- 172.21.15.164:0/3071960738 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f709c19fa60 con 0x7f709c100a40 2024-09-19T08:45:46.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.629+0000 7f70a1875640 1 --2- 172.21.15.164:0/3071960738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f709c100a40 0x7f709c06a410 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7098009230 tx=0x7f7098004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:46.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.630+0000 7f7092ffd640 1 -- 172.21.15.164:0/3071960738 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709803f070 con 0x7f709c100a40 2024-09-19T08:45:46.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.630+0000 7f70a3ad9640 1 -- 172.21.15.164:0/3071960738 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f709c19fc90 con 0x7f709c100a40 2024-09-19T08:45:46.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.630+0000 7f7092ffd640 1 -- 172.21.15.164:0/3071960738 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7098003cd0 con 0x7f709c100a40 2024-09-19T08:45:46.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.630+0000 7f7092ffd640 1 -- 172.21.15.164:0/3071960738 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7098035070 con 0x7f709c100a40 2024-09-19T08:45:46.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.630+0000 7f70a3ad9640 1 -- 172.21.15.164:0/3071960738 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f709c1a01a0 con 0x7f709c100a40 2024-09-19T08:45:46.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.631+0000 7f70a3ad9640 1 -- 172.21.15.164:0/3071960738 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f709c100e40 con 0x7f709c100a40 2024-09-19T08:45:46.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.632+0000 7f7092ffd640 1 -- 172.21.15.164:0/3071960738 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 19) ==== 138373+0+0 (secure 0 0 0) 0x7f709803b460 con 0x7f709c100a40 2024-09-19T08:45:46.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.632+0000 7f7092ffd640 1 --2- 172.21.15.164:0/3071960738 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f70880a33c0 0x7f70880a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:46.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.632+0000 7f70a1074640 1 -- 172.21.15.164:0/3071960738 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f70880a33c0 msgr2=0x7f70880a5880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.164:6800/1094864383 2024-09-19T08:45:46.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.633+0000 7f7092ffd640 1 -- 172.21.15.164:0/3071960738 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f70980efca0 con 0x7f709c100a40 2024-09-19T08:45:46.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.633+0000 7f70a1074640 1 --2- 172.21.15.164:0/3071960738 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f70880a33c0 0x7f70880a5880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T08:45:46.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.637+0000 7f7092ffd640 1 -- 172.21.15.164:0/3071960738 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f70980bc370 con 0x7f709c100a40 2024-09-19T08:45:46.781 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:46 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:46.512+0000 7fb0a0f40040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T08:45:46.782 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:46 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:46.576+0000 7fb0a0f40040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T08:45:46.782 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:46 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:46.744+0000 7fb0a0f40040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T08:45:46.818 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:45:46 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:45:46.551+0000 7f2842019040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T08:45:46.818 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:45:46 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:45:46.617+0000 7f2842019040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T08:45:46.818 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:45:46 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:45:46.700+0000 7f2842019040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T08:45:46.818 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:45:46 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:45:46.758+0000 7f2842019040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T08:45:46.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.834+0000 7f70a1074640 1 -- 172.21.15.164:0/3071960738 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f70880a33c0 msgr2=0x7f70880a5880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.164:6800/1094864383 2024-09-19T08:45:46.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.834+0000 7f70a1074640 1 --2- 172.21.15.164:0/3071960738 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f70880a33c0 0x7f70880a5880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T08:45:46.844 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:45:46 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:45:46.644+0000 7f1dbdede040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T08:45:46.844 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:45:46 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:45:46.725+0000 7f1dbdede040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T08:45:46.844 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:45:46 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:45:46.784+0000 7f1dbdede040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T08:45:46.844 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:45:46 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:45:46.844+0000 7f1dbdede040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T08:45:46.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.850+0000 7f70a3ad9640 1 -- 172.21.15.164:0/3071960738 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f709c062df0 con 0x7f709c100a40 2024-09-19T08:45:46.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.906+0000 7f7092ffd640 1 -- 172.21.15.164:0/3071960738 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v28) ==== 117+0+0 (secure 0 0 0) 0x7f70980c0670 con 0x7f709c100a40 2024-09-19T08:45:46.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.908+0000 7f70a3ad9640 1 -- 172.21.15.164:0/3071960738 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f70880a33c0 msgr2=0x7f70880a5880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:45:46.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.908+0000 7f70a3ad9640 1 --2- 172.21.15.164:0/3071960738 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f70880a33c0 0x7f70880a5880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:46.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.908+0000 7f70a3ad9640 1 -- 172.21.15.164:0/3071960738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f709c100a40 msgr2=0x7f709c06a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:46.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.908+0000 7f70a3ad9640 1 --2- 172.21.15.164:0/3071960738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f709c100a40 0x7f709c06a410 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7098009230 tx=0x7f7098004270 comp rx=0 tx=0).stop 2024-09-19T08:45:46.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.908+0000 7f70a3ad9640 1 -- 172.21.15.164:0/3071960738 shutdown_connections 2024-09-19T08:45:46.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.908+0000 7f70a3ad9640 1 --2- 172.21.15.164:0/3071960738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f709c1025f0 0x7f709c19f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:46.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.908+0000 7f70a3ad9640 1 --2- 172.21.15.164:0/3071960738 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f709c101c30 0x7f709c19cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:46.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.908+0000 7f70a3ad9640 1 --2- 172.21.15.164:0/3071960738 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f70880a33c0 0x7f70880a5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:46.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.908+0000 7f70a3ad9640 1 --2- 172.21.15.164:0/3071960738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f709c100a40 0x7f709c06a410 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:46.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.908+0000 7f70a3ad9640 1 -- 172.21.15.164:0/3071960738 >> 172.21.15.164:0/3071960738 conn(0x7f709c0fc1e0 msgr2=0x7f709c104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:46.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.908+0000 7f70a3ad9640 1 -- 172.21.15.164:0/3071960738 shutdown_connections 2024-09-19T08:45:46.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:46.908+0000 7f70a3ad9640 1 -- 172.21.15.164:0/3071960738 wait complete. 2024-09-19T08:45:46.921 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-19T08:45:46.933 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-19T08:45:46.944 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:45:47.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.047+0000 7fbbada8d640 1 -- 172.21.15.164:0/2922210807 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbba8101c00 msgr2=0x7fbba8102080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:47.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.047+0000 7fbbada8d640 1 --2- 172.21.15.164:0/2922210807 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbba8101c00 0x7fbba8102080 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbb90009860 tx=0x7fbb9002a340 comp rx=0 tx=0).stop 2024-09-19T08:45:47.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.048+0000 7fbbada8d640 1 -- 172.21.15.164:0/2922210807 shutdown_connections 2024-09-19T08:45:47.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.048+0000 7fbbada8d640 1 --2- 172.21.15.164:0/2922210807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbba81025c0 0x7fbba8108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.048+0000 7fbbada8d640 1 --2- 172.21.15.164:0/2922210807 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbba8101c00 0x7fbba8102080 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.048+0000 7fbbada8d640 1 --2- 172.21.15.164:0/2922210807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbba8100a10 0x7fbba8100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.048+0000 7fbbada8d640 1 -- 172.21.15.164:0/2922210807 >> 172.21.15.164:0/2922210807 conn(0x7fbba80fc170 msgr2=0x7fbba80fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:47.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.048+0000 7fbbada8d640 1 -- 172.21.15.164:0/2922210807 shutdown_connections 2024-09-19T08:45:47.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.048+0000 7fbbada8d640 1 -- 172.21.15.164:0/2922210807 wait complete. 2024-09-19T08:45:47.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.048+0000 7fbbada8d640 1 Processor -- start 2024-09-19T08:45:47.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.048+0000 7fbbada8d640 1 -- start start 2024-09-19T08:45:47.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.048+0000 7fbbada8d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbba8100a10 0x7fbba8196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.048+0000 7fbbada8d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbba8101c00 0x7fbba81972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.048+0000 7fbbada8d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbba81025c0 0x7fbba819e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.048+0000 7fbbada8d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbba810c690 con 0x7fbba8101c00 2024-09-19T08:45:47.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.048+0000 7fbbada8d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbba810c510 con 0x7fbba8100a10 2024-09-19T08:45:47.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.048+0000 7fbbada8d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbba810c810 con 0x7fbba81025c0 2024-09-19T08:45:47.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.049+0000 7fbb9effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbba8101c00 0x7fbba81972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:47.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.049+0000 7fbba77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbba8100a10 0x7fbba8196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:47.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.049+0000 7fbba77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbba8100a10 0x7fbba8196d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:45268/0 (socket says 172.21.15.164:45268) 2024-09-19T08:45:47.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.049+0000 7fbba77fe640 1 -- 172.21.15.164:0/1694342777 learned_addr learned my addr 172.21.15.164:0/1694342777 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:47.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.049+0000 7fbb9effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbba8101c00 0x7fbba81972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46328/0 (socket says 172.21.15.164:46328) 2024-09-19T08:45:47.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.049+0000 7fbba7fff640 1 --2- 172.21.15.164:0/1694342777 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbba81025c0 0x7fbba819e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:47.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.049+0000 7fbba77fe640 1 -- 172.21.15.164:0/1694342777 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbba81025c0 msgr2=0x7fbba819e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:47.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.049+0000 7fbba77fe640 1 --2- 172.21.15.164:0/1694342777 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbba81025c0 0x7fbba819e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.049+0000 7fbba77fe640 1 -- 172.21.15.164:0/1694342777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbba8101c00 msgr2=0x7fbba81972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:47.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.049+0000 7fbba77fe640 1 --2- 172.21.15.164:0/1694342777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbba8101c00 0x7fbba81972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.049+0000 7fbba77fe640 1 -- 172.21.15.164:0/1694342777 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba819e990 con 0x7fbba8100a10 2024-09-19T08:45:47.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.049+0000 7fbba77fe640 1 --2- 172.21.15.164:0/1694342777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbba8100a10 0x7fbba8196d70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbb9800e9f0 tx=0x7fbb9800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:47.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.049+0000 7fbba57fa640 1 -- 172.21.15.164:0/1694342777 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9800ccc0 con 0x7fbba8100a10 2024-09-19T08:45:47.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.049+0000 7fbbada8d640 1 -- 172.21.15.164:0/1694342777 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbba819ec80 con 0x7fbba8100a10 2024-09-19T08:45:47.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.049+0000 7fbba57fa640 1 -- 172.21.15.164:0/1694342777 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb980035f0 con 0x7fbba8100a10 2024-09-19T08:45:47.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.049+0000 7fbba57fa640 1 -- 172.21.15.164:0/1694342777 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9800ce60 con 0x7fbba8100a10 2024-09-19T08:45:47.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.049+0000 7fbbada8d640 1 -- 172.21.15.164:0/1694342777 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbba819f1a0 con 0x7fbba8100a10 2024-09-19T08:45:47.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.050+0000 7fbbada8d640 1 -- 172.21.15.164:0/1694342777 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbba8107e50 con 0x7fbba8100a10 2024-09-19T08:45:47.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.051+0000 7fbba57fa640 1 -- 172.21.15.164:0/1694342777 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 19) ==== 138373+0+0 (secure 0 0 0) 0x7fbb980037a0 con 0x7fbba8100a10 2024-09-19T08:45:47.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.053+0000 7fbba57fa640 1 --2- 172.21.15.164:0/1694342777 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fbb940a32a0 0x7fbb940a5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.053+0000 7fbb9effd640 1 -- 172.21.15.164:0/1694342777 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fbb940a32a0 msgr2=0x7fbb940a5760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.164:6800/1094864383 2024-09-19T08:45:47.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.053+0000 7fbba57fa640 1 -- 172.21.15.164:0/1694342777 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7fbb980cfdf0 con 0x7fbba8100a10 2024-09-19T08:45:47.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.053+0000 7fbb9effd640 1 --2- 172.21.15.164:0/1694342777 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fbb940a32a0 0x7fbb940a5760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T08:45:47.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.057+0000 7fbba57fa640 1 -- 172.21.15.164:0/1694342777 <== mon.2 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbb9809cb10 con 0x7fbba8100a10 2024-09-19T08:45:47.099 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:46 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3071960738' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-19T08:45:47.100 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:46 smithi164 ceph-mon[30928]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-19T08:45:47.100 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:46 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:46.813+0000 7fb0a0f40040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T08:45:47.100 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:46 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:46.882+0000 7fb0a0f40040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T08:45:47.100 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:46 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:46.970+0000 7fb0a0f40040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T08:45:47.100 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:47 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:47.034+0000 7fb0a0f40040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T08:45:47.100 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:47 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:47.099+0000 7fb0a0f40040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T08:45:47.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:46 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3071960738' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-19T08:45:47.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:46 smithi170 ceph-mon[35072]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-19T08:45:47.160 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:45:46 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:45:46.818+0000 7f2842019040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T08:45:47.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:46 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3071960738' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-19T08:45:47.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:46 smithi173 ceph-mon[35306]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-19T08:45:47.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.232+0000 7fbbada8d640 1 -- 172.21.15.164:0/1694342777 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbba802da70 con 0x7fbba8100a10 2024-09-19T08:45:47.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.234+0000 7fbba57fa640 1 -- 172.21.15.164:0/1694342777 <== mon.2 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22702 (secure 0 0 0) 0x7fbb9809c410 con 0x7fbba8100a10 2024-09-19T08:45:47.234 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:45:47.235 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":82,"fsid":"1e4b0d74-7662-11ef-bceb-c7b262605968","created":"2024-09-19T08:36:09.564509+0000","modified":"2024-09-19T08:44:02.811236+0000","last_up_change":"2024-09-19T08:43:01.260852+0000","last_in_change":"2024-09-19T08:42:37.492464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:39:13.961749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:43:49.613471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T08:43:51.570675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33b11f37-6ac5-41fc-8cfd-1b08c7eef688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6803","nonce":663038243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6805","nonce":663038243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6809","nonce":663038243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6807","nonce":663038243}]},"public_addr":"172.21.15.164:6803/663038243","cluster_addr":"172.21.15.164:6805/663038243","heartbeat_back_addr":"172.21.15.164:6809/663038243","heartbeat_front_addr":"172.21.15.164:6807/663038243","state":["exists","up"]},{"osd":1,"uuid":"6de64b0a-46e2-4a98-8005-a326a64bc87d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6801","nonce":2167066371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6803","nonce":2167066371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6807","nonce":2167066371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6805","nonce":2167066371}]},"public_addr":"172.21.15.170:6801/2167066371","cluster_addr":"172.21.15.170:6803/2167066371","heartbeat_back_addr":"172.21.15.170:6807/2167066371","heartbeat_front_addr":"172.21.15.170:6805/2167066371","state":["exists","up"]},{"osd":2,"uuid":"d4158cc7-9822-42a7-9601-d65089907584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6801","nonce":1646561048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6803","nonce":1646561048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6807","nonce":1646561048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6805","nonce":1646561048}]},"public_addr":"172.21.15.173:6801/1646561048","cluster_addr":"172.21.15.173:6803/1646561048","heartbeat_back_addr":"172.21.15.173:6807/1646561048","heartbeat_front_addr":"172.21.15.173:6805/1646561048","state":["exists","up"]},{"osd":3,"uuid":"0cf3f7f1-3a8c-42ce-88a9-9bea52e02221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6811","nonce":406795560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6813","nonce":406795560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6817","nonce":406795560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6815","nonce":406795560}]},"public_addr":"172.21.15.164:6811/406795560","cluster_addr":"172.21.15.164:6813/406795560","heartbeat_back_addr":"172.21.15.164:6817/406795560","heartbeat_front_addr":"172.21.15.164:6815/406795560","state":["exists","up"]},{"osd":4,"uuid":"6dca048a-2860-4a02-8d8e-450b30bc1a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6809","nonce":4120832165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6811","nonce":4120832165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6815","nonce":4120832165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6813","nonce":4120832165}]},"public_addr":"172.21.15.170:6809/4120832165","cluster_addr":"172.21.15.170:6811/4120832165","heartbeat_back_addr":"172.21.15.170:6815/4120832165","heartbeat_front_addr":"172.21.15.170:6813/4120832165","state":["exists","up"]},{"osd":5,"uuid":"3822521f-69d6-4141-a83f-5fc9cecc7b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6809","nonce":2914653552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6811","nonce":2914653552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6815","nonce":2914653552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6813","nonce":2914653552}]},"public_addr":"172.21.15.173:6809/2914653552","cluster_addr":"172.21.15.173:6811/2914653552","heartbeat_back_addr":"172.21.15.173:6815/2914653552","heartbeat_front_addr":"172.21.15.173:6813/2914653552","state":["exists","up"]},{"osd":6,"uuid":"0f5a965e-2eb2-4c24-b089-577e0639c160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6819","nonce":1138341393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6821","nonce":1138341393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6825","nonce":1138341393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6823","nonce":1138341393}]},"public_addr":"172.21.15.164:6819/1138341393","cluster_addr":"172.21.15.164:6821/1138341393","heartbeat_back_addr":"172.21.15.164:6825/1138341393","heartbeat_front_addr":"172.21.15.164:6823/1138341393","state":["exists","up"]},{"osd":7,"uuid":"f16cb2c3-639b-46ae-a740-1bd28ca1863c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6817","nonce":2124160360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6819","nonce":2124160360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6823","nonce":2124160360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6821","nonce":2124160360}]},"public_addr":"172.21.15.170:6817/2124160360","cluster_addr":"172.21.15.170:6819/2124160360","heartbeat_back_addr":"172.21.15.170:6823/2124160360","heartbeat_front_addr":"172.21.15.170:6821/2124160360","state":["exists","up"]},{"osd":8,"uuid":"161cff01-46ad-48e3-b999-a5473a2ad618","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6817","nonce":2285175061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6819","nonce":2285175061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6823","nonce":2285175061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6821","nonce":2285175061}]},"public_addr":"172.21.15.173:6817/2285175061","cluster_addr":"172.21.15.173:6819/2285175061","heartbeat_back_addr":"172.21.15.173:6823/2285175061","heartbeat_front_addr":"172.21.15.173:6821/2285175061","state":["exists","up"]},{"osd":9,"uuid":"994666e8-8fbd-4231-ac67-81a79aaf7c2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6827","nonce":3918847952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6829","nonce":3918847952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6833","nonce":3918847952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6831","nonce":3918847952}]},"public_addr":"172.21.15.164:6827/3918847952","cluster_addr":"172.21.15.164:6829/3918847952","heartbeat_back_addr":"172.21.15.164:6833/3918847952","heartbeat_front_addr":"172.21.15.164:6831/3918847952","state":["exists","up"]},{"osd":10,"uuid":"ff5e9099-cdb4-4905-b888-5b67320ef76b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6825","nonce":2707848227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6827","nonce":2707848227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6831","nonce":2707848227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6829","nonce":2707848227}]},"public_addr":"172.21.15.170:6825/2707848227","cluster_addr":"172.21.15.170:6827/2707848227","heartbeat_back_addr":"172.21.15.170:6831/2707848227","heartbeat_front_addr":"172.21.15.170:6829/2707848227","state":["exists","up"]},{"osd":11,"uuid":"4d93088f-e791-4938-a207-12f15b56abd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6825","nonce":2831924738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6827","nonce":2831924738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6831","nonce":2831924738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6829","nonce":2831924738}]},"public_addr":"172.21.15.173:6825/2831924738","cluster_addr":"172.21.15.173:6827/2831924738","heartbeat_back_addr":"172.21.15.173:6831/2831924738","heartbeat_front_addr":"172.21.15.173:6829/2831924738","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:21.787352+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:46.012323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:39:11.353040+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:39:37.015124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:02.063379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:27.892865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:53.372041+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:41:17.855284+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:41:43.389882+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:08.956895+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:34.069779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:59.303897+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.164:0/339032808":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6801/1173822158":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6800/1173822158":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6801/2805828009":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1837164919":"2024-09-20T08:37:01.663065+0000","172.21.15.164:0/4018459143":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6800/2805828009":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/2770797546":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1905394497":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1369265436":"2024-09-20T08:36:33.856642+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:45:47.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.236+0000 7fbbada8d640 1 -- 172.21.15.164:0/1694342777 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fbb940a32a0 msgr2=0x7fbb940a5760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:45:47.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.236+0000 7fbbada8d640 1 --2- 172.21.15.164:0/1694342777 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fbb940a32a0 0x7fbb940a5760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.236+0000 7fbbada8d640 1 -- 172.21.15.164:0/1694342777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbba8100a10 msgr2=0x7fbba8196d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:47.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.236+0000 7fbbada8d640 1 --2- 172.21.15.164:0/1694342777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbba8100a10 0x7fbba8196d70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbb9800e9f0 tx=0x7fbb9800eeb0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.236+0000 7fbbada8d640 1 -- 172.21.15.164:0/1694342777 shutdown_connections 2024-09-19T08:45:47.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.236+0000 7fbbada8d640 1 --2- 172.21.15.164:0/1694342777 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7fbb940a32a0 0x7fbb940a5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.236+0000 7fbbada8d640 1 --2- 172.21.15.164:0/1694342777 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbba81025c0 0x7fbba819e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.236+0000 7fbbada8d640 1 --2- 172.21.15.164:0/1694342777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbba8101c00 0x7fbba81972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.236+0000 7fbbada8d640 1 --2- 172.21.15.164:0/1694342777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbba8100a10 0x7fbba8196d70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.236+0000 7fbbada8d640 1 -- 172.21.15.164:0/1694342777 >> 172.21.15.164:0/1694342777 conn(0x7fbba80fc170 msgr2=0x7fbba8105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:47.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.236+0000 7fbbada8d640 1 -- 172.21.15.164:0/1694342777 shutdown_connections 2024-09-19T08:45:47.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.236+0000 7fbbada8d640 1 -- 172.21.15.164:0/1694342777 wait complete. 2024-09-19T08:45:47.248 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T08:39:13.961749+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T08:43:49.613471+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T08:43:51.570675+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T08:45:47.249 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T08:45:47.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.353+0000 7f1eec99f640 1 -- 172.21.15.164:0/2939730335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ee8100900 msgr2=0x7f1ee8100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:47.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.353+0000 7f1eec99f640 1 --2- 172.21.15.164:0/2939730335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ee8100900 0x7f1ee8100d00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8009900 tx=0x7f1ed802a410 comp rx=0 tx=0).stop 2024-09-19T08:45:47.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.353+0000 7f1eec99f640 1 -- 172.21.15.164:0/2939730335 shutdown_connections 2024-09-19T08:45:47.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.353+0000 7f1eec99f640 1 --2- 172.21.15.164:0/2939730335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ee81024b0 0x7f1ee8108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.353+0000 7f1eec99f640 1 --2- 172.21.15.164:0/2939730335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ee8101af0 0x7f1ee8101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.353+0000 7f1eec99f640 1 --2- 172.21.15.164:0/2939730335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ee8100900 0x7f1ee8100d00 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.353+0000 7f1eec99f640 1 -- 172.21.15.164:0/2939730335 >> 172.21.15.164:0/2939730335 conn(0x7f1ee80fc0a0 msgr2=0x7f1ee80fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:47.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.353+0000 7f1eec99f640 1 -- 172.21.15.164:0/2939730335 shutdown_connections 2024-09-19T08:45:47.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.353+0000 7f1eec99f640 1 -- 172.21.15.164:0/2939730335 wait complete. 2024-09-19T08:45:47.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.354+0000 7f1eec99f640 1 Processor -- start 2024-09-19T08:45:47.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.354+0000 7f1eec99f640 1 -- start start 2024-09-19T08:45:47.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.354+0000 7f1eec99f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ee8100900 0x7f1ee8197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.354+0000 7f1eec99f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ee8101af0 0x7f1ee81982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.354+0000 7f1eec99f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ee81024b0 0x7f1ee819f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.354+0000 7f1eec99f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee810c580 con 0x7f1ee8100900 2024-09-19T08:45:47.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.354+0000 7f1eec99f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee810c400 con 0x7f1ee81024b0 2024-09-19T08:45:47.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.354+0000 7f1eec99f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee810c700 con 0x7f1ee8101af0 2024-09-19T08:45:47.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.355+0000 7f1ee77fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ee8100900 0x7f1ee8197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:47.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.355+0000 7f1ee77fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ee8100900 0x7f1ee8197d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46342/0 (socket says 172.21.15.164:46342) 2024-09-19T08:45:47.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.355+0000 7f1ee77fe640 1 -- 172.21.15.164:0/2932847315 learned_addr learned my addr 172.21.15.164:0/2932847315 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:47.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.355+0000 7f1ee6ffd640 1 --2- 172.21.15.164:0/2932847315 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ee8101af0 0x7f1ee81982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:47.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.355+0000 7f1ee7fff640 1 --2- 172.21.15.164:0/2932847315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ee81024b0 0x7f1ee819f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:47.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.355+0000 7f1ee77fe640 1 -- 172.21.15.164:0/2932847315 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ee8101af0 msgr2=0x7f1ee81982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:47.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.355+0000 7f1ee77fe640 1 --2- 172.21.15.164:0/2932847315 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ee8101af0 0x7f1ee81982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.355+0000 7f1ee77fe640 1 -- 172.21.15.164:0/2932847315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ee81024b0 msgr2=0x7f1ee819f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:47.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.355+0000 7f1ee77fe640 1 --2- 172.21.15.164:0/2932847315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ee81024b0 0x7f1ee819f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.355+0000 7f1ee77fe640 1 -- 172.21.15.164:0/2932847315 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee819f990 con 0x7f1ee8100900 2024-09-19T08:45:47.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.355+0000 7f1ee6ffd640 1 --2- 172.21.15.164:0/2932847315 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ee8101af0 0x7f1ee81982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:45:47.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.355+0000 7f1ee77fe640 1 --2- 172.21.15.164:0/2932847315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ee8100900 0x7f1ee8197d70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8031610 tx=0x7f1ed8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:47.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.355+0000 7f1ee4ff9640 1 -- 172.21.15.164:0/2932847315 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed803f030 con 0x7f1ee8100900 2024-09-19T08:45:47.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.355+0000 7f1ee4ff9640 1 -- 172.21.15.164:0/2932847315 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ed8003ad0 con 0x7f1ee8100900 2024-09-19T08:45:47.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.355+0000 7f1ee4ff9640 1 -- 172.21.15.164:0/2932847315 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed8035070 con 0x7f1ee8100900 2024-09-19T08:45:47.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.355+0000 7f1eec99f640 1 -- 172.21.15.164:0/2932847315 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ee819fc20 con 0x7f1ee8100900 2024-09-19T08:45:47.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.355+0000 7f1eec99f640 1 -- 172.21.15.164:0/2932847315 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ee81a0160 con 0x7f1ee8100900 2024-09-19T08:45:47.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.357+0000 7f1eec99f640 1 -- 172.21.15.164:0/2932847315 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ee81a0330 con 0x7f1ee8100900 2024-09-19T08:45:47.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.357+0000 7f1ee4ff9640 1 -- 172.21.15.164:0/2932847315 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 19) ==== 138373+0+0 (secure 0 0 0) 0x7f1ed8042030 con 0x7f1ee8100900 2024-09-19T08:45:47.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.358+0000 7f1ee4ff9640 1 --2- 172.21.15.164:0/2932847315 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1ed00a3320 0x7f1ed00a57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.358+0000 7f1ee6ffd640 1 -- 172.21.15.164:0/2932847315 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1ed00a3320 msgr2=0x7f1ed00a57e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.164:6800/1094864383 2024-09-19T08:45:47.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.358+0000 7f1ee4ff9640 1 -- 172.21.15.164:0/2932847315 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f1ed80efe00 con 0x7f1ee8100900 2024-09-19T08:45:47.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.358+0000 7f1ee6ffd640 1 --2- 172.21.15.164:0/2932847315 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1ed00a3320 0x7f1ed00a57e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T08:45:47.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.362+0000 7f1ee4ff9640 1 -- 172.21.15.164:0/2932847315 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1ed80bc570 con 0x7f1ee8100900 2024-09-19T08:45:47.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.542+0000 7f1eec99f640 1 -- 172.21.15.164:0/2932847315 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1ee81a0500 con 0x7f1ee8100900 2024-09-19T08:45:47.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.543+0000 7f1ee4ff9640 1 -- 172.21.15.164:0/2932847315 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f1ed80c0870 con 0x7f1ee8100900 2024-09-19T08:45:47.543 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-09-19T08:45:47.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.546+0000 7f1eec99f640 1 -- 172.21.15.164:0/2932847315 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1ed00a3320 msgr2=0x7f1ed00a57e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:45:47.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.546+0000 7f1eec99f640 1 --2- 172.21.15.164:0/2932847315 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1ed00a3320 0x7f1ed00a57e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.546+0000 7f1eec99f640 1 -- 172.21.15.164:0/2932847315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ee8100900 msgr2=0x7f1ee8197d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:47.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.546+0000 7f1eec99f640 1 --2- 172.21.15.164:0/2932847315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ee8100900 0x7f1ee8197d70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8031610 tx=0x7f1ed8004270 comp rx=0 tx=0).stop 2024-09-19T08:45:47.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.546+0000 7f1eec99f640 1 -- 172.21.15.164:0/2932847315 shutdown_connections 2024-09-19T08:45:47.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.546+0000 7f1eec99f640 1 --2- 172.21.15.164:0/2932847315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ee81024b0 0x7f1ee819f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.546+0000 7f1eec99f640 1 --2- 172.21.15.164:0/2932847315 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f1ed00a3320 0x7f1ed00a57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.546+0000 7f1eec99f640 1 --2- 172.21.15.164:0/2932847315 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ee8101af0 0x7f1ee81982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.546+0000 7f1eec99f640 1 --2- 172.21.15.164:0/2932847315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ee8100900 0x7f1ee8197d70 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.546+0000 7f1eec99f640 1 -- 172.21.15.164:0/2932847315 >> 172.21.15.164:0/2932847315 conn(0x7f1ee80fc0a0 msgr2=0x7f1ee8069330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:47.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.546+0000 7f1eec99f640 1 -- 172.21.15.164:0/2932847315 shutdown_connections 2024-09-19T08:45:47.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.546+0000 7f1eec99f640 1 -- 172.21.15.164:0/2932847315 wait complete. 2024-09-19T08:45:47.562 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T08:45:47.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.669+0000 7f26601c4640 1 -- 172.21.15.164:0/3425744243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2658104f30 msgr2=0x7f2658107320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:47.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.669+0000 7f26601c4640 1 --2- 172.21.15.164:0/3425744243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2658104f30 0x7f2658107320 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2648009900 tx=0x7f264802a410 comp rx=0 tx=0).stop 2024-09-19T08:45:47.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.669+0000 7f26601c4640 1 -- 172.21.15.164:0/3425744243 shutdown_connections 2024-09-19T08:45:47.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.669+0000 7f26601c4640 1 --2- 172.21.15.164:0/3425744243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2658107860 0x7f2658109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.669+0000 7f26601c4640 1 --2- 172.21.15.164:0/3425744243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2658104f30 0x7f2658107320 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.669+0000 7f26601c4640 1 --2- 172.21.15.164:0/3425744243 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2658102600 0x7f26581049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.669+0000 7f26601c4640 1 -- 172.21.15.164:0/3425744243 >> 172.21.15.164:0/3425744243 conn(0x7f26580fc140 msgr2=0x7f26580fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:47.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.669+0000 7f26601c4640 1 -- 172.21.15.164:0/3425744243 shutdown_connections 2024-09-19T08:45:47.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.669+0000 7f26601c4640 1 -- 172.21.15.164:0/3425744243 wait complete. 2024-09-19T08:45:47.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.669+0000 7f26601c4640 1 Processor -- start 2024-09-19T08:45:47.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.669+0000 7f26601c4640 1 -- start start 2024-09-19T08:45:47.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.670+0000 7f26601c4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2658102600 0x7f2658197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.670+0000 7f26601c4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2658104f30 0x7f2658198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.670+0000 7f26601c4640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2658107860 0x7f265819f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.670+0000 7f26601c4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f265810d7b0 con 0x7f2658104f30 2024-09-19T08:45:47.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.670+0000 7f26601c4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f265810d630 con 0x7f2658102600 2024-09-19T08:45:47.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.670+0000 7f26601c4640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f265810d930 con 0x7f2658107860 2024-09-19T08:45:47.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.670+0000 7f265d75f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2658104f30 0x7f2658198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:47.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.670+0000 7f265d75f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2658104f30 0x7f2658198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46360/0 (socket says 172.21.15.164:46360) 2024-09-19T08:45:47.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.670+0000 7f265d75f640 1 -- 172.21.15.164:0/2053344996 learned_addr learned my addr 172.21.15.164:0/2053344996 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:47.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.670+0000 7f265df60640 1 --2- 172.21.15.164:0/2053344996 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2658102600 0x7f2658197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:47.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.670+0000 7f265e761640 1 --2- 172.21.15.164:0/2053344996 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2658107860 0x7f265819f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:47.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.670+0000 7f265d75f640 1 -- 172.21.15.164:0/2053344996 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2658107860 msgr2=0x7f265819f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:47.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.670+0000 7f265d75f640 1 --2- 172.21.15.164:0/2053344996 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2658107860 0x7f265819f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.670+0000 7f265d75f640 1 -- 172.21.15.164:0/2053344996 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2658102600 msgr2=0x7f2658197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:47.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.670+0000 7f265d75f640 1 --2- 172.21.15.164:0/2053344996 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2658102600 0x7f2658197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.670+0000 7f265d75f640 1 -- 172.21.15.164:0/2053344996 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265819fb40 con 0x7f2658104f30 2024-09-19T08:45:47.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.670+0000 7f265e761640 1 --2- 172.21.15.164:0/2053344996 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2658107860 0x7f265819f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:47.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.670+0000 7f265d75f640 1 --2- 172.21.15.164:0/2053344996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2658104f30 0x7f2658198460 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2648031610 tx=0x7f2648004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:47.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.670+0000 7f264effd640 1 -- 172.21.15.164:0/2053344996 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264803f030 con 0x7f2658104f30 2024-09-19T08:45:47.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.670+0000 7f26601c4640 1 -- 172.21.15.164:0/2053344996 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f265819fdd0 con 0x7f2658104f30 2024-09-19T08:45:47.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.670+0000 7f264effd640 1 -- 172.21.15.164:0/2053344996 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2648003ad0 con 0x7f2658104f30 2024-09-19T08:45:47.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.670+0000 7f264effd640 1 -- 172.21.15.164:0/2053344996 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648035070 con 0x7f2658104f30 2024-09-19T08:45:47.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.670+0000 7f26601c4640 1 -- 172.21.15.164:0/2053344996 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26581a0260 con 0x7f2658104f30 2024-09-19T08:45:47.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.672+0000 7f26601c4640 1 -- 172.21.15.164:0/2053344996 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26580fdc30 con 0x7f2658104f30 2024-09-19T08:45:47.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.674+0000 7f264effd640 1 -- 172.21.15.164:0/2053344996 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 19) ==== 138373+0+0 (secure 0 0 0) 0x7f2648042030 con 0x7f2658104f30 2024-09-19T08:45:47.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.674+0000 7f264effd640 1 --2- 172.21.15.164:0/2053344996 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f26440a32a0 0x7f26440a5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.674+0000 7f265df60640 1 -- 172.21.15.164:0/2053344996 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f26440a32a0 msgr2=0x7f26440a5760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.164:6800/1094864383 2024-09-19T08:45:47.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.674+0000 7f264effd640 1 -- 172.21.15.164:0/2053344996 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8153+0+0 (secure 0 0 0) 0x7f26480f0270 con 0x7f2658104f30 2024-09-19T08:45:47.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.674+0000 7f265df60640 1 --2- 172.21.15.164:0/2053344996 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f26440a32a0 0x7f26440a5760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T08:45:47.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.677+0000 7f264effd640 1 -- 172.21.15.164:0/2053344996 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f26480bcf90 con 0x7f2658104f30 2024-09-19T08:45:47.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.851+0000 7f26601c4640 1 -- 172.21.15.164:0/2053344996 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f265802da10 con 0x7f2658104f30 2024-09-19T08:45:47.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.851+0000 7f264effd640 1 -- 172.21.15.164:0/2053344996 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) ==== 104+0+11 (secure 0 0 0) 0x7f26480bc890 con 0x7f2658104f30 2024-09-19T08:45:47.852 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-09-19T08:45:47.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.853+0000 7f26601c4640 1 -- 172.21.15.164:0/2053344996 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f26440a32a0 msgr2=0x7f26440a5760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:45:47.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.853+0000 7f26601c4640 1 --2- 172.21.15.164:0/2053344996 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f26440a32a0 0x7f26440a5760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.854+0000 7f26601c4640 1 -- 172.21.15.164:0/2053344996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2658104f30 msgr2=0x7f2658198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:47.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.854+0000 7f26601c4640 1 --2- 172.21.15.164:0/2053344996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2658104f30 0x7f2658198460 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2648031610 tx=0x7f2648004270 comp rx=0 tx=0).stop 2024-09-19T08:45:47.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.854+0000 7f26601c4640 1 -- 172.21.15.164:0/2053344996 shutdown_connections 2024-09-19T08:45:47.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.854+0000 7f26601c4640 1 --2- 172.21.15.164:0/2053344996 >> [v2:172.21.15.164:6800/1094864383,v1:172.21.15.164:6801/1094864383] conn(0x7f26440a32a0 0x7f26440a5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.854+0000 7f26601c4640 1 --2- 172.21.15.164:0/2053344996 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2658107860 0x7f265819f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.854+0000 7f26601c4640 1 --2- 172.21.15.164:0/2053344996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2658104f30 0x7f2658198460 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.854+0000 7f26601c4640 1 --2- 172.21.15.164:0/2053344996 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2658102600 0x7f2658197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.854+0000 7f26601c4640 1 -- 172.21.15.164:0/2053344996 >> 172.21.15.164:0/2053344996 conn(0x7f26580fc140 msgr2=0x7f2658102ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:47.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.854+0000 7f26601c4640 1 -- 172.21.15.164:0/2053344996 shutdown_connections 2024-09-19T08:45:47.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.854+0000 7f26601c4640 1 -- 172.21.15.164:0/2053344996 wait complete. 2024-09-19T08:45:47.865 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T08:45:47.909 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:45:47 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y[36438]: 2024-09-19T08:45:47.638+0000 7f1dbdede040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T08:45:47.909 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:45:47 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z[37109]: 2024-09-19T08:45:47.604+0000 7f2842019040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T08:45:47.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.977+0000 7fc1ce4f1640 1 -- 172.21.15.164:0/3424380900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1c8101c20 msgr2=0x7fc1c81020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:47.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.977+0000 7fc1ce4f1640 1 --2- 172.21.15.164:0/3424380900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1c8101c20 0x7fc1c81020a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc009860 tx=0x7fc1bc02a340 comp rx=0 tx=0).stop 2024-09-19T08:45:47.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.977+0000 7fc1ce4f1640 1 -- 172.21.15.164:0/3424380900 shutdown_connections 2024-09-19T08:45:47.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.977+0000 7fc1ce4f1640 1 --2- 172.21.15.164:0/3424380900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1c81025e0 0x7fc1c8108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.977+0000 7fc1ce4f1640 1 --2- 172.21.15.164:0/3424380900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1c8101c20 0x7fc1c81020a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.977+0000 7fc1ce4f1640 1 --2- 172.21.15.164:0/3424380900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1c8100a30 0x7fc1c8100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.977+0000 7fc1ce4f1640 1 -- 172.21.15.164:0/3424380900 >> 172.21.15.164:0/3424380900 conn(0x7fc1c80fc1b0 msgr2=0x7fc1c80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:47.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.977+0000 7fc1ce4f1640 1 -- 172.21.15.164:0/3424380900 shutdown_connections 2024-09-19T08:45:47.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.977+0000 7fc1ce4f1640 1 -- 172.21.15.164:0/3424380900 wait complete. 2024-09-19T08:45:47.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.978+0000 7fc1ce4f1640 1 Processor -- start 2024-09-19T08:45:47.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.978+0000 7fc1ce4f1640 1 -- start start 2024-09-19T08:45:47.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.979+0000 7fc1ce4f1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1c8100a30 0x7fc1c806aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.979+0000 7fc1ce4f1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1c8101c20 0x7fc1c819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.979+0000 7fc1ce4f1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1c81025e0 0x7fc1c819f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.979+0000 7fc1ce4f1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c810c6b0 con 0x7fc1c8101c20 2024-09-19T08:45:47.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.979+0000 7fc1ce4f1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c810c530 con 0x7fc1c81025e0 2024-09-19T08:45:47.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.979+0000 7fc1ce4f1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c810c830 con 0x7fc1c8100a30 2024-09-19T08:45:47.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.979+0000 7fc1c77fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1c8101c20 0x7fc1c819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:47.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.979+0000 7fc1c77fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1c8101c20 0x7fc1c819cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46394/0 (socket says 172.21.15.164:46394) 2024-09-19T08:45:47.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.979+0000 7fc1c77fe640 1 -- 172.21.15.164:0/4286810926 learned_addr learned my addr 172.21.15.164:0/4286810926 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:47.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.979+0000 7fc1cca8e640 1 --2- 172.21.15.164:0/4286810926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1c81025e0 0x7fc1c819f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:47.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.979+0000 7fc1c7fff640 1 --2- 172.21.15.164:0/4286810926 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1c8100a30 0x7fc1c806aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:47.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.979+0000 7fc1c77fe640 1 -- 172.21.15.164:0/4286810926 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1c8100a30 msgr2=0x7fc1c806aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:47.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.979+0000 7fc1c77fe640 1 --2- 172.21.15.164:0/4286810926 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1c8100a30 0x7fc1c806aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.979+0000 7fc1c77fe640 1 -- 172.21.15.164:0/4286810926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1c81025e0 msgr2=0x7fc1c819f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:47.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.979+0000 7fc1c77fe640 1 --2- 172.21.15.164:0/4286810926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1c81025e0 0x7fc1c819f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.979+0000 7fc1c77fe640 1 -- 172.21.15.164:0/4286810926 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c819fb20 con 0x7fc1c8101c20 2024-09-19T08:45:47.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.979+0000 7fc1cca8e640 1 --2- 172.21.15.164:0/4286810926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1c81025e0 0x7fc1c819f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:45:47.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.980+0000 7fc1c77fe640 1 --2- 172.21.15.164:0/4286810926 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1c8101c20 0x7fc1c819cbf0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc035040 tx=0x7fc1bc033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:47.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.980+0000 7fc1c57fa640 1 -- 172.21.15.164:0/4286810926 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc037040 con 0x7fc1c8101c20 2024-09-19T08:45:47.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.980+0000 7fc1ce4f1640 1 -- 172.21.15.164:0/4286810926 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c819fd50 con 0x7fc1c8101c20 2024-09-19T08:45:47.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.980+0000 7fc1ce4f1640 1 -- 172.21.15.164:0/4286810926 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c81a0230 con 0x7fc1c8101c20 2024-09-19T08:45:47.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.980+0000 7fc1c57fa640 1 -- 172.21.15.164:0/4286810926 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1bc030850 con 0x7fc1c8101c20 2024-09-19T08:45:47.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.980+0000 7fc1c57fa640 1 -- 172.21.15.164:0/4286810926 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc0046c0 con 0x7fc1c8101c20 2024-09-19T08:45:47.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.981+0000 7fc1ce4f1640 1 -- 172.21.15.164:0/4286810926 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1b40052c0 con 0x7fc1c8101c20 2024-09-19T08:45:47.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.983+0000 7fc1c57fa640 1 -- 172.21.15.164:0/4286810926 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 21) ==== 138040+0+0 (secure 0 0 0) 0x7fc1bc004050 con 0x7fc1c8101c20 2024-09-19T08:45:47.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.983+0000 7fc1c57fa640 1 -- 172.21.15.164:0/4286810926 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fc1bc0ef480 con 0x7fc1c8101c20 2024-09-19T08:45:47.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:47.987+0000 7fc1c57fa640 1 -- 172.21.15.164:0/4286810926 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc1bc0bc4f0 con 0x7fc1c8101c20 2024-09-19T08:45:48.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:47 smithi173 ceph-mon[35306]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-19T08:45:48.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:47 smithi173 ceph-mon[35306]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T08:45:48.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:47 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/1694342777' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:45:48.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:47 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2932847315' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T08:45:48.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:47 smithi173 ceph-mon[35306]: Standby manager daemon z restarted 2024-09-19T08:45:48.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:47 smithi173 ceph-mon[35306]: Standby manager daemon z started 2024-09-19T08:45:48.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:47 smithi173 ceph-mon[35306]: from='mgr.? 172.21.15.173:0/1133876732' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T08:45:48.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:47 smithi173 ceph-mon[35306]: from='mgr.? 172.21.15.173:0/1133876732' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T08:45:48.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:47 smithi173 ceph-mon[35306]: from='mgr.? 172.21.15.173:0/1133876732' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T08:45:48.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:47 smithi173 ceph-mon[35306]: from='mgr.? 172.21.15.173:0/1133876732' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T08:45:48.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:47 smithi173 ceph-mon[35306]: Standby manager daemon y restarted 2024-09-19T08:45:48.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:47 smithi173 ceph-mon[35306]: Standby manager daemon y started 2024-09-19T08:45:48.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:47 smithi173 ceph-mon[35306]: from='mgr.? 172.21.15.170:0/2554413846' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T08:45:48.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:47 smithi173 ceph-mon[35306]: from='mgr.? 172.21.15.170:0/2554413846' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T08:45:48.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:47 smithi173 ceph-mon[35306]: from='mgr.? 172.21.15.170:0/2554413846' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T08:45:48.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:47 smithi173 ceph-mon[35306]: from='mgr.? 172.21.15.170:0/2554413846' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T08:45:48.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:47 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2053344996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T08:45:48.204 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:47 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:47.957+0000 7fb0a0f40040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T08:45:48.204 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:47 smithi164 ceph-mon[30928]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-19T08:45:48.204 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:47 smithi164 ceph-mon[30928]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T08:45:48.204 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:47 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/1694342777' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:45:48.205 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:47 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2932847315' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T08:45:48.205 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:47 smithi164 ceph-mon[30928]: Standby manager daemon z restarted 2024-09-19T08:45:48.205 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:47 smithi164 ceph-mon[30928]: Standby manager daemon z started 2024-09-19T08:45:48.205 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:47 smithi164 ceph-mon[30928]: from='mgr.? 172.21.15.173:0/1133876732' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T08:45:48.205 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:47 smithi164 ceph-mon[30928]: from='mgr.? 172.21.15.173:0/1133876732' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T08:45:48.205 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:47 smithi164 ceph-mon[30928]: from='mgr.? 172.21.15.173:0/1133876732' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T08:45:48.205 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:47 smithi164 ceph-mon[30928]: from='mgr.? 172.21.15.173:0/1133876732' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T08:45:48.205 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:47 smithi164 ceph-mon[30928]: Standby manager daemon y restarted 2024-09-19T08:45:48.205 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:47 smithi164 ceph-mon[30928]: Standby manager daemon y started 2024-09-19T08:45:48.205 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:47 smithi164 ceph-mon[30928]: from='mgr.? 172.21.15.170:0/2554413846' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T08:45:48.205 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:47 smithi164 ceph-mon[30928]: from='mgr.? 172.21.15.170:0/2554413846' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T08:45:48.205 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:47 smithi164 ceph-mon[30928]: from='mgr.? 172.21.15.170:0/2554413846' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T08:45:48.205 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:47 smithi164 ceph-mon[30928]: from='mgr.? 172.21.15.170:0/2554413846' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T08:45:48.206 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:47 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2053344996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T08:45:48.233 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:47 smithi170 ceph-mon[35072]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-19T08:45:48.233 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:47 smithi170 ceph-mon[35072]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T08:45:48.233 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:47 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/1694342777' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:45:48.234 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:47 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2932847315' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T08:45:48.234 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:47 smithi170 ceph-mon[35072]: Standby manager daemon z restarted 2024-09-19T08:45:48.234 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:47 smithi170 ceph-mon[35072]: Standby manager daemon z started 2024-09-19T08:45:48.234 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:47 smithi170 ceph-mon[35072]: from='mgr.? 172.21.15.173:0/1133876732' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T08:45:48.234 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:47 smithi170 ceph-mon[35072]: from='mgr.? 172.21.15.173:0/1133876732' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T08:45:48.234 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:47 smithi170 ceph-mon[35072]: from='mgr.? 172.21.15.173:0/1133876732' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T08:45:48.234 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:47 smithi170 ceph-mon[35072]: from='mgr.? 172.21.15.173:0/1133876732' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T08:45:48.234 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:47 smithi170 ceph-mon[35072]: Standby manager daemon y restarted 2024-09-19T08:45:48.234 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:47 smithi170 ceph-mon[35072]: Standby manager daemon y started 2024-09-19T08:45:48.234 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:47 smithi170 ceph-mon[35072]: from='mgr.? 172.21.15.170:0/2554413846' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T08:45:48.234 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:47 smithi170 ceph-mon[35072]: from='mgr.? 172.21.15.170:0/2554413846' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T08:45:48.234 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:47 smithi170 ceph-mon[35072]: from='mgr.? 172.21.15.170:0/2554413846' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T08:45:48.234 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:47 smithi170 ceph-mon[35072]: from='mgr.? 172.21.15.170:0/2554413846' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T08:45:48.234 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:47 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2053344996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T08:45:48.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:48.256+0000 7fc1ce4f1640 1 -- 172.21.15.164:0/4286810926 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc1b4005880 con 0x7fc1c8101c20 2024-09-19T08:45:48.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:48.256+0000 7fc1c57fa640 1 -- 172.21.15.164:0/4286810926 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) ==== 100+0+11 (secure 0 0 0) 0x7fc1bc0bbdf0 con 0x7fc1c8101c20 2024-09-19T08:45:48.257 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-09-19T08:45:48.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:48.259+0000 7fc1ce4f1640 1 -- 172.21.15.164:0/4286810926 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1c8101c20 msgr2=0x7fc1c819cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:48.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:48.259+0000 7fc1ce4f1640 1 --2- 172.21.15.164:0/4286810926 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1c8101c20 0x7fc1c819cbf0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc035040 tx=0x7fc1bc033d90 comp rx=0 tx=0).stop 2024-09-19T08:45:48.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:48.259+0000 7fc1ce4f1640 1 -- 172.21.15.164:0/4286810926 shutdown_connections 2024-09-19T08:45:48.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:48.259+0000 7fc1ce4f1640 1 --2- 172.21.15.164:0/4286810926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1c81025e0 0x7fc1c819f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:48.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:48.259+0000 7fc1ce4f1640 1 --2- 172.21.15.164:0/4286810926 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1c8101c20 0x7fc1c819cbf0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:48.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:48.259+0000 7fc1ce4f1640 1 --2- 172.21.15.164:0/4286810926 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1c8100a30 0x7fc1c806aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:48.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:48.259+0000 7fc1ce4f1640 1 -- 172.21.15.164:0/4286810926 >> 172.21.15.164:0/4286810926 conn(0x7fc1c80fc1b0 msgr2=0x7fc1c8104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:48.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:48.259+0000 7fc1ce4f1640 1 -- 172.21.15.164:0/4286810926 shutdown_connections 2024-09-19T08:45:48.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:48.259+0000 7fc1ce4f1640 1 -- 172.21.15.164:0/4286810926 wait complete. 2024-09-19T08:45:48.270 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-19T08:45:48.271 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:45:48.532 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:48.214+0000 7fb00b893640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:45:48.532 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:48.214+0000 7fb00b893640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:45:48.532 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:48.214+0000 7fb00b893640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:45:48.532 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:48.214+0000 7fb00b893640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:45:48.532 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:48.214+0000 7fb00b893640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:45:48.532 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:48.214+0000 7fb00908e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:45:48.533 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:48.214+0000 7fb00908e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:45:48.533 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:48.214+0000 7fb00908e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:45:48.533 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:48.214+0000 7fb00908e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:45:48.533 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:48.214+0000 7fb00908e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:45:48.536 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:45:49.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.060+0000 7fa912510640 1 -- 172.21.15.164:0/2743709184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa90c106a80 msgr2=0x7fa90c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:49.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.060+0000 7fa912510640 1 --2- 172.21.15.164:0/2743709184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa90c106a80 0x7fa90c106e60 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc009930 tx=0x7fa8fc02a420 comp rx=0 tx=0).stop 2024-09-19T08:45:49.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.060+0000 7fa912510640 1 -- 172.21.15.164:0/2743709184 shutdown_connections 2024-09-19T08:45:49.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.060+0000 7fa912510640 1 --2- 172.21.15.164:0/2743709184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa90c101420 0x7fa90c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:49.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.060+0000 7fa912510640 1 --2- 172.21.15.164:0/2743709184 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa90c100a80 0x7fa90c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:49.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.060+0000 7fa912510640 1 --2- 172.21.15.164:0/2743709184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa90c106a80 0x7fa90c106e60 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:49.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.060+0000 7fa912510640 1 -- 172.21.15.164:0/2743709184 >> 172.21.15.164:0/2743709184 conn(0x7fa90c0fc7a0 msgr2=0x7fa90c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:49.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.060+0000 7fa912510640 1 -- 172.21.15.164:0/2743709184 shutdown_connections 2024-09-19T08:45:49.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.060+0000 7fa912510640 1 -- 172.21.15.164:0/2743709184 wait complete. 2024-09-19T08:45:49.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.061+0000 7fa912510640 1 Processor -- start 2024-09-19T08:45:49.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.061+0000 7fa912510640 1 -- start start 2024-09-19T08:45:49.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.061+0000 7fa912510640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa90c100a80 0x7fa90c1973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:49.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.062+0000 7fa912510640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa90c101420 0x7fa90c1978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:49.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.062+0000 7fa912510640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa90c106a80 0x7fa90c19bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:49.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.062+0000 7fa912510640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c110e80 con 0x7fa90c101420 2024-09-19T08:45:49.077 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.062+0000 7fa912510640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c110d00 con 0x7fa90c106a80 2024-09-19T08:45:49.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.062+0000 7fa912510640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c111000 con 0x7fa90c100a80 2024-09-19T08:45:49.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.062+0000 7fa90b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa90c101420 0x7fa90c1978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:49.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.062+0000 7fa90b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa90c101420 0x7fa90c1978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46508/0 (socket says 172.21.15.164:46508) 2024-09-19T08:45:49.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.062+0000 7fa90b7fe640 1 -- 172.21.15.164:0/2654565927 learned_addr learned my addr 172.21.15.164:0/2654565927 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:49.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.062+0000 7fa910aad640 1 --2- 172.21.15.164:0/2654565927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa90c106a80 0x7fa90c19bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:49.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.062+0000 7fa90bfff640 1 --2- 172.21.15.164:0/2654565927 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa90c100a80 0x7fa90c1973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:49.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.062+0000 7fa90b7fe640 1 -- 172.21.15.164:0/2654565927 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa90c100a80 msgr2=0x7fa90c1973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:49.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.062+0000 7fa90b7fe640 1 --2- 172.21.15.164:0/2654565927 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa90c100a80 0x7fa90c1973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:49.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.062+0000 7fa90b7fe640 1 -- 172.21.15.164:0/2654565927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa90c106a80 msgr2=0x7fa90c19bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:49.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.062+0000 7fa90b7fe640 1 --2- 172.21.15.164:0/2654565927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa90c106a80 0x7fa90c19bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:49.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.062+0000 7fa90b7fe640 1 -- 172.21.15.164:0/2654565927 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa90c19c350 con 0x7fa90c101420 2024-09-19T08:45:49.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.062+0000 7fa90b7fe640 1 --2- 172.21.15.164:0/2654565927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa90c101420 0x7fa90c1978e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa90000d770 tx=0x7fa90000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:49.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.062+0000 7fa9097fa640 1 -- 172.21.15.164:0/2654565927 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa900017070 con 0x7fa90c101420 2024-09-19T08:45:49.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.062+0000 7fa912510640 1 -- 172.21.15.164:0/2654565927 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa90c19c640 con 0x7fa90c101420 2024-09-19T08:45:49.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.062+0000 7fa9097fa640 1 -- 172.21.15.164:0/2654565927 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa900005500 con 0x7fa90c101420 2024-09-19T08:45:49.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.062+0000 7fa9097fa640 1 -- 172.21.15.164:0/2654565927 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa900004e00 con 0x7fa90c101420 2024-09-19T08:45:49.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.062+0000 7fa912510640 1 -- 172.21.15.164:0/2654565927 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa90c1a3f20 con 0x7fa90c101420 2024-09-19T08:45:49.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.063+0000 7fa912510640 1 -- 172.21.15.164:0/2654565927 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa90c100ee0 con 0x7fa90c101420 2024-09-19T08:45:49.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.065+0000 7fa9097fa640 1 -- 172.21.15.164:0/2654565927 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7fa900004fa0 con 0x7fa90c101420 2024-09-19T08:45:49.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.065+0000 7fa9097fa640 1 --2- 172.21.15.164:0/2654565927 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa8f40a3030 0x7fa8f40a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:49.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.065+0000 7fa90bfff640 1 --2- 172.21.15.164:0/2654565927 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa8f40a3030 0x7fa8f40a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:49.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.065+0000 7fa9097fa640 1 -- 172.21.15.164:0/2654565927 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fa9000cf470 con 0x7fa90c101420 2024-09-19T08:45:49.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.066+0000 7fa90bfff640 1 --2- 172.21.15.164:0/2654565927 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa8f40a3030 0x7fa8f40a54f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc000c00 tx=0x7fa8fc005a20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:49.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.069+0000 7fa9097fa640 1 -- 172.21.15.164:0/2654565927 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa90009c3d0 con 0x7fa90c101420 2024-09-19T08:45:49.216 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-09-19T08:45:49.216 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: Active manager daemon x restarted 2024-09-19T08:45:49.216 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: Activating manager daemon x 2024-09-19T08:45:49.216 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T08:45:49.216 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: mgrmap e21: x(active, starting, since 0.0137148s), standbys: z, y 2024-09-19T08:45:49.217 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:45:49.217 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:45:49.217 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:45:49.217 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T08:45:49.217 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T08:45:49.217 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T08:45:49.217 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T08:45:49.218 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T08:45:49.218 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T08:45:49.218 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T08:45:49.218 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T08:45:49.218 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T08:45:49.218 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T08:45:49.218 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T08:45:49.218 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T08:45:49.218 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T08:45:49.218 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T08:45:49.219 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T08:45:49.219 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:45:49.219 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:45:49.219 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:45:49.219 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:45:49.219 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:45:49.219 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:45:49.219 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:45:49.219 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:45:49.219 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:45:49.219 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:45:49.219 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:45:49.219 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:45:49.219 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T08:45:49.220 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T08:45:49.220 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T08:45:49.220 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: Manager daemon x is now available 2024-09-19T08:45:49.220 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:45:49.220 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T08:45:49.220 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T08:45:49.220 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:48 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/4286810926' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T08:45:49.236 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-09-19T08:45:49.236 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: Active manager daemon x restarted 2024-09-19T08:45:49.236 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: Activating manager daemon x 2024-09-19T08:45:49.236 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T08:45:49.237 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: mgrmap e21: x(active, starting, since 0.0137148s), standbys: z, y 2024-09-19T08:45:49.237 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:45:49.237 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:45:49.237 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:45:49.237 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T08:45:49.237 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T08:45:49.237 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T08:45:49.237 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T08:45:49.237 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T08:45:49.237 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T08:45:49.237 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T08:45:49.237 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T08:45:49.238 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T08:45:49.238 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T08:45:49.238 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T08:45:49.238 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T08:45:49.238 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T08:45:49.238 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T08:45:49.238 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T08:45:49.238 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:45:49.238 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:45:49.238 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:45:49.238 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:45:49.238 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:45:49.238 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:45:49.239 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:45:49.239 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:45:49.239 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:45:49.239 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:45:49.239 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:45:49.239 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:45:49.239 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T08:45:49.239 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T08:45:49.239 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T08:45:49.239 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: Manager daemon x is now available 2024-09-19T08:45:49.239 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:45:49.239 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T08:45:49.239 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T08:45:49.239 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:48 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/4286810926' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T08:45:49.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-09-19T08:45:49.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: Active manager daemon x restarted 2024-09-19T08:45:49.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: Activating manager daemon x 2024-09-19T08:45:49.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T08:45:49.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: mgrmap e21: x(active, starting, since 0.0137148s), standbys: z, y 2024-09-19T08:45:49.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T08:45:49.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T08:45:49.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T08:45:49.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T08:45:49.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T08:45:49.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T08:45:49.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T08:45:49.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T08:45:49.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T08:45:49.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T08:45:49.284 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T08:45:49.284 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T08:45:49.284 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T08:45:49.284 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T08:45:49.284 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T08:45:49.284 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T08:45:49.284 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T08:45:49.284 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T08:45:49.284 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T08:45:49.285 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T08:45:49.285 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T08:45:49.285 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T08:45:49.285 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T08:45:49.285 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:49 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:49.139+0000 7faff47ab640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:08:45:49] ENGINE Error in HTTPServer.serve 2024-09-19T08:45:49.285 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:49 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: Traceback (most recent call last): 2024-09-19T08:45:49.285 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:49 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T08:45:49.285 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:49 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: self._connections.run(self.expiration_interval) 2024-09-19T08:45:49.286 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:49 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T08:45:49.286 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:49 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: self._run(expiration_interval) 2024-09-19T08:45:49.286 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:49 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T08:45:49.286 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:49 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T08:45:49.286 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:49 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T08:45:49.286 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:49 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T08:45:49.286 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:49 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T08:45:49.286 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:49 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: s = self.context.wrap_socket( 2024-09-19T08:45:49.286 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:49 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T08:45:49.286 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:49 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: return self.sslsocket_class._create( 2024-09-19T08:45:49.286 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:49 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T08:45:49.286 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:49 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: self.do_handshake() 2024-09-19T08:45:49.287 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:49 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T08:45:49.287 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:49 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: self._sslobj.do_handshake() 2024-09-19T08:45:49.287 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:49 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T08:45:49.287 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:45:49 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:45:49.287 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T08:45:49.287 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T08:45:49.287 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T08:45:49.287 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T08:45:49.287 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T08:45:49.287 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T08:45:49.287 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T08:45:49.288 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T08:45:49.288 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T08:45:49.288 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T08:45:49.288 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: Manager daemon x is now available 2024-09-19T08:45:49.288 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:45:49.288 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T08:45:49.288 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T08:45:49.288 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:48 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/4286810926' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T08:45:49.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.312+0000 7fa912510640 1 -- 172.21.15.164:0/2654565927 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa90c198700 con 0x7fa90c101420 2024-09-19T08:45:49.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.313+0000 7fa9097fa640 1 -- 172.21.15.164:0/2654565927 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11984 (secure 0 0 0) 0x7fa90c198700 con 0x7fa90c101420 2024-09-19T08:45:49.314 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:45:49.314 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":28,"btime":"2024-09-19T08:45:46:903004+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:45:46.852789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:45:49.314 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 28 2024-09-19T08:45:49.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.315+0000 7fa912510640 1 -- 172.21.15.164:0/2654565927 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa8f40a3030 msgr2=0x7fa8f40a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:49.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.315+0000 7fa912510640 1 --2- 172.21.15.164:0/2654565927 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa8f40a3030 0x7fa8f40a54f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc000c00 tx=0x7fa8fc005a20 comp rx=0 tx=0).stop 2024-09-19T08:45:49.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.315+0000 7fa912510640 1 -- 172.21.15.164:0/2654565927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa90c101420 msgr2=0x7fa90c1978e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:49.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.315+0000 7fa912510640 1 --2- 172.21.15.164:0/2654565927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa90c101420 0x7fa90c1978e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa90000d770 tx=0x7fa90000dc30 comp rx=0 tx=0).stop 2024-09-19T08:45:49.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.315+0000 7fa912510640 1 -- 172.21.15.164:0/2654565927 shutdown_connections 2024-09-19T08:45:49.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.315+0000 7fa912510640 1 --2- 172.21.15.164:0/2654565927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa90c106a80 0x7fa90c19bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:49.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.315+0000 7fa912510640 1 --2- 172.21.15.164:0/2654565927 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa8f40a3030 0x7fa8f40a54f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:49.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.315+0000 7fa912510640 1 --2- 172.21.15.164:0/2654565927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa90c101420 0x7fa90c1978e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:49.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.315+0000 7fa912510640 1 --2- 172.21.15.164:0/2654565927 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa90c100a80 0x7fa90c1973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:49.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.315+0000 7fa912510640 1 -- 172.21.15.164:0/2654565927 >> 172.21.15.164:0/2654565927 conn(0x7fa90c0fc7a0 msgr2=0x7fa90c0fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:49.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.315+0000 7fa912510640 1 -- 172.21.15.164:0/2654565927 shutdown_connections 2024-09-19T08:45:49.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:49.315+0000 7fa912510640 1 -- 172.21.15.164:0/2654565927 wait complete. 2024-09-19T08:45:50.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:49 smithi164 ceph-mon[30928]: mgrmap e22: x(active, since 1.01655s), standbys: z, y 2024-09-19T08:45:50.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:49 smithi164 ceph-mon[30928]: [19/Sep/2024:08:45:49] ENGINE Bus STARTING 2024-09-19T08:45:50.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:49 smithi164 ceph-mon[30928]: [19/Sep/2024:08:45:49] ENGINE Serving on https://172.21.15.164:7150 2024-09-19T08:45:50.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:49 smithi164 ceph-mon[30928]: [19/Sep/2024:08:45:49] ENGINE Error in HTTPServer.serve 2024-09-19T08:45:50.032 INFO:journalctl@ceph.mon.a.smithi164.stdout: Traceback (most recent call last): 2024-09-19T08:45:50.032 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T08:45:50.032 INFO:journalctl@ceph.mon.a.smithi164.stdout: self._connections.run(self.expiration_interval) 2024-09-19T08:45:50.032 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T08:45:50.032 INFO:journalctl@ceph.mon.a.smithi164.stdout: self._run(expiration_interval) 2024-09-19T08:45:50.033 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T08:45:50.033 INFO:journalctl@ceph.mon.a.smithi164.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T08:45:50.033 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T08:45:50.033 INFO:journalctl@ceph.mon.a.smithi164.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T08:45:50.033 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T08:45:50.033 INFO:journalctl@ceph.mon.a.smithi164.stdout: s = self.context.wrap_socket( 2024-09-19T08:45:50.033 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T08:45:50.033 INFO:journalctl@ceph.mon.a.smithi164.stdout: return self.sslsocket_class._create( 2024-09-19T08:45:50.033 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T08:45:50.033 INFO:journalctl@ceph.mon.a.smithi164.stdout: self.do_handshake() 2024-09-19T08:45:50.034 INFO:journalctl@ceph.mon.a.smithi164.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T08:45:50.034 INFO:journalctl@ceph.mon.a.smithi164.stdout: self._sslobj.do_handshake() 2024-09-19T08:45:50.034 INFO:journalctl@ceph.mon.a.smithi164.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T08:45:50.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:49 smithi164 ceph-mon[30928]: [19/Sep/2024:08:45:49] ENGINE Serving on http://172.21.15.164:8765 2024-09-19T08:45:50.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:49 smithi164 ceph-mon[30928]: [19/Sep/2024:08:45:49] ENGINE Bus STARTED 2024-09-19T08:45:50.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:49 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2654565927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:45:50.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:49 smithi170 ceph-mon[35072]: mgrmap e22: x(active, since 1.01655s), standbys: z, y 2024-09-19T08:45:50.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:49 smithi170 ceph-mon[35072]: [19/Sep/2024:08:45:49] ENGINE Bus STARTING 2024-09-19T08:45:50.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:49 smithi170 ceph-mon[35072]: [19/Sep/2024:08:45:49] ENGINE Serving on https://172.21.15.164:7150 2024-09-19T08:45:50.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:49 smithi170 ceph-mon[35072]: [19/Sep/2024:08:45:49] ENGINE Error in HTTPServer.serve 2024-09-19T08:45:50.375 INFO:journalctl@ceph.mon.b.smithi170.stdout: Traceback (most recent call last): 2024-09-19T08:45:50.375 INFO:journalctl@ceph.mon.b.smithi170.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T08:45:50.375 INFO:journalctl@ceph.mon.b.smithi170.stdout: self._connections.run(self.expiration_interval) 2024-09-19T08:45:50.376 INFO:journalctl@ceph.mon.b.smithi170.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T08:45:50.376 INFO:journalctl@ceph.mon.b.smithi170.stdout: self._run(expiration_interval) 2024-09-19T08:45:50.376 INFO:journalctl@ceph.mon.b.smithi170.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T08:45:50.376 INFO:journalctl@ceph.mon.b.smithi170.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T08:45:50.376 INFO:journalctl@ceph.mon.b.smithi170.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T08:45:50.376 INFO:journalctl@ceph.mon.b.smithi170.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T08:45:50.377 INFO:journalctl@ceph.mon.b.smithi170.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T08:45:50.377 INFO:journalctl@ceph.mon.b.smithi170.stdout: s = self.context.wrap_socket( 2024-09-19T08:45:50.377 INFO:journalctl@ceph.mon.b.smithi170.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T08:45:50.377 INFO:journalctl@ceph.mon.b.smithi170.stdout: return self.sslsocket_class._create( 2024-09-19T08:45:50.377 INFO:journalctl@ceph.mon.b.smithi170.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T08:45:50.377 INFO:journalctl@ceph.mon.b.smithi170.stdout: self.do_handshake() 2024-09-19T08:45:50.377 INFO:journalctl@ceph.mon.b.smithi170.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T08:45:50.377 INFO:journalctl@ceph.mon.b.smithi170.stdout: self._sslobj.do_handshake() 2024-09-19T08:45:50.377 INFO:journalctl@ceph.mon.b.smithi170.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T08:45:50.377 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:49 smithi170 ceph-mon[35072]: [19/Sep/2024:08:45:49] ENGINE Serving on http://172.21.15.164:8765 2024-09-19T08:45:50.377 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:49 smithi170 ceph-mon[35072]: [19/Sep/2024:08:45:49] ENGINE Bus STARTED 2024-09-19T08:45:50.377 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:49 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2654565927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:45:50.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:49 smithi173 ceph-mon[35306]: mgrmap e22: x(active, since 1.01655s), standbys: z, y 2024-09-19T08:45:50.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:49 smithi173 ceph-mon[35306]: [19/Sep/2024:08:45:49] ENGINE Bus STARTING 2024-09-19T08:45:50.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:49 smithi173 ceph-mon[35306]: [19/Sep/2024:08:45:49] ENGINE Serving on https://172.21.15.164:7150 2024-09-19T08:45:50.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:49 smithi173 ceph-mon[35306]: [19/Sep/2024:08:45:49] ENGINE Error in HTTPServer.serve 2024-09-19T08:45:50.411 INFO:journalctl@ceph.mon.c.smithi173.stdout: Traceback (most recent call last): 2024-09-19T08:45:50.411 INFO:journalctl@ceph.mon.c.smithi173.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T08:45:50.411 INFO:journalctl@ceph.mon.c.smithi173.stdout: self._connections.run(self.expiration_interval) 2024-09-19T08:45:50.412 INFO:journalctl@ceph.mon.c.smithi173.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T08:45:50.412 INFO:journalctl@ceph.mon.c.smithi173.stdout: self._run(expiration_interval) 2024-09-19T08:45:50.412 INFO:journalctl@ceph.mon.c.smithi173.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T08:45:50.412 INFO:journalctl@ceph.mon.c.smithi173.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T08:45:50.412 INFO:journalctl@ceph.mon.c.smithi173.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T08:45:50.412 INFO:journalctl@ceph.mon.c.smithi173.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T08:45:50.412 INFO:journalctl@ceph.mon.c.smithi173.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T08:45:50.412 INFO:journalctl@ceph.mon.c.smithi173.stdout: s = self.context.wrap_socket( 2024-09-19T08:45:50.412 INFO:journalctl@ceph.mon.c.smithi173.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T08:45:50.412 INFO:journalctl@ceph.mon.c.smithi173.stdout: return self.sslsocket_class._create( 2024-09-19T08:45:50.412 INFO:journalctl@ceph.mon.c.smithi173.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T08:45:50.413 INFO:journalctl@ceph.mon.c.smithi173.stdout: self.do_handshake() 2024-09-19T08:45:50.413 INFO:journalctl@ceph.mon.c.smithi173.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T08:45:50.413 INFO:journalctl@ceph.mon.c.smithi173.stdout: self._sslobj.do_handshake() 2024-09-19T08:45:50.413 INFO:journalctl@ceph.mon.c.smithi173.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T08:45:50.413 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:49 smithi173 ceph-mon[35306]: [19/Sep/2024:08:45:49] ENGINE Serving on http://172.21.15.164:8765 2024-09-19T08:45:50.413 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:49 smithi173 ceph-mon[35306]: [19/Sep/2024:08:45:49] ENGINE Bus STARTED 2024-09-19T08:45:50.413 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:49 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2654565927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:45:50.773 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-19T08:45:50.773 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-19T08:45:50.774 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:45:50.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.880+0000 7f834d718640 1 -- 172.21.15.164:0/1756113441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8348100a30 msgr2=0x7f8348100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:50.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.880+0000 7f834d718640 1 --2- 172.21.15.164:0/1756113441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8348100a30 0x7f8348100e30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8340009ca0 tx=0x7f834002a8c0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.880+0000 7f834d718640 1 -- 172.21.15.164:0/1756113441 shutdown_connections 2024-09-19T08:45:50.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.880+0000 7f834d718640 1 --2- 172.21.15.164:0/1756113441 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83481025e0 0x7f8348108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.880+0000 7f834d718640 1 --2- 172.21.15.164:0/1756113441 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8348101c20 0x7f83481020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.880+0000 7f834d718640 1 --2- 172.21.15.164:0/1756113441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8348100a30 0x7f8348100e30 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.880+0000 7f834d718640 1 -- 172.21.15.164:0/1756113441 >> 172.21.15.164:0/1756113441 conn(0x7f83480fc1b0 msgr2=0x7f83480fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:50.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.880+0000 7f834d718640 1 -- 172.21.15.164:0/1756113441 shutdown_connections 2024-09-19T08:45:50.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.880+0000 7f834d718640 1 -- 172.21.15.164:0/1756113441 wait complete. 2024-09-19T08:45:50.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.880+0000 7f834d718640 1 Processor -- start 2024-09-19T08:45:50.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.881+0000 7f834d718640 1 -- start start 2024-09-19T08:45:50.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.881+0000 7f834d718640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8348100a30 0x7f834806ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:50.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.881+0000 7f834d718640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8348101c20 0x7f834819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:50.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.881+0000 7f834d718640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83481025e0 0x7f83481a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:50.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.881+0000 7f834d718640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f834810c6b0 con 0x7f8348101c20 2024-09-19T08:45:50.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.881+0000 7f834d718640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f834810c530 con 0x7f83481025e0 2024-09-19T08:45:50.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.881+0000 7f834d718640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f834810c830 con 0x7f8348100a30 2024-09-19T08:45:50.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.881+0000 7f83467fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8348101c20 0x7f834819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:50.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.881+0000 7f83467fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8348101c20 0x7f834819f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46536/0 (socket says 172.21.15.164:46536) 2024-09-19T08:45:50.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.881+0000 7f83467fc640 1 -- 172.21.15.164:0/3907043624 learned_addr learned my addr 172.21.15.164:0/3907043624 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:50.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.881+0000 7f8346ffd640 1 --2- 172.21.15.164:0/3907043624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8348100a30 0x7f834806ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:50.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.881+0000 7f83477fe640 1 --2- 172.21.15.164:0/3907043624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83481025e0 0x7f83481a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:50.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.881+0000 7f83467fc640 1 -- 172.21.15.164:0/3907043624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8348100a30 msgr2=0x7f834806ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:50.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.881+0000 7f83467fc640 1 --2- 172.21.15.164:0/3907043624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8348100a30 0x7f834806ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.881+0000 7f83467fc640 1 -- 172.21.15.164:0/3907043624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83481025e0 msgr2=0x7f83481a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:50.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.881+0000 7f83467fc640 1 --2- 172.21.15.164:0/3907043624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83481025e0 0x7f83481a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.881+0000 7f83467fc640 1 -- 172.21.15.164:0/3907043624 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83481a1d50 con 0x7f8348101c20 2024-09-19T08:45:50.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.881+0000 7f83477fe640 1 --2- 172.21.15.164:0/3907043624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83481025e0 0x7f83481a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:50.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.881+0000 7f83467fc640 1 --2- 172.21.15.164:0/3907043624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8348101c20 0x7f834819f010 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f833800e9f0 tx=0x7f833800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:50.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.882+0000 7f832ffff640 1 -- 172.21.15.164:0/3907043624 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833800cd80 con 0x7f8348101c20 2024-09-19T08:45:50.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.882+0000 7f832ffff640 1 -- 172.21.15.164:0/3907043624 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8338005500 con 0x7f8348101c20 2024-09-19T08:45:50.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.882+0000 7f834d718640 1 -- 172.21.15.164:0/3907043624 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83481a1fe0 con 0x7f8348101c20 2024-09-19T08:45:50.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.882+0000 7f832ffff640 1 -- 172.21.15.164:0/3907043624 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8338010690 con 0x7f8348101c20 2024-09-19T08:45:50.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.882+0000 7f834d718640 1 -- 172.21.15.164:0/3907043624 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83481a2450 con 0x7f8348101c20 2024-09-19T08:45:50.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.883+0000 7f834d718640 1 -- 172.21.15.164:0/3907043624 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8348107ee0 con 0x7f8348101c20 2024-09-19T08:45:50.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.884+0000 7f832ffff640 1 -- 172.21.15.164:0/3907043624 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 23) ==== 138476+0+0 (secure 0 0 0) 0x7f83380046c0 con 0x7f8348101c20 2024-09-19T08:45:50.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.885+0000 7f832ffff640 1 --2- 172.21.15.164:0/3907043624 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f83300a33f0 0x7f83300a58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:50.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.885+0000 7f832ffff640 1 -- 172.21.15.164:0/3907043624 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f83380d01a0 con 0x7f8348101c20 2024-09-19T08:45:50.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.885+0000 7f8346ffd640 1 --2- 172.21.15.164:0/3907043624 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f83300a33f0 0x7f83300a58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:50.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.886+0000 7f8346ffd640 1 --2- 172.21.15.164:0/3907043624 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f83300a33f0 0x7f83300a58b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8340006010 tx=0x7f8340009360 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:50.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:50.890+0000 7f832ffff640 1 -- 172.21.15.164:0/3907043624 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f833809cbf0 con 0x7f8348101c20 2024-09-19T08:45:51.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:50 smithi164 ceph-mon[30928]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:45:51.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:50 smithi164 ceph-mon[30928]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-09-19T08:45:51.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:50 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:51.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:50 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:51.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:50 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:51.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:50 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:51.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:50 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:51.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:50 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:51.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:50 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:51.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:50 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:51.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:50 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:51.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:50 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:51.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:50 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:51.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:50 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:51.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:50 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:51.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:50 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:51.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:50 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:51.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:50 smithi164 ceph-mon[30928]: Adjusting osd_memory_target on smithi170 to 176.0M 2024-09-19T08:45:51.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:50 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:51.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:50 smithi164 ceph-mon[30928]: Unable to set osd_memory_target on smithi170 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-19T08:45:51.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:50 smithi164 ceph-mon[30928]: Adjusting osd_memory_target on smithi173 to 176.0M 2024-09-19T08:45:51.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:50 smithi164 ceph-mon[30928]: Unable to set osd_memory_target on smithi173 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-19T08:45:51.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.088+0000 7f834d718640 1 -- 172.21.15.164:0/3907043624 --> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8348062630 con 0x7f83300a33f0 2024-09-19T08:45:51.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.095+0000 7f832ffff640 1 -- 172.21.15.164:0/3907043624 <== mgr.15201 v2:172.21.15.164:6800/173486514 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258041 (secure 0 0 0) 0x7f8348062630 con 0x7f83300a33f0 2024-09-19T08:45:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:45:51.100 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-09-19T08:45:51.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.098+0000 7f834d718640 1 -- 172.21.15.164:0/3907043624 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f83300a33f0 msgr2=0x7f83300a58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:51.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.098+0000 7f834d718640 1 --2- 172.21.15.164:0/3907043624 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f83300a33f0 0x7f83300a58b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8340006010 tx=0x7f8340009360 comp rx=0 tx=0).stop 2024-09-19T08:45:51.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.098+0000 7f834d718640 1 -- 172.21.15.164:0/3907043624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8348101c20 msgr2=0x7f834819f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:51.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.098+0000 7f834d718640 1 --2- 172.21.15.164:0/3907043624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8348101c20 0x7f834819f010 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f833800e9f0 tx=0x7f833800edf0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.098+0000 7f834d718640 1 -- 172.21.15.164:0/3907043624 shutdown_connections 2024-09-19T08:45:51.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.098+0000 7f834d718640 1 --2- 172.21.15.164:0/3907043624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83481025e0 0x7f83481a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.098+0000 7f834d718640 1 --2- 172.21.15.164:0/3907043624 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f83300a33f0 0x7f83300a58b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.098+0000 7f834d718640 1 --2- 172.21.15.164:0/3907043624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8348101c20 0x7f834819f010 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.098+0000 7f834d718640 1 --2- 172.21.15.164:0/3907043624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8348100a30 0x7f834806ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.098+0000 7f834d718640 1 -- 172.21.15.164:0/3907043624 >> 172.21.15.164:0/3907043624 conn(0x7f83480fc1b0 msgr2=0x7f83481050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:51.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.098+0000 7f834d718640 1 -- 172.21.15.164:0/3907043624 shutdown_connections 2024-09-19T08:45:51.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.098+0000 7f834d718640 1 -- 172.21.15.164:0/3907043624 wait complete. 2024-09-19T08:45:51.111 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-09-19T08:45:49.981301+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1190407,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1326,"num_read_kb":830,"num_write":557,"num_write_kb":4404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":940,"ondisk_log_size":940,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":389792,"kb_used_data":11376,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124552032,"statfs":{"total":1151940427776,"available":1151541280768,"internally_reserved":0,"allocated":11649024,"data_stored":8896335,"data_compressed":220029,"data_compressed_allocated":282624,"data_compressed_original":3416064,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.991213"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229715+0000","last_change":"2024-09-19T08:43:52.766709+0000","last_active":"2024-09-19T08:45:48.229715+0000","last_peered":"2024-09-19T08:45:48.229715+0000","last_clean":"2024-09-19T08:45:48.229715+0000","last_became_active":"2024-09-19T08:43:52.765970+0000","last_became_peered":"2024-09-19T08:43:52.765970+0000","last_unstale":"2024-09-19T08:45:48.229715+0000","last_undegraded":"2024-09-19T08:45:48.229715+0000","last_fullsized":"2024-09-19T08:45:48.229715+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:12.176870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.230605+0000","last_change":"2024-09-19T08:43:52.766590+0000","last_active":"2024-09-19T08:45:48.230605+0000","last_peered":"2024-09-19T08:45:48.230605+0000","last_clean":"2024-09-19T08:45:48.230605+0000","last_became_active":"2024-09-19T08:43:52.766255+0000","last_became_peered":"2024-09-19T08:43:52.766255+0000","last_unstale":"2024-09-19T08:45:48.230605+0000","last_undegraded":"2024-09-19T08:45:48.230605+0000","last_fullsized":"2024-09-19T08:45:48.230605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:15:41.359806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.230306+0000","last_change":"2024-09-19T08:43:52.761956+0000","last_active":"2024-09-19T08:45:48.230306+0000","last_peered":"2024-09-19T08:45:48.230306+0000","last_clean":"2024-09-19T08:45:48.230306+0000","last_became_active":"2024-09-19T08:43:52.761644+0000","last_became_peered":"2024-09-19T08:43:52.761644+0000","last_unstale":"2024-09-19T08:45:48.230306+0000","last_undegraded":"2024-09-19T08:45:48.230306+0000","last_fullsized":"2024-09-19T08:45:48.230306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:13.770361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.966146+0000","last_change":"2024-09-19T08:43:52.764131+0000","last_active":"2024-09-19T08:45:47.966146+0000","last_peered":"2024-09-19T08:45:47.966146+0000","last_clean":"2024-09-19T08:45:47.966146+0000","last_became_active":"2024-09-19T08:43:52.763812+0000","last_became_peered":"2024-09-19T08:43:52.763812+0000","last_unstale":"2024-09-19T08:45:47.966146+0000","last_undegraded":"2024-09-19T08:45:47.966146+0000","last_fullsized":"2024-09-19T08:45:47.966146+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:32.272157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065874+0000","last_change":"2024-09-19T08:43:52.754673+0000","last_active":"2024-09-19T08:45:48.065874+0000","last_peered":"2024-09-19T08:45:48.065874+0000","last_clean":"2024-09-19T08:45:48.065874+0000","last_became_active":"2024-09-19T08:43:52.754144+0000","last_became_peered":"2024-09-19T08:43:52.754144+0000","last_unstale":"2024-09-19T08:45:48.065874+0000","last_undegraded":"2024-09-19T08:45:48.065874+0000","last_fullsized":"2024-09-19T08:45:48.065874+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:16.069260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.076773+0000","last_change":"2024-09-19T08:43:52.758939+0000","last_active":"2024-09-19T08:45:48.076773+0000","last_peered":"2024-09-19T08:45:48.076773+0000","last_clean":"2024-09-19T08:45:48.076773+0000","last_became_active":"2024-09-19T08:43:52.758611+0000","last_became_peered":"2024-09-19T08:43:52.758611+0000","last_unstale":"2024-09-19T08:45:48.076773+0000","last_undegraded":"2024-09-19T08:45:48.076773+0000","last_fullsized":"2024-09-19T08:45:48.076773+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:35:56.463545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221260+0000","last_change":"2024-09-19T08:43:52.770261+0000","last_active":"2024-09-19T08:45:48.221260+0000","last_peered":"2024-09-19T08:45:48.221260+0000","last_clean":"2024-09-19T08:45:48.221260+0000","last_became_active":"2024-09-19T08:43:52.768253+0000","last_became_peered":"2024-09-19T08:43:52.768253+0000","last_unstale":"2024-09-19T08:45:48.221260+0000","last_undegraded":"2024-09-19T08:45:48.221260+0000","last_fullsized":"2024-09-19T08:45:48.221260+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:40.756988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065907+0000","last_change":"2024-09-19T08:43:52.768564+0000","last_active":"2024-09-19T08:45:48.065907+0000","last_peered":"2024-09-19T08:45:48.065907+0000","last_clean":"2024-09-19T08:45:48.065907+0000","last_became_active":"2024-09-19T08:43:52.768260+0000","last_became_peered":"2024-09-19T08:43:52.768260+0000","last_unstale":"2024-09-19T08:45:48.065907+0000","last_undegraded":"2024-09-19T08:45:48.065907+0000","last_fullsized":"2024-09-19T08:45:48.065907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:16.528115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.222053+0000","last_change":"2024-09-19T08:43:52.769804+0000","last_active":"2024-09-19T08:45:48.222053+0000","last_peered":"2024-09-19T08:45:48.222053+0000","last_clean":"2024-09-19T08:45:48.222053+0000","last_became_active":"2024-09-19T08:43:52.769504+0000","last_became_peered":"2024-09-19T08:43:52.769504+0000","last_unstale":"2024-09-19T08:45:48.222053+0000","last_undegraded":"2024-09-19T08:45:48.222053+0000","last_fullsized":"2024-09-19T08:45:48.222053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:22:38.514410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.077170+0000","last_change":"2024-09-19T08:43:52.763278+0000","last_active":"2024-09-19T08:45:48.077170+0000","last_peered":"2024-09-19T08:45:48.077170+0000","last_clean":"2024-09-19T08:45:48.077170+0000","last_became_active":"2024-09-19T08:43:52.762985+0000","last_became_peered":"2024-09-19T08:43:52.762985+0000","last_unstale":"2024-09-19T08:45:48.077170+0000","last_undegraded":"2024-09-19T08:45:48.077170+0000","last_fullsized":"2024-09-19T08:45:48.077170+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:30.048782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221488+0000","last_change":"2024-09-19T08:43:52.770864+0000","last_active":"2024-09-19T08:45:48.221488+0000","last_peered":"2024-09-19T08:45:48.221488+0000","last_clean":"2024-09-19T08:45:48.221488+0000","last_became_active":"2024-09-19T08:43:52.770518+0000","last_became_peered":"2024-09-19T08:43:52.770518+0000","last_unstale":"2024-09-19T08:45:48.221488+0000","last_undegraded":"2024-09-19T08:45:48.221488+0000","last_fullsized":"2024-09-19T08:45:48.221488+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:08.931452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.966171+0000","last_change":"2024-09-19T08:43:52.757464+0000","last_active":"2024-09-19T08:45:47.966171+0000","last_peered":"2024-09-19T08:45:47.966171+0000","last_clean":"2024-09-19T08:45:47.966171+0000","last_became_active":"2024-09-19T08:43:52.757101+0000","last_became_peered":"2024-09-19T08:43:52.757101+0000","last_unstale":"2024-09-19T08:45:47.966171+0000","last_undegraded":"2024-09-19T08:45:47.966171+0000","last_fullsized":"2024-09-19T08:45:47.966171+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:17.395735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.224010+0000","last_change":"2024-09-19T08:43:52.767063+0000","last_active":"2024-09-19T08:45:48.224010+0000","last_peered":"2024-09-19T08:45:48.224010+0000","last_clean":"2024-09-19T08:45:48.224010+0000","last_became_active":"2024-09-19T08:43:52.766755+0000","last_became_peered":"2024-09-19T08:43:52.766755+0000","last_unstale":"2024-09-19T08:45:48.224010+0000","last_undegraded":"2024-09-19T08:45:48.224010+0000","last_fullsized":"2024-09-19T08:45:48.224010+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:30.530937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.230150+0000","last_change":"2024-09-19T08:43:52.766515+0000","last_active":"2024-09-19T08:45:48.230150+0000","last_peered":"2024-09-19T08:45:48.230150+0000","last_clean":"2024-09-19T08:45:48.230150+0000","last_became_active":"2024-09-19T08:43:52.765847+0000","last_became_peered":"2024-09-19T08:43:52.765847+0000","last_unstale":"2024-09-19T08:45:48.230150+0000","last_undegraded":"2024-09-19T08:45:48.230150+0000","last_fullsized":"2024-09-19T08:45:48.230150+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:50:35.948663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229847+0000","last_change":"2024-09-19T08:43:52.773702+0000","last_active":"2024-09-19T08:45:48.229847+0000","last_peered":"2024-09-19T08:45:48.229847+0000","last_clean":"2024-09-19T08:45:48.229847+0000","last_became_active":"2024-09-19T08:43:52.773324+0000","last_became_peered":"2024-09-19T08:43:52.773324+0000","last_unstale":"2024-09-19T08:45:48.229847+0000","last_undegraded":"2024-09-19T08:45:48.229847+0000","last_fullsized":"2024-09-19T08:45:48.229847+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:34.368014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221253+0000","last_change":"2024-09-19T08:43:52.762745+0000","last_active":"2024-09-19T08:45:48.221253+0000","last_peered":"2024-09-19T08:45:48.221253+0000","last_clean":"2024-09-19T08:45:48.221253+0000","last_became_active":"2024-09-19T08:43:52.762172+0000","last_became_peered":"2024-09-19T08:43:52.762172+0000","last_unstale":"2024-09-19T08:45:48.221253+0000","last_undegraded":"2024-09-19T08:45:48.221253+0000","last_fullsized":"2024-09-19T08:45:48.221253+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:57.369775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229743+0000","last_change":"2024-09-19T08:43:52.767118+0000","last_active":"2024-09-19T08:45:48.229743+0000","last_peered":"2024-09-19T08:45:48.229743+0000","last_clean":"2024-09-19T08:45:48.229743+0000","last_became_active":"2024-09-19T08:43:52.766388+0000","last_became_peered":"2024-09-19T08:43:52.766388+0000","last_unstale":"2024-09-19T08:45:48.229743+0000","last_undegraded":"2024-09-19T08:45:48.229743+0000","last_fullsized":"2024-09-19T08:45:48.229743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:34.182923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.967257+0000","last_change":"2024-09-19T08:43:52.765008+0000","last_active":"2024-09-19T08:45:47.967257+0000","last_peered":"2024-09-19T08:45:47.967257+0000","last_clean":"2024-09-19T08:45:47.967257+0000","last_became_active":"2024-09-19T08:43:52.764558+0000","last_became_peered":"2024-09-19T08:43:52.764558+0000","last_unstale":"2024-09-19T08:45:47.967257+0000","last_undegraded":"2024-09-19T08:45:47.967257+0000","last_fullsized":"2024-09-19T08:45:47.967257+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:52:42.756421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968722+0000","last_change":"2024-09-19T08:43:52.764376+0000","last_active":"2024-09-19T08:45:47.968722+0000","last_peered":"2024-09-19T08:45:47.968722+0000","last_clean":"2024-09-19T08:45:47.968722+0000","last_became_active":"2024-09-19T08:43:52.764048+0000","last_became_peered":"2024-09-19T08:43:52.764048+0000","last_unstale":"2024-09-19T08:45:47.968722+0000","last_undegraded":"2024-09-19T08:45:47.968722+0000","last_fullsized":"2024-09-19T08:45:47.968722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:56.732094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229417+0000","last_change":"2024-09-19T08:44:02.827925+0000","last_active":"2024-09-19T08:45:48.229417+0000","last_peered":"2024-09-19T08:45:48.229417+0000","last_clean":"2024-09-19T08:45:48.229417+0000","last_became_active":"2024-09-19T08:44:02.827269+0000","last_became_peered":"2024-09-19T08:44:02.827269+0000","last_unstale":"2024-09-19T08:45:48.229417+0000","last_undegraded":"2024-09-19T08:45:48.229417+0000","last_fullsized":"2024-09-19T08:45:48.229417+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:31:54.556786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.224157+0000","last_change":"2024-09-19T08:43:52.762190+0000","last_active":"2024-09-19T08:45:48.224157+0000","last_peered":"2024-09-19T08:45:48.224157+0000","last_clean":"2024-09-19T08:45:48.224157+0000","last_became_active":"2024-09-19T08:43:52.761818+0000","last_became_peered":"2024-09-19T08:43:52.761818+0000","last_unstale":"2024-09-19T08:45:48.224157+0000","last_undegraded":"2024-09-19T08:45:48.224157+0000","last_fullsized":"2024-09-19T08:45:48.224157+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:19:55.246779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221492+0000","last_change":"2024-09-19T08:43:52.761069+0000","last_active":"2024-09-19T08:45:48.221492+0000","last_peered":"2024-09-19T08:45:48.221492+0000","last_clean":"2024-09-19T08:45:48.221492+0000","last_became_active":"2024-09-19T08:43:52.760705+0000","last_became_peered":"2024-09-19T08:43:52.760705+0000","last_unstale":"2024-09-19T08:45:48.221492+0000","last_undegraded":"2024-09-19T08:45:48.221492+0000","last_fullsized":"2024-09-19T08:45:48.221492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:46:27.059026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065963+0000","last_change":"2024-09-19T08:43:52.773449+0000","last_active":"2024-09-19T08:45:48.065963+0000","last_peered":"2024-09-19T08:45:48.065963+0000","last_clean":"2024-09-19T08:45:48.065963+0000","last_became_active":"2024-09-19T08:43:52.772929+0000","last_became_peered":"2024-09-19T08:43:52.772929+0000","last_unstale":"2024-09-19T08:45:48.065963+0000","last_undegraded":"2024-09-19T08:45:48.065963+0000","last_fullsized":"2024-09-19T08:45:48.065963+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:00:38.929199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074018+0000","last_change":"2024-09-19T08:43:52.773885+0000","last_active":"2024-09-19T08:45:48.074018+0000","last_peered":"2024-09-19T08:45:48.074018+0000","last_clean":"2024-09-19T08:45:48.074018+0000","last_became_active":"2024-09-19T08:43:52.772856+0000","last_became_peered":"2024-09-19T08:43:52.772856+0000","last_unstale":"2024-09-19T08:45:48.074018+0000","last_undegraded":"2024-09-19T08:45:48.074018+0000","last_fullsized":"2024-09-19T08:45:48.074018+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:13:22.581335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.067279+0000","last_change":"2024-09-19T08:43:52.762909+0000","last_active":"2024-09-19T08:45:48.067279+0000","last_peered":"2024-09-19T08:45:48.067279+0000","last_clean":"2024-09-19T08:45:48.067279+0000","last_became_active":"2024-09-19T08:43:52.762600+0000","last_became_peered":"2024-09-19T08:43:52.762600+0000","last_unstale":"2024-09-19T08:45:48.067279+0000","last_undegraded":"2024-09-19T08:45:48.067279+0000","last_fullsized":"2024-09-19T08:45:48.067279+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:46.098294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968939+0000","last_change":"2024-09-19T08:43:52.762861+0000","last_active":"2024-09-19T08:45:47.968939+0000","last_peered":"2024-09-19T08:45:47.968939+0000","last_clean":"2024-09-19T08:45:47.968939+0000","last_became_active":"2024-09-19T08:43:52.762525+0000","last_became_peered":"2024-09-19T08:43:52.762525+0000","last_unstale":"2024-09-19T08:45:47.968939+0000","last_undegraded":"2024-09-19T08:45:47.968939+0000","last_fullsized":"2024-09-19T08:45:47.968939+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:54.717249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229552+0000","last_change":"2024-09-19T08:43:52.768370+0000","last_active":"2024-09-19T08:45:48.229552+0000","last_peered":"2024-09-19T08:45:48.229552+0000","last_clean":"2024-09-19T08:45:48.229552+0000","last_became_active":"2024-09-19T08:43:52.768055+0000","last_became_peered":"2024-09-19T08:43:52.768055+0000","last_unstale":"2024-09-19T08:45:48.229552+0000","last_undegraded":"2024-09-19T08:45:48.229552+0000","last_fullsized":"2024-09-19T08:45:48.229552+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:48.212862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.224066+0000","last_change":"2024-09-19T08:43:52.758541+0000","last_active":"2024-09-19T08:45:48.224066+0000","last_peered":"2024-09-19T08:45:48.224066+0000","last_clean":"2024-09-19T08:45:48.224066+0000","last_became_active":"2024-09-19T08:43:52.758122+0000","last_became_peered":"2024-09-19T08:43:52.758122+0000","last_unstale":"2024-09-19T08:45:48.224066+0000","last_undegraded":"2024-09-19T08:45:48.224066+0000","last_fullsized":"2024-09-19T08:45:48.224066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:54:40.681880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.231790+0000","last_change":"2024-09-19T08:43:52.758109+0000","last_active":"2024-09-19T08:45:48.231790+0000","last_peered":"2024-09-19T08:45:48.231790+0000","last_clean":"2024-09-19T08:45:48.231790+0000","last_became_active":"2024-09-19T08:43:52.757808+0000","last_became_peered":"2024-09-19T08:43:52.757808+0000","last_unstale":"2024-09-19T08:45:48.231790+0000","last_undegraded":"2024-09-19T08:45:48.231790+0000","last_fullsized":"2024-09-19T08:45:48.231790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:02:47.614562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968701+0000","last_change":"2024-09-19T08:43:52.772501+0000","last_active":"2024-09-19T08:45:47.968701+0000","last_peered":"2024-09-19T08:45:47.968701+0000","last_clean":"2024-09-19T08:45:47.968701+0000","last_became_active":"2024-09-19T08:43:52.771611+0000","last_became_peered":"2024-09-19T08:43:52.771611+0000","last_unstale":"2024-09-19T08:45:47.968701+0000","last_undegraded":"2024-09-19T08:45:47.968701+0000","last_fullsized":"2024-09-19T08:45:47.968701+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:49.537237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968630+0000","last_change":"2024-09-19T08:43:52.773664+0000","last_active":"2024-09-19T08:45:47.968630+0000","last_peered":"2024-09-19T08:45:47.968630+0000","last_clean":"2024-09-19T08:45:47.968630+0000","last_became_active":"2024-09-19T08:43:52.773014+0000","last_became_peered":"2024-09-19T08:43:52.773014+0000","last_unstale":"2024-09-19T08:45:47.968630+0000","last_undegraded":"2024-09-19T08:45:47.968630+0000","last_fullsized":"2024-09-19T08:45:47.968630+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:03.448825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229497+0000","last_change":"2024-09-19T08:43:52.769578+0000","last_active":"2024-09-19T08:45:48.229497+0000","last_peered":"2024-09-19T08:45:48.229497+0000","last_clean":"2024-09-19T08:45:48.229497+0000","last_became_active":"2024-09-19T08:43:52.769252+0000","last_became_peered":"2024-09-19T08:43:52.769252+0000","last_unstale":"2024-09-19T08:45:48.229497+0000","last_undegraded":"2024-09-19T08:45:48.229497+0000","last_fullsized":"2024-09-19T08:45:48.229497+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:27.936118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065844+0000","last_change":"2024-09-19T08:43:52.770453+0000","last_active":"2024-09-19T08:45:48.065844+0000","last_peered":"2024-09-19T08:45:48.065844+0000","last_clean":"2024-09-19T08:45:48.065844+0000","last_became_active":"2024-09-19T08:43:52.770155+0000","last_became_peered":"2024-09-19T08:43:52.770155+0000","last_unstale":"2024-09-19T08:45:48.065844+0000","last_undegraded":"2024-09-19T08:45:48.065844+0000","last_fullsized":"2024-09-19T08:45:48.065844+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:12.893572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'8","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.782770+0000","last_change":"2024-09-19T08:43:50.751316+0000","last_active":"2024-09-19T08:45:48.782770+0000","last_peered":"2024-09-19T08:45:48.782770+0000","last_clean":"2024-09-19T08:45:48.782770+0000","last_became_active":"2024-09-19T08:43:50.750972+0000","last_became_peered":"2024-09-19T08:43:50.750972+0000","last_unstale":"2024-09-19T08:45:48.782770+0000","last_undegraded":"2024-09-19T08:45:48.782770+0000","last_fullsized":"2024-09-19T08:45:48.782770+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:40.654729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1549,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074105+0000","last_change":"2024-09-19T08:43:52.773979+0000","last_active":"2024-09-19T08:45:48.074105+0000","last_peered":"2024-09-19T08:45:48.074105+0000","last_clean":"2024-09-19T08:45:48.074105+0000","last_became_active":"2024-09-19T08:43:52.773648+0000","last_became_peered":"2024-09-19T08:43:52.773648+0000","last_unstale":"2024-09-19T08:45:48.074105+0000","last_undegraded":"2024-09-19T08:45:48.074105+0000","last_fullsized":"2024-09-19T08:45:48.074105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:36.061209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.076810+0000","last_change":"2024-09-19T08:43:50.750704+0000","last_active":"2024-09-19T08:45:48.076810+0000","last_peered":"2024-09-19T08:45:48.076810+0000","last_clean":"2024-09-19T08:45:48.076810+0000","last_became_active":"2024-09-19T08:43:50.750399+0000","last_became_peered":"2024-09-19T08:43:50.750399+0000","last_unstale":"2024-09-19T08:45:48.076810+0000","last_undegraded":"2024-09-19T08:45:48.076810+0000","last_fullsized":"2024-09-19T08:45:48.076810+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:25.447320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.223615+0000","last_change":"2024-09-19T08:43:50.744919+0000","last_active":"2024-09-19T08:45:48.223615+0000","last_peered":"2024-09-19T08:45:48.223615+0000","last_clean":"2024-09-19T08:45:48.223615+0000","last_became_active":"2024-09-19T08:43:50.744604+0000","last_became_peered":"2024-09-19T08:43:50.744604+0000","last_unstale":"2024-09-19T08:45:48.223615+0000","last_undegraded":"2024-09-19T08:45:48.223615+0000","last_fullsized":"2024-09-19T08:45:48.223615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:25:06.410093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065913+0000","last_change":"2024-09-19T08:43:52.766779+0000","last_active":"2024-09-19T08:45:48.065913+0000","last_peered":"2024-09-19T08:45:48.065913+0000","last_clean":"2024-09-19T08:45:48.065913+0000","last_became_active":"2024-09-19T08:43:52.766484+0000","last_became_peered":"2024-09-19T08:43:52.766484+0000","last_unstale":"2024-09-19T08:45:48.065913+0000","last_undegraded":"2024-09-19T08:45:48.065913+0000","last_fullsized":"2024-09-19T08:45:48.065913+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:17:29.350778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074289+0000","last_change":"2024-09-19T08:43:50.751282+0000","last_active":"2024-09-19T08:45:48.074289+0000","last_peered":"2024-09-19T08:45:48.074289+0000","last_clean":"2024-09-19T08:45:48.074289+0000","last_became_active":"2024-09-19T08:43:50.750754+0000","last_became_peered":"2024-09-19T08:43:50.750754+0000","last_unstale":"2024-09-19T08:45:48.074289+0000","last_undegraded":"2024-09-19T08:45:48.074289+0000","last_fullsized":"2024-09-19T08:45:48.074289+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:11.661626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.967219+0000","last_change":"2024-09-19T08:43:52.756442+0000","last_active":"2024-09-19T08:45:47.967219+0000","last_peered":"2024-09-19T08:45:47.967219+0000","last_clean":"2024-09-19T08:45:47.967219+0000","last_became_active":"2024-09-19T08:43:52.756116+0000","last_became_peered":"2024-09-19T08:43:52.756116+0000","last_unstale":"2024-09-19T08:45:47.967219+0000","last_undegraded":"2024-09-19T08:45:47.967219+0000","last_fullsized":"2024-09-19T08:45:47.967219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:40.029272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'8","reported_seq":242,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.782133+0000","last_change":"2024-09-19T08:43:50.749637+0000","last_active":"2024-09-19T08:45:48.782133+0000","last_peered":"2024-09-19T08:45:48.782133+0000","last_clean":"2024-09-19T08:45:48.782133+0000","last_became_active":"2024-09-19T08:43:50.749307+0000","last_became_peered":"2024-09-19T08:43:50.749307+0000","last_unstale":"2024-09-19T08:45:48.782133+0000","last_undegraded":"2024-09-19T08:45:48.782133+0000","last_fullsized":"2024-09-19T08:45:48.782133+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:23:33.409671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":210,"num_read_kb":210,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.222016+0000","last_change":"2024-09-19T08:43:52.763592+0000","last_active":"2024-09-19T08:45:48.222016+0000","last_peered":"2024-09-19T08:45:48.222016+0000","last_clean":"2024-09-19T08:45:48.222016+0000","last_became_active":"2024-09-19T08:43:52.763146+0000","last_became_peered":"2024-09-19T08:43:52.763146+0000","last_unstale":"2024-09-19T08:45:48.222016+0000","last_undegraded":"2024-09-19T08:45:48.222016+0000","last_fullsized":"2024-09-19T08:45:48.222016+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:32.194465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221748+0000","last_change":"2024-09-19T08:43:50.749184+0000","last_active":"2024-09-19T08:45:48.221748+0000","last_peered":"2024-09-19T08:45:48.221748+0000","last_clean":"2024-09-19T08:45:48.221748+0000","last_became_active":"2024-09-19T08:43:50.748844+0000","last_became_peered":"2024-09-19T08:43:50.748844+0000","last_unstale":"2024-09-19T08:45:48.221748+0000","last_undegraded":"2024-09-19T08:45:48.221748+0000","last_fullsized":"2024-09-19T08:45:48.221748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:15.334759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.231209+0000","last_change":"2024-09-19T08:43:52.768696+0000","last_active":"2024-09-19T08:45:48.231209+0000","last_peered":"2024-09-19T08:45:48.231209+0000","last_clean":"2024-09-19T08:45:48.231209+0000","last_became_active":"2024-09-19T08:43:52.768374+0000","last_became_peered":"2024-09-19T08:43:52.768374+0000","last_unstale":"2024-09-19T08:45:48.231209+0000","last_undegraded":"2024-09-19T08:45:48.231209+0000","last_fullsized":"2024-09-19T08:45:48.231209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:34:39.343041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074342+0000","last_change":"2024-09-19T08:43:51.026270+0000","last_active":"2024-09-19T08:45:48.074342+0000","last_peered":"2024-09-19T08:45:48.074342+0000","last_clean":"2024-09-19T08:45:48.074342+0000","last_became_active":"2024-09-19T08:43:51.025823+0000","last_became_peered":"2024-09-19T08:43:51.025823+0000","last_unstale":"2024-09-19T08:45:48.074342+0000","last_undegraded":"2024-09-19T08:45:48.074342+0000","last_fullsized":"2024-09-19T08:45:48.074342+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:00.480438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221712+0000","last_change":"2024-09-19T08:43:52.759555+0000","last_active":"2024-09-19T08:45:48.221712+0000","last_peered":"2024-09-19T08:45:48.221712+0000","last_clean":"2024-09-19T08:45:48.221712+0000","last_became_active":"2024-09-19T08:43:52.759223+0000","last_became_peered":"2024-09-19T08:43:52.759223+0000","last_unstale":"2024-09-19T08:45:48.221712+0000","last_undegraded":"2024-09-19T08:45:48.221712+0000","last_fullsized":"2024-09-19T08:45:48.221712+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:57:06.451323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'36","reported_seq":283,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.782998+0000","last_change":"2024-09-19T08:43:50.751924+0000","last_active":"2024-09-19T08:45:48.782998+0000","last_peered":"2024-09-19T08:45:48.782998+0000","last_clean":"2024-09-19T08:45:48.782998+0000","last_became_active":"2024-09-19T08:43:50.751117+0000","last_became_peered":"2024-09-19T08:43:50.751117+0000","last_unstale":"2024-09-19T08:45:48.782998+0000","last_undegraded":"2024-09-19T08:45:48.782998+0000","last_fullsized":"2024-09-19T08:45:48.782998+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:33:57.196283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65927,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":223,"num_read_kb":176,"num_write":38,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066156+0000","last_change":"2024-09-19T08:43:52.762260+0000","last_active":"2024-09-19T08:45:48.066156+0000","last_peered":"2024-09-19T08:45:48.066156+0000","last_clean":"2024-09-19T08:45:48.066156+0000","last_became_active":"2024-09-19T08:43:52.761923+0000","last_became_peered":"2024-09-19T08:43:52.761923+0000","last_unstale":"2024-09-19T08:45:48.066156+0000","last_undegraded":"2024-09-19T08:45:48.066156+0000","last_fullsized":"2024-09-19T08:45:48.066156+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:39:39.503332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968758+0000","last_change":"2024-09-19T08:43:50.749745+0000","last_active":"2024-09-19T08:45:47.968758+0000","last_peered":"2024-09-19T08:45:47.968758+0000","last_clean":"2024-09-19T08:45:47.968758+0000","last_became_active":"2024-09-19T08:43:50.749073+0000","last_became_peered":"2024-09-19T08:43:50.749073+0000","last_unstale":"2024-09-19T08:45:47.968758+0000","last_undegraded":"2024-09-19T08:45:47.968758+0000","last_fullsized":"2024-09-19T08:45:47.968758+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:48:11.965841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.223734+0000","last_change":"2024-09-19T08:43:52.761495+0000","last_active":"2024-09-19T08:45:48.223734+0000","last_peered":"2024-09-19T08:45:48.223734+0000","last_clean":"2024-09-19T08:45:48.223734+0000","last_became_active":"2024-09-19T08:43:52.760425+0000","last_became_peered":"2024-09-19T08:43:52.760425+0000","last_unstale":"2024-09-19T08:45:48.223734+0000","last_undegraded":"2024-09-19T08:45:48.223734+0000","last_fullsized":"2024-09-19T08:45:48.223734+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:57.689284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074090+0000","last_change":"2024-09-19T08:43:50.749563+0000","last_active":"2024-09-19T08:45:48.074090+0000","last_peered":"2024-09-19T08:45:48.074090+0000","last_clean":"2024-09-19T08:45:48.074090+0000","last_became_active":"2024-09-19T08:43:50.749244+0000","last_became_peered":"2024-09-19T08:43:50.749244+0000","last_unstale":"2024-09-19T08:45:48.074090+0000","last_undegraded":"2024-09-19T08:45:48.074090+0000","last_fullsized":"2024-09-19T08:45:48.074090+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:59:02.312883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221429+0000","last_change":"2024-09-19T08:43:52.767087+0000","last_active":"2024-09-19T08:45:48.221429+0000","last_peered":"2024-09-19T08:45:48.221429+0000","last_clean":"2024-09-19T08:45:48.221429+0000","last_became_active":"2024-09-19T08:43:52.766730+0000","last_became_peered":"2024-09-19T08:43:52.766730+0000","last_unstale":"2024-09-19T08:45:48.221429+0000","last_undegraded":"2024-09-19T08:45:48.221429+0000","last_fullsized":"2024-09-19T08:45:48.221429+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:04.040896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.966237+0000","last_change":"2024-09-19T08:43:50.749659+0000","last_active":"2024-09-19T08:45:47.966237+0000","last_peered":"2024-09-19T08:45:47.966237+0000","last_clean":"2024-09-19T08:45:47.966237+0000","last_became_active":"2024-09-19T08:43:50.749314+0000","last_became_peered":"2024-09-19T08:43:50.749314+0000","last_unstale":"2024-09-19T08:45:47.966237+0000","last_undegraded":"2024-09-19T08:45:47.966237+0000","last_fullsized":"2024-09-19T08:45:47.966237+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:08.576209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221691+0000","last_change":"2024-09-19T08:43:52.767928+0000","last_active":"2024-09-19T08:45:48.221691+0000","last_peered":"2024-09-19T08:45:48.221691+0000","last_clean":"2024-09-19T08:45:48.221691+0000","last_became_active":"2024-09-19T08:43:52.767580+0000","last_became_peered":"2024-09-19T08:43:52.767580+0000","last_unstale":"2024-09-19T08:45:48.221691+0000","last_undegraded":"2024-09-19T08:45:48.221691+0000","last_fullsized":"2024-09-19T08:45:48.221691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:45.344693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229348+0000","last_change":"2024-09-19T08:44:02.833887+0000","last_active":"2024-09-19T08:45:48.229348+0000","last_peered":"2024-09-19T08:45:48.229348+0000","last_clean":"2024-09-19T08:45:48.229348+0000","last_became_active":"2024-09-19T08:44:02.821624+0000","last_became_peered":"2024-09-19T08:44:02.821624+0000","last_unstale":"2024-09-19T08:45:48.229348+0000","last_undegraded":"2024-09-19T08:45:48.229348+0000","last_fullsized":"2024-09-19T08:45:48.229348+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:54:36.131949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.967019+0000","last_change":"2024-09-19T08:44:03.080503+0000","last_active":"2024-09-19T08:45:47.967019+0000","last_peered":"2024-09-19T08:45:47.967019+0000","last_clean":"2024-09-19T08:45:47.967019+0000","last_became_active":"2024-09-19T08:44:03.080129+0000","last_became_peered":"2024-09-19T08:44:03.080129+0000","last_unstale":"2024-09-19T08:45:47.967019+0000","last_undegraded":"2024-09-19T08:45:47.967019+0000","last_fullsized":"2024-09-19T08:45:47.967019+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:01:37.034795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066192+0000","last_change":"2024-09-19T08:43:50.747624+0000","last_active":"2024-09-19T08:45:48.066192+0000","last_peered":"2024-09-19T08:45:48.066192+0000","last_clean":"2024-09-19T08:45:48.066192+0000","last_became_active":"2024-09-19T08:43:50.747263+0000","last_became_peered":"2024-09-19T08:43:50.747263+0000","last_unstale":"2024-09-19T08:45:48.066192+0000","last_undegraded":"2024-09-19T08:45:48.066192+0000","last_fullsized":"2024-09-19T08:45:48.066192+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:18.580106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229525+0000","last_change":"2024-09-19T08:43:52.770051+0000","last_active":"2024-09-19T08:45:48.229525+0000","last_peered":"2024-09-19T08:45:48.229525+0000","last_clean":"2024-09-19T08:45:48.229525+0000","last_became_active":"2024-09-19T08:43:52.769727+0000","last_became_peered":"2024-09-19T08:43:52.769727+0000","last_unstale":"2024-09-19T08:45:48.229525+0000","last_undegraded":"2024-09-19T08:45:48.229525+0000","last_fullsized":"2024-09-19T08:45:48.229525+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:34.954897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.223984+0000","last_change":"2024-09-19T08:43:50.746696+0000","last_active":"2024-09-19T08:45:48.223984+0000","last_peered":"2024-09-19T08:45:48.223984+0000","last_clean":"2024-09-19T08:45:48.223984+0000","last_became_active":"2024-09-19T08:43:50.746246+0000","last_became_peered":"2024-09-19T08:43:50.746246+0000","last_unstale":"2024-09-19T08:45:48.223984+0000","last_undegraded":"2024-09-19T08:45:48.223984+0000","last_fullsized":"2024-09-19T08:45:48.223984+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:38:39.935348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.222174+0000","last_change":"2024-09-19T08:43:52.765199+0000","last_active":"2024-09-19T08:45:48.222174+0000","last_peered":"2024-09-19T08:45:48.222174+0000","last_clean":"2024-09-19T08:45:48.222174+0000","last_became_active":"2024-09-19T08:43:52.764896+0000","last_became_peered":"2024-09-19T08:43:52.764896+0000","last_unstale":"2024-09-19T08:45:48.222174+0000","last_undegraded":"2024-09-19T08:45:48.222174+0000","last_fullsized":"2024-09-19T08:45:48.222174+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:02:59.570018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'41","reported_seq":73,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.186388+0000","last_change":"2024-09-19T08:43:50.747242+0000","last_active":"2024-09-19T08:45:48.186388+0000","last_peered":"2024-09-19T08:45:48.186388+0000","last_clean":"2024-09-19T08:45:48.186388+0000","last_became_active":"2024-09-19T08:43:50.746853+0000","last_became_peered":"2024-09-19T08:43:50.746853+0000","last_unstale":"2024-09-19T08:45:48.186388+0000","last_undegraded":"2024-09-19T08:45:48.186388+0000","last_fullsized":"2024-09-19T08:45:48.186388+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:21.545635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221579+0000","last_change":"2024-09-19T08:44:02.826535+0000","last_active":"2024-09-19T08:45:48.221579+0000","last_peered":"2024-09-19T08:45:48.221579+0000","last_clean":"2024-09-19T08:45:48.221579+0000","last_became_active":"2024-09-19T08:44:02.826113+0000","last_became_peered":"2024-09-19T08:44:02.826113+0000","last_unstale":"2024-09-19T08:45:48.221579+0000","last_undegraded":"2024-09-19T08:45:48.221579+0000","last_fullsized":"2024-09-19T08:45:48.221579+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:20.247732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'8","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.782660+0000","last_change":"2024-09-19T08:43:51.131540+0000","last_active":"2024-09-19T08:45:48.782660+0000","last_peered":"2024-09-19T08:45:48.782660+0000","last_clean":"2024-09-19T08:45:48.782660+0000","last_became_active":"2024-09-19T08:43:51.131170+0000","last_became_peered":"2024-09-19T08:43:51.131170+0000","last_unstale":"2024-09-19T08:45:48.782660+0000","last_undegraded":"2024-09-19T08:45:48.782660+0000","last_fullsized":"2024-09-19T08:45:48.782660+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:43.635612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221593+0000","last_change":"2024-09-19T08:43:52.763308+0000","last_active":"2024-09-19T08:45:48.221593+0000","last_peered":"2024-09-19T08:45:48.221593+0000","last_clean":"2024-09-19T08:45:48.221593+0000","last_became_active":"2024-09-19T08:43:52.762859+0000","last_became_peered":"2024-09-19T08:43:52.762859+0000","last_unstale":"2024-09-19T08:45:48.221593+0000","last_undegraded":"2024-09-19T08:45:48.221593+0000","last_fullsized":"2024-09-19T08:45:48.221593+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:32:35.083361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":350,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221461+0000","last_change":"2024-09-19T08:42:39.567408+0000","last_active":"2024-09-19T08:45:48.221461+0000","last_peered":"2024-09-19T08:45:48.221461+0000","last_clean":"2024-09-19T08:45:48.221461+0000","last_became_active":"2024-09-19T08:42:39.566967+0000","last_became_peered":"2024-09-19T08:42:39.566967+0000","last_unstale":"2024-09-19T08:45:48.221461+0000","last_undegraded":"2024-09-19T08:45:48.221461+0000","last_fullsized":"2024-09-19T08:45:48.221461+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:14.287944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:14.287944+0000","last_clean_scrub_stamp":"2024-09-19T08:39:14.287944+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:31:03.245386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.966263+0000","last_change":"2024-09-19T08:43:50.751097+0000","last_active":"2024-09-19T08:45:47.966263+0000","last_peered":"2024-09-19T08:45:47.966263+0000","last_clean":"2024-09-19T08:45:47.966263+0000","last_became_active":"2024-09-19T08:43:50.750341+0000","last_became_peered":"2024-09-19T08:43:50.750341+0000","last_unstale":"2024-09-19T08:45:47.966263+0000","last_undegraded":"2024-09-19T08:45:47.966263+0000","last_fullsized":"2024-09-19T08:45:47.966263+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:25.694535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066373+0000","last_change":"2024-09-19T08:43:52.767905+0000","last_active":"2024-09-19T08:45:48.066373+0000","last_peered":"2024-09-19T08:45:48.066373+0000","last_clean":"2024-09-19T08:45:48.066373+0000","last_became_active":"2024-09-19T08:43:52.767601+0000","last_became_peered":"2024-09-19T08:43:52.767601+0000","last_unstale":"2024-09-19T08:45:48.066373+0000","last_undegraded":"2024-09-19T08:45:48.066373+0000","last_fullsized":"2024-09-19T08:45:48.066373+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:14.519879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221338+0000","last_change":"2024-09-19T08:43:50.751397+0000","last_active":"2024-09-19T08:45:48.221338+0000","last_peered":"2024-09-19T08:45:48.221338+0000","last_clean":"2024-09-19T08:45:48.221338+0000","last_became_active":"2024-09-19T08:43:50.751079+0000","last_became_peered":"2024-09-19T08:43:50.751079+0000","last_unstale":"2024-09-19T08:45:48.221338+0000","last_undegraded":"2024-09-19T08:45:48.221338+0000","last_fullsized":"2024-09-19T08:45:48.221338+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:29.904786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.223616+0000","last_change":"2024-09-19T08:43:52.773348+0000","last_active":"2024-09-19T08:45:48.223616+0000","last_peered":"2024-09-19T08:45:48.223616+0000","last_clean":"2024-09-19T08:45:48.223616+0000","last_became_active":"2024-09-19T08:43:52.773011+0000","last_became_peered":"2024-09-19T08:43:52.773011+0000","last_unstale":"2024-09-19T08:45:48.223616+0000","last_undegraded":"2024-09-19T08:45:48.223616+0000","last_fullsized":"2024-09-19T08:45:48.223616+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:10.806578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":135,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.223789+0000","last_change":"2024-09-19T08:43:51.025300+0000","last_active":"2024-09-19T08:45:48.223789+0000","last_peered":"2024-09-19T08:45:48.223789+0000","last_clean":"2024-09-19T08:45:48.223789+0000","last_became_active":"2024-09-19T08:43:51.024936+0000","last_became_peered":"2024-09-19T08:43:51.024936+0000","last_unstale":"2024-09-19T08:45:48.223789+0000","last_undegraded":"2024-09-19T08:45:48.223789+0000","last_fullsized":"2024-09-19T08:45:48.223789+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:03.186066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074063+0000","last_change":"2024-09-19T08:43:52.774962+0000","last_active":"2024-09-19T08:45:48.074063+0000","last_peered":"2024-09-19T08:45:48.074063+0000","last_clean":"2024-09-19T08:45:48.074063+0000","last_became_active":"2024-09-19T08:43:52.774598+0000","last_became_peered":"2024-09-19T08:43:52.774598+0000","last_unstale":"2024-09-19T08:45:48.074063+0000","last_undegraded":"2024-09-19T08:45:48.074063+0000","last_fullsized":"2024-09-19T08:45:48.074063+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:02.687136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'7","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.782526+0000","last_change":"2024-09-19T08:43:50.748180+0000","last_active":"2024-09-19T08:45:48.782526+0000","last_peered":"2024-09-19T08:45:48.782526+0000","last_clean":"2024-09-19T08:45:48.782526+0000","last_became_active":"2024-09-19T08:43:50.747853+0000","last_became_peered":"2024-09-19T08:43:50.747853+0000","last_unstale":"2024-09-19T08:45:48.782526+0000","last_undegraded":"2024-09-19T08:45:48.782526+0000","last_fullsized":"2024-09-19T08:45:48.782526+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:56.047259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.077066+0000","last_change":"2024-09-19T08:43:52.770253+0000","last_active":"2024-09-19T08:45:48.077066+0000","last_peered":"2024-09-19T08:45:48.077066+0000","last_clean":"2024-09-19T08:45:48.077066+0000","last_became_active":"2024-09-19T08:43:52.769951+0000","last_became_peered":"2024-09-19T08:43:52.769951+0000","last_unstale":"2024-09-19T08:45:48.077066+0000","last_undegraded":"2024-09-19T08:45:48.077066+0000","last_fullsized":"2024-09-19T08:45:48.077066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:02.353144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.967260+0000","last_change":"2024-09-19T08:43:51.026453+0000","last_active":"2024-09-19T08:45:47.967260+0000","last_peered":"2024-09-19T08:45:47.967260+0000","last_clean":"2024-09-19T08:45:47.967260+0000","last_became_active":"2024-09-19T08:43:51.025873+0000","last_became_peered":"2024-09-19T08:43:51.025873+0000","last_unstale":"2024-09-19T08:45:47.967260+0000","last_undegraded":"2024-09-19T08:45:47.967260+0000","last_fullsized":"2024-09-19T08:45:47.967260+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:25:17.475074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.076927+0000","last_change":"2024-09-19T08:43:52.761700+0000","last_active":"2024-09-19T08:45:48.076927+0000","last_peered":"2024-09-19T08:45:48.076927+0000","last_clean":"2024-09-19T08:45:48.076927+0000","last_became_active":"2024-09-19T08:43:52.755527+0000","last_became_peered":"2024-09-19T08:43:52.755527+0000","last_unstale":"2024-09-19T08:45:48.076927+0000","last_undegraded":"2024-09-19T08:45:48.076927+0000","last_fullsized":"2024-09-19T08:45:48.076927+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:18:43.171284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066299+0000","last_change":"2024-09-19T08:43:51.026136+0000","last_active":"2024-09-19T08:45:48.066299+0000","last_peered":"2024-09-19T08:45:48.066299+0000","last_clean":"2024-09-19T08:45:48.066299+0000","last_became_active":"2024-09-19T08:43:51.025740+0000","last_became_peered":"2024-09-19T08:43:51.025740+0000","last_unstale":"2024-09-19T08:45:48.066299+0000","last_undegraded":"2024-09-19T08:45:48.066299+0000","last_fullsized":"2024-09-19T08:45:48.066299+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:17:32.741083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.224195+0000","last_change":"2024-09-19T08:43:52.763808+0000","last_active":"2024-09-19T08:45:48.224195+0000","last_peered":"2024-09-19T08:45:48.224195+0000","last_clean":"2024-09-19T08:45:48.224195+0000","last_became_active":"2024-09-19T08:43:52.763509+0000","last_became_peered":"2024-09-19T08:43:52.763509+0000","last_unstale":"2024-09-19T08:45:48.224195+0000","last_undegraded":"2024-09-19T08:45:48.224195+0000","last_fullsized":"2024-09-19T08:45:48.224195+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:45:08.385419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'51","reported_seq":98,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.189380+0000","last_change":"2024-09-19T08:43:50.744213+0000","last_active":"2024-09-19T08:45:48.189380+0000","last_peered":"2024-09-19T08:45:48.189380+0000","last_clean":"2024-09-19T08:45:48.189380+0000","last_became_active":"2024-09-19T08:43:50.743848+0000","last_became_peered":"2024-09-19T08:43:50.743848+0000","last_unstale":"2024-09-19T08:45:48.189380+0000","last_undegraded":"2024-09-19T08:45:48.189380+0000","last_fullsized":"2024-09-19T08:45:48.189380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:26.613055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":90,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.067394+0000","last_change":"2024-09-19T08:43:52.767102+0000","last_active":"2024-09-19T08:45:48.067394+0000","last_peered":"2024-09-19T08:45:48.067394+0000","last_clean":"2024-09-19T08:45:48.067394+0000","last_became_active":"2024-09-19T08:43:52.766262+0000","last_became_peered":"2024-09-19T08:43:52.766262+0000","last_unstale":"2024-09-19T08:45:48.067394+0000","last_undegraded":"2024-09-19T08:45:48.067394+0000","last_fullsized":"2024-09-19T08:45:48.067394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:35.740462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.966479+0000","last_change":"2024-09-19T08:43:51.131742+0000","last_active":"2024-09-19T08:45:47.966479+0000","last_peered":"2024-09-19T08:45:47.966479+0000","last_clean":"2024-09-19T08:45:47.966479+0000","last_became_active":"2024-09-19T08:43:51.131388+0000","last_became_peered":"2024-09-19T08:43:51.131388+0000","last_unstale":"2024-09-19T08:45:47.966479+0000","last_undegraded":"2024-09-19T08:45:47.966479+0000","last_fullsized":"2024-09-19T08:45:47.966479+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:26.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.230731+0000","last_change":"2024-09-19T08:43:52.756004+0000","last_active":"2024-09-19T08:45:48.230731+0000","last_peered":"2024-09-19T08:45:48.230731+0000","last_clean":"2024-09-19T08:45:48.230731+0000","last_became_active":"2024-09-19T08:43:52.755596+0000","last_became_peered":"2024-09-19T08:43:52.755596+0000","last_unstale":"2024-09-19T08:45:48.230731+0000","last_undegraded":"2024-09-19T08:45:48.230731+0000","last_fullsized":"2024-09-19T08:45:48.230731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:35.942375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221620+0000","last_change":"2024-09-19T08:44:02.835986+0000","last_active":"2024-09-19T08:45:48.221620+0000","last_peered":"2024-09-19T08:45:48.221620+0000","last_clean":"2024-09-19T08:45:48.221620+0000","last_became_active":"2024-09-19T08:44:02.826314+0000","last_became_peered":"2024-09-19T08:44:02.826314+0000","last_unstale":"2024-09-19T08:45:48.221620+0000","last_undegraded":"2024-09-19T08:45:48.221620+0000","last_fullsized":"2024-09-19T08:45:48.221620+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:16.943989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065838+0000","last_change":"2024-09-19T08:43:52.766582+0000","last_active":"2024-09-19T08:45:48.065838+0000","last_peered":"2024-09-19T08:45:48.065838+0000","last_clean":"2024-09-19T08:45:48.065838+0000","last_became_active":"2024-09-19T08:43:52.766232+0000","last_became_peered":"2024-09-19T08:43:52.766232+0000","last_unstale":"2024-09-19T08:45:48.065838+0000","last_undegraded":"2024-09-19T08:45:48.065838+0000","last_fullsized":"2024-09-19T08:45:48.065838+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:43:42.319422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066195+0000","last_change":"2024-09-19T08:43:50.747611+0000","last_active":"2024-09-19T08:45:48.066195+0000","last_peered":"2024-09-19T08:45:48.066195+0000","last_clean":"2024-09-19T08:45:48.066195+0000","last_became_active":"2024-09-19T08:43:50.747263+0000","last_became_peered":"2024-09-19T08:43:50.747263+0000","last_unstale":"2024-09-19T08:45:48.066195+0000","last_undegraded":"2024-09-19T08:45:48.066195+0000","last_fullsized":"2024-09-19T08:45:48.066195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:14.717820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229532+0000","last_change":"2024-09-19T08:43:52.766493+0000","last_active":"2024-09-19T08:45:48.229532+0000","last_peered":"2024-09-19T08:45:48.229532+0000","last_clean":"2024-09-19T08:45:48.229532+0000","last_became_active":"2024-09-19T08:43:52.766158+0000","last_became_peered":"2024-09-19T08:43:52.766158+0000","last_unstale":"2024-09-19T08:45:48.229532+0000","last_undegraded":"2024-09-19T08:45:48.229532+0000","last_fullsized":"2024-09-19T08:45:48.229532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:59.175949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968841+0000","last_change":"2024-09-19T08:43:50.751368+0000","last_active":"2024-09-19T08:45:47.968841+0000","last_peered":"2024-09-19T08:45:47.968841+0000","last_clean":"2024-09-19T08:45:47.968841+0000","last_became_active":"2024-09-19T08:43:50.751038+0000","last_became_peered":"2024-09-19T08:43:50.751038+0000","last_unstale":"2024-09-19T08:45:47.968841+0000","last_undegraded":"2024-09-19T08:45:47.968841+0000","last_fullsized":"2024-09-19T08:45:47.968841+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:46.302996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066722+0000","last_change":"2024-09-19T08:43:52.756998+0000","last_active":"2024-09-19T08:45:48.066722+0000","last_peered":"2024-09-19T08:45:48.066722+0000","last_clean":"2024-09-19T08:45:48.066722+0000","last_became_active":"2024-09-19T08:43:52.756661+0000","last_became_peered":"2024-09-19T08:43:52.756661+0000","last_unstale":"2024-09-19T08:45:48.066722+0000","last_undegraded":"2024-09-19T08:45:48.066722+0000","last_fullsized":"2024-09-19T08:45:48.066722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:15.182574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065908+0000","last_change":"2024-09-19T08:43:50.750095+0000","last_active":"2024-09-19T08:45:48.065908+0000","last_peered":"2024-09-19T08:45:48.065908+0000","last_clean":"2024-09-19T08:45:48.065908+0000","last_became_active":"2024-09-19T08:43:50.749672+0000","last_became_peered":"2024-09-19T08:43:50.749672+0000","last_unstale":"2024-09-19T08:45:48.065908+0000","last_undegraded":"2024-09-19T08:45:48.065908+0000","last_fullsized":"2024-09-19T08:45:48.065908+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:25:34.978094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221996+0000","last_change":"2024-09-19T08:43:52.764097+0000","last_active":"2024-09-19T08:45:48.221996+0000","last_peered":"2024-09-19T08:45:48.221996+0000","last_clean":"2024-09-19T08:45:48.221996+0000","last_became_active":"2024-09-19T08:43:52.763773+0000","last_became_peered":"2024-09-19T08:43:52.763773+0000","last_unstale":"2024-09-19T08:45:48.221996+0000","last_undegraded":"2024-09-19T08:45:48.221996+0000","last_fullsized":"2024-09-19T08:45:48.221996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:01:42.098460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074171+0000","last_change":"2024-09-19T08:44:02.825238+0000","last_active":"2024-09-19T08:45:48.074171+0000","last_peered":"2024-09-19T08:45:48.074171+0000","last_clean":"2024-09-19T08:45:48.074171+0000","last_became_active":"2024-09-19T08:44:02.824873+0000","last_became_peered":"2024-09-19T08:44:02.824873+0000","last_unstale":"2024-09-19T08:45:48.074171+0000","last_undegraded":"2024-09-19T08:45:48.074171+0000","last_fullsized":"2024-09-19T08:45:48.074171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:38.896082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066088+0000","last_change":"2024-09-19T08:43:50.748789+0000","last_active":"2024-09-19T08:45:48.066088+0000","last_peered":"2024-09-19T08:45:48.066088+0000","last_clean":"2024-09-19T08:45:48.066088+0000","last_became_active":"2024-09-19T08:43:50.748464+0000","last_became_peered":"2024-09-19T08:43:50.748464+0000","last_unstale":"2024-09-19T08:45:48.066088+0000","last_undegraded":"2024-09-19T08:45:48.066088+0000","last_fullsized":"2024-09-19T08:45:48.066088+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:36:45.905639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221312+0000","last_change":"2024-09-19T08:43:52.756608+0000","last_active":"2024-09-19T08:45:48.221312+0000","last_peered":"2024-09-19T08:45:48.221312+0000","last_clean":"2024-09-19T08:45:48.221312+0000","last_became_active":"2024-09-19T08:43:52.756227+0000","last_became_peered":"2024-09-19T08:43:52.756227+0000","last_unstale":"2024-09-19T08:45:48.221312+0000","last_undegraded":"2024-09-19T08:45:48.221312+0000","last_fullsized":"2024-09-19T08:45:48.221312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:29:58.085584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'13","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066253+0000","last_change":"2024-09-19T08:43:50.749135+0000","last_active":"2024-09-19T08:45:48.066253+0000","last_peered":"2024-09-19T08:45:48.066253+0000","last_clean":"2024-09-19T08:45:48.066253+0000","last_became_active":"2024-09-19T08:43:50.748755+0000","last_became_peered":"2024-09-19T08:43:50.748755+0000","last_unstale":"2024-09-19T08:45:48.066253+0000","last_undegraded":"2024-09-19T08:45:48.066253+0000","last_fullsized":"2024-09-19T08:45:48.066253+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:55.079780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.231077+0000","last_change":"2024-09-19T08:43:52.763277+0000","last_active":"2024-09-19T08:45:48.231077+0000","last_peered":"2024-09-19T08:45:48.231077+0000","last_clean":"2024-09-19T08:45:48.231077+0000","last_became_active":"2024-09-19T08:43:52.762941+0000","last_became_peered":"2024-09-19T08:43:52.762941+0000","last_unstale":"2024-09-19T08:45:48.231077+0000","last_undegraded":"2024-09-19T08:45:48.231077+0000","last_fullsized":"2024-09-19T08:45:48.231077+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:35.531946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.067406+0000","last_change":"2024-09-19T08:43:51.025350+0000","last_active":"2024-09-19T08:45:48.067406+0000","last_peered":"2024-09-19T08:45:48.067406+0000","last_clean":"2024-09-19T08:45:48.067406+0000","last_became_active":"2024-09-19T08:43:51.025046+0000","last_became_peered":"2024-09-19T08:43:51.025046+0000","last_unstale":"2024-09-19T08:45:48.067406+0000","last_undegraded":"2024-09-19T08:45:48.067406+0000","last_fullsized":"2024-09-19T08:45:48.067406+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:27.595545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066247+0000","last_change":"2024-09-19T08:43:52.767917+0000","last_active":"2024-09-19T08:45:48.066247+0000","last_peered":"2024-09-19T08:45:48.066247+0000","last_clean":"2024-09-19T08:45:48.066247+0000","last_became_active":"2024-09-19T08:43:52.767575+0000","last_became_peered":"2024-09-19T08:43:52.767575+0000","last_unstale":"2024-09-19T08:45:48.066247+0000","last_undegraded":"2024-09-19T08:45:48.066247+0000","last_fullsized":"2024-09-19T08:45:48.066247+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:05.950399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221116+0000","last_change":"2024-09-19T08:43:50.746400+0000","last_active":"2024-09-19T08:45:48.221116+0000","last_peered":"2024-09-19T08:45:48.221116+0000","last_clean":"2024-09-19T08:45:48.221116+0000","last_became_active":"2024-09-19T08:43:50.745891+0000","last_became_peered":"2024-09-19T08:43:50.745891+0000","last_unstale":"2024-09-19T08:45:48.221116+0000","last_undegraded":"2024-09-19T08:45:48.221116+0000","last_fullsized":"2024-09-19T08:45:48.221116+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:03.098202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599805,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1190,"num_read_kb":711,"num_write":374,"num_write_kb":1464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":712704,"data_stored":1799415,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":301,"ondisk_log_size":301,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":109335,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579941,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":446056,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808874,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1089536,"data_stored":1232787,"data_compressed":45500,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005102,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":962560,"data_stored":838280,"data_compressed":22045,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234035,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":946176,"data_stored":838429,"data_compressed":22045,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462968,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":983040,"data_stored":1039165,"data_compressed":39344,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593790,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":445450,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822722,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1015808,"data_stored":1169657,"data_compressed":45500,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986247,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":444525,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215180,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":969188,"data_compressed":28201,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1003520,"data_stored":513578,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":983040,"data_stored":512290,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":937984,"data_stored":446930,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":68465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":69753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135347,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":199263,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:45:51.116 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:45:51.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.253+0000 7f726a39f640 1 -- 172.21.15.164:0/432848220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7264107890 msgr2=0x7f7264109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:51.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.253+0000 7f726a39f640 1 --2- 172.21.15.164:0/432848220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7264107890 0x7f7264109c80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f725c009ca0 tx=0x7f725c02a8c0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.253+0000 7f726a39f640 1 -- 172.21.15.164:0/432848220 shutdown_connections 2024-09-19T08:45:51.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.253+0000 7f726a39f640 1 --2- 172.21.15.164:0/432848220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7264107890 0x7f7264109c80 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.253+0000 7f726a39f640 1 --2- 172.21.15.164:0/432848220 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7264104f60 0x7f7264107350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.253+0000 7f726a39f640 1 --2- 172.21.15.164:0/432848220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7264102630 0x7f7264104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.253+0000 7f726a39f640 1 -- 172.21.15.164:0/432848220 >> 172.21.15.164:0/432848220 conn(0x7f72640fc170 msgr2=0x7f72640fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:51.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.253+0000 7f726a39f640 1 -- 172.21.15.164:0/432848220 shutdown_connections 2024-09-19T08:45:51.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.253+0000 7f726a39f640 1 -- 172.21.15.164:0/432848220 wait complete. 2024-09-19T08:45:51.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.253+0000 7f726a39f640 1 Processor -- start 2024-09-19T08:45:51.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.253+0000 7f726a39f640 1 -- start start 2024-09-19T08:45:51.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.254+0000 7f726a39f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7264102630 0x7f7264101fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:51.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.254+0000 7f726a39f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7264104f60 0x7f72641005e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:51.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.254+0000 7f726a39f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7264107890 0x7f7264100b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:51.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.254+0000 7f726a39f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f726410d7d0 con 0x7f7264104f60 2024-09-19T08:45:51.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.254+0000 7f726a39f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f726410d650 con 0x7f7264102630 2024-09-19T08:45:51.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.254+0000 7f726a39f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f726410d950 con 0x7f7264107890 2024-09-19T08:45:51.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.254+0000 7f72637fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7264104f60 0x7f72641005e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:51.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.254+0000 7f72637fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7264104f60 0x7f72641005e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46556/0 (socket says 172.21.15.164:46556) 2024-09-19T08:45:51.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.254+0000 7f72637fe640 1 -- 172.21.15.164:0/2531010062 learned_addr learned my addr 172.21.15.164:0/2531010062 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:51.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.254+0000 7f726893c640 1 --2- 172.21.15.164:0/2531010062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7264107890 0x7f7264100b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:51.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.254+0000 7f7263fff640 1 --2- 172.21.15.164:0/2531010062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7264102630 0x7f7264101fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:51.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.254+0000 7f72637fe640 1 -- 172.21.15.164:0/2531010062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7264107890 msgr2=0x7f7264100b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:51.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.254+0000 7f72637fe640 1 --2- 172.21.15.164:0/2531010062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7264107890 0x7f7264100b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.254+0000 7f72637fe640 1 -- 172.21.15.164:0/2531010062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7264102630 msgr2=0x7f7264101fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:51.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.254+0000 7f72637fe640 1 --2- 172.21.15.164:0/2531010062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7264102630 0x7f7264101fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.254+0000 7f72637fe640 1 -- 172.21.15.164:0/2531010062 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72641013b0 con 0x7f7264104f60 2024-09-19T08:45:51.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.254+0000 7f726893c640 1 --2- 172.21.15.164:0/2531010062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7264107890 0x7f7264100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:45:51.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.254+0000 7f7263fff640 1 --2- 172.21.15.164:0/2531010062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7264102630 0x7f7264101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:51.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.254+0000 7f72637fe640 1 --2- 172.21.15.164:0/2531010062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7264104f60 0x7f72641005e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f725800e9f0 tx=0x7f725800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:51.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.255+0000 7f72617fa640 1 -- 172.21.15.164:0/2531010062 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725800ccc0 con 0x7f7264104f60 2024-09-19T08:45:51.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.255+0000 7f726a39f640 1 -- 172.21.15.164:0/2531010062 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72641a1f80 con 0x7f7264104f60 2024-09-19T08:45:51.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.255+0000 7f72617fa640 1 -- 172.21.15.164:0/2531010062 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7258005500 con 0x7f7264104f60 2024-09-19T08:45:51.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.255+0000 7f72617fa640 1 -- 172.21.15.164:0/2531010062 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7258010690 con 0x7f7264104f60 2024-09-19T08:45:51.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.255+0000 7f726a39f640 1 -- 172.21.15.164:0/2531010062 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72641a2390 con 0x7f7264104f60 2024-09-19T08:45:51.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.256+0000 7f726a39f640 1 -- 172.21.15.164:0/2531010062 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72640fdc60 con 0x7f7264104f60 2024-09-19T08:45:51.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.257+0000 7f72617fa640 1 -- 172.21.15.164:0/2531010062 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 23) ==== 138476+0+0 (secure 0 0 0) 0x7f72580046c0 con 0x7f7264104f60 2024-09-19T08:45:51.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.258+0000 7f72617fa640 1 --2- 172.21.15.164:0/2531010062 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f724c0a33c0 0x7f724c0a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:51.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.258+0000 7f7263fff640 1 --2- 172.21.15.164:0/2531010062 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f724c0a33c0 0x7f724c0a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:51.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.258+0000 7f72617fa640 1 -- 172.21.15.164:0/2531010062 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f72580d0270 con 0x7f7264104f60 2024-09-19T08:45:51.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.258+0000 7f7263fff640 1 --2- 172.21.15.164:0/2531010062 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f724c0a33c0 0x7f724c0a5880 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7254009c20 tx=0x7f7254005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:51.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.265+0000 7f72617fa640 1 -- 172.21.15.164:0/2531010062 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7258054210 con 0x7f7264104f60 2024-09-19T08:45:51.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:50 smithi170 ceph-mon[35072]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:45:51.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:50 smithi170 ceph-mon[35072]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-09-19T08:45:51.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:50 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:51.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:50 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:51.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:50 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:51.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:50 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:51.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:50 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:51.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:50 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:51.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:50 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:51.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:50 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:51.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:50 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:51.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:50 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:51.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:50 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:51.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:50 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:51.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:50 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:51.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:50 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:51.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:50 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:51.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:50 smithi170 ceph-mon[35072]: Adjusting osd_memory_target on smithi170 to 176.0M 2024-09-19T08:45:51.377 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:50 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:51.377 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:50 smithi170 ceph-mon[35072]: Unable to set osd_memory_target on smithi170 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-19T08:45:51.377 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:50 smithi170 ceph-mon[35072]: Adjusting osd_memory_target on smithi173 to 176.0M 2024-09-19T08:45:51.377 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:50 smithi170 ceph-mon[35072]: Unable to set osd_memory_target on smithi173 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-19T08:45:51.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:50 smithi173 ceph-mon[35306]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:45:51.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:50 smithi173 ceph-mon[35306]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-09-19T08:45:51.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:50 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:51.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:50 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:51.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:50 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:51.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:50 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:51.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:50 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:51.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:50 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:51.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:50 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:51.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:50 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:51.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:50 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:51.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:50 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:51.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:50 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:51.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:50 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:51.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:50 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:51.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:50 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:51.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:50 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:51.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:50 smithi173 ceph-mon[35306]: Adjusting osd_memory_target on smithi170 to 176.0M 2024-09-19T08:45:51.413 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:50 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:51.413 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:50 smithi173 ceph-mon[35306]: Unable to set osd_memory_target on smithi170 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-19T08:45:51.413 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:50 smithi173 ceph-mon[35306]: Adjusting osd_memory_target on smithi173 to 176.0M 2024-09-19T08:45:51.413 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:50 smithi173 ceph-mon[35306]: Unable to set osd_memory_target on smithi173 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-19T08:45:51.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.437+0000 7f726a39f640 1 -- 172.21.15.164:0/2531010062 --> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7264062630 con 0x7f724c0a33c0 2024-09-19T08:45:51.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.443+0000 7f72617fa640 1 -- 172.21.15.164:0/2531010062 <== mgr.15201 v2:172.21.15.164:6800/173486514 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258041 (secure 0 0 0) 0x7f7264062630 con 0x7f724c0a33c0 2024-09-19T08:45:51.446 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:45:51.448 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-09-19T08:45:51.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.446+0000 7f726a39f640 1 -- 172.21.15.164:0/2531010062 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f724c0a33c0 msgr2=0x7f724c0a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:51.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.446+0000 7f726a39f640 1 --2- 172.21.15.164:0/2531010062 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f724c0a33c0 0x7f724c0a5880 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7254009c20 tx=0x7f7254005f90 comp rx=0 tx=0).stop 2024-09-19T08:45:51.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.446+0000 7f726a39f640 1 -- 172.21.15.164:0/2531010062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7264104f60 msgr2=0x7f72641005e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:51.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.446+0000 7f726a39f640 1 --2- 172.21.15.164:0/2531010062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7264104f60 0x7f72641005e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f725800e9f0 tx=0x7f725800eeb0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.446+0000 7f726a39f640 1 -- 172.21.15.164:0/2531010062 shutdown_connections 2024-09-19T08:45:51.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.446+0000 7f726a39f640 1 --2- 172.21.15.164:0/2531010062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7264107890 0x7f7264100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.446+0000 7f726a39f640 1 --2- 172.21.15.164:0/2531010062 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f724c0a33c0 0x7f724c0a5880 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.446+0000 7f726a39f640 1 --2- 172.21.15.164:0/2531010062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7264104f60 0x7f72641005e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.446+0000 7f726a39f640 1 --2- 172.21.15.164:0/2531010062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7264102630 0x7f7264101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.446+0000 7f726a39f640 1 -- 172.21.15.164:0/2531010062 >> 172.21.15.164:0/2531010062 conn(0x7f72640fc170 msgr2=0x7f7264102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:51.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.446+0000 7f726a39f640 1 -- 172.21.15.164:0/2531010062 shutdown_connections 2024-09-19T08:45:51.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.446+0000 7f726a39f640 1 -- 172.21.15.164:0/2531010062 wait complete. 2024-09-19T08:45:51.458 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-09-19T08:45:49.981301+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1190407,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1326,"num_read_kb":830,"num_write":557,"num_write_kb":4404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":940,"ondisk_log_size":940,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":389792,"kb_used_data":11376,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124552032,"statfs":{"total":1151940427776,"available":1151541280768,"internally_reserved":0,"allocated":11649024,"data_stored":8896335,"data_compressed":220029,"data_compressed_allocated":282624,"data_compressed_original":3416064,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.991213"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229715+0000","last_change":"2024-09-19T08:43:52.766709+0000","last_active":"2024-09-19T08:45:48.229715+0000","last_peered":"2024-09-19T08:45:48.229715+0000","last_clean":"2024-09-19T08:45:48.229715+0000","last_became_active":"2024-09-19T08:43:52.765970+0000","last_became_peered":"2024-09-19T08:43:52.765970+0000","last_unstale":"2024-09-19T08:45:48.229715+0000","last_undegraded":"2024-09-19T08:45:48.229715+0000","last_fullsized":"2024-09-19T08:45:48.229715+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:12.176870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.230605+0000","last_change":"2024-09-19T08:43:52.766590+0000","last_active":"2024-09-19T08:45:48.230605+0000","last_peered":"2024-09-19T08:45:48.230605+0000","last_clean":"2024-09-19T08:45:48.230605+0000","last_became_active":"2024-09-19T08:43:52.766255+0000","last_became_peered":"2024-09-19T08:43:52.766255+0000","last_unstale":"2024-09-19T08:45:48.230605+0000","last_undegraded":"2024-09-19T08:45:48.230605+0000","last_fullsized":"2024-09-19T08:45:48.230605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:15:41.359806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.230306+0000","last_change":"2024-09-19T08:43:52.761956+0000","last_active":"2024-09-19T08:45:48.230306+0000","last_peered":"2024-09-19T08:45:48.230306+0000","last_clean":"2024-09-19T08:45:48.230306+0000","last_became_active":"2024-09-19T08:43:52.761644+0000","last_became_peered":"2024-09-19T08:43:52.761644+0000","last_unstale":"2024-09-19T08:45:48.230306+0000","last_undegraded":"2024-09-19T08:45:48.230306+0000","last_fullsized":"2024-09-19T08:45:48.230306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:13.770361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.966146+0000","last_change":"2024-09-19T08:43:52.764131+0000","last_active":"2024-09-19T08:45:47.966146+0000","last_peered":"2024-09-19T08:45:47.966146+0000","last_clean":"2024-09-19T08:45:47.966146+0000","last_became_active":"2024-09-19T08:43:52.763812+0000","last_became_peered":"2024-09-19T08:43:52.763812+0000","last_unstale":"2024-09-19T08:45:47.966146+0000","last_undegraded":"2024-09-19T08:45:47.966146+0000","last_fullsized":"2024-09-19T08:45:47.966146+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:32.272157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065874+0000","last_change":"2024-09-19T08:43:52.754673+0000","last_active":"2024-09-19T08:45:48.065874+0000","last_peered":"2024-09-19T08:45:48.065874+0000","last_clean":"2024-09-19T08:45:48.065874+0000","last_became_active":"2024-09-19T08:43:52.754144+0000","last_became_peered":"2024-09-19T08:43:52.754144+0000","last_unstale":"2024-09-19T08:45:48.065874+0000","last_undegraded":"2024-09-19T08:45:48.065874+0000","last_fullsized":"2024-09-19T08:45:48.065874+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:16.069260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.076773+0000","last_change":"2024-09-19T08:43:52.758939+0000","last_active":"2024-09-19T08:45:48.076773+0000","last_peered":"2024-09-19T08:45:48.076773+0000","last_clean":"2024-09-19T08:45:48.076773+0000","last_became_active":"2024-09-19T08:43:52.758611+0000","last_became_peered":"2024-09-19T08:43:52.758611+0000","last_unstale":"2024-09-19T08:45:48.076773+0000","last_undegraded":"2024-09-19T08:45:48.076773+0000","last_fullsized":"2024-09-19T08:45:48.076773+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:35:56.463545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221260+0000","last_change":"2024-09-19T08:43:52.770261+0000","last_active":"2024-09-19T08:45:48.221260+0000","last_peered":"2024-09-19T08:45:48.221260+0000","last_clean":"2024-09-19T08:45:48.221260+0000","last_became_active":"2024-09-19T08:43:52.768253+0000","last_became_peered":"2024-09-19T08:43:52.768253+0000","last_unstale":"2024-09-19T08:45:48.221260+0000","last_undegraded":"2024-09-19T08:45:48.221260+0000","last_fullsized":"2024-09-19T08:45:48.221260+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:40.756988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065907+0000","last_change":"2024-09-19T08:43:52.768564+0000","last_active":"2024-09-19T08:45:48.065907+0000","last_peered":"2024-09-19T08:45:48.065907+0000","last_clean":"2024-09-19T08:45:48.065907+0000","last_became_active":"2024-09-19T08:43:52.768260+0000","last_became_peered":"2024-09-19T08:43:52.768260+0000","last_unstale":"2024-09-19T08:45:48.065907+0000","last_undegraded":"2024-09-19T08:45:48.065907+0000","last_fullsized":"2024-09-19T08:45:48.065907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:16.528115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.222053+0000","last_change":"2024-09-19T08:43:52.769804+0000","last_active":"2024-09-19T08:45:48.222053+0000","last_peered":"2024-09-19T08:45:48.222053+0000","last_clean":"2024-09-19T08:45:48.222053+0000","last_became_active":"2024-09-19T08:43:52.769504+0000","last_became_peered":"2024-09-19T08:43:52.769504+0000","last_unstale":"2024-09-19T08:45:48.222053+0000","last_undegraded":"2024-09-19T08:45:48.222053+0000","last_fullsized":"2024-09-19T08:45:48.222053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:22:38.514410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.077170+0000","last_change":"2024-09-19T08:43:52.763278+0000","last_active":"2024-09-19T08:45:48.077170+0000","last_peered":"2024-09-19T08:45:48.077170+0000","last_clean":"2024-09-19T08:45:48.077170+0000","last_became_active":"2024-09-19T08:43:52.762985+0000","last_became_peered":"2024-09-19T08:43:52.762985+0000","last_unstale":"2024-09-19T08:45:48.077170+0000","last_undegraded":"2024-09-19T08:45:48.077170+0000","last_fullsized":"2024-09-19T08:45:48.077170+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:30.048782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221488+0000","last_change":"2024-09-19T08:43:52.770864+0000","last_active":"2024-09-19T08:45:48.221488+0000","last_peered":"2024-09-19T08:45:48.221488+0000","last_clean":"2024-09-19T08:45:48.221488+0000","last_became_active":"2024-09-19T08:43:52.770518+0000","last_became_peered":"2024-09-19T08:43:52.770518+0000","last_unstale":"2024-09-19T08:45:48.221488+0000","last_undegraded":"2024-09-19T08:45:48.221488+0000","last_fullsized":"2024-09-19T08:45:48.221488+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:08.931452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.966171+0000","last_change":"2024-09-19T08:43:52.757464+0000","last_active":"2024-09-19T08:45:47.966171+0000","last_peered":"2024-09-19T08:45:47.966171+0000","last_clean":"2024-09-19T08:45:47.966171+0000","last_became_active":"2024-09-19T08:43:52.757101+0000","last_became_peered":"2024-09-19T08:43:52.757101+0000","last_unstale":"2024-09-19T08:45:47.966171+0000","last_undegraded":"2024-09-19T08:45:47.966171+0000","last_fullsized":"2024-09-19T08:45:47.966171+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:17.395735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.224010+0000","last_change":"2024-09-19T08:43:52.767063+0000","last_active":"2024-09-19T08:45:48.224010+0000","last_peered":"2024-09-19T08:45:48.224010+0000","last_clean":"2024-09-19T08:45:48.224010+0000","last_became_active":"2024-09-19T08:43:52.766755+0000","last_became_peered":"2024-09-19T08:43:52.766755+0000","last_unstale":"2024-09-19T08:45:48.224010+0000","last_undegraded":"2024-09-19T08:45:48.224010+0000","last_fullsized":"2024-09-19T08:45:48.224010+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:30.530937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.230150+0000","last_change":"2024-09-19T08:43:52.766515+0000","last_active":"2024-09-19T08:45:48.230150+0000","last_peered":"2024-09-19T08:45:48.230150+0000","last_clean":"2024-09-19T08:45:48.230150+0000","last_became_active":"2024-09-19T08:43:52.765847+0000","last_became_peered":"2024-09-19T08:43:52.765847+0000","last_unstale":"2024-09-19T08:45:48.230150+0000","last_undegraded":"2024-09-19T08:45:48.230150+0000","last_fullsized":"2024-09-19T08:45:48.230150+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:50:35.948663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229847+0000","last_change":"2024-09-19T08:43:52.773702+0000","last_active":"2024-09-19T08:45:48.229847+0000","last_peered":"2024-09-19T08:45:48.229847+0000","last_clean":"2024-09-19T08:45:48.229847+0000","last_became_active":"2024-09-19T08:43:52.773324+0000","last_became_peered":"2024-09-19T08:43:52.773324+0000","last_unstale":"2024-09-19T08:45:48.229847+0000","last_undegraded":"2024-09-19T08:45:48.229847+0000","last_fullsized":"2024-09-19T08:45:48.229847+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:34.368014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221253+0000","last_change":"2024-09-19T08:43:52.762745+0000","last_active":"2024-09-19T08:45:48.221253+0000","last_peered":"2024-09-19T08:45:48.221253+0000","last_clean":"2024-09-19T08:45:48.221253+0000","last_became_active":"2024-09-19T08:43:52.762172+0000","last_became_peered":"2024-09-19T08:43:52.762172+0000","last_unstale":"2024-09-19T08:45:48.221253+0000","last_undegraded":"2024-09-19T08:45:48.221253+0000","last_fullsized":"2024-09-19T08:45:48.221253+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:57.369775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229743+0000","last_change":"2024-09-19T08:43:52.767118+0000","last_active":"2024-09-19T08:45:48.229743+0000","last_peered":"2024-09-19T08:45:48.229743+0000","last_clean":"2024-09-19T08:45:48.229743+0000","last_became_active":"2024-09-19T08:43:52.766388+0000","last_became_peered":"2024-09-19T08:43:52.766388+0000","last_unstale":"2024-09-19T08:45:48.229743+0000","last_undegraded":"2024-09-19T08:45:48.229743+0000","last_fullsized":"2024-09-19T08:45:48.229743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:34.182923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.967257+0000","last_change":"2024-09-19T08:43:52.765008+0000","last_active":"2024-09-19T08:45:47.967257+0000","last_peered":"2024-09-19T08:45:47.967257+0000","last_clean":"2024-09-19T08:45:47.967257+0000","last_became_active":"2024-09-19T08:43:52.764558+0000","last_became_peered":"2024-09-19T08:43:52.764558+0000","last_unstale":"2024-09-19T08:45:47.967257+0000","last_undegraded":"2024-09-19T08:45:47.967257+0000","last_fullsized":"2024-09-19T08:45:47.967257+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:52:42.756421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968722+0000","last_change":"2024-09-19T08:43:52.764376+0000","last_active":"2024-09-19T08:45:47.968722+0000","last_peered":"2024-09-19T08:45:47.968722+0000","last_clean":"2024-09-19T08:45:47.968722+0000","last_became_active":"2024-09-19T08:43:52.764048+0000","last_became_peered":"2024-09-19T08:43:52.764048+0000","last_unstale":"2024-09-19T08:45:47.968722+0000","last_undegraded":"2024-09-19T08:45:47.968722+0000","last_fullsized":"2024-09-19T08:45:47.968722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:56.732094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229417+0000","last_change":"2024-09-19T08:44:02.827925+0000","last_active":"2024-09-19T08:45:48.229417+0000","last_peered":"2024-09-19T08:45:48.229417+0000","last_clean":"2024-09-19T08:45:48.229417+0000","last_became_active":"2024-09-19T08:44:02.827269+0000","last_became_peered":"2024-09-19T08:44:02.827269+0000","last_unstale":"2024-09-19T08:45:48.229417+0000","last_undegraded":"2024-09-19T08:45:48.229417+0000","last_fullsized":"2024-09-19T08:45:48.229417+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:31:54.556786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.224157+0000","last_change":"2024-09-19T08:43:52.762190+0000","last_active":"2024-09-19T08:45:48.224157+0000","last_peered":"2024-09-19T08:45:48.224157+0000","last_clean":"2024-09-19T08:45:48.224157+0000","last_became_active":"2024-09-19T08:43:52.761818+0000","last_became_peered":"2024-09-19T08:43:52.761818+0000","last_unstale":"2024-09-19T08:45:48.224157+0000","last_undegraded":"2024-09-19T08:45:48.224157+0000","last_fullsized":"2024-09-19T08:45:48.224157+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:19:55.246779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221492+0000","last_change":"2024-09-19T08:43:52.761069+0000","last_active":"2024-09-19T08:45:48.221492+0000","last_peered":"2024-09-19T08:45:48.221492+0000","last_clean":"2024-09-19T08:45:48.221492+0000","last_became_active":"2024-09-19T08:43:52.760705+0000","last_became_peered":"2024-09-19T08:43:52.760705+0000","last_unstale":"2024-09-19T08:45:48.221492+0000","last_undegraded":"2024-09-19T08:45:48.221492+0000","last_fullsized":"2024-09-19T08:45:48.221492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:46:27.059026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065963+0000","last_change":"2024-09-19T08:43:52.773449+0000","last_active":"2024-09-19T08:45:48.065963+0000","last_peered":"2024-09-19T08:45:48.065963+0000","last_clean":"2024-09-19T08:45:48.065963+0000","last_became_active":"2024-09-19T08:43:52.772929+0000","last_became_peered":"2024-09-19T08:43:52.772929+0000","last_unstale":"2024-09-19T08:45:48.065963+0000","last_undegraded":"2024-09-19T08:45:48.065963+0000","last_fullsized":"2024-09-19T08:45:48.065963+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:00:38.929199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074018+0000","last_change":"2024-09-19T08:43:52.773885+0000","last_active":"2024-09-19T08:45:48.074018+0000","last_peered":"2024-09-19T08:45:48.074018+0000","last_clean":"2024-09-19T08:45:48.074018+0000","last_became_active":"2024-09-19T08:43:52.772856+0000","last_became_peered":"2024-09-19T08:43:52.772856+0000","last_unstale":"2024-09-19T08:45:48.074018+0000","last_undegraded":"2024-09-19T08:45:48.074018+0000","last_fullsized":"2024-09-19T08:45:48.074018+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:13:22.581335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.067279+0000","last_change":"2024-09-19T08:43:52.762909+0000","last_active":"2024-09-19T08:45:48.067279+0000","last_peered":"2024-09-19T08:45:48.067279+0000","last_clean":"2024-09-19T08:45:48.067279+0000","last_became_active":"2024-09-19T08:43:52.762600+0000","last_became_peered":"2024-09-19T08:43:52.762600+0000","last_unstale":"2024-09-19T08:45:48.067279+0000","last_undegraded":"2024-09-19T08:45:48.067279+0000","last_fullsized":"2024-09-19T08:45:48.067279+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:46.098294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968939+0000","last_change":"2024-09-19T08:43:52.762861+0000","last_active":"2024-09-19T08:45:47.968939+0000","last_peered":"2024-09-19T08:45:47.968939+0000","last_clean":"2024-09-19T08:45:47.968939+0000","last_became_active":"2024-09-19T08:43:52.762525+0000","last_became_peered":"2024-09-19T08:43:52.762525+0000","last_unstale":"2024-09-19T08:45:47.968939+0000","last_undegraded":"2024-09-19T08:45:47.968939+0000","last_fullsized":"2024-09-19T08:45:47.968939+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:54.717249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229552+0000","last_change":"2024-09-19T08:43:52.768370+0000","last_active":"2024-09-19T08:45:48.229552+0000","last_peered":"2024-09-19T08:45:48.229552+0000","last_clean":"2024-09-19T08:45:48.229552+0000","last_became_active":"2024-09-19T08:43:52.768055+0000","last_became_peered":"2024-09-19T08:43:52.768055+0000","last_unstale":"2024-09-19T08:45:48.229552+0000","last_undegraded":"2024-09-19T08:45:48.229552+0000","last_fullsized":"2024-09-19T08:45:48.229552+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:48.212862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.224066+0000","last_change":"2024-09-19T08:43:52.758541+0000","last_active":"2024-09-19T08:45:48.224066+0000","last_peered":"2024-09-19T08:45:48.224066+0000","last_clean":"2024-09-19T08:45:48.224066+0000","last_became_active":"2024-09-19T08:43:52.758122+0000","last_became_peered":"2024-09-19T08:43:52.758122+0000","last_unstale":"2024-09-19T08:45:48.224066+0000","last_undegraded":"2024-09-19T08:45:48.224066+0000","last_fullsized":"2024-09-19T08:45:48.224066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:54:40.681880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.231790+0000","last_change":"2024-09-19T08:43:52.758109+0000","last_active":"2024-09-19T08:45:48.231790+0000","last_peered":"2024-09-19T08:45:48.231790+0000","last_clean":"2024-09-19T08:45:48.231790+0000","last_became_active":"2024-09-19T08:43:52.757808+0000","last_became_peered":"2024-09-19T08:43:52.757808+0000","last_unstale":"2024-09-19T08:45:48.231790+0000","last_undegraded":"2024-09-19T08:45:48.231790+0000","last_fullsized":"2024-09-19T08:45:48.231790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:02:47.614562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968701+0000","last_change":"2024-09-19T08:43:52.772501+0000","last_active":"2024-09-19T08:45:47.968701+0000","last_peered":"2024-09-19T08:45:47.968701+0000","last_clean":"2024-09-19T08:45:47.968701+0000","last_became_active":"2024-09-19T08:43:52.771611+0000","last_became_peered":"2024-09-19T08:43:52.771611+0000","last_unstale":"2024-09-19T08:45:47.968701+0000","last_undegraded":"2024-09-19T08:45:47.968701+0000","last_fullsized":"2024-09-19T08:45:47.968701+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:49.537237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968630+0000","last_change":"2024-09-19T08:43:52.773664+0000","last_active":"2024-09-19T08:45:47.968630+0000","last_peered":"2024-09-19T08:45:47.968630+0000","last_clean":"2024-09-19T08:45:47.968630+0000","last_became_active":"2024-09-19T08:43:52.773014+0000","last_became_peered":"2024-09-19T08:43:52.773014+0000","last_unstale":"2024-09-19T08:45:47.968630+0000","last_undegraded":"2024-09-19T08:45:47.968630+0000","last_fullsized":"2024-09-19T08:45:47.968630+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:03.448825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229497+0000","last_change":"2024-09-19T08:43:52.769578+0000","last_active":"2024-09-19T08:45:48.229497+0000","last_peered":"2024-09-19T08:45:48.229497+0000","last_clean":"2024-09-19T08:45:48.229497+0000","last_became_active":"2024-09-19T08:43:52.769252+0000","last_became_peered":"2024-09-19T08:43:52.769252+0000","last_unstale":"2024-09-19T08:45:48.229497+0000","last_undegraded":"2024-09-19T08:45:48.229497+0000","last_fullsized":"2024-09-19T08:45:48.229497+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:27.936118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065844+0000","last_change":"2024-09-19T08:43:52.770453+0000","last_active":"2024-09-19T08:45:48.065844+0000","last_peered":"2024-09-19T08:45:48.065844+0000","last_clean":"2024-09-19T08:45:48.065844+0000","last_became_active":"2024-09-19T08:43:52.770155+0000","last_became_peered":"2024-09-19T08:43:52.770155+0000","last_unstale":"2024-09-19T08:45:48.065844+0000","last_undegraded":"2024-09-19T08:45:48.065844+0000","last_fullsized":"2024-09-19T08:45:48.065844+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:12.893572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'8","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.782770+0000","last_change":"2024-09-19T08:43:50.751316+0000","last_active":"2024-09-19T08:45:48.782770+0000","last_peered":"2024-09-19T08:45:48.782770+0000","last_clean":"2024-09-19T08:45:48.782770+0000","last_became_active":"2024-09-19T08:43:50.750972+0000","last_became_peered":"2024-09-19T08:43:50.750972+0000","last_unstale":"2024-09-19T08:45:48.782770+0000","last_undegraded":"2024-09-19T08:45:48.782770+0000","last_fullsized":"2024-09-19T08:45:48.782770+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:40.654729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1549,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074105+0000","last_change":"2024-09-19T08:43:52.773979+0000","last_active":"2024-09-19T08:45:48.074105+0000","last_peered":"2024-09-19T08:45:48.074105+0000","last_clean":"2024-09-19T08:45:48.074105+0000","last_became_active":"2024-09-19T08:43:52.773648+0000","last_became_peered":"2024-09-19T08:43:52.773648+0000","last_unstale":"2024-09-19T08:45:48.074105+0000","last_undegraded":"2024-09-19T08:45:48.074105+0000","last_fullsized":"2024-09-19T08:45:48.074105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:36.061209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.076810+0000","last_change":"2024-09-19T08:43:50.750704+0000","last_active":"2024-09-19T08:45:48.076810+0000","last_peered":"2024-09-19T08:45:48.076810+0000","last_clean":"2024-09-19T08:45:48.076810+0000","last_became_active":"2024-09-19T08:43:50.750399+0000","last_became_peered":"2024-09-19T08:43:50.750399+0000","last_unstale":"2024-09-19T08:45:48.076810+0000","last_undegraded":"2024-09-19T08:45:48.076810+0000","last_fullsized":"2024-09-19T08:45:48.076810+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:25.447320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.223615+0000","last_change":"2024-09-19T08:43:50.744919+0000","last_active":"2024-09-19T08:45:48.223615+0000","last_peered":"2024-09-19T08:45:48.223615+0000","last_clean":"2024-09-19T08:45:48.223615+0000","last_became_active":"2024-09-19T08:43:50.744604+0000","last_became_peered":"2024-09-19T08:43:50.744604+0000","last_unstale":"2024-09-19T08:45:48.223615+0000","last_undegraded":"2024-09-19T08:45:48.223615+0000","last_fullsized":"2024-09-19T08:45:48.223615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:25:06.410093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065913+0000","last_change":"2024-09-19T08:43:52.766779+0000","last_active":"2024-09-19T08:45:48.065913+0000","last_peered":"2024-09-19T08:45:48.065913+0000","last_clean":"2024-09-19T08:45:48.065913+0000","last_became_active":"2024-09-19T08:43:52.766484+0000","last_became_peered":"2024-09-19T08:43:52.766484+0000","last_unstale":"2024-09-19T08:45:48.065913+0000","last_undegraded":"2024-09-19T08:45:48.065913+0000","last_fullsized":"2024-09-19T08:45:48.065913+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:17:29.350778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074289+0000","last_change":"2024-09-19T08:43:50.751282+0000","last_active":"2024-09-19T08:45:48.074289+0000","last_peered":"2024-09-19T08:45:48.074289+0000","last_clean":"2024-09-19T08:45:48.074289+0000","last_became_active":"2024-09-19T08:43:50.750754+0000","last_became_peered":"2024-09-19T08:43:50.750754+0000","last_unstale":"2024-09-19T08:45:48.074289+0000","last_undegraded":"2024-09-19T08:45:48.074289+0000","last_fullsized":"2024-09-19T08:45:48.074289+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:11.661626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.967219+0000","last_change":"2024-09-19T08:43:52.756442+0000","last_active":"2024-09-19T08:45:47.967219+0000","last_peered":"2024-09-19T08:45:47.967219+0000","last_clean":"2024-09-19T08:45:47.967219+0000","last_became_active":"2024-09-19T08:43:52.756116+0000","last_became_peered":"2024-09-19T08:43:52.756116+0000","last_unstale":"2024-09-19T08:45:47.967219+0000","last_undegraded":"2024-09-19T08:45:47.967219+0000","last_fullsized":"2024-09-19T08:45:47.967219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:40.029272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'8","reported_seq":242,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.782133+0000","last_change":"2024-09-19T08:43:50.749637+0000","last_active":"2024-09-19T08:45:48.782133+0000","last_peered":"2024-09-19T08:45:48.782133+0000","last_clean":"2024-09-19T08:45:48.782133+0000","last_became_active":"2024-09-19T08:43:50.749307+0000","last_became_peered":"2024-09-19T08:43:50.749307+0000","last_unstale":"2024-09-19T08:45:48.782133+0000","last_undegraded":"2024-09-19T08:45:48.782133+0000","last_fullsized":"2024-09-19T08:45:48.782133+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:23:33.409671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":210,"num_read_kb":210,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.222016+0000","last_change":"2024-09-19T08:43:52.763592+0000","last_active":"2024-09-19T08:45:48.222016+0000","last_peered":"2024-09-19T08:45:48.222016+0000","last_clean":"2024-09-19T08:45:48.222016+0000","last_became_active":"2024-09-19T08:43:52.763146+0000","last_became_peered":"2024-09-19T08:43:52.763146+0000","last_unstale":"2024-09-19T08:45:48.222016+0000","last_undegraded":"2024-09-19T08:45:48.222016+0000","last_fullsized":"2024-09-19T08:45:48.222016+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:32.194465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221748+0000","last_change":"2024-09-19T08:43:50.749184+0000","last_active":"2024-09-19T08:45:48.221748+0000","last_peered":"2024-09-19T08:45:48.221748+0000","last_clean":"2024-09-19T08:45:48.221748+0000","last_became_active":"2024-09-19T08:43:50.748844+0000","last_became_peered":"2024-09-19T08:43:50.748844+0000","last_unstale":"2024-09-19T08:45:48.221748+0000","last_undegraded":"2024-09-19T08:45:48.221748+0000","last_fullsized":"2024-09-19T08:45:48.221748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:15.334759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.231209+0000","last_change":"2024-09-19T08:43:52.768696+0000","last_active":"2024-09-19T08:45:48.231209+0000","last_peered":"2024-09-19T08:45:48.231209+0000","last_clean":"2024-09-19T08:45:48.231209+0000","last_became_active":"2024-09-19T08:43:52.768374+0000","last_became_peered":"2024-09-19T08:43:52.768374+0000","last_unstale":"2024-09-19T08:45:48.231209+0000","last_undegraded":"2024-09-19T08:45:48.231209+0000","last_fullsized":"2024-09-19T08:45:48.231209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:34:39.343041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074342+0000","last_change":"2024-09-19T08:43:51.026270+0000","last_active":"2024-09-19T08:45:48.074342+0000","last_peered":"2024-09-19T08:45:48.074342+0000","last_clean":"2024-09-19T08:45:48.074342+0000","last_became_active":"2024-09-19T08:43:51.025823+0000","last_became_peered":"2024-09-19T08:43:51.025823+0000","last_unstale":"2024-09-19T08:45:48.074342+0000","last_undegraded":"2024-09-19T08:45:48.074342+0000","last_fullsized":"2024-09-19T08:45:48.074342+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:00.480438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221712+0000","last_change":"2024-09-19T08:43:52.759555+0000","last_active":"2024-09-19T08:45:48.221712+0000","last_peered":"2024-09-19T08:45:48.221712+0000","last_clean":"2024-09-19T08:45:48.221712+0000","last_became_active":"2024-09-19T08:43:52.759223+0000","last_became_peered":"2024-09-19T08:43:52.759223+0000","last_unstale":"2024-09-19T08:45:48.221712+0000","last_undegraded":"2024-09-19T08:45:48.221712+0000","last_fullsized":"2024-09-19T08:45:48.221712+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:57:06.451323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'36","reported_seq":283,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.782998+0000","last_change":"2024-09-19T08:43:50.751924+0000","last_active":"2024-09-19T08:45:48.782998+0000","last_peered":"2024-09-19T08:45:48.782998+0000","last_clean":"2024-09-19T08:45:48.782998+0000","last_became_active":"2024-09-19T08:43:50.751117+0000","last_became_peered":"2024-09-19T08:43:50.751117+0000","last_unstale":"2024-09-19T08:45:48.782998+0000","last_undegraded":"2024-09-19T08:45:48.782998+0000","last_fullsized":"2024-09-19T08:45:48.782998+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:33:57.196283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65927,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":223,"num_read_kb":176,"num_write":38,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066156+0000","last_change":"2024-09-19T08:43:52.762260+0000","last_active":"2024-09-19T08:45:48.066156+0000","last_peered":"2024-09-19T08:45:48.066156+0000","last_clean":"2024-09-19T08:45:48.066156+0000","last_became_active":"2024-09-19T08:43:52.761923+0000","last_became_peered":"2024-09-19T08:43:52.761923+0000","last_unstale":"2024-09-19T08:45:48.066156+0000","last_undegraded":"2024-09-19T08:45:48.066156+0000","last_fullsized":"2024-09-19T08:45:48.066156+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:39:39.503332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968758+0000","last_change":"2024-09-19T08:43:50.749745+0000","last_active":"2024-09-19T08:45:47.968758+0000","last_peered":"2024-09-19T08:45:47.968758+0000","last_clean":"2024-09-19T08:45:47.968758+0000","last_became_active":"2024-09-19T08:43:50.749073+0000","last_became_peered":"2024-09-19T08:43:50.749073+0000","last_unstale":"2024-09-19T08:45:47.968758+0000","last_undegraded":"2024-09-19T08:45:47.968758+0000","last_fullsized":"2024-09-19T08:45:47.968758+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:48:11.965841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.223734+0000","last_change":"2024-09-19T08:43:52.761495+0000","last_active":"2024-09-19T08:45:48.223734+0000","last_peered":"2024-09-19T08:45:48.223734+0000","last_clean":"2024-09-19T08:45:48.223734+0000","last_became_active":"2024-09-19T08:43:52.760425+0000","last_became_peered":"2024-09-19T08:43:52.760425+0000","last_unstale":"2024-09-19T08:45:48.223734+0000","last_undegraded":"2024-09-19T08:45:48.223734+0000","last_fullsized":"2024-09-19T08:45:48.223734+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:57.689284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074090+0000","last_change":"2024-09-19T08:43:50.749563+0000","last_active":"2024-09-19T08:45:48.074090+0000","last_peered":"2024-09-19T08:45:48.074090+0000","last_clean":"2024-09-19T08:45:48.074090+0000","last_became_active":"2024-09-19T08:43:50.749244+0000","last_became_peered":"2024-09-19T08:43:50.749244+0000","last_unstale":"2024-09-19T08:45:48.074090+0000","last_undegraded":"2024-09-19T08:45:48.074090+0000","last_fullsized":"2024-09-19T08:45:48.074090+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:59:02.312883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221429+0000","last_change":"2024-09-19T08:43:52.767087+0000","last_active":"2024-09-19T08:45:48.221429+0000","last_peered":"2024-09-19T08:45:48.221429+0000","last_clean":"2024-09-19T08:45:48.221429+0000","last_became_active":"2024-09-19T08:43:52.766730+0000","last_became_peered":"2024-09-19T08:43:52.766730+0000","last_unstale":"2024-09-19T08:45:48.221429+0000","last_undegraded":"2024-09-19T08:45:48.221429+0000","last_fullsized":"2024-09-19T08:45:48.221429+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:04.040896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.966237+0000","last_change":"2024-09-19T08:43:50.749659+0000","last_active":"2024-09-19T08:45:47.966237+0000","last_peered":"2024-09-19T08:45:47.966237+0000","last_clean":"2024-09-19T08:45:47.966237+0000","last_became_active":"2024-09-19T08:43:50.749314+0000","last_became_peered":"2024-09-19T08:43:50.749314+0000","last_unstale":"2024-09-19T08:45:47.966237+0000","last_undegraded":"2024-09-19T08:45:47.966237+0000","last_fullsized":"2024-09-19T08:45:47.966237+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:08.576209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221691+0000","last_change":"2024-09-19T08:43:52.767928+0000","last_active":"2024-09-19T08:45:48.221691+0000","last_peered":"2024-09-19T08:45:48.221691+0000","last_clean":"2024-09-19T08:45:48.221691+0000","last_became_active":"2024-09-19T08:43:52.767580+0000","last_became_peered":"2024-09-19T08:43:52.767580+0000","last_unstale":"2024-09-19T08:45:48.221691+0000","last_undegraded":"2024-09-19T08:45:48.221691+0000","last_fullsized":"2024-09-19T08:45:48.221691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:45.344693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229348+0000","last_change":"2024-09-19T08:44:02.833887+0000","last_active":"2024-09-19T08:45:48.229348+0000","last_peered":"2024-09-19T08:45:48.229348+0000","last_clean":"2024-09-19T08:45:48.229348+0000","last_became_active":"2024-09-19T08:44:02.821624+0000","last_became_peered":"2024-09-19T08:44:02.821624+0000","last_unstale":"2024-09-19T08:45:48.229348+0000","last_undegraded":"2024-09-19T08:45:48.229348+0000","last_fullsized":"2024-09-19T08:45:48.229348+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:54:36.131949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.967019+0000","last_change":"2024-09-19T08:44:03.080503+0000","last_active":"2024-09-19T08:45:47.967019+0000","last_peered":"2024-09-19T08:45:47.967019+0000","last_clean":"2024-09-19T08:45:47.967019+0000","last_became_active":"2024-09-19T08:44:03.080129+0000","last_became_peered":"2024-09-19T08:44:03.080129+0000","last_unstale":"2024-09-19T08:45:47.967019+0000","last_undegraded":"2024-09-19T08:45:47.967019+0000","last_fullsized":"2024-09-19T08:45:47.967019+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:01:37.034795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066192+0000","last_change":"2024-09-19T08:43:50.747624+0000","last_active":"2024-09-19T08:45:48.066192+0000","last_peered":"2024-09-19T08:45:48.066192+0000","last_clean":"2024-09-19T08:45:48.066192+0000","last_became_active":"2024-09-19T08:43:50.747263+0000","last_became_peered":"2024-09-19T08:43:50.747263+0000","last_unstale":"2024-09-19T08:45:48.066192+0000","last_undegraded":"2024-09-19T08:45:48.066192+0000","last_fullsized":"2024-09-19T08:45:48.066192+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:18.580106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229525+0000","last_change":"2024-09-19T08:43:52.770051+0000","last_active":"2024-09-19T08:45:48.229525+0000","last_peered":"2024-09-19T08:45:48.229525+0000","last_clean":"2024-09-19T08:45:48.229525+0000","last_became_active":"2024-09-19T08:43:52.769727+0000","last_became_peered":"2024-09-19T08:43:52.769727+0000","last_unstale":"2024-09-19T08:45:48.229525+0000","last_undegraded":"2024-09-19T08:45:48.229525+0000","last_fullsized":"2024-09-19T08:45:48.229525+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:34.954897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.223984+0000","last_change":"2024-09-19T08:43:50.746696+0000","last_active":"2024-09-19T08:45:48.223984+0000","last_peered":"2024-09-19T08:45:48.223984+0000","last_clean":"2024-09-19T08:45:48.223984+0000","last_became_active":"2024-09-19T08:43:50.746246+0000","last_became_peered":"2024-09-19T08:43:50.746246+0000","last_unstale":"2024-09-19T08:45:48.223984+0000","last_undegraded":"2024-09-19T08:45:48.223984+0000","last_fullsized":"2024-09-19T08:45:48.223984+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:38:39.935348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.222174+0000","last_change":"2024-09-19T08:43:52.765199+0000","last_active":"2024-09-19T08:45:48.222174+0000","last_peered":"2024-09-19T08:45:48.222174+0000","last_clean":"2024-09-19T08:45:48.222174+0000","last_became_active":"2024-09-19T08:43:52.764896+0000","last_became_peered":"2024-09-19T08:43:52.764896+0000","last_unstale":"2024-09-19T08:45:48.222174+0000","last_undegraded":"2024-09-19T08:45:48.222174+0000","last_fullsized":"2024-09-19T08:45:48.222174+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:02:59.570018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'41","reported_seq":73,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.186388+0000","last_change":"2024-09-19T08:43:50.747242+0000","last_active":"2024-09-19T08:45:48.186388+0000","last_peered":"2024-09-19T08:45:48.186388+0000","last_clean":"2024-09-19T08:45:48.186388+0000","last_became_active":"2024-09-19T08:43:50.746853+0000","last_became_peered":"2024-09-19T08:43:50.746853+0000","last_unstale":"2024-09-19T08:45:48.186388+0000","last_undegraded":"2024-09-19T08:45:48.186388+0000","last_fullsized":"2024-09-19T08:45:48.186388+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:21.545635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221579+0000","last_change":"2024-09-19T08:44:02.826535+0000","last_active":"2024-09-19T08:45:48.221579+0000","last_peered":"2024-09-19T08:45:48.221579+0000","last_clean":"2024-09-19T08:45:48.221579+0000","last_became_active":"2024-09-19T08:44:02.826113+0000","last_became_peered":"2024-09-19T08:44:02.826113+0000","last_unstale":"2024-09-19T08:45:48.221579+0000","last_undegraded":"2024-09-19T08:45:48.221579+0000","last_fullsized":"2024-09-19T08:45:48.221579+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:20.247732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'8","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.782660+0000","last_change":"2024-09-19T08:43:51.131540+0000","last_active":"2024-09-19T08:45:48.782660+0000","last_peered":"2024-09-19T08:45:48.782660+0000","last_clean":"2024-09-19T08:45:48.782660+0000","last_became_active":"2024-09-19T08:43:51.131170+0000","last_became_peered":"2024-09-19T08:43:51.131170+0000","last_unstale":"2024-09-19T08:45:48.782660+0000","last_undegraded":"2024-09-19T08:45:48.782660+0000","last_fullsized":"2024-09-19T08:45:48.782660+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:43.635612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221593+0000","last_change":"2024-09-19T08:43:52.763308+0000","last_active":"2024-09-19T08:45:48.221593+0000","last_peered":"2024-09-19T08:45:48.221593+0000","last_clean":"2024-09-19T08:45:48.221593+0000","last_became_active":"2024-09-19T08:43:52.762859+0000","last_became_peered":"2024-09-19T08:43:52.762859+0000","last_unstale":"2024-09-19T08:45:48.221593+0000","last_undegraded":"2024-09-19T08:45:48.221593+0000","last_fullsized":"2024-09-19T08:45:48.221593+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:32:35.083361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":350,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221461+0000","last_change":"2024-09-19T08:42:39.567408+0000","last_active":"2024-09-19T08:45:48.221461+0000","last_peered":"2024-09-19T08:45:48.221461+0000","last_clean":"2024-09-19T08:45:48.221461+0000","last_became_active":"2024-09-19T08:42:39.566967+0000","last_became_peered":"2024-09-19T08:42:39.566967+0000","last_unstale":"2024-09-19T08:45:48.221461+0000","last_undegraded":"2024-09-19T08:45:48.221461+0000","last_fullsized":"2024-09-19T08:45:48.221461+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:14.287944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:14.287944+0000","last_clean_scrub_stamp":"2024-09-19T08:39:14.287944+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:31:03.245386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.966263+0000","last_change":"2024-09-19T08:43:50.751097+0000","last_active":"2024-09-19T08:45:47.966263+0000","last_peered":"2024-09-19T08:45:47.966263+0000","last_clean":"2024-09-19T08:45:47.966263+0000","last_became_active":"2024-09-19T08:43:50.750341+0000","last_became_peered":"2024-09-19T08:43:50.750341+0000","last_unstale":"2024-09-19T08:45:47.966263+0000","last_undegraded":"2024-09-19T08:45:47.966263+0000","last_fullsized":"2024-09-19T08:45:47.966263+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:25.694535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066373+0000","last_change":"2024-09-19T08:43:52.767905+0000","last_active":"2024-09-19T08:45:48.066373+0000","last_peered":"2024-09-19T08:45:48.066373+0000","last_clean":"2024-09-19T08:45:48.066373+0000","last_became_active":"2024-09-19T08:43:52.767601+0000","last_became_peered":"2024-09-19T08:43:52.767601+0000","last_unstale":"2024-09-19T08:45:48.066373+0000","last_undegraded":"2024-09-19T08:45:48.066373+0000","last_fullsized":"2024-09-19T08:45:48.066373+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:14.519879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221338+0000","last_change":"2024-09-19T08:43:50.751397+0000","last_active":"2024-09-19T08:45:48.221338+0000","last_peered":"2024-09-19T08:45:48.221338+0000","last_clean":"2024-09-19T08:45:48.221338+0000","last_became_active":"2024-09-19T08:43:50.751079+0000","last_became_peered":"2024-09-19T08:43:50.751079+0000","last_unstale":"2024-09-19T08:45:48.221338+0000","last_undegraded":"2024-09-19T08:45:48.221338+0000","last_fullsized":"2024-09-19T08:45:48.221338+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:29.904786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.223616+0000","last_change":"2024-09-19T08:43:52.773348+0000","last_active":"2024-09-19T08:45:48.223616+0000","last_peered":"2024-09-19T08:45:48.223616+0000","last_clean":"2024-09-19T08:45:48.223616+0000","last_became_active":"2024-09-19T08:43:52.773011+0000","last_became_peered":"2024-09-19T08:43:52.773011+0000","last_unstale":"2024-09-19T08:45:48.223616+0000","last_undegraded":"2024-09-19T08:45:48.223616+0000","last_fullsized":"2024-09-19T08:45:48.223616+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:10.806578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":135,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.223789+0000","last_change":"2024-09-19T08:43:51.025300+0000","last_active":"2024-09-19T08:45:48.223789+0000","last_peered":"2024-09-19T08:45:48.223789+0000","last_clean":"2024-09-19T08:45:48.223789+0000","last_became_active":"2024-09-19T08:43:51.024936+0000","last_became_peered":"2024-09-19T08:43:51.024936+0000","last_unstale":"2024-09-19T08:45:48.223789+0000","last_undegraded":"2024-09-19T08:45:48.223789+0000","last_fullsized":"2024-09-19T08:45:48.223789+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:03.186066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074063+0000","last_change":"2024-09-19T08:43:52.774962+0000","last_active":"2024-09-19T08:45:48.074063+0000","last_peered":"2024-09-19T08:45:48.074063+0000","last_clean":"2024-09-19T08:45:48.074063+0000","last_became_active":"2024-09-19T08:43:52.774598+0000","last_became_peered":"2024-09-19T08:43:52.774598+0000","last_unstale":"2024-09-19T08:45:48.074063+0000","last_undegraded":"2024-09-19T08:45:48.074063+0000","last_fullsized":"2024-09-19T08:45:48.074063+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:02.687136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'7","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.782526+0000","last_change":"2024-09-19T08:43:50.748180+0000","last_active":"2024-09-19T08:45:48.782526+0000","last_peered":"2024-09-19T08:45:48.782526+0000","last_clean":"2024-09-19T08:45:48.782526+0000","last_became_active":"2024-09-19T08:43:50.747853+0000","last_became_peered":"2024-09-19T08:43:50.747853+0000","last_unstale":"2024-09-19T08:45:48.782526+0000","last_undegraded":"2024-09-19T08:45:48.782526+0000","last_fullsized":"2024-09-19T08:45:48.782526+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:56.047259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.077066+0000","last_change":"2024-09-19T08:43:52.770253+0000","last_active":"2024-09-19T08:45:48.077066+0000","last_peered":"2024-09-19T08:45:48.077066+0000","last_clean":"2024-09-19T08:45:48.077066+0000","last_became_active":"2024-09-19T08:43:52.769951+0000","last_became_peered":"2024-09-19T08:43:52.769951+0000","last_unstale":"2024-09-19T08:45:48.077066+0000","last_undegraded":"2024-09-19T08:45:48.077066+0000","last_fullsized":"2024-09-19T08:45:48.077066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:02.353144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.967260+0000","last_change":"2024-09-19T08:43:51.026453+0000","last_active":"2024-09-19T08:45:47.967260+0000","last_peered":"2024-09-19T08:45:47.967260+0000","last_clean":"2024-09-19T08:45:47.967260+0000","last_became_active":"2024-09-19T08:43:51.025873+0000","last_became_peered":"2024-09-19T08:43:51.025873+0000","last_unstale":"2024-09-19T08:45:47.967260+0000","last_undegraded":"2024-09-19T08:45:47.967260+0000","last_fullsized":"2024-09-19T08:45:47.967260+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:25:17.475074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.076927+0000","last_change":"2024-09-19T08:43:52.761700+0000","last_active":"2024-09-19T08:45:48.076927+0000","last_peered":"2024-09-19T08:45:48.076927+0000","last_clean":"2024-09-19T08:45:48.076927+0000","last_became_active":"2024-09-19T08:43:52.755527+0000","last_became_peered":"2024-09-19T08:43:52.755527+0000","last_unstale":"2024-09-19T08:45:48.076927+0000","last_undegraded":"2024-09-19T08:45:48.076927+0000","last_fullsized":"2024-09-19T08:45:48.076927+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:18:43.171284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066299+0000","last_change":"2024-09-19T08:43:51.026136+0000","last_active":"2024-09-19T08:45:48.066299+0000","last_peered":"2024-09-19T08:45:48.066299+0000","last_clean":"2024-09-19T08:45:48.066299+0000","last_became_active":"2024-09-19T08:43:51.025740+0000","last_became_peered":"2024-09-19T08:43:51.025740+0000","last_unstale":"2024-09-19T08:45:48.066299+0000","last_undegraded":"2024-09-19T08:45:48.066299+0000","last_fullsized":"2024-09-19T08:45:48.066299+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:17:32.741083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.224195+0000","last_change":"2024-09-19T08:43:52.763808+0000","last_active":"2024-09-19T08:45:48.224195+0000","last_peered":"2024-09-19T08:45:48.224195+0000","last_clean":"2024-09-19T08:45:48.224195+0000","last_became_active":"2024-09-19T08:43:52.763509+0000","last_became_peered":"2024-09-19T08:43:52.763509+0000","last_unstale":"2024-09-19T08:45:48.224195+0000","last_undegraded":"2024-09-19T08:45:48.224195+0000","last_fullsized":"2024-09-19T08:45:48.224195+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:45:08.385419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'51","reported_seq":98,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.189380+0000","last_change":"2024-09-19T08:43:50.744213+0000","last_active":"2024-09-19T08:45:48.189380+0000","last_peered":"2024-09-19T08:45:48.189380+0000","last_clean":"2024-09-19T08:45:48.189380+0000","last_became_active":"2024-09-19T08:43:50.743848+0000","last_became_peered":"2024-09-19T08:43:50.743848+0000","last_unstale":"2024-09-19T08:45:48.189380+0000","last_undegraded":"2024-09-19T08:45:48.189380+0000","last_fullsized":"2024-09-19T08:45:48.189380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:26.613055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":90,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.067394+0000","last_change":"2024-09-19T08:43:52.767102+0000","last_active":"2024-09-19T08:45:48.067394+0000","last_peered":"2024-09-19T08:45:48.067394+0000","last_clean":"2024-09-19T08:45:48.067394+0000","last_became_active":"2024-09-19T08:43:52.766262+0000","last_became_peered":"2024-09-19T08:43:52.766262+0000","last_unstale":"2024-09-19T08:45:48.067394+0000","last_undegraded":"2024-09-19T08:45:48.067394+0000","last_fullsized":"2024-09-19T08:45:48.067394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:35.740462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.966479+0000","last_change":"2024-09-19T08:43:51.131742+0000","last_active":"2024-09-19T08:45:47.966479+0000","last_peered":"2024-09-19T08:45:47.966479+0000","last_clean":"2024-09-19T08:45:47.966479+0000","last_became_active":"2024-09-19T08:43:51.131388+0000","last_became_peered":"2024-09-19T08:43:51.131388+0000","last_unstale":"2024-09-19T08:45:47.966479+0000","last_undegraded":"2024-09-19T08:45:47.966479+0000","last_fullsized":"2024-09-19T08:45:47.966479+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:26.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.230731+0000","last_change":"2024-09-19T08:43:52.756004+0000","last_active":"2024-09-19T08:45:48.230731+0000","last_peered":"2024-09-19T08:45:48.230731+0000","last_clean":"2024-09-19T08:45:48.230731+0000","last_became_active":"2024-09-19T08:43:52.755596+0000","last_became_peered":"2024-09-19T08:43:52.755596+0000","last_unstale":"2024-09-19T08:45:48.230731+0000","last_undegraded":"2024-09-19T08:45:48.230731+0000","last_fullsized":"2024-09-19T08:45:48.230731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:35.942375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221620+0000","last_change":"2024-09-19T08:44:02.835986+0000","last_active":"2024-09-19T08:45:48.221620+0000","last_peered":"2024-09-19T08:45:48.221620+0000","last_clean":"2024-09-19T08:45:48.221620+0000","last_became_active":"2024-09-19T08:44:02.826314+0000","last_became_peered":"2024-09-19T08:44:02.826314+0000","last_unstale":"2024-09-19T08:45:48.221620+0000","last_undegraded":"2024-09-19T08:45:48.221620+0000","last_fullsized":"2024-09-19T08:45:48.221620+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:16.943989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065838+0000","last_change":"2024-09-19T08:43:52.766582+0000","last_active":"2024-09-19T08:45:48.065838+0000","last_peered":"2024-09-19T08:45:48.065838+0000","last_clean":"2024-09-19T08:45:48.065838+0000","last_became_active":"2024-09-19T08:43:52.766232+0000","last_became_peered":"2024-09-19T08:43:52.766232+0000","last_unstale":"2024-09-19T08:45:48.065838+0000","last_undegraded":"2024-09-19T08:45:48.065838+0000","last_fullsized":"2024-09-19T08:45:48.065838+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:43:42.319422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066195+0000","last_change":"2024-09-19T08:43:50.747611+0000","last_active":"2024-09-19T08:45:48.066195+0000","last_peered":"2024-09-19T08:45:48.066195+0000","last_clean":"2024-09-19T08:45:48.066195+0000","last_became_active":"2024-09-19T08:43:50.747263+0000","last_became_peered":"2024-09-19T08:43:50.747263+0000","last_unstale":"2024-09-19T08:45:48.066195+0000","last_undegraded":"2024-09-19T08:45:48.066195+0000","last_fullsized":"2024-09-19T08:45:48.066195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:14.717820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229532+0000","last_change":"2024-09-19T08:43:52.766493+0000","last_active":"2024-09-19T08:45:48.229532+0000","last_peered":"2024-09-19T08:45:48.229532+0000","last_clean":"2024-09-19T08:45:48.229532+0000","last_became_active":"2024-09-19T08:43:52.766158+0000","last_became_peered":"2024-09-19T08:43:52.766158+0000","last_unstale":"2024-09-19T08:45:48.229532+0000","last_undegraded":"2024-09-19T08:45:48.229532+0000","last_fullsized":"2024-09-19T08:45:48.229532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:59.175949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968841+0000","last_change":"2024-09-19T08:43:50.751368+0000","last_active":"2024-09-19T08:45:47.968841+0000","last_peered":"2024-09-19T08:45:47.968841+0000","last_clean":"2024-09-19T08:45:47.968841+0000","last_became_active":"2024-09-19T08:43:50.751038+0000","last_became_peered":"2024-09-19T08:43:50.751038+0000","last_unstale":"2024-09-19T08:45:47.968841+0000","last_undegraded":"2024-09-19T08:45:47.968841+0000","last_fullsized":"2024-09-19T08:45:47.968841+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:46.302996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066722+0000","last_change":"2024-09-19T08:43:52.756998+0000","last_active":"2024-09-19T08:45:48.066722+0000","last_peered":"2024-09-19T08:45:48.066722+0000","last_clean":"2024-09-19T08:45:48.066722+0000","last_became_active":"2024-09-19T08:43:52.756661+0000","last_became_peered":"2024-09-19T08:43:52.756661+0000","last_unstale":"2024-09-19T08:45:48.066722+0000","last_undegraded":"2024-09-19T08:45:48.066722+0000","last_fullsized":"2024-09-19T08:45:48.066722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:15.182574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065908+0000","last_change":"2024-09-19T08:43:50.750095+0000","last_active":"2024-09-19T08:45:48.065908+0000","last_peered":"2024-09-19T08:45:48.065908+0000","last_clean":"2024-09-19T08:45:48.065908+0000","last_became_active":"2024-09-19T08:43:50.749672+0000","last_became_peered":"2024-09-19T08:43:50.749672+0000","last_unstale":"2024-09-19T08:45:48.065908+0000","last_undegraded":"2024-09-19T08:45:48.065908+0000","last_fullsized":"2024-09-19T08:45:48.065908+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:25:34.978094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221996+0000","last_change":"2024-09-19T08:43:52.764097+0000","last_active":"2024-09-19T08:45:48.221996+0000","last_peered":"2024-09-19T08:45:48.221996+0000","last_clean":"2024-09-19T08:45:48.221996+0000","last_became_active":"2024-09-19T08:43:52.763773+0000","last_became_peered":"2024-09-19T08:43:52.763773+0000","last_unstale":"2024-09-19T08:45:48.221996+0000","last_undegraded":"2024-09-19T08:45:48.221996+0000","last_fullsized":"2024-09-19T08:45:48.221996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:01:42.098460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074171+0000","last_change":"2024-09-19T08:44:02.825238+0000","last_active":"2024-09-19T08:45:48.074171+0000","last_peered":"2024-09-19T08:45:48.074171+0000","last_clean":"2024-09-19T08:45:48.074171+0000","last_became_active":"2024-09-19T08:44:02.824873+0000","last_became_peered":"2024-09-19T08:44:02.824873+0000","last_unstale":"2024-09-19T08:45:48.074171+0000","last_undegraded":"2024-09-19T08:45:48.074171+0000","last_fullsized":"2024-09-19T08:45:48.074171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:38.896082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066088+0000","last_change":"2024-09-19T08:43:50.748789+0000","last_active":"2024-09-19T08:45:48.066088+0000","last_peered":"2024-09-19T08:45:48.066088+0000","last_clean":"2024-09-19T08:45:48.066088+0000","last_became_active":"2024-09-19T08:43:50.748464+0000","last_became_peered":"2024-09-19T08:43:50.748464+0000","last_unstale":"2024-09-19T08:45:48.066088+0000","last_undegraded":"2024-09-19T08:45:48.066088+0000","last_fullsized":"2024-09-19T08:45:48.066088+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:36:45.905639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221312+0000","last_change":"2024-09-19T08:43:52.756608+0000","last_active":"2024-09-19T08:45:48.221312+0000","last_peered":"2024-09-19T08:45:48.221312+0000","last_clean":"2024-09-19T08:45:48.221312+0000","last_became_active":"2024-09-19T08:43:52.756227+0000","last_became_peered":"2024-09-19T08:43:52.756227+0000","last_unstale":"2024-09-19T08:45:48.221312+0000","last_undegraded":"2024-09-19T08:45:48.221312+0000","last_fullsized":"2024-09-19T08:45:48.221312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:29:58.085584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'13","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066253+0000","last_change":"2024-09-19T08:43:50.749135+0000","last_active":"2024-09-19T08:45:48.066253+0000","last_peered":"2024-09-19T08:45:48.066253+0000","last_clean":"2024-09-19T08:45:48.066253+0000","last_became_active":"2024-09-19T08:43:50.748755+0000","last_became_peered":"2024-09-19T08:43:50.748755+0000","last_unstale":"2024-09-19T08:45:48.066253+0000","last_undegraded":"2024-09-19T08:45:48.066253+0000","last_fullsized":"2024-09-19T08:45:48.066253+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:55.079780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.231077+0000","last_change":"2024-09-19T08:43:52.763277+0000","last_active":"2024-09-19T08:45:48.231077+0000","last_peered":"2024-09-19T08:45:48.231077+0000","last_clean":"2024-09-19T08:45:48.231077+0000","last_became_active":"2024-09-19T08:43:52.762941+0000","last_became_peered":"2024-09-19T08:43:52.762941+0000","last_unstale":"2024-09-19T08:45:48.231077+0000","last_undegraded":"2024-09-19T08:45:48.231077+0000","last_fullsized":"2024-09-19T08:45:48.231077+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:35.531946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.067406+0000","last_change":"2024-09-19T08:43:51.025350+0000","last_active":"2024-09-19T08:45:48.067406+0000","last_peered":"2024-09-19T08:45:48.067406+0000","last_clean":"2024-09-19T08:45:48.067406+0000","last_became_active":"2024-09-19T08:43:51.025046+0000","last_became_peered":"2024-09-19T08:43:51.025046+0000","last_unstale":"2024-09-19T08:45:48.067406+0000","last_undegraded":"2024-09-19T08:45:48.067406+0000","last_fullsized":"2024-09-19T08:45:48.067406+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:27.595545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066247+0000","last_change":"2024-09-19T08:43:52.767917+0000","last_active":"2024-09-19T08:45:48.066247+0000","last_peered":"2024-09-19T08:45:48.066247+0000","last_clean":"2024-09-19T08:45:48.066247+0000","last_became_active":"2024-09-19T08:43:52.767575+0000","last_became_peered":"2024-09-19T08:43:52.767575+0000","last_unstale":"2024-09-19T08:45:48.066247+0000","last_undegraded":"2024-09-19T08:45:48.066247+0000","last_fullsized":"2024-09-19T08:45:48.066247+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:05.950399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221116+0000","last_change":"2024-09-19T08:43:50.746400+0000","last_active":"2024-09-19T08:45:48.221116+0000","last_peered":"2024-09-19T08:45:48.221116+0000","last_clean":"2024-09-19T08:45:48.221116+0000","last_became_active":"2024-09-19T08:43:50.745891+0000","last_became_peered":"2024-09-19T08:43:50.745891+0000","last_unstale":"2024-09-19T08:45:48.221116+0000","last_undegraded":"2024-09-19T08:45:48.221116+0000","last_fullsized":"2024-09-19T08:45:48.221116+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:03.098202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599805,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1190,"num_read_kb":711,"num_write":374,"num_write_kb":1464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":712704,"data_stored":1799415,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":301,"ondisk_log_size":301,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":109335,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579941,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":446056,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808874,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1089536,"data_stored":1232787,"data_compressed":45500,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005102,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":962560,"data_stored":838280,"data_compressed":22045,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234035,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":946176,"data_stored":838429,"data_compressed":22045,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462968,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":983040,"data_stored":1039165,"data_compressed":39344,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593790,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":445450,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822722,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1015808,"data_stored":1169657,"data_compressed":45500,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986247,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":444525,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215180,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":969188,"data_compressed":28201,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1003520,"data_stored":513578,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":983040,"data_stored":512290,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":937984,"data_stored":446930,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":68465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":69753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135347,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":199263,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:45:51.463 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-19T08:45:51.463 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:45:51.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.607+0000 7fccacd68640 1 -- 172.21.15.164:0/2916760289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcca8102590 msgr2=0x7fcca8104980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:51.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.607+0000 7fccacd68640 1 --2- 172.21.15.164:0/2916760289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcca8102590 0x7fcca8104980 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fcc98009900 tx=0x7fcc9802a410 comp rx=0 tx=0).stop 2024-09-19T08:45:51.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.607+0000 7fccacd68640 1 -- 172.21.15.164:0/2916760289 shutdown_connections 2024-09-19T08:45:51.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.607+0000 7fccacd68640 1 --2- 172.21.15.164:0/2916760289 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcca81077f0 0x7fcca8109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.607+0000 7fccacd68640 1 --2- 172.21.15.164:0/2916760289 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcca8104ec0 0x7fcca81072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.607+0000 7fccacd68640 1 --2- 172.21.15.164:0/2916760289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcca8102590 0x7fcca8104980 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.607+0000 7fccacd68640 1 -- 172.21.15.164:0/2916760289 >> 172.21.15.164:0/2916760289 conn(0x7fcca80fc170 msgr2=0x7fcca80fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:51.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.607+0000 7fccacd68640 1 -- 172.21.15.164:0/2916760289 shutdown_connections 2024-09-19T08:45:51.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.607+0000 7fccacd68640 1 -- 172.21.15.164:0/2916760289 wait complete. 2024-09-19T08:45:51.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.608+0000 7fccacd68640 1 Processor -- start 2024-09-19T08:45:51.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.608+0000 7fccacd68640 1 -- start start 2024-09-19T08:45:51.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.608+0000 7fccacd68640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcca8102590 0x7fcca819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:51.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.608+0000 7fccacd68640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcca8104ec0 0x7fcca819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:51.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.608+0000 7fccacd68640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcca81077f0 0x7fcca81a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:51.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.608+0000 7fccacd68640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcca810d840 con 0x7fcca8102590 2024-09-19T08:45:51.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.608+0000 7fccacd68640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcca810d6c0 con 0x7fcca8104ec0 2024-09-19T08:45:51.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.608+0000 7fccacd68640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcca810d9c0 con 0x7fcca81077f0 2024-09-19T08:45:51.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.608+0000 7fcca659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcca8102590 0x7fcca819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:51.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.608+0000 7fcca659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcca8102590 0x7fcca819a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46586/0 (socket says 172.21.15.164:46586) 2024-09-19T08:45:51.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.608+0000 7fcca659c640 1 -- 172.21.15.164:0/121002825 learned_addr learned my addr 172.21.15.164:0/121002825 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:51.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.608+0000 7fcca5d9b640 1 --2- 172.21.15.164:0/121002825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcca8104ec0 0x7fcca819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:51.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.609+0000 7fcca6d9d640 1 --2- 172.21.15.164:0/121002825 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcca81077f0 0x7fcca81a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:51.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.609+0000 7fcca659c640 1 -- 172.21.15.164:0/121002825 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcca81077f0 msgr2=0x7fcca81a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:51.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.609+0000 7fcca659c640 1 --2- 172.21.15.164:0/121002825 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcca81077f0 0x7fcca81a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.609+0000 7fcca659c640 1 -- 172.21.15.164:0/121002825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcca8104ec0 msgr2=0x7fcca819a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:51.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.609+0000 7fcca659c640 1 --2- 172.21.15.164:0/121002825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcca8104ec0 0x7fcca819a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.609+0000 7fcca659c640 1 -- 172.21.15.164:0/121002825 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca81a1ce0 con 0x7fcca8102590 2024-09-19T08:45:51.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.609+0000 7fcca659c640 1 --2- 172.21.15.164:0/121002825 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcca8102590 0x7fcca819a0c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcc98031440 tx=0x7fcc98032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:51.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.609+0000 7fcc977fe640 1 -- 172.21.15.164:0/121002825 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9803f030 con 0x7fcca8102590 2024-09-19T08:45:51.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.609+0000 7fccacd68640 1 -- 172.21.15.164:0/121002825 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcca81a1f70 con 0x7fcca8102590 2024-09-19T08:45:51.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.609+0000 7fcc977fe640 1 -- 172.21.15.164:0/121002825 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc98003ce0 con 0x7fcca8102590 2024-09-19T08:45:51.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.609+0000 7fcc977fe640 1 -- 172.21.15.164:0/121002825 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc98035070 con 0x7fcca8102590 2024-09-19T08:45:51.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.609+0000 7fccacd68640 1 -- 172.21.15.164:0/121002825 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcca81a2380 con 0x7fcca8102590 2024-09-19T08:45:51.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.610+0000 7fccacd68640 1 -- 172.21.15.164:0/121002825 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcca8109330 con 0x7fcca8102590 2024-09-19T08:45:51.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.611+0000 7fcc977fe640 1 -- 172.21.15.164:0/121002825 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 23) ==== 138476+0+0 (secure 0 0 0) 0x7fcc98042030 con 0x7fcca8102590 2024-09-19T08:45:51.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.612+0000 7fcc977fe640 1 --2- 172.21.15.164:0/121002825 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fcc900a33c0 0x7fcc900a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:51.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.612+0000 7fcca5d9b640 1 --2- 172.21.15.164:0/121002825 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fcc900a33c0 0x7fcc900a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:51.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.612+0000 7fcc977fe640 1 -- 172.21.15.164:0/121002825 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fcc980f06c0 con 0x7fcca8102590 2024-09-19T08:45:51.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.613+0000 7fcca5d9b640 1 --2- 172.21.15.164:0/121002825 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fcc900a33c0 0x7fcc900a5880 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcca819b5d0 tx=0x7fcc9c005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:51.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.619+0000 7fcc977fe640 1 -- 172.21.15.164:0/121002825 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcc980bd110 con 0x7fcca8102590 2024-09-19T08:45:51.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.801+0000 7fccacd68640 1 -- 172.21.15.164:0/121002825 --> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcca8062630 con 0x7fcc900a33c0 2024-09-19T08:45:51.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.809+0000 7fcc977fe640 1 -- 172.21.15.164:0/121002825 <== mgr.15201 v2:172.21.15.164:6800/173486514 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258041 (secure 0 0 0) 0x7fcca8062630 con 0x7fcc900a33c0 2024-09-19T08:45:51.812 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:45:51.814 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-09-19T08:45:51.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.812+0000 7fccacd68640 1 -- 172.21.15.164:0/121002825 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fcc900a33c0 msgr2=0x7fcc900a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:51.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.812+0000 7fccacd68640 1 --2- 172.21.15.164:0/121002825 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fcc900a33c0 0x7fcc900a5880 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcca819b5d0 tx=0x7fcc9c005f90 comp rx=0 tx=0).stop 2024-09-19T08:45:51.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.812+0000 7fccacd68640 1 -- 172.21.15.164:0/121002825 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcca8102590 msgr2=0x7fcca819a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:51.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.812+0000 7fccacd68640 1 --2- 172.21.15.164:0/121002825 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcca8102590 0x7fcca819a0c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcc98031440 tx=0x7fcc98032ce0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.812+0000 7fccacd68640 1 -- 172.21.15.164:0/121002825 shutdown_connections 2024-09-19T08:45:51.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.812+0000 7fccacd68640 1 --2- 172.21.15.164:0/121002825 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcca81077f0 0x7fcca81a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.812+0000 7fccacd68640 1 --2- 172.21.15.164:0/121002825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcca8104ec0 0x7fcca819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.812+0000 7fccacd68640 1 --2- 172.21.15.164:0/121002825 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fcc900a33c0 0x7fcc900a5880 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.812+0000 7fccacd68640 1 --2- 172.21.15.164:0/121002825 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcca8102590 0x7fcca819a0c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.812+0000 7fccacd68640 1 -- 172.21.15.164:0/121002825 >> 172.21.15.164:0/121002825 conn(0x7fcca80fc170 msgr2=0x7fcca8101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:51.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.812+0000 7fccacd68640 1 -- 172.21.15.164:0/121002825 shutdown_connections 2024-09-19T08:45:51.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:51.812+0000 7fccacd68640 1 -- 172.21.15.164:0/121002825 wait complete. 2024-09-19T08:45:51.824 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-09-19T08:45:49.981301+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1190407,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1326,"num_read_kb":830,"num_write":557,"num_write_kb":4404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":940,"ondisk_log_size":940,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":389792,"kb_used_data":11376,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124552032,"statfs":{"total":1151940427776,"available":1151541280768,"internally_reserved":0,"allocated":11649024,"data_stored":8896335,"data_compressed":220029,"data_compressed_allocated":282624,"data_compressed_original":3416064,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.991213"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229715+0000","last_change":"2024-09-19T08:43:52.766709+0000","last_active":"2024-09-19T08:45:48.229715+0000","last_peered":"2024-09-19T08:45:48.229715+0000","last_clean":"2024-09-19T08:45:48.229715+0000","last_became_active":"2024-09-19T08:43:52.765970+0000","last_became_peered":"2024-09-19T08:43:52.765970+0000","last_unstale":"2024-09-19T08:45:48.229715+0000","last_undegraded":"2024-09-19T08:45:48.229715+0000","last_fullsized":"2024-09-19T08:45:48.229715+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:12.176870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.230605+0000","last_change":"2024-09-19T08:43:52.766590+0000","last_active":"2024-09-19T08:45:48.230605+0000","last_peered":"2024-09-19T08:45:48.230605+0000","last_clean":"2024-09-19T08:45:48.230605+0000","last_became_active":"2024-09-19T08:43:52.766255+0000","last_became_peered":"2024-09-19T08:43:52.766255+0000","last_unstale":"2024-09-19T08:45:48.230605+0000","last_undegraded":"2024-09-19T08:45:48.230605+0000","last_fullsized":"2024-09-19T08:45:48.230605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:15:41.359806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.230306+0000","last_change":"2024-09-19T08:43:52.761956+0000","last_active":"2024-09-19T08:45:48.230306+0000","last_peered":"2024-09-19T08:45:48.230306+0000","last_clean":"2024-09-19T08:45:48.230306+0000","last_became_active":"2024-09-19T08:43:52.761644+0000","last_became_peered":"2024-09-19T08:43:52.761644+0000","last_unstale":"2024-09-19T08:45:48.230306+0000","last_undegraded":"2024-09-19T08:45:48.230306+0000","last_fullsized":"2024-09-19T08:45:48.230306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:13.770361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.966146+0000","last_change":"2024-09-19T08:43:52.764131+0000","last_active":"2024-09-19T08:45:47.966146+0000","last_peered":"2024-09-19T08:45:47.966146+0000","last_clean":"2024-09-19T08:45:47.966146+0000","last_became_active":"2024-09-19T08:43:52.763812+0000","last_became_peered":"2024-09-19T08:43:52.763812+0000","last_unstale":"2024-09-19T08:45:47.966146+0000","last_undegraded":"2024-09-19T08:45:47.966146+0000","last_fullsized":"2024-09-19T08:45:47.966146+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:32.272157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065874+0000","last_change":"2024-09-19T08:43:52.754673+0000","last_active":"2024-09-19T08:45:48.065874+0000","last_peered":"2024-09-19T08:45:48.065874+0000","last_clean":"2024-09-19T08:45:48.065874+0000","last_became_active":"2024-09-19T08:43:52.754144+0000","last_became_peered":"2024-09-19T08:43:52.754144+0000","last_unstale":"2024-09-19T08:45:48.065874+0000","last_undegraded":"2024-09-19T08:45:48.065874+0000","last_fullsized":"2024-09-19T08:45:48.065874+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:16.069260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.076773+0000","last_change":"2024-09-19T08:43:52.758939+0000","last_active":"2024-09-19T08:45:48.076773+0000","last_peered":"2024-09-19T08:45:48.076773+0000","last_clean":"2024-09-19T08:45:48.076773+0000","last_became_active":"2024-09-19T08:43:52.758611+0000","last_became_peered":"2024-09-19T08:43:52.758611+0000","last_unstale":"2024-09-19T08:45:48.076773+0000","last_undegraded":"2024-09-19T08:45:48.076773+0000","last_fullsized":"2024-09-19T08:45:48.076773+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:35:56.463545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221260+0000","last_change":"2024-09-19T08:43:52.770261+0000","last_active":"2024-09-19T08:45:48.221260+0000","last_peered":"2024-09-19T08:45:48.221260+0000","last_clean":"2024-09-19T08:45:48.221260+0000","last_became_active":"2024-09-19T08:43:52.768253+0000","last_became_peered":"2024-09-19T08:43:52.768253+0000","last_unstale":"2024-09-19T08:45:48.221260+0000","last_undegraded":"2024-09-19T08:45:48.221260+0000","last_fullsized":"2024-09-19T08:45:48.221260+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:40.756988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065907+0000","last_change":"2024-09-19T08:43:52.768564+0000","last_active":"2024-09-19T08:45:48.065907+0000","last_peered":"2024-09-19T08:45:48.065907+0000","last_clean":"2024-09-19T08:45:48.065907+0000","last_became_active":"2024-09-19T08:43:52.768260+0000","last_became_peered":"2024-09-19T08:43:52.768260+0000","last_unstale":"2024-09-19T08:45:48.065907+0000","last_undegraded":"2024-09-19T08:45:48.065907+0000","last_fullsized":"2024-09-19T08:45:48.065907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:16.528115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.222053+0000","last_change":"2024-09-19T08:43:52.769804+0000","last_active":"2024-09-19T08:45:48.222053+0000","last_peered":"2024-09-19T08:45:48.222053+0000","last_clean":"2024-09-19T08:45:48.222053+0000","last_became_active":"2024-09-19T08:43:52.769504+0000","last_became_peered":"2024-09-19T08:43:52.769504+0000","last_unstale":"2024-09-19T08:45:48.222053+0000","last_undegraded":"2024-09-19T08:45:48.222053+0000","last_fullsized":"2024-09-19T08:45:48.222053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:22:38.514410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.077170+0000","last_change":"2024-09-19T08:43:52.763278+0000","last_active":"2024-09-19T08:45:48.077170+0000","last_peered":"2024-09-19T08:45:48.077170+0000","last_clean":"2024-09-19T08:45:48.077170+0000","last_became_active":"2024-09-19T08:43:52.762985+0000","last_became_peered":"2024-09-19T08:43:52.762985+0000","last_unstale":"2024-09-19T08:45:48.077170+0000","last_undegraded":"2024-09-19T08:45:48.077170+0000","last_fullsized":"2024-09-19T08:45:48.077170+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:30.048782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221488+0000","last_change":"2024-09-19T08:43:52.770864+0000","last_active":"2024-09-19T08:45:48.221488+0000","last_peered":"2024-09-19T08:45:48.221488+0000","last_clean":"2024-09-19T08:45:48.221488+0000","last_became_active":"2024-09-19T08:43:52.770518+0000","last_became_peered":"2024-09-19T08:43:52.770518+0000","last_unstale":"2024-09-19T08:45:48.221488+0000","last_undegraded":"2024-09-19T08:45:48.221488+0000","last_fullsized":"2024-09-19T08:45:48.221488+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:08.931452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.966171+0000","last_change":"2024-09-19T08:43:52.757464+0000","last_active":"2024-09-19T08:45:47.966171+0000","last_peered":"2024-09-19T08:45:47.966171+0000","last_clean":"2024-09-19T08:45:47.966171+0000","last_became_active":"2024-09-19T08:43:52.757101+0000","last_became_peered":"2024-09-19T08:43:52.757101+0000","last_unstale":"2024-09-19T08:45:47.966171+0000","last_undegraded":"2024-09-19T08:45:47.966171+0000","last_fullsized":"2024-09-19T08:45:47.966171+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:17.395735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.224010+0000","last_change":"2024-09-19T08:43:52.767063+0000","last_active":"2024-09-19T08:45:48.224010+0000","last_peered":"2024-09-19T08:45:48.224010+0000","last_clean":"2024-09-19T08:45:48.224010+0000","last_became_active":"2024-09-19T08:43:52.766755+0000","last_became_peered":"2024-09-19T08:43:52.766755+0000","last_unstale":"2024-09-19T08:45:48.224010+0000","last_undegraded":"2024-09-19T08:45:48.224010+0000","last_fullsized":"2024-09-19T08:45:48.224010+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:30.530937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.230150+0000","last_change":"2024-09-19T08:43:52.766515+0000","last_active":"2024-09-19T08:45:48.230150+0000","last_peered":"2024-09-19T08:45:48.230150+0000","last_clean":"2024-09-19T08:45:48.230150+0000","last_became_active":"2024-09-19T08:43:52.765847+0000","last_became_peered":"2024-09-19T08:43:52.765847+0000","last_unstale":"2024-09-19T08:45:48.230150+0000","last_undegraded":"2024-09-19T08:45:48.230150+0000","last_fullsized":"2024-09-19T08:45:48.230150+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:50:35.948663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229847+0000","last_change":"2024-09-19T08:43:52.773702+0000","last_active":"2024-09-19T08:45:48.229847+0000","last_peered":"2024-09-19T08:45:48.229847+0000","last_clean":"2024-09-19T08:45:48.229847+0000","last_became_active":"2024-09-19T08:43:52.773324+0000","last_became_peered":"2024-09-19T08:43:52.773324+0000","last_unstale":"2024-09-19T08:45:48.229847+0000","last_undegraded":"2024-09-19T08:45:48.229847+0000","last_fullsized":"2024-09-19T08:45:48.229847+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:34.368014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221253+0000","last_change":"2024-09-19T08:43:52.762745+0000","last_active":"2024-09-19T08:45:48.221253+0000","last_peered":"2024-09-19T08:45:48.221253+0000","last_clean":"2024-09-19T08:45:48.221253+0000","last_became_active":"2024-09-19T08:43:52.762172+0000","last_became_peered":"2024-09-19T08:43:52.762172+0000","last_unstale":"2024-09-19T08:45:48.221253+0000","last_undegraded":"2024-09-19T08:45:48.221253+0000","last_fullsized":"2024-09-19T08:45:48.221253+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:57.369775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229743+0000","last_change":"2024-09-19T08:43:52.767118+0000","last_active":"2024-09-19T08:45:48.229743+0000","last_peered":"2024-09-19T08:45:48.229743+0000","last_clean":"2024-09-19T08:45:48.229743+0000","last_became_active":"2024-09-19T08:43:52.766388+0000","last_became_peered":"2024-09-19T08:43:52.766388+0000","last_unstale":"2024-09-19T08:45:48.229743+0000","last_undegraded":"2024-09-19T08:45:48.229743+0000","last_fullsized":"2024-09-19T08:45:48.229743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:34.182923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.967257+0000","last_change":"2024-09-19T08:43:52.765008+0000","last_active":"2024-09-19T08:45:47.967257+0000","last_peered":"2024-09-19T08:45:47.967257+0000","last_clean":"2024-09-19T08:45:47.967257+0000","last_became_active":"2024-09-19T08:43:52.764558+0000","last_became_peered":"2024-09-19T08:43:52.764558+0000","last_unstale":"2024-09-19T08:45:47.967257+0000","last_undegraded":"2024-09-19T08:45:47.967257+0000","last_fullsized":"2024-09-19T08:45:47.967257+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:52:42.756421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968722+0000","last_change":"2024-09-19T08:43:52.764376+0000","last_active":"2024-09-19T08:45:47.968722+0000","last_peered":"2024-09-19T08:45:47.968722+0000","last_clean":"2024-09-19T08:45:47.968722+0000","last_became_active":"2024-09-19T08:43:52.764048+0000","last_became_peered":"2024-09-19T08:43:52.764048+0000","last_unstale":"2024-09-19T08:45:47.968722+0000","last_undegraded":"2024-09-19T08:45:47.968722+0000","last_fullsized":"2024-09-19T08:45:47.968722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:56.732094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229417+0000","last_change":"2024-09-19T08:44:02.827925+0000","last_active":"2024-09-19T08:45:48.229417+0000","last_peered":"2024-09-19T08:45:48.229417+0000","last_clean":"2024-09-19T08:45:48.229417+0000","last_became_active":"2024-09-19T08:44:02.827269+0000","last_became_peered":"2024-09-19T08:44:02.827269+0000","last_unstale":"2024-09-19T08:45:48.229417+0000","last_undegraded":"2024-09-19T08:45:48.229417+0000","last_fullsized":"2024-09-19T08:45:48.229417+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:31:54.556786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.224157+0000","last_change":"2024-09-19T08:43:52.762190+0000","last_active":"2024-09-19T08:45:48.224157+0000","last_peered":"2024-09-19T08:45:48.224157+0000","last_clean":"2024-09-19T08:45:48.224157+0000","last_became_active":"2024-09-19T08:43:52.761818+0000","last_became_peered":"2024-09-19T08:43:52.761818+0000","last_unstale":"2024-09-19T08:45:48.224157+0000","last_undegraded":"2024-09-19T08:45:48.224157+0000","last_fullsized":"2024-09-19T08:45:48.224157+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:19:55.246779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221492+0000","last_change":"2024-09-19T08:43:52.761069+0000","last_active":"2024-09-19T08:45:48.221492+0000","last_peered":"2024-09-19T08:45:48.221492+0000","last_clean":"2024-09-19T08:45:48.221492+0000","last_became_active":"2024-09-19T08:43:52.760705+0000","last_became_peered":"2024-09-19T08:43:52.760705+0000","last_unstale":"2024-09-19T08:45:48.221492+0000","last_undegraded":"2024-09-19T08:45:48.221492+0000","last_fullsized":"2024-09-19T08:45:48.221492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:46:27.059026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065963+0000","last_change":"2024-09-19T08:43:52.773449+0000","last_active":"2024-09-19T08:45:48.065963+0000","last_peered":"2024-09-19T08:45:48.065963+0000","last_clean":"2024-09-19T08:45:48.065963+0000","last_became_active":"2024-09-19T08:43:52.772929+0000","last_became_peered":"2024-09-19T08:43:52.772929+0000","last_unstale":"2024-09-19T08:45:48.065963+0000","last_undegraded":"2024-09-19T08:45:48.065963+0000","last_fullsized":"2024-09-19T08:45:48.065963+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:00:38.929199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074018+0000","last_change":"2024-09-19T08:43:52.773885+0000","last_active":"2024-09-19T08:45:48.074018+0000","last_peered":"2024-09-19T08:45:48.074018+0000","last_clean":"2024-09-19T08:45:48.074018+0000","last_became_active":"2024-09-19T08:43:52.772856+0000","last_became_peered":"2024-09-19T08:43:52.772856+0000","last_unstale":"2024-09-19T08:45:48.074018+0000","last_undegraded":"2024-09-19T08:45:48.074018+0000","last_fullsized":"2024-09-19T08:45:48.074018+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:13:22.581335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.067279+0000","last_change":"2024-09-19T08:43:52.762909+0000","last_active":"2024-09-19T08:45:48.067279+0000","last_peered":"2024-09-19T08:45:48.067279+0000","last_clean":"2024-09-19T08:45:48.067279+0000","last_became_active":"2024-09-19T08:43:52.762600+0000","last_became_peered":"2024-09-19T08:43:52.762600+0000","last_unstale":"2024-09-19T08:45:48.067279+0000","last_undegraded":"2024-09-19T08:45:48.067279+0000","last_fullsized":"2024-09-19T08:45:48.067279+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:46.098294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968939+0000","last_change":"2024-09-19T08:43:52.762861+0000","last_active":"2024-09-19T08:45:47.968939+0000","last_peered":"2024-09-19T08:45:47.968939+0000","last_clean":"2024-09-19T08:45:47.968939+0000","last_became_active":"2024-09-19T08:43:52.762525+0000","last_became_peered":"2024-09-19T08:43:52.762525+0000","last_unstale":"2024-09-19T08:45:47.968939+0000","last_undegraded":"2024-09-19T08:45:47.968939+0000","last_fullsized":"2024-09-19T08:45:47.968939+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:54.717249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229552+0000","last_change":"2024-09-19T08:43:52.768370+0000","last_active":"2024-09-19T08:45:48.229552+0000","last_peered":"2024-09-19T08:45:48.229552+0000","last_clean":"2024-09-19T08:45:48.229552+0000","last_became_active":"2024-09-19T08:43:52.768055+0000","last_became_peered":"2024-09-19T08:43:52.768055+0000","last_unstale":"2024-09-19T08:45:48.229552+0000","last_undegraded":"2024-09-19T08:45:48.229552+0000","last_fullsized":"2024-09-19T08:45:48.229552+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:48.212862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.224066+0000","last_change":"2024-09-19T08:43:52.758541+0000","last_active":"2024-09-19T08:45:48.224066+0000","last_peered":"2024-09-19T08:45:48.224066+0000","last_clean":"2024-09-19T08:45:48.224066+0000","last_became_active":"2024-09-19T08:43:52.758122+0000","last_became_peered":"2024-09-19T08:43:52.758122+0000","last_unstale":"2024-09-19T08:45:48.224066+0000","last_undegraded":"2024-09-19T08:45:48.224066+0000","last_fullsized":"2024-09-19T08:45:48.224066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:54:40.681880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.231790+0000","last_change":"2024-09-19T08:43:52.758109+0000","last_active":"2024-09-19T08:45:48.231790+0000","last_peered":"2024-09-19T08:45:48.231790+0000","last_clean":"2024-09-19T08:45:48.231790+0000","last_became_active":"2024-09-19T08:43:52.757808+0000","last_became_peered":"2024-09-19T08:43:52.757808+0000","last_unstale":"2024-09-19T08:45:48.231790+0000","last_undegraded":"2024-09-19T08:45:48.231790+0000","last_fullsized":"2024-09-19T08:45:48.231790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:02:47.614562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968701+0000","last_change":"2024-09-19T08:43:52.772501+0000","last_active":"2024-09-19T08:45:47.968701+0000","last_peered":"2024-09-19T08:45:47.968701+0000","last_clean":"2024-09-19T08:45:47.968701+0000","last_became_active":"2024-09-19T08:43:52.771611+0000","last_became_peered":"2024-09-19T08:43:52.771611+0000","last_unstale":"2024-09-19T08:45:47.968701+0000","last_undegraded":"2024-09-19T08:45:47.968701+0000","last_fullsized":"2024-09-19T08:45:47.968701+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:49.537237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968630+0000","last_change":"2024-09-19T08:43:52.773664+0000","last_active":"2024-09-19T08:45:47.968630+0000","last_peered":"2024-09-19T08:45:47.968630+0000","last_clean":"2024-09-19T08:45:47.968630+0000","last_became_active":"2024-09-19T08:43:52.773014+0000","last_became_peered":"2024-09-19T08:43:52.773014+0000","last_unstale":"2024-09-19T08:45:47.968630+0000","last_undegraded":"2024-09-19T08:45:47.968630+0000","last_fullsized":"2024-09-19T08:45:47.968630+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:03.448825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229497+0000","last_change":"2024-09-19T08:43:52.769578+0000","last_active":"2024-09-19T08:45:48.229497+0000","last_peered":"2024-09-19T08:45:48.229497+0000","last_clean":"2024-09-19T08:45:48.229497+0000","last_became_active":"2024-09-19T08:43:52.769252+0000","last_became_peered":"2024-09-19T08:43:52.769252+0000","last_unstale":"2024-09-19T08:45:48.229497+0000","last_undegraded":"2024-09-19T08:45:48.229497+0000","last_fullsized":"2024-09-19T08:45:48.229497+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:27.936118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065844+0000","last_change":"2024-09-19T08:43:52.770453+0000","last_active":"2024-09-19T08:45:48.065844+0000","last_peered":"2024-09-19T08:45:48.065844+0000","last_clean":"2024-09-19T08:45:48.065844+0000","last_became_active":"2024-09-19T08:43:52.770155+0000","last_became_peered":"2024-09-19T08:43:52.770155+0000","last_unstale":"2024-09-19T08:45:48.065844+0000","last_undegraded":"2024-09-19T08:45:48.065844+0000","last_fullsized":"2024-09-19T08:45:48.065844+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:12.893572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'8","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.782770+0000","last_change":"2024-09-19T08:43:50.751316+0000","last_active":"2024-09-19T08:45:48.782770+0000","last_peered":"2024-09-19T08:45:48.782770+0000","last_clean":"2024-09-19T08:45:48.782770+0000","last_became_active":"2024-09-19T08:43:50.750972+0000","last_became_peered":"2024-09-19T08:43:50.750972+0000","last_unstale":"2024-09-19T08:45:48.782770+0000","last_undegraded":"2024-09-19T08:45:48.782770+0000","last_fullsized":"2024-09-19T08:45:48.782770+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:40.654729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1549,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074105+0000","last_change":"2024-09-19T08:43:52.773979+0000","last_active":"2024-09-19T08:45:48.074105+0000","last_peered":"2024-09-19T08:45:48.074105+0000","last_clean":"2024-09-19T08:45:48.074105+0000","last_became_active":"2024-09-19T08:43:52.773648+0000","last_became_peered":"2024-09-19T08:43:52.773648+0000","last_unstale":"2024-09-19T08:45:48.074105+0000","last_undegraded":"2024-09-19T08:45:48.074105+0000","last_fullsized":"2024-09-19T08:45:48.074105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:36.061209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.076810+0000","last_change":"2024-09-19T08:43:50.750704+0000","last_active":"2024-09-19T08:45:48.076810+0000","last_peered":"2024-09-19T08:45:48.076810+0000","last_clean":"2024-09-19T08:45:48.076810+0000","last_became_active":"2024-09-19T08:43:50.750399+0000","last_became_peered":"2024-09-19T08:43:50.750399+0000","last_unstale":"2024-09-19T08:45:48.076810+0000","last_undegraded":"2024-09-19T08:45:48.076810+0000","last_fullsized":"2024-09-19T08:45:48.076810+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:25.447320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.223615+0000","last_change":"2024-09-19T08:43:50.744919+0000","last_active":"2024-09-19T08:45:48.223615+0000","last_peered":"2024-09-19T08:45:48.223615+0000","last_clean":"2024-09-19T08:45:48.223615+0000","last_became_active":"2024-09-19T08:43:50.744604+0000","last_became_peered":"2024-09-19T08:43:50.744604+0000","last_unstale":"2024-09-19T08:45:48.223615+0000","last_undegraded":"2024-09-19T08:45:48.223615+0000","last_fullsized":"2024-09-19T08:45:48.223615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:25:06.410093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065913+0000","last_change":"2024-09-19T08:43:52.766779+0000","last_active":"2024-09-19T08:45:48.065913+0000","last_peered":"2024-09-19T08:45:48.065913+0000","last_clean":"2024-09-19T08:45:48.065913+0000","last_became_active":"2024-09-19T08:43:52.766484+0000","last_became_peered":"2024-09-19T08:43:52.766484+0000","last_unstale":"2024-09-19T08:45:48.065913+0000","last_undegraded":"2024-09-19T08:45:48.065913+0000","last_fullsized":"2024-09-19T08:45:48.065913+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:17:29.350778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074289+0000","last_change":"2024-09-19T08:43:50.751282+0000","last_active":"2024-09-19T08:45:48.074289+0000","last_peered":"2024-09-19T08:45:48.074289+0000","last_clean":"2024-09-19T08:45:48.074289+0000","last_became_active":"2024-09-19T08:43:50.750754+0000","last_became_peered":"2024-09-19T08:43:50.750754+0000","last_unstale":"2024-09-19T08:45:48.074289+0000","last_undegraded":"2024-09-19T08:45:48.074289+0000","last_fullsized":"2024-09-19T08:45:48.074289+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:11.661626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.967219+0000","last_change":"2024-09-19T08:43:52.756442+0000","last_active":"2024-09-19T08:45:47.967219+0000","last_peered":"2024-09-19T08:45:47.967219+0000","last_clean":"2024-09-19T08:45:47.967219+0000","last_became_active":"2024-09-19T08:43:52.756116+0000","last_became_peered":"2024-09-19T08:43:52.756116+0000","last_unstale":"2024-09-19T08:45:47.967219+0000","last_undegraded":"2024-09-19T08:45:47.967219+0000","last_fullsized":"2024-09-19T08:45:47.967219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:40.029272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'8","reported_seq":242,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.782133+0000","last_change":"2024-09-19T08:43:50.749637+0000","last_active":"2024-09-19T08:45:48.782133+0000","last_peered":"2024-09-19T08:45:48.782133+0000","last_clean":"2024-09-19T08:45:48.782133+0000","last_became_active":"2024-09-19T08:43:50.749307+0000","last_became_peered":"2024-09-19T08:43:50.749307+0000","last_unstale":"2024-09-19T08:45:48.782133+0000","last_undegraded":"2024-09-19T08:45:48.782133+0000","last_fullsized":"2024-09-19T08:45:48.782133+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:23:33.409671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":210,"num_read_kb":210,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.222016+0000","last_change":"2024-09-19T08:43:52.763592+0000","last_active":"2024-09-19T08:45:48.222016+0000","last_peered":"2024-09-19T08:45:48.222016+0000","last_clean":"2024-09-19T08:45:48.222016+0000","last_became_active":"2024-09-19T08:43:52.763146+0000","last_became_peered":"2024-09-19T08:43:52.763146+0000","last_unstale":"2024-09-19T08:45:48.222016+0000","last_undegraded":"2024-09-19T08:45:48.222016+0000","last_fullsized":"2024-09-19T08:45:48.222016+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:32.194465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221748+0000","last_change":"2024-09-19T08:43:50.749184+0000","last_active":"2024-09-19T08:45:48.221748+0000","last_peered":"2024-09-19T08:45:48.221748+0000","last_clean":"2024-09-19T08:45:48.221748+0000","last_became_active":"2024-09-19T08:43:50.748844+0000","last_became_peered":"2024-09-19T08:43:50.748844+0000","last_unstale":"2024-09-19T08:45:48.221748+0000","last_undegraded":"2024-09-19T08:45:48.221748+0000","last_fullsized":"2024-09-19T08:45:48.221748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:15.334759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.231209+0000","last_change":"2024-09-19T08:43:52.768696+0000","last_active":"2024-09-19T08:45:48.231209+0000","last_peered":"2024-09-19T08:45:48.231209+0000","last_clean":"2024-09-19T08:45:48.231209+0000","last_became_active":"2024-09-19T08:43:52.768374+0000","last_became_peered":"2024-09-19T08:43:52.768374+0000","last_unstale":"2024-09-19T08:45:48.231209+0000","last_undegraded":"2024-09-19T08:45:48.231209+0000","last_fullsized":"2024-09-19T08:45:48.231209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:34:39.343041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074342+0000","last_change":"2024-09-19T08:43:51.026270+0000","last_active":"2024-09-19T08:45:48.074342+0000","last_peered":"2024-09-19T08:45:48.074342+0000","last_clean":"2024-09-19T08:45:48.074342+0000","last_became_active":"2024-09-19T08:43:51.025823+0000","last_became_peered":"2024-09-19T08:43:51.025823+0000","last_unstale":"2024-09-19T08:45:48.074342+0000","last_undegraded":"2024-09-19T08:45:48.074342+0000","last_fullsized":"2024-09-19T08:45:48.074342+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:00.480438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221712+0000","last_change":"2024-09-19T08:43:52.759555+0000","last_active":"2024-09-19T08:45:48.221712+0000","last_peered":"2024-09-19T08:45:48.221712+0000","last_clean":"2024-09-19T08:45:48.221712+0000","last_became_active":"2024-09-19T08:43:52.759223+0000","last_became_peered":"2024-09-19T08:43:52.759223+0000","last_unstale":"2024-09-19T08:45:48.221712+0000","last_undegraded":"2024-09-19T08:45:48.221712+0000","last_fullsized":"2024-09-19T08:45:48.221712+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:57:06.451323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'36","reported_seq":283,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.782998+0000","last_change":"2024-09-19T08:43:50.751924+0000","last_active":"2024-09-19T08:45:48.782998+0000","last_peered":"2024-09-19T08:45:48.782998+0000","last_clean":"2024-09-19T08:45:48.782998+0000","last_became_active":"2024-09-19T08:43:50.751117+0000","last_became_peered":"2024-09-19T08:43:50.751117+0000","last_unstale":"2024-09-19T08:45:48.782998+0000","last_undegraded":"2024-09-19T08:45:48.782998+0000","last_fullsized":"2024-09-19T08:45:48.782998+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:33:57.196283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65927,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":223,"num_read_kb":176,"num_write":38,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066156+0000","last_change":"2024-09-19T08:43:52.762260+0000","last_active":"2024-09-19T08:45:48.066156+0000","last_peered":"2024-09-19T08:45:48.066156+0000","last_clean":"2024-09-19T08:45:48.066156+0000","last_became_active":"2024-09-19T08:43:52.761923+0000","last_became_peered":"2024-09-19T08:43:52.761923+0000","last_unstale":"2024-09-19T08:45:48.066156+0000","last_undegraded":"2024-09-19T08:45:48.066156+0000","last_fullsized":"2024-09-19T08:45:48.066156+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:39:39.503332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968758+0000","last_change":"2024-09-19T08:43:50.749745+0000","last_active":"2024-09-19T08:45:47.968758+0000","last_peered":"2024-09-19T08:45:47.968758+0000","last_clean":"2024-09-19T08:45:47.968758+0000","last_became_active":"2024-09-19T08:43:50.749073+0000","last_became_peered":"2024-09-19T08:43:50.749073+0000","last_unstale":"2024-09-19T08:45:47.968758+0000","last_undegraded":"2024-09-19T08:45:47.968758+0000","last_fullsized":"2024-09-19T08:45:47.968758+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:48:11.965841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.223734+0000","last_change":"2024-09-19T08:43:52.761495+0000","last_active":"2024-09-19T08:45:48.223734+0000","last_peered":"2024-09-19T08:45:48.223734+0000","last_clean":"2024-09-19T08:45:48.223734+0000","last_became_active":"2024-09-19T08:43:52.760425+0000","last_became_peered":"2024-09-19T08:43:52.760425+0000","last_unstale":"2024-09-19T08:45:48.223734+0000","last_undegraded":"2024-09-19T08:45:48.223734+0000","last_fullsized":"2024-09-19T08:45:48.223734+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:57.689284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074090+0000","last_change":"2024-09-19T08:43:50.749563+0000","last_active":"2024-09-19T08:45:48.074090+0000","last_peered":"2024-09-19T08:45:48.074090+0000","last_clean":"2024-09-19T08:45:48.074090+0000","last_became_active":"2024-09-19T08:43:50.749244+0000","last_became_peered":"2024-09-19T08:43:50.749244+0000","last_unstale":"2024-09-19T08:45:48.074090+0000","last_undegraded":"2024-09-19T08:45:48.074090+0000","last_fullsized":"2024-09-19T08:45:48.074090+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:59:02.312883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221429+0000","last_change":"2024-09-19T08:43:52.767087+0000","last_active":"2024-09-19T08:45:48.221429+0000","last_peered":"2024-09-19T08:45:48.221429+0000","last_clean":"2024-09-19T08:45:48.221429+0000","last_became_active":"2024-09-19T08:43:52.766730+0000","last_became_peered":"2024-09-19T08:43:52.766730+0000","last_unstale":"2024-09-19T08:45:48.221429+0000","last_undegraded":"2024-09-19T08:45:48.221429+0000","last_fullsized":"2024-09-19T08:45:48.221429+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:04.040896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.966237+0000","last_change":"2024-09-19T08:43:50.749659+0000","last_active":"2024-09-19T08:45:47.966237+0000","last_peered":"2024-09-19T08:45:47.966237+0000","last_clean":"2024-09-19T08:45:47.966237+0000","last_became_active":"2024-09-19T08:43:50.749314+0000","last_became_peered":"2024-09-19T08:43:50.749314+0000","last_unstale":"2024-09-19T08:45:47.966237+0000","last_undegraded":"2024-09-19T08:45:47.966237+0000","last_fullsized":"2024-09-19T08:45:47.966237+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:08.576209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221691+0000","last_change":"2024-09-19T08:43:52.767928+0000","last_active":"2024-09-19T08:45:48.221691+0000","last_peered":"2024-09-19T08:45:48.221691+0000","last_clean":"2024-09-19T08:45:48.221691+0000","last_became_active":"2024-09-19T08:43:52.767580+0000","last_became_peered":"2024-09-19T08:43:52.767580+0000","last_unstale":"2024-09-19T08:45:48.221691+0000","last_undegraded":"2024-09-19T08:45:48.221691+0000","last_fullsized":"2024-09-19T08:45:48.221691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:45.344693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229348+0000","last_change":"2024-09-19T08:44:02.833887+0000","last_active":"2024-09-19T08:45:48.229348+0000","last_peered":"2024-09-19T08:45:48.229348+0000","last_clean":"2024-09-19T08:45:48.229348+0000","last_became_active":"2024-09-19T08:44:02.821624+0000","last_became_peered":"2024-09-19T08:44:02.821624+0000","last_unstale":"2024-09-19T08:45:48.229348+0000","last_undegraded":"2024-09-19T08:45:48.229348+0000","last_fullsized":"2024-09-19T08:45:48.229348+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:54:36.131949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.967019+0000","last_change":"2024-09-19T08:44:03.080503+0000","last_active":"2024-09-19T08:45:47.967019+0000","last_peered":"2024-09-19T08:45:47.967019+0000","last_clean":"2024-09-19T08:45:47.967019+0000","last_became_active":"2024-09-19T08:44:03.080129+0000","last_became_peered":"2024-09-19T08:44:03.080129+0000","last_unstale":"2024-09-19T08:45:47.967019+0000","last_undegraded":"2024-09-19T08:45:47.967019+0000","last_fullsized":"2024-09-19T08:45:47.967019+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:01:37.034795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066192+0000","last_change":"2024-09-19T08:43:50.747624+0000","last_active":"2024-09-19T08:45:48.066192+0000","last_peered":"2024-09-19T08:45:48.066192+0000","last_clean":"2024-09-19T08:45:48.066192+0000","last_became_active":"2024-09-19T08:43:50.747263+0000","last_became_peered":"2024-09-19T08:43:50.747263+0000","last_unstale":"2024-09-19T08:45:48.066192+0000","last_undegraded":"2024-09-19T08:45:48.066192+0000","last_fullsized":"2024-09-19T08:45:48.066192+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:18.580106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229525+0000","last_change":"2024-09-19T08:43:52.770051+0000","last_active":"2024-09-19T08:45:48.229525+0000","last_peered":"2024-09-19T08:45:48.229525+0000","last_clean":"2024-09-19T08:45:48.229525+0000","last_became_active":"2024-09-19T08:43:52.769727+0000","last_became_peered":"2024-09-19T08:43:52.769727+0000","last_unstale":"2024-09-19T08:45:48.229525+0000","last_undegraded":"2024-09-19T08:45:48.229525+0000","last_fullsized":"2024-09-19T08:45:48.229525+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:34.954897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.223984+0000","last_change":"2024-09-19T08:43:50.746696+0000","last_active":"2024-09-19T08:45:48.223984+0000","last_peered":"2024-09-19T08:45:48.223984+0000","last_clean":"2024-09-19T08:45:48.223984+0000","last_became_active":"2024-09-19T08:43:50.746246+0000","last_became_peered":"2024-09-19T08:43:50.746246+0000","last_unstale":"2024-09-19T08:45:48.223984+0000","last_undegraded":"2024-09-19T08:45:48.223984+0000","last_fullsized":"2024-09-19T08:45:48.223984+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:38:39.935348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.222174+0000","last_change":"2024-09-19T08:43:52.765199+0000","last_active":"2024-09-19T08:45:48.222174+0000","last_peered":"2024-09-19T08:45:48.222174+0000","last_clean":"2024-09-19T08:45:48.222174+0000","last_became_active":"2024-09-19T08:43:52.764896+0000","last_became_peered":"2024-09-19T08:43:52.764896+0000","last_unstale":"2024-09-19T08:45:48.222174+0000","last_undegraded":"2024-09-19T08:45:48.222174+0000","last_fullsized":"2024-09-19T08:45:48.222174+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:02:59.570018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'41","reported_seq":73,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.186388+0000","last_change":"2024-09-19T08:43:50.747242+0000","last_active":"2024-09-19T08:45:48.186388+0000","last_peered":"2024-09-19T08:45:48.186388+0000","last_clean":"2024-09-19T08:45:48.186388+0000","last_became_active":"2024-09-19T08:43:50.746853+0000","last_became_peered":"2024-09-19T08:43:50.746853+0000","last_unstale":"2024-09-19T08:45:48.186388+0000","last_undegraded":"2024-09-19T08:45:48.186388+0000","last_fullsized":"2024-09-19T08:45:48.186388+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:21.545635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221579+0000","last_change":"2024-09-19T08:44:02.826535+0000","last_active":"2024-09-19T08:45:48.221579+0000","last_peered":"2024-09-19T08:45:48.221579+0000","last_clean":"2024-09-19T08:45:48.221579+0000","last_became_active":"2024-09-19T08:44:02.826113+0000","last_became_peered":"2024-09-19T08:44:02.826113+0000","last_unstale":"2024-09-19T08:45:48.221579+0000","last_undegraded":"2024-09-19T08:45:48.221579+0000","last_fullsized":"2024-09-19T08:45:48.221579+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:20.247732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'8","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.782660+0000","last_change":"2024-09-19T08:43:51.131540+0000","last_active":"2024-09-19T08:45:48.782660+0000","last_peered":"2024-09-19T08:45:48.782660+0000","last_clean":"2024-09-19T08:45:48.782660+0000","last_became_active":"2024-09-19T08:43:51.131170+0000","last_became_peered":"2024-09-19T08:43:51.131170+0000","last_unstale":"2024-09-19T08:45:48.782660+0000","last_undegraded":"2024-09-19T08:45:48.782660+0000","last_fullsized":"2024-09-19T08:45:48.782660+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:43.635612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221593+0000","last_change":"2024-09-19T08:43:52.763308+0000","last_active":"2024-09-19T08:45:48.221593+0000","last_peered":"2024-09-19T08:45:48.221593+0000","last_clean":"2024-09-19T08:45:48.221593+0000","last_became_active":"2024-09-19T08:43:52.762859+0000","last_became_peered":"2024-09-19T08:43:52.762859+0000","last_unstale":"2024-09-19T08:45:48.221593+0000","last_undegraded":"2024-09-19T08:45:48.221593+0000","last_fullsized":"2024-09-19T08:45:48.221593+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:32:35.083361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":350,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221461+0000","last_change":"2024-09-19T08:42:39.567408+0000","last_active":"2024-09-19T08:45:48.221461+0000","last_peered":"2024-09-19T08:45:48.221461+0000","last_clean":"2024-09-19T08:45:48.221461+0000","last_became_active":"2024-09-19T08:42:39.566967+0000","last_became_peered":"2024-09-19T08:42:39.566967+0000","last_unstale":"2024-09-19T08:45:48.221461+0000","last_undegraded":"2024-09-19T08:45:48.221461+0000","last_fullsized":"2024-09-19T08:45:48.221461+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:14.287944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:14.287944+0000","last_clean_scrub_stamp":"2024-09-19T08:39:14.287944+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:31:03.245386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.966263+0000","last_change":"2024-09-19T08:43:50.751097+0000","last_active":"2024-09-19T08:45:47.966263+0000","last_peered":"2024-09-19T08:45:47.966263+0000","last_clean":"2024-09-19T08:45:47.966263+0000","last_became_active":"2024-09-19T08:43:50.750341+0000","last_became_peered":"2024-09-19T08:43:50.750341+0000","last_unstale":"2024-09-19T08:45:47.966263+0000","last_undegraded":"2024-09-19T08:45:47.966263+0000","last_fullsized":"2024-09-19T08:45:47.966263+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:25.694535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066373+0000","last_change":"2024-09-19T08:43:52.767905+0000","last_active":"2024-09-19T08:45:48.066373+0000","last_peered":"2024-09-19T08:45:48.066373+0000","last_clean":"2024-09-19T08:45:48.066373+0000","last_became_active":"2024-09-19T08:43:52.767601+0000","last_became_peered":"2024-09-19T08:43:52.767601+0000","last_unstale":"2024-09-19T08:45:48.066373+0000","last_undegraded":"2024-09-19T08:45:48.066373+0000","last_fullsized":"2024-09-19T08:45:48.066373+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:14.519879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221338+0000","last_change":"2024-09-19T08:43:50.751397+0000","last_active":"2024-09-19T08:45:48.221338+0000","last_peered":"2024-09-19T08:45:48.221338+0000","last_clean":"2024-09-19T08:45:48.221338+0000","last_became_active":"2024-09-19T08:43:50.751079+0000","last_became_peered":"2024-09-19T08:43:50.751079+0000","last_unstale":"2024-09-19T08:45:48.221338+0000","last_undegraded":"2024-09-19T08:45:48.221338+0000","last_fullsized":"2024-09-19T08:45:48.221338+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:29.904786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.223616+0000","last_change":"2024-09-19T08:43:52.773348+0000","last_active":"2024-09-19T08:45:48.223616+0000","last_peered":"2024-09-19T08:45:48.223616+0000","last_clean":"2024-09-19T08:45:48.223616+0000","last_became_active":"2024-09-19T08:43:52.773011+0000","last_became_peered":"2024-09-19T08:43:52.773011+0000","last_unstale":"2024-09-19T08:45:48.223616+0000","last_undegraded":"2024-09-19T08:45:48.223616+0000","last_fullsized":"2024-09-19T08:45:48.223616+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:10.806578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":135,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.223789+0000","last_change":"2024-09-19T08:43:51.025300+0000","last_active":"2024-09-19T08:45:48.223789+0000","last_peered":"2024-09-19T08:45:48.223789+0000","last_clean":"2024-09-19T08:45:48.223789+0000","last_became_active":"2024-09-19T08:43:51.024936+0000","last_became_peered":"2024-09-19T08:43:51.024936+0000","last_unstale":"2024-09-19T08:45:48.223789+0000","last_undegraded":"2024-09-19T08:45:48.223789+0000","last_fullsized":"2024-09-19T08:45:48.223789+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:03.186066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074063+0000","last_change":"2024-09-19T08:43:52.774962+0000","last_active":"2024-09-19T08:45:48.074063+0000","last_peered":"2024-09-19T08:45:48.074063+0000","last_clean":"2024-09-19T08:45:48.074063+0000","last_became_active":"2024-09-19T08:43:52.774598+0000","last_became_peered":"2024-09-19T08:43:52.774598+0000","last_unstale":"2024-09-19T08:45:48.074063+0000","last_undegraded":"2024-09-19T08:45:48.074063+0000","last_fullsized":"2024-09-19T08:45:48.074063+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:02.687136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'7","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.782526+0000","last_change":"2024-09-19T08:43:50.748180+0000","last_active":"2024-09-19T08:45:48.782526+0000","last_peered":"2024-09-19T08:45:48.782526+0000","last_clean":"2024-09-19T08:45:48.782526+0000","last_became_active":"2024-09-19T08:43:50.747853+0000","last_became_peered":"2024-09-19T08:43:50.747853+0000","last_unstale":"2024-09-19T08:45:48.782526+0000","last_undegraded":"2024-09-19T08:45:48.782526+0000","last_fullsized":"2024-09-19T08:45:48.782526+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:56.047259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.077066+0000","last_change":"2024-09-19T08:43:52.770253+0000","last_active":"2024-09-19T08:45:48.077066+0000","last_peered":"2024-09-19T08:45:48.077066+0000","last_clean":"2024-09-19T08:45:48.077066+0000","last_became_active":"2024-09-19T08:43:52.769951+0000","last_became_peered":"2024-09-19T08:43:52.769951+0000","last_unstale":"2024-09-19T08:45:48.077066+0000","last_undegraded":"2024-09-19T08:45:48.077066+0000","last_fullsized":"2024-09-19T08:45:48.077066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:02.353144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.967260+0000","last_change":"2024-09-19T08:43:51.026453+0000","last_active":"2024-09-19T08:45:47.967260+0000","last_peered":"2024-09-19T08:45:47.967260+0000","last_clean":"2024-09-19T08:45:47.967260+0000","last_became_active":"2024-09-19T08:43:51.025873+0000","last_became_peered":"2024-09-19T08:43:51.025873+0000","last_unstale":"2024-09-19T08:45:47.967260+0000","last_undegraded":"2024-09-19T08:45:47.967260+0000","last_fullsized":"2024-09-19T08:45:47.967260+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:25:17.475074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.076927+0000","last_change":"2024-09-19T08:43:52.761700+0000","last_active":"2024-09-19T08:45:48.076927+0000","last_peered":"2024-09-19T08:45:48.076927+0000","last_clean":"2024-09-19T08:45:48.076927+0000","last_became_active":"2024-09-19T08:43:52.755527+0000","last_became_peered":"2024-09-19T08:43:52.755527+0000","last_unstale":"2024-09-19T08:45:48.076927+0000","last_undegraded":"2024-09-19T08:45:48.076927+0000","last_fullsized":"2024-09-19T08:45:48.076927+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:18:43.171284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066299+0000","last_change":"2024-09-19T08:43:51.026136+0000","last_active":"2024-09-19T08:45:48.066299+0000","last_peered":"2024-09-19T08:45:48.066299+0000","last_clean":"2024-09-19T08:45:48.066299+0000","last_became_active":"2024-09-19T08:43:51.025740+0000","last_became_peered":"2024-09-19T08:43:51.025740+0000","last_unstale":"2024-09-19T08:45:48.066299+0000","last_undegraded":"2024-09-19T08:45:48.066299+0000","last_fullsized":"2024-09-19T08:45:48.066299+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:17:32.741083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.224195+0000","last_change":"2024-09-19T08:43:52.763808+0000","last_active":"2024-09-19T08:45:48.224195+0000","last_peered":"2024-09-19T08:45:48.224195+0000","last_clean":"2024-09-19T08:45:48.224195+0000","last_became_active":"2024-09-19T08:43:52.763509+0000","last_became_peered":"2024-09-19T08:43:52.763509+0000","last_unstale":"2024-09-19T08:45:48.224195+0000","last_undegraded":"2024-09-19T08:45:48.224195+0000","last_fullsized":"2024-09-19T08:45:48.224195+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:45:08.385419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'51","reported_seq":98,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.189380+0000","last_change":"2024-09-19T08:43:50.744213+0000","last_active":"2024-09-19T08:45:48.189380+0000","last_peered":"2024-09-19T08:45:48.189380+0000","last_clean":"2024-09-19T08:45:48.189380+0000","last_became_active":"2024-09-19T08:43:50.743848+0000","last_became_peered":"2024-09-19T08:43:50.743848+0000","last_unstale":"2024-09-19T08:45:48.189380+0000","last_undegraded":"2024-09-19T08:45:48.189380+0000","last_fullsized":"2024-09-19T08:45:48.189380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:26.613055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":90,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.067394+0000","last_change":"2024-09-19T08:43:52.767102+0000","last_active":"2024-09-19T08:45:48.067394+0000","last_peered":"2024-09-19T08:45:48.067394+0000","last_clean":"2024-09-19T08:45:48.067394+0000","last_became_active":"2024-09-19T08:43:52.766262+0000","last_became_peered":"2024-09-19T08:43:52.766262+0000","last_unstale":"2024-09-19T08:45:48.067394+0000","last_undegraded":"2024-09-19T08:45:48.067394+0000","last_fullsized":"2024-09-19T08:45:48.067394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:35.740462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.966479+0000","last_change":"2024-09-19T08:43:51.131742+0000","last_active":"2024-09-19T08:45:47.966479+0000","last_peered":"2024-09-19T08:45:47.966479+0000","last_clean":"2024-09-19T08:45:47.966479+0000","last_became_active":"2024-09-19T08:43:51.131388+0000","last_became_peered":"2024-09-19T08:43:51.131388+0000","last_unstale":"2024-09-19T08:45:47.966479+0000","last_undegraded":"2024-09-19T08:45:47.966479+0000","last_fullsized":"2024-09-19T08:45:47.966479+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:26.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.230731+0000","last_change":"2024-09-19T08:43:52.756004+0000","last_active":"2024-09-19T08:45:48.230731+0000","last_peered":"2024-09-19T08:45:48.230731+0000","last_clean":"2024-09-19T08:45:48.230731+0000","last_became_active":"2024-09-19T08:43:52.755596+0000","last_became_peered":"2024-09-19T08:43:52.755596+0000","last_unstale":"2024-09-19T08:45:48.230731+0000","last_undegraded":"2024-09-19T08:45:48.230731+0000","last_fullsized":"2024-09-19T08:45:48.230731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:35.942375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221620+0000","last_change":"2024-09-19T08:44:02.835986+0000","last_active":"2024-09-19T08:45:48.221620+0000","last_peered":"2024-09-19T08:45:48.221620+0000","last_clean":"2024-09-19T08:45:48.221620+0000","last_became_active":"2024-09-19T08:44:02.826314+0000","last_became_peered":"2024-09-19T08:44:02.826314+0000","last_unstale":"2024-09-19T08:45:48.221620+0000","last_undegraded":"2024-09-19T08:45:48.221620+0000","last_fullsized":"2024-09-19T08:45:48.221620+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:16.943989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065838+0000","last_change":"2024-09-19T08:43:52.766582+0000","last_active":"2024-09-19T08:45:48.065838+0000","last_peered":"2024-09-19T08:45:48.065838+0000","last_clean":"2024-09-19T08:45:48.065838+0000","last_became_active":"2024-09-19T08:43:52.766232+0000","last_became_peered":"2024-09-19T08:43:52.766232+0000","last_unstale":"2024-09-19T08:45:48.065838+0000","last_undegraded":"2024-09-19T08:45:48.065838+0000","last_fullsized":"2024-09-19T08:45:48.065838+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:43:42.319422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066195+0000","last_change":"2024-09-19T08:43:50.747611+0000","last_active":"2024-09-19T08:45:48.066195+0000","last_peered":"2024-09-19T08:45:48.066195+0000","last_clean":"2024-09-19T08:45:48.066195+0000","last_became_active":"2024-09-19T08:43:50.747263+0000","last_became_peered":"2024-09-19T08:43:50.747263+0000","last_unstale":"2024-09-19T08:45:48.066195+0000","last_undegraded":"2024-09-19T08:45:48.066195+0000","last_fullsized":"2024-09-19T08:45:48.066195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:14.717820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229532+0000","last_change":"2024-09-19T08:43:52.766493+0000","last_active":"2024-09-19T08:45:48.229532+0000","last_peered":"2024-09-19T08:45:48.229532+0000","last_clean":"2024-09-19T08:45:48.229532+0000","last_became_active":"2024-09-19T08:43:52.766158+0000","last_became_peered":"2024-09-19T08:43:52.766158+0000","last_unstale":"2024-09-19T08:45:48.229532+0000","last_undegraded":"2024-09-19T08:45:48.229532+0000","last_fullsized":"2024-09-19T08:45:48.229532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:59.175949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968841+0000","last_change":"2024-09-19T08:43:50.751368+0000","last_active":"2024-09-19T08:45:47.968841+0000","last_peered":"2024-09-19T08:45:47.968841+0000","last_clean":"2024-09-19T08:45:47.968841+0000","last_became_active":"2024-09-19T08:43:50.751038+0000","last_became_peered":"2024-09-19T08:43:50.751038+0000","last_unstale":"2024-09-19T08:45:47.968841+0000","last_undegraded":"2024-09-19T08:45:47.968841+0000","last_fullsized":"2024-09-19T08:45:47.968841+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:46.302996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066722+0000","last_change":"2024-09-19T08:43:52.756998+0000","last_active":"2024-09-19T08:45:48.066722+0000","last_peered":"2024-09-19T08:45:48.066722+0000","last_clean":"2024-09-19T08:45:48.066722+0000","last_became_active":"2024-09-19T08:43:52.756661+0000","last_became_peered":"2024-09-19T08:43:52.756661+0000","last_unstale":"2024-09-19T08:45:48.066722+0000","last_undegraded":"2024-09-19T08:45:48.066722+0000","last_fullsized":"2024-09-19T08:45:48.066722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:15.182574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065908+0000","last_change":"2024-09-19T08:43:50.750095+0000","last_active":"2024-09-19T08:45:48.065908+0000","last_peered":"2024-09-19T08:45:48.065908+0000","last_clean":"2024-09-19T08:45:48.065908+0000","last_became_active":"2024-09-19T08:43:50.749672+0000","last_became_peered":"2024-09-19T08:43:50.749672+0000","last_unstale":"2024-09-19T08:45:48.065908+0000","last_undegraded":"2024-09-19T08:45:48.065908+0000","last_fullsized":"2024-09-19T08:45:48.065908+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:25:34.978094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221996+0000","last_change":"2024-09-19T08:43:52.764097+0000","last_active":"2024-09-19T08:45:48.221996+0000","last_peered":"2024-09-19T08:45:48.221996+0000","last_clean":"2024-09-19T08:45:48.221996+0000","last_became_active":"2024-09-19T08:43:52.763773+0000","last_became_peered":"2024-09-19T08:43:52.763773+0000","last_unstale":"2024-09-19T08:45:48.221996+0000","last_undegraded":"2024-09-19T08:45:48.221996+0000","last_fullsized":"2024-09-19T08:45:48.221996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:01:42.098460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074171+0000","last_change":"2024-09-19T08:44:02.825238+0000","last_active":"2024-09-19T08:45:48.074171+0000","last_peered":"2024-09-19T08:45:48.074171+0000","last_clean":"2024-09-19T08:45:48.074171+0000","last_became_active":"2024-09-19T08:44:02.824873+0000","last_became_peered":"2024-09-19T08:44:02.824873+0000","last_unstale":"2024-09-19T08:45:48.074171+0000","last_undegraded":"2024-09-19T08:45:48.074171+0000","last_fullsized":"2024-09-19T08:45:48.074171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:38.896082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066088+0000","last_change":"2024-09-19T08:43:50.748789+0000","last_active":"2024-09-19T08:45:48.066088+0000","last_peered":"2024-09-19T08:45:48.066088+0000","last_clean":"2024-09-19T08:45:48.066088+0000","last_became_active":"2024-09-19T08:43:50.748464+0000","last_became_peered":"2024-09-19T08:43:50.748464+0000","last_unstale":"2024-09-19T08:45:48.066088+0000","last_undegraded":"2024-09-19T08:45:48.066088+0000","last_fullsized":"2024-09-19T08:45:48.066088+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:36:45.905639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221312+0000","last_change":"2024-09-19T08:43:52.756608+0000","last_active":"2024-09-19T08:45:48.221312+0000","last_peered":"2024-09-19T08:45:48.221312+0000","last_clean":"2024-09-19T08:45:48.221312+0000","last_became_active":"2024-09-19T08:43:52.756227+0000","last_became_peered":"2024-09-19T08:43:52.756227+0000","last_unstale":"2024-09-19T08:45:48.221312+0000","last_undegraded":"2024-09-19T08:45:48.221312+0000","last_fullsized":"2024-09-19T08:45:48.221312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:29:58.085584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'13","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066253+0000","last_change":"2024-09-19T08:43:50.749135+0000","last_active":"2024-09-19T08:45:48.066253+0000","last_peered":"2024-09-19T08:45:48.066253+0000","last_clean":"2024-09-19T08:45:48.066253+0000","last_became_active":"2024-09-19T08:43:50.748755+0000","last_became_peered":"2024-09-19T08:43:50.748755+0000","last_unstale":"2024-09-19T08:45:48.066253+0000","last_undegraded":"2024-09-19T08:45:48.066253+0000","last_fullsized":"2024-09-19T08:45:48.066253+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:55.079780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.231077+0000","last_change":"2024-09-19T08:43:52.763277+0000","last_active":"2024-09-19T08:45:48.231077+0000","last_peered":"2024-09-19T08:45:48.231077+0000","last_clean":"2024-09-19T08:45:48.231077+0000","last_became_active":"2024-09-19T08:43:52.762941+0000","last_became_peered":"2024-09-19T08:43:52.762941+0000","last_unstale":"2024-09-19T08:45:48.231077+0000","last_undegraded":"2024-09-19T08:45:48.231077+0000","last_fullsized":"2024-09-19T08:45:48.231077+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:35.531946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.067406+0000","last_change":"2024-09-19T08:43:51.025350+0000","last_active":"2024-09-19T08:45:48.067406+0000","last_peered":"2024-09-19T08:45:48.067406+0000","last_clean":"2024-09-19T08:45:48.067406+0000","last_became_active":"2024-09-19T08:43:51.025046+0000","last_became_peered":"2024-09-19T08:43:51.025046+0000","last_unstale":"2024-09-19T08:45:48.067406+0000","last_undegraded":"2024-09-19T08:45:48.067406+0000","last_fullsized":"2024-09-19T08:45:48.067406+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:27.595545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066247+0000","last_change":"2024-09-19T08:43:52.767917+0000","last_active":"2024-09-19T08:45:48.066247+0000","last_peered":"2024-09-19T08:45:48.066247+0000","last_clean":"2024-09-19T08:45:48.066247+0000","last_became_active":"2024-09-19T08:43:52.767575+0000","last_became_peered":"2024-09-19T08:43:52.767575+0000","last_unstale":"2024-09-19T08:45:48.066247+0000","last_undegraded":"2024-09-19T08:45:48.066247+0000","last_fullsized":"2024-09-19T08:45:48.066247+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:05.950399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221116+0000","last_change":"2024-09-19T08:43:50.746400+0000","last_active":"2024-09-19T08:45:48.221116+0000","last_peered":"2024-09-19T08:45:48.221116+0000","last_clean":"2024-09-19T08:45:48.221116+0000","last_became_active":"2024-09-19T08:43:50.745891+0000","last_became_peered":"2024-09-19T08:43:50.745891+0000","last_unstale":"2024-09-19T08:45:48.221116+0000","last_undegraded":"2024-09-19T08:45:48.221116+0000","last_fullsized":"2024-09-19T08:45:48.221116+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:03.098202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599805,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1190,"num_read_kb":711,"num_write":374,"num_write_kb":1464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":712704,"data_stored":1799415,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":301,"ondisk_log_size":301,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":109335,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579941,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":446056,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808874,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1089536,"data_stored":1232787,"data_compressed":45500,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005102,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":962560,"data_stored":838280,"data_compressed":22045,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234035,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":946176,"data_stored":838429,"data_compressed":22045,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462968,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":983040,"data_stored":1039165,"data_compressed":39344,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593790,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":445450,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822722,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1015808,"data_stored":1169657,"data_compressed":45500,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986247,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":444525,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215180,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":969188,"data_compressed":28201,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1003520,"data_stored":513578,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":983040,"data_stored":512290,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":937984,"data_stored":446930,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":68465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":69753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135347,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":199263,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:45:51.829 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:45:52.126 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:45:52.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:52 smithi170 ceph-mon[35072]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:45:52.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:52 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:52.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:52 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:52.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:52 smithi170 ceph-mon[35072]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:45:52.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:52 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:52.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:52 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:52.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:52 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:52.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:52 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:52.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:52 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:52.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:52 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:52.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:52 smithi170 ceph-mon[35072]: Adjusting osd_memory_target on smithi164 to 176.0M 2024-09-19T08:45:52.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:52 smithi170 ceph-mon[35072]: Unable to set osd_memory_target on smithi164 to 184581017: error parsing value: Value '184581017' is below minimum 939524096 2024-09-19T08:45:52.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:52 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:45:52.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:52 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:45:52.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:52 smithi170 ceph-mon[35072]: Updating smithi164:/etc/ceph/ceph.conf 2024-09-19T08:45:52.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:52 smithi170 ceph-mon[35072]: Updating smithi170:/etc/ceph/ceph.conf 2024-09-19T08:45:52.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:52 smithi170 ceph-mon[35072]: Updating smithi173:/etc/ceph/ceph.conf 2024-09-19T08:45:52.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:52 smithi170 ceph-mon[35072]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:45:52.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:52 smithi173 ceph-mon[35306]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:45:52.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:52 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:52.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:52 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:52.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:52 smithi173 ceph-mon[35306]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:45:52.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:52 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:52.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:52 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:52.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:52 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:52.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:52 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:52.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:52 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:52.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:52 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:52.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:52 smithi173 ceph-mon[35306]: Adjusting osd_memory_target on smithi164 to 176.0M 2024-09-19T08:45:52.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:52 smithi173 ceph-mon[35306]: Unable to set osd_memory_target on smithi164 to 184581017: error parsing value: Value '184581017' is below minimum 939524096 2024-09-19T08:45:52.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:52 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:45:52.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:52 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:45:52.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:52 smithi173 ceph-mon[35306]: Updating smithi164:/etc/ceph/ceph.conf 2024-09-19T08:45:52.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:52 smithi173 ceph-mon[35306]: Updating smithi170:/etc/ceph/ceph.conf 2024-09-19T08:45:52.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:52 smithi173 ceph-mon[35306]: Updating smithi173:/etc/ceph/ceph.conf 2024-09-19T08:45:52.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:52 smithi173 ceph-mon[35306]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:45:52.434 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:52 smithi164 ceph-mon[30928]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:45:52.434 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:52 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:52.434 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:52 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:52.434 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:52 smithi164 ceph-mon[30928]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:45:52.434 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:52 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:52.435 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:52 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:52.435 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:52 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:52.435 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:52 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:52.435 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:52 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:52.435 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:52 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:45:52.435 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:52 smithi164 ceph-mon[30928]: Adjusting osd_memory_target on smithi164 to 176.0M 2024-09-19T08:45:52.435 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:52 smithi164 ceph-mon[30928]: Unable to set osd_memory_target on smithi164 to 184581017: error parsing value: Value '184581017' is below minimum 939524096 2024-09-19T08:45:52.435 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:52 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:45:52.435 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:52 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:45:52.435 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:52 smithi164 ceph-mon[30928]: Updating smithi164:/etc/ceph/ceph.conf 2024-09-19T08:45:52.435 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:52 smithi164 ceph-mon[30928]: Updating smithi170:/etc/ceph/ceph.conf 2024-09-19T08:45:52.435 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:52 smithi164 ceph-mon[30928]: Updating smithi173:/etc/ceph/ceph.conf 2024-09-19T08:45:52.436 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:52 smithi164 ceph-mon[30928]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:45:52.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.687+0000 7f667cba8640 1 -- 172.21.15.164:0/815715036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6678100e20 msgr2=0x7f6678101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:52.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.687+0000 7f667cba8640 1 --2- 172.21.15.164:0/815715036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6678100e20 0x7f6678101200 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6668009900 tx=0x7f666802a480 comp rx=0 tx=0).stop 2024-09-19T08:45:52.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.687+0000 7f667cba8640 1 -- 172.21.15.164:0/815715036 shutdown_connections 2024-09-19T08:45:52.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.687+0000 7f667cba8640 1 --2- 172.21.15.164:0/815715036 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6678102ef0 0x7f667810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:52.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.687+0000 7f667cba8640 1 --2- 172.21.15.164:0/815715036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6678102570 0x7f66781029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:52.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.687+0000 7f667cba8640 1 --2- 172.21.15.164:0/815715036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6678100e20 0x7f6678101200 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:52.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.687+0000 7f667cba8640 1 -- 172.21.15.164:0/815715036 >> 172.21.15.164:0/815715036 conn(0x7f66780fc7d0 msgr2=0x7f66780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:52.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.687+0000 7f667cba8640 1 -- 172.21.15.164:0/815715036 shutdown_connections 2024-09-19T08:45:52.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.687+0000 7f667cba8640 1 -- 172.21.15.164:0/815715036 wait complete. 2024-09-19T08:45:52.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.688+0000 7f667cba8640 1 Processor -- start 2024-09-19T08:45:52.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.688+0000 7f667cba8640 1 -- start start 2024-09-19T08:45:52.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.688+0000 7f667cba8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6678100e20 0x7f667819e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:52.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.688+0000 7f667cba8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6678102570 0x7f667819e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:52.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.688+0000 7f667cba8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6678102ef0 0x7f6678196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:52.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.688+0000 7f667cba8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6678110f80 con 0x7f6678102ef0 2024-09-19T08:45:52.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.688+0000 7f667cba8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6678110e00 con 0x7f6678100e20 2024-09-19T08:45:52.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.688+0000 7f667cba8640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6678111100 con 0x7f6678102570 2024-09-19T08:45:52.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.688+0000 7f6676d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6678102ef0 0x7f6678196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:52.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.688+0000 7f6676d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6678102ef0 0x7f6678196480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46604/0 (socket says 172.21.15.164:46604) 2024-09-19T08:45:52.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.688+0000 7f6676d9d640 1 -- 172.21.15.164:0/3808012376 learned_addr learned my addr 172.21.15.164:0/3808012376 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:52.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.688+0000 7f667659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6678100e20 0x7f667819e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:52.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.688+0000 7f6675d9b640 1 --2- 172.21.15.164:0/3808012376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6678102570 0x7f667819e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:52.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.689+0000 7f6676d9d640 1 -- 172.21.15.164:0/3808012376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6678102570 msgr2=0x7f667819e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:52.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.689+0000 7f6676d9d640 1 --2- 172.21.15.164:0/3808012376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6678102570 0x7f667819e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:52.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.689+0000 7f6676d9d640 1 -- 172.21.15.164:0/3808012376 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6678100e20 msgr2=0x7f667819e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:52.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.689+0000 7f6676d9d640 1 --2- 172.21.15.164:0/3808012376 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6678100e20 0x7f667819e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:52.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.689+0000 7f6676d9d640 1 -- 172.21.15.164:0/3808012376 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6678196d40 con 0x7f6678102ef0 2024-09-19T08:45:52.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.689+0000 7f667659c640 1 --2- 172.21.15.164:0/3808012376 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6678100e20 0x7f667819e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:45:52.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.689+0000 7f6676d9d640 1 --2- 172.21.15.164:0/3808012376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6678102ef0 0x7f6678196480 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f667000eb10 tx=0x7f667000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:52.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.689+0000 7f66677fe640 1 -- 172.21.15.164:0/3808012376 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6670004b20 con 0x7f6678102ef0 2024-09-19T08:45:52.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.689+0000 7f667cba8640 1 -- 172.21.15.164:0/3808012376 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6678197030 con 0x7f6678102ef0 2024-09-19T08:45:52.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.689+0000 7f66677fe640 1 -- 172.21.15.164:0/3808012376 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6670004cc0 con 0x7f6678102ef0 2024-09-19T08:45:52.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.689+0000 7f667cba8640 1 -- 172.21.15.164:0/3808012376 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66781b3e20 con 0x7f6678102ef0 2024-09-19T08:45:52.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.689+0000 7f66677fe640 1 -- 172.21.15.164:0/3808012376 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6670010760 con 0x7f6678102ef0 2024-09-19T08:45:52.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.690+0000 7f667cba8640 1 -- 172.21.15.164:0/3808012376 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6678101200 con 0x7f6678102ef0 2024-09-19T08:45:52.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.691+0000 7f66677fe640 1 -- 172.21.15.164:0/3808012376 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 23) ==== 138476+0+0 (secure 0 0 0) 0x7f6670005000 con 0x7f6678102ef0 2024-09-19T08:45:52.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.692+0000 7f66677fe640 1 --2- 172.21.15.164:0/3808012376 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f66600a33b0 0x7f66600a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:52.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.693+0000 7f667659c640 1 --2- 172.21.15.164:0/3808012376 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f66600a33b0 0x7f66600a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:52.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.693+0000 7f66677fe640 1 -- 172.21.15.164:0/3808012376 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f66700d09b0 con 0x7f6678102ef0 2024-09-19T08:45:52.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.693+0000 7f667659c640 1 --2- 172.21.15.164:0/3808012376 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f66600a33b0 0x7f66600a5870 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6668009230 tx=0x7f66680058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:52.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.696+0000 7f66677fe640 1 -- 172.21.15.164:0/3808012376 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f667009d3a0 con 0x7f6678102ef0 2024-09-19T08:45:52.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.798+0000 7f66677fe640 1 -- 172.21.15.164:0/3808012376 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f6670099310 con 0x7f6678102ef0 2024-09-19T08:45:52.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.923+0000 7f667cba8640 1 -- 172.21.15.164:0/3808012376 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6678197620 con 0x7f6678102ef0 2024-09-19T08:45:52.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.924+0000 7f66677fe640 1 -- 172.21.15.164:0/3808012376 <== mon.0 v2:172.21.15.164:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11984 (secure 0 0 0) 0x7f66700cdbe0 con 0x7f6678102ef0 2024-09-19T08:45:52.924 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:45:52.925 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":28,"btime":"2024-09-19T08:45:46:903004+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:45:46.852789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:45:52.925 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 28 2024-09-19T08:45:52.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.926+0000 7f667cba8640 1 -- 172.21.15.164:0/3808012376 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f66600a33b0 msgr2=0x7f66600a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:52.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.926+0000 7f667cba8640 1 --2- 172.21.15.164:0/3808012376 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f66600a33b0 0x7f66600a5870 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6668009230 tx=0x7f66680058d0 comp rx=0 tx=0).stop 2024-09-19T08:45:52.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.926+0000 7f667cba8640 1 -- 172.21.15.164:0/3808012376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6678102ef0 msgr2=0x7f6678196480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:52.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.926+0000 7f667cba8640 1 --2- 172.21.15.164:0/3808012376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6678102ef0 0x7f6678196480 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f667000eb10 tx=0x7f667000c2d0 comp rx=0 tx=0).stop 2024-09-19T08:45:52.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.926+0000 7f667cba8640 1 -- 172.21.15.164:0/3808012376 shutdown_connections 2024-09-19T08:45:52.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.926+0000 7f667cba8640 1 --2- 172.21.15.164:0/3808012376 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f66600a33b0 0x7f66600a5870 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:52.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.926+0000 7f667cba8640 1 --2- 172.21.15.164:0/3808012376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6678102ef0 0x7f6678196480 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:52.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.926+0000 7f667cba8640 1 --2- 172.21.15.164:0/3808012376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6678102570 0x7f667819e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:52.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.926+0000 7f667cba8640 1 --2- 172.21.15.164:0/3808012376 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6678100e20 0x7f667819e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:52.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.926+0000 7f667cba8640 1 -- 172.21.15.164:0/3808012376 >> 172.21.15.164:0/3808012376 conn(0x7f66780fc7d0 msgr2=0x7f667806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:52.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.926+0000 7f667cba8640 1 -- 172.21.15.164:0/3808012376 shutdown_connections 2024-09-19T08:45:52.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:52.927+0000 7f667cba8640 1 -- 172.21.15.164:0/3808012376 wait complete. 2024-09-19T08:45:53.756 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T08:45:54.005 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:45:54.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:53 smithi164 ceph-mon[30928]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:45:54.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:53 smithi164 ceph-mon[30928]: Updating smithi164:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:45:54.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:53 smithi164 ceph-mon[30928]: Updating smithi170:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:45:54.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:53 smithi164 ceph-mon[30928]: Updating smithi173:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:45:54.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:53 smithi164 ceph-mon[30928]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:45:54.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:53 smithi164 ceph-mon[30928]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:45:54.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:53 smithi164 ceph-mon[30928]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:45:54.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:53 smithi164 ceph-mon[30928]: Updating smithi170:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T08:45:54.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:53 smithi164 ceph-mon[30928]: Updating smithi164:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T08:45:54.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:53 smithi164 ceph-mon[30928]: Updating smithi173:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T08:45:54.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:53 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:54.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:53 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:54.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:53 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:54.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:53 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:54.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:53 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:54.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:53 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:54.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:53 smithi164 ceph-mon[30928]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-09-19T08:45:54.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:53 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:54.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:53 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:45:54.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:53 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:45:54.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:53 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:45:54.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:53 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:54.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:53 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3808012376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:45:54.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:53 smithi170 ceph-mon[35072]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:45:54.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:53 smithi170 ceph-mon[35072]: Updating smithi164:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:45:54.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:53 smithi170 ceph-mon[35072]: Updating smithi170:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:45:54.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:53 smithi170 ceph-mon[35072]: Updating smithi173:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:45:54.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:53 smithi170 ceph-mon[35072]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:45:54.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:53 smithi170 ceph-mon[35072]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:45:54.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:53 smithi170 ceph-mon[35072]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:45:54.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:53 smithi170 ceph-mon[35072]: Updating smithi170:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T08:45:54.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:53 smithi170 ceph-mon[35072]: Updating smithi164:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T08:45:54.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:53 smithi170 ceph-mon[35072]: Updating smithi173:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T08:45:54.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:53 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:54.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:53 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:54.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:53 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:54.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:53 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:54.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:53 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:54.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:53 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:54.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:53 smithi170 ceph-mon[35072]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-09-19T08:45:54.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:53 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:54.127 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:53 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:45:54.127 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:53 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:45:54.127 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:53 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:45:54.127 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:53 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:54.127 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:53 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3808012376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:45:54.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:53 smithi173 ceph-mon[35306]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:45:54.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:53 smithi173 ceph-mon[35306]: Updating smithi164:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:45:54.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:53 smithi173 ceph-mon[35306]: Updating smithi170:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:45:54.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:53 smithi173 ceph-mon[35306]: Updating smithi173:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T08:45:54.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:53 smithi173 ceph-mon[35306]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:45:54.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:53 smithi173 ceph-mon[35306]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:45:54.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:53 smithi173 ceph-mon[35306]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-09-19T08:45:54.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:53 smithi173 ceph-mon[35306]: Updating smithi170:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T08:45:54.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:53 smithi173 ceph-mon[35306]: Updating smithi164:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T08:45:54.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:53 smithi173 ceph-mon[35306]: Updating smithi173:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T08:45:54.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:53 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:54.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:53 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:54.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:53 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:54.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:53 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:54.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:53 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:54.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:53 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:54.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:53 smithi173 ceph-mon[35306]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-09-19T08:45:54.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:53 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:54.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:53 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:45:54.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:53 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:45:54.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:53 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:45:54.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:53 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:54.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:53 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3808012376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:45:54.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.472+0000 7fbcab422640 1 -- 172.21.15.164:0/1148326722 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbca4102ef0 msgr2=0x7fbca410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:54.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.472+0000 7fbcab422640 1 --2- 172.21.15.164:0/1148326722 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbca4102ef0 0x7fbca410da40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c009ca0 tx=0x7fbc9c02a950 comp rx=0 tx=0).stop 2024-09-19T08:45:54.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.472+0000 7fbcab422640 1 -- 172.21.15.164:0/1148326722 shutdown_connections 2024-09-19T08:45:54.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.472+0000 7fbcab422640 1 --2- 172.21.15.164:0/1148326722 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbca4102ef0 0x7fbca410da40 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:54.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.472+0000 7fbcab422640 1 --2- 172.21.15.164:0/1148326722 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbca4102570 0x7fbca41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:54.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.472+0000 7fbcab422640 1 --2- 172.21.15.164:0/1148326722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbca4100e20 0x7fbca4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:54.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.472+0000 7fbcab422640 1 -- 172.21.15.164:0/1148326722 >> 172.21.15.164:0/1148326722 conn(0x7fbca40fc7d0 msgr2=0x7fbca40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:54.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.472+0000 7fbcab422640 1 -- 172.21.15.164:0/1148326722 shutdown_connections 2024-09-19T08:45:54.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.472+0000 7fbcab422640 1 -- 172.21.15.164:0/1148326722 wait complete. 2024-09-19T08:45:54.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.473+0000 7fbcab422640 1 Processor -- start 2024-09-19T08:45:54.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.473+0000 7fbcab422640 1 -- start start 2024-09-19T08:45:54.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.473+0000 7fbcab422640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbca4100e20 0x7fbca4199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:54.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.473+0000 7fbcab422640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbca4102570 0x7fbca41998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:54.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.473+0000 7fbcab422640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbca4102ef0 0x7fbca41a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:54.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.473+0000 7fbcab422640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbca4110f80 con 0x7fbca4102ef0 2024-09-19T08:45:54.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.473+0000 7fbcab422640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbca4110e00 con 0x7fbca4102570 2024-09-19T08:45:54.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.473+0000 7fbcab422640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbca4111100 con 0x7fbca4100e20 2024-09-19T08:45:54.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.473+0000 7fbca99bf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbca4102ef0 0x7fbca41a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:54.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.473+0000 7fbca99bf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbca4102ef0 0x7fbca41a43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39178/0 (socket says 172.21.15.164:39178) 2024-09-19T08:45:54.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.473+0000 7fbca99bf640 1 -- 172.21.15.164:0/967050130 learned_addr learned my addr 172.21.15.164:0/967050130 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:54.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.473+0000 7fbca89bd640 1 --2- 172.21.15.164:0/967050130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbca4102570 0x7fbca41998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:54.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.473+0000 7fbca91be640 1 --2- 172.21.15.164:0/967050130 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbca4100e20 0x7fbca4199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:54.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.473+0000 7fbca99bf640 1 -- 172.21.15.164:0/967050130 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbca4100e20 msgr2=0x7fbca4199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:54.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.473+0000 7fbca99bf640 1 --2- 172.21.15.164:0/967050130 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbca4100e20 0x7fbca4199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:54.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.473+0000 7fbca99bf640 1 -- 172.21.15.164:0/967050130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbca4102570 msgr2=0x7fbca41998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:54.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.473+0000 7fbca99bf640 1 --2- 172.21.15.164:0/967050130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbca4102570 0x7fbca41998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:54.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.473+0000 7fbca99bf640 1 -- 172.21.15.164:0/967050130 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca41a4930 con 0x7fbca4102ef0 2024-09-19T08:45:54.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.474+0000 7fbca99bf640 1 --2- 172.21.15.164:0/967050130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbca4102ef0 0x7fbca41a43f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c02ae70 tx=0x7fbc9c02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:54.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.474+0000 7fbc9a7fc640 1 -- 172.21.15.164:0/967050130 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c034040 con 0x7fbca4102ef0 2024-09-19T08:45:54.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.474+0000 7fbc9a7fc640 1 -- 172.21.15.164:0/967050130 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc9c030920 con 0x7fbca4102ef0 2024-09-19T08:45:54.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.474+0000 7fbcab422640 1 -- 172.21.15.164:0/967050130 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbca41a4b00 con 0x7fbca4102ef0 2024-09-19T08:45:54.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.474+0000 7fbc9a7fc640 1 -- 172.21.15.164:0/967050130 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c045480 con 0x7fbca4102ef0 2024-09-19T08:45:54.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.474+0000 7fbcab422640 1 -- 172.21.15.164:0/967050130 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbca41a4f50 con 0x7fbca4102ef0 2024-09-19T08:45:54.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.475+0000 7fbcab422640 1 -- 172.21.15.164:0/967050130 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbca4101200 con 0x7fbca4102ef0 2024-09-19T08:45:54.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.476+0000 7fbc9a7fc640 1 -- 172.21.15.164:0/967050130 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7fbc9c037020 con 0x7fbca4102ef0 2024-09-19T08:45:54.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.477+0000 7fbc9a7fc640 1 --2- 172.21.15.164:0/967050130 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fbc900a3440 0x7fbc900a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:54.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.477+0000 7fbca91be640 1 --2- 172.21.15.164:0/967050130 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fbc900a3440 0x7fbc900a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:54.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.477+0000 7fbc9a7fc640 1 -- 172.21.15.164:0/967050130 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fbc9c0f0c50 con 0x7fbca4102ef0 2024-09-19T08:45:54.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.477+0000 7fbca91be640 1 --2- 172.21.15.164:0/967050130 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fbc900a3440 0x7fbc900a5900 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbca00058c0 tx=0x7fbca000a4c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:54.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.481+0000 7fbc9a7fc640 1 -- 172.21.15.164:0/967050130 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbc9c0f6020 con 0x7fbca4102ef0 2024-09-19T08:45:54.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.650+0000 7fbcab422640 1 -- 172.21.15.164:0/967050130 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbca41a51b0 con 0x7fbca4102ef0 2024-09-19T08:45:54.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.651+0000 7fbc9a7fc640 1 -- 172.21.15.164:0/967050130 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+23086 (secure 0 0 0) 0x7fbc9c0bd640 con 0x7fbca4102ef0 2024-09-19T08:45:54.652 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:45:54.653 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":83,"fsid":"1e4b0d74-7662-11ef-bceb-c7b262605968","created":"2024-09-19T08:36:09.564509+0000","modified":"2024-09-19T08:45:47.960176+0000","last_up_change":"2024-09-19T08:43:01.260852+0000","last_in_change":"2024-09-19T08:42:37.492464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:39:13.961749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:43:49.613471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T08:43:51.570675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33b11f37-6ac5-41fc-8cfd-1b08c7eef688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6803","nonce":663038243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6805","nonce":663038243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6809","nonce":663038243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6807","nonce":663038243}]},"public_addr":"172.21.15.164:6803/663038243","cluster_addr":"172.21.15.164:6805/663038243","heartbeat_back_addr":"172.21.15.164:6809/663038243","heartbeat_front_addr":"172.21.15.164:6807/663038243","state":["exists","up"]},{"osd":1,"uuid":"6de64b0a-46e2-4a98-8005-a326a64bc87d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6801","nonce":2167066371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6803","nonce":2167066371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6807","nonce":2167066371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6805","nonce":2167066371}]},"public_addr":"172.21.15.170:6801/2167066371","cluster_addr":"172.21.15.170:6803/2167066371","heartbeat_back_addr":"172.21.15.170:6807/2167066371","heartbeat_front_addr":"172.21.15.170:6805/2167066371","state":["exists","up"]},{"osd":2,"uuid":"d4158cc7-9822-42a7-9601-d65089907584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6801","nonce":1646561048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6803","nonce":1646561048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6807","nonce":1646561048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6805","nonce":1646561048}]},"public_addr":"172.21.15.173:6801/1646561048","cluster_addr":"172.21.15.173:6803/1646561048","heartbeat_back_addr":"172.21.15.173:6807/1646561048","heartbeat_front_addr":"172.21.15.173:6805/1646561048","state":["exists","up"]},{"osd":3,"uuid":"0cf3f7f1-3a8c-42ce-88a9-9bea52e02221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6811","nonce":406795560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6813","nonce":406795560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6817","nonce":406795560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6815","nonce":406795560}]},"public_addr":"172.21.15.164:6811/406795560","cluster_addr":"172.21.15.164:6813/406795560","heartbeat_back_addr":"172.21.15.164:6817/406795560","heartbeat_front_addr":"172.21.15.164:6815/406795560","state":["exists","up"]},{"osd":4,"uuid":"6dca048a-2860-4a02-8d8e-450b30bc1a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6809","nonce":4120832165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6811","nonce":4120832165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6815","nonce":4120832165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6813","nonce":4120832165}]},"public_addr":"172.21.15.170:6809/4120832165","cluster_addr":"172.21.15.170:6811/4120832165","heartbeat_back_addr":"172.21.15.170:6815/4120832165","heartbeat_front_addr":"172.21.15.170:6813/4120832165","state":["exists","up"]},{"osd":5,"uuid":"3822521f-69d6-4141-a83f-5fc9cecc7b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6809","nonce":2914653552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6811","nonce":2914653552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6815","nonce":2914653552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6813","nonce":2914653552}]},"public_addr":"172.21.15.173:6809/2914653552","cluster_addr":"172.21.15.173:6811/2914653552","heartbeat_back_addr":"172.21.15.173:6815/2914653552","heartbeat_front_addr":"172.21.15.173:6813/2914653552","state":["exists","up"]},{"osd":6,"uuid":"0f5a965e-2eb2-4c24-b089-577e0639c160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6819","nonce":1138341393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6821","nonce":1138341393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6825","nonce":1138341393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6823","nonce":1138341393}]},"public_addr":"172.21.15.164:6819/1138341393","cluster_addr":"172.21.15.164:6821/1138341393","heartbeat_back_addr":"172.21.15.164:6825/1138341393","heartbeat_front_addr":"172.21.15.164:6823/1138341393","state":["exists","up"]},{"osd":7,"uuid":"f16cb2c3-639b-46ae-a740-1bd28ca1863c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6817","nonce":2124160360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6819","nonce":2124160360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6823","nonce":2124160360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6821","nonce":2124160360}]},"public_addr":"172.21.15.170:6817/2124160360","cluster_addr":"172.21.15.170:6819/2124160360","heartbeat_back_addr":"172.21.15.170:6823/2124160360","heartbeat_front_addr":"172.21.15.170:6821/2124160360","state":["exists","up"]},{"osd":8,"uuid":"161cff01-46ad-48e3-b999-a5473a2ad618","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6817","nonce":2285175061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6819","nonce":2285175061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6823","nonce":2285175061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6821","nonce":2285175061}]},"public_addr":"172.21.15.173:6817/2285175061","cluster_addr":"172.21.15.173:6819/2285175061","heartbeat_back_addr":"172.21.15.173:6823/2285175061","heartbeat_front_addr":"172.21.15.173:6821/2285175061","state":["exists","up"]},{"osd":9,"uuid":"994666e8-8fbd-4231-ac67-81a79aaf7c2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6827","nonce":3918847952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6829","nonce":3918847952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6833","nonce":3918847952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6831","nonce":3918847952}]},"public_addr":"172.21.15.164:6827/3918847952","cluster_addr":"172.21.15.164:6829/3918847952","heartbeat_back_addr":"172.21.15.164:6833/3918847952","heartbeat_front_addr":"172.21.15.164:6831/3918847952","state":["exists","up"]},{"osd":10,"uuid":"ff5e9099-cdb4-4905-b888-5b67320ef76b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6825","nonce":2707848227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6827","nonce":2707848227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6831","nonce":2707848227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6829","nonce":2707848227}]},"public_addr":"172.21.15.170:6825/2707848227","cluster_addr":"172.21.15.170:6827/2707848227","heartbeat_back_addr":"172.21.15.170:6831/2707848227","heartbeat_front_addr":"172.21.15.170:6829/2707848227","state":["exists","up"]},{"osd":11,"uuid":"4d93088f-e791-4938-a207-12f15b56abd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6825","nonce":2831924738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6827","nonce":2831924738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6831","nonce":2831924738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6829","nonce":2831924738}]},"public_addr":"172.21.15.173:6825/2831924738","cluster_addr":"172.21.15.173:6827/2831924738","heartbeat_back_addr":"172.21.15.173:6831/2831924738","heartbeat_front_addr":"172.21.15.173:6829/2831924738","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:21.787352+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:46.012323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:39:11.353040+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:39:37.015124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:02.063379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:27.892865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:53.372041+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:41:17.855284+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:41:43.389882+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:08.956895+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:34.069779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:59.303897+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.164:0/3424843982":"2024-09-20T08:45:47.960147+0000","172.21.15.164:0/1369265436":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1807120560":"2024-09-20T08:45:47.960147+0000","172.21.15.164:6801/1094864383":"2024-09-20T08:45:47.960147+0000","172.21.15.164:0/1905394497":"2024-09-20T08:36:33.856642+0000","172.21.15.164:6800/2805828009":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/2770797546":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1837164919":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6800/1173822158":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6800/1094864383":"2024-09-20T08:45:47.960147+0000","172.21.15.164:6801/1173822158":"2024-09-20T08:37:01.663065+0000","172.21.15.164:0/339032808":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6801/2805828009":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/2087067799":"2024-09-20T08:45:47.960147+0000","172.21.15.164:0/4018459143":"2024-09-20T08:37:01.663065+0000","172.21.15.164:0/2727626049":"2024-09-20T08:45:47.960147+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:45:54.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.654+0000 7fbcab422640 1 -- 172.21.15.164:0/967050130 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fbc900a3440 msgr2=0x7fbc900a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:54.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.654+0000 7fbcab422640 1 --2- 172.21.15.164:0/967050130 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fbc900a3440 0x7fbc900a5900 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbca00058c0 tx=0x7fbca000a4c0 comp rx=0 tx=0).stop 2024-09-19T08:45:54.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.654+0000 7fbcab422640 1 -- 172.21.15.164:0/967050130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbca4102ef0 msgr2=0x7fbca41a43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:54.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.654+0000 7fbcab422640 1 --2- 172.21.15.164:0/967050130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbca4102ef0 0x7fbca41a43f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c02ae70 tx=0x7fbc9c02f5d0 comp rx=0 tx=0).stop 2024-09-19T08:45:54.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.654+0000 7fbcab422640 1 -- 172.21.15.164:0/967050130 shutdown_connections 2024-09-19T08:45:54.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.654+0000 7fbcab422640 1 --2- 172.21.15.164:0/967050130 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fbc900a3440 0x7fbc900a5900 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:54.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.654+0000 7fbcab422640 1 --2- 172.21.15.164:0/967050130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbca4102ef0 0x7fbca41a43f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:54.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.654+0000 7fbcab422640 1 --2- 172.21.15.164:0/967050130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbca4102570 0x7fbca41998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:54.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.654+0000 7fbcab422640 1 --2- 172.21.15.164:0/967050130 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbca4100e20 0x7fbca4199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:54.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.654+0000 7fbcab422640 1 -- 172.21.15.164:0/967050130 >> 172.21.15.164:0/967050130 conn(0x7fbca40fc7d0 msgr2=0x7fbca4106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:54.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.655+0000 7fbcab422640 1 -- 172.21.15.164:0/967050130 shutdown_connections 2024-09-19T08:45:54.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:54.655+0000 7fbcab422640 1 -- 172.21.15.164:0/967050130 wait complete. 2024-09-19T08:45:54.973 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:54 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/967050130' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:45:55.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:54 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/967050130' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:45:55.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:54 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/967050130' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:45:55.328 DEBUG:tasks.fwd_scrub:Yielding 2024-09-19T08:45:55.328 INFO:teuthology.run_tasks:Running task exec... 2024-09-19T08:45:55.337 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T08:45:55.337 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi164.front.sepia.ceph.com 2024-09-19T08:45:55.337 DEBUG:teuthology.orchestra.run.smithi164:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-19T08:45:55.338 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-19T08:45:55.338 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T08:45:55.338 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T08:45:55.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.443+0000 7fdc8b62e640 1 -- 172.21.15.164:0/290634793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc841069c0 msgr2=0x7fdc84106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:55.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.443+0000 7fdc8b62e640 1 --2- 172.21.15.164:0/290634793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc841069c0 0x7fdc84106da0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fdc80009930 tx=0x7fdc8002a420 comp rx=0 tx=0).stop 2024-09-19T08:45:55.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.443+0000 7fdc8b62e640 1 -- 172.21.15.164:0/290634793 shutdown_connections 2024-09-19T08:45:55.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.443+0000 7fdc8b62e640 1 --2- 172.21.15.164:0/290634793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc84101350 0x7fdc8410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.443+0000 7fdc8b62e640 1 --2- 172.21.15.164:0/290634793 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc841009b0 0x7fdc84100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.443+0000 7fdc8b62e640 1 --2- 172.21.15.164:0/290634793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc841069c0 0x7fdc84106da0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.443+0000 7fdc8b62e640 1 -- 172.21.15.164:0/290634793 >> 172.21.15.164:0/290634793 conn(0x7fdc84077990 msgr2=0x7fdc840fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:55.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.443+0000 7fdc8b62e640 1 -- 172.21.15.164:0/290634793 shutdown_connections 2024-09-19T08:45:55.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.443+0000 7fdc8b62e640 1 -- 172.21.15.164:0/290634793 wait complete. 2024-09-19T08:45:55.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.443+0000 7fdc8b62e640 1 Processor -- start 2024-09-19T08:45:55.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.444+0000 7fdc8b62e640 1 -- start start 2024-09-19T08:45:55.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.444+0000 7fdc8b62e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc841009b0 0x7fdc84197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:55.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.444+0000 7fdc8b62e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc84101350 0x7fdc841977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:55.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.444+0000 7fdc8b62e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc841069c0 0x7fdc8419bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:55.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.444+0000 7fdc8b62e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdc84110e00 con 0x7fdc84101350 2024-09-19T08:45:55.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.444+0000 7fdc8b62e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdc84110c80 con 0x7fdc841009b0 2024-09-19T08:45:55.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.444+0000 7fdc8b62e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdc84110f80 con 0x7fdc841069c0 2024-09-19T08:45:55.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.444+0000 7fdc88bc9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc84101350 0x7fdc841977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:55.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.444+0000 7fdc88bc9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc84101350 0x7fdc841977b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39200/0 (socket says 172.21.15.164:39200) 2024-09-19T08:45:55.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.444+0000 7fdc88bc9640 1 -- 172.21.15.164:0/2500027340 learned_addr learned my addr 172.21.15.164:0/2500027340 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:55.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.444+0000 7fdc89bcb640 1 --2- 172.21.15.164:0/2500027340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc841069c0 0x7fdc8419bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:55.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.444+0000 7fdc893ca640 1 --2- 172.21.15.164:0/2500027340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc841009b0 0x7fdc84197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:55.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.444+0000 7fdc88bc9640 1 -- 172.21.15.164:0/2500027340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc841069c0 msgr2=0x7fdc8419bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:55.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.444+0000 7fdc88bc9640 1 --2- 172.21.15.164:0/2500027340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc841069c0 0x7fdc8419bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.444+0000 7fdc88bc9640 1 -- 172.21.15.164:0/2500027340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc841009b0 msgr2=0x7fdc84197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:55.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.444+0000 7fdc88bc9640 1 --2- 172.21.15.164:0/2500027340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc841009b0 0x7fdc84197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.444+0000 7fdc88bc9640 1 -- 172.21.15.164:0/2500027340 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc8419c260 con 0x7fdc84101350 2024-09-19T08:45:55.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.444+0000 7fdc893ca640 1 --2- 172.21.15.164:0/2500027340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc841009b0 0x7fdc84197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:55.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.444+0000 7fdc88bc9640 1 --2- 172.21.15.164:0/2500027340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc84101350 0x7fdc841977b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fdc7800d770 tx=0x7fdc7800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:55.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.445+0000 7fdc767fc640 1 -- 172.21.15.164:0/2500027340 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc78017070 con 0x7fdc84101350 2024-09-19T08:45:55.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.445+0000 7fdc767fc640 1 -- 172.21.15.164:0/2500027340 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc78005500 con 0x7fdc84101350 2024-09-19T08:45:55.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.445+0000 7fdc8b62e640 1 -- 172.21.15.164:0/2500027340 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc8419c550 con 0x7fdc84101350 2024-09-19T08:45:55.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.445+0000 7fdc767fc640 1 -- 172.21.15.164:0/2500027340 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc78004e00 con 0x7fdc84101350 2024-09-19T08:45:55.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.445+0000 7fdc8b62e640 1 -- 172.21.15.164:0/2500027340 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc841a3de0 con 0x7fdc84101350 2024-09-19T08:45:55.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.446+0000 7fdc8b62e640 1 -- 172.21.15.164:0/2500027340 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc84100e10 con 0x7fdc84101350 2024-09-19T08:45:55.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.447+0000 7fdc767fc640 1 -- 172.21.15.164:0/2500027340 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7fdc78004fa0 con 0x7fdc84101350 2024-09-19T08:45:55.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.448+0000 7fdc767fc640 1 --2- 172.21.15.164:0/2500027340 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fdc700a3580 0x7fdc700a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:55.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.448+0000 7fdc893ca640 1 --2- 172.21.15.164:0/2500027340 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fdc700a3580 0x7fdc700a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:55.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.448+0000 7fdc767fc640 1 -- 172.21.15.164:0/2500027340 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fdc780cfc50 con 0x7fdc84101350 2024-09-19T08:45:55.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.448+0000 7fdc893ca640 1 --2- 172.21.15.164:0/2500027340 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fdc700a3580 0x7fdc700a5a40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdc80000c00 tx=0x7fdc80005a20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:55.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.452+0000 7fdc767fc640 1 -- 172.21.15.164:0/2500027340 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdc7809c5b0 con 0x7fdc84101350 2024-09-19T08:45:55.597 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:45:55.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.640+0000 7fdc8b62e640 1 -- 172.21.15.164:0/2500027340 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7fdc841a4080 con 0x7fdc84101350 2024-09-19T08:45:55.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.644+0000 7fdc767fc640 1 -- 172.21.15.164:0/2500027340 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v32)=0 v32) ==== 129+0+0 (secure 0 0 0) 0x7fdc7809beb0 con 0x7fdc84101350 2024-09-19T08:45:55.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.646+0000 7fdc8b62e640 1 -- 172.21.15.164:0/2500027340 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fdc700a3580 msgr2=0x7fdc700a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:55.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.646+0000 7fdc8b62e640 1 --2- 172.21.15.164:0/2500027340 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fdc700a3580 0x7fdc700a5a40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdc80000c00 tx=0x7fdc80005a20 comp rx=0 tx=0).stop 2024-09-19T08:45:55.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.646+0000 7fdc8b62e640 1 -- 172.21.15.164:0/2500027340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc84101350 msgr2=0x7fdc841977b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:55.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.646+0000 7fdc8b62e640 1 --2- 172.21.15.164:0/2500027340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc84101350 0x7fdc841977b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fdc7800d770 tx=0x7fdc7800dc30 comp rx=0 tx=0).stop 2024-09-19T08:45:55.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.646+0000 7fdc8b62e640 1 -- 172.21.15.164:0/2500027340 shutdown_connections 2024-09-19T08:45:55.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.646+0000 7fdc8b62e640 1 --2- 172.21.15.164:0/2500027340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc841069c0 0x7fdc8419bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.646+0000 7fdc8b62e640 1 --2- 172.21.15.164:0/2500027340 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fdc700a3580 0x7fdc700a5a40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.646+0000 7fdc8b62e640 1 --2- 172.21.15.164:0/2500027340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc84101350 0x7fdc841977b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.646+0000 7fdc8b62e640 1 --2- 172.21.15.164:0/2500027340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc841009b0 0x7fdc84197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.646+0000 7fdc8b62e640 1 -- 172.21.15.164:0/2500027340 >> 172.21.15.164:0/2500027340 conn(0x7fdc84077990 msgr2=0x7fdc84103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:55.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.646+0000 7fdc8b62e640 1 -- 172.21.15.164:0/2500027340 shutdown_connections 2024-09-19T08:45:55.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.646+0000 7fdc8b62e640 1 -- 172.21.15.164:0/2500027340 wait complete. 2024-09-19T08:45:55.659 DEBUG:teuthology.orchestra.run.smithi164:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-19T08:45:55.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.764+0000 7f458cc65640 1 -- 172.21.15.164:0/2356879309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4588107b80 msgr2=0x7f4588107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:55.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.764+0000 7f458cc65640 1 --2- 172.21.15.164:0/2356879309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4588107b80 0x7f4588107f60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4578009930 tx=0x7f457802a420 comp rx=0 tx=0).stop 2024-09-19T08:45:55.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.764+0000 7f458cc65640 1 -- 172.21.15.164:0/2356879309 shutdown_connections 2024-09-19T08:45:55.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.764+0000 7f458cc65640 1 --2- 172.21.15.164:0/2356879309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4588102480 0x7f458810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.764+0000 7f458cc65640 1 --2- 172.21.15.164:0/2356879309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4588101ae0 0x7f4588101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.764+0000 7f458cc65640 1 --2- 172.21.15.164:0/2356879309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4588107b80 0x7f4588107f60 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.764+0000 7f458cc65640 1 -- 172.21.15.164:0/2356879309 >> 172.21.15.164:0/2356879309 conn(0x7f45880fd7e0 msgr2=0x7f45880ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:55.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.764+0000 7f458cc65640 1 -- 172.21.15.164:0/2356879309 shutdown_connections 2024-09-19T08:45:55.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.764+0000 7f458cc65640 1 -- 172.21.15.164:0/2356879309 wait complete. 2024-09-19T08:45:55.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.765+0000 7f458cc65640 1 Processor -- start 2024-09-19T08:45:55.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.765+0000 7f458cc65640 1 -- start start 2024-09-19T08:45:55.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.765+0000 7f458cc65640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4588101ae0 0x7f4588198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:55.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.765+0000 7f458cc65640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4588102480 0x7f45881988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:55.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.765+0000 7f458cc65640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4588107b80 0x7f458819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:55.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.765+0000 7f458cc65640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4588111f30 con 0x7f4588101ae0 2024-09-19T08:45:55.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.765+0000 7f458cc65640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4588111db0 con 0x7f4588102480 2024-09-19T08:45:55.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.765+0000 7f458cc65640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f45881120b0 con 0x7f4588107b80 2024-09-19T08:45:55.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.765+0000 7f458659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4588101ae0 0x7f4588198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:55.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.766+0000 7f458659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4588101ae0 0x7f4588198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39214/0 (socket says 172.21.15.164:39214) 2024-09-19T08:45:55.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.766+0000 7f458659c640 1 -- 172.21.15.164:0/157362804 learned_addr learned my addr 172.21.15.164:0/157362804 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:55.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.766+0000 7f4586d9d640 1 --2- 172.21.15.164:0/157362804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4588107b80 0x7f458819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:55.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.766+0000 7f4585d9b640 1 --2- 172.21.15.164:0/157362804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4588102480 0x7f45881988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:55.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.766+0000 7f458659c640 1 -- 172.21.15.164:0/157362804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4588107b80 msgr2=0x7f458819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:55.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.766+0000 7f458659c640 1 --2- 172.21.15.164:0/157362804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4588107b80 0x7f458819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.766+0000 7f458659c640 1 -- 172.21.15.164:0/157362804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4588102480 msgr2=0x7f45881988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:55.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.766+0000 7f458659c640 1 --2- 172.21.15.164:0/157362804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4588102480 0x7f45881988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.766+0000 7f458659c640 1 -- 172.21.15.164:0/157362804 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458819d370 con 0x7f4588101ae0 2024-09-19T08:45:55.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.766+0000 7f458659c640 1 --2- 172.21.15.164:0/157362804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4588101ae0 0x7f4588198380 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4578009230 tx=0x7f4578004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:55.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.766+0000 7f45777fe640 1 -- 172.21.15.164:0/157362804 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457803f030 con 0x7f4588101ae0 2024-09-19T08:45:55.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.766+0000 7f45777fe640 1 -- 172.21.15.164:0/157362804 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4578003ad0 con 0x7f4588101ae0 2024-09-19T08:45:55.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.766+0000 7f458cc65640 1 -- 172.21.15.164:0/157362804 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f458819d600 con 0x7f4588101ae0 2024-09-19T08:45:55.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.766+0000 7f45777fe640 1 -- 172.21.15.164:0/157362804 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4578035070 con 0x7f4588101ae0 2024-09-19T08:45:55.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.766+0000 7f458cc65640 1 -- 172.21.15.164:0/157362804 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45881a4ef0 con 0x7f4588101ae0 2024-09-19T08:45:55.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.767+0000 7f458cc65640 1 -- 172.21.15.164:0/157362804 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45881031e0 con 0x7f4588101ae0 2024-09-19T08:45:55.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.768+0000 7f45777fe640 1 -- 172.21.15.164:0/157362804 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f4578042030 con 0x7f4588101ae0 2024-09-19T08:45:55.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.769+0000 7f45777fe640 1 --2- 172.21.15.164:0/157362804 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f45700a3500 0x7f45700a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:55.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.769+0000 7f45777fe640 1 -- 172.21.15.164:0/157362804 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f45780effe0 con 0x7f4588101ae0 2024-09-19T08:45:55.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.769+0000 7f4585d9b640 1 --2- 172.21.15.164:0/157362804 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f45700a3500 0x7f45700a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:55.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.770+0000 7f4585d9b640 1 --2- 172.21.15.164:0/157362804 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f45700a3500 0x7f45700a59c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f457c005fd0 tx=0x7f457c005850 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:55.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.773+0000 7f45777fe640 1 -- 172.21.15.164:0/157362804 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f45780bc8c0 con 0x7f4588101ae0 2024-09-19T08:45:55.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.947+0000 7f458cc65640 1 -- 172.21.15.164:0/157362804 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f45881a50c0 con 0x7f4588101ae0 2024-09-19T08:45:55.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.951+0000 7f45777fe640 1 -- 172.21.15.164:0/157362804 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v33)=0 v33) ==== 138+0+0 (secure 0 0 0) 0x7f45780bc1c0 con 0x7f4588101ae0 2024-09-19T08:45:55.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.954+0000 7f458cc65640 1 -- 172.21.15.164:0/157362804 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f45700a3500 msgr2=0x7f45700a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:55.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.954+0000 7f458cc65640 1 --2- 172.21.15.164:0/157362804 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f45700a3500 0x7f45700a59c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f457c005fd0 tx=0x7f457c005850 comp rx=0 tx=0).stop 2024-09-19T08:45:55.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.954+0000 7f458cc65640 1 -- 172.21.15.164:0/157362804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4588101ae0 msgr2=0x7f4588198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:55.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.954+0000 7f458cc65640 1 --2- 172.21.15.164:0/157362804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4588101ae0 0x7f4588198380 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4578009230 tx=0x7f4578004270 comp rx=0 tx=0).stop 2024-09-19T08:45:55.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.954+0000 7f458cc65640 1 -- 172.21.15.164:0/157362804 shutdown_connections 2024-09-19T08:45:55.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.954+0000 7f458cc65640 1 --2- 172.21.15.164:0/157362804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4588107b80 0x7f458819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.954+0000 7f458cc65640 1 --2- 172.21.15.164:0/157362804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4588102480 0x7f45881988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.954+0000 7f458cc65640 1 --2- 172.21.15.164:0/157362804 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f45700a3500 0x7f45700a59c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.954+0000 7f458cc65640 1 --2- 172.21.15.164:0/157362804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4588101ae0 0x7f4588198380 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.954+0000 7f458cc65640 1 -- 172.21.15.164:0/157362804 >> 172.21.15.164:0/157362804 conn(0x7f45880fd7e0 msgr2=0x7f45880feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:55.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.954+0000 7f458cc65640 1 -- 172.21.15.164:0/157362804 shutdown_connections 2024-09-19T08:45:55.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:55.954+0000 7f458cc65640 1 -- 172.21.15.164:0/157362804 wait complete. 2024-09-19T08:45:55.967 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:55 smithi164 ceph-mon[30928]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 0 B/s wr, 26 op/s 2024-09-19T08:45:55.967 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:55 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2500027340' entity='client.admin' 2024-09-19T08:45:55.967 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:55 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:45:55.967 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:55 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:45:55.967 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:55 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:45:55.967 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:55 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:55.968 DEBUG:teuthology.orchestra.run.smithi164:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-19T08:45:56.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.059+0000 7f1387569640 1 -- 172.21.15.164:0/891465608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1380106a80 msgr2=0x7f1380106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.059+0000 7f1387569640 1 --2- 172.21.15.164:0/891465608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1380106a80 0x7f1380106e60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1370009930 tx=0x7f137002a420 comp rx=0 tx=0).stop 2024-09-19T08:45:56.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.059+0000 7f1387569640 1 -- 172.21.15.164:0/891465608 shutdown_connections 2024-09-19T08:45:56.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.059+0000 7f1387569640 1 --2- 172.21.15.164:0/891465608 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1380101420 0x7f138010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.059+0000 7f1387569640 1 --2- 172.21.15.164:0/891465608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1380100a80 0x7f1380100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.059+0000 7f1387569640 1 --2- 172.21.15.164:0/891465608 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1380106a80 0x7f1380106e60 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.059+0000 7f1387569640 1 -- 172.21.15.164:0/891465608 >> 172.21.15.164:0/891465608 conn(0x7f13800fc7a0 msgr2=0x7f13800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:56.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.059+0000 7f1387569640 1 -- 172.21.15.164:0/891465608 shutdown_connections 2024-09-19T08:45:56.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.059+0000 7f1387569640 1 -- 172.21.15.164:0/891465608 wait complete. 2024-09-19T08:45:56.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.060+0000 7f1387569640 1 Processor -- start 2024-09-19T08:45:56.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.060+0000 7f1387569640 1 -- start start 2024-09-19T08:45:56.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.060+0000 7f1387569640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1380100a80 0x7f138019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.060+0000 7f1387569640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1380101420 0x7f138019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.060+0000 7f1387569640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1380106a80 0x7f138019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.060+0000 7f1387569640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1380110e80 con 0x7f1380101420 2024-09-19T08:45:56.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.060+0000 7f1387569640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1380110d00 con 0x7f1380100a80 2024-09-19T08:45:56.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.060+0000 7f1387569640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1380111000 con 0x7f1380106a80 2024-09-19T08:45:56.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.060+0000 7f1384b04640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1380101420 0x7f138019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.060+0000 7f1384b04640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1380101420 0x7f138019aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39230/0 (socket says 172.21.15.164:39230) 2024-09-19T08:45:56.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.060+0000 7f1384b04640 1 -- 172.21.15.164:0/3012013258 learned_addr learned my addr 172.21.15.164:0/3012013258 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:56.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.060+0000 7f1385b06640 1 --2- 172.21.15.164:0/3012013258 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1380106a80 0x7f138019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.060+0000 7f1385305640 1 --2- 172.21.15.164:0/3012013258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1380100a80 0x7f138019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.061+0000 7f1384b04640 1 -- 172.21.15.164:0/3012013258 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1380106a80 msgr2=0x7f138019ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.061+0000 7f1384b04640 1 --2- 172.21.15.164:0/3012013258 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1380106a80 0x7f138019ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.061+0000 7f1384b04640 1 -- 172.21.15.164:0/3012013258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1380100a80 msgr2=0x7f138019a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.061+0000 7f1384b04640 1 --2- 172.21.15.164:0/3012013258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1380100a80 0x7f138019a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.061+0000 7f1384b04640 1 -- 172.21.15.164:0/3012013258 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f138019f560 con 0x7f1380101420 2024-09-19T08:45:56.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.061+0000 7f1385305640 1 --2- 172.21.15.164:0/3012013258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1380100a80 0x7f138019a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:56.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.061+0000 7f1384b04640 1 --2- 172.21.15.164:0/3012013258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1380101420 0x7f138019aaf0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f137c00c920 tx=0x7f137c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:56.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.061+0000 7f13767fc640 1 -- 172.21.15.164:0/3012013258 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c007c70 con 0x7f1380101420 2024-09-19T08:45:56.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.061+0000 7f1387569640 1 -- 172.21.15.164:0/3012013258 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f138019f850 con 0x7f1380101420 2024-09-19T08:45:56.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.061+0000 7f13767fc640 1 -- 172.21.15.164:0/3012013258 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f137c007e10 con 0x7f1380101420 2024-09-19T08:45:56.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.061+0000 7f13767fc640 1 -- 172.21.15.164:0/3012013258 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c018890 con 0x7f1380101420 2024-09-19T08:45:56.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.061+0000 7f1387569640 1 -- 172.21.15.164:0/3012013258 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13801a7130 con 0x7f1380101420 2024-09-19T08:45:56.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.063+0000 7f13767fc640 1 -- 172.21.15.164:0/3012013258 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f137c018a70 con 0x7f1380101420 2024-09-19T08:45:56.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.064+0000 7f1389942740 1 Processor -- start 2024-09-19T08:45:56.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.064+0000 7f1389942740 1 -- start start 2024-09-19T08:45:56.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.064+0000 7f1389942740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556574f5bb60 0x556574f5bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.064+0000 7f1389942740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556574f5c480 0x556574f66800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.064+0000 7f1389942740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556574f66d40 0x556574f69130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.064+0000 7f1389942740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556574ec1280 con 0x556574f66d40 2024-09-19T08:45:56.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.064+0000 7f1389942740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556574e68b10 con 0x556574f5c480 2024-09-19T08:45:56.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.064+0000 7f1389942740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556574ebd240 con 0x556574f5bb60 2024-09-19T08:45:56.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.064+0000 7f1385b06640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556574f5bb60 0x556574f5bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.064+0000 7f1384b04640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556574f66d40 0x556574f69130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.064+0000 7f1385b06640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556574f5bb60 0x556574f5bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:48936/0 (socket says 172.21.15.164:48936) 2024-09-19T08:45:56.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.064+0000 7f1385b06640 1 -- 172.21.15.164:0/962081183 learned_addr learned my addr 172.21.15.164:0/962081183 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:56.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.064+0000 7f1385305640 1 --2- 172.21.15.164:0/962081183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556574f5c480 0x556574f66800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.064+0000 7f13767fc640 1 --2- 172.21.15.164:0/3012013258 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f136c0a3440 0x7f136c0a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.065+0000 7f1385b06640 1 -- 172.21.15.164:0/962081183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556574f5c480 msgr2=0x556574f66800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:56.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.065+0000 7f1385b06640 1 --2- 172.21.15.164:0/962081183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556574f5c480 0x556574f66800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.065+0000 7f1385b06640 1 -- 172.21.15.164:0/962081183 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556574f66d40 msgr2=0x556574f69130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:56.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.065+0000 7f1385b06640 1 --2- 172.21.15.164:0/962081183 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556574f66d40 0x556574f69130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.065+0000 7f1385b06640 1 -- 172.21.15.164:0/962081183 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556574e751f0 con 0x556574f5bb60 2024-09-19T08:45:56.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.065+0000 7f1384b04640 1 --2- 172.21.15.164:0/962081183 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556574f66d40 0x556574f69130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:45:56.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.065+0000 7f1385b06640 1 --2- 172.21.15.164:0/3012013258 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f136c0a3440 0x7f136c0a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.065+0000 7f13767fc640 1 -- 172.21.15.164:0/3012013258 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f137c0cffc0 con 0x7f1380101420 2024-09-19T08:45:56.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.065+0000 7f1385b06640 1 --2- 172.21.15.164:0/962081183 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556574f5bb60 0x556574f5bf40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1378011590 tx=0x7f1378011a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c5825e92ce28542 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:56.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.065+0000 7f136b7fe640 1 -- 172.21.15.164:0/962081183 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1378017050 con 0x556574f5bb60 2024-09-19T08:45:56.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.065+0000 7f1385b06640 1 --2- 172.21.15.164:0/3012013258 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f136c0a3440 0x7f136c0a5900 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1378022000 tx=0x7f1378023000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:56.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.065+0000 7f136b7fe640 1 -- 172.21.15.164:0/962081183 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f137800f7a0 con 0x556574f5bb60 2024-09-19T08:45:56.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.065+0000 7f136b7fe640 1 -- 172.21.15.164:0/962081183 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137801a5a0 con 0x556574f5bb60 2024-09-19T08:45:56.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.066+0000 7f1389942740 1 -- 172.21.15.164:0/962081183 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556574f5bb60 msgr2=0x556574f5bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.066+0000 7f1389942740 1 --2- 172.21.15.164:0/962081183 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556574f5bb60 0x556574f5bf40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1378011590 tx=0x7f1378011a50 comp rx=0 tx=0).stop 2024-09-19T08:45:56.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.066+0000 7f1389942740 1 -- 172.21.15.164:0/962081183 shutdown_connections 2024-09-19T08:45:56.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.066+0000 7f1389942740 1 --2- 172.21.15.164:0/962081183 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556574f66d40 0x556574f69130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.066+0000 7f1389942740 1 --2- 172.21.15.164:0/962081183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556574f5c480 0x556574f66800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.066+0000 7f1389942740 1 --2- 172.21.15.164:0/962081183 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556574f5bb60 0x556574f5bf40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.066+0000 7f1389942740 1 -- 172.21.15.164:0/962081183 >> 172.21.15.164:0/962081183 conn(0x556574f580f0 msgr2=0x556574f584d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:56.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.066+0000 7f1389942740 1 -- 172.21.15.164:0/962081183 shutdown_connections 2024-09-19T08:45:56.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.066+0000 7f1389942740 1 -- 172.21.15.164:0/962081183 wait complete. 2024-09-19T08:45:56.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.066+0000 7f1389942740 1 -- start start 2024-09-19T08:45:56.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.066+0000 7f1389942740 1 Processor -- start 2024-09-19T08:45:56.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.067+0000 7f1389942740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556574f5bb60 0x556574f63c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.067+0000 7f1389942740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556574f5c480 0x556574f641a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.067+0000 7f1389942740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556574f66d40 0x556574f646e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.067+0000 7f1389942740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556574f69910 con 0x556574f66d40 2024-09-19T08:45:56.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.067+0000 7f1389942740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556574e7bf10 con 0x556574f5c480 2024-09-19T08:45:56.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.067+0000 7f1389942740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556574f69a90 con 0x556574f5bb60 2024-09-19T08:45:56.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.067+0000 7f1389942740 10 client.0 _ll_register_callbacks cb 0x556574e44a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:45:56.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.067+0000 7f1385b06640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556574f66d40 0x556574f646e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.067+0000 7f1385305640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556574f5bb60 0x556574f63c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.067+0000 7f1385305640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556574f5bb60 0x556574f63c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:48950/0 (socket says 172.21.15.164:48950) 2024-09-19T08:45:56.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.067+0000 7f1385305640 1 -- 172.21.15.164:0/1604778199 learned_addr learned my addr 172.21.15.164:0/1604778199 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:56.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.067+0000 7f1384b04640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556574f5c480 0x556574f641a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.067+0000 7f1385b06640 1 -- 172.21.15.164:0/1604778199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556574f5bb60 msgr2=0x556574f63c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:56.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.067+0000 7f1385b06640 1 --2- 172.21.15.164:0/1604778199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556574f5bb60 0x556574f63c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.067+0000 7f1385b06640 1 -- 172.21.15.164:0/1604778199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556574f5c480 msgr2=0x556574f641a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:56.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.067+0000 7f1385b06640 1 --2- 172.21.15.164:0/1604778199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556574f5c480 0x556574f641a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.067+0000 7f1385b06640 1 -- 172.21.15.164:0/1604778199 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556574c326a0 con 0x556574f66d40 2024-09-19T08:45:56.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.067+0000 7f1385305640 1 --2- 172.21.15.164:0/1604778199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556574f5bb60 0x556574f63c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:45:56.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.067+0000 7f1384b04640 1 --2- 172.21.15.164:0/1604778199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556574f5c480 0x556574f641a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:45:56.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.068+0000 7f1385b06640 1 --2- 172.21.15.164:0/1604778199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556574f66d40 0x556574f646e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f137800ffd0 tx=0x7f137800ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d611c7dafa92366 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:56.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.068+0000 7f1368ff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:45:56.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.068+0000 7f1368ff9640 1 -- 172.21.15.164:0/1604778199 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1378034060 con 0x556574f66d40 2024-09-19T08:45:56.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.068+0000 7f1368ff9640 1 -- 172.21.15.164:0/1604778199 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f137801bb90 con 0x556574f66d40 2024-09-19T08:45:56.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.068+0000 7f1389942740 1 -- 172.21.15.164:0/1604778199 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556574ece7f0 con 0x556574f66d40 2024-09-19T08:45:56.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.068+0000 7f1368ff9640 1 -- 172.21.15.164:0/1604778199 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1378017020 con 0x556574f66d40 2024-09-19T08:45:56.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.068+0000 7f1389942740 10 client.15339 fetch_fsmap learned FSMap version 28 2024-09-19T08:45:56.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.068+0000 7f1389942740 1 -- 172.21.15.164:0/1604778199 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556574f75550 con 0x556574f66d40 2024-09-19T08:45:56.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.068+0000 7f1368ff9640 1 -- 172.21.15.164:0/1604778199 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556574ece7f0 con 0x556574f66d40 2024-09-19T08:45:56.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.068+0000 7f1368ff9640 1 -- 172.21.15.164:0/1604778199 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1378037030 con 0x556574f66d40 2024-09-19T08:45:56.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.068+0000 7f1389942740 10 client.15339 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:45:56.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.068+0000 7f1389942740 10 client.15339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:45:56.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.068+0000 7f1389942740 20 client.15339 populate_metadata read hostname 'smithi164' 2024-09-19T08:45:56.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.068+0000 7f1389942740 1 --2- 172.21.15.164:0/1604778199 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556574f769b0 0x556574f78da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.068+0000 7f1389942740 4 client.15339 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:45:56.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.068+0000 7f1389942740 1 -- 172.21.15.164:0/1604778199 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556574ec8d00 con 0x556574f769b0 2024-09-19T08:45:56.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.069+0000 7f1385305640 1 --2- 172.21.15.164:0/1604778199 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556574f769b0 0x556574f78da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.069+0000 7f1385305640 1 --2- 172.21.15.164:0/1604778199 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556574f769b0 0x556574f78da0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9ffaba6b8c13151 server_cookie=2a45eff3020163e7 in_seq=0 out_seq=0 2024-09-19T08:45:56.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.069+0000 7f1368ff9640 10 client.15339 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:45:56.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.070+0000 7f1368ff9640 1 -- 172.21.15.164:0/1604778199 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556574ec8d00 con 0x556574f769b0 2024-09-19T08:45:56.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.070+0000 7f1368ff9640 10 client.15339 handle_command_reply: tid=0 2024-09-19T08:45:56.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.070+0000 7f1389942740 1 client.15339 shutdown 2024-09-19T08:45:56.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.070+0000 7f1389942740 1 -- 172.21.15.164:0/1604778199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556574f66d40 msgr2=0x556574f646e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.070+0000 7f1389942740 1 --2- 172.21.15.164:0/1604778199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556574f66d40 0x556574f646e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f137800ffd0 tx=0x7f137800ebc0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.070+0000 7f1389942740 1 -- 172.21.15.164:0/1604778199 shutdown_connections 2024-09-19T08:45:56.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.070+0000 7f1389942740 1 --2- 172.21.15.164:0/1604778199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556574f66d40 0x556574f646e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.070+0000 7f1389942740 1 --2- 172.21.15.164:0/1604778199 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556574f769b0 0x556574f78da0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.070+0000 7f1389942740 1 --2- 172.21.15.164:0/1604778199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556574f5c480 0x556574f641a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.070+0000 7f1389942740 1 --2- 172.21.15.164:0/1604778199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556574f5bb60 0x556574f63c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.070+0000 7f1368ff9640 0 client.15339 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:45:56.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.070+0000 7f1389942740 1 -- 172.21.15.164:0/1604778199 >> 172.21.15.164:0/1604778199 conn(0x556574f580f0 msgr2=0x556574f5ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:56.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.070+0000 7f1389942740 1 -- 172.21.15.164:0/1604778199 shutdown_connections 2024-09-19T08:45:56.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.070+0000 7f1389942740 1 -- 172.21.15.164:0/1604778199 wait complete. 2024-09-19T08:45:56.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.070+0000 7f1389942740 20 client.15339 trim_cache size 0 max 16384 2024-09-19T08:45:56.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.086+0000 7f71d14c9640 1 -- 172.21.15.164:0/789795408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71cc101350 msgr2=0x7f71cc10d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.086+0000 7f71d14c9640 1 --2- 172.21.15.164:0/789795408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71cc101350 0x7f71cc10d880 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f71c4009cd0 tx=0x7f71c402a8e0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.086+0000 7f71d14c9640 1 -- 172.21.15.164:0/789795408 shutdown_connections 2024-09-19T08:45:56.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.086+0000 7f71d14c9640 1 --2- 172.21.15.164:0/789795408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71cc101350 0x7f71cc10d880 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.086+0000 7f71d14c9640 1 --2- 172.21.15.164:0/789795408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71cc1009b0 0x7f71cc100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.086+0000 7f71d14c9640 1 --2- 172.21.15.164:0/789795408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71cc1069c0 0x7f71cc106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.087+0000 7f71d14c9640 1 -- 172.21.15.164:0/789795408 >> 172.21.15.164:0/789795408 conn(0x7f71cc077990 msgr2=0x7f71cc0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:56.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.087+0000 7f71d14c9640 1 -- 172.21.15.164:0/789795408 shutdown_connections 2024-09-19T08:45:56.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.087+0000 7f71d14c9640 1 -- 172.21.15.164:0/789795408 wait complete. 2024-09-19T08:45:56.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.087+0000 7f71d14c9640 1 Processor -- start 2024-09-19T08:45:56.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.087+0000 7f71d14c9640 1 -- start start 2024-09-19T08:45:56.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.087+0000 7f71d14c9640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71cc1009b0 0x7f71cc19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.087+0000 7f71d14c9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71cc101350 0x7f71cc19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.087+0000 7f71d14c9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71cc1069c0 0x7f71cc19ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.088+0000 7f71d14c9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc110e00 con 0x7f71cc101350 2024-09-19T08:45:56.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.088+0000 7f71d14c9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc110c80 con 0x7f71cc1069c0 2024-09-19T08:45:56.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.088+0000 7f71d14c9640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc110f80 con 0x7f71cc1009b0 2024-09-19T08:45:56.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.088+0000 7f71ca7fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71cc101350 0x7f71cc19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.088+0000 7f71ca7fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71cc101350 0x7f71cc19aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39272/0 (socket says 172.21.15.164:39272) 2024-09-19T08:45:56.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.088+0000 7f71ca7fc640 1 -- 172.21.15.164:0/373542985 learned_addr learned my addr 172.21.15.164:0/373542985 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:56.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.088+0000 7f71caffd640 1 --2- 172.21.15.164:0/373542985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71cc1009b0 0x7f71cc19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.088+0000 7f71cb7fe640 1 --2- 172.21.15.164:0/373542985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71cc1069c0 0x7f71cc19ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.088+0000 7f71ca7fc640 1 -- 172.21.15.164:0/373542985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71cc1009b0 msgr2=0x7f71cc19a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.088+0000 7f71ca7fc640 1 --2- 172.21.15.164:0/373542985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71cc1009b0 0x7f71cc19a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.088+0000 7f71ca7fc640 1 -- 172.21.15.164:0/373542985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71cc1069c0 msgr2=0x7f71cc19ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.088+0000 7f71ca7fc640 1 --2- 172.21.15.164:0/373542985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71cc1069c0 0x7f71cc19ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.088+0000 7f71ca7fc640 1 -- 172.21.15.164:0/373542985 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71cc19f550 con 0x7f71cc101350 2024-09-19T08:45:56.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.088+0000 7f1389942740 1 Processor -- start 2024-09-19T08:45:56.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.088+0000 7f1389942740 1 -- start start 2024-09-19T08:45:56.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.088+0000 7f71ca7fc640 1 --2- 172.21.15.164:0/373542985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71cc101350 0x7f71cc19aaa0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f71c000e9f0 tx=0x7f71c000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:56.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.088+0000 7f1389942740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556574f5bb60 0x556574f63580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.088+0000 7f1389942740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556574f5c480 0x556574f7e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.088+0000 7f1389942740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556574f66d40 0x556574f7e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.088+0000 7f1389942740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f13801af930 con 0x556574f66d40 2024-09-19T08:45:56.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.088+0000 7f1389942740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f13801b1470 con 0x556574f5bb60 2024-09-19T08:45:56.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.088+0000 7f71b3fff640 1 -- 172.21.15.164:0/373542985 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c000cd80 con 0x7f71cc101350 2024-09-19T08:45:56.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.088+0000 7f71b3fff640 1 -- 172.21.15.164:0/373542985 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71c0005500 con 0x7f71cc101350 2024-09-19T08:45:56.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.088+0000 7f71d14c9640 1 -- 172.21.15.164:0/373542985 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71cc19f840 con 0x7f71cc101350 2024-09-19T08:45:56.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.088+0000 7f1389942740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f13801b0000 con 0x556574f5c480 2024-09-19T08:45:56.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.088+0000 7f1385305640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556574f5bb60 0x556574f63580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.088+0000 7f71b3fff640 1 -- 172.21.15.164:0/373542985 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c0010690 con 0x7f71cc101350 2024-09-19T08:45:56.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.088+0000 7f1385305640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556574f5bb60 0x556574f63580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:57560/0 (socket says 172.21.15.164:57560) 2024-09-19T08:45:56.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.088+0000 7f1385305640 1 -- 172.21.15.164:0/1231212979 learned_addr learned my addr 172.21.15.164:0/1231212979 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:56.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.088+0000 7f1384b04640 1 --2- 172.21.15.164:0/1231212979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556574f5c480 0x556574f7e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.088+0000 7f1385b06640 1 --2- 172.21.15.164:0/1231212979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556574f66d40 0x556574f7e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.089+0000 7f71d14c9640 1 -- 172.21.15.164:0/373542985 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71cc1a70d0 con 0x7f71cc101350 2024-09-19T08:45:56.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.089+0000 7f1385305640 1 -- 172.21.15.164:0/1231212979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556574f5c480 msgr2=0x556574f7e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:56.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.089+0000 7f1385305640 1 --2- 172.21.15.164:0/1231212979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556574f5c480 0x556574f7e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.089+0000 7f1385305640 1 -- 172.21.15.164:0/1231212979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556574f66d40 msgr2=0x556574f7e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:56.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.089+0000 7f1385305640 1 --2- 172.21.15.164:0/1231212979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556574f66d40 0x556574f7e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.089+0000 7f1385305640 1 -- 172.21.15.164:0/1231212979 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556574f53830 con 0x556574f5bb60 2024-09-19T08:45:56.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.089+0000 7f1385b06640 1 --2- 172.21.15.164:0/1231212979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556574f66d40 0x556574f7e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:45:56.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.089+0000 7f1385305640 1 --2- 172.21.15.164:0/1231212979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556574f5bb60 0x556574f63580 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1370006010 tx=0x7f1370031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc2de7c162e0d73e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:56.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.089+0000 7f136affd640 1 -- 172.21.15.164:0/1231212979 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1370035040 con 0x556574f5bb60 2024-09-19T08:45:56.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.089+0000 7f136affd640 1 -- 172.21.15.164:0/1231212979 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f137004a070 con 0x556574f5bb60 2024-09-19T08:45:56.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.089+0000 7f136affd640 1 -- 172.21.15.164:0/1231212979 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1370004990 con 0x556574f5bb60 2024-09-19T08:45:56.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.090+0000 7f71d14c9640 1 -- 172.21.15.164:0/373542985 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71cc100e10 con 0x7f71cc101350 2024-09-19T08:45:56.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.090+0000 7f1389942740 1 -- 172.21.15.164:0/1231212979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556574f5bb60 msgr2=0x556574f63580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.090+0000 7f1389942740 1 --2- 172.21.15.164:0/1231212979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556574f5bb60 0x556574f63580 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1370006010 tx=0x7f1370031400 comp rx=0 tx=0).stop 2024-09-19T08:45:56.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.090+0000 7f1389942740 1 -- 172.21.15.164:0/1231212979 shutdown_connections 2024-09-19T08:45:56.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.090+0000 7f1389942740 1 --2- 172.21.15.164:0/1231212979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556574f66d40 0x556574f7e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.090+0000 7f1389942740 1 --2- 172.21.15.164:0/1231212979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556574f5c480 0x556574f7e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.090+0000 7f1389942740 1 --2- 172.21.15.164:0/1231212979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556574f5bb60 0x556574f63580 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.090+0000 7f1389942740 1 -- 172.21.15.164:0/1231212979 >> 172.21.15.164:0/1231212979 conn(0x556574f580f0 msgr2=0x556574f65bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:56.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.090+0000 7f1389942740 1 -- 172.21.15.164:0/1231212979 shutdown_connections 2024-09-19T08:45:56.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.090+0000 7f1389942740 1 -- 172.21.15.164:0/1231212979 wait complete. 2024-09-19T08:45:56.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.090+0000 7f1389942740 1 -- start start 2024-09-19T08:45:56.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.091+0000 7f1389942740 1 Processor -- start 2024-09-19T08:45:56.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.091+0000 7f1389942740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556574f5bb60 0x556574f6eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.091+0000 7f1389942740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556574f5c480 0x556574f6f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.091+0000 7f1389942740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556574f66d40 0x556574f60500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.091+0000 7f1389942740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f13801b0da0 con 0x556574f5c480 2024-09-19T08:45:56.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.091+0000 7f1389942740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f13801b06d0 con 0x556574f5bb60 2024-09-19T08:45:56.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.091+0000 7f1389942740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f13801ad910 con 0x556574f66d40 2024-09-19T08:45:56.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.091+0000 7f1384b04640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556574f5c480 0x556574f6f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.091+0000 7f1389942740 10 client.0 _ll_register_callbacks cb 0x7f1378011f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:45:56.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.091+0000 7f1384b04640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556574f5c480 0x556574f6f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39282/0 (socket says 172.21.15.164:39282) 2024-09-19T08:45:56.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.091+0000 7f1384b04640 1 -- 172.21.15.164:0/2955674388 learned_addr learned my addr 172.21.15.164:0/2955674388 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:56.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.091+0000 7f1385305640 1 --2- 172.21.15.164:0/2955674388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556574f5bb60 0x556574f6eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.091+0000 7f1385b06640 1 --2- 172.21.15.164:0/2955674388 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556574f66d40 0x556574f60500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.092+0000 7f1384b04640 1 -- 172.21.15.164:0/2955674388 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556574f66d40 msgr2=0x556574f60500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:56.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.092+0000 7f1384b04640 1 --2- 172.21.15.164:0/2955674388 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556574f66d40 0x556574f60500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.092+0000 7f1384b04640 1 -- 172.21.15.164:0/2955674388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556574f5bb60 msgr2=0x556574f6eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:56.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.092+0000 7f71b3fff640 1 -- 172.21.15.164:0/373542985 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f71c00046c0 con 0x7f71cc101350 2024-09-19T08:45:56.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.092+0000 7f1384b04640 1 --2- 172.21.15.164:0/2955674388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556574f5bb60 0x556574f6eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.092+0000 7f1384b04640 1 -- 172.21.15.164:0/2955674388 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556574f60d30 con 0x556574f5c480 2024-09-19T08:45:56.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.092+0000 7f1384b04640 1 --2- 172.21.15.164:0/2955674388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556574f5c480 0x556574f6f400 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f137c007590 tx=0x7f137c00a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2550eeb933b96b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:56.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.092+0000 7f1368ff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:45:56.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.092+0000 7f1368ff9640 1 -- 172.21.15.164:0/2955674388 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c09bb00 con 0x556574f5c480 2024-09-19T08:45:56.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.092+0000 7f1368ff9640 1 -- 172.21.15.164:0/2955674388 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f137c0850b0 con 0x556574f5c480 2024-09-19T08:45:56.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.092+0000 7f1389942740 1 -- 172.21.15.164:0/2955674388 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f137004a070 con 0x556574f5c480 2024-09-19T08:45:56.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.093+0000 7f71b3fff640 1 --2- 172.21.15.164:0/373542985 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f71b40a3580 0x7f71b40a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.093+0000 7f71caffd640 1 --2- 172.21.15.164:0/373542985 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f71b40a3580 0x7f71b40a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.093+0000 7f71b3fff640 1 -- 172.21.15.164:0/373542985 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f71c00d0280 con 0x7f71cc101350 2024-09-19T08:45:56.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.093+0000 7f71caffd640 1 --2- 172.21.15.164:0/373542985 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f71b40a3580 0x7f71b40a5a40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009b50 tx=0x7f71bc005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:56.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.097+0000 7f1368ff9640 1 -- 172.21.15.164:0/2955674388 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f137c09c8a0 con 0x556574f5c480 2024-09-19T08:45:56.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.097+0000 7f1389942740 10 client.15351 fetch_fsmap learned FSMap version 28 2024-09-19T08:45:56.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.097+0000 7f1389942740 1 -- 172.21.15.164:0/2955674388 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556574fe0e20 con 0x556574f5c480 2024-09-19T08:45:56.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.097+0000 7f71b3fff640 1 -- 172.21.15.164:0/373542985 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f71c009cb60 con 0x7f71cc101350 2024-09-19T08:45:56.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.097+0000 7f1368ff9640 1 -- 172.21.15.164:0/2955674388 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f137004a070 con 0x556574f5c480 2024-09-19T08:45:56.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.097+0000 7f1368ff9640 1 -- 172.21.15.164:0/2955674388 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f137c0bc0b0 con 0x556574f5c480 2024-09-19T08:45:56.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.097+0000 7f1389942740 10 client.15351 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:45:56.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.097+0000 7f1389942740 10 client.15351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:45:56.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.097+0000 7f1389942740 20 client.15351 populate_metadata read hostname 'smithi164' 2024-09-19T08:45:56.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.097+0000 7f1389942740 1 --2- 172.21.15.164:0/2955674388 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556574fe1520 0x556574fe3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.097+0000 7f1389942740 4 client.15351 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T08:45:56.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.097+0000 7f1389942740 1 -- 172.21.15.164:0/2955674388 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556574e45110 con 0x556574fe1520 2024-09-19T08:45:56.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.098+0000 7f1385305640 1 --2- 172.21.15.164:0/2955674388 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556574fe1520 0x556574fe3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.098+0000 7f1368ff9640 10 client.15351 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:45:56.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.098+0000 7f1385305640 1 --2- 172.21.15.164:0/2955674388 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556574fe1520 0x556574fe3940 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=851e1556df9fc297 server_cookie=dfdceb00c220e768 in_seq=0 out_seq=0 2024-09-19T08:45:56.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.099+0000 7f1368ff9640 1 -- 172.21.15.164:0/2955674388 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556574e45110 con 0x556574fe1520 2024-09-19T08:45:56.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.099+0000 7f1368ff9640 10 client.15351 handle_command_reply: tid=0 2024-09-19T08:45:56.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.099+0000 7f1389942740 1 client.15351 shutdown 2024-09-19T08:45:56.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.099+0000 7f1389942740 1 -- 172.21.15.164:0/2955674388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556574f5c480 msgr2=0x556574f6f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.099+0000 7f1389942740 1 --2- 172.21.15.164:0/2955674388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556574f5c480 0x556574f6f400 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f137c007590 tx=0x7f137c00a630 comp rx=0 tx=0).stop 2024-09-19T08:45:56.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.099+0000 7f1389942740 1 -- 172.21.15.164:0/2955674388 shutdown_connections 2024-09-19T08:45:56.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.099+0000 7f1389942740 1 --2- 172.21.15.164:0/2955674388 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556574f66d40 0x556574f60500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.100+0000 7f1389942740 1 --2- 172.21.15.164:0/2955674388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556574f5c480 0x556574f6f400 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.100+0000 7f1389942740 1 --2- 172.21.15.164:0/2955674388 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556574fe1520 0x556574fe3940 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.100+0000 7f1389942740 1 --2- 172.21.15.164:0/2955674388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556574f5bb60 0x556574f6eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.100+0000 7f1389942740 1 -- 172.21.15.164:0/2955674388 >> 172.21.15.164:0/2955674388 conn(0x556574f580f0 msgr2=0x556574f5d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:56.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.100+0000 7f1368ff9640 0 client.15351 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:45:56.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.100+0000 7f1389942740 1 -- 172.21.15.164:0/2955674388 shutdown_connections 2024-09-19T08:45:56.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.100+0000 7f1389942740 1 -- 172.21.15.164:0/2955674388 wait complete. 2024-09-19T08:45:56.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.100+0000 7f1389942740 20 client.15351 trim_cache size 0 max 16384 2024-09-19T08:45:56.107 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:45:56.107 INFO:teuthology.orchestra.run.smithi164.stdout: "return_code": 0, 2024-09-19T08:45:56.107 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_tag": "1e77c761-d10c-42ee-bf2c-62f79ddff737", 2024-09-19T08:45:56.107 INFO:teuthology.orchestra.run.smithi164.stdout: "mode": "asynchronous" 2024-09-19T08:45:56.107 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:45:56.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.100+0000 7f1387569640 1 -- 172.21.15.164:0/3012013258 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f136c0a3440 msgr2=0x7f136c0a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.100+0000 7f1387569640 1 --2- 172.21.15.164:0/3012013258 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f136c0a3440 0x7f136c0a5900 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1378022000 tx=0x7f1378023000 comp rx=0 tx=0).stop 2024-09-19T08:45:56.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.100+0000 7f1387569640 1 -- 172.21.15.164:0/3012013258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1380101420 msgr2=0x7f138019aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.100+0000 7f1387569640 1 --2- 172.21.15.164:0/3012013258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1380101420 0x7f138019aaf0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f137c00c920 tx=0x7f137c00cde0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.100+0000 7f1387569640 1 -- 172.21.15.164:0/3012013258 shutdown_connections 2024-09-19T08:45:56.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.100+0000 7f1387569640 1 --2- 172.21.15.164:0/3012013258 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1380106a80 0x7f138019ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.100+0000 7f1387569640 1 --2- 172.21.15.164:0/3012013258 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f136c0a3440 0x7f136c0a5900 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.100+0000 7f1387569640 1 --2- 172.21.15.164:0/3012013258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1380101420 0x7f138019aaf0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.100+0000 7f1387569640 1 --2- 172.21.15.164:0/3012013258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1380100a80 0x7f138019a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.100+0000 7f1387569640 1 -- 172.21.15.164:0/3012013258 >> 172.21.15.164:0/3012013258 conn(0x7f13800fc7a0 msgr2=0x7f13800fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:56.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.101+0000 7f1387569640 1 -- 172.21.15.164:0/3012013258 shutdown_connections 2024-09-19T08:45:56.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.101+0000 7f1387569640 1 -- 172.21.15.164:0/3012013258 wait complete. 2024-09-19T08:45:56.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:55 smithi170 ceph-mon[35072]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 0 B/s wr, 26 op/s 2024-09-19T08:45:56.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:55 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2500027340' entity='client.admin' 2024-09-19T08:45:56.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:55 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:45:56.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:55 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:45:56.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:55 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:45:56.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:55 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:56.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:55 smithi173 ceph-mon[35306]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 0 B/s wr, 26 op/s 2024-09-19T08:45:56.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:55 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2500027340' entity='client.admin' 2024-09-19T08:45:56.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:55 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:45:56.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:55 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:45:56.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:55 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:45:56.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:55 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:56.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.269+0000 7f71d14c9640 1 -- 172.21.15.164:0/373542985 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f71cc0fe480 con 0x7f71cc101350 2024-09-19T08:45:56.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.273+0000 7f71b3fff640 1 -- 172.21.15.164:0/373542985 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v34)=0 v34) ==== 133+0+0 (secure 0 0 0) 0x7f71c009c460 con 0x7f71cc101350 2024-09-19T08:45:56.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.275+0000 7f71d14c9640 1 -- 172.21.15.164:0/373542985 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f71b40a3580 msgr2=0x7f71b40a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.275+0000 7f71d14c9640 1 --2- 172.21.15.164:0/373542985 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f71b40a3580 0x7f71b40a5a40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009b50 tx=0x7f71bc005f90 comp rx=0 tx=0).stop 2024-09-19T08:45:56.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.275+0000 7f71d14c9640 1 -- 172.21.15.164:0/373542985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71cc101350 msgr2=0x7f71cc19aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.275+0000 7f71d14c9640 1 --2- 172.21.15.164:0/373542985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71cc101350 0x7f71cc19aaa0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f71c000e9f0 tx=0x7f71c000edf0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.276+0000 7f71d14c9640 1 -- 172.21.15.164:0/373542985 shutdown_connections 2024-09-19T08:45:56.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.276+0000 7f71d14c9640 1 --2- 172.21.15.164:0/373542985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71cc1069c0 0x7f71cc19ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.276+0000 7f71d14c9640 1 --2- 172.21.15.164:0/373542985 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f71b40a3580 0x7f71b40a5a40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.276+0000 7f71d14c9640 1 --2- 172.21.15.164:0/373542985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71cc101350 0x7f71cc19aaa0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.276+0000 7f71d14c9640 1 --2- 172.21.15.164:0/373542985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71cc1009b0 0x7f71cc19a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.276+0000 7f71d14c9640 1 -- 172.21.15.164:0/373542985 >> 172.21.15.164:0/373542985 conn(0x7f71cc077990 msgr2=0x7f71cc1048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:56.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.276+0000 7f71d14c9640 1 -- 172.21.15.164:0/373542985 shutdown_connections 2024-09-19T08:45:56.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.276+0000 7f71d14c9640 1 -- 172.21.15.164:0/373542985 wait complete. 2024-09-19T08:45:56.289 DEBUG:teuthology.orchestra.run.smithi164:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-19T08:45:56.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.403+0000 7f8124142640 1 -- 172.21.15.164:0/713963535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f811c1009e0 msgr2=0x7f811c100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.403+0000 7f8124142640 1 --2- 172.21.15.164:0/713963535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f811c1009e0 0x7f811c100e40 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8110009930 tx=0x7f811002a420 comp rx=0 tx=0).stop 2024-09-19T08:45:56.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.403+0000 7f8124142640 1 -- 172.21.15.164:0/713963535 shutdown_connections 2024-09-19T08:45:56.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.403+0000 7f8124142640 1 --2- 172.21.15.164:0/713963535 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f811c101380 0x7f811c10d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.403+0000 7f8124142640 1 --2- 172.21.15.164:0/713963535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f811c1009e0 0x7f811c100e40 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.403+0000 7f8124142640 1 --2- 172.21.15.164:0/713963535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f811c1069f0 0x7f811c106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.403+0000 7f8124142640 1 -- 172.21.15.164:0/713963535 >> 172.21.15.164:0/713963535 conn(0x7f811c0fc6c0 msgr2=0x7f811c0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:56.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.403+0000 7f8124142640 1 -- 172.21.15.164:0/713963535 shutdown_connections 2024-09-19T08:45:56.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.403+0000 7f8124142640 1 -- 172.21.15.164:0/713963535 wait complete. 2024-09-19T08:45:56.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.403+0000 7f8124142640 1 Processor -- start 2024-09-19T08:45:56.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.403+0000 7f8124142640 1 -- start start 2024-09-19T08:45:56.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.404+0000 7f8124142640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f811c1009e0 0x7f811c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.404+0000 7f8124142640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f811c101380 0x7f811c198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.404+0000 7f8124142640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f811c1069f0 0x7f811c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.404+0000 7f8124142640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f811c110e80 con 0x7f811c1069f0 2024-09-19T08:45:56.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.404+0000 7f8124142640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f811c110d00 con 0x7f811c101380 2024-09-19T08:45:56.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.404+0000 7f8124142640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f811c111000 con 0x7f811c1009e0 2024-09-19T08:45:56.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.404+0000 7f81226df640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f811c1069f0 0x7f811c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.404+0000 7f81226df640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f811c1069f0 0x7f811c19cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39306/0 (socket says 172.21.15.164:39306) 2024-09-19T08:45:56.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.404+0000 7f81226df640 1 -- 172.21.15.164:0/2980641248 learned_addr learned my addr 172.21.15.164:0/2980641248 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:56.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.404+0000 7f8121ede640 1 --2- 172.21.15.164:0/2980641248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f811c1009e0 0x7f811c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.404+0000 7f81216dd640 1 --2- 172.21.15.164:0/2980641248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f811c101380 0x7f811c198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.404+0000 7f81226df640 1 -- 172.21.15.164:0/2980641248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f811c1009e0 msgr2=0x7f811c198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.404+0000 7f81226df640 1 --2- 172.21.15.164:0/2980641248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f811c1009e0 0x7f811c198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.404+0000 7f81226df640 1 -- 172.21.15.164:0/2980641248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f811c101380 msgr2=0x7f811c198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.404+0000 7f81226df640 1 --2- 172.21.15.164:0/2980641248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f811c101380 0x7f811c198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.404+0000 7f81226df640 1 -- 172.21.15.164:0/2980641248 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f811c19d330 con 0x7f811c1069f0 2024-09-19T08:45:56.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.404+0000 7f81226df640 1 --2- 172.21.15.164:0/2980641248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f811c1069f0 0x7f811c19cc00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f810c00eaa0 tx=0x7f810c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:56.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.404+0000 7f811affd640 1 -- 172.21.15.164:0/2980641248 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c00cdb0 con 0x7f811c1069f0 2024-09-19T08:45:56.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.404+0000 7f811affd640 1 -- 172.21.15.164:0/2980641248 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f810c005020 con 0x7f811c1069f0 2024-09-19T08:45:56.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.404+0000 7f811affd640 1 -- 172.21.15.164:0/2980641248 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c0116b0 con 0x7f811c1069f0 2024-09-19T08:45:56.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.404+0000 7f8124142640 1 -- 172.21.15.164:0/2980641248 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f811c19d620 con 0x7f811c1069f0 2024-09-19T08:45:56.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.404+0000 7f8124142640 1 -- 172.21.15.164:0/2980641248 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f811c1a4eb0 con 0x7f811c1069f0 2024-09-19T08:45:56.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.405+0000 7f8124142640 1 -- 172.21.15.164:0/2980641248 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f811c100e40 con 0x7f811c1069f0 2024-09-19T08:45:56.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.407+0000 7f811affd640 1 -- 172.21.15.164:0/2980641248 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f810c005560 con 0x7f811c1069f0 2024-09-19T08:45:56.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.407+0000 7f811affd640 1 --2- 172.21.15.164:0/2980641248 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f81040a3580 0x7f81040a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.407+0000 7f8121ede640 1 --2- 172.21.15.164:0/2980641248 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f81040a3580 0x7f81040a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.407+0000 7f811affd640 1 -- 172.21.15.164:0/2980641248 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f810c0d0000 con 0x7f811c1069f0 2024-09-19T08:45:56.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.408+0000 7f8121ede640 1 --2- 172.21.15.164:0/2980641248 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f81040a3580 0x7f81040a5a40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f81140058c0 tx=0x7f811400a2d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:56.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.412+0000 7f811affd640 1 -- 172.21.15.164:0/2980641248 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f810c016030 con 0x7f811c1069f0 2024-09-19T08:45:56.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.600+0000 7f8124142640 1 -- 172.21.15.164:0/2980641248 --> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f811c062630 con 0x7f81040a3580 2024-09-19T08:45:56.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.665+0000 7f811affd640 1 -- 172.21.15.164:0/2980641248 <== mgr.15201 v2:172.21.15.164:6800/173486514 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f811c062630 con 0x7f81040a3580 2024-09-19T08:45:56.665 INFO:teuthology.orchestra.run.smithi164.stdout:Schedule set for path / 2024-09-19T08:45:56.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.667+0000 7f8124142640 1 -- 172.21.15.164:0/2980641248 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f81040a3580 msgr2=0x7f81040a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.667+0000 7f8124142640 1 --2- 172.21.15.164:0/2980641248 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f81040a3580 0x7f81040a5a40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f81140058c0 tx=0x7f811400a2d0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.667+0000 7f8124142640 1 -- 172.21.15.164:0/2980641248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f811c1069f0 msgr2=0x7f811c19cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.667+0000 7f8124142640 1 --2- 172.21.15.164:0/2980641248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f811c1069f0 0x7f811c19cc00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f810c00eaa0 tx=0x7f810c00ef60 comp rx=0 tx=0).stop 2024-09-19T08:45:56.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.667+0000 7f8124142640 1 -- 172.21.15.164:0/2980641248 shutdown_connections 2024-09-19T08:45:56.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.667+0000 7f8124142640 1 --2- 172.21.15.164:0/2980641248 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f81040a3580 0x7f81040a5a40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.667+0000 7f8124142640 1 --2- 172.21.15.164:0/2980641248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f811c1069f0 0x7f811c19cc00 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.667+0000 7f8124142640 1 --2- 172.21.15.164:0/2980641248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f811c101380 0x7f811c198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.667+0000 7f8124142640 1 --2- 172.21.15.164:0/2980641248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f811c1009e0 0x7f811c198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.667+0000 7f8124142640 1 -- 172.21.15.164:0/2980641248 >> 172.21.15.164:0/2980641248 conn(0x7f811c0fc6c0 msgr2=0x7f811c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:56.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.667+0000 7f8124142640 1 -- 172.21.15.164:0/2980641248 shutdown_connections 2024-09-19T08:45:56.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.667+0000 7f8124142640 1 -- 172.21.15.164:0/2980641248 wait complete. 2024-09-19T08:45:56.679 DEBUG:teuthology.orchestra.run.smithi164:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-19T08:45:56.781 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:45:56.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.787+0000 7f385f25e640 1 -- 172.21.15.164:0/3057001253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3858107b70 msgr2=0x7f3858107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.787+0000 7f385f25e640 1 --2- 172.21.15.164:0/3057001253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3858107b70 0x7f3858107f50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f384c009930 tx=0x7f384c02a420 comp rx=0 tx=0).stop 2024-09-19T08:45:56.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.787+0000 7f385f25e640 1 -- 172.21.15.164:0/3057001253 shutdown_connections 2024-09-19T08:45:56.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.787+0000 7f385f25e640 1 --2- 172.21.15.164:0/3057001253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3858102470 0x7f385810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.787+0000 7f385f25e640 1 --2- 172.21.15.164:0/3057001253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3858101ad0 0x7f3858101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.787+0000 7f385f25e640 1 --2- 172.21.15.164:0/3057001253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3858107b70 0x7f3858107f50 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.787+0000 7f385f25e640 1 -- 172.21.15.164:0/3057001253 >> 172.21.15.164:0/3057001253 conn(0x7f38580fd7d0 msgr2=0x7f38580ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:56.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.787+0000 7f385f25e640 1 -- 172.21.15.164:0/3057001253 shutdown_connections 2024-09-19T08:45:56.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.787+0000 7f385f25e640 1 -- 172.21.15.164:0/3057001253 wait complete. 2024-09-19T08:45:56.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.787+0000 7f385f25e640 1 Processor -- start 2024-09-19T08:45:56.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.787+0000 7f385f25e640 1 -- start start 2024-09-19T08:45:56.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.787+0000 7f385f25e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3858101ad0 0x7f38580744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.787+0000 7f385f25e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3858102470 0x7f3858074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.788+0000 7f385f25e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3858107b70 0x7f3858074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.788+0000 7f385f25e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3858111f20 con 0x7f3858101ad0 2024-09-19T08:45:56.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.788+0000 7f385f25e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3858111da0 con 0x7f3858107b70 2024-09-19T08:45:56.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.788+0000 7f385f25e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f38581120a0 con 0x7f3858102470 2024-09-19T08:45:56.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.788+0000 7f385cffa640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3858101ad0 0x7f38580744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.788+0000 7f385cffa640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3858101ad0 0x7f38580744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39320/0 (socket says 172.21.15.164:39320) 2024-09-19T08:45:56.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.788+0000 7f385cffa640 1 -- 172.21.15.164:0/456285827 learned_addr learned my addr 172.21.15.164:0/456285827 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:56.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.788+0000 7f385d7fb640 1 --2- 172.21.15.164:0/456285827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3858107b70 0x7f3858074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.788+0000 7f385cffa640 1 -- 172.21.15.164:0/456285827 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3858102470 msgr2=0x7f3858074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.788+0000 7f3857fff640 1 --2- 172.21.15.164:0/456285827 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3858102470 0x7f3858074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.788+0000 7f385cffa640 1 --2- 172.21.15.164:0/456285827 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3858102470 0x7f3858074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.788+0000 7f385cffa640 1 -- 172.21.15.164:0/456285827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3858107b70 msgr2=0x7f3858074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.788+0000 7f385cffa640 1 --2- 172.21.15.164:0/456285827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3858107b70 0x7f3858074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.788+0000 7f385cffa640 1 -- 172.21.15.164:0/456285827 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38581a6850 con 0x7f3858101ad0 2024-09-19T08:45:56.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.788+0000 7f385d7fb640 1 --2- 172.21.15.164:0/456285827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3858107b70 0x7f3858074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:56.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.788+0000 7f385cffa640 1 --2- 172.21.15.164:0/456285827 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3858101ad0 0x7f38580744f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f384c009230 tx=0x7f384c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:56.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.788+0000 7f3855ffb640 1 -- 172.21.15.164:0/456285827 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c03f030 con 0x7f3858101ad0 2024-09-19T08:45:56.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.788+0000 7f385f25e640 1 -- 172.21.15.164:0/456285827 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38581a6ae0 con 0x7f3858101ad0 2024-09-19T08:45:56.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.788+0000 7f3855ffb640 1 -- 172.21.15.164:0/456285827 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f384c003ad0 con 0x7f3858101ad0 2024-09-19T08:45:56.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.788+0000 7f3855ffb640 1 -- 172.21.15.164:0/456285827 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c035070 con 0x7f3858101ad0 2024-09-19T08:45:56.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.789+0000 7f385f25e640 1 -- 172.21.15.164:0/456285827 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38581a7020 con 0x7f3858101ad0 2024-09-19T08:45:56.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.790+0000 7f385f25e640 1 -- 172.21.15.164:0/456285827 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3858101f30 con 0x7f3858101ad0 2024-09-19T08:45:56.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.791+0000 7f3855ffb640 1 -- 172.21.15.164:0/456285827 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f384c042030 con 0x7f3858101ad0 2024-09-19T08:45:56.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.792+0000 7f3855ffb640 1 --2- 172.21.15.164:0/456285827 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f38480a3500 0x7f38480a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.792+0000 7f3855ffb640 1 -- 172.21.15.164:0/456285827 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f384c0f0450 con 0x7f3858101ad0 2024-09-19T08:45:56.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.792+0000 7f3857fff640 1 --2- 172.21.15.164:0/456285827 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f38480a3500 0x7f38480a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.792+0000 7f3857fff640 1 --2- 172.21.15.164:0/456285827 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f38480a3500 0x7f38480a59c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3844005fd0 tx=0x7f3844005850 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:56.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.797+0000 7f3855ffb640 1 -- 172.21.15.164:0/456285827 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f384c0bcd30 con 0x7f3858101ad0 2024-09-19T08:45:56.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:56.998+0000 7f385f25e640 1 -- 172.21.15.164:0/456285827 --> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f38580ff710 con 0x7f38480a3500 2024-09-19T08:45:57.039 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:45:57.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.059+0000 7f3855ffb640 1 -- 172.21.15.164:0/456285827 <== mgr.15201 v2:172.21.15.164:6800/173486514 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f38580ff710 con 0x7f38480a3500 2024-09-19T08:45:57.060 INFO:teuthology.orchestra.run.smithi164.stdout:Retention added to path / 2024-09-19T08:45:57.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.061+0000 7f385f25e640 1 -- 172.21.15.164:0/456285827 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f38480a3500 msgr2=0x7f38480a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:57.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.061+0000 7f385f25e640 1 --2- 172.21.15.164:0/456285827 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f38480a3500 0x7f38480a59c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3844005fd0 tx=0x7f3844005850 comp rx=0 tx=0).stop 2024-09-19T08:45:57.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.061+0000 7f385f25e640 1 -- 172.21.15.164:0/456285827 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3858101ad0 msgr2=0x7f38580744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:57.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.061+0000 7f385f25e640 1 --2- 172.21.15.164:0/456285827 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3858101ad0 0x7f38580744f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f384c009230 tx=0x7f384c004270 comp rx=0 tx=0).stop 2024-09-19T08:45:57.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.061+0000 7f385f25e640 1 -- 172.21.15.164:0/456285827 shutdown_connections 2024-09-19T08:45:57.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.061+0000 7f385f25e640 1 --2- 172.21.15.164:0/456285827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3858107b70 0x7f3858074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.061+0000 7f385f25e640 1 --2- 172.21.15.164:0/456285827 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3858102470 0x7f3858074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.061+0000 7f385f25e640 1 --2- 172.21.15.164:0/456285827 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f38480a3500 0x7f38480a59c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.061+0000 7f385f25e640 1 --2- 172.21.15.164:0/456285827 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3858101ad0 0x7f38580744f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.061+0000 7f385f25e640 1 -- 172.21.15.164:0/456285827 >> 172.21.15.164:0/456285827 conn(0x7f38580fd7d0 msgr2=0x7f38580fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:57.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.061+0000 7f385f25e640 1 -- 172.21.15.164:0/456285827 shutdown_connections 2024-09-19T08:45:57.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.061+0000 7f385f25e640 1 -- 172.21.15.164:0/456285827 wait complete. 2024-09-19T08:45:57.073 DEBUG:teuthology.orchestra.run.smithi164:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-19T08:45:57.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.217+0000 7f53c4e35640 1 -- 172.21.15.164:0/81998685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53c0105af0 msgr2=0x7f53c0105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:57.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.217+0000 7f53c4e35640 1 --2- 172.21.15.164:0/81998685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53c0105af0 0x7f53c0105ed0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f53b0009900 tx=0x7f53b002a480 comp rx=0 tx=0).stop 2024-09-19T08:45:57.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.217+0000 7f53c4e35640 1 -- 172.21.15.164:0/81998685 shutdown_connections 2024-09-19T08:45:57.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.217+0000 7f53c4e35640 1 --2- 172.21.15.164:0/81998685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53c0102050 0x7f53c010d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.217+0000 7f53c4e35640 1 --2- 172.21.15.164:0/81998685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53c01016d0 0x7f53c0101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.217+0000 7f53c4e35640 1 --2- 172.21.15.164:0/81998685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53c0105af0 0x7f53c0105ed0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.217+0000 7f53c4e35640 1 -- 172.21.15.164:0/81998685 >> 172.21.15.164:0/81998685 conn(0x7f53c0075ec0 msgr2=0x7f53c00762d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:57.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.217+0000 7f53c4e35640 1 -- 172.21.15.164:0/81998685 shutdown_connections 2024-09-19T08:45:57.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.217+0000 7f53c4e35640 1 -- 172.21.15.164:0/81998685 wait complete. 2024-09-19T08:45:57.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.218+0000 7f53c4e35640 1 Processor -- start 2024-09-19T08:45:57.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.218+0000 7f53c4e35640 1 -- start start 2024-09-19T08:45:57.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.218+0000 7f53c4e35640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53c01016d0 0x7f53c0197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:57.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.218+0000 7f53c4e35640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53c0102050 0x7f53c0197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:57.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.218+0000 7f53c4e35640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53c0105af0 0x7f53c019bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:57.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.218+0000 7f53c4e35640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f53c0110e30 con 0x7f53c01016d0 2024-09-19T08:45:57.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.218+0000 7f53c4e35640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f53c0110cb0 con 0x7f53c0102050 2024-09-19T08:45:57.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.218+0000 7f53c4e35640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f53c0110fb0 con 0x7f53c0105af0 2024-09-19T08:45:57.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.218+0000 7f53be59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53c01016d0 0x7f53c0197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:57.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.218+0000 7f53be59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53c01016d0 0x7f53c0197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39342/0 (socket says 172.21.15.164:39342) 2024-09-19T08:45:57.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.218+0000 7f53be59c640 1 -- 172.21.15.164:0/521711437 learned_addr learned my addr 172.21.15.164:0/521711437 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:57.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.219+0000 7f53bed9d640 1 --2- 172.21.15.164:0/521711437 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53c0105af0 0x7f53c019bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:57.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.219+0000 7f53bdd9b640 1 --2- 172.21.15.164:0/521711437 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53c0102050 0x7f53c0197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:57.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.219+0000 7f53be59c640 1 -- 172.21.15.164:0/521711437 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53c0105af0 msgr2=0x7f53c019bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:57.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.219+0000 7f53be59c640 1 --2- 172.21.15.164:0/521711437 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53c0105af0 0x7f53c019bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.219+0000 7f53be59c640 1 -- 172.21.15.164:0/521711437 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53c0102050 msgr2=0x7f53c0197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:57.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.219+0000 7f53be59c640 1 --2- 172.21.15.164:0/521711437 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53c0102050 0x7f53c0197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.219+0000 7f53be59c640 1 -- 172.21.15.164:0/521711437 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c019c210 con 0x7f53c01016d0 2024-09-19T08:45:57.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.219+0000 7f53be59c640 1 --2- 172.21.15.164:0/521711437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53c01016d0 0x7f53c0197220 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f53b0004770 tx=0x7f53b0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:57.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.219+0000 7f53af7fe640 1 -- 172.21.15.164:0/521711437 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b003f070 con 0x7f53c01016d0 2024-09-19T08:45:57.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.219+0000 7f53af7fe640 1 -- 172.21.15.164:0/521711437 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53b0031ca0 con 0x7f53c01016d0 2024-09-19T08:45:57.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.219+0000 7f53af7fe640 1 -- 172.21.15.164:0/521711437 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b0035030 con 0x7f53c01016d0 2024-09-19T08:45:57.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.219+0000 7f53c4e35640 1 -- 172.21.15.164:0/521711437 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53c019c4a0 con 0x7f53c01016d0 2024-09-19T08:45:57.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.219+0000 7f53c4e35640 1 -- 172.21.15.164:0/521711437 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53c01a3d90 con 0x7f53c01016d0 2024-09-19T08:45:57.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.220+0000 7f53c4e35640 1 -- 172.21.15.164:0/521711437 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53c0101b10 con 0x7f53c01016d0 2024-09-19T08:45:57.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.221+0000 7f53af7fe640 1 -- 172.21.15.164:0/521711437 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f53b0042070 con 0x7f53c01016d0 2024-09-19T08:45:57.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.222+0000 7f53af7fe640 1 --2- 172.21.15.164:0/521711437 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f53a80a3500 0x7f53a80a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:57.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.222+0000 7f53bdd9b640 1 --2- 172.21.15.164:0/521711437 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f53a80a3500 0x7f53a80a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:57.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.222+0000 7f53af7fe640 1 -- 172.21.15.164:0/521711437 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f53b00f0410 con 0x7f53c01016d0 2024-09-19T08:45:57.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.222+0000 7f53bdd9b640 1 --2- 172.21.15.164:0/521711437 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f53a80a3500 0x7f53a80a59c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f53c0076430 tx=0x7f53b4005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:57.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.226+0000 7f53af7fe640 1 -- 172.21.15.164:0/521711437 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f53b00bc760 con 0x7f53c01016d0 2024-09-19T08:45:57.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:56 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/157362804' entity='client.admin' 2024-09-19T08:45:57.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:56 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:45:57.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:56 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:45:57.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:56 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:45:57.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:56 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:57.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:56 smithi164 ceph-mon[30928]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 0 B/s wr, 24 op/s 2024-09-19T08:45:57.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:56 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/373542985' entity='client.admin' 2024-09-19T08:45:57.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:56 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:45:57.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:56 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:45:57.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:56 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:45:57.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:56 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:57.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:56 smithi164 ceph-mon[30928]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:45:57.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:56 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/157362804' entity='client.admin' 2024-09-19T08:45:57.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:56 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:45:57.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:56 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:45:57.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:56 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:45:57.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:56 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:57.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:56 smithi170 ceph-mon[35072]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 0 B/s wr, 24 op/s 2024-09-19T08:45:57.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:56 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/373542985' entity='client.admin' 2024-09-19T08:45:57.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:56 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:45:57.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:56 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:45:57.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:56 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:45:57.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:56 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:57.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:56 smithi170 ceph-mon[35072]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:45:57.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:56 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/157362804' entity='client.admin' 2024-09-19T08:45:57.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:56 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:45:57.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:56 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:45:57.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:56 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:45:57.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:56 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:57.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:56 smithi173 ceph-mon[35306]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 0 B/s wr, 24 op/s 2024-09-19T08:45:57.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:56 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/373542985' entity='client.admin' 2024-09-19T08:45:57.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:56 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:45:57.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:56 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:45:57.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:56 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:45:57.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:56 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:45:57.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:56 smithi173 ceph-mon[35306]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:45:57.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.433+0000 7f53c4e35640 1 -- 172.21.15.164:0/521711437 --> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f53c0062630 con 0x7f53a80a3500 2024-09-19T08:45:57.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.439+0000 7f53af7fe640 1 -- 172.21.15.164:0/521711437 <== mgr.15201 v2:172.21.15.164:6800/173486514 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f53c0062630 con 0x7f53a80a3500 2024-09-19T08:45:57.440 INFO:teuthology.orchestra.run.smithi164.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-19T00:00:00", "created": "2024-09-19T08:45:56", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-19T08:45:57.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.441+0000 7f53c4e35640 1 -- 172.21.15.164:0/521711437 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f53a80a3500 msgr2=0x7f53a80a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:57.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.441+0000 7f53c4e35640 1 --2- 172.21.15.164:0/521711437 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f53a80a3500 0x7f53a80a59c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f53c0076430 tx=0x7f53b4005f90 comp rx=0 tx=0).stop 2024-09-19T08:45:57.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.441+0000 7f53c4e35640 1 -- 172.21.15.164:0/521711437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53c01016d0 msgr2=0x7f53c0197220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:57.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.441+0000 7f53c4e35640 1 --2- 172.21.15.164:0/521711437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53c01016d0 0x7f53c0197220 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f53b0004770 tx=0x7f53b0003c10 comp rx=0 tx=0).stop 2024-09-19T08:45:57.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.442+0000 7f53c4e35640 1 -- 172.21.15.164:0/521711437 shutdown_connections 2024-09-19T08:45:57.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.442+0000 7f53c4e35640 1 --2- 172.21.15.164:0/521711437 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53c0105af0 0x7f53c019bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.442+0000 7f53c4e35640 1 --2- 172.21.15.164:0/521711437 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53c0102050 0x7f53c0197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.442+0000 7f53c4e35640 1 --2- 172.21.15.164:0/521711437 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f53a80a3500 0x7f53a80a59c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.442+0000 7f53c4e35640 1 --2- 172.21.15.164:0/521711437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53c01016d0 0x7f53c0197220 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.442+0000 7f53c4e35640 1 -- 172.21.15.164:0/521711437 >> 172.21.15.164:0/521711437 conn(0x7f53c0075ec0 msgr2=0x7f53c00ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:57.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.442+0000 7f53c4e35640 1 -- 172.21.15.164:0/521711437 shutdown_connections 2024-09-19T08:45:57.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.442+0000 7f53c4e35640 1 -- 172.21.15.164:0/521711437 wait complete. 2024-09-19T08:45:57.454 DEBUG:teuthology.orchestra.run.smithi164:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-19T08:45:57.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.529+0000 7fd45fd02640 1 -- 172.21.15.164:0/1710559950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd458102570 msgr2=0x7fd4581029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:57.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.529+0000 7fd45fd02640 1 --2- 172.21.15.164:0/1710559950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd458102570 0x7fd4581029b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd44c009900 tx=0x7fd44c02a480 comp rx=0 tx=0).stop 2024-09-19T08:45:57.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.529+0000 7fd45fd02640 1 -- 172.21.15.164:0/1710559950 shutdown_connections 2024-09-19T08:45:57.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.529+0000 7fd45fd02640 1 --2- 172.21.15.164:0/1710559950 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd458102ef0 0x7fd45810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.529+0000 7fd45fd02640 1 --2- 172.21.15.164:0/1710559950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd458102570 0x7fd4581029b0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.529+0000 7fd45fd02640 1 --2- 172.21.15.164:0/1710559950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd458100e20 0x7fd458101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.529+0000 7fd45fd02640 1 -- 172.21.15.164:0/1710559950 >> 172.21.15.164:0/1710559950 conn(0x7fd4580fc7d0 msgr2=0x7fd4580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:57.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.529+0000 7fd45fd02640 1 -- 172.21.15.164:0/1710559950 shutdown_connections 2024-09-19T08:45:57.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.529+0000 7fd45fd02640 1 -- 172.21.15.164:0/1710559950 wait complete. 2024-09-19T08:45:57.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.530+0000 7fd45fd02640 1 Processor -- start 2024-09-19T08:45:57.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.530+0000 7fd45fd02640 1 -- start start 2024-09-19T08:45:57.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.530+0000 7fd45fd02640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd458100e20 0x7fd4581991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:57.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.530+0000 7fd45fd02640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd458102570 0x7fd458199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:57.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.530+0000 7fd45fd02640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd458102ef0 0x7fd4581a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:57.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.530+0000 7fd45fd02640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd458110f50 con 0x7fd458102ef0 2024-09-19T08:45:57.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.530+0000 7fd45fd02640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd458110dd0 con 0x7fd458100e20 2024-09-19T08:45:57.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.530+0000 7fd45fd02640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd4581110d0 con 0x7fd458102570 2024-09-19T08:45:57.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.530+0000 7fd45e29f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd458102ef0 0x7fd4581a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:57.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.530+0000 7fd45e29f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd458102ef0 0x7fd4581a4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39354/0 (socket says 172.21.15.164:39354) 2024-09-19T08:45:57.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.530+0000 7fd45e29f640 1 -- 172.21.15.164:0/1283372116 learned_addr learned my addr 172.21.15.164:0/1283372116 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:57.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.530+0000 7fd45da9e640 1 --2- 172.21.15.164:0/1283372116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd458100e20 0x7fd4581991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:57.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.530+0000 7fd45d29d640 1 --2- 172.21.15.164:0/1283372116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd458102570 0x7fd458199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:57.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.531+0000 7fd45e29f640 1 -- 172.21.15.164:0/1283372116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd458102570 msgr2=0x7fd458199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:57.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.531+0000 7fd45e29f640 1 --2- 172.21.15.164:0/1283372116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd458102570 0x7fd458199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.531+0000 7fd45e29f640 1 -- 172.21.15.164:0/1283372116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd458100e20 msgr2=0x7fd4581991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:57.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.531+0000 7fd45e29f640 1 --2- 172.21.15.164:0/1283372116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd458100e20 0x7fd4581991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.531+0000 7fd45e29f640 1 -- 172.21.15.164:0/1283372116 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4581a47b0 con 0x7fd458102ef0 2024-09-19T08:45:57.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.531+0000 7fd45da9e640 1 --2- 172.21.15.164:0/1283372116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd458100e20 0x7fd4581991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:45:57.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.531+0000 7fd45e29f640 1 --2- 172.21.15.164:0/1283372116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd458102ef0 0x7fd4581a4270 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd44800c970 tx=0x7fd44800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:57.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.531+0000 7fd456ffd640 1 -- 172.21.15.164:0/1283372116 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd448007bb0 con 0x7fd458102ef0 2024-09-19T08:45:57.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.531+0000 7fd45fd02640 1 -- 172.21.15.164:0/1283372116 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4581a4980 con 0x7fd458102ef0 2024-09-19T08:45:57.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.531+0000 7fd456ffd640 1 -- 172.21.15.164:0/1283372116 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd448007d50 con 0x7fd458102ef0 2024-09-19T08:45:57.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.531+0000 7fd456ffd640 1 -- 172.21.15.164:0/1283372116 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd448018800 con 0x7fd458102ef0 2024-09-19T08:45:57.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.531+0000 7fd45fd02640 1 -- 172.21.15.164:0/1283372116 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4581a4e90 con 0x7fd458102ef0 2024-09-19T08:45:57.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.534+0000 7fd456ffd640 1 -- 172.21.15.164:0/1283372116 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7fd44800fa50 con 0x7fd458102ef0 2024-09-19T08:45:57.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.534+0000 7fd4620db740 1 Processor -- start 2024-09-19T08:45:57.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.534+0000 7fd4620db740 1 -- start start 2024-09-19T08:45:57.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.534+0000 7fd4620db740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b30edb4a0 0x556b30edb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:57.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.534+0000 7fd4620db740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b30edbdc0 0x556b30ee6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:57.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.534+0000 7fd4620db740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b30ee6680 0x556b30ee8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:57.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.534+0000 7fd4620db740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556b30e412e0 con 0x556b30edb4a0 2024-09-19T08:45:57.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.534+0000 7fd4620db740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556b30de8ce0 con 0x556b30edbdc0 2024-09-19T08:45:57.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.534+0000 7fd4620db740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556b30e3d2a0 con 0x556b30ee6680 2024-09-19T08:45:57.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.534+0000 7fd45d29d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b30edb4a0 0x556b30edb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:57.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.534+0000 7fd45d29d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b30edb4a0 0x556b30edb880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39362/0 (socket says 172.21.15.164:39362) 2024-09-19T08:45:57.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.534+0000 7fd45d29d640 1 -- 172.21.15.164:0/2487829428 learned_addr learned my addr 172.21.15.164:0/2487829428 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:57.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.534+0000 7fd45da9e640 1 --2- 172.21.15.164:0/2487829428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b30edbdc0 0x556b30ee6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:57.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.534+0000 7fd45d29d640 1 --2- 172.21.15.164:0/2487829428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b30ee6680 0x556b30ee8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:57.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.535+0000 7fd456ffd640 1 --2- 172.21.15.164:0/1283372116 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd4400a3440 0x7fd4400a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:57.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.535+0000 7fd45d29d640 1 -- 172.21.15.164:0/2487829428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b30ee6680 msgr2=0x556b30ee8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:57.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.535+0000 7fd45d29d640 1 --2- 172.21.15.164:0/2487829428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b30ee6680 0x556b30ee8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.535+0000 7fd45d29d640 1 -- 172.21.15.164:0/2487829428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b30edbdc0 msgr2=0x556b30ee6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:57.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.535+0000 7fd45d29d640 1 --2- 172.21.15.164:0/2487829428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b30edbdc0 0x556b30ee6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.535+0000 7fd45d29d640 1 -- 172.21.15.164:0/2487829428 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b30dcd440 con 0x556b30edb4a0 2024-09-19T08:45:57.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.535+0000 7fd45e29f640 1 --2- 172.21.15.164:0/1283372116 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd4400a3440 0x7fd4400a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:57.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.535+0000 7fd45d29d640 1 --2- 172.21.15.164:0/2487829428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b30edb4a0 0x556b30edb880 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd44c02a9a0 tx=0x7fd44c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26fad70aa08d33d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:57.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.535+0000 7fd456ffd640 1 -- 172.21.15.164:0/1283372116 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fd4480cfda0 con 0x7fd458102ef0 2024-09-19T08:45:57.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.535+0000 7fd43ffff640 1 -- 172.21.15.164:0/2487829428 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c035070 con 0x556b30edb4a0 2024-09-19T08:45:57.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.535+0000 7fd43ffff640 1 -- 172.21.15.164:0/2487829428 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd44c003d50 con 0x556b30edb4a0 2024-09-19T08:45:57.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.535+0000 7fd43ffff640 1 -- 172.21.15.164:0/2487829428 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c03b460 con 0x556b30edb4a0 2024-09-19T08:45:57.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.536+0000 7fd4620db740 1 -- 172.21.15.164:0/2487829428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b30edb4a0 msgr2=0x556b30edb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:57.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.536+0000 7fd4620db740 1 --2- 172.21.15.164:0/2487829428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b30edb4a0 0x556b30edb880 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd44c02a9a0 tx=0x7fd44c005c00 comp rx=0 tx=0).stop 2024-09-19T08:45:57.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.536+0000 7fd45e29f640 1 --2- 172.21.15.164:0/1283372116 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd4400a3440 0x7fd4400a5900 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd4480cd500 tx=0x7fd44806edb0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:57.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.536+0000 7fd4620db740 1 -- 172.21.15.164:0/2487829428 shutdown_connections 2024-09-19T08:45:57.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.536+0000 7fd4620db740 1 --2- 172.21.15.164:0/2487829428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b30ee6680 0x556b30ee8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.536+0000 7fd4620db740 1 --2- 172.21.15.164:0/2487829428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b30edbdc0 0x556b30ee6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.536+0000 7fd4620db740 1 --2- 172.21.15.164:0/2487829428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b30edb4a0 0x556b30edb880 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.536+0000 7fd4620db740 1 -- 172.21.15.164:0/2487829428 >> 172.21.15.164:0/2487829428 conn(0x556b30ed7a30 msgr2=0x556b30ed7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:57.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.537+0000 7fd4620db740 1 -- 172.21.15.164:0/2487829428 shutdown_connections 2024-09-19T08:45:57.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.537+0000 7fd4620db740 1 -- 172.21.15.164:0/2487829428 wait complete. 2024-09-19T08:45:57.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.537+0000 7fd4620db740 1 -- start start 2024-09-19T08:45:57.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.537+0000 7fd4620db740 1 Processor -- start 2024-09-19T08:45:57.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.537+0000 7fd4620db740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b30edb4a0 0x556b30ee5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:57.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.537+0000 7fd4620db740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b30edbdc0 0x556b30ee31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:57.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.537+0000 7fd4620db740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b30ee6680 0x556b30ee3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:57.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.537+0000 7fd4620db740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556b30eed660 con 0x556b30ee6680 2024-09-19T08:45:57.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.537+0000 7fd4620db740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556b30dfc050 con 0x556b30edb4a0 2024-09-19T08:45:57.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.537+0000 7fd4620db740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556b30eed7e0 con 0x556b30edbdc0 2024-09-19T08:45:57.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.538+0000 7fd4620db740 10 client.0 _ll_register_callbacks cb 0x556b30e3c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:45:57.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.538+0000 7fd45d29d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b30ee6680 0x556b30ee3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:57.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.538+0000 7fd45d29d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b30ee6680 0x556b30ee3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39366/0 (socket says 172.21.15.164:39366) 2024-09-19T08:45:57.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.538+0000 7fd45d29d640 1 -- 172.21.15.164:0/2525752991 learned_addr learned my addr 172.21.15.164:0/2525752991 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:57.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.538+0000 7fd45da9e640 1 --2- 172.21.15.164:0/2525752991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b30edbdc0 0x556b30ee31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:57.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.538+0000 7fd45d29d640 1 --2- 172.21.15.164:0/2525752991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b30edb4a0 0x556b30ee5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:57.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.538+0000 7fd45da9e640 1 -- 172.21.15.164:0/2525752991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b30edb4a0 msgr2=0x556b30ee5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:57.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.538+0000 7fd45da9e640 1 --2- 172.21.15.164:0/2525752991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b30edb4a0 0x556b30ee5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.538+0000 7fd45da9e640 1 -- 172.21.15.164:0/2525752991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b30ee6680 msgr2=0x556b30ee3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:57.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.538+0000 7fd45da9e640 1 --2- 172.21.15.164:0/2525752991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b30ee6680 0x556b30ee3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.538+0000 7fd45da9e640 1 -- 172.21.15.164:0/2525752991 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b30bb23c0 con 0x556b30edbdc0 2024-09-19T08:45:57.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.538+0000 7fd45d29d640 1 --2- 172.21.15.164:0/2525752991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b30ee6680 0x556b30ee3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:45:57.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.538+0000 7fd45da9e640 1 --2- 172.21.15.164:0/2525752991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b30edbdc0 0x556b30ee31e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd450011ef0 tx=0x7fd45000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db54f2a35ebb3f3b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:57.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.538+0000 7fd43d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-09-19T08:45:57.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.538+0000 7fd43d7fa640 1 -- 172.21.15.164:0/2525752991 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd450004d20 con 0x556b30edbdc0 2024-09-19T08:45:57.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.538+0000 7fd4620db740 1 -- 172.21.15.164:0/2525752991 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd44c003d50 con 0x556b30edbdc0 2024-09-19T08:45:57.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.538+0000 7fd43d7fa640 1 -- 172.21.15.164:0/2525752991 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fd45000d550 con 0x556b30edbdc0 2024-09-19T08:45:57.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.539+0000 7fd43d7fa640 1 -- 172.21.15.164:0/2525752991 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd45000d750 con 0x556b30edbdc0 2024-09-19T08:45:57.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.539+0000 7fd4620db740 10 client.24964 fetch_fsmap learned FSMap version 28 2024-09-19T08:45:57.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.539+0000 7fd4620db740 1 -- 172.21.15.164:0/2525752991 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556b30ef9060 con 0x556b30edbdc0 2024-09-19T08:45:57.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.539+0000 7fd43d7fa640 1 -- 172.21.15.164:0/2525752991 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd44c003d50 con 0x556b30edbdc0 2024-09-19T08:45:57.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.539+0000 7fd43d7fa640 1 -- 172.21.15.164:0/2525752991 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd45001ba50 con 0x556b30edbdc0 2024-09-19T08:45:57.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.539+0000 7fd4620db740 10 client.24964 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:45:57.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.539+0000 7fd4620db740 10 client.24964 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:45:57.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.539+0000 7fd4620db740 20 client.24964 populate_metadata read hostname 'smithi164' 2024-09-19T08:45:57.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.539+0000 7fd4620db740 1 --2- 172.21.15.164:0/2525752991 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556b30efa510 0x556b30efc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:57.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.539+0000 7fd4620db740 4 client.24964 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:45:57.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.539+0000 7fd4620db740 1 -- 172.21.15.164:0/2525752991 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b30e4e850 con 0x556b30efa510 2024-09-19T08:45:57.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.540+0000 7fd45d29d640 1 --2- 172.21.15.164:0/2525752991 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556b30efa510 0x556b30efc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:57.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.540+0000 7fd43d7fa640 10 client.24964 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:45:57.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.540+0000 7fd45d29d640 1 --2- 172.21.15.164:0/2525752991 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556b30efa510 0x556b30efc930 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3e5c16172628696 server_cookie=1a9c2c7f1bea784e in_seq=0 out_seq=0 2024-09-19T08:45:57.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.541+0000 7fd43d7fa640 1 -- 172.21.15.164:0/2525752991 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556b30e4e850 con 0x556b30efa510 2024-09-19T08:45:57.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.541+0000 7fd43d7fa640 10 client.24964 handle_command_reply: tid=0 2024-09-19T08:45:57.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.541+0000 7fd4620db740 1 client.24964 shutdown 2024-09-19T08:45:57.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.541+0000 7fd4620db740 1 -- 172.21.15.164:0/2525752991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b30edbdc0 msgr2=0x556b30ee31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:57.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.541+0000 7fd4620db740 1 --2- 172.21.15.164:0/2525752991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b30edbdc0 0x556b30ee31e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd450011ef0 tx=0x7fd45000f790 comp rx=0 tx=0).stop 2024-09-19T08:45:57.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.541+0000 7fd4620db740 1 -- 172.21.15.164:0/2525752991 shutdown_connections 2024-09-19T08:45:57.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.541+0000 7fd4620db740 1 --2- 172.21.15.164:0/2525752991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b30ee6680 0x556b30ee3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.541+0000 7fd4620db740 1 --2- 172.21.15.164:0/2525752991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b30edbdc0 0x556b30ee31e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.541+0000 7fd4620db740 1 --2- 172.21.15.164:0/2525752991 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556b30efa510 0x556b30efc930 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.541+0000 7fd4620db740 1 --2- 172.21.15.164:0/2525752991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b30edb4a0 0x556b30ee5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.541+0000 7fd4620db740 1 -- 172.21.15.164:0/2525752991 >> 172.21.15.164:0/2525752991 conn(0x556b30ed7a30 msgr2=0x556b30edd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:57.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.541+0000 7fd43d7fa640 0 client.24964 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:45:57.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.541+0000 7fd4620db740 1 -- 172.21.15.164:0/2525752991 shutdown_connections 2024-09-19T08:45:57.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.541+0000 7fd4620db740 1 -- 172.21.15.164:0/2525752991 wait complete. 2024-09-19T08:45:57.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.541+0000 7fd4620db740 20 client.24964 trim_cache size 0 max 16384 2024-09-19T08:45:57.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.558+0000 7fd4620db740 1 Processor -- start 2024-09-19T08:45:57.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.558+0000 7fd4620db740 1 -- start start 2024-09-19T08:45:57.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.558+0000 7fd4620db740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b30edb4a0 0x556b30ef68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:57.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.558+0000 7fd4620db740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b30edbdc0 0x556b30ede900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:57.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.558+0000 7fd4620db740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b30ee6680 0x556b30ee21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:57.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.558+0000 7fd4620db740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd4581ab300 con 0x556b30edbdc0 2024-09-19T08:45:57.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.558+0000 7fd4620db740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd4581ace40 con 0x556b30ee6680 2024-09-19T08:45:57.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.558+0000 7fd4620db740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd4581ab9d0 con 0x556b30edb4a0 2024-09-19T08:45:57.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.558+0000 7fd45da9e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b30edbdc0 0x556b30ede900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:57.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.558+0000 7fd45da9e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b30edbdc0 0x556b30ede900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39374/0 (socket says 172.21.15.164:39374) 2024-09-19T08:45:57.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.558+0000 7fd45da9e640 1 -- 172.21.15.164:0/4281160358 learned_addr learned my addr 172.21.15.164:0/4281160358 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:57.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.559+0000 7fd45d29d640 1 --2- 172.21.15.164:0/4281160358 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b30edb4a0 0x556b30ef68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:57.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.559+0000 7fd45d29d640 1 --2- 172.21.15.164:0/4281160358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b30ee6680 0x556b30ee21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:57.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.559+0000 7fd45da9e640 1 -- 172.21.15.164:0/4281160358 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b30edb4a0 msgr2=0x556b30ef68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:57.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.559+0000 7fd45da9e640 1 --2- 172.21.15.164:0/4281160358 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b30edb4a0 0x556b30ef68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.559+0000 7fd45da9e640 1 -- 172.21.15.164:0/4281160358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b30ee6680 msgr2=0x556b30ee21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:57.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.559+0000 7fd45da9e640 1 --2- 172.21.15.164:0/4281160358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b30ee6680 0x556b30ee21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.559+0000 7fd45da9e640 1 -- 172.21.15.164:0/4281160358 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b30edbbf0 con 0x556b30edbdc0 2024-09-19T08:45:57.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.559+0000 7fd45da9e640 1 --2- 172.21.15.164:0/4281160358 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b30edbdc0 0x556b30ede900 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd45001cff0 tx=0x7fd450011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db70b9cb9dac8a8d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:57.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.559+0000 7fd43f7fe640 1 -- 172.21.15.164:0/4281160358 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd450033070 con 0x556b30edbdc0 2024-09-19T08:45:57.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.560+0000 7fd43f7fe640 1 -- 172.21.15.164:0/4281160358 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd45002a050 con 0x556b30edbdc0 2024-09-19T08:45:57.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.560+0000 7fd43f7fe640 1 -- 172.21.15.164:0/4281160358 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd450028030 con 0x556b30edbdc0 2024-09-19T08:45:57.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.560+0000 7fd4620db740 1 -- 172.21.15.164:0/4281160358 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b30edbdc0 msgr2=0x556b30ede900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:57.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.560+0000 7fd4620db740 1 --2- 172.21.15.164:0/4281160358 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b30edbdc0 0x556b30ede900 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd45001cff0 tx=0x7fd450011b90 comp rx=0 tx=0).stop 2024-09-19T08:45:57.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.560+0000 7fd4620db740 1 -- 172.21.15.164:0/4281160358 shutdown_connections 2024-09-19T08:45:57.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.560+0000 7fd4620db740 1 --2- 172.21.15.164:0/4281160358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b30ee6680 0x556b30ee21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.560+0000 7fd4620db740 1 --2- 172.21.15.164:0/4281160358 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b30edbdc0 0x556b30ede900 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.560+0000 7fd4620db740 1 --2- 172.21.15.164:0/4281160358 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b30edb4a0 0x556b30ef68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.560+0000 7fd4620db740 1 -- 172.21.15.164:0/4281160358 >> 172.21.15.164:0/4281160358 conn(0x556b30ed7a30 msgr2=0x556b30ee1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:57.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.560+0000 7fd4620db740 1 -- 172.21.15.164:0/4281160358 shutdown_connections 2024-09-19T08:45:57.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.561+0000 7fd4620db740 1 -- 172.21.15.164:0/4281160358 wait complete. 2024-09-19T08:45:57.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.561+0000 7fd4620db740 1 -- start start 2024-09-19T08:45:57.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.561+0000 7fd4620db740 1 Processor -- start 2024-09-19T08:45:57.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.562+0000 7fd4620db740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b30edb4a0 0x556b30eedb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:57.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.562+0000 7fd4620db740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b30edbdc0 0x556b30eee060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:57.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.562+0000 7fd4620db740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b30ee6680 0x556b30f631a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:57.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.562+0000 7fd4620db740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd4581ac770 con 0x556b30ee6680 2024-09-19T08:45:57.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.562+0000 7fd4620db740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd4581ac0a0 con 0x556b30edb4a0 2024-09-19T08:45:57.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.562+0000 7fd4620db740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd4581a9340 con 0x556b30edbdc0 2024-09-19T08:45:57.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.562+0000 7fd4620db740 10 client.0 _ll_register_callbacks cb 0x7fd450007da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:45:57.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.562+0000 7fd45d29d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b30ee6680 0x556b30f631a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:57.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.562+0000 7fd45da9e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b30edbdc0 0x556b30eee060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:57.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.562+0000 7fd45d29d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b30ee6680 0x556b30f631a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39392/0 (socket says 172.21.15.164:39392) 2024-09-19T08:45:57.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.562+0000 7fd45d29d640 1 -- 172.21.15.164:0/699816770 learned_addr learned my addr 172.21.15.164:0/699816770 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:57.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.562+0000 7fd45d29d640 1 --2- 172.21.15.164:0/699816770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b30edb4a0 0x556b30eedb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:57.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.562+0000 7fd45d29d640 1 -- 172.21.15.164:0/699816770 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b30edbdc0 msgr2=0x556b30eee060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:57.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.562+0000 7fd45d29d640 1 --2- 172.21.15.164:0/699816770 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b30edbdc0 0x556b30eee060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.562+0000 7fd45d29d640 1 -- 172.21.15.164:0/699816770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b30edb4a0 msgr2=0x556b30eedb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:57.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.562+0000 7fd45d29d640 1 --2- 172.21.15.164:0/699816770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b30edb4a0 0x556b30eedb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.562+0000 7fd45d29d640 1 -- 172.21.15.164:0/699816770 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b30eee930 con 0x556b30ee6680 2024-09-19T08:45:57.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.562+0000 7fd45da9e640 1 --2- 172.21.15.164:0/699816770 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b30edbdc0 0x556b30eee060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:45:57.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.562+0000 7fd45d29d640 1 --2- 172.21.15.164:0/699816770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b30ee6680 0x556b30f631a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd44c031b80 tx=0x7fd44c054040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e104e4028fe547 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:57.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.562+0000 7fd43d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:45:57.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.563+0000 7fd43d7fa640 1 -- 172.21.15.164:0/699816770 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c035040 con 0x556b30ee6680 2024-09-19T08:45:57.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.563+0000 7fd4620db740 1 -- 172.21.15.164:0/699816770 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd45002a050 con 0x556b30ee6680 2024-09-19T08:45:57.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.563+0000 7fd43d7fa640 1 -- 172.21.15.164:0/699816770 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fd44c003670 con 0x556b30ee6680 2024-09-19T08:45:57.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.563+0000 7fd43d7fa640 1 -- 172.21.15.164:0/699816770 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd44c003870 con 0x556b30ee6680 2024-09-19T08:45:57.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.563+0000 7fd4620db740 10 client.15390 fetch_fsmap learned FSMap version 28 2024-09-19T08:45:57.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.563+0000 7fd4620db740 1 -- 172.21.15.164:0/699816770 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556b30f647f0 con 0x556b30ee6680 2024-09-19T08:45:57.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.563+0000 7fd43d7fa640 1 -- 172.21.15.164:0/699816770 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd45002a050 con 0x556b30ee6680 2024-09-19T08:45:57.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.563+0000 7fd43d7fa640 1 -- 172.21.15.164:0/699816770 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd44c04b440 con 0x556b30ee6680 2024-09-19T08:45:57.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.563+0000 7fd4620db740 10 client.15390 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:45:57.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.564+0000 7fd4620db740 10 client.15390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:45:57.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.564+0000 7fd4620db740 20 client.15390 populate_metadata read hostname 'smithi164' 2024-09-19T08:45:57.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.564+0000 7fd4620db740 1 --2- 172.21.15.164:0/699816770 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556b30f64ae0 0x556b30f66ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:57.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.564+0000 7fd4620db740 4 client.15390 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:45:57.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.564+0000 7fd4620db740 1 -- 172.21.15.164:0/699816770 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556b30e48d60 con 0x556b30f64ae0 2024-09-19T08:45:57.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.564+0000 7fd45da9e640 1 --2- 172.21.15.164:0/699816770 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556b30f64ae0 0x556b30f66ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:57.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.564+0000 7fd45da9e640 1 --2- 172.21.15.164:0/699816770 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556b30f64ae0 0x556b30f66ed0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=553f8e472a2ed390 server_cookie=9fe30911e54744c0 in_seq=0 out_seq=0 2024-09-19T08:45:57.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.564+0000 7fd43d7fa640 10 client.15390 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:45:57.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.564+0000 7f006212c640 1 -- 172.21.15.164:0/72756537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f005c102050 msgr2=0x7f005c10d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:57.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.564+0000 7f006212c640 1 --2- 172.21.15.164:0/72756537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f005c102050 0x7f005c10d7d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0054009ca0 tx=0x7f005402a950 comp rx=0 tx=0).stop 2024-09-19T08:45:57.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.565+0000 7f006212c640 1 -- 172.21.15.164:0/72756537 shutdown_connections 2024-09-19T08:45:57.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.565+0000 7f006212c640 1 --2- 172.21.15.164:0/72756537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f005c102050 0x7f005c10d7d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.565+0000 7f006212c640 1 --2- 172.21.15.164:0/72756537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f005c1016d0 0x7f005c101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.565+0000 7f006212c640 1 --2- 172.21.15.164:0/72756537 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f005c105af0 0x7f005c105ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.565+0000 7f006212c640 1 -- 172.21.15.164:0/72756537 >> 172.21.15.164:0/72756537 conn(0x7f005c075f20 msgr2=0x7f005c076330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:57.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.565+0000 7f006212c640 1 -- 172.21.15.164:0/72756537 shutdown_connections 2024-09-19T08:45:57.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.565+0000 7f006212c640 1 -- 172.21.15.164:0/72756537 wait complete. 2024-09-19T08:45:57.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.565+0000 7fd43d7fa640 1 -- 172.21.15.164:0/699816770 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556b30e48d60 con 0x556b30f64ae0 2024-09-19T08:45:57.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.565+0000 7fd43d7fa640 10 client.15390 handle_command_reply: tid=0 2024-09-19T08:45:57.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.565+0000 7fd4620db740 1 client.15390 shutdown 2024-09-19T08:45:57.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.565+0000 7fd4620db740 1 -- 172.21.15.164:0/699816770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b30ee6680 msgr2=0x556b30f631a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:57.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.565+0000 7fd4620db740 1 --2- 172.21.15.164:0/699816770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b30ee6680 0x556b30f631a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd44c031b80 tx=0x7fd44c054040 comp rx=0 tx=0).stop 2024-09-19T08:45:57.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.565+0000 7fd4620db740 1 -- 172.21.15.164:0/699816770 shutdown_connections 2024-09-19T08:45:57.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.565+0000 7fd4620db740 1 --2- 172.21.15.164:0/699816770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b30ee6680 0x556b30f631a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.565+0000 7fd4620db740 1 --2- 172.21.15.164:0/699816770 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b30edbdc0 0x556b30eee060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.565+0000 7fd4620db740 1 --2- 172.21.15.164:0/699816770 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556b30f64ae0 0x556b30f66ed0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.565+0000 7fd4620db740 1 --2- 172.21.15.164:0/699816770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b30edb4a0 0x556b30eedb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.565+0000 7fd4620db740 1 -- 172.21.15.164:0/699816770 >> 172.21.15.164:0/699816770 conn(0x556b30ed7a30 msgr2=0x556b30ed6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:57.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.565+0000 7fd43d7fa640 0 client.15390 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:45:57.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.565+0000 7fd4620db740 1 -- 172.21.15.164:0/699816770 shutdown_connections 2024-09-19T08:45:57.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.565+0000 7f006212c640 1 Processor -- start 2024-09-19T08:45:57.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.565+0000 7fd4620db740 1 -- 172.21.15.164:0/699816770 wait complete. 2024-09-19T08:45:57.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.566+0000 7fd4620db740 20 client.15390 trim_cache size 0 max 16384 2024-09-19T08:45:57.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.566+0000 7f006212c640 1 -- start start 2024-09-19T08:45:57.572 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:45:57.572 INFO:teuthology.orchestra.run.smithi164.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T08:45:57.573 INFO:teuthology.orchestra.run.smithi164.stdout: "scrubs": { 2024-09-19T08:45:57.573 INFO:teuthology.orchestra.run.smithi164.stdout: "1e77c761-d10c-42ee-bf2c-62f79ddff737": { 2024-09-19T08:45:57.573 INFO:teuthology.orchestra.run.smithi164.stdout: "path": "/", 2024-09-19T08:45:57.573 INFO:teuthology.orchestra.run.smithi164.stdout: "tag": "1e77c761-d10c-42ee-bf2c-62f79ddff737", 2024-09-19T08:45:57.573 INFO:teuthology.orchestra.run.smithi164.stdout: "options": "recursive,force" 2024-09-19T08:45:57.573 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:45:57.573 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:45:57.573 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:45:57.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.566+0000 7f006212c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f005c1016d0 0x7f005c198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:57.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.566+0000 7f006212c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f005c102050 0x7f005c198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:57.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.566+0000 7f006212c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f005c105af0 0x7f005c19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:57.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.566+0000 7f006212c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f005c110e30 con 0x7f005c105af0 2024-09-19T08:45:57.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.566+0000 7f005bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f005c105af0 0x7f005c19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:57.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.566+0000 7f005bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f005c105af0 0x7f005c19cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39398/0 (socket says 172.21.15.164:39398) 2024-09-19T08:45:57.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.566+0000 7f005bfff640 1 -- 172.21.15.164:0/3873315852 learned_addr learned my addr 172.21.15.164:0/3873315852 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:57.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.566+0000 7f005b7fe640 1 --2- 172.21.15.164:0/3873315852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f005c1016d0 0x7f005c198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:57.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.566+0000 7f006212c640 1 -- 172.21.15.164:0/3873315852 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f005c110cb0 con 0x7f005c102050 2024-09-19T08:45:57.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.566+0000 7f005affd640 1 --2- 172.21.15.164:0/3873315852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f005c102050 0x7f005c198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:57.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.566+0000 7fd45fd02640 1 -- 172.21.15.164:0/1283372116 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd4400a3440 msgr2=0x7fd4400a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:57.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.566+0000 7fd45fd02640 1 --2- 172.21.15.164:0/1283372116 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd4400a3440 0x7fd4400a5900 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd4480cd500 tx=0x7fd44806edb0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.566+0000 7fd45fd02640 1 -- 172.21.15.164:0/1283372116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd458102ef0 msgr2=0x7fd4581a4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:57.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.566+0000 7f006212c640 1 -- 172.21.15.164:0/3873315852 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f005c110fb0 con 0x7f005c1016d0 2024-09-19T08:45:57.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.566+0000 7fd45fd02640 1 --2- 172.21.15.164:0/1283372116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd458102ef0 0x7fd4581a4270 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd44800c970 tx=0x7fd44800ce30 comp rx=0 tx=0).stop 2024-09-19T08:45:57.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.567+0000 7fd45fd02640 1 -- 172.21.15.164:0/1283372116 shutdown_connections 2024-09-19T08:45:57.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.567+0000 7fd45fd02640 1 --2- 172.21.15.164:0/1283372116 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd4400a3440 0x7fd4400a5900 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.567+0000 7fd45fd02640 1 --2- 172.21.15.164:0/1283372116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd458102ef0 0x7fd4581a4270 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.567+0000 7fd45fd02640 1 --2- 172.21.15.164:0/1283372116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd458102570 0x7fd458199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.567+0000 7fd45fd02640 1 --2- 172.21.15.164:0/1283372116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd458100e20 0x7fd4581991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.567+0000 7fd45fd02640 1 -- 172.21.15.164:0/1283372116 >> 172.21.15.164:0/1283372116 conn(0x7fd4580fc7d0 msgr2=0x7fd458106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:57.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.567+0000 7f005affd640 1 -- 172.21.15.164:0/3873315852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f005c1016d0 msgr2=0x7f005c198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:57.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.567+0000 7f005affd640 1 --2- 172.21.15.164:0/3873315852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f005c1016d0 0x7f005c198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.567+0000 7f005affd640 1 -- 172.21.15.164:0/3873315852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f005c105af0 msgr2=0x7f005c19cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:57.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.567+0000 7f005affd640 1 --2- 172.21.15.164:0/3873315852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f005c105af0 0x7f005c19cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.567+0000 7f005affd640 1 -- 172.21.15.164:0/3873315852 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f005c19d300 con 0x7f005c102050 2024-09-19T08:45:57.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.567+0000 7f005bfff640 1 --2- 172.21.15.164:0/3873315852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f005c105af0 0x7f005c19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:45:57.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.567+0000 7fd45fd02640 1 -- 172.21.15.164:0/1283372116 shutdown_connections 2024-09-19T08:45:57.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.567+0000 7fd45fd02640 1 -- 172.21.15.164:0/1283372116 wait complete. 2024-09-19T08:45:57.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.567+0000 7f005affd640 1 --2- 172.21.15.164:0/3873315852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f005c102050 0x7f005c198850 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f005000d6d0 tx=0x7f005000db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:57.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.567+0000 7f0058ff9640 1 -- 172.21.15.164:0/3873315852 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0050017070 con 0x7f005c102050 2024-09-19T08:45:57.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.568+0000 7f006212c640 1 -- 172.21.15.164:0/3873315852 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f005c19d5f0 con 0x7f005c102050 2024-09-19T08:45:57.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.568+0000 7f0058ff9640 1 -- 172.21.15.164:0/3873315852 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0050009820 con 0x7f005c102050 2024-09-19T08:45:57.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.568+0000 7f0058ff9640 1 -- 172.21.15.164:0/3873315852 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0050004cd0 con 0x7f005c102050 2024-09-19T08:45:57.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.568+0000 7f006212c640 1 -- 172.21.15.164:0/3873315852 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f005c1a4e80 con 0x7f005c102050 2024-09-19T08:45:57.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.569+0000 7f006212c640 1 -- 172.21.15.164:0/3873315852 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f005c105ed0 con 0x7f005c102050 2024-09-19T08:45:57.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.570+0000 7f0058ff9640 1 -- 172.21.15.164:0/3873315852 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f00500099d0 con 0x7f005c102050 2024-09-19T08:45:57.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.571+0000 7f0058ff9640 1 --2- 172.21.15.164:0/3873315852 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f00440a3500 0x7f00440a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:57.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.572+0000 7f005b7fe640 1 --2- 172.21.15.164:0/3873315852 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f00440a3500 0x7f00440a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:57.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.572+0000 7f0058ff9640 1 -- 172.21.15.164:0/3873315852 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f00500cfd50 con 0x7f005c102050 2024-09-19T08:45:57.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.572+0000 7f005b7fe640 1 --2- 172.21.15.164:0/3873315852 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f00440a3500 0x7f00440a59c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f004c0095f0 tx=0x7f004c005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:57.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.574+0000 7f0058ff9640 1 -- 172.21.15.164:0/3873315852 <== mon.2 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f005009bf80 con 0x7f005c102050 2024-09-19T08:45:57.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.770+0000 7f006212c640 1 -- 172.21.15.164:0/3873315852 --> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f005c103d60 con 0x7f00440a3500 2024-09-19T08:45:57.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.777+0000 7f0058ff9640 1 -- 172.21.15.164:0/3873315852 <== mgr.15201 v2:172.21.15.164:6800/173486514 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f005c103d60 con 0x7f00440a3500 2024-09-19T08:45:57.777 INFO:teuthology.orchestra.run.smithi164.stdout:/ 1m 6m3h 2024-09-19T08:45:57.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.779+0000 7f006212c640 1 -- 172.21.15.164:0/3873315852 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f00440a3500 msgr2=0x7f00440a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:57.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.779+0000 7f006212c640 1 --2- 172.21.15.164:0/3873315852 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f00440a3500 0x7f00440a59c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f004c0095f0 tx=0x7f004c005f90 comp rx=0 tx=0).stop 2024-09-19T08:45:57.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.779+0000 7f006212c640 1 -- 172.21.15.164:0/3873315852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f005c102050 msgr2=0x7f005c198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:57.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.779+0000 7f006212c640 1 --2- 172.21.15.164:0/3873315852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f005c102050 0x7f005c198850 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f005000d6d0 tx=0x7f005000db90 comp rx=0 tx=0).stop 2024-09-19T08:45:57.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.779+0000 7f006212c640 1 -- 172.21.15.164:0/3873315852 shutdown_connections 2024-09-19T08:45:57.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.779+0000 7f006212c640 1 --2- 172.21.15.164:0/3873315852 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f00440a3500 0x7f00440a59c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.779+0000 7f006212c640 1 --2- 172.21.15.164:0/3873315852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f005c105af0 0x7f005c19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.779+0000 7f006212c640 1 --2- 172.21.15.164:0/3873315852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f005c102050 0x7f005c198850 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.779+0000 7f006212c640 1 --2- 172.21.15.164:0/3873315852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f005c1016d0 0x7f005c198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.779+0000 7f006212c640 1 -- 172.21.15.164:0/3873315852 >> 172.21.15.164:0/3873315852 conn(0x7f005c075f20 msgr2=0x7f005c103650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:57.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.779+0000 7f006212c640 1 -- 172.21.15.164:0/3873315852 shutdown_connections 2024-09-19T08:45:57.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:57.779+0000 7f006212c640 1 -- 172.21.15.164:0/3873315852 wait complete. 2024-09-19T08:45:57.791 DEBUG:teuthology.orchestra.run.smithi164:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-19T08:45:57.818 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-19T08:45:57.829 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-19T08:45:57.844 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T08:45:57.844 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T08:45:58.216 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:57 smithi164 ceph-mon[30928]: scrub queued for path: / 2024-09-19T08:45:58.216 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:57 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:45:58.216 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:57 smithi164 ceph-mon[30928]: scrub summary: active paths [/] 2024-09-19T08:45:58.217 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:57 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:45:58.217 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:57 smithi164 ceph-mon[30928]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:45:58.217 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:57 smithi164 ceph-mon[30928]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:45:58.217 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:57 smithi164 ceph-mon[30928]: from='client.25001 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:45:58.246 INFO:tasks.cephfs.filesystem:scrub status for tag:1e77c761-d10c-42ee-bf2c-62f79ddff737 - {'path': '/', 'tag': '1e77c761-d10c-42ee-bf2c-62f79ddff737', 'options': 'recursive,force'} 2024-09-19T08:45:58.267 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:45:58.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:57 smithi170 ceph-mon[35072]: scrub queued for path: / 2024-09-19T08:45:58.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:57 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:45:58.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:57 smithi170 ceph-mon[35072]: scrub summary: active paths [/] 2024-09-19T08:45:58.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:57 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:45:58.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:57 smithi170 ceph-mon[35072]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:45:58.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:57 smithi170 ceph-mon[35072]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:45:58.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:57 smithi170 ceph-mon[35072]: from='client.25001 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:45:58.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:57 smithi173 ceph-mon[35306]: scrub queued for path: / 2024-09-19T08:45:58.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:57 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:45:58.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:57 smithi173 ceph-mon[35306]: scrub summary: active paths [/] 2024-09-19T08:45:58.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:57 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:45:58.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:57 smithi173 ceph-mon[35306]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:45:58.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:57 smithi173 ceph-mon[35306]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:45:58.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:57 smithi173 ceph-mon[35306]: from='client.25001 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:45:58.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.733+0000 7f8c2b3e7640 1 -- 172.21.15.164:0/1395613963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c24102570 msgr2=0x7f8c241029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:58.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.733+0000 7f8c2b3e7640 1 --2- 172.21.15.164:0/1395613963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c24102570 0x7f8c241029b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8c14009900 tx=0x7f8c1402a480 comp rx=0 tx=0).stop 2024-09-19T08:45:58.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.733+0000 7f8c2b3e7640 1 -- 172.21.15.164:0/1395613963 shutdown_connections 2024-09-19T08:45:58.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.733+0000 7f8c2b3e7640 1 --2- 172.21.15.164:0/1395613963 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c24102ef0 0x7f8c2410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:58.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.733+0000 7f8c2b3e7640 1 --2- 172.21.15.164:0/1395613963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c24102570 0x7f8c241029b0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:58.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.733+0000 7f8c2b3e7640 1 --2- 172.21.15.164:0/1395613963 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c24100e20 0x7f8c24101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:58.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.733+0000 7f8c2b3e7640 1 -- 172.21.15.164:0/1395613963 >> 172.21.15.164:0/1395613963 conn(0x7f8c240fc7d0 msgr2=0x7f8c240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:58.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.733+0000 7f8c2b3e7640 1 -- 172.21.15.164:0/1395613963 shutdown_connections 2024-09-19T08:45:58.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.733+0000 7f8c2b3e7640 1 -- 172.21.15.164:0/1395613963 wait complete. 2024-09-19T08:45:58.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.733+0000 7f8c2b3e7640 1 Processor -- start 2024-09-19T08:45:58.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.733+0000 7f8c2b3e7640 1 -- start start 2024-09-19T08:45:58.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.733+0000 7f8c2b3e7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c24100e20 0x7f8c240fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:58.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.733+0000 7f8c2b3e7640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c24102570 0x7f8c240fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:58.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.734+0000 7f8c2b3e7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c24102ef0 0x7f8c240fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:58.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.734+0000 7f8c2b3e7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8c24110f80 con 0x7f8c24102ef0 2024-09-19T08:45:58.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.734+0000 7f8c2b3e7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8c24110e00 con 0x7f8c24100e20 2024-09-19T08:45:58.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.734+0000 7f8c2b3e7640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8c24111100 con 0x7f8c24102570 2024-09-19T08:45:58.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.734+0000 7f8c29984640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c24102ef0 0x7f8c240fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:58.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.734+0000 7f8c29984640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c24102ef0 0x7f8c240fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39414/0 (socket says 172.21.15.164:39414) 2024-09-19T08:45:58.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.734+0000 7f8c29984640 1 -- 172.21.15.164:0/1944946338 learned_addr learned my addr 172.21.15.164:0/1944946338 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:45:58.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.734+0000 7f8c29183640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c24100e20 0x7f8c240fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:58.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.734+0000 7f8c28982640 1 --2- 172.21.15.164:0/1944946338 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c24102570 0x7f8c240fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:58.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.734+0000 7f8c29984640 1 -- 172.21.15.164:0/1944946338 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c24102570 msgr2=0x7f8c240fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:58.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.734+0000 7f8c29984640 1 --2- 172.21.15.164:0/1944946338 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c24102570 0x7f8c240fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:58.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.734+0000 7f8c29984640 1 -- 172.21.15.164:0/1944946338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c24100e20 msgr2=0x7f8c240fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:58.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.734+0000 7f8c29984640 1 --2- 172.21.15.164:0/1944946338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c24100e20 0x7f8c240fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:58.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.734+0000 7f8c29984640 1 -- 172.21.15.164:0/1944946338 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c241a6990 con 0x7f8c24102ef0 2024-09-19T08:45:58.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.734+0000 7f8c29984640 1 --2- 172.21.15.164:0/1944946338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c24102ef0 0x7f8c240fd700 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c00eaa0 tx=0x7f8c1c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:58.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.734+0000 7f8c1a7fc640 1 -- 172.21.15.164:0/1944946338 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c00cdb0 con 0x7f8c24102ef0 2024-09-19T08:45:58.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.734+0000 7f8c2b3e7640 1 -- 172.21.15.164:0/1944946338 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c241a6c80 con 0x7f8c24102ef0 2024-09-19T08:45:58.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.734+0000 7f8c1a7fc640 1 -- 172.21.15.164:0/1944946338 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c1c005020 con 0x7f8c24102ef0 2024-09-19T08:45:58.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.734+0000 7f8c1a7fc640 1 -- 172.21.15.164:0/1944946338 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c0116d0 con 0x7f8c24102ef0 2024-09-19T08:45:58.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.734+0000 7f8c2b3e7640 1 -- 172.21.15.164:0/1944946338 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c241a71c0 con 0x7f8c24102ef0 2024-09-19T08:45:58.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.735+0000 7f8c2b3e7640 1 -- 172.21.15.164:0/1944946338 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c24101200 con 0x7f8c24102ef0 2024-09-19T08:45:58.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.737+0000 7f8c1a7fc640 1 -- 172.21.15.164:0/1944946338 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f8c1c0054d0 con 0x7f8c24102ef0 2024-09-19T08:45:58.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.738+0000 7f8c1a7fc640 1 --2- 172.21.15.164:0/1944946338 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f8c100a3440 0x7f8c100a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:58.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.738+0000 7f8c29183640 1 --2- 172.21.15.164:0/1944946338 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f8c100a3440 0x7f8c100a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:58.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.738+0000 7f8c1a7fc640 1 -- 172.21.15.164:0/1944946338 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f8c1c0d0360 con 0x7f8c24102ef0 2024-09-19T08:45:58.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.738+0000 7f8c29183640 1 --2- 172.21.15.164:0/1944946338 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f8c100a3440 0x7f8c100a5900 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000bbe0 tx=0x7f8c2000b390 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:58.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.741+0000 7f8c1a7fc640 1 -- 172.21.15.164:0/1944946338 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8c1c016030 con 0x7f8c24102ef0 2024-09-19T08:45:58.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.909+0000 7f8c2b3e7640 1 -- 172.21.15.164:0/1944946338 --> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8c2410a110 con 0x7f8c100a3440 2024-09-19T08:45:58.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.916+0000 7f8c1a7fc640 1 -- 172.21.15.164:0/1944946338 <== mgr.15201 v2:172.21.15.164:6800/173486514 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258047 (secure 0 0 0) 0x7f8c20010c30 con 0x7f8c100a3440 2024-09-19T08:45:58.918 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:45:58.919 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-09-19T08:45:58.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.919+0000 7f8c2b3e7640 1 -- 172.21.15.164:0/1944946338 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f8c100a3440 msgr2=0x7f8c100a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:58.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.919+0000 7f8c2b3e7640 1 --2- 172.21.15.164:0/1944946338 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f8c100a3440 0x7f8c100a5900 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000bbe0 tx=0x7f8c2000b390 comp rx=0 tx=0).stop 2024-09-19T08:45:58.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.919+0000 7f8c2b3e7640 1 -- 172.21.15.164:0/1944946338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c24102ef0 msgr2=0x7f8c240fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:58.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.919+0000 7f8c2b3e7640 1 --2- 172.21.15.164:0/1944946338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c24102ef0 0x7f8c240fd700 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c00eaa0 tx=0x7f8c1c00ef60 comp rx=0 tx=0).stop 2024-09-19T08:45:58.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.919+0000 7f8c2b3e7640 1 -- 172.21.15.164:0/1944946338 shutdown_connections 2024-09-19T08:45:58.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.919+0000 7f8c2b3e7640 1 --2- 172.21.15.164:0/1944946338 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f8c100a3440 0x7f8c100a5900 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:58.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.919+0000 7f8c2b3e7640 1 --2- 172.21.15.164:0/1944946338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c24102ef0 0x7f8c240fd700 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:58.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.919+0000 7f8c2b3e7640 1 --2- 172.21.15.164:0/1944946338 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c24102570 0x7f8c240fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:58.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.919+0000 7f8c2b3e7640 1 --2- 172.21.15.164:0/1944946338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c24100e20 0x7f8c240fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:58.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.919+0000 7f8c2b3e7640 1 -- 172.21.15.164:0/1944946338 >> 172.21.15.164:0/1944946338 conn(0x7f8c240fc7d0 msgr2=0x7f8c2406a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:58.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.919+0000 7f8c2b3e7640 1 -- 172.21.15.164:0/1944946338 shutdown_connections 2024-09-19T08:45:58.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:45:58.919+0000 7f8c2b3e7640 1 -- 172.21.15.164:0/1944946338 wait complete. 2024-09-19T08:45:59.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:45:58 smithi164 ceph-mon[30928]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 0 B/s wr, 19 op/s 2024-09-19T08:45:59.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:45:58 smithi170 ceph-mon[35072]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 0 B/s wr, 19 op/s 2024-09-19T08:45:59.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:45:58 smithi173 ceph-mon[35306]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 0 B/s wr, 19 op/s 2024-09-19T08:45:59.663 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-09-19T08:45:57.983100+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1190407,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1477,"num_read_kb":1066,"num_write":577,"num_write_kb":4404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":951,"ondisk_log_size":951,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":389776,"kb_used_data":11376,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124552048,"statfs":{"total":1151940427776,"available":1151541297152,"internally_reserved":0,"allocated":11649024,"data_stored":8897121,"data_compressed":220029,"data_compressed_allocated":282624,"data_compressed_original":3416064,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":151,"num_read_kb":236,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.993012"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229715+0000","last_change":"2024-09-19T08:43:52.766709+0000","last_active":"2024-09-19T08:45:48.229715+0000","last_peered":"2024-09-19T08:45:48.229715+0000","last_clean":"2024-09-19T08:45:48.229715+0000","last_became_active":"2024-09-19T08:43:52.765970+0000","last_became_peered":"2024-09-19T08:43:52.765970+0000","last_unstale":"2024-09-19T08:45:48.229715+0000","last_undegraded":"2024-09-19T08:45:48.229715+0000","last_fullsized":"2024-09-19T08:45:48.229715+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:12.176870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.230605+0000","last_change":"2024-09-19T08:43:52.766590+0000","last_active":"2024-09-19T08:45:48.230605+0000","last_peered":"2024-09-19T08:45:48.230605+0000","last_clean":"2024-09-19T08:45:48.230605+0000","last_became_active":"2024-09-19T08:43:52.766255+0000","last_became_peered":"2024-09-19T08:43:52.766255+0000","last_unstale":"2024-09-19T08:45:48.230605+0000","last_undegraded":"2024-09-19T08:45:48.230605+0000","last_fullsized":"2024-09-19T08:45:48.230605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:15:41.359806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.230306+0000","last_change":"2024-09-19T08:43:52.761956+0000","last_active":"2024-09-19T08:45:48.230306+0000","last_peered":"2024-09-19T08:45:48.230306+0000","last_clean":"2024-09-19T08:45:48.230306+0000","last_became_active":"2024-09-19T08:43:52.761644+0000","last_became_peered":"2024-09-19T08:43:52.761644+0000","last_unstale":"2024-09-19T08:45:48.230306+0000","last_undegraded":"2024-09-19T08:45:48.230306+0000","last_fullsized":"2024-09-19T08:45:48.230306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:13.770361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.966146+0000","last_change":"2024-09-19T08:43:52.764131+0000","last_active":"2024-09-19T08:45:47.966146+0000","last_peered":"2024-09-19T08:45:47.966146+0000","last_clean":"2024-09-19T08:45:47.966146+0000","last_became_active":"2024-09-19T08:43:52.763812+0000","last_became_peered":"2024-09-19T08:43:52.763812+0000","last_unstale":"2024-09-19T08:45:47.966146+0000","last_undegraded":"2024-09-19T08:45:47.966146+0000","last_fullsized":"2024-09-19T08:45:47.966146+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:32.272157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065874+0000","last_change":"2024-09-19T08:43:52.754673+0000","last_active":"2024-09-19T08:45:48.065874+0000","last_peered":"2024-09-19T08:45:48.065874+0000","last_clean":"2024-09-19T08:45:48.065874+0000","last_became_active":"2024-09-19T08:43:52.754144+0000","last_became_peered":"2024-09-19T08:43:52.754144+0000","last_unstale":"2024-09-19T08:45:48.065874+0000","last_undegraded":"2024-09-19T08:45:48.065874+0000","last_fullsized":"2024-09-19T08:45:48.065874+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:16.069260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.076773+0000","last_change":"2024-09-19T08:43:52.758939+0000","last_active":"2024-09-19T08:45:48.076773+0000","last_peered":"2024-09-19T08:45:48.076773+0000","last_clean":"2024-09-19T08:45:48.076773+0000","last_became_active":"2024-09-19T08:43:52.758611+0000","last_became_peered":"2024-09-19T08:43:52.758611+0000","last_unstale":"2024-09-19T08:45:48.076773+0000","last_undegraded":"2024-09-19T08:45:48.076773+0000","last_fullsized":"2024-09-19T08:45:48.076773+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:35:56.463545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221260+0000","last_change":"2024-09-19T08:43:52.770261+0000","last_active":"2024-09-19T08:45:48.221260+0000","last_peered":"2024-09-19T08:45:48.221260+0000","last_clean":"2024-09-19T08:45:48.221260+0000","last_became_active":"2024-09-19T08:43:52.768253+0000","last_became_peered":"2024-09-19T08:43:52.768253+0000","last_unstale":"2024-09-19T08:45:48.221260+0000","last_undegraded":"2024-09-19T08:45:48.221260+0000","last_fullsized":"2024-09-19T08:45:48.221260+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:40.756988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065907+0000","last_change":"2024-09-19T08:43:52.768564+0000","last_active":"2024-09-19T08:45:48.065907+0000","last_peered":"2024-09-19T08:45:48.065907+0000","last_clean":"2024-09-19T08:45:48.065907+0000","last_became_active":"2024-09-19T08:43:52.768260+0000","last_became_peered":"2024-09-19T08:43:52.768260+0000","last_unstale":"2024-09-19T08:45:48.065907+0000","last_undegraded":"2024-09-19T08:45:48.065907+0000","last_fullsized":"2024-09-19T08:45:48.065907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:16.528115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.222053+0000","last_change":"2024-09-19T08:43:52.769804+0000","last_active":"2024-09-19T08:45:48.222053+0000","last_peered":"2024-09-19T08:45:48.222053+0000","last_clean":"2024-09-19T08:45:48.222053+0000","last_became_active":"2024-09-19T08:43:52.769504+0000","last_became_peered":"2024-09-19T08:43:52.769504+0000","last_unstale":"2024-09-19T08:45:48.222053+0000","last_undegraded":"2024-09-19T08:45:48.222053+0000","last_fullsized":"2024-09-19T08:45:48.222053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:22:38.514410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.077170+0000","last_change":"2024-09-19T08:43:52.763278+0000","last_active":"2024-09-19T08:45:48.077170+0000","last_peered":"2024-09-19T08:45:48.077170+0000","last_clean":"2024-09-19T08:45:48.077170+0000","last_became_active":"2024-09-19T08:43:52.762985+0000","last_became_peered":"2024-09-19T08:43:52.762985+0000","last_unstale":"2024-09-19T08:45:48.077170+0000","last_undegraded":"2024-09-19T08:45:48.077170+0000","last_fullsized":"2024-09-19T08:45:48.077170+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:30.048782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221488+0000","last_change":"2024-09-19T08:43:52.770864+0000","last_active":"2024-09-19T08:45:48.221488+0000","last_peered":"2024-09-19T08:45:48.221488+0000","last_clean":"2024-09-19T08:45:48.221488+0000","last_became_active":"2024-09-19T08:43:52.770518+0000","last_became_peered":"2024-09-19T08:43:52.770518+0000","last_unstale":"2024-09-19T08:45:48.221488+0000","last_undegraded":"2024-09-19T08:45:48.221488+0000","last_fullsized":"2024-09-19T08:45:48.221488+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:08.931452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.966171+0000","last_change":"2024-09-19T08:43:52.757464+0000","last_active":"2024-09-19T08:45:47.966171+0000","last_peered":"2024-09-19T08:45:47.966171+0000","last_clean":"2024-09-19T08:45:47.966171+0000","last_became_active":"2024-09-19T08:43:52.757101+0000","last_became_peered":"2024-09-19T08:43:52.757101+0000","last_unstale":"2024-09-19T08:45:47.966171+0000","last_undegraded":"2024-09-19T08:45:47.966171+0000","last_fullsized":"2024-09-19T08:45:47.966171+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:17.395735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.224010+0000","last_change":"2024-09-19T08:43:52.767063+0000","last_active":"2024-09-19T08:45:48.224010+0000","last_peered":"2024-09-19T08:45:48.224010+0000","last_clean":"2024-09-19T08:45:48.224010+0000","last_became_active":"2024-09-19T08:43:52.766755+0000","last_became_peered":"2024-09-19T08:43:52.766755+0000","last_unstale":"2024-09-19T08:45:48.224010+0000","last_undegraded":"2024-09-19T08:45:48.224010+0000","last_fullsized":"2024-09-19T08:45:48.224010+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:30.530937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.230150+0000","last_change":"2024-09-19T08:43:52.766515+0000","last_active":"2024-09-19T08:45:48.230150+0000","last_peered":"2024-09-19T08:45:48.230150+0000","last_clean":"2024-09-19T08:45:48.230150+0000","last_became_active":"2024-09-19T08:43:52.765847+0000","last_became_peered":"2024-09-19T08:43:52.765847+0000","last_unstale":"2024-09-19T08:45:48.230150+0000","last_undegraded":"2024-09-19T08:45:48.230150+0000","last_fullsized":"2024-09-19T08:45:48.230150+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:50:35.948663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229847+0000","last_change":"2024-09-19T08:43:52.773702+0000","last_active":"2024-09-19T08:45:48.229847+0000","last_peered":"2024-09-19T08:45:48.229847+0000","last_clean":"2024-09-19T08:45:48.229847+0000","last_became_active":"2024-09-19T08:43:52.773324+0000","last_became_peered":"2024-09-19T08:43:52.773324+0000","last_unstale":"2024-09-19T08:45:48.229847+0000","last_undegraded":"2024-09-19T08:45:48.229847+0000","last_fullsized":"2024-09-19T08:45:48.229847+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:34.368014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221253+0000","last_change":"2024-09-19T08:43:52.762745+0000","last_active":"2024-09-19T08:45:48.221253+0000","last_peered":"2024-09-19T08:45:48.221253+0000","last_clean":"2024-09-19T08:45:48.221253+0000","last_became_active":"2024-09-19T08:43:52.762172+0000","last_became_peered":"2024-09-19T08:43:52.762172+0000","last_unstale":"2024-09-19T08:45:48.221253+0000","last_undegraded":"2024-09-19T08:45:48.221253+0000","last_fullsized":"2024-09-19T08:45:48.221253+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:57.369775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229743+0000","last_change":"2024-09-19T08:43:52.767118+0000","last_active":"2024-09-19T08:45:48.229743+0000","last_peered":"2024-09-19T08:45:48.229743+0000","last_clean":"2024-09-19T08:45:48.229743+0000","last_became_active":"2024-09-19T08:43:52.766388+0000","last_became_peered":"2024-09-19T08:43:52.766388+0000","last_unstale":"2024-09-19T08:45:48.229743+0000","last_undegraded":"2024-09-19T08:45:48.229743+0000","last_fullsized":"2024-09-19T08:45:48.229743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:34.182923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.967257+0000","last_change":"2024-09-19T08:43:52.765008+0000","last_active":"2024-09-19T08:45:47.967257+0000","last_peered":"2024-09-19T08:45:47.967257+0000","last_clean":"2024-09-19T08:45:47.967257+0000","last_became_active":"2024-09-19T08:43:52.764558+0000","last_became_peered":"2024-09-19T08:43:52.764558+0000","last_unstale":"2024-09-19T08:45:47.967257+0000","last_undegraded":"2024-09-19T08:45:47.967257+0000","last_fullsized":"2024-09-19T08:45:47.967257+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:52:42.756421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968722+0000","last_change":"2024-09-19T08:43:52.764376+0000","last_active":"2024-09-19T08:45:47.968722+0000","last_peered":"2024-09-19T08:45:47.968722+0000","last_clean":"2024-09-19T08:45:47.968722+0000","last_became_active":"2024-09-19T08:43:52.764048+0000","last_became_peered":"2024-09-19T08:43:52.764048+0000","last_unstale":"2024-09-19T08:45:47.968722+0000","last_undegraded":"2024-09-19T08:45:47.968722+0000","last_fullsized":"2024-09-19T08:45:47.968722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:56.732094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229417+0000","last_change":"2024-09-19T08:44:02.827925+0000","last_active":"2024-09-19T08:45:48.229417+0000","last_peered":"2024-09-19T08:45:48.229417+0000","last_clean":"2024-09-19T08:45:48.229417+0000","last_became_active":"2024-09-19T08:44:02.827269+0000","last_became_peered":"2024-09-19T08:44:02.827269+0000","last_unstale":"2024-09-19T08:45:48.229417+0000","last_undegraded":"2024-09-19T08:45:48.229417+0000","last_fullsized":"2024-09-19T08:45:48.229417+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:31:54.556786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.224157+0000","last_change":"2024-09-19T08:43:52.762190+0000","last_active":"2024-09-19T08:45:48.224157+0000","last_peered":"2024-09-19T08:45:48.224157+0000","last_clean":"2024-09-19T08:45:48.224157+0000","last_became_active":"2024-09-19T08:43:52.761818+0000","last_became_peered":"2024-09-19T08:43:52.761818+0000","last_unstale":"2024-09-19T08:45:48.224157+0000","last_undegraded":"2024-09-19T08:45:48.224157+0000","last_fullsized":"2024-09-19T08:45:48.224157+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:19:55.246779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221492+0000","last_change":"2024-09-19T08:43:52.761069+0000","last_active":"2024-09-19T08:45:48.221492+0000","last_peered":"2024-09-19T08:45:48.221492+0000","last_clean":"2024-09-19T08:45:48.221492+0000","last_became_active":"2024-09-19T08:43:52.760705+0000","last_became_peered":"2024-09-19T08:43:52.760705+0000","last_unstale":"2024-09-19T08:45:48.221492+0000","last_undegraded":"2024-09-19T08:45:48.221492+0000","last_fullsized":"2024-09-19T08:45:48.221492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:46:27.059026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065963+0000","last_change":"2024-09-19T08:43:52.773449+0000","last_active":"2024-09-19T08:45:48.065963+0000","last_peered":"2024-09-19T08:45:48.065963+0000","last_clean":"2024-09-19T08:45:48.065963+0000","last_became_active":"2024-09-19T08:43:52.772929+0000","last_became_peered":"2024-09-19T08:43:52.772929+0000","last_unstale":"2024-09-19T08:45:48.065963+0000","last_undegraded":"2024-09-19T08:45:48.065963+0000","last_fullsized":"2024-09-19T08:45:48.065963+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:00:38.929199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074018+0000","last_change":"2024-09-19T08:43:52.773885+0000","last_active":"2024-09-19T08:45:48.074018+0000","last_peered":"2024-09-19T08:45:48.074018+0000","last_clean":"2024-09-19T08:45:48.074018+0000","last_became_active":"2024-09-19T08:43:52.772856+0000","last_became_peered":"2024-09-19T08:43:52.772856+0000","last_unstale":"2024-09-19T08:45:48.074018+0000","last_undegraded":"2024-09-19T08:45:48.074018+0000","last_fullsized":"2024-09-19T08:45:48.074018+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:13:22.581335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.067279+0000","last_change":"2024-09-19T08:43:52.762909+0000","last_active":"2024-09-19T08:45:48.067279+0000","last_peered":"2024-09-19T08:45:48.067279+0000","last_clean":"2024-09-19T08:45:48.067279+0000","last_became_active":"2024-09-19T08:43:52.762600+0000","last_became_peered":"2024-09-19T08:43:52.762600+0000","last_unstale":"2024-09-19T08:45:48.067279+0000","last_undegraded":"2024-09-19T08:45:48.067279+0000","last_fullsized":"2024-09-19T08:45:48.067279+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:46.098294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968939+0000","last_change":"2024-09-19T08:43:52.762861+0000","last_active":"2024-09-19T08:45:47.968939+0000","last_peered":"2024-09-19T08:45:47.968939+0000","last_clean":"2024-09-19T08:45:47.968939+0000","last_became_active":"2024-09-19T08:43:52.762525+0000","last_became_peered":"2024-09-19T08:43:52.762525+0000","last_unstale":"2024-09-19T08:45:47.968939+0000","last_undegraded":"2024-09-19T08:45:47.968939+0000","last_fullsized":"2024-09-19T08:45:47.968939+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:54.717249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229552+0000","last_change":"2024-09-19T08:43:52.768370+0000","last_active":"2024-09-19T08:45:48.229552+0000","last_peered":"2024-09-19T08:45:48.229552+0000","last_clean":"2024-09-19T08:45:48.229552+0000","last_became_active":"2024-09-19T08:43:52.768055+0000","last_became_peered":"2024-09-19T08:43:52.768055+0000","last_unstale":"2024-09-19T08:45:48.229552+0000","last_undegraded":"2024-09-19T08:45:48.229552+0000","last_fullsized":"2024-09-19T08:45:48.229552+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:48.212862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.224066+0000","last_change":"2024-09-19T08:43:52.758541+0000","last_active":"2024-09-19T08:45:48.224066+0000","last_peered":"2024-09-19T08:45:48.224066+0000","last_clean":"2024-09-19T08:45:48.224066+0000","last_became_active":"2024-09-19T08:43:52.758122+0000","last_became_peered":"2024-09-19T08:43:52.758122+0000","last_unstale":"2024-09-19T08:45:48.224066+0000","last_undegraded":"2024-09-19T08:45:48.224066+0000","last_fullsized":"2024-09-19T08:45:48.224066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:54:40.681880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.231790+0000","last_change":"2024-09-19T08:43:52.758109+0000","last_active":"2024-09-19T08:45:48.231790+0000","last_peered":"2024-09-19T08:45:48.231790+0000","last_clean":"2024-09-19T08:45:48.231790+0000","last_became_active":"2024-09-19T08:43:52.757808+0000","last_became_peered":"2024-09-19T08:43:52.757808+0000","last_unstale":"2024-09-19T08:45:48.231790+0000","last_undegraded":"2024-09-19T08:45:48.231790+0000","last_fullsized":"2024-09-19T08:45:48.231790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:02:47.614562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968701+0000","last_change":"2024-09-19T08:43:52.772501+0000","last_active":"2024-09-19T08:45:47.968701+0000","last_peered":"2024-09-19T08:45:47.968701+0000","last_clean":"2024-09-19T08:45:47.968701+0000","last_became_active":"2024-09-19T08:43:52.771611+0000","last_became_peered":"2024-09-19T08:43:52.771611+0000","last_unstale":"2024-09-19T08:45:47.968701+0000","last_undegraded":"2024-09-19T08:45:47.968701+0000","last_fullsized":"2024-09-19T08:45:47.968701+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:49.537237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968630+0000","last_change":"2024-09-19T08:43:52.773664+0000","last_active":"2024-09-19T08:45:47.968630+0000","last_peered":"2024-09-19T08:45:47.968630+0000","last_clean":"2024-09-19T08:45:47.968630+0000","last_became_active":"2024-09-19T08:43:52.773014+0000","last_became_peered":"2024-09-19T08:43:52.773014+0000","last_unstale":"2024-09-19T08:45:47.968630+0000","last_undegraded":"2024-09-19T08:45:47.968630+0000","last_fullsized":"2024-09-19T08:45:47.968630+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:03.448825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229553+0000","last_change":"2024-09-19T08:43:52.769578+0000","last_active":"2024-09-19T08:45:48.229553+0000","last_peered":"2024-09-19T08:45:48.229553+0000","last_clean":"2024-09-19T08:45:48.229553+0000","last_became_active":"2024-09-19T08:43:52.769252+0000","last_became_peered":"2024-09-19T08:43:52.769252+0000","last_unstale":"2024-09-19T08:45:48.229553+0000","last_undegraded":"2024-09-19T08:45:48.229553+0000","last_fullsized":"2024-09-19T08:45:48.229553+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:27.936118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065844+0000","last_change":"2024-09-19T08:43:52.770453+0000","last_active":"2024-09-19T08:45:48.065844+0000","last_peered":"2024-09-19T08:45:48.065844+0000","last_clean":"2024-09-19T08:45:48.065844+0000","last_became_active":"2024-09-19T08:43:52.770155+0000","last_became_peered":"2024-09-19T08:43:52.770155+0000","last_unstale":"2024-09-19T08:45:48.065844+0000","last_undegraded":"2024-09-19T08:45:48.065844+0000","last_fullsized":"2024-09-19T08:45:48.065844+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:12.893572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'8","reported_seq":147,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:52.787924+0000","last_change":"2024-09-19T08:43:50.751316+0000","last_active":"2024-09-19T08:45:52.787924+0000","last_peered":"2024-09-19T08:45:52.787924+0000","last_clean":"2024-09-19T08:45:52.787924+0000","last_became_active":"2024-09-19T08:43:50.750972+0000","last_became_peered":"2024-09-19T08:43:50.750972+0000","last_unstale":"2024-09-19T08:45:52.787924+0000","last_undegraded":"2024-09-19T08:45:52.787924+0000","last_fullsized":"2024-09-19T08:45:52.787924+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:40.654729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1549,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074105+0000","last_change":"2024-09-19T08:43:52.773979+0000","last_active":"2024-09-19T08:45:48.074105+0000","last_peered":"2024-09-19T08:45:48.074105+0000","last_clean":"2024-09-19T08:45:48.074105+0000","last_became_active":"2024-09-19T08:43:52.773648+0000","last_became_peered":"2024-09-19T08:43:52.773648+0000","last_unstale":"2024-09-19T08:45:48.074105+0000","last_undegraded":"2024-09-19T08:45:48.074105+0000","last_fullsized":"2024-09-19T08:45:48.074105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:36.061209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.076810+0000","last_change":"2024-09-19T08:43:50.750704+0000","last_active":"2024-09-19T08:45:48.076810+0000","last_peered":"2024-09-19T08:45:48.076810+0000","last_clean":"2024-09-19T08:45:48.076810+0000","last_became_active":"2024-09-19T08:43:50.750399+0000","last_became_peered":"2024-09-19T08:43:50.750399+0000","last_unstale":"2024-09-19T08:45:48.076810+0000","last_undegraded":"2024-09-19T08:45:48.076810+0000","last_fullsized":"2024-09-19T08:45:48.076810+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:25.447320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.223615+0000","last_change":"2024-09-19T08:43:50.744919+0000","last_active":"2024-09-19T08:45:48.223615+0000","last_peered":"2024-09-19T08:45:48.223615+0000","last_clean":"2024-09-19T08:45:48.223615+0000","last_became_active":"2024-09-19T08:43:50.744604+0000","last_became_peered":"2024-09-19T08:43:50.744604+0000","last_unstale":"2024-09-19T08:45:48.223615+0000","last_undegraded":"2024-09-19T08:45:48.223615+0000","last_fullsized":"2024-09-19T08:45:48.223615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:25:06.410093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065913+0000","last_change":"2024-09-19T08:43:52.766779+0000","last_active":"2024-09-19T08:45:48.065913+0000","last_peered":"2024-09-19T08:45:48.065913+0000","last_clean":"2024-09-19T08:45:48.065913+0000","last_became_active":"2024-09-19T08:43:52.766484+0000","last_became_peered":"2024-09-19T08:43:52.766484+0000","last_unstale":"2024-09-19T08:45:48.065913+0000","last_undegraded":"2024-09-19T08:45:48.065913+0000","last_fullsized":"2024-09-19T08:45:48.065913+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:17:29.350778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074289+0000","last_change":"2024-09-19T08:43:50.751282+0000","last_active":"2024-09-19T08:45:48.074289+0000","last_peered":"2024-09-19T08:45:48.074289+0000","last_clean":"2024-09-19T08:45:48.074289+0000","last_became_active":"2024-09-19T08:43:50.750754+0000","last_became_peered":"2024-09-19T08:43:50.750754+0000","last_unstale":"2024-09-19T08:45:48.074289+0000","last_undegraded":"2024-09-19T08:45:48.074289+0000","last_fullsized":"2024-09-19T08:45:48.074289+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:11.661626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.967219+0000","last_change":"2024-09-19T08:43:52.756442+0000","last_active":"2024-09-19T08:45:47.967219+0000","last_peered":"2024-09-19T08:45:47.967219+0000","last_clean":"2024-09-19T08:45:47.967219+0000","last_became_active":"2024-09-19T08:43:52.756116+0000","last_became_peered":"2024-09-19T08:43:52.756116+0000","last_unstale":"2024-09-19T08:45:47.967219+0000","last_undegraded":"2024-09-19T08:45:47.967219+0000","last_fullsized":"2024-09-19T08:45:47.967219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:40.029272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'8","reported_seq":252,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:53.788457+0000","last_change":"2024-09-19T08:43:50.749637+0000","last_active":"2024-09-19T08:45:53.788457+0000","last_peered":"2024-09-19T08:45:53.788457+0000","last_clean":"2024-09-19T08:45:53.788457+0000","last_became_active":"2024-09-19T08:43:50.749307+0000","last_became_peered":"2024-09-19T08:43:50.749307+0000","last_unstale":"2024-09-19T08:45:53.788457+0000","last_undegraded":"2024-09-19T08:45:53.788457+0000","last_fullsized":"2024-09-19T08:45:53.788457+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:23:33.409671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":220,"num_read_kb":220,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.222016+0000","last_change":"2024-09-19T08:43:52.763592+0000","last_active":"2024-09-19T08:45:48.222016+0000","last_peered":"2024-09-19T08:45:48.222016+0000","last_clean":"2024-09-19T08:45:48.222016+0000","last_became_active":"2024-09-19T08:43:52.763146+0000","last_became_peered":"2024-09-19T08:43:52.763146+0000","last_unstale":"2024-09-19T08:45:48.222016+0000","last_undegraded":"2024-09-19T08:45:48.222016+0000","last_fullsized":"2024-09-19T08:45:48.222016+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:32.194465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221748+0000","last_change":"2024-09-19T08:43:50.749184+0000","last_active":"2024-09-19T08:45:48.221748+0000","last_peered":"2024-09-19T08:45:48.221748+0000","last_clean":"2024-09-19T08:45:48.221748+0000","last_became_active":"2024-09-19T08:43:50.748844+0000","last_became_peered":"2024-09-19T08:43:50.748844+0000","last_unstale":"2024-09-19T08:45:48.221748+0000","last_undegraded":"2024-09-19T08:45:48.221748+0000","last_fullsized":"2024-09-19T08:45:48.221748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:15.334759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.231209+0000","last_change":"2024-09-19T08:43:52.768696+0000","last_active":"2024-09-19T08:45:48.231209+0000","last_peered":"2024-09-19T08:45:48.231209+0000","last_clean":"2024-09-19T08:45:48.231209+0000","last_became_active":"2024-09-19T08:43:52.768374+0000","last_became_peered":"2024-09-19T08:43:52.768374+0000","last_unstale":"2024-09-19T08:45:48.231209+0000","last_undegraded":"2024-09-19T08:45:48.231209+0000","last_fullsized":"2024-09-19T08:45:48.231209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:34:39.343041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074342+0000","last_change":"2024-09-19T08:43:51.026270+0000","last_active":"2024-09-19T08:45:48.074342+0000","last_peered":"2024-09-19T08:45:48.074342+0000","last_clean":"2024-09-19T08:45:48.074342+0000","last_became_active":"2024-09-19T08:43:51.025823+0000","last_became_peered":"2024-09-19T08:43:51.025823+0000","last_unstale":"2024-09-19T08:45:48.074342+0000","last_undegraded":"2024-09-19T08:45:48.074342+0000","last_fullsized":"2024-09-19T08:45:48.074342+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:00.480438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221712+0000","last_change":"2024-09-19T08:43:52.759555+0000","last_active":"2024-09-19T08:45:48.221712+0000","last_peered":"2024-09-19T08:45:48.221712+0000","last_clean":"2024-09-19T08:45:48.221712+0000","last_became_active":"2024-09-19T08:43:52.759223+0000","last_became_peered":"2024-09-19T08:43:52.759223+0000","last_unstale":"2024-09-19T08:45:48.221712+0000","last_undegraded":"2024-09-19T08:45:48.221712+0000","last_fullsized":"2024-09-19T08:45:48.221712+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:57:06.451323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'36","reported_seq":293,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:53.788461+0000","last_change":"2024-09-19T08:43:50.751924+0000","last_active":"2024-09-19T08:45:53.788461+0000","last_peered":"2024-09-19T08:45:53.788461+0000","last_clean":"2024-09-19T08:45:53.788461+0000","last_became_active":"2024-09-19T08:43:50.751117+0000","last_became_peered":"2024-09-19T08:43:50.751117+0000","last_unstale":"2024-09-19T08:45:53.788461+0000","last_undegraded":"2024-09-19T08:45:53.788461+0000","last_fullsized":"2024-09-19T08:45:53.788461+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:33:57.196283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65927,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":233,"num_read_kb":181,"num_write":38,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066156+0000","last_change":"2024-09-19T08:43:52.762260+0000","last_active":"2024-09-19T08:45:48.066156+0000","last_peered":"2024-09-19T08:45:48.066156+0000","last_clean":"2024-09-19T08:45:48.066156+0000","last_became_active":"2024-09-19T08:43:52.761923+0000","last_became_peered":"2024-09-19T08:43:52.761923+0000","last_unstale":"2024-09-19T08:45:48.066156+0000","last_undegraded":"2024-09-19T08:45:48.066156+0000","last_fullsized":"2024-09-19T08:45:48.066156+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:39:39.503332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968758+0000","last_change":"2024-09-19T08:43:50.749745+0000","last_active":"2024-09-19T08:45:47.968758+0000","last_peered":"2024-09-19T08:45:47.968758+0000","last_clean":"2024-09-19T08:45:47.968758+0000","last_became_active":"2024-09-19T08:43:50.749073+0000","last_became_peered":"2024-09-19T08:43:50.749073+0000","last_unstale":"2024-09-19T08:45:47.968758+0000","last_undegraded":"2024-09-19T08:45:47.968758+0000","last_fullsized":"2024-09-19T08:45:47.968758+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:48:11.965841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.223734+0000","last_change":"2024-09-19T08:43:52.761495+0000","last_active":"2024-09-19T08:45:48.223734+0000","last_peered":"2024-09-19T08:45:48.223734+0000","last_clean":"2024-09-19T08:45:48.223734+0000","last_became_active":"2024-09-19T08:43:52.760425+0000","last_became_peered":"2024-09-19T08:43:52.760425+0000","last_unstale":"2024-09-19T08:45:48.223734+0000","last_undegraded":"2024-09-19T08:45:48.223734+0000","last_fullsized":"2024-09-19T08:45:48.223734+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:57.689284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074090+0000","last_change":"2024-09-19T08:43:50.749563+0000","last_active":"2024-09-19T08:45:48.074090+0000","last_peered":"2024-09-19T08:45:48.074090+0000","last_clean":"2024-09-19T08:45:48.074090+0000","last_became_active":"2024-09-19T08:43:50.749244+0000","last_became_peered":"2024-09-19T08:43:50.749244+0000","last_unstale":"2024-09-19T08:45:48.074090+0000","last_undegraded":"2024-09-19T08:45:48.074090+0000","last_fullsized":"2024-09-19T08:45:48.074090+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:59:02.312883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221429+0000","last_change":"2024-09-19T08:43:52.767087+0000","last_active":"2024-09-19T08:45:48.221429+0000","last_peered":"2024-09-19T08:45:48.221429+0000","last_clean":"2024-09-19T08:45:48.221429+0000","last_became_active":"2024-09-19T08:43:52.766730+0000","last_became_peered":"2024-09-19T08:43:52.766730+0000","last_unstale":"2024-09-19T08:45:48.221429+0000","last_undegraded":"2024-09-19T08:45:48.221429+0000","last_fullsized":"2024-09-19T08:45:48.221429+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:04.040896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:52.798017+0000","last_change":"2024-09-19T08:43:50.749659+0000","last_active":"2024-09-19T08:45:52.798017+0000","last_peered":"2024-09-19T08:45:52.798017+0000","last_clean":"2024-09-19T08:45:52.798017+0000","last_became_active":"2024-09-19T08:43:50.749314+0000","last_became_peered":"2024-09-19T08:43:50.749314+0000","last_unstale":"2024-09-19T08:45:52.798017+0000","last_undegraded":"2024-09-19T08:45:52.798017+0000","last_fullsized":"2024-09-19T08:45:52.798017+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:08.576209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221691+0000","last_change":"2024-09-19T08:43:52.767928+0000","last_active":"2024-09-19T08:45:48.221691+0000","last_peered":"2024-09-19T08:45:48.221691+0000","last_clean":"2024-09-19T08:45:48.221691+0000","last_became_active":"2024-09-19T08:43:52.767580+0000","last_became_peered":"2024-09-19T08:43:52.767580+0000","last_unstale":"2024-09-19T08:45:48.221691+0000","last_undegraded":"2024-09-19T08:45:48.221691+0000","last_fullsized":"2024-09-19T08:45:48.221691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:45.344693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229348+0000","last_change":"2024-09-19T08:44:02.833887+0000","last_active":"2024-09-19T08:45:48.229348+0000","last_peered":"2024-09-19T08:45:48.229348+0000","last_clean":"2024-09-19T08:45:48.229348+0000","last_became_active":"2024-09-19T08:44:02.821624+0000","last_became_peered":"2024-09-19T08:44:02.821624+0000","last_unstale":"2024-09-19T08:45:48.229348+0000","last_undegraded":"2024-09-19T08:45:48.229348+0000","last_fullsized":"2024-09-19T08:45:48.229348+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:54:36.131949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.967019+0000","last_change":"2024-09-19T08:44:03.080503+0000","last_active":"2024-09-19T08:45:47.967019+0000","last_peered":"2024-09-19T08:45:47.967019+0000","last_clean":"2024-09-19T08:45:47.967019+0000","last_became_active":"2024-09-19T08:44:03.080129+0000","last_became_peered":"2024-09-19T08:44:03.080129+0000","last_unstale":"2024-09-19T08:45:47.967019+0000","last_undegraded":"2024-09-19T08:45:47.967019+0000","last_fullsized":"2024-09-19T08:45:47.967019+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:01:37.034795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066192+0000","last_change":"2024-09-19T08:43:50.747624+0000","last_active":"2024-09-19T08:45:48.066192+0000","last_peered":"2024-09-19T08:45:48.066192+0000","last_clean":"2024-09-19T08:45:48.066192+0000","last_became_active":"2024-09-19T08:43:50.747263+0000","last_became_peered":"2024-09-19T08:43:50.747263+0000","last_unstale":"2024-09-19T08:45:48.066192+0000","last_undegraded":"2024-09-19T08:45:48.066192+0000","last_fullsized":"2024-09-19T08:45:48.066192+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:18.580106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229525+0000","last_change":"2024-09-19T08:43:52.770051+0000","last_active":"2024-09-19T08:45:48.229525+0000","last_peered":"2024-09-19T08:45:48.229525+0000","last_clean":"2024-09-19T08:45:48.229525+0000","last_became_active":"2024-09-19T08:43:52.769727+0000","last_became_peered":"2024-09-19T08:43:52.769727+0000","last_unstale":"2024-09-19T08:45:48.229525+0000","last_undegraded":"2024-09-19T08:45:48.229525+0000","last_fullsized":"2024-09-19T08:45:48.229525+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:34.954897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.223984+0000","last_change":"2024-09-19T08:43:50.746696+0000","last_active":"2024-09-19T08:45:48.223984+0000","last_peered":"2024-09-19T08:45:48.223984+0000","last_clean":"2024-09-19T08:45:48.223984+0000","last_became_active":"2024-09-19T08:43:50.746246+0000","last_became_peered":"2024-09-19T08:43:50.746246+0000","last_unstale":"2024-09-19T08:45:48.223984+0000","last_undegraded":"2024-09-19T08:45:48.223984+0000","last_fullsized":"2024-09-19T08:45:48.223984+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:38:39.935348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.222174+0000","last_change":"2024-09-19T08:43:52.765199+0000","last_active":"2024-09-19T08:45:48.222174+0000","last_peered":"2024-09-19T08:45:48.222174+0000","last_clean":"2024-09-19T08:45:48.222174+0000","last_became_active":"2024-09-19T08:43:52.764896+0000","last_became_peered":"2024-09-19T08:43:52.764896+0000","last_unstale":"2024-09-19T08:45:48.222174+0000","last_undegraded":"2024-09-19T08:45:48.222174+0000","last_fullsized":"2024-09-19T08:45:48.222174+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:02:59.570018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'41","reported_seq":73,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.186388+0000","last_change":"2024-09-19T08:43:50.747242+0000","last_active":"2024-09-19T08:45:48.186388+0000","last_peered":"2024-09-19T08:45:48.186388+0000","last_clean":"2024-09-19T08:45:48.186388+0000","last_became_active":"2024-09-19T08:43:50.746853+0000","last_became_peered":"2024-09-19T08:43:50.746853+0000","last_unstale":"2024-09-19T08:45:48.186388+0000","last_undegraded":"2024-09-19T08:45:48.186388+0000","last_fullsized":"2024-09-19T08:45:48.186388+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:21.545635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221579+0000","last_change":"2024-09-19T08:44:02.826535+0000","last_active":"2024-09-19T08:45:48.221579+0000","last_peered":"2024-09-19T08:45:48.221579+0000","last_clean":"2024-09-19T08:45:48.221579+0000","last_became_active":"2024-09-19T08:44:02.826113+0000","last_became_peered":"2024-09-19T08:44:02.826113+0000","last_unstale":"2024-09-19T08:45:48.221579+0000","last_undegraded":"2024-09-19T08:45:48.221579+0000","last_fullsized":"2024-09-19T08:45:48.221579+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:20.247732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'8","reported_seq":147,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:52.787737+0000","last_change":"2024-09-19T08:43:51.131540+0000","last_active":"2024-09-19T08:45:52.787737+0000","last_peered":"2024-09-19T08:45:52.787737+0000","last_clean":"2024-09-19T08:45:52.787737+0000","last_became_active":"2024-09-19T08:43:51.131170+0000","last_became_peered":"2024-09-19T08:43:51.131170+0000","last_unstale":"2024-09-19T08:45:52.787737+0000","last_undegraded":"2024-09-19T08:45:52.787737+0000","last_fullsized":"2024-09-19T08:45:52.787737+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:43.635612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":115,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221593+0000","last_change":"2024-09-19T08:43:52.763308+0000","last_active":"2024-09-19T08:45:48.221593+0000","last_peered":"2024-09-19T08:45:48.221593+0000","last_clean":"2024-09-19T08:45:48.221593+0000","last_became_active":"2024-09-19T08:43:52.762859+0000","last_became_peered":"2024-09-19T08:43:52.762859+0000","last_unstale":"2024-09-19T08:45:48.221593+0000","last_undegraded":"2024-09-19T08:45:48.221593+0000","last_fullsized":"2024-09-19T08:45:48.221593+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:32:35.083361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"83'148","reported_seq":394,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:50.085307+0000","last_change":"2024-09-19T08:42:39.567408+0000","last_active":"2024-09-19T08:45:50.085307+0000","last_peered":"2024-09-19T08:45:50.085307+0000","last_clean":"2024-09-19T08:45:50.085307+0000","last_became_active":"2024-09-19T08:42:39.566967+0000","last_became_peered":"2024-09-19T08:42:39.566967+0000","last_unstale":"2024-09-19T08:45:50.085307+0000","last_undegraded":"2024-09-19T08:45:50.085307+0000","last_fullsized":"2024-09-19T08:45:50.085307+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:14.287944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:14.287944+0000","last_clean_scrub_stamp":"2024-09-19T08:39:14.287944+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:31:03.245386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.966315+0000","last_change":"2024-09-19T08:43:50.751097+0000","last_active":"2024-09-19T08:45:47.966315+0000","last_peered":"2024-09-19T08:45:47.966315+0000","last_clean":"2024-09-19T08:45:47.966315+0000","last_became_active":"2024-09-19T08:43:50.750341+0000","last_became_peered":"2024-09-19T08:43:50.750341+0000","last_unstale":"2024-09-19T08:45:47.966315+0000","last_undegraded":"2024-09-19T08:45:47.966315+0000","last_fullsized":"2024-09-19T08:45:47.966315+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:25.694535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066373+0000","last_change":"2024-09-19T08:43:52.767905+0000","last_active":"2024-09-19T08:45:48.066373+0000","last_peered":"2024-09-19T08:45:48.066373+0000","last_clean":"2024-09-19T08:45:48.066373+0000","last_became_active":"2024-09-19T08:43:52.767601+0000","last_became_peered":"2024-09-19T08:43:52.767601+0000","last_unstale":"2024-09-19T08:45:48.066373+0000","last_undegraded":"2024-09-19T08:45:48.066373+0000","last_fullsized":"2024-09-19T08:45:48.066373+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:14.519879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221389+0000","last_change":"2024-09-19T08:43:50.751397+0000","last_active":"2024-09-19T08:45:48.221389+0000","last_peered":"2024-09-19T08:45:48.221389+0000","last_clean":"2024-09-19T08:45:48.221389+0000","last_became_active":"2024-09-19T08:43:50.751079+0000","last_became_peered":"2024-09-19T08:43:50.751079+0000","last_unstale":"2024-09-19T08:45:48.221389+0000","last_undegraded":"2024-09-19T08:45:48.221389+0000","last_fullsized":"2024-09-19T08:45:48.221389+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:29.904786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.223616+0000","last_change":"2024-09-19T08:43:52.773348+0000","last_active":"2024-09-19T08:45:48.223616+0000","last_peered":"2024-09-19T08:45:48.223616+0000","last_clean":"2024-09-19T08:45:48.223616+0000","last_became_active":"2024-09-19T08:43:52.773011+0000","last_became_peered":"2024-09-19T08:43:52.773011+0000","last_unstale":"2024-09-19T08:45:48.223616+0000","last_undegraded":"2024-09-19T08:45:48.223616+0000","last_fullsized":"2024-09-19T08:45:48.223616+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:10.806578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:53.798805+0000","last_change":"2024-09-19T08:43:51.025300+0000","last_active":"2024-09-19T08:45:53.798805+0000","last_peered":"2024-09-19T08:45:53.798805+0000","last_clean":"2024-09-19T08:45:53.798805+0000","last_became_active":"2024-09-19T08:43:51.024936+0000","last_became_peered":"2024-09-19T08:43:51.024936+0000","last_unstale":"2024-09-19T08:45:53.798805+0000","last_undegraded":"2024-09-19T08:45:53.798805+0000","last_fullsized":"2024-09-19T08:45:53.798805+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:03.186066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074063+0000","last_change":"2024-09-19T08:43:52.774962+0000","last_active":"2024-09-19T08:45:48.074063+0000","last_peered":"2024-09-19T08:45:48.074063+0000","last_clean":"2024-09-19T08:45:48.074063+0000","last_became_active":"2024-09-19T08:43:52.774598+0000","last_became_peered":"2024-09-19T08:43:52.774598+0000","last_unstale":"2024-09-19T08:45:48.074063+0000","last_undegraded":"2024-09-19T08:45:48.074063+0000","last_fullsized":"2024-09-19T08:45:48.074063+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:02.687136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'7","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:53.788873+0000","last_change":"2024-09-19T08:43:50.748180+0000","last_active":"2024-09-19T08:45:53.788873+0000","last_peered":"2024-09-19T08:45:53.788873+0000","last_clean":"2024-09-19T08:45:53.788873+0000","last_became_active":"2024-09-19T08:43:50.747853+0000","last_became_peered":"2024-09-19T08:43:50.747853+0000","last_unstale":"2024-09-19T08:45:53.788873+0000","last_undegraded":"2024-09-19T08:45:53.788873+0000","last_fullsized":"2024-09-19T08:45:53.788873+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:56.047259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.077066+0000","last_change":"2024-09-19T08:43:52.770253+0000","last_active":"2024-09-19T08:45:48.077066+0000","last_peered":"2024-09-19T08:45:48.077066+0000","last_clean":"2024-09-19T08:45:48.077066+0000","last_became_active":"2024-09-19T08:43:52.769951+0000","last_became_peered":"2024-09-19T08:43:52.769951+0000","last_unstale":"2024-09-19T08:45:48.077066+0000","last_undegraded":"2024-09-19T08:45:48.077066+0000","last_fullsized":"2024-09-19T08:45:48.077066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:02.353144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.967260+0000","last_change":"2024-09-19T08:43:51.026453+0000","last_active":"2024-09-19T08:45:47.967260+0000","last_peered":"2024-09-19T08:45:47.967260+0000","last_clean":"2024-09-19T08:45:47.967260+0000","last_became_active":"2024-09-19T08:43:51.025873+0000","last_became_peered":"2024-09-19T08:43:51.025873+0000","last_unstale":"2024-09-19T08:45:47.967260+0000","last_undegraded":"2024-09-19T08:45:47.967260+0000","last_fullsized":"2024-09-19T08:45:47.967260+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:25:17.475074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.076927+0000","last_change":"2024-09-19T08:43:52.761700+0000","last_active":"2024-09-19T08:45:48.076927+0000","last_peered":"2024-09-19T08:45:48.076927+0000","last_clean":"2024-09-19T08:45:48.076927+0000","last_became_active":"2024-09-19T08:43:52.755527+0000","last_became_peered":"2024-09-19T08:43:52.755527+0000","last_unstale":"2024-09-19T08:45:48.076927+0000","last_undegraded":"2024-09-19T08:45:48.076927+0000","last_fullsized":"2024-09-19T08:45:48.076927+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:18:43.171284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066299+0000","last_change":"2024-09-19T08:43:51.026136+0000","last_active":"2024-09-19T08:45:48.066299+0000","last_peered":"2024-09-19T08:45:48.066299+0000","last_clean":"2024-09-19T08:45:48.066299+0000","last_became_active":"2024-09-19T08:43:51.025740+0000","last_became_peered":"2024-09-19T08:43:51.025740+0000","last_unstale":"2024-09-19T08:45:48.066299+0000","last_undegraded":"2024-09-19T08:45:48.066299+0000","last_fullsized":"2024-09-19T08:45:48.066299+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:17:32.741083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.224195+0000","last_change":"2024-09-19T08:43:52.763808+0000","last_active":"2024-09-19T08:45:48.224195+0000","last_peered":"2024-09-19T08:45:48.224195+0000","last_clean":"2024-09-19T08:45:48.224195+0000","last_became_active":"2024-09-19T08:43:52.763509+0000","last_became_peered":"2024-09-19T08:43:52.763509+0000","last_unstale":"2024-09-19T08:45:48.224195+0000","last_undegraded":"2024-09-19T08:45:48.224195+0000","last_fullsized":"2024-09-19T08:45:48.224195+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:45:08.385419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'51","reported_seq":98,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.189380+0000","last_change":"2024-09-19T08:43:50.744213+0000","last_active":"2024-09-19T08:45:48.189380+0000","last_peered":"2024-09-19T08:45:48.189380+0000","last_clean":"2024-09-19T08:45:48.189380+0000","last_became_active":"2024-09-19T08:43:50.743848+0000","last_became_peered":"2024-09-19T08:43:50.743848+0000","last_unstale":"2024-09-19T08:45:48.189380+0000","last_undegraded":"2024-09-19T08:45:48.189380+0000","last_fullsized":"2024-09-19T08:45:48.189380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:26.613055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":90,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.067394+0000","last_change":"2024-09-19T08:43:52.767102+0000","last_active":"2024-09-19T08:45:48.067394+0000","last_peered":"2024-09-19T08:45:48.067394+0000","last_clean":"2024-09-19T08:45:48.067394+0000","last_became_active":"2024-09-19T08:43:52.766262+0000","last_became_peered":"2024-09-19T08:43:52.766262+0000","last_unstale":"2024-09-19T08:45:48.067394+0000","last_undegraded":"2024-09-19T08:45:48.067394+0000","last_fullsized":"2024-09-19T08:45:48.067394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:35.740462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.966479+0000","last_change":"2024-09-19T08:43:51.131742+0000","last_active":"2024-09-19T08:45:47.966479+0000","last_peered":"2024-09-19T08:45:47.966479+0000","last_clean":"2024-09-19T08:45:47.966479+0000","last_became_active":"2024-09-19T08:43:51.131388+0000","last_became_peered":"2024-09-19T08:43:51.131388+0000","last_unstale":"2024-09-19T08:45:47.966479+0000","last_undegraded":"2024-09-19T08:45:47.966479+0000","last_fullsized":"2024-09-19T08:45:47.966479+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:26.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.230731+0000","last_change":"2024-09-19T08:43:52.756004+0000","last_active":"2024-09-19T08:45:48.230731+0000","last_peered":"2024-09-19T08:45:48.230731+0000","last_clean":"2024-09-19T08:45:48.230731+0000","last_became_active":"2024-09-19T08:43:52.755596+0000","last_became_peered":"2024-09-19T08:43:52.755596+0000","last_unstale":"2024-09-19T08:45:48.230731+0000","last_undegraded":"2024-09-19T08:45:48.230731+0000","last_fullsized":"2024-09-19T08:45:48.230731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:35.942375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221620+0000","last_change":"2024-09-19T08:44:02.835986+0000","last_active":"2024-09-19T08:45:48.221620+0000","last_peered":"2024-09-19T08:45:48.221620+0000","last_clean":"2024-09-19T08:45:48.221620+0000","last_became_active":"2024-09-19T08:44:02.826314+0000","last_became_peered":"2024-09-19T08:44:02.826314+0000","last_unstale":"2024-09-19T08:45:48.221620+0000","last_undegraded":"2024-09-19T08:45:48.221620+0000","last_fullsized":"2024-09-19T08:45:48.221620+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:16.943989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065838+0000","last_change":"2024-09-19T08:43:52.766582+0000","last_active":"2024-09-19T08:45:48.065838+0000","last_peered":"2024-09-19T08:45:48.065838+0000","last_clean":"2024-09-19T08:45:48.065838+0000","last_became_active":"2024-09-19T08:43:52.766232+0000","last_became_peered":"2024-09-19T08:43:52.766232+0000","last_unstale":"2024-09-19T08:45:48.065838+0000","last_undegraded":"2024-09-19T08:45:48.065838+0000","last_fullsized":"2024-09-19T08:45:48.065838+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:43:42.319422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066195+0000","last_change":"2024-09-19T08:43:50.747611+0000","last_active":"2024-09-19T08:45:48.066195+0000","last_peered":"2024-09-19T08:45:48.066195+0000","last_clean":"2024-09-19T08:45:48.066195+0000","last_became_active":"2024-09-19T08:43:50.747263+0000","last_became_peered":"2024-09-19T08:43:50.747263+0000","last_unstale":"2024-09-19T08:45:48.066195+0000","last_undegraded":"2024-09-19T08:45:48.066195+0000","last_fullsized":"2024-09-19T08:45:48.066195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:14.717820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229532+0000","last_change":"2024-09-19T08:43:52.766493+0000","last_active":"2024-09-19T08:45:48.229532+0000","last_peered":"2024-09-19T08:45:48.229532+0000","last_clean":"2024-09-19T08:45:48.229532+0000","last_became_active":"2024-09-19T08:43:52.766158+0000","last_became_peered":"2024-09-19T08:43:52.766158+0000","last_unstale":"2024-09-19T08:45:48.229532+0000","last_undegraded":"2024-09-19T08:45:48.229532+0000","last_fullsized":"2024-09-19T08:45:48.229532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:59.175949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968841+0000","last_change":"2024-09-19T08:43:50.751368+0000","last_active":"2024-09-19T08:45:47.968841+0000","last_peered":"2024-09-19T08:45:47.968841+0000","last_clean":"2024-09-19T08:45:47.968841+0000","last_became_active":"2024-09-19T08:43:50.751038+0000","last_became_peered":"2024-09-19T08:43:50.751038+0000","last_unstale":"2024-09-19T08:45:47.968841+0000","last_undegraded":"2024-09-19T08:45:47.968841+0000","last_fullsized":"2024-09-19T08:45:47.968841+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:46.302996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066722+0000","last_change":"2024-09-19T08:43:52.756998+0000","last_active":"2024-09-19T08:45:48.066722+0000","last_peered":"2024-09-19T08:45:48.066722+0000","last_clean":"2024-09-19T08:45:48.066722+0000","last_became_active":"2024-09-19T08:43:52.756661+0000","last_became_peered":"2024-09-19T08:43:52.756661+0000","last_unstale":"2024-09-19T08:45:48.066722+0000","last_undegraded":"2024-09-19T08:45:48.066722+0000","last_fullsized":"2024-09-19T08:45:48.066722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:15.182574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065908+0000","last_change":"2024-09-19T08:43:50.750095+0000","last_active":"2024-09-19T08:45:48.065908+0000","last_peered":"2024-09-19T08:45:48.065908+0000","last_clean":"2024-09-19T08:45:48.065908+0000","last_became_active":"2024-09-19T08:43:50.749672+0000","last_became_peered":"2024-09-19T08:43:50.749672+0000","last_unstale":"2024-09-19T08:45:48.065908+0000","last_undegraded":"2024-09-19T08:45:48.065908+0000","last_fullsized":"2024-09-19T08:45:48.065908+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:25:34.978094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221996+0000","last_change":"2024-09-19T08:43:52.764097+0000","last_active":"2024-09-19T08:45:48.221996+0000","last_peered":"2024-09-19T08:45:48.221996+0000","last_clean":"2024-09-19T08:45:48.221996+0000","last_became_active":"2024-09-19T08:43:52.763773+0000","last_became_peered":"2024-09-19T08:43:52.763773+0000","last_unstale":"2024-09-19T08:45:48.221996+0000","last_undegraded":"2024-09-19T08:45:48.221996+0000","last_fullsized":"2024-09-19T08:45:48.221996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:01:42.098460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074171+0000","last_change":"2024-09-19T08:44:02.825238+0000","last_active":"2024-09-19T08:45:48.074171+0000","last_peered":"2024-09-19T08:45:48.074171+0000","last_clean":"2024-09-19T08:45:48.074171+0000","last_became_active":"2024-09-19T08:44:02.824873+0000","last_became_peered":"2024-09-19T08:44:02.824873+0000","last_unstale":"2024-09-19T08:45:48.074171+0000","last_undegraded":"2024-09-19T08:45:48.074171+0000","last_fullsized":"2024-09-19T08:45:48.074171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:38.896082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066088+0000","last_change":"2024-09-19T08:43:50.748789+0000","last_active":"2024-09-19T08:45:48.066088+0000","last_peered":"2024-09-19T08:45:48.066088+0000","last_clean":"2024-09-19T08:45:48.066088+0000","last_became_active":"2024-09-19T08:43:50.748464+0000","last_became_peered":"2024-09-19T08:43:50.748464+0000","last_unstale":"2024-09-19T08:45:48.066088+0000","last_undegraded":"2024-09-19T08:45:48.066088+0000","last_fullsized":"2024-09-19T08:45:48.066088+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:36:45.905639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221312+0000","last_change":"2024-09-19T08:43:52.756608+0000","last_active":"2024-09-19T08:45:48.221312+0000","last_peered":"2024-09-19T08:45:48.221312+0000","last_clean":"2024-09-19T08:45:48.221312+0000","last_became_active":"2024-09-19T08:43:52.756227+0000","last_became_peered":"2024-09-19T08:43:52.756227+0000","last_unstale":"2024-09-19T08:45:48.221312+0000","last_undegraded":"2024-09-19T08:45:48.221312+0000","last_fullsized":"2024-09-19T08:45:48.221312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:29:58.085584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'13","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066253+0000","last_change":"2024-09-19T08:43:50.749135+0000","last_active":"2024-09-19T08:45:48.066253+0000","last_peered":"2024-09-19T08:45:48.066253+0000","last_clean":"2024-09-19T08:45:48.066253+0000","last_became_active":"2024-09-19T08:43:50.748755+0000","last_became_peered":"2024-09-19T08:43:50.748755+0000","last_unstale":"2024-09-19T08:45:48.066253+0000","last_undegraded":"2024-09-19T08:45:48.066253+0000","last_fullsized":"2024-09-19T08:45:48.066253+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:55.079780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.231077+0000","last_change":"2024-09-19T08:43:52.763277+0000","last_active":"2024-09-19T08:45:48.231077+0000","last_peered":"2024-09-19T08:45:48.231077+0000","last_clean":"2024-09-19T08:45:48.231077+0000","last_became_active":"2024-09-19T08:43:52.762941+0000","last_became_peered":"2024-09-19T08:43:52.762941+0000","last_unstale":"2024-09-19T08:45:48.231077+0000","last_undegraded":"2024-09-19T08:45:48.231077+0000","last_fullsized":"2024-09-19T08:45:48.231077+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:35.531946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.067452+0000","last_change":"2024-09-19T08:43:51.025350+0000","last_active":"2024-09-19T08:45:48.067452+0000","last_peered":"2024-09-19T08:45:48.067452+0000","last_clean":"2024-09-19T08:45:48.067452+0000","last_became_active":"2024-09-19T08:43:51.025046+0000","last_became_peered":"2024-09-19T08:43:51.025046+0000","last_unstale":"2024-09-19T08:45:48.067452+0000","last_undegraded":"2024-09-19T08:45:48.067452+0000","last_fullsized":"2024-09-19T08:45:48.067452+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:27.595545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066292+0000","last_change":"2024-09-19T08:43:52.767917+0000","last_active":"2024-09-19T08:45:48.066292+0000","last_peered":"2024-09-19T08:45:48.066292+0000","last_clean":"2024-09-19T08:45:48.066292+0000","last_became_active":"2024-09-19T08:43:52.767575+0000","last_became_peered":"2024-09-19T08:43:52.767575+0000","last_unstale":"2024-09-19T08:45:48.066292+0000","last_undegraded":"2024-09-19T08:45:48.066292+0000","last_fullsized":"2024-09-19T08:45:48.066292+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:05.950399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:53.788467+0000","last_change":"2024-09-19T08:43:50.746400+0000","last_active":"2024-09-19T08:45:53.788467+0000","last_peered":"2024-09-19T08:45:53.788467+0000","last_clean":"2024-09-19T08:45:53.788467+0000","last_became_active":"2024-09-19T08:43:50.745891+0000","last_became_peered":"2024-09-19T08:43:50.745891+0000","last_unstale":"2024-09-19T08:45:53.788467+0000","last_undegraded":"2024-09-19T08:45:53.788467+0000","last_fullsized":"2024-09-19T08:45:53.788467+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:03.098202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599805,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1240,"num_read_kb":736,"num_write":374,"num_write_kb":1464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":712704,"data_stored":1799415,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":301,"ondisk_log_size":301,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":109335,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579942,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":446056,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808875,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1089536,"data_stored":1233573,"data_compressed":45500,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005103,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":962560,"data_stored":838280,"data_compressed":22045,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234036,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":946176,"data_stored":838429,"data_compressed":22045,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462969,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":983040,"data_stored":1039165,"data_compressed":39344,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593791,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":445450,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822723,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1015808,"data_stored":1169657,"data_compressed":45500,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986248,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":444525,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215181,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":969188,"data_compressed":28201,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1003520,"data_stored":513578,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":983040,"data_stored":512290,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":937984,"data_stored":446930,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":68465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":69753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135347,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":199263,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:45:59.672 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T08:45:59.919 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:46:00.337 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:46:00 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:46:00.067+0000 7fafe4e0c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:00.337 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:46:00 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:46:00.068+0000 7fafe4e0c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:00.337 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:46:00 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:46:00.068+0000 7fafe4e0c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:00.337 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:46:00 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:46:00.068+0000 7fafe4e0c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:00.337 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:46:00 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:46:00.068+0000 7fafe4e0c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:00.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.419+0000 7f3ce5e59640 1 -- 172.21.15.164:0/2481872525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ce0106f10 msgr2=0x7f3ce010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:00.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.419+0000 7f3ce5e59640 1 --2- 172.21.15.164:0/2481872525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ce0106f10 0x7f3ce010ea90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8009ca0 tx=0x7f3cd802a8c0 comp rx=0 tx=0).stop 2024-09-19T08:46:00.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.419+0000 7f3ce5e59640 1 -- 172.21.15.164:0/2481872525 shutdown_connections 2024-09-19T08:46:00.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.419+0000 7f3ce5e59640 1 --2- 172.21.15.164:0/2481872525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ce0106f10 0x7f3ce010ea90 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:00.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.419+0000 7f3ce5e59640 1 --2- 172.21.15.164:0/2481872525 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ce0102790 0x7f3ce01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:00.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.419+0000 7f3ce5e59640 1 --2- 172.21.15.164:0/2481872525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ce0101de0 0x7f3ce01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:00.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.419+0000 7f3ce5e59640 1 -- 172.21.15.164:0/2481872525 >> 172.21.15.164:0/2481872525 conn(0x7f3ce00fd880 msgr2=0x7f3ce00ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:00.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.419+0000 7f3ce5e59640 1 -- 172.21.15.164:0/2481872525 shutdown_connections 2024-09-19T08:46:00.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.419+0000 7f3ce5e59640 1 -- 172.21.15.164:0/2481872525 wait complete. 2024-09-19T08:46:00.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.420+0000 7f3ce5e59640 1 Processor -- start 2024-09-19T08:46:00.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.420+0000 7f3ce5e59640 1 -- start start 2024-09-19T08:46:00.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.420+0000 7f3ce5e59640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ce0101de0 0x7f3ce01983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:00.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.420+0000 7f3ce5e59640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ce0102790 0x7f3ce01988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:00.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.420+0000 7f3ce5e59640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ce0106f10 0x7f3ce019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:00.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.420+0000 7f3ce5e59640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce0111f70 con 0x7f3ce0102790 2024-09-19T08:46:00.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.420+0000 7f3ce5e59640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce0111df0 con 0x7f3ce0101de0 2024-09-19T08:46:00.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.420+0000 7f3ce5e59640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce01120f0 con 0x7f3ce0106f10 2024-09-19T08:46:00.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.420+0000 7f3cdeffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ce0102790 0x7f3ce01988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:00.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.421+0000 7f3cdeffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ce0102790 0x7f3ce01988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39460/0 (socket says 172.21.15.164:39460) 2024-09-19T08:46:00.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.421+0000 7f3cdeffd640 1 -- 172.21.15.164:0/1214500416 learned_addr learned my addr 172.21.15.164:0/1214500416 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:00.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.421+0000 7f3cdffff640 1 --2- 172.21.15.164:0/1214500416 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ce0106f10 0x7f3ce019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:00.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.421+0000 7f3cdf7fe640 1 --2- 172.21.15.164:0/1214500416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ce0101de0 0x7f3ce01983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:00.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.421+0000 7f3cdeffd640 1 -- 172.21.15.164:0/1214500416 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ce0106f10 msgr2=0x7f3ce019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:00.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.421+0000 7f3cdeffd640 1 --2- 172.21.15.164:0/1214500416 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ce0106f10 0x7f3ce019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:00.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.421+0000 7f3cdeffd640 1 -- 172.21.15.164:0/1214500416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ce0101de0 msgr2=0x7f3ce01983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:00.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.421+0000 7f3cdeffd640 1 --2- 172.21.15.164:0/1214500416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ce0101de0 0x7f3ce01983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:00.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.421+0000 7f3cdeffd640 1 -- 172.21.15.164:0/1214500416 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce019d360 con 0x7f3ce0102790 2024-09-19T08:46:00.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.421+0000 7f3cdf7fe640 1 --2- 172.21.15.164:0/1214500416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ce0101de0 0x7f3ce01983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:00.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.421+0000 7f3cdeffd640 1 --2- 172.21.15.164:0/1214500416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ce0102790 0x7f3ce01988f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3cd400d950 tx=0x7f3cd400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:00.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.421+0000 7f3cdcff9640 1 -- 172.21.15.164:0/1214500416 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4017030 con 0x7f3ce0102790 2024-09-19T08:46:00.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.421+0000 7f3ce5e59640 1 -- 172.21.15.164:0/1214500416 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ce019d650 con 0x7f3ce0102790 2024-09-19T08:46:00.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.421+0000 7f3cdcff9640 1 -- 172.21.15.164:0/1214500416 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cd4004020 con 0x7f3ce0102790 2024-09-19T08:46:00.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.421+0000 7f3cdcff9640 1 -- 172.21.15.164:0/1214500416 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd40047a0 con 0x7f3ce0102790 2024-09-19T08:46:00.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.421+0000 7f3ce5e59640 1 -- 172.21.15.164:0/1214500416 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ce01a4f30 con 0x7f3ce0102790 2024-09-19T08:46:00.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.422+0000 7f3ce5e59640 1 -- 172.21.15.164:0/1214500416 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ce01021c0 con 0x7f3ce0102790 2024-09-19T08:46:00.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.423+0000 7f3cdcff9640 1 -- 172.21.15.164:0/1214500416 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f3cd40041d0 con 0x7f3ce0102790 2024-09-19T08:46:00.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.424+0000 7f3cdcff9640 1 --2- 172.21.15.164:0/1214500416 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3cc80a3440 0x7f3cc80a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:00.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.424+0000 7f3cdf7fe640 1 --2- 172.21.15.164:0/1214500416 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3cc80a3440 0x7f3cc80a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:00.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.424+0000 7f3cdcff9640 1 -- 172.21.15.164:0/1214500416 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f3cd40cf7d0 con 0x7f3ce0102790 2024-09-19T08:46:00.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.425+0000 7f3cdf7fe640 1 --2- 172.21.15.164:0/1214500416 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3cc80a3440 0x7f3cc80a5900 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00058c0 tx=0x7f3cd0005850 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:00.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.428+0000 7f3cdcff9640 1 -- 172.21.15.164:0/1214500416 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3cd409c0b0 con 0x7f3ce0102790 2024-09-19T08:46:00.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.597+0000 7f3ce5e59640 1 -- 172.21.15.164:0/1214500416 --> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3ce0062630 con 0x7f3cc80a3440 2024-09-19T08:46:00.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.603+0000 7f3cdcff9640 1 -- 172.21.15.164:0/1214500416 <== mgr.15201 v2:172.21.15.164:6800/173486514 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258075 (secure 0 0 0) 0x7f3ce0062630 con 0x7f3cc80a3440 2024-09-19T08:46:00.606 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:46:00.607 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-09-19T08:46:00.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.607+0000 7f3ce5e59640 1 -- 172.21.15.164:0/1214500416 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3cc80a3440 msgr2=0x7f3cc80a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:00.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.607+0000 7f3ce5e59640 1 --2- 172.21.15.164:0/1214500416 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3cc80a3440 0x7f3cc80a5900 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00058c0 tx=0x7f3cd0005850 comp rx=0 tx=0).stop 2024-09-19T08:46:00.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.607+0000 7f3ce5e59640 1 -- 172.21.15.164:0/1214500416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ce0102790 msgr2=0x7f3ce01988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:00.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.607+0000 7f3ce5e59640 1 --2- 172.21.15.164:0/1214500416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ce0102790 0x7f3ce01988f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3cd400d950 tx=0x7f3cd400de10 comp rx=0 tx=0).stop 2024-09-19T08:46:00.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.607+0000 7f3ce5e59640 1 -- 172.21.15.164:0/1214500416 shutdown_connections 2024-09-19T08:46:00.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.607+0000 7f3ce5e59640 1 --2- 172.21.15.164:0/1214500416 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ce0106f10 0x7f3ce019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:00.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.607+0000 7f3ce5e59640 1 --2- 172.21.15.164:0/1214500416 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3cc80a3440 0x7f3cc80a5900 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:00.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.607+0000 7f3ce5e59640 1 --2- 172.21.15.164:0/1214500416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ce0102790 0x7f3ce01988f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:00.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.607+0000 7f3ce5e59640 1 --2- 172.21.15.164:0/1214500416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ce0101de0 0x7f3ce01983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:00.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.607+0000 7f3ce5e59640 1 -- 172.21.15.164:0/1214500416 >> 172.21.15.164:0/1214500416 conn(0x7f3ce00fd880 msgr2=0x7f3ce00feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:00.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.607+0000 7f3ce5e59640 1 -- 172.21.15.164:0/1214500416 shutdown_connections 2024-09-19T08:46:00.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:00.607+0000 7f3ce5e59640 1 -- 172.21.15.164:0/1214500416 wait complete. 2024-09-19T08:46:00.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:00 smithi173 ceph-mon[35306]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:46:01.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:00 smithi164 ceph-mon[30928]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:46:01.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:00 smithi170 ceph-mon[35072]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:46:01.471 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-09-19T08:45:59.983584+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1388177,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1783,"num_read_kb":1325,"num_write":704,"num_write_kb":5320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1031,"ondisk_log_size":1031,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":389868,"kb_used_data":11460,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124551956,"statfs":{"total":1151940427776,"available":1151541202944,"internally_reserved":0,"allocated":11735040,"data_stored":9490431,"data_compressed":247596,"data_compressed_allocated":319488,"data_compressed_original":3969024,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":197770,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":457,"num_read_kb":495,"num_write":147,"num_write_kb":916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.993496"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229741+0000","last_change":"2024-09-19T08:43:52.766709+0000","last_active":"2024-09-19T08:45:48.229741+0000","last_peered":"2024-09-19T08:45:48.229741+0000","last_clean":"2024-09-19T08:45:48.229741+0000","last_became_active":"2024-09-19T08:43:52.765970+0000","last_became_peered":"2024-09-19T08:43:52.765970+0000","last_unstale":"2024-09-19T08:45:48.229741+0000","last_undegraded":"2024-09-19T08:45:48.229741+0000","last_fullsized":"2024-09-19T08:45:48.229741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:12.176870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.230652+0000","last_change":"2024-09-19T08:43:52.766590+0000","last_active":"2024-09-19T08:45:48.230652+0000","last_peered":"2024-09-19T08:45:48.230652+0000","last_clean":"2024-09-19T08:45:48.230652+0000","last_became_active":"2024-09-19T08:43:52.766255+0000","last_became_peered":"2024-09-19T08:43:52.766255+0000","last_unstale":"2024-09-19T08:45:48.230652+0000","last_undegraded":"2024-09-19T08:45:48.230652+0000","last_fullsized":"2024-09-19T08:45:48.230652+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:15:41.359806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.230354+0000","last_change":"2024-09-19T08:43:52.761956+0000","last_active":"2024-09-19T08:45:48.230354+0000","last_peered":"2024-09-19T08:45:48.230354+0000","last_clean":"2024-09-19T08:45:48.230354+0000","last_became_active":"2024-09-19T08:43:52.761644+0000","last_became_peered":"2024-09-19T08:43:52.761644+0000","last_unstale":"2024-09-19T08:45:48.230354+0000","last_undegraded":"2024-09-19T08:45:48.230354+0000","last_fullsized":"2024-09-19T08:45:48.230354+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:13.770361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.966221+0000","last_change":"2024-09-19T08:43:52.764131+0000","last_active":"2024-09-19T08:45:47.966221+0000","last_peered":"2024-09-19T08:45:47.966221+0000","last_clean":"2024-09-19T08:45:47.966221+0000","last_became_active":"2024-09-19T08:43:52.763812+0000","last_became_peered":"2024-09-19T08:43:52.763812+0000","last_unstale":"2024-09-19T08:45:47.966221+0000","last_undegraded":"2024-09-19T08:45:47.966221+0000","last_fullsized":"2024-09-19T08:45:47.966221+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:32.272157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065874+0000","last_change":"2024-09-19T08:43:52.754673+0000","last_active":"2024-09-19T08:45:48.065874+0000","last_peered":"2024-09-19T08:45:48.065874+0000","last_clean":"2024-09-19T08:45:48.065874+0000","last_became_active":"2024-09-19T08:43:52.754144+0000","last_became_peered":"2024-09-19T08:43:52.754144+0000","last_unstale":"2024-09-19T08:45:48.065874+0000","last_undegraded":"2024-09-19T08:45:48.065874+0000","last_fullsized":"2024-09-19T08:45:48.065874+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:16.069260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.076773+0000","last_change":"2024-09-19T08:43:52.758939+0000","last_active":"2024-09-19T08:45:48.076773+0000","last_peered":"2024-09-19T08:45:48.076773+0000","last_clean":"2024-09-19T08:45:48.076773+0000","last_became_active":"2024-09-19T08:43:52.758611+0000","last_became_peered":"2024-09-19T08:43:52.758611+0000","last_unstale":"2024-09-19T08:45:48.076773+0000","last_undegraded":"2024-09-19T08:45:48.076773+0000","last_fullsized":"2024-09-19T08:45:48.076773+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:35:56.463545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221260+0000","last_change":"2024-09-19T08:43:52.770261+0000","last_active":"2024-09-19T08:45:48.221260+0000","last_peered":"2024-09-19T08:45:48.221260+0000","last_clean":"2024-09-19T08:45:48.221260+0000","last_became_active":"2024-09-19T08:43:52.768253+0000","last_became_peered":"2024-09-19T08:43:52.768253+0000","last_unstale":"2024-09-19T08:45:48.221260+0000","last_undegraded":"2024-09-19T08:45:48.221260+0000","last_fullsized":"2024-09-19T08:45:48.221260+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:40.756988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065907+0000","last_change":"2024-09-19T08:43:52.768564+0000","last_active":"2024-09-19T08:45:48.065907+0000","last_peered":"2024-09-19T08:45:48.065907+0000","last_clean":"2024-09-19T08:45:48.065907+0000","last_became_active":"2024-09-19T08:43:52.768260+0000","last_became_peered":"2024-09-19T08:43:52.768260+0000","last_unstale":"2024-09-19T08:45:48.065907+0000","last_undegraded":"2024-09-19T08:45:48.065907+0000","last_fullsized":"2024-09-19T08:45:48.065907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:16.528115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.222053+0000","last_change":"2024-09-19T08:43:52.769804+0000","last_active":"2024-09-19T08:45:48.222053+0000","last_peered":"2024-09-19T08:45:48.222053+0000","last_clean":"2024-09-19T08:45:48.222053+0000","last_became_active":"2024-09-19T08:43:52.769504+0000","last_became_peered":"2024-09-19T08:43:52.769504+0000","last_unstale":"2024-09-19T08:45:48.222053+0000","last_undegraded":"2024-09-19T08:45:48.222053+0000","last_fullsized":"2024-09-19T08:45:48.222053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:22:38.514410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.077170+0000","last_change":"2024-09-19T08:43:52.763278+0000","last_active":"2024-09-19T08:45:48.077170+0000","last_peered":"2024-09-19T08:45:48.077170+0000","last_clean":"2024-09-19T08:45:48.077170+0000","last_became_active":"2024-09-19T08:43:52.762985+0000","last_became_peered":"2024-09-19T08:43:52.762985+0000","last_unstale":"2024-09-19T08:45:48.077170+0000","last_undegraded":"2024-09-19T08:45:48.077170+0000","last_fullsized":"2024-09-19T08:45:48.077170+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:30.048782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221488+0000","last_change":"2024-09-19T08:43:52.770864+0000","last_active":"2024-09-19T08:45:48.221488+0000","last_peered":"2024-09-19T08:45:48.221488+0000","last_clean":"2024-09-19T08:45:48.221488+0000","last_became_active":"2024-09-19T08:43:52.770518+0000","last_became_peered":"2024-09-19T08:43:52.770518+0000","last_unstale":"2024-09-19T08:45:48.221488+0000","last_undegraded":"2024-09-19T08:45:48.221488+0000","last_fullsized":"2024-09-19T08:45:48.221488+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:08.931452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.966171+0000","last_change":"2024-09-19T08:43:52.757464+0000","last_active":"2024-09-19T08:45:47.966171+0000","last_peered":"2024-09-19T08:45:47.966171+0000","last_clean":"2024-09-19T08:45:47.966171+0000","last_became_active":"2024-09-19T08:43:52.757101+0000","last_became_peered":"2024-09-19T08:43:52.757101+0000","last_unstale":"2024-09-19T08:45:47.966171+0000","last_undegraded":"2024-09-19T08:45:47.966171+0000","last_fullsized":"2024-09-19T08:45:47.966171+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:17.395735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.224010+0000","last_change":"2024-09-19T08:43:52.767063+0000","last_active":"2024-09-19T08:45:48.224010+0000","last_peered":"2024-09-19T08:45:48.224010+0000","last_clean":"2024-09-19T08:45:48.224010+0000","last_became_active":"2024-09-19T08:43:52.766755+0000","last_became_peered":"2024-09-19T08:43:52.766755+0000","last_unstale":"2024-09-19T08:45:48.224010+0000","last_undegraded":"2024-09-19T08:45:48.224010+0000","last_fullsized":"2024-09-19T08:45:48.224010+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:30.530937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.230150+0000","last_change":"2024-09-19T08:43:52.766515+0000","last_active":"2024-09-19T08:45:48.230150+0000","last_peered":"2024-09-19T08:45:48.230150+0000","last_clean":"2024-09-19T08:45:48.230150+0000","last_became_active":"2024-09-19T08:43:52.765847+0000","last_became_peered":"2024-09-19T08:43:52.765847+0000","last_unstale":"2024-09-19T08:45:48.230150+0000","last_undegraded":"2024-09-19T08:45:48.230150+0000","last_fullsized":"2024-09-19T08:45:48.230150+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:50:35.948663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229847+0000","last_change":"2024-09-19T08:43:52.773702+0000","last_active":"2024-09-19T08:45:48.229847+0000","last_peered":"2024-09-19T08:45:48.229847+0000","last_clean":"2024-09-19T08:45:48.229847+0000","last_became_active":"2024-09-19T08:43:52.773324+0000","last_became_peered":"2024-09-19T08:43:52.773324+0000","last_unstale":"2024-09-19T08:45:48.229847+0000","last_undegraded":"2024-09-19T08:45:48.229847+0000","last_fullsized":"2024-09-19T08:45:48.229847+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:34.368014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221253+0000","last_change":"2024-09-19T08:43:52.762745+0000","last_active":"2024-09-19T08:45:48.221253+0000","last_peered":"2024-09-19T08:45:48.221253+0000","last_clean":"2024-09-19T08:45:48.221253+0000","last_became_active":"2024-09-19T08:43:52.762172+0000","last_became_peered":"2024-09-19T08:43:52.762172+0000","last_unstale":"2024-09-19T08:45:48.221253+0000","last_undegraded":"2024-09-19T08:45:48.221253+0000","last_fullsized":"2024-09-19T08:45:48.221253+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:57.369775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229743+0000","last_change":"2024-09-19T08:43:52.767118+0000","last_active":"2024-09-19T08:45:48.229743+0000","last_peered":"2024-09-19T08:45:48.229743+0000","last_clean":"2024-09-19T08:45:48.229743+0000","last_became_active":"2024-09-19T08:43:52.766388+0000","last_became_peered":"2024-09-19T08:43:52.766388+0000","last_unstale":"2024-09-19T08:45:48.229743+0000","last_undegraded":"2024-09-19T08:45:48.229743+0000","last_fullsized":"2024-09-19T08:45:48.229743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:34.182923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.967257+0000","last_change":"2024-09-19T08:43:52.765008+0000","last_active":"2024-09-19T08:45:47.967257+0000","last_peered":"2024-09-19T08:45:47.967257+0000","last_clean":"2024-09-19T08:45:47.967257+0000","last_became_active":"2024-09-19T08:43:52.764558+0000","last_became_peered":"2024-09-19T08:43:52.764558+0000","last_unstale":"2024-09-19T08:45:47.967257+0000","last_undegraded":"2024-09-19T08:45:47.967257+0000","last_fullsized":"2024-09-19T08:45:47.967257+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:52:42.756421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968722+0000","last_change":"2024-09-19T08:43:52.764376+0000","last_active":"2024-09-19T08:45:47.968722+0000","last_peered":"2024-09-19T08:45:47.968722+0000","last_clean":"2024-09-19T08:45:47.968722+0000","last_became_active":"2024-09-19T08:43:52.764048+0000","last_became_peered":"2024-09-19T08:43:52.764048+0000","last_unstale":"2024-09-19T08:45:47.968722+0000","last_undegraded":"2024-09-19T08:45:47.968722+0000","last_fullsized":"2024-09-19T08:45:47.968722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:56.732094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229417+0000","last_change":"2024-09-19T08:44:02.827925+0000","last_active":"2024-09-19T08:45:48.229417+0000","last_peered":"2024-09-19T08:45:48.229417+0000","last_clean":"2024-09-19T08:45:48.229417+0000","last_became_active":"2024-09-19T08:44:02.827269+0000","last_became_peered":"2024-09-19T08:44:02.827269+0000","last_unstale":"2024-09-19T08:45:48.229417+0000","last_undegraded":"2024-09-19T08:45:48.229417+0000","last_fullsized":"2024-09-19T08:45:48.229417+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:31:54.556786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.224157+0000","last_change":"2024-09-19T08:43:52.762190+0000","last_active":"2024-09-19T08:45:48.224157+0000","last_peered":"2024-09-19T08:45:48.224157+0000","last_clean":"2024-09-19T08:45:48.224157+0000","last_became_active":"2024-09-19T08:43:52.761818+0000","last_became_peered":"2024-09-19T08:43:52.761818+0000","last_unstale":"2024-09-19T08:45:48.224157+0000","last_undegraded":"2024-09-19T08:45:48.224157+0000","last_fullsized":"2024-09-19T08:45:48.224157+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:19:55.246779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221492+0000","last_change":"2024-09-19T08:43:52.761069+0000","last_active":"2024-09-19T08:45:48.221492+0000","last_peered":"2024-09-19T08:45:48.221492+0000","last_clean":"2024-09-19T08:45:48.221492+0000","last_became_active":"2024-09-19T08:43:52.760705+0000","last_became_peered":"2024-09-19T08:43:52.760705+0000","last_unstale":"2024-09-19T08:45:48.221492+0000","last_undegraded":"2024-09-19T08:45:48.221492+0000","last_fullsized":"2024-09-19T08:45:48.221492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:46:27.059026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065963+0000","last_change":"2024-09-19T08:43:52.773449+0000","last_active":"2024-09-19T08:45:48.065963+0000","last_peered":"2024-09-19T08:45:48.065963+0000","last_clean":"2024-09-19T08:45:48.065963+0000","last_became_active":"2024-09-19T08:43:52.772929+0000","last_became_peered":"2024-09-19T08:43:52.772929+0000","last_unstale":"2024-09-19T08:45:48.065963+0000","last_undegraded":"2024-09-19T08:45:48.065963+0000","last_fullsized":"2024-09-19T08:45:48.065963+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:00:38.929199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074018+0000","last_change":"2024-09-19T08:43:52.773885+0000","last_active":"2024-09-19T08:45:48.074018+0000","last_peered":"2024-09-19T08:45:48.074018+0000","last_clean":"2024-09-19T08:45:48.074018+0000","last_became_active":"2024-09-19T08:43:52.772856+0000","last_became_peered":"2024-09-19T08:43:52.772856+0000","last_unstale":"2024-09-19T08:45:48.074018+0000","last_undegraded":"2024-09-19T08:45:48.074018+0000","last_fullsized":"2024-09-19T08:45:48.074018+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:13:22.581335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.067279+0000","last_change":"2024-09-19T08:43:52.762909+0000","last_active":"2024-09-19T08:45:48.067279+0000","last_peered":"2024-09-19T08:45:48.067279+0000","last_clean":"2024-09-19T08:45:48.067279+0000","last_became_active":"2024-09-19T08:43:52.762600+0000","last_became_peered":"2024-09-19T08:43:52.762600+0000","last_unstale":"2024-09-19T08:45:48.067279+0000","last_undegraded":"2024-09-19T08:45:48.067279+0000","last_fullsized":"2024-09-19T08:45:48.067279+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:46.098294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968939+0000","last_change":"2024-09-19T08:43:52.762861+0000","last_active":"2024-09-19T08:45:47.968939+0000","last_peered":"2024-09-19T08:45:47.968939+0000","last_clean":"2024-09-19T08:45:47.968939+0000","last_became_active":"2024-09-19T08:43:52.762525+0000","last_became_peered":"2024-09-19T08:43:52.762525+0000","last_unstale":"2024-09-19T08:45:47.968939+0000","last_undegraded":"2024-09-19T08:45:47.968939+0000","last_fullsized":"2024-09-19T08:45:47.968939+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:54.717249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229602+0000","last_change":"2024-09-19T08:43:52.768370+0000","last_active":"2024-09-19T08:45:48.229602+0000","last_peered":"2024-09-19T08:45:48.229602+0000","last_clean":"2024-09-19T08:45:48.229602+0000","last_became_active":"2024-09-19T08:43:52.768055+0000","last_became_peered":"2024-09-19T08:43:52.768055+0000","last_unstale":"2024-09-19T08:45:48.229602+0000","last_undegraded":"2024-09-19T08:45:48.229602+0000","last_fullsized":"2024-09-19T08:45:48.229602+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:48.212862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.224066+0000","last_change":"2024-09-19T08:43:52.758541+0000","last_active":"2024-09-19T08:45:48.224066+0000","last_peered":"2024-09-19T08:45:48.224066+0000","last_clean":"2024-09-19T08:45:48.224066+0000","last_became_active":"2024-09-19T08:43:52.758122+0000","last_became_peered":"2024-09-19T08:43:52.758122+0000","last_unstale":"2024-09-19T08:45:48.224066+0000","last_undegraded":"2024-09-19T08:45:48.224066+0000","last_fullsized":"2024-09-19T08:45:48.224066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:54:40.681880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.231790+0000","last_change":"2024-09-19T08:43:52.758109+0000","last_active":"2024-09-19T08:45:48.231790+0000","last_peered":"2024-09-19T08:45:48.231790+0000","last_clean":"2024-09-19T08:45:48.231790+0000","last_became_active":"2024-09-19T08:43:52.757808+0000","last_became_peered":"2024-09-19T08:43:52.757808+0000","last_unstale":"2024-09-19T08:45:48.231790+0000","last_undegraded":"2024-09-19T08:45:48.231790+0000","last_fullsized":"2024-09-19T08:45:48.231790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:02:47.614562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968751+0000","last_change":"2024-09-19T08:43:52.772501+0000","last_active":"2024-09-19T08:45:47.968751+0000","last_peered":"2024-09-19T08:45:47.968751+0000","last_clean":"2024-09-19T08:45:47.968751+0000","last_became_active":"2024-09-19T08:43:52.771611+0000","last_became_peered":"2024-09-19T08:43:52.771611+0000","last_unstale":"2024-09-19T08:45:47.968751+0000","last_undegraded":"2024-09-19T08:45:47.968751+0000","last_fullsized":"2024-09-19T08:45:47.968751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:49.537237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968692+0000","last_change":"2024-09-19T08:43:52.773664+0000","last_active":"2024-09-19T08:45:47.968692+0000","last_peered":"2024-09-19T08:45:47.968692+0000","last_clean":"2024-09-19T08:45:47.968692+0000","last_became_active":"2024-09-19T08:43:52.773014+0000","last_became_peered":"2024-09-19T08:43:52.773014+0000","last_unstale":"2024-09-19T08:45:47.968692+0000","last_undegraded":"2024-09-19T08:45:47.968692+0000","last_fullsized":"2024-09-19T08:45:47.968692+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:03.448825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229553+0000","last_change":"2024-09-19T08:43:52.769578+0000","last_active":"2024-09-19T08:45:48.229553+0000","last_peered":"2024-09-19T08:45:48.229553+0000","last_clean":"2024-09-19T08:45:48.229553+0000","last_became_active":"2024-09-19T08:43:52.769252+0000","last_became_peered":"2024-09-19T08:43:52.769252+0000","last_unstale":"2024-09-19T08:45:48.229553+0000","last_undegraded":"2024-09-19T08:45:48.229553+0000","last_fullsized":"2024-09-19T08:45:48.229553+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:27.936118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065844+0000","last_change":"2024-09-19T08:43:52.770453+0000","last_active":"2024-09-19T08:45:48.065844+0000","last_peered":"2024-09-19T08:45:48.065844+0000","last_clean":"2024-09-19T08:45:48.065844+0000","last_became_active":"2024-09-19T08:43:52.770155+0000","last_became_peered":"2024-09-19T08:43:52.770155+0000","last_unstale":"2024-09-19T08:45:48.065844+0000","last_undegraded":"2024-09-19T08:45:48.065844+0000","last_fullsized":"2024-09-19T08:45:48.065844+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:12.893572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'8","reported_seq":152,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:57.794654+0000","last_change":"2024-09-19T08:43:50.751316+0000","last_active":"2024-09-19T08:45:57.794654+0000","last_peered":"2024-09-19T08:45:57.794654+0000","last_clean":"2024-09-19T08:45:57.794654+0000","last_became_active":"2024-09-19T08:43:50.750972+0000","last_became_peered":"2024-09-19T08:43:50.750972+0000","last_unstale":"2024-09-19T08:45:57.794654+0000","last_undegraded":"2024-09-19T08:45:57.794654+0000","last_fullsized":"2024-09-19T08:45:57.794654+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:40.654729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1549,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074105+0000","last_change":"2024-09-19T08:43:52.773979+0000","last_active":"2024-09-19T08:45:48.074105+0000","last_peered":"2024-09-19T08:45:48.074105+0000","last_clean":"2024-09-19T08:45:48.074105+0000","last_became_active":"2024-09-19T08:43:52.773648+0000","last_became_peered":"2024-09-19T08:43:52.773648+0000","last_unstale":"2024-09-19T08:45:48.074105+0000","last_undegraded":"2024-09-19T08:45:48.074105+0000","last_fullsized":"2024-09-19T08:45:48.074105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:36.061209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.076810+0000","last_change":"2024-09-19T08:43:50.750704+0000","last_active":"2024-09-19T08:45:48.076810+0000","last_peered":"2024-09-19T08:45:48.076810+0000","last_clean":"2024-09-19T08:45:48.076810+0000","last_became_active":"2024-09-19T08:43:50.750399+0000","last_became_peered":"2024-09-19T08:43:50.750399+0000","last_unstale":"2024-09-19T08:45:48.076810+0000","last_undegraded":"2024-09-19T08:45:48.076810+0000","last_fullsized":"2024-09-19T08:45:48.076810+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:25.447320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.223615+0000","last_change":"2024-09-19T08:43:50.744919+0000","last_active":"2024-09-19T08:45:48.223615+0000","last_peered":"2024-09-19T08:45:48.223615+0000","last_clean":"2024-09-19T08:45:48.223615+0000","last_became_active":"2024-09-19T08:43:50.744604+0000","last_became_peered":"2024-09-19T08:43:50.744604+0000","last_unstale":"2024-09-19T08:45:48.223615+0000","last_undegraded":"2024-09-19T08:45:48.223615+0000","last_fullsized":"2024-09-19T08:45:48.223615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:25:06.410093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065913+0000","last_change":"2024-09-19T08:43:52.766779+0000","last_active":"2024-09-19T08:45:48.065913+0000","last_peered":"2024-09-19T08:45:48.065913+0000","last_clean":"2024-09-19T08:45:48.065913+0000","last_became_active":"2024-09-19T08:43:52.766484+0000","last_became_peered":"2024-09-19T08:43:52.766484+0000","last_unstale":"2024-09-19T08:45:48.065913+0000","last_undegraded":"2024-09-19T08:45:48.065913+0000","last_fullsized":"2024-09-19T08:45:48.065913+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:17:29.350778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074289+0000","last_change":"2024-09-19T08:43:50.751282+0000","last_active":"2024-09-19T08:45:48.074289+0000","last_peered":"2024-09-19T08:45:48.074289+0000","last_clean":"2024-09-19T08:45:48.074289+0000","last_became_active":"2024-09-19T08:43:50.750754+0000","last_became_peered":"2024-09-19T08:43:50.750754+0000","last_unstale":"2024-09-19T08:45:48.074289+0000","last_undegraded":"2024-09-19T08:45:48.074289+0000","last_fullsized":"2024-09-19T08:45:48.074289+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:11.661626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.967219+0000","last_change":"2024-09-19T08:43:52.756442+0000","last_active":"2024-09-19T08:45:47.967219+0000","last_peered":"2024-09-19T08:45:47.967219+0000","last_clean":"2024-09-19T08:45:47.967219+0000","last_became_active":"2024-09-19T08:43:52.756116+0000","last_became_peered":"2024-09-19T08:43:52.756116+0000","last_unstale":"2024-09-19T08:45:47.967219+0000","last_undegraded":"2024-09-19T08:45:47.967219+0000","last_fullsized":"2024-09-19T08:45:47.967219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:40.029272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'8","reported_seq":262,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:58.795230+0000","last_change":"2024-09-19T08:43:50.749637+0000","last_active":"2024-09-19T08:45:58.795230+0000","last_peered":"2024-09-19T08:45:58.795230+0000","last_clean":"2024-09-19T08:45:58.795230+0000","last_became_active":"2024-09-19T08:43:50.749307+0000","last_became_peered":"2024-09-19T08:43:50.749307+0000","last_unstale":"2024-09-19T08:45:58.795230+0000","last_undegraded":"2024-09-19T08:45:58.795230+0000","last_fullsized":"2024-09-19T08:45:58.795230+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:23:33.409671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":230,"num_read_kb":230,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.222016+0000","last_change":"2024-09-19T08:43:52.763592+0000","last_active":"2024-09-19T08:45:48.222016+0000","last_peered":"2024-09-19T08:45:48.222016+0000","last_clean":"2024-09-19T08:45:48.222016+0000","last_became_active":"2024-09-19T08:43:52.763146+0000","last_became_peered":"2024-09-19T08:43:52.763146+0000","last_unstale":"2024-09-19T08:45:48.222016+0000","last_undegraded":"2024-09-19T08:45:48.222016+0000","last_fullsized":"2024-09-19T08:45:48.222016+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:32.194465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221748+0000","last_change":"2024-09-19T08:43:50.749184+0000","last_active":"2024-09-19T08:45:48.221748+0000","last_peered":"2024-09-19T08:45:48.221748+0000","last_clean":"2024-09-19T08:45:48.221748+0000","last_became_active":"2024-09-19T08:43:50.748844+0000","last_became_peered":"2024-09-19T08:43:50.748844+0000","last_unstale":"2024-09-19T08:45:48.221748+0000","last_undegraded":"2024-09-19T08:45:48.221748+0000","last_fullsized":"2024-09-19T08:45:48.221748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:15.334759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.231209+0000","last_change":"2024-09-19T08:43:52.768696+0000","last_active":"2024-09-19T08:45:48.231209+0000","last_peered":"2024-09-19T08:45:48.231209+0000","last_clean":"2024-09-19T08:45:48.231209+0000","last_became_active":"2024-09-19T08:43:52.768374+0000","last_became_peered":"2024-09-19T08:43:52.768374+0000","last_unstale":"2024-09-19T08:45:48.231209+0000","last_undegraded":"2024-09-19T08:45:48.231209+0000","last_fullsized":"2024-09-19T08:45:48.231209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:34:39.343041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074342+0000","last_change":"2024-09-19T08:43:51.026270+0000","last_active":"2024-09-19T08:45:48.074342+0000","last_peered":"2024-09-19T08:45:48.074342+0000","last_clean":"2024-09-19T08:45:48.074342+0000","last_became_active":"2024-09-19T08:43:51.025823+0000","last_became_peered":"2024-09-19T08:43:51.025823+0000","last_unstale":"2024-09-19T08:45:48.074342+0000","last_undegraded":"2024-09-19T08:45:48.074342+0000","last_fullsized":"2024-09-19T08:45:48.074342+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:00.480438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221712+0000","last_change":"2024-09-19T08:43:52.759555+0000","last_active":"2024-09-19T08:45:48.221712+0000","last_peered":"2024-09-19T08:45:48.221712+0000","last_clean":"2024-09-19T08:45:48.221712+0000","last_became_active":"2024-09-19T08:43:52.759223+0000","last_became_peered":"2024-09-19T08:43:52.759223+0000","last_unstale":"2024-09-19T08:45:48.221712+0000","last_undegraded":"2024-09-19T08:45:48.221712+0000","last_fullsized":"2024-09-19T08:45:48.221712+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:57:06.451323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'37","reported_seq":306,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:58.795355+0000","last_change":"2024-09-19T08:43:50.751924+0000","last_active":"2024-09-19T08:45:58.795355+0000","last_peered":"2024-09-19T08:45:58.795355+0000","last_clean":"2024-09-19T08:45:58.795355+0000","last_became_active":"2024-09-19T08:43:50.751117+0000","last_became_peered":"2024-09-19T08:43:50.751117+0000","last_unstale":"2024-09-19T08:45:58.795355+0000","last_undegraded":"2024-09-19T08:45:58.795355+0000","last_fullsized":"2024-09-19T08:45:58.795355+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:33:57.196283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67065,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":245,"num_read_kb":189,"num_write":39,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066156+0000","last_change":"2024-09-19T08:43:52.762260+0000","last_active":"2024-09-19T08:45:48.066156+0000","last_peered":"2024-09-19T08:45:48.066156+0000","last_clean":"2024-09-19T08:45:48.066156+0000","last_became_active":"2024-09-19T08:43:52.761923+0000","last_became_peered":"2024-09-19T08:43:52.761923+0000","last_unstale":"2024-09-19T08:45:48.066156+0000","last_undegraded":"2024-09-19T08:45:48.066156+0000","last_fullsized":"2024-09-19T08:45:48.066156+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:39:39.503332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968758+0000","last_change":"2024-09-19T08:43:50.749745+0000","last_active":"2024-09-19T08:45:47.968758+0000","last_peered":"2024-09-19T08:45:47.968758+0000","last_clean":"2024-09-19T08:45:47.968758+0000","last_became_active":"2024-09-19T08:43:50.749073+0000","last_became_peered":"2024-09-19T08:43:50.749073+0000","last_unstale":"2024-09-19T08:45:47.968758+0000","last_undegraded":"2024-09-19T08:45:47.968758+0000","last_fullsized":"2024-09-19T08:45:47.968758+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:48:11.965841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.223734+0000","last_change":"2024-09-19T08:43:52.761495+0000","last_active":"2024-09-19T08:45:48.223734+0000","last_peered":"2024-09-19T08:45:48.223734+0000","last_clean":"2024-09-19T08:45:48.223734+0000","last_became_active":"2024-09-19T08:43:52.760425+0000","last_became_peered":"2024-09-19T08:43:52.760425+0000","last_unstale":"2024-09-19T08:45:48.223734+0000","last_undegraded":"2024-09-19T08:45:48.223734+0000","last_fullsized":"2024-09-19T08:45:48.223734+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:57.689284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074090+0000","last_change":"2024-09-19T08:43:50.749563+0000","last_active":"2024-09-19T08:45:48.074090+0000","last_peered":"2024-09-19T08:45:48.074090+0000","last_clean":"2024-09-19T08:45:48.074090+0000","last_became_active":"2024-09-19T08:43:50.749244+0000","last_became_peered":"2024-09-19T08:43:50.749244+0000","last_unstale":"2024-09-19T08:45:48.074090+0000","last_undegraded":"2024-09-19T08:45:48.074090+0000","last_fullsized":"2024-09-19T08:45:48.074090+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:59:02.312883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221429+0000","last_change":"2024-09-19T08:43:52.767087+0000","last_active":"2024-09-19T08:45:48.221429+0000","last_peered":"2024-09-19T08:45:48.221429+0000","last_clean":"2024-09-19T08:45:48.221429+0000","last_became_active":"2024-09-19T08:43:52.766730+0000","last_became_peered":"2024-09-19T08:43:52.766730+0000","last_unstale":"2024-09-19T08:45:48.221429+0000","last_undegraded":"2024-09-19T08:45:48.221429+0000","last_fullsized":"2024-09-19T08:45:48.221429+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:04.040896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":146,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:57.806660+0000","last_change":"2024-09-19T08:43:50.749659+0000","last_active":"2024-09-19T08:45:57.806660+0000","last_peered":"2024-09-19T08:45:57.806660+0000","last_clean":"2024-09-19T08:45:57.806660+0000","last_became_active":"2024-09-19T08:43:50.749314+0000","last_became_peered":"2024-09-19T08:43:50.749314+0000","last_unstale":"2024-09-19T08:45:57.806660+0000","last_undegraded":"2024-09-19T08:45:57.806660+0000","last_fullsized":"2024-09-19T08:45:57.806660+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:08.576209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221691+0000","last_change":"2024-09-19T08:43:52.767928+0000","last_active":"2024-09-19T08:45:48.221691+0000","last_peered":"2024-09-19T08:45:48.221691+0000","last_clean":"2024-09-19T08:45:48.221691+0000","last_became_active":"2024-09-19T08:43:52.767580+0000","last_became_peered":"2024-09-19T08:43:52.767580+0000","last_unstale":"2024-09-19T08:45:48.221691+0000","last_undegraded":"2024-09-19T08:45:48.221691+0000","last_fullsized":"2024-09-19T08:45:48.221691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:45.344693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229348+0000","last_change":"2024-09-19T08:44:02.833887+0000","last_active":"2024-09-19T08:45:48.229348+0000","last_peered":"2024-09-19T08:45:48.229348+0000","last_clean":"2024-09-19T08:45:48.229348+0000","last_became_active":"2024-09-19T08:44:02.821624+0000","last_became_peered":"2024-09-19T08:44:02.821624+0000","last_unstale":"2024-09-19T08:45:48.229348+0000","last_undegraded":"2024-09-19T08:45:48.229348+0000","last_fullsized":"2024-09-19T08:45:48.229348+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:54:36.131949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.967019+0000","last_change":"2024-09-19T08:44:03.080503+0000","last_active":"2024-09-19T08:45:47.967019+0000","last_peered":"2024-09-19T08:45:47.967019+0000","last_clean":"2024-09-19T08:45:47.967019+0000","last_became_active":"2024-09-19T08:44:03.080129+0000","last_became_peered":"2024-09-19T08:44:03.080129+0000","last_unstale":"2024-09-19T08:45:47.967019+0000","last_undegraded":"2024-09-19T08:45:47.967019+0000","last_fullsized":"2024-09-19T08:45:47.967019+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:01:37.034795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066192+0000","last_change":"2024-09-19T08:43:50.747624+0000","last_active":"2024-09-19T08:45:48.066192+0000","last_peered":"2024-09-19T08:45:48.066192+0000","last_clean":"2024-09-19T08:45:48.066192+0000","last_became_active":"2024-09-19T08:43:50.747263+0000","last_became_peered":"2024-09-19T08:43:50.747263+0000","last_unstale":"2024-09-19T08:45:48.066192+0000","last_undegraded":"2024-09-19T08:45:48.066192+0000","last_fullsized":"2024-09-19T08:45:48.066192+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:18.580106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229525+0000","last_change":"2024-09-19T08:43:52.770051+0000","last_active":"2024-09-19T08:45:48.229525+0000","last_peered":"2024-09-19T08:45:48.229525+0000","last_clean":"2024-09-19T08:45:48.229525+0000","last_became_active":"2024-09-19T08:43:52.769727+0000","last_became_peered":"2024-09-19T08:43:52.769727+0000","last_unstale":"2024-09-19T08:45:48.229525+0000","last_undegraded":"2024-09-19T08:45:48.229525+0000","last_fullsized":"2024-09-19T08:45:48.229525+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:34.954897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.223984+0000","last_change":"2024-09-19T08:43:50.746696+0000","last_active":"2024-09-19T08:45:48.223984+0000","last_peered":"2024-09-19T08:45:48.223984+0000","last_clean":"2024-09-19T08:45:48.223984+0000","last_became_active":"2024-09-19T08:43:50.746246+0000","last_became_peered":"2024-09-19T08:43:50.746246+0000","last_unstale":"2024-09-19T08:45:48.223984+0000","last_undegraded":"2024-09-19T08:45:48.223984+0000","last_fullsized":"2024-09-19T08:45:48.223984+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:38:39.935348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.222174+0000","last_change":"2024-09-19T08:43:52.765199+0000","last_active":"2024-09-19T08:45:48.222174+0000","last_peered":"2024-09-19T08:45:48.222174+0000","last_clean":"2024-09-19T08:45:48.222174+0000","last_became_active":"2024-09-19T08:43:52.764896+0000","last_became_peered":"2024-09-19T08:43:52.764896+0000","last_unstale":"2024-09-19T08:45:48.222174+0000","last_undegraded":"2024-09-19T08:45:48.222174+0000","last_fullsized":"2024-09-19T08:45:48.222174+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:02:59.570018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'67","reported_seq":122,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:57.438919+0000","last_change":"2024-09-19T08:43:50.747242+0000","last_active":"2024-09-19T08:45:57.438919+0000","last_peered":"2024-09-19T08:45:57.438919+0000","last_clean":"2024-09-19T08:45:57.438919+0000","last_became_active":"2024-09-19T08:43:50.746853+0000","last_became_peered":"2024-09-19T08:43:50.746853+0000","last_unstale":"2024-09-19T08:45:57.438919+0000","last_undegraded":"2024-09-19T08:45:57.438919+0000","last_fullsized":"2024-09-19T08:45:57.438919+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:21.545635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221579+0000","last_change":"2024-09-19T08:44:02.826535+0000","last_active":"2024-09-19T08:45:48.221579+0000","last_peered":"2024-09-19T08:45:48.221579+0000","last_clean":"2024-09-19T08:45:48.221579+0000","last_became_active":"2024-09-19T08:44:02.826113+0000","last_became_peered":"2024-09-19T08:44:02.826113+0000","last_unstale":"2024-09-19T08:45:48.221579+0000","last_undegraded":"2024-09-19T08:45:48.221579+0000","last_fullsized":"2024-09-19T08:45:48.221579+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:20.247732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'8","reported_seq":152,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:57.794457+0000","last_change":"2024-09-19T08:43:51.131540+0000","last_active":"2024-09-19T08:45:57.794457+0000","last_peered":"2024-09-19T08:45:57.794457+0000","last_clean":"2024-09-19T08:45:57.794457+0000","last_became_active":"2024-09-19T08:43:51.131170+0000","last_became_peered":"2024-09-19T08:43:51.131170+0000","last_unstale":"2024-09-19T08:45:57.794457+0000","last_undegraded":"2024-09-19T08:45:57.794457+0000","last_fullsized":"2024-09-19T08:45:57.794457+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:43.635612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221593+0000","last_change":"2024-09-19T08:43:52.763308+0000","last_active":"2024-09-19T08:45:48.221593+0000","last_peered":"2024-09-19T08:45:48.221593+0000","last_clean":"2024-09-19T08:45:48.221593+0000","last_became_active":"2024-09-19T08:43:52.762859+0000","last_became_peered":"2024-09-19T08:43:52.762859+0000","last_unstale":"2024-09-19T08:45:48.221593+0000","last_undegraded":"2024-09-19T08:45:48.221593+0000","last_fullsized":"2024-09-19T08:45:48.221593+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:32:35.083361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"83'148","reported_seq":394,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:50.085307+0000","last_change":"2024-09-19T08:42:39.567408+0000","last_active":"2024-09-19T08:45:50.085307+0000","last_peered":"2024-09-19T08:45:50.085307+0000","last_clean":"2024-09-19T08:45:50.085307+0000","last_became_active":"2024-09-19T08:42:39.566967+0000","last_became_peered":"2024-09-19T08:42:39.566967+0000","last_unstale":"2024-09-19T08:45:50.085307+0000","last_undegraded":"2024-09-19T08:45:50.085307+0000","last_fullsized":"2024-09-19T08:45:50.085307+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:14.287944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:14.287944+0000","last_clean_scrub_stamp":"2024-09-19T08:39:14.287944+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:31:03.245386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.966315+0000","last_change":"2024-09-19T08:43:50.751097+0000","last_active":"2024-09-19T08:45:47.966315+0000","last_peered":"2024-09-19T08:45:47.966315+0000","last_clean":"2024-09-19T08:45:47.966315+0000","last_became_active":"2024-09-19T08:43:50.750341+0000","last_became_peered":"2024-09-19T08:43:50.750341+0000","last_unstale":"2024-09-19T08:45:47.966315+0000","last_undegraded":"2024-09-19T08:45:47.966315+0000","last_fullsized":"2024-09-19T08:45:47.966315+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:25.694535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066417+0000","last_change":"2024-09-19T08:43:52.767905+0000","last_active":"2024-09-19T08:45:48.066417+0000","last_peered":"2024-09-19T08:45:48.066417+0000","last_clean":"2024-09-19T08:45:48.066417+0000","last_became_active":"2024-09-19T08:43:52.767601+0000","last_became_peered":"2024-09-19T08:43:52.767601+0000","last_unstale":"2024-09-19T08:45:48.066417+0000","last_undegraded":"2024-09-19T08:45:48.066417+0000","last_fullsized":"2024-09-19T08:45:48.066417+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:14.519879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221389+0000","last_change":"2024-09-19T08:43:50.751397+0000","last_active":"2024-09-19T08:45:48.221389+0000","last_peered":"2024-09-19T08:45:48.221389+0000","last_clean":"2024-09-19T08:45:48.221389+0000","last_became_active":"2024-09-19T08:43:50.751079+0000","last_became_peered":"2024-09-19T08:43:50.751079+0000","last_unstale":"2024-09-19T08:45:48.221389+0000","last_undegraded":"2024-09-19T08:45:48.221389+0000","last_fullsized":"2024-09-19T08:45:48.221389+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:29.904786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.223683+0000","last_change":"2024-09-19T08:43:52.773348+0000","last_active":"2024-09-19T08:45:48.223683+0000","last_peered":"2024-09-19T08:45:48.223683+0000","last_clean":"2024-09-19T08:45:48.223683+0000","last_became_active":"2024-09-19T08:43:52.773011+0000","last_became_peered":"2024-09-19T08:43:52.773011+0000","last_unstale":"2024-09-19T08:45:48.223683+0000","last_undegraded":"2024-09-19T08:45:48.223683+0000","last_fullsized":"2024-09-19T08:45:48.223683+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:10.806578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":146,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:57.805846+0000","last_change":"2024-09-19T08:43:51.025300+0000","last_active":"2024-09-19T08:45:57.805846+0000","last_peered":"2024-09-19T08:45:57.805846+0000","last_clean":"2024-09-19T08:45:57.805846+0000","last_became_active":"2024-09-19T08:43:51.024936+0000","last_became_peered":"2024-09-19T08:43:51.024936+0000","last_unstale":"2024-09-19T08:45:57.805846+0000","last_undegraded":"2024-09-19T08:45:57.805846+0000","last_fullsized":"2024-09-19T08:45:57.805846+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:03.186066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074113+0000","last_change":"2024-09-19T08:43:52.774962+0000","last_active":"2024-09-19T08:45:48.074113+0000","last_peered":"2024-09-19T08:45:48.074113+0000","last_clean":"2024-09-19T08:45:48.074113+0000","last_became_active":"2024-09-19T08:43:52.774598+0000","last_became_peered":"2024-09-19T08:43:52.774598+0000","last_unstale":"2024-09-19T08:45:48.074113+0000","last_undegraded":"2024-09-19T08:45:48.074113+0000","last_fullsized":"2024-09-19T08:45:48.074113+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:02.687136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'7","reported_seq":148,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:57.794028+0000","last_change":"2024-09-19T08:43:50.748180+0000","last_active":"2024-09-19T08:45:57.794028+0000","last_peered":"2024-09-19T08:45:57.794028+0000","last_clean":"2024-09-19T08:45:57.794028+0000","last_became_active":"2024-09-19T08:43:50.747853+0000","last_became_peered":"2024-09-19T08:43:50.747853+0000","last_unstale":"2024-09-19T08:45:57.794028+0000","last_undegraded":"2024-09-19T08:45:57.794028+0000","last_fullsized":"2024-09-19T08:45:57.794028+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:56.047259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.077066+0000","last_change":"2024-09-19T08:43:52.770253+0000","last_active":"2024-09-19T08:45:48.077066+0000","last_peered":"2024-09-19T08:45:48.077066+0000","last_clean":"2024-09-19T08:45:48.077066+0000","last_became_active":"2024-09-19T08:43:52.769951+0000","last_became_peered":"2024-09-19T08:43:52.769951+0000","last_unstale":"2024-09-19T08:45:48.077066+0000","last_undegraded":"2024-09-19T08:45:48.077066+0000","last_fullsized":"2024-09-19T08:45:48.077066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:02.353144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.967308+0000","last_change":"2024-09-19T08:43:51.026453+0000","last_active":"2024-09-19T08:45:47.967308+0000","last_peered":"2024-09-19T08:45:47.967308+0000","last_clean":"2024-09-19T08:45:47.967308+0000","last_became_active":"2024-09-19T08:43:51.025873+0000","last_became_peered":"2024-09-19T08:43:51.025873+0000","last_unstale":"2024-09-19T08:45:47.967308+0000","last_undegraded":"2024-09-19T08:45:47.967308+0000","last_fullsized":"2024-09-19T08:45:47.967308+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:25:17.475074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.076927+0000","last_change":"2024-09-19T08:43:52.761700+0000","last_active":"2024-09-19T08:45:48.076927+0000","last_peered":"2024-09-19T08:45:48.076927+0000","last_clean":"2024-09-19T08:45:48.076927+0000","last_became_active":"2024-09-19T08:43:52.755527+0000","last_became_peered":"2024-09-19T08:43:52.755527+0000","last_unstale":"2024-09-19T08:45:48.076927+0000","last_undegraded":"2024-09-19T08:45:48.076927+0000","last_fullsized":"2024-09-19T08:45:48.076927+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:18:43.171284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066342+0000","last_change":"2024-09-19T08:43:51.026136+0000","last_active":"2024-09-19T08:45:48.066342+0000","last_peered":"2024-09-19T08:45:48.066342+0000","last_clean":"2024-09-19T08:45:48.066342+0000","last_became_active":"2024-09-19T08:43:51.025740+0000","last_became_peered":"2024-09-19T08:43:51.025740+0000","last_unstale":"2024-09-19T08:45:48.066342+0000","last_undegraded":"2024-09-19T08:45:48.066342+0000","last_fullsized":"2024-09-19T08:45:48.066342+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:17:32.741083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.224195+0000","last_change":"2024-09-19T08:43:52.763808+0000","last_active":"2024-09-19T08:45:48.224195+0000","last_peered":"2024-09-19T08:45:48.224195+0000","last_clean":"2024-09-19T08:45:48.224195+0000","last_became_active":"2024-09-19T08:43:52.763509+0000","last_became_peered":"2024-09-19T08:43:52.763509+0000","last_unstale":"2024-09-19T08:45:48.224195+0000","last_undegraded":"2024-09-19T08:45:48.224195+0000","last_fullsized":"2024-09-19T08:45:48.224195+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:45:08.385419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'104","reported_seq":221,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:57.778372+0000","last_change":"2024-09-19T08:43:50.744213+0000","last_active":"2024-09-19T08:45:57.778372+0000","last_peered":"2024-09-19T08:45:57.778372+0000","last_clean":"2024-09-19T08:45:57.778372+0000","last_became_active":"2024-09-19T08:43:50.743848+0000","last_became_peered":"2024-09-19T08:43:50.743848+0000","last_unstale":"2024-09-19T08:45:57.778372+0000","last_undegraded":"2024-09-19T08:45:57.778372+0000","last_fullsized":"2024-09-19T08:45:57.778372+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:26.613055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":189,"num_write_kb":1285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.067394+0000","last_change":"2024-09-19T08:43:52.767102+0000","last_active":"2024-09-19T08:45:48.067394+0000","last_peered":"2024-09-19T08:45:48.067394+0000","last_clean":"2024-09-19T08:45:48.067394+0000","last_became_active":"2024-09-19T08:43:52.766262+0000","last_became_peered":"2024-09-19T08:43:52.766262+0000","last_unstale":"2024-09-19T08:45:48.067394+0000","last_undegraded":"2024-09-19T08:45:48.067394+0000","last_fullsized":"2024-09-19T08:45:48.067394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:35.740462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.966517+0000","last_change":"2024-09-19T08:43:51.131742+0000","last_active":"2024-09-19T08:45:47.966517+0000","last_peered":"2024-09-19T08:45:47.966517+0000","last_clean":"2024-09-19T08:45:47.966517+0000","last_became_active":"2024-09-19T08:43:51.131388+0000","last_became_peered":"2024-09-19T08:43:51.131388+0000","last_unstale":"2024-09-19T08:45:47.966517+0000","last_undegraded":"2024-09-19T08:45:47.966517+0000","last_fullsized":"2024-09-19T08:45:47.966517+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:26.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.230731+0000","last_change":"2024-09-19T08:43:52.756004+0000","last_active":"2024-09-19T08:45:48.230731+0000","last_peered":"2024-09-19T08:45:48.230731+0000","last_clean":"2024-09-19T08:45:48.230731+0000","last_became_active":"2024-09-19T08:43:52.755596+0000","last_became_peered":"2024-09-19T08:43:52.755596+0000","last_unstale":"2024-09-19T08:45:48.230731+0000","last_undegraded":"2024-09-19T08:45:48.230731+0000","last_fullsized":"2024-09-19T08:45:48.230731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:35.942375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221620+0000","last_change":"2024-09-19T08:44:02.835986+0000","last_active":"2024-09-19T08:45:48.221620+0000","last_peered":"2024-09-19T08:45:48.221620+0000","last_clean":"2024-09-19T08:45:48.221620+0000","last_became_active":"2024-09-19T08:44:02.826314+0000","last_became_peered":"2024-09-19T08:44:02.826314+0000","last_unstale":"2024-09-19T08:45:48.221620+0000","last_undegraded":"2024-09-19T08:45:48.221620+0000","last_fullsized":"2024-09-19T08:45:48.221620+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:16.943989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065838+0000","last_change":"2024-09-19T08:43:52.766582+0000","last_active":"2024-09-19T08:45:48.065838+0000","last_peered":"2024-09-19T08:45:48.065838+0000","last_clean":"2024-09-19T08:45:48.065838+0000","last_became_active":"2024-09-19T08:43:52.766232+0000","last_became_peered":"2024-09-19T08:43:52.766232+0000","last_unstale":"2024-09-19T08:45:48.065838+0000","last_undegraded":"2024-09-19T08:45:48.065838+0000","last_fullsized":"2024-09-19T08:45:48.065838+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:43:42.319422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066195+0000","last_change":"2024-09-19T08:43:50.747611+0000","last_active":"2024-09-19T08:45:48.066195+0000","last_peered":"2024-09-19T08:45:48.066195+0000","last_clean":"2024-09-19T08:45:48.066195+0000","last_became_active":"2024-09-19T08:43:50.747263+0000","last_became_peered":"2024-09-19T08:43:50.747263+0000","last_unstale":"2024-09-19T08:45:48.066195+0000","last_undegraded":"2024-09-19T08:45:48.066195+0000","last_fullsized":"2024-09-19T08:45:48.066195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:14.717820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229532+0000","last_change":"2024-09-19T08:43:52.766493+0000","last_active":"2024-09-19T08:45:48.229532+0000","last_peered":"2024-09-19T08:45:48.229532+0000","last_clean":"2024-09-19T08:45:48.229532+0000","last_became_active":"2024-09-19T08:43:52.766158+0000","last_became_peered":"2024-09-19T08:43:52.766158+0000","last_unstale":"2024-09-19T08:45:48.229532+0000","last_undegraded":"2024-09-19T08:45:48.229532+0000","last_fullsized":"2024-09-19T08:45:48.229532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:59.175949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968841+0000","last_change":"2024-09-19T08:43:50.751368+0000","last_active":"2024-09-19T08:45:47.968841+0000","last_peered":"2024-09-19T08:45:47.968841+0000","last_clean":"2024-09-19T08:45:47.968841+0000","last_became_active":"2024-09-19T08:43:50.751038+0000","last_became_peered":"2024-09-19T08:43:50.751038+0000","last_unstale":"2024-09-19T08:45:47.968841+0000","last_undegraded":"2024-09-19T08:45:47.968841+0000","last_fullsized":"2024-09-19T08:45:47.968841+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:46.302996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066722+0000","last_change":"2024-09-19T08:43:52.756998+0000","last_active":"2024-09-19T08:45:48.066722+0000","last_peered":"2024-09-19T08:45:48.066722+0000","last_clean":"2024-09-19T08:45:48.066722+0000","last_became_active":"2024-09-19T08:43:52.756661+0000","last_became_peered":"2024-09-19T08:43:52.756661+0000","last_unstale":"2024-09-19T08:45:48.066722+0000","last_undegraded":"2024-09-19T08:45:48.066722+0000","last_fullsized":"2024-09-19T08:45:48.066722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:15.182574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065908+0000","last_change":"2024-09-19T08:43:50.750095+0000","last_active":"2024-09-19T08:45:48.065908+0000","last_peered":"2024-09-19T08:45:48.065908+0000","last_clean":"2024-09-19T08:45:48.065908+0000","last_became_active":"2024-09-19T08:43:50.749672+0000","last_became_peered":"2024-09-19T08:43:50.749672+0000","last_unstale":"2024-09-19T08:45:48.065908+0000","last_undegraded":"2024-09-19T08:45:48.065908+0000","last_fullsized":"2024-09-19T08:45:48.065908+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:25:34.978094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221996+0000","last_change":"2024-09-19T08:43:52.764097+0000","last_active":"2024-09-19T08:45:48.221996+0000","last_peered":"2024-09-19T08:45:48.221996+0000","last_clean":"2024-09-19T08:45:48.221996+0000","last_became_active":"2024-09-19T08:43:52.763773+0000","last_became_peered":"2024-09-19T08:43:52.763773+0000","last_unstale":"2024-09-19T08:45:48.221996+0000","last_undegraded":"2024-09-19T08:45:48.221996+0000","last_fullsized":"2024-09-19T08:45:48.221996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:01:42.098460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074171+0000","last_change":"2024-09-19T08:44:02.825238+0000","last_active":"2024-09-19T08:45:48.074171+0000","last_peered":"2024-09-19T08:45:48.074171+0000","last_clean":"2024-09-19T08:45:48.074171+0000","last_became_active":"2024-09-19T08:44:02.824873+0000","last_became_peered":"2024-09-19T08:44:02.824873+0000","last_unstale":"2024-09-19T08:45:48.074171+0000","last_undegraded":"2024-09-19T08:45:48.074171+0000","last_fullsized":"2024-09-19T08:45:48.074171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:38.896082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066088+0000","last_change":"2024-09-19T08:43:50.748789+0000","last_active":"2024-09-19T08:45:48.066088+0000","last_peered":"2024-09-19T08:45:48.066088+0000","last_clean":"2024-09-19T08:45:48.066088+0000","last_became_active":"2024-09-19T08:43:50.748464+0000","last_became_peered":"2024-09-19T08:43:50.748464+0000","last_unstale":"2024-09-19T08:45:48.066088+0000","last_undegraded":"2024-09-19T08:45:48.066088+0000","last_fullsized":"2024-09-19T08:45:48.066088+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:36:45.905639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221312+0000","last_change":"2024-09-19T08:43:52.756608+0000","last_active":"2024-09-19T08:45:48.221312+0000","last_peered":"2024-09-19T08:45:48.221312+0000","last_clean":"2024-09-19T08:45:48.221312+0000","last_became_active":"2024-09-19T08:43:52.756227+0000","last_became_peered":"2024-09-19T08:43:52.756227+0000","last_unstale":"2024-09-19T08:45:48.221312+0000","last_undegraded":"2024-09-19T08:45:48.221312+0000","last_fullsized":"2024-09-19T08:45:48.221312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:29:58.085584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'13","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066253+0000","last_change":"2024-09-19T08:43:50.749135+0000","last_active":"2024-09-19T08:45:48.066253+0000","last_peered":"2024-09-19T08:45:48.066253+0000","last_clean":"2024-09-19T08:45:48.066253+0000","last_became_active":"2024-09-19T08:43:50.748755+0000","last_became_peered":"2024-09-19T08:43:50.748755+0000","last_unstale":"2024-09-19T08:45:48.066253+0000","last_undegraded":"2024-09-19T08:45:48.066253+0000","last_fullsized":"2024-09-19T08:45:48.066253+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:55.079780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.231124+0000","last_change":"2024-09-19T08:43:52.763277+0000","last_active":"2024-09-19T08:45:48.231124+0000","last_peered":"2024-09-19T08:45:48.231124+0000","last_clean":"2024-09-19T08:45:48.231124+0000","last_became_active":"2024-09-19T08:43:52.762941+0000","last_became_peered":"2024-09-19T08:43:52.762941+0000","last_unstale":"2024-09-19T08:45:48.231124+0000","last_undegraded":"2024-09-19T08:45:48.231124+0000","last_fullsized":"2024-09-19T08:45:48.231124+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:35.531946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.067452+0000","last_change":"2024-09-19T08:43:51.025350+0000","last_active":"2024-09-19T08:45:48.067452+0000","last_peered":"2024-09-19T08:45:48.067452+0000","last_clean":"2024-09-19T08:45:48.067452+0000","last_became_active":"2024-09-19T08:43:51.025046+0000","last_became_peered":"2024-09-19T08:43:51.025046+0000","last_unstale":"2024-09-19T08:45:48.067452+0000","last_undegraded":"2024-09-19T08:45:48.067452+0000","last_fullsized":"2024-09-19T08:45:48.067452+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:27.595545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066292+0000","last_change":"2024-09-19T08:43:52.767917+0000","last_active":"2024-09-19T08:45:48.066292+0000","last_peered":"2024-09-19T08:45:48.066292+0000","last_clean":"2024-09-19T08:45:48.066292+0000","last_became_active":"2024-09-19T08:43:52.767575+0000","last_became_peered":"2024-09-19T08:43:52.767575+0000","last_unstale":"2024-09-19T08:45:48.066292+0000","last_undegraded":"2024-09-19T08:45:48.066292+0000","last_fullsized":"2024-09-19T08:45:48.066292+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:05.950399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":155,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:57.793622+0000","last_change":"2024-09-19T08:43:50.746400+0000","last_active":"2024-09-19T08:45:57.793622+0000","last_peered":"2024-09-19T08:45:57.793622+0000","last_clean":"2024-09-19T08:45:57.793622+0000","last_became_active":"2024-09-19T08:43:50.745891+0000","last_became_peered":"2024-09-19T08:43:50.745891+0000","last_unstale":"2024-09-19T08:45:57.793622+0000","last_undegraded":"2024-09-19T08:45:57.793622+0000","last_fullsized":"2024-09-19T08:45:57.793622+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:03.098202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":797575,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1546,"num_read_kb":995,"num_write":501,"num_write_kb":2380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":2392725,"data_compressed":103473,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":381,"ondisk_log_size":381,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":109335,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579943,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":446056,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808876,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1118208,"data_stored":1431343,"data_compressed":54546,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005104,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":838280,"data_compressed":22188,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234037,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":838429,"data_compressed":22188,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462970,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":983040,"data_stored":1039165,"data_compressed":39344,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593792,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":445450,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822724,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1040384,"data_stored":1366289,"data_compressed":54546,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986249,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":444525,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215182,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":999424,"data_stored":1165820,"data_compressed":37390,"data_compressed_allocated":49152,"data_compressed_original":708608,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1007616,"data_stored":514716,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":987136,"data_stored":513428,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":937984,"data_stored":446930,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":331979,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":397033,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:46:01.475 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T08:46:01.476 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T08:46:01.729 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:46:01.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:01 smithi164 ceph-mon[30928]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-09-19T08:46:01.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:01 smithi164 ceph-mon[30928]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:46:01.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:01 smithi173 ceph-mon[35306]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-09-19T08:46:01.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:01 smithi173 ceph-mon[35306]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:46:02.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:01 smithi170 ceph-mon[35072]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-09-19T08:46:02.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:01 smithi170 ceph-mon[35072]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:46:02.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.210+0000 7fce72f61640 1 -- 172.21.15.164:0/1970471474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce6c102780 msgr2=0x7fce6c1067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:02.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.210+0000 7fce72f61640 1 --2- 172.21.15.164:0/1970471474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce6c102780 0x7fce6c1067d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fce5c009ca0 tx=0x7fce5c02a8c0 comp rx=0 tx=0).stop 2024-09-19T08:46:02.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.210+0000 7fce72f61640 1 -- 172.21.15.164:0/1970471474 shutdown_connections 2024-09-19T08:46:02.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.210+0000 7fce72f61640 1 --2- 172.21.15.164:0/1970471474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce6c106f00 0x7fce6c10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:02.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.210+0000 7fce72f61640 1 --2- 172.21.15.164:0/1970471474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce6c102780 0x7fce6c1067d0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:02.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.210+0000 7fce72f61640 1 --2- 172.21.15.164:0/1970471474 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce6c101dd0 0x7fce6c1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:02.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.210+0000 7fce72f61640 1 -- 172.21.15.164:0/1970471474 >> 172.21.15.164:0/1970471474 conn(0x7fce6c0fd8b0 msgr2=0x7fce6c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:02.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.210+0000 7fce72f61640 1 -- 172.21.15.164:0/1970471474 shutdown_connections 2024-09-19T08:46:02.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.210+0000 7fce72f61640 1 -- 172.21.15.164:0/1970471474 wait complete. 2024-09-19T08:46:02.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.210+0000 7fce72f61640 1 Processor -- start 2024-09-19T08:46:02.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.211+0000 7fce72f61640 1 -- start start 2024-09-19T08:46:02.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.211+0000 7fce72f61640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce6c101dd0 0x7fce6c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:02.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.211+0000 7fce72f61640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce6c102780 0x7fce6c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:02.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.211+0000 7fce72f61640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce6c106f00 0x7fce6c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:02.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.211+0000 7fce72f61640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c111f60 con 0x7fce6c106f00 2024-09-19T08:46:02.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.211+0000 7fce72f61640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c111de0 con 0x7fce6c102780 2024-09-19T08:46:02.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.211+0000 7fce72f61640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c1120e0 con 0x7fce6c101dd0 2024-09-19T08:46:02.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.211+0000 7fce714fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce6c106f00 0x7fce6c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:02.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.211+0000 7fce714fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce6c106f00 0x7fce6c19cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39478/0 (socket says 172.21.15.164:39478) 2024-09-19T08:46:02.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.211+0000 7fce714fe640 1 -- 172.21.15.164:0/1023665506 learned_addr learned my addr 172.21.15.164:0/1023665506 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:02.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.211+0000 7fce6bfff640 1 --2- 172.21.15.164:0/1023665506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce6c102780 0x7fce6c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:02.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.211+0000 7fce70cfd640 1 --2- 172.21.15.164:0/1023665506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce6c101dd0 0x7fce6c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:02.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.211+0000 7fce714fe640 1 -- 172.21.15.164:0/1023665506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce6c101dd0 msgr2=0x7fce6c198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:02.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.211+0000 7fce714fe640 1 --2- 172.21.15.164:0/1023665506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce6c101dd0 0x7fce6c198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:02.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.211+0000 7fce714fe640 1 -- 172.21.15.164:0/1023665506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce6c102780 msgr2=0x7fce6c198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:02.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.211+0000 7fce714fe640 1 --2- 172.21.15.164:0/1023665506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce6c102780 0x7fce6c198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:02.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.211+0000 7fce714fe640 1 -- 172.21.15.164:0/1023665506 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce6c19d3f0 con 0x7fce6c106f00 2024-09-19T08:46:02.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.211+0000 7fce714fe640 1 --2- 172.21.15.164:0/1023665506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce6c106f00 0x7fce6c19cd10 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fce6400d750 tx=0x7fce6400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:02.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.212+0000 7fce69ffb640 1 -- 172.21.15.164:0/1023665506 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce64017070 con 0x7fce6c106f00 2024-09-19T08:46:02.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.212+0000 7fce72f61640 1 -- 172.21.15.164:0/1023665506 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce6c19d6e0 con 0x7fce6c106f00 2024-09-19T08:46:02.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.212+0000 7fce69ffb640 1 -- 172.21.15.164:0/1023665506 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce64005500 con 0x7fce6c106f00 2024-09-19T08:46:02.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.212+0000 7fce69ffb640 1 -- 172.21.15.164:0/1023665506 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce64004d90 con 0x7fce6c106f00 2024-09-19T08:46:02.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.212+0000 7fce72f61640 1 -- 172.21.15.164:0/1023665506 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce6c1a4fc0 con 0x7fce6c106f00 2024-09-19T08:46:02.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.213+0000 7fce72f61640 1 -- 172.21.15.164:0/1023665506 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce6c1021b0 con 0x7fce6c106f00 2024-09-19T08:46:02.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.214+0000 7fce69ffb640 1 -- 172.21.15.164:0/1023665506 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7fce64004f30 con 0x7fce6c106f00 2024-09-19T08:46:02.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.214+0000 7fce69ffb640 1 --2- 172.21.15.164:0/1023665506 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fce580a3440 0x7fce580a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:02.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.214+0000 7fce69ffb640 1 -- 172.21.15.164:0/1023665506 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fce640cf820 con 0x7fce6c106f00 2024-09-19T08:46:02.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.214+0000 7fce70cfd640 1 --2- 172.21.15.164:0/1023665506 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fce580a3440 0x7fce580a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:02.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.215+0000 7fce70cfd640 1 --2- 172.21.15.164:0/1023665506 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fce580a3440 0x7fce580a5900 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fce54005fd0 tx=0x7fce54005850 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:02.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.219+0000 7fce69ffb640 1 -- 172.21.15.164:0/1023665506 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fce6409c0a0 con 0x7fce6c106f00 2024-09-19T08:46:02.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.382+0000 7fce72f61640 1 -- 172.21.15.164:0/1023665506 --> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fce6c062630 con 0x7fce580a3440 2024-09-19T08:46:02.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.390+0000 7fce69ffb640 1 -- 172.21.15.164:0/1023665506 <== mgr.15201 v2:172.21.15.164:6800/173486514 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258076 (secure 0 0 0) 0x7fce6c062630 con 0x7fce580a3440 2024-09-19T08:46:02.393 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:46:02.394 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-09-19T08:46:02.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.394+0000 7fce72f61640 1 -- 172.21.15.164:0/1023665506 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fce580a3440 msgr2=0x7fce580a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:02.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.394+0000 7fce72f61640 1 --2- 172.21.15.164:0/1023665506 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fce580a3440 0x7fce580a5900 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fce54005fd0 tx=0x7fce54005850 comp rx=0 tx=0).stop 2024-09-19T08:46:02.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.394+0000 7fce72f61640 1 -- 172.21.15.164:0/1023665506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce6c106f00 msgr2=0x7fce6c19cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:02.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.394+0000 7fce72f61640 1 --2- 172.21.15.164:0/1023665506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce6c106f00 0x7fce6c19cd10 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fce6400d750 tx=0x7fce6400dc10 comp rx=0 tx=0).stop 2024-09-19T08:46:02.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.394+0000 7fce72f61640 1 -- 172.21.15.164:0/1023665506 shutdown_connections 2024-09-19T08:46:02.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.394+0000 7fce72f61640 1 --2- 172.21.15.164:0/1023665506 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fce580a3440 0x7fce580a5900 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:02.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.394+0000 7fce72f61640 1 --2- 172.21.15.164:0/1023665506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce6c106f00 0x7fce6c19cd10 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:02.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.394+0000 7fce72f61640 1 --2- 172.21.15.164:0/1023665506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce6c102780 0x7fce6c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:02.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.394+0000 7fce72f61640 1 --2- 172.21.15.164:0/1023665506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce6c101dd0 0x7fce6c198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:02.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.394+0000 7fce72f61640 1 -- 172.21.15.164:0/1023665506 >> 172.21.15.164:0/1023665506 conn(0x7fce6c0fd8b0 msgr2=0x7fce6c0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:02.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.394+0000 7fce72f61640 1 -- 172.21.15.164:0/1023665506 shutdown_connections 2024-09-19T08:46:02.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:02.394+0000 7fce72f61640 1 -- 172.21.15.164:0/1023665506 wait complete. 2024-09-19T08:46:03.118 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-09-19T08:46:01.984303+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1388177,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1783,"num_read_kb":1325,"num_write":704,"num_write_kb":5320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1031,"ondisk_log_size":1031,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":389868,"kb_used_data":11460,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124551956,"statfs":{"total":1151940427776,"available":1151541202944,"internally_reserved":0,"allocated":11735040,"data_stored":9490431,"data_compressed":247596,"data_compressed_allocated":319488,"data_compressed_original":3969024,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":197770,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":457,"num_read_kb":495,"num_write":147,"num_write_kb":916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003001"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229741+0000","last_change":"2024-09-19T08:43:52.766709+0000","last_active":"2024-09-19T08:45:48.229741+0000","last_peered":"2024-09-19T08:45:48.229741+0000","last_clean":"2024-09-19T08:45:48.229741+0000","last_became_active":"2024-09-19T08:43:52.765970+0000","last_became_peered":"2024-09-19T08:43:52.765970+0000","last_unstale":"2024-09-19T08:45:48.229741+0000","last_undegraded":"2024-09-19T08:45:48.229741+0000","last_fullsized":"2024-09-19T08:45:48.229741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:12.176870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.230652+0000","last_change":"2024-09-19T08:43:52.766590+0000","last_active":"2024-09-19T08:45:48.230652+0000","last_peered":"2024-09-19T08:45:48.230652+0000","last_clean":"2024-09-19T08:45:48.230652+0000","last_became_active":"2024-09-19T08:43:52.766255+0000","last_became_peered":"2024-09-19T08:43:52.766255+0000","last_unstale":"2024-09-19T08:45:48.230652+0000","last_undegraded":"2024-09-19T08:45:48.230652+0000","last_fullsized":"2024-09-19T08:45:48.230652+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:15:41.359806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.230354+0000","last_change":"2024-09-19T08:43:52.761956+0000","last_active":"2024-09-19T08:45:48.230354+0000","last_peered":"2024-09-19T08:45:48.230354+0000","last_clean":"2024-09-19T08:45:48.230354+0000","last_became_active":"2024-09-19T08:43:52.761644+0000","last_became_peered":"2024-09-19T08:43:52.761644+0000","last_unstale":"2024-09-19T08:45:48.230354+0000","last_undegraded":"2024-09-19T08:45:48.230354+0000","last_fullsized":"2024-09-19T08:45:48.230354+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:07:13.770361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.966221+0000","last_change":"2024-09-19T08:43:52.764131+0000","last_active":"2024-09-19T08:45:47.966221+0000","last_peered":"2024-09-19T08:45:47.966221+0000","last_clean":"2024-09-19T08:45:47.966221+0000","last_became_active":"2024-09-19T08:43:52.763812+0000","last_became_peered":"2024-09-19T08:43:52.763812+0000","last_unstale":"2024-09-19T08:45:47.966221+0000","last_undegraded":"2024-09-19T08:45:47.966221+0000","last_fullsized":"2024-09-19T08:45:47.966221+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:32.272157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065874+0000","last_change":"2024-09-19T08:43:52.754673+0000","last_active":"2024-09-19T08:45:48.065874+0000","last_peered":"2024-09-19T08:45:48.065874+0000","last_clean":"2024-09-19T08:45:48.065874+0000","last_became_active":"2024-09-19T08:43:52.754144+0000","last_became_peered":"2024-09-19T08:43:52.754144+0000","last_unstale":"2024-09-19T08:45:48.065874+0000","last_undegraded":"2024-09-19T08:45:48.065874+0000","last_fullsized":"2024-09-19T08:45:48.065874+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:16.069260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.076773+0000","last_change":"2024-09-19T08:43:52.758939+0000","last_active":"2024-09-19T08:45:48.076773+0000","last_peered":"2024-09-19T08:45:48.076773+0000","last_clean":"2024-09-19T08:45:48.076773+0000","last_became_active":"2024-09-19T08:43:52.758611+0000","last_became_peered":"2024-09-19T08:43:52.758611+0000","last_unstale":"2024-09-19T08:45:48.076773+0000","last_undegraded":"2024-09-19T08:45:48.076773+0000","last_fullsized":"2024-09-19T08:45:48.076773+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:35:56.463545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221260+0000","last_change":"2024-09-19T08:43:52.770261+0000","last_active":"2024-09-19T08:45:48.221260+0000","last_peered":"2024-09-19T08:45:48.221260+0000","last_clean":"2024-09-19T08:45:48.221260+0000","last_became_active":"2024-09-19T08:43:52.768253+0000","last_became_peered":"2024-09-19T08:43:52.768253+0000","last_unstale":"2024-09-19T08:45:48.221260+0000","last_undegraded":"2024-09-19T08:45:48.221260+0000","last_fullsized":"2024-09-19T08:45:48.221260+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:40.756988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065907+0000","last_change":"2024-09-19T08:43:52.768564+0000","last_active":"2024-09-19T08:45:48.065907+0000","last_peered":"2024-09-19T08:45:48.065907+0000","last_clean":"2024-09-19T08:45:48.065907+0000","last_became_active":"2024-09-19T08:43:52.768260+0000","last_became_peered":"2024-09-19T08:43:52.768260+0000","last_unstale":"2024-09-19T08:45:48.065907+0000","last_undegraded":"2024-09-19T08:45:48.065907+0000","last_fullsized":"2024-09-19T08:45:48.065907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:16.528115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.222053+0000","last_change":"2024-09-19T08:43:52.769804+0000","last_active":"2024-09-19T08:45:48.222053+0000","last_peered":"2024-09-19T08:45:48.222053+0000","last_clean":"2024-09-19T08:45:48.222053+0000","last_became_active":"2024-09-19T08:43:52.769504+0000","last_became_peered":"2024-09-19T08:43:52.769504+0000","last_unstale":"2024-09-19T08:45:48.222053+0000","last_undegraded":"2024-09-19T08:45:48.222053+0000","last_fullsized":"2024-09-19T08:45:48.222053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:22:38.514410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.077170+0000","last_change":"2024-09-19T08:43:52.763278+0000","last_active":"2024-09-19T08:45:48.077170+0000","last_peered":"2024-09-19T08:45:48.077170+0000","last_clean":"2024-09-19T08:45:48.077170+0000","last_became_active":"2024-09-19T08:43:52.762985+0000","last_became_peered":"2024-09-19T08:43:52.762985+0000","last_unstale":"2024-09-19T08:45:48.077170+0000","last_undegraded":"2024-09-19T08:45:48.077170+0000","last_fullsized":"2024-09-19T08:45:48.077170+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:30.048782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221488+0000","last_change":"2024-09-19T08:43:52.770864+0000","last_active":"2024-09-19T08:45:48.221488+0000","last_peered":"2024-09-19T08:45:48.221488+0000","last_clean":"2024-09-19T08:45:48.221488+0000","last_became_active":"2024-09-19T08:43:52.770518+0000","last_became_peered":"2024-09-19T08:43:52.770518+0000","last_unstale":"2024-09-19T08:45:48.221488+0000","last_undegraded":"2024-09-19T08:45:48.221488+0000","last_fullsized":"2024-09-19T08:45:48.221488+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:08.931452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.966171+0000","last_change":"2024-09-19T08:43:52.757464+0000","last_active":"2024-09-19T08:45:47.966171+0000","last_peered":"2024-09-19T08:45:47.966171+0000","last_clean":"2024-09-19T08:45:47.966171+0000","last_became_active":"2024-09-19T08:43:52.757101+0000","last_became_peered":"2024-09-19T08:43:52.757101+0000","last_unstale":"2024-09-19T08:45:47.966171+0000","last_undegraded":"2024-09-19T08:45:47.966171+0000","last_fullsized":"2024-09-19T08:45:47.966171+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:17.395735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.224010+0000","last_change":"2024-09-19T08:43:52.767063+0000","last_active":"2024-09-19T08:45:48.224010+0000","last_peered":"2024-09-19T08:45:48.224010+0000","last_clean":"2024-09-19T08:45:48.224010+0000","last_became_active":"2024-09-19T08:43:52.766755+0000","last_became_peered":"2024-09-19T08:43:52.766755+0000","last_unstale":"2024-09-19T08:45:48.224010+0000","last_undegraded":"2024-09-19T08:45:48.224010+0000","last_fullsized":"2024-09-19T08:45:48.224010+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:30.530937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.230150+0000","last_change":"2024-09-19T08:43:52.766515+0000","last_active":"2024-09-19T08:45:48.230150+0000","last_peered":"2024-09-19T08:45:48.230150+0000","last_clean":"2024-09-19T08:45:48.230150+0000","last_became_active":"2024-09-19T08:43:52.765847+0000","last_became_peered":"2024-09-19T08:43:52.765847+0000","last_unstale":"2024-09-19T08:45:48.230150+0000","last_undegraded":"2024-09-19T08:45:48.230150+0000","last_fullsized":"2024-09-19T08:45:48.230150+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:50:35.948663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229847+0000","last_change":"2024-09-19T08:43:52.773702+0000","last_active":"2024-09-19T08:45:48.229847+0000","last_peered":"2024-09-19T08:45:48.229847+0000","last_clean":"2024-09-19T08:45:48.229847+0000","last_became_active":"2024-09-19T08:43:52.773324+0000","last_became_peered":"2024-09-19T08:43:52.773324+0000","last_unstale":"2024-09-19T08:45:48.229847+0000","last_undegraded":"2024-09-19T08:45:48.229847+0000","last_fullsized":"2024-09-19T08:45:48.229847+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:34.368014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221253+0000","last_change":"2024-09-19T08:43:52.762745+0000","last_active":"2024-09-19T08:45:48.221253+0000","last_peered":"2024-09-19T08:45:48.221253+0000","last_clean":"2024-09-19T08:45:48.221253+0000","last_became_active":"2024-09-19T08:43:52.762172+0000","last_became_peered":"2024-09-19T08:43:52.762172+0000","last_unstale":"2024-09-19T08:45:48.221253+0000","last_undegraded":"2024-09-19T08:45:48.221253+0000","last_fullsized":"2024-09-19T08:45:48.221253+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:57.369775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229743+0000","last_change":"2024-09-19T08:43:52.767118+0000","last_active":"2024-09-19T08:45:48.229743+0000","last_peered":"2024-09-19T08:45:48.229743+0000","last_clean":"2024-09-19T08:45:48.229743+0000","last_became_active":"2024-09-19T08:43:52.766388+0000","last_became_peered":"2024-09-19T08:43:52.766388+0000","last_unstale":"2024-09-19T08:45:48.229743+0000","last_undegraded":"2024-09-19T08:45:48.229743+0000","last_fullsized":"2024-09-19T08:45:48.229743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:34.182923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.967257+0000","last_change":"2024-09-19T08:43:52.765008+0000","last_active":"2024-09-19T08:45:47.967257+0000","last_peered":"2024-09-19T08:45:47.967257+0000","last_clean":"2024-09-19T08:45:47.967257+0000","last_became_active":"2024-09-19T08:43:52.764558+0000","last_became_peered":"2024-09-19T08:43:52.764558+0000","last_unstale":"2024-09-19T08:45:47.967257+0000","last_undegraded":"2024-09-19T08:45:47.967257+0000","last_fullsized":"2024-09-19T08:45:47.967257+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:52:42.756421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968722+0000","last_change":"2024-09-19T08:43:52.764376+0000","last_active":"2024-09-19T08:45:47.968722+0000","last_peered":"2024-09-19T08:45:47.968722+0000","last_clean":"2024-09-19T08:45:47.968722+0000","last_became_active":"2024-09-19T08:43:52.764048+0000","last_became_peered":"2024-09-19T08:43:52.764048+0000","last_unstale":"2024-09-19T08:45:47.968722+0000","last_undegraded":"2024-09-19T08:45:47.968722+0000","last_fullsized":"2024-09-19T08:45:47.968722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:56.732094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229417+0000","last_change":"2024-09-19T08:44:02.827925+0000","last_active":"2024-09-19T08:45:48.229417+0000","last_peered":"2024-09-19T08:45:48.229417+0000","last_clean":"2024-09-19T08:45:48.229417+0000","last_became_active":"2024-09-19T08:44:02.827269+0000","last_became_peered":"2024-09-19T08:44:02.827269+0000","last_unstale":"2024-09-19T08:45:48.229417+0000","last_undegraded":"2024-09-19T08:45:48.229417+0000","last_fullsized":"2024-09-19T08:45:48.229417+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:31:54.556786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.224157+0000","last_change":"2024-09-19T08:43:52.762190+0000","last_active":"2024-09-19T08:45:48.224157+0000","last_peered":"2024-09-19T08:45:48.224157+0000","last_clean":"2024-09-19T08:45:48.224157+0000","last_became_active":"2024-09-19T08:43:52.761818+0000","last_became_peered":"2024-09-19T08:43:52.761818+0000","last_unstale":"2024-09-19T08:45:48.224157+0000","last_undegraded":"2024-09-19T08:45:48.224157+0000","last_fullsized":"2024-09-19T08:45:48.224157+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:19:55.246779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221492+0000","last_change":"2024-09-19T08:43:52.761069+0000","last_active":"2024-09-19T08:45:48.221492+0000","last_peered":"2024-09-19T08:45:48.221492+0000","last_clean":"2024-09-19T08:45:48.221492+0000","last_became_active":"2024-09-19T08:43:52.760705+0000","last_became_peered":"2024-09-19T08:43:52.760705+0000","last_unstale":"2024-09-19T08:45:48.221492+0000","last_undegraded":"2024-09-19T08:45:48.221492+0000","last_fullsized":"2024-09-19T08:45:48.221492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:46:27.059026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065963+0000","last_change":"2024-09-19T08:43:52.773449+0000","last_active":"2024-09-19T08:45:48.065963+0000","last_peered":"2024-09-19T08:45:48.065963+0000","last_clean":"2024-09-19T08:45:48.065963+0000","last_became_active":"2024-09-19T08:43:52.772929+0000","last_became_peered":"2024-09-19T08:43:52.772929+0000","last_unstale":"2024-09-19T08:45:48.065963+0000","last_undegraded":"2024-09-19T08:45:48.065963+0000","last_fullsized":"2024-09-19T08:45:48.065963+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:00:38.929199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074018+0000","last_change":"2024-09-19T08:43:52.773885+0000","last_active":"2024-09-19T08:45:48.074018+0000","last_peered":"2024-09-19T08:45:48.074018+0000","last_clean":"2024-09-19T08:45:48.074018+0000","last_became_active":"2024-09-19T08:43:52.772856+0000","last_became_peered":"2024-09-19T08:43:52.772856+0000","last_unstale":"2024-09-19T08:45:48.074018+0000","last_undegraded":"2024-09-19T08:45:48.074018+0000","last_fullsized":"2024-09-19T08:45:48.074018+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:13:22.581335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.067279+0000","last_change":"2024-09-19T08:43:52.762909+0000","last_active":"2024-09-19T08:45:48.067279+0000","last_peered":"2024-09-19T08:45:48.067279+0000","last_clean":"2024-09-19T08:45:48.067279+0000","last_became_active":"2024-09-19T08:43:52.762600+0000","last_became_peered":"2024-09-19T08:43:52.762600+0000","last_unstale":"2024-09-19T08:45:48.067279+0000","last_undegraded":"2024-09-19T08:45:48.067279+0000","last_fullsized":"2024-09-19T08:45:48.067279+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:46.098294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968939+0000","last_change":"2024-09-19T08:43:52.762861+0000","last_active":"2024-09-19T08:45:47.968939+0000","last_peered":"2024-09-19T08:45:47.968939+0000","last_clean":"2024-09-19T08:45:47.968939+0000","last_became_active":"2024-09-19T08:43:52.762525+0000","last_became_peered":"2024-09-19T08:43:52.762525+0000","last_unstale":"2024-09-19T08:45:47.968939+0000","last_undegraded":"2024-09-19T08:45:47.968939+0000","last_fullsized":"2024-09-19T08:45:47.968939+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:54.717249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229602+0000","last_change":"2024-09-19T08:43:52.768370+0000","last_active":"2024-09-19T08:45:48.229602+0000","last_peered":"2024-09-19T08:45:48.229602+0000","last_clean":"2024-09-19T08:45:48.229602+0000","last_became_active":"2024-09-19T08:43:52.768055+0000","last_became_peered":"2024-09-19T08:43:52.768055+0000","last_unstale":"2024-09-19T08:45:48.229602+0000","last_undegraded":"2024-09-19T08:45:48.229602+0000","last_fullsized":"2024-09-19T08:45:48.229602+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:48.212862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.224066+0000","last_change":"2024-09-19T08:43:52.758541+0000","last_active":"2024-09-19T08:45:48.224066+0000","last_peered":"2024-09-19T08:45:48.224066+0000","last_clean":"2024-09-19T08:45:48.224066+0000","last_became_active":"2024-09-19T08:43:52.758122+0000","last_became_peered":"2024-09-19T08:43:52.758122+0000","last_unstale":"2024-09-19T08:45:48.224066+0000","last_undegraded":"2024-09-19T08:45:48.224066+0000","last_fullsized":"2024-09-19T08:45:48.224066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:54:40.681880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.231790+0000","last_change":"2024-09-19T08:43:52.758109+0000","last_active":"2024-09-19T08:45:48.231790+0000","last_peered":"2024-09-19T08:45:48.231790+0000","last_clean":"2024-09-19T08:45:48.231790+0000","last_became_active":"2024-09-19T08:43:52.757808+0000","last_became_peered":"2024-09-19T08:43:52.757808+0000","last_unstale":"2024-09-19T08:45:48.231790+0000","last_undegraded":"2024-09-19T08:45:48.231790+0000","last_fullsized":"2024-09-19T08:45:48.231790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:02:47.614562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968751+0000","last_change":"2024-09-19T08:43:52.772501+0000","last_active":"2024-09-19T08:45:47.968751+0000","last_peered":"2024-09-19T08:45:47.968751+0000","last_clean":"2024-09-19T08:45:47.968751+0000","last_became_active":"2024-09-19T08:43:52.771611+0000","last_became_peered":"2024-09-19T08:43:52.771611+0000","last_unstale":"2024-09-19T08:45:47.968751+0000","last_undegraded":"2024-09-19T08:45:47.968751+0000","last_fullsized":"2024-09-19T08:45:47.968751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:49.537237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968692+0000","last_change":"2024-09-19T08:43:52.773664+0000","last_active":"2024-09-19T08:45:47.968692+0000","last_peered":"2024-09-19T08:45:47.968692+0000","last_clean":"2024-09-19T08:45:47.968692+0000","last_became_active":"2024-09-19T08:43:52.773014+0000","last_became_peered":"2024-09-19T08:43:52.773014+0000","last_unstale":"2024-09-19T08:45:47.968692+0000","last_undegraded":"2024-09-19T08:45:47.968692+0000","last_fullsized":"2024-09-19T08:45:47.968692+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:03.448825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229553+0000","last_change":"2024-09-19T08:43:52.769578+0000","last_active":"2024-09-19T08:45:48.229553+0000","last_peered":"2024-09-19T08:45:48.229553+0000","last_clean":"2024-09-19T08:45:48.229553+0000","last_became_active":"2024-09-19T08:43:52.769252+0000","last_became_peered":"2024-09-19T08:43:52.769252+0000","last_unstale":"2024-09-19T08:45:48.229553+0000","last_undegraded":"2024-09-19T08:45:48.229553+0000","last_fullsized":"2024-09-19T08:45:48.229553+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:27.936118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065844+0000","last_change":"2024-09-19T08:43:52.770453+0000","last_active":"2024-09-19T08:45:48.065844+0000","last_peered":"2024-09-19T08:45:48.065844+0000","last_clean":"2024-09-19T08:45:48.065844+0000","last_became_active":"2024-09-19T08:43:52.770155+0000","last_became_peered":"2024-09-19T08:43:52.770155+0000","last_unstale":"2024-09-19T08:45:48.065844+0000","last_undegraded":"2024-09-19T08:45:48.065844+0000","last_fullsized":"2024-09-19T08:45:48.065844+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:12.893572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'8","reported_seq":152,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:57.794654+0000","last_change":"2024-09-19T08:43:50.751316+0000","last_active":"2024-09-19T08:45:57.794654+0000","last_peered":"2024-09-19T08:45:57.794654+0000","last_clean":"2024-09-19T08:45:57.794654+0000","last_became_active":"2024-09-19T08:43:50.750972+0000","last_became_peered":"2024-09-19T08:43:50.750972+0000","last_unstale":"2024-09-19T08:45:57.794654+0000","last_undegraded":"2024-09-19T08:45:57.794654+0000","last_fullsized":"2024-09-19T08:45:57.794654+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:40.654729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1549,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074105+0000","last_change":"2024-09-19T08:43:52.773979+0000","last_active":"2024-09-19T08:45:48.074105+0000","last_peered":"2024-09-19T08:45:48.074105+0000","last_clean":"2024-09-19T08:45:48.074105+0000","last_became_active":"2024-09-19T08:43:52.773648+0000","last_became_peered":"2024-09-19T08:43:52.773648+0000","last_unstale":"2024-09-19T08:45:48.074105+0000","last_undegraded":"2024-09-19T08:45:48.074105+0000","last_fullsized":"2024-09-19T08:45:48.074105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:36.061209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.076810+0000","last_change":"2024-09-19T08:43:50.750704+0000","last_active":"2024-09-19T08:45:48.076810+0000","last_peered":"2024-09-19T08:45:48.076810+0000","last_clean":"2024-09-19T08:45:48.076810+0000","last_became_active":"2024-09-19T08:43:50.750399+0000","last_became_peered":"2024-09-19T08:43:50.750399+0000","last_unstale":"2024-09-19T08:45:48.076810+0000","last_undegraded":"2024-09-19T08:45:48.076810+0000","last_fullsized":"2024-09-19T08:45:48.076810+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:25.447320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.223615+0000","last_change":"2024-09-19T08:43:50.744919+0000","last_active":"2024-09-19T08:45:48.223615+0000","last_peered":"2024-09-19T08:45:48.223615+0000","last_clean":"2024-09-19T08:45:48.223615+0000","last_became_active":"2024-09-19T08:43:50.744604+0000","last_became_peered":"2024-09-19T08:43:50.744604+0000","last_unstale":"2024-09-19T08:45:48.223615+0000","last_undegraded":"2024-09-19T08:45:48.223615+0000","last_fullsized":"2024-09-19T08:45:48.223615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:25:06.410093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065913+0000","last_change":"2024-09-19T08:43:52.766779+0000","last_active":"2024-09-19T08:45:48.065913+0000","last_peered":"2024-09-19T08:45:48.065913+0000","last_clean":"2024-09-19T08:45:48.065913+0000","last_became_active":"2024-09-19T08:43:52.766484+0000","last_became_peered":"2024-09-19T08:43:52.766484+0000","last_unstale":"2024-09-19T08:45:48.065913+0000","last_undegraded":"2024-09-19T08:45:48.065913+0000","last_fullsized":"2024-09-19T08:45:48.065913+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:17:29.350778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074289+0000","last_change":"2024-09-19T08:43:50.751282+0000","last_active":"2024-09-19T08:45:48.074289+0000","last_peered":"2024-09-19T08:45:48.074289+0000","last_clean":"2024-09-19T08:45:48.074289+0000","last_became_active":"2024-09-19T08:43:50.750754+0000","last_became_peered":"2024-09-19T08:43:50.750754+0000","last_unstale":"2024-09-19T08:45:48.074289+0000","last_undegraded":"2024-09-19T08:45:48.074289+0000","last_fullsized":"2024-09-19T08:45:48.074289+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:11.661626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.967219+0000","last_change":"2024-09-19T08:43:52.756442+0000","last_active":"2024-09-19T08:45:47.967219+0000","last_peered":"2024-09-19T08:45:47.967219+0000","last_clean":"2024-09-19T08:45:47.967219+0000","last_became_active":"2024-09-19T08:43:52.756116+0000","last_became_peered":"2024-09-19T08:43:52.756116+0000","last_unstale":"2024-09-19T08:45:47.967219+0000","last_undegraded":"2024-09-19T08:45:47.967219+0000","last_fullsized":"2024-09-19T08:45:47.967219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:40.029272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'8","reported_seq":262,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:58.795230+0000","last_change":"2024-09-19T08:43:50.749637+0000","last_active":"2024-09-19T08:45:58.795230+0000","last_peered":"2024-09-19T08:45:58.795230+0000","last_clean":"2024-09-19T08:45:58.795230+0000","last_became_active":"2024-09-19T08:43:50.749307+0000","last_became_peered":"2024-09-19T08:43:50.749307+0000","last_unstale":"2024-09-19T08:45:58.795230+0000","last_undegraded":"2024-09-19T08:45:58.795230+0000","last_fullsized":"2024-09-19T08:45:58.795230+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:23:33.409671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":230,"num_read_kb":230,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.222016+0000","last_change":"2024-09-19T08:43:52.763592+0000","last_active":"2024-09-19T08:45:48.222016+0000","last_peered":"2024-09-19T08:45:48.222016+0000","last_clean":"2024-09-19T08:45:48.222016+0000","last_became_active":"2024-09-19T08:43:52.763146+0000","last_became_peered":"2024-09-19T08:43:52.763146+0000","last_unstale":"2024-09-19T08:45:48.222016+0000","last_undegraded":"2024-09-19T08:45:48.222016+0000","last_fullsized":"2024-09-19T08:45:48.222016+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:32.194465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221748+0000","last_change":"2024-09-19T08:43:50.749184+0000","last_active":"2024-09-19T08:45:48.221748+0000","last_peered":"2024-09-19T08:45:48.221748+0000","last_clean":"2024-09-19T08:45:48.221748+0000","last_became_active":"2024-09-19T08:43:50.748844+0000","last_became_peered":"2024-09-19T08:43:50.748844+0000","last_unstale":"2024-09-19T08:45:48.221748+0000","last_undegraded":"2024-09-19T08:45:48.221748+0000","last_fullsized":"2024-09-19T08:45:48.221748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:15.334759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.231209+0000","last_change":"2024-09-19T08:43:52.768696+0000","last_active":"2024-09-19T08:45:48.231209+0000","last_peered":"2024-09-19T08:45:48.231209+0000","last_clean":"2024-09-19T08:45:48.231209+0000","last_became_active":"2024-09-19T08:43:52.768374+0000","last_became_peered":"2024-09-19T08:43:52.768374+0000","last_unstale":"2024-09-19T08:45:48.231209+0000","last_undegraded":"2024-09-19T08:45:48.231209+0000","last_fullsized":"2024-09-19T08:45:48.231209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:34:39.343041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074342+0000","last_change":"2024-09-19T08:43:51.026270+0000","last_active":"2024-09-19T08:45:48.074342+0000","last_peered":"2024-09-19T08:45:48.074342+0000","last_clean":"2024-09-19T08:45:48.074342+0000","last_became_active":"2024-09-19T08:43:51.025823+0000","last_became_peered":"2024-09-19T08:43:51.025823+0000","last_unstale":"2024-09-19T08:45:48.074342+0000","last_undegraded":"2024-09-19T08:45:48.074342+0000","last_fullsized":"2024-09-19T08:45:48.074342+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:00.480438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221712+0000","last_change":"2024-09-19T08:43:52.759555+0000","last_active":"2024-09-19T08:45:48.221712+0000","last_peered":"2024-09-19T08:45:48.221712+0000","last_clean":"2024-09-19T08:45:48.221712+0000","last_became_active":"2024-09-19T08:43:52.759223+0000","last_became_peered":"2024-09-19T08:43:52.759223+0000","last_unstale":"2024-09-19T08:45:48.221712+0000","last_undegraded":"2024-09-19T08:45:48.221712+0000","last_fullsized":"2024-09-19T08:45:48.221712+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:57:06.451323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'37","reported_seq":306,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:58.795355+0000","last_change":"2024-09-19T08:43:50.751924+0000","last_active":"2024-09-19T08:45:58.795355+0000","last_peered":"2024-09-19T08:45:58.795355+0000","last_clean":"2024-09-19T08:45:58.795355+0000","last_became_active":"2024-09-19T08:43:50.751117+0000","last_became_peered":"2024-09-19T08:43:50.751117+0000","last_unstale":"2024-09-19T08:45:58.795355+0000","last_undegraded":"2024-09-19T08:45:58.795355+0000","last_fullsized":"2024-09-19T08:45:58.795355+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:33:57.196283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67065,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":245,"num_read_kb":189,"num_write":39,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066156+0000","last_change":"2024-09-19T08:43:52.762260+0000","last_active":"2024-09-19T08:45:48.066156+0000","last_peered":"2024-09-19T08:45:48.066156+0000","last_clean":"2024-09-19T08:45:48.066156+0000","last_became_active":"2024-09-19T08:43:52.761923+0000","last_became_peered":"2024-09-19T08:43:52.761923+0000","last_unstale":"2024-09-19T08:45:48.066156+0000","last_undegraded":"2024-09-19T08:45:48.066156+0000","last_fullsized":"2024-09-19T08:45:48.066156+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:39:39.503332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968758+0000","last_change":"2024-09-19T08:43:50.749745+0000","last_active":"2024-09-19T08:45:47.968758+0000","last_peered":"2024-09-19T08:45:47.968758+0000","last_clean":"2024-09-19T08:45:47.968758+0000","last_became_active":"2024-09-19T08:43:50.749073+0000","last_became_peered":"2024-09-19T08:43:50.749073+0000","last_unstale":"2024-09-19T08:45:47.968758+0000","last_undegraded":"2024-09-19T08:45:47.968758+0000","last_fullsized":"2024-09-19T08:45:47.968758+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:48:11.965841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.223734+0000","last_change":"2024-09-19T08:43:52.761495+0000","last_active":"2024-09-19T08:45:48.223734+0000","last_peered":"2024-09-19T08:45:48.223734+0000","last_clean":"2024-09-19T08:45:48.223734+0000","last_became_active":"2024-09-19T08:43:52.760425+0000","last_became_peered":"2024-09-19T08:43:52.760425+0000","last_unstale":"2024-09-19T08:45:48.223734+0000","last_undegraded":"2024-09-19T08:45:48.223734+0000","last_fullsized":"2024-09-19T08:45:48.223734+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:57.689284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074090+0000","last_change":"2024-09-19T08:43:50.749563+0000","last_active":"2024-09-19T08:45:48.074090+0000","last_peered":"2024-09-19T08:45:48.074090+0000","last_clean":"2024-09-19T08:45:48.074090+0000","last_became_active":"2024-09-19T08:43:50.749244+0000","last_became_peered":"2024-09-19T08:43:50.749244+0000","last_unstale":"2024-09-19T08:45:48.074090+0000","last_undegraded":"2024-09-19T08:45:48.074090+0000","last_fullsized":"2024-09-19T08:45:48.074090+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:59:02.312883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221429+0000","last_change":"2024-09-19T08:43:52.767087+0000","last_active":"2024-09-19T08:45:48.221429+0000","last_peered":"2024-09-19T08:45:48.221429+0000","last_clean":"2024-09-19T08:45:48.221429+0000","last_became_active":"2024-09-19T08:43:52.766730+0000","last_became_peered":"2024-09-19T08:43:52.766730+0000","last_unstale":"2024-09-19T08:45:48.221429+0000","last_undegraded":"2024-09-19T08:45:48.221429+0000","last_fullsized":"2024-09-19T08:45:48.221429+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:04.040896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":146,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:57.806660+0000","last_change":"2024-09-19T08:43:50.749659+0000","last_active":"2024-09-19T08:45:57.806660+0000","last_peered":"2024-09-19T08:45:57.806660+0000","last_clean":"2024-09-19T08:45:57.806660+0000","last_became_active":"2024-09-19T08:43:50.749314+0000","last_became_peered":"2024-09-19T08:43:50.749314+0000","last_unstale":"2024-09-19T08:45:57.806660+0000","last_undegraded":"2024-09-19T08:45:57.806660+0000","last_fullsized":"2024-09-19T08:45:57.806660+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:08.576209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221691+0000","last_change":"2024-09-19T08:43:52.767928+0000","last_active":"2024-09-19T08:45:48.221691+0000","last_peered":"2024-09-19T08:45:48.221691+0000","last_clean":"2024-09-19T08:45:48.221691+0000","last_became_active":"2024-09-19T08:43:52.767580+0000","last_became_peered":"2024-09-19T08:43:52.767580+0000","last_unstale":"2024-09-19T08:45:48.221691+0000","last_undegraded":"2024-09-19T08:45:48.221691+0000","last_fullsized":"2024-09-19T08:45:48.221691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:45.344693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229348+0000","last_change":"2024-09-19T08:44:02.833887+0000","last_active":"2024-09-19T08:45:48.229348+0000","last_peered":"2024-09-19T08:45:48.229348+0000","last_clean":"2024-09-19T08:45:48.229348+0000","last_became_active":"2024-09-19T08:44:02.821624+0000","last_became_peered":"2024-09-19T08:44:02.821624+0000","last_unstale":"2024-09-19T08:45:48.229348+0000","last_undegraded":"2024-09-19T08:45:48.229348+0000","last_fullsized":"2024-09-19T08:45:48.229348+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:54:36.131949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.967019+0000","last_change":"2024-09-19T08:44:03.080503+0000","last_active":"2024-09-19T08:45:47.967019+0000","last_peered":"2024-09-19T08:45:47.967019+0000","last_clean":"2024-09-19T08:45:47.967019+0000","last_became_active":"2024-09-19T08:44:03.080129+0000","last_became_peered":"2024-09-19T08:44:03.080129+0000","last_unstale":"2024-09-19T08:45:47.967019+0000","last_undegraded":"2024-09-19T08:45:47.967019+0000","last_fullsized":"2024-09-19T08:45:47.967019+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:01:37.034795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066192+0000","last_change":"2024-09-19T08:43:50.747624+0000","last_active":"2024-09-19T08:45:48.066192+0000","last_peered":"2024-09-19T08:45:48.066192+0000","last_clean":"2024-09-19T08:45:48.066192+0000","last_became_active":"2024-09-19T08:43:50.747263+0000","last_became_peered":"2024-09-19T08:43:50.747263+0000","last_unstale":"2024-09-19T08:45:48.066192+0000","last_undegraded":"2024-09-19T08:45:48.066192+0000","last_fullsized":"2024-09-19T08:45:48.066192+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:18.580106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229525+0000","last_change":"2024-09-19T08:43:52.770051+0000","last_active":"2024-09-19T08:45:48.229525+0000","last_peered":"2024-09-19T08:45:48.229525+0000","last_clean":"2024-09-19T08:45:48.229525+0000","last_became_active":"2024-09-19T08:43:52.769727+0000","last_became_peered":"2024-09-19T08:43:52.769727+0000","last_unstale":"2024-09-19T08:45:48.229525+0000","last_undegraded":"2024-09-19T08:45:48.229525+0000","last_fullsized":"2024-09-19T08:45:48.229525+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:34.954897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.223984+0000","last_change":"2024-09-19T08:43:50.746696+0000","last_active":"2024-09-19T08:45:48.223984+0000","last_peered":"2024-09-19T08:45:48.223984+0000","last_clean":"2024-09-19T08:45:48.223984+0000","last_became_active":"2024-09-19T08:43:50.746246+0000","last_became_peered":"2024-09-19T08:43:50.746246+0000","last_unstale":"2024-09-19T08:45:48.223984+0000","last_undegraded":"2024-09-19T08:45:48.223984+0000","last_fullsized":"2024-09-19T08:45:48.223984+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:38:39.935348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.222174+0000","last_change":"2024-09-19T08:43:52.765199+0000","last_active":"2024-09-19T08:45:48.222174+0000","last_peered":"2024-09-19T08:45:48.222174+0000","last_clean":"2024-09-19T08:45:48.222174+0000","last_became_active":"2024-09-19T08:43:52.764896+0000","last_became_peered":"2024-09-19T08:43:52.764896+0000","last_unstale":"2024-09-19T08:45:48.222174+0000","last_undegraded":"2024-09-19T08:45:48.222174+0000","last_fullsized":"2024-09-19T08:45:48.222174+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:02:59.570018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'67","reported_seq":122,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:57.438919+0000","last_change":"2024-09-19T08:43:50.747242+0000","last_active":"2024-09-19T08:45:57.438919+0000","last_peered":"2024-09-19T08:45:57.438919+0000","last_clean":"2024-09-19T08:45:57.438919+0000","last_became_active":"2024-09-19T08:43:50.746853+0000","last_became_peered":"2024-09-19T08:43:50.746853+0000","last_unstale":"2024-09-19T08:45:57.438919+0000","last_undegraded":"2024-09-19T08:45:57.438919+0000","last_fullsized":"2024-09-19T08:45:57.438919+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:21.545635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221579+0000","last_change":"2024-09-19T08:44:02.826535+0000","last_active":"2024-09-19T08:45:48.221579+0000","last_peered":"2024-09-19T08:45:48.221579+0000","last_clean":"2024-09-19T08:45:48.221579+0000","last_became_active":"2024-09-19T08:44:02.826113+0000","last_became_peered":"2024-09-19T08:44:02.826113+0000","last_unstale":"2024-09-19T08:45:48.221579+0000","last_undegraded":"2024-09-19T08:45:48.221579+0000","last_fullsized":"2024-09-19T08:45:48.221579+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:20.247732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'8","reported_seq":152,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:57.794457+0000","last_change":"2024-09-19T08:43:51.131540+0000","last_active":"2024-09-19T08:45:57.794457+0000","last_peered":"2024-09-19T08:45:57.794457+0000","last_clean":"2024-09-19T08:45:57.794457+0000","last_became_active":"2024-09-19T08:43:51.131170+0000","last_became_peered":"2024-09-19T08:43:51.131170+0000","last_unstale":"2024-09-19T08:45:57.794457+0000","last_undegraded":"2024-09-19T08:45:57.794457+0000","last_fullsized":"2024-09-19T08:45:57.794457+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:43.635612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221593+0000","last_change":"2024-09-19T08:43:52.763308+0000","last_active":"2024-09-19T08:45:48.221593+0000","last_peered":"2024-09-19T08:45:48.221593+0000","last_clean":"2024-09-19T08:45:48.221593+0000","last_became_active":"2024-09-19T08:43:52.762859+0000","last_became_peered":"2024-09-19T08:43:52.762859+0000","last_unstale":"2024-09-19T08:45:48.221593+0000","last_undegraded":"2024-09-19T08:45:48.221593+0000","last_fullsized":"2024-09-19T08:45:48.221593+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:32:35.083361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"83'148","reported_seq":394,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:50.085307+0000","last_change":"2024-09-19T08:42:39.567408+0000","last_active":"2024-09-19T08:45:50.085307+0000","last_peered":"2024-09-19T08:45:50.085307+0000","last_clean":"2024-09-19T08:45:50.085307+0000","last_became_active":"2024-09-19T08:42:39.566967+0000","last_became_peered":"2024-09-19T08:42:39.566967+0000","last_unstale":"2024-09-19T08:45:50.085307+0000","last_undegraded":"2024-09-19T08:45:50.085307+0000","last_fullsized":"2024-09-19T08:45:50.085307+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:14.287944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:14.287944+0000","last_clean_scrub_stamp":"2024-09-19T08:39:14.287944+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:31:03.245386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.966315+0000","last_change":"2024-09-19T08:43:50.751097+0000","last_active":"2024-09-19T08:45:47.966315+0000","last_peered":"2024-09-19T08:45:47.966315+0000","last_clean":"2024-09-19T08:45:47.966315+0000","last_became_active":"2024-09-19T08:43:50.750341+0000","last_became_peered":"2024-09-19T08:43:50.750341+0000","last_unstale":"2024-09-19T08:45:47.966315+0000","last_undegraded":"2024-09-19T08:45:47.966315+0000","last_fullsized":"2024-09-19T08:45:47.966315+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:25.694535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066417+0000","last_change":"2024-09-19T08:43:52.767905+0000","last_active":"2024-09-19T08:45:48.066417+0000","last_peered":"2024-09-19T08:45:48.066417+0000","last_clean":"2024-09-19T08:45:48.066417+0000","last_became_active":"2024-09-19T08:43:52.767601+0000","last_became_peered":"2024-09-19T08:43:52.767601+0000","last_unstale":"2024-09-19T08:45:48.066417+0000","last_undegraded":"2024-09-19T08:45:48.066417+0000","last_fullsized":"2024-09-19T08:45:48.066417+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:14.519879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221389+0000","last_change":"2024-09-19T08:43:50.751397+0000","last_active":"2024-09-19T08:45:48.221389+0000","last_peered":"2024-09-19T08:45:48.221389+0000","last_clean":"2024-09-19T08:45:48.221389+0000","last_became_active":"2024-09-19T08:43:50.751079+0000","last_became_peered":"2024-09-19T08:43:50.751079+0000","last_unstale":"2024-09-19T08:45:48.221389+0000","last_undegraded":"2024-09-19T08:45:48.221389+0000","last_fullsized":"2024-09-19T08:45:48.221389+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:29.904786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.223683+0000","last_change":"2024-09-19T08:43:52.773348+0000","last_active":"2024-09-19T08:45:48.223683+0000","last_peered":"2024-09-19T08:45:48.223683+0000","last_clean":"2024-09-19T08:45:48.223683+0000","last_became_active":"2024-09-19T08:43:52.773011+0000","last_became_peered":"2024-09-19T08:43:52.773011+0000","last_unstale":"2024-09-19T08:45:48.223683+0000","last_undegraded":"2024-09-19T08:45:48.223683+0000","last_fullsized":"2024-09-19T08:45:48.223683+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:10.806578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":146,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:57.805846+0000","last_change":"2024-09-19T08:43:51.025300+0000","last_active":"2024-09-19T08:45:57.805846+0000","last_peered":"2024-09-19T08:45:57.805846+0000","last_clean":"2024-09-19T08:45:57.805846+0000","last_became_active":"2024-09-19T08:43:51.024936+0000","last_became_peered":"2024-09-19T08:43:51.024936+0000","last_unstale":"2024-09-19T08:45:57.805846+0000","last_undegraded":"2024-09-19T08:45:57.805846+0000","last_fullsized":"2024-09-19T08:45:57.805846+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:03.186066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074113+0000","last_change":"2024-09-19T08:43:52.774962+0000","last_active":"2024-09-19T08:45:48.074113+0000","last_peered":"2024-09-19T08:45:48.074113+0000","last_clean":"2024-09-19T08:45:48.074113+0000","last_became_active":"2024-09-19T08:43:52.774598+0000","last_became_peered":"2024-09-19T08:43:52.774598+0000","last_unstale":"2024-09-19T08:45:48.074113+0000","last_undegraded":"2024-09-19T08:45:48.074113+0000","last_fullsized":"2024-09-19T08:45:48.074113+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:02.687136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'7","reported_seq":148,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:57.794028+0000","last_change":"2024-09-19T08:43:50.748180+0000","last_active":"2024-09-19T08:45:57.794028+0000","last_peered":"2024-09-19T08:45:57.794028+0000","last_clean":"2024-09-19T08:45:57.794028+0000","last_became_active":"2024-09-19T08:43:50.747853+0000","last_became_peered":"2024-09-19T08:43:50.747853+0000","last_unstale":"2024-09-19T08:45:57.794028+0000","last_undegraded":"2024-09-19T08:45:57.794028+0000","last_fullsized":"2024-09-19T08:45:57.794028+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:56.047259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.077066+0000","last_change":"2024-09-19T08:43:52.770253+0000","last_active":"2024-09-19T08:45:48.077066+0000","last_peered":"2024-09-19T08:45:48.077066+0000","last_clean":"2024-09-19T08:45:48.077066+0000","last_became_active":"2024-09-19T08:43:52.769951+0000","last_became_peered":"2024-09-19T08:43:52.769951+0000","last_unstale":"2024-09-19T08:45:48.077066+0000","last_undegraded":"2024-09-19T08:45:48.077066+0000","last_fullsized":"2024-09-19T08:45:48.077066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:02.353144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.967308+0000","last_change":"2024-09-19T08:43:51.026453+0000","last_active":"2024-09-19T08:45:47.967308+0000","last_peered":"2024-09-19T08:45:47.967308+0000","last_clean":"2024-09-19T08:45:47.967308+0000","last_became_active":"2024-09-19T08:43:51.025873+0000","last_became_peered":"2024-09-19T08:43:51.025873+0000","last_unstale":"2024-09-19T08:45:47.967308+0000","last_undegraded":"2024-09-19T08:45:47.967308+0000","last_fullsized":"2024-09-19T08:45:47.967308+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:25:17.475074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.076927+0000","last_change":"2024-09-19T08:43:52.761700+0000","last_active":"2024-09-19T08:45:48.076927+0000","last_peered":"2024-09-19T08:45:48.076927+0000","last_clean":"2024-09-19T08:45:48.076927+0000","last_became_active":"2024-09-19T08:43:52.755527+0000","last_became_peered":"2024-09-19T08:43:52.755527+0000","last_unstale":"2024-09-19T08:45:48.076927+0000","last_undegraded":"2024-09-19T08:45:48.076927+0000","last_fullsized":"2024-09-19T08:45:48.076927+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:18:43.171284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066342+0000","last_change":"2024-09-19T08:43:51.026136+0000","last_active":"2024-09-19T08:45:48.066342+0000","last_peered":"2024-09-19T08:45:48.066342+0000","last_clean":"2024-09-19T08:45:48.066342+0000","last_became_active":"2024-09-19T08:43:51.025740+0000","last_became_peered":"2024-09-19T08:43:51.025740+0000","last_unstale":"2024-09-19T08:45:48.066342+0000","last_undegraded":"2024-09-19T08:45:48.066342+0000","last_fullsized":"2024-09-19T08:45:48.066342+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:17:32.741083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.224195+0000","last_change":"2024-09-19T08:43:52.763808+0000","last_active":"2024-09-19T08:45:48.224195+0000","last_peered":"2024-09-19T08:45:48.224195+0000","last_clean":"2024-09-19T08:45:48.224195+0000","last_became_active":"2024-09-19T08:43:52.763509+0000","last_became_peered":"2024-09-19T08:43:52.763509+0000","last_unstale":"2024-09-19T08:45:48.224195+0000","last_undegraded":"2024-09-19T08:45:48.224195+0000","last_fullsized":"2024-09-19T08:45:48.224195+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:45:08.385419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'104","reported_seq":221,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:57.778372+0000","last_change":"2024-09-19T08:43:50.744213+0000","last_active":"2024-09-19T08:45:57.778372+0000","last_peered":"2024-09-19T08:45:57.778372+0000","last_clean":"2024-09-19T08:45:57.778372+0000","last_became_active":"2024-09-19T08:43:50.743848+0000","last_became_peered":"2024-09-19T08:43:50.743848+0000","last_unstale":"2024-09-19T08:45:57.778372+0000","last_undegraded":"2024-09-19T08:45:57.778372+0000","last_fullsized":"2024-09-19T08:45:57.778372+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:26.613055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":189,"num_write_kb":1285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.067394+0000","last_change":"2024-09-19T08:43:52.767102+0000","last_active":"2024-09-19T08:45:48.067394+0000","last_peered":"2024-09-19T08:45:48.067394+0000","last_clean":"2024-09-19T08:45:48.067394+0000","last_became_active":"2024-09-19T08:43:52.766262+0000","last_became_peered":"2024-09-19T08:43:52.766262+0000","last_unstale":"2024-09-19T08:45:48.067394+0000","last_undegraded":"2024-09-19T08:45:48.067394+0000","last_fullsized":"2024-09-19T08:45:48.067394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:35.740462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.966517+0000","last_change":"2024-09-19T08:43:51.131742+0000","last_active":"2024-09-19T08:45:47.966517+0000","last_peered":"2024-09-19T08:45:47.966517+0000","last_clean":"2024-09-19T08:45:47.966517+0000","last_became_active":"2024-09-19T08:43:51.131388+0000","last_became_peered":"2024-09-19T08:43:51.131388+0000","last_unstale":"2024-09-19T08:45:47.966517+0000","last_undegraded":"2024-09-19T08:45:47.966517+0000","last_fullsized":"2024-09-19T08:45:47.966517+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:26.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.230731+0000","last_change":"2024-09-19T08:43:52.756004+0000","last_active":"2024-09-19T08:45:48.230731+0000","last_peered":"2024-09-19T08:45:48.230731+0000","last_clean":"2024-09-19T08:45:48.230731+0000","last_became_active":"2024-09-19T08:43:52.755596+0000","last_became_peered":"2024-09-19T08:43:52.755596+0000","last_unstale":"2024-09-19T08:45:48.230731+0000","last_undegraded":"2024-09-19T08:45:48.230731+0000","last_fullsized":"2024-09-19T08:45:48.230731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:35.942375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221620+0000","last_change":"2024-09-19T08:44:02.835986+0000","last_active":"2024-09-19T08:45:48.221620+0000","last_peered":"2024-09-19T08:45:48.221620+0000","last_clean":"2024-09-19T08:45:48.221620+0000","last_became_active":"2024-09-19T08:44:02.826314+0000","last_became_peered":"2024-09-19T08:44:02.826314+0000","last_unstale":"2024-09-19T08:45:48.221620+0000","last_undegraded":"2024-09-19T08:45:48.221620+0000","last_fullsized":"2024-09-19T08:45:48.221620+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:16.943989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065838+0000","last_change":"2024-09-19T08:43:52.766582+0000","last_active":"2024-09-19T08:45:48.065838+0000","last_peered":"2024-09-19T08:45:48.065838+0000","last_clean":"2024-09-19T08:45:48.065838+0000","last_became_active":"2024-09-19T08:43:52.766232+0000","last_became_peered":"2024-09-19T08:43:52.766232+0000","last_unstale":"2024-09-19T08:45:48.065838+0000","last_undegraded":"2024-09-19T08:45:48.065838+0000","last_fullsized":"2024-09-19T08:45:48.065838+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:43:42.319422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066195+0000","last_change":"2024-09-19T08:43:50.747611+0000","last_active":"2024-09-19T08:45:48.066195+0000","last_peered":"2024-09-19T08:45:48.066195+0000","last_clean":"2024-09-19T08:45:48.066195+0000","last_became_active":"2024-09-19T08:43:50.747263+0000","last_became_peered":"2024-09-19T08:43:50.747263+0000","last_unstale":"2024-09-19T08:45:48.066195+0000","last_undegraded":"2024-09-19T08:45:48.066195+0000","last_fullsized":"2024-09-19T08:45:48.066195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:14.717820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.229532+0000","last_change":"2024-09-19T08:43:52.766493+0000","last_active":"2024-09-19T08:45:48.229532+0000","last_peered":"2024-09-19T08:45:48.229532+0000","last_clean":"2024-09-19T08:45:48.229532+0000","last_became_active":"2024-09-19T08:43:52.766158+0000","last_became_peered":"2024-09-19T08:43:52.766158+0000","last_unstale":"2024-09-19T08:45:48.229532+0000","last_undegraded":"2024-09-19T08:45:48.229532+0000","last_fullsized":"2024-09-19T08:45:48.229532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:59.175949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:47.968841+0000","last_change":"2024-09-19T08:43:50.751368+0000","last_active":"2024-09-19T08:45:47.968841+0000","last_peered":"2024-09-19T08:45:47.968841+0000","last_clean":"2024-09-19T08:45:47.968841+0000","last_became_active":"2024-09-19T08:43:50.751038+0000","last_became_peered":"2024-09-19T08:43:50.751038+0000","last_unstale":"2024-09-19T08:45:47.968841+0000","last_undegraded":"2024-09-19T08:45:47.968841+0000","last_fullsized":"2024-09-19T08:45:47.968841+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:46.302996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066722+0000","last_change":"2024-09-19T08:43:52.756998+0000","last_active":"2024-09-19T08:45:48.066722+0000","last_peered":"2024-09-19T08:45:48.066722+0000","last_clean":"2024-09-19T08:45:48.066722+0000","last_became_active":"2024-09-19T08:43:52.756661+0000","last_became_peered":"2024-09-19T08:43:52.756661+0000","last_unstale":"2024-09-19T08:45:48.066722+0000","last_undegraded":"2024-09-19T08:45:48.066722+0000","last_fullsized":"2024-09-19T08:45:48.066722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:15.182574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.065908+0000","last_change":"2024-09-19T08:43:50.750095+0000","last_active":"2024-09-19T08:45:48.065908+0000","last_peered":"2024-09-19T08:45:48.065908+0000","last_clean":"2024-09-19T08:45:48.065908+0000","last_became_active":"2024-09-19T08:43:50.749672+0000","last_became_peered":"2024-09-19T08:43:50.749672+0000","last_unstale":"2024-09-19T08:45:48.065908+0000","last_undegraded":"2024-09-19T08:45:48.065908+0000","last_fullsized":"2024-09-19T08:45:48.065908+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:25:34.978094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221996+0000","last_change":"2024-09-19T08:43:52.764097+0000","last_active":"2024-09-19T08:45:48.221996+0000","last_peered":"2024-09-19T08:45:48.221996+0000","last_clean":"2024-09-19T08:45:48.221996+0000","last_became_active":"2024-09-19T08:43:52.763773+0000","last_became_peered":"2024-09-19T08:43:52.763773+0000","last_unstale":"2024-09-19T08:45:48.221996+0000","last_undegraded":"2024-09-19T08:45:48.221996+0000","last_fullsized":"2024-09-19T08:45:48.221996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:01:42.098460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.074171+0000","last_change":"2024-09-19T08:44:02.825238+0000","last_active":"2024-09-19T08:45:48.074171+0000","last_peered":"2024-09-19T08:45:48.074171+0000","last_clean":"2024-09-19T08:45:48.074171+0000","last_became_active":"2024-09-19T08:44:02.824873+0000","last_became_peered":"2024-09-19T08:44:02.824873+0000","last_unstale":"2024-09-19T08:45:48.074171+0000","last_undegraded":"2024-09-19T08:45:48.074171+0000","last_fullsized":"2024-09-19T08:45:48.074171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:38.896082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066088+0000","last_change":"2024-09-19T08:43:50.748789+0000","last_active":"2024-09-19T08:45:48.066088+0000","last_peered":"2024-09-19T08:45:48.066088+0000","last_clean":"2024-09-19T08:45:48.066088+0000","last_became_active":"2024-09-19T08:43:50.748464+0000","last_became_peered":"2024-09-19T08:43:50.748464+0000","last_unstale":"2024-09-19T08:45:48.066088+0000","last_undegraded":"2024-09-19T08:45:48.066088+0000","last_fullsized":"2024-09-19T08:45:48.066088+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:36:45.905639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.221312+0000","last_change":"2024-09-19T08:43:52.756608+0000","last_active":"2024-09-19T08:45:48.221312+0000","last_peered":"2024-09-19T08:45:48.221312+0000","last_clean":"2024-09-19T08:45:48.221312+0000","last_became_active":"2024-09-19T08:43:52.756227+0000","last_became_peered":"2024-09-19T08:43:52.756227+0000","last_unstale":"2024-09-19T08:45:48.221312+0000","last_undegraded":"2024-09-19T08:45:48.221312+0000","last_fullsized":"2024-09-19T08:45:48.221312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:29:58.085584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'13","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066253+0000","last_change":"2024-09-19T08:43:50.749135+0000","last_active":"2024-09-19T08:45:48.066253+0000","last_peered":"2024-09-19T08:45:48.066253+0000","last_clean":"2024-09-19T08:45:48.066253+0000","last_became_active":"2024-09-19T08:43:50.748755+0000","last_became_peered":"2024-09-19T08:43:50.748755+0000","last_unstale":"2024-09-19T08:45:48.066253+0000","last_undegraded":"2024-09-19T08:45:48.066253+0000","last_fullsized":"2024-09-19T08:45:48.066253+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:55.079780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.231124+0000","last_change":"2024-09-19T08:43:52.763277+0000","last_active":"2024-09-19T08:45:48.231124+0000","last_peered":"2024-09-19T08:45:48.231124+0000","last_clean":"2024-09-19T08:45:48.231124+0000","last_became_active":"2024-09-19T08:43:52.762941+0000","last_became_peered":"2024-09-19T08:43:52.762941+0000","last_unstale":"2024-09-19T08:45:48.231124+0000","last_undegraded":"2024-09-19T08:45:48.231124+0000","last_fullsized":"2024-09-19T08:45:48.231124+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:35.531946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.067452+0000","last_change":"2024-09-19T08:43:51.025350+0000","last_active":"2024-09-19T08:45:48.067452+0000","last_peered":"2024-09-19T08:45:48.067452+0000","last_clean":"2024-09-19T08:45:48.067452+0000","last_became_active":"2024-09-19T08:43:51.025046+0000","last_became_peered":"2024-09-19T08:43:51.025046+0000","last_unstale":"2024-09-19T08:45:48.067452+0000","last_undegraded":"2024-09-19T08:45:48.067452+0000","last_fullsized":"2024-09-19T08:45:48.067452+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:27.595545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:48.066292+0000","last_change":"2024-09-19T08:43:52.767917+0000","last_active":"2024-09-19T08:45:48.066292+0000","last_peered":"2024-09-19T08:45:48.066292+0000","last_clean":"2024-09-19T08:45:48.066292+0000","last_became_active":"2024-09-19T08:43:52.767575+0000","last_became_peered":"2024-09-19T08:43:52.767575+0000","last_unstale":"2024-09-19T08:45:48.066292+0000","last_undegraded":"2024-09-19T08:45:48.066292+0000","last_fullsized":"2024-09-19T08:45:48.066292+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:51.737507+0000","last_clean_scrub_stamp":"2024-09-19T08:43:51.737507+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:05.950399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":155,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T08:45:57.793622+0000","last_change":"2024-09-19T08:43:50.746400+0000","last_active":"2024-09-19T08:45:57.793622+0000","last_peered":"2024-09-19T08:45:57.793622+0000","last_clean":"2024-09-19T08:45:57.793622+0000","last_became_active":"2024-09-19T08:43:50.745891+0000","last_became_peered":"2024-09-19T08:43:50.745891+0000","last_unstale":"2024-09-19T08:45:57.793622+0000","last_undegraded":"2024-09-19T08:45:57.793622+0000","last_fullsized":"2024-09-19T08:45:57.793622+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:49.726744+0000","last_clean_scrub_stamp":"2024-09-19T08:43:49.726744+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:03.098202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":797575,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1546,"num_read_kb":995,"num_write":501,"num_write_kb":2380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":2392725,"data_compressed":103473,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":381,"ondisk_log_size":381,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":109335,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579943,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":446056,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808876,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1118208,"data_stored":1431343,"data_compressed":54546,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005104,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":838280,"data_compressed":22188,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234037,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":838429,"data_compressed":22188,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462970,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":983040,"data_stored":1039165,"data_compressed":39344,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593792,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":445450,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822724,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1040384,"data_stored":1366289,"data_compressed":54546,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986249,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":444525,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215182,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":999424,"data_stored":1165820,"data_compressed":37390,"data_compressed_allocated":49152,"data_compressed_original":708608,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1007616,"data_stored":514716,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":987136,"data_stored":513428,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":937984,"data_stored":446930,"data_compressed":2899,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36445,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":331979,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":397033,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:46:03.123 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:46:03.372 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:46:03.754 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:03 smithi164 ceph-mon[30928]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-09-19T08:46:03.754 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:03 smithi164 ceph-mon[30928]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:46:03.754 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:03 smithi164 ceph-mon[30928]: mgrmap e25: x(active, since 14s), standbys: z, y 2024-09-19T08:46:03.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.834+0000 7fc6bdb50640 1 -- 172.21.15.164:0/953992639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6b8102570 msgr2=0x7fc6b81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:03.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.834+0000 7fc6bdb50640 1 --2- 172.21.15.164:0/953992639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6b8102570 0x7fc6b81029b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009900 tx=0x7fc6a002a480 comp rx=0 tx=0).stop 2024-09-19T08:46:03.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.834+0000 7fc6bdb50640 1 -- 172.21.15.164:0/953992639 shutdown_connections 2024-09-19T08:46:03.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.834+0000 7fc6bdb50640 1 --2- 172.21.15.164:0/953992639 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6b8102ef0 0x7fc6b810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:03.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.834+0000 7fc6bdb50640 1 --2- 172.21.15.164:0/953992639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6b8102570 0x7fc6b81029b0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:03.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.834+0000 7fc6bdb50640 1 --2- 172.21.15.164:0/953992639 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6b8100e20 0x7fc6b8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:03.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.834+0000 7fc6bdb50640 1 -- 172.21.15.164:0/953992639 >> 172.21.15.164:0/953992639 conn(0x7fc6b80fc7d0 msgr2=0x7fc6b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:03.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.834+0000 7fc6bdb50640 1 -- 172.21.15.164:0/953992639 shutdown_connections 2024-09-19T08:46:03.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.834+0000 7fc6bdb50640 1 -- 172.21.15.164:0/953992639 wait complete. 2024-09-19T08:46:03.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.834+0000 7fc6bdb50640 1 Processor -- start 2024-09-19T08:46:03.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.834+0000 7fc6bdb50640 1 -- start start 2024-09-19T08:46:03.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.835+0000 7fc6bdb50640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6b8100e20 0x7fc6b81984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:03.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.835+0000 7fc6bdb50640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6b8102570 0x7fc6b8198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:03.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.835+0000 7fc6bdb50640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6b8102ef0 0x7fc6b819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:03.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.835+0000 7fc6bdb50640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b8110f80 con 0x7fc6b8102570 2024-09-19T08:46:03.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.835+0000 7fc6bdb50640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b8110e00 con 0x7fc6b8102ef0 2024-09-19T08:46:03.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.835+0000 7fc6bdb50640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b8111100 con 0x7fc6b8100e20 2024-09-19T08:46:03.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.835+0000 7fc6affff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6b8102570 0x7fc6b8198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:03.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.835+0000 7fc6affff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6b8102570 0x7fc6b8198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38874/0 (socket says 172.21.15.164:38874) 2024-09-19T08:46:03.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.835+0000 7fc6affff640 1 -- 172.21.15.164:0/1304982222 learned_addr learned my addr 172.21.15.164:0/1304982222 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:03.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.835+0000 7fc6b7fff640 1 --2- 172.21.15.164:0/1304982222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6b8102ef0 0x7fc6b819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:03.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.835+0000 7fc6b77fe640 1 --2- 172.21.15.164:0/1304982222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6b8100e20 0x7fc6b81984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:03.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.835+0000 7fc6affff640 1 -- 172.21.15.164:0/1304982222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6b8100e20 msgr2=0x7fc6b81984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:03.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.835+0000 7fc6affff640 1 --2- 172.21.15.164:0/1304982222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6b8100e20 0x7fc6b81984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:03.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.835+0000 7fc6affff640 1 -- 172.21.15.164:0/1304982222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6b8102ef0 msgr2=0x7fc6b819cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:03.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.835+0000 7fc6affff640 1 --2- 172.21.15.164:0/1304982222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6b8102ef0 0x7fc6b819cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:03.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.835+0000 7fc6affff640 1 -- 172.21.15.164:0/1304982222 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6b819d4a0 con 0x7fc6b8102570 2024-09-19T08:46:03.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.835+0000 7fc6b7fff640 1 --2- 172.21.15.164:0/1304982222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6b8102ef0 0x7fc6b819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:03.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.835+0000 7fc6b77fe640 1 --2- 172.21.15.164:0/1304982222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6b8100e20 0x7fc6b81984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:03.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.835+0000 7fc6affff640 1 --2- 172.21.15.164:0/1304982222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6b8102570 0x7fc6b8198a30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0006010 tx=0x7fc6a0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:03.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.835+0000 7fc6b57fa640 1 -- 172.21.15.164:0/1304982222 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a003f070 con 0x7fc6b8102570 2024-09-19T08:46:03.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.835+0000 7fc6bdb50640 1 -- 172.21.15.164:0/1304982222 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6b819d730 con 0x7fc6b8102570 2024-09-19T08:46:03.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.835+0000 7fc6b57fa640 1 -- 172.21.15.164:0/1304982222 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6a0031c30 con 0x7fc6b8102570 2024-09-19T08:46:03.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.835+0000 7fc6b57fa640 1 -- 172.21.15.164:0/1304982222 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0035030 con 0x7fc6b8102570 2024-09-19T08:46:03.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.835+0000 7fc6bdb50640 1 -- 172.21.15.164:0/1304982222 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6b81a5070 con 0x7fc6b8102570 2024-09-19T08:46:03.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.836+0000 7fc6bdb50640 1 -- 172.21.15.164:0/1304982222 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6b81021b0 con 0x7fc6b8102570 2024-09-19T08:46:03.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.838+0000 7fc6b57fa640 1 -- 172.21.15.164:0/1304982222 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7fc6a0042070 con 0x7fc6b8102570 2024-09-19T08:46:03.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.838+0000 7fc6b57fa640 1 --2- 172.21.15.164:0/1304982222 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fc6a40a34d0 0x7fc6a40a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:03.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.838+0000 7fc6b77fe640 1 --2- 172.21.15.164:0/1304982222 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fc6a40a34d0 0x7fc6a40a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:03.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.838+0000 7fc6b57fa640 1 -- 172.21.15.164:0/1304982222 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fc6a00f0bc0 con 0x7fc6b8102570 2024-09-19T08:46:03.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.839+0000 7fc6b77fe640 1 --2- 172.21.15.164:0/1304982222 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fc6a40a34d0 0x7fc6a40a5990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8009b40 tx=0x7fc6a8005e50 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:03.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:03.841+0000 7fc6b57fa640 1 -- 172.21.15.164:0/1304982222 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc6a0031de0 con 0x7fc6b8102570 2024-09-19T08:46:04.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:04.048+0000 7fc6bdb50640 1 -- 172.21.15.164:0/1304982222 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6b81a5240 con 0x7fc6b8102570 2024-09-19T08:46:04.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:04.049+0000 7fc6b57fa640 1 -- 172.21.15.164:0/1304982222 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11984 (secure 0 0 0) 0x7fc6a00bd490 con 0x7fc6b8102570 2024-09-19T08:46:04.050 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:46:04.051 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 28 2024-09-19T08:46:04.051 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":28,"btime":"2024-09-19T08:45:46:903004+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:45:46.852789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:46:04.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:04.051+0000 7fc6bdb50640 1 -- 172.21.15.164:0/1304982222 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fc6a40a34d0 msgr2=0x7fc6a40a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:04.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:04.051+0000 7fc6bdb50640 1 --2- 172.21.15.164:0/1304982222 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fc6a40a34d0 0x7fc6a40a5990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8009b40 tx=0x7fc6a8005e50 comp rx=0 tx=0).stop 2024-09-19T08:46:04.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:04.051+0000 7fc6bdb50640 1 -- 172.21.15.164:0/1304982222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6b8102570 msgr2=0x7fc6b8198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:04.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:04.051+0000 7fc6bdb50640 1 --2- 172.21.15.164:0/1304982222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6b8102570 0x7fc6b8198a30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0006010 tx=0x7fc6a0003c10 comp rx=0 tx=0).stop 2024-09-19T08:46:04.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:04.051+0000 7fc6bdb50640 1 -- 172.21.15.164:0/1304982222 shutdown_connections 2024-09-19T08:46:04.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:04.051+0000 7fc6bdb50640 1 --2- 172.21.15.164:0/1304982222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6b8102ef0 0x7fc6b819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:04.051+0000 7fc6bdb50640 1 --2- 172.21.15.164:0/1304982222 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fc6a40a34d0 0x7fc6a40a5990 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:04.051+0000 7fc6bdb50640 1 --2- 172.21.15.164:0/1304982222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6b8102570 0x7fc6b8198a30 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:04.051+0000 7fc6bdb50640 1 --2- 172.21.15.164:0/1304982222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6b8100e20 0x7fc6b81984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:04.051+0000 7fc6bdb50640 1 -- 172.21.15.164:0/1304982222 >> 172.21.15.164:0/1304982222 conn(0x7fc6b80fc7d0 msgr2=0x7fc6b8069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:04.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:04.051+0000 7fc6bdb50640 1 -- 172.21.15.164:0/1304982222 shutdown_connections 2024-09-19T08:46:04.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:04.051+0000 7fc6bdb50640 1 -- 172.21.15.164:0/1304982222 wait complete. 2024-09-19T08:46:04.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:03 smithi170 ceph-mon[35072]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-09-19T08:46:04.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:03 smithi170 ceph-mon[35072]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:46:04.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:03 smithi170 ceph-mon[35072]: mgrmap e25: x(active, since 14s), standbys: z, y 2024-09-19T08:46:04.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:03 smithi173 ceph-mon[35306]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-09-19T08:46:04.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:03 smithi173 ceph-mon[35306]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T08:46:04.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:03 smithi173 ceph-mon[35306]: mgrmap e25: x(active, since 14s), standbys: z, y 2024-09-19T08:46:04.680 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:46:04.781 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:04 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/1304982222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:46:04.928 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:46:05.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:04 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/1304982222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:46:05.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:04 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/1304982222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:46:05.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.383+0000 7ff9318a7640 1 -- 172.21.15.164:0/603318634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff92c106a60 msgr2=0x7ff92c106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:05.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.383+0000 7ff9318a7640 1 --2- 172.21.15.164:0/603318634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff92c106a60 0x7ff92c106e40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff920009cd0 tx=0x7ff92002a8f0 comp rx=0 tx=0).stop 2024-09-19T08:46:05.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.383+0000 7ff9318a7640 1 -- 172.21.15.164:0/603318634 shutdown_connections 2024-09-19T08:46:05.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.383+0000 7ff9318a7640 1 --2- 172.21.15.164:0/603318634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff92c101400 0x7ff92c10d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:05.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.383+0000 7ff9318a7640 1 --2- 172.21.15.164:0/603318634 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff92c100a60 0x7ff92c100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:05.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.383+0000 7ff9318a7640 1 --2- 172.21.15.164:0/603318634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff92c106a60 0x7ff92c106e40 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:05.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.383+0000 7ff9318a7640 1 -- 172.21.15.164:0/603318634 >> 172.21.15.164:0/603318634 conn(0x7ff92c0fc780 msgr2=0x7ff92c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:05.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.383+0000 7ff9318a7640 1 -- 172.21.15.164:0/603318634 shutdown_connections 2024-09-19T08:46:05.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.383+0000 7ff9318a7640 1 -- 172.21.15.164:0/603318634 wait complete. 2024-09-19T08:46:05.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.383+0000 7ff9318a7640 1 Processor -- start 2024-09-19T08:46:05.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.384+0000 7ff9318a7640 1 -- start start 2024-09-19T08:46:05.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.384+0000 7ff9318a7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff92c100a60 0x7ff92c199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:05.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.384+0000 7ff9318a7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff92c101400 0x7ff92c1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:05.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.384+0000 7ff9318a7640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff92c106a60 0x7ff92c1a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:05.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.384+0000 7ff9318a7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c110e60 con 0x7ff92c101400 2024-09-19T08:46:05.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.384+0000 7ff9318a7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c110ce0 con 0x7ff92c100a60 2024-09-19T08:46:05.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.384+0000 7ff9318a7640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c110fe0 con 0x7ff92c106a60 2024-09-19T08:46:05.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.384+0000 7ff92a7fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff92c101400 0x7ff92c1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:05.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.384+0000 7ff92a7fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff92c101400 0x7ff92c1997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38898/0 (socket says 172.21.15.164:38898) 2024-09-19T08:46:05.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.384+0000 7ff92a7fc640 1 -- 172.21.15.164:0/3972692615 learned_addr learned my addr 172.21.15.164:0/3972692615 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:05.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.384+0000 7ff92affd640 1 --2- 172.21.15.164:0/3972692615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff92c100a60 0x7ff92c199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:05.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.384+0000 7ff92b7fe640 1 --2- 172.21.15.164:0/3972692615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff92c106a60 0x7ff92c1a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:05.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.384+0000 7ff92a7fc640 1 -- 172.21.15.164:0/3972692615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff92c106a60 msgr2=0x7ff92c1a4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:05.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.384+0000 7ff92a7fc640 1 --2- 172.21.15.164:0/3972692615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff92c106a60 0x7ff92c1a4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:05.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.384+0000 7ff92a7fc640 1 -- 172.21.15.164:0/3972692615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff92c100a60 msgr2=0x7ff92c199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:05.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.384+0000 7ff92a7fc640 1 --2- 172.21.15.164:0/3972692615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff92c100a60 0x7ff92c199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:05.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.384+0000 7ff92a7fc640 1 -- 172.21.15.164:0/3972692615 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff92c1a4850 con 0x7ff92c101400 2024-09-19T08:46:05.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.385+0000 7ff92a7fc640 1 --2- 172.21.15.164:0/3972692615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff92c101400 0x7ff92c1997d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff91c00d770 tx=0x7ff91c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:05.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.385+0000 7ff9308a5640 1 -- 172.21.15.164:0/3972692615 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91c017070 con 0x7ff92c101400 2024-09-19T08:46:05.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.385+0000 7ff9318a7640 1 -- 172.21.15.164:0/3972692615 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff92c1a4a20 con 0x7ff92c101400 2024-09-19T08:46:05.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.385+0000 7ff9308a5640 1 -- 172.21.15.164:0/3972692615 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff91c005500 con 0x7ff92c101400 2024-09-19T08:46:05.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.385+0000 7ff9308a5640 1 -- 172.21.15.164:0/3972692615 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91c004e00 con 0x7ff92c101400 2024-09-19T08:46:05.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.385+0000 7ff9318a7640 1 -- 172.21.15.164:0/3972692615 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff92c1a4f30 con 0x7ff92c101400 2024-09-19T08:46:05.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.386+0000 7ff9318a7640 1 -- 172.21.15.164:0/3972692615 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff92c100ec0 con 0x7ff92c101400 2024-09-19T08:46:05.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.387+0000 7ff9308a5640 1 -- 172.21.15.164:0/3972692615 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7ff91c004fa0 con 0x7ff92c101400 2024-09-19T08:46:05.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.388+0000 7ff9308a5640 1 --2- 172.21.15.164:0/3972692615 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff9140a34d0 0x7ff9140a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:05.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.388+0000 7ff92affd640 1 --2- 172.21.15.164:0/3972692615 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff9140a34d0 0x7ff9140a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:05.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.388+0000 7ff9308a5640 1 -- 172.21.15.164:0/3972692615 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7ff91c0cf9f0 con 0x7ff92c101400 2024-09-19T08:46:05.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.388+0000 7ff92affd640 1 --2- 172.21.15.164:0/3972692615 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff9140a34d0 0x7ff9140a5990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff920004af0 tx=0x7ff920004a40 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:05.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.392+0000 7ff9308a5640 1 -- 172.21.15.164:0/3972692615 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff91c09c1f0 con 0x7ff92c101400 2024-09-19T08:46:05.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.593+0000 7ff9318a7640 1 -- 172.21.15.164:0/3972692615 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff92c108fd0 con 0x7ff92c101400 2024-09-19T08:46:05.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.594+0000 7ff9308a5640 1 -- 172.21.15.164:0/3972692615 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11984 (secure 0 0 0) 0x7ff91c09baf0 con 0x7ff92c101400 2024-09-19T08:46:05.594 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:46:05.594 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":28,"btime":"2024-09-19T08:45:46:903004+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:45:46.852789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:46:05.594 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 28 2024-09-19T08:46:05.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.596+0000 7ff9318a7640 1 -- 172.21.15.164:0/3972692615 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff9140a34d0 msgr2=0x7ff9140a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:05.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.596+0000 7ff9318a7640 1 --2- 172.21.15.164:0/3972692615 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff9140a34d0 0x7ff9140a5990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff920004af0 tx=0x7ff920004a40 comp rx=0 tx=0).stop 2024-09-19T08:46:05.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.596+0000 7ff9318a7640 1 -- 172.21.15.164:0/3972692615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff92c101400 msgr2=0x7ff92c1997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:05.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.596+0000 7ff9318a7640 1 --2- 172.21.15.164:0/3972692615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff92c101400 0x7ff92c1997d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff91c00d770 tx=0x7ff91c00dc30 comp rx=0 tx=0).stop 2024-09-19T08:46:05.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.596+0000 7ff9318a7640 1 -- 172.21.15.164:0/3972692615 shutdown_connections 2024-09-19T08:46:05.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.596+0000 7ff9318a7640 1 --2- 172.21.15.164:0/3972692615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff92c106a60 0x7ff92c1a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:05.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.596+0000 7ff9318a7640 1 --2- 172.21.15.164:0/3972692615 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff9140a34d0 0x7ff9140a5990 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:05.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.596+0000 7ff9318a7640 1 --2- 172.21.15.164:0/3972692615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff92c101400 0x7ff92c1997d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:05.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.596+0000 7ff9318a7640 1 --2- 172.21.15.164:0/3972692615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff92c100a60 0x7ff92c199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:05.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.596+0000 7ff9318a7640 1 -- 172.21.15.164:0/3972692615 >> 172.21.15.164:0/3972692615 conn(0x7ff92c0fc780 msgr2=0x7ff92c0fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:05.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.596+0000 7ff9318a7640 1 -- 172.21.15.164:0/3972692615 shutdown_connections 2024-09-19T08:46:05.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:05.596+0000 7ff9318a7640 1 -- 172.21.15.164:0/3972692615 wait complete. 2024-09-19T08:46:05.975 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:05 smithi164 ceph-mon[30928]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 87 KiB/s wr, 64 op/s 2024-09-19T08:46:05.975 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:05 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3972692615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:46:06.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:05 smithi170 ceph-mon[35072]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 87 KiB/s wr, 64 op/s 2024-09-19T08:46:06.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:05 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3972692615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:46:06.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:05 smithi173 ceph-mon[35306]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 87 KiB/s wr, 64 op/s 2024-09-19T08:46:06.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:05 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3972692615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:46:06.319 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T08:46:06.562 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:46:07.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.084+0000 7f5af892a640 1 -- 172.21.15.164:0/2102603252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5af4102d40 msgr2=0x7f5af4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:07.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.084+0000 7f5af892a640 1 --2- 172.21.15.164:0/2102603252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5af4102d40 0x7f5af4103120 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4009ca0 tx=0x7f5ae402a950 comp rx=0 tx=0).stop 2024-09-19T08:46:07.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.085+0000 7f5af892a640 1 -- 172.21.15.164:0/2102603252 shutdown_connections 2024-09-19T08:46:07.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.085+0000 7f5af892a640 1 --2- 172.21.15.164:0/2102603252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5af410c170 0x7f5af410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:07.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.085+0000 7f5af892a640 1 --2- 172.21.15.164:0/2102603252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5af4103660 0x7f5af410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:07.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.085+0000 7f5af892a640 1 --2- 172.21.15.164:0/2102603252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5af4102d40 0x7f5af4103120 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:07.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.085+0000 7f5af892a640 1 -- 172.21.15.164:0/2102603252 >> 172.21.15.164:0/2102603252 conn(0x7f5af40fc7a0 msgr2=0x7f5af40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:07.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.085+0000 7f5af892a640 1 -- 172.21.15.164:0/2102603252 shutdown_connections 2024-09-19T08:46:07.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.085+0000 7f5af892a640 1 -- 172.21.15.164:0/2102603252 wait complete. 2024-09-19T08:46:07.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.085+0000 7f5af892a640 1 Processor -- start 2024-09-19T08:46:07.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.085+0000 7f5af892a640 1 -- start start 2024-09-19T08:46:07.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.086+0000 7f5af892a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5af4102d40 0x7f5af4198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:07.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.086+0000 7f5af892a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5af4103660 0x7f5af4198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:07.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.086+0000 7f5af892a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5af410c170 0x7f5af419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:07.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.086+0000 7f5af892a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5af4111f70 con 0x7f5af4102d40 2024-09-19T08:46:07.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.086+0000 7f5af892a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5af4111df0 con 0x7f5af410c170 2024-09-19T08:46:07.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.086+0000 7f5af892a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5af41120f0 con 0x7f5af4103660 2024-09-19T08:46:07.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.086+0000 7f5af259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5af4102d40 0x7f5af4198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:07.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.086+0000 7f5af259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5af4102d40 0x7f5af4198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38922/0 (socket says 172.21.15.164:38922) 2024-09-19T08:46:07.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.086+0000 7f5af259c640 1 -- 172.21.15.164:0/781954855 learned_addr learned my addr 172.21.15.164:0/781954855 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:07.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.086+0000 7f5af2d9d640 1 --2- 172.21.15.164:0/781954855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5af410c170 0x7f5af419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:07.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.086+0000 7f5af1d9b640 1 --2- 172.21.15.164:0/781954855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5af4103660 0x7f5af4198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:07.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.086+0000 7f5af259c640 1 -- 172.21.15.164:0/781954855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5af4103660 msgr2=0x7f5af4198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:07.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.086+0000 7f5af259c640 1 --2- 172.21.15.164:0/781954855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5af4103660 0x7f5af4198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:07.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.086+0000 7f5af259c640 1 -- 172.21.15.164:0/781954855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5af410c170 msgr2=0x7f5af419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:07.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.086+0000 7f5af259c640 1 --2- 172.21.15.164:0/781954855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5af410c170 0x7f5af419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:07.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.086+0000 7f5af259c640 1 -- 172.21.15.164:0/781954855 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af419d400 con 0x7f5af4102d40 2024-09-19T08:46:07.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.086+0000 7f5af259c640 1 --2- 172.21.15.164:0/781954855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5af4102d40 0x7f5af4198450 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4005cc0 tx=0x7f5ae402f630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:07.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.086+0000 7f5ae37fe640 1 -- 172.21.15.164:0/781954855 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4034040 con 0x7f5af4102d40 2024-09-19T08:46:07.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.086+0000 7f5ae37fe640 1 -- 172.21.15.164:0/781954855 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ae4030920 con 0x7f5af4102d40 2024-09-19T08:46:07.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.086+0000 7f5af892a640 1 -- 172.21.15.164:0/781954855 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af419d690 con 0x7f5af4102d40 2024-09-19T08:46:07.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.086+0000 7f5ae37fe640 1 -- 172.21.15.164:0/781954855 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4004830 con 0x7f5af4102d40 2024-09-19T08:46:07.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.086+0000 7f5af892a640 1 -- 172.21.15.164:0/781954855 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af41a4fd0 con 0x7f5af4102d40 2024-09-19T08:46:07.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.087+0000 7f5af892a640 1 -- 172.21.15.164:0/781954855 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5af406b070 con 0x7f5af4102d40 2024-09-19T08:46:07.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.088+0000 7f5ae37fe640 1 -- 172.21.15.164:0/781954855 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7f5ae4037020 con 0x7f5af4102d40 2024-09-19T08:46:07.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.089+0000 7f5ae37fe640 1 --2- 172.21.15.164:0/781954855 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f5adc0a34d0 0x7f5adc0a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:07.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.089+0000 7f5af1d9b640 1 --2- 172.21.15.164:0/781954855 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f5adc0a34d0 0x7f5adc0a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:07.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.090+0000 7f5ae37fe640 1 -- 172.21.15.164:0/781954855 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f5ae40f0ec0 con 0x7f5af4102d40 2024-09-19T08:46:07.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.090+0000 7f5af1d9b640 1 --2- 172.21.15.164:0/781954855 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f5adc0a34d0 0x7f5adc0a5990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5af410d5b0 tx=0x7f5ae800a560 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:07.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.094+0000 7f5ae37fe640 1 -- 172.21.15.164:0/781954855 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5ae40bd7d0 con 0x7f5af4102d40 2024-09-19T08:46:07.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.262+0000 7f5af892a640 1 -- 172.21.15.164:0/781954855 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5af4107210 con 0x7f5af4102d40 2024-09-19T08:46:07.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.264+0000 7f5ae37fe640 1 -- 172.21.15.164:0/781954855 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+23086 (secure 0 0 0) 0x7f5ae40bd0d0 con 0x7f5af4102d40 2024-09-19T08:46:07.264 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:46:07.264 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":83,"fsid":"1e4b0d74-7662-11ef-bceb-c7b262605968","created":"2024-09-19T08:36:09.564509+0000","modified":"2024-09-19T08:45:47.960176+0000","last_up_change":"2024-09-19T08:43:01.260852+0000","last_in_change":"2024-09-19T08:42:37.492464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:39:13.961749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:43:49.613471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T08:43:51.570675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33b11f37-6ac5-41fc-8cfd-1b08c7eef688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6803","nonce":663038243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6805","nonce":663038243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6809","nonce":663038243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6807","nonce":663038243}]},"public_addr":"172.21.15.164:6803/663038243","cluster_addr":"172.21.15.164:6805/663038243","heartbeat_back_addr":"172.21.15.164:6809/663038243","heartbeat_front_addr":"172.21.15.164:6807/663038243","state":["exists","up"]},{"osd":1,"uuid":"6de64b0a-46e2-4a98-8005-a326a64bc87d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6801","nonce":2167066371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6803","nonce":2167066371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6807","nonce":2167066371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6805","nonce":2167066371}]},"public_addr":"172.21.15.170:6801/2167066371","cluster_addr":"172.21.15.170:6803/2167066371","heartbeat_back_addr":"172.21.15.170:6807/2167066371","heartbeat_front_addr":"172.21.15.170:6805/2167066371","state":["exists","up"]},{"osd":2,"uuid":"d4158cc7-9822-42a7-9601-d65089907584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6801","nonce":1646561048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6803","nonce":1646561048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6807","nonce":1646561048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6805","nonce":1646561048}]},"public_addr":"172.21.15.173:6801/1646561048","cluster_addr":"172.21.15.173:6803/1646561048","heartbeat_back_addr":"172.21.15.173:6807/1646561048","heartbeat_front_addr":"172.21.15.173:6805/1646561048","state":["exists","up"]},{"osd":3,"uuid":"0cf3f7f1-3a8c-42ce-88a9-9bea52e02221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6811","nonce":406795560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6813","nonce":406795560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6817","nonce":406795560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6815","nonce":406795560}]},"public_addr":"172.21.15.164:6811/406795560","cluster_addr":"172.21.15.164:6813/406795560","heartbeat_back_addr":"172.21.15.164:6817/406795560","heartbeat_front_addr":"172.21.15.164:6815/406795560","state":["exists","up"]},{"osd":4,"uuid":"6dca048a-2860-4a02-8d8e-450b30bc1a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6809","nonce":4120832165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6811","nonce":4120832165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6815","nonce":4120832165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6813","nonce":4120832165}]},"public_addr":"172.21.15.170:6809/4120832165","cluster_addr":"172.21.15.170:6811/4120832165","heartbeat_back_addr":"172.21.15.170:6815/4120832165","heartbeat_front_addr":"172.21.15.170:6813/4120832165","state":["exists","up"]},{"osd":5,"uuid":"3822521f-69d6-4141-a83f-5fc9cecc7b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6809","nonce":2914653552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6811","nonce":2914653552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6815","nonce":2914653552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6813","nonce":2914653552}]},"public_addr":"172.21.15.173:6809/2914653552","cluster_addr":"172.21.15.173:6811/2914653552","heartbeat_back_addr":"172.21.15.173:6815/2914653552","heartbeat_front_addr":"172.21.15.173:6813/2914653552","state":["exists","up"]},{"osd":6,"uuid":"0f5a965e-2eb2-4c24-b089-577e0639c160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6819","nonce":1138341393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6821","nonce":1138341393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6825","nonce":1138341393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6823","nonce":1138341393}]},"public_addr":"172.21.15.164:6819/1138341393","cluster_addr":"172.21.15.164:6821/1138341393","heartbeat_back_addr":"172.21.15.164:6825/1138341393","heartbeat_front_addr":"172.21.15.164:6823/1138341393","state":["exists","up"]},{"osd":7,"uuid":"f16cb2c3-639b-46ae-a740-1bd28ca1863c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6817","nonce":2124160360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6819","nonce":2124160360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6823","nonce":2124160360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6821","nonce":2124160360}]},"public_addr":"172.21.15.170:6817/2124160360","cluster_addr":"172.21.15.170:6819/2124160360","heartbeat_back_addr":"172.21.15.170:6823/2124160360","heartbeat_front_addr":"172.21.15.170:6821/2124160360","state":["exists","up"]},{"osd":8,"uuid":"161cff01-46ad-48e3-b999-a5473a2ad618","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6817","nonce":2285175061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6819","nonce":2285175061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6823","nonce":2285175061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6821","nonce":2285175061}]},"public_addr":"172.21.15.173:6817/2285175061","cluster_addr":"172.21.15.173:6819/2285175061","heartbeat_back_addr":"172.21.15.173:6823/2285175061","heartbeat_front_addr":"172.21.15.173:6821/2285175061","state":["exists","up"]},{"osd":9,"uuid":"994666e8-8fbd-4231-ac67-81a79aaf7c2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6827","nonce":3918847952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6829","nonce":3918847952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6833","nonce":3918847952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6831","nonce":3918847952}]},"public_addr":"172.21.15.164:6827/3918847952","cluster_addr":"172.21.15.164:6829/3918847952","heartbeat_back_addr":"172.21.15.164:6833/3918847952","heartbeat_front_addr":"172.21.15.164:6831/3918847952","state":["exists","up"]},{"osd":10,"uuid":"ff5e9099-cdb4-4905-b888-5b67320ef76b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6825","nonce":2707848227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6827","nonce":2707848227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6831","nonce":2707848227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6829","nonce":2707848227}]},"public_addr":"172.21.15.170:6825/2707848227","cluster_addr":"172.21.15.170:6827/2707848227","heartbeat_back_addr":"172.21.15.170:6831/2707848227","heartbeat_front_addr":"172.21.15.170:6829/2707848227","state":["exists","up"]},{"osd":11,"uuid":"4d93088f-e791-4938-a207-12f15b56abd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6825","nonce":2831924738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6827","nonce":2831924738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6831","nonce":2831924738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6829","nonce":2831924738}]},"public_addr":"172.21.15.173:6825/2831924738","cluster_addr":"172.21.15.173:6827/2831924738","heartbeat_back_addr":"172.21.15.173:6831/2831924738","heartbeat_front_addr":"172.21.15.173:6829/2831924738","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:21.787352+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:46.012323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:39:11.353040+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:39:37.015124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:02.063379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:27.892865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:53.372041+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:41:17.855284+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:41:43.389882+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:08.956895+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:34.069779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:59.303897+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.164:0/3424843982":"2024-09-20T08:45:47.960147+0000","172.21.15.164:0/1369265436":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1807120560":"2024-09-20T08:45:47.960147+0000","172.21.15.164:6801/1094864383":"2024-09-20T08:45:47.960147+0000","172.21.15.164:0/1905394497":"2024-09-20T08:36:33.856642+0000","172.21.15.164:6800/2805828009":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/2770797546":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1837164919":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6800/1173822158":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6800/1094864383":"2024-09-20T08:45:47.960147+0000","172.21.15.164:6801/1173822158":"2024-09-20T08:37:01.663065+0000","172.21.15.164:0/339032808":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6801/2805828009":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/2087067799":"2024-09-20T08:45:47.960147+0000","172.21.15.164:0/4018459143":"2024-09-20T08:37:01.663065+0000","172.21.15.164:0/2727626049":"2024-09-20T08:45:47.960147+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:46:07.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.266+0000 7f5af892a640 1 -- 172.21.15.164:0/781954855 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f5adc0a34d0 msgr2=0x7f5adc0a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:07.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.266+0000 7f5af892a640 1 --2- 172.21.15.164:0/781954855 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f5adc0a34d0 0x7f5adc0a5990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5af410d5b0 tx=0x7f5ae800a560 comp rx=0 tx=0).stop 2024-09-19T08:46:07.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.266+0000 7f5af892a640 1 -- 172.21.15.164:0/781954855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5af4102d40 msgr2=0x7f5af4198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:07.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.266+0000 7f5af892a640 1 --2- 172.21.15.164:0/781954855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5af4102d40 0x7f5af4198450 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4005cc0 tx=0x7f5ae402f630 comp rx=0 tx=0).stop 2024-09-19T08:46:07.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.266+0000 7f5af892a640 1 -- 172.21.15.164:0/781954855 shutdown_connections 2024-09-19T08:46:07.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.266+0000 7f5af892a640 1 --2- 172.21.15.164:0/781954855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5af410c170 0x7f5af419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:07.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.266+0000 7f5af892a640 1 --2- 172.21.15.164:0/781954855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5af4103660 0x7f5af4198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:07.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.266+0000 7f5af892a640 1 --2- 172.21.15.164:0/781954855 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f5adc0a34d0 0x7f5adc0a5990 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:07.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.266+0000 7f5af892a640 1 --2- 172.21.15.164:0/781954855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5af4102d40 0x7f5af4198450 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:07.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.266+0000 7f5af892a640 1 -- 172.21.15.164:0/781954855 >> 172.21.15.164:0/781954855 conn(0x7f5af40fc7a0 msgr2=0x7f5af40fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:07.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.266+0000 7f5af892a640 1 -- 172.21.15.164:0/781954855 shutdown_connections 2024-09-19T08:46:07.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:07.266+0000 7f5af892a640 1 -- 172.21.15.164:0/781954855 wait complete. 2024-09-19T08:46:07.927 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:07 smithi164 ceph-mon[30928]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 99 KiB/s wr, 59 op/s 2024-09-19T08:46:07.927 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:07 smithi164 ceph-mon[30928]: scrub summary: idle 2024-09-19T08:46:07.927 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:07 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/781954855' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:46:07.953 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-19T08:46:08.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:07 smithi170 ceph-mon[35072]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 99 KiB/s wr, 59 op/s 2024-09-19T08:46:08.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:07 smithi170 ceph-mon[35072]: scrub summary: idle 2024-09-19T08:46:08.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:07 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/781954855' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:46:08.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:07 smithi173 ceph-mon[35306]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 99 KiB/s wr, 59 op/s 2024-09-19T08:46:08.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:07 smithi173 ceph-mon[35306]: scrub summary: idle 2024-09-19T08:46:08.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:07 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/781954855' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:46:08.200 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:46:09.377 INFO:teuthology.orchestra.run.smithi164.stdout:{"run_dir":"/var/run/ceph"} 2024-09-19T08:46:09.377 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-19T08:46:09.378 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 1747 2024-09-19T08:46:09.378 DEBUG:tasks.quiescer:Yielding 2024-09-19T08:46:09.378 INFO:teuthology.run_tasks:Running task workunit... 2024-09-19T08:46:09.393 INFO:tasks.workunit:Pulling workunits from ref 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T08:46:09.393 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-19T08:46:09.394 INFO:tasks.workunit:timeout=3h 2024-09-19T08:46:09.394 INFO:tasks.workunit:cleanup=True 2024-09-19T08:46:09.394 DEBUG:teuthology.orchestra.run.smithi164:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-19T08:46:09.394 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:46:09.414 INFO:teuthology.orchestra.run.smithi164.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-19T08:46:09.414 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-19T08:46:09.414 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-19T08:46:09.414 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:46:09.414 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-19T08:46:09.414 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-09-19 08:44:11.441668044 +0000 2024-09-19T08:46:09.414 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-09-19 08:44:11.441668044 +0000 2024-09-19T08:46:09.414 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-09-19 08:45:46.184868283 +0000 2024-09-19T08:46:09.414 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-09-19 08:44:11.441668044 +0000 2024-09-19T08:46:09.415 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-19T08:46:09.415 DEBUG:teuthology.orchestra.run.smithi164:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-19T08:46:09.488 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T08:46:09.544 INFO:tasks.workunit.client.0.smithi164.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-19T08:46:09.648 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:46:09.941 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:09 smithi164 ceph-mon[30928]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 99 KiB/s wr, 55 op/s 2024-09-19T08:46:10.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:09 smithi170 ceph-mon[35072]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 99 KiB/s wr, 55 op/s 2024-09-19T08:46:10.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:09 smithi173 ceph-mon[35306]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 99 KiB/s wr, 55 op/s 2024-09-19T08:46:10.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.155+0000 7fe5cf046640 1 -- 172.21.15.164:0/1235772943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5c8106f20 msgr2=0x7fe5c810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:10.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.155+0000 7fe5cf046640 1 --2- 172.21.15.164:0/1235772943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5c8106f20 0x7fe5c810eaa0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0009ca0 tx=0x7fe5c002a8c0 comp rx=0 tx=0).stop 2024-09-19T08:46:10.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.155+0000 7fe5cf046640 1 -- 172.21.15.164:0/1235772943 shutdown_connections 2024-09-19T08:46:10.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.155+0000 7fe5cf046640 1 --2- 172.21.15.164:0/1235772943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5c8106f20 0x7fe5c810eaa0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:10.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.155+0000 7fe5cf046640 1 --2- 172.21.15.164:0/1235772943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5c81027a0 0x7fe5c81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:10.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.155+0000 7fe5cf046640 1 --2- 172.21.15.164:0/1235772943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5c8101df0 0x7fe5c81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:10.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.155+0000 7fe5cf046640 1 -- 172.21.15.164:0/1235772943 >> 172.21.15.164:0/1235772943 conn(0x7fe5c80fd8b0 msgr2=0x7fe5c80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:10.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.155+0000 7fe5cf046640 1 -- 172.21.15.164:0/1235772943 shutdown_connections 2024-09-19T08:46:10.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.155+0000 7fe5cf046640 1 -- 172.21.15.164:0/1235772943 wait complete. 2024-09-19T08:46:10.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.155+0000 7fe5cf046640 1 Processor -- start 2024-09-19T08:46:10.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.155+0000 7fe5cf046640 1 -- start start 2024-09-19T08:46:10.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.155+0000 7fe5cf046640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5c8101df0 0x7fe5c80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:10.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.156+0000 7fe5cf046640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5c81027a0 0x7fe5c8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:10.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.156+0000 7fe5cf046640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5c8106f20 0x7fe5c8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:10.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.156+0000 7fe5cf046640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c8111f80 con 0x7fe5c8101df0 2024-09-19T08:46:10.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.156+0000 7fe5cf046640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c8111e00 con 0x7fe5c81027a0 2024-09-19T08:46:10.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.156+0000 7fe5cf046640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c8112100 con 0x7fe5c8106f20 2024-09-19T08:46:10.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.156+0000 7fe5ccde2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5c8101df0 0x7fe5c80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:10.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.156+0000 7fe5ccde2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5c8101df0 0x7fe5c80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38936/0 (socket says 172.21.15.164:38936) 2024-09-19T08:46:10.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.156+0000 7fe5ccde2640 1 -- 172.21.15.164:0/2623768678 learned_addr learned my addr 172.21.15.164:0/2623768678 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:10.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.156+0000 7fe5c7fff640 1 --2- 172.21.15.164:0/2623768678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5c81027a0 0x7fe5c8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:10.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.156+0000 7fe5cd5e3640 1 --2- 172.21.15.164:0/2623768678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5c8106f20 0x7fe5c8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:10.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.156+0000 7fe5ccde2640 1 -- 172.21.15.164:0/2623768678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5c8106f20 msgr2=0x7fe5c8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:10.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.156+0000 7fe5ccde2640 1 --2- 172.21.15.164:0/2623768678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5c8106f20 0x7fe5c8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:10.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.156+0000 7fe5ccde2640 1 -- 172.21.15.164:0/2623768678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5c81027a0 msgr2=0x7fe5c8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:10.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.156+0000 7fe5ccde2640 1 --2- 172.21.15.164:0/2623768678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5c81027a0 0x7fe5c8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:10.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.156+0000 7fe5ccde2640 1 -- 172.21.15.164:0/2623768678 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c81a67c0 con 0x7fe5c8101df0 2024-09-19T08:46:10.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.156+0000 7fe5cd5e3640 1 --2- 172.21.15.164:0/2623768678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5c8106f20 0x7fe5c8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:10.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.156+0000 7fe5ccde2640 1 --2- 172.21.15.164:0/2623768678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5c8101df0 0x7fe5c80744f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc00c920 tx=0x7fe5bc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:10.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.156+0000 7fe5c5ffb640 1 -- 172.21.15.164:0/2623768678 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc007c70 con 0x7fe5c8101df0 2024-09-19T08:46:10.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.156+0000 7fe5cf046640 1 -- 172.21.15.164:0/2623768678 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c81a6ab0 con 0x7fe5c8101df0 2024-09-19T08:46:10.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.156+0000 7fe5c5ffb640 1 -- 172.21.15.164:0/2623768678 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5bc007e10 con 0x7fe5c8101df0 2024-09-19T08:46:10.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.156+0000 7fe5c5ffb640 1 -- 172.21.15.164:0/2623768678 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc018890 con 0x7fe5c8101df0 2024-09-19T08:46:10.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.156+0000 7fe5cf046640 1 -- 172.21.15.164:0/2623768678 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c81a7040 con 0x7fe5c8101df0 2024-09-19T08:46:10.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.158+0000 7fe5cf046640 1 -- 172.21.15.164:0/2623768678 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5c81021d0 con 0x7fe5c8101df0 2024-09-19T08:46:10.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.159+0000 7fe5c5ffb640 1 -- 172.21.15.164:0/2623768678 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7fe5bc018a70 con 0x7fe5c8101df0 2024-09-19T08:46:10.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.159+0000 7fe5c5ffb640 1 --2- 172.21.15.164:0/2623768678 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe5b80a34d0 0x7fe5b80a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:10.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.159+0000 7fe5c7fff640 1 --2- 172.21.15.164:0/2623768678 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe5b80a34d0 0x7fe5b80a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:10.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.159+0000 7fe5c5ffb640 1 -- 172.21.15.164:0/2623768678 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fe5bc0d00b0 con 0x7fe5c8101df0 2024-09-19T08:46:10.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.160+0000 7fe5c7fff640 1 --2- 172.21.15.164:0/2623768678 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe5b80a34d0 0x7fe5b80a5990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8075f80 tx=0x7fe5b400b390 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:10.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.163+0000 7fe5c5ffb640 1 -- 172.21.15.164:0/2623768678 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe5bc014030 con 0x7fe5c8101df0 2024-09-19T08:46:10.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.376+0000 7fe5cf046640 1 -- 172.21.15.164:0/2623768678 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5c81a7340 con 0x7fe5c8101df0 2024-09-19T08:46:10.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.377+0000 7fe5c5ffb640 1 -- 172.21.15.164:0/2623768678 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11984 (secure 0 0 0) 0x7fe5bc09c8b0 con 0x7fe5c8101df0 2024-09-19T08:46:10.377 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:46:10.377 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":28,"btime":"2024-09-19T08:45:46:903004+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:45:46.852789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:46:10.377 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 28 2024-09-19T08:46:10.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.379+0000 7fe5cf046640 1 -- 172.21.15.164:0/2623768678 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe5b80a34d0 msgr2=0x7fe5b80a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:10.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.379+0000 7fe5cf046640 1 --2- 172.21.15.164:0/2623768678 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe5b80a34d0 0x7fe5b80a5990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8075f80 tx=0x7fe5b400b390 comp rx=0 tx=0).stop 2024-09-19T08:46:10.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.379+0000 7fe5cf046640 1 -- 172.21.15.164:0/2623768678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5c8101df0 msgr2=0x7fe5c80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:10.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.379+0000 7fe5cf046640 1 --2- 172.21.15.164:0/2623768678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5c8101df0 0x7fe5c80744f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc00c920 tx=0x7fe5bc00cde0 comp rx=0 tx=0).stop 2024-09-19T08:46:10.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.379+0000 7fe5cf046640 1 -- 172.21.15.164:0/2623768678 shutdown_connections 2024-09-19T08:46:10.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.379+0000 7fe5cf046640 1 --2- 172.21.15.164:0/2623768678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5c8106f20 0x7fe5c8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:10.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.379+0000 7fe5cf046640 1 --2- 172.21.15.164:0/2623768678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5c81027a0 0x7fe5c8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:10.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.379+0000 7fe5cf046640 1 --2- 172.21.15.164:0/2623768678 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe5b80a34d0 0x7fe5b80a5990 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:10.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.379+0000 7fe5cf046640 1 --2- 172.21.15.164:0/2623768678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5c8101df0 0x7fe5c80744f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:10.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.379+0000 7fe5cf046640 1 -- 172.21.15.164:0/2623768678 >> 172.21.15.164:0/2623768678 conn(0x7fe5c80fd8b0 msgr2=0x7fe5c80fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:10.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.380+0000 7fe5cf046640 1 -- 172.21.15.164:0/2623768678 shutdown_connections 2024-09-19T08:46:10.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:10.380+0000 7fe5cf046640 1 -- 172.21.15.164:0/2623768678 wait complete. 2024-09-19T08:46:11.020 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:10 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2623768678' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:46:11.048 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T08:43:53.386044+0000', 'modified': '2024-09-19T08:45:46.852789+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24479, 'mds_1': 24449, 'mds_2': 24442, 'mds_3': 24424, 'mds_4': 14595}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14583': {'gid': 14583, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6839/538276454', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6838', 'nonce': 538276454}, {'type': 'v1', 'addr': '172.21.15.164:6839', 'nonce': 538276454}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14595': {'gid': 14595, 'name': 'a', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.164:6841/2221659234', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6840', 'nonce': 2221659234}, {'type': 'v1', 'addr': '172.21.15.164:6841', 'nonce': 2221659234}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24401': {'gid': 24401, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6833/2726071671', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6832', 'nonce': 2726071671}, {'type': 'v1', 'addr': '172.21.15.170:6833', 'nonce': 2726071671}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24403': {'gid': 24403, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.173:6833/486946900', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6832', 'nonce': 486946900}, {'type': 'v1', 'addr': '172.21.15.173:6833', 'nonce': 486946900}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24418': {'gid': 24418, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.173:6835/947317705', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6834', 'nonce': 947317705}, {'type': 'v1', 'addr': '172.21.15.173:6835', 'nonce': 947317705}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6835/321254273', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6834', 'nonce': 321254273}, {'type': 'v1', 'addr': '172.21.15.170:6835', 'nonce': 321254273}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24424': {'gid': 24424, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.173:6837/2431090203', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6836', 'nonce': 2431090203}, {'type': 'v1', 'addr': '172.21.15.173:6837', 'nonce': 2431090203}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'i', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.173:6839/1089658493', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6838', 'nonce': 1089658493}, {'type': 'v1', 'addr': '172.21.15.173:6839', 'nonce': 1089658493}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'h', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.170:6837/2350302392', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6836', 'nonce': 2350302392}, {'type': 'v1', 'addr': '172.21.15.170:6837', 'nonce': 2350302392}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.170:6839/1854887138', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6838', 'nonce': 1854887138}, {'type': 'v1', 'addr': '172.21.15.170:6839', 'nonce': 1854887138}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24479, 'qdb_cluster': [14595, 24449, 24424, 24442, 24479]} 2024-09-19T08:46:11.048 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-19T08:46:11.049 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.a status 2024-09-19T08:46:11.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:10 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2623768678' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:46:11.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:10 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2623768678' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:46:11.336 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:46:11.743 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:11 smithi164 ceph-mon[30928]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 99 KiB/s wr, 60 op/s 2024-09-19T08:46:11.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.825+0000 7f7f9e56f640 1 -- 172.21.15.164:0/3671004232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f98102d30 msgr2=0x7f7f98103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:11.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.825+0000 7f7f9e56f640 1 --2- 172.21.15.164:0/3671004232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f98102d30 0x7f7f98103110 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009930 tx=0x7f7f8802a420 comp rx=0 tx=0).stop 2024-09-19T08:46:11.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.825+0000 7f7f9e56f640 1 -- 172.21.15.164:0/3671004232 shutdown_connections 2024-09-19T08:46:11.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.825+0000 7f7f9e56f640 1 --2- 172.21.15.164:0/3671004232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f981082a0 0x7f7f9810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.825+0000 7f7f9e56f640 1 --2- 172.21.15.164:0/3671004232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f981036e0 0x7f7f98107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.825+0000 7f7f9e56f640 1 --2- 172.21.15.164:0/3671004232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f98102d30 0x7f7f98103110 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.825+0000 7f7f9e56f640 1 -- 172.21.15.164:0/3671004232 >> 172.21.15.164:0/3671004232 conn(0x7f7f980fc7d0 msgr2=0x7f7f980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:11.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.825+0000 7f7f9e56f640 1 -- 172.21.15.164:0/3671004232 shutdown_connections 2024-09-19T08:46:11.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.825+0000 7f7f9e56f640 1 -- 172.21.15.164:0/3671004232 wait complete. 2024-09-19T08:46:11.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.826+0000 7f7f9e56f640 1 Processor -- start 2024-09-19T08:46:11.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.826+0000 7f7f9e56f640 1 -- start start 2024-09-19T08:46:11.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.826+0000 7f7f9e56f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f98102d30 0x7f7f98100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:11.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.826+0000 7f7f9e56f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f981036e0 0x7f7f98100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:11.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.826+0000 7f7f9e56f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f981082a0 0x7f7f98101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:11.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.826+0000 7f7f9e56f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7f981120c0 con 0x7f7f98102d30 2024-09-19T08:46:11.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.826+0000 7f7f9e56f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7f98111f40 con 0x7f7f981036e0 2024-09-19T08:46:11.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.826+0000 7f7f9e56f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7f98112240 con 0x7f7f981082a0 2024-09-19T08:46:11.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.826+0000 7f7f97fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f98102d30 0x7f7f98100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:11.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.826+0000 7f7f97fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f98102d30 0x7f7f98100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38944/0 (socket says 172.21.15.164:38944) 2024-09-19T08:46:11.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.826+0000 7f7f97fff640 1 -- 172.21.15.164:0/3027963448 learned_addr learned my addr 172.21.15.164:0/3027963448 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:11.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.826+0000 7f7f8ffff640 1 --2- 172.21.15.164:0/3027963448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f981036e0 0x7f7f98100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:11.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.826+0000 7f7f9cb0c640 1 --2- 172.21.15.164:0/3027963448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f981082a0 0x7f7f98101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:11.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.826+0000 7f7f97fff640 1 -- 172.21.15.164:0/3027963448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f981082a0 msgr2=0x7f7f98101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:11.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.826+0000 7f7f97fff640 1 --2- 172.21.15.164:0/3027963448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f981082a0 0x7f7f98101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.826+0000 7f7f97fff640 1 -- 172.21.15.164:0/3027963448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f981036e0 msgr2=0x7f7f98100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:11.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.826+0000 7f7f97fff640 1 --2- 172.21.15.164:0/3027963448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f981036e0 0x7f7f98100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.826+0000 7f7f97fff640 1 -- 172.21.15.164:0/3027963448 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f981a6ac0 con 0x7f7f98102d30 2024-09-19T08:46:11.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.827+0000 7f7f97fff640 1 --2- 172.21.15.164:0/3027963448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f98102d30 0x7f7f98100990 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009230 tx=0x7f7f88004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:11.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.827+0000 7f7f95ffb640 1 -- 172.21.15.164:0/3027963448 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8803f030 con 0x7f7f98102d30 2024-09-19T08:46:11.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.827+0000 7f7f95ffb640 1 -- 172.21.15.164:0/3027963448 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f88003ad0 con 0x7f7f98102d30 2024-09-19T08:46:11.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.827+0000 7f7f9e56f640 1 -- 172.21.15.164:0/3027963448 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f981a6d50 con 0x7f7f98102d30 2024-09-19T08:46:11.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.827+0000 7f7f95ffb640 1 -- 172.21.15.164:0/3027963448 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f88035070 con 0x7f7f98102d30 2024-09-19T08:46:11.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.827+0000 7f7f9e56f640 1 -- 172.21.15.164:0/3027963448 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f981a71d0 con 0x7f7f98102d30 2024-09-19T08:46:11.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.829+0000 7f7f95ffb640 1 -- 172.21.15.164:0/3027963448 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7f7f88042030 con 0x7f7f98102d30 2024-09-19T08:46:11.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.829+0000 7f7fa0948740 1 Processor -- start 2024-09-19T08:46:11.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.829+0000 7f7fa0948740 1 -- start start 2024-09-19T08:46:11.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.830+0000 7f7fa0948740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8327559d0 0x55c832755db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:11.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.830+0000 7f7fa0948740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8327562f0 0x55c832760670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:11.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.830+0000 7f7fa0948740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c832760bb0 0x55c832762fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:11.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.830+0000 7f7fa0948740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c8326bc090 con 0x55c832760bb0 2024-09-19T08:46:11.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.830+0000 7f7fa0948740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c832663a70 con 0x55c8327559d0 2024-09-19T08:46:11.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.830+0000 7f7fa0948740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c8326b8050 con 0x55c8327562f0 2024-09-19T08:46:11.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.830+0000 7f7f8ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c832760bb0 0x55c832762fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:11.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.830+0000 7f7f8ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c832760bb0 0x55c832762fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38950/0 (socket says 172.21.15.164:38950) 2024-09-19T08:46:11.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.830+0000 7f7f8ffff640 1 -- 172.21.15.164:0/3855394107 learned_addr learned my addr 172.21.15.164:0/3855394107 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:11.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.830+0000 7f7f9cb0c640 1 --2- 172.21.15.164:0/3855394107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8327559d0 0x55c832755db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:11.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.830+0000 7f7f97fff640 1 --2- 172.21.15.164:0/3855394107 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8327562f0 0x55c832760670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:11.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.830+0000 7f7f95ffb640 1 --2- 172.21.15.164:0/3027963448 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f7f840a34d0 0x7f7f840a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:11.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.830+0000 7f7f8ffff640 1 -- 172.21.15.164:0/3855394107 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8327562f0 msgr2=0x55c832760670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:11.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.830+0000 7f7f8ffff640 1 --2- 172.21.15.164:0/3855394107 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8327562f0 0x55c832760670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.830+0000 7f7f8ffff640 1 -- 172.21.15.164:0/3855394107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8327559d0 msgr2=0x55c832755db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:11.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.830+0000 7f7f8ffff640 1 --2- 172.21.15.164:0/3855394107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8327559d0 0x55c832755db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.830+0000 7f7f8ffff640 1 -- 172.21.15.164:0/3855394107 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c832670130 con 0x55c832760bb0 2024-09-19T08:46:11.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.830+0000 7f7f9cb0c640 1 --2- 172.21.15.164:0/3027963448 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f7f840a34d0 0x7f7f840a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:11.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.830+0000 7f7f8ffff640 1 --2- 172.21.15.164:0/3855394107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c832760bb0 0x55c832762fa0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7f8000b690 tx=0x7f7f8000bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e3f01b0bc4238 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:11.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.831+0000 7f7f95ffb640 1 -- 172.21.15.164:0/3027963448 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f7f880eff70 con 0x7f7f98102d30 2024-09-19T08:46:11.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.831+0000 7f7f8e7fc640 1 -- 172.21.15.164:0/3855394107 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f80003bd0 con 0x55c832760bb0 2024-09-19T08:46:11.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.831+0000 7f7f8e7fc640 1 -- 172.21.15.164:0/3855394107 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f80015070 con 0x55c832760bb0 2024-09-19T08:46:11.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.831+0000 7f7f8e7fc640 1 -- 172.21.15.164:0/3855394107 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f80004de0 con 0x55c832760bb0 2024-09-19T08:46:11.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.831+0000 7f7f9cb0c640 1 --2- 172.21.15.164:0/3027963448 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f7f840a34d0 0x7f7f840a5990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7f90005fd0 tx=0x7f7f90004a40 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:11.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.831+0000 7f7fa0948740 1 -- 172.21.15.164:0/3855394107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c832760bb0 msgr2=0x55c832762fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:11.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.831+0000 7f7fa0948740 1 --2- 172.21.15.164:0/3855394107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c832760bb0 0x55c832762fa0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7f8000b690 tx=0x7f7f8000bb50 comp rx=0 tx=0).stop 2024-09-19T08:46:11.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.831+0000 7f7fa0948740 1 -- 172.21.15.164:0/3855394107 shutdown_connections 2024-09-19T08:46:11.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.831+0000 7f7fa0948740 1 --2- 172.21.15.164:0/3855394107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c832760bb0 0x55c832762fa0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.831+0000 7f7fa0948740 1 --2- 172.21.15.164:0/3855394107 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8327562f0 0x55c832760670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.831+0000 7f7fa0948740 1 --2- 172.21.15.164:0/3855394107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8327559d0 0x55c832755db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.831+0000 7f7fa0948740 1 -- 172.21.15.164:0/3855394107 >> 172.21.15.164:0/3855394107 conn(0x55c832751f60 msgr2=0x55c832752340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:11.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.831+0000 7f7fa0948740 1 -- 172.21.15.164:0/3855394107 shutdown_connections 2024-09-19T08:46:11.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.831+0000 7f7fa0948740 1 -- 172.21.15.164:0/3855394107 wait complete. 2024-09-19T08:46:11.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.832+0000 7f7fa0948740 1 -- start start 2024-09-19T08:46:11.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.832+0000 7f7fa0948740 1 Processor -- start 2024-09-19T08:46:11.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.832+0000 7f7fa0948740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8327559d0 0x55c832760400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:11.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.832+0000 7f7fa0948740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8327562f0 0x55c83275d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:11.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.832+0000 7f7fa0948740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c832760bb0 0x55c83275dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:11.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.832+0000 7f7fa0948740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c832763780 con 0x55c8327559d0 2024-09-19T08:46:11.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.832+0000 7f7fa0948740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c832676f20 con 0x55c8327562f0 2024-09-19T08:46:11.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.832+0000 7f7fa0948740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c832763900 con 0x55c832760bb0 2024-09-19T08:46:11.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.832+0000 7f7fa0948740 10 client.0 _ll_register_callbacks cb 0x55c832640030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:11.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.832+0000 7f7f97fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8327559d0 0x55c832760400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:11.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.832+0000 7f7f97fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8327559d0 0x55c832760400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38954/0 (socket says 172.21.15.164:38954) 2024-09-19T08:46:11.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.832+0000 7f7f97fff640 1 -- 172.21.15.164:0/2403743215 learned_addr learned my addr 172.21.15.164:0/2403743215 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:11.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.832+0000 7f7f8ffff640 1 --2- 172.21.15.164:0/2403743215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8327562f0 0x55c83275d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:11.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.833+0000 7f7f9cb0c640 1 --2- 172.21.15.164:0/2403743215 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c832760bb0 0x55c83275dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:11.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.833+0000 7f7f97fff640 1 -- 172.21.15.164:0/2403743215 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c832760bb0 msgr2=0x55c83275dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:11.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.833+0000 7f7f97fff640 1 --2- 172.21.15.164:0/2403743215 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c832760bb0 0x55c83275dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.833+0000 7f7f97fff640 1 -- 172.21.15.164:0/2403743215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8327562f0 msgr2=0x55c83275d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:11.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.833+0000 7f7f97fff640 1 --2- 172.21.15.164:0/2403743215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8327562f0 0x55c83275d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.833+0000 7f7f97fff640 1 -- 172.21.15.164:0/2403743215 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c83242d2b0 con 0x55c8327559d0 2024-09-19T08:46:11.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.833+0000 7f7f97fff640 1 --2- 172.21.15.164:0/2403743215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8327559d0 0x55c832760400 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7f880b8650 tx=0x7f7f880f3670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80eac5cde717a5ae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:11.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.833+0000 7f7f777fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:46:11.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.833+0000 7f7f777fe640 1 -- 172.21.15.164:0/2403743215 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f88049050 con 0x55c8327559d0 2024-09-19T08:46:11.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.833+0000 7f7f777fe640 1 -- 172.21.15.164:0/2403743215 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f7f880dc0b0 con 0x55c8327559d0 2024-09-19T08:46:11.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.833+0000 7f7fa0948740 1 -- 172.21.15.164:0/2403743215 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f80015070 con 0x55c8327559d0 2024-09-19T08:46:11.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.833+0000 7f7f777fe640 1 -- 172.21.15.164:0/2403743215 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7f880bbb50 con 0x55c8327559d0 2024-09-19T08:46:11.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.833+0000 7f7fa0948740 10 client.15453 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:11.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.833+0000 7f7fa0948740 1 -- 172.21.15.164:0/2403743215 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c83276f050 con 0x55c8327559d0 2024-09-19T08:46:11.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.833+0000 7f7f777fe640 1 -- 172.21.15.164:0/2403743215 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f80015070 con 0x55c8327559d0 2024-09-19T08:46:11.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.833+0000 7f7f777fe640 1 -- 172.21.15.164:0/2403743215 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7f880b8860 con 0x55c8327559d0 2024-09-19T08:46:11.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.833+0000 7f7fa0948740 10 client.15453 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:11.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.834+0000 7f7fa0948740 10 client.15453 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T08:46:11.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.834+0000 7f7fa0948740 20 client.15453 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:11.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.834+0000 7f7fa0948740 1 --2- 172.21.15.164:0/2403743215 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55c832771630 0x55c832772aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:11.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.834+0000 7f7fa0948740 4 client.15453 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:46:11.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.834+0000 7f7fa0948740 1 -- 172.21.15.164:0/2403743215 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8326c9600 con 0x55c832771630 2024-09-19T08:46:11.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.834+0000 7f7f8ffff640 1 --2- 172.21.15.164:0/2403743215 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55c832771630 0x55c832772aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:11.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.834+0000 7f7f8ffff640 1 --2- 172.21.15.164:0/2403743215 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55c832771630 0x55c832772aa0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bf88d6a1636580d7 server_cookie=b2d1ce4a0d384401 in_seq=0 out_seq=0 2024-09-19T08:46:11.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.834+0000 7f7f777fe640 10 client.15453 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:46:11.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.835+0000 7f7f777fe640 1 -- 172.21.15.164:0/2403743215 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c8326c9600 con 0x55c832771630 2024-09-19T08:46:11.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.835+0000 7f7f777fe640 10 client.15453 handle_command_reply: tid=0 2024-09-19T08:46:11.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.835+0000 7f7fa0948740 1 client.15453 shutdown 2024-09-19T08:46:11.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.835+0000 7f7fa0948740 1 -- 172.21.15.164:0/2403743215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8327559d0 msgr2=0x55c832760400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:11.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.835+0000 7f7fa0948740 1 --2- 172.21.15.164:0/2403743215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8327559d0 0x55c832760400 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7f880b8650 tx=0x7f7f880f3670 comp rx=0 tx=0).stop 2024-09-19T08:46:11.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.835+0000 7f7fa0948740 1 -- 172.21.15.164:0/2403743215 shutdown_connections 2024-09-19T08:46:11.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.835+0000 7f7fa0948740 1 --2- 172.21.15.164:0/2403743215 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55c832771630 0x55c832772aa0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.835+0000 7f7fa0948740 1 --2- 172.21.15.164:0/2403743215 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c832760bb0 0x55c83275dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.835+0000 7f7fa0948740 1 --2- 172.21.15.164:0/2403743215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8327562f0 0x55c83275d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.835+0000 7f7f777fe640 0 client.15453 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:46:11.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.835+0000 7f7fa0948740 1 --2- 172.21.15.164:0/2403743215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c8327559d0 0x55c832760400 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.835+0000 7f7fa0948740 1 -- 172.21.15.164:0/2403743215 >> 172.21.15.164:0/2403743215 conn(0x55c832751f60 msgr2=0x55c832757c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:11.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.835+0000 7f7fa0948740 1 -- 172.21.15.164:0/2403743215 shutdown_connections 2024-09-19T08:46:11.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.835+0000 7f7fa0948740 1 -- 172.21.15.164:0/2403743215 wait complete. 2024-09-19T08:46:11.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.835+0000 7f7fa0948740 20 client.15453 trim_cache size 0 max 16384 2024-09-19T08:46:11.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.853+0000 7f7fa0948740 1 Processor -- start 2024-09-19T08:46:11.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.853+0000 7f7fa0948740 1 -- start start 2024-09-19T08:46:11.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.853+0000 7f7fa0948740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8327559d0 0x55c832772aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:11.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.853+0000 7f7fa0948740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8327562f0 0x55c832752340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:11.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.853+0000 7f7fa0948740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c832760bb0 0x55c832752880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:11.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.853+0000 7f7fa0948740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c832770de0 con 0x55c832760bb0 2024-09-19T08:46:11.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.853+0000 7f7fa0948740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c8327714b0 con 0x55c8327559d0 2024-09-19T08:46:11.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.853+0000 7f7fa0948740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c832771b80 con 0x55c8327562f0 2024-09-19T08:46:11.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.853+0000 7f7f9cb0c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c832760bb0 0x55c832752880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:11.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.853+0000 7f7f9cb0c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c832760bb0 0x55c832752880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38968/0 (socket says 172.21.15.164:38968) 2024-09-19T08:46:11.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.853+0000 7f7f9cb0c640 1 -- 172.21.15.164:0/2383964964 learned_addr learned my addr 172.21.15.164:0/2383964964 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:11.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.853+0000 7f7f8ffff640 1 --2- 172.21.15.164:0/2383964964 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8327562f0 0x55c832752340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:11.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.853+0000 7f7f97fff640 1 --2- 172.21.15.164:0/2383964964 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8327559d0 0x55c832772aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:11.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.853+0000 7f7f9cb0c640 1 -- 172.21.15.164:0/2383964964 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8327562f0 msgr2=0x55c832752340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:11.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.853+0000 7f7f9cb0c640 1 --2- 172.21.15.164:0/2383964964 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8327562f0 0x55c832752340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.853+0000 7f7f9cb0c640 1 -- 172.21.15.164:0/2383964964 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8327559d0 msgr2=0x55c832772aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:11.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.853+0000 7f7f9cb0c640 1 --2- 172.21.15.164:0/2383964964 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8327559d0 0x55c832772aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.853+0000 7f7f9cb0c640 1 -- 172.21.15.164:0/2383964964 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c832756ee0 con 0x55c832760bb0 2024-09-19T08:46:11.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.853+0000 7f7f97fff640 1 --2- 172.21.15.164:0/2383964964 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8327559d0 0x55c832772aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:11.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.854+0000 7f7f9cb0c640 1 --2- 172.21.15.164:0/2383964964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c832760bb0 0x55c832752880 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7f900124c0 tx=0x7f7f90016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d11c53d17117d60b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:11.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.854+0000 7f7f8dffb640 1 -- 172.21.15.164:0/2383964964 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f90012d30 con 0x55c832760bb0 2024-09-19T08:46:11.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.854+0000 7f7f8dffb640 1 -- 172.21.15.164:0/2383964964 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f900218f0 con 0x55c832760bb0 2024-09-19T08:46:11.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.854+0000 7f7f8dffb640 1 -- 172.21.15.164:0/2383964964 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f90010970 con 0x55c832760bb0 2024-09-19T08:46:11.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.854+0000 7f7fa0948740 1 -- 172.21.15.164:0/2383964964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c832760bb0 msgr2=0x55c832752880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:11.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.854+0000 7f7fa0948740 1 --2- 172.21.15.164:0/2383964964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c832760bb0 0x55c832752880 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7f900124c0 tx=0x7f7f90016000 comp rx=0 tx=0).stop 2024-09-19T08:46:11.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.855+0000 7f7fa0948740 1 -- 172.21.15.164:0/2383964964 shutdown_connections 2024-09-19T08:46:11.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.855+0000 7f7fa0948740 1 --2- 172.21.15.164:0/2383964964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c832760bb0 0x55c832752880 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.855+0000 7f7fa0948740 1 --2- 172.21.15.164:0/2383964964 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8327562f0 0x55c832752340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.855+0000 7f7fa0948740 1 --2- 172.21.15.164:0/2383964964 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8327559d0 0x55c832772aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.855+0000 7f7fa0948740 1 -- 172.21.15.164:0/2383964964 >> 172.21.15.164:0/2383964964 conn(0x55c832751f60 msgr2=0x55c83274cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:11.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.855+0000 7f7fa0948740 1 -- 172.21.15.164:0/2383964964 shutdown_connections 2024-09-19T08:46:11.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.855+0000 7f7fa0948740 1 -- 172.21.15.164:0/2383964964 wait complete. 2024-09-19T08:46:11.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.855+0000 7f7fa0948740 1 -- start start 2024-09-19T08:46:11.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.855+0000 7f7fa0948740 1 Processor -- start 2024-09-19T08:46:11.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.855+0000 7f7fa0948740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8327559d0 0x55c832774260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:11.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.855+0000 7f7fa0948740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8327562f0 0x55c8327747a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:11.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.855+0000 7f7fa0948740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c832760bb0 0x55c8327d9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:11.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.855+0000 7f7fa0948740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c832772920 con 0x55c832760bb0 2024-09-19T08:46:11.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.855+0000 7f7fa0948740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c832772250 con 0x55c8327562f0 2024-09-19T08:46:11.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.856+0000 7f7fa0948740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c832772ff0 con 0x55c8327559d0 2024-09-19T08:46:11.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.856+0000 7f7fa0948740 10 client.0 _ll_register_callbacks cb 0x7f7f880b8550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:11.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.856+0000 7f7f9cb0c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c832760bb0 0x55c8327d9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:11.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.856+0000 7f7f9cb0c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c832760bb0 0x55c8327d9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38974/0 (socket says 172.21.15.164:38974) 2024-09-19T08:46:11.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.856+0000 7f7f9cb0c640 1 -- 172.21.15.164:0/2606358274 learned_addr learned my addr 172.21.15.164:0/2606358274 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:11.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.856+0000 7f7f8ffff640 1 --2- 172.21.15.164:0/2606358274 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8327562f0 0x55c8327747a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:11.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.856+0000 7f7f97fff640 1 --2- 172.21.15.164:0/2606358274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8327559d0 0x55c832774260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:11.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.856+0000 7f7f9cb0c640 1 -- 172.21.15.164:0/2606358274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8327559d0 msgr2=0x55c832774260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:11.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.856+0000 7f7f9cb0c640 1 --2- 172.21.15.164:0/2606358274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8327559d0 0x55c832774260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.856+0000 7f7f9cb0c640 1 -- 172.21.15.164:0/2606358274 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8327562f0 msgr2=0x55c8327747a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:11.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.856+0000 7f7f9cb0c640 1 --2- 172.21.15.164:0/2606358274 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8327562f0 0x55c8327747a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.856+0000 7f7f9cb0c640 1 -- 172.21.15.164:0/2606358274 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8327da270 con 0x55c832760bb0 2024-09-19T08:46:11.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.856+0000 7f7f9cb0c640 1 --2- 172.21.15.164:0/2606358274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c832760bb0 0x55c8327d9d30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7f90004410 tx=0x7f7f90012ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98568da553830a25 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:11.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.856+0000 7f7f777fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:46:11.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.856+0000 7f7f777fe640 1 -- 172.21.15.164:0/2606358274 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f90021490 con 0x55c832760bb0 2024-09-19T08:46:11.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.856+0000 7f7f777fe640 1 -- 172.21.15.164:0/2606358274 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f7f90021630 con 0x55c832760bb0 2024-09-19T08:46:11.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.856+0000 7f7fa0948740 1 -- 172.21.15.164:0/2606358274 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f900218f0 con 0x55c832760bb0 2024-09-19T08:46:11.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.856+0000 7f7f777fe640 1 -- 172.21.15.164:0/2606358274 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7f9003c050 con 0x55c832760bb0 2024-09-19T08:46:11.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.856+0000 7f7fa0948740 10 client.15459 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:11.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.856+0000 7f7fa0948740 1 -- 172.21.15.164:0/2606358274 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c8327db520 con 0x55c832760bb0 2024-09-19T08:46:11.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.857+0000 7f7f777fe640 1 -- 172.21.15.164:0/2606358274 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f900218f0 con 0x55c832760bb0 2024-09-19T08:46:11.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.857+0000 7f7f777fe640 1 -- 172.21.15.164:0/2606358274 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7f9000f040 con 0x55c832760bb0 2024-09-19T08:46:11.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.857+0000 7f7fa0948740 10 client.15459 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:11.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.857+0000 7f7fa0948740 10 client.15459 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T08:46:11.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.857+0000 7f7fa0948740 20 client.15459 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:11.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.857+0000 7f7fa0948740 1 --2- 172.21.15.164:0/2606358274 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55c8327dfd80 0x55c8327dea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:11.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.857+0000 7f7fa0948740 4 client.15459 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:46:11.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.857+0000 7f7fa0948740 1 -- 172.21.15.164:0/2606358274 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "status"}) -- 0x55c8326c3b10 con 0x55c8327dfd80 2024-09-19T08:46:11.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.857+0000 7f7f97fff640 1 --2- 172.21.15.164:0/2606358274 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55c8327dfd80 0x55c8327dea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:11.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.857+0000 7f7f777fe640 10 client.15459 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:46:11.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.857+0000 7f7f97fff640 1 --2- 172.21.15.164:0/2606358274 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55c8327dfd80 0x55c8327dea90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2dba2c41fc774348 server_cookie=4b543c44e1320ed0 in_seq=0 out_seq=0 2024-09-19T08:46:11.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.858+0000 7f7f777fe640 1 -- 172.21.15.164:0/2606358274 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55c8326c3b10 con 0x55c8327dfd80 2024-09-19T08:46:11.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.858+0000 7f7f777fe640 10 client.15459 handle_command_reply: tid=0 2024-09-19T08:46:11.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.858+0000 7f7fa0948740 1 client.15459 shutdown 2024-09-19T08:46:11.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.858+0000 7f7fa0948740 1 -- 172.21.15.164:0/2606358274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c832760bb0 msgr2=0x55c8327d9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:11.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.858+0000 7f7fa0948740 1 --2- 172.21.15.164:0/2606358274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c832760bb0 0x55c8327d9d30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7f90004410 tx=0x7f7f90012ea0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.858+0000 7f7fa0948740 1 -- 172.21.15.164:0/2606358274 shutdown_connections 2024-09-19T08:46:11.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.858+0000 7f7fa0948740 1 --2- 172.21.15.164:0/2606358274 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55c8327dfd80 0x55c8327dea90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.858+0000 7f7fa0948740 1 --2- 172.21.15.164:0/2606358274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c832760bb0 0x55c8327d9d30 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.858+0000 7f7f777fe640 0 client.15459 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:46:11.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.858+0000 7f7fa0948740 1 --2- 172.21.15.164:0/2606358274 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8327562f0 0x55c8327747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.858+0000 7f7fa0948740 1 --2- 172.21.15.164:0/2606358274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8327559d0 0x55c832774260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.858+0000 7f7fa0948740 1 -- 172.21.15.164:0/2606358274 >> 172.21.15.164:0/2606358274 conn(0x55c832751f60 msgr2=0x55c832752340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:11.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.858+0000 7f7fa0948740 1 -- 172.21.15.164:0/2606358274 shutdown_connections 2024-09-19T08:46:11.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.858+0000 7f7fa0948740 1 -- 172.21.15.164:0/2606358274 wait complete. 2024-09-19T08:46:11.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.858+0000 7f7fa0948740 20 client.15459 trim_cache size 0 max 16384 2024-09-19T08:46:11.878 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:46:11.878 INFO:teuthology.orchestra.run.smithi164.stdout: "cluster_fsid": "1e4b0d74-7662-11ef-bceb-c7b262605968", 2024-09-19T08:46:11.878 INFO:teuthology.orchestra.run.smithi164.stdout: "whoami": 4, 2024-09-19T08:46:11.878 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 14595, 2024-09-19T08:46:11.878 INFO:teuthology.orchestra.run.smithi164.stdout: "want_state": "up:active", 2024-09-19T08:46:11.878 INFO:teuthology.orchestra.run.smithi164.stdout: "state": "up:active", 2024-09-19T08:46:11.878 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs", 2024-09-19T08:46:11.878 INFO:teuthology.orchestra.run.smithi164.stdout: "rank_uptime": 127.22934193, 2024-09-19T08:46:11.878 INFO:teuthology.orchestra.run.smithi164.stdout: "mdsmap_epoch": 28, 2024-09-19T08:46:11.878 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch": 83, 2024-09-19T08:46:11.878 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch_barrier": 82, 2024-09-19T08:46:11.878 INFO:teuthology.orchestra.run.smithi164.stdout: "uptime": 152.49015933300001 2024-09-19T08:46:11.879 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:46:11.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.859+0000 7f7f9e56f640 1 -- 172.21.15.164:0/3027963448 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f7f840a34d0 msgr2=0x7f7f840a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:11.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.859+0000 7f7f9e56f640 1 --2- 172.21.15.164:0/3027963448 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f7f840a34d0 0x7f7f840a5990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7f90005fd0 tx=0x7f7f90004a40 comp rx=0 tx=0).stop 2024-09-19T08:46:11.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.859+0000 7f7f9e56f640 1 -- 172.21.15.164:0/3027963448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f98102d30 msgr2=0x7f7f98100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:11.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.859+0000 7f7f9e56f640 1 --2- 172.21.15.164:0/3027963448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f98102d30 0x7f7f98100990 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009230 tx=0x7f7f88004270 comp rx=0 tx=0).stop 2024-09-19T08:46:11.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.859+0000 7f7f9e56f640 1 -- 172.21.15.164:0/3027963448 shutdown_connections 2024-09-19T08:46:11.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.859+0000 7f7f9e56f640 1 --2- 172.21.15.164:0/3027963448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f981082a0 0x7f7f98101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.859+0000 7f7f9e56f640 1 --2- 172.21.15.164:0/3027963448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f981036e0 0x7f7f98100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.859+0000 7f7f9e56f640 1 --2- 172.21.15.164:0/3027963448 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f7f840a34d0 0x7f7f840a5990 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.859+0000 7f7f9e56f640 1 --2- 172.21.15.164:0/3027963448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f98102d30 0x7f7f98100990 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.859+0000 7f7f9e56f640 1 -- 172.21.15.164:0/3027963448 >> 172.21.15.164:0/3027963448 conn(0x7f7f980fc7d0 msgr2=0x7f7f98105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:11.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.859+0000 7f7f9e56f640 1 -- 172.21.15.164:0/3027963448 shutdown_connections 2024-09-19T08:46:11.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:11.859+0000 7f7f9e56f640 1 -- 172.21.15.164:0/3027963448 wait complete. 2024-09-19T08:46:12.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:11 smithi170 ceph-mon[35072]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 99 KiB/s wr, 60 op/s 2024-09-19T08:46:12.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:11 smithi173 ceph-mon[35306]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 99 KiB/s wr, 60 op/s 2024-09-19T08:46:12.651 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T08:46:12.898 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:46:13.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.348+0000 7f7c0ec14640 1 -- 172.21.15.164:0/3252164901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c08101420 msgr2=0x7f7c0810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:13.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.348+0000 7f7c0ec14640 1 --2- 172.21.15.164:0/3252164901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c08101420 0x7f7c0810d950 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7c00009cd0 tx=0x7f7c0002a8e0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.348+0000 7f7c0ec14640 1 -- 172.21.15.164:0/3252164901 shutdown_connections 2024-09-19T08:46:13.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.348+0000 7f7c0ec14640 1 --2- 172.21.15.164:0/3252164901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c08101420 0x7f7c0810d950 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.348+0000 7f7c0ec14640 1 --2- 172.21.15.164:0/3252164901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c08100a80 0x7f7c08100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.348+0000 7f7c0ec14640 1 --2- 172.21.15.164:0/3252164901 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c08106a80 0x7f7c08106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.348+0000 7f7c0ec14640 1 -- 172.21.15.164:0/3252164901 >> 172.21.15.164:0/3252164901 conn(0x7f7c080fc7a0 msgr2=0x7f7c080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:13.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.348+0000 7f7c0ec14640 1 -- 172.21.15.164:0/3252164901 shutdown_connections 2024-09-19T08:46:13.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.348+0000 7f7c0ec14640 1 -- 172.21.15.164:0/3252164901 wait complete. 2024-09-19T08:46:13.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.349+0000 7f7c0ec14640 1 Processor -- start 2024-09-19T08:46:13.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.349+0000 7f7c0ec14640 1 -- start start 2024-09-19T08:46:13.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.349+0000 7f7c0ec14640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c08100a80 0x7f7c08199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:13.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.349+0000 7f7c0ec14640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c08101420 0x7f7c081996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:13.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.349+0000 7f7c0ec14640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c08106a80 0x7f7c081a4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:13.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.349+0000 7f7c0ec14640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7c08110e80 con 0x7f7c08100a80 2024-09-19T08:46:13.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.349+0000 7f7c0ec14640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7c08110d00 con 0x7f7c08106a80 2024-09-19T08:46:13.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.349+0000 7f7c0ec14640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7c08111000 con 0x7f7c08101420 2024-09-19T08:46:13.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.349+0000 7f7c0c9b0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c08100a80 0x7f7c08199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:13.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.349+0000 7f7c0c9b0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c08100a80 0x7f7c08199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39000/0 (socket says 172.21.15.164:39000) 2024-09-19T08:46:13.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.349+0000 7f7c0c9b0640 1 -- 172.21.15.164:0/1207414279 learned_addr learned my addr 172.21.15.164:0/1207414279 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:13.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.349+0000 7f7c0d1b1640 1 --2- 172.21.15.164:0/1207414279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c08106a80 0x7f7c081a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:13.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.349+0000 7f7bfffff640 1 --2- 172.21.15.164:0/1207414279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c08101420 0x7f7c081996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:13.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.349+0000 7f7c0c9b0640 1 -- 172.21.15.164:0/1207414279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c08101420 msgr2=0x7f7c081996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:13.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.349+0000 7f7c0c9b0640 1 --2- 172.21.15.164:0/1207414279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c08101420 0x7f7c081996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.349+0000 7f7c0c9b0640 1 -- 172.21.15.164:0/1207414279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c08106a80 msgr2=0x7f7c081a4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:13.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.349+0000 7f7c0c9b0640 1 --2- 172.21.15.164:0/1207414279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c08106a80 0x7f7c081a4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.349+0000 7f7c0c9b0640 1 -- 172.21.15.164:0/1207414279 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c081a4750 con 0x7f7c08100a80 2024-09-19T08:46:13.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.350+0000 7f7c0c9b0640 1 --2- 172.21.15.164:0/1207414279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c08100a80 0x7f7c08199190 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400c970 tx=0x7f7c0400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:13.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.350+0000 7f7bfdffb640 1 -- 172.21.15.164:0/1207414279 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04007bb0 con 0x7f7c08100a80 2024-09-19T08:46:13.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.350+0000 7f7bfdffb640 1 -- 172.21.15.164:0/1207414279 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c04007d50 con 0x7f7c08100a80 2024-09-19T08:46:13.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.350+0000 7f7bfdffb640 1 -- 172.21.15.164:0/1207414279 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0400f450 con 0x7f7c08100a80 2024-09-19T08:46:13.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.350+0000 7f7c0ec14640 1 -- 172.21.15.164:0/1207414279 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c081a4920 con 0x7f7c08100a80 2024-09-19T08:46:13.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.350+0000 7f7c0ec14640 1 -- 172.21.15.164:0/1207414279 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c081a4e10 con 0x7f7c08100a80 2024-09-19T08:46:13.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.352+0000 7f7bfdffb640 1 -- 172.21.15.164:0/1207414279 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7f7c04005020 con 0x7f7c08100a80 2024-09-19T08:46:13.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.352+0000 7f7c10fed740 1 Processor -- start 2024-09-19T08:46:13.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.352+0000 7f7c10fed740 1 -- start start 2024-09-19T08:46:13.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.353+0000 7f7bfdffb640 1 --2- 172.21.15.164:0/1207414279 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f7bf00a34d0 0x7f7bf00a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:13.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.353+0000 7f7c10fed740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560613c358d0 0x560613c35cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:13.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.353+0000 7f7c10fed740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560613c361f0 0x560613c44a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:13.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.353+0000 7f7c10fed740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560613c44fc0 0x560613c473b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:13.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.353+0000 7f7c10fed740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560613b9c090 con 0x560613c44fc0 2024-09-19T08:46:13.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.353+0000 7f7c10fed740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560613b43a70 con 0x560613c358d0 2024-09-19T08:46:13.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.353+0000 7f7c10fed740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x560613b98050 con 0x560613c361f0 2024-09-19T08:46:13.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.353+0000 7f7c0d1b1640 1 --2- 172.21.15.164:0/1207414279 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f7bf00a34d0 0x7f7bf00a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:13.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.353+0000 7f7bfdffb640 1 -- 172.21.15.164:0/1207414279 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f7c040cfcf0 con 0x7f7c08100a80 2024-09-19T08:46:13.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.353+0000 7f7c0d1b1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560613c44fc0 0x560613c473b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:13.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.353+0000 7f7c0d1b1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560613c44fc0 0x560613c473b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39016/0 (socket says 172.21.15.164:39016) 2024-09-19T08:46:13.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.353+0000 7f7c0d1b1640 1 -- 172.21.15.164:0/933189751 learned_addr learned my addr 172.21.15.164:0/933189751 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:13.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.353+0000 7f7c0c9b0640 1 --2- 172.21.15.164:0/933189751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560613c358d0 0x560613c35cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:13.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.353+0000 7f7bfffff640 1 --2- 172.21.15.164:0/933189751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560613c361f0 0x560613c44a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:13.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.353+0000 7f7c0d1b1640 1 -- 172.21.15.164:0/933189751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560613c361f0 msgr2=0x560613c44a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:13.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.353+0000 7f7c0d1b1640 1 --2- 172.21.15.164:0/933189751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560613c361f0 0x560613c44a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.353+0000 7f7c0d1b1640 1 -- 172.21.15.164:0/933189751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560613c358d0 msgr2=0x560613c35cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:13.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.353+0000 7f7c0d1b1640 1 --2- 172.21.15.164:0/933189751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560613c358d0 0x560613c35cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.353+0000 7f7c0d1b1640 1 -- 172.21.15.164:0/933189751 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560613b50130 con 0x560613c44fc0 2024-09-19T08:46:13.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.353+0000 7f7c0c9b0640 1 --2- 172.21.15.164:0/933189751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560613c358d0 0x560613c35cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:46:13.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.353+0000 7f7c0d1b1640 1 --2- 172.21.15.164:0/1207414279 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f7bf00a34d0 0x7f7bf00a5990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7c0002ae00 tx=0x7f7c00009a20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:13.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.353+0000 7f7c0d1b1640 1 --2- 172.21.15.164:0/933189751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560613c44fc0 0x560613c473b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7c00009cd0 tx=0x7f7c00048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e88c44187f67f941 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:13.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.354+0000 7f7beeffd640 1 -- 172.21.15.164:0/933189751 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0004f070 con 0x560613c44fc0 2024-09-19T08:46:13.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.354+0000 7f7beeffd640 1 -- 172.21.15.164:0/933189751 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c00003e00 con 0x560613c44fc0 2024-09-19T08:46:13.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.354+0000 7f7beeffd640 1 -- 172.21.15.164:0/933189751 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c00030cf0 con 0x560613c44fc0 2024-09-19T08:46:13.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.354+0000 7f7c10fed740 1 -- 172.21.15.164:0/933189751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560613c44fc0 msgr2=0x560613c473b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:13.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.354+0000 7f7c10fed740 1 --2- 172.21.15.164:0/933189751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560613c44fc0 0x560613c473b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7c00009cd0 tx=0x7f7c00048000 comp rx=0 tx=0).stop 2024-09-19T08:46:13.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.354+0000 7f7c10fed740 1 -- 172.21.15.164:0/933189751 shutdown_connections 2024-09-19T08:46:13.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.354+0000 7f7c10fed740 1 --2- 172.21.15.164:0/933189751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560613c44fc0 0x560613c473b0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.354+0000 7f7c10fed740 1 --2- 172.21.15.164:0/933189751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560613c361f0 0x560613c44a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.354+0000 7f7c10fed740 1 --2- 172.21.15.164:0/933189751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560613c358d0 0x560613c35cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.354+0000 7f7c10fed740 1 -- 172.21.15.164:0/933189751 >> 172.21.15.164:0/933189751 conn(0x560613c31f60 msgr2=0x560613c32340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:13.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.354+0000 7f7c10fed740 1 -- 172.21.15.164:0/933189751 shutdown_connections 2024-09-19T08:46:13.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.354+0000 7f7c10fed740 1 -- 172.21.15.164:0/933189751 wait complete. 2024-09-19T08:46:13.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.355+0000 7f7c10fed740 1 -- start start 2024-09-19T08:46:13.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.355+0000 7f7c10fed740 1 Processor -- start 2024-09-19T08:46:13.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.355+0000 7f7c10fed740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560613c358d0 0x560613c44850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:13.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.355+0000 7f7c10fed740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560613c361f0 0x560613c41ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:13.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.355+0000 7f7c10fed740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560613c44fc0 0x560613c420e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:13.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.355+0000 7f7c10fed740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560613c47b90 con 0x560613c44fc0 2024-09-19T08:46:13.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.355+0000 7f7c10fed740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560613b56f20 con 0x560613c361f0 2024-09-19T08:46:13.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.355+0000 7f7c10fed740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x560613c47d10 con 0x560613c358d0 2024-09-19T08:46:13.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.355+0000 7f7c10fed740 10 client.0 _ll_register_callbacks cb 0x560613b20030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:13.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.355+0000 7f7c0d1b1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560613c44fc0 0x560613c420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:13.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.355+0000 7f7c0d1b1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560613c44fc0 0x560613c420e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39030/0 (socket says 172.21.15.164:39030) 2024-09-19T08:46:13.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.355+0000 7f7c0d1b1640 1 -- 172.21.15.164:0/3544612132 learned_addr learned my addr 172.21.15.164:0/3544612132 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:13.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.355+0000 7f7bfffff640 1 --2- 172.21.15.164:0/3544612132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560613c361f0 0x560613c41ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:13.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.355+0000 7f7c0c9b0640 1 --2- 172.21.15.164:0/3544612132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560613c358d0 0x560613c44850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:13.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.356+0000 7f7c0d1b1640 1 -- 172.21.15.164:0/3544612132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560613c358d0 msgr2=0x560613c44850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:13.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.356+0000 7f7c0d1b1640 1 --2- 172.21.15.164:0/3544612132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560613c358d0 0x560613c44850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.356+0000 7f7c0d1b1640 1 -- 172.21.15.164:0/3544612132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560613c361f0 msgr2=0x560613c41ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:13.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.356+0000 7f7c0d1b1640 1 --2- 172.21.15.164:0/3544612132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560613c361f0 0x560613c41ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.356+0000 7f7c0d1b1640 1 -- 172.21.15.164:0/3544612132 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56061390d2b0 con 0x560613c44fc0 2024-09-19T08:46:13.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.356+0000 7f7bfffff640 1 --2- 172.21.15.164:0/3544612132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560613c361f0 0x560613c41ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:46:13.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.356+0000 7f7c0d1b1640 1 --2- 172.21.15.164:0/3544612132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560613c44fc0 0x560613c420e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7c00003fd0 tx=0x7f7c000048c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7335e660e279f976 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:13.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.356+0000 7f7be7fff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:46:13.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.356+0000 7f7be7fff640 1 -- 172.21.15.164:0/3544612132 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c00045040 con 0x560613c44fc0 2024-09-19T08:46:13.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.356+0000 7f7be7fff640 1 -- 172.21.15.164:0/3544612132 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f7c00030c50 con 0x560613c44fc0 2024-09-19T08:46:13.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.356+0000 7f7c10fed740 1 -- 172.21.15.164:0/3544612132 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c00003e00 con 0x560613c44fc0 2024-09-19T08:46:13.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.356+0000 7f7be7fff640 1 -- 172.21.15.164:0/3544612132 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7c0004f040 con 0x560613c44fc0 2024-09-19T08:46:13.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.356+0000 7f7c10fed740 10 client.15471 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:13.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.356+0000 7f7c10fed740 1 -- 172.21.15.164:0/3544612132 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560613c53560 con 0x560613c44fc0 2024-09-19T08:46:13.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.356+0000 7f7be7fff640 1 -- 172.21.15.164:0/3544612132 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c00003e00 con 0x560613c44fc0 2024-09-19T08:46:13.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.357+0000 7f7be7fff640 1 -- 172.21.15.164:0/3544612132 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7c00030070 con 0x560613c44fc0 2024-09-19T08:46:13.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.357+0000 7f7c10fed740 10 client.15471 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:13.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.357+0000 7f7c10fed740 10 client.15471 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T08:46:13.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.357+0000 7f7c10fed740 20 client.15471 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:13.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.357+0000 7f7c10fed740 1 --2- 172.21.15.164:0/3544612132 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x560613c548b0 0x560613c56a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:13.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.357+0000 7f7c10fed740 4 client.15471 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:46:13.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.357+0000 7f7c10fed740 1 -- 172.21.15.164:0/3544612132 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560613ba9600 con 0x560613c548b0 2024-09-19T08:46:13.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.357+0000 7f7c0c9b0640 1 --2- 172.21.15.164:0/3544612132 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x560613c548b0 0x560613c56a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:13.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.357+0000 7f7c0c9b0640 1 --2- 172.21.15.164:0/3544612132 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x560613c548b0 0x560613c56a70 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f2292f7a93373371 server_cookie=fc84dfb4fce61e7a in_seq=0 out_seq=0 2024-09-19T08:46:13.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.357+0000 7f7be7fff640 10 client.15471 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:46:13.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.358+0000 7f7be7fff640 1 -- 172.21.15.164:0/3544612132 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560613ba9600 con 0x560613c548b0 2024-09-19T08:46:13.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.358+0000 7f7be7fff640 10 client.15471 handle_command_reply: tid=0 2024-09-19T08:46:13.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.358+0000 7f7c10fed740 1 client.15471 shutdown 2024-09-19T08:46:13.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.358+0000 7f7c10fed740 1 -- 172.21.15.164:0/3544612132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560613c44fc0 msgr2=0x560613c420e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:13.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.358+0000 7f7c10fed740 1 --2- 172.21.15.164:0/3544612132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560613c44fc0 0x560613c420e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7c00003fd0 tx=0x7f7c000048c0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.358+0000 7f7c10fed740 1 -- 172.21.15.164:0/3544612132 shutdown_connections 2024-09-19T08:46:13.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.358+0000 7f7c10fed740 1 --2- 172.21.15.164:0/3544612132 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x560613c548b0 0x560613c56a70 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.358+0000 7f7c10fed740 1 --2- 172.21.15.164:0/3544612132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560613c44fc0 0x560613c420e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.358+0000 7f7be7fff640 0 client.15471 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:46:13.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.358+0000 7f7c10fed740 1 --2- 172.21.15.164:0/3544612132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560613c361f0 0x560613c41ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.358+0000 7f7c10fed740 1 --2- 172.21.15.164:0/3544612132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560613c358d0 0x560613c44850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.358+0000 7f7c10fed740 1 -- 172.21.15.164:0/3544612132 >> 172.21.15.164:0/3544612132 conn(0x560613c31f60 msgr2=0x560613c37cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:13.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.359+0000 7f7c10fed740 1 -- 172.21.15.164:0/3544612132 shutdown_connections 2024-09-19T08:46:13.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.359+0000 7f7c10fed740 1 -- 172.21.15.164:0/3544612132 wait complete. 2024-09-19T08:46:13.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.359+0000 7f7c10fed740 20 client.15471 trim_cache size 0 max 16384 2024-09-19T08:46:13.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.376+0000 7f7c10fed740 1 Processor -- start 2024-09-19T08:46:13.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.376+0000 7f7c10fed740 1 -- start start 2024-09-19T08:46:13.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.376+0000 7f7c10fed740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560613c358d0 0x560613c55600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:13.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.376+0000 7f7c10fed740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560613c361f0 0x560613c3d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:13.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.376+0000 7f7c10fed740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560613c44fc0 0x560613c3ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:13.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.376+0000 7f7c10fed740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560613c54db0 con 0x560613c361f0 2024-09-19T08:46:13.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.376+0000 7f7c10fed740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560613c55480 con 0x560613c358d0 2024-09-19T08:46:13.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.376+0000 7f7c10fed740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x560613c55b50 con 0x560613c44fc0 2024-09-19T08:46:13.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.376+0000 7f7bfffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560613c361f0 0x560613c3d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:13.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.376+0000 7f7bfffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560613c361f0 0x560613c3d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39044/0 (socket says 172.21.15.164:39044) 2024-09-19T08:46:13.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.376+0000 7f7bfffff640 1 -- 172.21.15.164:0/3698404535 learned_addr learned my addr 172.21.15.164:0/3698404535 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:13.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.377+0000 7f7c0c9b0640 1 --2- 172.21.15.164:0/3698404535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560613c358d0 0x560613c55600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:13.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.377+0000 7f7c0d1b1640 1 --2- 172.21.15.164:0/3698404535 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560613c44fc0 0x560613c3ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:13.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.377+0000 7f7bfffff640 1 -- 172.21.15.164:0/3698404535 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560613c44fc0 msgr2=0x560613c3ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:13.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.377+0000 7f7bfffff640 1 --2- 172.21.15.164:0/3698404535 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560613c44fc0 0x560613c3ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.377+0000 7f7bfffff640 1 -- 172.21.15.164:0/3698404535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560613c358d0 msgr2=0x560613c55600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:13.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.377+0000 7f7bfffff640 1 --2- 172.21.15.164:0/3698404535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560613c358d0 0x560613c55600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.377+0000 7f7bfffff640 1 -- 172.21.15.164:0/3698404535 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560613c31d90 con 0x560613c361f0 2024-09-19T08:46:13.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.377+0000 7f7bfffff640 1 --2- 172.21.15.164:0/3698404535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560613c361f0 0x560613c3d130 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800be50 tx=0x7f7bf8010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f4797488d40d19 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:13.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.377+0000 7f7bee7fc640 1 -- 172.21.15.164:0/3698404535 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf800e680 con 0x560613c361f0 2024-09-19T08:46:13.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.377+0000 7f7bee7fc640 1 -- 172.21.15.164:0/3698404535 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bf800ed00 con 0x560613c361f0 2024-09-19T08:46:13.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.377+0000 7f7bee7fc640 1 -- 172.21.15.164:0/3698404535 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf8018390 con 0x560613c361f0 2024-09-19T08:46:13.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.377+0000 7f7c10fed740 1 -- 172.21.15.164:0/3698404535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560613c361f0 msgr2=0x560613c3d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:13.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.377+0000 7f7c10fed740 1 --2- 172.21.15.164:0/3698404535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560613c361f0 0x560613c3d130 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800be50 tx=0x7f7bf8010800 comp rx=0 tx=0).stop 2024-09-19T08:46:13.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.378+0000 7f7c10fed740 1 -- 172.21.15.164:0/3698404535 shutdown_connections 2024-09-19T08:46:13.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.378+0000 7f7c10fed740 1 --2- 172.21.15.164:0/3698404535 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560613c44fc0 0x560613c3ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.378+0000 7f7c10fed740 1 --2- 172.21.15.164:0/3698404535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560613c361f0 0x560613c3d130 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.378+0000 7f7c10fed740 1 --2- 172.21.15.164:0/3698404535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560613c358d0 0x560613c55600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.378+0000 7f7c10fed740 1 -- 172.21.15.164:0/3698404535 >> 172.21.15.164:0/3698404535 conn(0x560613c31f60 msgr2=0x560613c3ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:13.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.378+0000 7f7c10fed740 1 -- 172.21.15.164:0/3698404535 shutdown_connections 2024-09-19T08:46:13.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.378+0000 7f7c10fed740 1 -- 172.21.15.164:0/3698404535 wait complete. 2024-09-19T08:46:13.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.378+0000 7f7c10fed740 1 -- start start 2024-09-19T08:46:13.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.378+0000 7f7c10fed740 1 Processor -- start 2024-09-19T08:46:13.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.378+0000 7f7c10fed740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560613c358d0 0x560613c3e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:13.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.378+0000 7f7c10fed740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560613c361f0 0x560613c3ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:13.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.378+0000 7f7c10fed740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560613c44fc0 0x560613c5dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:13.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.378+0000 7f7c10fed740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560613c568f0 con 0x560613c358d0 2024-09-19T08:46:13.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.378+0000 7f7c10fed740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560613c56220 con 0x560613c361f0 2024-09-19T08:46:13.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.378+0000 7f7c10fed740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x560613c56fc0 con 0x560613c44fc0 2024-09-19T08:46:13.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.378+0000 7f7c10fed740 10 client.0 _ll_register_callbacks cb 0x7f7c04004920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:13.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.378+0000 7f7c0c9b0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560613c358d0 0x560613c3e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:13.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.378+0000 7f7c0c9b0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560613c358d0 0x560613c3e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39058/0 (socket says 172.21.15.164:39058) 2024-09-19T08:46:13.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.378+0000 7f7c0c9b0640 1 -- 172.21.15.164:0/2119587386 learned_addr learned my addr 172.21.15.164:0/2119587386 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:13.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.379+0000 7f7bfffff640 1 --2- 172.21.15.164:0/2119587386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560613c361f0 0x560613c3ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:13.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.379+0000 7f7c0d1b1640 1 --2- 172.21.15.164:0/2119587386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560613c44fc0 0x560613c5dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:13.418 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:46:13.419 INFO:teuthology.orchestra.run.smithi164.stdout: "cluster_fsid": "1e4b0d74-7662-11ef-bceb-c7b262605968", 2024-09-19T08:46:13.419 INFO:teuthology.orchestra.run.smithi164.stdout: "whoami": 3, 2024-09-19T08:46:13.419 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 24424, 2024-09-19T08:46:13.419 INFO:teuthology.orchestra.run.smithi164.stdout: "want_state": "up:active", 2024-09-19T08:46:13.419 INFO:teuthology.orchestra.run.smithi164.stdout: "state": "up:active", 2024-09-19T08:46:13.419 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs", 2024-09-19T08:46:13.419 INFO:teuthology.orchestra.run.smithi164.stdout: "rank_uptime": 129.55054744099999, 2024-09-19T08:46:13.419 INFO:teuthology.orchestra.run.smithi164.stdout: "mdsmap_epoch": 28, 2024-09-19T08:46:13.419 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch": 83, 2024-09-19T08:46:13.419 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch_barrier": 82, 2024-09-19T08:46:13.419 INFO:teuthology.orchestra.run.smithi164.stdout: "uptime": 165.21760631800001 2024-09-19T08:46:13.419 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:46:13.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.379+0000 7f7c0c9b0640 1 -- 172.21.15.164:0/2119587386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560613c44fc0 msgr2=0x560613c5dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:13.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.379+0000 7f7c0c9b0640 1 --2- 172.21.15.164:0/2119587386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560613c44fc0 0x560613c5dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.379+0000 7f7c0c9b0640 1 -- 172.21.15.164:0/2119587386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560613c361f0 msgr2=0x560613c3ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:13.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.379+0000 7f7c0c9b0640 1 --2- 172.21.15.164:0/2119587386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560613c361f0 0x560613c3ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.379+0000 7f7c0c9b0640 1 -- 172.21.15.164:0/2119587386 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560613c32bf0 con 0x560613c358d0 2024-09-19T08:46:13.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.379+0000 7f7c0d1b1640 1 --2- 172.21.15.164:0/2119587386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560613c44fc0 0x560613c5dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:13.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.379+0000 7f7c0c9b0640 1 --2- 172.21.15.164:0/2119587386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560613c358d0 0x560613c3e590 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7c04098250 tx=0x7f7c040076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a6163a7562b33e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:13.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.379+0000 7f7be7fff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:46:13.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.379+0000 7f7be7fff640 1 -- 172.21.15.164:0/2119587386 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0401d040 con 0x560613c358d0 2024-09-19T08:46:13.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.379+0000 7f7be7fff640 1 -- 172.21.15.164:0/2119587386 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f7c04098460 con 0x560613c358d0 2024-09-19T08:46:13.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.379+0000 7f7c10fed740 1 -- 172.21.15.164:0/2119587386 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bf800ed00 con 0x560613c358d0 2024-09-19T08:46:13.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.379+0000 7f7be7fff640 1 -- 172.21.15.164:0/2119587386 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7c0409bdf0 con 0x560613c358d0 2024-09-19T08:46:13.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.379+0000 7f7c10fed740 10 client.15477 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:13.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.379+0000 7f7c10fed740 1 -- 172.21.15.164:0/2119587386 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560613c5f340 con 0x560613c358d0 2024-09-19T08:46:13.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.379+0000 7f7be7fff640 1 -- 172.21.15.164:0/2119587386 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bf800ed00 con 0x560613c358d0 2024-09-19T08:46:13.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.380+0000 7f7be7fff640 1 -- 172.21.15.164:0/2119587386 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7c04081070 con 0x560613c358d0 2024-09-19T08:46:13.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.380+0000 7f7c10fed740 10 client.15477 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:13.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.380+0000 7f7c10fed740 10 client.15477 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T08:46:13.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.380+0000 7f7c10fed740 20 client.15477 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:13.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.380+0000 7f7c10fed740 1 --2- 172.21.15.164:0/2119587386 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x560613c548b0 0x560613cc2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:13.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.380+0000 7f7c10fed740 4 client.15477 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:46:13.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.380+0000 7f7c10fed740 1 -- 172.21.15.164:0/2119587386 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "status"}) -- 0x560613ba3b10 con 0x560613c548b0 2024-09-19T08:46:13.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.380+0000 7f7bfffff640 1 --2- 172.21.15.164:0/2119587386 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x560613c548b0 0x560613cc2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:13.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.380+0000 7f7be7fff640 10 client.15477 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:46:13.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.380+0000 7f7bfffff640 1 --2- 172.21.15.164:0/2119587386 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x560613c548b0 0x560613cc2de0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e45baf016d11d5a9 server_cookie=b01275d4d7a4d05e in_seq=0 out_seq=0 2024-09-19T08:46:13.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.381+0000 7f7be7fff640 1 -- 172.21.15.164:0/2119587386 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x560613ba3b10 con 0x560613c548b0 2024-09-19T08:46:13.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.381+0000 7f7be7fff640 10 client.15477 handle_command_reply: tid=0 2024-09-19T08:46:13.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.381+0000 7f7c10fed740 1 client.15477 shutdown 2024-09-19T08:46:13.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.381+0000 7f7c10fed740 1 -- 172.21.15.164:0/2119587386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560613c358d0 msgr2=0x560613c3e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:13.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.381+0000 7f7c10fed740 1 --2- 172.21.15.164:0/2119587386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560613c358d0 0x560613c3e590 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7c04098250 tx=0x7f7c040076c0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.381+0000 7f7c10fed740 1 -- 172.21.15.164:0/2119587386 shutdown_connections 2024-09-19T08:46:13.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.381+0000 7f7c10fed740 1 --2- 172.21.15.164:0/2119587386 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x560613c548b0 0x560613cc2de0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.381+0000 7f7c10fed740 1 --2- 172.21.15.164:0/2119587386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560613c44fc0 0x560613c5dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.381+0000 7f7be7fff640 0 client.15477 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:46:13.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.381+0000 7f7c10fed740 1 --2- 172.21.15.164:0/2119587386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560613c361f0 0x560613c3ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.381+0000 7f7c10fed740 1 --2- 172.21.15.164:0/2119587386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560613c358d0 0x560613c3e590 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.381+0000 7f7c10fed740 1 -- 172.21.15.164:0/2119587386 >> 172.21.15.164:0/2119587386 conn(0x560613c31f60 msgr2=0x560613c3d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:13.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.381+0000 7f7c10fed740 1 -- 172.21.15.164:0/2119587386 shutdown_connections 2024-09-19T08:46:13.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.381+0000 7f7c10fed740 1 -- 172.21.15.164:0/2119587386 wait complete. 2024-09-19T08:46:13.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.381+0000 7f7c10fed740 20 client.15477 trim_cache size 0 max 16384 2024-09-19T08:46:13.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.381+0000 7f7c0ec14640 1 -- 172.21.15.164:0/1207414279 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f7bf00a34d0 msgr2=0x7f7bf00a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:13.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.381+0000 7f7c0ec14640 1 --2- 172.21.15.164:0/1207414279 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f7bf00a34d0 0x7f7bf00a5990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7c0002ae00 tx=0x7f7c00009a20 comp rx=0 tx=0).stop 2024-09-19T08:46:13.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.381+0000 7f7c0ec14640 1 -- 172.21.15.164:0/1207414279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c08100a80 msgr2=0x7f7c08199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:13.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.381+0000 7f7c0ec14640 1 --2- 172.21.15.164:0/1207414279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c08100a80 0x7f7c08199190 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400c970 tx=0x7f7c0400ce30 comp rx=0 tx=0).stop 2024-09-19T08:46:13.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.381+0000 7f7c0ec14640 1 -- 172.21.15.164:0/1207414279 shutdown_connections 2024-09-19T08:46:13.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.381+0000 7f7c0ec14640 1 --2- 172.21.15.164:0/1207414279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c08106a80 0x7f7c081a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.381+0000 7f7c0ec14640 1 --2- 172.21.15.164:0/1207414279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c08101420 0x7f7c081996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.381+0000 7f7c0ec14640 1 --2- 172.21.15.164:0/1207414279 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f7bf00a34d0 0x7f7bf00a5990 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.381+0000 7f7c0ec14640 1 --2- 172.21.15.164:0/1207414279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c08100a80 0x7f7c08199190 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.381+0000 7f7c0ec14640 1 -- 172.21.15.164:0/1207414279 >> 172.21.15.164:0/1207414279 conn(0x7f7c080fc7a0 msgr2=0x7f7c080fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:13.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.382+0000 7f7c0ec14640 1 -- 172.21.15.164:0/1207414279 shutdown_connections 2024-09-19T08:46:13.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:13.382+0000 7f7c0ec14640 1 -- 172.21.15.164:0/1207414279 wait complete. 2024-09-19T08:46:14.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:13 smithi164 ceph-mon[30928]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T08:46:14.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:13 smithi170 ceph-mon[35072]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T08:46:14.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:13 smithi173 ceph-mon[35306]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T08:46:14.187 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T08:46:14.432 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:46:14.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.921+0000 7f144d446640 1 -- 172.21.15.164:0/968279685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1448101430 msgr2=0x7f144810d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:14.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.921+0000 7f144d446640 1 --2- 172.21.15.164:0/968279685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1448101430 0x7f144810d960 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1440009930 tx=0x7f144002a420 comp rx=0 tx=0).stop 2024-09-19T08:46:14.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.921+0000 7f144d446640 1 -- 172.21.15.164:0/968279685 shutdown_connections 2024-09-19T08:46:14.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.921+0000 7f144d446640 1 --2- 172.21.15.164:0/968279685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1448101430 0x7f144810d960 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:14.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.921+0000 7f144d446640 1 --2- 172.21.15.164:0/968279685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1448100a90 0x7f1448100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:14.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.921+0000 7f144d446640 1 --2- 172.21.15.164:0/968279685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1448106a90 0x7f1448106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:14.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.921+0000 7f144d446640 1 -- 172.21.15.164:0/968279685 >> 172.21.15.164:0/968279685 conn(0x7f14480fc7d0 msgr2=0x7f14480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:14.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.921+0000 7f144d446640 1 -- 172.21.15.164:0/968279685 shutdown_connections 2024-09-19T08:46:14.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.921+0000 7f144d446640 1 -- 172.21.15.164:0/968279685 wait complete. 2024-09-19T08:46:14.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.921+0000 7f144d446640 1 Processor -- start 2024-09-19T08:46:14.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.922+0000 7f144d446640 1 -- start start 2024-09-19T08:46:14.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.922+0000 7f144d446640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1448100a90 0x7f14480fcd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:14.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.922+0000 7f144d446640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1448101430 0x7f14480fd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:14.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.922+0000 7f144d446640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1448106a90 0x7f14480fd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:14.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.922+0000 7f144d446640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1448110e90 con 0x7f1448106a90 2024-09-19T08:46:14.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.922+0000 7f144d446640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1448110d10 con 0x7f1448101430 2024-09-19T08:46:14.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.922+0000 7f144d446640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1448111010 con 0x7f1448100a90 2024-09-19T08:46:14.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.922+0000 7f14477fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1448106a90 0x7f14480fd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:14.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.922+0000 7f14477fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1448106a90 0x7f14480fd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42260/0 (socket says 172.21.15.164:42260) 2024-09-19T08:46:14.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.922+0000 7f14477fe640 1 -- 172.21.15.164:0/1200190777 learned_addr learned my addr 172.21.15.164:0/1200190777 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:14.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.922+0000 7f14467fc640 1 --2- 172.21.15.164:0/1200190777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1448101430 0x7f14480fd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:14.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.922+0000 7f1446ffd640 1 --2- 172.21.15.164:0/1200190777 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1448100a90 0x7f14480fcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:14.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.922+0000 7f14477fe640 1 -- 172.21.15.164:0/1200190777 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1448100a90 msgr2=0x7f14480fcd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:14.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.922+0000 7f14477fe640 1 --2- 172.21.15.164:0/1200190777 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1448100a90 0x7f14480fcd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:14.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.922+0000 7f14477fe640 1 -- 172.21.15.164:0/1200190777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1448101430 msgr2=0x7f14480fd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:14.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.922+0000 7f14477fe640 1 --2- 172.21.15.164:0/1200190777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1448101430 0x7f14480fd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:14.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.922+0000 7f14477fe640 1 -- 172.21.15.164:0/1200190777 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14481a68e0 con 0x7f1448106a90 2024-09-19T08:46:14.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.922+0000 7f1446ffd640 1 --2- 172.21.15.164:0/1200190777 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1448100a90 0x7f14480fcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:14.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.923+0000 7f14477fe640 1 --2- 172.21.15.164:0/1200190777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1448106a90 0x7f14480fd7f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1440005ab0 tx=0x7f14400042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:14.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.923+0000 7f142ffff640 1 -- 172.21.15.164:0/1200190777 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144003f030 con 0x7f1448106a90 2024-09-19T08:46:14.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.923+0000 7f142ffff640 1 -- 172.21.15.164:0/1200190777 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1440003ce0 con 0x7f1448106a90 2024-09-19T08:46:14.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.923+0000 7f144d446640 1 -- 172.21.15.164:0/1200190777 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14481a6b70 con 0x7f1448106a90 2024-09-19T08:46:14.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.923+0000 7f142ffff640 1 -- 172.21.15.164:0/1200190777 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1440035070 con 0x7f1448106a90 2024-09-19T08:46:14.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.923+0000 7f144d446640 1 -- 172.21.15.164:0/1200190777 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14481a7050 con 0x7f1448106a90 2024-09-19T08:46:14.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.925+0000 7f142ffff640 1 -- 172.21.15.164:0/1200190777 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7f1440042030 con 0x7f1448106a90 2024-09-19T08:46:14.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.925+0000 7f144f81f740 1 Processor -- start 2024-09-19T08:46:14.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.925+0000 7f144f81f740 1 -- start start 2024-09-19T08:46:14.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.925+0000 7f144f81f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600860509d0 0x560086050db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:14.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.925+0000 7f144f81f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5600860512f0 0x56008605b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:14.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.925+0000 7f144f81f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56008605bbb0 0x56008605dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:14.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.925+0000 7f144f81f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560085fb7090 con 0x5600860509d0 2024-09-19T08:46:14.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.925+0000 7f144f81f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560085f5ea70 con 0x56008605bbb0 2024-09-19T08:46:14.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.925+0000 7f144f81f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x560085fb3050 con 0x5600860512f0 2024-09-19T08:46:14.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.926+0000 7f14467fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600860509d0 0x560086050db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:14.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.926+0000 7f142ffff640 1 --2- 172.21.15.164:0/1200190777 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f14300a34d0 0x7f14300a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:14.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.926+0000 7f14467fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600860509d0 0x560086050db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42266/0 (socket says 172.21.15.164:42266) 2024-09-19T08:46:14.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.926+0000 7f14467fc640 1 -- 172.21.15.164:0/3214078624 learned_addr learned my addr 172.21.15.164:0/3214078624 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:14.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.926+0000 7f1446ffd640 1 --2- 172.21.15.164:0/3214078624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5600860512f0 0x56008605b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:14.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.926+0000 7f14477fe640 1 --2- 172.21.15.164:0/1200190777 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f14300a34d0 0x7f14300a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:14.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.926+0000 7f142ffff640 1 -- 172.21.15.164:0/1200190777 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f14400f0310 con 0x7f1448106a90 2024-09-19T08:46:14.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.926+0000 7f14467fc640 1 -- 172.21.15.164:0/3214078624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5600860512f0 msgr2=0x56008605b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:14.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.926+0000 7f14467fc640 1 --2- 172.21.15.164:0/3214078624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5600860512f0 0x56008605b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:14.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.926+0000 7f14467fc640 1 -- 172.21.15.164:0/3214078624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56008605bbb0 msgr2=0x56008605dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:14.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.926+0000 7f14467fc640 1 --2- 172.21.15.164:0/3214078624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56008605bbb0 0x56008605dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:14.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.926+0000 7f14467fc640 1 -- 172.21.15.164:0/3214078624 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560085f6b130 con 0x5600860509d0 2024-09-19T08:46:14.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.926+0000 7f14467fc640 1 --2- 172.21.15.164:0/3214078624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600860509d0 0x560086050db0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f143c00b790 tx=0x7f143c00bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af09bacc9389f6c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:14.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.926+0000 7f14477fe640 1 --2- 172.21.15.164:0/1200190777 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f14300a34d0 0x7f14300a5990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f14400ed9b0 tx=0x7f1440009260 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:14.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.926+0000 7f142d7fa640 1 -- 172.21.15.164:0/3214078624 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c00bea0 con 0x5600860509d0 2024-09-19T08:46:14.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.926+0000 7f142d7fa640 1 -- 172.21.15.164:0/3214078624 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f143c016030 con 0x5600860509d0 2024-09-19T08:46:14.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.926+0000 7f142d7fa640 1 -- 172.21.15.164:0/3214078624 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c004dd0 con 0x5600860509d0 2024-09-19T08:46:14.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.927+0000 7f144f81f740 1 -- 172.21.15.164:0/3214078624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600860509d0 msgr2=0x560086050db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:14.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.927+0000 7f144f81f740 1 --2- 172.21.15.164:0/3214078624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600860509d0 0x560086050db0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f143c00b790 tx=0x7f143c00bc50 comp rx=0 tx=0).stop 2024-09-19T08:46:14.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.927+0000 7f144f81f740 1 -- 172.21.15.164:0/3214078624 shutdown_connections 2024-09-19T08:46:14.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.927+0000 7f144f81f740 1 --2- 172.21.15.164:0/3214078624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56008605bbb0 0x56008605dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:14.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.927+0000 7f144f81f740 1 --2- 172.21.15.164:0/3214078624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5600860512f0 0x56008605b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:14.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.927+0000 7f144f81f740 1 --2- 172.21.15.164:0/3214078624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600860509d0 0x560086050db0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:14.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.927+0000 7f144f81f740 1 -- 172.21.15.164:0/3214078624 >> 172.21.15.164:0/3214078624 conn(0x56008604cf60 msgr2=0x56008604d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:14.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.927+0000 7f144f81f740 1 -- 172.21.15.164:0/3214078624 shutdown_connections 2024-09-19T08:46:14.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.927+0000 7f144f81f740 1 -- 172.21.15.164:0/3214078624 wait complete. 2024-09-19T08:46:14.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.927+0000 7f144f81f740 1 -- start start 2024-09-19T08:46:14.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.927+0000 7f144f81f740 1 Processor -- start 2024-09-19T08:46:14.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.928+0000 7f144f81f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600860509d0 0x56008605b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:14.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.928+0000 7f144f81f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600860512f0 0x560086058760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:14.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.928+0000 7f144f81f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56008605bbb0 0x560086058ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:14.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.928+0000 7f144f81f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56008605e780 con 0x5600860509d0 2024-09-19T08:46:14.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.928+0000 7f144f81f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560085f71f20 con 0x5600860512f0 2024-09-19T08:46:14.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.928+0000 7f144f81f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56008605e900 con 0x56008605bbb0 2024-09-19T08:46:14.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.928+0000 7f144f81f740 10 client.0 _ll_register_callbacks cb 0x560085f3b030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:14.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.928+0000 7f14467fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600860509d0 0x56008605b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:14.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.928+0000 7f14467fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600860509d0 0x56008605b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42280/0 (socket says 172.21.15.164:42280) 2024-09-19T08:46:14.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.928+0000 7f14467fc640 1 -- 172.21.15.164:0/1533426006 learned_addr learned my addr 172.21.15.164:0/1533426006 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:14.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.928+0000 7f14467fc640 1 -- 172.21.15.164:0/1533426006 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56008605bbb0 msgr2=0x560086058ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:14.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.928+0000 7f1446ffd640 1 --2- 172.21.15.164:0/1533426006 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600860512f0 0x560086058760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:14.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.928+0000 7f14467fc640 1 --2- 172.21.15.164:0/1533426006 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56008605bbb0 0x560086058ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:14.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.928+0000 7f14467fc640 1 -- 172.21.15.164:0/1533426006 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600860512f0 msgr2=0x560086058760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:14.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.928+0000 7f14467fc640 1 --2- 172.21.15.164:0/1533426006 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600860512f0 0x560086058760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:14.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.928+0000 7f14467fc640 1 -- 172.21.15.164:0/1533426006 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560085d282b0 con 0x5600860509d0 2024-09-19T08:46:14.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.928+0000 7f14467fc640 1 --2- 172.21.15.164:0/1533426006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600860509d0 0x56008605b410 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f143c004390 tx=0x7f143c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b519944b704a4747 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:14.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.928+0000 7f142aff5640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:46:14.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.928+0000 7f142aff5640 1 -- 172.21.15.164:0/1533426006 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c01b670 con 0x5600860509d0 2024-09-19T08:46:14.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.928+0000 7f142aff5640 1 -- 172.21.15.164:0/1533426006 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f143c01b810 con 0x5600860509d0 2024-09-19T08:46:14.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.928+0000 7f144f81f740 1 -- 172.21.15.164:0/1533426006 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f143c016030 con 0x5600860509d0 2024-09-19T08:46:14.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.929+0000 7f142aff5640 1 -- 172.21.15.164:0/1533426006 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f143c030050 con 0x5600860509d0 2024-09-19T08:46:14.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.929+0000 7f144f81f740 10 client.15489 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:14.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.929+0000 7f144f81f740 1 -- 172.21.15.164:0/1533426006 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56008606a0a0 con 0x5600860509d0 2024-09-19T08:46:14.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.929+0000 7f142aff5640 1 -- 172.21.15.164:0/1533426006 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f143c016030 con 0x5600860509d0 2024-09-19T08:46:14.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.929+0000 7f142aff5640 1 -- 172.21.15.164:0/1533426006 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f143c010ad0 con 0x5600860509d0 2024-09-19T08:46:14.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.929+0000 7f144f81f740 10 client.15489 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:14.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.929+0000 7f144f81f740 10 client.15489 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T08:46:14.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.929+0000 7f144f81f740 20 client.15489 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:14.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.929+0000 7f144f81f740 1 --2- 172.21.15.164:0/1533426006 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x56008606c4c0 0x56008606d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:14.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.929+0000 7f144f81f740 4 client.15489 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:46:14.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.929+0000 7f144f81f740 1 -- 172.21.15.164:0/1533426006 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560085fc4600 con 0x56008606c4c0 2024-09-19T08:46:14.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.930+0000 7f1446ffd640 1 --2- 172.21.15.164:0/1533426006 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x56008606c4c0 0x56008606d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:14.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.930+0000 7f142aff5640 10 client.15489 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:46:14.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.930+0000 7f1446ffd640 1 --2- 172.21.15.164:0/1533426006 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x56008606c4c0 0x56008606d930 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=466b1b43a9eabad7 server_cookie=cc7762c4320fb7a in_seq=0 out_seq=0 2024-09-19T08:46:14.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.931+0000 7f142aff5640 1 -- 172.21.15.164:0/1533426006 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560085fc4600 con 0x56008606c4c0 2024-09-19T08:46:14.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.931+0000 7f142aff5640 10 client.15489 handle_command_reply: tid=0 2024-09-19T08:46:14.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.931+0000 7f144f81f740 1 client.15489 shutdown 2024-09-19T08:46:14.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.931+0000 7f144f81f740 1 -- 172.21.15.164:0/1533426006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600860509d0 msgr2=0x56008605b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:14.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.931+0000 7f144f81f740 1 --2- 172.21.15.164:0/1533426006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600860509d0 0x56008605b410 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f143c004390 tx=0x7f143c0046c0 comp rx=0 tx=0).stop 2024-09-19T08:46:14.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.931+0000 7f144f81f740 1 -- 172.21.15.164:0/1533426006 shutdown_connections 2024-09-19T08:46:14.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.931+0000 7f144f81f740 1 --2- 172.21.15.164:0/1533426006 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x56008606c4c0 0x56008606d930 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:14.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.931+0000 7f144f81f740 1 --2- 172.21.15.164:0/1533426006 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56008605bbb0 0x560086058ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:14.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.931+0000 7f144f81f740 1 --2- 172.21.15.164:0/1533426006 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600860512f0 0x560086058760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:14.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.931+0000 7f142aff5640 0 client.15489 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:46:14.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.931+0000 7f144f81f740 1 --2- 172.21.15.164:0/1533426006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600860509d0 0x56008605b410 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:14.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.931+0000 7f144f81f740 1 -- 172.21.15.164:0/1533426006 >> 172.21.15.164:0/1533426006 conn(0x56008604cf60 msgr2=0x560086052c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:14.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.931+0000 7f144f81f740 1 -- 172.21.15.164:0/1533426006 shutdown_connections 2024-09-19T08:46:14.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.931+0000 7f144f81f740 1 -- 172.21.15.164:0/1533426006 wait complete. 2024-09-19T08:46:14.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.931+0000 7f144f81f740 20 client.15489 trim_cache size 0 max 16384 2024-09-19T08:46:14.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.948+0000 7f144f81f740 1 Processor -- start 2024-09-19T08:46:14.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.948+0000 7f144f81f740 1 -- start start 2024-09-19T08:46:14.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.949+0000 7f144f81f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600860509d0 0x56008606d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:14.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.949+0000 7f144f81f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5600860512f0 0x560086047ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:14.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.949+0000 7f144f81f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56008605bbb0 0x560086057030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:14.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.949+0000 7f144f81f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56008606bc70 con 0x56008605bbb0 2024-09-19T08:46:14.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.949+0000 7f144f81f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56008606c340 con 0x5600860509d0 2024-09-19T08:46:14.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.949+0000 7f144f81f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56008606ca10 con 0x5600860512f0 2024-09-19T08:46:14.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.949+0000 7f14467fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56008605bbb0 0x560086057030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:14.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.949+0000 7f14467fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56008605bbb0 0x560086057030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42296/0 (socket says 172.21.15.164:42296) 2024-09-19T08:46:14.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.949+0000 7f14467fc640 1 -- 172.21.15.164:0/2129859525 learned_addr learned my addr 172.21.15.164:0/2129859525 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:14.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.949+0000 7f14467fc640 1 --2- 172.21.15.164:0/2129859525 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600860509d0 0x56008606d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:14.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.949+0000 7f1446ffd640 1 --2- 172.21.15.164:0/2129859525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5600860512f0 0x560086047ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:14.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.949+0000 7f14467fc640 1 -- 172.21.15.164:0/2129859525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5600860512f0 msgr2=0x560086047ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:14.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.949+0000 7f14467fc640 1 --2- 172.21.15.164:0/2129859525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5600860512f0 0x560086047ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:14.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.949+0000 7f14467fc640 1 -- 172.21.15.164:0/2129859525 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600860509d0 msgr2=0x56008606d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:14.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.949+0000 7f14467fc640 1 --2- 172.21.15.164:0/2129859525 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600860509d0 0x56008606d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:14.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.949+0000 7f14467fc640 1 -- 172.21.15.164:0/2129859525 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56008605b780 con 0x56008605bbb0 2024-09-19T08:46:14.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.949+0000 7f1446ffd640 1 --2- 172.21.15.164:0/2129859525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5600860512f0 0x560086047ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:14.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.949+0000 7f14467fc640 1 --2- 172.21.15.164:0/2129859525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56008605bbb0 0x560086057030 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x5600860592c0 tx=0x7f143c00bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97cbdc4632a82721 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:14.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.949+0000 7f142cff9640 1 -- 172.21.15.164:0/2129859525 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c02a2d0 con 0x56008605bbb0 2024-09-19T08:46:14.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.949+0000 7f142cff9640 1 -- 172.21.15.164:0/2129859525 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f143c016070 con 0x56008605bbb0 2024-09-19T08:46:14.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.949+0000 7f142cff9640 1 -- 172.21.15.164:0/2129859525 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c010870 con 0x56008605bbb0 2024-09-19T08:46:14.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.950+0000 7f144f81f740 1 -- 172.21.15.164:0/2129859525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56008605bbb0 msgr2=0x560086057030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:14.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.950+0000 7f144f81f740 1 --2- 172.21.15.164:0/2129859525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56008605bbb0 0x560086057030 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x5600860592c0 tx=0x7f143c00bc50 comp rx=0 tx=0).stop 2024-09-19T08:46:14.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.950+0000 7f144f81f740 1 -- 172.21.15.164:0/2129859525 shutdown_connections 2024-09-19T08:46:14.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.950+0000 7f144f81f740 1 --2- 172.21.15.164:0/2129859525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56008605bbb0 0x560086057030 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:14.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.950+0000 7f144f81f740 1 --2- 172.21.15.164:0/2129859525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5600860512f0 0x560086047ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:14.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.950+0000 7f144f81f740 1 --2- 172.21.15.164:0/2129859525 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600860509d0 0x56008606d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:14.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.950+0000 7f144f81f740 1 -- 172.21.15.164:0/2129859525 >> 172.21.15.164:0/2129859525 conn(0x56008604cf60 msgr2=0x56008606b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:14.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.950+0000 7f144f81f740 1 -- 172.21.15.164:0/2129859525 shutdown_connections 2024-09-19T08:46:14.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.950+0000 7f144f81f740 1 -- 172.21.15.164:0/2129859525 wait complete. 2024-09-19T08:46:14.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.950+0000 7f144f81f740 1 -- start start 2024-09-19T08:46:14.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.950+0000 7f144f81f740 1 Processor -- start 2024-09-19T08:46:14.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.951+0000 7f144f81f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600860509d0 0x560086071150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:14.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.951+0000 7f144f81f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600860512f0 0x560086071690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:14.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.951+0000 7f144f81f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56008605bbb0 0x5600860d4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:14.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.951+0000 7f144f81f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56008606d7b0 con 0x5600860509d0 2024-09-19T08:46:14.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.951+0000 7f144f81f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56008606d0e0 con 0x5600860512f0 2024-09-19T08:46:14.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.951+0000 7f144f81f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56008606de80 con 0x56008605bbb0 2024-09-19T08:46:14.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.951+0000 7f144f81f740 10 client.0 _ll_register_callbacks cb 0x7f143c00b690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:14.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.951+0000 7f14467fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600860509d0 0x560086071150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:14.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.951+0000 7f14467fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600860509d0 0x560086071150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42310/0 (socket says 172.21.15.164:42310) 2024-09-19T08:46:14.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.951+0000 7f14467fc640 1 -- 172.21.15.164:0/2756736780 learned_addr learned my addr 172.21.15.164:0/2756736780 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:14.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.951+0000 7f1446ffd640 1 --2- 172.21.15.164:0/2756736780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600860512f0 0x560086071690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:14.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.951+0000 7f14467fc640 1 --2- 172.21.15.164:0/2756736780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56008605bbb0 0x5600860d4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:14.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.951+0000 7f14467fc640 1 -- 172.21.15.164:0/2756736780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56008605bbb0 msgr2=0x5600860d4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:14.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.951+0000 7f14467fc640 1 --2- 172.21.15.164:0/2756736780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56008605bbb0 0x5600860d4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:14.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.951+0000 7f14467fc640 1 -- 172.21.15.164:0/2756736780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600860512f0 msgr2=0x560086071690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:14.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.951+0000 7f14467fc640 1 --2- 172.21.15.164:0/2756736780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600860512f0 0x560086071690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:14.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.951+0000 7f14467fc640 1 -- 172.21.15.164:0/2756736780 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600860d5200 con 0x5600860509d0 2024-09-19T08:46:14.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.951+0000 7f14467fc640 1 --2- 172.21.15.164:0/2756736780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600860509d0 0x560086071150 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f143c004320 tx=0x7f143c00bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d3d6fca242c442 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:14.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.951+0000 7f142aff5640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:46:14.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.951+0000 7f142aff5640 1 -- 172.21.15.164:0/2756736780 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c02a0d0 con 0x5600860509d0 2024-09-19T08:46:14.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.951+0000 7f142aff5640 1 -- 172.21.15.164:0/2756736780 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f143c02a270 con 0x5600860509d0 2024-09-19T08:46:14.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.952+0000 7f144f81f740 1 -- 172.21.15.164:0/2756736780 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f143c016070 con 0x5600860509d0 2024-09-19T08:46:14.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.952+0000 7f142aff5640 1 -- 172.21.15.164:0/2756736780 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f143c02a470 con 0x5600860509d0 2024-09-19T08:46:14.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.952+0000 7f144f81f740 10 client.15495 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:14.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.952+0000 7f144f81f740 1 -- 172.21.15.164:0/2756736780 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5600860d6450 con 0x5600860509d0 2024-09-19T08:46:14.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.952+0000 7f142aff5640 1 -- 172.21.15.164:0/2756736780 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f143c016070 con 0x5600860509d0 2024-09-19T08:46:14.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.952+0000 7f142aff5640 1 -- 172.21.15.164:0/2756736780 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f143c03c030 con 0x5600860509d0 2024-09-19T08:46:14.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.952+0000 7f144f81f740 10 client.15495 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:14.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.952+0000 7f144f81f740 10 client.15495 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T08:46:14.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.952+0000 7f144f81f740 20 client.15495 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:14.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.952+0000 7f144f81f740 1 --2- 172.21.15.164:0/2756736780 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x5600860dac60 0x5600860d9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:14.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.952+0000 7f144f81f740 4 client.15495 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:46:14.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.952+0000 7f144f81f740 1 -- 172.21.15.164:0/2756736780 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "status"}) -- 0x560085fbeb10 con 0x5600860dac60 2024-09-19T08:46:14.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.952+0000 7f1446ffd640 1 --2- 172.21.15.164:0/2756736780 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x5600860dac60 0x5600860d9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:14.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.953+0000 7f1446ffd640 1 --2- 172.21.15.164:0/2756736780 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x5600860dac60 0x5600860d9970 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=237c18da429764b6 server_cookie=bf080c6056ad922 in_seq=0 out_seq=0 2024-09-19T08:46:14.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.953+0000 7f142aff5640 10 client.15495 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:46:14.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.953+0000 7f142aff5640 1 -- 172.21.15.164:0/2756736780 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560085fbeb10 con 0x5600860dac60 2024-09-19T08:46:14.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.953+0000 7f142aff5640 10 client.15495 handle_command_reply: tid=0 2024-09-19T08:46:14.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.953+0000 7f144f81f740 1 client.15495 shutdown 2024-09-19T08:46:14.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.953+0000 7f144f81f740 1 -- 172.21.15.164:0/2756736780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600860509d0 msgr2=0x560086071150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:14.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.953+0000 7f144f81f740 1 --2- 172.21.15.164:0/2756736780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600860509d0 0x560086071150 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f143c004320 tx=0x7f143c00bf70 comp rx=0 tx=0).stop 2024-09-19T08:46:14.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.953+0000 7f144f81f740 1 -- 172.21.15.164:0/2756736780 shutdown_connections 2024-09-19T08:46:14.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.953+0000 7f144f81f740 1 --2- 172.21.15.164:0/2756736780 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x5600860dac60 0x5600860d9970 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:14.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.953+0000 7f144f81f740 1 --2- 172.21.15.164:0/2756736780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56008605bbb0 0x5600860d4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:14.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.953+0000 7f144f81f740 1 --2- 172.21.15.164:0/2756736780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600860512f0 0x560086071690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:14.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.953+0000 7f144f81f740 1 --2- 172.21.15.164:0/2756736780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5600860509d0 0x560086071150 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:14.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.953+0000 7f144f81f740 1 -- 172.21.15.164:0/2756736780 >> 172.21.15.164:0/2756736780 conn(0x56008604cf60 msgr2=0x560086047ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:14.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.953+0000 7f142aff5640 0 client.15495 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:46:14.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.953+0000 7f144f81f740 1 -- 172.21.15.164:0/2756736780 shutdown_connections 2024-09-19T08:46:14.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.953+0000 7f144f81f740 1 -- 172.21.15.164:0/2756736780 wait complete. 2024-09-19T08:46:14.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.953+0000 7f144f81f740 20 client.15495 trim_cache size 0 max 16384 2024-09-19T08:46:14.962 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:46:14.962 INFO:teuthology.orchestra.run.smithi164.stdout: "cluster_fsid": "1e4b0d74-7662-11ef-bceb-c7b262605968", 2024-09-19T08:46:14.963 INFO:teuthology.orchestra.run.smithi164.stdout: "whoami": 2, 2024-09-19T08:46:14.963 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 24442, 2024-09-19T08:46:14.963 INFO:teuthology.orchestra.run.smithi164.stdout: "want_state": "up:active", 2024-09-19T08:46:14.963 INFO:teuthology.orchestra.run.smithi164.stdout: "state": "up:active", 2024-09-19T08:46:14.963 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs", 2024-09-19T08:46:14.963 INFO:teuthology.orchestra.run.smithi164.stdout: "rank_uptime": 133.13754990999999, 2024-09-19T08:46:14.963 INFO:teuthology.orchestra.run.smithi164.stdout: "mdsmap_epoch": 28, 2024-09-19T08:46:14.963 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch": 83, 2024-09-19T08:46:14.963 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch_barrier": 82, 2024-09-19T08:46:14.963 INFO:teuthology.orchestra.run.smithi164.stdout: "uptime": 161.565774686 2024-09-19T08:46:14.964 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:46:14.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.954+0000 7f144d446640 1 -- 172.21.15.164:0/1200190777 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f14300a34d0 msgr2=0x7f14300a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:14.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.954+0000 7f144d446640 1 --2- 172.21.15.164:0/1200190777 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f14300a34d0 0x7f14300a5990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f14400ed9b0 tx=0x7f1440009260 comp rx=0 tx=0).stop 2024-09-19T08:46:14.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.954+0000 7f144d446640 1 -- 172.21.15.164:0/1200190777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1448106a90 msgr2=0x7f14480fd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:14.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.954+0000 7f144d446640 1 --2- 172.21.15.164:0/1200190777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1448106a90 0x7f14480fd7f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1440005ab0 tx=0x7f14400042f0 comp rx=0 tx=0).stop 2024-09-19T08:46:14.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.954+0000 7f144d446640 1 -- 172.21.15.164:0/1200190777 shutdown_connections 2024-09-19T08:46:14.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.954+0000 7f144d446640 1 --2- 172.21.15.164:0/1200190777 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f14300a34d0 0x7f14300a5990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:14.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.954+0000 7f144d446640 1 --2- 172.21.15.164:0/1200190777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1448106a90 0x7f14480fd7f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:14.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.954+0000 7f144d446640 1 --2- 172.21.15.164:0/1200190777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1448101430 0x7f14480fd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:14.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.954+0000 7f144d446640 1 --2- 172.21.15.164:0/1200190777 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1448100a90 0x7f14480fcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:14.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.954+0000 7f144d446640 1 -- 172.21.15.164:0/1200190777 >> 172.21.15.164:0/1200190777 conn(0x7f14480fc7d0 msgr2=0x7f144806aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:14.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.954+0000 7f144d446640 1 -- 172.21.15.164:0/1200190777 shutdown_connections 2024-09-19T08:46:14.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:14.954+0000 7f144d446640 1 -- 172.21.15.164:0/1200190777 wait complete. 2024-09-19T08:46:15.692 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T08:46:15.940 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:46:15.941 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:15 smithi170 ceph-mon[35072]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-09-19T08:46:15.963 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:15 smithi173 ceph-mon[35306]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-09-19T08:46:16.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:15 smithi164 ceph-mon[30928]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-09-19T08:46:16.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.396+0000 7f068a575640 1 -- 172.21.15.164:0/2216641414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0684102ef0 msgr2=0x7f068410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:16.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.396+0000 7f068a575640 1 --2- 172.21.15.164:0/2216641414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0684102ef0 0x7f068410da40 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f067c009ca0 tx=0x7f067c02a950 comp rx=0 tx=0).stop 2024-09-19T08:46:16.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.396+0000 7f068a575640 1 -- 172.21.15.164:0/2216641414 shutdown_connections 2024-09-19T08:46:16.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.396+0000 7f068a575640 1 --2- 172.21.15.164:0/2216641414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0684102ef0 0x7f068410da40 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.396+0000 7f068a575640 1 --2- 172.21.15.164:0/2216641414 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0684102570 0x7f06841029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.396+0000 7f068a575640 1 --2- 172.21.15.164:0/2216641414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0684100e20 0x7f0684101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.396+0000 7f068a575640 1 -- 172.21.15.164:0/2216641414 >> 172.21.15.164:0/2216641414 conn(0x7f06840fc7d0 msgr2=0x7f06840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:16.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.396+0000 7f068a575640 1 -- 172.21.15.164:0/2216641414 shutdown_connections 2024-09-19T08:46:16.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.396+0000 7f068a575640 1 -- 172.21.15.164:0/2216641414 wait complete. 2024-09-19T08:46:16.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.396+0000 7f068a575640 1 Processor -- start 2024-09-19T08:46:16.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.396+0000 7f068a575640 1 -- start start 2024-09-19T08:46:16.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.397+0000 7f068a575640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0684100e20 0x7f0684197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:16.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.397+0000 7f068a575640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0684102570 0x7f0684197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:16.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.397+0000 7f068a575640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0684102ef0 0x7f068419bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:16.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.397+0000 7f068a575640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0684110f80 con 0x7f0684102ef0 2024-09-19T08:46:16.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.397+0000 7f068a575640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0684110e00 con 0x7f0684102570 2024-09-19T08:46:16.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.397+0000 7f068a575640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0684111100 con 0x7f0684100e20 2024-09-19T08:46:16.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.397+0000 7f0688b12640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0684102ef0 0x7f068419bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:16.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.397+0000 7f0688b12640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0684102ef0 0x7f068419bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42318/0 (socket says 172.21.15.164:42318) 2024-09-19T08:46:16.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.397+0000 7f0688b12640 1 -- 172.21.15.164:0/3248448309 learned_addr learned my addr 172.21.15.164:0/3248448309 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:16.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.397+0000 7f0683fff640 1 --2- 172.21.15.164:0/3248448309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0684100e20 0x7f0684197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:16.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.397+0000 7f06837fe640 1 --2- 172.21.15.164:0/3248448309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0684102570 0x7f0684197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:16.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.397+0000 7f0688b12640 1 -- 172.21.15.164:0/3248448309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0684100e20 msgr2=0x7f0684197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:16.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.397+0000 7f0688b12640 1 --2- 172.21.15.164:0/3248448309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0684100e20 0x7f0684197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.397+0000 7f0688b12640 1 -- 172.21.15.164:0/3248448309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0684102570 msgr2=0x7f0684197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:16.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.397+0000 7f0688b12640 1 --2- 172.21.15.164:0/3248448309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0684102570 0x7f0684197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.397+0000 7f0688b12640 1 -- 172.21.15.164:0/3248448309 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f068419c3f0 con 0x7f0684102ef0 2024-09-19T08:46:16.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.397+0000 7f0688b12640 1 --2- 172.21.15.164:0/3248448309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0684102ef0 0x7f068419bd10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f067c02ae70 tx=0x7f067c02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:16.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.398+0000 7f06817fa640 1 -- 172.21.15.164:0/3248448309 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c034040 con 0x7f0684102ef0 2024-09-19T08:46:16.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.398+0000 7f06817fa640 1 -- 172.21.15.164:0/3248448309 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f067c030920 con 0x7f0684102ef0 2024-09-19T08:46:16.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.398+0000 7f068a575640 1 -- 172.21.15.164:0/3248448309 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f068419c680 con 0x7f0684102ef0 2024-09-19T08:46:16.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.398+0000 7f06817fa640 1 -- 172.21.15.164:0/3248448309 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c045480 con 0x7f0684102ef0 2024-09-19T08:46:16.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.398+0000 7f068a575640 1 -- 172.21.15.164:0/3248448309 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06841a3fc0 con 0x7f0684102ef0 2024-09-19T08:46:16.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.400+0000 7f06817fa640 1 -- 172.21.15.164:0/3248448309 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7f067c037020 con 0x7f0684102ef0 2024-09-19T08:46:16.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.400+0000 7f068c94e740 1 Processor -- start 2024-09-19T08:46:16.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.400+0000 7f068c94e740 1 -- start start 2024-09-19T08:46:16.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.401+0000 7f068c94e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597d6ec0790 0x5597d6ec0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:16.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.401+0000 7f068c94e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597d6ebaf30 0x5597d6ebb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:16.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.401+0000 7f068c94e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597d6ebb8f0 0x5597d6ec75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:16.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.401+0000 7f068c94e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5597d6e1c090 con 0x5597d6ebb8f0 2024-09-19T08:46:16.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.401+0000 7f068c94e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5597d6dc3a70 con 0x5597d6ebaf30 2024-09-19T08:46:16.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.401+0000 7f068c94e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5597d6e18050 con 0x5597d6ec0790 2024-09-19T08:46:16.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.401+0000 7f06837fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597d6ebb8f0 0x5597d6ec75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:16.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.401+0000 7f06837fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597d6ec0790 0x5597d6ec0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:16.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.401+0000 7f06837fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597d6ec0790 0x5597d6ec0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:48702/0 (socket says 172.21.15.164:48702) 2024-09-19T08:46:16.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.401+0000 7f06837fe640 1 -- 172.21.15.164:0/3329945180 learned_addr learned my addr 172.21.15.164:0/3329945180 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:16.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.401+0000 7f0683fff640 1 --2- 172.21.15.164:0/3329945180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597d6ebaf30 0x5597d6ebb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:16.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.401+0000 7f06817fa640 1 --2- 172.21.15.164:0/3248448309 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f066c0a34d0 0x7f066c0a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:16.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.401+0000 7f06837fe640 1 -- 172.21.15.164:0/3329945180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597d6ebaf30 msgr2=0x5597d6ebb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:16.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.401+0000 7f06837fe640 1 --2- 172.21.15.164:0/3329945180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597d6ebaf30 0x5597d6ebb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.401+0000 7f06837fe640 1 -- 172.21.15.164:0/3329945180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597d6ebb8f0 msgr2=0x5597d6ec75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:16.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.401+0000 7f06837fe640 1 --2- 172.21.15.164:0/3329945180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597d6ebb8f0 0x5597d6ec75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.401+0000 7f06837fe640 1 -- 172.21.15.164:0/3329945180 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597d6dd0130 con 0x5597d6ec0790 2024-09-19T08:46:16.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.401+0000 7f0688b12640 1 --2- 172.21.15.164:0/3248448309 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f066c0a34d0 0x7f066c0a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:16.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.401+0000 7f06817fa640 1 -- 172.21.15.164:0/3248448309 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f067c0f0cb0 con 0x7f0684102ef0 2024-09-19T08:46:16.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.401+0000 7f06837fe640 1 --2- 172.21.15.164:0/3329945180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597d6ec0790 0x5597d6ec0b70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0678012730 tx=0x7f067800b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c55490b7508b099 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:16.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.402+0000 7f066a7fc640 1 -- 172.21.15.164:0/3329945180 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0678004050 con 0x5597d6ec0790 2024-09-19T08:46:16.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.402+0000 7f0688b12640 1 --2- 172.21.15.164:0/3248448309 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f066c0a34d0 0x7f066c0a5990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f067c0ee460 tx=0x7f067c0095c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:16.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.402+0000 7f066a7fc640 1 -- 172.21.15.164:0/3329945180 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0678005500 con 0x5597d6ec0790 2024-09-19T08:46:16.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.402+0000 7f066a7fc640 1 -- 172.21.15.164:0/3329945180 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0678004ce0 con 0x5597d6ec0790 2024-09-19T08:46:16.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.402+0000 7f068c94e740 1 -- 172.21.15.164:0/3329945180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597d6ec0790 msgr2=0x5597d6ec0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:16.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.402+0000 7f068c94e740 1 --2- 172.21.15.164:0/3329945180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597d6ec0790 0x5597d6ec0b70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0678012730 tx=0x7f067800b390 comp rx=0 tx=0).stop 2024-09-19T08:46:16.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.402+0000 7f068c94e740 1 -- 172.21.15.164:0/3329945180 shutdown_connections 2024-09-19T08:46:16.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.402+0000 7f068c94e740 1 --2- 172.21.15.164:0/3329945180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597d6ebb8f0 0x5597d6ec75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.402+0000 7f068c94e740 1 --2- 172.21.15.164:0/3329945180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597d6ebaf30 0x5597d6ebb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.402+0000 7f068c94e740 1 --2- 172.21.15.164:0/3329945180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597d6ec0790 0x5597d6ec0b70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.402+0000 7f068c94e740 1 -- 172.21.15.164:0/3329945180 >> 172.21.15.164:0/3329945180 conn(0x5597d6eb1a50 msgr2=0x5597d6eb1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:16.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.402+0000 7f068c94e740 1 -- 172.21.15.164:0/3329945180 shutdown_connections 2024-09-19T08:46:16.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.402+0000 7f068c94e740 1 -- 172.21.15.164:0/3329945180 wait complete. 2024-09-19T08:46:16.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.403+0000 7f068c94e740 1 -- start start 2024-09-19T08:46:16.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.403+0000 7f068c94e740 1 Processor -- start 2024-09-19T08:46:16.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.403+0000 7f068c94e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597d6ebaf30 0x5597d6ec73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:16.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.403+0000 7f068c94e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597d6ebb8f0 0x5597d6ec4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:16.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.403+0000 7f068c94e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597d6ec0790 0x5597d6ec4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:16.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.403+0000 7f068c94e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5597d6ec7c00 con 0x5597d6ebb8f0 2024-09-19T08:46:16.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.403+0000 7f068c94e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5597d6dd6f20 con 0x5597d6ebaf30 2024-09-19T08:46:16.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.403+0000 7f068c94e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5597d6ec7d80 con 0x5597d6ec0790 2024-09-19T08:46:16.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.403+0000 7f068c94e740 10 client.0 _ll_register_callbacks cb 0x5597d6da0030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:16.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.403+0000 7f0683fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597d6ebb8f0 0x5597d6ec4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:16.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.403+0000 7f0683fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597d6ebb8f0 0x5597d6ec4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42332/0 (socket says 172.21.15.164:42332) 2024-09-19T08:46:16.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.403+0000 7f0683fff640 1 -- 172.21.15.164:0/2248895138 learned_addr learned my addr 172.21.15.164:0/2248895138 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:16.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.403+0000 7f06837fe640 1 --2- 172.21.15.164:0/2248895138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597d6ebaf30 0x5597d6ec73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:16.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.403+0000 7f06837fe640 1 --2- 172.21.15.164:0/2248895138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597d6ec0790 0x5597d6ec4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:16.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.404+0000 7f0683fff640 1 -- 172.21.15.164:0/2248895138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597d6ec0790 msgr2=0x5597d6ec4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:16.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.404+0000 7f0683fff640 1 --2- 172.21.15.164:0/2248895138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597d6ec0790 0x5597d6ec4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.404+0000 7f0683fff640 1 -- 172.21.15.164:0/2248895138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597d6ebaf30 msgr2=0x5597d6ec73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:16.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.404+0000 7f0683fff640 1 --2- 172.21.15.164:0/2248895138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597d6ebaf30 0x5597d6ec73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.404+0000 7f0683fff640 1 -- 172.21.15.164:0/2248895138 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597d6b8d2b0 con 0x5597d6ebb8f0 2024-09-19T08:46:16.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.404+0000 7f0683fff640 1 --2- 172.21.15.164:0/2248895138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597d6ebb8f0 0x5597d6ec4700 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f067400ba20 tx=0x7f067400bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ad20fb03ed5c23 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:16.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.404+0000 7f06637fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:46:16.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.404+0000 7f06637fe640 1 -- 172.21.15.164:0/2248895138 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0674003e60 con 0x5597d6ebb8f0 2024-09-19T08:46:16.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.404+0000 7f06637fe640 1 -- 172.21.15.164:0/2248895138 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f067401b640 con 0x5597d6ebb8f0 2024-09-19T08:46:16.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.404+0000 7f068c94e740 1 -- 172.21.15.164:0/2248895138 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0678005500 con 0x5597d6ebb8f0 2024-09-19T08:46:16.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.404+0000 7f06637fe640 1 -- 172.21.15.164:0/2248895138 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f067401b840 con 0x5597d6ebb8f0 2024-09-19T08:46:16.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.404+0000 7f068c94e740 10 client.15507 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:16.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.404+0000 7f068c94e740 1 -- 172.21.15.164:0/2248895138 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5597d6ed35c0 con 0x5597d6ebb8f0 2024-09-19T08:46:16.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.404+0000 7f06637fe640 1 -- 172.21.15.164:0/2248895138 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0678005500 con 0x5597d6ebb8f0 2024-09-19T08:46:16.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.404+0000 7f06637fe640 1 -- 172.21.15.164:0/2248895138 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0674023020 con 0x5597d6ebb8f0 2024-09-19T08:46:16.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.404+0000 7f068c94e740 10 client.15507 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:16.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.404+0000 7f068c94e740 10 client.15507 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T08:46:16.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.405+0000 7f068c94e740 20 client.15507 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:16.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.405+0000 7f068c94e740 1 --2- 172.21.15.164:0/2248895138 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x5597d6ed5bf0 0x5597d6ed7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:16.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.405+0000 7f068c94e740 4 client.15507 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:46:16.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.405+0000 7f068c94e740 1 -- 172.21.15.164:0/2248895138 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597d6e29600 con 0x5597d6ed5bf0 2024-09-19T08:46:16.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.405+0000 7f06837fe640 1 --2- 172.21.15.164:0/2248895138 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x5597d6ed5bf0 0x5597d6ed7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:16.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.405+0000 7f06637fe640 10 client.15507 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:46:16.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.405+0000 7f06837fe640 1 --2- 172.21.15.164:0/2248895138 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x5597d6ed5bf0 0x5597d6ed7060 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aab6711c34a0c264 server_cookie=780d75a21717aa37 in_seq=0 out_seq=0 2024-09-19T08:46:16.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.406+0000 7f06637fe640 1 -- 172.21.15.164:0/2248895138 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5597d6e29600 con 0x5597d6ed5bf0 2024-09-19T08:46:16.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.406+0000 7f06637fe640 10 client.15507 handle_command_reply: tid=0 2024-09-19T08:46:16.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.406+0000 7f068c94e740 1 client.15507 shutdown 2024-09-19T08:46:16.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.406+0000 7f068c94e740 1 -- 172.21.15.164:0/2248895138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597d6ebb8f0 msgr2=0x5597d6ec4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:16.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.406+0000 7f068c94e740 1 --2- 172.21.15.164:0/2248895138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597d6ebb8f0 0x5597d6ec4700 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f067400ba20 tx=0x7f067400bee0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.406+0000 7f068c94e740 1 -- 172.21.15.164:0/2248895138 shutdown_connections 2024-09-19T08:46:16.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.406+0000 7f068c94e740 1 --2- 172.21.15.164:0/2248895138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597d6ec0790 0x5597d6ec4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.406+0000 7f068c94e740 1 --2- 172.21.15.164:0/2248895138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597d6ebb8f0 0x5597d6ec4700 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.406+0000 7f068c94e740 1 --2- 172.21.15.164:0/2248895138 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x5597d6ed5bf0 0x5597d6ed7060 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.406+0000 7f068c94e740 1 --2- 172.21.15.164:0/2248895138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597d6ebaf30 0x5597d6ec73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.406+0000 7f068c94e740 1 -- 172.21.15.164:0/2248895138 >> 172.21.15.164:0/2248895138 conn(0x5597d6eb1a50 msgr2=0x5597d6eb1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:16.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.406+0000 7f06637fe640 0 client.15507 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:46:16.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.406+0000 7f068c94e740 1 -- 172.21.15.164:0/2248895138 shutdown_connections 2024-09-19T08:46:16.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.406+0000 7f068c94e740 1 -- 172.21.15.164:0/2248895138 wait complete. 2024-09-19T08:46:16.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.406+0000 7f068c94e740 20 client.15507 trim_cache size 0 max 16384 2024-09-19T08:46:16.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.424+0000 7f068c94e740 1 Processor -- start 2024-09-19T08:46:16.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.424+0000 7f068c94e740 1 -- start start 2024-09-19T08:46:16.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.424+0000 7f068c94e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597d6ebaf30 0x5597d6ebfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:16.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.424+0000 7f068c94e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597d6ebb8f0 0x5597d6eb0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:16.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.424+0000 7f068c94e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597d6ec0790 0x5597d6eb0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:16.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.424+0000 7f068c94e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5597d6ed53a0 con 0x5597d6ebb8f0 2024-09-19T08:46:16.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.424+0000 7f068c94e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5597d6ed5a70 con 0x5597d6ec0790 2024-09-19T08:46:16.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.424+0000 7f068c94e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5597d6ed6140 con 0x5597d6ebaf30 2024-09-19T08:46:16.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.424+0000 7f0683fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597d6ebb8f0 0x5597d6eb0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:16.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.424+0000 7f0683fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597d6ebb8f0 0x5597d6eb0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42336/0 (socket says 172.21.15.164:42336) 2024-09-19T08:46:16.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.424+0000 7f0683fff640 1 -- 172.21.15.164:0/138550805 learned_addr learned my addr 172.21.15.164:0/138550805 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:16.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.425+0000 7f06837fe640 1 --2- 172.21.15.164:0/138550805 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597d6ebaf30 0x5597d6ebfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:16.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.425+0000 7f06837fe640 1 --2- 172.21.15.164:0/138550805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597d6ec0790 0x5597d6eb0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:16.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.425+0000 7f0683fff640 1 -- 172.21.15.164:0/138550805 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597d6ebaf30 msgr2=0x5597d6ebfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:16.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.425+0000 7f0683fff640 1 --2- 172.21.15.164:0/138550805 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597d6ebaf30 0x5597d6ebfe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.425+0000 7f0683fff640 1 -- 172.21.15.164:0/138550805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597d6ec0790 msgr2=0x5597d6eb0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:16.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.425+0000 7f0683fff640 1 --2- 172.21.15.164:0/138550805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597d6ec0790 0x5597d6eb0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.425+0000 7f0683fff640 1 -- 172.21.15.164:0/138550805 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597d6ec36d0 con 0x5597d6ebb8f0 2024-09-19T08:46:16.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.425+0000 7f06837fe640 1 --2- 172.21.15.164:0/138550805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597d6ec0790 0x5597d6eb0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:16.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.425+0000 7f0683fff640 1 --2- 172.21.15.164:0/138550805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597d6ebb8f0 0x5597d6eb0830 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0674000c00 tx=0x7f0674005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9e443876acaa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:16.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.425+0000 7f0669ffb640 1 -- 172.21.15.164:0/138550805 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0674011440 con 0x5597d6ebb8f0 2024-09-19T08:46:16.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.425+0000 7f0669ffb640 1 -- 172.21.15.164:0/138550805 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f067401b440 con 0x5597d6ebb8f0 2024-09-19T08:46:16.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.425+0000 7f0669ffb640 1 -- 172.21.15.164:0/138550805 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0674025460 con 0x5597d6ebb8f0 2024-09-19T08:46:16.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.426+0000 7f068c94e740 1 -- 172.21.15.164:0/138550805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597d6ebb8f0 msgr2=0x5597d6eb0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:16.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.426+0000 7f068c94e740 1 --2- 172.21.15.164:0/138550805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597d6ebb8f0 0x5597d6eb0830 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0674000c00 tx=0x7f0674005b90 comp rx=0 tx=0).stop 2024-09-19T08:46:16.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.426+0000 7f068c94e740 1 -- 172.21.15.164:0/138550805 shutdown_connections 2024-09-19T08:46:16.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.426+0000 7f068c94e740 1 --2- 172.21.15.164:0/138550805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597d6ec0790 0x5597d6eb0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.426+0000 7f068c94e740 1 --2- 172.21.15.164:0/138550805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597d6ebb8f0 0x5597d6eb0830 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.426+0000 7f068c94e740 1 --2- 172.21.15.164:0/138550805 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597d6ebaf30 0x5597d6ebfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.426+0000 7f068c94e740 1 -- 172.21.15.164:0/138550805 >> 172.21.15.164:0/138550805 conn(0x5597d6eb1a50 msgr2=0x5597d6ebf780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:16.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.426+0000 7f068c94e740 1 -- 172.21.15.164:0/138550805 shutdown_connections 2024-09-19T08:46:16.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.426+0000 7f068c94e740 1 -- 172.21.15.164:0/138550805 wait complete. 2024-09-19T08:46:16.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.426+0000 7f068c94e740 1 -- start start 2024-09-19T08:46:16.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.427+0000 7f068c94e740 1 Processor -- start 2024-09-19T08:46:16.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.427+0000 7f068c94e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597d6ebaf30 0x5597d6edf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:16.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.427+0000 7f068c94e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597d6ebb8f0 0x5597d6edf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:16.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.427+0000 7f068c94e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597d6ec0790 0x5597d6eda270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:16.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.427+0000 7f068c94e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5597d6ed6ee0 con 0x5597d6ebb8f0 2024-09-19T08:46:16.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.427+0000 7f068c94e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5597d6ed6810 con 0x5597d6ebaf30 2024-09-19T08:46:16.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.427+0000 7f068c94e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5597d6ed75b0 con 0x5597d6ec0790 2024-09-19T08:46:16.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.427+0000 7f068c94e740 10 client.0 _ll_register_callbacks cb 0x7f06740040e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:16.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.427+0000 7f0683fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597d6ebb8f0 0x5597d6edf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:16.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.427+0000 7f0683fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597d6ebb8f0 0x5597d6edf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42338/0 (socket says 172.21.15.164:42338) 2024-09-19T08:46:16.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.427+0000 7f0683fff640 1 -- 172.21.15.164:0/1925865279 learned_addr learned my addr 172.21.15.164:0/1925865279 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:16.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.427+0000 7f06837fe640 1 --2- 172.21.15.164:0/1925865279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597d6ebaf30 0x5597d6edf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:16.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.427+0000 7f06837fe640 1 --2- 172.21.15.164:0/1925865279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597d6ec0790 0x5597d6eda270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:16.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.427+0000 7f0683fff640 1 -- 172.21.15.164:0/1925865279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597d6ec0790 msgr2=0x5597d6eda270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:16.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.427+0000 7f0683fff640 1 --2- 172.21.15.164:0/1925865279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597d6ec0790 0x5597d6eda270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.427+0000 7f0683fff640 1 -- 172.21.15.164:0/1925865279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597d6ebaf30 msgr2=0x5597d6edf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:16.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.427+0000 7f0683fff640 1 --2- 172.21.15.164:0/1925865279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597d6ebaf30 0x5597d6edf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.427+0000 7f0683fff640 1 -- 172.21.15.164:0/1925865279 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597d6edff70 con 0x5597d6ebb8f0 2024-09-19T08:46:16.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.427+0000 7f0683fff640 1 --2- 172.21.15.164:0/1925865279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597d6ebb8f0 0x5597d6edf800 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f067400b9c0 tx=0x7f067401ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd578ef9097c94fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:16.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.427+0000 7f06637fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:46:16.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.428+0000 7f06637fe640 1 -- 172.21.15.164:0/1925865279 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06740050f0 con 0x5597d6ebb8f0 2024-09-19T08:46:16.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.428+0000 7f06637fe640 1 -- 172.21.15.164:0/1925865279 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f0674005510 con 0x5597d6ebb8f0 2024-09-19T08:46:16.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.428+0000 7f068c94e740 1 -- 172.21.15.164:0/1925865279 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f067401b440 con 0x5597d6ebb8f0 2024-09-19T08:46:16.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.428+0000 7f06637fe640 1 -- 172.21.15.164:0/1925865279 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0674025460 con 0x5597d6ebb8f0 2024-09-19T08:46:16.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.428+0000 7f068c94e740 10 client.15513 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:16.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.428+0000 7f068c94e740 1 -- 172.21.15.164:0/1925865279 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5597d6f40910 con 0x5597d6ebb8f0 2024-09-19T08:46:16.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.428+0000 7f06637fe640 1 -- 172.21.15.164:0/1925865279 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f067401b440 con 0x5597d6ebb8f0 2024-09-19T08:46:16.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.428+0000 7f06637fe640 1 -- 172.21.15.164:0/1925865279 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0674023040 con 0x5597d6ebb8f0 2024-09-19T08:46:16.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.428+0000 7f068c94e740 10 client.15513 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:16.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.428+0000 7f068c94e740 10 client.15513 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T08:46:16.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.428+0000 7f068c94e740 20 client.15513 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:16.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.428+0000 7f068c94e740 1 --2- 172.21.15.164:0/1925865279 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x5597d6f45100 0x5597d6f43e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:16.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.428+0000 7f068c94e740 4 client.15513 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:46:16.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.428+0000 7f068c94e740 1 -- 172.21.15.164:0/1925865279 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "status"}) -- 0x5597d6e23b10 con 0x5597d6f45100 2024-09-19T08:46:16.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.428+0000 7f06837fe640 1 --2- 172.21.15.164:0/1925865279 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x5597d6f45100 0x5597d6f43e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:16.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.429+0000 7f06637fe640 10 client.15513 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:46:16.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.429+0000 7f06837fe640 1 --2- 172.21.15.164:0/1925865279 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x5597d6f45100 0x5597d6f43e10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c1db64bf4dfc338c server_cookie=31c11c07332a655b in_seq=0 out_seq=0 2024-09-19T08:46:16.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.429+0000 7f06637fe640 1 -- 172.21.15.164:0/1925865279 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5597d6e23b10 con 0x5597d6f45100 2024-09-19T08:46:16.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.429+0000 7f06637fe640 10 client.15513 handle_command_reply: tid=0 2024-09-19T08:46:16.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.429+0000 7f068c94e740 1 client.15513 shutdown 2024-09-19T08:46:16.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.429+0000 7f068c94e740 1 -- 172.21.15.164:0/1925865279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597d6ebb8f0 msgr2=0x5597d6edf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:16.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.429+0000 7f068c94e740 1 --2- 172.21.15.164:0/1925865279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597d6ebb8f0 0x5597d6edf800 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f067400b9c0 tx=0x7f067401ce60 comp rx=0 tx=0).stop 2024-09-19T08:46:16.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.429+0000 7f068c94e740 1 -- 172.21.15.164:0/1925865279 shutdown_connections 2024-09-19T08:46:16.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.429+0000 7f068c94e740 1 --2- 172.21.15.164:0/1925865279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597d6ec0790 0x5597d6eda270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.429+0000 7f068c94e740 1 --2- 172.21.15.164:0/1925865279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597d6ebb8f0 0x5597d6edf800 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.429+0000 7f068c94e740 1 --2- 172.21.15.164:0/1925865279 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x5597d6f45100 0x5597d6f43e10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.429+0000 7f068c94e740 1 --2- 172.21.15.164:0/1925865279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597d6ebaf30 0x5597d6edf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.429+0000 7f06637fe640 0 client.15513 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:46:16.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.429+0000 7f068c94e740 1 -- 172.21.15.164:0/1925865279 >> 172.21.15.164:0/1925865279 conn(0x5597d6eb1a50 msgr2=0x5597d6eb0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:16.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.429+0000 7f068c94e740 1 -- 172.21.15.164:0/1925865279 shutdown_connections 2024-09-19T08:46:16.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.429+0000 7f068c94e740 1 -- 172.21.15.164:0/1925865279 wait complete. 2024-09-19T08:46:16.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.430+0000 7f068c94e740 20 client.15513 trim_cache size 0 max 16384 2024-09-19T08:46:16.436 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:46:16.436 INFO:teuthology.orchestra.run.smithi164.stdout: "cluster_fsid": "1e4b0d74-7662-11ef-bceb-c7b262605968", 2024-09-19T08:46:16.436 INFO:teuthology.orchestra.run.smithi164.stdout: "whoami": 1, 2024-09-19T08:46:16.436 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 24449, 2024-09-19T08:46:16.436 INFO:teuthology.orchestra.run.smithi164.stdout: "want_state": "up:active", 2024-09-19T08:46:16.436 INFO:teuthology.orchestra.run.smithi164.stdout: "state": "up:active", 2024-09-19T08:46:16.437 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs", 2024-09-19T08:46:16.437 INFO:teuthology.orchestra.run.smithi164.stdout: "rank_uptime": 135.63192552999999, 2024-09-19T08:46:16.437 INFO:teuthology.orchestra.run.smithi164.stdout: "mdsmap_epoch": 28, 2024-09-19T08:46:16.437 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch": 83, 2024-09-19T08:46:16.437 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch_barrier": 82, 2024-09-19T08:46:16.437 INFO:teuthology.orchestra.run.smithi164.stdout: "uptime": 165.60452535600001 2024-09-19T08:46:16.437 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:46:16.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.430+0000 7f068a575640 1 -- 172.21.15.164:0/3248448309 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f066c0a34d0 msgr2=0x7f066c0a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:16.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.430+0000 7f068a575640 1 --2- 172.21.15.164:0/3248448309 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f066c0a34d0 0x7f066c0a5990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f067c0ee460 tx=0x7f067c0095c0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.430+0000 7f068a575640 1 -- 172.21.15.164:0/3248448309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0684102ef0 msgr2=0x7f068419bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:16.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.430+0000 7f068a575640 1 --2- 172.21.15.164:0/3248448309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0684102ef0 0x7f068419bd10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f067c02ae70 tx=0x7f067c02f5d0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.430+0000 7f068a575640 1 -- 172.21.15.164:0/3248448309 shutdown_connections 2024-09-19T08:46:16.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.430+0000 7f068a575640 1 --2- 172.21.15.164:0/3248448309 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f066c0a34d0 0x7f066c0a5990 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.430+0000 7f068a575640 1 --2- 172.21.15.164:0/3248448309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0684102ef0 0x7f068419bd10 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.430+0000 7f068a575640 1 --2- 172.21.15.164:0/3248448309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0684102570 0x7f0684197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.430+0000 7f068a575640 1 --2- 172.21.15.164:0/3248448309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0684100e20 0x7f0684197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.430+0000 7f068a575640 1 -- 172.21.15.164:0/3248448309 >> 172.21.15.164:0/3248448309 conn(0x7f06840fc7d0 msgr2=0x7f06841068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:16.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.430+0000 7f068a575640 1 -- 172.21.15.164:0/3248448309 shutdown_connections 2024-09-19T08:46:16.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:16.430+0000 7f068a575640 1 -- 172.21.15.164:0/3248448309 wait complete. 2024-09-19T08:46:17.232 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T08:46:17.480 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:46:17.915 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:17 smithi164 ceph-mon[30928]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-19T08:46:18.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:17.996+0000 7fe0fd1c0640 1 -- 172.21.15.164:0/166874944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0f8100a80 msgr2=0x7fe0f8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:18.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:17.996+0000 7fe0fd1c0640 1 --2- 172.21.15.164:0/166874944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0f8100a80 0x7fe0f8100ee0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0009930 tx=0x7fe0e002a420 comp rx=0 tx=0).stop 2024-09-19T08:46:18.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:17.996+0000 7fe0fd1c0640 1 -- 172.21.15.164:0/166874944 shutdown_connections 2024-09-19T08:46:18.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:17.996+0000 7fe0fd1c0640 1 --2- 172.21.15.164:0/166874944 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0f8101420 0x7fe0f810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:17.996+0000 7fe0fd1c0640 1 --2- 172.21.15.164:0/166874944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0f8100a80 0x7fe0f8100ee0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:17.996+0000 7fe0fd1c0640 1 --2- 172.21.15.164:0/166874944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0f8106a80 0x7fe0f8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:17.996+0000 7fe0fd1c0640 1 -- 172.21.15.164:0/166874944 >> 172.21.15.164:0/166874944 conn(0x7fe0f80fc7a0 msgr2=0x7fe0f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:18.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:17.996+0000 7fe0fd1c0640 1 -- 172.21.15.164:0/166874944 shutdown_connections 2024-09-19T08:46:18.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:17.996+0000 7fe0fd1c0640 1 -- 172.21.15.164:0/166874944 wait complete. 2024-09-19T08:46:18.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:17.997+0000 7fe0fd1c0640 1 Processor -- start 2024-09-19T08:46:18.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:17.997+0000 7fe0fd1c0640 1 -- start start 2024-09-19T08:46:18.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:17.997+0000 7fe0fd1c0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0f8100a80 0x7fe0f81984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:18.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:17.997+0000 7fe0fd1c0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0f8101420 0x7fe0f81989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:18.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:17.997+0000 7fe0fd1c0640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0f8106a80 0x7fe0f819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:18.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:17.997+0000 7fe0fd1c0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f8110e80 con 0x7fe0f8101420 2024-09-19T08:46:18.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:17.997+0000 7fe0fd1c0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f8110d00 con 0x7fe0f8100a80 2024-09-19T08:46:18.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:17.997+0000 7fe0fd1c0640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f8111000 con 0x7fe0f8106a80 2024-09-19T08:46:18.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:17.997+0000 7fe0f659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0f8101420 0x7fe0f81989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:18.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:17.997+0000 7fe0f659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0f8101420 0x7fe0f81989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42356/0 (socket says 172.21.15.164:42356) 2024-09-19T08:46:18.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:17.997+0000 7fe0f659c640 1 -- 172.21.15.164:0/154173884 learned_addr learned my addr 172.21.15.164:0/154173884 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:18.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:17.997+0000 7fe0f759e640 1 --2- 172.21.15.164:0/154173884 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0f8106a80 0x7fe0f819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:18.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:17.997+0000 7fe0f6d9d640 1 --2- 172.21.15.164:0/154173884 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0f8100a80 0x7fe0f81984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:18.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:17.998+0000 7fe0f659c640 1 -- 172.21.15.164:0/154173884 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0f8106a80 msgr2=0x7fe0f819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:18.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:17.998+0000 7fe0f659c640 1 --2- 172.21.15.164:0/154173884 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0f8106a80 0x7fe0f819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:17.998+0000 7fe0f659c640 1 -- 172.21.15.164:0/154173884 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0f8100a80 msgr2=0x7fe0f81984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:18.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:17.998+0000 7fe0f659c640 1 --2- 172.21.15.164:0/154173884 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0f8100a80 0x7fe0f81984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:17.998+0000 7fe0f659c640 1 -- 172.21.15.164:0/154173884 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f819d460 con 0x7fe0f8101420 2024-09-19T08:46:18.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:17.998+0000 7fe0f659c640 1 --2- 172.21.15.164:0/154173884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0f8101420 0x7fe0f81989f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0009230 tx=0x7fe0e0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:18.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:17.998+0000 7fe0effff640 1 -- 172.21.15.164:0/154173884 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e003f030 con 0x7fe0f8101420 2024-09-19T08:46:18.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:17.998+0000 7fe0effff640 1 -- 172.21.15.164:0/154173884 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0e0003ad0 con 0x7fe0f8101420 2024-09-19T08:46:18.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:17.998+0000 7fe0fd1c0640 1 -- 172.21.15.164:0/154173884 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0f819d6f0 con 0x7fe0f8101420 2024-09-19T08:46:18.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:17.998+0000 7fe0effff640 1 -- 172.21.15.164:0/154173884 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0035070 con 0x7fe0f8101420 2024-09-19T08:46:18.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:17.998+0000 7fe0fd1c0640 1 -- 172.21.15.164:0/154173884 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0f81a5030 con 0x7fe0f8101420 2024-09-19T08:46:18.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.000+0000 7fe0effff640 1 -- 172.21.15.164:0/154173884 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7fe0e0042030 con 0x7fe0f8101420 2024-09-19T08:46:18.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.001+0000 7fe0ff599740 1 Processor -- start 2024-09-19T08:46:18.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.001+0000 7fe0ff599740 1 -- start start 2024-09-19T08:46:18.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.001+0000 7fe0ff599740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5617c29db9d0 0x5617c29dbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:18.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.001+0000 7fe0ff599740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5617c29dc2f0 0x5617c29e6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:18.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.001+0000 7fe0ff599740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617c29e6bb0 0x5617c29e8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:18.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.001+0000 7fe0ff599740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5617c2942090 con 0x5617c29e6bb0 2024-09-19T08:46:18.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.001+0000 7fe0ff599740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5617c28e9a70 con 0x5617c29dc2f0 2024-09-19T08:46:18.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.001+0000 7fe0ff599740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5617c293e050 con 0x5617c29db9d0 2024-09-19T08:46:18.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.001+0000 7fe0f659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617c29e6bb0 0x5617c29e8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:18.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.001+0000 7fe0f6d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5617c29dc2f0 0x5617c29e6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:18.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.001+0000 7fe0f6d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5617c29dc2f0 0x5617c29e6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:54074/0 (socket says 172.21.15.164:54074) 2024-09-19T08:46:18.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.001+0000 7fe0f6d9d640 1 -- 172.21.15.164:0/2665810132 learned_addr learned my addr 172.21.15.164:0/2665810132 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:18.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.001+0000 7fe0effff640 1 --2- 172.21.15.164:0/154173884 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe0e40a34d0 0x7fe0e40a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:18.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.001+0000 7fe0f759e640 1 --2- 172.21.15.164:0/2665810132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5617c29db9d0 0x5617c29dbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:18.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.002+0000 7fe0f6d9d640 1 -- 172.21.15.164:0/2665810132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5617c29db9d0 msgr2=0x5617c29dbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:18.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.002+0000 7fe0f759e640 1 --2- 172.21.15.164:0/154173884 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe0e40a34d0 0x7fe0e40a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:18.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.002+0000 7fe0f6d9d640 1 --2- 172.21.15.164:0/2665810132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5617c29db9d0 0x5617c29dbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.002+0000 7fe0f6d9d640 1 -- 172.21.15.164:0/2665810132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617c29e6bb0 msgr2=0x5617c29e8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:18.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.002+0000 7fe0f6d9d640 1 --2- 172.21.15.164:0/2665810132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617c29e6bb0 0x5617c29e8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.002+0000 7fe0f6d9d640 1 -- 172.21.15.164:0/2665810132 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617c28f6130 con 0x5617c29dc2f0 2024-09-19T08:46:18.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.002+0000 7fe0effff640 1 -- 172.21.15.164:0/154173884 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fe0e00f02d0 con 0x7fe0f8101420 2024-09-19T08:46:18.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.002+0000 7fe0f6d9d640 1 --2- 172.21.15.164:0/2665810132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5617c29dc2f0 0x5617c29e6670 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800aba0 tx=0x7fe0e800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8db70f0aa39f1571 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:18.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.002+0000 7fe0f759e640 1 --2- 172.21.15.164:0/154173884 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe0e40a34d0 0x7fe0e40a5990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0005fd0 tx=0x7fe0f0004ab0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:18.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.002+0000 7fe0ed7fa640 1 -- 172.21.15.164:0/2665810132 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e801d070 con 0x5617c29dc2f0 2024-09-19T08:46:18.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.002+0000 7fe0ed7fa640 1 -- 172.21.15.164:0/2665810132 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0e8018b20 con 0x5617c29dc2f0 2024-09-19T08:46:18.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.002+0000 7fe0ed7fa640 1 -- 172.21.15.164:0/2665810132 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8010af0 con 0x5617c29dc2f0 2024-09-19T08:46:18.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.003+0000 7fe0ff599740 1 -- 172.21.15.164:0/2665810132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5617c29dc2f0 msgr2=0x5617c29e6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:18.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.003+0000 7fe0ff599740 1 --2- 172.21.15.164:0/2665810132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5617c29dc2f0 0x5617c29e6670 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800aba0 tx=0x7fe0e800f790 comp rx=0 tx=0).stop 2024-09-19T08:46:18.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.003+0000 7fe0ff599740 1 -- 172.21.15.164:0/2665810132 shutdown_connections 2024-09-19T08:46:18.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.003+0000 7fe0ff599740 1 --2- 172.21.15.164:0/2665810132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617c29e6bb0 0x5617c29e8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.003+0000 7fe0ff599740 1 --2- 172.21.15.164:0/2665810132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5617c29dc2f0 0x5617c29e6670 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.003+0000 7fe0ff599740 1 --2- 172.21.15.164:0/2665810132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5617c29db9d0 0x5617c29dbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.003+0000 7fe0ff599740 1 -- 172.21.15.164:0/2665810132 >> 172.21.15.164:0/2665810132 conn(0x5617c29d7f60 msgr2=0x5617c29d8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:18.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.003+0000 7fe0ff599740 1 -- 172.21.15.164:0/2665810132 shutdown_connections 2024-09-19T08:46:18.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.003+0000 7fe0ff599740 1 -- 172.21.15.164:0/2665810132 wait complete. 2024-09-19T08:46:18.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.003+0000 7fe0ff599740 1 -- start start 2024-09-19T08:46:18.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.003+0000 7fe0ff599740 1 Processor -- start 2024-09-19T08:46:18.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.004+0000 7fe0ff599740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5617c29db9d0 0x5617c29e6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:18.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.004+0000 7fe0ff599740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5617c29dc2f0 0x5617c29e36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:18.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.004+0000 7fe0ff599740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617c29e6bb0 0x5617c29e3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:18.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.004+0000 7fe0ff599740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5617c29e9780 con 0x5617c29e6bb0 2024-09-19T08:46:18.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.004+0000 7fe0ff599740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5617c28fcf20 con 0x5617c29dc2f0 2024-09-19T08:46:18.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.004+0000 7fe0ff599740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5617c29e9900 con 0x5617c29db9d0 2024-09-19T08:46:18.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.004+0000 7fe0ff599740 10 client.0 _ll_register_callbacks cb 0x5617c28c6030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:18.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.004+0000 7fe0f759e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617c29e6bb0 0x5617c29e3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:18.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.004+0000 7fe0f759e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617c29e6bb0 0x5617c29e3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42374/0 (socket says 172.21.15.164:42374) 2024-09-19T08:46:18.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.004+0000 7fe0f759e640 1 -- 172.21.15.164:0/2457990111 learned_addr learned my addr 172.21.15.164:0/2457990111 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:18.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.004+0000 7fe0f659c640 1 --2- 172.21.15.164:0/2457990111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5617c29dc2f0 0x5617c29e36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:18.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.004+0000 7fe0f6d9d640 1 --2- 172.21.15.164:0/2457990111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5617c29db9d0 0x5617c29e6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:18.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.004+0000 7fe0f759e640 1 -- 172.21.15.164:0/2457990111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5617c29db9d0 msgr2=0x5617c29e6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:18.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.004+0000 7fe0f759e640 1 --2- 172.21.15.164:0/2457990111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5617c29db9d0 0x5617c29e6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.004+0000 7fe0f759e640 1 -- 172.21.15.164:0/2457990111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5617c29dc2f0 msgr2=0x5617c29e36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:18.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.004+0000 7fe0f759e640 1 --2- 172.21.15.164:0/2457990111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5617c29dc2f0 0x5617c29e36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.004+0000 7fe0f759e640 1 -- 172.21.15.164:0/2457990111 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617c26b32b0 con 0x5617c29e6bb0 2024-09-19T08:46:18.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.004+0000 7fe0f759e640 1 --2- 172.21.15.164:0/2457990111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617c29e6bb0 0x5617c29e3c10 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0010a90 tx=0x7fe0f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4141cd5d895a45c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:18.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.004+0000 7fe0de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:46:18.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.004+0000 7fe0de7fc640 1 -- 172.21.15.164:0/2457990111 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0018070 con 0x5617c29e6bb0 2024-09-19T08:46:18.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.004+0000 7fe0de7fc640 1 -- 172.21.15.164:0/2457990111 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fe0f0004b20 con 0x5617c29e6bb0 2024-09-19T08:46:18.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.004+0000 7fe0ff599740 1 -- 172.21.15.164:0/2457990111 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0e8018b20 con 0x5617c29e6bb0 2024-09-19T08:46:18.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.005+0000 7fe0de7fc640 1 -- 172.21.15.164:0/2457990111 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe0f0004d20 con 0x5617c29e6bb0 2024-09-19T08:46:18.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.005+0000 7fe0ff599740 10 client.15525 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:18.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.005+0000 7fe0ff599740 1 -- 172.21.15.164:0/2457990111 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5617c29f5060 con 0x5617c29e6bb0 2024-09-19T08:46:18.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.005+0000 7fe0de7fc640 1 -- 172.21.15.164:0/2457990111 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0e8018b20 con 0x5617c29e6bb0 2024-09-19T08:46:18.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.005+0000 7fe0de7fc640 1 -- 172.21.15.164:0/2457990111 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe0f0030020 con 0x5617c29e6bb0 2024-09-19T08:46:18.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.005+0000 7fe0ff599740 10 client.15525 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:18.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.005+0000 7fe0ff599740 10 client.15525 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T08:46:18.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.005+0000 7fe0ff599740 20 client.15525 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:18.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.005+0000 7fe0ff599740 1 --2- 172.21.15.164:0/2457990111 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5617c29f76a0 0x5617c29f8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:18.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.005+0000 7fe0ff599740 4 client.15525 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:46:18.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.005+0000 7fe0ff599740 1 -- 172.21.15.164:0/2457990111 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617c294f600 con 0x5617c29f76a0 2024-09-19T08:46:18.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.005+0000 7fe0f6d9d640 1 --2- 172.21.15.164:0/2457990111 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5617c29f76a0 0x5617c29f8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:18.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.006+0000 7fe0de7fc640 10 client.15525 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:46:18.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.006+0000 7fe0f6d9d640 1 --2- 172.21.15.164:0/2457990111 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5617c29f76a0 0x5617c29f8b10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc5fd1389bc5c5f5 server_cookie=4df8cb0cd9d432b in_seq=0 out_seq=0 2024-09-19T08:46:18.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.006+0000 7fe0de7fc640 1 -- 172.21.15.164:0/2457990111 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5617c294f600 con 0x5617c29f76a0 2024-09-19T08:46:18.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.006+0000 7fe0de7fc640 10 client.15525 handle_command_reply: tid=0 2024-09-19T08:46:18.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.006+0000 7fe0ff599740 1 client.15525 shutdown 2024-09-19T08:46:18.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.006+0000 7fe0ff599740 1 -- 172.21.15.164:0/2457990111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617c29e6bb0 msgr2=0x5617c29e3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:18.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.006+0000 7fe0ff599740 1 --2- 172.21.15.164:0/2457990111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617c29e6bb0 0x5617c29e3c10 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0010a90 tx=0x7fe0f0004a40 comp rx=0 tx=0).stop 2024-09-19T08:46:18.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.006+0000 7fe0ff599740 1 -- 172.21.15.164:0/2457990111 shutdown_connections 2024-09-19T08:46:18.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.007+0000 7fe0ff599740 1 --2- 172.21.15.164:0/2457990111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617c29e6bb0 0x5617c29e3c10 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.007+0000 7fe0ff599740 1 --2- 172.21.15.164:0/2457990111 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5617c29f76a0 0x5617c29f8b10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.007+0000 7fe0de7fc640 0 client.15525 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:46:18.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.007+0000 7fe0ff599740 1 --2- 172.21.15.164:0/2457990111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5617c29dc2f0 0x5617c29e36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.007+0000 7fe0ff599740 1 --2- 172.21.15.164:0/2457990111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5617c29db9d0 0x5617c29e6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.007+0000 7fe0ff599740 1 -- 172.21.15.164:0/2457990111 >> 172.21.15.164:0/2457990111 conn(0x5617c29d7f60 msgr2=0x5617c29ddc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:18.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.007+0000 7fe0ff599740 1 -- 172.21.15.164:0/2457990111 shutdown_connections 2024-09-19T08:46:18.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.007+0000 7fe0ff599740 1 -- 172.21.15.164:0/2457990111 wait complete. 2024-09-19T08:46:18.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.007+0000 7fe0ff599740 20 client.15525 trim_cache size 0 max 16384 2024-09-19T08:46:18.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.024+0000 7fe0ff599740 1 Processor -- start 2024-09-19T08:46:18.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.024+0000 7fe0ff599740 1 -- start start 2024-09-19T08:46:18.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.024+0000 7fe0ff599740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617c29db9d0 0x5617c29f8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:18.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.024+0000 7fe0ff599740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5617c29dc2f0 0x5617c29e3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:18.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.024+0000 7fe0ff599740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5617c29e6bb0 0x5617c29e3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:18.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.024+0000 7fe0ff599740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5617c29f6e50 con 0x5617c29db9d0 2024-09-19T08:46:18.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.024+0000 7fe0ff599740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5617c29f7520 con 0x5617c29dc2f0 2024-09-19T08:46:18.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.024+0000 7fe0ff599740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5617c29f7bf0 con 0x5617c29e6bb0 2024-09-19T08:46:18.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.025+0000 7fe0f6d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617c29db9d0 0x5617c29f8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:18.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.025+0000 7fe0f6d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617c29db9d0 0x5617c29f8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42386/0 (socket says 172.21.15.164:42386) 2024-09-19T08:46:18.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.025+0000 7fe0f6d9d640 1 -- 172.21.15.164:0/2616294941 learned_addr learned my addr 172.21.15.164:0/2616294941 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:18.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.025+0000 7fe0f759e640 1 --2- 172.21.15.164:0/2616294941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5617c29e6bb0 0x5617c29e3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:18.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.025+0000 7fe0f659c640 1 --2- 172.21.15.164:0/2616294941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5617c29dc2f0 0x5617c29e3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:18.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.025+0000 7fe0f6d9d640 1 -- 172.21.15.164:0/2616294941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5617c29e6bb0 msgr2=0x5617c29e3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:18.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.025+0000 7fe0f6d9d640 1 --2- 172.21.15.164:0/2616294941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5617c29e6bb0 0x5617c29e3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.025+0000 7fe0f6d9d640 1 -- 172.21.15.164:0/2616294941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5617c29dc2f0 msgr2=0x5617c29e3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:18.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.025+0000 7fe0f6d9d640 1 --2- 172.21.15.164:0/2616294941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5617c29dc2f0 0x5617c29e3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.025+0000 7fe0f6d9d640 1 -- 172.21.15.164:0/2616294941 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617c29d7d90 con 0x5617c29db9d0 2024-09-19T08:46:18.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.025+0000 7fe0f6d9d640 1 --2- 172.21.15.164:0/2616294941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617c29db9d0 0x5617c29f8b10 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8000c00 tx=0x7fe0e800aef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c52e5875b6ad0d36 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:18.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.025+0000 7fe0ecff9640 1 -- 172.21.15.164:0/2616294941 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8032070 con 0x5617c29db9d0 2024-09-19T08:46:18.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.025+0000 7fe0ecff9640 1 -- 172.21.15.164:0/2616294941 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0e801e040 con 0x5617c29db9d0 2024-09-19T08:46:18.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.025+0000 7fe0ecff9640 1 -- 172.21.15.164:0/2616294941 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e801d040 con 0x5617c29db9d0 2024-09-19T08:46:18.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.026+0000 7fe0ff599740 1 -- 172.21.15.164:0/2616294941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617c29db9d0 msgr2=0x5617c29f8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:18.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.026+0000 7fe0ff599740 1 --2- 172.21.15.164:0/2616294941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617c29db9d0 0x5617c29f8b10 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8000c00 tx=0x7fe0e800aef0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.026+0000 7fe0ff599740 1 -- 172.21.15.164:0/2616294941 shutdown_connections 2024-09-19T08:46:18.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.026+0000 7fe0ff599740 1 --2- 172.21.15.164:0/2616294941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5617c29e6bb0 0x5617c29e3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.026+0000 7fe0ff599740 1 --2- 172.21.15.164:0/2616294941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5617c29dc2f0 0x5617c29e3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.026+0000 7fe0ff599740 1 --2- 172.21.15.164:0/2616294941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617c29db9d0 0x5617c29f8b10 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.026+0000 7fe0ff599740 1 -- 172.21.15.164:0/2616294941 >> 172.21.15.164:0/2616294941 conn(0x5617c29d7f60 msgr2=0x5617c29e2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:18.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.026+0000 7fe0ff599740 1 -- 172.21.15.164:0/2616294941 shutdown_connections 2024-09-19T08:46:18.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.026+0000 7fe0ff599740 1 -- 172.21.15.164:0/2616294941 wait complete. 2024-09-19T08:46:18.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.026+0000 7fe0ff599740 1 -- start start 2024-09-19T08:46:18.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.026+0000 7fe0ff599740 1 Processor -- start 2024-09-19T08:46:18.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.027+0000 7fe0ff599740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617c29db9d0 0x5617c29fc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:18.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.027+0000 7fe0ff599740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5617c29dc2f0 0x5617c29fc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:18.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.027+0000 7fe0ff599740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5617c29e6bb0 0x5617c2a5fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:18.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.027+0000 7fe0ff599740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5617c29f8990 con 0x5617c29db9d0 2024-09-19T08:46:18.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.027+0000 7fe0ff599740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5617c29f82c0 con 0x5617c29e6bb0 2024-09-19T08:46:18.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.027+0000 7fe0ff599740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5617c29f9060 con 0x5617c29dc2f0 2024-09-19T08:46:18.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.027+0000 7fe0ff599740 10 client.0 _ll_register_callbacks cb 0x7fe0f0010740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:18.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.027+0000 7fe0f6d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617c29db9d0 0x5617c29fc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:18.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.027+0000 7fe0f6d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617c29db9d0 0x5617c29fc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42394/0 (socket says 172.21.15.164:42394) 2024-09-19T08:46:18.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.027+0000 7fe0f6d9d640 1 -- 172.21.15.164:0/3258721867 learned_addr learned my addr 172.21.15.164:0/3258721867 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:18.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.027+0000 7fe0f659c640 1 --2- 172.21.15.164:0/3258721867 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5617c29dc2f0 0x5617c29fc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:18.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.027+0000 7fe0f759e640 1 --2- 172.21.15.164:0/3258721867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5617c29e6bb0 0x5617c2a5fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:18.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.027+0000 7fe0f6d9d640 1 -- 172.21.15.164:0/3258721867 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5617c29dc2f0 msgr2=0x5617c29fc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:18.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.027+0000 7fe0f6d9d640 1 --2- 172.21.15.164:0/3258721867 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5617c29dc2f0 0x5617c29fc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.027+0000 7fe0f6d9d640 1 -- 172.21.15.164:0/3258721867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5617c29e6bb0 msgr2=0x5617c2a5fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:18.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.027+0000 7fe0f6d9d640 1 --2- 172.21.15.164:0/3258721867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5617c29e6bb0 0x5617c2a5fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.027+0000 7fe0f6d9d640 1 -- 172.21.15.164:0/3258721867 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617c2a602e0 con 0x5617c29db9d0 2024-09-19T08:46:18.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.027+0000 7fe0f759e640 1 --2- 172.21.15.164:0/3258721867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5617c29e6bb0 0x5617c2a5fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:18.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.027+0000 7fe0f659c640 1 --2- 172.21.15.164:0/3258721867 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5617c29dc2f0 0x5617c29fc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:18.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.027+0000 7fe0f6d9d640 1 --2- 172.21.15.164:0/3258721867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617c29db9d0 0x5617c29fc1a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800fc90 tx=0x7fe0e800fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a799233d4f4f1fbc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:18.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.027+0000 7fe0de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:46:18.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.027+0000 7fe0de7fc640 1 -- 172.21.15.164:0/3258721867 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e803a060 con 0x5617c29db9d0 2024-09-19T08:46:18.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.027+0000 7fe0ff599740 1 -- 172.21.15.164:0/3258721867 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0e801e040 con 0x5617c29db9d0 2024-09-19T08:46:18.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.027+0000 7fe0de7fc640 1 -- 172.21.15.164:0/3258721867 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fe0e8005030 con 0x5617c29db9d0 2024-09-19T08:46:18.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.028+0000 7fe0de7fc640 1 -- 172.21.15.164:0/3258721867 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe0e8032070 con 0x5617c29db9d0 2024-09-19T08:46:18.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.028+0000 7fe0ff599740 10 client.15531 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:18.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.028+0000 7fe0ff599740 1 -- 172.21.15.164:0/3258721867 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5617c2a614c0 con 0x5617c29db9d0 2024-09-19T08:46:18.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.028+0000 7fe0de7fc640 1 -- 172.21.15.164:0/3258721867 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0e801e040 con 0x5617c29db9d0 2024-09-19T08:46:18.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.028+0000 7fe0de7fc640 1 -- 172.21.15.164:0/3258721867 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe0e803d020 con 0x5617c29db9d0 2024-09-19T08:46:18.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.028+0000 7fe0ff599740 10 client.15531 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:18.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.028+0000 7fe0ff599740 10 client.15531 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T08:46:18.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.028+0000 7fe0ff599740 20 client.15531 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:18.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.028+0000 7fe0ff599740 1 --2- 172.21.15.164:0/3258721867 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5617c29f76a0 0x5617c2a64630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:18.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.028+0000 7fe0ff599740 4 client.15531 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:46:18.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.028+0000 7fe0ff599740 1 -- 172.21.15.164:0/3258721867 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "status"}) -- 0x5617c2949b10 con 0x5617c29f76a0 2024-09-19T08:46:18.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.028+0000 7fe0f659c640 1 --2- 172.21.15.164:0/3258721867 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5617c29f76a0 0x5617c2a64630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:18.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.029+0000 7fe0de7fc640 10 client.15531 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:46:18.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.029+0000 7fe0f659c640 1 --2- 172.21.15.164:0/3258721867 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5617c29f76a0 0x5617c2a64630 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7838a413bc8f966 server_cookie=ca3ad5b71061cbd3 in_seq=0 out_seq=0 2024-09-19T08:46:18.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.029+0000 7fe0de7fc640 1 -- 172.21.15.164:0/3258721867 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5617c2949b10 con 0x5617c29f76a0 2024-09-19T08:46:18.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.029+0000 7fe0de7fc640 10 client.15531 handle_command_reply: tid=0 2024-09-19T08:46:18.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.029+0000 7fe0ff599740 1 client.15531 shutdown 2024-09-19T08:46:18.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.029+0000 7fe0ff599740 1 -- 172.21.15.164:0/3258721867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617c29db9d0 msgr2=0x5617c29fc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:18.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.029+0000 7fe0ff599740 1 --2- 172.21.15.164:0/3258721867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617c29db9d0 0x5617c29fc1a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800fc90 tx=0x7fe0e800fd40 comp rx=0 tx=0).stop 2024-09-19T08:46:18.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.029+0000 7fe0ff599740 1 -- 172.21.15.164:0/3258721867 shutdown_connections 2024-09-19T08:46:18.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.029+0000 7fe0ff599740 1 --2- 172.21.15.164:0/3258721867 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5617c29f76a0 0x5617c2a64630 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.029+0000 7fe0ff599740 1 --2- 172.21.15.164:0/3258721867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5617c29e6bb0 0x5617c2a5fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.029+0000 7fe0ff599740 1 --2- 172.21.15.164:0/3258721867 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5617c29dc2f0 0x5617c29fc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.029+0000 7fe0ff599740 1 --2- 172.21.15.164:0/3258721867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617c29db9d0 0x5617c29fc1a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.029+0000 7fe0ff599740 1 -- 172.21.15.164:0/3258721867 >> 172.21.15.164:0/3258721867 conn(0x5617c29d7f60 msgr2=0x5617c29e3690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:18.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.029+0000 7fe0de7fc640 0 client.15531 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:46:18.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.029+0000 7fe0ff599740 1 -- 172.21.15.164:0/3258721867 shutdown_connections 2024-09-19T08:46:18.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.029+0000 7fe0ff599740 1 -- 172.21.15.164:0/3258721867 wait complete. 2024-09-19T08:46:18.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.029+0000 7fe0ff599740 20 client.15531 trim_cache size 0 max 16384 2024-09-19T08:46:18.046 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:46:18.046 INFO:teuthology.orchestra.run.smithi164.stdout: "cluster_fsid": "1e4b0d74-7662-11ef-bceb-c7b262605968", 2024-09-19T08:46:18.046 INFO:teuthology.orchestra.run.smithi164.stdout: "whoami": 0, 2024-09-19T08:46:18.046 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 24479, 2024-09-19T08:46:18.046 INFO:teuthology.orchestra.run.smithi164.stdout: "want_state": "up:active", 2024-09-19T08:46:18.047 INFO:teuthology.orchestra.run.smithi164.stdout: "state": "up:active", 2024-09-19T08:46:18.047 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs", 2024-09-19T08:46:18.047 INFO:teuthology.orchestra.run.smithi164.stdout: "rank_uptime": 144.26692326200001, 2024-09-19T08:46:18.047 INFO:teuthology.orchestra.run.smithi164.stdout: "mdsmap_epoch": 28, 2024-09-19T08:46:18.047 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch": 83, 2024-09-19T08:46:18.047 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch_barrier": 82, 2024-09-19T08:46:18.047 INFO:teuthology.orchestra.run.smithi164.stdout: "uptime": 155.981177876 2024-09-19T08:46:18.047 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:46:18.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.030+0000 7fe0fd1c0640 1 -- 172.21.15.164:0/154173884 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe0e40a34d0 msgr2=0x7fe0e40a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:18.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.030+0000 7fe0fd1c0640 1 --2- 172.21.15.164:0/154173884 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe0e40a34d0 0x7fe0e40a5990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0005fd0 tx=0x7fe0f0004ab0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.030+0000 7fe0fd1c0640 1 -- 172.21.15.164:0/154173884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0f8101420 msgr2=0x7fe0f81989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:18.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.030+0000 7fe0fd1c0640 1 --2- 172.21.15.164:0/154173884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0f8101420 0x7fe0f81989f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0009230 tx=0x7fe0e0004270 comp rx=0 tx=0).stop 2024-09-19T08:46:18.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.030+0000 7fe0fd1c0640 1 -- 172.21.15.164:0/154173884 shutdown_connections 2024-09-19T08:46:18.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.030+0000 7fe0fd1c0640 1 --2- 172.21.15.164:0/154173884 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0f8106a80 0x7fe0f819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.030+0000 7fe0fd1c0640 1 --2- 172.21.15.164:0/154173884 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe0e40a34d0 0x7fe0e40a5990 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.030+0000 7fe0fd1c0640 1 --2- 172.21.15.164:0/154173884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0f8101420 0x7fe0f81989f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.030+0000 7fe0fd1c0640 1 --2- 172.21.15.164:0/154173884 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0f8100a80 0x7fe0f81984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.030+0000 7fe0fd1c0640 1 -- 172.21.15.164:0/154173884 >> 172.21.15.164:0/154173884 conn(0x7fe0f80fc7a0 msgr2=0x7fe0f806aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:18.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.030+0000 7fe0fd1c0640 1 -- 172.21.15.164:0/154173884 shutdown_connections 2024-09-19T08:46:18.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:18.030+0000 7fe0fd1c0640 1 -- 172.21.15.164:0/154173884 wait complete. 2024-09-19T08:46:18.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:17 smithi170 ceph-mon[35072]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-19T08:46:18.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:17 smithi173 ceph-mon[35306]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-19T08:46:18.731 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-19T08:46:20.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:19 smithi164 ceph-mon[30928]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-19T08:46:20.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:19 smithi170 ceph-mon[35072]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-19T08:46:20.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:19 smithi173 ceph-mon[35306]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-19T08:46:22.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:21 smithi164 ceph-mon[30928]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-19T08:46:22.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:21 smithi170 ceph-mon[35072]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-19T08:46:22.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:21 smithi173 ceph-mon[35306]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-19T08:46:24.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:23 smithi164 ceph-mon[30928]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1023 B/s wr, 9 op/s 2024-09-19T08:46:24.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:23 smithi170 ceph-mon[35072]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1023 B/s wr, 9 op/s 2024-09-19T08:46:24.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:23 smithi173 ceph-mon[35306]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1023 B/s wr, 9 op/s 2024-09-19T08:46:26.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:25 smithi164 ceph-mon[30928]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-19T08:46:26.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:25 smithi170 ceph-mon[35072]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-19T08:46:26.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:25 smithi173 ceph-mon[35306]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-19T08:46:28.035 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:27 smithi164 ceph-mon[30928]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T08:46:28.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:27 smithi170 ceph-mon[35072]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T08:46:28.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:27 smithi173 ceph-mon[35306]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T08:46:28.247 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:46:28.502 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:46:28.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.949+0000 7f723351a640 1 -- 172.21.15.164:0/2602636184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f722c102570 msgr2=0x7f722c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:28.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.949+0000 7f723351a640 1 --2- 172.21.15.164:0/2602636184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f722c102570 0x7f722c1029b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f721c009900 tx=0x7f721c02a480 comp rx=0 tx=0).stop 2024-09-19T08:46:28.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.949+0000 7f723351a640 1 -- 172.21.15.164:0/2602636184 shutdown_connections 2024-09-19T08:46:28.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.949+0000 7f723351a640 1 --2- 172.21.15.164:0/2602636184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f722c102ef0 0x7f722c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.949+0000 7f723351a640 1 --2- 172.21.15.164:0/2602636184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f722c102570 0x7f722c1029b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.949+0000 7f723351a640 1 --2- 172.21.15.164:0/2602636184 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f722c100e20 0x7f722c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.949+0000 7f723351a640 1 -- 172.21.15.164:0/2602636184 >> 172.21.15.164:0/2602636184 conn(0x7f722c0fc7d0 msgr2=0x7f722c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:28.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.949+0000 7f723351a640 1 -- 172.21.15.164:0/2602636184 shutdown_connections 2024-09-19T08:46:28.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.949+0000 7f723351a640 1 -- 172.21.15.164:0/2602636184 wait complete. 2024-09-19T08:46:28.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.950+0000 7f723351a640 1 Processor -- start 2024-09-19T08:46:28.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.950+0000 7f723351a640 1 -- start start 2024-09-19T08:46:28.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.951+0000 7f723351a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f722c100e20 0x7f722c1a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:28.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.951+0000 7f723351a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f722c102570 0x7f722c1a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:28.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.951+0000 7f723351a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f722c102ef0 0x7f722c19ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:28.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.951+0000 7f723351a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f722c110f80 con 0x7f722c102ef0 2024-09-19T08:46:28.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.951+0000 7f723351a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f722c110e00 con 0x7f722c100e20 2024-09-19T08:46:28.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.951+0000 7f723351a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f722c111100 con 0x7f722c102570 2024-09-19T08:46:28.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.951+0000 7f7231ab7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f722c102ef0 0x7f722c19ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:28.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.951+0000 7f7231ab7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f722c102ef0 0x7f722c19ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35902/0 (socket says 172.21.15.164:35902) 2024-09-19T08:46:28.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.951+0000 7f7231ab7640 1 -- 172.21.15.164:0/2889118195 learned_addr learned my addr 172.21.15.164:0/2889118195 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:28.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.951+0000 7f72312b6640 1 --2- 172.21.15.164:0/2889118195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f722c100e20 0x7f722c1a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:28.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.951+0000 7f7230ab5640 1 --2- 172.21.15.164:0/2889118195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f722c102570 0x7f722c1a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:28.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.951+0000 7f7231ab7640 1 -- 172.21.15.164:0/2889118195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f722c102570 msgr2=0x7f722c1a4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:28.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.951+0000 7f7231ab7640 1 --2- 172.21.15.164:0/2889118195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f722c102570 0x7f722c1a4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.951+0000 7f7231ab7640 1 -- 172.21.15.164:0/2889118195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f722c100e20 msgr2=0x7f722c1a47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:28.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.951+0000 7f7231ab7640 1 --2- 172.21.15.164:0/2889118195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f722c100e20 0x7f722c1a47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.951+0000 7f7231ab7640 1 -- 172.21.15.164:0/2889118195 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f722c19f3f0 con 0x7f722c102ef0 2024-09-19T08:46:28.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.951+0000 7f7231ab7640 1 --2- 172.21.15.164:0/2889118195 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f722c102ef0 0x7f722c19ec20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7224009fd0 tx=0x7f722400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:28.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.952+0000 7f72227fc640 1 -- 172.21.15.164:0/2889118195 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7224017070 con 0x7f722c102ef0 2024-09-19T08:46:28.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.952+0000 7f72227fc640 1 -- 172.21.15.164:0/2889118195 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7224003e50 con 0x7f722c102ef0 2024-09-19T08:46:28.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.952+0000 7f723351a640 1 -- 172.21.15.164:0/2889118195 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f722c19f6e0 con 0x7f722c102ef0 2024-09-19T08:46:28.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.952+0000 7f72227fc640 1 -- 172.21.15.164:0/2889118195 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72240106c0 con 0x7f722c102ef0 2024-09-19T08:46:28.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.952+0000 7f723351a640 1 -- 172.21.15.164:0/2889118195 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f722c1ab610 con 0x7f722c102ef0 2024-09-19T08:46:28.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.954+0000 7f72227fc640 1 -- 172.21.15.164:0/2889118195 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7f7224003970 con 0x7f722c102ef0 2024-09-19T08:46:28.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.954+0000 7f72227fc640 1 --2- 172.21.15.164:0/2889118195 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f72180a34d0 0x7f72180a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:28.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.955+0000 7f72312b6640 1 --2- 172.21.15.164:0/2889118195 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f72180a34d0 0x7f72180a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:28.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.955+0000 7f72227fc640 1 -- 172.21.15.164:0/2889118195 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f72240cfcd0 con 0x7f722c102ef0 2024-09-19T08:46:28.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.955+0000 7f72312b6640 1 --2- 172.21.15.164:0/2889118195 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f72180a34d0 0x7f72180a5990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7228009bd0 tx=0x7f7228005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:28.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.955+0000 7f72358f3740 1 Processor -- start 2024-09-19T08:46:28.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.955+0000 7f72358f3740 1 -- start start 2024-09-19T08:46:28.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.956+0000 7f72358f3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563af8c6d4a0 0x563af8c6d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:28.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.956+0000 7f72358f3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563af8c6ddc0 0x563af8c78140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:28.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.956+0000 7f72358f3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563af8c78680 0x563af8c7aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:28.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.956+0000 7f72358f3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563af8bd32e0 con 0x563af8c6d4a0 2024-09-19T08:46:28.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.956+0000 7f72358f3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563af8b7ace0 con 0x563af8c6ddc0 2024-09-19T08:46:28.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.956+0000 7f72358f3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563af8bcf2a0 con 0x563af8c78680 2024-09-19T08:46:28.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.956+0000 7f72312b6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563af8c6d4a0 0x563af8c6d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:28.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.956+0000 7f72312b6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563af8c6d4a0 0x563af8c6d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35910/0 (socket says 172.21.15.164:35910) 2024-09-19T08:46:28.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.956+0000 7f72312b6640 1 -- 172.21.15.164:0/3689540124 learned_addr learned my addr 172.21.15.164:0/3689540124 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:28.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.956+0000 7f7231ab7640 1 --2- 172.21.15.164:0/3689540124 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563af8c78680 0x563af8c7aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:28.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.956+0000 7f7230ab5640 1 --2- 172.21.15.164:0/3689540124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563af8c6ddc0 0x563af8c78140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:28.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.956+0000 7f72312b6640 1 -- 172.21.15.164:0/3689540124 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563af8c78680 msgr2=0x563af8c7aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:28.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.956+0000 7f72312b6640 1 --2- 172.21.15.164:0/3689540124 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563af8c78680 0x563af8c7aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.956+0000 7f72312b6640 1 -- 172.21.15.164:0/3689540124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563af8c6ddc0 msgr2=0x563af8c78140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:28.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.956+0000 7f72312b6640 1 --2- 172.21.15.164:0/3689540124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563af8c6ddc0 0x563af8c78140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.956+0000 7f72312b6640 1 -- 172.21.15.164:0/3689540124 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563af8b5f440 con 0x563af8c6d4a0 2024-09-19T08:46:28.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.956+0000 7f72312b6640 1 --2- 172.21.15.164:0/3689540124 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563af8c6d4a0 0x563af8c6d880 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f72280058c0 tx=0x7f7228014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b19de9d127c63e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:28.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.956+0000 7f72177fe640 1 -- 172.21.15.164:0/3689540124 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7228005020 con 0x563af8c6d4a0 2024-09-19T08:46:28.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.957+0000 7f72177fe640 1 -- 172.21.15.164:0/3689540124 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72280047a0 con 0x563af8c6d4a0 2024-09-19T08:46:28.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.957+0000 7f72177fe640 1 -- 172.21.15.164:0/3689540124 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722801e780 con 0x563af8c6d4a0 2024-09-19T08:46:28.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.957+0000 7f72358f3740 1 -- 172.21.15.164:0/3689540124 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563af8c6d4a0 msgr2=0x563af8c6d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:28.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.957+0000 7f72358f3740 1 --2- 172.21.15.164:0/3689540124 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563af8c6d4a0 0x563af8c6d880 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f72280058c0 tx=0x7f7228014000 comp rx=0 tx=0).stop 2024-09-19T08:46:28.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.957+0000 7f72358f3740 1 -- 172.21.15.164:0/3689540124 shutdown_connections 2024-09-19T08:46:28.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.957+0000 7f72358f3740 1 --2- 172.21.15.164:0/3689540124 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563af8c78680 0x563af8c7aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.957+0000 7f72358f3740 1 --2- 172.21.15.164:0/3689540124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563af8c6ddc0 0x563af8c78140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.957+0000 7f72358f3740 1 --2- 172.21.15.164:0/3689540124 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563af8c6d4a0 0x563af8c6d880 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.957+0000 7f72358f3740 1 -- 172.21.15.164:0/3689540124 >> 172.21.15.164:0/3689540124 conn(0x563af8c69a30 msgr2=0x563af8c69e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:28.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.957+0000 7f72358f3740 1 -- 172.21.15.164:0/3689540124 shutdown_connections 2024-09-19T08:46:28.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.957+0000 7f72358f3740 1 -- 172.21.15.164:0/3689540124 wait complete. 2024-09-19T08:46:28.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.958+0000 7f72358f3740 1 -- start start 2024-09-19T08:46:28.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.958+0000 7f72358f3740 1 Processor -- start 2024-09-19T08:46:28.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.958+0000 7f72358f3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563af8c6d4a0 0x563af8c77f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:28.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.958+0000 7f72358f3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563af8c6ddc0 0x563af8c75210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:28.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.959+0000 7f72358f3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563af8c78680 0x563af8c75750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:28.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.959+0000 7f7230ab5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563af8c6ddc0 0x563af8c75210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:28.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.959+0000 7f7230ab5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563af8c6ddc0 0x563af8c75210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35920/0 (socket says 172.21.15.164:35920) 2024-09-19T08:46:28.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.959+0000 7f7230ab5640 1 -- 172.21.15.164:0/3892191738 learned_addr learned my addr 172.21.15.164:0/3892191738 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:28.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.959+0000 7f72358f3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563af8c7b250 con 0x563af8c6ddc0 2024-09-19T08:46:28.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.959+0000 7f72358f3740 1 -- 172.21.15.164:0/3892191738 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563af8b8e050 con 0x563af8c78680 2024-09-19T08:46:28.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.959+0000 7f72358f3740 1 -- 172.21.15.164:0/3892191738 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563af8c7b3d0 con 0x563af8c6d4a0 2024-09-19T08:46:28.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.959+0000 7f72312b6640 1 --2- 172.21.15.164:0/3892191738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563af8c6d4a0 0x563af8c77f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:28.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.959+0000 7f7231ab7640 1 --2- 172.21.15.164:0/3892191738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563af8c78680 0x563af8c75750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:28.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.959+0000 7f72312b6640 1 -- 172.21.15.164:0/3892191738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563af8c78680 msgr2=0x563af8c75750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:28.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.959+0000 7f72312b6640 1 --2- 172.21.15.164:0/3892191738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563af8c78680 0x563af8c75750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.959+0000 7f72312b6640 1 -- 172.21.15.164:0/3892191738 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563af8c6ddc0 msgr2=0x563af8c75210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:28.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.959+0000 7f72312b6640 1 --2- 172.21.15.164:0/3892191738 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563af8c6ddc0 0x563af8c75210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.959+0000 7f72312b6640 1 -- 172.21.15.164:0/3892191738 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563af89443c0 con 0x563af8c6d4a0 2024-09-19T08:46:28.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.959+0000 7f72358f3740 10 client.0 _ll_register_callbacks cb 0x563af8bce160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:28.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.959+0000 7f7230ab5640 1 --2- 172.21.15.164:0/3892191738 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563af8c6ddc0 0x563af8c75210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:46:28.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.959+0000 7f72358f3740 1 -- 172.21.15.164:0/3892191738 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72280047a0 con 0x563af8c6d4a0 2024-09-19T08:46:28.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.959+0000 7f72312b6640 1 --2- 172.21.15.164:0/3892191738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563af8c6d4a0 0x563af8c77f60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7228003e50 tx=0x7f72280043f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45c7d1056bb98a44 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:28.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.959+0000 7f7214ff9640 10 client.25099 ms_handle_connect on v2:172.21.15.173:3300/0 2024-09-19T08:46:28.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.960+0000 7f7214ff9640 1 -- 172.21.15.164:0/3892191738 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7228005210 con 0x563af8c6d4a0 2024-09-19T08:46:28.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.960+0000 7f7214ff9640 1 -- 172.21.15.164:0/3892191738 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f72280053b0 con 0x563af8c6d4a0 2024-09-19T08:46:28.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.960+0000 7f7214ff9640 1 -- 172.21.15.164:0/3892191738 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f72280055b0 con 0x563af8c6d4a0 2024-09-19T08:46:28.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.960+0000 7f72358f3740 10 client.25099 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:28.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.960+0000 7f72358f3740 1 -- 172.21.15.164:0/3892191738 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({fsmap=28}) -- 0x563af8c86a90 con 0x563af8c6d4a0 2024-09-19T08:46:28.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.960+0000 7f7214ff9640 1 -- 172.21.15.164:0/3892191738 <== mon.1 v2:172.21.15.173:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f722801e500 con 0x563af8c6d4a0 2024-09-19T08:46:28.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.960+0000 7f72358f3740 10 client.25099 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:28.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.960+0000 7f72358f3740 10 client.25099 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:46:28.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.960+0000 7f72358f3740 20 client.25099 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:28.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.960+0000 7f72358f3740 1 --2- 172.21.15.164:0/3892191738 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x563af8c87f20 0x563af8c8a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:28.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.961+0000 7f72358f3740 4 client.25099 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:46:28.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.961+0000 7f72358f3740 1 -- 172.21.15.164:0/3892191738 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563af8be0850 con 0x563af8c87f20 2024-09-19T08:46:28.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.961+0000 7f7230ab5640 1 --2- 172.21.15.164:0/3892191738 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x563af8c87f20 0x563af8c8a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:28.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.961+0000 7f7230ab5640 1 --2- 172.21.15.164:0/3892191738 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x563af8c87f20 0x563af8c8a310 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b12c5c40ea22cbb server_cookie=563bb7797c29e39 in_seq=0 out_seq=0 2024-09-19T08:46:28.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.961+0000 7f7214ff9640 10 client.25099 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:46:28.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.962+0000 7f7214ff9640 1 -- 172.21.15.164:0/3892191738 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563af8be0850 con 0x563af8c87f20 2024-09-19T08:46:28.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.962+0000 7f7214ff9640 10 client.25099 handle_command_reply: tid=0 2024-09-19T08:46:28.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.962+0000 7f72358f3740 1 client.25099 shutdown 2024-09-19T08:46:28.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.962+0000 7f72358f3740 1 -- 172.21.15.164:0/3892191738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563af8c6d4a0 msgr2=0x563af8c77f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:28.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.962+0000 7f72358f3740 1 --2- 172.21.15.164:0/3892191738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563af8c6d4a0 0x563af8c77f60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7228003e50 tx=0x7f72280043f0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.962+0000 7f72358f3740 1 -- 172.21.15.164:0/3892191738 shutdown_connections 2024-09-19T08:46:28.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.962+0000 7f72358f3740 1 --2- 172.21.15.164:0/3892191738 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x563af8c87f20 0x563af8c8a310 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.962+0000 7f72358f3740 1 --2- 172.21.15.164:0/3892191738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563af8c78680 0x563af8c75750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.962+0000 7f7214ff9640 0 client.25099 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:46:28.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.962+0000 7f72358f3740 1 --2- 172.21.15.164:0/3892191738 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563af8c6ddc0 0x563af8c75210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.962+0000 7f72358f3740 1 --2- 172.21.15.164:0/3892191738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563af8c6d4a0 0x563af8c77f60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.962+0000 7f72358f3740 1 -- 172.21.15.164:0/3892191738 >> 172.21.15.164:0/3892191738 conn(0x563af8c69a30 msgr2=0x563af8c68b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:28.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.962+0000 7f72358f3740 1 -- 172.21.15.164:0/3892191738 shutdown_connections 2024-09-19T08:46:28.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.962+0000 7f72358f3740 1 -- 172.21.15.164:0/3892191738 wait complete. 2024-09-19T08:46:28.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.962+0000 7f72358f3740 20 client.25099 trim_cache size 0 max 16384 2024-09-19T08:46:29.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.997+0000 7f72358f3740 1 Processor -- start 2024-09-19T08:46:29.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.997+0000 7f72358f3740 1 -- start start 2024-09-19T08:46:29.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.997+0000 7f72358f3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563af8c69a30 0x563af8c751d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:29.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.997+0000 7f72358f3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563af8c6d4a0 0x563af8c68a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:29.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.997+0000 7f72358f3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563af8c6ddc0 0x563af8c8d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:29.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.997+0000 7f72358f3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f722c1b3c10 con 0x563af8c6d4a0 2024-09-19T08:46:29.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.997+0000 7f72358f3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f722c1b5750 con 0x563af8c6ddc0 2024-09-19T08:46:29.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.997+0000 7f72358f3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f722c1b42e0 con 0x563af8c69a30 2024-09-19T08:46:29.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.997+0000 7f7230ab5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563af8c6d4a0 0x563af8c68a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:29.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.997+0000 7f7230ab5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563af8c6d4a0 0x563af8c68a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35934/0 (socket says 172.21.15.164:35934) 2024-09-19T08:46:29.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.997+0000 7f7230ab5640 1 -- 172.21.15.164:0/2558713402 learned_addr learned my addr 172.21.15.164:0/2558713402 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:29.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.997+0000 7f7231ab7640 1 --2- 172.21.15.164:0/2558713402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563af8c6ddc0 0x563af8c8d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:29.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.998+0000 7f72312b6640 1 --2- 172.21.15.164:0/2558713402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563af8c69a30 0x563af8c751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:29.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.998+0000 7f7230ab5640 1 -- 172.21.15.164:0/2558713402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563af8c69a30 msgr2=0x563af8c751d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:29.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.998+0000 7f7230ab5640 1 --2- 172.21.15.164:0/2558713402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563af8c69a30 0x563af8c751d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:29.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.998+0000 7f7230ab5640 1 -- 172.21.15.164:0/2558713402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563af8c6ddc0 msgr2=0x563af8c8d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:29.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.998+0000 7f7230ab5640 1 --2- 172.21.15.164:0/2558713402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563af8c6ddc0 0x563af8c8d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:29.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.998+0000 7f7230ab5640 1 -- 172.21.15.164:0/2558713402 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563af8c782d0 con 0x563af8c6d4a0 2024-09-19T08:46:29.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.998+0000 7f72312b6640 1 --2- 172.21.15.164:0/2558713402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563af8c69a30 0x563af8c751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:29.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.998+0000 7f7231ab7640 1 --2- 172.21.15.164:0/2558713402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563af8c6ddc0 0x563af8c8d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:29.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.998+0000 7f7230ab5640 1 --2- 172.21.15.164:0/2558713402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563af8c6d4a0 0x563af8c68a80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f721c02a9a0 tx=0x7f721c004420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532e63394830770b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:29.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.998+0000 7f7216ffd640 1 -- 172.21.15.164:0/2558713402 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721c035040 con 0x563af8c6d4a0 2024-09-19T08:46:29.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.998+0000 7f7216ffd640 1 -- 172.21.15.164:0/2558713402 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f721c003cd0 con 0x563af8c6d4a0 2024-09-19T08:46:29.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.998+0000 7f7216ffd640 1 -- 172.21.15.164:0/2558713402 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721c031ca0 con 0x563af8c6d4a0 2024-09-19T08:46:29.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.999+0000 7f72358f3740 1 -- 172.21.15.164:0/2558713402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563af8c6d4a0 msgr2=0x563af8c68a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:29.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.999+0000 7f72358f3740 1 --2- 172.21.15.164:0/2558713402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563af8c6d4a0 0x563af8c68a80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f721c02a9a0 tx=0x7f721c004420 comp rx=0 tx=0).stop 2024-09-19T08:46:29.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.999+0000 7f72358f3740 1 -- 172.21.15.164:0/2558713402 shutdown_connections 2024-09-19T08:46:29.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.999+0000 7f72358f3740 1 --2- 172.21.15.164:0/2558713402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563af8c6ddc0 0x563af8c8d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:29.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.999+0000 7f72358f3740 1 --2- 172.21.15.164:0/2558713402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563af8c6d4a0 0x563af8c68a80 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:29.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.999+0000 7f72358f3740 1 --2- 172.21.15.164:0/2558713402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563af8c69a30 0x563af8c751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:29.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.999+0000 7f72358f3740 1 -- 172.21.15.164:0/2558713402 >> 172.21.15.164:0/2558713402 conn(0x563af8c758b0 msgr2=0x563af8c68370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:29.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.999+0000 7f72358f3740 1 -- 172.21.15.164:0/2558713402 shutdown_connections 2024-09-19T08:46:29.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.999+0000 7f72358f3740 1 -- 172.21.15.164:0/2558713402 wait complete. 2024-09-19T08:46:29.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.999+0000 7f72358f3740 1 -- start start 2024-09-19T08:46:29.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:28.999+0000 7f72358f3740 1 Processor -- start 2024-09-19T08:46:29.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.000+0000 7f72358f3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563af8c69a30 0x563af8c8ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:29.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.000+0000 7f72358f3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563af8c6d4a0 0x563af8c8d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:29.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.000+0000 7f72358f3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563af8c6ddc0 0x563af8ce9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:29.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.000+0000 7f72358f3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f722c1b5080 con 0x563af8c6ddc0 2024-09-19T08:46:29.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.000+0000 7f72358f3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f722c1b49b0 con 0x563af8c6d4a0 2024-09-19T08:46:29.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.000+0000 7f72358f3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f722c1b1bf0 con 0x563af8c69a30 2024-09-19T08:46:29.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.000+0000 7f72358f3740 10 client.0 _ll_register_callbacks cb 0x7f7224004870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:29.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.000+0000 7f7231ab7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563af8c6ddc0 0x563af8ce9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:29.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.000+0000 7f7231ab7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563af8c6ddc0 0x563af8ce9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35946/0 (socket says 172.21.15.164:35946) 2024-09-19T08:46:29.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.000+0000 7f72312b6640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563af8c69a30 0x563af8c8ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:29.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.000+0000 7f7231ab7640 1 -- 172.21.15.164:0/2649398515 learned_addr learned my addr 172.21.15.164:0/2649398515 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:29.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.000+0000 7f7230ab5640 1 --2- 172.21.15.164:0/2649398515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563af8c6d4a0 0x563af8c8d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:29.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.000+0000 7f7231ab7640 1 -- 172.21.15.164:0/2649398515 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563af8c69a30 msgr2=0x563af8c8ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:29.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.000+0000 7f7231ab7640 1 --2- 172.21.15.164:0/2649398515 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563af8c69a30 0x563af8c8ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:29.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.000+0000 7f7231ab7640 1 -- 172.21.15.164:0/2649398515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563af8c6d4a0 msgr2=0x563af8c8d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:29.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.000+0000 7f7231ab7640 1 --2- 172.21.15.164:0/2649398515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563af8c6d4a0 0x563af8c8d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:29.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.001+0000 7f7231ab7640 1 -- 172.21.15.164:0/2649398515 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563af8cea1c0 con 0x563af8c6ddc0 2024-09-19T08:46:29.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.001+0000 7f72312b6640 1 --2- 172.21.15.164:0/2649398515 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563af8c69a30 0x563af8c8ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:46:29.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.001+0000 7f7231ab7640 1 --2- 172.21.15.164:0/2649398515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563af8c6ddc0 0x563af8ce9b20 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7224098280 tx=0x7f72240a0120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9a73b201dd4e56 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:29.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.001+0000 7f7214ff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:46:29.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.001+0000 7f7214ff9640 1 -- 172.21.15.164:0/2649398515 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7224020040 con 0x563af8c6ddc0 2024-09-19T08:46:29.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.001+0000 7f7214ff9640 1 -- 172.21.15.164:0/2649398515 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f72240cdb80 con 0x563af8c6ddc0 2024-09-19T08:46:29.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.001+0000 7f72358f3740 1 -- 172.21.15.164:0/2649398515 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f721c003cd0 con 0x563af8c6ddc0 2024-09-19T08:46:29.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.001+0000 7f7214ff9640 1 -- 172.21.15.164:0/2649398515 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f72240cdd80 con 0x563af8c6ddc0 2024-09-19T08:46:29.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.001+0000 7f72358f3740 10 client.15549 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:29.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.001+0000 7f72358f3740 1 -- 172.21.15.164:0/2649398515 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563af8cf27a0 con 0x563af8c6ddc0 2024-09-19T08:46:29.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.001+0000 7f7214ff9640 1 -- 172.21.15.164:0/2649398515 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72240cd590 con 0x563af8c6ddc0 2024-09-19T08:46:29.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.002+0000 7f7214ff9640 1 -- 172.21.15.164:0/2649398515 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f722409ace0 con 0x563af8c6ddc0 2024-09-19T08:46:29.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.002+0000 7f72358f3740 10 client.15549 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:29.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.002+0000 7f72358f3740 10 client.15549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:46:29.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.002+0000 7f72358f3740 20 client.15549 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:29.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.002+0000 7f72358f3740 1 --2- 172.21.15.164:0/2649398515 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x563af8c78680 0x563af8cf4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:29.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.002+0000 7f72358f3740 4 client.15549 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:46:29.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.002+0000 7f72358f3740 1 -- 172.21.15.164:0/2649398515 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f721c003cd0 con 0x563af8c78680 2024-09-19T08:46:29.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.002+0000 7f72312b6640 1 --2- 172.21.15.164:0/2649398515 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x563af8c78680 0x563af8cf4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:29.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.002+0000 7f72312b6640 1 --2- 172.21.15.164:0/2649398515 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x563af8c78680 0x563af8cf4f00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42e52bf09bbc103c server_cookie=9a59112a318a1fa7 in_seq=0 out_seq=0 2024-09-19T08:46:29.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.002+0000 7f7214ff9640 10 client.15549 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:46:29.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.003+0000 7f7214ff9640 1 -- 172.21.15.164:0/2649398515 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f721c003cd0 con 0x563af8c78680 2024-09-19T08:46:29.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.003+0000 7f7214ff9640 10 client.15549 handle_command_reply: tid=0 2024-09-19T08:46:29.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.003+0000 7f72358f3740 1 client.15549 shutdown 2024-09-19T08:46:29.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.003+0000 7f72358f3740 1 -- 172.21.15.164:0/2649398515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563af8c6ddc0 msgr2=0x563af8ce9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:29.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.003+0000 7f72358f3740 1 --2- 172.21.15.164:0/2649398515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563af8c6ddc0 0x563af8ce9b20 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7224098280 tx=0x7f72240a0120 comp rx=0 tx=0).stop 2024-09-19T08:46:29.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.003+0000 7f72358f3740 1 -- 172.21.15.164:0/2649398515 shutdown_connections 2024-09-19T08:46:29.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.003+0000 7f72358f3740 1 --2- 172.21.15.164:0/2649398515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563af8c6ddc0 0x563af8ce9b20 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:29.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.003+0000 7f72358f3740 1 --2- 172.21.15.164:0/2649398515 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x563af8c78680 0x563af8cf4f00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:29.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.003+0000 7f72358f3740 1 --2- 172.21.15.164:0/2649398515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563af8c6d4a0 0x563af8c8d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:29.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.003+0000 7f72358f3740 1 --2- 172.21.15.164:0/2649398515 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563af8c69a30 0x563af8c8ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:29.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.003+0000 7f72358f3740 1 -- 172.21.15.164:0/2649398515 >> 172.21.15.164:0/2649398515 conn(0x563af8c758b0 msgr2=0x563af8c68b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:29.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.003+0000 7f7214ff9640 0 client.15549 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:46:29.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.003+0000 7f72358f3740 1 -- 172.21.15.164:0/2649398515 shutdown_connections 2024-09-19T08:46:29.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.003+0000 7f72358f3740 1 -- 172.21.15.164:0/2649398515 wait complete. 2024-09-19T08:46:29.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.003+0000 7f72358f3740 20 client.15549 trim_cache size 0 max 16384 2024-09-19T08:46:29.013 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:46:29.014 INFO:teuthology.orchestra.run.smithi164.stdout: "status": "no active scrubs running", 2024-09-19T08:46:29.014 INFO:teuthology.orchestra.run.smithi164.stdout: "scrubs": {} 2024-09-19T08:46:29.014 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:46:29.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.004+0000 7f723351a640 1 -- 172.21.15.164:0/2889118195 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f72180a34d0 msgr2=0x7f72180a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:29.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.004+0000 7f723351a640 1 --2- 172.21.15.164:0/2889118195 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f72180a34d0 0x7f72180a5990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7228009bd0 tx=0x7f7228005f90 comp rx=0 tx=0).stop 2024-09-19T08:46:29.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.004+0000 7f723351a640 1 -- 172.21.15.164:0/2889118195 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f722c102ef0 msgr2=0x7f722c19ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:29.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.004+0000 7f723351a640 1 --2- 172.21.15.164:0/2889118195 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f722c102ef0 0x7f722c19ec20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7224009fd0 tx=0x7f722400b260 comp rx=0 tx=0).stop 2024-09-19T08:46:29.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.004+0000 7f723351a640 1 -- 172.21.15.164:0/2889118195 shutdown_connections 2024-09-19T08:46:29.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.004+0000 7f723351a640 1 --2- 172.21.15.164:0/2889118195 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f72180a34d0 0x7f72180a5990 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:29.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.004+0000 7f723351a640 1 --2- 172.21.15.164:0/2889118195 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f722c102ef0 0x7f722c19ec20 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:29.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.004+0000 7f723351a640 1 --2- 172.21.15.164:0/2889118195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f722c102570 0x7f722c1a4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:29.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.004+0000 7f723351a640 1 --2- 172.21.15.164:0/2889118195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f722c100e20 0x7f722c1a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:29.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.004+0000 7f723351a640 1 -- 172.21.15.164:0/2889118195 >> 172.21.15.164:0/2889118195 conn(0x7f722c0fc7d0 msgr2=0x7f722c06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:29.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.004+0000 7f723351a640 1 -- 172.21.15.164:0/2889118195 shutdown_connections 2024-09-19T08:46:29.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:29.004+0000 7f723351a640 1 -- 172.21.15.164:0/2889118195 wait complete. 2024-09-19T08:46:29.798 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T08:46:29.799 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:46:30.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:29 smithi164 ceph-mon[30928]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:46:30.055 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:46:30.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:29 smithi170 ceph-mon[35072]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:46:30.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:29 smithi173 ceph-mon[35306]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:46:30.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.528+0000 7f9581530640 1 -- 172.21.15.164:0/920371870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f957c100e20 msgr2=0x7f957c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:30.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.528+0000 7f9581530640 1 --2- 172.21.15.164:0/920371870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f957c100e20 0x7f957c101200 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f956c009980 tx=0x7f956c02c6b0 comp rx=0 tx=0).stop 2024-09-19T08:46:30.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.528+0000 7f9581530640 1 -- 172.21.15.164:0/920371870 shutdown_connections 2024-09-19T08:46:30.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.528+0000 7f9581530640 1 --2- 172.21.15.164:0/920371870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f957c102ef0 0x7f957c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:30.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.528+0000 7f9581530640 1 --2- 172.21.15.164:0/920371870 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f957c102570 0x7f957c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:30.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.528+0000 7f9581530640 1 --2- 172.21.15.164:0/920371870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f957c100e20 0x7f957c101200 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:30.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.528+0000 7f9581530640 1 -- 172.21.15.164:0/920371870 >> 172.21.15.164:0/920371870 conn(0x7f957c0fc7d0 msgr2=0x7f957c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:30.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.528+0000 7f9581530640 1 -- 172.21.15.164:0/920371870 shutdown_connections 2024-09-19T08:46:30.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.528+0000 7f9581530640 1 -- 172.21.15.164:0/920371870 wait complete. 2024-09-19T08:46:30.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.528+0000 7f9581530640 1 Processor -- start 2024-09-19T08:46:30.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.529+0000 7f9581530640 1 -- start start 2024-09-19T08:46:30.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.529+0000 7f9581530640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f957c100e20 0x7f957c1a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:30.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.529+0000 7f9581530640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f957c102570 0x7f957c1a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:30.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.529+0000 7f9581530640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f957c102ef0 0x7f957c19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:30.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.529+0000 7f9581530640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f957c10fe80 con 0x7f957c102570 2024-09-19T08:46:30.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.529+0000 7f9581530640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f957c10fd00 con 0x7f957c102ef0 2024-09-19T08:46:30.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.529+0000 7f9581530640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f957c110000 con 0x7f957c100e20 2024-09-19T08:46:30.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.529+0000 7f957a7fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f957c102570 0x7f957c1a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:30.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.529+0000 7f957a7fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f957c102570 0x7f957c1a0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35956/0 (socket says 172.21.15.164:35956) 2024-09-19T08:46:30.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.529+0000 7f957a7fc640 1 -- 172.21.15.164:0/242731844 learned_addr learned my addr 172.21.15.164:0/242731844 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:30.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.529+0000 7f957affd640 1 --2- 172.21.15.164:0/242731844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f957c100e20 0x7f957c1a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:30.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.529+0000 7f957b7fe640 1 --2- 172.21.15.164:0/242731844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f957c102ef0 0x7f957c19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:30.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.529+0000 7f957a7fc640 1 -- 172.21.15.164:0/242731844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f957c100e20 msgr2=0x7f957c1a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:30.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.529+0000 7f957a7fc640 1 --2- 172.21.15.164:0/242731844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f957c100e20 0x7f957c1a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:30.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.529+0000 7f957a7fc640 1 -- 172.21.15.164:0/242731844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f957c102ef0 msgr2=0x7f957c19a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:30.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.529+0000 7f957a7fc640 1 --2- 172.21.15.164:0/242731844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f957c102ef0 0x7f957c19a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:30.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.529+0000 7f957a7fc640 1 -- 172.21.15.164:0/242731844 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957c19acd0 con 0x7f957c102570 2024-09-19T08:46:30.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.529+0000 7f957b7fe640 1 --2- 172.21.15.164:0/242731844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f957c102ef0 0x7f957c19a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:30.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.530+0000 7f957a7fc640 1 --2- 172.21.15.164:0/242731844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f957c102570 0x7f957c1a0920 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f957000d770 tx=0x7f957000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:30.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.530+0000 7f9563fff640 1 -- 172.21.15.164:0/242731844 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9570017070 con 0x7f957c102570 2024-09-19T08:46:30.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.530+0000 7f9581530640 1 -- 172.21.15.164:0/242731844 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f957c19afc0 con 0x7f957c102570 2024-09-19T08:46:30.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.530+0000 7f9563fff640 1 -- 172.21.15.164:0/242731844 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9570005500 con 0x7f957c102570 2024-09-19T08:46:30.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.530+0000 7f9563fff640 1 -- 172.21.15.164:0/242731844 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9570004e00 con 0x7f957c102570 2024-09-19T08:46:30.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.530+0000 7f9581530640 1 -- 172.21.15.164:0/242731844 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f957c1a72b0 con 0x7f957c102570 2024-09-19T08:46:30.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.531+0000 7f9581530640 1 -- 172.21.15.164:0/242731844 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f957c101200 con 0x7f957c102570 2024-09-19T08:46:30.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.532+0000 7f9563fff640 1 -- 172.21.15.164:0/242731844 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7f9570004fa0 con 0x7f957c102570 2024-09-19T08:46:30.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.533+0000 7f9563fff640 1 --2- 172.21.15.164:0/242731844 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f95640a34d0 0x7f95640a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:30.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.533+0000 7f957affd640 1 --2- 172.21.15.164:0/242731844 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f95640a34d0 0x7f95640a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:30.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.533+0000 7f9563fff640 1 -- 172.21.15.164:0/242731844 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f95700cfe10 con 0x7f957c102570 2024-09-19T08:46:30.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.533+0000 7f957affd640 1 --2- 172.21.15.164:0/242731844 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f95640a34d0 0x7f95640a5990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f956c009230 tx=0x7f956c0058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:30.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.540+0000 7f9563fff640 1 -- 172.21.15.164:0/242731844 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f957009c610 con 0x7f957c102570 2024-09-19T08:46:30.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.751+0000 7f9581530640 1 -- 172.21.15.164:0/242731844 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f957c19bad0 con 0x7f957c102570 2024-09-19T08:46:30.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.752+0000 7f9563fff640 1 -- 172.21.15.164:0/242731844 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11984 (secure 0 0 0) 0x7f957c19bad0 con 0x7f957c102570 2024-09-19T08:46:30.751 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:46:30.752 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":28,"btime":"2024-09-19T08:45:46:903004+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:45:46.852789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:46:30.752 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 28 2024-09-19T08:46:30.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.754+0000 7f9581530640 1 -- 172.21.15.164:0/242731844 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f95640a34d0 msgr2=0x7f95640a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:30.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.754+0000 7f9581530640 1 --2- 172.21.15.164:0/242731844 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f95640a34d0 0x7f95640a5990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f956c009230 tx=0x7f956c0058d0 comp rx=0 tx=0).stop 2024-09-19T08:46:30.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.754+0000 7f9581530640 1 -- 172.21.15.164:0/242731844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f957c102570 msgr2=0x7f957c1a0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:30.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.754+0000 7f9581530640 1 --2- 172.21.15.164:0/242731844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f957c102570 0x7f957c1a0920 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f957000d770 tx=0x7f957000dc30 comp rx=0 tx=0).stop 2024-09-19T08:46:30.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.754+0000 7f9581530640 1 -- 172.21.15.164:0/242731844 shutdown_connections 2024-09-19T08:46:30.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.754+0000 7f9581530640 1 --2- 172.21.15.164:0/242731844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f957c102ef0 0x7f957c19a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:30.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.754+0000 7f9581530640 1 --2- 172.21.15.164:0/242731844 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f95640a34d0 0x7f95640a5990 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:30.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.754+0000 7f9581530640 1 --2- 172.21.15.164:0/242731844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f957c102570 0x7f957c1a0920 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:30.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.754+0000 7f9581530640 1 --2- 172.21.15.164:0/242731844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f957c100e20 0x7f957c1a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:30.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.754+0000 7f9581530640 1 -- 172.21.15.164:0/242731844 >> 172.21.15.164:0/242731844 conn(0x7f957c0fc7d0 msgr2=0x7f957c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:30.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.755+0000 7f9581530640 1 -- 172.21.15.164:0/242731844 shutdown_connections 2024-09-19T08:46:30.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:30.755+0000 7f9581530640 1 -- 172.21.15.164:0/242731844 wait complete. 2024-09-19T08:46:31.473 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T08:46:31.747 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:46:32.001 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:31 smithi164 ceph-mon[30928]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T08:46:32.001 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:31 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/242731844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:46:32.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:31 smithi170 ceph-mon[35072]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T08:46:32.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:31 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/242731844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:46:32.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:31 smithi173 ceph-mon[35306]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T08:46:32.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:31 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/242731844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:46:32.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.237+0000 7ff8a9379640 1 -- 172.21.15.164:0/1590727294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8a41082a0 msgr2=0x7ff8a410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:32.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.237+0000 7ff8a9379640 1 --2- 172.21.15.164:0/1590727294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8a41082a0 0x7ff8a410ec40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff89c009cd0 tx=0x7ff89c02a8e0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.237+0000 7ff8a9379640 1 -- 172.21.15.164:0/1590727294 shutdown_connections 2024-09-19T08:46:32.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.237+0000 7ff8a9379640 1 --2- 172.21.15.164:0/1590727294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8a41082a0 0x7ff8a410ec40 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.237+0000 7ff8a9379640 1 --2- 172.21.15.164:0/1590727294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8a41036e0 0x7ff8a4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.237+0000 7ff8a9379640 1 --2- 172.21.15.164:0/1590727294 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8a4102d30 0x7ff8a4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.237+0000 7ff8a9379640 1 -- 172.21.15.164:0/1590727294 >> 172.21.15.164:0/1590727294 conn(0x7ff8a40fc7d0 msgr2=0x7ff8a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:32.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.237+0000 7ff8a9379640 1 -- 172.21.15.164:0/1590727294 shutdown_connections 2024-09-19T08:46:32.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.237+0000 7ff8a9379640 1 -- 172.21.15.164:0/1590727294 wait complete. 2024-09-19T08:46:32.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.237+0000 7ff8a9379640 1 Processor -- start 2024-09-19T08:46:32.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.238+0000 7ff8a9379640 1 -- start start 2024-09-19T08:46:32.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.238+0000 7ff8a9379640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8a4102d30 0x7ff8a4100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:32.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.238+0000 7ff8a9379640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8a41036e0 0x7ff8a4100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:32.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.238+0000 7ff8a9379640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8a41082a0 0x7ff8a4101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:32.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.238+0000 7ff8a9379640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a41120a0 con 0x7ff8a41036e0 2024-09-19T08:46:32.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.238+0000 7ff8a9379640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a4111f20 con 0x7ff8a4102d30 2024-09-19T08:46:32.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.238+0000 7ff8a9379640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a4112220 con 0x7ff8a41082a0 2024-09-19T08:46:32.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.238+0000 7ff8a27fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8a41036e0 0x7ff8a4100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:32.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.238+0000 7ff8a27fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8a41036e0 0x7ff8a4100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35980/0 (socket says 172.21.15.164:35980) 2024-09-19T08:46:32.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.238+0000 7ff8a27fc640 1 -- 172.21.15.164:0/4238427086 learned_addr learned my addr 172.21.15.164:0/4238427086 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:32.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.238+0000 7ff8a2ffd640 1 --2- 172.21.15.164:0/4238427086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8a4102d30 0x7ff8a4100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:32.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.238+0000 7ff8a37fe640 1 --2- 172.21.15.164:0/4238427086 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8a41082a0 0x7ff8a4101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:32.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.239+0000 7ff8a27fc640 1 -- 172.21.15.164:0/4238427086 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8a41082a0 msgr2=0x7ff8a4101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:32.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.239+0000 7ff8a27fc640 1 --2- 172.21.15.164:0/4238427086 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8a41082a0 0x7ff8a4101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.239+0000 7ff8a27fc640 1 -- 172.21.15.164:0/4238427086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8a4102d30 msgr2=0x7ff8a4100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:32.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.239+0000 7ff8a27fc640 1 --2- 172.21.15.164:0/4238427086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8a4102d30 0x7ff8a4100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.239+0000 7ff8a27fc640 1 -- 172.21.15.164:0/4238427086 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a41a69b0 con 0x7ff8a41036e0 2024-09-19T08:46:32.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.239+0000 7ff8a27fc640 1 --2- 172.21.15.164:0/4238427086 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8a41036e0 0x7ff8a4100ed0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff89800c8c0 tx=0x7ff89800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:32.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.239+0000 7ff88bfff640 1 -- 172.21.15.164:0/4238427086 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898007bb0 con 0x7ff8a41036e0 2024-09-19T08:46:32.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.239+0000 7ff88bfff640 1 -- 172.21.15.164:0/4238427086 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff898007d50 con 0x7ff8a41036e0 2024-09-19T08:46:32.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.239+0000 7ff88bfff640 1 -- 172.21.15.164:0/4238427086 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898018830 con 0x7ff8a41036e0 2024-09-19T08:46:32.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.239+0000 7ff8a9379640 1 -- 172.21.15.164:0/4238427086 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8a41a6ca0 con 0x7ff8a41036e0 2024-09-19T08:46:32.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.239+0000 7ff8a9379640 1 -- 172.21.15.164:0/4238427086 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8a41a7190 con 0x7ff8a41036e0 2024-09-19T08:46:32.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.242+0000 7ff88bfff640 1 -- 172.21.15.164:0/4238427086 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7ff898018a10 con 0x7ff8a41036e0 2024-09-19T08:46:32.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.242+0000 7ff8ab752740 1 Processor -- start 2024-09-19T08:46:32.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.242+0000 7ff8ab752740 1 -- start start 2024-09-19T08:46:32.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.243+0000 7ff8ab752740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5625637bdb40 0x5625637bc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:32.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.243+0000 7ff8ab752740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625637bc5d0 0x5625637bca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:32.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.243+0000 7ff8ab752740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625637c7370 0x5625637c9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:32.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.243+0000 7ff8ab752740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5625637222f0 con 0x5625637c7370 2024-09-19T08:46:32.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.243+0000 7ff8ab752740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5625636c9cf0 con 0x5625637bdb40 2024-09-19T08:46:32.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.243+0000 7ff8ab752740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56256371e2b0 con 0x5625637bc5d0 2024-09-19T08:46:32.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.243+0000 7ff8a27fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625637c7370 0x5625637c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:32.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.243+0000 7ff8a27fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625637c7370 0x5625637c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35986/0 (socket says 172.21.15.164:35986) 2024-09-19T08:46:32.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.243+0000 7ff8a27fc640 1 -- 172.21.15.164:0/883703141 learned_addr learned my addr 172.21.15.164:0/883703141 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:32.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.243+0000 7ff8a2ffd640 1 --2- 172.21.15.164:0/883703141 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625637bc5d0 0x5625637bca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:32.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.243+0000 7ff8a37fe640 1 --2- 172.21.15.164:0/883703141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5625637bdb40 0x5625637bc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:32.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.243+0000 7ff88bfff640 1 --2- 172.21.15.164:0/4238427086 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff88c0a34d0 0x7ff88c0a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:32.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.243+0000 7ff8a27fc640 1 -- 172.21.15.164:0/883703141 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625637bc5d0 msgr2=0x5625637bca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:32.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.243+0000 7ff8a27fc640 1 --2- 172.21.15.164:0/883703141 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625637bc5d0 0x5625637bca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.243+0000 7ff8a27fc640 1 -- 172.21.15.164:0/883703141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5625637bdb40 msgr2=0x5625637bc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:32.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.243+0000 7ff8a27fc640 1 --2- 172.21.15.164:0/883703141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5625637bdb40 0x5625637bc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.243+0000 7ff8a27fc640 1 -- 172.21.15.164:0/883703141 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625636ae3a0 con 0x5625637c7370 2024-09-19T08:46:32.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.244+0000 7ff8a37fe640 1 --2- 172.21.15.164:0/4238427086 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff88c0a34d0 0x7ff88c0a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:32.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.244+0000 7ff88bfff640 1 -- 172.21.15.164:0/4238427086 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7ff8980cff90 con 0x7ff8a41036e0 2024-09-19T08:46:32.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.244+0000 7ff8a27fc640 1 --2- 172.21.15.164:0/883703141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625637c7370 0x5625637c9760 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff8980cd510 tx=0x7ff898066a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc1f3fdf9ef4770 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:32.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.244+0000 7ff8897fa640 1 -- 172.21.15.164:0/883703141 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89809b990 con 0x5625637c7370 2024-09-19T08:46:32.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.244+0000 7ff8897fa640 1 -- 172.21.15.164:0/883703141 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff89809c730 con 0x5625637c7370 2024-09-19T08:46:32.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.244+0000 7ff8897fa640 1 -- 172.21.15.164:0/883703141 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89806f360 con 0x5625637c7370 2024-09-19T08:46:32.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.245+0000 7ff8ab752740 1 -- 172.21.15.164:0/883703141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625637c7370 msgr2=0x5625637c9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:32.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.245+0000 7ff8a37fe640 1 --2- 172.21.15.164:0/4238427086 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff88c0a34d0 0x7ff88c0a5990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff89c006010 tx=0x7ff89c02a8e0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:32.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.245+0000 7ff8ab752740 1 --2- 172.21.15.164:0/883703141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625637c7370 0x5625637c9760 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff8980cd510 tx=0x7ff898066a20 comp rx=0 tx=0).stop 2024-09-19T08:46:32.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.245+0000 7ff8ab752740 1 -- 172.21.15.164:0/883703141 shutdown_connections 2024-09-19T08:46:32.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.245+0000 7ff8ab752740 1 --2- 172.21.15.164:0/883703141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625637c7370 0x5625637c9760 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.245+0000 7ff8ab752740 1 --2- 172.21.15.164:0/883703141 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625637bc5d0 0x5625637bca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.245+0000 7ff8ab752740 1 --2- 172.21.15.164:0/883703141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5625637bdb40 0x5625637bc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.245+0000 7ff8ab752740 1 -- 172.21.15.164:0/883703141 >> 172.21.15.164:0/883703141 conn(0x562563785310 msgr2=0x5625637b8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:32.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.245+0000 7ff8ab752740 1 -- 172.21.15.164:0/883703141 shutdown_connections 2024-09-19T08:46:32.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.245+0000 7ff8ab752740 1 -- 172.21.15.164:0/883703141 wait complete. 2024-09-19T08:46:32.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.245+0000 7ff8ab752740 1 -- start start 2024-09-19T08:46:32.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.246+0000 7ff8ab752740 1 Processor -- start 2024-09-19T08:46:32.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.246+0000 7ff8ab752740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625637bc5d0 0x5625637c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:32.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.246+0000 7ff8ab752740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5625637bdb40 0x5625637c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:32.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.246+0000 7ff8ab752740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625637c7370 0x5625637c44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:32.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.246+0000 7ff8ab752740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5625637ca010 con 0x5625637c7370 2024-09-19T08:46:32.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.246+0000 7ff8ab752740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5625636dd060 con 0x5625637bdb40 2024-09-19T08:46:32.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.246+0000 7ff8ab752740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5625637ca190 con 0x5625637bc5d0 2024-09-19T08:46:32.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.246+0000 7ff8ab752740 10 client.0 _ll_register_callbacks cb 0x5625634308c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:32.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.246+0000 7ff8a2ffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625637bc5d0 0x5625637c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:32.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.246+0000 7ff8a2ffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625637bc5d0 0x5625637c6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:53574/0 (socket says 172.21.15.164:53574) 2024-09-19T08:46:32.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.246+0000 7ff8a2ffd640 1 -- 172.21.15.164:0/2247204089 learned_addr learned my addr 172.21.15.164:0/2247204089 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:32.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.246+0000 7ff8a27fc640 1 --2- 172.21.15.164:0/2247204089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5625637bdb40 0x5625637c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:32.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.246+0000 7ff8a37fe640 1 --2- 172.21.15.164:0/2247204089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625637c7370 0x5625637c44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:32.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.246+0000 7ff8a2ffd640 1 -- 172.21.15.164:0/2247204089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5625637bdb40 msgr2=0x5625637c3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:32.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.246+0000 7ff8a2ffd640 1 --2- 172.21.15.164:0/2247204089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5625637bdb40 0x5625637c3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.246+0000 7ff8a2ffd640 1 -- 172.21.15.164:0/2247204089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625637c7370 msgr2=0x5625637c44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:32.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.246+0000 7ff8a2ffd640 1 --2- 172.21.15.164:0/2247204089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625637c7370 0x5625637c44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.246+0000 7ff8a2ffd640 1 -- 172.21.15.164:0/2247204089 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625634933c0 con 0x5625637bc5d0 2024-09-19T08:46:32.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.247+0000 7ff8a2ffd640 1 --2- 172.21.15.164:0/2247204089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625637bc5d0 0x5625637c6cc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff89400ba10 tx=0x7ff89400bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1461bd57e84fed5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:32.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.247+0000 7ff886ff5640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-09-19T08:46:32.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.247+0000 7ff886ff5640 1 -- 172.21.15.164:0/2247204089 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894003e80 con 0x5625637bc5d0 2024-09-19T08:46:32.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.247+0000 7ff886ff5640 1 -- 172.21.15.164:0/2247204089 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7ff894011440 con 0x5625637bc5d0 2024-09-19T08:46:32.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.247+0000 7ff8ab752740 1 -- 172.21.15.164:0/2247204089 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff89809c730 con 0x5625637bc5d0 2024-09-19T08:46:32.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.247+0000 7ff886ff5640 1 -- 172.21.15.164:0/2247204089 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff894011640 con 0x5625637bc5d0 2024-09-19T08:46:32.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.247+0000 7ff8ab752740 10 client.25123 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:32.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.247+0000 7ff8ab752740 1 -- 172.21.15.164:0/2247204089 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5625637d5ab0 con 0x5625637bc5d0 2024-09-19T08:46:32.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.248+0000 7ff886ff5640 1 -- 172.21.15.164:0/2247204089 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff894011cc0 con 0x5625637bc5d0 2024-09-19T08:46:32.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.248+0000 7ff886ff5640 1 -- 172.21.15.164:0/2247204089 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff89401bc00 con 0x5625637bc5d0 2024-09-19T08:46:32.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.248+0000 7ff8ab752740 10 client.25123 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:32.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.248+0000 7ff8ab752740 10 client.25123 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-19T08:46:32.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.248+0000 7ff8ab752740 20 client.25123 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:32.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.248+0000 7ff8ab752740 1 --2- 172.21.15.164:0/2247204089 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5625637d6f70 0x5625637d93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:32.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.248+0000 7ff8ab752740 4 client.25123 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:46:32.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.248+0000 7ff8ab752740 1 -- 172.21.15.164:0/2247204089 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff89809c730 con 0x5625637d6f70 2024-09-19T08:46:32.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.248+0000 7ff8a27fc640 1 --2- 172.21.15.164:0/2247204089 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5625637d6f70 0x5625637d93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:32.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.248+0000 7ff886ff5640 10 client.25123 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:46:32.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.248+0000 7ff8a27fc640 1 --2- 172.21.15.164:0/2247204089 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5625637d6f70 0x5625637d93c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3775c694ccaa6242 server_cookie=a2931186c07d9854 in_seq=0 out_seq=0 2024-09-19T08:46:32.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.249+0000 7ff886ff5640 1 -- 172.21.15.164:0/2247204089 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7ff89809c730 con 0x5625637d6f70 2024-09-19T08:46:32.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.249+0000 7ff886ff5640 10 client.25123 handle_command_reply: tid=0 2024-09-19T08:46:32.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.249+0000 7ff8ab752740 1 client.25123 shutdown 2024-09-19T08:46:32.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.249+0000 7ff8ab752740 1 -- 172.21.15.164:0/2247204089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625637bc5d0 msgr2=0x5625637c6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:32.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.250+0000 7ff8ab752740 1 --2- 172.21.15.164:0/2247204089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625637bc5d0 0x5625637c6cc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff89400ba10 tx=0x7ff89400bed0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.250+0000 7ff8ab752740 1 -- 172.21.15.164:0/2247204089 shutdown_connections 2024-09-19T08:46:32.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.250+0000 7ff8ab752740 1 --2- 172.21.15.164:0/2247204089 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5625637d6f70 0x5625637d93c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.250+0000 7ff8ab752740 1 --2- 172.21.15.164:0/2247204089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625637c7370 0x5625637c44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.250+0000 7ff8ab752740 1 --2- 172.21.15.164:0/2247204089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5625637bdb40 0x5625637c3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.250+0000 7ff8ab752740 1 --2- 172.21.15.164:0/2247204089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625637bc5d0 0x5625637c6cc0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.250+0000 7ff8ab752740 1 -- 172.21.15.164:0/2247204089 >> 172.21.15.164:0/2247204089 conn(0x562563785310 msgr2=0x5625637be960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:32.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.250+0000 7ff886ff5640 0 client.25123 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:46:32.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.250+0000 7ff8ab752740 1 -- 172.21.15.164:0/2247204089 shutdown_connections 2024-09-19T08:46:32.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.250+0000 7ff8ab752740 1 -- 172.21.15.164:0/2247204089 wait complete. 2024-09-19T08:46:32.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.250+0000 7ff8ab752740 20 client.25123 trim_cache size 0 max 16384 2024-09-19T08:46:32.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.268+0000 7ff8ab752740 1 Processor -- start 2024-09-19T08:46:32.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.268+0000 7ff8ab752740 1 -- start start 2024-09-19T08:46:32.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.268+0000 7ff8ab752740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625637bc5d0 0x5625637c2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:32.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.268+0000 7ff8ab752740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625637bdb40 0x5625637c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:32.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.268+0000 7ff8ab752740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5625637c7370 0x5625637bf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:32.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.268+0000 7ff8ab752740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a41af8e0 con 0x5625637bc5d0 2024-09-19T08:46:32.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.268+0000 7ff8ab752740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a41b1420 con 0x5625637c7370 2024-09-19T08:46:32.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.268+0000 7ff8ab752740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a41affb0 con 0x5625637bdb40 2024-09-19T08:46:32.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.268+0000 7ff8a2ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625637bc5d0 0x5625637c2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:32.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.268+0000 7ff8a27fc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625637bdb40 0x5625637c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:32.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.269+0000 7ff8a27fc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625637bdb40 0x5625637c5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:53582/0 (socket says 172.21.15.164:53582) 2024-09-19T08:46:32.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.269+0000 7ff8a27fc640 1 -- 172.21.15.164:0/2996131786 learned_addr learned my addr 172.21.15.164:0/2996131786 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:32.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.269+0000 7ff8a37fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5625637c7370 0x5625637bf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:32.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.269+0000 7ff8a37fe640 1 -- 172.21.15.164:0/2996131786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625637bdb40 msgr2=0x5625637c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:32.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.269+0000 7ff8a37fe640 1 --2- 172.21.15.164:0/2996131786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625637bdb40 0x5625637c5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.269+0000 7ff8a37fe640 1 -- 172.21.15.164:0/2996131786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625637bc5d0 msgr2=0x5625637c2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:32.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.269+0000 7ff8a37fe640 1 --2- 172.21.15.164:0/2996131786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625637bc5d0 0x5625637c2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.269+0000 7ff8a37fe640 1 -- 172.21.15.164:0/2996131786 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625637c4800 con 0x5625637c7370 2024-09-19T08:46:32.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.269+0000 7ff8a37fe640 1 --2- 172.21.15.164:0/2996131786 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5625637c7370 0x5625637bf120 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff89c009ca0 tx=0x7ff89c005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e9bf170a5a0feb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:32.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.269+0000 7ff888ff9640 1 -- 172.21.15.164:0/2996131786 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89c004590 con 0x5625637c7370 2024-09-19T08:46:32.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.269+0000 7ff888ff9640 1 -- 172.21.15.164:0/2996131786 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff89c036050 con 0x5625637c7370 2024-09-19T08:46:32.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.269+0000 7ff888ff9640 1 -- 172.21.15.164:0/2996131786 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89c03b660 con 0x5625637c7370 2024-09-19T08:46:32.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.270+0000 7ff8ab752740 1 -- 172.21.15.164:0/2996131786 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5625637c7370 msgr2=0x5625637bf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:32.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.270+0000 7ff8ab752740 1 --2- 172.21.15.164:0/2996131786 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5625637c7370 0x5625637bf120 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff89c009ca0 tx=0x7ff89c005ef0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.270+0000 7ff8ab752740 1 -- 172.21.15.164:0/2996131786 shutdown_connections 2024-09-19T08:46:32.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.270+0000 7ff8ab752740 1 --2- 172.21.15.164:0/2996131786 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5625637c7370 0x5625637bf120 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.270+0000 7ff8ab752740 1 --2- 172.21.15.164:0/2996131786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625637bdb40 0x5625637c5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.270+0000 7ff8ab752740 1 --2- 172.21.15.164:0/2996131786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625637bc5d0 0x5625637c2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.270+0000 7ff8ab752740 1 -- 172.21.15.164:0/2996131786 >> 172.21.15.164:0/2996131786 conn(0x562563785310 msgr2=0x5625637c57d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:32.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.270+0000 7ff8ab752740 1 -- 172.21.15.164:0/2996131786 shutdown_connections 2024-09-19T08:46:32.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.270+0000 7ff8ab752740 1 -- 172.21.15.164:0/2996131786 wait complete. 2024-09-19T08:46:32.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.271+0000 7ff8ab752740 1 -- start start 2024-09-19T08:46:32.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.271+0000 7ff8ab752740 1 Processor -- start 2024-09-19T08:46:32.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.271+0000 7ff8ab752740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625637bc5d0 0x5625637cf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:32.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.271+0000 7ff8ab752740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5625637bdb40 0x5625637cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:32.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.271+0000 7ff8ab752740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625637c7370 0x5625637cc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:32.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.271+0000 7ff8ab752740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a41b0d50 con 0x5625637bc5d0 2024-09-19T08:46:32.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.271+0000 7ff8ab752740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a41b0680 con 0x5625637bdb40 2024-09-19T08:46:32.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.271+0000 7ff8ab752740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a41ad8c0 con 0x5625637c7370 2024-09-19T08:46:32.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.271+0000 7ff8ab752740 10 client.0 _ll_register_callbacks cb 0x7ff8940042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:32.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.272+0000 7ff8a2ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625637bc5d0 0x5625637cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:32.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.272+0000 7ff8a2ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625637bc5d0 0x5625637cf450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36006/0 (socket says 172.21.15.164:36006) 2024-09-19T08:46:32.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.272+0000 7ff8a2ffd640 1 -- 172.21.15.164:0/927812749 learned_addr learned my addr 172.21.15.164:0/927812749 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:32.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.272+0000 7ff8a27fc640 1 --2- 172.21.15.164:0/927812749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5625637bdb40 0x5625637cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:32.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.272+0000 7ff8a37fe640 1 --2- 172.21.15.164:0/927812749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625637c7370 0x5625637cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:32.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.272+0000 7ff8a27fc640 1 -- 172.21.15.164:0/927812749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625637c7370 msgr2=0x5625637cc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:32.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.272+0000 7ff8a27fc640 1 --2- 172.21.15.164:0/927812749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625637c7370 0x5625637cc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.272+0000 7ff8a27fc640 1 -- 172.21.15.164:0/927812749 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625637bc5d0 msgr2=0x5625637cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:32.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.272+0000 7ff8a27fc640 1 --2- 172.21.15.164:0/927812749 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625637bc5d0 0x5625637cf450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.272+0000 7ff8a27fc640 1 -- 172.21.15.164:0/927812749 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625637d0080 con 0x5625637bdb40 2024-09-19T08:46:32.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.272+0000 7ff8a37fe640 1 --2- 172.21.15.164:0/927812749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625637c7370 0x5625637cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:46:32.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.272+0000 7ff8a27fc640 1 --2- 172.21.15.164:0/927812749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5625637bdb40 0x5625637cf990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff89809bfc0 tx=0x7ff89800fe90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d5702bd218f3a08 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:32.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.272+0000 7ff886ff5640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-19T08:46:32.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.273+0000 7ff886ff5640 1 -- 172.21.15.164:0/927812749 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89809b990 con 0x5625637bdb40 2024-09-19T08:46:32.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.273+0000 7ff886ff5640 1 -- 172.21.15.164:0/927812749 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7ff89809a330 con 0x5625637bdb40 2024-09-19T08:46:32.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.273+0000 7ff8ab752740 1 -- 172.21.15.164:0/927812749 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff89c036050 con 0x5625637bdb40 2024-09-19T08:46:32.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.273+0000 7ff886ff5640 1 -- 172.21.15.164:0/927812749 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff89809bb70 con 0x5625637bdb40 2024-09-19T08:46:32.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.273+0000 7ff8ab752740 10 client.25136 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:32.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.273+0000 7ff8ab752740 1 -- 172.21.15.164:0/927812749 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5625638422c0 con 0x5625637bdb40 2024-09-19T08:46:32.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.273+0000 7ff886ff5640 1 -- 172.21.15.164:0/927812749 <== mon.2 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff89c036050 con 0x5625637bdb40 2024-09-19T08:46:32.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.274+0000 7ff886ff5640 1 -- 172.21.15.164:0/927812749 <== mon.2 v2:172.21.15.170:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff8980986a0 con 0x5625637bdb40 2024-09-19T08:46:32.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.274+0000 7ff8ab752740 10 client.25136 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:32.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.274+0000 7ff8ab752740 10 client.25136 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-19T08:46:32.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.274+0000 7ff8ab752740 20 client.25136 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:32.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.274+0000 7ff8ab752740 1 --2- 172.21.15.164:0/927812749 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5625638426a0 0x562563844a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:32.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.274+0000 7ff8ab752740 4 client.25136 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:46:32.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.274+0000 7ff8ab752740 1 -- 172.21.15.164:0/927812749 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56256372f860 con 0x5625638426a0 2024-09-19T08:46:32.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.274+0000 7ff8a2ffd640 1 --2- 172.21.15.164:0/927812749 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5625638426a0 0x562563844a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:32.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.275+0000 7ff8a2ffd640 1 --2- 172.21.15.164:0/927812749 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5625638426a0 0x562563844a90 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a633fc182509de9c server_cookie=134c035af9b0dec0 in_seq=0 out_seq=0 2024-09-19T08:46:32.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.275+0000 7ff886ff5640 10 client.25136 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:46:32.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.275+0000 7ff886ff5640 1 -- 172.21.15.164:0/927812749 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56256372f860 con 0x5625638426a0 2024-09-19T08:46:32.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.275+0000 7ff886ff5640 10 client.25136 handle_command_reply: tid=0 2024-09-19T08:46:32.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.275+0000 7ff8ab752740 1 client.25136 shutdown 2024-09-19T08:46:32.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.275+0000 7ff8ab752740 1 -- 172.21.15.164:0/927812749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5625637bdb40 msgr2=0x5625637cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:32.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.275+0000 7ff8ab752740 1 --2- 172.21.15.164:0/927812749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5625637bdb40 0x5625637cf990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff89809bfc0 tx=0x7ff89800fe90 comp rx=0 tx=0).stop 2024-09-19T08:46:32.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.275+0000 7ff8ab752740 1 -- 172.21.15.164:0/927812749 shutdown_connections 2024-09-19T08:46:32.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.275+0000 7ff8ab752740 1 --2- 172.21.15.164:0/927812749 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5625638426a0 0x562563844a90 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.275+0000 7ff8ab752740 1 --2- 172.21.15.164:0/927812749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625637c7370 0x5625637cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.275+0000 7ff8ab752740 1 --2- 172.21.15.164:0/927812749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5625637bdb40 0x5625637cf990 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.275+0000 7ff886ff5640 0 client.25136 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:46:32.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.275+0000 7ff8ab752740 1 --2- 172.21.15.164:0/927812749 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5625637bc5d0 0x5625637cf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.275+0000 7ff8ab752740 1 -- 172.21.15.164:0/927812749 >> 172.21.15.164:0/927812749 conn(0x562563785310 msgr2=0x5625637b7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:32.280 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:46:32.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.275+0000 7ff8ab752740 1 -- 172.21.15.164:0/927812749 shutdown_connections 2024-09-19T08:46:32.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.275+0000 7ff8ab752740 1 -- 172.21.15.164:0/927812749 wait complete. 2024-09-19T08:46:32.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.276+0000 7ff8ab752740 20 client.25136 trim_cache size 0 max 16384 2024-09-19T08:46:32.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.276+0000 7ff8a9379640 1 -- 172.21.15.164:0/4238427086 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff88c0a34d0 msgr2=0x7ff88c0a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:32.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.276+0000 7ff8a9379640 1 --2- 172.21.15.164:0/4238427086 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff88c0a34d0 0x7ff88c0a5990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff89c006010 tx=0x7ff89c02a8e0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.276+0000 7ff8a9379640 1 -- 172.21.15.164:0/4238427086 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8a41036e0 msgr2=0x7ff8a4100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:32.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.276+0000 7ff8a9379640 1 --2- 172.21.15.164:0/4238427086 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8a41036e0 0x7ff8a4100ed0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff89800c8c0 tx=0x7ff89800cd80 comp rx=0 tx=0).stop 2024-09-19T08:46:32.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.276+0000 7ff8a9379640 1 -- 172.21.15.164:0/4238427086 shutdown_connections 2024-09-19T08:46:32.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.276+0000 7ff8a9379640 1 --2- 172.21.15.164:0/4238427086 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8a41082a0 0x7ff8a4101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.276+0000 7ff8a9379640 1 --2- 172.21.15.164:0/4238427086 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff88c0a34d0 0x7ff88c0a5990 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.276+0000 7ff8a9379640 1 --2- 172.21.15.164:0/4238427086 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8a41036e0 0x7ff8a4100ed0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.276+0000 7ff8a9379640 1 --2- 172.21.15.164:0/4238427086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8a4102d30 0x7ff8a4100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.276+0000 7ff8a9379640 1 -- 172.21.15.164:0/4238427086 >> 172.21.15.164:0/4238427086 conn(0x7ff8a40fc7d0 msgr2=0x7ff8a4106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:32.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.277+0000 7ff8a9379640 1 -- 172.21.15.164:0/4238427086 shutdown_connections 2024-09-19T08:46:32.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:32.277+0000 7ff8a9379640 1 -- 172.21.15.164:0/4238427086 wait complete. 2024-09-19T08:46:32.980 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T08:46:33.245 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:46:33.732 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 11.2 2024-09-19T08:46:33.733 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:46:33.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.764+0000 7fa26aa69640 1 -- 172.21.15.164:0/266186947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2640745c0 msgr2=0x7fa2640749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:33.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.764+0000 7fa26aa69640 1 --2- 172.21.15.164:0/266186947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2640745c0 0x7fa2640749a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa258009900 tx=0x7fa25802a480 comp rx=0 tx=0).stop 2024-09-19T08:46:33.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.764+0000 7fa26aa69640 1 -- 172.21.15.164:0/266186947 shutdown_connections 2024-09-19T08:46:33.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.764+0000 7fa26aa69640 1 --2- 172.21.15.164:0/266186947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa26410e430 0x7fa264110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.764+0000 7fa26aa69640 1 --2- 172.21.15.164:0/266186947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa264074ee0 0x7fa26410def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.764+0000 7fa26aa69640 1 --2- 172.21.15.164:0/266186947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2640745c0 0x7fa2640749a0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.764+0000 7fa26aa69640 1 -- 172.21.15.164:0/266186947 >> 172.21.15.164:0/266186947 conn(0x7fa2640fd8b0 msgr2=0x7fa2640ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:33.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.764+0000 7fa26aa69640 1 -- 172.21.15.164:0/266186947 shutdown_connections 2024-09-19T08:46:33.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.764+0000 7fa26aa69640 1 -- 172.21.15.164:0/266186947 wait complete. 2024-09-19T08:46:33.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.765+0000 7fa26aa69640 1 Processor -- start 2024-09-19T08:46:33.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.765+0000 7fa26aa69640 1 -- start start 2024-09-19T08:46:33.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.765+0000 7fa26aa69640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2640745c0 0x7fa26419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:33.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.765+0000 7fa26aa69640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa264074ee0 0x7fa26419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:33.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.765+0000 7fa26aa69640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa26410e430 0x7fa26419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:33.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.765+0000 7fa26aa69640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa2641141f0 con 0x7fa2640745c0 2024-09-19T08:46:33.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.765+0000 7fa26aa69640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa264114070 con 0x7fa26410e430 2024-09-19T08:46:33.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.765+0000 7fa26aa69640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa264114370 con 0x7fa264074ee0 2024-09-19T08:46:33.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.766+0000 7fa269006640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa26410e430 0x7fa26419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:33.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.766+0000 7fa269006640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa26410e430 0x7fa26419ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:51314/0 (socket says 172.21.15.164:51314) 2024-09-19T08:46:33.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.766+0000 7fa269006640 1 -- 172.21.15.164:0/2258292379 learned_addr learned my addr 172.21.15.164:0/2258292379 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:33.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.766+0000 7fa268805640 1 --2- 172.21.15.164:0/2258292379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2640745c0 0x7fa26419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:33.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.766+0000 7fa263fff640 1 --2- 172.21.15.164:0/2258292379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa264074ee0 0x7fa26419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:33.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.766+0000 7fa269006640 1 -- 172.21.15.164:0/2258292379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa264074ee0 msgr2=0x7fa26419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:33.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.766+0000 7fa269006640 1 --2- 172.21.15.164:0/2258292379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa264074ee0 0x7fa26419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.766+0000 7fa269006640 1 -- 172.21.15.164:0/2258292379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2640745c0 msgr2=0x7fa26419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:33.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.766+0000 7fa269006640 1 --2- 172.21.15.164:0/2258292379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2640745c0 0x7fa26419a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.766+0000 7fa269006640 1 -- 172.21.15.164:0/2258292379 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa26419f600 con 0x7fa26410e430 2024-09-19T08:46:33.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.766+0000 7fa269006640 1 --2- 172.21.15.164:0/2258292379 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa26410e430 0x7fa26419ef20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa25c00eb40 tx=0x7fa25c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:33.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.766+0000 7fa261ffb640 1 -- 172.21.15.164:0/2258292379 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25c004b20 con 0x7fa26410e430 2024-09-19T08:46:33.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.767+0000 7fa26aa69640 1 -- 172.21.15.164:0/2258292379 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa26419f8f0 con 0x7fa26410e430 2024-09-19T08:46:33.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.767+0000 7fa261ffb640 1 -- 172.21.15.164:0/2258292379 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa25c004cc0 con 0x7fa26410e430 2024-09-19T08:46:33.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.767+0000 7fa261ffb640 1 -- 172.21.15.164:0/2258292379 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25c010850 con 0x7fa26410e430 2024-09-19T08:46:33.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.767+0000 7fa26aa69640 1 -- 172.21.15.164:0/2258292379 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2641a71d0 con 0x7fa26410e430 2024-09-19T08:46:33.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.769+0000 7fa261ffb640 1 -- 172.21.15.164:0/2258292379 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7fa25c005020 con 0x7fa26410e430 2024-09-19T08:46:33.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.769+0000 7fa26ce42740 1 Processor -- start 2024-09-19T08:46:33.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.769+0000 7fa261ffb640 1 --2- 172.21.15.164:0/2258292379 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa2540a3550 0x7fa2540a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:33.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.770+0000 7fa26ce42740 1 -- start start 2024-09-19T08:46:33.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.770+0000 7fa261ffb640 1 -- 172.21.15.164:0/2258292379 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fa25c0d06b0 con 0x7fa26410e430 2024-09-19T08:46:33.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.770+0000 7fa268805640 1 --2- 172.21.15.164:0/2258292379 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa2540a3550 0x7fa2540a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:33.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.770+0000 7fa26ce42740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5571648e6b40 0x5571648e5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:33.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.770+0000 7fa26ce42740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5571648e55d0 0x5571648e5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:33.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.770+0000 7fa26ce42740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571648f0370 0x5571648f2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:33.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.770+0000 7fa26ce42740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55716484b2f0 con 0x5571648e6b40 2024-09-19T08:46:33.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.770+0000 7fa26ce42740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5571647f2cf0 con 0x5571648e55d0 2024-09-19T08:46:33.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.770+0000 7fa26ce42740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5571648472b0 con 0x5571648f0370 2024-09-19T08:46:33.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.770+0000 7fa268805640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5571648e6b40 0x5571648e5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:33.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.770+0000 7fa268805640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5571648e6b40 0x5571648e5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45368/0 (socket says 172.21.15.164:45368) 2024-09-19T08:46:33.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.770+0000 7fa268805640 1 -- 172.21.15.164:0/3703841451 learned_addr learned my addr 172.21.15.164:0/3703841451 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:33.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.770+0000 7fa263fff640 1 --2- 172.21.15.164:0/3703841451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5571648e55d0 0x5571648e5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:33.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.770+0000 7fa269006640 1 --2- 172.21.15.164:0/3703841451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571648f0370 0x5571648f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:33.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.770+0000 7fa268805640 1 -- 172.21.15.164:0/3703841451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571648f0370 msgr2=0x5571648f2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:33.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.770+0000 7fa268805640 1 --2- 172.21.15.164:0/3703841451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571648f0370 0x5571648f2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.770+0000 7fa268805640 1 -- 172.21.15.164:0/3703841451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5571648e55d0 msgr2=0x5571648e5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:33.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.770+0000 7fa268805640 1 --2- 172.21.15.164:0/3703841451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5571648e55d0 0x5571648e5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.770+0000 7fa268805640 1 -- 172.21.15.164:0/3703841451 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571647d73a0 con 0x5571648e6b40 2024-09-19T08:46:33.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.770+0000 7fa269006640 1 --2- 172.21.15.164:0/3703841451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571648f0370 0x5571648f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:33.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.770+0000 7fa263fff640 1 --2- 172.21.15.164:0/3703841451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5571648e55d0 0x5571648e5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:33.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.771+0000 7fa268805640 1 --2- 172.21.15.164:0/3703841451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5571648e6b40 0x5571648e5090 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa258009230 tx=0x7fa2580052e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4158742ca705a3fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:33.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.771+0000 7fa268805640 1 --2- 172.21.15.164:0/2258292379 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa2540a3550 0x7fa2540a5a10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa258005c40 tx=0x7fa25803a000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:33.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.771+0000 7fa24effd640 1 -- 172.21.15.164:0/3703841451 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25804e070 con 0x5571648e6b40 2024-09-19T08:46:33.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.771+0000 7fa24effd640 1 -- 172.21.15.164:0/3703841451 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa258004d30 con 0x5571648e6b40 2024-09-19T08:46:33.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.771+0000 7fa24effd640 1 -- 172.21.15.164:0/3703841451 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa258035040 con 0x5571648e6b40 2024-09-19T08:46:33.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.772+0000 7fa26ce42740 1 -- 172.21.15.164:0/3703841451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5571648e6b40 msgr2=0x5571648e5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:33.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.772+0000 7fa26ce42740 1 --2- 172.21.15.164:0/3703841451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5571648e6b40 0x5571648e5090 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa258009230 tx=0x7fa2580052e0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.772+0000 7fa26ce42740 1 -- 172.21.15.164:0/3703841451 shutdown_connections 2024-09-19T08:46:33.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.772+0000 7fa26ce42740 1 --2- 172.21.15.164:0/3703841451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571648f0370 0x5571648f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.772+0000 7fa26ce42740 1 --2- 172.21.15.164:0/3703841451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5571648e55d0 0x5571648e5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.772+0000 7fa26ce42740 1 --2- 172.21.15.164:0/3703841451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5571648e6b40 0x5571648e5090 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.772+0000 7fa26ce42740 1 -- 172.21.15.164:0/3703841451 >> 172.21.15.164:0/3703841451 conn(0x5571648ae310 msgr2=0x5571648e1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:33.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.772+0000 7fa26ce42740 1 -- 172.21.15.164:0/3703841451 shutdown_connections 2024-09-19T08:46:33.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.772+0000 7fa26ce42740 1 -- 172.21.15.164:0/3703841451 wait complete. 2024-09-19T08:46:33.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.773+0000 7fa26ce42740 1 -- start start 2024-09-19T08:46:33.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.773+0000 7fa26ce42740 1 Processor -- start 2024-09-19T08:46:33.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.773+0000 7fa26ce42740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571648e55d0 0x5571648efd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:33.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.773+0000 7fa26ce42740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5571648e6b40 0x5571648ed060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:33.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.773+0000 7fa26ce42740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5571648f0370 0x5571648ed6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:33.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.773+0000 7fa26ce42740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5571648f3010 con 0x5571648e6b40 2024-09-19T08:46:33.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.773+0000 7fa26ce42740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x557164806060 con 0x5571648f0370 2024-09-19T08:46:33.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.773+0000 7fa26ce42740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5571648f3190 con 0x5571648e55d0 2024-09-19T08:46:33.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.773+0000 7fa263fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5571648e6b40 0x5571648ed060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:33.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.773+0000 7fa263fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5571648e6b40 0x5571648ed060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45382/0 (socket says 172.21.15.164:45382) 2024-09-19T08:46:33.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.773+0000 7fa263fff640 1 -- 172.21.15.164:0/4074489012 learned_addr learned my addr 172.21.15.164:0/4074489012 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:33.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.773+0000 7fa26ce42740 10 client.0 _ll_register_callbacks cb 0x5571645598c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:33.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.773+0000 7fa268805640 1 --2- 172.21.15.164:0/4074489012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571648e55d0 0x5571648efd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:33.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.774+0000 7fa269006640 1 --2- 172.21.15.164:0/4074489012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5571648f0370 0x5571648ed6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:33.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.774+0000 7fa263fff640 1 -- 172.21.15.164:0/4074489012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571648e55d0 msgr2=0x5571648efd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:33.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.774+0000 7fa263fff640 1 --2- 172.21.15.164:0/4074489012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571648e55d0 0x5571648efd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.774+0000 7fa263fff640 1 -- 172.21.15.164:0/4074489012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5571648f0370 msgr2=0x5571648ed6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:33.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.774+0000 7fa263fff640 1 --2- 172.21.15.164:0/4074489012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5571648f0370 0x5571648ed6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.774+0000 7fa263fff640 1 -- 172.21.15.164:0/4074489012 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571645bc3c0 con 0x5571648e6b40 2024-09-19T08:46:33.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.774+0000 7fa269006640 1 --2- 172.21.15.164:0/4074489012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5571648f0370 0x5571648ed6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:33.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.774+0000 7fa263fff640 1 --2- 172.21.15.164:0/4074489012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5571648e6b40 0x5571648ed060 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa25000b9a0 tx=0x7fa25000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee2d2723f45442f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:33.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.774+0000 7fa243fff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:46:33.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.774+0000 7fa243fff640 1 -- 172.21.15.164:0/4074489012 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa250003ea0 con 0x5571648e6b40 2024-09-19T08:46:33.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.774+0000 7fa243fff640 1 -- 172.21.15.164:0/4074489012 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fa250005220 con 0x5571648e6b40 2024-09-19T08:46:33.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.774+0000 7fa26ce42740 1 -- 172.21.15.164:0/4074489012 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa258004d30 con 0x5571648e6b40 2024-09-19T08:46:33.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.774+0000 7fa243fff640 1 -- 172.21.15.164:0/4074489012 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa250005420 con 0x5571648e6b40 2024-09-19T08:46:33.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.774+0000 7fa26ce42740 10 client.15585 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:33.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.774+0000 7fa26ce42740 1 -- 172.21.15.164:0/4074489012 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5571648feb30 con 0x5571648e6b40 2024-09-19T08:46:33.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.775+0000 7fa243fff640 1 -- 172.21.15.164:0/4074489012 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa258004d30 con 0x5571648e6b40 2024-09-19T08:46:33.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.775+0000 7fa243fff640 1 -- 172.21.15.164:0/4074489012 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa250022020 con 0x5571648e6b40 2024-09-19T08:46:33.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.775+0000 7fa26ce42740 10 client.15585 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:33.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.775+0000 7fa26ce42740 10 client.15585 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T08:46:33.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.775+0000 7fa26ce42740 20 client.15585 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:33.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.775+0000 7fa26ce42740 1 --2- 172.21.15.164:0/4074489012 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x5571648fff90 0x5571649023d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:33.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.775+0000 7fa26ce42740 4 client.15585 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:46:33.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.775+0000 7fa26ce42740 1 -- 172.21.15.164:0/4074489012 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557164858860 con 0x5571648fff90 2024-09-19T08:46:33.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.775+0000 7fa268805640 1 --2- 172.21.15.164:0/4074489012 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x5571648fff90 0x5571649023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:33.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.776+0000 7fa243fff640 10 client.15585 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:46:33.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.776+0000 7fa268805640 1 --2- 172.21.15.164:0/4074489012 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x5571648fff90 0x5571649023d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bd70368eae5dc1d8 server_cookie=89fd3177f3830dc6 in_seq=0 out_seq=0 2024-09-19T08:46:33.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.776+0000 7fa243fff640 1 -- 172.21.15.164:0/4074489012 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557164858860 con 0x5571648fff90 2024-09-19T08:46:33.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.776+0000 7fa243fff640 10 client.15585 handle_command_reply: tid=0 2024-09-19T08:46:33.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.776+0000 7fa26ce42740 1 client.15585 shutdown 2024-09-19T08:46:33.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.776+0000 7fa26ce42740 1 -- 172.21.15.164:0/4074489012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5571648e6b40 msgr2=0x5571648ed060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:33.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.777+0000 7fa26ce42740 1 --2- 172.21.15.164:0/4074489012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5571648e6b40 0x5571648ed060 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa25000b9a0 tx=0x7fa25000be60 comp rx=0 tx=0).stop 2024-09-19T08:46:33.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.777+0000 7fa26ce42740 1 -- 172.21.15.164:0/4074489012 shutdown_connections 2024-09-19T08:46:33.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.777+0000 7fa26ce42740 1 --2- 172.21.15.164:0/4074489012 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x5571648fff90 0x5571649023d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.777+0000 7fa243fff640 0 client.15585 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:46:33.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.777+0000 7fa26ce42740 1 --2- 172.21.15.164:0/4074489012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5571648f0370 0x5571648ed6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.777+0000 7fa26ce42740 1 --2- 172.21.15.164:0/4074489012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5571648e6b40 0x5571648ed060 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.777+0000 7fa26ce42740 1 --2- 172.21.15.164:0/4074489012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571648e55d0 0x5571648efd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.777+0000 7fa26ce42740 1 -- 172.21.15.164:0/4074489012 >> 172.21.15.164:0/4074489012 conn(0x5571648ae310 msgr2=0x5571648e7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:33.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.777+0000 7fa26ce42740 1 -- 172.21.15.164:0/4074489012 shutdown_connections 2024-09-19T08:46:33.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.777+0000 7fa26ce42740 1 -- 172.21.15.164:0/4074489012 wait complete. 2024-09-19T08:46:33.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.777+0000 7fa26ce42740 20 client.15585 trim_cache size 0 max 16384 2024-09-19T08:46:33.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.791+0000 7fa26ce42740 1 Processor -- start 2024-09-19T08:46:33.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.791+0000 7fa26ce42740 1 -- start start 2024-09-19T08:46:33.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.792+0000 7fa26ce42740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5571648e55d0 0x5571648fc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:33.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.792+0000 7fa26ce42740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5571648e6b40 0x5571648e8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:33.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.792+0000 7fa26ce42740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571648f0370 0x5571648e8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:33.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.792+0000 7fa26ce42740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa2641af8f0 con 0x5571648e6b40 2024-09-19T08:46:33.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.792+0000 7fa26ce42740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa2641b1430 con 0x5571648e55d0 2024-09-19T08:46:33.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.792+0000 7fa26ce42740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa2641affc0 con 0x5571648f0370 2024-09-19T08:46:33.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.792+0000 7fa263fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5571648e6b40 0x5571648e8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:33.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.792+0000 7fa263fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5571648e6b40 0x5571648e8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45390/0 (socket says 172.21.15.164:45390) 2024-09-19T08:46:33.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.792+0000 7fa263fff640 1 -- 172.21.15.164:0/1390581630 learned_addr learned my addr 172.21.15.164:0/1390581630 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:33.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.792+0000 7fa268805640 1 --2- 172.21.15.164:0/1390581630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5571648e55d0 0x5571648fc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:33.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.792+0000 7fa269006640 1 --2- 172.21.15.164:0/1390581630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571648f0370 0x5571648e8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:33.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.792+0000 7fa263fff640 1 -- 172.21.15.164:0/1390581630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571648f0370 msgr2=0x5571648e8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:33.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.792+0000 7fa263fff640 1 --2- 172.21.15.164:0/1390581630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571648f0370 0x5571648e8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.792+0000 7fa263fff640 1 -- 172.21.15.164:0/1390581630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5571648e55d0 msgr2=0x5571648fc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:33.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.792+0000 7fa263fff640 1 --2- 172.21.15.164:0/1390581630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5571648e55d0 0x5571648fc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.792+0000 7fa263fff640 1 -- 172.21.15.164:0/1390581630 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571648ee190 con 0x5571648e6b40 2024-09-19T08:46:33.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.792+0000 7fa269006640 1 --2- 172.21.15.164:0/1390581630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571648f0370 0x5571648e8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:33.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.792+0000 7fa268805640 1 --2- 172.21.15.164:0/1390581630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5571648e55d0 0x5571648fc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:46:33.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.792+0000 7fa263fff640 1 --2- 172.21.15.164:0/1390581630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5571648e6b40 0x5571648e8120 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa250004240 tx=0x7fa2500044a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0de21474b42885 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:33.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.793+0000 7fa24e7fc640 1 -- 172.21.15.164:0/1390581630 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa250000e60 con 0x5571648e6b40 2024-09-19T08:46:33.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.793+0000 7fa24e7fc640 1 -- 172.21.15.164:0/1390581630 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2500039c0 con 0x5571648e6b40 2024-09-19T08:46:33.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.793+0000 7fa24e7fc640 1 -- 172.21.15.164:0/1390581630 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25002a070 con 0x5571648e6b40 2024-09-19T08:46:33.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.794+0000 7fa26ce42740 1 -- 172.21.15.164:0/1390581630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5571648e6b40 msgr2=0x5571648e8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:33.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.794+0000 7fa26ce42740 1 --2- 172.21.15.164:0/1390581630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5571648e6b40 0x5571648e8120 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa250004240 tx=0x7fa2500044a0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.794+0000 7fa26ce42740 1 -- 172.21.15.164:0/1390581630 shutdown_connections 2024-09-19T08:46:33.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.794+0000 7fa26ce42740 1 --2- 172.21.15.164:0/1390581630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571648f0370 0x5571648e8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.794+0000 7fa26ce42740 1 --2- 172.21.15.164:0/1390581630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5571648e6b40 0x5571648e8120 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.794+0000 7fa26ce42740 1 --2- 172.21.15.164:0/1390581630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5571648e55d0 0x5571648fc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.794+0000 7fa26ce42740 1 -- 172.21.15.164:0/1390581630 >> 172.21.15.164:0/1390581630 conn(0x5571648ae310 msgr2=0x5571648ec150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:33.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.794+0000 7fa26ce42740 1 -- 172.21.15.164:0/1390581630 shutdown_connections 2024-09-19T08:46:33.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.794+0000 7fa26ce42740 1 -- 172.21.15.164:0/1390581630 wait complete. 2024-09-19T08:46:33.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.794+0000 7fa26ce42740 1 -- start start 2024-09-19T08:46:33.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.794+0000 7fa26ce42740 1 Processor -- start 2024-09-19T08:46:33.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.795+0000 7fa26ce42740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571648e55d0 0x5571648f89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:33.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.795+0000 7fa26ce42740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5571648e6b40 0x5571648f3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:33.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.795+0000 7fa26ce42740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5571648f0370 0x5571648f4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:33.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.795+0000 7fa26ce42740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa2641b0d60 con 0x5571648f0370 2024-09-19T08:46:33.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.795+0000 7fa26ce42740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa2641b0690 con 0x5571648e6b40 2024-09-19T08:46:33.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.795+0000 7fa26ce42740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa2641ad8d0 con 0x5571648e55d0 2024-09-19T08:46:33.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.795+0000 7fa26ce42740 10 client.0 _ll_register_callbacks cb 0x7fa250004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:33.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.795+0000 7fa269006640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5571648f0370 0x5571648f4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:33.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.795+0000 7fa268805640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571648e55d0 0x5571648f89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:33.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.795+0000 7fa269006640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5571648f0370 0x5571648f4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45396/0 (socket says 172.21.15.164:45396) 2024-09-19T08:46:33.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.795+0000 7fa269006640 1 -- 172.21.15.164:0/39924438 learned_addr learned my addr 172.21.15.164:0/39924438 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:33.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.795+0000 7fa263fff640 1 --2- 172.21.15.164:0/39924438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5571648e6b40 0x5571648f3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:33.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.795+0000 7fa269006640 1 -- 172.21.15.164:0/39924438 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571648e55d0 msgr2=0x5571648f89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:33.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.795+0000 7fa269006640 1 --2- 172.21.15.164:0/39924438 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571648e55d0 0x5571648f89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.796+0000 7fa269006640 1 -- 172.21.15.164:0/39924438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5571648e6b40 msgr2=0x5571648f3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:33.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.796+0000 7fa269006640 1 --2- 172.21.15.164:0/39924438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5571648e6b40 0x5571648f3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.796+0000 7fa269006640 1 -- 172.21.15.164:0/39924438 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571648f48c0 con 0x5571648f0370 2024-09-19T08:46:33.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.796+0000 7fa269006640 1 --2- 172.21.15.164:0/39924438 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5571648f0370 0x5571648f4250 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa25c09c6e0 tx=0x7fa25c098b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93eafed302560543 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:33.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.796+0000 7fa243fff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:46:33.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.796+0000 7fa243fff640 1 -- 172.21.15.164:0/39924438 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25c0ba060 con 0x5571648f0370 2024-09-19T08:46:33.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.796+0000 7fa26ce42740 1 -- 172.21.15.164:0/39924438 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2500039c0 con 0x5571648f0370 2024-09-19T08:46:33.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.796+0000 7fa243fff640 1 -- 172.21.15.164:0/39924438 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fa25c0a4f30 con 0x5571648f0370 2024-09-19T08:46:33.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.796+0000 7fa243fff640 1 -- 172.21.15.164:0/39924438 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa25c0b4030 con 0x5571648f0370 2024-09-19T08:46:33.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.796+0000 7fa26ce42740 10 client.15591 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:33.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.796+0000 7fa26ce42740 1 -- 172.21.15.164:0/39924438 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55716496aae0 con 0x5571648f0370 2024-09-19T08:46:33.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.796+0000 7fa243fff640 1 -- 172.21.15.164:0/39924438 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2500039c0 con 0x5571648f0370 2024-09-19T08:46:33.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.797+0000 7fa243fff640 1 -- 172.21.15.164:0/39924438 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa25c0cdf70 con 0x5571648f0370 2024-09-19T08:46:33.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.797+0000 7fa26ce42740 10 client.15591 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:33.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.797+0000 7fa26ce42740 10 client.15591 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T08:46:33.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.797+0000 7fa26ce42740 20 client.15591 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:33.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.797+0000 7fa26ce42740 1 --2- 172.21.15.164:0/39924438 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x5571648fff90 0x55716496d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:33.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.797+0000 7fa26ce42740 4 client.15591 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:46:33.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.797+0000 7fa26ce42740 1 -- 172.21.15.164:0/39924438 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557164852d70 con 0x5571648fff90 2024-09-19T08:46:33.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.797+0000 7fa268805640 1 --2- 172.21.15.164:0/39924438 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x5571648fff90 0x55716496d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:33.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.798+0000 7fa268805640 1 --2- 172.21.15.164:0/39924438 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x5571648fff90 0x55716496d1f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9e9b9319f54141c0 server_cookie=ec6a5318bf455338 in_seq=0 out_seq=0 2024-09-19T08:46:33.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.798+0000 7fa243fff640 10 client.15591 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:46:33.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.798+0000 7fa243fff640 1 -- 172.21.15.164:0/39924438 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557164852d70 con 0x5571648fff90 2024-09-19T08:46:33.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.798+0000 7fa243fff640 10 client.15591 handle_command_reply: tid=0 2024-09-19T08:46:33.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.798+0000 7fa26ce42740 1 client.15591 shutdown 2024-09-19T08:46:33.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.798+0000 7fa26ce42740 1 -- 172.21.15.164:0/39924438 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5571648f0370 msgr2=0x5571648f4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:33.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.798+0000 7fa26ce42740 1 --2- 172.21.15.164:0/39924438 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5571648f0370 0x5571648f4250 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa25c09c6e0 tx=0x7fa25c098b20 comp rx=0 tx=0).stop 2024-09-19T08:46:33.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.798+0000 7fa26ce42740 1 -- 172.21.15.164:0/39924438 shutdown_connections 2024-09-19T08:46:33.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.798+0000 7fa26ce42740 1 --2- 172.21.15.164:0/39924438 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x5571648fff90 0x55716496d1f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.798+0000 7fa26ce42740 1 --2- 172.21.15.164:0/39924438 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5571648f0370 0x5571648f4250 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.798+0000 7fa243fff640 0 client.15591 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:46:33.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.798+0000 7fa26ce42740 1 --2- 172.21.15.164:0/39924438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5571648e6b40 0x5571648f3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.798+0000 7fa26ce42740 1 --2- 172.21.15.164:0/39924438 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571648e55d0 0x5571648f89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.798+0000 7fa26ce42740 1 -- 172.21.15.164:0/39924438 >> 172.21.15.164:0/39924438 conn(0x5571648ae310 msgr2=0x5571648ee8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:33.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.798+0000 7fa26ce42740 1 -- 172.21.15.164:0/39924438 shutdown_connections 2024-09-19T08:46:33.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.798+0000 7fa26ce42740 1 -- 172.21.15.164:0/39924438 wait complete. 2024-09-19T08:46:33.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.799+0000 7fa26ce42740 20 client.15591 trim_cache size 0 max 16384 2024-09-19T08:46:33.804 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:46:33.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.799+0000 7fa26aa69640 1 -- 172.21.15.164:0/2258292379 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa2540a3550 msgr2=0x7fa2540a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:33.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.799+0000 7fa26aa69640 1 --2- 172.21.15.164:0/2258292379 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa2540a3550 0x7fa2540a5a10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa258005c40 tx=0x7fa25803a000 comp rx=0 tx=0).stop 2024-09-19T08:46:33.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.799+0000 7fa26aa69640 1 -- 172.21.15.164:0/2258292379 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa26410e430 msgr2=0x7fa26419ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:33.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.799+0000 7fa26aa69640 1 --2- 172.21.15.164:0/2258292379 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa26410e430 0x7fa26419ef20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa25c00eb40 tx=0x7fa25c00c2d0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.799+0000 7fa26aa69640 1 -- 172.21.15.164:0/2258292379 shutdown_connections 2024-09-19T08:46:33.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.799+0000 7fa26aa69640 1 --2- 172.21.15.164:0/2258292379 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa26410e430 0x7fa26419ef20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.799+0000 7fa26aa69640 1 --2- 172.21.15.164:0/2258292379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa264074ee0 0x7fa26419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.799+0000 7fa26aa69640 1 --2- 172.21.15.164:0/2258292379 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa2540a3550 0x7fa2540a5a10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.799+0000 7fa26aa69640 1 --2- 172.21.15.164:0/2258292379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2640745c0 0x7fa26419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.799+0000 7fa26aa69640 1 -- 172.21.15.164:0/2258292379 >> 172.21.15.164:0/2258292379 conn(0x7fa2640fd8b0 msgr2=0x7fa2640fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:33.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.800+0000 7fa26aa69640 1 -- 172.21.15.164:0/2258292379 shutdown_connections 2024-09-19T08:46:33.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:33.800+0000 7fa26aa69640 1 -- 172.21.15.164:0/2258292379 wait complete. 2024-09-19T08:46:34.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:33 smithi164 ceph-mon[30928]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:46:34.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:33 smithi170 ceph-mon[35072]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:46:34.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:33 smithi173 ceph-mon[35306]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:46:34.590 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T08:46:34.614 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:46:35.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.092+0000 7fcb35bf8640 1 -- 172.21.15.164:0/1086188513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb30108290 msgr2=0x7fcb3010ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:35.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.092+0000 7fcb35bf8640 1 --2- 172.21.15.164:0/1086188513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb30108290 0x7fcb3010ec30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009cd0 tx=0x7fcb2802a8e0 comp rx=0 tx=0).stop 2024-09-19T08:46:35.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.092+0000 7fcb35bf8640 1 -- 172.21.15.164:0/1086188513 shutdown_connections 2024-09-19T08:46:35.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.092+0000 7fcb35bf8640 1 --2- 172.21.15.164:0/1086188513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb30108290 0x7fcb3010ec30 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:35.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.092+0000 7fcb35bf8640 1 --2- 172.21.15.164:0/1086188513 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb30103740 0x7fcb30107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:35.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.092+0000 7fcb35bf8640 1 --2- 172.21.15.164:0/1086188513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcb30102d90 0x7fcb30103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:35.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.092+0000 7fcb35bf8640 1 -- 172.21.15.164:0/1086188513 >> 172.21.15.164:0/1086188513 conn(0x7fcb300fc7d0 msgr2=0x7fcb300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:35.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.092+0000 7fcb35bf8640 1 -- 172.21.15.164:0/1086188513 shutdown_connections 2024-09-19T08:46:35.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.092+0000 7fcb35bf8640 1 -- 172.21.15.164:0/1086188513 wait complete. 2024-09-19T08:46:35.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.093+0000 7fcb35bf8640 1 Processor -- start 2024-09-19T08:46:35.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.093+0000 7fcb35bf8640 1 -- start start 2024-09-19T08:46:35.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.093+0000 7fcb35bf8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb30102d90 0x7fcb301984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:35.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.093+0000 7fcb35bf8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcb30103740 0x7fcb301989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:35.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.093+0000 7fcb35bf8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb30108290 0x7fcb3019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:35.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.093+0000 7fcb35bf8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcb30112090 con 0x7fcb30102d90 2024-09-19T08:46:35.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.093+0000 7fcb35bf8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcb30111f10 con 0x7fcb30108290 2024-09-19T08:46:35.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.093+0000 7fcb35bf8640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcb30112210 con 0x7fcb30103740 2024-09-19T08:46:35.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.094+0000 7fcb2f7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb30102d90 0x7fcb301984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:35.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.094+0000 7fcb2f7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb30102d90 0x7fcb301984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45416/0 (socket says 172.21.15.164:45416) 2024-09-19T08:46:35.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.094+0000 7fcb2f7fe640 1 -- 172.21.15.164:0/3857475713 learned_addr learned my addr 172.21.15.164:0/3857475713 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:35.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.094+0000 7fcb2effd640 1 --2- 172.21.15.164:0/3857475713 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcb30103740 0x7fcb301989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:35.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.094+0000 7fcb2ffff640 1 --2- 172.21.15.164:0/3857475713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb30108290 0x7fcb3019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:35.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.094+0000 7fcb2f7fe640 1 -- 172.21.15.164:0/3857475713 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcb30103740 msgr2=0x7fcb301989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:35.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.094+0000 7fcb2f7fe640 1 --2- 172.21.15.164:0/3857475713 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcb30103740 0x7fcb301989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:35.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.094+0000 7fcb2f7fe640 1 -- 172.21.15.164:0/3857475713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb30108290 msgr2=0x7fcb3019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:35.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.094+0000 7fcb2f7fe640 1 --2- 172.21.15.164:0/3857475713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb30108290 0x7fcb3019cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:35.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.094+0000 7fcb2f7fe640 1 -- 172.21.15.164:0/3857475713 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb3019d460 con 0x7fcb30102d90 2024-09-19T08:46:35.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.094+0000 7fcb2f7fe640 1 --2- 172.21.15.164:0/3857475713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb30102d90 0x7fcb301984b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000d770 tx=0x7fcb2000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:35.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.094+0000 7fcb2cff9640 1 -- 172.21.15.164:0/3857475713 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb20017070 con 0x7fcb30102d90 2024-09-19T08:46:35.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.094+0000 7fcb35bf8640 1 -- 172.21.15.164:0/3857475713 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb3019d750 con 0x7fcb30102d90 2024-09-19T08:46:35.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.094+0000 7fcb2cff9640 1 -- 172.21.15.164:0/3857475713 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb20005500 con 0x7fcb30102d90 2024-09-19T08:46:35.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.094+0000 7fcb2cff9640 1 -- 172.21.15.164:0/3857475713 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb20004e00 con 0x7fcb30102d90 2024-09-19T08:46:35.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.094+0000 7fcb35bf8640 1 -- 172.21.15.164:0/3857475713 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb301a5030 con 0x7fcb30102d90 2024-09-19T08:46:35.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.096+0000 7fcb35bf8640 1 -- 172.21.15.164:0/3857475713 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb30103170 con 0x7fcb30102d90 2024-09-19T08:46:35.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.097+0000 7fcb2cff9640 1 -- 172.21.15.164:0/3857475713 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7fcb20004fa0 con 0x7fcb30102d90 2024-09-19T08:46:35.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.097+0000 7fcb2cff9640 1 --2- 172.21.15.164:0/3857475713 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fcb180a34d0 0x7fcb180a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:35.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.098+0000 7fcb2effd640 1 --2- 172.21.15.164:0/3857475713 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fcb180a34d0 0x7fcb180a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:35.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.098+0000 7fcb2cff9640 1 -- 172.21.15.164:0/3857475713 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fcb200cfec0 con 0x7fcb30102d90 2024-09-19T08:46:35.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.098+0000 7fcb2effd640 1 --2- 172.21.15.164:0/3857475713 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fcb180a34d0 0x7fcb180a5990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcb300fdc40 tx=0x7fcb2400c4a0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:35.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.105+0000 7fcb2cff9640 1 -- 172.21.15.164:0/3857475713 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcb2009c6c0 con 0x7fcb30102d90 2024-09-19T08:46:35.147 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:46:35.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.338+0000 7fcb35bf8640 1 -- 172.21.15.164:0/3857475713 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb301a5200 con 0x7fcb30102d90 2024-09-19T08:46:35.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.338+0000 7fcb2cff9640 1 -- 172.21.15.164:0/3857475713 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11984 (secure 0 0 0) 0x7fcb2009bfc0 con 0x7fcb30102d90 2024-09-19T08:46:35.338 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:46:35.338 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":28,"btime":"2024-09-19T08:45:46:903004+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:45:46.852789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:46:35.339 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 28 2024-09-19T08:46:35.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.341+0000 7fcb35bf8640 1 -- 172.21.15.164:0/3857475713 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fcb180a34d0 msgr2=0x7fcb180a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:35.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.341+0000 7fcb35bf8640 1 --2- 172.21.15.164:0/3857475713 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fcb180a34d0 0x7fcb180a5990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcb300fdc40 tx=0x7fcb2400c4a0 comp rx=0 tx=0).stop 2024-09-19T08:46:35.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.341+0000 7fcb35bf8640 1 -- 172.21.15.164:0/3857475713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb30102d90 msgr2=0x7fcb301984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:35.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.341+0000 7fcb35bf8640 1 --2- 172.21.15.164:0/3857475713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb30102d90 0x7fcb301984b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000d770 tx=0x7fcb2000dc30 comp rx=0 tx=0).stop 2024-09-19T08:46:35.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.341+0000 7fcb35bf8640 1 -- 172.21.15.164:0/3857475713 shutdown_connections 2024-09-19T08:46:35.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.341+0000 7fcb35bf8640 1 --2- 172.21.15.164:0/3857475713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb30108290 0x7fcb3019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:35.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.341+0000 7fcb35bf8640 1 --2- 172.21.15.164:0/3857475713 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcb30103740 0x7fcb301989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:35.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.341+0000 7fcb35bf8640 1 --2- 172.21.15.164:0/3857475713 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fcb180a34d0 0x7fcb180a5990 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:35.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.341+0000 7fcb35bf8640 1 --2- 172.21.15.164:0/3857475713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb30102d90 0x7fcb301984b0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:35.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.341+0000 7fcb35bf8640 1 -- 172.21.15.164:0/3857475713 >> 172.21.15.164:0/3857475713 conn(0x7fcb300fc7d0 msgr2=0x7fcb300fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:35.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.341+0000 7fcb35bf8640 1 -- 172.21.15.164:0/3857475713 shutdown_connections 2024-09-19T08:46:35.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:35.341+0000 7fcb35bf8640 1 -- 172.21.15.164:0/3857475713 wait complete. 2024-09-19T08:46:35.899 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:35 smithi164 ceph-mon[30928]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T08:46:35.900 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:35 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3857475713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:46:36.045 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 131.2 --await-for 45.0' 2024-09-19T08:46:36.046 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 131.2 --await-for 45.0 2024-09-19T08:46:36.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:35 smithi170 ceph-mon[35072]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T08:46:36.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:35 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3857475713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:46:36.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:35 smithi173 ceph-mon[35306]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T08:46:36.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:35 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3857475713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:46:36.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.257+0000 7ffbd50a1640 1 -- 172.21.15.164:0/3667995514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbd01082a0 msgr2=0x7ffbd010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:36.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.257+0000 7ffbd50a1640 1 --2- 172.21.15.164:0/3667995514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbd01082a0 0x7ffbd010ec40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8009cd0 tx=0x7ffbc802a8e0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.257+0000 7ffbd50a1640 1 -- 172.21.15.164:0/3667995514 shutdown_connections 2024-09-19T08:46:36.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.257+0000 7ffbd50a1640 1 --2- 172.21.15.164:0/3667995514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbd01082a0 0x7ffbd010ec40 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.257+0000 7ffbd50a1640 1 --2- 172.21.15.164:0/3667995514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbd01036e0 0x7ffbd0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.257+0000 7ffbd50a1640 1 --2- 172.21.15.164:0/3667995514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbd0102d30 0x7ffbd0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.257+0000 7ffbd50a1640 1 -- 172.21.15.164:0/3667995514 >> 172.21.15.164:0/3667995514 conn(0x7ffbd00fc7d0 msgr2=0x7ffbd00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:36.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.257+0000 7ffbd50a1640 1 -- 172.21.15.164:0/3667995514 shutdown_connections 2024-09-19T08:46:36.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.257+0000 7ffbd50a1640 1 -- 172.21.15.164:0/3667995514 wait complete. 2024-09-19T08:46:36.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.258+0000 7ffbd50a1640 1 Processor -- start 2024-09-19T08:46:36.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.258+0000 7ffbd50a1640 1 -- start start 2024-09-19T08:46:36.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.259+0000 7ffbd50a1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbd0102d30 0x7ffbd019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:36.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.259+0000 7ffbd50a1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbd01036e0 0x7ffbd019ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:36.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.259+0000 7ffbd50a1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbd01082a0 0x7ffbd019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:36.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.259+0000 7ffbd50a1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd01120a0 con 0x7ffbd01082a0 2024-09-19T08:46:36.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.259+0000 7ffbd50a1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd0111f20 con 0x7ffbd01036e0 2024-09-19T08:46:36.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.259+0000 7ffbd50a1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd0112220 con 0x7ffbd0102d30 2024-09-19T08:46:36.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.259+0000 7ffbcf59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbd01082a0 0x7ffbd019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:36.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.259+0000 7ffbcf59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbd01082a0 0x7ffbd019f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45444/0 (socket says 172.21.15.164:45444) 2024-09-19T08:46:36.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.259+0000 7ffbcf59e640 1 -- 172.21.15.164:0/1405948681 learned_addr learned my addr 172.21.15.164:0/1405948681 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:36.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.259+0000 7ffbce59c640 1 --2- 172.21.15.164:0/1405948681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbd01036e0 0x7ffbd019ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:36.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.259+0000 7ffbced9d640 1 --2- 172.21.15.164:0/1405948681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbd0102d30 0x7ffbd019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:36.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.259+0000 7ffbcf59e640 1 -- 172.21.15.164:0/1405948681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbd0102d30 msgr2=0x7ffbd019a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:36.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.259+0000 7ffbcf59e640 1 --2- 172.21.15.164:0/1405948681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbd0102d30 0x7ffbd019a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.259+0000 7ffbcf59e640 1 -- 172.21.15.164:0/1405948681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbd01036e0 msgr2=0x7ffbd019ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:36.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.259+0000 7ffbcf59e640 1 --2- 172.21.15.164:0/1405948681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbd01036e0 0x7ffbd019ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.259+0000 7ffbcf59e640 1 -- 172.21.15.164:0/1405948681 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd019f6f0 con 0x7ffbd01082a0 2024-09-19T08:46:36.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.259+0000 7ffbce59c640 1 --2- 172.21.15.164:0/1405948681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbd01036e0 0x7ffbd019ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:36.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.259+0000 7ffbcf59e640 1 --2- 172.21.15.164:0/1405948681 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbd01082a0 0x7ffbd019f010 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8005cc0 tx=0x7ffbc8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:36.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.260+0000 7ffbb7fff640 1 -- 172.21.15.164:0/1405948681 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8004040 con 0x7ffbd01082a0 2024-09-19T08:46:36.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.260+0000 7ffbd50a1640 1 -- 172.21.15.164:0/1405948681 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbd019f980 con 0x7ffbd01082a0 2024-09-19T08:46:36.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.260+0000 7ffbb7fff640 1 -- 172.21.15.164:0/1405948681 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbc80046c0 con 0x7ffbd01082a0 2024-09-19T08:46:36.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.260+0000 7ffbb7fff640 1 -- 172.21.15.164:0/1405948681 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8046480 con 0x7ffbd01082a0 2024-09-19T08:46:36.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.260+0000 7ffbd50a1640 1 -- 172.21.15.164:0/1405948681 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbd01a72c0 con 0x7ffbd01082a0 2024-09-19T08:46:36.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.263+0000 7ffbd747a740 1 Processor -- start 2024-09-19T08:46:36.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.263+0000 7ffbb7fff640 1 -- 172.21.15.164:0/1405948681 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7ffbc80305a0 con 0x7ffbd01082a0 2024-09-19T08:46:36.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.263+0000 7ffbd747a740 1 -- start start 2024-09-19T08:46:36.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.263+0000 7ffbd747a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560912e74b40 0x560912e73090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:36.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.263+0000 7ffbd747a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560912e735d0 0x560912e73a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:36.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.263+0000 7ffbd747a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560912e7e370 0x560912e80760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:36.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.264+0000 7ffbd747a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560912dd92f0 con 0x560912e74b40 2024-09-19T08:46:36.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.264+0000 7ffbd747a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560912d80cf0 con 0x560912e735d0 2024-09-19T08:46:36.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.264+0000 7ffbd747a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x560912dd52b0 con 0x560912e7e370 2024-09-19T08:46:36.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.264+0000 7ffbce59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560912e74b40 0x560912e73090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:36.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.264+0000 7ffbced9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560912e735d0 0x560912e73a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:36.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.264+0000 7ffbced9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560912e735d0 0x560912e73a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:51372/0 (socket says 172.21.15.164:51372) 2024-09-19T08:46:36.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.264+0000 7ffbced9d640 1 -- 172.21.15.164:0/1284824736 learned_addr learned my addr 172.21.15.164:0/1284824736 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:36.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.264+0000 7ffbce59c640 1 --2- 172.21.15.164:0/1284824736 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560912e7e370 0x560912e80760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:36.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.265+0000 7ffbb7fff640 1 --2- 172.21.15.164:0/1405948681 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ffbb80a3550 0x7ffbb80a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:36.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.265+0000 7ffbce59c640 1 -- 172.21.15.164:0/1284824736 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560912e7e370 msgr2=0x560912e80760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:36.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.265+0000 7ffbce59c640 1 --2- 172.21.15.164:0/1284824736 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560912e7e370 0x560912e80760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.265+0000 7ffbce59c640 1 -- 172.21.15.164:0/1284824736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560912e735d0 msgr2=0x560912e73a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:36.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.265+0000 7ffbce59c640 1 --2- 172.21.15.164:0/1284824736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560912e735d0 0x560912e73a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.265+0000 7ffbce59c640 1 -- 172.21.15.164:0/1284824736 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560912d653a0 con 0x560912e74b40 2024-09-19T08:46:36.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.265+0000 7ffbcf59e640 1 --2- 172.21.15.164:0/1405948681 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ffbb80a3550 0x7ffbb80a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:36.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.265+0000 7ffbb7fff640 1 -- 172.21.15.164:0/1405948681 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7ffbc802e030 con 0x7ffbd01082a0 2024-09-19T08:46:36.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.265+0000 7ffbce59c640 1 --2- 172.21.15.164:0/1284824736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560912e74b40 0x560912e73090 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0102710 tx=0x7ffbc400c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a83e6d249531fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:36.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.265+0000 7ffbcf59e640 1 --2- 172.21.15.164:0/1405948681 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ffbb80a3550 0x7ffbb80a5a10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80eede0 tx=0x7ffbc808f040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:36.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.265+0000 7ffbb57fa640 1 -- 172.21.15.164:0/1284824736 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc40142d0 con 0x560912e74b40 2024-09-19T08:46:36.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.265+0000 7ffbb57fa640 1 -- 172.21.15.164:0/1284824736 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbc400f730 con 0x560912e74b40 2024-09-19T08:46:36.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.266+0000 7ffbb57fa640 1 -- 172.21.15.164:0/1284824736 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc40037f0 con 0x560912e74b40 2024-09-19T08:46:36.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.266+0000 7ffbd747a740 1 -- 172.21.15.164:0/1284824736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560912e74b40 msgr2=0x560912e73090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:36.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.266+0000 7ffbd747a740 1 --2- 172.21.15.164:0/1284824736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560912e74b40 0x560912e73090 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0102710 tx=0x7ffbc400c470 comp rx=0 tx=0).stop 2024-09-19T08:46:36.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.266+0000 7ffbd747a740 1 -- 172.21.15.164:0/1284824736 shutdown_connections 2024-09-19T08:46:36.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.266+0000 7ffbd747a740 1 --2- 172.21.15.164:0/1284824736 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560912e7e370 0x560912e80760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.266+0000 7ffbd747a740 1 --2- 172.21.15.164:0/1284824736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560912e735d0 0x560912e73a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.266+0000 7ffbd747a740 1 --2- 172.21.15.164:0/1284824736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560912e74b40 0x560912e73090 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.266+0000 7ffbd747a740 1 -- 172.21.15.164:0/1284824736 >> 172.21.15.164:0/1284824736 conn(0x560912e3c310 msgr2=0x560912e6fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:36.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.266+0000 7ffbd747a740 1 -- 172.21.15.164:0/1284824736 shutdown_connections 2024-09-19T08:46:36.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.266+0000 7ffbd747a740 1 -- 172.21.15.164:0/1284824736 wait complete. 2024-09-19T08:46:36.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.267+0000 7ffbd747a740 1 -- start start 2024-09-19T08:46:36.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.267+0000 7ffbd747a740 1 Processor -- start 2024-09-19T08:46:36.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.267+0000 7ffbd747a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560912e735d0 0x560912e7dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:36.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.267+0000 7ffbd747a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560912e74b40 0x560912e7b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:36.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.267+0000 7ffbd747a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560912e7e370 0x560912e7b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:36.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.267+0000 7ffbd747a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560912e81010 con 0x560912e7e370 2024-09-19T08:46:36.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.267+0000 7ffbd747a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560912d94060 con 0x560912e735d0 2024-09-19T08:46:36.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.267+0000 7ffbd747a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x560912e81190 con 0x560912e74b40 2024-09-19T08:46:36.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.267+0000 7ffbd747a740 10 client.0 _ll_register_callbacks cb 0x560912ae78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:36.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.267+0000 7ffbce59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560912e7e370 0x560912e7b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:36.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.267+0000 7ffbce59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560912e7e370 0x560912e7b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45450/0 (socket says 172.21.15.164:45450) 2024-09-19T08:46:36.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.267+0000 7ffbce59c640 1 -- 172.21.15.164:0/2835089138 learned_addr learned my addr 172.21.15.164:0/2835089138 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:36.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.267+0000 7ffbce59c640 1 --2- 172.21.15.164:0/2835089138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560912e735d0 0x560912e7dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:36.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.267+0000 7ffbced9d640 1 --2- 172.21.15.164:0/2835089138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560912e74b40 0x560912e7b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:36.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.267+0000 7ffbce59c640 1 -- 172.21.15.164:0/2835089138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560912e74b40 msgr2=0x560912e7b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:36.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.267+0000 7ffbce59c640 1 --2- 172.21.15.164:0/2835089138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560912e74b40 0x560912e7b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.268+0000 7ffbce59c640 1 -- 172.21.15.164:0/2835089138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560912e735d0 msgr2=0x560912e7dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:36.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.268+0000 7ffbce59c640 1 --2- 172.21.15.164:0/2835089138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560912e735d0 0x560912e7dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.268+0000 7ffbce59c640 1 -- 172.21.15.164:0/2835089138 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560912b4a3c0 con 0x560912e7e370 2024-09-19T08:46:36.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.268+0000 7ffbce59c640 1 --2- 172.21.15.164:0/2835089138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560912e7e370 0x560912e7b640 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4014fd0 tx=0x7ffbc400c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b96564ce7a5a3c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:36.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.268+0000 7ffbb2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:46:36.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.268+0000 7ffbb2ff5640 1 -- 172.21.15.164:0/2835089138 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4005250 con 0x560912e7e370 2024-09-19T08:46:36.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.268+0000 7ffbd747a740 1 -- 172.21.15.164:0/2835089138 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbc400f730 con 0x560912e7e370 2024-09-19T08:46:36.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.268+0000 7ffbb2ff5640 1 -- 172.21.15.164:0/2835089138 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7ffbc40053f0 con 0x560912e7e370 2024-09-19T08:46:36.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.268+0000 7ffbb2ff5640 1 -- 172.21.15.164:0/2835089138 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffbc40055f0 con 0x560912e7e370 2024-09-19T08:46:36.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.268+0000 7ffbd747a740 10 client.15609 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:36.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.268+0000 7ffbd747a740 1 -- 172.21.15.164:0/2835089138 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560912e952f0 con 0x560912e7e370 2024-09-19T08:46:36.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.268+0000 7ffbb2ff5640 1 -- 172.21.15.164:0/2835089138 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbc400f730 con 0x560912e7e370 2024-09-19T08:46:36.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.268+0000 7ffbb2ff5640 1 -- 172.21.15.164:0/2835089138 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ffbc4021760 con 0x560912e7e370 2024-09-19T08:46:36.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.268+0000 7ffbd747a740 10 client.15609 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:36.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.269+0000 7ffbd747a740 10 client.15609 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T08:46:36.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.269+0000 7ffbd747a740 20 client.15609 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:36.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.269+0000 7ffbd747a740 1 --2- 172.21.15.164:0/2835089138 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x560912e966a0 0x560912e98a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:36.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.269+0000 7ffbd747a740 4 client.15609 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:46:36.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.269+0000 7ffbd747a740 1 -- 172.21.15.164:0/2835089138 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560912de6860 con 0x560912e966a0 2024-09-19T08:46:36.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.269+0000 7ffbced9d640 1 --2- 172.21.15.164:0/2835089138 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x560912e966a0 0x560912e98a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:36.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.270+0000 7ffbced9d640 1 --2- 172.21.15.164:0/2835089138 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x560912e966a0 0x560912e98a90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d7385a238c6da603 server_cookie=3c4d88cd1b6b750f in_seq=0 out_seq=0 2024-09-19T08:46:36.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.270+0000 7ffbb2ff5640 10 client.15609 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:46:36.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.271+0000 7ffbb2ff5640 1 -- 172.21.15.164:0/2835089138 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560912de6860 con 0x560912e966a0 2024-09-19T08:46:36.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.271+0000 7ffbb2ff5640 10 client.15609 handle_command_reply: tid=0 2024-09-19T08:46:36.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.271+0000 7ffbd747a740 1 client.15609 shutdown 2024-09-19T08:46:36.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.271+0000 7ffbd747a740 1 -- 172.21.15.164:0/2835089138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560912e7e370 msgr2=0x560912e7b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:36.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.271+0000 7ffbd747a740 1 --2- 172.21.15.164:0/2835089138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560912e7e370 0x560912e7b640 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4014fd0 tx=0x7ffbc400c3b0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.271+0000 7ffbd747a740 1 -- 172.21.15.164:0/2835089138 shutdown_connections 2024-09-19T08:46:36.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.271+0000 7ffbd747a740 1 --2- 172.21.15.164:0/2835089138 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x560912e966a0 0x560912e98a90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.271+0000 7ffbd747a740 1 --2- 172.21.15.164:0/2835089138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560912e7e370 0x560912e7b640 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.271+0000 7ffbd747a740 1 --2- 172.21.15.164:0/2835089138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560912e74b40 0x560912e7b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.271+0000 7ffbd747a740 1 --2- 172.21.15.164:0/2835089138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560912e735d0 0x560912e7dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.271+0000 7ffbd747a740 1 -- 172.21.15.164:0/2835089138 >> 172.21.15.164:0/2835089138 conn(0x560912e3c310 msgr2=0x560912e75990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:36.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.271+0000 7ffbb2ff5640 0 client.15609 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:46:36.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.271+0000 7ffbd747a740 1 -- 172.21.15.164:0/2835089138 shutdown_connections 2024-09-19T08:46:36.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.271+0000 7ffbd747a740 1 -- 172.21.15.164:0/2835089138 wait complete. 2024-09-19T08:46:36.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.272+0000 7ffbd747a740 20 client.15609 trim_cache size 0 max 16384 2024-09-19T08:46:36.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.293+0000 7ffbd747a740 1 Processor -- start 2024-09-19T08:46:36.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.293+0000 7ffbd747a740 1 -- start start 2024-09-19T08:46:36.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.294+0000 7ffbd747a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560912e735d0 0x560912e92b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:36.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.294+0000 7ffbd747a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560912e74b40 0x560912e76120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:36.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.294+0000 7ffbd747a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560912e7e370 0x560912e76660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:36.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.294+0000 7ffbd747a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd01ab4b0 con 0x560912e735d0 2024-09-19T08:46:36.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.294+0000 7ffbd747a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd01acff0 con 0x560912e7e370 2024-09-19T08:46:36.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.294+0000 7ffbd747a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd01abb80 con 0x560912e74b40 2024-09-19T08:46:36.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.294+0000 7ffbce59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560912e735d0 0x560912e92b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:36.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.294+0000 7ffbce59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560912e735d0 0x560912e92b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45452/0 (socket says 172.21.15.164:45452) 2024-09-19T08:46:36.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.294+0000 7ffbce59c640 1 -- 172.21.15.164:0/2654395432 learned_addr learned my addr 172.21.15.164:0/2654395432 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:36.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.294+0000 7ffbced9d640 1 --2- 172.21.15.164:0/2654395432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560912e74b40 0x560912e76120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:36.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.294+0000 7ffbced9d640 1 -- 172.21.15.164:0/2654395432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560912e7e370 msgr2=0x560912e76660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:36.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.294+0000 7ffbced9d640 1 --2- 172.21.15.164:0/2654395432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560912e7e370 0x560912e76660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.294+0000 7ffbced9d640 1 -- 172.21.15.164:0/2654395432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560912e735d0 msgr2=0x560912e92b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:36.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.294+0000 7ffbced9d640 1 --2- 172.21.15.164:0/2654395432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560912e735d0 0x560912e92b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.294+0000 7ffbced9d640 1 -- 172.21.15.164:0/2654395432 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560912e74750 con 0x560912e74b40 2024-09-19T08:46:36.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.294+0000 7ffbce59c640 1 --2- 172.21.15.164:0/2654395432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560912e735d0 0x560912e92b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:46:36.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.295+0000 7ffbced9d640 1 --2- 172.21.15.164:0/2654395432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560912e74b40 0x560912e76120 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x560912e7c670 tx=0x7ffbc000b410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1adfc6dcc772f01a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:36.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.296+0000 7ffbb4ff9640 1 -- 172.21.15.164:0/2654395432 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0005220 con 0x560912e74b40 2024-09-19T08:46:36.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.296+0000 7ffbb4ff9640 1 -- 172.21.15.164:0/2654395432 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbc0004c80 con 0x560912e74b40 2024-09-19T08:46:36.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.296+0000 7ffbb4ff9640 1 -- 172.21.15.164:0/2654395432 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc001d9b0 con 0x560912e74b40 2024-09-19T08:46:36.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.297+0000 7ffbd747a740 1 -- 172.21.15.164:0/2654395432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560912e74b40 msgr2=0x560912e76120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:36.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.297+0000 7ffbd747a740 1 --2- 172.21.15.164:0/2654395432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560912e74b40 0x560912e76120 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x560912e7c670 tx=0x7ffbc000b410 comp rx=0 tx=0).stop 2024-09-19T08:46:36.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.297+0000 7ffbd747a740 1 -- 172.21.15.164:0/2654395432 shutdown_connections 2024-09-19T08:46:36.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.297+0000 7ffbd747a740 1 --2- 172.21.15.164:0/2654395432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560912e7e370 0x560912e76660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.297+0000 7ffbd747a740 1 --2- 172.21.15.164:0/2654395432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560912e74b40 0x560912e76120 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.297+0000 7ffbd747a740 1 --2- 172.21.15.164:0/2654395432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560912e735d0 0x560912e92b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.297+0000 7ffbd747a740 1 -- 172.21.15.164:0/2654395432 >> 172.21.15.164:0/2654395432 conn(0x560912e3c310 msgr2=0x560912e7d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:36.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.297+0000 7ffbd747a740 1 -- 172.21.15.164:0/2654395432 shutdown_connections 2024-09-19T08:46:36.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.297+0000 7ffbd747a740 1 -- 172.21.15.164:0/2654395432 wait complete. 2024-09-19T08:46:36.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.298+0000 7ffbd747a740 1 -- start start 2024-09-19T08:46:36.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.298+0000 7ffbd747a740 1 Processor -- start 2024-09-19T08:46:36.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.298+0000 7ffbd747a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560912e735d0 0x560912e8f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:36.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.298+0000 7ffbd747a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560912e74b40 0x560912e8a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:36.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.299+0000 7ffbd747a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560912e7e370 0x560912e8b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:36.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.299+0000 7ffbd747a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd01ac920 con 0x560912e735d0 2024-09-19T08:46:36.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.299+0000 7ffbce59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560912e735d0 0x560912e8f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:36.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.299+0000 7ffbce59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560912e735d0 0x560912e8f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45456/0 (socket says 172.21.15.164:45456) 2024-09-19T08:46:36.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.299+0000 7ffbce59c640 1 -- 172.21.15.164:0/3306478989 learned_addr learned my addr 172.21.15.164:0/3306478989 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:36.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.299+0000 7ffbd747a740 1 -- 172.21.15.164:0/3306478989 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd01ac250 con 0x560912e7e370 2024-09-19T08:46:36.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.299+0000 7ffbd747a740 1 -- 172.21.15.164:0/3306478989 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd01a9510 con 0x560912e74b40 2024-09-19T08:46:36.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.299+0000 7ffbced9d640 1 --2- 172.21.15.164:0/3306478989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560912e74b40 0x560912e8a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:36.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.299+0000 7ffbce59c640 1 -- 172.21.15.164:0/3306478989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560912e74b40 msgr2=0x560912e8a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:36.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.299+0000 7ffbce59c640 1 --2- 172.21.15.164:0/3306478989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560912e74b40 0x560912e8a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.299+0000 7ffbce59c640 1 -- 172.21.15.164:0/3306478989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560912e7e370 msgr2=0x560912e8b040 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:46:36.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.299+0000 7ffbce59c640 1 --2- 172.21.15.164:0/3306478989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560912e7e370 0x560912e8b040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.299+0000 7ffbce59c640 1 -- 172.21.15.164:0/3306478989 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560912f00030 con 0x560912e735d0 2024-09-19T08:46:36.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.299+0000 7ffbd747a740 10 client.0 _ll_register_callbacks cb 0x7ffbc000b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:36.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.299+0000 7ffbd747a740 1 -- 172.21.15.164:0/3306478989 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbc0004c80 con 0x560912e735d0 2024-09-19T08:46:36.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.299+0000 7ffbce59c640 1 --2- 172.21.15.164:0/3306478989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560912e735d0 0x560912e8f580 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4006010 tx=0x7ffbc4005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef51b9f800061da server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:36.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.299+0000 7ffbb2ff5640 10 client.15615 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:46:36.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.300+0000 7ffbb2ff5640 1 -- 172.21.15.164:0/3306478989 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc403d060 con 0x560912e735d0 2024-09-19T08:46:36.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.300+0000 7ffbb2ff5640 1 -- 172.21.15.164:0/3306478989 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7ffbc4021d20 con 0x560912e735d0 2024-09-19T08:46:36.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.300+0000 7ffbb2ff5640 1 -- 172.21.15.164:0/3306478989 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffbc4027040 con 0x560912e735d0 2024-09-19T08:46:36.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.300+0000 7ffbd747a740 10 client.15615 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:36.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.300+0000 7ffbd747a740 1 -- 172.21.15.164:0/3306478989 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({fsmap=28}) -- 0x560912f01050 con 0x560912e735d0 2024-09-19T08:46:36.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.300+0000 7ffbb2ff5640 1 -- 172.21.15.164:0/3306478989 <== mon.0 v2:172.21.15.164:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ffbc4021670 con 0x560912e735d0 2024-09-19T08:46:36.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.300+0000 7ffbd747a740 10 client.15615 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:36.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.300+0000 7ffbd747a740 10 client.15615 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T08:46:36.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.300+0000 7ffbd747a740 20 client.15615 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:36.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.300+0000 7ffbd747a740 1 --2- 172.21.15.164:0/3306478989 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x560912f01680 0x560912f03aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:36.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.301+0000 7ffbd747a740 4 client.15615 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:46:36.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.301+0000 7ffbd747a740 1 -- 172.21.15.164:0/3306478989 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560912de0d70 con 0x560912f01680 2024-09-19T08:46:36.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.301+0000 7ffbced9d640 1 --2- 172.21.15.164:0/3306478989 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x560912f01680 0x560912f03aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:36.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.301+0000 7ffbb2ff5640 10 client.15615 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:46:36.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.301+0000 7ffbced9d640 1 --2- 172.21.15.164:0/3306478989 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x560912f01680 0x560912f03aa0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=773f5ff5666e2007 server_cookie=9ed1e3e8d04e8ace in_seq=0 out_seq=0 2024-09-19T08:46:36.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.302+0000 7ffbb2ff5640 1 -- 172.21.15.164:0/3306478989 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560912de0d70 con 0x560912f01680 2024-09-19T08:46:36.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.302+0000 7ffbb2ff5640 10 client.15615 handle_command_reply: tid=0 2024-09-19T08:46:36.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.302+0000 7ffbd747a740 1 client.15615 shutdown 2024-09-19T08:46:36.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.302+0000 7ffbd747a740 1 -- 172.21.15.164:0/3306478989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560912e735d0 msgr2=0x560912e8f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:36.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.302+0000 7ffbd747a740 1 --2- 172.21.15.164:0/3306478989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560912e735d0 0x560912e8f580 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4006010 tx=0x7ffbc4005850 comp rx=0 tx=0).stop 2024-09-19T08:46:36.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.302+0000 7ffbd747a740 1 -- 172.21.15.164:0/3306478989 shutdown_connections 2024-09-19T08:46:36.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.302+0000 7ffbd747a740 1 --2- 172.21.15.164:0/3306478989 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x560912f01680 0x560912f03aa0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.302+0000 7ffbd747a740 1 --2- 172.21.15.164:0/3306478989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560912e7e370 0x560912e8b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.302+0000 7ffbd747a740 1 --2- 172.21.15.164:0/3306478989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560912e74b40 0x560912e8a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.302+0000 7ffbd747a740 1 --2- 172.21.15.164:0/3306478989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560912e735d0 0x560912e8f580 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.302+0000 7ffbb2ff5640 0 client.15615 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:46:36.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.302+0000 7ffbd747a740 1 -- 172.21.15.164:0/3306478989 >> 172.21.15.164:0/3306478989 conn(0x560912e3c310 msgr2=0x560912e797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:36.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.302+0000 7ffbd747a740 1 -- 172.21.15.164:0/3306478989 shutdown_connections 2024-09-19T08:46:36.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.302+0000 7ffbd747a740 1 -- 172.21.15.164:0/3306478989 wait complete. 2024-09-19T08:46:36.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.302+0000 7ffbd747a740 20 client.15615 trim_cache size 0 max 16384 2024-09-19T08:46:36.304 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:46:36.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.303+0000 7ffbd50a1640 1 -- 172.21.15.164:0/1405948681 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ffbb80a3550 msgr2=0x7ffbb80a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:36.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.303+0000 7ffbd50a1640 1 --2- 172.21.15.164:0/1405948681 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ffbb80a3550 0x7ffbb80a5a10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80eede0 tx=0x7ffbc808f040 comp rx=0 tx=0).stop 2024-09-19T08:46:36.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.303+0000 7ffbd50a1640 1 -- 172.21.15.164:0/1405948681 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbd01082a0 msgr2=0x7ffbd019f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:36.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.303+0000 7ffbd50a1640 1 --2- 172.21.15.164:0/1405948681 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbd01082a0 0x7ffbd019f010 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8005cc0 tx=0x7ffbc8003970 comp rx=0 tx=0).stop 2024-09-19T08:46:36.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.303+0000 7ffbd50a1640 1 -- 172.21.15.164:0/1405948681 shutdown_connections 2024-09-19T08:46:36.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.303+0000 7ffbd50a1640 1 --2- 172.21.15.164:0/1405948681 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ffbb80a3550 0x7ffbb80a5a10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.303+0000 7ffbd50a1640 1 --2- 172.21.15.164:0/1405948681 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbd01082a0 0x7ffbd019f010 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.303+0000 7ffbd50a1640 1 --2- 172.21.15.164:0/1405948681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbd01036e0 0x7ffbd019ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.303+0000 7ffbd50a1640 1 --2- 172.21.15.164:0/1405948681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbd0102d30 0x7ffbd019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.303+0000 7ffbd50a1640 1 -- 172.21.15.164:0/1405948681 >> 172.21.15.164:0/1405948681 conn(0x7ffbd00fc7d0 msgr2=0x7ffbd00fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:36.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.303+0000 7ffbd50a1640 1 -- 172.21.15.164:0/1405948681 shutdown_connections 2024-09-19T08:46:36.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:36.304+0000 7ffbd50a1640 1 -- 172.21.15.164:0/1405948681 wait complete. 2024-09-19T08:46:37.119 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T08:46:37.182 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:46:37.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.740+0000 7f1b39a41640 1 -- 172.21.15.164:0/943553154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b34102ef0 msgr2=0x7f1b3410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:37.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.740+0000 7f1b39a41640 1 --2- 172.21.15.164:0/943553154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b34102ef0 0x7f1b3410da40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c009ca0 tx=0x7f1b2c02a950 comp rx=0 tx=0).stop 2024-09-19T08:46:37.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.740+0000 7f1b39a41640 1 -- 172.21.15.164:0/943553154 shutdown_connections 2024-09-19T08:46:37.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.740+0000 7f1b39a41640 1 --2- 172.21.15.164:0/943553154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b34102ef0 0x7f1b3410da40 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.740+0000 7f1b39a41640 1 --2- 172.21.15.164:0/943553154 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b34102570 0x7f1b341029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.740+0000 7f1b39a41640 1 --2- 172.21.15.164:0/943553154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b34100e20 0x7f1b34101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.740+0000 7f1b39a41640 1 -- 172.21.15.164:0/943553154 >> 172.21.15.164:0/943553154 conn(0x7f1b340fc7d0 msgr2=0x7f1b340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:37.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.740+0000 7f1b39a41640 1 -- 172.21.15.164:0/943553154 shutdown_connections 2024-09-19T08:46:37.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.740+0000 7f1b39a41640 1 -- 172.21.15.164:0/943553154 wait complete. 2024-09-19T08:46:37.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.741+0000 7f1b39a41640 1 Processor -- start 2024-09-19T08:46:37.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.741+0000 7f1b39a41640 1 -- start start 2024-09-19T08:46:37.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.741+0000 7f1b39a41640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b34100e20 0x7f1b341973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.741+0000 7f1b39a41640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b34102570 0x7f1b34197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.741+0000 7f1b39a41640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b34102ef0 0x7f1b3419bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.741+0000 7f1b39a41640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1b34110fa0 con 0x7f1b34102ef0 2024-09-19T08:46:37.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.741+0000 7f1b39a41640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1b34110e20 con 0x7f1b34102570 2024-09-19T08:46:37.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.741+0000 7f1b39a41640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1b34111120 con 0x7f1b34100e20 2024-09-19T08:46:37.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.741+0000 7f1b337fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b34102ef0 0x7f1b3419bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:37.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.741+0000 7f1b337fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b34102ef0 0x7f1b3419bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45476/0 (socket says 172.21.15.164:45476) 2024-09-19T08:46:37.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.741+0000 7f1b337fe640 1 -- 172.21.15.164:0/2391095530 learned_addr learned my addr 172.21.15.164:0/2391095530 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:37.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.741+0000 7f1b327fc640 1 --2- 172.21.15.164:0/2391095530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b34102570 0x7f1b34197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:37.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.741+0000 7f1b32ffd640 1 --2- 172.21.15.164:0/2391095530 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b34100e20 0x7f1b341973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:37.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.742+0000 7f1b337fe640 1 -- 172.21.15.164:0/2391095530 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b34100e20 msgr2=0x7f1b341973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:37.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.742+0000 7f1b337fe640 1 --2- 172.21.15.164:0/2391095530 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b34100e20 0x7f1b341973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.742+0000 7f1b337fe640 1 -- 172.21.15.164:0/2391095530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b34102570 msgr2=0x7f1b34197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:37.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.742+0000 7f1b337fe640 1 --2- 172.21.15.164:0/2391095530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b34102570 0x7f1b34197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.742+0000 7f1b337fe640 1 -- 172.21.15.164:0/2391095530 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3419c390 con 0x7f1b34102ef0 2024-09-19T08:46:37.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.742+0000 7f1b337fe640 1 --2- 172.21.15.164:0/2391095530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b34102ef0 0x7f1b3419bcb0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c006010 tx=0x7f1b2c02f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:37.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.742+0000 7f1b38a3f640 1 -- 172.21.15.164:0/2391095530 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c034040 con 0x7f1b34102ef0 2024-09-19T08:46:37.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.742+0000 7f1b39a41640 1 -- 172.21.15.164:0/2391095530 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b3419c620 con 0x7f1b34102ef0 2024-09-19T08:46:37.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.742+0000 7f1b38a3f640 1 -- 172.21.15.164:0/2391095530 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b2c003e00 con 0x7f1b34102ef0 2024-09-19T08:46:37.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.742+0000 7f1b38a3f640 1 -- 172.21.15.164:0/2391095530 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c045480 con 0x7f1b34102ef0 2024-09-19T08:46:37.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.742+0000 7f1b39a41640 1 -- 172.21.15.164:0/2391095530 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b341a3f60 con 0x7f1b34102ef0 2024-09-19T08:46:37.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.744+0000 7f1b38a3f640 1 -- 172.21.15.164:0/2391095530 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7f1b2c037020 con 0x7f1b34102ef0 2024-09-19T08:46:37.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.745+0000 7f1b3be1a740 1 Processor -- start 2024-09-19T08:46:37.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.745+0000 7f1b3be1a740 1 -- start start 2024-09-19T08:46:37.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.745+0000 7f1b3be1a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e076ea8fc0 0x55e076ea7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.745+0000 7f1b3be1a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e076ea7d90 0x55e076ea81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.745+0000 7f1b3be1a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e076eb4a00 0x55e076eb6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.745+0000 7f1b3be1a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e076e0b2a0 con 0x55e076ea8fc0 2024-09-19T08:46:37.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.745+0000 7f1b3be1a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e076db2b10 con 0x55e076ea7d90 2024-09-19T08:46:37.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.745+0000 7f1b3be1a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e076e07260 con 0x55e076eb4a00 2024-09-19T08:46:37.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.745+0000 7f1b38a3f640 1 --2- 172.21.15.164:0/2391095530 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f1b1c0a3550 0x7f1b1c0a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.745+0000 7f1b327fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e076ea8fc0 0x55e076ea7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:37.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.745+0000 7f1b327fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e076ea8fc0 0x55e076ea7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45492/0 (socket says 172.21.15.164:45492) 2024-09-19T08:46:37.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.745+0000 7f1b327fc640 1 -- 172.21.15.164:0/3070858521 learned_addr learned my addr 172.21.15.164:0/3070858521 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:37.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.745+0000 7f1b32ffd640 1 --2- 172.21.15.164:0/3070858521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e076ea7d90 0x55e076ea81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:37.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.745+0000 7f1b337fe640 1 --2- 172.21.15.164:0/2391095530 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f1b1c0a3550 0x7f1b1c0a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:37.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.745+0000 7f1b327fc640 1 -- 172.21.15.164:0/3070858521 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e076eb4a00 msgr2=0x55e076eb6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:37.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.745+0000 7f1b327fc640 1 --2- 172.21.15.164:0/3070858521 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e076eb4a00 0x55e076eb6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.745+0000 7f1b327fc640 1 -- 172.21.15.164:0/3070858521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e076ea7d90 msgr2=0x55e076ea81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:37.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.745+0000 7f1b327fc640 1 --2- 172.21.15.164:0/3070858521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e076ea7d90 0x55e076ea81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.745+0000 7f1b327fc640 1 -- 172.21.15.164:0/3070858521 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e076b7cb90 con 0x55e076ea8fc0 2024-09-19T08:46:37.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.745+0000 7f1b32ffd640 1 --2- 172.21.15.164:0/3070858521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e076ea7d90 0x55e076ea81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:37.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.745+0000 7f1b38a3f640 1 -- 172.21.15.164:0/2391095530 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f1b2c0f0e20 con 0x7f1b34102ef0 2024-09-19T08:46:37.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.745+0000 7f1b327fc640 1 --2- 172.21.15.164:0/3070858521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e076ea8fc0 0x55e076ea7850 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1b28009b00 tx=0x7f1b2800e1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be15667965a29bca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:37.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.745+0000 7f1b337fe640 1 --2- 172.21.15.164:0/2391095530 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f1b1c0a3550 0x7f1b1c0a5a10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c0ee5d0 tx=0x7f1b2c004990 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:37.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.745+0000 7f1b19ffb640 1 -- 172.21.15.164:0/3070858521 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b28017070 con 0x55e076ea8fc0 2024-09-19T08:46:37.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.745+0000 7f1b19ffb640 1 -- 172.21.15.164:0/3070858521 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b2800ebb0 con 0x55e076ea8fc0 2024-09-19T08:46:37.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.745+0000 7f1b19ffb640 1 -- 172.21.15.164:0/3070858521 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b28012630 con 0x55e076ea8fc0 2024-09-19T08:46:37.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.746+0000 7f1b3be1a740 1 -- 172.21.15.164:0/3070858521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e076ea8fc0 msgr2=0x55e076ea7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:37.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.746+0000 7f1b3be1a740 1 --2- 172.21.15.164:0/3070858521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e076ea8fc0 0x55e076ea7850 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1b28009b00 tx=0x7f1b2800e1d0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.746+0000 7f1b3be1a740 1 -- 172.21.15.164:0/3070858521 shutdown_connections 2024-09-19T08:46:37.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.746+0000 7f1b3be1a740 1 --2- 172.21.15.164:0/3070858521 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e076eb4a00 0x55e076eb6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.746+0000 7f1b3be1a740 1 --2- 172.21.15.164:0/3070858521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e076ea7d90 0x55e076ea81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.746+0000 7f1b3be1a740 1 --2- 172.21.15.164:0/3070858521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e076ea8fc0 0x55e076ea7850 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.746+0000 7f1b3be1a740 1 -- 172.21.15.164:0/3070858521 >> 172.21.15.164:0/3070858521 conn(0x55e076ea3f00 msgr2=0x55e076ea42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:37.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.746+0000 7f1b3be1a740 1 -- 172.21.15.164:0/3070858521 shutdown_connections 2024-09-19T08:46:37.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.746+0000 7f1b3be1a740 1 -- 172.21.15.164:0/3070858521 wait complete. 2024-09-19T08:46:37.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.746+0000 7f1b3be1a740 1 -- start start 2024-09-19T08:46:37.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.747+0000 7f1b3be1a740 1 Processor -- start 2024-09-19T08:46:37.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.747+0000 7f1b3be1a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e076ea7d90 0x55e076eb1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.747+0000 7f1b3be1a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e076ea8fc0 0x55e076eb1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.747+0000 7f1b3be1a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e076eb4a00 0x55e076eb2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.747+0000 7f1b3be1a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e076eb7910 con 0x55e076ea8fc0 2024-09-19T08:46:37.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.747+0000 7f1b3be1a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e076d936e0 con 0x55e076ea7d90 2024-09-19T08:46:37.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.747+0000 7f1b3be1a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e076eb7a90 con 0x55e076eb4a00 2024-09-19T08:46:37.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.747+0000 7f1b3be1a740 10 client.0 _ll_register_callbacks cb 0x55e076db7a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:37.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.747+0000 7f1b32ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e076ea8fc0 0x55e076eb1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:37.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.747+0000 7f1b32ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e076ea8fc0 0x55e076eb1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45506/0 (socket says 172.21.15.164:45506) 2024-09-19T08:46:37.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.747+0000 7f1b32ffd640 1 -- 172.21.15.164:0/2380065464 learned_addr learned my addr 172.21.15.164:0/2380065464 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:37.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.747+0000 7f1b327fc640 1 --2- 172.21.15.164:0/2380065464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e076ea7d90 0x55e076eb1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:37.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.747+0000 7f1b327fc640 1 --2- 172.21.15.164:0/2380065464 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e076eb4a00 0x55e076eb2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:37.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.747+0000 7f1b32ffd640 1 -- 172.21.15.164:0/2380065464 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e076eb4a00 msgr2=0x55e076eb2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:37.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.748+0000 7f1b32ffd640 1 --2- 172.21.15.164:0/2380065464 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e076eb4a00 0x55e076eb2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.748+0000 7f1b32ffd640 1 -- 172.21.15.164:0/2380065464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e076ea7d90 msgr2=0x55e076eb1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:37.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.748+0000 7f1b32ffd640 1 --2- 172.21.15.164:0/2380065464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e076ea7d90 0x55e076eb1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.748+0000 7f1b32ffd640 1 -- 172.21.15.164:0/2380065464 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e076ec0f60 con 0x55e076ea8fc0 2024-09-19T08:46:37.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.748+0000 7f1b32ffd640 1 --2- 172.21.15.164:0/2380065464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e076ea8fc0 0x55e076eb1f40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1b240127f0 tx=0x7f1b24012cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2cc90dc3d5bfa8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:37.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.748+0000 7f1b12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:46:37.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.748+0000 7f1b12ffd640 1 -- 172.21.15.164:0/2380065464 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24010d60 con 0x55e076ea8fc0 2024-09-19T08:46:37.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.748+0000 7f1b3be1a740 1 -- 172.21.15.164:0/2380065464 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b2800ebb0 con 0x55e076ea8fc0 2024-09-19T08:46:37.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.748+0000 7f1b12ffd640 1 -- 172.21.15.164:0/2380065464 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f1b24005020 con 0x55e076ea8fc0 2024-09-19T08:46:37.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.748+0000 7f1b12ffd640 1 -- 172.21.15.164:0/2380065464 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1b24005220 con 0x55e076ea8fc0 2024-09-19T08:46:37.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.748+0000 7f1b3be1a740 10 client.15627 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:37.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.748+0000 7f1b3be1a740 1 -- 172.21.15.164:0/2380065464 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e076ec39a0 con 0x55e076ea8fc0 2024-09-19T08:46:37.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.748+0000 7f1b12ffd640 1 -- 172.21.15.164:0/2380065464 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b2800ebb0 con 0x55e076ea8fc0 2024-09-19T08:46:37.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.748+0000 7f1b12ffd640 1 -- 172.21.15.164:0/2380065464 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1b24022030 con 0x55e076ea8fc0 2024-09-19T08:46:37.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.748+0000 7f1b3be1a740 10 client.15627 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:37.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.749+0000 7f1b3be1a740 10 client.15627 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:46:37.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.749+0000 7f1b3be1a740 20 client.15627 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:37.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.749+0000 7f1b3be1a740 1 --2- 172.21.15.164:0/2380065464 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55e076ec4db0 0x55e076ec71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.749+0000 7f1b3be1a740 4 client.15627 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:46:37.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.749+0000 7f1b3be1a740 1 -- 172.21.15.164:0/2380065464 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e076e189d0 con 0x55e076ec4db0 2024-09-19T08:46:37.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.749+0000 7f1b327fc640 1 --2- 172.21.15.164:0/2380065464 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55e076ec4db0 0x55e076ec71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:37.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.749+0000 7f1b327fc640 1 --2- 172.21.15.164:0/2380065464 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55e076ec4db0 0x55e076ec71f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0aae0fee28afda7 server_cookie=298cc0a409444953 in_seq=0 out_seq=0 2024-09-19T08:46:37.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.749+0000 7f1b12ffd640 10 client.15627 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:46:37.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.750+0000 7f1b12ffd640 1 -- 172.21.15.164:0/2380065464 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e076e189d0 con 0x55e076ec4db0 2024-09-19T08:46:37.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.750+0000 7f1b12ffd640 10 client.15627 handle_command_reply: tid=0 2024-09-19T08:46:37.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.750+0000 7f1b3be1a740 1 client.15627 shutdown 2024-09-19T08:46:37.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.750+0000 7f1b3be1a740 1 -- 172.21.15.164:0/2380065464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e076ea8fc0 msgr2=0x55e076eb1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:37.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.750+0000 7f1b3be1a740 1 --2- 172.21.15.164:0/2380065464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e076ea8fc0 0x55e076eb1f40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1b240127f0 tx=0x7f1b24012cb0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.750+0000 7f1b3be1a740 1 -- 172.21.15.164:0/2380065464 shutdown_connections 2024-09-19T08:46:37.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.750+0000 7f1b3be1a740 1 --2- 172.21.15.164:0/2380065464 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e076eb4a00 0x55e076eb2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.750+0000 7f1b3be1a740 1 --2- 172.21.15.164:0/2380065464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e076ea8fc0 0x55e076eb1f40 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.750+0000 7f1b3be1a740 1 --2- 172.21.15.164:0/2380065464 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55e076ec4db0 0x55e076ec71f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.750+0000 7f1b3be1a740 1 --2- 172.21.15.164:0/2380065464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e076ea7d90 0x55e076eb1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.750+0000 7f1b3be1a740 1 -- 172.21.15.164:0/2380065464 >> 172.21.15.164:0/2380065464 conn(0x55e076ea3f00 msgr2=0x55e076ea9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:37.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.750+0000 7f1b12ffd640 0 client.15627 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:46:37.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.750+0000 7f1b3be1a740 1 -- 172.21.15.164:0/2380065464 shutdown_connections 2024-09-19T08:46:37.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.750+0000 7f1b3be1a740 1 -- 172.21.15.164:0/2380065464 wait complete. 2024-09-19T08:46:37.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.751+0000 7f1b3be1a740 20 client.15627 trim_cache size 0 max 16384 2024-09-19T08:46:37.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.769+0000 7f1b3be1a740 1 Processor -- start 2024-09-19T08:46:37.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.769+0000 7f1b3be1a740 1 -- start start 2024-09-19T08:46:37.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.770+0000 7f1b3be1a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e076ea7d90 0x55e076eccac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.770+0000 7f1b3be1a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e076ea8fc0 0x55e076ecd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.770+0000 7f1b3be1a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e076eb4a00 0x55e076ea2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.770+0000 7f1b3be1a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1b341ab4a0 con 0x55e076ea8fc0 2024-09-19T08:46:37.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.770+0000 7f1b3be1a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1b341acfe0 con 0x55e076eb4a00 2024-09-19T08:46:37.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.770+0000 7f1b3be1a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1b341abb70 con 0x55e076ea7d90 2024-09-19T08:46:37.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.770+0000 7f1b32ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e076ea8fc0 0x55e076ecd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:37.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.770+0000 7f1b32ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e076ea8fc0 0x55e076ecd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45520/0 (socket says 172.21.15.164:45520) 2024-09-19T08:46:37.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.770+0000 7f1b32ffd640 1 -- 172.21.15.164:0/3133893143 learned_addr learned my addr 172.21.15.164:0/3133893143 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:37.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.770+0000 7f1b327fc640 1 --2- 172.21.15.164:0/3133893143 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e076ea7d90 0x55e076eccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:37.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.770+0000 7f1b327fc640 1 --2- 172.21.15.164:0/3133893143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e076eb4a00 0x55e076ea2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:37.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.770+0000 7f1b32ffd640 1 -- 172.21.15.164:0/3133893143 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e076ea7d90 msgr2=0x55e076eccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:37.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.770+0000 7f1b32ffd640 1 --2- 172.21.15.164:0/3133893143 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e076ea7d90 0x55e076eccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.770+0000 7f1b32ffd640 1 -- 172.21.15.164:0/3133893143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e076eb4a00 msgr2=0x55e076ea2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:37.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.770+0000 7f1b32ffd640 1 --2- 172.21.15.164:0/3133893143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e076eb4a00 0x55e076ea2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.770+0000 7f1b32ffd640 1 -- 172.21.15.164:0/3133893143 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e076ec0bb0 con 0x55e076ea8fc0 2024-09-19T08:46:37.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.770+0000 7f1b327fc640 1 --2- 172.21.15.164:0/3133893143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e076eb4a00 0x55e076ea2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:37.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.771+0000 7f1b32ffd640 1 --2- 172.21.15.164:0/3133893143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e076ea8fc0 0x55e076ecd000 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1b240058c0 tx=0x7f1b2401a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99c49ab80e2c93e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:37.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.771+0000 7f1b197fa640 1 -- 172.21.15.164:0/3133893143 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2401ad90 con 0x55e076ea8fc0 2024-09-19T08:46:37.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.771+0000 7f1b197fa640 1 -- 172.21.15.164:0/3133893143 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b2401a980 con 0x55e076ea8fc0 2024-09-19T08:46:37.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.771+0000 7f1b197fa640 1 -- 172.21.15.164:0/3133893143 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b240035f0 con 0x55e076ea8fc0 2024-09-19T08:46:37.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.772+0000 7f1b3be1a740 1 -- 172.21.15.164:0/3133893143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e076ea8fc0 msgr2=0x55e076ecd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:37.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.772+0000 7f1b3be1a740 1 --2- 172.21.15.164:0/3133893143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e076ea8fc0 0x55e076ecd000 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1b240058c0 tx=0x7f1b2401a430 comp rx=0 tx=0).stop 2024-09-19T08:46:37.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.772+0000 7f1b3be1a740 1 -- 172.21.15.164:0/3133893143 shutdown_connections 2024-09-19T08:46:37.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.772+0000 7f1b3be1a740 1 --2- 172.21.15.164:0/3133893143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e076eb4a00 0x55e076ea2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.772+0000 7f1b3be1a740 1 --2- 172.21.15.164:0/3133893143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e076ea8fc0 0x55e076ecd000 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.772+0000 7f1b3be1a740 1 --2- 172.21.15.164:0/3133893143 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e076ea7d90 0x55e076eccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.772+0000 7f1b3be1a740 1 -- 172.21.15.164:0/3133893143 >> 172.21.15.164:0/3133893143 conn(0x55e076ea3f00 msgr2=0x55e076eb02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:37.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.772+0000 7f1b3be1a740 1 -- 172.21.15.164:0/3133893143 shutdown_connections 2024-09-19T08:46:37.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.772+0000 7f1b3be1a740 1 -- 172.21.15.164:0/3133893143 wait complete. 2024-09-19T08:46:37.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.772+0000 7f1b3be1a740 1 -- start start 2024-09-19T08:46:37.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.772+0000 7f1b3be1a740 1 Processor -- start 2024-09-19T08:46:37.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.773+0000 7f1b3be1a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e076ea7d90 0x55e076ebd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.773+0000 7f1b3be1a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e076eb4a00 0x55e076eb9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.773+0000 7f1b3be1a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e076eba230 0x55e076eba6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.773+0000 7f1b3be1a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1b341ac910 con 0x55e076eb4a00 2024-09-19T08:46:37.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.773+0000 7f1b3be1a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1b341ac240 con 0x55e076ea7d90 2024-09-19T08:46:37.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.773+0000 7f1b3be1a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1b341a94e0 con 0x55e076eba230 2024-09-19T08:46:37.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.773+0000 7f1b3be1a740 10 client.0 _ll_register_callbacks cb 0x7f1b2400aed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:37.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.773+0000 7f1b32ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e076eb4a00 0x55e076eb9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:37.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.773+0000 7f1b32ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e076eb4a00 0x55e076eb9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45534/0 (socket says 172.21.15.164:45534) 2024-09-19T08:46:37.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.773+0000 7f1b32ffd640 1 -- 172.21.15.164:0/4287403353 learned_addr learned my addr 172.21.15.164:0/4287403353 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:37.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.773+0000 7f1b327fc640 1 --2- 172.21.15.164:0/4287403353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e076ea7d90 0x55e076ebd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:37.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.773+0000 7f1b327fc640 1 --2- 172.21.15.164:0/4287403353 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e076eba230 0x55e076eba6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:37.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.773+0000 7f1b32ffd640 1 -- 172.21.15.164:0/4287403353 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e076eba230 msgr2=0x55e076eba6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:37.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.773+0000 7f1b32ffd640 1 --2- 172.21.15.164:0/4287403353 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e076eba230 0x55e076eba6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.773+0000 7f1b32ffd640 1 -- 172.21.15.164:0/4287403353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e076ea7d90 msgr2=0x55e076ebd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:37.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.773+0000 7f1b32ffd640 1 --2- 172.21.15.164:0/4287403353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e076ea7d90 0x55e076ebd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.773+0000 7f1b32ffd640 1 -- 172.21.15.164:0/4287403353 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e076f2fb50 con 0x55e076eb4a00 2024-09-19T08:46:37.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.773+0000 7f1b32ffd640 1 --2- 172.21.15.164:0/4287403353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e076eb4a00 0x55e076eb9cf0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1b2401a790 tx=0x7f1b24005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99ca47f8774f60a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:37.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.773+0000 7f1b12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:46:37.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.774+0000 7f1b12ffd640 1 -- 172.21.15.164:0/4287403353 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24031040 con 0x55e076eb4a00 2024-09-19T08:46:37.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.774+0000 7f1b3be1a740 1 -- 172.21.15.164:0/4287403353 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b2401a980 con 0x55e076eb4a00 2024-09-19T08:46:37.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.774+0000 7f1b12ffd640 1 -- 172.21.15.164:0/4287403353 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f1b2401ad30 con 0x55e076eb4a00 2024-09-19T08:46:37.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.774+0000 7f1b12ffd640 1 -- 172.21.15.164:0/4287403353 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1b240107a0 con 0x55e076eb4a00 2024-09-19T08:46:37.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.774+0000 7f1b3be1a740 10 client.15633 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:37.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.774+0000 7f1b3be1a740 1 -- 172.21.15.164:0/4287403353 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e076f30e10 con 0x55e076eb4a00 2024-09-19T08:46:37.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.774+0000 7f1b12ffd640 1 -- 172.21.15.164:0/4287403353 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b2401a980 con 0x55e076eb4a00 2024-09-19T08:46:37.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.774+0000 7f1b12ffd640 1 -- 172.21.15.164:0/4287403353 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1b2403f020 con 0x55e076eb4a00 2024-09-19T08:46:37.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.774+0000 7f1b3be1a740 10 client.15633 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:37.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.774+0000 7f1b3be1a740 10 client.15633 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:46:37.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.774+0000 7f1b3be1a740 20 client.15633 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:37.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.774+0000 7f1b3be1a740 1 --2- 172.21.15.164:0/4287403353 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55e076f31430 0x55e076f33850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.774+0000 7f1b3be1a740 4 client.15633 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.2, "await_for": 45.0}] 2024-09-19T08:46:37.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.775+0000 7f1b3be1a740 1 -- 172.21.15.164:0/4287403353 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.2, "await_for": 45.0}) -- 0x55e076e12d20 con 0x55e076f31430 2024-09-19T08:46:37.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.775+0000 7f1b327fc640 1 --2- 172.21.15.164:0/4287403353 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55e076f31430 0x55e076f33850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:37.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.775+0000 7f1b12ffd640 10 client.15633 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:46:37.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.775+0000 7f1b327fc640 1 --2- 172.21.15.164:0/4287403353 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55e076f31430 0x55e076f33850 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3724691d927e176 server_cookie=bf1657aa2b5467fc in_seq=0 out_seq=0 2024-09-19T08:46:37.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.787+0000 7f1b12ffd640 1 -- 172.21.15.164:0/4287403353 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55e076e12d20 con 0x55e076f31430 2024-09-19T08:46:37.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.787+0000 7f1b12ffd640 10 client.15633 handle_command_reply: tid=0 2024-09-19T08:46:37.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.787+0000 7f1b3be1a740 1 client.15633 shutdown 2024-09-19T08:46:37.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.787+0000 7f1b3be1a740 1 -- 172.21.15.164:0/4287403353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e076eb4a00 msgr2=0x55e076eb9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:37.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.787+0000 7f1b3be1a740 1 --2- 172.21.15.164:0/4287403353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e076eb4a00 0x55e076eb9cf0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1b2401a790 tx=0x7f1b24005770 comp rx=0 tx=0).stop 2024-09-19T08:46:37.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.787+0000 7f1b3be1a740 1 -- 172.21.15.164:0/4287403353 shutdown_connections 2024-09-19T08:46:37.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.787+0000 7f1b3be1a740 1 --2- 172.21.15.164:0/4287403353 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e076eba230 0x55e076eba6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.787+0000 7f1b3be1a740 1 --2- 172.21.15.164:0/4287403353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e076eb4a00 0x55e076eb9cf0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.787+0000 7f1b3be1a740 1 --2- 172.21.15.164:0/4287403353 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55e076f31430 0x55e076f33850 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.787+0000 7f1b3be1a740 1 --2- 172.21.15.164:0/4287403353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e076ea7d90 0x55e076ebd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.787+0000 7f1b12ffd640 0 client.15633 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:46:37.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.787+0000 7f1b3be1a740 1 -- 172.21.15.164:0/4287403353 >> 172.21.15.164:0/4287403353 conn(0x55e076ea3f00 msgr2=0x55e076ea2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:37.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.787+0000 7f1b3be1a740 1 -- 172.21.15.164:0/4287403353 shutdown_connections 2024-09-19T08:46:37.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.787+0000 7f1b3be1a740 1 -- 172.21.15.164:0/4287403353 wait complete. 2024-09-19T08:46:37.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.788+0000 7f1b3be1a740 20 client.15633 trim_cache size 0 max 16384 2024-09-19T08:46:37.789 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:46:37.789 INFO:teuthology.orchestra.run.smithi164.stdout: "epoch": 28, 2024-09-19T08:46:37.789 INFO:teuthology.orchestra.run.smithi164.stdout: "leader": 24479, 2024-09-19T08:46:37.790 INFO:teuthology.orchestra.run.smithi164.stdout: "set_version": 2, 2024-09-19T08:46:37.790 INFO:teuthology.orchestra.run.smithi164.stdout: "sets": { 2024-09-19T08:46:37.790 INFO:teuthology.orchestra.run.smithi164.stdout: "446cbd86": { 2024-09-19T08:46:37.790 INFO:teuthology.orchestra.run.smithi164.stdout: "version": 2, 2024-09-19T08:46:37.790 INFO:teuthology.orchestra.run.smithi164.stdout: "age_ref": 0.0, 2024-09-19T08:46:37.790 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:46:37.790 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "QUIESCED", 2024-09-19T08:46:37.790 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:46:37.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:46:37.790 INFO:teuthology.orchestra.run.smithi164.stdout: "timeout": 90.0, 2024-09-19T08:46:37.790 INFO:teuthology.orchestra.run.smithi164.stdout: "expiration": 131.2, 2024-09-19T08:46:37.790 INFO:teuthology.orchestra.run.smithi164.stdout: "members": { 2024-09-19T08:46:37.790 INFO:teuthology.orchestra.run.smithi164.stdout: "file:/": { 2024-09-19T08:46:37.791 INFO:teuthology.orchestra.run.smithi164.stdout: "excluded": false, 2024-09-19T08:46:37.791 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:46:37.791 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "QUIESCED", 2024-09-19T08:46:37.791 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:46:37.791 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:46:37.791 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:46:37.791 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:46:37.791 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:46:37.791 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:46:37.791 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:46:37.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.788+0000 7f1b39a41640 1 -- 172.21.15.164:0/2391095530 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f1b1c0a3550 msgr2=0x7f1b1c0a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:37.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.788+0000 7f1b39a41640 1 --2- 172.21.15.164:0/2391095530 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f1b1c0a3550 0x7f1b1c0a5a10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c0ee5d0 tx=0x7f1b2c004990 comp rx=0 tx=0).stop 2024-09-19T08:46:37.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.788+0000 7f1b39a41640 1 -- 172.21.15.164:0/2391095530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b34102ef0 msgr2=0x7f1b3419bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:37.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.788+0000 7f1b39a41640 1 --2- 172.21.15.164:0/2391095530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b34102ef0 0x7f1b3419bcb0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c006010 tx=0x7f1b2c02f6f0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.788+0000 7f1b39a41640 1 -- 172.21.15.164:0/2391095530 shutdown_connections 2024-09-19T08:46:37.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.788+0000 7f1b39a41640 1 --2- 172.21.15.164:0/2391095530 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f1b1c0a3550 0x7f1b1c0a5a10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.788+0000 7f1b39a41640 1 --2- 172.21.15.164:0/2391095530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b34102ef0 0x7f1b3419bcb0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.788+0000 7f1b39a41640 1 --2- 172.21.15.164:0/2391095530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b34102570 0x7f1b34197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.788+0000 7f1b39a41640 1 --2- 172.21.15.164:0/2391095530 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b34100e20 0x7f1b341973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.788+0000 7f1b39a41640 1 -- 172.21.15.164:0/2391095530 >> 172.21.15.164:0/2391095530 conn(0x7f1b340fc7d0 msgr2=0x7f1b34106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:37.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.788+0000 7f1b39a41640 1 -- 172.21.15.164:0/2391095530 shutdown_connections 2024-09-19T08:46:37.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:37.788+0000 7f1b39a41640 1 -- 172.21.15.164:0/2391095530 wait complete. 2024-09-19T08:46:37.799 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:46:38.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:37 smithi164 ceph-mon[30928]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-19T08:46:38.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:37 smithi170 ceph-mon[35072]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-19T08:46:38.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:37 smithi173 ceph-mon[35306]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-19T08:46:38.521 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '446cbd86', quiesce took 4.8 seconds. Will release after: 6.3999999999999995 2024-09-19T08:46:39.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.021+0000 7f7580378640 1 -- 172.21.15.164:0/3461472640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7578101410 msgr2=0x7f757810d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:39.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.021+0000 7f7580378640 1 --2- 172.21.15.164:0/3461472640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7578101410 0x7f757810d940 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7570009cd0 tx=0x7f757002a8e0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.021+0000 7f7580378640 1 -- 172.21.15.164:0/3461472640 shutdown_connections 2024-09-19T08:46:39.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.021+0000 7f7580378640 1 --2- 172.21.15.164:0/3461472640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7578101410 0x7f757810d940 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.021+0000 7f7580378640 1 --2- 172.21.15.164:0/3461472640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7578100a70 0x7f7578100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.021+0000 7f7580378640 1 --2- 172.21.15.164:0/3461472640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7578106a70 0x7f7578106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.021+0000 7f7580378640 1 -- 172.21.15.164:0/3461472640 >> 172.21.15.164:0/3461472640 conn(0x7f75780fc790 msgr2=0x7f75780febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:39.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.021+0000 7f7580378640 1 -- 172.21.15.164:0/3461472640 shutdown_connections 2024-09-19T08:46:39.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.021+0000 7f7580378640 1 -- 172.21.15.164:0/3461472640 wait complete. 2024-09-19T08:46:39.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.022+0000 7f7580378640 1 Processor -- start 2024-09-19T08:46:39.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.022+0000 7f7580378640 1 -- start start 2024-09-19T08:46:39.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.022+0000 7f7580378640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7578100a70 0x7f75781972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:39.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.022+0000 7f7580378640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7578101410 0x7f7578197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:39.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.022+0000 7f7580378640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7578106a70 0x7f757819bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:39.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.022+0000 7f7580378640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7578110e90 con 0x7f7578106a70 2024-09-19T08:46:39.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.022+0000 7f7580378640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7578110d10 con 0x7f7578100a70 2024-09-19T08:46:39.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.022+0000 7f7580378640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7578111010 con 0x7f7578101410 2024-09-19T08:46:39.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.022+0000 7f757e915640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7578106a70 0x7f757819bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:39.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.022+0000 7f757e915640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7578106a70 0x7f757819bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45564/0 (socket says 172.21.15.164:45564) 2024-09-19T08:46:39.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.022+0000 7f757e915640 1 -- 172.21.15.164:0/2849296574 learned_addr learned my addr 172.21.15.164:0/2849296574 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:39.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.023+0000 7f757e114640 1 --2- 172.21.15.164:0/2849296574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7578100a70 0x7f75781972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:39.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.023+0000 7f757d913640 1 --2- 172.21.15.164:0/2849296574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7578101410 0x7f7578197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:39.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.023+0000 7f757e915640 1 -- 172.21.15.164:0/2849296574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7578101410 msgr2=0x7f7578197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:39.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.023+0000 7f757e915640 1 --2- 172.21.15.164:0/2849296574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7578101410 0x7f7578197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.023+0000 7f757e915640 1 -- 172.21.15.164:0/2849296574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7578100a70 msgr2=0x7f75781972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:39.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.023+0000 7f757e915640 1 --2- 172.21.15.164:0/2849296574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7578100a70 0x7f75781972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.023+0000 7f757e915640 1 -- 172.21.15.164:0/2849296574 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757819c290 con 0x7f7578106a70 2024-09-19T08:46:39.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.023+0000 7f757e114640 1 --2- 172.21.15.164:0/2849296574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7578100a70 0x7f75781972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:39.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.023+0000 7f757e915640 1 --2- 172.21.15.164:0/2849296574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7578106a70 0x7f757819bbb0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f7570006010 tx=0x7f7570003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:39.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.023+0000 7f756f7fe640 1 -- 172.21.15.164:0/2849296574 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7570004450 con 0x7f7578106a70 2024-09-19T08:46:39.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.023+0000 7f7580378640 1 -- 172.21.15.164:0/2849296574 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f757819c520 con 0x7f7578106a70 2024-09-19T08:46:39.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.023+0000 7f756f7fe640 1 -- 172.21.15.164:0/2849296574 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7570003e10 con 0x7f7578106a70 2024-09-19T08:46:39.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.023+0000 7f756f7fe640 1 -- 172.21.15.164:0/2849296574 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75700466e0 con 0x7f7578106a70 2024-09-19T08:46:39.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.023+0000 7f7580378640 1 -- 172.21.15.164:0/2849296574 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f757806a790 con 0x7f7578106a70 2024-09-19T08:46:39.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.026+0000 7f756f7fe640 1 -- 172.21.15.164:0/2849296574 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7f75700305a0 con 0x7f7578106a70 2024-09-19T08:46:39.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.027+0000 7f7582751740 1 Processor -- start 2024-09-19T08:46:39.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.027+0000 7f7582751740 1 -- start start 2024-09-19T08:46:39.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.027+0000 7f7582751740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aca9d27b40 0x55aca9d26090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:39.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.027+0000 7f7582751740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aca9d265d0 0x55aca9d26a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:39.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.027+0000 7f7582751740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aca9d31370 0x55aca9d33760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:39.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.027+0000 7f7582751740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55aca9c8c2f0 con 0x55aca9d31370 2024-09-19T08:46:39.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.027+0000 7f7582751740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55aca9c33cf0 con 0x55aca9d265d0 2024-09-19T08:46:39.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.027+0000 7f7582751740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55aca9c882b0 con 0x55aca9d27b40 2024-09-19T08:46:39.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.027+0000 7f756f7fe640 1 --2- 172.21.15.164:0/2849296574 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f75640a3550 0x7f75640a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:39.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.027+0000 7f756f7fe640 1 -- 172.21.15.164:0/2849296574 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f757002e030 con 0x7f7578106a70 2024-09-19T08:46:39.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.027+0000 7f757e114640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aca9d265d0 0x55aca9d26a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:39.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.027+0000 7f757d913640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aca9d31370 0x55aca9d33760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:39.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.027+0000 7f757e114640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aca9d265d0 0x55aca9d26a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:51468/0 (socket says 172.21.15.164:51468) 2024-09-19T08:46:39.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.027+0000 7f757e114640 1 -- 172.21.15.164:0/1205309847 learned_addr learned my addr 172.21.15.164:0/1205309847 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:39.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.027+0000 7f757d913640 1 --2- 172.21.15.164:0/1205309847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aca9d27b40 0x55aca9d26090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:39.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.028+0000 7f757e915640 1 --2- 172.21.15.164:0/2849296574 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f75640a3550 0x7f75640a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:39.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.028+0000 7f757e114640 1 -- 172.21.15.164:0/1205309847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aca9d27b40 msgr2=0x55aca9d26090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:39.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.028+0000 7f757e114640 1 --2- 172.21.15.164:0/1205309847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aca9d27b40 0x55aca9d26090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.028+0000 7f757e114640 1 -- 172.21.15.164:0/1205309847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aca9d31370 msgr2=0x55aca9d33760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:39.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.028+0000 7f757e114640 1 --2- 172.21.15.164:0/1205309847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aca9d31370 0x55aca9d33760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.028+0000 7f757e114640 1 -- 172.21.15.164:0/1205309847 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aca9c183a0 con 0x55aca9d265d0 2024-09-19T08:46:39.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.028+0000 7f757d913640 1 --2- 172.21.15.164:0/1205309847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aca9d31370 0x55aca9d33760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:46:39.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.028+0000 7f757e114640 1 --2- 172.21.15.164:0/1205309847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aca9d265d0 0x55aca9d26a10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7574007850 tx=0x7f7574007d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1efe508898b9c40c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:39.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.029+0000 7f757e915640 1 --2- 172.21.15.164:0/2849296574 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f75640a3550 0x7f75640a5a10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f75700eede0 tx=0x7f75700095c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:39.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.029+0000 7f756cdea640 1 -- 172.21.15.164:0/1205309847 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757400d950 con 0x55aca9d265d0 2024-09-19T08:46:39.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.029+0000 7f756cdea640 1 -- 172.21.15.164:0/1205309847 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7574018070 con 0x55aca9d265d0 2024-09-19T08:46:39.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.029+0000 7f756cdea640 1 -- 172.21.15.164:0/1205309847 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75740123e0 con 0x55aca9d265d0 2024-09-19T08:46:39.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.030+0000 7f7582751740 1 -- 172.21.15.164:0/1205309847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aca9d265d0 msgr2=0x55aca9d26a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:39.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.030+0000 7f7582751740 1 --2- 172.21.15.164:0/1205309847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aca9d265d0 0x55aca9d26a10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7574007850 tx=0x7f7574007d10 comp rx=0 tx=0).stop 2024-09-19T08:46:39.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.030+0000 7f7582751740 1 -- 172.21.15.164:0/1205309847 shutdown_connections 2024-09-19T08:46:39.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.030+0000 7f7582751740 1 --2- 172.21.15.164:0/1205309847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aca9d31370 0x55aca9d33760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.030+0000 7f7582751740 1 --2- 172.21.15.164:0/1205309847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aca9d265d0 0x55aca9d26a10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.030+0000 7f7582751740 1 --2- 172.21.15.164:0/1205309847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aca9d27b40 0x55aca9d26090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.030+0000 7f7582751740 1 -- 172.21.15.164:0/1205309847 >> 172.21.15.164:0/1205309847 conn(0x55aca9cef310 msgr2=0x55aca9d22c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:39.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.030+0000 7f7582751740 1 -- 172.21.15.164:0/1205309847 shutdown_connections 2024-09-19T08:46:39.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.030+0000 7f7582751740 1 -- 172.21.15.164:0/1205309847 wait complete. 2024-09-19T08:46:39.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.030+0000 7f7582751740 1 -- start start 2024-09-19T08:46:39.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.031+0000 7f7582751740 1 Processor -- start 2024-09-19T08:46:39.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.031+0000 7f7582751740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aca9d265d0 0x55aca9d30d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:39.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.031+0000 7f7582751740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aca9d27b40 0x55aca9d2e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:39.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.031+0000 7f7582751740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aca9d31370 0x55aca9d2e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:39.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.031+0000 7f7582751740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55aca9d38480 con 0x55aca9d265d0 2024-09-19T08:46:39.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.031+0000 7f7582751740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55aca9c47060 con 0x55aca9d27b40 2024-09-19T08:46:39.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.031+0000 7f7582751740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55aca9d38600 con 0x55aca9d31370 2024-09-19T08:46:39.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.031+0000 7f7582751740 10 client.0 _ll_register_callbacks cb 0x55aca999a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:39.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.031+0000 7f757d913640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aca9d265d0 0x55aca9d30d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:39.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.031+0000 7f757d913640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aca9d265d0 0x55aca9d30d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45584/0 (socket says 172.21.15.164:45584) 2024-09-19T08:46:39.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.031+0000 7f757d913640 1 -- 172.21.15.164:0/1666598836 learned_addr learned my addr 172.21.15.164:0/1666598836 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:39.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.031+0000 7f757d913640 1 --2- 172.21.15.164:0/1666598836 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aca9d31370 0x55aca9d2e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:39.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.031+0000 7f757e114640 1 --2- 172.21.15.164:0/1666598836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aca9d27b40 0x55aca9d2e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:39.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.031+0000 7f757d913640 1 -- 172.21.15.164:0/1666598836 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aca9d31370 msgr2=0x55aca9d2e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:39.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.031+0000 7f757d913640 1 --2- 172.21.15.164:0/1666598836 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aca9d31370 0x55aca9d2e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.031+0000 7f757d913640 1 -- 172.21.15.164:0/1666598836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aca9d27b40 msgr2=0x55aca9d2e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:39.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.032+0000 7f757d913640 1 --2- 172.21.15.164:0/1666598836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aca9d27b40 0x55aca9d2e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.032+0000 7f757d913640 1 -- 172.21.15.164:0/1666598836 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aca99fd3c0 con 0x55aca9d265d0 2024-09-19T08:46:39.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.032+0000 7f757d913640 1 --2- 172.21.15.164:0/1666598836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aca9d265d0 0x55aca9d30d50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7568013e80 tx=0x7f756800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf6f4d92cae1770c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:39.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.032+0000 7f7561ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:46:39.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.032+0000 7f7561ffb640 1 -- 172.21.15.164:0/1666598836 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568013310 con 0x55aca9d265d0 2024-09-19T08:46:39.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.032+0000 7f7561ffb640 1 -- 172.21.15.164:0/1666598836 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f756800f9d0 con 0x55aca9d265d0 2024-09-19T08:46:39.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.032+0000 7f7582751740 1 -- 172.21.15.164:0/1666598836 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7574018070 con 0x55aca9d265d0 2024-09-19T08:46:39.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.032+0000 7f7561ffb640 1 -- 172.21.15.164:0/1666598836 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f756800fbd0 con 0x55aca9d265d0 2024-09-19T08:46:39.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.032+0000 7f7582751740 10 client.15645 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:39.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.032+0000 7f7582751740 1 -- 172.21.15.164:0/1666598836 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55aca9d43fb0 con 0x55aca9d265d0 2024-09-19T08:46:39.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.033+0000 7f7561ffb640 1 -- 172.21.15.164:0/1666598836 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7574018070 con 0x55aca9d265d0 2024-09-19T08:46:39.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.033+0000 7f7561ffb640 1 -- 172.21.15.164:0/1666598836 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7568028050 con 0x55aca9d265d0 2024-09-19T08:46:39.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.033+0000 7f7582751740 10 client.15645 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:39.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.033+0000 7f7582751740 10 client.15645 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T08:46:39.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.033+0000 7f7582751740 20 client.15645 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:39.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.033+0000 7f7582751740 1 --2- 172.21.15.164:0/1666598836 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55aca9d453c0 0x55aca9d477d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:39.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.033+0000 7f7582751740 4 client.15645 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:46:39.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.033+0000 7f7582751740 1 -- 172.21.15.164:0/1666598836 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aca9c99860 con 0x55aca9d453c0 2024-09-19T08:46:39.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.033+0000 7f757e114640 1 --2- 172.21.15.164:0/1666598836 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55aca9d453c0 0x55aca9d477d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:39.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.034+0000 7f757e114640 1 --2- 172.21.15.164:0/1666598836 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55aca9d453c0 0x55aca9d477d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2329170830adfed server_cookie=1ad65d1373983f51 in_seq=0 out_seq=0 2024-09-19T08:46:39.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.034+0000 7f7561ffb640 10 client.15645 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:46:39.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.034+0000 7f7561ffb640 1 -- 172.21.15.164:0/1666598836 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aca9c99860 con 0x55aca9d453c0 2024-09-19T08:46:39.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.034+0000 7f7561ffb640 10 client.15645 handle_command_reply: tid=0 2024-09-19T08:46:39.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.034+0000 7f7582751740 1 client.15645 shutdown 2024-09-19T08:46:39.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.034+0000 7f7582751740 1 -- 172.21.15.164:0/1666598836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aca9d265d0 msgr2=0x55aca9d30d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:39.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.034+0000 7f7582751740 1 --2- 172.21.15.164:0/1666598836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aca9d265d0 0x55aca9d30d50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7568013e80 tx=0x7f756800eb50 comp rx=0 tx=0).stop 2024-09-19T08:46:39.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.035+0000 7f7582751740 1 -- 172.21.15.164:0/1666598836 shutdown_connections 2024-09-19T08:46:39.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.035+0000 7f7582751740 1 --2- 172.21.15.164:0/1666598836 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aca9d31370 0x55aca9d2e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.035+0000 7f7582751740 1 --2- 172.21.15.164:0/1666598836 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55aca9d453c0 0x55aca9d477d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.035+0000 7f7582751740 1 --2- 172.21.15.164:0/1666598836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aca9d27b40 0x55aca9d2e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.035+0000 7f7582751740 1 --2- 172.21.15.164:0/1666598836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aca9d265d0 0x55aca9d30d50 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.035+0000 7f7561ffb640 0 client.15645 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:46:39.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.035+0000 7f7582751740 1 -- 172.21.15.164:0/1666598836 >> 172.21.15.164:0/1666598836 conn(0x55aca9cef310 msgr2=0x55aca9d28960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:39.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.035+0000 7f7582751740 1 -- 172.21.15.164:0/1666598836 shutdown_connections 2024-09-19T08:46:39.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.035+0000 7f7582751740 1 -- 172.21.15.164:0/1666598836 wait complete. 2024-09-19T08:46:39.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.035+0000 7f7582751740 20 client.15645 trim_cache size 0 max 16384 2024-09-19T08:46:39.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.048+0000 7f7582751740 1 Processor -- start 2024-09-19T08:46:39.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.048+0000 7f7582751740 1 -- start start 2024-09-19T08:46:39.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.048+0000 7f7582751740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aca9d265d0 0x55aca9d22d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:39.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.048+0000 7f7582751740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aca9d27b40 0x55aca9d2ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:39.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.048+0000 7f7582751740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aca9d31370 0x55aca9d29120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:39.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.048+0000 7f7582751740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f75781ab3a0 con 0x55aca9d265d0 2024-09-19T08:46:39.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.048+0000 7f7582751740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f75781acee0 con 0x55aca9d31370 2024-09-19T08:46:39.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.048+0000 7f7582751740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f75781aba70 con 0x55aca9d27b40 2024-09-19T08:46:39.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.048+0000 7f757d913640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aca9d265d0 0x55aca9d22d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:39.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.048+0000 7f757d913640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aca9d265d0 0x55aca9d22d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45590/0 (socket says 172.21.15.164:45590) 2024-09-19T08:46:39.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.048+0000 7f757d913640 1 -- 172.21.15.164:0/4154171181 learned_addr learned my addr 172.21.15.164:0/4154171181 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:39.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.048+0000 7f757e114640 1 --2- 172.21.15.164:0/4154171181 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aca9d27b40 0x55aca9d2ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:39.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.048+0000 7f757d913640 1 --2- 172.21.15.164:0/4154171181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aca9d31370 0x55aca9d29120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:39.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.048+0000 7f757d913640 1 -- 172.21.15.164:0/4154171181 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aca9d27b40 msgr2=0x55aca9d2ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:39.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.048+0000 7f757d913640 1 --2- 172.21.15.164:0/4154171181 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aca9d27b40 0x55aca9d2ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.048+0000 7f757d913640 1 -- 172.21.15.164:0/4154171181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aca9d31370 msgr2=0x55aca9d29120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:39.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.048+0000 7f757d913640 1 --2- 172.21.15.164:0/4154171181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aca9d31370 0x55aca9d29120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.048+0000 7f757d913640 1 -- 172.21.15.164:0/4154171181 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aca9d30970 con 0x55aca9d265d0 2024-09-19T08:46:39.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.049+0000 7f757d913640 1 --2- 172.21.15.164:0/4154171181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aca9d265d0 0x55aca9d22d70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7568013de0 tx=0x7f756800e410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967bcd58d9e04ca1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:39.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.049+0000 7f7563fff640 1 -- 172.21.15.164:0/4154171181 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568029370 con 0x55aca9d265d0 2024-09-19T08:46:39.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.049+0000 7f7563fff640 1 -- 172.21.15.164:0/4154171181 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75680055a0 con 0x55aca9d265d0 2024-09-19T08:46:39.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.049+0000 7f7563fff640 1 -- 172.21.15.164:0/4154171181 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568011440 con 0x55aca9d265d0 2024-09-19T08:46:39.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.050+0000 7f7582751740 1 -- 172.21.15.164:0/4154171181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aca9d265d0 msgr2=0x55aca9d22d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:39.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.050+0000 7f7582751740 1 --2- 172.21.15.164:0/4154171181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aca9d265d0 0x55aca9d22d70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7568013de0 tx=0x7f756800e410 comp rx=0 tx=0).stop 2024-09-19T08:46:39.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.050+0000 7f7582751740 1 -- 172.21.15.164:0/4154171181 shutdown_connections 2024-09-19T08:46:39.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.050+0000 7f7582751740 1 --2- 172.21.15.164:0/4154171181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aca9d31370 0x55aca9d29120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.050+0000 7f7582751740 1 --2- 172.21.15.164:0/4154171181 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aca9d27b40 0x55aca9d2ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.050+0000 7f7582751740 1 --2- 172.21.15.164:0/4154171181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aca9d265d0 0x55aca9d22d70 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.050+0000 7f7582751740 1 -- 172.21.15.164:0/4154171181 >> 172.21.15.164:0/4154171181 conn(0x55aca9cef310 msgr2=0x55aca9d2f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:39.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.050+0000 7f7582751740 1 -- 172.21.15.164:0/4154171181 shutdown_connections 2024-09-19T08:46:39.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.050+0000 7f7582751740 1 -- 172.21.15.164:0/4154171181 wait complete. 2024-09-19T08:46:39.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.050+0000 7f7582751740 1 -- start start 2024-09-19T08:46:39.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.050+0000 7f7582751740 1 Processor -- start 2024-09-19T08:46:39.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.050+0000 7f7582751740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aca9d265d0 0x55aca9d3de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:39.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.050+0000 7f7582751740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aca9d27b40 0x55aca9d2acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:39.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.050+0000 7f7582751740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aca9d31370 0x55aca9d2b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:39.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.050+0000 7f7582751740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f75781ac810 con 0x55aca9d27b40 2024-09-19T08:46:39.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.050+0000 7f7582751740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f75781ac140 con 0x55aca9d265d0 2024-09-19T08:46:39.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.050+0000 7f7582751740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f75781a93c0 con 0x55aca9d31370 2024-09-19T08:46:39.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.050+0000 7f7582751740 10 client.0 _ll_register_callbacks cb 0x7f7568010660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:39.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.051+0000 7f757e114640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aca9d27b40 0x55aca9d2acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:39.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.051+0000 7f757e114640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aca9d27b40 0x55aca9d2acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45598/0 (socket says 172.21.15.164:45598) 2024-09-19T08:46:39.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.051+0000 7f757e114640 1 -- 172.21.15.164:0/3496069281 learned_addr learned my addr 172.21.15.164:0/3496069281 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:39.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.051+0000 7f757d913640 1 --2- 172.21.15.164:0/3496069281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aca9d265d0 0x55aca9d3de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:39.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.051+0000 7f757d913640 1 --2- 172.21.15.164:0/3496069281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aca9d31370 0x55aca9d2b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:39.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.051+0000 7f757e114640 1 -- 172.21.15.164:0/3496069281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aca9d31370 msgr2=0x55aca9d2b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:39.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.051+0000 7f757e114640 1 --2- 172.21.15.164:0/3496069281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aca9d31370 0x55aca9d2b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.051+0000 7f757e114640 1 -- 172.21.15.164:0/3496069281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aca9d265d0 msgr2=0x55aca9d3de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:39.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.051+0000 7f757e114640 1 --2- 172.21.15.164:0/3496069281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aca9d265d0 0x55aca9d3de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.051+0000 7f757e114640 1 -- 172.21.15.164:0/3496069281 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aca9d2b8e0 con 0x55aca9d27b40 2024-09-19T08:46:39.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.051+0000 7f757e114640 1 --2- 172.21.15.164:0/3496069281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aca9d27b40 0x55aca9d2acb0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7574007d50 tx=0x7f7574006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b0f619cb16d5ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:39.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.051+0000 7f7561ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:46:39.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.051+0000 7f7561ffb640 1 -- 172.21.15.164:0/3496069281 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757400d950 con 0x55aca9d27b40 2024-09-19T08:46:39.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.051+0000 7f7561ffb640 1 -- 172.21.15.164:0/3496069281 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f757400daf0 con 0x55aca9d27b40 2024-09-19T08:46:39.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.051+0000 7f7582751740 1 -- 172.21.15.164:0/3496069281 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75680055a0 con 0x55aca9d27b40 2024-09-19T08:46:39.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.052+0000 7f7561ffb640 1 -- 172.21.15.164:0/3496069281 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7574036050 con 0x55aca9d27b40 2024-09-19T08:46:39.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.052+0000 7f7582751740 10 client.15651 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:39.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.052+0000 7f7582751740 1 -- 172.21.15.164:0/3496069281 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55aca9daff80 con 0x55aca9d27b40 2024-09-19T08:46:39.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.052+0000 7f7561ffb640 1 -- 172.21.15.164:0/3496069281 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75680055a0 con 0x55aca9d27b40 2024-09-19T08:46:39.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.052+0000 7f7561ffb640 1 -- 172.21.15.164:0/3496069281 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f757402e3d0 con 0x55aca9d27b40 2024-09-19T08:46:39.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.052+0000 7f7582751740 10 client.15651 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:39.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.052+0000 7f7582751740 10 client.15651 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T08:46:39.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.052+0000 7f7582751740 20 client.15651 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:39.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.052+0000 7f7582751740 1 --2- 172.21.15.164:0/3496069281 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55aca9d453c0 0x55aca9db2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:39.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.052+0000 7f7582751740 4 client.15651 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:46:39.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.052+0000 7f7582751740 1 -- 172.21.15.164:0/3496069281 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aca9c93d70 con 0x55aca9d453c0 2024-09-19T08:46:39.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.052+0000 7f757d913640 1 --2- 172.21.15.164:0/3496069281 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55aca9d453c0 0x55aca9db2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:39.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.053+0000 7f757d913640 1 --2- 172.21.15.164:0/3496069281 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55aca9d453c0 0x55aca9db2550 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=db8c20ae2b3e2fa2 server_cookie=e3d19e729eb0206b in_seq=0 out_seq=0 2024-09-19T08:46:39.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.053+0000 7f7561ffb640 10 client.15651 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:46:39.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.053+0000 7f7561ffb640 1 -- 172.21.15.164:0/3496069281 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aca9c93d70 con 0x55aca9d453c0 2024-09-19T08:46:39.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.053+0000 7f7561ffb640 10 client.15651 handle_command_reply: tid=0 2024-09-19T08:46:39.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.053+0000 7f7582751740 1 client.15651 shutdown 2024-09-19T08:46:39.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.053+0000 7f7582751740 1 -- 172.21.15.164:0/3496069281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aca9d27b40 msgr2=0x55aca9d2acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:39.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.053+0000 7f7582751740 1 --2- 172.21.15.164:0/3496069281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aca9d27b40 0x55aca9d2acb0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7574007d50 tx=0x7f7574006f90 comp rx=0 tx=0).stop 2024-09-19T08:46:39.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.053+0000 7f7582751740 1 -- 172.21.15.164:0/3496069281 shutdown_connections 2024-09-19T08:46:39.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.053+0000 7f7582751740 1 --2- 172.21.15.164:0/3496069281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aca9d31370 0x55aca9d2b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.053+0000 7f7582751740 1 --2- 172.21.15.164:0/3496069281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aca9d27b40 0x55aca9d2acb0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.053+0000 7f7582751740 1 --2- 172.21.15.164:0/3496069281 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55aca9d453c0 0x55aca9db2550 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.053+0000 7f7582751740 1 --2- 172.21.15.164:0/3496069281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aca9d265d0 0x55aca9d3de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.053+0000 7f7561ffb640 0 client.15651 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:46:39.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.053+0000 7f7582751740 1 -- 172.21.15.164:0/3496069281 >> 172.21.15.164:0/3496069281 conn(0x55aca9cef310 msgr2=0x55aca9d30080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:39.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.053+0000 7f7582751740 1 -- 172.21.15.164:0/3496069281 shutdown_connections 2024-09-19T08:46:39.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.053+0000 7f7582751740 1 -- 172.21.15.164:0/3496069281 wait complete. 2024-09-19T08:46:39.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.053+0000 7f7582751740 20 client.15651 trim_cache size 0 max 16384 2024-09-19T08:46:39.060 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:46:39.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.054+0000 7f7580378640 1 -- 172.21.15.164:0/2849296574 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f75640a3550 msgr2=0x7f75640a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:39.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.054+0000 7f7580378640 1 --2- 172.21.15.164:0/2849296574 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f75640a3550 0x7f75640a5a10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f75700eede0 tx=0x7f75700095c0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.054+0000 7f7580378640 1 -- 172.21.15.164:0/2849296574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7578106a70 msgr2=0x7f757819bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:39.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.054+0000 7f7580378640 1 --2- 172.21.15.164:0/2849296574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7578106a70 0x7f757819bbb0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f7570006010 tx=0x7f7570003c20 comp rx=0 tx=0).stop 2024-09-19T08:46:39.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.054+0000 7f7580378640 1 -- 172.21.15.164:0/2849296574 shutdown_connections 2024-09-19T08:46:39.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.054+0000 7f7580378640 1 --2- 172.21.15.164:0/2849296574 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f75640a3550 0x7f75640a5a10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.054+0000 7f7580378640 1 --2- 172.21.15.164:0/2849296574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7578106a70 0x7f757819bbb0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.054+0000 7f7580378640 1 --2- 172.21.15.164:0/2849296574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7578101410 0x7f7578197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.054+0000 7f7580378640 1 --2- 172.21.15.164:0/2849296574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7578100a70 0x7f75781972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.054+0000 7f7580378640 1 -- 172.21.15.164:0/2849296574 >> 172.21.15.164:0/2849296574 conn(0x7f75780fc790 msgr2=0x7f75780fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:39.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.054+0000 7f7580378640 1 -- 172.21.15.164:0/2849296574 shutdown_connections 2024-09-19T08:46:39.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:39.054+0000 7f7580378640 1 -- 172.21.15.164:0/2849296574 wait complete. 2024-09-19T08:46:39.869 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T08:46:40.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:39 smithi164 ceph-mon[30928]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:46:40.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:39 smithi170 ceph-mon[35072]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:46:40.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:39 smithi173 ceph-mon[35306]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:46:40.201 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:46:40.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.685+0000 7f9641cb3640 1 -- 172.21.15.164:0/1708783031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f963c102570 msgr2=0x7f963c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:40.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.685+0000 7f9641cb3640 1 --2- 172.21.15.164:0/1708783031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f963c102570 0x7f963c1029b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9630009900 tx=0x7f963002a480 comp rx=0 tx=0).stop 2024-09-19T08:46:40.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.685+0000 7f9641cb3640 1 -- 172.21.15.164:0/1708783031 shutdown_connections 2024-09-19T08:46:40.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.685+0000 7f9641cb3640 1 --2- 172.21.15.164:0/1708783031 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f963c102ef0 0x7f963c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.685+0000 7f9641cb3640 1 --2- 172.21.15.164:0/1708783031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f963c102570 0x7f963c1029b0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.685+0000 7f9641cb3640 1 --2- 172.21.15.164:0/1708783031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f963c100e20 0x7f963c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.685+0000 7f9641cb3640 1 -- 172.21.15.164:0/1708783031 >> 172.21.15.164:0/1708783031 conn(0x7f963c0fc7d0 msgr2=0x7f963c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:40.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.685+0000 7f9641cb3640 1 -- 172.21.15.164:0/1708783031 shutdown_connections 2024-09-19T08:46:40.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.685+0000 7f9641cb3640 1 -- 172.21.15.164:0/1708783031 wait complete. 2024-09-19T08:46:40.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.686+0000 7f9641cb3640 1 Processor -- start 2024-09-19T08:46:40.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.686+0000 7f9641cb3640 1 -- start start 2024-09-19T08:46:40.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.686+0000 7f9641cb3640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f963c100e20 0x7f963c1973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:40.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.686+0000 7f9641cb3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f963c102570 0x7f963c197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:40.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.686+0000 7f9641cb3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f963c102ef0 0x7f963c19bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:40.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.686+0000 7f9641cb3640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f963c110f80 con 0x7f963c102570 2024-09-19T08:46:40.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.686+0000 7f9641cb3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f963c110e00 con 0x7f963c102ef0 2024-09-19T08:46:40.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.686+0000 7f9641cb3640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f963c111100 con 0x7f963c100e20 2024-09-19T08:46:40.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.686+0000 7f963affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f963c102570 0x7f963c197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:40.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.687+0000 7f963affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f963c102570 0x7f963c197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45618/0 (socket says 172.21.15.164:45618) 2024-09-19T08:46:40.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.687+0000 7f963affd640 1 -- 172.21.15.164:0/786152143 learned_addr learned my addr 172.21.15.164:0/786152143 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:40.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.687+0000 7f963bfff640 1 --2- 172.21.15.164:0/786152143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f963c102ef0 0x7f963c19bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:40.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.687+0000 7f963bfff640 1 -- 172.21.15.164:0/786152143 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f963c100e20 msgr2=0x7f963c1973e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T08:46:40.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.687+0000 7f963bfff640 1 --2- 172.21.15.164:0/786152143 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f963c100e20 0x7f963c1973e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.687+0000 7f963bfff640 1 -- 172.21.15.164:0/786152143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f963c102570 msgr2=0x7f963c197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:40.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.687+0000 7f963bfff640 1 --2- 172.21.15.164:0/786152143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f963c102570 0x7f963c197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.687+0000 7f963bfff640 1 -- 172.21.15.164:0/786152143 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f963c19c390 con 0x7f963c102ef0 2024-09-19T08:46:40.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.687+0000 7f963affd640 1 --2- 172.21.15.164:0/786152143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f963c102570 0x7f963c197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:46:40.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.687+0000 7f963bfff640 1 --2- 172.21.15.164:0/786152143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f963c102ef0 0x7f963c19bcb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9634009fd0 tx=0x7f963400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:40.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.687+0000 7f9638ff9640 1 -- 172.21.15.164:0/786152143 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963400ce20 con 0x7f963c102ef0 2024-09-19T08:46:40.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.688+0000 7f9638ff9640 1 -- 172.21.15.164:0/786152143 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9634005020 con 0x7f963c102ef0 2024-09-19T08:46:40.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.688+0000 7f9641cb3640 1 -- 172.21.15.164:0/786152143 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f963c19c680 con 0x7f963c102ef0 2024-09-19T08:46:40.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.688+0000 7f9638ff9640 1 -- 172.21.15.164:0/786152143 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9634010720 con 0x7f963c102ef0 2024-09-19T08:46:40.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.688+0000 7f9641cb3640 1 -- 172.21.15.164:0/786152143 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f963c1a3f60 con 0x7f963c102ef0 2024-09-19T08:46:40.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.690+0000 7f9638ff9640 1 -- 172.21.15.164:0/786152143 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7f9634005540 con 0x7f963c102ef0 2024-09-19T08:46:40.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.691+0000 7f9638ff9640 1 --2- 172.21.15.164:0/786152143 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f96240a34d0 0x7f96240a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:40.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.691+0000 7f964408c740 1 Processor -- start 2024-09-19T08:46:40.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.691+0000 7f9638ff9640 1 -- 172.21.15.164:0/786152143 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f96340d05a0 con 0x7f963c102ef0 2024-09-19T08:46:40.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.691+0000 7f963b7fe640 1 --2- 172.21.15.164:0/786152143 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f96240a34d0 0x7f96240a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:40.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.691+0000 7f964408c740 1 -- start start 2024-09-19T08:46:40.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.691+0000 7f964408c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f750a1fb40 0x55f750a1e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:40.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.692+0000 7f964408c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f750a1e5d0 0x55f750a1ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:40.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.692+0000 7f964408c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f750a25350 0x55f750a257c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:40.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.692+0000 7f964408c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f7509842f0 con 0x55f750a1e5d0 2024-09-19T08:46:40.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.692+0000 7f964408c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f75092bcf0 con 0x55f750a25350 2024-09-19T08:46:40.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.692+0000 7f964408c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f7509802b0 con 0x55f750a1fb40 2024-09-19T08:46:40.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.692+0000 7f963affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f750a1e5d0 0x55f750a1ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:40.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.692+0000 7f963b7fe640 1 --2- 172.21.15.164:0/786152143 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f96240a34d0 0x7f96240a5990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f962c005fd0 tx=0x7f962c005c70 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:40.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.692+0000 7f963affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f750a1e5d0 0x55f750a1ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45632/0 (socket says 172.21.15.164:45632) 2024-09-19T08:46:40.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.692+0000 7f963affd640 1 -- 172.21.15.164:0/4105985843 learned_addr learned my addr 172.21.15.164:0/4105985843 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:40.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.692+0000 7f963b7fe640 1 --2- 172.21.15.164:0/4105985843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f750a1fb40 0x55f750a1e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:40.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.692+0000 7f963bfff640 1 --2- 172.21.15.164:0/4105985843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f750a25350 0x55f750a257c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:40.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.692+0000 7f963affd640 1 -- 172.21.15.164:0/4105985843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f750a1fb40 msgr2=0x55f750a1e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:40.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.692+0000 7f963affd640 1 --2- 172.21.15.164:0/4105985843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f750a1fb40 0x55f750a1e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.692+0000 7f963affd640 1 -- 172.21.15.164:0/4105985843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f750a25350 msgr2=0x55f750a257c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:40.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.692+0000 7f963affd640 1 --2- 172.21.15.164:0/4105985843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f750a25350 0x55f750a257c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.692+0000 7f963affd640 1 -- 172.21.15.164:0/4105985843 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7509103a0 con 0x55f750a1e5d0 2024-09-19T08:46:40.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.692+0000 7f963bfff640 1 --2- 172.21.15.164:0/4105985843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f750a25350 0x55f750a257c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:40.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.692+0000 7f963affd640 1 --2- 172.21.15.164:0/4105985843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f750a1e5d0 0x55f750a1ea10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9630000c00 tx=0x7f9630004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb2ec958ec92821a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:40.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.692+0000 7f9621ffb640 1 -- 172.21.15.164:0/4105985843 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9630040070 con 0x55f750a1e5d0 2024-09-19T08:46:40.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.693+0000 7f9621ffb640 1 -- 172.21.15.164:0/4105985843 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9630003cd0 con 0x55f750a1e5d0 2024-09-19T08:46:40.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.693+0000 7f964408c740 1 -- 172.21.15.164:0/4105985843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f750a1e5d0 msgr2=0x55f750a1ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:40.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.693+0000 7f964408c740 1 --2- 172.21.15.164:0/4105985843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f750a1e5d0 0x55f750a1ea10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9630000c00 tx=0x7f9630004470 comp rx=0 tx=0).stop 2024-09-19T08:46:40.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.693+0000 7f964408c740 1 -- 172.21.15.164:0/4105985843 shutdown_connections 2024-09-19T08:46:40.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.693+0000 7f964408c740 1 --2- 172.21.15.164:0/4105985843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f750a25350 0x55f750a257c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.693+0000 7f964408c740 1 --2- 172.21.15.164:0/4105985843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f750a1e5d0 0x55f750a1ea10 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.694+0000 7f964408c740 1 --2- 172.21.15.164:0/4105985843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f750a1fb40 0x55f750a1e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.694+0000 7f964408c740 1 -- 172.21.15.164:0/4105985843 >> 172.21.15.164:0/4105985843 conn(0x55f7509e7310 msgr2=0x55f750a1ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:40.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.694+0000 7f964408c740 1 -- 172.21.15.164:0/4105985843 shutdown_connections 2024-09-19T08:46:40.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.694+0000 7f964408c740 1 -- 172.21.15.164:0/4105985843 wait complete. 2024-09-19T08:46:40.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.694+0000 7f964408c740 1 -- start start 2024-09-19T08:46:40.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.694+0000 7f964408c740 1 Processor -- start 2024-09-19T08:46:40.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.695+0000 7f964408c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f750a1e5d0 0x55f750a30f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:40.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.695+0000 7f964408c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f750a1fb40 0x55f750a291b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:40.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.695+0000 7f964408c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f750a25350 0x55f750a296f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:40.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.695+0000 7f964408c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f750a26090 con 0x55f750a1fb40 2024-09-19T08:46:40.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.695+0000 7f963affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f750a1fb40 0x55f750a291b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:40.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.695+0000 7f963affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f750a1fb40 0x55f750a291b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45644/0 (socket says 172.21.15.164:45644) 2024-09-19T08:46:40.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.695+0000 7f963affd640 1 -- 172.21.15.164:0/3781710651 learned_addr learned my addr 172.21.15.164:0/3781710651 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:40.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.695+0000 7f964408c740 1 -- 172.21.15.164:0/3781710651 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f75093f060 con 0x55f750a25350 2024-09-19T08:46:40.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.695+0000 7f963bfff640 1 --2- 172.21.15.164:0/3781710651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f750a25350 0x55f750a296f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:40.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.695+0000 7f964408c740 1 -- 172.21.15.164:0/3781710651 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f750a26210 con 0x55f750a1e5d0 2024-09-19T08:46:40.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.696+0000 7f964408c740 10 client.0 _ll_register_callbacks cb 0x55f7506928c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:40.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.696+0000 7f963b7fe640 1 --2- 172.21.15.164:0/3781710651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f750a1e5d0 0x55f750a30f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:40.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.696+0000 7f963affd640 1 -- 172.21.15.164:0/3781710651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f750a1e5d0 msgr2=0x55f750a30f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:40.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.696+0000 7f963affd640 1 --2- 172.21.15.164:0/3781710651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f750a1e5d0 0x55f750a30f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.696+0000 7f963affd640 1 -- 172.21.15.164:0/3781710651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f750a25350 msgr2=0x55f750a296f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:40.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.696+0000 7f963affd640 1 --2- 172.21.15.164:0/3781710651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f750a25350 0x55f750a296f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.696+0000 7f963affd640 1 -- 172.21.15.164:0/3781710651 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7506f53c0 con 0x55f750a1fb40 2024-09-19T08:46:40.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.696+0000 7f963bfff640 1 --2- 172.21.15.164:0/3781710651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f750a25350 0x55f750a296f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:46:40.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.696+0000 7f963affd640 1 --2- 172.21.15.164:0/3781710651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f750a1fb40 0x55f750a291b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9630031750 tx=0x7f963002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15bd9f0cd133f8b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:40.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.696+0000 7f961affd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:46:40.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.696+0000 7f961affd640 1 -- 172.21.15.164:0/3781710651 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9630040070 con 0x55f750a1fb40 2024-09-19T08:46:40.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.696+0000 7f961affd640 1 -- 172.21.15.164:0/3781710651 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f9630003800 con 0x55f750a1fb40 2024-09-19T08:46:40.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.696+0000 7f964408c740 1 -- 172.21.15.164:0/3781710651 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9630003cd0 con 0x55f750a1fb40 2024-09-19T08:46:40.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.697+0000 7f961affd640 1 -- 172.21.15.164:0/3781710651 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9630009520 con 0x55f750a1fb40 2024-09-19T08:46:40.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.697+0000 7f964408c740 10 client.15663 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:40.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.697+0000 7f964408c740 1 -- 172.21.15.164:0/3781710651 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f750a40250 con 0x55f750a1fb40 2024-09-19T08:46:40.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.697+0000 7f961affd640 1 -- 172.21.15.164:0/3781710651 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9630003cd0 con 0x55f750a1fb40 2024-09-19T08:46:40.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.697+0000 7f961affd640 1 -- 172.21.15.164:0/3781710651 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9630057030 con 0x55f750a1fb40 2024-09-19T08:46:40.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.697+0000 7f964408c740 10 client.15663 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:40.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.697+0000 7f964408c740 10 client.15663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:46:40.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.697+0000 7f964408c740 20 client.15663 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:40.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.697+0000 7f964408c740 1 --2- 172.21.15.164:0/3781710651 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55f750a415a0 0x55f750a43990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:40.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.697+0000 7f964408c740 4 client.15663 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:46:40.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.697+0000 7f964408c740 1 -- 172.21.15.164:0/3781710651 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f750991860 con 0x55f750a415a0 2024-09-19T08:46:40.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.697+0000 7f963b7fe640 1 --2- 172.21.15.164:0/3781710651 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55f750a415a0 0x55f750a43990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:40.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.698+0000 7f963b7fe640 1 --2- 172.21.15.164:0/3781710651 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55f750a415a0 0x55f750a43990 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b256459fb29382b9 server_cookie=2afa06a85daed597 in_seq=0 out_seq=0 2024-09-19T08:46:40.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.698+0000 7f961affd640 10 client.15663 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:46:40.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.698+0000 7f961affd640 1 -- 172.21.15.164:0/3781710651 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f750991860 con 0x55f750a415a0 2024-09-19T08:46:40.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.698+0000 7f961affd640 10 client.15663 handle_command_reply: tid=0 2024-09-19T08:46:40.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.699+0000 7f964408c740 1 client.15663 shutdown 2024-09-19T08:46:40.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.699+0000 7f964408c740 1 -- 172.21.15.164:0/3781710651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f750a1fb40 msgr2=0x55f750a291b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:40.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.699+0000 7f964408c740 1 --2- 172.21.15.164:0/3781710651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f750a1fb40 0x55f750a291b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9630031750 tx=0x7f963002a480 comp rx=0 tx=0).stop 2024-09-19T08:46:40.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.699+0000 7f964408c740 1 -- 172.21.15.164:0/3781710651 shutdown_connections 2024-09-19T08:46:40.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.699+0000 7f964408c740 1 --2- 172.21.15.164:0/3781710651 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55f750a415a0 0x55f750a43990 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.699+0000 7f964408c740 1 --2- 172.21.15.164:0/3781710651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f750a25350 0x55f750a296f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.699+0000 7f964408c740 1 --2- 172.21.15.164:0/3781710651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f750a1fb40 0x55f750a291b0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.699+0000 7f964408c740 1 --2- 172.21.15.164:0/3781710651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f750a1e5d0 0x55f750a30f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.699+0000 7f964408c740 1 -- 172.21.15.164:0/3781710651 >> 172.21.15.164:0/3781710651 conn(0x55f7509e7310 msgr2=0x55f750a20960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:40.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.699+0000 7f961affd640 0 client.15663 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:46:40.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.699+0000 7f964408c740 1 -- 172.21.15.164:0/3781710651 shutdown_connections 2024-09-19T08:46:40.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.699+0000 7f964408c740 1 -- 172.21.15.164:0/3781710651 wait complete. 2024-09-19T08:46:40.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.699+0000 7f964408c740 20 client.15663 trim_cache size 0 max 16384 2024-09-19T08:46:40.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.712+0000 7f964408c740 1 Processor -- start 2024-09-19T08:46:40.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.712+0000 7f964408c740 1 -- start start 2024-09-19T08:46:40.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.712+0000 7f964408c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f750a1e5d0 0x55f750a26d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:40.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.712+0000 7f964408c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f750a1fb40 0x55f750a248b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:40.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.712+0000 7f964408c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f750a25350 0x55f750a230b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:40.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.713+0000 7f964408c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f963c1ab4e0 con 0x55f750a25350 2024-09-19T08:46:40.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.713+0000 7f963bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f750a25350 0x55f750a230b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:40.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.713+0000 7f963bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f750a25350 0x55f750a230b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45650/0 (socket says 172.21.15.164:45650) 2024-09-19T08:46:40.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.713+0000 7f963bfff640 1 -- 172.21.15.164:0/3501317235 learned_addr learned my addr 172.21.15.164:0/3501317235 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:40.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.713+0000 7f964408c740 1 -- 172.21.15.164:0/3501317235 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f963c1ad020 con 0x55f750a1e5d0 2024-09-19T08:46:40.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.713+0000 7f964408c740 1 -- 172.21.15.164:0/3501317235 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f963c1abbb0 con 0x55f750a1fb40 2024-09-19T08:46:40.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.713+0000 7f963b7fe640 1 --2- 172.21.15.164:0/3501317235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f750a1e5d0 0x55f750a26d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:40.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.713+0000 7f963affd640 1 --2- 172.21.15.164:0/3501317235 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f750a1fb40 0x55f750a248b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:40.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.713+0000 7f963bfff640 1 -- 172.21.15.164:0/3501317235 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f750a1fb40 msgr2=0x55f750a248b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:40.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.713+0000 7f963bfff640 1 --2- 172.21.15.164:0/3501317235 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f750a1fb40 0x55f750a248b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.713+0000 7f963bfff640 1 -- 172.21.15.164:0/3501317235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f750a1e5d0 msgr2=0x55f750a26d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:40.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.713+0000 7f963bfff640 1 --2- 172.21.15.164:0/3501317235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f750a1e5d0 0x55f750a26d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.713+0000 7f963bfff640 1 -- 172.21.15.164:0/3501317235 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f750a2a3b0 con 0x55f750a25350 2024-09-19T08:46:40.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.713+0000 7f963bfff640 1 --2- 172.21.15.164:0/3501317235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f750a25350 0x55f750a230b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f96340cdcf0 tx=0x7f9634098300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b23b1b39c7f69c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:40.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.713+0000 7f96217fa640 1 -- 172.21.15.164:0/3501317235 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96340b4020 con 0x55f750a25350 2024-09-19T08:46:40.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.714+0000 7f96217fa640 1 -- 172.21.15.164:0/3501317235 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96340a5310 con 0x55f750a25350 2024-09-19T08:46:40.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.714+0000 7f96217fa640 1 -- 172.21.15.164:0/3501317235 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96340a50c0 con 0x55f750a25350 2024-09-19T08:46:40.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.714+0000 7f964408c740 1 -- 172.21.15.164:0/3501317235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f750a25350 msgr2=0x55f750a230b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:40.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.714+0000 7f964408c740 1 --2- 172.21.15.164:0/3501317235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f750a25350 0x55f750a230b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f96340cdcf0 tx=0x7f9634098300 comp rx=0 tx=0).stop 2024-09-19T08:46:40.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.714+0000 7f964408c740 1 -- 172.21.15.164:0/3501317235 shutdown_connections 2024-09-19T08:46:40.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.714+0000 7f964408c740 1 --2- 172.21.15.164:0/3501317235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f750a25350 0x55f750a230b0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.714+0000 7f964408c740 1 --2- 172.21.15.164:0/3501317235 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f750a1fb40 0x55f750a248b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.714+0000 7f964408c740 1 --2- 172.21.15.164:0/3501317235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f750a1e5d0 0x55f750a26d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.714+0000 7f964408c740 1 -- 172.21.15.164:0/3501317235 >> 172.21.15.164:0/3501317235 conn(0x55f7509e7310 msgr2=0x55f750a241a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:40.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.714+0000 7f964408c740 1 -- 172.21.15.164:0/3501317235 shutdown_connections 2024-09-19T08:46:40.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.715+0000 7f964408c740 1 -- 172.21.15.164:0/3501317235 wait complete. 2024-09-19T08:46:40.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.715+0000 7f964408c740 1 -- start start 2024-09-19T08:46:40.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.715+0000 7f964408c740 1 Processor -- start 2024-09-19T08:46:40.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.716+0000 7f964408c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f750a1e5d0 0x55f750aa4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:40.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.716+0000 7f964408c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f750a1fb40 0x55f750aa4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:40.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.716+0000 7f964408c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f750a25350 0x55f750a46130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:40.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.716+0000 7f964408c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f963c1ac950 con 0x55f750a25350 2024-09-19T08:46:40.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.716+0000 7f964408c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f963c1ac280 con 0x55f750a1e5d0 2024-09-19T08:46:40.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.716+0000 7f964408c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f963c1a9540 con 0x55f750a1fb40 2024-09-19T08:46:40.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.716+0000 7f963b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f750a1e5d0 0x55f750aa4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:40.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.716+0000 7f963affd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f750a1fb40 0x55f750aa4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:40.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.716+0000 7f964408c740 10 client.0 _ll_register_callbacks cb 0x7f9630005e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:40.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.716+0000 7f963bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f750a25350 0x55f750a46130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:40.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.716+0000 7f963affd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f750a1fb40 0x55f750aa4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:42646/0 (socket says 172.21.15.164:42646) 2024-09-19T08:46:40.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.716+0000 7f963b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f750a1e5d0 0x55f750aa4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:51530/0 (socket says 172.21.15.164:51530) 2024-09-19T08:46:40.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.716+0000 7f963affd640 1 -- 172.21.15.164:0/713881534 learned_addr learned my addr 172.21.15.164:0/713881534 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:40.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.716+0000 7f963b7fe640 1 -- 172.21.15.164:0/713881534 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f750a1fb40 msgr2=0x55f750aa4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:40.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.716+0000 7f963b7fe640 1 --2- 172.21.15.164:0/713881534 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f750a1fb40 0x55f750aa4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.716+0000 7f963b7fe640 1 -- 172.21.15.164:0/713881534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f750a25350 msgr2=0x55f750a46130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:40.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.716+0000 7f963b7fe640 1 --2- 172.21.15.164:0/713881534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f750a25350 0x55f750a46130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.716+0000 7f963b7fe640 1 -- 172.21.15.164:0/713881534 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f750a1a4f0 con 0x55f750a1e5d0 2024-09-19T08:46:40.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.716+0000 7f963bfff640 1 --2- 172.21.15.164:0/713881534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f750a25350 0x55f750a46130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:46:40.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.717+0000 7f963b7fe640 1 --2- 172.21.15.164:0/713881534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f750a1e5d0 0x55f750aa4750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9630031c10 tx=0x7f962c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2883ef2a39864cff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:40.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.717+0000 7f961affd640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-19T08:46:40.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.717+0000 7f961affd640 1 -- 172.21.15.164:0/713881534 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c004220 con 0x55f750a1e5d0 2024-09-19T08:46:40.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.717+0000 7f964408c740 1 -- 172.21.15.164:0/713881534 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96340a5310 con 0x55f750a1e5d0 2024-09-19T08:46:40.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.717+0000 7f961affd640 1 -- 172.21.15.164:0/713881534 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f962c0043c0 con 0x55f750a1e5d0 2024-09-19T08:46:40.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.717+0000 7f961affd640 1 -- 172.21.15.164:0/713881534 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f962c0035f0 con 0x55f750a1e5d0 2024-09-19T08:46:40.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.717+0000 7f964408c740 10 client.25214 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:40.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.717+0000 7f964408c740 1 -- 172.21.15.164:0/713881534 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f750aabe20 con 0x55f750a1e5d0 2024-09-19T08:46:40.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.717+0000 7f961affd640 1 -- 172.21.15.164:0/713881534 <== mon.2 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96340a5310 con 0x55f750a1e5d0 2024-09-19T08:46:40.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.717+0000 7f961affd640 1 -- 172.21.15.164:0/713881534 <== mon.2 v2:172.21.15.170:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f962c020870 con 0x55f750a1e5d0 2024-09-19T08:46:40.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.717+0000 7f964408c740 10 client.25214 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:40.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.717+0000 7f964408c740 10 client.25214 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:46:40.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.717+0000 7f964408c740 20 client.25214 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:40.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.718+0000 7f964408c740 1 --2- 172.21.15.164:0/713881534 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55f750aac5a0 0x55f750aae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:40.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.718+0000 7f964408c740 4 client.25214 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:46:40.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.718+0000 7f964408c740 1 -- 172.21.15.164:0/713881534 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f75098bd70 con 0x55f750aac5a0 2024-09-19T08:46:40.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.718+0000 7f963affd640 1 --2- 172.21.15.164:0/713881534 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55f750aac5a0 0x55f750aae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:40.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.718+0000 7f961affd640 10 client.25214 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:46:40.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.718+0000 7f963affd640 1 --2- 172.21.15.164:0/713881534 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55f750aac5a0 0x55f750aae9c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4942abf24b77257 server_cookie=1e0d382a73905c43 in_seq=0 out_seq=0 2024-09-19T08:46:40.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.719+0000 7f961affd640 1 -- 172.21.15.164:0/713881534 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f75098bd70 con 0x55f750aac5a0 2024-09-19T08:46:40.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.719+0000 7f961affd640 10 client.25214 handle_command_reply: tid=0 2024-09-19T08:46:40.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.719+0000 7f964408c740 1 client.25214 shutdown 2024-09-19T08:46:40.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.719+0000 7f964408c740 1 -- 172.21.15.164:0/713881534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f750a1e5d0 msgr2=0x55f750aa4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:40.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.719+0000 7f964408c740 1 --2- 172.21.15.164:0/713881534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f750a1e5d0 0x55f750aa4750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9630031c10 tx=0x7f962c0046c0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.719+0000 7f964408c740 1 -- 172.21.15.164:0/713881534 shutdown_connections 2024-09-19T08:46:40.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.719+0000 7f964408c740 1 --2- 172.21.15.164:0/713881534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f750a25350 0x55f750a46130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.719+0000 7f964408c740 1 --2- 172.21.15.164:0/713881534 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f750a1fb40 0x55f750aa4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.719+0000 7f964408c740 1 --2- 172.21.15.164:0/713881534 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55f750aac5a0 0x55f750aae9c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.719+0000 7f964408c740 1 --2- 172.21.15.164:0/713881534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f750a1e5d0 0x55f750aa4750 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.719+0000 7f964408c740 1 -- 172.21.15.164:0/713881534 >> 172.21.15.164:0/713881534 conn(0x55f7509e7310 msgr2=0x55f750a19d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:40.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.719+0000 7f961affd640 0 client.25214 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:46:40.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.719+0000 7f964408c740 1 -- 172.21.15.164:0/713881534 shutdown_connections 2024-09-19T08:46:40.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.719+0000 7f964408c740 1 -- 172.21.15.164:0/713881534 wait complete. 2024-09-19T08:46:40.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.719+0000 7f964408c740 20 client.25214 trim_cache size 0 max 16384 2024-09-19T08:46:40.739 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:46:40.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.720+0000 7f9641cb3640 1 -- 172.21.15.164:0/786152143 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f96240a34d0 msgr2=0x7f96240a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:40.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.720+0000 7f9641cb3640 1 --2- 172.21.15.164:0/786152143 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f96240a34d0 0x7f96240a5990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f962c005fd0 tx=0x7f962c005c70 comp rx=0 tx=0).stop 2024-09-19T08:46:40.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.720+0000 7f9641cb3640 1 -- 172.21.15.164:0/786152143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f963c102ef0 msgr2=0x7f963c19bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:40.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.720+0000 7f9641cb3640 1 --2- 172.21.15.164:0/786152143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f963c102ef0 0x7f963c19bcb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9634009fd0 tx=0x7f963400ef10 comp rx=0 tx=0).stop 2024-09-19T08:46:40.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.720+0000 7f9641cb3640 1 -- 172.21.15.164:0/786152143 shutdown_connections 2024-09-19T08:46:40.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.720+0000 7f9641cb3640 1 --2- 172.21.15.164:0/786152143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f963c102ef0 0x7f963c19bcb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.720+0000 7f9641cb3640 1 --2- 172.21.15.164:0/786152143 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f96240a34d0 0x7f96240a5990 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.720+0000 7f9641cb3640 1 --2- 172.21.15.164:0/786152143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f963c102570 0x7f963c197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.720+0000 7f9641cb3640 1 --2- 172.21.15.164:0/786152143 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f963c100e20 0x7f963c1973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.720+0000 7f9641cb3640 1 -- 172.21.15.164:0/786152143 >> 172.21.15.164:0/786152143 conn(0x7f963c0fc7d0 msgr2=0x7f963c1068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:40.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.721+0000 7f9641cb3640 1 -- 172.21.15.164:0/786152143 shutdown_connections 2024-09-19T08:46:40.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:40.721+0000 7f9641cb3640 1 -- 172.21.15.164:0/786152143 wait complete. 2024-09-19T08:46:42.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:41 smithi164 ceph-mon[30928]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-19T08:46:42.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:41 smithi170 ceph-mon[35072]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-19T08:46:42.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:41 smithi173 ceph-mon[35306]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-19T08:46:42.630 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T08:46:42.630 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T08:46:42.960 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:46:44.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:43 smithi164 ceph-mon[30928]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-19T08:46:44.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:43 smithi170 ceph-mon[35072]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-19T08:46:44.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:43 smithi173 ceph-mon[35306]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-19T08:46:44.922 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:46:46.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:45 smithi164 ceph-mon[30928]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-19T08:46:46.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:45 smithi170 ceph-mon[35072]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-19T08:46:46.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:45 smithi173 ceph-mon[35306]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-19T08:46:46.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.282+0000 7f8848f48640 1 -- 172.21.15.164:0/4180747425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8844100100 msgr2=0x7f8844100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:46.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.282+0000 7f8848f48640 1 --2- 172.21.15.164:0/4180747425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8844100100 0x7f8844100560 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f882c009900 tx=0x7f882c02a410 comp rx=0 tx=0).stop 2024-09-19T08:46:46.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.283+0000 7f8848f48640 1 -- 172.21.15.164:0/4180747425 shutdown_connections 2024-09-19T08:46:46.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.283+0000 7f8848f48640 1 --2- 172.21.15.164:0/4180747425 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8844100aa0 0x7f884410ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:46.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.283+0000 7f8848f48640 1 --2- 172.21.15.164:0/4180747425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8844100100 0x7f8844100560 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:46.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.283+0000 7f8848f48640 1 --2- 172.21.15.164:0/4180747425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8844106b80 0x7f8844106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:46.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.283+0000 7f8848f48640 1 -- 172.21.15.164:0/4180747425 >> 172.21.15.164:0/4180747425 conn(0x7f8844077230 msgr2=0x7f88440fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:46.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.283+0000 7f8848f48640 1 -- 172.21.15.164:0/4180747425 shutdown_connections 2024-09-19T08:46:46.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.283+0000 7f8848f48640 1 -- 172.21.15.164:0/4180747425 wait complete. 2024-09-19T08:46:46.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.284+0000 7f8848f48640 1 Processor -- start 2024-09-19T08:46:46.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.284+0000 7f8848f48640 1 -- start start 2024-09-19T08:46:46.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.284+0000 7f8848f48640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8844100100 0x7f88441984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:46.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.284+0000 7f8848f48640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8844100aa0 0x7f8844198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:46.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.284+0000 7f8848f48640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8844106b80 0x7f884419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:46.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.284+0000 7f8848f48640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8844112060 con 0x7f8844100aa0 2024-09-19T08:46:46.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.284+0000 7f8848f48640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8844111ee0 con 0x7f8844106b80 2024-09-19T08:46:46.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.284+0000 7f8848f48640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f88441121e0 con 0x7f8844100100 2024-09-19T08:46:46.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.284+0000 7f883bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8844100aa0 0x7f8844198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:46.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.284+0000 7f884259c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8844100100 0x7f88441984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:46.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.284+0000 7f883bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8844100aa0 0x7f8844198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45690/0 (socket says 172.21.15.164:45690) 2024-09-19T08:46:46.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.284+0000 7f884259c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8844100100 0x7f88441984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:48976/0 (socket says 172.21.15.164:48976) 2024-09-19T08:46:46.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.284+0000 7f883bfff640 1 -- 172.21.15.164:0/2133515039 learned_addr learned my addr 172.21.15.164:0/2133515039 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:46.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.284+0000 7f8842d9d640 1 --2- 172.21.15.164:0/2133515039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8844106b80 0x7f884419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:46.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.284+0000 7f883bfff640 1 -- 172.21.15.164:0/2133515039 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8844100100 msgr2=0x7f88441984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:46.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.284+0000 7f883bfff640 1 --2- 172.21.15.164:0/2133515039 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8844100100 0x7f88441984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:46.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.284+0000 7f883bfff640 1 -- 172.21.15.164:0/2133515039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8844106b80 msgr2=0x7f884419cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:46.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.284+0000 7f883bfff640 1 --2- 172.21.15.164:0/2133515039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8844106b80 0x7f884419cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:46.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.284+0000 7f883bfff640 1 -- 172.21.15.164:0/2133515039 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f884419d490 con 0x7f8844100aa0 2024-09-19T08:46:46.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.284+0000 7f8842d9d640 1 --2- 172.21.15.164:0/2133515039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8844106b80 0x7f884419cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:46.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.285+0000 7f883bfff640 1 --2- 172.21.15.164:0/2133515039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8844100aa0 0x7f8844198a20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f882c004770 tx=0x7f882c003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:46.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.285+0000 7f883b7fe640 1 -- 172.21.15.164:0/2133515039 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c03f070 con 0x7f8844100aa0 2024-09-19T08:46:46.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.285+0000 7f8848f48640 1 -- 172.21.15.164:0/2133515039 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f884419d720 con 0x7f8844100aa0 2024-09-19T08:46:46.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.285+0000 7f8848f48640 1 -- 172.21.15.164:0/2133515039 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8844101210 con 0x7f8844100aa0 2024-09-19T08:46:46.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.285+0000 7f883b7fe640 1 -- 172.21.15.164:0/2133515039 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f882c031c90 con 0x7f8844100aa0 2024-09-19T08:46:46.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.285+0000 7f883b7fe640 1 -- 172.21.15.164:0/2133515039 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c035030 con 0x7f8844100aa0 2024-09-19T08:46:46.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.288+0000 7f884b321740 1 Processor -- start 2024-09-19T08:46:46.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.289+0000 7f884b321740 1 -- start start 2024-09-19T08:46:46.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.289+0000 7f883b7fe640 1 -- 172.21.15.164:0/2133515039 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7f882c042070 con 0x7f8844100aa0 2024-09-19T08:46:46.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.289+0000 7f884b321740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ba8f18b60 0x557ba8f18f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:46.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.289+0000 7f884b321740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ba8f19480 0x557ba8f23800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:46.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.289+0000 7f884b321740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ba8f23d40 0x557ba8f26130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:46.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.289+0000 7f884b321740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557ba8e7e280 con 0x557ba8f23d40 2024-09-19T08:46:46.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.289+0000 7f884b321740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x557ba8e25b10 con 0x557ba8f18b60 2024-09-19T08:46:46.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.289+0000 7f884b321740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x557ba8e7a240 con 0x557ba8f19480 2024-09-19T08:46:46.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.289+0000 7f8842d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ba8f18b60 0x557ba8f18f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:46.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.289+0000 7f884259c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ba8f19480 0x557ba8f23800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:46.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.289+0000 7f883bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ba8f23d40 0x557ba8f26130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:46.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.289+0000 7f8842d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ba8f18b60 0x557ba8f18f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:44350/0 (socket says 172.21.15.164:44350) 2024-09-19T08:46:46.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.289+0000 7f8842d9d640 1 -- 172.21.15.164:0/2550219089 learned_addr learned my addr 172.21.15.164:0/2550219089 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:46.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.290+0000 7f8842d9d640 1 -- 172.21.15.164:0/2550219089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ba8f19480 msgr2=0x557ba8f23800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:46.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.290+0000 7f8842d9d640 1 --2- 172.21.15.164:0/2550219089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ba8f19480 0x557ba8f23800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:46.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.290+0000 7f8842d9d640 1 -- 172.21.15.164:0/2550219089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ba8f23d40 msgr2=0x557ba8f26130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:46.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.290+0000 7f8842d9d640 1 --2- 172.21.15.164:0/2550219089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ba8f23d40 0x557ba8f26130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:46.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.290+0000 7f8842d9d640 1 -- 172.21.15.164:0/2550219089 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ba8e321f0 con 0x557ba8f18b60 2024-09-19T08:46:46.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.290+0000 7f883bfff640 1 --2- 172.21.15.164:0/2550219089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ba8f23d40 0x557ba8f26130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:46:46.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.290+0000 7f8842d9d640 1 --2- 172.21.15.164:0/2550219089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ba8f18b60 0x557ba8f18f40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f883c010480 tx=0x7f883c009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ab10358cbe2e65a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:46.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.290+0000 7f8838ff9640 1 -- 172.21.15.164:0/2550219089 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c011760 con 0x557ba8f18b60 2024-09-19T08:46:46.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.290+0000 7f8838ff9640 1 -- 172.21.15.164:0/2550219089 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f883c011de0 con 0x557ba8f18b60 2024-09-19T08:46:46.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.290+0000 7f8838ff9640 1 -- 172.21.15.164:0/2550219089 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c01a380 con 0x557ba8f18b60 2024-09-19T08:46:46.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.291+0000 7f883b7fe640 1 --2- 172.21.15.164:0/2133515039 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f88300a3550 0x7f88300a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:46.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.291+0000 7f884b321740 1 -- 172.21.15.164:0/2550219089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ba8f18b60 msgr2=0x557ba8f18f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:46.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.291+0000 7f884b321740 1 --2- 172.21.15.164:0/2550219089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ba8f18b60 0x557ba8f18f40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f883c010480 tx=0x7f883c009f90 comp rx=0 tx=0).stop 2024-09-19T08:46:46.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.291+0000 7f884b321740 1 -- 172.21.15.164:0/2550219089 shutdown_connections 2024-09-19T08:46:46.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.291+0000 7f884b321740 1 --2- 172.21.15.164:0/2550219089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ba8f23d40 0x557ba8f26130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:46.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.291+0000 7f884b321740 1 --2- 172.21.15.164:0/2550219089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ba8f19480 0x557ba8f23800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:46.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.291+0000 7f884b321740 1 --2- 172.21.15.164:0/2550219089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ba8f18b60 0x557ba8f18f40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:46.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.291+0000 7f884b321740 1 -- 172.21.15.164:0/2550219089 >> 172.21.15.164:0/2550219089 conn(0x557ba8f150f0 msgr2=0x557ba8f154d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:46.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.291+0000 7f883b7fe640 1 -- 172.21.15.164:0/2133515039 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f882c0f0640 con 0x7f8844100aa0 2024-09-19T08:46:46.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.292+0000 7f884259c640 1 --2- 172.21.15.164:0/2133515039 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f88300a3550 0x7f88300a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:46.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.292+0000 7f884b321740 1 -- 172.21.15.164:0/2550219089 shutdown_connections 2024-09-19T08:46:46.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.293+0000 7f884259c640 1 --2- 172.21.15.164:0/2133515039 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f88300a3550 0x7f88300a5a10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x557ba8f15f50 tx=0x7f883400b410 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:46.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.293+0000 7f884b321740 1 -- 172.21.15.164:0/2550219089 wait complete. 2024-09-19T08:46:46.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.293+0000 7f884b321740 1 -- start start 2024-09-19T08:46:46.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.293+0000 7f884b321740 1 Processor -- start 2024-09-19T08:46:46.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.294+0000 7f884b321740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ba8f18b60 0x557ba8f20c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:46.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.294+0000 7f884b321740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ba8f19480 0x557ba8f21190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:46.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.294+0000 7f884b321740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ba8f23d40 0x557ba8f216d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:46.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.294+0000 7f884b321740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557ba8f26910 con 0x557ba8f19480 2024-09-19T08:46:46.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.294+0000 7f884b321740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x557ba8e38f10 con 0x557ba8f18b60 2024-09-19T08:46:46.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.294+0000 7f884b321740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x557ba8f26a90 con 0x557ba8f23d40 2024-09-19T08:46:46.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.294+0000 7f884b321740 10 client.0 _ll_register_callbacks cb 0x557ba8e01a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:46.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.294+0000 7f883bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ba8f19480 0x557ba8f21190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:46.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.294+0000 7f883bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ba8f19480 0x557ba8f21190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45720/0 (socket says 172.21.15.164:45720) 2024-09-19T08:46:46.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.294+0000 7f883bfff640 1 -- 172.21.15.164:0/1105173546 learned_addr learned my addr 172.21.15.164:0/1105173546 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:46.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.294+0000 7f884259c640 1 --2- 172.21.15.164:0/1105173546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ba8f18b60 0x557ba8f20c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:46.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.294+0000 7f8842d9d640 1 --2- 172.21.15.164:0/1105173546 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ba8f23d40 0x557ba8f216d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:46.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.294+0000 7f884259c640 1 -- 172.21.15.164:0/1105173546 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ba8f23d40 msgr2=0x557ba8f216d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:46.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.294+0000 7f884259c640 1 --2- 172.21.15.164:0/1105173546 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ba8f23d40 0x557ba8f216d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:46.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.294+0000 7f884259c640 1 -- 172.21.15.164:0/1105173546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ba8f19480 msgr2=0x557ba8f21190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:46.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.294+0000 7f884259c640 1 --2- 172.21.15.164:0/1105173546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ba8f19480 0x557ba8f21190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:46.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.294+0000 7f884259c640 1 -- 172.21.15.164:0/1105173546 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ba8bef6a0 con 0x557ba8f18b60 2024-09-19T08:46:46.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.295+0000 7f883bfff640 1 --2- 172.21.15.164:0/1105173546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ba8f19480 0x557ba8f21190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:46:46.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.295+0000 7f884259c640 1 --2- 172.21.15.164:0/1105173546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ba8f18b60 0x557ba8f20c50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8834005fd0 tx=0x7f8834004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=123b9fa747ef3146 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:46.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.295+0000 7f8829ffb640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-19T08:46:46.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.295+0000 7f8829ffb640 1 -- 172.21.15.164:0/1105173546 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883401c6c0 con 0x557ba8f18b60 2024-09-19T08:46:46.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.295+0000 7f884b321740 1 -- 172.21.15.164:0/1105173546 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f883c011de0 con 0x557ba8f18b60 2024-09-19T08:46:46.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.295+0000 7f8829ffb640 1 -- 172.21.15.164:0/1105173546 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f883401c860 con 0x557ba8f18b60 2024-09-19T08:46:46.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.295+0000 7f8829ffb640 1 -- 172.21.15.164:0/1105173546 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f883401ca60 con 0x557ba8f18b60 2024-09-19T08:46:46.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.295+0000 7f884b321740 10 client.25226 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:46.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.295+0000 7f884b321740 1 -- 172.21.15.164:0/1105173546 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557ba8f3ae00 con 0x557ba8f18b60 2024-09-19T08:46:46.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.295+0000 7f8829ffb640 1 -- 172.21.15.164:0/1105173546 <== mon.2 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f883c011de0 con 0x557ba8f18b60 2024-09-19T08:46:46.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.296+0000 7f8829ffb640 1 -- 172.21.15.164:0/1105173546 <== mon.2 v2:172.21.15.170:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8834026070 con 0x557ba8f18b60 2024-09-19T08:46:46.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.296+0000 7f884b321740 10 client.25226 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:46.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.296+0000 7f884b321740 10 client.25226 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:46:46.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.296+0000 7f884b321740 20 client.25226 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:46.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.296+0000 7f884b321740 1 --2- 172.21.15.164:0/1105173546 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x557ba8f3c6f0 0x557ba8f3eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:46.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.296+0000 7f884b321740 4 client.25226 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:46:46.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.296+0000 7f884b321740 1 -- 172.21.15.164:0/1105173546 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ba8e8b7f0 con 0x557ba8f3c6f0 2024-09-19T08:46:46.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.296+0000 7f883bfff640 1 --2- 172.21.15.164:0/1105173546 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x557ba8f3c6f0 0x557ba8f3eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:46.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.297+0000 7f8829ffb640 10 client.25226 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:46:46.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.297+0000 7f883bfff640 1 --2- 172.21.15.164:0/1105173546 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x557ba8f3c6f0 0x557ba8f3eb90 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adf478ade26eabec server_cookie=f8638a06bc6b4bf2 in_seq=0 out_seq=0 2024-09-19T08:46:46.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.297+0000 7f8829ffb640 1 -- 172.21.15.164:0/1105173546 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557ba8e8b7f0 con 0x557ba8f3c6f0 2024-09-19T08:46:46.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.297+0000 7f8829ffb640 10 client.25226 handle_command_reply: tid=0 2024-09-19T08:46:46.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.297+0000 7f884b321740 1 client.25226 shutdown 2024-09-19T08:46:46.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.298+0000 7f884b321740 1 -- 172.21.15.164:0/1105173546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ba8f18b60 msgr2=0x557ba8f20c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:46.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.298+0000 7f884b321740 1 --2- 172.21.15.164:0/1105173546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ba8f18b60 0x557ba8f20c50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8834005fd0 tx=0x7f8834004730 comp rx=0 tx=0).stop 2024-09-19T08:46:46.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.298+0000 7f884b321740 1 -- 172.21.15.164:0/1105173546 shutdown_connections 2024-09-19T08:46:46.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.298+0000 7f884b321740 1 --2- 172.21.15.164:0/1105173546 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ba8f23d40 0x557ba8f216d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:46.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.298+0000 7f884b321740 1 --2- 172.21.15.164:0/1105173546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ba8f19480 0x557ba8f21190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:46.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.298+0000 7f884b321740 1 --2- 172.21.15.164:0/1105173546 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x557ba8f3c6f0 0x557ba8f3eb90 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:46.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.298+0000 7f8829ffb640 0 client.25226 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:46:46.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.298+0000 7f884b321740 1 --2- 172.21.15.164:0/1105173546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ba8f18b60 0x557ba8f20c50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:46.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.298+0000 7f884b321740 1 -- 172.21.15.164:0/1105173546 >> 172.21.15.164:0/1105173546 conn(0x557ba8f150f0 msgr2=0x557ba8f1add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:46.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.298+0000 7f884b321740 1 -- 172.21.15.164:0/1105173546 shutdown_connections 2024-09-19T08:46:46.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.298+0000 7f884b321740 1 -- 172.21.15.164:0/1105173546 wait complete. 2024-09-19T08:46:46.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.298+0000 7f884b321740 20 client.25226 trim_cache size 0 max 16384 2024-09-19T08:46:46.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.332+0000 7f884b321740 1 Processor -- start 2024-09-19T08:46:46.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.332+0000 7f884b321740 1 -- start start 2024-09-19T08:46:46.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.332+0000 7f884b321740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ba8f18b60 0x557ba8f1acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:46.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.332+0000 7f884b321740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ba8f19480 0x557ba8f44460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:46.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.332+0000 7f884b321740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ba8f23d40 0x557ba8f449a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:46.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.332+0000 7f884b321740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f88441ab450 con 0x557ba8f23d40 2024-09-19T08:46:46.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.332+0000 7f884b321740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f88441acf90 con 0x557ba8f18b60 2024-09-19T08:46:46.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.332+0000 7f884b321740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f88441abb20 con 0x557ba8f19480 2024-09-19T08:46:46.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.333+0000 7f883bfff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ba8f19480 0x557ba8f44460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:46.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.333+0000 7f884259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ba8f18b60 0x557ba8f1acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:46.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.333+0000 7f884259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ba8f18b60 0x557ba8f1acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:44364/0 (socket says 172.21.15.164:44364) 2024-09-19T08:46:46.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.333+0000 7f884259c640 1 -- 172.21.15.164:0/37624184 learned_addr learned my addr 172.21.15.164:0/37624184 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:46.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.333+0000 7f8842d9d640 1 --2- 172.21.15.164:0/37624184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ba8f23d40 0x557ba8f449a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:46.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.333+0000 7f884259c640 1 -- 172.21.15.164:0/37624184 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ba8f19480 msgr2=0x557ba8f44460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:46.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.333+0000 7f884259c640 1 --2- 172.21.15.164:0/37624184 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ba8f19480 0x557ba8f44460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:46.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.333+0000 7f884259c640 1 -- 172.21.15.164:0/37624184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ba8f23d40 msgr2=0x557ba8f449a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:46.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.333+0000 7f884259c640 1 --2- 172.21.15.164:0/37624184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ba8f23d40 0x557ba8f449a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:46.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.333+0000 7f884259c640 1 -- 172.21.15.164:0/37624184 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ba8f192b0 con 0x557ba8f18b60 2024-09-19T08:46:46.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.333+0000 7f8842d9d640 1 --2- 172.21.15.164:0/37624184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ba8f23d40 0x557ba8f449a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:46.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.333+0000 7f884259c640 1 --2- 172.21.15.164:0/37624184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ba8f18b60 0x557ba8f1acc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8834012010 tx=0x7f88340057d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a677d01cee23b754 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:46.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.333+0000 7f882bfff640 1 -- 172.21.15.164:0/37624184 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8834035030 con 0x557ba8f18b60 2024-09-19T08:46:46.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.334+0000 7f882bfff640 1 -- 172.21.15.164:0/37624184 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f883401fc90 con 0x557ba8f18b60 2024-09-19T08:46:46.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.334+0000 7f882bfff640 1 -- 172.21.15.164:0/37624184 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883400ed20 con 0x557ba8f18b60 2024-09-19T08:46:46.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.334+0000 7f884b321740 1 -- 172.21.15.164:0/37624184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ba8f18b60 msgr2=0x557ba8f1acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:46.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.334+0000 7f884b321740 1 --2- 172.21.15.164:0/37624184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ba8f18b60 0x557ba8f1acc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8834012010 tx=0x7f88340057d0 comp rx=0 tx=0).stop 2024-09-19T08:46:46.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.335+0000 7f884b321740 1 -- 172.21.15.164:0/37624184 shutdown_connections 2024-09-19T08:46:46.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.335+0000 7f884b321740 1 --2- 172.21.15.164:0/37624184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ba8f23d40 0x557ba8f449a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:46.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.335+0000 7f884b321740 1 --2- 172.21.15.164:0/37624184 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ba8f19480 0x557ba8f44460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:46.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.335+0000 7f884b321740 1 --2- 172.21.15.164:0/37624184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ba8f18b60 0x557ba8f1acc0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:46.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.335+0000 7f884b321740 1 -- 172.21.15.164:0/37624184 >> 172.21.15.164:0/37624184 conn(0x557ba8f150f0 msgr2=0x557ba8f1a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:46.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.335+0000 7f884b321740 1 -- 172.21.15.164:0/37624184 shutdown_connections 2024-09-19T08:46:46.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.335+0000 7f884b321740 1 -- 172.21.15.164:0/37624184 wait complete. 2024-09-19T08:46:46.598 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:46:46.598 INFO:teuthology.orchestra.run.smithi164.stdout: "return_code": 0, 2024-09-19T08:46:46.598 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_tag": "69f5a23e-905f-4b97-804c-e2c239821390", 2024-09-19T08:46:46.598 INFO:teuthology.orchestra.run.smithi164.stdout: "mode": "asynchronous" 2024-09-19T08:46:46.599 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:46:46.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.335+0000 7f884b321740 1 -- start start 2024-09-19T08:46:46.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.335+0000 7f884b321740 1 Processor -- start 2024-09-19T08:46:46.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.336+0000 7f884b321740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ba8f18b60 0x557ba8f2c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:46.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.336+0000 7f884b321740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ba8f19480 0x557ba8f1db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:46.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.336+0000 7f884b321740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ba8f23d40 0x557ba8f1e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:46.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.336+0000 7f884b321740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f88441ac8c0 con 0x557ba8f18b60 2024-09-19T08:46:46.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.336+0000 7f884b321740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f88441ac1f0 con 0x557ba8f23d40 2024-09-19T08:46:46.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.336+0000 7f884b321740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f88441a9490 con 0x557ba8f19480 2024-09-19T08:46:46.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.336+0000 7f883bfff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ba8f19480 0x557ba8f1db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:46.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.336+0000 7f883bfff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ba8f19480 0x557ba8f1db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:48998/0 (socket says 172.21.15.164:48998) 2024-09-19T08:46:46.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.336+0000 7f883bfff640 1 -- 172.21.15.164:0/3974511252 learned_addr learned my addr 172.21.15.164:0/3974511252 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:46.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.336+0000 7f884b321740 10 client.0 _ll_register_callbacks cb 0x7f8834011f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:46.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.336+0000 7f884259c640 1 --2- 172.21.15.164:0/3974511252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ba8f18b60 0x557ba8f2c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:46.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.336+0000 7f8842d9d640 1 --2- 172.21.15.164:0/3974511252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ba8f23d40 0x557ba8f1e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:46.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.336+0000 7f883bfff640 1 -- 172.21.15.164:0/3974511252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ba8f23d40 msgr2=0x557ba8f1e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:46.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.336+0000 7f883bfff640 1 --2- 172.21.15.164:0/3974511252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ba8f23d40 0x557ba8f1e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:46.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.336+0000 7f883bfff640 1 -- 172.21.15.164:0/3974511252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ba8f18b60 msgr2=0x557ba8f2c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:46.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.336+0000 7f883bfff640 1 --2- 172.21.15.164:0/3974511252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ba8f18b60 0x557ba8f2c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:46.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.336+0000 7f883bfff640 1 -- 172.21.15.164:0/3974511252 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ba8fa6430 con 0x557ba8f19480 2024-09-19T08:46:46.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.336+0000 7f884259c640 1 --2- 172.21.15.164:0/3974511252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ba8f18b60 0x557ba8f2c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:46:46.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.337+0000 7f883bfff640 1 --2- 172.21.15.164:0/3974511252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ba8f19480 0x557ba8f1db90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f882c0bc740 tx=0x7f882c0f6530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc4503febd02799d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:46.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.337+0000 7f8829ffb640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-09-19T08:46:46.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.337+0000 7f8829ffb640 1 -- 172.21.15.164:0/3974511252 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c049040 con 0x557ba8f19480 2024-09-19T08:46:46.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.337+0000 7f8829ffb640 1 -- 172.21.15.164:0/3974511252 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f882c03a3c0 con 0x557ba8f19480 2024-09-19T08:46:46.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.337+0000 7f884b321740 1 -- 172.21.15.164:0/3974511252 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f883401fc90 con 0x557ba8f19480 2024-09-19T08:46:46.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.337+0000 7f8829ffb640 1 -- 172.21.15.164:0/3974511252 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f882c0da050 con 0x557ba8f19480 2024-09-19T08:46:46.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.337+0000 7f884b321740 10 client.25228 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:46.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.337+0000 7f884b321740 1 -- 172.21.15.164:0/3974511252 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557ba8fa7630 con 0x557ba8f19480 2024-09-19T08:46:46.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.337+0000 7f8829ffb640 1 -- 172.21.15.164:0/3974511252 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f882c0bceb0 con 0x557ba8f19480 2024-09-19T08:46:46.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.338+0000 7f8829ffb640 1 -- 172.21.15.164:0/3974511252 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f882c03b460 con 0x557ba8f19480 2024-09-19T08:46:46.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.338+0000 7f884b321740 10 client.25228 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:46.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.338+0000 7f884b321740 10 client.25228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:46:46.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.338+0000 7f884b321740 20 client.25228 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:46.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.338+0000 7f884b321740 1 --2- 172.21.15.164:0/3974511252 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x557ba8f3c6f0 0x557ba8fa9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:46.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.338+0000 7f884b321740 4 client.25228 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T08:46:46.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.338+0000 7f884b321740 1 -- 172.21.15.164:0/3974511252 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f883401fc90 con 0x557ba8f3c6f0 2024-09-19T08:46:46.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.339+0000 7f884259c640 1 --2- 172.21.15.164:0/3974511252 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x557ba8f3c6f0 0x557ba8fa9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:46.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.339+0000 7f8829ffb640 10 client.25228 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:46:46.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.339+0000 7f884259c640 1 --2- 172.21.15.164:0/3974511252 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x557ba8f3c6f0 0x557ba8fa9d20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15b908ed770e7516 server_cookie=c7beffd6fa245687 in_seq=0 out_seq=0 2024-09-19T08:46:46.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.340+0000 7f8829ffb640 1 -- 172.21.15.164:0/3974511252 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f883401fc90 con 0x557ba8f3c6f0 2024-09-19T08:46:46.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.340+0000 7f8829ffb640 10 client.25228 handle_command_reply: tid=0 2024-09-19T08:46:46.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.340+0000 7f884b321740 1 client.25228 shutdown 2024-09-19T08:46:46.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.341+0000 7f884b321740 1 -- 172.21.15.164:0/3974511252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ba8f19480 msgr2=0x557ba8f1db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:46.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.341+0000 7f884b321740 1 --2- 172.21.15.164:0/3974511252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ba8f19480 0x557ba8f1db90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f882c0bc740 tx=0x7f882c0f6530 comp rx=0 tx=0).stop 2024-09-19T08:46:46.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.341+0000 7f884b321740 1 -- 172.21.15.164:0/3974511252 shutdown_connections 2024-09-19T08:46:46.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.341+0000 7f884b321740 1 --2- 172.21.15.164:0/3974511252 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x557ba8f3c6f0 0x557ba8fa9d20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:46.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.341+0000 7f884b321740 1 --2- 172.21.15.164:0/3974511252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ba8f23d40 0x557ba8f1e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:46.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.341+0000 7f884b321740 1 --2- 172.21.15.164:0/3974511252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ba8f19480 0x557ba8f1db90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:46.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.341+0000 7f884b321740 1 --2- 172.21.15.164:0/3974511252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557ba8f18b60 0x557ba8f2c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:46.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.341+0000 7f884b321740 1 -- 172.21.15.164:0/3974511252 >> 172.21.15.164:0/3974511252 conn(0x557ba8f150f0 msgr2=0x557ba8f1add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:46.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.341+0000 7f8829ffb640 0 client.25228 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:46:46.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.341+0000 7f884b321740 1 -- 172.21.15.164:0/3974511252 shutdown_connections 2024-09-19T08:46:46.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.341+0000 7f884b321740 1 -- 172.21.15.164:0/3974511252 wait complete. 2024-09-19T08:46:46.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.341+0000 7f884b321740 20 client.25228 trim_cache size 0 max 16384 2024-09-19T08:46:46.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.342+0000 7f8848f48640 1 -- 172.21.15.164:0/2133515039 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f88300a3550 msgr2=0x7f88300a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:46.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.342+0000 7f8848f48640 1 --2- 172.21.15.164:0/2133515039 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f88300a3550 0x7f88300a5a10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x557ba8f15f50 tx=0x7f883400b410 comp rx=0 tx=0).stop 2024-09-19T08:46:46.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.342+0000 7f8848f48640 1 -- 172.21.15.164:0/2133515039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8844100aa0 msgr2=0x7f8844198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:46.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.342+0000 7f8848f48640 1 --2- 172.21.15.164:0/2133515039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8844100aa0 0x7f8844198a20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f882c004770 tx=0x7f882c003f80 comp rx=0 tx=0).stop 2024-09-19T08:46:46.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.342+0000 7f8848f48640 1 -- 172.21.15.164:0/2133515039 shutdown_connections 2024-09-19T08:46:46.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.342+0000 7f8848f48640 1 --2- 172.21.15.164:0/2133515039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8844106b80 0x7f884419cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:46.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.342+0000 7f8848f48640 1 --2- 172.21.15.164:0/2133515039 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f88300a3550 0x7f88300a5a10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:46.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.342+0000 7f8848f48640 1 --2- 172.21.15.164:0/2133515039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8844100aa0 0x7f8844198a20 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:46.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.342+0000 7f8848f48640 1 --2- 172.21.15.164:0/2133515039 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8844100100 0x7f88441984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:46.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.342+0000 7f8848f48640 1 -- 172.21.15.164:0/2133515039 >> 172.21.15.164:0/2133515039 conn(0x7f8844077230 msgr2=0x7f8844104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:46.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.342+0000 7f8848f48640 1 -- 172.21.15.164:0/2133515039 shutdown_connections 2024-09-19T08:46:46.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:46.343+0000 7f8848f48640 1 -- 172.21.15.164:0/2133515039 wait complete. 2024-09-19T08:46:47.415 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:46:47.445 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:46:47.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.956+0000 7fa4851ea640 1 -- 172.21.15.164:0/4188193571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa480108300 msgr2=0x7fa48010ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:47.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.956+0000 7fa4851ea640 1 --2- 172.21.15.164:0/4188193571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa480108300 0x7fa48010ec70 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa478009cd0 tx=0x7fa47802a8d0 comp rx=0 tx=0).stop 2024-09-19T08:46:47.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.956+0000 7fa4851ea640 1 -- 172.21.15.164:0/4188193571 shutdown_connections 2024-09-19T08:46:47.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.956+0000 7fa4851ea640 1 --2- 172.21.15.164:0/4188193571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa480108300 0x7fa48010ec70 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:47.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.956+0000 7fa4851ea640 1 --2- 172.21.15.164:0/4188193571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa480103780 0x7fa480107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:47.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.956+0000 7fa4851ea640 1 --2- 172.21.15.164:0/4188193571 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa480102dd0 0x7fa4801031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:47.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.956+0000 7fa4851ea640 1 -- 172.21.15.164:0/4188193571 >> 172.21.15.164:0/4188193571 conn(0x7fa4800fc7d0 msgr2=0x7fa4800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:47.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.956+0000 7fa4851ea640 1 -- 172.21.15.164:0/4188193571 shutdown_connections 2024-09-19T08:46:47.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.956+0000 7fa4851ea640 1 -- 172.21.15.164:0/4188193571 wait complete. 2024-09-19T08:46:47.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.957+0000 7fa4851ea640 1 Processor -- start 2024-09-19T08:46:47.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.957+0000 7fa4851ea640 1 -- start start 2024-09-19T08:46:47.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.958+0000 7fa4851ea640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa480102dd0 0x7fa480198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:47.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.958+0000 7fa4851ea640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa480103780 0x7fa480198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:47.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.958+0000 7fa4851ea640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa480108300 0x7fa48019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:47.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.958+0000 7fa4851ea640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa4801120d0 con 0x7fa480108300 2024-09-19T08:46:47.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.958+0000 7fa4851ea640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa480111f50 con 0x7fa480102dd0 2024-09-19T08:46:47.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.958+0000 7fa4851ea640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa480112250 con 0x7fa480103780 2024-09-19T08:46:47.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.958+0000 7fa47f59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa480108300 0x7fa48019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:47.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.958+0000 7fa47ed9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa480102dd0 0x7fa480198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:47.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.958+0000 7fa47e59c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa480103780 0x7fa480198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:47.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.958+0000 7fa47ed9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa480102dd0 0x7fa480198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:44394/0 (socket says 172.21.15.164:44394) 2024-09-19T08:46:47.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.958+0000 7fa47ed9d640 1 -- 172.21.15.164:0/733117666 learned_addr learned my addr 172.21.15.164:0/733117666 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:47.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.958+0000 7fa47f59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa480108300 0x7fa48019ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45758/0 (socket says 172.21.15.164:45758) 2024-09-19T08:46:47.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.958+0000 7fa47ed9d640 1 -- 172.21.15.164:0/733117666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa480103780 msgr2=0x7fa480198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:47.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.958+0000 7fa47ed9d640 1 --2- 172.21.15.164:0/733117666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa480103780 0x7fa480198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:47.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.958+0000 7fa47ed9d640 1 -- 172.21.15.164:0/733117666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa480108300 msgr2=0x7fa48019ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:47.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.959+0000 7fa47ed9d640 1 --2- 172.21.15.164:0/733117666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa480108300 0x7fa48019ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:47.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.959+0000 7fa47ed9d640 1 -- 172.21.15.164:0/733117666 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48019d540 con 0x7fa480102dd0 2024-09-19T08:46:47.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.959+0000 7fa47f59e640 1 --2- 172.21.15.164:0/733117666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa480108300 0x7fa48019ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:46:47.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.959+0000 7fa47ed9d640 1 --2- 172.21.15.164:0/733117666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa480102dd0 0x7fa480198590 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa47000d770 tx=0x7fa47000dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:47.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.959+0000 7fa467fff640 1 -- 172.21.15.164:0/733117666 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470017070 con 0x7fa480102dd0 2024-09-19T08:46:47.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.959+0000 7fa467fff640 1 -- 172.21.15.164:0/733117666 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa470005500 con 0x7fa480102dd0 2024-09-19T08:46:47.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.959+0000 7fa467fff640 1 -- 172.21.15.164:0/733117666 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470010460 con 0x7fa480102dd0 2024-09-19T08:46:47.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.959+0000 7fa4851ea640 1 -- 172.21.15.164:0/733117666 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa48019d830 con 0x7fa480102dd0 2024-09-19T08:46:47.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.959+0000 7fa4851ea640 1 -- 172.21.15.164:0/733117666 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa48019dc60 con 0x7fa480102dd0 2024-09-19T08:46:47.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.961+0000 7fa4851ea640 1 -- 172.21.15.164:0/733117666 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4801031b0 con 0x7fa480102dd0 2024-09-19T08:46:47.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.962+0000 7fa467fff640 1 -- 172.21.15.164:0/733117666 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7fa470009c30 con 0x7fa480102dd0 2024-09-19T08:46:47.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.963+0000 7fa467fff640 1 --2- 172.21.15.164:0/733117666 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa4680a3550 0x7fa4680a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:47.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.964+0000 7fa47e59c640 1 --2- 172.21.15.164:0/733117666 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa4680a3550 0x7fa4680a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:47.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.964+0000 7fa467fff640 1 -- 172.21.15.164:0/733117666 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fa4700cfa60 con 0x7fa480102dd0 2024-09-19T08:46:47.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.964+0000 7fa47e59c640 1 --2- 172.21.15.164:0/733117666 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa4680a3550 0x7fa4680a5a10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa4800fee50 tx=0x7fa47400b410 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:47.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:47.968+0000 7fa467fff640 1 -- 172.21.15.164:0/733117666 <== mon.2 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa4700056b0 con 0x7fa480102dd0 2024-09-19T08:46:47.988 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:47 smithi164 ceph-mon[30928]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-19T08:46:47.988 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:47 smithi164 ceph-mon[30928]: scrub queued for path: / 2024-09-19T08:46:47.988 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:47 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:46:47.988 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:47 smithi164 ceph-mon[30928]: scrub summary: active paths [/] 2024-09-19T08:46:47.988 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:47 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:46:47.988 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:47 smithi164 ceph-mon[30928]: scrub summary: idle 2024-09-19T08:46:48.028 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:46:48.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:47 smithi170 ceph-mon[35072]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-19T08:46:48.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:47 smithi170 ceph-mon[35072]: scrub queued for path: / 2024-09-19T08:46:48.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:47 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:46:48.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:47 smithi170 ceph-mon[35072]: scrub summary: active paths [/] 2024-09-19T08:46:48.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:47 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:46:48.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:47 smithi170 ceph-mon[35072]: scrub summary: idle 2024-09-19T08:46:48.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:47 smithi173 ceph-mon[35306]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-19T08:46:48.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:47 smithi173 ceph-mon[35306]: scrub queued for path: / 2024-09-19T08:46:48.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:47 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:46:48.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:47 smithi173 ceph-mon[35306]: scrub summary: active paths [/] 2024-09-19T08:46:48.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:47 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:46:48.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:47 smithi173 ceph-mon[35306]: scrub summary: idle 2024-09-19T08:46:48.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.354+0000 7fa4851ea640 1 -- 172.21.15.164:0/733117666 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4801998f0 con 0x7fa480102dd0 2024-09-19T08:46:48.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.356+0000 7fa467fff640 1 -- 172.21.15.164:0/733117666 <== mon.2 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11984 (secure 0 0 0) 0x7fa47009bbb0 con 0x7fa480102dd0 2024-09-19T08:46:48.357 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:46:48.357 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":28,"btime":"2024-09-19T08:45:46:903004+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:45:46.852789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:46:48.357 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 28 2024-09-19T08:46:48.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.359+0000 7fa4851ea640 1 -- 172.21.15.164:0/733117666 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa4680a3550 msgr2=0x7fa4680a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:48.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.359+0000 7fa4851ea640 1 --2- 172.21.15.164:0/733117666 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa4680a3550 0x7fa4680a5a10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa4800fee50 tx=0x7fa47400b410 comp rx=0 tx=0).stop 2024-09-19T08:46:48.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.359+0000 7fa4851ea640 1 -- 172.21.15.164:0/733117666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa480102dd0 msgr2=0x7fa480198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:48.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.359+0000 7fa4851ea640 1 --2- 172.21.15.164:0/733117666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa480102dd0 0x7fa480198590 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa47000d770 tx=0x7fa47000dc30 comp rx=0 tx=0).stop 2024-09-19T08:46:48.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.359+0000 7fa4851ea640 1 -- 172.21.15.164:0/733117666 shutdown_connections 2024-09-19T08:46:48.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.359+0000 7fa4851ea640 1 --2- 172.21.15.164:0/733117666 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa4680a3550 0x7fa4680a5a10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.359+0000 7fa4851ea640 1 --2- 172.21.15.164:0/733117666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa480108300 0x7fa48019ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.359+0000 7fa4851ea640 1 --2- 172.21.15.164:0/733117666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa480103780 0x7fa480198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.359+0000 7fa4851ea640 1 --2- 172.21.15.164:0/733117666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa480102dd0 0x7fa480198590 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.359+0000 7fa4851ea640 1 -- 172.21.15.164:0/733117666 >> 172.21.15.164:0/733117666 conn(0x7fa4800fc7d0 msgr2=0x7fa480105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:48.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.361+0000 7fa4851ea640 1 -- 172.21.15.164:0/733117666 shutdown_connections 2024-09-19T08:46:48.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.361+0000 7fa4851ea640 1 -- 172.21.15.164:0/733117666 wait complete. 2024-09-19T08:46:48.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.624+0000 7f2279157640 1 -- 172.21.15.164:0/1125325450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22741027a0 msgr2=0x7f22741067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:48.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.624+0000 7f2279157640 1 --2- 172.21.15.164:0/1125325450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22741027a0 0x7f22741067f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f2268009900 tx=0x7f226802a410 comp rx=0 tx=0).stop 2024-09-19T08:46:48.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.625+0000 7f2279157640 1 -- 172.21.15.164:0/1125325450 shutdown_connections 2024-09-19T08:46:48.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.625+0000 7f2279157640 1 --2- 172.21.15.164:0/1125325450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2274106f20 0x7f227410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.625+0000 7f2279157640 1 --2- 172.21.15.164:0/1125325450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22741027a0 0x7f22741067f0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.625+0000 7f2279157640 1 --2- 172.21.15.164:0/1125325450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2274101df0 0x7f22741021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.625+0000 7f2279157640 1 -- 172.21.15.164:0/1125325450 >> 172.21.15.164:0/1125325450 conn(0x7f22740fd8f0 msgr2=0x7f22740ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:48.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.625+0000 7f2279157640 1 -- 172.21.15.164:0/1125325450 shutdown_connections 2024-09-19T08:46:48.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.625+0000 7f2279157640 1 -- 172.21.15.164:0/1125325450 wait complete. 2024-09-19T08:46:48.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.626+0000 7f2279157640 1 Processor -- start 2024-09-19T08:46:48.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.626+0000 7f2279157640 1 -- start start 2024-09-19T08:46:48.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.626+0000 7f2279157640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2274101df0 0x7f2274198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:48.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.626+0000 7f2279157640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2274106f20 0x7f2274198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:48.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.626+0000 7f2279157640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f227419cf40 0x7f227419d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:48.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.626+0000 7f2279157640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2274111f80 con 0x7f2274106f20 2024-09-19T08:46:48.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.626+0000 7f2279157640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2274111e00 con 0x7f2274101df0 2024-09-19T08:46:48.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.626+0000 7f2279157640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2274112100 con 0x7f227419cf40 2024-09-19T08:46:48.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.627+0000 7f2272d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2274101df0 0x7f2274198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:48.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.627+0000 7f227359e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f227419cf40 0x7f227419d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:48.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.627+0000 7f2272d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2274101df0 0x7f2274198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:44412/0 (socket says 172.21.15.164:44412) 2024-09-19T08:46:48.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.627+0000 7f2272d9d640 1 -- 172.21.15.164:0/3528414950 learned_addr learned my addr 172.21.15.164:0/3528414950 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:48.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.627+0000 7f227259c640 1 --2- 172.21.15.164:0/3528414950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2274106f20 0x7f2274198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:48.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.627+0000 7f227359e640 1 -- 172.21.15.164:0/3528414950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2274101df0 msgr2=0x7f2274198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:48.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.627+0000 7f227359e640 1 --2- 172.21.15.164:0/3528414950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2274101df0 0x7f2274198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.627+0000 7f227359e640 1 -- 172.21.15.164:0/3528414950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2274106f20 msgr2=0x7f2274198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:48.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.627+0000 7f227359e640 1 --2- 172.21.15.164:0/3528414950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2274106f20 0x7f2274198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.627+0000 7f227359e640 1 -- 172.21.15.164:0/3528414950 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f227419da00 con 0x7f227419cf40 2024-09-19T08:46:48.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.627+0000 7f227259c640 1 --2- 172.21.15.164:0/3528414950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2274106f20 0x7f2274198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:48.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.627+0000 7f227359e640 1 --2- 172.21.15.164:0/3528414950 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f227419cf40 0x7f227419d3f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f226c00e940 tx=0x7f226c00ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:48.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.628+0000 7f225bfff640 1 -- 172.21.15.164:0/3528414950 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c004d20 con 0x7f227419cf40 2024-09-19T08:46:48.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.628+0000 7f2279157640 1 -- 172.21.15.164:0/3528414950 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f227419dcf0 con 0x7f227419cf40 2024-09-19T08:46:48.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.628+0000 7f225bfff640 1 -- 172.21.15.164:0/3528414950 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f226c005020 con 0x7f227419cf40 2024-09-19T08:46:48.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.628+0000 7f225bfff640 1 -- 172.21.15.164:0/3528414950 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c0116b0 con 0x7f227419cf40 2024-09-19T08:46:48.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.628+0000 7f2279157640 1 -- 172.21.15.164:0/3528414950 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2274074700 con 0x7f227419cf40 2024-09-19T08:46:48.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.631+0000 7f227b530740 1 Processor -- start 2024-09-19T08:46:48.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.631+0000 7f225bfff640 1 -- 172.21.15.164:0/3528414950 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7f226c0054d0 con 0x7f227419cf40 2024-09-19T08:46:48.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.631+0000 7f227b530740 1 -- start start 2024-09-19T08:46:48.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.632+0000 7f227b530740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b19f364a0 0x559b19f36880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:48.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.632+0000 7f227259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b19f364a0 0x559b19f36880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:48.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.632+0000 7f227259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b19f364a0 0x559b19f36880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:44414/0 (socket says 172.21.15.164:44414) 2024-09-19T08:46:48.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.632+0000 7f227b530740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b19f36dc0 0x559b19f41140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:48.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.632+0000 7f225bfff640 1 --2- 172.21.15.164:0/3528414950 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f225c0a3550 0x7f225c0a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:48.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.632+0000 7f227b530740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b19f41680 0x559b19f43a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:48.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.632+0000 7f227b530740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559b19e9c2e0 con 0x559b19f41680 2024-09-19T08:46:48.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.632+0000 7f227b530740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x559b19e43ce0 con 0x559b19f364a0 2024-09-19T08:46:48.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.632+0000 7f227b530740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559b19e982a0 con 0x559b19f36dc0 2024-09-19T08:46:48.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.632+0000 7f227259c640 1 -- 172.21.15.164:0/1398697167 learned_addr learned my addr 172.21.15.164:0/1398697167 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:48.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.632+0000 7f227359e640 1 --2- 172.21.15.164:0/1398697167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b19f41680 0x559b19f43a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:48.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.632+0000 7f2272d9d640 1 --2- 172.21.15.164:0/1398697167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b19f36dc0 0x559b19f41140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:48.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.633+0000 7f227259c640 1 --2- 172.21.15.164:0/3528414950 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f225c0a3550 0x7f225c0a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:48.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.633+0000 7f225bfff640 1 -- 172.21.15.164:0/3528414950 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f226c0cffc0 con 0x7f227419cf40 2024-09-19T08:46:48.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.633+0000 7f227359e640 1 -- 172.21.15.164:0/1398697167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b19f36dc0 msgr2=0x559b19f41140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:48.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.633+0000 7f227359e640 1 --2- 172.21.15.164:0/1398697167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b19f36dc0 0x559b19f41140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.633+0000 7f227359e640 1 -- 172.21.15.164:0/1398697167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b19f364a0 msgr2=0x559b19f36880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:48.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.633+0000 7f227359e640 1 --2- 172.21.15.164:0/1398697167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b19f364a0 0x559b19f36880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.633+0000 7f227359e640 1 -- 172.21.15.164:0/1398697167 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b19e28440 con 0x559b19f41680 2024-09-19T08:46:48.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.633+0000 7f227359e640 1 --2- 172.21.15.164:0/1398697167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b19f41680 0x559b19f43a70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f226c0cd600 tx=0x7f226c098480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd7414d46a9cf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:48.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.633+0000 7f22597fa640 1 -- 172.21.15.164:0/1398697167 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c01f070 con 0x559b19f41680 2024-09-19T08:46:48.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.634+0000 7f22597fa640 1 -- 172.21.15.164:0/1398697167 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f226c09c7c0 con 0x559b19f41680 2024-09-19T08:46:48.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.634+0000 7f22597fa640 1 -- 172.21.15.164:0/1398697167 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c0a48a0 con 0x559b19f41680 2024-09-19T08:46:48.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.634+0000 7f227b530740 1 -- 172.21.15.164:0/1398697167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b19f41680 msgr2=0x559b19f43a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:48.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.634+0000 7f227b530740 1 --2- 172.21.15.164:0/1398697167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b19f41680 0x559b19f43a70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f226c0cd600 tx=0x7f226c098480 comp rx=0 tx=0).stop 2024-09-19T08:46:48.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.634+0000 7f227259c640 1 --2- 172.21.15.164:0/3528414950 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f225c0a3550 0x7f225c0a5a10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2268000c00 tx=0x7f2268009560 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:48.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.635+0000 7f227b530740 1 -- 172.21.15.164:0/1398697167 shutdown_connections 2024-09-19T08:46:48.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.635+0000 7f227b530740 1 --2- 172.21.15.164:0/1398697167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b19f41680 0x559b19f43a70 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.635+0000 7f227b530740 1 --2- 172.21.15.164:0/1398697167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b19f36dc0 0x559b19f41140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.635+0000 7f227b530740 1 --2- 172.21.15.164:0/1398697167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b19f364a0 0x559b19f36880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.635+0000 7f227b530740 1 -- 172.21.15.164:0/1398697167 >> 172.21.15.164:0/1398697167 conn(0x559b19f32a30 msgr2=0x559b19f32e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:48.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.635+0000 7f227b530740 1 -- 172.21.15.164:0/1398697167 shutdown_connections 2024-09-19T08:46:48.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.635+0000 7f227b530740 1 -- 172.21.15.164:0/1398697167 wait complete. 2024-09-19T08:46:48.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.635+0000 7f227b530740 1 -- start start 2024-09-19T08:46:48.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.635+0000 7f227b530740 1 Processor -- start 2024-09-19T08:46:48.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.635+0000 7f227b530740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b19f364a0 0x559b19f41030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:48.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.636+0000 7f227b530740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b19f36dc0 0x559b19f3e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:48.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.636+0000 7f227259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b19f36dc0 0x559b19f3e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:48.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.636+0000 7f227259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b19f36dc0 0x559b19f3e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45788/0 (socket says 172.21.15.164:45788) 2024-09-19T08:46:48.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.636+0000 7f227b530740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b19f41680 0x559b19f3e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:48.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.636+0000 7f227259c640 1 -- 172.21.15.164:0/2079326763 learned_addr learned my addr 172.21.15.164:0/2079326763 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:48.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.636+0000 7f227b530740 1 -- 172.21.15.164:0/2079326763 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559b19f441f0 con 0x559b19f36dc0 2024-09-19T08:46:48.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.636+0000 7f227b530740 1 -- 172.21.15.164:0/2079326763 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x559b19e57050 con 0x559b19f41680 2024-09-19T08:46:48.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.636+0000 7f227b530740 1 -- 172.21.15.164:0/2079326763 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559b19f44370 con 0x559b19f364a0 2024-09-19T08:46:48.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.636+0000 7f227b530740 10 client.0 _ll_register_callbacks cb 0x559b19e97160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:48.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.636+0000 7f2272d9d640 1 --2- 172.21.15.164:0/2079326763 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b19f364a0 0x559b19f41030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:48.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.636+0000 7f227359e640 1 --2- 172.21.15.164:0/2079326763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b19f41680 0x559b19f3e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:48.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.636+0000 7f227259c640 1 -- 172.21.15.164:0/2079326763 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b19f364a0 msgr2=0x559b19f41030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:48.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.636+0000 7f227259c640 1 --2- 172.21.15.164:0/2079326763 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b19f364a0 0x559b19f41030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.636+0000 7f227259c640 1 -- 172.21.15.164:0/2079326763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b19f41680 msgr2=0x559b19f3e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:48.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.636+0000 7f227259c640 1 --2- 172.21.15.164:0/2079326763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b19f41680 0x559b19f3e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.636+0000 7f227259c640 1 -- 172.21.15.164:0/2079326763 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b19c0d3c0 con 0x559b19f36dc0 2024-09-19T08:46:48.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.636+0000 7f2272d9d640 1 --2- 172.21.15.164:0/2079326763 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b19f364a0 0x559b19f41030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:46:48.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.637+0000 7f227259c640 1 --2- 172.21.15.164:0/2079326763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b19f36dc0 0x559b19f3e2e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f2268009230 tx=0x7f226804b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4fc27581908d265 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:48.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.637+0000 7f2256ff5640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:46:48.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.637+0000 7f2256ff5640 1 -- 172.21.15.164:0/2079326763 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2268035050 con 0x559b19f36dc0 2024-09-19T08:46:48.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.637+0000 7f2256ff5640 1 -- 172.21.15.164:0/2079326763 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f2268005230 con 0x559b19f36dc0 2024-09-19T08:46:48.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.637+0000 7f227b530740 1 -- 172.21.15.164:0/2079326763 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f226c09c7c0 con 0x559b19f36dc0 2024-09-19T08:46:48.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.637+0000 7f2256ff5640 1 -- 172.21.15.164:0/2079326763 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f226805e050 con 0x559b19f36dc0 2024-09-19T08:46:48.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.637+0000 7f227b530740 10 client.15705 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:48.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.637+0000 7f227b530740 1 -- 172.21.15.164:0/2079326763 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559b19f4fbe0 con 0x559b19f36dc0 2024-09-19T08:46:48.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.637+0000 7f2256ff5640 1 -- 172.21.15.164:0/2079326763 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2268049040 con 0x559b19f36dc0 2024-09-19T08:46:48.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.637+0000 7f2256ff5640 1 -- 172.21.15.164:0/2079326763 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2268055070 con 0x559b19f36dc0 2024-09-19T08:46:48.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.638+0000 7f227b530740 10 client.15705 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:48.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.638+0000 7f227b530740 10 client.15705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:46:48.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.638+0000 7f227b530740 20 client.15705 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:48.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.638+0000 7f227b530740 1 --2- 172.21.15.164:0/2079326763 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x559b19f51040 0x559b19f53460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:48.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.638+0000 7f227b530740 4 client.15705 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:46:48.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.638+0000 7f227b530740 1 -- 172.21.15.164:0/2079326763 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f226c09c7c0 con 0x559b19f51040 2024-09-19T08:46:48.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.638+0000 7f2272d9d640 1 --2- 172.21.15.164:0/2079326763 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x559b19f51040 0x559b19f53460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:48.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.638+0000 7f2272d9d640 1 --2- 172.21.15.164:0/2079326763 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x559b19f51040 0x559b19f53460 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=559ee4508ec48f31 server_cookie=6200922c1ef8717d in_seq=0 out_seq=0 2024-09-19T08:46:48.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.638+0000 7f2256ff5640 10 client.15705 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:46:48.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.639+0000 7f2256ff5640 1 -- 172.21.15.164:0/2079326763 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f226c09c7c0 con 0x559b19f51040 2024-09-19T08:46:48.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.639+0000 7f2256ff5640 10 client.15705 handle_command_reply: tid=0 2024-09-19T08:46:48.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.639+0000 7f227b530740 1 client.15705 shutdown 2024-09-19T08:46:48.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.639+0000 7f227b530740 1 -- 172.21.15.164:0/2079326763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b19f36dc0 msgr2=0x559b19f3e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:48.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.639+0000 7f227b530740 1 --2- 172.21.15.164:0/2079326763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b19f36dc0 0x559b19f3e2e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f2268009230 tx=0x7f226804b000 comp rx=0 tx=0).stop 2024-09-19T08:46:48.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.639+0000 7f227b530740 1 -- 172.21.15.164:0/2079326763 shutdown_connections 2024-09-19T08:46:48.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.639+0000 7f227b530740 1 --2- 172.21.15.164:0/2079326763 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x559b19f51040 0x559b19f53460 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.639+0000 7f227b530740 1 --2- 172.21.15.164:0/2079326763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b19f41680 0x559b19f3e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.639+0000 7f227b530740 1 --2- 172.21.15.164:0/2079326763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b19f36dc0 0x559b19f3e2e0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.639+0000 7f227b530740 1 --2- 172.21.15.164:0/2079326763 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b19f364a0 0x559b19f41030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.639+0000 7f227b530740 1 -- 172.21.15.164:0/2079326763 >> 172.21.15.164:0/2079326763 conn(0x559b19f32a30 msgr2=0x559b19f38760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:48.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.639+0000 7f2256ff5640 0 client.15705 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:46:48.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.640+0000 7f227b530740 1 -- 172.21.15.164:0/2079326763 shutdown_connections 2024-09-19T08:46:48.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.640+0000 7f227b530740 1 -- 172.21.15.164:0/2079326763 wait complete. 2024-09-19T08:46:48.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.640+0000 7f227b530740 20 client.15705 trim_cache size 0 max 16384 2024-09-19T08:46:48.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.657+0000 7f227b530740 1 Processor -- start 2024-09-19T08:46:48.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.657+0000 7f227b530740 1 -- start start 2024-09-19T08:46:48.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.657+0000 7f227b530740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b19f364a0 0x559b19f4d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:48.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.657+0000 7f227b530740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b19f36dc0 0x559b19f58d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:48.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.657+0000 7f227b530740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b19f41680 0x559b19f59270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:48.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.657+0000 7f227b530740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f22741aba50 con 0x559b19f364a0 2024-09-19T08:46:48.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.657+0000 7f227b530740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f22741ad590 con 0x559b19f41680 2024-09-19T08:46:48.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.657+0000 7f227b530740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f22741ac120 con 0x559b19f36dc0 2024-09-19T08:46:48.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.657+0000 7f2272d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b19f364a0 0x559b19f4d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:48.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.657+0000 7f2272d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b19f364a0 0x559b19f4d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45802/0 (socket says 172.21.15.164:45802) 2024-09-19T08:46:48.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.657+0000 7f2272d9d640 1 -- 172.21.15.164:0/893639329 learned_addr learned my addr 172.21.15.164:0/893639329 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:48.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.657+0000 7f227359e640 1 --2- 172.21.15.164:0/893639329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b19f41680 0x559b19f59270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:48.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.657+0000 7f227259c640 1 --2- 172.21.15.164:0/893639329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b19f36dc0 0x559b19f58d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:48.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.658+0000 7f2272d9d640 1 -- 172.21.15.164:0/893639329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b19f36dc0 msgr2=0x559b19f58d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:48.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.658+0000 7f2272d9d640 1 --2- 172.21.15.164:0/893639329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b19f36dc0 0x559b19f58d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.658+0000 7f2272d9d640 1 -- 172.21.15.164:0/893639329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b19f41680 msgr2=0x559b19f59270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:48.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.658+0000 7f2272d9d640 1 --2- 172.21.15.164:0/893639329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b19f41680 0x559b19f59270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.658+0000 7f2272d9d640 1 -- 172.21.15.164:0/893639329 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b19f3ebf0 con 0x559b19f364a0 2024-09-19T08:46:48.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.658+0000 7f2272d9d640 1 --2- 172.21.15.164:0/893639329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b19f364a0 0x559b19f4d450 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f226400ba70 tx=0x7f226400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53ef92bfdb84169 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:48.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.658+0000 7f2258ff9640 1 -- 172.21.15.164:0/893639329 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2264005020 con 0x559b19f364a0 2024-09-19T08:46:48.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.658+0000 7f2258ff9640 1 -- 172.21.15.164:0/893639329 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22640047a0 con 0x559b19f364a0 2024-09-19T08:46:48.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.658+0000 7f2258ff9640 1 -- 172.21.15.164:0/893639329 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2264026730 con 0x559b19f364a0 2024-09-19T08:46:48.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.659+0000 7f227b530740 1 -- 172.21.15.164:0/893639329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b19f364a0 msgr2=0x559b19f4d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:48.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.659+0000 7f227b530740 1 --2- 172.21.15.164:0/893639329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b19f364a0 0x559b19f4d450 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f226400ba70 tx=0x7f226400b480 comp rx=0 tx=0).stop 2024-09-19T08:46:48.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.659+0000 7f227b530740 1 -- 172.21.15.164:0/893639329 shutdown_connections 2024-09-19T08:46:48.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.659+0000 7f227b530740 1 --2- 172.21.15.164:0/893639329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b19f41680 0x559b19f59270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.659+0000 7f227b530740 1 --2- 172.21.15.164:0/893639329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b19f36dc0 0x559b19f58d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.659+0000 7f227b530740 1 --2- 172.21.15.164:0/893639329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b19f364a0 0x559b19f4d450 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.659+0000 7f227b530740 1 -- 172.21.15.164:0/893639329 >> 172.21.15.164:0/893639329 conn(0x559b19f32a30 msgr2=0x559b19f3d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:48.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.660+0000 7f227b530740 1 -- 172.21.15.164:0/893639329 shutdown_connections 2024-09-19T08:46:48.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.660+0000 7f227b530740 1 -- 172.21.15.164:0/893639329 wait complete. 2024-09-19T08:46:48.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.660+0000 7f227b530740 1 -- start start 2024-09-19T08:46:48.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.660+0000 7f227b530740 1 Processor -- start 2024-09-19T08:46:48.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.660+0000 7f227b530740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b19f364a0 0x559b19f44d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:48.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.660+0000 7f227b530740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b19f36dc0 0x559b19f45240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:48.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.660+0000 7f227b530740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b19f41680 0x559b19fb3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:48.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.660+0000 7f227b530740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f22741acec0 con 0x559b19f364a0 2024-09-19T08:46:48.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.660+0000 7f227b530740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f22741ac7f0 con 0x559b19f36dc0 2024-09-19T08:46:48.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.660+0000 7f227b530740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f22741a9a50 con 0x559b19f41680 2024-09-19T08:46:48.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.660+0000 7f227b530740 10 client.0 _ll_register_callbacks cb 0x7f2268009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:48.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.661+0000 7f2272d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b19f364a0 0x559b19f44d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:48.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.661+0000 7f227359e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b19f41680 0x559b19fb3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:48.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.661+0000 7f227359e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b19f41680 0x559b19fb3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:49094/0 (socket says 172.21.15.164:49094) 2024-09-19T08:46:48.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.661+0000 7f227359e640 1 -- 172.21.15.164:0/1297053621 learned_addr learned my addr 172.21.15.164:0/1297053621 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:48.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.661+0000 7f227259c640 1 --2- 172.21.15.164:0/1297053621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b19f36dc0 0x559b19f45240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:48.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.661+0000 7f227359e640 1 -- 172.21.15.164:0/1297053621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b19f36dc0 msgr2=0x559b19f45240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:48.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.661+0000 7f227359e640 1 --2- 172.21.15.164:0/1297053621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b19f36dc0 0x559b19f45240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.661+0000 7f227359e640 1 -- 172.21.15.164:0/1297053621 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b19f364a0 msgr2=0x559b19f44d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:48.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.661+0000 7f227359e640 1 --2- 172.21.15.164:0/1297053621 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b19f364a0 0x559b19f44d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.661+0000 7f227359e640 1 -- 172.21.15.164:0/1297053621 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b19f36bf0 con 0x559b19f41680 2024-09-19T08:46:48.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.661+0000 7f2272d9d640 1 --2- 172.21.15.164:0/1297053621 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b19f364a0 0x559b19f44d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:46:48.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.662+0000 7f227359e640 1 --2- 172.21.15.164:0/1297053621 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b19f41680 0x559b19fb3d30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f226c0cd710 tx=0x7f226c0a04e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc18f5a224ebf69b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:48.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.662+0000 7f2256ff5640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-09-19T08:46:48.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.662+0000 7f2256ff5640 1 -- 172.21.15.164:0/1297053621 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c0b5060 con 0x559b19f41680 2024-09-19T08:46:48.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.662+0000 7f2256ff5640 1 -- 172.21.15.164:0/1297053621 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f226c0a0150 con 0x559b19f41680 2024-09-19T08:46:48.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.662+0000 7f227b530740 1 -- 172.21.15.164:0/1297053621 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22640047a0 con 0x559b19f41680 2024-09-19T08:46:48.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.662+0000 7f2256ff5640 1 -- 172.21.15.164:0/1297053621 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f226c01f040 con 0x559b19f41680 2024-09-19T08:46:48.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.662+0000 7f227b530740 10 client.25252 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:48.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.662+0000 7f227b530740 1 -- 172.21.15.164:0/1297053621 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559b19fbb4f0 con 0x559b19f41680 2024-09-19T08:46:48.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.662+0000 7f2256ff5640 1 -- 172.21.15.164:0/1297053621 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f226c0a4d30 con 0x559b19f41680 2024-09-19T08:46:48.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.663+0000 7f2256ff5640 1 -- 172.21.15.164:0/1297053621 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f226c00f040 con 0x559b19f41680 2024-09-19T08:46:48.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.663+0000 7f227b530740 10 client.25252 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:48.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.663+0000 7f227b530740 10 client.25252 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:46:48.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.663+0000 7f227b530740 20 client.25252 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:48.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.663+0000 7f227b530740 1 --2- 172.21.15.164:0/1297053621 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x559b19fbb7e0 0x559b19fbdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:48.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.663+0000 7f227b530740 4 client.25252 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:46:48.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.663+0000 7f227b530740 1 -- 172.21.15.164:0/1297053621 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f22640047a0 con 0x559b19fbb7e0 2024-09-19T08:46:48.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.663+0000 7f2272d9d640 1 --2- 172.21.15.164:0/1297053621 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x559b19fbb7e0 0x559b19fbdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:48.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.664+0000 7f2272d9d640 1 --2- 172.21.15.164:0/1297053621 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x559b19fbb7e0 0x559b19fbdbd0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a35bec9842a6e0b1 server_cookie=2dfdb6349f943b26 in_seq=0 out_seq=0 2024-09-19T08:46:48.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.664+0000 7f2256ff5640 10 client.25252 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:46:48.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.664+0000 7f2256ff5640 1 -- 172.21.15.164:0/1297053621 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f22640047a0 con 0x559b19fbb7e0 2024-09-19T08:46:48.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.664+0000 7f2256ff5640 10 client.25252 handle_command_reply: tid=0 2024-09-19T08:46:48.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.664+0000 7f227b530740 1 client.25252 shutdown 2024-09-19T08:46:48.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.664+0000 7f227b530740 1 -- 172.21.15.164:0/1297053621 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b19f41680 msgr2=0x559b19fb3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:48.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.664+0000 7f227b530740 1 --2- 172.21.15.164:0/1297053621 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b19f41680 0x559b19fb3d30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f226c0cd710 tx=0x7f226c0a04e0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.664+0000 7f227b530740 1 -- 172.21.15.164:0/1297053621 shutdown_connections 2024-09-19T08:46:48.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.664+0000 7f227b530740 1 --2- 172.21.15.164:0/1297053621 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b19f41680 0x559b19fb3d30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.664+0000 7f227b530740 1 --2- 172.21.15.164:0/1297053621 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x559b19fbb7e0 0x559b19fbdbd0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.664+0000 7f227b530740 1 --2- 172.21.15.164:0/1297053621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b19f36dc0 0x559b19f45240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.664+0000 7f227b530740 1 --2- 172.21.15.164:0/1297053621 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559b19f364a0 0x559b19f44d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.664+0000 7f227b530740 1 -- 172.21.15.164:0/1297053621 >> 172.21.15.164:0/1297053621 conn(0x559b19f32a30 msgr2=0x559b19f39090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:48.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.665+0000 7f2256ff5640 0 client.25252 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:46:48.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.665+0000 7f227b530740 1 -- 172.21.15.164:0/1297053621 shutdown_connections 2024-09-19T08:46:48.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.665+0000 7f227b530740 1 -- 172.21.15.164:0/1297053621 wait complete. 2024-09-19T08:46:48.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.665+0000 7f227b530740 20 client.25252 trim_cache size 0 max 16384 2024-09-19T08:46:48.670 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:46:48.670 INFO:teuthology.orchestra.run.smithi164.stdout: "status": "no active scrubs running", 2024-09-19T08:46:48.670 INFO:teuthology.orchestra.run.smithi164.stdout: "scrubs": {} 2024-09-19T08:46:48.670 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:46:48.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.665+0000 7f2279157640 1 -- 172.21.15.164:0/3528414950 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f225c0a3550 msgr2=0x7f225c0a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:48.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.665+0000 7f2279157640 1 --2- 172.21.15.164:0/3528414950 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f225c0a3550 0x7f225c0a5a10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2268000c00 tx=0x7f2268009560 comp rx=0 tx=0).stop 2024-09-19T08:46:48.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.666+0000 7f2279157640 1 -- 172.21.15.164:0/3528414950 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f227419cf40 msgr2=0x7f227419d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:48.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.666+0000 7f2279157640 1 --2- 172.21.15.164:0/3528414950 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f227419cf40 0x7f227419d3f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f226c00e940 tx=0x7f226c00ee00 comp rx=0 tx=0).stop 2024-09-19T08:46:48.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.666+0000 7f2279157640 1 -- 172.21.15.164:0/3528414950 shutdown_connections 2024-09-19T08:46:48.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.666+0000 7f2279157640 1 --2- 172.21.15.164:0/3528414950 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f227419cf40 0x7f227419d3f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.666+0000 7f2279157640 1 --2- 172.21.15.164:0/3528414950 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f225c0a3550 0x7f225c0a5a10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.666+0000 7f2279157640 1 --2- 172.21.15.164:0/3528414950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2274106f20 0x7f2274198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.666+0000 7f2279157640 1 --2- 172.21.15.164:0/3528414950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2274101df0 0x7f2274198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:48.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.666+0000 7f2279157640 1 -- 172.21.15.164:0/3528414950 >> 172.21.15.164:0/3528414950 conn(0x7f22740fd8f0 msgr2=0x7f22740fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:48.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.666+0000 7f2279157640 1 -- 172.21.15.164:0/3528414950 shutdown_connections 2024-09-19T08:46:48.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:48.666+0000 7f2279157640 1 -- 172.21.15.164:0/3528414950 wait complete. 2024-09-19T08:46:48.859 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:48 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/733117666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:46:49.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:48 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/733117666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:46:49.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:48 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/733117666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:46:49.525 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T08:46:49.526 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:46:49.743 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:49 smithi164 ceph-mon[30928]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-19T08:46:49.935 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id 446cbd86 --release --await' 2024-09-19T08:46:49.935 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id 446cbd86 --release --await 2024-09-19T08:46:49.999 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:46:50.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:49 smithi170 ceph-mon[35072]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-19T08:46:50.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:49 smithi173 ceph-mon[35306]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-19T08:46:50.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.636+0000 7fd0d0f5e640 1 -- 172.21.15.164:0/3932019788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0cc102570 msgr2=0x7fd0cc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:50.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.636+0000 7fd0d0f5e640 1 --2- 172.21.15.164:0/3932019788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0cc102570 0x7fd0cc1029b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0009900 tx=0x7fd0c002a480 comp rx=0 tx=0).stop 2024-09-19T08:46:50.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.636+0000 7fd0d0f5e640 1 -- 172.21.15.164:0/3932019788 shutdown_connections 2024-09-19T08:46:50.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.636+0000 7fd0d0f5e640 1 --2- 172.21.15.164:0/3932019788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0cc102ef0 0x7fd0cc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.636+0000 7fd0d0f5e640 1 --2- 172.21.15.164:0/3932019788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0cc102570 0x7fd0cc1029b0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.636+0000 7fd0d0f5e640 1 --2- 172.21.15.164:0/3932019788 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0cc100e20 0x7fd0cc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.636+0000 7fd0d0f5e640 1 -- 172.21.15.164:0/3932019788 >> 172.21.15.164:0/3932019788 conn(0x7fd0cc0fc7d0 msgr2=0x7fd0cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:50.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.636+0000 7fd0d0f5e640 1 -- 172.21.15.164:0/3932019788 shutdown_connections 2024-09-19T08:46:50.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.636+0000 7fd0d0f5e640 1 -- 172.21.15.164:0/3932019788 wait complete. 2024-09-19T08:46:50.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.637+0000 7fd0d0f5e640 1 Processor -- start 2024-09-19T08:46:50.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.637+0000 7fd0d0f5e640 1 -- start start 2024-09-19T08:46:50.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.637+0000 7fd0d0f5e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0cc100e20 0x7fd0cc197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:50.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.637+0000 7fd0d0f5e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0cc102570 0x7fd0cc1978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:50.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.637+0000 7fd0d0f5e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0cc102ef0 0x7fd0cc19bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:50.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.637+0000 7fd0d0f5e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc110f80 con 0x7fd0cc102570 2024-09-19T08:46:50.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.637+0000 7fd0d0f5e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc110e00 con 0x7fd0cc102ef0 2024-09-19T08:46:50.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.637+0000 7fd0d0f5e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc111100 con 0x7fd0cc100e20 2024-09-19T08:46:50.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.637+0000 7fd0c9d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0cc102570 0x7fd0cc1978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:50.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.638+0000 7fd0c9d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0cc102570 0x7fd0cc1978c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45832/0 (socket says 172.21.15.164:45832) 2024-09-19T08:46:50.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.638+0000 7fd0c9d9b640 1 -- 172.21.15.164:0/500219862 learned_addr learned my addr 172.21.15.164:0/500219862 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:50.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.638+0000 7fd0cad9d640 1 --2- 172.21.15.164:0/500219862 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0cc102ef0 0x7fd0cc19bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:50.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.638+0000 7fd0ca59c640 1 --2- 172.21.15.164:0/500219862 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0cc100e20 0x7fd0cc197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:50.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.638+0000 7fd0cad9d640 1 -- 172.21.15.164:0/500219862 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0cc100e20 msgr2=0x7fd0cc197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:50.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.638+0000 7fd0cad9d640 1 --2- 172.21.15.164:0/500219862 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0cc100e20 0x7fd0cc197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.638+0000 7fd0cad9d640 1 -- 172.21.15.164:0/500219862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0cc102570 msgr2=0x7fd0cc1978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:50.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.638+0000 7fd0cad9d640 1 --2- 172.21.15.164:0/500219862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0cc102570 0x7fd0cc1978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.638+0000 7fd0cad9d640 1 -- 172.21.15.164:0/500219862 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0cc19c330 con 0x7fd0cc102ef0 2024-09-19T08:46:50.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.638+0000 7fd0c9d9b640 1 --2- 172.21.15.164:0/500219862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0cc102570 0x7fd0cc1978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:46:50.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.638+0000 7fd0cad9d640 1 --2- 172.21.15.164:0/500219862 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0cc102ef0 0x7fd0cc19bc50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4009fd0 tx=0x7fd0c400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:50.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.639+0000 7fd0bb7fe640 1 -- 172.21.15.164:0/500219862 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c400ce20 con 0x7fd0cc102ef0 2024-09-19T08:46:50.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.639+0000 7fd0d0f5e640 1 -- 172.21.15.164:0/500219862 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0cc19c620 con 0x7fd0cc102ef0 2024-09-19T08:46:50.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.639+0000 7fd0d0f5e640 1 -- 172.21.15.164:0/500219862 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0cc069f70 con 0x7fd0cc102ef0 2024-09-19T08:46:50.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.639+0000 7fd0bb7fe640 1 -- 172.21.15.164:0/500219862 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0c4005020 con 0x7fd0cc102ef0 2024-09-19T08:46:50.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.639+0000 7fd0bb7fe640 1 -- 172.21.15.164:0/500219862 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c4010720 con 0x7fd0cc102ef0 2024-09-19T08:46:50.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.640+0000 7fd0d0f5e640 1 -- 172.21.15.164:0/500219862 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0ac005290 con 0x7fd0cc102ef0 2024-09-19T08:46:50.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.641+0000 7fd0bb7fe640 1 -- 172.21.15.164:0/500219862 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7fd0c4004a40 con 0x7fd0cc102ef0 2024-09-19T08:46:50.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.642+0000 7fd0bb7fe640 1 --2- 172.21.15.164:0/500219862 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd0b40a3550 0x7fd0b40a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:50.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.642+0000 7fd0ca59c640 1 --2- 172.21.15.164:0/500219862 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd0b40a3550 0x7fd0b40a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:50.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.642+0000 7fd0bb7fe640 1 -- 172.21.15.164:0/500219862 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fd0c40d0820 con 0x7fd0cc102ef0 2024-09-19T08:46:50.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.643+0000 7fd0ca59c640 1 --2- 172.21.15.164:0/500219862 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd0b40a3550 0x7fd0b40a5a10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009bd0 tx=0x7fd0bc005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:50.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.645+0000 7fd0bb7fe640 1 -- 172.21.15.164:0/500219862 <== mon.2 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd0c409cfc0 con 0x7fd0cc102ef0 2024-09-19T08:46:50.748 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:46:50.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.878+0000 7fd0d0f5e640 1 -- 172.21.15.164:0/500219862 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0ac005580 con 0x7fd0cc102ef0 2024-09-19T08:46:50.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.879+0000 7fd0bb7fe640 1 -- 172.21.15.164:0/500219862 <== mon.2 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11984 (secure 0 0 0) 0x7fd0c409c8c0 con 0x7fd0cc102ef0 2024-09-19T08:46:50.880 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:46:50.880 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":28,"btime":"2024-09-19T08:45:46:903004+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:45:46.852789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:46:50.880 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 28 2024-09-19T08:46:50.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.882+0000 7fd0d0f5e640 1 -- 172.21.15.164:0/500219862 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd0b40a3550 msgr2=0x7fd0b40a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:50.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.882+0000 7fd0d0f5e640 1 --2- 172.21.15.164:0/500219862 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd0b40a3550 0x7fd0b40a5a10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009bd0 tx=0x7fd0bc005f90 comp rx=0 tx=0).stop 2024-09-19T08:46:50.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.882+0000 7fd0d0f5e640 1 -- 172.21.15.164:0/500219862 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0cc102ef0 msgr2=0x7fd0cc19bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:50.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.882+0000 7fd0d0f5e640 1 --2- 172.21.15.164:0/500219862 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0cc102ef0 0x7fd0cc19bc50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4009fd0 tx=0x7fd0c400ef10 comp rx=0 tx=0).stop 2024-09-19T08:46:50.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.882+0000 7fd0d0f5e640 1 -- 172.21.15.164:0/500219862 shutdown_connections 2024-09-19T08:46:50.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.882+0000 7fd0d0f5e640 1 --2- 172.21.15.164:0/500219862 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0cc102ef0 0x7fd0cc19bc50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.882+0000 7fd0d0f5e640 1 --2- 172.21.15.164:0/500219862 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd0b40a3550 0x7fd0b40a5a10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.882+0000 7fd0d0f5e640 1 --2- 172.21.15.164:0/500219862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0cc102570 0x7fd0cc1978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.882+0000 7fd0d0f5e640 1 --2- 172.21.15.164:0/500219862 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0cc100e20 0x7fd0cc197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.882+0000 7fd0d0f5e640 1 -- 172.21.15.164:0/500219862 >> 172.21.15.164:0/500219862 conn(0x7fd0cc0fc7d0 msgr2=0x7fd0cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:50.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.882+0000 7fd0d0f5e640 1 -- 172.21.15.164:0/500219862 shutdown_connections 2024-09-19T08:46:50.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:50.882+0000 7fd0d0f5e640 1 -- 172.21.15.164:0/500219862 wait complete. 2024-09-19T08:46:51.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.445+0000 7ff8b6977640 1 -- 172.21.15.164:0/3946960700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8b0102d30 msgr2=0x7ff8b0103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:51.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.445+0000 7ff8b6977640 1 --2- 172.21.15.164:0/3946960700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8b0102d30 0x7ff8b0103110 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0009930 tx=0x7ff8a002a420 comp rx=0 tx=0).stop 2024-09-19T08:46:51.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.445+0000 7ff8b6977640 1 -- 172.21.15.164:0/3946960700 shutdown_connections 2024-09-19T08:46:51.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.445+0000 7ff8b6977640 1 --2- 172.21.15.164:0/3946960700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8b01082a0 0x7ff8b010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.445+0000 7ff8b6977640 1 --2- 172.21.15.164:0/3946960700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8b01036e0 0x7ff8b0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.445+0000 7ff8b6977640 1 --2- 172.21.15.164:0/3946960700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8b0102d30 0x7ff8b0103110 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.445+0000 7ff8b6977640 1 -- 172.21.15.164:0/3946960700 >> 172.21.15.164:0/3946960700 conn(0x7ff8b00fc7d0 msgr2=0x7ff8b00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:51.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.445+0000 7ff8b6977640 1 -- 172.21.15.164:0/3946960700 shutdown_connections 2024-09-19T08:46:51.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.445+0000 7ff8b6977640 1 -- 172.21.15.164:0/3946960700 wait complete. 2024-09-19T08:46:51.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.445+0000 7ff8b6977640 1 Processor -- start 2024-09-19T08:46:51.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.446+0000 7ff8b6977640 1 -- start start 2024-09-19T08:46:51.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.446+0000 7ff8b6977640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8b0102d30 0x7ff8b01984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:51.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.446+0000 7ff8b6977640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8b01036e0 0x7ff8b0198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:51.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.446+0000 7ff8b6977640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8b01082a0 0x7ff8b019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:51.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.446+0000 7ff8b6977640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b01120a0 con 0x7ff8b0102d30 2024-09-19T08:46:51.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.446+0000 7ff8b6977640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b0111f20 con 0x7ff8b01036e0 2024-09-19T08:46:51.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.446+0000 7ff8b6977640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b0112220 con 0x7ff8b01082a0 2024-09-19T08:46:51.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.446+0000 7ff8affff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8b0102d30 0x7ff8b01984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:51.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.446+0000 7ff8affff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8b0102d30 0x7ff8b01984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45842/0 (socket says 172.21.15.164:45842) 2024-09-19T08:46:51.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.446+0000 7ff8affff640 1 -- 172.21.15.164:0/2941604444 learned_addr learned my addr 172.21.15.164:0/2941604444 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:51.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.446+0000 7ff8af7fe640 1 --2- 172.21.15.164:0/2941604444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8b01036e0 0x7ff8b0198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:51.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.447+0000 7ff8b4f14640 1 --2- 172.21.15.164:0/2941604444 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8b01082a0 0x7ff8b019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:51.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.447+0000 7ff8affff640 1 -- 172.21.15.164:0/2941604444 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8b01082a0 msgr2=0x7ff8b019cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:51.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.447+0000 7ff8affff640 1 --2- 172.21.15.164:0/2941604444 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8b01082a0 0x7ff8b019cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.447+0000 7ff8affff640 1 -- 172.21.15.164:0/2941604444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8b01036e0 msgr2=0x7ff8b0198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:51.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.447+0000 7ff8affff640 1 --2- 172.21.15.164:0/2941604444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8b01036e0 0x7ff8b0198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.447+0000 7ff8affff640 1 -- 172.21.15.164:0/2941604444 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b019d4a0 con 0x7ff8b0102d30 2024-09-19T08:46:51.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.447+0000 7ff8b4f14640 1 --2- 172.21.15.164:0/2941604444 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8b01082a0 0x7ff8b019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:51.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.447+0000 7ff8affff640 1 --2- 172.21.15.164:0/2941604444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8b0102d30 0x7ff8b01984f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0009230 tx=0x7ff8a0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:51.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.447+0000 7ff8ad7fa640 1 -- 172.21.15.164:0/2941604444 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a003f030 con 0x7ff8b0102d30 2024-09-19T08:46:51.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.447+0000 7ff8ad7fa640 1 -- 172.21.15.164:0/2941604444 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8a0003ad0 con 0x7ff8b0102d30 2024-09-19T08:46:51.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.447+0000 7ff8ad7fa640 1 -- 172.21.15.164:0/2941604444 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a003f030 con 0x7ff8b0102d30 2024-09-19T08:46:51.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.447+0000 7ff8b6977640 1 -- 172.21.15.164:0/2941604444 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b019d730 con 0x7ff8b0102d30 2024-09-19T08:46:51.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.448+0000 7ff8b6977640 1 -- 172.21.15.164:0/2941604444 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b019dbc0 con 0x7ff8b0102d30 2024-09-19T08:46:51.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.452+0000 7ff8ad7fa640 1 -- 172.21.15.164:0/2941604444 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7ff8a0042030 con 0x7ff8b0102d30 2024-09-19T08:46:51.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.453+0000 7ff8ad7fa640 1 --2- 172.21.15.164:0/2941604444 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff8980a38d0 0x7ff8980a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:51.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.453+0000 7ff8b8d50740 1 Processor -- start 2024-09-19T08:46:51.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.453+0000 7ff8b8d50740 1 -- start start 2024-09-19T08:46:51.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.453+0000 7ff8ad7fa640 1 -- 172.21.15.164:0/2941604444 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7ff8a0040070 con 0x7ff8b0102d30 2024-09-19T08:46:51.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.453+0000 7ff8af7fe640 1 --2- 172.21.15.164:0/2941604444 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff8980a38d0 0x7ff8980a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:51.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.453+0000 7ff8b8d50740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c1ee33f00 0x563c1ee342e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:51.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.453+0000 7ff8b8d50740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c1ee34820 0x563c1ee43110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:51.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.453+0000 7ff8b8d50740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c1ee43650 0x563c1ee45a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:51.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.453+0000 7ff8b8d50740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563c1ed99620 con 0x563c1ee34820 2024-09-19T08:46:51.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.453+0000 7ff8b8d50740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563c1ed40e50 con 0x563c1ee43650 2024-09-19T08:46:51.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.453+0000 7ff8b8d50740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563c1ed955e0 con 0x563c1ee33f00 2024-09-19T08:46:51.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.454+0000 7ff8affff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c1ee33f00 0x563c1ee342e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:51.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.454+0000 7ff8af7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c1ee34820 0x563c1ee43110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:51.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.454+0000 7ff8af7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c1ee34820 0x563c1ee43110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45846/0 (socket says 172.21.15.164:45846) 2024-09-19T08:46:51.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.454+0000 7ff8af7fe640 1 -- 172.21.15.164:0/544263103 learned_addr learned my addr 172.21.15.164:0/544263103 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:51.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.454+0000 7ff8b4f14640 1 --2- 172.21.15.164:0/544263103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c1ee43650 0x563c1ee45a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:51.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.454+0000 7ff8af7fe640 1 --2- 172.21.15.164:0/2941604444 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff8980a38d0 0x7ff8980a5d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff8b00fedc0 tx=0x7ff8a400f3c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:51.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.454+0000 7ff8b4f14640 1 -- 172.21.15.164:0/544263103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c1ee33f00 msgr2=0x563c1ee342e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:51.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.454+0000 7ff8b4f14640 1 --2- 172.21.15.164:0/544263103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c1ee33f00 0x563c1ee342e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.454+0000 7ff8b4f14640 1 -- 172.21.15.164:0/544263103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c1ee34820 msgr2=0x563c1ee43110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:51.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.454+0000 7ff8b4f14640 1 --2- 172.21.15.164:0/544263103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c1ee34820 0x563c1ee43110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.454+0000 7ff8b4f14640 1 -- 172.21.15.164:0/544263103 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c1ed31d40 con 0x563c1ee43650 2024-09-19T08:46:51.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.455+0000 7ff8b4f14640 1 --2- 172.21.15.164:0/544263103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c1ee43650 0x563c1ee45a40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8009e40 tx=0x7ff8a800f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88e6ebffbd864e53 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:51.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.455+0000 7ff8967fc640 1 -- 172.21.15.164:0/544263103 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a8003b70 con 0x563c1ee43650 2024-09-19T08:46:51.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.455+0000 7ff8967fc640 1 -- 172.21.15.164:0/544263103 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8a8003d10 con 0x563c1ee43650 2024-09-19T08:46:51.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.455+0000 7ff8967fc640 1 -- 172.21.15.164:0/544263103 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a8011940 con 0x563c1ee43650 2024-09-19T08:46:51.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.456+0000 7ff8b8d50740 1 -- 172.21.15.164:0/544263103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c1ee43650 msgr2=0x563c1ee45a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:51.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.456+0000 7ff8b8d50740 1 --2- 172.21.15.164:0/544263103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c1ee43650 0x563c1ee45a40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8009e40 tx=0x7ff8a800f730 comp rx=0 tx=0).stop 2024-09-19T08:46:51.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.456+0000 7ff8b8d50740 1 -- 172.21.15.164:0/544263103 shutdown_connections 2024-09-19T08:46:51.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.456+0000 7ff8b8d50740 1 --2- 172.21.15.164:0/544263103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c1ee43650 0x563c1ee45a40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.456+0000 7ff8b8d50740 1 --2- 172.21.15.164:0/544263103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c1ee34820 0x563c1ee43110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.456+0000 7ff8b8d50740 1 --2- 172.21.15.164:0/544263103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c1ee33f00 0x563c1ee342e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.456+0000 7ff8b8d50740 1 -- 172.21.15.164:0/544263103 >> 172.21.15.164:0/544263103 conn(0x563c1ee305f0 msgr2=0x563c1ee309d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:51.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.456+0000 7ff8b8d50740 1 -- 172.21.15.164:0/544263103 shutdown_connections 2024-09-19T08:46:51.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.456+0000 7ff8b8d50740 1 -- 172.21.15.164:0/544263103 wait complete. 2024-09-19T08:46:51.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.457+0000 7ff8b8d50740 1 -- start start 2024-09-19T08:46:51.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.457+0000 7ff8b8d50740 1 Processor -- start 2024-09-19T08:46:51.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.458+0000 7ff8b8d50740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c1ee33f00 0x563c1ee40500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:51.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.458+0000 7ff8b8d50740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c1ee34820 0x563c1ee40a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:51.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.458+0000 7ff8affff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c1ee33f00 0x563c1ee40500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:51.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.458+0000 7ff8affff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c1ee33f00 0x563c1ee40500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:44484/0 (socket says 172.21.15.164:44484) 2024-09-19T08:46:51.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.458+0000 7ff8af7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c1ee34820 0x563c1ee40a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:51.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.458+0000 7ff8af7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c1ee34820 0x563c1ee40a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45854/0 (socket says 172.21.15.164:45854) 2024-09-19T08:46:51.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.458+0000 7ff8affff640 1 -- 172.21.15.164:0/3174655938 learned_addr learned my addr 172.21.15.164:0/3174655938 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:51.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.458+0000 7ff8b8d50740 1 --2- 172.21.15.164:0/3174655938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c1ee43650 0x563c1ee40f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:51.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.458+0000 7ff8b8d50740 1 -- 172.21.15.164:0/3174655938 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563c1ee46270 con 0x563c1ee34820 2024-09-19T08:46:51.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.458+0000 7ff8b8d50740 1 -- 172.21.15.164:0/3174655938 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563c1ed21f10 con 0x563c1ee33f00 2024-09-19T08:46:51.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.458+0000 7ff8b8d50740 1 -- 172.21.15.164:0/3174655938 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563c1ee463f0 con 0x563c1ee43650 2024-09-19T08:46:51.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.459+0000 7ff8b4f14640 1 --2- 172.21.15.164:0/3174655938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c1ee43650 0x563c1ee40f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:51.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.459+0000 7ff8af7fe640 1 -- 172.21.15.164:0/3174655938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c1ee43650 msgr2=0x563c1ee40f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:51.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.459+0000 7ff8af7fe640 1 --2- 172.21.15.164:0/3174655938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c1ee43650 0x563c1ee40f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.459+0000 7ff8af7fe640 1 -- 172.21.15.164:0/3174655938 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c1ee33f00 msgr2=0x563c1ee40500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:51.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.459+0000 7ff8af7fe640 1 --2- 172.21.15.164:0/3174655938 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c1ee33f00 0x563c1ee40500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.459+0000 7ff8af7fe640 1 -- 172.21.15.164:0/3174655938 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c1eaa8580 con 0x563c1ee34820 2024-09-19T08:46:51.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.460+0000 7ff8af7fe640 1 --2- 172.21.15.164:0/3174655938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c1ee34820 0x563c1ee40a40 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4005d90 tx=0x7ff8a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f7359e5be5a2767 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:51.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.460+0000 7ff88f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:46:51.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.460+0000 7ff88f7fe640 1 -- 172.21.15.164:0/3174655938 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4018070 con 0x563c1ee34820 2024-09-19T08:46:51.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.460+0000 7ff88f7fe640 1 -- 172.21.15.164:0/3174655938 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7ff8a401b5c0 con 0x563c1ee34820 2024-09-19T08:46:51.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.460+0000 7ff88f7fe640 10 client.0 inactive, discarding osd_map(83..83 src has 1..83) 2024-09-19T08:46:51.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.461+0000 7ff8b8d50740 10 client.0 _ll_register_callbacks cb 0x563c1ed45db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:51.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.461+0000 7ff8b8d50740 1 -- 172.21.15.164:0/3174655938 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8a8003d10 con 0x563c1ee34820 2024-09-19T08:46:51.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.461+0000 7ff88f7fe640 1 -- 172.21.15.164:0/3174655938 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff8a401b7c0 con 0x563c1ee34820 2024-09-19T08:46:51.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.461+0000 7ff8b8d50740 10 client.15729 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:51.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.462+0000 7ff8b8d50740 1 -- 172.21.15.164:0/3174655938 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({fsmap=28}) -- 0x563c1eb72960 con 0x563c1ee34820 2024-09-19T08:46:51.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.462+0000 7ff88f7fe640 1 -- 172.21.15.164:0/3174655938 <== mon.0 v2:172.21.15.164:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff8a401b960 con 0x563c1ee34820 2024-09-19T08:46:51.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.462+0000 7ff8b8d50740 10 client.15729 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:51.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.463+0000 7ff8b8d50740 10 client.15729 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:46:51.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.463+0000 7ff8b8d50740 20 client.15729 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:51.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.463+0000 7ff8b8d50740 1 --2- 172.21.15.164:0/3174655938 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x563c1ee52fe0 0x563c1ee55480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:51.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.463+0000 7ff8b8d50740 4 client.15729 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:46:51.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.463+0000 7ff8b8d50740 1 -- 172.21.15.164:0/3174655938 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c1eda6b90 con 0x563c1ee52fe0 2024-09-19T08:46:51.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.463+0000 7ff8affff640 1 --2- 172.21.15.164:0/3174655938 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x563c1ee52fe0 0x563c1ee55480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:51.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.464+0000 7ff88f7fe640 10 client.15729 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:46:51.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.464+0000 7ff8affff640 1 --2- 172.21.15.164:0/3174655938 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x563c1ee52fe0 0x563c1ee55480 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2af5a97e20ebc6a server_cookie=9b58cde5d90aa160 in_seq=0 out_seq=0 2024-09-19T08:46:51.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.465+0000 7ff88f7fe640 1 -- 172.21.15.164:0/3174655938 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563c1eda6b90 con 0x563c1ee52fe0 2024-09-19T08:46:51.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.465+0000 7ff88f7fe640 10 client.15729 handle_command_reply: tid=0 2024-09-19T08:46:51.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.465+0000 7ff8b8d50740 1 client.15729 shutdown 2024-09-19T08:46:51.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.465+0000 7ff8b8d50740 1 -- 172.21.15.164:0/3174655938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c1ee34820 msgr2=0x563c1ee40a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:51.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.465+0000 7ff8b8d50740 1 --2- 172.21.15.164:0/3174655938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c1ee34820 0x563c1ee40a40 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4005d90 tx=0x7ff8a4004730 comp rx=0 tx=0).stop 2024-09-19T08:46:51.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.466+0000 7ff8b8d50740 1 -- 172.21.15.164:0/3174655938 shutdown_connections 2024-09-19T08:46:51.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.466+0000 7ff8b8d50740 1 --2- 172.21.15.164:0/3174655938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c1ee43650 0x563c1ee40f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.466+0000 7ff8b8d50740 1 --2- 172.21.15.164:0/3174655938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c1ee34820 0x563c1ee40a40 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.466+0000 7ff8b8d50740 1 --2- 172.21.15.164:0/3174655938 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x563c1ee52fe0 0x563c1ee55480 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.466+0000 7ff8b8d50740 1 --2- 172.21.15.164:0/3174655938 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c1ee33f00 0x563c1ee40500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.466+0000 7ff88f7fe640 0 client.15729 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:46:51.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.466+0000 7ff8b8d50740 1 -- 172.21.15.164:0/3174655938 >> 172.21.15.164:0/3174655938 conn(0x563c1ee305f0 msgr2=0x563c1ee36350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:51.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.466+0000 7ff8b8d50740 1 -- 172.21.15.164:0/3174655938 shutdown_connections 2024-09-19T08:46:51.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.466+0000 7ff8b8d50740 1 -- 172.21.15.164:0/3174655938 wait complete. 2024-09-19T08:46:51.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.466+0000 7ff8b8d50740 20 client.15729 trim_cache size 0 max 16384 2024-09-19T08:46:51.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.485+0000 7ff8b8d50740 1 Processor -- start 2024-09-19T08:46:51.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.485+0000 7ff8b8d50740 1 -- start start 2024-09-19T08:46:51.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.485+0000 7ff8b8d50740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c1ee33f00 0x563c1ee4ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:51.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.485+0000 7ff8b8d50740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c1ee34820 0x563c1ee4f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:51.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.485+0000 7ff8b8d50740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c1ee43650 0x563c1ee3aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:51.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.485+0000 7ff8b8d50740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b01ab470 con 0x563c1ee43650 2024-09-19T08:46:51.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.485+0000 7ff8b8d50740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b01acfb0 con 0x563c1ee33f00 2024-09-19T08:46:51.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.485+0000 7ff8b8d50740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b01abb40 con 0x563c1ee34820 2024-09-19T08:46:51.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.485+0000 7ff8b4f14640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c1ee43650 0x563c1ee3aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:51.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.485+0000 7ff8b4f14640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c1ee43650 0x563c1ee3aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45858/0 (socket says 172.21.15.164:45858) 2024-09-19T08:46:51.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.485+0000 7ff8b4f14640 1 -- 172.21.15.164:0/1062416853 learned_addr learned my addr 172.21.15.164:0/1062416853 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:51.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.485+0000 7ff8af7fe640 1 --2- 172.21.15.164:0/1062416853 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c1ee34820 0x563c1ee4f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:51.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.485+0000 7ff8affff640 1 --2- 172.21.15.164:0/1062416853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c1ee33f00 0x563c1ee4ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:51.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.486+0000 7ff8b4f14640 1 -- 172.21.15.164:0/1062416853 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c1ee34820 msgr2=0x563c1ee4f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:51.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.486+0000 7ff8b4f14640 1 --2- 172.21.15.164:0/1062416853 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c1ee34820 0x563c1ee4f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.486+0000 7ff8b4f14640 1 -- 172.21.15.164:0/1062416853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c1ee33f00 msgr2=0x563c1ee4ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:51.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.486+0000 7ff8b4f14640 1 --2- 172.21.15.164:0/1062416853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c1ee33f00 0x563c1ee4ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.486+0000 7ff8b4f14640 1 -- 172.21.15.164:0/1062416853 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c1ee45d80 con 0x563c1ee43650 2024-09-19T08:46:51.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.486+0000 7ff8b4f14640 1 --2- 172.21.15.164:0/1062416853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c1ee43650 0x563c1ee3aa40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8011550 tx=0x7ff8a80098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5007b01911819c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:51.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.486+0000 7ff895ffb640 1 -- 172.21.15.164:0/1062416853 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a8003b70 con 0x563c1ee43650 2024-09-19T08:46:51.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.486+0000 7ff895ffb640 1 -- 172.21.15.164:0/1062416853 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8a8011b90 con 0x563c1ee43650 2024-09-19T08:46:51.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.486+0000 7ff895ffb640 1 -- 172.21.15.164:0/1062416853 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a80055b0 con 0x563c1ee43650 2024-09-19T08:46:51.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.487+0000 7ff8b8d50740 1 -- 172.21.15.164:0/1062416853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c1ee43650 msgr2=0x563c1ee3aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:51.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.487+0000 7ff8b8d50740 1 --2- 172.21.15.164:0/1062416853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c1ee43650 0x563c1ee3aa40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8011550 tx=0x7ff8a80098e0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.487+0000 7ff8b8d50740 1 -- 172.21.15.164:0/1062416853 shutdown_connections 2024-09-19T08:46:51.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.487+0000 7ff8b8d50740 1 --2- 172.21.15.164:0/1062416853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c1ee43650 0x563c1ee3aa40 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.487+0000 7ff8b8d50740 1 --2- 172.21.15.164:0/1062416853 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c1ee34820 0x563c1ee4f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.487+0000 7ff8b8d50740 1 --2- 172.21.15.164:0/1062416853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c1ee33f00 0x563c1ee4ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.487+0000 7ff8b8d50740 1 -- 172.21.15.164:0/1062416853 >> 172.21.15.164:0/1062416853 conn(0x563c1ee305f0 msgr2=0x563c1ee4e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:51.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.487+0000 7ff8b8d50740 1 -- 172.21.15.164:0/1062416853 shutdown_connections 2024-09-19T08:46:51.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.487+0000 7ff8b8d50740 1 -- 172.21.15.164:0/1062416853 wait complete. 2024-09-19T08:46:51.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.487+0000 7ff8b8d50740 1 -- start start 2024-09-19T08:46:51.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.488+0000 7ff8b8d50740 1 Processor -- start 2024-09-19T08:46:51.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.488+0000 7ff8b8d50740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c1ee33f00 0x563c1ee38600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:51.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.488+0000 7ff8b8d50740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c1ee34820 0x563c1ee38b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:51.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.488+0000 7ff8b8d50740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c1ee43650 0x563c1ee4b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:51.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.488+0000 7ff8b8d50740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b01ac8e0 con 0x563c1ee34820 2024-09-19T08:46:51.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.488+0000 7ff8b8d50740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b01ac210 con 0x563c1ee33f00 2024-09-19T08:46:51.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.488+0000 7ff8b8d50740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b01a9470 con 0x563c1ee43650 2024-09-19T08:46:51.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.488+0000 7ff8b8d50740 10 client.0 _ll_register_callbacks cb 0x7ff8a4010ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:51.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.488+0000 7ff8af7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c1ee34820 0x563c1ee38b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:51.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.488+0000 7ff8af7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c1ee34820 0x563c1ee38b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45860/0 (socket says 172.21.15.164:45860) 2024-09-19T08:46:51.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.488+0000 7ff8af7fe640 1 -- 172.21.15.164:0/3685190087 learned_addr learned my addr 172.21.15.164:0/3685190087 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:51.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.488+0000 7ff8b4f14640 1 --2- 172.21.15.164:0/3685190087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c1ee43650 0x563c1ee4b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:51.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.489+0000 7ff8affff640 1 --2- 172.21.15.164:0/3685190087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c1ee33f00 0x563c1ee38600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:51.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.489+0000 7ff8af7fe640 1 -- 172.21.15.164:0/3685190087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c1ee43650 msgr2=0x563c1ee4b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:51.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.489+0000 7ff8af7fe640 1 --2- 172.21.15.164:0/3685190087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c1ee43650 0x563c1ee4b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.489+0000 7ff8af7fe640 1 -- 172.21.15.164:0/3685190087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c1ee33f00 msgr2=0x563c1ee38600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:51.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.489+0000 7ff8af7fe640 1 --2- 172.21.15.164:0/3685190087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c1ee33f00 0x563c1ee38600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.489+0000 7ff8af7fe640 1 -- 172.21.15.164:0/3685190087 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c1ee4bc70 con 0x563c1ee34820 2024-09-19T08:46:51.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.489+0000 7ff8af7fe640 1 --2- 172.21.15.164:0/3685190087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c1ee34820 0x563c1ee38b40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4005d90 tx=0x7ff8a400a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b9f5e42a94df6c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:51.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.489+0000 7ff88f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:46:51.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.489+0000 7ff88f7fe640 1 -- 172.21.15.164:0/3685190087 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4018040 con 0x563c1ee34820 2024-09-19T08:46:51.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.489+0000 7ff88f7fe640 1 -- 172.21.15.164:0/3685190087 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7ff8a401b5c0 con 0x563c1ee34820 2024-09-19T08:46:51.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.489+0000 7ff8b8d50740 1 -- 172.21.15.164:0/3685190087 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8a8011b90 con 0x563c1ee34820 2024-09-19T08:46:51.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.490+0000 7ff88f7fe640 1 -- 172.21.15.164:0/3685190087 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff8a4005be0 con 0x563c1ee34820 2024-09-19T08:46:51.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.490+0000 7ff8b8d50740 10 client.15735 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:51.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.490+0000 7ff8b8d50740 1 -- 172.21.15.164:0/3685190087 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563c1eec2f70 con 0x563c1ee34820 2024-09-19T08:46:51.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.490+0000 7ff88f7fe640 1 -- 172.21.15.164:0/3685190087 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8a8003d10 con 0x563c1ee34820 2024-09-19T08:46:51.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.490+0000 7ff88f7fe640 1 -- 172.21.15.164:0/3685190087 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff8a4041050 con 0x563c1ee34820 2024-09-19T08:46:51.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.490+0000 7ff8b8d50740 10 client.15735 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:51.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.490+0000 7ff8b8d50740 10 client.15735 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:46:51.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.490+0000 7ff8b8d50740 20 client.15735 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:51.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.490+0000 7ff8b8d50740 1 --2- 172.21.15.164:0/3685190087 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x563c1eec36f0 0x563c1eec5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:51.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.490+0000 7ff8b8d50740 4 client.15735 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "446cbd86", "release": true, "await": true}] 2024-09-19T08:46:51.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.490+0000 7ff8b8d50740 1 -- 172.21.15.164:0/3685190087 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "quiesce db", "set_id": "446cbd86", "release": true, "await": true}) -- 0x7ff8a8011b90 con 0x563c1eec36f0 2024-09-19T08:46:51.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.490+0000 7ff8affff640 1 --2- 172.21.15.164:0/3685190087 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x563c1eec36f0 0x563c1eec5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:51.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.491+0000 7ff88f7fe640 10 client.15735 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:46:51.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.491+0000 7ff8affff640 1 --2- 172.21.15.164:0/3685190087 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x563c1eec36f0 0x563c1eec5b10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=731cadd3f33f799a server_cookie=cfeb721415729b8b in_seq=0 out_seq=0 2024-09-19T08:46:51.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.492+0000 7ff88f7fe640 1 -- 172.21.15.164:0/3685190087 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7ff8a8011b90 con 0x563c1eec36f0 2024-09-19T08:46:51.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.492+0000 7ff88f7fe640 10 client.15735 handle_command_reply: tid=0 2024-09-19T08:46:51.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.492+0000 7ff8b8d50740 1 client.15735 shutdown 2024-09-19T08:46:51.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.493+0000 7ff8b8d50740 1 -- 172.21.15.164:0/3685190087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c1ee34820 msgr2=0x563c1ee38b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:51.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.493+0000 7ff8b8d50740 1 --2- 172.21.15.164:0/3685190087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c1ee34820 0x563c1ee38b40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4005d90 tx=0x7ff8a400a2d0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.493+0000 7ff8b8d50740 1 -- 172.21.15.164:0/3685190087 shutdown_connections 2024-09-19T08:46:51.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.493+0000 7ff8b8d50740 1 --2- 172.21.15.164:0/3685190087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c1ee43650 0x563c1ee4b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.493+0000 7ff8b8d50740 1 --2- 172.21.15.164:0/3685190087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c1ee34820 0x563c1ee38b40 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.493+0000 7ff8b8d50740 1 --2- 172.21.15.164:0/3685190087 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x563c1eec36f0 0x563c1eec5b10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.493+0000 7ff8b8d50740 1 --2- 172.21.15.164:0/3685190087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c1ee33f00 0x563c1ee38600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.493+0000 7ff8b8d50740 1 -- 172.21.15.164:0/3685190087 >> 172.21.15.164:0/3685190087 conn(0x563c1ee305f0 msgr2=0x563c1ee4ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:51.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.493+0000 7ff88f7fe640 0 client.15735 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:46:51.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.493+0000 7ff8b8d50740 1 -- 172.21.15.164:0/3685190087 shutdown_connections 2024-09-19T08:46:51.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.493+0000 7ff8b8d50740 1 -- 172.21.15.164:0/3685190087 wait complete. 2024-09-19T08:46:51.498 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.493+0000 7ff8b8d50740 20 client.15735 trim_cache size 0 max 16384 2024-09-19T08:46:51.498 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:46:51.498 INFO:teuthology.orchestra.run.smithi164.stdout: "epoch": 28, 2024-09-19T08:46:51.498 INFO:teuthology.orchestra.run.smithi164.stdout: "leader": 24479, 2024-09-19T08:46:51.498 INFO:teuthology.orchestra.run.smithi164.stdout: "set_version": 4, 2024-09-19T08:46:51.498 INFO:teuthology.orchestra.run.smithi164.stdout: "sets": { 2024-09-19T08:46:51.498 INFO:teuthology.orchestra.run.smithi164.stdout: "446cbd86": { 2024-09-19T08:46:51.498 INFO:teuthology.orchestra.run.smithi164.stdout: "version": 4, 2024-09-19T08:46:51.498 INFO:teuthology.orchestra.run.smithi164.stdout: "age_ref": 0.0, 2024-09-19T08:46:51.498 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:46:51.499 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "RELEASED", 2024-09-19T08:46:51.499 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:46:51.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:46:51.499 INFO:teuthology.orchestra.run.smithi164.stdout: "timeout": 90.0, 2024-09-19T08:46:51.499 INFO:teuthology.orchestra.run.smithi164.stdout: "expiration": 131.2, 2024-09-19T08:46:51.499 INFO:teuthology.orchestra.run.smithi164.stdout: "members": { 2024-09-19T08:46:51.499 INFO:teuthology.orchestra.run.smithi164.stdout: "file:/": { 2024-09-19T08:46:51.499 INFO:teuthology.orchestra.run.smithi164.stdout: "excluded": false, 2024-09-19T08:46:51.499 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:46:51.499 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "RELEASED", 2024-09-19T08:46:51.499 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:46:51.499 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:46:51.499 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:46:51.499 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:46:51.500 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:46:51.500 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:46:51.500 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:46:51.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.494+0000 7ff8b6977640 1 -- 172.21.15.164:0/2941604444 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff8980a38d0 msgr2=0x7ff8980a5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:51.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.494+0000 7ff8b6977640 1 --2- 172.21.15.164:0/2941604444 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff8980a38d0 0x7ff8980a5d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff8b00fedc0 tx=0x7ff8a400f3c0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.494+0000 7ff8b6977640 1 -- 172.21.15.164:0/2941604444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8b0102d30 msgr2=0x7ff8b01984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:51.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.494+0000 7ff8b6977640 1 --2- 172.21.15.164:0/2941604444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8b0102d30 0x7ff8b01984f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0009230 tx=0x7ff8a0004270 comp rx=0 tx=0).stop 2024-09-19T08:46:51.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.494+0000 7ff8b6977640 1 -- 172.21.15.164:0/2941604444 shutdown_connections 2024-09-19T08:46:51.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.494+0000 7ff8b6977640 1 --2- 172.21.15.164:0/2941604444 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8b01082a0 0x7ff8b019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.494+0000 7ff8b6977640 1 --2- 172.21.15.164:0/2941604444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8b01036e0 0x7ff8b0198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.494+0000 7ff8b6977640 1 --2- 172.21.15.164:0/2941604444 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff8980a38d0 0x7ff8980a5d90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.494+0000 7ff8b6977640 1 --2- 172.21.15.164:0/2941604444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8b0102d30 0x7ff8b01984f0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.494+0000 7ff8b6977640 1 -- 172.21.15.164:0/2941604444 >> 172.21.15.164:0/2941604444 conn(0x7ff8b00fc7d0 msgr2=0x7ff8b0105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:51.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.494+0000 7ff8b6977640 1 -- 172.21.15.164:0/2941604444 shutdown_connections 2024-09-19T08:46:51.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:51.494+0000 7ff8b6977640 1 -- 172.21.15.164:0/2941604444 wait complete. 2024-09-19T08:46:51.927 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T08:46:52.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:51 smithi164 ceph-mon[30928]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 853 B/s wr, 13 op/s 2024-09-19T08:46:52.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:51 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/500219862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:46:52.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:51 smithi170 ceph-mon[35072]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 853 B/s wr, 13 op/s 2024-09-19T08:46:52.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:51 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/500219862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:46:52.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:51 smithi173 ceph-mon[35306]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 853 B/s wr, 13 op/s 2024-09-19T08:46:52.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:51 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/500219862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:46:52.717 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '446cbd86', total seconds elapsed: 19.0 2024-09-19T08:46:52.821 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:46:53.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.384+0000 7fb5a8782640 1 -- 172.21.15.164:0/4274402537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5a0101400 msgr2=0x7fb5a010d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:53.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.384+0000 7fb5a8782640 1 --2- 172.21.15.164:0/4274402537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5a0101400 0x7fb5a010d930 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb598009cd0 tx=0x7fb59802a8e0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.384+0000 7fb5a8782640 1 -- 172.21.15.164:0/4274402537 shutdown_connections 2024-09-19T08:46:53.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.384+0000 7fb5a8782640 1 --2- 172.21.15.164:0/4274402537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5a0101400 0x7fb5a010d930 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.384+0000 7fb5a8782640 1 --2- 172.21.15.164:0/4274402537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5a0100a60 0x7fb5a0100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.384+0000 7fb5a8782640 1 --2- 172.21.15.164:0/4274402537 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5a0106a60 0x7fb5a0106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.384+0000 7fb5a8782640 1 -- 172.21.15.164:0/4274402537 >> 172.21.15.164:0/4274402537 conn(0x7fb5a00fc780 msgr2=0x7fb5a00feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:53.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.384+0000 7fb5a8782640 1 -- 172.21.15.164:0/4274402537 shutdown_connections 2024-09-19T08:46:53.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.384+0000 7fb5a8782640 1 -- 172.21.15.164:0/4274402537 wait complete. 2024-09-19T08:46:53.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.385+0000 7fb5a8782640 1 Processor -- start 2024-09-19T08:46:53.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.385+0000 7fb5a8782640 1 -- start start 2024-09-19T08:46:53.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.385+0000 7fb5a8782640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5a0100a60 0x7fb5a0198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:53.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.385+0000 7fb5a8782640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5a0101400 0x7fb5a0198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:53.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.385+0000 7fb5a8782640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5a0106a60 0x7fb5a019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:53.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.385+0000 7fb5a8782640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a0110e60 con 0x7fb5a0106a60 2024-09-19T08:46:53.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.385+0000 7fb5a8782640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a0110ce0 con 0x7fb5a0101400 2024-09-19T08:46:53.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.385+0000 7fb5a8782640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a0110fe0 con 0x7fb5a0100a60 2024-09-19T08:46:53.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.386+0000 7fb5a6d1f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5a0106a60 0x7fb5a019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:53.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.386+0000 7fb5a6d1f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5a0106a60 0x7fb5a019ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45884/0 (socket says 172.21.15.164:45884) 2024-09-19T08:46:53.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.386+0000 7fb5a6d1f640 1 -- 172.21.15.164:0/719096589 learned_addr learned my addr 172.21.15.164:0/719096589 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:53.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.386+0000 7fb5a651e640 1 --2- 172.21.15.164:0/719096589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5a0100a60 0x7fb5a0198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:53.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.386+0000 7fb5a5d1d640 1 --2- 172.21.15.164:0/719096589 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5a0101400 0x7fb5a0198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:53.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.386+0000 7fb5a6d1f640 1 -- 172.21.15.164:0/719096589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5a0100a60 msgr2=0x7fb5a0198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:53.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.386+0000 7fb5a6d1f640 1 --2- 172.21.15.164:0/719096589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5a0100a60 0x7fb5a0198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.386+0000 7fb5a6d1f640 1 -- 172.21.15.164:0/719096589 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5a0101400 msgr2=0x7fb5a0198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:53.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.386+0000 7fb5a6d1f640 1 --2- 172.21.15.164:0/719096589 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5a0101400 0x7fb5a0198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.386+0000 7fb5a6d1f640 1 -- 172.21.15.164:0/719096589 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5a019d3d0 con 0x7fb5a0106a60 2024-09-19T08:46:53.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.386+0000 7fb5a6d1f640 1 --2- 172.21.15.164:0/719096589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5a0106a60 0x7fb5a019ccf0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb598005cc0 tx=0x7fb598003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:53.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.387+0000 7fb5977fe640 1 -- 172.21.15.164:0/719096589 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb598004040 con 0x7fb5a0106a60 2024-09-19T08:46:53.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.387+0000 7fb5977fe640 1 -- 172.21.15.164:0/719096589 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5980046c0 con 0x7fb5a0106a60 2024-09-19T08:46:53.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.387+0000 7fb5a8782640 1 -- 172.21.15.164:0/719096589 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5a019d660 con 0x7fb5a0106a60 2024-09-19T08:46:53.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.387+0000 7fb5977fe640 1 -- 172.21.15.164:0/719096589 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb598046480 con 0x7fb5a0106a60 2024-09-19T08:46:53.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.388+0000 7fb5a8782640 1 -- 172.21.15.164:0/719096589 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5a01a4fa0 con 0x7fb5a0106a60 2024-09-19T08:46:53.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.391+0000 7fb5977fe640 1 -- 172.21.15.164:0/719096589 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7fb5980305a0 con 0x7fb5a0106a60 2024-09-19T08:46:53.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.391+0000 7fb5977fe640 1 --2- 172.21.15.164:0/719096589 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb58c0a3550 0x7fb58c0a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:53.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.392+0000 7fb5aab5b740 1 Processor -- start 2024-09-19T08:46:53.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.392+0000 7fb5977fe640 1 -- 172.21.15.164:0/719096589 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fb59802e030 con 0x7fb5a0106a60 2024-09-19T08:46:53.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.392+0000 7fb5a651e640 1 --2- 172.21.15.164:0/719096589 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb58c0a3550 0x7fb58c0a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:53.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.392+0000 7fb5aab5b740 1 -- start start 2024-09-19T08:46:53.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.392+0000 7fb5aab5b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555622582f50 0x5556225814a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:53.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.392+0000 7fb5aab5b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5556225819e0 0x555622581e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:53.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.392+0000 7fb5aab5b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55562258c780 0x55562258eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:53.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.392+0000 7fb5aab5b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5556224e32f0 con 0x55562258c780 2024-09-19T08:46:53.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.392+0000 7fb5aab5b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55562248acf0 con 0x555622582f50 2024-09-19T08:46:53.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.392+0000 7fb5aab5b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5556224df2b0 con 0x5556225819e0 2024-09-19T08:46:53.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.393+0000 7fb5a5d1d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5556225819e0 0x555622581e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:53.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.393+0000 7fb5a5d1d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5556225819e0 0x555622581e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:49184/0 (socket says 172.21.15.164:49184) 2024-09-19T08:46:53.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.393+0000 7fb5a6d1f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55562258c780 0x55562258eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:53.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.393+0000 7fb5a5d1d640 1 -- 172.21.15.164:0/2298277256 learned_addr learned my addr 172.21.15.164:0/2298277256 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:53.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.393+0000 7fb5a651e640 1 --2- 172.21.15.164:0/719096589 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb58c0a3550 0x7fb58c0a5a10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb59c005d00 tx=0x7fb59c00b410 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:53.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.393+0000 7fb5a6d1f640 1 -- 172.21.15.164:0/2298277256 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5556225819e0 msgr2=0x555622581e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:53.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.393+0000 7fb5a6d1f640 1 --2- 172.21.15.164:0/2298277256 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5556225819e0 0x555622581e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.393+0000 7fb5a6d1f640 1 -- 172.21.15.164:0/2298277256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555622582f50 msgr2=0x5556225814a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:53.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.393+0000 7fb5a651e640 1 --2- 172.21.15.164:0/2298277256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555622582f50 0x5556225814a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:53.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.393+0000 7fb5a6d1f640 1 --2- 172.21.15.164:0/2298277256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555622582f50 0x5556225814a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.393+0000 7fb5a6d1f640 1 -- 172.21.15.164:0/2298277256 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55562246f3a0 con 0x55562258c780 2024-09-19T08:46:53.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.393+0000 7fb5a651e640 1 --2- 172.21.15.164:0/2298277256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555622582f50 0x5556225814a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:53.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.393+0000 7fb5a6d1f640 1 --2- 172.21.15.164:0/2298277256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55562258c780 0x55562258eb70 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb5980ee6e0 tx=0x7fb598003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffbaa0aaec2dcae1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:53.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.394+0000 7fb594ff9640 1 -- 172.21.15.164:0/2298277256 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5980bd8a0 con 0x55562258c780 2024-09-19T08:46:53.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.394+0000 7fb594ff9640 1 -- 172.21.15.164:0/2298277256 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb598036020 con 0x55562258c780 2024-09-19T08:46:53.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.394+0000 7fb594ff9640 1 -- 172.21.15.164:0/2298277256 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5980c1110 con 0x55562258c780 2024-09-19T08:46:53.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.395+0000 7fb5aab5b740 1 -- 172.21.15.164:0/2298277256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55562258c780 msgr2=0x55562258eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:53.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.395+0000 7fb5aab5b740 1 --2- 172.21.15.164:0/2298277256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55562258c780 0x55562258eb70 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb5980ee6e0 tx=0x7fb598003f70 comp rx=0 tx=0).stop 2024-09-19T08:46:53.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.395+0000 7fb5aab5b740 1 -- 172.21.15.164:0/2298277256 shutdown_connections 2024-09-19T08:46:53.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.395+0000 7fb5aab5b740 1 --2- 172.21.15.164:0/2298277256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55562258c780 0x55562258eb70 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.395+0000 7fb5aab5b740 1 --2- 172.21.15.164:0/2298277256 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5556225819e0 0x555622581e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.395+0000 7fb5aab5b740 1 --2- 172.21.15.164:0/2298277256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555622582f50 0x5556225814a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.395+0000 7fb5aab5b740 1 -- 172.21.15.164:0/2298277256 >> 172.21.15.164:0/2298277256 conn(0x555622546310 msgr2=0x55562257e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:53.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.395+0000 7fb5aab5b740 1 -- 172.21.15.164:0/2298277256 shutdown_connections 2024-09-19T08:46:53.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.395+0000 7fb5aab5b740 1 -- 172.21.15.164:0/2298277256 wait complete. 2024-09-19T08:46:53.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.395+0000 7fb5aab5b740 1 -- start start 2024-09-19T08:46:53.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.395+0000 7fb5aab5b740 1 Processor -- start 2024-09-19T08:46:53.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.396+0000 7fb5aab5b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5556225819e0 0x55562258c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:53.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.396+0000 7fb5aab5b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555622582f50 0x5556225894a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:53.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.396+0000 7fb5aab5b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55562258c780 0x555622589be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:53.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.396+0000 7fb5aab5b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55562258f240 con 0x5556225819e0 2024-09-19T08:46:53.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.396+0000 7fb5aab5b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55562249e060 con 0x55562258c780 2024-09-19T08:46:53.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.396+0000 7fb5aab5b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55562258f3c0 con 0x555622582f50 2024-09-19T08:46:53.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.396+0000 7fb5aab5b740 10 client.0 _ll_register_callbacks cb 0x5556221f18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:53.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.396+0000 7fb5a5d1d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555622582f50 0x5556225894a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:53.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.396+0000 7fb5a651e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5556225819e0 0x55562258c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:53.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.396+0000 7fb5a5d1d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555622582f50 0x5556225894a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:49200/0 (socket says 172.21.15.164:49200) 2024-09-19T08:46:53.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.396+0000 7fb5a6d1f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55562258c780 0x555622589be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:53.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.396+0000 7fb5a5d1d640 1 -- 172.21.15.164:0/2007730472 learned_addr learned my addr 172.21.15.164:0/2007730472 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:53.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.396+0000 7fb5a5d1d640 1 -- 172.21.15.164:0/2007730472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55562258c780 msgr2=0x555622589be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:53.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.396+0000 7fb5a5d1d640 1 --2- 172.21.15.164:0/2007730472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55562258c780 0x555622589be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.396+0000 7fb5a5d1d640 1 -- 172.21.15.164:0/2007730472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5556225819e0 msgr2=0x55562258c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:53.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.397+0000 7fb5a5d1d640 1 --2- 172.21.15.164:0/2007730472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5556225819e0 0x55562258c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.397+0000 7fb5a5d1d640 1 -- 172.21.15.164:0/2007730472 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556222543c0 con 0x555622582f50 2024-09-19T08:46:53.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.397+0000 7fb5a651e640 1 --2- 172.21.15.164:0/2007730472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5556225819e0 0x55562258c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:46:53.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.397+0000 7fb5a5d1d640 1 --2- 172.21.15.164:0/2007730472 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555622582f50 0x5556225894a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb59000bad0 tx=0x7fb59000bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d517c944c4eb236 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:53.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.397+0000 7fb589ffb640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-09-19T08:46:53.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.397+0000 7fb589ffb640 1 -- 172.21.15.164:0/2007730472 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb590011440 con 0x555622582f50 2024-09-19T08:46:53.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.397+0000 7fb5aab5b740 1 -- 172.21.15.164:0/2007730472 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb598036020 con 0x555622582f50 2024-09-19T08:46:53.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.397+0000 7fb589ffb640 1 -- 172.21.15.164:0/2007730472 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fb5900115e0 con 0x555622582f50 2024-09-19T08:46:53.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.397+0000 7fb589ffb640 1 -- 172.21.15.164:0/2007730472 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb590011820 con 0x555622582f50 2024-09-19T08:46:53.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.397+0000 7fb5aab5b740 10 client.25285 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:53.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.397+0000 7fb5aab5b740 1 -- 172.21.15.164:0/2007730472 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55562257af10 con 0x555622582f50 2024-09-19T08:46:53.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.398+0000 7fb589ffb640 1 -- 172.21.15.164:0/2007730472 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb598036020 con 0x555622582f50 2024-09-19T08:46:53.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.398+0000 7fb589ffb640 1 -- 172.21.15.164:0/2007730472 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb59001bb80 con 0x555622582f50 2024-09-19T08:46:53.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.398+0000 7fb5aab5b740 10 client.25285 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:53.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.398+0000 7fb5aab5b740 10 client.25285 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-19T08:46:53.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.398+0000 7fb5aab5b740 20 client.25285 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:53.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.398+0000 7fb5aab5b740 1 --2- 172.21.15.164:0/2007730472 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55562257c790 0x55562257cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:53.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.398+0000 7fb5a651e640 1 --2- 172.21.15.164:0/2007730472 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55562257c790 0x55562257cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:53.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.399+0000 7fb5a651e640 1 --2- 172.21.15.164:0/2007730472 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55562257c790 0x55562257cc20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=378ae85fa3110806 server_cookie=f2ad85c871a21085 in_seq=0 out_seq=0 2024-09-19T08:46:53.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.399+0000 7fb5aab5b740 4 client.25285 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:46:53.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.399+0000 7fb5aab5b740 1 -- 172.21.15.164:0/2007730472 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556224f0860 con 0x55562257c790 2024-09-19T08:46:53.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.399+0000 7fb589ffb640 10 client.25285 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:46:53.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.401+0000 7fb589ffb640 1 -- 172.21.15.164:0/2007730472 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5556224f0860 con 0x55562257c790 2024-09-19T08:46:53.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.401+0000 7fb589ffb640 10 client.25285 handle_command_reply: tid=0 2024-09-19T08:46:53.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.401+0000 7fb5aab5b740 1 client.25285 shutdown 2024-09-19T08:46:53.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.401+0000 7fb5aab5b740 1 -- 172.21.15.164:0/2007730472 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555622582f50 msgr2=0x5556225894a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:53.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.401+0000 7fb5aab5b740 1 --2- 172.21.15.164:0/2007730472 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555622582f50 0x5556225894a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb59000bad0 tx=0x7fb59000bed0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.401+0000 7fb5aab5b740 1 -- 172.21.15.164:0/2007730472 shutdown_connections 2024-09-19T08:46:53.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.401+0000 7fb5aab5b740 1 --2- 172.21.15.164:0/2007730472 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55562257c790 0x55562257cc20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.401+0000 7fb5aab5b740 1 --2- 172.21.15.164:0/2007730472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55562258c780 0x555622589be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.401+0000 7fb5aab5b740 1 --2- 172.21.15.164:0/2007730472 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555622582f50 0x5556225894a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.401+0000 7fb5aab5b740 1 --2- 172.21.15.164:0/2007730472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5556225819e0 0x55562258c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.401+0000 7fb5aab5b740 1 -- 172.21.15.164:0/2007730472 >> 172.21.15.164:0/2007730472 conn(0x555622546310 msgr2=0x555622583da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:53.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.401+0000 7fb589ffb640 0 client.25285 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:46:53.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.401+0000 7fb5aab5b740 1 -- 172.21.15.164:0/2007730472 shutdown_connections 2024-09-19T08:46:53.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.401+0000 7fb5aab5b740 1 -- 172.21.15.164:0/2007730472 wait complete. 2024-09-19T08:46:53.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.401+0000 7fb5aab5b740 20 client.25285 trim_cache size 0 max 16384 2024-09-19T08:46:53.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.426+0000 7fb5aab5b740 1 Processor -- start 2024-09-19T08:46:53.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.426+0000 7fb5aab5b740 1 -- start start 2024-09-19T08:46:53.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.426+0000 7fb5aab5b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5556225819e0 0x555622588480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:53.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.426+0000 7fb5aab5b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555622582f50 0x555622584530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:53.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.426+0000 7fb5aab5b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55562258c780 0x555622584a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:53.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.426+0000 7fb5aab5b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a01ab4e0 con 0x5556225819e0 2024-09-19T08:46:53.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.426+0000 7fb5aab5b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a01ad020 con 0x555622582f50 2024-09-19T08:46:53.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.426+0000 7fb5aab5b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a01abbb0 con 0x55562258c780 2024-09-19T08:46:53.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.427+0000 7fb5a5d1d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555622582f50 0x555622584530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:53.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.427+0000 7fb5a5d1d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555622582f50 0x555622584530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:41908/0 (socket says 172.21.15.164:41908) 2024-09-19T08:46:53.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.427+0000 7fb5a5d1d640 1 -- 172.21.15.164:0/2523789267 learned_addr learned my addr 172.21.15.164:0/2523789267 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:53.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.427+0000 7fb5a651e640 1 --2- 172.21.15.164:0/2523789267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5556225819e0 0x555622588480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:53.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.427+0000 7fb5a6d1f640 1 --2- 172.21.15.164:0/2523789267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55562258c780 0x555622584a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:53.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.427+0000 7fb5a5d1d640 1 -- 172.21.15.164:0/2523789267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55562258c780 msgr2=0x555622584a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:53.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.427+0000 7fb5a5d1d640 1 --2- 172.21.15.164:0/2523789267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55562258c780 0x555622584a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.427+0000 7fb5a5d1d640 1 -- 172.21.15.164:0/2523789267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5556225819e0 msgr2=0x555622588480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:53.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.427+0000 7fb5a5d1d640 1 --2- 172.21.15.164:0/2523789267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5556225819e0 0x555622588480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.427+0000 7fb5a5d1d640 1 -- 172.21.15.164:0/2523789267 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555622582b70 con 0x555622582f50 2024-09-19T08:46:53.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.427+0000 7fb5a651e640 1 --2- 172.21.15.164:0/2523789267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5556225819e0 0x555622588480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:53.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.427+0000 7fb5a5d1d640 1 --2- 172.21.15.164:0/2523789267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555622582f50 0x555622584530 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb590004280 tx=0x7fb5900044e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db2de44d571b1837 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:53.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.428+0000 7fb58bfff640 1 -- 172.21.15.164:0/2523789267 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb590028070 con 0x555622582f50 2024-09-19T08:46:53.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.428+0000 7fb58bfff640 1 -- 172.21.15.164:0/2523789267 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb59000b480 con 0x555622582f50 2024-09-19T08:46:53.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.428+0000 7fb58bfff640 1 -- 172.21.15.164:0/2523789267 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb590011c20 con 0x555622582f50 2024-09-19T08:46:53.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.429+0000 7fb5aab5b740 1 -- 172.21.15.164:0/2523789267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555622582f50 msgr2=0x555622584530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:53.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.429+0000 7fb5aab5b740 1 --2- 172.21.15.164:0/2523789267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555622582f50 0x555622584530 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb590004280 tx=0x7fb5900044e0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.429+0000 7fb5aab5b740 1 -- 172.21.15.164:0/2523789267 shutdown_connections 2024-09-19T08:46:53.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.429+0000 7fb5aab5b740 1 --2- 172.21.15.164:0/2523789267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55562258c780 0x555622584a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.429+0000 7fb5aab5b740 1 --2- 172.21.15.164:0/2523789267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555622582f50 0x555622584530 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.429+0000 7fb5aab5b740 1 --2- 172.21.15.164:0/2523789267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5556225819e0 0x555622588480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.429+0000 7fb5aab5b740 1 -- 172.21.15.164:0/2523789267 >> 172.21.15.164:0/2523789267 conn(0x555622546310 msgr2=0x555622587d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:53.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.429+0000 7fb5aab5b740 1 -- 172.21.15.164:0/2523789267 shutdown_connections 2024-09-19T08:46:53.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.429+0000 7fb5aab5b740 1 -- 172.21.15.164:0/2523789267 wait complete. 2024-09-19T08:46:53.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.429+0000 7fb5aab5b740 1 -- start start 2024-09-19T08:46:53.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.429+0000 7fb5aab5b740 1 Processor -- start 2024-09-19T08:46:53.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.430+0000 7fb5aab5b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5556225819e0 0x5556225a94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:53.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.430+0000 7fb5aab5b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555622582f50 0x5556225a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:53.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.430+0000 7fb5aab5b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55562258c780 0x5556225a3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:53.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.430+0000 7fb5aab5b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a01ac950 con 0x55562258c780 2024-09-19T08:46:53.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.430+0000 7fb5aab5b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a01ac280 con 0x555622582f50 2024-09-19T08:46:53.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.430+0000 7fb5aab5b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a01a9500 con 0x5556225819e0 2024-09-19T08:46:53.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.430+0000 7fb5aab5b740 10 client.0 _ll_register_callbacks cb 0x7fb5900040a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:53.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.430+0000 7fb5a651e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5556225819e0 0x5556225a94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:53.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.430+0000 7fb5a651e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5556225819e0 0x5556225a94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:35108/0 (socket says 172.21.15.164:35108) 2024-09-19T08:46:53.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.430+0000 7fb5a651e640 1 -- 172.21.15.164:0/4197610022 learned_addr learned my addr 172.21.15.164:0/4197610022 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.430+0000 7fb5a6d1f640 1 --2- 172.21.15.164:0/4197610022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55562258c780 0x5556225a3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.431+0000 7fb5a5d1d640 1 --2- 172.21.15.164:0/4197610022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555622582f50 0x5556225a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.431+0000 7fb5a651e640 1 -- 172.21.15.164:0/4197610022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555622582f50 msgr2=0x5556225a37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.431+0000 7fb5a651e640 1 --2- 172.21.15.164:0/4197610022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555622582f50 0x5556225a37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.431+0000 7fb5a651e640 1 -- 172.21.15.164:0/4197610022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55562258c780 msgr2=0x5556225a3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.431+0000 7fb5a651e640 1 --2- 172.21.15.164:0/4197610022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55562258c780 0x5556225a3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.431+0000 7fb5a651e640 1 -- 172.21.15.164:0/4197610022 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555622609bc0 con 0x5556225819e0 2024-09-19T08:46:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.431+0000 7fb5a6d1f640 1 --2- 172.21.15.164:0/4197610022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55562258c780 0x5556225a3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:46:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.431+0000 7fb5a651e640 1 --2- 172.21.15.164:0/4197610022 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5556225819e0 0x5556225a94b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb59c010ee0 tx=0x7fb59c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88d7820b55d99016 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.431+0000 7fb589ffb640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-09-19T08:46:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.431+0000 7fb589ffb640 1 -- 172.21.15.164:0/4197610022 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c011d10 con 0x5556225819e0 2024-09-19T08:46:53.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.431+0000 7fb589ffb640 1 -- 172.21.15.164:0/4197610022 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fb59c02a420 con 0x5556225819e0 2024-09-19T08:46:53.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.431+0000 7fb5aab5b740 1 -- 172.21.15.164:0/4197610022 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb59000b480 con 0x5556225819e0 2024-09-19T08:46:53.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.432+0000 7fb589ffb640 1 -- 172.21.15.164:0/4197610022 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb59c02a620 con 0x5556225819e0 2024-09-19T08:46:53.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.432+0000 7fb5aab5b740 10 client.25291 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:53.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.432+0000 7fb5aab5b740 1 -- 172.21.15.164:0/4197610022 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55562260ae10 con 0x5556225819e0 2024-09-19T08:46:53.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.432+0000 7fb589ffb640 1 -- 172.21.15.164:0/4197610022 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb59000b480 con 0x5556225819e0 2024-09-19T08:46:53.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.432+0000 7fb589ffb640 1 -- 172.21.15.164:0/4197610022 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb59c032b00 con 0x5556225819e0 2024-09-19T08:46:53.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.432+0000 7fb5aab5b740 10 client.25291 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:53.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.432+0000 7fb5aab5b740 10 client.25291 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-19T08:46:53.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.432+0000 7fb5aab5b740 20 client.25291 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:53.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.432+0000 7fb5aab5b740 1 --2- 172.21.15.164:0/4197610022 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55562260b540 0x55562260d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:53.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.433+0000 7fb5aab5b740 4 client.25291 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:46:53.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.433+0000 7fb5aab5b740 1 -- 172.21.15.164:0/4197610022 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5556224ead70 con 0x55562260b540 2024-09-19T08:46:53.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.433+0000 7fb5a5d1d640 1 --2- 172.21.15.164:0/4197610022 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55562260b540 0x55562260d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:53.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.433+0000 7fb5a5d1d640 1 --2- 172.21.15.164:0/4197610022 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55562260b540 0x55562260d980 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=abe466be109baa6e server_cookie=a89d33beaeee2ab6 in_seq=0 out_seq=0 2024-09-19T08:46:53.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.433+0000 7fb589ffb640 10 client.25291 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:46:53.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.433+0000 7fb589ffb640 1 -- 172.21.15.164:0/4197610022 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5556224ead70 con 0x55562260b540 2024-09-19T08:46:53.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.433+0000 7fb589ffb640 10 client.25291 handle_command_reply: tid=0 2024-09-19T08:46:53.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.433+0000 7fb5aab5b740 1 client.25291 shutdown 2024-09-19T08:46:53.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.433+0000 7fb5aab5b740 1 -- 172.21.15.164:0/4197610022 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5556225819e0 msgr2=0x5556225a94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:53.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.433+0000 7fb5aab5b740 1 --2- 172.21.15.164:0/4197610022 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5556225819e0 0x5556225a94b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb59c010ee0 tx=0x7fb59c004730 comp rx=0 tx=0).stop 2024-09-19T08:46:53.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.434+0000 7fb5aab5b740 1 -- 172.21.15.164:0/4197610022 shutdown_connections 2024-09-19T08:46:53.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.434+0000 7fb5aab5b740 1 --2- 172.21.15.164:0/4197610022 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55562260b540 0x55562260d980 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.434+0000 7fb5aab5b740 1 --2- 172.21.15.164:0/4197610022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55562258c780 0x5556225a3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.434+0000 7fb589ffb640 0 client.25291 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:46:53.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.434+0000 7fb5aab5b740 1 --2- 172.21.15.164:0/4197610022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555622582f50 0x5556225a37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.434+0000 7fb5aab5b740 1 --2- 172.21.15.164:0/4197610022 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5556225819e0 0x5556225a94b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.434+0000 7fb5aab5b740 1 -- 172.21.15.164:0/4197610022 >> 172.21.15.164:0/4197610022 conn(0x555622546310 msgr2=0x555622588590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:53.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.434+0000 7fb5aab5b740 1 -- 172.21.15.164:0/4197610022 shutdown_connections 2024-09-19T08:46:53.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.434+0000 7fb5aab5b740 1 -- 172.21.15.164:0/4197610022 wait complete. 2024-09-19T08:46:53.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.434+0000 7fb5aab5b740 20 client.25291 trim_cache size 0 max 16384 2024-09-19T08:46:53.440 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:46:53.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.435+0000 7fb5a8782640 1 -- 172.21.15.164:0/719096589 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb58c0a3550 msgr2=0x7fb58c0a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:53.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.435+0000 7fb5a8782640 1 --2- 172.21.15.164:0/719096589 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb58c0a3550 0x7fb58c0a5a10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb59c005d00 tx=0x7fb59c00b410 comp rx=0 tx=0).stop 2024-09-19T08:46:53.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.435+0000 7fb5a8782640 1 -- 172.21.15.164:0/719096589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5a0106a60 msgr2=0x7fb5a019ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:53.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.435+0000 7fb5a8782640 1 --2- 172.21.15.164:0/719096589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5a0106a60 0x7fb5a019ccf0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb598005cc0 tx=0x7fb598003970 comp rx=0 tx=0).stop 2024-09-19T08:46:53.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.435+0000 7fb5a8782640 1 -- 172.21.15.164:0/719096589 shutdown_connections 2024-09-19T08:46:53.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.435+0000 7fb5a8782640 1 --2- 172.21.15.164:0/719096589 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb58c0a3550 0x7fb58c0a5a10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.435+0000 7fb5a8782640 1 --2- 172.21.15.164:0/719096589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5a0106a60 0x7fb5a019ccf0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.435+0000 7fb5a8782640 1 --2- 172.21.15.164:0/719096589 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5a0101400 0x7fb5a0198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.435+0000 7fb5a8782640 1 --2- 172.21.15.164:0/719096589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5a0100a60 0x7fb5a0198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.435+0000 7fb5a8782640 1 -- 172.21.15.164:0/719096589 >> 172.21.15.164:0/719096589 conn(0x7fb5a00fc780 msgr2=0x7fb5a006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:53.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.435+0000 7fb5a8782640 1 -- 172.21.15.164:0/719096589 shutdown_connections 2024-09-19T08:46:53.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:53.435+0000 7fb5a8782640 1 -- 172.21.15.164:0/719096589 wait complete. 2024-09-19T08:46:54.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:53 smithi164 ceph-mon[30928]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T08:46:54.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:53 smithi170 ceph-mon[35072]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T08:46:54.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:53 smithi173 ceph-mon[35306]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T08:46:54.293 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T08:46:54.576 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:46:55.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.148+0000 7f45cfa88640 1 -- 172.21.15.164:0/3368837069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45c8100aa0 msgr2=0x7f45c810ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:55.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.148+0000 7f45cfa88640 1 --2- 172.21.15.164:0/3368837069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45c8100aa0 0x7f45c810ea20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f45c0009900 tx=0x7f45c002a410 comp rx=0 tx=0).stop 2024-09-19T08:46:55.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.148+0000 7f45cfa88640 1 -- 172.21.15.164:0/3368837069 shutdown_connections 2024-09-19T08:46:55.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.148+0000 7f45cfa88640 1 --2- 172.21.15.164:0/3368837069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45c8100aa0 0x7f45c810ea20 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:55.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.148+0000 7f45cfa88640 1 --2- 172.21.15.164:0/3368837069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45c8100100 0x7f45c8100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:55.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.148+0000 7f45cfa88640 1 --2- 172.21.15.164:0/3368837069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45c8106b80 0x7f45c8106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:55.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.148+0000 7f45cfa88640 1 -- 172.21.15.164:0/3368837069 >> 172.21.15.164:0/3368837069 conn(0x7f45c8077230 msgr2=0x7f45c80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:55.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.148+0000 7f45cfa88640 1 -- 172.21.15.164:0/3368837069 shutdown_connections 2024-09-19T08:46:55.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.148+0000 7f45cfa88640 1 -- 172.21.15.164:0/3368837069 wait complete. 2024-09-19T08:46:55.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.149+0000 7f45cfa88640 1 Processor -- start 2024-09-19T08:46:55.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.149+0000 7f45cfa88640 1 -- start start 2024-09-19T08:46:55.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.149+0000 7f45cfa88640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45c8100100 0x7f45c8073710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:55.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.149+0000 7f45cfa88640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45c8100aa0 0x7f45c8073c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:55.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.149+0000 7f45cfa88640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45c8106b80 0x7f45c806c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:55.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.150+0000 7f45cfa88640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f45c8112050 con 0x7f45c8100100 2024-09-19T08:46:55.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.150+0000 7f45cfa88640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f45c8111ed0 con 0x7f45c8106b80 2024-09-19T08:46:55.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.150+0000 7f45cfa88640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f45c81121d0 con 0x7f45c8100aa0 2024-09-19T08:46:55.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.150+0000 7f45cd824640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45c8100100 0x7f45c8073710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:55.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.150+0000 7f45ce025640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45c8106b80 0x7f45c806c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:55.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.150+0000 7f45ce025640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45c8106b80 0x7f45c806c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:41932/0 (socket says 172.21.15.164:41932) 2024-09-19T08:46:55.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.150+0000 7f45ce025640 1 -- 172.21.15.164:0/4030939593 learned_addr learned my addr 172.21.15.164:0/4030939593 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:55.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.150+0000 7f45cd023640 1 --2- 172.21.15.164:0/4030939593 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45c8100aa0 0x7f45c8073c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:55.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.150+0000 7f45ce025640 1 -- 172.21.15.164:0/4030939593 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45c8100aa0 msgr2=0x7f45c8073c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:55.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.150+0000 7f45ce025640 1 --2- 172.21.15.164:0/4030939593 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45c8100aa0 0x7f45c8073c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:55.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.150+0000 7f45ce025640 1 -- 172.21.15.164:0/4030939593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45c8100100 msgr2=0x7f45c8073710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:55.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.150+0000 7f45ce025640 1 --2- 172.21.15.164:0/4030939593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45c8100100 0x7f45c8073710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:55.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.150+0000 7f45ce025640 1 -- 172.21.15.164:0/4030939593 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c806cfa0 con 0x7f45c8106b80 2024-09-19T08:46:55.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.151+0000 7f45cd824640 1 --2- 172.21.15.164:0/4030939593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45c8100100 0x7f45c8073710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:46:55.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.151+0000 7f45ce025640 1 --2- 172.21.15.164:0/4030939593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45c8106b80 0x7f45c806c780 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f45c0005940 tx=0x7f45c0030650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:55.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.151+0000 7f45beffd640 1 -- 172.21.15.164:0/4030939593 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c003f070 con 0x7f45c8106b80 2024-09-19T08:46:55.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.151+0000 7f45cfa88640 1 -- 172.21.15.164:0/4030939593 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c806d230 con 0x7f45c8106b80 2024-09-19T08:46:55.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.151+0000 7f45beffd640 1 -- 172.21.15.164:0/4030939593 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45c0004460 con 0x7f45c8106b80 2024-09-19T08:46:55.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.151+0000 7f45cfa88640 1 -- 172.21.15.164:0/4030939593 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c8071240 con 0x7f45c8106b80 2024-09-19T08:46:55.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.151+0000 7f45beffd640 1 -- 172.21.15.164:0/4030939593 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c0035030 con 0x7f45c8106b80 2024-09-19T08:46:55.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.154+0000 7f45beffd640 1 -- 172.21.15.164:0/4030939593 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7f45c0042070 con 0x7f45c8106b80 2024-09-19T08:46:55.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.154+0000 7f45beffd640 1 --2- 172.21.15.164:0/4030939593 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f45b40a3550 0x7f45b40a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:55.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.154+0000 7f45beffd640 1 -- 172.21.15.164:0/4030939593 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f45c00f0b20 con 0x7f45c8106b80 2024-09-19T08:46:55.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.154+0000 7f45cd824640 1 --2- 172.21.15.164:0/4030939593 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f45b40a3550 0x7f45b40a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:55.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.155+0000 7f45d1e61740 1 Processor -- start 2024-09-19T08:46:55.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.155+0000 7f45d1e61740 1 -- start start 2024-09-19T08:46:55.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.155+0000 7f45cd824640 1 --2- 172.21.15.164:0/4030939593 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f45b40a3550 0x7f45b40a5a10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f45c400f5d0 tx=0x7f45c40075b0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:55.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.155+0000 7f45d1e61740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd1dc9b40 0x55cdd1dc8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:55.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.155+0000 7f45d1e61740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd1dc85d0 0x55cdd1dc8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:55.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.155+0000 7f45d1e61740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd1dd3370 0x55cdd1dd5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:55.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.155+0000 7f45d1e61740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cdd1d2e2f0 con 0x55cdd1dd3370 2024-09-19T08:46:55.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.155+0000 7f45d1e61740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55cdd1cd5cf0 con 0x55cdd1dc9b40 2024-09-19T08:46:55.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.155+0000 7f45d1e61740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55cdd1d2a2b0 con 0x55cdd1dc85d0 2024-09-19T08:46:55.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.156+0000 7f45cd023640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd1dc85d0 0x55cdd1dc8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:55.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.156+0000 7f45cd824640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd1dc9b40 0x55cdd1dc8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:55.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.156+0000 7f45cd824640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd1dc9b40 0x55cdd1dc8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:41936/0 (socket says 172.21.15.164:41936) 2024-09-19T08:46:55.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.156+0000 7f45cd023640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd1dc85d0 0x55cdd1dc8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:35138/0 (socket says 172.21.15.164:35138) 2024-09-19T08:46:55.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.156+0000 7f45cd824640 1 -- 172.21.15.164:0/464669748 learned_addr learned my addr 172.21.15.164:0/464669748 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:55.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.156+0000 7f45ce025640 1 --2- 172.21.15.164:0/464669748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd1dd3370 0x55cdd1dd5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:55.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.156+0000 7f45cd824640 1 -- 172.21.15.164:0/464669748 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd1dc85d0 msgr2=0x55cdd1dc8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:55.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.156+0000 7f45cd824640 1 --2- 172.21.15.164:0/464669748 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd1dc85d0 0x55cdd1dc8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:55.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.156+0000 7f45cd824640 1 -- 172.21.15.164:0/464669748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd1dd3370 msgr2=0x55cdd1dd5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:55.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.156+0000 7f45cd824640 1 --2- 172.21.15.164:0/464669748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd1dd3370 0x55cdd1dd5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:55.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.156+0000 7f45cd824640 1 -- 172.21.15.164:0/464669748 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdd1cba3a0 con 0x55cdd1dc9b40 2024-09-19T08:46:55.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.156+0000 7f45ce025640 1 --2- 172.21.15.164:0/464669748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd1dd3370 0x55cdd1dd5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:55.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.157+0000 7f45cd824640 1 --2- 172.21.15.164:0/464669748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd1dc9b40 0x55cdd1dc8090 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f45c4007d10 tx=0x7f45c4012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2e5b5a5025d5983 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:55.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.157+0000 7f45b3fff640 1 -- 172.21.15.164:0/464669748 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c402b070 con 0x55cdd1dc9b40 2024-09-19T08:46:55.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.157+0000 7f45b3fff640 1 -- 172.21.15.164:0/464669748 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45c400dcd0 con 0x55cdd1dc9b40 2024-09-19T08:46:55.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.157+0000 7f45b3fff640 1 -- 172.21.15.164:0/464669748 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c4010430 con 0x55cdd1dc9b40 2024-09-19T08:46:55.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.158+0000 7f45d1e61740 1 -- 172.21.15.164:0/464669748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd1dc9b40 msgr2=0x55cdd1dc8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:55.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.158+0000 7f45d1e61740 1 --2- 172.21.15.164:0/464669748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd1dc9b40 0x55cdd1dc8090 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f45c4007d10 tx=0x7f45c4012000 comp rx=0 tx=0).stop 2024-09-19T08:46:55.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.158+0000 7f45d1e61740 1 -- 172.21.15.164:0/464669748 shutdown_connections 2024-09-19T08:46:55.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.158+0000 7f45d1e61740 1 --2- 172.21.15.164:0/464669748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd1dd3370 0x55cdd1dd5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:55.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.158+0000 7f45d1e61740 1 --2- 172.21.15.164:0/464669748 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd1dc85d0 0x55cdd1dc8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:55.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.158+0000 7f45d1e61740 1 --2- 172.21.15.164:0/464669748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd1dc9b40 0x55cdd1dc8090 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:55.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.158+0000 7f45d1e61740 1 -- 172.21.15.164:0/464669748 >> 172.21.15.164:0/464669748 conn(0x55cdd1d91310 msgr2=0x55cdd1dc4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:55.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.158+0000 7f45d1e61740 1 -- 172.21.15.164:0/464669748 shutdown_connections 2024-09-19T08:46:55.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.158+0000 7f45d1e61740 1 -- 172.21.15.164:0/464669748 wait complete. 2024-09-19T08:46:55.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.158+0000 7f45d1e61740 1 -- start start 2024-09-19T08:46:55.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.158+0000 7f45d1e61740 1 Processor -- start 2024-09-19T08:46:55.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.159+0000 7f45d1e61740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd1dc85d0 0x55cdd1dd2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:55.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.159+0000 7f45d1e61740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd1dc9b40 0x55cdd1dd0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:55.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.159+0000 7f45d1e61740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd1dd3370 0x55cdd1dd0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:55.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.159+0000 7f45d1e61740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cdd1dd6070 con 0x55cdd1dc85d0 2024-09-19T08:46:55.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.159+0000 7f45d1e61740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55cdd1ce9060 con 0x55cdd1dc9b40 2024-09-19T08:46:55.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.159+0000 7f45d1e61740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55cdd1dd61f0 con 0x55cdd1dd3370 2024-09-19T08:46:55.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.159+0000 7f45d1e61740 10 client.0 _ll_register_callbacks cb 0x55cdd1a3c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:55.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.159+0000 7f45ce025640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd1dd3370 0x55cdd1dd0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:55.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.159+0000 7f45ce025640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd1dd3370 0x55cdd1dd0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:35148/0 (socket says 172.21.15.164:35148) 2024-09-19T08:46:55.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.159+0000 7f45ce025640 1 -- 172.21.15.164:0/4023951192 learned_addr learned my addr 172.21.15.164:0/4023951192 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:55.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.159+0000 7f45cd824640 1 --2- 172.21.15.164:0/4023951192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd1dc85d0 0x55cdd1dd2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:55.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.159+0000 7f45cd023640 1 --2- 172.21.15.164:0/4023951192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd1dc9b40 0x55cdd1dd0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:55.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.160+0000 7f45ce025640 1 -- 172.21.15.164:0/4023951192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd1dc9b40 msgr2=0x55cdd1dd0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:55.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.160+0000 7f45ce025640 1 --2- 172.21.15.164:0/4023951192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd1dc9b40 0x55cdd1dd0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:55.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.160+0000 7f45ce025640 1 -- 172.21.15.164:0/4023951192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd1dc85d0 msgr2=0x55cdd1dd2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:55.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.160+0000 7f45ce025640 1 --2- 172.21.15.164:0/4023951192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd1dc85d0 0x55cdd1dd2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:55.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.160+0000 7f45ce025640 1 -- 172.21.15.164:0/4023951192 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdd1a9f3c0 con 0x55cdd1dd3370 2024-09-19T08:46:55.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.160+0000 7f45cd824640 1 --2- 172.21.15.164:0/4023951192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd1dc85d0 0x55cdd1dd2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:46:55.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.160+0000 7f45ce025640 1 --2- 172.21.15.164:0/4023951192 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd1dd3370 0x55cdd1dd0650 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f45c00bc9f0 tx=0x7f45c00bc650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83e81ce5e7e39a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:55.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.160+0000 7f45b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-09-19T08:46:55.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.160+0000 7f45b17fa640 1 -- 172.21.15.164:0/4023951192 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c00c10a0 con 0x55cdd1dd3370 2024-09-19T08:46:55.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.160+0000 7f45d1e61740 1 -- 172.21.15.164:0/4023951192 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45c400dcd0 con 0x55cdd1dd3370 2024-09-19T08:46:55.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.160+0000 7f45b17fa640 1 -- 172.21.15.164:0/4023951192 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f45c00b9360 con 0x55cdd1dd3370 2024-09-19T08:46:55.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.160+0000 7f45b17fa640 1 -- 172.21.15.164:0/4023951192 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f45c00c5960 con 0x55cdd1dd3370 2024-09-19T08:46:55.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.160+0000 7f45d1e61740 10 client.25303 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:55.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.160+0000 7f45d1e61740 1 -- 172.21.15.164:0/4023951192 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cdd1dd6070 con 0x55cdd1dd3370 2024-09-19T08:46:55.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.160+0000 7f45b17fa640 1 -- 172.21.15.164:0/4023951192 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45c400dcd0 con 0x55cdd1dd3370 2024-09-19T08:46:55.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.161+0000 7f45b17fa640 1 -- 172.21.15.164:0/4023951192 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f45c00ee440 con 0x55cdd1dd3370 2024-09-19T08:46:55.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.161+0000 7f45d1e61740 10 client.25303 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:55.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.161+0000 7f45d1e61740 10 client.25303 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T08:46:55.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.161+0000 7f45d1e61740 20 client.25303 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:55.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.161+0000 7f45d1e61740 1 --2- 172.21.15.164:0/4023951192 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55cdd1deb770 0x55cdd1dedb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:55.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.161+0000 7f45d1e61740 4 client.25303 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:46:55.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.161+0000 7f45d1e61740 1 -- 172.21.15.164:0/4023951192 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cdd1d3b860 con 0x55cdd1deb770 2024-09-19T08:46:55.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.161+0000 7f45cd824640 1 --2- 172.21.15.164:0/4023951192 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55cdd1deb770 0x55cdd1dedb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:55.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.161+0000 7f45cd824640 1 --2- 172.21.15.164:0/4023951192 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55cdd1deb770 0x55cdd1dedb80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e8523bc3bc5e1173 server_cookie=b30d9a02b7c0d2b9 in_seq=0 out_seq=0 2024-09-19T08:46:55.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.161+0000 7f45b17fa640 10 client.25303 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:46:55.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.162+0000 7f45b17fa640 1 -- 172.21.15.164:0/4023951192 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cdd1d3b860 con 0x55cdd1deb770 2024-09-19T08:46:55.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.162+0000 7f45b17fa640 10 client.25303 handle_command_reply: tid=0 2024-09-19T08:46:55.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.162+0000 7f45d1e61740 1 client.25303 shutdown 2024-09-19T08:46:55.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.162+0000 7f45d1e61740 1 -- 172.21.15.164:0/4023951192 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd1dd3370 msgr2=0x55cdd1dd0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:55.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.162+0000 7f45d1e61740 1 --2- 172.21.15.164:0/4023951192 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd1dd3370 0x55cdd1dd0650 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f45c00bc9f0 tx=0x7f45c00bc650 comp rx=0 tx=0).stop 2024-09-19T08:46:55.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.162+0000 7f45d1e61740 1 -- 172.21.15.164:0/4023951192 shutdown_connections 2024-09-19T08:46:55.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.162+0000 7f45d1e61740 1 --2- 172.21.15.164:0/4023951192 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55cdd1deb770 0x55cdd1dedb80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:55.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.162+0000 7f45d1e61740 1 --2- 172.21.15.164:0/4023951192 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd1dd3370 0x55cdd1dd0650 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:55.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.162+0000 7f45d1e61740 1 --2- 172.21.15.164:0/4023951192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd1dc9b40 0x55cdd1dd0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:55.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.162+0000 7f45d1e61740 1 --2- 172.21.15.164:0/4023951192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd1dc85d0 0x55cdd1dd2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:55.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.162+0000 7f45d1e61740 1 -- 172.21.15.164:0/4023951192 >> 172.21.15.164:0/4023951192 conn(0x55cdd1d91310 msgr2=0x55cdd1dca960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:55.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.162+0000 7f45b17fa640 0 client.25303 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:46:55.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.163+0000 7f45d1e61740 1 -- 172.21.15.164:0/4023951192 shutdown_connections 2024-09-19T08:46:55.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.163+0000 7f45d1e61740 1 -- 172.21.15.164:0/4023951192 wait complete. 2024-09-19T08:46:55.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.163+0000 7f45d1e61740 20 client.25303 trim_cache size 0 max 16384 2024-09-19T08:46:55.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.175+0000 7f45d1e61740 1 Processor -- start 2024-09-19T08:46:55.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.176+0000 7f45d1e61740 1 -- start start 2024-09-19T08:46:55.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.176+0000 7f45d1e61740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd1dc85d0 0x55cdd1dd1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:55.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.176+0000 7f45d1e61740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd1dc9b40 0x55cdd1dcb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:55.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.176+0000 7f45d1e61740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd1dd3370 0x55cdd1dcb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:55.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.176+0000 7f45d1e61740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f45c81b3c40 con 0x55cdd1dc85d0 2024-09-19T08:46:55.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.176+0000 7f45d1e61740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f45c81b5780 con 0x55cdd1dc9b40 2024-09-19T08:46:55.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.176+0000 7f45d1e61740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f45c81b4310 con 0x55cdd1dd3370 2024-09-19T08:46:55.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.176+0000 7f45cd824640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd1dc85d0 0x55cdd1dd1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:55.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.176+0000 7f45cd824640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd1dc85d0 0x55cdd1dd1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59196/0 (socket says 172.21.15.164:59196) 2024-09-19T08:46:55.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.176+0000 7f45cd824640 1 -- 172.21.15.164:0/1800800030 learned_addr learned my addr 172.21.15.164:0/1800800030 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:55.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.176+0000 7f45cd023640 1 --2- 172.21.15.164:0/1800800030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd1dc9b40 0x55cdd1dcb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:55.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.176+0000 7f45cd023640 1 -- 172.21.15.164:0/1800800030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd1dd3370 msgr2=0x55cdd1dcb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:55.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.176+0000 7f45cd023640 1 --2- 172.21.15.164:0/1800800030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd1dd3370 0x55cdd1dcb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:55.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.176+0000 7f45cd023640 1 -- 172.21.15.164:0/1800800030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd1dc85d0 msgr2=0x55cdd1dd1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:55.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.176+0000 7f45cd023640 1 --2- 172.21.15.164:0/1800800030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd1dc85d0 0x55cdd1dd1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:55.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.176+0000 7f45cd023640 1 -- 172.21.15.164:0/1800800030 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdd1dc44f0 con 0x55cdd1dc9b40 2024-09-19T08:46:55.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.177+0000 7f45cd023640 1 --2- 172.21.15.164:0/1800800030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd1dc9b40 0x55cdd1dcb120 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55cdd1dd1680 tx=0x7f45b8012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9afde3fd39110a4e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:55.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.177+0000 7f45b37fe640 1 -- 172.21.15.164:0/1800800030 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b800bc60 con 0x55cdd1dc9b40 2024-09-19T08:46:55.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.177+0000 7f45b37fe640 1 -- 172.21.15.164:0/1800800030 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45b800be00 con 0x55cdd1dc9b40 2024-09-19T08:46:55.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.177+0000 7f45b37fe640 1 -- 172.21.15.164:0/1800800030 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b800f890 con 0x55cdd1dc9b40 2024-09-19T08:46:55.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.178+0000 7f45d1e61740 1 -- 172.21.15.164:0/1800800030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd1dc9b40 msgr2=0x55cdd1dcb120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:55.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.178+0000 7f45d1e61740 1 --2- 172.21.15.164:0/1800800030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd1dc9b40 0x55cdd1dcb120 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55cdd1dd1680 tx=0x7f45b8012820 comp rx=0 tx=0).stop 2024-09-19T08:46:55.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.178+0000 7f45d1e61740 1 -- 172.21.15.164:0/1800800030 shutdown_connections 2024-09-19T08:46:55.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.178+0000 7f45d1e61740 1 --2- 172.21.15.164:0/1800800030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd1dd3370 0x55cdd1dcb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:55.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.178+0000 7f45d1e61740 1 --2- 172.21.15.164:0/1800800030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd1dc9b40 0x55cdd1dcb120 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:55.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.178+0000 7f45d1e61740 1 --2- 172.21.15.164:0/1800800030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd1dc85d0 0x55cdd1dd1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:55.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.178+0000 7f45d1e61740 1 -- 172.21.15.164:0/1800800030 >> 172.21.15.164:0/1800800030 conn(0x55cdd1d91310 msgr2=0x55cdd1dd18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:55.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.178+0000 7f45d1e61740 1 -- 172.21.15.164:0/1800800030 shutdown_connections 2024-09-19T08:46:55.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.178+0000 7f45d1e61740 1 -- 172.21.15.164:0/1800800030 wait complete. 2024-09-19T08:46:55.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.178+0000 7f45d1e61740 1 -- start start 2024-09-19T08:46:55.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.178+0000 7f45d1e61740 1 Processor -- start 2024-09-19T08:46:55.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.179+0000 7f45d1e61740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd1dc85d0 0x55cdd1ddfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:55.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.179+0000 7f45d1e61740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd1dc9b40 0x55cdd1de01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:55.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.179+0000 7f45d1e61740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd1dd3370 0x55cdd1de0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:55.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.179+0000 7f45d1e61740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f45c81b50b0 con 0x55cdd1dc85d0 2024-09-19T08:46:55.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.179+0000 7f45d1e61740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f45c81b49e0 con 0x55cdd1dd3370 2024-09-19T08:46:55.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.179+0000 7f45d1e61740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f45c81b1c20 con 0x55cdd1dc9b40 2024-09-19T08:46:55.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.179+0000 7f45d1e61740 10 client.0 _ll_register_callbacks cb 0x7f45c0003850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:55.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.179+0000 7f45cd824640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd1dc85d0 0x55cdd1ddfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:55.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.179+0000 7f45cd824640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd1dc85d0 0x55cdd1ddfc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59210/0 (socket says 172.21.15.164:59210) 2024-09-19T08:46:55.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.179+0000 7f45cd824640 1 -- 172.21.15.164:0/689175357 learned_addr learned my addr 172.21.15.164:0/689175357 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:55.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.179+0000 7f45cd023640 1 --2- 172.21.15.164:0/689175357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd1dc9b40 0x55cdd1de01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:55.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.179+0000 7f45cd824640 1 -- 172.21.15.164:0/689175357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd1dc9b40 msgr2=0x55cdd1de01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:55.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.179+0000 7f45cd824640 1 --2- 172.21.15.164:0/689175357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd1dc9b40 0x55cdd1de01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:55.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.179+0000 7f45cd824640 1 -- 172.21.15.164:0/689175357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd1dd3370 msgr2=0x55cdd1de0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:55.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.179+0000 7f45cd824640 1 --2- 172.21.15.164:0/689175357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd1dd3370 0x55cdd1de0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:55.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.179+0000 7f45cd824640 1 -- 172.21.15.164:0/689175357 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdd1e55660 con 0x55cdd1dc85d0 2024-09-19T08:46:55.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.179+0000 7f45cd824640 1 --2- 172.21.15.164:0/689175357 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd1dc85d0 0x55cdd1ddfc80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f45c4021010 tx=0x7f45c401baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83edad28c28563d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:55.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.180+0000 7f45b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:46:55.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.180+0000 7f45b17fa640 1 -- 172.21.15.164:0/689175357 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c402b040 con 0x55cdd1dc85d0 2024-09-19T08:46:55.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.180+0000 7f45d1e61740 1 -- 172.21.15.164:0/689175357 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45b800be00 con 0x55cdd1dc85d0 2024-09-19T08:46:55.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.180+0000 7f45b17fa640 1 -- 172.21.15.164:0/689175357 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f45c401bcc0 con 0x55cdd1dc85d0 2024-09-19T08:46:55.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.180+0000 7f45b17fa640 1 -- 172.21.15.164:0/689175357 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f45c401d3e0 con 0x55cdd1dc85d0 2024-09-19T08:46:55.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.180+0000 7f45d1e61740 10 client.15771 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:55.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.180+0000 7f45d1e61740 1 -- 172.21.15.164:0/689175357 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cdd1e56900 con 0x55cdd1dc85d0 2024-09-19T08:46:55.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.180+0000 7f45b17fa640 1 -- 172.21.15.164:0/689175357 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45b800be00 con 0x55cdd1dc85d0 2024-09-19T08:46:55.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.180+0000 7f45b17fa640 1 -- 172.21.15.164:0/689175357 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f45c4032620 con 0x55cdd1dc85d0 2024-09-19T08:46:55.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.180+0000 7f45d1e61740 10 client.15771 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:55.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.180+0000 7f45d1e61740 10 client.15771 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T08:46:55.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.181+0000 7f45d1e61740 20 client.15771 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:55.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.181+0000 7f45d1e61740 1 --2- 172.21.15.164:0/689175357 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55cdd1e56f10 0x55cdd1e59370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:55.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.181+0000 7f45d1e61740 4 client.15771 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:46:55.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.181+0000 7f45d1e61740 1 -- 172.21.15.164:0/689175357 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cdd1d35d70 con 0x55cdd1e56f10 2024-09-19T08:46:55.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.181+0000 7f45cd023640 1 --2- 172.21.15.164:0/689175357 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55cdd1e56f10 0x55cdd1e59370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:55.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.181+0000 7f45cd023640 1 --2- 172.21.15.164:0/689175357 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55cdd1e56f10 0x55cdd1e59370 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c83ac41bf35a61f8 server_cookie=ec5066a9f3ca6fad in_seq=0 out_seq=0 2024-09-19T08:46:55.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.181+0000 7f45b17fa640 10 client.15771 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:46:55.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.182+0000 7f45b17fa640 1 -- 172.21.15.164:0/689175357 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cdd1d35d70 con 0x55cdd1e56f10 2024-09-19T08:46:55.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.182+0000 7f45b17fa640 10 client.15771 handle_command_reply: tid=0 2024-09-19T08:46:55.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.182+0000 7f45d1e61740 1 client.15771 shutdown 2024-09-19T08:46:55.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.182+0000 7f45d1e61740 1 -- 172.21.15.164:0/689175357 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd1dc85d0 msgr2=0x55cdd1ddfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:55.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.182+0000 7f45d1e61740 1 --2- 172.21.15.164:0/689175357 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd1dc85d0 0x55cdd1ddfc80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f45c4021010 tx=0x7f45c401baf0 comp rx=0 tx=0).stop 2024-09-19T08:46:55.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.182+0000 7f45d1e61740 1 -- 172.21.15.164:0/689175357 shutdown_connections 2024-09-19T08:46:55.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.182+0000 7f45d1e61740 1 --2- 172.21.15.164:0/689175357 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55cdd1e56f10 0x55cdd1e59370 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:55.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.182+0000 7f45d1e61740 1 --2- 172.21.15.164:0/689175357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd1dd3370 0x55cdd1de0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:55.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.182+0000 7f45d1e61740 1 --2- 172.21.15.164:0/689175357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd1dc9b40 0x55cdd1de01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:55.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.182+0000 7f45d1e61740 1 --2- 172.21.15.164:0/689175357 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd1dc85d0 0x55cdd1ddfc80 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:55.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.182+0000 7f45d1e61740 1 -- 172.21.15.164:0/689175357 >> 172.21.15.164:0/689175357 conn(0x55cdd1d91310 msgr2=0x55cdd1dd20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:55.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.182+0000 7f45b17fa640 0 client.15771 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:46:55.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.182+0000 7f45d1e61740 1 -- 172.21.15.164:0/689175357 shutdown_connections 2024-09-19T08:46:55.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.182+0000 7f45d1e61740 1 -- 172.21.15.164:0/689175357 wait complete. 2024-09-19T08:46:55.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.182+0000 7f45d1e61740 20 client.15771 trim_cache size 0 max 16384 2024-09-19T08:46:55.198 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:46:55.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.183+0000 7f45cfa88640 1 -- 172.21.15.164:0/4030939593 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f45b40a3550 msgr2=0x7f45b40a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:55.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.183+0000 7f45cfa88640 1 --2- 172.21.15.164:0/4030939593 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f45b40a3550 0x7f45b40a5a10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f45c400f5d0 tx=0x7f45c40075b0 comp rx=0 tx=0).stop 2024-09-19T08:46:55.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.183+0000 7f45cfa88640 1 -- 172.21.15.164:0/4030939593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45c8106b80 msgr2=0x7f45c806c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:55.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.183+0000 7f45cfa88640 1 --2- 172.21.15.164:0/4030939593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45c8106b80 0x7f45c806c780 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f45c0005940 tx=0x7f45c0030650 comp rx=0 tx=0).stop 2024-09-19T08:46:55.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.183+0000 7f45cfa88640 1 -- 172.21.15.164:0/4030939593 shutdown_connections 2024-09-19T08:46:55.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.183+0000 7f45cfa88640 1 --2- 172.21.15.164:0/4030939593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45c8106b80 0x7f45c806c780 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:55.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.183+0000 7f45cfa88640 1 --2- 172.21.15.164:0/4030939593 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45c8100aa0 0x7f45c8073c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:55.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.183+0000 7f45cfa88640 1 --2- 172.21.15.164:0/4030939593 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f45b40a3550 0x7f45b40a5a10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:55.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.183+0000 7f45cfa88640 1 --2- 172.21.15.164:0/4030939593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45c8100100 0x7f45c8073710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:55.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.183+0000 7f45cfa88640 1 -- 172.21.15.164:0/4030939593 >> 172.21.15.164:0/4030939593 conn(0x7f45c8077230 msgr2=0x7f45c8104820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:55.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.183+0000 7f45cfa88640 1 -- 172.21.15.164:0/4030939593 shutdown_connections 2024-09-19T08:46:55.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:55.183+0000 7f45cfa88640 1 -- 172.21.15.164:0/4030939593 wait complete. 2024-09-19T08:46:56.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:55 smithi164 ceph-mon[30928]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-19T08:46:56.091 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T08:46:56.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:55 smithi170 ceph-mon[35072]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-19T08:46:56.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:55 smithi173 ceph-mon[35306]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-19T08:46:56.361 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:46:56.756 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:56 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:46:56.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.946+0000 7ffbc5d5c640 1 -- 172.21.15.164:0/1886692547 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbc0101470 msgr2=0x7ffbc0101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:56.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.946+0000 7ffbc5d5c640 1 --2- 172.21.15.164:0/1886692547 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbc0101470 0x7ffbc0101850 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0009900 tx=0x7ffbb002a3f0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.946+0000 7ffbc5d5c640 1 -- 172.21.15.164:0/1886692547 shutdown_connections 2024-09-19T08:46:56.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.946+0000 7ffbc5d5c640 1 --2- 172.21.15.164:0/1886692547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbc0105de0 0x7ffbc010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.946+0000 7ffbc5d5c640 1 --2- 172.21.15.164:0/1886692547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbc0101e20 0x7ffbc01056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.946+0000 7ffbc5d5c640 1 --2- 172.21.15.164:0/1886692547 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbc0101470 0x7ffbc0101850 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.946+0000 7ffbc5d5c640 1 -- 172.21.15.164:0/1886692547 >> 172.21.15.164:0/1886692547 conn(0x7ffbc0077230 msgr2=0x7ffbc00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:56.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.946+0000 7ffbc5d5c640 1 -- 172.21.15.164:0/1886692547 shutdown_connections 2024-09-19T08:46:56.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.946+0000 7ffbc5d5c640 1 -- 172.21.15.164:0/1886692547 wait complete. 2024-09-19T08:46:56.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.947+0000 7ffbc5d5c640 1 Processor -- start 2024-09-19T08:46:56.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.947+0000 7ffbc5d5c640 1 -- start start 2024-09-19T08:46:56.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.948+0000 7ffbc5d5c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbc0101470 0x7ffbc019f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:56.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.948+0000 7ffbc5d5c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbc0101e20 0x7ffbc019f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:56.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.948+0000 7ffbc5d5c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbc0105de0 0x7ffbc010cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:56.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.948+0000 7ffbc5d5c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc0111f60 con 0x7ffbc0101e20 2024-09-19T08:46:56.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.948+0000 7ffbc5d5c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc0111de0 con 0x7ffbc0101470 2024-09-19T08:46:56.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.948+0000 7ffbc5d5c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc01120e0 con 0x7ffbc0105de0 2024-09-19T08:46:56.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.948+0000 7ffbbeffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbc0101e20 0x7ffbc019f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:56.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.948+0000 7ffbbeffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbc0101e20 0x7ffbc019f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59236/0 (socket says 172.21.15.164:59236) 2024-09-19T08:46:56.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.948+0000 7ffbbeffd640 1 -- 172.21.15.164:0/2797848316 learned_addr learned my addr 172.21.15.164:0/2797848316 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:56.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.948+0000 7ffbbffff640 1 --2- 172.21.15.164:0/2797848316 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbc0105de0 0x7ffbc010cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:56.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.948+0000 7ffbbf7fe640 1 --2- 172.21.15.164:0/2797848316 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbc0101470 0x7ffbc019f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:56.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.949+0000 7ffbbffff640 1 -- 172.21.15.164:0/2797848316 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbc0101470 msgr2=0x7ffbc019f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:56.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.949+0000 7ffbbffff640 1 --2- 172.21.15.164:0/2797848316 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbc0101470 0x7ffbc019f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.949+0000 7ffbbffff640 1 -- 172.21.15.164:0/2797848316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbc0101e20 msgr2=0x7ffbc019f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:56.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.949+0000 7ffbbffff640 1 --2- 172.21.15.164:0/2797848316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbc0101e20 0x7ffbc019f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.949+0000 7ffbbffff640 1 -- 172.21.15.164:0/2797848316 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc010d420 con 0x7ffbc0105de0 2024-09-19T08:46:56.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.949+0000 7ffbbeffd640 1 --2- 172.21.15.164:0/2797848316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbc0101e20 0x7ffbc019f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:46:56.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.949+0000 7ffbbffff640 1 --2- 172.21.15.164:0/2797848316 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbc0105de0 0x7ffbc010cb60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffbb800ec50 tx=0x7ffbb800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:56.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.949+0000 7ffbbcff9640 1 -- 172.21.15.164:0/2797848316 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb8003b70 con 0x7ffbc0105de0 2024-09-19T08:46:56.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.949+0000 7ffbc5d5c640 1 -- 172.21.15.164:0/2797848316 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbc010d710 con 0x7ffbc0105de0 2024-09-19T08:46:56.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.949+0000 7ffbc5d5c640 1 -- 172.21.15.164:0/2797848316 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbc019ad00 con 0x7ffbc0105de0 2024-09-19T08:46:56.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.950+0000 7ffbbcff9640 1 -- 172.21.15.164:0/2797848316 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbb8003d10 con 0x7ffbc0105de0 2024-09-19T08:46:56.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.950+0000 7ffbbcff9640 1 -- 172.21.15.164:0/2797848316 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb8010890 con 0x7ffbc0105de0 2024-09-19T08:46:56.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.953+0000 7ffbbcff9640 1 -- 172.21.15.164:0/2797848316 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7ffbb8010a30 con 0x7ffbc0105de0 2024-09-19T08:46:56.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.954+0000 7ffbc8135740 1 Processor -- start 2024-09-19T08:46:56.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.954+0000 7ffbc8135740 1 -- start start 2024-09-19T08:46:56.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.955+0000 7ffbc8135740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56378d61bb40 0x56378d61a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:56.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.955+0000 7ffbc8135740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56378d61a5d0 0x56378d61aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:56.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.955+0000 7ffbbcff9640 1 --2- 172.21.15.164:0/2797848316 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ffba80a35d0 0x7ffba80a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:56.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.955+0000 7ffbc8135740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56378d625370 0x56378d627760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:56.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.955+0000 7ffbbcff9640 1 -- 172.21.15.164:0/2797848316 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7ffbb80d1570 con 0x7ffbc0105de0 2024-09-19T08:46:56.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.955+0000 7ffbc8135740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56378d5802f0 con 0x56378d625370 2024-09-19T08:46:56.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.955+0000 7ffbc8135740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56378d527cf0 con 0x56378d61bb40 2024-09-19T08:46:56.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.955+0000 7ffbc8135740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56378d57c2b0 con 0x56378d61a5d0 2024-09-19T08:46:56.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.955+0000 7ffbbeffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56378d61bb40 0x56378d61a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:56.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.955+0000 7ffbbeffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56378d61bb40 0x56378d61a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:42010/0 (socket says 172.21.15.164:42010) 2024-09-19T08:46:56.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.955+0000 7ffbbeffd640 1 -- 172.21.15.164:0/768296644 learned_addr learned my addr 172.21.15.164:0/768296644 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:56.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.955+0000 7ffbbffff640 1 --2- 172.21.15.164:0/2797848316 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ffba80a35d0 0x7ffba80a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:56.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.955+0000 7ffbbeffd640 1 --2- 172.21.15.164:0/768296644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56378d625370 0x56378d627760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:56.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.955+0000 7ffbbf7fe640 1 --2- 172.21.15.164:0/768296644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56378d61a5d0 0x56378d61aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:56.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.955+0000 7ffbbeffd640 1 -- 172.21.15.164:0/768296644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56378d61a5d0 msgr2=0x56378d61aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:56.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.955+0000 7ffbbeffd640 1 --2- 172.21.15.164:0/768296644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56378d61a5d0 0x56378d61aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.955+0000 7ffbbeffd640 1 -- 172.21.15.164:0/768296644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56378d625370 msgr2=0x56378d627760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:56.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.955+0000 7ffbbeffd640 1 --2- 172.21.15.164:0/768296644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56378d625370 0x56378d627760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.955+0000 7ffbbeffd640 1 -- 172.21.15.164:0/768296644 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56378d50c3a0 con 0x56378d61bb40 2024-09-19T08:46:56.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.956+0000 7ffbbeffd640 1 --2- 172.21.15.164:0/768296644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56378d61bb40 0x56378d61a090 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400bfd0 tx=0x7ffbb400b700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ded7c2bfd30a0092 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:56.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.956+0000 7ffbbffff640 1 --2- 172.21.15.164:0/2797848316 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ffba80a35d0 0x7ffba80a5a90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffbb80cebb0 tx=0x7ffbb8009b50 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:56.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.957+0000 7ffba5ffb640 1 -- 172.21.15.164:0/768296644 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4005020 con 0x56378d61bb40 2024-09-19T08:46:56.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.957+0000 7ffba5ffb640 1 -- 172.21.15.164:0/768296644 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbb40056a0 con 0x56378d61bb40 2024-09-19T08:46:56.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.957+0000 7ffba5ffb640 1 -- 172.21.15.164:0/768296644 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4004a00 con 0x56378d61bb40 2024-09-19T08:46:56.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.958+0000 7ffbc8135740 1 -- 172.21.15.164:0/768296644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56378d61bb40 msgr2=0x56378d61a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:56.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.958+0000 7ffbc8135740 1 --2- 172.21.15.164:0/768296644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56378d61bb40 0x56378d61a090 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400bfd0 tx=0x7ffbb400b700 comp rx=0 tx=0).stop 2024-09-19T08:46:56.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.958+0000 7ffbc8135740 1 -- 172.21.15.164:0/768296644 shutdown_connections 2024-09-19T08:46:56.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.958+0000 7ffbc8135740 1 --2- 172.21.15.164:0/768296644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56378d625370 0x56378d627760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.958+0000 7ffbc8135740 1 --2- 172.21.15.164:0/768296644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56378d61a5d0 0x56378d61aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.958+0000 7ffbc8135740 1 --2- 172.21.15.164:0/768296644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56378d61bb40 0x56378d61a090 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400bfd0 tx=0x7ffbb400b700 comp rx=0 tx=0).stop 2024-09-19T08:46:56.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.958+0000 7ffbc8135740 1 -- 172.21.15.164:0/768296644 >> 172.21.15.164:0/768296644 conn(0x56378d5e3310 msgr2=0x56378d616c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:56.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.958+0000 7ffbc8135740 1 -- 172.21.15.164:0/768296644 shutdown_connections 2024-09-19T08:46:56.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.958+0000 7ffbc8135740 1 -- 172.21.15.164:0/768296644 wait complete. 2024-09-19T08:46:56.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.959+0000 7ffbc8135740 1 -- start start 2024-09-19T08:46:56.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.959+0000 7ffbc8135740 1 Processor -- start 2024-09-19T08:46:56.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.960+0000 7ffbc8135740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56378d61a5d0 0x56378d6221d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:56.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.960+0000 7ffbc8135740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56378d625370 0x56378d622710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:56.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.960+0000 7ffbc8135740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56378d622c50 0x56378d6315b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:56.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.960+0000 7ffbc8135740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56378d628070 con 0x56378d625370 2024-09-19T08:46:56.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.960+0000 7ffbbeffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56378d61a5d0 0x56378d6221d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:56.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.960+0000 7ffbbf7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56378d625370 0x56378d622710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:56.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.960+0000 7ffbc8135740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56378d53b060 con 0x56378d61a5d0 2024-09-19T08:46:56.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.960+0000 7ffbc8135740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56378d6281f0 con 0x56378d622c50 2024-09-19T08:46:56.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.960+0000 7ffbbf7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56378d625370 0x56378d622710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59260/0 (socket says 172.21.15.164:59260) 2024-09-19T08:46:56.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.960+0000 7ffbbf7fe640 1 -- 172.21.15.164:0/1012294556 learned_addr learned my addr 172.21.15.164:0/1012294556 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:56.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.960+0000 7ffbc8135740 10 client.0 _ll_register_callbacks cb 0x56378d28e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:56.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.960+0000 7ffbbeffd640 1 --2- 172.21.15.164:0/1012294556 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56378d622c50 0x56378d6315b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:56.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.960+0000 7ffbbf7fe640 1 -- 172.21.15.164:0/1012294556 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56378d622c50 msgr2=0x56378d6315b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:56.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.960+0000 7ffbbf7fe640 1 --2- 172.21.15.164:0/1012294556 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56378d622c50 0x56378d6315b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.960+0000 7ffbbf7fe640 1 -- 172.21.15.164:0/1012294556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56378d61a5d0 msgr2=0x56378d6221d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:56.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.960+0000 7ffbbf7fe640 1 --2- 172.21.15.164:0/1012294556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56378d61a5d0 0x56378d6221d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.960+0000 7ffbbf7fe640 1 -- 172.21.15.164:0/1012294556 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56378d2f13c0 con 0x56378d625370 2024-09-19T08:46:56.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.961+0000 7ffbbeffd640 1 --2- 172.21.15.164:0/1012294556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56378d61a5d0 0x56378d6221d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:56.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.961+0000 7ffbbf7fe640 1 --2- 172.21.15.164:0/1012294556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56378d625370 0x56378d622710 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0005ab0 tx=0x7ffbb0031860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bfb2f8000274fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:56.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.961+0000 7ffb9effd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:46:56.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.961+0000 7ffb9effd640 1 -- 172.21.15.164:0/1012294556 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0035070 con 0x56378d625370 2024-09-19T08:46:56.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.961+0000 7ffb9effd640 1 -- 172.21.15.164:0/1012294556 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7ffbb0004240 con 0x56378d625370 2024-09-19T08:46:56.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.961+0000 7ffbc8135740 1 -- 172.21.15.164:0/1012294556 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbb40056a0 con 0x56378d625370 2024-09-19T08:46:56.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.962+0000 7ffb9effd640 1 -- 172.21.15.164:0/1012294556 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffbb0005d50 con 0x56378d625370 2024-09-19T08:46:56.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.962+0000 7ffbc8135740 10 client.15783 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:56.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.962+0000 7ffbc8135740 1 -- 172.21.15.164:0/1012294556 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56378d634140 con 0x56378d625370 2024-09-19T08:46:56.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.962+0000 7ffb9effd640 1 -- 172.21.15.164:0/1012294556 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbb40056a0 con 0x56378d625370 2024-09-19T08:46:56.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.962+0000 7ffb9effd640 1 -- 172.21.15.164:0/1012294556 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ffbb00049f0 con 0x56378d625370 2024-09-19T08:46:56.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.962+0000 7ffbc8135740 10 client.15783 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:56.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.962+0000 7ffbc8135740 10 client.15783 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T08:46:56.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.962+0000 7ffbc8135740 20 client.15783 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:56.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.962+0000 7ffbc8135740 1 --2- 172.21.15.164:0/1012294556 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x56378d635600 0x56378d637a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:56.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.962+0000 7ffbc8135740 4 client.15783 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:46:56.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.962+0000 7ffbc8135740 1 -- 172.21.15.164:0/1012294556 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56378d58d860 con 0x56378d635600 2024-09-19T08:46:56.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.963+0000 7ffbbeffd640 1 --2- 172.21.15.164:0/1012294556 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x56378d635600 0x56378d637a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:56.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.963+0000 7ffbbeffd640 1 --2- 172.21.15.164:0/1012294556 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x56378d635600 0x56378d637a80 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=def50c6f58361beb server_cookie=f7657eb2732fc700 in_seq=0 out_seq=0 2024-09-19T08:46:56.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.963+0000 7ffb9effd640 10 client.15783 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:46:56.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.964+0000 7ffb9effd640 1 -- 172.21.15.164:0/1012294556 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56378d58d860 con 0x56378d635600 2024-09-19T08:46:56.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.964+0000 7ffb9effd640 10 client.15783 handle_command_reply: tid=0 2024-09-19T08:46:56.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.964+0000 7ffbc8135740 1 client.15783 shutdown 2024-09-19T08:46:56.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.964+0000 7ffbc8135740 1 -- 172.21.15.164:0/1012294556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56378d625370 msgr2=0x56378d622710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:56.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.964+0000 7ffbc8135740 1 --2- 172.21.15.164:0/1012294556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56378d625370 0x56378d622710 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0005ab0 tx=0x7ffbb0031860 comp rx=0 tx=0).stop 2024-09-19T08:46:56.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.964+0000 7ffbc8135740 1 -- 172.21.15.164:0/1012294556 shutdown_connections 2024-09-19T08:46:56.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.964+0000 7ffbc8135740 1 --2- 172.21.15.164:0/1012294556 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x56378d635600 0x56378d637a80 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.964+0000 7ffbc8135740 1 --2- 172.21.15.164:0/1012294556 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56378d622c50 0x56378d6315b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.964+0000 7ffbc8135740 1 --2- 172.21.15.164:0/1012294556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56378d625370 0x56378d622710 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.964+0000 7ffbc8135740 1 --2- 172.21.15.164:0/1012294556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56378d61a5d0 0x56378d6221d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.964+0000 7ffb9effd640 0 client.15783 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:46:56.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.964+0000 7ffbc8135740 1 -- 172.21.15.164:0/1012294556 >> 172.21.15.164:0/1012294556 conn(0x56378d5e3310 msgr2=0x56378d61c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:56.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.965+0000 7ffbc8135740 1 -- 172.21.15.164:0/1012294556 shutdown_connections 2024-09-19T08:46:56.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.965+0000 7ffbc8135740 1 -- 172.21.15.164:0/1012294556 wait complete. 2024-09-19T08:46:56.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.965+0000 7ffbc8135740 20 client.15783 trim_cache size 0 max 16384 2024-09-19T08:46:56.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.990+0000 7ffbc8135740 1 Processor -- start 2024-09-19T08:46:56.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.990+0000 7ffbc8135740 1 -- start start 2024-09-19T08:46:56.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.990+0000 7ffbc8135740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56378d61a5d0 0x56378d616e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:56.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.990+0000 7ffbbeffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56378d61a5d0 0x56378d616e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:56.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.991+0000 7ffbbeffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56378d61a5d0 0x56378d616e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59276/0 (socket says 172.21.15.164:59276) 2024-09-19T08:46:56.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.991+0000 7ffbbeffd640 1 -- 172.21.15.164:0/2882374483 learned_addr learned my addr 172.21.15.164:0/2882374483 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:56.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.991+0000 7ffbc8135740 1 --2- 172.21.15.164:0/2882374483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56378d622c50 0x56378d621a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:56.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.991+0000 7ffbc8135740 1 --2- 172.21.15.164:0/2882374483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56378d625370 0x56378d6313a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:56.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.991+0000 7ffbc8135740 1 -- 172.21.15.164:0/2882374483 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac00e5d0 con 0x56378d61a5d0 2024-09-19T08:46:56.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.991+0000 7ffbc8135740 1 -- 172.21.15.164:0/2882374483 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac010110 con 0x56378d622c50 2024-09-19T08:46:56.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.991+0000 7ffbc8135740 1 -- 172.21.15.164:0/2882374483 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac00eca0 con 0x56378d625370 2024-09-19T08:46:56.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.991+0000 7ffbbf7fe640 1 --2- 172.21.15.164:0/2882374483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56378d622c50 0x56378d621a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:56.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.991+0000 7ffbbeffd640 1 -- 172.21.15.164:0/2882374483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56378d625370 msgr2=0x56378d6313a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:46:56.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.991+0000 7ffbbeffd640 1 --2- 172.21.15.164:0/2882374483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56378d625370 0x56378d6313a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.991+0000 7ffbbeffd640 1 -- 172.21.15.164:0/2882374483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56378d622c50 msgr2=0x56378d621a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:56.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.991+0000 7ffbbeffd640 1 --2- 172.21.15.164:0/2882374483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56378d622c50 0x56378d621a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.991+0000 7ffbbeffd640 1 -- 172.21.15.164:0/2882374483 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56378d61d9f0 con 0x56378d61a5d0 2024-09-19T08:46:56.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.992+0000 7ffbbeffd640 1 --2- 172.21.15.164:0/2882374483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56378d61a5d0 0x56378d616e10 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4005fa0 tx=0x7ffbb4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73248c42fc5892b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:56.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.992+0000 7ffba57fa640 1 -- 172.21.15.164:0/2882374483 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4027070 con 0x56378d61a5d0 2024-09-19T08:46:56.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.992+0000 7ffba57fa640 1 -- 172.21.15.164:0/2882374483 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbb4005550 con 0x56378d61a5d0 2024-09-19T08:46:56.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.992+0000 7ffba57fa640 1 -- 172.21.15.164:0/2882374483 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb402ce80 con 0x56378d61a5d0 2024-09-19T08:46:56.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.993+0000 7ffbc8135740 1 -- 172.21.15.164:0/2882374483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56378d61a5d0 msgr2=0x56378d616e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:56.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.993+0000 7ffbc8135740 1 --2- 172.21.15.164:0/2882374483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56378d61a5d0 0x56378d616e10 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4005fa0 tx=0x7ffbb4005fd0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.993+0000 7ffbc8135740 1 -- 172.21.15.164:0/2882374483 shutdown_connections 2024-09-19T08:46:56.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.993+0000 7ffbc8135740 1 --2- 172.21.15.164:0/2882374483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56378d625370 0x56378d6313a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.993+0000 7ffbc8135740 1 --2- 172.21.15.164:0/2882374483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56378d622c50 0x56378d621a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.993+0000 7ffbc8135740 1 --2- 172.21.15.164:0/2882374483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56378d61a5d0 0x56378d616e10 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.993+0000 7ffbc8135740 1 -- 172.21.15.164:0/2882374483 >> 172.21.15.164:0/2882374483 conn(0x56378d5e3310 msgr2=0x56378d616700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:56.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.994+0000 7ffbc8135740 1 -- 172.21.15.164:0/2882374483 shutdown_connections 2024-09-19T08:46:56.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.994+0000 7ffbc8135740 1 -- 172.21.15.164:0/2882374483 wait complete. 2024-09-19T08:46:56.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.994+0000 7ffbc8135740 1 -- start start 2024-09-19T08:46:56.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.994+0000 7ffbc8135740 1 Processor -- start 2024-09-19T08:46:56.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.995+0000 7ffbc8135740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56378d61a5d0 0x56378d62d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:56.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.995+0000 7ffbbeffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56378d61a5d0 0x56378d62d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:56.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.995+0000 7ffbbeffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56378d61a5d0 0x56378d62d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59290/0 (socket says 172.21.15.164:59290) 2024-09-19T08:46:56.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.995+0000 7ffbbeffd640 1 -- 172.21.15.164:0/720443855 learned_addr learned my addr 172.21.15.164:0/720443855 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:56.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.995+0000 7ffbc8135740 1 --2- 172.21.15.164:0/720443855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56378d622c50 0x56378d62ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:56.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.995+0000 7ffbc8135740 1 --2- 172.21.15.164:0/720443855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56378d625370 0x56378d628d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:56.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.995+0000 7ffbc8135740 1 -- 172.21.15.164:0/720443855 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac00fa40 con 0x56378d61a5d0 2024-09-19T08:46:56.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.995+0000 7ffbc8135740 1 -- 172.21.15.164:0/720443855 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac00f370 con 0x56378d625370 2024-09-19T08:46:56.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.995+0000 7ffbc8135740 1 -- 172.21.15.164:0/720443855 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac00c5b0 con 0x56378d622c50 2024-09-19T08:46:56.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.996+0000 7ffbbf7fe640 1 --2- 172.21.15.164:0/720443855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56378d622c50 0x56378d62ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:56.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.996+0000 7ffbbeffd640 1 -- 172.21.15.164:0/720443855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56378d622c50 msgr2=0x56378d62ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:56.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.996+0000 7ffbbeffd640 1 --2- 172.21.15.164:0/720443855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56378d622c50 0x56378d62ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.996+0000 7ffbbeffd640 1 -- 172.21.15.164:0/720443855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56378d625370 msgr2=0x56378d628d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:46:56.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.996+0000 7ffbbeffd640 1 --2- 172.21.15.164:0/720443855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56378d625370 0x56378d628d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.996+0000 7ffbbeffd640 1 -- 172.21.15.164:0/720443855 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56378d629540 con 0x56378d61a5d0 2024-09-19T08:46:56.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.996+0000 7ffbc8135740 10 client.0 _ll_register_callbacks cb 0x7ffbb80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:56.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.996+0000 7ffbc8135740 1 -- 172.21.15.164:0/720443855 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbb4005550 con 0x56378d61a5d0 2024-09-19T08:46:56.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.996+0000 7ffbbeffd640 1 --2- 172.21.15.164:0/720443855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56378d61a5d0 0x56378d62d870 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4005930 tx=0x7ffbb4005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90d7ee7b5d768251 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:56.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.996+0000 7ffb9effd640 10 client.15789 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:46:56.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.997+0000 7ffb9effd640 1 -- 172.21.15.164:0/720443855 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4027070 con 0x56378d61a5d0 2024-09-19T08:46:56.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.997+0000 7ffb9effd640 1 -- 172.21.15.164:0/720443855 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7ffbb4005310 con 0x56378d61a5d0 2024-09-19T08:46:56.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.997+0000 7ffb9effd640 1 -- 172.21.15.164:0/720443855 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffbb40035f0 con 0x56378d61a5d0 2024-09-19T08:46:56.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.997+0000 7ffbc8135740 10 client.15789 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:56.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.997+0000 7ffbc8135740 1 -- 172.21.15.164:0/720443855 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({fsmap=28}) -- 0x56378d69fd70 con 0x56378d61a5d0 2024-09-19T08:46:56.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.997+0000 7ffb9effd640 1 -- 172.21.15.164:0/720443855 <== mon.0 v2:172.21.15.164:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ffbb4034070 con 0x56378d61a5d0 2024-09-19T08:46:56.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.998+0000 7ffbc8135740 10 client.15789 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:57.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.998+0000 7ffbc8135740 10 client.15789 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T08:46:57.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.998+0000 7ffbc8135740 20 client.15789 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:57.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.998+0000 7ffbc8135740 1 --2- 172.21.15.164:0/720443855 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x56378d635600 0x56378d6a20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:57.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.998+0000 7ffbc8135740 4 client.15789 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:46:57.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.998+0000 7ffbc8135740 1 -- 172.21.15.164:0/720443855 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56378d587d70 con 0x56378d635600 2024-09-19T08:46:57.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.998+0000 7ffbbf7fe640 1 --2- 172.21.15.164:0/720443855 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x56378d635600 0x56378d6a20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:57.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.998+0000 7ffb9effd640 10 client.15789 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:46:57.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.998+0000 7ffbbf7fe640 1 --2- 172.21.15.164:0/720443855 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x56378d635600 0x56378d6a20e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=64a1782d372d7379 server_cookie=5425e448b6c6b504 in_seq=0 out_seq=0 2024-09-19T08:46:57.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.998+0000 7ffb9effd640 1 -- 172.21.15.164:0/720443855 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56378d587d70 con 0x56378d635600 2024-09-19T08:46:57.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.998+0000 7ffb9effd640 10 client.15789 handle_command_reply: tid=0 2024-09-19T08:46:57.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.998+0000 7ffbc8135740 1 client.15789 shutdown 2024-09-19T08:46:57.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.999+0000 7ffbc8135740 1 -- 172.21.15.164:0/720443855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56378d61a5d0 msgr2=0x56378d62d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:57.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.999+0000 7ffbc8135740 1 --2- 172.21.15.164:0/720443855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56378d61a5d0 0x56378d62d870 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4005930 tx=0x7ffbb4005770 comp rx=0 tx=0).stop 2024-09-19T08:46:57.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.999+0000 7ffbc8135740 1 -- 172.21.15.164:0/720443855 shutdown_connections 2024-09-19T08:46:57.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.999+0000 7ffbc8135740 1 --2- 172.21.15.164:0/720443855 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x56378d635600 0x56378d6a20e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:57.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.999+0000 7ffb9effd640 0 client.15789 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:46:57.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.999+0000 7ffbc8135740 1 --2- 172.21.15.164:0/720443855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56378d625370 0x56378d628d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:57.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.999+0000 7ffbc8135740 1 --2- 172.21.15.164:0/720443855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56378d622c50 0x56378d62ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:57.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.999+0000 7ffbc8135740 1 --2- 172.21.15.164:0/720443855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56378d61a5d0 0x56378d62d870 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:57.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.999+0000 7ffbc8135740 1 -- 172.21.15.164:0/720443855 >> 172.21.15.164:0/720443855 conn(0x56378d5e3310 msgr2=0x56378d623a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:57.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:56.999+0000 7ffbc8135740 1 -- 172.21.15.164:0/720443855 shutdown_connections 2024-09-19T08:46:57.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:57.000+0000 7ffbc8135740 1 -- 172.21.15.164:0/720443855 wait complete. 2024-09-19T08:46:57.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:57.000+0000 7ffbc8135740 20 client.15789 trim_cache size 0 max 16384 2024-09-19T08:46:57.003 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:46:57.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:57.001+0000 7ffbc5d5c640 1 -- 172.21.15.164:0/2797848316 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ffba80a35d0 msgr2=0x7ffba80a5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:57.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:57.001+0000 7ffbc5d5c640 1 --2- 172.21.15.164:0/2797848316 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ffba80a35d0 0x7ffba80a5a90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffbb80cebb0 tx=0x7ffbb8009b50 comp rx=0 tx=0).stop 2024-09-19T08:46:57.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:57.001+0000 7ffbc5d5c640 1 -- 172.21.15.164:0/2797848316 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbc0105de0 msgr2=0x7ffbc010cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:57.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:57.001+0000 7ffbc5d5c640 1 --2- 172.21.15.164:0/2797848316 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbc0105de0 0x7ffbc010cb60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffbb800ec50 tx=0x7ffbb800c6a0 comp rx=0 tx=0).stop 2024-09-19T08:46:57.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:57.001+0000 7ffbc5d5c640 1 -- 172.21.15.164:0/2797848316 shutdown_connections 2024-09-19T08:46:57.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:57.001+0000 7ffbc5d5c640 1 --2- 172.21.15.164:0/2797848316 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbc0105de0 0x7ffbc010cb60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:57.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:57.001+0000 7ffbc5d5c640 1 --2- 172.21.15.164:0/2797848316 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ffba80a35d0 0x7ffba80a5a90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:57.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:57.001+0000 7ffbc5d5c640 1 --2- 172.21.15.164:0/2797848316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbc0101e20 0x7ffbc019f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:57.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:57.001+0000 7ffbc5d5c640 1 --2- 172.21.15.164:0/2797848316 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbc0101470 0x7ffbc019f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:57.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:57.001+0000 7ffbc5d5c640 1 -- 172.21.15.164:0/2797848316 >> 172.21.15.164:0/2797848316 conn(0x7ffbc0077230 msgr2=0x7ffbc0103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:57.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:57.001+0000 7ffbc5d5c640 1 -- 172.21.15.164:0/2797848316 shutdown_connections 2024-09-19T08:46:57.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:57.001+0000 7ffbc5d5c640 1 -- 172.21.15.164:0/2797848316 wait complete. 2024-09-19T08:46:57.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:56 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:46:57.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:56 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:46:57.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:57 smithi164 ceph-mon[30928]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T08:46:57.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:57 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:46:57.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:57 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:46:57.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:57 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:46:57.783 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:57 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:46:57.783 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:57 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:46:57.898 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T08:46:58.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:57 smithi170 ceph-mon[35072]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T08:46:58.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:57 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:46:58.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:57 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:46:58.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:57 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:46:58.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:57 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:46:58.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:57 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:46:58.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:57 smithi173 ceph-mon[35306]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T08:46:58.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:57 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:46:58.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:57 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:46:58.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:57 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:46:58.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:57 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:46:58.162 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:57 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:46:58.162 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:46:58.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.654+0000 7fa2ad396640 1 -- 172.21.15.164:0/1958272696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2a8074620 msgr2=0x7fa2a8074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:58.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.654+0000 7fa2ad396640 1 --2- 172.21.15.164:0/1958272696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2a8074620 0x7fa2a8074a00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa298009900 tx=0x7fa29802a480 comp rx=0 tx=0).stop 2024-09-19T08:46:58.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.654+0000 7fa2ad396640 1 -- 172.21.15.164:0/1958272696 shutdown_connections 2024-09-19T08:46:58.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.654+0000 7fa2ad396640 1 --2- 172.21.15.164:0/1958272696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2a810e3c0 0x7fa2a8110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.654+0000 7fa2ad396640 1 --2- 172.21.15.164:0/1958272696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2a8074f40 0x7fa2a810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.654+0000 7fa2ad396640 1 --2- 172.21.15.164:0/1958272696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2a8074620 0x7fa2a8074a00 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.654+0000 7fa2ad396640 1 -- 172.21.15.164:0/1958272696 >> 172.21.15.164:0/1958272696 conn(0x7fa2a80fd8c0 msgr2=0x7fa2a80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:58.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.654+0000 7fa2ad396640 1 -- 172.21.15.164:0/1958272696 shutdown_connections 2024-09-19T08:46:58.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.654+0000 7fa2ad396640 1 -- 172.21.15.164:0/1958272696 wait complete. 2024-09-19T08:46:58.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.655+0000 7fa2ad396640 1 Processor -- start 2024-09-19T08:46:58.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.655+0000 7fa2ad396640 1 -- start start 2024-09-19T08:46:58.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.656+0000 7fa2ad396640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2a8074620 0x7fa2a819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:58.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.656+0000 7fa2ad396640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2a8074f40 0x7fa2a819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:58.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.656+0000 7fa2a6ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2a8074620 0x7fa2a819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:58.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.656+0000 7fa2a6ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2a8074620 0x7fa2a819a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59302/0 (socket says 172.21.15.164:59302) 2024-09-19T08:46:58.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.656+0000 7fa2a6ffd640 1 -- 172.21.15.164:0/156200502 learned_addr learned my addr 172.21.15.164:0/156200502 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:58.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.656+0000 7fa2ad396640 1 --2- 172.21.15.164:0/156200502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2a810e3c0 0x7fa2a819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:58.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.656+0000 7fa2ad396640 1 -- 172.21.15.164:0/156200502 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a8114170 con 0x7fa2a8074620 2024-09-19T08:46:58.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.656+0000 7fa2ad396640 1 -- 172.21.15.164:0/156200502 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a8113ff0 con 0x7fa2a8074f40 2024-09-19T08:46:58.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.656+0000 7fa2ad396640 1 -- 172.21.15.164:0/156200502 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81142f0 con 0x7fa2a810e3c0 2024-09-19T08:46:58.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.656+0000 7fa2a67fc640 1 --2- 172.21.15.164:0/156200502 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2a8074f40 0x7fa2a819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:58.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.656+0000 7fa2a77fe640 1 --2- 172.21.15.164:0/156200502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2a810e3c0 0x7fa2a819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:58.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.657+0000 7fa2a77fe640 1 -- 172.21.15.164:0/156200502 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2a8074f40 msgr2=0x7fa2a819ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:58.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.657+0000 7fa2a77fe640 1 --2- 172.21.15.164:0/156200502 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2a8074f40 0x7fa2a819ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.657+0000 7fa2a77fe640 1 -- 172.21.15.164:0/156200502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2a8074620 msgr2=0x7fa2a819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:58.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.657+0000 7fa2a77fe640 1 --2- 172.21.15.164:0/156200502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2a8074620 0x7fa2a819a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.657+0000 7fa2a77fe640 1 -- 172.21.15.164:0/156200502 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a819f600 con 0x7fa2a810e3c0 2024-09-19T08:46:58.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.657+0000 7fa2a77fe640 1 --2- 172.21.15.164:0/156200502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2a810e3c0 0x7fa2a819ef20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009fd0 tx=0x7fa2a000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:58.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.657+0000 7fa28ffff640 1 -- 172.21.15.164:0/156200502 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a000ce20 con 0x7fa2a810e3c0 2024-09-19T08:46:58.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.657+0000 7fa2ad396640 1 -- 172.21.15.164:0/156200502 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a819f8f0 con 0x7fa2a810e3c0 2024-09-19T08:46:58.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.657+0000 7fa28ffff640 1 -- 172.21.15.164:0/156200502 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2a0005020 con 0x7fa2a810e3c0 2024-09-19T08:46:58.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.657+0000 7fa28ffff640 1 -- 172.21.15.164:0/156200502 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0010720 con 0x7fa2a810e3c0 2024-09-19T08:46:58.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.657+0000 7fa2ad396640 1 -- 172.21.15.164:0/156200502 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a81a71d0 con 0x7fa2a810e3c0 2024-09-19T08:46:58.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.661+0000 7fa2af76f740 1 Processor -- start 2024-09-19T08:46:58.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.661+0000 7fa28ffff640 1 -- 172.21.15.164:0/156200502 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7fa2a0005540 con 0x7fa2a810e3c0 2024-09-19T08:46:58.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.662+0000 7fa2af76f740 1 -- start start 2024-09-19T08:46:58.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.662+0000 7fa2af76f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ab06e7b40 0x561ab06e6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:58.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.662+0000 7fa2af76f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561ab06e65d0 0x561ab06e6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:58.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.662+0000 7fa2af76f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ab06f1370 0x561ab06f3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:58.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.662+0000 7fa2af76f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561ab064c2f0 con 0x561ab06e7b40 2024-09-19T08:46:58.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.662+0000 7fa2af76f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561ab05f3cf0 con 0x561ab06f1370 2024-09-19T08:46:58.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.662+0000 7fa2af76f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x561ab06482b0 con 0x561ab06e65d0 2024-09-19T08:46:58.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.662+0000 7fa28ffff640 1 --2- 172.21.15.164:0/156200502 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa2900a34d0 0x7fa2900a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:58.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.662+0000 7fa2a67fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ab06e7b40 0x561ab06e6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:58.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.662+0000 7fa2a67fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ab06e7b40 0x561ab06e6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59314/0 (socket says 172.21.15.164:59314) 2024-09-19T08:46:58.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.662+0000 7fa2a67fc640 1 -- 172.21.15.164:0/2379646495 learned_addr learned my addr 172.21.15.164:0/2379646495 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:58.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.662+0000 7fa2a77fe640 1 --2- 172.21.15.164:0/156200502 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa2900a34d0 0x7fa2900a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:58.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.663+0000 7fa28ffff640 1 -- 172.21.15.164:0/156200502 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fa2a00d0550 con 0x7fa2a810e3c0 2024-09-19T08:46:58.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.663+0000 7fa2a6ffd640 1 --2- 172.21.15.164:0/2379646495 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561ab06e65d0 0x561ab06e6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:58.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.663+0000 7fa2a67fc640 1 --2- 172.21.15.164:0/2379646495 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ab06f1370 0x561ab06f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:58.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.663+0000 7fa2a67fc640 1 -- 172.21.15.164:0/2379646495 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561ab06e65d0 msgr2=0x561ab06e6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:58.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.663+0000 7fa2a67fc640 1 --2- 172.21.15.164:0/2379646495 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561ab06e65d0 0x561ab06e6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.663+0000 7fa2a67fc640 1 -- 172.21.15.164:0/2379646495 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ab06f1370 msgr2=0x561ab06f3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:58.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.663+0000 7fa2a67fc640 1 --2- 172.21.15.164:0/2379646495 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ab06f1370 0x561ab06f3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.663+0000 7fa2a67fc640 1 -- 172.21.15.164:0/2379646495 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ab05d83a0 con 0x561ab06e7b40 2024-09-19T08:46:58.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.663+0000 7fa2a77fe640 1 --2- 172.21.15.164:0/156200502 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa2900a34d0 0x7fa2900a5990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00cdb90 tx=0x7fa2a0098160 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:58.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.663+0000 7fa2a6ffd640 1 --2- 172.21.15.164:0/2379646495 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561ab06e65d0 0x561ab06e6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:58.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.664+0000 7fa2a67fc640 1 --2- 172.21.15.164:0/2379646495 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ab06e7b40 0x561ab06e6090 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa29c005bc0 tx=0x7fa29c009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ba87eb2779e769 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:58.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.664+0000 7fa28d7fa640 1 -- 172.21.15.164:0/2379646495 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c004030 con 0x561ab06e7b40 2024-09-19T08:46:58.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.664+0000 7fa28d7fa640 1 -- 172.21.15.164:0/2379646495 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa29c0041d0 con 0x561ab06e7b40 2024-09-19T08:46:58.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.664+0000 7fa28d7fa640 1 -- 172.21.15.164:0/2379646495 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c00f560 con 0x561ab06e7b40 2024-09-19T08:46:58.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.665+0000 7fa2af76f740 1 -- 172.21.15.164:0/2379646495 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ab06e7b40 msgr2=0x561ab06e6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:58.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.665+0000 7fa2af76f740 1 --2- 172.21.15.164:0/2379646495 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ab06e7b40 0x561ab06e6090 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa29c005bc0 tx=0x7fa29c009850 comp rx=0 tx=0).stop 2024-09-19T08:46:58.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.665+0000 7fa2af76f740 1 -- 172.21.15.164:0/2379646495 shutdown_connections 2024-09-19T08:46:58.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.665+0000 7fa2af76f740 1 --2- 172.21.15.164:0/2379646495 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ab06f1370 0x561ab06f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.665+0000 7fa2af76f740 1 --2- 172.21.15.164:0/2379646495 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561ab06e65d0 0x561ab06e6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.665+0000 7fa2af76f740 1 --2- 172.21.15.164:0/2379646495 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ab06e7b40 0x561ab06e6090 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.665+0000 7fa2af76f740 1 -- 172.21.15.164:0/2379646495 >> 172.21.15.164:0/2379646495 conn(0x561ab06af310 msgr2=0x561ab06e2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:58.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.665+0000 7fa2af76f740 1 -- 172.21.15.164:0/2379646495 shutdown_connections 2024-09-19T08:46:58.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.665+0000 7fa2af76f740 1 -- 172.21.15.164:0/2379646495 wait complete. 2024-09-19T08:46:58.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.666+0000 7fa2af76f740 1 -- start start 2024-09-19T08:46:58.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.666+0000 7fa2af76f740 1 Processor -- start 2024-09-19T08:46:58.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.667+0000 7fa2af76f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561ab06e65d0 0x561ab06f0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:58.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.667+0000 7fa2af76f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ab06e7b40 0x561ab06edfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:58.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.667+0000 7fa2af76f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ab06f1370 0x561ab06ee5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:58.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.667+0000 7fa2a6ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ab06e7b40 0x561ab06edfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:58.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.667+0000 7fa2a6ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ab06e7b40 0x561ab06edfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59330/0 (socket says 172.21.15.164:59330) 2024-09-19T08:46:58.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.667+0000 7fa2a67fc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561ab06e65d0 0x561ab06f0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:58.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.667+0000 7fa2a67fc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561ab06e65d0 0x561ab06f0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:35242/0 (socket says 172.21.15.164:35242) 2024-09-19T08:46:58.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.667+0000 7fa2a6ffd640 1 -- 172.21.15.164:0/2023548566 learned_addr learned my addr 172.21.15.164:0/2023548566 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:58.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.667+0000 7fa2af76f740 1 -- 172.21.15.164:0/2023548566 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561ab06f4070 con 0x561ab06e7b40 2024-09-19T08:46:58.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.667+0000 7fa2af76f740 1 -- 172.21.15.164:0/2023548566 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561ab0607060 con 0x561ab06f1370 2024-09-19T08:46:58.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.667+0000 7fa2af76f740 1 -- 172.21.15.164:0/2023548566 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x561ab06f41f0 con 0x561ab06e65d0 2024-09-19T08:46:58.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.667+0000 7fa2a6ffd640 1 -- 172.21.15.164:0/2023548566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561ab06e65d0 msgr2=0x561ab06f0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:58.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.667+0000 7fa2a6ffd640 1 --2- 172.21.15.164:0/2023548566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561ab06e65d0 0x561ab06f0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.667+0000 7fa2a6ffd640 1 -- 172.21.15.164:0/2023548566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ab06f1370 msgr2=0x561ab06ee5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:58.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.668+0000 7fa2a67fc640 1 --2- 172.21.15.164:0/2023548566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ab06f1370 0x561ab06ee5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:58.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.668+0000 7fa2a6ffd640 1 --2- 172.21.15.164:0/2023548566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ab06f1370 0x561ab06ee5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.668+0000 7fa2a6ffd640 1 -- 172.21.15.164:0/2023548566 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ab03bd3c0 con 0x561ab06e7b40 2024-09-19T08:46:58.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.668+0000 7fa2a6ffd640 1 --2- 172.21.15.164:0/2023548566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ab06e7b40 0x561ab06edfe0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa298004750 tx=0x7fa298005450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7809746c8950f03c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:58.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.668+0000 7fa28aff5640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:46:58.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.668+0000 7fa2af76f740 10 client.0 _ll_register_callbacks cb 0x561ab035a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:58.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.668+0000 7fa2af76f740 1 -- 172.21.15.164:0/2023548566 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa29c0041d0 con 0x561ab06e7b40 2024-09-19T08:46:58.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.668+0000 7fa28aff5640 1 -- 172.21.15.164:0/2023548566 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298035020 con 0x561ab06e7b40 2024-09-19T08:46:58.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.668+0000 7fa28aff5640 1 -- 172.21.15.164:0/2023548566 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fa298004040 con 0x561ab06e7b40 2024-09-19T08:46:58.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.668+0000 7fa28aff5640 1 -- 172.21.15.164:0/2023548566 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa29804e050 con 0x561ab06e7b40 2024-09-19T08:46:58.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.668+0000 7fa2af76f740 10 client.15801 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:58.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.668+0000 7fa2af76f740 1 -- 172.21.15.164:0/2023548566 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({fsmap=28}) -- 0x561ab06ff8c0 con 0x561ab06e7b40 2024-09-19T08:46:58.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.669+0000 7fa28aff5640 1 -- 172.21.15.164:0/2023548566 <== mon.0 v2:172.21.15.164:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa29803a040 con 0x561ab06e7b40 2024-09-19T08:46:58.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.669+0000 7fa2af76f740 10 client.15801 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:58.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.669+0000 7fa2af76f740 10 client.15801 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T08:46:58.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.669+0000 7fa2af76f740 20 client.15801 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:58.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.669+0000 7fa2af76f740 1 --2- 172.21.15.164:0/2023548566 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x561ab0700c90 0x561ab0703120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:58.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.669+0000 7fa2af76f740 4 client.15801 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:46:58.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.669+0000 7fa2af76f740 1 -- 172.21.15.164:0/2023548566 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ab0659860 con 0x561ab0700c90 2024-09-19T08:46:58.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.669+0000 7fa2a67fc640 1 --2- 172.21.15.164:0/2023548566 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x561ab0700c90 0x561ab0703120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:58.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.670+0000 7fa2a67fc640 1 --2- 172.21.15.164:0/2023548566 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x561ab0700c90 0x561ab0703120 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=561f7f938364ba13 server_cookie=271413e691fd5f83 in_seq=0 out_seq=0 2024-09-19T08:46:58.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.670+0000 7fa28aff5640 10 client.15801 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:46:58.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.670+0000 7fa28aff5640 1 -- 172.21.15.164:0/2023548566 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561ab0659860 con 0x561ab0700c90 2024-09-19T08:46:58.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.670+0000 7fa28aff5640 10 client.15801 handle_command_reply: tid=0 2024-09-19T08:46:58.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.670+0000 7fa2af76f740 1 client.15801 shutdown 2024-09-19T08:46:58.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.671+0000 7fa2af76f740 1 -- 172.21.15.164:0/2023548566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ab06e7b40 msgr2=0x561ab06edfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:58.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.671+0000 7fa2af76f740 1 --2- 172.21.15.164:0/2023548566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ab06e7b40 0x561ab06edfe0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa298004750 tx=0x7fa298005450 comp rx=0 tx=0).stop 2024-09-19T08:46:58.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.671+0000 7fa2af76f740 1 -- 172.21.15.164:0/2023548566 shutdown_connections 2024-09-19T08:46:58.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.671+0000 7fa2af76f740 1 --2- 172.21.15.164:0/2023548566 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x561ab0700c90 0x561ab0703120 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.671+0000 7fa28aff5640 0 client.15801 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:46:58.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.671+0000 7fa2af76f740 1 --2- 172.21.15.164:0/2023548566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ab06f1370 0x561ab06ee5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.671+0000 7fa2af76f740 1 --2- 172.21.15.164:0/2023548566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ab06e7b40 0x561ab06edfe0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.671+0000 7fa2af76f740 1 --2- 172.21.15.164:0/2023548566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561ab06e65d0 0x561ab06f0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.671+0000 7fa2af76f740 1 -- 172.21.15.164:0/2023548566 >> 172.21.15.164:0/2023548566 conn(0x561ab06af310 msgr2=0x561ab06e8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:58.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.671+0000 7fa2af76f740 1 -- 172.21.15.164:0/2023548566 shutdown_connections 2024-09-19T08:46:58.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.671+0000 7fa2af76f740 1 -- 172.21.15.164:0/2023548566 wait complete. 2024-09-19T08:46:58.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.671+0000 7fa2af76f740 20 client.15801 trim_cache size 0 max 16384 2024-09-19T08:46:58.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.696+0000 7fa2af76f740 1 Processor -- start 2024-09-19T08:46:58.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.696+0000 7fa2af76f740 1 -- start start 2024-09-19T08:46:58.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.696+0000 7fa2af76f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561ab06e65d0 0x561ab06fd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:58.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.696+0000 7fa2af76f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ab06e7b40 0x561ab06ec8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:58.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.696+0000 7fa2af76f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ab06f1370 0x561ab0708a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:58.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.696+0000 7fa2af76f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81af910 con 0x561ab06e7b40 2024-09-19T08:46:58.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.696+0000 7fa2af76f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81b1450 con 0x561ab06f1370 2024-09-19T08:46:58.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.696+0000 7fa2af76f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81affe0 con 0x561ab06e65d0 2024-09-19T08:46:58.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.696+0000 7fa2a6ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ab06e7b40 0x561ab06ec8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:58.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.696+0000 7fa2a6ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ab06e7b40 0x561ab06ec8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59346/0 (socket says 172.21.15.164:59346) 2024-09-19T08:46:58.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.696+0000 7fa2a6ffd640 1 -- 172.21.15.164:0/1981937352 learned_addr learned my addr 172.21.15.164:0/1981937352 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:58.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.697+0000 7fa2a67fc640 1 --2- 172.21.15.164:0/1981937352 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ab06f1370 0x561ab0708a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:58.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.697+0000 7fa2a67fc640 1 --2- 172.21.15.164:0/1981937352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561ab06e65d0 0x561ab06fd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:58.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.697+0000 7fa2a6ffd640 1 -- 172.21.15.164:0/1981937352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561ab06e65d0 msgr2=0x561ab06fd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:58.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.697+0000 7fa2a6ffd640 1 --2- 172.21.15.164:0/1981937352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561ab06e65d0 0x561ab06fd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.697+0000 7fa2a6ffd640 1 -- 172.21.15.164:0/1981937352 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ab06f1370 msgr2=0x561ab0708a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:58.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.697+0000 7fa2a6ffd640 1 --2- 172.21.15.164:0/1981937352 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ab06f1370 0x561ab0708a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.697+0000 7fa2a6ffd640 1 -- 172.21.15.164:0/1981937352 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ab06f10a0 con 0x561ab06e7b40 2024-09-19T08:46:58.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.697+0000 7fa2a67fc640 1 --2- 172.21.15.164:0/1981937352 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ab06f1370 0x561ab0708a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:58.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.697+0000 7fa2a6ffd640 1 --2- 172.21.15.164:0/1981937352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ab06e7b40 0x561ab06ec8b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa2980098d0 tx=0x7fa298040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f141919eca7c3e91 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:58.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.697+0000 7fa28cff9640 1 -- 172.21.15.164:0/1981937352 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298057070 con 0x561ab06e7b40 2024-09-19T08:46:58.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.697+0000 7fa28cff9640 1 -- 172.21.15.164:0/1981937352 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa29c0041d0 con 0x561ab06e7b40 2024-09-19T08:46:58.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.697+0000 7fa28cff9640 1 -- 172.21.15.164:0/1981937352 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298035070 con 0x561ab06e7b40 2024-09-19T08:46:58.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.698+0000 7fa2af76f740 1 -- 172.21.15.164:0/1981937352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ab06e7b40 msgr2=0x561ab06ec8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:58.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.698+0000 7fa2af76f740 1 --2- 172.21.15.164:0/1981937352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ab06e7b40 0x561ab06ec8b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa2980098d0 tx=0x7fa298040000 comp rx=0 tx=0).stop 2024-09-19T08:46:58.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.698+0000 7fa2af76f740 1 -- 172.21.15.164:0/1981937352 shutdown_connections 2024-09-19T08:46:58.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.698+0000 7fa2af76f740 1 --2- 172.21.15.164:0/1981937352 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ab06f1370 0x561ab0708a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.698+0000 7fa2af76f740 1 --2- 172.21.15.164:0/1981937352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ab06e7b40 0x561ab06ec8b0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.698+0000 7fa2af76f740 1 --2- 172.21.15.164:0/1981937352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561ab06e65d0 0x561ab06fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.698+0000 7fa2af76f740 1 -- 172.21.15.164:0/1981937352 >> 172.21.15.164:0/1981937352 conn(0x561ab06af310 msgr2=0x561ab06e8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:58.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.699+0000 7fa2af76f740 1 -- 172.21.15.164:0/1981937352 shutdown_connections 2024-09-19T08:46:58.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.699+0000 7fa2af76f740 1 -- 172.21.15.164:0/1981937352 wait complete. 2024-09-19T08:46:58.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.699+0000 7fa2af76f740 1 -- start start 2024-09-19T08:46:58.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.699+0000 7fa2af76f740 1 Processor -- start 2024-09-19T08:46:58.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.699+0000 7fa2af76f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561ab06e65d0 0x561ab06f5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:58.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.699+0000 7fa2af76f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ab06e7b40 0x561ab06f56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:58.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.699+0000 7fa2af76f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ab06f1370 0x561ab06f5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:58.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.699+0000 7fa2af76f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81b0d80 con 0x561ab06f1370 2024-09-19T08:46:58.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.699+0000 7fa2af76f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81b06b0 con 0x561ab06e7b40 2024-09-19T08:46:58.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.699+0000 7fa2af76f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81ad910 con 0x561ab06e65d0 2024-09-19T08:46:58.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.699+0000 7fa2af76f740 10 client.0 _ll_register_callbacks cb 0x7fa2a00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:58.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.700+0000 7fa2a67fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ab06f1370 0x561ab06f5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:58.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.700+0000 7fa2a67fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ab06f1370 0x561ab06f5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59356/0 (socket says 172.21.15.164:59356) 2024-09-19T08:46:58.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.700+0000 7fa2a67fc640 1 -- 172.21.15.164:0/1825052880 learned_addr learned my addr 172.21.15.164:0/1825052880 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:46:58.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.700+0000 7fa2a67fc640 1 --2- 172.21.15.164:0/1825052880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561ab06e65d0 0x561ab06f5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:58.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.700+0000 7fa2a6ffd640 1 --2- 172.21.15.164:0/1825052880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ab06e7b40 0x561ab06f56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:58.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.700+0000 7fa2a6ffd640 1 -- 172.21.15.164:0/1825052880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561ab06e65d0 msgr2=0x561ab06f5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:58.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.700+0000 7fa2a6ffd640 1 --2- 172.21.15.164:0/1825052880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561ab06e65d0 0x561ab06f5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.700+0000 7fa2a6ffd640 1 -- 172.21.15.164:0/1825052880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ab06f1370 msgr2=0x561ab06f5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:58.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.700+0000 7fa2a6ffd640 1 --2- 172.21.15.164:0/1825052880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ab06f1370 0x561ab06f5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.700+0000 7fa2a6ffd640 1 -- 172.21.15.164:0/1825052880 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ab076aa00 con 0x561ab06e7b40 2024-09-19T08:46:58.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.700+0000 7fa2a67fc640 1 --2- 172.21.15.164:0/1825052880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ab06f1370 0x561ab06f5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:46:58.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.700+0000 7fa2a6ffd640 1 --2- 172.21.15.164:0/1825052880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ab06e7b40 0x561ab06f56d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa2980058a0 tx=0x7fa298005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc1b89afb76e4f24 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:58.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.700+0000 7fa28aff5640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-19T08:46:58.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.700+0000 7fa28aff5640 1 -- 172.21.15.164:0/1825052880 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298035070 con 0x561ab06e7b40 2024-09-19T08:46:58.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.701+0000 7fa2af76f740 1 -- 172.21.15.164:0/1825052880 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa29c0041d0 con 0x561ab06e7b40 2024-09-19T08:46:58.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.701+0000 7fa28aff5640 1 -- 172.21.15.164:0/1825052880 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fa298004d70 con 0x561ab06e7b40 2024-09-19T08:46:58.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.701+0000 7fa28aff5640 1 -- 172.21.15.164:0/1825052880 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa298057040 con 0x561ab06e7b40 2024-09-19T08:46:58.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.701+0000 7fa2af76f740 10 client.25328 fetch_fsmap learned FSMap version 28 2024-09-19T08:46:58.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.701+0000 7fa2af76f740 1 -- 172.21.15.164:0/1825052880 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561ab076bc90 con 0x561ab06e7b40 2024-09-19T08:46:58.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.701+0000 7fa28aff5640 1 -- 172.21.15.164:0/1825052880 <== mon.2 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa298004860 con 0x561ab06e7b40 2024-09-19T08:46:58.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.701+0000 7fa28aff5640 1 -- 172.21.15.164:0/1825052880 <== mon.2 v2:172.21.15.170:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa29805a070 con 0x561ab06e7b40 2024-09-19T08:46:58.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.701+0000 7fa2af76f740 10 client.25328 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:46:58.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.701+0000 7fa2af76f740 10 client.25328 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T08:46:58.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.702+0000 7fa2af76f740 20 client.25328 populate_metadata read hostname 'smithi164' 2024-09-19T08:46:58.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.702+0000 7fa2af76f740 1 --2- 172.21.15.164:0/1825052880 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x561ab076c2a0 0x561ab076e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:58.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.702+0000 7fa2af76f740 4 client.25328 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:46:58.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.702+0000 7fa2af76f740 1 -- 172.21.15.164:0/1825052880 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa29c0041d0 con 0x561ab076c2a0 2024-09-19T08:46:58.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.702+0000 7fa2a67fc640 1 --2- 172.21.15.164:0/1825052880 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x561ab076c2a0 0x561ab076e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:58.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.702+0000 7fa2a67fc640 1 --2- 172.21.15.164:0/1825052880 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x561ab076c2a0 0x561ab076e690 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a1141fed5c2a33ec server_cookie=d48df05193049693 in_seq=0 out_seq=0 2024-09-19T08:46:58.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.702+0000 7fa28aff5640 10 client.25328 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:46:58.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.702+0000 7fa28aff5640 1 -- 172.21.15.164:0/1825052880 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa29c0041d0 con 0x561ab076c2a0 2024-09-19T08:46:58.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.703+0000 7fa28aff5640 10 client.25328 handle_command_reply: tid=0 2024-09-19T08:46:58.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.703+0000 7fa2af76f740 1 client.25328 shutdown 2024-09-19T08:46:58.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.703+0000 7fa2af76f740 1 -- 172.21.15.164:0/1825052880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ab06e7b40 msgr2=0x561ab06f56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:58.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.703+0000 7fa2af76f740 1 --2- 172.21.15.164:0/1825052880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ab06e7b40 0x561ab06f56d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa2980058a0 tx=0x7fa298005b50 comp rx=0 tx=0).stop 2024-09-19T08:46:58.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.703+0000 7fa2af76f740 1 -- 172.21.15.164:0/1825052880 shutdown_connections 2024-09-19T08:46:58.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.703+0000 7fa2af76f740 1 --2- 172.21.15.164:0/1825052880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ab06f1370 0x561ab06f5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.703+0000 7fa2af76f740 1 --2- 172.21.15.164:0/1825052880 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x561ab076c2a0 0x561ab076e690 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.703+0000 7fa2af76f740 1 --2- 172.21.15.164:0/1825052880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ab06e7b40 0x561ab06f56d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.703+0000 7fa2af76f740 1 --2- 172.21.15.164:0/1825052880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561ab06e65d0 0x561ab06f5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.703+0000 7fa2af76f740 1 -- 172.21.15.164:0/1825052880 >> 172.21.15.164:0/1825052880 conn(0x561ab06af310 msgr2=0x561ab06e14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:58.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.703+0000 7fa28aff5640 0 client.25328 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:46:58.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.703+0000 7fa2af76f740 1 -- 172.21.15.164:0/1825052880 shutdown_connections 2024-09-19T08:46:58.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.703+0000 7fa2af76f740 1 -- 172.21.15.164:0/1825052880 wait complete. 2024-09-19T08:46:58.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.703+0000 7fa2af76f740 20 client.25328 trim_cache size 0 max 16384 2024-09-19T08:46:58.710 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:46:58.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.704+0000 7fa2ad396640 1 -- 172.21.15.164:0/156200502 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa2900a34d0 msgr2=0x7fa2900a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:58.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.704+0000 7fa2ad396640 1 --2- 172.21.15.164:0/156200502 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa2900a34d0 0x7fa2900a5990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00cdb90 tx=0x7fa2a0098160 comp rx=0 tx=0).stop 2024-09-19T08:46:58.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.704+0000 7fa2ad396640 1 -- 172.21.15.164:0/156200502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2a810e3c0 msgr2=0x7fa2a819ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:58.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.704+0000 7fa2ad396640 1 --2- 172.21.15.164:0/156200502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2a810e3c0 0x7fa2a819ef20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009fd0 tx=0x7fa2a000ef10 comp rx=0 tx=0).stop 2024-09-19T08:46:58.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.704+0000 7fa2ad396640 1 -- 172.21.15.164:0/156200502 shutdown_connections 2024-09-19T08:46:58.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.704+0000 7fa2ad396640 1 --2- 172.21.15.164:0/156200502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2a810e3c0 0x7fa2a819ef20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.704+0000 7fa2ad396640 1 --2- 172.21.15.164:0/156200502 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2a8074f40 0x7fa2a819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.704+0000 7fa2ad396640 1 --2- 172.21.15.164:0/156200502 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa2900a34d0 0x7fa2900a5990 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.704+0000 7fa2ad396640 1 --2- 172.21.15.164:0/156200502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2a8074620 0x7fa2a819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.704+0000 7fa2ad396640 1 -- 172.21.15.164:0/156200502 >> 172.21.15.164:0/156200502 conn(0x7fa2a80fd8c0 msgr2=0x7fa2a80fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:58.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.704+0000 7fa2ad396640 1 -- 172.21.15.164:0/156200502 shutdown_connections 2024-09-19T08:46:58.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:46:58.704+0000 7fa2ad396640 1 -- 172.21.15.164:0/156200502 wait complete. 2024-09-19T08:46:59.610 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T08:46:59.891 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:47:00.013 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:46:59 smithi164 ceph-mon[30928]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T08:47:00.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:46:59 smithi170 ceph-mon[35072]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T08:47:00.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:46:59 smithi173 ceph-mon[35306]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T08:47:00.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.472+0000 7f2b26ef9640 1 -- 172.21.15.164:0/1484353520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b20101410 msgr2=0x7f2b2010d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:00.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.472+0000 7f2b26ef9640 1 --2- 172.21.15.164:0/1484353520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b20101410 0x7f2b2010d940 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2b18009890 tx=0x7f2b1802a380 comp rx=0 tx=0).stop 2024-09-19T08:47:00.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.472+0000 7f2b26ef9640 1 -- 172.21.15.164:0/1484353520 shutdown_connections 2024-09-19T08:47:00.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.472+0000 7f2b26ef9640 1 --2- 172.21.15.164:0/1484353520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b20101410 0x7f2b2010d940 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.472+0000 7f2b26ef9640 1 --2- 172.21.15.164:0/1484353520 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b20100a70 0x7f2b20100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.472+0000 7f2b26ef9640 1 --2- 172.21.15.164:0/1484353520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b20106a70 0x7f2b20106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.472+0000 7f2b26ef9640 1 -- 172.21.15.164:0/1484353520 >> 172.21.15.164:0/1484353520 conn(0x7f2b200fc7d0 msgr2=0x7f2b200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:00.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.472+0000 7f2b26ef9640 1 -- 172.21.15.164:0/1484353520 shutdown_connections 2024-09-19T08:47:00.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.472+0000 7f2b26ef9640 1 -- 172.21.15.164:0/1484353520 wait complete. 2024-09-19T08:47:00.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.473+0000 7f2b26ef9640 1 Processor -- start 2024-09-19T08:47:00.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.473+0000 7f2b26ef9640 1 -- start start 2024-09-19T08:47:00.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.473+0000 7f2b26ef9640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b20100a70 0x7f2b2019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:00.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.473+0000 7f2b26ef9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b20101410 0x7f2b2019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:00.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.473+0000 7f2b26ef9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b20106a70 0x7f2b2019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:00.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.474+0000 7f2b26ef9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2b20110e70 con 0x7f2b20106a70 2024-09-19T08:47:00.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.474+0000 7f2b26ef9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2b20110cf0 con 0x7f2b20101410 2024-09-19T08:47:00.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.474+0000 7f2b26ef9640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2b20110ff0 con 0x7f2b20100a70 2024-09-19T08:47:00.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.474+0000 7f2b25496640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b20106a70 0x7f2b2019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:00.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.474+0000 7f2b25496640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b20106a70 0x7f2b2019ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59362/0 (socket says 172.21.15.164:59362) 2024-09-19T08:47:00.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.474+0000 7f2b25496640 1 -- 172.21.15.164:0/3725213578 learned_addr learned my addr 172.21.15.164:0/3725213578 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:00.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.474+0000 7f2b17fff640 1 --2- 172.21.15.164:0/3725213578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b20101410 0x7f2b2019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:00.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.474+0000 7f2b25496640 1 -- 172.21.15.164:0/3725213578 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b20100a70 msgr2=0x7f2b2019a4c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:47:00.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.474+0000 7f2b25496640 1 --2- 172.21.15.164:0/3725213578 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b20100a70 0x7f2b2019a4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.474+0000 7f2b25496640 1 -- 172.21.15.164:0/3725213578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b20101410 msgr2=0x7f2b2019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:00.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.474+0000 7f2b25496640 1 --2- 172.21.15.164:0/3725213578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b20101410 0x7f2b2019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.474+0000 7f2b25496640 1 -- 172.21.15.164:0/3725213578 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b2019f470 con 0x7f2b20106a70 2024-09-19T08:47:00.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.474+0000 7f2b17fff640 1 --2- 172.21.15.164:0/3725213578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b20101410 0x7f2b2019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:47:00.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.474+0000 7f2b25496640 1 --2- 172.21.15.164:0/3725213578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b20106a70 0x7f2b2019ed90 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2b18009780 tx=0x7f2b18033d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:00.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.475+0000 7f2b15ffb640 1 -- 172.21.15.164:0/3725213578 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1803f020 con 0x7f2b20106a70 2024-09-19T08:47:00.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.475+0000 7f2b15ffb640 1 -- 172.21.15.164:0/3725213578 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b18004240 con 0x7f2b20106a70 2024-09-19T08:47:00.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.475+0000 7f2b15ffb640 1 -- 172.21.15.164:0/3725213578 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b18037040 con 0x7f2b20106a70 2024-09-19T08:47:00.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.475+0000 7f2b26ef9640 1 -- 172.21.15.164:0/3725213578 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b2019f700 con 0x7f2b20106a70 2024-09-19T08:47:00.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.475+0000 7f2b26ef9640 1 -- 172.21.15.164:0/3725213578 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b2019fbb0 con 0x7f2b20106a70 2024-09-19T08:47:00.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.478+0000 7f2b15ffb640 1 -- 172.21.15.164:0/3725213578 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7f2b18042020 con 0x7f2b20106a70 2024-09-19T08:47:00.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.478+0000 7f2b15ffb640 1 --2- 172.21.15.164:0/3725213578 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f2b080a3480 0x7f2b080a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:00.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.479+0000 7f2b15ffb640 1 -- 172.21.15.164:0/3725213578 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f2b180ef330 con 0x7f2b20106a70 2024-09-19T08:47:00.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.479+0000 7f2b24c95640 1 --2- 172.21.15.164:0/3725213578 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f2b080a3480 0x7f2b080a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:00.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.479+0000 7f2b292d2740 1 Processor -- start 2024-09-19T08:47:00.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.479+0000 7f2b292d2740 1 -- start start 2024-09-19T08:47:00.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.479+0000 7f2b292d2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a2e18b40 0x55d2a2e17090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:00.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.479+0000 7f2b292d2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2a2e175d0 0x55d2a2e17a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:00.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.479+0000 7f2b292d2740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2a2e22370 0x55d2a2e24760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:00.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.480+0000 7f2b292d2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d2a2d7d2f0 con 0x55d2a2e18b40 2024-09-19T08:47:00.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.480+0000 7f2b292d2740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d2a2d24cf0 con 0x55d2a2e22370 2024-09-19T08:47:00.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.480+0000 7f2b292d2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d2a2d792b0 con 0x55d2a2e175d0 2024-09-19T08:47:00.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.480+0000 7f2b17fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2a2e175d0 0x55d2a2e17a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:00.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.480+0000 7f2b25496640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2a2e22370 0x55d2a2e24760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:00.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.480+0000 7f2b25496640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2a2e22370 0x55d2a2e24760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:42094/0 (socket says 172.21.15.164:42094) 2024-09-19T08:47:00.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.480+0000 7f2b25496640 1 -- 172.21.15.164:0/3960428930 learned_addr learned my addr 172.21.15.164:0/3960428930 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:00.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.480+0000 7f2b24c95640 1 --2- 172.21.15.164:0/3960428930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a2e18b40 0x55d2a2e17090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:00.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.480+0000 7f2b25496640 1 -- 172.21.15.164:0/3960428930 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2a2e175d0 msgr2=0x55d2a2e17a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:00.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.480+0000 7f2b25496640 1 --2- 172.21.15.164:0/3960428930 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2a2e175d0 0x55d2a2e17a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.480+0000 7f2b25496640 1 -- 172.21.15.164:0/3960428930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a2e18b40 msgr2=0x55d2a2e17090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:00.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.480+0000 7f2b25496640 1 --2- 172.21.15.164:0/3960428930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a2e18b40 0x55d2a2e17090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.480+0000 7f2b25496640 1 -- 172.21.15.164:0/3960428930 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2a2d093a0 con 0x55d2a2e22370 2024-09-19T08:47:00.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.480+0000 7f2b24c95640 1 --2- 172.21.15.164:0/3960428930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a2e18b40 0x55d2a2e17090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:00.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.480+0000 7f2b24c95640 1 --2- 172.21.15.164:0/3725213578 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f2b080a3480 0x7f2b080a5940 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c005fd0 tx=0x7f2b1c009500 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:00.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.481+0000 7f2b25496640 1 --2- 172.21.15.164:0/3960428930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2a2e22370 0x55d2a2e24760 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2b180b7f90 tx=0x7f2b180bb390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ec9c16f844f5025 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:00.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.481+0000 7f2b06ffd640 1 -- 172.21.15.164:0/3960428930 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b180bbcc0 con 0x55d2a2e22370 2024-09-19T08:47:00.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.481+0000 7f2b06ffd640 1 -- 172.21.15.164:0/3960428930 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b180bb5c0 con 0x55d2a2e22370 2024-09-19T08:47:00.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.481+0000 7f2b06ffd640 1 -- 172.21.15.164:0/3960428930 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b180b7c30 con 0x55d2a2e22370 2024-09-19T08:47:00.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.482+0000 7f2b292d2740 1 -- 172.21.15.164:0/3960428930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2a2e22370 msgr2=0x55d2a2e24760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:00.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.482+0000 7f2b292d2740 1 --2- 172.21.15.164:0/3960428930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2a2e22370 0x55d2a2e24760 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2b180b7f90 tx=0x7f2b180bb390 comp rx=0 tx=0).stop 2024-09-19T08:47:00.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.482+0000 7f2b292d2740 1 -- 172.21.15.164:0/3960428930 shutdown_connections 2024-09-19T08:47:00.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.482+0000 7f2b292d2740 1 --2- 172.21.15.164:0/3960428930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2a2e22370 0x55d2a2e24760 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.482+0000 7f2b292d2740 1 --2- 172.21.15.164:0/3960428930 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2a2e175d0 0x55d2a2e17a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.482+0000 7f2b292d2740 1 --2- 172.21.15.164:0/3960428930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a2e18b40 0x55d2a2e17090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.482+0000 7f2b292d2740 1 -- 172.21.15.164:0/3960428930 >> 172.21.15.164:0/3960428930 conn(0x55d2a2de0310 msgr2=0x55d2a2e13c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:00.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.482+0000 7f2b292d2740 1 -- 172.21.15.164:0/3960428930 shutdown_connections 2024-09-19T08:47:00.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.482+0000 7f2b292d2740 1 -- 172.21.15.164:0/3960428930 wait complete. 2024-09-19T08:47:00.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.482+0000 7f2b292d2740 1 -- start start 2024-09-19T08:47:00.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.482+0000 7f2b292d2740 1 Processor -- start 2024-09-19T08:47:00.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.483+0000 7f2b292d2740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2a2e175d0 0x55d2a2e21dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:00.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.483+0000 7f2b292d2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2a2e18b40 0x55d2a2e1f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:00.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.483+0000 7f2b292d2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a2e22370 0x55d2a2e1f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:00.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.483+0000 7f2b292d2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d2a2d24cf0 con 0x55d2a2e22370 2024-09-19T08:47:00.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.483+0000 7f2b292d2740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d2a2d38060 con 0x55d2a2e175d0 2024-09-19T08:47:00.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.483+0000 7f2b292d2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d2a2e24e30 con 0x55d2a2e18b40 2024-09-19T08:47:00.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.483+0000 7f2b24c95640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2a2e175d0 0x55d2a2e21dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:00.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.483+0000 7f2b25496640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a2e22370 0x55d2a2e1f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:00.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.483+0000 7f2b24c95640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2a2e175d0 0x55d2a2e21dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:42100/0 (socket says 172.21.15.164:42100) 2024-09-19T08:47:00.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.483+0000 7f2b24c95640 1 -- 172.21.15.164:0/652586092 learned_addr learned my addr 172.21.15.164:0/652586092 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:00.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.483+0000 7f2b17fff640 1 --2- 172.21.15.164:0/652586092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2a2e18b40 0x55d2a2e1f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:00.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.484+0000 7f2b292d2740 10 client.0 _ll_register_callbacks cb 0x55d2a2a8b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:47:00.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.484+0000 7f2b17fff640 1 -- 172.21.15.164:0/652586092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2a2e175d0 msgr2=0x55d2a2e21dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:00.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.484+0000 7f2b17fff640 1 --2- 172.21.15.164:0/652586092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2a2e175d0 0x55d2a2e21dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.484+0000 7f2b17fff640 1 -- 172.21.15.164:0/652586092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a2e22370 msgr2=0x55d2a2e1f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:00.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.484+0000 7f2b17fff640 1 --2- 172.21.15.164:0/652586092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a2e22370 0x55d2a2e1f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.484+0000 7f2b17fff640 1 -- 172.21.15.164:0/652586092 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2a2aee3c0 con 0x55d2a2e18b40 2024-09-19T08:47:00.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.484+0000 7f2b24c95640 1 --2- 172.21.15.164:0/652586092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2a2e175d0 0x55d2a2e21dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:47:00.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.484+0000 7f2b17fff640 1 --2- 172.21.15.164:0/652586092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2a2e18b40 0x55d2a2e1f070 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2b200ff120 tx=0x7f2b10012b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e9dd40bbd22e788 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:00.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.484+0000 7f2afffff640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-09-19T08:47:00.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.484+0000 7f2afffff640 1 -- 172.21.15.164:0/652586092 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10011b70 con 0x55d2a2e18b40 2024-09-19T08:47:00.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.484+0000 7f2b292d2740 1 -- 172.21.15.164:0/652586092 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b180bb5c0 con 0x55d2a2e18b40 2024-09-19T08:47:00.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.484+0000 7f2afffff640 1 -- 172.21.15.164:0/652586092 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f2b10011d10 con 0x55d2a2e18b40 2024-09-19T08:47:00.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.484+0000 7f2afffff640 1 -- 172.21.15.164:0/652586092 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2b10005020 con 0x55d2a2e18b40 2024-09-19T08:47:00.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.485+0000 7f2b292d2740 10 client.25345 fetch_fsmap learned FSMap version 28 2024-09-19T08:47:00.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.485+0000 7f2b292d2740 1 -- 172.21.15.164:0/652586092 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d2a2e38ec0 con 0x55d2a2e18b40 2024-09-19T08:47:00.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.485+0000 7f2afffff640 1 -- 172.21.15.164:0/652586092 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b180bb5c0 con 0x55d2a2e18b40 2024-09-19T08:47:00.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.485+0000 7f2afffff640 1 -- 172.21.15.164:0/652586092 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2b10035020 con 0x55d2a2e18b40 2024-09-19T08:47:00.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.485+0000 7f2b292d2740 10 client.25345 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:47:00.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.485+0000 7f2b292d2740 10 client.25345 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:47:00.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.485+0000 7f2b292d2740 20 client.25345 populate_metadata read hostname 'smithi164' 2024-09-19T08:47:00.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.485+0000 7f2b292d2740 1 --2- 172.21.15.164:0/652586092 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55d2a2e3a320 0x55d2a2e3c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:00.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.485+0000 7f2b292d2740 4 client.25345 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:47:00.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.485+0000 7f2b292d2740 1 -- 172.21.15.164:0/652586092 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2a2d8a860 con 0x55d2a2e3a320 2024-09-19T08:47:00.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.485+0000 7f2b24c95640 1 --2- 172.21.15.164:0/652586092 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55d2a2e3a320 0x55d2a2e3c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:00.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.486+0000 7f2afffff640 10 client.25345 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:47:00.498 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.486+0000 7f2b24c95640 1 --2- 172.21.15.164:0/652586092 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55d2a2e3a320 0x55d2a2e3c710 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb64d66de4e3a6eb server_cookie=bc367421452d846f in_seq=0 out_seq=0 2024-09-19T08:47:00.498 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.486+0000 7f2afffff640 1 -- 172.21.15.164:0/652586092 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d2a2d8a860 con 0x55d2a2e3a320 2024-09-19T08:47:00.498 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.486+0000 7f2afffff640 10 client.25345 handle_command_reply: tid=0 2024-09-19T08:47:00.498 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.487+0000 7f2b292d2740 1 client.25345 shutdown 2024-09-19T08:47:00.498 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.487+0000 7f2b292d2740 1 -- 172.21.15.164:0/652586092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2a2e18b40 msgr2=0x55d2a2e1f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:00.498 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.487+0000 7f2b292d2740 1 --2- 172.21.15.164:0/652586092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2a2e18b40 0x55d2a2e1f070 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2b200ff120 tx=0x7f2b10012b80 comp rx=0 tx=0).stop 2024-09-19T08:47:00.498 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.487+0000 7f2b292d2740 1 -- 172.21.15.164:0/652586092 shutdown_connections 2024-09-19T08:47:00.498 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.487+0000 7f2b292d2740 1 --2- 172.21.15.164:0/652586092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a2e22370 0x55d2a2e1f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.487+0000 7f2b292d2740 1 --2- 172.21.15.164:0/652586092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2a2e18b40 0x55d2a2e1f070 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.487+0000 7f2b292d2740 1 --2- 172.21.15.164:0/652586092 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55d2a2e3a320 0x55d2a2e3c710 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.487+0000 7f2b292d2740 1 --2- 172.21.15.164:0/652586092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2a2e175d0 0x55d2a2e21dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.487+0000 7f2afffff640 0 client.25345 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:47:00.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.487+0000 7f2b292d2740 1 -- 172.21.15.164:0/652586092 >> 172.21.15.164:0/652586092 conn(0x55d2a2de0310 msgr2=0x55d2a2e199c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:00.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.487+0000 7f2b292d2740 1 -- 172.21.15.164:0/652586092 shutdown_connections 2024-09-19T08:47:00.499 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.487+0000 7f2b292d2740 1 -- 172.21.15.164:0/652586092 wait complete. 2024-09-19T08:47:00.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.487+0000 7f2b292d2740 20 client.25345 trim_cache size 0 max 16384 2024-09-19T08:47:00.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.512+0000 7f2b292d2740 1 Processor -- start 2024-09-19T08:47:00.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.513+0000 7f2b292d2740 1 -- start start 2024-09-19T08:47:00.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.513+0000 7f2b292d2740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2a2e175d0 0x55d2a2e1e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:00.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.513+0000 7f2b292d2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a2e18b40 0x55d2a2e20fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:00.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.513+0000 7f2b292d2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2a2e22370 0x55d2a2e1a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:00.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.513+0000 7f2b292d2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2006fc60 con 0x55d2a2e18b40 2024-09-19T08:47:00.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.513+0000 7f2b292d2740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2b200717a0 con 0x55d2a2e175d0 2024-09-19T08:47:00.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.513+0000 7f2b292d2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2b20070330 con 0x55d2a2e22370 2024-09-19T08:47:00.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.513+0000 7f2b17fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a2e18b40 0x55d2a2e20fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:00.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.513+0000 7f2b17fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a2e18b40 0x55d2a2e20fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59384/0 (socket says 172.21.15.164:59384) 2024-09-19T08:47:00.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.513+0000 7f2b17fff640 1 -- 172.21.15.164:0/1706061847 learned_addr learned my addr 172.21.15.164:0/1706061847 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:00.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.513+0000 7f2b24c95640 1 --2- 172.21.15.164:0/1706061847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2a2e175d0 0x55d2a2e1e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:00.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.513+0000 7f2b25496640 1 --2- 172.21.15.164:0/1706061847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2a2e22370 0x55d2a2e1a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:00.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.514+0000 7f2b24c95640 1 -- 172.21.15.164:0/1706061847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2a2e22370 msgr2=0x55d2a2e1a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:00.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.514+0000 7f2b24c95640 1 --2- 172.21.15.164:0/1706061847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2a2e22370 0x55d2a2e1a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.515 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.514+0000 7f2b24c95640 1 -- 172.21.15.164:0/1706061847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a2e18b40 msgr2=0x55d2a2e20fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:00.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.514+0000 7f2b24c95640 1 --2- 172.21.15.164:0/1706061847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a2e18b40 0x55d2a2e20fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.514+0000 7f2b24c95640 1 -- 172.21.15.164:0/1706061847 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2a2e1fb70 con 0x55d2a2e175d0 2024-09-19T08:47:00.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.514+0000 7f2b17fff640 1 --2- 172.21.15.164:0/1706061847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a2e18b40 0x55d2a2e20fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:47:00.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.514+0000 7f2b24c95640 1 --2- 172.21.15.164:0/1706061847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2a2e175d0 0x55d2a2e1e930 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c010920 tx=0x7f2b1c017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ca06b1cbd30e787 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:00.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.514+0000 7f2b067fc640 1 -- 172.21.15.164:0/1706061847 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c014070 con 0x55d2a2e175d0 2024-09-19T08:47:00.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.514+0000 7f2b067fc640 1 -- 172.21.15.164:0/1706061847 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b1c020a00 con 0x55d2a2e175d0 2024-09-19T08:47:00.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.514+0000 7f2b067fc640 1 -- 172.21.15.164:0/1706061847 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c005610 con 0x55d2a2e175d0 2024-09-19T08:47:00.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.515+0000 7f2b292d2740 1 -- 172.21.15.164:0/1706061847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2a2e175d0 msgr2=0x55d2a2e1e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:00.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.515+0000 7f2b292d2740 1 --2- 172.21.15.164:0/1706061847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2a2e175d0 0x55d2a2e1e930 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c010920 tx=0x7f2b1c017000 comp rx=0 tx=0).stop 2024-09-19T08:47:00.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.515+0000 7f2b292d2740 1 -- 172.21.15.164:0/1706061847 shutdown_connections 2024-09-19T08:47:00.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.515+0000 7f2b292d2740 1 --2- 172.21.15.164:0/1706061847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2a2e22370 0x55d2a2e1a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.515+0000 7f2b292d2740 1 --2- 172.21.15.164:0/1706061847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a2e18b40 0x55d2a2e20fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.515+0000 7f2b292d2740 1 --2- 172.21.15.164:0/1706061847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2a2e175d0 0x55d2a2e1e930 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.515+0000 7f2b292d2740 1 -- 172.21.15.164:0/1706061847 >> 172.21.15.164:0/1706061847 conn(0x55d2a2de0310 msgr2=0x55d2a2e208d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:00.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.515+0000 7f2b292d2740 1 -- 172.21.15.164:0/1706061847 shutdown_connections 2024-09-19T08:47:00.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.515+0000 7f2b292d2740 1 -- 172.21.15.164:0/1706061847 wait complete. 2024-09-19T08:47:00.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.515+0000 7f2b292d2740 1 -- start start 2024-09-19T08:47:00.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.516+0000 7f2b292d2740 1 Processor -- start 2024-09-19T08:47:00.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.516+0000 7f2b292d2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2a2e175d0 0x55d2a2e32d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:00.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.516+0000 7f2b292d2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a2e18b40 0x55d2a2e2e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:00.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.516+0000 7f2b292d2740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2a2e22370 0x55d2a2e2e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:00.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.516+0000 7f2b292d2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2b200710d0 con 0x55d2a2e18b40 2024-09-19T08:47:00.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.516+0000 7f2b292d2740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2b20070a00 con 0x55d2a2e22370 2024-09-19T08:47:00.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.516+0000 7f2b292d2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2006dc60 con 0x55d2a2e175d0 2024-09-19T08:47:00.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.516+0000 7f2b25496640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2a2e22370 0x55d2a2e2e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:00.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.516+0000 7f2b292d2740 10 client.0 _ll_register_callbacks cb 0x7f2b100097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:47:00.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.516+0000 7f2b25496640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2a2e22370 0x55d2a2e2e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:42122/0 (socket says 172.21.15.164:42122) 2024-09-19T08:47:00.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.516+0000 7f2b25496640 1 -- 172.21.15.164:0/472560889 learned_addr learned my addr 172.21.15.164:0/472560889 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:00.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.517+0000 7f2b24c95640 1 --2- 172.21.15.164:0/472560889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2a2e175d0 0x55d2a2e32d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:00.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.517+0000 7f2b17fff640 1 --2- 172.21.15.164:0/472560889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a2e18b40 0x55d2a2e2e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:00.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.517+0000 7f2b25496640 1 -- 172.21.15.164:0/472560889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2a2e175d0 msgr2=0x55d2a2e32d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:00.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.517+0000 7f2b25496640 1 --2- 172.21.15.164:0/472560889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2a2e175d0 0x55d2a2e32d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.517+0000 7f2b25496640 1 -- 172.21.15.164:0/472560889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a2e18b40 msgr2=0x55d2a2e2e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:00.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.517+0000 7f2b25496640 1 --2- 172.21.15.164:0/472560889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a2e18b40 0x55d2a2e2e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.517+0000 7f2b25496640 1 -- 172.21.15.164:0/472560889 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2a2e2eb40 con 0x55d2a2e22370 2024-09-19T08:47:00.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.517+0000 7f2b24c95640 1 --2- 172.21.15.164:0/472560889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2a2e175d0 0x55d2a2e32d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:00.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.517+0000 7f2b17fff640 1 --2- 172.21.15.164:0/472560889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a2e18b40 0x55d2a2e2e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:47:00.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.517+0000 7f2b25496640 1 --2- 172.21.15.164:0/472560889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2a2e22370 0x55d2a2e2e580 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2b180bb550 tx=0x7f2b180bb4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b342e5170c96534 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:00.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.517+0000 7f2afffff640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-19T08:47:00.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.517+0000 7f2afffff640 1 -- 172.21.15.164:0/472560889 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b180db070 con 0x55d2a2e22370 2024-09-19T08:47:00.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.517+0000 7f2afffff640 1 -- 172.21.15.164:0/472560889 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f2b18003d30 con 0x55d2a2e22370 2024-09-19T08:47:00.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.517+0000 7f2b292d2740 1 -- 172.21.15.164:0/472560889 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b1c020a00 con 0x55d2a2e22370 2024-09-19T08:47:00.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.517+0000 7f2afffff640 1 -- 172.21.15.164:0/472560889 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2b180bbcc0 con 0x55d2a2e22370 2024-09-19T08:47:00.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.517+0000 7f2b292d2740 10 client.25346 fetch_fsmap learned FSMap version 28 2024-09-19T08:47:00.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.517+0000 7f2b292d2740 1 -- 172.21.15.164:0/472560889 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d2a2ea4d60 con 0x55d2a2e22370 2024-09-19T08:47:00.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.518+0000 7f2afffff640 1 -- 172.21.15.164:0/472560889 <== mon.2 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b180c4230 con 0x55d2a2e22370 2024-09-19T08:47:00.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.518+0000 7f2afffff640 1 -- 172.21.15.164:0/472560889 <== mon.2 v2:172.21.15.170:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2b1803a030 con 0x55d2a2e22370 2024-09-19T08:47:00.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.518+0000 7f2b292d2740 10 client.25346 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:47:00.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.518+0000 7f2b292d2740 10 client.25346 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:47:00.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.518+0000 7f2b292d2740 20 client.25346 populate_metadata read hostname 'smithi164' 2024-09-19T08:47:00.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.518+0000 7f2b292d2740 1 --2- 172.21.15.164:0/472560889 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55d2a2ea5240 0x55d2a2ea7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:00.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.518+0000 7f2b292d2740 4 client.25346 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:47:00.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.518+0000 7f2b292d2740 1 -- 172.21.15.164:0/472560889 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f2b1c020a00 con 0x55d2a2ea5240 2024-09-19T08:47:00.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.518+0000 7f2b24c95640 1 --2- 172.21.15.164:0/472560889 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55d2a2ea5240 0x55d2a2ea7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:00.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.519+0000 7f2afffff640 10 client.25346 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:47:00.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.519+0000 7f2b24c95640 1 --2- 172.21.15.164:0/472560889 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55d2a2ea5240 0x55d2a2ea7630 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba1669b24293b12b server_cookie=453710e6af9223b9 in_seq=0 out_seq=0 2024-09-19T08:47:00.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.519+0000 7f2afffff640 1 -- 172.21.15.164:0/472560889 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f2b1c020a00 con 0x55d2a2ea5240 2024-09-19T08:47:00.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.519+0000 7f2afffff640 10 client.25346 handle_command_reply: tid=0 2024-09-19T08:47:00.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.519+0000 7f2b292d2740 1 client.25346 shutdown 2024-09-19T08:47:00.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.519+0000 7f2b292d2740 1 -- 172.21.15.164:0/472560889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2a2e22370 msgr2=0x55d2a2e2e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:00.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.519+0000 7f2b292d2740 1 --2- 172.21.15.164:0/472560889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2a2e22370 0x55d2a2e2e580 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2b180bb550 tx=0x7f2b180bb4c0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.520+0000 7f2b292d2740 1 -- 172.21.15.164:0/472560889 shutdown_connections 2024-09-19T08:47:00.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.520+0000 7f2b292d2740 1 --2- 172.21.15.164:0/472560889 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55d2a2ea5240 0x55d2a2ea7630 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.520+0000 7f2b292d2740 1 --2- 172.21.15.164:0/472560889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2a2e22370 0x55d2a2e2e580 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.520+0000 7f2afffff640 0 client.25346 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:47:00.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.520+0000 7f2b292d2740 1 --2- 172.21.15.164:0/472560889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2a2e18b40 0x55d2a2e2e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.520+0000 7f2b292d2740 1 --2- 172.21.15.164:0/472560889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2a2e175d0 0x55d2a2e32d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.520+0000 7f2b292d2740 1 -- 172.21.15.164:0/472560889 >> 172.21.15.164:0/472560889 conn(0x55d2a2de0310 msgr2=0x55d2a2e210f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:00.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.520+0000 7f2b292d2740 1 -- 172.21.15.164:0/472560889 shutdown_connections 2024-09-19T08:47:00.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.520+0000 7f2b292d2740 1 -- 172.21.15.164:0/472560889 wait complete. 2024-09-19T08:47:00.525 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:47:00.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.520+0000 7f2b292d2740 20 client.25346 trim_cache size 0 max 16384 2024-09-19T08:47:00.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.521+0000 7f2b26ef9640 1 -- 172.21.15.164:0/3725213578 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f2b080a3480 msgr2=0x7f2b080a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:00.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.521+0000 7f2b26ef9640 1 --2- 172.21.15.164:0/3725213578 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f2b080a3480 0x7f2b080a5940 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c005fd0 tx=0x7f2b1c009500 comp rx=0 tx=0).stop 2024-09-19T08:47:00.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.521+0000 7f2b26ef9640 1 -- 172.21.15.164:0/3725213578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b20106a70 msgr2=0x7f2b2019ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:00.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.521+0000 7f2b26ef9640 1 --2- 172.21.15.164:0/3725213578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b20106a70 0x7f2b2019ed90 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2b18009780 tx=0x7f2b18033d10 comp rx=0 tx=0).stop 2024-09-19T08:47:00.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.521+0000 7f2b26ef9640 1 -- 172.21.15.164:0/3725213578 shutdown_connections 2024-09-19T08:47:00.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.521+0000 7f2b26ef9640 1 --2- 172.21.15.164:0/3725213578 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f2b080a3480 0x7f2b080a5940 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.521+0000 7f2b26ef9640 1 --2- 172.21.15.164:0/3725213578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b20106a70 0x7f2b2019ed90 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.521+0000 7f2b26ef9640 1 --2- 172.21.15.164:0/3725213578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b20101410 0x7f2b2019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.521+0000 7f2b26ef9640 1 --2- 172.21.15.164:0/3725213578 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b20100a70 0x7f2b2019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.521+0000 7f2b26ef9640 1 -- 172.21.15.164:0/3725213578 >> 172.21.15.164:0/3725213578 conn(0x7f2b200fc7d0 msgr2=0x7f2b2006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:00.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.521+0000 7f2b26ef9640 1 -- 172.21.15.164:0/3725213578 shutdown_connections 2024-09-19T08:47:00.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:00.521+0000 7f2b26ef9640 1 -- 172.21.15.164:0/3725213578 wait complete. 2024-09-19T08:47:02.018 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:01 smithi164 ceph-mon[30928]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-19T08:47:02.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:01 smithi170 ceph-mon[35072]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-19T08:47:02.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:01 smithi173 ceph-mon[35306]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-19T08:47:02.328 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T08:47:02.329 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T08:47:02.619 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:47:03.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.140+0000 7f072322b640 1 -- 172.21.15.164:0/1817517013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f071c102df0 msgr2=0x7f071c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:03.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.140+0000 7f072322b640 1 --2- 172.21.15.164:0/1817517013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f071c102df0 0x7f071c1031d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0710009900 tx=0x7f071002a410 comp rx=0 tx=0).stop 2024-09-19T08:47:03.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.140+0000 7f072322b640 1 -- 172.21.15.164:0/1817517013 shutdown_connections 2024-09-19T08:47:03.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.140+0000 7f072322b640 1 --2- 172.21.15.164:0/1817517013 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f071c10c2f0 0x7f071c10e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.140+0000 7f072322b640 1 --2- 172.21.15.164:0/1817517013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f071c1037a0 0x7f071c10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.140+0000 7f072322b640 1 --2- 172.21.15.164:0/1817517013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f071c102df0 0x7f071c1031d0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.140+0000 7f072322b640 1 -- 172.21.15.164:0/1817517013 >> 172.21.15.164:0/1817517013 conn(0x7f071c0fc7d0 msgr2=0x7f071c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:03.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.140+0000 7f072322b640 1 -- 172.21.15.164:0/1817517013 shutdown_connections 2024-09-19T08:47:03.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.140+0000 7f072322b640 1 -- 172.21.15.164:0/1817517013 wait complete. 2024-09-19T08:47:03.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.141+0000 7f072322b640 1 Processor -- start 2024-09-19T08:47:03.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.141+0000 7f072322b640 1 -- start start 2024-09-19T08:47:03.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.141+0000 7f072322b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f071c102df0 0x7f071c073d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.141+0000 7f072322b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f071c1037a0 0x7f071c06bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.142+0000 7f072322b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f071c10c2f0 0x7f071c06c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.142+0000 7f0720fc7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f071c102df0 0x7f071c073d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.142+0000 7f0720fc7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f071c102df0 0x7f071c073d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59404/0 (socket says 172.21.15.164:59404) 2024-09-19T08:47:03.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.142+0000 7f0720fc7640 1 -- 172.21.15.164:0/4080419028 learned_addr learned my addr 172.21.15.164:0/4080419028 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:03.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.142+0000 7f072322b640 1 -- 172.21.15.164:0/4080419028 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f071c112110 con 0x7f071c102df0 2024-09-19T08:47:03.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.142+0000 7f072322b640 1 -- 172.21.15.164:0/4080419028 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f071c111f90 con 0x7f071c10c2f0 2024-09-19T08:47:03.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.142+0000 7f072322b640 1 -- 172.21.15.164:0/4080419028 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f071c112290 con 0x7f071c1037a0 2024-09-19T08:47:03.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.142+0000 7f071bfff640 1 --2- 172.21.15.164:0/4080419028 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f071c1037a0 0x7f071c06bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.142+0000 7f07217c8640 1 --2- 172.21.15.164:0/4080419028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f071c10c2f0 0x7f071c06c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.142+0000 7f0720fc7640 1 -- 172.21.15.164:0/4080419028 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f071c1037a0 msgr2=0x7f071c06bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:03.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.142+0000 7f0720fc7640 1 --2- 172.21.15.164:0/4080419028 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f071c1037a0 0x7f071c06bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.142+0000 7f0720fc7640 1 -- 172.21.15.164:0/4080419028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f071c10c2f0 msgr2=0x7f071c06c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:03.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.142+0000 7f0720fc7640 1 --2- 172.21.15.164:0/4080419028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f071c10c2f0 0x7f071c06c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.142+0000 7f0720fc7640 1 -- 172.21.15.164:0/4080419028 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f071c06ca40 con 0x7f071c102df0 2024-09-19T08:47:03.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.142+0000 7f07217c8640 1 --2- 172.21.15.164:0/4080419028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f071c10c2f0 0x7f071c06c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:47:03.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.142+0000 7f0720fc7640 1 --2- 172.21.15.164:0/4080419028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f071c102df0 0x7f071c073d10 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0710006010 tx=0x7f0710004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:03.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.143+0000 7f0719ffb640 1 -- 172.21.15.164:0/4080419028 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071003f030 con 0x7f071c102df0 2024-09-19T08:47:03.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.143+0000 7f072322b640 1 -- 172.21.15.164:0/4080419028 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f071c100980 con 0x7f071c102df0 2024-09-19T08:47:03.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.143+0000 7f0719ffb640 1 -- 172.21.15.164:0/4080419028 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0710004440 con 0x7f071c102df0 2024-09-19T08:47:03.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.143+0000 7f0719ffb640 1 -- 172.21.15.164:0/4080419028 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0710035070 con 0x7f071c102df0 2024-09-19T08:47:03.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.143+0000 7f072322b640 1 -- 172.21.15.164:0/4080419028 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f071c100eb0 con 0x7f071c102df0 2024-09-19T08:47:03.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.146+0000 7f0719ffb640 1 -- 172.21.15.164:0/4080419028 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7f0710042030 con 0x7f071c102df0 2024-09-19T08:47:03.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.146+0000 7f0725604740 1 Processor -- start 2024-09-19T08:47:03.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.147+0000 7f0719ffb640 1 --2- 172.21.15.164:0/4080419028 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f070c0a35d0 0x7f070c0a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.147+0000 7f0725604740 1 -- start start 2024-09-19T08:47:03.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.147+0000 7f0719ffb640 1 -- 172.21.15.164:0/4080419028 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f07100f0310 con 0x7f071c102df0 2024-09-19T08:47:03.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.148+0000 7f0725604740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597123d5370 0x5597123d5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.148+0000 7f0725604740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597123d5c90 0x5597123e0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.148+0000 7f0725604740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597123e0550 0x5597123e2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.148+0000 7f0725604740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559712332280 con 0x5597123d5c90 2024-09-19T08:47:03.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.148+0000 7f0725604740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5597122d9b10 con 0x5597123e0550 2024-09-19T08:47:03.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.148+0000 7f0725604740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55971232e240 con 0x5597123d5370 2024-09-19T08:47:03.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.148+0000 7f071bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597123d5c90 0x5597123e0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.148+0000 7f071bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597123d5c90 0x5597123e0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59418/0 (socket says 172.21.15.164:59418) 2024-09-19T08:47:03.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.148+0000 7f071bfff640 1 -- 172.21.15.164:0/1426295362 learned_addr learned my addr 172.21.15.164:0/1426295362 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:03.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.148+0000 7f0720fc7640 1 --2- 172.21.15.164:0/1426295362 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597123d5370 0x5597123d5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.148+0000 7f071bfff640 1 -- 172.21.15.164:0/1426295362 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597123d5370 msgr2=0x5597123d5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:03.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.148+0000 7f071bfff640 1 --2- 172.21.15.164:0/1426295362 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597123d5370 0x5597123d5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.148+0000 7f07217c8640 1 --2- 172.21.15.164:0/4080419028 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f070c0a35d0 0x7f070c0a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.148+0000 7f071bfff640 1 -- 172.21.15.164:0/1426295362 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597123e0550 msgr2=0x5597123e2940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:47:03.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.148+0000 7f071bfff640 1 --2- 172.21.15.164:0/1426295362 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597123e0550 0x5597123e2940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.148+0000 7f071bfff640 1 -- 172.21.15.164:0/1426295362 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597122e61f0 con 0x5597123d5c90 2024-09-19T08:47:03.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.148+0000 7f0720fc7640 1 --2- 172.21.15.164:0/1426295362 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597123d5370 0x5597123d5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:47:03.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.149+0000 7f071bfff640 1 --2- 172.21.15.164:0/1426295362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597123d5c90 0x5597123e0010 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0708009c20 tx=0x7f0708011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd11750735fa00a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:03.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.149+0000 7f0706ffd640 1 -- 172.21.15.164:0/1426295362 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0708004050 con 0x5597123d5c90 2024-09-19T08:47:03.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.149+0000 7f0706ffd640 1 -- 172.21.15.164:0/1426295362 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0708005500 con 0x5597123d5c90 2024-09-19T08:47:03.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.149+0000 7f0706ffd640 1 -- 172.21.15.164:0/1426295362 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07080035f0 con 0x5597123d5c90 2024-09-19T08:47:03.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.150+0000 7f0725604740 1 -- 172.21.15.164:0/1426295362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597123d5c90 msgr2=0x5597123e0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:03.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.150+0000 7f07217c8640 1 --2- 172.21.15.164:0/4080419028 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f070c0a35d0 0x7f070c0a5a90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x5597123d2c80 tx=0x7f0714009500 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:03.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.150+0000 7f0725604740 1 --2- 172.21.15.164:0/1426295362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597123d5c90 0x5597123e0010 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0708009c20 tx=0x7f0708011810 comp rx=0 tx=0).stop 2024-09-19T08:47:03.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.150+0000 7f0725604740 1 -- 172.21.15.164:0/1426295362 shutdown_connections 2024-09-19T08:47:03.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.150+0000 7f0725604740 1 --2- 172.21.15.164:0/1426295362 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597123e0550 0x5597123e2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.150+0000 7f0725604740 1 --2- 172.21.15.164:0/1426295362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597123d5c90 0x5597123e0010 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.150+0000 7f0725604740 1 --2- 172.21.15.164:0/1426295362 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597123d5370 0x5597123d5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.150+0000 7f0725604740 1 -- 172.21.15.164:0/1426295362 >> 172.21.15.164:0/1426295362 conn(0x5597123d1900 msgr2=0x5597123d1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:03.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.150+0000 7f0725604740 1 -- 172.21.15.164:0/1426295362 shutdown_connections 2024-09-19T08:47:03.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.150+0000 7f0725604740 1 -- 172.21.15.164:0/1426295362 wait complete. 2024-09-19T08:47:03.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.151+0000 7f0725604740 1 -- start start 2024-09-19T08:47:03.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.151+0000 7f0725604740 1 Processor -- start 2024-09-19T08:47:03.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.151+0000 7f0725604740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597123d5370 0x5597123dd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.151+0000 7f0725604740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597123d5c90 0x5597123dd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.151+0000 7f0725604740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597123e0550 0x5597123ddeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.152+0000 7f0725604740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5597123e3120 con 0x5597123d5c90 2024-09-19T08:47:03.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.152+0000 7f0725604740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5597122ecf10 con 0x5597123d5370 2024-09-19T08:47:03.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.152+0000 7f0725604740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5597123e32a0 con 0x5597123e0550 2024-09-19T08:47:03.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.152+0000 7f0725604740 10 client.0 _ll_register_callbacks cb 0x5597122b5a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:47:03.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.152+0000 7f071bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597123d5c90 0x5597123dd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.152+0000 7f0720fc7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597123d5370 0x5597123dd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.152+0000 7f071bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597123d5c90 0x5597123dd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59428/0 (socket says 172.21.15.164:59428) 2024-09-19T08:47:03.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.152+0000 7f071bfff640 1 -- 172.21.15.164:0/2181112432 learned_addr learned my addr 172.21.15.164:0/2181112432 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:03.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.152+0000 7f07217c8640 1 --2- 172.21.15.164:0/2181112432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597123e0550 0x5597123ddeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.152+0000 7f071bfff640 1 -- 172.21.15.164:0/2181112432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597123e0550 msgr2=0x5597123ddeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:03.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.152+0000 7f071bfff640 1 --2- 172.21.15.164:0/2181112432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597123e0550 0x5597123ddeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.152+0000 7f071bfff640 1 -- 172.21.15.164:0/2181112432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597123d5370 msgr2=0x5597123dd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:03.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.152+0000 7f071bfff640 1 --2- 172.21.15.164:0/2181112432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597123d5370 0x5597123dd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.152+0000 7f071bfff640 1 -- 172.21.15.164:0/2181112432 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597120a36a0 con 0x5597123d5c90 2024-09-19T08:47:03.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.152+0000 7f071bfff640 1 --2- 172.21.15.164:0/2181112432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597123d5c90 0x5597123dd970 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f07080050c0 tx=0x7f07080050f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0eec2269261396e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:03.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.152+0000 7f06fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:47:03.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.153+0000 7f06fbfff640 1 -- 172.21.15.164:0/2181112432 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0708029070 con 0x5597123d5c90 2024-09-19T08:47:03.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.153+0000 7f06fbfff640 1 -- 172.21.15.164:0/2181112432 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f0708004480 con 0x5597123d5c90 2024-09-19T08:47:03.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.153+0000 7f0725604740 1 -- 172.21.15.164:0/2181112432 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0708005500 con 0x5597123d5c90 2024-09-19T08:47:03.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.153+0000 7f06fbfff640 1 -- 172.21.15.164:0/2181112432 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0708035050 con 0x5597123d5c90 2024-09-19T08:47:03.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.153+0000 7f0725604740 10 client.15834 fetch_fsmap learned FSMap version 28 2024-09-19T08:47:03.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.153+0000 7f0725604740 1 -- 172.21.15.164:0/2181112432 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5597123eed20 con 0x5597123d5c90 2024-09-19T08:47:03.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.153+0000 7f06fbfff640 1 -- 172.21.15.164:0/2181112432 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0708005310 con 0x5597123d5c90 2024-09-19T08:47:03.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.153+0000 7f06fbfff640 1 -- 172.21.15.164:0/2181112432 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0708023660 con 0x5597123d5c90 2024-09-19T08:47:03.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.153+0000 7f0725604740 10 client.15834 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:47:03.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.153+0000 7f0725604740 10 client.15834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:47:03.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.153+0000 7f0725604740 20 client.15834 populate_metadata read hostname 'smithi164' 2024-09-19T08:47:03.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.154+0000 7f0725604740 1 --2- 172.21.15.164:0/2181112432 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5597123f0070 0x5597123f2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.154+0000 7f0725604740 4 client.15834 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:47:03.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.154+0000 7f0725604740 1 -- 172.21.15.164:0/2181112432 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0708005500 con 0x5597123f0070 2024-09-19T08:47:03.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.154+0000 7f0720fc7640 1 --2- 172.21.15.164:0/2181112432 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5597123f0070 0x5597123f2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.155+0000 7f06fbfff640 10 client.15834 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:47:03.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.155+0000 7f0720fc7640 1 --2- 172.21.15.164:0/2181112432 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5597123f0070 0x5597123f2460 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f88f86734b7339b server_cookie=6589a9f1cbc364b4 in_seq=0 out_seq=0 2024-09-19T08:47:03.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.155+0000 7f06fbfff640 1 -- 172.21.15.164:0/2181112432 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f0708005500 con 0x5597123f0070 2024-09-19T08:47:03.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.155+0000 7f06fbfff640 10 client.15834 handle_command_reply: tid=0 2024-09-19T08:47:03.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.155+0000 7f0725604740 1 client.15834 shutdown 2024-09-19T08:47:03.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.155+0000 7f0725604740 1 -- 172.21.15.164:0/2181112432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597123d5c90 msgr2=0x5597123dd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:03.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.155+0000 7f0725604740 1 --2- 172.21.15.164:0/2181112432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597123d5c90 0x5597123dd970 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f07080050c0 tx=0x7f07080050f0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.156+0000 7f0725604740 1 -- 172.21.15.164:0/2181112432 shutdown_connections 2024-09-19T08:47:03.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.156+0000 7f0725604740 1 --2- 172.21.15.164:0/2181112432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597123e0550 0x5597123ddeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.156+0000 7f0725604740 1 --2- 172.21.15.164:0/2181112432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597123d5c90 0x5597123dd970 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.156+0000 7f0725604740 1 --2- 172.21.15.164:0/2181112432 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5597123f0070 0x5597123f2460 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.156+0000 7f06fbfff640 0 client.15834 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:47:03.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.156+0000 7f0725604740 1 --2- 172.21.15.164:0/2181112432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597123d5370 0x5597123dd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.156+0000 7f0725604740 1 -- 172.21.15.164:0/2181112432 >> 172.21.15.164:0/2181112432 conn(0x5597123d1900 msgr2=0x5597123c8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:03.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.156+0000 7f0725604740 1 -- 172.21.15.164:0/2181112432 shutdown_connections 2024-09-19T08:47:03.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.156+0000 7f0725604740 1 -- 172.21.15.164:0/2181112432 wait complete. 2024-09-19T08:47:03.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.156+0000 7f0725604740 20 client.15834 trim_cache size 0 max 16384 2024-09-19T08:47:03.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.190+0000 7f0725604740 1 Processor -- start 2024-09-19T08:47:03.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.190+0000 7f0725604740 1 -- start start 2024-09-19T08:47:03.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.190+0000 7f0725604740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597123d5370 0x5597123ddd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.190+0000 7f0725604740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597123e0550 0x5597123f7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.190+0000 7f0725604740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597123f80c0 0x5597123da6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.190+0000 7f0725604740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f071c1b3d80 con 0x5597123f80c0 2024-09-19T08:47:03.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.190+0000 7f0725604740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f071c1b58c0 con 0x5597123d5370 2024-09-19T08:47:03.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.190+0000 7f0725604740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f071c1b4450 con 0x5597123e0550 2024-09-19T08:47:03.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.190+0000 7f07217c8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597123f80c0 0x5597123da6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.190+0000 7f07217c8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597123f80c0 0x5597123da6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59438/0 (socket says 172.21.15.164:59438) 2024-09-19T08:47:03.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.190+0000 7f07217c8640 1 -- 172.21.15.164:0/3464522396 learned_addr learned my addr 172.21.15.164:0/3464522396 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:03.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.190+0000 7f0720fc7640 1 --2- 172.21.15.164:0/3464522396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597123d5370 0x5597123ddd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.191+0000 7f071bfff640 1 --2- 172.21.15.164:0/3464522396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597123e0550 0x5597123f7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.191+0000 7f0720fc7640 1 -- 172.21.15.164:0/3464522396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597123e0550 msgr2=0x5597123f7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:03.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.191+0000 7f0720fc7640 1 --2- 172.21.15.164:0/3464522396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597123e0550 0x5597123f7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.191+0000 7f0720fc7640 1 -- 172.21.15.164:0/3464522396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597123f80c0 msgr2=0x5597123da6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:03.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.191+0000 7f0720fc7640 1 --2- 172.21.15.164:0/3464522396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597123f80c0 0x5597123da6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.191+0000 7f0720fc7640 1 -- 172.21.15.164:0/3464522396 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597123d1730 con 0x5597123d5370 2024-09-19T08:47:03.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.191+0000 7f071bfff640 1 --2- 172.21.15.164:0/3464522396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597123e0550 0x5597123f7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:47:03.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.191+0000 7f0720fc7640 1 --2- 172.21.15.164:0/3464522396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597123d5370 0x5597123ddd00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f07100ed8f0 tx=0x7f0710009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56be27fb616ea0c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:03.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.191+0000 7f07067fc640 1 -- 172.21.15.164:0/3464522396 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07100d9070 con 0x5597123d5370 2024-09-19T08:47:03.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.191+0000 7f07067fc640 1 -- 172.21.15.164:0/3464522396 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07100bc540 con 0x5597123d5370 2024-09-19T08:47:03.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.191+0000 7f07067fc640 1 -- 172.21.15.164:0/3464522396 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0710049040 con 0x5597123d5370 2024-09-19T08:47:03.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.192+0000 7f0725604740 1 -- 172.21.15.164:0/3464522396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597123d5370 msgr2=0x5597123ddd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:03.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.192+0000 7f0725604740 1 --2- 172.21.15.164:0/3464522396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597123d5370 0x5597123ddd00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f07100ed8f0 tx=0x7f0710009560 comp rx=0 tx=0).stop 2024-09-19T08:47:03.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.193+0000 7f0725604740 1 -- 172.21.15.164:0/3464522396 shutdown_connections 2024-09-19T08:47:03.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.193+0000 7f0725604740 1 --2- 172.21.15.164:0/3464522396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597123f80c0 0x5597123da6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.193+0000 7f0725604740 1 --2- 172.21.15.164:0/3464522396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597123e0550 0x5597123f7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.193+0000 7f0725604740 1 --2- 172.21.15.164:0/3464522396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597123d5370 0x5597123ddd00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.193+0000 7f0725604740 1 -- 172.21.15.164:0/3464522396 >> 172.21.15.164:0/3464522396 conn(0x5597123d1900 msgr2=0x5597123df3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:03.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.193+0000 7f0725604740 1 -- 172.21.15.164:0/3464522396 shutdown_connections 2024-09-19T08:47:03.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.193+0000 7f0725604740 1 -- 172.21.15.164:0/3464522396 wait complete. 2024-09-19T08:47:03.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.193+0000 7f0725604740 1 -- start start 2024-09-19T08:47:03.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.193+0000 7f0725604740 1 Processor -- start 2024-09-19T08:47:03.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.194+0000 7f0725604740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597123d5370 0x5597123e54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.194+0000 7f0725604740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597123e0550 0x5597123e5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.194+0000 7f0725604740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597123f80c0 0x5597123e5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.194+0000 7f0725604740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f071c1b51f0 con 0x5597123f80c0 2024-09-19T08:47:03.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.194+0000 7f0725604740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f071c1b4b20 con 0x5597123e0550 2024-09-19T08:47:03.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.194+0000 7f0725604740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f071c1b1d60 con 0x5597123d5370 2024-09-19T08:47:03.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.194+0000 7f071bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597123e0550 0x5597123e5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.194+0000 7f0720fc7640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597123d5370 0x5597123e54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.194+0000 7f071bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597123e0550 0x5597123e5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:42162/0 (socket says 172.21.15.164:42162) 2024-09-19T08:47:03.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.194+0000 7f071bfff640 1 -- 172.21.15.164:0/4278326962 learned_addr learned my addr 172.21.15.164:0/4278326962 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:03.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.194+0000 7f07217c8640 1 --2- 172.21.15.164:0/4278326962 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597123f80c0 0x5597123e5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.195+0000 7f0725604740 10 client.0 _ll_register_callbacks cb 0x7f0708009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:47:03.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.195+0000 7f071bfff640 1 -- 172.21.15.164:0/4278326962 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597123d5370 msgr2=0x5597123e54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:03.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.195+0000 7f071bfff640 1 --2- 172.21.15.164:0/4278326962 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597123d5370 0x5597123e54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.195+0000 7f071bfff640 1 -- 172.21.15.164:0/4278326962 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597123f80c0 msgr2=0x5597123e5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:03.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.195+0000 7f071bfff640 1 --2- 172.21.15.164:0/4278326962 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597123f80c0 0x5597123e5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.195+0000 7f071bfff640 1 -- 172.21.15.164:0/4278326962 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597123d2a90 con 0x5597123e0550 2024-09-19T08:47:03.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.195+0000 7f071bfff640 1 --2- 172.21.15.164:0/4278326962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597123e0550 0x5597123e5a00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0708019430 tx=0x7f0708004e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8a527bd8316d394 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:03.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.195+0000 7f06fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-19T08:47:03.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.195+0000 7f06fbfff640 1 -- 172.21.15.164:0/4278326962 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0708029040 con 0x5597123e0550 2024-09-19T08:47:03.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.195+0000 7f06fbfff640 1 -- 172.21.15.164:0/4278326962 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f0708023630 con 0x5597123e0550 2024-09-19T08:47:03.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.195+0000 7f0725604740 1 -- 172.21.15.164:0/4278326962 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07100bc540 con 0x5597123e0550 2024-09-19T08:47:03.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.196+0000 7f06fbfff640 1 -- 172.21.15.164:0/4278326962 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0708023830 con 0x5597123e0550 2024-09-19T08:47:03.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.196+0000 7f0725604740 10 client.25358 fetch_fsmap learned FSMap version 28 2024-09-19T08:47:03.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.196+0000 7f0725604740 1 -- 172.21.15.164:0/4278326962 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55971245bc30 con 0x5597123e0550 2024-09-19T08:47:03.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.196+0000 7f06fbfff640 1 -- 172.21.15.164:0/4278326962 <== mon.2 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0708032070 con 0x5597123e0550 2024-09-19T08:47:03.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.196+0000 7f06fbfff640 1 -- 172.21.15.164:0/4278326962 <== mon.2 v2:172.21.15.170:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0708040020 con 0x5597123e0550 2024-09-19T08:47:03.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.196+0000 7f0725604740 10 client.25358 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:47:03.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.196+0000 7f0725604740 10 client.25358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:47:03.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.196+0000 7f0725604740 20 client.25358 populate_metadata read hostname 'smithi164' 2024-09-19T08:47:03.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.196+0000 7f0725604740 1 --2- 172.21.15.164:0/4278326962 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55971245c1e0 0x55971245e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.196+0000 7f0725604740 4 client.25358 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T08:47:03.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.196+0000 7f0725604740 1 -- 172.21.15.164:0/4278326962 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f07100bc540 con 0x55971245c1e0 2024-09-19T08:47:03.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.196+0000 7f0720fc7640 1 --2- 172.21.15.164:0/4278326962 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55971245c1e0 0x55971245e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.197+0000 7f06fbfff640 10 client.25358 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:47:03.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.197+0000 7f0720fc7640 1 --2- 172.21.15.164:0/4278326962 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55971245c1e0 0x55971245e600 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef8c5af1911fa629 server_cookie=da05f333112ed5de in_seq=0 out_seq=0 2024-09-19T08:47:03.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.198+0000 7f06fbfff640 1 -- 172.21.15.164:0/4278326962 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f07100bc540 con 0x55971245c1e0 2024-09-19T08:47:03.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.198+0000 7f06fbfff640 10 client.25358 handle_command_reply: tid=0 2024-09-19T08:47:03.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.198+0000 7f0725604740 1 client.25358 shutdown 2024-09-19T08:47:03.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.198+0000 7f0725604740 1 -- 172.21.15.164:0/4278326962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597123e0550 msgr2=0x5597123e5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:03.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.198+0000 7f0725604740 1 --2- 172.21.15.164:0/4278326962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597123e0550 0x5597123e5a00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0708019430 tx=0x7f0708004e20 comp rx=0 tx=0).stop 2024-09-19T08:47:03.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.198+0000 7f0725604740 1 -- 172.21.15.164:0/4278326962 shutdown_connections 2024-09-19T08:47:03.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.198+0000 7f0725604740 1 --2- 172.21.15.164:0/4278326962 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597123f80c0 0x5597123e5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.198+0000 7f0725604740 1 --2- 172.21.15.164:0/4278326962 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55971245c1e0 0x55971245e600 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.198+0000 7f0725604740 1 --2- 172.21.15.164:0/4278326962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5597123e0550 0x5597123e5a00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.198+0000 7f0725604740 1 --2- 172.21.15.164:0/4278326962 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5597123d5370 0x5597123e54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.198+0000 7f0725604740 1 -- 172.21.15.164:0/4278326962 >> 172.21.15.164:0/4278326962 conn(0x5597123d1900 msgr2=0x5597123d6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:03.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.198+0000 7f06fbfff640 0 client.25358 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:47:03.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.199+0000 7f0725604740 1 -- 172.21.15.164:0/4278326962 shutdown_connections 2024-09-19T08:47:03.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.199+0000 7f0725604740 1 -- 172.21.15.164:0/4278326962 wait complete. 2024-09-19T08:47:03.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.199+0000 7f0725604740 20 client.25358 trim_cache size 0 max 16384 2024-09-19T08:47:03.205 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:47:03.206 INFO:teuthology.orchestra.run.smithi164.stdout: "return_code": 0, 2024-09-19T08:47:03.206 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_tag": "4617d585-2e46-40d5-9b74-875f518b408f", 2024-09-19T08:47:03.206 INFO:teuthology.orchestra.run.smithi164.stdout: "mode": "asynchronous" 2024-09-19T08:47:03.206 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:47:03.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.200+0000 7f072322b640 1 -- 172.21.15.164:0/4080419028 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f070c0a35d0 msgr2=0x7f070c0a5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:03.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.200+0000 7f072322b640 1 --2- 172.21.15.164:0/4080419028 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f070c0a35d0 0x7f070c0a5a90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x5597123d2c80 tx=0x7f0714009500 comp rx=0 tx=0).stop 2024-09-19T08:47:03.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.200+0000 7f072322b640 1 -- 172.21.15.164:0/4080419028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f071c102df0 msgr2=0x7f071c073d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:03.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.200+0000 7f072322b640 1 --2- 172.21.15.164:0/4080419028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f071c102df0 0x7f071c073d10 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0710006010 tx=0x7f0710004040 comp rx=0 tx=0).stop 2024-09-19T08:47:03.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.201+0000 7f072322b640 1 -- 172.21.15.164:0/4080419028 shutdown_connections 2024-09-19T08:47:03.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.201+0000 7f072322b640 1 --2- 172.21.15.164:0/4080419028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f071c10c2f0 0x7f071c06c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.201+0000 7f072322b640 1 --2- 172.21.15.164:0/4080419028 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f071c1037a0 0x7f071c06bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.201+0000 7f072322b640 1 --2- 172.21.15.164:0/4080419028 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f070c0a35d0 0x7f070c0a5a90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.201+0000 7f072322b640 1 --2- 172.21.15.164:0/4080419028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f071c102df0 0x7f071c073d10 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.201+0000 7f072322b640 1 -- 172.21.15.164:0/4080419028 >> 172.21.15.164:0/4080419028 conn(0x7f071c0fc7d0 msgr2=0x7f071c0fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:03.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.201+0000 7f072322b640 1 -- 172.21.15.164:0/4080419028 shutdown_connections 2024-09-19T08:47:03.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:03.201+0000 7f072322b640 1 -- 172.21.15.164:0/4080419028 wait complete. 2024-09-19T08:47:04.018 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:03 smithi164 ceph-mon[30928]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:47:04.019 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:47:04.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:03 smithi170 ceph-mon[35072]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:47:04.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:03 smithi173 ceph-mon[35306]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:47:04.291 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:47:04.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.786+0000 7f8fa87a1640 1 -- 172.21.15.164:0/2138175324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fa0106a70 msgr2=0x7f8fa0106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:04.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.786+0000 7f8fa87a1640 1 --2- 172.21.15.164:0/2138175324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fa0106a70 0x7f8fa0106e50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c009930 tx=0x7f8f9c02a420 comp rx=0 tx=0).stop 2024-09-19T08:47:04.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.786+0000 7f8fa87a1640 1 -- 172.21.15.164:0/2138175324 shutdown_connections 2024-09-19T08:47:04.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.786+0000 7f8fa87a1640 1 --2- 172.21.15.164:0/2138175324 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8fa0101410 0x7f8fa010d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.786+0000 7f8fa87a1640 1 --2- 172.21.15.164:0/2138175324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fa0100a70 0x7f8fa0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.786+0000 7f8fa87a1640 1 --2- 172.21.15.164:0/2138175324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fa0106a70 0x7f8fa0106e50 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.786+0000 7f8fa87a1640 1 -- 172.21.15.164:0/2138175324 >> 172.21.15.164:0/2138175324 conn(0x7f8fa00fc790 msgr2=0x7f8fa00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:04.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.786+0000 7f8fa87a1640 1 -- 172.21.15.164:0/2138175324 shutdown_connections 2024-09-19T08:47:04.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.787+0000 7f8fa87a1640 1 -- 172.21.15.164:0/2138175324 wait complete. 2024-09-19T08:47:04.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.787+0000 7f8fa87a1640 1 Processor -- start 2024-09-19T08:47:04.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.788+0000 7f8fa87a1640 1 -- start start 2024-09-19T08:47:04.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.788+0000 7f8fa87a1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fa0100a70 0x7f8fa01992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:04.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.788+0000 7f8fa87a1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8fa0101410 0x7f8fa0199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:04.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.788+0000 7f8fa87a1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fa0106a70 0x7f8fa01a4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:04.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.788+0000 7f8fa87a1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa0110e70 con 0x7f8fa0106a70 2024-09-19T08:47:04.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.788+0000 7f8fa87a1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa0110cf0 con 0x7f8fa0100a70 2024-09-19T08:47:04.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.788+0000 7f8fa87a1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa0110ff0 con 0x7f8fa0101410 2024-09-19T08:47:04.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.788+0000 7f8fa6d3e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fa0106a70 0x7f8fa01a4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:04.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.788+0000 7f8fa6d3e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fa0106a70 0x7f8fa01a4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45364/0 (socket says 172.21.15.164:45364) 2024-09-19T08:47:04.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.788+0000 7f8fa6d3e640 1 -- 172.21.15.164:0/620259889 learned_addr learned my addr 172.21.15.164:0/620259889 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:04.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.788+0000 7f8fa5d3c640 1 --2- 172.21.15.164:0/620259889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8fa0101410 0x7f8fa0199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:04.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.788+0000 7f8fa6d3e640 1 -- 172.21.15.164:0/620259889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8fa0101410 msgr2=0x7f8fa0199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:04.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.788+0000 7f8fa6d3e640 1 --2- 172.21.15.164:0/620259889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8fa0101410 0x7f8fa0199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.788+0000 7f8fa6d3e640 1 -- 172.21.15.164:0/620259889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fa0100a70 msgr2=0x7f8fa01992c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T08:47:04.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.788+0000 7f8fa6d3e640 1 --2- 172.21.15.164:0/620259889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fa0100a70 0x7f8fa01992c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.788+0000 7f8fa6d3e640 1 -- 172.21.15.164:0/620259889 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fa01a4880 con 0x7f8fa0106a70 2024-09-19T08:47:04.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.789+0000 7f8fa6d3e640 1 --2- 172.21.15.164:0/620259889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fa0106a70 0x7f8fa01a4340 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f8f9800eb10 tx=0x7f8f9800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:04.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.789+0000 7f8f977fe640 1 -- 172.21.15.164:0/620259889 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f98004b20 con 0x7f8fa0106a70 2024-09-19T08:47:04.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.789+0000 7f8fa87a1640 1 -- 172.21.15.164:0/620259889 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fa01a4a50 con 0x7f8fa0106a70 2024-09-19T08:47:04.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.789+0000 7f8f977fe640 1 -- 172.21.15.164:0/620259889 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f98004cc0 con 0x7f8fa0106a70 2024-09-19T08:47:04.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.789+0000 7f8fa87a1640 1 -- 172.21.15.164:0/620259889 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fa0197410 con 0x7f8fa0106a70 2024-09-19T08:47:04.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.789+0000 7f8f977fe640 1 -- 172.21.15.164:0/620259889 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f98010760 con 0x7f8fa0106a70 2024-09-19T08:47:04.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.792+0000 7f8faab7a740 1 Processor -- start 2024-09-19T08:47:04.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.792+0000 7f8f977fe640 1 -- 172.21.15.164:0/620259889 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7f8f98010900 con 0x7f8fa0106a70 2024-09-19T08:47:04.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.792+0000 7f8faab7a740 1 -- start start 2024-09-19T08:47:04.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.793+0000 7f8faab7a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556c8a8244a0 0x556c8a824880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:04.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.793+0000 7f8faab7a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c8a824dc0 0x556c8a82f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:04.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.793+0000 7f8faab7a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c8a82f680 0x556c8a831a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:04.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.793+0000 7f8faab7a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556c8a78a2e0 con 0x556c8a824dc0 2024-09-19T08:47:04.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.793+0000 7f8faab7a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556c8a731ce0 con 0x556c8a82f680 2024-09-19T08:47:04.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.793+0000 7f8faab7a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556c8a7862a0 con 0x556c8a8244a0 2024-09-19T08:47:04.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.793+0000 7f8fa653d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c8a824dc0 0x556c8a82f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:04.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.793+0000 7f8fa653d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c8a824dc0 0x556c8a82f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45380/0 (socket says 172.21.15.164:45380) 2024-09-19T08:47:04.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.793+0000 7f8fa653d640 1 -- 172.21.15.164:0/1533402631 learned_addr learned my addr 172.21.15.164:0/1533402631 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:04.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.793+0000 7f8f977fe640 1 --2- 172.21.15.164:0/620259889 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f8f8c0a3550 0x7f8f8c0a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:04.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.793+0000 7f8fa5d3c640 1 --2- 172.21.15.164:0/1533402631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c8a82f680 0x556c8a831a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:04.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.793+0000 7f8fa653d640 1 -- 172.21.15.164:0/1533402631 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556c8a8244a0 msgr2=0x556c8a824880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:04.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.793+0000 7f8fa5d3c640 1 --2- 172.21.15.164:0/1533402631 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556c8a8244a0 0x556c8a824880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:04.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.793+0000 7f8f977fe640 1 -- 172.21.15.164:0/620259889 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f8f980d0610 con 0x7f8fa0106a70 2024-09-19T08:47:04.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.793+0000 7f8fa6d3e640 1 --2- 172.21.15.164:0/620259889 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f8f8c0a3550 0x7f8f8c0a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:04.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.794+0000 7f8fa653d640 1 --2- 172.21.15.164:0/1533402631 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556c8a8244a0 0x556c8a824880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.794+0000 7f8fa653d640 1 -- 172.21.15.164:0/1533402631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c8a82f680 msgr2=0x556c8a831a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:04.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.794+0000 7f8fa653d640 1 --2- 172.21.15.164:0/1533402631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c8a82f680 0x556c8a831a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.794+0000 7f8fa653d640 1 -- 172.21.15.164:0/1533402631 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c8a716440 con 0x556c8a824dc0 2024-09-19T08:47:04.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.794+0000 7f8fa5d3c640 1 --2- 172.21.15.164:0/1533402631 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556c8a8244a0 0x556c8a824880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:04.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.794+0000 7f8fa653d640 1 --2- 172.21.15.164:0/1533402631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c8a824dc0 0x556c8a82f140 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c006010 tx=0x7f8f9c031510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f82ee4b9bcc353 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:04.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.794+0000 7f8f94ff9640 1 -- 172.21.15.164:0/1533402631 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c03e070 con 0x556c8a824dc0 2024-09-19T08:47:04.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.794+0000 7f8f94ff9640 1 -- 172.21.15.164:0/1533402631 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f9c031870 con 0x556c8a824dc0 2024-09-19T08:47:04.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.794+0000 7f8f94ff9640 1 -- 172.21.15.164:0/1533402631 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c035040 con 0x556c8a824dc0 2024-09-19T08:47:04.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.795+0000 7f8faab7a740 1 -- 172.21.15.164:0/1533402631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c8a824dc0 msgr2=0x556c8a82f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:04.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.795+0000 7f8faab7a740 1 --2- 172.21.15.164:0/1533402631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c8a824dc0 0x556c8a82f140 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c006010 tx=0x7f8f9c031510 comp rx=0 tx=0).stop 2024-09-19T08:47:04.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.795+0000 7f8fa6d3e640 1 --2- 172.21.15.164:0/620259889 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f8f8c0a3550 0x7f8f8c0a5a10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8f980cdcb0 tx=0x7f8f98000f40 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:04.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.795+0000 7f8faab7a740 1 -- 172.21.15.164:0/1533402631 shutdown_connections 2024-09-19T08:47:04.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.795+0000 7f8faab7a740 1 --2- 172.21.15.164:0/1533402631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c8a82f680 0x556c8a831a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.795+0000 7f8faab7a740 1 --2- 172.21.15.164:0/1533402631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c8a824dc0 0x556c8a82f140 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.795+0000 7f8faab7a740 1 --2- 172.21.15.164:0/1533402631 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556c8a8244a0 0x556c8a824880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.795+0000 7f8faab7a740 1 -- 172.21.15.164:0/1533402631 >> 172.21.15.164:0/1533402631 conn(0x556c8a820a30 msgr2=0x556c8a820e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:04.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.796+0000 7f8faab7a740 1 -- 172.21.15.164:0/1533402631 shutdown_connections 2024-09-19T08:47:04.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.796+0000 7f8faab7a740 1 -- 172.21.15.164:0/1533402631 wait complete. 2024-09-19T08:47:04.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.796+0000 7f8faab7a740 1 -- start start 2024-09-19T08:47:04.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.796+0000 7f8faab7a740 1 Processor -- start 2024-09-19T08:47:04.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.797+0000 7f8faab7a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556c8a8244a0 0x556c8a82ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:04.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.797+0000 7f8faab7a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c8a824dc0 0x556c8a82c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:04.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.797+0000 7f8faab7a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c8a82f680 0x556c8a82c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:04.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.797+0000 7f8fa5d3c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556c8a8244a0 0x556c8a82ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:04.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.797+0000 7f8fa5d3c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556c8a8244a0 0x556c8a82ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:46172/0 (socket says 172.21.15.164:46172) 2024-09-19T08:47:04.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.797+0000 7f8fa5d3c640 1 -- 172.21.15.164:0/1112041592 learned_addr learned my addr 172.21.15.164:0/1112041592 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:04.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.797+0000 7f8fa653d640 1 --2- 172.21.15.164:0/1112041592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c8a824dc0 0x556c8a82c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:04.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.797+0000 7f8faab7a740 1 -- 172.21.15.164:0/1112041592 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556c8a832250 con 0x556c8a82f680 2024-09-19T08:47:04.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.797+0000 7f8faab7a740 1 -- 172.21.15.164:0/1112041592 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556c8a745050 con 0x556c8a824dc0 2024-09-19T08:47:04.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.797+0000 7f8faab7a740 1 -- 172.21.15.164:0/1112041592 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556c8a8323d0 con 0x556c8a8244a0 2024-09-19T08:47:04.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.797+0000 7f8fa5d3c640 1 --2- 172.21.15.164:0/1112041592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c8a82f680 0x556c8a82c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:04.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.797+0000 7f8fa653d640 1 -- 172.21.15.164:0/1112041592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556c8a8244a0 msgr2=0x556c8a82ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:04.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.797+0000 7f8fa653d640 1 --2- 172.21.15.164:0/1112041592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556c8a8244a0 0x556c8a82ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.797+0000 7f8fa653d640 1 -- 172.21.15.164:0/1112041592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c8a82f680 msgr2=0x556c8a82c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:04.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.798+0000 7f8fa653d640 1 --2- 172.21.15.164:0/1112041592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c8a82f680 0x556c8a82c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.798+0000 7f8faab7a740 10 client.0 _ll_register_callbacks cb 0x556c8a785160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:47:04.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.798+0000 7f8fa653d640 1 -- 172.21.15.164:0/1112041592 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c8a4fb3c0 con 0x556c8a824dc0 2024-09-19T08:47:04.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.798+0000 7f8fa5d3c640 1 --2- 172.21.15.164:0/1112041592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c8a82f680 0x556c8a82c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:47:04.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.798+0000 7f8faab7a740 1 -- 172.21.15.164:0/1112041592 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f9c031870 con 0x556c8a824dc0 2024-09-19T08:47:04.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.798+0000 7f8fa653d640 1 --2- 172.21.15.164:0/1112041592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c8a824dc0 0x556c8a82c1d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c0096c0 tx=0x7f8f9c005280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=288040ff5655f38d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:04.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.798+0000 7f8f89ffb640 10 client.25367 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-19T08:47:04.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.798+0000 7f8f89ffb640 1 -- 172.21.15.164:0/1112041592 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c04b020 con 0x556c8a824dc0 2024-09-19T08:47:04.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.798+0000 7f8f89ffb640 1 -- 172.21.15.164:0/1112041592 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f8f9c004250 con 0x556c8a824dc0 2024-09-19T08:47:04.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.798+0000 7f8f89ffb640 1 -- 172.21.15.164:0/1112041592 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8f9c03e070 con 0x556c8a824dc0 2024-09-19T08:47:04.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.798+0000 7f8faab7a740 10 client.25367 fetch_fsmap learned FSMap version 28 2024-09-19T08:47:04.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.798+0000 7f8faab7a740 1 -- 172.21.15.164:0/1112041592 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({fsmap=28}) -- 0x556c8a83d990 con 0x556c8a824dc0 2024-09-19T08:47:04.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.799+0000 7f8f89ffb640 1 -- 172.21.15.164:0/1112041592 <== mon.2 v2:172.21.15.170:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8f9c0046c0 con 0x556c8a824dc0 2024-09-19T08:47:04.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.799+0000 7f8faab7a740 10 client.25367 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:47:04.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.799+0000 7f8faab7a740 10 client.25367 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:47:04.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.799+0000 7f8faab7a740 20 client.25367 populate_metadata read hostname 'smithi164' 2024-09-19T08:47:04.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.799+0000 7f8faab7a740 1 --2- 172.21.15.164:0/1112041592 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556c8a83ece0 0x556c8a8410d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:04.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.799+0000 7f8faab7a740 4 client.25367 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:47:04.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.799+0000 7f8faab7a740 1 -- 172.21.15.164:0/1112041592 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c8a797850 con 0x556c8a83ece0 2024-09-19T08:47:04.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.799+0000 7f8fa5d3c640 1 --2- 172.21.15.164:0/1112041592 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556c8a83ece0 0x556c8a8410d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:04.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.800+0000 7f8f89ffb640 10 client.25367 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:47:04.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.800+0000 7f8fa5d3c640 1 --2- 172.21.15.164:0/1112041592 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556c8a83ece0 0x556c8a8410d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3469ace73a436de server_cookie=103a1bcded06c5d8 in_seq=0 out_seq=0 2024-09-19T08:47:04.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.800+0000 7f8f89ffb640 1 -- 172.21.15.164:0/1112041592 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556c8a797850 con 0x556c8a83ece0 2024-09-19T08:47:04.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.800+0000 7f8f89ffb640 10 client.25367 handle_command_reply: tid=0 2024-09-19T08:47:04.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.800+0000 7f8faab7a740 1 client.25367 shutdown 2024-09-19T08:47:04.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.800+0000 7f8faab7a740 1 -- 172.21.15.164:0/1112041592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c8a824dc0 msgr2=0x556c8a82c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:04.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.800+0000 7f8faab7a740 1 --2- 172.21.15.164:0/1112041592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c8a824dc0 0x556c8a82c1d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c0096c0 tx=0x7f8f9c005280 comp rx=0 tx=0).stop 2024-09-19T08:47:04.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.800+0000 7f8faab7a740 1 -- 172.21.15.164:0/1112041592 shutdown_connections 2024-09-19T08:47:04.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.800+0000 7f8faab7a740 1 --2- 172.21.15.164:0/1112041592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c8a82f680 0x556c8a82c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.800+0000 7f8faab7a740 1 --2- 172.21.15.164:0/1112041592 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556c8a83ece0 0x556c8a8410d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.801+0000 7f8faab7a740 1 --2- 172.21.15.164:0/1112041592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c8a824dc0 0x556c8a82c1d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.801+0000 7f8faab7a740 1 --2- 172.21.15.164:0/1112041592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556c8a8244a0 0x556c8a82ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.801+0000 7f8f89ffb640 0 client.25367 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:47:04.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.801+0000 7f8faab7a740 1 -- 172.21.15.164:0/1112041592 >> 172.21.15.164:0/1112041592 conn(0x556c8a820a30 msgr2=0x556c8a826710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:04.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.801+0000 7f8faab7a740 1 -- 172.21.15.164:0/1112041592 shutdown_connections 2024-09-19T08:47:04.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.801+0000 7f8faab7a740 1 -- 172.21.15.164:0/1112041592 wait complete. 2024-09-19T08:47:04.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.801+0000 7f8faab7a740 20 client.25367 trim_cache size 0 max 16384 2024-09-19T08:47:04.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.834+0000 7f8faab7a740 1 Processor -- start 2024-09-19T08:47:04.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.834+0000 7f8faab7a740 1 -- start start 2024-09-19T08:47:04.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.834+0000 7f8faab7a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c8a8244a0 0x556c8a829430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:04.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.834+0000 7f8faab7a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556c8a824dc0 0x556c8a829970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:04.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.834+0000 7f8faab7a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c8a82f680 0x556c8a81f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:04.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.834+0000 7f8faab7a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa01af920 con 0x556c8a8244a0 2024-09-19T08:47:04.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.834+0000 7f8faab7a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa01b1460 con 0x556c8a82f680 2024-09-19T08:47:04.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.834+0000 7f8faab7a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa01afff0 con 0x556c8a824dc0 2024-09-19T08:47:04.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.834+0000 7f8fa5d3c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c8a8244a0 0x556c8a829430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:04.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.834+0000 7f8fa5d3c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c8a8244a0 0x556c8a829430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45394/0 (socket says 172.21.15.164:45394) 2024-09-19T08:47:04.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.834+0000 7f8fa5d3c640 1 -- 172.21.15.164:0/2326493285 learned_addr learned my addr 172.21.15.164:0/2326493285 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:04.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.835+0000 7f8fa653d640 1 --2- 172.21.15.164:0/2326493285 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556c8a824dc0 0x556c8a829970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:04.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.835+0000 7f8fa5d3c640 1 --2- 172.21.15.164:0/2326493285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c8a82f680 0x556c8a81f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:04.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.835+0000 7f8fa5d3c640 1 -- 172.21.15.164:0/2326493285 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556c8a824dc0 msgr2=0x556c8a829970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:04.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.835+0000 7f8fa5d3c640 1 --2- 172.21.15.164:0/2326493285 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556c8a824dc0 0x556c8a829970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.835+0000 7f8fa5d3c640 1 -- 172.21.15.164:0/2326493285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c8a82f680 msgr2=0x556c8a81f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:04.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.835+0000 7f8fa5d3c640 1 --2- 172.21.15.164:0/2326493285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c8a82f680 0x556c8a81f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.835+0000 7f8fa5d3c640 1 -- 172.21.15.164:0/2326493285 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c8a82a720 con 0x556c8a8244a0 2024-09-19T08:47:04.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.835+0000 7f8fa5d3c640 1 --2- 172.21.15.164:0/2326493285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c8a8244a0 0x556c8a829430 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8f9000bbc0 tx=0x7f8f90004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c79f454637644b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:04.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.835+0000 7f8f8bfff640 1 -- 172.21.15.164:0/2326493285 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f90034030 con 0x556c8a8244a0 2024-09-19T08:47:04.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.836+0000 7f8f8bfff640 1 -- 172.21.15.164:0/2326493285 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f90021890 con 0x556c8a8244a0 2024-09-19T08:47:04.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.836+0000 7f8f8bfff640 1 -- 172.21.15.164:0/2326493285 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f90025040 con 0x556c8a8244a0 2024-09-19T08:47:04.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.836+0000 7f8faab7a740 1 -- 172.21.15.164:0/2326493285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c8a8244a0 msgr2=0x556c8a829430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:04.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.836+0000 7f8faab7a740 1 --2- 172.21.15.164:0/2326493285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c8a8244a0 0x556c8a829430 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8f9000bbc0 tx=0x7f8f90004730 comp rx=0 tx=0).stop 2024-09-19T08:47:04.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.837+0000 7f8faab7a740 1 -- 172.21.15.164:0/2326493285 shutdown_connections 2024-09-19T08:47:04.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.837+0000 7f8faab7a740 1 --2- 172.21.15.164:0/2326493285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c8a82f680 0x556c8a81f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.837+0000 7f8faab7a740 1 --2- 172.21.15.164:0/2326493285 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556c8a824dc0 0x556c8a829970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.837+0000 7f8faab7a740 1 --2- 172.21.15.164:0/2326493285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c8a8244a0 0x556c8a829430 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.837+0000 7f8faab7a740 1 -- 172.21.15.164:0/2326493285 >> 172.21.15.164:0/2326493285 conn(0x556c8a820a30 msgr2=0x556c8a82b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:04.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.837+0000 7f8faab7a740 1 -- 172.21.15.164:0/2326493285 shutdown_connections 2024-09-19T08:47:04.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.837+0000 7f8faab7a740 1 -- 172.21.15.164:0/2326493285 wait complete. 2024-09-19T08:47:04.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.837+0000 7f8faab7a740 1 -- start start 2024-09-19T08:47:04.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.837+0000 7f8faab7a740 1 Processor -- start 2024-09-19T08:47:04.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.838+0000 7f8faab7a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556c8a8244a0 0x556c8a843500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:04.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.838+0000 7f8faab7a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c8a824dc0 0x556c8a843a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:04.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.838+0000 7f8faab7a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c8a82f680 0x556c8a8a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:04.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.838+0000 7f8fa5d3c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c8a82f680 0x556c8a8a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:04.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.838+0000 7f8fa5d3c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c8a82f680 0x556c8a8a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45404/0 (socket says 172.21.15.164:45404) 2024-09-19T08:47:04.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.838+0000 7f8fa5d3c640 1 -- 172.21.15.164:0/678398483 learned_addr learned my addr 172.21.15.164:0/678398483 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:04.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.838+0000 7f8faab7a740 1 -- 172.21.15.164:0/678398483 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa01b0d90 con 0x556c8a82f680 2024-09-19T08:47:04.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.838+0000 7f8faab7a740 1 -- 172.21.15.164:0/678398483 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa01b06c0 con 0x556c8a824dc0 2024-09-19T08:47:04.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.838+0000 7f8faab7a740 1 -- 172.21.15.164:0/678398483 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa01ad940 con 0x556c8a8244a0 2024-09-19T08:47:04.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.838+0000 7f8fa653d640 1 --2- 172.21.15.164:0/678398483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c8a824dc0 0x556c8a843a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:04.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.839+0000 7f8fa5d3c640 1 -- 172.21.15.164:0/678398483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556c8a8244a0 msgr2=0x556c8a843500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:04.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.839+0000 7f8fa5d3c640 1 --2- 172.21.15.164:0/678398483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556c8a8244a0 0x556c8a843500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.839+0000 7f8fa5d3c640 1 -- 172.21.15.164:0/678398483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c8a824dc0 msgr2=0x556c8a843a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:04.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.839+0000 7f8fa5d3c640 1 --2- 172.21.15.164:0/678398483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c8a824dc0 0x556c8a843a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.839+0000 7f8fa5d3c640 1 -- 172.21.15.164:0/678398483 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c8a8a0c80 con 0x556c8a82f680 2024-09-19T08:47:04.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.839+0000 7f8faab7a740 10 client.0 _ll_register_callbacks cb 0x7f8f9c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:47:04.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.839+0000 7f8faab7a740 1 -- 172.21.15.164:0/678398483 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f90021890 con 0x556c8a82f680 2024-09-19T08:47:04.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.839+0000 7f8fa5d3c640 1 --2- 172.21.15.164:0/678398483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c8a82f680 0x556c8a8a05e0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8f9000b7f0 tx=0x7f8f90005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8f8ab7537dd7d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:04.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.839+0000 7f8f89ffb640 10 client.15858 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:47:04.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.839+0000 7f8f89ffb640 1 -- 172.21.15.164:0/678398483 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9003d030 con 0x556c8a82f680 2024-09-19T08:47:04.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.839+0000 7f8f89ffb640 1 -- 172.21.15.164:0/678398483 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f8f90021a40 con 0x556c8a82f680 2024-09-19T08:47:04.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.840+0000 7f8f89ffb640 1 -- 172.21.15.164:0/678398483 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8f90034040 con 0x556c8a82f680 2024-09-19T08:47:04.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.840+0000 7f8faab7a740 10 client.15858 fetch_fsmap learned FSMap version 28 2024-09-19T08:47:04.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.840+0000 7f8faab7a740 1 -- 172.21.15.164:0/678398483 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({fsmap=28}) -- 0x556c8a8a8f50 con 0x556c8a82f680 2024-09-19T08:47:04.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.840+0000 7f8f89ffb640 1 -- 172.21.15.164:0/678398483 <== mon.0 v2:172.21.15.164:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8f90005390 con 0x556c8a82f680 2024-09-19T08:47:04.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.840+0000 7f8faab7a740 10 client.15858 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:47:04.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.840+0000 7f8faab7a740 10 client.15858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:47:04.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.840+0000 7f8faab7a740 20 client.15858 populate_metadata read hostname 'smithi164' 2024-09-19T08:47:04.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.840+0000 7f8faab7a740 1 --2- 172.21.15.164:0/678398483 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556c8a8a9560 0x556c8a8ab9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:04.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.840+0000 7f8faab7a740 4 client.15858 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:47:04.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.840+0000 7f8faab7a740 1 -- 172.21.15.164:0/678398483 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556c8a791d60 con 0x556c8a8a9560 2024-09-19T08:47:04.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.840+0000 7f8fa653d640 1 --2- 172.21.15.164:0/678398483 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556c8a8a9560 0x556c8a8ab9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:04.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.841+0000 7f8f89ffb640 10 client.15858 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:47:04.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.841+0000 7f8fa653d640 1 --2- 172.21.15.164:0/678398483 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556c8a8a9560 0x556c8a8ab9b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4032114a2d8d9db server_cookie=a8d0addd954d54e1 in_seq=0 out_seq=0 2024-09-19T08:47:04.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.841+0000 7f8f89ffb640 1 -- 172.21.15.164:0/678398483 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556c8a791d60 con 0x556c8a8a9560 2024-09-19T08:47:04.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.841+0000 7f8f89ffb640 10 client.15858 handle_command_reply: tid=0 2024-09-19T08:47:04.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.841+0000 7f8faab7a740 1 client.15858 shutdown 2024-09-19T08:47:04.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.841+0000 7f8faab7a740 1 -- 172.21.15.164:0/678398483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c8a82f680 msgr2=0x556c8a8a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:04.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.841+0000 7f8faab7a740 1 --2- 172.21.15.164:0/678398483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c8a82f680 0x556c8a8a05e0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8f9000b7f0 tx=0x7f8f90005020 comp rx=0 tx=0).stop 2024-09-19T08:47:04.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.841+0000 7f8faab7a740 1 -- 172.21.15.164:0/678398483 shutdown_connections 2024-09-19T08:47:04.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.841+0000 7f8faab7a740 1 --2- 172.21.15.164:0/678398483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c8a82f680 0x556c8a8a05e0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.841+0000 7f8faab7a740 1 --2- 172.21.15.164:0/678398483 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556c8a8a9560 0x556c8a8ab9b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.841+0000 7f8f89ffb640 0 client.15858 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:47:04.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.841+0000 7f8faab7a740 1 --2- 172.21.15.164:0/678398483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c8a824dc0 0x556c8a843a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.841+0000 7f8faab7a740 1 --2- 172.21.15.164:0/678398483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556c8a8244a0 0x556c8a843500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.841+0000 7f8faab7a740 1 -- 172.21.15.164:0/678398483 >> 172.21.15.164:0/678398483 conn(0x556c8a820a30 msgr2=0x556c8a82da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:04.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.842+0000 7f8faab7a740 1 -- 172.21.15.164:0/678398483 shutdown_connections 2024-09-19T08:47:04.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.842+0000 7f8faab7a740 1 -- 172.21.15.164:0/678398483 wait complete. 2024-09-19T08:47:04.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.842+0000 7f8faab7a740 20 client.15858 trim_cache size 0 max 16384 2024-09-19T08:47:04.846 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:47:04.846 INFO:teuthology.orchestra.run.smithi164.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T08:47:04.846 INFO:teuthology.orchestra.run.smithi164.stdout: "scrubs": { 2024-09-19T08:47:04.847 INFO:teuthology.orchestra.run.smithi164.stdout: "4617d585-2e46-40d5-9b74-875f518b408f": { 2024-09-19T08:47:04.847 INFO:teuthology.orchestra.run.smithi164.stdout: "path": "/", 2024-09-19T08:47:04.847 INFO:teuthology.orchestra.run.smithi164.stdout: "tag": "4617d585-2e46-40d5-9b74-875f518b408f", 2024-09-19T08:47:04.847 INFO:teuthology.orchestra.run.smithi164.stdout: "options": "recursive,force" 2024-09-19T08:47:04.847 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:47:04.847 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:47:04.847 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:47:04.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.842+0000 7f8fa87a1640 1 -- 172.21.15.164:0/620259889 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f8f8c0a3550 msgr2=0x7f8f8c0a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:04.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.842+0000 7f8fa87a1640 1 --2- 172.21.15.164:0/620259889 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f8f8c0a3550 0x7f8f8c0a5a10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8f980cdcb0 tx=0x7f8f98000f40 comp rx=0 tx=0).stop 2024-09-19T08:47:04.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.842+0000 7f8fa87a1640 1 -- 172.21.15.164:0/620259889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fa0106a70 msgr2=0x7f8fa01a4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:04.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.843+0000 7f8fa87a1640 1 --2- 172.21.15.164:0/620259889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fa0106a70 0x7f8fa01a4340 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f8f9800eb10 tx=0x7f8f9800c2d0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.843+0000 7f8fa87a1640 1 -- 172.21.15.164:0/620259889 shutdown_connections 2024-09-19T08:47:04.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.843+0000 7f8fa87a1640 1 --2- 172.21.15.164:0/620259889 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f8f8c0a3550 0x7f8f8c0a5a10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.843+0000 7f8fa87a1640 1 --2- 172.21.15.164:0/620259889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fa0106a70 0x7f8fa01a4340 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.843+0000 7f8fa87a1640 1 --2- 172.21.15.164:0/620259889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8fa0101410 0x7f8fa0199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.843+0000 7f8fa87a1640 1 --2- 172.21.15.164:0/620259889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fa0100a70 0x7f8fa01992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.843+0000 7f8fa87a1640 1 -- 172.21.15.164:0/620259889 >> 172.21.15.164:0/620259889 conn(0x7f8fa00fc790 msgr2=0x7f8fa00fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:04.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.843+0000 7f8fa87a1640 1 -- 172.21.15.164:0/620259889 shutdown_connections 2024-09-19T08:47:04.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:04.843+0000 7f8fa87a1640 1 -- 172.21.15.164:0/620259889 wait complete. 2024-09-19T08:47:05.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:04 smithi164 ceph-mon[30928]: scrub queued for path: / 2024-09-19T08:47:05.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:04 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:47:05.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:04 smithi164 ceph-mon[30928]: scrub summary: active paths [/] 2024-09-19T08:47:05.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:04 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:47:05.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:04 smithi170 ceph-mon[35072]: scrub queued for path: / 2024-09-19T08:47:05.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:04 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:47:05.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:04 smithi170 ceph-mon[35072]: scrub summary: active paths [/] 2024-09-19T08:47:05.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:04 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:47:05.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:04 smithi173 ceph-mon[35306]: scrub queued for path: / 2024-09-19T08:47:05.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:04 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:47:05.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:04 smithi173 ceph-mon[35306]: scrub summary: active paths [/] 2024-09-19T08:47:05.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:04 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:47:05.619 INFO:tasks.cephfs.filesystem:scrub status for tag:4617d585-2e46-40d5-9b74-875f518b408f - {'path': '/', 'tag': '4617d585-2e46-40d5-9b74-875f518b408f', 'options': 'recursive,force'} 2024-09-19T08:47:06.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:05 smithi164 ceph-mon[30928]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-19T08:47:06.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:05 smithi170 ceph-mon[35072]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-19T08:47:06.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:05 smithi173 ceph-mon[35306]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-19T08:47:08.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:07 smithi164 ceph-mon[30928]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T08:47:08.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:07 smithi164 ceph-mon[30928]: scrub summary: idle 2024-09-19T08:47:08.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:07 smithi170 ceph-mon[35072]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T08:47:08.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:07 smithi170 ceph-mon[35072]: scrub summary: idle 2024-09-19T08:47:08.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:07 smithi173 ceph-mon[35306]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T08:47:08.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:07 smithi173 ceph-mon[35306]: scrub summary: idle 2024-09-19T08:47:10.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:09 smithi164 ceph-mon[30928]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T08:47:10.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:09 smithi170 ceph-mon[35072]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T08:47:10.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:09 smithi173 ceph-mon[35306]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T08:47:10.387 INFO:tasks.workunit.client.0.smithi164.stderr:Note: switching to '6fad5dd73c93545e52f57f681e140a9c3529c11e'. 2024-09-19T08:47:10.387 INFO:tasks.workunit.client.0.smithi164.stderr: 2024-09-19T08:47:10.388 INFO:tasks.workunit.client.0.smithi164.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-19T08:47:10.388 INFO:tasks.workunit.client.0.smithi164.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-19T08:47:10.388 INFO:tasks.workunit.client.0.smithi164.stderr:state without impacting any branches by switching back to a branch. 2024-09-19T08:47:10.388 INFO:tasks.workunit.client.0.smithi164.stderr: 2024-09-19T08:47:10.388 INFO:tasks.workunit.client.0.smithi164.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-19T08:47:10.388 INFO:tasks.workunit.client.0.smithi164.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-19T08:47:10.388 INFO:tasks.workunit.client.0.smithi164.stderr: 2024-09-19T08:47:10.388 INFO:tasks.workunit.client.0.smithi164.stderr: git switch -c 2024-09-19T08:47:10.388 INFO:tasks.workunit.client.0.smithi164.stderr: 2024-09-19T08:47:10.388 INFO:tasks.workunit.client.0.smithi164.stderr:Or undo this operation with: 2024-09-19T08:47:10.388 INFO:tasks.workunit.client.0.smithi164.stderr: 2024-09-19T08:47:10.388 INFO:tasks.workunit.client.0.smithi164.stderr: git switch - 2024-09-19T08:47:10.389 INFO:tasks.workunit.client.0.smithi164.stderr: 2024-09-19T08:47:10.389 INFO:tasks.workunit.client.0.smithi164.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-19T08:47:10.389 INFO:tasks.workunit.client.0.smithi164.stderr: 2024-09-19T08:47:10.389 INFO:tasks.workunit.client.0.smithi164.stderr:HEAD is now at 6fad5dd73c9 Merge pull request #59812 from neha-ojha/wip-rtd-squid 2024-09-19T08:47:10.393 DEBUG:teuthology.orchestra.run.smithi164:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-19T08:47:10.423 INFO:tasks.workunit.client.0.smithi164.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-19T08:47:10.425 INFO:tasks.workunit.client.0.smithi164.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T08:47:10.425 INFO:tasks.workunit.client.0.smithi164.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-19T08:47:11.206 INFO:tasks.workunit.client.0.smithi164.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-19T08:47:11.280 INFO:tasks.workunit.client.0.smithi164.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-19T08:47:11.322 INFO:tasks.workunit.client.0.smithi164.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T08:47:11.324 INFO:tasks.workunit.client.0.smithi164.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T08:47:11.324 INFO:tasks.workunit.client.0.smithi164.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-19T08:47:11.364 INFO:tasks.workunit.client.0.smithi164.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T08:47:11.368 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-19T08:47:11.368 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-19T08:47:11.426 INFO:tasks.workunit:Running workunits matching suites/iogen.sh on client.0... 2024-09-19T08:47:11.427 INFO:tasks.workunit:Running workunit suites/iogen.sh... 2024-09-19T08:47:11.427 DEBUG:teuthology.orchestra.run.smithi164:workunit test suites/iogen.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iogen.sh 2024-09-19T08:47:11.490 INFO:tasks.workunit.client.0.smithi164.stdout:getting iogen 2024-09-19T08:47:11.490 INFO:tasks.workunit.client.0.smithi164.stderr:+ echo 'getting iogen' 2024-09-19T08:47:11.491 INFO:tasks.workunit.client.0.smithi164.stderr:+ wget http://download.ceph.com/qa/iogen_3.1p0.tar 2024-09-19T08:47:11.607 INFO:tasks.workunit.client.0.smithi164.stderr:--2024-09-19 08:47:11-- http://download.ceph.com/qa/iogen_3.1p0.tar 2024-09-19T08:47:11.607 INFO:tasks.workunit.client.0.smithi164.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-09-19T08:47:11.687 INFO:tasks.workunit.client.0.smithi164.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-09-19T08:47:11.768 INFO:tasks.workunit.client.0.smithi164.stderr:HTTP request sent, awaiting response... 200 OK 2024-09-19T08:47:11.768 INFO:tasks.workunit.client.0.smithi164.stderr:Length: 8071 (7.9K) [application/octet-stream] 2024-09-19T08:47:11.768 INFO:tasks.workunit.client.0.smithi164.stderr:Saving to: ‘iogen_3.1p0.tar’ 2024-09-19T08:47:11.771 INFO:tasks.workunit.client.0.smithi164.stderr: 2024-09-19T08:47:11.772 INFO:tasks.workunit.client.0.smithi164.stderr: 0K ....... 100% 1.13G=0s 2024-09-19T08:47:11.772 INFO:tasks.workunit.client.0.smithi164.stderr: 2024-09-19T08:47:11.772 INFO:tasks.workunit.client.0.smithi164.stderr:2024-09-19 08:47:11 (1.13 GB/s) - ‘iogen_3.1p0.tar’ saved [8071/8071] 2024-09-19T08:47:11.772 INFO:tasks.workunit.client.0.smithi164.stderr: 2024-09-19T08:47:11.772 INFO:tasks.workunit.client.0.smithi164.stderr:+ tar -xvzf iogen_3.1p0.tar 2024-09-19T08:47:11.783 INFO:tasks.workunit.client.0.smithi164.stdout:iogen_3.1p0 2024-09-19T08:47:11.785 INFO:tasks.workunit.client.0.smithi164.stdout:iogen_3.1p0/src 2024-09-19T08:47:11.785 INFO:tasks.workunit.client.0.smithi164.stdout:iogen_3.1p0/src/iogen.8 2024-09-19T08:47:11.786 INFO:tasks.workunit.client.0.smithi164.stdout:iogen_3.1p0/src/iogen.c 2024-09-19T08:47:11.787 INFO:tasks.workunit.client.0.smithi164.stdout:iogen_3.1p0/README 2024-09-19T08:47:11.788 INFO:tasks.workunit.client.0.smithi164.stdout:iogen_3.1p0/Makefile.bsd 2024-09-19T08:47:11.789 INFO:tasks.workunit.client.0.smithi164.stdout:iogen_3.1p0/Makefile 2024-09-19T08:47:11.790 INFO:tasks.workunit.client.0.smithi164.stderr:+ cd iogen_3.1p0 2024-09-19T08:47:11.791 INFO:tasks.workunit.client.0.smithi164.stdout:making iogen 2024-09-19T08:47:11.791 INFO:tasks.workunit.client.0.smithi164.stderr:+ echo 'making iogen' 2024-09-19T08:47:11.791 INFO:tasks.workunit.client.0.smithi164.stderr:+ make 2024-09-19T08:47:11.798 INFO:tasks.workunit.client.0.smithi164.stdout:gcc -D_FILE_OFFSET_BITS=64 -DVERSION=\"3.1p0\" -Dstrlcpy=strncpy -Darc4random=rand -O2 src/iogen.c -o iogen 2024-09-19T08:47:12.022 INFO:tasks.workunit.client.0.smithi164.stdout:nroff -mandoc -Tascii src/iogen.8 > iogen.cat8 2024-09-19T08:47:12.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:11 smithi164 ceph-mon[30928]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-19T08:47:12.120 INFO:tasks.workunit.client.0.smithi164.stdout:running iogen 2024-09-19T08:47:12.122 INFO:tasks.workunit.client.0.smithi164.stderr:+ echo 'running iogen' 2024-09-19T08:47:12.122 INFO:tasks.workunit.client.0.smithi164.stderr:+ ./iogen -n 5 -s 2g 2024-09-19T08:47:12.122 INFO:tasks.workunit.client.0.smithi164.stderr:child spawned 90081 2024-09-19T08:47:12.122 INFO:tasks.workunit.client.0.smithi164.stderr:child spawned 90082 2024-09-19T08:47:12.123 INFO:tasks.workunit.client.0.smithi164.stderr:child spawned 90083 2024-09-19T08:47:12.123 INFO:tasks.workunit.client.0.smithi164.stderr:child spawned 90084 2024-09-19T08:47:12.123 INFO:tasks.workunit.client.0.smithi164.stderr:child spawned 90085 2024-09-19T08:47:12.123 INFO:tasks.workunit.client.0.smithi164.stdout:sleep for 10 min 2024-09-19T08:47:12.124 INFO:tasks.workunit.client.0.smithi164.stderr:+ echo 'sleep for 10 min' 2024-09-19T08:47:12.124 INFO:tasks.workunit.client.0.smithi164.stderr:+ sleep 600 2024-09-19T08:47:12.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:11 smithi170 ceph-mon[35072]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-19T08:47:12.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:11 smithi173 ceph-mon[35306]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-19T08:47:13.518 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T08:47:13.518 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.35 2024-09-19T08:47:13.519 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:47:14.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:13 smithi164 ceph-mon[30928]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T08:47:14.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:13 smithi170 ceph-mon[35072]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T08:47:14.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:13 smithi173 ceph-mon[35306]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T08:47:14.273 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:47:16.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:15 smithi170 ceph-mon[35072]: pgmap v46: 97 pgs: 97 active+clean; 681 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 57 MiB/s wr, 40 op/s 2024-09-19T08:47:16.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:15 smithi173 ceph-mon[35306]: pgmap v46: 97 pgs: 97 active+clean; 681 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 57 MiB/s wr, 40 op/s 2024-09-19T08:47:16.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:15 smithi164 ceph-mon[30928]: pgmap v46: 97 pgs: 97 active+clean; 681 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 57 MiB/s wr, 40 op/s 2024-09-19T08:47:17.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.703+0000 7f71bc2ba640 1 -- 172.21.15.164:0/1654772717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71b4102d90 msgr2=0x7f71b4103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:17.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.703+0000 7f71bc2ba640 1 --2- 172.21.15.164:0/1654772717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71b4102d90 0x7f71b4103170 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009930 tx=0x7f71b002a420 comp rx=0 tx=0).stop 2024-09-19T08:47:17.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.703+0000 7f71bc2ba640 1 -- 172.21.15.164:0/1654772717 shutdown_connections 2024-09-19T08:47:17.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.703+0000 7f71bc2ba640 1 --2- 172.21.15.164:0/1654772717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71b4108290 0x7f71b410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:17.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.703+0000 7f71bc2ba640 1 --2- 172.21.15.164:0/1654772717 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71b4103740 0x7f71b4107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:17.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.703+0000 7f71bc2ba640 1 --2- 172.21.15.164:0/1654772717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71b4102d90 0x7f71b4103170 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:17.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.703+0000 7f71bc2ba640 1 -- 172.21.15.164:0/1654772717 >> 172.21.15.164:0/1654772717 conn(0x7f71b40fc7d0 msgr2=0x7f71b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:17.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.703+0000 7f71bc2ba640 1 -- 172.21.15.164:0/1654772717 shutdown_connections 2024-09-19T08:47:17.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.703+0000 7f71bc2ba640 1 -- 172.21.15.164:0/1654772717 wait complete. 2024-09-19T08:47:17.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.704+0000 7f71bc2ba640 1 Processor -- start 2024-09-19T08:47:17.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.704+0000 7f71bc2ba640 1 -- start start 2024-09-19T08:47:17.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.705+0000 7f71bc2ba640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71b4102d90 0x7f71b410b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:17.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.705+0000 7f71bc2ba640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71b4103740 0x7f71b410b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:17.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.705+0000 7f71bc2ba640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71b4108290 0x7f71b41aabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:17.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.705+0000 7f71bc2ba640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f71b4101880 con 0x7f71b4108290 2024-09-19T08:47:17.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.705+0000 7f71bc2ba640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f71b4101700 con 0x7f71b4103740 2024-09-19T08:47:17.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.705+0000 7f71bc2ba640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f71b4101a00 con 0x7f71b4102d90 2024-09-19T08:47:17.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.705+0000 7f71ba857640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71b4108290 0x7f71b41aabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:17.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.705+0000 7f71ba857640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71b4108290 0x7f71b41aabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33762/0 (socket says 172.21.15.164:33762) 2024-09-19T08:47:17.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.705+0000 7f71ba857640 1 -- 172.21.15.164:0/880169867 learned_addr learned my addr 172.21.15.164:0/880169867 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:17.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.705+0000 7f71b9855640 1 --2- 172.21.15.164:0/880169867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71b4103740 0x7f71b410b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:17.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.705+0000 7f71ba857640 1 -- 172.21.15.164:0/880169867 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71b4102d90 msgr2=0x7f71b410b1a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T08:47:17.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.705+0000 7f71ba857640 1 --2- 172.21.15.164:0/880169867 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71b4102d90 0x7f71b410b1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:17.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.705+0000 7f71ba857640 1 -- 172.21.15.164:0/880169867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71b4103740 msgr2=0x7f71b410b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:17.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.705+0000 7f71ba857640 1 --2- 172.21.15.164:0/880169867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71b4103740 0x7f71b410b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:17.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.705+0000 7f71ba857640 1 -- 172.21.15.164:0/880169867 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b41ab100 con 0x7f71b4108290 2024-09-19T08:47:17.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.706+0000 7f71b9855640 1 --2- 172.21.15.164:0/880169867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71b4103740 0x7f71b410b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:17.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.706+0000 7f71ba857640 1 --2- 172.21.15.164:0/880169867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71b4108290 0x7f71b41aabc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00b970 tx=0x7f71ac00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:17.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.706+0000 7f71ab7fe640 1 -- 172.21.15.164:0/880169867 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac019070 con 0x7f71b4108290 2024-09-19T08:47:17.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.706+0000 7f71ab7fe640 1 -- 172.21.15.164:0/880169867 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71ac014440 con 0x7f71b4108290 2024-09-19T08:47:17.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.706+0000 7f71ab7fe640 1 -- 172.21.15.164:0/880169867 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac007650 con 0x7f71b4108290 2024-09-19T08:47:17.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.706+0000 7f71bc2ba640 1 -- 172.21.15.164:0/880169867 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71b41ab330 con 0x7f71b4108290 2024-09-19T08:47:17.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.706+0000 7f71bc2ba640 1 -- 172.21.15.164:0/880169867 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71b41ab7f0 con 0x7f71b4108290 2024-09-19T08:47:17.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.708+0000 7f71bc2ba640 1 -- 172.21.15.164:0/880169867 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71b406a7f0 con 0x7f71b4108290 2024-09-19T08:47:17.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.710+0000 7f71ab7fe640 1 -- 172.21.15.164:0/880169867 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7f71ac01c070 con 0x7f71b4108290 2024-09-19T08:47:17.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.712+0000 7f71ab7fe640 1 --2- 172.21.15.164:0/880169867 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f71a00a3590 0x7f71a00a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:17.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.712+0000 7f71ab7fe640 1 -- 172.21.15.164:0/880169867 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f71ac0cf430 con 0x7f71b4108290 2024-09-19T08:47:17.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.713+0000 7f71ba056640 1 --2- 172.21.15.164:0/880169867 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f71a00a3590 0x7f71a00a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:17.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.713+0000 7f71ba056640 1 --2- 172.21.15.164:0/880169867 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f71a00a3590 0x7f71a00a5a50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f71b0005c20 tx=0x7f71b0005fd0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:17.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:17.718+0000 7f71ab7fe640 1 -- 172.21.15.164:0/880169867 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f71ac09bd40 con 0x7f71b4108290 2024-09-19T08:47:18.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:17 smithi170 ceph-mon[35072]: pgmap v47: 97 pgs: 97 active+clean; 1.1 GiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 95 MiB/s wr, 91 op/s 2024-09-19T08:47:18.140 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:47:18.141 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":28,"btime":"2024-09-19T08:45:46:903004+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:45:46.852789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:47:18.141 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 28 2024-09-19T08:47:18.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:18.133+0000 7f71bc2ba640 1 -- 172.21.15.164:0/880169867 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71b41abb10 con 0x7f71b4108290 2024-09-19T08:47:18.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:18.137+0000 7f71ab7fe640 1 -- 172.21.15.164:0/880169867 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11984 (secure 0 0 0) 0x7f71ac097070 con 0x7f71b4108290 2024-09-19T08:47:18.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:18.147+0000 7f71a97ba640 1 -- 172.21.15.164:0/880169867 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f71a00a3590 msgr2=0x7f71a00a5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:18.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:18.147+0000 7f71a97ba640 1 --2- 172.21.15.164:0/880169867 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f71a00a3590 0x7f71a00a5a50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f71b0005c20 tx=0x7f71b0005fd0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:18.147+0000 7f71a97ba640 1 -- 172.21.15.164:0/880169867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71b4108290 msgr2=0x7f71b41aabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:18.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:18.147+0000 7f71a97ba640 1 --2- 172.21.15.164:0/880169867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71b4108290 0x7f71b41aabc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00b970 tx=0x7f71ac00be30 comp rx=0 tx=0).stop 2024-09-19T08:47:18.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:18.148+0000 7f71a97ba640 1 -- 172.21.15.164:0/880169867 shutdown_connections 2024-09-19T08:47:18.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:18.148+0000 7f71a97ba640 1 --2- 172.21.15.164:0/880169867 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f71a00a3590 0x7f71a00a5a50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:18.148+0000 7f71a97ba640 1 --2- 172.21.15.164:0/880169867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71b4108290 0x7f71b41aabc0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:18.148+0000 7f71a97ba640 1 --2- 172.21.15.164:0/880169867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71b4103740 0x7f71b410b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:18.148+0000 7f71a97ba640 1 --2- 172.21.15.164:0/880169867 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71b4102d90 0x7f71b410b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:18.148+0000 7f71a97ba640 1 -- 172.21.15.164:0/880169867 >> 172.21.15.164:0/880169867 conn(0x7f71b40fc7d0 msgr2=0x7f71b4114c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:18.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:18.150+0000 7f71a97ba640 1 -- 172.21.15.164:0/880169867 shutdown_connections 2024-09-19T08:47:18.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:18.151+0000 7f71a97ba640 1 -- 172.21.15.164:0/880169867 wait complete. 2024-09-19T08:47:18.160 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:17 smithi164 ceph-mon[30928]: pgmap v47: 97 pgs: 97 active+clean; 1.1 GiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 95 MiB/s wr, 91 op/s 2024-09-19T08:47:18.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:17 smithi173 ceph-mon[35306]: pgmap v47: 97 pgs: 97 active+clean; 1.1 GiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 95 MiB/s wr, 91 op/s 2024-09-19T08:47:19.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:18 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/880169867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:47:19.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:18 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/880169867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:47:19.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:18 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/880169867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:47:20.123 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:19 smithi164 ceph-mon[30928]: pgmap v48: 97 pgs: 97 active+clean; 1.1 GiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 95 MiB/s wr, 91 op/s 2024-09-19T08:47:20.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:19 smithi170 ceph-mon[35072]: pgmap v48: 97 pgs: 97 active+clean; 1.1 GiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 95 MiB/s wr, 91 op/s 2024-09-19T08:47:20.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:19 smithi173 ceph-mon[35306]: pgmap v48: 97 pgs: 97 active+clean; 1.1 GiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 95 MiB/s wr, 91 op/s 2024-09-19T08:47:20.311 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0' 2024-09-19T08:47:20.311 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0 2024-09-19T08:47:21.370 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:47:22.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:21 smithi173 ceph-mon[35306]: pgmap v49: 97 pgs: 97 active+clean; 5.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 457 MiB/s wr, 352 op/s 2024-09-19T08:47:22.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:21 smithi173 ceph-mon[35306]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-09-19T08:47:22.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:21 smithi164 ceph-mon[30928]: pgmap v49: 97 pgs: 97 active+clean; 5.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 457 MiB/s wr, 352 op/s 2024-09-19T08:47:22.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:21 smithi164 ceph-mon[30928]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-09-19T08:47:22.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:21 smithi170 ceph-mon[35072]: pgmap v49: 97 pgs: 97 active+clean; 5.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 457 MiB/s wr, 352 op/s 2024-09-19T08:47:22.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:21 smithi170 ceph-mon[35072]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-09-19T08:47:23.109 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:22 smithi164 ceph-mon[30928]: pgmap v50: 97 pgs: 97 active+clean; 5.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 457 MiB/s wr, 347 op/s 2024-09-19T08:47:23.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.207+0000 7fe4b29ab640 1 -- 172.21.15.164:0/1132542266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4ac10cf30 msgr2=0x7fe4ac119410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:23.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.207+0000 7fe4b29ab640 1 --2- 172.21.15.164:0/1132542266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4ac10cf30 0x7fe4ac119410 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe49c009890 tx=0x7fe49c02a380 comp rx=0 tx=0).stop 2024-09-19T08:47:23.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.209+0000 7fe4b29ab640 1 -- 172.21.15.164:0/1132542266 shutdown_connections 2024-09-19T08:47:23.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.209+0000 7fe4b29ab640 1 --2- 172.21.15.164:0/1132542266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4ac10cf30 0x7fe4ac119410 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.209+0000 7fe4b29ab640 1 --2- 172.21.15.164:0/1132542266 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4ac10c590 0x7fe4ac10c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.209+0000 7fe4b29ab640 1 --2- 172.21.15.164:0/1132542266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4ac112640 0x7fe4ac112a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.209+0000 7fe4b29ab640 1 -- 172.21.15.164:0/1132542266 >> 172.21.15.164:0/1132542266 conn(0x7fe4ac06cdc0 msgr2=0x7fe4ac06d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:23.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.209+0000 7fe4b29ab640 1 -- 172.21.15.164:0/1132542266 shutdown_connections 2024-09-19T08:47:23.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.209+0000 7fe4b29ab640 1 -- 172.21.15.164:0/1132542266 wait complete. 2024-09-19T08:47:23.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.210+0000 7fe4b29ab640 1 Processor -- start 2024-09-19T08:47:23.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.211+0000 7fe4b29ab640 1 -- start start 2024-09-19T08:47:23.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.211+0000 7fe4b29ab640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4ac10c590 0x7fe4ac117f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:23.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.211+0000 7fe4b29ab640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4ac10cf30 0x7fe4ac118460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:23.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.211+0000 7fe4b29ab640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4ac112640 0x7fe4ac1189a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:23.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.211+0000 7fe4b29ab640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac06edf0 con 0x7fe4ac10cf30 2024-09-19T08:47:23.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.211+0000 7fe4b29ab640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac06ec70 con 0x7fe4ac112640 2024-09-19T08:47:23.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.211+0000 7fe4b29ab640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac06ef70 con 0x7fe4ac10c590 2024-09-19T08:47:23.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.212+0000 7fe4b0f48640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4ac112640 0x7fe4ac1189a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:23.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.212+0000 7fe4b0f48640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4ac112640 0x7fe4ac1189a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:47932/0 (socket says 172.21.15.164:47932) 2024-09-19T08:47:23.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.212+0000 7fe4b0f48640 1 -- 172.21.15.164:0/1001464389 learned_addr learned my addr 172.21.15.164:0/1001464389 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:23.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.212+0000 7fe4ab7fe640 1 --2- 172.21.15.164:0/1001464389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4ac10cf30 0x7fe4ac118460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:23.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.213+0000 7fe4b0f48640 1 -- 172.21.15.164:0/1001464389 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4ac10c590 msgr2=0x7fe4ac117f20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T08:47:23.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.213+0000 7fe4b0f48640 1 --2- 172.21.15.164:0/1001464389 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4ac10c590 0x7fe4ac117f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.213+0000 7fe4b0f48640 1 -- 172.21.15.164:0/1001464389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4ac10cf30 msgr2=0x7fe4ac118460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:23.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.213+0000 7fe4b0f48640 1 --2- 172.21.15.164:0/1001464389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4ac10cf30 0x7fe4ac118460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.213+0000 7fe4b0f48640 1 -- 172.21.15.164:0/1001464389 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4ac1bbd70 con 0x7fe4ac112640 2024-09-19T08:47:23.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.213+0000 7fe4b0f48640 1 --2- 172.21.15.164:0/1001464389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4ac112640 0x7fe4ac1189a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe49c009860 tx=0x7fe49c033d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:23.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.214+0000 7fe4a97fa640 1 -- 172.21.15.164:0/1001464389 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c037040 con 0x7fe4ac112640 2024-09-19T08:47:23.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.214+0000 7fe4b29ab640 1 -- 172.21.15.164:0/1001464389 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4ac1bc000 con 0x7fe4ac112640 2024-09-19T08:47:23.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.214+0000 7fe4b29ab640 1 -- 172.21.15.164:0/1001464389 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4ac1bc4e0 con 0x7fe4ac112640 2024-09-19T08:47:23.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.215+0000 7fe4a97fa640 1 -- 172.21.15.164:0/1001464389 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe49c040070 con 0x7fe4ac112640 2024-09-19T08:47:23.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.216+0000 7fe4a97fa640 1 -- 172.21.15.164:0/1001464389 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c003bd0 con 0x7fe4ac112640 2024-09-19T08:47:23.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.220+0000 7fe4b4d84740 1 Processor -- start 2024-09-19T08:47:23.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.222+0000 7fe4a97fa640 1 -- 172.21.15.164:0/1001464389 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fe49c004250 con 0x7fe4ac112640 2024-09-19T08:47:23.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.223+0000 7fe4b4d84740 1 -- start start 2024-09-19T08:47:23.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.224+0000 7fe4b4d84740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560192122850 0x560192122c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:23.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.224+0000 7fe4b4d84740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560192123170 0x56019212d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:23.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.225+0000 7fe4b4d84740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019212da30 0x56019212fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:23.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.225+0000 7fe4b4d84740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560192087160 con 0x56019212da30 2024-09-19T08:47:23.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.225+0000 7fe4b4d84740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56019202e720 con 0x560192122850 2024-09-19T08:47:23.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.225+0000 7fe4b4d84740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x560192083120 con 0x560192123170 2024-09-19T08:47:23.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.225+0000 7fe4abfff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560192123170 0x56019212d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:23.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.225+0000 7fe4abfff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560192123170 0x56019212d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:42018/0 (socket says 172.21.15.164:42018) 2024-09-19T08:47:23.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.225+0000 7fe4abfff640 1 -- 172.21.15.164:0/2420403118 learned_addr learned my addr 172.21.15.164:0/2420403118 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:23.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.226+0000 7fe4a97fa640 1 --2- 172.21.15.164:0/1001464389 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe4940a35c0 0x7fe4940a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:23.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.227+0000 7fe4b0f48640 1 --2- 172.21.15.164:0/1001464389 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe4940a35c0 0x7fe4940a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:23.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.228+0000 7fe4a97fa640 1 -- 172.21.15.164:0/1001464389 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fe49c0ef490 con 0x7fe4ac112640 2024-09-19T08:47:23.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.228+0000 7fe4ab7fe640 1 --2- 172.21.15.164:0/2420403118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560192122850 0x560192122c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:23.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.228+0000 7fe4ab7fe640 1 --2- 172.21.15.164:0/2420403118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019212da30 0x56019212fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:23.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.228+0000 7fe4abfff640 1 -- 172.21.15.164:0/2420403118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560192122850 msgr2=0x560192122c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:23.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.228+0000 7fe4abfff640 1 --2- 172.21.15.164:0/2420403118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560192122850 0x560192122c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.228+0000 7fe4abfff640 1 -- 172.21.15.164:0/2420403118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019212da30 msgr2=0x56019212fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:23.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.228+0000 7fe4abfff640 1 --2- 172.21.15.164:0/2420403118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019212da30 0x56019212fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.228+0000 7fe4abfff640 1 -- 172.21.15.164:0/2420403118 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560191df8ba0 con 0x560192123170 2024-09-19T08:47:23.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.228+0000 7fe4ab7fe640 1 --2- 172.21.15.164:0/2420403118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019212da30 0x56019212fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:23.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.229+0000 7fe4b0f48640 1 --2- 172.21.15.164:0/1001464389 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe4940a35c0 0x7fe4940a5a80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe49c0c3fd0 tx=0x7fe49c0b7400 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:23.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.230+0000 7fe4abfff640 1 --2- 172.21.15.164:0/2420403118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560192123170 0x56019212d4f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4009db0 tx=0x7fe4a400e760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83d35fbd4519b665 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:23.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.233+0000 7fe4927fc640 1 -- 172.21.15.164:0/2420403118 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4003b70 con 0x560192123170 2024-09-19T08:47:23.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.233+0000 7fe4927fc640 1 -- 172.21.15.164:0/2420403118 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4a4003d10 con 0x560192123170 2024-09-19T08:47:23.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.233+0000 7fe4927fc640 1 -- 172.21.15.164:0/2420403118 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a400fbb0 con 0x560192123170 2024-09-19T08:47:23.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.235+0000 7fe4b4d84740 1 -- 172.21.15.164:0/2420403118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560192123170 msgr2=0x56019212d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:23.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.235+0000 7fe4b4d84740 1 --2- 172.21.15.164:0/2420403118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560192123170 0x56019212d4f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4009db0 tx=0x7fe4a400e760 comp rx=0 tx=0).stop 2024-09-19T08:47:23.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.235+0000 7fe4b4d84740 1 -- 172.21.15.164:0/2420403118 shutdown_connections 2024-09-19T08:47:23.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.235+0000 7fe4b4d84740 1 --2- 172.21.15.164:0/2420403118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019212da30 0x56019212fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.235+0000 7fe4b4d84740 1 --2- 172.21.15.164:0/2420403118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560192123170 0x56019212d4f0 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4009db0 tx=0x7fe4a400e760 comp rx=0 tx=0).stop 2024-09-19T08:47:23.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.235+0000 7fe4b4d84740 1 --2- 172.21.15.164:0/2420403118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560192122850 0x560192122c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.235+0000 7fe4b4d84740 1 -- 172.21.15.164:0/2420403118 >> 172.21.15.164:0/2420403118 conn(0x56019211ede0 msgr2=0x56019211f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:23.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.235+0000 7fe4b4d84740 1 -- 172.21.15.164:0/2420403118 shutdown_connections 2024-09-19T08:47:23.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.236+0000 7fe4b4d84740 1 -- 172.21.15.164:0/2420403118 wait complete. 2024-09-19T08:47:23.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.236+0000 7fe4b4d84740 1 -- start start 2024-09-19T08:47:23.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.237+0000 7fe4b4d84740 1 Processor -- start 2024-09-19T08:47:23.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.239+0000 7fe4b4d84740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560192122850 0x56019212aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:23.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.239+0000 7fe4b4d84740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56019212da30 0x56019212afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:23.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.239+0000 7fe4ab7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560192122850 0x56019212aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:23.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.239+0000 7fe4ab7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560192122850 0x56019212aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33800/0 (socket says 172.21.15.164:33800) 2024-09-19T08:47:23.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.239+0000 7fe4ab7fe640 1 -- 172.21.15.164:0/3283536380 learned_addr learned my addr 172.21.15.164:0/3283536380 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:23.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.240+0000 7fe4b4d84740 1 --2- 172.21.15.164:0/3283536380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56019212b510 0x560192139b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:23.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.240+0000 7fe4b4d84740 1 -- 172.21.15.164:0/3283536380 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560192130600 con 0x560192122850 2024-09-19T08:47:23.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.240+0000 7fe4b4d84740 1 -- 172.21.15.164:0/3283536380 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56019200f900 con 0x56019212da30 2024-09-19T08:47:23.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.240+0000 7fe4b4d84740 1 -- 172.21.15.164:0/3283536380 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x560192130780 con 0x56019212b510 2024-09-19T08:47:23.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.240+0000 7fe4b4d84740 10 client.0 _ll_register_callbacks cb 0x56019200d780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:47:23.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.240+0000 7fe4abfff640 1 --2- 172.21.15.164:0/3283536380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56019212da30 0x56019212afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:23.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.241+0000 7fe4ab7fe640 1 -- 172.21.15.164:0/3283536380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56019212b510 msgr2=0x560192139b00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:47:23.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.241+0000 7fe4ab7fe640 1 --2- 172.21.15.164:0/3283536380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56019212b510 0x560192139b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.241+0000 7fe4ab7fe640 1 -- 172.21.15.164:0/3283536380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56019212da30 msgr2=0x56019212afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:23.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.241+0000 7fe4ab7fe640 1 --2- 172.21.15.164:0/3283536380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56019212da30 0x56019212afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.241+0000 7fe4ab7fe640 1 -- 172.21.15.164:0/3283536380 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56019204fc50 con 0x560192122850 2024-09-19T08:47:23.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.241+0000 7fe4ab7fe640 1 --2- 172.21.15.164:0/3283536380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560192122850 0x56019212aa90 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0012530 tx=0x7fe4a00129f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f60622744496d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:23.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.241+0000 7fe48b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:47:23.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.241+0000 7fe4b4d84740 1 -- 172.21.15.164:0/3283536380 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4a4003d10 con 0x560192122850 2024-09-19T08:47:23.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.241+0000 7fe48b7fe640 1 -- 172.21.15.164:0/3283536380 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0019030 con 0x560192122850 2024-09-19T08:47:23.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.241+0000 7fe48b7fe640 1 -- 172.21.15.164:0/3283536380 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fe4a000f040 con 0x560192122850 2024-09-19T08:47:23.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.242+0000 7fe48b7fe640 1 -- 172.21.15.164:0/3283536380 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe4a0014680 con 0x560192122850 2024-09-19T08:47:23.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.242+0000 7fe4b4d84740 10 client.15873 fetch_fsmap learned FSMap version 28 2024-09-19T08:47:23.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.242+0000 7fe4b4d84740 1 -- 172.21.15.164:0/3283536380 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({fsmap=28}) -- 0x56019213c750 con 0x560192122850 2024-09-19T08:47:23.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.242+0000 7fe48b7fe640 1 -- 172.21.15.164:0/3283536380 <== mon.0 v2:172.21.15.164:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe4a0014820 con 0x560192122850 2024-09-19T08:47:23.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.242+0000 7fe4b4d84740 10 client.15873 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:47:23.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.242+0000 7fe4b4d84740 10 client.15873 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:47:23.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.242+0000 7fe4b4d84740 20 client.15873 populate_metadata read hostname 'smithi164' 2024-09-19T08:47:23.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.243+0000 7fe4b4d84740 1 --2- 172.21.15.164:0/3283536380 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56019213db50 0x56019213ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:23.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.243+0000 7fe4b4d84740 4 client.15873 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:47:23.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.243+0000 7fe4b4d84740 1 -- 172.21.15.164:0/3283536380 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe4a4003d10 con 0x56019213db50 2024-09-19T08:47:23.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.243+0000 7fe4abfff640 1 --2- 172.21.15.164:0/3283536380 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56019213db50 0x56019213ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:23.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.247+0000 7fe4abfff640 1 --2- 172.21.15.164:0/3283536380 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56019213db50 0x56019213ffa0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88b8b22f88942804 server_cookie=6e5e6d48b93969e9 in_seq=0 out_seq=0 2024-09-19T08:47:23.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.247+0000 7fe48b7fe640 10 client.15873 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:47:23.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.251+0000 7fe48b7fe640 1 -- 172.21.15.164:0/3283536380 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fe4a4003d10 con 0x56019213db50 2024-09-19T08:47:23.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.251+0000 7fe48b7fe640 10 client.15873 handle_command_reply: tid=0 2024-09-19T08:47:23.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.252+0000 7fe4b4d84740 1 client.15873 shutdown 2024-09-19T08:47:23.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.252+0000 7fe4b4d84740 1 -- 172.21.15.164:0/3283536380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560192122850 msgr2=0x56019212aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:23.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.252+0000 7fe4b4d84740 1 --2- 172.21.15.164:0/3283536380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560192122850 0x56019212aa90 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0012530 tx=0x7fe4a00129f0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.252+0000 7fe4b4d84740 1 -- 172.21.15.164:0/3283536380 shutdown_connections 2024-09-19T08:47:23.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.252+0000 7fe4b4d84740 1 --2- 172.21.15.164:0/3283536380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56019212b510 0x560192139b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.253+0000 7fe4b4d84740 1 --2- 172.21.15.164:0/3283536380 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56019213db50 0x56019213ffa0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.253+0000 7fe48b7fe640 0 client.15873 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:47:23.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.253+0000 7fe4b4d84740 1 --2- 172.21.15.164:0/3283536380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56019212da30 0x56019212afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.253+0000 7fe4b4d84740 1 --2- 172.21.15.164:0/3283536380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560192122850 0x56019212aa90 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.253+0000 7fe4b4d84740 1 -- 172.21.15.164:0/3283536380 >> 172.21.15.164:0/3283536380 conn(0x56019211ede0 msgr2=0x560192124b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:23.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.253+0000 7fe4b4d84740 1 -- 172.21.15.164:0/3283536380 shutdown_connections 2024-09-19T08:47:23.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.253+0000 7fe4b4d84740 1 -- 172.21.15.164:0/3283536380 wait complete. 2024-09-19T08:47:23.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.253+0000 7fe4b4d84740 20 client.15873 trim_cache size 0 max 16384 2024-09-19T08:47:23.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.294+0000 7fe4b4d84740 1 Processor -- start 2024-09-19T08:47:23.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.294+0000 7fe4b4d84740 1 -- start start 2024-09-19T08:47:23.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.295+0000 7fe4b4d84740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560192122850 0x5601921398f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:23.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.295+0000 7fe4b4d84740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56019212b510 0x560192124a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:23.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.295+0000 7fe4b4d84740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019212da30 0x5601921458b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:23.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.295+0000 7fe4b4d84740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe49800e540 con 0x56019212da30 2024-09-19T08:47:23.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.295+0000 7fe4b4d84740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe498010080 con 0x560192122850 2024-09-19T08:47:23.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.295+0000 7fe4b4d84740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe49800ec10 con 0x56019212b510 2024-09-19T08:47:23.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.296+0000 7fe4ab7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560192122850 0x5601921398f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:23.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.296+0000 7fe4ab7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560192122850 0x5601921398f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:47962/0 (socket says 172.21.15.164:47962) 2024-09-19T08:47:23.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.296+0000 7fe4ab7fe640 1 -- 172.21.15.164:0/1654718941 learned_addr learned my addr 172.21.15.164:0/1654718941 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:23.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.297+0000 7fe4abfff640 1 --2- 172.21.15.164:0/1654718941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56019212b510 0x560192124a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:23.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.297+0000 7fe4ab7fe640 1 --2- 172.21.15.164:0/1654718941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019212da30 0x5601921458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:23.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.297+0000 7fe4ab7fe640 1 -- 172.21.15.164:0/1654718941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56019212b510 msgr2=0x560192124a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:23.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.297+0000 7fe4ab7fe640 1 --2- 172.21.15.164:0/1654718941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56019212b510 0x560192124a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.297+0000 7fe4ab7fe640 1 -- 172.21.15.164:0/1654718941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019212da30 msgr2=0x5601921458b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:23.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.297+0000 7fe4ab7fe640 1 --2- 172.21.15.164:0/1654718941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019212da30 0x5601921458b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.297+0000 7fe4ab7fe640 1 -- 172.21.15.164:0/1654718941 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560192130190 con 0x560192122850 2024-09-19T08:47:23.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.298+0000 7fe4abfff640 1 --2- 172.21.15.164:0/1654718941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56019212b510 0x560192124a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:47:23.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.300+0000 7fe4ab7fe640 1 --2- 172.21.15.164:0/1654718941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560192122850 0x5601921398f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe4a00148a0 tx=0x7fe4a000b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63f2211a4aef8e47 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:23.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.300+0000 7fe491ffb640 1 -- 172.21.15.164:0/1654718941 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a002f040 con 0x560192122850 2024-09-19T08:47:23.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.300+0000 7fe491ffb640 1 -- 172.21.15.164:0/1654718941 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4a00036b0 con 0x560192122850 2024-09-19T08:47:23.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.302+0000 7fe4b4d84740 1 -- 172.21.15.164:0/1654718941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560192122850 msgr2=0x5601921398f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:23.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.302+0000 7fe4b4d84740 1 --2- 172.21.15.164:0/1654718941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560192122850 0x5601921398f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe4a00148a0 tx=0x7fe4a000b000 comp rx=0 tx=0).stop 2024-09-19T08:47:23.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.303+0000 7fe4b4d84740 1 -- 172.21.15.164:0/1654718941 shutdown_connections 2024-09-19T08:47:23.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.303+0000 7fe4b4d84740 1 --2- 172.21.15.164:0/1654718941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019212da30 0x5601921458b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.303+0000 7fe4b4d84740 1 --2- 172.21.15.164:0/1654718941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56019212b510 0x560192124a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.303+0000 7fe4b4d84740 1 --2- 172.21.15.164:0/1654718941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560192122850 0x5601921398f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.303+0000 7fe4b4d84740 1 -- 172.21.15.164:0/1654718941 >> 172.21.15.164:0/1654718941 conn(0x56019211ede0 msgr2=0x5601921242f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:23.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.308+0000 7fe4b4d84740 1 -- 172.21.15.164:0/1654718941 shutdown_connections 2024-09-19T08:47:23.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.308+0000 7fe4b4d84740 1 -- 172.21.15.164:0/1654718941 wait complete. 2024-09-19T08:47:23.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.308+0000 7fe4b4d84740 1 -- start start 2024-09-19T08:47:23.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.309+0000 7fe4b4d84740 1 Processor -- start 2024-09-19T08:47:23.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.310+0000 7fe4b4d84740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560192122850 0x560192135a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:23.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.310+0000 7fe4b4d84740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56019212b510 0x560192135fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:23.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.310+0000 7fe4b4d84740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019212da30 0x560192132a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:23.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.310+0000 7fe4b4d84740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe49800f9b0 con 0x56019212da30 2024-09-19T08:47:23.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.310+0000 7fe4b4d84740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe49800f2e0 con 0x560192122850 2024-09-19T08:47:23.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.310+0000 7fe4b4d84740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe49800c520 con 0x56019212b510 2024-09-19T08:47:23.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.310+0000 7fe4b4d84740 10 client.0 _ll_register_callbacks cb 0x7fe4a0009630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:47:23.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.319+0000 7fe4abfff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56019212b510 0x560192135fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:23.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.319+0000 7fe4abfff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56019212b510 0x560192135fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:42042/0 (socket says 172.21.15.164:42042) 2024-09-19T08:47:23.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.319+0000 7fe4abfff640 1 -- 172.21.15.164:0/2100356416 learned_addr learned my addr 172.21.15.164:0/2100356416 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:23.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.320+0000 7fe4ab7fe640 1 --2- 172.21.15.164:0/2100356416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019212da30 0x560192132a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:23.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.320+0000 7fe4ab7fe640 1 --2- 172.21.15.164:0/2100356416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560192122850 0x560192135a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:23.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.326+0000 7fe4ab7fe640 1 -- 172.21.15.164:0/2100356416 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56019212b510 msgr2=0x560192135fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:23.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.326+0000 7fe4ab7fe640 1 --2- 172.21.15.164:0/2100356416 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56019212b510 0x560192135fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.326+0000 7fe4ab7fe640 1 -- 172.21.15.164:0/2100356416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019212da30 msgr2=0x560192132a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:23.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.326+0000 7fe4ab7fe640 1 --2- 172.21.15.164:0/2100356416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019212da30 0x560192132a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.326+0000 7fe4ab7fe640 1 -- 172.21.15.164:0/2100356416 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560192132fa0 con 0x560192122850 2024-09-19T08:47:23.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.338+0000 7fe4ab7fe640 1 --2- 172.21.15.164:0/2100356416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560192122850 0x560192135a60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe4a00094f0 tx=0x7fe4a0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1625cc68be8729b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:23.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.338+0000 7fe48b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-19T08:47:23.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.342+0000 7fe48b7fe640 1 -- 172.21.15.164:0/2100356416 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a00041c0 con 0x560192122850 2024-09-19T08:47:23.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.342+0000 7fe48b7fe640 1 -- 172.21.15.164:0/2100356416 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fe4a0004360 con 0x560192122850 2024-09-19T08:47:23.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.344+0000 7fe4b4d84740 1 -- 172.21.15.164:0/2100356416 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4a00036b0 con 0x560192122850 2024-09-19T08:47:23.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.346+0000 7fe48b7fe640 1 -- 172.21.15.164:0/2100356416 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe4a0004560 con 0x560192122850 2024-09-19T08:47:23.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.355+0000 7fe4b4d84740 10 client.25391 fetch_fsmap learned FSMap version 28 2024-09-19T08:47:23.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.355+0000 7fe4b4d84740 1 -- 172.21.15.164:0/2100356416 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5601921a90d0 con 0x560192122850 2024-09-19T08:47:23.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.359+0000 7fe48b7fe640 1 -- 172.21.15.164:0/2100356416 <== mon.2 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4a00036b0 con 0x560192122850 2024-09-19T08:47:23.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.359+0000 7fe48b7fe640 1 -- 172.21.15.164:0/2100356416 <== mon.2 v2:172.21.15.170:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe4a00155f0 con 0x560192122850 2024-09-19T08:47:23.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.359+0000 7fe4b4d84740 10 client.25391 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:47:23.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.359+0000 7fe4b4d84740 10 client.25391 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:47:23.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.359+0000 7fe4b4d84740 20 client.25391 populate_metadata read hostname 'smithi164' 2024-09-19T08:47:23.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.359+0000 7fe4b4d84740 1 --2- 172.21.15.164:0/2100356416 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56019213db50 0x5601921ab820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:23.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.360+0000 7fe4b4d84740 4 client.25391 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}] 2024-09-19T08:47:23.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.360+0000 7fe4b4d84740 1 -- 172.21.15.164:0/2100356416 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}) -- 0x560192094890 con 0x56019213db50 2024-09-19T08:47:23.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.363+0000 7fe4abfff640 1 --2- 172.21.15.164:0/2100356416 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56019213db50 0x5601921ab820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:23.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.371+0000 7fe4abfff640 1 --2- 172.21.15.164:0/2100356416 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56019213db50 0x5601921ab820 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b72c5f262786fdb6 server_cookie=d37b3a8fb099c565 in_seq=0 out_seq=0 2024-09-19T08:47:23.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.371+0000 7fe48b7fe640 10 client.25391 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:47:23.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:22 smithi170 ceph-mon[35072]: pgmap v50: 97 pgs: 97 active+clean; 5.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 457 MiB/s wr, 347 op/s 2024-09-19T08:47:23.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:22 smithi173 ceph-mon[35306]: pgmap v50: 97 pgs: 97 active+clean; 5.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 457 MiB/s wr, 347 op/s 2024-09-19T08:47:23.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.591+0000 7fe48b7fe640 1 -- 172.21.15.164:0/2100356416 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x560192094890 con 0x56019213db50 2024-09-19T08:47:23.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.592+0000 7fe48b7fe640 10 client.25391 handle_command_reply: tid=0 2024-09-19T08:47:23.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.592+0000 7fe4b4d84740 1 client.25391 shutdown 2024-09-19T08:47:23.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.592+0000 7fe4b4d84740 1 -- 172.21.15.164:0/2100356416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560192122850 msgr2=0x560192135a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:23.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.592+0000 7fe4b4d84740 1 --2- 172.21.15.164:0/2100356416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560192122850 0x560192135a60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe4a00094f0 tx=0x7fe4a0000bc0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.592+0000 7fe4b4d84740 1 -- 172.21.15.164:0/2100356416 shutdown_connections 2024-09-19T08:47:23.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.592+0000 7fe4b4d84740 1 --2- 172.21.15.164:0/2100356416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56019212da30 0x560192132a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.592+0000 7fe4b4d84740 1 --2- 172.21.15.164:0/2100356416 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56019212b510 0x560192135fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.592+0000 7fe4b4d84740 1 --2- 172.21.15.164:0/2100356416 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56019213db50 0x5601921ab820 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.592+0000 7fe4b4d84740 1 --2- 172.21.15.164:0/2100356416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560192122850 0x560192135a60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.592+0000 7fe4b4d84740 1 -- 172.21.15.164:0/2100356416 >> 172.21.15.164:0/2100356416 conn(0x56019211ede0 msgr2=0x560192124b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:23.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.593+0000 7fe48b7fe640 0 client.25391 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:47:23.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.593+0000 7fe4b4d84740 1 -- 172.21.15.164:0/2100356416 shutdown_connections 2024-09-19T08:47:23.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.593+0000 7fe4b4d84740 1 -- 172.21.15.164:0/2100356416 wait complete. 2024-09-19T08:47:23.594 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:47:23.594 INFO:teuthology.orchestra.run.smithi164.stdout: "epoch": 28, 2024-09-19T08:47:23.594 INFO:teuthology.orchestra.run.smithi164.stdout: "leader": 24479, 2024-09-19T08:47:23.594 INFO:teuthology.orchestra.run.smithi164.stdout: "set_version": 6, 2024-09-19T08:47:23.594 INFO:teuthology.orchestra.run.smithi164.stdout: "sets": { 2024-09-19T08:47:23.595 INFO:teuthology.orchestra.run.smithi164.stdout: "96eb57aa": { 2024-09-19T08:47:23.595 INFO:teuthology.orchestra.run.smithi164.stdout: "version": 6, 2024-09-19T08:47:23.595 INFO:teuthology.orchestra.run.smithi164.stdout: "age_ref": 0.0, 2024-09-19T08:47:23.595 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:47:23.595 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "QUIESCED", 2024-09-19T08:47:23.595 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:47:23.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:47:23.595 INFO:teuthology.orchestra.run.smithi164.stdout: "timeout": 90.0, 2024-09-19T08:47:23.595 INFO:teuthology.orchestra.run.smithi164.stdout: "expiration": 140.3, 2024-09-19T08:47:23.595 INFO:teuthology.orchestra.run.smithi164.stdout: "members": { 2024-09-19T08:47:23.595 INFO:teuthology.orchestra.run.smithi164.stdout: "file:/": { 2024-09-19T08:47:23.595 INFO:teuthology.orchestra.run.smithi164.stdout: "excluded": false, 2024-09-19T08:47:23.595 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:47:23.595 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "QUIESCED", 2024-09-19T08:47:23.596 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:47:23.596 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:47:23.596 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:47:23.596 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:47:23.596 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:47:23.596 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:47:23.596 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:47:23.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.593+0000 7fe4b4d84740 20 client.25391 trim_cache size 0 max 16384 2024-09-19T08:47:23.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.595+0000 7fe4b29ab640 1 -- 172.21.15.164:0/1001464389 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe4940a35c0 msgr2=0x7fe4940a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:23.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.595+0000 7fe4b29ab640 1 --2- 172.21.15.164:0/1001464389 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe4940a35c0 0x7fe4940a5a80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe49c0c3fd0 tx=0x7fe49c0b7400 comp rx=0 tx=0).stop 2024-09-19T08:47:23.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.595+0000 7fe4b29ab640 1 -- 172.21.15.164:0/1001464389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4ac112640 msgr2=0x7fe4ac1189a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:23.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.595+0000 7fe4b29ab640 1 --2- 172.21.15.164:0/1001464389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4ac112640 0x7fe4ac1189a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe49c009860 tx=0x7fe49c033d10 comp rx=0 tx=0).stop 2024-09-19T08:47:23.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.595+0000 7fe4b29ab640 1 -- 172.21.15.164:0/1001464389 shutdown_connections 2024-09-19T08:47:23.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.595+0000 7fe4b29ab640 1 --2- 172.21.15.164:0/1001464389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4ac112640 0x7fe4ac1189a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.595+0000 7fe4b29ab640 1 --2- 172.21.15.164:0/1001464389 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe4940a35c0 0x7fe4940a5a80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.595+0000 7fe4b29ab640 1 --2- 172.21.15.164:0/1001464389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4ac10cf30 0x7fe4ac118460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.595+0000 7fe4b29ab640 1 --2- 172.21.15.164:0/1001464389 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4ac10c590 0x7fe4ac117f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.595+0000 7fe4b29ab640 1 -- 172.21.15.164:0/1001464389 >> 172.21.15.164:0/1001464389 conn(0x7fe4ac06cdc0 msgr2=0x7fe4ac10e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:23.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.595+0000 7fe4b29ab640 1 -- 172.21.15.164:0/1001464389 shutdown_connections 2024-09-19T08:47:23.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:23.595+0000 7fe4b29ab640 1 -- 172.21.15.164:0/1001464389 wait complete. 2024-09-19T08:47:24.291 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '96eb57aa', quiesce took 10.8 seconds. Will release after: 9.55 2024-09-19T08:47:25.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:25 smithi170 ceph-mon[35072]: pgmap v51: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 662 MiB/s wr, 682 op/s 2024-09-19T08:47:25.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:25 smithi173 ceph-mon[35306]: pgmap v51: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 662 MiB/s wr, 682 op/s 2024-09-19T08:47:25.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:25 smithi164 ceph-mon[30928]: pgmap v51: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 662 MiB/s wr, 682 op/s 2024-09-19T08:47:27.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:27 smithi170 ceph-mon[35072]: pgmap v52: 97 pgs: 97 active+clean; 8.9 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 741 MiB/s wr, 881 op/s 2024-09-19T08:47:27.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:27 smithi173 ceph-mon[35306]: pgmap v52: 97 pgs: 97 active+clean; 8.9 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 741 MiB/s wr, 881 op/s 2024-09-19T08:47:27.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:27 smithi164 ceph-mon[30928]: pgmap v52: 97 pgs: 97 active+clean; 8.9 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 741 MiB/s wr, 881 op/s 2024-09-19T08:47:29.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:29 smithi170 ceph-mon[35072]: pgmap v53: 97 pgs: 97 active+clean; 8.9 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 702 MiB/s wr, 810 op/s 2024-09-19T08:47:29.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:29 smithi173 ceph-mon[35306]: pgmap v53: 97 pgs: 97 active+clean; 8.9 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 702 MiB/s wr, 810 op/s 2024-09-19T08:47:29.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:29 smithi164 ceph-mon[30928]: pgmap v53: 97 pgs: 97 active+clean; 8.9 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 702 MiB/s wr, 810 op/s 2024-09-19T08:47:31.334 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:31 smithi164 ceph-mon[30928]: pgmap v54: 97 pgs: 97 active+clean; 9.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 706 MiB/s wr, 817 op/s 2024-09-19T08:47:31.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:31 smithi170 ceph-mon[35072]: pgmap v54: 97 pgs: 97 active+clean; 9.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 706 MiB/s wr, 817 op/s 2024-09-19T08:47:31.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:31 smithi173 ceph-mon[35306]: pgmap v54: 97 pgs: 97 active+clean; 9.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 706 MiB/s wr, 817 op/s 2024-09-19T08:47:33.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:33 smithi170 ceph-mon[35072]: pgmap v55: 97 pgs: 97 active+clean; 9.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 344 MiB/s wr, 557 op/s 2024-09-19T08:47:33.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:33 smithi173 ceph-mon[35306]: pgmap v55: 97 pgs: 97 active+clean; 9.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 344 MiB/s wr, 557 op/s 2024-09-19T08:47:33.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:33 smithi164 ceph-mon[30928]: pgmap v55: 97 pgs: 97 active+clean; 9.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 344 MiB/s wr, 557 op/s 2024-09-19T08:47:33.842 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:47:34.216 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:47:35.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.117+0000 7ff8b8633640 1 -- 172.21.15.164:0/4274778369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8b0101de0 msgr2=0x7ff8b01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:35.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.117+0000 7ff8b8633640 1 --2- 172.21.15.164:0/4274778369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8b0101de0 0x7ff8b01021c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0009900 tx=0x7ff8a002a410 comp rx=0 tx=0).stop 2024-09-19T08:47:35.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.117+0000 7ff8b8633640 1 -- 172.21.15.164:0/4274778369 shutdown_connections 2024-09-19T08:47:35.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.117+0000 7ff8b8633640 1 --2- 172.21.15.164:0/4274778369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8b0106f10 0x7ff8b010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:35.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.117+0000 7ff8b8633640 1 --2- 172.21.15.164:0/4274778369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8b0102790 0x7ff8b01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:35.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.117+0000 7ff8b8633640 1 --2- 172.21.15.164:0/4274778369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8b0101de0 0x7ff8b01021c0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:35.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.117+0000 7ff8b8633640 1 -- 172.21.15.164:0/4274778369 >> 172.21.15.164:0/4274778369 conn(0x7ff8b00fd8c0 msgr2=0x7ff8b00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:35.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.117+0000 7ff8b8633640 1 -- 172.21.15.164:0/4274778369 shutdown_connections 2024-09-19T08:47:35.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.117+0000 7ff8b8633640 1 -- 172.21.15.164:0/4274778369 wait complete. 2024-09-19T08:47:35.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.118+0000 7ff8b8633640 1 Processor -- start 2024-09-19T08:47:35.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.118+0000 7ff8b8633640 1 -- start start 2024-09-19T08:47:35.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.118+0000 7ff8b8633640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8b0101de0 0x7ff8b0198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:35.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.118+0000 7ff8b8633640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8b0102790 0x7ff8b01989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:35.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.118+0000 7ff8b8633640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8b0106f10 0x7ff8b019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:35.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.118+0000 7ff8b8633640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b0111f90 con 0x7ff8b0106f10 2024-09-19T08:47:35.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.118+0000 7ff8b8633640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b0111e10 con 0x7ff8b0101de0 2024-09-19T08:47:35.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.118+0000 7ff8b8633640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b0112110 con 0x7ff8b0102790 2024-09-19T08:47:35.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.118+0000 7ff8b6bd0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8b0106f10 0x7ff8b019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:35.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.118+0000 7ff8b63cf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8b0101de0 0x7ff8b0198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:35.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.118+0000 7ff8b6bd0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8b0106f10 0x7ff8b019cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49918/0 (socket says 172.21.15.164:49918) 2024-09-19T08:47:35.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.118+0000 7ff8b5bce640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8b0102790 0x7ff8b01989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:35.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.118+0000 7ff8b6bd0640 1 -- 172.21.15.164:0/1569635506 learned_addr learned my addr 172.21.15.164:0/1569635506 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:35.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.118+0000 7ff8b6bd0640 1 -- 172.21.15.164:0/1569635506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8b0102790 msgr2=0x7ff8b01989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:35.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.118+0000 7ff8b6bd0640 1 --2- 172.21.15.164:0/1569635506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8b0102790 0x7ff8b01989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:35.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.118+0000 7ff8b6bd0640 1 -- 172.21.15.164:0/1569635506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8b0101de0 msgr2=0x7ff8b0198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:35.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.118+0000 7ff8b6bd0640 1 --2- 172.21.15.164:0/1569635506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8b0101de0 0x7ff8b0198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:35.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.118+0000 7ff8b6bd0640 1 -- 172.21.15.164:0/1569635506 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b019d440 con 0x7ff8b0106f10 2024-09-19T08:47:35.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.119+0000 7ff8b6bd0640 1 --2- 172.21.15.164:0/1569635506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8b0106f10 0x7ff8b019cd60 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff8a800b970 tx=0x7ff8a800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:35.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.119+0000 7ff8a77fe640 1 -- 172.21.15.164:0/1569635506 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a8019070 con 0x7ff8b0106f10 2024-09-19T08:47:35.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.119+0000 7ff8a77fe640 1 -- 172.21.15.164:0/1569635506 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8a800cb40 con 0x7ff8b0106f10 2024-09-19T08:47:35.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.119+0000 7ff8b8633640 1 -- 172.21.15.164:0/1569635506 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b019d730 con 0x7ff8b0106f10 2024-09-19T08:47:35.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.119+0000 7ff8a77fe640 1 -- 172.21.15.164:0/1569635506 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a8007650 con 0x7ff8b0106f10 2024-09-19T08:47:35.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.119+0000 7ff8b8633640 1 -- 172.21.15.164:0/1569635506 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b01a5010 con 0x7ff8b0106f10 2024-09-19T08:47:35.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.120+0000 7ff8b8633640 1 -- 172.21.15.164:0/1569635506 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8b01021c0 con 0x7ff8b0106f10 2024-09-19T08:47:35.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.121+0000 7ff8a77fe640 1 -- 172.21.15.164:0/1569635506 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7ff8a801c070 con 0x7ff8b0106f10 2024-09-19T08:47:35.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.121+0000 7ff8a77fe640 1 --2- 172.21.15.164:0/1569635506 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff89c0a33b0 0x7ff89c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:35.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.122+0000 7ff8b63cf640 1 --2- 172.21.15.164:0/1569635506 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff89c0a33b0 0x7ff89c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:35.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.122+0000 7ff8a77fe640 1 -- 172.21.15.164:0/1569635506 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7ff8a80cf100 con 0x7ff8b0106f10 2024-09-19T08:47:35.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.122+0000 7ff8b63cf640 1 --2- 172.21.15.164:0/1569635506 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff89c0a33b0 0x7ff89c0a5870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0000c00 tx=0x7ff8a00058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:35.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.126+0000 7ff8a77fe640 1 -- 172.21.15.164:0/1569635506 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff8a809b610 con 0x7ff8b0106f10 2024-09-19T08:47:35.281 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:35 smithi164 ceph-mon[30928]: pgmap v56: 97 pgs: 97 active+clean; 9.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 354 MiB/s wr, 578 op/s 2024-09-19T08:47:35.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.327+0000 7ff8b8633640 1 -- 172.21.15.164:0/1569635506 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8b002d7c0 con 0x7ff8b0106f10 2024-09-19T08:47:35.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.328+0000 7ff8a77fe640 1 -- 172.21.15.164:0/1569635506 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11984 (secure 0 0 0) 0x7ff8a809f910 con 0x7ff8b0106f10 2024-09-19T08:47:35.328 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:47:35.328 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":28,"btime":"2024-09-19T08:45:46:903004+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:45:46.852789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:47:35.328 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 28 2024-09-19T08:47:35.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.330+0000 7ff8b8633640 1 -- 172.21.15.164:0/1569635506 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff89c0a33b0 msgr2=0x7ff89c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:35.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.330+0000 7ff8b8633640 1 --2- 172.21.15.164:0/1569635506 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff89c0a33b0 0x7ff89c0a5870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0000c00 tx=0x7ff8a00058d0 comp rx=0 tx=0).stop 2024-09-19T08:47:35.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.330+0000 7ff8b8633640 1 -- 172.21.15.164:0/1569635506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8b0106f10 msgr2=0x7ff8b019cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:35.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.330+0000 7ff8b8633640 1 --2- 172.21.15.164:0/1569635506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8b0106f10 0x7ff8b019cd60 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff8a800b970 tx=0x7ff8a800be30 comp rx=0 tx=0).stop 2024-09-19T08:47:35.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.330+0000 7ff8b8633640 1 -- 172.21.15.164:0/1569635506 shutdown_connections 2024-09-19T08:47:35.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.330+0000 7ff8b8633640 1 --2- 172.21.15.164:0/1569635506 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff89c0a33b0 0x7ff89c0a5870 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:35.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.330+0000 7ff8b8633640 1 --2- 172.21.15.164:0/1569635506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8b0106f10 0x7ff8b019cd60 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:35.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.330+0000 7ff8b8633640 1 --2- 172.21.15.164:0/1569635506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8b0102790 0x7ff8b01989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:35.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.330+0000 7ff8b8633640 1 --2- 172.21.15.164:0/1569635506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8b0101de0 0x7ff8b0198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:35.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.330+0000 7ff8b8633640 1 -- 172.21.15.164:0/1569635506 >> 172.21.15.164:0/1569635506 conn(0x7ff8b00fd8c0 msgr2=0x7ff8b00fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:35.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.330+0000 7ff8b8633640 1 -- 172.21.15.164:0/1569635506 shutdown_connections 2024-09-19T08:47:35.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:35.330+0000 7ff8b8633640 1 -- 172.21.15.164:0/1569635506 wait complete. 2024-09-19T08:47:35.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:35 smithi170 ceph-mon[35072]: pgmap v56: 97 pgs: 97 active+clean; 9.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 354 MiB/s wr, 578 op/s 2024-09-19T08:47:35.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:35 smithi173 ceph-mon[35306]: pgmap v56: 97 pgs: 97 active+clean; 9.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 354 MiB/s wr, 578 op/s 2024-09-19T08:47:35.621 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:47:36.181 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:36 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/1569635506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:47:36.208 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id 96eb57aa --release --await' 2024-09-19T08:47:36.208 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id 96eb57aa --release --await 2024-09-19T08:47:36.302 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:47:36.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:36 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/1569635506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:47:36.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:36 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/1569635506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:47:36.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.811+0000 7f63b51a6640 1 -- 172.21.15.164:0/1741099357 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63b0103740 msgr2=0x7f63b0107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:36.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.811+0000 7f63b51a6640 1 --2- 172.21.15.164:0/1741099357 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63b0103740 0x7f63b0107b80 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6398009930 tx=0x7f639802a420 comp rx=0 tx=0).stop 2024-09-19T08:47:36.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.812+0000 7f63b51a6640 1 -- 172.21.15.164:0/1741099357 shutdown_connections 2024-09-19T08:47:36.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.812+0000 7f63b51a6640 1 --2- 172.21.15.164:0/1741099357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63b0108290 0x7f63b010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.812+0000 7f63b51a6640 1 --2- 172.21.15.164:0/1741099357 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63b0103740 0x7f63b0107b80 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.812+0000 7f63b51a6640 1 --2- 172.21.15.164:0/1741099357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63b0102d90 0x7f63b0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.812+0000 7f63b51a6640 1 -- 172.21.15.164:0/1741099357 >> 172.21.15.164:0/1741099357 conn(0x7f63b00fc7d0 msgr2=0x7f63b00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:36.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.812+0000 7f63b51a6640 1 -- 172.21.15.164:0/1741099357 shutdown_connections 2024-09-19T08:47:36.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.812+0000 7f63b51a6640 1 -- 172.21.15.164:0/1741099357 wait complete. 2024-09-19T08:47:36.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.812+0000 7f63b51a6640 1 Processor -- start 2024-09-19T08:47:36.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.813+0000 7f63b51a6640 1 -- start start 2024-09-19T08:47:36.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.813+0000 7f63b51a6640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63b0102d90 0x7f63b0198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:36.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.813+0000 7f63b51a6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63b0103740 0x7f63b0198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:36.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.813+0000 7f63b51a6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63b0108290 0x7f63b019d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:36.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.813+0000 7f63b51a6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f63b0112130 con 0x7f63b0108290 2024-09-19T08:47:36.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.813+0000 7f63b51a6640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f63b0111fb0 con 0x7f63b0103740 2024-09-19T08:47:36.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.813+0000 7f63b51a6640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01122b0 con 0x7f63b0102d90 2024-09-19T08:47:36.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.813+0000 7f63af59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63b0108290 0x7f63b019d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:36.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.813+0000 7f63af59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63b0108290 0x7f63b019d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49936/0 (socket says 172.21.15.164:49936) 2024-09-19T08:47:36.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.813+0000 7f63af59e640 1 -- 172.21.15.164:0/3676580604 learned_addr learned my addr 172.21.15.164:0/3676580604 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:36.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.813+0000 7f63aed9d640 1 --2- 172.21.15.164:0/3676580604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63b0102d90 0x7f63b0198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:36.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.813+0000 7f63ae59c640 1 --2- 172.21.15.164:0/3676580604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63b0103740 0x7f63b0198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:36.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.813+0000 7f63af59e640 1 -- 172.21.15.164:0/3676580604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63b0102d90 msgr2=0x7f63b0198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:36.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.813+0000 7f63af59e640 1 --2- 172.21.15.164:0/3676580604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63b0102d90 0x7f63b0198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.813+0000 7f63af59e640 1 -- 172.21.15.164:0/3676580604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63b0103740 msgr2=0x7f63b0198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:36.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.813+0000 7f63af59e640 1 --2- 172.21.15.164:0/3676580604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63b0103740 0x7f63b0198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.813+0000 7f63af59e640 1 -- 172.21.15.164:0/3676580604 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b019d6e0 con 0x7f63b0108290 2024-09-19T08:47:36.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.814+0000 7f63af59e640 1 --2- 172.21.15.164:0/3676580604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63b0108290 0x7f63b019d000 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009fd0 tx=0x7f63a000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:36.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.814+0000 7f63a7fff640 1 -- 172.21.15.164:0/3676580604 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0011830 con 0x7f63b0108290 2024-09-19T08:47:36.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.814+0000 7f63b51a6640 1 -- 172.21.15.164:0/3676580604 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b019d9d0 con 0x7f63b0108290 2024-09-19T08:47:36.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.814+0000 7f63a7fff640 1 -- 172.21.15.164:0/3676580604 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63a0016070 con 0x7f63b0108290 2024-09-19T08:47:36.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.814+0000 7f63a7fff640 1 -- 172.21.15.164:0/3676580604 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a000f2e0 con 0x7f63b0108290 2024-09-19T08:47:36.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.814+0000 7f63b51a6640 1 -- 172.21.15.164:0/3676580604 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b01a52b0 con 0x7f63b0108290 2024-09-19T08:47:36.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.816+0000 7f63a7fff640 1 -- 172.21.15.164:0/3676580604 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f63a00119d0 con 0x7f63b0108290 2024-09-19T08:47:36.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.816+0000 7f63b757f740 1 Processor -- start 2024-09-19T08:47:36.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.816+0000 7f63b757f740 1 -- start start 2024-09-19T08:47:36.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.817+0000 7f63b757f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb76ed94a0 0x55bb76ed9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:36.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.817+0000 7f63b757f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bb76ed9dc0 0x55bb76ee4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:36.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.817+0000 7f63b757f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb76ee4680 0x55bb76ee6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:36.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.817+0000 7f63b757f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bb76e3f2e0 con 0x55bb76ee4680 2024-09-19T08:47:36.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.817+0000 7f63b757f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55bb76de6ce0 con 0x55bb76ed94a0 2024-09-19T08:47:36.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.817+0000 7f63b757f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55bb76e3b2a0 con 0x55bb76ed9dc0 2024-09-19T08:47:36.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.817+0000 7f63ae59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb76ee4680 0x55bb76ee6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:36.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.817+0000 7f63ae59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb76ee4680 0x55bb76ee6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49948/0 (socket says 172.21.15.164:49948) 2024-09-19T08:47:36.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.817+0000 7f63ae59c640 1 -- 172.21.15.164:0/1333683780 learned_addr learned my addr 172.21.15.164:0/1333683780 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:36.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.817+0000 7f63aed9d640 1 --2- 172.21.15.164:0/1333683780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bb76ed9dc0 0x55bb76ee4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:36.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.817+0000 7f63ae59c640 1 --2- 172.21.15.164:0/1333683780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb76ed94a0 0x55bb76ed9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:36.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.817+0000 7f63a7fff640 1 --2- 172.21.15.164:0/3676580604 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f639c0a33b0 0x7f639c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:36.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.817+0000 7f63af59e640 1 --2- 172.21.15.164:0/3676580604 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f639c0a33b0 0x7f639c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:36.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.817+0000 7f63aed9d640 1 -- 172.21.15.164:0/1333683780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb76ed94a0 msgr2=0x55bb76ed9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:36.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.817+0000 7f63aed9d640 1 --2- 172.21.15.164:0/1333683780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb76ed94a0 0x55bb76ed9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.817+0000 7f63aed9d640 1 -- 172.21.15.164:0/1333683780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb76ee4680 msgr2=0x55bb76ee6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:36.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.817+0000 7f63aed9d640 1 --2- 172.21.15.164:0/1333683780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb76ee4680 0x55bb76ee6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.818+0000 7f63aed9d640 1 -- 172.21.15.164:0/1333683780 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb76dcb440 con 0x55bb76ed9dc0 2024-09-19T08:47:36.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.818+0000 7f63a7fff640 1 -- 172.21.15.164:0/3676580604 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f63a00cf200 con 0x7f63b0108290 2024-09-19T08:47:36.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.818+0000 7f63aed9d640 1 --2- 172.21.15.164:0/1333683780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bb76ed9dc0 0x55bb76ee4140 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f63a800a430 tx=0x7f63a80096d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a77f01946893325 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:36.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.818+0000 7f63af59e640 1 --2- 172.21.15.164:0/3676580604 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f639c0a33b0 0x7f639c0a5870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f63a0097fd0 tx=0x7f63a006d380 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:36.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.818+0000 7f63a57fa640 1 -- 172.21.15.164:0/1333683780 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a8013a10 con 0x55bb76ed9dc0 2024-09-19T08:47:36.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.818+0000 7f63a57fa640 1 -- 172.21.15.164:0/1333683780 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63a8017070 con 0x55bb76ed9dc0 2024-09-19T08:47:36.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.818+0000 7f63a57fa640 1 -- 172.21.15.164:0/1333683780 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a801a3c0 con 0x55bb76ed9dc0 2024-09-19T08:47:36.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.819+0000 7f63b757f740 1 -- 172.21.15.164:0/1333683780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bb76ed9dc0 msgr2=0x55bb76ee4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:36.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.819+0000 7f63b757f740 1 --2- 172.21.15.164:0/1333683780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bb76ed9dc0 0x55bb76ee4140 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f63a800a430 tx=0x7f63a80096d0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.819+0000 7f63b757f740 1 -- 172.21.15.164:0/1333683780 shutdown_connections 2024-09-19T08:47:36.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.819+0000 7f63b757f740 1 --2- 172.21.15.164:0/1333683780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb76ee4680 0x55bb76ee6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.819+0000 7f63b757f740 1 --2- 172.21.15.164:0/1333683780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bb76ed9dc0 0x55bb76ee4140 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.819+0000 7f63b757f740 1 --2- 172.21.15.164:0/1333683780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb76ed94a0 0x55bb76ed9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.819+0000 7f63b757f740 1 -- 172.21.15.164:0/1333683780 >> 172.21.15.164:0/1333683780 conn(0x55bb76ed5a30 msgr2=0x55bb76ed5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:36.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.819+0000 7f63b757f740 1 -- 172.21.15.164:0/1333683780 shutdown_connections 2024-09-19T08:47:36.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.819+0000 7f63b757f740 1 -- 172.21.15.164:0/1333683780 wait complete. 2024-09-19T08:47:36.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.819+0000 7f63b757f740 1 -- start start 2024-09-19T08:47:36.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.819+0000 7f63b757f740 1 Processor -- start 2024-09-19T08:47:36.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.820+0000 7f63b757f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bb76ed94a0 0x55bb76ee3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:36.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.820+0000 7f63b757f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb76ed9dc0 0x55bb76ee11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:36.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.820+0000 7f63b757f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb76ee4680 0x55bb76ee16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:36.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.820+0000 7f63b757f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bb76ee7250 con 0x55bb76ee4680 2024-09-19T08:47:36.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.820+0000 7f63b757f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55bb76dfa050 con 0x55bb76ed9dc0 2024-09-19T08:47:36.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.820+0000 7f63b757f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55bb76ee73d0 con 0x55bb76ed94a0 2024-09-19T08:47:36.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.820+0000 7f63b757f740 10 client.0 _ll_register_callbacks cb 0x55bb76e3a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:47:36.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.820+0000 7f63ae59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb76ee4680 0x55bb76ee16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:36.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.820+0000 7f63ae59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb76ee4680 0x55bb76ee16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49964/0 (socket says 172.21.15.164:49964) 2024-09-19T08:47:36.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.820+0000 7f63ae59c640 1 -- 172.21.15.164:0/4028498738 learned_addr learned my addr 172.21.15.164:0/4028498738 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:36.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.820+0000 7f63ae59c640 1 --2- 172.21.15.164:0/4028498738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bb76ed94a0 0x55bb76ee3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:36.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.820+0000 7f63aed9d640 1 --2- 172.21.15.164:0/4028498738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb76ed9dc0 0x55bb76ee11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:36.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.820+0000 7f63ae59c640 1 -- 172.21.15.164:0/4028498738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bb76ed94a0 msgr2=0x55bb76ee3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:36.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.820+0000 7f63ae59c640 1 --2- 172.21.15.164:0/4028498738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bb76ed94a0 0x55bb76ee3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.820+0000 7f63ae59c640 1 -- 172.21.15.164:0/4028498738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb76ed9dc0 msgr2=0x55bb76ee11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:36.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.820+0000 7f63ae59c640 1 --2- 172.21.15.164:0/4028498738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb76ed9dc0 0x55bb76ee11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.820+0000 7f63ae59c640 1 -- 172.21.15.164:0/4028498738 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb76bb03c0 con 0x55bb76ee4680 2024-09-19T08:47:36.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.820+0000 7f63aed9d640 1 --2- 172.21.15.164:0/4028498738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb76ed9dc0 0x55bb76ee11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:47:36.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.820+0000 7f63ae59c640 1 --2- 172.21.15.164:0/4028498738 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb76ee4680 0x55bb76ee16f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6398009930 tx=0x7f6398041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded9cc985eaaa2a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:36.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.820+0000 7f63967fc640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:47:36.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.820+0000 7f63967fc640 1 -- 172.21.15.164:0/4028498738 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6398035030 con 0x55bb76ee4680 2024-09-19T08:47:36.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.820+0000 7f63b757f740 1 -- 172.21.15.164:0/4028498738 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63a8017070 con 0x55bb76ee4680 2024-09-19T08:47:36.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.820+0000 7f63967fc640 1 -- 172.21.15.164:0/4028498738 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f6398004180 con 0x55bb76ee4680 2024-09-19T08:47:36.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.821+0000 7f63967fc640 1 -- 172.21.15.164:0/4028498738 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6398004380 con 0x55bb76ee4680 2024-09-19T08:47:36.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.821+0000 7f63b757f740 10 client.15897 fetch_fsmap learned FSMap version 28 2024-09-19T08:47:36.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.821+0000 7f63b757f740 1 -- 172.21.15.164:0/4028498738 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bb76ef2b60 con 0x55bb76ee4680 2024-09-19T08:47:36.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.821+0000 7f63967fc640 1 -- 172.21.15.164:0/4028498738 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63a8017070 con 0x55bb76ee4680 2024-09-19T08:47:36.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.821+0000 7f63967fc640 1 -- 172.21.15.164:0/4028498738 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6398054050 con 0x55bb76ee4680 2024-09-19T08:47:36.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.821+0000 7f63b757f740 10 client.15897 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:47:36.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.821+0000 7f63b757f740 10 client.15897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:47:36.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.821+0000 7f63b757f740 20 client.15897 populate_metadata read hostname 'smithi164' 2024-09-19T08:47:36.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.821+0000 7f63b757f740 1 --2- 172.21.15.164:0/4028498738 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55bb76ef3f70 0x55bb76ef6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:36.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.821+0000 7f63b757f740 4 client.15897 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:47:36.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.821+0000 7f63b757f740 1 -- 172.21.15.164:0/4028498738 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb76e4c850 con 0x55bb76ef3f70 2024-09-19T08:47:36.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.821+0000 7f63aed9d640 1 --2- 172.21.15.164:0/4028498738 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55bb76ef3f70 0x55bb76ef6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:36.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.822+0000 7f63aed9d640 1 --2- 172.21.15.164:0/4028498738 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55bb76ef3f70 0x55bb76ef6360 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a41baef31a89a0c4 server_cookie=bec3d18d58d8cc45 in_seq=0 out_seq=0 2024-09-19T08:47:36.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.822+0000 7f63967fc640 10 client.15897 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:47:36.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.822+0000 7f63967fc640 1 -- 172.21.15.164:0/4028498738 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bb76e4c850 con 0x55bb76ef3f70 2024-09-19T08:47:36.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.822+0000 7f63967fc640 10 client.15897 handle_command_reply: tid=0 2024-09-19T08:47:36.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.822+0000 7f63b757f740 1 client.15897 shutdown 2024-09-19T08:47:36.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.822+0000 7f63b757f740 1 -- 172.21.15.164:0/4028498738 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb76ee4680 msgr2=0x55bb76ee16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:36.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.822+0000 7f63b757f740 1 --2- 172.21.15.164:0/4028498738 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb76ee4680 0x55bb76ee16f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6398009930 tx=0x7f6398041040 comp rx=0 tx=0).stop 2024-09-19T08:47:36.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.822+0000 7f63b757f740 1 -- 172.21.15.164:0/4028498738 shutdown_connections 2024-09-19T08:47:36.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.823+0000 7f63b757f740 1 --2- 172.21.15.164:0/4028498738 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb76ee4680 0x55bb76ee16f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.823+0000 7f63b757f740 1 --2- 172.21.15.164:0/4028498738 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55bb76ef3f70 0x55bb76ef6360 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.823+0000 7f63b757f740 1 --2- 172.21.15.164:0/4028498738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb76ed9dc0 0x55bb76ee11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.823+0000 7f63b757f740 1 --2- 172.21.15.164:0/4028498738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bb76ed94a0 0x55bb76ee3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.823+0000 7f63967fc640 0 client.15897 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:47:36.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.823+0000 7f63b757f740 1 -- 172.21.15.164:0/4028498738 >> 172.21.15.164:0/4028498738 conn(0x55bb76ed5a30 msgr2=0x55bb76edb710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:36.837 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.823+0000 7f63b757f740 1 -- 172.21.15.164:0/4028498738 shutdown_connections 2024-09-19T08:47:36.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.823+0000 7f63b757f740 1 -- 172.21.15.164:0/4028498738 wait complete. 2024-09-19T08:47:36.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.823+0000 7f63b757f740 20 client.15897 trim_cache size 0 max 16384 2024-09-19T08:47:36.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.840+0000 7f63b757f740 1 Processor -- start 2024-09-19T08:47:36.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.840+0000 7f63b757f740 1 -- start start 2024-09-19T08:47:36.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.840+0000 7f63b757f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb76ed94a0 0x55bb76efbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:36.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.840+0000 7f63b757f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bb76ed9dc0 0x55bb76efc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:36.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.840+0000 7f63b757f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb76ee4680 0x55bb76ede410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:36.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.840+0000 7f63b757f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01ab600 con 0x55bb76ed94a0 2024-09-19T08:47:36.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.840+0000 7f63b757f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01ad140 con 0x55bb76ee4680 2024-09-19T08:47:36.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.840+0000 7f63b757f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01abcd0 con 0x55bb76ed9dc0 2024-09-19T08:47:36.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.840+0000 7f63ae59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb76ed94a0 0x55bb76efbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:36.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.840+0000 7f63ae59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb76ed94a0 0x55bb76efbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49970/0 (socket says 172.21.15.164:49970) 2024-09-19T08:47:36.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.840+0000 7f63ae59c640 1 -- 172.21.15.164:0/1239347619 learned_addr learned my addr 172.21.15.164:0/1239347619 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:36.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.840+0000 7f63aed9d640 1 --2- 172.21.15.164:0/1239347619 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bb76ed9dc0 0x55bb76efc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:36.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.840+0000 7f63ae59c640 1 -- 172.21.15.164:0/1239347619 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bb76ed9dc0 msgr2=0x55bb76efc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:36.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.840+0000 7f63ae59c640 1 --2- 172.21.15.164:0/1239347619 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bb76ed9dc0 0x55bb76efc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.840+0000 7f63ae59c640 1 -- 172.21.15.164:0/1239347619 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb76ee4680 msgr2=0x55bb76ede410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:36.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.840+0000 7f63ae59c640 1 --2- 172.21.15.164:0/1239347619 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb76ee4680 0x55bb76ede410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.841+0000 7f63ae59c640 1 -- 172.21.15.164:0/1239347619 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb76ef66d0 con 0x55bb76ed94a0 2024-09-19T08:47:36.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.841+0000 7f63ae59c640 1 --2- 172.21.15.164:0/1239347619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb76ed94a0 0x55bb76efbbe0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6398031460 tx=0x7f6398032db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e28de9001c91b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:36.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.841+0000 7f63a4ff9640 1 -- 172.21.15.164:0/1239347619 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6398035070 con 0x55bb76ed94a0 2024-09-19T08:47:36.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.841+0000 7f63a4ff9640 1 -- 172.21.15.164:0/1239347619 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f639805a020 con 0x55bb76ed94a0 2024-09-19T08:47:36.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.841+0000 7f63a4ff9640 1 -- 172.21.15.164:0/1239347619 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6398003c90 con 0x55bb76ed94a0 2024-09-19T08:47:36.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.842+0000 7f63b757f740 1 -- 172.21.15.164:0/1239347619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb76ed94a0 msgr2=0x55bb76efbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:36.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.842+0000 7f63b757f740 1 --2- 172.21.15.164:0/1239347619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb76ed94a0 0x55bb76efbbe0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6398031460 tx=0x7f6398032db0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.842+0000 7f63b757f740 1 -- 172.21.15.164:0/1239347619 shutdown_connections 2024-09-19T08:47:36.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.842+0000 7f63b757f740 1 --2- 172.21.15.164:0/1239347619 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb76ee4680 0x55bb76ede410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.842+0000 7f63b757f740 1 --2- 172.21.15.164:0/1239347619 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bb76ed9dc0 0x55bb76efc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.842+0000 7f63b757f740 1 --2- 172.21.15.164:0/1239347619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb76ed94a0 0x55bb76efbbe0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.842+0000 7f63b757f740 1 -- 172.21.15.164:0/1239347619 >> 172.21.15.164:0/1239347619 conn(0x55bb76ed5a30 msgr2=0x55bb76ee3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:36.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.842+0000 7f63b757f740 1 -- 172.21.15.164:0/1239347619 shutdown_connections 2024-09-19T08:47:36.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.842+0000 7f63b757f740 1 -- 172.21.15.164:0/1239347619 wait complete. 2024-09-19T08:47:36.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.842+0000 7f63b757f740 1 -- start start 2024-09-19T08:47:36.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.842+0000 7f63b757f740 1 Processor -- start 2024-09-19T08:47:36.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.843+0000 7f63b757f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb76ed94a0 0x55bb76eea700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:36.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.843+0000 7f63b757f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb76ed9dc0 0x55bb76eeac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:36.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.843+0000 7f63b757f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bb76ee4680 0x55bb76eeb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:36.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.843+0000 7f63b757f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01aca70 con 0x55bb76ed9dc0 2024-09-19T08:47:36.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.843+0000 7f63b757f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01ac3a0 con 0x55bb76ed94a0 2024-09-19T08:47:36.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.843+0000 7f63b757f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01a9640 con 0x55bb76ee4680 2024-09-19T08:47:36.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.843+0000 7f63b757f740 10 client.0 _ll_register_callbacks cb 0x7f63980314b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:47:36.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.843+0000 7f63aed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb76ed9dc0 0x55bb76eeac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:36.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.843+0000 7f63aed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb76ed9dc0 0x55bb76eeac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49976/0 (socket says 172.21.15.164:49976) 2024-09-19T08:47:36.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.843+0000 7f63aed9d640 1 -- 172.21.15.164:0/832510213 learned_addr learned my addr 172.21.15.164:0/832510213 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:36.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.843+0000 7f63ae59c640 1 --2- 172.21.15.164:0/832510213 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bb76ee4680 0x55bb76eeb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:36.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.843+0000 7f63ae59c640 1 --2- 172.21.15.164:0/832510213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb76ed94a0 0x55bb76eea700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:36.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.843+0000 7f63aed9d640 1 -- 172.21.15.164:0/832510213 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bb76ee4680 msgr2=0x55bb76eeb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:36.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.843+0000 7f63aed9d640 1 --2- 172.21.15.164:0/832510213 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bb76ee4680 0x55bb76eeb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.843+0000 7f63aed9d640 1 -- 172.21.15.164:0/832510213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb76ed94a0 msgr2=0x55bb76eea700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:36.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.843+0000 7f63aed9d640 1 --2- 172.21.15.164:0/832510213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb76ed94a0 0x55bb76eea700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.843+0000 7f63aed9d640 1 -- 172.21.15.164:0/832510213 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb76f5deb0 con 0x55bb76ed9dc0 2024-09-19T08:47:36.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.843+0000 7f63ae59c640 1 --2- 172.21.15.164:0/832510213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb76ed94a0 0x55bb76eea700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:47:36.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.843+0000 7f63aed9d640 1 --2- 172.21.15.164:0/832510213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb76ed9dc0 0x55bb76eeac40 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f63a801bf40 tx=0x7f63a80098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a57df077488a186 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:36.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.843+0000 7f63967fc640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:47:36.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.843+0000 7f63967fc640 1 -- 172.21.15.164:0/832510213 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a8033060 con 0x55bb76ed9dc0 2024-09-19T08:47:36.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.843+0000 7f63967fc640 1 -- 172.21.15.164:0/832510213 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f63a8013ce0 con 0x55bb76ed9dc0 2024-09-19T08:47:36.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.843+0000 7f63b757f740 1 -- 172.21.15.164:0/832510213 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f639805a020 con 0x55bb76ed9dc0 2024-09-19T08:47:36.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.844+0000 7f63967fc640 1 -- 172.21.15.164:0/832510213 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f63a801b490 con 0x55bb76ed9dc0 2024-09-19T08:47:36.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.844+0000 7f63b757f740 10 client.15903 fetch_fsmap learned FSMap version 28 2024-09-19T08:47:36.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.844+0000 7f63b757f740 1 -- 172.21.15.164:0/832510213 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bb76f5f1c0 con 0x55bb76ed9dc0 2024-09-19T08:47:36.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.844+0000 7f63967fc640 1 -- 172.21.15.164:0/832510213 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f639805a020 con 0x55bb76ed9dc0 2024-09-19T08:47:36.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.844+0000 7f63967fc640 1 -- 172.21.15.164:0/832510213 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f63a8036020 con 0x55bb76ed9dc0 2024-09-19T08:47:36.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.844+0000 7f63b757f740 10 client.15903 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:47:36.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.844+0000 7f63b757f740 10 client.15903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:47:36.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.844+0000 7f63b757f740 20 client.15903 populate_metadata read hostname 'smithi164' 2024-09-19T08:47:36.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.844+0000 7f63b757f740 1 --2- 172.21.15.164:0/832510213 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55bb76f5f750 0x55bb76f61b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:36.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.844+0000 7f63b757f740 4 client.15903 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:47:36.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.844+0000 7f63b757f740 1 -- 172.21.15.164:0/832510213 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bb76ee1a80 con 0x55bb76f5f750 2024-09-19T08:47:36.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.844+0000 7f63ae59c640 1 --2- 172.21.15.164:0/832510213 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55bb76f5f750 0x55bb76f61b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:36.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.845+0000 7f63ae59c640 1 --2- 172.21.15.164:0/832510213 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55bb76f5f750 0x55bb76f61b40 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=220860579fb94600 server_cookie=73f1811098dc7f71 in_seq=0 out_seq=0 2024-09-19T08:47:36.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.845+0000 7f63967fc640 10 client.15903 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:47:36.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.845+0000 7f63967fc640 1 -- 172.21.15.164:0/832510213 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bb76ee1a80 con 0x55bb76f5f750 2024-09-19T08:47:36.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.845+0000 7f63967fc640 10 client.15903 handle_command_reply: tid=0 2024-09-19T08:47:36.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.845+0000 7f63b757f740 1 client.15903 shutdown 2024-09-19T08:47:36.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.845+0000 7f63b757f740 1 -- 172.21.15.164:0/832510213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb76ed9dc0 msgr2=0x55bb76eeac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:36.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.845+0000 7f63b757f740 1 --2- 172.21.15.164:0/832510213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb76ed9dc0 0x55bb76eeac40 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f63a801bf40 tx=0x7f63a80098f0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.845+0000 7f63b757f740 1 -- 172.21.15.164:0/832510213 shutdown_connections 2024-09-19T08:47:36.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.845+0000 7f63b757f740 1 --2- 172.21.15.164:0/832510213 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bb76ee4680 0x55bb76eeb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.845+0000 7f63b757f740 1 --2- 172.21.15.164:0/832510213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb76ed9dc0 0x55bb76eeac40 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.845+0000 7f63b757f740 1 --2- 172.21.15.164:0/832510213 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55bb76f5f750 0x55bb76f61b40 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.845+0000 7f63b757f740 1 --2- 172.21.15.164:0/832510213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb76ed94a0 0x55bb76eea700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.845+0000 7f63967fc640 0 client.15903 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:47:36.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.845+0000 7f63b757f740 1 -- 172.21.15.164:0/832510213 >> 172.21.15.164:0/832510213 conn(0x55bb76ed5a30 msgr2=0x55bb76ed4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:36.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.845+0000 7f63b757f740 1 -- 172.21.15.164:0/832510213 shutdown_connections 2024-09-19T08:47:36.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.845+0000 7f63b757f740 1 -- 172.21.15.164:0/832510213 wait complete. 2024-09-19T08:47:36.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.845+0000 7f63b757f740 20 client.15903 trim_cache size 0 max 16384 2024-09-19T08:47:36.853 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:47:36.853 INFO:teuthology.orchestra.run.smithi164.stdout: "status": "no active scrubs running", 2024-09-19T08:47:36.853 INFO:teuthology.orchestra.run.smithi164.stdout: "scrubs": {} 2024-09-19T08:47:36.853 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:47:36.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.846+0000 7f63b51a6640 1 -- 172.21.15.164:0/3676580604 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f639c0a33b0 msgr2=0x7f639c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:36.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.846+0000 7f63b51a6640 1 --2- 172.21.15.164:0/3676580604 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f639c0a33b0 0x7f639c0a5870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f63a0097fd0 tx=0x7f63a006d380 comp rx=0 tx=0).stop 2024-09-19T08:47:36.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.846+0000 7f63b51a6640 1 -- 172.21.15.164:0/3676580604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63b0108290 msgr2=0x7f63b019d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:36.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.846+0000 7f63b51a6640 1 --2- 172.21.15.164:0/3676580604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63b0108290 0x7f63b019d000 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009fd0 tx=0x7f63a000b8e0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.846+0000 7f63b51a6640 1 -- 172.21.15.164:0/3676580604 shutdown_connections 2024-09-19T08:47:36.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.846+0000 7f63b51a6640 1 --2- 172.21.15.164:0/3676580604 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f639c0a33b0 0x7f639c0a5870 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.846+0000 7f63b51a6640 1 --2- 172.21.15.164:0/3676580604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63b0108290 0x7f63b019d000 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.846+0000 7f63b51a6640 1 --2- 172.21.15.164:0/3676580604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63b0103740 0x7f63b0198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.846+0000 7f63b51a6640 1 --2- 172.21.15.164:0/3676580604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63b0102d90 0x7f63b0198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.846+0000 7f63b51a6640 1 -- 172.21.15.164:0/3676580604 >> 172.21.15.164:0/3676580604 conn(0x7f63b00fc7d0 msgr2=0x7f63b0105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:36.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.847+0000 7f63b51a6640 1 -- 172.21.15.164:0/3676580604 shutdown_connections 2024-09-19T08:47:36.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:36.847+0000 7f63b51a6640 1 -- 172.21.15.164:0/3676580604 wait complete. 2024-09-19T08:47:37.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:37 smithi170 ceph-mon[35072]: pgmap v57: 97 pgs: 97 active+clean; 9.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 156 MiB/s wr, 266 op/s 2024-09-19T08:47:37.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:37 smithi173 ceph-mon[35306]: pgmap v57: 97 pgs: 97 active+clean; 9.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 156 MiB/s wr, 266 op/s 2024-09-19T08:47:37.473 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:37 smithi164 ceph-mon[30928]: pgmap v57: 97 pgs: 97 active+clean; 9.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 156 MiB/s wr, 266 op/s 2024-09-19T08:47:37.499 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:47:37.501 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T08:47:37.501 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:47:38.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.138+0000 7fe41447b640 1 -- 172.21.15.164:0/2661407442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe40c102790 msgr2=0x7fe40c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:38.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.138+0000 7fe41447b640 1 --2- 172.21.15.164:0/2661407442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe40c102790 0x7fe40c1067e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc009900 tx=0x7fe3fc02a410 comp rx=0 tx=0).stop 2024-09-19T08:47:38.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.138+0000 7fe41447b640 1 -- 172.21.15.164:0/2661407442 shutdown_connections 2024-09-19T08:47:38.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.138+0000 7fe41447b640 1 --2- 172.21.15.164:0/2661407442 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe40c106f10 0x7fe40c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.138+0000 7fe41447b640 1 --2- 172.21.15.164:0/2661407442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe40c102790 0x7fe40c1067e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.138+0000 7fe41447b640 1 --2- 172.21.15.164:0/2661407442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe40c101de0 0x7fe40c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.138+0000 7fe41447b640 1 -- 172.21.15.164:0/2661407442 >> 172.21.15.164:0/2661407442 conn(0x7fe40c0fd8c0 msgr2=0x7fe40c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:38.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.138+0000 7fe41447b640 1 -- 172.21.15.164:0/2661407442 shutdown_connections 2024-09-19T08:47:38.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.138+0000 7fe41447b640 1 -- 172.21.15.164:0/2661407442 wait complete. 2024-09-19T08:47:38.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.139+0000 7fe41447b640 1 Processor -- start 2024-09-19T08:47:38.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.139+0000 7fe41447b640 1 -- start start 2024-09-19T08:47:38.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.139+0000 7fe41447b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe40c101de0 0x7fe40c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.139+0000 7fe41447b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe40c102790 0x7fe40c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.139+0000 7fe41447b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe40c106f10 0x7fe40c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.139+0000 7fe41447b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c111f30 con 0x7fe40c106f10 2024-09-19T08:47:38.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.139+0000 7fe41447b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c111db0 con 0x7fe40c101de0 2024-09-19T08:47:38.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.139+0000 7fe41447b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c1120b0 con 0x7fe40c102790 2024-09-19T08:47:38.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.139+0000 7fe412a18640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe40c106f10 0x7fe40c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.139+0000 7fe412a18640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe40c106f10 0x7fe40c19cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50002/0 (socket says 172.21.15.164:50002) 2024-09-19T08:47:38.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.139+0000 7fe412a18640 1 -- 172.21.15.164:0/415449111 learned_addr learned my addr 172.21.15.164:0/415449111 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:38.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.139+0000 7fe412217640 1 --2- 172.21.15.164:0/415449111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe40c101de0 0x7fe40c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.139+0000 7fe411a16640 1 --2- 172.21.15.164:0/415449111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe40c102790 0x7fe40c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.139+0000 7fe412a18640 1 -- 172.21.15.164:0/415449111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe40c102790 msgr2=0x7fe40c1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:38.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.139+0000 7fe412a18640 1 --2- 172.21.15.164:0/415449111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe40c102790 0x7fe40c1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.139+0000 7fe412a18640 1 -- 172.21.15.164:0/415449111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe40c101de0 msgr2=0x7fe40c198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:38.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.139+0000 7fe412a18640 1 --2- 172.21.15.164:0/415449111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe40c101de0 0x7fe40c198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.139+0000 7fe412a18640 1 -- 172.21.15.164:0/415449111 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe40c19d340 con 0x7fe40c106f10 2024-09-19T08:47:38.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.139+0000 7fe411a16640 1 --2- 172.21.15.164:0/415449111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe40c102790 0x7fe40c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:38.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.140+0000 7fe412a18640 1 --2- 172.21.15.164:0/415449111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe40c106f10 0x7fe40c19cc60 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe40400cce0 tx=0x7fe40400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:38.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.140+0000 7fe4037fe640 1 -- 172.21.15.164:0/415449111 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40400f450 con 0x7fe40c106f10 2024-09-19T08:47:38.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.140+0000 7fe4037fe640 1 -- 172.21.15.164:0/415449111 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe40400fad0 con 0x7fe40c106f10 2024-09-19T08:47:38.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.140+0000 7fe41447b640 1 -- 172.21.15.164:0/415449111 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe40c19d630 con 0x7fe40c106f10 2024-09-19T08:47:38.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.140+0000 7fe4037fe640 1 -- 172.21.15.164:0/415449111 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe404018c10 con 0x7fe40c106f10 2024-09-19T08:47:38.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.140+0000 7fe41447b640 1 -- 172.21.15.164:0/415449111 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe40c1a4f10 con 0x7fe40c106f10 2024-09-19T08:47:38.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.143+0000 7fe4037fe640 1 -- 172.21.15.164:0/415449111 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fe40400f5f0 con 0x7fe40c106f10 2024-09-19T08:47:38.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.143+0000 7fe416854740 1 Processor -- start 2024-09-19T08:47:38.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.143+0000 7fe416854740 1 -- start start 2024-09-19T08:47:38.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.143+0000 7fe416854740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd8dd0f060 0x55cd8dd0f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.143+0000 7fe416854740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8dd0f980 0x55cd8dd19d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.143+0000 7fe416854740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8dd1a240 0x55cd8dd1c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.143+0000 7fe416854740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cd8dc74620 con 0x55cd8dd1a240 2024-09-19T08:47:38.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.143+0000 7fe416854740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55cd8dc1be50 con 0x55cd8dd0f980 2024-09-19T08:47:38.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.143+0000 7fe416854740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55cd8dc705e0 con 0x55cd8dd0f060 2024-09-19T08:47:38.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.144+0000 7fe411a16640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8dd1a240 0x55cd8dd1c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.144+0000 7fe411a16640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8dd1a240 0x55cd8dd1c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50014/0 (socket says 172.21.15.164:50014) 2024-09-19T08:47:38.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.144+0000 7fe411a16640 1 -- 172.21.15.164:0/3020756068 learned_addr learned my addr 172.21.15.164:0/3020756068 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:38.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.144+0000 7fe412217640 1 --2- 172.21.15.164:0/3020756068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8dd0f980 0x55cd8dd19d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.144+0000 7fe411a16640 1 --2- 172.21.15.164:0/3020756068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd8dd0f060 0x55cd8dd0f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.144+0000 7fe4037fe640 1 --2- 172.21.15.164:0/415449111 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe3f80a33b0 0x7fe3f80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.144+0000 7fe411a16640 1 -- 172.21.15.164:0/3020756068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd8dd0f060 msgr2=0x55cd8dd0f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:38.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.144+0000 7fe411a16640 1 --2- 172.21.15.164:0/3020756068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd8dd0f060 0x55cd8dd0f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.144+0000 7fe411a16640 1 -- 172.21.15.164:0/3020756068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8dd0f980 msgr2=0x55cd8dd19d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:38.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.144+0000 7fe411a16640 1 --2- 172.21.15.164:0/3020756068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8dd0f980 0x55cd8dd19d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.144+0000 7fe411a16640 1 -- 172.21.15.164:0/3020756068 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd8dc0cd40 con 0x55cd8dd1a240 2024-09-19T08:47:38.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.144+0000 7fe412a18640 1 --2- 172.21.15.164:0/415449111 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe3f80a33b0 0x7fe3f80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.144+0000 7fe4037fe640 1 -- 172.21.15.164:0/415449111 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fe4040d03a0 con 0x7fe40c106f10 2024-09-19T08:47:38.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.144+0000 7fe411a16640 1 --2- 172.21.15.164:0/3020756068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8dd1a240 0x55cd8dd1c630 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc032fb0 tx=0x7fe3fc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a263eff214aee5b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:38.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.144+0000 7fe400ff9640 1 -- 172.21.15.164:0/3020756068 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc035040 con 0x55cd8dd1a240 2024-09-19T08:47:38.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.144+0000 7fe400ff9640 1 -- 172.21.15.164:0/3020756068 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3fc045070 con 0x55cd8dd1a240 2024-09-19T08:47:38.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.144+0000 7fe400ff9640 1 -- 172.21.15.164:0/3020756068 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc003bc0 con 0x55cd8dd1a240 2024-09-19T08:47:38.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.145+0000 7fe416854740 1 -- 172.21.15.164:0/3020756068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8dd1a240 msgr2=0x55cd8dd1c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:38.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.145+0000 7fe412a18640 1 --2- 172.21.15.164:0/415449111 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe3f80a33b0 0x7fe3f80a5870 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe4040cdc10 tx=0x7fe404091b80 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:38.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.145+0000 7fe416854740 1 --2- 172.21.15.164:0/3020756068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8dd1a240 0x55cd8dd1c630 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc032fb0 tx=0x7fe3fc000bc0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.145+0000 7fe416854740 1 -- 172.21.15.164:0/3020756068 shutdown_connections 2024-09-19T08:47:38.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.145+0000 7fe416854740 1 --2- 172.21.15.164:0/3020756068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8dd1a240 0x55cd8dd1c630 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.145+0000 7fe416854740 1 --2- 172.21.15.164:0/3020756068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8dd0f980 0x55cd8dd19d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.145+0000 7fe416854740 1 --2- 172.21.15.164:0/3020756068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd8dd0f060 0x55cd8dd0f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.145+0000 7fe416854740 1 -- 172.21.15.164:0/3020756068 >> 172.21.15.164:0/3020756068 conn(0x55cd8dd0b5f0 msgr2=0x55cd8dd0b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:38.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.145+0000 7fe416854740 1 -- 172.21.15.164:0/3020756068 shutdown_connections 2024-09-19T08:47:38.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.145+0000 7fe416854740 1 -- 172.21.15.164:0/3020756068 wait complete. 2024-09-19T08:47:38.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.146+0000 7fe416854740 1 -- start start 2024-09-19T08:47:38.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.146+0000 7fe416854740 1 Processor -- start 2024-09-19T08:47:38.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.146+0000 7fe416854740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8dd0f060 0x55cd8dd17130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.146+0000 7fe416854740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8dd0f980 0x55cd8dd17670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.146+0000 7fe416854740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd8dd1a240 0x55cd8dd17bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.146+0000 7fe416854740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cd8dd21220 con 0x55cd8dd0f060 2024-09-19T08:47:38.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.146+0000 7fe416854740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55cd8dbfcf10 con 0x55cd8dd0f980 2024-09-19T08:47:38.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.146+0000 7fe416854740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55cd8dd213a0 con 0x55cd8dd1a240 2024-09-19T08:47:38.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.146+0000 7fe416854740 10 client.0 _ll_register_callbacks cb 0x55cd8dc20db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:47:38.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.146+0000 7fe411a16640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8dd0f060 0x55cd8dd17130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.146+0000 7fe411a16640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8dd0f060 0x55cd8dd17130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50030/0 (socket says 172.21.15.164:50030) 2024-09-19T08:47:38.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.146+0000 7fe411a16640 1 -- 172.21.15.164:0/1106031669 learned_addr learned my addr 172.21.15.164:0/1106031669 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:38.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.146+0000 7fe412217640 1 --2- 172.21.15.164:0/1106031669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8dd0f980 0x55cd8dd17670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.146+0000 7fe411a16640 1 -- 172.21.15.164:0/1106031669 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd8dd1a240 msgr2=0x55cd8dd17bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:47:38.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.146+0000 7fe411a16640 1 --2- 172.21.15.164:0/1106031669 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd8dd1a240 0x55cd8dd17bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.146+0000 7fe411a16640 1 -- 172.21.15.164:0/1106031669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8dd0f980 msgr2=0x55cd8dd17670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:38.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.146+0000 7fe411a16640 1 --2- 172.21.15.164:0/1106031669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8dd0f980 0x55cd8dd17670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.146+0000 7fe411a16640 1 -- 172.21.15.164:0/1106031669 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd8d983580 con 0x55cd8dd0f060 2024-09-19T08:47:38.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.147+0000 7fe411a16640 1 --2- 172.21.15.164:0/1106031669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8dd0f060 0x55cd8dd17130 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc031400 tx=0x7fe3fc031ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23bd82ec4ec7442d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:38.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.147+0000 7fe3f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:47:38.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.147+0000 7fe3f5ffb640 1 -- 172.21.15.164:0/1106031669 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc04d070 con 0x55cd8dd0f060 2024-09-19T08:47:38.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.147+0000 7fe3f5ffb640 1 -- 172.21.15.164:0/1106031669 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fe3fc03caa0 con 0x55cd8dd0f060 2024-09-19T08:47:38.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.147+0000 7fe416854740 1 -- 172.21.15.164:0/1106031669 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3fc045070 con 0x55cd8dd0f060 2024-09-19T08:47:38.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.147+0000 7fe3f5ffb640 1 -- 172.21.15.164:0/1106031669 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe3fc059050 con 0x55cd8dd0f060 2024-09-19T08:47:38.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.147+0000 7fe416854740 10 client.15915 fetch_fsmap learned FSMap version 28 2024-09-19T08:47:38.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.147+0000 7fe416854740 1 -- 172.21.15.164:0/1106031669 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cd8da4d960 con 0x55cd8dd0f060 2024-09-19T08:47:38.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.147+0000 7fe3f5ffb640 1 -- 172.21.15.164:0/1106031669 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3fc045070 con 0x55cd8dd0f060 2024-09-19T08:47:38.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.147+0000 7fe3f5ffb640 1 -- 172.21.15.164:0/1106031669 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe3fc03a050 con 0x55cd8dd0f060 2024-09-19T08:47:38.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.147+0000 7fe416854740 10 client.15915 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:47:38.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.147+0000 7fe416854740 10 client.15915 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:47:38.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.147+0000 7fe416854740 20 client.15915 populate_metadata read hostname 'smithi164' 2024-09-19T08:47:38.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.147+0000 7fe416854740 1 --2- 172.21.15.164:0/1106031669 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55cd8dd2e050 0x55cd8dd30440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.147+0000 7fe416854740 4 client.15915 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:47:38.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.147+0000 7fe416854740 1 -- 172.21.15.164:0/1106031669 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd8dc81b90 con 0x55cd8dd2e050 2024-09-19T08:47:38.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.148+0000 7fe412217640 1 --2- 172.21.15.164:0/1106031669 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55cd8dd2e050 0x55cd8dd30440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.148+0000 7fe412217640 1 --2- 172.21.15.164:0/1106031669 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55cd8dd2e050 0x55cd8dd30440 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=170e4e157c17ec04 server_cookie=69fadc12012453c in_seq=0 out_seq=0 2024-09-19T08:47:38.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.148+0000 7fe3f5ffb640 10 client.15915 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:47:38.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.149+0000 7fe3f5ffb640 1 -- 172.21.15.164:0/1106031669 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cd8dc81b90 con 0x55cd8dd2e050 2024-09-19T08:47:38.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.149+0000 7fe3f5ffb640 10 client.15915 handle_command_reply: tid=0 2024-09-19T08:47:38.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.149+0000 7fe416854740 1 client.15915 shutdown 2024-09-19T08:47:38.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.149+0000 7fe416854740 1 -- 172.21.15.164:0/1106031669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8dd0f060 msgr2=0x55cd8dd17130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:38.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.149+0000 7fe416854740 1 --2- 172.21.15.164:0/1106031669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8dd0f060 0x55cd8dd17130 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc031400 tx=0x7fe3fc031ab0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.149+0000 7fe416854740 1 -- 172.21.15.164:0/1106031669 shutdown_connections 2024-09-19T08:47:38.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.149+0000 7fe416854740 1 --2- 172.21.15.164:0/1106031669 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd8dd1a240 0x55cd8dd17bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.149+0000 7fe416854740 1 --2- 172.21.15.164:0/1106031669 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55cd8dd2e050 0x55cd8dd30440 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.149+0000 7fe416854740 1 --2- 172.21.15.164:0/1106031669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8dd0f980 0x55cd8dd17670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.149+0000 7fe416854740 1 --2- 172.21.15.164:0/1106031669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8dd0f060 0x55cd8dd17130 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.149+0000 7fe416854740 1 -- 172.21.15.164:0/1106031669 >> 172.21.15.164:0/1106031669 conn(0x55cd8dd0b5f0 msgr2=0x55cd8dd112d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:38.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.149+0000 7fe3f5ffb640 0 client.15915 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:47:38.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.149+0000 7fe416854740 1 -- 172.21.15.164:0/1106031669 shutdown_connections 2024-09-19T08:47:38.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.149+0000 7fe416854740 1 -- 172.21.15.164:0/1106031669 wait complete. 2024-09-19T08:47:38.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.149+0000 7fe416854740 20 client.15915 trim_cache size 0 max 16384 2024-09-19T08:47:38.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.166+0000 7fe416854740 1 Processor -- start 2024-09-19T08:47:38.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.166+0000 7fe416854740 1 -- start start 2024-09-19T08:47:38.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.166+0000 7fe416854740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8dd0f060 0x55cd8dd19000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.166+0000 7fe416854740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd8dd0f980 0x55cd8dd143b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.166+0000 7fe416854740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8dd1a240 0x55cd8dd148f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.166+0000 7fe416854740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c1ab420 con 0x55cd8dd1a240 2024-09-19T08:47:38.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.166+0000 7fe416854740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c1acf60 con 0x55cd8dd0f060 2024-09-19T08:47:38.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.166+0000 7fe416854740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c1abaf0 con 0x55cd8dd0f980 2024-09-19T08:47:38.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.166+0000 7fe411a16640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8dd1a240 0x55cd8dd148f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.166+0000 7fe411a16640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8dd1a240 0x55cd8dd148f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50038/0 (socket says 172.21.15.164:50038) 2024-09-19T08:47:38.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.166+0000 7fe411a16640 1 -- 172.21.15.164:0/2317185132 learned_addr learned my addr 172.21.15.164:0/2317185132 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:38.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.166+0000 7fe411a16640 1 --2- 172.21.15.164:0/2317185132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8dd0f060 0x55cd8dd19000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.166+0000 7fe412217640 1 --2- 172.21.15.164:0/2317185132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd8dd0f980 0x55cd8dd143b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.166+0000 7fe411a16640 1 -- 172.21.15.164:0/2317185132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd8dd0f980 msgr2=0x55cd8dd143b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:38.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.166+0000 7fe411a16640 1 --2- 172.21.15.164:0/2317185132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd8dd0f980 0x55cd8dd143b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.166+0000 7fe411a16640 1 -- 172.21.15.164:0/2317185132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8dd0f060 msgr2=0x55cd8dd19000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:38.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.166+0000 7fe411a16640 1 --2- 172.21.15.164:0/2317185132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8dd0f060 0x55cd8dd19000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.166+0000 7fe411a16640 1 -- 172.21.15.164:0/2317185132 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd8dd0f7b0 con 0x55cd8dd1a240 2024-09-19T08:47:38.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.166+0000 7fe412217640 1 --2- 172.21.15.164:0/2317185132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd8dd0f980 0x55cd8dd143b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:47:38.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.167+0000 7fe411a16640 1 --2- 172.21.15.164:0/2317185132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8dd1a240 0x55cd8dd148f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x55cd8dd2ce50 tx=0x7fe3fc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9554fff9a9b993d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:38.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.167+0000 7fe3f7fff640 1 -- 172.21.15.164:0/2317185132 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc04d070 con 0x55cd8dd1a240 2024-09-19T08:47:38.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.167+0000 7fe3f7fff640 1 -- 172.21.15.164:0/2317185132 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3fc045030 con 0x55cd8dd1a240 2024-09-19T08:47:38.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.167+0000 7fe3f7fff640 1 -- 172.21.15.164:0/2317185132 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc035040 con 0x55cd8dd1a240 2024-09-19T08:47:38.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.167+0000 7fe416854740 1 -- 172.21.15.164:0/2317185132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8dd1a240 msgr2=0x55cd8dd148f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:38.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.167+0000 7fe416854740 1 --2- 172.21.15.164:0/2317185132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8dd1a240 0x55cd8dd148f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x55cd8dd2ce50 tx=0x7fe3fc000bc0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.167+0000 7fe416854740 1 -- 172.21.15.164:0/2317185132 shutdown_connections 2024-09-19T08:47:38.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.167+0000 7fe416854740 1 --2- 172.21.15.164:0/2317185132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8dd1a240 0x55cd8dd148f0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.167+0000 7fe416854740 1 --2- 172.21.15.164:0/2317185132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd8dd0f980 0x55cd8dd143b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.167+0000 7fe416854740 1 --2- 172.21.15.164:0/2317185132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8dd0f060 0x55cd8dd19000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.167+0000 7fe416854740 1 -- 172.21.15.164:0/2317185132 >> 172.21.15.164:0/2317185132 conn(0x55cd8dd0b5f0 msgr2=0x55cd8dd29e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:38.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.167+0000 7fe416854740 1 -- 172.21.15.164:0/2317185132 shutdown_connections 2024-09-19T08:47:38.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.168+0000 7fe416854740 1 -- 172.21.15.164:0/2317185132 wait complete. 2024-09-19T08:47:38.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.168+0000 7fe416854740 1 -- start start 2024-09-19T08:47:38.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.168+0000 7fe416854740 1 Processor -- start 2024-09-19T08:47:38.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.168+0000 7fe416854740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8dd0f060 0x55cd8dd31e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.168+0000 7fe416854740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd8dd0f980 0x55cd8dd32350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.168+0000 7fe416854740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8dd1a240 0x55cd8dd332b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.168+0000 7fe416854740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c1ac890 con 0x55cd8dd1a240 2024-09-19T08:47:38.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.168+0000 7fe416854740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c1ac1c0 con 0x55cd8dd0f060 2024-09-19T08:47:38.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.168+0000 7fe416854740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c1a9420 con 0x55cd8dd0f980 2024-09-19T08:47:38.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.168+0000 7fe416854740 10 client.0 _ll_register_callbacks cb 0x7fe3fc0319e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:47:38.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.168+0000 7fe411a16640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8dd1a240 0x55cd8dd332b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.168+0000 7fe411a16640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8dd1a240 0x55cd8dd332b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50044/0 (socket says 172.21.15.164:50044) 2024-09-19T08:47:38.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.168+0000 7fe411a16640 1 -- 172.21.15.164:0/2112231915 learned_addr learned my addr 172.21.15.164:0/2112231915 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:38.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.168+0000 7fe412217640 1 --2- 172.21.15.164:0/2112231915 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd8dd0f980 0x55cd8dd32350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.168+0000 7fe411a16640 1 --2- 172.21.15.164:0/2112231915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8dd0f060 0x55cd8dd31e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.169+0000 7fe411a16640 1 -- 172.21.15.164:0/2112231915 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd8dd0f980 msgr2=0x55cd8dd32350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:38.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.169+0000 7fe411a16640 1 --2- 172.21.15.164:0/2112231915 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd8dd0f980 0x55cd8dd32350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.169+0000 7fe411a16640 1 -- 172.21.15.164:0/2112231915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8dd0f060 msgr2=0x55cd8dd31e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:38.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.169+0000 7fe411a16640 1 --2- 172.21.15.164:0/2112231915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8dd0f060 0x55cd8dd31e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.169+0000 7fe411a16640 1 -- 172.21.15.164:0/2112231915 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd8dd33ae0 con 0x55cd8dd1a240 2024-09-19T08:47:38.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.169+0000 7fe411a16640 1 --2- 172.21.15.164:0/2112231915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8dd1a240 0x55cd8dd332b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc049c90 tx=0x7fe3fc009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f238a7df03b95f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:38.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.169+0000 7fe3f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:47:38.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.169+0000 7fe3f5ffb640 1 -- 172.21.15.164:0/2112231915 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc035070 con 0x55cd8dd1a240 2024-09-19T08:47:38.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.169+0000 7fe3f5ffb640 1 -- 172.21.15.164:0/2112231915 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fe3fc031a90 con 0x55cd8dd1a240 2024-09-19T08:47:38.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.169+0000 7fe416854740 1 -- 172.21.15.164:0/2112231915 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3fc045030 con 0x55cd8dd1a240 2024-09-19T08:47:38.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.169+0000 7fe3f5ffb640 1 -- 172.21.15.164:0/2112231915 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe3fc04d040 con 0x55cd8dd1a240 2024-09-19T08:47:38.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.169+0000 7fe416854740 10 client.15921 fetch_fsmap learned FSMap version 28 2024-09-19T08:47:38.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.169+0000 7fe416854740 1 -- 172.21.15.164:0/2112231915 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cd8dd995b0 con 0x55cd8dd1a240 2024-09-19T08:47:38.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.169+0000 7fe3f5ffb640 1 -- 172.21.15.164:0/2112231915 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3fc045030 con 0x55cd8dd1a240 2024-09-19T08:47:38.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.170+0000 7fe3f5ffb640 1 -- 172.21.15.164:0/2112231915 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe3fc066020 con 0x55cd8dd1a240 2024-09-19T08:47:38.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.170+0000 7fe416854740 10 client.15921 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:47:38.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.170+0000 7fe416854740 10 client.15921 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:47:38.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.170+0000 7fe416854740 20 client.15921 populate_metadata read hostname 'smithi164' 2024-09-19T08:47:38.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.170+0000 7fe416854740 1 --2- 172.21.15.164:0/2112231915 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55cd8dd99c90 0x55cd8dd9c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.170+0000 7fe416854740 4 client.15921 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "96eb57aa", "release": true, "await": true}] 2024-09-19T08:47:38.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.170+0000 7fe416854740 1 -- 172.21.15.164:0/2112231915 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "quiesce db", "set_id": "96eb57aa", "release": true, "await": true}) -- 0x55cd8dc7c0a0 con 0x55cd8dd99c90 2024-09-19T08:47:38.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.170+0000 7fe412217640 1 --2- 172.21.15.164:0/2112231915 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55cd8dd99c90 0x55cd8dd9c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.170+0000 7fe3f5ffb640 10 client.15921 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:47:38.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.170+0000 7fe412217640 1 --2- 172.21.15.164:0/2112231915 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55cd8dd99c90 0x55cd8dd9c0b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=333c1a5790db1ba5 server_cookie=306c0e914ab15f87 in_seq=0 out_seq=0 2024-09-19T08:47:38.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.174+0000 7fe3f5ffb640 1 -- 172.21.15.164:0/2112231915 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55cd8dc7c0a0 con 0x55cd8dd99c90 2024-09-19T08:47:38.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.174+0000 7fe3f5ffb640 10 client.15921 handle_command_reply: tid=0 2024-09-19T08:47:38.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.174+0000 7fe416854740 1 client.15921 shutdown 2024-09-19T08:47:38.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.174+0000 7fe416854740 1 -- 172.21.15.164:0/2112231915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8dd1a240 msgr2=0x55cd8dd332b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:38.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.174+0000 7fe416854740 1 --2- 172.21.15.164:0/2112231915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8dd1a240 0x55cd8dd332b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc049c90 tx=0x7fe3fc009640 comp rx=0 tx=0).stop 2024-09-19T08:47:38.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.175+0000 7fe416854740 1 -- 172.21.15.164:0/2112231915 shutdown_connections 2024-09-19T08:47:38.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.175+0000 7fe416854740 1 --2- 172.21.15.164:0/2112231915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8dd1a240 0x55cd8dd332b0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.175+0000 7fe416854740 1 --2- 172.21.15.164:0/2112231915 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd8dd0f980 0x55cd8dd32350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.175+0000 7fe416854740 1 --2- 172.21.15.164:0/2112231915 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55cd8dd99c90 0x55cd8dd9c0b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.175+0000 7fe3f5ffb640 0 client.15921 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:47:38.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.175+0000 7fe416854740 1 --2- 172.21.15.164:0/2112231915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8dd0f060 0x55cd8dd31e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.175+0000 7fe416854740 1 -- 172.21.15.164:0/2112231915 >> 172.21.15.164:0/2112231915 conn(0x55cd8dd0b5f0 msgr2=0x55cd8dd295f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:38.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.175+0000 7fe416854740 1 -- 172.21.15.164:0/2112231915 shutdown_connections 2024-09-19T08:47:38.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.175+0000 7fe416854740 1 -- 172.21.15.164:0/2112231915 wait complete. 2024-09-19T08:47:38.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.175+0000 7fe416854740 20 client.15921 trim_cache size 0 max 16384 2024-09-19T08:47:38.323 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:47:38.323 INFO:teuthology.orchestra.run.smithi164.stdout: "epoch": 28, 2024-09-19T08:47:38.323 INFO:teuthology.orchestra.run.smithi164.stdout: "leader": 24479, 2024-09-19T08:47:38.323 INFO:teuthology.orchestra.run.smithi164.stdout: "set_version": 8, 2024-09-19T08:47:38.323 INFO:teuthology.orchestra.run.smithi164.stdout: "sets": { 2024-09-19T08:47:38.323 INFO:teuthology.orchestra.run.smithi164.stdout: "96eb57aa": { 2024-09-19T08:47:38.323 INFO:teuthology.orchestra.run.smithi164.stdout: "version": 8, 2024-09-19T08:47:38.323 INFO:teuthology.orchestra.run.smithi164.stdout: "age_ref": 0.0, 2024-09-19T08:47:38.323 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:47:38.323 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "RELEASED", 2024-09-19T08:47:38.323 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:47:38.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:47:38.324 INFO:teuthology.orchestra.run.smithi164.stdout: "timeout": 90.0, 2024-09-19T08:47:38.324 INFO:teuthology.orchestra.run.smithi164.stdout: "expiration": 140.3, 2024-09-19T08:47:38.324 INFO:teuthology.orchestra.run.smithi164.stdout: "members": { 2024-09-19T08:47:38.324 INFO:teuthology.orchestra.run.smithi164.stdout: "file:/": { 2024-09-19T08:47:38.324 INFO:teuthology.orchestra.run.smithi164.stdout: "excluded": false, 2024-09-19T08:47:38.324 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:47:38.324 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "RELEASED", 2024-09-19T08:47:38.324 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:47:38.324 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:47:38.324 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:47:38.324 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:47:38.324 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:47:38.325 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:47:38.325 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:47:38.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.176+0000 7fe41447b640 1 -- 172.21.15.164:0/415449111 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe3f80a33b0 msgr2=0x7fe3f80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:38.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.176+0000 7fe41447b640 1 --2- 172.21.15.164:0/415449111 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe3f80a33b0 0x7fe3f80a5870 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe4040cdc10 tx=0x7fe404091b80 comp rx=0 tx=0).stop 2024-09-19T08:47:38.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.176+0000 7fe41447b640 1 -- 172.21.15.164:0/415449111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe40c106f10 msgr2=0x7fe40c19cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:38.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.176+0000 7fe41447b640 1 --2- 172.21.15.164:0/415449111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe40c106f10 0x7fe40c19cc60 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe40400cce0 tx=0x7fe40400a620 comp rx=0 tx=0).stop 2024-09-19T08:47:38.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.176+0000 7fe41447b640 1 -- 172.21.15.164:0/415449111 shutdown_connections 2024-09-19T08:47:38.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.176+0000 7fe41447b640 1 --2- 172.21.15.164:0/415449111 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe3f80a33b0 0x7fe3f80a5870 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.176+0000 7fe41447b640 1 --2- 172.21.15.164:0/415449111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe40c106f10 0x7fe40c19cc60 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.176+0000 7fe41447b640 1 --2- 172.21.15.164:0/415449111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe40c102790 0x7fe40c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.176+0000 7fe41447b640 1 --2- 172.21.15.164:0/415449111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe40c101de0 0x7fe40c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.176+0000 7fe41447b640 1 -- 172.21.15.164:0/415449111 >> 172.21.15.164:0/415449111 conn(0x7fe40c0fd8c0 msgr2=0x7fe40c0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:38.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.176+0000 7fe41447b640 1 -- 172.21.15.164:0/415449111 shutdown_connections 2024-09-19T08:47:38.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:38.176+0000 7fe41447b640 1 -- 172.21.15.164:0/415449111 wait complete. 2024-09-19T08:47:38.720 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:47:39.285 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:39 smithi164 ceph-mon[30928]: pgmap v58: 97 pgs: 97 active+clean; 9.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 21 MiB/s wr, 50 op/s 2024-09-19T08:47:39.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:39 smithi170 ceph-mon[35072]: pgmap v58: 97 pgs: 97 active+clean; 9.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 21 MiB/s wr, 50 op/s 2024-09-19T08:47:39.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:39 smithi173 ceph-mon[35306]: pgmap v58: 97 pgs: 97 active+clean; 9.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 21 MiB/s wr, 50 op/s 2024-09-19T08:47:41.132 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '96eb57aa', total seconds elapsed: 27.6 2024-09-19T08:47:41.133 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.35 2024-09-19T08:47:41.133 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:47:41.225 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:41 smithi164 ceph-mon[30928]: pgmap v59: 97 pgs: 97 active+clean; 9.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 104 MiB/s wr, 709 op/s 2024-09-19T08:47:41.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:41 smithi170 ceph-mon[35072]: pgmap v59: 97 pgs: 97 active+clean; 9.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 104 MiB/s wr, 709 op/s 2024-09-19T08:47:41.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:41 smithi173 ceph-mon[35306]: pgmap v59: 97 pgs: 97 active+clean; 9.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 104 MiB/s wr, 709 op/s 2024-09-19T08:47:42.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.604+0000 7fe14136e640 1 -- 172.21.15.164:0/2484891688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe13c102ef0 msgr2=0x7fe13c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:42.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.604+0000 7fe14136e640 1 --2- 172.21.15.164:0/2484891688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe13c102ef0 0x7fe13c10da40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe134009c10 tx=0x7fe13402a790 comp rx=0 tx=0).stop 2024-09-19T08:47:42.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.604+0000 7fe14136e640 1 -- 172.21.15.164:0/2484891688 shutdown_connections 2024-09-19T08:47:42.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.604+0000 7fe14136e640 1 --2- 172.21.15.164:0/2484891688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe13c102ef0 0x7fe13c10da40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:42.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.604+0000 7fe14136e640 1 --2- 172.21.15.164:0/2484891688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe13c102570 0x7fe13c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:42.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.604+0000 7fe14136e640 1 --2- 172.21.15.164:0/2484891688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe13c100e20 0x7fe13c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:42.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.604+0000 7fe14136e640 1 -- 172.21.15.164:0/2484891688 >> 172.21.15.164:0/2484891688 conn(0x7fe13c0fc7d0 msgr2=0x7fe13c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:42.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.605+0000 7fe14136e640 1 -- 172.21.15.164:0/2484891688 shutdown_connections 2024-09-19T08:47:42.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.605+0000 7fe14136e640 1 -- 172.21.15.164:0/2484891688 wait complete. 2024-09-19T08:47:42.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.608+0000 7fe14136e640 1 Processor -- start 2024-09-19T08:47:42.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.609+0000 7fe14136e640 1 -- start start 2024-09-19T08:47:42.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.609+0000 7fe14136e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe13c100e20 0x7fe13c1974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:42.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.610+0000 7fe14136e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe13c102570 0x7fe13c1979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:42.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.610+0000 7fe14136e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe13c102ef0 0x7fe13c19bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:42.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.610+0000 7fe14136e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe13c110fc0 con 0x7fe13c102ef0 2024-09-19T08:47:42.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.610+0000 7fe14136e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe13c110e40 con 0x7fe13c102570 2024-09-19T08:47:42.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.610+0000 7fe14136e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe13c111140 con 0x7fe13c100e20 2024-09-19T08:47:42.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.610+0000 7fe13affd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe13c100e20 0x7fe13c1974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:42.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.611+0000 7fe13affd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe13c100e20 0x7fe13c1974a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:51374/0 (socket says 172.21.15.164:51374) 2024-09-19T08:47:42.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.611+0000 7fe13affd640 1 -- 172.21.15.164:0/195114688 learned_addr learned my addr 172.21.15.164:0/195114688 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:42.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.611+0000 7fe13a7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe13c102570 0x7fe13c1979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:42.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.612+0000 7fe13b7fe640 1 --2- 172.21.15.164:0/195114688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe13c102ef0 0x7fe13c19bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:42.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.613+0000 7fe13a7fc640 1 -- 172.21.15.164:0/195114688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe13c100e20 msgr2=0x7fe13c1974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:42.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.613+0000 7fe13a7fc640 1 --2- 172.21.15.164:0/195114688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe13c100e20 0x7fe13c1974a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:42.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.613+0000 7fe13a7fc640 1 -- 172.21.15.164:0/195114688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe13c102ef0 msgr2=0x7fe13c19bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:42.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.613+0000 7fe13a7fc640 1 --2- 172.21.15.164:0/195114688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe13c102ef0 0x7fe13c19bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:42.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.613+0000 7fe13a7fc640 1 -- 172.21.15.164:0/195114688 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe13c19c450 con 0x7fe13c102570 2024-09-19T08:47:42.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.614+0000 7fe13a7fc640 1 --2- 172.21.15.164:0/195114688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe13c102570 0x7fe13c1979e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe12c00e790 tx=0x7fe12c00ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:42.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.618+0000 7fe123fff640 1 -- 172.21.15.164:0/195114688 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c00eea0 con 0x7fe13c102570 2024-09-19T08:47:42.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.618+0000 7fe14136e640 1 -- 172.21.15.164:0/195114688 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe13c19c740 con 0x7fe13c102570 2024-09-19T08:47:42.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.618+0000 7fe14136e640 1 -- 172.21.15.164:0/195114688 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe13c1a4020 con 0x7fe13c102570 2024-09-19T08:47:42.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.618+0000 7fe123fff640 1 -- 172.21.15.164:0/195114688 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe12c00ccc0 con 0x7fe13c102570 2024-09-19T08:47:42.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.619+0000 7fe123fff640 1 -- 172.21.15.164:0/195114688 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c018750 con 0x7fe13c102570 2024-09-19T08:47:42.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.632+0000 7fe123fff640 1 -- 172.21.15.164:0/195114688 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fe12c0188f0 con 0x7fe13c102570 2024-09-19T08:47:42.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.633+0000 7fe123fff640 1 --2- 172.21.15.164:0/195114688 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe1240a35c0 0x7fe1240a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:42.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.634+0000 7fe123fff640 1 -- 172.21.15.164:0/195114688 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fe12c0cfcb0 con 0x7fe13c102570 2024-09-19T08:47:42.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.634+0000 7fe13affd640 1 --2- 172.21.15.164:0/195114688 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe1240a35c0 0x7fe1240a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:42.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.635+0000 7fe13affd640 1 --2- 172.21.15.164:0/195114688 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe1240a35c0 0x7fe1240a5a80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe130009a30 tx=0x7fe130009190 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:42.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.644+0000 7fe14136e640 1 -- 172.21.15.164:0/195114688 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe128005290 con 0x7fe13c102570 2024-09-19T08:47:42.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:42.661+0000 7fe123fff640 1 -- 172.21.15.164:0/195114688 <== mon.2 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe12c09c050 con 0x7fe13c102570 2024-09-19T08:47:42.946 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:47:43.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:43.172+0000 7fe14136e640 1 -- 172.21.15.164:0/195114688 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe128005850 con 0x7fe13c102570 2024-09-19T08:47:43.175 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:47:43.175 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":28,"btime":"2024-09-19T08:45:46:903004+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:45:46.852789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:47:43.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:43.173+0000 7fe123fff640 1 -- 172.21.15.164:0/195114688 <== mon.2 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11984 (secure 0 0 0) 0x7fe12c0a0350 con 0x7fe13c102570 2024-09-19T08:47:43.176 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 28 2024-09-19T08:47:43.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:43.187+0000 7fe121ffb640 1 -- 172.21.15.164:0/195114688 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe1240a35c0 msgr2=0x7fe1240a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:43.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:43.187+0000 7fe121ffb640 1 --2- 172.21.15.164:0/195114688 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe1240a35c0 0x7fe1240a5a80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe130009a30 tx=0x7fe130009190 comp rx=0 tx=0).stop 2024-09-19T08:47:43.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:43.187+0000 7fe121ffb640 1 -- 172.21.15.164:0/195114688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe13c102570 msgr2=0x7fe13c1979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:43.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:43.187+0000 7fe121ffb640 1 --2- 172.21.15.164:0/195114688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe13c102570 0x7fe13c1979e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe12c00e790 tx=0x7fe12c00ec50 comp rx=0 tx=0).stop 2024-09-19T08:47:43.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:43.187+0000 7fe121ffb640 1 -- 172.21.15.164:0/195114688 shutdown_connections 2024-09-19T08:47:43.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:43.188+0000 7fe121ffb640 1 --2- 172.21.15.164:0/195114688 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe1240a35c0 0x7fe1240a5a80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:43.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:43.188+0000 7fe121ffb640 1 --2- 172.21.15.164:0/195114688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe13c102ef0 0x7fe13c19bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:43.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:43.188+0000 7fe121ffb640 1 --2- 172.21.15.164:0/195114688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe13c102570 0x7fe13c1979e0 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe12c00e790 tx=0x7fe12c00ec50 comp rx=0 tx=0).stop 2024-09-19T08:47:43.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:43.188+0000 7fe121ffb640 1 --2- 172.21.15.164:0/195114688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe13c100e20 0x7fe13c1974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:43.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:43.188+0000 7fe121ffb640 1 -- 172.21.15.164:0/195114688 >> 172.21.15.164:0/195114688 conn(0x7fe13c0fc7d0 msgr2=0x7fe13c1048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:43.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:43.192+0000 7fe121ffb640 1 -- 172.21.15.164:0/195114688 shutdown_connections 2024-09-19T08:47:43.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:43.195+0000 7fe121ffb640 1 -- 172.21.15.164:0/195114688 wait complete. 2024-09-19T08:47:43.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:43 smithi170 ceph-mon[35072]: pgmap v60: 97 pgs: 97 active+clean; 9.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 100 MiB/s wr, 702 op/s 2024-09-19T08:47:43.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:43 smithi173 ceph-mon[35306]: pgmap v60: 97 pgs: 97 active+clean; 9.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 100 MiB/s wr, 702 op/s 2024-09-19T08:47:43.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:43 smithi164 ceph-mon[30928]: pgmap v60: 97 pgs: 97 active+clean; 9.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 100 MiB/s wr, 702 op/s 2024-09-19T08:47:44.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:44 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/195114688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:47:44.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:44 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/195114688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:47:44.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:44 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/195114688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:47:45.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:45 smithi173 ceph-mon[35306]: pgmap v61: 97 pgs: 97 active+clean; 9.8 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 307 MiB/s wr, 1.77k op/s 2024-09-19T08:47:45.463 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:45 smithi164 ceph-mon[30928]: pgmap v61: 97 pgs: 97 active+clean; 9.8 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 307 MiB/s wr, 1.77k op/s 2024-09-19T08:47:45.526 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:45 smithi170 ceph-mon[35072]: pgmap v61: 97 pgs: 97 active+clean; 9.8 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 307 MiB/s wr, 1.77k op/s 2024-09-19T08:47:46.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.495+0000 7fd20b59e640 1 -- 172.21.15.164:0/9209757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd20c101270 msgr2=0x7fd20c069f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:46.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.495+0000 7fd20b59e640 1 --2- 172.21.15.164:0/9209757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd20c101270 0x7fd20c069f70 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8009900 tx=0x7fd1f802a440 comp rx=0 tx=0).stop 2024-09-19T08:47:46.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.496+0000 7fd20b59e640 1 -- 172.21.15.164:0/9209757 shutdown_connections 2024-09-19T08:47:46.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.496+0000 7fd20b59e640 1 --2- 172.21.15.164:0/9209757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd20c101270 0x7fd20c069f70 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:46.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.496+0000 7fd20b59e640 1 --2- 172.21.15.164:0/9209757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd20c1008d0 0x7fd20c100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:46.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.496+0000 7fd20b59e640 1 --2- 172.21.15.164:0/9209757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd20c1068d0 0x7fd20c106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:46.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.496+0000 7fd20b59e640 1 -- 172.21.15.164:0/9209757 >> 172.21.15.164:0/9209757 conn(0x7fd20c0fc610 msgr2=0x7fd20c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:46.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.500+0000 7fd20b59e640 1 -- 172.21.15.164:0/9209757 shutdown_connections 2024-09-19T08:47:46.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.501+0000 7fd20b59e640 1 -- 172.21.15.164:0/9209757 wait complete. 2024-09-19T08:47:46.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.502+0000 7fd20b59e640 1 Processor -- start 2024-09-19T08:47:46.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.502+0000 7fd20b59e640 1 -- start start 2024-09-19T08:47:46.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.502+0000 7fd20b59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd20c1008d0 0x7fd20c110c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:46.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.503+0000 7fd20b59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd20c101270 0x7fd20c111170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:46.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.503+0000 7fd20b59e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd20c1068d0 0x7fd20c1116b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:46.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.503+0000 7fd20b59e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd20c06bf40 con 0x7fd20c1008d0 2024-09-19T08:47:46.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.503+0000 7fd20b59e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd20c06bdc0 con 0x7fd20c101270 2024-09-19T08:47:46.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.503+0000 7fd20b59e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd20c06c0c0 con 0x7fd20c1068d0 2024-09-19T08:47:46.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.505+0000 7fd20a59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd20c1008d0 0x7fd20c110c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:46.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.505+0000 7fd20ad9d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd20c1068d0 0x7fd20c1116b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:46.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.505+0000 7fd20ad9d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd20c1068d0 0x7fd20c1116b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:55228/0 (socket says 172.21.15.164:55228) 2024-09-19T08:47:46.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.505+0000 7fd20ad9d640 1 -- 172.21.15.164:0/1920091585 learned_addr learned my addr 172.21.15.164:0/1920091585 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:46.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.506+0000 7fd209d9b640 1 --2- 172.21.15.164:0/1920091585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd20c101270 0x7fd20c111170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:46.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.506+0000 7fd20a59c640 1 -- 172.21.15.164:0/1920091585 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd20c1068d0 msgr2=0x7fd20c1116b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:46.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.507+0000 7fd20a59c640 1 --2- 172.21.15.164:0/1920091585 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd20c1068d0 0x7fd20c1116b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:46.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.507+0000 7fd20a59c640 1 -- 172.21.15.164:0/1920091585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd20c101270 msgr2=0x7fd20c111170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:46.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.507+0000 7fd20a59c640 1 --2- 172.21.15.164:0/1920091585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd20c101270 0x7fd20c111170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:46.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.507+0000 7fd20a59c640 1 -- 172.21.15.164:0/1920091585 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd20c073a50 con 0x7fd20c1008d0 2024-09-19T08:47:46.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.508+0000 7fd20a59c640 1 --2- 172.21.15.164:0/1920091585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd20c1008d0 0x7fd20c110c30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009fd0 tx=0x7fd1fc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:46.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.510+0000 7fd2037fe640 1 -- 172.21.15.164:0/1920091585 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc015020 con 0x7fd20c1008d0 2024-09-19T08:47:46.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.511+0000 7fd2037fe640 1 -- 172.21.15.164:0/1920091585 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1fc011d10 con 0x7fd20c1008d0 2024-09-19T08:47:46.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.511+0000 7fd2037fe640 1 -- 172.21.15.164:0/1920091585 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc00f2e0 con 0x7fd20c1008d0 2024-09-19T08:47:46.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.512+0000 7fd20b59e640 1 -- 172.21.15.164:0/1920091585 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd20c073ce0 con 0x7fd20c1008d0 2024-09-19T08:47:46.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.512+0000 7fd20b59e640 1 -- 172.21.15.164:0/1920091585 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd20c074140 con 0x7fd20c1008d0 2024-09-19T08:47:46.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.516+0000 7fd20b59e640 1 -- 172.21.15.164:0/1920091585 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd20c102010 con 0x7fd20c1008d0 2024-09-19T08:47:46.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.519+0000 7fd2037fe640 1 -- 172.21.15.164:0/1920091585 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fd1fc018020 con 0x7fd20c1008d0 2024-09-19T08:47:46.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.521+0000 7fd2037fe640 1 --2- 172.21.15.164:0/1920091585 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd1f00a35c0 0x7fd1f00a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:46.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.521+0000 7fd209d9b640 1 --2- 172.21.15.164:0/1920091585 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd1f00a35c0 0x7fd1f00a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:46.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.525+0000 7fd2037fe640 1 -- 172.21.15.164:0/1920091585 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fd1fc0cf9b0 con 0x7fd20c1008d0 2024-09-19T08:47:46.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.528+0000 7fd209d9b640 1 --2- 172.21.15.164:0/1920091585 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd1f00a35c0 0x7fd1f00a5a80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd20c1123a0 tx=0x7fd2040072f0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:46.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:46.532+0000 7fd2037fe640 1 -- 172.21.15.164:0/1920091585 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd1fc09c530 con 0x7fd20c1008d0 2024-09-19T08:47:46.869 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T08:47:47.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:47.093+0000 7fd20b59e640 1 -- 172.21.15.164:0/1920091585 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd20c069ff0 con 0x7fd20c1008d0 2024-09-19T08:47:47.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:47.096+0000 7fd2037fe640 1 -- 172.21.15.164:0/1920091585 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11984 (secure 0 0 0) 0x7fd20c069ff0 con 0x7fd20c1008d0 2024-09-19T08:47:47.097 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:47:47.097 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":28,"btime":"2024-09-19T08:45:46:903004+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:45:46.852789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:47:47.097 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 28 2024-09-19T08:47:47.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:47.103+0000 7fd20b59e640 1 -- 172.21.15.164:0/1920091585 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd1f00a35c0 msgr2=0x7fd1f00a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:47.103+0000 7fd20b59e640 1 --2- 172.21.15.164:0/1920091585 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd1f00a35c0 0x7fd1f00a5a80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd20c1123a0 tx=0x7fd2040072f0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:47.103+0000 7fd20b59e640 1 -- 172.21.15.164:0/1920091585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd20c1008d0 msgr2=0x7fd20c110c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:47.103+0000 7fd20b59e640 1 --2- 172.21.15.164:0/1920091585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd20c1008d0 0x7fd20c110c30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009fd0 tx=0x7fd1fc00b870 comp rx=0 tx=0).stop 2024-09-19T08:47:47.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:47.104+0000 7fd20b59e640 1 -- 172.21.15.164:0/1920091585 shutdown_connections 2024-09-19T08:47:47.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:47.104+0000 7fd20b59e640 1 --2- 172.21.15.164:0/1920091585 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd20c1068d0 0x7fd20c1116b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:47.104+0000 7fd20b59e640 1 --2- 172.21.15.164:0/1920091585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd20c101270 0x7fd20c111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:47.104+0000 7fd20b59e640 1 --2- 172.21.15.164:0/1920091585 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd1f00a35c0 0x7fd1f00a5a80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:47.104+0000 7fd20b59e640 1 --2- 172.21.15.164:0/1920091585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd20c1008d0 0x7fd20c110c30 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:47.105+0000 7fd20b59e640 1 -- 172.21.15.164:0/1920091585 >> 172.21.15.164:0/1920091585 conn(0x7fd20c0fc610 msgr2=0x7fd20c10e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:47.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:47.105+0000 7fd20b59e640 1 -- 172.21.15.164:0/1920091585 shutdown_connections 2024-09-19T08:47:47.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:47.105+0000 7fd20b59e640 1 -- 172.21.15.164:0/1920091585 wait complete. 2024-09-19T08:47:47.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:47 smithi173 ceph-mon[35306]: pgmap v62: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 424 MiB/s wr, 2.40k op/s 2024-09-19T08:47:47.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:47 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/1920091585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:47:47.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:47 smithi164 ceph-mon[30928]: pgmap v62: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 424 MiB/s wr, 2.40k op/s 2024-09-19T08:47:47.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:47 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/1920091585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:47:47.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:47 smithi170 ceph-mon[35072]: pgmap v62: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 424 MiB/s wr, 2.40k op/s 2024-09-19T08:47:47.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:47 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/1920091585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:47:48.969 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0' 2024-09-19T08:47:48.969 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0 2024-09-19T08:47:49.187 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:49 smithi164 ceph-mon[30928]: pgmap v63: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 417 MiB/s wr, 2.38k op/s 2024-09-19T08:47:49.283 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:47:49.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:49 smithi170 ceph-mon[35072]: pgmap v63: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 417 MiB/s wr, 2.38k op/s 2024-09-19T08:47:49.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:49 smithi173 ceph-mon[35306]: pgmap v63: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 417 MiB/s wr, 2.38k op/s 2024-09-19T08:47:51.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.936+0000 7fc70134b640 1 -- 172.21.15.164:0/1714978255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6fc102ef0 msgr2=0x7fc6fc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:51.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.936+0000 7fc70134b640 1 --2- 172.21.15.164:0/1714978255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6fc102ef0 0x7fc6fc10da40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4009cd0 tx=0x7fc6f402a9c0 comp rx=0 tx=0).stop 2024-09-19T08:47:51.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.938+0000 7fc70134b640 1 -- 172.21.15.164:0/1714978255 shutdown_connections 2024-09-19T08:47:51.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.938+0000 7fc70134b640 1 --2- 172.21.15.164:0/1714978255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6fc102ef0 0x7fc6fc10da40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:51.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.938+0000 7fc70134b640 1 --2- 172.21.15.164:0/1714978255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6fc102570 0x7fc6fc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:51.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.938+0000 7fc70134b640 1 --2- 172.21.15.164:0/1714978255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6fc100e20 0x7fc6fc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:51.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.938+0000 7fc70134b640 1 -- 172.21.15.164:0/1714978255 >> 172.21.15.164:0/1714978255 conn(0x7fc6fc0fc7d0 msgr2=0x7fc6fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:51.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.945+0000 7fc70134b640 1 -- 172.21.15.164:0/1714978255 shutdown_connections 2024-09-19T08:47:51.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.948+0000 7fc70134b640 1 -- 172.21.15.164:0/1714978255 wait complete. 2024-09-19T08:47:51.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.949+0000 7fc70134b640 1 Processor -- start 2024-09-19T08:47:51.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.950+0000 7fc70134b640 1 -- start start 2024-09-19T08:47:51.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.951+0000 7fc70134b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6fc100e20 0x7fc6fc107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:51.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.951+0000 7fc70134b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6fc102570 0x7fc6fc108020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:51.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.951+0000 7fc70134b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6fc102ef0 0x7fc6fc108560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:51.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.951+0000 7fc70134b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc6fc110fc0 con 0x7fc6fc102ef0 2024-09-19T08:47:51.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.951+0000 7fc70134b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc6fc110e40 con 0x7fc6fc102570 2024-09-19T08:47:51.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.951+0000 7fc70134b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc6fc111140 con 0x7fc6fc100e20 2024-09-19T08:47:51.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.957+0000 7fc6faffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6fc100e20 0x7fc6fc107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:51.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.957+0000 7fc6fb7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6fc102ef0 0x7fc6fc108560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:51.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.957+0000 7fc6fb7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6fc102ef0 0x7fc6fc108560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57766/0 (socket says 172.21.15.164:57766) 2024-09-19T08:47:51.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.957+0000 7fc6fb7fe640 1 -- 172.21.15.164:0/2991700107 learned_addr learned my addr 172.21.15.164:0/2991700107 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:51.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.957+0000 7fc6fa7fc640 1 --2- 172.21.15.164:0/2991700107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6fc102570 0x7fc6fc108020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:51.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.961+0000 7fc6fa7fc640 1 -- 172.21.15.164:0/2991700107 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6fc100e20 msgr2=0x7fc6fc107ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:51.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.961+0000 7fc6fa7fc640 1 --2- 172.21.15.164:0/2991700107 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6fc100e20 0x7fc6fc107ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:51.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.961+0000 7fc6fa7fc640 1 -- 172.21.15.164:0/2991700107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6fc102ef0 msgr2=0x7fc6fc108560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:51.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.961+0000 7fc6fa7fc640 1 --2- 172.21.15.164:0/2991700107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6fc102ef0 0x7fc6fc108560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:51.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.961+0000 7fc6fa7fc640 1 -- 172.21.15.164:0/2991700107 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6fc19c6c0 con 0x7fc6fc102570 2024-09-19T08:47:51.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.963+0000 7fc6fa7fc640 1 --2- 172.21.15.164:0/2991700107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6fc102570 0x7fc6fc108020 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc6e400d6d0 tx=0x7fc6e400db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:51.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.964+0000 7fc6f3fff640 1 -- 172.21.15.164:0/2991700107 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4017070 con 0x7fc6fc102570 2024-09-19T08:47:51.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.964+0000 7fc6f3fff640 1 -- 172.21.15.164:0/2991700107 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6e4009820 con 0x7fc6fc102570 2024-09-19T08:47:51.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.964+0000 7fc6f3fff640 1 -- 172.21.15.164:0/2991700107 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4004d00 con 0x7fc6fc102570 2024-09-19T08:47:51.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.966+0000 7fc70134b640 1 -- 172.21.15.164:0/2991700107 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6fc19c9b0 con 0x7fc6fc102570 2024-09-19T08:47:51.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.966+0000 7fc70134b640 1 -- 172.21.15.164:0/2991700107 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6fc19cea0 con 0x7fc6fc102570 2024-09-19T08:47:51.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.980+0000 7fc703724740 1 Processor -- start 2024-09-19T08:47:51.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.980+0000 7fc703724740 1 -- start start 2024-09-19T08:47:51.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.981+0000 7fc703724740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4257d3b40 0x55d4257d2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:51.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.981+0000 7fc703724740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4257d25d0 0x55d4257d2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:51.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.981+0000 7fc703724740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4257dd370 0x55d4257df760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:51.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.981+0000 7fc703724740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d4257382f0 con 0x55d4257d25d0 2024-09-19T08:47:51.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.981+0000 7fc703724740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d4256dfcf0 con 0x55d4257dd370 2024-09-19T08:47:51.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.981+0000 7fc703724740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d4257342b0 con 0x55d4257d3b40 2024-09-19T08:47:51.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.984+0000 7fc6fb7fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4257d3b40 0x55d4257d2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:51.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.987+0000 7fc6fa7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4257dd370 0x55d4257df760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:51.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.988+0000 7fc6fb7fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4257d3b40 0x55d4257d2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:55260/0 (socket says 172.21.15.164:55260) 2024-09-19T08:47:51.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.988+0000 7fc6fb7fe640 1 -- 172.21.15.164:0/717289564 learned_addr learned my addr 172.21.15.164:0/717289564 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:51.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.988+0000 7fc6f3fff640 1 -- 172.21.15.164:0/2991700107 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fc6e4005220 con 0x7fc6fc102570 2024-09-19T08:47:51.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.989+0000 7fc6f3fff640 1 --2- 172.21.15.164:0/2991700107 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fc6e80a35c0 0x7fc6e80a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:51.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.990+0000 7fc6f3fff640 1 -- 172.21.15.164:0/2991700107 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fc6e40cf580 con 0x7fc6fc102570 2024-09-19T08:47:51.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.990+0000 7fc6fb7fe640 1 -- 172.21.15.164:0/717289564 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4257dd370 msgr2=0x55d4257df760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:51.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.990+0000 7fc6fb7fe640 1 --2- 172.21.15.164:0/717289564 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4257dd370 0x55d4257df760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:51.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.990+0000 7fc6fb7fe640 1 -- 172.21.15.164:0/717289564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4257d25d0 msgr2=0x55d4257d2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:51.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.990+0000 7fc6fb7fe640 1 --2- 172.21.15.164:0/717289564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4257d25d0 0x55d4257d2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:51.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.990+0000 7fc6fb7fe640 1 -- 172.21.15.164:0/717289564 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4256c43a0 con 0x55d4257d3b40 2024-09-19T08:47:51.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.991+0000 7fc6fb7fe640 1 --2- 172.21.15.164:0/2991700107 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fc6e80a35c0 0x7fc6e80a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:51.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.992+0000 7fc6fb7fe640 1 --2- 172.21.15.164:0/717289564 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4257d3b40 0x55d4257d2090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4005c40 tx=0x7fc6f402fc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=818b096f9809a9b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:51.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.992+0000 7fc6fb7fe640 1 --2- 172.21.15.164:0/2991700107 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fc6e80a35c0 0x7fc6e80a5a80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4030040 tx=0x7fc6f4046000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:51.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.996+0000 7fc6f17fa640 1 -- 172.21.15.164:0/717289564 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4034040 con 0x55d4257d3b40 2024-09-19T08:47:51.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.996+0000 7fc6f17fa640 1 -- 172.21.15.164:0/717289564 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6f4046950 con 0x55d4257d3b40 2024-09-19T08:47:51.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.997+0000 7fc703724740 1 -- 172.21.15.164:0/717289564 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4257d3b40 msgr2=0x55d4257d2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:51.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:50.997+0000 7fc703724740 1 --2- 172.21.15.164:0/717289564 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4257d3b40 0x55d4257d2090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4005c40 tx=0x7fc6f402fc60 comp rx=0 tx=0).stop 2024-09-19T08:47:51.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.002+0000 7fc703724740 1 -- 172.21.15.164:0/717289564 shutdown_connections 2024-09-19T08:47:51.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.002+0000 7fc703724740 1 --2- 172.21.15.164:0/717289564 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4257dd370 0x55d4257df760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:51.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.002+0000 7fc703724740 1 --2- 172.21.15.164:0/717289564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4257d25d0 0x55d4257d2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:51.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.002+0000 7fc703724740 1 --2- 172.21.15.164:0/717289564 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4257d3b40 0x55d4257d2090 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:51.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.002+0000 7fc703724740 1 -- 172.21.15.164:0/717289564 >> 172.21.15.164:0/717289564 conn(0x55d42579b310 msgr2=0x55d4257cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:51.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.002+0000 7fc703724740 1 -- 172.21.15.164:0/717289564 shutdown_connections 2024-09-19T08:47:51.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.002+0000 7fc703724740 1 -- 172.21.15.164:0/717289564 wait complete. 2024-09-19T08:47:51.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.002+0000 7fc703724740 1 -- start start 2024-09-19T08:47:51.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.003+0000 7fc703724740 1 Processor -- start 2024-09-19T08:47:51.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.003+0000 7fc703724740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4257d25d0 0x55d4257dcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:51.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.003+0000 7fc703724740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4257d3b40 0x55d4257d9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:51.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.003+0000 7fc703724740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4257dd370 0x55d4257da440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:51.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.003+0000 7fc703724740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d4257e0060 con 0x55d4257dd370 2024-09-19T08:47:51.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.003+0000 7fc703724740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d4256f3060 con 0x55d4257d25d0 2024-09-19T08:47:51.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.003+0000 7fc703724740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d4257e01e0 con 0x55d4257d3b40 2024-09-19T08:47:51.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.003+0000 7fc703724740 10 client.0 _ll_register_callbacks cb 0x55d4254468c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:47:51.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.006+0000 7fc6faffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4257d25d0 0x55d4257dcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:51.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.006+0000 7fc6faffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4257d25d0 0x55d4257dcc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:41492/0 (socket says 172.21.15.164:41492) 2024-09-19T08:47:51.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.006+0000 7fc6faffd640 1 -- 172.21.15.164:0/3869113483 learned_addr learned my addr 172.21.15.164:0/3869113483 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:51.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.006+0000 7fc6fb7fe640 1 --2- 172.21.15.164:0/3869113483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4257dd370 0x55d4257da440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:51.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.006+0000 7fc6faffd640 1 -- 172.21.15.164:0/3869113483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4257d3b40 msgr2=0x55d4257d9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:51.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.006+0000 7fc6faffd640 1 --2- 172.21.15.164:0/3869113483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4257d3b40 0x55d4257d9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:51.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.007+0000 7fc6faffd640 1 -- 172.21.15.164:0/3869113483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4257dd370 msgr2=0x55d4257da440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:51.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.007+0000 7fc6faffd640 1 --2- 172.21.15.164:0/3869113483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4257dd370 0x55d4257da440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:51.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.007+0000 7fc6faffd640 1 -- 172.21.15.164:0/3869113483 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4254a93c0 con 0x55d4257d25d0 2024-09-19T08:47:51.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.007+0000 7fc6faffd640 1 --2- 172.21.15.164:0/3869113483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4257d25d0 0x55d4257dcc50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec006fd0 tx=0x7fc6ec00bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=546b2e953fd354e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:51.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.007+0000 7fc6e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-19T08:47:51.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.008+0000 7fc6e27fc640 1 -- 172.21.15.164:0/3869113483 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec00de30 con 0x55d4257d25d0 2024-09-19T08:47:51.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.008+0000 7fc703724740 1 -- 172.21.15.164:0/3869113483 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6f4046950 con 0x55d4257d25d0 2024-09-19T08:47:51.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.008+0000 7fc6e27fc640 1 -- 172.21.15.164:0/3869113483 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fc6ec0125f0 con 0x55d4257d25d0 2024-09-19T08:47:51.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.008+0000 7fc6e27fc640 1 -- 172.21.15.164:0/3869113483 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc6ec021050 con 0x55d4257d25d0 2024-09-19T08:47:51.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.009+0000 7fc703724740 10 client.25439 fetch_fsmap learned FSMap version 28 2024-09-19T08:47:51.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.009+0000 7fc703724740 1 -- 172.21.15.164:0/3869113483 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d42580db60 con 0x55d4257d25d0 2024-09-19T08:47:51.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.015+0000 7fc6e27fc640 1 -- 172.21.15.164:0/3869113483 <== mon.2 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6f4046950 con 0x55d4257d25d0 2024-09-19T08:47:51.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.016+0000 7fc6e27fc640 1 -- 172.21.15.164:0/3869113483 <== mon.2 v2:172.21.15.170:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc6ec013c80 con 0x55d4257d25d0 2024-09-19T08:47:51.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.016+0000 7fc703724740 10 client.25439 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:47:51.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.016+0000 7fc703724740 10 client.25439 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-19T08:47:51.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.016+0000 7fc703724740 20 client.25439 populate_metadata read hostname 'smithi164' 2024-09-19T08:47:51.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.016+0000 7fc703724740 1 --2- 172.21.15.164:0/3869113483 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55d42580f020 0x55d4258114e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:51.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.016+0000 7fc703724740 4 client.25439 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:47:51.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.016+0000 7fc703724740 1 -- 172.21.15.164:0/3869113483 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d425745860 con 0x55d42580f020 2024-09-19T08:47:51.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.018+0000 7fc6fa7fc640 1 --2- 172.21.15.164:0/3869113483 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55d42580f020 0x55d4258114e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:51.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.019+0000 7fc6e27fc640 10 client.25439 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:47:51.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.019+0000 7fc6fa7fc640 1 --2- 172.21.15.164:0/3869113483 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55d42580f020 0x55d4258114e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7103b41c903e61e8 server_cookie=bb78e91d104006df in_seq=0 out_seq=0 2024-09-19T08:47:51.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.022+0000 7fc6e27fc640 1 -- 172.21.15.164:0/3869113483 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d425745860 con 0x55d42580f020 2024-09-19T08:47:51.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.022+0000 7fc6e27fc640 10 client.25439 handle_command_reply: tid=0 2024-09-19T08:47:51.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.022+0000 7fc703724740 1 client.25439 shutdown 2024-09-19T08:47:51.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.022+0000 7fc703724740 1 -- 172.21.15.164:0/3869113483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4257d25d0 msgr2=0x55d4257dcc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:51.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.022+0000 7fc703724740 1 --2- 172.21.15.164:0/3869113483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4257d25d0 0x55d4257dcc50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec006fd0 tx=0x7fc6ec00bf90 comp rx=0 tx=0).stop 2024-09-19T08:47:51.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.022+0000 7fc703724740 1 -- 172.21.15.164:0/3869113483 shutdown_connections 2024-09-19T08:47:51.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.022+0000 7fc703724740 1 --2- 172.21.15.164:0/3869113483 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55d42580f020 0x55d4258114e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:51.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.022+0000 7fc703724740 1 --2- 172.21.15.164:0/3869113483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4257dd370 0x55d4257da440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:51.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.022+0000 7fc703724740 1 --2- 172.21.15.164:0/3869113483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4257d3b40 0x55d4257d9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:51.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.022+0000 7fc703724740 1 --2- 172.21.15.164:0/3869113483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4257d25d0 0x55d4257dcc50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:51.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.022+0000 7fc703724740 1 -- 172.21.15.164:0/3869113483 >> 172.21.15.164:0/3869113483 conn(0x55d42579b310 msgr2=0x55d4257d4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:51.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.023+0000 7fc6e27fc640 0 client.25439 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:47:51.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.023+0000 7fc703724740 1 -- 172.21.15.164:0/3869113483 shutdown_connections 2024-09-19T08:47:51.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.023+0000 7fc703724740 1 -- 172.21.15.164:0/3869113483 wait complete. 2024-09-19T08:47:51.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.023+0000 7fc703724740 20 client.25439 trim_cache size 0 max 16384 2024-09-19T08:47:51.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.083+0000 7fc703724740 1 Processor -- start 2024-09-19T08:47:51.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.083+0000 7fc703724740 1 -- start start 2024-09-19T08:47:51.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.084+0000 7fc703724740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4257d25d0 0x55d4257d87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:51.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.084+0000 7fc703724740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4257d3b40 0x55d4257d8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:51.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.084+0000 7fc703724740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4257dd370 0x55d425816db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:51.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.084+0000 7fc703724740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc6dc00e740 con 0x55d4257dd370 2024-09-19T08:47:51.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.084+0000 7fc703724740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc6dc010280 con 0x55d4257d3b40 2024-09-19T08:47:51.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.084+0000 7fc703724740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc6dc00ee10 con 0x55d4257d25d0 2024-09-19T08:47:51.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.087+0000 7fc6fa7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4257d3b40 0x55d4257d8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:51.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.087+0000 7fc6faffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4257d25d0 0x55d4257d87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:51.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.087+0000 7fc6fa7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4257d3b40 0x55d4257d8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:41494/0 (socket says 172.21.15.164:41494) 2024-09-19T08:47:51.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.087+0000 7fc6fa7fc640 1 -- 172.21.15.164:0/665608401 learned_addr learned my addr 172.21.15.164:0/665608401 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:51.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.091+0000 7fc6fb7fe640 1 --2- 172.21.15.164:0/665608401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4257dd370 0x55d425816db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:51.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.091+0000 7fc6fa7fc640 1 -- 172.21.15.164:0/665608401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4257d25d0 msgr2=0x55d4257d87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:51.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.091+0000 7fc6fa7fc640 1 --2- 172.21.15.164:0/665608401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4257d25d0 0x55d4257d87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:51.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.091+0000 7fc6fa7fc640 1 -- 172.21.15.164:0/665608401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4257dd370 msgr2=0x55d425816db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:51.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.091+0000 7fc6fa7fc640 1 --2- 172.21.15.164:0/665608401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4257dd370 0x55d425816db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:51.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.091+0000 7fc6fa7fc640 1 -- 172.21.15.164:0/665608401 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4257dcfc0 con 0x55d4257d3b40 2024-09-19T08:47:51.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.101+0000 7fc6fa7fc640 1 --2- 172.21.15.164:0/665608401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4257d3b40 0x55d4257d8d10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc6e409b880 tx=0x7fc6e40cf040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d97584337ea200ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:51.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.107+0000 7fc6f0ff9640 1 -- 172.21.15.164:0/665608401 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e409bff0 con 0x55d4257d3b40 2024-09-19T08:47:51.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.107+0000 7fc6f0ff9640 1 -- 172.21.15.164:0/665608401 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6e409b8f0 con 0x55d4257d3b40 2024-09-19T08:47:51.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.109+0000 7fc703724740 1 -- 172.21.15.164:0/665608401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4257d3b40 msgr2=0x55d4257d8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:51.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.109+0000 7fc703724740 1 --2- 172.21.15.164:0/665608401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4257d3b40 0x55d4257d8d10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc6e409b880 tx=0x7fc6e40cf040 comp rx=0 tx=0).stop 2024-09-19T08:47:51.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.109+0000 7fc703724740 1 -- 172.21.15.164:0/665608401 shutdown_connections 2024-09-19T08:47:51.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.109+0000 7fc703724740 1 --2- 172.21.15.164:0/665608401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4257dd370 0x55d425816db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:51.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.109+0000 7fc703724740 1 --2- 172.21.15.164:0/665608401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4257d3b40 0x55d4257d8d10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:51.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.109+0000 7fc703724740 1 --2- 172.21.15.164:0/665608401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4257d25d0 0x55d4257d87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:51.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.109+0000 7fc703724740 1 -- 172.21.15.164:0/665608401 >> 172.21.15.164:0/665608401 conn(0x55d42579b310 msgr2=0x55d4257d9ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:51.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.114+0000 7fc703724740 1 -- 172.21.15.164:0/665608401 shutdown_connections 2024-09-19T08:47:51.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.114+0000 7fc703724740 1 -- 172.21.15.164:0/665608401 wait complete. 2024-09-19T08:47:51.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.115+0000 7fc703724740 1 -- start start 2024-09-19T08:47:51.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.115+0000 7fc703724740 1 Processor -- start 2024-09-19T08:47:51.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.115+0000 7fc703724740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4257d25d0 0x55d4257e5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:51.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.115+0000 7fc703724740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4257d3b40 0x55d4257e5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:51.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.115+0000 7fc703724740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4257dd370 0x55d4257e0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:51.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.115+0000 7fc703724740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc6dc00fbb0 con 0x55d4257d3b40 2024-09-19T08:47:51.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.115+0000 7fc703724740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc6dc00f4e0 con 0x55d4257d25d0 2024-09-19T08:47:51.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.115+0000 7fc703724740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc6dc00c720 con 0x55d4257dd370 2024-09-19T08:47:51.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.115+0000 7fc703724740 10 client.0 _ll_register_callbacks cb 0x7fc6ec00b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:47:51.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.116+0000 7fc6fb7fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4257dd370 0x55d4257e0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:51.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.116+0000 7fc6fb7fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4257dd370 0x55d4257e0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:55272/0 (socket says 172.21.15.164:55272) 2024-09-19T08:47:51.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.116+0000 7fc6fb7fe640 1 -- 172.21.15.164:0/3730764627 learned_addr learned my addr 172.21.15.164:0/3730764627 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:51.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.117+0000 7fc6fa7fc640 1 --2- 172.21.15.164:0/3730764627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4257d3b40 0x55d4257e5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:51.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.118+0000 7fc6faffd640 1 --2- 172.21.15.164:0/3730764627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4257d25d0 0x55d4257e5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:51.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.118+0000 7fc6fb7fe640 1 -- 172.21.15.164:0/3730764627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4257d25d0 msgr2=0x55d4257e5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:51.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.118+0000 7fc6fb7fe640 1 --2- 172.21.15.164:0/3730764627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4257d25d0 0x55d4257e5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:51.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.118+0000 7fc6fb7fe640 1 -- 172.21.15.164:0/3730764627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4257d3b40 msgr2=0x55d4257e5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:51.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.118+0000 7fc6fb7fe640 1 --2- 172.21.15.164:0/3730764627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4257d3b40 0x55d4257e5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:51.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.118+0000 7fc6fb7fe640 1 -- 172.21.15.164:0/3730764627 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4257e14a0 con 0x55d4257dd370 2024-09-19T08:47:51.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.118+0000 7fc6fa7fc640 1 --2- 172.21.15.164:0/3730764627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4257d3b40 0x55d4257e5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:47:51.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.123+0000 7fc6fb7fe640 1 --2- 172.21.15.164:0/3730764627 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4257dd370 0x55d4257e0c70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4005c40 tx=0x7fc6f404fb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50fdcef94a48da4c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:51.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.124+0000 7fc6e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-09-19T08:47:51.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.128+0000 7fc6e27fc640 1 -- 172.21.15.164:0/3730764627 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4053040 con 0x55d4257dd370 2024-09-19T08:47:51.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.129+0000 7fc6e27fc640 1 -- 172.21.15.164:0/3730764627 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fc6f4030b90 con 0x55d4257dd370 2024-09-19T08:47:51.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.130+0000 7fc703724740 1 -- 172.21.15.164:0/3730764627 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6e409b8f0 con 0x55d4257dd370 2024-09-19T08:47:51.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.133+0000 7fc6e27fc640 1 -- 172.21.15.164:0/3730764627 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc6f4034040 con 0x55d4257dd370 2024-09-19T08:47:51.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.136+0000 7fc703724740 10 client.25438 fetch_fsmap learned FSMap version 28 2024-09-19T08:47:51.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.136+0000 7fc703724740 1 -- 172.21.15.164:0/3730764627 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d42588a930 con 0x55d4257dd370 2024-09-19T08:47:51.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.139+0000 7fc6e27fc640 1 -- 172.21.15.164:0/3730764627 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6e409b8f0 con 0x55d4257dd370 2024-09-19T08:47:51.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.139+0000 7fc6e27fc640 1 -- 172.21.15.164:0/3730764627 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc6f4056070 con 0x55d4257dd370 2024-09-19T08:47:51.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.140+0000 7fc703724740 10 client.25438 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:47:51.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.140+0000 7fc703724740 10 client.25438 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-19T08:47:51.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.140+0000 7fc703724740 20 client.25438 populate_metadata read hostname 'smithi164' 2024-09-19T08:47:51.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.140+0000 7fc703724740 1 --2- 172.21.15.164:0/3730764627 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55d42588b0c0 0x55d42588d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:51.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.140+0000 7fc703724740 4 client.25438 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:47:51.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.140+0000 7fc703724740 1 -- 172.21.15.164:0/3730764627 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d42573fd70 con 0x55d42588b0c0 2024-09-19T08:47:51.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.142+0000 7fc6faffd640 1 --2- 172.21.15.164:0/3730764627 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55d42588b0c0 0x55d42588d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:51.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.151+0000 7fc6faffd640 1 --2- 172.21.15.164:0/3730764627 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55d42588b0c0 0x55d42588d4e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d2d01f70f0435265 server_cookie=fa5e7bd0b8af905e in_seq=0 out_seq=0 2024-09-19T08:47:51.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.151+0000 7fc6e27fc640 10 client.25438 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:47:51.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.155+0000 7fc6e27fc640 1 -- 172.21.15.164:0/3730764627 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d42573fd70 con 0x55d42588b0c0 2024-09-19T08:47:51.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.155+0000 7fc6e27fc640 10 client.25438 handle_command_reply: tid=0 2024-09-19T08:47:51.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.155+0000 7fc703724740 1 client.25438 shutdown 2024-09-19T08:47:51.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.160+0000 7fc703724740 1 -- 172.21.15.164:0/3730764627 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4257dd370 msgr2=0x55d4257e0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:51.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.160+0000 7fc703724740 1 --2- 172.21.15.164:0/3730764627 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4257dd370 0x55d4257e0c70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4005c40 tx=0x7fc6f404fb00 comp rx=0 tx=0).stop 2024-09-19T08:47:51.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.162+0000 7fc703724740 1 -- 172.21.15.164:0/3730764627 shutdown_connections 2024-09-19T08:47:51.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.162+0000 7fc703724740 1 --2- 172.21.15.164:0/3730764627 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55d42588b0c0 0x55d42588d4e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:51.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.162+0000 7fc703724740 1 --2- 172.21.15.164:0/3730764627 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4257dd370 0x55d4257e0c70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:51.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.162+0000 7fc703724740 1 --2- 172.21.15.164:0/3730764627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4257d3b40 0x55d4257e5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:51.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.162+0000 7fc703724740 1 --2- 172.21.15.164:0/3730764627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4257d25d0 0x55d4257e5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:51.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.162+0000 7fc703724740 1 -- 172.21.15.164:0/3730764627 >> 172.21.15.164:0/3730764627 conn(0x55d42579b310 msgr2=0x55d4257d8660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:51.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.163+0000 7fc6e27fc640 0 client.25438 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:47:51.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.163+0000 7fc703724740 1 -- 172.21.15.164:0/3730764627 shutdown_connections 2024-09-19T08:47:51.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.164+0000 7fc703724740 1 -- 172.21.15.164:0/3730764627 wait complete. 2024-09-19T08:47:51.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.164+0000 7fc703724740 20 client.25438 trim_cache size 0 max 16384 2024-09-19T08:47:51.167 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:47:51.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.172+0000 7fc70134b640 1 -- 172.21.15.164:0/2991700107 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fc6e80a35c0 msgr2=0x7fc6e80a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:51.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.172+0000 7fc70134b640 1 --2- 172.21.15.164:0/2991700107 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fc6e80a35c0 0x7fc6e80a5a80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4030040 tx=0x7fc6f4046000 comp rx=0 tx=0).stop 2024-09-19T08:47:51.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.172+0000 7fc70134b640 1 -- 172.21.15.164:0/2991700107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6fc102570 msgr2=0x7fc6fc108020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:51.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.172+0000 7fc70134b640 1 --2- 172.21.15.164:0/2991700107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6fc102570 0x7fc6fc108020 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc6e400d6d0 tx=0x7fc6e400db90 comp rx=0 tx=0).stop 2024-09-19T08:47:51.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.172+0000 7fc70134b640 1 -- 172.21.15.164:0/2991700107 shutdown_connections 2024-09-19T08:47:51.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.172+0000 7fc70134b640 1 --2- 172.21.15.164:0/2991700107 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fc6e80a35c0 0x7fc6e80a5a80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:51.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.172+0000 7fc70134b640 1 --2- 172.21.15.164:0/2991700107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6fc102ef0 0x7fc6fc108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:51.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.172+0000 7fc70134b640 1 --2- 172.21.15.164:0/2991700107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6fc102570 0x7fc6fc108020 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:51.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.172+0000 7fc70134b640 1 --2- 172.21.15.164:0/2991700107 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6fc100e20 0x7fc6fc107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:51.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.172+0000 7fc70134b640 1 -- 172.21.15.164:0/2991700107 >> 172.21.15.164:0/2991700107 conn(0x7fc6fc0fc7d0 msgr2=0x7fc6fc069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:51.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.177+0000 7fc70134b640 1 -- 172.21.15.164:0/2991700107 shutdown_connections 2024-09-19T08:47:51.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:51.177+0000 7fc70134b640 1 -- 172.21.15.164:0/2991700107 wait complete. 2024-09-19T08:47:51.214 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:51 smithi164 ceph-mon[30928]: pgmap v64: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 748 MiB/s wr, 3.48k op/s 2024-09-19T08:47:51.624 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:51 smithi170 ceph-mon[35072]: pgmap v64: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 748 MiB/s wr, 3.48k op/s 2024-09-19T08:47:51.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:51 smithi173 ceph-mon[35306]: pgmap v64: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 748 MiB/s wr, 3.48k op/s 2024-09-19T08:47:52.408 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:47:53.536 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T08:47:53.537 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:53 smithi164 ceph-mon[30928]: pgmap v65: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 665 MiB/s wr, 2.82k op/s 2024-09-19T08:47:53.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:53 smithi170 ceph-mon[35072]: pgmap v65: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 665 MiB/s wr, 2.82k op/s 2024-09-19T08:47:53.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:53 smithi173 ceph-mon[35306]: pgmap v65: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 665 MiB/s wr, 2.82k op/s 2024-09-19T08:47:54.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.811+0000 7f8c5456b640 1 -- 172.21.15.164:0/122705825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c4c108da0 msgr2=0x7f8c4c10cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:54.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.811+0000 7f8c5456b640 1 --2- 172.21.15.164:0/122705825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c4c108da0 0x7f8c4c10cdf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c009900 tx=0x7f8c3c02a420 comp rx=0 tx=0).stop 2024-09-19T08:47:54.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.812+0000 7f8c5456b640 1 -- 172.21.15.164:0/122705825 shutdown_connections 2024-09-19T08:47:54.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.812+0000 7f8c5456b640 1 --2- 172.21.15.164:0/122705825 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c4c10d520 0x7f8c4c1150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:54.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.812+0000 7f8c5456b640 1 --2- 172.21.15.164:0/122705825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c4c108da0 0x7f8c4c10cdf0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:54.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.812+0000 7f8c5456b640 1 --2- 172.21.15.164:0/122705825 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c4c1083f0 0x7f8c4c1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:54.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.812+0000 7f8c5456b640 1 -- 172.21.15.164:0/122705825 >> 172.21.15.164:0/122705825 conn(0x7f8c4c077af0 msgr2=0x7f8c4c079f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:54.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.813+0000 7f8c5456b640 1 -- 172.21.15.164:0/122705825 shutdown_connections 2024-09-19T08:47:54.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.813+0000 7f8c5456b640 1 -- 172.21.15.164:0/122705825 wait complete. 2024-09-19T08:47:54.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.815+0000 7f8c5456b640 1 Processor -- start 2024-09-19T08:47:54.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.816+0000 7f8c5456b640 1 -- start start 2024-09-19T08:47:54.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.816+0000 7f8c5456b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c4c1083f0 0x7f8c4c1a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:54.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.817+0000 7f8c5456b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c4c108da0 0x7f8c4c1a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:54.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.817+0000 7f8c5456b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c4c10d520 0x7f8c4c19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:54.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.817+0000 7f8c5456b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4c118520 con 0x7f8c4c1083f0 2024-09-19T08:47:54.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.817+0000 7f8c5456b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4c1183a0 con 0x7f8c4c10d520 2024-09-19T08:47:54.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.817+0000 7f8c5456b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4c1186a0 con 0x7f8c4c108da0 2024-09-19T08:47:54.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.817+0000 7f8c51b06640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c4c108da0 0x7f8c4c1a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:54.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.818+0000 7f8c51b06640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c4c108da0 0x7f8c4c1a4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:45120/0 (socket says 172.21.15.164:45120) 2024-09-19T08:47:54.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.818+0000 7f8c52b08640 1 --2- 172.21.15.164:0/1361256525 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c4c10d520 0x7f8c4c19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:54.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.818+0000 7f8c51b06640 1 -- 172.21.15.164:0/1361256525 learned_addr learned my addr 172.21.15.164:0/1361256525 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:54.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.819+0000 7f8c52b08640 1 -- 172.21.15.164:0/1361256525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c4c108da0 msgr2=0x7f8c4c1a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:54.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.819+0000 7f8c52b08640 1 --2- 172.21.15.164:0/1361256525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c4c108da0 0x7f8c4c1a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:54.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.819+0000 7f8c52b08640 1 -- 172.21.15.164:0/1361256525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c4c1083f0 msgr2=0x7f8c4c1a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:54.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.819+0000 7f8c52b08640 1 --2- 172.21.15.164:0/1361256525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c4c1083f0 0x7f8c4c1a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:54.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.819+0000 7f8c52b08640 1 -- 172.21.15.164:0/1361256525 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c4c19eff0 con 0x7f8c4c10d520 2024-09-19T08:47:54.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.820+0000 7f8c52b08640 1 --2- 172.21.15.164:0/1361256525 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c4c10d520 0x7f8c4c19e850 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009fd0 tx=0x7f8c4400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:54.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.824+0000 7f8c437fe640 1 -- 172.21.15.164:0/1361256525 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4400ce20 con 0x7f8c4c10d520 2024-09-19T08:47:54.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.824+0000 7f8c5456b640 1 -- 172.21.15.164:0/1361256525 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c4c19f2e0 con 0x7f8c4c10d520 2024-09-19T08:47:54.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.825+0000 7f8c5456b640 1 -- 172.21.15.164:0/1361256525 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c4c1ab530 con 0x7f8c4c10d520 2024-09-19T08:47:54.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.825+0000 7f8c437fe640 1 -- 172.21.15.164:0/1361256525 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c44005020 con 0x7f8c4c10d520 2024-09-19T08:47:54.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.825+0000 7f8c437fe640 1 -- 172.21.15.164:0/1361256525 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c44010720 con 0x7f8c4c10d520 2024-09-19T08:47:54.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.833+0000 7f8c56944740 1 Processor -- start 2024-09-19T08:47:54.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.833+0000 7f8c56944740 1 -- start start 2024-09-19T08:47:54.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.834+0000 7f8c56944740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56433e4f4850 0x56433e4f4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:54.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.834+0000 7f8c56944740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56433e4f5170 0x56433e4ff4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:54.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.834+0000 7f8c56944740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56433e4ffa30 0x56433e501e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:54.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.835+0000 7f8c56944740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56433e459160 con 0x56433e4f5170 2024-09-19T08:47:54.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.835+0000 7f8c56944740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56433e400720 con 0x56433e4ffa30 2024-09-19T08:47:54.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.835+0000 7f8c56944740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56433e455120 con 0x56433e4f4850 2024-09-19T08:47:54.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.835+0000 7f8c437fe640 1 -- 172.21.15.164:0/1361256525 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f8c44004a40 con 0x7f8c4c10d520 2024-09-19T08:47:54.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.835+0000 7f8c51b06640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56433e4f4850 0x56433e4f4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:54.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.835+0000 7f8c51b06640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56433e4f4850 0x56433e4f4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:45132/0 (socket says 172.21.15.164:45132) 2024-09-19T08:47:54.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.835+0000 7f8c51b06640 1 -- 172.21.15.164:0/1146163727 learned_addr learned my addr 172.21.15.164:0/1146163727 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:54.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.836+0000 7f8c51b06640 1 --2- 172.21.15.164:0/1146163727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56433e4ffa30 0x56433e501e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:54.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.836+0000 7f8c52307640 1 --2- 172.21.15.164:0/1146163727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56433e4f5170 0x56433e4ff4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:54.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.836+0000 7f8c51b06640 1 -- 172.21.15.164:0/1146163727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56433e4ffa30 msgr2=0x56433e501e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:54.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.836+0000 7f8c51b06640 1 --2- 172.21.15.164:0/1146163727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56433e4ffa30 0x56433e501e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:54.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.836+0000 7f8c51b06640 1 -- 172.21.15.164:0/1146163727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56433e4f5170 msgr2=0x56433e4ff4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:54.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.836+0000 7f8c51b06640 1 --2- 172.21.15.164:0/1146163727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56433e4f5170 0x56433e4ff4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:54.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.836+0000 7f8c51b06640 1 -- 172.21.15.164:0/1146163727 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56433e1caba0 con 0x56433e4f4850 2024-09-19T08:47:54.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.837+0000 7f8c51b06640 1 --2- 172.21.15.164:0/1146163727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56433e4f4850 0x56433e4f4c30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0096c0 tx=0x7f8c3c0095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d0aad011b9e88d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:54.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.838+0000 7f8c437fe640 1 --2- 172.21.15.164:0/1361256525 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f8c380a3580 0x7f8c380a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:54.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.840+0000 7f8c52307640 1 --2- 172.21.15.164:0/1361256525 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f8c380a3580 0x7f8c380a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:54.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.840+0000 7f8c437fe640 1 -- 172.21.15.164:0/1361256525 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f8c440d0280 con 0x7f8c4c10d520 2024-09-19T08:47:54.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.840+0000 7f8c40ff9640 1 -- 172.21.15.164:0/1146163727 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c004230 con 0x56433e4f4850 2024-09-19T08:47:54.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.841+0000 7f8c40ff9640 1 -- 172.21.15.164:0/1146163727 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c3c005000 con 0x56433e4f4850 2024-09-19T08:47:54.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.842+0000 7f8c56944740 1 -- 172.21.15.164:0/1146163727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56433e4f4850 msgr2=0x56433e4f4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:54.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.842+0000 7f8c56944740 1 --2- 172.21.15.164:0/1146163727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56433e4f4850 0x56433e4f4c30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0096c0 tx=0x7f8c3c0095c0 comp rx=0 tx=0).stop 2024-09-19T08:47:54.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.842+0000 7f8c56944740 1 -- 172.21.15.164:0/1146163727 shutdown_connections 2024-09-19T08:47:54.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.842+0000 7f8c56944740 1 --2- 172.21.15.164:0/1146163727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56433e4ffa30 0x56433e501e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:54.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.842+0000 7f8c56944740 1 --2- 172.21.15.164:0/1146163727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56433e4f5170 0x56433e4ff4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:54.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.842+0000 7f8c56944740 1 --2- 172.21.15.164:0/1146163727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56433e4f4850 0x56433e4f4c30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:54.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.842+0000 7f8c56944740 1 -- 172.21.15.164:0/1146163727 >> 172.21.15.164:0/1146163727 conn(0x56433e4f0de0 msgr2=0x56433e4f11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:54.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.843+0000 7f8c56944740 1 -- 172.21.15.164:0/1146163727 shutdown_connections 2024-09-19T08:47:54.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.843+0000 7f8c56944740 1 -- 172.21.15.164:0/1146163727 wait complete. 2024-09-19T08:47:54.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.843+0000 7f8c56944740 1 -- start start 2024-09-19T08:47:54.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.843+0000 7f8c56944740 1 Processor -- start 2024-09-19T08:47:54.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.844+0000 7f8c56944740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56433e4f5170 0x56433e4ff4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:54.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.844+0000 7f8c56944740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56433e4ffa30 0x56433e4fc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:54.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.845+0000 7f8c56944740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56433e4fce70 0x56433e50bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:54.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.845+0000 7f8c56944740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56433e502650 con 0x56433e4f5170 2024-09-19T08:47:54.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.845+0000 7f8c56944740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56433e3e1900 con 0x56433e4fce70 2024-09-19T08:47:54.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.845+0000 7f8c56944740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56433e5027d0 con 0x56433e4ffa30 2024-09-19T08:47:54.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.845+0000 7f8c56944740 10 client.0 _ll_register_callbacks cb 0x56433e3df780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:47:54.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.845+0000 7f8c52307640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56433e4f5170 0x56433e4ff4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:54.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.845+0000 7f8c52307640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56433e4f5170 0x56433e4ff4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47348/0 (socket says 172.21.15.164:47348) 2024-09-19T08:47:54.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.845+0000 7f8c52307640 1 -- 172.21.15.164:0/1014664091 learned_addr learned my addr 172.21.15.164:0/1014664091 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:54.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.845+0000 7f8c51b06640 1 --2- 172.21.15.164:0/1014664091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56433e4ffa30 0x56433e4fc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:54.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.846+0000 7f8c52b08640 1 --2- 172.21.15.164:0/1014664091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56433e4fce70 0x56433e50bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:54.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.846+0000 7f8c52307640 1 -- 172.21.15.164:0/1014664091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56433e4ffa30 msgr2=0x56433e4fc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:54.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.846+0000 7f8c52307640 1 --2- 172.21.15.164:0/1014664091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56433e4ffa30 0x56433e4fc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:54.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.846+0000 7f8c52307640 1 -- 172.21.15.164:0/1014664091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56433e4fce70 msgr2=0x56433e50bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:54.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.846+0000 7f8c52307640 1 --2- 172.21.15.164:0/1014664091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56433e4fce70 0x56433e50bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:54.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.846+0000 7f8c52307640 1 -- 172.21.15.164:0/1014664091 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56433e421c50 con 0x56433e4f5170 2024-09-19T08:47:54.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.846+0000 7f8c56944740 1 -- 172.21.15.164:0/1014664091 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56433e466890 con 0x56433e4f5170 2024-09-19T08:47:54.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.847+0000 7f8c52307640 1 --2- 172.21.15.164:0/1361256525 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f8c380a3580 0x7f8c380a5a40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8c480138c0 tx=0x7f8c48009f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:54.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.847+0000 7f8c52307640 1 --2- 172.21.15.164:0/1014664091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56433e4f5170 0x56433e4ff4d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8c48005a70 tx=0x7f8c48005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2100fdf08c3c1e2c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:54.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.847+0000 7f8c35ffb640 10 client.15954 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:47:54.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.847+0000 7f8c35ffb640 1 -- 172.21.15.164:0/1014664091 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4802f050 con 0x56433e4f5170 2024-09-19T08:47:54.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.848+0000 7f8c35ffb640 1 -- 172.21.15.164:0/1014664091 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f8c4802a440 con 0x56433e4f5170 2024-09-19T08:47:54.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.849+0000 7f8c35ffb640 1 -- 172.21.15.164:0/1014664091 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8c48025070 con 0x56433e4f5170 2024-09-19T08:47:54.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.859+0000 7f8c56944740 10 client.15954 fetch_fsmap learned FSMap version 28 2024-09-19T08:47:54.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.859+0000 7f8c56944740 1 -- 172.21.15.164:0/1014664091 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({fsmap=28}) -- 0x56433e50e640 con 0x56433e4f5170 2024-09-19T08:47:54.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.864+0000 7f8c35ffb640 1 -- 172.21.15.164:0/1014664091 <== mon.0 v2:172.21.15.164:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8c48032020 con 0x56433e4f5170 2024-09-19T08:47:54.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.864+0000 7f8c56944740 10 client.15954 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:47:54.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.864+0000 7f8c56944740 10 client.15954 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:47:54.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.864+0000 7f8c56944740 20 client.15954 populate_metadata read hostname 'smithi164' 2024-09-19T08:47:54.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.864+0000 7f8c56944740 1 --2- 172.21.15.164:0/1014664091 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56433e510000 0x56433e512460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:54.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.864+0000 7f8c56944740 4 client.15954 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:47:54.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.864+0000 7f8c56944740 1 -- 172.21.15.164:0/1014664091 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56433e460be0 con 0x56433e510000 2024-09-19T08:47:54.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.864+0000 7f8c51b06640 1 --2- 172.21.15.164:0/1014664091 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56433e510000 0x56433e512460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:54.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.865+0000 7f8c51b06640 1 --2- 172.21.15.164:0/1014664091 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56433e510000 0x56433e512460 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2917e5a11b6a70fd server_cookie=e4b2cef59b9b3030 in_seq=0 out_seq=0 2024-09-19T08:47:54.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.865+0000 7f8c35ffb640 10 client.15954 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:47:54.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.866+0000 7f8c35ffb640 1 -- 172.21.15.164:0/1014664091 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56433e460be0 con 0x56433e510000 2024-09-19T08:47:54.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.866+0000 7f8c35ffb640 10 client.15954 handle_command_reply: tid=0 2024-09-19T08:47:54.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.866+0000 7f8c56944740 1 client.15954 shutdown 2024-09-19T08:47:54.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.867+0000 7f8c56944740 1 -- 172.21.15.164:0/1014664091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56433e4f5170 msgr2=0x56433e4ff4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:54.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.867+0000 7f8c56944740 1 --2- 172.21.15.164:0/1014664091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56433e4f5170 0x56433e4ff4d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8c48005a70 tx=0x7f8c48005f90 comp rx=0 tx=0).stop 2024-09-19T08:47:54.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.869+0000 7f8c56944740 1 -- 172.21.15.164:0/1014664091 shutdown_connections 2024-09-19T08:47:54.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.869+0000 7f8c56944740 1 --2- 172.21.15.164:0/1014664091 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56433e510000 0x56433e512460 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:54.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.869+0000 7f8c56944740 1 --2- 172.21.15.164:0/1014664091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56433e4fce70 0x56433e50bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:54.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.869+0000 7f8c56944740 1 --2- 172.21.15.164:0/1014664091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56433e4ffa30 0x56433e4fc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:54.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.869+0000 7f8c56944740 1 --2- 172.21.15.164:0/1014664091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56433e4f5170 0x56433e4ff4d0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:54.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.869+0000 7f8c56944740 1 -- 172.21.15.164:0/1014664091 >> 172.21.15.164:0/1014664091 conn(0x56433e4f0de0 msgr2=0x56433e4f6320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:54.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.869+0000 7f8c35ffb640 0 client.15954 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:47:54.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.870+0000 7f8c56944740 1 -- 172.21.15.164:0/1014664091 shutdown_connections 2024-09-19T08:47:54.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.870+0000 7f8c56944740 1 -- 172.21.15.164:0/1014664091 wait complete. 2024-09-19T08:47:54.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.870+0000 7f8c56944740 20 client.15954 trim_cache size 0 max 16384 2024-09-19T08:47:54.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.914+0000 7f8c56944740 1 Processor -- start 2024-09-19T08:47:54.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.915+0000 7f8c56944740 1 -- start start 2024-09-19T08:47:54.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.917+0000 7f8c56944740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56433e4f5170 0x56433e50b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:54.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.917+0000 7f8c56944740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56433e4fce70 0x56433e50bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:54.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.918+0000 7f8c56944740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56433e4ffa30 0x56433e4fc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:54.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.918+0000 7f8c56944740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3000e590 con 0x56433e4fce70 2024-09-19T08:47:54.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.918+0000 7f8c56944740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8c300100d0 con 0x56433e4f5170 2024-09-19T08:47:54.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.918+0000 7f8c56944740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3000ec60 con 0x56433e4ffa30 2024-09-19T08:47:54.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.928+0000 7f8c51b06640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56433e4fce70 0x56433e50bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:54.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.929+0000 7f8c51b06640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56433e4fce70 0x56433e50bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47356/0 (socket says 172.21.15.164:47356) 2024-09-19T08:47:54.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.929+0000 7f8c51b06640 1 -- 172.21.15.164:0/3959384955 learned_addr learned my addr 172.21.15.164:0/3959384955 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:54.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.929+0000 7f8c52307640 1 --2- 172.21.15.164:0/3959384955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56433e4f5170 0x56433e50b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:54.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.929+0000 7f8c52b08640 1 --2- 172.21.15.164:0/3959384955 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56433e4ffa30 0x56433e4fc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:54.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.930+0000 7f8c51b06640 1 -- 172.21.15.164:0/3959384955 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56433e4ffa30 msgr2=0x56433e4fc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:54.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.930+0000 7f8c51b06640 1 --2- 172.21.15.164:0/3959384955 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56433e4ffa30 0x56433e4fc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:54.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.930+0000 7f8c51b06640 1 -- 172.21.15.164:0/3959384955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56433e4f5170 msgr2=0x56433e50b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:54.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.930+0000 7f8c51b06640 1 --2- 172.21.15.164:0/3959384955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56433e4f5170 0x56433e50b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:54.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.930+0000 7f8c51b06640 1 -- 172.21.15.164:0/3959384955 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56433e4ff840 con 0x56433e4fce70 2024-09-19T08:47:54.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.931+0000 7f8c51b06640 1 --2- 172.21.15.164:0/3959384955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56433e4fce70 0x56433e50bed0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c048000 tx=0x7f8c3c041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf77932188ff991 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:54.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.934+0000 7f8c37fff640 1 -- 172.21.15.164:0/3959384955 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c0035f0 con 0x56433e4fce70 2024-09-19T08:47:54.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.934+0000 7f8c37fff640 1 -- 172.21.15.164:0/3959384955 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c3c004c90 con 0x56433e4fce70 2024-09-19T08:47:54.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.934+0000 7f8c37fff640 1 -- 172.21.15.164:0/3959384955 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c005240 con 0x56433e4fce70 2024-09-19T08:47:54.954 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:47:54.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.938+0000 7f8c56944740 1 -- 172.21.15.164:0/3959384955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56433e4fce70 msgr2=0x56433e50bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:54.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.938+0000 7f8c56944740 1 --2- 172.21.15.164:0/3959384955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56433e4fce70 0x56433e50bed0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c048000 tx=0x7f8c3c041040 comp rx=0 tx=0).stop 2024-09-19T08:47:54.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.942+0000 7f8c56944740 1 -- 172.21.15.164:0/3959384955 shutdown_connections 2024-09-19T08:47:54.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.942+0000 7f8c56944740 1 --2- 172.21.15.164:0/3959384955 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56433e4ffa30 0x56433e4fc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:54.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.942+0000 7f8c56944740 1 --2- 172.21.15.164:0/3959384955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56433e4fce70 0x56433e50bed0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:54.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.942+0000 7f8c56944740 1 --2- 172.21.15.164:0/3959384955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56433e4f5170 0x56433e50b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:54.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.942+0000 7f8c56944740 1 -- 172.21.15.164:0/3959384955 >> 172.21.15.164:0/3959384955 conn(0x56433e4f0de0 msgr2=0x56433e4f7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:54.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.943+0000 7f8c56944740 1 -- 172.21.15.164:0/3959384955 shutdown_connections 2024-09-19T08:47:54.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.943+0000 7f8c56944740 1 -- 172.21.15.164:0/3959384955 wait complete. 2024-09-19T08:47:54.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.944+0000 7f8c56944740 1 -- start start 2024-09-19T08:47:54.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.945+0000 7f8c56944740 1 Processor -- start 2024-09-19T08:47:54.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.952+0000 7f8c56944740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56433e4f5170 0x56433e507d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:54.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.952+0000 7f8c56944740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56433e4fce70 0x56433e5082d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:54.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.952+0000 7f8c56944740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56433e4ffa30 0x56433e4f9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:54.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.953+0000 7f8c56944740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3000fa00 con 0x56433e4ffa30 2024-09-19T08:47:54.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.953+0000 7f8c56944740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3000f330 con 0x56433e4fce70 2024-09-19T08:47:54.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.953+0000 7f8c56944740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3000c570 con 0x56433e4f5170 2024-09-19T08:47:54.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.953+0000 7f8c56944740 10 client.0 _ll_register_callbacks cb 0x7f8c48009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:47:54.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.959+0000 7f8c52b08640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56433e4ffa30 0x56433e4f9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:54.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.959+0000 7f8c52b08640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56433e4ffa30 0x56433e4f9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47364/0 (socket says 172.21.15.164:47364) 2024-09-19T08:47:54.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.959+0000 7f8c52b08640 1 -- 172.21.15.164:0/3376058095 learned_addr learned my addr 172.21.15.164:0/3376058095 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:54.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.959+0000 7f8c52307640 1 --2- 172.21.15.164:0/3376058095 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56433e4f5170 0x56433e507d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:54.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.959+0000 7f8c51b06640 1 --2- 172.21.15.164:0/3376058095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56433e4fce70 0x56433e5082d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:54.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.971+0000 7f8c52b08640 1 -- 172.21.15.164:0/3376058095 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56433e4f5170 msgr2=0x56433e507d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:54.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.971+0000 7f8c52b08640 1 --2- 172.21.15.164:0/3376058095 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56433e4f5170 0x56433e507d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:54.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.971+0000 7f8c52b08640 1 -- 172.21.15.164:0/3376058095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56433e4fce70 msgr2=0x56433e5082d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:54.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.971+0000 7f8c52b08640 1 --2- 172.21.15.164:0/3376058095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56433e4fce70 0x56433e5082d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:54.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.971+0000 7f8c52b08640 1 -- 172.21.15.164:0/3376058095 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56433e4fa280 con 0x56433e4ffa30 2024-09-19T08:47:54.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.974+0000 7f8c52b08640 1 --2- 172.21.15.164:0/3376058095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56433e4ffa30 0x56433e4f9a00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8c440989d0 tx=0x7f8c4409cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ae0696bbad95fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:54.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.974+0000 7f8c35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:47:54.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.975+0000 7f8c35ffb640 1 -- 172.21.15.164:0/3376058095 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c440ba060 con 0x56433e4ffa30 2024-09-19T08:47:54.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.975+0000 7f8c56944740 1 -- 172.21.15.164:0/3376058095 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c3c004c90 con 0x56433e4ffa30 2024-09-19T08:47:54.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.976+0000 7f8c35ffb640 1 -- 172.21.15.164:0/3376058095 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f8c440a4d50 con 0x56433e4ffa30 2024-09-19T08:47:54.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.977+0000 7f8c35ffb640 1 -- 172.21.15.164:0/3376058095 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8c440aa050 con 0x56433e4ffa30 2024-09-19T08:47:54.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.978+0000 7f8c56944740 10 client.15960 fetch_fsmap learned FSMap version 28 2024-09-19T08:47:54.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.978+0000 7f8c56944740 1 -- 172.21.15.164:0/3376058095 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56433e500ee0 con 0x56433e4ffa30 2024-09-19T08:47:54.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.979+0000 7f8c35ffb640 1 -- 172.21.15.164:0/3376058095 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c440cde60 con 0x56433e4ffa30 2024-09-19T08:47:54.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.979+0000 7f8c35ffb640 1 -- 172.21.15.164:0/3376058095 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8c4409b430 con 0x56433e4ffa30 2024-09-19T08:47:54.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.982+0000 7f8c56944740 10 client.15960 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:47:54.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.982+0000 7f8c56944740 10 client.15960 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:47:54.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.982+0000 7f8c56944740 20 client.15960 populate_metadata read hostname 'smithi164' 2024-09-19T08:47:54.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.982+0000 7f8c56944740 1 --2- 172.21.15.164:0/3376058095 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56433e57afc0 0x56433e57d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:54.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.982+0000 7f8c56944740 4 client.15960 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}] 2024-09-19T08:47:54.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.982+0000 7f8c56944740 1 -- 172.21.15.164:0/3376058095 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}) -- 0x7f8c3c004c90 con 0x56433e57afc0 2024-09-19T08:47:54.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:54.992+0000 7f8c52307640 1 --2- 172.21.15.164:0/3376058095 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56433e57afc0 0x56433e57d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:55.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:55.000+0000 7f8c52307640 1 --2- 172.21.15.164:0/3376058095 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56433e57afc0 0x56433e57d3b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e907ad744a485e server_cookie=80dd5e740754f942 in_seq=0 out_seq=0 2024-09-19T08:47:55.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:55.001+0000 7f8c35ffb640 10 client.15960 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:47:55.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:55.265+0000 7f8c35ffb640 1 -- 172.21.15.164:0/3376058095 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f8c3c004c90 con 0x56433e57afc0 2024-09-19T08:47:55.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:55.265+0000 7f8c35ffb640 10 client.15960 handle_command_reply: tid=0 2024-09-19T08:47:55.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:55.265+0000 7f8c56944740 1 client.15960 shutdown 2024-09-19T08:47:55.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:55.266+0000 7f8c56944740 1 -- 172.21.15.164:0/3376058095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56433e4ffa30 msgr2=0x56433e4f9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:55.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:55.266+0000 7f8c56944740 1 --2- 172.21.15.164:0/3376058095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56433e4ffa30 0x56433e4f9a00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8c440989d0 tx=0x7f8c4409cc70 comp rx=0 tx=0).stop 2024-09-19T08:47:55.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:55.266+0000 7f8c56944740 1 -- 172.21.15.164:0/3376058095 shutdown_connections 2024-09-19T08:47:55.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:55.266+0000 7f8c56944740 1 --2- 172.21.15.164:0/3376058095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56433e4ffa30 0x56433e4f9a00 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:55.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:55.266+0000 7f8c56944740 1 --2- 172.21.15.164:0/3376058095 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56433e57afc0 0x56433e57d3b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:55.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:55.266+0000 7f8c56944740 1 --2- 172.21.15.164:0/3376058095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56433e4fce70 0x56433e5082d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:55.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:55.266+0000 7f8c56944740 1 --2- 172.21.15.164:0/3376058095 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56433e4f5170 0x56433e507d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:55.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:55.266+0000 7f8c35ffb640 0 client.15960 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:47:55.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:55.266+0000 7f8c56944740 1 -- 172.21.15.164:0/3376058095 >> 172.21.15.164:0/3376058095 conn(0x56433e4f0de0 msgr2=0x56433e4fbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:55.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:55.266+0000 7f8c56944740 1 -- 172.21.15.164:0/3376058095 shutdown_connections 2024-09-19T08:47:55.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:55.266+0000 7f8c56944740 1 -- 172.21.15.164:0/3376058095 wait complete. 2024-09-19T08:47:55.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:55.266+0000 7f8c56944740 20 client.15960 trim_cache size 0 max 16384 2024-09-19T08:47:55.268 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:47:55.268 INFO:teuthology.orchestra.run.smithi164.stdout: "epoch": 28, 2024-09-19T08:47:55.268 INFO:teuthology.orchestra.run.smithi164.stdout: "leader": 24479, 2024-09-19T08:47:55.269 INFO:teuthology.orchestra.run.smithi164.stdout: "set_version": 10, 2024-09-19T08:47:55.269 INFO:teuthology.orchestra.run.smithi164.stdout: "sets": { 2024-09-19T08:47:55.269 INFO:teuthology.orchestra.run.smithi164.stdout: "2966dfc9": { 2024-09-19T08:47:55.269 INFO:teuthology.orchestra.run.smithi164.stdout: "version": 10, 2024-09-19T08:47:55.269 INFO:teuthology.orchestra.run.smithi164.stdout: "age_ref": 0.0, 2024-09-19T08:47:55.269 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:47:55.269 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "QUIESCED", 2024-09-19T08:47:55.269 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:47:55.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:47:55.269 INFO:teuthology.orchestra.run.smithi164.stdout: "timeout": 90.0, 2024-09-19T08:47:55.269 INFO:teuthology.orchestra.run.smithi164.stdout: "expiration": 140.3, 2024-09-19T08:47:55.270 INFO:teuthology.orchestra.run.smithi164.stdout: "members": { 2024-09-19T08:47:55.270 INFO:teuthology.orchestra.run.smithi164.stdout: "file:/": { 2024-09-19T08:47:55.270 INFO:teuthology.orchestra.run.smithi164.stdout: "excluded": false, 2024-09-19T08:47:55.270 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:47:55.270 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "QUIESCED", 2024-09-19T08:47:55.270 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:47:55.270 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:47:55.270 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:47:55.270 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:47:55.270 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:47:55.270 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:47:55.270 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:47:55.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:55.267+0000 7f8c5456b640 1 -- 172.21.15.164:0/1361256525 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f8c380a3580 msgr2=0x7f8c380a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:55.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:55.267+0000 7f8c5456b640 1 --2- 172.21.15.164:0/1361256525 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f8c380a3580 0x7f8c380a5a40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8c480138c0 tx=0x7f8c48009f90 comp rx=0 tx=0).stop 2024-09-19T08:47:55.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:55.267+0000 7f8c5456b640 1 -- 172.21.15.164:0/1361256525 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c4c10d520 msgr2=0x7f8c4c19e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:55.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:55.267+0000 7f8c5456b640 1 --2- 172.21.15.164:0/1361256525 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c4c10d520 0x7f8c4c19e850 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009fd0 tx=0x7f8c4400ef10 comp rx=0 tx=0).stop 2024-09-19T08:47:55.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:55.268+0000 7f8c5456b640 1 -- 172.21.15.164:0/1361256525 shutdown_connections 2024-09-19T08:47:55.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:55.268+0000 7f8c5456b640 1 --2- 172.21.15.164:0/1361256525 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c4c10d520 0x7f8c4c19e850 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:55.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:55.268+0000 7f8c5456b640 1 --2- 172.21.15.164:0/1361256525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c4c108da0 0x7f8c4c1a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:55.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:55.268+0000 7f8c5456b640 1 --2- 172.21.15.164:0/1361256525 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f8c380a3580 0x7f8c380a5a40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:55.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:55.268+0000 7f8c5456b640 1 --2- 172.21.15.164:0/1361256525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c4c1083f0 0x7f8c4c1a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:55.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:55.268+0000 7f8c5456b640 1 -- 172.21.15.164:0/1361256525 >> 172.21.15.164:0/1361256525 conn(0x7f8c4c077af0 msgr2=0x7f8c4c078de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:55.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:55.268+0000 7f8c5456b640 1 -- 172.21.15.164:0/1361256525 shutdown_connections 2024-09-19T08:47:55.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:55.268+0000 7f8c5456b640 1 -- 172.21.15.164:0/1361256525 wait complete. 2024-09-19T08:47:55.493 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:55 smithi164 ceph-mon[30928]: pgmap v66: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 859 MiB/s wr, 3.80k op/s 2024-09-19T08:47:55.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:55 smithi170 ceph-mon[35072]: pgmap v66: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 859 MiB/s wr, 3.80k op/s 2024-09-19T08:47:55.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:55 smithi173 ceph-mon[35306]: pgmap v66: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 859 MiB/s wr, 3.80k op/s 2024-09-19T08:47:56.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:55.999+0000 7f75f803c640 1 -- 172.21.15.164:0/176715630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75f0102d30 msgr2=0x7f75f0103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:56.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:55.999+0000 7f75f803c640 1 --2- 172.21.15.164:0/176715630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75f0102d30 0x7f75f0103110 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f75e0009cd0 tx=0x7f75e002a910 comp rx=0 tx=0).stop 2024-09-19T08:47:56.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.000+0000 7f75f803c640 1 -- 172.21.15.164:0/176715630 shutdown_connections 2024-09-19T08:47:56.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.000+0000 7f75f803c640 1 --2- 172.21.15.164:0/176715630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75f01082a0 0x7f75f010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:56.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.000+0000 7f75f803c640 1 --2- 172.21.15.164:0/176715630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75f01036e0 0x7f75f0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:56.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.000+0000 7f75f803c640 1 --2- 172.21.15.164:0/176715630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75f0102d30 0x7f75f0103110 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:56.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.000+0000 7f75f803c640 1 -- 172.21.15.164:0/176715630 >> 172.21.15.164:0/176715630 conn(0x7f75f00fc7d0 msgr2=0x7f75f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:56.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.000+0000 7f75f803c640 1 -- 172.21.15.164:0/176715630 shutdown_connections 2024-09-19T08:47:56.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.000+0000 7f75f803c640 1 -- 172.21.15.164:0/176715630 wait complete. 2024-09-19T08:47:56.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.000+0000 7f75f803c640 1 Processor -- start 2024-09-19T08:47:56.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.000+0000 7f75f803c640 1 -- start start 2024-09-19T08:47:56.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.000+0000 7f75f803c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75f0102d30 0x7f75f0198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:56.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.000+0000 7f75f803c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75f01036e0 0x7f75f0198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:56.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.000+0000 7f75f803c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75f01082a0 0x7f75f019cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:56.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.000+0000 7f75f803c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f75f01120e0 con 0x7f75f0102d30 2024-09-19T08:47:56.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.000+0000 7f75f803c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f75f0111f60 con 0x7f75f01036e0 2024-09-19T08:47:56.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.000+0000 7f75f803c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f75f0112260 con 0x7f75f01082a0 2024-09-19T08:47:56.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.001+0000 7f75f5dd8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75f0102d30 0x7f75f0198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:56.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.001+0000 7f75f5dd8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75f0102d30 0x7f75f0198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47372/0 (socket says 172.21.15.164:47372) 2024-09-19T08:47:56.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.001+0000 7f75f5dd8640 1 -- 172.21.15.164:0/2071406409 learned_addr learned my addr 172.21.15.164:0/2071406409 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:56.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.001+0000 7f75f55d7640 1 --2- 172.21.15.164:0/2071406409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75f01036e0 0x7f75f0198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:56.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.001+0000 7f75f65d9640 1 --2- 172.21.15.164:0/2071406409 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75f01082a0 0x7f75f019cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:56.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.001+0000 7f75f5dd8640 1 -- 172.21.15.164:0/2071406409 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75f01082a0 msgr2=0x7f75f019cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:56.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.001+0000 7f75f5dd8640 1 --2- 172.21.15.164:0/2071406409 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75f01082a0 0x7f75f019cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:56.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.001+0000 7f75f5dd8640 1 -- 172.21.15.164:0/2071406409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75f01036e0 msgr2=0x7f75f0198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:56.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.001+0000 7f75f5dd8640 1 --2- 172.21.15.164:0/2071406409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75f01036e0 0x7f75f0198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:56.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.001+0000 7f75f5dd8640 1 -- 172.21.15.164:0/2071406409 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75f019d5d0 con 0x7f75f0102d30 2024-09-19T08:47:56.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.001+0000 7f75f5dd8640 1 --2- 172.21.15.164:0/2071406409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75f0102d30 0x7f75f0198620 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f75e00098d0 tx=0x7f75e0031460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:56.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.001+0000 7f75eeffd640 1 -- 172.21.15.164:0/2071406409 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e0031850 con 0x7f75f0102d30 2024-09-19T08:47:56.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.001+0000 7f75eeffd640 1 -- 172.21.15.164:0/2071406409 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75e0004520 con 0x7f75f0102d30 2024-09-19T08:47:56.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.001+0000 7f75f803c640 1 -- 172.21.15.164:0/2071406409 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75f019d860 con 0x7f75f0102d30 2024-09-19T08:47:56.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.001+0000 7f75eeffd640 1 -- 172.21.15.164:0/2071406409 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e002fe20 con 0x7f75f0102d30 2024-09-19T08:47:56.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.001+0000 7f75f803c640 1 -- 172.21.15.164:0/2071406409 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75f01a51a0 con 0x7f75f0102d30 2024-09-19T08:47:56.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.003+0000 7f75eeffd640 1 -- 172.21.15.164:0/2071406409 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f75e0004040 con 0x7f75f0102d30 2024-09-19T08:47:56.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.004+0000 7f75fa415740 1 Processor -- start 2024-09-19T08:47:56.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.004+0000 7f75fa415740 1 -- start start 2024-09-19T08:47:56.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.004+0000 7f75fa415740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578dbfb7b40 0x5578dbfb6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:56.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.004+0000 7f75fa415740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5578dbfb65d0 0x5578dbfb6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:56.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.004+0000 7f75fa415740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578dbfc1370 0x5578dbfc3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:56.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.005+0000 7f75fa415740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5578dbf1c2f0 con 0x5578dbfb7b40 2024-09-19T08:47:56.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.005+0000 7f75fa415740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5578dbec3cf0 con 0x5578dbfb65d0 2024-09-19T08:47:56.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.005+0000 7f75fa415740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5578dbf182b0 con 0x5578dbfc1370 2024-09-19T08:47:56.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.005+0000 7f75eeffd640 1 --2- 172.21.15.164:0/2071406409 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f75d80a33b0 0x7f75d80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:56.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.005+0000 7f75f65d9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578dbfb7b40 0x5578dbfb6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:56.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.005+0000 7f75f65d9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578dbfb7b40 0x5578dbfb6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47388/0 (socket says 172.21.15.164:47388) 2024-09-19T08:47:56.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.005+0000 7f75f65d9640 1 -- 172.21.15.164:0/1504062103 learned_addr learned my addr 172.21.15.164:0/1504062103 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:56.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.005+0000 7f75f65d9640 1 --2- 172.21.15.164:0/2071406409 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f75d80a33b0 0x7f75d80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:56.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.005+0000 7f75eeffd640 1 -- 172.21.15.164:0/2071406409 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f75e00f12a0 con 0x7f75f0102d30 2024-09-19T08:47:56.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.005+0000 7f75f5dd8640 1 --2- 172.21.15.164:0/1504062103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5578dbfb65d0 0x5578dbfb6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:56.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.005+0000 7f75f55d7640 1 --2- 172.21.15.164:0/1504062103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578dbfc1370 0x5578dbfc3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:56.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.005+0000 7f75f65d9640 1 -- 172.21.15.164:0/1504062103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578dbfc1370 msgr2=0x5578dbfc3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:56.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.005+0000 7f75f65d9640 1 --2- 172.21.15.164:0/1504062103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578dbfc1370 0x5578dbfc3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:56.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.005+0000 7f75f65d9640 1 -- 172.21.15.164:0/1504062103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5578dbfb65d0 msgr2=0x5578dbfb6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:56.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.005+0000 7f75f65d9640 1 --2- 172.21.15.164:0/1504062103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5578dbfb65d0 0x5578dbfb6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:56.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.005+0000 7f75f65d9640 1 -- 172.21.15.164:0/1504062103 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578dbea83a0 con 0x5578dbfb7b40 2024-09-19T08:47:56.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.005+0000 7f75f65d9640 1 --2- 172.21.15.164:0/1504062103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578dbfb7b40 0x5578dbfb6090 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f75e8012780 tx=0x7f75e8012c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df115c1fdd2c768d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:56.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.005+0000 7f75d7fff640 1 -- 172.21.15.164:0/1504062103 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e8022020 con 0x5578dbfb7b40 2024-09-19T08:47:56.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.005+0000 7f75f65d9640 1 --2- 172.21.15.164:0/2071406409 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f75d80a33b0 0x7f75d80a5870 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f75e8020000 tx=0x7f75e801f000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:56.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.005+0000 7f75d7fff640 1 -- 172.21.15.164:0/1504062103 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75e8003ad0 con 0x5578dbfb7b40 2024-09-19T08:47:56.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.006+0000 7f75d7fff640 1 -- 172.21.15.164:0/1504062103 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e801e7f0 con 0x5578dbfb7b40 2024-09-19T08:47:56.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.006+0000 7f75fa415740 1 -- 172.21.15.164:0/1504062103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578dbfb7b40 msgr2=0x5578dbfb6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:56.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.006+0000 7f75fa415740 1 --2- 172.21.15.164:0/1504062103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578dbfb7b40 0x5578dbfb6090 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f75e8012780 tx=0x7f75e8012c40 comp rx=0 tx=0).stop 2024-09-19T08:47:56.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.006+0000 7f75fa415740 1 -- 172.21.15.164:0/1504062103 shutdown_connections 2024-09-19T08:47:56.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.006+0000 7f75fa415740 1 --2- 172.21.15.164:0/1504062103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578dbfc1370 0x5578dbfc3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:56.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.006+0000 7f75fa415740 1 --2- 172.21.15.164:0/1504062103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5578dbfb65d0 0x5578dbfb6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:56.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.006+0000 7f75fa415740 1 --2- 172.21.15.164:0/1504062103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578dbfb7b40 0x5578dbfb6090 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:56.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.006+0000 7f75fa415740 1 -- 172.21.15.164:0/1504062103 >> 172.21.15.164:0/1504062103 conn(0x5578dbf7f310 msgr2=0x5578dbfb2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:56.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.006+0000 7f75fa415740 1 -- 172.21.15.164:0/1504062103 shutdown_connections 2024-09-19T08:47:56.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.006+0000 7f75fa415740 1 -- 172.21.15.164:0/1504062103 wait complete. 2024-09-19T08:47:56.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.007+0000 7f75fa415740 1 -- start start 2024-09-19T08:47:56.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.007+0000 7f75fa415740 1 Processor -- start 2024-09-19T08:47:56.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.007+0000 7f75fa415740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578dbfb65d0 0x5578dbfc0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:56.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.007+0000 7f75fa415740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578dbfb7b40 0x5578dbfbe040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:56.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.007+0000 7f75fa415740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5578dbfc1370 0x5578dbfbe650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:56.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.007+0000 7f75fa415740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5578dbfc4010 con 0x5578dbfb7b40 2024-09-19T08:47:56.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.007+0000 7f75fa415740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5578dbed7060 con 0x5578dbfc1370 2024-09-19T08:47:56.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.007+0000 7f75fa415740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5578dbfc4190 con 0x5578dbfb65d0 2024-09-19T08:47:56.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.007+0000 7f75fa415740 10 client.0 _ll_register_callbacks cb 0x5578dbc2a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:47:56.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.007+0000 7f75f55d7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578dbfb7b40 0x5578dbfbe040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:56.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.007+0000 7f75f55d7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578dbfb7b40 0x5578dbfbe040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47390/0 (socket says 172.21.15.164:47390) 2024-09-19T08:47:56.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.007+0000 7f75f55d7640 1 -- 172.21.15.164:0/3065411811 learned_addr learned my addr 172.21.15.164:0/3065411811 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:56.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.007+0000 7f75f5dd8640 1 --2- 172.21.15.164:0/3065411811 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578dbfb65d0 0x5578dbfc0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:56.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.007+0000 7f75f65d9640 1 --2- 172.21.15.164:0/3065411811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5578dbfc1370 0x5578dbfbe650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:56.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.008+0000 7f75f55d7640 1 -- 172.21.15.164:0/3065411811 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578dbfb65d0 msgr2=0x5578dbfc0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:56.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.008+0000 7f75f55d7640 1 --2- 172.21.15.164:0/3065411811 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578dbfb65d0 0x5578dbfc0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:56.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.008+0000 7f75f55d7640 1 -- 172.21.15.164:0/3065411811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5578dbfc1370 msgr2=0x5578dbfbe650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:56.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.008+0000 7f75f55d7640 1 --2- 172.21.15.164:0/3065411811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5578dbfc1370 0x5578dbfbe650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:56.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.008+0000 7f75f55d7640 1 -- 172.21.15.164:0/3065411811 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578dbc8d3c0 con 0x5578dbfb7b40 2024-09-19T08:47:56.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.008+0000 7f75f55d7640 1 --2- 172.21.15.164:0/3065411811 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578dbfb7b40 0x5578dbfbe040 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009f60 tx=0x7f75e4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7856a2769e2f5d6f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:56.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.008+0000 7f75d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:47:56.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.008+0000 7f75d57fa640 1 -- 172.21.15.164:0/3065411811 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e4017020 con 0x5578dbfb7b40 2024-09-19T08:47:56.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.008+0000 7f75d57fa640 1 -- 172.21.15.164:0/3065411811 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f75e4013410 con 0x5578dbfb7b40 2024-09-19T08:47:56.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.008+0000 7f75fa415740 1 -- 172.21.15.164:0/3065411811 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75e8003ad0 con 0x5578dbfb7b40 2024-09-19T08:47:56.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.008+0000 7f75d57fa640 1 -- 172.21.15.164:0/3065411811 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f75e4022050 con 0x5578dbfb7b40 2024-09-19T08:47:56.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.008+0000 7f75fa415740 10 client.15972 fetch_fsmap learned FSMap version 28 2024-09-19T08:47:56.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.008+0000 7f75fa415740 1 -- 172.21.15.164:0/3065411811 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5578dbfcfb40 con 0x5578dbfb7b40 2024-09-19T08:47:56.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.008+0000 7f75d57fa640 1 -- 172.21.15.164:0/3065411811 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75e8003ad0 con 0x5578dbfb7b40 2024-09-19T08:47:56.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.008+0000 7f75d57fa640 1 -- 172.21.15.164:0/3065411811 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f75e4012050 con 0x5578dbfb7b40 2024-09-19T08:47:56.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.008+0000 7f75fa415740 10 client.15972 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:47:56.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.008+0000 7f75fa415740 10 client.15972 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T08:47:56.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.008+0000 7f75fa415740 20 client.15972 populate_metadata read hostname 'smithi164' 2024-09-19T08:47:56.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.008+0000 7f75fa415740 1 --2- 172.21.15.164:0/3065411811 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x5578dbfd10a0 0x5578dbfd34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:56.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.008+0000 7f75fa415740 4 client.15972 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:47:56.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.008+0000 7f75fa415740 1 -- 172.21.15.164:0/3065411811 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578dbf29860 con 0x5578dbfd10a0 2024-09-19T08:47:56.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.009+0000 7f75f5dd8640 1 --2- 172.21.15.164:0/3065411811 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x5578dbfd10a0 0x5578dbfd34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:56.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.009+0000 7f75d57fa640 10 client.15972 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:47:56.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.009+0000 7f75f5dd8640 1 --2- 172.21.15.164:0/3065411811 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x5578dbfd10a0 0x5578dbfd34c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f3c7487ff1b84131 server_cookie=bf28dbd22a4f139f in_seq=0 out_seq=0 2024-09-19T08:47:56.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.010+0000 7f75d57fa640 1 -- 172.21.15.164:0/3065411811 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5578dbf29860 con 0x5578dbfd10a0 2024-09-19T08:47:56.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.010+0000 7f75d57fa640 10 client.15972 handle_command_reply: tid=0 2024-09-19T08:47:56.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.010+0000 7f75fa415740 1 client.15972 shutdown 2024-09-19T08:47:56.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.010+0000 7f75fa415740 1 -- 172.21.15.164:0/3065411811 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578dbfb7b40 msgr2=0x5578dbfbe040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:56.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.010+0000 7f75fa415740 1 --2- 172.21.15.164:0/3065411811 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578dbfb7b40 0x5578dbfbe040 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009f60 tx=0x7f75e4009f90 comp rx=0 tx=0).stop 2024-09-19T08:47:56.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.010+0000 7f75fa415740 1 -- 172.21.15.164:0/3065411811 shutdown_connections 2024-09-19T08:47:56.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.010+0000 7f75fa415740 1 --2- 172.21.15.164:0/3065411811 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x5578dbfd10a0 0x5578dbfd34c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:56.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.010+0000 7f75fa415740 1 --2- 172.21.15.164:0/3065411811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5578dbfc1370 0x5578dbfbe650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:56.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.010+0000 7f75fa415740 1 --2- 172.21.15.164:0/3065411811 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578dbfb7b40 0x5578dbfbe040 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:56.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.010+0000 7f75fa415740 1 --2- 172.21.15.164:0/3065411811 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578dbfb65d0 0x5578dbfc0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:56.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.010+0000 7f75d57fa640 0 client.15972 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:47:56.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.010+0000 7f75fa415740 1 -- 172.21.15.164:0/3065411811 >> 172.21.15.164:0/3065411811 conn(0x5578dbf7f310 msgr2=0x5578dbfb8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:56.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.010+0000 7f75fa415740 1 -- 172.21.15.164:0/3065411811 shutdown_connections 2024-09-19T08:47:56.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.010+0000 7f75fa415740 1 -- 172.21.15.164:0/3065411811 wait complete. 2024-09-19T08:47:56.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.010+0000 7f75fa415740 20 client.15972 trim_cache size 0 max 16384 2024-09-19T08:47:56.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.023+0000 7f75fa415740 1 Processor -- start 2024-09-19T08:47:56.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.023+0000 7f75fa415740 1 -- start start 2024-09-19T08:47:56.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.023+0000 7f75fa415740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578dbfb65d0 0x5578dbfb2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:56.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.023+0000 7f75fa415740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5578dbfb7b40 0x5578dbfcd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:56.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.023+0000 7f75fa415740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578dbfc1370 0x5578dbfbd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:56.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.023+0000 7f75fa415740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f75f01ad750 con 0x5578dbfb65d0 2024-09-19T08:47:56.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.023+0000 7f75fa415740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f75f01af290 con 0x5578dbfb7b40 2024-09-19T08:47:56.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.023+0000 7f75fa415740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f75f01ade20 con 0x5578dbfc1370 2024-09-19T08:47:56.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.023+0000 7f75f5dd8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578dbfb65d0 0x5578dbfb2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:56.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.023+0000 7f75f5dd8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578dbfb65d0 0x5578dbfb2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47406/0 (socket says 172.21.15.164:47406) 2024-09-19T08:47:56.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.023+0000 7f75f5dd8640 1 -- 172.21.15.164:0/3912453312 learned_addr learned my addr 172.21.15.164:0/3912453312 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:56.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.024+0000 7f75f55d7640 1 --2- 172.21.15.164:0/3912453312 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5578dbfb7b40 0x5578dbfcd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:56.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.024+0000 7f75f65d9640 1 --2- 172.21.15.164:0/3912453312 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578dbfc1370 0x5578dbfbd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:56.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.024+0000 7f75f5dd8640 1 -- 172.21.15.164:0/3912453312 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578dbfc1370 msgr2=0x5578dbfbd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:56.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.024+0000 7f75f5dd8640 1 --2- 172.21.15.164:0/3912453312 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578dbfc1370 0x5578dbfbd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:56.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.024+0000 7f75f5dd8640 1 -- 172.21.15.164:0/3912453312 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5578dbfb7b40 msgr2=0x5578dbfcd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:56.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.024+0000 7f75f5dd8640 1 --2- 172.21.15.164:0/3912453312 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5578dbfb7b40 0x5578dbfcd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:56.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.024+0000 7f75f5dd8640 1 -- 172.21.15.164:0/3912453312 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578dbfb2510 con 0x5578dbfb65d0 2024-09-19T08:47:56.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.024+0000 7f75f5dd8640 1 --2- 172.21.15.164:0/3912453312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578dbfb65d0 0x5578dbfb2d70 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f75e0006010 tx=0x7f75e002f8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f30403bf4d3100d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:56.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.024+0000 7f75d77fe640 1 -- 172.21.15.164:0/3912453312 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e00bdda0 con 0x5578dbfb65d0 2024-09-19T08:47:56.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.024+0000 7f75d77fe640 1 -- 172.21.15.164:0/3912453312 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75e0036040 con 0x5578dbfb65d0 2024-09-19T08:47:56.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.024+0000 7f75d77fe640 1 -- 172.21.15.164:0/3912453312 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e00bb9a0 con 0x5578dbfb65d0 2024-09-19T08:47:56.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.025+0000 7f75fa415740 1 -- 172.21.15.164:0/3912453312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578dbfb65d0 msgr2=0x5578dbfb2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:56.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.025+0000 7f75fa415740 1 --2- 172.21.15.164:0/3912453312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578dbfb65d0 0x5578dbfb2d70 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f75e0006010 tx=0x7f75e002f8c0 comp rx=0 tx=0).stop 2024-09-19T08:47:56.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.025+0000 7f75fa415740 1 -- 172.21.15.164:0/3912453312 shutdown_connections 2024-09-19T08:47:56.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.025+0000 7f75fa415740 1 --2- 172.21.15.164:0/3912453312 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578dbfc1370 0x5578dbfbd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:56.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.025+0000 7f75fa415740 1 --2- 172.21.15.164:0/3912453312 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5578dbfb7b40 0x5578dbfcd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:56.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.025+0000 7f75fa415740 1 --2- 172.21.15.164:0/3912453312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578dbfb65d0 0x5578dbfb2d70 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:56.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.025+0000 7f75fa415740 1 -- 172.21.15.164:0/3912453312 >> 172.21.15.164:0/3912453312 conn(0x5578dbf7f310 msgr2=0x5578dbfbc910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:56.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.025+0000 7f75fa415740 1 -- 172.21.15.164:0/3912453312 shutdown_connections 2024-09-19T08:47:56.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.025+0000 7f75fa415740 1 -- 172.21.15.164:0/3912453312 wait complete. 2024-09-19T08:47:56.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.025+0000 7f75fa415740 1 -- start start 2024-09-19T08:47:56.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.025+0000 7f75fa415740 1 Processor -- start 2024-09-19T08:47:56.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.026+0000 7f75fa415740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578dbfb65d0 0x5578dbfc99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:56.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.026+0000 7f75fa415740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578dbfb7b40 0x5578dbfc4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:56.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.026+0000 7f75fa415740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5578dbfc1370 0x5578dbfc5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:56.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.026+0000 7f75fa415740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f75f01aebc0 con 0x5578dbfb7b40 2024-09-19T08:47:56.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.026+0000 7f75fa415740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f75f01ae4f0 con 0x5578dbfc1370 2024-09-19T08:47:56.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.026+0000 7f75fa415740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f75f01ab770 con 0x5578dbfb65d0 2024-09-19T08:47:56.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.026+0000 7f75fa415740 10 client.0 _ll_register_callbacks cb 0x7f75e4006f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:47:56.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.026+0000 7f75f55d7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578dbfb7b40 0x5578dbfc4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:56.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.026+0000 7f75f55d7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578dbfb7b40 0x5578dbfc4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47416/0 (socket says 172.21.15.164:47416) 2024-09-19T08:47:56.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.026+0000 7f75f55d7640 1 -- 172.21.15.164:0/782464812 learned_addr learned my addr 172.21.15.164:0/782464812 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:56.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.026+0000 7f75f65d9640 1 --2- 172.21.15.164:0/782464812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5578dbfc1370 0x5578dbfc5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:56.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.026+0000 7f75f5dd8640 1 --2- 172.21.15.164:0/782464812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578dbfb65d0 0x5578dbfc99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:56.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.026+0000 7f75f55d7640 1 -- 172.21.15.164:0/782464812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578dbfb65d0 msgr2=0x5578dbfc99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:56.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.026+0000 7f75f55d7640 1 --2- 172.21.15.164:0/782464812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578dbfb65d0 0x5578dbfc99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:56.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.026+0000 7f75f55d7640 1 -- 172.21.15.164:0/782464812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5578dbfc1370 msgr2=0x5578dbfc5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:56.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.026+0000 7f75f55d7640 1 --2- 172.21.15.164:0/782464812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5578dbfc1370 0x5578dbfc5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:56.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.026+0000 7f75f55d7640 1 -- 172.21.15.164:0/782464812 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578dbfc58c0 con 0x5578dbfb7b40 2024-09-19T08:47:56.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.026+0000 7f75f55d7640 1 --2- 172.21.15.164:0/782464812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578dbfb7b40 0x5578dbfc4d10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f75e401b8c0 tx=0x7f75e4009bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f395dd081978382 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:56.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.026+0000 7f75d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:47:56.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.026+0000 7f75d57fa640 1 -- 172.21.15.164:0/782464812 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e402c070 con 0x5578dbfb7b40 2024-09-19T08:47:56.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.026+0000 7f75d57fa640 1 -- 172.21.15.164:0/782464812 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f75e401b5a0 con 0x5578dbfb7b40 2024-09-19T08:47:56.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.026+0000 7f75fa415740 1 -- 172.21.15.164:0/782464812 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75e0036040 con 0x5578dbfb7b40 2024-09-19T08:47:56.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.027+0000 7f75d57fa640 1 -- 172.21.15.164:0/782464812 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f75e4037050 con 0x5578dbfb7b40 2024-09-19T08:47:56.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.027+0000 7f75fa415740 10 client.15978 fetch_fsmap learned FSMap version 28 2024-09-19T08:47:56.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.027+0000 7f75fa415740 1 -- 172.21.15.164:0/782464812 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5578dc03bc60 con 0x5578dbfb7b40 2024-09-19T08:47:56.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.027+0000 7f75d57fa640 1 -- 172.21.15.164:0/782464812 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75e0036040 con 0x5578dbfb7b40 2024-09-19T08:47:56.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.027+0000 7f75d57fa640 1 -- 172.21.15.164:0/782464812 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f75e4012040 con 0x5578dbfb7b40 2024-09-19T08:47:56.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.027+0000 7f75fa415740 10 client.15978 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:47:56.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.027+0000 7f75fa415740 10 client.15978 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T08:47:56.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.027+0000 7f75fa415740 20 client.15978 populate_metadata read hostname 'smithi164' 2024-09-19T08:47:56.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.027+0000 7f75fa415740 1 --2- 172.21.15.164:0/782464812 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x5578dc03c410 0x5578dc03e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:56.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.027+0000 7f75fa415740 4 client.15978 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:47:56.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.027+0000 7f75fa415740 1 -- 172.21.15.164:0/782464812 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5578dbf23d70 con 0x5578dc03c410 2024-09-19T08:47:56.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.027+0000 7f75f5dd8640 1 --2- 172.21.15.164:0/782464812 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x5578dc03c410 0x5578dc03e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:56.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.028+0000 7f75f5dd8640 1 --2- 172.21.15.164:0/782464812 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x5578dc03c410 0x5578dc03e830 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=77d8a5e30184dfcb server_cookie=bbcc6ce4e4d37705 in_seq=0 out_seq=0 2024-09-19T08:47:56.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.028+0000 7f75d57fa640 10 client.15978 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:47:56.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.028+0000 7f75d57fa640 1 -- 172.21.15.164:0/782464812 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5578dbf23d70 con 0x5578dc03c410 2024-09-19T08:47:56.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.028+0000 7f75d57fa640 10 client.15978 handle_command_reply: tid=0 2024-09-19T08:47:56.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.028+0000 7f75fa415740 1 client.15978 shutdown 2024-09-19T08:47:56.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.028+0000 7f75fa415740 1 -- 172.21.15.164:0/782464812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578dbfb7b40 msgr2=0x5578dbfc4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:56.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.028+0000 7f75fa415740 1 --2- 172.21.15.164:0/782464812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578dbfb7b40 0x5578dbfc4d10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f75e401b8c0 tx=0x7f75e4009bc0 comp rx=0 tx=0).stop 2024-09-19T08:47:56.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.028+0000 7f75fa415740 1 -- 172.21.15.164:0/782464812 shutdown_connections 2024-09-19T08:47:56.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.028+0000 7f75fa415740 1 --2- 172.21.15.164:0/782464812 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x5578dc03c410 0x5578dc03e830 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:56.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.028+0000 7f75fa415740 1 --2- 172.21.15.164:0/782464812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5578dbfc1370 0x5578dbfc5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:56.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.028+0000 7f75fa415740 1 --2- 172.21.15.164:0/782464812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5578dbfb7b40 0x5578dbfc4d10 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:56.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.028+0000 7f75d57fa640 0 client.15978 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:47:56.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.028+0000 7f75fa415740 1 --2- 172.21.15.164:0/782464812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578dbfb65d0 0x5578dbfc99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:56.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.028+0000 7f75fa415740 1 -- 172.21.15.164:0/782464812 >> 172.21.15.164:0/782464812 conn(0x5578dbf7f310 msgr2=0x5578dbfbd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:56.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.028+0000 7f75fa415740 1 -- 172.21.15.164:0/782464812 shutdown_connections 2024-09-19T08:47:56.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.028+0000 7f75fa415740 1 -- 172.21.15.164:0/782464812 wait complete. 2024-09-19T08:47:56.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.028+0000 7f75fa415740 20 client.15978 trim_cache size 0 max 16384 2024-09-19T08:47:56.062 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:47:56.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.029+0000 7f75f803c640 1 -- 172.21.15.164:0/2071406409 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f75d80a33b0 msgr2=0x7f75d80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:56.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.029+0000 7f75f803c640 1 --2- 172.21.15.164:0/2071406409 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f75d80a33b0 0x7f75d80a5870 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f75e8020000 tx=0x7f75e801f000 comp rx=0 tx=0).stop 2024-09-19T08:47:56.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.029+0000 7f75f803c640 1 -- 172.21.15.164:0/2071406409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75f0102d30 msgr2=0x7f75f0198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:56.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.029+0000 7f75f803c640 1 --2- 172.21.15.164:0/2071406409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75f0102d30 0x7f75f0198620 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f75e00098d0 tx=0x7f75e0031460 comp rx=0 tx=0).stop 2024-09-19T08:47:56.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.029+0000 7f75f803c640 1 -- 172.21.15.164:0/2071406409 shutdown_connections 2024-09-19T08:47:56.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.029+0000 7f75f803c640 1 --2- 172.21.15.164:0/2071406409 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75f01082a0 0x7f75f019cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:56.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.029+0000 7f75f803c640 1 --2- 172.21.15.164:0/2071406409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75f01036e0 0x7f75f0198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:56.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.029+0000 7f75f803c640 1 --2- 172.21.15.164:0/2071406409 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f75d80a33b0 0x7f75d80a5870 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:56.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.029+0000 7f75f803c640 1 --2- 172.21.15.164:0/2071406409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75f0102d30 0x7f75f0198620 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:56.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.029+0000 7f75f803c640 1 -- 172.21.15.164:0/2071406409 >> 172.21.15.164:0/2071406409 conn(0x7f75f00fc7d0 msgr2=0x7f75f0105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:56.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.029+0000 7f75f803c640 1 -- 172.21.15.164:0/2071406409 shutdown_connections 2024-09-19T08:47:56.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:56.029+0000 7f75f803c640 1 -- 172.21.15.164:0/2071406409 wait complete. 2024-09-19T08:47:56.736 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2966dfc9', quiesce took 15.6 seconds. Will release after: 4.750000000000002 2024-09-19T08:47:57.221 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T08:47:57.473 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:47:57.531 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:57 smithi164 ceph-mon[30928]: pgmap v67: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 770 MiB/s wr, 3.32k op/s 2024-09-19T08:47:57.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:57 smithi170 ceph-mon[35072]: pgmap v67: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 770 MiB/s wr, 3.32k op/s 2024-09-19T08:47:57.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:57 smithi173 ceph-mon[35306]: pgmap v67: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 770 MiB/s wr, 3.32k op/s 2024-09-19T08:47:58.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.118+0000 7f8852e3c640 1 -- 172.21.15.164:0/1734188832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88440a5140 msgr2=0x7f88440b7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:58.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.118+0000 7f8852e3c640 1 --2- 172.21.15.164:0/1734188832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88440a5140 0x7f88440b7000 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f883c01a910 tx=0x7f883c03b500 comp rx=0 tx=0).stop 2024-09-19T08:47:58.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.119+0000 7f8852e3c640 1 -- 172.21.15.164:0/1734188832 shutdown_connections 2024-09-19T08:47:58.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.119+0000 7f8852e3c640 1 --2- 172.21.15.164:0/1734188832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88440b7540 0x7f88440b9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:58.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.119+0000 7f8852e3c640 1 --2- 172.21.15.164:0/1734188832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88440a5140 0x7f88440b7000 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:58.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.119+0000 7f8852e3c640 1 --2- 172.21.15.164:0/1734188832 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88440a4820 0x7f88440a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:58.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.119+0000 7f8852e3c640 1 -- 172.21.15.164:0/1734188832 >> 172.21.15.164:0/1734188832 conn(0x7f884401a0e0 msgr2=0x7f884401a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:58.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.120+0000 7f8852e3c640 1 -- 172.21.15.164:0/1734188832 shutdown_connections 2024-09-19T08:47:58.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.120+0000 7f8852e3c640 1 -- 172.21.15.164:0/1734188832 wait complete. 2024-09-19T08:47:58.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.121+0000 7f8852e3c640 1 Processor -- start 2024-09-19T08:47:58.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.121+0000 7f8852e3c640 1 -- start start 2024-09-19T08:47:58.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.122+0000 7f8852e3c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88440a4820 0x7f88440b6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:58.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.122+0000 7f8852e3c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88440b7540 0x7f88440b6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:58.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.122+0000 7f8852e3c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88440b2950 0x7f88440b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:58.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.122+0000 7f8852e3c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f88440af170 con 0x7f88440b7540 2024-09-19T08:47:58.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.122+0000 7f8852e3c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f88440aeff0 con 0x7f88440a4820 2024-09-19T08:47:58.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.122+0000 7f8852e3c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f88440af2f0 con 0x7f88440b2950 2024-09-19T08:47:58.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.123+0000 7f885263b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88440b2950 0x7f88440b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:58.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.123+0000 7f885263b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88440b2950 0x7f88440b2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:45218/0 (socket says 172.21.15.164:45218) 2024-09-19T08:47:58.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.123+0000 7f885263b640 1 -- 172.21.15.164:0/3684170653 learned_addr learned my addr 172.21.15.164:0/3684170653 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:58.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.123+0000 7f8851639640 1 --2- 172.21.15.164:0/3684170653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88440b7540 0x7f88440b6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:58.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.124+0000 7f885263b640 1 -- 172.21.15.164:0/3684170653 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88440a4820 msgr2=0x7f88440b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:58.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.124+0000 7f885263b640 1 --2- 172.21.15.164:0/3684170653 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88440a4820 0x7f88440b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:58.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.124+0000 7f885263b640 1 -- 172.21.15.164:0/3684170653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88440b7540 msgr2=0x7f88440b6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:58.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.124+0000 7f885263b640 1 --2- 172.21.15.164:0/3684170653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88440b7540 0x7f88440b6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:58.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.124+0000 7f885263b640 1 -- 172.21.15.164:0/3684170653 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88440b3670 con 0x7f88440b2950 2024-09-19T08:47:58.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.125+0000 7f885263b640 1 --2- 172.21.15.164:0/3684170653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88440b2950 0x7f88440b2db0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f884c0518b0 tx=0x7f884c06ef70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:58.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.126+0000 7f8842ffd640 1 -- 172.21.15.164:0/3684170653 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884c06bbd0 con 0x7f88440b2950 2024-09-19T08:47:58.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.126+0000 7f8852e3c640 1 -- 172.21.15.164:0/3684170653 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f884400b810 con 0x7f88440b2950 2024-09-19T08:47:58.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.126+0000 7f8852e3c640 1 -- 172.21.15.164:0/3684170653 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f884400bcd0 con 0x7f88440b2950 2024-09-19T08:47:58.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.126+0000 7f8842ffd640 1 -- 172.21.15.164:0/3684170653 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f884c06bd70 con 0x7f88440b2950 2024-09-19T08:47:58.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.126+0000 7f8842ffd640 1 -- 172.21.15.164:0/3684170653 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884c078650 con 0x7f88440b2950 2024-09-19T08:47:58.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.135+0000 7f8856477740 1 Processor -- start 2024-09-19T08:47:58.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.137+0000 7f8856477740 1 -- start start 2024-09-19T08:47:58.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.137+0000 7f8856477740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715a4eecf0 0x55715a4ed240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:58.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.137+0000 7f8856477740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55715a4ed780 0x55715a4edbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:58.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.137+0000 7f8856477740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55715a4f8520 0x55715a4fa910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:58.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.137+0000 7f8856477740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55715a4532f0 con 0x55715a4eecf0 2024-09-19T08:47:58.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.137+0000 7f8856477740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55715a3facf0 con 0x55715a4f8520 2024-09-19T08:47:58.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.137+0000 7f8856477740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55715a44f2b0 con 0x55715a4ed780 2024-09-19T08:47:58.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.137+0000 7f8842ffd640 1 -- 172.21.15.164:0/3684170653 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f884c080030 con 0x7f88440b2950 2024-09-19T08:47:58.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.138+0000 7f8851639640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55715a4f8520 0x55715a4fa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:58.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.138+0000 7f8851639640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55715a4f8520 0x55715a4fa910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:59540/0 (socket says 172.21.15.164:59540) 2024-09-19T08:47:58.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.138+0000 7f8851639640 1 -- 172.21.15.164:0/3671306672 learned_addr learned my addr 172.21.15.164:0/3671306672 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:58.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.138+0000 7f8851e3a640 1 --2- 172.21.15.164:0/3671306672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55715a4ed780 0x55715a4edbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:58.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.140+0000 7f8842ffd640 1 --2- 172.21.15.164:0/3684170653 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f88380a3500 0x7f88380a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:58.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.140+0000 7f8851639640 1 --2- 172.21.15.164:0/3671306672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715a4eecf0 0x55715a4ed240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:58.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.140+0000 7f8851e3a640 1 -- 172.21.15.164:0/3671306672 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55715a4f8520 msgr2=0x55715a4fa910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:58.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.140+0000 7f8851e3a640 1 --2- 172.21.15.164:0/3671306672 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55715a4f8520 0x55715a4fa910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:58.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.140+0000 7f8851e3a640 1 -- 172.21.15.164:0/3671306672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715a4eecf0 msgr2=0x55715a4ed240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:58.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.140+0000 7f8851e3a640 1 --2- 172.21.15.164:0/3671306672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715a4eecf0 0x55715a4ed240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:58.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.140+0000 7f8851e3a640 1 -- 172.21.15.164:0/3671306672 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55715a3df3a0 con 0x55715a4ed780 2024-09-19T08:47:58.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.141+0000 7f8851e3a640 1 --2- 172.21.15.164:0/3671306672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55715a4ed780 0x55715a4edbc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8848009b40 tx=0x7f884800e350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49864fccfbaa4a46 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:58.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.141+0000 7f885263b640 1 --2- 172.21.15.164:0/3684170653 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f88380a3500 0x7f88380a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:58.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.141+0000 7f8842ffd640 1 -- 172.21.15.164:0/3684170653 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f884c133460 con 0x7f88440b2950 2024-09-19T08:47:58.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.141+0000 7f8837fff640 1 -- 172.21.15.164:0/3671306672 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884800ed80 con 0x55715a4ed780 2024-09-19T08:47:58.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.141+0000 7f8837fff640 1 -- 172.21.15.164:0/3671306672 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8848005020 con 0x55715a4ed780 2024-09-19T08:47:58.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.141+0000 7f8837fff640 1 -- 172.21.15.164:0/3671306672 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8848004b60 con 0x55715a4ed780 2024-09-19T08:47:58.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.143+0000 7f8856477740 1 -- 172.21.15.164:0/3671306672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55715a4ed780 msgr2=0x55715a4edbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:58.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.143+0000 7f885263b640 1 --2- 172.21.15.164:0/3684170653 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f88380a3500 0x7f88380a59c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f884c051220 tx=0x7f884c071040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:58.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.143+0000 7f8856477740 1 --2- 172.21.15.164:0/3671306672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55715a4ed780 0x55715a4edbc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8848009b40 tx=0x7f884800e350 comp rx=0 tx=0).stop 2024-09-19T08:47:58.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.143+0000 7f8856477740 1 -- 172.21.15.164:0/3671306672 shutdown_connections 2024-09-19T08:47:58.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.143+0000 7f8856477740 1 --2- 172.21.15.164:0/3671306672 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55715a4f8520 0x55715a4fa910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:58.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.143+0000 7f8856477740 1 --2- 172.21.15.164:0/3671306672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55715a4ed780 0x55715a4edbc0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:58.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.143+0000 7f8856477740 1 --2- 172.21.15.164:0/3671306672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715a4eecf0 0x55715a4ed240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:58.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.143+0000 7f8856477740 1 -- 172.21.15.164:0/3671306672 >> 172.21.15.164:0/3671306672 conn(0x55715a4b6310 msgr2=0x55715a4e9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:58.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.144+0000 7f8856477740 1 -- 172.21.15.164:0/3671306672 shutdown_connections 2024-09-19T08:47:58.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.144+0000 7f8856477740 1 -- 172.21.15.164:0/3671306672 wait complete. 2024-09-19T08:47:58.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.145+0000 7f8856477740 1 -- start start 2024-09-19T08:47:58.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.145+0000 7f8856477740 1 Processor -- start 2024-09-19T08:47:58.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.146+0000 7f8856477740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715a4ed780 0x55715a4f7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:58.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.146+0000 7f8856477740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55715a4eecf0 0x55715a4f5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:58.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.146+0000 7f8856477740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55715a4f8520 0x55715a4f56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:58.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.146+0000 7f8856477740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55715a4fb220 con 0x55715a4ed780 2024-09-19T08:47:58.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.146+0000 7f8856477740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55715a40e060 con 0x55715a4f8520 2024-09-19T08:47:58.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.146+0000 7f8856477740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55715a4fb3a0 con 0x55715a4eecf0 2024-09-19T08:47:58.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.146+0000 7f8856477740 10 client.0 _ll_register_callbacks cb 0x55715a1618c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:47:58.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.147+0000 7f8851639640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715a4ed780 0x55715a4f7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:58.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.147+0000 7f8851639640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715a4ed780 0x55715a4f7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47460/0 (socket says 172.21.15.164:47460) 2024-09-19T08:47:58.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.147+0000 7f8851639640 1 -- 172.21.15.164:0/943293660 learned_addr learned my addr 172.21.15.164:0/943293660 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:58.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.148+0000 7f8851639640 1 -- 172.21.15.164:0/943293660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55715a4eecf0 msgr2=0x55715a4f5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:58.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.148+0000 7f8851639640 1 --2- 172.21.15.164:0/943293660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55715a4eecf0 0x55715a4f5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:58.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.148+0000 7f8851639640 1 -- 172.21.15.164:0/943293660 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55715a4f8520 msgr2=0x55715a4f56a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:47:58.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.148+0000 7f8851639640 1 --2- 172.21.15.164:0/943293660 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55715a4f8520 0x55715a4f56a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:58.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.148+0000 7f8851639640 1 -- 172.21.15.164:0/943293660 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55715a1c43c0 con 0x55715a4ed780 2024-09-19T08:47:58.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.148+0000 7f8851639640 1 --2- 172.21.15.164:0/943293660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715a4ed780 0x55715a4f7e90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f883c044040 tx=0x7f883c005580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1cbe2b559f57e98 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:58.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.148+0000 7f88357fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:47:58.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.149+0000 7f88357fa640 1 -- 172.21.15.164:0/943293660 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c0055f0 con 0x55715a4ed780 2024-09-19T08:47:58.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.149+0000 7f88357fa640 1 -- 172.21.15.164:0/943293660 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f883c019040 con 0x55715a4ed780 2024-09-19T08:47:58.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.149+0000 7f8856477740 1 -- 172.21.15.164:0/943293660 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8848005020 con 0x55715a4ed780 2024-09-19T08:47:58.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.150+0000 7f88357fa640 1 -- 172.21.15.164:0/943293660 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f883c0048d0 con 0x55715a4ed780 2024-09-19T08:47:58.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.150+0000 7f8856477740 10 client.15990 fetch_fsmap learned FSMap version 28 2024-09-19T08:47:58.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.150+0000 7f8856477740 1 -- 172.21.15.164:0/943293660 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55715a506cf0 con 0x55715a4ed780 2024-09-19T08:47:58.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.150+0000 7f88357fa640 1 -- 172.21.15.164:0/943293660 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f883c040510 con 0x55715a4ed780 2024-09-19T08:47:58.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.151+0000 7f88357fa640 1 -- 172.21.15.164:0/943293660 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f883c050440 con 0x55715a4ed780 2024-09-19T08:47:58.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.151+0000 7f8856477740 10 client.15990 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:47:58.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.151+0000 7f8856477740 10 client.15990 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T08:47:58.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.151+0000 7f8856477740 20 client.15990 populate_metadata read hostname 'smithi164' 2024-09-19T08:47:58.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.151+0000 7f8856477740 1 --2- 172.21.15.164:0/943293660 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55715a508250 0x55715a50a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:58.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.151+0000 7f8856477740 4 client.15990 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:47:58.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.151+0000 7f8856477740 1 -- 172.21.15.164:0/943293660 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8848005020 con 0x55715a508250 2024-09-19T08:47:58.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.152+0000 7f8851e3a640 1 --2- 172.21.15.164:0/943293660 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55715a508250 0x55715a50a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:58.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.155+0000 7f8851e3a640 1 --2- 172.21.15.164:0/943293660 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55715a508250 0x55715a50a680 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=74fff79e4db56f93 server_cookie=9ab31294da48084c in_seq=0 out_seq=0 2024-09-19T08:47:58.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.155+0000 7f88357fa640 10 client.15990 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:47:58.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.156+0000 7f88357fa640 1 -- 172.21.15.164:0/943293660 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f8848005020 con 0x55715a508250 2024-09-19T08:47:58.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.156+0000 7f88357fa640 10 client.15990 handle_command_reply: tid=0 2024-09-19T08:47:58.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.156+0000 7f8856477740 1 client.15990 shutdown 2024-09-19T08:47:58.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.157+0000 7f8856477740 1 -- 172.21.15.164:0/943293660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715a4ed780 msgr2=0x55715a4f7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:58.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.157+0000 7f8856477740 1 --2- 172.21.15.164:0/943293660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715a4ed780 0x55715a4f7e90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f883c044040 tx=0x7f883c005580 comp rx=0 tx=0).stop 2024-09-19T08:47:58.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.157+0000 7f8856477740 1 -- 172.21.15.164:0/943293660 shutdown_connections 2024-09-19T08:47:58.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.157+0000 7f8856477740 1 --2- 172.21.15.164:0/943293660 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55715a508250 0x55715a50a680 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:58.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.157+0000 7f8856477740 1 --2- 172.21.15.164:0/943293660 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55715a4f8520 0x55715a4f56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:58.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.157+0000 7f8856477740 1 --2- 172.21.15.164:0/943293660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55715a4eecf0 0x55715a4f5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:58.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.157+0000 7f88357fa640 0 client.15990 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:47:58.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.157+0000 7f8856477740 1 --2- 172.21.15.164:0/943293660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715a4ed780 0x55715a4f7e90 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:58.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.157+0000 7f8856477740 1 -- 172.21.15.164:0/943293660 >> 172.21.15.164:0/943293660 conn(0x55715a4b6310 msgr2=0x55715a4efae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:58.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.157+0000 7f8856477740 1 -- 172.21.15.164:0/943293660 shutdown_connections 2024-09-19T08:47:58.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.157+0000 7f8856477740 1 -- 172.21.15.164:0/943293660 wait complete. 2024-09-19T08:47:58.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.157+0000 7f8856477740 20 client.15990 trim_cache size 0 max 16384 2024-09-19T08:47:58.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.174+0000 7f8856477740 1 Processor -- start 2024-09-19T08:47:58.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.174+0000 7f8856477740 1 -- start start 2024-09-19T08:47:58.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.175+0000 7f8856477740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55715a4ed780 0x55715a4f5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:58.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.175+0000 7f8856477740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715a4eecf0 0x55715a504550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:58.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.175+0000 7f8856477740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55715a4f8520 0x55715a4e4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:58.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.175+0000 7f8856477740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f883000e5c0 con 0x55715a4eecf0 2024-09-19T08:47:58.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.175+0000 7f8856477740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8830010100 con 0x55715a4ed780 2024-09-19T08:47:58.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.175+0000 7f8856477740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f883000ec90 con 0x55715a4f8520 2024-09-19T08:47:58.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.175+0000 7f8851e3a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715a4eecf0 0x55715a504550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:58.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.175+0000 7f8851e3a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715a4eecf0 0x55715a504550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47466/0 (socket says 172.21.15.164:47466) 2024-09-19T08:47:58.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.175+0000 7f8851e3a640 1 -- 172.21.15.164:0/2958419612 learned_addr learned my addr 172.21.15.164:0/2958419612 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:58.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.175+0000 7f8851639640 1 --2- 172.21.15.164:0/2958419612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55715a4f8520 0x55715a4e4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:58.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.175+0000 7f8851639640 1 --2- 172.21.15.164:0/2958419612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55715a4ed780 0x55715a4f5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:58.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.176+0000 7f8851e3a640 1 -- 172.21.15.164:0/2958419612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55715a4f8520 msgr2=0x55715a4e4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:58.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.176+0000 7f8851e3a640 1 --2- 172.21.15.164:0/2958419612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55715a4f8520 0x55715a4e4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:58.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.176+0000 7f8851e3a640 1 -- 172.21.15.164:0/2958419612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55715a4ed780 msgr2=0x55715a4f5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:58.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.176+0000 7f8851e3a640 1 --2- 172.21.15.164:0/2958419612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55715a4ed780 0x55715a4f5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:58.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.176+0000 7f8851e3a640 1 -- 172.21.15.164:0/2958419612 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55715a4f3690 con 0x55715a4eecf0 2024-09-19T08:47:58.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.176+0000 7f8851e3a640 1 --2- 172.21.15.164:0/2958419612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715a4eecf0 0x55715a504550 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f8848009b40 tx=0x7f8848005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=243cd1ec99015c55 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:58.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.177+0000 7f88377fe640 1 -- 172.21.15.164:0/2958419612 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884800ee00 con 0x55715a4eecf0 2024-09-19T08:47:58.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.177+0000 7f88377fe640 1 -- 172.21.15.164:0/2958419612 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8848004520 con 0x55715a4eecf0 2024-09-19T08:47:58.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.177+0000 7f88377fe640 1 -- 172.21.15.164:0/2958419612 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8848018470 con 0x55715a4eecf0 2024-09-19T08:47:58.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.178+0000 7f8856477740 1 -- 172.21.15.164:0/2958419612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715a4eecf0 msgr2=0x55715a504550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:58.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.178+0000 7f8856477740 1 --2- 172.21.15.164:0/2958419612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715a4eecf0 0x55715a504550 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f8848009b40 tx=0x7f8848005730 comp rx=0 tx=0).stop 2024-09-19T08:47:58.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.178+0000 7f8856477740 1 -- 172.21.15.164:0/2958419612 shutdown_connections 2024-09-19T08:47:58.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.178+0000 7f8856477740 1 --2- 172.21.15.164:0/2958419612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55715a4f8520 0x55715a4e4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:58.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.178+0000 7f8856477740 1 --2- 172.21.15.164:0/2958419612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715a4eecf0 0x55715a504550 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:58.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.178+0000 7f8856477740 1 --2- 172.21.15.164:0/2958419612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55715a4ed780 0x55715a4f5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:58.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.178+0000 7f8856477740 1 -- 172.21.15.164:0/2958419612 >> 172.21.15.164:0/2958419612 conn(0x55715a4b6310 msgr2=0x55715a4f4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:58.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.178+0000 7f8856477740 1 -- 172.21.15.164:0/2958419612 shutdown_connections 2024-09-19T08:47:58.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.178+0000 7f8856477740 1 -- 172.21.15.164:0/2958419612 wait complete. 2024-09-19T08:47:58.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.179+0000 7f8856477740 1 -- start start 2024-09-19T08:47:58.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.179+0000 7f8856477740 1 Processor -- start 2024-09-19T08:47:58.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.179+0000 7f8856477740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55715a4ed780 0x55715a500b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:58.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.179+0000 7f8856477740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55715a4eecf0 0x55715a4f2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:58.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.179+0000 7f8856477740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715a4f8520 0x55715a4f2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:58.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.179+0000 7f8856477740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f883000fa30 con 0x55715a4f8520 2024-09-19T08:47:58.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.179+0000 7f8856477740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f883000f360 con 0x55715a4eecf0 2024-09-19T08:47:58.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.179+0000 7f8856477740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f883000c5a0 con 0x55715a4ed780 2024-09-19T08:47:58.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.179+0000 7f8856477740 10 client.0 _ll_register_callbacks cb 0x7f883c01a790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:47:58.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.180+0000 7f8851639640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715a4f8520 0x55715a4f2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:58.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.180+0000 7f8851639640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715a4f8520 0x55715a4f2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47468/0 (socket says 172.21.15.164:47468) 2024-09-19T08:47:58.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.180+0000 7f8851639640 1 -- 172.21.15.164:0/3917284917 learned_addr learned my addr 172.21.15.164:0/3917284917 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:58.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.180+0000 7f8851e3a640 1 --2- 172.21.15.164:0/3917284917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55715a4eecf0 0x55715a4f2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:58.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.180+0000 7f8851639640 1 -- 172.21.15.164:0/3917284917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55715a4ed780 msgr2=0x55715a500b40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:47:58.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.180+0000 7f8851639640 1 --2- 172.21.15.164:0/3917284917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55715a4ed780 0x55715a500b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:58.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.180+0000 7f8851639640 1 -- 172.21.15.164:0/3917284917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55715a4eecf0 msgr2=0x55715a4f2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:58.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.180+0000 7f8851639640 1 --2- 172.21.15.164:0/3917284917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55715a4eecf0 0x55715a4f2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:58.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.180+0000 7f8851639640 1 -- 172.21.15.164:0/3917284917 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55715a4f2f90 con 0x55715a4f8520 2024-09-19T08:47:58.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.180+0000 7f8851e3a640 1 --2- 172.21.15.164:0/3917284917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55715a4eecf0 0x55715a4f2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:58.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.180+0000 7f8851639640 1 --2- 172.21.15.164:0/3917284917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715a4f8520 0x55715a4f2a50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f883c0508e0 tx=0x7f883c005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99267fb0f87ae4cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:58.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.180+0000 7f88357fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:47:58.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.181+0000 7f88357fa640 1 -- 172.21.15.164:0/3917284917 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c066040 con 0x55715a4f8520 2024-09-19T08:47:58.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.181+0000 7f88357fa640 1 -- 172.21.15.164:0/3917284917 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f883c050ad0 con 0x55715a4f8520 2024-09-19T08:47:58.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.181+0000 7f8856477740 1 -- 172.21.15.164:0/3917284917 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8848004520 con 0x55715a4f8520 2024-09-19T08:47:58.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.181+0000 7f88357fa640 1 -- 172.21.15.164:0/3917284917 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f883c050650 con 0x55715a4f8520 2024-09-19T08:47:58.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.181+0000 7f8856477740 10 client.15996 fetch_fsmap learned FSMap version 28 2024-09-19T08:47:58.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.181+0000 7f8856477740 1 -- 172.21.15.164:0/3917284917 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55715a572f60 con 0x55715a4f8520 2024-09-19T08:47:58.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.182+0000 7f88357fa640 1 -- 172.21.15.164:0/3917284917 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8848004520 con 0x55715a4f8520 2024-09-19T08:47:58.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.182+0000 7f88357fa640 1 -- 172.21.15.164:0/3917284917 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f883c075020 con 0x55715a4f8520 2024-09-19T08:47:58.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.182+0000 7f8856477740 10 client.15996 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:47:58.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.182+0000 7f8856477740 10 client.15996 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T08:47:58.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.182+0000 7f8856477740 20 client.15996 populate_metadata read hostname 'smithi164' 2024-09-19T08:47:58.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.182+0000 7f8856477740 1 --2- 172.21.15.164:0/3917284917 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55715a5735d0 0x55715a5759f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:58.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.183+0000 7f8856477740 4 client.15996 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:47:58.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.183+0000 7f8856477740 1 -- 172.21.15.164:0/3917284917 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55715a460860 con 0x55715a5735d0 2024-09-19T08:47:58.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.183+0000 7f8851e3a640 1 --2- 172.21.15.164:0/3917284917 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55715a5735d0 0x55715a5759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:58.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.186+0000 7f8851e3a640 1 --2- 172.21.15.164:0/3917284917 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55715a5735d0 0x55715a5759f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=78b743d9acb3ef16 server_cookie=a231041df7493e3b in_seq=0 out_seq=0 2024-09-19T08:47:58.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.186+0000 7f88357fa640 10 client.15996 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:47:58.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.186+0000 7f88357fa640 1 -- 172.21.15.164:0/3917284917 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55715a460860 con 0x55715a5735d0 2024-09-19T08:47:58.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.186+0000 7f88357fa640 10 client.15996 handle_command_reply: tid=0 2024-09-19T08:47:58.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.186+0000 7f8856477740 1 client.15996 shutdown 2024-09-19T08:47:58.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.187+0000 7f8856477740 1 -- 172.21.15.164:0/3917284917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715a4f8520 msgr2=0x55715a4f2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:58.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.187+0000 7f8856477740 1 --2- 172.21.15.164:0/3917284917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715a4f8520 0x55715a4f2a50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f883c0508e0 tx=0x7f883c005770 comp rx=0 tx=0).stop 2024-09-19T08:47:58.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.187+0000 7f8856477740 1 -- 172.21.15.164:0/3917284917 shutdown_connections 2024-09-19T08:47:58.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.187+0000 7f8856477740 1 --2- 172.21.15.164:0/3917284917 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55715a5735d0 0x55715a5759f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:58.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.187+0000 7f8856477740 1 --2- 172.21.15.164:0/3917284917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715a4f8520 0x55715a4f2a50 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:58.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.187+0000 7f8856477740 1 --2- 172.21.15.164:0/3917284917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55715a4eecf0 0x55715a4f2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:58.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.187+0000 7f88357fa640 0 client.15996 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:47:58.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.187+0000 7f8856477740 1 --2- 172.21.15.164:0/3917284917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55715a4ed780 0x55715a500b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:58.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.187+0000 7f8856477740 1 -- 172.21.15.164:0/3917284917 >> 172.21.15.164:0/3917284917 conn(0x55715a4b6310 msgr2=0x55715a4e8690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:58.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.187+0000 7f8856477740 1 -- 172.21.15.164:0/3917284917 shutdown_connections 2024-09-19T08:47:58.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.187+0000 7f8856477740 1 -- 172.21.15.164:0/3917284917 wait complete. 2024-09-19T08:47:58.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.187+0000 7f8856477740 20 client.15996 trim_cache size 0 max 16384 2024-09-19T08:47:58.342 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:47:58.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.188+0000 7f8852e3c640 1 -- 172.21.15.164:0/3684170653 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f88380a3500 msgr2=0x7f88380a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:58.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.188+0000 7f8852e3c640 1 --2- 172.21.15.164:0/3684170653 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f88380a3500 0x7f88380a59c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f884c051220 tx=0x7f884c071040 comp rx=0 tx=0).stop 2024-09-19T08:47:58.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.188+0000 7f8852e3c640 1 -- 172.21.15.164:0/3684170653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88440b2950 msgr2=0x7f88440b2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:58.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.189+0000 7f8852e3c640 1 --2- 172.21.15.164:0/3684170653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88440b2950 0x7f88440b2db0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f884c0518b0 tx=0x7f884c06ef70 comp rx=0 tx=0).stop 2024-09-19T08:47:58.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.189+0000 7f8852e3c640 1 -- 172.21.15.164:0/3684170653 shutdown_connections 2024-09-19T08:47:58.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.189+0000 7f8852e3c640 1 --2- 172.21.15.164:0/3684170653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88440b2950 0x7f88440b2db0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:58.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.189+0000 7f8852e3c640 1 --2- 172.21.15.164:0/3684170653 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f88380a3500 0x7f88380a59c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:58.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.189+0000 7f8852e3c640 1 --2- 172.21.15.164:0/3684170653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88440b7540 0x7f88440b6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:58.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.189+0000 7f8852e3c640 1 --2- 172.21.15.164:0/3684170653 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88440a4820 0x7f88440b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:58.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.189+0000 7f8852e3c640 1 -- 172.21.15.164:0/3684170653 >> 172.21.15.164:0/3684170653 conn(0x7f884401a0e0 msgr2=0x7f88440a3810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:58.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.189+0000 7f8852e3c640 1 -- 172.21.15.164:0/3684170653 shutdown_connections 2024-09-19T08:47:58.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:58.190+0000 7f8852e3c640 1 -- 172.21.15.164:0/3684170653 wait complete. 2024-09-19T08:47:58.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:58 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:47:58.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:58 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:47:58.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:58 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:47:58.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:58 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:47:58.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:58 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:47:58.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:58 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:47:58.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:58 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:47:58.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:58 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:47:58.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:58 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:47:58.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:58 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:47:58.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:58 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:47:58.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:58 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:47:58.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:58 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:47:58.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:58 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:47:58.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:58 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:47:59.234 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T08:47:59.478 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:47:59.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:59 smithi164 ceph-mon[30928]: pgmap v68: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 642 MiB/s wr, 2.67k op/s 2024-09-19T08:47:59.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:59 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:47:59.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:59 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:47:59.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:47:59 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:47:59.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:59 smithi170 ceph-mon[35072]: pgmap v68: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 642 MiB/s wr, 2.67k op/s 2024-09-19T08:47:59.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:59 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:47:59.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:59 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:47:59.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:47:59 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:47:59.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:59 smithi173 ceph-mon[35306]: pgmap v68: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 642 MiB/s wr, 2.67k op/s 2024-09-19T08:47:59.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:59 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:47:59.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:59 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:47:59.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:47:59 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:47:59.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.975+0000 7f83c9bef640 1 -- 172.21.15.164:0/3599311577 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83c4106a80 msgr2=0x7f83c4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:59.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.975+0000 7f83c9bef640 1 --2- 172.21.15.164:0/3599311577 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83c4106a80 0x7f83c4106e60 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f83b4009930 tx=0x7f83b402a420 comp rx=0 tx=0).stop 2024-09-19T08:47:59.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.975+0000 7f83c9bef640 1 -- 172.21.15.164:0/3599311577 shutdown_connections 2024-09-19T08:47:59.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.975+0000 7f83c9bef640 1 --2- 172.21.15.164:0/3599311577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83c4101420 0x7f83c410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:59.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.975+0000 7f83c9bef640 1 --2- 172.21.15.164:0/3599311577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83c4100a80 0x7f83c4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:59.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.975+0000 7f83c9bef640 1 --2- 172.21.15.164:0/3599311577 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83c4106a80 0x7f83c4106e60 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:59.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.975+0000 7f83c9bef640 1 -- 172.21.15.164:0/3599311577 >> 172.21.15.164:0/3599311577 conn(0x7f83c40fc7a0 msgr2=0x7f83c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:59.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.975+0000 7f83c9bef640 1 -- 172.21.15.164:0/3599311577 shutdown_connections 2024-09-19T08:47:59.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.975+0000 7f83c9bef640 1 -- 172.21.15.164:0/3599311577 wait complete. 2024-09-19T08:47:59.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.976+0000 7f83c9bef640 1 Processor -- start 2024-09-19T08:47:59.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.977+0000 7f83c9bef640 1 -- start start 2024-09-19T08:47:59.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.977+0000 7f83c9bef640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83c4100a80 0x7f83c41992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:59.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.977+0000 7f83c9bef640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83c4101420 0x7f83c4199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:59.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.977+0000 7f83c9bef640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83c4106a80 0x7f83c41a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:59.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.977+0000 7f83c9bef640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f83c4110e80 con 0x7f83c4100a80 2024-09-19T08:47:59.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.977+0000 7f83c9bef640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f83c4110d00 con 0x7f83c4101420 2024-09-19T08:47:59.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.977+0000 7f83c9bef640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f83c4111000 con 0x7f83c4106a80 2024-09-19T08:47:59.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.977+0000 7f83c37fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83c4100a80 0x7f83c41992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:59.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.977+0000 7f83c37fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83c4100a80 0x7f83c41992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47484/0 (socket says 172.21.15.164:47484) 2024-09-19T08:47:59.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.977+0000 7f83c37fe640 1 -- 172.21.15.164:0/3858978431 learned_addr learned my addr 172.21.15.164:0/3858978431 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:59.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.977+0000 7f83c2ffd640 1 --2- 172.21.15.164:0/3858978431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83c4101420 0x7f83c4199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:59.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.977+0000 7f83c3fff640 1 --2- 172.21.15.164:0/3858978431 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83c4106a80 0x7f83c41a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:59.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.977+0000 7f83c37fe640 1 -- 172.21.15.164:0/3858978431 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83c4106a80 msgr2=0x7f83c41a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:59.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.977+0000 7f83c37fe640 1 --2- 172.21.15.164:0/3858978431 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83c4106a80 0x7f83c41a4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:59.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.977+0000 7f83c37fe640 1 -- 172.21.15.164:0/3858978431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83c4101420 msgr2=0x7f83c4199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:59.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.977+0000 7f83c37fe640 1 --2- 172.21.15.164:0/3858978431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83c4101420 0x7f83c4199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:59.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.977+0000 7f83c37fe640 1 -- 172.21.15.164:0/3858978431 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c41a48b0 con 0x7f83c4100a80 2024-09-19T08:47:59.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.978+0000 7f83c37fe640 1 --2- 172.21.15.164:0/3858978431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83c4100a80 0x7f83c41992f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f83b4009230 tx=0x7f83b4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:59.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.978+0000 7f83c0ff9640 1 -- 172.21.15.164:0/3858978431 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b403f030 con 0x7f83c4100a80 2024-09-19T08:47:59.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.978+0000 7f83c0ff9640 1 -- 172.21.15.164:0/3858978431 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83b4003ad0 con 0x7f83c4100a80 2024-09-19T08:47:59.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.978+0000 7f83c9bef640 1 -- 172.21.15.164:0/3858978431 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83c41a4a80 con 0x7f83c4100a80 2024-09-19T08:47:59.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.978+0000 7f83c0ff9640 1 -- 172.21.15.164:0/3858978431 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b4035070 con 0x7f83c4100a80 2024-09-19T08:47:59.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.978+0000 7f83c9bef640 1 -- 172.21.15.164:0/3858978431 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83c41a4ed0 con 0x7f83c4100a80 2024-09-19T08:47:59.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.981+0000 7f83c0ff9640 1 -- 172.21.15.164:0/3858978431 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f83b4042030 con 0x7f83c4100a80 2024-09-19T08:47:59.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.981+0000 7f83cbfc8740 1 Processor -- start 2024-09-19T08:47:59.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.981+0000 7f83cbfc8740 1 -- start start 2024-09-19T08:47:59.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.981+0000 7f83cbfc8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbd829bb40 0x55cbd829a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:59.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.981+0000 7f83cbfc8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cbd829a5d0 0x55cbd829aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:59.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.981+0000 7f83cbfc8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbd82a9780 0x55cbd82abb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:59.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.981+0000 7f83cbfc8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cbd82002f0 con 0x55cbd829bb40 2024-09-19T08:47:59.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.981+0000 7f83cbfc8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55cbd81a7cf0 con 0x55cbd82a9780 2024-09-19T08:47:59.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.981+0000 7f83cbfc8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55cbd81fc2b0 con 0x55cbd829a5d0 2024-09-19T08:47:59.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.981+0000 7f83c3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbd829bb40 0x55cbd829a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:59.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.981+0000 7f83c3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbd829bb40 0x55cbd829a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47486/0 (socket says 172.21.15.164:47486) 2024-09-19T08:47:59.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.981+0000 7f83c3fff640 1 -- 172.21.15.164:0/1953502065 learned_addr learned my addr 172.21.15.164:0/1953502065 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:59.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.981+0000 7f83c0ff9640 1 --2- 172.21.15.164:0/3858978431 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f83ac0a33b0 0x7f83ac0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:59.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.981+0000 7f83c2ffd640 1 --2- 172.21.15.164:0/1953502065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbd82a9780 0x55cbd82abb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:59.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.982+0000 7f83c37fe640 1 --2- 172.21.15.164:0/1953502065 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cbd829a5d0 0x55cbd829aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:59.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.982+0000 7f83c3fff640 1 -- 172.21.15.164:0/1953502065 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cbd829a5d0 msgr2=0x55cbd829aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:59.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.982+0000 7f83c3fff640 1 --2- 172.21.15.164:0/1953502065 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cbd829a5d0 0x55cbd829aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:59.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.982+0000 7f83c3fff640 1 -- 172.21.15.164:0/1953502065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbd82a9780 msgr2=0x55cbd82abb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:59.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.982+0000 7f83c3fff640 1 --2- 172.21.15.164:0/1953502065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbd82a9780 0x55cbd82abb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:59.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.982+0000 7f83c3fff640 1 -- 172.21.15.164:0/1953502065 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbd818c3a0 con 0x55cbd829bb40 2024-09-19T08:47:59.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.982+0000 7f83c37fe640 1 --2- 172.21.15.164:0/1953502065 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cbd829a5d0 0x55cbd829aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:59.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.982+0000 7f83c0ff9640 1 -- 172.21.15.164:0/3858978431 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f83b40f0020 con 0x7f83c4100a80 2024-09-19T08:47:59.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.982+0000 7f83c3fff640 1 --2- 172.21.15.164:0/3858978431 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f83ac0a33b0 0x7f83ac0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:59.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.982+0000 7f83c3fff640 1 --2- 172.21.15.164:0/1953502065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbd829bb40 0x55cbd829a090 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f83bc010440 tx=0x7f83bc010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b08f345a513e417 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:59.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.982+0000 7f83a9ffb640 1 -- 172.21.15.164:0/1953502065 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc022070 con 0x55cbd829bb40 2024-09-19T08:47:59.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.982+0000 7f83a9ffb640 1 -- 172.21.15.164:0/1953502065 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83bc01c390 con 0x55cbd829bb40 2024-09-19T08:47:59.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.982+0000 7f83a9ffb640 1 -- 172.21.15.164:0/1953502065 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc025600 con 0x55cbd829bb40 2024-09-19T08:47:59.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.983+0000 7f83cbfc8740 1 -- 172.21.15.164:0/1953502065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbd829bb40 msgr2=0x55cbd829a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:59.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.983+0000 7f83c3fff640 1 --2- 172.21.15.164:0/3858978431 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f83ac0a33b0 0x7f83ac0a5870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f83bc01f000 tx=0x7f83bc01d000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:59.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.983+0000 7f83cbfc8740 1 --2- 172.21.15.164:0/1953502065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbd829bb40 0x55cbd829a090 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f83bc010440 tx=0x7f83bc010900 comp rx=0 tx=0).stop 2024-09-19T08:47:59.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.983+0000 7f83cbfc8740 1 -- 172.21.15.164:0/1953502065 shutdown_connections 2024-09-19T08:47:59.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.983+0000 7f83cbfc8740 1 --2- 172.21.15.164:0/1953502065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbd82a9780 0x55cbd82abb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:59.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.983+0000 7f83cbfc8740 1 --2- 172.21.15.164:0/1953502065 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cbd829a5d0 0x55cbd829aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:59.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.983+0000 7f83cbfc8740 1 --2- 172.21.15.164:0/1953502065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbd829bb40 0x55cbd829a090 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:59.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.983+0000 7f83cbfc8740 1 -- 172.21.15.164:0/1953502065 >> 172.21.15.164:0/1953502065 conn(0x55cbd8263310 msgr2=0x55cbd8296c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:59.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.983+0000 7f83cbfc8740 1 -- 172.21.15.164:0/1953502065 shutdown_connections 2024-09-19T08:47:59.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.983+0000 7f83cbfc8740 1 -- 172.21.15.164:0/1953502065 wait complete. 2024-09-19T08:47:59.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.983+0000 7f83cbfc8740 1 -- start start 2024-09-19T08:47:59.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.984+0000 7f83cbfc8740 1 Processor -- start 2024-09-19T08:47:59.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.984+0000 7f83cbfc8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cbd829a5d0 0x55cbd82a91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:59.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.984+0000 7f83cbfc8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbd829bb40 0x55cbd82a6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:59.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.984+0000 7f83cbfc8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbd82a9780 0x55cbd82a6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:59.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.984+0000 7f83cbfc8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cbd82ac480 con 0x55cbd82a9780 2024-09-19T08:47:59.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.984+0000 7f83cbfc8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55cbd81bb060 con 0x55cbd829bb40 2024-09-19T08:47:59.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.984+0000 7f83cbfc8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55cbd82ac600 con 0x55cbd829a5d0 2024-09-19T08:47:59.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.984+0000 7f83cbfc8740 10 client.0 _ll_register_callbacks cb 0x55cbd7f0e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:47:59.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.984+0000 7f83c3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbd82a9780 0x55cbd82a6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:59.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.984+0000 7f83c3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbd82a9780 0x55cbd82a6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47488/0 (socket says 172.21.15.164:47488) 2024-09-19T08:47:59.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.984+0000 7f83c3fff640 1 -- 172.21.15.164:0/1978539393 learned_addr learned my addr 172.21.15.164:0/1978539393 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:47:59.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.984+0000 7f83c37fe640 1 --2- 172.21.15.164:0/1978539393 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cbd829a5d0 0x55cbd82a91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:59.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.984+0000 7f83c2ffd640 1 --2- 172.21.15.164:0/1978539393 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbd829bb40 0x55cbd82a6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:59.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.984+0000 7f83c3fff640 1 -- 172.21.15.164:0/1978539393 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cbd829a5d0 msgr2=0x55cbd82a91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:59.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.984+0000 7f83c3fff640 1 --2- 172.21.15.164:0/1978539393 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cbd829a5d0 0x55cbd82a91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:59.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.984+0000 7f83c3fff640 1 -- 172.21.15.164:0/1978539393 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbd829bb40 msgr2=0x55cbd82a6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:59.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.984+0000 7f83c3fff640 1 --2- 172.21.15.164:0/1978539393 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbd829bb40 0x55cbd82a6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:59.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.984+0000 7f83c3fff640 1 -- 172.21.15.164:0/1978539393 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbd7f713c0 con 0x55cbd82a9780 2024-09-19T08:47:59.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.984+0000 7f83c2ffd640 1 --2- 172.21.15.164:0/1978539393 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbd829bb40 0x55cbd82a6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:47:59.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.985+0000 7f83c3fff640 1 --2- 172.21.15.164:0/1978539393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbd82a9780 0x55cbd82a6bf0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009640 tx=0x7f83bc01c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2bfa3fc5c52b318 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:59.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.985+0000 7f83a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:47:59.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.985+0000 7f83a2ffd640 1 -- 172.21.15.164:0/1978539393 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc033060 con 0x55cbd82a9780 2024-09-19T08:47:59.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.985+0000 7f83a2ffd640 1 -- 172.21.15.164:0/1978539393 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f83bc01ddd0 con 0x55cbd82a9780 2024-09-19T08:47:59.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.985+0000 7f83cbfc8740 1 -- 172.21.15.164:0/1978539393 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83bc01c390 con 0x55cbd82a9780 2024-09-19T08:47:59.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.985+0000 7f83a2ffd640 1 -- 172.21.15.164:0/1978539393 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f83bc022040 con 0x55cbd82a9780 2024-09-19T08:47:59.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.985+0000 7f83cbfc8740 10 client.16008 fetch_fsmap learned FSMap version 28 2024-09-19T08:47:59.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.985+0000 7f83cbfc8740 1 -- 172.21.15.164:0/1978539393 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cbd82b7ed0 con 0x55cbd82a9780 2024-09-19T08:47:59.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.985+0000 7f83a2ffd640 1 -- 172.21.15.164:0/1978539393 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83bc01c390 con 0x55cbd82a9780 2024-09-19T08:47:59.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.985+0000 7f83a2ffd640 1 -- 172.21.15.164:0/1978539393 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f83bc036020 con 0x55cbd82a9780 2024-09-19T08:47:59.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.985+0000 7f83cbfc8740 10 client.16008 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:47:59.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.985+0000 7f83cbfc8740 10 client.16008 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T08:47:59.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.985+0000 7f83cbfc8740 20 client.16008 populate_metadata read hostname 'smithi164' 2024-09-19T08:47:59.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.985+0000 7f83cbfc8740 1 --2- 172.21.15.164:0/1978539393 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55cbd82b9330 0x55cbd82bb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:59.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.985+0000 7f83cbfc8740 4 client.16008 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:47:59.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.985+0000 7f83cbfc8740 1 -- 172.21.15.164:0/1978539393 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbd820d860 con 0x55cbd82b9330 2024-09-19T08:47:59.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.986+0000 7f83c37fe640 1 --2- 172.21.15.164:0/1978539393 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55cbd82b9330 0x55cbd82bb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:59.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.986+0000 7f83a2ffd640 10 client.16008 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:47:59.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.986+0000 7f83c37fe640 1 --2- 172.21.15.164:0/1978539393 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55cbd82b9330 0x55cbd82bb720 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1aa22ad0aecd05e server_cookie=7a63e98101604813 in_seq=0 out_seq=0 2024-09-19T08:47:59.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.987+0000 7f83a2ffd640 1 -- 172.21.15.164:0/1978539393 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cbd820d860 con 0x55cbd82b9330 2024-09-19T08:47:59.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.987+0000 7f83a2ffd640 10 client.16008 handle_command_reply: tid=0 2024-09-19T08:47:59.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.987+0000 7f83cbfc8740 1 client.16008 shutdown 2024-09-19T08:47:59.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.987+0000 7f83cbfc8740 1 -- 172.21.15.164:0/1978539393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbd82a9780 msgr2=0x55cbd82a6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:59.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.987+0000 7f83cbfc8740 1 --2- 172.21.15.164:0/1978539393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbd82a9780 0x55cbd82a6bf0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009640 tx=0x7f83bc01c7d0 comp rx=0 tx=0).stop 2024-09-19T08:47:59.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.987+0000 7f83cbfc8740 1 -- 172.21.15.164:0/1978539393 shutdown_connections 2024-09-19T08:47:59.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.987+0000 7f83cbfc8740 1 --2- 172.21.15.164:0/1978539393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbd82a9780 0x55cbd82a6bf0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:59.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.987+0000 7f83cbfc8740 1 --2- 172.21.15.164:0/1978539393 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55cbd82b9330 0x55cbd82bb720 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:59.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.987+0000 7f83cbfc8740 1 --2- 172.21.15.164:0/1978539393 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbd829bb40 0x55cbd82a6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:59.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.987+0000 7f83cbfc8740 1 --2- 172.21.15.164:0/1978539393 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cbd829a5d0 0x55cbd82a91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:59.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.987+0000 7f83cbfc8740 1 -- 172.21.15.164:0/1978539393 >> 172.21.15.164:0/1978539393 conn(0x55cbd8263310 msgr2=0x55cbd829c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:59.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.987+0000 7f83a2ffd640 0 client.16008 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:47:59.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.987+0000 7f83cbfc8740 1 -- 172.21.15.164:0/1978539393 shutdown_connections 2024-09-19T08:47:59.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.987+0000 7f83cbfc8740 1 -- 172.21.15.164:0/1978539393 wait complete. 2024-09-19T08:47:59.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:47:59.987+0000 7f83cbfc8740 20 client.16008 trim_cache size 0 max 16384 2024-09-19T08:48:00.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.000+0000 7f83cbfc8740 1 Processor -- start 2024-09-19T08:48:00.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.000+0000 7f83cbfc8740 1 -- start start 2024-09-19T08:48:00.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.000+0000 7f83cbfc8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cbd829a5d0 0x55cbd8296e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:00.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.000+0000 7f83cbfc8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbd829bb40 0x55cbd82c0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:00.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.000+0000 7f83cbfc8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbd82a9780 0x55cbd82c1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:00.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.000+0000 7f83cbfc8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f83c41ab5b0 con 0x55cbd829bb40 2024-09-19T08:48:00.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.000+0000 7f83cbfc8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f83c41ad0f0 con 0x55cbd82a9780 2024-09-19T08:48:00.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.000+0000 7f83cbfc8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f83c41abc80 con 0x55cbd829a5d0 2024-09-19T08:48:00.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.000+0000 7f83c2ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbd829bb40 0x55cbd82c0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:00.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.001+0000 7f83c2ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbd829bb40 0x55cbd82c0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47502/0 (socket says 172.21.15.164:47502) 2024-09-19T08:48:00.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.001+0000 7f83c2ffd640 1 -- 172.21.15.164:0/2650195009 learned_addr learned my addr 172.21.15.164:0/2650195009 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:00.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.001+0000 7f83c3fff640 1 --2- 172.21.15.164:0/2650195009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbd82a9780 0x55cbd82c1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:00.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.001+0000 7f83c37fe640 1 --2- 172.21.15.164:0/2650195009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cbd829a5d0 0x55cbd8296e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:00.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.001+0000 7f83c2ffd640 1 -- 172.21.15.164:0/2650195009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cbd829a5d0 msgr2=0x55cbd8296e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:00.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.001+0000 7f83c2ffd640 1 --2- 172.21.15.164:0/2650195009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cbd829a5d0 0x55cbd8296e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:00.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.001+0000 7f83c2ffd640 1 -- 172.21.15.164:0/2650195009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbd82a9780 msgr2=0x55cbd82c1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:00.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.001+0000 7f83c2ffd640 1 --2- 172.21.15.164:0/2650195009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbd82a9780 0x55cbd82c1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:00.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.001+0000 7f83c2ffd640 1 -- 172.21.15.164:0/2650195009 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbd8297960 con 0x55cbd829bb40 2024-09-19T08:48:00.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.001+0000 7f83c37fe640 1 --2- 172.21.15.164:0/2650195009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cbd829a5d0 0x55cbd8296e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:48:00.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.001+0000 7f83c3fff640 1 --2- 172.21.15.164:0/2650195009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbd82a9780 0x55cbd82c1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:48:00.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.001+0000 7f83c2ffd640 1 --2- 172.21.15.164:0/2650195009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbd829bb40 0x55cbd82c0f10 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f83b8009c50 tx=0x7f83b8010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33001576c902d103 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:00.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.001+0000 7f83a97fa640 1 -- 172.21.15.164:0/2650195009 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b8021070 con 0x55cbd829bb40 2024-09-19T08:48:00.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.001+0000 7f83a97fa640 1 -- 172.21.15.164:0/2650195009 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83b80035f0 con 0x55cbd829bb40 2024-09-19T08:48:00.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.001+0000 7f83a97fa640 1 -- 172.21.15.164:0/2650195009 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b801c670 con 0x55cbd829bb40 2024-09-19T08:48:00.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.002+0000 7f83cbfc8740 1 -- 172.21.15.164:0/2650195009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbd829bb40 msgr2=0x55cbd82c0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:00.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.002+0000 7f83cbfc8740 1 --2- 172.21.15.164:0/2650195009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbd829bb40 0x55cbd82c0f10 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f83b8009c50 tx=0x7f83b8010800 comp rx=0 tx=0).stop 2024-09-19T08:48:00.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.002+0000 7f83cbfc8740 1 -- 172.21.15.164:0/2650195009 shutdown_connections 2024-09-19T08:48:00.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.002+0000 7f83cbfc8740 1 --2- 172.21.15.164:0/2650195009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbd82a9780 0x55cbd82c1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:00.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.002+0000 7f83cbfc8740 1 --2- 172.21.15.164:0/2650195009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbd829bb40 0x55cbd82c0f10 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:00.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.002+0000 7f83cbfc8740 1 --2- 172.21.15.164:0/2650195009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cbd829a5d0 0x55cbd8296e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:00.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.002+0000 7f83cbfc8740 1 -- 172.21.15.164:0/2650195009 >> 172.21.15.164:0/2650195009 conn(0x55cbd8263310 msgr2=0x55cbd8296700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:00.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.002+0000 7f83cbfc8740 1 -- 172.21.15.164:0/2650195009 shutdown_connections 2024-09-19T08:48:00.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.002+0000 7f83cbfc8740 1 -- 172.21.15.164:0/2650195009 wait complete. 2024-09-19T08:48:00.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.002+0000 7f83cbfc8740 1 -- start start 2024-09-19T08:48:00.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.002+0000 7f83cbfc8740 1 Processor -- start 2024-09-19T08:48:00.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.003+0000 7f83cbfc8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbd829a5d0 0x55cbd82acff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:00.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.003+0000 7f83cbfc8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cbd829bb40 0x55cbd82ad530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:00.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.003+0000 7f83cbfc8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbd82a9780 0x55cbd8321ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:00.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.003+0000 7f83cbfc8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f83c41aca20 con 0x55cbd829a5d0 2024-09-19T08:48:00.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.003+0000 7f83cbfc8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f83c41ac350 con 0x55cbd82a9780 2024-09-19T08:48:00.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.003+0000 7f83cbfc8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f83c41a95b0 con 0x55cbd829bb40 2024-09-19T08:48:00.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.003+0000 7f83cbfc8740 10 client.0 _ll_register_callbacks cb 0x7f83bc020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:48:00.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.003+0000 7f83c37fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbd829a5d0 0x55cbd82acff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:00.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.003+0000 7f83c37fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbd829a5d0 0x55cbd82acff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47512/0 (socket says 172.21.15.164:47512) 2024-09-19T08:48:00.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.003+0000 7f83c37fe640 1 -- 172.21.15.164:0/192645657 learned_addr learned my addr 172.21.15.164:0/192645657 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:00.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.003+0000 7f83c2ffd640 1 --2- 172.21.15.164:0/192645657 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cbd829bb40 0x55cbd82ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:00.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.003+0000 7f83c3fff640 1 --2- 172.21.15.164:0/192645657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbd82a9780 0x55cbd8321ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:00.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.003+0000 7f83c37fe640 1 -- 172.21.15.164:0/192645657 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cbd829bb40 msgr2=0x55cbd82ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:00.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.003+0000 7f83c37fe640 1 --2- 172.21.15.164:0/192645657 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cbd829bb40 0x55cbd82ad530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:00.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.003+0000 7f83c37fe640 1 -- 172.21.15.164:0/192645657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbd82a9780 msgr2=0x55cbd8321ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:00.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.003+0000 7f83c37fe640 1 --2- 172.21.15.164:0/192645657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbd82a9780 0x55cbd8321ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:00.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.003+0000 7f83c37fe640 1 -- 172.21.15.164:0/192645657 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbd83224c0 con 0x55cbd829a5d0 2024-09-19T08:48:00.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.003+0000 7f83c37fe640 1 --2- 172.21.15.164:0/192645657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbd829a5d0 0x55cbd82acff0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f83b40edde0 tx=0x7f83b408c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e124ec26fd834a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:00.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.003+0000 7f83a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:48:00.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.004+0000 7f83a2ffd640 1 -- 172.21.15.164:0/192645657 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b4049040 con 0x55cbd829a5d0 2024-09-19T08:48:00.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.004+0000 7f83a2ffd640 1 -- 172.21.15.164:0/192645657 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f83b40eda60 con 0x55cbd829a5d0 2024-09-19T08:48:00.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.004+0000 7f83cbfc8740 1 -- 172.21.15.164:0/192645657 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83b80035f0 con 0x55cbd829a5d0 2024-09-19T08:48:00.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.004+0000 7f83a2ffd640 1 -- 172.21.15.164:0/192645657 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f83b40c4fe0 con 0x55cbd829a5d0 2024-09-19T08:48:00.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.004+0000 7f83cbfc8740 10 client.16014 fetch_fsmap learned FSMap version 28 2024-09-19T08:48:00.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.004+0000 7f83cbfc8740 1 -- 172.21.15.164:0/192645657 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cbd8323790 con 0x55cbd829a5d0 2024-09-19T08:48:00.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.004+0000 7f83a2ffd640 1 -- 172.21.15.164:0/192645657 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83b80035f0 con 0x55cbd829a5d0 2024-09-19T08:48:00.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.004+0000 7f83a2ffd640 1 -- 172.21.15.164:0/192645657 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f83b4031bd0 con 0x55cbd829a5d0 2024-09-19T08:48:00.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.004+0000 7f83cbfc8740 10 client.16014 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:48:00.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.004+0000 7f83cbfc8740 10 client.16014 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T08:48:00.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.004+0000 7f83cbfc8740 20 client.16014 populate_metadata read hostname 'smithi164' 2024-09-19T08:48:00.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.004+0000 7f83cbfc8740 1 --2- 172.21.15.164:0/192645657 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55cbd8323f00 0x55cbd83262f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:00.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.004+0000 7f83cbfc8740 4 client.16014 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:48:00.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.004+0000 7f83cbfc8740 1 -- 172.21.15.164:0/192645657 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cbd8207d70 con 0x55cbd8323f00 2024-09-19T08:48:00.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.004+0000 7f83c2ffd640 1 --2- 172.21.15.164:0/192645657 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55cbd8323f00 0x55cbd83262f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:00.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.005+0000 7f83c2ffd640 1 --2- 172.21.15.164:0/192645657 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55cbd8323f00 0x55cbd83262f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1653b59e4a22f3df server_cookie=791059b4dca351bf in_seq=0 out_seq=0 2024-09-19T08:48:00.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.005+0000 7f83a2ffd640 10 client.16014 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:48:00.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.005+0000 7f83a2ffd640 1 -- 172.21.15.164:0/192645657 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cbd8207d70 con 0x55cbd8323f00 2024-09-19T08:48:00.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.005+0000 7f83a2ffd640 10 client.16014 handle_command_reply: tid=0 2024-09-19T08:48:00.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.005+0000 7f83cbfc8740 1 client.16014 shutdown 2024-09-19T08:48:00.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.005+0000 7f83cbfc8740 1 -- 172.21.15.164:0/192645657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbd829a5d0 msgr2=0x55cbd82acff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:00.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.005+0000 7f83cbfc8740 1 --2- 172.21.15.164:0/192645657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbd829a5d0 0x55cbd82acff0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f83b40edde0 tx=0x7f83b408c040 comp rx=0 tx=0).stop 2024-09-19T08:48:00.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.005+0000 7f83cbfc8740 1 -- 172.21.15.164:0/192645657 shutdown_connections 2024-09-19T08:48:00.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.005+0000 7f83cbfc8740 1 --2- 172.21.15.164:0/192645657 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55cbd8323f00 0x55cbd83262f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:00.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.005+0000 7f83cbfc8740 1 --2- 172.21.15.164:0/192645657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbd82a9780 0x55cbd8321ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:00.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.005+0000 7f83cbfc8740 1 --2- 172.21.15.164:0/192645657 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cbd829bb40 0x55cbd82ad530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:00.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.005+0000 7f83cbfc8740 1 --2- 172.21.15.164:0/192645657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbd829a5d0 0x55cbd82acff0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:00.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.005+0000 7f83cbfc8740 1 -- 172.21.15.164:0/192645657 >> 172.21.15.164:0/192645657 conn(0x55cbd8263310 msgr2=0x55cbd82a5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:00.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.005+0000 7f83a2ffd640 0 client.16014 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:48:00.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.005+0000 7f83cbfc8740 1 -- 172.21.15.164:0/192645657 shutdown_connections 2024-09-19T08:48:00.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.005+0000 7f83cbfc8740 1 -- 172.21.15.164:0/192645657 wait complete. 2024-09-19T08:48:00.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.005+0000 7f83cbfc8740 20 client.16014 trim_cache size 0 max 16384 2024-09-19T08:48:00.013 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:48:00.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.006+0000 7f83c9bef640 1 -- 172.21.15.164:0/3858978431 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f83ac0a33b0 msgr2=0x7f83ac0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:00.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.006+0000 7f83c9bef640 1 --2- 172.21.15.164:0/3858978431 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f83ac0a33b0 0x7f83ac0a5870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f83bc01f000 tx=0x7f83bc01d000 comp rx=0 tx=0).stop 2024-09-19T08:48:00.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.006+0000 7f83c9bef640 1 -- 172.21.15.164:0/3858978431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83c4100a80 msgr2=0x7f83c41992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:00.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.006+0000 7f83c9bef640 1 --2- 172.21.15.164:0/3858978431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83c4100a80 0x7f83c41992f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f83b4009230 tx=0x7f83b4004270 comp rx=0 tx=0).stop 2024-09-19T08:48:00.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.006+0000 7f83c9bef640 1 -- 172.21.15.164:0/3858978431 shutdown_connections 2024-09-19T08:48:00.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.006+0000 7f83c9bef640 1 --2- 172.21.15.164:0/3858978431 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83c4106a80 0x7f83c41a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:00.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.006+0000 7f83c9bef640 1 --2- 172.21.15.164:0/3858978431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83c4101420 0x7f83c4199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:00.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.006+0000 7f83c9bef640 1 --2- 172.21.15.164:0/3858978431 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f83ac0a33b0 0x7f83ac0a5870 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:00.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.006+0000 7f83c9bef640 1 --2- 172.21.15.164:0/3858978431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83c4100a80 0x7f83c41992f0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:00.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.006+0000 7f83c9bef640 1 -- 172.21.15.164:0/3858978431 >> 172.21.15.164:0/3858978431 conn(0x7f83c40fc7a0 msgr2=0x7f83c40fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:00.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.006+0000 7f83c9bef640 1 -- 172.21.15.164:0/3858978431 shutdown_connections 2024-09-19T08:48:00.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:00.006+0000 7f83c9bef640 1 -- 172.21.15.164:0/3858978431 wait complete. 2024-09-19T08:48:00.853 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T08:48:01.098 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:48:01.479 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:01 smithi164 ceph-mon[30928]: pgmap v69: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 770 MiB/s wr, 2.77k op/s 2024-09-19T08:48:01.487 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:48:01.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.563+0000 7fe542f8f640 1 -- 172.21.15.164:0/3707957252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe53c100e20 msgr2=0x7fe53c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:01.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.563+0000 7fe542f8f640 1 --2- 172.21.15.164:0/3707957252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe53c100e20 0x7fe53c101200 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe530009900 tx=0x7fe53002a480 comp rx=0 tx=0).stop 2024-09-19T08:48:01.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.563+0000 7fe542f8f640 1 -- 172.21.15.164:0/3707957252 shutdown_connections 2024-09-19T08:48:01.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.563+0000 7fe542f8f640 1 --2- 172.21.15.164:0/3707957252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe53c102ef0 0x7fe53c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:01.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.563+0000 7fe542f8f640 1 --2- 172.21.15.164:0/3707957252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe53c102570 0x7fe53c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:01.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.563+0000 7fe542f8f640 1 --2- 172.21.15.164:0/3707957252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe53c100e20 0x7fe53c101200 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:01.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.563+0000 7fe542f8f640 1 -- 172.21.15.164:0/3707957252 >> 172.21.15.164:0/3707957252 conn(0x7fe53c0fc7d0 msgr2=0x7fe53c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:01.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.563+0000 7fe542f8f640 1 -- 172.21.15.164:0/3707957252 shutdown_connections 2024-09-19T08:48:01.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.563+0000 7fe542f8f640 1 -- 172.21.15.164:0/3707957252 wait complete. 2024-09-19T08:48:01.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.564+0000 7fe542f8f640 1 Processor -- start 2024-09-19T08:48:01.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.564+0000 7fe542f8f640 1 -- start start 2024-09-19T08:48:01.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.564+0000 7fe542f8f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe53c100e20 0x7fe53c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:01.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.564+0000 7fe542f8f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe53c102570 0x7fe53c19ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:01.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.564+0000 7fe542f8f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe53c102ef0 0x7fe53c19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:01.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.564+0000 7fe542f8f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c110f80 con 0x7fe53c102570 2024-09-19T08:48:01.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.564+0000 7fe542f8f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c110e00 con 0x7fe53c100e20 2024-09-19T08:48:01.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.564+0000 7fe542f8f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c111100 con 0x7fe53c102ef0 2024-09-19T08:48:01.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.564+0000 7fe53bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe53c102570 0x7fe53c19ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:01.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.564+0000 7fe53bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe53c102570 0x7fe53c19ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47524/0 (socket says 172.21.15.164:47524) 2024-09-19T08:48:01.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.564+0000 7fe53bfff640 1 -- 172.21.15.164:0/33513951 learned_addr learned my addr 172.21.15.164:0/33513951 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:01.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.564+0000 7fe540d2b640 1 --2- 172.21.15.164:0/33513951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe53c100e20 0x7fe53c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:01.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.564+0000 7fe54152c640 1 --2- 172.21.15.164:0/33513951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe53c102ef0 0x7fe53c19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:01.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.564+0000 7fe53bfff640 1 -- 172.21.15.164:0/33513951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe53c102ef0 msgr2=0x7fe53c19efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:01.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.564+0000 7fe53bfff640 1 --2- 172.21.15.164:0/33513951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe53c102ef0 0x7fe53c19efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:01.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.564+0000 7fe53bfff640 1 -- 172.21.15.164:0/33513951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe53c100e20 msgr2=0x7fe53c19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:01.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.564+0000 7fe53bfff640 1 --2- 172.21.15.164:0/33513951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe53c100e20 0x7fe53c19a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:01.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.564+0000 7fe53bfff640 1 -- 172.21.15.164:0/33513951 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe53c19f6b0 con 0x7fe53c102570 2024-09-19T08:48:01.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.565+0000 7fe53bfff640 1 --2- 172.21.15.164:0/33513951 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe53c102570 0x7fe53c19ac40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe52800d950 tx=0x7fe52800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:01.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.565+0000 7fe539ffb640 1 -- 172.21.15.164:0/33513951 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528017070 con 0x7fe53c102570 2024-09-19T08:48:01.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.565+0000 7fe542f8f640 1 -- 172.21.15.164:0/33513951 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe53c19f9a0 con 0x7fe53c102570 2024-09-19T08:48:01.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.565+0000 7fe539ffb640 1 -- 172.21.15.164:0/33513951 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe528009c30 con 0x7fe53c102570 2024-09-19T08:48:01.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.565+0000 7fe539ffb640 1 -- 172.21.15.164:0/33513951 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52800f6b0 con 0x7fe53c102570 2024-09-19T08:48:01.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.565+0000 7fe542f8f640 1 -- 172.21.15.164:0/33513951 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe53c1a7280 con 0x7fe53c102570 2024-09-19T08:48:01.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.568+0000 7fe539ffb640 1 -- 172.21.15.164:0/33513951 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fe5280109d0 con 0x7fe53c102570 2024-09-19T08:48:01.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.568+0000 7fe545368740 1 Processor -- start 2024-09-19T08:48:01.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.568+0000 7fe545368740 1 -- start start 2024-09-19T08:48:01.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.569+0000 7fe545368740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556545dd1b40 0x556545dd0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:01.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.569+0000 7fe545368740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556545dd05d0 0x556545dd0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:01.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.569+0000 7fe545368740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556545ddb370 0x556545ddd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:01.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.569+0000 7fe545368740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556545d362f0 con 0x556545ddb370 2024-09-19T08:48:01.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.569+0000 7fe545368740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556545cddcf0 con 0x556545dd05d0 2024-09-19T08:48:01.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.569+0000 7fe545368740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556545d322b0 con 0x556545dd1b40 2024-09-19T08:48:01.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.569+0000 7fe539ffb640 1 --2- 172.21.15.164:0/33513951 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe52c0a3430 0x7fe52c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:01.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.569+0000 7fe54152c640 1 --2- 172.21.15.164:0/33513951 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe52c0a3430 0x7fe52c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:01.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.569+0000 7fe53bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556545ddb370 0x556545ddd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:01.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.569+0000 7fe539ffb640 1 -- 172.21.15.164:0/33513951 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fe5280cf800 con 0x7fe53c102570 2024-09-19T08:48:01.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.569+0000 7fe53bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556545ddb370 0x556545ddd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47528/0 (socket says 172.21.15.164:47528) 2024-09-19T08:48:01.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.569+0000 7fe53bfff640 1 -- 172.21.15.164:0/2306965606 learned_addr learned my addr 172.21.15.164:0/2306965606 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:01.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.569+0000 7fe540d2b640 1 --2- 172.21.15.164:0/2306965606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556545dd05d0 0x556545dd0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:01.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.569+0000 7fe54152c640 1 --2- 172.21.15.164:0/2306965606 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556545dd1b40 0x556545dd0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:01.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.569+0000 7fe53bfff640 1 -- 172.21.15.164:0/2306965606 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556545dd1b40 msgr2=0x556545dd0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:01.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.569+0000 7fe53bfff640 1 --2- 172.21.15.164:0/2306965606 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556545dd1b40 0x556545dd0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:01.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.569+0000 7fe53bfff640 1 -- 172.21.15.164:0/2306965606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556545dd05d0 msgr2=0x556545dd0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:01.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.569+0000 7fe53bfff640 1 --2- 172.21.15.164:0/2306965606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556545dd05d0 0x556545dd0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:01.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.569+0000 7fe53bfff640 1 -- 172.21.15.164:0/2306965606 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556545cc23a0 con 0x556545ddb370 2024-09-19T08:48:01.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.569+0000 7fe540d2b640 1 --2- 172.21.15.164:0/2306965606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556545dd05d0 0x556545dd0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:48:01.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.569+0000 7fe53bfff640 1 --2- 172.21.15.164:0/2306965606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556545ddb370 0x556545ddd760 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe5280cd020 tx=0x7fe528010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9cfb1c69e055660 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:01.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.570+0000 7fe54152c640 1 --2- 172.21.15.164:0/33513951 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe52c0a3430 0x7fe52c0a58f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe534005fd0 tx=0x7fe534009de0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:01.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.570+0000 7fe526ffd640 1 -- 172.21.15.164:0/2306965606 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528020070 con 0x556545ddb370 2024-09-19T08:48:01.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.570+0000 7fe526ffd640 1 -- 172.21.15.164:0/2306965606 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5280a47c0 con 0x556545ddb370 2024-09-19T08:48:01.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.570+0000 7fe526ffd640 1 -- 172.21.15.164:0/2306965606 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52809b2b0 con 0x556545ddb370 2024-09-19T08:48:01.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.570+0000 7fe545368740 1 -- 172.21.15.164:0/2306965606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556545ddb370 msgr2=0x556545ddd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:01.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.570+0000 7fe545368740 1 --2- 172.21.15.164:0/2306965606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556545ddb370 0x556545ddd760 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe5280cd020 tx=0x7fe528010d70 comp rx=0 tx=0).stop 2024-09-19T08:48:01.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.570+0000 7fe545368740 1 -- 172.21.15.164:0/2306965606 shutdown_connections 2024-09-19T08:48:01.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.570+0000 7fe545368740 1 --2- 172.21.15.164:0/2306965606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556545ddb370 0x556545ddd760 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:01.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.570+0000 7fe545368740 1 --2- 172.21.15.164:0/2306965606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556545dd05d0 0x556545dd0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:01.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.570+0000 7fe545368740 1 --2- 172.21.15.164:0/2306965606 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556545dd1b40 0x556545dd0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:01.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.570+0000 7fe545368740 1 -- 172.21.15.164:0/2306965606 >> 172.21.15.164:0/2306965606 conn(0x556545d99310 msgr2=0x556545dccc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:01.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.570+0000 7fe545368740 1 -- 172.21.15.164:0/2306965606 shutdown_connections 2024-09-19T08:48:01.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.571+0000 7fe545368740 1 -- 172.21.15.164:0/2306965606 wait complete. 2024-09-19T08:48:01.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.571+0000 7fe545368740 1 -- start start 2024-09-19T08:48:01.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.571+0000 7fe545368740 1 Processor -- start 2024-09-19T08:48:01.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.571+0000 7fe545368740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556545dd05d0 0x556545ddae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:01.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.571+0000 7fe545368740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556545dd1b40 0x556545dd80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:01.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.571+0000 7fe545368740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556545ddb370 0x556545dd8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:01.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.571+0000 7fe545368740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556545dddfb0 con 0x556545dd05d0 2024-09-19T08:48:01.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.571+0000 7fe545368740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556545cf1060 con 0x556545ddb370 2024-09-19T08:48:01.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.571+0000 7fe545368740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556545dde130 con 0x556545dd1b40 2024-09-19T08:48:01.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.572+0000 7fe545368740 10 client.0 _ll_register_callbacks cb 0x556545a448c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:48:01.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.572+0000 7fe540d2b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556545dd05d0 0x556545ddae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:01.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.572+0000 7fe540d2b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556545dd05d0 0x556545ddae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47538/0 (socket says 172.21.15.164:47538) 2024-09-19T08:48:01.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.572+0000 7fe540d2b640 1 -- 172.21.15.164:0/3180118667 learned_addr learned my addr 172.21.15.164:0/3180118667 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:01.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.572+0000 7fe53bfff640 1 --2- 172.21.15.164:0/3180118667 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556545dd1b40 0x556545dd80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:01.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.572+0000 7fe54152c640 1 --2- 172.21.15.164:0/3180118667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556545ddb370 0x556545dd8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:01.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.572+0000 7fe540d2b640 1 -- 172.21.15.164:0/3180118667 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556545dd1b40 msgr2=0x556545dd80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:01.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.572+0000 7fe540d2b640 1 --2- 172.21.15.164:0/3180118667 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556545dd1b40 0x556545dd80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:01.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.572+0000 7fe540d2b640 1 -- 172.21.15.164:0/3180118667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556545ddb370 msgr2=0x556545dd8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:01.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.572+0000 7fe540d2b640 1 --2- 172.21.15.164:0/3180118667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556545ddb370 0x556545dd8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:01.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.572+0000 7fe540d2b640 1 -- 172.21.15.164:0/3180118667 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556545aa73c0 con 0x556545dd05d0 2024-09-19T08:48:01.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.572+0000 7fe54152c640 1 --2- 172.21.15.164:0/3180118667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556545ddb370 0x556545dd8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:48:01.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.572+0000 7fe540d2b640 1 --2- 172.21.15.164:0/3180118667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556545dd05d0 0x556545ddae00 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe5300314e0 tx=0x7fe530004420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a31be7026fabd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:01.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.572+0000 7fe51bfff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:48:01.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.572+0000 7fe51bfff640 1 -- 172.21.15.164:0/3180118667 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530035040 con 0x556545dd05d0 2024-09-19T08:48:01.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.572+0000 7fe51bfff640 1 -- 172.21.15.164:0/3180118667 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fe530003ac0 con 0x556545dd05d0 2024-09-19T08:48:01.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.572+0000 7fe545368740 1 -- 172.21.15.164:0/3180118667 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5280a47c0 con 0x556545dd05d0 2024-09-19T08:48:01.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.573+0000 7fe51bfff640 1 -- 172.21.15.164:0/3180118667 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe53004e050 con 0x556545dd05d0 2024-09-19T08:48:01.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.573+0000 7fe545368740 10 client.16026 fetch_fsmap learned FSMap version 28 2024-09-19T08:48:01.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.573+0000 7fe545368740 1 -- 172.21.15.164:0/3180118667 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556545de9b50 con 0x556545dd05d0 2024-09-19T08:48:01.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.573+0000 7fe51bfff640 1 -- 172.21.15.164:0/3180118667 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5280a47c0 con 0x556545dd05d0 2024-09-19T08:48:01.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.573+0000 7fe51bfff640 1 -- 172.21.15.164:0/3180118667 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe53003c440 con 0x556545dd05d0 2024-09-19T08:48:01.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.573+0000 7fe545368740 10 client.16026 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:48:01.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.573+0000 7fe545368740 10 client.16026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:48:01.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.573+0000 7fe545368740 20 client.16026 populate_metadata read hostname 'smithi164' 2024-09-19T08:48:01.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.573+0000 7fe545368740 1 --2- 172.21.15.164:0/3180118667 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556545deb010 0x556545ded460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:01.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.573+0000 7fe545368740 4 client.16026 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:48:01.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.573+0000 7fe545368740 1 -- 172.21.15.164:0/3180118667 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556545d43860 con 0x556545deb010 2024-09-19T08:48:01.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.574+0000 7fe53bfff640 1 --2- 172.21.15.164:0/3180118667 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556545deb010 0x556545ded460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:01.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.574+0000 7fe51bfff640 10 client.16026 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:48:01.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.574+0000 7fe53bfff640 1 --2- 172.21.15.164:0/3180118667 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556545deb010 0x556545ded460 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a5cb832510fd543 server_cookie=86788cd80eed93ea in_seq=0 out_seq=0 2024-09-19T08:48:01.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.575+0000 7fe51bfff640 1 -- 172.21.15.164:0/3180118667 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556545d43860 con 0x556545deb010 2024-09-19T08:48:01.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.575+0000 7fe51bfff640 10 client.16026 handle_command_reply: tid=0 2024-09-19T08:48:01.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.575+0000 7fe545368740 1 client.16026 shutdown 2024-09-19T08:48:01.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.575+0000 7fe545368740 1 -- 172.21.15.164:0/3180118667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556545dd05d0 msgr2=0x556545ddae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:01.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.575+0000 7fe545368740 1 --2- 172.21.15.164:0/3180118667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556545dd05d0 0x556545ddae00 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe5300314e0 tx=0x7fe530004420 comp rx=0 tx=0).stop 2024-09-19T08:48:01.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.575+0000 7fe545368740 1 -- 172.21.15.164:0/3180118667 shutdown_connections 2024-09-19T08:48:01.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.575+0000 7fe545368740 1 --2- 172.21.15.164:0/3180118667 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556545deb010 0x556545ded460 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:01.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.575+0000 7fe545368740 1 --2- 172.21.15.164:0/3180118667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556545ddb370 0x556545dd8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:01.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.575+0000 7fe545368740 1 --2- 172.21.15.164:0/3180118667 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556545dd1b40 0x556545dd80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:01.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.575+0000 7fe545368740 1 --2- 172.21.15.164:0/3180118667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556545dd05d0 0x556545ddae00 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:01.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.575+0000 7fe545368740 1 -- 172.21.15.164:0/3180118667 >> 172.21.15.164:0/3180118667 conn(0x556545d99310 msgr2=0x556545dd2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:01.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.575+0000 7fe51bfff640 0 client.16026 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:48:01.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.575+0000 7fe545368740 1 -- 172.21.15.164:0/3180118667 shutdown_connections 2024-09-19T08:48:01.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.575+0000 7fe545368740 1 -- 172.21.15.164:0/3180118667 wait complete. 2024-09-19T08:48:01.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.575+0000 7fe545368740 20 client.16026 trim_cache size 0 max 16384 2024-09-19T08:48:01.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.588+0000 7fe545368740 1 Processor -- start 2024-09-19T08:48:01.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.588+0000 7fe545368740 1 -- start start 2024-09-19T08:48:01.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.588+0000 7fe545368740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556545dd05d0 0x556545ddc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:01.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.588+0000 7fe545368740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556545dd1b40 0x556545ddcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:01.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.588+0000 7fe545368740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556545ddb370 0x556545df2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:01.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.588+0000 7fe545368740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c1af7c0 con 0x556545dd1b40 2024-09-19T08:48:01.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.588+0000 7fe545368740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c1b1300 con 0x556545dd05d0 2024-09-19T08:48:01.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.588+0000 7fe545368740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c1afe90 con 0x556545ddb370 2024-09-19T08:48:01.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.588+0000 7fe53bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556545dd1b40 0x556545ddcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:01.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.588+0000 7fe53bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556545dd1b40 0x556545ddcca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47540/0 (socket says 172.21.15.164:47540) 2024-09-19T08:48:01.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.588+0000 7fe53bfff640 1 -- 172.21.15.164:0/3571039293 learned_addr learned my addr 172.21.15.164:0/3571039293 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:01.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.589+0000 7fe540d2b640 1 --2- 172.21.15.164:0/3571039293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556545dd05d0 0x556545ddc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:01.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.589+0000 7fe54152c640 1 --2- 172.21.15.164:0/3571039293 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556545ddb370 0x556545df2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:01.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.589+0000 7fe53bfff640 1 -- 172.21.15.164:0/3571039293 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556545ddb370 msgr2=0x556545df2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:01.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.589+0000 7fe53bfff640 1 --2- 172.21.15.164:0/3571039293 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556545ddb370 0x556545df2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:01.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.589+0000 7fe53bfff640 1 -- 172.21.15.164:0/3571039293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556545dd05d0 msgr2=0x556545ddc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:01.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.589+0000 7fe53bfff640 1 --2- 172.21.15.164:0/3571039293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556545dd05d0 0x556545ddc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:01.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.589+0000 7fe53bfff640 1 -- 172.21.15.164:0/3571039293 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556545dddc60 con 0x556545dd1b40 2024-09-19T08:48:01.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.589+0000 7fe540d2b640 1 --2- 172.21.15.164:0/3571039293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556545dd05d0 0x556545ddc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:48:01.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.589+0000 7fe53bfff640 1 --2- 172.21.15.164:0/3571039293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556545dd1b40 0x556545ddcca0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe5280cd070 tx=0x7fe528086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8cf317ad238b741 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:01.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.589+0000 7fe5267fc640 1 -- 172.21.15.164:0/3571039293 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5280b9070 con 0x556545dd1b40 2024-09-19T08:48:01.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.589+0000 7fe5267fc640 1 -- 172.21.15.164:0/3571039293 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe528098270 con 0x556545dd1b40 2024-09-19T08:48:01.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.589+0000 7fe5267fc640 1 -- 172.21.15.164:0/3571039293 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528020050 con 0x556545dd1b40 2024-09-19T08:48:01.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.590+0000 7fe545368740 1 -- 172.21.15.164:0/3571039293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556545dd1b40 msgr2=0x556545ddcca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:01.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.590+0000 7fe545368740 1 --2- 172.21.15.164:0/3571039293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556545dd1b40 0x556545ddcca0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe5280cd070 tx=0x7fe528086040 comp rx=0 tx=0).stop 2024-09-19T08:48:01.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.590+0000 7fe545368740 1 -- 172.21.15.164:0/3571039293 shutdown_connections 2024-09-19T08:48:01.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.590+0000 7fe545368740 1 --2- 172.21.15.164:0/3571039293 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556545ddb370 0x556545df2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:01.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.590+0000 7fe545368740 1 --2- 172.21.15.164:0/3571039293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556545dd1b40 0x556545ddcca0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:01.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.590+0000 7fe545368740 1 --2- 172.21.15.164:0/3571039293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556545dd05d0 0x556545ddc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:01.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.590+0000 7fe545368740 1 -- 172.21.15.164:0/3571039293 >> 172.21.15.164:0/3571039293 conn(0x556545d99310 msgr2=0x556545dd3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:01.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.590+0000 7fe545368740 1 -- 172.21.15.164:0/3571039293 shutdown_connections 2024-09-19T08:48:01.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.590+0000 7fe545368740 1 -- 172.21.15.164:0/3571039293 wait complete. 2024-09-19T08:48:01.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.590+0000 7fe545368740 1 -- start start 2024-09-19T08:48:01.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.590+0000 7fe545368740 1 Processor -- start 2024-09-19T08:48:01.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.591+0000 7fe545368740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556545dd05d0 0x556545de3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:01.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.591+0000 7fe545368740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556545ddb370 0x556545ddbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:01.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.591+0000 7fe545368740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556545ddc520 0x556545ddc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:01.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.591+0000 7fe545368740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c1b0c30 con 0x556545ddb370 2024-09-19T08:48:01.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.591+0000 7fe545368740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c1b0560 con 0x556545ddc520 2024-09-19T08:48:01.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.591+0000 7fe545368740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c1ad7a0 con 0x556545dd05d0 2024-09-19T08:48:01.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.591+0000 7fe545368740 10 client.0 _ll_register_callbacks cb 0x7fe530009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:48:01.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.591+0000 7fe53bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556545ddb370 0x556545ddbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:01.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.591+0000 7fe540d2b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556545dd05d0 0x556545de3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:01.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.591+0000 7fe53bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556545ddb370 0x556545ddbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47544/0 (socket says 172.21.15.164:47544) 2024-09-19T08:48:01.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.591+0000 7fe53bfff640 1 -- 172.21.15.164:0/1006533131 learned_addr learned my addr 172.21.15.164:0/1006533131 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:01.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.591+0000 7fe54152c640 1 --2- 172.21.15.164:0/1006533131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556545ddc520 0x556545ddc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:01.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.591+0000 7fe53bfff640 1 -- 172.21.15.164:0/1006533131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556545dd05d0 msgr2=0x556545de3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:01.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.591+0000 7fe53bfff640 1 --2- 172.21.15.164:0/1006533131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556545dd05d0 0x556545de3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:01.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.591+0000 7fe53bfff640 1 -- 172.21.15.164:0/1006533131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556545ddc520 msgr2=0x556545ddc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:01.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.591+0000 7fe53bfff640 1 --2- 172.21.15.164:0/1006533131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556545ddc520 0x556545ddc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:01.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.591+0000 7fe53bfff640 1 -- 172.21.15.164:0/1006533131 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556545de1710 con 0x556545ddb370 2024-09-19T08:48:01.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.591+0000 7fe540d2b640 1 --2- 172.21.15.164:0/1006533131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556545dd05d0 0x556545de3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:48:01.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.591+0000 7fe54152c640 1 --2- 172.21.15.164:0/1006533131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556545ddc520 0x556545ddc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:48:01.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.592+0000 7fe53bfff640 1 --2- 172.21.15.164:0/1006533131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556545ddb370 0x556545ddbfe0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe5280a00c0 tx=0x7fe528005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e12531d51d5e897 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:01.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.592+0000 7fe51bfff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:48:01.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.592+0000 7fe51bfff640 1 -- 172.21.15.164:0/1006533131 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52807f060 con 0x556545ddb370 2024-09-19T08:48:01.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.592+0000 7fe51bfff640 1 -- 172.21.15.164:0/1006533131 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fe5280ba470 con 0x556545ddb370 2024-09-19T08:48:01.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.592+0000 7fe545368740 1 -- 172.21.15.164:0/1006533131 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe528098270 con 0x556545ddb370 2024-09-19T08:48:01.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.592+0000 7fe51bfff640 1 -- 172.21.15.164:0/1006533131 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe528020020 con 0x556545ddb370 2024-09-19T08:48:01.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.592+0000 7fe545368740 10 client.16032 fetch_fsmap learned FSMap version 28 2024-09-19T08:48:01.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.592+0000 7fe545368740 1 -- 172.21.15.164:0/1006533131 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556545e56100 con 0x556545ddb370 2024-09-19T08:48:01.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.592+0000 7fe51bfff640 1 -- 172.21.15.164:0/1006533131 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe528098270 con 0x556545ddb370 2024-09-19T08:48:01.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.592+0000 7fe51bfff640 1 -- 172.21.15.164:0/1006533131 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe528082030 con 0x556545ddb370 2024-09-19T08:48:01.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.592+0000 7fe545368740 10 client.16032 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:48:01.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.592+0000 7fe545368740 10 client.16032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:48:01.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.592+0000 7fe545368740 20 client.16032 populate_metadata read hostname 'smithi164' 2024-09-19T08:48:01.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.592+0000 7fe545368740 1 --2- 172.21.15.164:0/1006533131 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556545e56320 0x556545e58710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:01.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.592+0000 7fe545368740 4 client.16032 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:48:01.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.592+0000 7fe545368740 1 -- 172.21.15.164:0/1006533131 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556545d3dd70 con 0x556545e56320 2024-09-19T08:48:01.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.593+0000 7fe540d2b640 1 --2- 172.21.15.164:0/1006533131 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556545e56320 0x556545e58710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:01.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.593+0000 7fe540d2b640 1 --2- 172.21.15.164:0/1006533131 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556545e56320 0x556545e58710 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bbba3e4e91ce37a server_cookie=e156bf9d072473a9 in_seq=0 out_seq=0 2024-09-19T08:48:01.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.593+0000 7fe51bfff640 10 client.16032 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:48:01.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.593+0000 7fe51bfff640 1 -- 172.21.15.164:0/1006533131 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556545d3dd70 con 0x556545e56320 2024-09-19T08:48:01.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.593+0000 7fe51bfff640 10 client.16032 handle_command_reply: tid=0 2024-09-19T08:48:01.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.593+0000 7fe545368740 1 client.16032 shutdown 2024-09-19T08:48:01.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.594+0000 7fe545368740 1 -- 172.21.15.164:0/1006533131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556545ddb370 msgr2=0x556545ddbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:01.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.594+0000 7fe545368740 1 --2- 172.21.15.164:0/1006533131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556545ddb370 0x556545ddbfe0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe5280a00c0 tx=0x7fe528005880 comp rx=0 tx=0).stop 2024-09-19T08:48:01.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.594+0000 7fe545368740 1 -- 172.21.15.164:0/1006533131 shutdown_connections 2024-09-19T08:48:01.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.594+0000 7fe545368740 1 --2- 172.21.15.164:0/1006533131 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556545e56320 0x556545e58710 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:01.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.594+0000 7fe545368740 1 --2- 172.21.15.164:0/1006533131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556545ddc520 0x556545ddc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:01.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.594+0000 7fe545368740 1 --2- 172.21.15.164:0/1006533131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556545ddb370 0x556545ddbfe0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:01.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.594+0000 7fe545368740 1 --2- 172.21.15.164:0/1006533131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556545dd05d0 0x556545de3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:01.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.594+0000 7fe545368740 1 -- 172.21.15.164:0/1006533131 >> 172.21.15.164:0/1006533131 conn(0x556545d99310 msgr2=0x556545dd6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:01.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.594+0000 7fe51bfff640 0 client.16032 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:48:01.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.594+0000 7fe545368740 1 -- 172.21.15.164:0/1006533131 shutdown_connections 2024-09-19T08:48:01.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.594+0000 7fe545368740 1 -- 172.21.15.164:0/1006533131 wait complete. 2024-09-19T08:48:01.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.594+0000 7fe545368740 20 client.16032 trim_cache size 0 max 16384 2024-09-19T08:48:01.603 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:48:01.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.594+0000 7fe542f8f640 1 -- 172.21.15.164:0/33513951 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe52c0a3430 msgr2=0x7fe52c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:01.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.594+0000 7fe542f8f640 1 --2- 172.21.15.164:0/33513951 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe52c0a3430 0x7fe52c0a58f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe534005fd0 tx=0x7fe534009de0 comp rx=0 tx=0).stop 2024-09-19T08:48:01.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.594+0000 7fe542f8f640 1 -- 172.21.15.164:0/33513951 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe53c102570 msgr2=0x7fe53c19ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:01.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.594+0000 7fe542f8f640 1 --2- 172.21.15.164:0/33513951 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe53c102570 0x7fe53c19ac40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe52800d950 tx=0x7fe52800de10 comp rx=0 tx=0).stop 2024-09-19T08:48:01.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.595+0000 7fe542f8f640 1 -- 172.21.15.164:0/33513951 shutdown_connections 2024-09-19T08:48:01.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.595+0000 7fe542f8f640 1 --2- 172.21.15.164:0/33513951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe53c102ef0 0x7fe53c19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:01.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.595+0000 7fe542f8f640 1 --2- 172.21.15.164:0/33513951 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe52c0a3430 0x7fe52c0a58f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:01.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.595+0000 7fe542f8f640 1 --2- 172.21.15.164:0/33513951 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe53c102570 0x7fe53c19ac40 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:01.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.595+0000 7fe542f8f640 1 --2- 172.21.15.164:0/33513951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe53c100e20 0x7fe53c19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:01.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.595+0000 7fe542f8f640 1 -- 172.21.15.164:0/33513951 >> 172.21.15.164:0/33513951 conn(0x7fe53c0fc7d0 msgr2=0x7fe53c0ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:01.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.595+0000 7fe542f8f640 1 -- 172.21.15.164:0/33513951 shutdown_connections 2024-09-19T08:48:01.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:01.595+0000 7fe542f8f640 1 -- 172.21.15.164:0/33513951 wait complete. 2024-09-19T08:48:01.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:01 smithi170 ceph-mon[35072]: pgmap v69: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 770 MiB/s wr, 2.77k op/s 2024-09-19T08:48:01.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:01 smithi173 ceph-mon[35306]: pgmap v69: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 770 MiB/s wr, 2.77k op/s 2024-09-19T08:48:02.581 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:48:03.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.083+0000 7f9904979640 1 -- 172.21.15.164:0/1913893629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9900100ca0 msgr2=0x7f9900106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:03.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.083+0000 7f9904979640 1 --2- 172.21.15.164:0/1913893629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9900100ca0 0x7f9900106840 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009930 tx=0x7f98f402a420 comp rx=0 tx=0).stop 2024-09-19T08:48:03.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.083+0000 7f9904979640 1 -- 172.21.15.164:0/1913893629 shutdown_connections 2024-09-19T08:48:03.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.083+0000 7f9904979640 1 --2- 172.21.15.164:0/1913893629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9900106f50 0x7f990010eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:03.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.083+0000 7f9904979640 1 --2- 172.21.15.164:0/1913893629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9900100ca0 0x7f9900106840 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:03.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.083+0000 7f9904979640 1 --2- 172.21.15.164:0/1913893629 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99001002f0 0x7f99001006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:03.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.083+0000 7f9904979640 1 -- 172.21.15.164:0/1913893629 >> 172.21.15.164:0/1913893629 conn(0x7f9900077210 msgr2=0x7f99000fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:03.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.084+0000 7f9904979640 1 -- 172.21.15.164:0/1913893629 shutdown_connections 2024-09-19T08:48:03.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.084+0000 7f9904979640 1 -- 172.21.15.164:0/1913893629 wait complete. 2024-09-19T08:48:03.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.084+0000 7f9904979640 1 Processor -- start 2024-09-19T08:48:03.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.084+0000 7f9904979640 1 -- start start 2024-09-19T08:48:03.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.085+0000 7f9904979640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99001002f0 0x7f990019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:03.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.085+0000 7f9904979640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9900100ca0 0x7f990019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:03.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.085+0000 7f9904979640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9900106f50 0x7f990019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:03.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.085+0000 7f9904979640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9900111f60 con 0x7f99001002f0 2024-09-19T08:48:03.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.085+0000 7f9904979640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9900111de0 con 0x7f9900100ca0 2024-09-19T08:48:03.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.085+0000 7f9904979640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f99001120e0 con 0x7f9900106f50 2024-09-19T08:48:03.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.085+0000 7f98fe59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99001002f0 0x7f990019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:03.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.085+0000 7f98fe59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99001002f0 0x7f990019a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47562/0 (socket says 172.21.15.164:47562) 2024-09-19T08:48:03.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.085+0000 7f98fe59c640 1 -- 172.21.15.164:0/2700486514 learned_addr learned my addr 172.21.15.164:0/2700486514 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:03.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.085+0000 7f98fed9d640 1 --2- 172.21.15.164:0/2700486514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9900106f50 0x7f990019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:03.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.085+0000 7f98fdd9b640 1 --2- 172.21.15.164:0/2700486514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9900100ca0 0x7f990019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:03.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.085+0000 7f98fe59c640 1 -- 172.21.15.164:0/2700486514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9900106f50 msgr2=0x7f990019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:03.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.085+0000 7f98fe59c640 1 --2- 172.21.15.164:0/2700486514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9900106f50 0x7f990019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:03.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.085+0000 7f98fe59c640 1 -- 172.21.15.164:0/2700486514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9900100ca0 msgr2=0x7f990019abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:03.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.085+0000 7f98fe59c640 1 --2- 172.21.15.164:0/2700486514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9900100ca0 0x7f990019abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:03.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.085+0000 7f98fe59c640 1 -- 172.21.15.164:0/2700486514 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f990019f630 con 0x7f99001002f0 2024-09-19T08:48:03.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.085+0000 7f98fe59c640 1 --2- 172.21.15.164:0/2700486514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99001002f0 0x7f990019a680 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f98f000cce0 tx=0x7f98f000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:03.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.085+0000 7f98ef7fe640 1 -- 172.21.15.164:0/2700486514 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f000f650 con 0x7f99001002f0 2024-09-19T08:48:03.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.085+0000 7f9904979640 1 -- 172.21.15.164:0/2700486514 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f990019f920 con 0x7f99001002f0 2024-09-19T08:48:03.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.085+0000 7f98ef7fe640 1 -- 172.21.15.164:0/2700486514 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98f000fcd0 con 0x7f99001002f0 2024-09-19T08:48:03.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.085+0000 7f9904979640 1 -- 172.21.15.164:0/2700486514 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99001a7200 con 0x7f99001002f0 2024-09-19T08:48:03.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.085+0000 7f98ef7fe640 1 -- 172.21.15.164:0/2700486514 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f0018c70 con 0x7f99001002f0 2024-09-19T08:48:03.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.087+0000 7f9904979640 1 -- 172.21.15.164:0/2700486514 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98e0005290 con 0x7f99001002f0 2024-09-19T08:48:03.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.088+0000 7f98ef7fe640 1 -- 172.21.15.164:0/2700486514 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f98f0018400 con 0x7f99001002f0 2024-09-19T08:48:03.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.088+0000 7f98ef7fe640 1 --2- 172.21.15.164:0/2700486514 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f98e80a3430 0x7f98e80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:03.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.088+0000 7f98fdd9b640 1 --2- 172.21.15.164:0/2700486514 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f98e80a3430 0x7f98e80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:03.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.088+0000 7f98ef7fe640 1 -- 172.21.15.164:0/2700486514 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f98f00d0000 con 0x7f99001002f0 2024-09-19T08:48:03.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.089+0000 7f98fdd9b640 1 --2- 172.21.15.164:0/2700486514 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f98e80a3430 0x7f98e80a58f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f98f4004770 tx=0x7f98f40058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:03.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.093+0000 7f98ef7fe640 1 -- 172.21.15.164:0/2700486514 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f98f0014030 con 0x7f99001002f0 2024-09-19T08:48:03.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:03 smithi164 ceph-mon[30928]: pgmap v70: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 439 MiB/s wr, 1.67k op/s 2024-09-19T08:48:03.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.347+0000 7f9904979640 1 -- 172.21.15.164:0/2700486514 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98e0005850 con 0x7f99001002f0 2024-09-19T08:48:03.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.349+0000 7f98ef7fe640 1 -- 172.21.15.164:0/2700486514 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11984 (secure 0 0 0) 0x7f98f009ca50 con 0x7f99001002f0 2024-09-19T08:48:03.351 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:48:03.351 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":28,"btime":"2024-09-19T08:45:46:903004+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:45:46.852789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:48:03.352 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 28 2024-09-19T08:48:03.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.355+0000 7f9904979640 1 -- 172.21.15.164:0/2700486514 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f98e80a3430 msgr2=0x7f98e80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:03.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.356+0000 7f9904979640 1 --2- 172.21.15.164:0/2700486514 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f98e80a3430 0x7f98e80a58f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f98f4004770 tx=0x7f98f40058d0 comp rx=0 tx=0).stop 2024-09-19T08:48:03.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.356+0000 7f9904979640 1 -- 172.21.15.164:0/2700486514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99001002f0 msgr2=0x7f990019a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:03.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.356+0000 7f9904979640 1 --2- 172.21.15.164:0/2700486514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99001002f0 0x7f990019a680 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f98f000cce0 tx=0x7f98f000a620 comp rx=0 tx=0).stop 2024-09-19T08:48:03.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.358+0000 7f9904979640 1 -- 172.21.15.164:0/2700486514 shutdown_connections 2024-09-19T08:48:03.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.358+0000 7f9904979640 1 --2- 172.21.15.164:0/2700486514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9900106f50 0x7f990019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:03.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.358+0000 7f9904979640 1 --2- 172.21.15.164:0/2700486514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9900100ca0 0x7f990019abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:03.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.358+0000 7f9904979640 1 --2- 172.21.15.164:0/2700486514 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f98e80a3430 0x7f98e80a58f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:03.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.358+0000 7f9904979640 1 --2- 172.21.15.164:0/2700486514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99001002f0 0x7f990019a680 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:03.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.358+0000 7f9904979640 1 -- 172.21.15.164:0/2700486514 >> 172.21.15.164:0/2700486514 conn(0x7f9900077210 msgr2=0x7f9900105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:03.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.365+0000 7f9904979640 1 -- 172.21.15.164:0/2700486514 shutdown_connections 2024-09-19T08:48:03.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:03.365+0000 7f9904979640 1 -- 172.21.15.164:0/2700486514 wait complete. 2024-09-19T08:48:03.474 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T08:48:03.474 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T08:48:03.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:03 smithi170 ceph-mon[35072]: pgmap v70: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 439 MiB/s wr, 1.67k op/s 2024-09-19T08:48:03.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:03 smithi173 ceph-mon[35306]: pgmap v70: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 439 MiB/s wr, 1.67k op/s 2024-09-19T08:48:04.388 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id 2966dfc9 --release --await' 2024-09-19T08:48:04.389 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id 2966dfc9 --release --await 2024-09-19T08:48:04.424 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:48:04.531 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:04 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2700486514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:48:04.624 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:04 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2700486514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:48:04.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:04 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2700486514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:48:04.983 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:48:04.983 INFO:teuthology.orchestra.run.smithi164.stdout: "return_code": 0, 2024-09-19T08:48:04.983 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_tag": "204f4680-1dc6-4701-a0ff-1640d5067bc7", 2024-09-19T08:48:04.983 INFO:teuthology.orchestra.run.smithi164.stdout: "mode": "asynchronous" 2024-09-19T08:48:04.983 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:48:04.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.933+0000 7fc4ddbed640 1 -- 172.21.15.164:0/1056013204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4d810c170 msgr2=0x7fc4d810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:04.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.933+0000 7fc4ddbed640 1 --2- 172.21.15.164:0/1056013204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4d810c170 0x7fc4d810e610 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0009900 tx=0x7fc4d002a480 comp rx=0 tx=0).stop 2024-09-19T08:48:04.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.933+0000 7fc4ddbed640 1 -- 172.21.15.164:0/1056013204 shutdown_connections 2024-09-19T08:48:04.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.933+0000 7fc4ddbed640 1 --2- 172.21.15.164:0/1056013204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4d810c170 0x7fc4d810e610 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:04.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.933+0000 7fc4ddbed640 1 --2- 172.21.15.164:0/1056013204 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4d8103660 0x7fc4d810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:04.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.934+0000 7fc4ddbed640 1 --2- 172.21.15.164:0/1056013204 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4d8102d40 0x7fc4d8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:04.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.934+0000 7fc4ddbed640 1 -- 172.21.15.164:0/1056013204 >> 172.21.15.164:0/1056013204 conn(0x7fc4d80fc7a0 msgr2=0x7fc4d80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:04.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.934+0000 7fc4ddbed640 1 -- 172.21.15.164:0/1056013204 shutdown_connections 2024-09-19T08:48:04.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.934+0000 7fc4ddbed640 1 -- 172.21.15.164:0/1056013204 wait complete. 2024-09-19T08:48:04.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.934+0000 7fc4ddbed640 1 Processor -- start 2024-09-19T08:48:04.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.935+0000 7fc4ddbed640 1 -- start start 2024-09-19T08:48:04.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.935+0000 7fc4ddbed640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4d8102d40 0x7fc4d8198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:04.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.935+0000 7fc4ddbed640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4d8103660 0x7fc4d81989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:04.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.935+0000 7fc4ddbed640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4d810c170 0x7fc4d819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:04.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.935+0000 7fc4ddbed640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d8111f50 con 0x7fc4d810c170 2024-09-19T08:48:04.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.935+0000 7fc4ddbed640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d8111dd0 con 0x7fc4d8103660 2024-09-19T08:48:04.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.935+0000 7fc4ddbed640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d81120d0 con 0x7fc4d8102d40 2024-09-19T08:48:04.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.935+0000 7fc4d7fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4d810c170 0x7fc4d819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:04.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.935+0000 7fc4d7fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4d810c170 0x7fc4d819cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59096/0 (socket says 172.21.15.164:59096) 2024-09-19T08:48:04.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.935+0000 7fc4d7fff640 1 -- 172.21.15.164:0/1224704625 learned_addr learned my addr 172.21.15.164:0/1224704625 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:04.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.935+0000 7fc4d6ffd640 1 --2- 172.21.15.164:0/1224704625 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4d8103660 0x7fc4d81989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:04.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.935+0000 7fc4d77fe640 1 --2- 172.21.15.164:0/1224704625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4d8102d40 0x7fc4d8198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:04.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.935+0000 7fc4d7fff640 1 -- 172.21.15.164:0/1224704625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4d8102d40 msgr2=0x7fc4d8198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:04.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.935+0000 7fc4d7fff640 1 --2- 172.21.15.164:0/1224704625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4d8102d40 0x7fc4d8198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:04.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.935+0000 7fc4d7fff640 1 -- 172.21.15.164:0/1224704625 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4d8103660 msgr2=0x7fc4d81989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:04.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.935+0000 7fc4d7fff640 1 --2- 172.21.15.164:0/1224704625 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4d8103660 0x7fc4d81989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:04.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.935+0000 7fc4d7fff640 1 -- 172.21.15.164:0/1224704625 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d819d410 con 0x7fc4d810c170 2024-09-19T08:48:04.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.935+0000 7fc4d77fe640 1 --2- 172.21.15.164:0/1224704625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4d8102d40 0x7fc4d8198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:48:04.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.935+0000 7fc4d7fff640 1 --2- 172.21.15.164:0/1224704625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4d810c170 0x7fc4d819cd30 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0005ab0 tx=0x7fc4d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:04.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.936+0000 7fc4d4ff9640 1 -- 172.21.15.164:0/1224704625 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d003f030 con 0x7fc4d810c170 2024-09-19T08:48:04.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.936+0000 7fc4ddbed640 1 -- 172.21.15.164:0/1224704625 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4d819d6a0 con 0x7fc4d810c170 2024-09-19T08:48:04.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.936+0000 7fc4d4ff9640 1 -- 172.21.15.164:0/1224704625 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4d0003ce0 con 0x7fc4d810c170 2024-09-19T08:48:04.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.936+0000 7fc4d4ff9640 1 -- 172.21.15.164:0/1224704625 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d0035070 con 0x7fc4d810c170 2024-09-19T08:48:04.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.936+0000 7fc4ddbed640 1 -- 172.21.15.164:0/1224704625 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4d81a4fe0 con 0x7fc4d810c170 2024-09-19T08:48:04.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.938+0000 7fc4d4ff9640 1 -- 172.21.15.164:0/1224704625 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fc4d0042030 con 0x7fc4d810c170 2024-09-19T08:48:04.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.938+0000 7fc4dffc6740 1 Processor -- start 2024-09-19T08:48:04.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.938+0000 7fc4dffc6740 1 -- start start 2024-09-19T08:48:04.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.938+0000 7fc4d4ff9640 1 --2- 172.21.15.164:0/1224704625 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fc4c00a33b0 0x7fc4c00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:04.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.938+0000 7fc4dffc6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56239b60 0x561c56239f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:04.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.939+0000 7fc4dffc6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c5623a480 0x561c56244800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:04.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.939+0000 7fc4dffc6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c56244d40 0x561c56247130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:04.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.939+0000 7fc4dffc6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561c5619f280 con 0x561c56239b60 2024-09-19T08:48:04.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.939+0000 7fc4dffc6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561c56146b10 con 0x561c5623a480 2024-09-19T08:48:04.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.939+0000 7fc4dffc6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x561c5619b240 con 0x561c56244d40 2024-09-19T08:48:04.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.939+0000 7fc4d6ffd640 1 --2- 172.21.15.164:0/1224704625 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fc4c00a33b0 0x7fc4c00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:04.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.939+0000 7fc4d77fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56239b60 0x561c56239f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:04.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.939+0000 7fc4d4ff9640 1 -- 172.21.15.164:0/1224704625 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fc4d00f04f0 con 0x7fc4d810c170 2024-09-19T08:48:04.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.939+0000 7fc4d77fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56239b60 0x561c56239f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59104/0 (socket says 172.21.15.164:59104) 2024-09-19T08:48:04.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.939+0000 7fc4d77fe640 1 -- 172.21.15.164:0/2015881657 learned_addr learned my addr 172.21.15.164:0/2015881657 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:04.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.939+0000 7fc4d6ffd640 1 --2- 172.21.15.164:0/2015881657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c5623a480 0x561c56244800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:04.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.939+0000 7fc4d77fe640 1 -- 172.21.15.164:0/2015881657 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c56244d40 msgr2=0x561c56247130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:04.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.939+0000 7fc4d77fe640 1 --2- 172.21.15.164:0/2015881657 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c56244d40 0x561c56247130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:04.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.939+0000 7fc4d77fe640 1 -- 172.21.15.164:0/2015881657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c5623a480 msgr2=0x561c56244800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:04.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.939+0000 7fc4d77fe640 1 --2- 172.21.15.164:0/2015881657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c5623a480 0x561c56244800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:04.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.939+0000 7fc4d77fe640 1 -- 172.21.15.164:0/2015881657 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c561531f0 con 0x561c56239b60 2024-09-19T08:48:04.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.939+0000 7fc4d6ffd640 1 --2- 172.21.15.164:0/1224704625 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fc4c00a33b0 0x7fc4c00a5870 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc4d80fedc0 tx=0x7fc4cc013830 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:04.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.939+0000 7fc4d77fe640 1 --2- 172.21.15.164:0/2015881657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56239b60 0x561c56239f40 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc4c80103a0 tx=0x7fc4c8010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82fd709d8de577ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:04.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.939+0000 7fc4bdffb640 1 -- 172.21.15.164:0/2015881657 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c8016070 con 0x561c56239b60 2024-09-19T08:48:04.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.939+0000 7fc4bdffb640 1 -- 172.21.15.164:0/2015881657 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c8012bf0 con 0x561c56239b60 2024-09-19T08:48:04.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.939+0000 7fc4bdffb640 1 -- 172.21.15.164:0/2015881657 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c801bc80 con 0x561c56239b60 2024-09-19T08:48:04.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.940+0000 7fc4dffc6740 1 -- 172.21.15.164:0/2015881657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56239b60 msgr2=0x561c56239f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:04.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.940+0000 7fc4dffc6740 1 --2- 172.21.15.164:0/2015881657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56239b60 0x561c56239f40 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc4c80103a0 tx=0x7fc4c8010860 comp rx=0 tx=0).stop 2024-09-19T08:48:04.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.940+0000 7fc4dffc6740 1 -- 172.21.15.164:0/2015881657 shutdown_connections 2024-09-19T08:48:04.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.940+0000 7fc4dffc6740 1 --2- 172.21.15.164:0/2015881657 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c56244d40 0x561c56247130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:04.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.940+0000 7fc4dffc6740 1 --2- 172.21.15.164:0/2015881657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c5623a480 0x561c56244800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:04.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.940+0000 7fc4dffc6740 1 --2- 172.21.15.164:0/2015881657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56239b60 0x561c56239f40 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:04.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.940+0000 7fc4dffc6740 1 -- 172.21.15.164:0/2015881657 >> 172.21.15.164:0/2015881657 conn(0x561c562360f0 msgr2=0x561c562364d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:04.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.940+0000 7fc4dffc6740 1 -- 172.21.15.164:0/2015881657 shutdown_connections 2024-09-19T08:48:04.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.940+0000 7fc4dffc6740 1 -- 172.21.15.164:0/2015881657 wait complete. 2024-09-19T08:48:04.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.940+0000 7fc4dffc6740 1 -- start start 2024-09-19T08:48:04.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.941+0000 7fc4dffc6740 1 Processor -- start 2024-09-19T08:48:04.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.941+0000 7fc4dffc6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c56239b60 0x561c56241c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:04.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.941+0000 7fc4dffc6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5623a480 0x561c56242160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:04.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.941+0000 7fc4dffc6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c56244d40 0x561c562426a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:04.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.941+0000 7fc4dffc6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561c5624bd20 con 0x561c5623a480 2024-09-19T08:48:04.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.941+0000 7fc4dffc6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561c56159f10 con 0x561c56244d40 2024-09-19T08:48:04.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.941+0000 7fc4dffc6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x561c5624bea0 con 0x561c56239b60 2024-09-19T08:48:04.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.941+0000 7fc4dffc6740 10 client.0 _ll_register_callbacks cb 0x561c56122a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:48:04.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.941+0000 7fc4d6ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5623a480 0x561c56242160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:04.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.941+0000 7fc4d6ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5623a480 0x561c56242160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59114/0 (socket says 172.21.15.164:59114) 2024-09-19T08:48:04.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.941+0000 7fc4d6ffd640 1 -- 172.21.15.164:0/1613735972 learned_addr learned my addr 172.21.15.164:0/1613735972 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:04.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.941+0000 7fc4d77fe640 1 --2- 172.21.15.164:0/1613735972 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c56239b60 0x561c56241c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:04.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.941+0000 7fc4d7fff640 1 --2- 172.21.15.164:0/1613735972 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c56244d40 0x561c562426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:04.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.941+0000 7fc4d6ffd640 1 -- 172.21.15.164:0/1613735972 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c56239b60 msgr2=0x561c56241c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:04.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.941+0000 7fc4d6ffd640 1 --2- 172.21.15.164:0/1613735972 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c56239b60 0x561c56241c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:04.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.941+0000 7fc4d6ffd640 1 -- 172.21.15.164:0/1613735972 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c56244d40 msgr2=0x561c562426a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:04.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.941+0000 7fc4d6ffd640 1 --2- 172.21.15.164:0/1613735972 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c56244d40 0x561c562426a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:04.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.941+0000 7fc4d6ffd640 1 -- 172.21.15.164:0/1613735972 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c55f106a0 con 0x561c5623a480 2024-09-19T08:48:04.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.942+0000 7fc4d6ffd640 1 --2- 172.21.15.164:0/1613735972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5623a480 0x561c56242160 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc00eb90 tx=0x7fc4cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae9e0def5ae8150 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:04.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.942+0000 7fc4b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:48:04.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.942+0000 7fc4b6ffd640 1 -- 172.21.15.164:0/1613735972 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc011d70 con 0x561c5623a480 2024-09-19T08:48:04.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.942+0000 7fc4dffc6740 1 -- 172.21.15.164:0/1613735972 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4c8012bf0 con 0x561c5623a480 2024-09-19T08:48:04.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.942+0000 7fc4b6ffd640 1 -- 172.21.15.164:0/1613735972 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fc4cc01e440 con 0x561c5623a480 2024-09-19T08:48:04.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.942+0000 7fc4b6ffd640 1 -- 172.21.15.164:0/1613735972 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc4cc01e640 con 0x561c5623a480 2024-09-19T08:48:04.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.942+0000 7fc4dffc6740 10 client.16050 fetch_fsmap learned FSMap version 28 2024-09-19T08:48:04.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.942+0000 7fc4dffc6740 1 -- 172.21.15.164:0/1613735972 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561c56257920 con 0x561c5623a480 2024-09-19T08:48:04.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.942+0000 7fc4b6ffd640 1 -- 172.21.15.164:0/1613735972 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c8012bf0 con 0x561c5623a480 2024-09-19T08:48:04.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.942+0000 7fc4b6ffd640 1 -- 172.21.15.164:0/1613735972 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc4cc030020 con 0x561c5623a480 2024-09-19T08:48:04.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.942+0000 7fc4dffc6740 10 client.16050 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:48:04.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.942+0000 7fc4dffc6740 10 client.16050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:48:04.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.942+0000 7fc4dffc6740 20 client.16050 populate_metadata read hostname 'smithi164' 2024-09-19T08:48:04.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.942+0000 7fc4dffc6740 1 --2- 172.21.15.164:0/1613735972 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x561c56258d30 0x561c5625b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:04.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.942+0000 7fc4dffc6740 4 client.16050 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:48:04.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.942+0000 7fc4dffc6740 1 -- 172.21.15.164:0/1613735972 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c561ac7f0 con 0x561c56258d30 2024-09-19T08:48:04.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.943+0000 7fc4d77fe640 1 --2- 172.21.15.164:0/1613735972 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x561c56258d30 0x561c5625b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:04.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.943+0000 7fc4b6ffd640 10 client.16050 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:48:04.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.943+0000 7fc4d77fe640 1 --2- 172.21.15.164:0/1613735972 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x561c56258d30 0x561c5625b150 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45d8c30a1c79da6 server_cookie=22b788c9740d0c02 in_seq=0 out_seq=0 2024-09-19T08:48:04.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.944+0000 7fc4b6ffd640 1 -- 172.21.15.164:0/1613735972 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561c561ac7f0 con 0x561c56258d30 2024-09-19T08:48:04.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.944+0000 7fc4b6ffd640 10 client.16050 handle_command_reply: tid=0 2024-09-19T08:48:04.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.944+0000 7fc4dffc6740 1 client.16050 shutdown 2024-09-19T08:48:04.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.944+0000 7fc4dffc6740 1 -- 172.21.15.164:0/1613735972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5623a480 msgr2=0x561c56242160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:04.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.944+0000 7fc4dffc6740 1 --2- 172.21.15.164:0/1613735972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5623a480 0x561c56242160 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc00eb90 tx=0x7fc4cc004730 comp rx=0 tx=0).stop 2024-09-19T08:48:04.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.944+0000 7fc4dffc6740 1 -- 172.21.15.164:0/1613735972 shutdown_connections 2024-09-19T08:48:04.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.944+0000 7fc4dffc6740 1 --2- 172.21.15.164:0/1613735972 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x561c56258d30 0x561c5625b150 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:04.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.944+0000 7fc4dffc6740 1 --2- 172.21.15.164:0/1613735972 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c56244d40 0x561c562426a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:04.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.944+0000 7fc4dffc6740 1 --2- 172.21.15.164:0/1613735972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5623a480 0x561c56242160 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:04.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.944+0000 7fc4dffc6740 1 --2- 172.21.15.164:0/1613735972 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c56239b60 0x561c56241c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:04.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.944+0000 7fc4b6ffd640 0 client.16050 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:48:04.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.945+0000 7fc4dffc6740 1 -- 172.21.15.164:0/1613735972 >> 172.21.15.164:0/1613735972 conn(0x561c562360f0 msgr2=0x561c5623be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:04.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.945+0000 7fc4dffc6740 1 -- 172.21.15.164:0/1613735972 shutdown_connections 2024-09-19T08:48:04.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.945+0000 7fc4dffc6740 1 -- 172.21.15.164:0/1613735972 wait complete. 2024-09-19T08:48:04.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.945+0000 7fc4dffc6740 20 client.16050 trim_cache size 0 max 16384 2024-09-19T08:48:04.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.962+0000 7fc4dffc6740 1 Processor -- start 2024-09-19T08:48:04.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.962+0000 7fc4dffc6740 1 -- start start 2024-09-19T08:48:04.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.962+0000 7fc4dffc6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c56239b60 0x561c56260a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:04.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.962+0000 7fc4dffc6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5623a480 0x561c56260f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:04.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.962+0000 7fc4dffc6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c56244d40 0x561c56243370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:04.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.962+0000 7fc4dffc6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d81ab520 con 0x561c5623a480 2024-09-19T08:48:04.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.962+0000 7fc4dffc6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d81ad060 con 0x561c56244d40 2024-09-19T08:48:04.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.962+0000 7fc4dffc6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d81abbf0 con 0x561c56239b60 2024-09-19T08:48:04.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.962+0000 7fc4d6ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5623a480 0x561c56260f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:04.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.962+0000 7fc4d6ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5623a480 0x561c56260f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59124/0 (socket says 172.21.15.164:59124) 2024-09-19T08:48:04.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.962+0000 7fc4d6ffd640 1 -- 172.21.15.164:0/2754681454 learned_addr learned my addr 172.21.15.164:0/2754681454 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:04.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.962+0000 7fc4d6ffd640 1 -- 172.21.15.164:0/2754681454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c56239b60 msgr2=0x561c56260a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:04.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.962+0000 7fc4d6ffd640 1 --2- 172.21.15.164:0/2754681454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c56239b60 0x561c56260a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:04.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.962+0000 7fc4d6ffd640 1 -- 172.21.15.164:0/2754681454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c56244d40 msgr2=0x561c56243370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:04.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.962+0000 7fc4d6ffd640 1 --2- 172.21.15.164:0/2754681454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c56244d40 0x561c56243370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:04.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.962+0000 7fc4d6ffd640 1 -- 172.21.15.164:0/2754681454 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c56235f20 con 0x561c5623a480 2024-09-19T08:48:04.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.962+0000 7fc4d6ffd640 1 --2- 172.21.15.164:0/2754681454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5623a480 0x561c56260f40 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc000c00 tx=0x7fc4cc005f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a565514f27e7a4ae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:04.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.962+0000 7fc4bd7fa640 1 -- 172.21.15.164:0/2754681454 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc032e60 con 0x561c5623a480 2024-09-19T08:48:04.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.962+0000 7fc4bd7fa640 1 -- 172.21.15.164:0/2754681454 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4cc0117b0 con 0x561c5623a480 2024-09-19T08:48:04.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.963+0000 7fc4bd7fa640 1 -- 172.21.15.164:0/2754681454 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc032460 con 0x561c5623a480 2024-09-19T08:48:04.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.963+0000 7fc4dffc6740 1 -- 172.21.15.164:0/2754681454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5623a480 msgr2=0x561c56260f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:04.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.963+0000 7fc4dffc6740 1 --2- 172.21.15.164:0/2754681454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5623a480 0x561c56260f40 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc000c00 tx=0x7fc4cc005f50 comp rx=0 tx=0).stop 2024-09-19T08:48:04.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.963+0000 7fc4dffc6740 1 -- 172.21.15.164:0/2754681454 shutdown_connections 2024-09-19T08:48:04.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.963+0000 7fc4dffc6740 1 --2- 172.21.15.164:0/2754681454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c56244d40 0x561c56243370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:04.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.963+0000 7fc4dffc6740 1 --2- 172.21.15.164:0/2754681454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5623a480 0x561c56260f40 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:04.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.963+0000 7fc4dffc6740 1 --2- 172.21.15.164:0/2754681454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c56239b60 0x561c56260a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:04.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.963+0000 7fc4dffc6740 1 -- 172.21.15.164:0/2754681454 >> 172.21.15.164:0/2754681454 conn(0x561c562360f0 msgr2=0x561c5623be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:04.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.963+0000 7fc4dffc6740 1 -- 172.21.15.164:0/2754681454 shutdown_connections 2024-09-19T08:48:04.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.963+0000 7fc4dffc6740 1 -- 172.21.15.164:0/2754681454 wait complete. 2024-09-19T08:48:04.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.963+0000 7fc4dffc6740 1 -- start start 2024-09-19T08:48:04.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.964+0000 7fc4dffc6740 1 Processor -- start 2024-09-19T08:48:04.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.964+0000 7fc4dffc6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c56239b60 0x561c56251960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:04.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.964+0000 7fc4dffc6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c5623a480 0x561c5624cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:04.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.964+0000 7fc4dffc6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56244d40 0x561c5624d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:04.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.964+0000 7fc4dffc6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d81ac990 con 0x561c56244d40 2024-09-19T08:48:04.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.964+0000 7fc4dffc6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d81ac2c0 con 0x561c56239b60 2024-09-19T08:48:04.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.964+0000 7fc4dffc6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d81a9520 con 0x561c5623a480 2024-09-19T08:48:04.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.964+0000 7fc4dffc6740 10 client.0 _ll_register_callbacks cb 0x7fc4cc00a890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:48:04.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.964+0000 7fc4d7fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56244d40 0x561c5624d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:04.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.964+0000 7fc4d7fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56244d40 0x561c5624d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59126/0 (socket says 172.21.15.164:59126) 2024-09-19T08:48:04.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.964+0000 7fc4d7fff640 1 -- 172.21.15.164:0/2754564523 learned_addr learned my addr 172.21.15.164:0/2754564523 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:04.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.964+0000 7fc4d6ffd640 1 --2- 172.21.15.164:0/2754564523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c5623a480 0x561c5624cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:04.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.964+0000 7fc4d77fe640 1 --2- 172.21.15.164:0/2754564523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c56239b60 0x561c56251960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:04.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.964+0000 7fc4d7fff640 1 -- 172.21.15.164:0/2754564523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c5623a480 msgr2=0x561c5624cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:04.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.964+0000 7fc4d7fff640 1 --2- 172.21.15.164:0/2754564523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c5623a480 0x561c5624cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:04.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.964+0000 7fc4d7fff640 1 -- 172.21.15.164:0/2754564523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c56239b60 msgr2=0x561c56251960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:04.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.964+0000 7fc4d7fff640 1 --2- 172.21.15.164:0/2754564523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c56239b60 0x561c56251960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:04.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.964+0000 7fc4d7fff640 1 -- 172.21.15.164:0/2754564523 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c5624dab0 con 0x561c56244d40 2024-09-19T08:48:04.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.964+0000 7fc4d6ffd640 1 --2- 172.21.15.164:0/2754564523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c5623a480 0x561c5624cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:48:04.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.964+0000 7fc4d77fe640 1 --2- 172.21.15.164:0/2754564523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c56239b60 0x561c56251960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:48:04.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.965+0000 7fc4d7fff640 1 --2- 172.21.15.164:0/2754564523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56244d40 0x561c5624d280 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0009a20 tx=0x7fc4d0009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62457beef6828b7c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:04.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.965+0000 7fc4b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:48:04.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.965+0000 7fc4b6ffd640 1 -- 172.21.15.164:0/2754564523 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d0049040 con 0x561c56244d40 2024-09-19T08:48:04.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.965+0000 7fc4b6ffd640 1 -- 172.21.15.164:0/2754564523 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fc4d00b8e50 con 0x561c56244d40 2024-09-19T08:48:04.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.965+0000 7fc4dffc6740 1 -- 172.21.15.164:0/2754564523 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4cc0117b0 con 0x561c56244d40 2024-09-19T08:48:04.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.965+0000 7fc4b6ffd640 1 -- 172.21.15.164:0/2754564523 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc4d00bc140 con 0x561c56244d40 2024-09-19T08:48:04.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.965+0000 7fc4dffc6740 10 client.16056 fetch_fsmap learned FSMap version 28 2024-09-19T08:48:04.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.965+0000 7fc4dffc6740 1 -- 172.21.15.164:0/2754564523 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561c562c3b20 con 0x561c56244d40 2024-09-19T08:48:04.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.965+0000 7fc4b6ffd640 1 -- 172.21.15.164:0/2754564523 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4cc0117b0 con 0x561c56244d40 2024-09-19T08:48:04.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.965+0000 7fc4b6ffd640 1 -- 172.21.15.164:0/2754564523 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc4d003bc10 con 0x561c56244d40 2024-09-19T08:48:04.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.965+0000 7fc4dffc6740 10 client.16056 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:48:04.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.965+0000 7fc4dffc6740 10 client.16056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:48:04.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.965+0000 7fc4dffc6740 20 client.16056 populate_metadata read hostname 'smithi164' 2024-09-19T08:48:04.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.965+0000 7fc4dffc6740 1 --2- 172.21.15.164:0/2754564523 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x561c562c4290 0x561c562c6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:04.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.965+0000 7fc4dffc6740 4 client.16056 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T08:48:04.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.965+0000 7fc4dffc6740 1 -- 172.21.15.164:0/2754564523 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561c561a6d00 con 0x561c562c4290 2024-09-19T08:48:04.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.966+0000 7fc4d77fe640 1 --2- 172.21.15.164:0/2754564523 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x561c562c4290 0x561c562c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:04.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.966+0000 7fc4b6ffd640 10 client.16056 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:48:04.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.966+0000 7fc4d77fe640 1 --2- 172.21.15.164:0/2754564523 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x561c562c4290 0x561c562c6680 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f891f2c4096e49c3 server_cookie=201c2317ae17cf87 in_seq=0 out_seq=0 2024-09-19T08:48:04.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.967+0000 7fc4b6ffd640 1 -- 172.21.15.164:0/2754564523 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561c561a6d00 con 0x561c562c4290 2024-09-19T08:48:04.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.967+0000 7fc4b6ffd640 10 client.16056 handle_command_reply: tid=0 2024-09-19T08:48:05.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.967+0000 7fc4dffc6740 1 client.16056 shutdown 2024-09-19T08:48:05.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.967+0000 7fc4dffc6740 1 -- 172.21.15.164:0/2754564523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56244d40 msgr2=0x561c5624d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:05.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.967+0000 7fc4dffc6740 1 --2- 172.21.15.164:0/2754564523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56244d40 0x561c5624d280 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0009a20 tx=0x7fc4d0009560 comp rx=0 tx=0).stop 2024-09-19T08:48:05.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.967+0000 7fc4dffc6740 1 -- 172.21.15.164:0/2754564523 shutdown_connections 2024-09-19T08:48:05.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.967+0000 7fc4dffc6740 1 --2- 172.21.15.164:0/2754564523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56244d40 0x561c5624d280 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:05.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.967+0000 7fc4dffc6740 1 --2- 172.21.15.164:0/2754564523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c5623a480 0x561c5624cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:05.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.967+0000 7fc4dffc6740 1 --2- 172.21.15.164:0/2754564523 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x561c562c4290 0x561c562c6680 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:05.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.967+0000 7fc4dffc6740 1 --2- 172.21.15.164:0/2754564523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c56239b60 0x561c56251960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:05.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.967+0000 7fc4b6ffd640 0 client.16056 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:48:05.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.967+0000 7fc4dffc6740 1 -- 172.21.15.164:0/2754564523 >> 172.21.15.164:0/2754564523 conn(0x561c562360f0 msgr2=0x561c562403c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:05.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.968+0000 7fc4dffc6740 1 -- 172.21.15.164:0/2754564523 shutdown_connections 2024-09-19T08:48:05.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.968+0000 7fc4dffc6740 1 -- 172.21.15.164:0/2754564523 wait complete. 2024-09-19T08:48:05.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.968+0000 7fc4dffc6740 20 client.16056 trim_cache size 0 max 16384 2024-09-19T08:48:05.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.968+0000 7fc4ddbed640 1 -- 172.21.15.164:0/1224704625 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fc4c00a33b0 msgr2=0x7fc4c00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:05.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.968+0000 7fc4ddbed640 1 --2- 172.21.15.164:0/1224704625 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fc4c00a33b0 0x7fc4c00a5870 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc4d80fedc0 tx=0x7fc4cc013830 comp rx=0 tx=0).stop 2024-09-19T08:48:05.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.968+0000 7fc4ddbed640 1 -- 172.21.15.164:0/1224704625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4d810c170 msgr2=0x7fc4d819cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:05.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.968+0000 7fc4ddbed640 1 --2- 172.21.15.164:0/1224704625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4d810c170 0x7fc4d819cd30 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0005ab0 tx=0x7fc4d00042f0 comp rx=0 tx=0).stop 2024-09-19T08:48:05.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.968+0000 7fc4ddbed640 1 -- 172.21.15.164:0/1224704625 shutdown_connections 2024-09-19T08:48:05.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.968+0000 7fc4ddbed640 1 --2- 172.21.15.164:0/1224704625 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fc4c00a33b0 0x7fc4c00a5870 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:05.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.968+0000 7fc4ddbed640 1 --2- 172.21.15.164:0/1224704625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4d810c170 0x7fc4d819cd30 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:05.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.968+0000 7fc4ddbed640 1 --2- 172.21.15.164:0/1224704625 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4d8103660 0x7fc4d81989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:05.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.968+0000 7fc4ddbed640 1 --2- 172.21.15.164:0/1224704625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4d8102d40 0x7fc4d8198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:05.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.968+0000 7fc4ddbed640 1 -- 172.21.15.164:0/1224704625 >> 172.21.15.164:0/1224704625 conn(0x7fc4d80fc7a0 msgr2=0x7fc4d80fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:05.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.968+0000 7fc4ddbed640 1 -- 172.21.15.164:0/1224704625 shutdown_connections 2024-09-19T08:48:05.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:04.968+0000 7fc4ddbed640 1 -- 172.21.15.164:0/1224704625 wait complete. 2024-09-19T08:48:05.228 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:48:05.531 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:05 smithi164 ceph-mon[30928]: pgmap v71: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 462 MiB/s wr, 1.71k op/s 2024-09-19T08:48:05.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:05 smithi164 ceph-mon[30928]: scrub queued for path: / 2024-09-19T08:48:05.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:05 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:48:05.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:05 smithi164 ceph-mon[30928]: scrub summary: active paths [/] 2024-09-19T08:48:05.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:05 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:48:05.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:05 smithi170 ceph-mon[35072]: pgmap v71: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 462 MiB/s wr, 1.71k op/s 2024-09-19T08:48:05.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:05 smithi170 ceph-mon[35072]: scrub queued for path: / 2024-09-19T08:48:05.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:05 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:48:05.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:05 smithi170 ceph-mon[35072]: scrub summary: active paths [/] 2024-09-19T08:48:05.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:05 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:48:05.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:05 smithi173 ceph-mon[35306]: pgmap v71: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 462 MiB/s wr, 1.71k op/s 2024-09-19T08:48:05.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:05 smithi173 ceph-mon[35306]: scrub queued for path: / 2024-09-19T08:48:05.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:05 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:48:05.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:05 smithi173 ceph-mon[35306]: scrub summary: active paths [/] 2024-09-19T08:48:05.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:05 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:48:05.733 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:48:06.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.236+0000 7f1177790640 1 -- 172.21.15.164:0/1910296040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1170102790 msgr2=0x7f11701067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:06.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.236+0000 7f1177790640 1 --2- 172.21.15.164:0/1910296040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1170102790 0x7f11701067e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f1164009900 tx=0x7f116402a410 comp rx=0 tx=0).stop 2024-09-19T08:48:06.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.236+0000 7f1177790640 1 -- 172.21.15.164:0/1910296040 shutdown_connections 2024-09-19T08:48:06.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.236+0000 7f1177790640 1 --2- 172.21.15.164:0/1910296040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1170106f10 0x7f117010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.236+0000 7f1177790640 1 --2- 172.21.15.164:0/1910296040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1170102790 0x7f11701067e0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.236+0000 7f1177790640 1 --2- 172.21.15.164:0/1910296040 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1170101de0 0x7f11701021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.236+0000 7f1177790640 1 -- 172.21.15.164:0/1910296040 >> 172.21.15.164:0/1910296040 conn(0x7f11700fd8c0 msgr2=0x7f11700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:06.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.236+0000 7f1177790640 1 -- 172.21.15.164:0/1910296040 shutdown_connections 2024-09-19T08:48:06.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.236+0000 7f1177790640 1 -- 172.21.15.164:0/1910296040 wait complete. 2024-09-19T08:48:06.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.237+0000 7f1177790640 1 Processor -- start 2024-09-19T08:48:06.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.237+0000 7f1177790640 1 -- start start 2024-09-19T08:48:06.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.237+0000 7f1177790640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1170101de0 0x7f1170198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:06.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.237+0000 7f1177790640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1170102790 0x7f1170198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:06.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.237+0000 7f1177790640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1170106f10 0x7f117019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:06.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.237+0000 7f1177790640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1170111f90 con 0x7f1170106f10 2024-09-19T08:48:06.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.237+0000 7f1177790640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1170111e10 con 0x7f1170102790 2024-09-19T08:48:06.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.237+0000 7f1177790640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1170112110 con 0x7f1170101de0 2024-09-19T08:48:06.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.237+0000 7f1175d2d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1170106f10 0x7f117019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:06.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.237+0000 7f1175d2d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1170106f10 0x7f117019cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59142/0 (socket says 172.21.15.164:59142) 2024-09-19T08:48:06.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.237+0000 7f1175d2d640 1 -- 172.21.15.164:0/555047533 learned_addr learned my addr 172.21.15.164:0/555047533 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:06.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.237+0000 7f117552c640 1 --2- 172.21.15.164:0/555047533 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1170101de0 0x7f1170198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:06.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.237+0000 7f1174d2b640 1 --2- 172.21.15.164:0/555047533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1170102790 0x7f1170198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:06.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.237+0000 7f1175d2d640 1 -- 172.21.15.164:0/555047533 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1170101de0 msgr2=0x7f1170198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:06.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.237+0000 7f1175d2d640 1 --2- 172.21.15.164:0/555047533 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1170101de0 0x7f1170198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.237+0000 7f1175d2d640 1 -- 172.21.15.164:0/555047533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1170102790 msgr2=0x7f1170198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:06.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.237+0000 7f1175d2d640 1 --2- 172.21.15.164:0/555047533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1170102790 0x7f1170198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.237+0000 7f1175d2d640 1 -- 172.21.15.164:0/555047533 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f117019d3c0 con 0x7f1170106f10 2024-09-19T08:48:06.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.237+0000 7f1175d2d640 1 --2- 172.21.15.164:0/555047533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1170106f10 0x7f117019cce0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f116800cce0 tx=0x7f116800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:06.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.238+0000 7f11627fc640 1 -- 172.21.15.164:0/555047533 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116800f450 con 0x7f1170106f10 2024-09-19T08:48:06.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.238+0000 7f11627fc640 1 -- 172.21.15.164:0/555047533 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f116800fad0 con 0x7f1170106f10 2024-09-19T08:48:06.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.238+0000 7f11627fc640 1 -- 172.21.15.164:0/555047533 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1168018b60 con 0x7f1170106f10 2024-09-19T08:48:06.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.238+0000 7f1177790640 1 -- 172.21.15.164:0/555047533 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f117019d6b0 con 0x7f1170106f10 2024-09-19T08:48:06.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.238+0000 7f1177790640 1 -- 172.21.15.164:0/555047533 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11701a4f90 con 0x7f1170106f10 2024-09-19T08:48:06.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.240+0000 7f11627fc640 1 -- 172.21.15.164:0/555047533 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f116800fc80 con 0x7f1170106f10 2024-09-19T08:48:06.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.241+0000 7f1179b69740 1 Processor -- start 2024-09-19T08:48:06.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.241+0000 7f1179b69740 1 -- start start 2024-09-19T08:48:06.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.241+0000 7f1179b69740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56392e6db060 0x56392e6db440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:06.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.241+0000 7f1179b69740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56392e6db980 0x56392e6e5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:06.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.241+0000 7f1179b69740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56392e6e6240 0x56392e6e8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:06.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.241+0000 7f1179b69740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56392e640620 con 0x56392e6e6240 2024-09-19T08:48:06.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.241+0000 7f1179b69740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56392e5e7e50 con 0x56392e6db060 2024-09-19T08:48:06.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.241+0000 7f1179b69740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56392e63c5e0 con 0x56392e6db980 2024-09-19T08:48:06.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.241+0000 7f1174d2b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56392e6e6240 0x56392e6e8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:06.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.241+0000 7f11627fc640 1 --2- 172.21.15.164:0/555047533 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f115c0a33b0 0x7f115c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:06.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.241+0000 7f1174d2b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56392e6e6240 0x56392e6e8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59158/0 (socket says 172.21.15.164:59158) 2024-09-19T08:48:06.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.241+0000 7f1174d2b640 1 -- 172.21.15.164:0/4294488981 learned_addr learned my addr 172.21.15.164:0/4294488981 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:06.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.241+0000 7f117552c640 1 --2- 172.21.15.164:0/4294488981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56392e6db980 0x56392e6e5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:06.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.241+0000 7f1175d2d640 1 --2- 172.21.15.164:0/555047533 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f115c0a33b0 0x7f115c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:06.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.241+0000 7f1174d2b640 1 --2- 172.21.15.164:0/4294488981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56392e6db060 0x56392e6db440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:06.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.241+0000 7f11627fc640 1 -- 172.21.15.164:0/555047533 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f11680d0240 con 0x7f1170106f10 2024-09-19T08:48:06.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.241+0000 7f117552c640 1 -- 172.21.15.164:0/4294488981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56392e6db060 msgr2=0x56392e6db440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:06.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.241+0000 7f117552c640 1 --2- 172.21.15.164:0/4294488981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56392e6db060 0x56392e6db440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.241+0000 7f117552c640 1 -- 172.21.15.164:0/4294488981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56392e6e6240 msgr2=0x56392e6e8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:06.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.241+0000 7f117552c640 1 --2- 172.21.15.164:0/4294488981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56392e6e6240 0x56392e6e8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.241+0000 7f117552c640 1 -- 172.21.15.164:0/4294488981 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56392e5d8d40 con 0x56392e6db980 2024-09-19T08:48:06.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.241+0000 7f1174d2b640 1 --2- 172.21.15.164:0/4294488981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56392e6e6240 0x56392e6e8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:48:06.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.242+0000 7f117552c640 1 --2- 172.21.15.164:0/4294488981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56392e6db980 0x56392e6e5d00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f116c0099a0 tx=0x7f116c009e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c52abebf5dfa3af1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:06.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.242+0000 7f1175d2d640 1 --2- 172.21.15.164:0/555047533 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f115c0a33b0 0x7f115c0a5870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1168098990 tx=0x7f116806e2b0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:06.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.242+0000 7f115b7fe640 1 -- 172.21.15.164:0/4294488981 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c017070 con 0x56392e6db980 2024-09-19T08:48:06.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.242+0000 7f115b7fe640 1 -- 172.21.15.164:0/4294488981 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f116c013470 con 0x56392e6db980 2024-09-19T08:48:06.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.242+0000 7f115b7fe640 1 -- 172.21.15.164:0/4294488981 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c012610 con 0x56392e6db980 2024-09-19T08:48:06.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.243+0000 7f1179b69740 1 -- 172.21.15.164:0/4294488981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56392e6db980 msgr2=0x56392e6e5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:06.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.243+0000 7f1179b69740 1 --2- 172.21.15.164:0/4294488981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56392e6db980 0x56392e6e5d00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f116c0099a0 tx=0x7f116c009e60 comp rx=0 tx=0).stop 2024-09-19T08:48:06.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.243+0000 7f1179b69740 1 -- 172.21.15.164:0/4294488981 shutdown_connections 2024-09-19T08:48:06.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.243+0000 7f1179b69740 1 --2- 172.21.15.164:0/4294488981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56392e6e6240 0x56392e6e8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.243+0000 7f1179b69740 1 --2- 172.21.15.164:0/4294488981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56392e6db980 0x56392e6e5d00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.243+0000 7f1179b69740 1 --2- 172.21.15.164:0/4294488981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56392e6db060 0x56392e6db440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.243+0000 7f1179b69740 1 -- 172.21.15.164:0/4294488981 >> 172.21.15.164:0/4294488981 conn(0x56392e6d75f0 msgr2=0x56392e6d79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:06.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.243+0000 7f1179b69740 1 -- 172.21.15.164:0/4294488981 shutdown_connections 2024-09-19T08:48:06.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.243+0000 7f1179b69740 1 -- 172.21.15.164:0/4294488981 wait complete. 2024-09-19T08:48:06.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.243+0000 7f1179b69740 1 -- start start 2024-09-19T08:48:06.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.243+0000 7f1179b69740 1 Processor -- start 2024-09-19T08:48:06.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.244+0000 7f1179b69740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56392e6db060 0x56392e6e30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:06.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.244+0000 7f1179b69740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56392e6db980 0x56392e6e3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:06.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.244+0000 7f1179b69740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56392e6e6240 0x56392e6e3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:06.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.244+0000 7f1179b69740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56392e6e8e10 con 0x56392e6e6240 2024-09-19T08:48:06.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.244+0000 7f1179b69740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56392e5c8f10 con 0x56392e6db980 2024-09-19T08:48:06.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.244+0000 7f1179b69740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56392e6e8f90 con 0x56392e6db060 2024-09-19T08:48:06.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.244+0000 7f1179b69740 10 client.0 _ll_register_callbacks cb 0x56392e5ecdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:48:06.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.244+0000 7f1174d2b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56392e6e6240 0x56392e6e3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:06.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.244+0000 7f1174d2b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56392e6e6240 0x56392e6e3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59170/0 (socket says 172.21.15.164:59170) 2024-09-19T08:48:06.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.244+0000 7f1174d2b640 1 -- 172.21.15.164:0/1803599325 learned_addr learned my addr 172.21.15.164:0/1803599325 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:06.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.244+0000 7f1174d2b640 1 --2- 172.21.15.164:0/1803599325 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56392e6db060 0x56392e6e30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:06.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.244+0000 7f117552c640 1 --2- 172.21.15.164:0/1803599325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56392e6db980 0x56392e6e3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:06.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.244+0000 7f1174d2b640 1 -- 172.21.15.164:0/1803599325 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56392e6db060 msgr2=0x56392e6e30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:06.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.244+0000 7f1174d2b640 1 --2- 172.21.15.164:0/1803599325 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56392e6db060 0x56392e6e30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.244+0000 7f1174d2b640 1 -- 172.21.15.164:0/1803599325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56392e6db980 msgr2=0x56392e6e3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:06.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.244+0000 7f1174d2b640 1 --2- 172.21.15.164:0/1803599325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56392e6db980 0x56392e6e3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.244+0000 7f1174d2b640 1 -- 172.21.15.164:0/1803599325 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56392e34f580 con 0x56392e6e6240 2024-09-19T08:48:06.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.244+0000 7f1174d2b640 1 --2- 172.21.15.164:0/1803599325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56392e6e6240 0x56392e6e3b70 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1164033000 tx=0x7f1164000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c73371f16830e121 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:06.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.244+0000 7f1158ff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:48:06.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.245+0000 7f1158ff9640 1 -- 172.21.15.164:0/1803599325 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164046040 con 0x56392e6e6240 2024-09-19T08:48:06.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.245+0000 7f1179b69740 1 -- 172.21.15.164:0/1803599325 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f116c013470 con 0x56392e6e6240 2024-09-19T08:48:06.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.245+0000 7f1158ff9640 1 -- 172.21.15.164:0/1803599325 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f1164004450 con 0x56392e6e6240 2024-09-19T08:48:06.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.245+0000 7f1158ff9640 1 -- 172.21.15.164:0/1803599325 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1164035040 con 0x56392e6e6240 2024-09-19T08:48:06.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.245+0000 7f1179b69740 10 client.16068 fetch_fsmap learned FSMap version 28 2024-09-19T08:48:06.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.245+0000 7f1179b69740 1 -- 172.21.15.164:0/1803599325 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56392e419960 con 0x56392e6e6240 2024-09-19T08:48:06.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.245+0000 7f1158ff9640 1 -- 172.21.15.164:0/1803599325 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f116c013470 con 0x56392e6e6240 2024-09-19T08:48:06.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.245+0000 7f1158ff9640 1 -- 172.21.15.164:0/1803599325 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1164052050 con 0x56392e6e6240 2024-09-19T08:48:06.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.245+0000 7f1179b69740 10 client.16068 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:48:06.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.245+0000 7f1179b69740 10 client.16068 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:48:06.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.245+0000 7f1179b69740 20 client.16068 populate_metadata read hostname 'smithi164' 2024-09-19T08:48:06.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.245+0000 7f1179b69740 1 --2- 172.21.15.164:0/1803599325 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56392e6f5d00 0x56392e6f80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:06.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.245+0000 7f1179b69740 4 client.16068 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:48:06.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.245+0000 7f1179b69740 1 -- 172.21.15.164:0/1803599325 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56392e64db90 con 0x56392e6f5d00 2024-09-19T08:48:06.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.246+0000 7f117552c640 1 --2- 172.21.15.164:0/1803599325 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56392e6f5d00 0x56392e6f80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:06.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.246+0000 7f1158ff9640 10 client.16068 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:48:06.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.246+0000 7f117552c640 1 --2- 172.21.15.164:0/1803599325 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56392e6f5d00 0x56392e6f80f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c7e609047f3f547 server_cookie=362c4162ce414a7c in_seq=0 out_seq=0 2024-09-19T08:48:06.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.247+0000 7f1158ff9640 1 -- 172.21.15.164:0/1803599325 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56392e64db90 con 0x56392e6f5d00 2024-09-19T08:48:06.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.247+0000 7f1158ff9640 10 client.16068 handle_command_reply: tid=0 2024-09-19T08:48:06.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.247+0000 7f1179b69740 1 client.16068 shutdown 2024-09-19T08:48:06.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.247+0000 7f1179b69740 1 -- 172.21.15.164:0/1803599325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56392e6e6240 msgr2=0x56392e6e3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:06.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.247+0000 7f1179b69740 1 --2- 172.21.15.164:0/1803599325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56392e6e6240 0x56392e6e3b70 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1164033000 tx=0x7f1164000bc0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.247+0000 7f1179b69740 1 -- 172.21.15.164:0/1803599325 shutdown_connections 2024-09-19T08:48:06.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.247+0000 7f1179b69740 1 --2- 172.21.15.164:0/1803599325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56392e6e6240 0x56392e6e3b70 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.247+0000 7f1179b69740 1 --2- 172.21.15.164:0/1803599325 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56392e6f5d00 0x56392e6f80f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.247+0000 7f1179b69740 1 --2- 172.21.15.164:0/1803599325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56392e6db980 0x56392e6e3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.247+0000 7f1158ff9640 0 client.16068 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:48:06.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.247+0000 7f1179b69740 1 --2- 172.21.15.164:0/1803599325 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56392e6db060 0x56392e6e30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.247+0000 7f1179b69740 1 -- 172.21.15.164:0/1803599325 >> 172.21.15.164:0/1803599325 conn(0x56392e6d75f0 msgr2=0x56392e6dd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:06.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.247+0000 7f1179b69740 1 -- 172.21.15.164:0/1803599325 shutdown_connections 2024-09-19T08:48:06.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.247+0000 7f1179b69740 1 -- 172.21.15.164:0/1803599325 wait complete. 2024-09-19T08:48:06.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.247+0000 7f1179b69740 20 client.16068 trim_cache size 0 max 16384 2024-09-19T08:48:06.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.266+0000 7f1179b69740 1 Processor -- start 2024-09-19T08:48:06.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.266+0000 7f1179b69740 1 -- start start 2024-09-19T08:48:06.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.266+0000 7f1179b69740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56392e6db060 0x56392e6e4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:06.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.266+0000 7f1179b69740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56392e6db980 0x56392e6ddac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:06.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.266+0000 7f1179b69740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56392e6e6240 0x56392e6fd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:06.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.266+0000 7f1179b69740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f11701ad670 con 0x56392e6db060 2024-09-19T08:48:06.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.266+0000 7f1179b69740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f11701af1b0 con 0x56392e6e6240 2024-09-19T08:48:06.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.266+0000 7f1179b69740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f11701add40 con 0x56392e6db980 2024-09-19T08:48:06.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.266+0000 7f1174d2b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56392e6db060 0x56392e6e4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:06.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.266+0000 7f1174d2b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56392e6db060 0x56392e6e4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59182/0 (socket says 172.21.15.164:59182) 2024-09-19T08:48:06.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.266+0000 7f1174d2b640 1 -- 172.21.15.164:0/1215816775 learned_addr learned my addr 172.21.15.164:0/1215816775 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:06.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.267+0000 7f1174d2b640 1 -- 172.21.15.164:0/1215816775 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56392e6db980 msgr2=0x56392e6ddac0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:48:06.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.267+0000 7f1174d2b640 1 --2- 172.21.15.164:0/1215816775 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56392e6db980 0x56392e6ddac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.267+0000 7f1174d2b640 1 -- 172.21.15.164:0/1215816775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56392e6e6240 msgr2=0x56392e6fd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:06.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.267+0000 7f1174d2b640 1 --2- 172.21.15.164:0/1215816775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56392e6e6240 0x56392e6fd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.267+0000 7f1174d2b640 1 -- 172.21.15.164:0/1215816775 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56392e6d8280 con 0x56392e6db060 2024-09-19T08:48:06.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.267+0000 7f1174d2b640 1 --2- 172.21.15.164:0/1215816775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56392e6db060 0x56392e6e4fc0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1164031840 tx=0x7f11640095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc83f34c03e38b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:06.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.267+0000 7f115affd640 1 -- 172.21.15.164:0/1215816775 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116403d450 con 0x56392e6db060 2024-09-19T08:48:06.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.267+0000 7f115affd640 1 -- 172.21.15.164:0/1215816775 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1164058020 con 0x56392e6db060 2024-09-19T08:48:06.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.267+0000 7f115affd640 1 -- 172.21.15.164:0/1215816775 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164003ad0 con 0x56392e6db060 2024-09-19T08:48:06.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.268+0000 7f1179b69740 1 -- 172.21.15.164:0/1215816775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56392e6db060 msgr2=0x56392e6e4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:06.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.268+0000 7f1179b69740 1 --2- 172.21.15.164:0/1215816775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56392e6db060 0x56392e6e4fc0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1164031840 tx=0x7f11640095f0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.269+0000 7f1179b69740 1 -- 172.21.15.164:0/1215816775 shutdown_connections 2024-09-19T08:48:06.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.269+0000 7f1179b69740 1 --2- 172.21.15.164:0/1215816775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56392e6e6240 0x56392e6fd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.269+0000 7f1179b69740 1 --2- 172.21.15.164:0/1215816775 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56392e6db980 0x56392e6ddac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.269+0000 7f1179b69740 1 --2- 172.21.15.164:0/1215816775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56392e6db060 0x56392e6e4fc0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.269+0000 7f1179b69740 1 -- 172.21.15.164:0/1215816775 >> 172.21.15.164:0/1215816775 conn(0x56392e6d75f0 msgr2=0x56392e6d6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:06.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.269+0000 7f1179b69740 1 -- 172.21.15.164:0/1215816775 shutdown_connections 2024-09-19T08:48:06.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.269+0000 7f1179b69740 1 -- 172.21.15.164:0/1215816775 wait complete. 2024-09-19T08:48:06.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.269+0000 7f1179b69740 1 -- start start 2024-09-19T08:48:06.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.270+0000 7f1179b69740 1 Processor -- start 2024-09-19T08:48:06.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.270+0000 7f1179b69740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56392e6db060 0x56392e6eaa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:06.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.270+0000 7f1179b69740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56392e6db980 0x56392e6eaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:06.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.270+0000 7f1179b69740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56392e6e6240 0x56392e758900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:06.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.270+0000 7f1179b69740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f11701aeae0 con 0x56392e6db060 2024-09-19T08:48:06.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.270+0000 7f1179b69740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f11701ae410 con 0x56392e6e6240 2024-09-19T08:48:06.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.270+0000 7f1179b69740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f11701ab670 con 0x56392e6db980 2024-09-19T08:48:06.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.270+0000 7f1179b69740 10 client.0 _ll_register_callbacks cb 0x7f11640315d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:48:06.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.270+0000 7f1174d2b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56392e6db060 0x56392e6eaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:06.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.270+0000 7f1174d2b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56392e6db060 0x56392e6eaa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59196/0 (socket says 172.21.15.164:59196) 2024-09-19T08:48:06.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.270+0000 7f1174d2b640 1 -- 172.21.15.164:0/485276778 learned_addr learned my addr 172.21.15.164:0/485276778 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:06.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.271+0000 7f117552c640 1 --2- 172.21.15.164:0/485276778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56392e6db980 0x56392e6eaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:06.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.271+0000 7f117552c640 1 -- 172.21.15.164:0/485276778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56392e6e6240 msgr2=0x56392e758900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:06.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.271+0000 7f117552c640 1 --2- 172.21.15.164:0/485276778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56392e6e6240 0x56392e758900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.271+0000 7f117552c640 1 -- 172.21.15.164:0/485276778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56392e6db060 msgr2=0x56392e6eaa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:06.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.271+0000 7f117552c640 1 --2- 172.21.15.164:0/485276778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56392e6db060 0x56392e6eaa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.271+0000 7f117552c640 1 -- 172.21.15.164:0/485276778 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56392e758fa0 con 0x56392e6db980 2024-09-19T08:48:06.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.271+0000 7f1174d2b640 1 --2- 172.21.15.164:0/485276778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56392e6db060 0x56392e6eaa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:48:06.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.271+0000 7f117552c640 1 --2- 172.21.15.164:0/485276778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56392e6db980 0x56392e6eaf90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f116c004810 tx=0x7f116c013e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5313966709cc1c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:06.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.271+0000 7f1158ff9640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-09-19T08:48:06.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.271+0000 7f1158ff9640 1 -- 172.21.15.164:0/485276778 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c017040 con 0x56392e6db980 2024-09-19T08:48:06.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.271+0000 7f1158ff9640 1 -- 172.21.15.164:0/485276778 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f116c013680 con 0x56392e6db980 2024-09-19T08:48:06.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.271+0000 7f1179b69740 1 -- 172.21.15.164:0/485276778 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1164058020 con 0x56392e6db980 2024-09-19T08:48:06.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.272+0000 7f1158ff9640 1 -- 172.21.15.164:0/485276778 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f116c013880 con 0x56392e6db980 2024-09-19T08:48:06.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.272+0000 7f1179b69740 10 client.25525 fetch_fsmap learned FSMap version 28 2024-09-19T08:48:06.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.272+0000 7f1179b69740 1 -- 172.21.15.164:0/485276778 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56392e761500 con 0x56392e6db980 2024-09-19T08:48:06.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.272+0000 7f1158ff9640 1 -- 172.21.15.164:0/485276778 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1164058020 con 0x56392e6db980 2024-09-19T08:48:06.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.272+0000 7f1158ff9640 1 -- 172.21.15.164:0/485276778 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f116c01c450 con 0x56392e6db980 2024-09-19T08:48:06.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.272+0000 7f1179b69740 10 client.25525 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:48:06.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.272+0000 7f1179b69740 10 client.25525 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:48:06.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.272+0000 7f1179b69740 20 client.25525 populate_metadata read hostname 'smithi164' 2024-09-19T08:48:06.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.272+0000 7f1179b69740 1 --2- 172.21.15.164:0/485276778 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56392e6f5d00 0x56392e763a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:06.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.272+0000 7f1179b69740 4 client.25525 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2966dfc9", "release": true, "await": true}] 2024-09-19T08:48:06.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.272+0000 7f1179b69740 1 -- 172.21.15.164:0/485276778 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2966dfc9", "release": true, "await": true}) -- 0x56392e6480a0 con 0x56392e6f5d00 2024-09-19T08:48:06.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.272+0000 7f1174d2b640 1 --2- 172.21.15.164:0/485276778 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56392e6f5d00 0x56392e763a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:06.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.273+0000 7f1174d2b640 1 --2- 172.21.15.164:0/485276778 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56392e6f5d00 0x56392e763a10 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2c980400ae33b34 server_cookie=d6383e19b40cadbe in_seq=0 out_seq=0 2024-09-19T08:48:06.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.273+0000 7f1158ff9640 10 client.25525 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:48:06.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.278+0000 7f1158ff9640 1 -- 172.21.15.164:0/485276778 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56392e6480a0 con 0x56392e6f5d00 2024-09-19T08:48:06.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.278+0000 7f1158ff9640 10 client.25525 handle_command_reply: tid=0 2024-09-19T08:48:06.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.278+0000 7f1179b69740 1 client.25525 shutdown 2024-09-19T08:48:06.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.279+0000 7f1179b69740 1 -- 172.21.15.164:0/485276778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56392e6db980 msgr2=0x56392e6eaf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:06.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.279+0000 7f1179b69740 1 --2- 172.21.15.164:0/485276778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56392e6db980 0x56392e6eaf90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f116c004810 tx=0x7f116c013e10 comp rx=0 tx=0).stop 2024-09-19T08:48:06.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.280+0000 7f1179b69740 1 -- 172.21.15.164:0/485276778 shutdown_connections 2024-09-19T08:48:06.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.280+0000 7f1179b69740 1 --2- 172.21.15.164:0/485276778 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56392e6f5d00 0x56392e763a10 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.280+0000 7f1179b69740 1 --2- 172.21.15.164:0/485276778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56392e6e6240 0x56392e758900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.280+0000 7f1179b69740 1 --2- 172.21.15.164:0/485276778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56392e6db980 0x56392e6eaf90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.280+0000 7f1179b69740 1 --2- 172.21.15.164:0/485276778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56392e6db060 0x56392e6eaa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.280+0000 7f1158ff9640 0 client.25525 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:48:06.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.280+0000 7f1179b69740 1 -- 172.21.15.164:0/485276778 >> 172.21.15.164:0/485276778 conn(0x56392e6d75f0 msgr2=0x56392e6d5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:06.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.281+0000 7f1179b69740 1 -- 172.21.15.164:0/485276778 shutdown_connections 2024-09-19T08:48:06.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.281+0000 7f1179b69740 1 -- 172.21.15.164:0/485276778 wait complete. 2024-09-19T08:48:06.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.281+0000 7f1179b69740 20 client.25525 trim_cache size 0 max 16384 2024-09-19T08:48:06.282 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:48:06.282 INFO:teuthology.orchestra.run.smithi164.stdout: "epoch": 28, 2024-09-19T08:48:06.282 INFO:teuthology.orchestra.run.smithi164.stdout: "leader": 24479, 2024-09-19T08:48:06.282 INFO:teuthology.orchestra.run.smithi164.stdout: "set_version": 12, 2024-09-19T08:48:06.282 INFO:teuthology.orchestra.run.smithi164.stdout: "sets": { 2024-09-19T08:48:06.282 INFO:teuthology.orchestra.run.smithi164.stdout: "2966dfc9": { 2024-09-19T08:48:06.282 INFO:teuthology.orchestra.run.smithi164.stdout: "version": 12, 2024-09-19T08:48:06.282 INFO:teuthology.orchestra.run.smithi164.stdout: "age_ref": 0.0, 2024-09-19T08:48:06.282 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:48:06.282 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "RELEASED", 2024-09-19T08:48:06.282 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:48:06.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:48:06.282 INFO:teuthology.orchestra.run.smithi164.stdout: "timeout": 90.0, 2024-09-19T08:48:06.283 INFO:teuthology.orchestra.run.smithi164.stdout: "expiration": 140.3, 2024-09-19T08:48:06.283 INFO:teuthology.orchestra.run.smithi164.stdout: "members": { 2024-09-19T08:48:06.283 INFO:teuthology.orchestra.run.smithi164.stdout: "file:/": { 2024-09-19T08:48:06.283 INFO:teuthology.orchestra.run.smithi164.stdout: "excluded": false, 2024-09-19T08:48:06.283 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:48:06.283 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "RELEASED", 2024-09-19T08:48:06.283 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:48:06.283 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:48:06.283 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:48:06.283 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:48:06.283 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:48:06.283 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:48:06.283 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:48:06.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.282+0000 7f1177790640 1 -- 172.21.15.164:0/555047533 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f115c0a33b0 msgr2=0x7f115c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:06.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.282+0000 7f1177790640 1 --2- 172.21.15.164:0/555047533 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f115c0a33b0 0x7f115c0a5870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1168098990 tx=0x7f116806e2b0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.282+0000 7f1177790640 1 -- 172.21.15.164:0/555047533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1170106f10 msgr2=0x7f117019cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:06.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.282+0000 7f1177790640 1 --2- 172.21.15.164:0/555047533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1170106f10 0x7f117019cce0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f116800cce0 tx=0x7f116800a620 comp rx=0 tx=0).stop 2024-09-19T08:48:06.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.282+0000 7f1177790640 1 -- 172.21.15.164:0/555047533 shutdown_connections 2024-09-19T08:48:06.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.282+0000 7f1177790640 1 --2- 172.21.15.164:0/555047533 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f115c0a33b0 0x7f115c0a5870 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.282+0000 7f1177790640 1 --2- 172.21.15.164:0/555047533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1170106f10 0x7f117019cce0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.282+0000 7f1177790640 1 --2- 172.21.15.164:0/555047533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1170102790 0x7f1170198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.282+0000 7f1177790640 1 --2- 172.21.15.164:0/555047533 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1170101de0 0x7f1170198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:06.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.282+0000 7f1177790640 1 -- 172.21.15.164:0/555047533 >> 172.21.15.164:0/555047533 conn(0x7f11700fd8c0 msgr2=0x7f11700fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:06.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.283+0000 7f1177790640 1 -- 172.21.15.164:0/555047533 shutdown_connections 2024-09-19T08:48:06.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:06.283+0000 7f1177790640 1 -- 172.21.15.164:0/555047533 wait complete. 2024-09-19T08:48:06.292 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:48:07.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:07 smithi164 ceph-mon[30928]: pgmap v72: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 278 MiB/s wr, 746 op/s 2024-09-19T08:48:07.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:07 smithi164 ceph-mon[30928]: scrub summary: idle 2024-09-19T08:48:07.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:07 smithi170 ceph-mon[35072]: pgmap v72: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 278 MiB/s wr, 746 op/s 2024-09-19T08:48:07.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:07 smithi170 ceph-mon[35072]: scrub summary: idle 2024-09-19T08:48:07.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:07 smithi173 ceph-mon[35306]: pgmap v72: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 278 MiB/s wr, 746 op/s 2024-09-19T08:48:07.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:07 smithi173 ceph-mon[35306]: scrub summary: idle 2024-09-19T08:48:09.534 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:09 smithi164 ceph-mon[30928]: pgmap v73: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 161 MiB/s wr, 153 op/s 2024-09-19T08:48:09.624 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:09 smithi170 ceph-mon[35072]: pgmap v73: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 161 MiB/s wr, 153 op/s 2024-09-19T08:48:09.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:09 smithi173 ceph-mon[35306]: pgmap v73: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 161 MiB/s wr, 153 op/s 2024-09-19T08:48:10.354 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2966dfc9', total seconds elapsed: 29.2 2024-09-19T08:48:10.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.740+0000 7fce86d87640 1 -- 172.21.15.164:0/4061212645 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce8011b140 msgr2=0x7fce8011d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:10.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.740+0000 7fce86d87640 1 --2- 172.21.15.164:0/4061212645 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce8011b140 0x7fce8011d530 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fce78009ca0 tx=0x7fce7802a8e0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.740+0000 7fce86d87640 1 -- 172.21.15.164:0/4061212645 shutdown_connections 2024-09-19T08:48:10.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.740+0000 7fce86d87640 1 --2- 172.21.15.164:0/4061212645 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce8011b140 0x7fce8011d530 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.740+0000 7fce86d87640 1 --2- 172.21.15.164:0/4061212645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce8006ea30 0x7fce8006ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.740+0000 7fce86d87640 1 --2- 172.21.15.164:0/4061212645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce80110220 0x7fce8006e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.740+0000 7fce86d87640 1 -- 172.21.15.164:0/4061212645 >> 172.21.15.164:0/4061212645 conn(0x7fce80073210 msgr2=0x7fce80073620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:10.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.740+0000 7fce86d87640 1 -- 172.21.15.164:0/4061212645 shutdown_connections 2024-09-19T08:48:10.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.740+0000 7fce86d87640 1 -- 172.21.15.164:0/4061212645 wait complete. 2024-09-19T08:48:10.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.741+0000 7fce86d87640 1 Processor -- start 2024-09-19T08:48:10.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.741+0000 7fce86d87640 1 -- start start 2024-09-19T08:48:10.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.742+0000 7fce86d87640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce8006ea30 0x7fce80119640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:10.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.742+0000 7fce86d87640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce80110220 0x7fce80119b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:10.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.742+0000 7fce86d87640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce8011b140 0x7fce801128b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:10.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.742+0000 7fce86d87640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fce80120cf0 con 0x7fce8011b140 2024-09-19T08:48:10.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.742+0000 7fce86d87640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fce80120b70 con 0x7fce80110220 2024-09-19T08:48:10.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.742+0000 7fce86d87640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fce80120e70 con 0x7fce8006ea30 2024-09-19T08:48:10.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.746+0000 7fce85324640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce8011b140 0x7fce801128b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:10.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.746+0000 7fce85324640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce8011b140 0x7fce801128b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59202/0 (socket says 172.21.15.164:59202) 2024-09-19T08:48:10.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.746+0000 7fce85324640 1 -- 172.21.15.164:0/2205061698 learned_addr learned my addr 172.21.15.164:0/2205061698 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:10.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.747+0000 7fce7ffff640 1 --2- 172.21.15.164:0/2205061698 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce80110220 0x7fce80119b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:10.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.747+0000 7fce84b23640 1 --2- 172.21.15.164:0/2205061698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce8006ea30 0x7fce80119640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:10.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.748+0000 7fce84b23640 1 -- 172.21.15.164:0/2205061698 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce80110220 msgr2=0x7fce80119b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:10.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.748+0000 7fce84b23640 1 --2- 172.21.15.164:0/2205061698 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce80110220 0x7fce80119b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.748+0000 7fce84b23640 1 -- 172.21.15.164:0/2205061698 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce8011b140 msgr2=0x7fce801128b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:10.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.748+0000 7fce84b23640 1 --2- 172.21.15.164:0/2205061698 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce8011b140 0x7fce801128b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.748+0000 7fce84b23640 1 -- 172.21.15.164:0/2205061698 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce80113170 con 0x7fce8006ea30 2024-09-19T08:48:10.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.750+0000 7fce84b23640 1 --2- 172.21.15.164:0/2205061698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce8006ea30 0x7fce80119640 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fce7400b3f0 tx=0x7fce7400b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:10.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.750+0000 7fce7dffb640 1 -- 172.21.15.164:0/2205061698 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce74015020 con 0x7fce8006ea30 2024-09-19T08:48:10.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.750+0000 7fce86d87640 1 -- 172.21.15.164:0/2205061698 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce80113460 con 0x7fce8006ea30 2024-09-19T08:48:10.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.751+0000 7fce86d87640 1 -- 172.21.15.164:0/2205061698 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce80117400 con 0x7fce8006ea30 2024-09-19T08:48:10.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.751+0000 7fce7dffb640 1 -- 172.21.15.164:0/2205061698 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce74011d10 con 0x7fce8006ea30 2024-09-19T08:48:10.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.751+0000 7fce7dffb640 1 -- 172.21.15.164:0/2205061698 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7400f2e0 con 0x7fce8006ea30 2024-09-19T08:48:10.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.761+0000 7fce89160740 1 Processor -- start 2024-09-19T08:48:10.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.761+0000 7fce89160740 1 -- start start 2024-09-19T08:48:10.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.762+0000 7fce89160740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556061954670 0x556061954a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:10.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.762+0000 7fce89160740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556061954f90 0x55606195f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:10.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.762+0000 7fce89160740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606195f850 0x556061961c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:10.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.762+0000 7fce89160740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5560618ba2e0 con 0x55606195f850 2024-09-19T08:48:10.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.762+0000 7fce89160740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556061861ce0 con 0x556061954f90 2024-09-19T08:48:10.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.762+0000 7fce89160740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5560618b62a0 con 0x556061954670 2024-09-19T08:48:10.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.763+0000 7fce7dffb640 1 -- 172.21.15.164:0/2205061698 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fce74018020 con 0x7fce8006ea30 2024-09-19T08:48:10.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.764+0000 7fce85324640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556061954670 0x556061954a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:10.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.764+0000 7fce7ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606195f850 0x556061961c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:10.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.764+0000 7fce7ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606195f850 0x556061961c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59216/0 (socket says 172.21.15.164:59216) 2024-09-19T08:48:10.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.764+0000 7fce7ffff640 1 -- 172.21.15.164:0/1128252418 learned_addr learned my addr 172.21.15.164:0/1128252418 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:10.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.766+0000 7fce7ffff640 1 -- 172.21.15.164:0/1128252418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556061954670 msgr2=0x556061954a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:10.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.766+0000 7fce7ffff640 1 --2- 172.21.15.164:0/1128252418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556061954670 0x556061954a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.766+0000 7fce7ffff640 1 -- 172.21.15.164:0/1128252418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556061954f90 msgr2=0x55606195f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:10.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.766+0000 7fce84b23640 1 --2- 172.21.15.164:0/1128252418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556061954f90 0x55606195f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:10.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.766+0000 7fce7ffff640 1 --2- 172.21.15.164:0/1128252418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556061954f90 0x55606195f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.766+0000 7fce7ffff640 1 -- 172.21.15.164:0/1128252418 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556061846440 con 0x55606195f850 2024-09-19T08:48:10.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.766+0000 7fce7dffb640 1 --2- 172.21.15.164:0/2205061698 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fce680a3500 0x7fce680a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:10.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.767+0000 7fce7dffb640 1 -- 172.21.15.164:0/2205061698 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fce740cf2f0 con 0x7fce8006ea30 2024-09-19T08:48:10.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.767+0000 7fce7ffff640 1 --2- 172.21.15.164:0/1128252418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606195f850 0x556061961c40 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fce7000bc20 tx=0x7fce7000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d8feea4f2cbdef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:10.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.767+0000 7fce66ffd640 1 -- 172.21.15.164:0/1128252418 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce70015070 con 0x55606195f850 2024-09-19T08:48:10.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.767+0000 7fce66ffd640 1 -- 172.21.15.164:0/1128252418 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce700035f0 con 0x55606195f850 2024-09-19T08:48:10.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.767+0000 7fce66ffd640 1 -- 172.21.15.164:0/1128252418 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce700116a0 con 0x55606195f850 2024-09-19T08:48:10.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.769+0000 7fce89160740 1 -- 172.21.15.164:0/1128252418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606195f850 msgr2=0x556061961c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:10.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.769+0000 7fce89160740 1 --2- 172.21.15.164:0/1128252418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606195f850 0x556061961c40 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fce7000bc20 tx=0x7fce7000f730 comp rx=0 tx=0).stop 2024-09-19T08:48:10.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.769+0000 7fce85324640 1 --2- 172.21.15.164:0/2205061698 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fce680a3500 0x7fce680a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:10.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.769+0000 7fce89160740 1 -- 172.21.15.164:0/1128252418 shutdown_connections 2024-09-19T08:48:10.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.769+0000 7fce89160740 1 --2- 172.21.15.164:0/1128252418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606195f850 0x556061961c40 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.769+0000 7fce89160740 1 --2- 172.21.15.164:0/1128252418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556061954f90 0x55606195f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.769+0000 7fce89160740 1 --2- 172.21.15.164:0/1128252418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556061954670 0x556061954a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.769+0000 7fce89160740 1 -- 172.21.15.164:0/1128252418 >> 172.21.15.164:0/1128252418 conn(0x556061950c00 msgr2=0x556061950fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:10.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.770+0000 7fce85324640 1 --2- 172.21.15.164:0/2205061698 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fce680a3500 0x7fce680a59c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fce78006010 tx=0x7fce780315f0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:10.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.770+0000 7fce89160740 1 -- 172.21.15.164:0/1128252418 shutdown_connections 2024-09-19T08:48:10.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.771+0000 7fce89160740 1 -- 172.21.15.164:0/1128252418 wait complete. 2024-09-19T08:48:10.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.771+0000 7fce89160740 1 -- start start 2024-09-19T08:48:10.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.772+0000 7fce89160740 1 Processor -- start 2024-09-19T08:48:10.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.772+0000 7fce89160740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556061954670 0x55606195f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:10.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.772+0000 7fce89160740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556061954f90 0x55606195c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:10.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.772+0000 7fce89160740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55606195f850 0x55606195c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:10.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.772+0000 7fce89160740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556061962420 con 0x556061954670 2024-09-19T08:48:10.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.772+0000 7fce89160740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556061875050 con 0x55606195f850 2024-09-19T08:48:10.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.772+0000 7fce89160740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5560619625a0 con 0x556061954f90 2024-09-19T08:48:10.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.773+0000 7fce89160740 10 client.0 _ll_register_callbacks cb 0x5560618b5160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:48:10.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.773+0000 7fce85324640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55606195f850 0x55606195c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:10.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.773+0000 7fce85324640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55606195f850 0x55606195c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:50206/0 (socket says 172.21.15.164:50206) 2024-09-19T08:48:10.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.773+0000 7fce85324640 1 -- 172.21.15.164:0/2714767405 learned_addr learned my addr 172.21.15.164:0/2714767405 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:10.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.774+0000 7fce85324640 1 -- 172.21.15.164:0/2714767405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556061954f90 msgr2=0x55606195c3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:48:10.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.775+0000 7fce85324640 1 --2- 172.21.15.164:0/2714767405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556061954f90 0x55606195c3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.775+0000 7fce85324640 1 -- 172.21.15.164:0/2714767405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556061954670 msgr2=0x55606195f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:10.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.775+0000 7fce85324640 1 --2- 172.21.15.164:0/2714767405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556061954670 0x55606195f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.775+0000 7fce85324640 1 -- 172.21.15.164:0/2714767405 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55606162b3c0 con 0x55606195f850 2024-09-19T08:48:10.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.778+0000 7fce85324640 1 --2- 172.21.15.164:0/2714767405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55606195f850 0x55606195c910 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fce78005de0 tx=0x7fce780095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19075ed181a6f862 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:10.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.778+0000 7fce5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-19T08:48:10.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.781+0000 7fce5ffff640 1 -- 172.21.15.164:0/2714767405 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7804c070 con 0x55606195f850 2024-09-19T08:48:10.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.781+0000 7fce5ffff640 1 -- 172.21.15.164:0/2714767405 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fce780047d0 con 0x55606195f850 2024-09-19T08:48:10.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.781+0000 7fce89160740 1 -- 172.21.15.164:0/2714767405 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce700035f0 con 0x55606195f850 2024-09-19T08:48:10.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.782+0000 7fce5ffff640 1 -- 172.21.15.164:0/2714767405 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fce78056050 con 0x55606195f850 2024-09-19T08:48:10.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.783+0000 7fce89160740 10 client.25556 fetch_fsmap learned FSMap version 28 2024-09-19T08:48:10.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.783+0000 7fce89160740 1 -- 172.21.15.164:0/2714767405 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55606196de30 con 0x55606195f850 2024-09-19T08:48:10.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.786+0000 7fce5ffff640 1 -- 172.21.15.164:0/2714767405 <== mon.2 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce700035f0 con 0x55606195f850 2024-09-19T08:48:10.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.787+0000 7fce5ffff640 1 -- 172.21.15.164:0/2714767405 <== mon.2 v2:172.21.15.170:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fce780488b0 con 0x55606195f850 2024-09-19T08:48:10.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.788+0000 7fce89160740 10 client.25556 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:48:10.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.788+0000 7fce89160740 10 client.25556 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:48:10.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.788+0000 7fce89160740 20 client.25556 populate_metadata read hostname 'smithi164' 2024-09-19T08:48:10.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.788+0000 7fce89160740 1 --2- 172.21.15.164:0/2714767405 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55606196f820 0x556061971ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:10.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.788+0000 7fce89160740 4 client.25556 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:48:10.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.788+0000 7fce89160740 1 -- 172.21.15.164:0/2714767405 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560618c7850 con 0x55606196f820 2024-09-19T08:48:10.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.790+0000 7fce84b23640 1 --2- 172.21.15.164:0/2714767405 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55606196f820 0x556061971ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:10.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.797+0000 7fce84b23640 1 --2- 172.21.15.164:0/2714767405 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55606196f820 0x556061971ce0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11d00f709e0b5bf1 server_cookie=8970108a92de000e in_seq=0 out_seq=0 2024-09-19T08:48:10.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.798+0000 7fce5ffff640 10 client.25556 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:48:10.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.798+0000 7fce5ffff640 1 -- 172.21.15.164:0/2714767405 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5560618c7850 con 0x55606196f820 2024-09-19T08:48:10.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.798+0000 7fce5ffff640 10 client.25556 handle_command_reply: tid=0 2024-09-19T08:48:10.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.798+0000 7fce89160740 1 client.25556 shutdown 2024-09-19T08:48:10.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.800+0000 7fce89160740 1 -- 172.21.15.164:0/2714767405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55606195f850 msgr2=0x55606195c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:10.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.800+0000 7fce89160740 1 --2- 172.21.15.164:0/2714767405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55606195f850 0x55606195c910 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fce78005de0 tx=0x7fce780095c0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.802+0000 7fce89160740 1 -- 172.21.15.164:0/2714767405 shutdown_connections 2024-09-19T08:48:10.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.802+0000 7fce89160740 1 --2- 172.21.15.164:0/2714767405 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55606196f820 0x556061971ce0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.802+0000 7fce89160740 1 --2- 172.21.15.164:0/2714767405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55606195f850 0x55606195c910 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.802+0000 7fce89160740 1 --2- 172.21.15.164:0/2714767405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556061954f90 0x55606195c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.802+0000 7fce89160740 1 --2- 172.21.15.164:0/2714767405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556061954670 0x55606195f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.802+0000 7fce89160740 1 -- 172.21.15.164:0/2714767405 >> 172.21.15.164:0/2714767405 conn(0x556061950c00 msgr2=0x5560619568e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:10.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.802+0000 7fce5ffff640 0 client.25556 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:48:10.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.806+0000 7fce89160740 1 -- 172.21.15.164:0/2714767405 shutdown_connections 2024-09-19T08:48:10.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.807+0000 7fce89160740 1 -- 172.21.15.164:0/2714767405 wait complete. 2024-09-19T08:48:10.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.807+0000 7fce89160740 20 client.25556 trim_cache size 0 max 16384 2024-09-19T08:48:10.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.843+0000 7fce89160740 1 Processor -- start 2024-09-19T08:48:10.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.843+0000 7fce89160740 1 -- start start 2024-09-19T08:48:10.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.843+0000 7fce89160740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556061954670 0x55606196b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:10.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.843+0000 7fce89160740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556061954f90 0x55606194bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:10.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.843+0000 7fce89160740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606195f850 0x55606194fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:10.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.843+0000 7fce89160740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c00e6e0 con 0x55606195f850 2024-09-19T08:48:10.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.843+0000 7fce89160740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c010220 con 0x556061954f90 2024-09-19T08:48:10.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.843+0000 7fce89160740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c00edb0 con 0x556061954670 2024-09-19T08:48:10.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.844+0000 7fce85324640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606195f850 0x55606194fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:10.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.844+0000 7fce85324640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606195f850 0x55606194fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59228/0 (socket says 172.21.15.164:59228) 2024-09-19T08:48:10.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.844+0000 7fce85324640 1 -- 172.21.15.164:0/2637944468 learned_addr learned my addr 172.21.15.164:0/2637944468 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:10.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.844+0000 7fce85324640 1 -- 172.21.15.164:0/2637944468 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556061954670 msgr2=0x55606196b710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:48:10.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.844+0000 7fce85324640 1 --2- 172.21.15.164:0/2637944468 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556061954670 0x55606196b710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.844+0000 7fce85324640 1 -- 172.21.15.164:0/2637944468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556061954f90 msgr2=0x55606194bb90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:48:10.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.844+0000 7fce85324640 1 --2- 172.21.15.164:0/2637944468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556061954f90 0x55606194bb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.844+0000 7fce85324640 1 -- 172.21.15.164:0/2637944468 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55606195b380 con 0x55606195f850 2024-09-19T08:48:10.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.844+0000 7fce85324640 1 --2- 172.21.15.164:0/2637944468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606195f850 0x55606194fc10 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fce78048cd0 tx=0x7fce78031e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=189fb9995a258dc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:10.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.845+0000 7fce667fc640 1 -- 172.21.15.164:0/2637944468 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7804c070 con 0x55606195f850 2024-09-19T08:48:10.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.845+0000 7fce667fc640 1 -- 172.21.15.164:0/2637944468 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce78036040 con 0x55606195f850 2024-09-19T08:48:10.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.845+0000 7fce667fc640 1 -- 172.21.15.164:0/2637944468 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7803bb20 con 0x55606195f850 2024-09-19T08:48:10.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.847+0000 7fce89160740 1 -- 172.21.15.164:0/2637944468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606195f850 msgr2=0x55606194fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:10.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.847+0000 7fce89160740 1 --2- 172.21.15.164:0/2637944468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606195f850 0x55606194fc10 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fce78048cd0 tx=0x7fce78031e10 comp rx=0 tx=0).stop 2024-09-19T08:48:10.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.847+0000 7fce89160740 1 -- 172.21.15.164:0/2637944468 shutdown_connections 2024-09-19T08:48:10.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.847+0000 7fce89160740 1 --2- 172.21.15.164:0/2637944468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606195f850 0x55606194fc10 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.847+0000 7fce89160740 1 --2- 172.21.15.164:0/2637944468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556061954f90 0x55606194bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.847+0000 7fce89160740 1 --2- 172.21.15.164:0/2637944468 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556061954670 0x55606196b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.847+0000 7fce89160740 1 -- 172.21.15.164:0/2637944468 >> 172.21.15.164:0/2637944468 conn(0x556061950c00 msgr2=0x55606194f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:10.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.847+0000 7fce89160740 1 -- 172.21.15.164:0/2637944468 shutdown_connections 2024-09-19T08:48:10.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.848+0000 7fce89160740 1 -- 172.21.15.164:0/2637944468 wait complete. 2024-09-19T08:48:10.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.849+0000 7fce89160740 1 -- start start 2024-09-19T08:48:10.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.849+0000 7fce89160740 1 Processor -- start 2024-09-19T08:48:10.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.850+0000 7fce89160740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556061954670 0x556061963e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:10.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.850+0000 7fce89160740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556061954f90 0x5560619643b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:10.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.850+0000 7fce89160740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606195f850 0x5560619d3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:10.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.850+0000 7fce89160740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c00fb50 con 0x55606195f850 2024-09-19T08:48:10.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.850+0000 7fce84b23640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556061954670 0x556061963e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:10.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.850+0000 7fce84b23640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556061954670 0x556061963e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:50236/0 (socket says 172.21.15.164:50236) 2024-09-19T08:48:10.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.850+0000 7fce84b23640 1 -- 172.21.15.164:0/3502337412 learned_addr learned my addr 172.21.15.164:0/3502337412 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:10.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.851+0000 7fce89160740 1 -- 172.21.15.164:0/3502337412 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c00f480 con 0x556061954670 2024-09-19T08:48:10.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.851+0000 7fce89160740 1 -- 172.21.15.164:0/3502337412 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c00c6c0 con 0x556061954f90 2024-09-19T08:48:10.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.851+0000 7fce7ffff640 1 --2- 172.21.15.164:0/3502337412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556061954f90 0x5560619643b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:10.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.851+0000 7fce89160740 10 client.0 _ll_register_callbacks cb 0x7fce8006f5a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:48:10.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.851+0000 7fce85324640 1 --2- 172.21.15.164:0/3502337412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606195f850 0x5560619d3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:10.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.852+0000 7fce85324640 1 -- 172.21.15.164:0/3502337412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556061954f90 msgr2=0x5560619643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:10.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.852+0000 7fce85324640 1 --2- 172.21.15.164:0/3502337412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556061954f90 0x5560619643b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.852+0000 7fce85324640 1 -- 172.21.15.164:0/3502337412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556061954670 msgr2=0x556061963e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:10.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.852+0000 7fce85324640 1 --2- 172.21.15.164:0/3502337412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556061954670 0x556061963e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.852+0000 7fce85324640 1 -- 172.21.15.164:0/3502337412 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55606195f490 con 0x55606195f850 2024-09-19T08:48:10.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.853+0000 7fce84b23640 1 --2- 172.21.15.164:0/3502337412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556061954670 0x556061963e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:48:10.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.853+0000 7fce85324640 1 --2- 172.21.15.164:0/3502337412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606195f850 0x5560619d3590 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fce780319a0 tx=0x7fce780044e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d666755a6b8ed2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:10.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.853+0000 7fce7ffff640 1 --2- 172.21.15.164:0/3502337412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556061954f90 0x5560619643b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:48:10.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.853+0000 7fce5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:48:10.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.854+0000 7fce5ffff640 1 -- 172.21.15.164:0/3502337412 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7804c040 con 0x55606195f850 2024-09-19T08:48:10.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.854+0000 7fce89160740 1 -- 172.21.15.164:0/3502337412 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce78036040 con 0x55606195f850 2024-09-19T08:48:10.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.854+0000 7fce5ffff640 1 -- 172.21.15.164:0/3502337412 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fce7803b7a0 con 0x55606195f850 2024-09-19T08:48:10.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.854+0000 7fce5ffff640 1 -- 172.21.15.164:0/3502337412 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fce78074050 con 0x55606195f850 2024-09-19T08:48:10.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.854+0000 7fce89160740 10 client.16086 fetch_fsmap learned FSMap version 28 2024-09-19T08:48:10.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.854+0000 7fce89160740 1 -- 172.21.15.164:0/3502337412 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5560619dad90 con 0x55606195f850 2024-09-19T08:48:10.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.855+0000 7fce5ffff640 1 -- 172.21.15.164:0/3502337412 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce78036040 con 0x55606195f850 2024-09-19T08:48:10.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.855+0000 7fce5ffff640 1 -- 172.21.15.164:0/3502337412 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fce78030c30 con 0x55606195f850 2024-09-19T08:48:10.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.855+0000 7fce89160740 10 client.16086 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:48:10.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.855+0000 7fce89160740 10 client.16086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:48:10.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.855+0000 7fce89160740 20 client.16086 populate_metadata read hostname 'smithi164' 2024-09-19T08:48:10.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.855+0000 7fce89160740 1 --2- 172.21.15.164:0/3502337412 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5560619db390 0x5560619dd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:10.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.855+0000 7fce89160740 4 client.16086 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:48:10.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.856+0000 7fce89160740 1 -- 172.21.15.164:0/3502337412 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5560618c1d60 con 0x5560619db390 2024-09-19T08:48:10.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.858+0000 7fce84b23640 1 --2- 172.21.15.164:0/3502337412 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5560619db390 0x5560619dd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:10.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.860+0000 7fce84b23640 1 --2- 172.21.15.164:0/3502337412 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5560619db390 0x5560619dd780 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9af99ddd07f22364 server_cookie=4878b496374e464b in_seq=0 out_seq=0 2024-09-19T08:48:10.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.860+0000 7fce5ffff640 10 client.16086 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:48:10.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.860+0000 7fce5ffff640 1 -- 172.21.15.164:0/3502337412 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5560618c1d60 con 0x5560619db390 2024-09-19T08:48:10.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.860+0000 7fce5ffff640 10 client.16086 handle_command_reply: tid=0 2024-09-19T08:48:10.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.860+0000 7fce89160740 1 client.16086 shutdown 2024-09-19T08:48:10.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.861+0000 7fce89160740 1 -- 172.21.15.164:0/3502337412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606195f850 msgr2=0x5560619d3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:10.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.861+0000 7fce89160740 1 --2- 172.21.15.164:0/3502337412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606195f850 0x5560619d3590 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fce780319a0 tx=0x7fce780044e0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.861+0000 7fce89160740 1 -- 172.21.15.164:0/3502337412 shutdown_connections 2024-09-19T08:48:10.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.861+0000 7fce89160740 1 --2- 172.21.15.164:0/3502337412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606195f850 0x5560619d3590 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.861+0000 7fce89160740 1 --2- 172.21.15.164:0/3502337412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556061954f90 0x5560619643b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.861+0000 7fce89160740 1 --2- 172.21.15.164:0/3502337412 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5560619db390 0x5560619dd780 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.861+0000 7fce89160740 1 --2- 172.21.15.164:0/3502337412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556061954670 0x556061963e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.861+0000 7fce89160740 1 -- 172.21.15.164:0/3502337412 >> 172.21.15.164:0/3502337412 conn(0x556061950c00 msgr2=0x55606195e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:10.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.861+0000 7fce5ffff640 0 client.16086 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:48:10.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.862+0000 7fce89160740 1 -- 172.21.15.164:0/3502337412 shutdown_connections 2024-09-19T08:48:10.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.862+0000 7fce89160740 1 -- 172.21.15.164:0/3502337412 wait complete. 2024-09-19T08:48:10.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.863+0000 7fce89160740 20 client.16086 trim_cache size 0 max 16384 2024-09-19T08:48:10.863 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:48:10.864 INFO:teuthology.orchestra.run.smithi164.stdout: "status": "no active scrubs running", 2024-09-19T08:48:10.864 INFO:teuthology.orchestra.run.smithi164.stdout: "scrubs": {} 2024-09-19T08:48:10.864 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:48:10.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.865+0000 7fce86d87640 1 -- 172.21.15.164:0/2205061698 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fce680a3500 msgr2=0x7fce680a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:10.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.865+0000 7fce86d87640 1 --2- 172.21.15.164:0/2205061698 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fce680a3500 0x7fce680a59c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fce78006010 tx=0x7fce780315f0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.865+0000 7fce86d87640 1 -- 172.21.15.164:0/2205061698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce8006ea30 msgr2=0x7fce80119640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:10.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.865+0000 7fce86d87640 1 --2- 172.21.15.164:0/2205061698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce8006ea30 0x7fce80119640 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fce7400b3f0 tx=0x7fce7400b8b0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.865+0000 7fce86d87640 1 -- 172.21.15.164:0/2205061698 shutdown_connections 2024-09-19T08:48:10.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.865+0000 7fce86d87640 1 --2- 172.21.15.164:0/2205061698 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fce680a3500 0x7fce680a59c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.865+0000 7fce86d87640 1 --2- 172.21.15.164:0/2205061698 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce8011b140 0x7fce801128b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.865+0000 7fce86d87640 1 --2- 172.21.15.164:0/2205061698 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce80110220 0x7fce80119b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.865+0000 7fce86d87640 1 --2- 172.21.15.164:0/2205061698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce8006ea30 0x7fce80119640 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.865+0000 7fce86d87640 1 -- 172.21.15.164:0/2205061698 >> 172.21.15.164:0/2205061698 conn(0x7fce80073210 msgr2=0x7fce8006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:10.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.865+0000 7fce86d87640 1 -- 172.21.15.164:0/2205061698 shutdown_connections 2024-09-19T08:48:10.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:10.865+0000 7fce86d87640 1 -- 172.21.15.164:0/2205061698 wait complete. 2024-09-19T08:48:12.039 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:11 smithi164 ceph-mon[30928]: pgmap v74: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 360 MiB/s wr, 1.48k op/s 2024-09-19T08:48:12.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:11 smithi170 ceph-mon[35072]: pgmap v74: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 360 MiB/s wr, 1.48k op/s 2024-09-19T08:48:12.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:11 smithi173 ceph-mon[35306]: pgmap v74: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 360 MiB/s wr, 1.48k op/s 2024-09-19T08:48:13.891 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T08:48:13.891 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:48:14.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:13 smithi164 ceph-mon[30928]: pgmap v75: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 231 MiB/s wr, 1.38k op/s 2024-09-19T08:48:14.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:13 smithi170 ceph-mon[35072]: pgmap v75: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 231 MiB/s wr, 1.38k op/s 2024-09-19T08:48:14.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:13 smithi173 ceph-mon[35306]: pgmap v75: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 231 MiB/s wr, 1.38k op/s 2024-09-19T08:48:14.503 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:48:16.035 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:15 smithi164 ceph-mon[30928]: pgmap v76: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 439 MiB/s wr, 3.26k op/s 2024-09-19T08:48:16.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:15 smithi170 ceph-mon[35072]: pgmap v76: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 439 MiB/s wr, 3.26k op/s 2024-09-19T08:48:16.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:15 smithi173 ceph-mon[35306]: pgmap v76: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 439 MiB/s wr, 3.26k op/s 2024-09-19T08:48:17.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.606+0000 7f6702ffb640 1 -- 172.21.15.164:0/891376711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66fc10ea10 msgr2=0x7f66fc115360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:17.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.606+0000 7f6702ffb640 1 --2- 172.21.15.164:0/891376711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66fc10ea10 0x7f66fc115360 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f66f400bd70 tx=0x7f66f402c470 comp rx=0 tx=0).stop 2024-09-19T08:48:17.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.606+0000 7f6702ffb640 1 -- 172.21.15.164:0/891376711 shutdown_connections 2024-09-19T08:48:17.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.606+0000 7f6702ffb640 1 --2- 172.21.15.164:0/891376711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66fc10ea10 0x7f66fc115360 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:17.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.606+0000 7f6702ffb640 1 --2- 172.21.15.164:0/891376711 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66fc109e00 0x7f66fc10e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:17.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.606+0000 7f6702ffb640 1 --2- 172.21.15.164:0/891376711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66fc109450 0x7f66fc109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:17.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.606+0000 7f6702ffb640 1 -- 172.21.15.164:0/891376711 >> 172.21.15.164:0/891376711 conn(0x7f66fc073660 msgr2=0x7f66fc073a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:17.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.617+0000 7f6702ffb640 1 -- 172.21.15.164:0/891376711 shutdown_connections 2024-09-19T08:48:17.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.621+0000 7f6702ffb640 1 -- 172.21.15.164:0/891376711 wait complete. 2024-09-19T08:48:17.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.622+0000 7f6702ffb640 1 Processor -- start 2024-09-19T08:48:17.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.622+0000 7f6702ffb640 1 -- start start 2024-09-19T08:48:17.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.627+0000 7f6702ffb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66fc109450 0x7f66fc1a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:17.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.627+0000 7f6702ffb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66fc109e00 0x7f66fc1a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:17.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.627+0000 7f6702ffb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66fc10ea10 0x7f66fc19e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:17.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.627+0000 7f6702ffb640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc118380 con 0x7f66fc109e00 2024-09-19T08:48:17.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.627+0000 7f6702ffb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc118200 con 0x7f66fc109450 2024-09-19T08:48:17.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.627+0000 7f6702ffb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc118500 con 0x7f66fc10ea10 2024-09-19T08:48:17.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.628+0000 7f6701ff9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66fc109450 0x7f66fc1a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:17.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.628+0000 7f67017f8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66fc109e00 0x7f66fc1a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:17.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.628+0000 7f67017f8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66fc109e00 0x7f66fc1a5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:55608/0 (socket says 172.21.15.164:55608) 2024-09-19T08:48:17.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.628+0000 7f67017f8640 1 -- 172.21.15.164:0/2186375468 learned_addr learned my addr 172.21.15.164:0/2186375468 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:17.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.629+0000 7f67017f8640 1 -- 172.21.15.164:0/2186375468 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66fc10ea10 msgr2=0x7f66fc19e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:17.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.629+0000 7f67017f8640 1 --2- 172.21.15.164:0/2186375468 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66fc10ea10 0x7f66fc19e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:17.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.629+0000 7f67017f8640 1 -- 172.21.15.164:0/2186375468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66fc109450 msgr2=0x7f66fc1a53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:17.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.629+0000 7f67017f8640 1 --2- 172.21.15.164:0/2186375468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66fc109450 0x7f66fc1a53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:17.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.629+0000 7f67017f8640 1 -- 172.21.15.164:0/2186375468 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66fc19ed10 con 0x7f66fc109e00 2024-09-19T08:48:17.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.629+0000 7f67017f8640 1 --2- 172.21.15.164:0/2186375468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66fc109e00 0x7f66fc1a5920 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f66ec0078c0 tx=0x7f66ec007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:17.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.630+0000 7f66f2ffd640 1 -- 172.21.15.164:0/2186375468 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec018020 con 0x7f66fc109e00 2024-09-19T08:48:17.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.630+0000 7f66f2ffd640 1 -- 172.21.15.164:0/2186375468 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66ec014470 con 0x7f66fc109e00 2024-09-19T08:48:17.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.630+0000 7f66f2ffd640 1 -- 172.21.15.164:0/2186375468 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec013630 con 0x7f66fc109e00 2024-09-19T08:48:17.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.631+0000 7f6702ffb640 1 -- 172.21.15.164:0/2186375468 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66fc19f000 con 0x7f66fc109e00 2024-09-19T08:48:17.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.631+0000 7f6702ffb640 1 -- 172.21.15.164:0/2186375468 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66fc1a30e0 con 0x7f66fc109e00 2024-09-19T08:48:17.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.634+0000 7f66f0ff9640 1 -- 172.21.15.164:0/2186375468 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66fc06eef0 con 0x7f66fc109e00 2024-09-19T08:48:17.467 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:48:17.468 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":28,"btime":"2024-09-19T08:45:46:903004+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:45:46.852789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:48:17.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.642+0000 7f66f2ffd640 1 -- 172.21.15.164:0/2186375468 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f66ec01b020 con 0x7f66fc109e00 2024-09-19T08:48:17.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.644+0000 7f66f2ffd640 1 --2- 172.21.15.164:0/2186375468 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f66e80a35c0 0x7f66e80a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:17.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.645+0000 7f6701ff9640 1 --2- 172.21.15.164:0/2186375468 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f66e80a35c0 0x7f66e80a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:17.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.645+0000 7f66f2ffd640 1 -- 172.21.15.164:0/2186375468 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f66ec0d0250 con 0x7f66fc109e00 2024-09-19T08:48:17.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.645+0000 7f6701ff9640 1 --2- 172.21.15.164:0/2186375468 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f66e80a35c0 0x7f66e80a5a80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f66f8005fd0 tx=0x7f66f8005850 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:17.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:16.649+0000 7f66f2ffd640 1 -- 172.21.15.164:0/2186375468 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f66ec022020 con 0x7f66fc109e00 2024-09-19T08:48:17.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:17.009+0000 7f66f0ff9640 1 -- 172.21.15.164:0/2186375468 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66fc19f9a0 con 0x7f66fc109e00 2024-09-19T08:48:17.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:17.012+0000 7f66f2ffd640 1 -- 172.21.15.164:0/2186375468 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11984 (secure 0 0 0) 0x7f66ec09ccc0 con 0x7f66fc109e00 2024-09-19T08:48:17.469 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 28 2024-09-19T08:48:17.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:17.022+0000 7f66f0ff9640 1 -- 172.21.15.164:0/2186375468 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f66e80a35c0 msgr2=0x7f66e80a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:17.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:17.022+0000 7f66f0ff9640 1 --2- 172.21.15.164:0/2186375468 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f66e80a35c0 0x7f66e80a5a80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f66f8005fd0 tx=0x7f66f8005850 comp rx=0 tx=0).stop 2024-09-19T08:48:17.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:17.022+0000 7f66f0ff9640 1 -- 172.21.15.164:0/2186375468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66fc109e00 msgr2=0x7f66fc1a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:17.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:17.022+0000 7f66f0ff9640 1 --2- 172.21.15.164:0/2186375468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66fc109e00 0x7f66fc1a5920 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f66ec0078c0 tx=0x7f66ec007d80 comp rx=0 tx=0).stop 2024-09-19T08:48:17.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:17.025+0000 7f66f0ff9640 1 -- 172.21.15.164:0/2186375468 shutdown_connections 2024-09-19T08:48:17.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:17.025+0000 7f66f0ff9640 1 --2- 172.21.15.164:0/2186375468 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66fc10ea10 0x7f66fc19e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:17.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:17.025+0000 7f66f0ff9640 1 --2- 172.21.15.164:0/2186375468 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f66e80a35c0 0x7f66e80a5a80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:17.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:17.025+0000 7f66f0ff9640 1 --2- 172.21.15.164:0/2186375468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66fc109e00 0x7f66fc1a5920 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:17.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:17.025+0000 7f66f0ff9640 1 --2- 172.21.15.164:0/2186375468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66fc109450 0x7f66fc1a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:17.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:17.025+0000 7f66f0ff9640 1 -- 172.21.15.164:0/2186375468 >> 172.21.15.164:0/2186375468 conn(0x7f66fc073660 msgr2=0x7f66fc06d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:17.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:17.027+0000 7f66f0ff9640 1 -- 172.21.15.164:0/2186375468 shutdown_connections 2024-09-19T08:48:17.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:17.027+0000 7f66f0ff9640 1 -- 172.21.15.164:0/2186375468 wait complete. 2024-09-19T08:48:18.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:17 smithi164 ceph-mon[30928]: pgmap v77: 97 pgs: 97 active+clean; 13 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 529 MiB/s wr, 4.23k op/s 2024-09-19T08:48:18.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:17 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2186375468' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:48:18.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:17 smithi170 ceph-mon[35072]: pgmap v77: 97 pgs: 97 active+clean; 13 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 529 MiB/s wr, 4.23k op/s 2024-09-19T08:48:18.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:17 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2186375468' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:48:18.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:17 smithi173 ceph-mon[35306]: pgmap v77: 97 pgs: 97 active+clean; 13 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 529 MiB/s wr, 4.23k op/s 2024-09-19T08:48:18.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:17 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2186375468' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:48:18.279 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T08:48:18.664 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:48:20.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:19 smithi164 ceph-mon[30928]: pgmap v78: 97 pgs: 97 active+clean; 13 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 519 MiB/s wr, 4.22k op/s 2024-09-19T08:48:20.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:19 smithi170 ceph-mon[35072]: pgmap v78: 97 pgs: 97 active+clean; 13 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 519 MiB/s wr, 4.22k op/s 2024-09-19T08:48:20.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:19 smithi173 ceph-mon[35306]: pgmap v78: 97 pgs: 97 active+clean; 13 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 519 MiB/s wr, 4.22k op/s 2024-09-19T08:48:22.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:21 smithi164 ceph-mon[30928]: pgmap v79: 97 pgs: 97 active+clean; 14 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 840 MiB/s wr, 6.57k op/s 2024-09-19T08:48:22.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:21 smithi170 ceph-mon[35072]: pgmap v79: 97 pgs: 97 active+clean; 14 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 840 MiB/s wr, 6.57k op/s 2024-09-19T08:48:22.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:21 smithi173 ceph-mon[35306]: pgmap v79: 97 pgs: 97 active+clean; 14 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 840 MiB/s wr, 6.57k op/s 2024-09-19T08:48:22.475 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:48:22.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.184+0000 7fdf22a11640 1 -- 172.21.15.164:0/3463211204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf1c11b140 msgr2=0x7fdf1c11d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:22.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.184+0000 7fdf22a11640 1 --2- 172.21.15.164:0/3463211204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf1c11b140 0x7fdf1c11d530 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fdf14009ca0 tx=0x7fdf1402a8e0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.184+0000 7fdf22a11640 1 -- 172.21.15.164:0/3463211204 shutdown_connections 2024-09-19T08:48:22.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.184+0000 7fdf22a11640 1 --2- 172.21.15.164:0/3463211204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf1c11b140 0x7fdf1c11d530 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.184+0000 7fdf22a11640 1 --2- 172.21.15.164:0/3463211204 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf1c06ea30 0x7fdf1c06ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.185+0000 7fdf22a11640 1 --2- 172.21.15.164:0/3463211204 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf1c110190 0x7fdf1c06e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.185+0000 7fdf22a11640 1 -- 172.21.15.164:0/3463211204 >> 172.21.15.164:0/3463211204 conn(0x7fdf1c073180 msgr2=0x7fdf1c073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:22.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.185+0000 7fdf22a11640 1 -- 172.21.15.164:0/3463211204 shutdown_connections 2024-09-19T08:48:22.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.189+0000 7fdf22a11640 1 -- 172.21.15.164:0/3463211204 wait complete. 2024-09-19T08:48:22.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.190+0000 7fdf22a11640 1 Processor -- start 2024-09-19T08:48:22.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.190+0000 7fdf22a11640 1 -- start start 2024-09-19T08:48:22.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.191+0000 7fdf22a11640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf1c06ea30 0x7fdf1c1a70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:22.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.191+0000 7fdf22a11640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf1c110190 0x7fdf1c1a7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:22.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.191+0000 7fdf22a11640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf1c11b140 0x7fdf1c1ab9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:22.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.191+0000 7fdf22a11640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c120c40 con 0x7fdf1c110190 2024-09-19T08:48:22.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.191+0000 7fdf22a11640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c120ac0 con 0x7fdf1c11b140 2024-09-19T08:48:22.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.191+0000 7fdf22a11640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c120dc0 con 0x7fdf1c06ea30 2024-09-19T08:48:22.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.192+0000 7fdf2120e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf1c110190 0x7fdf1c1a7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:22.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.192+0000 7fdf2120e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf1c110190 0x7fdf1c1a7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:55616/0 (socket says 172.21.15.164:55616) 2024-09-19T08:48:22.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.192+0000 7fdf2120e640 1 -- 172.21.15.164:0/679056680 learned_addr learned my addr 172.21.15.164:0/679056680 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:22.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.192+0000 7fdf21a0f640 1 --2- 172.21.15.164:0/679056680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf1c06ea30 0x7fdf1c1a70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:22.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.193+0000 7fdf22210640 1 --2- 172.21.15.164:0/679056680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf1c11b140 0x7fdf1c1ab9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:22.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.194+0000 7fdf21a0f640 1 -- 172.21.15.164:0/679056680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf1c11b140 msgr2=0x7fdf1c1ab9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:22.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.194+0000 7fdf21a0f640 1 --2- 172.21.15.164:0/679056680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf1c11b140 0x7fdf1c1ab9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.194+0000 7fdf21a0f640 1 -- 172.21.15.164:0/679056680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf1c110190 msgr2=0x7fdf1c1a7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:22.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.194+0000 7fdf21a0f640 1 --2- 172.21.15.164:0/679056680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf1c110190 0x7fdf1c1a7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.194+0000 7fdf21a0f640 1 -- 172.21.15.164:0/679056680 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1c1ac0a0 con 0x7fdf1c06ea30 2024-09-19T08:48:22.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.194+0000 7fdf21a0f640 1 --2- 172.21.15.164:0/679056680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf1c06ea30 0x7fdf1c1a70d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800d6d0 tx=0x7fdf1800db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:22.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.195+0000 7fdf12ffd640 1 -- 172.21.15.164:0/679056680 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf18017070 con 0x7fdf1c06ea30 2024-09-19T08:48:22.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.195+0000 7fdf22a11640 1 -- 172.21.15.164:0/679056680 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf1c1ac390 con 0x7fdf1c06ea30 2024-09-19T08:48:22.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.195+0000 7fdf22a11640 1 -- 172.21.15.164:0/679056680 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf1c110920 con 0x7fdf1c06ea30 2024-09-19T08:48:22.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.195+0000 7fdf12ffd640 1 -- 172.21.15.164:0/679056680 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf18009820 con 0x7fdf1c06ea30 2024-09-19T08:48:22.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.197+0000 7fdf12ffd640 1 -- 172.21.15.164:0/679056680 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf18004bf0 con 0x7fdf1c06ea30 2024-09-19T08:48:22.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.206+0000 7fdf12ffd640 1 -- 172.21.15.164:0/679056680 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fdf180099d0 con 0x7fdf1c06ea30 2024-09-19T08:48:22.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.207+0000 7fdf12ffd640 1 --2- 172.21.15.164:0/679056680 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fdf080a32d0 0x7fdf080a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:22.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.208+0000 7fdf12ffd640 1 -- 172.21.15.164:0/679056680 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fdf180cf7b0 con 0x7fdf1c06ea30 2024-09-19T08:48:22.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.210+0000 7fdf2120e640 1 --2- 172.21.15.164:0/679056680 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fdf080a32d0 0x7fdf080a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:22.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.224+0000 7fdf2604c740 1 Processor -- start 2024-09-19T08:48:22.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.225+0000 7fdf2604c740 1 -- start start 2024-09-19T08:48:22.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.226+0000 7fdf2604c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ba9381cf0 0x563ba9380240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:22.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.227+0000 7fdf2604c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ba9380780 0x563ba9380bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:22.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.227+0000 7fdf2604c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ba938b520 0x563ba938d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:22.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.227+0000 7fdf2604c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563ba92e62f0 con 0x563ba9380780 2024-09-19T08:48:22.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.227+0000 7fdf2604c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563ba928dcf0 con 0x563ba9381cf0 2024-09-19T08:48:22.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.227+0000 7fdf2604c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563ba92e22b0 con 0x563ba938b520 2024-09-19T08:48:22.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.228+0000 7fdf2120e640 1 --2- 172.21.15.164:0/679056680 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fdf080a32d0 0x7fdf080a5790 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c10e690 tx=0x7fdf0c009190 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:22.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.229+0000 7fdf2120e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ba9380780 0x563ba9380bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:22.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.229+0000 7fdf2120e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ba9380780 0x563ba9380bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:55628/0 (socket says 172.21.15.164:55628) 2024-09-19T08:48:22.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.229+0000 7fdf2120e640 1 -- 172.21.15.164:0/175742907 learned_addr learned my addr 172.21.15.164:0/175742907 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:22.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.231+0000 7fdf22210640 1 --2- 172.21.15.164:0/175742907 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ba938b520 0x563ba938d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:22.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.235+0000 7fdf2120e640 1 -- 172.21.15.164:0/175742907 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ba938b520 msgr2=0x563ba938d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:22.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.235+0000 7fdf2120e640 1 --2- 172.21.15.164:0/175742907 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ba938b520 0x563ba938d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.235+0000 7fdf2120e640 1 -- 172.21.15.164:0/175742907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ba9381cf0 msgr2=0x563ba9380240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:48:22.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.235+0000 7fdf2120e640 1 --2- 172.21.15.164:0/175742907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ba9381cf0 0x563ba9380240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.235+0000 7fdf2120e640 1 -- 172.21.15.164:0/175742907 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ba92723a0 con 0x563ba9380780 2024-09-19T08:48:22.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.240+0000 7fdf2120e640 1 --2- 172.21.15.164:0/175742907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ba9380780 0x563ba9380bc0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00ec20 tx=0x7fdf0c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a984e86a0372b8d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:22.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.241+0000 7fdf07fff640 1 -- 172.21.15.164:0/175742907 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c01b040 con 0x563ba9380780 2024-09-19T08:48:22.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.241+0000 7fdf07fff640 1 -- 172.21.15.164:0/175742907 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf0c024070 con 0x563ba9380780 2024-09-19T08:48:22.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.241+0000 7fdf07fff640 1 -- 172.21.15.164:0/175742907 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c01f3f0 con 0x563ba9380780 2024-09-19T08:48:22.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.242+0000 7fdf2604c740 1 -- 172.21.15.164:0/175742907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ba9380780 msgr2=0x563ba9380bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:22.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.242+0000 7fdf2604c740 1 --2- 172.21.15.164:0/175742907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ba9380780 0x563ba9380bc0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00ec20 tx=0x7fdf0c011000 comp rx=0 tx=0).stop 2024-09-19T08:48:22.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.242+0000 7fdf2604c740 1 -- 172.21.15.164:0/175742907 shutdown_connections 2024-09-19T08:48:22.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.242+0000 7fdf2604c740 1 --2- 172.21.15.164:0/175742907 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ba938b520 0x563ba938d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.242+0000 7fdf2604c740 1 --2- 172.21.15.164:0/175742907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ba9380780 0x563ba9380bc0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.242+0000 7fdf2604c740 1 --2- 172.21.15.164:0/175742907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ba9381cf0 0x563ba9380240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.242+0000 7fdf2604c740 1 -- 172.21.15.164:0/175742907 >> 172.21.15.164:0/175742907 conn(0x563ba9349310 msgr2=0x563ba937ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:22.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.247+0000 7fdf2604c740 1 -- 172.21.15.164:0/175742907 shutdown_connections 2024-09-19T08:48:22.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.247+0000 7fdf2604c740 1 -- 172.21.15.164:0/175742907 wait complete. 2024-09-19T08:48:22.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.248+0000 7fdf2604c740 1 -- start start 2024-09-19T08:48:22.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.248+0000 7fdf2604c740 1 Processor -- start 2024-09-19T08:48:22.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.249+0000 7fdf2604c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ba9381cf0 0x563ba9388450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:22.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.249+0000 7fdf2604c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ba938b520 0x563ba9388990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:22.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.249+0000 7fdf2604c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ba9388ed0 0x563ba9397730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:22.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.249+0000 7fdf2604c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563ba938e220 con 0x563ba9388ed0 2024-09-19T08:48:22.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.249+0000 7fdf2604c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563ba92a1060 con 0x563ba938b520 2024-09-19T08:48:22.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.249+0000 7fdf2604c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563ba938e3a0 con 0x563ba9381cf0 2024-09-19T08:48:22.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.249+0000 7fdf2604c740 10 client.0 _ll_register_callbacks cb 0x563ba8ff48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:48:22.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.250+0000 7fdf22210640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ba9388ed0 0x563ba9397730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:22.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.250+0000 7fdf22210640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ba9388ed0 0x563ba9397730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:55638/0 (socket says 172.21.15.164:55638) 2024-09-19T08:48:22.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.250+0000 7fdf22210640 1 -- 172.21.15.164:0/716389450 learned_addr learned my addr 172.21.15.164:0/716389450 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:22.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.251+0000 7fdf22210640 1 -- 172.21.15.164:0/716389450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ba9381cf0 msgr2=0x563ba9388450 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:48:22.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.251+0000 7fdf22210640 1 --2- 172.21.15.164:0/716389450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ba9381cf0 0x563ba9388450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.251+0000 7fdf22210640 1 -- 172.21.15.164:0/716389450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ba938b520 msgr2=0x563ba9388990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:48:22.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.251+0000 7fdf22210640 1 --2- 172.21.15.164:0/716389450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ba938b520 0x563ba9388990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.251+0000 7fdf22210640 1 -- 172.21.15.164:0/716389450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ba90573c0 con 0x563ba9388ed0 2024-09-19T08:48:22.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.251+0000 7fdf22210640 1 --2- 172.21.15.164:0/716389450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ba9388ed0 0x563ba9397730 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fdf14009ca0 tx=0x7fdf14031c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caadab0374b999e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:22.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.251+0000 7fdf057fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:48:22.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.252+0000 7fdf057fa640 1 -- 172.21.15.164:0/716389450 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf14003970 con 0x563ba9388ed0 2024-09-19T08:48:22.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.252+0000 7fdf2604c740 1 -- 172.21.15.164:0/716389450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf0c024070 con 0x563ba9388ed0 2024-09-19T08:48:22.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.252+0000 7fdf057fa640 1 -- 172.21.15.164:0/716389450 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fdf14003b10 con 0x563ba9388ed0 2024-09-19T08:48:22.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.252+0000 7fdf057fa640 1 -- 172.21.15.164:0/716389450 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdf14003d10 con 0x563ba9388ed0 2024-09-19T08:48:22.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.252+0000 7fdf2604c740 10 client.16104 fetch_fsmap learned FSMap version 28 2024-09-19T08:48:22.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.252+0000 7fdf2604c740 1 -- 172.21.15.164:0/716389450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563ba939a3c0 con 0x563ba9388ed0 2024-09-19T08:48:22.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.253+0000 7fdf057fa640 1 -- 172.21.15.164:0/716389450 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf0c024070 con 0x563ba9388ed0 2024-09-19T08:48:22.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.255+0000 7fdf057fa640 1 -- 172.21.15.164:0/716389450 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdf1404e020 con 0x563ba9388ed0 2024-09-19T08:48:22.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.255+0000 7fdf2604c740 10 client.16104 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:48:22.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.255+0000 7fdf2604c740 10 client.16104 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-19T08:48:22.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.255+0000 7fdf2604c740 20 client.16104 populate_metadata read hostname 'smithi164' 2024-09-19T08:48:22.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.255+0000 7fdf2604c740 1 --2- 172.21.15.164:0/716389450 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x563ba939bc50 0x563ba939e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:22.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.255+0000 7fdf2604c740 4 client.16104 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:48:22.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.255+0000 7fdf2604c740 1 -- 172.21.15.164:0/716389450 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ba92f3860 con 0x563ba939bc50 2024-09-19T08:48:22.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.256+0000 7fdf21a0f640 1 --2- 172.21.15.164:0/716389450 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x563ba939bc50 0x563ba939e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:22.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.263+0000 7fdf21a0f640 1 --2- 172.21.15.164:0/716389450 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x563ba939bc50 0x563ba939e0f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=30ea0f12695f8852 server_cookie=e8ceb8da145cbdf4 in_seq=0 out_seq=0 2024-09-19T08:48:22.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.263+0000 7fdf057fa640 10 client.16104 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:48:22.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.265+0000 7fdf057fa640 1 -- 172.21.15.164:0/716389450 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563ba92f3860 con 0x563ba939bc50 2024-09-19T08:48:22.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.265+0000 7fdf057fa640 10 client.16104 handle_command_reply: tid=0 2024-09-19T08:48:22.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.265+0000 7fdf2604c740 1 client.16104 shutdown 2024-09-19T08:48:22.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.266+0000 7fdf2604c740 1 -- 172.21.15.164:0/716389450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ba9388ed0 msgr2=0x563ba9397730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:22.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.266+0000 7fdf2604c740 1 --2- 172.21.15.164:0/716389450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ba9388ed0 0x563ba9397730 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fdf14009ca0 tx=0x7fdf14031c80 comp rx=0 tx=0).stop 2024-09-19T08:48:22.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.267+0000 7fdf2604c740 1 -- 172.21.15.164:0/716389450 shutdown_connections 2024-09-19T08:48:22.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.267+0000 7fdf2604c740 1 --2- 172.21.15.164:0/716389450 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x563ba939bc50 0x563ba939e0f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.267+0000 7fdf2604c740 1 --2- 172.21.15.164:0/716389450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ba9388ed0 0x563ba9397730 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.267+0000 7fdf2604c740 1 --2- 172.21.15.164:0/716389450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ba938b520 0x563ba9388990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.267+0000 7fdf2604c740 1 --2- 172.21.15.164:0/716389450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ba9381cf0 0x563ba9388450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.267+0000 7fdf2604c740 1 -- 172.21.15.164:0/716389450 >> 172.21.15.164:0/716389450 conn(0x563ba9349310 msgr2=0x563ba9382ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:22.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.267+0000 7fdf057fa640 0 client.16104 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:48:22.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.268+0000 7fdf2604c740 1 -- 172.21.15.164:0/716389450 shutdown_connections 2024-09-19T08:48:22.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.268+0000 7fdf2604c740 1 -- 172.21.15.164:0/716389450 wait complete. 2024-09-19T08:48:22.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.268+0000 7fdf2604c740 20 client.16104 trim_cache size 0 max 16384 2024-09-19T08:48:22.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.297+0000 7fdf2604c740 1 Processor -- start 2024-09-19T08:48:22.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.300+0000 7fdf2604c740 1 -- start start 2024-09-19T08:48:22.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.300+0000 7fdf2604c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ba9349310 0x563ba9383bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:22.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.300+0000 7fdf2604c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ba9381cf0 0x563ba9397520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:22.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.300+0000 7fdf2604c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ba9388ed0 0x563ba93a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:22.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.300+0000 7fdf2604c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0000e590 con 0x563ba9388ed0 2024-09-19T08:48:22.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.300+0000 7fdf2604c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdf000100d0 con 0x563ba9381cf0 2024-09-19T08:48:22.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.300+0000 7fdf2604c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0000ec60 con 0x563ba9349310 2024-09-19T08:48:22.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.300+0000 7fdf22210640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ba9388ed0 0x563ba93a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:22.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.300+0000 7fdf22210640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ba9388ed0 0x563ba93a39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:55646/0 (socket says 172.21.15.164:55646) 2024-09-19T08:48:22.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.300+0000 7fdf22210640 1 -- 172.21.15.164:0/1301156344 learned_addr learned my addr 172.21.15.164:0/1301156344 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:22.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.301+0000 7fdf2120e640 1 --2- 172.21.15.164:0/1301156344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ba9381cf0 0x563ba9397520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:22.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.301+0000 7fdf21a0f640 1 --2- 172.21.15.164:0/1301156344 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ba9349310 0x563ba9383bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:22.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.301+0000 7fdf22210640 1 -- 172.21.15.164:0/1301156344 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ba9349310 msgr2=0x563ba9383bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:22.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.301+0000 7fdf22210640 1 --2- 172.21.15.164:0/1301156344 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ba9349310 0x563ba9383bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.301+0000 7fdf22210640 1 -- 172.21.15.164:0/1301156344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ba9381cf0 msgr2=0x563ba9397520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:22.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.301+0000 7fdf22210640 1 --2- 172.21.15.164:0/1301156344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ba9381cf0 0x563ba9397520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.301+0000 7fdf22210640 1 -- 172.21.15.164:0/1301156344 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ba938adc0 con 0x563ba9388ed0 2024-09-19T08:48:22.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.301+0000 7fdf22210640 1 --2- 172.21.15.164:0/1301156344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ba9388ed0 0x563ba93a39c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fdf14009bf0 tx=0x7fdf1402a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d684d8597d1fc2e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:22.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.303+0000 7fdf077fe640 1 -- 172.21.15.164:0/1301156344 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf14055030 con 0x563ba9388ed0 2024-09-19T08:48:22.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.303+0000 7fdf077fe640 1 -- 172.21.15.164:0/1301156344 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf1403f030 con 0x563ba9388ed0 2024-09-19T08:48:22.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.303+0000 7fdf077fe640 1 -- 172.21.15.164:0/1301156344 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf14050980 con 0x563ba9388ed0 2024-09-19T08:48:22.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.304+0000 7fdf2604c740 1 -- 172.21.15.164:0/1301156344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ba9388ed0 msgr2=0x563ba93a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:22.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.304+0000 7fdf2604c740 1 --2- 172.21.15.164:0/1301156344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ba9388ed0 0x563ba93a39c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fdf14009bf0 tx=0x7fdf1402a8e0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.305+0000 7fdf2604c740 1 -- 172.21.15.164:0/1301156344 shutdown_connections 2024-09-19T08:48:22.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.305+0000 7fdf2604c740 1 --2- 172.21.15.164:0/1301156344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ba9388ed0 0x563ba93a39c0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.305+0000 7fdf2604c740 1 --2- 172.21.15.164:0/1301156344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ba9381cf0 0x563ba9397520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.305+0000 7fdf2604c740 1 --2- 172.21.15.164:0/1301156344 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ba9349310 0x563ba9383bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.305+0000 7fdf2604c740 1 -- 172.21.15.164:0/1301156344 >> 172.21.15.164:0/1301156344 conn(0x563ba9388af0 msgr2=0x563ba93834c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:22.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.305+0000 7fdf2604c740 1 -- 172.21.15.164:0/1301156344 shutdown_connections 2024-09-19T08:48:22.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.305+0000 7fdf2604c740 1 -- 172.21.15.164:0/1301156344 wait complete. 2024-09-19T08:48:22.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.306+0000 7fdf2604c740 1 -- start start 2024-09-19T08:48:22.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.306+0000 7fdf2604c740 1 Processor -- start 2024-09-19T08:48:22.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.307+0000 7fdf2604c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ba9349310 0x563ba938bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:22.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.307+0000 7fdf2604c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ba9381cf0 0x563ba938c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:22.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.307+0000 7fdf2604c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ba9388ed0 0x563ba9392690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:22.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.307+0000 7fdf2604c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0000fa00 con 0x563ba9349310 2024-09-19T08:48:22.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.307+0000 7fdf2604c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0000f330 con 0x563ba9388ed0 2024-09-19T08:48:22.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.307+0000 7fdf2604c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0000c620 con 0x563ba9381cf0 2024-09-19T08:48:22.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.307+0000 7fdf2604c740 10 client.0 _ll_register_callbacks cb 0x7fdf140099a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:48:22.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.308+0000 7fdf2120e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ba9381cf0 0x563ba938c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:22.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.309+0000 7fdf22210640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ba9388ed0 0x563ba9392690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:22.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.309+0000 7fdf22210640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ba9388ed0 0x563ba9392690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:39040/0 (socket says 172.21.15.164:39040) 2024-09-19T08:48:22.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.309+0000 7fdf22210640 1 -- 172.21.15.164:0/1287541 learned_addr learned my addr 172.21.15.164:0/1287541 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:22.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.309+0000 7fdf21a0f640 1 --2- 172.21.15.164:0/1287541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ba9349310 0x563ba938bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:22.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.310+0000 7fdf22210640 1 -- 172.21.15.164:0/1287541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ba9381cf0 msgr2=0x563ba938c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:22.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.310+0000 7fdf22210640 1 --2- 172.21.15.164:0/1287541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ba9381cf0 0x563ba938c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.310+0000 7fdf22210640 1 -- 172.21.15.164:0/1287541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ba9349310 msgr2=0x563ba938bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:22.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.310+0000 7fdf22210640 1 --2- 172.21.15.164:0/1287541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ba9349310 0x563ba938bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.310+0000 7fdf22210640 1 -- 172.21.15.164:0/1287541 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ba9392bd0 con 0x563ba9388ed0 2024-09-19T08:48:22.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.310+0000 7fdf22210640 1 --2- 172.21.15.164:0/1287541 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ba9388ed0 0x563ba9392690 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdf140319a0 tx=0x7fdf140505d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f390f081931cec4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:22.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.310+0000 7fdf057fa640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-19T08:48:22.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.311+0000 7fdf057fa640 1 -- 172.21.15.164:0/1287541 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1405f060 con 0x563ba9388ed0 2024-09-19T08:48:22.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.311+0000 7fdf057fa640 1 -- 172.21.15.164:0/1287541 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fdf14050b10 con 0x563ba9388ed0 2024-09-19T08:48:22.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.313+0000 7fdf2604c740 1 -- 172.21.15.164:0/1287541 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf1403f030 con 0x563ba9388ed0 2024-09-19T08:48:22.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.314+0000 7fdf057fa640 1 -- 172.21.15.164:0/1287541 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdf14050d10 con 0x563ba9388ed0 2024-09-19T08:48:22.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.314+0000 7fdf2604c740 10 client.25568 fetch_fsmap learned FSMap version 28 2024-09-19T08:48:22.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.314+0000 7fdf2604c740 1 -- 172.21.15.164:0/1287541 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563ba9393e30 con 0x563ba9388ed0 2024-09-19T08:48:22.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.315+0000 7fdf057fa640 1 -- 172.21.15.164:0/1287541 <== mon.2 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf1403f030 con 0x563ba9388ed0 2024-09-19T08:48:22.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.315+0000 7fdf057fa640 1 -- 172.21.15.164:0/1287541 <== mon.2 v2:172.21.15.170:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdf1404e040 con 0x563ba9388ed0 2024-09-19T08:48:22.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.316+0000 7fdf2604c740 10 client.25568 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:48:22.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.316+0000 7fdf2604c740 10 client.25568 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-19T08:48:22.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.316+0000 7fdf2604c740 20 client.25568 populate_metadata read hostname 'smithi164' 2024-09-19T08:48:22.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.316+0000 7fdf2604c740 1 --2- 172.21.15.164:0/1287541 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x563ba938b520 0x563ba94089a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:22.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.316+0000 7fdf2604c740 4 client.25568 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:48:22.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.316+0000 7fdf2604c740 1 -- 172.21.15.164:0/1287541 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563ba92edd70 con 0x563ba938b520 2024-09-19T08:48:22.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.318+0000 7fdf21a0f640 1 --2- 172.21.15.164:0/1287541 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x563ba938b520 0x563ba94089a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:22.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.325+0000 7fdf21a0f640 1 --2- 172.21.15.164:0/1287541 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x563ba938b520 0x563ba94089a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f46e1d7e00efcd48 server_cookie=a31e7c04404899aa in_seq=0 out_seq=0 2024-09-19T08:48:22.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.325+0000 7fdf057fa640 10 client.25568 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:48:22.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.326+0000 7fdf057fa640 1 -- 172.21.15.164:0/1287541 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563ba92edd70 con 0x563ba938b520 2024-09-19T08:48:22.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.326+0000 7fdf057fa640 10 client.25568 handle_command_reply: tid=0 2024-09-19T08:48:22.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.326+0000 7fdf2604c740 1 client.25568 shutdown 2024-09-19T08:48:22.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.328+0000 7fdf2604c740 1 -- 172.21.15.164:0/1287541 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ba9388ed0 msgr2=0x563ba9392690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:22.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.328+0000 7fdf2604c740 1 --2- 172.21.15.164:0/1287541 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ba9388ed0 0x563ba9392690 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdf140319a0 tx=0x7fdf140505d0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.329+0000 7fdf2604c740 1 -- 172.21.15.164:0/1287541 shutdown_connections 2024-09-19T08:48:22.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.329+0000 7fdf2604c740 1 --2- 172.21.15.164:0/1287541 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x563ba938b520 0x563ba94089a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.329+0000 7fdf2604c740 1 --2- 172.21.15.164:0/1287541 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ba9388ed0 0x563ba9392690 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.329+0000 7fdf2604c740 1 --2- 172.21.15.164:0/1287541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ba9381cf0 0x563ba938c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.329+0000 7fdf2604c740 1 --2- 172.21.15.164:0/1287541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ba9349310 0x563ba938bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.329+0000 7fdf2604c740 1 -- 172.21.15.164:0/1287541 >> 172.21.15.164:0/1287541 conn(0x563ba9388af0 msgr2=0x563ba9386d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:22.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.329+0000 7fdf057fa640 0 client.25568 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:48:22.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.329+0000 7fdf2604c740 1 -- 172.21.15.164:0/1287541 shutdown_connections 2024-09-19T08:48:22.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.329+0000 7fdf2604c740 1 -- 172.21.15.164:0/1287541 wait complete. 2024-09-19T08:48:22.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.330+0000 7fdf2604c740 20 client.25568 trim_cache size 0 max 16384 2024-09-19T08:48:22.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.334+0000 7fdf22a11640 1 -- 172.21.15.164:0/679056680 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fdf080a32d0 msgr2=0x7fdf080a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:22.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.334+0000 7fdf22a11640 1 --2- 172.21.15.164:0/679056680 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fdf080a32d0 0x7fdf080a5790 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c10e690 tx=0x7fdf0c009190 comp rx=0 tx=0).stop 2024-09-19T08:48:22.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.335+0000 7fdf22a11640 1 -- 172.21.15.164:0/679056680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf1c06ea30 msgr2=0x7fdf1c1a70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:22.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.335+0000 7fdf22a11640 1 --2- 172.21.15.164:0/679056680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf1c06ea30 0x7fdf1c1a70d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800d6d0 tx=0x7fdf1800db90 comp rx=0 tx=0).stop 2024-09-19T08:48:22.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.336+0000 7fdf22a11640 1 -- 172.21.15.164:0/679056680 shutdown_connections 2024-09-19T08:48:22.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.336+0000 7fdf22a11640 1 --2- 172.21.15.164:0/679056680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf1c11b140 0x7fdf1c1ab9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.336+0000 7fdf22a11640 1 --2- 172.21.15.164:0/679056680 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fdf080a32d0 0x7fdf080a5790 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.336+0000 7fdf22a11640 1 --2- 172.21.15.164:0/679056680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf1c110190 0x7fdf1c1a7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.336+0000 7fdf22a11640 1 --2- 172.21.15.164:0/679056680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf1c06ea30 0x7fdf1c1a70d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:22.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.336+0000 7fdf22a11640 1 -- 172.21.15.164:0/679056680 >> 172.21.15.164:0/679056680 conn(0x7fdf1c073180 msgr2=0x7fdf1c119270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:22.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.338+0000 7fdf22a11640 1 -- 172.21.15.164:0/679056680 shutdown_connections 2024-09-19T08:48:22.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:22.338+0000 7fdf22a11640 1 -- 172.21.15.164:0/679056680 wait complete. 2024-09-19T08:48:23.656 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T08:48:23.791 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:23 smithi164 ceph-mon[30928]: pgmap v80: 97 pgs: 97 active+clean; 14 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 641 MiB/s wr, 5.24k op/s 2024-09-19T08:48:24.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:23 smithi170 ceph-mon[35072]: pgmap v80: 97 pgs: 97 active+clean; 14 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 641 MiB/s wr, 5.24k op/s 2024-09-19T08:48:24.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:23 smithi173 ceph-mon[35306]: pgmap v80: 97 pgs: 97 active+clean; 14 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 641 MiB/s wr, 5.24k op/s 2024-09-19T08:48:24.282 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:48:26.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:25 smithi170 ceph-mon[35072]: pgmap v81: 97 pgs: 97 active+clean; 15 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 854 MiB/s wr, 7.16k op/s 2024-09-19T08:48:26.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:25 smithi173 ceph-mon[35306]: pgmap v81: 97 pgs: 97 active+clean; 15 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 854 MiB/s wr, 7.16k op/s 2024-09-19T08:48:26.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:25 smithi164 ceph-mon[30928]: pgmap v81: 97 pgs: 97 active+clean; 15 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 854 MiB/s wr, 7.16k op/s 2024-09-19T08:48:28.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:27 smithi170 ceph-mon[35072]: pgmap v82: 97 pgs: 97 active+clean; 15 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 768 MiB/s wr, 6.43k op/s 2024-09-19T08:48:28.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:27 smithi173 ceph-mon[35306]: pgmap v82: 97 pgs: 97 active+clean; 15 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 768 MiB/s wr, 6.43k op/s 2024-09-19T08:48:28.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:27 smithi164 ceph-mon[30928]: pgmap v82: 97 pgs: 97 active+clean; 15 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 768 MiB/s wr, 6.43k op/s 2024-09-19T08:48:28.852 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:48:28.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.620+0000 7f3d3b685640 1 -- 172.21.15.164:0/1717246034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d34112a30 msgr2=0x7f3d34114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:28.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.620+0000 7f3d3b685640 1 --2- 172.21.15.164:0/1717246034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d34112a30 0x7f3d34114e20 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3d3001a910 tx=0x7f3d3003b430 comp rx=0 tx=0).stop 2024-09-19T08:48:28.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.623+0000 7f3d3b685640 1 -- 172.21.15.164:0/1717246034 shutdown_connections 2024-09-19T08:48:28.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.623+0000 7f3d3b685640 1 --2- 172.21.15.164:0/1717246034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d34112a30 0x7f3d34114e20 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.623+0000 7f3d3b685640 1 --2- 172.21.15.164:0/1717246034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d34077040 0x7f3d341124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.623+0000 7f3d3b685640 1 --2- 172.21.15.164:0/1717246034 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d34076720 0x7f3d34076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.623+0000 7f3d3b685640 1 -- 172.21.15.164:0/1717246034 >> 172.21.15.164:0/1717246034 conn(0x7f3d340ffa90 msgr2=0x7f3d34101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:28.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.624+0000 7f3d3b685640 1 -- 172.21.15.164:0/1717246034 shutdown_connections 2024-09-19T08:48:28.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.627+0000 7f3d3b685640 1 -- 172.21.15.164:0/1717246034 wait complete. 2024-09-19T08:48:28.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.628+0000 7f3d3b685640 1 Processor -- start 2024-09-19T08:48:28.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.629+0000 7f3d3b685640 1 -- start start 2024-09-19T08:48:28.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.629+0000 7f3d3b685640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d34076720 0x7f3d34112150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:28.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.629+0000 7f3d3b685640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d34077040 0x7f3d3410e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:28.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.630+0000 7f3d3b685640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d34112a30 0x7f3d3410e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:28.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.630+0000 7f3d3b685640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3d34106630 con 0x7f3d34112a30 2024-09-19T08:48:28.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.630+0000 7f3d3b685640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3d341064b0 con 0x7f3d34076720 2024-09-19T08:48:28.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.630+0000 7f3d3b685640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3d341067b0 con 0x7f3d34077040 2024-09-19T08:48:28.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.631+0000 7f3d38c20640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d34077040 0x7f3d3410e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:28.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.631+0000 7f3d38c20640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d34077040 0x7f3d3410e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:39348/0 (socket says 172.21.15.164:39348) 2024-09-19T08:48:28.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.631+0000 7f3d38c20640 1 -- 172.21.15.164:0/494433370 learned_addr learned my addr 172.21.15.164:0/494433370 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:28.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.635+0000 7f3d39421640 1 --2- 172.21.15.164:0/494433370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d34076720 0x7f3d34112150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:28.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.635+0000 7f3d38c20640 1 -- 172.21.15.164:0/494433370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d34076720 msgr2=0x7f3d34112150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:28.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.635+0000 7f3d38c20640 1 --2- 172.21.15.164:0/494433370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d34076720 0x7f3d34112150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.635+0000 7f3d38c20640 1 -- 172.21.15.164:0/494433370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d34112a30 msgr2=0x7f3d3410e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:28.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.635+0000 7f3d38c20640 1 --2- 172.21.15.164:0/494433370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d34112a30 0x7f3d3410e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.635+0000 7f3d38c20640 1 -- 172.21.15.164:0/494433370 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d3410ee80 con 0x7f3d34077040 2024-09-19T08:48:28.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.636+0000 7f3d38c20640 1 --2- 172.21.15.164:0/494433370 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d34077040 0x7f3d3410e240 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c00eab0 tx=0x7f3d2c00ef70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:28.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.637+0000 7f3d2a7fc640 1 -- 172.21.15.164:0/494433370 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c00ce40 con 0x7f3d34077040 2024-09-19T08:48:28.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.637+0000 7f3d2a7fc640 1 -- 172.21.15.164:0/494433370 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d2c011950 con 0x7f3d34077040 2024-09-19T08:48:28.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.637+0000 7f3d2a7fc640 1 -- 172.21.15.164:0/494433370 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c010aa0 con 0x7f3d34077040 2024-09-19T08:48:28.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.637+0000 7f3d3b685640 1 -- 172.21.15.164:0/494433370 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d3406bc10 con 0x7f3d34077040 2024-09-19T08:48:28.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.637+0000 7f3d3b685640 1 -- 172.21.15.164:0/494433370 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d3406c100 con 0x7f3d34077040 2024-09-19T08:48:28.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.643+0000 7f3d3da5e740 1 Processor -- start 2024-09-19T08:48:28.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.645+0000 7f3d3da5e740 1 -- start start 2024-09-19T08:48:28.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.646+0000 7f3d3da5e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a35901b40 0x562a35900090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:28.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.646+0000 7f3d3da5e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a359005d0 0x562a35900a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:28.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.646+0000 7f3d3da5e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a3590b370 0x562a3590d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:28.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.646+0000 7f3d3da5e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562a358662f0 con 0x562a359005d0 2024-09-19T08:48:28.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.646+0000 7f3d3da5e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562a3580dcf0 con 0x562a35901b40 2024-09-19T08:48:28.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.646+0000 7f3d3da5e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562a358622b0 con 0x562a3590b370 2024-09-19T08:48:28.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.646+0000 7f3d39c22640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a35901b40 0x562a35900090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:28.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.647+0000 7f3d39c22640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a35901b40 0x562a35900090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:44590/0 (socket says 172.21.15.164:44590) 2024-09-19T08:48:28.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.647+0000 7f3d39c22640 1 -- 172.21.15.164:0/891959451 learned_addr learned my addr 172.21.15.164:0/891959451 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:28.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.647+0000 7f3d38c20640 1 --2- 172.21.15.164:0/891959451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a3590b370 0x562a3590d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:28.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.647+0000 7f3d39c22640 1 -- 172.21.15.164:0/891959451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a3590b370 msgr2=0x562a3590d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:28.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.647+0000 7f3d39c22640 1 --2- 172.21.15.164:0/891959451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a3590b370 0x562a3590d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.647+0000 7f3d39c22640 1 -- 172.21.15.164:0/891959451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a359005d0 msgr2=0x562a35900a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:28.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.647+0000 7f3d39c22640 1 --2- 172.21.15.164:0/891959451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a359005d0 0x562a35900a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.647+0000 7f3d39c22640 1 -- 172.21.15.164:0/891959451 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a357f23a0 con 0x562a35901b40 2024-09-19T08:48:28.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.648+0000 7f3d39c22640 1 --2- 172.21.15.164:0/891959451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a35901b40 0x562a35900090 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3d3003b9a0 tx=0x7f3d30004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=861b48a6ba1288bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:28.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.649+0000 7f3d2a7fc640 1 -- 172.21.15.164:0/494433370 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f3d2c010c40 con 0x7f3d34077040 2024-09-19T08:48:28.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.650+0000 7f3d2a7fc640 1 --2- 172.21.15.164:0/494433370 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3d200a35c0 0x7f3d200a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:28.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.650+0000 7f3d2a7fc640 1 -- 172.21.15.164:0/494433370 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f3d2c0d02e0 con 0x7f3d34077040 2024-09-19T08:48:28.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.653+0000 7f3d1f7fe640 1 -- 172.21.15.164:0/891959451 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3004f020 con 0x562a35901b40 2024-09-19T08:48:28.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.653+0000 7f3d1f7fe640 1 -- 172.21.15.164:0/891959451 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d300048c0 con 0x562a35901b40 2024-09-19T08:48:28.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.655+0000 7f3d3da5e740 1 -- 172.21.15.164:0/891959451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a35901b40 msgr2=0x562a35900090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:28.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.655+0000 7f3d3da5e740 1 --2- 172.21.15.164:0/891959451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a35901b40 0x562a35900090 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3d3003b9a0 tx=0x7f3d30004750 comp rx=0 tx=0).stop 2024-09-19T08:48:28.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.655+0000 7f3d39421640 1 --2- 172.21.15.164:0/494433370 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3d200a35c0 0x7f3d200a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:28.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.660+0000 7f3d39421640 1 --2- 172.21.15.164:0/494433370 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3d200a35c0 0x7f3d200a5a80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x562a358fdac0 tx=0x7f3d240097d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:28.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.661+0000 7f3d3da5e740 1 -- 172.21.15.164:0/891959451 shutdown_connections 2024-09-19T08:48:28.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.661+0000 7f3d3da5e740 1 --2- 172.21.15.164:0/891959451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a3590b370 0x562a3590d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.661+0000 7f3d3da5e740 1 --2- 172.21.15.164:0/891959451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a359005d0 0x562a35900a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.661+0000 7f3d3da5e740 1 --2- 172.21.15.164:0/891959451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a35901b40 0x562a35900090 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.661+0000 7f3d3da5e740 1 -- 172.21.15.164:0/891959451 >> 172.21.15.164:0/891959451 conn(0x562a358c9310 msgr2=0x562a358fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:28.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.662+0000 7f3d3da5e740 1 -- 172.21.15.164:0/891959451 shutdown_connections 2024-09-19T08:48:28.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.662+0000 7f3d3da5e740 1 -- 172.21.15.164:0/891959451 wait complete. 2024-09-19T08:48:28.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.663+0000 7f3d3da5e740 1 -- start start 2024-09-19T08:48:28.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.663+0000 7f3d3da5e740 1 Processor -- start 2024-09-19T08:48:28.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.664+0000 7f3d3da5e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a359005d0 0x562a3590aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:28.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.664+0000 7f3d3da5e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a35901b40 0x562a35907f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:28.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.664+0000 7f3d3da5e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a3590b370 0x562a35908490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:28.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.664+0000 7f3d3da5e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562a3590e060 con 0x562a359005d0 2024-09-19T08:48:28.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.664+0000 7f3d3da5e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562a35821060 con 0x562a3590b370 2024-09-19T08:48:28.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.664+0000 7f3d3da5e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562a3590e1e0 con 0x562a35901b40 2024-09-19T08:48:28.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.664+0000 7f3d3da5e740 10 client.0 _ll_register_callbacks cb 0x562a355748c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:48:28.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.664+0000 7f3d39421640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a359005d0 0x562a3590aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:28.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.664+0000 7f3d39421640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a359005d0 0x562a3590aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:56038/0 (socket says 172.21.15.164:56038) 2024-09-19T08:48:28.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.664+0000 7f3d39421640 1 -- 172.21.15.164:0/50852683 learned_addr learned my addr 172.21.15.164:0/50852683 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:28.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.665+0000 7f3d38c20640 1 --2- 172.21.15.164:0/50852683 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a35901b40 0x562a35907f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:28.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.665+0000 7f3d39c22640 1 --2- 172.21.15.164:0/50852683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a3590b370 0x562a35908490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:28.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.665+0000 7f3d39421640 1 -- 172.21.15.164:0/50852683 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a35901b40 msgr2=0x562a35907f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:28.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.665+0000 7f3d39421640 1 --2- 172.21.15.164:0/50852683 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a35901b40 0x562a35907f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.665+0000 7f3d39421640 1 -- 172.21.15.164:0/50852683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a3590b370 msgr2=0x562a35908490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:28.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.665+0000 7f3d39421640 1 --2- 172.21.15.164:0/50852683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a3590b370 0x562a35908490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.665+0000 7f3d39421640 1 -- 172.21.15.164:0/50852683 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a355d73c0 con 0x562a359005d0 2024-09-19T08:48:28.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.666+0000 7f3d1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:48:28.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.666+0000 7f3d39421640 1 --2- 172.21.15.164:0/50852683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a359005d0 0x562a3590aca0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f3d240128a0 tx=0x7f3d24005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4460148eee4c296c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:28.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.667+0000 7f3d1cff9640 1 -- 172.21.15.164:0/50852683 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2400fce0 con 0x562a359005d0 2024-09-19T08:48:28.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.667+0000 7f3d3da5e740 1 -- 172.21.15.164:0/50852683 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d300048c0 con 0x562a359005d0 2024-09-19T08:48:28.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.667+0000 7f3d1cff9640 1 -- 172.21.15.164:0/50852683 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f3d2400fae0 con 0x562a359005d0 2024-09-19T08:48:28.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.669+0000 7f3d1cff9640 1 -- 172.21.15.164:0/50852683 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3d2400fce0 con 0x562a359005d0 2024-09-19T08:48:28.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.670+0000 7f3d3da5e740 10 client.16116 fetch_fsmap learned FSMap version 28 2024-09-19T08:48:28.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.670+0000 7f3d3da5e740 1 -- 172.21.15.164:0/50852683 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562a35919b50 con 0x562a359005d0 2024-09-19T08:48:28.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.672+0000 7f3d1cff9640 1 -- 172.21.15.164:0/50852683 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d300048c0 con 0x562a359005d0 2024-09-19T08:48:28.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.672+0000 7f3d1cff9640 1 -- 172.21.15.164:0/50852683 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3d2401f420 con 0x562a359005d0 2024-09-19T08:48:28.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.673+0000 7f3d3da5e740 10 client.16116 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:48:28.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.673+0000 7f3d3da5e740 10 client.16116 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T08:48:28.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.673+0000 7f3d3da5e740 20 client.16116 populate_metadata read hostname 'smithi164' 2024-09-19T08:48:28.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.673+0000 7f3d3da5e740 1 --2- 172.21.15.164:0/50852683 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x562a3591b3a0 0x562a3591d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:28.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.673+0000 7f3d3da5e740 4 client.16116 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:48:28.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.673+0000 7f3d3da5e740 1 -- 172.21.15.164:0/50852683 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a35873860 con 0x562a3591b3a0 2024-09-19T08:48:28.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.674+0000 7f3d38c20640 1 --2- 172.21.15.164:0/50852683 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x562a3591b3a0 0x562a3591d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:28.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.677+0000 7f3d1cff9640 10 client.16116 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:48:28.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.677+0000 7f3d38c20640 1 --2- 172.21.15.164:0/50852683 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x562a3591b3a0 0x562a3591d810 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=eeb7ffb3a90c8b63 server_cookie=51eaab0ac35d6cce in_seq=0 out_seq=0 2024-09-19T08:48:28.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.679+0000 7f3d1cff9640 1 -- 172.21.15.164:0/50852683 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562a35873860 con 0x562a3591b3a0 2024-09-19T08:48:28.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.679+0000 7f3d1cff9640 10 client.16116 handle_command_reply: tid=0 2024-09-19T08:48:28.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.679+0000 7f3d3da5e740 1 client.16116 shutdown 2024-09-19T08:48:28.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.680+0000 7f3d3da5e740 1 -- 172.21.15.164:0/50852683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a359005d0 msgr2=0x562a3590aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:28.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.681+0000 7f3d3da5e740 1 --2- 172.21.15.164:0/50852683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a359005d0 0x562a3590aca0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f3d240128a0 tx=0x7f3d24005f90 comp rx=0 tx=0).stop 2024-09-19T08:48:28.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.688+0000 7f3d3da5e740 1 -- 172.21.15.164:0/50852683 shutdown_connections 2024-09-19T08:48:28.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.688+0000 7f3d3da5e740 1 --2- 172.21.15.164:0/50852683 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x562a3591b3a0 0x562a3591d810 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.688+0000 7f3d1cff9640 0 client.16116 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:48:28.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.688+0000 7f3d3da5e740 1 --2- 172.21.15.164:0/50852683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a3590b370 0x562a35908490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.688+0000 7f3d3da5e740 1 --2- 172.21.15.164:0/50852683 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a35901b40 0x562a35907f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.688+0000 7f3d3da5e740 1 --2- 172.21.15.164:0/50852683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a359005d0 0x562a3590aca0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.689+0000 7f3d3da5e740 1 -- 172.21.15.164:0/50852683 >> 172.21.15.164:0/50852683 conn(0x562a358c9310 msgr2=0x562a35902930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:28.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.689+0000 7f3d3da5e740 1 -- 172.21.15.164:0/50852683 shutdown_connections 2024-09-19T08:48:28.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.690+0000 7f3d3da5e740 1 -- 172.21.15.164:0/50852683 wait complete. 2024-09-19T08:48:28.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.690+0000 7f3d3da5e740 20 client.16116 trim_cache size 0 max 16384 2024-09-19T08:48:28.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.739+0000 7f3d3da5e740 1 Processor -- start 2024-09-19T08:48:28.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.754+0000 7f3d3da5e740 1 -- start start 2024-09-19T08:48:28.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.754+0000 7f3d3da5e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a359005d0 0x562a35909ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:28.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.755+0000 7f3d3da5e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a35901b40 0x562a35903120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:28.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.755+0000 7f3d3da5e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3590b370 0x562a35903660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:28.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.755+0000 7f3d3da5e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3d341b3f50 con 0x562a3590b370 2024-09-19T08:48:28.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.755+0000 7f3d3da5e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3d341b5a90 con 0x562a359005d0 2024-09-19T08:48:28.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.755+0000 7f3d3da5e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3d341b4620 con 0x562a35901b40 2024-09-19T08:48:28.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.756+0000 7f3d39421640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a359005d0 0x562a35909ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:28.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.756+0000 7f3d38c20640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a35901b40 0x562a35903120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:28.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.756+0000 7f3d38c20640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a35901b40 0x562a35903120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:39360/0 (socket says 172.21.15.164:39360) 2024-09-19T08:48:28.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.756+0000 7f3d38c20640 1 -- 172.21.15.164:0/739211933 learned_addr learned my addr 172.21.15.164:0/739211933 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:28.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.756+0000 7f3d39c22640 1 --2- 172.21.15.164:0/739211933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3590b370 0x562a35903660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:28.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.757+0000 7f3d38c20640 1 -- 172.21.15.164:0/739211933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a359005d0 msgr2=0x562a35909ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:28.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.757+0000 7f3d38c20640 1 --2- 172.21.15.164:0/739211933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a359005d0 0x562a35909ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.757+0000 7f3d38c20640 1 -- 172.21.15.164:0/739211933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3590b370 msgr2=0x562a35903660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:28.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.757+0000 7f3d38c20640 1 --2- 172.21.15.164:0/739211933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3590b370 0x562a35903660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.757+0000 7f3d38c20640 1 -- 172.21.15.164:0/739211933 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a35903ba0 con 0x562a35901b40 2024-09-19T08:48:28.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.759+0000 7f3d38c20640 1 --2- 172.21.15.164:0/739211933 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a35901b40 0x562a35903120 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c00c700 tx=0x7f3d2c0d27e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e1e5b1c3d6752e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:28.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.761+0000 7f3d1effd640 1 -- 172.21.15.164:0/739211933 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c09c110 con 0x562a35901b40 2024-09-19T08:48:28.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.761+0000 7f3d1effd640 1 -- 172.21.15.164:0/739211933 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562a35873860 con 0x562a35901b40 2024-09-19T08:48:28.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.761+0000 7f3d1effd640 1 -- 172.21.15.164:0/739211933 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c09bf30 con 0x562a35901b40 2024-09-19T08:48:28.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.767+0000 7f3d3da5e740 1 -- 172.21.15.164:0/739211933 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a35901b40 msgr2=0x562a35903120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:28.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.767+0000 7f3d3da5e740 1 --2- 172.21.15.164:0/739211933 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a35901b40 0x562a35903120 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c00c700 tx=0x7f3d2c0d27e0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.768+0000 7f3d3da5e740 1 -- 172.21.15.164:0/739211933 shutdown_connections 2024-09-19T08:48:28.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.768+0000 7f3d3da5e740 1 --2- 172.21.15.164:0/739211933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3590b370 0x562a35903660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.768+0000 7f3d3da5e740 1 --2- 172.21.15.164:0/739211933 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a35901b40 0x562a35903120 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.768+0000 7f3d3da5e740 1 --2- 172.21.15.164:0/739211933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a359005d0 0x562a35909ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.768+0000 7f3d3da5e740 1 -- 172.21.15.164:0/739211933 >> 172.21.15.164:0/739211933 conn(0x562a358c9310 msgr2=0x562a359097b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:28.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.773+0000 7f3d3da5e740 1 -- 172.21.15.164:0/739211933 shutdown_connections 2024-09-19T08:48:28.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.773+0000 7f3d3da5e740 1 -- 172.21.15.164:0/739211933 wait complete. 2024-09-19T08:48:28.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.774+0000 7f3d3da5e740 1 -- start start 2024-09-19T08:48:28.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.775+0000 7f3d3da5e740 1 Processor -- start 2024-09-19T08:48:28.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.776+0000 7f3d3da5e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a359005d0 0x562a35913c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:28.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.777+0000 7f3d3da5e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a35901b40 0x562a35904d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:28.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.777+0000 7f3d39421640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a359005d0 0x562a35913c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:28.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.777+0000 7f3d39421640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a359005d0 0x562a35913c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:44602/0 (socket says 172.21.15.164:44602) 2024-09-19T08:48:28.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.777+0000 7f3d39421640 1 -- 172.21.15.164:0/57663605 learned_addr learned my addr 172.21.15.164:0/57663605 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:28.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.777+0000 7f3d3da5e740 1 --2- 172.21.15.164:0/57663605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3590b370 0x562a359052f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:28.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.778+0000 7f3d3da5e740 1 -- 172.21.15.164:0/57663605 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3d341b53c0 con 0x562a3590b370 2024-09-19T08:48:28.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.778+0000 7f3d3da5e740 1 -- 172.21.15.164:0/57663605 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3d341b4cf0 con 0x562a359005d0 2024-09-19T08:48:28.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.778+0000 7f3d3da5e740 1 -- 172.21.15.164:0/57663605 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3d34072970 con 0x562a35901b40 2024-09-19T08:48:28.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.778+0000 7f3d38c20640 1 --2- 172.21.15.164:0/57663605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a35901b40 0x562a35904d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:28.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.778+0000 7f3d3da5e740 10 client.0 _ll_register_callbacks cb 0x7f3d24009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:48:28.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.782+0000 7f3d39c22640 1 --2- 172.21.15.164:0/57663605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3590b370 0x562a359052f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:28.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.785+0000 7f3d38c20640 1 -- 172.21.15.164:0/57663605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a359005d0 msgr2=0x562a35913c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:28.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.785+0000 7f3d38c20640 1 --2- 172.21.15.164:0/57663605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a359005d0 0x562a35913c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.785+0000 7f3d38c20640 1 -- 172.21.15.164:0/57663605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3590b370 msgr2=0x562a359052f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:28.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.785+0000 7f3d38c20640 1 --2- 172.21.15.164:0/57663605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3590b370 0x562a359052f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.785+0000 7f3d38c20640 1 -- 172.21.15.164:0/57663605 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a35905b20 con 0x562a35901b40 2024-09-19T08:48:28.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.789+0000 7f3d38c20640 1 --2- 172.21.15.164:0/57663605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a35901b40 0x562a35904d50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x562a35901ff0 tx=0x7f3d2c098ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=347fc9a2911f299f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:28.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.790+0000 7f3d1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-09-19T08:48:28.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.790+0000 7f3d1cff9640 1 -- 172.21.15.164:0/57663605 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c09c110 con 0x562a35901b40 2024-09-19T08:48:28.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.790+0000 7f3d3da5e740 1 -- 172.21.15.164:0/57663605 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562a35873860 con 0x562a35901b40 2024-09-19T08:48:28.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.790+0000 7f3d1cff9640 1 -- 172.21.15.164:0/57663605 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f3d2c0202b0 con 0x562a35901b40 2024-09-19T08:48:28.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.791+0000 7f3d1cff9640 1 -- 172.21.15.164:0/57663605 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3d2c0204b0 con 0x562a35901b40 2024-09-19T08:48:28.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.792+0000 7f3d3da5e740 10 client.25561 fetch_fsmap learned FSMap version 28 2024-09-19T08:48:28.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.792+0000 7f3d3da5e740 1 -- 172.21.15.164:0/57663605 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562a35986170 con 0x562a35901b40 2024-09-19T08:48:28.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.795+0000 7f3d1cff9640 1 -- 172.21.15.164:0/57663605 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d2c0ba070 con 0x562a35901b40 2024-09-19T08:48:28.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.799+0000 7f3d1cff9640 1 -- 172.21.15.164:0/57663605 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3d2c0ad020 con 0x562a35901b40 2024-09-19T08:48:28.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.800+0000 7f3d3da5e740 10 client.25561 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:48:28.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.800+0000 7f3d3da5e740 10 client.25561 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T08:48:28.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.800+0000 7f3d3da5e740 20 client.25561 populate_metadata read hostname 'smithi164' 2024-09-19T08:48:28.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.800+0000 7f3d3da5e740 1 --2- 172.21.15.164:0/57663605 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x562a3591b3a0 0x562a35988840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:28.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.800+0000 7f3d3da5e740 4 client.25561 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:48:28.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.800+0000 7f3d3da5e740 1 -- 172.21.15.164:0/57663605 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562a35873860 con 0x562a3591b3a0 2024-09-19T08:48:28.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.809+0000 7f3d39421640 1 --2- 172.21.15.164:0/57663605 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x562a3591b3a0 0x562a35988840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:28.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.822+0000 7f3d39421640 1 --2- 172.21.15.164:0/57663605 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x562a3591b3a0 0x562a35988840 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4787dec6b44748d9 server_cookie=bf084668746ea4d9 in_seq=0 out_seq=0 2024-09-19T08:48:28.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.822+0000 7f3d1cff9640 10 client.25561 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:48:28.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.824+0000 7f3d1cff9640 1 -- 172.21.15.164:0/57663605 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562a35873860 con 0x562a3591b3a0 2024-09-19T08:48:28.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.824+0000 7f3d1cff9640 10 client.25561 handle_command_reply: tid=0 2024-09-19T08:48:28.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.824+0000 7f3d3da5e740 1 client.25561 shutdown 2024-09-19T08:48:28.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.831+0000 7f3d3da5e740 1 -- 172.21.15.164:0/57663605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a35901b40 msgr2=0x562a35904d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:28.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.831+0000 7f3d3da5e740 1 --2- 172.21.15.164:0/57663605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a35901b40 0x562a35904d50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x562a35901ff0 tx=0x7f3d2c098ed0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.837+0000 7f3d3da5e740 1 -- 172.21.15.164:0/57663605 shutdown_connections 2024-09-19T08:48:28.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.837+0000 7f3d3da5e740 1 --2- 172.21.15.164:0/57663605 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x562a3591b3a0 0x562a35988840 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.838+0000 7f3d3da5e740 1 --2- 172.21.15.164:0/57663605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a3590b370 0x562a359052f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.838+0000 7f3d1cff9640 0 client.25561 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:48:28.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.838+0000 7f3d3da5e740 1 --2- 172.21.15.164:0/57663605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a35901b40 0x562a35904d50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.838+0000 7f3d3da5e740 1 --2- 172.21.15.164:0/57663605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a359005d0 0x562a35913c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.838+0000 7f3d3da5e740 1 -- 172.21.15.164:0/57663605 >> 172.21.15.164:0/57663605 conn(0x562a358c9310 msgr2=0x562a358fbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:28.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.838+0000 7f3d3da5e740 1 -- 172.21.15.164:0/57663605 shutdown_connections 2024-09-19T08:48:28.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.839+0000 7f3d3da5e740 1 -- 172.21.15.164:0/57663605 wait complete. 2024-09-19T08:48:28.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.839+0000 7f3d3da5e740 20 client.25561 trim_cache size 0 max 16384 2024-09-19T08:48:28.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.842+0000 7f3d3b685640 1 -- 172.21.15.164:0/494433370 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3d200a35c0 msgr2=0x7f3d200a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:28.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.842+0000 7f3d3b685640 1 --2- 172.21.15.164:0/494433370 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3d200a35c0 0x7f3d200a5a80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x562a358fdac0 tx=0x7f3d240097d0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.842+0000 7f3d3b685640 1 -- 172.21.15.164:0/494433370 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d34077040 msgr2=0x7f3d3410e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:28.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.842+0000 7f3d3b685640 1 --2- 172.21.15.164:0/494433370 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d34077040 0x7f3d3410e240 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c00eab0 tx=0x7f3d2c00ef70 comp rx=0 tx=0).stop 2024-09-19T08:48:28.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.843+0000 7f3d3b685640 1 -- 172.21.15.164:0/494433370 shutdown_connections 2024-09-19T08:48:28.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.843+0000 7f3d3b685640 1 --2- 172.21.15.164:0/494433370 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3d200a35c0 0x7f3d200a5a80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.843+0000 7f3d3b685640 1 --2- 172.21.15.164:0/494433370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d34112a30 0x7f3d3410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.843+0000 7f3d3b685640 1 --2- 172.21.15.164:0/494433370 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d34077040 0x7f3d3410e240 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.843+0000 7f3d3b685640 1 --2- 172.21.15.164:0/494433370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d34076720 0x7f3d34112150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:28.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.843+0000 7f3d3b685640 1 -- 172.21.15.164:0/494433370 >> 172.21.15.164:0/494433370 conn(0x7f3d340ffa90 msgr2=0x7f3d34101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:28.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.843+0000 7f3d3b685640 1 -- 172.21.15.164:0/494433370 shutdown_connections 2024-09-19T08:48:28.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:28.843+0000 7f3d3b685640 1 -- 172.21.15.164:0/494433370 wait complete. 2024-09-19T08:48:30.038 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:29 smithi164 ceph-mon[30928]: pgmap v83: 97 pgs: 97 active+clean; 15 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 656 MiB/s wr, 5.42k op/s 2024-09-19T08:48:30.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:29 smithi170 ceph-mon[35072]: pgmap v83: 97 pgs: 97 active+clean; 15 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 656 MiB/s wr, 5.42k op/s 2024-09-19T08:48:30.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:29 smithi173 ceph-mon[35306]: pgmap v83: 97 pgs: 97 active+clean; 15 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 656 MiB/s wr, 5.42k op/s 2024-09-19T08:48:31.271 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T08:48:32.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:31 smithi170 ceph-mon[35072]: pgmap v84: 97 pgs: 97 active+clean; 16 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 832 MiB/s wr, 6.61k op/s 2024-09-19T08:48:32.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:31 smithi173 ceph-mon[35306]: pgmap v84: 97 pgs: 97 active+clean; 16 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 832 MiB/s wr, 6.61k op/s 2024-09-19T08:48:32.285 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:31 smithi164 ceph-mon[30928]: pgmap v84: 97 pgs: 97 active+clean; 16 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 832 MiB/s wr, 6.61k op/s 2024-09-19T08:48:32.392 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:48:33.977 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:33 smithi164 ceph-mon[30928]: pgmap v85: 97 pgs: 97 active+clean; 17 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 575 MiB/s wr, 4.55k op/s 2024-09-19T08:48:33.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.975+0000 7fefb06d6640 1 -- 172.21.15.164:0/3488210299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefa8108c30 msgr2=0x7fefa810cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:33.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.975+0000 7fefb06d6640 1 --2- 172.21.15.164:0/3488210299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefa8108c30 0x7fefa810cc70 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fef98009900 tx=0x7fef9802a410 comp rx=0 tx=0).stop 2024-09-19T08:48:33.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.975+0000 7fefb06d6640 1 -- 172.21.15.164:0/3488210299 shutdown_connections 2024-09-19T08:48:33.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.975+0000 7fefb06d6640 1 --2- 172.21.15.164:0/3488210299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa810d3a0 0x7fefa8114f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:33.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.975+0000 7fefb06d6640 1 --2- 172.21.15.164:0/3488210299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefa8108c30 0x7fefa810cc70 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:33.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.975+0000 7fefb06d6640 1 --2- 172.21.15.164:0/3488210299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefa8108280 0x7fefa8108660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:33.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.975+0000 7fefb06d6640 1 -- 172.21.15.164:0/3488210299 >> 172.21.15.164:0/3488210299 conn(0x7fefa8077a00 msgr2=0x7fefa8079e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:33.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.975+0000 7fefb06d6640 1 -- 172.21.15.164:0/3488210299 shutdown_connections 2024-09-19T08:48:33.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.975+0000 7fefb06d6640 1 -- 172.21.15.164:0/3488210299 wait complete. 2024-09-19T08:48:33.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.977+0000 7fefb06d6640 1 Processor -- start 2024-09-19T08:48:33.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.977+0000 7fefb06d6640 1 -- start start 2024-09-19T08:48:33.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.978+0000 7fefb06d6640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefa8108280 0x7fefa80827d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:33.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.978+0000 7fefb06d6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa8108c30 0x7fefa8082d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:33.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.978+0000 7fefb06d6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefa810d3a0 0x7fefa807ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:33.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.978+0000 7fefb06d6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fefa81184b0 con 0x7fefa810d3a0 2024-09-19T08:48:33.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.978+0000 7fefb06d6640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fefa8118330 con 0x7fefa8108c30 2024-09-19T08:48:33.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.978+0000 7fefb06d6640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fefa8118630 con 0x7fefa8108280 2024-09-19T08:48:33.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.978+0000 7fefaec73640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefa810d3a0 0x7fefa807ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:33.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.978+0000 7fefaec73640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefa810d3a0 0x7fefa807ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58846/0 (socket says 172.21.15.164:58846) 2024-09-19T08:48:33.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.978+0000 7fefaec73640 1 -- 172.21.15.164:0/286148492 learned_addr learned my addr 172.21.15.164:0/286148492 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:33.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.979+0000 7fefaec73640 1 -- 172.21.15.164:0/286148492 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefa8108280 msgr2=0x7fefa80827d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T08:48:33.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.979+0000 7fefaec73640 1 --2- 172.21.15.164:0/286148492 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefa8108280 0x7fefa80827d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:33.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.979+0000 7fefaec73640 1 -- 172.21.15.164:0/286148492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa8108c30 msgr2=0x7fefa8082d10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T08:48:33.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.979+0000 7fefaec73640 1 --2- 172.21.15.164:0/286148492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa8108c30 0x7fefa8082d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:33.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.979+0000 7fefaec73640 1 -- 172.21.15.164:0/286148492 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa807c300 con 0x7fefa810d3a0 2024-09-19T08:48:33.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.979+0000 7fefaec73640 1 --2- 172.21.15.164:0/286148492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefa810d3a0 0x7fefa807ba40 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fefa400b410 tx=0x7fefa400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:33.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.980+0000 7fef9f7fe640 1 -- 172.21.15.164:0/286148492 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4015020 con 0x7fefa810d3a0 2024-09-19T08:48:33.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.980+0000 7fef9f7fe640 1 -- 172.21.15.164:0/286148492 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa4011d10 con 0x7fefa810d3a0 2024-09-19T08:48:33.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.980+0000 7fef9f7fe640 1 -- 172.21.15.164:0/286148492 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4010e20 con 0x7fefa810d3a0 2024-09-19T08:48:33.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.980+0000 7fefb06d6640 1 -- 172.21.15.164:0/286148492 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa807c5f0 con 0x7fefa810d3a0 2024-09-19T08:48:33.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.980+0000 7fefb06d6640 1 -- 172.21.15.164:0/286148492 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa8080590 con 0x7fefa810d3a0 2024-09-19T08:48:33.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.989+0000 7fef9f7fe640 1 -- 172.21.15.164:0/286148492 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fefa4018020 con 0x7fefa810d3a0 2024-09-19T08:48:33.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.990+0000 7fef9f7fe640 1 --2- 172.21.15.164:0/286148492 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fef940a3580 0x7fef940a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:33.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.991+0000 7fef9f7fe640 1 -- 172.21.15.164:0/286148492 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fefa40d0410 con 0x7fefa810d3a0 2024-09-19T08:48:33.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.991+0000 7fefb2aaf740 1 Processor -- start 2024-09-19T08:48:33.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.991+0000 7fefae472640 1 --2- 172.21.15.164:0/286148492 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fef940a3580 0x7fef940a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:33.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.992+0000 7fefb2aaf740 1 -- start start 2024-09-19T08:48:33.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.992+0000 7fefb2aaf740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639863accf0 0x5639863ab240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:33.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.992+0000 7fefb2aaf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639863ab780 0x5639863abbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:33.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.992+0000 7fefb2aaf740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5639863b6520 0x5639863b8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:33.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.992+0000 7fefb2aaf740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5639863112f0 con 0x5639863accf0 2024-09-19T08:48:33.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.992+0000 7fefb2aaf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5639862b8cf0 con 0x5639863ab780 2024-09-19T08:48:33.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.992+0000 7fefb2aaf740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56398630d2b0 con 0x5639863b6520 2024-09-19T08:48:33.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.993+0000 7fefae472640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639863accf0 0x5639863ab240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:33.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.993+0000 7fefae472640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639863accf0 0x5639863ab240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58848/0 (socket says 172.21.15.164:58848) 2024-09-19T08:48:33.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.993+0000 7fefae472640 1 -- 172.21.15.164:0/1827670257 learned_addr learned my addr 172.21.15.164:0/1827670257 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:33.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.993+0000 7fefadc71640 1 --2- 172.21.15.164:0/1827670257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639863ab780 0x5639863abbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:33.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.993+0000 7fefaec73640 1 --2- 172.21.15.164:0/1827670257 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5639863b6520 0x5639863b8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:33.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.993+0000 7fefae472640 1 --2- 172.21.15.164:0/286148492 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fef940a3580 0x7fef940a5a40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fefa00126c0 tx=0x7fefa0012ac0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:33.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.994+0000 7fefae472640 1 -- 172.21.15.164:0/1827670257 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5639863b6520 msgr2=0x5639863b8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:33.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.994+0000 7fefae472640 1 --2- 172.21.15.164:0/1827670257 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5639863b6520 0x5639863b8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:33.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.994+0000 7fefae472640 1 -- 172.21.15.164:0/1827670257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639863ab780 msgr2=0x5639863abbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:33.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.994+0000 7fefae472640 1 --2- 172.21.15.164:0/1827670257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639863ab780 0x5639863abbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:33.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.994+0000 7fefae472640 1 -- 172.21.15.164:0/1827670257 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56398629d3a0 con 0x5639863accf0 2024-09-19T08:48:33.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.995+0000 7fefae472640 1 --2- 172.21.15.164:0/1827670257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639863accf0 0x5639863ab240 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fefa00109c0 tx=0x7fefa001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8dc967450c4ec95 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:33.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.995+0000 7fef9cff9640 1 -- 172.21.15.164:0/1827670257 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0012030 con 0x5639863accf0 2024-09-19T08:48:33.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.995+0000 7fef9cff9640 1 -- 172.21.15.164:0/1827670257 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa00121d0 con 0x5639863accf0 2024-09-19T08:48:33.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.996+0000 7fef9cff9640 1 -- 172.21.15.164:0/1827670257 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa00053a0 con 0x5639863accf0 2024-09-19T08:48:33.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.996+0000 7fefb2aaf740 1 -- 172.21.15.164:0/1827670257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639863accf0 msgr2=0x5639863ab240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:33.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.996+0000 7fefb2aaf740 1 --2- 172.21.15.164:0/1827670257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639863accf0 0x5639863ab240 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fefa00109c0 tx=0x7fefa001d000 comp rx=0 tx=0).stop 2024-09-19T08:48:33.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.996+0000 7fefb2aaf740 1 -- 172.21.15.164:0/1827670257 shutdown_connections 2024-09-19T08:48:33.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.996+0000 7fefb2aaf740 1 --2- 172.21.15.164:0/1827670257 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5639863b6520 0x5639863b8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:33.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.996+0000 7fefb2aaf740 1 --2- 172.21.15.164:0/1827670257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639863ab780 0x5639863abbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:33.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.996+0000 7fefb2aaf740 1 --2- 172.21.15.164:0/1827670257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639863accf0 0x5639863ab240 secure :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fefa00109c0 tx=0x7fefa001d000 comp rx=0 tx=0).stop 2024-09-19T08:48:33.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.996+0000 7fefb2aaf740 1 -- 172.21.15.164:0/1827670257 >> 172.21.15.164:0/1827670257 conn(0x563986374310 msgr2=0x5639863a7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:33.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.997+0000 7fefb2aaf740 1 -- 172.21.15.164:0/1827670257 shutdown_connections 2024-09-19T08:48:33.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.997+0000 7fefb2aaf740 1 -- 172.21.15.164:0/1827670257 wait complete. 2024-09-19T08:48:33.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.998+0000 7fefb2aaf740 1 -- start start 2024-09-19T08:48:33.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.998+0000 7fefb2aaf740 1 Processor -- start 2024-09-19T08:48:33.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.998+0000 7fefb2aaf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639863ab780 0x5639863b32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:33.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.998+0000 7fefb2aaf740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5639863b6520 0x5639863b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:33.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.998+0000 7fefb2aaf740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639863b3d30 0x5639863d3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:33.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.998+0000 7fefb2aaf740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5639863ca210 con 0x5639863b3d30 2024-09-19T08:48:33.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.998+0000 7fefb2aaf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5639862cc060 con 0x5639863ab780 2024-09-19T08:48:33.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.998+0000 7fefb2aaf740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5639863ca390 con 0x5639863b6520 2024-09-19T08:48:33.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.998+0000 7fefb2aaf740 10 client.0 _ll_register_callbacks cb 0x56398601f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:48:33.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.999+0000 7fefaec73640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639863b3d30 0x5639863d3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:33.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.999+0000 7fefaec73640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639863b3d30 0x5639863d3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58864/0 (socket says 172.21.15.164:58864) 2024-09-19T08:48:33.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:33.999+0000 7fefaec73640 1 -- 172.21.15.164:0/4022520890 learned_addr learned my addr 172.21.15.164:0/4022520890 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:34.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.000+0000 7fefaec73640 1 -- 172.21.15.164:0/4022520890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5639863b6520 msgr2=0x5639863b37f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:48:34.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.000+0000 7fefaec73640 1 --2- 172.21.15.164:0/4022520890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5639863b6520 0x5639863b37f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:34.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.000+0000 7fefaec73640 1 -- 172.21.15.164:0/4022520890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639863ab780 msgr2=0x5639863b32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:34.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.000+0000 7fefaec73640 1 --2- 172.21.15.164:0/4022520890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639863ab780 0x5639863b32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:34.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.000+0000 7fefaec73640 1 -- 172.21.15.164:0/4022520890 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639860823c0 con 0x5639863b3d30 2024-09-19T08:48:34.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.000+0000 7fefaec73640 1 --2- 172.21.15.164:0/4022520890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639863b3d30 0x5639863d3740 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fefa4098c70 tx=0x7fefa409c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=613fc4a63e1175fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:34.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.001+0000 7fef91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:48:34.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.001+0000 7fef91ffb640 1 -- 172.21.15.164:0/4022520890 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa40b4070 con 0x5639863b3d30 2024-09-19T08:48:34.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.001+0000 7fefb2aaf740 1 -- 172.21.15.164:0/4022520890 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefa00121d0 con 0x5639863b3d30 2024-09-19T08:48:34.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.001+0000 7fef91ffb640 1 -- 172.21.15.164:0/4022520890 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fefa409db40 con 0x5639863b3d30 2024-09-19T08:48:34.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.003+0000 7fef91ffb640 1 -- 172.21.15.164:0/4022520890 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fefa40cdff0 con 0x5639863b3d30 2024-09-19T08:48:34.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.003+0000 7fefb2aaf740 10 client.16134 fetch_fsmap learned FSMap version 28 2024-09-19T08:48:34.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.003+0000 7fefb2aaf740 1 -- 172.21.15.164:0/4022520890 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5639863d6250 con 0x5639863b3d30 2024-09-19T08:48:34.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.004+0000 7fef91ffb640 1 -- 172.21.15.164:0/4022520890 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa00121d0 con 0x5639863b3d30 2024-09-19T08:48:34.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.004+0000 7fef91ffb640 1 -- 172.21.15.164:0/4022520890 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fefa4098560 con 0x5639863b3d30 2024-09-19T08:48:34.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.004+0000 7fefb2aaf740 10 client.16134 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:48:34.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.004+0000 7fefb2aaf740 10 client.16134 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T08:48:34.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.004+0000 7fefb2aaf740 20 client.16134 populate_metadata read hostname 'smithi164' 2024-09-19T08:48:34.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.004+0000 7fefb2aaf740 1 --2- 172.21.15.164:0/4022520890 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x5639863d7b50 0x5639863da010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:34.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.004+0000 7fefb2aaf740 4 client.16134 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:48:34.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.004+0000 7fefb2aaf740 1 -- 172.21.15.164:0/4022520890 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56398631e860 con 0x5639863d7b50 2024-09-19T08:48:34.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.008+0000 7fefae472640 1 --2- 172.21.15.164:0/4022520890 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x5639863d7b50 0x5639863da010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:34.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.013+0000 7fefae472640 1 --2- 172.21.15.164:0/4022520890 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x5639863d7b50 0x5639863da010 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=355611b2f5f070c0 server_cookie=4fc370be91316afb in_seq=0 out_seq=0 2024-09-19T08:48:34.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.013+0000 7fef91ffb640 10 client.16134 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:48:34.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.016+0000 7fef91ffb640 1 -- 172.21.15.164:0/4022520890 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56398631e860 con 0x5639863d7b50 2024-09-19T08:48:34.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.016+0000 7fef91ffb640 10 client.16134 handle_command_reply: tid=0 2024-09-19T08:48:34.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.016+0000 7fefb2aaf740 1 client.16134 shutdown 2024-09-19T08:48:34.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.017+0000 7fefb2aaf740 1 -- 172.21.15.164:0/4022520890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639863b3d30 msgr2=0x5639863d3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:34.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.017+0000 7fefb2aaf740 1 --2- 172.21.15.164:0/4022520890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639863b3d30 0x5639863d3740 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fefa4098c70 tx=0x7fefa409c8d0 comp rx=0 tx=0).stop 2024-09-19T08:48:34.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.018+0000 7fefb2aaf740 1 -- 172.21.15.164:0/4022520890 shutdown_connections 2024-09-19T08:48:34.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.018+0000 7fefb2aaf740 1 --2- 172.21.15.164:0/4022520890 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x5639863d7b50 0x5639863da010 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:34.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.018+0000 7fefb2aaf740 1 --2- 172.21.15.164:0/4022520890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639863b3d30 0x5639863d3740 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:34.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.018+0000 7fefb2aaf740 1 --2- 172.21.15.164:0/4022520890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5639863b6520 0x5639863b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:34.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.018+0000 7fefb2aaf740 1 --2- 172.21.15.164:0/4022520890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639863ab780 0x5639863b32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:34.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.018+0000 7fefb2aaf740 1 -- 172.21.15.164:0/4022520890 >> 172.21.15.164:0/4022520890 conn(0x563986374310 msgr2=0x5639863adae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:34.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.018+0000 7fef91ffb640 0 client.16134 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:48:34.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.020+0000 7fefb2aaf740 1 -- 172.21.15.164:0/4022520890 shutdown_connections 2024-09-19T08:48:34.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.020+0000 7fefb2aaf740 1 -- 172.21.15.164:0/4022520890 wait complete. 2024-09-19T08:48:34.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.020+0000 7fefb2aaf740 20 client.16134 trim_cache size 0 max 16384 2024-09-19T08:48:34.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.061+0000 7fefb2aaf740 1 Processor -- start 2024-09-19T08:48:34.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.061+0000 7fefb2aaf740 1 -- start start 2024-09-19T08:48:34.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.061+0000 7fefb2aaf740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5639863ab780 0x5639863b5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:34.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.062+0000 7fefb2aaf740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639863b3d30 0x5639863d3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:34.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.062+0000 7fefb2aaf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639863b6520 0x5639863d3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:34.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.062+0000 7fefb2aaf740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c00e6d0 con 0x5639863b3d30 2024-09-19T08:48:34.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.062+0000 7fefb2aaf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c010210 con 0x5639863b6520 2024-09-19T08:48:34.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.062+0000 7fefb2aaf740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c00eda0 con 0x5639863ab780 2024-09-19T08:48:34.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.063+0000 7fefadc71640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639863b3d30 0x5639863d3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:34.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.063+0000 7fefadc71640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639863b3d30 0x5639863d3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58880/0 (socket says 172.21.15.164:58880) 2024-09-19T08:48:34.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.063+0000 7fefadc71640 1 -- 172.21.15.164:0/2222681353 learned_addr learned my addr 172.21.15.164:0/2222681353 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:34.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.063+0000 7fefadc71640 1 -- 172.21.15.164:0/2222681353 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5639863ab780 msgr2=0x5639863b5220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:48:34.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.063+0000 7fefadc71640 1 --2- 172.21.15.164:0/2222681353 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5639863ab780 0x5639863b5220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:34.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.063+0000 7fefadc71640 1 -- 172.21.15.164:0/2222681353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639863b6520 msgr2=0x5639863d3a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:48:34.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.063+0000 7fefadc71640 1 --2- 172.21.15.164:0/2222681353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639863b6520 0x5639863d3a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:34.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.063+0000 7fefadc71640 1 -- 172.21.15.164:0/2222681353 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639863b2260 con 0x5639863b3d30 2024-09-19T08:48:34.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.064+0000 7fefadc71640 1 --2- 172.21.15.164:0/2222681353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639863b3d30 0x5639863d3530 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fef98009230 tx=0x7fef98005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934b4a621ad608f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:34.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.065+0000 7fef93fff640 1 -- 172.21.15.164:0/2222681353 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef98040070 con 0x5639863b3d30 2024-09-19T08:48:34.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.065+0000 7fef93fff640 1 -- 172.21.15.164:0/2222681353 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef98004300 con 0x5639863b3d30 2024-09-19T08:48:34.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.065+0000 7fef93fff640 1 -- 172.21.15.164:0/2222681353 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef98035040 con 0x5639863b3d30 2024-09-19T08:48:34.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.073+0000 7fefb2aaf740 1 -- 172.21.15.164:0/2222681353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639863b3d30 msgr2=0x5639863d3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:34.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.073+0000 7fefb2aaf740 1 --2- 172.21.15.164:0/2222681353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639863b3d30 0x5639863d3530 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fef98009230 tx=0x7fef98005d10 comp rx=0 tx=0).stop 2024-09-19T08:48:34.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.074+0000 7fefb2aaf740 1 -- 172.21.15.164:0/2222681353 shutdown_connections 2024-09-19T08:48:34.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.074+0000 7fefb2aaf740 1 --2- 172.21.15.164:0/2222681353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639863b6520 0x5639863d3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:34.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.074+0000 7fefb2aaf740 1 --2- 172.21.15.164:0/2222681353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639863b3d30 0x5639863d3530 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:34.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.074+0000 7fefb2aaf740 1 --2- 172.21.15.164:0/2222681353 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5639863ab780 0x5639863b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:34.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.074+0000 7fefb2aaf740 1 -- 172.21.15.164:0/2222681353 >> 172.21.15.164:0/2222681353 conn(0x563986374310 msgr2=0x5639863b4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:34.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.078+0000 7fefb2aaf740 1 -- 172.21.15.164:0/2222681353 shutdown_connections 2024-09-19T08:48:34.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.078+0000 7fefb2aaf740 1 -- 172.21.15.164:0/2222681353 wait complete. 2024-09-19T08:48:34.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.079+0000 7fefb2aaf740 1 -- start start 2024-09-19T08:48:34.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.080+0000 7fefb2aaf740 1 Processor -- start 2024-09-19T08:48:34.080 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.081+0000 7fefb2aaf740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639863ab780 0x5639863ca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:34.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.081+0000 7fefb2aaf740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5639863b3d30 0x5639863caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:34.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.081+0000 7fefb2aaf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639863b6520 0x5639864408e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:34.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.081+0000 7fefb2aaf740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c00fb40 con 0x5639863ab780 2024-09-19T08:48:34.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.081+0000 7fefb2aaf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c00f470 con 0x5639863b6520 2024-09-19T08:48:34.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.081+0000 7fefb2aaf740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c00c6b0 con 0x5639863b3d30 2024-09-19T08:48:34.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.082+0000 7fefb2aaf740 10 client.0 _ll_register_callbacks cb 0x7fefa409c350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:48:34.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.082+0000 7fefae472640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639863ab780 0x5639863ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:34.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.082+0000 7fefae472640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639863ab780 0x5639863ca760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58882/0 (socket says 172.21.15.164:58882) 2024-09-19T08:48:34.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.082+0000 7fefae472640 1 -- 172.21.15.164:0/3347495022 learned_addr learned my addr 172.21.15.164:0/3347495022 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:34.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.083+0000 7fefae472640 1 -- 172.21.15.164:0/3347495022 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5639863b3d30 msgr2=0x5639863caca0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:48:34.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.083+0000 7fefae472640 1 --2- 172.21.15.164:0/3347495022 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5639863b3d30 0x5639863caca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:34.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.083+0000 7fefae472640 1 -- 172.21.15.164:0/3347495022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639863b6520 msgr2=0x5639864408e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:48:34.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.083+0000 7fefae472640 1 --2- 172.21.15.164:0/3347495022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639863b6520 0x5639864408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:34.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.083+0000 7fefae472640 1 -- 172.21.15.164:0/3347495022 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639863b8c80 con 0x5639863ab780 2024-09-19T08:48:34.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.084+0000 7fefae472640 1 --2- 172.21.15.164:0/3347495022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639863ab780 0x5639863ca760 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fefa0013fd0 tx=0x7fefa0005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc36cbab32131473 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:34.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.087+0000 7fef91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:48:34.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.087+0000 7fef91ffb640 1 -- 172.21.15.164:0/3347495022 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0039060 con 0x5639863ab780 2024-09-19T08:48:34.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.087+0000 7fef91ffb640 1 -- 172.21.15.164:0/3347495022 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fefa001dc30 con 0x5639863ab780 2024-09-19T08:48:34.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.096+0000 7fefb2aaf740 1 -- 172.21.15.164:0/3347495022 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef98004300 con 0x5639863ab780 2024-09-19T08:48:34.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.102+0000 7fef91ffb640 1 -- 172.21.15.164:0/3347495022 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fefa0009b20 con 0x5639863ab780 2024-09-19T08:48:34.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.104+0000 7fefb2aaf740 10 client.16140 fetch_fsmap learned FSMap version 28 2024-09-19T08:48:34.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.104+0000 7fefb2aaf740 1 -- 172.21.15.164:0/3347495022 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563986441dd0 con 0x5639863ab780 2024-09-19T08:48:34.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.107+0000 7fef91ffb640 1 -- 172.21.15.164:0/3347495022 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef98004300 con 0x5639863ab780 2024-09-19T08:48:34.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.108+0000 7fef91ffb640 1 -- 172.21.15.164:0/3347495022 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fefa0033030 con 0x5639863ab780 2024-09-19T08:48:34.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.108+0000 7fefb2aaf740 10 client.16140 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:48:34.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.108+0000 7fefb2aaf740 10 client.16140 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T08:48:34.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.108+0000 7fefb2aaf740 20 client.16140 populate_metadata read hostname 'smithi164' 2024-09-19T08:48:34.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.108+0000 7fefb2aaf740 1 --2- 172.21.15.164:0/3347495022 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x5639864421d0 0x5639864445c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:34.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.108+0000 7fefb2aaf740 4 client.16140 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:48:34.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.108+0000 7fefb2aaf740 1 -- 172.21.15.164:0/3347495022 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563986318d70 con 0x5639864421d0 2024-09-19T08:48:34.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.110+0000 7fefadc71640 1 --2- 172.21.15.164:0/3347495022 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x5639864421d0 0x5639864445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:34.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.116+0000 7fefadc71640 1 --2- 172.21.15.164:0/3347495022 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x5639864421d0 0x5639864445c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ea8965ea1af7c28 server_cookie=f24c72d728e101d2 in_seq=0 out_seq=0 2024-09-19T08:48:34.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.116+0000 7fef91ffb640 10 client.16140 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:48:34.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.116+0000 7fef91ffb640 1 -- 172.21.15.164:0/3347495022 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563986318d70 con 0x5639864421d0 2024-09-19T08:48:34.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.116+0000 7fef91ffb640 10 client.16140 handle_command_reply: tid=0 2024-09-19T08:48:34.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.116+0000 7fefb2aaf740 1 client.16140 shutdown 2024-09-19T08:48:34.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.119+0000 7fefb2aaf740 1 -- 172.21.15.164:0/3347495022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639863ab780 msgr2=0x5639863ca760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:34.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.119+0000 7fefb2aaf740 1 --2- 172.21.15.164:0/3347495022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639863ab780 0x5639863ca760 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fefa0013fd0 tx=0x7fefa0005f70 comp rx=0 tx=0).stop 2024-09-19T08:48:34.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:33 smithi170 ceph-mon[35072]: pgmap v85: 97 pgs: 97 active+clean; 17 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 575 MiB/s wr, 4.55k op/s 2024-09-19T08:48:34.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.129+0000 7fefb2aaf740 1 -- 172.21.15.164:0/3347495022 shutdown_connections 2024-09-19T08:48:34.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.129+0000 7fefb2aaf740 1 --2- 172.21.15.164:0/3347495022 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x5639864421d0 0x5639864445c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:34.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.129+0000 7fefb2aaf740 1 --2- 172.21.15.164:0/3347495022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639863b6520 0x5639864408e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:34.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.129+0000 7fefb2aaf740 1 --2- 172.21.15.164:0/3347495022 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5639863b3d30 0x5639863caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:34.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.129+0000 7fefb2aaf740 1 --2- 172.21.15.164:0/3347495022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639863ab780 0x5639863ca760 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:34.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.129+0000 7fefb2aaf740 1 -- 172.21.15.164:0/3347495022 >> 172.21.15.164:0/3347495022 conn(0x563986374310 msgr2=0x5639863b5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:34.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.130+0000 7fef91ffb640 0 client.16140 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:48:34.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.132+0000 7fefb2aaf740 1 -- 172.21.15.164:0/3347495022 shutdown_connections 2024-09-19T08:48:34.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.133+0000 7fefb2aaf740 1 -- 172.21.15.164:0/3347495022 wait complete. 2024-09-19T08:48:34.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.134+0000 7fefb2aaf740 20 client.16140 trim_cache size 0 max 16384 2024-09-19T08:48:34.140 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:48:34.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.145+0000 7fefb06d6640 1 -- 172.21.15.164:0/286148492 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fef940a3580 msgr2=0x7fef940a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:34.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.145+0000 7fefb06d6640 1 --2- 172.21.15.164:0/286148492 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fef940a3580 0x7fef940a5a40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fefa00126c0 tx=0x7fefa0012ac0 comp rx=0 tx=0).stop 2024-09-19T08:48:34.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.145+0000 7fefb06d6640 1 -- 172.21.15.164:0/286148492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefa810d3a0 msgr2=0x7fefa807ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:34.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.145+0000 7fefb06d6640 1 --2- 172.21.15.164:0/286148492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefa810d3a0 0x7fefa807ba40 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fefa400b410 tx=0x7fefa400b8d0 comp rx=0 tx=0).stop 2024-09-19T08:48:34.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.147+0000 7fefb06d6640 1 -- 172.21.15.164:0/286148492 shutdown_connections 2024-09-19T08:48:34.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.147+0000 7fefb06d6640 1 --2- 172.21.15.164:0/286148492 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fef940a3580 0x7fef940a5a40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:34.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.147+0000 7fefb06d6640 1 --2- 172.21.15.164:0/286148492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefa810d3a0 0x7fefa807ba40 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:34.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.147+0000 7fefb06d6640 1 --2- 172.21.15.164:0/286148492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa8108c30 0x7fefa8082d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:34.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.147+0000 7fefb06d6640 1 --2- 172.21.15.164:0/286148492 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefa8108280 0x7fefa80827d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:34.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.147+0000 7fefb06d6640 1 -- 172.21.15.164:0/286148492 >> 172.21.15.164:0/286148492 conn(0x7fefa8077a00 msgr2=0x7fefa8078d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:34.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.155+0000 7fefb06d6640 1 -- 172.21.15.164:0/286148492 shutdown_connections 2024-09-19T08:48:34.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:34.155+0000 7fefb06d6640 1 -- 172.21.15.164:0/286148492 wait complete. 2024-09-19T08:48:34.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:33 smithi173 ceph-mon[35306]: pgmap v85: 97 pgs: 97 active+clean; 17 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 575 MiB/s wr, 4.55k op/s 2024-09-19T08:48:36.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:35 smithi170 ceph-mon[35072]: pgmap v86: 97 pgs: 97 active+clean; 17 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 797 MiB/s wr, 4.81k op/s 2024-09-19T08:48:36.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:35 smithi173 ceph-mon[35306]: pgmap v86: 97 pgs: 97 active+clean; 17 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 797 MiB/s wr, 4.81k op/s 2024-09-19T08:48:36.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:35 smithi164 ceph-mon[30928]: pgmap v86: 97 pgs: 97 active+clean; 17 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 797 MiB/s wr, 4.81k op/s 2024-09-19T08:48:36.610 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T08:48:37.815 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:48:38.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:37 smithi173 ceph-mon[35306]: pgmap v87: 97 pgs: 97 active+clean; 17 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 711 MiB/s wr, 3.12k op/s 2024-09-19T08:48:38.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:37 smithi164 ceph-mon[30928]: pgmap v87: 97 pgs: 97 active+clean; 17 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 711 MiB/s wr, 3.12k op/s 2024-09-19T08:48:38.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:37 smithi170 ceph-mon[35072]: pgmap v87: 97 pgs: 97 active+clean; 17 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 711 MiB/s wr, 3.12k op/s 2024-09-19T08:48:39.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.813+0000 7ff488944640 1 -- 172.21.15.164:0/1635602046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff48410d530 msgr2=0x7ff4841150b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:39.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.813+0000 7ff488944640 1 --2- 172.21.15.164:0/1635602046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff48410d530 0x7ff4841150b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff47c009ca0 tx=0x7ff47c02a8f0 comp rx=0 tx=0).stop 2024-09-19T08:48:39.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.813+0000 7ff488944640 1 -- 172.21.15.164:0/1635602046 shutdown_connections 2024-09-19T08:48:39.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.813+0000 7ff488944640 1 --2- 172.21.15.164:0/1635602046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff48410d530 0x7ff4841150b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:39.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.813+0000 7ff488944640 1 --2- 172.21.15.164:0/1635602046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff484108e00 0x7ff48410ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:39.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.813+0000 7ff488944640 1 --2- 172.21.15.164:0/1635602046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff484108450 0x7ff484108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:39.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.813+0000 7ff488944640 1 -- 172.21.15.164:0/1635602046 >> 172.21.15.164:0/1635602046 conn(0x7ff484077ba0 msgr2=0x7ff484079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:39.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.823+0000 7ff488944640 1 -- 172.21.15.164:0/1635602046 shutdown_connections 2024-09-19T08:48:39.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.828+0000 7ff488944640 1 -- 172.21.15.164:0/1635602046 wait complete. 2024-09-19T08:48:39.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.829+0000 7ff488944640 1 Processor -- start 2024-09-19T08:48:39.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.833+0000 7ff488944640 1 -- start start 2024-09-19T08:48:39.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.834+0000 7ff488944640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff484108450 0x7ff484082590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:39.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.834+0000 7ff488944640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff484108e00 0x7ff484082ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:39.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.835+0000 7ff488944640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff48410d530 0x7ff48407b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:39.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.835+0000 7ff488944640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff484118620 con 0x7ff484108e00 2024-09-19T08:48:39.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.835+0000 7ff488944640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff4841184a0 con 0x7ff484108450 2024-09-19T08:48:39.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.835+0000 7ff488944640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff4841187a0 con 0x7ff48410d530 2024-09-19T08:48:39.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.836+0000 7ff482d9d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff48410d530 0x7ff48407b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:39.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.836+0000 7ff479d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff484108e00 0x7ff484082ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:39.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.836+0000 7ff482d9d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff48410d530 0x7ff48407b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:32944/0 (socket says 172.21.15.164:32944) 2024-09-19T08:48:39.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.836+0000 7ff482d9d640 1 -- 172.21.15.164:0/3471634368 learned_addr learned my addr 172.21.15.164:0/3471634368 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:39.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.836+0000 7ff48259c640 1 --2- 172.21.15.164:0/3471634368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff484108450 0x7ff484082590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:39.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.837+0000 7ff482d9d640 1 -- 172.21.15.164:0/3471634368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff484108450 msgr2=0x7ff484082590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:39.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.837+0000 7ff482d9d640 1 --2- 172.21.15.164:0/3471634368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff484108450 0x7ff484082590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:39.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.837+0000 7ff482d9d640 1 -- 172.21.15.164:0/3471634368 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff484108e00 msgr2=0x7ff484082ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:39.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.837+0000 7ff482d9d640 1 --2- 172.21.15.164:0/3471634368 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff484108e00 0x7ff484082ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:39.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.837+0000 7ff482d9d640 1 -- 172.21.15.164:0/3471634368 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff48407bee0 con 0x7ff48410d530 2024-09-19T08:48:39.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.838+0000 7ff482d9d640 1 --2- 172.21.15.164:0/3471634368 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff48410d530 0x7ff48407b620 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff47c004af0 tx=0x7ff47c003f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:39.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.840+0000 7ff47bfff640 1 -- 172.21.15.164:0/3471634368 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c031700 con 0x7ff48410d530 2024-09-19T08:48:39.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.841+0000 7ff47bfff640 1 -- 172.21.15.164:0/3471634368 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff47c02d070 con 0x7ff48410d530 2024-09-19T08:48:39.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.841+0000 7ff488944640 1 -- 172.21.15.164:0/3471634368 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff48407c110 con 0x7ff48410d530 2024-09-19T08:48:39.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.841+0000 7ff488944640 1 -- 172.21.15.164:0/3471634368 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4840800c0 con 0x7ff48410d530 2024-09-19T08:48:39.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.847+0000 7ff48ad1d740 1 Processor -- start 2024-09-19T08:48:39.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.853+0000 7ff47bfff640 1 -- 172.21.15.164:0/3471634368 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c02fb00 con 0x7ff48410d530 2024-09-19T08:48:39.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.857+0000 7ff48ad1d740 1 -- start start 2024-09-19T08:48:39.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.858+0000 7ff48ad1d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb5cca0e00 0x55eb5cc9f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:39.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.858+0000 7ff48ad1d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb5cc9f890 0x55eb5cc9fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:39.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.858+0000 7ff48ad1d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb5ccaa630 0x55eb5ccaca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:39.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.858+0000 7ff48ad1d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55eb5cc052f0 con 0x55eb5cca0e00 2024-09-19T08:48:39.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.858+0000 7ff48ad1d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55eb5cbaccf0 con 0x55eb5cc9f890 2024-09-19T08:48:39.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.858+0000 7ff48ad1d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55eb5cc012b0 con 0x55eb5ccaa630 2024-09-19T08:48:39.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.858+0000 7ff47bfff640 1 -- 172.21.15.164:0/3471634368 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7ff47c02fd60 con 0x7ff48410d530 2024-09-19T08:48:39.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.860+0000 7ff47bfff640 1 --2- 172.21.15.164:0/3471634368 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff4700a3580 0x7ff4700a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:39.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.860+0000 7ff47bfff640 1 -- 172.21.15.164:0/3471634368 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7ff47c0f18f0 con 0x7ff48410d530 2024-09-19T08:48:39.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.860+0000 7ff479d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb5cca0e00 0x55eb5cc9f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:39.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.861+0000 7ff479d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb5cca0e00 0x55eb5cc9f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58912/0 (socket says 172.21.15.164:58912) 2024-09-19T08:48:39.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.861+0000 7ff479d9b640 1 -- 172.21.15.164:0/486362684 learned_addr learned my addr 172.21.15.164:0/486362684 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:39.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.861+0000 7ff479d9b640 1 --2- 172.21.15.164:0/486362684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb5ccaa630 0x55eb5ccaca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:39.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.861+0000 7ff48259c640 1 --2- 172.21.15.164:0/486362684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb5cc9f890 0x55eb5cc9fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:39.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.862+0000 7ff482d9d640 1 --2- 172.21.15.164:0/3471634368 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff4700a3580 0x7ff4700a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:39.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.862+0000 7ff479d9b640 1 -- 172.21.15.164:0/486362684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb5cc9f890 msgr2=0x55eb5cc9fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:39.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.862+0000 7ff479d9b640 1 --2- 172.21.15.164:0/486362684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb5cc9f890 0x55eb5cc9fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:39.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.862+0000 7ff479d9b640 1 -- 172.21.15.164:0/486362684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb5cca0e00 msgr2=0x55eb5cc9f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:39.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.862+0000 7ff479d9b640 1 --2- 172.21.15.164:0/486362684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb5cca0e00 0x55eb5cc9f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:39.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.862+0000 7ff479d9b640 1 -- 172.21.15.164:0/486362684 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb5cb913a0 con 0x55eb5ccaa630 2024-09-19T08:48:39.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.863+0000 7ff479d9b640 1 --2- 172.21.15.164:0/486362684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb5ccaa630 0x55eb5ccaca20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff46c013710 tx=0x7ff46c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e72e193bea40be9d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:39.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.864+0000 7ff46bfff640 1 -- 172.21.15.164:0/486362684 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c017020 con 0x55eb5ccaa630 2024-09-19T08:48:39.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.864+0000 7ff46bfff640 1 -- 172.21.15.164:0/486362684 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff46c013110 con 0x55eb5ccaa630 2024-09-19T08:48:39.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.864+0000 7ff46bfff640 1 -- 172.21.15.164:0/486362684 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c00f920 con 0x55eb5ccaa630 2024-09-19T08:48:39.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.865+0000 7ff48ad1d740 1 -- 172.21.15.164:0/486362684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb5ccaa630 msgr2=0x55eb5ccaca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:39.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.865+0000 7ff48ad1d740 1 --2- 172.21.15.164:0/486362684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb5ccaa630 0x55eb5ccaca20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff46c013710 tx=0x7ff46c004730 comp rx=0 tx=0).stop 2024-09-19T08:48:39.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.865+0000 7ff482d9d640 1 --2- 172.21.15.164:0/3471634368 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff4700a3580 0x7ff4700a5a40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0be960 tx=0x7ff47c005fb0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:39.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.866+0000 7ff48ad1d740 1 -- 172.21.15.164:0/486362684 shutdown_connections 2024-09-19T08:48:39.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.866+0000 7ff48ad1d740 1 --2- 172.21.15.164:0/486362684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb5ccaa630 0x55eb5ccaca20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:39.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.866+0000 7ff48ad1d740 1 --2- 172.21.15.164:0/486362684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb5cc9f890 0x55eb5cc9fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:39.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.866+0000 7ff48ad1d740 1 --2- 172.21.15.164:0/486362684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb5cca0e00 0x55eb5cc9f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:39.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.866+0000 7ff48ad1d740 1 -- 172.21.15.164:0/486362684 >> 172.21.15.164:0/486362684 conn(0x55eb5cc68310 msgr2=0x55eb5cc9bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:39.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.867+0000 7ff48ad1d740 1 -- 172.21.15.164:0/486362684 shutdown_connections 2024-09-19T08:48:39.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.867+0000 7ff48ad1d740 1 -- 172.21.15.164:0/486362684 wait complete. 2024-09-19T08:48:39.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.867+0000 7ff48ad1d740 1 -- start start 2024-09-19T08:48:39.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.867+0000 7ff48ad1d740 1 Processor -- start 2024-09-19T08:48:39.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.868+0000 7ff48ad1d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb5cc9f890 0x55eb5cca9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:39.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.868+0000 7ff48ad1d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb5cca0e00 0x55eb5cca7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:39.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.868+0000 7ff48ad1d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb5ccaa630 0x55eb5cca76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:39.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.869+0000 7ff48ad1d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55eb5ccad240 con 0x55eb5cc9f890 2024-09-19T08:48:39.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.869+0000 7ff48ad1d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55eb5cbc0060 con 0x55eb5ccaa630 2024-09-19T08:48:39.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.869+0000 7ff48ad1d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55eb5ccad3c0 con 0x55eb5cca0e00 2024-09-19T08:48:39.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.869+0000 7ff48ad1d740 10 client.0 _ll_register_callbacks cb 0x55eb5c9138c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:48:39.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.870+0000 7ff479d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb5cc9f890 0x55eb5cca9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:39.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.870+0000 7ff479d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb5cc9f890 0x55eb5cca9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58922/0 (socket says 172.21.15.164:58922) 2024-09-19T08:48:39.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.870+0000 7ff479d9b640 1 -- 172.21.15.164:0/3799831700 learned_addr learned my addr 172.21.15.164:0/3799831700 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:39.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.870+0000 7ff479d9b640 1 --2- 172.21.15.164:0/3799831700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb5ccaa630 0x55eb5cca76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:39.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.870+0000 7ff48259c640 1 --2- 172.21.15.164:0/3799831700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb5cca0e00 0x55eb5cca7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:39.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.872+0000 7ff479d9b640 1 -- 172.21.15.164:0/3799831700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb5cca0e00 msgr2=0x55eb5cca7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:39.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.872+0000 7ff479d9b640 1 --2- 172.21.15.164:0/3799831700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb5cca0e00 0x55eb5cca7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:39.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.872+0000 7ff479d9b640 1 -- 172.21.15.164:0/3799831700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb5cc9f890 msgr2=0x55eb5cca9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:39.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.872+0000 7ff479d9b640 1 --2- 172.21.15.164:0/3799831700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb5cc9f890 0x55eb5cca9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:39.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.872+0000 7ff479d9b640 1 -- 172.21.15.164:0/3799831700 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb5c9763c0 con 0x55eb5ccaa630 2024-09-19T08:48:39.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.879+0000 7ff479d9b640 1 --2- 172.21.15.164:0/3799831700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb5ccaa630 0x55eb5cca76c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff46c0133d0 tx=0x7ff46c00f500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6e230b41c51ff68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:39.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.879+0000 7ff469ffb640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-19T08:48:39.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.881+0000 7ff469ffb640 1 -- 172.21.15.164:0/3799831700 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c017030 con 0x55eb5ccaa630 2024-09-19T08:48:39.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.882+0000 7ff469ffb640 1 -- 172.21.15.164:0/3799831700 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7ff46c004460 con 0x55eb5ccaa630 2024-09-19T08:48:39.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.882+0000 7ff48ad1d740 1 -- 172.21.15.164:0/3799831700 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff46c013110 con 0x55eb5ccaa630 2024-09-19T08:48:39.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.891+0000 7ff469ffb640 1 -- 172.21.15.164:0/3799831700 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff46c00f7b0 con 0x55eb5ccaa630 2024-09-19T08:48:39.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.891+0000 7ff48ad1d740 10 client.25598 fetch_fsmap learned FSMap version 28 2024-09-19T08:48:39.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.891+0000 7ff48ad1d740 1 -- 172.21.15.164:0/3799831700 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eb5ccb8bd0 con 0x55eb5ccaa630 2024-09-19T08:48:39.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.892+0000 7ff469ffb640 1 -- 172.21.15.164:0/3799831700 <== mon.2 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff46c00fe30 con 0x55eb5ccaa630 2024-09-19T08:48:39.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.892+0000 7ff469ffb640 1 -- 172.21.15.164:0/3799831700 <== mon.2 v2:172.21.15.170:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff46c003b50 con 0x55eb5ccaa630 2024-09-19T08:48:39.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.892+0000 7ff48ad1d740 10 client.25598 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:48:39.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.892+0000 7ff48ad1d740 10 client.25598 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T08:48:39.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.892+0000 7ff48ad1d740 20 client.25598 populate_metadata read hostname 'smithi164' 2024-09-19T08:48:39.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.892+0000 7ff48ad1d740 1 --2- 172.21.15.164:0/3799831700 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55eb5ccb9fe0 0x55eb5ccbc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:39.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.892+0000 7ff48ad1d740 4 client.25598 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:48:39.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.892+0000 7ff48ad1d740 1 -- 172.21.15.164:0/3799831700 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff46c013110 con 0x55eb5ccb9fe0 2024-09-19T08:48:39.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.893+0000 7ff48259c640 1 --2- 172.21.15.164:0/3799831700 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55eb5ccb9fe0 0x55eb5ccbc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:39.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.899+0000 7ff48259c640 1 --2- 172.21.15.164:0/3799831700 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55eb5ccb9fe0 0x55eb5ccbc3d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d7604bcd88a37f50 server_cookie=950ef126bbbb0eb7 in_seq=0 out_seq=0 2024-09-19T08:48:39.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.900+0000 7ff469ffb640 10 client.25598 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:48:39.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.903+0000 7ff469ffb640 1 -- 172.21.15.164:0/3799831700 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7ff46c013110 con 0x55eb5ccb9fe0 2024-09-19T08:48:39.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.903+0000 7ff469ffb640 10 client.25598 handle_command_reply: tid=0 2024-09-19T08:48:39.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.903+0000 7ff48ad1d740 1 client.25598 shutdown 2024-09-19T08:48:39.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.917+0000 7ff48ad1d740 1 -- 172.21.15.164:0/3799831700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb5ccaa630 msgr2=0x55eb5cca76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:39.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.917+0000 7ff48ad1d740 1 --2- 172.21.15.164:0/3799831700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb5ccaa630 0x55eb5cca76c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff46c0133d0 tx=0x7ff46c00f500 comp rx=0 tx=0).stop 2024-09-19T08:48:39.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.919+0000 7ff48ad1d740 1 -- 172.21.15.164:0/3799831700 shutdown_connections 2024-09-19T08:48:39.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.919+0000 7ff48ad1d740 1 --2- 172.21.15.164:0/3799831700 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55eb5ccb9fe0 0x55eb5ccbc3d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:39.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.919+0000 7ff469ffb640 0 client.25598 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:48:39.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.919+0000 7ff48ad1d740 1 --2- 172.21.15.164:0/3799831700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb5ccaa630 0x55eb5cca76c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:39.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.919+0000 7ff48ad1d740 1 --2- 172.21.15.164:0/3799831700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb5cca0e00 0x55eb5cca7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:39.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.919+0000 7ff48ad1d740 1 --2- 172.21.15.164:0/3799831700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb5cc9f890 0x55eb5cca9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:39.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.919+0000 7ff48ad1d740 1 -- 172.21.15.164:0/3799831700 >> 172.21.15.164:0/3799831700 conn(0x55eb5cc68310 msgr2=0x55eb5cc9b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:39.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.919+0000 7ff48ad1d740 1 -- 172.21.15.164:0/3799831700 shutdown_connections 2024-09-19T08:48:39.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.919+0000 7ff48ad1d740 1 -- 172.21.15.164:0/3799831700 wait complete. 2024-09-19T08:48:39.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.920+0000 7ff48ad1d740 20 client.25598 trim_cache size 0 max 16384 2024-09-19T08:48:39.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.987+0000 7ff48ad1d740 1 Processor -- start 2024-09-19T08:48:39.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.988+0000 7ff48ad1d740 1 -- start start 2024-09-19T08:48:39.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.989+0000 7ff48ad1d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb5cc9f890 0x55eb5ccb6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:39.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.990+0000 7ff48ad1d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb5cca0e00 0x55eb5cca7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:39.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.990+0000 7ff48ad1d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb5ccaa630 0x55eb5cca6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:39.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.990+0000 7ff48ad1d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff4841bc600 con 0x55eb5ccaa630 2024-09-19T08:48:39.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.990+0000 7ff48ad1d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff4841be140 con 0x55eb5cc9f890 2024-09-19T08:48:39.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.991+0000 7ff48ad1d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff4841bccd0 con 0x55eb5cca0e00 2024-09-19T08:48:39.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.991+0000 7ff479d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb5ccaa630 0x55eb5cca6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:39.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.991+0000 7ff479d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb5ccaa630 0x55eb5cca6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58924/0 (socket says 172.21.15.164:58924) 2024-09-19T08:48:39.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.991+0000 7ff479d9b640 1 -- 172.21.15.164:0/804559261 learned_addr learned my addr 172.21.15.164:0/804559261 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:39.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.991+0000 7ff479d9b640 1 -- 172.21.15.164:0/804559261 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb5cca0e00 msgr2=0x55eb5cca7610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:48:39.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.991+0000 7ff479d9b640 1 --2- 172.21.15.164:0/804559261 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb5cca0e00 0x55eb5cca7610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:39.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.991+0000 7ff479d9b640 1 -- 172.21.15.164:0/804559261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb5cc9f890 msgr2=0x55eb5ccb6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:39.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.991+0000 7ff479d9b640 1 --2- 172.21.15.164:0/804559261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb5cc9f890 0x55eb5ccb6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:39.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.991+0000 7ff479d9b640 1 -- 172.21.15.164:0/804559261 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb5cc75800 con 0x55eb5ccaa630 2024-09-19T08:48:39.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.992+0000 7ff479d9b640 1 --2- 172.21.15.164:0/804559261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb5ccaa630 0x55eb5cca6160 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff46c016040 tx=0x7ff46c013350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29bbb2331cabbb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:39.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.993+0000 7ff46b7fe640 1 -- 172.21.15.164:0/804559261 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c017030 con 0x55eb5ccaa630 2024-09-19T08:48:39.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.993+0000 7ff46b7fe640 1 -- 172.21.15.164:0/804559261 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff46c0099f0 con 0x55eb5ccaa630 2024-09-19T08:48:39.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.993+0000 7ff46b7fe640 1 -- 172.21.15.164:0/804559261 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c026040 con 0x55eb5ccaa630 2024-09-19T08:48:39.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.997+0000 7ff48ad1d740 1 -- 172.21.15.164:0/804559261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb5ccaa630 msgr2=0x55eb5cca6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:39.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.997+0000 7ff48ad1d740 1 --2- 172.21.15.164:0/804559261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb5ccaa630 0x55eb5cca6160 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff46c016040 tx=0x7ff46c013350 comp rx=0 tx=0).stop 2024-09-19T08:48:39.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.997+0000 7ff48ad1d740 1 -- 172.21.15.164:0/804559261 shutdown_connections 2024-09-19T08:48:39.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.997+0000 7ff48ad1d740 1 --2- 172.21.15.164:0/804559261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb5ccaa630 0x55eb5cca6160 secure :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff46c016040 tx=0x7ff46c013350 comp rx=0 tx=0).stop 2024-09-19T08:48:39.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.997+0000 7ff48ad1d740 1 --2- 172.21.15.164:0/804559261 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb5cca0e00 0x55eb5cca7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:39.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.997+0000 7ff48ad1d740 1 --2- 172.21.15.164:0/804559261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb5cc9f890 0x55eb5ccb6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:39.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:39.997+0000 7ff48ad1d740 1 -- 172.21.15.164:0/804559261 >> 172.21.15.164:0/804559261 conn(0x55eb5cc68310 msgr2=0x55eb5cca5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:40.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.001+0000 7ff48ad1d740 1 -- 172.21.15.164:0/804559261 shutdown_connections 2024-09-19T08:48:40.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.001+0000 7ff48ad1d740 1 -- 172.21.15.164:0/804559261 wait complete. 2024-09-19T08:48:40.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.002+0000 7ff48ad1d740 1 -- start start 2024-09-19T08:48:40.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.002+0000 7ff48ad1d740 1 Processor -- start 2024-09-19T08:48:40.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.004+0000 7ff48ad1d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb5cc9f890 0x55eb5ccb2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:40.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.004+0000 7ff48ad1d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb5cca0e00 0x55eb5ccb2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:40.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.004+0000 7ff48ad1d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb5ccb9fe0 0x55eb5ccadae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:40.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.005+0000 7ff48ad1d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff4841bda70 con 0x55eb5cc9f890 2024-09-19T08:48:40.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.005+0000 7ff48ad1d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff4841bd3a0 con 0x55eb5cca0e00 2024-09-19T08:48:40.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.005+0000 7ff48ad1d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff4841ba600 con 0x55eb5ccb9fe0 2024-09-19T08:48:40.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.005+0000 7ff48ad1d740 10 client.0 _ll_register_callbacks cb 0x7ff46c009dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:48:40.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.011+0000 7ff48259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb5cca0e00 0x55eb5ccb2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:40.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.014+0000 7ff479d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb5cc9f890 0x55eb5ccb2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:40.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.014+0000 7ff479d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb5cc9f890 0x55eb5ccb2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58934/0 (socket says 172.21.15.164:58934) 2024-09-19T08:48:40.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.014+0000 7ff479d9b640 1 -- 172.21.15.164:0/3887017663 learned_addr learned my addr 172.21.15.164:0/3887017663 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:40.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.014+0000 7ff479d9b640 1 --2- 172.21.15.164:0/3887017663 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb5ccb9fe0 0x55eb5ccadae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:40.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.025+0000 7ff48259c640 1 -- 172.21.15.164:0/3887017663 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb5ccb9fe0 msgr2=0x55eb5ccadae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:40.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.025+0000 7ff48259c640 1 --2- 172.21.15.164:0/3887017663 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb5ccb9fe0 0x55eb5ccadae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:40.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.025+0000 7ff48259c640 1 -- 172.21.15.164:0/3887017663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb5cc9f890 msgr2=0x55eb5ccb2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:40.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.025+0000 7ff48259c640 1 --2- 172.21.15.164:0/3887017663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb5cc9f890 0x55eb5ccb2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:40.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.025+0000 7ff48259c640 1 -- 172.21.15.164:0/3887017663 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb5ccae2e0 con 0x55eb5cca0e00 2024-09-19T08:48:40.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.036+0000 7ff48259c640 1 --2- 172.21.15.164:0/3887017663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb5cca0e00 0x55eb5ccb2bd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55eb5ccbca00 tx=0x7ff4740157f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70b83761d998e432 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:40.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.037+0000 7ff4697fa640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-19T08:48:40.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.042+0000 7ff4697fa640 1 -- 172.21.15.164:0/3887017663 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff474021020 con 0x55eb5cca0e00 2024-09-19T08:48:40.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.042+0000 7ff48ad1d740 1 -- 172.21.15.164:0/3887017663 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55eb5cc12860 con 0x55eb5cca0e00 2024-09-19T08:48:40.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.042+0000 7ff4697fa640 1 -- 172.21.15.164:0/3887017663 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7ff47401d4e0 con 0x55eb5cca0e00 2024-09-19T08:48:40.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.047+0000 7ff4697fa640 1 -- 172.21.15.164:0/3887017663 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff47402c050 con 0x55eb5cca0e00 2024-09-19T08:48:40.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.047+0000 7ff48ad1d740 10 client.25601 fetch_fsmap learned FSMap version 28 2024-09-19T08:48:40.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.048+0000 7ff48ad1d740 1 -- 172.21.15.164:0/3887017663 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eb5cd244d0 con 0x55eb5cca0e00 2024-09-19T08:48:40.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.050+0000 7ff4697fa640 1 -- 172.21.15.164:0/3887017663 <== mon.2 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff47401dbc0 con 0x55eb5cca0e00 2024-09-19T08:48:40.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.050+0000 7ff4697fa640 1 -- 172.21.15.164:0/3887017663 <== mon.2 v2:172.21.15.170:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff474013aa0 con 0x55eb5cca0e00 2024-09-19T08:48:40.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.050+0000 7ff48ad1d740 10 client.25601 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:48:40.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.050+0000 7ff48ad1d740 10 client.25601 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T08:48:40.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.051+0000 7ff48ad1d740 20 client.25601 populate_metadata read hostname 'smithi164' 2024-09-19T08:48:40.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.051+0000 7ff48ad1d740 1 --2- 172.21.15.164:0/3887017663 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55eb5cd24c80 0x55eb5cd270a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:40.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.051+0000 7ff48ad1d740 4 client.25601 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:48:40.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.051+0000 7ff48ad1d740 1 -- 172.21.15.164:0/3887017663 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eb5cc12860 con 0x55eb5cd24c80 2024-09-19T08:48:40.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.053+0000 7ff479d9b640 1 --2- 172.21.15.164:0/3887017663 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55eb5cd24c80 0x55eb5cd270a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:40.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.063+0000 7ff4697fa640 10 client.25601 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:48:40.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.063+0000 7ff479d9b640 1 --2- 172.21.15.164:0/3887017663 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55eb5cd24c80 0x55eb5cd270a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=25051453ccdf43e8 server_cookie=29b5948b798100e8 in_seq=0 out_seq=0 2024-09-19T08:48:40.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.066+0000 7ff4697fa640 1 -- 172.21.15.164:0/3887017663 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eb5cc12860 con 0x55eb5cd24c80 2024-09-19T08:48:40.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.066+0000 7ff4697fa640 10 client.25601 handle_command_reply: tid=0 2024-09-19T08:48:40.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.066+0000 7ff48ad1d740 1 client.25601 shutdown 2024-09-19T08:48:40.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.067+0000 7ff48ad1d740 1 -- 172.21.15.164:0/3887017663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb5cca0e00 msgr2=0x55eb5ccb2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:40.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.067+0000 7ff48ad1d740 1 --2- 172.21.15.164:0/3887017663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb5cca0e00 0x55eb5ccb2bd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55eb5ccbca00 tx=0x7ff4740157f0 comp rx=0 tx=0).stop 2024-09-19T08:48:40.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.067+0000 7ff48ad1d740 1 -- 172.21.15.164:0/3887017663 shutdown_connections 2024-09-19T08:48:40.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.067+0000 7ff48ad1d740 1 --2- 172.21.15.164:0/3887017663 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eb5ccb9fe0 0x55eb5ccadae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:40.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.067+0000 7ff48ad1d740 1 --2- 172.21.15.164:0/3887017663 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55eb5cd24c80 0x55eb5cd270a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:40.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.067+0000 7ff48ad1d740 1 --2- 172.21.15.164:0/3887017663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb5cca0e00 0x55eb5ccb2bd0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:40.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.067+0000 7ff48ad1d740 1 --2- 172.21.15.164:0/3887017663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb5cc9f890 0x55eb5ccb2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:40.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.067+0000 7ff48ad1d740 1 -- 172.21.15.164:0/3887017663 >> 172.21.15.164:0/3887017663 conn(0x55eb5cc68310 msgr2=0x55eb5cca6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:40.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.067+0000 7ff4697fa640 0 client.25601 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:48:40.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.071+0000 7ff48ad1d740 1 -- 172.21.15.164:0/3887017663 shutdown_connections 2024-09-19T08:48:40.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.072+0000 7ff48ad1d740 1 -- 172.21.15.164:0/3887017663 wait complete. 2024-09-19T08:48:40.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.076+0000 7ff48ad1d740 20 client.25601 trim_cache size 0 max 16384 2024-09-19T08:48:40.080 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:48:40.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.078+0000 7ff47959a640 1 -- 172.21.15.164:0/3471634368 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff4700a3580 msgr2=0x7ff4700a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:40.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.078+0000 7ff47959a640 1 --2- 172.21.15.164:0/3471634368 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff4700a3580 0x7ff4700a5a40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0be960 tx=0x7ff47c005fb0 comp rx=0 tx=0).stop 2024-09-19T08:48:40.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.079+0000 7ff47959a640 1 -- 172.21.15.164:0/3471634368 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff48410d530 msgr2=0x7ff48407b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:40.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.079+0000 7ff47959a640 1 --2- 172.21.15.164:0/3471634368 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff48410d530 0x7ff48407b620 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff47c004af0 tx=0x7ff47c003f30 comp rx=0 tx=0).stop 2024-09-19T08:48:40.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.083+0000 7ff47959a640 1 -- 172.21.15.164:0/3471634368 shutdown_connections 2024-09-19T08:48:40.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.083+0000 7ff47959a640 1 --2- 172.21.15.164:0/3471634368 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff48410d530 0x7ff48407b620 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:40.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.083+0000 7ff47959a640 1 --2- 172.21.15.164:0/3471634368 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff4700a3580 0x7ff4700a5a40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:40.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.083+0000 7ff47959a640 1 --2- 172.21.15.164:0/3471634368 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff484108e00 0x7ff484082ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:40.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.083+0000 7ff47959a640 1 --2- 172.21.15.164:0/3471634368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff484108450 0x7ff484082590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:40.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.083+0000 7ff47959a640 1 -- 172.21.15.164:0/3471634368 >> 172.21.15.164:0/3471634368 conn(0x7ff484077ba0 msgr2=0x7ff4840783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:40.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.084+0000 7ff47959a640 1 -- 172.21.15.164:0/3471634368 shutdown_connections 2024-09-19T08:48:40.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:40.084+0000 7ff47959a640 1 -- 172.21.15.164:0/3471634368 wait complete. 2024-09-19T08:48:40.140 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:39 smithi164 ceph-mon[30928]: pgmap v88: 97 pgs: 97 active+clean; 17 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 590 MiB/s wr, 1.97k op/s 2024-09-19T08:48:40.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:39 smithi173 ceph-mon[35306]: pgmap v88: 97 pgs: 97 active+clean; 17 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 590 MiB/s wr, 1.97k op/s 2024-09-19T08:48:40.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:39 smithi170 ceph-mon[35072]: pgmap v88: 97 pgs: 97 active+clean; 17 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 590 MiB/s wr, 1.97k op/s 2024-09-19T08:48:42.219 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T08:48:42.220 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:41 smithi164 ceph-mon[30928]: pgmap v89: 97 pgs: 97 active+clean; 19 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 834 MiB/s wr, 3.10k op/s 2024-09-19T08:48:42.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:41 smithi170 ceph-mon[35072]: pgmap v89: 97 pgs: 97 active+clean; 19 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 834 MiB/s wr, 3.10k op/s 2024-09-19T08:48:42.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:41 smithi173 ceph-mon[35306]: pgmap v89: 97 pgs: 97 active+clean; 19 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 834 MiB/s wr, 3.10k op/s 2024-09-19T08:48:43.284 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:48:44.285 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:43 smithi164 ceph-mon[30928]: pgmap v90: 97 pgs: 97 active+clean; 19 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 729 MiB/s wr, 2.25k op/s 2024-09-19T08:48:44.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:43 smithi170 ceph-mon[35072]: pgmap v90: 97 pgs: 97 active+clean; 19 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 729 MiB/s wr, 2.25k op/s 2024-09-19T08:48:44.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:43 smithi173 ceph-mon[35306]: pgmap v90: 97 pgs: 97 active+clean; 19 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 729 MiB/s wr, 2.25k op/s 2024-09-19T08:48:45.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.152+0000 7f4fed64e640 1 -- 172.21.15.164:0/3235993272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fe806b8e0 msgr2=0x7f4fe806bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:45.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.152+0000 7f4fed64e640 1 --2- 172.21.15.164:0/3235993272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fe806b8e0 0x7f4fe806bcc0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8009860 tx=0x7f4fd802a410 comp rx=0 tx=0).stop 2024-09-19T08:48:45.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.153+0000 7f4fed64e640 1 -- 172.21.15.164:0/3235993272 shutdown_connections 2024-09-19T08:48:45.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.153+0000 7f4fed64e640 1 --2- 172.21.15.164:0/3235993272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fe811b550 0x7f4fe811d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:45.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.153+0000 7f4fed64e640 1 --2- 172.21.15.164:0/3235993272 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fe806c200 0x7f4fe811b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:45.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.153+0000 7f4fed64e640 1 --2- 172.21.15.164:0/3235993272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fe806b8e0 0x7f4fe806bcc0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:45.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.153+0000 7f4fed64e640 1 -- 172.21.15.164:0/3235993272 >> 172.21.15.164:0/3235993272 conn(0x7f4fe8071630 msgr2=0x7f4fe8073a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:45.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.153+0000 7f4fed64e640 1 -- 172.21.15.164:0/3235993272 shutdown_connections 2024-09-19T08:48:45.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.153+0000 7f4fed64e640 1 -- 172.21.15.164:0/3235993272 wait complete. 2024-09-19T08:48:45.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.154+0000 7f4fed64e640 1 Processor -- start 2024-09-19T08:48:45.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.154+0000 7f4fed64e640 1 -- start start 2024-09-19T08:48:45.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.155+0000 7f4fed64e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fe806b8e0 0x7f4fe811aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:45.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.155+0000 7f4fed64e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fe806c200 0x7f4fe811afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:45.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.155+0000 7f4fed64e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fe811b550 0x7f4fe8113d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:45.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.155+0000 7f4fed64e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe8120e00 con 0x7f4fe811b550 2024-09-19T08:48:45.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.155+0000 7f4fed64e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe8120c80 con 0x7f4fe806b8e0 2024-09-19T08:48:45.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.155+0000 7f4fed64e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe8120f80 con 0x7f4fe806c200 2024-09-19T08:48:45.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.156+0000 7f4fe67fc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fe806c200 0x7f4fe811afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:45.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.156+0000 7f4fe67fc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fe806c200 0x7f4fe811afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:48484/0 (socket says 172.21.15.164:48484) 2024-09-19T08:48:45.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.156+0000 7f4fe67fc640 1 -- 172.21.15.164:0/713061832 learned_addr learned my addr 172.21.15.164:0/713061832 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:45.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.158+0000 7f4fe6ffd640 1 --2- 172.21.15.164:0/713061832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fe806b8e0 0x7f4fe811aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:45.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.158+0000 7f4fe77fe640 1 --2- 172.21.15.164:0/713061832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fe811b550 0x7f4fe8113d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:45.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.161+0000 7f4fe67fc640 1 -- 172.21.15.164:0/713061832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fe806b8e0 msgr2=0x7f4fe811aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:45.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.161+0000 7f4fe67fc640 1 --2- 172.21.15.164:0/713061832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fe806b8e0 0x7f4fe811aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:45.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.161+0000 7f4fe67fc640 1 -- 172.21.15.164:0/713061832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fe811b550 msgr2=0x7f4fe8113d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:45.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.161+0000 7f4fe67fc640 1 --2- 172.21.15.164:0/713061832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fe811b550 0x7f4fe8113d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:45.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.161+0000 7f4fe67fc640 1 -- 172.21.15.164:0/713061832 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe81145c0 con 0x7f4fe806c200 2024-09-19T08:48:45.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.164+0000 7f4fe67fc640 1 --2- 172.21.15.164:0/713061832 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fe806c200 0x7f4fe811afd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000efb0 tx=0x7f4fe000c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:45.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.168+0000 7f4fcffff640 1 -- 172.21.15.164:0/713061832 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe001b070 con 0x7f4fe806c200 2024-09-19T08:48:45.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.169+0000 7f4fed64e640 1 -- 172.21.15.164:0/713061832 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fe81148b0 con 0x7f4fe806c200 2024-09-19T08:48:45.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.169+0000 7f4fed64e640 1 -- 172.21.15.164:0/713061832 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fe8118850 con 0x7f4fe806c200 2024-09-19T08:48:45.169 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.169+0000 7f4fcffff640 1 -- 172.21.15.164:0/713061832 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe000ba50 con 0x7f4fe806c200 2024-09-19T08:48:45.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.169+0000 7f4fcffff640 1 -- 172.21.15.164:0/713061832 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0016c60 con 0x7f4fe806c200 2024-09-19T08:48:45.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.174+0000 7f4fcffff640 1 -- 172.21.15.164:0/713061832 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f4fe001e070 con 0x7f4fe806c200 2024-09-19T08:48:45.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.177+0000 7f4fcffff640 1 --2- 172.21.15.164:0/713061832 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f4fd00a35c0 0x7f4fd00a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:45.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.178+0000 7f4fcffff640 1 -- 172.21.15.164:0/713061832 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f4fe00cfac0 con 0x7f4fe806c200 2024-09-19T08:48:45.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.179+0000 7f4fefa27740 1 Processor -- start 2024-09-19T08:48:45.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.179+0000 7f4fefa27740 1 -- start start 2024-09-19T08:48:45.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.187+0000 7f4fe6ffd640 1 --2- 172.21.15.164:0/713061832 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f4fd00a35c0 0x7f4fd00a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:45.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.187+0000 7f4fefa27740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562fc5948b40 0x562fc5947090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:45.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.187+0000 7f4fefa27740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fc59475d0 0x562fc5947a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:45.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.187+0000 7f4fefa27740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fc5952370 0x562fc5954760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:45.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.187+0000 7f4fefa27740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562fc58ad2f0 con 0x562fc59475d0 2024-09-19T08:48:45.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.187+0000 7f4fefa27740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562fc5854cf0 con 0x562fc5952370 2024-09-19T08:48:45.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.187+0000 7f4fefa27740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562fc58a92b0 con 0x562fc5948b40 2024-09-19T08:48:45.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.188+0000 7f4fe67fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fc59475d0 0x562fc5947a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:45.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.188+0000 7f4fe67fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fc59475d0 0x562fc5947a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42106/0 (socket says 172.21.15.164:42106) 2024-09-19T08:48:45.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.188+0000 7f4fe67fc640 1 -- 172.21.15.164:0/4167517913 learned_addr learned my addr 172.21.15.164:0/4167517913 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:45.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.188+0000 7f4fe67fc640 1 -- 172.21.15.164:0/4167517913 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562fc5948b40 msgr2=0x562fc5947090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:48:45.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.188+0000 7f4fe67fc640 1 --2- 172.21.15.164:0/4167517913 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562fc5948b40 0x562fc5947090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:45.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.188+0000 7f4fe67fc640 1 -- 172.21.15.164:0/4167517913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fc5952370 msgr2=0x562fc5954760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:48:45.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.188+0000 7f4fe67fc640 1 --2- 172.21.15.164:0/4167517913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fc5952370 0x562fc5954760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:45.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.188+0000 7f4fe67fc640 1 -- 172.21.15.164:0/4167517913 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fc58393a0 con 0x562fc59475d0 2024-09-19T08:48:45.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.189+0000 7f4fe67fc640 1 --2- 172.21.15.164:0/4167517913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fc59475d0 0x562fc5947a10 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4fe00982c0 tx=0x7f4fe009b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460e1b21f7f78c49 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:45.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.195+0000 7f4fcd7fa640 1 -- 172.21.15.164:0/4167517913 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe009a220 con 0x562fc59475d0 2024-09-19T08:48:45.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.195+0000 7f4fe6ffd640 1 --2- 172.21.15.164:0/713061832 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f4fd00a35c0 0x7f4fd00a5a80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8031000 tx=0x7f4fd8030e60 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:45.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.195+0000 7f4fcd7fa640 1 -- 172.21.15.164:0/4167517913 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe009f030 con 0x562fc59475d0 2024-09-19T08:48:45.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.195+0000 7f4fcd7fa640 1 -- 172.21.15.164:0/4167517913 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe009b2b0 con 0x562fc59475d0 2024-09-19T08:48:45.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.197+0000 7f4fefa27740 1 -- 172.21.15.164:0/4167517913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fc59475d0 msgr2=0x562fc5947a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:45.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.197+0000 7f4fefa27740 1 --2- 172.21.15.164:0/4167517913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fc59475d0 0x562fc5947a10 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4fe00982c0 tx=0x7f4fe009b8f0 comp rx=0 tx=0).stop 2024-09-19T08:48:45.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.197+0000 7f4fefa27740 1 -- 172.21.15.164:0/4167517913 shutdown_connections 2024-09-19T08:48:45.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.197+0000 7f4fefa27740 1 --2- 172.21.15.164:0/4167517913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fc5952370 0x562fc5954760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:45.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.197+0000 7f4fefa27740 1 --2- 172.21.15.164:0/4167517913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fc59475d0 0x562fc5947a10 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:45.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.197+0000 7f4fefa27740 1 --2- 172.21.15.164:0/4167517913 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562fc5948b40 0x562fc5947090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:45.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.197+0000 7f4fefa27740 1 -- 172.21.15.164:0/4167517913 >> 172.21.15.164:0/4167517913 conn(0x562fc5910310 msgr2=0x562fc5943c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:45.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.200+0000 7f4fefa27740 1 -- 172.21.15.164:0/4167517913 shutdown_connections 2024-09-19T08:48:45.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.201+0000 7f4fefa27740 1 -- 172.21.15.164:0/4167517913 wait complete. 2024-09-19T08:48:45.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.201+0000 7f4fefa27740 1 -- start start 2024-09-19T08:48:45.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.201+0000 7f4fefa27740 1 Processor -- start 2024-09-19T08:48:45.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.203+0000 7f4fefa27740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562fc59475d0 0x562fc5951e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:45.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.203+0000 7f4fefa27740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fc5948b40 0x562fc594f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:45.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.203+0000 7f4fefa27740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fc5952370 0x562fc594f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:45.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.203+0000 7f4fefa27740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562fc5955070 con 0x562fc5952370 2024-09-19T08:48:45.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.203+0000 7f4fefa27740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562fc5868060 con 0x562fc5948b40 2024-09-19T08:48:45.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.203+0000 7f4fefa27740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562fc59551f0 con 0x562fc59475d0 2024-09-19T08:48:45.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.204+0000 7f4fefa27740 10 client.0 _ll_register_callbacks cb 0x562fc55bb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:48:45.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.204+0000 7f4fe6ffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562fc59475d0 0x562fc5951e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:45.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.204+0000 7f4fe6ffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562fc59475d0 0x562fc5951e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:48498/0 (socket says 172.21.15.164:48498) 2024-09-19T08:48:45.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.204+0000 7f4fe6ffd640 1 -- 172.21.15.164:0/154313068 learned_addr learned my addr 172.21.15.164:0/154313068 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:45.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.205+0000 7f4fe77fe640 1 --2- 172.21.15.164:0/154313068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fc5952370 0x562fc594f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:45.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.205+0000 7f4fe77fe640 1 -- 172.21.15.164:0/154313068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562fc59475d0 msgr2=0x562fc5951e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:45.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.205+0000 7f4fe77fe640 1 --2- 172.21.15.164:0/154313068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562fc59475d0 0x562fc5951e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:45.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.205+0000 7f4fe77fe640 1 -- 172.21.15.164:0/154313068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fc5948b40 msgr2=0x562fc594f0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:48:45.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.205+0000 7f4fe77fe640 1 --2- 172.21.15.164:0/154313068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fc5948b40 0x562fc594f0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:45.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.206+0000 7f4fe77fe640 1 -- 172.21.15.164:0/154313068 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fc561e3c0 con 0x562fc5952370 2024-09-19T08:48:45.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.206+0000 7f4fe77fe640 1 --2- 172.21.15.164:0/154313068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fc5952370 0x562fc594f840 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc00e1d0 tx=0x7f4fdc00e690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc25c3d3b68873 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:45.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.206+0000 7f4fcaff5640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:48:45.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.207+0000 7f4fcaff5640 1 -- 172.21.15.164:0/154313068 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc017070 con 0x562fc5952370 2024-09-19T08:48:45.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.207+0000 7f4fefa27740 1 -- 172.21.15.164:0/154313068 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fe009f030 con 0x562fc5952370 2024-09-19T08:48:45.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.207+0000 7f4fcaff5640 1 -- 172.21.15.164:0/154313068 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f4fdc00ee40 con 0x562fc5952370 2024-09-19T08:48:45.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.208+0000 7f4fcaff5640 1 -- 172.21.15.164:0/154313068 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4fdc023050 con 0x562fc5952370 2024-09-19T08:48:45.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.208+0000 7f4fefa27740 10 client.16170 fetch_fsmap learned FSMap version 28 2024-09-19T08:48:45.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.208+0000 7f4fefa27740 1 -- 172.21.15.164:0/154313068 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562fc5960d00 con 0x562fc5952370 2024-09-19T08:48:45.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.210+0000 7f4fcaff5640 1 -- 172.21.15.164:0/154313068 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe009f030 con 0x562fc5952370 2024-09-19T08:48:45.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.211+0000 7f4fcaff5640 1 -- 172.21.15.164:0/154313068 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4fdc0129a0 con 0x562fc5952370 2024-09-19T08:48:45.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.211+0000 7f4fefa27740 10 client.16170 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:48:45.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.211+0000 7f4fefa27740 10 client.16170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:48:45.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.212+0000 7f4fefa27740 20 client.16170 populate_metadata read hostname 'smithi164' 2024-09-19T08:48:45.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.212+0000 7f4fefa27740 1 --2- 172.21.15.164:0/154313068 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x562fc5962580 0x562fc5964a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:45.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.212+0000 7f4fefa27740 4 client.16170 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:48:45.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.212+0000 7f4fefa27740 1 -- 172.21.15.164:0/154313068 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fc58ba860 con 0x562fc5962580 2024-09-19T08:48:45.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.216+0000 7f4fe6ffd640 1 --2- 172.21.15.164:0/154313068 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x562fc5962580 0x562fc5964a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:45.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.219+0000 7f4fe6ffd640 1 --2- 172.21.15.164:0/154313068 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x562fc5962580 0x562fc5964a00 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=330e64d39fc2174e server_cookie=5711bab6bdacfdcf in_seq=0 out_seq=0 2024-09-19T08:48:45.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.219+0000 7f4fcaff5640 10 client.16170 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:48:45.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.220+0000 7f4fcaff5640 1 -- 172.21.15.164:0/154313068 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562fc58ba860 con 0x562fc5962580 2024-09-19T08:48:45.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.220+0000 7f4fcaff5640 10 client.16170 handle_command_reply: tid=0 2024-09-19T08:48:45.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.220+0000 7f4fefa27740 1 client.16170 shutdown 2024-09-19T08:48:45.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.220+0000 7f4fefa27740 1 -- 172.21.15.164:0/154313068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fc5952370 msgr2=0x562fc594f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:45.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.220+0000 7f4fefa27740 1 --2- 172.21.15.164:0/154313068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fc5952370 0x562fc594f840 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc00e1d0 tx=0x7f4fdc00e690 comp rx=0 tx=0).stop 2024-09-19T08:48:45.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.221+0000 7f4fefa27740 1 -- 172.21.15.164:0/154313068 shutdown_connections 2024-09-19T08:48:45.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.221+0000 7f4fefa27740 1 --2- 172.21.15.164:0/154313068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fc5952370 0x562fc594f840 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:45.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.221+0000 7f4fefa27740 1 --2- 172.21.15.164:0/154313068 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x562fc5962580 0x562fc5964a00 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:45.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.221+0000 7f4fefa27740 1 --2- 172.21.15.164:0/154313068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fc5948b40 0x562fc594f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:45.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.221+0000 7f4fcaff5640 0 client.16170 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:48:45.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.221+0000 7f4fefa27740 1 --2- 172.21.15.164:0/154313068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562fc59475d0 0x562fc5951e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:45.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.221+0000 7f4fefa27740 1 -- 172.21.15.164:0/154313068 >> 172.21.15.164:0/154313068 conn(0x562fc5910310 msgr2=0x562fc5949990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:45.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.224+0000 7f4fefa27740 1 -- 172.21.15.164:0/154313068 shutdown_connections 2024-09-19T08:48:45.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.224+0000 7f4fefa27740 1 -- 172.21.15.164:0/154313068 wait complete. 2024-09-19T08:48:45.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.224+0000 7f4fefa27740 20 client.16170 trim_cache size 0 max 16384 2024-09-19T08:48:45.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.251+0000 7f4fefa27740 1 Processor -- start 2024-09-19T08:48:45.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.251+0000 7f4fefa27740 1 -- start start 2024-09-19T08:48:45.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.252+0000 7f4fefa27740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fc59475d0 0x562fc5943d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:45.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.252+0000 7f4fefa27740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fc5948b40 0x562fc594e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:45.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.252+0000 7f4fefa27740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562fc5952370 0x562fc596a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:45.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.252+0000 7f4fefa27740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd400e490 con 0x562fc59475d0 2024-09-19T08:48:45.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.252+0000 7f4fefa27740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd400ffd0 con 0x562fc5948b40 2024-09-19T08:48:45.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.252+0000 7f4fefa27740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd400eb60 con 0x562fc5952370 2024-09-19T08:48:45.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.252+0000 7f4fe6ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fc59475d0 0x562fc5943d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:45.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.252+0000 7f4fe6ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fc59475d0 0x562fc5943d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42128/0 (socket says 172.21.15.164:42128) 2024-09-19T08:48:45.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.252+0000 7f4fe6ffd640 1 -- 172.21.15.164:0/2048341690 learned_addr learned my addr 172.21.15.164:0/2048341690 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:45.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.253+0000 7f4fe67fc640 1 --2- 172.21.15.164:0/2048341690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fc5948b40 0x562fc594e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:45.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.253+0000 7f4fe77fe640 1 --2- 172.21.15.164:0/2048341690 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562fc5952370 0x562fc596a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:45.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.253+0000 7f4fe6ffd640 1 -- 172.21.15.164:0/2048341690 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562fc5952370 msgr2=0x562fc596a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:45.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.253+0000 7f4fe6ffd640 1 --2- 172.21.15.164:0/2048341690 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562fc5952370 0x562fc596a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:45.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.253+0000 7f4fe6ffd640 1 -- 172.21.15.164:0/2048341690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fc5948b40 msgr2=0x562fc594e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:45.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.253+0000 7f4fe6ffd640 1 --2- 172.21.15.164:0/2048341690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fc5948b40 0x562fc594e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:45.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.253+0000 7f4fe6ffd640 1 -- 172.21.15.164:0/2048341690 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fc5952170 con 0x562fc59475d0 2024-09-19T08:48:45.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.254+0000 7f4fe6ffd640 1 --2- 172.21.15.164:0/2048341690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fc59475d0 0x562fc5943d70 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f4fd802aa00 tx=0x7f4fd803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b449d00de2e3918 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:45.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.254+0000 7f4fccff9640 1 -- 172.21.15.164:0/2048341690 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd80305a0 con 0x562fc59475d0 2024-09-19T08:48:45.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.254+0000 7f4fccff9640 1 -- 172.21.15.164:0/2048341690 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fd8059070 con 0x562fc59475d0 2024-09-19T08:48:45.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.254+0000 7f4fccff9640 1 -- 172.21.15.164:0/2048341690 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8053690 con 0x562fc59475d0 2024-09-19T08:48:45.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.255+0000 7f4fefa27740 1 -- 172.21.15.164:0/2048341690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fc59475d0 msgr2=0x562fc5943d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:45.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.255+0000 7f4fefa27740 1 --2- 172.21.15.164:0/2048341690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fc59475d0 0x562fc5943d70 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f4fd802aa00 tx=0x7f4fd803e000 comp rx=0 tx=0).stop 2024-09-19T08:48:45.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.257+0000 7f4fefa27740 1 -- 172.21.15.164:0/2048341690 shutdown_connections 2024-09-19T08:48:45.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.257+0000 7f4fefa27740 1 --2- 172.21.15.164:0/2048341690 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562fc5952370 0x562fc596a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:45.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.257+0000 7f4fefa27740 1 --2- 172.21.15.164:0/2048341690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fc5948b40 0x562fc594e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:45.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.257+0000 7f4fefa27740 1 --2- 172.21.15.164:0/2048341690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fc59475d0 0x562fc5943d70 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:45.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.257+0000 7f4fefa27740 1 -- 172.21.15.164:0/2048341690 >> 172.21.15.164:0/2048341690 conn(0x562fc5910310 msgr2=0x562fc594d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:45.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.258+0000 7f4fefa27740 1 -- 172.21.15.164:0/2048341690 shutdown_connections 2024-09-19T08:48:45.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.258+0000 7f4fefa27740 1 -- 172.21.15.164:0/2048341690 wait complete. 2024-09-19T08:48:45.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.258+0000 7f4fefa27740 1 -- start start 2024-09-19T08:48:45.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.258+0000 7f4fefa27740 1 Processor -- start 2024-09-19T08:48:45.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.259+0000 7f4fefa27740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562fc59475d0 0x562fc595aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:45.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.259+0000 7f4fefa27740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fc5948b40 0x562fc594bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:45.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.259+0000 7f4fefa27740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fc5952370 0x562fc594c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:45.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.259+0000 7f4fefa27740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd400f900 con 0x562fc5948b40 2024-09-19T08:48:45.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.259+0000 7f4fefa27740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd400f230 con 0x562fc5952370 2024-09-19T08:48:45.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.259+0000 7f4fefa27740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd400c520 con 0x562fc59475d0 2024-09-19T08:48:45.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.259+0000 7f4fefa27740 10 client.0 _ll_register_callbacks cb 0x7f4fdc009330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:48:45.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.261+0000 7f4fe6ffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562fc59475d0 0x562fc595aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:45.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.261+0000 7f4fe6ffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562fc59475d0 0x562fc595aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:48502/0 (socket says 172.21.15.164:48502) 2024-09-19T08:48:45.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.261+0000 7f4fe6ffd640 1 -- 172.21.15.164:0/797044804 learned_addr learned my addr 172.21.15.164:0/797044804 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:45.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.262+0000 7f4fe77fe640 1 --2- 172.21.15.164:0/797044804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fc5952370 0x562fc594c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:45.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.262+0000 7f4fe6ffd640 1 -- 172.21.15.164:0/797044804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fc5952370 msgr2=0x562fc594c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:45.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.262+0000 7f4fe6ffd640 1 --2- 172.21.15.164:0/797044804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fc5952370 0x562fc594c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:45.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.262+0000 7f4fe6ffd640 1 -- 172.21.15.164:0/797044804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fc5948b40 msgr2=0x562fc594bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:45.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.262+0000 7f4fe6ffd640 1 --2- 172.21.15.164:0/797044804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fc5948b40 0x562fc594bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:45.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.262+0000 7f4fe6ffd640 1 -- 172.21.15.164:0/797044804 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fc594c7a0 con 0x562fc59475d0 2024-09-19T08:48:45.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.264+0000 7f4fe6ffd640 1 --2- 172.21.15.164:0/797044804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562fc59475d0 0x562fc595aa40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8032ff0 tx=0x7f4fd80305d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddc428e37ba36ea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:45.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.264+0000 7f4fcaff5640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-09-19T08:48:45.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.265+0000 7f4fcaff5640 1 -- 172.21.15.164:0/797044804 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd80037f0 con 0x562fc59475d0 2024-09-19T08:48:45.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.265+0000 7f4fcaff5640 1 -- 172.21.15.164:0/797044804 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f4fd8003990 con 0x562fc59475d0 2024-09-19T08:48:45.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.266+0000 7f4fefa27740 1 -- 172.21.15.164:0/797044804 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fd8059070 con 0x562fc59475d0 2024-09-19T08:48:45.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.267+0000 7f4fcaff5640 1 -- 172.21.15.164:0/797044804 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4fd8003bf0 con 0x562fc59475d0 2024-09-19T08:48:45.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.267+0000 7f4fefa27740 10 client.25597 fetch_fsmap learned FSMap version 28 2024-09-19T08:48:45.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.268+0000 7f4fefa27740 1 -- 172.21.15.164:0/797044804 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562fc59cd190 con 0x562fc59475d0 2024-09-19T08:48:45.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.268+0000 7f4fcaff5640 1 -- 172.21.15.164:0/797044804 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fd8059070 con 0x562fc59475d0 2024-09-19T08:48:45.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.269+0000 7f4fcaff5640 1 -- 172.21.15.164:0/797044804 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4fd8031400 con 0x562fc59475d0 2024-09-19T08:48:45.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.269+0000 7f4fefa27740 10 client.25597 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:48:45.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.269+0000 7f4fefa27740 10 client.25597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:48:45.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.269+0000 7f4fefa27740 20 client.25597 populate_metadata read hostname 'smithi164' 2024-09-19T08:48:45.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.269+0000 7f4fefa27740 1 --2- 172.21.15.164:0/797044804 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x562fc5962580 0x562fc59cf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:45.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.269+0000 7f4fefa27740 4 client.25597 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:48:45.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.269+0000 7f4fefa27740 1 -- 172.21.15.164:0/797044804 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562fc58b4d70 con 0x562fc5962580 2024-09-19T08:48:45.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.270+0000 7f4fe67fc640 1 --2- 172.21.15.164:0/797044804 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x562fc5962580 0x562fc59cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:45.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.271+0000 7f4fe67fc640 1 --2- 172.21.15.164:0/797044804 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x562fc5962580 0x562fc59cf820 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6050b273a2a65aed server_cookie=d384cfc21ab7bd88 in_seq=0 out_seq=0 2024-09-19T08:48:45.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.271+0000 7f4fcaff5640 10 client.25597 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:48:45.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.271+0000 7f4fcaff5640 1 -- 172.21.15.164:0/797044804 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562fc58b4d70 con 0x562fc5962580 2024-09-19T08:48:45.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.271+0000 7f4fcaff5640 10 client.25597 handle_command_reply: tid=0 2024-09-19T08:48:45.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.272+0000 7f4fefa27740 1 client.25597 shutdown 2024-09-19T08:48:45.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.272+0000 7f4fefa27740 1 -- 172.21.15.164:0/797044804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562fc59475d0 msgr2=0x562fc595aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:45.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.272+0000 7f4fefa27740 1 --2- 172.21.15.164:0/797044804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562fc59475d0 0x562fc595aa40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8032ff0 tx=0x7f4fd80305d0 comp rx=0 tx=0).stop 2024-09-19T08:48:45.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.272+0000 7f4fefa27740 1 -- 172.21.15.164:0/797044804 shutdown_connections 2024-09-19T08:48:45.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.272+0000 7f4fefa27740 1 --2- 172.21.15.164:0/797044804 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x562fc5962580 0x562fc59cf820 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:45.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.272+0000 7f4fefa27740 1 --2- 172.21.15.164:0/797044804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fc5952370 0x562fc594c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:45.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.272+0000 7f4fefa27740 1 --2- 172.21.15.164:0/797044804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562fc5948b40 0x562fc594bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:45.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.272+0000 7f4fefa27740 1 --2- 172.21.15.164:0/797044804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562fc59475d0 0x562fc595aa40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:45.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.272+0000 7f4fefa27740 1 -- 172.21.15.164:0/797044804 >> 172.21.15.164:0/797044804 conn(0x562fc5910310 msgr2=0x562fc594e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:45.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.273+0000 7f4fcaff5640 0 client.25597 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:48:45.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.273+0000 7f4fefa27740 1 -- 172.21.15.164:0/797044804 shutdown_connections 2024-09-19T08:48:45.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.273+0000 7f4fefa27740 1 -- 172.21.15.164:0/797044804 wait complete. 2024-09-19T08:48:45.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.274+0000 7f4fefa27740 20 client.25597 trim_cache size 0 max 16384 2024-09-19T08:48:45.277 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:48:45.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.278+0000 7f4fed64e640 1 -- 172.21.15.164:0/713061832 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f4fd00a35c0 msgr2=0x7f4fd00a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:45.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.278+0000 7f4fed64e640 1 --2- 172.21.15.164:0/713061832 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f4fd00a35c0 0x7f4fd00a5a80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8031000 tx=0x7f4fd8030e60 comp rx=0 tx=0).stop 2024-09-19T08:48:45.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.278+0000 7f4fed64e640 1 -- 172.21.15.164:0/713061832 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fe806c200 msgr2=0x7f4fe811afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:45.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.278+0000 7f4fed64e640 1 --2- 172.21.15.164:0/713061832 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fe806c200 0x7f4fe811afd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000efb0 tx=0x7f4fe000c800 comp rx=0 tx=0).stop 2024-09-19T08:48:45.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.280+0000 7f4fed64e640 1 -- 172.21.15.164:0/713061832 shutdown_connections 2024-09-19T08:48:45.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.280+0000 7f4fed64e640 1 --2- 172.21.15.164:0/713061832 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f4fd00a35c0 0x7f4fd00a5a80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:45.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.280+0000 7f4fed64e640 1 --2- 172.21.15.164:0/713061832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fe811b550 0x7f4fe8113d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:45.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.280+0000 7f4fed64e640 1 --2- 172.21.15.164:0/713061832 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fe806c200 0x7f4fe811afd0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:45.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.280+0000 7f4fed64e640 1 --2- 172.21.15.164:0/713061832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fe806b8e0 0x7f4fe811aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:45.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.280+0000 7f4fed64e640 1 -- 172.21.15.164:0/713061832 >> 172.21.15.164:0/713061832 conn(0x7f4fe8071630 msgr2=0x7f4fe8072230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:45.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.282+0000 7f4fed64e640 1 -- 172.21.15.164:0/713061832 shutdown_connections 2024-09-19T08:48:45.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:45.282+0000 7f4fed64e640 1 -- 172.21.15.164:0/713061832 wait complete. 2024-09-19T08:48:46.212 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:45 smithi173 ceph-mon[35306]: pgmap v91: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 862 MiB/s wr, 2.78k op/s 2024-09-19T08:48:46.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:45 smithi164 ceph-mon[30928]: pgmap v91: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 862 MiB/s wr, 2.78k op/s 2024-09-19T08:48:46.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:45 smithi170 ceph-mon[35072]: pgmap v91: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 862 MiB/s wr, 2.78k op/s 2024-09-19T08:48:48.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:47 smithi164 ceph-mon[30928]: pgmap v92: 97 pgs: 97 active+clean; 20 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 759 MiB/s wr, 2.98k op/s 2024-09-19T08:48:48.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:47 smithi170 ceph-mon[35072]: pgmap v92: 97 pgs: 97 active+clean; 20 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 759 MiB/s wr, 2.98k op/s 2024-09-19T08:48:48.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:47 smithi173 ceph-mon[35306]: pgmap v92: 97 pgs: 97 active+clean; 20 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 759 MiB/s wr, 2.98k op/s 2024-09-19T08:48:48.560 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T08:48:48.561 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T08:48:49.669 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:48:50.288 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:49 smithi164 ceph-mon[30928]: pgmap v93: 97 pgs: 97 active+clean; 20 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 632 MiB/s wr, 2.75k op/s 2024-09-19T08:48:50.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:49 smithi170 ceph-mon[35072]: pgmap v93: 97 pgs: 97 active+clean; 20 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 632 MiB/s wr, 2.75k op/s 2024-09-19T08:48:50.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:49 smithi173 ceph-mon[35306]: pgmap v93: 97 pgs: 97 active+clean; 20 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 632 MiB/s wr, 2.75k op/s 2024-09-19T08:48:51.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.109+0000 7f3e45a95640 1 -- 172.21.15.164:0/512517866 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e4010d9b0 msgr2=0x7f3e40115510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:51.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.109+0000 7f3e45a95640 1 --2- 172.21.15.164:0/512517866 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e4010d9b0 0x7f3e40115510 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3e380076d0 tx=0x7f3e3802b370 comp rx=0 tx=0).stop 2024-09-19T08:48:51.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.111+0000 7f3e45a95640 1 -- 172.21.15.164:0/512517866 shutdown_connections 2024-09-19T08:48:51.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.111+0000 7f3e45a95640 1 --2- 172.21.15.164:0/512517866 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e4010d9b0 0x7f3e40115510 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:51.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.111+0000 7f3e45a95640 1 --2- 172.21.15.164:0/512517866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e4010ef30 0x7f3e4010d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:51.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.111+0000 7f3e45a95640 1 --2- 172.21.15.164:0/512517866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e4010e580 0x7f3e4010e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:51.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.111+0000 7f3e45a95640 1 -- 172.21.15.164:0/512517866 >> 172.21.15.164:0/512517866 conn(0x7f3e4006e140 msgr2=0x7f3e4006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:51.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.115+0000 7f3e45a95640 1 -- 172.21.15.164:0/512517866 shutdown_connections 2024-09-19T08:48:51.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.116+0000 7f3e45a95640 1 -- 172.21.15.164:0/512517866 wait complete. 2024-09-19T08:48:51.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.117+0000 7f3e45a95640 1 Processor -- start 2024-09-19T08:48:51.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.117+0000 7f3e45a95640 1 -- start start 2024-09-19T08:48:51.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.117+0000 7f3e45a95640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e4010d9b0 0x7f3e4011e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:51.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.118+0000 7f3e45a95640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e4010e580 0x7f3e4011ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:51.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.118+0000 7f3e45a95640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e4010ef30 0x7f3e40117800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:51.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.118+0000 7f3e45a95640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3e40120d70 con 0x7f3e4010e580 2024-09-19T08:48:51.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.118+0000 7f3e45a95640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3e40120bf0 con 0x7f3e4010d9b0 2024-09-19T08:48:51.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.118+0000 7f3e45a95640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3e40120ef0 con 0x7f3e4010ef30 2024-09-19T08:48:51.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.119+0000 7f3e3effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e4010e580 0x7f3e4011ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:51.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.119+0000 7f3e3effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e4010e580 0x7f3e4011ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42150/0 (socket says 172.21.15.164:42150) 2024-09-19T08:48:51.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.119+0000 7f3e3effd640 1 -- 172.21.15.164:0/6066425 learned_addr learned my addr 172.21.15.164:0/6066425 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:51.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.119+0000 7f3e3ffff640 1 --2- 172.21.15.164:0/6066425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e4010ef30 0x7f3e40117800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:51.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.120+0000 7f3e3effd640 1 -- 172.21.15.164:0/6066425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e4010ef30 msgr2=0x7f3e40117800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:51.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.120+0000 7f3e3effd640 1 --2- 172.21.15.164:0/6066425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e4010ef30 0x7f3e40117800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:51.121 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.120+0000 7f3e3effd640 1 -- 172.21.15.164:0/6066425 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e4010d9b0 msgr2=0x7f3e4011e590 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T08:48:51.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.120+0000 7f3e3effd640 1 --2- 172.21.15.164:0/6066425 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e4010d9b0 0x7f3e4011e590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:51.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.120+0000 7f3e3effd640 1 -- 172.21.15.164:0/6066425 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e401180c0 con 0x7f3e4010e580 2024-09-19T08:48:51.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.122+0000 7f3e3effd640 1 --2- 172.21.15.164:0/6066425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e4010e580 0x7f3e4011ead0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400c970 tx=0x7f3e3400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:51.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.123+0000 7f3e3cff9640 1 -- 172.21.15.164:0/6066425 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34007bb0 con 0x7f3e4010e580 2024-09-19T08:48:51.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.123+0000 7f3e45a95640 1 -- 172.21.15.164:0/6066425 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e401183b0 con 0x7f3e4010e580 2024-09-19T08:48:51.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.124+0000 7f3e45a95640 1 -- 172.21.15.164:0/6066425 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e4011c340 con 0x7f3e4010e580 2024-09-19T08:48:51.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.123+0000 7f3e3cff9640 1 -- 172.21.15.164:0/6066425 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e34007d50 con 0x7f3e4010e580 2024-09-19T08:48:51.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.133+0000 7f3e3cff9640 1 -- 172.21.15.164:0/6066425 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34007bb0 con 0x7f3e4010e580 2024-09-19T08:48:51.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.136+0000 7f3e47e6e740 1 Processor -- start 2024-09-19T08:48:51.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.137+0000 7f3e47e6e740 1 -- start start 2024-09-19T08:48:51.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.137+0000 7f3e47e6e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bfa05e5b60 0x55bfa05e5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:51.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.137+0000 7f3e47e6e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa05e6480 0x55bfa05f0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:51.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.137+0000 7f3e47e6e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bfa05f0d40 0x55bfa05f3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:51.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.137+0000 7f3e47e6e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bfa054b280 con 0x55bfa05e6480 2024-09-19T08:48:51.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.137+0000 7f3e47e6e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55bfa04f2b10 con 0x55bfa05f0d40 2024-09-19T08:48:51.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.137+0000 7f3e47e6e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55bfa0547240 con 0x55bfa05e5b60 2024-09-19T08:48:51.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.139+0000 7f3e3ffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bfa05e5b60 0x55bfa05e5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:51.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.139+0000 7f3e3ffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bfa05e5b60 0x55bfa05e5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:48528/0 (socket says 172.21.15.164:48528) 2024-09-19T08:48:51.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.139+0000 7f3e3ffff640 1 -- 172.21.15.164:0/1905569751 learned_addr learned my addr 172.21.15.164:0/1905569751 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:51.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.141+0000 7f3e3f7fe640 1 --2- 172.21.15.164:0/1905569751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa05e6480 0x55bfa05f0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:51.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.143+0000 7f3e3f7fe640 1 -- 172.21.15.164:0/1905569751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bfa05e5b60 msgr2=0x55bfa05e5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:51.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.143+0000 7f3e3f7fe640 1 --2- 172.21.15.164:0/1905569751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bfa05e5b60 0x55bfa05e5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:51.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.143+0000 7f3e3f7fe640 1 -- 172.21.15.164:0/1905569751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bfa05f0d40 msgr2=0x55bfa05f3130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:48:51.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.143+0000 7f3e3f7fe640 1 --2- 172.21.15.164:0/1905569751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bfa05f0d40 0x55bfa05f3130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:51.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.143+0000 7f3e3f7fe640 1 -- 172.21.15.164:0/1905569751 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfa04ff1f0 con 0x55bfa05e6480 2024-09-19T08:48:51.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.143+0000 7f3e3cff9640 1 -- 172.21.15.164:0/6066425 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f3e3400f450 con 0x7f3e4010e580 2024-09-19T08:48:51.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.144+0000 7f3e3f7fe640 1 --2- 172.21.15.164:0/1905569751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa05e6480 0x55bfa05f0800 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3e30009a00 tx=0x7f3e30009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899447b3a58fed0d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:51.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.144+0000 7f3e3cff9640 1 --2- 172.21.15.164:0/6066425 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3e280a35b0 0x7f3e280a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:51.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.145+0000 7f3e3cff9640 1 -- 172.21.15.164:0/6066425 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f3e34099fa0 con 0x7f3e4010e580 2024-09-19T08:48:51.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.145+0000 7f3e25ffb640 1 -- 172.21.15.164:0/1905569751 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3000ec50 con 0x55bfa05e6480 2024-09-19T08:48:51.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.145+0000 7f3e25ffb640 1 -- 172.21.15.164:0/1905569751 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e3000edf0 con 0x55bfa05e6480 2024-09-19T08:48:51.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.146+0000 7f3e3ffff640 1 --2- 172.21.15.164:0/6066425 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3e280a35b0 0x7f3e280a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:51.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.146+0000 7f3e25ffb640 1 -- 172.21.15.164:0/1905569751 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e30004a20 con 0x55bfa05e6480 2024-09-19T08:48:51.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.146+0000 7f3e47e6e740 1 -- 172.21.15.164:0/1905569751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa05e6480 msgr2=0x55bfa05f0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:51.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.146+0000 7f3e47e6e740 1 --2- 172.21.15.164:0/1905569751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa05e6480 0x55bfa05f0800 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3e30009a00 tx=0x7f3e30009ec0 comp rx=0 tx=0).stop 2024-09-19T08:48:51.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.148+0000 7f3e47e6e740 1 -- 172.21.15.164:0/1905569751 shutdown_connections 2024-09-19T08:48:51.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.148+0000 7f3e47e6e740 1 --2- 172.21.15.164:0/1905569751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bfa05f0d40 0x55bfa05f3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:51.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.148+0000 7f3e47e6e740 1 --2- 172.21.15.164:0/1905569751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa05e6480 0x55bfa05f0800 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:51.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.148+0000 7f3e47e6e740 1 --2- 172.21.15.164:0/1905569751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bfa05e5b60 0x55bfa05e5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:51.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.148+0000 7f3e47e6e740 1 -- 172.21.15.164:0/1905569751 >> 172.21.15.164:0/1905569751 conn(0x55bfa05e20f0 msgr2=0x55bfa05e24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:51.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.148+0000 7f3e47e6e740 1 -- 172.21.15.164:0/1905569751 shutdown_connections 2024-09-19T08:48:51.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.148+0000 7f3e47e6e740 1 -- 172.21.15.164:0/1905569751 wait complete. 2024-09-19T08:48:51.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.148+0000 7f3e3ffff640 1 --2- 172.21.15.164:0/6066425 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3e280a35b0 0x7f3e280a5a70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3e38005f00 tx=0x7f3e38004a40 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:51.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.149+0000 7f3e47e6e740 1 -- start start 2024-09-19T08:48:51.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.149+0000 7f3e47e6e740 1 Processor -- start 2024-09-19T08:48:51.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.150+0000 7f3e47e6e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa05e5b60 0x55bfa05f0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:51.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.150+0000 7f3e47e6e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bfa05e6480 0x55bfa05edad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:51.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.150+0000 7f3e47e6e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bfa05f0d40 0x55bfa05ee260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:51.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.151+0000 7f3e47e6e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bfa05f3960 con 0x55bfa05e5b60 2024-09-19T08:48:51.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.151+0000 7f3e47e6e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55bfa0505f10 con 0x55bfa05e6480 2024-09-19T08:48:51.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.151+0000 7f3e47e6e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55bfa05f3ae0 con 0x55bfa05f0d40 2024-09-19T08:48:51.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.151+0000 7f3e47e6e740 10 client.0 _ll_register_callbacks cb 0x55bfa04cea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:48:51.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.152+0000 7f3e3f7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa05e5b60 0x55bfa05f0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:51.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.152+0000 7f3e3effd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bfa05e6480 0x55bfa05edad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:51.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.152+0000 7f3e3effd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bfa05e6480 0x55bfa05edad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:47392/0 (socket says 172.21.15.164:47392) 2024-09-19T08:48:51.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.152+0000 7f3e3effd640 1 -- 172.21.15.164:0/339847010 learned_addr learned my addr 172.21.15.164:0/339847010 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:51.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.153+0000 7f3e3f7fe640 1 -- 172.21.15.164:0/339847010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bfa05f0d40 msgr2=0x55bfa05ee260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:48:51.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.153+0000 7f3e3f7fe640 1 --2- 172.21.15.164:0/339847010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bfa05f0d40 0x55bfa05ee260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:51.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.153+0000 7f3e3f7fe640 1 -- 172.21.15.164:0/339847010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bfa05e6480 msgr2=0x55bfa05edad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:51.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.153+0000 7f3e3f7fe640 1 --2- 172.21.15.164:0/339847010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bfa05e6480 0x55bfa05edad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:51.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.153+0000 7f3e3f7fe640 1 -- 172.21.15.164:0/339847010 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfa02bc6a0 con 0x55bfa05e5b60 2024-09-19T08:48:51.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.154+0000 7f3e3f7fe640 1 --2- 172.21.15.164:0/339847010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa05e5b60 0x55bfa05f0780 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3e30006010 tx=0x7f3e3001a490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35bd64c5f3512da8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:51.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.155+0000 7f3e1effd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:48:51.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.162+0000 7f3e1effd640 1 -- 172.21.15.164:0/339847010 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e30028060 con 0x55bfa05e5b60 2024-09-19T08:48:51.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.162+0000 7f3e47e6e740 1 -- 172.21.15.164:0/339847010 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e3000edf0 con 0x55bfa05e5b60 2024-09-19T08:48:51.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.162+0000 7f3e1effd640 1 -- 172.21.15.164:0/339847010 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f3e3001a5c0 con 0x55bfa05e5b60 2024-09-19T08:48:51.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.166+0000 7f3e1effd640 1 -- 172.21.15.164:0/339847010 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3e3001a7c0 con 0x55bfa05e5b60 2024-09-19T08:48:51.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.167+0000 7f3e47e6e740 10 client.16185 fetch_fsmap learned FSMap version 28 2024-09-19T08:48:51.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.167+0000 7f3e47e6e740 1 -- 172.21.15.164:0/339847010 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bfa05ff4c0 con 0x55bfa05e5b60 2024-09-19T08:48:51.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.168+0000 7f3e1effd640 1 -- 172.21.15.164:0/339847010 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e3000edf0 con 0x55bfa05e5b60 2024-09-19T08:48:51.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.169+0000 7f3e1effd640 1 -- 172.21.15.164:0/339847010 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3e30018af0 con 0x55bfa05e5b60 2024-09-19T08:48:51.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.170+0000 7f3e47e6e740 10 client.16185 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:48:51.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.170+0000 7f3e47e6e740 10 client.16185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:48:51.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.170+0000 7f3e47e6e740 20 client.16185 populate_metadata read hostname 'smithi164' 2024-09-19T08:48:51.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.170+0000 7f3e47e6e740 1 --2- 172.21.15.164:0/339847010 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55bfa0600bc0 0x55bfa0603040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:51.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.170+0000 7f3e47e6e740 4 client.16185 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:48:51.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.170+0000 7f3e47e6e740 1 -- 172.21.15.164:0/339847010 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfa05587f0 con 0x55bfa0600bc0 2024-09-19T08:48:51.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.170+0000 7f3e3effd640 1 --2- 172.21.15.164:0/339847010 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55bfa0600bc0 0x55bfa0603040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:51.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.171+0000 7f3e3effd640 1 --2- 172.21.15.164:0/339847010 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55bfa0600bc0 0x55bfa0603040 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed375741e3a9c48a server_cookie=2bc07a85a4566b44 in_seq=0 out_seq=0 2024-09-19T08:48:51.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.171+0000 7f3e1effd640 10 client.16185 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:48:51.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.175+0000 7f3e1effd640 1 -- 172.21.15.164:0/339847010 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bfa05587f0 con 0x55bfa0600bc0 2024-09-19T08:48:51.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.175+0000 7f3e1effd640 10 client.16185 handle_command_reply: tid=0 2024-09-19T08:48:51.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.175+0000 7f3e47e6e740 1 client.16185 shutdown 2024-09-19T08:48:51.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.175+0000 7f3e47e6e740 1 -- 172.21.15.164:0/339847010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa05e5b60 msgr2=0x55bfa05f0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:51.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.175+0000 7f3e47e6e740 1 --2- 172.21.15.164:0/339847010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa05e5b60 0x55bfa05f0780 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3e30006010 tx=0x7f3e3001a490 comp rx=0 tx=0).stop 2024-09-19T08:48:51.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.176+0000 7f3e47e6e740 1 -- 172.21.15.164:0/339847010 shutdown_connections 2024-09-19T08:48:51.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.176+0000 7f3e47e6e740 1 --2- 172.21.15.164:0/339847010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bfa05f0d40 0x55bfa05ee260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:51.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.176+0000 7f3e47e6e740 1 --2- 172.21.15.164:0/339847010 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55bfa0600bc0 0x55bfa0603040 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:51.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.176+0000 7f3e47e6e740 1 --2- 172.21.15.164:0/339847010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bfa05e6480 0x55bfa05edad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:51.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.176+0000 7f3e47e6e740 1 --2- 172.21.15.164:0/339847010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa05e5b60 0x55bfa05f0780 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:51.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.176+0000 7f3e1effd640 0 client.16185 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:48:51.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.176+0000 7f3e47e6e740 1 -- 172.21.15.164:0/339847010 >> 172.21.15.164:0/339847010 conn(0x55bfa05e20f0 msgr2=0x55bfa05e7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:51.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.176+0000 7f3e47e6e740 1 -- 172.21.15.164:0/339847010 shutdown_connections 2024-09-19T08:48:51.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.176+0000 7f3e47e6e740 1 -- 172.21.15.164:0/339847010 wait complete. 2024-09-19T08:48:51.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.178+0000 7f3e47e6e740 20 client.16185 trim_cache size 0 max 16384 2024-09-19T08:48:51.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.234+0000 7f3e47e6e740 1 Processor -- start 2024-09-19T08:48:51.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.235+0000 7f3e47e6e740 1 -- start start 2024-09-19T08:48:51.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.235+0000 7f3e47e6e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bfa05e5b60 0x55bfa05ec3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:51.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.236+0000 7f3e47e6e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa05e6480 0x55bfa05ec900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:51.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.236+0000 7f3e47e6e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bfa05f0d40 0x55bfa0608950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:51.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.236+0000 7f3e47e6e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c00e550 con 0x55bfa05e6480 2024-09-19T08:48:51.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.236+0000 7f3e47e6e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c010090 con 0x55bfa05e5b60 2024-09-19T08:48:51.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.236+0000 7f3e47e6e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c00ec20 con 0x55bfa05f0d40 2024-09-19T08:48:51.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.240+0000 7f3e3ffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bfa05f0d40 0x55bfa0608950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:51.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.240+0000 7f3e3f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bfa05e5b60 0x55bfa05ec3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:51.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.240+0000 7f3e3effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa05e6480 0x55bfa05ec900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:51.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.240+0000 7f3e3effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa05e6480 0x55bfa05ec900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42180/0 (socket says 172.21.15.164:42180) 2024-09-19T08:48:51.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.240+0000 7f3e3effd640 1 -- 172.21.15.164:0/1572953095 learned_addr learned my addr 172.21.15.164:0/1572953095 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:51.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.245+0000 7f3e3effd640 1 -- 172.21.15.164:0/1572953095 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bfa05f0d40 msgr2=0x55bfa0608950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:51.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.245+0000 7f3e3effd640 1 --2- 172.21.15.164:0/1572953095 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bfa05f0d40 0x55bfa0608950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:51.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.245+0000 7f3e3effd640 1 -- 172.21.15.164:0/1572953095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bfa05e5b60 msgr2=0x55bfa05ec3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:51.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.245+0000 7f3e3effd640 1 --2- 172.21.15.164:0/1572953095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bfa05e5b60 0x55bfa05ec3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:51.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.245+0000 7f3e3effd640 1 -- 172.21.15.164:0/1572953095 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfa05e62b0 con 0x55bfa05e6480 2024-09-19T08:48:51.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.258+0000 7f3e3effd640 1 --2- 172.21.15.164:0/1572953095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa05e6480 0x55bfa05ec900 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3e3401e000 tx=0x7f3e340a1280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9429cdb23a091 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:51.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.263+0000 7f3e257fa640 1 -- 172.21.15.164:0/1572953095 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34098930 con 0x55bfa05e6480 2024-09-19T08:48:51.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.263+0000 7f3e257fa640 1 -- 172.21.15.164:0/1572953095 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e3401ae50 con 0x55bfa05e6480 2024-09-19T08:48:51.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.263+0000 7f3e257fa640 1 -- 172.21.15.164:0/1572953095 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34098ad0 con 0x55bfa05e6480 2024-09-19T08:48:51.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.265+0000 7f3e47e6e740 1 -- 172.21.15.164:0/1572953095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa05e6480 msgr2=0x55bfa05ec900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:51.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.265+0000 7f3e47e6e740 1 --2- 172.21.15.164:0/1572953095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa05e6480 0x55bfa05ec900 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3e3401e000 tx=0x7f3e340a1280 comp rx=0 tx=0).stop 2024-09-19T08:48:51.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.271+0000 7f3e47e6e740 1 -- 172.21.15.164:0/1572953095 shutdown_connections 2024-09-19T08:48:51.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.271+0000 7f3e47e6e740 1 --2- 172.21.15.164:0/1572953095 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bfa05f0d40 0x55bfa0608950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:51.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.271+0000 7f3e47e6e740 1 --2- 172.21.15.164:0/1572953095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa05e6480 0x55bfa05ec900 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:51.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.271+0000 7f3e47e6e740 1 --2- 172.21.15.164:0/1572953095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bfa05e5b60 0x55bfa05ec3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:51.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.271+0000 7f3e47e6e740 1 -- 172.21.15.164:0/1572953095 >> 172.21.15.164:0/1572953095 conn(0x55bfa05e20f0 msgr2=0x55bfa05e8de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:51.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.272+0000 7f3e47e6e740 1 -- 172.21.15.164:0/1572953095 shutdown_connections 2024-09-19T08:48:51.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.272+0000 7f3e47e6e740 1 -- 172.21.15.164:0/1572953095 wait complete. 2024-09-19T08:48:51.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.273+0000 7f3e47e6e740 1 -- start start 2024-09-19T08:48:51.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.274+0000 7f3e47e6e740 1 Processor -- start 2024-09-19T08:48:51.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.285+0000 7f3e47e6e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa05e5b60 0x55bfa05f8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:51.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.285+0000 7f3e47e6e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bfa05e6480 0x55bfa05f9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:51.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.286+0000 7f3e47e6e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bfa05f0d40 0x55bfa05ead70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:51.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.286+0000 7f3e3effd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bfa05e6480 0x55bfa05f9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:51.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.286+0000 7f3e3effd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bfa05e6480 0x55bfa05f9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:48568/0 (socket says 172.21.15.164:48568) 2024-09-19T08:48:51.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.286+0000 7f3e3effd640 1 -- 172.21.15.164:0/3364409369 learned_addr learned my addr 172.21.15.164:0/3364409369 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:51.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.286+0000 7f3e3ffff640 1 --2- 172.21.15.164:0/3364409369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bfa05f0d40 0x55bfa05ead70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:51.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.286+0000 7f3e47e6e740 1 -- 172.21.15.164:0/3364409369 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c00f9c0 con 0x55bfa05e5b60 2024-09-19T08:48:51.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.286+0000 7f3e47e6e740 1 -- 172.21.15.164:0/3364409369 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c00f2f0 con 0x55bfa05f0d40 2024-09-19T08:48:51.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.286+0000 7f3e47e6e740 1 -- 172.21.15.164:0/3364409369 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c00c5e0 con 0x55bfa05e6480 2024-09-19T08:48:51.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.286+0000 7f3e47e6e740 10 client.0 _ll_register_callbacks cb 0x7f3e30005410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:48:51.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.287+0000 7f3e3f7fe640 1 --2- 172.21.15.164:0/3364409369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa05e5b60 0x55bfa05f8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:51.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.288+0000 7f3e3ffff640 1 -- 172.21.15.164:0/3364409369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bfa05e6480 msgr2=0x55bfa05f9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:51.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.288+0000 7f3e3ffff640 1 --2- 172.21.15.164:0/3364409369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bfa05e6480 0x55bfa05f9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:51.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.288+0000 7f3e3ffff640 1 -- 172.21.15.164:0/3364409369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa05e5b60 msgr2=0x55bfa05f8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:51.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.288+0000 7f3e3ffff640 1 --2- 172.21.15.164:0/3364409369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa05e5b60 0x55bfa05f8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:51.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.288+0000 7f3e3ffff640 1 -- 172.21.15.164:0/3364409369 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfa05eb550 con 0x55bfa05f0d40 2024-09-19T08:48:51.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.289+0000 7f3e3ffff640 1 --2- 172.21.15.164:0/3364409369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bfa05f0d40 0x55bfa05ead70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3e38033fd0 tx=0x7f3e3802b370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60938789e902358d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:51.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.292+0000 7f3e1effd640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-19T08:48:51.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.294+0000 7f3e1effd640 1 -- 172.21.15.164:0/3364409369 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3804d030 con 0x55bfa05f0d40 2024-09-19T08:48:51.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.294+0000 7f3e1effd640 1 -- 172.21.15.164:0/3364409369 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f3e38004040 con 0x55bfa05f0d40 2024-09-19T08:48:51.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.296+0000 7f3e47e6e740 1 -- 172.21.15.164:0/3364409369 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e3401ae50 con 0x55bfa05f0d40 2024-09-19T08:48:51.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.297+0000 7f3e1effd640 1 -- 172.21.15.164:0/3364409369 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3e38004240 con 0x55bfa05f0d40 2024-09-19T08:48:51.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.297+0000 7f3e47e6e740 10 client.25622 fetch_fsmap learned FSMap version 28 2024-09-19T08:48:51.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.298+0000 7f3e47e6e740 1 -- 172.21.15.164:0/3364409369 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bfa05f2200 con 0x55bfa05f0d40 2024-09-19T08:48:51.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.300+0000 7f3e1effd640 1 -- 172.21.15.164:0/3364409369 <== mon.2 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e3803e440 con 0x55bfa05f0d40 2024-09-19T08:48:51.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.302+0000 7f3e1effd640 1 -- 172.21.15.164:0/3364409369 <== mon.2 v2:172.21.15.170:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3e3805c420 con 0x55bfa05f0d40 2024-09-19T08:48:51.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.302+0000 7f3e47e6e740 10 client.25622 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:48:51.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.302+0000 7f3e47e6e740 10 client.25622 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:48:51.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.302+0000 7f3e47e6e740 20 client.25622 populate_metadata read hostname 'smithi164' 2024-09-19T08:48:51.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.302+0000 7f3e47e6e740 1 --2- 172.21.15.164:0/3364409369 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55bfa066bb70 0x55bfa066df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:51.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.302+0000 7f3e47e6e740 4 client.25622 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T08:48:51.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.302+0000 7f3e47e6e740 1 -- 172.21.15.164:0/3364409369 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f3e3401ae50 con 0x55bfa066bb70 2024-09-19T08:48:51.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.305+0000 7f3e3f7fe640 1 --2- 172.21.15.164:0/3364409369 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55bfa066bb70 0x55bfa066df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:51.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.317+0000 7f3e3f7fe640 1 --2- 172.21.15.164:0/3364409369 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55bfa066bb70 0x55bfa066df90 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ef3652f7f22efa1 server_cookie=8faede27043c8b33 in_seq=0 out_seq=0 2024-09-19T08:48:51.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.317+0000 7f3e1effd640 10 client.25622 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:48:51.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.319+0000 7f3e1effd640 1 -- 172.21.15.164:0/3364409369 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f3e3401ae50 con 0x55bfa066bb70 2024-09-19T08:48:51.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.319+0000 7f3e1effd640 10 client.25622 handle_command_reply: tid=0 2024-09-19T08:48:51.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.319+0000 7f3e47e6e740 1 client.25622 shutdown 2024-09-19T08:48:51.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.323+0000 7f3e47e6e740 1 -- 172.21.15.164:0/3364409369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bfa05f0d40 msgr2=0x55bfa05ead70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:51.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.323+0000 7f3e47e6e740 1 --2- 172.21.15.164:0/3364409369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bfa05f0d40 0x55bfa05ead70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3e38033fd0 tx=0x7f3e3802b370 comp rx=0 tx=0).stop 2024-09-19T08:48:51.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.323+0000 7f3e47e6e740 1 -- 172.21.15.164:0/3364409369 shutdown_connections 2024-09-19T08:48:51.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.323+0000 7f3e47e6e740 1 --2- 172.21.15.164:0/3364409369 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55bfa066bb70 0x55bfa066df90 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:51.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.323+0000 7f3e47e6e740 1 --2- 172.21.15.164:0/3364409369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bfa05f0d40 0x55bfa05ead70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:51.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.323+0000 7f3e1effd640 0 client.25622 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:48:51.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.323+0000 7f3e47e6e740 1 --2- 172.21.15.164:0/3364409369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bfa05e6480 0x55bfa05f9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:51.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.323+0000 7f3e47e6e740 1 --2- 172.21.15.164:0/3364409369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa05e5b60 0x55bfa05f8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:51.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.323+0000 7f3e47e6e740 1 -- 172.21.15.164:0/3364409369 >> 172.21.15.164:0/3364409369 conn(0x55bfa05e20f0 msgr2=0x55bfa05ec340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:51.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.325+0000 7f3e47e6e740 1 -- 172.21.15.164:0/3364409369 shutdown_connections 2024-09-19T08:48:51.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.325+0000 7f3e47e6e740 1 -- 172.21.15.164:0/3364409369 wait complete. 2024-09-19T08:48:51.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.325+0000 7f3e47e6e740 20 client.25622 trim_cache size 0 max 16384 2024-09-19T08:48:51.329 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:48:51.329 INFO:teuthology.orchestra.run.smithi164.stdout: "return_code": 0, 2024-09-19T08:48:51.329 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_tag": "5a6514d6-9350-48fd-90b6-df3d5b04e773", 2024-09-19T08:48:51.329 INFO:teuthology.orchestra.run.smithi164.stdout: "mode": "asynchronous" 2024-09-19T08:48:51.330 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:48:51.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.329+0000 7f3e45a95640 1 -- 172.21.15.164:0/6066425 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3e280a35b0 msgr2=0x7f3e280a5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:51.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.329+0000 7f3e45a95640 1 --2- 172.21.15.164:0/6066425 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3e280a35b0 0x7f3e280a5a70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3e38005f00 tx=0x7f3e38004a40 comp rx=0 tx=0).stop 2024-09-19T08:48:51.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.329+0000 7f3e45a95640 1 -- 172.21.15.164:0/6066425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e4010e580 msgr2=0x7f3e4011ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:51.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.329+0000 7f3e45a95640 1 --2- 172.21.15.164:0/6066425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e4010e580 0x7f3e4011ead0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400c970 tx=0x7f3e3400ce30 comp rx=0 tx=0).stop 2024-09-19T08:48:51.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.331+0000 7f3e45a95640 1 -- 172.21.15.164:0/6066425 shutdown_connections 2024-09-19T08:48:51.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.331+0000 7f3e45a95640 1 --2- 172.21.15.164:0/6066425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e4010ef30 0x7f3e40117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:51.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.331+0000 7f3e45a95640 1 --2- 172.21.15.164:0/6066425 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3e280a35b0 0x7f3e280a5a70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:51.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.331+0000 7f3e45a95640 1 --2- 172.21.15.164:0/6066425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e4010e580 0x7f3e4011ead0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:51.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.331+0000 7f3e45a95640 1 --2- 172.21.15.164:0/6066425 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e4010d9b0 0x7f3e4011e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:51.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.331+0000 7f3e45a95640 1 -- 172.21.15.164:0/6066425 >> 172.21.15.164:0/6066425 conn(0x7f3e4006e140 msgr2=0x7f3e4010b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:51.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.332+0000 7f3e45a95640 1 -- 172.21.15.164:0/6066425 shutdown_connections 2024-09-19T08:48:51.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:51.332+0000 7f3e45a95640 1 -- 172.21.15.164:0/6066425 wait complete. 2024-09-19T08:48:52.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:51 smithi164 ceph-mon[30928]: pgmap v94: 97 pgs: 97 active+clean; 20 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 881 MiB/s wr, 3.54k op/s 2024-09-19T08:48:52.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:51 smithi170 ceph-mon[35072]: pgmap v94: 97 pgs: 97 active+clean; 20 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 881 MiB/s wr, 3.54k op/s 2024-09-19T08:48:52.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:51 smithi173 ceph-mon[35306]: pgmap v94: 97 pgs: 97 active+clean; 20 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 881 MiB/s wr, 3.54k op/s 2024-09-19T08:48:53.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:53 smithi164 ceph-mon[30928]: scrub queued for path: / 2024-09-19T08:48:53.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:53 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:48:53.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:53 smithi164 ceph-mon[30928]: scrub summary: active paths [/] 2024-09-19T08:48:53.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:53 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:48:53.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:53 smithi164 ceph-mon[30928]: scrub summary: idle 2024-09-19T08:48:53.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:53 smithi170 ceph-mon[35072]: scrub queued for path: / 2024-09-19T08:48:53.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:53 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:48:53.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:53 smithi170 ceph-mon[35072]: scrub summary: active paths [/] 2024-09-19T08:48:53.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:53 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:48:53.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:53 smithi170 ceph-mon[35072]: scrub summary: idle 2024-09-19T08:48:53.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:53 smithi173 ceph-mon[35306]: scrub queued for path: / 2024-09-19T08:48:53.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:53 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:48:53.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:53 smithi173 ceph-mon[35306]: scrub summary: active paths [/] 2024-09-19T08:48:53.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:53 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:48:53.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:53 smithi173 ceph-mon[35306]: scrub summary: idle 2024-09-19T08:48:53.676 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:48:54.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:54 smithi164 ceph-mon[30928]: pgmap v95: 97 pgs: 97 active+clean; 20 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 710 MiB/s wr, 2.57k op/s 2024-09-19T08:48:54.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:54 smithi170 ceph-mon[35072]: pgmap v95: 97 pgs: 97 active+clean; 20 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 710 MiB/s wr, 2.57k op/s 2024-09-19T08:48:54.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:54 smithi173 ceph-mon[35306]: pgmap v95: 97 pgs: 97 active+clean; 20 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 710 MiB/s wr, 2.57k op/s 2024-09-19T08:48:54.584 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:48:55.897 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:48:55.897 INFO:teuthology.orchestra.run.smithi164.stdout: "status": "no active scrubs running", 2024-09-19T08:48:55.897 INFO:teuthology.orchestra.run.smithi164.stdout: "scrubs": {} 2024-09-19T08:48:55.897 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:48:55.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.543+0000 7f9f0a426640 1 -- 172.21.15.164:0/4216972345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f04105cb0 msgr2=0x7f9f0410d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:55.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.543+0000 7f9f0a426640 1 --2- 172.21.15.164:0/4216972345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f04105cb0 0x7f9f0410d870 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f9f00009900 tx=0x7f9f0002a440 comp rx=0 tx=0).stop 2024-09-19T08:48:55.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.543+0000 7f9f0a426640 1 -- 172.21.15.164:0/4216972345 shutdown_connections 2024-09-19T08:48:55.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.543+0000 7f9f0a426640 1 --2- 172.21.15.164:0/4216972345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f04105cb0 0x7f9f0410d870 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:55.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.543+0000 7f9f0a426640 1 --2- 172.21.15.164:0/4216972345 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f04101c60 0x7f9f04105580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:55.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.543+0000 7f9f0a426640 1 --2- 172.21.15.164:0/4216972345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f041012b0 0x7f9f04101690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:55.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.543+0000 7f9f0a426640 1 -- 172.21.15.164:0/4216972345 >> 172.21.15.164:0/4216972345 conn(0x7f9f04077070 msgr2=0x7f9f040fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:55.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.544+0000 7f9f0a426640 1 -- 172.21.15.164:0/4216972345 shutdown_connections 2024-09-19T08:48:55.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.544+0000 7f9f0a426640 1 -- 172.21.15.164:0/4216972345 wait complete. 2024-09-19T08:48:55.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.545+0000 7f9f0a426640 1 Processor -- start 2024-09-19T08:48:55.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.546+0000 7f9f0a426640 1 -- start start 2024-09-19T08:48:55.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.546+0000 7f9f0a426640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f041012b0 0x7f9f04197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:55.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.546+0000 7f9f0a426640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f04101c60 0x7f9f04197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:55.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.547+0000 7f9f0a426640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f04105cb0 0x7f9f0419bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:55.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.547+0000 7f9f0a426640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9f04110e30 con 0x7f9f04101c60 2024-09-19T08:48:55.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.547+0000 7f9f0a426640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9f04110cb0 con 0x7f9f04105cb0 2024-09-19T08:48:55.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.547+0000 7f9f0a426640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9f04110fb0 con 0x7f9f041012b0 2024-09-19T08:48:55.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.547+0000 7f9f09424640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f041012b0 0x7f9f04197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:55.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.547+0000 7f9f09c25640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f04105cb0 0x7f9f0419bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:55.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.547+0000 7f9f09424640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f041012b0 0x7f9f04197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:38372/0 (socket says 172.21.15.164:38372) 2024-09-19T08:48:55.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.547+0000 7f9f09424640 1 -- 172.21.15.164:0/2929180485 learned_addr learned my addr 172.21.15.164:0/2929180485 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:55.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.550+0000 7f9f09424640 1 -- 172.21.15.164:0/2929180485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f04105cb0 msgr2=0x7f9f0419bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:55.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.550+0000 7f9f09424640 1 --2- 172.21.15.164:0/2929180485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f04105cb0 0x7f9f0419bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:55.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.550+0000 7f9f09424640 1 -- 172.21.15.164:0/2929180485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f04101c60 msgr2=0x7f9f04197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:55.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.550+0000 7f9f09424640 1 --2- 172.21.15.164:0/2929180485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f04101c60 0x7f9f04197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:55.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.550+0000 7f9f09424640 1 -- 172.21.15.164:0/2929180485 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f0419c2e0 con 0x7f9f041012b0 2024-09-19T08:48:55.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.555+0000 7f9f09424640 1 --2- 172.21.15.164:0/2929180485 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f041012b0 0x7f9f04197330 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00ca30 tx=0x7f9efc00cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:55.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.559+0000 7f9efa7fc640 1 -- 172.21.15.164:0/2929180485 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc019070 con 0x7f9f041012b0 2024-09-19T08:48:55.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.559+0000 7f9efa7fc640 1 -- 172.21.15.164:0/2929180485 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9efc008ea0 con 0x7f9f041012b0 2024-09-19T08:48:55.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.559+0000 7f9efa7fc640 1 -- 172.21.15.164:0/2929180485 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc014480 con 0x7f9f041012b0 2024-09-19T08:48:55.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.560+0000 7f9f0a426640 1 -- 172.21.15.164:0/2929180485 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f0419c5d0 con 0x7f9f041012b0 2024-09-19T08:48:55.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.560+0000 7f9f0a426640 1 -- 172.21.15.164:0/2929180485 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f040fff40 con 0x7f9f041012b0 2024-09-19T08:48:55.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.565+0000 7f9f0da61740 1 Processor -- start 2024-09-19T08:48:55.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.565+0000 7f9f0da61740 1 -- start start 2024-09-19T08:48:55.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.566+0000 7f9f0da61740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55556ce23430 0x55556ce23810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:55.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.566+0000 7f9f0da61740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55556ce23d50 0x55556ce3f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:55.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.566+0000 7f9f0da61740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55556ce3f860 0x55556ce41c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:55.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.566+0000 7f9f0da61740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55556cd892e0 con 0x55556ce23d50 2024-09-19T08:48:55.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.566+0000 7f9f0da61740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55556cd30ce0 con 0x55556ce3f860 2024-09-19T08:48:55.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.566+0000 7f9f0da61740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55556cd852a0 con 0x55556ce23430 2024-09-19T08:48:55.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.566+0000 7f9f09c25640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55556ce23430 0x55556ce23810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:55.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.568+0000 7f9f08c23640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55556ce3f860 0x55556ce41c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:55.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.568+0000 7f9f08c23640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55556ce3f860 0x55556ce41c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:41078/0 (socket says 172.21.15.164:41078) 2024-09-19T08:48:55.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.568+0000 7f9f08c23640 1 -- 172.21.15.164:0/1710487977 learned_addr learned my addr 172.21.15.164:0/1710487977 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:55.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.570+0000 7f9f09c25640 1 -- 172.21.15.164:0/1710487977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55556ce3f860 msgr2=0x55556ce41c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:55.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.570+0000 7f9f09c25640 1 --2- 172.21.15.164:0/1710487977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55556ce3f860 0x55556ce41c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:55.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.570+0000 7f9f09c25640 1 -- 172.21.15.164:0/1710487977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55556ce23d50 msgr2=0x55556ce3f320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:48:55.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.570+0000 7f9f09c25640 1 --2- 172.21.15.164:0/1710487977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55556ce23d50 0x55556ce3f320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:55.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.570+0000 7f9f09c25640 1 -- 172.21.15.164:0/1710487977 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55556cd15440 con 0x55556ce23430 2024-09-19T08:48:55.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.571+0000 7f9efa7fc640 1 -- 172.21.15.164:0/2929180485 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f9efc01c070 con 0x7f9f041012b0 2024-09-19T08:48:55.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.575+0000 7f9efa7fc640 1 --2- 172.21.15.164:0/2929180485 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f9ef00a3580 0x7f9ef00a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:55.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.575+0000 7f9efa7fc640 1 -- 172.21.15.164:0/2929180485 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f9efc0cf220 con 0x7f9f041012b0 2024-09-19T08:48:55.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.577+0000 7f9f09c25640 1 --2- 172.21.15.164:0/1710487977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55556ce23430 0x55556ce23810 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9f00006010 tx=0x7f9f000316d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fb72b4d261cb69 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:55.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.579+0000 7f9f09424640 1 --2- 172.21.15.164:0/2929180485 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f9ef00a3580 0x7f9ef00a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:55.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.580+0000 7f9eef7fe640 1 -- 172.21.15.164:0/1710487977 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f00035040 con 0x55556ce23430 2024-09-19T08:48:55.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.581+0000 7f9eef7fe640 1 -- 172.21.15.164:0/1710487977 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f00005020 con 0x55556ce23430 2024-09-19T08:48:55.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.582+0000 7f9f0da61740 1 -- 172.21.15.164:0/1710487977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55556ce23430 msgr2=0x55556ce23810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:55.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.582+0000 7f9f0da61740 1 --2- 172.21.15.164:0/1710487977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55556ce23430 0x55556ce23810 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9f00006010 tx=0x7f9f000316d0 comp rx=0 tx=0).stop 2024-09-19T08:48:55.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.584+0000 7f9f0da61740 1 -- 172.21.15.164:0/1710487977 shutdown_connections 2024-09-19T08:48:55.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.584+0000 7f9f0da61740 1 --2- 172.21.15.164:0/1710487977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55556ce3f860 0x55556ce41c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:55.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.584+0000 7f9f0da61740 1 --2- 172.21.15.164:0/1710487977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55556ce23d50 0x55556ce3f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:55.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.584+0000 7f9f0da61740 1 --2- 172.21.15.164:0/1710487977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55556ce23430 0x55556ce23810 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:55.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.584+0000 7f9f0da61740 1 -- 172.21.15.164:0/1710487977 >> 172.21.15.164:0/1710487977 conn(0x55556ce1fc00 msgr2=0x55556ce1ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:55.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.585+0000 7f9f09424640 1 --2- 172.21.15.164:0/2929180485 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f9ef00a3580 0x7f9ef00a5a40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55556ce24580 tx=0x7f9efc00b040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:55.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.585+0000 7f9f0da61740 1 -- 172.21.15.164:0/1710487977 shutdown_connections 2024-09-19T08:48:55.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.585+0000 7f9f0da61740 1 -- 172.21.15.164:0/1710487977 wait complete. 2024-09-19T08:48:55.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.586+0000 7f9f0da61740 1 -- start start 2024-09-19T08:48:55.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.586+0000 7f9f0da61740 1 Processor -- start 2024-09-19T08:48:55.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.587+0000 7f9f0da61740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55556ce23430 0x55556ce3f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:55.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.587+0000 7f9f0da61740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55556ce23d50 0x55556ce3c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:55.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.587+0000 7f9f0da61740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55556ce3f860 0x55556ce3c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:55.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.587+0000 7f9f0da61740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55556ce42430 con 0x55556ce23d50 2024-09-19T08:48:55.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.587+0000 7f9f0da61740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55556cd44050 con 0x55556ce23430 2024-09-19T08:48:55.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.588+0000 7f9f0da61740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55556ce425b0 con 0x55556ce3f860 2024-09-19T08:48:55.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.588+0000 7f9f0da61740 10 client.0 _ll_register_callbacks cb 0x55556cd84160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:48:55.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.588+0000 7f9f08c23640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55556ce23d50 0x55556ce3c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:55.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.588+0000 7f9f08c23640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55556ce23d50 0x55556ce3c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:53034/0 (socket says 172.21.15.164:53034) 2024-09-19T08:48:55.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.588+0000 7f9f08c23640 1 -- 172.21.15.164:0/3088475875 learned_addr learned my addr 172.21.15.164:0/3088475875 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:55.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.590+0000 7f9f08c23640 1 -- 172.21.15.164:0/3088475875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55556ce3f860 msgr2=0x55556ce3c930 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:48:55.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.590+0000 7f9f08c23640 1 --2- 172.21.15.164:0/3088475875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55556ce3f860 0x55556ce3c930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:55.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.590+0000 7f9f08c23640 1 -- 172.21.15.164:0/3088475875 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55556ce23430 msgr2=0x55556ce3f0f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:48:55.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.590+0000 7f9f08c23640 1 --2- 172.21.15.164:0/3088475875 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55556ce23430 0x55556ce3f0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:55.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.590+0000 7f9f08c23640 1 -- 172.21.15.164:0/3088475875 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55556cafa3c0 con 0x55556ce23d50 2024-09-19T08:48:55.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.597+0000 7f9f08c23640 1 --2- 172.21.15.164:0/3088475875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55556ce23d50 0x55556ce3c3f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f9ef400e250 tx=0x7f9ef400e710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36287ec1e092a4ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:55.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.599+0000 7f9eecff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:48:55.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.599+0000 7f9eecff9640 1 -- 172.21.15.164:0/3088475875 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4017070 con 0x55556ce23d50 2024-09-19T08:48:55.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.599+0000 7f9f0da61740 1 -- 172.21.15.164:0/3088475875 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f00005020 con 0x55556ce23d50 2024-09-19T08:48:55.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.599+0000 7f9eecff9640 1 -- 172.21.15.164:0/3088475875 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f9ef4013680 con 0x55556ce23d50 2024-09-19T08:48:55.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.600+0000 7f9eecff9640 1 -- 172.21.15.164:0/3088475875 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9ef4023050 con 0x55556ce23d50 2024-09-19T08:48:55.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.600+0000 7f9f0da61740 10 client.16197 fetch_fsmap learned FSMap version 28 2024-09-19T08:48:55.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.600+0000 7f9f0da61740 1 -- 172.21.15.164:0/3088475875 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55556ce4ded0 con 0x55556ce23d50 2024-09-19T08:48:55.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.607+0000 7f9eecff9640 1 -- 172.21.15.164:0/3088475875 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f00005020 con 0x55556ce23d50 2024-09-19T08:48:55.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.607+0000 7f9eecff9640 1 -- 172.21.15.164:0/3088475875 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9ef40129e0 con 0x55556ce23d50 2024-09-19T08:48:55.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.611+0000 7f9f0da61740 10 client.16197 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:48:55.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.611+0000 7f9f0da61740 10 client.16197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:48:55.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.611+0000 7f9f0da61740 20 client.16197 populate_metadata read hostname 'smithi164' 2024-09-19T08:48:55.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.611+0000 7f9f0da61740 1 --2- 172.21.15.164:0/3088475875 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55556ce4f330 0x55556ce51780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:55.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.611+0000 7f9f0da61740 4 client.16197 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:48:55.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.611+0000 7f9f0da61740 1 -- 172.21.15.164:0/3088475875 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55556cd96850 con 0x55556ce4f330 2024-09-19T08:48:55.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.612+0000 7f9f09c25640 1 --2- 172.21.15.164:0/3088475875 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55556ce4f330 0x55556ce51780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:55.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.625+0000 7f9eecff9640 10 client.16197 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:48:55.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.625+0000 7f9f09c25640 1 --2- 172.21.15.164:0/3088475875 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55556ce4f330 0x55556ce51780 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f39b244bce6f2dfc server_cookie=7571b4ac2253fe6e in_seq=0 out_seq=0 2024-09-19T08:48:55.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.627+0000 7f9eecff9640 1 -- 172.21.15.164:0/3088475875 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55556cd96850 con 0x55556ce4f330 2024-09-19T08:48:55.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.627+0000 7f9eecff9640 10 client.16197 handle_command_reply: tid=0 2024-09-19T08:48:55.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.632+0000 7f9f0da61740 1 client.16197 shutdown 2024-09-19T08:48:55.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.633+0000 7f9f0da61740 1 -- 172.21.15.164:0/3088475875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55556ce23d50 msgr2=0x55556ce3c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:55.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.633+0000 7f9f0da61740 1 --2- 172.21.15.164:0/3088475875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55556ce23d50 0x55556ce3c3f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f9ef400e250 tx=0x7f9ef400e710 comp rx=0 tx=0).stop 2024-09-19T08:48:55.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.633+0000 7f9f0da61740 1 -- 172.21.15.164:0/3088475875 shutdown_connections 2024-09-19T08:48:55.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.633+0000 7f9f0da61740 1 --2- 172.21.15.164:0/3088475875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55556ce3f860 0x55556ce3c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:55.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.633+0000 7f9f0da61740 1 --2- 172.21.15.164:0/3088475875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55556ce23d50 0x55556ce3c3f0 secure :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f9ef400e250 tx=0x7f9ef400e710 comp rx=0 tx=0).stop 2024-09-19T08:48:55.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.633+0000 7f9f0da61740 1 --2- 172.21.15.164:0/3088475875 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55556ce4f330 0x55556ce51780 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:55.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.633+0000 7f9f0da61740 1 --2- 172.21.15.164:0/3088475875 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55556ce23430 0x55556ce3f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:55.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.633+0000 7f9f0da61740 1 -- 172.21.15.164:0/3088475875 >> 172.21.15.164:0/3088475875 conn(0x55556ce1fc00 msgr2=0x55556ce258a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:55.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.633+0000 7f9eecff9640 0 client.16197 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:48:55.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.633+0000 7f9f0da61740 1 -- 172.21.15.164:0/3088475875 shutdown_connections 2024-09-19T08:48:55.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.633+0000 7f9f0da61740 1 -- 172.21.15.164:0/3088475875 wait complete. 2024-09-19T08:48:55.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.634+0000 7f9f0da61740 20 client.16197 trim_cache size 0 max 16384 2024-09-19T08:48:55.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.673+0000 7f9f0da61740 1 Processor -- start 2024-09-19T08:48:55.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.674+0000 7f9f0da61740 1 -- start start 2024-09-19T08:48:55.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.675+0000 7f9f0da61740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55556ce23430 0x55556ce4b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:55.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.675+0000 7f9f0da61740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55556ce3f860 0x55556ce37ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:55.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.677+0000 7f9f0da61740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55556ce3ca20 0x55556ce3e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:55.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.678+0000 7f9f08c23640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55556ce3f860 0x55556ce37ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:55.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.678+0000 7f9f08c23640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55556ce3f860 0x55556ce37ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:53048/0 (socket says 172.21.15.164:53048) 2024-09-19T08:48:55.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.679+0000 7f9f08c23640 1 -- 172.21.15.164:0/2447230183 learned_addr learned my addr 172.21.15.164:0/2447230183 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:55.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.680+0000 7f9f0da61740 1 -- 172.21.15.164:0/2447230183 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9f04073160 con 0x55556ce3f860 2024-09-19T08:48:55.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.680+0000 7f9f0da61740 1 -- 172.21.15.164:0/2447230183 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9f041a4740 con 0x55556ce23430 2024-09-19T08:48:55.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.680+0000 7f9f0da61740 1 -- 172.21.15.164:0/2447230183 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9f04073830 con 0x55556ce3ca20 2024-09-19T08:48:55.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.681+0000 7f9f08c23640 1 -- 172.21.15.164:0/2447230183 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55556ce3ca20 msgr2=0x55556ce3e360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:48:55.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.681+0000 7f9f08c23640 1 --2- 172.21.15.164:0/2447230183 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55556ce3ca20 0x55556ce3e360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:55.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.681+0000 7f9f08c23640 1 -- 172.21.15.164:0/2447230183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55556ce23430 msgr2=0x55556ce4b750 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:48:55.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.681+0000 7f9f08c23640 1 --2- 172.21.15.164:0/2447230183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55556ce23430 0x55556ce4b750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:55.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.681+0000 7f9f08c23640 1 -- 172.21.15.164:0/2447230183 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55556ce41fb0 con 0x55556ce3f860 2024-09-19T08:48:55.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.681+0000 7f9f08c23640 1 --2- 172.21.15.164:0/2447230183 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55556ce3f860 0x55556ce37ae0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4000ea0 tx=0x7f9ef401b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a402d8855da15ade server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:55.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.683+0000 7f9eeeffd640 1 -- 172.21.15.164:0/2447230183 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4017040 con 0x55556ce3f860 2024-09-19T08:48:55.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.683+0000 7f9eeeffd640 1 -- 172.21.15.164:0/2447230183 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ef401bd00 con 0x55556ce3f860 2024-09-19T08:48:55.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.683+0000 7f9eeeffd640 1 -- 172.21.15.164:0/2447230183 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4013970 con 0x55556ce3f860 2024-09-19T08:48:55.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.687+0000 7f9f0da61740 1 -- 172.21.15.164:0/2447230183 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55556ce3f860 msgr2=0x55556ce37ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:55.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.687+0000 7f9f0da61740 1 --2- 172.21.15.164:0/2447230183 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55556ce3f860 0x55556ce37ae0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4000ea0 tx=0x7f9ef401b660 comp rx=0 tx=0).stop 2024-09-19T08:48:55.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.687+0000 7f9f0da61740 1 -- 172.21.15.164:0/2447230183 shutdown_connections 2024-09-19T08:48:55.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.687+0000 7f9f0da61740 1 --2- 172.21.15.164:0/2447230183 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55556ce3ca20 0x55556ce3e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:55.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.687+0000 7f9f0da61740 1 --2- 172.21.15.164:0/2447230183 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55556ce3f860 0x55556ce37ae0 secure :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4000ea0 tx=0x7f9ef401b660 comp rx=0 tx=0).stop 2024-09-19T08:48:55.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.687+0000 7f9f0da61740 1 --2- 172.21.15.164:0/2447230183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55556ce23430 0x55556ce4b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:55.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.687+0000 7f9f0da61740 1 -- 172.21.15.164:0/2447230183 >> 172.21.15.164:0/2447230183 conn(0x55556ce1fc00 msgr2=0x55556ce3dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:55.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.691+0000 7f9f0da61740 1 -- 172.21.15.164:0/2447230183 shutdown_connections 2024-09-19T08:48:55.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.691+0000 7f9f0da61740 1 -- 172.21.15.164:0/2447230183 wait complete. 2024-09-19T08:48:55.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.692+0000 7f9f0da61740 1 -- start start 2024-09-19T08:48:55.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.692+0000 7f9f0da61740 1 Processor -- start 2024-09-19T08:48:55.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.693+0000 7f9f0da61740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55556ce23430 0x55556ce43e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:55.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.694+0000 7f9f0da61740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55556ce3ca20 0x55556ce443d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:55.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.694+0000 7f9f0da61740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55556ceb30a0 0x55556ceb3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:55.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.694+0000 7f9f0da61740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9f041a4070 con 0x55556ce23430 2024-09-19T08:48:55.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.694+0000 7f9f0da61740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9f04073f00 con 0x55556ceb30a0 2024-09-19T08:48:55.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.694+0000 7f9f0da61740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9f04071140 con 0x55556ce3ca20 2024-09-19T08:48:55.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.694+0000 7f9f0da61740 10 client.0 _ll_register_callbacks cb 0x7f9ef4009380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:48:55.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.694+0000 7f9f09c25640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55556ceb30a0 0x55556ceb3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:55.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.694+0000 7f9f09c25640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55556ceb30a0 0x55556ceb3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:41098/0 (socket says 172.21.15.164:41098) 2024-09-19T08:48:55.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.694+0000 7f9f09c25640 1 -- 172.21.15.164:0/858809819 learned_addr learned my addr 172.21.15.164:0/858809819 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:48:55.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.694+0000 7f9f08c23640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55556ce3ca20 0x55556ce443d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:55.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.696+0000 7f9f09c25640 1 --2- 172.21.15.164:0/858809819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55556ce23430 0x55556ce43e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:55.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.697+0000 7f9f09c25640 1 -- 172.21.15.164:0/858809819 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55556ce3ca20 msgr2=0x55556ce443d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:55.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.697+0000 7f9f09c25640 1 --2- 172.21.15.164:0/858809819 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55556ce3ca20 0x55556ce443d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:55.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.697+0000 7f9f09c25640 1 -- 172.21.15.164:0/858809819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55556ce23430 msgr2=0x55556ce43e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:55.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.697+0000 7f9f09c25640 1 --2- 172.21.15.164:0/858809819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55556ce23430 0x55556ce43e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:55.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.697+0000 7f9f09c25640 1 -- 172.21.15.164:0/858809819 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55556ceb3a90 con 0x55556ceb30a0 2024-09-19T08:48:55.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.703+0000 7f9f09c25640 1 --2- 172.21.15.164:0/858809819 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55556ceb30a0 0x55556ceb3520 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9f00000c00 tx=0x7f9f000045b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b7c89574c47e80d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:55.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.703+0000 7f9eecff9640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-19T08:48:55.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.708+0000 7f9eecff9640 1 -- 172.21.15.164:0/858809819 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f00035030 con 0x55556ceb30a0 2024-09-19T08:48:55.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.708+0000 7f9eecff9640 1 -- 172.21.15.164:0/858809819 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f9f00061650 con 0x55556ceb30a0 2024-09-19T08:48:55.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.709+0000 7f9f0da61740 1 -- 172.21.15.164:0/858809819 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ef401bd00 con 0x55556ceb30a0 2024-09-19T08:48:55.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.715+0000 7f9eecff9640 1 -- 172.21.15.164:0/858809819 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9f00058040 con 0x55556ceb30a0 2024-09-19T08:48:55.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.716+0000 7f9f0da61740 10 client.25631 fetch_fsmap learned FSMap version 28 2024-09-19T08:48:55.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.716+0000 7f9f0da61740 1 -- 172.21.15.164:0/858809819 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55556cebae40 con 0x55556ceb30a0 2024-09-19T08:48:55.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.720+0000 7f9eecff9640 1 -- 172.21.15.164:0/858809819 <== mon.2 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f00063070 con 0x55556ceb30a0 2024-09-19T08:48:55.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.720+0000 7f9eecff9640 1 -- 172.21.15.164:0/858809819 <== mon.2 v2:172.21.15.170:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9f0003a040 con 0x55556ceb30a0 2024-09-19T08:48:55.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.720+0000 7f9f0da61740 10 client.25631 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:48:55.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.720+0000 7f9f0da61740 10 client.25631 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:48:55.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.720+0000 7f9f0da61740 20 client.25631 populate_metadata read hostname 'smithi164' 2024-09-19T08:48:55.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.720+0000 7f9f0da61740 1 --2- 172.21.15.164:0/858809819 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55556cebb120 0x55556cebd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:55.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.720+0000 7f9f0da61740 4 client.25631 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:48:55.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.720+0000 7f9f0da61740 1 -- 172.21.15.164:0/858809819 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f9ef401bd00 con 0x55556cebb120 2024-09-19T08:48:55.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.721+0000 7f9f08c23640 1 --2- 172.21.15.164:0/858809819 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55556cebb120 0x55556cebd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:55.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.730+0000 7f9eecff9640 10 client.25631 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:48:55.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.730+0000 7f9f08c23640 1 --2- 172.21.15.164:0/858809819 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55556cebb120 0x55556cebd510 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e04e13a6caa399d8 server_cookie=6442b4931db11a94 in_seq=0 out_seq=0 2024-09-19T08:48:55.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.730+0000 7f9eecff9640 1 -- 172.21.15.164:0/858809819 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f9ef401bd00 con 0x55556cebb120 2024-09-19T08:48:55.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.730+0000 7f9eecff9640 10 client.25631 handle_command_reply: tid=0 2024-09-19T08:48:55.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.730+0000 7f9f0da61740 1 client.25631 shutdown 2024-09-19T08:48:55.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.730+0000 7f9f0da61740 1 -- 172.21.15.164:0/858809819 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55556ceb30a0 msgr2=0x55556ceb3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:55.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.730+0000 7f9f0da61740 1 --2- 172.21.15.164:0/858809819 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55556ceb30a0 0x55556ceb3520 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9f00000c00 tx=0x7f9f000045b0 comp rx=0 tx=0).stop 2024-09-19T08:48:55.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.730+0000 7f9f0da61740 1 -- 172.21.15.164:0/858809819 shutdown_connections 2024-09-19T08:48:55.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.730+0000 7f9f0da61740 1 --2- 172.21.15.164:0/858809819 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55556cebb120 0x55556cebd510 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:55.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.731+0000 7f9f0da61740 1 --2- 172.21.15.164:0/858809819 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55556ceb30a0 0x55556ceb3520 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:55.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.731+0000 7f9f0da61740 1 --2- 172.21.15.164:0/858809819 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55556ce3ca20 0x55556ce443d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:55.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.731+0000 7f9f0da61740 1 --2- 172.21.15.164:0/858809819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55556ce23430 0x55556ce43e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:55.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.731+0000 7f9f0da61740 1 -- 172.21.15.164:0/858809819 >> 172.21.15.164:0/858809819 conn(0x55556ce1fc00 msgr2=0x55556ce3e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:55.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.731+0000 7f9eecff9640 0 client.25631 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:48:55.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.732+0000 7f9f0da61740 1 -- 172.21.15.164:0/858809819 shutdown_connections 2024-09-19T08:48:55.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.732+0000 7f9f0da61740 1 -- 172.21.15.164:0/858809819 wait complete. 2024-09-19T08:48:55.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.732+0000 7f9f0da61740 20 client.25631 trim_cache size 0 max 16384 2024-09-19T08:48:55.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.734+0000 7f9f0a426640 1 -- 172.21.15.164:0/2929180485 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f9ef00a3580 msgr2=0x7f9ef00a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:55.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.734+0000 7f9f0a426640 1 --2- 172.21.15.164:0/2929180485 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f9ef00a3580 0x7f9ef00a5a40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55556ce24580 tx=0x7f9efc00b040 comp rx=0 tx=0).stop 2024-09-19T08:48:55.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.734+0000 7f9f0a426640 1 -- 172.21.15.164:0/2929180485 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f041012b0 msgr2=0x7f9f04197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:55.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.734+0000 7f9f0a426640 1 --2- 172.21.15.164:0/2929180485 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f041012b0 0x7f9f04197330 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00ca30 tx=0x7f9efc00cef0 comp rx=0 tx=0).stop 2024-09-19T08:48:55.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.734+0000 7f9f0a426640 1 -- 172.21.15.164:0/2929180485 shutdown_connections 2024-09-19T08:48:55.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.734+0000 7f9f0a426640 1 --2- 172.21.15.164:0/2929180485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f04105cb0 0x7f9f0419bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:55.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.734+0000 7f9f0a426640 1 --2- 172.21.15.164:0/2929180485 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f9ef00a3580 0x7f9ef00a5a40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:55.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.734+0000 7f9f0a426640 1 --2- 172.21.15.164:0/2929180485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f04101c60 0x7f9f04197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:55.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.734+0000 7f9f0a426640 1 --2- 172.21.15.164:0/2929180485 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f041012b0 0x7f9f04197330 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:55.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.734+0000 7f9f0a426640 1 -- 172.21.15.164:0/2929180485 >> 172.21.15.164:0/2929180485 conn(0x7f9f04077070 msgr2=0x7f9f04104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:55.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.734+0000 7f9f0a426640 1 -- 172.21.15.164:0/2929180485 shutdown_connections 2024-09-19T08:48:55.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:48:55.734+0000 7f9f0a426640 1 -- 172.21.15.164:0/2929180485 wait complete. 2024-09-19T08:48:56.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:56 smithi164 ceph-mon[30928]: pgmap v96: 97 pgs: 97 active+clean; 20 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 740 MiB/s wr, 2.46k op/s 2024-09-19T08:48:56.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:56 smithi170 ceph-mon[35072]: pgmap v96: 97 pgs: 97 active+clean; 20 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 740 MiB/s wr, 2.46k op/s 2024-09-19T08:48:56.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:56 smithi173 ceph-mon[35306]: pgmap v96: 97 pgs: 97 active+clean; 20 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 740 MiB/s wr, 2.46k op/s 2024-09-19T08:48:58.303 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T08:48:58.304 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:48:58.304 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:58 smithi164 ceph-mon[30928]: pgmap v97: 97 pgs: 97 active+clean; 20 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 737 MiB/s wr, 2.11k op/s 2024-09-19T08:48:58.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:58 smithi170 ceph-mon[35072]: pgmap v97: 97 pgs: 97 active+clean; 20 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 737 MiB/s wr, 2.11k op/s 2024-09-19T08:48:58.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:58 smithi173 ceph-mon[35306]: pgmap v97: 97 pgs: 97 active+clean; 20 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 737 MiB/s wr, 2.11k op/s 2024-09-19T08:48:59.311 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:48:59.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:48:59 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:48:59.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:48:59 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:48:59.518 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:48:59 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:49:00.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:00 smithi170 ceph-mon[35072]: pgmap v98: 97 pgs: 97 active+clean; 20 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 655 MiB/s wr, 1.76k op/s 2024-09-19T08:49:00.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:00 smithi173 ceph-mon[35306]: pgmap v98: 97 pgs: 97 active+clean; 20 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 655 MiB/s wr, 1.76k op/s 2024-09-19T08:49:00.519 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:00 smithi164 ceph-mon[30928]: pgmap v98: 97 pgs: 97 active+clean; 20 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 655 MiB/s wr, 1.76k op/s 2024-09-19T08:49:00.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.780+0000 7f43d17b5640 1 -- 172.21.15.164:0/3481229013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43cc109830 msgr2=0x7f43cc109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:00.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.780+0000 7f43d17b5640 1 --2- 172.21.15.164:0/3481229013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43cc109830 0x7f43cc109c90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f43c0009900 tx=0x7f43c002a440 comp rx=0 tx=0).stop 2024-09-19T08:49:00.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.782+0000 7f43d17b5640 1 -- 172.21.15.164:0/3481229013 shutdown_connections 2024-09-19T08:49:00.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.782+0000 7f43d17b5640 1 --2- 172.21.15.164:0/3481229013 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43cc11b140 0x7f43cc11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:00.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.782+0000 7f43d17b5640 1 --2- 172.21.15.164:0/3481229013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43cc109830 0x7f43cc109c90 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:00.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.782+0000 7f43d17b5640 1 --2- 172.21.15.164:0/3481229013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43cc109450 0x7f43cc110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:00.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.782+0000 7f43d17b5640 1 -- 172.21.15.164:0/3481229013 >> 172.21.15.164:0/3481229013 conn(0x7f43cc06e140 msgr2=0x7f43cc06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:00.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.789+0000 7f43d17b5640 1 -- 172.21.15.164:0/3481229013 shutdown_connections 2024-09-19T08:49:00.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.793+0000 7f43d17b5640 1 -- 172.21.15.164:0/3481229013 wait complete. 2024-09-19T08:49:00.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.795+0000 7f43d17b5640 1 Processor -- start 2024-09-19T08:49:00.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.796+0000 7f43d17b5640 1 -- start start 2024-09-19T08:49:00.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.796+0000 7f43d17b5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43cc109830 0x7f43cc1a7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:00.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.801+0000 7f43d17b5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43cc11b140 0x7f43cc1a7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:00.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.801+0000 7f43d17b5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43cc1abd00 0x7f43cc1ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:00.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.802+0000 7f43caffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43cc109830 0x7f43cc1a7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:00.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.802+0000 7f43caffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43cc109830 0x7f43cc1a7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:41120/0 (socket says 172.21.15.164:41120) 2024-09-19T08:49:00.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.802+0000 7f43caffd640 1 -- 172.21.15.164:0/1960389209 learned_addr learned my addr 172.21.15.164:0/1960389209 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:00.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.802+0000 7f43d17b5640 1 -- 172.21.15.164:0/1960389209 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f43cc120e30 con 0x7f43cc11b140 2024-09-19T08:49:00.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.802+0000 7f43d17b5640 1 -- 172.21.15.164:0/1960389209 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f43cc120cb0 con 0x7f43cc109830 2024-09-19T08:49:00.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.802+0000 7f43d17b5640 1 -- 172.21.15.164:0/1960389209 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f43cc120fb0 con 0x7f43cc1abd00 2024-09-19T08:49:00.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.803+0000 7f43cb7fe640 1 --2- 172.21.15.164:0/1960389209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43cc1abd00 0x7f43cc1ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:00.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.804+0000 7f43ca7fc640 1 --2- 172.21.15.164:0/1960389209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43cc11b140 0x7f43cc1a7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:00.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.807+0000 7f43cb7fe640 1 -- 172.21.15.164:0/1960389209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43cc109830 msgr2=0x7f43cc1a7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:00.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.807+0000 7f43cb7fe640 1 --2- 172.21.15.164:0/1960389209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43cc109830 0x7f43cc1a7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:00.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.807+0000 7f43cb7fe640 1 -- 172.21.15.164:0/1960389209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43cc11b140 msgr2=0x7f43cc1a7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:00.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.807+0000 7f43cb7fe640 1 --2- 172.21.15.164:0/1960389209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43cc11b140 0x7f43cc1a7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:00.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.807+0000 7f43cb7fe640 1 -- 172.21.15.164:0/1960389209 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43cc1ac7c0 con 0x7f43cc1abd00 2024-09-19T08:49:00.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.807+0000 7f43cb7fe640 1 --2- 172.21.15.164:0/1960389209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43cc1abd00 0x7f43cc1ac1b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f43c400ea80 tx=0x7f43c400ef40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:00.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.808+0000 7f43b3fff640 1 -- 172.21.15.164:0/1960389209 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c4004b20 con 0x7f43cc1abd00 2024-09-19T08:49:00.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.808+0000 7f43b3fff640 1 -- 172.21.15.164:0/1960389209 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43c4004cc0 con 0x7f43cc1abd00 2024-09-19T08:49:00.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.808+0000 7f43d17b5640 1 -- 172.21.15.164:0/1960389209 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43cc1acab0 con 0x7f43cc1abd00 2024-09-19T08:49:00.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.808+0000 7f43b3fff640 1 -- 172.21.15.164:0/1960389209 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c4010720 con 0x7f43cc1abd00 2024-09-19T08:49:00.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.808+0000 7f43d17b5640 1 -- 172.21.15.164:0/1960389209 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43cc1b41e0 con 0x7f43cc1abd00 2024-09-19T08:49:00.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.810+0000 7f43d17b5640 1 -- 172.21.15.164:0/1960389209 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43cc1a8790 con 0x7f43cc1abd00 2024-09-19T08:49:00.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.819+0000 7f43b3fff640 1 -- 172.21.15.164:0/1960389209 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f43c4005020 con 0x7f43cc1abd00 2024-09-19T08:49:00.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.820+0000 7f43b3fff640 1 --2- 172.21.15.164:0/1960389209 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f43b40a3400 0x7f43b40a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:00.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.820+0000 7f43b3fff640 1 -- 172.21.15.164:0/1960389209 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f43c40d0440 con 0x7f43cc1abd00 2024-09-19T08:49:00.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.823+0000 7f43caffd640 1 --2- 172.21.15.164:0/1960389209 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f43b40a3400 0x7f43b40a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:00.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.826+0000 7f43caffd640 1 --2- 172.21.15.164:0/1960389209 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f43b40a3400 0x7f43b40a58c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f43bc005d00 tx=0x7f43bc00b390 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:00.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:00.827+0000 7f43b3fff640 1 -- 172.21.15.164:0/1960389209 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f43c409ce90 con 0x7f43cc1abd00 2024-09-19T08:49:01.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:01.310+0000 7f43d17b5640 1 -- 172.21.15.164:0/1960389209 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43cc10b580 con 0x7f43cc1abd00 2024-09-19T08:49:01.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:01.320+0000 7f43b3fff640 1 -- 172.21.15.164:0/1960389209 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11984 (secure 0 0 0) 0x7f43c409c790 con 0x7f43cc1abd00 2024-09-19T08:49:01.323 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:49:01.323 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":28,"btime":"2024-09-19T08:45:46:903004+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:45:46.852789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:49:01.324 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 28 2024-09-19T08:49:01.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:01.329+0000 7f43d17b5640 1 -- 172.21.15.164:0/1960389209 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f43b40a3400 msgr2=0x7f43b40a58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:01.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:01.329+0000 7f43d17b5640 1 --2- 172.21.15.164:0/1960389209 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f43b40a3400 0x7f43b40a58c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f43bc005d00 tx=0x7f43bc00b390 comp rx=0 tx=0).stop 2024-09-19T08:49:01.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:01.329+0000 7f43d17b5640 1 -- 172.21.15.164:0/1960389209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43cc1abd00 msgr2=0x7f43cc1ac1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:01.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:01.329+0000 7f43d17b5640 1 --2- 172.21.15.164:0/1960389209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43cc1abd00 0x7f43cc1ac1b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f43c400ea80 tx=0x7f43c400ef40 comp rx=0 tx=0).stop 2024-09-19T08:49:01.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:01.330+0000 7f43d17b5640 1 -- 172.21.15.164:0/1960389209 shutdown_connections 2024-09-19T08:49:01.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:01.330+0000 7f43d17b5640 1 --2- 172.21.15.164:0/1960389209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43cc1abd00 0x7f43cc1ac1b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:01.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:01.330+0000 7f43d17b5640 1 --2- 172.21.15.164:0/1960389209 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f43b40a3400 0x7f43b40a58c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:01.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:01.330+0000 7f43d17b5640 1 --2- 172.21.15.164:0/1960389209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43cc11b140 0x7f43cc1a7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:01.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:01.330+0000 7f43d17b5640 1 --2- 172.21.15.164:0/1960389209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43cc109830 0x7f43cc1a7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:01.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:01.330+0000 7f43d17b5640 1 -- 172.21.15.164:0/1960389209 >> 172.21.15.164:0/1960389209 conn(0x7f43cc06e140 msgr2=0x7f43cc11c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:01.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:01.330+0000 7f43d17b5640 1 -- 172.21.15.164:0/1960389209 shutdown_connections 2024-09-19T08:49:01.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:01.336+0000 7f43d17b5640 1 -- 172.21.15.164:0/1960389209 wait complete. 2024-09-19T08:49:01.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:01 smithi170 ceph-mon[35072]: pgmap v99: 97 pgs: 97 active+clean; 20 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 900 MiB/s wr, 2.22k op/s 2024-09-19T08:49:01.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:01 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:49:01.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:01 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:49:01.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:01 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:49:01.406 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:01 smithi164 ceph-mon[30928]: pgmap v99: 97 pgs: 97 active+clean; 20 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 900 MiB/s wr, 2.22k op/s 2024-09-19T08:49:01.406 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:01 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:49:01.406 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:01 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:49:01.406 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:01 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:49:01.413 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:01 smithi173 ceph-mon[35306]: pgmap v99: 97 pgs: 97 active+clean; 20 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 900 MiB/s wr, 2.22k op/s 2024-09-19T08:49:01.413 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:01 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:49:01.413 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:01 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:49:01.413 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:01 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:49:02.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:02 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/1960389209' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:49:02.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:02 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/1960389209' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:49:02.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:02 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/1960389209' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:49:03.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:03 smithi173 ceph-mon[35306]: pgmap v100: 97 pgs: 97 active+clean; 20 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 727 MiB/s wr, 1.59k op/s 2024-09-19T08:49:03.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:03 smithi164 ceph-mon[30928]: pgmap v100: 97 pgs: 97 active+clean; 20 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 727 MiB/s wr, 1.59k op/s 2024-09-19T08:49:03.624 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:03 smithi170 ceph-mon[35072]: pgmap v100: 97 pgs: 97 active+clean; 20 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 727 MiB/s wr, 1.59k op/s 2024-09-19T08:49:05.172 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T08:49:05.285 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:05 smithi164 ceph-mon[30928]: pgmap v101: 97 pgs: 97 active+clean; 20 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 758 MiB/s wr, 1.86k op/s 2024-09-19T08:49:05.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:05 smithi170 ceph-mon[35072]: pgmap v101: 97 pgs: 97 active+clean; 20 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 758 MiB/s wr, 1.86k op/s 2024-09-19T08:49:05.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:05 smithi173 ceph-mon[35306]: pgmap v101: 97 pgs: 97 active+clean; 20 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 758 MiB/s wr, 1.86k op/s 2024-09-19T08:49:06.041 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:49:07.505 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:07 smithi164 ceph-mon[30928]: pgmap v102: 97 pgs: 97 active+clean; 21 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 898 MiB/s wr, 2.75k op/s 2024-09-19T08:49:07.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.562+0000 7f74c455d640 1 -- 172.21.15.164:0/2674723941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74bc102390 msgr2=0x7f74bc10e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:07.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.562+0000 7f74c455d640 1 --2- 172.21.15.164:0/2674723941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74bc102390 0x7f74bc10e9d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f74b4009cd0 tx=0x7f74b402a910 comp rx=0 tx=0).stop 2024-09-19T08:49:07.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.564+0000 7f74c455d640 1 -- 172.21.15.164:0/2674723941 shutdown_connections 2024-09-19T08:49:07.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.564+0000 7f74c455d640 1 --2- 172.21.15.164:0/2674723941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74bc102390 0x7f74bc10e9d0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.564+0000 7f74c455d640 1 --2- 172.21.15.164:0/2674723941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74bc1019f0 0x7f74bc101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.564+0000 7f74c455d640 1 --2- 172.21.15.164:0/2674723941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74bc107b00 0x7f74bc107ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.564+0000 7f74c455d640 1 -- 172.21.15.164:0/2674723941 >> 172.21.15.164:0/2674723941 conn(0x7f74bc0fd710 msgr2=0x7f74bc0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:07.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.564+0000 7f74c455d640 1 -- 172.21.15.164:0/2674723941 shutdown_connections 2024-09-19T08:49:07.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.565+0000 7f74c455d640 1 -- 172.21.15.164:0/2674723941 wait complete. 2024-09-19T08:49:07.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.566+0000 7f74c455d640 1 Processor -- start 2024-09-19T08:49:07.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.566+0000 7f74c455d640 1 -- start start 2024-09-19T08:49:07.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.566+0000 7f74c455d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74bc1019f0 0x7f74bc1162a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:07.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.566+0000 7f74c455d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74bc102390 0x7f74bc1167e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:07.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.566+0000 7f74c455d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74bc107b00 0x7f74bc1113c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:07.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.566+0000 7f74c455d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc10fd80 con 0x7f74bc1019f0 2024-09-19T08:49:07.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.566+0000 7f74c455d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc10fc00 con 0x7f74bc107b00 2024-09-19T08:49:07.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.566+0000 7f74c455d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc10ff00 con 0x7f74bc102390 2024-09-19T08:49:07.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.567+0000 7f74c22f9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74bc1019f0 0x7f74bc1162a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:07.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.567+0000 7f74c22f9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74bc1019f0 0x7f74bc1162a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43272/0 (socket says 172.21.15.164:43272) 2024-09-19T08:49:07.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.567+0000 7f74c22f9640 1 -- 172.21.15.164:0/3577752475 learned_addr learned my addr 172.21.15.164:0/3577752475 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:07.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.568+0000 7f74c22f9640 1 -- 172.21.15.164:0/3577752475 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74bc102390 msgr2=0x7f74bc1167e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T08:49:07.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.568+0000 7f74c22f9640 1 --2- 172.21.15.164:0/3577752475 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74bc102390 0x7f74bc1167e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.568+0000 7f74c22f9640 1 -- 172.21.15.164:0/3577752475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74bc107b00 msgr2=0x7f74bc1113c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T08:49:07.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.569+0000 7f74c22f9640 1 --2- 172.21.15.164:0/3577752475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74bc107b00 0x7f74bc1113c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.569+0000 7f74c22f9640 1 -- 172.21.15.164:0/3577752475 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74bc111af0 con 0x7f74bc1019f0 2024-09-19T08:49:07.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.570+0000 7f74c22f9640 1 --2- 172.21.15.164:0/3577752475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74bc1019f0 0x7f74bc1162a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f74b800b450 tx=0x7f74b800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:07.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.570+0000 7f74b37fe640 1 -- 172.21.15.164:0/3577752475 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b8015020 con 0x7f74bc1019f0 2024-09-19T08:49:07.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.571+0000 7f74c455d640 1 -- 172.21.15.164:0/3577752475 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74bc111de0 con 0x7f74bc1019f0 2024-09-19T08:49:07.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.571+0000 7f74c455d640 1 -- 172.21.15.164:0/3577752475 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74bc06bc10 con 0x7f74bc1019f0 2024-09-19T08:49:07.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.571+0000 7f74b37fe640 1 -- 172.21.15.164:0/3577752475 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74b8011d10 con 0x7f74bc1019f0 2024-09-19T08:49:07.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.571+0000 7f74b37fe640 1 -- 172.21.15.164:0/3577752475 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b800f2e0 con 0x7f74bc1019f0 2024-09-19T08:49:07.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.578+0000 7f74c6936740 1 Processor -- start 2024-09-19T08:49:07.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.578+0000 7f74c6936740 1 -- start start 2024-09-19T08:49:07.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.579+0000 7f74c6936740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566be79bcf0 0x5566be79a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:07.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.579+0000 7f74b37fe640 1 -- 172.21.15.164:0/3577752475 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f74b8018020 con 0x7f74bc1019f0 2024-09-19T08:49:07.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.579+0000 7f74c6936740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566be79a780 0x5566be79abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:07.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.579+0000 7f74c6936740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5566be7a5520 0x5566be7a7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:07.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.579+0000 7f74c6936740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5566be7002f0 con 0x5566be79bcf0 2024-09-19T08:49:07.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.579+0000 7f74c6936740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5566be6a7cf0 con 0x5566be79a780 2024-09-19T08:49:07.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.579+0000 7f74c6936740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5566be6fc2b0 con 0x5566be7a5520 2024-09-19T08:49:07.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.580+0000 7f74c2afa640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566be79bcf0 0x5566be79a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:07.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.580+0000 7f74c2afa640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566be79bcf0 0x5566be79a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43280/0 (socket says 172.21.15.164:43280) 2024-09-19T08:49:07.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.580+0000 7f74c2afa640 1 -- 172.21.15.164:0/1243304008 learned_addr learned my addr 172.21.15.164:0/1243304008 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:07.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.580+0000 7f74c1af8640 1 --2- 172.21.15.164:0/1243304008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5566be7a5520 0x5566be7a7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:07.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.581+0000 7f74b37fe640 1 --2- 172.21.15.164:0/3577752475 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f74a80a35c0 0x7f74a80a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:07.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.581+0000 7f74c1af8640 1 -- 172.21.15.164:0/1243304008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566be79a780 msgr2=0x5566be79abc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:49:07.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.581+0000 7f74b37fe640 1 -- 172.21.15.164:0/3577752475 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f74b80cf350 con 0x7f74bc1019f0 2024-09-19T08:49:07.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.582+0000 7f74c2afa640 1 --2- 172.21.15.164:0/3577752475 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f74a80a35c0 0x7f74a80a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:07.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.582+0000 7f74c1af8640 1 --2- 172.21.15.164:0/1243304008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566be79a780 0x5566be79abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.582+0000 7f74c1af8640 1 -- 172.21.15.164:0/1243304008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566be79bcf0 msgr2=0x5566be79a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:07.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.583+0000 7f74c1af8640 1 --2- 172.21.15.164:0/1243304008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566be79bcf0 0x5566be79a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.583+0000 7f74c1af8640 1 -- 172.21.15.164:0/1243304008 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566be68c3a0 con 0x5566be7a5520 2024-09-19T08:49:07.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.583+0000 7f74c2afa640 1 --2- 172.21.15.164:0/1243304008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566be79bcf0 0x5566be79a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:49:07.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.583+0000 7f74c2afa640 1 --2- 172.21.15.164:0/3577752475 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f74a80a35c0 0x7f74a80a5a80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f74b4009ca0 tx=0x7f74b40095c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:07.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.584+0000 7f74c1af8640 1 --2- 172.21.15.164:0/1243304008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5566be7a5520 0x5566be7a7910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f74bc112c10 tx=0x7f74ac00be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7627e2489056d47 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:07.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.584+0000 7f74b0ff9640 1 -- 172.21.15.164:0/1243304008 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac016070 con 0x5566be7a5520 2024-09-19T08:49:07.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.584+0000 7f74b0ff9640 1 -- 172.21.15.164:0/1243304008 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74ac005500 con 0x5566be7a5520 2024-09-19T08:49:07.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.586+0000 7f74c6936740 1 -- 172.21.15.164:0/1243304008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5566be7a5520 msgr2=0x5566be7a7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:07.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.586+0000 7f74c6936740 1 --2- 172.21.15.164:0/1243304008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5566be7a5520 0x5566be7a7910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f74bc112c10 tx=0x7f74ac00be30 comp rx=0 tx=0).stop 2024-09-19T08:49:07.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.586+0000 7f74c6936740 1 -- 172.21.15.164:0/1243304008 shutdown_connections 2024-09-19T08:49:07.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.586+0000 7f74c6936740 1 --2- 172.21.15.164:0/1243304008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5566be7a5520 0x5566be7a7910 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.586+0000 7f74c6936740 1 --2- 172.21.15.164:0/1243304008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566be79a780 0x5566be79abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.586+0000 7f74c6936740 1 --2- 172.21.15.164:0/1243304008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566be79bcf0 0x5566be79a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.586+0000 7f74c6936740 1 -- 172.21.15.164:0/1243304008 >> 172.21.15.164:0/1243304008 conn(0x5566be763310 msgr2=0x5566be796e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:07.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.586+0000 7f74c6936740 1 -- 172.21.15.164:0/1243304008 shutdown_connections 2024-09-19T08:49:07.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.586+0000 7f74c6936740 1 -- 172.21.15.164:0/1243304008 wait complete. 2024-09-19T08:49:07.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.587+0000 7f74c6936740 1 -- start start 2024-09-19T08:49:07.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.587+0000 7f74c6936740 1 Processor -- start 2024-09-19T08:49:07.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.588+0000 7f74c6936740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566be79a780 0x5566be7a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:07.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.588+0000 7f74c6936740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566be79bcf0 0x5566be7a2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:07.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.588+0000 7f74c6936740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5566be7a5520 0x5566be7a25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:07.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.588+0000 7f74c6936740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5566be7a8270 con 0x5566be79bcf0 2024-09-19T08:49:07.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.588+0000 7f74c6936740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5566be6bb060 con 0x5566be79a780 2024-09-19T08:49:07.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.588+0000 7f74c6936740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5566be7a83f0 con 0x5566be7a5520 2024-09-19T08:49:07.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.588+0000 7f74c2afa640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5566be7a5520 0x5566be7a25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:07.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.588+0000 7f74c2afa640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5566be7a5520 0x5566be7a25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:59770/0 (socket says 172.21.15.164:59770) 2024-09-19T08:49:07.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.588+0000 7f74c2afa640 1 -- 172.21.15.164:0/4284579814 learned_addr learned my addr 172.21.15.164:0/4284579814 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:07.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.588+0000 7f74c6936740 10 client.0 _ll_register_callbacks cb 0x5566be40e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:49:07.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.589+0000 7f74c1af8640 1 --2- 172.21.15.164:0/4284579814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566be79bcf0 0x5566be7a2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:07.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.589+0000 7f74c1af8640 1 -- 172.21.15.164:0/4284579814 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5566be7a5520 msgr2=0x5566be7a25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:07.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.589+0000 7f74c1af8640 1 --2- 172.21.15.164:0/4284579814 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5566be7a5520 0x5566be7a25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.589+0000 7f74c1af8640 1 -- 172.21.15.164:0/4284579814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566be79a780 msgr2=0x5566be7a4de0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:49:07.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.589+0000 7f74c1af8640 1 --2- 172.21.15.164:0/4284579814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566be79a780 0x5566be7a4de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.590+0000 7f74c1af8640 1 -- 172.21.15.164:0/4284579814 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566be4713c0 con 0x5566be79bcf0 2024-09-19T08:49:07.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.590+0000 7f74c1af8640 1 --2- 172.21.15.164:0/4284579814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566be79bcf0 0x5566be7a2090 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f74bc112c10 tx=0x7f74ac005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d9ffb848e1cb86 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:07.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.592+0000 7f74a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:49:07.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.592+0000 7f74a5ffb640 1 -- 172.21.15.164:0/4284579814 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac016070 con 0x5566be79bcf0 2024-09-19T08:49:07.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.592+0000 7f74a5ffb640 1 -- 172.21.15.164:0/4284579814 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f74ac004040 con 0x5566be79bcf0 2024-09-19T08:49:07.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.592+0000 7f74c6936740 1 -- 172.21.15.164:0/4284579814 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74ac005500 con 0x5566be79bcf0 2024-09-19T08:49:07.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.593+0000 7f74a5ffb640 1 -- 172.21.15.164:0/4284579814 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f74ac02a050 con 0x5566be79bcf0 2024-09-19T08:49:07.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.593+0000 7f74c6936740 10 client.16221 fetch_fsmap learned FSMap version 28 2024-09-19T08:49:07.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.593+0000 7f74c6936740 1 -- 172.21.15.164:0/4284579814 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5566be7b3ce0 con 0x5566be79bcf0 2024-09-19T08:49:07.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.594+0000 7f74a5ffb640 1 -- 172.21.15.164:0/4284579814 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74ac005500 con 0x5566be79bcf0 2024-09-19T08:49:07.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.594+0000 7f74a5ffb640 1 -- 172.21.15.164:0/4284579814 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f74ac010b10 con 0x5566be79bcf0 2024-09-19T08:49:07.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.594+0000 7f74c6936740 10 client.16221 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:49:07.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.594+0000 7f74c6936740 10 client.16221 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-19T08:49:07.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.595+0000 7f74c6936740 20 client.16221 populate_metadata read hostname 'smithi164' 2024-09-19T08:49:07.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.595+0000 7f74c6936740 1 --2- 172.21.15.164:0/4284579814 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5566be7b5700 0x5566be7b7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:07.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.595+0000 7f74c6936740 4 client.16221 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:49:07.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.595+0000 7f74c6936740 1 -- 172.21.15.164:0/4284579814 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566be70d860 con 0x5566be7b5700 2024-09-19T08:49:07.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.605+0000 7f74c22f9640 1 --2- 172.21.15.164:0/4284579814 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5566be7b5700 0x5566be7b7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:07.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.607+0000 7f74c22f9640 1 --2- 172.21.15.164:0/4284579814 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5566be7b5700 0x5566be7b7b60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=622a7b671d6f0db6 server_cookie=6e5999c73f9901f5 in_seq=0 out_seq=0 2024-09-19T08:49:07.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.607+0000 7f74a5ffb640 10 client.16221 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:49:07.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.612+0000 7f74a5ffb640 1 -- 172.21.15.164:0/4284579814 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5566be70d860 con 0x5566be7b5700 2024-09-19T08:49:07.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.612+0000 7f74a5ffb640 10 client.16221 handle_command_reply: tid=0 2024-09-19T08:49:07.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.613+0000 7f74c6936740 1 client.16221 shutdown 2024-09-19T08:49:07.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.614+0000 7f74c6936740 1 -- 172.21.15.164:0/4284579814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566be79bcf0 msgr2=0x5566be7a2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:07.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.614+0000 7f74c6936740 1 --2- 172.21.15.164:0/4284579814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566be79bcf0 0x5566be7a2090 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f74bc112c10 tx=0x7f74ac005250 comp rx=0 tx=0).stop 2024-09-19T08:49:07.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.617+0000 7f74c6936740 1 -- 172.21.15.164:0/4284579814 shutdown_connections 2024-09-19T08:49:07.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.617+0000 7f74c6936740 1 --2- 172.21.15.164:0/4284579814 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5566be7b5700 0x5566be7b7b60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.617+0000 7f74c6936740 1 --2- 172.21.15.164:0/4284579814 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5566be7a5520 0x5566be7a25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.617+0000 7f74c6936740 1 --2- 172.21.15.164:0/4284579814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566be79bcf0 0x5566be7a2090 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.617+0000 7f74c6936740 1 --2- 172.21.15.164:0/4284579814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566be79a780 0x5566be7a4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.617+0000 7f74c6936740 1 -- 172.21.15.164:0/4284579814 >> 172.21.15.164:0/4284579814 conn(0x5566be763310 msgr2=0x5566be79cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:07.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.617+0000 7f74a5ffb640 0 client.16221 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:49:07.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.622+0000 7f74c6936740 1 -- 172.21.15.164:0/4284579814 shutdown_connections 2024-09-19T08:49:07.624 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:07 smithi170 ceph-mon[35072]: pgmap v102: 97 pgs: 97 active+clean; 21 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 898 MiB/s wr, 2.75k op/s 2024-09-19T08:49:07.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.625+0000 7f74c6936740 1 -- 172.21.15.164:0/4284579814 wait complete. 2024-09-19T08:49:07.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.627+0000 7f74c6936740 20 client.16221 trim_cache size 0 max 16384 2024-09-19T08:49:07.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:07 smithi173 ceph-mon[35306]: pgmap v102: 97 pgs: 97 active+clean; 21 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 898 MiB/s wr, 2.75k op/s 2024-09-19T08:49:07.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.668+0000 7f74c6936740 1 Processor -- start 2024-09-19T08:49:07.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.669+0000 7f74c6936740 1 -- start start 2024-09-19T08:49:07.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.669+0000 7f74c6936740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566be79a780 0x5566be7b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:07.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.669+0000 7f74c6936740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566be79bcf0 0x5566be79d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:07.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.670+0000 7f74c6936740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5566be7a5520 0x5566be7bd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:07.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.670+0000 7f74c6936740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f74a000e740 con 0x5566be79bcf0 2024-09-19T08:49:07.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.670+0000 7f74c6936740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f74a0010280 con 0x5566be79a780 2024-09-19T08:49:07.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.670+0000 7f74c6936740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f74a000ee10 con 0x5566be7a5520 2024-09-19T08:49:07.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.670+0000 7f74c1af8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566be79bcf0 0x5566be79d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:07.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.670+0000 7f74c1af8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566be79bcf0 0x5566be79d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43298/0 (socket says 172.21.15.164:43298) 2024-09-19T08:49:07.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.670+0000 7f74c2afa640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5566be7a5520 0x5566be7bd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:07.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.670+0000 7f74c1af8640 1 -- 172.21.15.164:0/3780734457 learned_addr learned my addr 172.21.15.164:0/3780734457 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:07.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.672+0000 7f74c1af8640 1 -- 172.21.15.164:0/3780734457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5566be7a5520 msgr2=0x5566be7bd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:07.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.672+0000 7f74c1af8640 1 --2- 172.21.15.164:0/3780734457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5566be7a5520 0x5566be7bd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.672+0000 7f74c1af8640 1 -- 172.21.15.164:0/3780734457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566be79a780 msgr2=0x5566be7b15f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:49:07.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.672+0000 7f74c1af8640 1 --2- 172.21.15.164:0/3780734457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566be79a780 0x5566be7b15f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.672+0000 7f74c1af8640 1 -- 172.21.15.164:0/3780734457 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566be797350 con 0x5566be79bcf0 2024-09-19T08:49:07.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.673+0000 7f74c1af8640 1 --2- 172.21.15.164:0/3780734457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566be79bcf0 0x5566be79d4c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f74bc112c10 tx=0x7f74ac011fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31068238008501c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:07.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.674+0000 7f74a7fff640 1 -- 172.21.15.164:0/3780734457 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac016070 con 0x5566be79bcf0 2024-09-19T08:49:07.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.674+0000 7f74a7fff640 1 -- 172.21.15.164:0/3780734457 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74ac02d340 con 0x5566be79bcf0 2024-09-19T08:49:07.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.675+0000 7f74c6936740 1 -- 172.21.15.164:0/3780734457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566be79bcf0 msgr2=0x5566be79d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:07.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.675+0000 7f74c6936740 1 --2- 172.21.15.164:0/3780734457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566be79bcf0 0x5566be79d4c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f74bc112c10 tx=0x7f74ac011fa0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.676+0000 7f74c6936740 1 -- 172.21.15.164:0/3780734457 shutdown_connections 2024-09-19T08:49:07.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.676+0000 7f74c6936740 1 --2- 172.21.15.164:0/3780734457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5566be7a5520 0x5566be7bd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.676+0000 7f74c6936740 1 --2- 172.21.15.164:0/3780734457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566be79bcf0 0x5566be79d4c0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.676+0000 7f74c6936740 1 --2- 172.21.15.164:0/3780734457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566be79a780 0x5566be7b15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.676+0000 7f74c6936740 1 -- 172.21.15.164:0/3780734457 >> 172.21.15.164:0/3780734457 conn(0x5566be763310 msgr2=0x5566be7a0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:07.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.682+0000 7f74c6936740 1 -- 172.21.15.164:0/3780734457 shutdown_connections 2024-09-19T08:49:07.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.682+0000 7f74c6936740 1 -- 172.21.15.164:0/3780734457 wait complete. 2024-09-19T08:49:07.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.682+0000 7f74c6936740 1 -- start start 2024-09-19T08:49:07.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.682+0000 7f74c6936740 1 Processor -- start 2024-09-19T08:49:07.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.683+0000 7f74c6936740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566be79a780 0x5566be7a87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:07.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.683+0000 7f74c6936740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5566be79bcf0 0x5566be7a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:07.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.683+0000 7f74c6936740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566be7a5520 0x5566be81e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:07.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.683+0000 7f74c6936740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f74a000fbb0 con 0x5566be7a5520 2024-09-19T08:49:07.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.683+0000 7f74c6936740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f74a000f4e0 con 0x5566be79a780 2024-09-19T08:49:07.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.683+0000 7f74c6936740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f74a000c720 con 0x5566be79bcf0 2024-09-19T08:49:07.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.683+0000 7f74c6936740 10 client.0 _ll_register_callbacks cb 0x7f74ac00b820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:49:07.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.686+0000 7f74c22f9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566be79a780 0x5566be7a87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:07.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.686+0000 7f74c22f9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566be79a780 0x5566be7a87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:35268/0 (socket says 172.21.15.164:35268) 2024-09-19T08:49:07.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.686+0000 7f74c22f9640 1 -- 172.21.15.164:0/437421917 learned_addr learned my addr 172.21.15.164:0/437421917 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:07.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.686+0000 7f74c1af8640 1 --2- 172.21.15.164:0/437421917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5566be79bcf0 0x5566be7a8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:07.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.695+0000 7f74c2afa640 1 --2- 172.21.15.164:0/437421917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566be7a5520 0x5566be81e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:07.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.695+0000 7f74c1af8640 1 -- 172.21.15.164:0/437421917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566be79a780 msgr2=0x5566be7a87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:07.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.695+0000 7f74c1af8640 1 --2- 172.21.15.164:0/437421917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566be79a780 0x5566be7a87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.695+0000 7f74c1af8640 1 -- 172.21.15.164:0/437421917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566be7a5520 msgr2=0x5566be81e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:07.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.695+0000 7f74c1af8640 1 --2- 172.21.15.164:0/437421917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566be7a5520 0x5566be81e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.695+0000 7f74c1af8640 1 -- 172.21.15.164:0/437421917 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566be797b10 con 0x5566be79bcf0 2024-09-19T08:49:07.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.705+0000 7f74c1af8640 1 --2- 172.21.15.164:0/437421917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5566be79bcf0 0x5566be7a8d20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x5566be79d3f0 tx=0x7f74ac0054a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a34e4e8046d0e5b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:07.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.705+0000 7f74a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-09-19T08:49:07.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.707+0000 7f74a5ffb640 1 -- 172.21.15.164:0/437421917 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac016070 con 0x5566be79bcf0 2024-09-19T08:49:07.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.707+0000 7f74a5ffb640 1 -- 172.21.15.164:0/437421917 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f74ac034040 con 0x5566be79bcf0 2024-09-19T08:49:07.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.708+0000 7f74c6936740 1 -- 172.21.15.164:0/437421917 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74ac02d340 con 0x5566be79bcf0 2024-09-19T08:49:07.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.709+0000 7f74a5ffb640 1 -- 172.21.15.164:0/437421917 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f74ac00b5a0 con 0x5566be79bcf0 2024-09-19T08:49:07.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.709+0000 7f74c6936740 10 client.25636 fetch_fsmap learned FSMap version 28 2024-09-19T08:49:07.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.709+0000 7f74c6936740 1 -- 172.21.15.164:0/437421917 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5566be81f8e0 con 0x5566be79bcf0 2024-09-19T08:49:07.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.710+0000 7f74a5ffb640 1 -- 172.21.15.164:0/437421917 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74ac02d340 con 0x5566be79bcf0 2024-09-19T08:49:07.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.710+0000 7f74a5ffb640 1 -- 172.21.15.164:0/437421917 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f74ac004160 con 0x5566be79bcf0 2024-09-19T08:49:07.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.710+0000 7f74c6936740 10 client.25636 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:49:07.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.710+0000 7f74c6936740 10 client.25636 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-19T08:49:07.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.710+0000 7f74c6936740 20 client.25636 populate_metadata read hostname 'smithi164' 2024-09-19T08:49:07.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.710+0000 7f74c6936740 1 --2- 172.21.15.164:0/437421917 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5566be81fe70 0x5566be822290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:07.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.710+0000 7f74c6936740 4 client.25636 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:49:07.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.710+0000 7f74c6936740 1 -- 172.21.15.164:0/437421917 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5566be707d70 con 0x5566be81fe70 2024-09-19T08:49:07.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.714+0000 7f74c22f9640 1 --2- 172.21.15.164:0/437421917 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5566be81fe70 0x5566be822290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:07.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.716+0000 7f74c22f9640 1 --2- 172.21.15.164:0/437421917 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5566be81fe70 0x5566be822290 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=623fce2f8cf59541 server_cookie=8fb2ca3e072e8edd in_seq=0 out_seq=0 2024-09-19T08:49:07.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.716+0000 7f74a5ffb640 10 client.25636 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:49:07.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.717+0000 7f74a5ffb640 1 -- 172.21.15.164:0/437421917 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5566be707d70 con 0x5566be81fe70 2024-09-19T08:49:07.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.717+0000 7f74a5ffb640 10 client.25636 handle_command_reply: tid=0 2024-09-19T08:49:07.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.717+0000 7f74c6936740 1 client.25636 shutdown 2024-09-19T08:49:07.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.721+0000 7f74c6936740 1 -- 172.21.15.164:0/437421917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5566be79bcf0 msgr2=0x5566be7a8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:07.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.721+0000 7f74c6936740 1 --2- 172.21.15.164:0/437421917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5566be79bcf0 0x5566be7a8d20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x5566be79d3f0 tx=0x7f74ac0054a0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.725+0000 7f74c6936740 1 -- 172.21.15.164:0/437421917 shutdown_connections 2024-09-19T08:49:07.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.725+0000 7f74c6936740 1 --2- 172.21.15.164:0/437421917 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5566be81fe70 0x5566be822290 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.725+0000 7f74c6936740 1 --2- 172.21.15.164:0/437421917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566be7a5520 0x5566be81e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.725+0000 7f74c6936740 1 --2- 172.21.15.164:0/437421917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5566be79bcf0 0x5566be7a8d20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.725+0000 7f74c6936740 1 --2- 172.21.15.164:0/437421917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566be79a780 0x5566be7a87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.725+0000 7f74c6936740 1 -- 172.21.15.164:0/437421917 >> 172.21.15.164:0/437421917 conn(0x5566be763310 msgr2=0x5566be79d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:07.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.725+0000 7f74a5ffb640 0 client.25636 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:49:07.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.725+0000 7f74c6936740 1 -- 172.21.15.164:0/437421917 shutdown_connections 2024-09-19T08:49:07.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.730+0000 7f74c6936740 1 -- 172.21.15.164:0/437421917 wait complete. 2024-09-19T08:49:07.735 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:49:07.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.734+0000 7f74c6936740 20 client.25636 trim_cache size 0 max 16384 2024-09-19T08:49:07.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.737+0000 7f74c455d640 1 -- 172.21.15.164:0/3577752475 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f74a80a35c0 msgr2=0x7f74a80a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:07.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.737+0000 7f74c455d640 1 --2- 172.21.15.164:0/3577752475 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f74a80a35c0 0x7f74a80a5a80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f74b4009ca0 tx=0x7f74b40095c0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.737+0000 7f74c455d640 1 -- 172.21.15.164:0/3577752475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74bc1019f0 msgr2=0x7f74bc1162a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:07.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.737+0000 7f74c455d640 1 --2- 172.21.15.164:0/3577752475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74bc1019f0 0x7f74bc1162a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f74b800b450 tx=0x7f74b800b910 comp rx=0 tx=0).stop 2024-09-19T08:49:07.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.738+0000 7f74c455d640 1 -- 172.21.15.164:0/3577752475 shutdown_connections 2024-09-19T08:49:07.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.738+0000 7f74c455d640 1 --2- 172.21.15.164:0/3577752475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74bc107b00 0x7f74bc1113c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.738+0000 7f74c455d640 1 --2- 172.21.15.164:0/3577752475 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74bc102390 0x7f74bc1167e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.738+0000 7f74c455d640 1 --2- 172.21.15.164:0/3577752475 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f74a80a35c0 0x7f74a80a5a80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.738+0000 7f74c455d640 1 --2- 172.21.15.164:0/3577752475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74bc1019f0 0x7f74bc1162a0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:07.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.738+0000 7f74c455d640 1 -- 172.21.15.164:0/3577752475 >> 172.21.15.164:0/3577752475 conn(0x7f74bc0fd710 msgr2=0x7f74bc0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:07.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.741+0000 7f74c455d640 1 -- 172.21.15.164:0/3577752475 shutdown_connections 2024-09-19T08:49:07.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:07.741+0000 7f74c455d640 1 -- 172.21.15.164:0/3577752475 wait complete. 2024-09-19T08:49:09.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:09 smithi164 ceph-mon[30928]: pgmap v103: 97 pgs: 97 active+clean; 21 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 703 MiB/s wr, 2.27k op/s 2024-09-19T08:49:09.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:09 smithi170 ceph-mon[35072]: pgmap v103: 97 pgs: 97 active+clean; 21 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 703 MiB/s wr, 2.27k op/s 2024-09-19T08:49:09.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:09 smithi173 ceph-mon[35306]: pgmap v103: 97 pgs: 97 active+clean; 21 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 703 MiB/s wr, 2.27k op/s 2024-09-19T08:49:11.267 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T08:49:11.536 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:11 smithi164 ceph-mon[30928]: pgmap v104: 97 pgs: 97 active+clean; 22 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 899 MiB/s wr, 3.69k op/s 2024-09-19T08:49:11.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:11 smithi170 ceph-mon[35072]: pgmap v104: 97 pgs: 97 active+clean; 22 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 899 MiB/s wr, 3.69k op/s 2024-09-19T08:49:11.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:11 smithi173 ceph-mon[35306]: pgmap v104: 97 pgs: 97 active+clean; 22 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 899 MiB/s wr, 3.69k op/s 2024-09-19T08:49:12.292 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:49:13.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:13 smithi164 ceph-mon[30928]: pgmap v105: 97 pgs: 97 active+clean; 22 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 721 MiB/s wr, 3.68k op/s 2024-09-19T08:49:13.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:13 smithi170 ceph-mon[35072]: pgmap v105: 97 pgs: 97 active+clean; 22 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 721 MiB/s wr, 3.68k op/s 2024-09-19T08:49:13.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:13 smithi173 ceph-mon[35306]: pgmap v105: 97 pgs: 97 active+clean; 22 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 721 MiB/s wr, 3.68k op/s 2024-09-19T08:49:14.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.930+0000 7f1475a6a640 1 -- 172.21.15.164:0/3948835063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f147006e4f0 msgr2=0x7f147006e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:14.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.930+0000 7f1475a6a640 1 --2- 172.21.15.164:0/3948835063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f147006e4f0 0x7f147006e8d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1468009c10 tx=0x7f146802a720 comp rx=0 tx=0).stop 2024-09-19T08:49:14.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.938+0000 7f1475a6a640 1 -- 172.21.15.164:0/3948835063 shutdown_connections 2024-09-19T08:49:14.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.938+0000 7f1475a6a640 1 --2- 172.21.15.164:0/3948835063 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1470114ab0 0x7f1470116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:14.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.938+0000 7f1475a6a640 1 --2- 172.21.15.164:0/3948835063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f147006ee10 0x7f1470114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:14.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.938+0000 7f1475a6a640 1 --2- 172.21.15.164:0/3948835063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f147006e4f0 0x7f147006e8d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:14.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.938+0000 7f1475a6a640 1 -- 172.21.15.164:0/3948835063 >> 172.21.15.164:0/3948835063 conn(0x7f1470073180 msgr2=0x7f1470073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:14.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.938+0000 7f1475a6a640 1 -- 172.21.15.164:0/3948835063 shutdown_connections 2024-09-19T08:49:14.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.938+0000 7f1475a6a640 1 -- 172.21.15.164:0/3948835063 wait complete. 2024-09-19T08:49:14.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.944+0000 7f1475a6a640 1 Processor -- start 2024-09-19T08:49:14.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.944+0000 7f1475a6a640 1 -- start start 2024-09-19T08:49:14.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.945+0000 7f1475a6a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f147006e4f0 0x7f14701a7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:14.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.945+0000 7f1475a6a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f147006ee10 0x7f14701a75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:14.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.945+0000 7f1475a6a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1470114ab0 0x7f14701ab940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:14.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.945+0000 7f1475a6a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1470118370 con 0x7f1470114ab0 2024-09-19T08:49:14.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.945+0000 7f1475a6a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f14701181f0 con 0x7f147006e4f0 2024-09-19T08:49:14.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.945+0000 7f1475a6a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f14701184f0 con 0x7f147006ee10 2024-09-19T08:49:14.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.945+0000 7f1475269640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1470114ab0 0x7f14701ab940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:14.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.945+0000 7f1475269640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1470114ab0 0x7f14701ab940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58590/0 (socket says 172.21.15.164:58590) 2024-09-19T08:49:14.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.945+0000 7f1475269640 1 -- 172.21.15.164:0/3782794143 learned_addr learned my addr 172.21.15.164:0/3782794143 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:14.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.947+0000 7f1475269640 1 -- 172.21.15.164:0/3782794143 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f147006ee10 msgr2=0x7f14701a75b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T08:49:14.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.947+0000 7f1475269640 1 --2- 172.21.15.164:0/3782794143 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f147006ee10 0x7f14701a75b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:14.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.947+0000 7f1475269640 1 -- 172.21.15.164:0/3782794143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f147006e4f0 msgr2=0x7f14701a7070 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T08:49:14.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.947+0000 7f1475269640 1 --2- 172.21.15.164:0/3782794143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f147006e4f0 0x7f14701a7070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:14.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.947+0000 7f1475269640 1 -- 172.21.15.164:0/3782794143 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14701ac040 con 0x7f1470114ab0 2024-09-19T08:49:14.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.948+0000 7f1475269640 1 --2- 172.21.15.164:0/3782794143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1470114ab0 0x7f14701ab940 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f146400d680 tx=0x7f146400db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:14.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.955+0000 7f146dffb640 1 -- 172.21.15.164:0/3782794143 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1464017070 con 0x7f1470114ab0 2024-09-19T08:49:14.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.955+0000 7f146dffb640 1 -- 172.21.15.164:0/3782794143 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1464009820 con 0x7f1470114ab0 2024-09-19T08:49:14.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.955+0000 7f146dffb640 1 -- 172.21.15.164:0/3782794143 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1464004d00 con 0x7f1470114ab0 2024-09-19T08:49:14.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.956+0000 7f1475a6a640 1 -- 172.21.15.164:0/3782794143 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14701ac330 con 0x7f1470114ab0 2024-09-19T08:49:14.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.957+0000 7f1475a6a640 1 -- 172.21.15.164:0/3782794143 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f147019e5c0 con 0x7f1470114ab0 2024-09-19T08:49:14.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.967+0000 7f14790a5740 1 Processor -- start 2024-09-19T08:49:14.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.967+0000 7f14790a5740 1 -- start start 2024-09-19T08:49:14.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.967+0000 7f14790a5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602f6bd1cf0 0x5602f6bd0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:14.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.967+0000 7f14790a5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5602f6bd0780 0x5602f6bd0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:14.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.967+0000 7f14790a5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f6bdb520 0x5602f6bdd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:14.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.967+0000 7f14790a5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5602f6b362f0 con 0x5602f6bd1cf0 2024-09-19T08:49:14.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.967+0000 7f14790a5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5602f6addcf0 con 0x5602f6bd0780 2024-09-19T08:49:14.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.967+0000 7f14790a5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5602f6b322b0 con 0x5602f6bdb520 2024-09-19T08:49:14.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.974+0000 7f146ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602f6bd1cf0 0x5602f6bd0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:14.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.974+0000 7f146ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602f6bd1cf0 0x5602f6bd0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58592/0 (socket says 172.21.15.164:58592) 2024-09-19T08:49:14.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.974+0000 7f146ffff640 1 -- 172.21.15.164:0/1151072916 learned_addr learned my addr 172.21.15.164:0/1151072916 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:14.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.976+0000 7f146dffb640 1 -- 172.21.15.164:0/3782794143 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f14640099d0 con 0x7f1470114ab0 2024-09-19T08:49:14.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.978+0000 7f1474a68640 1 --2- 172.21.15.164:0/1151072916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5602f6bd0780 0x5602f6bd0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:14.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.978+0000 7f146ffff640 1 --2- 172.21.15.164:0/1151072916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f6bdb520 0x5602f6bdd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:14.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.980+0000 7f146dffb640 1 --2- 172.21.15.164:0/3782794143 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f14580a3570 0x7f14580a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:14.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.980+0000 7f146dffb640 1 -- 172.21.15.164:0/3782794143 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f14640cf680 con 0x7f1470114ab0 2024-09-19T08:49:14.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.983+0000 7f146ffff640 1 -- 172.21.15.164:0/1151072916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f6bdb520 msgr2=0x5602f6bdd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:14.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.983+0000 7f146ffff640 1 --2- 172.21.15.164:0/1151072916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f6bdb520 0x5602f6bdd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:14.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.983+0000 7f146ffff640 1 -- 172.21.15.164:0/1151072916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5602f6bd0780 msgr2=0x5602f6bd0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:14.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.983+0000 7f146ffff640 1 --2- 172.21.15.164:0/1151072916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5602f6bd0780 0x5602f6bd0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:14.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.983+0000 7f146ffff640 1 -- 172.21.15.164:0/1151072916 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602f6ac23a0 con 0x5602f6bd1cf0 2024-09-19T08:49:14.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.985+0000 7f146ffff640 1 --2- 172.21.15.164:0/1151072916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602f6bd1cf0 0x5602f6bd0240 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x5602f6bd1220 tx=0x7f146000b2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c9ad35ddf11be0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:14.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.986+0000 7f1456ffd640 1 -- 172.21.15.164:0/1151072916 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1460005220 con 0x5602f6bd1cf0 2024-09-19T08:49:14.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.986+0000 7f1456ffd640 1 -- 172.21.15.164:0/1151072916 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1460004c80 con 0x5602f6bd1cf0 2024-09-19T08:49:14.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.989+0000 7f1475269640 1 --2- 172.21.15.164:0/3782794143 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f14580a3570 0x7f14580a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:14.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.992+0000 7f14790a5740 1 -- 172.21.15.164:0/1151072916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602f6bd1cf0 msgr2=0x5602f6bd0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:14.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.992+0000 7f14790a5740 1 --2- 172.21.15.164:0/1151072916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602f6bd1cf0 0x5602f6bd0240 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x5602f6bd1220 tx=0x7f146000b2d0 comp rx=0 tx=0).stop 2024-09-19T08:49:14.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.995+0000 7f14790a5740 1 -- 172.21.15.164:0/1151072916 shutdown_connections 2024-09-19T08:49:14.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.995+0000 7f14790a5740 1 --2- 172.21.15.164:0/1151072916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f6bdb520 0x5602f6bdd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:14.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.995+0000 7f14790a5740 1 --2- 172.21.15.164:0/1151072916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5602f6bd0780 0x5602f6bd0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:14.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.995+0000 7f14790a5740 1 --2- 172.21.15.164:0/1151072916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602f6bd1cf0 0x5602f6bd0240 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:14.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:14.995+0000 7f14790a5740 1 -- 172.21.15.164:0/1151072916 >> 172.21.15.164:0/1151072916 conn(0x5602f6b99310 msgr2=0x5602f6bcce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:15.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.002+0000 7f14790a5740 1 -- 172.21.15.164:0/1151072916 shutdown_connections 2024-09-19T08:49:15.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.002+0000 7f14790a5740 1 -- 172.21.15.164:0/1151072916 wait complete. 2024-09-19T08:49:15.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.004+0000 7f14790a5740 1 -- start start 2024-09-19T08:49:15.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.005+0000 7f14790a5740 1 Processor -- start 2024-09-19T08:49:15.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.006+0000 7f14790a5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f6bd0780 0x5602f6bdaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:15.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.006+0000 7f14790a5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5602f6bd1cf0 0x5602f6bd8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:15.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.007+0000 7f14790a5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602f6bdb520 0x5602f6bd8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:15.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.007+0000 7f14790a5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5602f6bde270 con 0x5602f6bdb520 2024-09-19T08:49:15.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.007+0000 7f14790a5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5602f6af1060 con 0x5602f6bd1cf0 2024-09-19T08:49:15.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.007+0000 7f14790a5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5602f6bde3f0 con 0x5602f6bd0780 2024-09-19T08:49:15.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.007+0000 7f14790a5740 10 client.0 _ll_register_callbacks cb 0x5602f68448c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:49:15.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.011+0000 7f146ffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f6bd0780 0x5602f6bdaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:15.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.011+0000 7f146ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602f6bdb520 0x5602f6bd8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:15.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.011+0000 7f146ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602f6bdb520 0x5602f6bd8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58598/0 (socket says 172.21.15.164:58598) 2024-09-19T08:49:15.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.011+0000 7f146ffff640 1 -- 172.21.15.164:0/688669011 learned_addr learned my addr 172.21.15.164:0/688669011 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:15.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.011+0000 7f1475269640 1 --2- 172.21.15.164:0/3782794143 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f14580a3570 0x7f14580a5a30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1464098340 tx=0x7f14640046c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:15.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.011+0000 7f1474a68640 1 --2- 172.21.15.164:0/688669011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5602f6bd1cf0 0x5602f6bd8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:15.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.016+0000 7f146ffff640 1 -- 172.21.15.164:0/688669011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5602f6bd1cf0 msgr2=0x5602f6bd8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:15.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.016+0000 7f146ffff640 1 --2- 172.21.15.164:0/688669011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5602f6bd1cf0 0x5602f6bd8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:15.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.016+0000 7f146ffff640 1 -- 172.21.15.164:0/688669011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602f6bdb520 msgr2=0x5602f6bd8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:15.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.016+0000 7f146ffff640 1 --2- 172.21.15.164:0/688669011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602f6bdb520 0x5602f6bd8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:15.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.016+0000 7f146ffff640 1 -- 172.21.15.164:0/688669011 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602f68a73c0 con 0x5602f6bd0780 2024-09-19T08:49:15.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.020+0000 7f146ffff640 1 --2- 172.21.15.164:0/688669011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f6bd0780 0x5602f6bdaf90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1460005d70 tx=0x7f146000b2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15df986a3da0e660 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:15.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.021+0000 7f144ffff640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-09-19T08:49:15.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.025+0000 7f144ffff640 1 -- 172.21.15.164:0/688669011 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14600035f0 con 0x5602f6bd0780 2024-09-19T08:49:15.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.025+0000 7f14790a5740 1 -- 172.21.15.164:0/688669011 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5602f6b43860 con 0x5602f6bd0780 2024-09-19T08:49:15.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.025+0000 7f144ffff640 1 -- 172.21.15.164:0/688669011 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f1460003790 con 0x5602f6bd0780 2024-09-19T08:49:15.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.026+0000 7f144ffff640 1 -- 172.21.15.164:0/688669011 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1460003990 con 0x5602f6bd0780 2024-09-19T08:49:15.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.026+0000 7f14790a5740 10 client.25645 fetch_fsmap learned FSMap version 28 2024-09-19T08:49:15.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.026+0000 7f14790a5740 1 -- 172.21.15.164:0/688669011 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5602f6c0be90 con 0x5602f6bd0780 2024-09-19T08:49:15.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.029+0000 7f144ffff640 1 -- 172.21.15.164:0/688669011 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5602f6b43860 con 0x5602f6bd0780 2024-09-19T08:49:15.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.029+0000 7f144ffff640 1 -- 172.21.15.164:0/688669011 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1460014ce0 con 0x5602f6bd0780 2024-09-19T08:49:15.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.031+0000 7f14790a5740 10 client.25645 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:49:15.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.031+0000 7f14790a5740 10 client.25645 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T08:49:15.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.031+0000 7f14790a5740 20 client.25645 populate_metadata read hostname 'smithi164' 2024-09-19T08:49:15.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.031+0000 7f14790a5740 1 --2- 172.21.15.164:0/688669011 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x5602f6c0d580 0x5602f6c0f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:15.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.031+0000 7f14790a5740 4 client.25645 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:49:15.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.031+0000 7f14790a5740 1 -- 172.21.15.164:0/688669011 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602f6b3dd70 con 0x5602f6c0d580 2024-09-19T08:49:15.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.034+0000 7f1474a68640 1 --2- 172.21.15.164:0/688669011 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x5602f6c0d580 0x5602f6c0f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:15.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.046+0000 7f1474a68640 1 --2- 172.21.15.164:0/688669011 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x5602f6c0d580 0x5602f6c0f9b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9ec9e706b797cb72 server_cookie=df166b0775433bfd in_seq=0 out_seq=0 2024-09-19T08:49:15.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.047+0000 7f144ffff640 10 client.25645 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:49:15.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.050+0000 7f144ffff640 1 -- 172.21.15.164:0/688669011 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5602f6b3dd70 con 0x5602f6c0d580 2024-09-19T08:49:15.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.050+0000 7f144ffff640 10 client.25645 handle_command_reply: tid=0 2024-09-19T08:49:15.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.050+0000 7f14790a5740 1 client.25645 shutdown 2024-09-19T08:49:15.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.051+0000 7f14790a5740 1 -- 172.21.15.164:0/688669011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f6bd0780 msgr2=0x5602f6bdaf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:15.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.051+0000 7f14790a5740 1 --2- 172.21.15.164:0/688669011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f6bd0780 0x5602f6bdaf90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1460005d70 tx=0x7f146000b2d0 comp rx=0 tx=0).stop 2024-09-19T08:49:15.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.051+0000 7f14790a5740 1 -- 172.21.15.164:0/688669011 shutdown_connections 2024-09-19T08:49:15.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.051+0000 7f14790a5740 1 --2- 172.21.15.164:0/688669011 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x5602f6c0d580 0x5602f6c0f9b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:15.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.051+0000 7f14790a5740 1 --2- 172.21.15.164:0/688669011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602f6bdb520 0x5602f6bd8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:15.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.051+0000 7f14790a5740 1 --2- 172.21.15.164:0/688669011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5602f6bd1cf0 0x5602f6bd8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:15.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.051+0000 7f14790a5740 1 --2- 172.21.15.164:0/688669011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f6bd0780 0x5602f6bdaf90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:15.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.051+0000 7f144ffff640 0 client.25645 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:49:15.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.051+0000 7f14790a5740 1 -- 172.21.15.164:0/688669011 >> 172.21.15.164:0/688669011 conn(0x5602f6b99310 msgr2=0x5602f6bd2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:15.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.052+0000 7f14790a5740 1 -- 172.21.15.164:0/688669011 shutdown_connections 2024-09-19T08:49:15.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.052+0000 7f14790a5740 1 -- 172.21.15.164:0/688669011 wait complete. 2024-09-19T08:49:15.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.054+0000 7f14790a5740 20 client.25645 trim_cache size 0 max 16384 2024-09-19T08:49:15.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.086+0000 7f14790a5740 1 Processor -- start 2024-09-19T08:49:15.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.086+0000 7f14790a5740 1 -- start start 2024-09-19T08:49:15.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.087+0000 7f14790a5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5602f6bd0780 0x5602f6bcc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:15.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.087+0000 7f14790a5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602f6bd1cf0 0x5602f6bd32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:15.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.087+0000 7f14790a5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f6bdb520 0x5602f6bd3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:15.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.087+0000 7f14790a5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f14700744f0 con 0x5602f6bd1cf0 2024-09-19T08:49:15.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.087+0000 7f14790a5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1470076030 con 0x5602f6bd0780 2024-09-19T08:49:15.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.087+0000 7f14790a5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1470074bc0 con 0x5602f6bdb520 2024-09-19T08:49:15.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.090+0000 7f1474a68640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602f6bd1cf0 0x5602f6bd32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:15.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.090+0000 7f1474a68640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602f6bd1cf0 0x5602f6bd32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58608/0 (socket says 172.21.15.164:58608) 2024-09-19T08:49:15.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.090+0000 7f1474a68640 1 -- 172.21.15.164:0/2452301739 learned_addr learned my addr 172.21.15.164:0/2452301739 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:15.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.091+0000 7f146ffff640 1 --2- 172.21.15.164:0/2452301739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5602f6bd0780 0x5602f6bcc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:15.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.091+0000 7f146ffff640 1 --2- 172.21.15.164:0/2452301739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f6bdb520 0x5602f6bd3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:15.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.098+0000 7f146ffff640 1 -- 172.21.15.164:0/2452301739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f6bdb520 msgr2=0x5602f6bd3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:15.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.098+0000 7f146ffff640 1 --2- 172.21.15.164:0/2452301739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f6bdb520 0x5602f6bd3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:15.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.098+0000 7f146ffff640 1 -- 172.21.15.164:0/2452301739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602f6bd1cf0 msgr2=0x5602f6bd32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:15.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.098+0000 7f146ffff640 1 --2- 172.21.15.164:0/2452301739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602f6bd1cf0 0x5602f6bd32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:15.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.098+0000 7f146ffff640 1 -- 172.21.15.164:0/2452301739 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602f6bd8cf0 con 0x5602f6bd0780 2024-09-19T08:49:15.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.103+0000 7f146ffff640 1 --2- 172.21.15.164:0/2452301739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5602f6bd0780 0x5602f6bcc160 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1460001eb0 tx=0x7f146000b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4ed51f03d8b8e64 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:15.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.107+0000 7f14567fc640 1 -- 172.21.15.164:0/2452301739 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14600035f0 con 0x5602f6bd0780 2024-09-19T08:49:15.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.107+0000 7f14567fc640 1 -- 172.21.15.164:0/2452301739 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1460003790 con 0x5602f6bd0780 2024-09-19T08:49:15.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.109+0000 7f14790a5740 1 -- 172.21.15.164:0/2452301739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5602f6bd0780 msgr2=0x5602f6bcc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:15.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.109+0000 7f14790a5740 1 --2- 172.21.15.164:0/2452301739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5602f6bd0780 0x5602f6bcc160 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1460001eb0 tx=0x7f146000b260 comp rx=0 tx=0).stop 2024-09-19T08:49:15.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.113+0000 7f14790a5740 1 -- 172.21.15.164:0/2452301739 shutdown_connections 2024-09-19T08:49:15.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.113+0000 7f14790a5740 1 --2- 172.21.15.164:0/2452301739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f6bdb520 0x5602f6bd3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:15.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.113+0000 7f14790a5740 1 --2- 172.21.15.164:0/2452301739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602f6bd1cf0 0x5602f6bd32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:15.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.113+0000 7f14790a5740 1 --2- 172.21.15.164:0/2452301739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5602f6bd0780 0x5602f6bcc160 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:15.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.113+0000 7f14790a5740 1 -- 172.21.15.164:0/2452301739 >> 172.21.15.164:0/2452301739 conn(0x5602f6b99310 msgr2=0x5602f6bd6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:15.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.114+0000 7f14790a5740 1 -- 172.21.15.164:0/2452301739 shutdown_connections 2024-09-19T08:49:15.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.116+0000 7f14790a5740 1 -- 172.21.15.164:0/2452301739 wait complete. 2024-09-19T08:49:15.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.118+0000 7f14790a5740 1 -- start start 2024-09-19T08:49:15.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.118+0000 7f14790a5740 1 Processor -- start 2024-09-19T08:49:15.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.119+0000 7f14790a5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f6bd0780 0x5602f6bd4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:15.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.119+0000 7f14790a5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602f6bd1cf0 0x5602f6bd4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:15.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.119+0000 7f14790a5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5602f6bdb520 0x5602f6c76210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:15.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.119+0000 7f14790a5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1470075960 con 0x5602f6bd1cf0 2024-09-19T08:49:15.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.119+0000 7f14790a5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1470075290 con 0x5602f6bdb520 2024-09-19T08:49:15.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.119+0000 7f14790a5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f14701a4e60 con 0x5602f6bd0780 2024-09-19T08:49:15.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.119+0000 7f14790a5740 10 client.0 _ll_register_callbacks cb 0x7f146000bce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:49:15.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.122+0000 7f146ffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f6bd0780 0x5602f6bd4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:15.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.122+0000 7f146ffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f6bd0780 0x5602f6bd4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:60882/0 (socket says 172.21.15.164:60882) 2024-09-19T08:49:15.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.122+0000 7f146ffff640 1 -- 172.21.15.164:0/702867162 learned_addr learned my addr 172.21.15.164:0/702867162 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:15.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.122+0000 7f146ffff640 1 --2- 172.21.15.164:0/702867162 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5602f6bdb520 0x5602f6c76210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:15.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.123+0000 7f146ffff640 1 -- 172.21.15.164:0/702867162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f6bd0780 msgr2=0x5602f6bd4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:15.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.123+0000 7f146ffff640 1 --2- 172.21.15.164:0/702867162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f6bd0780 0x5602f6bd4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:15.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.123+0000 7f146ffff640 1 -- 172.21.15.164:0/702867162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602f6bd1cf0 msgr2=0x5602f6bd4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:15.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.123+0000 7f146ffff640 1 --2- 172.21.15.164:0/702867162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602f6bd1cf0 0x5602f6bd4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:15.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.123+0000 7f146ffff640 1 -- 172.21.15.164:0/702867162 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602f6bd58c0 con 0x5602f6bdb520 2024-09-19T08:49:15.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.124+0000 7f146ffff640 1 --2- 172.21.15.164:0/702867162 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5602f6bdb520 0x5602f6c76210 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1460005dc0 tx=0x7f146000b2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94a4d032ce30abc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:15.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.127+0000 7f144ffff640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-19T08:49:15.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.127+0000 7f144ffff640 1 -- 172.21.15.164:0/702867162 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14600035f0 con 0x5602f6bdb520 2024-09-19T08:49:15.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.127+0000 7f14790a5740 1 -- 172.21.15.164:0/702867162 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1460003790 con 0x5602f6bdb520 2024-09-19T08:49:15.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.127+0000 7f144ffff640 1 -- 172.21.15.164:0/702867162 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f1460014030 con 0x5602f6bdb520 2024-09-19T08:49:15.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.129+0000 7f144ffff640 1 -- 172.21.15.164:0/702867162 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1460050050 con 0x5602f6bdb520 2024-09-19T08:49:15.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.133+0000 7f14790a5740 10 client.25655 fetch_fsmap learned FSMap version 28 2024-09-19T08:49:15.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.133+0000 7f14790a5740 1 -- 172.21.15.164:0/702867162 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5602f6c77880 con 0x5602f6bdb520 2024-09-19T08:49:15.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.138+0000 7f144ffff640 1 -- 172.21.15.164:0/702867162 <== mon.2 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1460003790 con 0x5602f6bdb520 2024-09-19T08:49:15.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.138+0000 7f144ffff640 1 -- 172.21.15.164:0/702867162 <== mon.2 v2:172.21.15.170:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1460034020 con 0x5602f6bdb520 2024-09-19T08:49:15.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.139+0000 7f14790a5740 10 client.25655 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:49:15.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.139+0000 7f14790a5740 10 client.25655 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T08:49:15.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.139+0000 7f14790a5740 20 client.25655 populate_metadata read hostname 'smithi164' 2024-09-19T08:49:15.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.139+0000 7f14790a5740 1 --2- 172.21.15.164:0/702867162 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x5602f6c77ea0 0x5602f6c7a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:15.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.140+0000 7f14790a5740 4 client.25655 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:49:15.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.140+0000 7f14790a5740 1 -- 172.21.15.164:0/702867162 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5602f6aba170 con 0x5602f6c77ea0 2024-09-19T08:49:15.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.144+0000 7f1474a68640 1 --2- 172.21.15.164:0/702867162 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x5602f6c77ea0 0x5602f6c7a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:15.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.145+0000 7f1474a68640 1 --2- 172.21.15.164:0/702867162 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x5602f6c77ea0 0x5602f6c7a2c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=90dcae3f4cf73cfe server_cookie=56275b1f8b02d493 in_seq=0 out_seq=0 2024-09-19T08:49:15.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.146+0000 7f144ffff640 10 client.25655 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:49:15.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.146+0000 7f144ffff640 1 -- 172.21.15.164:0/702867162 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5602f6aba170 con 0x5602f6c77ea0 2024-09-19T08:49:15.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.146+0000 7f144ffff640 10 client.25655 handle_command_reply: tid=0 2024-09-19T08:49:15.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.146+0000 7f14790a5740 1 client.25655 shutdown 2024-09-19T08:49:15.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.147+0000 7f14790a5740 1 -- 172.21.15.164:0/702867162 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5602f6bdb520 msgr2=0x5602f6c76210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:15.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.147+0000 7f14790a5740 1 --2- 172.21.15.164:0/702867162 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5602f6bdb520 0x5602f6c76210 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1460005dc0 tx=0x7f146000b2d0 comp rx=0 tx=0).stop 2024-09-19T08:49:15.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.148+0000 7f14790a5740 1 -- 172.21.15.164:0/702867162 shutdown_connections 2024-09-19T08:49:15.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.148+0000 7f14790a5740 1 --2- 172.21.15.164:0/702867162 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x5602f6c77ea0 0x5602f6c7a2c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:15.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.148+0000 7f14790a5740 1 --2- 172.21.15.164:0/702867162 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5602f6bdb520 0x5602f6c76210 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:15.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.148+0000 7f14790a5740 1 --2- 172.21.15.164:0/702867162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602f6bd1cf0 0x5602f6bd4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:15.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.148+0000 7f14790a5740 1 --2- 172.21.15.164:0/702867162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f6bd0780 0x5602f6bd4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:15.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.148+0000 7f14790a5740 1 -- 172.21.15.164:0/702867162 >> 172.21.15.164:0/702867162 conn(0x5602f6b99310 msgr2=0x5602f6bd67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:15.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.148+0000 7f144ffff640 0 client.25655 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:49:15.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.149+0000 7f14790a5740 1 -- 172.21.15.164:0/702867162 shutdown_connections 2024-09-19T08:49:15.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.150+0000 7f14790a5740 1 -- 172.21.15.164:0/702867162 wait complete. 2024-09-19T08:49:15.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.151+0000 7f14790a5740 20 client.25655 trim_cache size 0 max 16384 2024-09-19T08:49:15.157 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:49:15.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.154+0000 7f1475a6a640 1 -- 172.21.15.164:0/3782794143 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f14580a3570 msgr2=0x7f14580a5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:15.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.154+0000 7f1475a6a640 1 --2- 172.21.15.164:0/3782794143 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f14580a3570 0x7f14580a5a30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1464098340 tx=0x7f14640046c0 comp rx=0 tx=0).stop 2024-09-19T08:49:15.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.154+0000 7f1475a6a640 1 -- 172.21.15.164:0/3782794143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1470114ab0 msgr2=0x7f14701ab940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:15.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.154+0000 7f1475a6a640 1 --2- 172.21.15.164:0/3782794143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1470114ab0 0x7f14701ab940 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f146400d680 tx=0x7f146400db40 comp rx=0 tx=0).stop 2024-09-19T08:49:15.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.162+0000 7f1475a6a640 1 -- 172.21.15.164:0/3782794143 shutdown_connections 2024-09-19T08:49:15.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.162+0000 7f1475a6a640 1 --2- 172.21.15.164:0/3782794143 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f14580a3570 0x7f14580a5a30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:15.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.162+0000 7f1475a6a640 1 --2- 172.21.15.164:0/3782794143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1470114ab0 0x7f14701ab940 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:15.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.162+0000 7f1475a6a640 1 --2- 172.21.15.164:0/3782794143 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f147006ee10 0x7f14701a75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:15.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.162+0000 7f1475a6a640 1 --2- 172.21.15.164:0/3782794143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f147006e4f0 0x7f14701a7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:15.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.162+0000 7f1475a6a640 1 -- 172.21.15.164:0/3782794143 >> 172.21.15.164:0/3782794143 conn(0x7f1470073180 msgr2=0x7f147006b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:15.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.163+0000 7f1475a6a640 1 -- 172.21.15.164:0/3782794143 shutdown_connections 2024-09-19T08:49:15.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:15.163+0000 7f1475a6a640 1 -- 172.21.15.164:0/3782794143 wait complete. 2024-09-19T08:49:15.541 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:15 smithi164 ceph-mon[30928]: pgmap v106: 97 pgs: 97 active+clean; 23 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 699 MiB/s wr, 3.74k op/s 2024-09-19T08:49:15.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:15 smithi170 ceph-mon[35072]: pgmap v106: 97 pgs: 97 active+clean; 23 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 699 MiB/s wr, 3.74k op/s 2024-09-19T08:49:15.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:15 smithi173 ceph-mon[35306]: pgmap v106: 97 pgs: 97 active+clean; 23 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 699 MiB/s wr, 3.74k op/s 2024-09-19T08:49:17.509 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T08:49:17.597 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:17 smithi164 ceph-mon[30928]: pgmap v107: 97 pgs: 97 active+clean; 25 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 798 MiB/s wr, 4.00k op/s 2024-09-19T08:49:17.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:17 smithi170 ceph-mon[35072]: pgmap v107: 97 pgs: 97 active+clean; 25 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 798 MiB/s wr, 4.00k op/s 2024-09-19T08:49:17.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:17 smithi173 ceph-mon[35306]: pgmap v107: 97 pgs: 97 active+clean; 25 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 798 MiB/s wr, 4.00k op/s 2024-09-19T08:49:18.470 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:49:19.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:19 smithi170 ceph-mon[35072]: pgmap v108: 97 pgs: 97 active+clean; 25 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 594 MiB/s wr, 2.94k op/s 2024-09-19T08:49:19.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:19 smithi173 ceph-mon[35306]: pgmap v108: 97 pgs: 97 active+clean; 25 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 594 MiB/s wr, 2.94k op/s 2024-09-19T08:49:19.783 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:19 smithi164 ceph-mon[30928]: pgmap v108: 97 pgs: 97 active+clean; 25 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 594 MiB/s wr, 2.94k op/s 2024-09-19T08:49:20.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.346+0000 7f7676a56640 1 -- 172.21.15.164:0/1684207279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7670110190 msgr2=0x7f767006e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:20.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.346+0000 7f7676a56640 1 --2- 172.21.15.164:0/1684207279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7670110190 0x7f767006e4f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f766c009900 tx=0x7f766c02a440 comp rx=0 tx=0).stop 2024-09-19T08:49:20.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.350+0000 7f7676a56640 1 -- 172.21.15.164:0/1684207279 shutdown_connections 2024-09-19T08:49:20.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.350+0000 7f7676a56640 1 --2- 172.21.15.164:0/1684207279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f767011b140 0x7f767011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.350+0000 7f7676a56640 1 --2- 172.21.15.164:0/1684207279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f767006ea30 0x7f767006ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.350+0000 7f7676a56640 1 --2- 172.21.15.164:0/1684207279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7670110190 0x7f767006e4f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.351+0000 7f7676a56640 1 -- 172.21.15.164:0/1684207279 >> 172.21.15.164:0/1684207279 conn(0x7f7670073180 msgr2=0x7f7670073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:20.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.351+0000 7f7676a56640 1 -- 172.21.15.164:0/1684207279 shutdown_connections 2024-09-19T08:49:20.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.351+0000 7f7676a56640 1 -- 172.21.15.164:0/1684207279 wait complete. 2024-09-19T08:49:20.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.352+0000 7f7676a56640 1 Processor -- start 2024-09-19T08:49:20.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.352+0000 7f7676a56640 1 -- start start 2024-09-19T08:49:20.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.353+0000 7f7676a56640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f767006ea30 0x7f76701186f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:20.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.353+0000 7f7676a56640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7670110190 0x7f7670111780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:20.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.353+0000 7f7676a56640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f767011b140 0x7f7670111cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:20.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.353+0000 7f7676a56640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7670120be0 con 0x7f767006ea30 2024-09-19T08:49:20.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.353+0000 7f7676a56640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7670120a60 con 0x7f767011b140 2024-09-19T08:49:20.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.353+0000 7f7676a56640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7670120d60 con 0x7f7670110190 2024-09-19T08:49:20.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.353+0000 7f7675253640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7670110190 0x7f7670111780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:20.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.354+0000 7f7675253640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7670110190 0x7f7670111780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:60894/0 (socket says 172.21.15.164:60894) 2024-09-19T08:49:20.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.354+0000 7f7675253640 1 -- 172.21.15.164:0/1003715367 learned_addr learned my addr 172.21.15.164:0/1003715367 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:20.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.354+0000 7f7676255640 1 --2- 172.21.15.164:0/1003715367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f767011b140 0x7f7670111cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:20.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.354+0000 7f7675a54640 1 --2- 172.21.15.164:0/1003715367 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f767006ea30 0x7f76701186f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:20.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.354+0000 7f7675253640 1 -- 172.21.15.164:0/1003715367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f767011b140 msgr2=0x7f7670111cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:20.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.354+0000 7f7675253640 1 --2- 172.21.15.164:0/1003715367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f767011b140 0x7f7670111cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.354+0000 7f7675253640 1 -- 172.21.15.164:0/1003715367 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f767006ea30 msgr2=0x7f76701186f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:20.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.354+0000 7f7675253640 1 --2- 172.21.15.164:0/1003715367 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f767006ea30 0x7f76701186f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.354+0000 7f7675253640 1 -- 172.21.15.164:0/1003715367 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7670112280 con 0x7f7670110190 2024-09-19T08:49:20.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.356+0000 7f7675253640 1 --2- 172.21.15.164:0/1003715367 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7670110190 0x7f7670111780 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f766000e9f0 tx=0x7f766000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:20.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.356+0000 7f7666ffd640 1 -- 172.21.15.164:0/1003715367 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766000cd80 con 0x7f7670110190 2024-09-19T08:49:20.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.356+0000 7f7676a56640 1 -- 172.21.15.164:0/1003715367 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7670112570 con 0x7f7670110190 2024-09-19T08:49:20.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.356+0000 7f7676a56640 1 -- 172.21.15.164:0/1003715367 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7670116590 con 0x7f7670110190 2024-09-19T08:49:20.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.357+0000 7f7666ffd640 1 -- 172.21.15.164:0/1003715367 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7660010910 con 0x7f7670110190 2024-09-19T08:49:20.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.357+0000 7f7666ffd640 1 -- 172.21.15.164:0/1003715367 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7660018990 con 0x7f7670110190 2024-09-19T08:49:20.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.367+0000 7f767a091740 1 Processor -- start 2024-09-19T08:49:20.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.367+0000 7f767a091740 1 -- start start 2024-09-19T08:49:20.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.368+0000 7f767a091740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3d3445d20 0x55c3d3444270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:20.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.368+0000 7f767a091740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3d34447b0 0x55c3d3444bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:20.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.374+0000 7f7676255640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3d3445d20 0x55c3d3444270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:20.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.374+0000 7f7676255640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3d3445d20 0x55c3d3444270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58648/0 (socket says 172.21.15.164:58648) 2024-09-19T08:49:20.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.374+0000 7f7676255640 1 -- 172.21.15.164:0/1865498090 learned_addr learned my addr 172.21.15.164:0/1865498090 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:20.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.374+0000 7f767a091740 1 --2- 172.21.15.164:0/1865498090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3d344f550 0x55c3d3451940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:20.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.374+0000 7f767a091740 1 -- 172.21.15.164:0/1865498090 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c3d33aa2f0 con 0x55c3d3445d20 2024-09-19T08:49:20.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.374+0000 7f767a091740 1 -- 172.21.15.164:0/1865498090 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c3d3351cf0 con 0x55c3d34447b0 2024-09-19T08:49:20.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.374+0000 7f767a091740 1 -- 172.21.15.164:0/1865498090 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c3d33a62b0 con 0x55c3d344f550 2024-09-19T08:49:20.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.375+0000 7f7666ffd640 1 -- 172.21.15.164:0/1003715367 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f7660018b30 con 0x7f7670110190 2024-09-19T08:49:20.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.375+0000 7f7675a54640 1 --2- 172.21.15.164:0/1865498090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3d34447b0 0x55c3d3444bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:20.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.376+0000 7f7675253640 1 --2- 172.21.15.164:0/1865498090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3d344f550 0x55c3d3451940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:20.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.376+0000 7f7675a54640 1 -- 172.21.15.164:0/1865498090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3d344f550 msgr2=0x55c3d3451940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:20.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.376+0000 7f7675a54640 1 --2- 172.21.15.164:0/1865498090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3d344f550 0x55c3d3451940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.376+0000 7f7675a54640 1 -- 172.21.15.164:0/1865498090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3d3445d20 msgr2=0x55c3d3444270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:20.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.376+0000 7f7675a54640 1 --2- 172.21.15.164:0/1865498090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3d3445d20 0x55c3d3444270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.376+0000 7f7675a54640 1 -- 172.21.15.164:0/1865498090 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3d33363a0 con 0x55c3d34447b0 2024-09-19T08:49:20.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.377+0000 7f7675a54640 1 --2- 172.21.15.164:0/1865498090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3d34447b0 0x55c3d3444bf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f766c009230 tx=0x7f766c0094b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79217e70a7642429 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:20.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.378+0000 7f7666ffd640 1 --2- 172.21.15.164:0/1003715367 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f765c0a3500 0x7f765c0a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:20.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.378+0000 7f765bfff640 1 -- 172.21.15.164:0/1865498090 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c040070 con 0x55c3d34447b0 2024-09-19T08:49:20.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.378+0000 7f765bfff640 1 -- 172.21.15.164:0/1865498090 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f766c004e10 con 0x55c3d34447b0 2024-09-19T08:49:20.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.379+0000 7f7666ffd640 1 -- 172.21.15.164:0/1003715367 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f76600d10b0 con 0x7f7670110190 2024-09-19T08:49:20.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.380+0000 7f7676255640 1 --2- 172.21.15.164:0/1003715367 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f765c0a3500 0x7f765c0a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:20.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.380+0000 7f767a091740 1 -- 172.21.15.164:0/1865498090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3d34447b0 msgr2=0x55c3d3444bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:20.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.380+0000 7f767a091740 1 --2- 172.21.15.164:0/1865498090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3d34447b0 0x55c3d3444bf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f766c009230 tx=0x7f766c0094b0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.381+0000 7f767a091740 1 -- 172.21.15.164:0/1865498090 shutdown_connections 2024-09-19T08:49:20.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.381+0000 7f767a091740 1 --2- 172.21.15.164:0/1865498090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3d344f550 0x55c3d3451940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.381+0000 7f767a091740 1 --2- 172.21.15.164:0/1865498090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3d34447b0 0x55c3d3444bf0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.381+0000 7f767a091740 1 --2- 172.21.15.164:0/1865498090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3d3445d20 0x55c3d3444270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.381+0000 7f767a091740 1 -- 172.21.15.164:0/1865498090 >> 172.21.15.164:0/1865498090 conn(0x55c3d340d310 msgr2=0x55c3d3440e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:20.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.381+0000 7f767a091740 1 -- 172.21.15.164:0/1865498090 shutdown_connections 2024-09-19T08:49:20.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.381+0000 7f767a091740 1 -- 172.21.15.164:0/1865498090 wait complete. 2024-09-19T08:49:20.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.381+0000 7f7676255640 1 --2- 172.21.15.164:0/1003715367 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f765c0a3500 0x7f765c0a59c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f76680123f0 tx=0x7f7668009550 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:20.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.382+0000 7f767a091740 1 -- start start 2024-09-19T08:49:20.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.382+0000 7f767a091740 1 Processor -- start 2024-09-19T08:49:20.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.383+0000 7f767a091740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3d34447b0 0x55c3d344ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:20.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.384+0000 7f767a091740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3d3445d20 0x55c3d344c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:20.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.384+0000 7f767a091740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3d344f550 0x55c3d344c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:20.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.384+0000 7f767a091740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c3d3452240 con 0x55c3d344f550 2024-09-19T08:49:20.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.384+0000 7f767a091740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c3d3365060 con 0x55c3d34447b0 2024-09-19T08:49:20.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.384+0000 7f767a091740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c3d34523c0 con 0x55c3d3445d20 2024-09-19T08:49:20.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.384+0000 7f767a091740 10 client.0 _ll_register_callbacks cb 0x55c3d30b88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:49:20.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.384+0000 7f7676255640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3d344f550 0x55c3d344c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:20.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.384+0000 7f7676255640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3d344f550 0x55c3d344c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58652/0 (socket says 172.21.15.164:58652) 2024-09-19T08:49:20.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.384+0000 7f7676255640 1 -- 172.21.15.164:0/31330207 learned_addr learned my addr 172.21.15.164:0/31330207 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:20.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.385+0000 7f7675253640 1 --2- 172.21.15.164:0/31330207 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3d3445d20 0x55c3d344c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:20.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.385+0000 7f7675a54640 1 --2- 172.21.15.164:0/31330207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3d34447b0 0x55c3d344ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:20.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.385+0000 7f7676255640 1 -- 172.21.15.164:0/31330207 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3d3445d20 msgr2=0x55c3d344c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:20.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.385+0000 7f7676255640 1 --2- 172.21.15.164:0/31330207 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3d3445d20 0x55c3d344c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.385+0000 7f7676255640 1 -- 172.21.15.164:0/31330207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3d34447b0 msgr2=0x55c3d344ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:20.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.385+0000 7f7676255640 1 --2- 172.21.15.164:0/31330207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3d34447b0 0x55c3d344ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.385+0000 7f7676255640 1 -- 172.21.15.164:0/31330207 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3d311b3c0 con 0x55c3d344f550 2024-09-19T08:49:20.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.386+0000 7f7676255640 1 --2- 172.21.15.164:0/31330207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3d344f550 0x55c3d344c630 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f7668003eb0 tx=0x7f7668004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d3a0941258ccc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:20.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.386+0000 7f76597fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:49:20.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.386+0000 7f76597fa640 1 -- 172.21.15.164:0/31330207 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766801f8f0 con 0x55c3d344f550 2024-09-19T08:49:20.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.386+0000 7f76597fa640 1 -- 172.21.15.164:0/31330207 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f766801fa90 con 0x55c3d344f550 2024-09-19T08:49:20.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.387+0000 7f767a091740 1 -- 172.21.15.164:0/31330207 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c3d33b7860 con 0x55c3d344f550 2024-09-19T08:49:20.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.387+0000 7f76597fa640 1 -- 172.21.15.164:0/31330207 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f766801fcd0 con 0x55c3d344f550 2024-09-19T08:49:20.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.388+0000 7f767a091740 10 client.16251 fetch_fsmap learned FSMap version 28 2024-09-19T08:49:20.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.388+0000 7f767a091740 1 -- 172.21.15.164:0/31330207 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c3d345dd00 con 0x55c3d344f550 2024-09-19T08:49:20.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.389+0000 7f76597fa640 1 -- 172.21.15.164:0/31330207 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7668031900 con 0x55c3d344f550 2024-09-19T08:49:20.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.389+0000 7f76597fa640 1 -- 172.21.15.164:0/31330207 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f766800f040 con 0x55c3d344f550 2024-09-19T08:49:20.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.390+0000 7f767a091740 10 client.16251 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:49:20.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.390+0000 7f767a091740 10 client.16251 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T08:49:20.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.390+0000 7f767a091740 20 client.16251 populate_metadata read hostname 'smithi164' 2024-09-19T08:49:20.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.390+0000 7f767a091740 1 --2- 172.21.15.164:0/31330207 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55c3d345f1c0 0x55c3d3461650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:20.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.390+0000 7f767a091740 4 client.16251 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:49:20.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.390+0000 7f767a091740 1 -- 172.21.15.164:0/31330207 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3d33b7860 con 0x55c3d345f1c0 2024-09-19T08:49:20.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.391+0000 7f7675a54640 1 --2- 172.21.15.164:0/31330207 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55c3d345f1c0 0x55c3d3461650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:20.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.394+0000 7f7675a54640 1 --2- 172.21.15.164:0/31330207 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55c3d345f1c0 0x55c3d3461650 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be713a2e7d836d7a server_cookie=42ae098ad531ec87 in_seq=0 out_seq=0 2024-09-19T08:49:20.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.394+0000 7f76597fa640 10 client.16251 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:49:20.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.397+0000 7f76597fa640 1 -- 172.21.15.164:0/31330207 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c3d33b7860 con 0x55c3d345f1c0 2024-09-19T08:49:20.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.397+0000 7f76597fa640 10 client.16251 handle_command_reply: tid=0 2024-09-19T08:49:20.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.397+0000 7f767a091740 1 client.16251 shutdown 2024-09-19T08:49:20.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.397+0000 7f767a091740 1 -- 172.21.15.164:0/31330207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3d344f550 msgr2=0x55c3d344c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:20.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.397+0000 7f767a091740 1 --2- 172.21.15.164:0/31330207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3d344f550 0x55c3d344c630 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f7668003eb0 tx=0x7f7668004a40 comp rx=0 tx=0).stop 2024-09-19T08:49:20.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.398+0000 7f767a091740 1 -- 172.21.15.164:0/31330207 shutdown_connections 2024-09-19T08:49:20.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.398+0000 7f767a091740 1 --2- 172.21.15.164:0/31330207 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55c3d345f1c0 0x55c3d3461650 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.398+0000 7f76597fa640 0 client.16251 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:49:20.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.398+0000 7f767a091740 1 --2- 172.21.15.164:0/31330207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3d344f550 0x55c3d344c630 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.398+0000 7f767a091740 1 --2- 172.21.15.164:0/31330207 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3d3445d20 0x55c3d344c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.398+0000 7f767a091740 1 --2- 172.21.15.164:0/31330207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3d34447b0 0x55c3d344ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.398+0000 7f767a091740 1 -- 172.21.15.164:0/31330207 >> 172.21.15.164:0/31330207 conn(0x55c3d340d310 msgr2=0x55c3d3446b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:20.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.408+0000 7f767a091740 1 -- 172.21.15.164:0/31330207 shutdown_connections 2024-09-19T08:49:20.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.409+0000 7f767a091740 1 -- 172.21.15.164:0/31330207 wait complete. 2024-09-19T08:49:20.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.413+0000 7f767a091740 20 client.16251 trim_cache size 0 max 16384 2024-09-19T08:49:20.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.448+0000 7f767a091740 1 Processor -- start 2024-09-19T08:49:20.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.453+0000 7f767a091740 1 -- start start 2024-09-19T08:49:20.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.453+0000 7f767a091740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3d340d310 0x55c3d345b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:20.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.453+0000 7f767a091740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3d34447b0 0x55c3d3451510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:20.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.453+0000 7f767a091740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3d3445d20 0x55c3d3450940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:20.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.453+0000 7f767a091740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f765400e740 con 0x55c3d340d310 2024-09-19T08:49:20.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.453+0000 7f767a091740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7654010280 con 0x55c3d34447b0 2024-09-19T08:49:20.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.453+0000 7f767a091740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f765400ee10 con 0x55c3d3445d20 2024-09-19T08:49:20.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.454+0000 7f7675a54640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3d340d310 0x55c3d345b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:20.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.455+0000 7f7675a54640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3d340d310 0x55c3d345b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58666/0 (socket says 172.21.15.164:58666) 2024-09-19T08:49:20.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.455+0000 7f7675a54640 1 -- 172.21.15.164:0/1285524233 learned_addr learned my addr 172.21.15.164:0/1285524233 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:20.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.455+0000 7f7676255640 1 --2- 172.21.15.164:0/1285524233 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3d3445d20 0x55c3d3450940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:20.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.456+0000 7f7675a54640 1 -- 172.21.15.164:0/1285524233 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3d3445d20 msgr2=0x55c3d3450940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:20.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.456+0000 7f7675a54640 1 --2- 172.21.15.164:0/1285524233 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3d3445d20 0x55c3d3450940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.456+0000 7f7675a54640 1 -- 172.21.15.164:0/1285524233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3d34447b0 msgr2=0x55c3d3451510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:20.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.456+0000 7f7675a54640 1 --2- 172.21.15.164:0/1285524233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3d34447b0 0x55c3d3451510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.456+0000 7f7675a54640 1 -- 172.21.15.164:0/1285524233 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3d344a640 con 0x55c3d340d310 2024-09-19T08:49:20.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.456+0000 7f7675a54640 1 --2- 172.21.15.164:0/1285524233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3d340d310 0x55c3d345b590 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f766c005640 tx=0x7f766c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e66ffe2deca297 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:20.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.462+0000 7f765b7fe640 1 -- 172.21.15.164:0/1285524233 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c040040 con 0x55c3d340d310 2024-09-19T08:49:20.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.462+0000 7f765b7fe640 1 -- 172.21.15.164:0/1285524233 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f766c0048c0 con 0x55c3d340d310 2024-09-19T08:49:20.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.462+0000 7f765b7fe640 1 -- 172.21.15.164:0/1285524233 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c031440 con 0x55c3d340d310 2024-09-19T08:49:20.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.463+0000 7f767a091740 1 -- 172.21.15.164:0/1285524233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3d340d310 msgr2=0x55c3d345b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:20.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.463+0000 7f767a091740 1 --2- 172.21.15.164:0/1285524233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3d340d310 0x55c3d345b590 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f766c005640 tx=0x7f766c005fd0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.464+0000 7f767a091740 1 -- 172.21.15.164:0/1285524233 shutdown_connections 2024-09-19T08:49:20.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.464+0000 7f767a091740 1 --2- 172.21.15.164:0/1285524233 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3d3445d20 0x55c3d3450940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.464+0000 7f767a091740 1 --2- 172.21.15.164:0/1285524233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3d34447b0 0x55c3d3451510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.464+0000 7f767a091740 1 --2- 172.21.15.164:0/1285524233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3d340d310 0x55c3d345b590 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.464+0000 7f767a091740 1 -- 172.21.15.164:0/1285524233 >> 172.21.15.164:0/1285524233 conn(0x55c3d344c790 msgr2=0x55c3d344b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:20.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.466+0000 7f767a091740 1 -- 172.21.15.164:0/1285524233 shutdown_connections 2024-09-19T08:49:20.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.466+0000 7f767a091740 1 -- 172.21.15.164:0/1285524233 wait complete. 2024-09-19T08:49:20.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.467+0000 7f767a091740 1 -- start start 2024-09-19T08:49:20.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.467+0000 7f767a091740 1 Processor -- start 2024-09-19T08:49:20.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.468+0000 7f767a091740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3d340d310 0x55c3d3457eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:20.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.468+0000 7f767a091740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3d34447b0 0x55c3d3450700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:20.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.468+0000 7f767a091740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3d3445d20 0x55c3d3450c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:20.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.468+0000 7f767a091740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f765400fbb0 con 0x55c3d34447b0 2024-09-19T08:49:20.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.468+0000 7f767a091740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f765400f4e0 con 0x55c3d3445d20 2024-09-19T08:49:20.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.468+0000 7f767a091740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f765400c720 con 0x55c3d340d310 2024-09-19T08:49:20.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.468+0000 7f767a091740 10 client.0 _ll_register_callbacks cb 0x7f7668009c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:49:20.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.469+0000 7f7676255640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3d3445d20 0x55c3d3450c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:20.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.470+0000 7f7675a54640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3d340d310 0x55c3d3457eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:20.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.470+0000 7f7676255640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3d3445d20 0x55c3d3450c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:38466/0 (socket says 172.21.15.164:38466) 2024-09-19T08:49:20.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.470+0000 7f7676255640 1 -- 172.21.15.164:0/950516591 learned_addr learned my addr 172.21.15.164:0/950516591 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:20.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.472+0000 7f7675253640 1 --2- 172.21.15.164:0/950516591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3d34447b0 0x55c3d3450700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:20.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.473+0000 7f7675a54640 1 -- 172.21.15.164:0/950516591 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3d3445d20 msgr2=0x55c3d3450c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:20.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.473+0000 7f7675a54640 1 --2- 172.21.15.164:0/950516591 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3d3445d20 0x55c3d3450c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.473+0000 7f7675a54640 1 -- 172.21.15.164:0/950516591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3d34447b0 msgr2=0x55c3d3450700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:20.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.473+0000 7f7675a54640 1 --2- 172.21.15.164:0/950516591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3d34447b0 0x55c3d3450700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.473+0000 7f7675a54640 1 -- 172.21.15.164:0/950516591 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3d3451420 con 0x55c3d340d310 2024-09-19T08:49:20.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.474+0000 7f7675a54640 1 --2- 172.21.15.164:0/950516591 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3d340d310 0x55c3d3457eb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f766c009850 tx=0x7f766c045b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1aa21494a3628f0b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:20.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.474+0000 7f76597fa640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-09-19T08:49:20.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.476+0000 7f76597fa640 1 -- 172.21.15.164:0/950516591 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c04a030 con 0x55c3d340d310 2024-09-19T08:49:20.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.476+0000 7f76597fa640 1 -- 172.21.15.164:0/950516591 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f766c004fd0 con 0x55c3d340d310 2024-09-19T08:49:20.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.476+0000 7f767a091740 1 -- 172.21.15.164:0/950516591 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f766c0048c0 con 0x55c3d340d310 2024-09-19T08:49:20.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.477+0000 7f76597fa640 1 -- 172.21.15.164:0/950516591 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f766c040040 con 0x55c3d340d310 2024-09-19T08:49:20.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.478+0000 7f767a091740 10 client.25669 fetch_fsmap learned FSMap version 28 2024-09-19T08:49:20.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.479+0000 7f767a091740 1 -- 172.21.15.164:0/950516591 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c3d34c9ef0 con 0x55c3d340d310 2024-09-19T08:49:20.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.480+0000 7f76597fa640 1 -- 172.21.15.164:0/950516591 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f766c0048c0 con 0x55c3d340d310 2024-09-19T08:49:20.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.480+0000 7f76597fa640 1 -- 172.21.15.164:0/950516591 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f766c063920 con 0x55c3d340d310 2024-09-19T08:49:20.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.480+0000 7f767a091740 10 client.25669 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:49:20.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.480+0000 7f767a091740 10 client.25669 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T08:49:20.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.480+0000 7f767a091740 20 client.25669 populate_metadata read hostname 'smithi164' 2024-09-19T08:49:20.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.480+0000 7f767a091740 1 --2- 172.21.15.164:0/950516591 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55c3d344f550 0x55c3d34cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:20.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.480+0000 7f767a091740 4 client.25669 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:49:20.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.480+0000 7f767a091740 1 -- 172.21.15.164:0/950516591 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c3d33b1d70 con 0x55c3d344f550 2024-09-19T08:49:20.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.481+0000 7f7675253640 1 --2- 172.21.15.164:0/950516591 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55c3d344f550 0x55c3d34cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:20.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.482+0000 7f76597fa640 10 client.25669 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:49:20.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.483+0000 7f7675253640 1 --2- 172.21.15.164:0/950516591 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55c3d344f550 0x55c3d34cc4c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7fdd6a6b437790fa server_cookie=29c2341d72ced729 in_seq=0 out_seq=0 2024-09-19T08:49:20.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.483+0000 7f76597fa640 1 -- 172.21.15.164:0/950516591 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c3d33b1d70 con 0x55c3d344f550 2024-09-19T08:49:20.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.483+0000 7f76597fa640 10 client.25669 handle_command_reply: tid=0 2024-09-19T08:49:20.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.483+0000 7f767a091740 1 client.25669 shutdown 2024-09-19T08:49:20.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.486+0000 7f767a091740 1 -- 172.21.15.164:0/950516591 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3d340d310 msgr2=0x55c3d3457eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:20.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.486+0000 7f767a091740 1 --2- 172.21.15.164:0/950516591 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3d340d310 0x55c3d3457eb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f766c009850 tx=0x7f766c045b30 comp rx=0 tx=0).stop 2024-09-19T08:49:20.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.490+0000 7f767a091740 1 -- 172.21.15.164:0/950516591 shutdown_connections 2024-09-19T08:49:20.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.490+0000 7f767a091740 1 --2- 172.21.15.164:0/950516591 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55c3d344f550 0x55c3d34cc4c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.490+0000 7f767a091740 1 --2- 172.21.15.164:0/950516591 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3d3445d20 0x55c3d3450c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.490+0000 7f767a091740 1 --2- 172.21.15.164:0/950516591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3d34447b0 0x55c3d3450700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.490+0000 7f767a091740 1 --2- 172.21.15.164:0/950516591 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3d340d310 0x55c3d3457eb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.490+0000 7f76597fa640 0 client.25669 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:49:20.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.490+0000 7f767a091740 1 -- 172.21.15.164:0/950516591 >> 172.21.15.164:0/950516591 conn(0x55c3d344c790 msgr2=0x55c3d344d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:20.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.491+0000 7f767a091740 1 -- 172.21.15.164:0/950516591 shutdown_connections 2024-09-19T08:49:20.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.491+0000 7f767a091740 1 -- 172.21.15.164:0/950516591 wait complete. 2024-09-19T08:49:20.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.491+0000 7f767a091740 20 client.25669 trim_cache size 0 max 16384 2024-09-19T08:49:20.494 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:49:20.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.493+0000 7f7676a56640 1 -- 172.21.15.164:0/1003715367 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f765c0a3500 msgr2=0x7f765c0a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:20.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.493+0000 7f7676a56640 1 --2- 172.21.15.164:0/1003715367 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f765c0a3500 0x7f765c0a59c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f76680123f0 tx=0x7f7668009550 comp rx=0 tx=0).stop 2024-09-19T08:49:20.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.493+0000 7f7676a56640 1 -- 172.21.15.164:0/1003715367 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7670110190 msgr2=0x7f7670111780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:20.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.493+0000 7f7676a56640 1 --2- 172.21.15.164:0/1003715367 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7670110190 0x7f7670111780 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f766000e9f0 tx=0x7f766000eeb0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.494+0000 7f7676a56640 1 -- 172.21.15.164:0/1003715367 shutdown_connections 2024-09-19T08:49:20.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.494+0000 7f7676a56640 1 --2- 172.21.15.164:0/1003715367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f767011b140 0x7f7670111cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.494+0000 7f7676a56640 1 --2- 172.21.15.164:0/1003715367 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7670110190 0x7f7670111780 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.494+0000 7f7676a56640 1 --2- 172.21.15.164:0/1003715367 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f765c0a3500 0x7f765c0a59c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.494+0000 7f7676a56640 1 --2- 172.21.15.164:0/1003715367 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f767006ea30 0x7f76701186f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:20.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.494+0000 7f7676a56640 1 -- 172.21.15.164:0/1003715367 >> 172.21.15.164:0/1003715367 conn(0x7f7670073180 msgr2=0x7f767010e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:20.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.494+0000 7f7676a56640 1 -- 172.21.15.164:0/1003715367 shutdown_connections 2024-09-19T08:49:20.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:20.494+0000 7f7676a56640 1 -- 172.21.15.164:0/1003715367 wait complete. 2024-09-19T08:49:21.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:21 smithi173 ceph-mon[35306]: pgmap v109: 97 pgs: 97 active+clean; 26 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 844 MiB/s wr, 3.59k op/s 2024-09-19T08:49:21.666 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:21 smithi164 ceph-mon[30928]: pgmap v109: 97 pgs: 97 active+clean; 26 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 844 MiB/s wr, 3.59k op/s 2024-09-19T08:49:21.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:21 smithi170 ceph-mon[35072]: pgmap v109: 97 pgs: 97 active+clean; 26 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 844 MiB/s wr, 3.59k op/s 2024-09-19T08:49:22.817 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T08:49:23.782 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:49:23.793 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:23 smithi164 ceph-mon[30928]: pgmap v110: 97 pgs: 97 active+clean; 27 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 687 MiB/s wr, 2.33k op/s 2024-09-19T08:49:23.876 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:23 smithi170 ceph-mon[35072]: pgmap v110: 97 pgs: 97 active+clean; 27 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 687 MiB/s wr, 2.33k op/s 2024-09-19T08:49:23.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:23 smithi173 ceph-mon[35306]: pgmap v110: 97 pgs: 97 active+clean; 27 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 687 MiB/s wr, 2.33k op/s 2024-09-19T08:49:25.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:25 smithi164 ceph-mon[30928]: pgmap v111: 97 pgs: 97 active+clean; 27 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 682 MiB/s wr, 2.04k op/s 2024-09-19T08:49:25.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:25 smithi170 ceph-mon[35072]: pgmap v111: 97 pgs: 97 active+clean; 27 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 682 MiB/s wr, 2.04k op/s 2024-09-19T08:49:25.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:25 smithi173 ceph-mon[35306]: pgmap v111: 97 pgs: 97 active+clean; 27 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 682 MiB/s wr, 2.04k op/s 2024-09-19T08:49:25.940 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 16.6 2024-09-19T08:49:25.941 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:49:26.532 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:49:26.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.072+0000 7f1be77fe640 1 -- 172.21.15.164:0/3612200180 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc0035f0 con 0x7f1be0095a50 2024-09-19T08:49:26.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.072+0000 7f1bedd5f640 1 -- 172.21.15.164:0/3612200180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1be0095a50 msgr2=0x7f1be0095ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:26.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.072+0000 7f1bedd5f640 1 --2- 172.21.15.164:0/3612200180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1be0095a50 0x7f1be0095ed0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc009900 tx=0x7f1bdc02a440 comp rx=0 tx=0).stop 2024-09-19T08:49:26.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.073+0000 7f1bedd5f640 1 -- 172.21.15.164:0/3612200180 shutdown_connections 2024-09-19T08:49:26.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.073+0000 7f1bedd5f640 1 --2- 172.21.15.164:0/3612200180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1be0096410 0x7f1be00a2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:26.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.073+0000 7f1bedd5f640 1 --2- 172.21.15.164:0/3612200180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1be0095a50 0x7f1be0095ed0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:26.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.073+0000 7f1bedd5f640 1 --2- 172.21.15.164:0/3612200180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1be009b650 0x7f1be009ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:26.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.073+0000 7f1bedd5f640 1 -- 172.21.15.164:0/3612200180 >> 172.21.15.164:0/3612200180 conn(0x7f1be000b320 msgr2=0x7f1be000b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:26.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.073+0000 7f1bedd5f640 1 -- 172.21.15.164:0/3612200180 shutdown_connections 2024-09-19T08:49:26.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.073+0000 7f1bedd5f640 1 -- 172.21.15.164:0/3612200180 wait complete. 2024-09-19T08:49:26.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.074+0000 7f1bedd5f640 1 Processor -- start 2024-09-19T08:49:26.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.074+0000 7f1bedd5f640 1 -- start start 2024-09-19T08:49:26.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.074+0000 7f1bedd5f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1be0095a50 0x7f1be00a9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:26.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.074+0000 7f1bedd5f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1be0096410 0x7f1be00aa060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:26.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.074+0000 7f1bedd5f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1be009b650 0x7f1be00a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:26.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.074+0000 7f1bedd5f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1be0008d80 con 0x7f1be0095a50 2024-09-19T08:49:26.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.074+0000 7f1bedd5f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1be0008c00 con 0x7f1be009b650 2024-09-19T08:49:26.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.074+0000 7f1bedd5f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1be0008f00 con 0x7f1be0096410 2024-09-19T08:49:26.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.075+0000 7f1be7fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1be0096410 0x7f1be00aa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:26.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.075+0000 7f1be7fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1be0096410 0x7f1be00aa060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:34532/0 (socket says 172.21.15.164:34532) 2024-09-19T08:49:26.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.075+0000 7f1be7fff640 1 -- 172.21.15.164:0/2809391984 learned_addr learned my addr 172.21.15.164:0/2809391984 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:26.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.075+0000 7f1becd5d640 1 --2- 172.21.15.164:0/2809391984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1be0095a50 0x7f1be00a9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:26.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.075+0000 7f1bed55e640 1 --2- 172.21.15.164:0/2809391984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1be009b650 0x7f1be00a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:26.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.076+0000 7f1be7fff640 1 -- 172.21.15.164:0/2809391984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1be009b650 msgr2=0x7f1be00a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:26.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.076+0000 7f1be7fff640 1 --2- 172.21.15.164:0/2809391984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1be009b650 0x7f1be00a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:26.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.076+0000 7f1be7fff640 1 -- 172.21.15.164:0/2809391984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1be0095a50 msgr2=0x7f1be00a9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:26.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.076+0000 7f1be7fff640 1 --2- 172.21.15.164:0/2809391984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1be0095a50 0x7f1be00a9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:26.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.076+0000 7f1be7fff640 1 -- 172.21.15.164:0/2809391984 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be00a5500 con 0x7f1be0096410 2024-09-19T08:49:26.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.078+0000 7f1be7fff640 1 --2- 172.21.15.164:0/2809391984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1be0096410 0x7f1be00aa060 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc005940 tx=0x7f1bdc003f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:26.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.078+0000 7f1be5ffb640 1 -- 172.21.15.164:0/2809391984 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc040030 con 0x7f1be0096410 2024-09-19T08:49:26.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.078+0000 7f1be5ffb640 1 -- 172.21.15.164:0/2809391984 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bdc003660 con 0x7f1be0096410 2024-09-19T08:49:26.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.078+0000 7f1be5ffb640 1 -- 172.21.15.164:0/2809391984 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc0318d0 con 0x7f1be0096410 2024-09-19T08:49:26.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.079+0000 7f1bedd5f640 1 -- 172.21.15.164:0/2809391984 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1be00a5790 con 0x7f1be0096410 2024-09-19T08:49:26.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.079+0000 7f1bedd5f640 1 -- 172.21.15.164:0/2809391984 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1be00af840 con 0x7f1be0096410 2024-09-19T08:49:26.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.084+0000 7f1bf139a740 1 Processor -- start 2024-09-19T08:49:26.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.084+0000 7f1bf139a740 1 -- start start 2024-09-19T08:49:26.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.085+0000 7f1bf139a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605b80e2cf0 0x5605b80e1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:26.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.085+0000 7f1bf139a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605b80e1780 0x5605b80e1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:26.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.085+0000 7f1bf139a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605b80ec520 0x5605b80ee910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:26.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.085+0000 7f1bf139a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5605b80472f0 con 0x5605b80ec520 2024-09-19T08:49:26.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.085+0000 7f1bf139a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5605b7feecf0 con 0x5605b80e2cf0 2024-09-19T08:49:26.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.085+0000 7f1bf139a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5605b80432b0 con 0x5605b80e1780 2024-09-19T08:49:26.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.085+0000 7f1bed55e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605b80e2cf0 0x5605b80e1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:26.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.085+0000 7f1bed55e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605b80e2cf0 0x5605b80e1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:54616/0 (socket says 172.21.15.164:54616) 2024-09-19T08:49:26.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.085+0000 7f1bed55e640 1 -- 172.21.15.164:0/3996497427 learned_addr learned my addr 172.21.15.164:0/3996497427 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:26.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.085+0000 7f1becd5d640 1 --2- 172.21.15.164:0/3996497427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605b80e1780 0x5605b80e1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:26.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.086+0000 7f1becd5d640 1 -- 172.21.15.164:0/3996497427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605b80e2cf0 msgr2=0x5605b80e1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:26.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.086+0000 7f1becd5d640 1 --2- 172.21.15.164:0/3996497427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605b80e2cf0 0x5605b80e1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:26.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.086+0000 7f1becd5d640 1 -- 172.21.15.164:0/3996497427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605b80ec520 msgr2=0x5605b80ee910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:49:26.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.086+0000 7f1becd5d640 1 --2- 172.21.15.164:0/3996497427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605b80ec520 0x5605b80ee910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:26.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.086+0000 7f1becd5d640 1 -- 172.21.15.164:0/3996497427 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605b7fd33a0 con 0x5605b80e1780 2024-09-19T08:49:26.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.087+0000 7f1becd5d640 1 --2- 172.21.15.164:0/3996497427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605b80e1780 0x5605b80e1bc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1be8050280 tx=0x7f1be8068f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43ff011aa5482d81 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:26.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.088+0000 7f1bceffd640 1 -- 172.21.15.164:0/3996497427 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be8070a30 con 0x5605b80e1780 2024-09-19T08:49:26.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.088+0000 7f1bceffd640 1 -- 172.21.15.164:0/3996497427 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1be8070bd0 con 0x5605b80e1780 2024-09-19T08:49:26.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.088+0000 7f1bceffd640 1 -- 172.21.15.164:0/3996497427 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be806d620 con 0x5605b80e1780 2024-09-19T08:49:26.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.090+0000 7f1bf139a740 1 -- 172.21.15.164:0/3996497427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605b80e1780 msgr2=0x5605b80e1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:26.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.090+0000 7f1bf139a740 1 --2- 172.21.15.164:0/3996497427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605b80e1780 0x5605b80e1bc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1be8050280 tx=0x7f1be8068f60 comp rx=0 tx=0).stop 2024-09-19T08:49:26.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.090+0000 7f1be5ffb640 1 -- 172.21.15.164:0/2809391984 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f1bdc043030 con 0x7f1be0096410 2024-09-19T08:49:26.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.091+0000 7f1bf139a740 1 -- 172.21.15.164:0/3996497427 shutdown_connections 2024-09-19T08:49:26.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.091+0000 7f1bf139a740 1 --2- 172.21.15.164:0/3996497427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605b80ec520 0x5605b80ee910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:26.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.091+0000 7f1bf139a740 1 --2- 172.21.15.164:0/3996497427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605b80e1780 0x5605b80e1bc0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:26.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.091+0000 7f1bf139a740 1 --2- 172.21.15.164:0/3996497427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605b80e2cf0 0x5605b80e1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:26.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.091+0000 7f1bf139a740 1 -- 172.21.15.164:0/3996497427 >> 172.21.15.164:0/3996497427 conn(0x5605b80aa310 msgr2=0x5605b80dde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:26.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.092+0000 7f1bf139a740 1 -- 172.21.15.164:0/3996497427 shutdown_connections 2024-09-19T08:49:26.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.092+0000 7f1bf139a740 1 -- 172.21.15.164:0/3996497427 wait complete. 2024-09-19T08:49:26.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.092+0000 7f1be5ffb640 1 --2- 172.21.15.164:0/2809391984 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f1bd00a33d0 0x7f1bd00a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:26.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.092+0000 7f1bf139a740 1 -- start start 2024-09-19T08:49:26.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.093+0000 7f1bf139a740 1 Processor -- start 2024-09-19T08:49:26.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.093+0000 7f1be5ffb640 1 -- 172.21.15.164:0/2809391984 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f1bdc0f0620 con 0x7f1be0096410 2024-09-19T08:49:26.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.094+0000 7f1bf139a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605b80e1780 0x5605b80ebed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:26.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.094+0000 7f1bf139a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605b80e2cf0 0x5605b80e9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:26.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.094+0000 7f1becd5d640 1 --2- 172.21.15.164:0/2809391984 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f1bd00a33d0 0x7f1bd00a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:26.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.094+0000 7f1be7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605b80e2cf0 0x5605b80e9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:26.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.094+0000 7f1be7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605b80e2cf0 0x5605b80e9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:54626/0 (socket says 172.21.15.164:54626) 2024-09-19T08:49:26.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.094+0000 7f1be7fff640 1 -- 172.21.15.164:0/2252476498 learned_addr learned my addr 172.21.15.164:0/2252476498 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:26.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.094+0000 7f1bf139a740 1 --2- 172.21.15.164:0/2252476498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605b80ec520 0x5605b80e9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:26.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.094+0000 7f1bf139a740 1 -- 172.21.15.164:0/2252476498 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5605b80ef220 con 0x5605b80ec520 2024-09-19T08:49:26.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.095+0000 7f1bf139a740 1 -- 172.21.15.164:0/2252476498 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5605b8002060 con 0x5605b80e2cf0 2024-09-19T08:49:26.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.095+0000 7f1bf139a740 1 -- 172.21.15.164:0/2252476498 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5605b80ef3a0 con 0x5605b80e1780 2024-09-19T08:49:26.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.095+0000 7f1becd5d640 1 --2- 172.21.15.164:0/2252476498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605b80e1780 0x5605b80ebed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:26.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.095+0000 7f1bf139a740 10 client.0 _ll_register_callbacks cb 0x5605b7d558c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:49:26.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.096+0000 7f1be7fff640 1 -- 172.21.15.164:0/2252476498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605b80e1780 msgr2=0x5605b80ebed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:26.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.096+0000 7f1be7fff640 1 --2- 172.21.15.164:0/2252476498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605b80e1780 0x5605b80ebed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:26.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.096+0000 7f1be7fff640 1 -- 172.21.15.164:0/2252476498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605b80ec520 msgr2=0x5605b80e9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:26.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.096+0000 7f1be7fff640 1 --2- 172.21.15.164:0/2252476498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605b80ec520 0x5605b80e9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:26.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.096+0000 7f1be7fff640 1 -- 172.21.15.164:0/2252476498 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605b7db83c0 con 0x5605b80e2cf0 2024-09-19T08:49:26.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.096+0000 7f1becd5d640 1 --2- 172.21.15.164:0/2809391984 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f1bd00a33d0 0x7f1bd00a5890 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1be8067bf0 tx=0x7f1be806b390 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:26.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.097+0000 7f1be7fff640 1 --2- 172.21.15.164:0/2252476498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605b80e2cf0 0x5605b80e9180 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc0bc610 tx=0x7f1bdc0bd150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39f34b2e3fcd45af server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:26.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.099+0000 7f1bc7fff640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-19T08:49:26.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.100+0000 7f1bc7fff640 1 -- 172.21.15.164:0/2252476498 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc0c5580 con 0x5605b80e2cf0 2024-09-19T08:49:26.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.100+0000 7f1bc7fff640 1 -- 172.21.15.164:0/2252476498 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f1bdc0bac10 con 0x5605b80e2cf0 2024-09-19T08:49:26.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.105+0000 7f1bf139a740 1 -- 172.21.15.164:0/2252476498 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1be8070bd0 con 0x5605b80e2cf0 2024-09-19T08:49:26.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.105+0000 7f1bc7fff640 1 -- 172.21.15.164:0/2252476498 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1bdc03cbb0 con 0x5605b80e2cf0 2024-09-19T08:49:26.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.105+0000 7f1bf139a740 10 client.25679 fetch_fsmap learned FSMap version 28 2024-09-19T08:49:26.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.105+0000 7f1bf139a740 1 -- 172.21.15.164:0/2252476498 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5605b80ef220 con 0x5605b80e2cf0 2024-09-19T08:49:26.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.106+0000 7f1bc7fff640 1 -- 172.21.15.164:0/2252476498 <== mon.2 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1be8070bd0 con 0x5605b80e2cf0 2024-09-19T08:49:26.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.108+0000 7f1bc7fff640 1 -- 172.21.15.164:0/2252476498 <== mon.2 v2:172.21.15.170:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1bdc04d060 con 0x5605b80e2cf0 2024-09-19T08:49:26.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.108+0000 7f1bf139a740 10 client.25679 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:49:26.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.108+0000 7f1bf139a740 10 client.25679 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T08:49:26.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.108+0000 7f1bf139a740 20 client.25679 populate_metadata read hostname 'smithi164' 2024-09-19T08:49:26.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.108+0000 7f1bf139a740 1 --2- 172.21.15.164:0/2252476498 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x5605b80fc030 0x5605b80fe420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:26.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.108+0000 7f1bf139a740 4 client.25679 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:49:26.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.108+0000 7f1bf139a740 1 -- 172.21.15.164:0/2252476498 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605b8054860 con 0x5605b80fc030 2024-09-19T08:49:26.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.110+0000 7f1becd5d640 1 --2- 172.21.15.164:0/2252476498 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x5605b80fc030 0x5605b80fe420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:26.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.115+0000 7f1becd5d640 1 --2- 172.21.15.164:0/2252476498 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x5605b80fc030 0x5605b80fe420 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2db49b0c218ca7a6 server_cookie=9aacda6a29be0c7d in_seq=0 out_seq=0 2024-09-19T08:49:26.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.115+0000 7f1bc7fff640 10 client.25679 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:49:26.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.118+0000 7f1bc7fff640 1 -- 172.21.15.164:0/2252476498 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5605b8054860 con 0x5605b80fc030 2024-09-19T08:49:26.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.118+0000 7f1bc7fff640 10 client.25679 handle_command_reply: tid=0 2024-09-19T08:49:26.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.118+0000 7f1bf139a740 1 client.25679 shutdown 2024-09-19T08:49:26.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.118+0000 7f1bf139a740 1 -- 172.21.15.164:0/2252476498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605b80e2cf0 msgr2=0x5605b80e9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:26.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.118+0000 7f1bf139a740 1 --2- 172.21.15.164:0/2252476498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605b80e2cf0 0x5605b80e9180 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc0bc610 tx=0x7f1bdc0bd150 comp rx=0 tx=0).stop 2024-09-19T08:49:26.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.119+0000 7f1bf139a740 1 -- 172.21.15.164:0/2252476498 shutdown_connections 2024-09-19T08:49:26.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.119+0000 7f1bf139a740 1 --2- 172.21.15.164:0/2252476498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605b80ec520 0x5605b80e9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:26.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.119+0000 7f1bf139a740 1 --2- 172.21.15.164:0/2252476498 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x5605b80fc030 0x5605b80fe420 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:26.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.119+0000 7f1bf139a740 1 --2- 172.21.15.164:0/2252476498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605b80e2cf0 0x5605b80e9180 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:26.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.119+0000 7f1bf139a740 1 --2- 172.21.15.164:0/2252476498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605b80e1780 0x5605b80ebed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:26.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.119+0000 7f1bf139a740 1 -- 172.21.15.164:0/2252476498 >> 172.21.15.164:0/2252476498 conn(0x5605b80aa310 msgr2=0x5605b80e3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:26.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.119+0000 7f1bc7fff640 0 client.25679 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:49:26.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.120+0000 7f1bf139a740 1 -- 172.21.15.164:0/2252476498 shutdown_connections 2024-09-19T08:49:26.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.120+0000 7f1bf139a740 1 -- 172.21.15.164:0/2252476498 wait complete. 2024-09-19T08:49:26.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.120+0000 7f1bf139a740 20 client.25679 trim_cache size 0 max 16384 2024-09-19T08:49:26.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.154+0000 7f1bf139a740 1 Processor -- start 2024-09-19T08:49:26.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.154+0000 7f1bf139a740 1 -- start start 2024-09-19T08:49:26.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.154+0000 7f1bf139a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605b80e1780 0x5605b80f8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:26.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.155+0000 7f1bf139a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605b80e2cf0 0x5605b80ed910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:26.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.155+0000 7f1bf139a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605b80ec520 0x5605b80ede50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:26.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.155+0000 7f1bf139a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1be0160df0 con 0x5605b80ec520 2024-09-19T08:49:26.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.155+0000 7f1bf139a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1be0162930 con 0x5605b80e2cf0 2024-09-19T08:49:26.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.155+0000 7f1bf139a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1be01614c0 con 0x5605b80e1780 2024-09-19T08:49:26.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.155+0000 7f1bed55e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605b80ec520 0x5605b80ede50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:26.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.155+0000 7f1becd5d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605b80e1780 0x5605b80f8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:26.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.156+0000 7f1be7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605b80e2cf0 0x5605b80ed910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:26.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.156+0000 7f1becd5d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605b80e1780 0x5605b80f8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:34572/0 (socket says 172.21.15.164:34572) 2024-09-19T08:49:26.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.156+0000 7f1becd5d640 1 -- 172.21.15.164:0/3739228920 learned_addr learned my addr 172.21.15.164:0/3739228920 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:26.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.158+0000 7f1becd5d640 1 -- 172.21.15.164:0/3739228920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605b80e2cf0 msgr2=0x5605b80ed910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:26.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.158+0000 7f1becd5d640 1 --2- 172.21.15.164:0/3739228920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605b80e2cf0 0x5605b80ed910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:26.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.158+0000 7f1becd5d640 1 -- 172.21.15.164:0/3739228920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605b80ec520 msgr2=0x5605b80ede50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:26.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.158+0000 7f1becd5d640 1 --2- 172.21.15.164:0/3739228920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605b80ec520 0x5605b80ede50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:26.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.158+0000 7f1becd5d640 1 -- 172.21.15.164:0/3739228920 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605b80e2550 con 0x5605b80e1780 2024-09-19T08:49:26.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.159+0000 7f1becd5d640 1 --2- 172.21.15.164:0/3739228920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605b80e1780 0x5605b80f8520 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1be8051220 tx=0x7f1be808e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=197228a845ce55f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:26.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.159+0000 7f1bce7fc640 1 -- 172.21.15.164:0/3739228920 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be8070b50 con 0x5605b80e1780 2024-09-19T08:49:26.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.159+0000 7f1bce7fc640 1 -- 172.21.15.164:0/3739228920 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1be8089040 con 0x5605b80e1780 2024-09-19T08:49:26.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.160+0000 7f1bf139a740 1 -- 172.21.15.164:0/3739228920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605b80e1780 msgr2=0x5605b80f8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:26.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.160+0000 7f1bf139a740 1 --2- 172.21.15.164:0/3739228920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605b80e1780 0x5605b80f8520 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1be8051220 tx=0x7f1be808e000 comp rx=0 tx=0).stop 2024-09-19T08:49:26.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.165+0000 7f1bf139a740 1 -- 172.21.15.164:0/3739228920 shutdown_connections 2024-09-19T08:49:26.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.165+0000 7f1bf139a740 1 --2- 172.21.15.164:0/3739228920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605b80ec520 0x5605b80ede50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:26.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.165+0000 7f1bf139a740 1 --2- 172.21.15.164:0/3739228920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605b80e2cf0 0x5605b80ed910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:26.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.165+0000 7f1bf139a740 1 --2- 172.21.15.164:0/3739228920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605b80e1780 0x5605b80f8520 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:26.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.165+0000 7f1bf139a740 1 -- 172.21.15.164:0/3739228920 >> 172.21.15.164:0/3739228920 conn(0x5605b80aa310 msgr2=0x5605b80dd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:26.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.166+0000 7f1bf139a740 1 -- 172.21.15.164:0/3739228920 shutdown_connections 2024-09-19T08:49:26.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.166+0000 7f1bf139a740 1 -- 172.21.15.164:0/3739228920 wait complete. 2024-09-19T08:49:26.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.167+0000 7f1bf139a740 1 -- start start 2024-09-19T08:49:26.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.167+0000 7f1bf139a740 1 Processor -- start 2024-09-19T08:49:26.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.168+0000 7f1bf139a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605b80e1780 0x5605b80f4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:26.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.168+0000 7f1bf139a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605b80e2cf0 0x5605b8103400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:26.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.168+0000 7f1bf139a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605b80ec520 0x5605b8103940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:26.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.168+0000 7f1bf139a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1be0162260 con 0x5605b80ec520 2024-09-19T08:49:26.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.168+0000 7f1bf139a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1be0161b90 con 0x5605b80e2cf0 2024-09-19T08:49:26.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.168+0000 7f1bf139a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1be015eea0 con 0x5605b80e1780 2024-09-19T08:49:26.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.168+0000 7f1bf139a740 10 client.0 _ll_register_callbacks cb 0x7f1bdc03cf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:49:26.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.169+0000 7f1becd5d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605b80e1780 0x5605b80f4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:26.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.169+0000 7f1becd5d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605b80e1780 0x5605b80f4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:34576/0 (socket says 172.21.15.164:34576) 2024-09-19T08:49:26.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.169+0000 7f1becd5d640 1 -- 172.21.15.164:0/1220519903 learned_addr learned my addr 172.21.15.164:0/1220519903 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:26.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.169+0000 7f1be7fff640 1 --2- 172.21.15.164:0/1220519903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605b80e2cf0 0x5605b8103400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:26.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.169+0000 7f1becd5d640 1 -- 172.21.15.164:0/1220519903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605b80e2cf0 msgr2=0x5605b8103400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:26.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.169+0000 7f1becd5d640 1 --2- 172.21.15.164:0/1220519903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605b80e2cf0 0x5605b8103400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:26.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.169+0000 7f1becd5d640 1 -- 172.21.15.164:0/1220519903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605b80ec520 msgr2=0x5605b8103940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:26.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.169+0000 7f1becd5d640 1 --2- 172.21.15.164:0/1220519903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605b80ec520 0x5605b8103940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:26.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.169+0000 7f1becd5d640 1 -- 172.21.15.164:0/1220519903 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605b80ee390 con 0x5605b80e1780 2024-09-19T08:49:26.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.170+0000 7f1becd5d640 1 --2- 172.21.15.164:0/1220519903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605b80e1780 0x5605b80f4c10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1be80679f0 tx=0x7f1be80681c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5803c4655ef3a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:26.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.170+0000 7f1bc7fff640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-09-19T08:49:26.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.170+0000 7f1bc7fff640 1 -- 172.21.15.164:0/1220519903 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be8070b50 con 0x5605b80e1780 2024-09-19T08:49:26.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.170+0000 7f1bf139a740 1 -- 172.21.15.164:0/1220519903 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1be8089040 con 0x5605b80e1780 2024-09-19T08:49:26.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.170+0000 7f1bc7fff640 1 -- 172.21.15.164:0/1220519903 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f1be8062630 con 0x5605b80e1780 2024-09-19T08:49:26.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.170+0000 7f1bc7fff640 1 -- 172.21.15.164:0/1220519903 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1be8062830 con 0x5605b80e1780 2024-09-19T08:49:26.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.174+0000 7f1bf139a740 10 client.25684 fetch_fsmap learned FSMap version 28 2024-09-19T08:49:26.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.174+0000 7f1bf139a740 1 -- 172.21.15.164:0/1220519903 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5605b80ec330 con 0x5605b80e1780 2024-09-19T08:49:26.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.175+0000 7f1bc7fff640 1 -- 172.21.15.164:0/1220519903 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1be8089040 con 0x5605b80e1780 2024-09-19T08:49:26.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.175+0000 7f1bc7fff640 1 -- 172.21.15.164:0/1220519903 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1be80b5050 con 0x5605b80e1780 2024-09-19T08:49:26.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.175+0000 7f1bf139a740 10 client.25684 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:49:26.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.175+0000 7f1bf139a740 10 client.25684 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T08:49:26.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.175+0000 7f1bf139a740 20 client.25684 populate_metadata read hostname 'smithi164' 2024-09-19T08:49:26.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.175+0000 7f1bf139a740 1 --2- 172.21.15.164:0/1220519903 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x5605b8166cc0 0x5605b81690b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:26.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.175+0000 7f1bf139a740 4 client.25684 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:49:26.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.175+0000 7f1bf139a740 1 -- 172.21.15.164:0/1220519903 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5605b804ed70 con 0x5605b8166cc0 2024-09-19T08:49:26.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.176+0000 7f1be7fff640 1 --2- 172.21.15.164:0/1220519903 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x5605b8166cc0 0x5605b81690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:26.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.180+0000 7f1be7fff640 1 --2- 172.21.15.164:0/1220519903 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x5605b8166cc0 0x5605b81690b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eaddece8746537c2 server_cookie=58c8a70d577d9412 in_seq=0 out_seq=0 2024-09-19T08:49:26.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.181+0000 7f1bc7fff640 10 client.25684 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:49:26.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.182+0000 7f1bc7fff640 1 -- 172.21.15.164:0/1220519903 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5605b804ed70 con 0x5605b8166cc0 2024-09-19T08:49:26.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.182+0000 7f1bc7fff640 10 client.25684 handle_command_reply: tid=0 2024-09-19T08:49:26.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.182+0000 7f1bf139a740 1 client.25684 shutdown 2024-09-19T08:49:26.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.186+0000 7f1bf139a740 1 -- 172.21.15.164:0/1220519903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605b80e1780 msgr2=0x5605b80f4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:26.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.186+0000 7f1bf139a740 1 --2- 172.21.15.164:0/1220519903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605b80e1780 0x5605b80f4c10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1be80679f0 tx=0x7f1be80681c0 comp rx=0 tx=0).stop 2024-09-19T08:49:26.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.186+0000 7f1bf139a740 1 -- 172.21.15.164:0/1220519903 shutdown_connections 2024-09-19T08:49:26.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.186+0000 7f1bf139a740 1 --2- 172.21.15.164:0/1220519903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605b80ec520 0x5605b8103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:26.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.186+0000 7f1bf139a740 1 --2- 172.21.15.164:0/1220519903 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x5605b8166cc0 0x5605b81690b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:26.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.186+0000 7f1bf139a740 1 --2- 172.21.15.164:0/1220519903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605b80e2cf0 0x5605b8103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:26.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.186+0000 7f1bf139a740 1 --2- 172.21.15.164:0/1220519903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605b80e1780 0x5605b80f4c10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:26.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.186+0000 7f1bf139a740 1 -- 172.21.15.164:0/1220519903 >> 172.21.15.164:0/1220519903 conn(0x5605b80aa310 msgr2=0x5605b80e8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:26.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.186+0000 7f1bc7fff640 0 client.25684 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:49:26.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.187+0000 7f1bf139a740 1 -- 172.21.15.164:0/1220519903 shutdown_connections 2024-09-19T08:49:26.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.191+0000 7f1bf139a740 1 -- 172.21.15.164:0/1220519903 wait complete. 2024-09-19T08:49:26.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.191+0000 7f1bf139a740 20 client.25684 trim_cache size 0 max 16384 2024-09-19T08:49:26.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.194+0000 7f1bedd5f640 1 -- 172.21.15.164:0/2809391984 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f1bd00a33d0 msgr2=0x7f1bd00a5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:26.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.194+0000 7f1bedd5f640 1 --2- 172.21.15.164:0/2809391984 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f1bd00a33d0 0x7f1bd00a5890 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1be8067bf0 tx=0x7f1be806b390 comp rx=0 tx=0).stop 2024-09-19T08:49:26.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.195+0000 7f1bedd5f640 1 -- 172.21.15.164:0/2809391984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1be0096410 msgr2=0x7f1be00aa060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:26.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.195+0000 7f1bedd5f640 1 --2- 172.21.15.164:0/2809391984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1be0096410 0x7f1be00aa060 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc005940 tx=0x7f1bdc003f60 comp rx=0 tx=0).stop 2024-09-19T08:49:26.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.195+0000 7f1bedd5f640 1 -- 172.21.15.164:0/2809391984 shutdown_connections 2024-09-19T08:49:26.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.195+0000 7f1bedd5f640 1 --2- 172.21.15.164:0/2809391984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1be009b650 0x7f1be00a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:26.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.195+0000 7f1bedd5f640 1 --2- 172.21.15.164:0/2809391984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1be0096410 0x7f1be00aa060 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc005940 tx=0x7f1bdc003f60 comp rx=0 tx=0).stop 2024-09-19T08:49:26.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.195+0000 7f1bedd5f640 1 --2- 172.21.15.164:0/2809391984 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f1bd00a33d0 0x7f1bd00a5890 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:26.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.195+0000 7f1bedd5f640 1 --2- 172.21.15.164:0/2809391984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1be0095a50 0x7f1be00a9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:26.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.195+0000 7f1bedd5f640 1 -- 172.21.15.164:0/2809391984 >> 172.21.15.164:0/2809391984 conn(0x7f1be000b320 msgr2=0x7f1be0094620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:26.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.195+0000 7f1bedd5f640 1 -- 172.21.15.164:0/2809391984 shutdown_connections 2024-09-19T08:49:26.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:26.196+0000 7f1bedd5f640 1 -- 172.21.15.164:0/2809391984 wait complete. 2024-09-19T08:49:27.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:27 smithi164 ceph-mon[30928]: pgmap v112: 97 pgs: 97 active+clean; 29 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 896 MiB/s wr, 2.39k op/s 2024-09-19T08:49:27.874 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:27 smithi170 ceph-mon[35072]: pgmap v112: 97 pgs: 97 active+clean; 29 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 896 MiB/s wr, 2.39k op/s 2024-09-19T08:49:27.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:27 smithi173 ceph-mon[35306]: pgmap v112: 97 pgs: 97 active+clean; 29 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 896 MiB/s wr, 2.39k op/s 2024-09-19T08:49:28.577 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T08:49:28.606 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:49:29.785 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:29 smithi164 ceph-mon[30928]: pgmap v113: 97 pgs: 97 active+clean; 29 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 693 MiB/s wr, 1.70k op/s 2024-09-19T08:49:29.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:29 smithi170 ceph-mon[35072]: pgmap v113: 97 pgs: 97 active+clean; 29 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 693 MiB/s wr, 1.70k op/s 2024-09-19T08:49:29.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:29 smithi173 ceph-mon[35306]: pgmap v113: 97 pgs: 97 active+clean; 29 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 693 MiB/s wr, 1.70k op/s 2024-09-19T08:49:30.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.186+0000 7fb0bbbfc640 1 -- 172.21.15.164:0/2796363537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0b4109e00 msgr2=0x7fb0b40742d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:30.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.186+0000 7fb0bbbfc640 1 --2- 172.21.15.164:0/2796363537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0b4109e00 0x7fb0b40742d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac00bc00 tx=0x7fb0ac02e690 comp rx=0 tx=0).stop 2024-09-19T08:49:30.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.190+0000 7fb0bbbfc640 1 -- 172.21.15.164:0/2796363537 shutdown_connections 2024-09-19T08:49:30.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.190+0000 7fb0bbbfc640 1 --2- 172.21.15.164:0/2796363537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0b4074990 0x7fb0b407b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:30.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.190+0000 7fb0bbbfc640 1 --2- 172.21.15.164:0/2796363537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0b4109e00 0x7fb0b40742d0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:30.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.190+0000 7fb0bbbfc640 1 --2- 172.21.15.164:0/2796363537 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0b4109450 0x7fb0b4109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:30.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.190+0000 7fb0bbbfc640 1 -- 172.21.15.164:0/2796363537 >> 172.21.15.164:0/2796363537 conn(0x7fb0b406e140 msgr2=0x7fb0b406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:30.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.191+0000 7fb0bbbfc640 1 -- 172.21.15.164:0/2796363537 shutdown_connections 2024-09-19T08:49:30.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.191+0000 7fb0bbbfc640 1 -- 172.21.15.164:0/2796363537 wait complete. 2024-09-19T08:49:30.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.192+0000 7fb0bbbfc640 1 Processor -- start 2024-09-19T08:49:30.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.193+0000 7fb0bbbfc640 1 -- start start 2024-09-19T08:49:30.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.193+0000 7fb0bbbfc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0b4074990 0x7fb0b407b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:30.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.193+0000 7fb0bbbfc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0b4109450 0x7fb0b40761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:30.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.193+0000 7fb0bbbfc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0b4109e00 0x7fb0b40766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:30.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.193+0000 7fb0bbbfc640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b407e780 con 0x7fb0b4109e00 2024-09-19T08:49:30.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.193+0000 7fb0bbbfc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b407e600 con 0x7fb0b4109450 2024-09-19T08:49:30.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.193+0000 7fb0bbbfc640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b407e900 con 0x7fb0b4074990 2024-09-19T08:49:30.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.197+0000 7fb0b9197640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0b4109450 0x7fb0b40761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:30.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.197+0000 7fb0ba199640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0b4109e00 0x7fb0b40766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:30.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.197+0000 7fb0ba199640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0b4109e00 0x7fb0b40766e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59752/0 (socket says 172.21.15.164:59752) 2024-09-19T08:49:30.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.197+0000 7fb0ba199640 1 -- 172.21.15.164:0/2180228248 learned_addr learned my addr 172.21.15.164:0/2180228248 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:30.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.198+0000 7fb0b9998640 1 --2- 172.21.15.164:0/2180228248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0b4074990 0x7fb0b407b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:30.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.199+0000 7fb0ba199640 1 -- 172.21.15.164:0/2180228248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0b4074990 msgr2=0x7fb0b407b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:30.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.199+0000 7fb0ba199640 1 --2- 172.21.15.164:0/2180228248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0b4074990 0x7fb0b407b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:30.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.199+0000 7fb0ba199640 1 -- 172.21.15.164:0/2180228248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0b4109450 msgr2=0x7fb0b40761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:30.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.199+0000 7fb0ba199640 1 --2- 172.21.15.164:0/2180228248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0b4109450 0x7fb0b40761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:30.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.199+0000 7fb0ba199640 1 -- 172.21.15.164:0/2180228248 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b4076fa0 con 0x7fb0b4109e00 2024-09-19T08:49:30.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.202+0000 7fb0ba199640 1 --2- 172.21.15.164:0/2180228248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0b4109e00 0x7fb0b40766e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb0a400d750 tx=0x7fb0a400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:30.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.203+0000 7fb0aaffd640 1 -- 172.21.15.164:0/2180228248 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4017070 con 0x7fb0b4109e00 2024-09-19T08:49:30.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.204+0000 7fb0bbbfc640 1 -- 172.21.15.164:0/2180228248 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b41bbfa0 con 0x7fb0b4109e00 2024-09-19T08:49:30.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.204+0000 7fb0bbbfc640 1 -- 172.21.15.164:0/2180228248 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b41bc340 con 0x7fb0b4109e00 2024-09-19T08:49:30.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.204+0000 7fb0aaffd640 1 -- 172.21.15.164:0/2180228248 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0a4005500 con 0x7fb0b4109e00 2024-09-19T08:49:30.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.204+0000 7fb0aaffd640 1 -- 172.21.15.164:0/2180228248 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4004d60 con 0x7fb0b4109e00 2024-09-19T08:49:30.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.208+0000 7fb0bbbfc640 1 -- 172.21.15.164:0/2180228248 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb098005290 con 0x7fb0b4109e00 2024-09-19T08:49:30.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.214+0000 7fb0aaffd640 1 -- 172.21.15.164:0/2180228248 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fb0a40046c0 con 0x7fb0b4109e00 2024-09-19T08:49:30.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.216+0000 7fb0aaffd640 1 --2- 172.21.15.164:0/2180228248 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb0a00a35c0 0x7fb0a00a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:30.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.216+0000 7fb0aaffd640 1 -- 172.21.15.164:0/2180228248 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fb0a40cf800 con 0x7fb0b4109e00 2024-09-19T08:49:30.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.218+0000 7fb0b9998640 1 --2- 172.21.15.164:0/2180228248 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb0a00a35c0 0x7fb0a00a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:30.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.228+0000 7fb0b9998640 1 --2- 172.21.15.164:0/2180228248 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb0a00a35c0 0x7fb0a00a5a80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0005f30 tx=0x7fb0b000c040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:30.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.228+0000 7fb0aaffd640 1 -- 172.21.15.164:0/2180228248 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb0a409c2f0 con 0x7fb0b4109e00 2024-09-19T08:49:30.436 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:49:30.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.807+0000 7fb0bbbfc640 1 -- 172.21.15.164:0/2180228248 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb098005580 con 0x7fb0b4109e00 2024-09-19T08:49:30.813 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:49:30.813 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":28,"btime":"2024-09-19T08:45:46:903004+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:45:46.852789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:49:30.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.810+0000 7fb0aaffd640 1 -- 172.21.15.164:0/2180228248 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11984 (secure 0 0 0) 0x7fb0a409bbf0 con 0x7fb0b4109e00 2024-09-19T08:49:30.813 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 28 2024-09-19T08:49:30.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.816+0000 7fb0bbbfc640 1 -- 172.21.15.164:0/2180228248 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb0a00a35c0 msgr2=0x7fb0a00a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:30.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.816+0000 7fb0bbbfc640 1 --2- 172.21.15.164:0/2180228248 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb0a00a35c0 0x7fb0a00a5a80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0005f30 tx=0x7fb0b000c040 comp rx=0 tx=0).stop 2024-09-19T08:49:30.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.816+0000 7fb0bbbfc640 1 -- 172.21.15.164:0/2180228248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0b4109e00 msgr2=0x7fb0b40766e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:30.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.816+0000 7fb0bbbfc640 1 --2- 172.21.15.164:0/2180228248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0b4109e00 0x7fb0b40766e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb0a400d750 tx=0x7fb0a400dc10 comp rx=0 tx=0).stop 2024-09-19T08:49:30.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.817+0000 7fb0bbbfc640 1 -- 172.21.15.164:0/2180228248 shutdown_connections 2024-09-19T08:49:30.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.817+0000 7fb0bbbfc640 1 --2- 172.21.15.164:0/2180228248 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb0a00a35c0 0x7fb0a00a5a80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:30.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.817+0000 7fb0bbbfc640 1 --2- 172.21.15.164:0/2180228248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0b4109e00 0x7fb0b40766e0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:30.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.817+0000 7fb0bbbfc640 1 --2- 172.21.15.164:0/2180228248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0b4109450 0x7fb0b40761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:30.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.817+0000 7fb0bbbfc640 1 --2- 172.21.15.164:0/2180228248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0b4074990 0x7fb0b407b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:30.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.817+0000 7fb0bbbfc640 1 -- 172.21.15.164:0/2180228248 >> 172.21.15.164:0/2180228248 conn(0x7fb0b406e140 msgr2=0x7fb0b4072390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:30.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.822+0000 7fb0bbbfc640 1 -- 172.21.15.164:0/2180228248 shutdown_connections 2024-09-19T08:49:30.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:30.824+0000 7fb0bbbfc640 1 -- 172.21.15.164:0/2180228248 wait complete. 2024-09-19T08:49:31.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:31 smithi170 ceph-mon[35072]: pgmap v114: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 906 MiB/s wr, 2.21k op/s 2024-09-19T08:49:31.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:31 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2180228248' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:49:31.914 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:31 smithi173 ceph-mon[35306]: pgmap v114: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 906 MiB/s wr, 2.21k op/s 2024-09-19T08:49:31.914 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:31 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2180228248' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:49:32.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:31 smithi164 ceph-mon[30928]: pgmap v114: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 906 MiB/s wr, 2.21k op/s 2024-09-19T08:49:32.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:31 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2180228248' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:49:33.695 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0' 2024-09-19T08:49:33.696 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0 2024-09-19T08:49:33.716 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:49:33.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.603+0000 7eff497f5640 1 -- 172.21.15.164:0/3985805745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff44109450 msgr2=0x7eff44109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:33.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.603+0000 7eff497f5640 1 --2- 172.21.15.164:0/3985805745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff44109450 0x7eff44109830 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7eff2c009900 tx=0x7eff2c02a420 comp rx=0 tx=0).stop 2024-09-19T08:49:33.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.603+0000 7eff497f5640 1 -- 172.21.15.164:0/3985805745 shutdown_connections 2024-09-19T08:49:33.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.603+0000 7eff497f5640 1 --2- 172.21.15.164:0/3985805745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff4410e890 0x7eff4410ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:33.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.603+0000 7eff497f5640 1 --2- 172.21.15.164:0/3985805745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff44109d70 0x7eff4410e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:33.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.603+0000 7eff497f5640 1 --2- 172.21.15.164:0/3985805745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff44109450 0x7eff44109830 secure :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7eff2c009900 tx=0x7eff2c02a420 comp rx=0 tx=0).stop 2024-09-19T08:49:33.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.604+0000 7eff497f5640 1 -- 172.21.15.164:0/3985805745 >> 172.21.15.164:0/3985805745 conn(0x7eff4406e2d0 msgr2=0x7eff4406e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:33.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.610+0000 7eff497f5640 1 -- 172.21.15.164:0/3985805745 shutdown_connections 2024-09-19T08:49:33.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.616+0000 7eff497f5640 1 -- 172.21.15.164:0/3985805745 wait complete. 2024-09-19T08:49:33.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.617+0000 7eff497f5640 1 Processor -- start 2024-09-19T08:49:33.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.619+0000 7eff497f5640 1 -- start start 2024-09-19T08:49:33.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.619+0000 7eff497f5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff44109d70 0x7eff4411ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:33.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.619+0000 7eff497f5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff4410e890 0x7eff4411f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:33.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.619+0000 7eff497f5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff44124b90 0x7eff44119f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:33.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.619+0000 7eff497f5640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7eff441164f0 con 0x7eff44109d70 2024-09-19T08:49:33.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.619+0000 7eff497f5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7eff44118810 con 0x7eff4410e890 2024-09-19T08:49:33.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.619+0000 7eff497f5640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7eff4411a4d0 con 0x7eff44124b90 2024-09-19T08:49:33.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.620+0000 7eff437fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff44124b90 0x7eff44119f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:33.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.620+0000 7eff437fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff44124b90 0x7eff44119f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:34592/0 (socket says 172.21.15.164:34592) 2024-09-19T08:49:33.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.620+0000 7eff437fe640 1 -- 172.21.15.164:0/4012148408 learned_addr learned my addr 172.21.15.164:0/4012148408 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:33.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.623+0000 7eff42ffd640 1 --2- 172.21.15.164:0/4012148408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff44109d70 0x7eff4411ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:33.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.624+0000 7eff427fc640 1 --2- 172.21.15.164:0/4012148408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff4410e890 0x7eff4411f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:33.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.625+0000 7eff437fe640 1 -- 172.21.15.164:0/4012148408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff4410e890 msgr2=0x7eff4411f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:33.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.625+0000 7eff437fe640 1 --2- 172.21.15.164:0/4012148408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff4410e890 0x7eff4411f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:33.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.625+0000 7eff437fe640 1 -- 172.21.15.164:0/4012148408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff44109d70 msgr2=0x7eff4411ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:33.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.625+0000 7eff437fe640 1 --2- 172.21.15.164:0/4012148408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff44109d70 0x7eff4411ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:33.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.625+0000 7eff437fe640 1 -- 172.21.15.164:0/4012148408 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff4411a6d0 con 0x7eff44124b90 2024-09-19T08:49:33.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.626+0000 7eff437fe640 1 --2- 172.21.15.164:0/4012148408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff44124b90 0x7eff44119f90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7eff3c00b3f0 tx=0x7eff3c00b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:33.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.626+0000 7eff3bfff640 1 -- 172.21.15.164:0/4012148408 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c015020 con 0x7eff44124b90 2024-09-19T08:49:33.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.626+0000 7eff497f5640 1 -- 172.21.15.164:0/4012148408 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff4411a9c0 con 0x7eff44124b90 2024-09-19T08:49:33.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.626+0000 7eff497f5640 1 -- 172.21.15.164:0/4012148408 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff441bd160 con 0x7eff44124b90 2024-09-19T08:49:33.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.626+0000 7eff3bfff640 1 -- 172.21.15.164:0/4012148408 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff3c011d10 con 0x7eff44124b90 2024-09-19T08:49:33.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.626+0000 7eff3bfff640 1 -- 172.21.15.164:0/4012148408 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c00f2e0 con 0x7eff44124b90 2024-09-19T08:49:33.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.631+0000 7eff3bfff640 1 -- 172.21.15.164:0/4012148408 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7eff3c018020 con 0x7eff44124b90 2024-09-19T08:49:33.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.632+0000 7eff3bfff640 1 --2- 172.21.15.164:0/4012148408 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7eff300a3500 0x7eff300a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:33.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.633+0000 7eff3bfff640 1 -- 172.21.15.164:0/4012148408 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7eff3c0d0550 con 0x7eff44124b90 2024-09-19T08:49:33.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.634+0000 7eff42ffd640 1 --2- 172.21.15.164:0/4012148408 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7eff300a3500 0x7eff300a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:33.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.638+0000 7eff42ffd640 1 --2- 172.21.15.164:0/4012148408 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7eff300a3500 0x7eff300a59c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7eff2c02a990 tx=0x7eff2c000bc0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:33.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.645+0000 7eff4bbce740 1 Processor -- start 2024-09-19T08:49:33.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.645+0000 7eff4bbce740 1 -- start start 2024-09-19T08:49:33.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.646+0000 7eff4bbce740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557255719b40 0x557255718090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:33.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.647+0000 7eff4bbce740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5572557185d0 0x557255718a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:33.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.647+0000 7eff4bbce740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557255723370 0x557255725760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:33.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.648+0000 7eff4bbce740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55725567e2f0 con 0x557255723370 2024-09-19T08:49:33.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.648+0000 7eff4bbce740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x557255625cf0 con 0x5572557185d0 2024-09-19T08:49:33.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.648+0000 7eff4bbce740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55725567a2b0 con 0x557255719b40 2024-09-19T08:49:33.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.649+0000 7eff437fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557255723370 0x557255725760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:33.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.649+0000 7eff437fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557255723370 0x557255725760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59766/0 (socket says 172.21.15.164:59766) 2024-09-19T08:49:33.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.649+0000 7eff437fe640 1 -- 172.21.15.164:0/1491372725 learned_addr learned my addr 172.21.15.164:0/1491372725 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:33.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.650+0000 7eff437fe640 1 -- 172.21.15.164:0/1491372725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557255719b40 msgr2=0x557255718090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:49:33.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.650+0000 7eff437fe640 1 --2- 172.21.15.164:0/1491372725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557255719b40 0x557255718090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:33.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.650+0000 7eff437fe640 1 -- 172.21.15.164:0/1491372725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5572557185d0 msgr2=0x557255718a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:49:33.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.650+0000 7eff437fe640 1 --2- 172.21.15.164:0/1491372725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5572557185d0 0x557255718a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:33.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.650+0000 7eff437fe640 1 -- 172.21.15.164:0/1491372725 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55725560a3a0 con 0x557255723370 2024-09-19T08:49:33.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.650+0000 7eff437fe640 1 --2- 172.21.15.164:0/1491372725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557255723370 0x557255725760 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7eff3c00bdb0 tx=0x7eff3c09d110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ed255e9cac7ee8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:33.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.653+0000 7eff397fa640 1 -- 172.21.15.164:0/1491372725 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c0a0d00 con 0x557255723370 2024-09-19T08:49:33.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.653+0000 7eff397fa640 1 -- 172.21.15.164:0/1491372725 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff3c0a55c0 con 0x557255723370 2024-09-19T08:49:33.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.653+0000 7eff397fa640 1 -- 172.21.15.164:0/1491372725 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c09bda0 con 0x557255723370 2024-09-19T08:49:33.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.655+0000 7eff4bbce740 1 -- 172.21.15.164:0/1491372725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557255723370 msgr2=0x557255725760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:33.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.655+0000 7eff4bbce740 1 --2- 172.21.15.164:0/1491372725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557255723370 0x557255725760 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7eff3c00bdb0 tx=0x7eff3c09d110 comp rx=0 tx=0).stop 2024-09-19T08:49:33.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.656+0000 7eff4bbce740 1 -- 172.21.15.164:0/1491372725 shutdown_connections 2024-09-19T08:49:33.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.656+0000 7eff4bbce740 1 --2- 172.21.15.164:0/1491372725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557255723370 0x557255725760 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:33.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.656+0000 7eff4bbce740 1 --2- 172.21.15.164:0/1491372725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5572557185d0 0x557255718a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:33.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.656+0000 7eff4bbce740 1 --2- 172.21.15.164:0/1491372725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557255719b40 0x557255718090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:33.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.656+0000 7eff4bbce740 1 -- 172.21.15.164:0/1491372725 >> 172.21.15.164:0/1491372725 conn(0x5572556e1310 msgr2=0x557255714c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:33.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.661+0000 7eff4bbce740 1 -- 172.21.15.164:0/1491372725 shutdown_connections 2024-09-19T08:49:33.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.661+0000 7eff4bbce740 1 -- 172.21.15.164:0/1491372725 wait complete. 2024-09-19T08:49:33.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.662+0000 7eff4bbce740 1 -- start start 2024-09-19T08:49:33.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.663+0000 7eff4bbce740 1 Processor -- start 2024-09-19T08:49:33.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.664+0000 7eff4bbce740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572557185d0 0x557255722dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:33.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.665+0000 7eff4bbce740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557255719b40 0x557255720080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:33.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.665+0000 7eff4bbce740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557255723370 0x557255720810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:33.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.665+0000 7eff4bbce740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557255726070 con 0x5572557185d0 2024-09-19T08:49:33.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.665+0000 7eff4bbce740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x557255639060 con 0x557255723370 2024-09-19T08:49:33.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.665+0000 7eff4bbce740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5572557261f0 con 0x557255719b40 2024-09-19T08:49:33.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.665+0000 7eff4bbce740 10 client.0 _ll_register_callbacks cb 0x55725538c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:49:33.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.666+0000 7eff42ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572557185d0 0x557255722dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:33.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.666+0000 7eff427fc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557255719b40 0x557255720080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:33.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.666+0000 7eff42ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572557185d0 0x557255722dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59770/0 (socket says 172.21.15.164:59770) 2024-09-19T08:49:33.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.666+0000 7eff42ffd640 1 -- 172.21.15.164:0/111254798 learned_addr learned my addr 172.21.15.164:0/111254798 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:33.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.666+0000 7eff437fe640 1 --2- 172.21.15.164:0/111254798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557255723370 0x557255720810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:33.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.667+0000 7eff42ffd640 1 -- 172.21.15.164:0/111254798 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557255719b40 msgr2=0x557255720080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:33.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.667+0000 7eff42ffd640 1 --2- 172.21.15.164:0/111254798 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557255719b40 0x557255720080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:33.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.667+0000 7eff42ffd640 1 -- 172.21.15.164:0/111254798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557255723370 msgr2=0x557255720810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:33.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.667+0000 7eff42ffd640 1 --2- 172.21.15.164:0/111254798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557255723370 0x557255720810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:33.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.667+0000 7eff42ffd640 1 -- 172.21.15.164:0/111254798 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572553ef3c0 con 0x5572557185d0 2024-09-19T08:49:33.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.668+0000 7eff42ffd640 1 --2- 172.21.15.164:0/111254798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572557185d0 0x557255722dd0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7eff2c0096c0 tx=0x7eff2c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d590cf1d55db6295 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:33.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.668+0000 7eff2a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:49:33.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.669+0000 7eff2a7fc640 1 -- 172.21.15.164:0/111254798 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2c04c020 con 0x5572557185d0 2024-09-19T08:49:33.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.670+0000 7eff2a7fc640 1 -- 172.21.15.164:0/111254798 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7eff2c0037f0 con 0x5572557185d0 2024-09-19T08:49:33.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.670+0000 7eff4bbce740 1 -- 172.21.15.164:0/111254798 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff3c0a55c0 con 0x5572557185d0 2024-09-19T08:49:33.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.671+0000 7eff2a7fc640 1 -- 172.21.15.164:0/111254798 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7eff2c035050 con 0x5572557185d0 2024-09-19T08:49:33.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.671+0000 7eff4bbce740 10 client.16284 fetch_fsmap learned FSMap version 28 2024-09-19T08:49:33.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.671+0000 7eff4bbce740 1 -- 172.21.15.164:0/111254798 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557255731bf0 con 0x5572557185d0 2024-09-19T08:49:33.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.673+0000 7eff2a7fc640 1 -- 172.21.15.164:0/111254798 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff2c031e10 con 0x5572557185d0 2024-09-19T08:49:33.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.675+0000 7eff2a7fc640 1 -- 172.21.15.164:0/111254798 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7eff2c0039f0 con 0x5572557185d0 2024-09-19T08:49:33.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.676+0000 7eff4bbce740 10 client.16284 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:49:33.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.676+0000 7eff4bbce740 10 client.16284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:49:33.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.676+0000 7eff4bbce740 20 client.16284 populate_metadata read hostname 'smithi164' 2024-09-19T08:49:33.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.676+0000 7eff4bbce740 1 --2- 172.21.15.164:0/111254798 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x557255733000 0x557255735430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:33.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.676+0000 7eff4bbce740 4 client.16284 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:49:33.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.676+0000 7eff4bbce740 1 -- 172.21.15.164:0/111254798 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7eff3c0a55c0 con 0x557255733000 2024-09-19T08:49:33.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.677+0000 7eff427fc640 1 --2- 172.21.15.164:0/111254798 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x557255733000 0x557255735430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:33.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.678+0000 7eff427fc640 1 --2- 172.21.15.164:0/111254798 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x557255733000 0x557255735430 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df914d8a7ab1bf59 server_cookie=807feae79a30acc5 in_seq=0 out_seq=0 2024-09-19T08:49:33.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.678+0000 7eff2a7fc640 10 client.16284 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:49:33.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.682+0000 7eff2a7fc640 1 -- 172.21.15.164:0/111254798 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7eff3c0a55c0 con 0x557255733000 2024-09-19T08:49:33.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.682+0000 7eff2a7fc640 10 client.16284 handle_command_reply: tid=0 2024-09-19T08:49:33.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.683+0000 7eff4bbce740 1 client.16284 shutdown 2024-09-19T08:49:33.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.685+0000 7eff4bbce740 1 -- 172.21.15.164:0/111254798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572557185d0 msgr2=0x557255722dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:33.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.685+0000 7eff4bbce740 1 --2- 172.21.15.164:0/111254798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572557185d0 0x557255722dd0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7eff2c0096c0 tx=0x7eff2c009260 comp rx=0 tx=0).stop 2024-09-19T08:49:33.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.692+0000 7eff4bbce740 1 -- 172.21.15.164:0/111254798 shutdown_connections 2024-09-19T08:49:33.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.692+0000 7eff4bbce740 1 --2- 172.21.15.164:0/111254798 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x557255733000 0x557255735430 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:33.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.692+0000 7eff2a7fc640 0 client.16284 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:49:33.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.692+0000 7eff4bbce740 1 --2- 172.21.15.164:0/111254798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557255723370 0x557255720810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:33.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.692+0000 7eff4bbce740 1 --2- 172.21.15.164:0/111254798 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557255719b40 0x557255720080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:33.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.692+0000 7eff4bbce740 1 --2- 172.21.15.164:0/111254798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572557185d0 0x557255722dd0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:33.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.692+0000 7eff4bbce740 1 -- 172.21.15.164:0/111254798 >> 172.21.15.164:0/111254798 conn(0x5572556e1310 msgr2=0x55725571a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:33.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.696+0000 7eff4bbce740 1 -- 172.21.15.164:0/111254798 shutdown_connections 2024-09-19T08:49:33.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.698+0000 7eff4bbce740 1 -- 172.21.15.164:0/111254798 wait complete. 2024-09-19T08:49:33.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.699+0000 7eff4bbce740 20 client.16284 trim_cache size 0 max 16384 2024-09-19T08:49:33.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.741+0000 7eff4bbce740 1 Processor -- start 2024-09-19T08:49:33.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.750+0000 7eff4bbce740 1 -- start start 2024-09-19T08:49:33.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.751+0000 7eff4bbce740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5572557185d0 0x557255722dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:33.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.751+0000 7eff4bbce740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557255719b40 0x55725571f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:33.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.751+0000 7eff4bbce740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557255723370 0x557255713c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:33.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.751+0000 7eff4bbce740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7eff2400e690 con 0x557255723370 2024-09-19T08:49:33.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.751+0000 7eff4bbce740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7eff240101d0 con 0x5572557185d0 2024-09-19T08:49:33.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.751+0000 7eff4bbce740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7eff2400ed60 con 0x557255719b40 2024-09-19T08:49:33.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.755+0000 7eff42ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5572557185d0 0x557255722dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:33.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.755+0000 7eff427fc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557255719b40 0x55725571f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:33.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.755+0000 7eff42ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5572557185d0 0x557255722dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:54704/0 (socket says 172.21.15.164:54704) 2024-09-19T08:49:33.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.755+0000 7eff42ffd640 1 -- 172.21.15.164:0/3576986526 learned_addr learned my addr 172.21.15.164:0/3576986526 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:33.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.757+0000 7eff437fe640 1 --2- 172.21.15.164:0/3576986526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557255723370 0x557255713c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:33.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.760+0000 7eff42ffd640 1 -- 172.21.15.164:0/3576986526 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557255719b40 msgr2=0x55725571f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:33.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.760+0000 7eff42ffd640 1 --2- 172.21.15.164:0/3576986526 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557255719b40 0x55725571f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:33.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.760+0000 7eff42ffd640 1 -- 172.21.15.164:0/3576986526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557255723370 msgr2=0x557255713c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:33.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.760+0000 7eff42ffd640 1 --2- 172.21.15.164:0/3576986526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557255723370 0x557255713c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:33.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.760+0000 7eff42ffd640 1 -- 172.21.15.164:0/3576986526 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557255725ad0 con 0x5572557185d0 2024-09-19T08:49:33.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.766+0000 7eff42ffd640 1 --2- 172.21.15.164:0/3576986526 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5572557185d0 0x557255722dd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7eff2c053040 tx=0x7eff2c005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eecdab9e46497e68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:33.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.772+0000 7eff38ff9640 1 -- 172.21.15.164:0/3576986526 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2c05e070 con 0x5572557185d0 2024-09-19T08:49:33.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.772+0000 7eff38ff9640 1 -- 172.21.15.164:0/3576986526 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff2c048de0 con 0x5572557185d0 2024-09-19T08:49:33.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.772+0000 7eff38ff9640 1 -- 172.21.15.164:0/3576986526 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2c05a850 con 0x5572557185d0 2024-09-19T08:49:33.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.775+0000 7eff4bbce740 1 -- 172.21.15.164:0/3576986526 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5572557185d0 msgr2=0x557255722dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:33.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.775+0000 7eff4bbce740 1 --2- 172.21.15.164:0/3576986526 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5572557185d0 0x557255722dd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7eff2c053040 tx=0x7eff2c005bd0 comp rx=0 tx=0).stop 2024-09-19T08:49:33.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.775+0000 7eff4bbce740 1 -- 172.21.15.164:0/3576986526 shutdown_connections 2024-09-19T08:49:33.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.775+0000 7eff4bbce740 1 --2- 172.21.15.164:0/3576986526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557255723370 0x557255713c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:33.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.775+0000 7eff4bbce740 1 --2- 172.21.15.164:0/3576986526 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557255719b40 0x55725571f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:33.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.775+0000 7eff4bbce740 1 --2- 172.21.15.164:0/3576986526 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5572557185d0 0x557255722dd0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:33.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.775+0000 7eff4bbce740 1 -- 172.21.15.164:0/3576986526 >> 172.21.15.164:0/3576986526 conn(0x5572556e1310 msgr2=0x557255713500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:33.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.783+0000 7eff4bbce740 1 -- 172.21.15.164:0/3576986526 shutdown_connections 2024-09-19T08:49:33.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.783+0000 7eff4bbce740 1 -- 172.21.15.164:0/3576986526 wait complete. 2024-09-19T08:49:33.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.783+0000 7eff4bbce740 1 -- start start 2024-09-19T08:49:33.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.784+0000 7eff4bbce740 1 Processor -- start 2024-09-19T08:49:33.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.784+0000 7eff4bbce740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5572557185d0 0x55725572b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:33.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.785+0000 7eff4bbce740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557255719b40 0x55725572bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:33.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.785+0000 7eff4bbce740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557255723370 0x55725571c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:33.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.785+0000 7eff4bbce740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7eff2400fb00 con 0x557255719b40 2024-09-19T08:49:33.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.785+0000 7eff4bbce740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7eff2400f430 con 0x5572557185d0 2024-09-19T08:49:33.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.785+0000 7eff4bbce740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7eff2400c670 con 0x557255723370 2024-09-19T08:49:33.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.785+0000 7eff4bbce740 10 client.0 _ll_register_callbacks cb 0x7eff2c03f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:49:33.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.790+0000 7eff427fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557255719b40 0x55725572bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:33.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.790+0000 7eff427fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557255719b40 0x55725572bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59790/0 (socket says 172.21.15.164:59790) 2024-09-19T08:49:33.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.790+0000 7eff427fc640 1 -- 172.21.15.164:0/2574202903 learned_addr learned my addr 172.21.15.164:0/2574202903 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:33.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.791+0000 7eff437fe640 1 --2- 172.21.15.164:0/2574202903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557255723370 0x55725571c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:33.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.791+0000 7eff42ffd640 1 --2- 172.21.15.164:0/2574202903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5572557185d0 0x55725572b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:33.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.793+0000 7eff427fc640 1 -- 172.21.15.164:0/2574202903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557255723370 msgr2=0x55725571c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:33.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.793+0000 7eff427fc640 1 --2- 172.21.15.164:0/2574202903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557255723370 0x55725571c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:33.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.793+0000 7eff427fc640 1 -- 172.21.15.164:0/2574202903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5572557185d0 msgr2=0x55725572b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:33.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.793+0000 7eff427fc640 1 --2- 172.21.15.164:0/2574202903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5572557185d0 0x55725572b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:33.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.793+0000 7eff427fc640 1 -- 172.21.15.164:0/2574202903 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55725571d140 con 0x557255719b40 2024-09-19T08:49:33.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.795+0000 7eff427fc640 1 --2- 172.21.15.164:0/2574202903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557255719b40 0x55725572bc80 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7eff4411b750 tx=0x7eff34014b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e18d01665be9ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:33.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.795+0000 7eff2a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:49:33.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.796+0000 7eff2a7fc640 1 -- 172.21.15.164:0/2574202903 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3401f070 con 0x557255719b40 2024-09-19T08:49:33.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.797+0000 7eff4bbce740 1 -- 172.21.15.164:0/2574202903 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff2c048de0 con 0x557255719b40 2024-09-19T08:49:33.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.797+0000 7eff2a7fc640 1 -- 172.21.15.164:0/2574202903 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7eff34012db0 con 0x557255719b40 2024-09-19T08:49:33.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.798+0000 7eff2a7fc640 1 -- 172.21.15.164:0/2574202903 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7eff34029050 con 0x557255719b40 2024-09-19T08:49:33.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.798+0000 7eff4bbce740 10 client.16290 fetch_fsmap learned FSMap version 28 2024-09-19T08:49:33.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.798+0000 7eff4bbce740 1 -- 172.21.15.164:0/2574202903 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55725579d900 con 0x557255719b40 2024-09-19T08:49:33.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.799+0000 7eff2a7fc640 1 -- 172.21.15.164:0/2574202903 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff2c048de0 con 0x557255719b40 2024-09-19T08:49:33.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.802+0000 7eff2a7fc640 1 -- 172.21.15.164:0/2574202903 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7eff3402c020 con 0x557255719b40 2024-09-19T08:49:33.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.802+0000 7eff4bbce740 10 client.16290 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:49:33.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.802+0000 7eff4bbce740 10 client.16290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:49:33.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.802+0000 7eff4bbce740 20 client.16290 populate_metadata read hostname 'smithi164' 2024-09-19T08:49:33.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.802+0000 7eff4bbce740 1 --2- 172.21.15.164:0/2574202903 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x557255733000 0x5572557a0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:33.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.802+0000 7eff4bbce740 4 client.16290 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:49:33.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.802+0000 7eff4bbce740 1 -- 172.21.15.164:0/2574202903 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55725568b860 con 0x557255733000 2024-09-19T08:49:33.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.808+0000 7eff42ffd640 1 --2- 172.21.15.164:0/2574202903 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x557255733000 0x5572557a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:33.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.827+0000 7eff2a7fc640 10 client.16290 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:49:33.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.827+0000 7eff42ffd640 1 --2- 172.21.15.164:0/2574202903 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x557255733000 0x5572557a0090 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6eea8ea196cb9933 server_cookie=4d5959c914c7250a in_seq=0 out_seq=0 2024-09-19T08:49:33.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.834+0000 7eff2a7fc640 1 -- 172.21.15.164:0/2574202903 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55725568b860 con 0x557255733000 2024-09-19T08:49:33.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.835+0000 7eff2a7fc640 10 client.16290 handle_command_reply: tid=0 2024-09-19T08:49:33.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.835+0000 7eff4bbce740 1 client.16290 shutdown 2024-09-19T08:49:33.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.837+0000 7eff4bbce740 1 -- 172.21.15.164:0/2574202903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557255719b40 msgr2=0x55725572bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:33.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.837+0000 7eff4bbce740 1 --2- 172.21.15.164:0/2574202903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557255719b40 0x55725572bc80 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7eff4411b750 tx=0x7eff34014b60 comp rx=0 tx=0).stop 2024-09-19T08:49:33.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.837+0000 7eff4bbce740 1 -- 172.21.15.164:0/2574202903 shutdown_connections 2024-09-19T08:49:33.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.837+0000 7eff4bbce740 1 --2- 172.21.15.164:0/2574202903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557255723370 0x55725571c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:33.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.837+0000 7eff4bbce740 1 --2- 172.21.15.164:0/2574202903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557255719b40 0x55725572bc80 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:33.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.837+0000 7eff4bbce740 1 --2- 172.21.15.164:0/2574202903 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x557255733000 0x5572557a0090 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:33.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.837+0000 7eff4bbce740 1 --2- 172.21.15.164:0/2574202903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5572557185d0 0x55725572b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:33.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.837+0000 7eff4bbce740 1 -- 172.21.15.164:0/2574202903 >> 172.21.15.164:0/2574202903 conn(0x5572556e1310 msgr2=0x557255713d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:33.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.837+0000 7eff2a7fc640 0 client.16290 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:49:33.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.840+0000 7eff4bbce740 1 -- 172.21.15.164:0/2574202903 shutdown_connections 2024-09-19T08:49:33.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.840+0000 7eff4bbce740 1 -- 172.21.15.164:0/2574202903 wait complete. 2024-09-19T08:49:33.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.840+0000 7eff4bbce740 20 client.16290 trim_cache size 0 max 16384 2024-09-19T08:49:33.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.841+0000 7eff497f5640 1 -- 172.21.15.164:0/4012148408 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7eff300a3500 msgr2=0x7eff300a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:33.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.841+0000 7eff497f5640 1 --2- 172.21.15.164:0/4012148408 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7eff300a3500 0x7eff300a59c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7eff2c02a990 tx=0x7eff2c000bc0 comp rx=0 tx=0).stop 2024-09-19T08:49:33.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.841+0000 7eff497f5640 1 -- 172.21.15.164:0/4012148408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff44124b90 msgr2=0x7eff44119f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:33.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.841+0000 7eff497f5640 1 --2- 172.21.15.164:0/4012148408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff44124b90 0x7eff44119f90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7eff3c00b3f0 tx=0x7eff3c00b8b0 comp rx=0 tx=0).stop 2024-09-19T08:49:33.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.842+0000 7eff497f5640 1 -- 172.21.15.164:0/4012148408 shutdown_connections 2024-09-19T08:49:33.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.842+0000 7eff497f5640 1 --2- 172.21.15.164:0/4012148408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff44124b90 0x7eff44119f90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:33.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.842+0000 7eff497f5640 1 --2- 172.21.15.164:0/4012148408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff4410e890 0x7eff4411f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:33.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.842+0000 7eff497f5640 1 --2- 172.21.15.164:0/4012148408 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7eff300a3500 0x7eff300a59c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:33.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.842+0000 7eff497f5640 1 --2- 172.21.15.164:0/4012148408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff44109d70 0x7eff4411ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:33.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.842+0000 7eff497f5640 1 -- 172.21.15.164:0/4012148408 >> 172.21.15.164:0/4012148408 conn(0x7eff4406e2d0 msgr2=0x7eff440707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:33.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.842+0000 7eff497f5640 1 -- 172.21.15.164:0/4012148408 shutdown_connections 2024-09-19T08:49:33.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:32.842+0000 7eff497f5640 1 -- 172.21.15.164:0/4012148408 wait complete. 2024-09-19T08:49:33.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:33 smithi170 ceph-mon[35072]: pgmap v115: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 729 MiB/s wr, 1.77k op/s 2024-09-19T08:49:33.883 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:33 smithi164 ceph-mon[30928]: pgmap v115: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 729 MiB/s wr, 1.77k op/s 2024-09-19T08:49:33.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:33 smithi173 ceph-mon[35306]: pgmap v115: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 729 MiB/s wr, 1.77k op/s 2024-09-19T08:49:35.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:35 smithi170 ceph-mon[35072]: pgmap v116: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 714 MiB/s wr, 1.61k op/s 2024-09-19T08:49:35.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:35 smithi173 ceph-mon[35306]: pgmap v116: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 714 MiB/s wr, 1.61k op/s 2024-09-19T08:49:36.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:35 smithi164 ceph-mon[30928]: pgmap v116: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 714 MiB/s wr, 1.61k op/s 2024-09-19T08:49:36.306 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:49:37.286 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T08:49:37.287 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T08:49:37.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:37 smithi173 ceph-mon[35306]: pgmap v117: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 881 MiB/s wr, 1.97k op/s 2024-09-19T08:49:37.997 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:37 smithi164 ceph-mon[30928]: pgmap v117: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 881 MiB/s wr, 1.97k op/s 2024-09-19T08:49:38.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.006+0000 7f3d6e48e640 1 -- 172.21.15.164:0/3069927602 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d68105d90 msgr2=0x7f3d68106220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:38.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.006+0000 7f3d6e48e640 1 --2- 172.21.15.164:0/3069927602 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d68105d90 0x7f3d68106220 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3d64009be0 tx=0x7f3d6402a760 comp rx=0 tx=0).stop 2024-09-19T08:49:38.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.006+0000 7f3d6e48e640 1 -- 172.21.15.164:0/3069927602 shutdown_connections 2024-09-19T08:49:38.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.006+0000 7f3d6e48e640 1 --2- 172.21.15.164:0/3069927602 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d68105d90 0x7f3d68106220 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:38.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.006+0000 7f3d6e48e640 1 --2- 172.21.15.164:0/3069927602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d681012a0 0x7f3d68105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:38.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.006+0000 7f3d6e48e640 1 --2- 172.21.15.164:0/3069927602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d68100980 0x7f3d68100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:38.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.006+0000 7f3d6e48e640 1 -- 172.21.15.164:0/3069927602 >> 172.21.15.164:0/3069927602 conn(0x7f3d680fc610 msgr2=0x7f3d680fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:38.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.010+0000 7f3d6e48e640 1 -- 172.21.15.164:0/3069927602 shutdown_connections 2024-09-19T08:49:38.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.010+0000 7f3d6e48e640 1 -- 172.21.15.164:0/3069927602 wait complete. 2024-09-19T08:49:38.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.011+0000 7f3d6e48e640 1 Processor -- start 2024-09-19T08:49:38.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.011+0000 7f3d6e48e640 1 -- start start 2024-09-19T08:49:38.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.017+0000 7f3d6e48e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d68100980 0x7f3d681125d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:38.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.017+0000 7f3d6e48e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d681012a0 0x7f3d68112b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:38.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.017+0000 7f3d6e48e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d6810d840 0x7f3d6810dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:38.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.017+0000 7f3d6e48e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3d681184a0 con 0x7f3d6810d840 2024-09-19T08:49:38.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.018+0000 7f3d6e48e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3d68118320 con 0x7f3d681012a0 2024-09-19T08:49:38.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.018+0000 7f3d6e48e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3d68118620 con 0x7f3d68100980 2024-09-19T08:49:38.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.019+0000 7f3d6d48c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d68100980 0x7f3d681125d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:38.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.019+0000 7f3d6cc8b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d681012a0 0x7f3d68112b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:38.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.023+0000 7f3d6d48c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d68100980 0x7f3d681125d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:47746/0 (socket says 172.21.15.164:47746) 2024-09-19T08:49:38.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.023+0000 7f3d6d48c640 1 -- 172.21.15.164:0/2724260204 learned_addr learned my addr 172.21.15.164:0/2724260204 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:38.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.027+0000 7f3d6d48c640 1 -- 172.21.15.164:0/2724260204 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d681012a0 msgr2=0x7f3d68112b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:38.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.027+0000 7f3d6d48c640 1 --2- 172.21.15.164:0/2724260204 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d681012a0 0x7f3d68112b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:38.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.027+0000 7f3d6d48c640 1 -- 172.21.15.164:0/2724260204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d6810d840 msgr2=0x7f3d6810dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:38.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.027+0000 7f3d6d48c640 1 --2- 172.21.15.164:0/2724260204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d6810d840 0x7f3d6810dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:38.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.027+0000 7f3d6d48c640 1 -- 172.21.15.164:0/2724260204 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6810e3f0 con 0x7f3d68100980 2024-09-19T08:49:38.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.027+0000 7f3d6d48c640 1 --2- 172.21.15.164:0/2724260204 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d68100980 0x7f3d681125d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3d6000e9f0 tx=0x7f3d6000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:38.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.027+0000 7f3d5e7fc640 1 -- 172.21.15.164:0/2724260204 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6000ccc0 con 0x7f3d68100980 2024-09-19T08:49:38.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.028+0000 7f3d6e48e640 1 -- 172.21.15.164:0/2724260204 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d68111340 con 0x7f3d68100980 2024-09-19T08:49:38.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.028+0000 7f3d6e48e640 1 -- 172.21.15.164:0/2724260204 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d68111880 con 0x7f3d68100980 2024-09-19T08:49:38.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.033+0000 7f3d71ac9740 1 Processor -- start 2024-09-19T08:49:38.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.035+0000 7f3d71ac9740 1 -- start start 2024-09-19T08:49:38.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.036+0000 7f3d71ac9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56552ee7d910 0x56552ee7dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:38.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.036+0000 7f3d71ac9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56552ee7e230 0x56552ee885b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:38.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.036+0000 7f3d71ac9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56552ee88af0 0x56552ee8aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:38.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.036+0000 7f3d71ac9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56552edd22a0 con 0x56552ee7d910 2024-09-19T08:49:38.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.036+0000 7f3d71ac9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56552ed79b10 con 0x56552ee7e230 2024-09-19T08:49:38.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.036+0000 7f3d71ac9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56552edce260 con 0x56552ee88af0 2024-09-19T08:49:38.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.036+0000 7f3d5e7fc640 1 -- 172.21.15.164:0/2724260204 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d60005500 con 0x7f3d68100980 2024-09-19T08:49:38.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.036+0000 7f3d5e7fc640 1 -- 172.21.15.164:0/2724260204 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d60010690 con 0x7f3d68100980 2024-09-19T08:49:38.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.036+0000 7f3d5e7fc640 1 -- 172.21.15.164:0/2724260204 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f3d600108f0 con 0x7f3d68100980 2024-09-19T08:49:38.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.037+0000 7f3d6d48c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56552ee7e230 0x56552ee885b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:38.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.040+0000 7f3d6d48c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56552ee7e230 0x56552ee885b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:44100/0 (socket says 172.21.15.164:44100) 2024-09-19T08:49:38.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.040+0000 7f3d6d48c640 1 -- 172.21.15.164:0/3503801199 learned_addr learned my addr 172.21.15.164:0/3503801199 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:38.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.040+0000 7f3d6cc8b640 1 --2- 172.21.15.164:0/3503801199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56552ee88af0 0x56552ee8aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:38.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.049+0000 7f3d6dc8d640 1 --2- 172.21.15.164:0/3503801199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56552ee7d910 0x56552ee7dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:38.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.050+0000 7f3d6d48c640 1 -- 172.21.15.164:0/3503801199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56552ee88af0 msgr2=0x56552ee8aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:38.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.050+0000 7f3d6d48c640 1 --2- 172.21.15.164:0/3503801199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56552ee88af0 0x56552ee8aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:38.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.050+0000 7f3d6d48c640 1 -- 172.21.15.164:0/3503801199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56552ee7d910 msgr2=0x56552ee7dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:38.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.050+0000 7f3d6d48c640 1 --2- 172.21.15.164:0/3503801199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56552ee7d910 0x56552ee7dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:38.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.050+0000 7f3d6d48c640 1 -- 172.21.15.164:0/3503801199 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56552eb43b90 con 0x56552ee7e230 2024-09-19T08:49:38.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.050+0000 7f3d6d48c640 1 --2- 172.21.15.164:0/3503801199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56552ee7e230 0x56552ee885b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3d600ce770 tx=0x7f3d60017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b778ea76e8f174c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:38.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.052+0000 7f3d5e7fc640 1 --2- 172.21.15.164:0/2724260204 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3d540a35f0 0x7f3d540a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:38.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.053+0000 7f3d5e7fc640 1 -- 172.21.15.164:0/2724260204 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f3d600d1050 con 0x7f3d68100980 2024-09-19T08:49:38.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.054+0000 7f3d537fe640 1 -- 172.21.15.164:0/3503801199 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d600d1630 con 0x56552ee7e230 2024-09-19T08:49:38.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.054+0000 7f3d537fe640 1 -- 172.21.15.164:0/3503801199 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d600d1cb0 con 0x56552ee7e230 2024-09-19T08:49:38.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.055+0000 7f3d71ac9740 1 -- 172.21.15.164:0/3503801199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56552ee7e230 msgr2=0x56552ee885b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:38.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.055+0000 7f3d71ac9740 1 --2- 172.21.15.164:0/3503801199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56552ee7e230 0x56552ee885b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3d600ce770 tx=0x7f3d60017000 comp rx=0 tx=0).stop 2024-09-19T08:49:38.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.062+0000 7f3d6dc8d640 1 --2- 172.21.15.164:0/2724260204 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3d540a35f0 0x7f3d540a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:38.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.062+0000 7f3d71ac9740 1 -- 172.21.15.164:0/3503801199 shutdown_connections 2024-09-19T08:49:38.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.062+0000 7f3d71ac9740 1 --2- 172.21.15.164:0/3503801199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56552ee88af0 0x56552ee8aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:38.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.062+0000 7f3d71ac9740 1 --2- 172.21.15.164:0/3503801199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56552ee7e230 0x56552ee885b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:38.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.062+0000 7f3d71ac9740 1 --2- 172.21.15.164:0/3503801199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56552ee7d910 0x56552ee7dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:38.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.062+0000 7f3d71ac9740 1 -- 172.21.15.164:0/3503801199 >> 172.21.15.164:0/3503801199 conn(0x56552ee79ea0 msgr2=0x56552ee7a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:38.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.069+0000 7f3d71ac9740 1 -- 172.21.15.164:0/3503801199 shutdown_connections 2024-09-19T08:49:38.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.071+0000 7f3d6dc8d640 1 --2- 172.21.15.164:0/2724260204 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3d540a35f0 0x7f3d540a5ab0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3d64031040 tx=0x7f3d64005e30 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:38.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.071+0000 7f3d71ac9740 1 -- 172.21.15.164:0/3503801199 wait complete. 2024-09-19T08:49:38.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.071+0000 7f3d71ac9740 1 -- start start 2024-09-19T08:49:38.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.072+0000 7f3d71ac9740 1 Processor -- start 2024-09-19T08:49:38.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.072+0000 7f3d71ac9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56552ee7d910 0x56552ee85a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:38.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.072+0000 7f3d71ac9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56552ee7e230 0x56552ee85fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:38.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.072+0000 7f3d71ac9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56552ee88af0 0x56552ee864e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:38.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.072+0000 7f3d71ac9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56552ee68b70 con 0x56552ee7e230 2024-09-19T08:49:38.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.072+0000 7f3d71ac9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56552ed5a6e0 con 0x56552ee7d910 2024-09-19T08:49:38.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.072+0000 7f3d71ac9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56552ee68cf0 con 0x56552ee88af0 2024-09-19T08:49:38.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.072+0000 7f3d71ac9740 10 client.0 _ll_register_callbacks cb 0x56552ed7ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:49:38.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.077+0000 7f3d6d48c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56552ee7d910 0x56552ee85a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:38.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.077+0000 7f3d6dc8d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56552ee88af0 0x56552ee864e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:38.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.077+0000 7f3d6d48c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56552ee7d910 0x56552ee85a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:44112/0 (socket says 172.21.15.164:44112) 2024-09-19T08:49:38.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.077+0000 7f3d6d48c640 1 -- 172.21.15.164:0/2838298028 learned_addr learned my addr 172.21.15.164:0/2838298028 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:38.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.081+0000 7f3d6cc8b640 1 --2- 172.21.15.164:0/2838298028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56552ee7e230 0x56552ee85fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:38.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.084+0000 7f3d6d48c640 1 -- 172.21.15.164:0/2838298028 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56552ee88af0 msgr2=0x56552ee864e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:38.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.084+0000 7f3d6d48c640 1 --2- 172.21.15.164:0/2838298028 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56552ee88af0 0x56552ee864e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:38.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.084+0000 7f3d6d48c640 1 -- 172.21.15.164:0/2838298028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56552ee7e230 msgr2=0x56552ee85fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:38.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.084+0000 7f3d6d48c640 1 --2- 172.21.15.164:0/2838298028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56552ee7e230 0x56552ee85fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:38.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.084+0000 7f3d6d48c640 1 -- 172.21.15.164:0/2838298028 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56552ee721c0 con 0x56552ee7d910 2024-09-19T08:49:38.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.089+0000 7f3d6d48c640 1 --2- 172.21.15.164:0/2838298028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56552ee7d910 0x56552ee85a60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3d6000e9c0 tx=0x7f3d60005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=293234a5997495e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:38.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.089+0000 7f3d50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-19T08:49:38.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.093+0000 7f3d50ff9640 1 -- 172.21.15.164:0/2838298028 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d60099660 con 0x56552ee7d910 2024-09-19T08:49:38.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.093+0000 7f3d50ff9640 1 -- 172.21.15.164:0/2838298028 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f3d600ceab0 con 0x56552ee7d910 2024-09-19T08:49:38.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.097+0000 7f3d71ac9740 1 -- 172.21.15.164:0/2838298028 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d600d1cb0 con 0x56552ee7d910 2024-09-19T08:49:38.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.099+0000 7f3d50ff9640 1 -- 172.21.15.164:0/2838298028 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3d600056b0 con 0x56552ee7d910 2024-09-19T08:49:38.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.099+0000 7f3d71ac9740 10 client.25703 fetch_fsmap learned FSMap version 28 2024-09-19T08:49:38.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.099+0000 7f3d71ac9740 1 -- 172.21.15.164:0/2838298028 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56552ee68b70 con 0x56552ee7d910 2024-09-19T08:49:38.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.103+0000 7f3d50ff9640 1 -- 172.21.15.164:0/2838298028 <== mon.2 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d600d1cb0 con 0x56552ee7d910 2024-09-19T08:49:38.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.103+0000 7f3d50ff9640 1 -- 172.21.15.164:0/2838298028 <== mon.2 v2:172.21.15.170:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3d60010be0 con 0x56552ee7d910 2024-09-19T08:49:38.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.103+0000 7f3d71ac9740 10 client.25703 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:49:38.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.103+0000 7f3d71ac9740 10 client.25703 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:49:38.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.103+0000 7f3d71ac9740 20 client.25703 populate_metadata read hostname 'smithi164' 2024-09-19T08:49:38.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.103+0000 7f3d71ac9740 1 --2- 172.21.15.164:0/2838298028 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56552ee76000 0x56552ee78440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:38.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.103+0000 7f3d71ac9740 4 client.25703 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:49:38.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.103+0000 7f3d71ac9740 1 -- 172.21.15.164:0/2838298028 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56552eddf9d0 con 0x56552ee76000 2024-09-19T08:49:38.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.107+0000 7f3d6cc8b640 1 --2- 172.21.15.164:0/2838298028 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56552ee76000 0x56552ee78440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:38.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.113+0000 7f3d6cc8b640 1 --2- 172.21.15.164:0/2838298028 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56552ee76000 0x56552ee78440 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21d769ee837a5074 server_cookie=63a0829a6f3c3cf3 in_seq=0 out_seq=0 2024-09-19T08:49:38.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.113+0000 7f3d50ff9640 10 client.25703 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:49:38.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.114+0000 7f3d50ff9640 1 -- 172.21.15.164:0/2838298028 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56552eddf9d0 con 0x56552ee76000 2024-09-19T08:49:38.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.114+0000 7f3d50ff9640 10 client.25703 handle_command_reply: tid=0 2024-09-19T08:49:38.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.115+0000 7f3d71ac9740 1 client.25703 shutdown 2024-09-19T08:49:38.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.115+0000 7f3d71ac9740 1 -- 172.21.15.164:0/2838298028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56552ee7d910 msgr2=0x56552ee85a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:38.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.115+0000 7f3d71ac9740 1 --2- 172.21.15.164:0/2838298028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56552ee7d910 0x56552ee85a60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3d6000e9c0 tx=0x7f3d60005a20 comp rx=0 tx=0).stop 2024-09-19T08:49:38.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.115+0000 7f3d71ac9740 1 -- 172.21.15.164:0/2838298028 shutdown_connections 2024-09-19T08:49:38.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.115+0000 7f3d71ac9740 1 --2- 172.21.15.164:0/2838298028 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56552ee88af0 0x56552ee864e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:38.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.115+0000 7f3d71ac9740 1 --2- 172.21.15.164:0/2838298028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56552ee7e230 0x56552ee85fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:38.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.115+0000 7f3d71ac9740 1 --2- 172.21.15.164:0/2838298028 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56552ee76000 0x56552ee78440 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:38.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.115+0000 7f3d71ac9740 1 --2- 172.21.15.164:0/2838298028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56552ee7d910 0x56552ee85a60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:38.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.115+0000 7f3d71ac9740 1 -- 172.21.15.164:0/2838298028 >> 172.21.15.164:0/2838298028 conn(0x56552ee79ea0 msgr2=0x56552ee7fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:38.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.115+0000 7f3d50ff9640 0 client.25703 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:49:38.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.116+0000 7f3d71ac9740 1 -- 172.21.15.164:0/2838298028 shutdown_connections 2024-09-19T08:49:38.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.118+0000 7f3d71ac9740 1 -- 172.21.15.164:0/2838298028 wait complete. 2024-09-19T08:49:38.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.118+0000 7f3d71ac9740 20 client.25703 trim_cache size 0 max 16384 2024-09-19T08:49:38.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:37 smithi170 ceph-mon[35072]: pgmap v117: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 881 MiB/s wr, 1.97k op/s 2024-09-19T08:49:38.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.157+0000 7f3d71ac9740 1 Processor -- start 2024-09-19T08:49:38.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.159+0000 7f3d71ac9740 1 -- start start 2024-09-19T08:49:38.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.159+0000 7f3d71ac9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56552ee7d910 0x56552ee86330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:38.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.160+0000 7f3d71ac9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56552ee7e230 0x56552eeb2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:38.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.160+0000 7f3d71ac9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56552ee88af0 0x56552eeb29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:38.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.160+0000 7f3d71ac9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4c00e740 con 0x56552ee88af0 2024-09-19T08:49:38.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.160+0000 7f3d71ac9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4c010280 con 0x56552ee7e230 2024-09-19T08:49:38.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.160+0000 7f3d71ac9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4c00ee10 con 0x56552ee7d910 2024-09-19T08:49:38.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.160+0000 7f3d6d48c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56552ee7d910 0x56552ee86330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:38.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.160+0000 7f3d6d48c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56552ee7d910 0x56552ee86330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:47780/0 (socket says 172.21.15.164:47780) 2024-09-19T08:49:38.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.160+0000 7f3d6d48c640 1 -- 172.21.15.164:0/3003976378 learned_addr learned my addr 172.21.15.164:0/3003976378 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:38.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.160+0000 7f3d6cc8b640 1 --2- 172.21.15.164:0/3003976378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56552ee7e230 0x56552eeb2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:38.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.162+0000 7f3d6d48c640 1 -- 172.21.15.164:0/3003976378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56552ee7e230 msgr2=0x56552eeb2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:38.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.162+0000 7f3d6d48c640 1 --2- 172.21.15.164:0/3003976378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56552ee7e230 0x56552eeb2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:38.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.162+0000 7f3d6d48c640 1 -- 172.21.15.164:0/3003976378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56552ee88af0 msgr2=0x56552eeb29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:38.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.162+0000 7f3d6d48c640 1 --2- 172.21.15.164:0/3003976378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56552ee88af0 0x56552eeb29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:38.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.162+0000 7f3d6d48c640 1 -- 172.21.15.164:0/3003976378 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56552ee7b030 con 0x56552ee7d910 2024-09-19T08:49:38.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.163+0000 7f3d6d48c640 1 --2- 172.21.15.164:0/3003976378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56552ee7d910 0x56552ee86330 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3d60099d90 tx=0x7f3d600ce910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eafb1b556b966a62 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:38.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.169+0000 7f3d52ffd640 1 -- 172.21.15.164:0/3003976378 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6009cd20 con 0x56552ee7d910 2024-09-19T08:49:38.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.169+0000 7f3d52ffd640 1 -- 172.21.15.164:0/3003976378 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d600a1770 con 0x56552ee7d910 2024-09-19T08:49:38.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.169+0000 7f3d52ffd640 1 -- 172.21.15.164:0/3003976378 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6009cf00 con 0x56552ee7d910 2024-09-19T08:49:38.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.170+0000 7f3d71ac9740 1 -- 172.21.15.164:0/3003976378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56552ee7d910 msgr2=0x56552ee86330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:38.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.170+0000 7f3d71ac9740 1 --2- 172.21.15.164:0/3003976378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56552ee7d910 0x56552ee86330 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3d60099d90 tx=0x7f3d600ce910 comp rx=0 tx=0).stop 2024-09-19T08:49:38.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.174+0000 7f3d71ac9740 1 -- 172.21.15.164:0/3003976378 shutdown_connections 2024-09-19T08:49:38.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.174+0000 7f3d71ac9740 1 --2- 172.21.15.164:0/3003976378 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56552ee88af0 0x56552eeb29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:38.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.174+0000 7f3d71ac9740 1 --2- 172.21.15.164:0/3003976378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56552ee7e230 0x56552eeb2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:38.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.174+0000 7f3d71ac9740 1 --2- 172.21.15.164:0/3003976378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56552ee7d910 0x56552ee86330 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:38.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.174+0000 7f3d71ac9740 1 -- 172.21.15.164:0/3003976378 >> 172.21.15.164:0/3003976378 conn(0x56552ee79ea0 msgr2=0x56552ee87ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:38.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.175+0000 7f3d71ac9740 1 -- 172.21.15.164:0/3003976378 shutdown_connections 2024-09-19T08:49:38.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.175+0000 7f3d71ac9740 1 -- 172.21.15.164:0/3003976378 wait complete. 2024-09-19T08:49:38.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.176+0000 7f3d71ac9740 1 -- start start 2024-09-19T08:49:38.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.176+0000 7f3d71ac9740 1 Processor -- start 2024-09-19T08:49:38.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.177+0000 7f3d71ac9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56552ee7d910 0x56552ee690a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:38.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.177+0000 7f3d71ac9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56552ee7e230 0x56552ee695e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:38.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.178+0000 7f3d71ac9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56552ee88af0 0x56552ef13420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:38.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.178+0000 7f3d71ac9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4c00fbb0 con 0x56552ee7e230 2024-09-19T08:49:38.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.178+0000 7f3d71ac9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4c00f4e0 con 0x56552ee88af0 2024-09-19T08:49:38.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.178+0000 7f3d71ac9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4c00c720 con 0x56552ee7d910 2024-09-19T08:49:38.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.178+0000 7f3d71ac9740 10 client.0 _ll_register_callbacks cb 0x7f3d600ce7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:49:38.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.179+0000 7f3d6d48c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56552ee7d910 0x56552ee690a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:38.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.179+0000 7f3d6dc8d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56552ee88af0 0x56552ef13420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:38.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.180+0000 7f3d6d48c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56552ee7d910 0x56552ee690a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:47792/0 (socket says 172.21.15.164:47792) 2024-09-19T08:49:38.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.180+0000 7f3d6d48c640 1 -- 172.21.15.164:0/2347070169 learned_addr learned my addr 172.21.15.164:0/2347070169 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:38.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.180+0000 7f3d6d48c640 1 -- 172.21.15.164:0/2347070169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56552ee88af0 msgr2=0x56552ef13420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:38.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.180+0000 7f3d6cc8b640 1 --2- 172.21.15.164:0/2347070169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56552ee7e230 0x56552ee695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:38.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.180+0000 7f3d6d48c640 1 --2- 172.21.15.164:0/2347070169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56552ee88af0 0x56552ef13420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:38.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.180+0000 7f3d6d48c640 1 -- 172.21.15.164:0/2347070169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56552ee7e230 msgr2=0x56552ee695e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:38.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.181+0000 7f3d6d48c640 1 --2- 172.21.15.164:0/2347070169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56552ee7e230 0x56552ee695e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:38.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.181+0000 7f3d6d48c640 1 -- 172.21.15.164:0/2347070169 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56552ee69ee0 con 0x56552ee7d910 2024-09-19T08:49:38.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.181+0000 7f3d6cc8b640 1 --2- 172.21.15.164:0/2347070169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56552ee7e230 0x56552ee695e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:49:38.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.185+0000 7f3d6d48c640 1 --2- 172.21.15.164:0/2347070169 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56552ee7d910 0x56552ee690a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3d600d1330 tx=0x7f3d60099770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23078541f95df24f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:38.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.186+0000 7f3d50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-09-19T08:49:38.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.188+0000 7f3d50ff9640 1 -- 172.21.15.164:0/2347070169 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d600ba070 con 0x56552ee7d910 2024-09-19T08:49:38.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.188+0000 7f3d71ac9740 1 -- 172.21.15.164:0/2347070169 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d600a1770 con 0x56552ee7d910 2024-09-19T08:49:38.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.188+0000 7f3d50ff9640 1 -- 172.21.15.164:0/2347070169 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f3d6009d620 con 0x56552ee7d910 2024-09-19T08:49:38.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.189+0000 7f3d50ff9640 1 -- 172.21.15.164:0/2347070169 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3d6009d190 con 0x56552ee7d910 2024-09-19T08:49:38.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.190+0000 7f3d71ac9740 10 client.25714 fetch_fsmap learned FSMap version 28 2024-09-19T08:49:38.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.190+0000 7f3d71ac9740 1 -- 172.21.15.164:0/2347070169 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56552ee69ee0 con 0x56552ee7d910 2024-09-19T08:49:38.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.196+0000 7f3d50ff9640 1 -- 172.21.15.164:0/2347070169 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d600a1770 con 0x56552ee7d910 2024-09-19T08:49:38.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.196+0000 7f3d50ff9640 1 -- 172.21.15.164:0/2347070169 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3d600a5ba0 con 0x56552ee7d910 2024-09-19T08:49:38.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.197+0000 7f3d71ac9740 10 client.25714 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:49:38.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.197+0000 7f3d71ac9740 10 client.25714 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:49:38.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.197+0000 7f3d71ac9740 20 client.25714 populate_metadata read hostname 'smithi164' 2024-09-19T08:49:38.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.197+0000 7f3d71ac9740 1 --2- 172.21.15.164:0/2347070169 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56552ef14d90 0x56552ef17180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:38.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.197+0000 7f3d71ac9740 4 client.25714 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}] 2024-09-19T08:49:38.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.197+0000 7f3d71ac9740 1 -- 172.21.15.164:0/2347070169 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}) -- 0x56552edd9d20 con 0x56552ef14d90 2024-09-19T08:49:38.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.198+0000 7f3d6cc8b640 1 --2- 172.21.15.164:0/2347070169 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56552ef14d90 0x56552ef17180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:38.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.204+0000 7f3d6cc8b640 1 --2- 172.21.15.164:0/2347070169 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56552ef14d90 0x56552ef17180 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2fd155c1f18f4f8 server_cookie=e3f6b49aa3bf0021 in_seq=0 out_seq=0 2024-09-19T08:49:38.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.204+0000 7f3d50ff9640 10 client.25714 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:49:38.235 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:49:38.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.359+0000 7f3d50ff9640 1 -- 172.21.15.164:0/2347070169 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56552edd9d20 con 0x56552ef14d90 2024-09-19T08:49:38.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.359+0000 7f3d50ff9640 10 client.25714 handle_command_reply: tid=0 2024-09-19T08:49:38.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.359+0000 7f3d71ac9740 1 client.25714 shutdown 2024-09-19T08:49:38.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.359+0000 7f3d71ac9740 1 -- 172.21.15.164:0/2347070169 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56552ee7d910 msgr2=0x56552ee690a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:38.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.359+0000 7f3d71ac9740 1 --2- 172.21.15.164:0/2347070169 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56552ee7d910 0x56552ee690a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3d600d1330 tx=0x7f3d60099770 comp rx=0 tx=0).stop 2024-09-19T08:49:38.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.359+0000 7f3d71ac9740 1 -- 172.21.15.164:0/2347070169 shutdown_connections 2024-09-19T08:49:38.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.359+0000 7f3d71ac9740 1 --2- 172.21.15.164:0/2347070169 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56552ef14d90 0x56552ef17180 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:38.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.359+0000 7f3d71ac9740 1 --2- 172.21.15.164:0/2347070169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56552ee88af0 0x56552ef13420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:38.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.359+0000 7f3d71ac9740 1 --2- 172.21.15.164:0/2347070169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56552ee7e230 0x56552ee695e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:38.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.359+0000 7f3d71ac9740 1 --2- 172.21.15.164:0/2347070169 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56552ee7d910 0x56552ee690a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:38.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.359+0000 7f3d71ac9740 1 -- 172.21.15.164:0/2347070169 >> 172.21.15.164:0/2347070169 conn(0x56552ee79ea0 msgr2=0x56552ee677e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:38.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.359+0000 7f3d50ff9640 0 client.25714 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:49:38.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.360+0000 7f3d71ac9740 1 -- 172.21.15.164:0/2347070169 shutdown_connections 2024-09-19T08:49:38.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.360+0000 7f3d71ac9740 1 -- 172.21.15.164:0/2347070169 wait complete. 2024-09-19T08:49:38.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.360+0000 7f3d71ac9740 20 client.25714 trim_cache size 0 max 16384 2024-09-19T08:49:38.362 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:49:38.362 INFO:teuthology.orchestra.run.smithi164.stdout: "epoch": 28, 2024-09-19T08:49:38.362 INFO:teuthology.orchestra.run.smithi164.stdout: "leader": 24479, 2024-09-19T08:49:38.362 INFO:teuthology.orchestra.run.smithi164.stdout: "set_version": 14, 2024-09-19T08:49:38.362 INFO:teuthology.orchestra.run.smithi164.stdout: "sets": { 2024-09-19T08:49:38.362 INFO:teuthology.orchestra.run.smithi164.stdout: "1bb05f3d": { 2024-09-19T08:49:38.362 INFO:teuthology.orchestra.run.smithi164.stdout: "version": 14, 2024-09-19T08:49:38.362 INFO:teuthology.orchestra.run.smithi164.stdout: "age_ref": 0.0, 2024-09-19T08:49:38.362 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:49:38.363 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "QUIESCED", 2024-09-19T08:49:38.363 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:49:38.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:49:38.363 INFO:teuthology.orchestra.run.smithi164.stdout: "timeout": 90.0, 2024-09-19T08:49:38.363 INFO:teuthology.orchestra.run.smithi164.stdout: "expiration": 136.6, 2024-09-19T08:49:38.363 INFO:teuthology.orchestra.run.smithi164.stdout: "members": { 2024-09-19T08:49:38.363 INFO:teuthology.orchestra.run.smithi164.stdout: "file:/": { 2024-09-19T08:49:38.363 INFO:teuthology.orchestra.run.smithi164.stdout: "excluded": false, 2024-09-19T08:49:38.363 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:49:38.363 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "QUIESCED", 2024-09-19T08:49:38.363 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:49:38.363 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:49:38.363 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:49:38.364 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:49:38.364 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:49:38.364 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:49:38.364 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:49:38.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.360+0000 7f3d6e48e640 1 -- 172.21.15.164:0/2724260204 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3d540a35f0 msgr2=0x7f3d540a5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:38.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.360+0000 7f3d6e48e640 1 --2- 172.21.15.164:0/2724260204 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3d540a35f0 0x7f3d540a5ab0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3d64031040 tx=0x7f3d64005e30 comp rx=0 tx=0).stop 2024-09-19T08:49:38.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.360+0000 7f3d6e48e640 1 -- 172.21.15.164:0/2724260204 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d68100980 msgr2=0x7f3d681125d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:38.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.360+0000 7f3d6e48e640 1 --2- 172.21.15.164:0/2724260204 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d68100980 0x7f3d681125d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3d6000e9f0 tx=0x7f3d6000eeb0 comp rx=0 tx=0).stop 2024-09-19T08:49:38.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.360+0000 7f3d6e48e640 1 -- 172.21.15.164:0/2724260204 shutdown_connections 2024-09-19T08:49:38.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.360+0000 7f3d6e48e640 1 --2- 172.21.15.164:0/2724260204 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3d540a35f0 0x7f3d540a5ab0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:38.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.360+0000 7f3d6e48e640 1 --2- 172.21.15.164:0/2724260204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d6810d840 0x7f3d6810dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:38.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.360+0000 7f3d6e48e640 1 --2- 172.21.15.164:0/2724260204 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d681012a0 0x7f3d68112b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:38.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.360+0000 7f3d6e48e640 1 --2- 172.21.15.164:0/2724260204 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d68100980 0x7f3d681125d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:38.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.360+0000 7f3d6e48e640 1 -- 172.21.15.164:0/2724260204 >> 172.21.15.164:0/2724260204 conn(0x7f3d680fc610 msgr2=0x7f3d680fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:38.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.361+0000 7f3d6e48e640 1 -- 172.21.15.164:0/2724260204 shutdown_connections 2024-09-19T08:49:38.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:38.361+0000 7f3d6e48e640 1 -- 172.21.15.164:0/2724260204 wait complete. 2024-09-19T08:49:39.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.637+0000 7fd1f287a640 1 -- 172.21.15.164:0/2378755286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1ec101420 msgr2=0x7fd1ec10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:39.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.637+0000 7fd1f287a640 1 --2- 172.21.15.164:0/2378755286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1ec101420 0x7fd1ec10d950 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4009cd0 tx=0x7fd1e402a8e0 comp rx=0 tx=0).stop 2024-09-19T08:49:39.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.637+0000 7fd1f287a640 1 -- 172.21.15.164:0/2378755286 shutdown_connections 2024-09-19T08:49:39.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.637+0000 7fd1f287a640 1 --2- 172.21.15.164:0/2378755286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1ec101420 0x7fd1ec10d950 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:39.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.637+0000 7fd1f287a640 1 --2- 172.21.15.164:0/2378755286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1ec100a80 0x7fd1ec100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:39.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.637+0000 7fd1f287a640 1 --2- 172.21.15.164:0/2378755286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1ec106a80 0x7fd1ec106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:39.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.637+0000 7fd1f287a640 1 -- 172.21.15.164:0/2378755286 >> 172.21.15.164:0/2378755286 conn(0x7fd1ec0fc7a0 msgr2=0x7fd1ec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:39.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.637+0000 7fd1f287a640 1 -- 172.21.15.164:0/2378755286 shutdown_connections 2024-09-19T08:49:39.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.637+0000 7fd1f287a640 1 -- 172.21.15.164:0/2378755286 wait complete. 2024-09-19T08:49:39.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.637+0000 7fd1f287a640 1 Processor -- start 2024-09-19T08:49:39.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.638+0000 7fd1f287a640 1 -- start start 2024-09-19T08:49:39.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.638+0000 7fd1f287a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1ec100a80 0x7fd1ec198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:39.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.638+0000 7fd1f287a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1ec101420 0x7fd1ec1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:39.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.638+0000 7fd1f287a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1ec106a80 0x7fd1ec19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:39.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.638+0000 7fd1f287a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec110ea0 con 0x7fd1ec106a80 2024-09-19T08:49:39.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.638+0000 7fd1f287a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec110d20 con 0x7fd1ec100a80 2024-09-19T08:49:39.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.638+0000 7fd1f287a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec111020 con 0x7fd1ec101420 2024-09-19T08:49:39.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.638+0000 7fd1f0e17640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1ec106a80 0x7fd1ec19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:39.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.638+0000 7fd1f0e17640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1ec106a80 0x7fd1ec19cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36178/0 (socket says 172.21.15.164:36178) 2024-09-19T08:49:39.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.638+0000 7fd1f0e17640 1 -- 172.21.15.164:0/3843905046 learned_addr learned my addr 172.21.15.164:0/3843905046 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:39.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.638+0000 7fd1eb7fe640 1 --2- 172.21.15.164:0/3843905046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1ec101420 0x7fd1ec1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:39.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.638+0000 7fd1ebfff640 1 --2- 172.21.15.164:0/3843905046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1ec100a80 0x7fd1ec198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:39.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.638+0000 7fd1f0e17640 1 -- 172.21.15.164:0/3843905046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1ec101420 msgr2=0x7fd1ec1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:39.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.638+0000 7fd1f0e17640 1 --2- 172.21.15.164:0/3843905046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1ec101420 0x7fd1ec1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:39.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.638+0000 7fd1f0e17640 1 -- 172.21.15.164:0/3843905046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1ec100a80 msgr2=0x7fd1ec198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:39.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.638+0000 7fd1f0e17640 1 --2- 172.21.15.164:0/3843905046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1ec100a80 0x7fd1ec198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:39.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.638+0000 7fd1f0e17640 1 -- 172.21.15.164:0/3843905046 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1ec19d420 con 0x7fd1ec106a80 2024-09-19T08:49:39.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.639+0000 7fd1f0e17640 1 --2- 172.21.15.164:0/3843905046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1ec106a80 0x7fd1ec19cd40 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4006010 tx=0x7fd1e4003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:39.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.639+0000 7fd1e97fa640 1 -- 172.21.15.164:0/3843905046 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4004450 con 0x7fd1ec106a80 2024-09-19T08:49:39.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.639+0000 7fd1e97fa640 1 -- 172.21.15.164:0/3843905046 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1e4003e10 con 0x7fd1ec106a80 2024-09-19T08:49:39.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.639+0000 7fd1f287a640 1 -- 172.21.15.164:0/3843905046 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1ec19d6b0 con 0x7fd1ec106a80 2024-09-19T08:49:39.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.639+0000 7fd1e97fa640 1 -- 172.21.15.164:0/3843905046 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4046480 con 0x7fd1ec106a80 2024-09-19T08:49:39.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.639+0000 7fd1f287a640 1 -- 172.21.15.164:0/3843905046 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1ec1a4ff0 con 0x7fd1ec106a80 2024-09-19T08:49:39.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.641+0000 7fd1e97fa640 1 -- 172.21.15.164:0/3843905046 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fd1e40045f0 con 0x7fd1ec106a80 2024-09-19T08:49:39.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.641+0000 7fd1f4c53740 1 Processor -- start 2024-09-19T08:49:39.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.642+0000 7fd1f4c53740 1 -- start start 2024-09-19T08:49:39.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.642+0000 7fd1f4c53740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576b08cfb60 0x5576b08cff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:39.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.642+0000 7fd1f4c53740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576b08d0480 0x5576b08da800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:39.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.642+0000 7fd1f4c53740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576b08dad40 0x5576b08dd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:39.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.642+0000 7fd1f4c53740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5576b0835280 con 0x5576b08d0480 2024-09-19T08:49:39.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.642+0000 7fd1f4c53740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5576b07dcb10 con 0x5576b08cfb60 2024-09-19T08:49:39.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.642+0000 7fd1f4c53740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5576b0831240 con 0x5576b08dad40 2024-09-19T08:49:39.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.642+0000 7fd1ebfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576b08d0480 0x5576b08da800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:39.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.642+0000 7fd1ebfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576b08d0480 0x5576b08da800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36194/0 (socket says 172.21.15.164:36194) 2024-09-19T08:49:39.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.642+0000 7fd1ebfff640 1 -- 172.21.15.164:0/46803567 learned_addr learned my addr 172.21.15.164:0/46803567 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:39.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.642+0000 7fd1e97fa640 1 --2- 172.21.15.164:0/3843905046 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd1d40a33b0 0x7fd1d40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:39.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.642+0000 7fd1eb7fe640 1 --2- 172.21.15.164:0/46803567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576b08cfb60 0x5576b08cff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:39.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.642+0000 7fd1eb7fe640 1 --2- 172.21.15.164:0/46803567 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576b08dad40 0x5576b08dd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:39.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.642+0000 7fd1f0e17640 1 --2- 172.21.15.164:0/3843905046 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd1d40a33b0 0x7fd1d40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:39.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.642+0000 7fd1ebfff640 1 -- 172.21.15.164:0/46803567 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576b08dad40 msgr2=0x5576b08dd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:39.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.642+0000 7fd1ebfff640 1 --2- 172.21.15.164:0/46803567 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576b08dad40 0x5576b08dd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:39.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.642+0000 7fd1ebfff640 1 -- 172.21.15.164:0/46803567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576b08cfb60 msgr2=0x5576b08cff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:39.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.642+0000 7fd1ebfff640 1 --2- 172.21.15.164:0/46803567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576b08cfb60 0x5576b08cff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:39.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.642+0000 7fd1ebfff640 1 -- 172.21.15.164:0/46803567 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576b07e91f0 con 0x5576b08d0480 2024-09-19T08:49:39.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.642+0000 7fd1e97fa640 1 -- 172.21.15.164:0/3843905046 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fd1e402e030 con 0x7fd1ec106a80 2024-09-19T08:49:39.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.643+0000 7fd1ebfff640 1 --2- 172.21.15.164:0/46803567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576b08d0480 0x5576b08da800 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009a70 tx=0x7fd1dc009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df6cce7f4335a0b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:39.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.643+0000 7fd1f0e17640 1 --2- 172.21.15.164:0/3843905046 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd1d40a33b0 0x7fd1d40a5870 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd1e40b9d30 tx=0x7fd1e4088210 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:39.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.643+0000 7fd1d27fc640 1 -- 172.21.15.164:0/46803567 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc017070 con 0x5576b08d0480 2024-09-19T08:49:39.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.643+0000 7fd1d27fc640 1 -- 172.21.15.164:0/46803567 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1dc013470 con 0x5576b08d0480 2024-09-19T08:49:39.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.643+0000 7fd1d27fc640 1 -- 172.21.15.164:0/46803567 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc0125f0 con 0x5576b08d0480 2024-09-19T08:49:39.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.643+0000 7fd1f4c53740 1 -- 172.21.15.164:0/46803567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576b08d0480 msgr2=0x5576b08da800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:39.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.643+0000 7fd1f4c53740 1 --2- 172.21.15.164:0/46803567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576b08d0480 0x5576b08da800 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009a70 tx=0x7fd1dc009f30 comp rx=0 tx=0).stop 2024-09-19T08:49:39.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.643+0000 7fd1f4c53740 1 -- 172.21.15.164:0/46803567 shutdown_connections 2024-09-19T08:49:39.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.643+0000 7fd1f4c53740 1 --2- 172.21.15.164:0/46803567 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576b08dad40 0x5576b08dd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:39.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.643+0000 7fd1f4c53740 1 --2- 172.21.15.164:0/46803567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576b08d0480 0x5576b08da800 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:39.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.643+0000 7fd1f4c53740 1 --2- 172.21.15.164:0/46803567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576b08cfb60 0x5576b08cff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:39.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.643+0000 7fd1f4c53740 1 -- 172.21.15.164:0/46803567 >> 172.21.15.164:0/46803567 conn(0x5576b08cc0f0 msgr2=0x5576b08cc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:39.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.644+0000 7fd1f4c53740 1 -- 172.21.15.164:0/46803567 shutdown_connections 2024-09-19T08:49:39.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.644+0000 7fd1f4c53740 1 -- 172.21.15.164:0/46803567 wait complete. 2024-09-19T08:49:39.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.644+0000 7fd1f4c53740 1 -- start start 2024-09-19T08:49:39.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.644+0000 7fd1f4c53740 1 Processor -- start 2024-09-19T08:49:39.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.644+0000 7fd1f4c53740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576b08cfb60 0x5576b08da800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:39.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.644+0000 7fd1f4c53740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576b08d0480 0x5576b08d7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:39.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.644+0000 7fd1f4c53740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576b08dad40 0x5576b08d82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:39.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.644+0000 7fd1f4c53740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5576b08dd910 con 0x5576b08dad40 2024-09-19T08:49:39.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.644+0000 7fd1f4c53740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5576b07eff10 con 0x5576b08cfb60 2024-09-19T08:49:39.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.644+0000 7fd1f4c53740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5576b08dda90 con 0x5576b08d0480 2024-09-19T08:49:39.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.644+0000 7fd1f4c53740 10 client.0 _ll_register_callbacks cb 0x5576b07b8a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:49:39.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.644+0000 7fd1eb7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576b08dad40 0x5576b08d82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:39.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.645+0000 7fd1eb7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576b08dad40 0x5576b08d82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36206/0 (socket says 172.21.15.164:36206) 2024-09-19T08:49:39.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.645+0000 7fd1eb7fe640 1 -- 172.21.15.164:0/1694965666 learned_addr learned my addr 172.21.15.164:0/1694965666 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:39.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.645+0000 7fd1ebfff640 1 --2- 172.21.15.164:0/1694965666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576b08d0480 0x5576b08d7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:39.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.645+0000 7fd1eb7fe640 1 --2- 172.21.15.164:0/1694965666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576b08cfb60 0x5576b08da800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:39.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.645+0000 7fd1eb7fe640 1 -- 172.21.15.164:0/1694965666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576b08d0480 msgr2=0x5576b08d7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:39.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.645+0000 7fd1eb7fe640 1 --2- 172.21.15.164:0/1694965666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576b08d0480 0x5576b08d7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:39.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.645+0000 7fd1eb7fe640 1 -- 172.21.15.164:0/1694965666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576b08cfb60 msgr2=0x5576b08da800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:39.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.645+0000 7fd1eb7fe640 1 --2- 172.21.15.164:0/1694965666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576b08cfb60 0x5576b08da800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:39.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.645+0000 7fd1eb7fe640 1 -- 172.21.15.164:0/1694965666 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576b05a66a0 con 0x5576b08dad40 2024-09-19T08:49:39.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.645+0000 7fd1ebfff640 1 --2- 172.21.15.164:0/1694965666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576b08d0480 0x5576b08d7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:49:39.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.645+0000 7fd1eb7fe640 1 --2- 172.21.15.164:0/1694965666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576b08dad40 0x5576b08d82e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000f640 tx=0x7fd1e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8adb990302c91dd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:39.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.645+0000 7fd1cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:49:39.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.645+0000 7fd1cb7fe640 1 -- 172.21.15.164:0/1694965666 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0004020 con 0x5576b08dad40 2024-09-19T08:49:39.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.645+0000 7fd1cb7fe640 1 -- 172.21.15.164:0/1694965666 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fd1e00041c0 con 0x5576b08dad40 2024-09-19T08:49:39.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.645+0000 7fd1f4c53740 1 -- 172.21.15.164:0/1694965666 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1dc013470 con 0x5576b08dad40 2024-09-19T08:49:39.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.645+0000 7fd1cb7fe640 1 -- 172.21.15.164:0/1694965666 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd1e00043c0 con 0x5576b08dad40 2024-09-19T08:49:39.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.645+0000 7fd1f4c53740 10 client.16311 fetch_fsmap learned FSMap version 28 2024-09-19T08:49:39.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.645+0000 7fd1f4c53740 1 -- 172.21.15.164:0/1694965666 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5576b08e94c0 con 0x5576b08dad40 2024-09-19T08:49:39.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.646+0000 7fd1cb7fe640 1 -- 172.21.15.164:0/1694965666 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1dc013470 con 0x5576b08dad40 2024-09-19T08:49:39.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.646+0000 7fd1cb7fe640 1 -- 172.21.15.164:0/1694965666 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd1e0020040 con 0x5576b08dad40 2024-09-19T08:49:39.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.646+0000 7fd1f4c53740 10 client.16311 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:49:39.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.646+0000 7fd1f4c53740 10 client.16311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:49:39.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.646+0000 7fd1f4c53740 20 client.16311 populate_metadata read hostname 'smithi164' 2024-09-19T08:49:39.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.646+0000 7fd1f4c53740 1 --2- 172.21.15.164:0/1694965666 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5576b08ea980 0x5576b08ecdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:39.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.646+0000 7fd1f4c53740 4 client.16311 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:49:39.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.646+0000 7fd1f4c53740 1 -- 172.21.15.164:0/1694965666 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576b08427f0 con 0x5576b08ea980 2024-09-19T08:49:39.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.646+0000 7fd1ebfff640 1 --2- 172.21.15.164:0/1694965666 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5576b08ea980 0x5576b08ecdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:39.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.646+0000 7fd1cb7fe640 10 client.16311 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:49:39.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.646+0000 7fd1ebfff640 1 --2- 172.21.15.164:0/1694965666 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5576b08ea980 0x5576b08ecdb0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d12f7eb34593e173 server_cookie=881ac1a108d3b2b1 in_seq=0 out_seq=0 2024-09-19T08:49:39.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.647+0000 7fd1cb7fe640 1 -- 172.21.15.164:0/1694965666 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5576b08427f0 con 0x5576b08ea980 2024-09-19T08:49:39.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.647+0000 7fd1cb7fe640 10 client.16311 handle_command_reply: tid=0 2024-09-19T08:49:39.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.647+0000 7fd1f4c53740 1 client.16311 shutdown 2024-09-19T08:49:39.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.647+0000 7fd1f4c53740 1 -- 172.21.15.164:0/1694965666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576b08dad40 msgr2=0x5576b08d82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:39.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.647+0000 7fd1f4c53740 1 --2- 172.21.15.164:0/1694965666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576b08dad40 0x5576b08d82e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000f640 tx=0x7fd1e00046c0 comp rx=0 tx=0).stop 2024-09-19T08:49:39.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.647+0000 7fd1f4c53740 1 -- 172.21.15.164:0/1694965666 shutdown_connections 2024-09-19T08:49:39.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.647+0000 7fd1f4c53740 1 --2- 172.21.15.164:0/1694965666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576b08dad40 0x5576b08d82e0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:39.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.647+0000 7fd1f4c53740 1 --2- 172.21.15.164:0/1694965666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576b08d0480 0x5576b08d7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:39.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.647+0000 7fd1f4c53740 1 --2- 172.21.15.164:0/1694965666 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5576b08ea980 0x5576b08ecdb0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:39.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.647+0000 7fd1f4c53740 1 --2- 172.21.15.164:0/1694965666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576b08cfb60 0x5576b08da800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:39.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.647+0000 7fd1f4c53740 1 -- 172.21.15.164:0/1694965666 >> 172.21.15.164:0/1694965666 conn(0x5576b08cc0f0 msgr2=0x5576b08cb270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:39.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.647+0000 7fd1cb7fe640 0 client.16311 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:49:39.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.647+0000 7fd1f4c53740 1 -- 172.21.15.164:0/1694965666 shutdown_connections 2024-09-19T08:49:39.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.647+0000 7fd1f4c53740 1 -- 172.21.15.164:0/1694965666 wait complete. 2024-09-19T08:49:39.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.647+0000 7fd1f4c53740 20 client.16311 trim_cache size 0 max 16384 2024-09-19T08:49:39.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.665+0000 7fd1f4c53740 1 Processor -- start 2024-09-19T08:49:39.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.665+0000 7fd1f4c53740 1 -- start start 2024-09-19T08:49:39.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.665+0000 7fd1f4c53740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576b08cfb60 0x5576b08d6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:39.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.665+0000 7fd1f4c53740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576b08d0480 0x5576b08d6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:39.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.665+0000 7fd1f4c53740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576b08dad40 0x5576b08f2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:39.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.665+0000 7fd1f4c53740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec1ad640 con 0x5576b08dad40 2024-09-19T08:49:39.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.665+0000 7fd1f4c53740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec1af180 con 0x5576b08d0480 2024-09-19T08:49:39.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.665+0000 7fd1f4c53740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec1add10 con 0x5576b08cfb60 2024-09-19T08:49:39.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.665+0000 7fd1eb7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576b08dad40 0x5576b08f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:39.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.665+0000 7fd1ebfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576b08d0480 0x5576b08d6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:39.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.665+0000 7fd1ebfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576b08d0480 0x5576b08d6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:44146/0 (socket says 172.21.15.164:44146) 2024-09-19T08:49:39.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.665+0000 7fd1ebfff640 1 -- 172.21.15.164:0/2548406166 learned_addr learned my addr 172.21.15.164:0/2548406166 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:39.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.665+0000 7fd1eb7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576b08dad40 0x5576b08f2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36216/0 (socket says 172.21.15.164:36216) 2024-09-19T08:49:39.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.665+0000 7fd1eb7fe640 1 --2- 172.21.15.164:0/2548406166 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576b08cfb60 0x5576b08d6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:39.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.665+0000 7fd1eb7fe640 1 -- 172.21.15.164:0/2548406166 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576b08cfb60 msgr2=0x5576b08d6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:39.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.665+0000 7fd1eb7fe640 1 --2- 172.21.15.164:0/2548406166 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576b08cfb60 0x5576b08d6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:39.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.665+0000 7fd1eb7fe640 1 -- 172.21.15.164:0/2548406166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576b08d0480 msgr2=0x5576b08d6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:39.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.665+0000 7fd1eb7fe640 1 --2- 172.21.15.164:0/2548406166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576b08d0480 0x5576b08d6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:39.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.665+0000 7fd1eb7fe640 1 -- 172.21.15.164:0/2548406166 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576b08d02b0 con 0x5576b08dad40 2024-09-19T08:49:39.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.665+0000 7fd1ebfff640 1 --2- 172.21.15.164:0/2548406166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576b08d0480 0x5576b08d6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:49:39.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.666+0000 7fd1eb7fe640 1 --2- 172.21.15.164:0/2548406166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576b08dad40 0x5576b08f2680 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0011bf0 tx=0x7fd1e0001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=266d8f2eabf0d9a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:39.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.666+0000 7fd1d1ffb640 1 -- 172.21.15.164:0/2548406166 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0004730 con 0x5576b08dad40 2024-09-19T08:49:39.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.666+0000 7fd1d1ffb640 1 -- 172.21.15.164:0/2548406166 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1e00048d0 con 0x5576b08dad40 2024-09-19T08:49:39.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.666+0000 7fd1d1ffb640 1 -- 172.21.15.164:0/2548406166 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0005520 con 0x5576b08dad40 2024-09-19T08:49:39.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.666+0000 7fd1f4c53740 1 -- 172.21.15.164:0/2548406166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576b08dad40 msgr2=0x5576b08f2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:39.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.666+0000 7fd1f4c53740 1 --2- 172.21.15.164:0/2548406166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576b08dad40 0x5576b08f2680 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0011bf0 tx=0x7fd1e0001e50 comp rx=0 tx=0).stop 2024-09-19T08:49:39.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.666+0000 7fd1f4c53740 1 -- 172.21.15.164:0/2548406166 shutdown_connections 2024-09-19T08:49:39.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.666+0000 7fd1f4c53740 1 --2- 172.21.15.164:0/2548406166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576b08dad40 0x5576b08f2680 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:39.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.666+0000 7fd1f4c53740 1 --2- 172.21.15.164:0/2548406166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576b08d0480 0x5576b08d6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:39.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.666+0000 7fd1f4c53740 1 --2- 172.21.15.164:0/2548406166 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576b08cfb60 0x5576b08d6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:39.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.666+0000 7fd1f4c53740 1 -- 172.21.15.164:0/2548406166 >> 172.21.15.164:0/2548406166 conn(0x5576b08cc0f0 msgr2=0x5576b08d2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:39.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.667+0000 7fd1f4c53740 1 -- 172.21.15.164:0/2548406166 shutdown_connections 2024-09-19T08:49:39.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.667+0000 7fd1f4c53740 1 -- 172.21.15.164:0/2548406166 wait complete. 2024-09-19T08:49:39.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.667+0000 7fd1f4c53740 1 -- start start 2024-09-19T08:49:39.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.667+0000 7fd1f4c53740 1 Processor -- start 2024-09-19T08:49:39.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.667+0000 7fd1f4c53740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576b08cfb60 0x5576b08e32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:39.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.667+0000 7fd1f4c53740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576b08d0480 0x5576b08de5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:39.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.667+0000 7fd1f4c53740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576b08dad40 0x5576b08deb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:39.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.667+0000 7fd1f4c53740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec1aeab0 con 0x5576b08cfb60 2024-09-19T08:49:39.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.667+0000 7fd1f4c53740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec1ae3e0 con 0x5576b08d0480 2024-09-19T08:49:39.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.667+0000 7fd1f4c53740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec1ab640 con 0x5576b08dad40 2024-09-19T08:49:39.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.667+0000 7fd1f4c53740 10 client.0 _ll_register_callbacks cb 0x7fd1e000ac50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:49:39.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.667+0000 7fd1eb7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576b08cfb60 0x5576b08e32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:39.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.668+0000 7fd1eb7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576b08cfb60 0x5576b08e32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36224/0 (socket says 172.21.15.164:36224) 2024-09-19T08:49:39.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.668+0000 7fd1eb7fe640 1 -- 172.21.15.164:0/4031735980 learned_addr learned my addr 172.21.15.164:0/4031735980 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:39.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.668+0000 7fd1ebfff640 1 --2- 172.21.15.164:0/4031735980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576b08d0480 0x5576b08de5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:39.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.668+0000 7fd1eb7fe640 1 -- 172.21.15.164:0/4031735980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576b08dad40 msgr2=0x5576b08deb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:39.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.668+0000 7fd1eb7fe640 1 --2- 172.21.15.164:0/4031735980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576b08dad40 0x5576b08deb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:39.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.668+0000 7fd1eb7fe640 1 -- 172.21.15.164:0/4031735980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576b08d0480 msgr2=0x5576b08de5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:39.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.668+0000 7fd1eb7fe640 1 --2- 172.21.15.164:0/4031735980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576b08d0480 0x5576b08de5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:39.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.668+0000 7fd1eb7fe640 1 -- 172.21.15.164:0/4031735980 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576b08df150 con 0x5576b08cfb60 2024-09-19T08:49:39.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.668+0000 7fd1eb7fe640 1 --2- 172.21.15.164:0/4031735980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576b08cfb60 0x5576b08e32a0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd1e00058c0 tx=0x7fd1e000a350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70d56389cfd94f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:39.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.668+0000 7fd1cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:49:39.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.668+0000 7fd1cb7fe640 1 -- 172.21.15.164:0/4031735980 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e003c060 con 0x5576b08cfb60 2024-09-19T08:49:39.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.668+0000 7fd1cb7fe640 1 -- 172.21.15.164:0/4031735980 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fd1e0005480 con 0x5576b08cfb60 2024-09-19T08:49:39.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.668+0000 7fd1f4c53740 1 -- 172.21.15.164:0/4031735980 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1e00048d0 con 0x5576b08cfb60 2024-09-19T08:49:39.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.668+0000 7fd1cb7fe640 1 -- 172.21.15.164:0/4031735980 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd1e0011040 con 0x5576b08cfb60 2024-09-19T08:49:39.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.668+0000 7fd1f4c53740 10 client.16317 fetch_fsmap learned FSMap version 28 2024-09-19T08:49:39.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.668+0000 7fd1f4c53740 1 -- 172.21.15.164:0/4031735980 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5576b0955450 con 0x5576b08cfb60 2024-09-19T08:49:39.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.668+0000 7fd1cb7fe640 1 -- 172.21.15.164:0/4031735980 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1e00048d0 con 0x5576b08cfb60 2024-09-19T08:49:39.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.669+0000 7fd1cb7fe640 1 -- 172.21.15.164:0/4031735980 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd1e003f020 con 0x5576b08cfb60 2024-09-19T08:49:39.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.669+0000 7fd1f4c53740 10 client.16317 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:49:39.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.669+0000 7fd1f4c53740 10 client.16317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:49:39.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.669+0000 7fd1f4c53740 20 client.16317 populate_metadata read hostname 'smithi164' 2024-09-19T08:49:39.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.669+0000 7fd1f4c53740 1 --2- 172.21.15.164:0/4031735980 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5576b0955ac0 0x5576b0957ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:39.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.669+0000 7fd1f4c53740 4 client.16317 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T08:49:39.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.669+0000 7fd1f4c53740 1 -- 172.21.15.164:0/4031735980 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5576b083cd00 con 0x5576b0955ac0 2024-09-19T08:49:39.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.669+0000 7fd1ebfff640 1 --2- 172.21.15.164:0/4031735980 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5576b0955ac0 0x5576b0957ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:39.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.669+0000 7fd1ebfff640 1 --2- 172.21.15.164:0/4031735980 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5576b0955ac0 0x5576b0957ee0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b324b49d110b5154 server_cookie=dc3789b113871430 in_seq=0 out_seq=0 2024-09-19T08:49:39.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.669+0000 7fd1cb7fe640 10 client.16317 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:49:39.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.670+0000 7fd1cb7fe640 1 -- 172.21.15.164:0/4031735980 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5576b083cd00 con 0x5576b0955ac0 2024-09-19T08:49:39.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.670+0000 7fd1cb7fe640 10 client.16317 handle_command_reply: tid=0 2024-09-19T08:49:39.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.670+0000 7fd1f4c53740 1 client.16317 shutdown 2024-09-19T08:49:39.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.670+0000 7fd1f4c53740 1 -- 172.21.15.164:0/4031735980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576b08cfb60 msgr2=0x5576b08e32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:39.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.670+0000 7fd1f4c53740 1 --2- 172.21.15.164:0/4031735980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576b08cfb60 0x5576b08e32a0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd1e00058c0 tx=0x7fd1e000a350 comp rx=0 tx=0).stop 2024-09-19T08:49:39.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.671+0000 7fd1f4c53740 1 -- 172.21.15.164:0/4031735980 shutdown_connections 2024-09-19T08:49:39.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.671+0000 7fd1f4c53740 1 --2- 172.21.15.164:0/4031735980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576b08dad40 0x5576b08deb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:39.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.671+0000 7fd1f4c53740 1 --2- 172.21.15.164:0/4031735980 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5576b0955ac0 0x5576b0957ee0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:39.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.671+0000 7fd1f4c53740 1 --2- 172.21.15.164:0/4031735980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576b08d0480 0x5576b08de5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:39.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.671+0000 7fd1f4c53740 1 --2- 172.21.15.164:0/4031735980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576b08cfb60 0x5576b08e32a0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:39.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.671+0000 7fd1cb7fe640 0 client.16317 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:49:39.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.671+0000 7fd1f4c53740 1 -- 172.21.15.164:0/4031735980 >> 172.21.15.164:0/4031735980 conn(0x5576b08cc0f0 msgr2=0x5576b08d63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:39.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.671+0000 7fd1f4c53740 1 -- 172.21.15.164:0/4031735980 shutdown_connections 2024-09-19T08:49:39.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.671+0000 7fd1f4c53740 1 -- 172.21.15.164:0/4031735980 wait complete. 2024-09-19T08:49:39.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.671+0000 7fd1f4c53740 20 client.16317 trim_cache size 0 max 16384 2024-09-19T08:49:39.679 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:49:39.680 INFO:teuthology.orchestra.run.smithi164.stdout: "return_code": 0, 2024-09-19T08:49:39.680 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_tag": "0b935302-e641-4de9-a7b9-9f1fd5fdca64", 2024-09-19T08:49:39.680 INFO:teuthology.orchestra.run.smithi164.stdout: "mode": "asynchronous" 2024-09-19T08:49:39.680 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:49:39.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.672+0000 7fd1f287a640 1 -- 172.21.15.164:0/3843905046 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd1d40a33b0 msgr2=0x7fd1d40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:39.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.672+0000 7fd1f287a640 1 --2- 172.21.15.164:0/3843905046 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd1d40a33b0 0x7fd1d40a5870 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd1e40b9d30 tx=0x7fd1e4088210 comp rx=0 tx=0).stop 2024-09-19T08:49:39.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.672+0000 7fd1f287a640 1 -- 172.21.15.164:0/3843905046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1ec106a80 msgr2=0x7fd1ec19cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:39.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.672+0000 7fd1f287a640 1 --2- 172.21.15.164:0/3843905046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1ec106a80 0x7fd1ec19cd40 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4006010 tx=0x7fd1e4003c20 comp rx=0 tx=0).stop 2024-09-19T08:49:39.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.672+0000 7fd1f287a640 1 -- 172.21.15.164:0/3843905046 shutdown_connections 2024-09-19T08:49:39.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.672+0000 7fd1f287a640 1 --2- 172.21.15.164:0/3843905046 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd1d40a33b0 0x7fd1d40a5870 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:39.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.672+0000 7fd1f287a640 1 --2- 172.21.15.164:0/3843905046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1ec106a80 0x7fd1ec19cd40 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:39.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.672+0000 7fd1f287a640 1 --2- 172.21.15.164:0/3843905046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1ec101420 0x7fd1ec1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:39.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.672+0000 7fd1f287a640 1 --2- 172.21.15.164:0/3843905046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1ec100a80 0x7fd1ec198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:39.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.672+0000 7fd1f287a640 1 -- 172.21.15.164:0/3843905046 >> 172.21.15.164:0/3843905046 conn(0x7fd1ec0fc7a0 msgr2=0x7fd1ec06a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:39.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.672+0000 7fd1f287a640 1 -- 172.21.15.164:0/3843905046 shutdown_connections 2024-09-19T08:49:39.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:39.672+0000 7fd1f287a640 1 -- 172.21.15.164:0/3843905046 wait complete. 2024-09-19T08:49:39.705 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:39 smithi164 ceph-mon[30928]: pgmap v118: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 653 MiB/s wr, 1.48k op/s 2024-09-19T08:49:39.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:39 smithi173 ceph-mon[35306]: pgmap v118: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 653 MiB/s wr, 1.48k op/s 2024-09-19T08:49:40.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:39 smithi170 ceph-mon[35072]: pgmap v118: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 653 MiB/s wr, 1.48k op/s 2024-09-19T08:49:40.493 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1bb05f3d', quiesce took 14.6 seconds. Will release after: 2.0000000000000018 2024-09-19T08:49:40.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:40 smithi164 ceph-mon[30928]: scrub queued for path: / 2024-09-19T08:49:40.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:40 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:49:40.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:40 smithi164 ceph-mon[30928]: scrub summary: active paths [/] 2024-09-19T08:49:40.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:40 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:49:40.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:40 smithi173 ceph-mon[35306]: scrub queued for path: / 2024-09-19T08:49:40.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:40 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:49:40.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:40 smithi173 ceph-mon[35306]: scrub summary: active paths [/] 2024-09-19T08:49:40.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:40 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:49:41.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:40 smithi170 ceph-mon[35072]: scrub queued for path: / 2024-09-19T08:49:41.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:40 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:49:41.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:40 smithi170 ceph-mon[35072]: scrub summary: active paths [/] 2024-09-19T08:49:41.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:40 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:49:41.204 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:49:41.457 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:49:41.875 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:41 smithi164 ceph-mon[30928]: pgmap v119: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 878 MiB/s wr, 2.59k op/s 2024-09-19T08:49:41.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:41 smithi173 ceph-mon[35306]: pgmap v119: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 878 MiB/s wr, 2.59k op/s 2024-09-19T08:49:41.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.957+0000 7fe608e3b640 1 -- 172.21.15.164:0/1965179901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe60410c170 msgr2=0x7fe60410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:41.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.957+0000 7fe608e3b640 1 --2- 172.21.15.164:0/1965179901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe60410c170 0x7fe60410e610 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc009ca0 tx=0x7fe5fc02a950 comp rx=0 tx=0).stop 2024-09-19T08:49:41.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.957+0000 7fe608e3b640 1 -- 172.21.15.164:0/1965179901 shutdown_connections 2024-09-19T08:49:41.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.957+0000 7fe608e3b640 1 --2- 172.21.15.164:0/1965179901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe60410c170 0x7fe60410e610 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:41.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.957+0000 7fe608e3b640 1 --2- 172.21.15.164:0/1965179901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe604103660 0x7fe60410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:41.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.957+0000 7fe608e3b640 1 --2- 172.21.15.164:0/1965179901 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe604102d40 0x7fe604103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:41.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.957+0000 7fe608e3b640 1 -- 172.21.15.164:0/1965179901 >> 172.21.15.164:0/1965179901 conn(0x7fe6040fc7a0 msgr2=0x7fe6040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:41.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.957+0000 7fe608e3b640 1 -- 172.21.15.164:0/1965179901 shutdown_connections 2024-09-19T08:49:41.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.957+0000 7fe608e3b640 1 -- 172.21.15.164:0/1965179901 wait complete. 2024-09-19T08:49:41.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.958+0000 7fe608e3b640 1 Processor -- start 2024-09-19T08:49:41.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.958+0000 7fe608e3b640 1 -- start start 2024-09-19T08:49:41.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.958+0000 7fe608e3b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe604102d40 0x7fe6041983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:41.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.958+0000 7fe608e3b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe604103660 0x7fe604198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:41.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.958+0000 7fe608e3b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe60410c170 0x7fe60419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:41.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.958+0000 7fe608e3b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe604111f70 con 0x7fe60410c170 2024-09-19T08:49:41.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.958+0000 7fe608e3b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe604111df0 con 0x7fe604102d40 2024-09-19T08:49:41.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.958+0000 7fe608e3b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe6041120f0 con 0x7fe604103660 2024-09-19T08:49:41.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.958+0000 7fe602d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe60410c170 0x7fe60419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:41.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.958+0000 7fe602d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe60410c170 0x7fe60419cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36254/0 (socket says 172.21.15.164:36254) 2024-09-19T08:49:41.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.958+0000 7fe602d9d640 1 -- 172.21.15.164:0/3720894595 learned_addr learned my addr 172.21.15.164:0/3720894595 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:41.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.958+0000 7fe60259c640 1 --2- 172.21.15.164:0/3720894595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe604102d40 0x7fe6041983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:41.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.958+0000 7fe601d9b640 1 --2- 172.21.15.164:0/3720894595 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe604103660 0x7fe604198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:41.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.958+0000 7fe602d9d640 1 -- 172.21.15.164:0/3720894595 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe604103660 msgr2=0x7fe604198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:41.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.958+0000 7fe602d9d640 1 --2- 172.21.15.164:0/3720894595 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe604103660 0x7fe604198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:41.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.958+0000 7fe602d9d640 1 -- 172.21.15.164:0/3720894595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe604102d40 msgr2=0x7fe6041983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:41.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.958+0000 7fe602d9d640 1 --2- 172.21.15.164:0/3720894595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe604102d40 0x7fe6041983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:41.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.958+0000 7fe602d9d640 1 -- 172.21.15.164:0/3720894595 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe60419d380 con 0x7fe60410c170 2024-09-19T08:49:41.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.959+0000 7fe602d9d640 1 --2- 172.21.15.164:0/3720894595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe60410c170 0x7fe60419cca0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc004af0 tx=0x7fe5fc02f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:41.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.959+0000 7fe5f37fe640 1 -- 172.21.15.164:0/3720894595 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc034040 con 0x7fe60410c170 2024-09-19T08:49:41.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.959+0000 7fe608e3b640 1 -- 172.21.15.164:0/3720894595 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe60419d610 con 0x7fe60410c170 2024-09-19T08:49:41.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.959+0000 7fe5f37fe640 1 -- 172.21.15.164:0/3720894595 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5fc030920 con 0x7fe60410c170 2024-09-19T08:49:41.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.959+0000 7fe5f37fe640 1 -- 172.21.15.164:0/3720894595 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc0047b0 con 0x7fe60410c170 2024-09-19T08:49:41.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.959+0000 7fe608e3b640 1 -- 172.21.15.164:0/3720894595 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6041a4f50 con 0x7fe60410c170 2024-09-19T08:49:41.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.961+0000 7fe5f37fe640 1 -- 172.21.15.164:0/3720894595 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fe5fc037020 con 0x7fe60410c170 2024-09-19T08:49:41.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.961+0000 7fe60b214740 1 Processor -- start 2024-09-19T08:49:41.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.962+0000 7fe60b214740 1 -- start start 2024-09-19T08:49:41.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.962+0000 7fe60b214740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55576e9314a0 0x55576e931880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:41.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.962+0000 7fe60b214740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576e931dc0 0x55576e93c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:41.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.962+0000 7fe60b214740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55576e93c680 0x55576e93ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:41.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.962+0000 7fe60b214740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55576e8972e0 con 0x55576e931dc0 2024-09-19T08:49:41.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.962+0000 7fe60b214740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55576e83ece0 con 0x55576e93c680 2024-09-19T08:49:41.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.962+0000 7fe60b214740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55576e8932a0 con 0x55576e9314a0 2024-09-19T08:49:41.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.962+0000 7fe60259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576e931dc0 0x55576e93c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:41.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.962+0000 7fe60259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576e931dc0 0x55576e93c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36270/0 (socket says 172.21.15.164:36270) 2024-09-19T08:49:41.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.962+0000 7fe60259c640 1 -- 172.21.15.164:0/2150712275 learned_addr learned my addr 172.21.15.164:0/2150712275 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:41.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.962+0000 7fe5f37fe640 1 --2- 172.21.15.164:0/3720894595 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe5ec0a33b0 0x7fe5ec0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:41.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.962+0000 7fe601d9b640 1 --2- 172.21.15.164:0/2150712275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55576e93c680 0x55576e93ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:41.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.962+0000 7fe601d9b640 1 --2- 172.21.15.164:0/2150712275 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55576e9314a0 0x55576e931880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:41.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.962+0000 7fe60259c640 1 -- 172.21.15.164:0/2150712275 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55576e9314a0 msgr2=0x55576e931880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:41.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.962+0000 7fe60259c640 1 --2- 172.21.15.164:0/2150712275 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55576e9314a0 0x55576e931880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:41.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.962+0000 7fe60259c640 1 -- 172.21.15.164:0/2150712275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55576e93c680 msgr2=0x55576e93ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:41.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.962+0000 7fe60259c640 1 --2- 172.21.15.164:0/2150712275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55576e93c680 0x55576e93ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:41.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.962+0000 7fe60259c640 1 -- 172.21.15.164:0/2150712275 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55576e823440 con 0x55576e931dc0 2024-09-19T08:49:41.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.962+0000 7fe601d9b640 1 --2- 172.21.15.164:0/2150712275 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55576e9314a0 0x55576e931880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:49:41.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.962+0000 7fe602d9d640 1 --2- 172.21.15.164:0/3720894595 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe5ec0a33b0 0x7fe5ec0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:41.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.962+0000 7fe5f37fe640 1 -- 172.21.15.164:0/3720894595 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fe5fc0f0960 con 0x7fe60410c170 2024-09-19T08:49:41.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.962+0000 7fe60259c640 1 --2- 172.21.15.164:0/2150712275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576e931dc0 0x55576e93c140 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4009cf0 tx=0x7fe5f4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbaf9341991f2b62 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:41.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.963+0000 7fe5f0ff9640 1 -- 172.21.15.164:0/2150712275 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4004230 con 0x55576e931dc0 2024-09-19T08:49:41.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.963+0000 7fe5f0ff9640 1 -- 172.21.15.164:0/2150712275 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5f40043d0 con 0x55576e931dc0 2024-09-19T08:49:41.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.963+0000 7fe5f0ff9640 1 -- 172.21.15.164:0/2150712275 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4019960 con 0x55576e931dc0 2024-09-19T08:49:41.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.963+0000 7fe60b214740 1 -- 172.21.15.164:0/2150712275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576e931dc0 msgr2=0x55576e93c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:41.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.963+0000 7fe602d9d640 1 --2- 172.21.15.164:0/3720894595 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe5ec0a33b0 0x7fe5ec0a5870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc0bca60 tx=0x7fe5fc004250 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:41.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.963+0000 7fe60b214740 1 --2- 172.21.15.164:0/2150712275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576e931dc0 0x55576e93c140 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4009cf0 tx=0x7fe5f4011810 comp rx=0 tx=0).stop 2024-09-19T08:49:41.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.963+0000 7fe60b214740 1 -- 172.21.15.164:0/2150712275 shutdown_connections 2024-09-19T08:49:41.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.963+0000 7fe60b214740 1 --2- 172.21.15.164:0/2150712275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55576e93c680 0x55576e93ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:41.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.963+0000 7fe60b214740 1 --2- 172.21.15.164:0/2150712275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576e931dc0 0x55576e93c140 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:41.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.963+0000 7fe60b214740 1 --2- 172.21.15.164:0/2150712275 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55576e9314a0 0x55576e931880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:41.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.963+0000 7fe60b214740 1 -- 172.21.15.164:0/2150712275 >> 172.21.15.164:0/2150712275 conn(0x55576e92da30 msgr2=0x55576e92de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:41.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.964+0000 7fe60b214740 1 -- 172.21.15.164:0/2150712275 shutdown_connections 2024-09-19T08:49:41.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.964+0000 7fe60b214740 1 -- 172.21.15.164:0/2150712275 wait complete. 2024-09-19T08:49:41.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.964+0000 7fe60b214740 1 -- start start 2024-09-19T08:49:41.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.964+0000 7fe60b214740 1 Processor -- start 2024-09-19T08:49:41.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.964+0000 7fe60b214740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55576e9314a0 0x55576e93bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:41.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.964+0000 7fe60b214740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576e931dc0 0x55576e939220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:41.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.964+0000 7fe60b214740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55576e93c680 0x55576e939760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:41.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.964+0000 7fe60b214740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55576e93f250 con 0x55576e931dc0 2024-09-19T08:49:41.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.964+0000 7fe60b214740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55576e852050 con 0x55576e9314a0 2024-09-19T08:49:41.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.964+0000 7fe60b214740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55576e93f3d0 con 0x55576e93c680 2024-09-19T08:49:41.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.964+0000 7fe60b214740 10 client.0 _ll_register_callbacks cb 0x55576e892160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:49:41.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.964+0000 7fe60259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576e931dc0 0x55576e939220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:41.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.964+0000 7fe60259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576e931dc0 0x55576e939220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36286/0 (socket says 172.21.15.164:36286) 2024-09-19T08:49:41.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.964+0000 7fe60259c640 1 -- 172.21.15.164:0/4126848093 learned_addr learned my addr 172.21.15.164:0/4126848093 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:41.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.964+0000 7fe601d9b640 1 --2- 172.21.15.164:0/4126848093 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55576e9314a0 0x55576e93bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:41.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.965+0000 7fe60259c640 1 -- 172.21.15.164:0/4126848093 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55576e93c680 msgr2=0x55576e939760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:41.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.965+0000 7fe60259c640 1 --2- 172.21.15.164:0/4126848093 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55576e93c680 0x55576e939760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:41.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.965+0000 7fe60259c640 1 -- 172.21.15.164:0/4126848093 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55576e9314a0 msgr2=0x55576e93bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:41.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.965+0000 7fe60259c640 1 --2- 172.21.15.164:0/4126848093 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55576e9314a0 0x55576e93bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:41.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.965+0000 7fe60259c640 1 -- 172.21.15.164:0/4126848093 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55576e6083c0 con 0x55576e931dc0 2024-09-19T08:49:41.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.965+0000 7fe601d9b640 1 --2- 172.21.15.164:0/4126848093 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55576e9314a0 0x55576e93bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:49:41.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.965+0000 7fe60259c640 1 --2- 172.21.15.164:0/4126848093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576e931dc0 0x55576e939220 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4009730 tx=0x7fe5f4003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=215ed57e2ca9d5f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:41.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.965+0000 7fe5e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:49:41.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.965+0000 7fe5e9ffb640 1 -- 172.21.15.164:0/4126848093 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4027070 con 0x55576e931dc0 2024-09-19T08:49:41.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.965+0000 7fe5e9ffb640 1 -- 172.21.15.164:0/4126848093 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fe5f4005240 con 0x55576e931dc0 2024-09-19T08:49:41.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.965+0000 7fe60b214740 1 -- 172.21.15.164:0/4126848093 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5f40043d0 con 0x55576e931dc0 2024-09-19T08:49:41.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.965+0000 7fe5e9ffb640 1 -- 172.21.15.164:0/4126848093 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe5f4033050 con 0x55576e931dc0 2024-09-19T08:49:41.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.965+0000 7fe60b214740 10 client.16329 fetch_fsmap learned FSMap version 28 2024-09-19T08:49:41.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.965+0000 7fe60b214740 1 -- 172.21.15.164:0/4126848093 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55576e94ab30 con 0x55576e931dc0 2024-09-19T08:49:41.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.965+0000 7fe5e9ffb640 1 -- 172.21.15.164:0/4126848093 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5f40043d0 con 0x55576e931dc0 2024-09-19T08:49:41.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.965+0000 7fe5e9ffb640 1 -- 172.21.15.164:0/4126848093 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe5f4004bc0 con 0x55576e931dc0 2024-09-19T08:49:41.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.965+0000 7fe60b214740 10 client.16329 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:49:41.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.965+0000 7fe60b214740 10 client.16329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:49:41.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.966+0000 7fe60b214740 20 client.16329 populate_metadata read hostname 'smithi164' 2024-09-19T08:49:41.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.966+0000 7fe60b214740 1 --2- 172.21.15.164:0/4126848093 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55576e94bee0 0x55576e94e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:41.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.966+0000 7fe60b214740 4 client.16329 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:49:41.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.966+0000 7fe60b214740 1 -- 172.21.15.164:0/4126848093 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55576e8a4850 con 0x55576e94bee0 2024-09-19T08:49:41.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.966+0000 7fe601d9b640 1 --2- 172.21.15.164:0/4126848093 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55576e94bee0 0x55576e94e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:41.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.966+0000 7fe601d9b640 1 --2- 172.21.15.164:0/4126848093 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55576e94bee0 0x55576e94e2d0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64114883451badc1 server_cookie=37581c3215e9091 in_seq=0 out_seq=0 2024-09-19T08:49:41.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.966+0000 7fe5e9ffb640 10 client.16329 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:49:41.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.967+0000 7fe5e9ffb640 1 -- 172.21.15.164:0/4126848093 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55576e8a4850 con 0x55576e94bee0 2024-09-19T08:49:41.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.967+0000 7fe5e9ffb640 10 client.16329 handle_command_reply: tid=0 2024-09-19T08:49:41.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.967+0000 7fe60b214740 1 client.16329 shutdown 2024-09-19T08:49:41.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.967+0000 7fe60b214740 1 -- 172.21.15.164:0/4126848093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576e931dc0 msgr2=0x55576e939220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:41.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.967+0000 7fe60b214740 1 --2- 172.21.15.164:0/4126848093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576e931dc0 0x55576e939220 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4009730 tx=0x7fe5f4003820 comp rx=0 tx=0).stop 2024-09-19T08:49:41.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.967+0000 7fe60b214740 1 -- 172.21.15.164:0/4126848093 shutdown_connections 2024-09-19T08:49:41.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.967+0000 7fe60b214740 1 --2- 172.21.15.164:0/4126848093 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55576e93c680 0x55576e939760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:41.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.967+0000 7fe60b214740 1 --2- 172.21.15.164:0/4126848093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576e931dc0 0x55576e939220 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:41.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.967+0000 7fe60b214740 1 --2- 172.21.15.164:0/4126848093 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55576e94bee0 0x55576e94e2d0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:41.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.967+0000 7fe5e9ffb640 0 client.16329 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:49:41.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.967+0000 7fe60b214740 1 --2- 172.21.15.164:0/4126848093 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55576e9314a0 0x55576e93bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:41.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.967+0000 7fe60b214740 1 -- 172.21.15.164:0/4126848093 >> 172.21.15.164:0/4126848093 conn(0x55576e92da30 msgr2=0x55576e933710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:41.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.967+0000 7fe60b214740 1 -- 172.21.15.164:0/4126848093 shutdown_connections 2024-09-19T08:49:41.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.967+0000 7fe60b214740 1 -- 172.21.15.164:0/4126848093 wait complete. 2024-09-19T08:49:41.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.967+0000 7fe60b214740 20 client.16329 trim_cache size 0 max 16384 2024-09-19T08:49:41.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.984+0000 7fe60b214740 1 Processor -- start 2024-09-19T08:49:41.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.984+0000 7fe60b214740 1 -- start start 2024-09-19T08:49:41.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.985+0000 7fe60b214740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55576e9314a0 0x55576e937af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:41.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.985+0000 7fe60b214740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576e931dc0 0x55576e953ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:41.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.985+0000 7fe60b214740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55576e93c680 0x55576e954020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:41.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.985+0000 7fe60b214740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe6041ad630 con 0x55576e931dc0 2024-09-19T08:49:41.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.985+0000 7fe60b214740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe6041af170 con 0x55576e93c680 2024-09-19T08:49:41.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.985+0000 7fe60b214740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe6041add00 con 0x55576e9314a0 2024-09-19T08:49:41.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.985+0000 7fe60259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576e931dc0 0x55576e953ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:41.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.985+0000 7fe60259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576e931dc0 0x55576e953ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36294/0 (socket says 172.21.15.164:36294) 2024-09-19T08:49:41.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.985+0000 7fe60259c640 1 -- 172.21.15.164:0/1491228716 learned_addr learned my addr 172.21.15.164:0/1491228716 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:41.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.985+0000 7fe601d9b640 1 --2- 172.21.15.164:0/1491228716 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55576e9314a0 0x55576e937af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:41.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.985+0000 7fe601d9b640 1 --2- 172.21.15.164:0/1491228716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55576e93c680 0x55576e954020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:41.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.985+0000 7fe60259c640 1 -- 172.21.15.164:0/1491228716 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55576e9314a0 msgr2=0x55576e937af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:41.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.985+0000 7fe60259c640 1 --2- 172.21.15.164:0/1491228716 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55576e9314a0 0x55576e937af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:41.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.985+0000 7fe60259c640 1 -- 172.21.15.164:0/1491228716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55576e93c680 msgr2=0x55576e954020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:41.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.985+0000 7fe60259c640 1 --2- 172.21.15.164:0/1491228716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55576e93c680 0x55576e954020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:41.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.985+0000 7fe60259c640 1 -- 172.21.15.164:0/1491228716 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55576e931220 con 0x55576e931dc0 2024-09-19T08:49:41.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.985+0000 7fe601d9b640 1 --2- 172.21.15.164:0/1491228716 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55576e9314a0 0x55576e937af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:49:41.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.985+0000 7fe60259c640 1 --2- 172.21.15.164:0/1491228716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576e931dc0 0x55576e953ae0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40096f0 tx=0x7fe5f4005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02d796152450ea7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:41.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.985+0000 7fe5ebfff640 1 -- 172.21.15.164:0/1491228716 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4030070 con 0x55576e931dc0 2024-09-19T08:49:41.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.985+0000 7fe5ebfff640 1 -- 172.21.15.164:0/1491228716 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5f4003e30 con 0x55576e931dc0 2024-09-19T08:49:41.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.985+0000 7fe5ebfff640 1 -- 172.21.15.164:0/1491228716 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4027030 con 0x55576e931dc0 2024-09-19T08:49:41.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.986+0000 7fe60b214740 1 -- 172.21.15.164:0/1491228716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576e931dc0 msgr2=0x55576e953ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:41.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.986+0000 7fe60b214740 1 --2- 172.21.15.164:0/1491228716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576e931dc0 0x55576e953ae0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40096f0 tx=0x7fe5f4005d30 comp rx=0 tx=0).stop 2024-09-19T08:49:41.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.986+0000 7fe60b214740 1 -- 172.21.15.164:0/1491228716 shutdown_connections 2024-09-19T08:49:41.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.986+0000 7fe60b214740 1 --2- 172.21.15.164:0/1491228716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55576e93c680 0x55576e954020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:41.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.986+0000 7fe60b214740 1 --2- 172.21.15.164:0/1491228716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576e931dc0 0x55576e953ae0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:41.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.986+0000 7fe60b214740 1 --2- 172.21.15.164:0/1491228716 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55576e9314a0 0x55576e937af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:41.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.986+0000 7fe60b214740 1 -- 172.21.15.164:0/1491228716 >> 172.21.15.164:0/1491228716 conn(0x55576e92da30 msgr2=0x55576e93b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:41.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.986+0000 7fe60b214740 1 -- 172.21.15.164:0/1491228716 shutdown_connections 2024-09-19T08:49:41.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.986+0000 7fe60b214740 1 -- 172.21.15.164:0/1491228716 wait complete. 2024-09-19T08:49:41.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.986+0000 7fe60b214740 1 -- start start 2024-09-19T08:49:41.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.986+0000 7fe60b214740 1 Processor -- start 2024-09-19T08:49:41.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.987+0000 7fe60b214740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576e9314a0 0x55576e944c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:41.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.987+0000 7fe60b214740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55576e931dc0 0x55576e942030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:41.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.987+0000 7fe60b214740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55576e93c680 0x55576e942570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:41.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.987+0000 7fe60b214740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe6041aeaa0 con 0x55576e9314a0 2024-09-19T08:49:41.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.987+0000 7fe60b214740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe6041ae3d0 con 0x55576e93c680 2024-09-19T08:49:41.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.987+0000 7fe60b214740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe6041ab610 con 0x55576e931dc0 2024-09-19T08:49:41.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.987+0000 7fe60b214740 10 client.0 _ll_register_callbacks cb 0x7fe5f4009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:49:41.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.987+0000 7fe601d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576e9314a0 0x55576e944c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:41.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.987+0000 7fe601d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576e9314a0 0x55576e944c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36300/0 (socket says 172.21.15.164:36300) 2024-09-19T08:49:41.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.987+0000 7fe601d9b640 1 -- 172.21.15.164:0/4050226685 learned_addr learned my addr 172.21.15.164:0/4050226685 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:41.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.987+0000 7fe60259c640 1 --2- 172.21.15.164:0/4050226685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55576e931dc0 0x55576e942030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:41.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.987+0000 7fe601d9b640 1 --2- 172.21.15.164:0/4050226685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55576e93c680 0x55576e942570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:41.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.987+0000 7fe601d9b640 1 -- 172.21.15.164:0/4050226685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55576e931dc0 msgr2=0x55576e942030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:41.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.987+0000 7fe601d9b640 1 --2- 172.21.15.164:0/4050226685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55576e931dc0 0x55576e942030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:41.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.987+0000 7fe601d9b640 1 -- 172.21.15.164:0/4050226685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55576e93c680 msgr2=0x55576e942570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:41.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.987+0000 7fe601d9b640 1 --2- 172.21.15.164:0/4050226685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55576e93c680 0x55576e942570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:41.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.987+0000 7fe601d9b640 1 -- 172.21.15.164:0/4050226685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55576e942c60 con 0x55576e9314a0 2024-09-19T08:49:41.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.987+0000 7fe601d9b640 1 --2- 172.21.15.164:0/4050226685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576e9314a0 0x55576e944c40 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8000c00 tx=0x7fe5f4003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=610cd1679cde2a36 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:41.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.987+0000 7fe5e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:49:41.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.987+0000 7fe5e9ffb640 1 -- 172.21.15.164:0/4050226685 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f801f8d0 con 0x55576e9314a0 2024-09-19T08:49:41.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.988+0000 7fe5e9ffb640 1 -- 172.21.15.164:0/4050226685 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fe5f801fa70 con 0x55576e9314a0 2024-09-19T08:49:41.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.988+0000 7fe60b214740 1 -- 172.21.15.164:0/4050226685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5f4003e30 con 0x55576e9314a0 2024-09-19T08:49:41.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.988+0000 7fe5e9ffb640 1 -- 172.21.15.164:0/4050226685 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe5f801fc70 con 0x55576e9314a0 2024-09-19T08:49:41.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.988+0000 7fe60b214740 10 client.16335 fetch_fsmap learned FSMap version 28 2024-09-19T08:49:41.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.988+0000 7fe60b214740 1 -- 172.21.15.164:0/4050226685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55576e9b69f0 con 0x55576e9314a0 2024-09-19T08:49:41.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.988+0000 7fe5e9ffb640 1 -- 172.21.15.164:0/4050226685 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5f4003e30 con 0x55576e9314a0 2024-09-19T08:49:41.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.988+0000 7fe5e9ffb640 1 -- 172.21.15.164:0/4050226685 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe5f801b040 con 0x55576e9314a0 2024-09-19T08:49:41.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.988+0000 7fe60b214740 10 client.16335 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:49:41.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.988+0000 7fe60b214740 10 client.16335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:49:41.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.988+0000 7fe60b214740 20 client.16335 populate_metadata read hostname 'smithi164' 2024-09-19T08:49:41.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.988+0000 7fe60b214740 1 --2- 172.21.15.164:0/4050226685 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55576e94bee0 0x55576e9b9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:41.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.988+0000 7fe60b214740 4 client.16335 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:49:41.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.988+0000 7fe60b214740 1 -- 172.21.15.164:0/4050226685 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55576e94e870 con 0x55576e94bee0 2024-09-19T08:49:41.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.988+0000 7fe60259c640 1 --2- 172.21.15.164:0/4050226685 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55576e94bee0 0x55576e9b9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:41.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.989+0000 7fe5e9ffb640 10 client.16335 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:49:41.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.989+0000 7fe60259c640 1 --2- 172.21.15.164:0/4050226685 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55576e94bee0 0x55576e9b9180 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=475ff0d7b9fb349a server_cookie=7727f0b1e1b86bec in_seq=0 out_seq=0 2024-09-19T08:49:41.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.989+0000 7fe5e9ffb640 1 -- 172.21.15.164:0/4050226685 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55576e94e870 con 0x55576e94bee0 2024-09-19T08:49:41.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.989+0000 7fe5e9ffb640 10 client.16335 handle_command_reply: tid=0 2024-09-19T08:49:41.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.989+0000 7fe60b214740 1 client.16335 shutdown 2024-09-19T08:49:41.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.989+0000 7fe60b214740 1 -- 172.21.15.164:0/4050226685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576e9314a0 msgr2=0x55576e944c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:41.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.989+0000 7fe60b214740 1 --2- 172.21.15.164:0/4050226685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576e9314a0 0x55576e944c40 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8000c00 tx=0x7fe5f4003d50 comp rx=0 tx=0).stop 2024-09-19T08:49:41.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.989+0000 7fe60b214740 1 -- 172.21.15.164:0/4050226685 shutdown_connections 2024-09-19T08:49:41.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.989+0000 7fe60b214740 1 --2- 172.21.15.164:0/4050226685 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55576e94bee0 0x55576e9b9180 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:41.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.989+0000 7fe60b214740 1 --2- 172.21.15.164:0/4050226685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55576e93c680 0x55576e942570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:41.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.989+0000 7fe60b214740 1 --2- 172.21.15.164:0/4050226685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55576e931dc0 0x55576e942030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:41.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.989+0000 7fe5e9ffb640 0 client.16335 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:49:41.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.989+0000 7fe60b214740 1 --2- 172.21.15.164:0/4050226685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55576e9314a0 0x55576e944c40 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:41.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.989+0000 7fe60b214740 1 -- 172.21.15.164:0/4050226685 >> 172.21.15.164:0/4050226685 conn(0x55576e92da30 msgr2=0x55576e92c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:41.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.989+0000 7fe60b214740 1 -- 172.21.15.164:0/4050226685 shutdown_connections 2024-09-19T08:49:41.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.989+0000 7fe60b214740 1 -- 172.21.15.164:0/4050226685 wait complete. 2024-09-19T08:49:41.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.989+0000 7fe60b214740 20 client.16335 trim_cache size 0 max 16384 2024-09-19T08:49:41.998 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:49:41.998 INFO:teuthology.orchestra.run.smithi164.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T08:49:41.999 INFO:teuthology.orchestra.run.smithi164.stdout: "scrubs": { 2024-09-19T08:49:41.999 INFO:teuthology.orchestra.run.smithi164.stdout: "0b935302-e641-4de9-a7b9-9f1fd5fdca64": { 2024-09-19T08:49:41.999 INFO:teuthology.orchestra.run.smithi164.stdout: "path": "/", 2024-09-19T08:49:41.999 INFO:teuthology.orchestra.run.smithi164.stdout: "tag": "0b935302-e641-4de9-a7b9-9f1fd5fdca64", 2024-09-19T08:49:41.999 INFO:teuthology.orchestra.run.smithi164.stdout: "options": "recursive,force" 2024-09-19T08:49:41.999 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:49:41.999 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:49:41.999 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:49:42.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.990+0000 7fe608e3b640 1 -- 172.21.15.164:0/3720894595 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe5ec0a33b0 msgr2=0x7fe5ec0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:42.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.990+0000 7fe608e3b640 1 --2- 172.21.15.164:0/3720894595 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe5ec0a33b0 0x7fe5ec0a5870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc0bca60 tx=0x7fe5fc004250 comp rx=0 tx=0).stop 2024-09-19T08:49:42.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.990+0000 7fe608e3b640 1 -- 172.21.15.164:0/3720894595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe60410c170 msgr2=0x7fe60419cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:42.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.990+0000 7fe608e3b640 1 --2- 172.21.15.164:0/3720894595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe60410c170 0x7fe60419cca0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc004af0 tx=0x7fe5fc02f5b0 comp rx=0 tx=0).stop 2024-09-19T08:49:42.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.990+0000 7fe608e3b640 1 -- 172.21.15.164:0/3720894595 shutdown_connections 2024-09-19T08:49:42.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.990+0000 7fe608e3b640 1 --2- 172.21.15.164:0/3720894595 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe5ec0a33b0 0x7fe5ec0a5870 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:42.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.990+0000 7fe608e3b640 1 --2- 172.21.15.164:0/3720894595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe60410c170 0x7fe60419cca0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:42.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.990+0000 7fe608e3b640 1 --2- 172.21.15.164:0/3720894595 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe604103660 0x7fe604198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:42.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.990+0000 7fe608e3b640 1 --2- 172.21.15.164:0/3720894595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe604102d40 0x7fe6041983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:42.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.990+0000 7fe608e3b640 1 -- 172.21.15.164:0/3720894595 >> 172.21.15.164:0/3720894595 conn(0x7fe6040fc7a0 msgr2=0x7fe6040fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:42.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.990+0000 7fe608e3b640 1 -- 172.21.15.164:0/3720894595 shutdown_connections 2024-09-19T08:49:42.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:41.990+0000 7fe608e3b640 1 -- 172.21.15.164:0/3720894595 wait complete. 2024-09-19T08:49:42.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:41 smithi170 ceph-mon[35072]: pgmap v119: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 878 MiB/s wr, 2.59k op/s 2024-09-19T08:49:42.494 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:49:42.781 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:42 smithi164 ceph-mon[30928]: scrub summary: idle 2024-09-19T08:49:42.850 INFO:tasks.cephfs.filesystem:scrub status for tag:0b935302-e641-4de9-a7b9-9f1fd5fdca64 - {'path': '/', 'tag': '0b935302-e641-4de9-a7b9-9f1fd5fdca64', 'options': 'recursive,force'} 2024-09-19T08:49:42.927 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:49:43.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:42 smithi170 ceph-mon[35072]: scrub summary: idle 2024-09-19T08:49:43.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:42 smithi173 ceph-mon[35306]: scrub summary: idle 2024-09-19T08:49:43.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.413+0000 7f4f29706640 1 -- 172.21.15.164:0/1959740997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f24106f10 msgr2=0x7f4f2410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:43.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.413+0000 7f4f29706640 1 --2- 172.21.15.164:0/1959740997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f24106f10 0x7f4f2410ea90 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c009ca0 tx=0x7f4f1c02a8c0 comp rx=0 tx=0).stop 2024-09-19T08:49:43.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.413+0000 7f4f29706640 1 -- 172.21.15.164:0/1959740997 shutdown_connections 2024-09-19T08:49:43.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.413+0000 7f4f29706640 1 --2- 172.21.15.164:0/1959740997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f24106f10 0x7f4f2410ea90 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:43.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.413+0000 7f4f29706640 1 --2- 172.21.15.164:0/1959740997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f24102790 0x7f4f241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:43.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.413+0000 7f4f29706640 1 --2- 172.21.15.164:0/1959740997 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f24101de0 0x7f4f241021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:43.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.413+0000 7f4f29706640 1 -- 172.21.15.164:0/1959740997 >> 172.21.15.164:0/1959740997 conn(0x7f4f240fd8c0 msgr2=0x7f4f240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:43.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.413+0000 7f4f29706640 1 -- 172.21.15.164:0/1959740997 shutdown_connections 2024-09-19T08:49:43.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.413+0000 7f4f29706640 1 -- 172.21.15.164:0/1959740997 wait complete. 2024-09-19T08:49:43.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.414+0000 7f4f29706640 1 Processor -- start 2024-09-19T08:49:43.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.414+0000 7f4f29706640 1 -- start start 2024-09-19T08:49:43.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.414+0000 7f4f29706640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f24101de0 0x7f4f241983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:43.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.414+0000 7f4f29706640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f24102790 0x7f4f24198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:43.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.414+0000 7f4f29706640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f24106f10 0x7f4f2419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:43.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.414+0000 7f4f29706640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4f24111f70 con 0x7f4f24106f10 2024-09-19T08:49:43.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.414+0000 7f4f29706640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4f24111df0 con 0x7f4f24101de0 2024-09-19T08:49:43.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.414+0000 7f4f29706640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4f241120f0 con 0x7f4f24102790 2024-09-19T08:49:43.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.414+0000 7f4f237fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f24106f10 0x7f4f2419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:43.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.414+0000 7f4f237fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f24106f10 0x7f4f2419ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36318/0 (socket says 172.21.15.164:36318) 2024-09-19T08:49:43.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.415+0000 7f4f237fe640 1 -- 172.21.15.164:0/2411454083 learned_addr learned my addr 172.21.15.164:0/2411454083 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:43.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.415+0000 7f4f22ffd640 1 --2- 172.21.15.164:0/2411454083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f24101de0 0x7f4f241983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:43.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.415+0000 7f4f227fc640 1 --2- 172.21.15.164:0/2411454083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f24102790 0x7f4f24198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:43.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.415+0000 7f4f237fe640 1 -- 172.21.15.164:0/2411454083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f24102790 msgr2=0x7f4f24198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:43.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.415+0000 7f4f237fe640 1 --2- 172.21.15.164:0/2411454083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f24102790 0x7f4f24198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:43.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.415+0000 7f4f237fe640 1 -- 172.21.15.164:0/2411454083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f24101de0 msgr2=0x7f4f241983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:43.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.415+0000 7f4f237fe640 1 --2- 172.21.15.164:0/2411454083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f24101de0 0x7f4f241983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:43.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.415+0000 7f4f237fe640 1 -- 172.21.15.164:0/2411454083 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f2419d3a0 con 0x7f4f24106f10 2024-09-19T08:49:43.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.415+0000 7f4f237fe640 1 --2- 172.21.15.164:0/2411454083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f24106f10 0x7f4f2419ccc0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c031a20 tx=0x7f4f1c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:43.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.415+0000 7f4f0bfff640 1 -- 172.21.15.164:0/2411454083 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c003c30 con 0x7f4f24106f10 2024-09-19T08:49:43.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.415+0000 7f4f0bfff640 1 -- 172.21.15.164:0/2411454083 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f1c003dd0 con 0x7f4f24106f10 2024-09-19T08:49:43.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.415+0000 7f4f29706640 1 -- 172.21.15.164:0/2411454083 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f2419d630 con 0x7f4f24106f10 2024-09-19T08:49:43.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.415+0000 7f4f0bfff640 1 -- 172.21.15.164:0/2411454083 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c02fcf0 con 0x7f4f24106f10 2024-09-19T08:49:43.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.415+0000 7f4f29706640 1 -- 172.21.15.164:0/2411454083 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f241a4f70 con 0x7f4f24106f10 2024-09-19T08:49:43.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.416+0000 7f4f29706640 1 -- 172.21.15.164:0/2411454083 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f241021c0 con 0x7f4f24106f10 2024-09-19T08:49:43.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.418+0000 7f4f0bfff640 1 -- 172.21.15.164:0/2411454083 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f4f1c004b40 con 0x7f4f24106f10 2024-09-19T08:49:43.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.418+0000 7f4f0bfff640 1 --2- 172.21.15.164:0/2411454083 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f4f0c0a33b0 0x7f4f0c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:43.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.418+0000 7f4f22ffd640 1 --2- 172.21.15.164:0/2411454083 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f4f0c0a33b0 0x7f4f0c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:43.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.418+0000 7f4f0bfff640 1 -- 172.21.15.164:0/2411454083 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f4f1c0f0fb0 con 0x7f4f24106f10 2024-09-19T08:49:43.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.419+0000 7f4f22ffd640 1 --2- 172.21.15.164:0/2411454083 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f4f0c0a33b0 0x7f4f0c0a5870 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4f1400bbe0 tx=0x7f4f1400b390 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:43.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.421+0000 7f4f0bfff640 1 -- 172.21.15.164:0/2411454083 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4f1c0bdba0 con 0x7f4f24106f10 2024-09-19T08:49:43.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.624+0000 7f4f29706640 1 -- 172.21.15.164:0/2411454083 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f24199750 con 0x7f4f24106f10 2024-09-19T08:49:43.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.625+0000 7f4f0bfff640 1 -- 172.21.15.164:0/2411454083 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11984 (secure 0 0 0) 0x7f4f1c02fe90 con 0x7f4f24106f10 2024-09-19T08:49:43.626 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:49:43.626 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":28,"btime":"2024-09-19T08:45:46:903004+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:45:46.852789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:49:43.626 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 28 2024-09-19T08:49:43.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.627+0000 7f4f29706640 1 -- 172.21.15.164:0/2411454083 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f4f0c0a33b0 msgr2=0x7f4f0c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:43.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.627+0000 7f4f29706640 1 --2- 172.21.15.164:0/2411454083 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f4f0c0a33b0 0x7f4f0c0a5870 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4f1400bbe0 tx=0x7f4f1400b390 comp rx=0 tx=0).stop 2024-09-19T08:49:43.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.627+0000 7f4f29706640 1 -- 172.21.15.164:0/2411454083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f24106f10 msgr2=0x7f4f2419ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:43.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.627+0000 7f4f29706640 1 --2- 172.21.15.164:0/2411454083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f24106f10 0x7f4f2419ccc0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c031a20 tx=0x7f4f1c031a50 comp rx=0 tx=0).stop 2024-09-19T08:49:43.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.627+0000 7f4f29706640 1 -- 172.21.15.164:0/2411454083 shutdown_connections 2024-09-19T08:49:43.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.627+0000 7f4f29706640 1 --2- 172.21.15.164:0/2411454083 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f4f0c0a33b0 0x7f4f0c0a5870 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:43.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.627+0000 7f4f29706640 1 --2- 172.21.15.164:0/2411454083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f24106f10 0x7f4f2419ccc0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:43.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.627+0000 7f4f29706640 1 --2- 172.21.15.164:0/2411454083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f24102790 0x7f4f24198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:43.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.627+0000 7f4f29706640 1 --2- 172.21.15.164:0/2411454083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f24101de0 0x7f4f241983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:43.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.627+0000 7f4f29706640 1 -- 172.21.15.164:0/2411454083 >> 172.21.15.164:0/2411454083 conn(0x7f4f240fd8c0 msgr2=0x7f4f240feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:43.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.627+0000 7f4f29706640 1 -- 172.21.15.164:0/2411454083 shutdown_connections 2024-09-19T08:49:43.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:43.627+0000 7f4f29706640 1 -- 172.21.15.164:0/2411454083 wait complete. 2024-09-19T08:49:43.948 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:43 smithi164 ceph-mon[30928]: pgmap v120: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 667 MiB/s wr, 2.27k op/s 2024-09-19T08:49:43.948 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:43 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2411454083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:49:44.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:43 smithi170 ceph-mon[35072]: pgmap v120: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 667 MiB/s wr, 2.27k op/s 2024-09-19T08:49:44.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:43 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2411454083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:49:44.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:43 smithi173 ceph-mon[35306]: pgmap v120: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 667 MiB/s wr, 2.27k op/s 2024-09-19T08:49:44.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:43 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2411454083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:49:45.111 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id 1bb05f3d --release --await' 2024-09-19T08:49:45.111 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id 1bb05f3d --release --await 2024-09-19T08:49:45.359 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:49:45.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:45 smithi164 ceph-mon[30928]: pgmap v121: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 596 MiB/s wr, 2.08k op/s 2024-09-19T08:49:45.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.882+0000 7f68be99c640 1 -- 172.21.15.164:0/4097926855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68b8101b40 msgr2=0x7f68b810da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:45.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.882+0000 7f68be99c640 1 --2- 172.21.15.164:0/4097926855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68b8101b40 0x7f68b810da00 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f68b0009ca0 tx=0x7f68b002a8c0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.882+0000 7f68be99c640 1 -- 172.21.15.164:0/4097926855 shutdown_connections 2024-09-19T08:49:45.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.882+0000 7f68be99c640 1 --2- 172.21.15.164:0/4097926855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68b8101b40 0x7f68b810da00 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.882+0000 7f68be99c640 1 --2- 172.21.15.164:0/4097926855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68b81011a0 0x7f68b8101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.882+0000 7f68be99c640 1 --2- 172.21.15.164:0/4097926855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68b8106c30 0x7f68b8107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.882+0000 7f68be99c640 1 -- 172.21.15.164:0/4097926855 >> 172.21.15.164:0/4097926855 conn(0x7f68b8077210 msgr2=0x7f68b80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:45.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.882+0000 7f68be99c640 1 -- 172.21.15.164:0/4097926855 shutdown_connections 2024-09-19T08:49:45.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.882+0000 7f68be99c640 1 -- 172.21.15.164:0/4097926855 wait complete. 2024-09-19T08:49:45.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.882+0000 7f68be99c640 1 Processor -- start 2024-09-19T08:49:45.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.883+0000 7f68be99c640 1 -- start start 2024-09-19T08:49:45.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.883+0000 7f68be99c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68b81011a0 0x7f68b810e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:45.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.883+0000 7f68be99c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68b8101b40 0x7f68b810e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:45.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.883+0000 7f68be99c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68b8106c30 0x7f68b810ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:45.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.883+0000 7f68be99c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f68b81120c0 con 0x7f68b81011a0 2024-09-19T08:49:45.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.883+0000 7f68be99c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f68b8111f40 con 0x7f68b8101b40 2024-09-19T08:49:45.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.883+0000 7f68be99c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f68b8112240 con 0x7f68b8106c30 2024-09-19T08:49:45.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.883+0000 7f68b7fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68b81011a0 0x7f68b810e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:45.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.883+0000 7f68b7fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68b81011a0 0x7f68b810e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48400/0 (socket says 172.21.15.164:48400) 2024-09-19T08:49:45.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.883+0000 7f68b7fff640 1 -- 172.21.15.164:0/627700175 learned_addr learned my addr 172.21.15.164:0/627700175 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:45.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.883+0000 7f68b77fe640 1 --2- 172.21.15.164:0/627700175 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68b8101b40 0x7f68b810e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:45.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.883+0000 7f68bcf39640 1 --2- 172.21.15.164:0/627700175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68b8106c30 0x7f68b810ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:45.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.883+0000 7f68b7fff640 1 -- 172.21.15.164:0/627700175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68b8106c30 msgr2=0x7f68b810ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:45.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.883+0000 7f68b7fff640 1 --2- 172.21.15.164:0/627700175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68b8106c30 0x7f68b810ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.883+0000 7f68b7fff640 1 -- 172.21.15.164:0/627700175 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68b8101b40 msgr2=0x7f68b810e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:45.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.883+0000 7f68b7fff640 1 --2- 172.21.15.164:0/627700175 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68b8101b40 0x7f68b810e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.883+0000 7f68b7fff640 1 -- 172.21.15.164:0/627700175 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b81a6a20 con 0x7f68b81011a0 2024-09-19T08:49:45.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.883+0000 7f68b77fe640 1 --2- 172.21.15.164:0/627700175 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68b8101b40 0x7f68b810e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:49:45.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.884+0000 7f68b7fff640 1 --2- 172.21.15.164:0/627700175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68b81011a0 0x7f68b810e490 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f68a800b4a0 tx=0x7f68a800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:45.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.884+0000 7f68b57fa640 1 -- 172.21.15.164:0/627700175 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8014070 con 0x7f68b81011a0 2024-09-19T08:49:45.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.884+0000 7f68be99c640 1 -- 172.21.15.164:0/627700175 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b81a6d10 con 0x7f68b81011a0 2024-09-19T08:49:45.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.884+0000 7f68b57fa640 1 -- 172.21.15.164:0/627700175 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68a8010af0 con 0x7f68b81011a0 2024-09-19T08:49:45.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.884+0000 7f68b57fa640 1 -- 172.21.15.164:0/627700175 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8017d10 con 0x7f68b81011a0 2024-09-19T08:49:45.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.884+0000 7f68be99c640 1 -- 172.21.15.164:0/627700175 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b81a71a0 con 0x7f68b81011a0 2024-09-19T08:49:45.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.886+0000 7f68b57fa640 1 -- 172.21.15.164:0/627700175 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f68a8017070 con 0x7f68b81011a0 2024-09-19T08:49:45.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.886+0000 7f68c0d75740 1 Processor -- start 2024-09-19T08:49:45.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.886+0000 7f68c0d75740 1 -- start start 2024-09-19T08:49:45.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.887+0000 7f68c0d75740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be3f5b7060 0x55be3f5b7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:45.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.887+0000 7f68c0d75740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be3f5b7980 0x55be3f5c1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:45.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.887+0000 7f68c0d75740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be3f5c2240 0x55be3f5c4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:45.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.887+0000 7f68c0d75740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55be3f51c620 con 0x55be3f5c2240 2024-09-19T08:49:45.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.887+0000 7f68c0d75740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55be3f4c3e50 con 0x55be3f5b7980 2024-09-19T08:49:45.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.887+0000 7f68c0d75740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55be3f5185e0 con 0x55be3f5b7060 2024-09-19T08:49:45.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.887+0000 7f68b57fa640 1 --2- 172.21.15.164:0/627700175 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f68a00a33b0 0x7f68a00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:45.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.887+0000 7f68b77fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be3f5c2240 0x55be3f5c4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:45.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.887+0000 7f68b77fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be3f5c2240 0x55be3f5c4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48414/0 (socket says 172.21.15.164:48414) 2024-09-19T08:49:45.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.887+0000 7f68b77fe640 1 -- 172.21.15.164:0/3491208483 learned_addr learned my addr 172.21.15.164:0/3491208483 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:45.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.887+0000 7f68bcf39640 1 --2- 172.21.15.164:0/3491208483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be3f5b7060 0x55be3f5b7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:45.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.887+0000 7f68b7fff640 1 --2- 172.21.15.164:0/3491208483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be3f5b7980 0x55be3f5c1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:45.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.887+0000 7f68bcf39640 1 --2- 172.21.15.164:0/627700175 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f68a00a33b0 0x7f68a00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:45.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.887+0000 7f68b57fa640 1 -- 172.21.15.164:0/627700175 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f68a80cef70 con 0x7f68b81011a0 2024-09-19T08:49:45.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.887+0000 7f68b77fe640 1 -- 172.21.15.164:0/3491208483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be3f5b7060 msgr2=0x55be3f5b7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:45.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.887+0000 7f68b77fe640 1 --2- 172.21.15.164:0/3491208483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be3f5b7060 0x55be3f5b7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.887+0000 7f68b77fe640 1 -- 172.21.15.164:0/3491208483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be3f5b7980 msgr2=0x55be3f5c1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:45.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.887+0000 7f68b77fe640 1 --2- 172.21.15.164:0/3491208483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be3f5b7980 0x55be3f5c1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.887+0000 7f68b77fe640 1 -- 172.21.15.164:0/3491208483 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be3f4b4d40 con 0x55be3f5c2240 2024-09-19T08:49:45.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.887+0000 7f68b77fe640 1 --2- 172.21.15.164:0/3491208483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be3f5c2240 0x55be3f5c4630 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f68ac010480 tx=0x7f68ac010940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749520368298f315 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:45.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.887+0000 7f68bcf39640 1 --2- 172.21.15.164:0/627700175 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f68a00a33b0 0x7f68a00a5870 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f68b0031f80 tx=0x7f68b00095c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:45.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.887+0000 7f689e7fc640 1 -- 172.21.15.164:0/3491208483 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac011760 con 0x55be3f5c2240 2024-09-19T08:49:45.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.888+0000 7f689e7fc640 1 -- 172.21.15.164:0/3491208483 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68ac011de0 con 0x55be3f5c2240 2024-09-19T08:49:45.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.888+0000 7f689e7fc640 1 -- 172.21.15.164:0/3491208483 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac00e430 con 0x55be3f5c2240 2024-09-19T08:49:45.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.888+0000 7f68c0d75740 1 -- 172.21.15.164:0/3491208483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be3f5c2240 msgr2=0x55be3f5c4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:45.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.888+0000 7f68c0d75740 1 --2- 172.21.15.164:0/3491208483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be3f5c2240 0x55be3f5c4630 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f68ac010480 tx=0x7f68ac010940 comp rx=0 tx=0).stop 2024-09-19T08:49:45.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.888+0000 7f68c0d75740 1 -- 172.21.15.164:0/3491208483 shutdown_connections 2024-09-19T08:49:45.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.888+0000 7f68c0d75740 1 --2- 172.21.15.164:0/3491208483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be3f5c2240 0x55be3f5c4630 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.888+0000 7f68c0d75740 1 --2- 172.21.15.164:0/3491208483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be3f5b7980 0x55be3f5c1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.888+0000 7f68c0d75740 1 --2- 172.21.15.164:0/3491208483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be3f5b7060 0x55be3f5b7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.888+0000 7f68c0d75740 1 -- 172.21.15.164:0/3491208483 >> 172.21.15.164:0/3491208483 conn(0x55be3f5b35f0 msgr2=0x55be3f5b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:45.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.888+0000 7f68c0d75740 1 -- 172.21.15.164:0/3491208483 shutdown_connections 2024-09-19T08:49:45.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.888+0000 7f68c0d75740 1 -- 172.21.15.164:0/3491208483 wait complete. 2024-09-19T08:49:45.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.889+0000 7f68c0d75740 1 -- start start 2024-09-19T08:49:45.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.889+0000 7f68c0d75740 1 Processor -- start 2024-09-19T08:49:45.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.889+0000 7f68c0d75740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be3f5b7060 0x55be3f5bf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:45.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.889+0000 7f68c0d75740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be3f5b7980 0x55be3f5bf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:45.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.889+0000 7f68c0d75740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be3f5c2240 0x55be3f5bfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:45.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.889+0000 7f68c0d75740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55be3f5c4e10 con 0x55be3f5b7980 2024-09-19T08:49:45.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.889+0000 7f68c0d75740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55be3f4a4f10 con 0x55be3f5c2240 2024-09-19T08:49:45.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.889+0000 7f68c0d75740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55be3f5c4f90 con 0x55be3f5b7060 2024-09-19T08:49:45.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.889+0000 7f68c0d75740 10 client.0 _ll_register_callbacks cb 0x55be3f4c8db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:49:45.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.889+0000 7f68b77fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be3f5b7980 0x55be3f5bf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:45.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.889+0000 7f68b77fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be3f5b7980 0x55be3f5bf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48420/0 (socket says 172.21.15.164:48420) 2024-09-19T08:49:45.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.889+0000 7f68b77fe640 1 -- 172.21.15.164:0/2439622048 learned_addr learned my addr 172.21.15.164:0/2439622048 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:45.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.889+0000 7f68bcf39640 1 --2- 172.21.15.164:0/2439622048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be3f5c2240 0x55be3f5bfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:45.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.889+0000 7f68b7fff640 1 --2- 172.21.15.164:0/2439622048 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be3f5b7060 0x55be3f5bf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:45.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.889+0000 7f68b77fe640 1 -- 172.21.15.164:0/2439622048 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be3f5b7060 msgr2=0x55be3f5bf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:45.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.889+0000 7f68b77fe640 1 --2- 172.21.15.164:0/2439622048 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be3f5b7060 0x55be3f5bf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.889+0000 7f68b77fe640 1 -- 172.21.15.164:0/2439622048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be3f5c2240 msgr2=0x55be3f5bfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:45.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.889+0000 7f68b77fe640 1 --2- 172.21.15.164:0/2439622048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be3f5c2240 0x55be3f5bfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.889+0000 7f68b77fe640 1 -- 172.21.15.164:0/2439622048 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be3f22b580 con 0x55be3f5b7980 2024-09-19T08:49:45.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.890+0000 7f68b7fff640 1 --2- 172.21.15.164:0/2439622048 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be3f5b7060 0x55be3f5bf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:49:45.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.890+0000 7f68b77fe640 1 --2- 172.21.15.164:0/2439622048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be3f5b7980 0x55be3f5bf610 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f68ac01cb70 tx=0x7f68ac01c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ce93d1015d67c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:45.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.890+0000 7f68977fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:49:45.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.890+0000 7f68977fe640 1 -- 172.21.15.164:0/2439622048 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac02a060 con 0x55be3f5b7980 2024-09-19T08:49:45.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.890+0000 7f68977fe640 1 -- 172.21.15.164:0/2439622048 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f68ac005250 con 0x55be3f5b7980 2024-09-19T08:49:45.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.890+0000 7f68c0d75740 1 -- 172.21.15.164:0/2439622048 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68ac011de0 con 0x55be3f5b7980 2024-09-19T08:49:45.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.890+0000 7f68977fe640 1 -- 172.21.15.164:0/2439622048 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f68ac005450 con 0x55be3f5b7980 2024-09-19T08:49:45.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.890+0000 7f68c0d75740 10 client.16353 fetch_fsmap learned FSMap version 28 2024-09-19T08:49:45.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.890+0000 7f68c0d75740 1 -- 172.21.15.164:0/2439622048 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55be3f2f5960 con 0x55be3f5b7980 2024-09-19T08:49:45.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.890+0000 7f68977fe640 1 -- 172.21.15.164:0/2439622048 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68ac011de0 con 0x55be3f5b7980 2024-09-19T08:49:45.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.890+0000 7f68977fe640 1 -- 172.21.15.164:0/2439622048 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f68ac02d020 con 0x55be3f5b7980 2024-09-19T08:49:45.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.890+0000 7f68c0d75740 10 client.16353 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:49:45.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.890+0000 7f68c0d75740 10 client.16353 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:49:45.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.890+0000 7f68c0d75740 20 client.16353 populate_metadata read hostname 'smithi164' 2024-09-19T08:49:45.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.891+0000 7f68c0d75740 1 --2- 172.21.15.164:0/2439622048 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55be3f5d1cc0 0x55be3f5d40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:45.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.891+0000 7f68c0d75740 4 client.16353 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:49:45.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.891+0000 7f68c0d75740 1 -- 172.21.15.164:0/2439622048 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be3f529b90 con 0x55be3f5d1cc0 2024-09-19T08:49:45.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.891+0000 7f68b7fff640 1 --2- 172.21.15.164:0/2439622048 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55be3f5d1cc0 0x55be3f5d40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:45.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.891+0000 7f68b7fff640 1 --2- 172.21.15.164:0/2439622048 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55be3f5d1cc0 0x55be3f5d40b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=564aa8bda5ef1e63 server_cookie=31a4bf8927a32ed2 in_seq=0 out_seq=0 2024-09-19T08:49:45.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.891+0000 7f68977fe640 10 client.16353 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:49:45.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.892+0000 7f68977fe640 1 -- 172.21.15.164:0/2439622048 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55be3f529b90 con 0x55be3f5d1cc0 2024-09-19T08:49:45.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.892+0000 7f68977fe640 10 client.16353 handle_command_reply: tid=0 2024-09-19T08:49:45.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.892+0000 7f68c0d75740 1 client.16353 shutdown 2024-09-19T08:49:45.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.892+0000 7f68c0d75740 1 -- 172.21.15.164:0/2439622048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be3f5b7980 msgr2=0x55be3f5bf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:45.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.892+0000 7f68c0d75740 1 --2- 172.21.15.164:0/2439622048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be3f5b7980 0x55be3f5bf610 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f68ac01cb70 tx=0x7f68ac01c6b0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.892+0000 7f68c0d75740 1 -- 172.21.15.164:0/2439622048 shutdown_connections 2024-09-19T08:49:45.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.892+0000 7f68c0d75740 1 --2- 172.21.15.164:0/2439622048 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55be3f5d1cc0 0x55be3f5d40b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.892+0000 7f68c0d75740 1 --2- 172.21.15.164:0/2439622048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be3f5c2240 0x55be3f5bfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.892+0000 7f68c0d75740 1 --2- 172.21.15.164:0/2439622048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be3f5b7980 0x55be3f5bf610 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.892+0000 7f68c0d75740 1 --2- 172.21.15.164:0/2439622048 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be3f5b7060 0x55be3f5bf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.892+0000 7f68977fe640 0 client.16353 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:49:45.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.892+0000 7f68c0d75740 1 -- 172.21.15.164:0/2439622048 >> 172.21.15.164:0/2439622048 conn(0x55be3f5b35f0 msgr2=0x55be3f5b92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:45.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.892+0000 7f68c0d75740 1 -- 172.21.15.164:0/2439622048 shutdown_connections 2024-09-19T08:49:45.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.892+0000 7f68c0d75740 1 -- 172.21.15.164:0/2439622048 wait complete. 2024-09-19T08:49:45.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.892+0000 7f68c0d75740 20 client.16353 trim_cache size 0 max 16384 2024-09-19T08:49:45.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.909+0000 7f68c0d75740 1 Processor -- start 2024-09-19T08:49:45.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.909+0000 7f68c0d75740 1 -- start start 2024-09-19T08:49:45.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.909+0000 7f68c0d75740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be3f5b7060 0x55be3f5d96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:45.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.909+0000 7f68c0d75740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be3f5b7980 0x55be3f5d9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:45.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.909+0000 7f68c0d75740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be3f5c2240 0x55be3f5bc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:45.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.909+0000 7f68c0d75740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f68b81af9a0 con 0x55be3f5c2240 2024-09-19T08:49:45.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.909+0000 7f68c0d75740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f68b81b14e0 con 0x55be3f5b7980 2024-09-19T08:49:45.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.909+0000 7f68c0d75740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f68b81b0070 con 0x55be3f5b7060 2024-09-19T08:49:45.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.909+0000 7f68bcf39640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be3f5c2240 0x55be3f5bc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:45.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.909+0000 7f68bcf39640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be3f5c2240 0x55be3f5bc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48422/0 (socket says 172.21.15.164:48422) 2024-09-19T08:49:45.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.909+0000 7f68bcf39640 1 -- 172.21.15.164:0/647441146 learned_addr learned my addr 172.21.15.164:0/647441146 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:45.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.910+0000 7f68b7fff640 1 --2- 172.21.15.164:0/647441146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be3f5b7060 0x55be3f5d96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:45.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.910+0000 7f68b77fe640 1 --2- 172.21.15.164:0/647441146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be3f5b7980 0x55be3f5d9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:45.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.910+0000 7f68bcf39640 1 -- 172.21.15.164:0/647441146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be3f5b7060 msgr2=0x55be3f5d96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:45.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.910+0000 7f68bcf39640 1 --2- 172.21.15.164:0/647441146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be3f5b7060 0x55be3f5d96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.910+0000 7f68bcf39640 1 -- 172.21.15.164:0/647441146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be3f5b7980 msgr2=0x55be3f5d9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:45.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.910+0000 7f68bcf39640 1 --2- 172.21.15.164:0/647441146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be3f5b7980 0x55be3f5d9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.910+0000 7f68bcf39640 1 -- 172.21.15.164:0/647441146 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be3f5b77b0 con 0x55be3f5c2240 2024-09-19T08:49:45.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.910+0000 7f68b77fe640 1 --2- 172.21.15.164:0/647441146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be3f5b7980 0x55be3f5d9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:49:45.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.910+0000 7f68bcf39640 1 --2- 172.21.15.164:0/647441146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be3f5c2240 0x55be3f5bc3b0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f68b0009ca0 tx=0x7f68b003a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d95b7b4aa4a1fea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:45.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.910+0000 7f689dffb640 1 -- 172.21.15.164:0/647441146 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b0003af0 con 0x55be3f5c2240 2024-09-19T08:49:45.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.910+0000 7f689dffb640 1 -- 172.21.15.164:0/647441146 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68b0036040 con 0x55be3f5c2240 2024-09-19T08:49:45.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.910+0000 7f689dffb640 1 -- 172.21.15.164:0/647441146 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b002f870 con 0x55be3f5c2240 2024-09-19T08:49:45.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.911+0000 7f68c0d75740 1 -- 172.21.15.164:0/647441146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be3f5c2240 msgr2=0x55be3f5bc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:45.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.911+0000 7f68c0d75740 1 --2- 172.21.15.164:0/647441146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be3f5c2240 0x55be3f5bc3b0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f68b0009ca0 tx=0x7f68b003a000 comp rx=0 tx=0).stop 2024-09-19T08:49:45.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.911+0000 7f68c0d75740 1 -- 172.21.15.164:0/647441146 shutdown_connections 2024-09-19T08:49:45.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.911+0000 7f68c0d75740 1 --2- 172.21.15.164:0/647441146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be3f5c2240 0x55be3f5bc3b0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.911+0000 7f68c0d75740 1 --2- 172.21.15.164:0/647441146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be3f5b7980 0x55be3f5d9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.911+0000 7f68c0d75740 1 --2- 172.21.15.164:0/647441146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be3f5b7060 0x55be3f5d96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.911+0000 7f68c0d75740 1 -- 172.21.15.164:0/647441146 >> 172.21.15.164:0/647441146 conn(0x55be3f5b35f0 msgr2=0x55be3f5bda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:45.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.911+0000 7f68c0d75740 1 -- 172.21.15.164:0/647441146 shutdown_connections 2024-09-19T08:49:45.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.911+0000 7f68c0d75740 1 -- 172.21.15.164:0/647441146 wait complete. 2024-09-19T08:49:45.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.911+0000 7f68c0d75740 1 -- start start 2024-09-19T08:49:45.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.911+0000 7f68c0d75740 1 Processor -- start 2024-09-19T08:49:45.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.912+0000 7f68c0d75740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be3f5b7060 0x55be3f5ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:45.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.912+0000 7f68c0d75740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be3f5b7980 0x55be3f5ca930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:45.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.912+0000 7f68c0d75740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be3f5c2240 0x55be3f5c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:45.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.912+0000 7f68c0d75740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f68b81b0e10 con 0x55be3f5b7060 2024-09-19T08:49:45.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.912+0000 7f68c0d75740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f68b81b0740 con 0x55be3f5c2240 2024-09-19T08:49:45.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.912+0000 7f68c0d75740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f68b81ad980 con 0x55be3f5b7980 2024-09-19T08:49:45.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.912+0000 7f68c0d75740 10 client.0 _ll_register_callbacks cb 0x7f68ac01c7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:49:45.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.912+0000 7f68b7fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be3f5b7060 0x55be3f5ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:45.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.912+0000 7f68b7fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be3f5b7060 0x55be3f5ca3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48428/0 (socket says 172.21.15.164:48428) 2024-09-19T08:49:45.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.912+0000 7f68b7fff640 1 -- 172.21.15.164:0/2511693286 learned_addr learned my addr 172.21.15.164:0/2511693286 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:49:45.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.912+0000 7f68b77fe640 1 --2- 172.21.15.164:0/2511693286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be3f5b7980 0x55be3f5ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:45.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.912+0000 7f68bcf39640 1 --2- 172.21.15.164:0/2511693286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be3f5c2240 0x55be3f5c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:45.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.912+0000 7f68b7fff640 1 -- 172.21.15.164:0/2511693286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be3f5b7980 msgr2=0x55be3f5ca930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:45.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.912+0000 7f68b7fff640 1 --2- 172.21.15.164:0/2511693286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be3f5b7980 0x55be3f5ca930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.912+0000 7f68b7fff640 1 -- 172.21.15.164:0/2511693286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be3f5c2240 msgr2=0x55be3f5c71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:49:45.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.912+0000 7f68b7fff640 1 --2- 172.21.15.164:0/2511693286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be3f5c2240 0x55be3f5c71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.912+0000 7f68b7fff640 1 -- 172.21.15.164:0/2511693286 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be3f5c78f0 con 0x55be3f5b7060 2024-09-19T08:49:45.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.912+0000 7f68b7fff640 1 --2- 172.21.15.164:0/2511693286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be3f5b7060 0x55be3f5ca3f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f68a8050700 tx=0x7f68a8066260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43b300c42ee71 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:49:45.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.912+0000 7f68977fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:49:45.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.912+0000 7f68977fe640 1 -- 172.21.15.164:0/2511693286 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a801e040 con 0x55be3f5b7060 2024-09-19T08:49:45.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.912+0000 7f68977fe640 1 -- 172.21.15.164:0/2511693286 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f68a80cc9b0 con 0x55be3f5b7060 2024-09-19T08:49:45.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.912+0000 7f68c0d75740 1 -- 172.21.15.164:0/2511693286 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68b0036040 con 0x55be3f5b7060 2024-09-19T08:49:45.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.913+0000 7f68977fe640 1 -- 172.21.15.164:0/2511693286 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f68a80a4040 con 0x55be3f5b7060 2024-09-19T08:49:45.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.913+0000 7f68c0d75740 10 client.16359 fetch_fsmap learned FSMap version 28 2024-09-19T08:49:45.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.913+0000 7f68c0d75740 1 -- 172.21.15.164:0/2511693286 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55be3f63d0a0 con 0x55be3f5b7060 2024-09-19T08:49:45.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.913+0000 7f68977fe640 1 -- 172.21.15.164:0/2511693286 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68b0036040 con 0x55be3f5b7060 2024-09-19T08:49:45.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.913+0000 7f68977fe640 1 -- 172.21.15.164:0/2511693286 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f68a80d10b0 con 0x55be3f5b7060 2024-09-19T08:49:45.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.913+0000 7f68c0d75740 10 client.16359 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T08:49:45.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.913+0000 7f68c0d75740 10 client.16359 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:49:45.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.913+0000 7f68c0d75740 20 client.16359 populate_metadata read hostname 'smithi164' 2024-09-19T08:49:45.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.913+0000 7f68c0d75740 1 --2- 172.21.15.164:0/2511693286 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55be3f63d770 0x55be3f63fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:49:45.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.913+0000 7f68c0d75740 4 client.16359 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1bb05f3d", "release": true, "await": true}] 2024-09-19T08:49:45.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.913+0000 7f68c0d75740 1 -- 172.21.15.164:0/2511693286 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1bb05f3d", "release": true, "await": true}) -- 0x55be3f5240a0 con 0x55be3f63d770 2024-09-19T08:49:45.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.913+0000 7f68b77fe640 1 --2- 172.21.15.164:0/2511693286 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55be3f63d770 0x55be3f63fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:49:45.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.914+0000 7f68b77fe640 1 --2- 172.21.15.164:0/2511693286 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55be3f63d770 0x55be3f63fb60 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b33b1af00de15e69 server_cookie=567fe03270b2056c in_seq=0 out_seq=0 2024-09-19T08:49:45.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.914+0000 7f68977fe640 10 client.16359 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:49:45.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.918+0000 7f68977fe640 1 -- 172.21.15.164:0/2511693286 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55be3f5240a0 con 0x55be3f63d770 2024-09-19T08:49:45.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.918+0000 7f68977fe640 10 client.16359 handle_command_reply: tid=0 2024-09-19T08:49:45.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.918+0000 7f68c0d75740 1 client.16359 shutdown 2024-09-19T08:49:45.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.918+0000 7f68c0d75740 1 -- 172.21.15.164:0/2511693286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be3f5b7060 msgr2=0x55be3f5ca3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:45.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.918+0000 7f68c0d75740 1 --2- 172.21.15.164:0/2511693286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be3f5b7060 0x55be3f5ca3f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f68a8050700 tx=0x7f68a8066260 comp rx=0 tx=0).stop 2024-09-19T08:49:45.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.918+0000 7f68c0d75740 1 -- 172.21.15.164:0/2511693286 shutdown_connections 2024-09-19T08:49:45.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.918+0000 7f68c0d75740 1 --2- 172.21.15.164:0/2511693286 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55be3f63d770 0x55be3f63fb60 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.918+0000 7f68c0d75740 1 --2- 172.21.15.164:0/2511693286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be3f5c2240 0x55be3f5c71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.918+0000 7f68c0d75740 1 --2- 172.21.15.164:0/2511693286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be3f5b7980 0x55be3f5ca930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.918+0000 7f68977fe640 0 client.16359 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:49:45.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.918+0000 7f68c0d75740 1 --2- 172.21.15.164:0/2511693286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be3f5b7060 0x55be3f5ca3f0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.918+0000 7f68c0d75740 1 -- 172.21.15.164:0/2511693286 >> 172.21.15.164:0/2511693286 conn(0x55be3f5b35f0 msgr2=0x55be3f5b1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:45.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.918+0000 7f68c0d75740 1 -- 172.21.15.164:0/2511693286 shutdown_connections 2024-09-19T08:49:45.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.918+0000 7f68c0d75740 1 -- 172.21.15.164:0/2511693286 wait complete. 2024-09-19T08:49:45.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.918+0000 7f68c0d75740 20 client.16359 trim_cache size 0 max 16384 2024-09-19T08:49:45.921 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:49:45.921 INFO:teuthology.orchestra.run.smithi164.stdout: "epoch": 28, 2024-09-19T08:49:45.921 INFO:teuthology.orchestra.run.smithi164.stdout: "leader": 24479, 2024-09-19T08:49:45.921 INFO:teuthology.orchestra.run.smithi164.stdout: "set_version": 16, 2024-09-19T08:49:45.922 INFO:teuthology.orchestra.run.smithi164.stdout: "sets": { 2024-09-19T08:49:45.922 INFO:teuthology.orchestra.run.smithi164.stdout: "1bb05f3d": { 2024-09-19T08:49:45.922 INFO:teuthology.orchestra.run.smithi164.stdout: "version": 16, 2024-09-19T08:49:45.922 INFO:teuthology.orchestra.run.smithi164.stdout: "age_ref": 0.0, 2024-09-19T08:49:45.922 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:49:45.922 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "RELEASED", 2024-09-19T08:49:45.922 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:49:45.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:49:45.922 INFO:teuthology.orchestra.run.smithi164.stdout: "timeout": 90.0, 2024-09-19T08:49:45.922 INFO:teuthology.orchestra.run.smithi164.stdout: "expiration": 136.6, 2024-09-19T08:49:45.922 INFO:teuthology.orchestra.run.smithi164.stdout: "members": { 2024-09-19T08:49:45.922 INFO:teuthology.orchestra.run.smithi164.stdout: "file:/": { 2024-09-19T08:49:45.922 INFO:teuthology.orchestra.run.smithi164.stdout: "excluded": false, 2024-09-19T08:49:45.923 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:49:45.923 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "RELEASED", 2024-09-19T08:49:45.923 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:49:45.923 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:49:45.923 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:49:45.923 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:49:45.923 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:49:45.923 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:49:45.923 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:49:45.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.919+0000 7f68be99c640 1 -- 172.21.15.164:0/627700175 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f68a00a33b0 msgr2=0x7f68a00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:45.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.919+0000 7f68be99c640 1 --2- 172.21.15.164:0/627700175 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f68a00a33b0 0x7f68a00a5870 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f68b0031f80 tx=0x7f68b00095c0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.919+0000 7f68be99c640 1 -- 172.21.15.164:0/627700175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68b81011a0 msgr2=0x7f68b810e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:49:45.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.919+0000 7f68be99c640 1 --2- 172.21.15.164:0/627700175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68b81011a0 0x7f68b810e490 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f68a800b4a0 tx=0x7f68a800b960 comp rx=0 tx=0).stop 2024-09-19T08:49:45.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.919+0000 7f68be99c640 1 -- 172.21.15.164:0/627700175 shutdown_connections 2024-09-19T08:49:45.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.919+0000 7f68be99c640 1 --2- 172.21.15.164:0/627700175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68b8106c30 0x7f68b810ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.919+0000 7f68be99c640 1 --2- 172.21.15.164:0/627700175 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68b8101b40 0x7f68b810e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.919+0000 7f68be99c640 1 --2- 172.21.15.164:0/627700175 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f68a00a33b0 0x7f68a00a5870 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.919+0000 7f68be99c640 1 --2- 172.21.15.164:0/627700175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68b81011a0 0x7f68b810e490 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:49:45.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.919+0000 7f68be99c640 1 -- 172.21.15.164:0/627700175 >> 172.21.15.164:0/627700175 conn(0x7f68b8077210 msgr2=0x7f68b8103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:49:45.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.919+0000 7f68be99c640 1 -- 172.21.15.164:0/627700175 shutdown_connections 2024-09-19T08:49:45.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:49:45.919+0000 7f68be99c640 1 -- 172.21.15.164:0/627700175 wait complete. 2024-09-19T08:49:46.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:45 smithi170 ceph-mon[35072]: pgmap v121: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 596 MiB/s wr, 2.08k op/s 2024-09-19T08:49:46.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:45 smithi173 ceph-mon[35306]: pgmap v121: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 596 MiB/s wr, 2.08k op/s 2024-09-19T08:49:48.039 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:47 smithi164 ceph-mon[30928]: pgmap v122: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 546 MiB/s wr, 2.05k op/s 2024-09-19T08:49:48.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:47 smithi170 ceph-mon[35072]: pgmap v122: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 546 MiB/s wr, 2.05k op/s 2024-09-19T08:49:48.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:47 smithi173 ceph-mon[35306]: pgmap v122: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 546 MiB/s wr, 2.05k op/s 2024-09-19T08:49:48.223 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1bb05f3d', total seconds elapsed: 22.3 2024-09-19T08:49:50.035 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:49 smithi164 ceph-mon[30928]: pgmap v123: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 333 MiB/s wr, 1.64k op/s 2024-09-19T08:49:50.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:49 smithi170 ceph-mon[35072]: pgmap v123: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 333 MiB/s wr, 1.64k op/s 2024-09-19T08:49:50.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:49 smithi173 ceph-mon[35306]: pgmap v123: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 333 MiB/s wr, 1.64k op/s 2024-09-19T08:49:52.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:51 smithi170 ceph-mon[35072]: pgmap v124: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 504 MiB/s wr, 2.48k op/s 2024-09-19T08:49:52.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:51 smithi173 ceph-mon[35306]: pgmap v124: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 504 MiB/s wr, 2.48k op/s 2024-09-19T08:49:52.284 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:51 smithi164 ceph-mon[30928]: pgmap v124: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 504 MiB/s wr, 2.48k op/s 2024-09-19T08:49:54.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:53 smithi170 ceph-mon[35072]: pgmap v125: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 359 MiB/s wr, 1.75k op/s 2024-09-19T08:49:54.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:53 smithi173 ceph-mon[35306]: pgmap v125: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 359 MiB/s wr, 1.75k op/s 2024-09-19T08:49:54.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:53 smithi164 ceph-mon[30928]: pgmap v125: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 359 MiB/s wr, 1.75k op/s 2024-09-19T08:49:56.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:55 smithi170 ceph-mon[35072]: pgmap v126: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 378 MiB/s wr, 1.72k op/s 2024-09-19T08:49:56.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:55 smithi173 ceph-mon[35306]: pgmap v126: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 378 MiB/s wr, 1.72k op/s 2024-09-19T08:49:56.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:55 smithi164 ceph-mon[30928]: pgmap v126: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 378 MiB/s wr, 1.72k op/s 2024-09-19T08:49:58.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:57 smithi170 ceph-mon[35072]: pgmap v127: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 563 MiB/s wr, 2.32k op/s 2024-09-19T08:49:58.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:57 smithi173 ceph-mon[35306]: pgmap v127: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 563 MiB/s wr, 2.32k op/s 2024-09-19T08:49:58.284 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:57 smithi164 ceph-mon[30928]: pgmap v127: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 563 MiB/s wr, 2.32k op/s 2024-09-19T08:49:59.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:49:58 smithi173 ceph-mon[35306]: pgmap v128: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 635 MiB/s wr, 2.41k op/s 2024-09-19T08:49:59.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:49:58 smithi164 ceph-mon[30928]: pgmap v128: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 635 MiB/s wr, 2.41k op/s 2024-09-19T08:49:59.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:49:58 smithi170 ceph-mon[35072]: pgmap v128: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 635 MiB/s wr, 2.41k op/s 2024-09-19T08:50:00.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:00 smithi170 ceph-mon[35072]: overall HEALTH_OK 2024-09-19T08:50:00.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:00 smithi173 ceph-mon[35306]: overall HEALTH_OK 2024-09-19T08:50:00.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:00 smithi164 ceph-mon[30928]: overall HEALTH_OK 2024-09-19T08:50:01.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:01 smithi170 ceph-mon[35072]: pgmap v129: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 893 MiB/s wr, 2.87k op/s 2024-09-19T08:50:01.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:01 smithi173 ceph-mon[35306]: pgmap v129: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 893 MiB/s wr, 2.87k op/s 2024-09-19T08:50:01.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:01 smithi164 ceph-mon[30928]: pgmap v129: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 893 MiB/s wr, 2.87k op/s 2024-09-19T08:50:02.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:02 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:50:02.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:02 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:50:02.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:02 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:50:03.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:03 smithi170 ceph-mon[35072]: pgmap v130: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 766 MiB/s wr, 2.09k op/s 2024-09-19T08:50:03.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:03 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:50:03.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:03 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:50:03.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:03 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:50:03.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:03 smithi173 ceph-mon[35306]: pgmap v130: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 766 MiB/s wr, 2.09k op/s 2024-09-19T08:50:03.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:03 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:50:03.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:03 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:50:03.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:03 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:50:03.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:03 smithi164 ceph-mon[30928]: pgmap v130: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 766 MiB/s wr, 2.09k op/s 2024-09-19T08:50:03.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:03 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:50:03.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:03 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:50:03.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:03 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:50:05.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:05 smithi164 ceph-mon[30928]: pgmap v131: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 756 MiB/s wr, 1.85k op/s 2024-09-19T08:50:05.624 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:05 smithi170 ceph-mon[35072]: pgmap v131: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 756 MiB/s wr, 1.85k op/s 2024-09-19T08:50:05.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:05 smithi173 ceph-mon[35306]: pgmap v131: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 756 MiB/s wr, 1.85k op/s 2024-09-19T08:50:07.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:07 smithi164 ceph-mon[30928]: pgmap v132: 97 pgs: 97 active+clean; 30 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 900 MiB/s wr, 2.17k op/s 2024-09-19T08:50:07.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:07 smithi170 ceph-mon[35072]: pgmap v132: 97 pgs: 97 active+clean; 30 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 900 MiB/s wr, 2.17k op/s 2024-09-19T08:50:07.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:07 smithi173 ceph-mon[35306]: pgmap v132: 97 pgs: 97 active+clean; 30 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 900 MiB/s wr, 2.17k op/s 2024-09-19T08:50:09.457 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:09 smithi164 ceph-mon[30928]: pgmap v133: 97 pgs: 97 active+clean; 31 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 791 MiB/s wr, 1.90k op/s 2024-09-19T08:50:09.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:09 smithi170 ceph-mon[35072]: pgmap v133: 97 pgs: 97 active+clean; 31 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 791 MiB/s wr, 1.90k op/s 2024-09-19T08:50:09.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:09 smithi173 ceph-mon[35306]: pgmap v133: 97 pgs: 97 active+clean; 31 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 791 MiB/s wr, 1.90k op/s 2024-09-19T08:50:11.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:11 smithi164 ceph-mon[30928]: pgmap v134: 97 pgs: 97 active+clean; 32 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 906 MiB/s wr, 3.21k op/s 2024-09-19T08:50:11.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:11 smithi170 ceph-mon[35072]: pgmap v134: 97 pgs: 97 active+clean; 32 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 906 MiB/s wr, 3.21k op/s 2024-09-19T08:50:11.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:11 smithi173 ceph-mon[35306]: pgmap v134: 97 pgs: 97 active+clean; 32 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 906 MiB/s wr, 3.21k op/s 2024-09-19T08:50:12.852 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:50:13.491 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:50:13.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:13 smithi170 ceph-mon[35072]: pgmap v135: 97 pgs: 97 active+clean; 32 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 740 MiB/s wr, 3.51k op/s 2024-09-19T08:50:13.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:13 smithi170 ceph-mon[35072]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T08:50:13.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:13 smithi173 ceph-mon[35306]: pgmap v135: 97 pgs: 97 active+clean; 32 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 740 MiB/s wr, 3.51k op/s 2024-09-19T08:50:13.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:13 smithi173 ceph-mon[35306]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T08:50:13.691 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:13 smithi164 ceph-mon[30928]: pgmap v135: 97 pgs: 97 active+clean; 32 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 740 MiB/s wr, 3.51k op/s 2024-09-19T08:50:13.691 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:13 smithi164 ceph-mon[30928]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T08:50:15.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:15 smithi170 ceph-mon[35072]: pgmap v136: 97 pgs: 97 active+clean; 32 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 704 MiB/s wr, 3.58k op/s 2024-09-19T08:50:15.635 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:15 smithi164 ceph-mon[30928]: pgmap v136: 97 pgs: 97 active+clean; 32 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 704 MiB/s wr, 3.58k op/s 2024-09-19T08:50:15.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:15 smithi173 ceph-mon[35306]: pgmap v136: 97 pgs: 97 active+clean; 32 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 704 MiB/s wr, 3.58k op/s 2024-09-19T08:50:17.314 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:50:17.314 INFO:teuthology.orchestra.run.smithi164.stdout: "status": "no active scrubs running", 2024-09-19T08:50:17.314 INFO:teuthology.orchestra.run.smithi164.stdout: "scrubs": {} 2024-09-19T08:50:17.314 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:50:17.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.685+0000 7f3070eda640 1 -- 172.21.15.164:0/380496796 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f306c109830 msgr2=0x7f306c109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:17.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.685+0000 7f3070eda640 1 --2- 172.21.15.164:0/380496796 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f306c109830 0x7f306c109c90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f305c009900 tx=0x7f305c02a400 comp rx=0 tx=0).stop 2024-09-19T08:50:17.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.688+0000 7f3070eda640 1 -- 172.21.15.164:0/380496796 shutdown_connections 2024-09-19T08:50:17.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.688+0000 7f3070eda640 1 --2- 172.21.15.164:0/380496796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f306c11b140 0x7f306c11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:17.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.688+0000 7f3070eda640 1 --2- 172.21.15.164:0/380496796 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f306c109830 0x7f306c109c90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:17.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.688+0000 7f3070eda640 1 --2- 172.21.15.164:0/380496796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f306c109450 0x7f306c110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:17.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.688+0000 7f3070eda640 1 -- 172.21.15.164:0/380496796 >> 172.21.15.164:0/380496796 conn(0x7f306c06e140 msgr2=0x7f306c06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:17.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.690+0000 7f3070eda640 1 -- 172.21.15.164:0/380496796 shutdown_connections 2024-09-19T08:50:17.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.692+0000 7f3070eda640 1 -- 172.21.15.164:0/380496796 wait complete. 2024-09-19T08:50:17.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.693+0000 7f3070eda640 1 Processor -- start 2024-09-19T08:50:17.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.693+0000 7f3070eda640 1 -- start start 2024-09-19T08:50:17.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.693+0000 7f3070eda640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f306c109450 0x7f306c11a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:17.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.693+0000 7f3070eda640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f306c109830 0x7f306c11ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:17.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.693+0000 7f3070eda640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f306c11b140 0x7f306c114740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:17.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.693+0000 7f3070eda640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f306c111060 con 0x7f306c109830 2024-09-19T08:50:17.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.693+0000 7f3070eda640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f306c110ee0 con 0x7f306c11b140 2024-09-19T08:50:17.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.693+0000 7f3070eda640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f306c1111e0 con 0x7f306c109450 2024-09-19T08:50:17.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.694+0000 7f306ad9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f306c11b140 0x7f306c114740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:17.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.694+0000 7f306ad9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f306c11b140 0x7f306c114740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:39364/0 (socket says 172.21.15.164:39364) 2024-09-19T08:50:17.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.694+0000 7f306ad9d640 1 -- 172.21.15.164:0/1244906179 learned_addr learned my addr 172.21.15.164:0/1244906179 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:17.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.694+0000 7f306ad9d640 1 -- 172.21.15.164:0/1244906179 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f306c109450 msgr2=0x7f306c11a5e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T08:50:17.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.694+0000 7f306ad9d640 1 --2- 172.21.15.164:0/1244906179 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f306c109450 0x7f306c11a5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:17.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.694+0000 7f306ad9d640 1 -- 172.21.15.164:0/1244906179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f306c109830 msgr2=0x7f306c11ab20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T08:50:17.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.694+0000 7f306ad9d640 1 --2- 172.21.15.164:0/1244906179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f306c109830 0x7f306c11ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:17.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.694+0000 7f306ad9d640 1 -- 172.21.15.164:0/1244906179 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f306c115000 con 0x7f306c11b140 2024-09-19T08:50:17.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.697+0000 7f306ad9d640 1 --2- 172.21.15.164:0/1244906179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f306c11b140 0x7f306c114740 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f306400e9f0 tx=0x7f306400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:17.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.699+0000 7f305b7fe640 1 -- 172.21.15.164:0/1244906179 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306400cd80 con 0x7f306c11b140 2024-09-19T08:50:17.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.700+0000 7f3070eda640 1 -- 172.21.15.164:0/1244906179 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f306c1152f0 con 0x7f306c11b140 2024-09-19T08:50:17.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.700+0000 7f3070eda640 1 -- 172.21.15.164:0/1244906179 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f306c1bc5f0 con 0x7f306c11b140 2024-09-19T08:50:17.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.700+0000 7f305b7fe640 1 -- 172.21.15.164:0/1244906179 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3064005020 con 0x7f306c11b140 2024-09-19T08:50:17.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.700+0000 7f305b7fe640 1 -- 172.21.15.164:0/1244906179 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30640116d0 con 0x7f306c11b140 2024-09-19T08:50:17.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.721+0000 7f30732b3740 1 Processor -- start 2024-09-19T08:50:17.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.722+0000 7f30732b3740 1 -- start start 2024-09-19T08:50:17.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.723+0000 7f305b7fe640 1 -- 172.21.15.164:0/1244906179 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f3064004a40 con 0x7f306c11b140 2024-09-19T08:50:17.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.725+0000 7f30732b3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576c65ca4a0 0x5576c65ca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:17.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.725+0000 7f30732b3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576c65cadc0 0x5576c65d5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:17.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.725+0000 7f30732b3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576c65d5680 0x5576c65d7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:17.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.725+0000 7f30732b3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5576c65302e0 con 0x5576c65ca4a0 2024-09-19T08:50:17.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.725+0000 7f30732b3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5576c64d7ce0 con 0x5576c65d5680 2024-09-19T08:50:17.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.725+0000 7f30732b3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5576c652c2a0 con 0x5576c65cadc0 2024-09-19T08:50:17.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.726+0000 7f305b7fe640 1 --2- 172.21.15.164:0/1244906179 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f30540a35c0 0x7f30540a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:17.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.726+0000 7f3069d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576c65d5680 0x5576c65d7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:17.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.726+0000 7f3069d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576c65d5680 0x5576c65d7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:39372/0 (socket says 172.21.15.164:39372) 2024-09-19T08:50:17.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.726+0000 7f3069d9b640 1 -- 172.21.15.164:0/2816108102 learned_addr learned my addr 172.21.15.164:0/2816108102 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:17.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.727+0000 7f305b7fe640 1 -- 172.21.15.164:0/1244906179 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f30640d01b0 con 0x7f306c11b140 2024-09-19T08:50:17.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.728+0000 7f3069d9b640 1 -- 172.21.15.164:0/2816108102 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576c65cadc0 msgr2=0x5576c65d5140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:50:17.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.728+0000 7f3069d9b640 1 --2- 172.21.15.164:0/2816108102 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576c65cadc0 0x5576c65d5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:17.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.728+0000 7f3069d9b640 1 -- 172.21.15.164:0/2816108102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576c65ca4a0 msgr2=0x5576c65ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:17.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.728+0000 7f3069d9b640 1 --2- 172.21.15.164:0/2816108102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576c65ca4a0 0x5576c65ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:17.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.728+0000 7f3069d9b640 1 -- 172.21.15.164:0/2816108102 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576c64bc440 con 0x5576c65d5680 2024-09-19T08:50:17.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.729+0000 7f3069d9b640 1 --2- 172.21.15.164:0/2816108102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576c65d5680 0x5576c65d7a70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f305c005dc0 tx=0x7f305c031500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28519373e609a830 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:17.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.730+0000 7f306ad9d640 1 --2- 172.21.15.164:0/1244906179 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f30540a35c0 0x7f30540a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:17.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.732+0000 7f3058ff9640 1 -- 172.21.15.164:0/2816108102 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c040070 con 0x5576c65d5680 2024-09-19T08:50:17.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.732+0000 7f3058ff9640 1 -- 172.21.15.164:0/2816108102 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f305c031d10 con 0x5576c65d5680 2024-09-19T08:50:17.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.732+0000 7f3058ff9640 1 -- 172.21.15.164:0/2816108102 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c035030 con 0x5576c65d5680 2024-09-19T08:50:17.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.733+0000 7f30732b3740 1 -- 172.21.15.164:0/2816108102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576c65d5680 msgr2=0x5576c65d7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:17.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.733+0000 7f30732b3740 1 --2- 172.21.15.164:0/2816108102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576c65d5680 0x5576c65d7a70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f305c005dc0 tx=0x7f305c031500 comp rx=0 tx=0).stop 2024-09-19T08:50:17.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.734+0000 7f306ad9d640 1 --2- 172.21.15.164:0/1244906179 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f30540a35c0 0x7f30540a5a80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f306409c430 tx=0x7f3064005b20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:17.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.735+0000 7f30732b3740 1 -- 172.21.15.164:0/2816108102 shutdown_connections 2024-09-19T08:50:17.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.735+0000 7f30732b3740 1 --2- 172.21.15.164:0/2816108102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576c65d5680 0x5576c65d7a70 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:17.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.735+0000 7f30732b3740 1 --2- 172.21.15.164:0/2816108102 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576c65cadc0 0x5576c65d5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:17.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.735+0000 7f30732b3740 1 --2- 172.21.15.164:0/2816108102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576c65ca4a0 0x5576c65ca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:17.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.735+0000 7f30732b3740 1 -- 172.21.15.164:0/2816108102 >> 172.21.15.164:0/2816108102 conn(0x5576c65c6a30 msgr2=0x5576c65c6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:17.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.738+0000 7f30732b3740 1 -- 172.21.15.164:0/2816108102 shutdown_connections 2024-09-19T08:50:17.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.738+0000 7f30732b3740 1 -- 172.21.15.164:0/2816108102 wait complete. 2024-09-19T08:50:17.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.739+0000 7f30732b3740 1 -- start start 2024-09-19T08:50:17.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.739+0000 7f30732b3740 1 Processor -- start 2024-09-19T08:50:17.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.740+0000 7f30732b3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576c65ca4a0 0x5576c65d4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:17.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.740+0000 7f30732b3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576c65cadc0 0x5576c65d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:17.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.740+0000 7f3069d9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576c65ca4a0 0x5576c65d4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:17.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.741+0000 7f3069d9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576c65ca4a0 0x5576c65d4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:33678/0 (socket says 172.21.15.164:33678) 2024-09-19T08:50:17.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.741+0000 7f3069d9b640 1 -- 172.21.15.164:0/363523852 learned_addr learned my addr 172.21.15.164:0/363523852 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:17.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.741+0000 7f30732b3740 1 --2- 172.21.15.164:0/363523852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576c65d5680 0x5576c65d27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:17.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.741+0000 7f30732b3740 1 -- 172.21.15.164:0/363523852 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5576c65d8250 con 0x5576c65cadc0 2024-09-19T08:50:17.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.741+0000 7f30732b3740 1 -- 172.21.15.164:0/363523852 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5576c64eb050 con 0x5576c65d5680 2024-09-19T08:50:17.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.741+0000 7f30732b3740 1 -- 172.21.15.164:0/363523852 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5576c65d83d0 con 0x5576c65ca4a0 2024-09-19T08:50:17.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.741+0000 7f3069d9b640 1 --2- 172.21.15.164:0/363523852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576c65d5680 0x5576c65d27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:17.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.742+0000 7f3069d9b640 1 -- 172.21.15.164:0/363523852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576c65d5680 msgr2=0x5576c65d27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:17.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.742+0000 7f3069d9b640 1 --2- 172.21.15.164:0/363523852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576c65d5680 0x5576c65d27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:17.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.742+0000 7f3069d9b640 1 -- 172.21.15.164:0/363523852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576c65cadc0 msgr2=0x5576c65d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:17.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.742+0000 7f3069d9b640 1 --2- 172.21.15.164:0/363523852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576c65cadc0 0x5576c65d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:17.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.742+0000 7f3069d9b640 1 -- 172.21.15.164:0/363523852 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576c62a13c0 con 0x5576c65ca4a0 2024-09-19T08:50:17.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.742+0000 7f30732b3740 10 client.0 _ll_register_callbacks cb 0x5576c652b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:50:17.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.742+0000 7f3069d9b640 1 --2- 172.21.15.164:0/363523852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576c65ca4a0 0x5576c65d4fc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f305c032fd0 tx=0x7f305c03cb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0ded66048206e13 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:17.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.743+0000 7f30732b3740 1 -- 172.21.15.164:0/363523852 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f305c031d10 con 0x5576c65ca4a0 2024-09-19T08:50:17.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.744+0000 7f3051ffb640 10 client.25765 ms_handle_connect on v2:172.21.15.173:3300/0 2024-09-19T08:50:17.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.744+0000 7f3051ffb640 1 -- 172.21.15.164:0/363523852 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c04d030 con 0x5576c65ca4a0 2024-09-19T08:50:17.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.745+0000 7f3051ffb640 1 -- 172.21.15.164:0/363523852 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f305c031900 con 0x5576c65ca4a0 2024-09-19T08:50:17.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.745+0000 7f3051ffb640 1 -- 172.21.15.164:0/363523852 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f305c059050 con 0x5576c65ca4a0 2024-09-19T08:50:17.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.748+0000 7f30732b3740 10 client.25765 fetch_fsmap learned FSMap version 29 2024-09-19T08:50:17.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.748+0000 7f30732b3740 1 -- 172.21.15.164:0/363523852 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({fsmap=29}) -- 0x5576c65e3890 con 0x5576c65ca4a0 2024-09-19T08:50:17.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.750+0000 7f3051ffb640 1 -- 172.21.15.164:0/363523852 <== mon.1 v2:172.21.15.173:3300/0 4 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f305c03a040 con 0x5576c65ca4a0 2024-09-19T08:50:17.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.750+0000 7f30732b3740 10 client.25765 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T08:50:17.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.750+0000 7f30732b3740 10 client.25765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:50:17.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.750+0000 7f30732b3740 20 client.25765 populate_metadata read hostname 'smithi164' 2024-09-19T08:50:17.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.750+0000 7f30732b3740 1 --2- 172.21.15.164:0/363523852 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5576c65e4be0 0x5576c65e6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:17.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.751+0000 7f30732b3740 4 client.25765 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:50:17.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.751+0000 7f30732b3740 1 -- 172.21.15.164:0/363523852 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576c653d850 con 0x5576c65e4be0 2024-09-19T08:50:17.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.755+0000 7f306a59c640 1 --2- 172.21.15.164:0/363523852 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5576c65e4be0 0x5576c65e6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:17.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.759+0000 7f306a59c640 1 --2- 172.21.15.164:0/363523852 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5576c65e4be0 0x5576c65e6fd0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a76fe8b6aa74336 server_cookie=f3ebda41b2e0a96b in_seq=0 out_seq=0 2024-09-19T08:50:17.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.760+0000 7f3051ffb640 10 client.25765 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:50:17.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.760+0000 7f3051ffb640 1 -- 172.21.15.164:0/363523852 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5576c653d850 con 0x5576c65e4be0 2024-09-19T08:50:17.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.761+0000 7f3051ffb640 10 client.25765 handle_command_reply: tid=0 2024-09-19T08:50:17.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.761+0000 7f30732b3740 1 client.25765 shutdown 2024-09-19T08:50:17.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.764+0000 7f30732b3740 1 -- 172.21.15.164:0/363523852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576c65ca4a0 msgr2=0x5576c65d4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:17.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.764+0000 7f30732b3740 1 --2- 172.21.15.164:0/363523852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576c65ca4a0 0x5576c65d4fc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f305c032fd0 tx=0x7f305c03cb20 comp rx=0 tx=0).stop 2024-09-19T08:50:17.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.764+0000 7f30732b3740 1 -- 172.21.15.164:0/363523852 shutdown_connections 2024-09-19T08:50:17.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.764+0000 7f30732b3740 1 --2- 172.21.15.164:0/363523852 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5576c65e4be0 0x5576c65e6fd0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:17.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.764+0000 7f30732b3740 1 --2- 172.21.15.164:0/363523852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576c65d5680 0x5576c65d27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:17.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.764+0000 7f3051ffb640 0 client.25765 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:50:17.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.765+0000 7f30732b3740 1 --2- 172.21.15.164:0/363523852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576c65cadc0 0x5576c65d2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:17.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.765+0000 7f30732b3740 1 --2- 172.21.15.164:0/363523852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576c65ca4a0 0x5576c65d4fc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:17.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.765+0000 7f30732b3740 1 -- 172.21.15.164:0/363523852 >> 172.21.15.164:0/363523852 conn(0x5576c65c6a30 msgr2=0x5576c65cc760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:17.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.765+0000 7f30732b3740 1 -- 172.21.15.164:0/363523852 shutdown_connections 2024-09-19T08:50:17.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.765+0000 7f30732b3740 1 -- 172.21.15.164:0/363523852 wait complete. 2024-09-19T08:50:17.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.768+0000 7f30732b3740 20 client.25765 trim_cache size 0 max 16384 2024-09-19T08:50:17.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.813+0000 7f30732b3740 1 Processor -- start 2024-09-19T08:50:17.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.813+0000 7f30732b3740 1 -- start start 2024-09-19T08:50:17.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.814+0000 7f30732b3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576c65ca4a0 0x5576c65e1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:17.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.814+0000 7f30732b3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576c65cadc0 0x5576c65ec780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:17.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.814+0000 7f3069d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576c65ca4a0 0x5576c65e1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:17.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.814+0000 7f3069d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576c65ca4a0 0x5576c65e1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:39390/0 (socket says 172.21.15.164:39390) 2024-09-19T08:50:17.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.814+0000 7f3069d9b640 1 -- 172.21.15.164:0/1783921186 learned_addr learned my addr 172.21.15.164:0/1783921186 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:17.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.814+0000 7f30732b3740 1 --2- 172.21.15.164:0/1783921186 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576c65d5680 0x5576c65eccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:17.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.814+0000 7f30732b3740 1 -- 172.21.15.164:0/1783921186 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f304c00e670 con 0x5576c65cadc0 2024-09-19T08:50:17.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.814+0000 7f30732b3740 1 -- 172.21.15.164:0/1783921186 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f304c0101e0 con 0x5576c65ca4a0 2024-09-19T08:50:17.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.814+0000 7f30732b3740 1 -- 172.21.15.164:0/1783921186 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f304c00ed40 con 0x5576c65d5680 2024-09-19T08:50:17.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.816+0000 7f3069d9b640 1 --2- 172.21.15.164:0/1783921186 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576c65d5680 0x5576c65eccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:17.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.816+0000 7f3069d9b640 1 -- 172.21.15.164:0/1783921186 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576c65d5680 msgr2=0x5576c65eccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:17.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.816+0000 7f3069d9b640 1 --2- 172.21.15.164:0/1783921186 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576c65d5680 0x5576c65eccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:17.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.816+0000 7f3069d9b640 1 -- 172.21.15.164:0/1783921186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576c65cadc0 msgr2=0x5576c65ec780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:17.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.816+0000 7f3069d9b640 1 --2- 172.21.15.164:0/1783921186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576c65cadc0 0x5576c65ec780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:17.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.816+0000 7f3069d9b640 1 -- 172.21.15.164:0/1783921186 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576c65ca2d0 con 0x5576c65ca4a0 2024-09-19T08:50:17.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.817+0000 7f3069d9b640 1 --2- 172.21.15.164:0/1783921186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576c65ca4a0 0x5576c65e1540 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f305c02abf0 tx=0x7f305c046000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d4293e826ef2ac9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:17.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.818+0000 7f3053fff640 1 -- 172.21.15.164:0/1783921186 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c04d030 con 0x5576c65ca4a0 2024-09-19T08:50:17.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.818+0000 7f3053fff640 1 -- 172.21.15.164:0/1783921186 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f305c031d10 con 0x5576c65ca4a0 2024-09-19T08:50:17.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.819+0000 7f30732b3740 1 -- 172.21.15.164:0/1783921186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576c65ca4a0 msgr2=0x5576c65e1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:17.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.819+0000 7f30732b3740 1 --2- 172.21.15.164:0/1783921186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576c65ca4a0 0x5576c65e1540 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f305c02abf0 tx=0x7f305c046000 comp rx=0 tx=0).stop 2024-09-19T08:50:17.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.820+0000 7f30732b3740 1 -- 172.21.15.164:0/1783921186 shutdown_connections 2024-09-19T08:50:17.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.820+0000 7f30732b3740 1 --2- 172.21.15.164:0/1783921186 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576c65d5680 0x5576c65eccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:17.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.820+0000 7f30732b3740 1 --2- 172.21.15.164:0/1783921186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576c65cadc0 0x5576c65ec780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:17.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.820+0000 7f30732b3740 1 --2- 172.21.15.164:0/1783921186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576c65ca4a0 0x5576c65e1540 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:17.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.820+0000 7f30732b3740 1 -- 172.21.15.164:0/1783921186 >> 172.21.15.164:0/1783921186 conn(0x5576c65c6a30 msgr2=0x5576c65cc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:17.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.820+0000 7f30732b3740 1 -- 172.21.15.164:0/1783921186 shutdown_connections 2024-09-19T08:50:17.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.820+0000 7f30732b3740 1 -- 172.21.15.164:0/1783921186 wait complete. 2024-09-19T08:50:17.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.821+0000 7f30732b3740 1 -- start start 2024-09-19T08:50:17.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.821+0000 7f30732b3740 1 Processor -- start 2024-09-19T08:50:17.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.822+0000 7f30732b3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576c65cadc0 0x5576c65dd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:17.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.822+0000 7f30732b3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576c65d5680 0x5576c65ddb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:17.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.822+0000 7f30732b3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576c65e4be0 0x5576c65da5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:17.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.822+0000 7f30732b3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f304c00fae0 con 0x5576c65cadc0 2024-09-19T08:50:17.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.822+0000 7f30732b3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f304c00f410 con 0x5576c65d5680 2024-09-19T08:50:17.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.822+0000 7f30732b3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f304c00c650 con 0x5576c65e4be0 2024-09-19T08:50:17.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.822+0000 7f30732b3740 10 client.0 _ll_register_callbacks cb 0x7f305c031400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:50:17.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.825+0000 7f306a59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576c65d5680 0x5576c65ddb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:17.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.825+0000 7f306a59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576c65d5680 0x5576c65ddb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:39402/0 (socket says 172.21.15.164:39402) 2024-09-19T08:50:17.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.825+0000 7f306a59c640 1 -- 172.21.15.164:0/2222112854 learned_addr learned my addr 172.21.15.164:0/2222112854 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:17.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.825+0000 7f3069d9b640 1 --2- 172.21.15.164:0/2222112854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576c65cadc0 0x5576c65dd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:17.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.825+0000 7f3069d9b640 1 --2- 172.21.15.164:0/2222112854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576c65e4be0 0x5576c65da5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:17.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.825+0000 7f306a59c640 1 -- 172.21.15.164:0/2222112854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576c65e4be0 msgr2=0x5576c65da5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:17.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.825+0000 7f306a59c640 1 --2- 172.21.15.164:0/2222112854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576c65e4be0 0x5576c65da5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:17.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.825+0000 7f306a59c640 1 -- 172.21.15.164:0/2222112854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576c65cadc0 msgr2=0x5576c65dd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:17.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.825+0000 7f306a59c640 1 --2- 172.21.15.164:0/2222112854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576c65cadc0 0x5576c65dd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:17.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.825+0000 7f306a59c640 1 -- 172.21.15.164:0/2222112854 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576c65de1d0 con 0x5576c65d5680 2024-09-19T08:50:17.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.831+0000 7f306a59c640 1 --2- 172.21.15.164:0/2222112854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576c65d5680 0x5576c65ddb10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3060004810 tx=0x7f3060009f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b4cac8300723ab4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:17.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.831+0000 7f3051ffb640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-19T08:50:17.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.834+0000 7f3051ffb640 1 -- 172.21.15.164:0/2222112854 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3060022060 con 0x5576c65d5680 2024-09-19T08:50:17.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.835+0000 7f30732b3740 1 -- 172.21.15.164:0/2222112854 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5576c6537d60 con 0x5576c65d5680 2024-09-19T08:50:17.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.835+0000 7f3051ffb640 1 -- 172.21.15.164:0/2222112854 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f3060012be0 con 0x5576c65d5680 2024-09-19T08:50:17.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.835+0000 7f3051ffb640 1 -- 172.21.15.164:0/2222112854 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3060012de0 con 0x5576c65d5680 2024-09-19T08:50:17.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.835+0000 7f30732b3740 10 client.25781 fetch_fsmap learned FSMap version 29 2024-09-19T08:50:17.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.836+0000 7f30732b3740 1 -- 172.21.15.164:0/2222112854 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5576c664fd90 con 0x5576c65d5680 2024-09-19T08:50:17.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.837+0000 7f3051ffb640 1 -- 172.21.15.164:0/2222112854 <== mon.2 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5576c6537d60 con 0x5576c65d5680 2024-09-19T08:50:17.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.837+0000 7f3051ffb640 1 -- 172.21.15.164:0/2222112854 <== mon.2 v2:172.21.15.170:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f3060025020 con 0x5576c65d5680 2024-09-19T08:50:17.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.837+0000 7f30732b3740 10 client.25781 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T08:50:17.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.837+0000 7f30732b3740 10 client.25781 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:50:17.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.837+0000 7f30732b3740 20 client.25781 populate_metadata read hostname 'smithi164' 2024-09-19T08:50:17.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.837+0000 7f30732b3740 1 --2- 172.21.15.164:0/2222112854 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5576c6650570 0x5576c6652990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:17.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.837+0000 7f30732b3740 4 client.25781 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:50:17.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.837+0000 7f30732b3740 1 -- 172.21.15.164:0/2222112854 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5576c64b4210 con 0x5576c6650570 2024-09-19T08:50:17.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.840+0000 7f3069d9b640 1 --2- 172.21.15.164:0/2222112854 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5576c6650570 0x5576c6652990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:17.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.848+0000 7f3069d9b640 1 --2- 172.21.15.164:0/2222112854 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5576c6650570 0x5576c6652990 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d691cdb7226d887b server_cookie=8aff78e268b73318 in_seq=0 out_seq=0 2024-09-19T08:50:17.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.848+0000 7f3051ffb640 10 client.25781 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:50:17.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.851+0000 7f3051ffb640 1 -- 172.21.15.164:0/2222112854 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5576c64b4210 con 0x5576c6650570 2024-09-19T08:50:17.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.851+0000 7f3051ffb640 10 client.25781 handle_command_reply: tid=0 2024-09-19T08:50:17.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.851+0000 7f30732b3740 1 client.25781 shutdown 2024-09-19T08:50:17.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.851+0000 7f30732b3740 1 -- 172.21.15.164:0/2222112854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576c65d5680 msgr2=0x5576c65ddb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:17.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.851+0000 7f30732b3740 1 --2- 172.21.15.164:0/2222112854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576c65d5680 0x5576c65ddb10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3060004810 tx=0x7f3060009f10 comp rx=0 tx=0).stop 2024-09-19T08:50:17.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.851+0000 7f30732b3740 1 -- 172.21.15.164:0/2222112854 shutdown_connections 2024-09-19T08:50:17.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.851+0000 7f30732b3740 1 --2- 172.21.15.164:0/2222112854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5576c65e4be0 0x5576c65da5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:17.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.851+0000 7f30732b3740 1 --2- 172.21.15.164:0/2222112854 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5576c6650570 0x5576c6652990 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:17.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.851+0000 7f30732b3740 1 --2- 172.21.15.164:0/2222112854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576c65d5680 0x5576c65ddb10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:17.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.851+0000 7f30732b3740 1 --2- 172.21.15.164:0/2222112854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5576c65cadc0 0x5576c65dd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:17.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.851+0000 7f30732b3740 1 -- 172.21.15.164:0/2222112854 >> 172.21.15.164:0/2222112854 conn(0x5576c65c6a30 msgr2=0x5576c65d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:17.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.851+0000 7f3051ffb640 0 client.25781 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:50:17.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.852+0000 7f30732b3740 1 -- 172.21.15.164:0/2222112854 shutdown_connections 2024-09-19T08:50:17.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.852+0000 7f30732b3740 1 -- 172.21.15.164:0/2222112854 wait complete. 2024-09-19T08:50:17.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.852+0000 7f30732b3740 20 client.25781 trim_cache size 0 max 16384 2024-09-19T08:50:17.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.855+0000 7f3070eda640 1 -- 172.21.15.164:0/1244906179 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f30540a35c0 msgr2=0x7f30540a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:17.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.855+0000 7f3070eda640 1 --2- 172.21.15.164:0/1244906179 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f30540a35c0 0x7f30540a5a80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f306409c430 tx=0x7f3064005b20 comp rx=0 tx=0).stop 2024-09-19T08:50:17.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.855+0000 7f3070eda640 1 -- 172.21.15.164:0/1244906179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f306c11b140 msgr2=0x7f306c114740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:17.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.855+0000 7f3070eda640 1 --2- 172.21.15.164:0/1244906179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f306c11b140 0x7f306c114740 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f306400e9f0 tx=0x7f306400eeb0 comp rx=0 tx=0).stop 2024-09-19T08:50:17.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.856+0000 7f3070eda640 1 -- 172.21.15.164:0/1244906179 shutdown_connections 2024-09-19T08:50:17.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.856+0000 7f3070eda640 1 --2- 172.21.15.164:0/1244906179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f306c11b140 0x7f306c114740 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:17.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.856+0000 7f3070eda640 1 --2- 172.21.15.164:0/1244906179 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f30540a35c0 0x7f30540a5a80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:17.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.856+0000 7f3070eda640 1 --2- 172.21.15.164:0/1244906179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f306c109830 0x7f306c11ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:17.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.856+0000 7f3070eda640 1 --2- 172.21.15.164:0/1244906179 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f306c109450 0x7f306c11a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:17.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.856+0000 7f3070eda640 1 -- 172.21.15.164:0/1244906179 >> 172.21.15.164:0/1244906179 conn(0x7f306c06e140 msgr2=0x7f306c070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:17.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.857+0000 7f3070eda640 1 -- 172.21.15.164:0/1244906179 shutdown_connections 2024-09-19T08:50:17.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:16.857+0000 7f3070eda640 1 -- 172.21.15.164:0/1244906179 wait complete. 2024-09-19T08:50:17.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:17 smithi170 ceph-mon[35072]: pgmap v137: 97 pgs: 97 active+clean; 33 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 790 MiB/s wr, 4.23k op/s 2024-09-19T08:50:17.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:17 smithi173 ceph-mon[35306]: pgmap v137: 97 pgs: 97 active+clean; 33 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 790 MiB/s wr, 4.23k op/s 2024-09-19T08:50:17.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:17 smithi164 ceph-mon[30928]: pgmap v137: 97 pgs: 97 active+clean; 33 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 790 MiB/s wr, 4.23k op/s 2024-09-19T08:50:18.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:18 smithi170 ceph-mon[35072]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T08:50:18.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:18 smithi173 ceph-mon[35306]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T08:50:18.715 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:18 smithi164 ceph-mon[30928]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T08:50:19.053 INFO:tasks.quiescer.fs.[cephfs]:Total duration (59.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T08:50:19.053 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.8 2024-09-19T08:50:19.054 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:50:19.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:19 smithi164 ceph-mon[30928]: pgmap v138: 97 pgs: 97 active+clean; 34 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 654 MiB/s wr, 4.01k op/s 2024-09-19T08:50:19.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:19 smithi173 ceph-mon[35306]: pgmap v138: 97 pgs: 97 active+clean; 34 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 654 MiB/s wr, 4.01k op/s 2024-09-19T08:50:19.815 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T08:50:19.816 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:50:19.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:19 smithi170 ceph-mon[35072]: pgmap v138: 97 pgs: 97 active+clean; 34 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 654 MiB/s wr, 4.01k op/s 2024-09-19T08:50:20.314 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:50:21.396 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:21 smithi164 ceph-mon[30928]: pgmap v139: 97 pgs: 97 active+clean; 36 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 757 MiB/s wr, 4.27k op/s 2024-09-19T08:50:21.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:21 smithi173 ceph-mon[35306]: pgmap v139: 97 pgs: 97 active+clean; 36 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 757 MiB/s wr, 4.27k op/s 2024-09-19T08:50:21.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.763+0000 7fefd6375640 1 -- 172.21.15.164:0/1707615822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefd011b140 msgr2=0x7fefd011d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:21.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.763+0000 7fefd6375640 1 --2- 172.21.15.164:0/1707615822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefd011b140 0x7fefd011d530 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fefc8009ca0 tx=0x7fefc802a8f0 comp rx=0 tx=0).stop 2024-09-19T08:50:21.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.764+0000 7fefd6375640 1 -- 172.21.15.164:0/1707615822 shutdown_connections 2024-09-19T08:50:21.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.764+0000 7fefd6375640 1 --2- 172.21.15.164:0/1707615822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefd011b140 0x7fefd011d530 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:21.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.764+0000 7fefd6375640 1 --2- 172.21.15.164:0/1707615822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefd006ea30 0x7fefd006ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:21.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.764+0000 7fefd6375640 1 --2- 172.21.15.164:0/1707615822 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefd0110190 0x7fefd006e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:21.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.764+0000 7fefd6375640 1 -- 172.21.15.164:0/1707615822 >> 172.21.15.164:0/1707615822 conn(0x7fefd0073180 msgr2=0x7fefd0073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:21.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.764+0000 7fefd6375640 1 -- 172.21.15.164:0/1707615822 shutdown_connections 2024-09-19T08:50:21.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.765+0000 7fefd6375640 1 -- 172.21.15.164:0/1707615822 wait complete. 2024-09-19T08:50:21.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.775+0000 7fefd6375640 1 Processor -- start 2024-09-19T08:50:21.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.786+0000 7fefd6375640 1 -- start start 2024-09-19T08:50:21.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.787+0000 7fefd6375640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefd006ea30 0x7fefd01a7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:21.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.788+0000 7fefd6375640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefd0110190 0x7fefd01a7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:21.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.788+0000 7fefd6375640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefd011b140 0x7fefd01abb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:21.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.788+0000 7fefd6375640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fefd0120c70 con 0x7fefd006ea30 2024-09-19T08:50:21.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.788+0000 7fefd6375640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fefd0120af0 con 0x7fefd011b140 2024-09-19T08:50:21.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.788+0000 7fefd6375640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fefd0120df0 con 0x7fefd0110190 2024-09-19T08:50:21.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.788+0000 7fefd5373640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefd006ea30 0x7fefd01a7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:21.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.789+0000 7fefd5b74640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefd011b140 0x7fefd01abb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:21.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.789+0000 7fefd5b74640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefd011b140 0x7fefd01abb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:39422/0 (socket says 172.21.15.164:39422) 2024-09-19T08:50:21.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.789+0000 7fefd5b74640 1 -- 172.21.15.164:0/1742883318 learned_addr learned my addr 172.21.15.164:0/1742883318 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:21.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.790+0000 7fefd5373640 1 -- 172.21.15.164:0/1742883318 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefd0110190 msgr2=0x7fefd01a7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:21.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.790+0000 7fefd5373640 1 --2- 172.21.15.164:0/1742883318 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefd0110190 0x7fefd01a7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:21.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.790+0000 7fefd5373640 1 -- 172.21.15.164:0/1742883318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefd011b140 msgr2=0x7fefd01abb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:21.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.790+0000 7fefd5373640 1 --2- 172.21.15.164:0/1742883318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefd011b140 0x7fefd01abb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:21.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.790+0000 7fefd5373640 1 -- 172.21.15.164:0/1742883318 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd01ac200 con 0x7fefd006ea30 2024-09-19T08:50:21.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.791+0000 7fefd5373640 1 --2- 172.21.15.164:0/1742883318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefd006ea30 0x7fefd01a7250 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fefcc00b3f0 tx=0x7fefcc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:21.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.791+0000 7fefc67fc640 1 -- 172.21.15.164:0/1742883318 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc015020 con 0x7fefd006ea30 2024-09-19T08:50:21.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.792+0000 7fefd6375640 1 -- 172.21.15.164:0/1742883318 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefd01ac4f0 con 0x7fefd006ea30 2024-09-19T08:50:21.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.792+0000 7fefd6375640 1 -- 172.21.15.164:0/1742883318 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefd01b3dd0 con 0x7fefd006ea30 2024-09-19T08:50:21.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.795+0000 7fefc67fc640 1 -- 172.21.15.164:0/1742883318 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefcc011d10 con 0x7fefd006ea30 2024-09-19T08:50:21.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.795+0000 7fefc67fc640 1 -- 172.21.15.164:0/1742883318 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc010e60 con 0x7fefd006ea30 2024-09-19T08:50:21.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.802+0000 7fefc67fc640 1 -- 172.21.15.164:0/1742883318 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fefcc018020 con 0x7fefd006ea30 2024-09-19T08:50:21.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.801+0000 7fefbbfff640 1 -- 172.21.15.164:0/1742883318 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefd006eef0 con 0x7fefd006ea30 2024-09-19T08:50:21.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.805+0000 7fefc67fc640 1 --2- 172.21.15.164:0/1742883318 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fefbc0a35c0 0x7fefbc0a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:21.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.805+0000 7fefd4b72640 1 --2- 172.21.15.164:0/1742883318 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fefbc0a35c0 0x7fefbc0a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:21.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.806+0000 7fefc67fc640 1 -- 172.21.15.164:0/1742883318 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fefcc0cf340 con 0x7fefd006ea30 2024-09-19T08:50:21.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.812+0000 7fefd4b72640 1 --2- 172.21.15.164:0/1742883318 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fefbc0a35c0 0x7fefbc0a5a80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fefd011acd0 tx=0x7fefc0006c40 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:21.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:21.819+0000 7fefc67fc640 1 -- 172.21.15.164:0/1742883318 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fefcc01f020 con 0x7fefd006ea30 2024-09-19T08:50:21.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:21 smithi170 ceph-mon[35072]: pgmap v139: 97 pgs: 97 active+clean; 36 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 757 MiB/s wr, 4.27k op/s 2024-09-19T08:50:21.989 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:50:22.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:22.321+0000 7fefbbfff640 1 -- 172.21.15.164:0/1742883318 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fefd01a85b0 con 0x7fefd006ea30 2024-09-19T08:50:22.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:22.327+0000 7fefc67fc640 1 -- 172.21.15.164:0/1742883318 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11991 (secure 0 0 0) 0x7fefcc09be40 con 0x7fefd006ea30 2024-09-19T08:50:22.327 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:50:22.327 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":30,"btime":"2024-09-19T08:50:17:326587+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:50:17.071374+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:50:22.328 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 30 2024-09-19T08:50:22.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:22.333+0000 7fefbbfff640 1 -- 172.21.15.164:0/1742883318 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fefbc0a35c0 msgr2=0x7fefbc0a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:22.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:22.333+0000 7fefbbfff640 1 --2- 172.21.15.164:0/1742883318 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fefbc0a35c0 0x7fefbc0a5a80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fefd011acd0 tx=0x7fefc0006c40 comp rx=0 tx=0).stop 2024-09-19T08:50:22.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:22.334+0000 7fefbbfff640 1 -- 172.21.15.164:0/1742883318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefd006ea30 msgr2=0x7fefd01a7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:22.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:22.334+0000 7fefbbfff640 1 --2- 172.21.15.164:0/1742883318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefd006ea30 0x7fefd01a7250 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fefcc00b3f0 tx=0x7fefcc00b8b0 comp rx=0 tx=0).stop 2024-09-19T08:50:22.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:22.336+0000 7fefbbfff640 1 -- 172.21.15.164:0/1742883318 shutdown_connections 2024-09-19T08:50:22.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:22.336+0000 7fefbbfff640 1 --2- 172.21.15.164:0/1742883318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefd011b140 0x7fefd01abb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:22.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:22.336+0000 7fefbbfff640 1 --2- 172.21.15.164:0/1742883318 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefd0110190 0x7fefd01a7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:22.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:22.336+0000 7fefbbfff640 1 --2- 172.21.15.164:0/1742883318 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fefbc0a35c0 0x7fefbc0a5a80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:22.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:22.336+0000 7fefbbfff640 1 --2- 172.21.15.164:0/1742883318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefd006ea30 0x7fefd01a7250 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:22.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:22.336+0000 7fefbbfff640 1 -- 172.21.15.164:0/1742883318 >> 172.21.15.164:0/1742883318 conn(0x7fefd0073180 msgr2=0x7fefd01194d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:22.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:22.337+0000 7fefbbfff640 1 -- 172.21.15.164:0/1742883318 shutdown_connections 2024-09-19T08:50:22.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:22.337+0000 7fefbbfff640 1 -- 172.21.15.164:0/1742883318 wait complete. 2024-09-19T08:50:22.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:22 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/1742883318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:50:22.874 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:22 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/1742883318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:50:22.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:22 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/1742883318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:50:23.783 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:23 smithi164 ceph-mon[30928]: pgmap v140: 97 pgs: 97 active+clean; 37 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 640 MiB/s wr, 2.88k op/s 2024-09-19T08:50:23.874 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:23 smithi170 ceph-mon[35072]: pgmap v140: 97 pgs: 97 active+clean; 37 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 640 MiB/s wr, 2.88k op/s 2024-09-19T08:50:23.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:23 smithi173 ceph-mon[35306]: pgmap v140: 97 pgs: 97 active+clean; 37 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 640 MiB/s wr, 2.88k op/s 2024-09-19T08:50:25.711 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0' 2024-09-19T08:50:25.711 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0 2024-09-19T08:50:25.862 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:25 smithi164 ceph-mon[30928]: pgmap v141: 97 pgs: 97 active+clean; 37 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 634 MiB/s wr, 2.24k op/s 2024-09-19T08:50:25.862 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:25 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/1733040598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:50:25.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:25 smithi170 ceph-mon[35072]: pgmap v141: 97 pgs: 97 active+clean; 37 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 634 MiB/s wr, 2.24k op/s 2024-09-19T08:50:25.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:25 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/1733040598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:50:25.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:25 smithi173 ceph-mon[35306]: pgmap v141: 97 pgs: 97 active+clean; 37 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 634 MiB/s wr, 2.24k op/s 2024-09-19T08:50:25.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:25 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/1733040598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:50:26.326 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:50:26.327 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":30,"btime":"2024-09-19T08:50:17:326587+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:50:17.071374+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:50:26.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.600+0000 7f712359e640 1 -- 172.21.15.164:0/3748527321 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f712411b140 msgr2=0x7f712411d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:26.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.600+0000 7f712359e640 1 --2- 172.21.15.164:0/3748527321 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f712411b140 0x7f712411d530 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7118009900 tx=0x7f711802a440 comp rx=0 tx=0).stop 2024-09-19T08:50:26.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.602+0000 7f712359e640 1 -- 172.21.15.164:0/3748527321 shutdown_connections 2024-09-19T08:50:26.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.602+0000 7f712359e640 1 --2- 172.21.15.164:0/3748527321 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f712411b140 0x7f712411d530 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:26.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.602+0000 7f712359e640 1 --2- 172.21.15.164:0/3748527321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f712406ea30 0x7f712406ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:26.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.602+0000 7f712359e640 1 --2- 172.21.15.164:0/3748527321 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7124110190 0x7f712406e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:26.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.602+0000 7f712359e640 1 -- 172.21.15.164:0/3748527321 >> 172.21.15.164:0/3748527321 conn(0x7f7124073180 msgr2=0x7f7124073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:26.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.603+0000 7f712359e640 1 -- 172.21.15.164:0/3748527321 shutdown_connections 2024-09-19T08:50:26.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.604+0000 7f712359e640 1 -- 172.21.15.164:0/3748527321 wait complete. 2024-09-19T08:50:26.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.606+0000 7f712359e640 1 Processor -- start 2024-09-19T08:50:26.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.606+0000 7f712359e640 1 -- start start 2024-09-19T08:50:26.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.606+0000 7f712359e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f712406ea30 0x7f71241a7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:26.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.606+0000 7f712359e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7124110190 0x7f71241a7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:26.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.607+0000 7f712359e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f712411b140 0x7f71241ab900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:26.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.607+0000 7f712359e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7124120b90 con 0x7f7124110190 2024-09-19T08:50:26.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.607+0000 7f712359e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7124120a10 con 0x7f712411b140 2024-09-19T08:50:26.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.607+0000 7f712359e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7124120d10 con 0x7f712406ea30 2024-09-19T08:50:26.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.608+0000 7f7122d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f712411b140 0x7f71241ab900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:26.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.608+0000 7f7122d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f712411b140 0x7f71241ab900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:60512/0 (socket says 172.21.15.164:60512) 2024-09-19T08:50:26.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.608+0000 7f7122d9d640 1 -- 172.21.15.164:0/1733040598 learned_addr learned my addr 172.21.15.164:0/1733040598 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:26.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.612+0000 7f712259c640 1 --2- 172.21.15.164:0/1733040598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f712406ea30 0x7f71241a7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:26.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.612+0000 7f7121d9b640 1 --2- 172.21.15.164:0/1733040598 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7124110190 0x7f71241a7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:26.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.612+0000 7f7122d9d640 1 -- 172.21.15.164:0/1733040598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f712406ea30 msgr2=0x7f71241a7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:26.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.612+0000 7f7122d9d640 1 --2- 172.21.15.164:0/1733040598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f712406ea30 0x7f71241a7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:26.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.612+0000 7f7122d9d640 1 -- 172.21.15.164:0/1733040598 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7124110190 msgr2=0x7f71241a7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:26.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.612+0000 7f7122d9d640 1 --2- 172.21.15.164:0/1733040598 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7124110190 0x7f71241a7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:26.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.612+0000 7f7122d9d640 1 -- 172.21.15.164:0/1733040598 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71241abfe0 con 0x7f712411b140 2024-09-19T08:50:26.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.616+0000 7f7122d9d640 1 --2- 172.21.15.164:0/1733040598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f712411b140 0x7f71241ab900 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f71180058c0 tx=0x7f7118005000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:26.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.617+0000 7f71137fe640 1 -- 172.21.15.164:0/1733040598 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118040030 con 0x7f712411b140 2024-09-19T08:50:26.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.618+0000 7f712359e640 1 -- 172.21.15.164:0/1733040598 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71241ac270 con 0x7f712411b140 2024-09-19T08:50:26.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.618+0000 7f712359e640 1 -- 172.21.15.164:0/1733040598 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71241b3bb0 con 0x7f712411b140 2024-09-19T08:50:26.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.618+0000 7f71137fe640 1 -- 172.21.15.164:0/1733040598 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7118005380 con 0x7f712411b140 2024-09-19T08:50:26.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.618+0000 7f71137fe640 1 -- 172.21.15.164:0/1733040598 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711803c730 con 0x7f712411b140 2024-09-19T08:50:26.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.624+0000 7f712359e640 1 -- 172.21.15.164:0/1733040598 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7104005290 con 0x7f712411b140 2024-09-19T08:50:26.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.627+0000 7f71137fe640 1 -- 172.21.15.164:0/1733040598 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f7118043030 con 0x7f712411b140 2024-09-19T08:50:26.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.628+0000 7f71137fe640 1 --2- 172.21.15.164:0/1733040598 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f710c0a3500 0x7f710c0a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:26.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.629+0000 7f71137fe640 1 -- 172.21.15.164:0/1733040598 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f71180f02d0 con 0x7f712411b140 2024-09-19T08:50:26.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.634+0000 7f712259c640 1 --2- 172.21.15.164:0/1733040598 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f710c0a3500 0x7f710c0a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:26.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.635+0000 7f712259c640 1 --2- 172.21.15.164:0/1733040598 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f710c0a3500 0x7f710c0a59c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f711c005fd0 tx=0x7f711c009200 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:26.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:24.641+0000 7f71137fe640 1 -- 172.21.15.164:0/1733040598 <== mon.2 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f71180bcc20 con 0x7f712411b140 2024-09-19T08:50:26.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:25.219+0000 7f712359e640 1 -- 172.21.15.164:0/1733040598 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7104005850 con 0x7f712411b140 2024-09-19T08:50:26.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:25.221+0000 7f71137fe640 1 -- 172.21.15.164:0/1733040598 <== mon.2 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11991 (secure 0 0 0) 0x7f71180bc520 con 0x7f712411b140 2024-09-19T08:50:26.331 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 30 2024-09-19T08:50:26.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:25.229+0000 7f712359e640 1 -- 172.21.15.164:0/1733040598 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f710c0a3500 msgr2=0x7f710c0a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:26.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:25.229+0000 7f712359e640 1 --2- 172.21.15.164:0/1733040598 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f710c0a3500 0x7f710c0a59c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f711c005fd0 tx=0x7f711c009200 comp rx=0 tx=0).stop 2024-09-19T08:50:26.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:25.229+0000 7f712359e640 1 -- 172.21.15.164:0/1733040598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f712411b140 msgr2=0x7f71241ab900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:26.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:25.229+0000 7f712359e640 1 --2- 172.21.15.164:0/1733040598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f712411b140 0x7f71241ab900 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f71180058c0 tx=0x7f7118005000 comp rx=0 tx=0).stop 2024-09-19T08:50:26.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:25.235+0000 7f712359e640 1 -- 172.21.15.164:0/1733040598 shutdown_connections 2024-09-19T08:50:26.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:25.235+0000 7f712359e640 1 --2- 172.21.15.164:0/1733040598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f712411b140 0x7f71241ab900 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:26.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:25.235+0000 7f712359e640 1 --2- 172.21.15.164:0/1733040598 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f710c0a3500 0x7f710c0a59c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:26.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:25.235+0000 7f712359e640 1 --2- 172.21.15.164:0/1733040598 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7124110190 0x7f71241a7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:26.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:25.235+0000 7f712359e640 1 --2- 172.21.15.164:0/1733040598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f712406ea30 0x7f71241a7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:26.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:25.235+0000 7f712359e640 1 -- 172.21.15.164:0/1733040598 >> 172.21.15.164:0/1733040598 conn(0x7f7124073180 msgr2=0x7f712410e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:26.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:25.238+0000 7f712359e640 1 -- 172.21.15.164:0/1733040598 shutdown_connections 2024-09-19T08:50:26.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:25.238+0000 7f712359e640 1 -- 172.21.15.164:0/1733040598 wait complete. 2024-09-19T08:50:27.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:27 smithi170 ceph-mon[35072]: pgmap v142: 97 pgs: 97 active+clean; 38 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 765 MiB/s wr, 2.49k op/s 2024-09-19T08:50:27.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:27 smithi173 ceph-mon[35306]: pgmap v142: 97 pgs: 97 active+clean; 38 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 765 MiB/s wr, 2.49k op/s 2024-09-19T08:50:28.036 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:27 smithi164 ceph-mon[30928]: pgmap v142: 97 pgs: 97 active+clean; 38 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 765 MiB/s wr, 2.49k op/s 2024-09-19T08:50:28.569 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T08:50:28.697 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:50:29.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:29 smithi164 ceph-mon[30928]: pgmap v143: 97 pgs: 97 active+clean; 38 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 686 MiB/s wr, 2.00k op/s 2024-09-19T08:50:29.874 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:29 smithi170 ceph-mon[35072]: pgmap v143: 97 pgs: 97 active+clean; 38 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 686 MiB/s wr, 2.00k op/s 2024-09-19T08:50:29.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:29 smithi173 ceph-mon[35306]: pgmap v143: 97 pgs: 97 active+clean; 38 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 686 MiB/s wr, 2.00k op/s 2024-09-19T08:50:30.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.400+0000 7f91ae98b640 1 -- 172.21.15.164:0/3790253599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91a8100980 msgr2=0x7f91a8100d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:30.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.400+0000 7f91ae98b640 1 --2- 172.21.15.164:0/3790253599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91a8100980 0x7f91a8100d60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009ca0 tx=0x7f91a002a960 comp rx=0 tx=0).stop 2024-09-19T08:50:30.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.411+0000 7f91ae98b640 1 -- 172.21.15.164:0/3790253599 shutdown_connections 2024-09-19T08:50:30.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.411+0000 7f91ae98b640 1 --2- 172.21.15.164:0/3790253599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91a8105d90 0x7f91a8106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.411+0000 7f91ae98b640 1 --2- 172.21.15.164:0/3790253599 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91a81012a0 0x7f91a8105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.411+0000 7f91ae98b640 1 --2- 172.21.15.164:0/3790253599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91a8100980 0x7f91a8100d60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.411+0000 7f91ae98b640 1 -- 172.21.15.164:0/3790253599 >> 172.21.15.164:0/3790253599 conn(0x7f91a80fc610 msgr2=0x7f91a80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:30.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.413+0000 7f91ae98b640 1 -- 172.21.15.164:0/3790253599 shutdown_connections 2024-09-19T08:50:30.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.415+0000 7f91ae98b640 1 -- 172.21.15.164:0/3790253599 wait complete. 2024-09-19T08:50:30.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.416+0000 7f91ae98b640 1 Processor -- start 2024-09-19T08:50:30.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.416+0000 7f91ae98b640 1 -- start start 2024-09-19T08:50:30.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.417+0000 7f91ae98b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91a8100980 0x7f91a81148e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:30.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.417+0000 7f91ae98b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91a81012a0 0x7f91a8114e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:30.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.417+0000 7f91ae98b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91a8105d90 0x7f91a810e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:30.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.417+0000 7f91ae98b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f91a8107350 con 0x7f91a8105d90 2024-09-19T08:50:30.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.417+0000 7f91ae98b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f91a81071d0 con 0x7f91a81012a0 2024-09-19T08:50:30.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.417+0000 7f91ae98b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f91a81074d0 con 0x7f91a8100980 2024-09-19T08:50:30.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.420+0000 7f91ad188640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91a81012a0 0x7f91a8114e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:30.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.420+0000 7f91ad188640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91a81012a0 0x7f91a8114e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:60532/0 (socket says 172.21.15.164:60532) 2024-09-19T08:50:30.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.420+0000 7f91ad188640 1 -- 172.21.15.164:0/1254220680 learned_addr learned my addr 172.21.15.164:0/1254220680 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:30.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.420+0000 7f91ae18a640 1 --2- 172.21.15.164:0/1254220680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91a8105d90 0x7f91a810e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:30.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.420+0000 7f91ad989640 1 --2- 172.21.15.164:0/1254220680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91a8100980 0x7f91a81148e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:30.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.423+0000 7f91ad188640 1 -- 172.21.15.164:0/1254220680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91a8100980 msgr2=0x7f91a81148e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:30.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.423+0000 7f91ad188640 1 --2- 172.21.15.164:0/1254220680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91a8100980 0x7f91a81148e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.423+0000 7f91ad188640 1 -- 172.21.15.164:0/1254220680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91a8105d90 msgr2=0x7f91a810e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:30.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.423+0000 7f91ad188640 1 --2- 172.21.15.164:0/1254220680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91a8105d90 0x7f91a810e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.423+0000 7f91ad188640 1 -- 172.21.15.164:0/1254220680 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a810f200 con 0x7f91a81012a0 2024-09-19T08:50:30.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.431+0000 7f91ad188640 1 --2- 172.21.15.164:0/1254220680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91a81012a0 0x7f91a8114e20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f919800b3c0 tx=0x7f919800b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:30.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.431+0000 7f919effd640 1 -- 172.21.15.164:0/1254220680 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9198011830 con 0x7f91a81012a0 2024-09-19T08:50:30.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.432+0000 7f91ae98b640 1 -- 172.21.15.164:0/1254220680 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91a810f4f0 con 0x7f91a81012a0 2024-09-19T08:50:30.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.432+0000 7f91ae98b640 1 -- 172.21.15.164:0/1254220680 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91a811e0f0 con 0x7f91a81012a0 2024-09-19T08:50:30.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.438+0000 7f91b1fc6740 1 Processor -- start 2024-09-19T08:50:30.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.438+0000 7f91b1fc6740 1 -- start start 2024-09-19T08:50:30.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.438+0000 7f919effd640 1 -- 172.21.15.164:0/1254220680 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9198016070 con 0x7f91a81012a0 2024-09-19T08:50:30.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.438+0000 7f919effd640 1 -- 172.21.15.164:0/1254220680 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919800f2e0 con 0x7f91a81012a0 2024-09-19T08:50:30.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.444+0000 7f91b1fc6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642d7a19230 0x5642d7a17a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:30.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.444+0000 7f91b1fc6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d7a17f40 0x5642d7a18380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:30.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.444+0000 7f91b1fc6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642d7a228a0 0x5642d7a24c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:30.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.444+0000 7f91b1fc6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5642d797d2a0 con 0x5642d7a228a0 2024-09-19T08:50:30.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.444+0000 7f91b1fc6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5642d7924b10 con 0x5642d7a19230 2024-09-19T08:50:30.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.444+0000 7f91b1fc6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5642d7979260 con 0x5642d7a17f40 2024-09-19T08:50:30.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.447+0000 7f91ae18a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642d7a19230 0x5642d7a17a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:30.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.448+0000 7f91ad188640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642d7a228a0 0x5642d7a24c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:30.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.448+0000 7f91ad188640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642d7a228a0 0x5642d7a24c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46006/0 (socket says 172.21.15.164:46006) 2024-09-19T08:50:30.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.448+0000 7f91ad188640 1 -- 172.21.15.164:0/1803515969 learned_addr learned my addr 172.21.15.164:0/1803515969 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:30.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.450+0000 7f91ae18a640 1 -- 172.21.15.164:0/1803515969 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d7a17f40 msgr2=0x5642d7a18380 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:50:30.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.450+0000 7f91ae18a640 1 --2- 172.21.15.164:0/1803515969 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d7a17f40 0x5642d7a18380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.450+0000 7f91ae18a640 1 -- 172.21.15.164:0/1803515969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642d7a228a0 msgr2=0x5642d7a24c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:30.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.450+0000 7f91ae18a640 1 --2- 172.21.15.164:0/1803515969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642d7a228a0 0x5642d7a24c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.450+0000 7f91ae18a640 1 -- 172.21.15.164:0/1803515969 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642d76eeb90 con 0x5642d7a19230 2024-09-19T08:50:30.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.451+0000 7f91ae18a640 1 --2- 172.21.15.164:0/1803515969 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642d7a19230 0x5642d7a17a00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f91a400b6c0 tx=0x7f91a400bb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e6d74e54e76a24c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:30.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.456+0000 7f91ae98b640 1 -- 172.21.15.164:0/1803515969 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a400d740 con 0x5642d7a19230 2024-09-19T08:50:30.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.456+0000 7f91ae98b640 1 -- 172.21.15.164:0/1803515969 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91a4019070 con 0x5642d7a19230 2024-09-19T08:50:30.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.457+0000 7f919effd640 1 -- 172.21.15.164:0/1254220680 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f91980119d0 con 0x7f91a81012a0 2024-09-19T08:50:30.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.457+0000 7f91b1fc6740 1 -- 172.21.15.164:0/1803515969 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642d7a19230 msgr2=0x5642d7a17a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:30.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.457+0000 7f91b1fc6740 1 --2- 172.21.15.164:0/1803515969 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642d7a19230 0x5642d7a17a00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f91a400b6c0 tx=0x7f91a400bb80 comp rx=0 tx=0).stop 2024-09-19T08:50:30.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.458+0000 7f919effd640 1 --2- 172.21.15.164:0/1254220680 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f91940a34f0 0x7f91940a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:30.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.458+0000 7f919effd640 1 -- 172.21.15.164:0/1254220680 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f91980d4960 con 0x7f91a81012a0 2024-09-19T08:50:30.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.459+0000 7f91b1fc6740 1 -- 172.21.15.164:0/1803515969 shutdown_connections 2024-09-19T08:50:30.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.459+0000 7f91b1fc6740 1 --2- 172.21.15.164:0/1803515969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642d7a228a0 0x5642d7a24c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.459+0000 7f91b1fc6740 1 --2- 172.21.15.164:0/1803515969 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d7a17f40 0x5642d7a18380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.459+0000 7f91b1fc6740 1 --2- 172.21.15.164:0/1803515969 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642d7a19230 0x5642d7a17a00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.459+0000 7f91b1fc6740 1 -- 172.21.15.164:0/1803515969 >> 172.21.15.164:0/1803515969 conn(0x5642d7a14190 msgr2=0x5642d7a14570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:30.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.470+0000 7f91ae18a640 1 --2- 172.21.15.164:0/1254220680 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f91940a34f0 0x7f91940a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:30.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.473+0000 7f91b1fc6740 1 -- 172.21.15.164:0/1803515969 shutdown_connections 2024-09-19T08:50:30.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.476+0000 7f91b1fc6740 1 -- 172.21.15.164:0/1803515969 wait complete. 2024-09-19T08:50:30.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.477+0000 7f91b1fc6740 1 -- start start 2024-09-19T08:50:30.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.478+0000 7f91b1fc6740 1 Processor -- start 2024-09-19T08:50:30.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.485+0000 7f91ae18a640 1 --2- 172.21.15.164:0/1254220680 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f91940a34f0 0x7f91940a59b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f91a400d530 tx=0x7f91a4000bc0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:30.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.487+0000 7f91b1fc6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642d7a17f40 0x5642d7a222d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:30.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.487+0000 7f91b1fc6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642d7a19230 0x5642d7a1f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:30.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.487+0000 7f91b1fc6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d7a228a0 0x5642d7a1fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:30.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.487+0000 7f91b1fc6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5642d7a257b0 con 0x5642d7a17f40 2024-09-19T08:50:30.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.487+0000 7f91b1fc6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5642d79056e0 con 0x5642d7a19230 2024-09-19T08:50:30.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.487+0000 7f91b1fc6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5642d7a25930 con 0x5642d7a228a0 2024-09-19T08:50:30.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.487+0000 7f91b1fc6740 10 client.0 _ll_register_callbacks cb 0x5642d7929a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:50:30.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.490+0000 7f91ad188640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642d7a19230 0x5642d7a1f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:30.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.490+0000 7f91ae18a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d7a228a0 0x5642d7a1fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:30.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.490+0000 7f91ad188640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642d7a19230 0x5642d7a1f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:60536/0 (socket says 172.21.15.164:60536) 2024-09-19T08:50:30.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.490+0000 7f91ad188640 1 -- 172.21.15.164:0/610055742 learned_addr learned my addr 172.21.15.164:0/610055742 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:30.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.492+0000 7f91ad989640 1 --2- 172.21.15.164:0/610055742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642d7a17f40 0x5642d7a222d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:30.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.492+0000 7f91ad188640 1 -- 172.21.15.164:0/610055742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d7a228a0 msgr2=0x5642d7a1fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:30.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.492+0000 7f91ad188640 1 --2- 172.21.15.164:0/610055742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d7a228a0 0x5642d7a1fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.492+0000 7f91ad188640 1 -- 172.21.15.164:0/610055742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642d7a17f40 msgr2=0x5642d7a222d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:30.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.492+0000 7f91ad188640 1 --2- 172.21.15.164:0/610055742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642d7a17f40 0x5642d7a222d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.492+0000 7f91ad188640 1 -- 172.21.15.164:0/610055742 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642d7a2ed40 con 0x5642d7a19230 2024-09-19T08:50:30.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.492+0000 7f91ad989640 1 --2- 172.21.15.164:0/610055742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642d7a17f40 0x5642d7a222d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:50:30.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.493+0000 7f91ad188640 1 --2- 172.21.15.164:0/610055742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642d7a19230 0x5642d7a1f580 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f91980a09c0 tx=0x7f919809d550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=801c2541ef359862 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:30.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.495+0000 7f91917fa640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-19T08:50:30.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.497+0000 7f91917fa640 1 -- 172.21.15.164:0/610055742 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91980d2550 con 0x5642d7a19230 2024-09-19T08:50:30.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.497+0000 7f91917fa640 1 -- 172.21.15.164:0/610055742 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f919800eda0 con 0x5642d7a19230 2024-09-19T08:50:30.497 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.498+0000 7f91b1fc6740 1 -- 172.21.15.164:0/610055742 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91a4019070 con 0x5642d7a19230 2024-09-19T08:50:30.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.502+0000 7f91917fa640 1 -- 172.21.15.164:0/610055742 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f91980a5070 con 0x5642d7a19230 2024-09-19T08:50:30.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.502+0000 7f91b1fc6740 10 client.25796 fetch_fsmap learned FSMap version 30 2024-09-19T08:50:30.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.502+0000 7f91b1fc6740 1 -- 172.21.15.164:0/610055742 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5642d7a315a0 con 0x5642d7a19230 2024-09-19T08:50:30.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.507+0000 7f91917fa640 1 -- 172.21.15.164:0/610055742 <== mon.2 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91980a9930 con 0x5642d7a19230 2024-09-19T08:50:30.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.507+0000 7f91917fa640 1 -- 172.21.15.164:0/610055742 <== mon.2 v2:172.21.15.170:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f919809c980 con 0x5642d7a19230 2024-09-19T08:50:30.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.507+0000 7f91b1fc6740 10 client.25796 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:50:30.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.507+0000 7f91b1fc6740 10 client.25796 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:50:30.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.508+0000 7f91b1fc6740 20 client.25796 populate_metadata read hostname 'smithi164' 2024-09-19T08:50:30.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.508+0000 7f91b1fc6740 1 --2- 172.21.15.164:0/610055742 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5642d7a32f60 0x5642d7a35380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:30.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.508+0000 7f91b1fc6740 4 client.25796 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:50:30.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.508+0000 7f91b1fc6740 1 -- 172.21.15.164:0/610055742 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f91a4019070 con 0x5642d7a32f60 2024-09-19T08:50:30.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.513+0000 7f91ad989640 1 --2- 172.21.15.164:0/610055742 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5642d7a32f60 0x5642d7a35380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:30.523 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.523+0000 7f91ad989640 1 --2- 172.21.15.164:0/610055742 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5642d7a32f60 0x5642d7a35380 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=181b56d0f55465cb server_cookie=bb3b841393cd3707 in_seq=0 out_seq=0 2024-09-19T08:50:30.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.523+0000 7f91917fa640 10 client.25796 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:50:30.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.526+0000 7f91917fa640 1 -- 172.21.15.164:0/610055742 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f91a4019070 con 0x5642d7a32f60 2024-09-19T08:50:30.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.526+0000 7f91917fa640 10 client.25796 handle_command_reply: tid=0 2024-09-19T08:50:30.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.526+0000 7f91b1fc6740 1 client.25796 shutdown 2024-09-19T08:50:30.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.530+0000 7f91b1fc6740 1 -- 172.21.15.164:0/610055742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642d7a19230 msgr2=0x5642d7a1f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:30.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.530+0000 7f91b1fc6740 1 --2- 172.21.15.164:0/610055742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642d7a19230 0x5642d7a1f580 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f91980a09c0 tx=0x7f919809d550 comp rx=0 tx=0).stop 2024-09-19T08:50:30.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.536+0000 7f91b1fc6740 1 -- 172.21.15.164:0/610055742 shutdown_connections 2024-09-19T08:50:30.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.536+0000 7f91b1fc6740 1 --2- 172.21.15.164:0/610055742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d7a228a0 0x5642d7a1fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.536+0000 7f91b1fc6740 1 --2- 172.21.15.164:0/610055742 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5642d7a32f60 0x5642d7a35380 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.536+0000 7f91b1fc6740 1 --2- 172.21.15.164:0/610055742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642d7a19230 0x5642d7a1f580 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.536+0000 7f91b1fc6740 1 --2- 172.21.15.164:0/610055742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642d7a17f40 0x5642d7a222d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.536+0000 7f91b1fc6740 1 -- 172.21.15.164:0/610055742 >> 172.21.15.164:0/610055742 conn(0x5642d7a14190 msgr2=0x5642d7a13470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:30.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.536+0000 7f91917fa640 0 client.25796 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:50:30.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.537+0000 7f91b1fc6740 1 -- 172.21.15.164:0/610055742 shutdown_connections 2024-09-19T08:50:30.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.539+0000 7f91b1fc6740 1 -- 172.21.15.164:0/610055742 wait complete. 2024-09-19T08:50:30.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.540+0000 7f91b1fc6740 20 client.25796 trim_cache size 0 max 16384 2024-09-19T08:50:30.582 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:50:30.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.583+0000 7f91b1fc6740 1 Processor -- start 2024-09-19T08:50:30.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.583+0000 7f91b1fc6740 1 -- start start 2024-09-19T08:50:30.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.583+0000 7f91b1fc6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642d7a17f40 0x5642d7a1a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:30.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.584+0000 7f91b1fc6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d7a19230 0x5642d7a1abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:30.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.584+0000 7f91b1fc6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642d7a228a0 0x5642d7a3ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:30.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.584+0000 7f91b1fc6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f918c00e6f0 con 0x5642d7a17f40 2024-09-19T08:50:30.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.584+0000 7f91b1fc6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f918c0102f0 con 0x5642d7a228a0 2024-09-19T08:50:30.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.584+0000 7f91b1fc6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f918c00edc0 con 0x5642d7a19230 2024-09-19T08:50:30.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.585+0000 7f91ad188640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d7a19230 0x5642d7a1abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:30.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.585+0000 7f91ad188640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d7a19230 0x5642d7a1abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:35480/0 (socket says 172.21.15.164:35480) 2024-09-19T08:50:30.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.585+0000 7f91ad188640 1 -- 172.21.15.164:0/4106773769 learned_addr learned my addr 172.21.15.164:0/4106773769 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:30.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.585+0000 7f91ae18a640 1 --2- 172.21.15.164:0/4106773769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642d7a228a0 0x5642d7a3ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:30.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.587+0000 7f91ad989640 1 --2- 172.21.15.164:0/4106773769 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642d7a17f40 0x5642d7a1a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:30.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.588+0000 7f91ad188640 1 -- 172.21.15.164:0/4106773769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642d7a228a0 msgr2=0x5642d7a3ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:30.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.588+0000 7f91ad188640 1 --2- 172.21.15.164:0/4106773769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642d7a228a0 0x5642d7a3ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.588+0000 7f91ad188640 1 -- 172.21.15.164:0/4106773769 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642d7a17f40 msgr2=0x5642d7a1a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:30.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.588+0000 7f91ad188640 1 --2- 172.21.15.164:0/4106773769 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642d7a17f40 0x5642d7a1a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.588+0000 7f91ad188640 1 -- 172.21.15.164:0/4106773769 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642d7a20310 con 0x5642d7a19230 2024-09-19T08:50:30.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.589+0000 7f91ad188640 1 --2- 172.21.15.164:0/4106773769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d7a19230 0x5642d7a1abf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f91980a9400 tx=0x7f919809d4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d564e4525003a2ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:30.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.589+0000 7f91937fe640 1 -- 172.21.15.164:0/4106773769 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919806a080 con 0x5642d7a19230 2024-09-19T08:50:30.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.589+0000 7f91937fe640 1 -- 172.21.15.164:0/4106773769 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91980a9930 con 0x5642d7a19230 2024-09-19T08:50:30.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.589+0000 7f91937fe640 1 -- 172.21.15.164:0/4106773769 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919800eba0 con 0x5642d7a19230 2024-09-19T08:50:30.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.591+0000 7f91b1fc6740 1 -- 172.21.15.164:0/4106773769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d7a19230 msgr2=0x5642d7a1abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:30.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.591+0000 7f91b1fc6740 1 --2- 172.21.15.164:0/4106773769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d7a19230 0x5642d7a1abf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f91980a9400 tx=0x7f919809d4b0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.591+0000 7f91b1fc6740 1 -- 172.21.15.164:0/4106773769 shutdown_connections 2024-09-19T08:50:30.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.591+0000 7f91b1fc6740 1 --2- 172.21.15.164:0/4106773769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642d7a228a0 0x5642d7a3ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.591+0000 7f91b1fc6740 1 --2- 172.21.15.164:0/4106773769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d7a19230 0x5642d7a1abf0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.591+0000 7f91b1fc6740 1 --2- 172.21.15.164:0/4106773769 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642d7a17f40 0x5642d7a1a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.591+0000 7f91b1fc6740 1 -- 172.21.15.164:0/4106773769 >> 172.21.15.164:0/4106773769 conn(0x5642d7a14190 msgr2=0x5642d7a13470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:30.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.595+0000 7f91b1fc6740 1 -- 172.21.15.164:0/4106773769 shutdown_connections 2024-09-19T08:50:30.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.595+0000 7f91b1fc6740 1 -- 172.21.15.164:0/4106773769 wait complete. 2024-09-19T08:50:30.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.596+0000 7f91b1fc6740 1 -- start start 2024-09-19T08:50:30.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.596+0000 7f91b1fc6740 1 Processor -- start 2024-09-19T08:50:30.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.597+0000 7f91b1fc6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642d7a17f40 0x5642d7a271b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:30.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.597+0000 7f91b1fc6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d7a19230 0x5642d7a276f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:30.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.597+0000 7f91b1fc6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642d7a228a0 0x5642d7a9cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:30.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.597+0000 7f91b1fc6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f918c00fb60 con 0x5642d7a17f40 2024-09-19T08:50:30.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.597+0000 7f91b1fc6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f918c00f490 con 0x5642d7a228a0 2024-09-19T08:50:30.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.597+0000 7f91b1fc6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f918c00c6d0 con 0x5642d7a19230 2024-09-19T08:50:30.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.597+0000 7f91b1fc6740 10 client.0 _ll_register_callbacks cb 0x7f91980a0b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:50:30.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.599+0000 7f91ae18a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642d7a228a0 0x5642d7a9cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:30.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.599+0000 7f91ae18a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642d7a228a0 0x5642d7a9cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:60550/0 (socket says 172.21.15.164:60550) 2024-09-19T08:50:30.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.599+0000 7f91ae18a640 1 -- 172.21.15.164:0/1755934695 learned_addr learned my addr 172.21.15.164:0/1755934695 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:30.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.599+0000 7f91ad188640 1 --2- 172.21.15.164:0/1755934695 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d7a19230 0x5642d7a276f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:30.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.599+0000 7f91ad989640 1 --2- 172.21.15.164:0/1755934695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642d7a17f40 0x5642d7a271b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:30.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.603+0000 7f91ae18a640 1 -- 172.21.15.164:0/1755934695 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d7a19230 msgr2=0x5642d7a276f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:30.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.604+0000 7f91ae18a640 1 --2- 172.21.15.164:0/1755934695 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d7a19230 0x5642d7a276f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.604+0000 7f91ae18a640 1 -- 172.21.15.164:0/1755934695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642d7a17f40 msgr2=0x5642d7a271b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:30.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.604+0000 7f91ae18a640 1 --2- 172.21.15.164:0/1755934695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642d7a17f40 0x5642d7a271b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.604+0000 7f91ae18a640 1 -- 172.21.15.164:0/1755934695 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642d7aae1e0 con 0x5642d7a228a0 2024-09-19T08:50:30.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.606+0000 7f91ae18a640 1 --2- 172.21.15.164:0/1755934695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642d7a228a0 0x5642d7a9cc60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f91a4006f70 tx=0x7f91a40149c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bff8a47bba2ea50 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:30.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.606+0000 7f91917fa640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-19T08:50:30.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.607+0000 7f91917fa640 1 -- 172.21.15.164:0/1755934695 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a402e020 con 0x5642d7a228a0 2024-09-19T08:50:30.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.607+0000 7f91b1fc6740 1 -- 172.21.15.164:0/1755934695 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91980a9930 con 0x5642d7a228a0 2024-09-19T08:50:30.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.607+0000 7f91917fa640 1 -- 172.21.15.164:0/1755934695 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f91a4012c60 con 0x5642d7a228a0 2024-09-19T08:50:30.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.607+0000 7f91917fa640 1 -- 172.21.15.164:0/1755934695 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f91a4039050 con 0x5642d7a228a0 2024-09-19T08:50:30.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.611+0000 7f91b1fc6740 10 client.25802 fetch_fsmap learned FSMap version 30 2024-09-19T08:50:30.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.611+0000 7f91b1fc6740 1 -- 172.21.15.164:0/1755934695 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5642d7aaf4a0 con 0x5642d7a228a0 2024-09-19T08:50:30.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.612+0000 7f91917fa640 1 -- 172.21.15.164:0/1755934695 <== mon.2 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91980a9930 con 0x5642d7a228a0 2024-09-19T08:50:30.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.612+0000 7f91917fa640 1 -- 172.21.15.164:0/1755934695 <== mon.2 v2:172.21.15.170:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f91a4013070 con 0x5642d7a228a0 2024-09-19T08:50:30.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.617+0000 7f91b1fc6740 10 client.25802 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:50:30.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.617+0000 7f91b1fc6740 10 client.25802 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:50:30.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.617+0000 7f91b1fc6740 20 client.25802 populate_metadata read hostname 'smithi164' 2024-09-19T08:50:30.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.617+0000 7f91b1fc6740 1 --2- 172.21.15.164:0/1755934695 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5642d7aafaf0 0x5642d7ab1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:30.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.617+0000 7f91b1fc6740 4 client.25802 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}] 2024-09-19T08:50:30.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.617+0000 7f91b1fc6740 1 -- 172.21.15.164:0/1755934695 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}) -- 0x5642d798a9d0 con 0x5642d7aafaf0 2024-09-19T08:50:30.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.620+0000 7f91ad989640 1 --2- 172.21.15.164:0/1755934695 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5642d7aafaf0 0x5642d7ab1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:30.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.632+0000 7f91917fa640 10 client.25802 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:50:30.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.632+0000 7f91ad989640 1 --2- 172.21.15.164:0/1755934695 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5642d7aafaf0 0x5642d7ab1f10 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=957a802164fc930e server_cookie=2cd4675626479224 in_seq=0 out_seq=0 2024-09-19T08:50:30.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.952+0000 7f91917fa640 1 -- 172.21.15.164:0/1755934695 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5642d798a9d0 con 0x5642d7aafaf0 2024-09-19T08:50:30.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.952+0000 7f91917fa640 10 client.25802 handle_command_reply: tid=0 2024-09-19T08:50:30.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.952+0000 7f91b1fc6740 1 client.25802 shutdown 2024-09-19T08:50:30.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.953+0000 7f91b1fc6740 1 -- 172.21.15.164:0/1755934695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642d7a228a0 msgr2=0x5642d7a9cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:30.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.953+0000 7f91b1fc6740 1 --2- 172.21.15.164:0/1755934695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642d7a228a0 0x5642d7a9cc60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f91a4006f70 tx=0x7f91a40149c0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.953+0000 7f91b1fc6740 1 -- 172.21.15.164:0/1755934695 shutdown_connections 2024-09-19T08:50:30.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.953+0000 7f91b1fc6740 1 --2- 172.21.15.164:0/1755934695 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5642d7aafaf0 0x5642d7ab1f10 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.953+0000 7f91b1fc6740 1 --2- 172.21.15.164:0/1755934695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642d7a228a0 0x5642d7a9cc60 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.953+0000 7f91b1fc6740 1 --2- 172.21.15.164:0/1755934695 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d7a19230 0x5642d7a276f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.953+0000 7f91b1fc6740 1 --2- 172.21.15.164:0/1755934695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642d7a17f40 0x5642d7a271b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.953+0000 7f91b1fc6740 1 -- 172.21.15.164:0/1755934695 >> 172.21.15.164:0/1755934695 conn(0x5642d7a14190 msgr2=0x5642d7a2db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:30.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.953+0000 7f91917fa640 0 client.25802 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:50:30.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.954+0000 7f91b1fc6740 1 -- 172.21.15.164:0/1755934695 shutdown_connections 2024-09-19T08:50:30.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.954+0000 7f91b1fc6740 1 -- 172.21.15.164:0/1755934695 wait complete. 2024-09-19T08:50:30.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.954+0000 7f91b1fc6740 20 client.25802 trim_cache size 0 max 16384 2024-09-19T08:50:30.955 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:50:30.955 INFO:teuthology.orchestra.run.smithi164.stdout: "epoch": 30, 2024-09-19T08:50:30.955 INFO:teuthology.orchestra.run.smithi164.stdout: "leader": 24479, 2024-09-19T08:50:30.955 INFO:teuthology.orchestra.run.smithi164.stdout: "set_version": 18, 2024-09-19T08:50:30.955 INFO:teuthology.orchestra.run.smithi164.stdout: "sets": { 2024-09-19T08:50:30.956 INFO:teuthology.orchestra.run.smithi164.stdout: "ea4bdac6": { 2024-09-19T08:50:30.956 INFO:teuthology.orchestra.run.smithi164.stdout: "version": 18, 2024-09-19T08:50:30.956 INFO:teuthology.orchestra.run.smithi164.stdout: "age_ref": 0.0, 2024-09-19T08:50:30.956 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:50:30.956 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "QUIESCED", 2024-09-19T08:50:30.956 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:50:30.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:50:30.956 INFO:teuthology.orchestra.run.smithi164.stdout: "timeout": 90.0, 2024-09-19T08:50:30.956 INFO:teuthology.orchestra.run.smithi164.stdout: "expiration": 149.8, 2024-09-19T08:50:30.956 INFO:teuthology.orchestra.run.smithi164.stdout: "members": { 2024-09-19T08:50:30.956 INFO:teuthology.orchestra.run.smithi164.stdout: "file:/": { 2024-09-19T08:50:30.956 INFO:teuthology.orchestra.run.smithi164.stdout: "excluded": false, 2024-09-19T08:50:30.956 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:50:30.957 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "QUIESCED", 2024-09-19T08:50:30.957 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:50:30.957 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:50:30.957 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:50:30.957 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:50:30.957 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:50:30.957 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:50:30.957 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:50:30.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.956+0000 7f919cdea640 1 -- 172.21.15.164:0/1254220680 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f91940a34f0 msgr2=0x7f91940a59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:30.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.956+0000 7f919cdea640 1 --2- 172.21.15.164:0/1254220680 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f91940a34f0 0x7f91940a59b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f91a400d530 tx=0x7f91a4000bc0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.956+0000 7f919cdea640 1 -- 172.21.15.164:0/1254220680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91a81012a0 msgr2=0x7f91a8114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:30.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.956+0000 7f919cdea640 1 --2- 172.21.15.164:0/1254220680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91a81012a0 0x7f91a8114e20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f919800b3c0 tx=0x7f919800b880 comp rx=0 tx=0).stop 2024-09-19T08:50:30.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.956+0000 7f919cdea640 1 -- 172.21.15.164:0/1254220680 shutdown_connections 2024-09-19T08:50:30.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.956+0000 7f919cdea640 1 --2- 172.21.15.164:0/1254220680 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f91940a34f0 0x7f91940a59b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.956+0000 7f919cdea640 1 --2- 172.21.15.164:0/1254220680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91a8105d90 0x7f91a810e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.956+0000 7f919cdea640 1 --2- 172.21.15.164:0/1254220680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91a81012a0 0x7f91a8114e20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.956+0000 7f919cdea640 1 --2- 172.21.15.164:0/1254220680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91a8100980 0x7f91a81148e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:30.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.956+0000 7f919cdea640 1 -- 172.21.15.164:0/1254220680 >> 172.21.15.164:0/1254220680 conn(0x7f91a80fc610 msgr2=0x7f91a8103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:30.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.956+0000 7f919cdea640 1 -- 172.21.15.164:0/1254220680 shutdown_connections 2024-09-19T08:50:30.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:30.957+0000 7f919cdea640 1 -- 172.21.15.164:0/1254220680 wait complete. 2024-09-19T08:50:31.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.654+0000 7f4e67993640 1 -- 172.21.15.164:0/1643101843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e60106f10 msgr2=0x7f4e6010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:31.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.654+0000 7f4e67993640 1 --2- 172.21.15.164:0/1643101843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e60106f10 0x7f4e6010ea90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4e58009ca0 tx=0x7f4e5802a8c0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.654+0000 7f4e67993640 1 -- 172.21.15.164:0/1643101843 shutdown_connections 2024-09-19T08:50:31.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.654+0000 7f4e67993640 1 --2- 172.21.15.164:0/1643101843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e60106f10 0x7f4e6010ea90 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.654+0000 7f4e67993640 1 --2- 172.21.15.164:0/1643101843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e60102790 0x7f4e601067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.654+0000 7f4e67993640 1 --2- 172.21.15.164:0/1643101843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e60101de0 0x7f4e601021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.654+0000 7f4e67993640 1 -- 172.21.15.164:0/1643101843 >> 172.21.15.164:0/1643101843 conn(0x7f4e600fd8c0 msgr2=0x7f4e600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:31.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.655+0000 7f4e67993640 1 -- 172.21.15.164:0/1643101843 shutdown_connections 2024-09-19T08:50:31.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.655+0000 7f4e67993640 1 -- 172.21.15.164:0/1643101843 wait complete. 2024-09-19T08:50:31.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.655+0000 7f4e67993640 1 Processor -- start 2024-09-19T08:50:31.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.655+0000 7f4e67993640 1 -- start start 2024-09-19T08:50:31.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.655+0000 7f4e67993640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e60101de0 0x7f4e600744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:31.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.655+0000 7f4e67993640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e60102790 0x7f4e60074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:31.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.655+0000 7f4e67993640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e60106f10 0x7f4e60074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:31.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.655+0000 7f4e67993640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4e60111f70 con 0x7f4e60102790 2024-09-19T08:50:31.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.655+0000 7f4e67993640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4e60111df0 con 0x7f4e60106f10 2024-09-19T08:50:31.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.655+0000 7f4e67993640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4e601120f0 con 0x7f4e60101de0 2024-09-19T08:50:31.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.656+0000 7f4e64f2e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e60102790 0x7f4e60074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:31.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.656+0000 7f4e64f2e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e60102790 0x7f4e60074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46050/0 (socket says 172.21.15.164:46050) 2024-09-19T08:50:31.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.656+0000 7f4e64f2e640 1 -- 172.21.15.164:0/1853844771 learned_addr learned my addr 172.21.15.164:0/1853844771 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:31.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.656+0000 7f4e65f30640 1 --2- 172.21.15.164:0/1853844771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e60106f10 0x7f4e60074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:31.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.656+0000 7f4e6572f640 1 --2- 172.21.15.164:0/1853844771 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e60101de0 0x7f4e600744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:31.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.656+0000 7f4e64f2e640 1 -- 172.21.15.164:0/1853844771 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e60101de0 msgr2=0x7f4e600744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:31.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.656+0000 7f4e64f2e640 1 --2- 172.21.15.164:0/1853844771 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e60101de0 0x7f4e600744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.656+0000 7f4e64f2e640 1 -- 172.21.15.164:0/1853844771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e60106f10 msgr2=0x7f4e60074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:31.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.656+0000 7f4e64f2e640 1 --2- 172.21.15.164:0/1853844771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e60106f10 0x7f4e60074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.656+0000 7f4e64f2e640 1 -- 172.21.15.164:0/1853844771 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e601a68b0 con 0x7f4e60102790 2024-09-19T08:50:31.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.656+0000 7f4e6572f640 1 --2- 172.21.15.164:0/1853844771 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e60101de0 0x7f4e600744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:50:31.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.656+0000 7f4e65f30640 1 --2- 172.21.15.164:0/1853844771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e60106f10 0x7f4e60074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:50:31.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.656+0000 7f4e64f2e640 1 --2- 172.21.15.164:0/1853844771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e60102790 0x7f4e60074a30 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000e9f0 tx=0x7f4e5000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:31.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.656+0000 7f4e567fc640 1 -- 172.21.15.164:0/1853844771 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5000cd80 con 0x7f4e60102790 2024-09-19T08:50:31.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.656+0000 7f4e567fc640 1 -- 172.21.15.164:0/1853844771 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e50005500 con 0x7f4e60102790 2024-09-19T08:50:31.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.656+0000 7f4e67993640 1 -- 172.21.15.164:0/1853844771 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e601a6ba0 con 0x7f4e60102790 2024-09-19T08:50:31.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.656+0000 7f4e567fc640 1 -- 172.21.15.164:0/1853844771 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e50010690 con 0x7f4e60102790 2024-09-19T08:50:31.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.656+0000 7f4e67993640 1 -- 172.21.15.164:0/1853844771 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e601a7090 con 0x7f4e60102790 2024-09-19T08:50:31.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.659+0000 7f4e567fc640 1 -- 172.21.15.164:0/1853844771 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f4e500046c0 con 0x7f4e60102790 2024-09-19T08:50:31.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.659+0000 7f4e69d6c740 1 Processor -- start 2024-09-19T08:50:31.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.659+0000 7f4e69d6c740 1 -- start start 2024-09-19T08:50:31.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.659+0000 7f4e69d6c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abd9e42b40 0x55abd9e41090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:31.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.659+0000 7f4e69d6c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55abd9e415d0 0x55abd9e41a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:31.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.659+0000 7f4e69d6c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55abd9e4c370 0x55abd9e4e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:31.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.659+0000 7f4e69d6c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55abd9da72f0 con 0x55abd9e4c370 2024-09-19T08:50:31.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.659+0000 7f4e69d6c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55abd9d4ecf0 con 0x55abd9e415d0 2024-09-19T08:50:31.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.659+0000 7f4e69d6c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55abd9da32b0 con 0x55abd9e42b40 2024-09-19T08:50:31.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.659+0000 7f4e64f2e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55abd9e4c370 0x55abd9e4e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:31.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.659+0000 7f4e65f30640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abd9e42b40 0x55abd9e41090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:31.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.659+0000 7f4e64f2e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55abd9e4c370 0x55abd9e4e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46056/0 (socket says 172.21.15.164:46056) 2024-09-19T08:50:31.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.659+0000 7f4e64f2e640 1 -- 172.21.15.164:0/3542532051 learned_addr learned my addr 172.21.15.164:0/3542532051 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:31.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.659+0000 7f4e6572f640 1 --2- 172.21.15.164:0/3542532051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55abd9e415d0 0x55abd9e41a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:31.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.660+0000 7f4e65f30640 1 -- 172.21.15.164:0/3542532051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55abd9e415d0 msgr2=0x55abd9e41a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:31.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.660+0000 7f4e65f30640 1 --2- 172.21.15.164:0/3542532051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55abd9e415d0 0x55abd9e41a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.660+0000 7f4e65f30640 1 -- 172.21.15.164:0/3542532051 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55abd9e4c370 msgr2=0x55abd9e4e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:31.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.660+0000 7f4e65f30640 1 --2- 172.21.15.164:0/3542532051 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55abd9e4c370 0x55abd9e4e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.660+0000 7f4e65f30640 1 -- 172.21.15.164:0/3542532051 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abd9d333a0 con 0x55abd9e42b40 2024-09-19T08:50:31.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.660+0000 7f4e567fc640 1 --2- 172.21.15.164:0/1853844771 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f4e4c0a33b0 0x7f4e4c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:31.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.660+0000 7f4e6572f640 1 --2- 172.21.15.164:0/1853844771 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f4e4c0a33b0 0x7f4e4c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:31.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.660+0000 7f4e65f30640 1 --2- 172.21.15.164:0/3542532051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abd9e42b40 0x55abd9e41090 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4e58006010 tx=0x7f4e58009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=122bfffac4b5c3d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:31.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.660+0000 7f4e567fc640 1 -- 172.21.15.164:0/1853844771 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f4e500d00d0 con 0x7f4e60102790 2024-09-19T08:50:31.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.660+0000 7f4e4b7fe640 1 -- 172.21.15.164:0/3542532051 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e58003b70 con 0x55abd9e42b40 2024-09-19T08:50:31.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.660+0000 7f4e4b7fe640 1 -- 172.21.15.164:0/3542532051 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e58036040 con 0x55abd9e42b40 2024-09-19T08:50:31.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.660+0000 7f4e4b7fe640 1 -- 172.21.15.164:0/3542532051 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e580046a0 con 0x55abd9e42b40 2024-09-19T08:50:31.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.661+0000 7f4e69d6c740 1 -- 172.21.15.164:0/3542532051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abd9e42b40 msgr2=0x55abd9e41090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:31.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.661+0000 7f4e6572f640 1 --2- 172.21.15.164:0/1853844771 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f4e4c0a33b0 0x7f4e4c0a5870 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55abd9e3eac0 tx=0x7f4e5c00b5a0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:31.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.661+0000 7f4e69d6c740 1 --2- 172.21.15.164:0/3542532051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abd9e42b40 0x55abd9e41090 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4e58006010 tx=0x7f4e58009800 comp rx=0 tx=0).stop 2024-09-19T08:50:31.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.661+0000 7f4e69d6c740 1 -- 172.21.15.164:0/3542532051 shutdown_connections 2024-09-19T08:50:31.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.661+0000 7f4e69d6c740 1 --2- 172.21.15.164:0/3542532051 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55abd9e4c370 0x55abd9e4e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.661+0000 7f4e69d6c740 1 --2- 172.21.15.164:0/3542532051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55abd9e415d0 0x55abd9e41a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.661+0000 7f4e69d6c740 1 --2- 172.21.15.164:0/3542532051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abd9e42b40 0x55abd9e41090 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.661+0000 7f4e69d6c740 1 -- 172.21.15.164:0/3542532051 >> 172.21.15.164:0/3542532051 conn(0x55abd9e0a310 msgr2=0x55abd9e3dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:31.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.661+0000 7f4e69d6c740 1 -- 172.21.15.164:0/3542532051 shutdown_connections 2024-09-19T08:50:31.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.661+0000 7f4e69d6c740 1 -- 172.21.15.164:0/3542532051 wait complete. 2024-09-19T08:50:31.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.661+0000 7f4e69d6c740 1 -- start start 2024-09-19T08:50:31.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.661+0000 7f4e69d6c740 1 Processor -- start 2024-09-19T08:50:31.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.662+0000 7f4e69d6c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abd9e415d0 0x55abd9e4bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:31.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.662+0000 7f4e69d6c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55abd9e42b40 0x55abd9e49000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:31.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.662+0000 7f4e69d6c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55abd9e4c370 0x55abd9e495d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:31.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.662+0000 7f4e69d6c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55abd9e4ee90 con 0x55abd9e42b40 2024-09-19T08:50:31.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.662+0000 7f4e69d6c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55abd9d62060 con 0x55abd9e4c370 2024-09-19T08:50:31.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.662+0000 7f4e69d6c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55abd9e4f010 con 0x55abd9e415d0 2024-09-19T08:50:31.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.662+0000 7f4e69d6c740 10 client.0 _ll_register_callbacks cb 0x55abd9ab58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:50:31.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.662+0000 7f4e64f2e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55abd9e42b40 0x55abd9e49000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:31.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.662+0000 7f4e64f2e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55abd9e42b40 0x55abd9e49000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46066/0 (socket says 172.21.15.164:46066) 2024-09-19T08:50:31.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.662+0000 7f4e64f2e640 1 -- 172.21.15.164:0/1631369446 learned_addr learned my addr 172.21.15.164:0/1631369446 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:31.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.662+0000 7f4e6572f640 1 --2- 172.21.15.164:0/1631369446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abd9e415d0 0x55abd9e4bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:31.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.662+0000 7f4e65f30640 1 --2- 172.21.15.164:0/1631369446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55abd9e4c370 0x55abd9e495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:31.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.662+0000 7f4e64f2e640 1 -- 172.21.15.164:0/1631369446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abd9e415d0 msgr2=0x55abd9e4bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:31.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.662+0000 7f4e64f2e640 1 --2- 172.21.15.164:0/1631369446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abd9e415d0 0x55abd9e4bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.662+0000 7f4e64f2e640 1 -- 172.21.15.164:0/1631369446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55abd9e4c370 msgr2=0x55abd9e495d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:31.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.662+0000 7f4e64f2e640 1 --2- 172.21.15.164:0/1631369446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55abd9e4c370 0x55abd9e495d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.662+0000 7f4e64f2e640 1 -- 172.21.15.164:0/1631369446 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abd9b183c0 con 0x55abd9e42b40 2024-09-19T08:50:31.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.662+0000 7f4e64f2e640 1 --2- 172.21.15.164:0/1631369446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55abd9e42b40 0x55abd9e49000 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4e5009c3f0 tx=0x7f4e500d3850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992dd9a93da5102a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:31.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.662+0000 7f4e48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:50:31.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.662+0000 7f4e48ff9640 1 -- 172.21.15.164:0/1631369446 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5009c230 con 0x55abd9e42b40 2024-09-19T08:50:31.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.663+0000 7f4e69d6c740 1 -- 172.21.15.164:0/1631369446 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e58036040 con 0x55abd9e42b40 2024-09-19T08:50:31.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.663+0000 7f4e48ff9640 1 -- 172.21.15.164:0/1631369446 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f4e500a4c40 con 0x55abd9e42b40 2024-09-19T08:50:31.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.663+0000 7f4e48ff9640 1 -- 172.21.15.164:0/1631369446 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4e5009bdc0 con 0x55abd9e42b40 2024-09-19T08:50:31.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.663+0000 7f4e69d6c740 10 client.16395 fetch_fsmap learned FSMap version 30 2024-09-19T08:50:31.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.663+0000 7f4e69d6c740 1 -- 172.21.15.164:0/1631369446 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55abd9e5a970 con 0x55abd9e42b40 2024-09-19T08:50:31.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.663+0000 7f4e48ff9640 1 -- 172.21.15.164:0/1631369446 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e58036040 con 0x55abd9e42b40 2024-09-19T08:50:31.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.663+0000 7f4e48ff9640 1 -- 172.21.15.164:0/1631369446 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f4e500d2110 con 0x55abd9e42b40 2024-09-19T08:50:31.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.663+0000 7f4e69d6c740 10 client.16395 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:50:31.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.663+0000 7f4e69d6c740 10 client.16395 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-19T08:50:31.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.663+0000 7f4e69d6c740 20 client.16395 populate_metadata read hostname 'smithi164' 2024-09-19T08:50:31.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.663+0000 7f4e69d6c740 1 --2- 172.21.15.164:0/1631369446 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55abd9e5be30 0x55abd9e5e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:31.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.663+0000 7f4e69d6c740 4 client.16395 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:50:31.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.663+0000 7f4e69d6c740 1 -- 172.21.15.164:0/1631369446 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abd9db4860 con 0x55abd9e5be30 2024-09-19T08:50:31.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.663+0000 7f4e6572f640 1 --2- 172.21.15.164:0/1631369446 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55abd9e5be30 0x55abd9e5e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:31.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.664+0000 7f4e6572f640 1 --2- 172.21.15.164:0/1631369446 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55abd9e5be30 0x55abd9e5e240 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=10e69557611344c8 server_cookie=3fd83ede9caf0e40 in_seq=0 out_seq=0 2024-09-19T08:50:31.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.664+0000 7f4e48ff9640 10 client.16395 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:50:31.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.664+0000 7f4e48ff9640 1 -- 172.21.15.164:0/1631369446 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55abd9db4860 con 0x55abd9e5be30 2024-09-19T08:50:31.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.664+0000 7f4e48ff9640 10 client.16395 handle_command_reply: tid=0 2024-09-19T08:50:31.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.664+0000 7f4e69d6c740 1 client.16395 shutdown 2024-09-19T08:50:31.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.664+0000 7f4e69d6c740 1 -- 172.21.15.164:0/1631369446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55abd9e42b40 msgr2=0x55abd9e49000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:31.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.664+0000 7f4e69d6c740 1 --2- 172.21.15.164:0/1631369446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55abd9e42b40 0x55abd9e49000 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4e5009c3f0 tx=0x7f4e500d3850 comp rx=0 tx=0).stop 2024-09-19T08:50:31.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.664+0000 7f4e69d6c740 1 -- 172.21.15.164:0/1631369446 shutdown_connections 2024-09-19T08:50:31.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.664+0000 7f4e69d6c740 1 --2- 172.21.15.164:0/1631369446 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55abd9e5be30 0x55abd9e5e240 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.664+0000 7f4e69d6c740 1 --2- 172.21.15.164:0/1631369446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55abd9e4c370 0x55abd9e495d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.664+0000 7f4e69d6c740 1 --2- 172.21.15.164:0/1631369446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55abd9e42b40 0x55abd9e49000 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.664+0000 7f4e48ff9640 0 client.16395 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:50:31.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.664+0000 7f4e69d6c740 1 --2- 172.21.15.164:0/1631369446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abd9e415d0 0x55abd9e4bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.664+0000 7f4e69d6c740 1 -- 172.21.15.164:0/1631369446 >> 172.21.15.164:0/1631369446 conn(0x55abd9e0a310 msgr2=0x55abd9e43960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:31.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.664+0000 7f4e69d6c740 1 -- 172.21.15.164:0/1631369446 shutdown_connections 2024-09-19T08:50:31.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.664+0000 7f4e69d6c740 1 -- 172.21.15.164:0/1631369446 wait complete. 2024-09-19T08:50:31.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.665+0000 7f4e69d6c740 20 client.16395 trim_cache size 0 max 16384 2024-09-19T08:50:31.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.677+0000 7f4e69d6c740 1 Processor -- start 2024-09-19T08:50:31.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.677+0000 7f4e69d6c740 1 -- start start 2024-09-19T08:50:31.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.678+0000 7f4e69d6c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55abd9e415d0 0x55abd9e58130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:31.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.678+0000 7f4e69d6c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abd9e42b40 0x55abd9e63ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:31.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.678+0000 7f4e69d6c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55abd9e4c370 0x55abd9e64000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:31.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.678+0000 7f4e69d6c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4e601af820 con 0x55abd9e4c370 2024-09-19T08:50:31.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.678+0000 7f4e69d6c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4e601b1420 con 0x55abd9e415d0 2024-09-19T08:50:31.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.678+0000 7f4e69d6c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4e601afef0 con 0x55abd9e42b40 2024-09-19T08:50:31.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.678+0000 7f4e65f30640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55abd9e4c370 0x55abd9e64000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:31.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.678+0000 7f4e65f30640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55abd9e4c370 0x55abd9e64000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46068/0 (socket says 172.21.15.164:46068) 2024-09-19T08:50:31.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.678+0000 7f4e65f30640 1 -- 172.21.15.164:0/3333548141 learned_addr learned my addr 172.21.15.164:0/3333548141 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:31.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.678+0000 7f4e6572f640 1 --2- 172.21.15.164:0/3333548141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55abd9e415d0 0x55abd9e58130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:31.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.678+0000 7f4e64f2e640 1 --2- 172.21.15.164:0/3333548141 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abd9e42b40 0x55abd9e63ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:31.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.678+0000 7f4e65f30640 1 -- 172.21.15.164:0/3333548141 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abd9e42b40 msgr2=0x55abd9e63ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:31.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.678+0000 7f4e65f30640 1 --2- 172.21.15.164:0/3333548141 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abd9e42b40 0x55abd9e63ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.678+0000 7f4e65f30640 1 -- 172.21.15.164:0/3333548141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55abd9e415d0 msgr2=0x55abd9e58130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:31.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.678+0000 7f4e65f30640 1 --2- 172.21.15.164:0/3333548141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55abd9e415d0 0x55abd9e58130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.678+0000 7f4e65f30640 1 -- 172.21.15.164:0/3333548141 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abd9e44120 con 0x55abd9e4c370 2024-09-19T08:50:31.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.678+0000 7f4e65f30640 1 --2- 172.21.15.164:0/3333548141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55abd9e4c370 0x55abd9e64000 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4e58009ca0 tx=0x7f4e58004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a95c18a5155350d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:31.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.678+0000 7f4e4affd640 1 -- 172.21.15.164:0/3333548141 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e580043b0 con 0x55abd9e4c370 2024-09-19T08:50:31.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.678+0000 7f4e4affd640 1 -- 172.21.15.164:0/3333548141 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e58036040 con 0x55abd9e4c370 2024-09-19T08:50:31.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.678+0000 7f4e4affd640 1 -- 172.21.15.164:0/3333548141 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e58003bd0 con 0x55abd9e4c370 2024-09-19T08:50:31.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.679+0000 7f4e69d6c740 1 -- 172.21.15.164:0/3333548141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55abd9e4c370 msgr2=0x55abd9e64000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:31.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.679+0000 7f4e69d6c740 1 --2- 172.21.15.164:0/3333548141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55abd9e4c370 0x55abd9e64000 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4e58009ca0 tx=0x7f4e58004280 comp rx=0 tx=0).stop 2024-09-19T08:50:31.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.679+0000 7f4e69d6c740 1 -- 172.21.15.164:0/3333548141 shutdown_connections 2024-09-19T08:50:31.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.679+0000 7f4e69d6c740 1 --2- 172.21.15.164:0/3333548141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55abd9e4c370 0x55abd9e64000 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.679+0000 7f4e69d6c740 1 --2- 172.21.15.164:0/3333548141 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abd9e42b40 0x55abd9e63ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.679+0000 7f4e69d6c740 1 --2- 172.21.15.164:0/3333548141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55abd9e415d0 0x55abd9e58130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.679+0000 7f4e69d6c740 1 -- 172.21.15.164:0/3333548141 >> 172.21.15.164:0/3333548141 conn(0x55abd9e0a310 msgr2=0x55abd9e480f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:31.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.679+0000 7f4e69d6c740 1 -- 172.21.15.164:0/3333548141 shutdown_connections 2024-09-19T08:50:31.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.679+0000 7f4e69d6c740 1 -- 172.21.15.164:0/3333548141 wait complete. 2024-09-19T08:50:31.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.679+0000 7f4e69d6c740 1 -- start start 2024-09-19T08:50:31.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.680+0000 7f4e69d6c740 1 Processor -- start 2024-09-19T08:50:31.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.680+0000 7f4e69d6c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abd9e415d0 0x55abd9e542a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:31.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.680+0000 7f4e69d6c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55abd9e42b40 0x55abd9e547e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:31.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.680+0000 7f4e69d6c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55abd9e4c370 0x55abd9e51270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:31.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.680+0000 7f4e64f2e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55abd9e42b40 0x55abd9e547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:31.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.680+0000 7f4e64f2e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55abd9e42b40 0x55abd9e547e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46082/0 (socket says 172.21.15.164:46082) 2024-09-19T08:50:31.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.680+0000 7f4e64f2e640 1 -- 172.21.15.164:0/2336712346 learned_addr learned my addr 172.21.15.164:0/2336712346 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:31.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.680+0000 7f4e69d6c740 1 -- 172.21.15.164:0/2336712346 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4e601b0c90 con 0x55abd9e42b40 2024-09-19T08:50:31.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.680+0000 7f4e69d6c740 1 -- 172.21.15.164:0/2336712346 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4e601b05c0 con 0x55abd9e4c370 2024-09-19T08:50:31.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.680+0000 7f4e69d6c740 1 -- 172.21.15.164:0/2336712346 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4e601ad820 con 0x55abd9e415d0 2024-09-19T08:50:31.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.680+0000 7f4e65f30640 1 --2- 172.21.15.164:0/2336712346 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55abd9e4c370 0x55abd9e51270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:31.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.680+0000 7f4e6572f640 1 --2- 172.21.15.164:0/2336712346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abd9e415d0 0x55abd9e542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:31.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.681+0000 7f4e69d6c740 10 client.0 _ll_register_callbacks cb 0x7f4e5000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:50:31.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.681+0000 7f4e64f2e640 1 -- 172.21.15.164:0/2336712346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abd9e415d0 msgr2=0x55abd9e542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:31.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.681+0000 7f4e64f2e640 1 --2- 172.21.15.164:0/2336712346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abd9e415d0 0x55abd9e542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.681+0000 7f4e64f2e640 1 -- 172.21.15.164:0/2336712346 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55abd9e4c370 msgr2=0x55abd9e51270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:31.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.681+0000 7f4e64f2e640 1 --2- 172.21.15.164:0/2336712346 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55abd9e4c370 0x55abd9e51270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.681+0000 7f4e64f2e640 1 -- 172.21.15.164:0/2336712346 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abd9e517b0 con 0x55abd9e42b40 2024-09-19T08:50:31.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.681+0000 7f4e65f30640 1 --2- 172.21.15.164:0/2336712346 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55abd9e4c370 0x55abd9e51270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:50:31.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.681+0000 7f4e64f2e640 1 --2- 172.21.15.164:0/2336712346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55abd9e42b40 0x55abd9e547e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4e50098e30 tx=0x7f4e500d3ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e2f4e2f6eb4ab8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:31.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.681+0000 7f4e48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:50:31.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.681+0000 7f4e48ff9640 1 -- 172.21.15.164:0/2336712346 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5009bfa0 con 0x55abd9e42b40 2024-09-19T08:50:31.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.681+0000 7f4e48ff9640 1 -- 172.21.15.164:0/2336712346 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f4e5009b8b0 con 0x55abd9e42b40 2024-09-19T08:50:31.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.681+0000 7f4e69d6c740 1 -- 172.21.15.164:0/2336712346 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e58036040 con 0x55abd9e42b40 2024-09-19T08:50:31.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.681+0000 7f4e48ff9640 1 -- 172.21.15.164:0/2336712346 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4e5009c230 con 0x55abd9e42b40 2024-09-19T08:50:31.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.681+0000 7f4e69d6c740 10 client.16401 fetch_fsmap learned FSMap version 30 2024-09-19T08:50:31.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.681+0000 7f4e69d6c740 1 -- 172.21.15.164:0/2336712346 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55abd9ec7120 con 0x55abd9e42b40 2024-09-19T08:50:31.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.681+0000 7f4e48ff9640 1 -- 172.21.15.164:0/2336712346 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e58036040 con 0x55abd9e42b40 2024-09-19T08:50:31.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.682+0000 7f4e48ff9640 1 -- 172.21.15.164:0/2336712346 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f4e5009b320 con 0x55abd9e42b40 2024-09-19T08:50:31.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.682+0000 7f4e69d6c740 10 client.16401 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:50:31.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.682+0000 7f4e69d6c740 10 client.16401 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-19T08:50:31.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.682+0000 7f4e69d6c740 20 client.16401 populate_metadata read hostname 'smithi164' 2024-09-19T08:50:31.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.682+0000 7f4e69d6c740 1 --2- 172.21.15.164:0/2336712346 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55abd9ec76b0 0x55abd9ec9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:31.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.682+0000 7f4e69d6c740 4 client.16401 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:50:31.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.682+0000 7f4e69d6c740 1 -- 172.21.15.164:0/2336712346 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55abd9daed70 con 0x55abd9ec76b0 2024-09-19T08:50:31.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.682+0000 7f4e6572f640 1 --2- 172.21.15.164:0/2336712346 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55abd9ec76b0 0x55abd9ec9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:31.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.682+0000 7f4e48ff9640 10 client.16401 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:50:31.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.682+0000 7f4e6572f640 1 --2- 172.21.15.164:0/2336712346 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55abd9ec76b0 0x55abd9ec9ad0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5fc74659289b8b37 server_cookie=cae99862b083d499 in_seq=0 out_seq=0 2024-09-19T08:50:31.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.682+0000 7f4e48ff9640 1 -- 172.21.15.164:0/2336712346 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55abd9daed70 con 0x55abd9ec76b0 2024-09-19T08:50:31.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.682+0000 7f4e48ff9640 10 client.16401 handle_command_reply: tid=0 2024-09-19T08:50:31.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.682+0000 7f4e69d6c740 1 client.16401 shutdown 2024-09-19T08:50:31.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.683+0000 7f4e69d6c740 1 -- 172.21.15.164:0/2336712346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55abd9e42b40 msgr2=0x55abd9e547e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:31.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.683+0000 7f4e69d6c740 1 --2- 172.21.15.164:0/2336712346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55abd9e42b40 0x55abd9e547e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4e50098e30 tx=0x7f4e500d3ac0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.683+0000 7f4e69d6c740 1 -- 172.21.15.164:0/2336712346 shutdown_connections 2024-09-19T08:50:31.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.683+0000 7f4e69d6c740 1 --2- 172.21.15.164:0/2336712346 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55abd9ec76b0 0x55abd9ec9ad0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.683+0000 7f4e69d6c740 1 --2- 172.21.15.164:0/2336712346 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55abd9e4c370 0x55abd9e51270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.683+0000 7f4e48ff9640 0 client.16401 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:50:31.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.683+0000 7f4e69d6c740 1 --2- 172.21.15.164:0/2336712346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55abd9e42b40 0x55abd9e547e0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.683+0000 7f4e69d6c740 1 --2- 172.21.15.164:0/2336712346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abd9e415d0 0x55abd9e542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.683+0000 7f4e69d6c740 1 -- 172.21.15.164:0/2336712346 >> 172.21.15.164:0/2336712346 conn(0x55abd9e0a310 msgr2=0x55abd9e430e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:31.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.683+0000 7f4e69d6c740 1 -- 172.21.15.164:0/2336712346 shutdown_connections 2024-09-19T08:50:31.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.683+0000 7f4e69d6c740 1 -- 172.21.15.164:0/2336712346 wait complete. 2024-09-19T08:50:31.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.683+0000 7f4e69d6c740 20 client.16401 trim_cache size 0 max 16384 2024-09-19T08:50:31.693 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:50:31.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.683+0000 7f4e67993640 1 -- 172.21.15.164:0/1853844771 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f4e4c0a33b0 msgr2=0x7f4e4c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:31.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.683+0000 7f4e67993640 1 --2- 172.21.15.164:0/1853844771 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f4e4c0a33b0 0x7f4e4c0a5870 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55abd9e3eac0 tx=0x7f4e5c00b5a0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.683+0000 7f4e67993640 1 -- 172.21.15.164:0/1853844771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e60102790 msgr2=0x7f4e60074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:31.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.683+0000 7f4e67993640 1 --2- 172.21.15.164:0/1853844771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e60102790 0x7f4e60074a30 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000e9f0 tx=0x7f4e5000edf0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.683+0000 7f4e67993640 1 -- 172.21.15.164:0/1853844771 shutdown_connections 2024-09-19T08:50:31.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.683+0000 7f4e67993640 1 --2- 172.21.15.164:0/1853844771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e60106f10 0x7f4e60074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.683+0000 7f4e67993640 1 --2- 172.21.15.164:0/1853844771 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f4e4c0a33b0 0x7f4e4c0a5870 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.683+0000 7f4e67993640 1 --2- 172.21.15.164:0/1853844771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e60102790 0x7f4e60074a30 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.683+0000 7f4e67993640 1 --2- 172.21.15.164:0/1853844771 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e60101de0 0x7f4e600744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:31.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.684+0000 7f4e67993640 1 -- 172.21.15.164:0/1853844771 >> 172.21.15.164:0/1853844771 conn(0x7f4e600fd8c0 msgr2=0x7f4e600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:31.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.684+0000 7f4e67993640 1 -- 172.21.15.164:0/1853844771 shutdown_connections 2024-09-19T08:50:31.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:31.684+0000 7f4e67993640 1 -- 172.21.15.164:0/1853844771 wait complete. 2024-09-19T08:50:31.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:31 smithi164 ceph-mon[30928]: pgmap v144: 97 pgs: 97 active+clean; 39 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 826 MiB/s wr, 2.23k op/s 2024-09-19T08:50:31.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:31 smithi170 ceph-mon[35072]: pgmap v144: 97 pgs: 97 active+clean; 39 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 826 MiB/s wr, 2.23k op/s 2024-09-19T08:50:31.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:31 smithi173 ceph-mon[35306]: pgmap v144: 97 pgs: 97 active+clean; 39 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 826 MiB/s wr, 2.23k op/s 2024-09-19T08:50:32.718 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T08:50:33.478 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ea4bdac6', quiesce took 14.4 seconds. Will release after: 15.4 2024-09-19T08:50:33.510 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:50:33.754 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:33 smithi164 ceph-mon[30928]: pgmap v145: 97 pgs: 97 active+clean; 40 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 715 MiB/s wr, 1.72k op/s 2024-09-19T08:50:33.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:33 smithi170 ceph-mon[35072]: pgmap v145: 97 pgs: 97 active+clean; 40 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 715 MiB/s wr, 1.72k op/s 2024-09-19T08:50:33.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:33 smithi173 ceph-mon[35306]: pgmap v145: 97 pgs: 97 active+clean; 40 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 715 MiB/s wr, 1.72k op/s 2024-09-19T08:50:34.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.994+0000 7f180682f640 1 -- 172.21.15.164:0/3917179943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1800106f20 msgr2=0x7f180010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:34.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.994+0000 7f180682f640 1 --2- 172.21.15.164:0/3917179943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1800106f20 0x7f180010eaa0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f17f8009ca0 tx=0x7f17f802a8c0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.994+0000 7f180682f640 1 -- 172.21.15.164:0/3917179943 shutdown_connections 2024-09-19T08:50:34.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.994+0000 7f180682f640 1 --2- 172.21.15.164:0/3917179943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1800106f20 0x7f180010eaa0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.994+0000 7f180682f640 1 --2- 172.21.15.164:0/3917179943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18001027a0 0x7f18001067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.994+0000 7f180682f640 1 --2- 172.21.15.164:0/3917179943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1800101df0 0x7f18001021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.994+0000 7f180682f640 1 -- 172.21.15.164:0/3917179943 >> 172.21.15.164:0/3917179943 conn(0x7f18000fd8f0 msgr2=0x7f18000ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:34.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.994+0000 7f180682f640 1 -- 172.21.15.164:0/3917179943 shutdown_connections 2024-09-19T08:50:34.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.994+0000 7f180682f640 1 -- 172.21.15.164:0/3917179943 wait complete. 2024-09-19T08:50:34.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.994+0000 7f180682f640 1 Processor -- start 2024-09-19T08:50:34.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.994+0000 7f180682f640 1 -- start start 2024-09-19T08:50:34.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.995+0000 7f180682f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1800101df0 0x7f18001984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:34.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.995+0000 7f180682f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18001027a0 0x7f18001989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:34.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.995+0000 7f180682f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1800106f20 0x7f180019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:34.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.995+0000 7f180682f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1800111fc0 con 0x7f18001027a0 2024-09-19T08:50:34.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.995+0000 7f180682f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1800111e40 con 0x7f1800106f20 2024-09-19T08:50:34.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.995+0000 7f180682f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1800112140 con 0x7f1800101df0 2024-09-19T08:50:34.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.995+0000 7f17ff7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18001027a0 0x7f18001989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:34.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.995+0000 7f17ff7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18001027a0 0x7f18001989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60148/0 (socket says 172.21.15.164:60148) 2024-09-19T08:50:34.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.995+0000 7f17ff7fe640 1 -- 172.21.15.164:0/4088842383 learned_addr learned my addr 172.21.15.164:0/4088842383 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:34.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.995+0000 7f1804dcc640 1 --2- 172.21.15.164:0/4088842383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1800106f20 0x7f180019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:34.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.995+0000 7f17fffff640 1 --2- 172.21.15.164:0/4088842383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1800101df0 0x7f18001984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:34.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.995+0000 7f17ff7fe640 1 -- 172.21.15.164:0/4088842383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1800101df0 msgr2=0x7f18001984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:34.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.995+0000 7f17ff7fe640 1 --2- 172.21.15.164:0/4088842383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1800101df0 0x7f18001984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.995+0000 7f17ff7fe640 1 -- 172.21.15.164:0/4088842383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1800106f20 msgr2=0x7f180019cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:34.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.995+0000 7f17ff7fe640 1 --2- 172.21.15.164:0/4088842383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1800106f20 0x7f180019cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.995+0000 7f17ff7fe640 1 -- 172.21.15.164:0/4088842383 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f180019d450 con 0x7f18001027a0 2024-09-19T08:50:34.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.995+0000 7f17ff7fe640 1 --2- 172.21.15.164:0/4088842383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18001027a0 0x7f18001989e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f17f400b4a0 tx=0x7f17f400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:34.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.995+0000 7f17fd7fa640 1 -- 172.21.15.164:0/4088842383 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f4014070 con 0x7f18001027a0 2024-09-19T08:50:34.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.996+0000 7f180682f640 1 -- 172.21.15.164:0/4088842383 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f180019d740 con 0x7f18001027a0 2024-09-19T08:50:34.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.996+0000 7f17fd7fa640 1 -- 172.21.15.164:0/4088842383 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17f4010af0 con 0x7f18001027a0 2024-09-19T08:50:34.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.996+0000 7f17fd7fa640 1 -- 172.21.15.164:0/4088842383 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f4017d10 con 0x7f18001027a0 2024-09-19T08:50:34.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.996+0000 7f180682f640 1 -- 172.21.15.164:0/4088842383 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18001a5020 con 0x7f18001027a0 2024-09-19T08:50:34.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.998+0000 7f17fd7fa640 1 -- 172.21.15.164:0/4088842383 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f17f4017070 con 0x7f18001027a0 2024-09-19T08:50:34.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.998+0000 7f1808c08740 1 Processor -- start 2024-09-19T08:50:34.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.998+0000 7f1808c08740 1 -- start start 2024-09-19T08:50:34.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.998+0000 7f1808c08740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eae42b9b40 0x55eae42b8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:34.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.998+0000 7f1808c08740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eae42b85d0 0x55eae42b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:34.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.998+0000 7f1808c08740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eae42c3370 0x55eae42c5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:34.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.998+0000 7f1808c08740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55eae421e2f0 con 0x55eae42c3370 2024-09-19T08:50:34.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.998+0000 7f1808c08740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55eae41c5cf0 con 0x55eae42b85d0 2024-09-19T08:50:34.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.998+0000 7f1808c08740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55eae421a2b0 con 0x55eae42b9b40 2024-09-19T08:50:34.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.999+0000 7f17ff7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eae42c3370 0x55eae42c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:34.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.999+0000 7f17fd7fa640 1 --2- 172.21.15.164:0/4088842383 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f17e80a33b0 0x7f17e80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:34.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.999+0000 7f17ff7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eae42c3370 0x55eae42c5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60150/0 (socket says 172.21.15.164:60150) 2024-09-19T08:50:34.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.999+0000 7f17ff7fe640 1 -- 172.21.15.164:0/1903889723 learned_addr learned my addr 172.21.15.164:0/1903889723 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:34.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.999+0000 7f17fffff640 1 --2- 172.21.15.164:0/1903889723 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eae42b85d0 0x55eae42b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:34.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.999+0000 7f1804dcc640 1 --2- 172.21.15.164:0/1903889723 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eae42b9b40 0x55eae42b8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:34.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.999+0000 7f1804dcc640 1 --2- 172.21.15.164:0/4088842383 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f17e80a33b0 0x7f17e80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:34.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.999+0000 7f17fd7fa640 1 -- 172.21.15.164:0/4088842383 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f17f40cf080 con 0x7f18001027a0 2024-09-19T08:50:34.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.999+0000 7f17ff7fe640 1 -- 172.21.15.164:0/1903889723 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eae42b9b40 msgr2=0x55eae42b8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:34.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.999+0000 7f17ff7fe640 1 --2- 172.21.15.164:0/1903889723 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eae42b9b40 0x55eae42b8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.999+0000 7f17ff7fe640 1 -- 172.21.15.164:0/1903889723 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eae42b85d0 msgr2=0x55eae42b8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:34.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.999+0000 7f17ff7fe640 1 --2- 172.21.15.164:0/1903889723 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eae42b85d0 0x55eae42b8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.999+0000 7f17ff7fe640 1 -- 172.21.15.164:0/1903889723 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eae41aa3a0 con 0x55eae42c3370 2024-09-19T08:50:34.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.999+0000 7f17ff7fe640 1 --2- 172.21.15.164:0/1903889723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eae42c3370 0x55eae42c5760 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f17f4097e50 tx=0x7f17f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5735e35c48ed6e4b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:34.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.999+0000 7f1804dcc640 1 --2- 172.21.15.164:0/4088842383 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f17e80a33b0 0x7f17e80a5870 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f17f8031f80 tx=0x7f17f80095c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:34.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.999+0000 7f17e67fc640 1 -- 172.21.15.164:0/1903889723 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f401e070 con 0x55eae42c3370 2024-09-19T08:50:34.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:33.999+0000 7f17e67fc640 1 -- 172.21.15.164:0/1903889723 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17f409bb80 con 0x55eae42c3370 2024-09-19T08:50:34.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.000+0000 7f17e67fc640 1 -- 172.21.15.164:0/1903889723 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f40cca60 con 0x55eae42c3370 2024-09-19T08:50:34.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.000+0000 7f1808c08740 1 -- 172.21.15.164:0/1903889723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eae42c3370 msgr2=0x55eae42c5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:34.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.000+0000 7f1808c08740 1 --2- 172.21.15.164:0/1903889723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eae42c3370 0x55eae42c5760 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f17f4097e50 tx=0x7f17f40046c0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.000+0000 7f1808c08740 1 -- 172.21.15.164:0/1903889723 shutdown_connections 2024-09-19T08:50:34.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.000+0000 7f1808c08740 1 --2- 172.21.15.164:0/1903889723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eae42c3370 0x55eae42c5760 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.000+0000 7f1808c08740 1 --2- 172.21.15.164:0/1903889723 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eae42b85d0 0x55eae42b8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.000+0000 7f1808c08740 1 --2- 172.21.15.164:0/1903889723 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eae42b9b40 0x55eae42b8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.000+0000 7f1808c08740 1 -- 172.21.15.164:0/1903889723 >> 172.21.15.164:0/1903889723 conn(0x55eae4281310 msgr2=0x55eae42b4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:34.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.000+0000 7f1808c08740 1 -- 172.21.15.164:0/1903889723 shutdown_connections 2024-09-19T08:50:34.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.000+0000 7f1808c08740 1 -- 172.21.15.164:0/1903889723 wait complete. 2024-09-19T08:50:34.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.001+0000 7f1808c08740 1 -- start start 2024-09-19T08:50:34.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.001+0000 7f1808c08740 1 Processor -- start 2024-09-19T08:50:34.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.001+0000 7f1808c08740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eae42b85d0 0x55eae42c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:34.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.001+0000 7f1808c08740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eae42b9b40 0x55eae42c0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:34.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.001+0000 7f1808c08740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eae42c3370 0x55eae42c07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:34.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.001+0000 7f1808c08740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55eae42c6070 con 0x55eae42b85d0 2024-09-19T08:50:34.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.001+0000 7f1808c08740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55eae41d9060 con 0x55eae42c3370 2024-09-19T08:50:34.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.001+0000 7f1808c08740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55eae42c61f0 con 0x55eae42b9b40 2024-09-19T08:50:34.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.001+0000 7f1808c08740 10 client.0 _ll_register_callbacks cb 0x55eae3f2c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:50:34.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.001+0000 7f17fffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eae42b85d0 0x55eae42c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:34.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.001+0000 7f17fffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eae42b85d0 0x55eae42c2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60158/0 (socket says 172.21.15.164:60158) 2024-09-19T08:50:34.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.001+0000 7f17fffff640 1 -- 172.21.15.164:0/2131920947 learned_addr learned my addr 172.21.15.164:0/2131920947 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:34.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.001+0000 7f1804dcc640 1 --2- 172.21.15.164:0/2131920947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eae42c3370 0x55eae42c07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:34.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.001+0000 7f17ff7fe640 1 --2- 172.21.15.164:0/2131920947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eae42b9b40 0x55eae42c0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:34.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.002+0000 7f17fffff640 1 -- 172.21.15.164:0/2131920947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eae42b9b40 msgr2=0x55eae42c0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:34.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.002+0000 7f17fffff640 1 --2- 172.21.15.164:0/2131920947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eae42b9b40 0x55eae42c0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.002+0000 7f17fffff640 1 -- 172.21.15.164:0/2131920947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eae42c3370 msgr2=0x55eae42c07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:34.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.002+0000 7f17fffff640 1 --2- 172.21.15.164:0/2131920947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eae42c3370 0x55eae42c07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.002+0000 7f17fffff640 1 -- 172.21.15.164:0/2131920947 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eae3f8f3c0 con 0x55eae42b85d0 2024-09-19T08:50:34.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.002+0000 7f17fffff640 1 --2- 172.21.15.164:0/2131920947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eae42b85d0 0x55eae42c2da0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f17f0011780 tx=0x7f17f0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=813ed22101b1f0a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:34.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.002+0000 7f17df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:50:34.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.002+0000 7f17df7fe640 1 -- 172.21.15.164:0/2131920947 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f001e060 con 0x55eae42b85d0 2024-09-19T08:50:34.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.002+0000 7f17df7fe640 1 -- 172.21.15.164:0/2131920947 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f17f000fdb0 con 0x55eae42b85d0 2024-09-19T08:50:34.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.002+0000 7f1808c08740 1 -- 172.21.15.164:0/2131920947 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17f409bb80 con 0x55eae42b85d0 2024-09-19T08:50:34.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.002+0000 7f17df7fe640 1 -- 172.21.15.164:0/2131920947 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f17f0023440 con 0x55eae42b85d0 2024-09-19T08:50:34.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.002+0000 7f1808c08740 10 client.16413 fetch_fsmap learned FSMap version 30 2024-09-19T08:50:34.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.002+0000 7f1808c08740 1 -- 172.21.15.164:0/2131920947 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55eae42d1bb0 con 0x55eae42b85d0 2024-09-19T08:50:34.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.002+0000 7f17df7fe640 1 -- 172.21.15.164:0/2131920947 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17f409bb80 con 0x55eae42b85d0 2024-09-19T08:50:34.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.003+0000 7f17df7fe640 1 -- 172.21.15.164:0/2131920947 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f17f0021020 con 0x55eae42b85d0 2024-09-19T08:50:34.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.003+0000 7f1808c08740 10 client.16413 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:50:34.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.003+0000 7f1808c08740 10 client.16413 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T08:50:34.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.003+0000 7f1808c08740 20 client.16413 populate_metadata read hostname 'smithi164' 2024-09-19T08:50:34.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.003+0000 7f1808c08740 1 --2- 172.21.15.164:0/2131920947 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55eae42d3070 0x55eae42d5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:34.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.003+0000 7f1808c08740 4 client.16413 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:50:34.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.003+0000 7f1808c08740 1 -- 172.21.15.164:0/2131920947 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eae422b860 con 0x55eae42d3070 2024-09-19T08:50:34.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.003+0000 7f17ff7fe640 1 --2- 172.21.15.164:0/2131920947 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55eae42d3070 0x55eae42d5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:34.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.003+0000 7f17ff7fe640 1 --2- 172.21.15.164:0/2131920947 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55eae42d3070 0x55eae42d5460 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=830790233ef3e510 server_cookie=6ea266571551d3e0 in_seq=0 out_seq=0 2024-09-19T08:50:34.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.003+0000 7f17df7fe640 10 client.16413 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:50:34.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.004+0000 7f17df7fe640 1 -- 172.21.15.164:0/2131920947 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eae422b860 con 0x55eae42d3070 2024-09-19T08:50:34.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.004+0000 7f17df7fe640 10 client.16413 handle_command_reply: tid=0 2024-09-19T08:50:34.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.004+0000 7f1808c08740 1 client.16413 shutdown 2024-09-19T08:50:34.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.004+0000 7f1808c08740 1 -- 172.21.15.164:0/2131920947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eae42b85d0 msgr2=0x55eae42c2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:34.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.004+0000 7f1808c08740 1 --2- 172.21.15.164:0/2131920947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eae42b85d0 0x55eae42c2da0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f17f0011780 tx=0x7f17f0011c40 comp rx=0 tx=0).stop 2024-09-19T08:50:34.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.004+0000 7f1808c08740 1 -- 172.21.15.164:0/2131920947 shutdown_connections 2024-09-19T08:50:34.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.004+0000 7f1808c08740 1 --2- 172.21.15.164:0/2131920947 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55eae42d3070 0x55eae42d5460 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.004+0000 7f17df7fe640 0 client.16413 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:50:34.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.004+0000 7f1808c08740 1 --2- 172.21.15.164:0/2131920947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eae42c3370 0x55eae42c07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.004+0000 7f1808c08740 1 --2- 172.21.15.164:0/2131920947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eae42b9b40 0x55eae42c0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.004+0000 7f1808c08740 1 --2- 172.21.15.164:0/2131920947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eae42b85d0 0x55eae42c2da0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.004+0000 7f1808c08740 1 -- 172.21.15.164:0/2131920947 >> 172.21.15.164:0/2131920947 conn(0x55eae4281310 msgr2=0x55eae42ba990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:34.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.004+0000 7f1808c08740 1 -- 172.21.15.164:0/2131920947 shutdown_connections 2024-09-19T08:50:34.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.004+0000 7f1808c08740 1 -- 172.21.15.164:0/2131920947 wait complete. 2024-09-19T08:50:34.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.005+0000 7f1808c08740 20 client.16413 trim_cache size 0 max 16384 2024-09-19T08:50:34.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.017+0000 7f1808c08740 1 Processor -- start 2024-09-19T08:50:34.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.017+0000 7f1808c08740 1 -- start start 2024-09-19T08:50:34.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.017+0000 7f1808c08740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eae42b85d0 0x55eae42b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:34.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.017+0000 7f1808c08740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eae42b9b40 0x55eae42bb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:34.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.017+0000 7f1808c08740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eae42c3370 0x55eae42bb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:34.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.017+0000 7f1808c08740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f18001ad7e0 con 0x55eae42c3370 2024-09-19T08:50:34.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.017+0000 7f1808c08740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f18001af3e0 con 0x55eae42b9b40 2024-09-19T08:50:34.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.017+0000 7f1808c08740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f18001adeb0 con 0x55eae42b85d0 2024-09-19T08:50:34.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.018+0000 7f17ff7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eae42b9b40 0x55eae42bb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:34.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.018+0000 7f17fffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eae42b85d0 0x55eae42b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:34.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.018+0000 7f17ff7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eae42b9b40 0x55eae42bb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:53578/0 (socket says 172.21.15.164:53578) 2024-09-19T08:50:34.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.018+0000 7f17ff7fe640 1 -- 172.21.15.164:0/1161927623 learned_addr learned my addr 172.21.15.164:0/1161927623 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:34.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.018+0000 7f1804dcc640 1 --2- 172.21.15.164:0/1161927623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eae42c3370 0x55eae42bb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:34.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.018+0000 7f17fffff640 1 -- 172.21.15.164:0/1161927623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eae42b9b40 msgr2=0x55eae42bb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:34.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.018+0000 7f17fffff640 1 --2- 172.21.15.164:0/1161927623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eae42b9b40 0x55eae42bb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.018+0000 7f17fffff640 1 -- 172.21.15.164:0/1161927623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eae42c3370 msgr2=0x55eae42bb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:34.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.018+0000 7f17fffff640 1 --2- 172.21.15.164:0/1161927623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eae42c3370 0x55eae42bb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.018+0000 7f17fffff640 1 -- 172.21.15.164:0/1161927623 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eae42b83f0 con 0x55eae42b85d0 2024-09-19T08:50:34.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.018+0000 7f17fffff640 1 --2- 172.21.15.164:0/1161927623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eae42b85d0 0x55eae42b3bf0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f17f0011750 tx=0x7f17f0005670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=871bc25ec27ee0f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:34.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.018+0000 7f17e5ffb640 1 -- 172.21.15.164:0/1161927623 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f000fdb0 con 0x55eae42b85d0 2024-09-19T08:50:34.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.019+0000 7f17e5ffb640 1 -- 172.21.15.164:0/1161927623 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17f000a810 con 0x55eae42b85d0 2024-09-19T08:50:34.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.019+0000 7f17e5ffb640 1 -- 172.21.15.164:0/1161927623 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f0003870 con 0x55eae42b85d0 2024-09-19T08:50:34.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.019+0000 7f1808c08740 1 -- 172.21.15.164:0/1161927623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eae42b85d0 msgr2=0x55eae42b3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:34.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.019+0000 7f1808c08740 1 --2- 172.21.15.164:0/1161927623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eae42b85d0 0x55eae42b3bf0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f17f0011750 tx=0x7f17f0005670 comp rx=0 tx=0).stop 2024-09-19T08:50:34.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.019+0000 7f1808c08740 1 -- 172.21.15.164:0/1161927623 shutdown_connections 2024-09-19T08:50:34.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.019+0000 7f1808c08740 1 --2- 172.21.15.164:0/1161927623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eae42c3370 0x55eae42bb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.019+0000 7f1808c08740 1 --2- 172.21.15.164:0/1161927623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eae42b9b40 0x55eae42bb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.019+0000 7f1808c08740 1 --2- 172.21.15.164:0/1161927623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eae42b85d0 0x55eae42b3bf0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.019+0000 7f1808c08740 1 -- 172.21.15.164:0/1161927623 >> 172.21.15.164:0/1161927623 conn(0x55eae4281310 msgr2=0x55eae42b34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:34.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.019+0000 7f1808c08740 1 -- 172.21.15.164:0/1161927623 shutdown_connections 2024-09-19T08:50:34.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.019+0000 7f1808c08740 1 -- 172.21.15.164:0/1161927623 wait complete. 2024-09-19T08:50:34.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.020+0000 7f1808c08740 1 -- start start 2024-09-19T08:50:34.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.020+0000 7f1808c08740 1 Processor -- start 2024-09-19T08:50:34.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.020+0000 7f1808c08740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eae42b85d0 0x55eae42c7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:34.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.020+0000 7f1808c08740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eae42b9b40 0x55eae42c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:34.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.020+0000 7f1808c08740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eae42c3370 0x55eae42cb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:34.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.020+0000 7f1808c08740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f18001aec50 con 0x55eae42b9b40 2024-09-19T08:50:34.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.020+0000 7f1808c08740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f18001ae580 con 0x55eae42b85d0 2024-09-19T08:50:34.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.020+0000 7f1808c08740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f18001ab7e0 con 0x55eae42c3370 2024-09-19T08:50:34.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.020+0000 7f1808c08740 10 client.0 _ll_register_callbacks cb 0x7f17f000c690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:50:34.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.020+0000 7f17ff7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eae42b9b40 0x55eae42c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:34.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.020+0000 7f17ff7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eae42b9b40 0x55eae42c7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60172/0 (socket says 172.21.15.164:60172) 2024-09-19T08:50:34.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.020+0000 7f17ff7fe640 1 -- 172.21.15.164:0/929889317 learned_addr learned my addr 172.21.15.164:0/929889317 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:34.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.021+0000 7f17fffff640 1 --2- 172.21.15.164:0/929889317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eae42b85d0 0x55eae42c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:34.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.021+0000 7f1804dcc640 1 --2- 172.21.15.164:0/929889317 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eae42c3370 0x55eae42cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:34.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.021+0000 7f17ff7fe640 1 -- 172.21.15.164:0/929889317 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eae42c3370 msgr2=0x55eae42cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:34.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.021+0000 7f17ff7fe640 1 --2- 172.21.15.164:0/929889317 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eae42c3370 0x55eae42cb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.021+0000 7f17ff7fe640 1 -- 172.21.15.164:0/929889317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eae42b85d0 msgr2=0x55eae42c7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:34.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.021+0000 7f17ff7fe640 1 --2- 172.21.15.164:0/929889317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eae42b85d0 0x55eae42c7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.021+0000 7f17ff7fe640 1 -- 172.21.15.164:0/929889317 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eae42cbd10 con 0x55eae42b9b40 2024-09-19T08:50:34.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.021+0000 7f1804dcc640 1 --2- 172.21.15.164:0/929889317 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eae42c3370 0x55eae42cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:50:34.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.021+0000 7f17ff7fe640 1 --2- 172.21.15.164:0/929889317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eae42b9b40 0x55eae42c7d70 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f17f40cf320 tx=0x7f17f409b380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e23798a9d740638 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:34.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.021+0000 7f17df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:50:34.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.021+0000 7f17df7fe640 1 -- 172.21.15.164:0/929889317 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f401e040 con 0x55eae42b9b40 2024-09-19T08:50:34.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.021+0000 7f17df7fe640 1 -- 172.21.15.164:0/929889317 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f17f4097af0 con 0x55eae42b9b40 2024-09-19T08:50:34.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.021+0000 7f1808c08740 1 -- 172.21.15.164:0/929889317 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17f000a810 con 0x55eae42b9b40 2024-09-19T08:50:34.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.021+0000 7f17df7fe640 1 -- 172.21.15.164:0/929889317 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f17f4010de0 con 0x55eae42b9b40 2024-09-19T08:50:34.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.021+0000 7f1808c08740 10 client.16419 fetch_fsmap learned FSMap version 30 2024-09-19T08:50:34.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.021+0000 7f1808c08740 1 -- 172.21.15.164:0/929889317 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55eae4337520 con 0x55eae42b9b40 2024-09-19T08:50:34.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.022+0000 7f17df7fe640 1 -- 172.21.15.164:0/929889317 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17f000a810 con 0x55eae42b9b40 2024-09-19T08:50:34.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.022+0000 7f17df7fe640 1 -- 172.21.15.164:0/929889317 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f17f40ccad0 con 0x55eae42b9b40 2024-09-19T08:50:34.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.022+0000 7f1808c08740 10 client.16419 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:50:34.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.022+0000 7f1808c08740 10 client.16419 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T08:50:34.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.022+0000 7f1808c08740 20 client.16419 populate_metadata read hostname 'smithi164' 2024-09-19T08:50:34.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.022+0000 7f1808c08740 1 --2- 172.21.15.164:0/929889317 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55eae42d3070 0x55eae4348940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:34.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.022+0000 7f1808c08740 4 client.16419 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:50:34.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.022+0000 7f1808c08740 1 -- 172.21.15.164:0/929889317 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eae4225d70 con 0x55eae42d3070 2024-09-19T08:50:34.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.022+0000 7f17fffff640 1 --2- 172.21.15.164:0/929889317 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55eae42d3070 0x55eae4348940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:34.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.023+0000 7f17fffff640 1 --2- 172.21.15.164:0/929889317 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55eae42d3070 0x55eae4348940 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=30c2b80b3c77897e server_cookie=f1fff22deced55aa in_seq=0 out_seq=0 2024-09-19T08:50:34.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.023+0000 7f17df7fe640 10 client.16419 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:50:34.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.023+0000 7f17df7fe640 1 -- 172.21.15.164:0/929889317 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eae4225d70 con 0x55eae42d3070 2024-09-19T08:50:34.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.023+0000 7f17df7fe640 10 client.16419 handle_command_reply: tid=0 2024-09-19T08:50:34.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.023+0000 7f1808c08740 1 client.16419 shutdown 2024-09-19T08:50:34.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.023+0000 7f1808c08740 1 -- 172.21.15.164:0/929889317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eae42b9b40 msgr2=0x55eae42c7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:34.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.023+0000 7f1808c08740 1 --2- 172.21.15.164:0/929889317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eae42b9b40 0x55eae42c7d70 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f17f40cf320 tx=0x7f17f409b380 comp rx=0 tx=0).stop 2024-09-19T08:50:34.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.023+0000 7f1808c08740 1 -- 172.21.15.164:0/929889317 shutdown_connections 2024-09-19T08:50:34.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.023+0000 7f1808c08740 1 --2- 172.21.15.164:0/929889317 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55eae42d3070 0x55eae4348940 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.023+0000 7f1808c08740 1 --2- 172.21.15.164:0/929889317 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eae42c3370 0x55eae42cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.023+0000 7f1808c08740 1 --2- 172.21.15.164:0/929889317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eae42b9b40 0x55eae42c7d70 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.023+0000 7f1808c08740 1 --2- 172.21.15.164:0/929889317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eae42b85d0 0x55eae42c7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.023+0000 7f1808c08740 1 -- 172.21.15.164:0/929889317 >> 172.21.15.164:0/929889317 conn(0x55eae4281310 msgr2=0x55eae42b3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:34.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.023+0000 7f17df7fe640 0 client.16419 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:50:34.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.023+0000 7f1808c08740 1 -- 172.21.15.164:0/929889317 shutdown_connections 2024-09-19T08:50:34.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.023+0000 7f1808c08740 1 -- 172.21.15.164:0/929889317 wait complete. 2024-09-19T08:50:34.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.023+0000 7f1808c08740 20 client.16419 trim_cache size 0 max 16384 2024-09-19T08:50:34.030 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:50:34.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.024+0000 7f180682f640 1 -- 172.21.15.164:0/4088842383 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f17e80a33b0 msgr2=0x7f17e80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:34.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.024+0000 7f180682f640 1 --2- 172.21.15.164:0/4088842383 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f17e80a33b0 0x7f17e80a5870 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f17f8031f80 tx=0x7f17f80095c0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.024+0000 7f180682f640 1 -- 172.21.15.164:0/4088842383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18001027a0 msgr2=0x7f18001989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:34.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.024+0000 7f180682f640 1 --2- 172.21.15.164:0/4088842383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18001027a0 0x7f18001989e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f17f400b4a0 tx=0x7f17f400b960 comp rx=0 tx=0).stop 2024-09-19T08:50:34.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.024+0000 7f180682f640 1 -- 172.21.15.164:0/4088842383 shutdown_connections 2024-09-19T08:50:34.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.024+0000 7f180682f640 1 --2- 172.21.15.164:0/4088842383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1800106f20 0x7f180019cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.024+0000 7f180682f640 1 --2- 172.21.15.164:0/4088842383 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f17e80a33b0 0x7f17e80a5870 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.024+0000 7f180682f640 1 --2- 172.21.15.164:0/4088842383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18001027a0 0x7f18001989e0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.024+0000 7f180682f640 1 --2- 172.21.15.164:0/4088842383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1800101df0 0x7f18001984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:34.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.024+0000 7f180682f640 1 -- 172.21.15.164:0/4088842383 >> 172.21.15.164:0/4088842383 conn(0x7f18000fd8f0 msgr2=0x7f18000fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:34.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.024+0000 7f180682f640 1 -- 172.21.15.164:0/4088842383 shutdown_connections 2024-09-19T08:50:34.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:34.024+0000 7f180682f640 1 -- 172.21.15.164:0/4088842383 wait complete. 2024-09-19T08:50:34.816 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T08:50:35.105 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:50:35.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.760+0000 7f407ac7c640 1 -- 172.21.15.164:0/1594231711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4074102ef0 msgr2=0x7f407410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:35.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.760+0000 7f407ac7c640 1 --2- 172.21.15.164:0/1594231711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4074102ef0 0x7f407410da40 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f406c009ca0 tx=0x7f406c02a950 comp rx=0 tx=0).stop 2024-09-19T08:50:35.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.760+0000 7f407ac7c640 1 -- 172.21.15.164:0/1594231711 shutdown_connections 2024-09-19T08:50:35.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.760+0000 7f407ac7c640 1 --2- 172.21.15.164:0/1594231711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4074102ef0 0x7f407410da40 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.760+0000 7f407ac7c640 1 --2- 172.21.15.164:0/1594231711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4074102570 0x7f40741029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.760+0000 7f407ac7c640 1 --2- 172.21.15.164:0/1594231711 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4074100e20 0x7f4074101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.760+0000 7f407ac7c640 1 -- 172.21.15.164:0/1594231711 >> 172.21.15.164:0/1594231711 conn(0x7f40740fc7d0 msgr2=0x7f40740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:35.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.760+0000 7f407ac7c640 1 -- 172.21.15.164:0/1594231711 shutdown_connections 2024-09-19T08:50:35.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.760+0000 7f407ac7c640 1 -- 172.21.15.164:0/1594231711 wait complete. 2024-09-19T08:50:35.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.761+0000 7f407ac7c640 1 Processor -- start 2024-09-19T08:50:35.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.761+0000 7f407ac7c640 1 -- start start 2024-09-19T08:50:35.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.761+0000 7f407ac7c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4074100e20 0x7f4074199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:35.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.761+0000 7f407ac7c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4074102570 0x7f40741998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:35.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.761+0000 7f407ac7c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4074102ef0 0x7f40741a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:35.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.761+0000 7f407ac7c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4074110f80 con 0x7f4074102ef0 2024-09-19T08:50:35.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.761+0000 7f407ac7c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4074110e00 con 0x7f4074102570 2024-09-19T08:50:35.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.761+0000 7f407ac7c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4074111100 con 0x7f4074100e20 2024-09-19T08:50:35.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.761+0000 7f4079219640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4074102ef0 0x7f40741a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:35.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.761+0000 7f4079219640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4074102ef0 0x7f40741a43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60194/0 (socket says 172.21.15.164:60194) 2024-09-19T08:50:35.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.761+0000 7f4079219640 1 -- 172.21.15.164:0/2048514596 learned_addr learned my addr 172.21.15.164:0/2048514596 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:35.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.761+0000 7f4073fff640 1 --2- 172.21.15.164:0/2048514596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4074102570 0x7f40741998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:35.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.761+0000 7f4078a18640 1 --2- 172.21.15.164:0/2048514596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4074100e20 0x7f4074199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:35.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.761+0000 7f4079219640 1 -- 172.21.15.164:0/2048514596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4074100e20 msgr2=0x7f4074199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:35.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.761+0000 7f4079219640 1 --2- 172.21.15.164:0/2048514596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4074100e20 0x7f4074199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.761+0000 7f4079219640 1 -- 172.21.15.164:0/2048514596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4074102570 msgr2=0x7f40741998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:35.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.762+0000 7f4079219640 1 --2- 172.21.15.164:0/2048514596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4074102570 0x7f40741998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.762+0000 7f4079219640 1 -- 172.21.15.164:0/2048514596 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40741a4930 con 0x7f4074102ef0 2024-09-19T08:50:35.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.762+0000 7f4079219640 1 --2- 172.21.15.164:0/2048514596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4074102ef0 0x7f40741a43f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f406c02ae70 tx=0x7f406c02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:35.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.762+0000 7f4071ffb640 1 -- 172.21.15.164:0/2048514596 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406c034040 con 0x7f4074102ef0 2024-09-19T08:50:35.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.762+0000 7f407ac7c640 1 -- 172.21.15.164:0/2048514596 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40741a4b00 con 0x7f4074102ef0 2024-09-19T08:50:35.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.762+0000 7f4071ffb640 1 -- 172.21.15.164:0/2048514596 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f406c030920 con 0x7f4074102ef0 2024-09-19T08:50:35.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.762+0000 7f4071ffb640 1 -- 172.21.15.164:0/2048514596 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406c045480 con 0x7f4074102ef0 2024-09-19T08:50:35.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.762+0000 7f407ac7c640 1 -- 172.21.15.164:0/2048514596 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40741a4fb0 con 0x7f4074102ef0 2024-09-19T08:50:35.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.764+0000 7f4071ffb640 1 -- 172.21.15.164:0/2048514596 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f406c037020 con 0x7f4074102ef0 2024-09-19T08:50:35.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.765+0000 7f407d055740 1 Processor -- start 2024-09-19T08:50:35.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.765+0000 7f407d055740 1 -- start start 2024-09-19T08:50:35.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.765+0000 7f407d055740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556805671b40 0x556805670090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:35.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.765+0000 7f407d055740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5568056705d0 0x556805670a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:35.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.765+0000 7f407d055740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680567b370 0x55680567d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:35.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.765+0000 7f407d055740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5568055d62f0 con 0x55680567b370 2024-09-19T08:50:35.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.765+0000 7f407d055740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55680557dcf0 con 0x556805671b40 2024-09-19T08:50:35.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.765+0000 7f4071ffb640 1 --2- 172.21.15.164:0/2048514596 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f40640a33b0 0x7f40640a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:35.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.765+0000 7f407d055740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5568055d22b0 con 0x5568056705d0 2024-09-19T08:50:35.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.765+0000 7f4073fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680567b370 0x55680567d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:35.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.765+0000 7f4078a18640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5568056705d0 0x556805670a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:35.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.765+0000 7f4079219640 1 --2- 172.21.15.164:0/2048514596 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f40640a33b0 0x7f40640a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:35.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.765+0000 7f4073fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680567b370 0x55680567d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60196/0 (socket says 172.21.15.164:60196) 2024-09-19T08:50:35.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.765+0000 7f4073fff640 1 -- 172.21.15.164:0/2757255630 learned_addr learned my addr 172.21.15.164:0/2757255630 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:35.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.765+0000 7f4073fff640 1 --2- 172.21.15.164:0/2757255630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556805671b40 0x556805670090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:35.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.765+0000 7f4071ffb640 1 -- 172.21.15.164:0/2048514596 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f406c0f09b0 con 0x7f4074102ef0 2024-09-19T08:50:35.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.765+0000 7f4073fff640 1 -- 172.21.15.164:0/2757255630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5568056705d0 msgr2=0x556805670a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:35.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.765+0000 7f4073fff640 1 --2- 172.21.15.164:0/2757255630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5568056705d0 0x556805670a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.765+0000 7f4073fff640 1 -- 172.21.15.164:0/2757255630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556805671b40 msgr2=0x556805670090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:35.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.765+0000 7f4073fff640 1 --2- 172.21.15.164:0/2757255630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556805671b40 0x556805670090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.765+0000 7f4073fff640 1 -- 172.21.15.164:0/2757255630 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568055623a0 con 0x55680567b370 2024-09-19T08:50:35.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.765+0000 7f4078a18640 1 --2- 172.21.15.164:0/2757255630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5568056705d0 0x556805670a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:50:35.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.766+0000 7f4073fff640 1 --2- 172.21.15.164:0/2757255630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680567b370 0x55680567d760 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4060012720 tx=0x7f406000b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f328eb1e3b6f96 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:35.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.766+0000 7f4079219640 1 --2- 172.21.15.164:0/2048514596 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f40640a33b0 0x7f40640a5870 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f406c0b9270 tx=0x7f406c0b90b0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:35.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.766+0000 7f405effd640 1 -- 172.21.15.164:0/2757255630 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4060012ca0 con 0x55680567b370 2024-09-19T08:50:35.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.766+0000 7f405effd640 1 -- 172.21.15.164:0/2757255630 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4060012e40 con 0x55680567b370 2024-09-19T08:50:35.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.766+0000 7f405effd640 1 -- 172.21.15.164:0/2757255630 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4060004a00 con 0x55680567b370 2024-09-19T08:50:35.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.766+0000 7f407d055740 1 -- 172.21.15.164:0/2757255630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680567b370 msgr2=0x55680567d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:35.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.766+0000 7f407d055740 1 --2- 172.21.15.164:0/2757255630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680567b370 0x55680567d760 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4060012720 tx=0x7f406000b6e0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.766+0000 7f407d055740 1 -- 172.21.15.164:0/2757255630 shutdown_connections 2024-09-19T08:50:35.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.766+0000 7f407d055740 1 --2- 172.21.15.164:0/2757255630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680567b370 0x55680567d760 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.766+0000 7f407d055740 1 --2- 172.21.15.164:0/2757255630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5568056705d0 0x556805670a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.766+0000 7f407d055740 1 --2- 172.21.15.164:0/2757255630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556805671b40 0x556805670090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.766+0000 7f407d055740 1 -- 172.21.15.164:0/2757255630 >> 172.21.15.164:0/2757255630 conn(0x556805639310 msgr2=0x55680566cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:35.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.767+0000 7f407d055740 1 -- 172.21.15.164:0/2757255630 shutdown_connections 2024-09-19T08:50:35.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.767+0000 7f407d055740 1 -- 172.21.15.164:0/2757255630 wait complete. 2024-09-19T08:50:35.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.767+0000 7f407d055740 1 -- start start 2024-09-19T08:50:35.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.767+0000 7f407d055740 1 Processor -- start 2024-09-19T08:50:35.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.767+0000 7f407d055740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568056705d0 0x55680567ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:35.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.767+0000 7f407d055740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805671b40 0x556805677fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:35.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.767+0000 7f407d055740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680567b370 0x556805678570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:35.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.767+0000 7f407d055740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55680567e070 con 0x55680567b370 2024-09-19T08:50:35.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.767+0000 7f407d055740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556805591060 con 0x5568056705d0 2024-09-19T08:50:35.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.767+0000 7f407d055740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55680567e1f0 con 0x556805671b40 2024-09-19T08:50:35.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.768+0000 7f407d055740 10 client.0 _ll_register_callbacks cb 0x5568052e48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:50:35.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.768+0000 7f4073fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680567b370 0x556805678570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:35.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.768+0000 7f4073fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680567b370 0x556805678570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60200/0 (socket says 172.21.15.164:60200) 2024-09-19T08:50:35.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.768+0000 7f4073fff640 1 -- 172.21.15.164:0/2091242347 learned_addr learned my addr 172.21.15.164:0/2091242347 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:35.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.768+0000 7f4078a18640 1 --2- 172.21.15.164:0/2091242347 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805671b40 0x556805677fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:35.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.768+0000 7f4073fff640 1 --2- 172.21.15.164:0/2091242347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568056705d0 0x55680567ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:35.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.768+0000 7f4073fff640 1 -- 172.21.15.164:0/2091242347 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805671b40 msgr2=0x556805677fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:35.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.768+0000 7f4073fff640 1 --2- 172.21.15.164:0/2091242347 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805671b40 0x556805677fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.768+0000 7f4073fff640 1 -- 172.21.15.164:0/2091242347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568056705d0 msgr2=0x55680567ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:35.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.768+0000 7f4073fff640 1 --2- 172.21.15.164:0/2091242347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568056705d0 0x55680567ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.768+0000 7f4073fff640 1 -- 172.21.15.164:0/2091242347 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568053473c0 con 0x55680567b370 2024-09-19T08:50:35.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.768+0000 7f4073fff640 1 --2- 172.21.15.164:0/2091242347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680567b370 0x556805678570 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f40600054f0 tx=0x7f4060021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a515f8f9f6e6767 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:35.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.768+0000 7f4053fff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:50:35.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.768+0000 7f4053fff640 1 -- 172.21.15.164:0/2091242347 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4060004460 con 0x55680567b370 2024-09-19T08:50:35.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.768+0000 7f4053fff640 1 -- 172.21.15.164:0/2091242347 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f4060010740 con 0x55680567b370 2024-09-19T08:50:35.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.768+0000 7f407d055740 1 -- 172.21.15.164:0/2091242347 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5568055e3860 con 0x55680567b370 2024-09-19T08:50:35.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.768+0000 7f4053fff640 1 -- 172.21.15.164:0/2091242347 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4060032050 con 0x55680567b370 2024-09-19T08:50:35.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.768+0000 7f407d055740 10 client.16431 fetch_fsmap learned FSMap version 30 2024-09-19T08:50:35.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.768+0000 7f407d055740 1 -- 172.21.15.164:0/2091242347 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556805689b20 con 0x55680567b370 2024-09-19T08:50:35.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.769+0000 7f4053fff640 1 -- 172.21.15.164:0/2091242347 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5568055e3860 con 0x55680567b370 2024-09-19T08:50:35.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.769+0000 7f4053fff640 1 -- 172.21.15.164:0/2091242347 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f4060004bc0 con 0x55680567b370 2024-09-19T08:50:35.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.769+0000 7f407d055740 10 client.16431 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:50:35.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.769+0000 7f407d055740 10 client.16431 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T08:50:35.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.769+0000 7f407d055740 20 client.16431 populate_metadata read hostname 'smithi164' 2024-09-19T08:50:35.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.769+0000 7f407d055740 1 --2- 172.21.15.164:0/2091242347 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55680568ae70 0x55680568d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:35.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.769+0000 7f407d055740 4 client.16431 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:50:35.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.769+0000 7f407d055740 1 -- 172.21.15.164:0/2091242347 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568055ddd70 con 0x55680568ae70 2024-09-19T08:50:35.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.769+0000 7f4078a18640 1 --2- 172.21.15.164:0/2091242347 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55680568ae70 0x55680568d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:35.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.770+0000 7f4078a18640 1 --2- 172.21.15.164:0/2091242347 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55680568ae70 0x55680568d260 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=97959f8661e45769 server_cookie=71fdbef84c141a81 in_seq=0 out_seq=0 2024-09-19T08:50:35.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.770+0000 7f4053fff640 10 client.16431 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:50:35.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.770+0000 7f4053fff640 1 -- 172.21.15.164:0/2091242347 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5568055ddd70 con 0x55680568ae70 2024-09-19T08:50:35.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.770+0000 7f4053fff640 10 client.16431 handle_command_reply: tid=0 2024-09-19T08:50:35.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.770+0000 7f407d055740 1 client.16431 shutdown 2024-09-19T08:50:35.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.770+0000 7f407d055740 1 -- 172.21.15.164:0/2091242347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680567b370 msgr2=0x556805678570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:35.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.770+0000 7f407d055740 1 --2- 172.21.15.164:0/2091242347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680567b370 0x556805678570 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f40600054f0 tx=0x7f4060021040 comp rx=0 tx=0).stop 2024-09-19T08:50:35.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.770+0000 7f407d055740 1 -- 172.21.15.164:0/2091242347 shutdown_connections 2024-09-19T08:50:35.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.770+0000 7f407d055740 1 --2- 172.21.15.164:0/2091242347 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55680568ae70 0x55680568d260 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.770+0000 7f407d055740 1 --2- 172.21.15.164:0/2091242347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680567b370 0x556805678570 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.770+0000 7f407d055740 1 --2- 172.21.15.164:0/2091242347 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805671b40 0x556805677fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.770+0000 7f4053fff640 0 client.16431 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:50:35.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.770+0000 7f407d055740 1 --2- 172.21.15.164:0/2091242347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568056705d0 0x55680567ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.770+0000 7f407d055740 1 -- 172.21.15.164:0/2091242347 >> 172.21.15.164:0/2091242347 conn(0x556805639310 msgr2=0x556805672960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:35.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.771+0000 7f407d055740 1 -- 172.21.15.164:0/2091242347 shutdown_connections 2024-09-19T08:50:35.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.771+0000 7f407d055740 1 -- 172.21.15.164:0/2091242347 wait complete. 2024-09-19T08:50:35.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.771+0000 7f407d055740 20 client.16431 trim_cache size 0 max 16384 2024-09-19T08:50:35.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.783+0000 7f407d055740 1 Processor -- start 2024-09-19T08:50:35.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.783+0000 7f407d055740 1 -- start start 2024-09-19T08:50:35.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.784+0000 7f407d055740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5568056705d0 0x556805687370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:35.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.784+0000 7f407d055740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556805671b40 0x55680566cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:35.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.784+0000 7f407d055740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55680567b370 0x5568056838c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:35.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.784+0000 7f407d055740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f40741ad650 con 0x556805671b40 2024-09-19T08:50:35.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.784+0000 7f407d055740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f40741af250 con 0x55680567b370 2024-09-19T08:50:35.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.784+0000 7f407d055740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f40741add20 con 0x5568056705d0 2024-09-19T08:50:35.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.784+0000 7f4078a18640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556805671b40 0x55680566cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:35.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.784+0000 7f4078a18640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556805671b40 0x55680566cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60206/0 (socket says 172.21.15.164:60206) 2024-09-19T08:50:35.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.784+0000 7f4078a18640 1 -- 172.21.15.164:0/2946839892 learned_addr learned my addr 172.21.15.164:0/2946839892 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:35.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.784+0000 7f4078a18640 1 -- 172.21.15.164:0/2946839892 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5568056705d0 msgr2=0x556805687370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:35.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.784+0000 7f4078a18640 1 --2- 172.21.15.164:0/2946839892 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5568056705d0 0x556805687370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.784+0000 7f4078a18640 1 -- 172.21.15.164:0/2946839892 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55680567b370 msgr2=0x5568056838c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:35.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.784+0000 7f4078a18640 1 --2- 172.21.15.164:0/2946839892 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55680567b370 0x5568056838c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.784+0000 7f4078a18640 1 -- 172.21.15.164:0/2946839892 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55680568d840 con 0x556805671b40 2024-09-19T08:50:35.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.784+0000 7f4078a18640 1 --2- 172.21.15.164:0/2946839892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556805671b40 0x55680566cc80 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4068005fd0 tx=0x7f406800b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2d00cad66d7ea1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:35.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.784+0000 7f405e7fc640 1 -- 172.21.15.164:0/2946839892 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4068004220 con 0x556805671b40 2024-09-19T08:50:35.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.784+0000 7f405e7fc640 1 -- 172.21.15.164:0/2946839892 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40680043c0 con 0x556805671b40 2024-09-19T08:50:35.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.784+0000 7f405e7fc640 1 -- 172.21.15.164:0/2946839892 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40680035f0 con 0x556805671b40 2024-09-19T08:50:35.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.785+0000 7f407d055740 1 -- 172.21.15.164:0/2946839892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556805671b40 msgr2=0x55680566cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:35.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.785+0000 7f407d055740 1 --2- 172.21.15.164:0/2946839892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556805671b40 0x55680566cc80 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4068005fd0 tx=0x7f406800b410 comp rx=0 tx=0).stop 2024-09-19T08:50:35.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.785+0000 7f407d055740 1 -- 172.21.15.164:0/2946839892 shutdown_connections 2024-09-19T08:50:35.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.785+0000 7f407d055740 1 --2- 172.21.15.164:0/2946839892 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55680567b370 0x5568056838c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.785+0000 7f407d055740 1 --2- 172.21.15.164:0/2946839892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556805671b40 0x55680566cc80 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.785+0000 7f407d055740 1 --2- 172.21.15.164:0/2946839892 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5568056705d0 0x556805687370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.785+0000 7f407d055740 1 -- 172.21.15.164:0/2946839892 >> 172.21.15.164:0/2946839892 conn(0x556805639310 msgr2=0x5568056770c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:35.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.785+0000 7f407d055740 1 -- 172.21.15.164:0/2946839892 shutdown_connections 2024-09-19T08:50:35.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.785+0000 7f407d055740 1 -- 172.21.15.164:0/2946839892 wait complete. 2024-09-19T08:50:35.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.785+0000 7f407d055740 1 -- start start 2024-09-19T08:50:35.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.785+0000 7f407d055740 1 Processor -- start 2024-09-19T08:50:35.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.786+0000 7f407d055740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568056705d0 0x5568056927e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:35.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.786+0000 7f407d055740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805671b40 0x556805692d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:35.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.786+0000 7f407d055740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680567b370 0x5568056f0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:35.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.786+0000 7f407d055740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f40741aeac0 con 0x55680567b370 2024-09-19T08:50:35.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.786+0000 7f407d055740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f40741ae3f0 con 0x5568056705d0 2024-09-19T08:50:35.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.786+0000 7f407d055740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f40741ab630 con 0x556805671b40 2024-09-19T08:50:35.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.786+0000 7f407d055740 10 client.0 _ll_register_callbacks cb 0x7f406000bdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:50:35.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.786+0000 7f4073fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680567b370 0x5568056f0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:35.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.786+0000 7f4073fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680567b370 0x5568056f0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60218/0 (socket says 172.21.15.164:60218) 2024-09-19T08:50:35.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.786+0000 7f4073fff640 1 -- 172.21.15.164:0/1637670318 learned_addr learned my addr 172.21.15.164:0/1637670318 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:35.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.786+0000 7f4073fff640 1 --2- 172.21.15.164:0/1637670318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568056705d0 0x5568056927e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:35.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.786+0000 7f4078a18640 1 --2- 172.21.15.164:0/1637670318 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805671b40 0x556805692d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:35.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.786+0000 7f4073fff640 1 -- 172.21.15.164:0/1637670318 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805671b40 msgr2=0x556805692d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:35.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.786+0000 7f4073fff640 1 --2- 172.21.15.164:0/1637670318 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805671b40 0x556805692d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.786+0000 7f4073fff640 1 -- 172.21.15.164:0/1637670318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568056705d0 msgr2=0x5568056927e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:35.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.786+0000 7f4073fff640 1 --2- 172.21.15.164:0/1637670318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568056705d0 0x5568056927e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.786+0000 7f4073fff640 1 -- 172.21.15.164:0/1637670318 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568056f1580 con 0x55680567b370 2024-09-19T08:50:35.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.786+0000 7f4073fff640 1 --2- 172.21.15.164:0/1637670318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680567b370 0x5568056f0ee0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f40600039f0 tx=0x7f4060017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c56763b6f800503 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:35.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.786+0000 7f4053fff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:50:35.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.787+0000 7f4053fff640 1 -- 172.21.15.164:0/1637670318 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40600120e0 con 0x55680567b370 2024-09-19T08:50:35.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.787+0000 7f4053fff640 1 -- 172.21.15.164:0/1637670318 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f4060029310 con 0x55680567b370 2024-09-19T08:50:35.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.787+0000 7f407d055740 1 -- 172.21.15.164:0/1637670318 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40680043c0 con 0x55680567b370 2024-09-19T08:50:35.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.787+0000 7f4053fff640 1 -- 172.21.15.164:0/1637670318 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f406000b7b0 con 0x55680567b370 2024-09-19T08:50:35.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.787+0000 7f407d055740 10 client.16437 fetch_fsmap learned FSMap version 30 2024-09-19T08:50:35.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.787+0000 7f407d055740 1 -- 172.21.15.164:0/1637670318 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5568056f9b10 con 0x55680567b370 2024-09-19T08:50:35.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.787+0000 7f4053fff640 1 -- 172.21.15.164:0/1637670318 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40680043c0 con 0x55680567b370 2024-09-19T08:50:35.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.787+0000 7f4053fff640 1 -- 172.21.15.164:0/1637670318 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f40600108d0 con 0x55680567b370 2024-09-19T08:50:35.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.787+0000 7f407d055740 10 client.16437 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:50:35.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.787+0000 7f407d055740 10 client.16437 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T08:50:35.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.787+0000 7f407d055740 20 client.16437 populate_metadata read hostname 'smithi164' 2024-09-19T08:50:35.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.787+0000 7f407d055740 1 --2- 172.21.15.164:0/1637670318 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x5568056fa170 0x5568056fc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:35.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.787+0000 7f407d055740 4 client.16437 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:50:35.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.787+0000 7f407d055740 1 -- 172.21.15.164:0/1637670318 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55680555a170 con 0x5568056fa170 2024-09-19T08:50:35.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.788+0000 7f4078a18640 1 --2- 172.21.15.164:0/1637670318 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x5568056fa170 0x5568056fc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:35.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.788+0000 7f4053fff640 10 client.16437 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:50:35.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.788+0000 7f4078a18640 1 --2- 172.21.15.164:0/1637670318 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x5568056fa170 0x5568056fc560 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cfa6977445abcaaa server_cookie=d7f668ae84cefbf4 in_seq=0 out_seq=0 2024-09-19T08:50:35.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.788+0000 7f4053fff640 1 -- 172.21.15.164:0/1637670318 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55680555a170 con 0x5568056fa170 2024-09-19T08:50:35.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.788+0000 7f4053fff640 10 client.16437 handle_command_reply: tid=0 2024-09-19T08:50:35.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.788+0000 7f407d055740 1 client.16437 shutdown 2024-09-19T08:50:35.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.788+0000 7f407d055740 1 -- 172.21.15.164:0/1637670318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680567b370 msgr2=0x5568056f0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:35.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.788+0000 7f407d055740 1 --2- 172.21.15.164:0/1637670318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680567b370 0x5568056f0ee0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f40600039f0 tx=0x7f4060017040 comp rx=0 tx=0).stop 2024-09-19T08:50:35.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.788+0000 7f407d055740 1 -- 172.21.15.164:0/1637670318 shutdown_connections 2024-09-19T08:50:35.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.788+0000 7f407d055740 1 --2- 172.21.15.164:0/1637670318 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x5568056fa170 0x5568056fc560 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.788+0000 7f407d055740 1 --2- 172.21.15.164:0/1637670318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55680567b370 0x5568056f0ee0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.788+0000 7f407d055740 1 --2- 172.21.15.164:0/1637670318 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805671b40 0x556805692d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.788+0000 7f407d055740 1 --2- 172.21.15.164:0/1637670318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568056705d0 0x5568056927e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.788+0000 7f407d055740 1 -- 172.21.15.164:0/1637670318 >> 172.21.15.164:0/1637670318 conn(0x556805639310 msgr2=0x55680566b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:35.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.788+0000 7f4053fff640 0 client.16437 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:50:35.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.788+0000 7f407d055740 1 -- 172.21.15.164:0/1637670318 shutdown_connections 2024-09-19T08:50:35.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.788+0000 7f407d055740 1 -- 172.21.15.164:0/1637670318 wait complete. 2024-09-19T08:50:35.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.789+0000 7f407d055740 20 client.16437 trim_cache size 0 max 16384 2024-09-19T08:50:35.803 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:50:35.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.789+0000 7f407ac7c640 1 -- 172.21.15.164:0/2048514596 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f40640a33b0 msgr2=0x7f40640a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:35.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.789+0000 7f407ac7c640 1 --2- 172.21.15.164:0/2048514596 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f40640a33b0 0x7f40640a5870 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f406c0b9270 tx=0x7f406c0b90b0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.789+0000 7f407ac7c640 1 -- 172.21.15.164:0/2048514596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4074102ef0 msgr2=0x7f40741a43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:35.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.789+0000 7f407ac7c640 1 --2- 172.21.15.164:0/2048514596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4074102ef0 0x7f40741a43f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f406c02ae70 tx=0x7f406c02f5d0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.789+0000 7f407ac7c640 1 -- 172.21.15.164:0/2048514596 shutdown_connections 2024-09-19T08:50:35.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.789+0000 7f407ac7c640 1 --2- 172.21.15.164:0/2048514596 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f40640a33b0 0x7f40640a5870 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.789+0000 7f407ac7c640 1 --2- 172.21.15.164:0/2048514596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4074102ef0 0x7f40741a43f0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.789+0000 7f407ac7c640 1 --2- 172.21.15.164:0/2048514596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4074102570 0x7f40741998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.789+0000 7f407ac7c640 1 --2- 172.21.15.164:0/2048514596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4074100e20 0x7f4074199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:35.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.789+0000 7f407ac7c640 1 -- 172.21.15.164:0/2048514596 >> 172.21.15.164:0/2048514596 conn(0x7f40740fc7d0 msgr2=0x7f4074106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:35.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.789+0000 7f407ac7c640 1 -- 172.21.15.164:0/2048514596 shutdown_connections 2024-09-19T08:50:35.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:35.789+0000 7f407ac7c640 1 -- 172.21.15.164:0/2048514596 wait complete. 2024-09-19T08:50:35.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:35 smithi170 ceph-mon[35072]: pgmap v146: 97 pgs: 97 active+clean; 40 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 697 MiB/s wr, 1.67k op/s 2024-09-19T08:50:35.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:35 smithi173 ceph-mon[35306]: pgmap v146: 97 pgs: 97 active+clean; 40 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 697 MiB/s wr, 1.67k op/s 2024-09-19T08:50:36.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:35 smithi164 ceph-mon[30928]: pgmap v146: 97 pgs: 97 active+clean; 40 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 697 MiB/s wr, 1.67k op/s 2024-09-19T08:50:36.804 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T08:50:37.050 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:50:37.541 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:50:37.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.501+0000 7fdd1d579640 1 -- 172.21.15.164:0/1851080045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd18100e20 msgr2=0x7fdd18101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:37.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.501+0000 7fdd1d579640 1 --2- 172.21.15.164:0/1851080045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd18100e20 0x7fdd18101200 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fdd08009900 tx=0x7fdd0802a480 comp rx=0 tx=0).stop 2024-09-19T08:50:37.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.501+0000 7fdd1d579640 1 -- 172.21.15.164:0/1851080045 shutdown_connections 2024-09-19T08:50:37.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.501+0000 7fdd1d579640 1 --2- 172.21.15.164:0/1851080045 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd18102ef0 0x7fdd1810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:37.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.501+0000 7fdd1d579640 1 --2- 172.21.15.164:0/1851080045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd18102570 0x7fdd181029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:37.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.501+0000 7fdd1d579640 1 --2- 172.21.15.164:0/1851080045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd18100e20 0x7fdd18101200 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:37.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.501+0000 7fdd1d579640 1 -- 172.21.15.164:0/1851080045 >> 172.21.15.164:0/1851080045 conn(0x7fdd180fc7d0 msgr2=0x7fdd180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:37.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.501+0000 7fdd1d579640 1 -- 172.21.15.164:0/1851080045 shutdown_connections 2024-09-19T08:50:37.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.501+0000 7fdd1d579640 1 -- 172.21.15.164:0/1851080045 wait complete. 2024-09-19T08:50:37.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.501+0000 7fdd1d579640 1 Processor -- start 2024-09-19T08:50:37.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.501+0000 7fdd1d579640 1 -- start start 2024-09-19T08:50:37.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.502+0000 7fdd1d579640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd18100e20 0x7fdd18198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:37.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.502+0000 7fdd1d579640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd18102570 0x7fdd18198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:37.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.502+0000 7fdd1d579640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd18102ef0 0x7fdd1819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:37.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.502+0000 7fdd1d579640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdd18110fa0 con 0x7fdd18102ef0 2024-09-19T08:50:37.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.502+0000 7fdd1d579640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdd18110e20 con 0x7fdd18100e20 2024-09-19T08:50:37.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.502+0000 7fdd1d579640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdd18111120 con 0x7fdd18102570 2024-09-19T08:50:37.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.502+0000 7fdd177fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd18102ef0 0x7fdd1819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:37.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.502+0000 7fdd177fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd18102ef0 0x7fdd1819cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60240/0 (socket says 172.21.15.164:60240) 2024-09-19T08:50:37.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.502+0000 7fdd177fe640 1 -- 172.21.15.164:0/2646874731 learned_addr learned my addr 172.21.15.164:0/2646874731 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:37.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.502+0000 7fdd16ffd640 1 --2- 172.21.15.164:0/2646874731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd18100e20 0x7fdd18198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:37.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.502+0000 7fdd0ffff640 1 --2- 172.21.15.164:0/2646874731 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd18102570 0x7fdd18198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:37.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.502+0000 7fdd177fe640 1 -- 172.21.15.164:0/2646874731 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd18102570 msgr2=0x7fdd18198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:37.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.502+0000 7fdd177fe640 1 --2- 172.21.15.164:0/2646874731 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd18102570 0x7fdd18198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:37.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.502+0000 7fdd177fe640 1 -- 172.21.15.164:0/2646874731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd18100e20 msgr2=0x7fdd18198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:37.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.502+0000 7fdd177fe640 1 --2- 172.21.15.164:0/2646874731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd18100e20 0x7fdd18198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:37.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.502+0000 7fdd177fe640 1 -- 172.21.15.164:0/2646874731 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd1819d4c0 con 0x7fdd18102ef0 2024-09-19T08:50:37.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.502+0000 7fdd16ffd640 1 --2- 172.21.15.164:0/2646874731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd18100e20 0x7fdd18198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:50:37.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.502+0000 7fdd177fe640 1 --2- 172.21.15.164:0/2646874731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd18102ef0 0x7fdd1819cde0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000b970 tx=0x7fdd1000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:37.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.503+0000 7fdd14ff9640 1 -- 172.21.15.164:0/2646874731 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd10019070 con 0x7fdd18102ef0 2024-09-19T08:50:37.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.503+0000 7fdd1d579640 1 -- 172.21.15.164:0/2646874731 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd1819d7b0 con 0x7fdd18102ef0 2024-09-19T08:50:37.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.503+0000 7fdd14ff9640 1 -- 172.21.15.164:0/2646874731 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd10014440 con 0x7fdd18102ef0 2024-09-19T08:50:37.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.503+0000 7fdd14ff9640 1 -- 172.21.15.164:0/2646874731 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd100075d0 con 0x7fdd18102ef0 2024-09-19T08:50:37.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.503+0000 7fdd1d579640 1 -- 172.21.15.164:0/2646874731 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd181a5090 con 0x7fdd18102ef0 2024-09-19T08:50:37.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.505+0000 7fdd14ff9640 1 -- 172.21.15.164:0/2646874731 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fdd1001c070 con 0x7fdd18102ef0 2024-09-19T08:50:37.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.505+0000 7fdd1f952740 1 Processor -- start 2024-09-19T08:50:37.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.505+0000 7fdd1f952740 1 -- start start 2024-09-19T08:50:37.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.506+0000 7fdd1f952740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557934647b40 0x557934646090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:37.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.506+0000 7fdd1f952740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579346465d0 0x557934646a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:37.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.506+0000 7fdd1f952740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557934651370 0x557934653760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:37.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.506+0000 7fdd1f952740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5579345ac2f0 con 0x557934647b40 2024-09-19T08:50:37.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.506+0000 7fdd1f952740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x557934553cf0 con 0x5579346465d0 2024-09-19T08:50:37.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.506+0000 7fdd1f952740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5579345a82b0 con 0x557934651370 2024-09-19T08:50:37.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.506+0000 7fdd14ff9640 1 --2- 172.21.15.164:0/2646874731 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fdd040a33b0 0x7fdd040a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:37.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.506+0000 7fdd0ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557934647b40 0x557934646090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:37.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.506+0000 7fdd0ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557934647b40 0x557934646090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60250/0 (socket says 172.21.15.164:60250) 2024-09-19T08:50:37.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.506+0000 7fdd0ffff640 1 -- 172.21.15.164:0/1631163090 learned_addr learned my addr 172.21.15.164:0/1631163090 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:37.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.506+0000 7fdd16ffd640 1 --2- 172.21.15.164:0/1631163090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579346465d0 0x557934646a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:37.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.506+0000 7fdd0ffff640 1 --2- 172.21.15.164:0/1631163090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557934651370 0x557934653760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:37.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.506+0000 7fdd14ff9640 1 -- 172.21.15.164:0/2646874731 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fdd100cf0a0 con 0x7fdd18102ef0 2024-09-19T08:50:37.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.506+0000 7fdd0ffff640 1 -- 172.21.15.164:0/1631163090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557934651370 msgr2=0x557934653760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:37.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.506+0000 7fdd0ffff640 1 --2- 172.21.15.164:0/1631163090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557934651370 0x557934653760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:37.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.506+0000 7fdd0ffff640 1 -- 172.21.15.164:0/1631163090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579346465d0 msgr2=0x557934646a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:37.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.506+0000 7fdd0ffff640 1 --2- 172.21.15.164:0/1631163090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579346465d0 0x557934646a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:37.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.506+0000 7fdd177fe640 1 --2- 172.21.15.164:0/2646874731 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fdd040a33b0 0x7fdd040a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:37.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.506+0000 7fdd0ffff640 1 -- 172.21.15.164:0/1631163090 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579345383a0 con 0x557934647b40 2024-09-19T08:50:37.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.507+0000 7fdd0ffff640 1 --2- 172.21.15.164:0/1631163090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557934647b40 0x557934646090 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fdd00013560 tx=0x7fdd0000a800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9083ebcfc7cff9e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:37.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.507+0000 7fdd0d7fa640 1 -- 172.21.15.164:0/1631163090 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd00019030 con 0x557934647b40 2024-09-19T08:50:37.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.507+0000 7fdd0d7fa640 1 -- 172.21.15.164:0/1631163090 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd000110d0 con 0x557934647b40 2024-09-19T08:50:37.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.507+0000 7fdd0d7fa640 1 -- 172.21.15.164:0/1631163090 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0000f8e0 con 0x557934647b40 2024-09-19T08:50:37.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.507+0000 7fdd177fe640 1 --2- 172.21.15.164:0/2646874731 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fdd040a33b0 0x7fdd040a5870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdd100ccac0 tx=0x7fdd1000c510 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:37.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.508+0000 7fdd1f952740 1 -- 172.21.15.164:0/1631163090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557934647b40 msgr2=0x557934646090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:37.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.508+0000 7fdd1f952740 1 --2- 172.21.15.164:0/1631163090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557934647b40 0x557934646090 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fdd00013560 tx=0x7fdd0000a800 comp rx=0 tx=0).stop 2024-09-19T08:50:37.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.508+0000 7fdd1f952740 1 -- 172.21.15.164:0/1631163090 shutdown_connections 2024-09-19T08:50:37.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.508+0000 7fdd1f952740 1 --2- 172.21.15.164:0/1631163090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557934651370 0x557934653760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:37.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.508+0000 7fdd1f952740 1 --2- 172.21.15.164:0/1631163090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579346465d0 0x557934646a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:37.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.508+0000 7fdd1f952740 1 --2- 172.21.15.164:0/1631163090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557934647b40 0x557934646090 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:37.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.508+0000 7fdd1f952740 1 -- 172.21.15.164:0/1631163090 >> 172.21.15.164:0/1631163090 conn(0x55793460f310 msgr2=0x557934642c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:37.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.508+0000 7fdd1f952740 1 -- 172.21.15.164:0/1631163090 shutdown_connections 2024-09-19T08:50:37.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.508+0000 7fdd1f952740 1 -- 172.21.15.164:0/1631163090 wait complete. 2024-09-19T08:50:37.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.508+0000 7fdd1f952740 1 -- start start 2024-09-19T08:50:37.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.508+0000 7fdd1f952740 1 Processor -- start 2024-09-19T08:50:37.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.508+0000 7fdd1f952740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5579346465d0 0x557934650d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:37.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.508+0000 7fdd1f952740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557934647b40 0x55793464dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:37.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.508+0000 7fdd1f952740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557934651370 0x55793464e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:37.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.508+0000 7fdd1f952740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557934654070 con 0x557934651370 2024-09-19T08:50:37.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.508+0000 7fdd1f952740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x557934567060 con 0x557934647b40 2024-09-19T08:50:37.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.508+0000 7fdd1f952740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5579346541f0 con 0x5579346465d0 2024-09-19T08:50:37.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.509+0000 7fdd1f952740 10 client.0 _ll_register_callbacks cb 0x5579342ba8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:50:37.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.509+0000 7fdd0ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557934651370 0x55793464e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:37.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.509+0000 7fdd0ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557934651370 0x55793464e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60262/0 (socket says 172.21.15.164:60262) 2024-09-19T08:50:37.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.509+0000 7fdd0ffff640 1 -- 172.21.15.164:0/4091557687 learned_addr learned my addr 172.21.15.164:0/4091557687 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:37.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.509+0000 7fdd0ffff640 1 --2- 172.21.15.164:0/4091557687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5579346465d0 0x557934650d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:37.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.509+0000 7fdd16ffd640 1 --2- 172.21.15.164:0/4091557687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557934647b40 0x55793464dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:37.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.509+0000 7fdd0ffff640 1 -- 172.21.15.164:0/4091557687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5579346465d0 msgr2=0x557934650d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:37.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.509+0000 7fdd0ffff640 1 --2- 172.21.15.164:0/4091557687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5579346465d0 0x557934650d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:37.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.509+0000 7fdd0ffff640 1 -- 172.21.15.164:0/4091557687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557934647b40 msgr2=0x55793464dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:37.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.509+0000 7fdd0ffff640 1 --2- 172.21.15.164:0/4091557687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557934647b40 0x55793464dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:37.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.509+0000 7fdd0ffff640 1 -- 172.21.15.164:0/4091557687 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55793431d3c0 con 0x557934651370 2024-09-19T08:50:37.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.509+0000 7fdd0ffff640 1 --2- 172.21.15.164:0/4091557687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557934651370 0x55793464e5d0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fdd0000f5c0 tx=0x7fdd00015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c03f1495dd231e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:37.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.509+0000 7fdcfe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:50:37.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.509+0000 7fdcfe7fc640 1 -- 172.21.15.164:0/4091557687 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd00019070 con 0x557934651370 2024-09-19T08:50:37.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.509+0000 7fdcfe7fc640 1 -- 172.21.15.164:0/4091557687 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fdd0000f660 con 0x557934651370 2024-09-19T08:50:37.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.509+0000 7fdd1f952740 1 -- 172.21.15.164:0/4091557687 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd000110d0 con 0x557934651370 2024-09-19T08:50:37.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.510+0000 7fdcfe7fc640 1 -- 172.21.15.164:0/4091557687 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdd0000f860 con 0x557934651370 2024-09-19T08:50:37.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.510+0000 7fdd1f952740 10 client.16449 fetch_fsmap learned FSMap version 30 2024-09-19T08:50:37.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.510+0000 7fdd1f952740 1 -- 172.21.15.164:0/4091557687 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55793465fad0 con 0x557934651370 2024-09-19T08:50:37.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.510+0000 7fdcfe7fc640 1 -- 172.21.15.164:0/4091557687 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd000110d0 con 0x557934651370 2024-09-19T08:50:37.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.510+0000 7fdcfe7fc640 1 -- 172.21.15.164:0/4091557687 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fdd0001e7c0 con 0x557934651370 2024-09-19T08:50:37.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.510+0000 7fdd1f952740 10 client.16449 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:50:37.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.510+0000 7fdd1f952740 10 client.16449 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T08:50:37.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.510+0000 7fdd1f952740 20 client.16449 populate_metadata read hostname 'smithi164' 2024-09-19T08:50:37.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.510+0000 7fdd1f952740 1 --2- 172.21.15.164:0/4091557687 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x557934660e80 0x557934663270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:37.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.510+0000 7fdd1f952740 4 client.16449 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:50:37.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.510+0000 7fdd1f952740 1 -- 172.21.15.164:0/4091557687 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579345b9860 con 0x557934660e80 2024-09-19T08:50:37.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.510+0000 7fdd16ffd640 1 --2- 172.21.15.164:0/4091557687 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x557934660e80 0x557934663270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:37.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.511+0000 7fdd16ffd640 1 --2- 172.21.15.164:0/4091557687 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x557934660e80 0x557934663270 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4567cba82d39245a server_cookie=c957f382350bcbf in_seq=0 out_seq=0 2024-09-19T08:50:37.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.511+0000 7fdcfe7fc640 10 client.16449 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:50:37.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.511+0000 7fdcfe7fc640 1 -- 172.21.15.164:0/4091557687 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5579345b9860 con 0x557934660e80 2024-09-19T08:50:37.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.511+0000 7fdcfe7fc640 10 client.16449 handle_command_reply: tid=0 2024-09-19T08:50:37.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.512+0000 7fdd1f952740 1 client.16449 shutdown 2024-09-19T08:50:37.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.512+0000 7fdd1f952740 1 -- 172.21.15.164:0/4091557687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557934651370 msgr2=0x55793464e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:37.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.512+0000 7fdd1f952740 1 --2- 172.21.15.164:0/4091557687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557934651370 0x55793464e5d0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fdd0000f5c0 tx=0x7fdd00015040 comp rx=0 tx=0).stop 2024-09-19T08:50:37.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.512+0000 7fdd1f952740 1 -- 172.21.15.164:0/4091557687 shutdown_connections 2024-09-19T08:50:37.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.512+0000 7fdd1f952740 1 --2- 172.21.15.164:0/4091557687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557934651370 0x55793464e5d0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:37.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.512+0000 7fdd1f952740 1 --2- 172.21.15.164:0/4091557687 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x557934660e80 0x557934663270 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:37.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.512+0000 7fdd1f952740 1 --2- 172.21.15.164:0/4091557687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557934647b40 0x55793464dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:37.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.512+0000 7fdd1f952740 1 --2- 172.21.15.164:0/4091557687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5579346465d0 0x557934650d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:37.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.512+0000 7fdcfe7fc640 0 client.16449 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:50:37.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.512+0000 7fdd1f952740 1 -- 172.21.15.164:0/4091557687 >> 172.21.15.164:0/4091557687 conn(0x55793460f310 msgr2=0x557934648960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:37.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.512+0000 7fdd1f952740 1 -- 172.21.15.164:0/4091557687 shutdown_connections 2024-09-19T08:50:37.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.512+0000 7fdd1f952740 1 -- 172.21.15.164:0/4091557687 wait complete. 2024-09-19T08:50:37.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.512+0000 7fdd1f952740 20 client.16449 trim_cache size 0 max 16384 2024-09-19T08:50:37.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.525+0000 7fdd1f952740 1 Processor -- start 2024-09-19T08:50:37.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.525+0000 7fdd1f952740 1 -- start start 2024-09-19T08:50:37.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.525+0000 7fdd1f952740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579346465d0 0x55793464d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:37.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.525+0000 7fdd1f952740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557934647b40 0x557934649120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:37.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.525+0000 7fdd1f952740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557934651370 0x557934649660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:37.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.525+0000 7fdd1f952740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdd181ad710 con 0x5579346465d0 2024-09-19T08:50:37.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.525+0000 7fdd1f952740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdd181af310 con 0x557934651370 2024-09-19T08:50:37.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.525+0000 7fdd1f952740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdd181adde0 con 0x557934647b40 2024-09-19T08:50:37.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.525+0000 7fdd0ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579346465d0 0x55793464d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:37.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.525+0000 7fdd0ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579346465d0 0x55793464d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60270/0 (socket says 172.21.15.164:60270) 2024-09-19T08:50:37.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.525+0000 7fdd0ffff640 1 -- 172.21.15.164:0/639812803 learned_addr learned my addr 172.21.15.164:0/639812803 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:37.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.525+0000 7fdd0ffff640 1 -- 172.21.15.164:0/639812803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557934647b40 msgr2=0x557934649120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:37.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.525+0000 7fdd16ffd640 1 --2- 172.21.15.164:0/639812803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557934647b40 0x557934649120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:37.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.525+0000 7fdd0ffff640 1 --2- 172.21.15.164:0/639812803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557934647b40 0x557934649120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:37.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.525+0000 7fdd0ffff640 1 -- 172.21.15.164:0/639812803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557934651370 msgr2=0x557934649660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:37.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.525+0000 7fdd0ffff640 1 --2- 172.21.15.164:0/639812803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557934651370 0x557934649660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:37.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.525+0000 7fdd0ffff640 1 -- 172.21.15.164:0/639812803 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557934643960 con 0x5579346465d0 2024-09-19T08:50:37.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.525+0000 7fdd0ffff640 1 --2- 172.21.15.164:0/639812803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579346465d0 0x55793464d790 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fdd0001eb90 tx=0x7fdd00011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8611a4c3cc0cd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:37.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.526+0000 7fdd0cff9640 1 -- 172.21.15.164:0/639812803 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd00019070 con 0x5579346465d0 2024-09-19T08:50:37.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.526+0000 7fdd0cff9640 1 -- 172.21.15.164:0/639812803 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd0000f5f0 con 0x5579346465d0 2024-09-19T08:50:37.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.526+0000 7fdd0cff9640 1 -- 172.21.15.164:0/639812803 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0000fa00 con 0x5579346465d0 2024-09-19T08:50:37.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.526+0000 7fdd1f952740 1 -- 172.21.15.164:0/639812803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579346465d0 msgr2=0x55793464d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:37.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.526+0000 7fdd1f952740 1 --2- 172.21.15.164:0/639812803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579346465d0 0x55793464d790 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fdd0001eb90 tx=0x7fdd00011a60 comp rx=0 tx=0).stop 2024-09-19T08:50:37.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.526+0000 7fdd1f952740 1 -- 172.21.15.164:0/639812803 shutdown_connections 2024-09-19T08:50:37.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.526+0000 7fdd1f952740 1 --2- 172.21.15.164:0/639812803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557934651370 0x557934649660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:37.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.526+0000 7fdd1f952740 1 --2- 172.21.15.164:0/639812803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557934647b40 0x557934649120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:37.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.526+0000 7fdd1f952740 1 --2- 172.21.15.164:0/639812803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579346465d0 0x55793464d790 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:37.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.526+0000 7fdd1f952740 1 -- 172.21.15.164:0/639812803 >> 172.21.15.164:0/639812803 conn(0x55793460f310 msgr2=0x557934650060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:37.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.527+0000 7fdd1f952740 1 -- 172.21.15.164:0/639812803 shutdown_connections 2024-09-19T08:50:37.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.527+0000 7fdd1f952740 1 -- 172.21.15.164:0/639812803 wait complete. 2024-09-19T08:50:37.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.527+0000 7fdd1f952740 1 -- start start 2024-09-19T08:50:37.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.527+0000 7fdd1f952740 1 Processor -- start 2024-09-19T08:50:37.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.527+0000 7fdd1f952740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579346465d0 0x557934654bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:37.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.527+0000 7fdd1f952740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557934647b40 0x557934655100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:37.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.527+0000 7fdd1f952740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557934651370 0x5579346c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:37.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.527+0000 7fdd1f952740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdd181aeb80 con 0x5579346465d0 2024-09-19T08:50:37.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.527+0000 7fdd1f952740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdd181ae4b0 con 0x557934647b40 2024-09-19T08:50:37.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.527+0000 7fdd1f952740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdd181ab730 con 0x557934651370 2024-09-19T08:50:37.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.527+0000 7fdd1f952740 10 client.0 _ll_register_callbacks cb 0x7fdd08005dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:50:37.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.528+0000 7fdd0ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579346465d0 0x557934654bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:37.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.528+0000 7fdd0ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579346465d0 0x557934654bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60274/0 (socket says 172.21.15.164:60274) 2024-09-19T08:50:37.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.528+0000 7fdd0ffff640 1 -- 172.21.15.164:0/713937681 learned_addr learned my addr 172.21.15.164:0/713937681 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:37.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.528+0000 7fdd16ffd640 1 --2- 172.21.15.164:0/713937681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557934647b40 0x557934655100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:37.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.528+0000 7fdd0ffff640 1 -- 172.21.15.164:0/713937681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557934651370 msgr2=0x5579346c7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:37.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.528+0000 7fdd0ffff640 1 --2- 172.21.15.164:0/713937681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557934651370 0x5579346c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:37.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.528+0000 7fdd0ffff640 1 -- 172.21.15.164:0/713937681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557934647b40 msgr2=0x557934655100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:37.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.528+0000 7fdd0ffff640 1 --2- 172.21.15.164:0/713937681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557934647b40 0x557934655100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:37.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.528+0000 7fdd0ffff640 1 -- 172.21.15.164:0/713937681 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579346c8340 con 0x5579346465d0 2024-09-19T08:50:37.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.528+0000 7fdd0ffff640 1 --2- 172.21.15.164:0/713937681 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579346465d0 0x557934654bc0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fdd0001ed80 tx=0x7fdd0000a940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf598ea6241f775 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:37.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.528+0000 7fdcfe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:50:37.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.528+0000 7fdcfe7fc640 1 -- 172.21.15.164:0/713937681 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0003d060 con 0x5579346465d0 2024-09-19T08:50:37.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.528+0000 7fdcfe7fc640 1 -- 172.21.15.164:0/713937681 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fdd000110d0 con 0x5579346465d0 2024-09-19T08:50:37.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.528+0000 7fdd1f952740 1 -- 172.21.15.164:0/713937681 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd0000f5f0 con 0x5579346465d0 2024-09-19T08:50:37.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.529+0000 7fdcfe7fc640 1 -- 172.21.15.164:0/713937681 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdd00019070 con 0x5579346465d0 2024-09-19T08:50:37.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.529+0000 7fdd1f952740 10 client.16455 fetch_fsmap learned FSMap version 30 2024-09-19T08:50:37.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.529+0000 7fdd1f952740 1 -- 172.21.15.164:0/713937681 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5579346cf6b0 con 0x5579346465d0 2024-09-19T08:50:37.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.529+0000 7fdcfe7fc640 1 -- 172.21.15.164:0/713937681 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd0000f5f0 con 0x5579346465d0 2024-09-19T08:50:37.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.529+0000 7fdcfe7fc640 1 -- 172.21.15.164:0/713937681 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fdd00040030 con 0x5579346465d0 2024-09-19T08:50:37.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.529+0000 7fdd1f952740 10 client.16455 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:50:37.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.529+0000 7fdd1f952740 10 client.16455 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T08:50:37.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.529+0000 7fdd1f952740 20 client.16455 populate_metadata read hostname 'smithi164' 2024-09-19T08:50:37.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.529+0000 7fdd1f952740 1 --2- 172.21.15.164:0/713937681 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x5579346cfcc0 0x5579346d20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:37.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.529+0000 7fdd1f952740 4 client.16455 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:50:37.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.529+0000 7fdd1f952740 1 -- 172.21.15.164:0/713937681 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5579345b3d70 con 0x5579346cfcc0 2024-09-19T08:50:37.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.529+0000 7fdd16ffd640 1 --2- 172.21.15.164:0/713937681 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x5579346cfcc0 0x5579346d20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:37.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.530+0000 7fdd16ffd640 1 --2- 172.21.15.164:0/713937681 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x5579346cfcc0 0x5579346d20b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bd54fa9a36d04f8f server_cookie=87187b04940932eb in_seq=0 out_seq=0 2024-09-19T08:50:37.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.530+0000 7fdcfe7fc640 10 client.16455 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:50:37.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.530+0000 7fdcfe7fc640 1 -- 172.21.15.164:0/713937681 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5579345b3d70 con 0x5579346cfcc0 2024-09-19T08:50:37.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.530+0000 7fdcfe7fc640 10 client.16455 handle_command_reply: tid=0 2024-09-19T08:50:37.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.530+0000 7fdd1f952740 1 client.16455 shutdown 2024-09-19T08:50:37.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.530+0000 7fdd1f952740 1 -- 172.21.15.164:0/713937681 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579346465d0 msgr2=0x557934654bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:37.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.530+0000 7fdd1f952740 1 --2- 172.21.15.164:0/713937681 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579346465d0 0x557934654bc0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fdd0001ed80 tx=0x7fdd0000a940 comp rx=0 tx=0).stop 2024-09-19T08:50:37.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.530+0000 7fdd1f952740 1 -- 172.21.15.164:0/713937681 shutdown_connections 2024-09-19T08:50:37.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.530+0000 7fdd1f952740 1 --2- 172.21.15.164:0/713937681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557934651370 0x5579346c7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:37.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.530+0000 7fdd1f952740 1 --2- 172.21.15.164:0/713937681 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x5579346cfcc0 0x5579346d20b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:37.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.530+0000 7fdd1f952740 1 --2- 172.21.15.164:0/713937681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557934647b40 0x557934655100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:37.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.530+0000 7fdd1f952740 1 --2- 172.21.15.164:0/713937681 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579346465d0 0x557934654bc0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:37.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.530+0000 7fdd1f952740 1 -- 172.21.15.164:0/713937681 >> 172.21.15.164:0/713937681 conn(0x55793460f310 msgr2=0x5579346414e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:37.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.530+0000 7fdcfe7fc640 0 client.16455 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:50:37.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.530+0000 7fdd1f952740 1 -- 172.21.15.164:0/713937681 shutdown_connections 2024-09-19T08:50:37.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.530+0000 7fdd1f952740 1 -- 172.21.15.164:0/713937681 wait complete. 2024-09-19T08:50:37.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.530+0000 7fdd1f952740 20 client.16455 trim_cache size 0 max 16384 2024-09-19T08:50:37.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.531+0000 7fdd1d579640 1 -- 172.21.15.164:0/2646874731 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fdd040a33b0 msgr2=0x7fdd040a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:37.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.531+0000 7fdd1d579640 1 --2- 172.21.15.164:0/2646874731 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fdd040a33b0 0x7fdd040a5870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdd100ccac0 tx=0x7fdd1000c510 comp rx=0 tx=0).stop 2024-09-19T08:50:37.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.531+0000 7fdd1d579640 1 -- 172.21.15.164:0/2646874731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd18102ef0 msgr2=0x7fdd1819cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:37.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.531+0000 7fdd1d579640 1 --2- 172.21.15.164:0/2646874731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd18102ef0 0x7fdd1819cde0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000b970 tx=0x7fdd1000bd70 comp rx=0 tx=0).stop 2024-09-19T08:50:37.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.531+0000 7fdd1d579640 1 -- 172.21.15.164:0/2646874731 shutdown_connections 2024-09-19T08:50:37.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.531+0000 7fdd1d579640 1 --2- 172.21.15.164:0/2646874731 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fdd040a33b0 0x7fdd040a5870 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:37.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.531+0000 7fdd1d579640 1 --2- 172.21.15.164:0/2646874731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd18102ef0 0x7fdd1819cde0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:37.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.531+0000 7fdd1d579640 1 --2- 172.21.15.164:0/2646874731 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd18102570 0x7fdd18198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:37.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.531+0000 7fdd1d579640 1 --2- 172.21.15.164:0/2646874731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd18100e20 0x7fdd18198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:37.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.531+0000 7fdd1d579640 1 -- 172.21.15.164:0/2646874731 >> 172.21.15.164:0/2646874731 conn(0x7fdd180fc7d0 msgr2=0x7fdd1806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:37.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.531+0000 7fdd1d579640 1 -- 172.21.15.164:0/2646874731 shutdown_connections 2024-09-19T08:50:37.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:37.531+0000 7fdd1d579640 1 -- 172.21.15.164:0/2646874731 wait complete. 2024-09-19T08:50:37.874 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:37 smithi170 ceph-mon[35072]: pgmap v147: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 656 MiB/s wr, 1.52k op/s 2024-09-19T08:50:37.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:37 smithi173 ceph-mon[35306]: pgmap v147: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 656 MiB/s wr, 1.52k op/s 2024-09-19T08:50:38.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:37 smithi164 ceph-mon[30928]: pgmap v147: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 656 MiB/s wr, 1.52k op/s 2024-09-19T08:50:38.449 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T08:50:38.699 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:50:39.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.166+0000 7f52ba44e640 1 -- 172.21.15.164:0/1435952420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52b4100e20 msgr2=0x7f52b4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:39.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.166+0000 7f52ba44e640 1 --2- 172.21.15.164:0/1435952420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52b4100e20 0x7f52b4101200 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f52a4009900 tx=0x7f52a402a480 comp rx=0 tx=0).stop 2024-09-19T08:50:39.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.166+0000 7f52ba44e640 1 -- 172.21.15.164:0/1435952420 shutdown_connections 2024-09-19T08:50:39.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.166+0000 7f52ba44e640 1 --2- 172.21.15.164:0/1435952420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52b4102ef0 0x7f52b410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:39.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.166+0000 7f52ba44e640 1 --2- 172.21.15.164:0/1435952420 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52b4102570 0x7f52b41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:39.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.166+0000 7f52ba44e640 1 --2- 172.21.15.164:0/1435952420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52b4100e20 0x7f52b4101200 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:39.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.166+0000 7f52ba44e640 1 -- 172.21.15.164:0/1435952420 >> 172.21.15.164:0/1435952420 conn(0x7f52b40fc7d0 msgr2=0x7f52b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:39.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.166+0000 7f52ba44e640 1 -- 172.21.15.164:0/1435952420 shutdown_connections 2024-09-19T08:50:39.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.166+0000 7f52ba44e640 1 -- 172.21.15.164:0/1435952420 wait complete. 2024-09-19T08:50:39.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.167+0000 7f52ba44e640 1 Processor -- start 2024-09-19T08:50:39.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.167+0000 7f52ba44e640 1 -- start start 2024-09-19T08:50:39.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.167+0000 7f52ba44e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52b4100e20 0x7f52b4103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:39.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.167+0000 7f52ba44e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52b4102570 0x7f52b4103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:39.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.167+0000 7f52ba44e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52b4102ef0 0x7f52b41041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:39.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.167+0000 7f52ba44e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f52b4110f80 con 0x7f52b4102570 2024-09-19T08:50:39.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.167+0000 7f52ba44e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f52b4110e00 con 0x7f52b4102ef0 2024-09-19T08:50:39.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.167+0000 7f52ba44e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f52b4111100 con 0x7f52b4100e20 2024-09-19T08:50:39.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.167+0000 7f52abfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52b4102570 0x7f52b4103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:39.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.167+0000 7f52abfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52b4102570 0x7f52b4103c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60282/0 (socket says 172.21.15.164:60282) 2024-09-19T08:50:39.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.167+0000 7f52abfff640 1 -- 172.21.15.164:0/3611209411 learned_addr learned my addr 172.21.15.164:0/3611209411 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:39.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.167+0000 7f52b3fff640 1 --2- 172.21.15.164:0/3611209411 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52b4100e20 0x7f52b4103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:39.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.167+0000 7f52b89eb640 1 --2- 172.21.15.164:0/3611209411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52b4102ef0 0x7f52b41041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:39.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.167+0000 7f52abfff640 1 -- 172.21.15.164:0/3611209411 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52b4100e20 msgr2=0x7f52b4103750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:39.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.167+0000 7f52abfff640 1 --2- 172.21.15.164:0/3611209411 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52b4100e20 0x7f52b4103750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:39.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.167+0000 7f52abfff640 1 -- 172.21.15.164:0/3611209411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52b4102ef0 msgr2=0x7f52b41041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:39.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.167+0000 7f52abfff640 1 --2- 172.21.15.164:0/3611209411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52b4102ef0 0x7f52b41041d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:39.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.167+0000 7f52abfff640 1 -- 172.21.15.164:0/3611209411 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b41a6a10 con 0x7f52b4102570 2024-09-19T08:50:39.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.168+0000 7f52abfff640 1 --2- 172.21.15.164:0/3611209411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52b4102570 0x7f52b4103c90 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f529c00d950 tx=0x7f529c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:39.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.168+0000 7f52b1ffb640 1 -- 172.21.15.164:0/3611209411 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c017070 con 0x7f52b4102570 2024-09-19T08:50:39.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.168+0000 7f52b1ffb640 1 -- 172.21.15.164:0/3611209411 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f529c009c30 con 0x7f52b4102570 2024-09-19T08:50:39.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.168+0000 7f52ba44e640 1 -- 172.21.15.164:0/3611209411 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b41a6d00 con 0x7f52b4102570 2024-09-19T08:50:39.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.168+0000 7f52b1ffb640 1 -- 172.21.15.164:0/3611209411 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c00f6b0 con 0x7f52b4102570 2024-09-19T08:50:39.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.168+0000 7f52ba44e640 1 -- 172.21.15.164:0/3611209411 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b41a7230 con 0x7f52b4102570 2024-09-19T08:50:39.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.170+0000 7f52b1ffb640 1 -- 172.21.15.164:0/3611209411 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f529c0109d0 con 0x7f52b4102570 2024-09-19T08:50:39.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.170+0000 7f52bc827740 1 Processor -- start 2024-09-19T08:50:39.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.170+0000 7f52bc827740 1 -- start start 2024-09-19T08:50:39.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.171+0000 7f52b1ffb640 1 --2- 172.21.15.164:0/3611209411 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f52a00a3430 0x7f52a00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:39.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.171+0000 7f52bc827740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5612a2f4eb40 0x5612a2f4d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:39.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.171+0000 7f52bc827740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612a2f4d5d0 0x5612a2f4da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:39.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.171+0000 7f52bc827740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612a2f58370 0x5612a2f5a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:39.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.171+0000 7f52bc827740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5612a2eb32f0 con 0x5612a2f4d5d0 2024-09-19T08:50:39.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.171+0000 7f52bc827740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5612a2e5acf0 con 0x5612a2f58370 2024-09-19T08:50:39.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.171+0000 7f52bc827740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5612a2eaf2b0 con 0x5612a2f4eb40 2024-09-19T08:50:39.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.171+0000 7f52b89eb640 1 --2- 172.21.15.164:0/3611209411 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f52a00a3430 0x7f52a00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:39.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.171+0000 7f52abfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612a2f4d5d0 0x5612a2f4da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:39.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.171+0000 7f52abfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612a2f4d5d0 0x5612a2f4da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60286/0 (socket says 172.21.15.164:60286) 2024-09-19T08:50:39.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.171+0000 7f52abfff640 1 -- 172.21.15.164:0/2183202203 learned_addr learned my addr 172.21.15.164:0/2183202203 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:39.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.171+0000 7f52b3fff640 1 --2- 172.21.15.164:0/2183202203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5612a2f4eb40 0x5612a2f4d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:39.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.171+0000 7f52b1ffb640 1 -- 172.21.15.164:0/3611209411 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f529c0cf950 con 0x7f52b4102570 2024-09-19T08:50:39.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.171+0000 7f52abfff640 1 -- 172.21.15.164:0/2183202203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5612a2f4eb40 msgr2=0x5612a2f4d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:39.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.171+0000 7f52abfff640 1 --2- 172.21.15.164:0/2183202203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5612a2f4eb40 0x5612a2f4d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:39.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.171+0000 7f52abfff640 1 -- 172.21.15.164:0/2183202203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612a2f58370 msgr2=0x5612a2f5a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:39.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.171+0000 7f52abfff640 1 --2- 172.21.15.164:0/2183202203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612a2f58370 0x5612a2f5a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:39.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.171+0000 7f52abfff640 1 -- 172.21.15.164:0/2183202203 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612a2e3f3a0 con 0x5612a2f4d5d0 2024-09-19T08:50:39.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.171+0000 7f52b89eb640 1 --2- 172.21.15.164:0/3611209411 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f52a00a3430 0x7f52a00a58f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f52ac005cc0 tx=0x7f52ac00a710 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:39.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.171+0000 7f52abfff640 1 --2- 172.21.15.164:0/2183202203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612a2f4d5d0 0x5612a2f4da10 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f529c0cd020 tx=0x7f529c00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bfa28ef994303ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:39.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.171+0000 7f52aa7fc640 1 -- 172.21.15.164:0/2183202203 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c020070 con 0x5612a2f4d5d0 2024-09-19T08:50:39.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.171+0000 7f52aa7fc640 1 -- 172.21.15.164:0/2183202203 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f529c09c340 con 0x5612a2f4d5d0 2024-09-19T08:50:39.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.172+0000 7f52aa7fc640 1 -- 172.21.15.164:0/2183202203 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c0a48b0 con 0x5612a2f4d5d0 2024-09-19T08:50:39.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.172+0000 7f52bc827740 1 -- 172.21.15.164:0/2183202203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612a2f4d5d0 msgr2=0x5612a2f4da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:39.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.172+0000 7f52bc827740 1 --2- 172.21.15.164:0/2183202203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612a2f4d5d0 0x5612a2f4da10 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f529c0cd020 tx=0x7f529c00bf60 comp rx=0 tx=0).stop 2024-09-19T08:50:39.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.172+0000 7f52bc827740 1 -- 172.21.15.164:0/2183202203 shutdown_connections 2024-09-19T08:50:39.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.172+0000 7f52bc827740 1 --2- 172.21.15.164:0/2183202203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612a2f58370 0x5612a2f5a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:39.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.172+0000 7f52bc827740 1 --2- 172.21.15.164:0/2183202203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612a2f4d5d0 0x5612a2f4da10 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:39.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.172+0000 7f52bc827740 1 --2- 172.21.15.164:0/2183202203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5612a2f4eb40 0x5612a2f4d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:39.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.172+0000 7f52bc827740 1 -- 172.21.15.164:0/2183202203 >> 172.21.15.164:0/2183202203 conn(0x5612a2f16310 msgr2=0x5612a2f49c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:39.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.172+0000 7f52bc827740 1 -- 172.21.15.164:0/2183202203 shutdown_connections 2024-09-19T08:50:39.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.172+0000 7f52bc827740 1 -- 172.21.15.164:0/2183202203 wait complete. 2024-09-19T08:50:39.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.173+0000 7f52bc827740 1 -- start start 2024-09-19T08:50:39.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.173+0000 7f52bc827740 1 Processor -- start 2024-09-19T08:50:39.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.173+0000 7f52bc827740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612a2f4d5d0 0x5612a2f57d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:39.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.173+0000 7f52bc827740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612a2f4eb40 0x5612a2f55000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:39.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.173+0000 7f52bc827740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5612a2f58370 0x5612a2f555d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:39.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.173+0000 7f52bc827740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5612a2f5b070 con 0x5612a2f4eb40 2024-09-19T08:50:39.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.173+0000 7f52bc827740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5612a2e6e060 con 0x5612a2f4d5d0 2024-09-19T08:50:39.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.173+0000 7f52bc827740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5612a2f5b1f0 con 0x5612a2f58370 2024-09-19T08:50:39.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.173+0000 7f52bc827740 10 client.0 _ll_register_callbacks cb 0x5612a2bc18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:50:39.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.173+0000 7f52abfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612a2f4eb40 0x5612a2f55000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:39.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.173+0000 7f52abfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612a2f4eb40 0x5612a2f55000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60298/0 (socket says 172.21.15.164:60298) 2024-09-19T08:50:39.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.173+0000 7f52abfff640 1 -- 172.21.15.164:0/886831941 learned_addr learned my addr 172.21.15.164:0/886831941 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:39.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.173+0000 7f52b3fff640 1 --2- 172.21.15.164:0/886831941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612a2f4d5d0 0x5612a2f57d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:39.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.173+0000 7f52b89eb640 1 --2- 172.21.15.164:0/886831941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5612a2f58370 0x5612a2f555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:39.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.174+0000 7f52abfff640 1 -- 172.21.15.164:0/886831941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5612a2f58370 msgr2=0x5612a2f555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:39.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.174+0000 7f52abfff640 1 --2- 172.21.15.164:0/886831941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5612a2f58370 0x5612a2f555d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:39.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.174+0000 7f52abfff640 1 -- 172.21.15.164:0/886831941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612a2f4d5d0 msgr2=0x5612a2f57d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:39.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.174+0000 7f52abfff640 1 --2- 172.21.15.164:0/886831941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612a2f4d5d0 0x5612a2f57d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:39.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.174+0000 7f52abfff640 1 -- 172.21.15.164:0/886831941 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612a2c243c0 con 0x5612a2f4eb40 2024-09-19T08:50:39.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.174+0000 7f52abfff640 1 --2- 172.21.15.164:0/886831941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612a2f4eb40 0x5612a2f55000 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f529c009610 tx=0x7f529c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c100160ed56c021 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:39.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.174+0000 7f52937fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:50:39.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.174+0000 7f52937fe640 1 -- 172.21.15.164:0/886831941 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c020050 con 0x5612a2f4eb40 2024-09-19T08:50:39.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.174+0000 7f52937fe640 1 -- 172.21.15.164:0/886831941 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f529c0ad3b0 con 0x5612a2f4eb40 2024-09-19T08:50:39.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.174+0000 7f52bc827740 1 -- 172.21.15.164:0/886831941 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f529c09c340 con 0x5612a2f4eb40 2024-09-19T08:50:39.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.174+0000 7f52937fe640 1 -- 172.21.15.164:0/886831941 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f529c0ae740 con 0x5612a2f4eb40 2024-09-19T08:50:39.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.174+0000 7f52bc827740 10 client.16467 fetch_fsmap learned FSMap version 30 2024-09-19T08:50:39.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.174+0000 7f52bc827740 1 -- 172.21.15.164:0/886831941 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5612a2f66b20 con 0x5612a2f4eb40 2024-09-19T08:50:39.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.174+0000 7f52937fe640 1 -- 172.21.15.164:0/886831941 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f529c09c340 con 0x5612a2f4eb40 2024-09-19T08:50:39.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.174+0000 7f52937fe640 1 -- 172.21.15.164:0/886831941 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f529c0cd3c0 con 0x5612a2f4eb40 2024-09-19T08:50:39.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.174+0000 7f52bc827740 10 client.16467 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:50:39.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.174+0000 7f52bc827740 10 client.16467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:50:39.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.174+0000 7f52bc827740 20 client.16467 populate_metadata read hostname 'smithi164' 2024-09-19T08:50:39.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.175+0000 7f52bc827740 1 --2- 172.21.15.164:0/886831941 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5612a2f67e70 0x5612a2f6a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:39.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.175+0000 7f52bc827740 4 client.16467 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:50:39.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.175+0000 7f52bc827740 1 -- 172.21.15.164:0/886831941 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612a2ec0860 con 0x5612a2f67e70 2024-09-19T08:50:39.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.175+0000 7f52b3fff640 1 --2- 172.21.15.164:0/886831941 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5612a2f67e70 0x5612a2f6a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:39.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.175+0000 7f52b3fff640 1 --2- 172.21.15.164:0/886831941 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5612a2f67e70 0x5612a2f6a260 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e4dc1260c51eff6 server_cookie=d68aae6efb5ca5a4 in_seq=0 out_seq=0 2024-09-19T08:50:39.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.175+0000 7f52937fe640 10 client.16467 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:50:39.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.176+0000 7f52937fe640 1 -- 172.21.15.164:0/886831941 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5612a2ec0860 con 0x5612a2f67e70 2024-09-19T08:50:39.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.176+0000 7f52937fe640 10 client.16467 handle_command_reply: tid=0 2024-09-19T08:50:39.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.176+0000 7f52bc827740 1 client.16467 shutdown 2024-09-19T08:50:39.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.176+0000 7f52bc827740 1 -- 172.21.15.164:0/886831941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612a2f4eb40 msgr2=0x5612a2f55000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:39.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.176+0000 7f52bc827740 1 --2- 172.21.15.164:0/886831941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612a2f4eb40 0x5612a2f55000 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f529c009610 tx=0x7f529c005880 comp rx=0 tx=0).stop 2024-09-19T08:50:39.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.176+0000 7f52bc827740 1 -- 172.21.15.164:0/886831941 shutdown_connections 2024-09-19T08:50:39.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.176+0000 7f52bc827740 1 --2- 172.21.15.164:0/886831941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5612a2f58370 0x5612a2f555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:39.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.176+0000 7f52bc827740 1 --2- 172.21.15.164:0/886831941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612a2f4eb40 0x5612a2f55000 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:39.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.176+0000 7f52bc827740 1 --2- 172.21.15.164:0/886831941 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5612a2f67e70 0x5612a2f6a260 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:39.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.176+0000 7f52937fe640 0 client.16467 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:50:39.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.176+0000 7f52bc827740 1 --2- 172.21.15.164:0/886831941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612a2f4d5d0 0x5612a2f57d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:39.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.176+0000 7f52bc827740 1 -- 172.21.15.164:0/886831941 >> 172.21.15.164:0/886831941 conn(0x5612a2f16310 msgr2=0x5612a2f4f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:39.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.176+0000 7f52bc827740 1 -- 172.21.15.164:0/886831941 shutdown_connections 2024-09-19T08:50:39.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.176+0000 7f52bc827740 1 -- 172.21.15.164:0/886831941 wait complete. 2024-09-19T08:50:39.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.177+0000 7f52bc827740 20 client.16467 trim_cache size 0 max 16384 2024-09-19T08:50:39.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.190+0000 7f52bc827740 1 Processor -- start 2024-09-19T08:50:39.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.190+0000 7f52bc827740 1 -- start start 2024-09-19T08:50:39.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.190+0000 7f52bc827740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612a2f4d5d0 0x5612a2f56f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:39.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.190+0000 7f52bc827740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612a2f4eb40 0x5612a2f6f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:39.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.190+0000 7f52bc827740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5612a2f58370 0x5612a2f6fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:39.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.190+0000 7f52bc827740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f52b41afa20 con 0x5612a2f4eb40 2024-09-19T08:50:39.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.190+0000 7f52bc827740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f52b41b1620 con 0x5612a2f4d5d0 2024-09-19T08:50:39.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.190+0000 7f52bc827740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f52b41b00f0 con 0x5612a2f58370 2024-09-19T08:50:39.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.190+0000 7f52abfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612a2f4eb40 0x5612a2f6f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:39.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.190+0000 7f52abfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612a2f4eb40 0x5612a2f6f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60304/0 (socket says 172.21.15.164:60304) 2024-09-19T08:50:39.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.190+0000 7f52abfff640 1 -- 172.21.15.164:0/2626188773 learned_addr learned my addr 172.21.15.164:0/2626188773 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:39.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.190+0000 7f52b3fff640 1 --2- 172.21.15.164:0/2626188773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612a2f4d5d0 0x5612a2f56f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:39.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.190+0000 7f52b89eb640 1 --2- 172.21.15.164:0/2626188773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5612a2f58370 0x5612a2f6fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:39.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.190+0000 7f52abfff640 1 -- 172.21.15.164:0/2626188773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5612a2f58370 msgr2=0x5612a2f6fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:39.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.190+0000 7f52abfff640 1 --2- 172.21.15.164:0/2626188773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5612a2f58370 0x5612a2f6fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:39.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.190+0000 7f52abfff640 1 -- 172.21.15.164:0/2626188773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612a2f4d5d0 msgr2=0x5612a2f56f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:39.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.190+0000 7f52abfff640 1 --2- 172.21.15.164:0/2626188773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612a2f4d5d0 0x5612a2f56f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:39.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.190+0000 7f52abfff640 1 -- 172.21.15.164:0/2626188773 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612a2f5b210 con 0x5612a2f4eb40 2024-09-19T08:50:39.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.190+0000 7f52b3fff640 1 --2- 172.21.15.164:0/2626188773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612a2f4d5d0 0x5612a2f56f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:50:39.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.190+0000 7f52abfff640 1 --2- 172.21.15.164:0/2626188773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612a2f4eb40 0x5612a2f6f8f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f529c012040 tx=0x7f529c00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4b7e328f6a68ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:39.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.191+0000 7f52a9ffb640 1 -- 172.21.15.164:0/2626188773 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c09c340 con 0x5612a2f4eb40 2024-09-19T08:50:39.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.191+0000 7f52a9ffb640 1 -- 172.21.15.164:0/2626188773 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f529c009de0 con 0x5612a2f4eb40 2024-09-19T08:50:39.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.191+0000 7f52a9ffb640 1 -- 172.21.15.164:0/2626188773 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c0ae510 con 0x5612a2f4eb40 2024-09-19T08:50:39.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.191+0000 7f52bc827740 1 -- 172.21.15.164:0/2626188773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612a2f4eb40 msgr2=0x5612a2f6f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:39.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.191+0000 7f52bc827740 1 --2- 172.21.15.164:0/2626188773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612a2f4eb40 0x5612a2f6f8f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f529c012040 tx=0x7f529c00bf60 comp rx=0 tx=0).stop 2024-09-19T08:50:39.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.191+0000 7f52bc827740 1 -- 172.21.15.164:0/2626188773 shutdown_connections 2024-09-19T08:50:39.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.191+0000 7f52bc827740 1 --2- 172.21.15.164:0/2626188773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5612a2f58370 0x5612a2f6fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:39.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.191+0000 7f52bc827740 1 --2- 172.21.15.164:0/2626188773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612a2f4eb40 0x5612a2f6f8f0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:39.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.191+0000 7f52bc827740 1 --2- 172.21.15.164:0/2626188773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612a2f4d5d0 0x5612a2f56f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:39.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.191+0000 7f52bc827740 1 -- 172.21.15.164:0/2626188773 >> 172.21.15.164:0/2626188773 conn(0x5612a2f16310 msgr2=0x5612a2f56860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:39.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.191+0000 7f52bc827740 1 -- 172.21.15.164:0/2626188773 shutdown_connections 2024-09-19T08:50:39.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.191+0000 7f52bc827740 1 -- 172.21.15.164:0/2626188773 wait complete. 2024-09-19T08:50:39.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.192+0000 7f52bc827740 1 -- start start 2024-09-19T08:50:39.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.192+0000 7f52bc827740 1 Processor -- start 2024-09-19T08:50:39.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.192+0000 7f52bc827740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5612a2f4d5d0 0x5612a2f604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:39.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.192+0000 7f52bc827740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612a2f4eb40 0x5612a2f609f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:39.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.192+0000 7f52bc827740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612a2f58370 0x5612a2f5d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:39.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.192+0000 7f52bc827740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f52b41b0e90 con 0x5612a2f58370 2024-09-19T08:50:39.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.192+0000 7f52bc827740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f52b41b07c0 con 0x5612a2f4eb40 2024-09-19T08:50:39.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.192+0000 7f52bc827740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f52b41ada00 con 0x5612a2f4d5d0 2024-09-19T08:50:39.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.192+0000 7f52bc827740 10 client.0 _ll_register_callbacks cb 0x7f52a4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:50:39.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.193+0000 7f52b89eb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612a2f58370 0x5612a2f5d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:39.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.193+0000 7f52b89eb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612a2f58370 0x5612a2f5d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60306/0 (socket says 172.21.15.164:60306) 2024-09-19T08:50:39.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.193+0000 7f52b89eb640 1 -- 172.21.15.164:0/1456867345 learned_addr learned my addr 172.21.15.164:0/1456867345 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:39.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.193+0000 7f52b3fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5612a2f4d5d0 0x5612a2f604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:39.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.193+0000 7f52abfff640 1 --2- 172.21.15.164:0/1456867345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612a2f4eb40 0x5612a2f609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:39.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.193+0000 7f52b89eb640 1 -- 172.21.15.164:0/1456867345 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5612a2f4d5d0 msgr2=0x5612a2f604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:39.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.193+0000 7f52b89eb640 1 --2- 172.21.15.164:0/1456867345 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5612a2f4d5d0 0x5612a2f604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:39.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.193+0000 7f52b89eb640 1 -- 172.21.15.164:0/1456867345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612a2f4eb40 msgr2=0x5612a2f609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:39.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.193+0000 7f52b89eb640 1 --2- 172.21.15.164:0/1456867345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612a2f4eb40 0x5612a2f609f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:39.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.193+0000 7f52b89eb640 1 -- 172.21.15.164:0/1456867345 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612a2f50120 con 0x5612a2f58370 2024-09-19T08:50:39.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.193+0000 7f52b3fff640 1 --2- 172.21.15.164:0/1456867345 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5612a2f4d5d0 0x5612a2f604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:50:39.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.193+0000 7f52b89eb640 1 --2- 172.21.15.164:0/1456867345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612a2f58370 0x5612a2f5d480 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f52ac012dd0 tx=0x7f52ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35361e54932abbce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:39.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.193+0000 7f52937fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:50:39.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.193+0000 7f52937fe640 1 -- 172.21.15.164:0/1456867345 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac021410 con 0x5612a2f58370 2024-09-19T08:50:39.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.193+0000 7f52937fe640 1 -- 172.21.15.164:0/1456867345 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f52ac0215b0 con 0x5612a2f58370 2024-09-19T08:50:39.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.193+0000 7f52bc827740 1 -- 172.21.15.164:0/1456867345 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f529c009de0 con 0x5612a2f58370 2024-09-19T08:50:39.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.193+0000 7f52937fe640 1 -- 172.21.15.164:0/1456867345 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f52ac003970 con 0x5612a2f58370 2024-09-19T08:50:39.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.193+0000 7f52bc827740 10 client.16473 fetch_fsmap learned FSMap version 30 2024-09-19T08:50:39.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.193+0000 7f52bc827740 1 -- 172.21.15.164:0/1456867345 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5612a2fd2d70 con 0x5612a2f58370 2024-09-19T08:50:39.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.194+0000 7f52937fe640 1 -- 172.21.15.164:0/1456867345 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f529c009de0 con 0x5612a2f58370 2024-09-19T08:50:39.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.194+0000 7f52937fe640 1 -- 172.21.15.164:0/1456867345 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f52ac031050 con 0x5612a2f58370 2024-09-19T08:50:39.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.194+0000 7f52bc827740 10 client.16473 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:50:39.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.194+0000 7f52bc827740 10 client.16473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:50:39.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.194+0000 7f52bc827740 20 client.16473 populate_metadata read hostname 'smithi164' 2024-09-19T08:50:39.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.194+0000 7f52bc827740 1 --2- 172.21.15.164:0/1456867345 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5612a2f67e70 0x5612a2fd55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:39.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.194+0000 7f52bc827740 4 client.16473 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:50:39.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.194+0000 7f52bc827740 1 -- 172.21.15.164:0/1456867345 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5612a2ebad70 con 0x5612a2f67e70 2024-09-19T08:50:39.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.194+0000 7f52b3fff640 1 --2- 172.21.15.164:0/1456867345 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5612a2f67e70 0x5612a2fd55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:39.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.194+0000 7f52937fe640 10 client.16473 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:50:39.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.194+0000 7f52b3fff640 1 --2- 172.21.15.164:0/1456867345 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5612a2f67e70 0x5612a2fd55d0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acd473b4aca18e25 server_cookie=5fcbaf24eb81f88e in_seq=0 out_seq=0 2024-09-19T08:50:39.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.195+0000 7f52937fe640 1 -- 172.21.15.164:0/1456867345 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5612a2ebad70 con 0x5612a2f67e70 2024-09-19T08:50:39.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.195+0000 7f52937fe640 10 client.16473 handle_command_reply: tid=0 2024-09-19T08:50:39.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.195+0000 7f52bc827740 1 client.16473 shutdown 2024-09-19T08:50:39.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.195+0000 7f52bc827740 1 -- 172.21.15.164:0/1456867345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612a2f58370 msgr2=0x5612a2f5d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:39.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.195+0000 7f52bc827740 1 --2- 172.21.15.164:0/1456867345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612a2f58370 0x5612a2f5d480 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f52ac012dd0 tx=0x7f52ac004a40 comp rx=0 tx=0).stop 2024-09-19T08:50:39.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.195+0000 7f52bc827740 1 -- 172.21.15.164:0/1456867345 shutdown_connections 2024-09-19T08:50:39.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.195+0000 7f52bc827740 1 --2- 172.21.15.164:0/1456867345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612a2f58370 0x5612a2f5d480 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:39.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.195+0000 7f52bc827740 1 --2- 172.21.15.164:0/1456867345 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5612a2f67e70 0x5612a2fd55d0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:39.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.195+0000 7f52bc827740 1 --2- 172.21.15.164:0/1456867345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612a2f4eb40 0x5612a2f609f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:39.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.195+0000 7f52bc827740 1 --2- 172.21.15.164:0/1456867345 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5612a2f4d5d0 0x5612a2f604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:39.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.195+0000 7f52bc827740 1 -- 172.21.15.164:0/1456867345 >> 172.21.15.164:0/1456867345 conn(0x5612a2f16310 msgr2=0x5612a2f57080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:39.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.195+0000 7f52937fe640 0 client.16473 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:50:39.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.195+0000 7f52bc827740 1 -- 172.21.15.164:0/1456867345 shutdown_connections 2024-09-19T08:50:39.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.195+0000 7f52bc827740 1 -- 172.21.15.164:0/1456867345 wait complete. 2024-09-19T08:50:39.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.195+0000 7f52bc827740 20 client.16473 trim_cache size 0 max 16384 2024-09-19T08:50:39.206 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:50:39.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.196+0000 7f52ba44e640 1 -- 172.21.15.164:0/3611209411 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f52a00a3430 msgr2=0x7f52a00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:39.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.196+0000 7f52ba44e640 1 --2- 172.21.15.164:0/3611209411 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f52a00a3430 0x7f52a00a58f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f52ac005cc0 tx=0x7f52ac00a710 comp rx=0 tx=0).stop 2024-09-19T08:50:39.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.196+0000 7f52ba44e640 1 -- 172.21.15.164:0/3611209411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52b4102570 msgr2=0x7f52b4103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:39.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.196+0000 7f52ba44e640 1 --2- 172.21.15.164:0/3611209411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52b4102570 0x7f52b4103c90 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f529c00d950 tx=0x7f529c00de10 comp rx=0 tx=0).stop 2024-09-19T08:50:39.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.196+0000 7f52ba44e640 1 -- 172.21.15.164:0/3611209411 shutdown_connections 2024-09-19T08:50:39.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.196+0000 7f52ba44e640 1 --2- 172.21.15.164:0/3611209411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52b4102ef0 0x7f52b41041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:39.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.196+0000 7f52ba44e640 1 --2- 172.21.15.164:0/3611209411 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f52a00a3430 0x7f52a00a58f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:39.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.196+0000 7f52ba44e640 1 --2- 172.21.15.164:0/3611209411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52b4102570 0x7f52b4103c90 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:39.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.196+0000 7f52ba44e640 1 --2- 172.21.15.164:0/3611209411 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52b4100e20 0x7f52b4103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:39.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.196+0000 7f52ba44e640 1 -- 172.21.15.164:0/3611209411 >> 172.21.15.164:0/3611209411 conn(0x7f52b40fc7d0 msgr2=0x7f52b406a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:39.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.196+0000 7f52ba44e640 1 -- 172.21.15.164:0/3611209411 shutdown_connections 2024-09-19T08:50:39.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:39.196+0000 7f52ba44e640 1 -- 172.21.15.164:0/3611209411 wait complete. 2024-09-19T08:50:39.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:39 smithi164 ceph-mon[30928]: pgmap v148: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 480 MiB/s wr, 1.06k op/s 2024-09-19T08:50:39.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:39 smithi170 ceph-mon[35072]: pgmap v148: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 480 MiB/s wr, 1.06k op/s 2024-09-19T08:50:39.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:39 smithi173 ceph-mon[35306]: pgmap v148: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 480 MiB/s wr, 1.06k op/s 2024-09-19T08:50:41.011 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T08:50:41.012 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T08:50:41.274 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:50:41.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:41 smithi164 ceph-mon[30928]: pgmap v149: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 417 MiB/s wr, 824 op/s 2024-09-19T08:50:41.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:41 smithi170 ceph-mon[35072]: pgmap v149: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 417 MiB/s wr, 824 op/s 2024-09-19T08:50:41.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:41 smithi173 ceph-mon[35306]: pgmap v149: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 417 MiB/s wr, 824 op/s 2024-09-19T08:50:42.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.121+0000 7ff6a2108640 1 -- 172.21.15.164:0/1344943069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff69c100a80 msgr2=0x7ff69c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:42.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.121+0000 7ff6a2108640 1 --2- 172.21.15.164:0/1344943069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff69c100a80 0x7ff69c100ee0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff68c009930 tx=0x7ff68c02a420 comp rx=0 tx=0).stop 2024-09-19T08:50:42.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.121+0000 7ff6a2108640 1 -- 172.21.15.164:0/1344943069 shutdown_connections 2024-09-19T08:50:42.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.121+0000 7ff6a2108640 1 --2- 172.21.15.164:0/1344943069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff69c101420 0x7ff69c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:42.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.121+0000 7ff6a2108640 1 --2- 172.21.15.164:0/1344943069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff69c100a80 0x7ff69c100ee0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:42.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.121+0000 7ff6a2108640 1 --2- 172.21.15.164:0/1344943069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff69c106a80 0x7ff69c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:42.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.121+0000 7ff6a2108640 1 -- 172.21.15.164:0/1344943069 >> 172.21.15.164:0/1344943069 conn(0x7ff69c0fc7a0 msgr2=0x7ff69c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:42.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.121+0000 7ff6a2108640 1 -- 172.21.15.164:0/1344943069 shutdown_connections 2024-09-19T08:50:42.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.121+0000 7ff6a2108640 1 -- 172.21.15.164:0/1344943069 wait complete. 2024-09-19T08:50:42.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.122+0000 7ff6a2108640 1 Processor -- start 2024-09-19T08:50:42.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.122+0000 7ff6a2108640 1 -- start start 2024-09-19T08:50:42.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.122+0000 7ff6a2108640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff69c100a80 0x7ff69c1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:42.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.122+0000 7ff6a2108640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff69c101420 0x7ff69c197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:42.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.122+0000 7ff6a2108640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff69c106a80 0x7ff69c19bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:42.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.122+0000 7ff6a2108640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c110e80 con 0x7ff69c100a80 2024-09-19T08:50:42.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.122+0000 7ff6a2108640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c110d00 con 0x7ff69c106a80 2024-09-19T08:50:42.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.123+0000 7ff6a2108640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c111000 con 0x7ff69c101420 2024-09-19T08:50:42.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.123+0000 7ff69b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff69c100a80 0x7ff69c1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:42.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.123+0000 7ff69b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff69c100a80 0x7ff69c1972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60330/0 (socket says 172.21.15.164:60330) 2024-09-19T08:50:42.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.123+0000 7ff69b7fe640 1 -- 172.21.15.164:0/3546874134 learned_addr learned my addr 172.21.15.164:0/3546874134 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:42.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.123+0000 7ff69affd640 1 --2- 172.21.15.164:0/3546874134 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff69c101420 0x7ff69c197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:42.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.123+0000 7ff69bfff640 1 --2- 172.21.15.164:0/3546874134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff69c106a80 0x7ff69c19bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:42.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.123+0000 7ff69b7fe640 1 -- 172.21.15.164:0/3546874134 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff69c101420 msgr2=0x7ff69c197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:42.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.123+0000 7ff69b7fe640 1 --2- 172.21.15.164:0/3546874134 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff69c101420 0x7ff69c197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:42.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.123+0000 7ff69b7fe640 1 -- 172.21.15.164:0/3546874134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff69c106a80 msgr2=0x7ff69c19bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:42.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.123+0000 7ff69b7fe640 1 --2- 172.21.15.164:0/3546874134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff69c106a80 0x7ff69c19bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:42.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.123+0000 7ff69b7fe640 1 -- 172.21.15.164:0/3546874134 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69c19c270 con 0x7ff69c100a80 2024-09-19T08:50:42.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.123+0000 7ff69affd640 1 --2- 172.21.15.164:0/3546874134 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff69c101420 0x7ff69c197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:50:42.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.123+0000 7ff69bfff640 1 --2- 172.21.15.164:0/3546874134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff69c106a80 0x7ff69c19bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:50:42.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.123+0000 7ff69b7fe640 1 --2- 172.21.15.164:0/3546874134 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff69c100a80 0x7ff69c1972c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff69000c920 tx=0x7ff69000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:42.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.123+0000 7ff698ff9640 1 -- 172.21.15.164:0/3546874134 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff690007c70 con 0x7ff69c100a80 2024-09-19T08:50:42.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.123+0000 7ff6a2108640 1 -- 172.21.15.164:0/3546874134 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff69c19c560 con 0x7ff69c100a80 2024-09-19T08:50:42.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.123+0000 7ff698ff9640 1 -- 172.21.15.164:0/3546874134 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff690007e10 con 0x7ff69c100a80 2024-09-19T08:50:42.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.123+0000 7ff698ff9640 1 -- 172.21.15.164:0/3546874134 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff690018890 con 0x7ff69c100a80 2024-09-19T08:50:42.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.123+0000 7ff6a2108640 1 -- 172.21.15.164:0/3546874134 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff69c1a3e40 con 0x7ff69c100a80 2024-09-19T08:50:42.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.125+0000 7ff698ff9640 1 -- 172.21.15.164:0/3546874134 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7ff690018a70 con 0x7ff69c100a80 2024-09-19T08:50:42.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.126+0000 7ff6a44e1740 1 Processor -- start 2024-09-19T08:50:42.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.126+0000 7ff698ff9640 1 --2- 172.21.15.164:0/3546874134 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff6840a33b0 0x7ff6840a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:42.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.126+0000 7ff6a44e1740 1 -- start start 2024-09-19T08:50:42.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.126+0000 7ff69affd640 1 --2- 172.21.15.164:0/3546874134 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff6840a33b0 0x7ff6840a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:42.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.126+0000 7ff698ff9640 1 -- 172.21.15.164:0/3546874134 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7ff6900cfe30 con 0x7ff69c100a80 2024-09-19T08:50:42.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.126+0000 7ff6a44e1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555960419d70 0x55596041a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:42.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.126+0000 7ff6a44e1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55596041a690 0x555960424a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:42.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.126+0000 7ff6a44e1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555960424f50 0x555960427340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:42.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.126+0000 7ff6a44e1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55596037d280 con 0x555960419d70 2024-09-19T08:50:42.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.126+0000 7ff6a44e1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555960324b10 con 0x555960424f50 2024-09-19T08:50:42.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.126+0000 7ff6a44e1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555960379240 con 0x55596041a690 2024-09-19T08:50:42.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.126+0000 7ff69b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555960419d70 0x55596041a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:42.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.127+0000 7ff69b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555960419d70 0x55596041a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60342/0 (socket says 172.21.15.164:60342) 2024-09-19T08:50:42.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.127+0000 7ff69b7fe640 1 -- 172.21.15.164:0/2557351942 learned_addr learned my addr 172.21.15.164:0/2557351942 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:42.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.127+0000 7ff69bfff640 1 --2- 172.21.15.164:0/2557351942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555960424f50 0x555960427340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:42.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.127+0000 7ff69affd640 1 --2- 172.21.15.164:0/2557351942 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55596041a690 0x555960424a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:42.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.127+0000 7ff69affd640 1 --2- 172.21.15.164:0/3546874134 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff6840a33b0 0x7ff6840a5870 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff68c000c00 tx=0x7ff68c005a20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:42.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.127+0000 7ff69b7fe640 1 -- 172.21.15.164:0/2557351942 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55596041a690 msgr2=0x555960424a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:42.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.127+0000 7ff69b7fe640 1 --2- 172.21.15.164:0/2557351942 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55596041a690 0x555960424a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:42.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.127+0000 7ff69b7fe640 1 -- 172.21.15.164:0/2557351942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555960424f50 msgr2=0x555960427340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:42.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.127+0000 7ff69b7fe640 1 --2- 172.21.15.164:0/2557351942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555960424f50 0x555960427340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:42.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.127+0000 7ff69b7fe640 1 -- 172.21.15.164:0/2557351942 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559603311f0 con 0x555960419d70 2024-09-19T08:50:42.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.127+0000 7ff69b7fe640 1 --2- 172.21.15.164:0/2557351942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555960419d70 0x55596041a150 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff690098b50 tx=0x7ff69009c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd32835529d1d88a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:42.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.127+0000 7ff681ffb640 1 -- 172.21.15.164:0/2557351942 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6900b5070 con 0x555960419d70 2024-09-19T08:50:42.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.127+0000 7ff681ffb640 1 -- 172.21.15.164:0/2557351942 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff69009a480 con 0x555960419d70 2024-09-19T08:50:42.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.127+0000 7ff681ffb640 1 -- 172.21.15.164:0/2557351942 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6900987f0 con 0x555960419d70 2024-09-19T08:50:42.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.128+0000 7ff6a44e1740 1 -- 172.21.15.164:0/2557351942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555960419d70 msgr2=0x55596041a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:42.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.128+0000 7ff6a44e1740 1 --2- 172.21.15.164:0/2557351942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555960419d70 0x55596041a150 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff690098b50 tx=0x7ff69009c180 comp rx=0 tx=0).stop 2024-09-19T08:50:42.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.128+0000 7ff6a44e1740 1 -- 172.21.15.164:0/2557351942 shutdown_connections 2024-09-19T08:50:42.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.128+0000 7ff6a44e1740 1 --2- 172.21.15.164:0/2557351942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555960424f50 0x555960427340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:42.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.128+0000 7ff6a44e1740 1 --2- 172.21.15.164:0/2557351942 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55596041a690 0x555960424a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:42.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.128+0000 7ff6a44e1740 1 --2- 172.21.15.164:0/2557351942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555960419d70 0x55596041a150 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:42.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.128+0000 7ff6a44e1740 1 -- 172.21.15.164:0/2557351942 >> 172.21.15.164:0/2557351942 conn(0x555960416300 msgr2=0x5559604166e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:42.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.128+0000 7ff6a44e1740 1 -- 172.21.15.164:0/2557351942 shutdown_connections 2024-09-19T08:50:42.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.128+0000 7ff6a44e1740 1 -- 172.21.15.164:0/2557351942 wait complete. 2024-09-19T08:50:42.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.128+0000 7ff6a44e1740 1 -- start start 2024-09-19T08:50:42.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.128+0000 7ff6a44e1740 1 Processor -- start 2024-09-19T08:50:42.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.129+0000 7ff6a44e1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555960419d70 0x555960421f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:42.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.129+0000 7ff6a44e1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55596041a690 0x555960422440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:42.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.129+0000 7ff6a44e1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555960424f50 0x555960422980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:42.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.129+0000 7ff6a44e1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555960427b20 con 0x555960419d70 2024-09-19T08:50:42.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.129+0000 7ff6a44e1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555960337f10 con 0x555960424f50 2024-09-19T08:50:42.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.129+0000 7ff6a44e1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555960427ca0 con 0x55596041a690 2024-09-19T08:50:42.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.129+0000 7ff6a44e1740 10 client.0 _ll_register_callbacks cb 0x555960300a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:50:42.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.129+0000 7ff69b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555960419d70 0x555960421f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:42.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.129+0000 7ff69b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555960419d70 0x555960421f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60344/0 (socket says 172.21.15.164:60344) 2024-09-19T08:50:42.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.129+0000 7ff69b7fe640 1 -- 172.21.15.164:0/3317756387 learned_addr learned my addr 172.21.15.164:0/3317756387 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:42.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.129+0000 7ff69affd640 1 --2- 172.21.15.164:0/3317756387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55596041a690 0x555960422440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:42.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.129+0000 7ff69bfff640 1 --2- 172.21.15.164:0/3317756387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555960424f50 0x555960422980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:42.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.129+0000 7ff69b7fe640 1 -- 172.21.15.164:0/3317756387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55596041a690 msgr2=0x555960422440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:42.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.129+0000 7ff69b7fe640 1 --2- 172.21.15.164:0/3317756387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55596041a690 0x555960422440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:42.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.129+0000 7ff69b7fe640 1 -- 172.21.15.164:0/3317756387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555960424f50 msgr2=0x555960422980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:42.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.129+0000 7ff69b7fe640 1 --2- 172.21.15.164:0/3317756387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555960424f50 0x555960422980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:42.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.129+0000 7ff69b7fe640 1 -- 172.21.15.164:0/3317756387 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559600ee6a0 con 0x555960419d70 2024-09-19T08:50:42.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.129+0000 7ff69b7fe640 1 --2- 172.21.15.164:0/3317756387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555960419d70 0x555960421f00 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff6900d0150 tx=0x7ff690018f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5b4c238a5add8f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:42.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.129+0000 7ff67affd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:50:42.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.129+0000 7ff67affd640 1 -- 172.21.15.164:0/3317756387 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6900b5040 con 0x555960419d70 2024-09-19T08:50:42.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.130+0000 7ff67affd640 1 -- 172.21.15.164:0/3317756387 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7ff6900cd8e0 con 0x555960419d70 2024-09-19T08:50:42.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.130+0000 7ff6a44e1740 1 -- 172.21.15.164:0/3317756387 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff69009a480 con 0x555960419d70 2024-09-19T08:50:42.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.130+0000 7ff67affd640 1 -- 172.21.15.164:0/3317756387 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff6900987f0 con 0x555960419d70 2024-09-19T08:50:42.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.130+0000 7ff6a44e1740 10 client.16485 fetch_fsmap learned FSMap version 30 2024-09-19T08:50:42.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.130+0000 7ff6a44e1740 1 -- 172.21.15.164:0/3317756387 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555960433800 con 0x555960419d70 2024-09-19T08:50:42.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.130+0000 7ff67affd640 1 -- 172.21.15.164:0/3317756387 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff69009a480 con 0x555960419d70 2024-09-19T08:50:42.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.130+0000 7ff67affd640 1 -- 172.21.15.164:0/3317756387 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7ff6900a4960 con 0x555960419d70 2024-09-19T08:50:42.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.130+0000 7ff6a44e1740 10 client.16485 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:50:42.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.130+0000 7ff6a44e1740 10 client.16485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:50:42.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.130+0000 7ff6a44e1740 20 client.16485 populate_metadata read hostname 'smithi164' 2024-09-19T08:50:42.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.130+0000 7ff6a44e1740 1 --2- 172.21.15.164:0/3317756387 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555960434b50 0x555960436f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:42.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.130+0000 7ff6a44e1740 4 client.16485 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:50:42.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.130+0000 7ff6a44e1740 1 -- 172.21.15.164:0/3317756387 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55596038a7f0 con 0x555960434b50 2024-09-19T08:50:42.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.130+0000 7ff69affd640 1 --2- 172.21.15.164:0/3317756387 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555960434b50 0x555960436f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:42.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.131+0000 7ff67affd640 10 client.16485 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:50:42.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.131+0000 7ff69affd640 1 --2- 172.21.15.164:0/3317756387 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555960434b50 0x555960436f40 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38293841d3b4c5fc server_cookie=95243d2627c68382 in_seq=0 out_seq=0 2024-09-19T08:50:42.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.131+0000 7ff67affd640 1 -- 172.21.15.164:0/3317756387 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55596038a7f0 con 0x555960434b50 2024-09-19T08:50:42.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.131+0000 7ff67affd640 10 client.16485 handle_command_reply: tid=0 2024-09-19T08:50:42.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.131+0000 7ff6a44e1740 1 client.16485 shutdown 2024-09-19T08:50:42.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.132+0000 7ff6a44e1740 1 -- 172.21.15.164:0/3317756387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555960419d70 msgr2=0x555960421f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:42.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.132+0000 7ff6a44e1740 1 --2- 172.21.15.164:0/3317756387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555960419d70 0x555960421f00 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff6900d0150 tx=0x7ff690018f90 comp rx=0 tx=0).stop 2024-09-19T08:50:42.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.132+0000 7ff6a44e1740 1 -- 172.21.15.164:0/3317756387 shutdown_connections 2024-09-19T08:50:42.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.132+0000 7ff6a44e1740 1 --2- 172.21.15.164:0/3317756387 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555960434b50 0x555960436f40 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:42.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.132+0000 7ff6a44e1740 1 --2- 172.21.15.164:0/3317756387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555960424f50 0x555960422980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:42.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.132+0000 7ff6a44e1740 1 --2- 172.21.15.164:0/3317756387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55596041a690 0x555960422440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:42.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.132+0000 7ff6a44e1740 1 --2- 172.21.15.164:0/3317756387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555960419d70 0x555960421f00 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:42.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.132+0000 7ff6a44e1740 1 -- 172.21.15.164:0/3317756387 >> 172.21.15.164:0/3317756387 conn(0x555960416300 msgr2=0x555960413270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:42.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.132+0000 7ff67affd640 0 client.16485 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:50:42.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.132+0000 7ff6a44e1740 1 -- 172.21.15.164:0/3317756387 shutdown_connections 2024-09-19T08:50:42.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.132+0000 7ff6a44e1740 1 -- 172.21.15.164:0/3317756387 wait complete. 2024-09-19T08:50:42.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.132+0000 7ff6a44e1740 20 client.16485 trim_cache size 0 max 16384 2024-09-19T08:50:42.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.149+0000 7ff6a44e1740 1 Processor -- start 2024-09-19T08:50:42.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.149+0000 7ff6a44e1740 1 -- start start 2024-09-19T08:50:42.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.149+0000 7ff6a44e1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555960419d70 0x55596041c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:42.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.149+0000 7ff6a44e1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55596041a690 0x555960439f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:42.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.149+0000 7ff6a44e1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555960424f50 0x55596043a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:42.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.149+0000 7ff6a44e1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c1ab520 con 0x555960419d70 2024-09-19T08:50:42.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.149+0000 7ff6a44e1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c1ad120 con 0x555960424f50 2024-09-19T08:50:42.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.149+0000 7ff6a44e1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c1abbf0 con 0x55596041a690 2024-09-19T08:50:42.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.149+0000 7ff69b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555960419d70 0x55596041c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:42.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.149+0000 7ff69b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555960419d70 0x55596041c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60358/0 (socket says 172.21.15.164:60358) 2024-09-19T08:50:42.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.149+0000 7ff69b7fe640 1 -- 172.21.15.164:0/397906904 learned_addr learned my addr 172.21.15.164:0/397906904 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:42.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.149+0000 7ff69bfff640 1 --2- 172.21.15.164:0/397906904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555960424f50 0x55596043a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:42.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.149+0000 7ff69affd640 1 --2- 172.21.15.164:0/397906904 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55596041a690 0x555960439f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:42.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.149+0000 7ff69b7fe640 1 -- 172.21.15.164:0/397906904 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55596041a690 msgr2=0x555960439f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:42.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.149+0000 7ff69b7fe640 1 --2- 172.21.15.164:0/397906904 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55596041a690 0x555960439f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:42.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.149+0000 7ff69b7fe640 1 -- 172.21.15.164:0/397906904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555960424f50 msgr2=0x55596043a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:42.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.149+0000 7ff69b7fe640 1 --2- 172.21.15.164:0/397906904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555960424f50 0x55596043a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:42.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.149+0000 7ff69b7fe640 1 -- 172.21.15.164:0/397906904 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559604173b0 con 0x555960419d70 2024-09-19T08:50:42.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.150+0000 7ff69b7fe640 1 --2- 172.21.15.164:0/397906904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555960419d70 0x55596041c7d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff6900cdbf0 tx=0x7ff69009c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6bbea3cfb32bbdb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:42.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.150+0000 7ff6817fa640 1 -- 172.21.15.164:0/397906904 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6900b5040 con 0x555960419d70 2024-09-19T08:50:42.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.150+0000 7ff6817fa640 1 -- 172.21.15.164:0/397906904 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6900bc0b0 con 0x555960419d70 2024-09-19T08:50:42.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.150+0000 7ff6817fa640 1 -- 172.21.15.164:0/397906904 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6900a4960 con 0x555960419d70 2024-09-19T08:50:42.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.150+0000 7ff6a44e1740 1 -- 172.21.15.164:0/397906904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555960419d70 msgr2=0x55596041c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:42.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.150+0000 7ff6a44e1740 1 --2- 172.21.15.164:0/397906904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555960419d70 0x55596041c7d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff6900cdbf0 tx=0x7ff69009c180 comp rx=0 tx=0).stop 2024-09-19T08:50:42.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.150+0000 7ff6a44e1740 1 -- 172.21.15.164:0/397906904 shutdown_connections 2024-09-19T08:50:42.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.150+0000 7ff6a44e1740 1 --2- 172.21.15.164:0/397906904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555960424f50 0x55596043a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:42.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.150+0000 7ff6a44e1740 1 --2- 172.21.15.164:0/397906904 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55596041a690 0x555960439f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:42.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.150+0000 7ff6a44e1740 1 --2- 172.21.15.164:0/397906904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555960419d70 0x55596041c7d0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:42.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.150+0000 7ff6a44e1740 1 -- 172.21.15.164:0/397906904 >> 172.21.15.164:0/397906904 conn(0x555960416300 msgr2=0x555960423e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:42.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.151+0000 7ff6a44e1740 1 -- 172.21.15.164:0/397906904 shutdown_connections 2024-09-19T08:50:42.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.151+0000 7ff6a44e1740 1 -- 172.21.15.164:0/397906904 wait complete. 2024-09-19T08:50:42.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.151+0000 7ff6a44e1740 1 -- start start 2024-09-19T08:50:42.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.151+0000 7ff6a44e1740 1 Processor -- start 2024-09-19T08:50:42.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.151+0000 7ff6a44e1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555960419d70 0x555960438900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:42.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.151+0000 7ff6a44e1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55596041a690 0x555960438e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:42.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.151+0000 7ff6a44e1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555960424f50 0x555960439d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:42.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.151+0000 7ff6a44e1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c1ac990 con 0x555960424f50 2024-09-19T08:50:42.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.151+0000 7ff6a44e1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c1ac2c0 con 0x555960419d70 2024-09-19T08:50:42.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.151+0000 7ff6a44e1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c1a9520 con 0x55596041a690 2024-09-19T08:50:42.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.151+0000 7ff6a44e1740 10 client.0 _ll_register_callbacks cb 0x7ff6900b3070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:50:42.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.152+0000 7ff69affd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55596041a690 0x555960438e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:42.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.152+0000 7ff69affd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55596041a690 0x555960438e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:51134/0 (socket says 172.21.15.164:51134) 2024-09-19T08:50:42.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.152+0000 7ff69bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555960424f50 0x555960439d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:42.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.152+0000 7ff69affd640 1 -- 172.21.15.164:0/1861688414 learned_addr learned my addr 172.21.15.164:0/1861688414 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:42.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.152+0000 7ff69b7fe640 1 --2- 172.21.15.164:0/1861688414 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555960419d70 0x555960438900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:42.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.152+0000 7ff69affd640 1 -- 172.21.15.164:0/1861688414 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555960419d70 msgr2=0x555960438900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:42.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.152+0000 7ff69affd640 1 --2- 172.21.15.164:0/1861688414 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555960419d70 0x555960438900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:42.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.152+0000 7ff69affd640 1 -- 172.21.15.164:0/1861688414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555960424f50 msgr2=0x555960439d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:42.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.152+0000 7ff69affd640 1 --2- 172.21.15.164:0/1861688414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555960424f50 0x555960439d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:42.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.152+0000 7ff69affd640 1 -- 172.21.15.164:0/1861688414 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55596043a590 con 0x55596041a690 2024-09-19T08:50:42.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.152+0000 7ff69affd640 1 --2- 172.21.15.164:0/1861688414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55596041a690 0x555960438e40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff68c009850 tx=0x7ff68c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6db141256994be1a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:42.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.152+0000 7ff67affd640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-09-19T08:50:42.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.152+0000 7ff67affd640 1 -- 172.21.15.164:0/1861688414 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c057020 con 0x55596041a690 2024-09-19T08:50:42.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.152+0000 7ff67affd640 1 -- 172.21.15.164:0/1861688414 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7ff68c005450 con 0x55596041a690 2024-09-19T08:50:42.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.152+0000 7ff6a44e1740 1 -- 172.21.15.164:0/1861688414 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6900bc0b0 con 0x55596041a690 2024-09-19T08:50:42.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.152+0000 7ff67affd640 1 -- 172.21.15.164:0/1861688414 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff68c062050 con 0x55596041a690 2024-09-19T08:50:42.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.152+0000 7ff6a44e1740 10 client.25876 fetch_fsmap learned FSMap version 30 2024-09-19T08:50:42.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.152+0000 7ff6a44e1740 1 -- 172.21.15.164:0/1861688414 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5559604a0090 con 0x55596041a690 2024-09-19T08:50:42.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.153+0000 7ff67affd640 1 -- 172.21.15.164:0/1861688414 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6900bc0b0 con 0x55596041a690 2024-09-19T08:50:42.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.153+0000 7ff67affd640 1 -- 172.21.15.164:0/1861688414 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7ff68c0036f0 con 0x55596041a690 2024-09-19T08:50:42.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.153+0000 7ff6a44e1740 10 client.25876 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:50:42.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.153+0000 7ff6a44e1740 10 client.25876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:50:42.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.153+0000 7ff6a44e1740 20 client.25876 populate_metadata read hostname 'smithi164' 2024-09-19T08:50:42.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.153+0000 7ff6a44e1740 1 --2- 172.21.15.164:0/1861688414 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5559604a0950 0x5559604a2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:42.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.153+0000 7ff6a44e1740 4 client.25876 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T08:50:42.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.153+0000 7ff6a44e1740 1 -- 172.21.15.164:0/1861688414 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555960384d00 con 0x5559604a0950 2024-09-19T08:50:42.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.153+0000 7ff69b7fe640 1 --2- 172.21.15.164:0/1861688414 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5559604a0950 0x5559604a2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:42.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.153+0000 7ff69b7fe640 1 --2- 172.21.15.164:0/1861688414 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5559604a0950 0x5559604a2d70 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75ab05518729ff00 server_cookie=fe2b6b3ab07006e8 in_seq=0 out_seq=0 2024-09-19T08:50:42.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.153+0000 7ff67affd640 10 client.25876 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:50:42.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.154+0000 7ff67affd640 1 -- 172.21.15.164:0/1861688414 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555960384d00 con 0x5559604a0950 2024-09-19T08:50:42.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.154+0000 7ff67affd640 10 client.25876 handle_command_reply: tid=0 2024-09-19T08:50:42.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.154+0000 7ff6a44e1740 1 client.25876 shutdown 2024-09-19T08:50:42.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.154+0000 7ff6a44e1740 1 -- 172.21.15.164:0/1861688414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55596041a690 msgr2=0x555960438e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:42.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.154+0000 7ff6a44e1740 1 --2- 172.21.15.164:0/1861688414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55596041a690 0x555960438e40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff68c009850 tx=0x7ff68c009260 comp rx=0 tx=0).stop 2024-09-19T08:50:42.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.154+0000 7ff6a44e1740 1 -- 172.21.15.164:0/1861688414 shutdown_connections 2024-09-19T08:50:42.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.154+0000 7ff6a44e1740 1 --2- 172.21.15.164:0/1861688414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555960424f50 0x555960439d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:42.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.154+0000 7ff6a44e1740 1 --2- 172.21.15.164:0/1861688414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55596041a690 0x555960438e40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:42.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.154+0000 7ff6a44e1740 1 --2- 172.21.15.164:0/1861688414 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5559604a0950 0x5559604a2d70 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:42.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.154+0000 7ff6a44e1740 1 --2- 172.21.15.164:0/1861688414 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555960419d70 0x555960438900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:42.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.154+0000 7ff67affd640 0 client.25876 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:50:42.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.154+0000 7ff6a44e1740 1 -- 172.21.15.164:0/1861688414 >> 172.21.15.164:0/1861688414 conn(0x555960416300 msgr2=0x55596042fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:42.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.155+0000 7ff6a44e1740 1 -- 172.21.15.164:0/1861688414 shutdown_connections 2024-09-19T08:50:42.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.155+0000 7ff6a44e1740 1 -- 172.21.15.164:0/1861688414 wait complete. 2024-09-19T08:50:42.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.155+0000 7ff6a44e1740 20 client.25876 trim_cache size 0 max 16384 2024-09-19T08:50:42.162 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:50:42.162 INFO:teuthology.orchestra.run.smithi164.stdout: "return_code": 0, 2024-09-19T08:50:42.162 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_tag": "818851bb-1873-440f-b8a8-31ad8880d06d", 2024-09-19T08:50:42.162 INFO:teuthology.orchestra.run.smithi164.stdout: "mode": "asynchronous" 2024-09-19T08:50:42.162 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:50:42.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.156+0000 7ff6a2108640 1 -- 172.21.15.164:0/3546874134 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff6840a33b0 msgr2=0x7ff6840a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:42.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.156+0000 7ff6a2108640 1 --2- 172.21.15.164:0/3546874134 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff6840a33b0 0x7ff6840a5870 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff68c000c00 tx=0x7ff68c005a20 comp rx=0 tx=0).stop 2024-09-19T08:50:42.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.156+0000 7ff6a2108640 1 -- 172.21.15.164:0/3546874134 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff69c100a80 msgr2=0x7ff69c1972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:42.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.156+0000 7ff6a2108640 1 --2- 172.21.15.164:0/3546874134 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff69c100a80 0x7ff69c1972c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff69000c920 tx=0x7ff69000cde0 comp rx=0 tx=0).stop 2024-09-19T08:50:42.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.156+0000 7ff6a2108640 1 -- 172.21.15.164:0/3546874134 shutdown_connections 2024-09-19T08:50:42.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.156+0000 7ff6a2108640 1 --2- 172.21.15.164:0/3546874134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff69c106a80 0x7ff69c19bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:42.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.156+0000 7ff6a2108640 1 --2- 172.21.15.164:0/3546874134 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff69c101420 0x7ff69c197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:42.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.156+0000 7ff6a2108640 1 --2- 172.21.15.164:0/3546874134 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff6840a33b0 0x7ff6840a5870 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:42.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.156+0000 7ff6a2108640 1 --2- 172.21.15.164:0/3546874134 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff69c100a80 0x7ff69c1972c0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:42.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.156+0000 7ff6a2108640 1 -- 172.21.15.164:0/3546874134 >> 172.21.15.164:0/3546874134 conn(0x7ff69c0fc7a0 msgr2=0x7ff69c0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:42.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.156+0000 7ff6a2108640 1 -- 172.21.15.164:0/3546874134 shutdown_connections 2024-09-19T08:50:42.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:42.156+0000 7ff6a2108640 1 -- 172.21.15.164:0/3546874134 wait complete. 2024-09-19T08:50:42.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:42 smithi170 ceph-mon[35072]: scrub queued for path: / 2024-09-19T08:50:42.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:42 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:50:42.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:42 smithi170 ceph-mon[35072]: scrub summary: active paths [/] 2024-09-19T08:50:42.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:42 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:50:42.883 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:42 smithi164 ceph-mon[30928]: scrub queued for path: / 2024-09-19T08:50:42.883 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:42 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:50:42.883 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:42 smithi164 ceph-mon[30928]: scrub summary: active paths [/] 2024-09-19T08:50:42.883 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:42 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:50:42.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:42 smithi173 ceph-mon[35306]: scrub queued for path: / 2024-09-19T08:50:42.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:42 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:50:42.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:42 smithi173 ceph-mon[35306]: scrub summary: active paths [/] 2024-09-19T08:50:42.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:42 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:50:42.912 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:50:43.156 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:50:43.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.668+0000 7f0e6e60c640 1 -- 172.21.15.164:0/1558284418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e68106f20 msgr2=0x7f0e6810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:43.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.668+0000 7f0e6e60c640 1 --2- 172.21.15.164:0/1558284418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e68106f20 0x7f0e6810eaa0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f0e60009ca0 tx=0x7f0e6002a8c0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.668+0000 7f0e6e60c640 1 -- 172.21.15.164:0/1558284418 shutdown_connections 2024-09-19T08:50:43.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.668+0000 7f0e6e60c640 1 --2- 172.21.15.164:0/1558284418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e68106f20 0x7f0e6810eaa0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.668+0000 7f0e6e60c640 1 --2- 172.21.15.164:0/1558284418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e681027a0 0x7f0e681067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.668+0000 7f0e6e60c640 1 --2- 172.21.15.164:0/1558284418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e68101df0 0x7f0e681021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.668+0000 7f0e6e60c640 1 -- 172.21.15.164:0/1558284418 >> 172.21.15.164:0/1558284418 conn(0x7f0e680fd8b0 msgr2=0x7f0e680ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:43.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.668+0000 7f0e6e60c640 1 -- 172.21.15.164:0/1558284418 shutdown_connections 2024-09-19T08:50:43.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.668+0000 7f0e6e60c640 1 -- 172.21.15.164:0/1558284418 wait complete. 2024-09-19T08:50:43.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.669+0000 7f0e6e60c640 1 Processor -- start 2024-09-19T08:50:43.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.669+0000 7f0e6e60c640 1 -- start start 2024-09-19T08:50:43.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.669+0000 7f0e6e60c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e68101df0 0x7f0e68198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:43.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.669+0000 7f0e6e60c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e681027a0 0x7f0e681988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:43.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.669+0000 7f0e6e60c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e68106f20 0x7f0e6819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:43.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.669+0000 7f0e6e60c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0e68111f80 con 0x7f0e681027a0 2024-09-19T08:50:43.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.669+0000 7f0e6e60c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0e68111e00 con 0x7f0e68101df0 2024-09-19T08:50:43.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.669+0000 7f0e6e60c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0e68112100 con 0x7f0e68106f20 2024-09-19T08:50:43.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.669+0000 7f0e677fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e681027a0 0x7f0e681988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:43.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.669+0000 7f0e677fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e681027a0 0x7f0e681988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48516/0 (socket says 172.21.15.164:48516) 2024-09-19T08:50:43.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.669+0000 7f0e677fe640 1 -- 172.21.15.164:0/1957475696 learned_addr learned my addr 172.21.15.164:0/1957475696 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:43.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.669+0000 7f0e67fff640 1 --2- 172.21.15.164:0/1957475696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e68101df0 0x7f0e68198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:43.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.670+0000 7f0e6cba9640 1 --2- 172.21.15.164:0/1957475696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e68106f20 0x7f0e6819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:43.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.670+0000 7f0e677fe640 1 -- 172.21.15.164:0/1957475696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e68106f20 msgr2=0x7f0e6819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:43.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.670+0000 7f0e677fe640 1 --2- 172.21.15.164:0/1957475696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e68106f20 0x7f0e6819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.670+0000 7f0e677fe640 1 -- 172.21.15.164:0/1957475696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e68101df0 msgr2=0x7f0e68198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:43.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.670+0000 7f0e677fe640 1 --2- 172.21.15.164:0/1957475696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e68101df0 0x7f0e68198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.670+0000 7f0e677fe640 1 -- 172.21.15.164:0/1957475696 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e6819d310 con 0x7f0e681027a0 2024-09-19T08:50:43.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.670+0000 7f0e677fe640 1 --2- 172.21.15.164:0/1957475696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e681027a0 0x7f0e681988a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c00d770 tx=0x7f0e5c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:43.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.670+0000 7f0e657fa640 1 -- 172.21.15.164:0/1957475696 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c017070 con 0x7f0e681027a0 2024-09-19T08:50:43.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.670+0000 7f0e6e60c640 1 -- 172.21.15.164:0/1957475696 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e6819d600 con 0x7f0e681027a0 2024-09-19T08:50:43.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.670+0000 7f0e657fa640 1 -- 172.21.15.164:0/1957475696 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e5c005500 con 0x7f0e681027a0 2024-09-19T08:50:43.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.670+0000 7f0e657fa640 1 -- 172.21.15.164:0/1957475696 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c004e00 con 0x7f0e681027a0 2024-09-19T08:50:43.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.670+0000 7f0e6e60c640 1 -- 172.21.15.164:0/1957475696 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e681a4ee0 con 0x7f0e681027a0 2024-09-19T08:50:43.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.672+0000 7f0e657fa640 1 -- 172.21.15.164:0/1957475696 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f0e5c004fa0 con 0x7f0e681027a0 2024-09-19T08:50:43.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.672+0000 7f0e709e5740 1 Processor -- start 2024-09-19T08:50:43.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.673+0000 7f0e709e5740 1 -- start start 2024-09-19T08:50:43.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.673+0000 7f0e709e5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592782558b0 0x559278255c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:43.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.673+0000 7f0e709e5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592782561d0 0x559278260550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:43.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.673+0000 7f0e709e5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559278260a90 0x559278262e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:43.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.673+0000 7f0e709e5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5592781b72e0 con 0x5592782558b0 2024-09-19T08:50:43.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.673+0000 7f0e709e5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55927815ece0 con 0x5592782561d0 2024-09-19T08:50:43.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.673+0000 7f0e709e5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5592781b32a0 con 0x559278260a90 2024-09-19T08:50:43.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.673+0000 7f0e6cba9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592782558b0 0x559278255c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:43.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.673+0000 7f0e6cba9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592782558b0 0x559278255c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48524/0 (socket says 172.21.15.164:48524) 2024-09-19T08:50:43.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.673+0000 7f0e6cba9640 1 -- 172.21.15.164:0/1545445829 learned_addr learned my addr 172.21.15.164:0/1545445829 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:43.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.673+0000 7f0e657fa640 1 --2- 172.21.15.164:0/1957475696 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f0e500a33b0 0x7f0e500a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:43.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.673+0000 7f0e677fe640 1 --2- 172.21.15.164:0/1545445829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559278260a90 0x559278262e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:43.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.673+0000 7f0e67fff640 1 --2- 172.21.15.164:0/1545445829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592782561d0 0x559278260550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:43.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.673+0000 7f0e6cba9640 1 --2- 172.21.15.164:0/1957475696 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f0e500a33b0 0x7f0e500a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:43.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.673+0000 7f0e657fa640 1 -- 172.21.15.164:0/1957475696 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f0e5c0cf670 con 0x7f0e681027a0 2024-09-19T08:50:43.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.673+0000 7f0e6cba9640 1 -- 172.21.15.164:0/1545445829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559278260a90 msgr2=0x559278262e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:43.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.673+0000 7f0e6cba9640 1 --2- 172.21.15.164:0/1545445829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559278260a90 0x559278262e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.673+0000 7f0e6cba9640 1 -- 172.21.15.164:0/1545445829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592782561d0 msgr2=0x559278260550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:43.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.673+0000 7f0e6cba9640 1 --2- 172.21.15.164:0/1545445829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592782561d0 0x559278260550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.673+0000 7f0e6cba9640 1 -- 172.21.15.164:0/1545445829 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559278143440 con 0x5592782558b0 2024-09-19T08:50:43.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.674+0000 7f0e6cba9640 1 --2- 172.21.15.164:0/1545445829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592782558b0 0x559278255c90 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f0e60005fe0 tx=0x7f0e60009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3bb8b384c78a89 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:43.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.674+0000 7f0e4e7fc640 1 -- 172.21.15.164:0/1545445829 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e60003e10 con 0x5592782558b0 2024-09-19T08:50:43.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.674+0000 7f0e4e7fc640 1 -- 172.21.15.164:0/1545445829 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e6004e070 con 0x5592782558b0 2024-09-19T08:50:43.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.674+0000 7f0e4e7fc640 1 -- 172.21.15.164:0/1545445829 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6002fb70 con 0x5592782558b0 2024-09-19T08:50:43.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.674+0000 7f0e709e5740 1 -- 172.21.15.164:0/1545445829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592782558b0 msgr2=0x559278255c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:43.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.674+0000 7f0e6cba9640 1 --2- 172.21.15.164:0/1957475696 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f0e500a33b0 0x7f0e500a5870 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0e6002f540 tx=0x7f0e6004b000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:43.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.674+0000 7f0e709e5740 1 --2- 172.21.15.164:0/1545445829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592782558b0 0x559278255c90 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f0e60005fe0 tx=0x7f0e60009a00 comp rx=0 tx=0).stop 2024-09-19T08:50:43.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.675+0000 7f0e709e5740 1 -- 172.21.15.164:0/1545445829 shutdown_connections 2024-09-19T08:50:43.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.675+0000 7f0e709e5740 1 --2- 172.21.15.164:0/1545445829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559278260a90 0x559278262e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.675+0000 7f0e709e5740 1 --2- 172.21.15.164:0/1545445829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592782561d0 0x559278260550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.675+0000 7f0e709e5740 1 --2- 172.21.15.164:0/1545445829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592782558b0 0x559278255c90 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.675+0000 7f0e709e5740 1 -- 172.21.15.164:0/1545445829 >> 172.21.15.164:0/1545445829 conn(0x559278251e40 msgr2=0x559278252220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:43.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.675+0000 7f0e709e5740 1 -- 172.21.15.164:0/1545445829 shutdown_connections 2024-09-19T08:50:43.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.675+0000 7f0e709e5740 1 -- 172.21.15.164:0/1545445829 wait complete. 2024-09-19T08:50:43.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.675+0000 7f0e709e5740 1 -- start start 2024-09-19T08:50:43.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.675+0000 7f0e709e5740 1 Processor -- start 2024-09-19T08:50:43.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.676+0000 7f0e709e5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592782558b0 0x5592782603c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:43.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.676+0000 7f0e709e5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592782561d0 0x55927825d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:43.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.676+0000 7f0e709e5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559278260a90 0x55927825dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:43.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.676+0000 7f0e709e5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5592782634e0 con 0x5592782561d0 2024-09-19T08:50:43.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.676+0000 7f0e709e5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x559278172050 con 0x5592782558b0 2024-09-19T08:50:43.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.676+0000 7f0e709e5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559278263660 con 0x559278260a90 2024-09-19T08:50:43.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.676+0000 7f0e709e5740 10 client.0 _ll_register_callbacks cb 0x5592781b2160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:50:43.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.676+0000 7f0e677fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592782561d0 0x55927825d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:43.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.676+0000 7f0e677fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592782561d0 0x55927825d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48536/0 (socket says 172.21.15.164:48536) 2024-09-19T08:50:43.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.676+0000 7f0e677fe640 1 -- 172.21.15.164:0/2625160030 learned_addr learned my addr 172.21.15.164:0/2625160030 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:43.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.676+0000 7f0e67fff640 1 --2- 172.21.15.164:0/2625160030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592782558b0 0x5592782603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:43.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.676+0000 7f0e6cba9640 1 --2- 172.21.15.164:0/2625160030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559278260a90 0x55927825dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:43.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.676+0000 7f0e677fe640 1 -- 172.21.15.164:0/2625160030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559278260a90 msgr2=0x55927825dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:43.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.676+0000 7f0e677fe640 1 --2- 172.21.15.164:0/2625160030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559278260a90 0x55927825dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.676+0000 7f0e677fe640 1 -- 172.21.15.164:0/2625160030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592782558b0 msgr2=0x5592782603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:43.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.676+0000 7f0e677fe640 1 --2- 172.21.15.164:0/2625160030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592782558b0 0x5592782603c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.676+0000 7f0e677fe640 1 -- 172.21.15.164:0/2625160030 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559277f283c0 con 0x5592782561d0 2024-09-19T08:50:43.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.676+0000 7f0e6cba9640 1 --2- 172.21.15.164:0/2625160030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559278260a90 0x55927825dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:50:43.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.676+0000 7f0e67fff640 1 --2- 172.21.15.164:0/2625160030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592782558b0 0x5592782603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:50:43.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.676+0000 7f0e677fe640 1 --2- 172.21.15.164:0/2625160030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592782561d0 0x55927825d670 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c098390 tx=0x7f0e5c06b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab55eeb480342a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:43.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.676+0000 7f0e477fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:50:43.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.676+0000 7f0e477fe640 1 -- 172.21.15.164:0/2625160030 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c020030 con 0x5592782561d0 2024-09-19T08:50:43.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.676+0000 7f0e477fe640 1 -- 172.21.15.164:0/2625160030 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f0e5c009cc0 con 0x5592782561d0 2024-09-19T08:50:43.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.676+0000 7f0e709e5740 1 -- 172.21.15.164:0/2625160030 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e6004e070 con 0x5592782561d0 2024-09-19T08:50:43.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.676+0000 7f0e477fe640 1 -- 172.21.15.164:0/2625160030 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0e5c09c0c0 con 0x5592782561d0 2024-09-19T08:50:43.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.676+0000 7f0e709e5740 10 client.16503 fetch_fsmap learned FSMap version 30 2024-09-19T08:50:43.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.676+0000 7f0e709e5740 1 -- 172.21.15.164:0/2625160030 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55927826ee90 con 0x5592782561d0 2024-09-19T08:50:43.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.677+0000 7f0e477fe640 1 -- 172.21.15.164:0/2625160030 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e6004e070 con 0x5592782561d0 2024-09-19T08:50:43.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.677+0000 7f0e477fe640 1 -- 172.21.15.164:0/2625160030 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f0e5c098030 con 0x5592782561d0 2024-09-19T08:50:43.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.677+0000 7f0e709e5740 10 client.16503 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:50:43.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.677+0000 7f0e709e5740 10 client.16503 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:50:43.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.677+0000 7f0e709e5740 20 client.16503 populate_metadata read hostname 'smithi164' 2024-09-19T08:50:43.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.677+0000 7f0e709e5740 1 --2- 172.21.15.164:0/2625160030 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x559278270410 0x5592782728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:43.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.677+0000 7f0e709e5740 4 client.16503 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:50:43.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.677+0000 7f0e709e5740 1 -- 172.21.15.164:0/2625160030 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592781c4850 con 0x559278270410 2024-09-19T08:50:43.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.677+0000 7f0e67fff640 1 --2- 172.21.15.164:0/2625160030 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x559278270410 0x5592782728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:43.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.677+0000 7f0e67fff640 1 --2- 172.21.15.164:0/2625160030 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x559278270410 0x5592782728b0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28ec8dcc60f14fe4 server_cookie=fa0609635c9073dc in_seq=0 out_seq=0 2024-09-19T08:50:43.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.677+0000 7f0e477fe640 10 client.16503 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:50:43.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.678+0000 7f0e477fe640 1 -- 172.21.15.164:0/2625160030 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5592781c4850 con 0x559278270410 2024-09-19T08:50:43.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.678+0000 7f0e477fe640 10 client.16503 handle_command_reply: tid=0 2024-09-19T08:50:43.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.678+0000 7f0e709e5740 1 client.16503 shutdown 2024-09-19T08:50:43.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.678+0000 7f0e709e5740 1 -- 172.21.15.164:0/2625160030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592782561d0 msgr2=0x55927825d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:43.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.678+0000 7f0e709e5740 1 --2- 172.21.15.164:0/2625160030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592782561d0 0x55927825d670 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c098390 tx=0x7f0e5c06b5e0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.678+0000 7f0e709e5740 1 -- 172.21.15.164:0/2625160030 shutdown_connections 2024-09-19T08:50:43.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.678+0000 7f0e709e5740 1 --2- 172.21.15.164:0/2625160030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559278260a90 0x55927825dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.678+0000 7f0e709e5740 1 --2- 172.21.15.164:0/2625160030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592782561d0 0x55927825d670 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.678+0000 7f0e709e5740 1 --2- 172.21.15.164:0/2625160030 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x559278270410 0x5592782728b0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.678+0000 7f0e709e5740 1 --2- 172.21.15.164:0/2625160030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592782558b0 0x5592782603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.678+0000 7f0e709e5740 1 -- 172.21.15.164:0/2625160030 >> 172.21.15.164:0/2625160030 conn(0x559278251e40 msgr2=0x55927824cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:43.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.678+0000 7f0e477fe640 0 client.16503 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:50:43.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.678+0000 7f0e709e5740 1 -- 172.21.15.164:0/2625160030 shutdown_connections 2024-09-19T08:50:43.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.679+0000 7f0e709e5740 1 -- 172.21.15.164:0/2625160030 wait complete. 2024-09-19T08:50:43.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.679+0000 7f0e709e5740 20 client.16503 trim_cache size 0 max 16384 2024-09-19T08:50:43.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.695+0000 7f0e709e5740 1 Processor -- start 2024-09-19T08:50:43.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.695+0000 7f0e709e5740 1 -- start start 2024-09-19T08:50:43.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.695+0000 7f0e709e5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592782558b0 0x55927826c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:43.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.695+0000 7f0e709e5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592782561d0 0x559278278130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:43.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.695+0000 7f0e709e5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559278260a90 0x559278278670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:43.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.695+0000 7f0e709e5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0e681ab3f0 con 0x559278260a90 2024-09-19T08:50:43.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.695+0000 7f0e709e5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0e681acff0 con 0x5592782558b0 2024-09-19T08:50:43.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.695+0000 7f0e709e5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0e681abac0 con 0x5592782561d0 2024-09-19T08:50:43.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.696+0000 7f0e6cba9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559278260a90 0x559278278670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:43.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.696+0000 7f0e6cba9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559278260a90 0x559278278670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48552/0 (socket says 172.21.15.164:48552) 2024-09-19T08:50:43.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.696+0000 7f0e6cba9640 1 -- 172.21.15.164:0/4182424680 learned_addr learned my addr 172.21.15.164:0/4182424680 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:43.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.696+0000 7f0e67fff640 1 --2- 172.21.15.164:0/4182424680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592782558b0 0x55927826c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:43.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.696+0000 7f0e6cba9640 1 -- 172.21.15.164:0/4182424680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592782561d0 msgr2=0x559278278130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:43.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.696+0000 7f0e6cba9640 1 --2- 172.21.15.164:0/4182424680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592782561d0 0x559278278130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.696+0000 7f0e6cba9640 1 -- 172.21.15.164:0/4182424680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592782558b0 msgr2=0x55927826c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:43.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.696+0000 7f0e6cba9640 1 --2- 172.21.15.164:0/4182424680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592782558b0 0x55927826c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.696+0000 7f0e6cba9640 1 -- 172.21.15.164:0/4182424680 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559278260730 con 0x559278260a90 2024-09-19T08:50:43.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.696+0000 7f0e67fff640 1 --2- 172.21.15.164:0/4182424680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592782558b0 0x55927826c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:50:43.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.696+0000 7f0e6cba9640 1 --2- 172.21.15.164:0/4182424680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559278260a90 0x559278278670 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f0e6002ade0 tx=0x7f0e6003c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6067314eac111752 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:43.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.696+0000 7f0e4dffb640 1 -- 172.21.15.164:0/4182424680 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e60058040 con 0x559278260a90 2024-09-19T08:50:43.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.696+0000 7f0e4dffb640 1 -- 172.21.15.164:0/4182424680 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e6004e040 con 0x559278260a90 2024-09-19T08:50:43.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.696+0000 7f0e4dffb640 1 -- 172.21.15.164:0/4182424680 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e60004410 con 0x559278260a90 2024-09-19T08:50:43.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.697+0000 7f0e709e5740 1 -- 172.21.15.164:0/4182424680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559278260a90 msgr2=0x559278278670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:43.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.697+0000 7f0e709e5740 1 --2- 172.21.15.164:0/4182424680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559278260a90 0x559278278670 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f0e6002ade0 tx=0x7f0e6003c000 comp rx=0 tx=0).stop 2024-09-19T08:50:43.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.697+0000 7f0e709e5740 1 -- 172.21.15.164:0/4182424680 shutdown_connections 2024-09-19T08:50:43.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.697+0000 7f0e709e5740 1 --2- 172.21.15.164:0/4182424680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559278260a90 0x559278278670 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.697+0000 7f0e709e5740 1 --2- 172.21.15.164:0/4182424680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592782561d0 0x559278278130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.697+0000 7f0e709e5740 1 --2- 172.21.15.164:0/4182424680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592782558b0 0x55927826c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.697+0000 7f0e709e5740 1 -- 172.21.15.164:0/4182424680 >> 172.21.15.164:0/4182424680 conn(0x559278251e40 msgr2=0x55927825c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:43.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.697+0000 7f0e709e5740 1 -- 172.21.15.164:0/4182424680 shutdown_connections 2024-09-19T08:50:43.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.697+0000 7f0e709e5740 1 -- 172.21.15.164:0/4182424680 wait complete. 2024-09-19T08:50:43.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.697+0000 7f0e709e5740 1 -- start start 2024-09-19T08:50:43.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.697+0000 7f0e709e5740 1 Processor -- start 2024-09-19T08:50:43.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.698+0000 7f0e709e5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592782558b0 0x559278264040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:43.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.698+0000 7f0e709e5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592782561d0 0x559278264580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:43.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.698+0000 7f0e709e5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559278260a90 0x5592782d30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:43.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.698+0000 7f0e709e5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0e681ac860 con 0x559278260a90 2024-09-19T08:50:43.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.698+0000 7f0e709e5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0e681ac190 con 0x5592782558b0 2024-09-19T08:50:43.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.698+0000 7f0e709e5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0e681a93f0 con 0x5592782561d0 2024-09-19T08:50:43.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.698+0000 7f0e709e5740 10 client.0 _ll_register_callbacks cb 0x7f0e5c00de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:50:43.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.698+0000 7f0e6cba9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559278260a90 0x5592782d30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:43.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.698+0000 7f0e6cba9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559278260a90 0x5592782d30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48554/0 (socket says 172.21.15.164:48554) 2024-09-19T08:50:43.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.698+0000 7f0e6cba9640 1 -- 172.21.15.164:0/680767214 learned_addr learned my addr 172.21.15.164:0/680767214 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:43.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.698+0000 7f0e677fe640 1 --2- 172.21.15.164:0/680767214 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592782561d0 0x559278264580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:43.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.698+0000 7f0e67fff640 1 --2- 172.21.15.164:0/680767214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592782558b0 0x559278264040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:43.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.698+0000 7f0e6cba9640 1 -- 172.21.15.164:0/680767214 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592782561d0 msgr2=0x559278264580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:43.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.698+0000 7f0e6cba9640 1 --2- 172.21.15.164:0/680767214 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592782561d0 0x559278264580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.698+0000 7f0e6cba9640 1 -- 172.21.15.164:0/680767214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592782558b0 msgr2=0x559278264040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:43.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.698+0000 7f0e6cba9640 1 --2- 172.21.15.164:0/680767214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592782558b0 0x559278264040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.698+0000 7f0e6cba9640 1 -- 172.21.15.164:0/680767214 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592782d3700 con 0x559278260a90 2024-09-19T08:50:43.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.698+0000 7f0e67fff640 1 --2- 172.21.15.164:0/680767214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592782558b0 0x559278264040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:50:43.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.698+0000 7f0e6cba9640 1 --2- 172.21.15.164:0/680767214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559278260a90 0x5592782d30c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f0e600099b0 tx=0x7f0e600099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92122b65859e548 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:43.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.698+0000 7f0e477fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:50:43.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.698+0000 7f0e477fe640 1 -- 172.21.15.164:0/680767214 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e60058030 con 0x559278260a90 2024-09-19T08:50:43.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.699+0000 7f0e477fe640 1 -- 172.21.15.164:0/680767214 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f0e6002f610 con 0x559278260a90 2024-09-19T08:50:43.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.699+0000 7f0e709e5740 1 -- 172.21.15.164:0/680767214 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e6004e040 con 0x559278260a90 2024-09-19T08:50:43.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.699+0000 7f0e477fe640 1 -- 172.21.15.164:0/680767214 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0e6002fe90 con 0x559278260a90 2024-09-19T08:50:43.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.699+0000 7f0e709e5740 10 client.16509 fetch_fsmap learned FSMap version 30 2024-09-19T08:50:43.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.699+0000 7f0e709e5740 1 -- 172.21.15.164:0/680767214 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5592782daa10 con 0x559278260a90 2024-09-19T08:50:43.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.699+0000 7f0e477fe640 1 -- 172.21.15.164:0/680767214 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e6004e040 con 0x559278260a90 2024-09-19T08:50:43.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.699+0000 7f0e477fe640 1 -- 172.21.15.164:0/680767214 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f0e60076020 con 0x559278260a90 2024-09-19T08:50:43.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.699+0000 7f0e709e5740 10 client.16509 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:50:43.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.699+0000 7f0e709e5740 10 client.16509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:50:43.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.699+0000 7f0e709e5740 20 client.16509 populate_metadata read hostname 'smithi164' 2024-09-19T08:50:43.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.699+0000 7f0e709e5740 1 --2- 172.21.15.164:0/680767214 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5592782dae10 0x5592782dd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:43.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.699+0000 7f0e709e5740 4 client.16509 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:50:43.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.699+0000 7f0e709e5740 1 -- 172.21.15.164:0/680767214 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5592781bed60 con 0x5592782dae10 2024-09-19T08:50:43.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.700+0000 7f0e67fff640 1 --2- 172.21.15.164:0/680767214 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5592782dae10 0x5592782dd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:43.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.700+0000 7f0e67fff640 1 --2- 172.21.15.164:0/680767214 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5592782dae10 0x5592782dd230 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c9ce900b0974cb4 server_cookie=146ad3521104624c in_seq=0 out_seq=0 2024-09-19T08:50:43.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.700+0000 7f0e477fe640 10 client.16509 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:50:43.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.700+0000 7f0e477fe640 1 -- 172.21.15.164:0/680767214 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5592781bed60 con 0x5592782dae10 2024-09-19T08:50:43.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.700+0000 7f0e477fe640 10 client.16509 handle_command_reply: tid=0 2024-09-19T08:50:43.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.700+0000 7f0e709e5740 1 client.16509 shutdown 2024-09-19T08:50:43.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.700+0000 7f0e709e5740 1 -- 172.21.15.164:0/680767214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559278260a90 msgr2=0x5592782d30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:43.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.700+0000 7f0e709e5740 1 --2- 172.21.15.164:0/680767214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559278260a90 0x5592782d30c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f0e600099b0 tx=0x7f0e600099e0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.700+0000 7f0e709e5740 1 -- 172.21.15.164:0/680767214 shutdown_connections 2024-09-19T08:50:43.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.700+0000 7f0e709e5740 1 --2- 172.21.15.164:0/680767214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559278260a90 0x5592782d30c0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.700+0000 7f0e709e5740 1 --2- 172.21.15.164:0/680767214 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592782561d0 0x559278264580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.700+0000 7f0e709e5740 1 --2- 172.21.15.164:0/680767214 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5592782dae10 0x5592782dd230 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.700+0000 7f0e709e5740 1 --2- 172.21.15.164:0/680767214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592782558b0 0x559278264040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.700+0000 7f0e709e5740 1 -- 172.21.15.164:0/680767214 >> 172.21.15.164:0/680767214 conn(0x559278251e40 msgr2=0x55927825eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:43.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.700+0000 7f0e477fe640 0 client.16509 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:50:43.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.701+0000 7f0e709e5740 1 -- 172.21.15.164:0/680767214 shutdown_connections 2024-09-19T08:50:43.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.701+0000 7f0e709e5740 1 -- 172.21.15.164:0/680767214 wait complete. 2024-09-19T08:50:43.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.701+0000 7f0e709e5740 20 client.16509 trim_cache size 0 max 16384 2024-09-19T08:50:43.709 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:50:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T08:50:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: "scrubs": { 2024-09-19T08:50:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: "818851bb-1873-440f-b8a8-31ad8880d06d": { 2024-09-19T08:50:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: "path": "/", 2024-09-19T08:50:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: "tag": "818851bb-1873-440f-b8a8-31ad8880d06d", 2024-09-19T08:50:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: "options": "recursive,force" 2024-09-19T08:50:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:50:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:50:43.710 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:50:43.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.701+0000 7f0e6e60c640 1 -- 172.21.15.164:0/1957475696 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f0e500a33b0 msgr2=0x7f0e500a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:43.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.701+0000 7f0e6e60c640 1 --2- 172.21.15.164:0/1957475696 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f0e500a33b0 0x7f0e500a5870 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0e6002f540 tx=0x7f0e6004b000 comp rx=0 tx=0).stop 2024-09-19T08:50:43.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.701+0000 7f0e6e60c640 1 -- 172.21.15.164:0/1957475696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e681027a0 msgr2=0x7f0e681988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:43.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.701+0000 7f0e6e60c640 1 --2- 172.21.15.164:0/1957475696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e681027a0 0x7f0e681988a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c00d770 tx=0x7f0e5c00dc30 comp rx=0 tx=0).stop 2024-09-19T08:50:43.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.701+0000 7f0e6e60c640 1 -- 172.21.15.164:0/1957475696 shutdown_connections 2024-09-19T08:50:43.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.701+0000 7f0e6e60c640 1 --2- 172.21.15.164:0/1957475696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e68106f20 0x7f0e6819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.701+0000 7f0e6e60c640 1 --2- 172.21.15.164:0/1957475696 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f0e500a33b0 0x7f0e500a5870 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.701+0000 7f0e6e60c640 1 --2- 172.21.15.164:0/1957475696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e681027a0 0x7f0e681988a0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.701+0000 7f0e6e60c640 1 --2- 172.21.15.164:0/1957475696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e68101df0 0x7f0e68198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:43.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.701+0000 7f0e6e60c640 1 -- 172.21.15.164:0/1957475696 >> 172.21.15.164:0/1957475696 conn(0x7f0e680fd8b0 msgr2=0x7f0e680fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:43.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.701+0000 7f0e6e60c640 1 -- 172.21.15.164:0/1957475696 shutdown_connections 2024-09-19T08:50:43.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:43.701+0000 7f0e6e60c640 1 -- 172.21.15.164:0/1957475696 wait complete. 2024-09-19T08:50:43.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:43 smithi170 ceph-mon[35072]: scrub summary: active paths [/] 2024-09-19T08:50:43.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:43 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:50:43.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:43 smithi170 ceph-mon[35072]: scrub summary: active paths [/] 2024-09-19T08:50:43.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:43 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:50:43.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:43 smithi170 ceph-mon[35072]: pgmap v150: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 213 MiB/s wr, 336 op/s 2024-09-19T08:50:43.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:43 smithi173 ceph-mon[35306]: scrub summary: active paths [/] 2024-09-19T08:50:43.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:43 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:50:43.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:43 smithi173 ceph-mon[35306]: scrub summary: active paths [/] 2024-09-19T08:50:43.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:43 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:50:43.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:43 smithi173 ceph-mon[35306]: pgmap v150: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 213 MiB/s wr, 336 op/s 2024-09-19T08:50:44.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:43 smithi164 ceph-mon[30928]: scrub summary: active paths [/] 2024-09-19T08:50:44.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:43 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:50:44.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:43 smithi164 ceph-mon[30928]: scrub summary: active paths [/] 2024-09-19T08:50:44.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:43 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:50:44.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:43 smithi164 ceph-mon[30928]: pgmap v150: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 213 MiB/s wr, 336 op/s 2024-09-19T08:50:44.589 INFO:tasks.cephfs.filesystem:scrub status for tag:818851bb-1873-440f-b8a8-31ad8880d06d - {'path': '/', 'tag': '818851bb-1873-440f-b8a8-31ad8880d06d', 'options': 'recursive,force'} 2024-09-19T08:50:45.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:45 smithi170 ceph-mon[35072]: pgmap v151: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 147 MiB/s wr, 307 op/s 2024-09-19T08:50:45.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:45 smithi173 ceph-mon[35306]: pgmap v151: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 147 MiB/s wr, 307 op/s 2024-09-19T08:50:46.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:45 smithi164 ceph-mon[30928]: pgmap v151: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 147 MiB/s wr, 307 op/s 2024-09-19T08:50:47.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:47 smithi170 ceph-mon[35072]: pgmap v152: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 97 MiB/s wr, 404 op/s 2024-09-19T08:50:47.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:47 smithi173 ceph-mon[35306]: pgmap v152: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 97 MiB/s wr, 404 op/s 2024-09-19T08:50:48.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:47 smithi164 ceph-mon[30928]: pgmap v152: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 97 MiB/s wr, 404 op/s 2024-09-19T08:50:48.879 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:50:49.125 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:50:49.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.619+0000 7f996cb7b640 1 -- 172.21.15.164:0/3007883456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99681082a0 msgr2=0x7f996810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:49.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.619+0000 7f996cb7b640 1 --2- 172.21.15.164:0/3007883456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99681082a0 0x7f996810ec40 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9958009930 tx=0x7f995802a420 comp rx=0 tx=0).stop 2024-09-19T08:50:49.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.620+0000 7f996cb7b640 1 -- 172.21.15.164:0/3007883456 shutdown_connections 2024-09-19T08:50:49.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.620+0000 7f996cb7b640 1 --2- 172.21.15.164:0/3007883456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99681082a0 0x7f996810ec40 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:49.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.620+0000 7f996cb7b640 1 --2- 172.21.15.164:0/3007883456 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99681036e0 0x7f9968107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:49.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.620+0000 7f996cb7b640 1 --2- 172.21.15.164:0/3007883456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9968102d30 0x7f9968103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:49.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.620+0000 7f996cb7b640 1 -- 172.21.15.164:0/3007883456 >> 172.21.15.164:0/3007883456 conn(0x7f99680fc7d0 msgr2=0x7f99680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:49.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.620+0000 7f996cb7b640 1 -- 172.21.15.164:0/3007883456 shutdown_connections 2024-09-19T08:50:49.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.620+0000 7f996cb7b640 1 -- 172.21.15.164:0/3007883456 wait complete. 2024-09-19T08:50:49.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.620+0000 7f996cb7b640 1 Processor -- start 2024-09-19T08:50:49.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.620+0000 7f996cb7b640 1 -- start start 2024-09-19T08:50:49.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.621+0000 7f996cb7b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9968102d30 0x7f9968100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:49.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.621+0000 7f996cb7b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99681036e0 0x7f9968100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:49.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.621+0000 7f996cb7b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99681082a0 0x7f9968101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:49.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.621+0000 7f996cb7b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f99681120a0 con 0x7f9968102d30 2024-09-19T08:50:49.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.621+0000 7f996cb7b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9968111f20 con 0x7f99681082a0 2024-09-19T08:50:49.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.621+0000 7f996cb7b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9968112220 con 0x7f99681036e0 2024-09-19T08:50:49.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.621+0000 7f996659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9968102d30 0x7f9968100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:49.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.621+0000 7f996659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9968102d30 0x7f9968100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48570/0 (socket says 172.21.15.164:48570) 2024-09-19T08:50:49.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.621+0000 7f996659c640 1 -- 172.21.15.164:0/1827825283 learned_addr learned my addr 172.21.15.164:0/1827825283 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:49.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.621+0000 7f9966d9d640 1 --2- 172.21.15.164:0/1827825283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99681082a0 0x7f9968101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:49.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.621+0000 7f9965d9b640 1 --2- 172.21.15.164:0/1827825283 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99681036e0 0x7f9968100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:49.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.621+0000 7f996659c640 1 -- 172.21.15.164:0/1827825283 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99681036e0 msgr2=0x7f9968100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:49.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.621+0000 7f996659c640 1 --2- 172.21.15.164:0/1827825283 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99681036e0 0x7f9968100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:49.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.621+0000 7f996659c640 1 -- 172.21.15.164:0/1827825283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99681082a0 msgr2=0x7f9968101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:49.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.621+0000 7f996659c640 1 --2- 172.21.15.164:0/1827825283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99681082a0 0x7f9968101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:49.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.621+0000 7f996659c640 1 -- 172.21.15.164:0/1827825283 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99681a6aa0 con 0x7f9968102d30 2024-09-19T08:50:49.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.621+0000 7f996659c640 1 --2- 172.21.15.164:0/1827825283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9968102d30 0x7f9968100990 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f996000cce0 tx=0x7f996000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:49.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.621+0000 7f995f7fe640 1 -- 172.21.15.164:0/1827825283 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960004b20 con 0x7f9968102d30 2024-09-19T08:50:49.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.621+0000 7f996cb7b640 1 -- 172.21.15.164:0/1827825283 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99681a6d90 con 0x7f9968102d30 2024-09-19T08:50:49.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.621+0000 7f995f7fe640 1 -- 172.21.15.164:0/1827825283 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9960004cc0 con 0x7f9968102d30 2024-09-19T08:50:49.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.621+0000 7f995f7fe640 1 -- 172.21.15.164:0/1827825283 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996000f7d0 con 0x7f9968102d30 2024-09-19T08:50:49.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.621+0000 7f996cb7b640 1 -- 172.21.15.164:0/1827825283 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99681a7250 con 0x7f9968102d30 2024-09-19T08:50:49.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.622+0000 7f996cb7b640 1 -- 172.21.15.164:0/1827825283 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9968103110 con 0x7f9968102d30 2024-09-19T08:50:49.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.624+0000 7f995f7fe640 1 -- 172.21.15.164:0/1827825283 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f996000f970 con 0x7f9968102d30 2024-09-19T08:50:49.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.624+0000 7f995f7fe640 1 --2- 172.21.15.164:0/1827825283 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f99540a33b0 0x7f99540a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:49.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.624+0000 7f9965d9b640 1 --2- 172.21.15.164:0/1827825283 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f99540a33b0 0x7f99540a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:49.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.624+0000 7f995f7fe640 1 -- 172.21.15.164:0/1827825283 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f99600cfe10 con 0x7f9968102d30 2024-09-19T08:50:49.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.625+0000 7f9965d9b640 1 --2- 172.21.15.164:0/1827825283 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f99540a33b0 0x7f99540a5870 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9950005d90 tx=0x7f9950005d20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:49.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.628+0000 7f995f7fe640 1 -- 172.21.15.164:0/1827825283 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9960014030 con 0x7f9968102d30 2024-09-19T08:50:49.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.830+0000 7f996cb7b640 1 -- 172.21.15.164:0/1827825283 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f996810a2b0 con 0x7f9968102d30 2024-09-19T08:50:49.831 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:49 smithi164 ceph-mon[30928]: pgmap v153: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 41 MiB/s wr, 368 op/s 2024-09-19T08:50:49.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.831+0000 7f995f7fe640 1 -- 172.21.15.164:0/1827825283 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11991 (secure 0 0 0) 0x7f996009c860 con 0x7f9968102d30 2024-09-19T08:50:49.832 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:50:49.832 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":30,"btime":"2024-09-19T08:50:17:326587+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:50:17.071374+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:50:49.832 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 30 2024-09-19T08:50:49.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.833+0000 7f996cb7b640 1 -- 172.21.15.164:0/1827825283 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f99540a33b0 msgr2=0x7f99540a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:49.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.833+0000 7f996cb7b640 1 --2- 172.21.15.164:0/1827825283 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f99540a33b0 0x7f99540a5870 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9950005d90 tx=0x7f9950005d20 comp rx=0 tx=0).stop 2024-09-19T08:50:49.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.833+0000 7f996cb7b640 1 -- 172.21.15.164:0/1827825283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9968102d30 msgr2=0x7f9968100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:49.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.833+0000 7f996cb7b640 1 --2- 172.21.15.164:0/1827825283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9968102d30 0x7f9968100990 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f996000cce0 tx=0x7f996000a620 comp rx=0 tx=0).stop 2024-09-19T08:50:49.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.834+0000 7f996cb7b640 1 -- 172.21.15.164:0/1827825283 shutdown_connections 2024-09-19T08:50:49.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.834+0000 7f996cb7b640 1 --2- 172.21.15.164:0/1827825283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99681082a0 0x7f9968101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:49.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.834+0000 7f996cb7b640 1 --2- 172.21.15.164:0/1827825283 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99681036e0 0x7f9968100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:49.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.834+0000 7f996cb7b640 1 --2- 172.21.15.164:0/1827825283 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f99540a33b0 0x7f99540a5870 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:49.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.834+0000 7f996cb7b640 1 --2- 172.21.15.164:0/1827825283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9968102d30 0x7f9968100990 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:49.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.834+0000 7f996cb7b640 1 -- 172.21.15.164:0/1827825283 >> 172.21.15.164:0/1827825283 conn(0x7f99680fc7d0 msgr2=0x7f9968106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:49.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.834+0000 7f996cb7b640 1 -- 172.21.15.164:0/1827825283 shutdown_connections 2024-09-19T08:50:49.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:49.834+0000 7f996cb7b640 1 -- 172.21.15.164:0/1827825283 wait complete. 2024-09-19T08:50:49.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:49 smithi170 ceph-mon[35072]: pgmap v153: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 41 MiB/s wr, 368 op/s 2024-09-19T08:50:49.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:49 smithi173 ceph-mon[35306]: pgmap v153: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 41 MiB/s wr, 368 op/s 2024-09-19T08:50:50.641 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id ea4bdac6 --release --await' 2024-09-19T08:50:50.645 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id ea4bdac6 --release --await 2024-09-19T08:50:50.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:50 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/1827825283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:50:50.896 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:50:50.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:50 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/1827825283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:50:51.017 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:50 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/1827825283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:50:51.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.387+0000 7f8609c10640 1 -- 172.21.15.164:0/1676248910 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8604101de0 msgr2=0x7f86041021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:51.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.387+0000 7f8609c10640 1 --2- 172.21.15.164:0/1676248910 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8604101de0 0x7f86041021c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f85ec009900 tx=0x7f85ec02a410 comp rx=0 tx=0).stop 2024-09-19T08:50:51.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.387+0000 7f8609c10640 1 -- 172.21.15.164:0/1676248910 shutdown_connections 2024-09-19T08:50:51.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.387+0000 7f8609c10640 1 --2- 172.21.15.164:0/1676248910 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8604106f10 0x7f860410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:51.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.387+0000 7f8609c10640 1 --2- 172.21.15.164:0/1676248910 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8604102790 0x7f86041067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:51.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.387+0000 7f8609c10640 1 --2- 172.21.15.164:0/1676248910 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8604101de0 0x7f86041021c0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:51.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.387+0000 7f8609c10640 1 -- 172.21.15.164:0/1676248910 >> 172.21.15.164:0/1676248910 conn(0x7f86040fd8c0 msgr2=0x7f86040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:51.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.387+0000 7f8609c10640 1 -- 172.21.15.164:0/1676248910 shutdown_connections 2024-09-19T08:50:51.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.387+0000 7f8609c10640 1 -- 172.21.15.164:0/1676248910 wait complete. 2024-09-19T08:50:51.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.387+0000 7f8609c10640 1 Processor -- start 2024-09-19T08:50:51.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.388+0000 7f8609c10640 1 -- start start 2024-09-19T08:50:51.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.388+0000 7f8609c10640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8604101de0 0x7f86040744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:51.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.388+0000 7f8609c10640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8604102790 0x7f8604074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:51.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.388+0000 7f8609c10640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8604106f10 0x7f8604074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:51.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.388+0000 7f8609c10640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8604111f70 con 0x7f8604102790 2024-09-19T08:50:51.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.388+0000 7f8609c10640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8604111df0 con 0x7f8604106f10 2024-09-19T08:50:51.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.388+0000 7f8609c10640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f86041120f0 con 0x7f8604101de0 2024-09-19T08:50:51.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.388+0000 7f8602ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8604102790 0x7f8604074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:51.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.388+0000 7f8602ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8604102790 0x7f8604074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48592/0 (socket says 172.21.15.164:48592) 2024-09-19T08:50:51.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.388+0000 7f8602ffd640 1 -- 172.21.15.164:0/3001233319 learned_addr learned my addr 172.21.15.164:0/3001233319 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:51.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.388+0000 7f8603fff640 1 --2- 172.21.15.164:0/3001233319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8604106f10 0x7f8604074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:51.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.388+0000 7f86037fe640 1 --2- 172.21.15.164:0/3001233319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8604101de0 0x7f86040744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:51.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.388+0000 7f8602ffd640 1 -- 172.21.15.164:0/3001233319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8604101de0 msgr2=0x7f86040744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:51.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.388+0000 7f8602ffd640 1 --2- 172.21.15.164:0/3001233319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8604101de0 0x7f86040744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:51.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.388+0000 7f8602ffd640 1 -- 172.21.15.164:0/3001233319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8604106f10 msgr2=0x7f8604074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:51.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.388+0000 7f8602ffd640 1 --2- 172.21.15.164:0/3001233319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8604106f10 0x7f8604074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:51.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.388+0000 7f8602ffd640 1 -- 172.21.15.164:0/3001233319 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86041a6900 con 0x7f8604102790 2024-09-19T08:50:51.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.388+0000 7f8603fff640 1 --2- 172.21.15.164:0/3001233319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8604106f10 0x7f8604074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:50:51.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.388+0000 7f86037fe640 1 --2- 172.21.15.164:0/3001233319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8604101de0 0x7f86040744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:50:51.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.389+0000 7f8602ffd640 1 --2- 172.21.15.164:0/3001233319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8604102790 0x7f8604074a30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f85f400c890 tx=0x7f85f400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:51.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.389+0000 7f8600ff9640 1 -- 172.21.15.164:0/3001233319 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f4004220 con 0x7f8604102790 2024-09-19T08:50:51.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.389+0000 7f8600ff9640 1 -- 172.21.15.164:0/3001233319 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85f40043c0 con 0x7f8604102790 2024-09-19T08:50:51.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.389+0000 7f8609c10640 1 -- 172.21.15.164:0/3001233319 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86041a6bf0 con 0x7f8604102790 2024-09-19T08:50:51.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.389+0000 7f8600ff9640 1 -- 172.21.15.164:0/3001233319 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f40049e0 con 0x7f8604102790 2024-09-19T08:50:51.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.389+0000 7f8609c10640 1 -- 172.21.15.164:0/3001233319 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86041a7130 con 0x7f8604102790 2024-09-19T08:50:51.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.391+0000 7f8600ff9640 1 -- 172.21.15.164:0/3001233319 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f85f40079b0 con 0x7f8604102790 2024-09-19T08:50:51.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.391+0000 7f860bfe9740 1 Processor -- start 2024-09-19T08:50:51.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.392+0000 7f860bfe9740 1 -- start start 2024-09-19T08:50:51.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.392+0000 7f860bfe9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a5fb1e060 0x562a5fb1e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:51.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.392+0000 7f8600ff9640 1 --2- 172.21.15.164:0/3001233319 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f85f00a33b0 0x7f85f00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:51.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.392+0000 7f860bfe9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a5fb1e980 0x562a5fb28d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:51.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.392+0000 7f860bfe9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a5fb29240 0x562a5fb2b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:51.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.392+0000 7f860bfe9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562a5fa83620 con 0x562a5fb1e060 2024-09-19T08:50:51.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.392+0000 7f860bfe9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562a5fa2ae50 con 0x562a5fb1e980 2024-09-19T08:50:51.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.392+0000 7f860bfe9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562a5fa7f5e0 con 0x562a5fb29240 2024-09-19T08:50:51.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.392+0000 7f8603fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a5fb1e060 0x562a5fb1e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:51.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.392+0000 7f86037fe640 1 --2- 172.21.15.164:0/3001233319 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f85f00a33b0 0x7f85f00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:51.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.392+0000 7f8603fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a5fb1e060 0x562a5fb1e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48606/0 (socket says 172.21.15.164:48606) 2024-09-19T08:50:51.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.392+0000 7f8603fff640 1 -- 172.21.15.164:0/902667592 learned_addr learned my addr 172.21.15.164:0/902667592 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:51.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.392+0000 7f8600ff9640 1 -- 172.21.15.164:0/3001233319 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f85f40cfdc0 con 0x7f8604102790 2024-09-19T08:50:51.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.392+0000 7f8603fff640 1 --2- 172.21.15.164:0/902667592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a5fb29240 0x562a5fb2b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:51.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.392+0000 7f8602ffd640 1 --2- 172.21.15.164:0/902667592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a5fb1e980 0x562a5fb28d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:51.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.392+0000 7f8603fff640 1 -- 172.21.15.164:0/902667592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a5fb29240 msgr2=0x562a5fb2b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:51.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.392+0000 7f8603fff640 1 --2- 172.21.15.164:0/902667592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a5fb29240 0x562a5fb2b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:51.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.392+0000 7f8603fff640 1 -- 172.21.15.164:0/902667592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a5fb1e980 msgr2=0x562a5fb28d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:51.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.392+0000 7f8603fff640 1 --2- 172.21.15.164:0/902667592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a5fb1e980 0x562a5fb28d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:51.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.392+0000 7f8603fff640 1 -- 172.21.15.164:0/902667592 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a5fa1bd40 con 0x562a5fb1e060 2024-09-19T08:50:51.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.392+0000 7f8603fff640 1 --2- 172.21.15.164:0/902667592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a5fb1e060 0x562a5fb1e440 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f85fc009fc0 tx=0x7f85fc009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a68dba14ab48485 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:51.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.392+0000 7f86037fe640 1 --2- 172.21.15.164:0/3001233319 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f85f00a33b0 0x7f85f00a5870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f85ec005ab0 tx=0x7f85ec005a20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:51.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.393+0000 7f85f9ffb640 1 -- 172.21.15.164:0/902667592 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc01a030 con 0x562a5fb1e060 2024-09-19T08:50:51.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.393+0000 7f85f9ffb640 1 -- 172.21.15.164:0/902667592 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85fc014de0 con 0x562a5fb1e060 2024-09-19T08:50:51.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.393+0000 7f85f9ffb640 1 -- 172.21.15.164:0/902667592 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc0053b0 con 0x562a5fb1e060 2024-09-19T08:50:51.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.393+0000 7f860bfe9740 1 -- 172.21.15.164:0/902667592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a5fb1e060 msgr2=0x562a5fb1e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:51.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.393+0000 7f860bfe9740 1 --2- 172.21.15.164:0/902667592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a5fb1e060 0x562a5fb1e440 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f85fc009fc0 tx=0x7f85fc009910 comp rx=0 tx=0).stop 2024-09-19T08:50:51.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.393+0000 7f860bfe9740 1 -- 172.21.15.164:0/902667592 shutdown_connections 2024-09-19T08:50:51.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.393+0000 7f860bfe9740 1 --2- 172.21.15.164:0/902667592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a5fb29240 0x562a5fb2b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:51.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.393+0000 7f860bfe9740 1 --2- 172.21.15.164:0/902667592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a5fb1e980 0x562a5fb28d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:51.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.393+0000 7f860bfe9740 1 --2- 172.21.15.164:0/902667592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a5fb1e060 0x562a5fb1e440 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:51.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.393+0000 7f860bfe9740 1 -- 172.21.15.164:0/902667592 >> 172.21.15.164:0/902667592 conn(0x562a5fb1a5f0 msgr2=0x562a5fb1a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:51.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.393+0000 7f860bfe9740 1 -- 172.21.15.164:0/902667592 shutdown_connections 2024-09-19T08:50:51.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.393+0000 7f860bfe9740 1 -- 172.21.15.164:0/902667592 wait complete. 2024-09-19T08:50:51.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.394+0000 7f860bfe9740 1 -- start start 2024-09-19T08:50:51.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.394+0000 7f860bfe9740 1 Processor -- start 2024-09-19T08:50:51.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.394+0000 7f860bfe9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a5fb1e060 0x562a5fb26110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:51.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.394+0000 7f860bfe9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a5fb1e980 0x562a5fb26650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:51.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.394+0000 7f860bfe9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a5fb29240 0x562a5fb26b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:51.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.394+0000 7f860bfe9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562a5fb2be10 con 0x562a5fb29240 2024-09-19T08:50:51.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.394+0000 7f860bfe9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562a5fa0bf10 con 0x562a5fb1e060 2024-09-19T08:50:51.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.394+0000 7f860bfe9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562a5fb2bf90 con 0x562a5fb1e980 2024-09-19T08:50:51.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.394+0000 7f860bfe9740 10 client.0 _ll_register_callbacks cb 0x562a5fa2fdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:50:51.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.394+0000 7f8603fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a5fb29240 0x562a5fb26b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:51.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.394+0000 7f8603fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a5fb29240 0x562a5fb26b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48616/0 (socket says 172.21.15.164:48616) 2024-09-19T08:50:51.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.394+0000 7f8603fff640 1 -- 172.21.15.164:0/2956046159 learned_addr learned my addr 172.21.15.164:0/2956046159 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:51.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.394+0000 7f86037fe640 1 --2- 172.21.15.164:0/2956046159 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a5fb1e060 0x562a5fb26110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:51.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.395+0000 7f8603fff640 1 -- 172.21.15.164:0/2956046159 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a5fb1e980 msgr2=0x562a5fb26650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:51.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.395+0000 7f8602ffd640 1 --2- 172.21.15.164:0/2956046159 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a5fb1e980 0x562a5fb26650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:51.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.395+0000 7f8603fff640 1 --2- 172.21.15.164:0/2956046159 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a5fb1e980 0x562a5fb26650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:51.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.395+0000 7f8603fff640 1 -- 172.21.15.164:0/2956046159 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a5fb1e060 msgr2=0x562a5fb26110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:51.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.395+0000 7f8603fff640 1 --2- 172.21.15.164:0/2956046159 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a5fb1e060 0x562a5fb26110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:51.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.395+0000 7f8603fff640 1 -- 172.21.15.164:0/2956046159 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a5f792580 con 0x562a5fb29240 2024-09-19T08:50:51.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.395+0000 7f8602ffd640 1 --2- 172.21.15.164:0/2956046159 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a5fb1e980 0x562a5fb26650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:50:51.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.395+0000 7f8603fff640 1 --2- 172.21.15.164:0/2956046159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a5fb29240 0x562a5fb26b90 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f85fc0050f0 tx=0x7f85fc012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb7f3c082e95ee6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:51.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.395+0000 7f85e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:50:51.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.395+0000 7f85e2ffd640 1 -- 172.21.15.164:0/2956046159 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc01a070 con 0x562a5fb29240 2024-09-19T08:50:51.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.395+0000 7f85e2ffd640 1 -- 172.21.15.164:0/2956046159 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f85fc005190 con 0x562a5fb29240 2024-09-19T08:50:51.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.395+0000 7f860bfe9740 1 -- 172.21.15.164:0/2956046159 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85fc014de0 con 0x562a5fb29240 2024-09-19T08:50:51.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.395+0000 7f85e2ffd640 1 -- 172.21.15.164:0/2956046159 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f85fc004e50 con 0x562a5fb29240 2024-09-19T08:50:51.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.395+0000 7f860bfe9740 10 client.16527 fetch_fsmap learned FSMap version 30 2024-09-19T08:50:51.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.395+0000 7f860bfe9740 1 -- 172.21.15.164:0/2956046159 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562a5f85c960 con 0x562a5fb29240 2024-09-19T08:50:51.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.395+0000 7f85e2ffd640 1 -- 172.21.15.164:0/2956046159 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85fc014de0 con 0x562a5fb29240 2024-09-19T08:50:51.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.395+0000 7f85e2ffd640 1 -- 172.21.15.164:0/2956046159 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f85fc027070 con 0x562a5fb29240 2024-09-19T08:50:51.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.395+0000 7f860bfe9740 10 client.16527 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:50:51.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.395+0000 7f860bfe9740 10 client.16527 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:50:51.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.395+0000 7f860bfe9740 20 client.16527 populate_metadata read hostname 'smithi164' 2024-09-19T08:50:51.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.396+0000 7f860bfe9740 1 --2- 172.21.15.164:0/2956046159 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x562a5fb38ce0 0x562a5fb3b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:51.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.396+0000 7f860bfe9740 4 client.16527 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:50:51.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.396+0000 7f860bfe9740 1 -- 172.21.15.164:0/2956046159 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a5fa90b90 con 0x562a5fb38ce0 2024-09-19T08:50:51.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.396+0000 7f86037fe640 1 --2- 172.21.15.164:0/2956046159 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x562a5fb38ce0 0x562a5fb3b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:51.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.396+0000 7f86037fe640 1 --2- 172.21.15.164:0/2956046159 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x562a5fb38ce0 0x562a5fb3b0d0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e40bfe1b0e8300dc server_cookie=cbec75727276ec43 in_seq=0 out_seq=0 2024-09-19T08:50:51.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.396+0000 7f85e2ffd640 10 client.16527 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:50:51.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.397+0000 7f85e2ffd640 1 -- 172.21.15.164:0/2956046159 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562a5fa90b90 con 0x562a5fb38ce0 2024-09-19T08:50:51.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.397+0000 7f85e2ffd640 10 client.16527 handle_command_reply: tid=0 2024-09-19T08:50:51.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.397+0000 7f860bfe9740 1 client.16527 shutdown 2024-09-19T08:50:51.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.397+0000 7f860bfe9740 1 -- 172.21.15.164:0/2956046159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a5fb29240 msgr2=0x562a5fb26b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:51.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.397+0000 7f860bfe9740 1 --2- 172.21.15.164:0/2956046159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a5fb29240 0x562a5fb26b90 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f85fc0050f0 tx=0x7f85fc012bc0 comp rx=0 tx=0).stop 2024-09-19T08:50:51.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.397+0000 7f860bfe9740 1 -- 172.21.15.164:0/2956046159 shutdown_connections 2024-09-19T08:50:51.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.397+0000 7f860bfe9740 1 --2- 172.21.15.164:0/2956046159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a5fb29240 0x562a5fb26b90 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:51.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.397+0000 7f860bfe9740 1 --2- 172.21.15.164:0/2956046159 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a5fb1e980 0x562a5fb26650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:51.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.397+0000 7f860bfe9740 1 --2- 172.21.15.164:0/2956046159 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x562a5fb38ce0 0x562a5fb3b0d0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:51.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.397+0000 7f860bfe9740 1 --2- 172.21.15.164:0/2956046159 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a5fb1e060 0x562a5fb26110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:51.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.397+0000 7f860bfe9740 1 -- 172.21.15.164:0/2956046159 >> 172.21.15.164:0/2956046159 conn(0x562a5fb1a5f0 msgr2=0x562a5fb20320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:51.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.397+0000 7f85e2ffd640 0 client.16527 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:50:51.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.397+0000 7f860bfe9740 1 -- 172.21.15.164:0/2956046159 shutdown_connections 2024-09-19T08:50:51.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.397+0000 7f860bfe9740 1 -- 172.21.15.164:0/2956046159 wait complete. 2024-09-19T08:50:51.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.397+0000 7f860bfe9740 20 client.16527 trim_cache size 0 max 16384 2024-09-19T08:50:51.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.414+0000 7f860bfe9740 1 Processor -- start 2024-09-19T08:50:51.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.414+0000 7f860bfe9740 1 -- start start 2024-09-19T08:50:51.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.415+0000 7f860bfe9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a5fb1e060 0x562a5fb20ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:51.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.415+0000 7f860bfe9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a5fb1e980 0x562a5fb19640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:51.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.415+0000 7f860bfe9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a5fb29240 0x562a5fb408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:51.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.415+0000 7f860bfe9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f86041af990 con 0x562a5fb1e980 2024-09-19T08:50:51.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.415+0000 7f860bfe9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f86041b1590 con 0x562a5fb29240 2024-09-19T08:50:51.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.415+0000 7f860bfe9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f86041b0060 con 0x562a5fb1e060 2024-09-19T08:50:51.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.415+0000 7f8602ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a5fb1e980 0x562a5fb19640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:51.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.415+0000 7f8602ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a5fb1e980 0x562a5fb19640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48628/0 (socket says 172.21.15.164:48628) 2024-09-19T08:50:51.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.415+0000 7f8602ffd640 1 -- 172.21.15.164:0/129934766 learned_addr learned my addr 172.21.15.164:0/129934766 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:51.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.415+0000 7f8602ffd640 1 -- 172.21.15.164:0/129934766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a5fb1e060 msgr2=0x562a5fb20ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:51.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.415+0000 7f86037fe640 1 --2- 172.21.15.164:0/129934766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a5fb1e060 0x562a5fb20ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:51.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.415+0000 7f8603fff640 1 --2- 172.21.15.164:0/129934766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a5fb29240 0x562a5fb408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:51.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.415+0000 7f8602ffd640 1 --2- 172.21.15.164:0/129934766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a5fb1e060 0x562a5fb20ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:51.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.415+0000 7f8602ffd640 1 -- 172.21.15.164:0/129934766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a5fb29240 msgr2=0x562a5fb408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:51.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.415+0000 7f8602ffd640 1 --2- 172.21.15.164:0/129934766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a5fb29240 0x562a5fb408b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:51.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.415+0000 7f8602ffd640 1 -- 172.21.15.164:0/129934766 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a5fb1b6a0 con 0x562a5fb1e980 2024-09-19T08:50:51.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.415+0000 7f8602ffd640 1 --2- 172.21.15.164:0/129934766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a5fb1e980 0x562a5fb19640 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f85f409be10 tx=0x7f85f409bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c00040f5a3b8e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:51.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.415+0000 7f85f97fa640 1 -- 172.21.15.164:0/129934766 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f401d040 con 0x562a5fb1e980 2024-09-19T08:50:51.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.415+0000 7f85f97fa640 1 -- 172.21.15.164:0/129934766 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85f409c810 con 0x562a5fb1e980 2024-09-19T08:50:51.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.415+0000 7f85f97fa640 1 -- 172.21.15.164:0/129934766 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f40a4d80 con 0x562a5fb1e980 2024-09-19T08:50:51.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.416+0000 7f860bfe9740 1 -- 172.21.15.164:0/129934766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a5fb1e980 msgr2=0x562a5fb19640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:51.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.416+0000 7f860bfe9740 1 --2- 172.21.15.164:0/129934766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a5fb1e980 0x562a5fb19640 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f85f409be10 tx=0x7f85f409bd10 comp rx=0 tx=0).stop 2024-09-19T08:50:51.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.416+0000 7f860bfe9740 1 -- 172.21.15.164:0/129934766 shutdown_connections 2024-09-19T08:50:51.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.416+0000 7f860bfe9740 1 --2- 172.21.15.164:0/129934766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a5fb29240 0x562a5fb408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:51.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.416+0000 7f860bfe9740 1 --2- 172.21.15.164:0/129934766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a5fb1e980 0x562a5fb19640 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:51.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.416+0000 7f860bfe9740 1 --2- 172.21.15.164:0/129934766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a5fb1e060 0x562a5fb20ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:51.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.416+0000 7f860bfe9740 1 -- 172.21.15.164:0/129934766 >> 172.21.15.164:0/129934766 conn(0x562a5fb1a5f0 msgr2=0x562a5fb18f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:51.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.416+0000 7f860bfe9740 1 -- 172.21.15.164:0/129934766 shutdown_connections 2024-09-19T08:50:51.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.416+0000 7f860bfe9740 1 -- 172.21.15.164:0/129934766 wait complete. 2024-09-19T08:50:51.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.416+0000 7f860bfe9740 1 -- start start 2024-09-19T08:50:51.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.417+0000 7f860bfe9740 1 Processor -- start 2024-09-19T08:50:51.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.417+0000 7f860bfe9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a5fb1e060 0x562a5fb2d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:51.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.417+0000 7f860bfe9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a5fb1e980 0x562a5fb2dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:51.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.417+0000 7f860bfe9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a5fb29240 0x562a5fba2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:51.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.417+0000 7f860bfe9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f86041b0e00 con 0x562a5fb1e060 2024-09-19T08:50:51.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.417+0000 7f860bfe9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f86041b0730 con 0x562a5fb29240 2024-09-19T08:50:51.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.417+0000 7f860bfe9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f86041ad970 con 0x562a5fb1e980 2024-09-19T08:50:51.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.417+0000 7f860bfe9740 10 client.0 _ll_register_callbacks cb 0x7f85fc005020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:50:51.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.417+0000 7f86037fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a5fb1e060 0x562a5fb2d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:51.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.417+0000 7f86037fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a5fb1e060 0x562a5fb2d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48634/0 (socket says 172.21.15.164:48634) 2024-09-19T08:50:51.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.417+0000 7f86037fe640 1 -- 172.21.15.164:0/941733845 learned_addr learned my addr 172.21.15.164:0/941733845 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:51.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.417+0000 7f8602ffd640 1 --2- 172.21.15.164:0/941733845 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a5fb1e980 0x562a5fb2dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:51.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.417+0000 7f8603fff640 1 --2- 172.21.15.164:0/941733845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a5fb29240 0x562a5fba2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:51.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.417+0000 7f86037fe640 1 -- 172.21.15.164:0/941733845 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a5fb1e980 msgr2=0x562a5fb2dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:51.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.417+0000 7f86037fe640 1 --2- 172.21.15.164:0/941733845 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a5fb1e980 0x562a5fb2dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:51.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.417+0000 7f86037fe640 1 -- 172.21.15.164:0/941733845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a5fb29240 msgr2=0x562a5fba2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:50:51.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.417+0000 7f86037fe640 1 --2- 172.21.15.164:0/941733845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a5fb29240 0x562a5fba2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:51.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.417+0000 7f86037fe640 1 -- 172.21.15.164:0/941733845 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a5fb1b280 con 0x562a5fb1e060 2024-09-19T08:50:51.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.417+0000 7f8603fff640 1 --2- 172.21.15.164:0/941733845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a5fb29240 0x562a5fba2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:50:51.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.418+0000 7f86037fe640 1 --2- 172.21.15.164:0/941733845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a5fb1e060 0x562a5fb2d590 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f85ec02a930 tx=0x7f85ec009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b7e6bda03cfc77 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:51.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.418+0000 7f85e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:50:51.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.418+0000 7f85e2ffd640 1 -- 172.21.15.164:0/941733845 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec035050 con 0x562a5fb1e060 2024-09-19T08:50:51.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.418+0000 7f85e2ffd640 1 -- 172.21.15.164:0/941733845 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f85ec004250 con 0x562a5fb1e060 2024-09-19T08:50:51.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.418+0000 7f860bfe9740 1 -- 172.21.15.164:0/941733845 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85f409c810 con 0x562a5fb1e060 2024-09-19T08:50:51.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.418+0000 7f85e2ffd640 1 -- 172.21.15.164:0/941733845 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f85ec004450 con 0x562a5fb1e060 2024-09-19T08:50:51.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.418+0000 7f860bfe9740 10 client.16533 fetch_fsmap learned FSMap version 30 2024-09-19T08:50:51.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.418+0000 7f860bfe9740 1 -- 172.21.15.164:0/941733845 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562a5fba3e40 con 0x562a5fb1e060 2024-09-19T08:50:51.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.418+0000 7f85e2ffd640 1 -- 172.21.15.164:0/941733845 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85f409c810 con 0x562a5fb1e060 2024-09-19T08:50:51.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.418+0000 7f85e2ffd640 1 -- 172.21.15.164:0/941733845 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f85ec05a030 con 0x562a5fb1e060 2024-09-19T08:50:51.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.418+0000 7f860bfe9740 10 client.16533 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:50:51.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.418+0000 7f860bfe9740 10 client.16533 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:50:51.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.418+0000 7f860bfe9740 20 client.16533 populate_metadata read hostname 'smithi164' 2024-09-19T08:50:51.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.418+0000 7f860bfe9740 1 --2- 172.21.15.164:0/941733845 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x562a5fba4650 0x562a5fba6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:51.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.418+0000 7f860bfe9740 4 client.16533 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ea4bdac6", "release": true, "await": true}] 2024-09-19T08:50:51.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.418+0000 7f860bfe9740 1 -- 172.21.15.164:0/941733845 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ea4bdac6", "release": true, "await": true}) -- 0x562a5fb21d30 con 0x562a5fba4650 2024-09-19T08:50:51.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.419+0000 7f8602ffd640 1 --2- 172.21.15.164:0/941733845 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x562a5fba4650 0x562a5fba6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:51.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.419+0000 7f8602ffd640 1 --2- 172.21.15.164:0/941733845 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x562a5fba4650 0x562a5fba6a70 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e2f741a622275f5 server_cookie=5cde53d29d2a99da in_seq=0 out_seq=0 2024-09-19T08:50:51.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.419+0000 7f85e2ffd640 10 client.16533 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:50:51.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.423+0000 7f85e2ffd640 1 -- 172.21.15.164:0/941733845 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562a5fb21d30 con 0x562a5fba4650 2024-09-19T08:50:51.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.423+0000 7f85e2ffd640 10 client.16533 handle_command_reply: tid=0 2024-09-19T08:50:51.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.423+0000 7f860bfe9740 1 client.16533 shutdown 2024-09-19T08:50:51.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.425+0000 7f860bfe9740 1 -- 172.21.15.164:0/941733845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a5fb1e060 msgr2=0x562a5fb2d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:51.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.425+0000 7f860bfe9740 1 --2- 172.21.15.164:0/941733845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a5fb1e060 0x562a5fb2d590 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f85ec02a930 tx=0x7f85ec009260 comp rx=0 tx=0).stop 2024-09-19T08:50:51.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.425+0000 7f860bfe9740 1 -- 172.21.15.164:0/941733845 shutdown_connections 2024-09-19T08:50:51.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.425+0000 7f860bfe9740 1 --2- 172.21.15.164:0/941733845 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x562a5fba4650 0x562a5fba6a70 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:51.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.425+0000 7f860bfe9740 1 --2- 172.21.15.164:0/941733845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a5fb29240 0x562a5fba2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:51.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.425+0000 7f860bfe9740 1 --2- 172.21.15.164:0/941733845 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a5fb1e980 0x562a5fb2dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:51.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.425+0000 7f860bfe9740 1 --2- 172.21.15.164:0/941733845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a5fb1e060 0x562a5fb2d590 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:51.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.425+0000 7f860bfe9740 1 -- 172.21.15.164:0/941733845 >> 172.21.15.164:0/941733845 conn(0x562a5fb1a5f0 msgr2=0x562a5fb19750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:51.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.426+0000 7f85e2ffd640 0 client.16533 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:50:51.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.426+0000 7f860bfe9740 1 -- 172.21.15.164:0/941733845 shutdown_connections 2024-09-19T08:50:51.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.426+0000 7f860bfe9740 1 -- 172.21.15.164:0/941733845 wait complete. 2024-09-19T08:50:51.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.426+0000 7f860bfe9740 20 client.16533 trim_cache size 0 max 16384 2024-09-19T08:50:51.429 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:50:51.429 INFO:teuthology.orchestra.run.smithi164.stdout: "epoch": 30, 2024-09-19T08:50:51.429 INFO:teuthology.orchestra.run.smithi164.stdout: "leader": 24479, 2024-09-19T08:50:51.429 INFO:teuthology.orchestra.run.smithi164.stdout: "set_version": 20, 2024-09-19T08:50:51.429 INFO:teuthology.orchestra.run.smithi164.stdout: "sets": { 2024-09-19T08:50:51.429 INFO:teuthology.orchestra.run.smithi164.stdout: "ea4bdac6": { 2024-09-19T08:50:51.429 INFO:teuthology.orchestra.run.smithi164.stdout: "version": 20, 2024-09-19T08:50:51.429 INFO:teuthology.orchestra.run.smithi164.stdout: "age_ref": 0.0, 2024-09-19T08:50:51.429 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:50:51.429 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "RELEASED", 2024-09-19T08:50:51.429 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:50:51.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:50:51.430 INFO:teuthology.orchestra.run.smithi164.stdout: "timeout": 90.0, 2024-09-19T08:50:51.430 INFO:teuthology.orchestra.run.smithi164.stdout: "expiration": 149.8, 2024-09-19T08:50:51.430 INFO:teuthology.orchestra.run.smithi164.stdout: "members": { 2024-09-19T08:50:51.430 INFO:teuthology.orchestra.run.smithi164.stdout: "file:/": { 2024-09-19T08:50:51.430 INFO:teuthology.orchestra.run.smithi164.stdout: "excluded": false, 2024-09-19T08:50:51.430 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:50:51.430 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "RELEASED", 2024-09-19T08:50:51.430 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:50:51.430 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:50:51.430 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:50:51.430 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:50:51.430 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:50:51.430 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:50:51.431 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:50:51.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.427+0000 7f8609c10640 1 -- 172.21.15.164:0/3001233319 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f85f00a33b0 msgr2=0x7f85f00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:51.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.427+0000 7f8609c10640 1 --2- 172.21.15.164:0/3001233319 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f85f00a33b0 0x7f85f00a5870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f85ec005ab0 tx=0x7f85ec005a20 comp rx=0 tx=0).stop 2024-09-19T08:50:51.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.427+0000 7f8609c10640 1 -- 172.21.15.164:0/3001233319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8604102790 msgr2=0x7f8604074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:51.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.427+0000 7f8609c10640 1 --2- 172.21.15.164:0/3001233319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8604102790 0x7f8604074a30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f85f400c890 tx=0x7f85f400cd50 comp rx=0 tx=0).stop 2024-09-19T08:50:51.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.427+0000 7f8609c10640 1 -- 172.21.15.164:0/3001233319 shutdown_connections 2024-09-19T08:50:51.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.427+0000 7f8609c10640 1 --2- 172.21.15.164:0/3001233319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8604106f10 0x7f8604074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:51.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.427+0000 7f8609c10640 1 --2- 172.21.15.164:0/3001233319 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f85f00a33b0 0x7f85f00a5870 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:51.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.427+0000 7f8609c10640 1 --2- 172.21.15.164:0/3001233319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8604102790 0x7f8604074a30 secure :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f85f400c890 tx=0x7f85f400cd50 comp rx=0 tx=0).stop 2024-09-19T08:50:51.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.427+0000 7f8609c10640 1 --2- 172.21.15.164:0/3001233319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8604101de0 0x7f86040744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:51.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.427+0000 7f8609c10640 1 -- 172.21.15.164:0/3001233319 >> 172.21.15.164:0/3001233319 conn(0x7f86040fd8c0 msgr2=0x7f86040fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:51.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.428+0000 7f8609c10640 1 -- 172.21.15.164:0/3001233319 shutdown_connections 2024-09-19T08:50:51.432 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:51.428+0000 7f8609c10640 1 -- 172.21.15.164:0/3001233319 wait complete. 2024-09-19T08:50:51.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:51 smithi173 ceph-mon[35306]: pgmap v154: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 56 MiB/s wr, 392 op/s 2024-09-19T08:50:52.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:51 smithi164 ceph-mon[30928]: pgmap v154: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 56 MiB/s wr, 392 op/s 2024-09-19T08:50:52.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:51 smithi170 ceph-mon[35072]: pgmap v154: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 56 MiB/s wr, 392 op/s 2024-09-19T08:50:52.786 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:52 smithi164 ceph-mon[30928]: scrub summary: idle 2024-09-19T08:50:52.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:52 smithi173 ceph-mon[35306]: scrub summary: idle 2024-09-19T08:50:53.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:52 smithi170 ceph-mon[35072]: scrub summary: idle 2024-09-19T08:50:53.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:53 smithi173 ceph-mon[35306]: scrub summary: idle 2024-09-19T08:50:53.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:53 smithi173 ceph-mon[35306]: pgmap v155: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 51 MiB/s wr, 339 op/s 2024-09-19T08:50:54.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:53 smithi164 ceph-mon[30928]: scrub summary: idle 2024-09-19T08:50:54.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:53 smithi164 ceph-mon[30928]: pgmap v155: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 51 MiB/s wr, 339 op/s 2024-09-19T08:50:54.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:53 smithi170 ceph-mon[35072]: scrub summary: idle 2024-09-19T08:50:54.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:53 smithi170 ceph-mon[35072]: pgmap v155: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 51 MiB/s wr, 339 op/s 2024-09-19T08:50:54.547 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ea4bdac6', total seconds elapsed: 35.5 2024-09-19T08:50:54.547 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.8 2024-09-19T08:50:54.547 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:50:55.714 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:50:55.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:55 smithi173 ceph-mon[35306]: pgmap v156: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 86 MiB/s wr, 465 op/s 2024-09-19T08:50:55.914 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:55 smithi164 ceph-mon[30928]: pgmap v156: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 86 MiB/s wr, 465 op/s 2024-09-19T08:50:56.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:55 smithi170 ceph-mon[35072]: pgmap v156: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 86 MiB/s wr, 465 op/s 2024-09-19T08:50:57.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:56.999+0000 7f5b69ee0640 1 -- 172.21.15.164:0/1315162481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b6411b140 msgr2=0x7f5b6411d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:57.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:56.999+0000 7f5b69ee0640 1 --2- 172.21.15.164:0/1315162481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b6411b140 0x7f5b6411d530 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c009ca0 tx=0x7f5b5c02a8f0 comp rx=0 tx=0).stop 2024-09-19T08:50:57.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.000+0000 7f5b69ee0640 1 -- 172.21.15.164:0/1315162481 shutdown_connections 2024-09-19T08:50:57.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.000+0000 7f5b69ee0640 1 --2- 172.21.15.164:0/1315162481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b6411b140 0x7f5b6411d530 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:57.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.000+0000 7f5b69ee0640 1 --2- 172.21.15.164:0/1315162481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b64109830 0x7f5b64109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:57.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.000+0000 7f5b69ee0640 1 --2- 172.21.15.164:0/1315162481 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b64109450 0x7f5b64110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:57.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.000+0000 7f5b69ee0640 1 -- 172.21.15.164:0/1315162481 >> 172.21.15.164:0/1315162481 conn(0x7f5b6406e140 msgr2=0x7f5b6406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:57.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.003+0000 7f5b69ee0640 1 -- 172.21.15.164:0/1315162481 shutdown_connections 2024-09-19T08:50:57.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.007+0000 7f5b69ee0640 1 -- 172.21.15.164:0/1315162481 wait complete. 2024-09-19T08:50:57.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.008+0000 7f5b69ee0640 1 Processor -- start 2024-09-19T08:50:57.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.009+0000 7f5b69ee0640 1 -- start start 2024-09-19T08:50:57.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.010+0000 7f5b69ee0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b64109450 0x7f5b641a7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:57.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.010+0000 7f5b69ee0640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b64109830 0x7f5b641a78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:57.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.010+0000 7f5b69ee0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b6411b140 0x7f5b641abc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:57.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.010+0000 7f5b69ee0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5b64120e80 con 0x7f5b64109450 2024-09-19T08:50:57.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.010+0000 7f5b69ee0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5b64120d00 con 0x7f5b6411b140 2024-09-19T08:50:57.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.010+0000 7f5b69ee0640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5b64121000 con 0x7f5b64109830 2024-09-19T08:50:57.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.013+0000 7f5b63fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b6411b140 0x7f5b641abc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:57.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.015+0000 7f5b5bfff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b64109830 0x7f5b641a78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:57.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.015+0000 7f5b63fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b6411b140 0x7f5b641abc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:44908/0 (socket says 172.21.15.164:44908) 2024-09-19T08:50:57.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.015+0000 7f5b63fff640 1 -- 172.21.15.164:0/1323601029 learned_addr learned my addr 172.21.15.164:0/1323601029 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:50:57.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.019+0000 7f5b637fe640 1 --2- 172.21.15.164:0/1323601029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b64109450 0x7f5b641a7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:57.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.019+0000 7f5b63fff640 1 -- 172.21.15.164:0/1323601029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b64109830 msgr2=0x7f5b641a78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:57.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.019+0000 7f5b63fff640 1 --2- 172.21.15.164:0/1323601029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b64109830 0x7f5b641a78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:57.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.019+0000 7f5b63fff640 1 -- 172.21.15.164:0/1323601029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b64109450 msgr2=0x7f5b641a7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:57.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.019+0000 7f5b63fff640 1 --2- 172.21.15.164:0/1323601029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b64109450 0x7f5b641a7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:57.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.019+0000 7f5b63fff640 1 -- 172.21.15.164:0/1323601029 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b641ac350 con 0x7f5b6411b140 2024-09-19T08:50:57.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.025+0000 7f5b63fff640 1 --2- 172.21.15.164:0/1323601029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b6411b140 0x7f5b641abc50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c030da0 tx=0x7f5b5c030dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:57.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.030+0000 7f5b617fa640 1 -- 172.21.15.164:0/1323601029 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c003b70 con 0x7f5b6411b140 2024-09-19T08:50:57.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.030+0000 7f5b69ee0640 1 -- 172.21.15.164:0/1323601029 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b641ac5e0 con 0x7f5b6411b140 2024-09-19T08:50:57.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.030+0000 7f5b69ee0640 1 -- 172.21.15.164:0/1323601029 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b64110920 con 0x7f5b6411b140 2024-09-19T08:50:57.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.030+0000 7f5b617fa640 1 -- 172.21.15.164:0/1323601029 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b5c02d070 con 0x7f5b6411b140 2024-09-19T08:50:57.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.030+0000 7f5b617fa640 1 -- 172.21.15.164:0/1323601029 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c004600 con 0x7f5b6411b140 2024-09-19T08:50:57.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.036+0000 7f5b69ee0640 1 -- 172.21.15.164:0/1323601029 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b44005290 con 0x7f5b6411b140 2024-09-19T08:50:57.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.038+0000 7f5b617fa640 1 -- 172.21.15.164:0/1323601029 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f5b5c031480 con 0x7f5b6411b140 2024-09-19T08:50:57.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.039+0000 7f5b617fa640 1 --2- 172.21.15.164:0/1323601029 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f5b500a35c0 0x7f5b500a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:50:57.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.040+0000 7f5b617fa640 1 -- 172.21.15.164:0/1323601029 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f5b5c0f12b0 con 0x7f5b6411b140 2024-09-19T08:50:57.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.041+0000 7f5b617fa640 1 -- 172.21.15.164:0/1323601029 <== mon.2 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5b5c0bde30 con 0x7f5b6411b140 2024-09-19T08:50:57.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.043+0000 7f5b637fe640 1 --2- 172.21.15.164:0/1323601029 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f5b500a35c0 0x7f5b500a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:50:57.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.054+0000 7f5b637fe640 1 --2- 172.21.15.164:0/1323601029 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f5b500a35c0 0x7f5b500a5a80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5b54006fd0 tx=0x7f5b54008040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:50:57.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.529+0000 7f5b69ee0640 1 -- 172.21.15.164:0/1323601029 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b44005580 con 0x7f5b6411b140 2024-09-19T08:50:57.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.532+0000 7f5b617fa640 1 -- 172.21.15.164:0/1323601029 <== mon.2 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11991 (secure 0 0 0) 0x7f5b5c0bd730 con 0x7f5b6411b140 2024-09-19T08:50:57.533 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:50:57.533 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":30,"btime":"2024-09-19T08:50:17:326587+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:50:17.071374+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:50:57.533 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 30 2024-09-19T08:50:57.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.537+0000 7f5b69ee0640 1 -- 172.21.15.164:0/1323601029 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f5b500a35c0 msgr2=0x7f5b500a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:57.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.537+0000 7f5b69ee0640 1 --2- 172.21.15.164:0/1323601029 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f5b500a35c0 0x7f5b500a5a80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5b54006fd0 tx=0x7f5b54008040 comp rx=0 tx=0).stop 2024-09-19T08:50:57.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.537+0000 7f5b69ee0640 1 -- 172.21.15.164:0/1323601029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b6411b140 msgr2=0x7f5b641abc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:50:57.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.537+0000 7f5b69ee0640 1 --2- 172.21.15.164:0/1323601029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b6411b140 0x7f5b641abc50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c030da0 tx=0x7f5b5c030dd0 comp rx=0 tx=0).stop 2024-09-19T08:50:57.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.538+0000 7f5b69ee0640 1 -- 172.21.15.164:0/1323601029 shutdown_connections 2024-09-19T08:50:57.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.538+0000 7f5b69ee0640 1 --2- 172.21.15.164:0/1323601029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b6411b140 0x7f5b641abc50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:57.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.538+0000 7f5b69ee0640 1 --2- 172.21.15.164:0/1323601029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b64109830 0x7f5b641a78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:57.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.538+0000 7f5b69ee0640 1 --2- 172.21.15.164:0/1323601029 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f5b500a35c0 0x7f5b500a5a80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:57.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.538+0000 7f5b69ee0640 1 --2- 172.21.15.164:0/1323601029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b64109450 0x7f5b641a7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:50:57.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.538+0000 7f5b69ee0640 1 -- 172.21.15.164:0/1323601029 >> 172.21.15.164:0/1323601029 conn(0x7f5b6406e140 msgr2=0x7f5b6411be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:50:57.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.544+0000 7f5b69ee0640 1 -- 172.21.15.164:0/1323601029 shutdown_connections 2024-09-19T08:50:57.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:50:57.545+0000 7f5b69ee0640 1 -- 172.21.15.164:0/1323601029 wait complete. 2024-09-19T08:50:58.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:57 smithi164 ceph-mon[30928]: pgmap v157: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 204 MiB/s wr, 960 op/s 2024-09-19T08:50:58.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:57 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/1323601029' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:50:58.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:57 smithi170 ceph-mon[35072]: pgmap v157: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 204 MiB/s wr, 960 op/s 2024-09-19T08:50:58.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:57 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/1323601029' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:50:58.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:57 smithi173 ceph-mon[35306]: pgmap v157: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 204 MiB/s wr, 960 op/s 2024-09-19T08:50:58.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:57 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/1323601029' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:50:59.650 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0' 2024-09-19T08:50:59.651 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0 2024-09-19T08:51:00.035 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:50:59 smithi164 ceph-mon[30928]: pgmap v158: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 271 MiB/s wr, 1.15k op/s 2024-09-19T08:51:00.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:50:59 smithi170 ceph-mon[35072]: pgmap v158: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 271 MiB/s wr, 1.15k op/s 2024-09-19T08:51:00.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:50:59 smithi173 ceph-mon[35306]: pgmap v158: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 271 MiB/s wr, 1.15k op/s 2024-09-19T08:51:00.660 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:51:02.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:01 smithi164 ceph-mon[30928]: pgmap v159: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 469 MiB/s wr, 1.79k op/s 2024-09-19T08:51:02.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:01 smithi170 ceph-mon[35072]: pgmap v159: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 469 MiB/s wr, 1.79k op/s 2024-09-19T08:51:02.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:01 smithi173 ceph-mon[35306]: pgmap v159: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 469 MiB/s wr, 1.79k op/s 2024-09-19T08:51:02.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.045+0000 7f880633f640 1 -- 172.21.15.164:0/2695387477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8800101290 msgr2=0x7f880010d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:02.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.045+0000 7f880633f640 1 --2- 172.21.15.164:0/2695387477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8800101290 0x7f880010d7c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f87f400d910 tx=0x7f87f402e5c0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.045+0000 7f880633f640 1 -- 172.21.15.164:0/2695387477 shutdown_connections 2024-09-19T08:51:02.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.045+0000 7f880633f640 1 --2- 172.21.15.164:0/2695387477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8800101290 0x7f880010d7c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.045+0000 7f880633f640 1 --2- 172.21.15.164:0/2695387477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88001008f0 0x7f8800100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.045+0000 7f880633f640 1 --2- 172.21.15.164:0/2695387477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88001068f0 0x7f8800106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.045+0000 7f880633f640 1 -- 172.21.15.164:0/2695387477 >> 172.21.15.164:0/2695387477 conn(0x7f88000fc630 msgr2=0x7f88000fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:02.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.049+0000 7f880633f640 1 -- 172.21.15.164:0/2695387477 shutdown_connections 2024-09-19T08:51:02.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.049+0000 7f880633f640 1 -- 172.21.15.164:0/2695387477 wait complete. 2024-09-19T08:51:02.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.050+0000 7f880633f640 1 Processor -- start 2024-09-19T08:51:02.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.054+0000 7f880633f640 1 -- start start 2024-09-19T08:51:02.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.054+0000 7f880633f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88001008f0 0x7f88001a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:02.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.054+0000 7f880633f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8800101290 0x7f88001a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:02.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.054+0000 7f880633f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88001068f0 0x7f880019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:02.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.055+0000 7f880633f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f880010fca0 con 0x7f88001068f0 2024-09-19T08:51:02.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.055+0000 7f880633f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f880010fb20 con 0x7f8800101290 2024-09-19T08:51:02.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.055+0000 7f880633f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f880010fe20 con 0x7f88001008f0 2024-09-19T08:51:02.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.057+0000 7f87ff7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8800101290 0x7f88001a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:02.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.057+0000 7f87ff7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8800101290 0x7f88001a4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:44922/0 (socket says 172.21.15.164:44922) 2024-09-19T08:51:02.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.057+0000 7f87ff7fe640 1 -- 172.21.15.164:0/1572444977 learned_addr learned my addr 172.21.15.164:0/1572444977 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:02.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.058+0000 7f88048dc640 1 --2- 172.21.15.164:0/1572444977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88001068f0 0x7f880019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:02.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.058+0000 7f87fffff640 1 --2- 172.21.15.164:0/1572444977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88001008f0 0x7f88001a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:02.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.058+0000 7f87ff7fe640 1 -- 172.21.15.164:0/1572444977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88001008f0 msgr2=0x7f88001a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:02.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.058+0000 7f87ff7fe640 1 --2- 172.21.15.164:0/1572444977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88001008f0 0x7f88001a46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.058+0000 7f87ff7fe640 1 -- 172.21.15.164:0/1572444977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88001068f0 msgr2=0x7f880019e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:02.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.058+0000 7f87ff7fe640 1 --2- 172.21.15.164:0/1572444977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88001068f0 0x7f880019e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.058+0000 7f87ff7fe640 1 -- 172.21.15.164:0/1572444977 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f880019efc0 con 0x7f8800101290 2024-09-19T08:51:02.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.059+0000 7f87ff7fe640 1 --2- 172.21.15.164:0/1572444977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8800101290 0x7f88001a4be0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f87f000b4a0 tx=0x7f87f000b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:02.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.060+0000 7f87fd7fa640 1 -- 172.21.15.164:0/1572444977 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f0014070 con 0x7f8800101290 2024-09-19T08:51:02.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.060+0000 7f87fd7fa640 1 -- 172.21.15.164:0/1572444977 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87f0010af0 con 0x7f8800101290 2024-09-19T08:51:02.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.060+0000 7f87fd7fa640 1 -- 172.21.15.164:0/1572444977 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f0017c40 con 0x7f8800101290 2024-09-19T08:51:02.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.060+0000 7f880633f640 1 -- 172.21.15.164:0/1572444977 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f880019f2b0 con 0x7f8800101290 2024-09-19T08:51:02.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.060+0000 7f880633f640 1 -- 172.21.15.164:0/1572444977 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88001ab500 con 0x7f8800101290 2024-09-19T08:51:02.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.066+0000 7f8808718740 1 Processor -- start 2024-09-19T08:51:02.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.067+0000 7f8808718740 1 -- start start 2024-09-19T08:51:02.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.067+0000 7f8808718740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e6f53900 0x55d8e6f53ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:02.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.067+0000 7f8808718740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e6f54220 0x55d8e6f5e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:02.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.067+0000 7f8808718740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e6f5eae0 0x55d8e6f60ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:02.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.067+0000 7f8808718740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d8e6eb92a0 con 0x55d8e6f53900 2024-09-19T08:51:02.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.067+0000 7f8808718740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d8e6e60b10 con 0x55d8e6f54220 2024-09-19T08:51:02.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.067+0000 7f8808718740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d8e6eb5260 con 0x55d8e6f5eae0 2024-09-19T08:51:02.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.068+0000 7f88048dc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e6f53900 0x55d8e6f53ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:02.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.069+0000 7f87fffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e6f54220 0x55d8e6f5e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:02.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.069+0000 7f87fffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e6f54220 0x55d8e6f5e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:44932/0 (socket says 172.21.15.164:44932) 2024-09-19T08:51:02.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.069+0000 7f87fffff640 1 -- 172.21.15.164:0/4091671612 learned_addr learned my addr 172.21.15.164:0/4091671612 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:02.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.071+0000 7f87fffff640 1 -- 172.21.15.164:0/4091671612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e6f5eae0 msgr2=0x55d8e6f60ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:51:02.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.071+0000 7f87fffff640 1 --2- 172.21.15.164:0/4091671612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e6f5eae0 0x55d8e6f60ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.071+0000 7f87fffff640 1 -- 172.21.15.164:0/4091671612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e6f53900 msgr2=0x55d8e6f53ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:02.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.071+0000 7f87fffff640 1 --2- 172.21.15.164:0/4091671612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e6f53900 0x55d8e6f53ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.071+0000 7f87fffff640 1 -- 172.21.15.164:0/4091671612 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8e6c2ab90 con 0x55d8e6f54220 2024-09-19T08:51:02.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.071+0000 7f88048dc640 1 --2- 172.21.15.164:0/4091671612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e6f53900 0x55d8e6f53ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:51:02.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.072+0000 7f87fffff640 1 --2- 172.21.15.164:0/4091671612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e6f54220 0x55d8e6f5e5a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f87f8007e10 tx=0x7f87f800c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a68b0178c5f1ce5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:02.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.072+0000 7f87e67fc640 1 -- 172.21.15.164:0/4091671612 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f8015020 con 0x55d8e6f54220 2024-09-19T08:51:02.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.072+0000 7f87fd7fa640 1 -- 172.21.15.164:0/1572444977 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f87f0017070 con 0x7f8800101290 2024-09-19T08:51:02.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.073+0000 7f87e67fc640 1 -- 172.21.15.164:0/4091671612 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87f8011b10 con 0x55d8e6f54220 2024-09-19T08:51:02.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.074+0000 7f8808718740 1 -- 172.21.15.164:0/4091671612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e6f54220 msgr2=0x55d8e6f5e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:02.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.074+0000 7f8808718740 1 --2- 172.21.15.164:0/4091671612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e6f54220 0x55d8e6f5e5a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f87f8007e10 tx=0x7f87f800c560 comp rx=0 tx=0).stop 2024-09-19T08:51:02.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.075+0000 7f8808718740 1 -- 172.21.15.164:0/4091671612 shutdown_connections 2024-09-19T08:51:02.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.075+0000 7f8808718740 1 --2- 172.21.15.164:0/4091671612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e6f5eae0 0x55d8e6f60ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.075+0000 7f8808718740 1 --2- 172.21.15.164:0/4091671612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e6f54220 0x55d8e6f5e5a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.075+0000 7f8808718740 1 --2- 172.21.15.164:0/4091671612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e6f53900 0x55d8e6f53ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.075+0000 7f8808718740 1 -- 172.21.15.164:0/4091671612 >> 172.21.15.164:0/4091671612 conn(0x55d8e6f4fe90 msgr2=0x55d8e6f50270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:02.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.079+0000 7f87fd7fa640 1 --2- 172.21.15.164:0/1572444977 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f87e80a3480 0x7f87e80a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:02.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.080+0000 7f8808718740 1 -- 172.21.15.164:0/4091671612 shutdown_connections 2024-09-19T08:51:02.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.080+0000 7f87fd7fa640 1 -- 172.21.15.164:0/1572444977 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f87f0010670 con 0x7f8800101290 2024-09-19T08:51:02.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.081+0000 7f87fffff640 1 --2- 172.21.15.164:0/1572444977 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f87e80a3480 0x7f87e80a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:02.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.081+0000 7f8808718740 1 -- 172.21.15.164:0/4091671612 wait complete. 2024-09-19T08:51:02.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.082+0000 7f8808718740 1 -- start start 2024-09-19T08:51:02.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.082+0000 7f8808718740 1 Processor -- start 2024-09-19T08:51:02.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.083+0000 7f8808718740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e6f53900 0x55d8e6f5b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:02.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.083+0000 7f8808718740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e6f5e640 0x55d8e6f5be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:02.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.083+0000 7f8808718740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e6f5c480 0x55d8e6f6ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:02.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.083+0000 7f8808718740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d8e6f61750 con 0x55d8e6f5c480 2024-09-19T08:51:02.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.083+0000 7f8808718740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d8e6e416e0 con 0x55d8e6f53900 2024-09-19T08:51:02.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.083+0000 7f8808718740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d8e6f618d0 con 0x55d8e6f5e640 2024-09-19T08:51:02.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.083+0000 7f8808718740 10 client.0 _ll_register_callbacks cb 0x55d8e6e65a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:51:02.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.085+0000 7f87fffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e6f53900 0x55d8e6f5b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:02.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.085+0000 7f88048dc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e6f5c480 0x55d8e6f6ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:02.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.085+0000 7f88048dc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e6f5c480 0x55d8e6f6ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45192/0 (socket says 172.21.15.164:45192) 2024-09-19T08:51:02.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.085+0000 7f88048dc640 1 -- 172.21.15.164:0/1848767058 learned_addr learned my addr 172.21.15.164:0/1848767058 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:02.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.087+0000 7f88048dc640 1 -- 172.21.15.164:0/1848767058 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e6f5e640 msgr2=0x55d8e6f5be30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:51:02.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.087+0000 7f88048dc640 1 --2- 172.21.15.164:0/1848767058 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e6f5e640 0x55d8e6f5be30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.087+0000 7f88048dc640 1 -- 172.21.15.164:0/1848767058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e6f53900 msgr2=0x55d8e6f5b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:02.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.087+0000 7f88048dc640 1 --2- 172.21.15.164:0/1848767058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e6f53900 0x55d8e6f5b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.087+0000 7f88048dc640 1 -- 172.21.15.164:0/1848767058 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8e6f7c380 con 0x55d8e6f5c480 2024-09-19T08:51:02.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.087+0000 7f87fffff640 1 --2- 172.21.15.164:0/1572444977 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f87e80a3480 0x7f87e80a5940 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f87f8007260 tx=0x7f87f8010fc0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:02.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.088+0000 7f88048dc640 1 --2- 172.21.15.164:0/1848767058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e6f5c480 0x55d8e6f6ac10 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f87f4037040 tx=0x7f87f400c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c216cde6c5f7cfb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:02.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.088+0000 7f87df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:51:02.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.089+0000 7f87df7fe640 1 -- 172.21.15.164:0/1848767058 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f4004450 con 0x55d8e6f5c480 2024-09-19T08:51:02.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.089+0000 7f8808718740 1 -- 172.21.15.164:0/1848767058 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d8e6ec69d0 con 0x55d8e6f5c480 2024-09-19T08:51:02.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.089+0000 7f87df7fe640 1 -- 172.21.15.164:0/1848767058 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f87f400c980 con 0x55d8e6f5c480 2024-09-19T08:51:02.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.099+0000 7f87df7fe640 1 -- 172.21.15.164:0/1848767058 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f87f400cb80 con 0x55d8e6f5c480 2024-09-19T08:51:02.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.100+0000 7f8808718740 10 client.16548 fetch_fsmap learned FSMap version 30 2024-09-19T08:51:02.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.100+0000 7f8808718740 1 -- 172.21.15.164:0/1848767058 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d8e6f7ed80 con 0x55d8e6f5c480 2024-09-19T08:51:02.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.106+0000 7f87df7fe640 1 -- 172.21.15.164:0/1848767058 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d8e6ec69d0 con 0x55d8e6f5c480 2024-09-19T08:51:02.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.106+0000 7f87df7fe640 1 -- 172.21.15.164:0/1848767058 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f87f4050020 con 0x55d8e6f5c480 2024-09-19T08:51:02.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.106+0000 7f8808718740 10 client.16548 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:51:02.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.106+0000 7f8808718740 10 client.16548 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:51:02.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.106+0000 7f8808718740 20 client.16548 populate_metadata read hostname 'smithi164' 2024-09-19T08:51:02.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.106+0000 7f8808718740 1 --2- 172.21.15.164:0/1848767058 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55d8e6f806e0 0x55d8e6f82b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:02.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.106+0000 7f8808718740 4 client.16548 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:51:02.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.106+0000 7f8808718740 1 -- 172.21.15.164:0/1848767058 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8e6ec0d20 con 0x55d8e6f806e0 2024-09-19T08:51:02.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.109+0000 7f87fffff640 1 --2- 172.21.15.164:0/1848767058 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55d8e6f806e0 0x55d8e6f82b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:02.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.117+0000 7f87fffff640 1 --2- 172.21.15.164:0/1848767058 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55d8e6f806e0 0x55d8e6f82b20 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=873f81c017c317c6 server_cookie=a81303eed084e808 in_seq=0 out_seq=0 2024-09-19T08:51:02.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.117+0000 7f87df7fe640 10 client.16548 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:51:02.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.125+0000 7f87df7fe640 1 -- 172.21.15.164:0/1848767058 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d8e6ec0d20 con 0x55d8e6f806e0 2024-09-19T08:51:02.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.125+0000 7f87df7fe640 10 client.16548 handle_command_reply: tid=0 2024-09-19T08:51:02.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.125+0000 7f8808718740 1 client.16548 shutdown 2024-09-19T08:51:02.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.126+0000 7f8808718740 1 -- 172.21.15.164:0/1848767058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e6f5c480 msgr2=0x55d8e6f6ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:02.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.126+0000 7f8808718740 1 --2- 172.21.15.164:0/1848767058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e6f5c480 0x55d8e6f6ac10 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f87f4037040 tx=0x7f87f400c6d0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.126+0000 7f8808718740 1 -- 172.21.15.164:0/1848767058 shutdown_connections 2024-09-19T08:51:02.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.126+0000 7f8808718740 1 --2- 172.21.15.164:0/1848767058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e6f5c480 0x55d8e6f6ac10 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.126+0000 7f8808718740 1 --2- 172.21.15.164:0/1848767058 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e6f5e640 0x55d8e6f5be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.126+0000 7f8808718740 1 --2- 172.21.15.164:0/1848767058 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55d8e6f806e0 0x55d8e6f82b20 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.126+0000 7f8808718740 1 --2- 172.21.15.164:0/1848767058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e6f53900 0x55d8e6f5b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.126+0000 7f8808718740 1 -- 172.21.15.164:0/1848767058 >> 172.21.15.164:0/1848767058 conn(0x55d8e6f4fe90 msgr2=0x55d8e6f553a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:02.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.127+0000 7f87df7fe640 0 client.16548 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:51:02.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.132+0000 7f8808718740 1 -- 172.21.15.164:0/1848767058 shutdown_connections 2024-09-19T08:51:02.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.133+0000 7f8808718740 1 -- 172.21.15.164:0/1848767058 wait complete. 2024-09-19T08:51:02.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.133+0000 7f8808718740 20 client.16548 trim_cache size 0 max 16384 2024-09-19T08:51:02.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.181+0000 7f8808718740 1 Processor -- start 2024-09-19T08:51:02.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.182+0000 7f8808718740 1 -- start start 2024-09-19T08:51:02.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.182+0000 7f8808718740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e6f53900 0x55d8e6f6aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:02.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.183+0000 7f8808718740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e6f5c480 0x55d8e6f5a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:02.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.183+0000 7f8808718740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e6f5e640 0x55d8e6f5a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:02.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.183+0000 7f8808718740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f88001ab730 con 0x55d8e6f53900 2024-09-19T08:51:02.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.183+0000 7f8808718740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f88001ad1f0 con 0x55d8e6f5e640 2024-09-19T08:51:02.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.183+0000 7f8808718740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f88001abcc0 con 0x55d8e6f5c480 2024-09-19T08:51:02.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.183+0000 7f88048dc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e6f5e640 0x55d8e6f5a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:02.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.183+0000 7f88048dc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e6f5e640 0x55d8e6f5a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:44944/0 (socket says 172.21.15.164:44944) 2024-09-19T08:51:02.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.183+0000 7f88048dc640 1 -- 172.21.15.164:0/4225389819 learned_addr learned my addr 172.21.15.164:0/4225389819 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:02.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.183+0000 7f87fffff640 1 --2- 172.21.15.164:0/4225389819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e6f53900 0x55d8e6f6aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:02.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.184+0000 7f87ff7fe640 1 --2- 172.21.15.164:0/4225389819 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e6f5c480 0x55d8e6f5a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:02.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.185+0000 7f87fffff640 1 -- 172.21.15.164:0/4225389819 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e6f5c480 msgr2=0x55d8e6f5a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:02.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.185+0000 7f87fffff640 1 --2- 172.21.15.164:0/4225389819 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e6f5c480 0x55d8e6f5a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.185+0000 7f87fffff640 1 -- 172.21.15.164:0/4225389819 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e6f5e640 msgr2=0x55d8e6f5a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:02.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.185+0000 7f87fffff640 1 --2- 172.21.15.164:0/4225389819 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e6f5e640 0x55d8e6f5a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.185+0000 7f87fffff640 1 -- 172.21.15.164:0/4225389819 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8e6f6af40 con 0x55d8e6f53900 2024-09-19T08:51:02.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.190+0000 7f88048dc640 1 --2- 172.21.15.164:0/4225389819 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e6f5e640 0x55d8e6f5a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:51:02.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.190+0000 7f87fffff640 1 --2- 172.21.15.164:0/4225389819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e6f53900 0x55d8e6f6aa00 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f87f8007de0 tx=0x7f87f8025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c496e5a60b2e663 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:02.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.191+0000 7f87e5ffb640 1 -- 172.21.15.164:0/4225389819 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f8036070 con 0x55d8e6f53900 2024-09-19T08:51:02.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.191+0000 7f87e5ffb640 1 -- 172.21.15.164:0/4225389819 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87f8003cd0 con 0x55d8e6f53900 2024-09-19T08:51:02.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.191+0000 7f87e5ffb640 1 -- 172.21.15.164:0/4225389819 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f8015040 con 0x55d8e6f53900 2024-09-19T08:51:02.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.192+0000 7f8808718740 1 -- 172.21.15.164:0/4225389819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e6f53900 msgr2=0x55d8e6f6aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:02.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.192+0000 7f8808718740 1 --2- 172.21.15.164:0/4225389819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e6f53900 0x55d8e6f6aa00 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f87f8007de0 tx=0x7f87f8025000 comp rx=0 tx=0).stop 2024-09-19T08:51:02.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.195+0000 7f8808718740 1 -- 172.21.15.164:0/4225389819 shutdown_connections 2024-09-19T08:51:02.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.195+0000 7f8808718740 1 --2- 172.21.15.164:0/4225389819 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e6f5e640 0x55d8e6f5a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.195+0000 7f8808718740 1 --2- 172.21.15.164:0/4225389819 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e6f5c480 0x55d8e6f5a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.195+0000 7f8808718740 1 --2- 172.21.15.164:0/4225389819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e6f53900 0x55d8e6f6aa00 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.196+0000 7f8808718740 1 -- 172.21.15.164:0/4225389819 >> 172.21.15.164:0/4225389819 conn(0x55d8e6f4fe90 msgr2=0x55d8e6f56ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:02.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.198+0000 7f8808718740 1 -- 172.21.15.164:0/4225389819 shutdown_connections 2024-09-19T08:51:02.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.198+0000 7f8808718740 1 -- 172.21.15.164:0/4225389819 wait complete. 2024-09-19T08:51:02.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.199+0000 7f8808718740 1 -- start start 2024-09-19T08:51:02.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.200+0000 7f8808718740 1 Processor -- start 2024-09-19T08:51:02.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.201+0000 7f8808718740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e6f53900 0x55d8e6f59450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:02.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.205+0000 7f8808718740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e6f5c480 0x55d8e6f66b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:02.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.205+0000 7f87fffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e6f53900 0x55d8e6f59450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:02.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.206+0000 7f87fffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e6f53900 0x55d8e6f59450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45218/0 (socket says 172.21.15.164:45218) 2024-09-19T08:51:02.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.206+0000 7f87fffff640 1 -- 172.21.15.164:0/3307695091 learned_addr learned my addr 172.21.15.164:0/3307695091 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:02.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.207+0000 7f8808718740 1 --2- 172.21.15.164:0/3307695091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e6f5e640 0x55d8e6f670b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:02.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.207+0000 7f87ff7fe640 1 --2- 172.21.15.164:0/3307695091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e6f5c480 0x55d8e6f66b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:02.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.207+0000 7f8808718740 1 -- 172.21.15.164:0/3307695091 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f88001aca60 con 0x55d8e6f53900 2024-09-19T08:51:02.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.208+0000 7f8808718740 1 -- 172.21.15.164:0/3307695091 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f88001ac390 con 0x55d8e6f5c480 2024-09-19T08:51:02.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.208+0000 7f8808718740 1 -- 172.21.15.164:0/3307695091 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8800072300 con 0x55d8e6f5e640 2024-09-19T08:51:02.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.208+0000 7f88048dc640 1 --2- 172.21.15.164:0/3307695091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e6f5e640 0x55d8e6f670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:02.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.209+0000 7f8808718740 10 client.0 _ll_register_callbacks cb 0x7f87f400d4b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:51:02.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.210+0000 7f87ff7fe640 1 -- 172.21.15.164:0/3307695091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e6f5e640 msgr2=0x55d8e6f670b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:02.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.210+0000 7f87ff7fe640 1 --2- 172.21.15.164:0/3307695091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e6f5e640 0x55d8e6f670b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.210+0000 7f87ff7fe640 1 -- 172.21.15.164:0/3307695091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e6f53900 msgr2=0x55d8e6f59450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:02.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.210+0000 7f87ff7fe640 1 --2- 172.21.15.164:0/3307695091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e6f53900 0x55d8e6f59450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.210+0000 7f87ff7fe640 1 -- 172.21.15.164:0/3307695091 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8e6f5f9d0 con 0x55d8e6f5c480 2024-09-19T08:51:02.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.213+0000 7f87ff7fe640 1 --2- 172.21.15.164:0/3307695091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e6f5c480 0x55d8e6f66b70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f87f00ccc90 tx=0x7f87f009bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44d6c217cb676e80 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:02.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.213+0000 7f87df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-19T08:51:02.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.214+0000 7f87df7fe640 1 -- 172.21.15.164:0/3307695091 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f001e070 con 0x55d8e6f5c480 2024-09-19T08:51:02.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.214+0000 7f8808718740 1 -- 172.21.15.164:0/3307695091 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87f8003cd0 con 0x55d8e6f5c480 2024-09-19T08:51:02.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.214+0000 7f87df7fe640 1 -- 172.21.15.164:0/3307695091 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f87f0017e00 con 0x55d8e6f5c480 2024-09-19T08:51:02.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.214+0000 7f87df7fe640 1 -- 172.21.15.164:0/3307695091 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f87f00a4490 con 0x55d8e6f5c480 2024-09-19T08:51:02.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.214+0000 7f8808718740 10 client.25931 fetch_fsmap learned FSMap version 30 2024-09-19T08:51:02.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.215+0000 7f8808718740 1 -- 172.21.15.164:0/3307695091 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55d8e6feb480 con 0x55d8e6f5c480 2024-09-19T08:51:02.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.215+0000 7f87df7fe640 1 -- 172.21.15.164:0/3307695091 <== mon.2 v2:172.21.15.170:3300/0 4 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f87f00cce00 con 0x55d8e6f5c480 2024-09-19T08:51:02.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.216+0000 7f8808718740 10 client.25931 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:51:02.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.216+0000 7f8808718740 10 client.25931 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:51:02.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.216+0000 7f8808718740 20 client.25931 populate_metadata read hostname 'smithi164' 2024-09-19T08:51:02.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.216+0000 7f8808718740 1 --2- 172.21.15.164:0/3307695091 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55d8e6febb30 0x55d8e6fedf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:02.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.216+0000 7f8808718740 4 client.25931 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}] 2024-09-19T08:51:02.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.216+0000 7f8808718740 1 -- 172.21.15.164:0/3307695091 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}) -- 0x7f87f8003cd0 con 0x55d8e6febb30 2024-09-19T08:51:02.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.217+0000 7f87fffff640 1 --2- 172.21.15.164:0/3307695091 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55d8e6febb30 0x55d8e6fedf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:02.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.221+0000 7f87fffff640 1 --2- 172.21.15.164:0/3307695091 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55d8e6febb30 0x55d8e6fedf50 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32f91c343573049 server_cookie=401e37a97f38da37 in_seq=0 out_seq=0 2024-09-19T08:51:02.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.222+0000 7f87df7fe640 10 client.25931 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:51:02.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.504+0000 7f87df7fe640 1 -- 172.21.15.164:0/3307695091 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f87f8003cd0 con 0x55d8e6febb30 2024-09-19T08:51:02.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.504+0000 7f87df7fe640 10 client.25931 handle_command_reply: tid=0 2024-09-19T08:51:02.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.504+0000 7f8808718740 1 client.25931 shutdown 2024-09-19T08:51:02.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.504+0000 7f8808718740 1 -- 172.21.15.164:0/3307695091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e6f5c480 msgr2=0x55d8e6f66b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:02.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.504+0000 7f8808718740 1 --2- 172.21.15.164:0/3307695091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e6f5c480 0x55d8e6f66b70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f87f00ccc90 tx=0x7f87f009bf20 comp rx=0 tx=0).stop 2024-09-19T08:51:02.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.504+0000 7f8808718740 1 -- 172.21.15.164:0/3307695091 shutdown_connections 2024-09-19T08:51:02.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.504+0000 7f8808718740 1 --2- 172.21.15.164:0/3307695091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e6f5e640 0x55d8e6f670b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.504+0000 7f8808718740 1 --2- 172.21.15.164:0/3307695091 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55d8e6febb30 0x55d8e6fedf50 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.504+0000 7f87df7fe640 0 client.25931 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:51:02.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.504+0000 7f8808718740 1 --2- 172.21.15.164:0/3307695091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e6f5c480 0x55d8e6f66b70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.504+0000 7f8808718740 1 --2- 172.21.15.164:0/3307695091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e6f53900 0x55d8e6f59450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.504+0000 7f8808718740 1 -- 172.21.15.164:0/3307695091 >> 172.21.15.164:0/3307695091 conn(0x55d8e6f4fe90 msgr2=0x55d8e6f5a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:02.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.505+0000 7f8808718740 1 -- 172.21.15.164:0/3307695091 shutdown_connections 2024-09-19T08:51:02.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.505+0000 7f8808718740 1 -- 172.21.15.164:0/3307695091 wait complete. 2024-09-19T08:51:02.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.505+0000 7f8808718740 20 client.25931 trim_cache size 0 max 16384 2024-09-19T08:51:02.507 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:51:02.507 INFO:teuthology.orchestra.run.smithi164.stdout: "epoch": 30, 2024-09-19T08:51:02.508 INFO:teuthology.orchestra.run.smithi164.stdout: "leader": 24479, 2024-09-19T08:51:02.508 INFO:teuthology.orchestra.run.smithi164.stdout: "set_version": 22, 2024-09-19T08:51:02.508 INFO:teuthology.orchestra.run.smithi164.stdout: "sets": { 2024-09-19T08:51:02.508 INFO:teuthology.orchestra.run.smithi164.stdout: "21b3571c": { 2024-09-19T08:51:02.508 INFO:teuthology.orchestra.run.smithi164.stdout: "version": 22, 2024-09-19T08:51:02.508 INFO:teuthology.orchestra.run.smithi164.stdout: "age_ref": 0.0, 2024-09-19T08:51:02.508 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:51:02.508 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "QUIESCED", 2024-09-19T08:51:02.508 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:51:02.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:51:02.508 INFO:teuthology.orchestra.run.smithi164.stdout: "timeout": 90.0, 2024-09-19T08:51:02.508 INFO:teuthology.orchestra.run.smithi164.stdout: "expiration": 149.8, 2024-09-19T08:51:02.508 INFO:teuthology.orchestra.run.smithi164.stdout: "members": { 2024-09-19T08:51:02.509 INFO:teuthology.orchestra.run.smithi164.stdout: "file:/": { 2024-09-19T08:51:02.509 INFO:teuthology.orchestra.run.smithi164.stdout: "excluded": false, 2024-09-19T08:51:02.509 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:51:02.509 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "QUIESCED", 2024-09-19T08:51:02.509 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:51:02.509 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:51:02.509 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:51:02.509 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:51:02.509 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:51:02.509 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:51:02.509 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:51:02.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.506+0000 7f880633f640 1 -- 172.21.15.164:0/1572444977 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f87e80a3480 msgr2=0x7f87e80a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:02.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.506+0000 7f880633f640 1 --2- 172.21.15.164:0/1572444977 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f87e80a3480 0x7f87e80a5940 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f87f8007260 tx=0x7f87f8010fc0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.506+0000 7f880633f640 1 -- 172.21.15.164:0/1572444977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8800101290 msgr2=0x7f88001a4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:02.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.506+0000 7f880633f640 1 --2- 172.21.15.164:0/1572444977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8800101290 0x7f88001a4be0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f87f000b4a0 tx=0x7f87f000b960 comp rx=0 tx=0).stop 2024-09-19T08:51:02.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.506+0000 7f880633f640 1 -- 172.21.15.164:0/1572444977 shutdown_connections 2024-09-19T08:51:02.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.506+0000 7f880633f640 1 --2- 172.21.15.164:0/1572444977 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f87e80a3480 0x7f87e80a5940 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.506+0000 7f880633f640 1 --2- 172.21.15.164:0/1572444977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88001068f0 0x7f880019e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.506+0000 7f880633f640 1 --2- 172.21.15.164:0/1572444977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8800101290 0x7f88001a4be0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.506+0000 7f880633f640 1 --2- 172.21.15.164:0/1572444977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88001008f0 0x7f88001a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:02.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.506+0000 7f880633f640 1 -- 172.21.15.164:0/1572444977 >> 172.21.15.164:0/1572444977 conn(0x7f88000fc630 msgr2=0x7f88000fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:02.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.506+0000 7f880633f640 1 -- 172.21.15.164:0/1572444977 shutdown_connections 2024-09-19T08:51:02.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:02.506+0000 7f880633f640 1 -- 172.21.15.164:0/1572444977 wait complete. 2024-09-19T08:51:02.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:02 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:51:03.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:02 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:51:03.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:02 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:51:03.398 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '21b3571c', quiesce took 8.9 seconds. Will release after: 20.9 2024-09-19T08:51:04.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:03 smithi164 ceph-mon[30928]: pgmap v160: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 528 MiB/s wr, 2.04k op/s 2024-09-19T08:51:04.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:03 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:51:04.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:03 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:51:04.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:03 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:51:04.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:03 smithi170 ceph-mon[35072]: pgmap v160: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 528 MiB/s wr, 2.04k op/s 2024-09-19T08:51:04.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:03 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:51:04.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:03 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:51:04.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:03 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:51:04.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:03 smithi173 ceph-mon[35306]: pgmap v160: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 528 MiB/s wr, 2.04k op/s 2024-09-19T08:51:04.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:03 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:51:04.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:03 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:51:04.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:03 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:51:06.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:05 smithi164 ceph-mon[30928]: pgmap v161: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 607 MiB/s wr, 2.24k op/s 2024-09-19T08:51:06.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:05 smithi170 ceph-mon[35072]: pgmap v161: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 607 MiB/s wr, 2.24k op/s 2024-09-19T08:51:06.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:05 smithi173 ceph-mon[35306]: pgmap v161: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 607 MiB/s wr, 2.24k op/s 2024-09-19T08:51:08.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:07 smithi164 ceph-mon[30928]: pgmap v162: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 677 MiB/s wr, 2.40k op/s 2024-09-19T08:51:08.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:07 smithi170 ceph-mon[35072]: pgmap v162: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 677 MiB/s wr, 2.40k op/s 2024-09-19T08:51:08.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:07 smithi173 ceph-mon[35306]: pgmap v162: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 677 MiB/s wr, 2.40k op/s 2024-09-19T08:51:10.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:09 smithi164 ceph-mon[30928]: pgmap v163: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 560 MiB/s wr, 1.84k op/s 2024-09-19T08:51:10.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:09 smithi170 ceph-mon[35072]: pgmap v163: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 560 MiB/s wr, 1.84k op/s 2024-09-19T08:51:10.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:09 smithi173 ceph-mon[35306]: pgmap v163: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 560 MiB/s wr, 1.84k op/s 2024-09-19T08:51:12.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:11 smithi164 ceph-mon[30928]: pgmap v164: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 505 MiB/s wr, 1.53k op/s 2024-09-19T08:51:12.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:11 smithi170 ceph-mon[35072]: pgmap v164: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 505 MiB/s wr, 1.53k op/s 2024-09-19T08:51:12.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:11 smithi173 ceph-mon[35306]: pgmap v164: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 505 MiB/s wr, 1.53k op/s 2024-09-19T08:51:14.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:13 smithi164 ceph-mon[30928]: pgmap v165: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 306 MiB/s wr, 871 op/s 2024-09-19T08:51:14.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:13 smithi170 ceph-mon[35072]: pgmap v165: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 306 MiB/s wr, 871 op/s 2024-09-19T08:51:14.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:13 smithi173 ceph-mon[35306]: pgmap v165: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 306 MiB/s wr, 871 op/s 2024-09-19T08:51:14.590 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:51:14.852 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:51:15.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.376+0000 7f5e990c7640 1 -- 172.21.15.164:0/316148399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e94101ba0 msgr2=0x7f5e9410d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:15.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.376+0000 7f5e990c7640 1 --2- 172.21.15.164:0/316148399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e94101ba0 0x7f5e9410d8f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f5e84009ca0 tx=0x7f5e8402a8c0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.376+0000 7f5e990c7640 1 -- 172.21.15.164:0/316148399 shutdown_connections 2024-09-19T08:51:15.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.376+0000 7f5e990c7640 1 --2- 172.21.15.164:0/316148399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e94101ba0 0x7f5e9410d8f0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.376+0000 7f5e990c7640 1 --2- 172.21.15.164:0/316148399 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e94101220 0x7f5e94101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.376+0000 7f5e990c7640 1 --2- 172.21.15.164:0/316148399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e94105a70 0x7f5e94105e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.376+0000 7f5e990c7640 1 -- 172.21.15.164:0/316148399 >> 172.21.15.164:0/316148399 conn(0x7f5e940fbee0 msgr2=0x7f5e940fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:15.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.376+0000 7f5e990c7640 1 -- 172.21.15.164:0/316148399 shutdown_connections 2024-09-19T08:51:15.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.376+0000 7f5e990c7640 1 -- 172.21.15.164:0/316148399 wait complete. 2024-09-19T08:51:15.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.377+0000 7f5e990c7640 1 Processor -- start 2024-09-19T08:51:15.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.377+0000 7f5e990c7640 1 -- start start 2024-09-19T08:51:15.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.377+0000 7f5e990c7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e94101220 0x7f5e94198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:15.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.377+0000 7f5e990c7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e94101ba0 0x7f5e94198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:15.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.377+0000 7f5e990c7640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e94105a70 0x7f5e9419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:15.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.377+0000 7f5e990c7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5e94111010 con 0x7f5e94101ba0 2024-09-19T08:51:15.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.377+0000 7f5e990c7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5e94110e90 con 0x7f5e94101220 2024-09-19T08:51:15.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.377+0000 7f5e990c7640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5e94111190 con 0x7f5e94105a70 2024-09-19T08:51:15.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.377+0000 7f5e9259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e94101ba0 0x7f5e94198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:15.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.377+0000 7f5e9259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e94101ba0 0x7f5e94198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45036/0 (socket says 172.21.15.164:45036) 2024-09-19T08:51:15.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.377+0000 7f5e9259c640 1 -- 172.21.15.164:0/2904697270 learned_addr learned my addr 172.21.15.164:0/2904697270 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:15.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.377+0000 7f5e9359e640 1 --2- 172.21.15.164:0/2904697270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e94105a70 0x7f5e9419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:15.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.377+0000 7f5e92d9d640 1 --2- 172.21.15.164:0/2904697270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e94101220 0x7f5e94198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:15.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.377+0000 7f5e9259c640 1 -- 172.21.15.164:0/2904697270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e94105a70 msgr2=0x7f5e9419cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:15.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.377+0000 7f5e9259c640 1 --2- 172.21.15.164:0/2904697270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e94105a70 0x7f5e9419cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.378+0000 7f5e9259c640 1 -- 172.21.15.164:0/2904697270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e94101220 msgr2=0x7f5e94198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:15.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.378+0000 7f5e9259c640 1 --2- 172.21.15.164:0/2904697270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e94101220 0x7f5e94198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.378+0000 7f5e9259c640 1 -- 172.21.15.164:0/2904697270 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e9419d4d0 con 0x7f5e94101ba0 2024-09-19T08:51:15.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.378+0000 7f5e92d9d640 1 --2- 172.21.15.164:0/2904697270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e94101220 0x7f5e94198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:51:15.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.378+0000 7f5e9259c640 1 --2- 172.21.15.164:0/2904697270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e94101ba0 0x7f5e94198a60 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00b4a0 tx=0x7f5e7c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:15.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.378+0000 7f5e8bfff640 1 -- 172.21.15.164:0/2904697270 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c014070 con 0x7f5e94101ba0 2024-09-19T08:51:15.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.378+0000 7f5e8bfff640 1 -- 172.21.15.164:0/2904697270 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e7c010af0 con 0x7f5e94101ba0 2024-09-19T08:51:15.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.378+0000 7f5e990c7640 1 -- 172.21.15.164:0/2904697270 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e9419d7c0 con 0x7f5e94101ba0 2024-09-19T08:51:15.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.378+0000 7f5e8bfff640 1 -- 172.21.15.164:0/2904697270 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c017d10 con 0x7f5e94101ba0 2024-09-19T08:51:15.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.378+0000 7f5e990c7640 1 -- 172.21.15.164:0/2904697270 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e941a50a0 con 0x7f5e94101ba0 2024-09-19T08:51:15.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.380+0000 7f5e8bfff640 1 -- 172.21.15.164:0/2904697270 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f5e7c017070 con 0x7f5e94101ba0 2024-09-19T08:51:15.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.381+0000 7f5e9b4a0740 1 Processor -- start 2024-09-19T08:51:15.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.381+0000 7f5e9b4a0740 1 -- start start 2024-09-19T08:51:15.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.381+0000 7f5e9b4a0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607e38493a0 0x5607e3849780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:15.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.381+0000 7f5e9b4a0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607e3849cc0 0x5607e3858550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:15.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.381+0000 7f5e9b4a0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e3858a90 0x5607e385ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:15.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.381+0000 7f5e9b4a0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5607e37af2e0 con 0x5607e38493a0 2024-09-19T08:51:15.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.381+0000 7f5e9b4a0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5607e3756ce0 con 0x5607e3849cc0 2024-09-19T08:51:15.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.381+0000 7f5e9b4a0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5607e37ab2a0 con 0x5607e3858a90 2024-09-19T08:51:15.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.381+0000 7f5e8bfff640 1 --2- 172.21.15.164:0/2904697270 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f5e800a33b0 0x7f5e800a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:15.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.381+0000 7f5e9359e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607e38493a0 0x5607e3849780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:15.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.381+0000 7f5e9359e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607e38493a0 0x5607e3849780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45038/0 (socket says 172.21.15.164:45038) 2024-09-19T08:51:15.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.381+0000 7f5e9359e640 1 -- 172.21.15.164:0/1523300458 learned_addr learned my addr 172.21.15.164:0/1523300458 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:15.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.381+0000 7f5e92d9d640 1 --2- 172.21.15.164:0/1523300458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607e3849cc0 0x5607e3858550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:15.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.381+0000 7f5e9359e640 1 --2- 172.21.15.164:0/2904697270 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f5e800a33b0 0x7f5e800a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:15.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.381+0000 7f5e8bfff640 1 -- 172.21.15.164:0/2904697270 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f5e7c0cef70 con 0x7f5e94101ba0 2024-09-19T08:51:15.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.382+0000 7f5e9259c640 1 --2- 172.21.15.164:0/1523300458 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e3858a90 0x5607e385ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:15.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.382+0000 7f5e92d9d640 1 -- 172.21.15.164:0/1523300458 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e3858a90 msgr2=0x5607e385ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:15.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.382+0000 7f5e92d9d640 1 --2- 172.21.15.164:0/1523300458 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e3858a90 0x5607e385ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.382+0000 7f5e92d9d640 1 -- 172.21.15.164:0/1523300458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607e38493a0 msgr2=0x5607e3849780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:15.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.382+0000 7f5e92d9d640 1 --2- 172.21.15.164:0/1523300458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607e38493a0 0x5607e3849780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.382+0000 7f5e92d9d640 1 -- 172.21.15.164:0/1523300458 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607e373b440 con 0x5607e3849cc0 2024-09-19T08:51:15.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.382+0000 7f5e9259c640 1 --2- 172.21.15.164:0/1523300458 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e3858a90 0x5607e385ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:51:15.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.382+0000 7f5e92d9d640 1 --2- 172.21.15.164:0/1523300458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607e3849cc0 0x5607e3858550 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c007760 tx=0x7f5e8c007c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f93ef18741cdf978 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:15.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.382+0000 7f5e9359e640 1 --2- 172.21.15.164:0/2904697270 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f5e800a33b0 0x7f5e800a5870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5e8402f540 tx=0x7f5e840095c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:15.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.382+0000 7f5e897fa640 1 -- 172.21.15.164:0/1523300458 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8c00d950 con 0x5607e3849cc0 2024-09-19T08:51:15.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.382+0000 7f5e897fa640 1 -- 172.21.15.164:0/1523300458 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e8c018070 con 0x5607e3849cc0 2024-09-19T08:51:15.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.382+0000 7f5e897fa640 1 -- 172.21.15.164:0/1523300458 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8c013e10 con 0x5607e3849cc0 2024-09-19T08:51:15.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.383+0000 7f5e9b4a0740 1 -- 172.21.15.164:0/1523300458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607e3849cc0 msgr2=0x5607e3858550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:15.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.383+0000 7f5e9b4a0740 1 --2- 172.21.15.164:0/1523300458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607e3849cc0 0x5607e3858550 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c007760 tx=0x7f5e8c007c20 comp rx=0 tx=0).stop 2024-09-19T08:51:15.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.383+0000 7f5e9b4a0740 1 -- 172.21.15.164:0/1523300458 shutdown_connections 2024-09-19T08:51:15.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.383+0000 7f5e9b4a0740 1 --2- 172.21.15.164:0/1523300458 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e3858a90 0x5607e385ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.383+0000 7f5e9b4a0740 1 --2- 172.21.15.164:0/1523300458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607e3849cc0 0x5607e3858550 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.383+0000 7f5e9b4a0740 1 --2- 172.21.15.164:0/1523300458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607e38493a0 0x5607e3849780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.383+0000 7f5e9b4a0740 1 -- 172.21.15.164:0/1523300458 >> 172.21.15.164:0/1523300458 conn(0x5607e3845a30 msgr2=0x5607e3845e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:15.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.383+0000 7f5e9b4a0740 1 -- 172.21.15.164:0/1523300458 shutdown_connections 2024-09-19T08:51:15.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.383+0000 7f5e9b4a0740 1 -- 172.21.15.164:0/1523300458 wait complete. 2024-09-19T08:51:15.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.383+0000 7f5e9b4a0740 1 -- start start 2024-09-19T08:51:15.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.383+0000 7f5e9b4a0740 1 Processor -- start 2024-09-19T08:51:15.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.384+0000 7f5e9b4a0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607e38493a0 0x5607e38583d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:15.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.384+0000 7f5e9b4a0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e3849cc0 0x5607e3855680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:15.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.384+0000 7f5e9b4a0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607e3858a90 0x5607e3855bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:15.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.384+0000 7f5e9b4a0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5607e385b6c0 con 0x5607e3858a90 2024-09-19T08:51:15.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.384+0000 7f5e9b4a0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5607e376a050 con 0x5607e38493a0 2024-09-19T08:51:15.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.384+0000 7f5e9b4a0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5607e385b840 con 0x5607e3849cc0 2024-09-19T08:51:15.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.384+0000 7f5e9b4a0740 10 client.0 _ll_register_callbacks cb 0x5607e37aa160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:51:15.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.384+0000 7f5e9359e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607e3858a90 0x5607e3855bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:15.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.384+0000 7f5e9359e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607e3858a90 0x5607e3855bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45042/0 (socket says 172.21.15.164:45042) 2024-09-19T08:51:15.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.384+0000 7f5e9359e640 1 -- 172.21.15.164:0/1644636504 learned_addr learned my addr 172.21.15.164:0/1644636504 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:15.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.384+0000 7f5e92d9d640 1 --2- 172.21.15.164:0/1644636504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607e38493a0 0x5607e38583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:15.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.384+0000 7f5e9259c640 1 --2- 172.21.15.164:0/1644636504 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e3849cc0 0x5607e3855680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:15.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.384+0000 7f5e9359e640 1 -- 172.21.15.164:0/1644636504 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e3849cc0 msgr2=0x5607e3855680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:15.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.384+0000 7f5e9359e640 1 --2- 172.21.15.164:0/1644636504 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e3849cc0 0x5607e3855680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.384+0000 7f5e9359e640 1 -- 172.21.15.164:0/1644636504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607e38493a0 msgr2=0x5607e38583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:15.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.384+0000 7f5e9359e640 1 --2- 172.21.15.164:0/1644636504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607e38493a0 0x5607e38583d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.384+0000 7f5e9359e640 1 -- 172.21.15.164:0/1644636504 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607e35203c0 con 0x5607e3858a90 2024-09-19T08:51:15.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.384+0000 7f5e9359e640 1 --2- 172.21.15.164:0/1644636504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607e3858a90 0x5607e3855bc0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f5e84006010 tx=0x7f5e8402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=435cf6818b04d1a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:15.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.384+0000 7f5e7a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:51:15.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.385+0000 7f5e7a7fc640 1 -- 172.21.15.164:0/1644636504 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e84030dd0 con 0x5607e3858a90 2024-09-19T08:51:15.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.385+0000 7f5e9b4a0740 1 -- 172.21.15.164:0/1644636504 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e8c018070 con 0x5607e3858a90 2024-09-19T08:51:15.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.385+0000 7f5e7a7fc640 1 -- 172.21.15.164:0/1644636504 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f5e8402f610 con 0x5607e3858a90 2024-09-19T08:51:15.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.385+0000 7f5e7a7fc640 1 -- 172.21.15.164:0/1644636504 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5e84031990 con 0x5607e3858a90 2024-09-19T08:51:15.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.385+0000 7f5e9b4a0740 10 client.16566 fetch_fsmap learned FSMap version 30 2024-09-19T08:51:15.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.385+0000 7f5e9b4a0740 1 -- 172.21.15.164:0/1644636504 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5607e3867020 con 0x5607e3858a90 2024-09-19T08:51:15.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.385+0000 7f5e7a7fc640 1 -- 172.21.15.164:0/1644636504 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e8c018070 con 0x5607e3858a90 2024-09-19T08:51:15.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.385+0000 7f5e7a7fc640 1 -- 172.21.15.164:0/1644636504 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f5e8405e050 con 0x5607e3858a90 2024-09-19T08:51:15.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.385+0000 7f5e9b4a0740 10 client.16566 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:51:15.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.385+0000 7f5e9b4a0740 10 client.16566 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:51:15.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.385+0000 7f5e9b4a0740 20 client.16566 populate_metadata read hostname 'smithi164' 2024-09-19T08:51:15.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.385+0000 7f5e9b4a0740 1 --2- 172.21.15.164:0/1644636504 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5607e3868480 0x5607e386a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:15.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.385+0000 7f5e9b4a0740 4 client.16566 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:51:15.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.385+0000 7f5e9b4a0740 1 -- 172.21.15.164:0/1644636504 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607e37bc850 con 0x5607e3868480 2024-09-19T08:51:15.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.385+0000 7f5e92d9d640 1 --2- 172.21.15.164:0/1644636504 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5607e3868480 0x5607e386a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:15.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.386+0000 7f5e92d9d640 1 --2- 172.21.15.164:0/1644636504 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5607e3868480 0x5607e386a900 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ebfe5f61625fdf3 server_cookie=df0776dfca007203 in_seq=0 out_seq=0 2024-09-19T08:51:15.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.386+0000 7f5e7a7fc640 10 client.16566 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:51:15.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.387+0000 7f5e7a7fc640 1 -- 172.21.15.164:0/1644636504 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5607e37bc850 con 0x5607e3868480 2024-09-19T08:51:15.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.387+0000 7f5e7a7fc640 10 client.16566 handle_command_reply: tid=0 2024-09-19T08:51:15.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.387+0000 7f5e9b4a0740 1 client.16566 shutdown 2024-09-19T08:51:15.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.387+0000 7f5e9b4a0740 1 -- 172.21.15.164:0/1644636504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607e3858a90 msgr2=0x5607e3855bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:15.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.387+0000 7f5e9b4a0740 1 --2- 172.21.15.164:0/1644636504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607e3858a90 0x5607e3855bc0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f5e84006010 tx=0x7f5e8402a8c0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.387+0000 7f5e9b4a0740 1 -- 172.21.15.164:0/1644636504 shutdown_connections 2024-09-19T08:51:15.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.387+0000 7f5e9b4a0740 1 --2- 172.21.15.164:0/1644636504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607e3858a90 0x5607e3855bc0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.387+0000 7f5e9b4a0740 1 --2- 172.21.15.164:0/1644636504 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e3849cc0 0x5607e3855680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.387+0000 7f5e9b4a0740 1 --2- 172.21.15.164:0/1644636504 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5607e3868480 0x5607e386a900 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.387+0000 7f5e9b4a0740 1 --2- 172.21.15.164:0/1644636504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607e38493a0 0x5607e38583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.387+0000 7f5e9b4a0740 1 -- 172.21.15.164:0/1644636504 >> 172.21.15.164:0/1644636504 conn(0x5607e3845a30 msgr2=0x5607e384b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:15.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.387+0000 7f5e7a7fc640 0 client.16566 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:51:15.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.387+0000 7f5e9b4a0740 1 -- 172.21.15.164:0/1644636504 shutdown_connections 2024-09-19T08:51:15.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.387+0000 7f5e9b4a0740 1 -- 172.21.15.164:0/1644636504 wait complete. 2024-09-19T08:51:15.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.387+0000 7f5e9b4a0740 20 client.16566 trim_cache size 0 max 16384 2024-09-19T08:51:15.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.404+0000 7f5e9b4a0740 1 Processor -- start 2024-09-19T08:51:15.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.404+0000 7f5e9b4a0740 1 -- start start 2024-09-19T08:51:15.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.404+0000 7f5e9b4a0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e38493a0 0x5607e3864960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:15.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.404+0000 7f5e9b4a0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607e3849cc0 0x5607e38575f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:15.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.404+0000 7f5e9b4a0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607e3858a90 0x5607e38701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:15.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.404+0000 7f5e9b4a0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5e941ab5b0 con 0x5607e3858a90 2024-09-19T08:51:15.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.404+0000 7f5e9b4a0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5e941ad1b0 con 0x5607e3849cc0 2024-09-19T08:51:15.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.404+0000 7f5e9b4a0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5e941abc80 con 0x5607e38493a0 2024-09-19T08:51:15.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.405+0000 7f5e9359e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607e3858a90 0x5607e38701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:15.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.405+0000 7f5e9359e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607e3858a90 0x5607e38701d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45046/0 (socket says 172.21.15.164:45046) 2024-09-19T08:51:15.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.405+0000 7f5e9359e640 1 -- 172.21.15.164:0/4171658424 learned_addr learned my addr 172.21.15.164:0/4171658424 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:15.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.405+0000 7f5e9259c640 1 --2- 172.21.15.164:0/4171658424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607e3849cc0 0x5607e38575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:15.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.405+0000 7f5e9359e640 1 -- 172.21.15.164:0/4171658424 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e38493a0 msgr2=0x5607e3864960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:15.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.405+0000 7f5e9359e640 1 --2- 172.21.15.164:0/4171658424 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e38493a0 0x5607e3864960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.405+0000 7f5e9359e640 1 -- 172.21.15.164:0/4171658424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607e3849cc0 msgr2=0x5607e38575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:15.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.405+0000 7f5e9359e640 1 --2- 172.21.15.164:0/4171658424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607e3849cc0 0x5607e38575f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.405+0000 7f5e9359e640 1 -- 172.21.15.164:0/4171658424 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607e3858740 con 0x5607e3858a90 2024-09-19T08:51:15.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.405+0000 7f5e9359e640 1 --2- 172.21.15.164:0/4171658424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607e3858a90 0x5607e38701d0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f5e84009ca0 tx=0x7f5e8402b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2618321964d640f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:15.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.405+0000 7f5e88ff9640 1 -- 172.21.15.164:0/4171658424 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8404cc30 con 0x5607e3858a90 2024-09-19T08:51:15.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.405+0000 7f5e88ff9640 1 -- 172.21.15.164:0/4171658424 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e84036070 con 0x5607e3858a90 2024-09-19T08:51:15.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.405+0000 7f5e88ff9640 1 -- 172.21.15.164:0/4171658424 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8402f870 con 0x5607e3858a90 2024-09-19T08:51:15.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.406+0000 7f5e9b4a0740 1 -- 172.21.15.164:0/4171658424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607e3858a90 msgr2=0x5607e38701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:15.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.406+0000 7f5e9b4a0740 1 --2- 172.21.15.164:0/4171658424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607e3858a90 0x5607e38701d0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f5e84009ca0 tx=0x7f5e8402b040 comp rx=0 tx=0).stop 2024-09-19T08:51:15.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.406+0000 7f5e9b4a0740 1 -- 172.21.15.164:0/4171658424 shutdown_connections 2024-09-19T08:51:15.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.406+0000 7f5e9b4a0740 1 --2- 172.21.15.164:0/4171658424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607e3858a90 0x5607e38701d0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.406+0000 7f5e9b4a0740 1 --2- 172.21.15.164:0/4171658424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607e3849cc0 0x5607e38575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.406+0000 7f5e9b4a0740 1 --2- 172.21.15.164:0/4171658424 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e38493a0 0x5607e3864960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.406+0000 7f5e9b4a0740 1 -- 172.21.15.164:0/4171658424 >> 172.21.15.164:0/4171658424 conn(0x5607e3845a30 msgr2=0x5607e3856ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:15.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.406+0000 7f5e9b4a0740 1 -- 172.21.15.164:0/4171658424 shutdown_connections 2024-09-19T08:51:15.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.406+0000 7f5e9b4a0740 1 -- 172.21.15.164:0/4171658424 wait complete. 2024-09-19T08:51:15.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.406+0000 7f5e9b4a0740 1 -- start start 2024-09-19T08:51:15.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.406+0000 7f5e9b4a0740 1 Processor -- start 2024-09-19T08:51:15.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.407+0000 7f5e9b4a0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607e38493a0 0x5607e385c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:15.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.407+0000 7f5e9b4a0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607e3849cc0 0x5607e385cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:15.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.407+0000 7f5e9b4a0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e3858a90 0x5607e385d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:15.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.407+0000 7f5e9b4a0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5e941aca20 con 0x5607e3849cc0 2024-09-19T08:51:15.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.407+0000 7f5e9b4a0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5e941ac350 con 0x5607e38493a0 2024-09-19T08:51:15.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.407+0000 7f5e9b4a0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5e941a95b0 con 0x5607e3858a90 2024-09-19T08:51:15.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.407+0000 7f5e9b4a0740 10 client.0 _ll_register_callbacks cb 0x7f5e84031850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:51:15.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.407+0000 7f5e9259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607e3849cc0 0x5607e385cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:15.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.407+0000 7f5e9259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607e3849cc0 0x5607e385cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45062/0 (socket says 172.21.15.164:45062) 2024-09-19T08:51:15.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.407+0000 7f5e9259c640 1 -- 172.21.15.164:0/1141866986 learned_addr learned my addr 172.21.15.164:0/1141866986 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:15.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.407+0000 7f5e92d9d640 1 --2- 172.21.15.164:0/1141866986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607e38493a0 0x5607e385c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:15.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.407+0000 7f5e9359e640 1 --2- 172.21.15.164:0/1141866986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e3858a90 0x5607e385d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:15.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.407+0000 7f5e9259c640 1 -- 172.21.15.164:0/1141866986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e3858a90 msgr2=0x5607e385d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:15.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.407+0000 7f5e9259c640 1 --2- 172.21.15.164:0/1141866986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e3858a90 0x5607e385d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.407+0000 7f5e9259c640 1 -- 172.21.15.164:0/1141866986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607e38493a0 msgr2=0x5607e385c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:15.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.407+0000 7f5e9259c640 1 --2- 172.21.15.164:0/1141866986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607e38493a0 0x5607e385c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.407+0000 7f5e9259c640 1 -- 172.21.15.164:0/1141866986 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607e38d21d0 con 0x5607e3849cc0 2024-09-19T08:51:15.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.407+0000 7f5e9259c640 1 --2- 172.21.15.164:0/1141866986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607e3849cc0 0x5607e385cde0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c097da0 tx=0x7f5e7c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee0df5bf74cdce2d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:15.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.407+0000 7f5e7a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:51:15.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.407+0000 7f5e7a7fc640 1 -- 172.21.15.164:0/1141866986 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c01e040 con 0x5607e3849cc0 2024-09-19T08:51:15.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.407+0000 7f5e7a7fc640 1 -- 172.21.15.164:0/1141866986 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f5e7c0cc9b0 con 0x5607e3849cc0 2024-09-19T08:51:15.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.407+0000 7f5e9b4a0740 1 -- 172.21.15.164:0/1141866986 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e84036070 con 0x5607e3849cc0 2024-09-19T08:51:15.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.407+0000 7f5e7a7fc640 1 -- 172.21.15.164:0/1141866986 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5e7c09f780 con 0x5607e3849cc0 2024-09-19T08:51:15.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.407+0000 7f5e9b4a0740 10 client.16572 fetch_fsmap learned FSMap version 30 2024-09-19T08:51:15.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.407+0000 7f5e9b4a0740 1 -- 172.21.15.164:0/1141866986 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5607e38d3500 con 0x5607e3849cc0 2024-09-19T08:51:15.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.408+0000 7f5e7a7fc640 1 -- 172.21.15.164:0/1141866986 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e84036070 con 0x5607e3849cc0 2024-09-19T08:51:15.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.408+0000 7f5e7a7fc640 1 -- 172.21.15.164:0/1141866986 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f5e7c097a40 con 0x5607e3849cc0 2024-09-19T08:51:15.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.408+0000 7f5e9b4a0740 10 client.16572 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:51:15.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.408+0000 7f5e9b4a0740 10 client.16572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:51:15.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.408+0000 7f5e9b4a0740 20 client.16572 populate_metadata read hostname 'smithi164' 2024-09-19T08:51:15.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.408+0000 7f5e9b4a0740 1 --2- 172.21.15.164:0/1141866986 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5607e38d3d90 0x5607e38d61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:15.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.408+0000 7f5e9b4a0740 4 client.16572 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:51:15.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.408+0000 7f5e9b4a0740 1 -- 172.21.15.164:0/1141866986 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5607e37b6d60 con 0x5607e38d3d90 2024-09-19T08:51:15.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.408+0000 7f5e92d9d640 1 --2- 172.21.15.164:0/1141866986 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5607e38d3d90 0x5607e38d61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:15.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.408+0000 7f5e92d9d640 1 --2- 172.21.15.164:0/1141866986 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5607e38d3d90 0x5607e38d61b0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c64ece45a6588c7e server_cookie=246fab0e2cd044b8 in_seq=0 out_seq=0 2024-09-19T08:51:15.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.408+0000 7f5e7a7fc640 10 client.16572 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:51:15.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.409+0000 7f5e7a7fc640 1 -- 172.21.15.164:0/1141866986 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5607e37b6d60 con 0x5607e38d3d90 2024-09-19T08:51:15.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.409+0000 7f5e7a7fc640 10 client.16572 handle_command_reply: tid=0 2024-09-19T08:51:15.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.409+0000 7f5e9b4a0740 1 client.16572 shutdown 2024-09-19T08:51:15.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.409+0000 7f5e9b4a0740 1 -- 172.21.15.164:0/1141866986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607e3849cc0 msgr2=0x5607e385cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:15.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.409+0000 7f5e9b4a0740 1 --2- 172.21.15.164:0/1141866986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607e3849cc0 0x5607e385cde0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c097da0 tx=0x7f5e7c00bf90 comp rx=0 tx=0).stop 2024-09-19T08:51:15.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.409+0000 7f5e9b4a0740 1 -- 172.21.15.164:0/1141866986 shutdown_connections 2024-09-19T08:51:15.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.409+0000 7f5e9b4a0740 1 --2- 172.21.15.164:0/1141866986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e3858a90 0x5607e385d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.409+0000 7f5e9b4a0740 1 --2- 172.21.15.164:0/1141866986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607e3849cc0 0x5607e385cde0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.409+0000 7f5e9b4a0740 1 --2- 172.21.15.164:0/1141866986 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5607e38d3d90 0x5607e38d61b0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.409+0000 7f5e9b4a0740 1 --2- 172.21.15.164:0/1141866986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607e38493a0 0x5607e385c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.409+0000 7f5e9b4a0740 1 -- 172.21.15.164:0/1141866986 >> 172.21.15.164:0/1141866986 conn(0x5607e3845a30 msgr2=0x5607e384aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:15.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.409+0000 7f5e7a7fc640 0 client.16572 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:51:15.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.409+0000 7f5e9b4a0740 1 -- 172.21.15.164:0/1141866986 shutdown_connections 2024-09-19T08:51:15.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.409+0000 7f5e9b4a0740 1 -- 172.21.15.164:0/1141866986 wait complete. 2024-09-19T08:51:15.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.409+0000 7f5e9b4a0740 20 client.16572 trim_cache size 0 max 16384 2024-09-19T08:51:15.416 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:51:15.416 INFO:teuthology.orchestra.run.smithi164.stdout: "status": "no active scrubs running", 2024-09-19T08:51:15.416 INFO:teuthology.orchestra.run.smithi164.stdout: "scrubs": {} 2024-09-19T08:51:15.416 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:51:15.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.410+0000 7f5e990c7640 1 -- 172.21.15.164:0/2904697270 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f5e800a33b0 msgr2=0x7f5e800a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:15.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.410+0000 7f5e990c7640 1 --2- 172.21.15.164:0/2904697270 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f5e800a33b0 0x7f5e800a5870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5e8402f540 tx=0x7f5e840095c0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.410+0000 7f5e990c7640 1 -- 172.21.15.164:0/2904697270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e94101ba0 msgr2=0x7f5e94198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:15.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.410+0000 7f5e990c7640 1 --2- 172.21.15.164:0/2904697270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e94101ba0 0x7f5e94198a60 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00b4a0 tx=0x7f5e7c00b960 comp rx=0 tx=0).stop 2024-09-19T08:51:15.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.410+0000 7f5e990c7640 1 -- 172.21.15.164:0/2904697270 shutdown_connections 2024-09-19T08:51:15.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.410+0000 7f5e990c7640 1 --2- 172.21.15.164:0/2904697270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e94105a70 0x7f5e9419cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.410+0000 7f5e990c7640 1 --2- 172.21.15.164:0/2904697270 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f5e800a33b0 0x7f5e800a5870 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.410+0000 7f5e990c7640 1 --2- 172.21.15.164:0/2904697270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e94101ba0 0x7f5e94198a60 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.410+0000 7f5e990c7640 1 --2- 172.21.15.164:0/2904697270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e94101220 0x7f5e94198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:15.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.410+0000 7f5e990c7640 1 -- 172.21.15.164:0/2904697270 >> 172.21.15.164:0/2904697270 conn(0x7f5e940fbee0 msgr2=0x7f5e941002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:15.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.410+0000 7f5e990c7640 1 -- 172.21.15.164:0/2904697270 shutdown_connections 2024-09-19T08:51:15.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:15.410+0000 7f5e990c7640 1 -- 172.21.15.164:0/2904697270 wait complete. 2024-09-19T08:51:16.026 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:15 smithi164 ceph-mon[30928]: pgmap v166: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 236 MiB/s wr, 593 op/s 2024-09-19T08:51:16.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:15 smithi170 ceph-mon[35072]: pgmap v166: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 236 MiB/s wr, 593 op/s 2024-09-19T08:51:16.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:15 smithi173 ceph-mon[35306]: pgmap v166: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 236 MiB/s wr, 593 op/s 2024-09-19T08:51:16.253 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T08:51:16.253 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:51:16.503 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:51:17.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.990+0000 7f0a9af11640 1 -- 172.21.15.164:0/1938632366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a94106f50 msgr2=0x7f0a9410eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:17.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.990+0000 7f0a9af11640 1 --2- 172.21.15.164:0/1938632366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a94106f50 0x7f0a9410eaf0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c009cd0 tx=0x7f0a8c02a8e0 comp rx=0 tx=0).stop 2024-09-19T08:51:17.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.990+0000 7f0a9af11640 1 -- 172.21.15.164:0/1938632366 shutdown_connections 2024-09-19T08:51:17.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.990+0000 7f0a9af11640 1 --2- 172.21.15.164:0/1938632366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a94106f50 0x7f0a9410eaf0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:17.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.990+0000 7f0a9af11640 1 --2- 172.21.15.164:0/1938632366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a94100ca0 0x7f0a94106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:17.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.990+0000 7f0a9af11640 1 --2- 172.21.15.164:0/1938632366 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a941002f0 0x7f0a941006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:17.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.991+0000 7f0a9af11640 1 -- 172.21.15.164:0/1938632366 >> 172.21.15.164:0/1938632366 conn(0x7f0a94077210 msgr2=0x7f0a940fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:17.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.991+0000 7f0a9af11640 1 -- 172.21.15.164:0/1938632366 shutdown_connections 2024-09-19T08:51:17.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.991+0000 7f0a9af11640 1 -- 172.21.15.164:0/1938632366 wait complete. 2024-09-19T08:51:17.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.991+0000 7f0a9af11640 1 Processor -- start 2024-09-19T08:51:17.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.991+0000 7f0a9af11640 1 -- start start 2024-09-19T08:51:17.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.991+0000 7f0a9af11640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a941002f0 0x7f0a94198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:17.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.991+0000 7f0a9af11640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a94100ca0 0x7f0a94198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:17.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.991+0000 7f0a9af11640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a94106f50 0x7f0a9419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:17.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.991+0000 7f0a9af11640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0a94112020 con 0x7f0a94100ca0 2024-09-19T08:51:17.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.991+0000 7f0a9af11640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0a94111ea0 con 0x7f0a941002f0 2024-09-19T08:51:17.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.991+0000 7f0a9af11640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0a941121a0 con 0x7f0a94106f50 2024-09-19T08:51:17.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.991+0000 7f0a93fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a94100ca0 0x7f0a94198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:17.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.991+0000 7f0a93fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a94100ca0 0x7f0a94198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45082/0 (socket says 172.21.15.164:45082) 2024-09-19T08:51:17.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.991+0000 7f0a93fff640 1 -- 172.21.15.164:0/2566392751 learned_addr learned my addr 172.21.15.164:0/2566392751 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:17.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.992+0000 7f0a98cad640 1 --2- 172.21.15.164:0/2566392751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a941002f0 0x7f0a94198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:17.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.992+0000 7f0a994ae640 1 --2- 172.21.15.164:0/2566392751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a94106f50 0x7f0a9419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:17.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.992+0000 7f0a93fff640 1 -- 172.21.15.164:0/2566392751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a94106f50 msgr2=0x7f0a9419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:17.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.992+0000 7f0a93fff640 1 --2- 172.21.15.164:0/2566392751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a94106f50 0x7f0a9419cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:17.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.992+0000 7f0a93fff640 1 -- 172.21.15.164:0/2566392751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a941002f0 msgr2=0x7f0a94198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:17.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.992+0000 7f0a93fff640 1 --2- 172.21.15.164:0/2566392751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a941002f0 0x7f0a94198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:17.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.992+0000 7f0a93fff640 1 -- 172.21.15.164:0/2566392751 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a9419d4b0 con 0x7f0a94100ca0 2024-09-19T08:51:17.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.992+0000 7f0a93fff640 1 --2- 172.21.15.164:0/2566392751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a94100ca0 0x7f0a94198a40 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f0a8000b450 tx=0x7f0a8000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:17.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.992+0000 7f0a91ffb640 1 -- 172.21.15.164:0/2566392751 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80015020 con 0x7f0a94100ca0 2024-09-19T08:51:17.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.992+0000 7f0a91ffb640 1 -- 172.21.15.164:0/2566392751 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a80011d70 con 0x7f0a94100ca0 2024-09-19T08:51:17.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.992+0000 7f0a9af11640 1 -- 172.21.15.164:0/2566392751 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a9419d7a0 con 0x7f0a94100ca0 2024-09-19T08:51:17.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.992+0000 7f0a91ffb640 1 -- 172.21.15.164:0/2566392751 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8000f340 con 0x7f0a94100ca0 2024-09-19T08:51:17.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.992+0000 7f0a9af11640 1 -- 172.21.15.164:0/2566392751 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a941a5080 con 0x7f0a94100ca0 2024-09-19T08:51:17.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.993+0000 7f0a9af11640 1 -- 172.21.15.164:0/2566392751 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a941006d0 con 0x7f0a94100ca0 2024-09-19T08:51:17.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.994+0000 7f0a91ffb640 1 -- 172.21.15.164:0/2566392751 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f0a80018020 con 0x7f0a94100ca0 2024-09-19T08:51:17.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.995+0000 7f0a91ffb640 1 --2- 172.21.15.164:0/2566392751 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f0a840a33b0 0x7f0a840a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:17.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.995+0000 7f0a98cad640 1 --2- 172.21.15.164:0/2566392751 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f0a840a33b0 0x7f0a840a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:17.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.995+0000 7f0a91ffb640 1 -- 172.21.15.164:0/2566392751 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f0a800cf700 con 0x7f0a94100ca0 2024-09-19T08:51:17.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.995+0000 7f0a98cad640 1 --2- 172.21.15.164:0/2566392751 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f0a840a33b0 0x7f0a840a5870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0a88005fd0 tx=0x7f0a88005e20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:17.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:16.999+0000 7f0a91ffb640 1 -- 172.21.15.164:0/2566392751 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0a8001f020 con 0x7f0a94100ca0 2024-09-19T08:51:17.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:17.200+0000 7f0a9af11640 1 -- 172.21.15.164:0/2566392751 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a9402d7c0 con 0x7f0a94100ca0 2024-09-19T08:51:17.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:17.201+0000 7f0a91ffb640 1 -- 172.21.15.164:0/2566392751 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11991 (secure 0 0 0) 0x7f0a8000d040 con 0x7f0a94100ca0 2024-09-19T08:51:17.201 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:51:17.201 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":30,"btime":"2024-09-19T08:50:17:326587+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:50:17.071374+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:51:17.201 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 30 2024-09-19T08:51:17.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:17.203+0000 7f0a9af11640 1 -- 172.21.15.164:0/2566392751 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f0a840a33b0 msgr2=0x7f0a840a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:17.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:17.204+0000 7f0a9af11640 1 --2- 172.21.15.164:0/2566392751 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f0a840a33b0 0x7f0a840a5870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0a88005fd0 tx=0x7f0a88005e20 comp rx=0 tx=0).stop 2024-09-19T08:51:17.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:17.204+0000 7f0a9af11640 1 -- 172.21.15.164:0/2566392751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a94100ca0 msgr2=0x7f0a94198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:17.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:17.204+0000 7f0a9af11640 1 --2- 172.21.15.164:0/2566392751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a94100ca0 0x7f0a94198a40 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f0a8000b450 tx=0x7f0a8000b910 comp rx=0 tx=0).stop 2024-09-19T08:51:17.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:17.204+0000 7f0a9af11640 1 -- 172.21.15.164:0/2566392751 shutdown_connections 2024-09-19T08:51:17.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:17.204+0000 7f0a9af11640 1 --2- 172.21.15.164:0/2566392751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a94106f50 0x7f0a9419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:17.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:17.204+0000 7f0a9af11640 1 --2- 172.21.15.164:0/2566392751 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f0a840a33b0 0x7f0a840a5870 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:17.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:17.204+0000 7f0a9af11640 1 --2- 172.21.15.164:0/2566392751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a94100ca0 0x7f0a94198a40 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:17.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:17.204+0000 7f0a9af11640 1 --2- 172.21.15.164:0/2566392751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a941002f0 0x7f0a94198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:17.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:17.204+0000 7f0a9af11640 1 -- 172.21.15.164:0/2566392751 >> 172.21.15.164:0/2566392751 conn(0x7f0a94077210 msgr2=0x7f0a94105850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:17.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:17.204+0000 7f0a9af11640 1 -- 172.21.15.164:0/2566392751 shutdown_connections 2024-09-19T08:51:17.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:17.204+0000 7f0a9af11640 1 -- 172.21.15.164:0/2566392751 wait complete. 2024-09-19T08:51:18.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:17 smithi164 ceph-mon[30928]: pgmap v167: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 176 MiB/s wr, 418 op/s 2024-09-19T08:51:18.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:17 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2566392751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:51:18.061 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T08:51:18.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:17 smithi170 ceph-mon[35072]: pgmap v167: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 176 MiB/s wr, 418 op/s 2024-09-19T08:51:18.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:17 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2566392751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:51:18.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:17 smithi173 ceph-mon[35306]: pgmap v167: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 176 MiB/s wr, 418 op/s 2024-09-19T08:51:18.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:17 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2566392751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:51:18.307 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:51:18.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.811+0000 7f070c50a640 1 -- 172.21.15.164:0/2690031016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0704101420 msgr2=0x7f070410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:18.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.811+0000 7f070c50a640 1 --2- 172.21.15.164:0/2690031016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0704101420 0x7f070410d950 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f06fc009cd0 tx=0x7f06fc02a8e0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.811+0000 7f070c50a640 1 -- 172.21.15.164:0/2690031016 shutdown_connections 2024-09-19T08:51:18.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.811+0000 7f070c50a640 1 --2- 172.21.15.164:0/2690031016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0704101420 0x7f070410d950 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.811+0000 7f070c50a640 1 --2- 172.21.15.164:0/2690031016 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0704100a80 0x7f0704100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.811+0000 7f070c50a640 1 --2- 172.21.15.164:0/2690031016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0704106a80 0x7f0704106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.811+0000 7f070c50a640 1 -- 172.21.15.164:0/2690031016 >> 172.21.15.164:0/2690031016 conn(0x7f07040fc7a0 msgr2=0x7f07040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:18.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.811+0000 7f070c50a640 1 -- 172.21.15.164:0/2690031016 shutdown_connections 2024-09-19T08:51:18.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.811+0000 7f070c50a640 1 -- 172.21.15.164:0/2690031016 wait complete. 2024-09-19T08:51:18.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.811+0000 7f070c50a640 1 Processor -- start 2024-09-19T08:51:18.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.811+0000 7f070c50a640 1 -- start start 2024-09-19T08:51:18.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.812+0000 7f070c50a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0704100a80 0x7f0704198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:18.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.812+0000 7f070c50a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0704101420 0x7f0704198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:18.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.812+0000 7f070c50a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0704106a80 0x7f070419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:18.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.812+0000 7f070c50a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0704110e80 con 0x7f0704100a80 2024-09-19T08:51:18.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.812+0000 7f070c50a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0704110d00 con 0x7f0704106a80 2024-09-19T08:51:18.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.812+0000 7f070c50a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0704111000 con 0x7f0704101420 2024-09-19T08:51:18.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.812+0000 7f070a2a6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0704100a80 0x7f0704198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:18.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.812+0000 7f070a2a6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0704100a80 0x7f0704198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45106/0 (socket says 172.21.15.164:45106) 2024-09-19T08:51:18.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.812+0000 7f070a2a6640 1 -- 172.21.15.164:0/531833011 learned_addr learned my addr 172.21.15.164:0/531833011 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:18.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.812+0000 7f0709aa5640 1 --2- 172.21.15.164:0/531833011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0704101420 0x7f0704198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:18.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.812+0000 7f070aaa7640 1 --2- 172.21.15.164:0/531833011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0704106a80 0x7f070419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:18.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.812+0000 7f070a2a6640 1 -- 172.21.15.164:0/531833011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0704101420 msgr2=0x7f0704198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:18.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.812+0000 7f070a2a6640 1 --2- 172.21.15.164:0/531833011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0704101420 0x7f0704198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.812+0000 7f070a2a6640 1 -- 172.21.15.164:0/531833011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0704106a80 msgr2=0x7f070419ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:18.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.812+0000 7f070a2a6640 1 --2- 172.21.15.164:0/531833011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0704106a80 0x7f070419ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.812+0000 7f070a2a6640 1 -- 172.21.15.164:0/531833011 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f070419d3d0 con 0x7f0704100a80 2024-09-19T08:51:18.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.812+0000 7f070a2a6640 1 --2- 172.21.15.164:0/531833011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0704100a80 0x7f0704198420 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f06f800c920 tx=0x7f06f800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:18.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.812+0000 7f06f77fe640 1 -- 172.21.15.164:0/531833011 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f8007c70 con 0x7f0704100a80 2024-09-19T08:51:18.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.812+0000 7f06f77fe640 1 -- 172.21.15.164:0/531833011 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06f8007e10 con 0x7f0704100a80 2024-09-19T08:51:18.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.812+0000 7f06f77fe640 1 -- 172.21.15.164:0/531833011 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f8018890 con 0x7f0704100a80 2024-09-19T08:51:18.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.812+0000 7f070c50a640 1 -- 172.21.15.164:0/531833011 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f070419d6c0 con 0x7f0704100a80 2024-09-19T08:51:18.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.812+0000 7f070c50a640 1 -- 172.21.15.164:0/531833011 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07041a4fa0 con 0x7f0704100a80 2024-09-19T08:51:18.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.815+0000 7f06f77fe640 1 -- 172.21.15.164:0/531833011 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f06f8018a70 con 0x7f0704100a80 2024-09-19T08:51:18.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.815+0000 7f070e8e3740 1 Processor -- start 2024-09-19T08:51:18.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.815+0000 7f070e8e3740 1 -- start start 2024-09-19T08:51:18.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.815+0000 7f070e8e3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650cc3f7b40 0x5650cc3f6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:18.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.816+0000 7f070e8e3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5650cc3f65d0 0x5650cc3f6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:18.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.816+0000 7f070e8e3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650cc405780 0x5650cc407b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:18.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.816+0000 7f070e8e3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5650cc35c2f0 con 0x5650cc3f7b40 2024-09-19T08:51:18.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.816+0000 7f070e8e3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5650cc303cf0 con 0x5650cc405780 2024-09-19T08:51:18.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.816+0000 7f070e8e3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5650cc3582b0 con 0x5650cc3f65d0 2024-09-19T08:51:18.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.816+0000 7f06f77fe640 1 --2- 172.21.15.164:0/531833011 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f06f00a3430 0x7f06f00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:18.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.816+0000 7f070aaa7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650cc3f7b40 0x5650cc3f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:18.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.816+0000 7f070aaa7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650cc3f7b40 0x5650cc3f6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45112/0 (socket says 172.21.15.164:45112) 2024-09-19T08:51:18.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.816+0000 7f070aaa7640 1 -- 172.21.15.164:0/2505099673 learned_addr learned my addr 172.21.15.164:0/2505099673 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:18.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.816+0000 7f070a2a6640 1 --2- 172.21.15.164:0/2505099673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5650cc3f65d0 0x5650cc3f6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:18.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.816+0000 7f06f77fe640 1 -- 172.21.15.164:0/531833011 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f06f80cfce0 con 0x7f0704100a80 2024-09-19T08:51:18.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.816+0000 7f0709aa5640 1 --2- 172.21.15.164:0/2505099673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650cc405780 0x5650cc407b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:18.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.816+0000 7f070aaa7640 1 -- 172.21.15.164:0/2505099673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5650cc3f65d0 msgr2=0x5650cc3f6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:18.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.816+0000 7f070aaa7640 1 --2- 172.21.15.164:0/2505099673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5650cc3f65d0 0x5650cc3f6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.816+0000 7f070aaa7640 1 -- 172.21.15.164:0/2505099673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650cc405780 msgr2=0x5650cc407b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:18.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.816+0000 7f070aaa7640 1 --2- 172.21.15.164:0/2505099673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650cc405780 0x5650cc407b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.816+0000 7f070aaa7640 1 -- 172.21.15.164:0/2505099673 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650cc2e83a0 con 0x5650cc3f7b40 2024-09-19T08:51:18.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.816+0000 7f0709aa5640 1 --2- 172.21.15.164:0/2505099673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650cc405780 0x5650cc407b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:51:18.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.816+0000 7f070aaa7640 1 --2- 172.21.15.164:0/531833011 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f06f00a3430 0x7f06f00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:18.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.816+0000 7f070aaa7640 1 --2- 172.21.15.164:0/2505099673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650cc3f7b40 0x5650cc3f6090 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f06fc009430 tx=0x7f06fc031540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ceaa9a630a70602 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:18.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.817+0000 7f06f4ff9640 1 -- 172.21.15.164:0/2505099673 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc031c60 con 0x5650cc3f7b40 2024-09-19T08:51:18.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.817+0000 7f06f4ff9640 1 -- 172.21.15.164:0/2505099673 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06fc036040 con 0x5650cc3f7b40 2024-09-19T08:51:18.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.817+0000 7f06f4ff9640 1 -- 172.21.15.164:0/2505099673 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc004700 con 0x5650cc3f7b40 2024-09-19T08:51:18.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.817+0000 7f070aaa7640 1 --2- 172.21.15.164:0/531833011 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f06f00a3430 0x7f06f00a58f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f06fc04b000 tx=0x7f06fc048000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:18.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.817+0000 7f070e8e3740 1 -- 172.21.15.164:0/2505099673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650cc3f7b40 msgr2=0x5650cc3f6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:18.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.817+0000 7f070e8e3740 1 --2- 172.21.15.164:0/2505099673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650cc3f7b40 0x5650cc3f6090 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f06fc009430 tx=0x7f06fc031540 comp rx=0 tx=0).stop 2024-09-19T08:51:18.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.817+0000 7f070e8e3740 1 -- 172.21.15.164:0/2505099673 shutdown_connections 2024-09-19T08:51:18.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.817+0000 7f070e8e3740 1 --2- 172.21.15.164:0/2505099673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650cc405780 0x5650cc407b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.817+0000 7f070e8e3740 1 --2- 172.21.15.164:0/2505099673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5650cc3f65d0 0x5650cc3f6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.817+0000 7f070e8e3740 1 --2- 172.21.15.164:0/2505099673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650cc3f7b40 0x5650cc3f6090 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.817+0000 7f070e8e3740 1 -- 172.21.15.164:0/2505099673 >> 172.21.15.164:0/2505099673 conn(0x5650cc3bf310 msgr2=0x5650cc3f2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:18.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.817+0000 7f070e8e3740 1 -- 172.21.15.164:0/2505099673 shutdown_connections 2024-09-19T08:51:18.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.818+0000 7f070e8e3740 1 -- 172.21.15.164:0/2505099673 wait complete. 2024-09-19T08:51:18.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.818+0000 7f070e8e3740 1 -- start start 2024-09-19T08:51:18.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.818+0000 7f070e8e3740 1 Processor -- start 2024-09-19T08:51:18.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.818+0000 7f070e8e3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650cc3f65d0 0x5650cc405170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:18.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.818+0000 7f070e8e3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5650cc3f7b40 0x5650cc402420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:18.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.818+0000 7f070e8e3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650cc405780 0x5650cc402a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:18.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.818+0000 7f070e8e3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5650cc408480 con 0x5650cc405780 2024-09-19T08:51:18.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.818+0000 7f070e8e3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5650cc317060 con 0x5650cc3f65d0 2024-09-19T08:51:18.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.818+0000 7f070e8e3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5650cc408600 con 0x5650cc3f7b40 2024-09-19T08:51:18.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.818+0000 7f070e8e3740 10 client.0 _ll_register_callbacks cb 0x5650cc06a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:51:18.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.818+0000 7f070aaa7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650cc405780 0x5650cc402a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:18.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.818+0000 7f0709aa5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5650cc3f7b40 0x5650cc402420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:18.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.818+0000 7f0709aa5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5650cc3f7b40 0x5650cc402420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:42978/0 (socket says 172.21.15.164:42978) 2024-09-19T08:51:18.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.818+0000 7f070aaa7640 1 --2- 172.21.15.164:0/4229174572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650cc405780 0x5650cc402a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45128/0 (socket says 172.21.15.164:45128) 2024-09-19T08:51:18.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.818+0000 7f0709aa5640 1 -- 172.21.15.164:0/4229174572 learned_addr learned my addr 172.21.15.164:0/4229174572 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:18.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.818+0000 7f070a2a6640 1 --2- 172.21.15.164:0/4229174572 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650cc3f65d0 0x5650cc405170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:18.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.819+0000 7f0709aa5640 1 -- 172.21.15.164:0/4229174572 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650cc3f65d0 msgr2=0x5650cc405170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:18.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.819+0000 7f0709aa5640 1 --2- 172.21.15.164:0/4229174572 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650cc3f65d0 0x5650cc405170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.819+0000 7f0709aa5640 1 -- 172.21.15.164:0/4229174572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650cc405780 msgr2=0x5650cc402a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:18.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.819+0000 7f0709aa5640 1 --2- 172.21.15.164:0/4229174572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650cc405780 0x5650cc402a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.819+0000 7f0709aa5640 1 -- 172.21.15.164:0/4229174572 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650cc0cd3c0 con 0x5650cc3f7b40 2024-09-19T08:51:18.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.819+0000 7f0709aa5640 1 --2- 172.21.15.164:0/4229174572 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5650cc3f7b40 0x5650cc402420 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0700013710 tx=0x7f0700013bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de4c6c89dda57267 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:18.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.819+0000 7f06edffb640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-09-19T08:51:18.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.819+0000 7f06edffb640 1 -- 172.21.15.164:0/4229174572 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0700004460 con 0x5650cc3f7b40 2024-09-19T08:51:18.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.819+0000 7f070e8e3740 1 -- 172.21.15.164:0/4229174572 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06fc036040 con 0x5650cc3f7b40 2024-09-19T08:51:18.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.819+0000 7f06edffb640 1 -- 172.21.15.164:0/4229174572 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f0700011940 con 0x5650cc3f7b40 2024-09-19T08:51:18.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.819+0000 7f06edffb640 1 -- 172.21.15.164:0/4229174572 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0700011b40 con 0x5650cc3f7b40 2024-09-19T08:51:18.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.819+0000 7f070e8e3740 10 client.25933 fetch_fsmap learned FSMap version 30 2024-09-19T08:51:18.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.819+0000 7f070e8e3740 1 -- 172.21.15.164:0/4229174572 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5650cc413f40 con 0x5650cc3f7b40 2024-09-19T08:51:18.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.819+0000 7f06edffb640 1 -- 172.21.15.164:0/4229174572 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06fc036040 con 0x5650cc3f7b40 2024-09-19T08:51:18.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.820+0000 7f06edffb640 1 -- 172.21.15.164:0/4229174572 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f070001a070 con 0x5650cc3f7b40 2024-09-19T08:51:18.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.820+0000 7f070e8e3740 10 client.25933 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:51:18.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.820+0000 7f070e8e3740 10 client.25933 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-19T08:51:18.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.820+0000 7f070e8e3740 20 client.25933 populate_metadata read hostname 'smithi164' 2024-09-19T08:51:18.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.820+0000 7f070e8e3740 1 --2- 172.21.15.164:0/4229174572 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5650cc4154a0 0x5650cc4178c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:18.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.820+0000 7f070e8e3740 4 client.25933 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:51:18.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.820+0000 7f070e8e3740 1 -- 172.21.15.164:0/4229174572 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650cc369860 con 0x5650cc4154a0 2024-09-19T08:51:18.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.820+0000 7f070a2a6640 1 --2- 172.21.15.164:0/4229174572 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5650cc4154a0 0x5650cc4178c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:18.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.820+0000 7f070a2a6640 1 --2- 172.21.15.164:0/4229174572 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5650cc4154a0 0x5650cc4178c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cab0f6c725728aec server_cookie=cdb697fd66920b47 in_seq=0 out_seq=0 2024-09-19T08:51:18.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.820+0000 7f06edffb640 10 client.25933 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:51:18.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.821+0000 7f06edffb640 1 -- 172.21.15.164:0/4229174572 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5650cc369860 con 0x5650cc4154a0 2024-09-19T08:51:18.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.821+0000 7f06edffb640 10 client.25933 handle_command_reply: tid=0 2024-09-19T08:51:18.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.821+0000 7f070e8e3740 1 client.25933 shutdown 2024-09-19T08:51:18.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.821+0000 7f070e8e3740 1 -- 172.21.15.164:0/4229174572 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5650cc3f7b40 msgr2=0x5650cc402420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:18.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.821+0000 7f070e8e3740 1 --2- 172.21.15.164:0/4229174572 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5650cc3f7b40 0x5650cc402420 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0700013710 tx=0x7f0700013bd0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.821+0000 7f070e8e3740 1 -- 172.21.15.164:0/4229174572 shutdown_connections 2024-09-19T08:51:18.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.821+0000 7f070e8e3740 1 --2- 172.21.15.164:0/4229174572 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5650cc4154a0 0x5650cc4178c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.821+0000 7f070e8e3740 1 --2- 172.21.15.164:0/4229174572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650cc405780 0x5650cc402a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.821+0000 7f070e8e3740 1 --2- 172.21.15.164:0/4229174572 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5650cc3f7b40 0x5650cc402420 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.821+0000 7f06edffb640 0 client.25933 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:51:18.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.821+0000 7f070e8e3740 1 --2- 172.21.15.164:0/4229174572 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650cc3f65d0 0x5650cc405170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.821+0000 7f070e8e3740 1 -- 172.21.15.164:0/4229174572 >> 172.21.15.164:0/4229174572 conn(0x5650cc3bf310 msgr2=0x5650cc3f8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:18.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.821+0000 7f070e8e3740 1 -- 172.21.15.164:0/4229174572 shutdown_connections 2024-09-19T08:51:18.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.821+0000 7f070e8e3740 1 -- 172.21.15.164:0/4229174572 wait complete. 2024-09-19T08:51:18.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.821+0000 7f070e8e3740 20 client.25933 trim_cache size 0 max 16384 2024-09-19T08:51:18.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.834+0000 7f070e8e3740 1 Processor -- start 2024-09-19T08:51:18.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.835+0000 7f070e8e3740 1 -- start start 2024-09-19T08:51:18.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.835+0000 7f070e8e3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5650cc3f65d0 0x5650cc3f2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:18.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.835+0000 7f070e8e3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650cc3f7b40 0x5650cc411720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:18.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.835+0000 7f070e8e3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650cc405780 0x5650cc401460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:18.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.835+0000 7f070e8e3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f07041ab3f0 con 0x5650cc405780 2024-09-19T08:51:18.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.835+0000 7f070e8e3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f07041acff0 con 0x5650cc3f7b40 2024-09-19T08:51:18.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.835+0000 7f070e8e3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f07041abac0 con 0x5650cc3f65d0 2024-09-19T08:51:18.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.835+0000 7f070a2a6640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5650cc3f65d0 0x5650cc3f2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:18.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.835+0000 7f070a2a6640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5650cc3f65d0 0x5650cc3f2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:42994/0 (socket says 172.21.15.164:42994) 2024-09-19T08:51:18.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.835+0000 7f070a2a6640 1 -- 172.21.15.164:0/2159115705 learned_addr learned my addr 172.21.15.164:0/2159115705 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:18.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.835+0000 7f070aaa7640 1 --2- 172.21.15.164:0/2159115705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650cc405780 0x5650cc401460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:18.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.836+0000 7f0709aa5640 1 --2- 172.21.15.164:0/2159115705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650cc3f7b40 0x5650cc411720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:18.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.836+0000 7f070a2a6640 1 -- 172.21.15.164:0/2159115705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650cc3f7b40 msgr2=0x5650cc411720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:18.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.836+0000 7f070a2a6640 1 --2- 172.21.15.164:0/2159115705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650cc3f7b40 0x5650cc411720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.836+0000 7f070a2a6640 1 -- 172.21.15.164:0/2159115705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650cc405780 msgr2=0x5650cc401460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:18.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.836+0000 7f070a2a6640 1 --2- 172.21.15.164:0/2159115705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650cc405780 0x5650cc401460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.836+0000 7f070a2a6640 1 -- 172.21.15.164:0/2159115705 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650cc402d80 con 0x5650cc3f65d0 2024-09-19T08:51:18.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.836+0000 7f070a2a6640 1 --2- 172.21.15.164:0/2159115705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5650cc3f65d0 0x5650cc3f2d70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f06f80cd500 tx=0x7f06f8013b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11941819dd249933 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:18.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.837+0000 7f06effff640 1 -- 172.21.15.164:0/2159115705 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f80a4ca0 con 0x5650cc3f65d0 2024-09-19T08:51:18.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.837+0000 7f06effff640 1 -- 172.21.15.164:0/2159115705 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06f8084070 con 0x5650cc3f65d0 2024-09-19T08:51:18.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.837+0000 7f06effff640 1 -- 172.21.15.164:0/2159115705 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f8015d70 con 0x5650cc3f65d0 2024-09-19T08:51:18.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.838+0000 7f070e8e3740 1 -- 172.21.15.164:0/2159115705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5650cc3f65d0 msgr2=0x5650cc3f2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:18.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.838+0000 7f070e8e3740 1 --2- 172.21.15.164:0/2159115705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5650cc3f65d0 0x5650cc3f2d70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f06f80cd500 tx=0x7f06f8013b80 comp rx=0 tx=0).stop 2024-09-19T08:51:18.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.838+0000 7f070e8e3740 1 -- 172.21.15.164:0/2159115705 shutdown_connections 2024-09-19T08:51:18.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.838+0000 7f070e8e3740 1 --2- 172.21.15.164:0/2159115705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650cc405780 0x5650cc401460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.838+0000 7f070e8e3740 1 --2- 172.21.15.164:0/2159115705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650cc3f7b40 0x5650cc411720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.838+0000 7f070e8e3740 1 --2- 172.21.15.164:0/2159115705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5650cc3f65d0 0x5650cc3f2d70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.838+0000 7f070e8e3740 1 -- 172.21.15.164:0/2159115705 >> 172.21.15.164:0/2159115705 conn(0x5650cc3bf310 msgr2=0x5650cc400d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:18.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.838+0000 7f070e8e3740 1 -- 172.21.15.164:0/2159115705 shutdown_connections 2024-09-19T08:51:18.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.838+0000 7f070e8e3740 1 -- 172.21.15.164:0/2159115705 wait complete. 2024-09-19T08:51:18.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.838+0000 7f070e8e3740 1 -- start start 2024-09-19T08:51:18.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.838+0000 7f070e8e3740 1 Processor -- start 2024-09-19T08:51:18.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.839+0000 7f070e8e3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5650cc3f65d0 0x5650cc40dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:18.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.839+0000 7f070e8e3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650cc3f7b40 0x5650cc40e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:18.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.839+0000 7f070e8e3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650cc405780 0x5650cc40aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:18.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.839+0000 7f070e8e3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f07041ac860 con 0x5650cc405780 2024-09-19T08:51:18.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.839+0000 7f070e8e3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f07041ac190 con 0x5650cc3f7b40 2024-09-19T08:51:18.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.839+0000 7f070e8e3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f07041a93f0 con 0x5650cc3f65d0 2024-09-19T08:51:18.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.839+0000 7f070e8e3740 10 client.0 _ll_register_callbacks cb 0x7f0700009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:51:18.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.839+0000 7f070aaa7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650cc405780 0x5650cc40aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:18.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.839+0000 7f070aaa7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650cc405780 0x5650cc40aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45148/0 (socket says 172.21.15.164:45148) 2024-09-19T08:51:18.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.839+0000 7f070aaa7640 1 -- 172.21.15.164:0/3481549397 learned_addr learned my addr 172.21.15.164:0/3481549397 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:18.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.839+0000 7f070a2a6640 1 --2- 172.21.15.164:0/3481549397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5650cc3f65d0 0x5650cc40dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:18.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.839+0000 7f0709aa5640 1 --2- 172.21.15.164:0/3481549397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650cc3f7b40 0x5650cc40e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:18.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.840+0000 7f070aaa7640 1 -- 172.21.15.164:0/3481549397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5650cc3f65d0 msgr2=0x5650cc40dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:18.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.840+0000 7f070aaa7640 1 --2- 172.21.15.164:0/3481549397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5650cc3f65d0 0x5650cc40dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.840+0000 7f070aaa7640 1 -- 172.21.15.164:0/3481549397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650cc3f7b40 msgr2=0x5650cc40e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:18.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.840+0000 7f070aaa7640 1 --2- 172.21.15.164:0/3481549397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650cc3f7b40 0x5650cc40e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.840+0000 7f070aaa7640 1 -- 172.21.15.164:0/3481549397 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650cc40b2e0 con 0x5650cc405780 2024-09-19T08:51:18.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.840+0000 7f070aaa7640 1 --2- 172.21.15.164:0/3481549397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650cc405780 0x5650cc40aab0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f06fc02ae00 tx=0x7f06fc0043e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3fb5aaf8613c138 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:18.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.840+0000 7f06edffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:51:18.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.840+0000 7f06edffb640 1 -- 172.21.15.164:0/3481549397 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc009890 con 0x5650cc405780 2024-09-19T08:51:18.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.840+0000 7f06edffb640 1 -- 172.21.15.164:0/3481549397 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f06fc031c60 con 0x5650cc405780 2024-09-19T08:51:18.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.840+0000 7f070e8e3740 1 -- 172.21.15.164:0/3481549397 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06f8084070 con 0x5650cc405780 2024-09-19T08:51:18.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.841+0000 7f06edffb640 1 -- 172.21.15.164:0/3481549397 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f06fc003970 con 0x5650cc405780 2024-09-19T08:51:18.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.841+0000 7f070e8e3740 10 client.16596 fetch_fsmap learned FSMap version 30 2024-09-19T08:51:18.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.841+0000 7f070e8e3740 1 -- 172.21.15.164:0/3481549397 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5650cc3fadc0 con 0x5650cc405780 2024-09-19T08:51:18.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.844+0000 7f06edffb640 1 -- 172.21.15.164:0/3481549397 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06f8084070 con 0x5650cc405780 2024-09-19T08:51:18.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.844+0000 7f06edffb640 1 -- 172.21.15.164:0/3481549397 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f06fc030070 con 0x5650cc405780 2024-09-19T08:51:18.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.845+0000 7f070e8e3740 10 client.16596 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:51:18.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.846+0000 7f070e8e3740 10 client.16596 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-19T08:51:18.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.846+0000 7f070e8e3740 20 client.16596 populate_metadata read hostname 'smithi164' 2024-09-19T08:51:18.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.846+0000 7f070e8e3740 1 --2- 172.21.15.164:0/3481549397 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5650cc484950 0x5650cc486d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:18.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.846+0000 7f070e8e3740 4 client.16596 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:51:18.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.846+0000 7f070e8e3740 1 -- 172.21.15.164:0/3481549397 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5650cc363d70 con 0x5650cc484950 2024-09-19T08:51:18.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.848+0000 7f070a2a6640 1 --2- 172.21.15.164:0/3481549397 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5650cc484950 0x5650cc486d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:18.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.851+0000 7f070a2a6640 1 --2- 172.21.15.164:0/3481549397 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5650cc484950 0x5650cc486d40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=686c4d4c1addef53 server_cookie=83222af4e3ddda3a in_seq=0 out_seq=0 2024-09-19T08:51:18.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.851+0000 7f06edffb640 10 client.16596 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:51:18.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.852+0000 7f06edffb640 1 -- 172.21.15.164:0/3481549397 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5650cc363d70 con 0x5650cc484950 2024-09-19T08:51:18.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.852+0000 7f06edffb640 10 client.16596 handle_command_reply: tid=0 2024-09-19T08:51:18.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.852+0000 7f070e8e3740 1 client.16596 shutdown 2024-09-19T08:51:18.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.854+0000 7f070e8e3740 1 -- 172.21.15.164:0/3481549397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650cc405780 msgr2=0x5650cc40aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:18.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.854+0000 7f070e8e3740 1 --2- 172.21.15.164:0/3481549397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650cc405780 0x5650cc40aab0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f06fc02ae00 tx=0x7f06fc0043e0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.854+0000 7f070e8e3740 1 -- 172.21.15.164:0/3481549397 shutdown_connections 2024-09-19T08:51:18.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.854+0000 7f070e8e3740 1 --2- 172.21.15.164:0/3481549397 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5650cc484950 0x5650cc486d40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.854+0000 7f070e8e3740 1 --2- 172.21.15.164:0/3481549397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650cc405780 0x5650cc40aab0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.854+0000 7f070e8e3740 1 --2- 172.21.15.164:0/3481549397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650cc3f7b40 0x5650cc40e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.854+0000 7f070e8e3740 1 --2- 172.21.15.164:0/3481549397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5650cc3f65d0 0x5650cc40dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.854+0000 7f070e8e3740 1 -- 172.21.15.164:0/3481549397 >> 172.21.15.164:0/3481549397 conn(0x5650cc3bf310 msgr2=0x5650cc401570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:18.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.854+0000 7f06edffb640 0 client.16596 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:51:18.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.854+0000 7f070e8e3740 1 -- 172.21.15.164:0/3481549397 shutdown_connections 2024-09-19T08:51:18.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.854+0000 7f070e8e3740 1 -- 172.21.15.164:0/3481549397 wait complete. 2024-09-19T08:51:18.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.854+0000 7f070e8e3740 20 client.16596 trim_cache size 0 max 16384 2024-09-19T08:51:18.868 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:51:18.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.856+0000 7f070c50a640 1 -- 172.21.15.164:0/531833011 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f06f00a3430 msgr2=0x7f06f00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:18.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.856+0000 7f070c50a640 1 --2- 172.21.15.164:0/531833011 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f06f00a3430 0x7f06f00a58f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f06fc04b000 tx=0x7f06fc048000 comp rx=0 tx=0).stop 2024-09-19T08:51:18.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.856+0000 7f070c50a640 1 -- 172.21.15.164:0/531833011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0704100a80 msgr2=0x7f0704198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:18.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.856+0000 7f070c50a640 1 --2- 172.21.15.164:0/531833011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0704100a80 0x7f0704198420 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f06f800c920 tx=0x7f06f800cde0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.857+0000 7f070c50a640 1 -- 172.21.15.164:0/531833011 shutdown_connections 2024-09-19T08:51:18.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.857+0000 7f070c50a640 1 --2- 172.21.15.164:0/531833011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0704106a80 0x7f070419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.857+0000 7f070c50a640 1 --2- 172.21.15.164:0/531833011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0704101420 0x7f0704198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.857+0000 7f070c50a640 1 --2- 172.21.15.164:0/531833011 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f06f00a3430 0x7f06f00a58f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.857+0000 7f070c50a640 1 --2- 172.21.15.164:0/531833011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0704100a80 0x7f0704198420 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:18.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.857+0000 7f070c50a640 1 -- 172.21.15.164:0/531833011 >> 172.21.15.164:0/531833011 conn(0x7f07040fc7a0 msgr2=0x7f070406a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:18.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.865+0000 7f070c50a640 1 -- 172.21.15.164:0/531833011 shutdown_connections 2024-09-19T08:51:18.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:18.865+0000 7f070c50a640 1 -- 172.21.15.164:0/531833011 wait complete. 2024-09-19T08:51:20.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:19 smithi164 ceph-mon[30928]: pgmap v168: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 75 MiB/s wr, 95 op/s 2024-09-19T08:51:20.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:19 smithi170 ceph-mon[35072]: pgmap v168: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 75 MiB/s wr, 95 op/s 2024-09-19T08:51:20.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:19 smithi173 ceph-mon[35306]: pgmap v168: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 75 MiB/s wr, 95 op/s 2024-09-19T08:51:20.233 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T08:51:20.482 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:51:21.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.991+0000 7f3a3c94d640 1 -- 172.21.15.164:0/423704000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a381082a0 msgr2=0x7f3a3810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:21.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.991+0000 7f3a3c94d640 1 --2- 172.21.15.164:0/423704000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a381082a0 0x7f3a3810ec40 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f3a30009930 tx=0x7f3a3002a420 comp rx=0 tx=0).stop 2024-09-19T08:51:21.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.991+0000 7f3a3c94d640 1 -- 172.21.15.164:0/423704000 shutdown_connections 2024-09-19T08:51:21.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.991+0000 7f3a3c94d640 1 --2- 172.21.15.164:0/423704000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a381082a0 0x7f3a3810ec40 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:21.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.991+0000 7f3a3c94d640 1 --2- 172.21.15.164:0/423704000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a381036e0 0x7f3a38107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:21.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.991+0000 7f3a3c94d640 1 --2- 172.21.15.164:0/423704000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a38102d30 0x7f3a38103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:21.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.991+0000 7f3a3c94d640 1 -- 172.21.15.164:0/423704000 >> 172.21.15.164:0/423704000 conn(0x7f3a380fc7d0 msgr2=0x7f3a380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:21.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.991+0000 7f3a3c94d640 1 -- 172.21.15.164:0/423704000 shutdown_connections 2024-09-19T08:51:21.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.991+0000 7f3a3c94d640 1 -- 172.21.15.164:0/423704000 wait complete. 2024-09-19T08:51:21.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.992+0000 7f3a3c94d640 1 Processor -- start 2024-09-19T08:51:21.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.992+0000 7f3a3c94d640 1 -- start start 2024-09-19T08:51:21.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.993+0000 7f3a3c94d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a38102d30 0x7f3a381984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:21.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.993+0000 7f3a3c94d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a381036e0 0x7f3a38198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:21.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.993+0000 7f3a3c94d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a381082a0 0x7f3a3819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:21.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.993+0000 7f3a3c94d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3a381120a0 con 0x7f3a38102d30 2024-09-19T08:51:21.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.993+0000 7f3a3c94d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3a38111f20 con 0x7f3a381082a0 2024-09-19T08:51:21.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.993+0000 7f3a3c94d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3a38112220 con 0x7f3a381036e0 2024-09-19T08:51:21.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.993+0000 7f3a3659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a38102d30 0x7f3a381984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:21.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.993+0000 7f3a3659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a38102d30 0x7f3a381984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45162/0 (socket says 172.21.15.164:45162) 2024-09-19T08:51:21.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.993+0000 7f3a3659c640 1 -- 172.21.15.164:0/4073023931 learned_addr learned my addr 172.21.15.164:0/4073023931 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:21.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.993+0000 7f3a35d9b640 1 --2- 172.21.15.164:0/4073023931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a381036e0 0x7f3a38198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:21.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.993+0000 7f3a36d9d640 1 --2- 172.21.15.164:0/4073023931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a381082a0 0x7f3a3819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:21.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.993+0000 7f3a3659c640 1 -- 172.21.15.164:0/4073023931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a381036e0 msgr2=0x7f3a38198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:21.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.993+0000 7f3a3659c640 1 --2- 172.21.15.164:0/4073023931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a381036e0 0x7f3a38198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:21.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.993+0000 7f3a3659c640 1 -- 172.21.15.164:0/4073023931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a381082a0 msgr2=0x7f3a3819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:21.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.993+0000 7f3a3659c640 1 --2- 172.21.15.164:0/4073023931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a381082a0 0x7f3a3819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:21.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.993+0000 7f3a3659c640 1 -- 172.21.15.164:0/4073023931 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a3819d480 con 0x7f3a38102d30 2024-09-19T08:51:21.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.993+0000 7f3a35d9b640 1 --2- 172.21.15.164:0/4073023931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a381036e0 0x7f3a38198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:51:21.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.993+0000 7f3a36d9d640 1 --2- 172.21.15.164:0/4073023931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a381082a0 0x7f3a3819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:51:21.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.993+0000 7f3a3659c640 1 --2- 172.21.15.164:0/4073023931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a38102d30 0x7f3a381984d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3a2800cce0 tx=0x7f3a2800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:21.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.993+0000 7f3a277fe640 1 -- 172.21.15.164:0/4073023931 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2800f650 con 0x7f3a38102d30 2024-09-19T08:51:21.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.994+0000 7f3a3c94d640 1 -- 172.21.15.164:0/4073023931 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a3819d770 con 0x7f3a38102d30 2024-09-19T08:51:21.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.994+0000 7f3a277fe640 1 -- 172.21.15.164:0/4073023931 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a2800fcd0 con 0x7f3a38102d30 2024-09-19T08:51:21.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.994+0000 7f3a277fe640 1 -- 172.21.15.164:0/4073023931 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a28018c70 con 0x7f3a38102d30 2024-09-19T08:51:21.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.994+0000 7f3a3c94d640 1 -- 172.21.15.164:0/4073023931 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a381a5050 con 0x7f3a38102d30 2024-09-19T08:51:21.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.996+0000 7f3a277fe640 1 -- 172.21.15.164:0/4073023931 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f3a2800f7f0 con 0x7f3a38102d30 2024-09-19T08:51:21.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.996+0000 7f3a3ed26740 1 Processor -- start 2024-09-19T08:51:21.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.997+0000 7f3a3ed26740 1 -- start start 2024-09-19T08:51:21.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.997+0000 7f3a3ed26740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e8e890b40 0x561e8e88f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:21.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.997+0000 7f3a3ed26740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e8e88f5d0 0x561e8e88fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:21.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.997+0000 7f3a3ed26740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8e89a370 0x561e8e89c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:21.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.997+0000 7f3a3ed26740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561e8e7f52f0 con 0x561e8e89a370 2024-09-19T08:51:21.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.997+0000 7f3a3ed26740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561e8e79ccf0 con 0x561e8e88f5d0 2024-09-19T08:51:21.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.997+0000 7f3a3ed26740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x561e8e7f12b0 con 0x561e8e890b40 2024-09-19T08:51:21.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.997+0000 7f3a277fe640 1 --2- 172.21.15.164:0/4073023931 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3a200a33b0 0x7f3a200a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:21.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.997+0000 7f3a35d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8e89a370 0x561e8e89c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:21.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.997+0000 7f3a35d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8e89a370 0x561e8e89c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45178/0 (socket says 172.21.15.164:45178) 2024-09-19T08:51:21.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.997+0000 7f3a36d9d640 1 --2- 172.21.15.164:0/4073023931 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3a200a33b0 0x7f3a200a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:21.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.997+0000 7f3a35d9b640 1 -- 172.21.15.164:0/2049242080 learned_addr learned my addr 172.21.15.164:0/2049242080 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:21.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.997+0000 7f3a277fe640 1 -- 172.21.15.164:0/4073023931 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f3a280d0220 con 0x7f3a38102d30 2024-09-19T08:51:21.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.997+0000 7f3a3659c640 1 --2- 172.21.15.164:0/2049242080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e8e88f5d0 0x561e8e88fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:21.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.997+0000 7f3a36d9d640 1 --2- 172.21.15.164:0/2049242080 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e8e890b40 0x561e8e88f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:21.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.997+0000 7f3a35d9b640 1 -- 172.21.15.164:0/2049242080 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e8e890b40 msgr2=0x561e8e88f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:21.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.997+0000 7f3a35d9b640 1 --2- 172.21.15.164:0/2049242080 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e8e890b40 0x561e8e88f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:21.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.997+0000 7f3a35d9b640 1 -- 172.21.15.164:0/2049242080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e8e88f5d0 msgr2=0x561e8e88fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:21.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.997+0000 7f3a35d9b640 1 --2- 172.21.15.164:0/2049242080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e8e88f5d0 0x561e8e88fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:21.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.997+0000 7f3a35d9b640 1 -- 172.21.15.164:0/2049242080 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e8e7813a0 con 0x561e8e89a370 2024-09-19T08:51:21.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.997+0000 7f3a3659c640 1 --2- 172.21.15.164:0/2049242080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e8e88f5d0 0x561e8e88fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:51:21.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.998+0000 7f3a35d9b640 1 --2- 172.21.15.164:0/2049242080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8e89a370 0x561e8e89c760 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c009b30 tx=0x7f3a2c00e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dbef4015012c0f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:21.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.998+0000 7f3a36d9d640 1 --2- 172.21.15.164:0/4073023931 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3a200a33b0 0x7f3a200a5870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3a30031550 tx=0x7f3a30005d50 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:21.044 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:51:21.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.998+0000 7f3a24ff9640 1 -- 172.21.15.164:0/2049242080 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c016070 con 0x561e8e89a370 2024-09-19T08:51:21.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.998+0000 7f3a24ff9640 1 -- 172.21.15.164:0/2049242080 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a2c00ed20 con 0x561e8e89a370 2024-09-19T08:51:21.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.998+0000 7f3a24ff9640 1 -- 172.21.15.164:0/2049242080 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c010710 con 0x561e8e89a370 2024-09-19T08:51:21.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.999+0000 7f3a3ed26740 1 -- 172.21.15.164:0/2049242080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8e89a370 msgr2=0x561e8e89c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:21.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.999+0000 7f3a3ed26740 1 --2- 172.21.15.164:0/2049242080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8e89a370 0x561e8e89c760 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c009b30 tx=0x7f3a2c00e2f0 comp rx=0 tx=0).stop 2024-09-19T08:51:21.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.999+0000 7f3a3ed26740 1 -- 172.21.15.164:0/2049242080 shutdown_connections 2024-09-19T08:51:21.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.999+0000 7f3a3ed26740 1 --2- 172.21.15.164:0/2049242080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8e89a370 0x561e8e89c760 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:21.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.999+0000 7f3a3ed26740 1 --2- 172.21.15.164:0/2049242080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e8e88f5d0 0x561e8e88fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:21.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.999+0000 7f3a3ed26740 1 --2- 172.21.15.164:0/2049242080 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e8e890b40 0x561e8e88f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:21.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.999+0000 7f3a3ed26740 1 -- 172.21.15.164:0/2049242080 >> 172.21.15.164:0/2049242080 conn(0x561e8e858310 msgr2=0x561e8e88bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:21.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.999+0000 7f3a3ed26740 1 -- 172.21.15.164:0/2049242080 shutdown_connections 2024-09-19T08:51:21.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.999+0000 7f3a3ed26740 1 -- 172.21.15.164:0/2049242080 wait complete. 2024-09-19T08:51:21.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.999+0000 7f3a3ed26740 1 -- start start 2024-09-19T08:51:21.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:20.999+0000 7f3a3ed26740 1 Processor -- start 2024-09-19T08:51:21.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.000+0000 7f3a3ed26740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e8e88f5d0 0x561e8e899ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:21.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.000+0000 7f3a3ed26740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8e890b40 0x561e8e896f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:21.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.000+0000 7f3a3ed26740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e8e89a370 0x561e8e8974f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:21.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.000+0000 7f3a3ed26740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561e8e89d070 con 0x561e8e890b40 2024-09-19T08:51:21.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.000+0000 7f3a3ed26740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561e8e7b0060 con 0x561e8e89a370 2024-09-19T08:51:21.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.000+0000 7f3a3ed26740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x561e8e89d1f0 con 0x561e8e88f5d0 2024-09-19T08:51:21.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.000+0000 7f3a3ed26740 10 client.0 _ll_register_callbacks cb 0x561e8e5038c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:51:21.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.000+0000 7f3a35d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8e890b40 0x561e8e896f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:21.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.000+0000 7f3a35d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8e890b40 0x561e8e896f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45186/0 (socket says 172.21.15.164:45186) 2024-09-19T08:51:21.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.000+0000 7f3a35d9b640 1 -- 172.21.15.164:0/3393857662 learned_addr learned my addr 172.21.15.164:0/3393857662 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:21.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.000+0000 7f3a36d9d640 1 --2- 172.21.15.164:0/3393857662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e8e89a370 0x561e8e8974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:21.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.000+0000 7f3a3659c640 1 --2- 172.21.15.164:0/3393857662 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e8e88f5d0 0x561e8e899ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:21.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.000+0000 7f3a35d9b640 1 -- 172.21.15.164:0/3393857662 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e8e88f5d0 msgr2=0x561e8e899ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:21.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.000+0000 7f3a35d9b640 1 --2- 172.21.15.164:0/3393857662 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e8e88f5d0 0x561e8e899ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:21.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.000+0000 7f3a35d9b640 1 -- 172.21.15.164:0/3393857662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e8e89a370 msgr2=0x561e8e8974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:21.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.000+0000 7f3a35d9b640 1 --2- 172.21.15.164:0/3393857662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e8e89a370 0x561e8e8974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:21.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.000+0000 7f3a35d9b640 1 -- 172.21.15.164:0/3393857662 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e8e5663c0 con 0x561e8e890b40 2024-09-19T08:51:21.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.000+0000 7f3a36d9d640 1 --2- 172.21.15.164:0/3393857662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e8e89a370 0x561e8e8974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:51:21.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.000+0000 7f3a35d9b640 1 --2- 172.21.15.164:0/3393857662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8e890b40 0x561e8e896f90 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c011f60 tx=0x7f3a2c0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677b0d33e154ddc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:21.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.000+0000 7f3a1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:51:21.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.000+0000 7f3a1dffb640 1 -- 172.21.15.164:0/3393857662 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c016050 con 0x561e8e890b40 2024-09-19T08:51:21.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.000+0000 7f3a1dffb640 1 -- 172.21.15.164:0/3393857662 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f3a2c0119c0 con 0x561e8e890b40 2024-09-19T08:51:21.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.001+0000 7f3a3ed26740 1 -- 172.21.15.164:0/3393857662 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a2c00ed20 con 0x561e8e890b40 2024-09-19T08:51:21.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.001+0000 7f3a1dffb640 1 -- 172.21.15.164:0/3393857662 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3a2c0046c0 con 0x561e8e890b40 2024-09-19T08:51:21.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.001+0000 7f3a3ed26740 10 client.16608 fetch_fsmap learned FSMap version 30 2024-09-19T08:51:21.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.001+0000 7f3a3ed26740 1 -- 172.21.15.164:0/3393857662 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561e8e8a8a80 con 0x561e8e890b40 2024-09-19T08:51:21.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.001+0000 7f3a1dffb640 1 -- 172.21.15.164:0/3393857662 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a2c00ed20 con 0x561e8e890b40 2024-09-19T08:51:21.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.001+0000 7f3a1dffb640 1 -- 172.21.15.164:0/3393857662 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f3a2c02b050 con 0x561e8e890b40 2024-09-19T08:51:21.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.001+0000 7f3a3ed26740 10 client.16608 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:51:21.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.001+0000 7f3a3ed26740 10 client.16608 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T08:51:21.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.001+0000 7f3a3ed26740 20 client.16608 populate_metadata read hostname 'smithi164' 2024-09-19T08:51:21.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.001+0000 7f3a3ed26740 1 --2- 172.21.15.164:0/3393857662 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x561e8e8a9e30 0x561e8e8ac220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:21.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.001+0000 7f3a3ed26740 4 client.16608 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:51:21.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.001+0000 7f3a3ed26740 1 -- 172.21.15.164:0/3393857662 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e8e802860 con 0x561e8e8a9e30 2024-09-19T08:51:21.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.001+0000 7f3a3659c640 1 --2- 172.21.15.164:0/3393857662 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x561e8e8a9e30 0x561e8e8ac220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:21.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.002+0000 7f3a1dffb640 10 client.16608 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:51:21.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.002+0000 7f3a3659c640 1 --2- 172.21.15.164:0/3393857662 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x561e8e8a9e30 0x561e8e8ac220 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6e1f29c7c764e35e server_cookie=825aa0018e155630 in_seq=0 out_seq=0 2024-09-19T08:51:21.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.002+0000 7f3a1dffb640 1 -- 172.21.15.164:0/3393857662 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561e8e802860 con 0x561e8e8a9e30 2024-09-19T08:51:21.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.002+0000 7f3a1dffb640 10 client.16608 handle_command_reply: tid=0 2024-09-19T08:51:21.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.003+0000 7f3a3ed26740 1 client.16608 shutdown 2024-09-19T08:51:21.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.003+0000 7f3a3ed26740 1 -- 172.21.15.164:0/3393857662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8e890b40 msgr2=0x561e8e896f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:21.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.003+0000 7f3a3ed26740 1 --2- 172.21.15.164:0/3393857662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8e890b40 0x561e8e896f90 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c011f60 tx=0x7f3a2c0056a0 comp rx=0 tx=0).stop 2024-09-19T08:51:21.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.003+0000 7f3a3ed26740 1 -- 172.21.15.164:0/3393857662 shutdown_connections 2024-09-19T08:51:21.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.003+0000 7f3a3ed26740 1 --2- 172.21.15.164:0/3393857662 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x561e8e8a9e30 0x561e8e8ac220 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:21.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.003+0000 7f3a3ed26740 1 --2- 172.21.15.164:0/3393857662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e8e89a370 0x561e8e8974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:21.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.003+0000 7f3a3ed26740 1 --2- 172.21.15.164:0/3393857662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8e890b40 0x561e8e896f90 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:21.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.003+0000 7f3a3ed26740 1 --2- 172.21.15.164:0/3393857662 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e8e88f5d0 0x561e8e899ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:21.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.003+0000 7f3a3ed26740 1 -- 172.21.15.164:0/3393857662 >> 172.21.15.164:0/3393857662 conn(0x561e8e858310 msgr2=0x561e8e891960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:21.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.003+0000 7f3a1dffb640 0 client.16608 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:51:21.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.003+0000 7f3a3ed26740 1 -- 172.21.15.164:0/3393857662 shutdown_connections 2024-09-19T08:51:21.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.003+0000 7f3a3ed26740 1 -- 172.21.15.164:0/3393857662 wait complete. 2024-09-19T08:51:21.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.003+0000 7f3a3ed26740 20 client.16608 trim_cache size 0 max 16384 2024-09-19T08:51:21.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.016+0000 7f3a3ed26740 1 Processor -- start 2024-09-19T08:51:21.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.016+0000 7f3a3ed26740 1 -- start start 2024-09-19T08:51:21.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.016+0000 7f3a3ed26740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8e88f5d0 0x561e8e8a6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:21.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.016+0000 7f3a3ed26740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e8e890b40 0x561e8e898f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:21.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.016+0000 7f3a3ed26740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e8e89a370 0x561e8e8b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:21.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.016+0000 7f3a3ed26740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3a381ad710 con 0x561e8e88f5d0 2024-09-19T08:51:21.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.016+0000 7f3a3ed26740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3a381af310 con 0x561e8e89a370 2024-09-19T08:51:21.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.016+0000 7f3a3ed26740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3a381adde0 con 0x561e8e890b40 2024-09-19T08:51:21.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.016+0000 7f3a3659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8e88f5d0 0x561e8e8a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:21.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.016+0000 7f3a3659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8e88f5d0 0x561e8e8a6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45200/0 (socket says 172.21.15.164:45200) 2024-09-19T08:51:21.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.016+0000 7f3a3659c640 1 -- 172.21.15.164:0/2388706871 learned_addr learned my addr 172.21.15.164:0/2388706871 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:21.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.016+0000 7f3a35d9b640 1 --2- 172.21.15.164:0/2388706871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e8e890b40 0x561e8e898f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:21.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.017+0000 7f3a3659c640 1 -- 172.21.15.164:0/2388706871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e8e890b40 msgr2=0x561e8e898f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:21.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.017+0000 7f3a3659c640 1 --2- 172.21.15.164:0/2388706871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e8e890b40 0x561e8e898f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:21.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.017+0000 7f3a36d9d640 1 --2- 172.21.15.164:0/2388706871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e8e89a370 0x561e8e8b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:21.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.017+0000 7f3a3659c640 1 -- 172.21.15.164:0/2388706871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e8e89a370 msgr2=0x561e8e8b18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:21.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.017+0000 7f3a3659c640 1 --2- 172.21.15.164:0/2388706871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e8e89a370 0x561e8e8b18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:21.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.017+0000 7f3a3659c640 1 -- 172.21.15.164:0/2388706871 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e8e892120 con 0x561e8e88f5d0 2024-09-19T08:51:21.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.017+0000 7f3a35d9b640 1 --2- 172.21.15.164:0/2388706871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e8e890b40 0x561e8e898f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:51:21.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.017+0000 7f3a36d9d640 1 --2- 172.21.15.164:0/2388706871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e8e89a370 0x561e8e8b18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:51:21.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.017+0000 7f3a3659c640 1 --2- 172.21.15.164:0/2388706871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8e88f5d0 0x561e8e8a6310 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f3a2809c500 tx=0x7f3a28011040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc7609f9815967d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:21.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.017+0000 7f3a1ffff640 1 -- 172.21.15.164:0/2388706871 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2809cc70 con 0x561e8e88f5d0 2024-09-19T08:51:21.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.017+0000 7f3a1ffff640 1 -- 172.21.15.164:0/2388706871 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a28085070 con 0x561e8e88f5d0 2024-09-19T08:51:21.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.017+0000 7f3a1ffff640 1 -- 172.21.15.164:0/2388706871 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2800fae0 con 0x561e8e88f5d0 2024-09-19T08:51:21.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.018+0000 7f3a3ed26740 1 -- 172.21.15.164:0/2388706871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8e88f5d0 msgr2=0x561e8e8a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:21.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.018+0000 7f3a3ed26740 1 --2- 172.21.15.164:0/2388706871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8e88f5d0 0x561e8e8a6310 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f3a2809c500 tx=0x7f3a28011040 comp rx=0 tx=0).stop 2024-09-19T08:51:21.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.018+0000 7f3a3ed26740 1 -- 172.21.15.164:0/2388706871 shutdown_connections 2024-09-19T08:51:21.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.018+0000 7f3a3ed26740 1 --2- 172.21.15.164:0/2388706871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e8e89a370 0x561e8e8b18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:21.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.018+0000 7f3a3ed26740 1 --2- 172.21.15.164:0/2388706871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e8e890b40 0x561e8e898f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:21.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.018+0000 7f3a3ed26740 1 --2- 172.21.15.164:0/2388706871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8e88f5d0 0x561e8e8a6310 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:21.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.018+0000 7f3a3ed26740 1 -- 172.21.15.164:0/2388706871 >> 172.21.15.164:0/2388706871 conn(0x561e8e858310 msgr2=0x561e8e8987f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:21.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.018+0000 7f3a3ed26740 1 -- 172.21.15.164:0/2388706871 shutdown_connections 2024-09-19T08:51:21.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.018+0000 7f3a3ed26740 1 -- 172.21.15.164:0/2388706871 wait complete. 2024-09-19T08:51:21.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.018+0000 7f3a3ed26740 1 -- start start 2024-09-19T08:51:21.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.018+0000 7f3a3ed26740 1 Processor -- start 2024-09-19T08:51:21.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.018+0000 7f3a3ed26740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8e88f5d0 0x561e8e8a26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:21.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.018+0000 7f3a3ed26740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e8e890b40 0x561e8e8a2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:21.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.018+0000 7f3a3ed26740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e8e89a370 0x561e8e893940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:21.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.018+0000 7f3a3ed26740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3a381aeb80 con 0x561e8e88f5d0 2024-09-19T08:51:21.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.018+0000 7f3a3ed26740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3a381ae4b0 con 0x561e8e890b40 2024-09-19T08:51:21.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.018+0000 7f3a3ed26740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3a381ab6f0 con 0x561e8e89a370 2024-09-19T08:51:21.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.018+0000 7f3a3ed26740 10 client.0 _ll_register_callbacks cb 0x7f3a28007f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:51:21.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.018+0000 7f3a3659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8e88f5d0 0x561e8e8a26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:21.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.019+0000 7f3a3659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8e88f5d0 0x561e8e8a26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45206/0 (socket says 172.21.15.164:45206) 2024-09-19T08:51:21.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.019+0000 7f3a3659c640 1 -- 172.21.15.164:0/1414393551 learned_addr learned my addr 172.21.15.164:0/1414393551 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:21.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.019+0000 7f3a3659c640 1 -- 172.21.15.164:0/1414393551 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e8e89a370 msgr2=0x561e8e893940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:21.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.019+0000 7f3a3659c640 1 --2- 172.21.15.164:0/1414393551 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e8e89a370 0x561e8e893940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:21.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.019+0000 7f3a3659c640 1 -- 172.21.15.164:0/1414393551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e8e890b40 msgr2=0x561e8e8a2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:21.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.019+0000 7f3a3659c640 1 --2- 172.21.15.164:0/1414393551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e8e890b40 0x561e8e8a2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:21.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.019+0000 7f3a3659c640 1 -- 172.21.15.164:0/1414393551 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e8e894170 con 0x561e8e88f5d0 2024-09-19T08:51:21.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.019+0000 7f3a3659c640 1 --2- 172.21.15.164:0/1414393551 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8e88f5d0 0x561e8e8a26f0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f3a280cdfe0 tx=0x7f3a2809ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692f2bb38b013086 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:21.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.019+0000 7f3a1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:51:21.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.019+0000 7f3a1dffb640 1 -- 172.21.15.164:0/1414393551 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a280b9070 con 0x561e8e88f5d0 2024-09-19T08:51:21.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.019+0000 7f3a1dffb640 1 -- 172.21.15.164:0/1414393551 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f3a28098be0 con 0x561e8e88f5d0 2024-09-19T08:51:21.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.019+0000 7f3a3ed26740 1 -- 172.21.15.164:0/1414393551 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a28085070 con 0x561e8e88f5d0 2024-09-19T08:51:21.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.019+0000 7f3a1dffb640 1 -- 172.21.15.164:0/1414393551 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3a280a0920 con 0x561e8e88f5d0 2024-09-19T08:51:21.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.019+0000 7f3a3ed26740 10 client.16614 fetch_fsmap learned FSMap version 30 2024-09-19T08:51:21.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.019+0000 7f3a3ed26740 1 -- 172.21.15.164:0/1414393551 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561e8e914390 con 0x561e8e88f5d0 2024-09-19T08:51:21.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.019+0000 7f3a1dffb640 1 -- 172.21.15.164:0/1414393551 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a28085070 con 0x561e8e88f5d0 2024-09-19T08:51:21.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.019+0000 7f3a1dffb640 1 -- 172.21.15.164:0/1414393551 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f3a2809b0a0 con 0x561e8e88f5d0 2024-09-19T08:51:21.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.020+0000 7f3a3ed26740 10 client.16614 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:51:21.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.020+0000 7f3a3ed26740 10 client.16614 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T08:51:21.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.020+0000 7f3a3ed26740 20 client.16614 populate_metadata read hostname 'smithi164' 2024-09-19T08:51:21.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.020+0000 7f3a3ed26740 1 --2- 172.21.15.164:0/1414393551 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x561e8e914ab0 0x561e8e916f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:21.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.020+0000 7f3a3ed26740 4 client.16614 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:51:21.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.020+0000 7f3a3ed26740 1 -- 172.21.15.164:0/1414393551 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561e8e7fcd70 con 0x561e8e914ab0 2024-09-19T08:51:21.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.020+0000 7f3a35d9b640 1 --2- 172.21.15.164:0/1414393551 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x561e8e914ab0 0x561e8e916f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:21.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.020+0000 7f3a1dffb640 10 client.16614 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:51:21.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.020+0000 7f3a35d9b640 1 --2- 172.21.15.164:0/1414393551 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x561e8e914ab0 0x561e8e916f10 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=23f9425773009bfe server_cookie=886705d4a5c5c462 in_seq=0 out_seq=0 2024-09-19T08:51:21.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.020+0000 7f3a1dffb640 1 -- 172.21.15.164:0/1414393551 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561e8e7fcd70 con 0x561e8e914ab0 2024-09-19T08:51:21.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.020+0000 7f3a1dffb640 10 client.16614 handle_command_reply: tid=0 2024-09-19T08:51:21.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.020+0000 7f3a3ed26740 1 client.16614 shutdown 2024-09-19T08:51:21.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.021+0000 7f3a3ed26740 1 -- 172.21.15.164:0/1414393551 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8e88f5d0 msgr2=0x561e8e8a26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:21.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.021+0000 7f3a3ed26740 1 --2- 172.21.15.164:0/1414393551 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8e88f5d0 0x561e8e8a26f0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f3a280cdfe0 tx=0x7f3a2809ac30 comp rx=0 tx=0).stop 2024-09-19T08:51:21.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.021+0000 7f3a3ed26740 1 -- 172.21.15.164:0/1414393551 shutdown_connections 2024-09-19T08:51:21.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.021+0000 7f3a3ed26740 1 --2- 172.21.15.164:0/1414393551 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x561e8e914ab0 0x561e8e916f10 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:21.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.021+0000 7f3a3ed26740 1 --2- 172.21.15.164:0/1414393551 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e8e89a370 0x561e8e893940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:21.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.021+0000 7f3a3ed26740 1 --2- 172.21.15.164:0/1414393551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e8e890b40 0x561e8e8a2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:21.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.021+0000 7f3a3ed26740 1 --2- 172.21.15.164:0/1414393551 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e8e88f5d0 0x561e8e8a26f0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:21.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.021+0000 7f3a1dffb640 0 client.16614 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:51:21.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.021+0000 7f3a3ed26740 1 -- 172.21.15.164:0/1414393551 >> 172.21.15.164:0/1414393551 conn(0x561e8e858310 msgr2=0x561e8e891930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:21.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.021+0000 7f3a3ed26740 1 -- 172.21.15.164:0/1414393551 shutdown_connections 2024-09-19T08:51:21.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.021+0000 7f3a3ed26740 1 -- 172.21.15.164:0/1414393551 wait complete. 2024-09-19T08:51:21.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.021+0000 7f3a3ed26740 20 client.16614 trim_cache size 0 max 16384 2024-09-19T08:51:21.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.021+0000 7f3a3c94d640 1 -- 172.21.15.164:0/4073023931 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3a200a33b0 msgr2=0x7f3a200a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:21.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.021+0000 7f3a3c94d640 1 --2- 172.21.15.164:0/4073023931 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3a200a33b0 0x7f3a200a5870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3a30031550 tx=0x7f3a30005d50 comp rx=0 tx=0).stop 2024-09-19T08:51:21.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.021+0000 7f3a3c94d640 1 -- 172.21.15.164:0/4073023931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a38102d30 msgr2=0x7f3a381984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:21.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.021+0000 7f3a3c94d640 1 --2- 172.21.15.164:0/4073023931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a38102d30 0x7f3a381984d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3a2800cce0 tx=0x7f3a2800a620 comp rx=0 tx=0).stop 2024-09-19T08:51:21.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.021+0000 7f3a3c94d640 1 -- 172.21.15.164:0/4073023931 shutdown_connections 2024-09-19T08:51:21.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.021+0000 7f3a3c94d640 1 --2- 172.21.15.164:0/4073023931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a381082a0 0x7f3a3819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:21.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.021+0000 7f3a3c94d640 1 --2- 172.21.15.164:0/4073023931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a381036e0 0x7f3a38198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:21.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.021+0000 7f3a3c94d640 1 --2- 172.21.15.164:0/4073023931 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3a200a33b0 0x7f3a200a5870 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:21.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.021+0000 7f3a3c94d640 1 --2- 172.21.15.164:0/4073023931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a38102d30 0x7f3a381984d0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:21.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.021+0000 7f3a3c94d640 1 -- 172.21.15.164:0/4073023931 >> 172.21.15.164:0/4073023931 conn(0x7f3a380fc7d0 msgr2=0x7f3a38106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:21.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.021+0000 7f3a3c94d640 1 -- 172.21.15.164:0/4073023931 shutdown_connections 2024-09-19T08:51:21.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:21.021+0000 7f3a3c94d640 1 -- 172.21.15.164:0/4073023931 wait complete. 2024-09-19T08:51:21.697 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T08:51:21.954 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:51:22.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:21 smithi164 ceph-mon[30928]: pgmap v169: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 87 MiB/s wr, 122 op/s 2024-09-19T08:51:22.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:21 smithi170 ceph-mon[35072]: pgmap v169: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 87 MiB/s wr, 122 op/s 2024-09-19T08:51:22.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:21 smithi173 ceph-mon[35306]: pgmap v169: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 87 MiB/s wr, 122 op/s 2024-09-19T08:51:22.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.447+0000 7ffbec1ce640 1 -- 172.21.15.164:0/28247297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbe4100b30 msgr2=0x7ffbe4100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:22.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.447+0000 7ffbec1ce640 1 --2- 172.21.15.164:0/28247297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbe4100b30 0x7ffbe4100f10 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0009900 tx=0x7ffbe002a470 comp rx=0 tx=0).stop 2024-09-19T08:51:22.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.447+0000 7ffbec1ce640 1 -- 172.21.15.164:0/28247297 shutdown_connections 2024-09-19T08:51:22.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.447+0000 7ffbec1ce640 1 --2- 172.21.15.164:0/28247297 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbe4105f40 0x7ffbe41063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:22.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.447+0000 7ffbec1ce640 1 --2- 172.21.15.164:0/28247297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbe4101450 0x7ffbe4105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:22.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.447+0000 7ffbec1ce640 1 --2- 172.21.15.164:0/28247297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbe4100b30 0x7ffbe4100f10 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:22.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.447+0000 7ffbec1ce640 1 -- 172.21.15.164:0/28247297 >> 172.21.15.164:0/28247297 conn(0x7ffbe40fc7a0 msgr2=0x7ffbe40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:22.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.447+0000 7ffbec1ce640 1 -- 172.21.15.164:0/28247297 shutdown_connections 2024-09-19T08:51:22.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.447+0000 7ffbec1ce640 1 -- 172.21.15.164:0/28247297 wait complete. 2024-09-19T08:51:22.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.448+0000 7ffbec1ce640 1 Processor -- start 2024-09-19T08:51:22.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.448+0000 7ffbec1ce640 1 -- start start 2024-09-19T08:51:22.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.448+0000 7ffbec1ce640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbe4100b30 0x7ffbe41a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:22.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.448+0000 7ffbec1ce640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbe4101450 0x7ffbe41a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:22.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.448+0000 7ffbec1ce640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbe4105f40 0x7ffbe419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:22.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.448+0000 7ffbec1ce640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe41185d0 con 0x7ffbe4100b30 2024-09-19T08:51:22.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.448+0000 7ffbec1ce640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe4118450 con 0x7ffbe4105f40 2024-09-19T08:51:22.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.448+0000 7ffbec1ce640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe4118750 con 0x7ffbe4101450 2024-09-19T08:51:22.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.448+0000 7ffbe9f6a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbe4100b30 0x7ffbe41a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:22.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.448+0000 7ffbe9f6a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbe4100b30 0x7ffbe41a46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45218/0 (socket says 172.21.15.164:45218) 2024-09-19T08:51:22.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.448+0000 7ffbe9f6a640 1 -- 172.21.15.164:0/274780440 learned_addr learned my addr 172.21.15.164:0/274780440 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:22.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.449+0000 7ffbea76b640 1 --2- 172.21.15.164:0/274780440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbe4105f40 0x7ffbe419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:22.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.449+0000 7ffbe9f6a640 1 -- 172.21.15.164:0/274780440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbe4101450 msgr2=0x7ffbe41a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:22.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.449+0000 7ffbe9f6a640 1 --2- 172.21.15.164:0/274780440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbe4101450 0x7ffbe41a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:22.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.449+0000 7ffbe9f6a640 1 -- 172.21.15.164:0/274780440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbe4105f40 msgr2=0x7ffbe419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:22.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.449+0000 7ffbe9f6a640 1 --2- 172.21.15.164:0/274780440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbe4105f40 0x7ffbe419e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:22.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.449+0000 7ffbe9f6a640 1 -- 172.21.15.164:0/274780440 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe419efc0 con 0x7ffbe4100b30 2024-09-19T08:51:22.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.449+0000 7ffbea76b640 1 --2- 172.21.15.164:0/274780440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbe4105f40 0x7ffbe419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:51:22.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.449+0000 7ffbe9f6a640 1 --2- 172.21.15.164:0/274780440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbe4100b30 0x7ffbe41a46d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0006010 tx=0x7ffbe00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:22.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.449+0000 7ffbdaffd640 1 -- 172.21.15.164:0/274780440 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe003f030 con 0x7ffbe4100b30 2024-09-19T08:51:22.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.449+0000 7ffbdaffd640 1 -- 172.21.15.164:0/274780440 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbe0003ce0 con 0x7ffbe4100b30 2024-09-19T08:51:22.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.449+0000 7ffbec1ce640 1 -- 172.21.15.164:0/274780440 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbe419f250 con 0x7ffbe4100b30 2024-09-19T08:51:22.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.449+0000 7ffbdaffd640 1 -- 172.21.15.164:0/274780440 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe0035070 con 0x7ffbe4100b30 2024-09-19T08:51:22.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.449+0000 7ffbec1ce640 1 -- 172.21.15.164:0/274780440 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbe41ab5a0 con 0x7ffbe4100b30 2024-09-19T08:51:22.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.451+0000 7ffbdaffd640 1 -- 172.21.15.164:0/274780440 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7ffbe0042030 con 0x7ffbe4100b30 2024-09-19T08:51:22.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.451+0000 7ffbee5a7740 1 Processor -- start 2024-09-19T08:51:22.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.452+0000 7ffbee5a7740 1 -- start start 2024-09-19T08:51:22.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.452+0000 7ffbee5a7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55818aa2ab40 0x55818aa29090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:22.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.452+0000 7ffbee5a7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55818aa295d0 0x55818aa29a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:22.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.452+0000 7ffbee5a7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55818aa34370 0x55818aa36760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:22.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.452+0000 7ffbee5a7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55818a98f2f0 con 0x55818aa295d0 2024-09-19T08:51:22.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.452+0000 7ffbee5a7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55818a936cf0 con 0x55818aa2ab40 2024-09-19T08:51:22.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.452+0000 7ffbee5a7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55818a98b2b0 con 0x55818aa34370 2024-09-19T08:51:22.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.452+0000 7ffbe9f6a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55818aa295d0 0x55818aa29a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:22.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.452+0000 7ffbdaffd640 1 --2- 172.21.15.164:0/274780440 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ffbd00a33b0 0x7ffbd00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:22.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.452+0000 7ffbe9f6a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55818aa295d0 0x55818aa29a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45234/0 (socket says 172.21.15.164:45234) 2024-09-19T08:51:22.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.452+0000 7ffbe9f6a640 1 -- 172.21.15.164:0/193205104 learned_addr learned my addr 172.21.15.164:0/193205104 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:22.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.452+0000 7ffbea76b640 1 --2- 172.21.15.164:0/193205104 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55818aa2ab40 0x55818aa29090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:22.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.452+0000 7ffbe9769640 1 --2- 172.21.15.164:0/193205104 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55818aa34370 0x55818aa36760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:22.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.452+0000 7ffbea76b640 1 --2- 172.21.15.164:0/274780440 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ffbd00a33b0 0x7ffbd00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:22.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.452+0000 7ffbdaffd640 1 -- 172.21.15.164:0/274780440 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7ffbe00f0480 con 0x7ffbe4100b30 2024-09-19T08:51:22.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.452+0000 7ffbe9f6a640 1 -- 172.21.15.164:0/193205104 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55818aa34370 msgr2=0x55818aa36760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:22.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.452+0000 7ffbe9f6a640 1 --2- 172.21.15.164:0/193205104 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55818aa34370 0x55818aa36760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:22.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.453+0000 7ffbe9f6a640 1 -- 172.21.15.164:0/193205104 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55818aa2ab40 msgr2=0x55818aa29090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:22.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.453+0000 7ffbe9f6a640 1 --2- 172.21.15.164:0/193205104 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55818aa2ab40 0x55818aa29090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:22.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.453+0000 7ffbe9f6a640 1 -- 172.21.15.164:0/193205104 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55818a91b3a0 con 0x55818aa295d0 2024-09-19T08:51:22.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.453+0000 7ffbe9f6a640 1 --2- 172.21.15.164:0/193205104 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55818aa295d0 0x55818aa29a10 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ffbe00b9200 tx=0x7ffbe002a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b435fc79b7d02f4f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:22.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.453+0000 7ffbea76b640 1 --2- 172.21.15.164:0/274780440 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ffbd00a33b0 0x7ffbd00a5870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc009440 tx=0x7ffbdc0093d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:22.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.453+0000 7ffbcffff640 1 -- 172.21.15.164:0/193205104 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe0049070 con 0x55818aa295d0 2024-09-19T08:51:22.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.453+0000 7ffbcffff640 1 -- 172.21.15.164:0/193205104 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbe003b580 con 0x55818aa295d0 2024-09-19T08:51:22.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.453+0000 7ffbcffff640 1 -- 172.21.15.164:0/193205104 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe00c54a0 con 0x55818aa295d0 2024-09-19T08:51:22.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.454+0000 7ffbee5a7740 1 -- 172.21.15.164:0/193205104 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55818aa295d0 msgr2=0x55818aa29a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:22.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.454+0000 7ffbee5a7740 1 --2- 172.21.15.164:0/193205104 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55818aa295d0 0x55818aa29a10 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ffbe00b9200 tx=0x7ffbe002a470 comp rx=0 tx=0).stop 2024-09-19T08:51:22.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.454+0000 7ffbee5a7740 1 -- 172.21.15.164:0/193205104 shutdown_connections 2024-09-19T08:51:22.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.454+0000 7ffbee5a7740 1 --2- 172.21.15.164:0/193205104 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55818aa34370 0x55818aa36760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:22.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.454+0000 7ffbee5a7740 1 --2- 172.21.15.164:0/193205104 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55818aa295d0 0x55818aa29a10 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:22.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.454+0000 7ffbee5a7740 1 --2- 172.21.15.164:0/193205104 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55818aa2ab40 0x55818aa29090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:22.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.454+0000 7ffbee5a7740 1 -- 172.21.15.164:0/193205104 >> 172.21.15.164:0/193205104 conn(0x55818a9f2310 msgr2=0x55818aa25c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:22.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.454+0000 7ffbee5a7740 1 -- 172.21.15.164:0/193205104 shutdown_connections 2024-09-19T08:51:22.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.454+0000 7ffbee5a7740 1 -- 172.21.15.164:0/193205104 wait complete. 2024-09-19T08:51:22.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.454+0000 7ffbee5a7740 1 -- start start 2024-09-19T08:51:22.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.454+0000 7ffbee5a7740 1 Processor -- start 2024-09-19T08:51:22.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.455+0000 7ffbee5a7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55818aa295d0 0x55818aa33db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:22.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.455+0000 7ffbee5a7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55818aa2ab40 0x55818aa31060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:22.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.455+0000 7ffbee5a7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55818aa34370 0x55818aa317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:22.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.455+0000 7ffbee5a7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55818aa36e30 con 0x55818aa295d0 2024-09-19T08:51:22.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.455+0000 7ffbee5a7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55818a94a060 con 0x55818aa2ab40 2024-09-19T08:51:22.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.455+0000 7ffbee5a7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55818aa36fb0 con 0x55818aa34370 2024-09-19T08:51:22.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.455+0000 7ffbee5a7740 10 client.0 _ll_register_callbacks cb 0x55818a69d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:51:22.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.455+0000 7ffbe9f6a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55818aa295d0 0x55818aa33db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:22.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.455+0000 7ffbe9f6a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55818aa295d0 0x55818aa33db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45250/0 (socket says 172.21.15.164:45250) 2024-09-19T08:51:22.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.455+0000 7ffbe9f6a640 1 -- 172.21.15.164:0/743961918 learned_addr learned my addr 172.21.15.164:0/743961918 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:22.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.455+0000 7ffbe9f6a640 1 -- 172.21.15.164:0/743961918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55818aa34370 msgr2=0x55818aa317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:22.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.455+0000 7ffbe9f6a640 1 --2- 172.21.15.164:0/743961918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55818aa34370 0x55818aa317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:22.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.455+0000 7ffbe9f6a640 1 -- 172.21.15.164:0/743961918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55818aa2ab40 msgr2=0x55818aa31060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:22.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.455+0000 7ffbe9f6a640 1 --2- 172.21.15.164:0/743961918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55818aa2ab40 0x55818aa31060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:22.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.455+0000 7ffbe9f6a640 1 -- 172.21.15.164:0/743961918 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55818a7003c0 con 0x55818aa295d0 2024-09-19T08:51:22.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.455+0000 7ffbe9f6a640 1 --2- 172.21.15.164:0/743961918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55818aa295d0 0x55818aa33db0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ffbe003d040 tx=0x7ffbe00bc210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9a04385eae81d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:22.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.455+0000 7ffbcd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:51:22.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.455+0000 7ffbcd7fa640 1 -- 172.21.15.164:0/743961918 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe00d9020 con 0x55818aa295d0 2024-09-19T08:51:22.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.455+0000 7ffbcd7fa640 1 -- 172.21.15.164:0/743961918 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7ffbe00ede60 con 0x55818aa295d0 2024-09-19T08:51:22.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.455+0000 7ffbee5a7740 1 -- 172.21.15.164:0/743961918 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55818a99c860 con 0x55818aa295d0 2024-09-19T08:51:22.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.455+0000 7ffbcd7fa640 1 -- 172.21.15.164:0/743961918 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffbe0049070 con 0x55818aa295d0 2024-09-19T08:51:22.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.455+0000 7ffbee5a7740 10 client.16626 fetch_fsmap learned FSMap version 30 2024-09-19T08:51:22.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.455+0000 7ffbee5a7740 1 -- 172.21.15.164:0/743961918 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55818aa36e30 con 0x55818aa295d0 2024-09-19T08:51:22.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.456+0000 7ffbcd7fa640 1 -- 172.21.15.164:0/743961918 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55818a99c860 con 0x55818aa295d0 2024-09-19T08:51:22.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.456+0000 7ffbcd7fa640 1 -- 172.21.15.164:0/743961918 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7ffbe00c5010 con 0x55818aa295d0 2024-09-19T08:51:22.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.456+0000 7ffbee5a7740 10 client.16626 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:51:22.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.456+0000 7ffbee5a7740 10 client.16626 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T08:51:22.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.456+0000 7ffbee5a7740 20 client.16626 populate_metadata read hostname 'smithi164' 2024-09-19T08:51:22.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.456+0000 7ffbee5a7740 1 --2- 172.21.15.164:0/743961918 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55818aa43990 0x55818aa45d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:22.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.456+0000 7ffbee5a7740 4 client.16626 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:51:22.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.456+0000 7ffbee5a7740 1 -- 172.21.15.164:0/743961918 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55818a996d70 con 0x55818aa43990 2024-09-19T08:51:22.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.456+0000 7ffbe9769640 1 --2- 172.21.15.164:0/743961918 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55818aa43990 0x55818aa45d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:22.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.457+0000 7ffbe9769640 1 --2- 172.21.15.164:0/743961918 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55818aa43990 0x55818aa45d80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=da5884426947b32e server_cookie=7b63c53538c328c7 in_seq=0 out_seq=0 2024-09-19T08:51:22.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.457+0000 7ffbcd7fa640 10 client.16626 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:51:22.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.457+0000 7ffbcd7fa640 1 -- 172.21.15.164:0/743961918 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55818a996d70 con 0x55818aa43990 2024-09-19T08:51:22.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.457+0000 7ffbcd7fa640 10 client.16626 handle_command_reply: tid=0 2024-09-19T08:51:22.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.457+0000 7ffbee5a7740 1 client.16626 shutdown 2024-09-19T08:51:22.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.457+0000 7ffbee5a7740 1 -- 172.21.15.164:0/743961918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55818aa295d0 msgr2=0x55818aa33db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:22.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.457+0000 7ffbee5a7740 1 --2- 172.21.15.164:0/743961918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55818aa295d0 0x55818aa33db0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ffbe003d040 tx=0x7ffbe00bc210 comp rx=0 tx=0).stop 2024-09-19T08:51:22.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.458+0000 7ffbee5a7740 1 -- 172.21.15.164:0/743961918 shutdown_connections 2024-09-19T08:51:22.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.458+0000 7ffbee5a7740 1 --2- 172.21.15.164:0/743961918 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55818aa43990 0x55818aa45d80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:22.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.458+0000 7ffbee5a7740 1 --2- 172.21.15.164:0/743961918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55818aa34370 0x55818aa317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:22.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.458+0000 7ffbee5a7740 1 --2- 172.21.15.164:0/743961918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55818aa2ab40 0x55818aa31060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:22.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.458+0000 7ffbcd7fa640 0 client.16626 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:51:22.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.458+0000 7ffbee5a7740 1 --2- 172.21.15.164:0/743961918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55818aa295d0 0x55818aa33db0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:22.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.458+0000 7ffbee5a7740 1 -- 172.21.15.164:0/743961918 >> 172.21.15.164:0/743961918 conn(0x55818a9f2310 msgr2=0x55818aa2b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:22.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.458+0000 7ffbee5a7740 1 -- 172.21.15.164:0/743961918 shutdown_connections 2024-09-19T08:51:22.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.458+0000 7ffbee5a7740 1 -- 172.21.15.164:0/743961918 wait complete. 2024-09-19T08:51:22.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.458+0000 7ffbee5a7740 20 client.16626 trim_cache size 0 max 16384 2024-09-19T08:51:22.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.470+0000 7ffbee5a7740 1 Processor -- start 2024-09-19T08:51:22.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.470+0000 7ffbee5a7740 1 -- start start 2024-09-19T08:51:22.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.470+0000 7ffbee5a7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55818aa295d0 0x55818aa2a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:22.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.471+0000 7ffbee5a7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55818aa2ab40 0x55818aa30040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:22.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.471+0000 7ffbee5a7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55818aa34370 0x55818aa4b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:22.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.471+0000 7ffbee5a7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe41b3d50 con 0x55818aa34370 2024-09-19T08:51:22.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.471+0000 7ffbee5a7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe41b5950 con 0x55818aa295d0 2024-09-19T08:51:22.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.471+0000 7ffbee5a7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe41b4420 con 0x55818aa2ab40 2024-09-19T08:51:22.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.471+0000 7ffbea76b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55818aa34370 0x55818aa4b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:22.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.471+0000 7ffbea76b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55818aa34370 0x55818aa4b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45262/0 (socket says 172.21.15.164:45262) 2024-09-19T08:51:22.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.471+0000 7ffbea76b640 1 -- 172.21.15.164:0/4008803696 learned_addr learned my addr 172.21.15.164:0/4008803696 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:22.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.471+0000 7ffbea76b640 1 -- 172.21.15.164:0/4008803696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55818aa2ab40 msgr2=0x55818aa30040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:22.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.471+0000 7ffbea76b640 1 --2- 172.21.15.164:0/4008803696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55818aa2ab40 0x55818aa30040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:22.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.471+0000 7ffbea76b640 1 -- 172.21.15.164:0/4008803696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55818aa295d0 msgr2=0x55818aa2a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:22.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.471+0000 7ffbe9f6a640 1 --2- 172.21.15.164:0/4008803696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55818aa295d0 0x55818aa2a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:22.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.471+0000 7ffbea76b640 1 --2- 172.21.15.164:0/4008803696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55818aa295d0 0x55818aa2a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:22.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.471+0000 7ffbea76b640 1 -- 172.21.15.164:0/4008803696 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55818aa26960 con 0x55818aa34370 2024-09-19T08:51:22.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.471+0000 7ffbe9f6a640 1 --2- 172.21.15.164:0/4008803696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55818aa295d0 0x55818aa2a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:51:22.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.471+0000 7ffbea76b640 1 --2- 172.21.15.164:0/4008803696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55818aa34370 0x55818aa4b5e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc009c20 tx=0x7ffbdc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94c30862cddf7b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:22.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.471+0000 7ffbcf7fe640 1 -- 172.21.15.164:0/4008803696 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc029070 con 0x55818aa34370 2024-09-19T08:51:22.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.471+0000 7ffbcf7fe640 1 -- 172.21.15.164:0/4008803696 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbdc00a980 con 0x55818aa34370 2024-09-19T08:51:22.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.471+0000 7ffbcf7fe640 1 -- 172.21.15.164:0/4008803696 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc013040 con 0x55818aa34370 2024-09-19T08:51:22.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.472+0000 7ffbee5a7740 1 -- 172.21.15.164:0/4008803696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55818aa34370 msgr2=0x55818aa4b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:22.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.472+0000 7ffbee5a7740 1 --2- 172.21.15.164:0/4008803696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55818aa34370 0x55818aa4b5e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc009c20 tx=0x7ffbdc004a40 comp rx=0 tx=0).stop 2024-09-19T08:51:22.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.472+0000 7ffbee5a7740 1 -- 172.21.15.164:0/4008803696 shutdown_connections 2024-09-19T08:51:22.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.472+0000 7ffbee5a7740 1 --2- 172.21.15.164:0/4008803696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55818aa34370 0x55818aa4b5e0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:22.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.472+0000 7ffbee5a7740 1 --2- 172.21.15.164:0/4008803696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55818aa2ab40 0x55818aa30040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:22.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.472+0000 7ffbee5a7740 1 --2- 172.21.15.164:0/4008803696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55818aa295d0 0x55818aa2a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:22.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.472+0000 7ffbee5a7740 1 -- 172.21.15.164:0/4008803696 >> 172.21.15.164:0/4008803696 conn(0x55818a9f2310 msgr2=0x55818aa2f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:22.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.472+0000 7ffbee5a7740 1 -- 172.21.15.164:0/4008803696 shutdown_connections 2024-09-19T08:51:22.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.472+0000 7ffbee5a7740 1 -- 172.21.15.164:0/4008803696 wait complete. 2024-09-19T08:51:22.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.472+0000 7ffbee5a7740 1 -- start start 2024-09-19T08:51:22.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.472+0000 7ffbee5a7740 1 Processor -- start 2024-09-19T08:51:22.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.473+0000 7ffbee5a7740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55818aa295d0 0x55818aa3c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:22.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.473+0000 7ffbee5a7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55818aa2ab40 0x55818aa37940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:22.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.473+0000 7ffbee5a7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55818aa34370 0x55818aa37e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:22.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.473+0000 7ffbee5a7740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe41b51c0 con 0x55818aa295d0 2024-09-19T08:51:22.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.473+0000 7ffbee5a7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe41b4af0 con 0x55818aa2ab40 2024-09-19T08:51:22.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.473+0000 7ffbee5a7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe41b1d30 con 0x55818aa34370 2024-09-19T08:51:22.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.473+0000 7ffbee5a7740 10 client.0 _ll_register_callbacks cb 0x7ffbdc009ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:51:22.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.473+0000 7ffbe9f6a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55818aa295d0 0x55818aa3c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:22.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.473+0000 7ffbe9f6a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55818aa295d0 0x55818aa3c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45276/0 (socket says 172.21.15.164:45276) 2024-09-19T08:51:22.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.473+0000 7ffbe9f6a640 1 -- 172.21.15.164:0/4292047159 learned_addr learned my addr 172.21.15.164:0/4292047159 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:22.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.473+0000 7ffbea76b640 1 --2- 172.21.15.164:0/4292047159 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55818aa34370 0x55818aa37e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:22.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.473+0000 7ffbe9769640 1 --2- 172.21.15.164:0/4292047159 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55818aa2ab40 0x55818aa37940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:22.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.473+0000 7ffbe9f6a640 1 -- 172.21.15.164:0/4292047159 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55818aa34370 msgr2=0x55818aa37e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:22.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.473+0000 7ffbe9f6a640 1 --2- 172.21.15.164:0/4292047159 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55818aa34370 0x55818aa37e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:22.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.473+0000 7ffbe9f6a640 1 -- 172.21.15.164:0/4292047159 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55818aa2ab40 msgr2=0x55818aa37940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:22.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.473+0000 7ffbe9f6a640 1 --2- 172.21.15.164:0/4292047159 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55818aa2ab40 0x55818aa37940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:22.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.473+0000 7ffbe9f6a640 1 -- 172.21.15.164:0/4292047159 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55818aa383f0 con 0x55818aa295d0 2024-09-19T08:51:22.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.473+0000 7ffbe9f6a640 1 --2- 172.21.15.164:0/4292047159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55818aa295d0 0x55818aa3c610 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0005940 tx=0x7ffbe00ee110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3bd1d401aa4e9a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:22.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.473+0000 7ffbcd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:51:22.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.474+0000 7ffbcd7fa640 1 -- 172.21.15.164:0/4292047159 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe00d9070 con 0x55818aa295d0 2024-09-19T08:51:22.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.474+0000 7ffbcd7fa640 1 -- 172.21.15.164:0/4292047159 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7ffbe0081dc0 con 0x55818aa295d0 2024-09-19T08:51:22.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.474+0000 7ffbee5a7740 1 -- 172.21.15.164:0/4292047159 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbdc00a980 con 0x55818aa295d0 2024-09-19T08:51:22.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.474+0000 7ffbcd7fa640 1 -- 172.21.15.164:0/4292047159 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffbe00bc600 con 0x55818aa295d0 2024-09-19T08:51:22.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.474+0000 7ffbee5a7740 10 client.16632 fetch_fsmap learned FSMap version 30 2024-09-19T08:51:22.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.474+0000 7ffbee5a7740 1 -- 172.21.15.164:0/4292047159 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55818aa383f0 con 0x55818aa295d0 2024-09-19T08:51:22.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.474+0000 7ffbcd7fa640 1 -- 172.21.15.164:0/4292047159 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbdc00a980 con 0x55818aa295d0 2024-09-19T08:51:22.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.474+0000 7ffbcd7fa640 1 -- 172.21.15.164:0/4292047159 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7ffbe00bdcd0 con 0x55818aa295d0 2024-09-19T08:51:22.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.474+0000 7ffbee5a7740 10 client.16632 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:51:22.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.474+0000 7ffbee5a7740 10 client.16632 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T08:51:22.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.474+0000 7ffbee5a7740 20 client.16632 populate_metadata read hostname 'smithi164' 2024-09-19T08:51:22.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.474+0000 7ffbee5a7740 1 --2- 172.21.15.164:0/4292047159 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55818aaae5e0 0x55818aab09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:22.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.474+0000 7ffbee5a7740 4 client.16632 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:51:22.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.474+0000 7ffbee5a7740 1 -- 172.21.15.164:0/4292047159 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55818a913170 con 0x55818aaae5e0 2024-09-19T08:51:22.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.474+0000 7ffbe9769640 1 --2- 172.21.15.164:0/4292047159 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55818aaae5e0 0x55818aab09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:22.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.475+0000 7ffbe9769640 1 --2- 172.21.15.164:0/4292047159 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55818aaae5e0 0x55818aab09d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=12005f37e4d5824e server_cookie=f857af9e283ab031 in_seq=0 out_seq=0 2024-09-19T08:51:22.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.475+0000 7ffbcd7fa640 10 client.16632 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:51:22.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.475+0000 7ffbcd7fa640 1 -- 172.21.15.164:0/4292047159 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55818a913170 con 0x55818aaae5e0 2024-09-19T08:51:22.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.475+0000 7ffbcd7fa640 10 client.16632 handle_command_reply: tid=0 2024-09-19T08:51:22.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.475+0000 7ffbee5a7740 1 client.16632 shutdown 2024-09-19T08:51:22.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.475+0000 7ffbee5a7740 1 -- 172.21.15.164:0/4292047159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55818aa295d0 msgr2=0x55818aa3c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:22.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.475+0000 7ffbee5a7740 1 --2- 172.21.15.164:0/4292047159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55818aa295d0 0x55818aa3c610 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0005940 tx=0x7ffbe00ee110 comp rx=0 tx=0).stop 2024-09-19T08:51:22.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.475+0000 7ffbee5a7740 1 -- 172.21.15.164:0/4292047159 shutdown_connections 2024-09-19T08:51:22.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.475+0000 7ffbee5a7740 1 --2- 172.21.15.164:0/4292047159 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55818aaae5e0 0x55818aab09d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:22.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.475+0000 7ffbee5a7740 1 --2- 172.21.15.164:0/4292047159 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55818aa34370 0x55818aa37e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:22.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.475+0000 7ffbee5a7740 1 --2- 172.21.15.164:0/4292047159 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55818aa2ab40 0x55818aa37940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:22.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.475+0000 7ffbee5a7740 1 --2- 172.21.15.164:0/4292047159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55818aa295d0 0x55818aa3c610 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:22.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.475+0000 7ffbcd7fa640 0 client.16632 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:51:22.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.475+0000 7ffbee5a7740 1 -- 172.21.15.164:0/4292047159 >> 172.21.15.164:0/4292047159 conn(0x55818a9f2310 msgr2=0x55818aa2b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:22.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.475+0000 7ffbee5a7740 1 -- 172.21.15.164:0/4292047159 shutdown_connections 2024-09-19T08:51:22.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.475+0000 7ffbee5a7740 1 -- 172.21.15.164:0/4292047159 wait complete. 2024-09-19T08:51:22.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.475+0000 7ffbee5a7740 20 client.16632 trim_cache size 0 max 16384 2024-09-19T08:51:22.492 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:51:22.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.476+0000 7ffbec1ce640 1 -- 172.21.15.164:0/274780440 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ffbd00a33b0 msgr2=0x7ffbd00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:22.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.476+0000 7ffbec1ce640 1 --2- 172.21.15.164:0/274780440 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ffbd00a33b0 0x7ffbd00a5870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc009440 tx=0x7ffbdc0093d0 comp rx=0 tx=0).stop 2024-09-19T08:51:22.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.476+0000 7ffbec1ce640 1 -- 172.21.15.164:0/274780440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbe4100b30 msgr2=0x7ffbe41a46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:22.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.476+0000 7ffbec1ce640 1 --2- 172.21.15.164:0/274780440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbe4100b30 0x7ffbe41a46d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0006010 tx=0x7ffbe00042f0 comp rx=0 tx=0).stop 2024-09-19T08:51:22.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.476+0000 7ffbec1ce640 1 -- 172.21.15.164:0/274780440 shutdown_connections 2024-09-19T08:51:22.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.476+0000 7ffbec1ce640 1 --2- 172.21.15.164:0/274780440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbe4105f40 0x7ffbe419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:22.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.476+0000 7ffbec1ce640 1 --2- 172.21.15.164:0/274780440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbe4101450 0x7ffbe41a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:22.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.476+0000 7ffbec1ce640 1 --2- 172.21.15.164:0/274780440 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ffbd00a33b0 0x7ffbd00a5870 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:22.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.476+0000 7ffbec1ce640 1 --2- 172.21.15.164:0/274780440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbe4100b30 0x7ffbe41a46d0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:22.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.476+0000 7ffbec1ce640 1 -- 172.21.15.164:0/274780440 >> 172.21.15.164:0/274780440 conn(0x7ffbe40fc7a0 msgr2=0x7ffbe410f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:22.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.476+0000 7ffbec1ce640 1 -- 172.21.15.164:0/274780440 shutdown_connections 2024-09-19T08:51:22.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:22.476+0000 7ffbec1ce640 1 -- 172.21.15.164:0/274780440 wait complete. 2024-09-19T08:51:23.214 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T08:51:23.461 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:51:23.906 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:23 smithi164 ceph-mon[30928]: pgmap v170: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 73 MiB/s wr, 112 op/s 2024-09-19T08:51:24.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.084+0000 7fcac644e640 1 -- 172.21.15.164:0/1327475163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcac0109450 msgr2=0x7fcac0110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:24.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.084+0000 7fcac644e640 1 --2- 172.21.15.164:0/1327475163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcac0109450 0x7fcac0110250 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009900 tx=0x7fcab002a410 comp rx=0 tx=0).stop 2024-09-19T08:51:24.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.085+0000 7fcac644e640 1 -- 172.21.15.164:0/1327475163 shutdown_connections 2024-09-19T08:51:24.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.085+0000 7fcac644e640 1 --2- 172.21.15.164:0/1327475163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcac011b140 0x7fcac011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:24.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.085+0000 7fcac644e640 1 --2- 172.21.15.164:0/1327475163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcac0109830 0x7fcac0109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:24.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.085+0000 7fcac644e640 1 --2- 172.21.15.164:0/1327475163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcac0109450 0x7fcac0110250 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:24.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.085+0000 7fcac644e640 1 -- 172.21.15.164:0/1327475163 >> 172.21.15.164:0/1327475163 conn(0x7fcac006e140 msgr2=0x7fcac006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:24.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.089+0000 7fcac644e640 1 -- 172.21.15.164:0/1327475163 shutdown_connections 2024-09-19T08:51:24.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.096+0000 7fcac644e640 1 -- 172.21.15.164:0/1327475163 wait complete. 2024-09-19T08:51:24.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.097+0000 7fcac644e640 1 Processor -- start 2024-09-19T08:51:24.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.097+0000 7fcac644e640 1 -- start start 2024-09-19T08:51:24.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.098+0000 7fcac644e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcac0109450 0x7fcac011a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:24.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.098+0000 7fcac644e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcac0109830 0x7fcac0114940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:24.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.098+0000 7fcac644e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcac011b140 0x7fcac0114e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:24.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.098+0000 7fcac644e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcac0111370 con 0x7fcac0109450 2024-09-19T08:51:24.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.098+0000 7fcac644e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01111f0 con 0x7fcac011b140 2024-09-19T08:51:24.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.098+0000 7fcac644e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01114f0 con 0x7fcac0109830 2024-09-19T08:51:24.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.100+0000 7fcabffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcac0109450 0x7fcac011a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:24.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.100+0000 7fcabffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcac0109450 0x7fcac011a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:32834/0 (socket says 172.21.15.164:32834) 2024-09-19T08:51:24.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.100+0000 7fcabffff640 1 -- 172.21.15.164:0/3674102540 learned_addr learned my addr 172.21.15.164:0/3674102540 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:24.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.104+0000 7fcabffff640 1 -- 172.21.15.164:0/3674102540 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcac0109830 msgr2=0x7fcac0114940 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T08:51:24.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.104+0000 7fcabffff640 1 --2- 172.21.15.164:0/3674102540 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcac0109830 0x7fcac0114940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:24.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.104+0000 7fcabffff640 1 -- 172.21.15.164:0/3674102540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcac011b140 msgr2=0x7fcac0114e80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T08:51:24.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.104+0000 7fcabffff640 1 --2- 172.21.15.164:0/3674102540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcac011b140 0x7fcac0114e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:24.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.104+0000 7fcabffff640 1 -- 172.21.15.164:0/3674102540 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac0115600 con 0x7fcac0109450 2024-09-19T08:51:24.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.108+0000 7fcabffff640 1 --2- 172.21.15.164:0/3674102540 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcac0109450 0x7fcac011a8c0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fcab0006010 tx=0x7fcab00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:24.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.109+0000 7fcabd7fa640 1 -- 172.21.15.164:0/3674102540 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab0040070 con 0x7fcac0109450 2024-09-19T08:51:24.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.109+0000 7fcabd7fa640 1 -- 172.21.15.164:0/3674102540 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab0031c80 con 0x7fcac0109450 2024-09-19T08:51:24.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.109+0000 7fcabd7fa640 1 -- 172.21.15.164:0/3674102540 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab0035020 con 0x7fcac0109450 2024-09-19T08:51:24.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.111+0000 7fcac644e640 1 -- 172.21.15.164:0/3674102540 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac01bc0e0 con 0x7fcac0109450 2024-09-19T08:51:24.116 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.112+0000 7fcac644e640 1 -- 172.21.15.164:0/3674102540 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac01bc4e0 con 0x7fcac0109450 2024-09-19T08:51:24.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.118+0000 7fcac8827740 1 Processor -- start 2024-09-19T08:51:24.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.119+0000 7fcac8827740 1 -- start start 2024-09-19T08:51:24.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.119+0000 7fcac8827740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fefc173b40 0x55fefc172090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:24.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.119+0000 7fcac8827740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fefc1725d0 0x55fefc172a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:24.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.119+0000 7fcac8827740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fefc17d370 0x55fefc17f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:24.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.119+0000 7fcac8827740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55fefc0d82f0 con 0x55fefc17d370 2024-09-19T08:51:24.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.119+0000 7fcac8827740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55fefc07fcf0 con 0x55fefc173b40 2024-09-19T08:51:24.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.119+0000 7fcac8827740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55fefc0d42b0 con 0x55fefc1725d0 2024-09-19T08:51:24.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.122+0000 7fcabffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fefc1725d0 0x55fefc172a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:24.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.122+0000 7fcabffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fefc1725d0 0x55fefc172a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:34528/0 (socket says 172.21.15.164:34528) 2024-09-19T08:51:24.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.122+0000 7fcabffff640 1 -- 172.21.15.164:0/1593051853 learned_addr learned my addr 172.21.15.164:0/1593051853 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:24.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.123+0000 7fcac49eb640 1 --2- 172.21.15.164:0/1593051853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fefc173b40 0x55fefc172090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:24.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.123+0000 7fcabf7fe640 1 --2- 172.21.15.164:0/1593051853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fefc17d370 0x55fefc17f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:24.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:23 smithi170 ceph-mon[35072]: pgmap v170: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 73 MiB/s wr, 112 op/s 2024-09-19T08:51:24.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.125+0000 7fcabd7fa640 1 -- 172.21.15.164:0/3674102540 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fcab0043070 con 0x7fcac0109450 2024-09-19T08:51:24.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.125+0000 7fcabf7fe640 1 -- 172.21.15.164:0/1593051853 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fefc1725d0 msgr2=0x55fefc172a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:24.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.125+0000 7fcabf7fe640 1 --2- 172.21.15.164:0/1593051853 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fefc1725d0 0x55fefc172a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:24.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.125+0000 7fcabf7fe640 1 -- 172.21.15.164:0/1593051853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fefc173b40 msgr2=0x55fefc172090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:24.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.125+0000 7fcabf7fe640 1 --2- 172.21.15.164:0/1593051853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fefc173b40 0x55fefc172090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:24.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.125+0000 7fcabf7fe640 1 -- 172.21.15.164:0/1593051853 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fefc0643a0 con 0x55fefc17d370 2024-09-19T08:51:24.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.125+0000 7fcabf7fe640 1 --2- 172.21.15.164:0/1593051853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fefc17d370 0x55fefc17f760 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fcac0116100 tx=0x7fcab4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef29a560f1ae387f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:24.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.126+0000 7fcabd7fa640 1 --2- 172.21.15.164:0/3674102540 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fcaa80a3580 0x7fcaa80a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:24.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.127+0000 7fcac49eb640 1 --2- 172.21.15.164:0/3674102540 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fcaa80a3580 0x7fcaa80a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:24.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.129+0000 7fcac49eb640 1 --2- 172.21.15.164:0/3674102540 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fcaa80a3580 0x7fcaa80a5a40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcab8005fd0 tx=0x7fcab8009890 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:24.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.129+0000 7fcabd7fa640 1 -- 172.21.15.164:0/3674102540 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fcab00f6780 con 0x7fcac0109450 2024-09-19T08:51:24.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.130+0000 7fcaa67fc640 1 -- 172.21.15.164:0/1593051853 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab4015020 con 0x55fefc17d370 2024-09-19T08:51:24.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.130+0000 7fcaa67fc640 1 -- 172.21.15.164:0/1593051853 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab400bd70 con 0x55fefc17d370 2024-09-19T08:51:24.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.131+0000 7fcac8827740 1 -- 172.21.15.164:0/1593051853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fefc17d370 msgr2=0x55fefc17f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:24.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.131+0000 7fcac8827740 1 --2- 172.21.15.164:0/1593051853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fefc17d370 0x55fefc17f760 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fcac0116100 tx=0x7fcab4009b20 comp rx=0 tx=0).stop 2024-09-19T08:51:24.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.133+0000 7fcac8827740 1 -- 172.21.15.164:0/1593051853 shutdown_connections 2024-09-19T08:51:24.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.133+0000 7fcac8827740 1 --2- 172.21.15.164:0/1593051853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fefc17d370 0x55fefc17f760 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:24.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.133+0000 7fcac8827740 1 --2- 172.21.15.164:0/1593051853 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fefc1725d0 0x55fefc172a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:24.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.133+0000 7fcac8827740 1 --2- 172.21.15.164:0/1593051853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fefc173b40 0x55fefc172090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:24.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.133+0000 7fcac8827740 1 -- 172.21.15.164:0/1593051853 >> 172.21.15.164:0/1593051853 conn(0x55fefc13b310 msgr2=0x55fefc16ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:24.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.134+0000 7fcac8827740 1 -- 172.21.15.164:0/1593051853 shutdown_connections 2024-09-19T08:51:24.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.134+0000 7fcac8827740 1 -- 172.21.15.164:0/1593051853 wait complete. 2024-09-19T08:51:24.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.135+0000 7fcac8827740 1 -- start start 2024-09-19T08:51:24.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.135+0000 7fcac8827740 1 Processor -- start 2024-09-19T08:51:24.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.137+0000 7fcac8827740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fefc1725d0 0x55fefc17cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:24.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.137+0000 7fcac8827740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fefc173b40 0x55fefc179f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:24.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.137+0000 7fcac8827740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fefc17d370 0x55fefc17a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:24.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.137+0000 7fcac8827740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55fefc1800c0 con 0x55fefc1725d0 2024-09-19T08:51:24.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.137+0000 7fcac8827740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55fefc093060 con 0x55fefc173b40 2024-09-19T08:51:24.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.137+0000 7fcac8827740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55fefc180240 con 0x55fefc17d370 2024-09-19T08:51:24.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.137+0000 7fcac8827740 10 client.0 _ll_register_callbacks cb 0x55fefbde68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:51:24.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.137+0000 7fcabffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fefc1725d0 0x55fefc17cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:24.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.137+0000 7fcabffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fefc1725d0 0x55fefc17cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:32854/0 (socket says 172.21.15.164:32854) 2024-09-19T08:51:24.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.137+0000 7fcabffff640 1 -- 172.21.15.164:0/3112723286 learned_addr learned my addr 172.21.15.164:0/3112723286 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:24.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.138+0000 7fcabf7fe640 1 --2- 172.21.15.164:0/3112723286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fefc173b40 0x55fefc179f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:24.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.138+0000 7fcac49eb640 1 --2- 172.21.15.164:0/3112723286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fefc17d370 0x55fefc17a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:24.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.139+0000 7fcabffff640 1 -- 172.21.15.164:0/3112723286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fefc17d370 msgr2=0x55fefc17a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:24.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.139+0000 7fcabffff640 1 --2- 172.21.15.164:0/3112723286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fefc17d370 0x55fefc17a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:24.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.139+0000 7fcabffff640 1 -- 172.21.15.164:0/3112723286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fefc173b40 msgr2=0x55fefc179f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:24.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.139+0000 7fcabffff640 1 --2- 172.21.15.164:0/3112723286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fefc173b40 0x55fefc179f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:24.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.139+0000 7fcabffff640 1 -- 172.21.15.164:0/3112723286 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fefbe493c0 con 0x55fefc1725d0 2024-09-19T08:51:24.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.140+0000 7fcabffff640 1 --2- 172.21.15.164:0/3112723286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fefc1725d0 0x55fefc17cc60 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fcab002a450 tx=0x7fcab00ad2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a1a3046d273346 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:24.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.141+0000 7fca9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:51:24.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.141+0000 7fca9f7fe640 1 -- 172.21.15.164:0/3112723286 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab00c25b0 con 0x55fefc1725d0 2024-09-19T08:51:24.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.141+0000 7fcac8827740 1 -- 172.21.15.164:0/3112723286 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcab400bd70 con 0x55fefc1725d0 2024-09-19T08:51:24.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.143+0000 7fca9f7fe640 1 -- 172.21.15.164:0/3112723286 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fcab00c2cd0 con 0x55fefc1725d0 2024-09-19T08:51:24.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.146+0000 7fca9f7fe640 1 -- 172.21.15.164:0/3112723286 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcab00c0bb0 con 0x55fefc1725d0 2024-09-19T08:51:24.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.146+0000 7fcac8827740 10 client.16644 fetch_fsmap learned FSMap version 30 2024-09-19T08:51:24.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.146+0000 7fcac8827740 1 -- 172.21.15.164:0/3112723286 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fefc1800c0 con 0x55fefc1725d0 2024-09-19T08:51:24.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.147+0000 7fca9f7fe640 1 -- 172.21.15.164:0/3112723286 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab400bd70 con 0x55fefc1725d0 2024-09-19T08:51:24.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.148+0000 7fca9f7fe640 1 -- 172.21.15.164:0/3112723286 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fcab00abce0 con 0x55fefc1725d0 2024-09-19T08:51:24.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.148+0000 7fcac8827740 10 client.16644 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:51:24.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.148+0000 7fcac8827740 10 client.16644 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T08:51:24.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.148+0000 7fcac8827740 20 client.16644 populate_metadata read hostname 'smithi164' 2024-09-19T08:51:24.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.148+0000 7fcac8827740 1 --2- 172.21.15.164:0/3112723286 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55fefc18ce20 0x55fefc18f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:24.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.148+0000 7fcac8827740 4 client.16644 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:51:24.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.148+0000 7fcac8827740 1 -- 172.21.15.164:0/3112723286 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fefc0e5860 con 0x55fefc18ce20 2024-09-19T08:51:24.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.154+0000 7fcabf7fe640 1 --2- 172.21.15.164:0/3112723286 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55fefc18ce20 0x55fefc18f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:24.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.158+0000 7fcabf7fe640 1 --2- 172.21.15.164:0/3112723286 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55fefc18ce20 0x55fefc18f2b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9669fe6bb7d7350d server_cookie=8f35e177f7f5f321 in_seq=0 out_seq=0 2024-09-19T08:51:24.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.158+0000 7fca9f7fe640 10 client.16644 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:51:24.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:23 smithi173 ceph-mon[35306]: pgmap v170: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 73 MiB/s wr, 112 op/s 2024-09-19T08:51:24.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.162+0000 7fca9f7fe640 1 -- 172.21.15.164:0/3112723286 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fefc0e5860 con 0x55fefc18ce20 2024-09-19T08:51:24.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.162+0000 7fca9f7fe640 10 client.16644 handle_command_reply: tid=0 2024-09-19T08:51:24.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.162+0000 7fcac8827740 1 client.16644 shutdown 2024-09-19T08:51:24.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.164+0000 7fcac8827740 1 -- 172.21.15.164:0/3112723286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fefc1725d0 msgr2=0x55fefc17cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:24.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.164+0000 7fcac8827740 1 --2- 172.21.15.164:0/3112723286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fefc1725d0 0x55fefc17cc60 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fcab002a450 tx=0x7fcab00ad2d0 comp rx=0 tx=0).stop 2024-09-19T08:51:24.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.164+0000 7fcac8827740 1 -- 172.21.15.164:0/3112723286 shutdown_connections 2024-09-19T08:51:24.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.164+0000 7fcac8827740 1 --2- 172.21.15.164:0/3112723286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fefc17d370 0x55fefc17a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:24.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.164+0000 7fcac8827740 1 --2- 172.21.15.164:0/3112723286 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55fefc18ce20 0x55fefc18f2b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:24.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.164+0000 7fcac8827740 1 --2- 172.21.15.164:0/3112723286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fefc173b40 0x55fefc179f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:24.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.164+0000 7fcac8827740 1 --2- 172.21.15.164:0/3112723286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fefc1725d0 0x55fefc17cc60 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:24.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.164+0000 7fcac8827740 1 -- 172.21.15.164:0/3112723286 >> 172.21.15.164:0/3112723286 conn(0x55fefc13b310 msgr2=0x55fefc174960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:24.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.164+0000 7fca9f7fe640 0 client.16644 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:51:24.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.164+0000 7fcac8827740 1 -- 172.21.15.164:0/3112723286 shutdown_connections 2024-09-19T08:51:24.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.164+0000 7fcac8827740 1 -- 172.21.15.164:0/3112723286 wait complete. 2024-09-19T08:51:24.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.165+0000 7fcac8827740 20 client.16644 trim_cache size 0 max 16384 2024-09-19T08:51:24.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.196+0000 7fcac8827740 1 Processor -- start 2024-09-19T08:51:24.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.196+0000 7fcac8827740 1 -- start start 2024-09-19T08:51:24.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.197+0000 7fcac8827740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fefc1725d0 0x55fefc1893e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:24.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.197+0000 7fcac8827740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fefc173b40 0x55fefc194b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:24.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.197+0000 7fcac8827740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fefc17d370 0x55fefc1950a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:24.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.197+0000 7fcac8827740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01c4ce0 con 0x55fefc1725d0 2024-09-19T08:51:24.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.197+0000 7fcac8827740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01c68e0 con 0x55fefc173b40 2024-09-19T08:51:24.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.197+0000 7fcac8827740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01c53b0 con 0x55fefc17d370 2024-09-19T08:51:24.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.197+0000 7fcabffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fefc1725d0 0x55fefc1893e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:24.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.197+0000 7fcabffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fefc1725d0 0x55fefc1893e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:32858/0 (socket says 172.21.15.164:32858) 2024-09-19T08:51:24.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.197+0000 7fcabffff640 1 -- 172.21.15.164:0/761817020 learned_addr learned my addr 172.21.15.164:0/761817020 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:24.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.197+0000 7fcabf7fe640 1 --2- 172.21.15.164:0/761817020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fefc173b40 0x55fefc194b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:24.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.198+0000 7fcac49eb640 1 --2- 172.21.15.164:0/761817020 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fefc17d370 0x55fefc1950a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:24.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.198+0000 7fcabffff640 1 -- 172.21.15.164:0/761817020 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fefc17d370 msgr2=0x55fefc1950a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:24.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.198+0000 7fcabffff640 1 --2- 172.21.15.164:0/761817020 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fefc17d370 0x55fefc1950a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:24.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.198+0000 7fcabffff640 1 -- 172.21.15.164:0/761817020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fefc173b40 msgr2=0x55fefc194b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:24.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.198+0000 7fcabffff640 1 --2- 172.21.15.164:0/761817020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fefc173b40 0x55fefc194b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:24.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.198+0000 7fcabffff640 1 -- 172.21.15.164:0/761817020 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fefc173780 con 0x55fefc1725d0 2024-09-19T08:51:24.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.198+0000 7fcac49eb640 1 --2- 172.21.15.164:0/761817020 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fefc17d370 0x55fefc1950a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:51:24.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.198+0000 7fcabffff640 1 --2- 172.21.15.164:0/761817020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fefc1725d0 0x55fefc1893e0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fcab00f6a80 tx=0x7fcab00c0a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aadca39a4b81d4f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:24.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.198+0000 7fcaa5ffb640 1 -- 172.21.15.164:0/761817020 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab00c23d0 con 0x55fefc1725d0 2024-09-19T08:51:24.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.198+0000 7fcaa5ffb640 1 -- 172.21.15.164:0/761817020 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab00c2f20 con 0x55fefc1725d0 2024-09-19T08:51:24.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.198+0000 7fcaa5ffb640 1 -- 172.21.15.164:0/761817020 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab00cb250 con 0x55fefc1725d0 2024-09-19T08:51:24.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.199+0000 7fcac8827740 1 -- 172.21.15.164:0/761817020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fefc1725d0 msgr2=0x55fefc1893e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:24.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.199+0000 7fcac8827740 1 --2- 172.21.15.164:0/761817020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fefc1725d0 0x55fefc1893e0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fcab00f6a80 tx=0x7fcab00c0a80 comp rx=0 tx=0).stop 2024-09-19T08:51:24.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.199+0000 7fcac8827740 1 -- 172.21.15.164:0/761817020 shutdown_connections 2024-09-19T08:51:24.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.199+0000 7fcac8827740 1 --2- 172.21.15.164:0/761817020 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fefc17d370 0x55fefc1950a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:24.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.199+0000 7fcac8827740 1 --2- 172.21.15.164:0/761817020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fefc173b40 0x55fefc194b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:24.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.199+0000 7fcac8827740 1 --2- 172.21.15.164:0/761817020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fefc1725d0 0x55fefc1893e0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:24.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.199+0000 7fcac8827740 1 -- 172.21.15.164:0/761817020 >> 172.21.15.164:0/761817020 conn(0x55fefc13b310 msgr2=0x55fefc179ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:24.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.199+0000 7fcac8827740 1 -- 172.21.15.164:0/761817020 shutdown_connections 2024-09-19T08:51:24.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.199+0000 7fcac8827740 1 -- 172.21.15.164:0/761817020 wait complete. 2024-09-19T08:51:24.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.200+0000 7fcac8827740 1 -- start start 2024-09-19T08:51:24.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.200+0000 7fcac8827740 1 Processor -- start 2024-09-19T08:51:24.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.200+0000 7fcac8827740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fefc1725d0 0x55fefc185c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:24.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.200+0000 7fcac8827740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fefc173b40 0x55fefc180f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:24.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.201+0000 7fcac8827740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fefc17d370 0x55fefc1814b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:24.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.201+0000 7fcac8827740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01c6150 con 0x55fefc173b40 2024-09-19T08:51:24.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.201+0000 7fcac8827740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01c5a80 con 0x55fefc1725d0 2024-09-19T08:51:24.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.201+0000 7fcac8827740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01c2cc0 con 0x55fefc17d370 2024-09-19T08:51:24.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.201+0000 7fcabf7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fefc173b40 0x55fefc180f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:24.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.201+0000 7fcabffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fefc1725d0 0x55fefc185c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:24.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.201+0000 7fcabffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fefc1725d0 0x55fefc185c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:44950/0 (socket says 172.21.15.164:44950) 2024-09-19T08:51:24.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.201+0000 7fcabffff640 1 -- 172.21.15.164:0/2876660493 learned_addr learned my addr 172.21.15.164:0/2876660493 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:24.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.201+0000 7fcabf7fe640 1 --2- 172.21.15.164:0/2876660493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fefc173b40 0x55fefc180f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:32860/0 (socket says 172.21.15.164:32860) 2024-09-19T08:51:24.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.201+0000 7fcac49eb640 1 --2- 172.21.15.164:0/2876660493 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fefc17d370 0x55fefc1814b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:24.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.201+0000 7fcac8827740 10 client.0 _ll_register_callbacks cb 0x7fcab00f69d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:51:24.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.202+0000 7fcabf7fe640 1 -- 172.21.15.164:0/2876660493 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fefc17d370 msgr2=0x55fefc1814b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:24.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.202+0000 7fcabf7fe640 1 --2- 172.21.15.164:0/2876660493 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fefc17d370 0x55fefc1814b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:24.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.202+0000 7fcabf7fe640 1 -- 172.21.15.164:0/2876660493 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fefc1725d0 msgr2=0x55fefc185c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:24.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.202+0000 7fcabf7fe640 1 --2- 172.21.15.164:0/2876660493 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fefc1725d0 0x55fefc185c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:24.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.202+0000 7fcabf7fe640 1 -- 172.21.15.164:0/2876660493 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fefc181c90 con 0x55fefc173b40 2024-09-19T08:51:24.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.202+0000 7fcabf7fe640 1 --2- 172.21.15.164:0/2876660493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fefc173b40 0x55fefc180f70 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fcab4011b30 tx=0x7fcab4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05f7d3658b2eb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:24.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.202+0000 7fca9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:51:24.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.202+0000 7fca9f7fe640 1 -- 172.21.15.164:0/2876660493 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab4027070 con 0x55fefc173b40 2024-09-19T08:51:24.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.202+0000 7fca9f7fe640 1 -- 172.21.15.164:0/2876660493 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7fcab4011620 con 0x55fefc173b40 2024-09-19T08:51:24.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.202+0000 7fcac8827740 1 -- 172.21.15.164:0/2876660493 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcab00c2f20 con 0x55fefc173b40 2024-09-19T08:51:24.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.202+0000 7fca9f7fe640 1 -- 172.21.15.164:0/2876660493 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcab4033050 con 0x55fefc173b40 2024-09-19T08:51:24.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.202+0000 7fcac8827740 10 client.16650 fetch_fsmap learned FSMap version 30 2024-09-19T08:51:24.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.202+0000 7fcac8827740 1 -- 172.21.15.164:0/2876660493 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fefc1f7a70 con 0x55fefc173b40 2024-09-19T08:51:24.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.203+0000 7fca9f7fe640 1 -- 172.21.15.164:0/2876660493 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab00c2f20 con 0x55fefc173b40 2024-09-19T08:51:24.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.203+0000 7fca9f7fe640 1 -- 172.21.15.164:0/2876660493 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fcab4003c10 con 0x55fefc173b40 2024-09-19T08:51:24.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.203+0000 7fcac8827740 10 client.16650 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:51:24.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.203+0000 7fcac8827740 10 client.16650 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T08:51:24.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.203+0000 7fcac8827740 20 client.16650 populate_metadata read hostname 'smithi164' 2024-09-19T08:51:24.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.203+0000 7fcac8827740 1 --2- 172.21.15.164:0/2876660493 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55fefc1f8060 0x55fefc1fa480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:24.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.203+0000 7fcac8827740 4 client.16650 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:51:24.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.203+0000 7fcac8827740 1 -- 172.21.15.164:0/2876660493 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fefc0dfd70 con 0x55fefc1f8060 2024-09-19T08:51:24.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.204+0000 7fcabffff640 1 --2- 172.21.15.164:0/2876660493 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55fefc1f8060 0x55fefc1fa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:24.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.205+0000 7fcabffff640 1 --2- 172.21.15.164:0/2876660493 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55fefc1f8060 0x55fefc1fa480 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a2d1c9986acfc223 server_cookie=43a985364e2c6f9c in_seq=0 out_seq=0 2024-09-19T08:51:24.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.205+0000 7fca9f7fe640 10 client.16650 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:51:24.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.205+0000 7fca9f7fe640 1 -- 172.21.15.164:0/2876660493 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fefc0dfd70 con 0x55fefc1f8060 2024-09-19T08:51:24.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.205+0000 7fca9f7fe640 10 client.16650 handle_command_reply: tid=0 2024-09-19T08:51:24.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.205+0000 7fcac8827740 1 client.16650 shutdown 2024-09-19T08:51:24.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.206+0000 7fcac8827740 1 -- 172.21.15.164:0/2876660493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fefc173b40 msgr2=0x55fefc180f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:24.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.206+0000 7fcac8827740 1 --2- 172.21.15.164:0/2876660493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fefc173b40 0x55fefc180f70 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fcab4011b30 tx=0x7fcab4011c40 comp rx=0 tx=0).stop 2024-09-19T08:51:24.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.206+0000 7fcac8827740 1 -- 172.21.15.164:0/2876660493 shutdown_connections 2024-09-19T08:51:24.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.206+0000 7fcac8827740 1 --2- 172.21.15.164:0/2876660493 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fefc17d370 0x55fefc1814b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:24.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.206+0000 7fcac8827740 1 --2- 172.21.15.164:0/2876660493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fefc173b40 0x55fefc180f70 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:24.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.206+0000 7fcac8827740 1 --2- 172.21.15.164:0/2876660493 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55fefc1f8060 0x55fefc1fa480 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:24.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.206+0000 7fcac8827740 1 --2- 172.21.15.164:0/2876660493 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fefc1725d0 0x55fefc185c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:24.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.206+0000 7fcac8827740 1 -- 172.21.15.164:0/2876660493 >> 172.21.15.164:0/2876660493 conn(0x55fefc13b310 msgr2=0x55fefc16d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:24.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.206+0000 7fca9f7fe640 0 client.16650 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:51:24.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.207+0000 7fcac8827740 1 -- 172.21.15.164:0/2876660493 shutdown_connections 2024-09-19T08:51:24.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.207+0000 7fcac8827740 1 -- 172.21.15.164:0/2876660493 wait complete. 2024-09-19T08:51:24.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.207+0000 7fcac8827740 20 client.16650 trim_cache size 0 max 16384 2024-09-19T08:51:24.213 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:51:24.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.208+0000 7fcac644e640 1 -- 172.21.15.164:0/3674102540 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fcaa80a3580 msgr2=0x7fcaa80a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:24.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.208+0000 7fcac644e640 1 --2- 172.21.15.164:0/3674102540 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fcaa80a3580 0x7fcaa80a5a40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcab8005fd0 tx=0x7fcab8009890 comp rx=0 tx=0).stop 2024-09-19T08:51:24.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.208+0000 7fcac644e640 1 -- 172.21.15.164:0/3674102540 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcac0109450 msgr2=0x7fcac011a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:24.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.209+0000 7fcac644e640 1 --2- 172.21.15.164:0/3674102540 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcac0109450 0x7fcac011a8c0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fcab0006010 tx=0x7fcab00045e0 comp rx=0 tx=0).stop 2024-09-19T08:51:24.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.209+0000 7fcac644e640 1 -- 172.21.15.164:0/3674102540 shutdown_connections 2024-09-19T08:51:24.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.209+0000 7fcac644e640 1 --2- 172.21.15.164:0/3674102540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcac011b140 0x7fcac0114e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:24.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.209+0000 7fcac644e640 1 --2- 172.21.15.164:0/3674102540 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcac0109830 0x7fcac0114940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:24.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.209+0000 7fcac644e640 1 --2- 172.21.15.164:0/3674102540 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fcaa80a3580 0x7fcaa80a5a40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:24.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.209+0000 7fcac644e640 1 --2- 172.21.15.164:0/3674102540 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcac0109450 0x7fcac011a8c0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:24.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.209+0000 7fcac644e640 1 -- 172.21.15.164:0/3674102540 >> 172.21.15.164:0/3674102540 conn(0x7fcac006e140 msgr2=0x7fcac0070c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:24.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.209+0000 7fcac644e640 1 -- 172.21.15.164:0/3674102540 shutdown_connections 2024-09-19T08:51:24.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:24.209+0000 7fcac644e640 1 -- 172.21.15.164:0/3674102540 wait complete. 2024-09-19T08:51:24.300 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:51:25.029 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T08:51:25.062 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:51:25.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.637+0000 7f1c091c5640 1 -- 172.21.15.164:0/587758199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c04101de0 msgr2=0x7f1c041021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:25.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.637+0000 7f1c091c5640 1 --2- 172.21.15.164:0/587758199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c04101de0 0x7f1c041021c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4009900 tx=0x7f1bf402a410 comp rx=0 tx=0).stop 2024-09-19T08:51:25.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.637+0000 7f1c091c5640 1 -- 172.21.15.164:0/587758199 shutdown_connections 2024-09-19T08:51:25.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.637+0000 7f1c091c5640 1 --2- 172.21.15.164:0/587758199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c04106f10 0x7f1c0410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:25.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.637+0000 7f1c091c5640 1 --2- 172.21.15.164:0/587758199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1c04102790 0x7f1c041067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:25.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.637+0000 7f1c091c5640 1 --2- 172.21.15.164:0/587758199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c04101de0 0x7f1c041021c0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:25.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.637+0000 7f1c091c5640 1 -- 172.21.15.164:0/587758199 >> 172.21.15.164:0/587758199 conn(0x7f1c040fd8c0 msgr2=0x7f1c040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:25.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.637+0000 7f1c091c5640 1 -- 172.21.15.164:0/587758199 shutdown_connections 2024-09-19T08:51:25.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.637+0000 7f1c091c5640 1 -- 172.21.15.164:0/587758199 wait complete. 2024-09-19T08:51:25.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.638+0000 7f1c091c5640 1 Processor -- start 2024-09-19T08:51:25.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.638+0000 7f1c091c5640 1 -- start start 2024-09-19T08:51:25.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.638+0000 7f1c091c5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c04101de0 0x7f1c040744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:25.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.638+0000 7f1c091c5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1c04102790 0x7f1c04074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:25.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.638+0000 7f1c091c5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c04106f10 0x7f1c04074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:25.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.638+0000 7f1c091c5640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1c04111f90 con 0x7f1c04101de0 2024-09-19T08:51:25.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.638+0000 7f1c091c5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1c04111e10 con 0x7f1c04106f10 2024-09-19T08:51:25.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.638+0000 7f1c091c5640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1c04112110 con 0x7f1c04102790 2024-09-19T08:51:25.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.638+0000 7f1c02d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c04101de0 0x7f1c040744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:25.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.638+0000 7f1c02d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c04101de0 0x7f1c040744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:32884/0 (socket says 172.21.15.164:32884) 2024-09-19T08:51:25.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.638+0000 7f1c02d9d640 1 -- 172.21.15.164:0/3901885056 learned_addr learned my addr 172.21.15.164:0/3901885056 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:25.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.638+0000 7f1c0259c640 1 --2- 172.21.15.164:0/3901885056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1c04102790 0x7f1c04074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:25.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.638+0000 7f1c0359e640 1 --2- 172.21.15.164:0/3901885056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c04106f10 0x7f1c04074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:25.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.639+0000 7f1c02d9d640 1 -- 172.21.15.164:0/3901885056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1c04102790 msgr2=0x7f1c04074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:25.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.639+0000 7f1c02d9d640 1 --2- 172.21.15.164:0/3901885056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1c04102790 0x7f1c04074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:25.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.639+0000 7f1c02d9d640 1 -- 172.21.15.164:0/3901885056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c04106f10 msgr2=0x7f1c04074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:25.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.639+0000 7f1c02d9d640 1 --2- 172.21.15.164:0/3901885056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c04106f10 0x7f1c04074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:25.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.639+0000 7f1c02d9d640 1 -- 172.21.15.164:0/3901885056 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c041a6920 con 0x7f1c04101de0 2024-09-19T08:51:25.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.639+0000 7f1c0359e640 1 --2- 172.21.15.164:0/3901885056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c04106f10 0x7f1c04074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:51:25.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.639+0000 7f1c02d9d640 1 --2- 172.21.15.164:0/3901885056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c04101de0 0x7f1c040744f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4005ab0 tx=0x7f1bf40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:25.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.639+0000 7f1bebfff640 1 -- 172.21.15.164:0/3901885056 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf403f030 con 0x7f1c04101de0 2024-09-19T08:51:25.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.639+0000 7f1c091c5640 1 -- 172.21.15.164:0/3901885056 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c041a6bb0 con 0x7f1c04101de0 2024-09-19T08:51:25.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.639+0000 7f1bebfff640 1 -- 172.21.15.164:0/3901885056 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bf4003ce0 con 0x7f1c04101de0 2024-09-19T08:51:25.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.639+0000 7f1bebfff640 1 -- 172.21.15.164:0/3901885056 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf4035070 con 0x7f1c04101de0 2024-09-19T08:51:25.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.639+0000 7f1c091c5640 1 -- 172.21.15.164:0/3901885056 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c041a70e0 con 0x7f1c04101de0 2024-09-19T08:51:25.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.640+0000 7f1c091c5640 1 -- 172.21.15.164:0/3901885056 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c041021c0 con 0x7f1c04101de0 2024-09-19T08:51:25.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.642+0000 7f1bebfff640 1 -- 172.21.15.164:0/3901885056 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f1bf4042030 con 0x7f1c04101de0 2024-09-19T08:51:25.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.643+0000 7f1bebfff640 1 --2- 172.21.15.164:0/3901885056 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f1bec0a3430 0x7f1bec0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:25.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.643+0000 7f1c0259c640 1 --2- 172.21.15.164:0/3901885056 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f1bec0a3430 0x7f1bec0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:25.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.643+0000 7f1bebfff640 1 -- 172.21.15.164:0/3901885056 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f1bf40f0780 con 0x7f1c04101de0 2024-09-19T08:51:25.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.643+0000 7f1c0259c640 1 --2- 172.21.15.164:0/3901885056 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f1bec0a3430 0x7f1bec0a58f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1c041087d0 tx=0x7f1bf8009110 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:25.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.646+0000 7f1bebfff640 1 -- 172.21.15.164:0/3901885056 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1bf40bd1d0 con 0x7f1c04101de0 2024-09-19T08:51:25.778 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:51:25.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.857+0000 7f1c091c5640 1 -- 172.21.15.164:0/3901885056 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c0410a150 con 0x7f1c04101de0 2024-09-19T08:51:25.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.858+0000 7f1bebfff640 1 -- 172.21.15.164:0/3901885056 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11991 (secure 0 0 0) 0x7f1bf40bcad0 con 0x7f1c04101de0 2024-09-19T08:51:25.858 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:51:25.858 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":30,"btime":"2024-09-19T08:50:17:326587+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:50:17.071374+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:51:25.858 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 30 2024-09-19T08:51:25.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.860+0000 7f1c091c5640 1 -- 172.21.15.164:0/3901885056 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f1bec0a3430 msgr2=0x7f1bec0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:25.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.860+0000 7f1c091c5640 1 --2- 172.21.15.164:0/3901885056 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f1bec0a3430 0x7f1bec0a58f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1c041087d0 tx=0x7f1bf8009110 comp rx=0 tx=0).stop 2024-09-19T08:51:25.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.860+0000 7f1c091c5640 1 -- 172.21.15.164:0/3901885056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c04101de0 msgr2=0x7f1c040744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:25.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.860+0000 7f1c091c5640 1 --2- 172.21.15.164:0/3901885056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c04101de0 0x7f1c040744f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4005ab0 tx=0x7f1bf40042f0 comp rx=0 tx=0).stop 2024-09-19T08:51:25.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.860+0000 7f1c091c5640 1 -- 172.21.15.164:0/3901885056 shutdown_connections 2024-09-19T08:51:25.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.860+0000 7f1c091c5640 1 --2- 172.21.15.164:0/3901885056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c04106f10 0x7f1c04074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:25.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.860+0000 7f1c091c5640 1 --2- 172.21.15.164:0/3901885056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1c04102790 0x7f1c04074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:25.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.860+0000 7f1c091c5640 1 --2- 172.21.15.164:0/3901885056 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f1bec0a3430 0x7f1bec0a58f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:25.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.860+0000 7f1c091c5640 1 --2- 172.21.15.164:0/3901885056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1c04101de0 0x7f1c040744f0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:25.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.860+0000 7f1c091c5640 1 -- 172.21.15.164:0/3901885056 >> 172.21.15.164:0/3901885056 conn(0x7f1c040fd8c0 msgr2=0x7f1c040feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:25.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.860+0000 7f1c091c5640 1 -- 172.21.15.164:0/3901885056 shutdown_connections 2024-09-19T08:51:25.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:25.860+0000 7f1c091c5640 1 -- 172.21.15.164:0/3901885056 wait complete. 2024-09-19T08:51:25.898 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:25 smithi164 ceph-mon[30928]: pgmap v171: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 75 MiB/s wr, 119 op/s 2024-09-19T08:51:26.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:25 smithi170 ceph-mon[35072]: pgmap v171: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 75 MiB/s wr, 119 op/s 2024-09-19T08:51:26.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:25 smithi173 ceph-mon[35306]: pgmap v171: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 75 MiB/s wr, 119 op/s 2024-09-19T08:51:26.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.554+0000 7f1503991640 1 -- 172.21.15.164:0/1047190260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14fc100a80 msgr2=0x7f14fc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:26.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.554+0000 7f1503991640 1 --2- 172.21.15.164:0/1047190260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14fc100a80 0x7f14fc100ee0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f14ec009930 tx=0x7f14ec02a420 comp rx=0 tx=0).stop 2024-09-19T08:51:26.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.554+0000 7f1503991640 1 -- 172.21.15.164:0/1047190260 shutdown_connections 2024-09-19T08:51:26.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.554+0000 7f1503991640 1 --2- 172.21.15.164:0/1047190260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14fc101420 0x7f14fc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.554+0000 7f1503991640 1 --2- 172.21.15.164:0/1047190260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14fc100a80 0x7f14fc100ee0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.554+0000 7f1503991640 1 --2- 172.21.15.164:0/1047190260 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14fc106a80 0x7f14fc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.554+0000 7f1503991640 1 -- 172.21.15.164:0/1047190260 >> 172.21.15.164:0/1047190260 conn(0x7f14fc0fc7a0 msgr2=0x7f14fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:26.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.554+0000 7f1503991640 1 -- 172.21.15.164:0/1047190260 shutdown_connections 2024-09-19T08:51:26.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.554+0000 7f1503991640 1 -- 172.21.15.164:0/1047190260 wait complete. 2024-09-19T08:51:26.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.554+0000 7f1503991640 1 Processor -- start 2024-09-19T08:51:26.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.554+0000 7f1503991640 1 -- start start 2024-09-19T08:51:26.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.555+0000 7f1503991640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14fc100a80 0x7f14fc1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:26.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.555+0000 7f1503991640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14fc101420 0x7f14fc198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:26.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.555+0000 7f1503991640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14fc106a80 0x7f14fc19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:26.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.555+0000 7f1503991640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc110ea0 con 0x7f14fc106a80 2024-09-19T08:51:26.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.555+0000 7f1503991640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc110d20 con 0x7f14fc100a80 2024-09-19T08:51:26.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.555+0000 7f1503991640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc111020 con 0x7f14fc101420 2024-09-19T08:51:26.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.555+0000 7f1501f2e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14fc106a80 0x7f14fc19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:26.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.555+0000 7f1501f2e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14fc106a80 0x7f14fc19cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:32914/0 (socket says 172.21.15.164:32914) 2024-09-19T08:51:26.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.555+0000 7f1501f2e640 1 -- 172.21.15.164:0/1111458855 learned_addr learned my addr 172.21.15.164:0/1111458855 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:26.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.555+0000 7f150172d640 1 --2- 172.21.15.164:0/1111458855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14fc100a80 0x7f14fc1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:26.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.555+0000 7f1500f2c640 1 --2- 172.21.15.164:0/1111458855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14fc101420 0x7f14fc198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:26.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.555+0000 7f1501f2e640 1 -- 172.21.15.164:0/1111458855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14fc101420 msgr2=0x7f14fc198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:26.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.555+0000 7f1501f2e640 1 --2- 172.21.15.164:0/1111458855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14fc101420 0x7f14fc198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.555+0000 7f1501f2e640 1 -- 172.21.15.164:0/1111458855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14fc100a80 msgr2=0x7f14fc1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:26.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.555+0000 7f1501f2e640 1 --2- 172.21.15.164:0/1111458855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14fc100a80 0x7f14fc1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.555+0000 7f1501f2e640 1 -- 172.21.15.164:0/1111458855 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14fc19d480 con 0x7f14fc106a80 2024-09-19T08:51:26.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.555+0000 7f1500f2c640 1 --2- 172.21.15.164:0/1111458855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14fc101420 0x7f14fc198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:51:26.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.555+0000 7f1501f2e640 1 --2- 172.21.15.164:0/1111458855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14fc106a80 0x7f14fc19cda0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009fd0 tx=0x7f14f400def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:26.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.555+0000 7f14f27fc640 1 -- 172.21.15.164:0/1111458855 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4017070 con 0x7f14fc106a80 2024-09-19T08:51:26.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.555+0000 7f1503991640 1 -- 172.21.15.164:0/1111458855 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14fc19d770 con 0x7f14fc106a80 2024-09-19T08:51:26.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.555+0000 7f14f27fc640 1 -- 172.21.15.164:0/1111458855 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14f4003e50 con 0x7f14fc106a80 2024-09-19T08:51:26.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.555+0000 7f14f27fc640 1 -- 172.21.15.164:0/1111458855 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4010460 con 0x7f14fc106a80 2024-09-19T08:51:26.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.555+0000 7f1503991640 1 -- 172.21.15.164:0/1111458855 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14fc1a5050 con 0x7f14fc106a80 2024-09-19T08:51:26.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.557+0000 7f14f27fc640 1 -- 172.21.15.164:0/1111458855 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f14f4003970 con 0x7f14fc106a80 2024-09-19T08:51:26.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.558+0000 7f1505d6a740 1 Processor -- start 2024-09-19T08:51:26.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.558+0000 7f1505d6a740 1 -- start start 2024-09-19T08:51:26.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.558+0000 7f1505d6a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dc081c5b40 0x55dc081c4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:26.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.558+0000 7f1505d6a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc081c45d0 0x55dc081c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:26.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.558+0000 7f1505d6a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc081cf370 0x55dc081d1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:26.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.558+0000 7f1505d6a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55dc0812a2f0 con 0x55dc081cf370 2024-09-19T08:51:26.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.558+0000 7f1505d6a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55dc080d1cf0 con 0x55dc081c45d0 2024-09-19T08:51:26.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.558+0000 7f1505d6a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55dc081262b0 con 0x55dc081c5b40 2024-09-19T08:51:26.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.558+0000 7f1500f2c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc081cf370 0x55dc081d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:26.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.558+0000 7f1500f2c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc081cf370 0x55dc081d1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:32916/0 (socket says 172.21.15.164:32916) 2024-09-19T08:51:26.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.558+0000 7f1500f2c640 1 -- 172.21.15.164:0/3405072354 learned_addr learned my addr 172.21.15.164:0/3405072354 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:26.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.558+0000 7f14f27fc640 1 --2- 172.21.15.164:0/1111458855 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f14e80a33b0 0x7f14e80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:26.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.558+0000 7f1500f2c640 1 --2- 172.21.15.164:0/3405072354 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dc081c5b40 0x55dc081c4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:26.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.558+0000 7f150172d640 1 --2- 172.21.15.164:0/3405072354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc081c45d0 0x55dc081c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:26.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.559+0000 7f1501f2e640 1 --2- 172.21.15.164:0/1111458855 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f14e80a33b0 0x7f14e80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:26.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.559+0000 7f1500f2c640 1 -- 172.21.15.164:0/3405072354 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dc081c5b40 msgr2=0x55dc081c4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:26.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.559+0000 7f1500f2c640 1 --2- 172.21.15.164:0/3405072354 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dc081c5b40 0x55dc081c4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.559+0000 7f1500f2c640 1 -- 172.21.15.164:0/3405072354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc081c45d0 msgr2=0x55dc081c4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:26.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.559+0000 7f1500f2c640 1 --2- 172.21.15.164:0/3405072354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc081c45d0 0x55dc081c4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.559+0000 7f1500f2c640 1 -- 172.21.15.164:0/3405072354 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc080b63a0 con 0x55dc081cf370 2024-09-19T08:51:26.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.559+0000 7f14f27fc640 1 -- 172.21.15.164:0/1111458855 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f14f40cfa30 con 0x7f14fc106a80 2024-09-19T08:51:26.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.559+0000 7f1500f2c640 1 --2- 172.21.15.164:0/3405072354 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc081cf370 0x55dc081d1760 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0057c0 tx=0x7f14ec000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7eec0a8b79daa3f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:26.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.559+0000 7f1501f2e640 1 --2- 172.21.15.164:0/1111458855 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f14e80a33b0 0x7f14e80a5870 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f14f4098180 tx=0x7f14f409baf0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:26.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.559+0000 7f14e77fe640 1 -- 172.21.15.164:0/3405072354 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec035040 con 0x55dc081cf370 2024-09-19T08:51:26.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.559+0000 7f14e77fe640 1 -- 172.21.15.164:0/3405072354 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14ec0035f0 con 0x55dc081cf370 2024-09-19T08:51:26.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.559+0000 7f14e77fe640 1 -- 172.21.15.164:0/3405072354 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec03c650 con 0x55dc081cf370 2024-09-19T08:51:26.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.560+0000 7f1505d6a740 1 -- 172.21.15.164:0/3405072354 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc081cf370 msgr2=0x55dc081d1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:26.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.560+0000 7f1505d6a740 1 --2- 172.21.15.164:0/3405072354 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc081cf370 0x55dc081d1760 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0057c0 tx=0x7f14ec000bc0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.560+0000 7f1505d6a740 1 -- 172.21.15.164:0/3405072354 shutdown_connections 2024-09-19T08:51:26.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.560+0000 7f1505d6a740 1 --2- 172.21.15.164:0/3405072354 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc081cf370 0x55dc081d1760 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.560+0000 7f1505d6a740 1 --2- 172.21.15.164:0/3405072354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc081c45d0 0x55dc081c4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.560+0000 7f1505d6a740 1 --2- 172.21.15.164:0/3405072354 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dc081c5b40 0x55dc081c4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.560+0000 7f1505d6a740 1 -- 172.21.15.164:0/3405072354 >> 172.21.15.164:0/3405072354 conn(0x55dc0818d310 msgr2=0x55dc081c0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:26.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.560+0000 7f1505d6a740 1 -- 172.21.15.164:0/3405072354 shutdown_connections 2024-09-19T08:51:26.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.560+0000 7f1505d6a740 1 -- 172.21.15.164:0/3405072354 wait complete. 2024-09-19T08:51:26.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.560+0000 7f1505d6a740 1 -- start start 2024-09-19T08:51:26.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.560+0000 7f1505d6a740 1 Processor -- start 2024-09-19T08:51:26.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.561+0000 7f1505d6a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc081c45d0 0x55dc081ced70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:26.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.561+0000 7f1505d6a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc081c5b40 0x55dc081cc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:26.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.561+0000 7f1505d6a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dc081cf370 0x55dc081cc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:26.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.561+0000 7f1505d6a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55dc081d2010 con 0x55dc081c5b40 2024-09-19T08:51:26.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.561+0000 7f1505d6a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55dc080e5060 con 0x55dc081c45d0 2024-09-19T08:51:26.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.561+0000 7f1505d6a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55dc081d2190 con 0x55dc081cf370 2024-09-19T08:51:26.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.561+0000 7f1505d6a740 10 client.0 _ll_register_callbacks cb 0x55dc07e388c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:51:26.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.561+0000 7f150172d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc081c5b40 0x55dc081cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:26.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.561+0000 7f150172d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc081c5b40 0x55dc081cc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:32924/0 (socket says 172.21.15.164:32924) 2024-09-19T08:51:26.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.561+0000 7f150172d640 1 -- 172.21.15.164:0/1654401201 learned_addr learned my addr 172.21.15.164:0/1654401201 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:26.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.561+0000 7f1500f2c640 1 --2- 172.21.15.164:0/1654401201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc081c45d0 0x55dc081ced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:26.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.561+0000 7f1500f2c640 1 --2- 172.21.15.164:0/1654401201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dc081cf370 0x55dc081cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:26.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.561+0000 7f150172d640 1 -- 172.21.15.164:0/1654401201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dc081cf370 msgr2=0x55dc081cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:26.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.561+0000 7f150172d640 1 --2- 172.21.15.164:0/1654401201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dc081cf370 0x55dc081cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.561+0000 7f150172d640 1 -- 172.21.15.164:0/1654401201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc081c45d0 msgr2=0x55dc081ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:26.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.561+0000 7f150172d640 1 --2- 172.21.15.164:0/1654401201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc081c45d0 0x55dc081ced70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.561+0000 7f150172d640 1 -- 172.21.15.164:0/1654401201 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc07e9b3c0 con 0x55dc081c5b40 2024-09-19T08:51:26.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.561+0000 7f1500f2c640 1 --2- 172.21.15.164:0/1654401201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dc081cf370 0x55dc081cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:51:26.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.562+0000 7f150172d640 1 --2- 172.21.15.164:0/1654401201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc081c5b40 0x55dc081cc020 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f14f8011540 tx=0x7f14f8011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81727240fbb866d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:26.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.562+0000 7f14e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:51:26.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.562+0000 7f14e4ff9640 1 -- 172.21.15.164:0/1654401201 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f800f6d0 con 0x55dc081c5b40 2024-09-19T08:51:26.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.562+0000 7f14e4ff9640 1 -- 172.21.15.164:0/1654401201 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f14f800f870 con 0x55dc081c5b40 2024-09-19T08:51:26.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.562+0000 7f1505d6a740 1 -- 172.21.15.164:0/1654401201 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14ec0035f0 con 0x55dc081c5b40 2024-09-19T08:51:26.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.562+0000 7f14e4ff9640 1 -- 172.21.15.164:0/1654401201 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f14f800fab0 con 0x55dc081c5b40 2024-09-19T08:51:26.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.562+0000 7f1505d6a740 10 client.16668 fetch_fsmap learned FSMap version 30 2024-09-19T08:51:26.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.562+0000 7f1505d6a740 1 -- 172.21.15.164:0/1654401201 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dc081ddb60 con 0x55dc081c5b40 2024-09-19T08:51:26.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.562+0000 7f14e4ff9640 1 -- 172.21.15.164:0/1654401201 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14ec0035f0 con 0x55dc081c5b40 2024-09-19T08:51:26.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.562+0000 7f14e4ff9640 1 -- 172.21.15.164:0/1654401201 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f14f8025020 con 0x55dc081c5b40 2024-09-19T08:51:26.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.562+0000 7f1505d6a740 10 client.16668 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:51:26.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.562+0000 7f1505d6a740 10 client.16668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:51:26.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.562+0000 7f1505d6a740 20 client.16668 populate_metadata read hostname 'smithi164' 2024-09-19T08:51:26.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.562+0000 7f1505d6a740 1 --2- 172.21.15.164:0/1654401201 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55dc081df040 0x55dc081e1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:26.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.562+0000 7f1505d6a740 4 client.16668 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:51:26.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.562+0000 7f1505d6a740 1 -- 172.21.15.164:0/1654401201 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc08137860 con 0x55dc081df040 2024-09-19T08:51:26.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.563+0000 7f1500f2c640 1 --2- 172.21.15.164:0/1654401201 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55dc081df040 0x55dc081e1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:26.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.563+0000 7f1500f2c640 1 --2- 172.21.15.164:0/1654401201 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55dc081df040 0x55dc081e1480 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30c2b758bc1bfe65 server_cookie=6cd71cd989f35b42 in_seq=0 out_seq=0 2024-09-19T08:51:26.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.563+0000 7f14e4ff9640 10 client.16668 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:51:26.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.563+0000 7f14e4ff9640 1 -- 172.21.15.164:0/1654401201 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dc08137860 con 0x55dc081df040 2024-09-19T08:51:26.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.564+0000 7f14e4ff9640 10 client.16668 handle_command_reply: tid=0 2024-09-19T08:51:26.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.564+0000 7f1505d6a740 1 client.16668 shutdown 2024-09-19T08:51:26.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.564+0000 7f1505d6a740 1 -- 172.21.15.164:0/1654401201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc081c5b40 msgr2=0x55dc081cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:26.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.564+0000 7f1505d6a740 1 --2- 172.21.15.164:0/1654401201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc081c5b40 0x55dc081cc020 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f14f8011540 tx=0x7f14f8011a00 comp rx=0 tx=0).stop 2024-09-19T08:51:26.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.564+0000 7f1505d6a740 1 -- 172.21.15.164:0/1654401201 shutdown_connections 2024-09-19T08:51:26.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.564+0000 7f1505d6a740 1 --2- 172.21.15.164:0/1654401201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dc081cf370 0x55dc081cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.564+0000 7f1505d6a740 1 --2- 172.21.15.164:0/1654401201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc081c5b40 0x55dc081cc020 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.564+0000 7f1505d6a740 1 --2- 172.21.15.164:0/1654401201 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55dc081df040 0x55dc081e1480 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.564+0000 7f1505d6a740 1 --2- 172.21.15.164:0/1654401201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc081c45d0 0x55dc081ced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.564+0000 7f14e4ff9640 0 client.16668 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:51:26.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.564+0000 7f1505d6a740 1 -- 172.21.15.164:0/1654401201 >> 172.21.15.164:0/1654401201 conn(0x55dc0818d310 msgr2=0x55dc081c6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:26.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.564+0000 7f1505d6a740 1 -- 172.21.15.164:0/1654401201 shutdown_connections 2024-09-19T08:51:26.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.564+0000 7f1505d6a740 1 -- 172.21.15.164:0/1654401201 wait complete. 2024-09-19T08:51:26.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.564+0000 7f1505d6a740 20 client.16668 trim_cache size 0 max 16384 2024-09-19T08:51:26.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.577+0000 7f1505d6a740 1 Processor -- start 2024-09-19T08:51:26.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.577+0000 7f1505d6a740 1 -- start start 2024-09-19T08:51:26.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.577+0000 7f1505d6a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dc081c45d0 0x55dc081c0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:26.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.577+0000 7f1505d6a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc081c5b40 0x55dc081cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:26.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.577+0000 7f1505d6a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc081cf370 0x55dc081db2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:26.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.577+0000 7f1505d6a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1ad740 con 0x55dc081cf370 2024-09-19T08:51:26.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.577+0000 7f1505d6a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1af340 con 0x55dc081c5b40 2024-09-19T08:51:26.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.577+0000 7f1505d6a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1ade10 con 0x55dc081c45d0 2024-09-19T08:51:26.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.577+0000 7f1500f2c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc081cf370 0x55dc081db2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:26.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.577+0000 7f1500f2c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc081cf370 0x55dc081db2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:32926/0 (socket says 172.21.15.164:32926) 2024-09-19T08:51:26.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.577+0000 7f1500f2c640 1 -- 172.21.15.164:0/2169483087 learned_addr learned my addr 172.21.15.164:0/2169483087 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:26.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.577+0000 7f150172d640 1 --2- 172.21.15.164:0/2169483087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc081c5b40 0x55dc081cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:26.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.577+0000 7f1500f2c640 1 -- 172.21.15.164:0/2169483087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dc081c45d0 msgr2=0x55dc081c0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:26.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.577+0000 7f1500f2c640 1 --2- 172.21.15.164:0/2169483087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dc081c45d0 0x55dc081c0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.577+0000 7f1500f2c640 1 -- 172.21.15.164:0/2169483087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc081c5b40 msgr2=0x55dc081cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:26.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.577+0000 7f1500f2c640 1 --2- 172.21.15.164:0/2169483087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc081c5b40 0x55dc081cb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.577+0000 7f1500f2c640 1 -- 172.21.15.164:0/2169483087 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc081c1960 con 0x55dc081cf370 2024-09-19T08:51:26.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.578+0000 7f1500f2c640 1 --2- 172.21.15.164:0/2169483087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc081cf370 0x55dc081db2b0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f14ec033000 tx=0x7f14ec02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c907a97987f38851 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:26.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.578+0000 7f14e6ffd640 1 -- 172.21.15.164:0/2169483087 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec05a070 con 0x55dc081cf370 2024-09-19T08:51:26.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.578+0000 7f14e6ffd640 1 -- 172.21.15.164:0/2169483087 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14ec004400 con 0x55dc081cf370 2024-09-19T08:51:26.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.578+0000 7f14e6ffd640 1 -- 172.21.15.164:0/2169483087 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec035040 con 0x55dc081cf370 2024-09-19T08:51:26.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.578+0000 7f1505d6a740 1 -- 172.21.15.164:0/2169483087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc081cf370 msgr2=0x55dc081db2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:26.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.578+0000 7f1505d6a740 1 --2- 172.21.15.164:0/2169483087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc081cf370 0x55dc081db2b0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f14ec033000 tx=0x7f14ec02a420 comp rx=0 tx=0).stop 2024-09-19T08:51:26.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.578+0000 7f1505d6a740 1 -- 172.21.15.164:0/2169483087 shutdown_connections 2024-09-19T08:51:26.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.578+0000 7f1505d6a740 1 --2- 172.21.15.164:0/2169483087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc081cf370 0x55dc081db2b0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.578+0000 7f1505d6a740 1 --2- 172.21.15.164:0/2169483087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc081c5b40 0x55dc081cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.578+0000 7f1505d6a740 1 --2- 172.21.15.164:0/2169483087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dc081c45d0 0x55dc081c0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.578+0000 7f1505d6a740 1 -- 172.21.15.164:0/2169483087 >> 172.21.15.164:0/2169483087 conn(0x55dc0818d310 msgr2=0x55dc081c0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:26.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.578+0000 7f1505d6a740 1 -- 172.21.15.164:0/2169483087 shutdown_connections 2024-09-19T08:51:26.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.579+0000 7f1505d6a740 1 -- 172.21.15.164:0/2169483087 wait complete. 2024-09-19T08:51:26.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.579+0000 7f1505d6a740 1 -- start start 2024-09-19T08:51:26.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.579+0000 7f1505d6a740 1 Processor -- start 2024-09-19T08:51:26.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.579+0000 7f1505d6a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dc081c45d0 0x55dc081c9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:26.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.579+0000 7f1505d6a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc081c5b40 0x55dc081c9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:26.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.579+0000 7f1505d6a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc081cf370 0x55dc081c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:26.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.579+0000 7f1505d6a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1aebb0 con 0x55dc081c5b40 2024-09-19T08:51:26.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.579+0000 7f1505d6a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1ae4e0 con 0x55dc081cf370 2024-09-19T08:51:26.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.579+0000 7f1505d6a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1ab740 con 0x55dc081c45d0 2024-09-19T08:51:26.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.579+0000 7f1505d6a740 10 client.0 _ll_register_callbacks cb 0x7f14f8009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:51:26.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.579+0000 7f150172d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc081c5b40 0x55dc081c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:26.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.579+0000 7f150172d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc081c5b40 0x55dc081c9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:32934/0 (socket says 172.21.15.164:32934) 2024-09-19T08:51:26.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.580+0000 7f150172d640 1 -- 172.21.15.164:0/844513150 learned_addr learned my addr 172.21.15.164:0/844513150 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:26.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.580+0000 7f1500f2c640 1 --2- 172.21.15.164:0/844513150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc081cf370 0x55dc081c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:26.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.580+0000 7f1500f2c640 1 --2- 172.21.15.164:0/844513150 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dc081c45d0 0x55dc081c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:26.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.580+0000 7f150172d640 1 -- 172.21.15.164:0/844513150 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dc081c45d0 msgr2=0x55dc081c9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:26.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.580+0000 7f150172d640 1 --2- 172.21.15.164:0/844513150 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dc081c45d0 0x55dc081c9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.580+0000 7f150172d640 1 -- 172.21.15.164:0/844513150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc081cf370 msgr2=0x55dc081c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:26.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.580+0000 7f150172d640 1 --2- 172.21.15.164:0/844513150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc081cf370 0x55dc081c9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.580+0000 7f150172d640 1 -- 172.21.15.164:0/844513150 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc081db7f0 con 0x55dc081c5b40 2024-09-19T08:51:26.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.580+0000 7f150172d640 1 --2- 172.21.15.164:0/844513150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc081c5b40 0x55dc081c9750 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f14f8004810 tx=0x7f14f8011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cad938c5cae4da7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:26.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.580+0000 7f14e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:51:26.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.580+0000 7f14e4ff9640 1 -- 172.21.15.164:0/844513150 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f8027660 con 0x55dc081c5b40 2024-09-19T08:51:26.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.580+0000 7f14e4ff9640 1 -- 172.21.15.164:0/844513150 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f14f8027800 con 0x55dc081c5b40 2024-09-19T08:51:26.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.580+0000 7f1505d6a740 1 -- 172.21.15.164:0/844513150 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14ec004400 con 0x55dc081c5b40 2024-09-19T08:51:26.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.580+0000 7f14e4ff9640 1 -- 172.21.15.164:0/844513150 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f14f800fe60 con 0x55dc081c5b40 2024-09-19T08:51:26.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.580+0000 7f1505d6a740 10 client.16674 fetch_fsmap learned FSMap version 30 2024-09-19T08:51:26.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.580+0000 7f1505d6a740 1 -- 172.21.15.164:0/844513150 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dc0824e440 con 0x55dc081c5b40 2024-09-19T08:51:26.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.581+0000 7f14e4ff9640 1 -- 172.21.15.164:0/844513150 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14ec004400 con 0x55dc081c5b40 2024-09-19T08:51:26.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.581+0000 7f14e4ff9640 1 -- 172.21.15.164:0/844513150 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f14f8038050 con 0x55dc081c5b40 2024-09-19T08:51:26.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.581+0000 7f1505d6a740 10 client.16674 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:51:26.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.581+0000 7f1505d6a740 10 client.16674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:51:26.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.581+0000 7f1505d6a740 20 client.16674 populate_metadata read hostname 'smithi164' 2024-09-19T08:51:26.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.581+0000 7f1505d6a740 1 --2- 172.21.15.164:0/844513150 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55dc0824ec60 0x55dc08251080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:26.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.581+0000 7f1505d6a740 4 client.16674 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:51:26.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.581+0000 7f1505d6a740 1 -- 172.21.15.164:0/844513150 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dc08131d70 con 0x55dc0824ec60 2024-09-19T08:51:26.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.581+0000 7f1500f2c640 1 --2- 172.21.15.164:0/844513150 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55dc0824ec60 0x55dc08251080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:26.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.581+0000 7f1500f2c640 1 --2- 172.21.15.164:0/844513150 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55dc0824ec60 0x55dc08251080 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4d94de9f9b352a server_cookie=bd3ccaa0cf45c4d7 in_seq=0 out_seq=0 2024-09-19T08:51:26.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.581+0000 7f14e4ff9640 10 client.16674 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:51:26.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.582+0000 7f14e4ff9640 1 -- 172.21.15.164:0/844513150 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dc08131d70 con 0x55dc0824ec60 2024-09-19T08:51:26.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.582+0000 7f14e4ff9640 10 client.16674 handle_command_reply: tid=0 2024-09-19T08:51:26.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.582+0000 7f1505d6a740 1 client.16674 shutdown 2024-09-19T08:51:26.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.582+0000 7f1505d6a740 1 -- 172.21.15.164:0/844513150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc081c5b40 msgr2=0x55dc081c9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:26.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.582+0000 7f1505d6a740 1 --2- 172.21.15.164:0/844513150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc081c5b40 0x55dc081c9750 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f14f8004810 tx=0x7f14f8011f90 comp rx=0 tx=0).stop 2024-09-19T08:51:26.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.582+0000 7f1505d6a740 1 -- 172.21.15.164:0/844513150 shutdown_connections 2024-09-19T08:51:26.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.582+0000 7f1505d6a740 1 --2- 172.21.15.164:0/844513150 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55dc0824ec60 0x55dc08251080 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.582+0000 7f1505d6a740 1 --2- 172.21.15.164:0/844513150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc081cf370 0x55dc081c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.582+0000 7f1505d6a740 1 --2- 172.21.15.164:0/844513150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dc081c5b40 0x55dc081c9750 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.582+0000 7f1505d6a740 1 --2- 172.21.15.164:0/844513150 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dc081c45d0 0x55dc081c9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.582+0000 7f1505d6a740 1 -- 172.21.15.164:0/844513150 >> 172.21.15.164:0/844513150 conn(0x55dc0818d310 msgr2=0x55dc081c6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:26.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.582+0000 7f14e4ff9640 0 client.16674 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:51:26.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.582+0000 7f1505d6a740 1 -- 172.21.15.164:0/844513150 shutdown_connections 2024-09-19T08:51:26.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.582+0000 7f1505d6a740 1 -- 172.21.15.164:0/844513150 wait complete. 2024-09-19T08:51:26.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.582+0000 7f1505d6a740 20 client.16674 trim_cache size 0 max 16384 2024-09-19T08:51:26.593 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:51:26.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.583+0000 7f1503991640 1 -- 172.21.15.164:0/1111458855 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f14e80a33b0 msgr2=0x7f14e80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:26.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.583+0000 7f1503991640 1 --2- 172.21.15.164:0/1111458855 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f14e80a33b0 0x7f14e80a5870 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f14f4098180 tx=0x7f14f409baf0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.583+0000 7f1503991640 1 -- 172.21.15.164:0/1111458855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14fc106a80 msgr2=0x7f14fc19cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:26.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.583+0000 7f1503991640 1 --2- 172.21.15.164:0/1111458855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14fc106a80 0x7f14fc19cda0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009fd0 tx=0x7f14f400def0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.583+0000 7f1503991640 1 -- 172.21.15.164:0/1111458855 shutdown_connections 2024-09-19T08:51:26.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.583+0000 7f1503991640 1 --2- 172.21.15.164:0/1111458855 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f14e80a33b0 0x7f14e80a5870 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.583+0000 7f1503991640 1 --2- 172.21.15.164:0/1111458855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14fc106a80 0x7f14fc19cda0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.583+0000 7f1503991640 1 --2- 172.21.15.164:0/1111458855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14fc101420 0x7f14fc198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.583+0000 7f1503991640 1 --2- 172.21.15.164:0/1111458855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14fc100a80 0x7f14fc1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:26.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.583+0000 7f1503991640 1 -- 172.21.15.164:0/1111458855 >> 172.21.15.164:0/1111458855 conn(0x7f14fc0fc7a0 msgr2=0x7f14fc06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:26.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.583+0000 7f1503991640 1 -- 172.21.15.164:0/1111458855 shutdown_connections 2024-09-19T08:51:26.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:26.583+0000 7f1503991640 1 -- 172.21.15.164:0/1111458855 wait complete. 2024-09-19T08:51:27.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:26 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3901885056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:51:27.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:26 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3901885056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:51:27.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:26 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3901885056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:51:27.843 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id 21b3571c --release --await' 2024-09-19T08:51:27.843 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id 21b3571c --release --await 2024-09-19T08:51:28.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:27 smithi164 ceph-mon[30928]: pgmap v172: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 84 MiB/s wr, 139 op/s 2024-09-19T08:51:28.092 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:51:28.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:27 smithi170 ceph-mon[35072]: pgmap v172: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 84 MiB/s wr, 139 op/s 2024-09-19T08:51:28.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:27 smithi173 ceph-mon[35306]: pgmap v172: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 84 MiB/s wr, 139 op/s 2024-09-19T08:51:28.298 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T08:51:28.298 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T08:51:28.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.556+0000 7f64580e2640 1 -- 172.21.15.164:0/2637222714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6450101df0 msgr2=0x7f64501021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:28.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.556+0000 7f64580e2640 1 --2- 172.21.15.164:0/2637222714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6450101df0 0x7f64501021d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f644c009900 tx=0x7f644c02a410 comp rx=0 tx=0).stop 2024-09-19T08:51:28.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.556+0000 7f64580e2640 1 -- 172.21.15.164:0/2637222714 shutdown_connections 2024-09-19T08:51:28.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.556+0000 7f64580e2640 1 --2- 172.21.15.164:0/2637222714 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6450106f20 0x7f645010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.556+0000 7f64580e2640 1 --2- 172.21.15.164:0/2637222714 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64501027a0 0x7f64501067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.556+0000 7f64580e2640 1 --2- 172.21.15.164:0/2637222714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6450101df0 0x7f64501021d0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.556+0000 7f64580e2640 1 -- 172.21.15.164:0/2637222714 >> 172.21.15.164:0/2637222714 conn(0x7f64500fd8f0 msgr2=0x7f64500ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:28.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.557+0000 7f64580e2640 1 -- 172.21.15.164:0/2637222714 shutdown_connections 2024-09-19T08:51:28.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.557+0000 7f64580e2640 1 -- 172.21.15.164:0/2637222714 wait complete. 2024-09-19T08:51:28.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.557+0000 7f64580e2640 1 Processor -- start 2024-09-19T08:51:28.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.557+0000 7f64580e2640 1 -- start start 2024-09-19T08:51:28.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.558+0000 7f64580e2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6450101df0 0x7f6450198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:28.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.558+0000 7f64580e2640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64501027a0 0x7f6450198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:28.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.558+0000 7f64580e2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6450106f20 0x7f645019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:28.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.558+0000 7f64580e2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6450111fa0 con 0x7f6450101df0 2024-09-19T08:51:28.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.558+0000 7f64580e2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6450111e20 con 0x7f6450106f20 2024-09-19T08:51:28.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.558+0000 7f64580e2640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6450112120 con 0x7f64501027a0 2024-09-19T08:51:28.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.558+0000 7f6455e7e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6450101df0 0x7f6450198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:28.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.558+0000 7f6455e7e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6450101df0 0x7f6450198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:32946/0 (socket says 172.21.15.164:32946) 2024-09-19T08:51:28.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.558+0000 7f6455e7e640 1 -- 172.21.15.164:0/1759486 learned_addr learned my addr 172.21.15.164:0/1759486 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:28.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.558+0000 7f645567d640 1 --2- 172.21.15.164:0/1759486 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64501027a0 0x7f6450198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:28.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.558+0000 7f645667f640 1 --2- 172.21.15.164:0/1759486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6450106f20 0x7f645019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:28.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.558+0000 7f6455e7e640 1 -- 172.21.15.164:0/1759486 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64501027a0 msgr2=0x7f6450198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:28.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.558+0000 7f6455e7e640 1 --2- 172.21.15.164:0/1759486 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64501027a0 0x7f6450198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.558+0000 7f6455e7e640 1 -- 172.21.15.164:0/1759486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6450106f20 msgr2=0x7f645019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:28.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.558+0000 7f6455e7e640 1 --2- 172.21.15.164:0/1759486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6450106f20 0x7f645019cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.558+0000 7f6455e7e640 1 -- 172.21.15.164:0/1759486 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645019d4b0 con 0x7f6450101df0 2024-09-19T08:51:28.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.558+0000 7f6455e7e640 1 --2- 172.21.15.164:0/1759486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6450101df0 0x7f6450198500 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f644c006010 tx=0x7f644c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:28.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.558+0000 7f6442ffd640 1 -- 172.21.15.164:0/1759486 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c03f030 con 0x7f6450101df0 2024-09-19T08:51:28.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.558+0000 7f6442ffd640 1 -- 172.21.15.164:0/1759486 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f644c003ce0 con 0x7f6450101df0 2024-09-19T08:51:28.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.558+0000 7f64580e2640 1 -- 172.21.15.164:0/1759486 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f645019d740 con 0x7f6450101df0 2024-09-19T08:51:28.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.558+0000 7f6442ffd640 1 -- 172.21.15.164:0/1759486 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c035070 con 0x7f6450101df0 2024-09-19T08:51:28.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.558+0000 7f64580e2640 1 -- 172.21.15.164:0/1759486 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64501a5080 con 0x7f6450101df0 2024-09-19T08:51:28.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.560+0000 7f6442ffd640 1 -- 172.21.15.164:0/1759486 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f644c042030 con 0x7f6450101df0 2024-09-19T08:51:28.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.561+0000 7f645a4bb740 1 Processor -- start 2024-09-19T08:51:28.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.561+0000 7f645a4bb740 1 -- start start 2024-09-19T08:51:28.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.561+0000 7f645a4bb740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e66bcbd060 0x55e66bcbd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:28.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.561+0000 7f6442ffd640 1 --2- 172.21.15.164:0/1759486 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f643c0a33b0 0x7f643c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:28.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.561+0000 7f645a4bb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e66bcbd980 0x55e66bcc7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:28.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.561+0000 7f645a4bb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e66bcc8240 0x55e66bcca630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:28.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.561+0000 7f645a4bb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e66bc22620 con 0x55e66bcc8240 2024-09-19T08:51:28.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.561+0000 7f645a4bb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e66bbc9e50 con 0x55e66bcbd980 2024-09-19T08:51:28.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.561+0000 7f645a4bb740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e66bc1e5e0 con 0x55e66bcbd060 2024-09-19T08:51:28.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.561+0000 7f6455e7e640 1 --2- 172.21.15.164:0/1759486 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f643c0a33b0 0x7f643c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:28.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.561+0000 7f645667f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e66bcc8240 0x55e66bcca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:28.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.561+0000 7f645667f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e66bcc8240 0x55e66bcca630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:32962/0 (socket says 172.21.15.164:32962) 2024-09-19T08:51:28.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.561+0000 7f645667f640 1 -- 172.21.15.164:0/854342416 learned_addr learned my addr 172.21.15.164:0/854342416 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:28.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.562+0000 7f645567d640 1 --2- 172.21.15.164:0/854342416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e66bcbd980 0x55e66bcc7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:28.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.562+0000 7f645667f640 1 --2- 172.21.15.164:0/854342416 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e66bcbd060 0x55e66bcbd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:28.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.562+0000 7f6442ffd640 1 -- 172.21.15.164:0/1759486 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f644c0f0480 con 0x7f6450101df0 2024-09-19T08:51:28.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.562+0000 7f645667f640 1 -- 172.21.15.164:0/854342416 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e66bcbd060 msgr2=0x55e66bcbd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:28.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.562+0000 7f645667f640 1 --2- 172.21.15.164:0/854342416 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e66bcbd060 0x55e66bcbd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.562+0000 7f645667f640 1 -- 172.21.15.164:0/854342416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e66bcbd980 msgr2=0x55e66bcc7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:28.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.562+0000 7f645667f640 1 --2- 172.21.15.164:0/854342416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e66bcbd980 0x55e66bcc7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.562+0000 7f645667f640 1 -- 172.21.15.164:0/854342416 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e66bbbad40 con 0x55e66bcc8240 2024-09-19T08:51:28.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.562+0000 7f6455e7e640 1 --2- 172.21.15.164:0/1759486 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f643c0a33b0 0x7f643c0a5870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f644c0b9200 tx=0x7f644c02a410 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:28.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.562+0000 7f645667f640 1 --2- 172.21.15.164:0/854342416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e66bcc8240 0x55e66bcca630 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6448014940 tx=0x7f64480017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4111ff0284b34791 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:28.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.562+0000 7f643bfff640 1 -- 172.21.15.164:0/854342416 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644801d070 con 0x55e66bcc8240 2024-09-19T08:51:28.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.562+0000 7f643bfff640 1 -- 172.21.15.164:0/854342416 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f644800de30 con 0x55e66bcc8240 2024-09-19T08:51:28.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.562+0000 7f643bfff640 1 -- 172.21.15.164:0/854342416 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6448018c20 con 0x55e66bcc8240 2024-09-19T08:51:28.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.563+0000 7f645a4bb740 1 -- 172.21.15.164:0/854342416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e66bcc8240 msgr2=0x55e66bcca630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:28.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.563+0000 7f645a4bb740 1 --2- 172.21.15.164:0/854342416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e66bcc8240 0x55e66bcca630 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6448014940 tx=0x7f64480017a0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.563+0000 7f645a4bb740 1 -- 172.21.15.164:0/854342416 shutdown_connections 2024-09-19T08:51:28.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.563+0000 7f645a4bb740 1 --2- 172.21.15.164:0/854342416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e66bcc8240 0x55e66bcca630 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.563+0000 7f645a4bb740 1 --2- 172.21.15.164:0/854342416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e66bcbd980 0x55e66bcc7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.563+0000 7f645a4bb740 1 --2- 172.21.15.164:0/854342416 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e66bcbd060 0x55e66bcbd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.563+0000 7f645a4bb740 1 -- 172.21.15.164:0/854342416 >> 172.21.15.164:0/854342416 conn(0x55e66bcb95f0 msgr2=0x55e66bcb99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:28.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.563+0000 7f645a4bb740 1 -- 172.21.15.164:0/854342416 shutdown_connections 2024-09-19T08:51:28.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.563+0000 7f645a4bb740 1 -- 172.21.15.164:0/854342416 wait complete. 2024-09-19T08:51:28.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.563+0000 7f645a4bb740 1 -- start start 2024-09-19T08:51:28.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.563+0000 7f645a4bb740 1 Processor -- start 2024-09-19T08:51:28.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.564+0000 7f645a4bb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e66bcbd060 0x55e66bcc5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:28.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.564+0000 7f645a4bb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e66bcbd980 0x55e66bcc5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:28.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.564+0000 7f645a4bb740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e66bcc8240 0x55e66bcc5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:28.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.564+0000 7f645a4bb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e66bccae10 con 0x55e66bcbd980 2024-09-19T08:51:28.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.564+0000 7f645a4bb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e66bbaaf10 con 0x55e66bcbd060 2024-09-19T08:51:28.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.564+0000 7f645a4bb740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e66bccaf90 con 0x55e66bcc8240 2024-09-19T08:51:28.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.564+0000 7f645a4bb740 10 client.0 _ll_register_callbacks cb 0x55e66bbcedb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:51:28.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.564+0000 7f645567d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e66bcbd980 0x55e66bcc5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:28.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.564+0000 7f645567d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e66bcbd980 0x55e66bcc5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:32974/0 (socket says 172.21.15.164:32974) 2024-09-19T08:51:28.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.564+0000 7f645567d640 1 -- 172.21.15.164:0/418554383 learned_addr learned my addr 172.21.15.164:0/418554383 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:28.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.564+0000 7f645667f640 1 --2- 172.21.15.164:0/418554383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e66bcbd060 0x55e66bcc5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:28.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.564+0000 7f645667f640 1 --2- 172.21.15.164:0/418554383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e66bcc8240 0x55e66bcc5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:28.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.564+0000 7f645567d640 1 -- 172.21.15.164:0/418554383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e66bcc8240 msgr2=0x55e66bcc5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:28.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.564+0000 7f645567d640 1 --2- 172.21.15.164:0/418554383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e66bcc8240 0x55e66bcc5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.564+0000 7f645567d640 1 -- 172.21.15.164:0/418554383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e66bcbd060 msgr2=0x55e66bcc5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:28.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.564+0000 7f645567d640 1 --2- 172.21.15.164:0/418554383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e66bcbd060 0x55e66bcc5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.564+0000 7f645567d640 1 -- 172.21.15.164:0/418554383 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e66b931580 con 0x55e66bcbd980 2024-09-19T08:51:28.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.564+0000 7f645667f640 1 --2- 172.21.15.164:0/418554383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e66bcc8240 0x55e66bcc5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:51:28.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.564+0000 7f645567d640 1 --2- 172.21.15.164:0/418554383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e66bcbd980 0x55e66bcc5750 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f644400bf30 tx=0x7f64440128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=920c3e3e7b696ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:28.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.564+0000 7f64397fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:51:28.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.564+0000 7f64397fa640 1 -- 172.21.15.164:0/418554383 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644401f060 con 0x55e66bcbd980 2024-09-19T08:51:28.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.564+0000 7f64397fa640 1 -- 172.21.15.164:0/418554383 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f64440108a0 con 0x55e66bcbd980 2024-09-19T08:51:28.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.564+0000 7f645a4bb740 1 -- 172.21.15.164:0/418554383 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f644800de30 con 0x55e66bcbd980 2024-09-19T08:51:28.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.565+0000 7f64397fa640 1 -- 172.21.15.164:0/418554383 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6444010aa0 con 0x55e66bcbd980 2024-09-19T08:51:28.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.565+0000 7f645a4bb740 10 client.16686 fetch_fsmap learned FSMap version 30 2024-09-19T08:51:28.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.565+0000 7f645a4bb740 1 -- 172.21.15.164:0/418554383 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e66b9fb960 con 0x55e66bcbd980 2024-09-19T08:51:28.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.565+0000 7f64397fa640 1 -- 172.21.15.164:0/418554383 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f644800de30 con 0x55e66bcbd980 2024-09-19T08:51:28.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.565+0000 7f64397fa640 1 -- 172.21.15.164:0/418554383 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f6444022020 con 0x55e66bcbd980 2024-09-19T08:51:28.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.565+0000 7f645a4bb740 10 client.16686 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:51:28.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.565+0000 7f645a4bb740 10 client.16686 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:51:28.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.565+0000 7f645a4bb740 20 client.16686 populate_metadata read hostname 'smithi164' 2024-09-19T08:51:28.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.565+0000 7f645a4bb740 1 --2- 172.21.15.164:0/418554383 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55e66bcd7ee0 0x55e66bcda2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:28.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.565+0000 7f645a4bb740 4 client.16686 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:51:28.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.565+0000 7f645a4bb740 1 -- 172.21.15.164:0/418554383 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e66bc2fb90 con 0x55e66bcd7ee0 2024-09-19T08:51:28.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.565+0000 7f645667f640 1 --2- 172.21.15.164:0/418554383 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55e66bcd7ee0 0x55e66bcda2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:28.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.566+0000 7f64397fa640 10 client.16686 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:51:28.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.566+0000 7f645667f640 1 --2- 172.21.15.164:0/418554383 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55e66bcd7ee0 0x55e66bcda2f0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cfa8faa37f33d2b server_cookie=9021a7006231e708 in_seq=0 out_seq=0 2024-09-19T08:51:28.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.566+0000 7f64397fa640 1 -- 172.21.15.164:0/418554383 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e66bc2fb90 con 0x55e66bcd7ee0 2024-09-19T08:51:28.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.566+0000 7f64397fa640 10 client.16686 handle_command_reply: tid=0 2024-09-19T08:51:28.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.566+0000 7f645a4bb740 1 client.16686 shutdown 2024-09-19T08:51:28.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.567+0000 7f645a4bb740 1 -- 172.21.15.164:0/418554383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e66bcbd980 msgr2=0x55e66bcc5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:28.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.567+0000 7f645a4bb740 1 --2- 172.21.15.164:0/418554383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e66bcbd980 0x55e66bcc5750 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f644400bf30 tx=0x7f64440128e0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.567+0000 7f645a4bb740 1 -- 172.21.15.164:0/418554383 shutdown_connections 2024-09-19T08:51:28.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.567+0000 7f645a4bb740 1 --2- 172.21.15.164:0/418554383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e66bcc8240 0x55e66bcc5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.567+0000 7f645a4bb740 1 --2- 172.21.15.164:0/418554383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e66bcbd980 0x55e66bcc5750 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.567+0000 7f645a4bb740 1 --2- 172.21.15.164:0/418554383 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55e66bcd7ee0 0x55e66bcda2f0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.567+0000 7f645a4bb740 1 --2- 172.21.15.164:0/418554383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e66bcbd060 0x55e66bcc5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.567+0000 7f645a4bb740 1 -- 172.21.15.164:0/418554383 >> 172.21.15.164:0/418554383 conn(0x55e66bcb95f0 msgr2=0x55e66bcbf2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:28.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.567+0000 7f64397fa640 0 client.16686 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:51:28.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.567+0000 7f645a4bb740 1 -- 172.21.15.164:0/418554383 shutdown_connections 2024-09-19T08:51:28.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.567+0000 7f645a4bb740 1 -- 172.21.15.164:0/418554383 wait complete. 2024-09-19T08:51:28.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.567+0000 7f645a4bb740 20 client.16686 trim_cache size 0 max 16384 2024-09-19T08:51:28.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.583+0000 7f645a4bb740 1 Processor -- start 2024-09-19T08:51:28.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.583+0000 7f645a4bb740 1 -- start start 2024-09-19T08:51:28.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.584+0000 7f645a4bb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e66bcbd060 0x55e66bcb8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:28.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.584+0000 7f645a4bb740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e66bcbd980 0x55e66bcdfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:28.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.584+0000 7f645a4bb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e66bcc8240 0x55e66bce0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:28.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.584+0000 7f645a4bb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f64501ab530 con 0x55e66bcbd060 2024-09-19T08:51:28.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.584+0000 7f645a4bb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f64501ad130 con 0x55e66bcc8240 2024-09-19T08:51:28.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.584+0000 7f645a4bb740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f64501abc00 con 0x55e66bcbd980 2024-09-19T08:51:28.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.584+0000 7f645667f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e66bcbd060 0x55e66bcb8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:28.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.584+0000 7f645667f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e66bcbd060 0x55e66bcb8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:32986/0 (socket says 172.21.15.164:32986) 2024-09-19T08:51:28.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.584+0000 7f645667f640 1 -- 172.21.15.164:0/1813300468 learned_addr learned my addr 172.21.15.164:0/1813300468 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:28.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.584+0000 7f645667f640 1 -- 172.21.15.164:0/1813300468 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e66bcbd980 msgr2=0x55e66bcdfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:28.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.584+0000 7f645667f640 1 --2- 172.21.15.164:0/1813300468 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e66bcbd980 0x55e66bcdfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.584+0000 7f645667f640 1 -- 172.21.15.164:0/1813300468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e66bcc8240 msgr2=0x55e66bce0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:28.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.584+0000 7f645667f640 1 --2- 172.21.15.164:0/1813300468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e66bcc8240 0x55e66bce0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.584+0000 7f645667f640 1 -- 172.21.15.164:0/1813300468 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e66bcc41c0 con 0x55e66bcbd060 2024-09-19T08:51:28.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.584+0000 7f645667f640 1 --2- 172.21.15.164:0/1813300468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e66bcbd060 0x55e66bcb8640 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f6448004b90 tx=0x7f6448001ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1600c934c330a08b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:28.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.584+0000 7f643b7fe640 1 -- 172.21.15.164:0/1813300468 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644802f030 con 0x55e66bcbd060 2024-09-19T08:51:28.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.584+0000 7f643b7fe640 1 -- 172.21.15.164:0/1813300468 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f644801e040 con 0x55e66bcbd060 2024-09-19T08:51:28.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.584+0000 7f643b7fe640 1 -- 172.21.15.164:0/1813300468 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644801d040 con 0x55e66bcbd060 2024-09-19T08:51:28.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.585+0000 7f645a4bb740 1 -- 172.21.15.164:0/1813300468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e66bcbd060 msgr2=0x55e66bcb8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:28.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.585+0000 7f645a4bb740 1 --2- 172.21.15.164:0/1813300468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e66bcbd060 0x55e66bcb8640 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f6448004b90 tx=0x7f6448001ec0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.585+0000 7f645a4bb740 1 -- 172.21.15.164:0/1813300468 shutdown_connections 2024-09-19T08:51:28.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.585+0000 7f645a4bb740 1 --2- 172.21.15.164:0/1813300468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e66bcc8240 0x55e66bce0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.585+0000 7f645a4bb740 1 --2- 172.21.15.164:0/1813300468 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e66bcbd980 0x55e66bcdfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.585+0000 7f645a4bb740 1 --2- 172.21.15.164:0/1813300468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e66bcbd060 0x55e66bcb8640 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.585+0000 7f645a4bb740 1 -- 172.21.15.164:0/1813300468 >> 172.21.15.164:0/1813300468 conn(0x55e66bcb95f0 msgr2=0x55e66bcb7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:28.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.585+0000 7f645a4bb740 1 -- 172.21.15.164:0/1813300468 shutdown_connections 2024-09-19T08:51:28.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.585+0000 7f645a4bb740 1 -- 172.21.15.164:0/1813300468 wait complete. 2024-09-19T08:51:28.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.585+0000 7f645a4bb740 1 -- start start 2024-09-19T08:51:28.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.586+0000 7f645a4bb740 1 Processor -- start 2024-09-19T08:51:28.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.586+0000 7f645a4bb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e66bcbd060 0x55e66bcd0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:28.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.586+0000 7f645a4bb740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e66bcbd980 0x55e66bcd0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:28.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.586+0000 7f645a4bb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e66bcc8240 0x55e66bccd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:28.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.586+0000 7f645a4bb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f64501ac9a0 con 0x55e66bcbd060 2024-09-19T08:51:28.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.586+0000 7f645a4bb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f64501ac2d0 con 0x55e66bcc8240 2024-09-19T08:51:28.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.586+0000 7f645a4bb740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f64501a9510 con 0x55e66bcbd980 2024-09-19T08:51:28.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.586+0000 7f645a4bb740 10 client.0 _ll_register_callbacks cb 0x7f644400bb70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:51:28.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.586+0000 7f645667f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e66bcbd060 0x55e66bcd0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:28.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.586+0000 7f645667f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e66bcbd060 0x55e66bcd0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:32994/0 (socket says 172.21.15.164:32994) 2024-09-19T08:51:28.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.586+0000 7f645667f640 1 -- 172.21.15.164:0/440626914 learned_addr learned my addr 172.21.15.164:0/440626914 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:28.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.586+0000 7f645567d640 1 --2- 172.21.15.164:0/440626914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e66bcbd980 0x55e66bcd0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:28.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.586+0000 7f645667f640 1 --2- 172.21.15.164:0/440626914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e66bcc8240 0x55e66bccd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:28.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.586+0000 7f645667f640 1 -- 172.21.15.164:0/440626914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e66bcbd980 msgr2=0x55e66bcd0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:28.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.586+0000 7f645667f640 1 --2- 172.21.15.164:0/440626914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e66bcbd980 0x55e66bcd0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.586+0000 7f645667f640 1 -- 172.21.15.164:0/440626914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e66bcc8240 msgr2=0x55e66bccd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:28.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.586+0000 7f645667f640 1 --2- 172.21.15.164:0/440626914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e66bcc8240 0x55e66bccd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.586+0000 7f645667f640 1 -- 172.21.15.164:0/440626914 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e66bccda20 con 0x55e66bcbd060 2024-09-19T08:51:28.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.587+0000 7f645667f640 1 --2- 172.21.15.164:0/440626914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e66bcbd060 0x55e66bcd0450 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f6448002030 tx=0x7f64480148c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32071a7398977680 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:28.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.587+0000 7f64397fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:51:28.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.587+0000 7f64397fa640 1 -- 172.21.15.164:0/440626914 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644803a030 con 0x55e66bcbd060 2024-09-19T08:51:28.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.587+0000 7f645a4bb740 1 -- 172.21.15.164:0/440626914 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f644801e040 con 0x55e66bcbd060 2024-09-19T08:51:28.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.587+0000 7f64397fa640 1 -- 172.21.15.164:0/440626914 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f6448018990 con 0x55e66bcbd060 2024-09-19T08:51:28.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.587+0000 7f64397fa640 1 -- 172.21.15.164:0/440626914 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f644801d040 con 0x55e66bcbd060 2024-09-19T08:51:28.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.587+0000 7f645a4bb740 10 client.16692 fetch_fsmap learned FSMap version 30 2024-09-19T08:51:28.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.587+0000 7f645a4bb740 1 -- 172.21.15.164:0/440626914 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e66bd43780 con 0x55e66bcbd060 2024-09-19T08:51:28.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.587+0000 7f64397fa640 1 -- 172.21.15.164:0/440626914 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f644801e040 con 0x55e66bcbd060 2024-09-19T08:51:28.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.587+0000 7f64397fa640 1 -- 172.21.15.164:0/440626914 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f6448024430 con 0x55e66bcbd060 2024-09-19T08:51:28.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.587+0000 7f645a4bb740 10 client.16692 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:51:28.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.587+0000 7f645a4bb740 10 client.16692 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:51:28.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.587+0000 7f645a4bb740 20 client.16692 populate_metadata read hostname 'smithi164' 2024-09-19T08:51:28.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.587+0000 7f645a4bb740 1 --2- 172.21.15.164:0/440626914 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55e66bd43ba0 0x55e66bd45f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:28.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.587+0000 7f645a4bb740 4 client.16692 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "21b3571c", "release": true, "await": true}] 2024-09-19T08:51:28.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.587+0000 7f645a4bb740 1 -- 172.21.15.164:0/440626914 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "quiesce db", "set_id": "21b3571c", "release": true, "await": true}) -- 0x55e66bc2a0a0 con 0x55e66bd43ba0 2024-09-19T08:51:28.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.588+0000 7f645567d640 1 --2- 172.21.15.164:0/440626914 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55e66bd43ba0 0x55e66bd45f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:28.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.588+0000 7f645567d640 1 --2- 172.21.15.164:0/440626914 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55e66bd43ba0 0x55e66bd45f90 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fc447801f53cee2 server_cookie=692ce35adf52fd6f in_seq=0 out_seq=0 2024-09-19T08:51:28.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.588+0000 7f64397fa640 10 client.16692 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:51:28.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.593+0000 7f64397fa640 1 -- 172.21.15.164:0/440626914 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e66bc2a0a0 con 0x55e66bd43ba0 2024-09-19T08:51:28.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.593+0000 7f64397fa640 10 client.16692 handle_command_reply: tid=0 2024-09-19T08:51:28.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.593+0000 7f645a4bb740 1 client.16692 shutdown 2024-09-19T08:51:28.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.593+0000 7f645a4bb740 1 -- 172.21.15.164:0/440626914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e66bcbd060 msgr2=0x55e66bcd0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:28.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.593+0000 7f645a4bb740 1 --2- 172.21.15.164:0/440626914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e66bcbd060 0x55e66bcd0450 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f6448002030 tx=0x7f64480148c0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.593+0000 7f645a4bb740 1 -- 172.21.15.164:0/440626914 shutdown_connections 2024-09-19T08:51:28.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.593+0000 7f645a4bb740 1 --2- 172.21.15.164:0/440626914 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55e66bd43ba0 0x55e66bd45f90 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.593+0000 7f645a4bb740 1 --2- 172.21.15.164:0/440626914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e66bcc8240 0x55e66bccd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.593+0000 7f645a4bb740 1 --2- 172.21.15.164:0/440626914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e66bcbd980 0x55e66bcd0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.593+0000 7f64397fa640 0 client.16692 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:51:28.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.593+0000 7f645a4bb740 1 --2- 172.21.15.164:0/440626914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e66bcbd060 0x55e66bcd0450 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.593+0000 7f645a4bb740 1 -- 172.21.15.164:0/440626914 >> 172.21.15.164:0/440626914 conn(0x55e66bcb95f0 msgr2=0x55e66bcb8750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:28.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.594+0000 7f645a4bb740 1 -- 172.21.15.164:0/440626914 shutdown_connections 2024-09-19T08:51:28.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.594+0000 7f645a4bb740 1 -- 172.21.15.164:0/440626914 wait complete. 2024-09-19T08:51:28.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.594+0000 7f645a4bb740 20 client.16692 trim_cache size 0 max 16384 2024-09-19T08:51:28.595 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:51:28.595 INFO:teuthology.orchestra.run.smithi164.stdout: "epoch": 30, 2024-09-19T08:51:28.595 INFO:teuthology.orchestra.run.smithi164.stdout: "leader": 24479, 2024-09-19T08:51:28.595 INFO:teuthology.orchestra.run.smithi164.stdout: "set_version": 24, 2024-09-19T08:51:28.595 INFO:teuthology.orchestra.run.smithi164.stdout: "sets": { 2024-09-19T08:51:28.595 INFO:teuthology.orchestra.run.smithi164.stdout: "21b3571c": { 2024-09-19T08:51:28.595 INFO:teuthology.orchestra.run.smithi164.stdout: "version": 24, 2024-09-19T08:51:28.595 INFO:teuthology.orchestra.run.smithi164.stdout: "age_ref": 0.0, 2024-09-19T08:51:28.595 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:51:28.595 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "RELEASED", 2024-09-19T08:51:28.596 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:51:28.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:51:28.596 INFO:teuthology.orchestra.run.smithi164.stdout: "timeout": 90.0, 2024-09-19T08:51:28.596 INFO:teuthology.orchestra.run.smithi164.stdout: "expiration": 149.8, 2024-09-19T08:51:28.596 INFO:teuthology.orchestra.run.smithi164.stdout: "members": { 2024-09-19T08:51:28.596 INFO:teuthology.orchestra.run.smithi164.stdout: "file:/": { 2024-09-19T08:51:28.596 INFO:teuthology.orchestra.run.smithi164.stdout: "excluded": false, 2024-09-19T08:51:28.596 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:51:28.596 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "RELEASED", 2024-09-19T08:51:28.596 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:51:28.596 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:51:28.596 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:51:28.597 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:51:28.597 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:51:28.597 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:51:28.597 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:51:28.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.595+0000 7f64580e2640 1 -- 172.21.15.164:0/1759486 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f643c0a33b0 msgr2=0x7f643c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:28.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.595+0000 7f64580e2640 1 --2- 172.21.15.164:0/1759486 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f643c0a33b0 0x7f643c0a5870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f644c0b9200 tx=0x7f644c02a410 comp rx=0 tx=0).stop 2024-09-19T08:51:28.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.595+0000 7f64580e2640 1 -- 172.21.15.164:0/1759486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6450101df0 msgr2=0x7f6450198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:28.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.595+0000 7f64580e2640 1 --2- 172.21.15.164:0/1759486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6450101df0 0x7f6450198500 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f644c006010 tx=0x7f644c0042f0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.595+0000 7f64580e2640 1 -- 172.21.15.164:0/1759486 shutdown_connections 2024-09-19T08:51:28.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.595+0000 7f64580e2640 1 --2- 172.21.15.164:0/1759486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6450106f20 0x7f645019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.595+0000 7f64580e2640 1 --2- 172.21.15.164:0/1759486 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64501027a0 0x7f6450198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.595+0000 7f64580e2640 1 --2- 172.21.15.164:0/1759486 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f643c0a33b0 0x7f643c0a5870 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.595+0000 7f64580e2640 1 --2- 172.21.15.164:0/1759486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6450101df0 0x7f6450198500 secure :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f644c006010 tx=0x7f644c0042f0 comp rx=0 tx=0).stop 2024-09-19T08:51:28.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.595+0000 7f64580e2640 1 -- 172.21.15.164:0/1759486 >> 172.21.15.164:0/1759486 conn(0x7f64500fd8f0 msgr2=0x7f64500fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:28.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.595+0000 7f64580e2640 1 -- 172.21.15.164:0/1759486 shutdown_connections 2024-09-19T08:51:28.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:28.596+0000 7f64580e2640 1 -- 172.21.15.164:0/1759486 wait complete. 2024-09-19T08:51:29.792 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '21b3571c', total seconds elapsed: 35.2 2024-09-19T08:51:29.809 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:51:29.982 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:29 smithi164 ceph-mon[30928]: pgmap v173: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 65 MiB/s wr, 107 op/s 2024-09-19T08:51:30.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:29 smithi170 ceph-mon[35072]: pgmap v173: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 65 MiB/s wr, 107 op/s 2024-09-19T08:51:30.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:29 smithi173 ceph-mon[35306]: pgmap v173: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 65 MiB/s wr, 107 op/s 2024-09-19T08:51:30.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.358+0000 7f91bac1b640 1 -- 172.21.15.164:0/2057482164 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91b4102d30 msgr2=0x7f91b4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:30.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.358+0000 7f91bac1b640 1 --2- 172.21.15.164:0/2057482164 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91b4102d30 0x7f91b4103110 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f91a8009860 tx=0x7f91a802a340 comp rx=0 tx=0).stop 2024-09-19T08:51:30.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.358+0000 7f91bac1b640 1 -- 172.21.15.164:0/2057482164 shutdown_connections 2024-09-19T08:51:30.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.358+0000 7f91bac1b640 1 --2- 172.21.15.164:0/2057482164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91b41082a0 0x7f91b410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.358+0000 7f91bac1b640 1 --2- 172.21.15.164:0/2057482164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91b41036e0 0x7f91b4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.358+0000 7f91bac1b640 1 --2- 172.21.15.164:0/2057482164 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91b4102d30 0x7f91b4103110 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.358+0000 7f91bac1b640 1 -- 172.21.15.164:0/2057482164 >> 172.21.15.164:0/2057482164 conn(0x7f91b40fc7d0 msgr2=0x7f91b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:30.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.359+0000 7f91bac1b640 1 -- 172.21.15.164:0/2057482164 shutdown_connections 2024-09-19T08:51:30.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.359+0000 7f91bac1b640 1 -- 172.21.15.164:0/2057482164 wait complete. 2024-09-19T08:51:30.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.360+0000 7f91bac1b640 1 Processor -- start 2024-09-19T08:51:30.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.360+0000 7f91bac1b640 1 -- start start 2024-09-19T08:51:30.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.360+0000 7f91bac1b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91b4102d30 0x7f91b41963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:30.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.361+0000 7f91bac1b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91b41036e0 0x7f91b41968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:30.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.361+0000 7f91bac1b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91b41082a0 0x7f91b419ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:30.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.361+0000 7f91b3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91b41036e0 0x7f91b41968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:30.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.361+0000 7f91b3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91b41036e0 0x7f91b41968f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33002/0 (socket says 172.21.15.164:33002) 2024-09-19T08:51:30.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.361+0000 7f91bac1b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f91b410fe90 con 0x7f91b41036e0 2024-09-19T08:51:30.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.361+0000 7f91bac1b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f91b410fd10 con 0x7f91b41082a0 2024-09-19T08:51:30.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.361+0000 7f91bac1b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f91b4110010 con 0x7f91b4102d30 2024-09-19T08:51:30.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.361+0000 7f91b3fff640 1 -- 172.21.15.164:0/226933976 learned_addr learned my addr 172.21.15.164:0/226933976 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:30.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.361+0000 7f91b89b7640 1 --2- 172.21.15.164:0/226933976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91b4102d30 0x7f91b41963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:30.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.361+0000 7f91b91b8640 1 --2- 172.21.15.164:0/226933976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91b41082a0 0x7f91b419ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:30.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.361+0000 7f91b3fff640 1 -- 172.21.15.164:0/226933976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91b4102d30 msgr2=0x7f91b41963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:30.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.361+0000 7f91b3fff640 1 --2- 172.21.15.164:0/226933976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91b4102d30 0x7f91b41963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.361+0000 7f91b3fff640 1 -- 172.21.15.164:0/226933976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91b41082a0 msgr2=0x7f91b419ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:30.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.361+0000 7f91b3fff640 1 --2- 172.21.15.164:0/226933976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91b41082a0 0x7f91b419ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.362+0000 7f91b3fff640 1 -- 172.21.15.164:0/226933976 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b419b360 con 0x7f91b41036e0 2024-09-19T08:51:30.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.362+0000 7f91b3fff640 1 --2- 172.21.15.164:0/226933976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91b41036e0 0x7f91b41968f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f91a0012bc0 tx=0x7f91a0010490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:30.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.362+0000 7f91b1ffb640 1 -- 172.21.15.164:0/226933976 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a0010ba0 con 0x7f91b41036e0 2024-09-19T08:51:30.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.362+0000 7f91b1ffb640 1 -- 172.21.15.164:0/226933976 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91a0010d40 con 0x7f91b41036e0 2024-09-19T08:51:30.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.362+0000 7f91bac1b640 1 -- 172.21.15.164:0/226933976 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b419b5f0 con 0x7f91b41036e0 2024-09-19T08:51:30.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.362+0000 7f91b1ffb640 1 -- 172.21.15.164:0/226933976 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a0004ad0 con 0x7f91b41036e0 2024-09-19T08:51:30.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.362+0000 7f91bac1b640 1 -- 172.21.15.164:0/226933976 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b41a2f30 con 0x7f91b41036e0 2024-09-19T08:51:30.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.366+0000 7f91b1ffb640 1 -- 172.21.15.164:0/226933976 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f91a001f020 con 0x7f91b41036e0 2024-09-19T08:51:30.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.366+0000 7f91bcff4740 1 Processor -- start 2024-09-19T08:51:30.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.366+0000 7f91bcff4740 1 -- start start 2024-09-19T08:51:30.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.367+0000 7f91bcff4740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555832eb7a60 0x555832eb7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:30.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.367+0000 7f91bcff4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555832eb8380 0x555832ecb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:30.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.367+0000 7f91bcff4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555832ecb550 0x555832ecd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:30.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.367+0000 7f91bcff4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555832e1d280 con 0x555832ecb550 2024-09-19T08:51:30.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.367+0000 7f91bcff4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555832dc4b10 con 0x555832eb8380 2024-09-19T08:51:30.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.367+0000 7f91bcff4740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555832e19240 con 0x555832eb7a60 2024-09-19T08:51:30.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.367+0000 7f91b3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555832ecb550 0x555832ecd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:30.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.367+0000 7f91b3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555832ecb550 0x555832ecd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33008/0 (socket says 172.21.15.164:33008) 2024-09-19T08:51:30.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.367+0000 7f91b3fff640 1 -- 172.21.15.164:0/1903105668 learned_addr learned my addr 172.21.15.164:0/1903105668 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:30.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.367+0000 7f91b91b8640 1 --2- 172.21.15.164:0/1903105668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555832eb7a60 0x555832eb7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:30.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.368+0000 7f91b1ffb640 1 --2- 172.21.15.164:0/226933976 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f91a40a3430 0x7f91a40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:30.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.368+0000 7f91b3fff640 1 -- 172.21.15.164:0/1903105668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555832eb7a60 msgr2=0x555832eb7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:30.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.368+0000 7f91b3fff640 1 --2- 172.21.15.164:0/1903105668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555832eb7a60 0x555832eb7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.368+0000 7f91b3fff640 1 -- 172.21.15.164:0/1903105668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555832eb8380 msgr2=0x555832ecb010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:51:30.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.368+0000 7f91b3fff640 1 --2- 172.21.15.164:0/1903105668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555832eb8380 0x555832ecb010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.368+0000 7f91b3fff640 1 -- 172.21.15.164:0/1903105668 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555832dd11f0 con 0x555832ecb550 2024-09-19T08:51:30.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.368+0000 7f91b91b8640 1 --2- 172.21.15.164:0/1903105668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555832eb7a60 0x555832eb7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:51:30.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.368+0000 7f91b91b8640 1 --2- 172.21.15.164:0/226933976 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f91a40a3430 0x7f91a40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:30.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.368+0000 7f91b1ffb640 1 -- 172.21.15.164:0/226933976 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f91a00d3820 con 0x7f91b41036e0 2024-09-19T08:51:30.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.368+0000 7f91b3fff640 1 --2- 172.21.15.164:0/1903105668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555832ecb550 0x555832ecd940 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f91a00d15e0 tx=0x7f91a006a4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4822d540af6d08 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:30.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.368+0000 7f91b91b8640 1 --2- 172.21.15.164:0/226933976 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f91a40a3430 0x7f91a40a58f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f91ac005fd0 tx=0x7f91ac004a40 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:30.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.369+0000 7f919effd640 1 -- 172.21.15.164:0/1903105668 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a009f6b0 con 0x555832ecb550 2024-09-19T08:51:30.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.369+0000 7f919effd640 1 -- 172.21.15.164:0/1903105668 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91a006a9c0 con 0x555832ecb550 2024-09-19T08:51:30.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.369+0000 7f919effd640 1 -- 172.21.15.164:0/1903105668 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a009f4d0 con 0x555832ecb550 2024-09-19T08:51:30.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.369+0000 7f91bcff4740 1 -- 172.21.15.164:0/1903105668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555832ecb550 msgr2=0x555832ecd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:30.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.369+0000 7f91bcff4740 1 --2- 172.21.15.164:0/1903105668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555832ecb550 0x555832ecd940 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f91a00d15e0 tx=0x7f91a006a4b0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.369+0000 7f91bcff4740 1 -- 172.21.15.164:0/1903105668 shutdown_connections 2024-09-19T08:51:30.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.369+0000 7f91bcff4740 1 --2- 172.21.15.164:0/1903105668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555832ecb550 0x555832ecd940 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.369+0000 7f91bcff4740 1 --2- 172.21.15.164:0/1903105668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555832eb8380 0x555832ecb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.369+0000 7f91bcff4740 1 --2- 172.21.15.164:0/1903105668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555832eb7a60 0x555832eb7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.370+0000 7f91bcff4740 1 -- 172.21.15.164:0/1903105668 >> 172.21.15.164:0/1903105668 conn(0x555832eb40f0 msgr2=0x555832eb44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:30.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.370+0000 7f91bcff4740 1 -- 172.21.15.164:0/1903105668 shutdown_connections 2024-09-19T08:51:30.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.370+0000 7f91bcff4740 1 -- 172.21.15.164:0/1903105668 wait complete. 2024-09-19T08:51:30.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.370+0000 7f91bcff4740 1 -- start start 2024-09-19T08:51:30.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.370+0000 7f91bcff4740 1 Processor -- start 2024-09-19T08:51:30.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.371+0000 7f91bcff4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555832eb7a60 0x555832ec83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:30.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.371+0000 7f91bcff4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555832eb8380 0x555832ec8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:30.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.371+0000 7f91bcff4740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555832ecb550 0x555832ec8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:30.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.371+0000 7f91bcff4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555832ece120 con 0x555832eb7a60 2024-09-19T08:51:30.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.371+0000 7f91bcff4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555832dd7f10 con 0x555832eb8380 2024-09-19T08:51:30.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.371+0000 7f91b89b7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555832eb7a60 0x555832ec83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:30.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.371+0000 7f91bcff4740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555832ece2a0 con 0x555832ecb550 2024-09-19T08:51:30.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.371+0000 7f91b89b7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555832eb7a60 0x555832ec83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33022/0 (socket says 172.21.15.164:33022) 2024-09-19T08:51:30.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.371+0000 7f91b91b8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555832ecb550 0x555832ec8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:30.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.372+0000 7f91b91b8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555832ecb550 0x555832ec8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:34696/0 (socket says 172.21.15.164:34696) 2024-09-19T08:51:30.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.372+0000 7f91b91b8640 1 -- 172.21.15.164:0/1898978384 learned_addr learned my addr 172.21.15.164:0/1898978384 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:30.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.372+0000 7f91bcff4740 10 client.0 _ll_register_callbacks cb 0x555832da0a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:51:30.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.371+0000 7f91b3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555832eb8380 0x555832ec8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:30.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.372+0000 7f91b91b8640 1 -- 172.21.15.164:0/1898978384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555832eb8380 msgr2=0x555832ec8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:30.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.372+0000 7f91b91b8640 1 --2- 172.21.15.164:0/1898978384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555832eb8380 0x555832ec8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.372+0000 7f91b91b8640 1 -- 172.21.15.164:0/1898978384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555832eb7a60 msgr2=0x555832ec83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:30.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.372+0000 7f91b91b8640 1 --2- 172.21.15.164:0/1898978384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555832eb7a60 0x555832ec83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.372+0000 7f91b91b8640 1 -- 172.21.15.164:0/1898978384 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555832b8e6a0 con 0x555832ecb550 2024-09-19T08:51:30.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.373+0000 7f91b91b8640 1 --2- 172.21.15.164:0/1898978384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555832ecb550 0x555832ec8e70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f91ac00ffd0 tx=0x7f91ac004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=831312912605e2e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:30.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.373+0000 7f9193fff640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-09-19T08:51:30.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.373+0000 7f9193fff640 1 -- 172.21.15.164:0/1898978384 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ac03b060 con 0x555832ecb550 2024-09-19T08:51:30.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.373+0000 7f9193fff640 1 -- 172.21.15.164:0/1898978384 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f91ac0195c0 con 0x555832ecb550 2024-09-19T08:51:30.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.373+0000 7f91bcff4740 1 -- 172.21.15.164:0/1898978384 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91a006a9c0 con 0x555832ecb550 2024-09-19T08:51:30.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.373+0000 7f9193fff640 1 -- 172.21.15.164:0/1898978384 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f91ac01ae30 con 0x555832ecb550 2024-09-19T08:51:30.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.373+0000 7f91bcff4740 10 client.25990 fetch_fsmap learned FSMap version 30 2024-09-19T08:51:30.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.373+0000 7f91bcff4740 1 -- 172.21.15.164:0/1898978384 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555832ed9d90 con 0x555832ecb550 2024-09-19T08:51:30.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.373+0000 7f9193fff640 1 -- 172.21.15.164:0/1898978384 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91a006a9c0 con 0x555832ecb550 2024-09-19T08:51:30.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.374+0000 7f9193fff640 1 -- 172.21.15.164:0/1898978384 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f91ac03e020 con 0x555832ecb550 2024-09-19T08:51:30.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.374+0000 7f91bcff4740 10 client.25990 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:51:30.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.374+0000 7f91bcff4740 10 client.25990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:51:30.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.374+0000 7f91bcff4740 20 client.25990 populate_metadata read hostname 'smithi164' 2024-09-19T08:51:30.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.374+0000 7f91bcff4740 1 --2- 172.21.15.164:0/1898978384 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555832edb2f0 0x555832edd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:30.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.374+0000 7f91bcff4740 4 client.25990 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:51:30.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.374+0000 7f91bcff4740 1 -- 172.21.15.164:0/1898978384 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555832e2a7f0 con 0x555832edb2f0 2024-09-19T08:51:30.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.374+0000 7f91b89b7640 1 --2- 172.21.15.164:0/1898978384 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555832edb2f0 0x555832edd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:30.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.376+0000 7f9193fff640 10 client.25990 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:51:30.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.376+0000 7f91b89b7640 1 --2- 172.21.15.164:0/1898978384 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555832edb2f0 0x555832edd750 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec823a66018ad838 server_cookie=b71ea2f69c4207c7 in_seq=0 out_seq=0 2024-09-19T08:51:30.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.377+0000 7f9193fff640 1 -- 172.21.15.164:0/1898978384 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555832e2a7f0 con 0x555832edb2f0 2024-09-19T08:51:30.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.377+0000 7f9193fff640 10 client.25990 handle_command_reply: tid=0 2024-09-19T08:51:30.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.378+0000 7f91bcff4740 1 client.25990 shutdown 2024-09-19T08:51:30.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.378+0000 7f91bcff4740 1 -- 172.21.15.164:0/1898978384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555832ecb550 msgr2=0x555832ec8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:30.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.378+0000 7f91bcff4740 1 --2- 172.21.15.164:0/1898978384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555832ecb550 0x555832ec8e70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f91ac00ffd0 tx=0x7f91ac004ab0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.378+0000 7f91bcff4740 1 -- 172.21.15.164:0/1898978384 shutdown_connections 2024-09-19T08:51:30.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.378+0000 7f91bcff4740 1 --2- 172.21.15.164:0/1898978384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555832ecb550 0x555832ec8e70 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.378+0000 7f91bcff4740 1 --2- 172.21.15.164:0/1898978384 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555832edb2f0 0x555832edd750 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.378+0000 7f91bcff4740 1 --2- 172.21.15.164:0/1898978384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555832eb8380 0x555832ec8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.378+0000 7f91bcff4740 1 --2- 172.21.15.164:0/1898978384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555832eb7a60 0x555832ec83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.378+0000 7f91bcff4740 1 -- 172.21.15.164:0/1898978384 >> 172.21.15.164:0/1898978384 conn(0x555832eb40f0 msgr2=0x555832eb9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:30.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.378+0000 7f9193fff640 0 client.25990 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:51:30.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.379+0000 7f91bcff4740 1 -- 172.21.15.164:0/1898978384 shutdown_connections 2024-09-19T08:51:30.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.379+0000 7f91bcff4740 1 -- 172.21.15.164:0/1898978384 wait complete. 2024-09-19T08:51:30.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.379+0000 7f91bcff4740 20 client.25990 trim_cache size 0 max 16384 2024-09-19T08:51:30.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.406+0000 7f91bcff4740 1 Processor -- start 2024-09-19T08:51:30.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.406+0000 7f91bcff4740 1 -- start start 2024-09-19T08:51:30.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.407+0000 7f91bcff4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555832eb7a60 0x555832eb9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:30.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.407+0000 7f91bcff4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555832eb8380 0x555832ee3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:30.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.407+0000 7f91bcff4740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555832ecb550 0x555832ee35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:30.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.407+0000 7f91bcff4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f91b4074150 con 0x555832eb7a60 2024-09-19T08:51:30.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.407+0000 7f91bcff4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f91b41a4a40 con 0x555832eb8380 2024-09-19T08:51:30.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.407+0000 7f91bcff4740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f91b41a3510 con 0x555832ecb550 2024-09-19T08:51:30.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.407+0000 7f91b89b7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555832eb7a60 0x555832eb9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:30.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.407+0000 7f91b3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555832eb8380 0x555832ee3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:30.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.407+0000 7f91b89b7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555832eb7a60 0x555832eb9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33034/0 (socket says 172.21.15.164:33034) 2024-09-19T08:51:30.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.407+0000 7f91b91b8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555832ecb550 0x555832ee35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:30.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.407+0000 7f91b89b7640 1 -- 172.21.15.164:0/3626382273 learned_addr learned my addr 172.21.15.164:0/3626382273 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:30.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.407+0000 7f91b91b8640 1 -- 172.21.15.164:0/3626382273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555832eb8380 msgr2=0x555832ee3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:30.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.407+0000 7f91b91b8640 1 --2- 172.21.15.164:0/3626382273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555832eb8380 0x555832ee3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.407+0000 7f91b91b8640 1 -- 172.21.15.164:0/3626382273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555832eb7a60 msgr2=0x555832eb9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:30.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.407+0000 7f91b91b8640 1 --2- 172.21.15.164:0/3626382273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555832eb7a60 0x555832eb9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.407+0000 7f91b91b8640 1 -- 172.21.15.164:0/3626382273 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555832ec31e0 con 0x555832ecb550 2024-09-19T08:51:30.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.408+0000 7f91b91b8640 1 --2- 172.21.15.164:0/3626382273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555832ecb550 0x555832ee35a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f91ac00d910 tx=0x7f91ac00dbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=136582ad8d36e433 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:30.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.408+0000 7f919e7fc640 1 -- 172.21.15.164:0/3626382273 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ac040440 con 0x555832ecb550 2024-09-19T08:51:30.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.408+0000 7f919e7fc640 1 -- 172.21.15.164:0/3626382273 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91ac0405e0 con 0x555832ecb550 2024-09-19T08:51:30.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.408+0000 7f919e7fc640 1 -- 172.21.15.164:0/3626382273 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ac036530 con 0x555832ecb550 2024-09-19T08:51:30.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.409+0000 7f91bcff4740 1 -- 172.21.15.164:0/3626382273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555832ecb550 msgr2=0x555832ee35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:30.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.409+0000 7f91bcff4740 1 --2- 172.21.15.164:0/3626382273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555832ecb550 0x555832ee35a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f91ac00d910 tx=0x7f91ac00dbf0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.409+0000 7f91bcff4740 1 -- 172.21.15.164:0/3626382273 shutdown_connections 2024-09-19T08:51:30.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.409+0000 7f91bcff4740 1 --2- 172.21.15.164:0/3626382273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555832ecb550 0x555832ee35a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.409+0000 7f91bcff4740 1 --2- 172.21.15.164:0/3626382273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555832eb8380 0x555832ee3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.409+0000 7f91bcff4740 1 --2- 172.21.15.164:0/3626382273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555832eb7a60 0x555832eb9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.409+0000 7f91bcff4740 1 -- 172.21.15.164:0/3626382273 >> 172.21.15.164:0/3626382273 conn(0x555832eb40f0 msgr2=0x555832eb9590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:30.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.409+0000 7f91bcff4740 1 -- 172.21.15.164:0/3626382273 shutdown_connections 2024-09-19T08:51:30.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.409+0000 7f91bcff4740 1 -- 172.21.15.164:0/3626382273 wait complete. 2024-09-19T08:51:30.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.410+0000 7f91bcff4740 1 -- start start 2024-09-19T08:51:30.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.410+0000 7f91bcff4740 1 Processor -- start 2024-09-19T08:51:30.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.410+0000 7f91bcff4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555832eb7a60 0x555832ed3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:30.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.410+0000 7f91bcff4740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555832eb8380 0x555832ecfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:30.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.410+0000 7f91bcff4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555832ecb550 0x555832ed0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:30.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.410+0000 7f91bcff4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f91b41a42b0 con 0x555832eb7a60 2024-09-19T08:51:30.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.410+0000 7f91bcff4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f91b41a3be0 con 0x555832ecb550 2024-09-19T08:51:30.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.410+0000 7f91bcff4740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f91b4072130 con 0x555832eb8380 2024-09-19T08:51:30.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.410+0000 7f91bcff4740 10 client.0 _ll_register_callbacks cb 0x7f91ac009cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:51:30.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.411+0000 7f91b89b7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555832eb7a60 0x555832ed3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:30.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.411+0000 7f91b89b7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555832eb7a60 0x555832ed3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33038/0 (socket says 172.21.15.164:33038) 2024-09-19T08:51:30.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.411+0000 7f91b89b7640 1 -- 172.21.15.164:0/3276428643 learned_addr learned my addr 172.21.15.164:0/3276428643 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:30.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.411+0000 7f91b91b8640 1 --2- 172.21.15.164:0/3276428643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555832ecb550 0x555832ed0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:30.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.411+0000 7f91b89b7640 1 -- 172.21.15.164:0/3276428643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555832eb8380 msgr2=0x555832ecfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:30.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.411+0000 7f91b89b7640 1 --2- 172.21.15.164:0/3276428643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555832eb8380 0x555832ecfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.411+0000 7f91b89b7640 1 -- 172.21.15.164:0/3276428643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555832ecb550 msgr2=0x555832ed0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:30.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.411+0000 7f91b89b7640 1 --2- 172.21.15.164:0/3276428643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555832ecb550 0x555832ed0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.411+0000 7f91b89b7640 1 -- 172.21.15.164:0/3276428643 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555832ed0840 con 0x555832eb7a60 2024-09-19T08:51:30.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.411+0000 7f91b89b7640 1 --2- 172.21.15.164:0/3276428643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555832eb7a60 0x555832ed3a00 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f91a8009830 tx=0x7f91a8030fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3731e7fc7e61cdb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:30.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.411+0000 7f9193fff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:51:30.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.411+0000 7f9193fff640 1 -- 172.21.15.164:0/3276428643 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a804b060 con 0x555832eb7a60 2024-09-19T08:51:30.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.412+0000 7f9193fff640 1 -- 172.21.15.164:0/3276428643 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7f91a80308c0 con 0x555832eb7a60 2024-09-19T08:51:30.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.412+0000 7f91bcff4740 1 -- 172.21.15.164:0/3276428643 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91ac0405e0 con 0x555832eb7a60 2024-09-19T08:51:30.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.412+0000 7f9193fff640 1 -- 172.21.15.164:0/3276428643 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f91a8037040 con 0x555832eb7a60 2024-09-19T08:51:30.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.412+0000 7f91bcff4740 10 client.16710 fetch_fsmap learned FSMap version 30 2024-09-19T08:51:30.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.412+0000 7f91bcff4740 1 -- 172.21.15.164:0/3276428643 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555832f46df0 con 0x555832eb7a60 2024-09-19T08:51:30.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.412+0000 7f9193fff640 1 -- 172.21.15.164:0/3276428643 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91ac0405e0 con 0x555832eb7a60 2024-09-19T08:51:30.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.413+0000 7f9193fff640 1 -- 172.21.15.164:0/3276428643 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f91a804e020 con 0x555832eb7a60 2024-09-19T08:51:30.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.413+0000 7f91bcff4740 10 client.16710 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:51:30.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.413+0000 7f91bcff4740 10 client.16710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:51:30.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.413+0000 7f91bcff4740 20 client.16710 populate_metadata read hostname 'smithi164' 2024-09-19T08:51:30.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.413+0000 7f91bcff4740 1 --2- 172.21.15.164:0/3276428643 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555832f47600 0x555832f49a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:30.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.413+0000 7f91bcff4740 4 client.16710 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T08:51:30.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.413+0000 7f91bcff4740 1 -- 172.21.15.164:0/3276428643 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555832e24d00 con 0x555832f47600 2024-09-19T08:51:30.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.413+0000 7f91b3fff640 1 --2- 172.21.15.164:0/3276428643 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555832f47600 0x555832f49a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:30.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.415+0000 7f9193fff640 10 client.16710 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:51:30.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.415+0000 7f91b3fff640 1 --2- 172.21.15.164:0/3276428643 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555832f47600 0x555832f49a60 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d13558e0c606f3c server_cookie=49e7b3a56de8f0bf in_seq=0 out_seq=0 2024-09-19T08:51:30.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.416+0000 7f9193fff640 1 -- 172.21.15.164:0/3276428643 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555832e24d00 con 0x555832f47600 2024-09-19T08:51:30.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.416+0000 7f9193fff640 10 client.16710 handle_command_reply: tid=0 2024-09-19T08:51:30.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.416+0000 7f91bcff4740 1 client.16710 shutdown 2024-09-19T08:51:30.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.417+0000 7f91bcff4740 1 -- 172.21.15.164:0/3276428643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555832eb7a60 msgr2=0x555832ed3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:30.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.417+0000 7f91bcff4740 1 --2- 172.21.15.164:0/3276428643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555832eb7a60 0x555832ed3a00 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f91a8009830 tx=0x7f91a8030fc0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.417+0000 7f91bcff4740 1 -- 172.21.15.164:0/3276428643 shutdown_connections 2024-09-19T08:51:30.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.417+0000 7f91bcff4740 1 --2- 172.21.15.164:0/3276428643 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555832f47600 0x555832f49a60 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.417+0000 7f91bcff4740 1 --2- 172.21.15.164:0/3276428643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555832ecb550 0x555832ed0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.417+0000 7f9193fff640 0 client.16710 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:51:30.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.417+0000 7f91bcff4740 1 --2- 172.21.15.164:0/3276428643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555832eb8380 0x555832ecfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.417+0000 7f91bcff4740 1 --2- 172.21.15.164:0/3276428643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555832eb7a60 0x555832ed3a00 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.417+0000 7f91bcff4740 1 -- 172.21.15.164:0/3276428643 >> 172.21.15.164:0/3276428643 conn(0x555832eb40f0 msgr2=0x555832ec8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:30.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.417+0000 7f91bcff4740 1 -- 172.21.15.164:0/3276428643 shutdown_connections 2024-09-19T08:51:30.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.417+0000 7f91bcff4740 1 -- 172.21.15.164:0/3276428643 wait complete. 2024-09-19T08:51:30.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.417+0000 7f91bcff4740 20 client.16710 trim_cache size 0 max 16384 2024-09-19T08:51:30.419 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:51:30.419 INFO:teuthology.orchestra.run.smithi164.stdout: "return_code": 0, 2024-09-19T08:51:30.419 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_tag": "f7b65f40-3553-424c-b61a-eb6e4f96c377", 2024-09-19T08:51:30.419 INFO:teuthology.orchestra.run.smithi164.stdout: "mode": "asynchronous" 2024-09-19T08:51:30.419 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:51:30.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.418+0000 7f91bac1b640 1 -- 172.21.15.164:0/226933976 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f91a40a3430 msgr2=0x7f91a40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:30.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.418+0000 7f91bac1b640 1 --2- 172.21.15.164:0/226933976 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f91a40a3430 0x7f91a40a58f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f91ac005fd0 tx=0x7f91ac004a40 comp rx=0 tx=0).stop 2024-09-19T08:51:30.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.418+0000 7f91bac1b640 1 -- 172.21.15.164:0/226933976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91b41036e0 msgr2=0x7f91b41968f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:30.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.418+0000 7f91bac1b640 1 --2- 172.21.15.164:0/226933976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91b41036e0 0x7f91b41968f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f91a0012bc0 tx=0x7f91a0010490 comp rx=0 tx=0).stop 2024-09-19T08:51:30.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.418+0000 7f91bac1b640 1 -- 172.21.15.164:0/226933976 shutdown_connections 2024-09-19T08:51:30.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.418+0000 7f91bac1b640 1 --2- 172.21.15.164:0/226933976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91b41082a0 0x7f91b419ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.418+0000 7f91bac1b640 1 --2- 172.21.15.164:0/226933976 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f91a40a3430 0x7f91a40a58f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.418+0000 7f91bac1b640 1 --2- 172.21.15.164:0/226933976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91b41036e0 0x7f91b41968f0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.418+0000 7f91bac1b640 1 --2- 172.21.15.164:0/226933976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91b4102d30 0x7f91b41963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:30.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.418+0000 7f91bac1b640 1 -- 172.21.15.164:0/226933976 >> 172.21.15.164:0/226933976 conn(0x7f91b40fc7d0 msgr2=0x7f91b4105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:30.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.418+0000 7f91bac1b640 1 -- 172.21.15.164:0/226933976 shutdown_connections 2024-09-19T08:51:30.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:30.418+0000 7f91bac1b640 1 -- 172.21.15.164:0/226933976 wait complete. 2024-09-19T08:51:31.504 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:51:32.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:31 smithi164 ceph-mon[30928]: pgmap v174: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 90 MiB/s wr, 228 op/s 2024-09-19T08:51:32.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:31 smithi164 ceph-mon[30928]: scrub queued for path: / 2024-09-19T08:51:32.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:31 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:51:32.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:31 smithi164 ceph-mon[30928]: scrub summary: active paths [/] 2024-09-19T08:51:32.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:31 smithi164 ceph-mon[30928]: scrub summary: active paths [/] 2024-09-19T08:51:32.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:31 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:51:32.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:31 smithi164 ceph-mon[30928]: scrub summary: active paths [/] 2024-09-19T08:51:32.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:31 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:51:32.034 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:31 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:51:32.121 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:51:32.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:31 smithi170 ceph-mon[35072]: pgmap v174: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 90 MiB/s wr, 228 op/s 2024-09-19T08:51:32.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:31 smithi170 ceph-mon[35072]: scrub queued for path: / 2024-09-19T08:51:32.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:31 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:51:32.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:31 smithi170 ceph-mon[35072]: scrub summary: active paths [/] 2024-09-19T08:51:32.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:31 smithi170 ceph-mon[35072]: scrub summary: active paths [/] 2024-09-19T08:51:32.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:31 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:51:32.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:31 smithi170 ceph-mon[35072]: scrub summary: active paths [/] 2024-09-19T08:51:32.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:31 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:51:32.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:31 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:51:32.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:31 smithi173 ceph-mon[35306]: pgmap v174: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 90 MiB/s wr, 228 op/s 2024-09-19T08:51:32.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:31 smithi173 ceph-mon[35306]: scrub queued for path: / 2024-09-19T08:51:32.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:31 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:51:32.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:31 smithi173 ceph-mon[35306]: scrub summary: active paths [/] 2024-09-19T08:51:32.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:31 smithi173 ceph-mon[35306]: scrub summary: active paths [/] 2024-09-19T08:51:32.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:31 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:51:32.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:31 smithi173 ceph-mon[35306]: scrub summary: active paths [/] 2024-09-19T08:51:32.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:31 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:51:32.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:31 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:51:34.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:33 smithi170 ceph-mon[35072]: pgmap v175: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 112 MiB/s wr, 450 op/s 2024-09-19T08:51:34.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:33 smithi173 ceph-mon[35306]: pgmap v175: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 112 MiB/s wr, 450 op/s 2024-09-19T08:51:34.298 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:33 smithi164 ceph-mon[30928]: pgmap v175: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 112 MiB/s wr, 450 op/s 2024-09-19T08:51:36.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:35 smithi170 ceph-mon[35072]: pgmap v176: 97 pgs: 97 active+clean; 41 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 171 MiB/s wr, 991 op/s 2024-09-19T08:51:36.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:35 smithi173 ceph-mon[35306]: pgmap v176: 97 pgs: 97 active+clean; 41 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 171 MiB/s wr, 991 op/s 2024-09-19T08:51:36.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:35 smithi164 ceph-mon[30928]: pgmap v176: 97 pgs: 97 active+clean; 41 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 171 MiB/s wr, 991 op/s 2024-09-19T08:51:37.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.926+0000 7faf33baa640 1 -- 172.21.15.164:0/1395825500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf2c1008c0 msgr2=0x7faf2c100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:37.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.926+0000 7faf33baa640 1 --2- 172.21.15.164:0/1395825500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf2c1008c0 0x7faf2c100ca0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7faf28009900 tx=0x7faf2802a410 comp rx=0 tx=0).stop 2024-09-19T08:51:37.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.926+0000 7faf33baa640 1 -- 172.21.15.164:0/1395825500 shutdown_connections 2024-09-19T08:51:37.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.926+0000 7faf33baa640 1 --2- 172.21.15.164:0/1395825500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf2c10e470 0x7faf2c110860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:37.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.926+0000 7faf33baa640 1 --2- 172.21.15.164:0/1395825500 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf2c101270 0x7faf2c10df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:37.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.926+0000 7faf33baa640 1 --2- 172.21.15.164:0/1395825500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf2c1008c0 0x7faf2c100ca0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:37.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.926+0000 7faf33baa640 1 -- 172.21.15.164:0/1395825500 >> 172.21.15.164:0/1395825500 conn(0x7faf2c0fc7d0 msgr2=0x7faf2c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:37.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.926+0000 7faf33baa640 1 -- 172.21.15.164:0/1395825500 shutdown_connections 2024-09-19T08:51:37.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.926+0000 7faf33baa640 1 -- 172.21.15.164:0/1395825500 wait complete. 2024-09-19T08:51:37.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.927+0000 7faf33baa640 1 Processor -- start 2024-09-19T08:51:37.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.927+0000 7faf33baa640 1 -- start start 2024-09-19T08:51:37.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.928+0000 7faf33baa640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf2c1008c0 0x7faf2c1a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:37.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.928+0000 7faf33baa640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf2c101270 0x7faf2c1a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:37.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.928+0000 7faf33baa640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf2c10e470 0x7faf2c19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:37.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.928+0000 7faf33baa640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c114250 con 0x7faf2c10e470 2024-09-19T08:51:37.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.928+0000 7faf33baa640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c1140d0 con 0x7faf2c101270 2024-09-19T08:51:37.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.928+0000 7faf33baa640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c1143d0 con 0x7faf2c1008c0 2024-09-19T08:51:37.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.928+0000 7faf32147640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf2c10e470 0x7faf2c19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:37.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.928+0000 7faf32147640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf2c10e470 0x7faf2c19a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47514/0 (socket says 172.21.15.164:47514) 2024-09-19T08:51:37.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.928+0000 7faf32147640 1 -- 172.21.15.164:0/738487006 learned_addr learned my addr 172.21.15.164:0/738487006 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:37.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.928+0000 7faf31145640 1 --2- 172.21.15.164:0/738487006 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf2c101270 0x7faf2c1a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:37.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.929+0000 7faf32147640 1 -- 172.21.15.164:0/738487006 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf2c1008c0 msgr2=0x7faf2c1a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:37.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.929+0000 7faf32147640 1 --2- 172.21.15.164:0/738487006 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf2c1008c0 0x7faf2c1a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:37.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.929+0000 7faf32147640 1 -- 172.21.15.164:0/738487006 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf2c101270 msgr2=0x7faf2c1a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:37.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.929+0000 7faf32147640 1 --2- 172.21.15.164:0/738487006 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf2c101270 0x7faf2c1a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:37.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.929+0000 7faf32147640 1 -- 172.21.15.164:0/738487006 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf2c19aed0 con 0x7faf2c10e470 2024-09-19T08:51:37.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.929+0000 7faf32147640 1 --2- 172.21.15.164:0/738487006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf2c10e470 0x7faf2c19a660 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7faf2400b930 tx=0x7faf2400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:37.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.929+0000 7faf22ffd640 1 -- 172.21.15.164:0/738487006 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf24019070 con 0x7faf2c10e470 2024-09-19T08:51:37.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.929+0000 7faf22ffd640 1 -- 172.21.15.164:0/738487006 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf24014440 con 0x7faf2c10e470 2024-09-19T08:51:37.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.929+0000 7faf22ffd640 1 -- 172.21.15.164:0/738487006 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf24012580 con 0x7faf2c10e470 2024-09-19T08:51:37.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.929+0000 7faf33baa640 1 -- 172.21.15.164:0/738487006 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf2c19b1c0 con 0x7faf2c10e470 2024-09-19T08:51:37.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.930+0000 7faf33baa640 1 -- 172.21.15.164:0/738487006 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf2c1a7340 con 0x7faf2c10e470 2024-09-19T08:51:37.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.933+0000 7faf35f83740 1 Processor -- start 2024-09-19T08:51:37.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.934+0000 7faf22ffd640 1 -- 172.21.15.164:0/738487006 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7faf2401c070 con 0x7faf2c10e470 2024-09-19T08:51:37.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.934+0000 7faf35f83740 1 -- start start 2024-09-19T08:51:37.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.934+0000 7faf35f83740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c06d8a54a0 0x55c06d8a5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:37.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.934+0000 7faf35f83740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c06d8a5dc0 0x55c06d8b0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:37.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.934+0000 7faf35f83740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c06d8b0680 0x55c06d8b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:37.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.934+0000 7faf35f83740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c06d80b2e0 con 0x55c06d8a5dc0 2024-09-19T08:51:37.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.934+0000 7faf35f83740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c06d7b2ce0 con 0x55c06d8b0680 2024-09-19T08:51:37.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.934+0000 7faf35f83740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c06d8072a0 con 0x55c06d8a54a0 2024-09-19T08:51:37.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.934+0000 7faf31145640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c06d8a54a0 0x55c06d8a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:37.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.935+0000 7faf31145640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c06d8a54a0 0x55c06d8a5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:34870/0 (socket says 172.21.15.164:34870) 2024-09-19T08:51:37.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.935+0000 7faf31145640 1 -- 172.21.15.164:0/2762901415 learned_addr learned my addr 172.21.15.164:0/2762901415 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:37.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.935+0000 7faf31946640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c06d8a5dc0 0x55c06d8b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:37.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.935+0000 7faf31145640 1 --2- 172.21.15.164:0/2762901415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c06d8b0680 0x55c06d8b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:37.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.935+0000 7faf31946640 1 -- 172.21.15.164:0/2762901415 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c06d8a54a0 msgr2=0x55c06d8a5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:37.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.935+0000 7faf31946640 1 --2- 172.21.15.164:0/2762901415 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c06d8a54a0 0x55c06d8a5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:37.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.935+0000 7faf31946640 1 -- 172.21.15.164:0/2762901415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c06d8b0680 msgr2=0x55c06d8b2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:37.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.935+0000 7faf31946640 1 --2- 172.21.15.164:0/2762901415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c06d8b0680 0x55c06d8b2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:37.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.935+0000 7faf31946640 1 -- 172.21.15.164:0/2762901415 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c06d797440 con 0x55c06d8a5dc0 2024-09-19T08:51:37.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.935+0000 7faf22ffd640 1 --2- 172.21.15.164:0/738487006 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7faf180a3430 0x7faf180a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:37.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.935+0000 7faf31145640 1 --2- 172.21.15.164:0/2762901415 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c06d8a54a0 0x55c06d8a5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:51:37.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.935+0000 7faf31946640 1 --2- 172.21.15.164:0/2762901415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c06d8a5dc0 0x55c06d8b0140 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7faf28009900 tx=0x7faf28005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f896baf901021c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:37.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.936+0000 7faf22ffd640 1 -- 172.21.15.164:0/738487006 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7faf240cf1e0 con 0x7faf2c10e470 2024-09-19T08:51:37.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.936+0000 7faf31145640 1 --2- 172.21.15.164:0/738487006 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7faf180a3430 0x7faf180a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:37.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.936+0000 7faf17fff640 1 -- 172.21.15.164:0/2762901415 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf28040070 con 0x55c06d8a5dc0 2024-09-19T08:51:37.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.936+0000 7faf17fff640 1 -- 172.21.15.164:0/2762901415 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf28004040 con 0x55c06d8a5dc0 2024-09-19T08:51:37.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.936+0000 7faf17fff640 1 -- 172.21.15.164:0/2762901415 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf28035040 con 0x55c06d8a5dc0 2024-09-19T08:51:37.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.937+0000 7faf31145640 1 --2- 172.21.15.164:0/738487006 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7faf180a3430 0x7faf180a58f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x55c06d8a2db0 tx=0x7faf1c008040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:37.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.937+0000 7faf35f83740 1 -- 172.21.15.164:0/2762901415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c06d8a5dc0 msgr2=0x55c06d8b0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:37.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.937+0000 7faf35f83740 1 --2- 172.21.15.164:0/2762901415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c06d8a5dc0 0x55c06d8b0140 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7faf28009900 tx=0x7faf28005d90 comp rx=0 tx=0).stop 2024-09-19T08:51:37.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.937+0000 7faf35f83740 1 -- 172.21.15.164:0/2762901415 shutdown_connections 2024-09-19T08:51:37.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.937+0000 7faf35f83740 1 --2- 172.21.15.164:0/2762901415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c06d8b0680 0x55c06d8b2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:37.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.937+0000 7faf35f83740 1 --2- 172.21.15.164:0/2762901415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c06d8a5dc0 0x55c06d8b0140 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:37.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.937+0000 7faf35f83740 1 --2- 172.21.15.164:0/2762901415 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c06d8a54a0 0x55c06d8a5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:37.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.938+0000 7faf35f83740 1 -- 172.21.15.164:0/2762901415 >> 172.21.15.164:0/2762901415 conn(0x55c06d8a1a30 msgr2=0x55c06d8a1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:37.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.938+0000 7faf35f83740 1 -- 172.21.15.164:0/2762901415 shutdown_connections 2024-09-19T08:51:37.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.938+0000 7faf35f83740 1 -- 172.21.15.164:0/2762901415 wait complete. 2024-09-19T08:51:37.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.939+0000 7faf35f83740 1 -- start start 2024-09-19T08:51:37.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.939+0000 7faf35f83740 1 Processor -- start 2024-09-19T08:51:37.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.940+0000 7faf35f83740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c06d8a54a0 0x55c06d8aff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:37.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.940+0000 7faf35f83740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c06d8a5dc0 0x55c06d8ad1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:37.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.940+0000 7faf35f83740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c06d8b0680 0x55c06d8ad6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:37.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.940+0000 7faf35f83740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c06d8b3250 con 0x55c06d8a5dc0 2024-09-19T08:51:37.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.940+0000 7faf35f83740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c06d7c6050 con 0x55c06d8a54a0 2024-09-19T08:51:37.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.940+0000 7faf31145640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c06d8a5dc0 0x55c06d8ad1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:37.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.940+0000 7faf35f83740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c06d8b33d0 con 0x55c06d8b0680 2024-09-19T08:51:37.733 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:51:37.733 INFO:teuthology.orchestra.run.smithi164.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T08:51:37.733 INFO:teuthology.orchestra.run.smithi164.stdout: "scrubs": { 2024-09-19T08:51:37.733 INFO:teuthology.orchestra.run.smithi164.stdout: "f7b65f40-3553-424c-b61a-eb6e4f96c377": { 2024-09-19T08:51:37.734 INFO:teuthology.orchestra.run.smithi164.stdout: "path": "/", 2024-09-19T08:51:37.734 INFO:teuthology.orchestra.run.smithi164.stdout: "tag": "f7b65f40-3553-424c-b61a-eb6e4f96c377", 2024-09-19T08:51:37.734 INFO:teuthology.orchestra.run.smithi164.stdout: "options": "recursive,force" 2024-09-19T08:51:37.734 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:51:37.734 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:51:37.734 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:51:37.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.940+0000 7faf31145640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c06d8a5dc0 0x55c06d8ad1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47530/0 (socket says 172.21.15.164:47530) 2024-09-19T08:51:37.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.940+0000 7faf31145640 1 -- 172.21.15.164:0/1159873920 learned_addr learned my addr 172.21.15.164:0/1159873920 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:37.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.940+0000 7faf35f83740 10 client.0 _ll_register_callbacks cb 0x55c06d806160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:51:37.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.941+0000 7faf31145640 1 -- 172.21.15.164:0/1159873920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c06d8b0680 msgr2=0x55c06d8ad6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:51:37.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.941+0000 7faf31145640 1 --2- 172.21.15.164:0/1159873920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c06d8b0680 0x55c06d8ad6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:37.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.941+0000 7faf31145640 1 -- 172.21.15.164:0/1159873920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c06d8a54a0 msgr2=0x55c06d8aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:37.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.941+0000 7faf31145640 1 --2- 172.21.15.164:0/1159873920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c06d8a54a0 0x55c06d8aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:37.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.941+0000 7faf31145640 1 -- 172.21.15.164:0/1159873920 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c06d57c3c0 con 0x55c06d8a5dc0 2024-09-19T08:51:37.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.941+0000 7faf31145640 1 --2- 172.21.15.164:0/1159873920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c06d8a5dc0 0x55c06d8ad1b0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7faf1c016760 tx=0x7faf1c009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dabc259181a6adf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:37.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.941+0000 7faf157fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:51:37.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.942+0000 7faf157fa640 1 -- 172.21.15.164:0/1159873920 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1c027020 con 0x55c06d8a5dc0 2024-09-19T08:51:37.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.942+0000 7faf35f83740 1 -- 172.21.15.164:0/1159873920 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf28004040 con 0x55c06d8a5dc0 2024-09-19T08:51:37.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.942+0000 7faf157fa640 1 -- 172.21.15.164:0/1159873920 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7faf1c0155b0 con 0x55c06d8a5dc0 2024-09-19T08:51:37.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.942+0000 7faf157fa640 1 -- 172.21.15.164:0/1159873920 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faf1c0059c0 con 0x55c06d8a5dc0 2024-09-19T08:51:37.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.942+0000 7faf35f83740 10 client.16722 fetch_fsmap learned FSMap version 30 2024-09-19T08:51:37.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.942+0000 7faf35f83740 1 -- 172.21.15.164:0/1159873920 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c06d8bec40 con 0x55c06d8a5dc0 2024-09-19T08:51:37.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.942+0000 7faf157fa640 1 -- 172.21.15.164:0/1159873920 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf28004040 con 0x55c06d8a5dc0 2024-09-19T08:51:37.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.943+0000 7faf157fa640 1 -- 172.21.15.164:0/1159873920 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7faf1c006270 con 0x55c06d8a5dc0 2024-09-19T08:51:37.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.943+0000 7faf35f83740 10 client.16722 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:51:37.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.943+0000 7faf35f83740 10 client.16722 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:51:37.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.943+0000 7faf35f83740 20 client.16722 populate_metadata read hostname 'smithi164' 2024-09-19T08:51:37.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.943+0000 7faf35f83740 1 --2- 172.21.15.164:0/1159873920 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c06d8c01a0 0x55c06d8c2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:37.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.943+0000 7faf35f83740 4 client.16722 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:51:37.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.943+0000 7faf35f83740 1 -- 172.21.15.164:0/1159873920 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c06d818850 con 0x55c06d8c01a0 2024-09-19T08:51:37.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.947+0000 7faf31946640 1 --2- 172.21.15.164:0/1159873920 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c06d8c01a0 0x55c06d8c2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:37.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.949+0000 7faf31946640 1 --2- 172.21.15.164:0/1159873920 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c06d8c01a0 0x55c06d8c2630 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0227f84113d309c server_cookie=6b026c892c417f8 in_seq=0 out_seq=0 2024-09-19T08:51:37.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.949+0000 7faf157fa640 10 client.16722 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:51:37.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.957+0000 7faf157fa640 1 -- 172.21.15.164:0/1159873920 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c06d818850 con 0x55c06d8c01a0 2024-09-19T08:51:37.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.958+0000 7faf157fa640 10 client.16722 handle_command_reply: tid=0 2024-09-19T08:51:37.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.958+0000 7faf35f83740 1 client.16722 shutdown 2024-09-19T08:51:37.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.958+0000 7faf35f83740 1 -- 172.21.15.164:0/1159873920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c06d8a5dc0 msgr2=0x55c06d8ad1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:37.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.958+0000 7faf35f83740 1 --2- 172.21.15.164:0/1159873920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c06d8a5dc0 0x55c06d8ad1b0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7faf1c016760 tx=0x7faf1c009110 comp rx=0 tx=0).stop 2024-09-19T08:51:37.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.958+0000 7faf35f83740 1 -- 172.21.15.164:0/1159873920 shutdown_connections 2024-09-19T08:51:37.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.958+0000 7faf35f83740 1 --2- 172.21.15.164:0/1159873920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c06d8b0680 0x55c06d8ad6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:37.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.958+0000 7faf35f83740 1 --2- 172.21.15.164:0/1159873920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c06d8a5dc0 0x55c06d8ad1b0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:37.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.958+0000 7faf35f83740 1 --2- 172.21.15.164:0/1159873920 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c06d8c01a0 0x55c06d8c2630 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:37.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.958+0000 7faf35f83740 1 --2- 172.21.15.164:0/1159873920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c06d8a54a0 0x55c06d8aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:37.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.958+0000 7faf157fa640 0 client.16722 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:51:37.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.958+0000 7faf35f83740 1 -- 172.21.15.164:0/1159873920 >> 172.21.15.164:0/1159873920 conn(0x55c06d8a1a30 msgr2=0x55c06d8a7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:37.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.958+0000 7faf35f83740 1 -- 172.21.15.164:0/1159873920 shutdown_connections 2024-09-19T08:51:37.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.958+0000 7faf35f83740 1 -- 172.21.15.164:0/1159873920 wait complete. 2024-09-19T08:51:37.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.959+0000 7faf35f83740 20 client.16722 trim_cache size 0 max 16384 2024-09-19T08:51:37.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.987+0000 7faf35f83740 1 Processor -- start 2024-09-19T08:51:37.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.987+0000 7faf35f83740 1 -- start start 2024-09-19T08:51:37.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.987+0000 7faf35f83740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c06d8a1a30 0x55c06d8bc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:37.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.988+0000 7faf35f83740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c06d8a54a0 0x55c06d8aba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:37.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.988+0000 7faf35f83740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c06d8a5dc0 0x55c06d8af120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:37.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.988+0000 7faf35f83740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c1af9e0 con 0x55c06d8a1a30 2024-09-19T08:51:37.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.988+0000 7faf35f83740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c1b15e0 con 0x55c06d8a5dc0 2024-09-19T08:51:37.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.988+0000 7faf35f83740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c1b00b0 con 0x55c06d8a54a0 2024-09-19T08:51:37.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.988+0000 7faf32147640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c06d8a5dc0 0x55c06d8af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:37.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.988+0000 7faf31145640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c06d8a54a0 0x55c06d8aba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:37.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.988+0000 7faf32147640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c06d8a5dc0 0x55c06d8af120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:56668/0 (socket says 172.21.15.164:56668) 2024-09-19T08:51:37.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.988+0000 7faf32147640 1 -- 172.21.15.164:0/3111149081 learned_addr learned my addr 172.21.15.164:0/3111149081 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:37.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.988+0000 7faf31946640 1 --2- 172.21.15.164:0/3111149081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c06d8a1a30 0x55c06d8bc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:37.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.989+0000 7faf31145640 1 -- 172.21.15.164:0/3111149081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c06d8a5dc0 msgr2=0x55c06d8af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:37.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.989+0000 7faf31145640 1 --2- 172.21.15.164:0/3111149081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c06d8a5dc0 0x55c06d8af120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:37.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.989+0000 7faf31145640 1 -- 172.21.15.164:0/3111149081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c06d8a1a30 msgr2=0x55c06d8bc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:37.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.989+0000 7faf31145640 1 --2- 172.21.15.164:0/3111149081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c06d8a1a30 0x55c06d8bc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:37.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.989+0000 7faf31145640 1 -- 172.21.15.164:0/3111149081 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c06d8ac160 con 0x55c06d8a54a0 2024-09-19T08:51:37.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.989+0000 7faf32147640 1 --2- 172.21.15.164:0/3111149081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c06d8a5dc0 0x55c06d8af120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:51:37.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.989+0000 7faf31145640 1 --2- 172.21.15.164:0/3111149081 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c06d8a54a0 0x55c06d8aba80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faf1c008010 tx=0x7faf1c0069d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1cc7ee9ea11e5be server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:37.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.992+0000 7faf177fe640 1 -- 172.21.15.164:0/3111149081 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1c03d070 con 0x55c06d8a54a0 2024-09-19T08:51:37.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.993+0000 7faf177fe640 1 -- 172.21.15.164:0/3111149081 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf1c01c030 con 0x55c06d8a54a0 2024-09-19T08:51:37.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.993+0000 7faf177fe640 1 -- 172.21.15.164:0/3111149081 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1c027040 con 0x55c06d8a54a0 2024-09-19T08:51:37.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.995+0000 7faf35f83740 1 -- 172.21.15.164:0/3111149081 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c06d8a54a0 msgr2=0x55c06d8aba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:37.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.995+0000 7faf35f83740 1 --2- 172.21.15.164:0/3111149081 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c06d8a54a0 0x55c06d8aba80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faf1c008010 tx=0x7faf1c0069d0 comp rx=0 tx=0).stop 2024-09-19T08:51:37.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.996+0000 7faf35f83740 1 -- 172.21.15.164:0/3111149081 shutdown_connections 2024-09-19T08:51:37.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.996+0000 7faf35f83740 1 --2- 172.21.15.164:0/3111149081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c06d8a5dc0 0x55c06d8af120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:37.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.996+0000 7faf35f83740 1 --2- 172.21.15.164:0/3111149081 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c06d8a54a0 0x55c06d8aba80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:37.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.996+0000 7faf35f83740 1 --2- 172.21.15.164:0/3111149081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c06d8a1a30 0x55c06d8bc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:37.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.996+0000 7faf35f83740 1 -- 172.21.15.164:0/3111149081 >> 172.21.15.164:0/3111149081 conn(0x55c06d8ad850 msgr2=0x55c06d8aea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:37.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.997+0000 7faf35f83740 1 -- 172.21.15.164:0/3111149081 shutdown_connections 2024-09-19T08:51:37.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.997+0000 7faf35f83740 1 -- 172.21.15.164:0/3111149081 wait complete. 2024-09-19T08:51:37.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.998+0000 7faf35f83740 1 -- start start 2024-09-19T08:51:37.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.998+0000 7faf35f83740 1 Processor -- start 2024-09-19T08:51:37.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:36.999+0000 7faf35f83740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c06d8a1a30 0x55c06d8b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:37.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.000+0000 7faf35f83740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c06d8a54a0 0x55c06d8b70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:37.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.000+0000 7faf35f83740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c06d8a5dc0 0x55c06d93af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:37.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.000+0000 7faf31145640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c06d8a54a0 0x55c06d8b70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:37.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.000+0000 7faf31145640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c06d8a54a0 0x55c06d8b70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:56678/0 (socket says 172.21.15.164:56678) 2024-09-19T08:51:37.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.000+0000 7faf31946640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c06d8a1a30 0x55c06d8b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:37.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.000+0000 7faf31946640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c06d8a1a30 0x55c06d8b6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:34896/0 (socket says 172.21.15.164:34896) 2024-09-19T08:51:37.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.000+0000 7faf31145640 1 -- 172.21.15.164:0/3652749328 learned_addr learned my addr 172.21.15.164:0/3652749328 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:51:37.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.001+0000 7faf35f83740 1 -- 172.21.15.164:0/3652749328 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c1b0e50 con 0x55c06d8a5dc0 2024-09-19T08:51:37.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.001+0000 7faf35f83740 1 -- 172.21.15.164:0/3652749328 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c1b0780 con 0x55c06d8a54a0 2024-09-19T08:51:37.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.001+0000 7faf35f83740 1 -- 172.21.15.164:0/3652749328 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c1ad9c0 con 0x55c06d8a1a30 2024-09-19T08:51:37.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.001+0000 7faf35f83740 10 client.0 _ll_register_callbacks cb 0x7faf1c009410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:51:37.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.002+0000 7faf32147640 1 --2- 172.21.15.164:0/3652749328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c06d8a5dc0 0x55c06d93af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:37.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.002+0000 7faf31145640 1 -- 172.21.15.164:0/3652749328 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c06d8a1a30 msgr2=0x55c06d8b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:37.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.002+0000 7faf31145640 1 --2- 172.21.15.164:0/3652749328 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c06d8a1a30 0x55c06d8b6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:37.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.002+0000 7faf31145640 1 -- 172.21.15.164:0/3652749328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c06d8a5dc0 msgr2=0x55c06d93af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:51:37.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.002+0000 7faf31145640 1 --2- 172.21.15.164:0/3652749328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c06d8a5dc0 0x55c06d93af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:37.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.002+0000 7faf31145640 1 -- 172.21.15.164:0/3652749328 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c06d93b4a0 con 0x55c06d8a54a0 2024-09-19T08:51:37.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.003+0000 7faf31145640 1 --2- 172.21.15.164:0/3652749328 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c06d8a54a0 0x55c06d8b70b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7faf1c006850 tx=0x7faf1c015e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ae58400b8e60f20 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:51:37.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.003+0000 7faf35f83740 1 -- 172.21.15.164:0/3652749328 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf1c01c030 con 0x55c06d8a54a0 2024-09-19T08:51:37.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.003+0000 7faf157fa640 10 client.26051 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-19T08:51:37.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.003+0000 7faf157fa640 1 -- 172.21.15.164:0/3652749328 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1c03d070 con 0x55c06d8a54a0 2024-09-19T08:51:37.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.004+0000 7faf157fa640 1 -- 172.21.15.164:0/3652749328 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8411+0+0 (secure 0 0 0) 0x7faf1c005d70 con 0x55c06d8a54a0 2024-09-19T08:51:37.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.004+0000 7faf157fa640 1 -- 172.21.15.164:0/3652749328 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faf1c027040 con 0x55c06d8a54a0 2024-09-19T08:51:37.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.004+0000 7faf35f83740 10 client.26051 fetch_fsmap learned FSMap version 30 2024-09-19T08:51:37.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.005+0000 7faf35f83740 1 -- 172.21.15.164:0/3652749328 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55c06d93c630 con 0x55c06d8a54a0 2024-09-19T08:51:37.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.008+0000 7faf157fa640 1 -- 172.21.15.164:0/3652749328 <== mon.2 v2:172.21.15.170:3300/0 4 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7faf1c005fb0 con 0x55c06d8a54a0 2024-09-19T08:51:37.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.008+0000 7faf35f83740 10 client.26051 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:51:37.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.008+0000 7faf35f83740 10 client.26051 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:51:37.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.008+0000 7faf35f83740 20 client.26051 populate_metadata read hostname 'smithi164' 2024-09-19T08:51:37.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.008+0000 7faf35f83740 1 --2- 172.21.15.164:0/3652749328 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c06d8b0680 0x55c06d93ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:51:37.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.008+0000 7faf35f83740 4 client.26051 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:51:37.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.008+0000 7faf35f83740 1 -- 172.21.15.164:0/3652749328 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7faf1c01c030 con 0x55c06d8b0680 2024-09-19T08:51:37.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.013+0000 7faf31946640 1 --2- 172.21.15.164:0/3652749328 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c06d8b0680 0x55c06d93ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:51:37.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.026+0000 7faf31946640 1 --2- 172.21.15.164:0/3652749328 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c06d8b0680 0x55c06d93ed10 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22babe1499401659 server_cookie=e41ca2501759973f in_seq=0 out_seq=0 2024-09-19T08:51:37.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.026+0000 7faf157fa640 10 client.26051 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:51:37.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.027+0000 7faf157fa640 1 -- 172.21.15.164:0/3652749328 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7faf1c01c030 con 0x55c06d8b0680 2024-09-19T08:51:37.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.028+0000 7faf157fa640 10 client.26051 handle_command_reply: tid=0 2024-09-19T08:51:37.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.028+0000 7faf35f83740 1 client.26051 shutdown 2024-09-19T08:51:37.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.028+0000 7faf35f83740 1 -- 172.21.15.164:0/3652749328 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c06d8a54a0 msgr2=0x55c06d8b70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:37.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.028+0000 7faf35f83740 1 --2- 172.21.15.164:0/3652749328 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c06d8a54a0 0x55c06d8b70b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7faf1c006850 tx=0x7faf1c015e10 comp rx=0 tx=0).stop 2024-09-19T08:51:37.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.029+0000 7faf35f83740 1 -- 172.21.15.164:0/3652749328 shutdown_connections 2024-09-19T08:51:37.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.029+0000 7faf35f83740 1 --2- 172.21.15.164:0/3652749328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c06d8a5dc0 0x55c06d93af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:37.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.029+0000 7faf35f83740 1 --2- 172.21.15.164:0/3652749328 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c06d8b0680 0x55c06d93ed10 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:37.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.029+0000 7faf35f83740 1 --2- 172.21.15.164:0/3652749328 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c06d8a54a0 0x55c06d8b70b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:37.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.029+0000 7faf35f83740 1 --2- 172.21.15.164:0/3652749328 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c06d8a1a30 0x55c06d8b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:37.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.029+0000 7faf35f83740 1 -- 172.21.15.164:0/3652749328 >> 172.21.15.164:0/3652749328 conn(0x55c06d8ad850 msgr2=0x55c06d8aea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:37.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.029+0000 7faf157fa640 0 client.26051 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:51:37.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.035+0000 7faf35f83740 1 -- 172.21.15.164:0/3652749328 shutdown_connections 2024-09-19T08:51:37.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.039+0000 7faf35f83740 1 -- 172.21.15.164:0/3652749328 wait complete. 2024-09-19T08:51:37.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.040+0000 7faf35f83740 20 client.26051 trim_cache size 0 max 16384 2024-09-19T08:51:37.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.047+0000 7faf33baa640 1 -- 172.21.15.164:0/738487006 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7faf180a3430 msgr2=0x7faf180a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:37.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.047+0000 7faf33baa640 1 --2- 172.21.15.164:0/738487006 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7faf180a3430 0x7faf180a58f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x55c06d8a2db0 tx=0x7faf1c008040 comp rx=0 tx=0).stop 2024-09-19T08:51:37.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.047+0000 7faf33baa640 1 -- 172.21.15.164:0/738487006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf2c10e470 msgr2=0x7faf2c19a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:51:37.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.047+0000 7faf33baa640 1 --2- 172.21.15.164:0/738487006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf2c10e470 0x7faf2c19a660 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7faf2400b930 tx=0x7faf2400bd30 comp rx=0 tx=0).stop 2024-09-19T08:51:37.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.047+0000 7faf33baa640 1 -- 172.21.15.164:0/738487006 shutdown_connections 2024-09-19T08:51:37.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.047+0000 7faf33baa640 1 --2- 172.21.15.164:0/738487006 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7faf180a3430 0x7faf180a58f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:37.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.047+0000 7faf33baa640 1 --2- 172.21.15.164:0/738487006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf2c10e470 0x7faf2c19a660 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:37.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.047+0000 7faf33baa640 1 --2- 172.21.15.164:0/738487006 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf2c101270 0x7faf2c1a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:37.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.047+0000 7faf33baa640 1 --2- 172.21.15.164:0/738487006 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf2c1008c0 0x7faf2c1a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:51:37.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.047+0000 7faf33baa640 1 -- 172.21.15.164:0/738487006 >> 172.21.15.164:0/738487006 conn(0x7faf2c0fc7d0 msgr2=0x7faf2c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:51:37.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.049+0000 7faf33baa640 1 -- 172.21.15.164:0/738487006 shutdown_connections 2024-09-19T08:51:37.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:51:37.050+0000 7faf33baa640 1 -- 172.21.15.164:0/738487006 wait complete. 2024-09-19T08:51:38.040 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:37 smithi164 ceph-mon[30928]: pgmap v177: 97 pgs: 97 active+clean; 41 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 295 MiB/s wr, 1.99k op/s 2024-09-19T08:51:38.040 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:37 smithi164 ceph-mon[30928]: scrub summary: idle 2024-09-19T08:51:38.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:37 smithi170 ceph-mon[35072]: pgmap v177: 97 pgs: 97 active+clean; 41 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 295 MiB/s wr, 1.99k op/s 2024-09-19T08:51:38.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:37 smithi170 ceph-mon[35072]: scrub summary: idle 2024-09-19T08:51:38.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:37 smithi173 ceph-mon[35306]: pgmap v177: 97 pgs: 97 active+clean; 41 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 295 MiB/s wr, 1.99k op/s 2024-09-19T08:51:38.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:37 smithi173 ceph-mon[35306]: scrub summary: idle 2024-09-19T08:51:39.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:38 smithi164 ceph-mon[30928]: scrub summary: idle 2024-09-19T08:51:39.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:38 smithi170 ceph-mon[35072]: scrub summary: idle 2024-09-19T08:51:39.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:38 smithi173 ceph-mon[35306]: scrub summary: idle 2024-09-19T08:51:39.915 INFO:tasks.cephfs.filesystem:scrub status for tag:f7b65f40-3553-424c-b61a-eb6e4f96c377 - {'path': '/', 'tag': 'f7b65f40-3553-424c-b61a-eb6e4f96c377', 'options': 'recursive,force'} 2024-09-19T08:51:40.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:39 smithi164 ceph-mon[30928]: pgmap v178: 97 pgs: 97 active+clean; 41 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 305 MiB/s wr, 2.11k op/s 2024-09-19T08:51:40.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:39 smithi173 ceph-mon[35306]: pgmap v178: 97 pgs: 97 active+clean; 41 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 305 MiB/s wr, 2.11k op/s 2024-09-19T08:51:40.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:39 smithi170 ceph-mon[35072]: pgmap v178: 97 pgs: 97 active+clean; 41 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 305 MiB/s wr, 2.11k op/s 2024-09-19T08:51:42.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:41 smithi173 ceph-mon[35306]: pgmap v179: 97 pgs: 97 active+clean; 43 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 515 MiB/s wr, 2.96k op/s 2024-09-19T08:51:42.285 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:41 smithi164 ceph-mon[30928]: pgmap v179: 97 pgs: 97 active+clean; 43 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 515 MiB/s wr, 2.96k op/s 2024-09-19T08:51:42.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:41 smithi170 ceph-mon[35072]: pgmap v179: 97 pgs: 97 active+clean; 43 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 515 MiB/s wr, 2.96k op/s 2024-09-19T08:51:44.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:43 smithi173 ceph-mon[35306]: pgmap v180: 97 pgs: 97 active+clean; 43 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 517 MiB/s wr, 2.98k op/s 2024-09-19T08:51:44.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:43 smithi164 ceph-mon[30928]: pgmap v180: 97 pgs: 97 active+clean; 43 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 517 MiB/s wr, 2.98k op/s 2024-09-19T08:51:44.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:43 smithi170 ceph-mon[35072]: pgmap v180: 97 pgs: 97 active+clean; 43 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 517 MiB/s wr, 2.98k op/s 2024-09-19T08:51:46.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:45 smithi173 ceph-mon[35306]: pgmap v181: 97 pgs: 97 active+clean; 45 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 600 MiB/s wr, 3.40k op/s 2024-09-19T08:51:46.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:45 smithi164 ceph-mon[30928]: pgmap v181: 97 pgs: 97 active+clean; 45 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 600 MiB/s wr, 3.40k op/s 2024-09-19T08:51:46.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:45 smithi170 ceph-mon[35072]: pgmap v181: 97 pgs: 97 active+clean; 45 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 600 MiB/s wr, 3.40k op/s 2024-09-19T08:51:48.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:47 smithi173 ceph-mon[35306]: pgmap v182: 97 pgs: 97 active+clean; 47 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 704 MiB/s wr, 3.64k op/s 2024-09-19T08:51:48.176 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:47 smithi170 ceph-mon[35072]: pgmap v182: 97 pgs: 97 active+clean; 47 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 704 MiB/s wr, 3.64k op/s 2024-09-19T08:51:48.285 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:47 smithi164 ceph-mon[30928]: pgmap v182: 97 pgs: 97 active+clean; 47 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 704 MiB/s wr, 3.64k op/s 2024-09-19T08:51:50.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:49 smithi164 ceph-mon[30928]: pgmap v183: 97 pgs: 97 active+clean; 47 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 609 MiB/s wr, 2.68k op/s 2024-09-19T08:51:50.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:49 smithi170 ceph-mon[35072]: pgmap v183: 97 pgs: 97 active+clean; 47 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 609 MiB/s wr, 2.68k op/s 2024-09-19T08:51:50.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:49 smithi173 ceph-mon[35306]: pgmap v183: 97 pgs: 97 active+clean; 47 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 609 MiB/s wr, 2.68k op/s 2024-09-19T08:51:52.292 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:51 smithi164 ceph-mon[30928]: pgmap v184: 97 pgs: 97 active+clean; 48 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 849 MiB/s wr, 2.79k op/s 2024-09-19T08:51:52.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:51 smithi170 ceph-mon[35072]: pgmap v184: 97 pgs: 97 active+clean; 48 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 849 MiB/s wr, 2.79k op/s 2024-09-19T08:51:52.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:51 smithi173 ceph-mon[35306]: pgmap v184: 97 pgs: 97 active+clean; 48 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 849 MiB/s wr, 2.79k op/s 2024-09-19T08:51:54.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:53 smithi164 ceph-mon[30928]: pgmap v185: 97 pgs: 97 active+clean; 48 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 679 MiB/s wr, 2.01k op/s 2024-09-19T08:51:54.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:53 smithi170 ceph-mon[35072]: pgmap v185: 97 pgs: 97 active+clean; 48 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 679 MiB/s wr, 2.01k op/s 2024-09-19T08:51:54.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:53 smithi173 ceph-mon[35306]: pgmap v185: 97 pgs: 97 active+clean; 48 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 679 MiB/s wr, 2.01k op/s 2024-09-19T08:51:56.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:55 smithi164 ceph-mon[30928]: pgmap v186: 97 pgs: 97 active+clean; 49 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 797 MiB/s wr, 2.29k op/s 2024-09-19T08:51:56.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:55 smithi170 ceph-mon[35072]: pgmap v186: 97 pgs: 97 active+clean; 49 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 797 MiB/s wr, 2.29k op/s 2024-09-19T08:51:56.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:55 smithi173 ceph-mon[35306]: pgmap v186: 97 pgs: 97 active+clean; 49 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 797 MiB/s wr, 2.29k op/s 2024-09-19T08:51:58.293 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:51:58 smithi164 ceph-mon[30928]: pgmap v187: 97 pgs: 97 active+clean; 49 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 838 MiB/s wr, 2.11k op/s 2024-09-19T08:51:58.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:51:58 smithi170 ceph-mon[35072]: pgmap v187: 97 pgs: 97 active+clean; 49 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 838 MiB/s wr, 2.11k op/s 2024-09-19T08:51:58.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:51:58 smithi173 ceph-mon[35306]: pgmap v187: 97 pgs: 97 active+clean; 49 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 838 MiB/s wr, 2.11k op/s 2024-09-19T08:52:00.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:00 smithi164 ceph-mon[30928]: pgmap v188: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 712 MiB/s wr, 1.42k op/s 2024-09-19T08:52:00.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:00 smithi170 ceph-mon[35072]: pgmap v188: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 712 MiB/s wr, 1.42k op/s 2024-09-19T08:52:00.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:00 smithi173 ceph-mon[35306]: pgmap v188: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 712 MiB/s wr, 1.42k op/s 2024-09-19T08:52:02.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:02 smithi170 ceph-mon[35072]: pgmap v189: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 932 MiB/s wr, 1.65k op/s 2024-09-19T08:52:02.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:02 smithi173 ceph-mon[35306]: pgmap v189: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 932 MiB/s wr, 1.65k op/s 2024-09-19T08:52:02.500 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:02 smithi164 ceph-mon[30928]: pgmap v189: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 932 MiB/s wr, 1.65k op/s 2024-09-19T08:52:04.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:04 smithi170 ceph-mon[35072]: pgmap v190: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 715 MiB/s wr, 1.51k op/s 2024-09-19T08:52:04.559 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:04 smithi170 ceph-mon[35072]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T08:52:04.559 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:04 smithi170 ceph-mon[35072]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T08:52:04.559 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:04 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:52:04.560 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:04 smithi164 ceph-mon[30928]: pgmap v190: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 715 MiB/s wr, 1.51k op/s 2024-09-19T08:52:04.560 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:04 smithi164 ceph-mon[30928]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T08:52:04.560 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:04 smithi164 ceph-mon[30928]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T08:52:04.560 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:04 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:52:04.561 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:04 smithi173 ceph-mon[35306]: pgmap v190: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 715 MiB/s wr, 1.51k op/s 2024-09-19T08:52:04.561 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:04 smithi173 ceph-mon[35306]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T08:52:04.561 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:04 smithi173 ceph-mon[35306]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T08:52:04.561 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:04 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:52:05.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:05 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:52:05.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:05 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:52:05.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:05 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:52:05.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:05 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:52:05.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:05 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:52:05.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:05 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:52:05.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:05 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:52:05.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:05 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:52:05.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:05 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:52:06.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:06 smithi170 ceph-mon[35072]: pgmap v192: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 815 MiB/s wr, 1.83k op/s 2024-09-19T08:52:06.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:06 smithi173 ceph-mon[35306]: pgmap v192: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 815 MiB/s wr, 1.83k op/s 2024-09-19T08:52:06.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:06 smithi164 ceph-mon[30928]: pgmap v192: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 815 MiB/s wr, 1.83k op/s 2024-09-19T08:52:08.349 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:08 smithi164 ceph-mon[30928]: pgmap v193: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 793 MiB/s wr, 1.90k op/s 2024-09-19T08:52:08.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:08 smithi170 ceph-mon[35072]: pgmap v193: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 793 MiB/s wr, 1.90k op/s 2024-09-19T08:52:08.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:08 smithi173 ceph-mon[35306]: pgmap v193: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 793 MiB/s wr, 1.90k op/s 2024-09-19T08:52:09.916 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:52:10.199 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:52:10.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:10 smithi164 ceph-mon[30928]: pgmap v194: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 754 MiB/s wr, 1.84k op/s 2024-09-19T08:52:10.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:10 smithi164 ceph-mon[30928]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T08:52:10.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:10 smithi170 ceph-mon[35072]: pgmap v194: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 754 MiB/s wr, 1.84k op/s 2024-09-19T08:52:10.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:10 smithi170 ceph-mon[35072]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T08:52:10.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:10 smithi173 ceph-mon[35306]: pgmap v194: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 754 MiB/s wr, 1.84k op/s 2024-09-19T08:52:10.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:10 smithi173 ceph-mon[35306]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T08:52:10.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.688+0000 7fb6960d4640 1 -- 172.21.15.164:0/540765690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb690100a80 msgr2=0x7fb690100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:10.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.688+0000 7fb6960d4640 1 --2- 172.21.15.164:0/540765690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb690100a80 0x7fb690100ee0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb684009930 tx=0x7fb68402a420 comp rx=0 tx=0).stop 2024-09-19T08:52:10.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.688+0000 7fb6960d4640 1 -- 172.21.15.164:0/540765690 shutdown_connections 2024-09-19T08:52:10.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.688+0000 7fb6960d4640 1 --2- 172.21.15.164:0/540765690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb690101420 0x7fb69010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:10.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.688+0000 7fb6960d4640 1 --2- 172.21.15.164:0/540765690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb690100a80 0x7fb690100ee0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:10.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.688+0000 7fb6960d4640 1 --2- 172.21.15.164:0/540765690 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb690106a80 0x7fb690106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:10.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.688+0000 7fb6960d4640 1 -- 172.21.15.164:0/540765690 >> 172.21.15.164:0/540765690 conn(0x7fb6900fc7a0 msgr2=0x7fb6900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:10.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.688+0000 7fb6960d4640 1 -- 172.21.15.164:0/540765690 shutdown_connections 2024-09-19T08:52:10.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.688+0000 7fb6960d4640 1 -- 172.21.15.164:0/540765690 wait complete. 2024-09-19T08:52:10.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.688+0000 7fb6960d4640 1 Processor -- start 2024-09-19T08:52:10.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.688+0000 7fb6960d4640 1 -- start start 2024-09-19T08:52:10.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.689+0000 7fb6960d4640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb690100a80 0x7fb690199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:10.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.689+0000 7fb6960d4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb690101420 0x7fb690199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:10.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.689+0000 7fb6960d4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb690106a80 0x7fb6901a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:10.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.689+0000 7fb6960d4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb690110e80 con 0x7fb690101420 2024-09-19T08:52:10.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.689+0000 7fb6960d4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb690110d00 con 0x7fb690106a80 2024-09-19T08:52:10.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.689+0000 7fb6960d4640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb690111000 con 0x7fb690100a80 2024-09-19T08:52:10.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.689+0000 7fb68effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb690101420 0x7fb690199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:10.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.689+0000 7fb68effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb690101420 0x7fb690199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49210/0 (socket says 172.21.15.164:49210) 2024-09-19T08:52:10.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.689+0000 7fb68effd640 1 -- 172.21.15.164:0/1961208381 learned_addr learned my addr 172.21.15.164:0/1961208381 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:10.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.689+0000 7fb68ffff640 1 --2- 172.21.15.164:0/1961208381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb690106a80 0x7fb6901a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:10.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.689+0000 7fb68f7fe640 1 --2- 172.21.15.164:0/1961208381 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb690100a80 0x7fb690199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:10.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.689+0000 7fb68effd640 1 -- 172.21.15.164:0/1961208381 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb690100a80 msgr2=0x7fb690199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:10.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.689+0000 7fb68effd640 1 --2- 172.21.15.164:0/1961208381 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb690100a80 0x7fb690199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:10.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.689+0000 7fb68effd640 1 -- 172.21.15.164:0/1961208381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb690106a80 msgr2=0x7fb6901a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:10.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.689+0000 7fb68effd640 1 --2- 172.21.15.164:0/1961208381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb690106a80 0x7fb6901a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:10.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.689+0000 7fb68effd640 1 -- 172.21.15.164:0/1961208381 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6901a48d0 con 0x7fb690101420 2024-09-19T08:52:10.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.689+0000 7fb68effd640 1 --2- 172.21.15.164:0/1961208381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb690101420 0x7fb690199850 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb684009230 tx=0x7fb684004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:10.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.689+0000 7fb68cff9640 1 -- 172.21.15.164:0/1961208381 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68403f030 con 0x7fb690101420 2024-09-19T08:52:10.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.689+0000 7fb6960d4640 1 -- 172.21.15.164:0/1961208381 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6901a4aa0 con 0x7fb690101420 2024-09-19T08:52:10.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.689+0000 7fb68cff9640 1 -- 172.21.15.164:0/1961208381 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb684003ad0 con 0x7fb690101420 2024-09-19T08:52:10.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.689+0000 7fb68cff9640 1 -- 172.21.15.164:0/1961208381 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb684035070 con 0x7fb690101420 2024-09-19T08:52:10.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.690+0000 7fb6960d4640 1 -- 172.21.15.164:0/1961208381 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6901a4f50 con 0x7fb690101420 2024-09-19T08:52:10.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.692+0000 7fb68cff9640 1 -- 172.21.15.164:0/1961208381 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fb684042030 con 0x7fb690101420 2024-09-19T08:52:10.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.692+0000 7fb6984ad740 1 Processor -- start 2024-09-19T08:52:10.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.692+0000 7fb6984ad740 1 -- start start 2024-09-19T08:52:10.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.692+0000 7fb6984ad740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555630c2a4a0 0x555630c2a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:10.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.692+0000 7fb6984ad740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555630c2adc0 0x555630c35140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:10.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.692+0000 7fb6984ad740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555630c35680 0x555630c37a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:10.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.692+0000 7fb6984ad740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555630b902e0 con 0x555630c2adc0 2024-09-19T08:52:10.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.692+0000 7fb6984ad740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555630b37ce0 con 0x555630c35680 2024-09-19T08:52:10.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.692+0000 7fb6984ad740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555630b8c2a0 con 0x555630c2a4a0 2024-09-19T08:52:10.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.692+0000 7fb68cff9640 1 --2- 172.21.15.164:0/1961208381 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb6780a33b0 0x7fb6780a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:10.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.692+0000 7fb68f7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555630c2adc0 0x555630c35140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:10.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.692+0000 7fb68f7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555630c2adc0 0x555630c35140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49224/0 (socket says 172.21.15.164:49224) 2024-09-19T08:52:10.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.692+0000 7fb68f7fe640 1 -- 172.21.15.164:0/3703203792 learned_addr learned my addr 172.21.15.164:0/3703203792 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:10.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.692+0000 7fb68ffff640 1 --2- 172.21.15.164:0/3703203792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555630c2a4a0 0x555630c2a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:10.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.693+0000 7fb68ffff640 1 --2- 172.21.15.164:0/1961208381 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb6780a33b0 0x7fb6780a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:10.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.693+0000 7fb68cff9640 1 -- 172.21.15.164:0/1961208381 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7fb6840f0250 con 0x7fb690101420 2024-09-19T08:52:10.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.693+0000 7fb68effd640 1 --2- 172.21.15.164:0/3703203792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555630c35680 0x555630c37a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:10.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.693+0000 7fb68f7fe640 1 -- 172.21.15.164:0/3703203792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555630c2a4a0 msgr2=0x555630c2a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:10.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.693+0000 7fb68f7fe640 1 --2- 172.21.15.164:0/3703203792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555630c2a4a0 0x555630c2a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:10.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.693+0000 7fb68f7fe640 1 -- 172.21.15.164:0/3703203792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555630c35680 msgr2=0x555630c37a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:10.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.693+0000 7fb68f7fe640 1 --2- 172.21.15.164:0/3703203792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555630c35680 0x555630c37a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:10.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.693+0000 7fb68f7fe640 1 -- 172.21.15.164:0/3703203792 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555630b1c440 con 0x555630c2adc0 2024-09-19T08:52:10.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.693+0000 7fb68f7fe640 1 --2- 172.21.15.164:0/3703203792 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555630c2adc0 0x555630c35140 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb68000bc20 tx=0x7fb68000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c375a637b5fac6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:10.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.693+0000 7fb68ffff640 1 --2- 172.21.15.164:0/1961208381 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb6780a33b0 0x7fb6780a5870 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb688009fd0 tx=0x7fb688009600 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:10.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.693+0000 7fb675ffb640 1 -- 172.21.15.164:0/3703203792 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb680015070 con 0x555630c2adc0 2024-09-19T08:52:10.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.693+0000 7fb675ffb640 1 -- 172.21.15.164:0/3703203792 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb680005500 con 0x555630c2adc0 2024-09-19T08:52:10.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.693+0000 7fb675ffb640 1 -- 172.21.15.164:0/3703203792 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6800116a0 con 0x555630c2adc0 2024-09-19T08:52:10.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.694+0000 7fb6984ad740 1 -- 172.21.15.164:0/3703203792 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555630c2adc0 msgr2=0x555630c35140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:10.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.694+0000 7fb6984ad740 1 --2- 172.21.15.164:0/3703203792 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555630c2adc0 0x555630c35140 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb68000bc20 tx=0x7fb68000f730 comp rx=0 tx=0).stop 2024-09-19T08:52:10.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.694+0000 7fb6984ad740 1 -- 172.21.15.164:0/3703203792 shutdown_connections 2024-09-19T08:52:10.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.694+0000 7fb6984ad740 1 --2- 172.21.15.164:0/3703203792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555630c35680 0x555630c37a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:10.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.694+0000 7fb6984ad740 1 --2- 172.21.15.164:0/3703203792 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555630c2adc0 0x555630c35140 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:10.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.694+0000 7fb6984ad740 1 --2- 172.21.15.164:0/3703203792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555630c2a4a0 0x555630c2a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:10.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.694+0000 7fb6984ad740 1 -- 172.21.15.164:0/3703203792 >> 172.21.15.164:0/3703203792 conn(0x555630c26a30 msgr2=0x555630c26e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:10.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.694+0000 7fb6984ad740 1 -- 172.21.15.164:0/3703203792 shutdown_connections 2024-09-19T08:52:10.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.694+0000 7fb6984ad740 1 -- 172.21.15.164:0/3703203792 wait complete. 2024-09-19T08:52:10.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.694+0000 7fb6984ad740 1 -- start start 2024-09-19T08:52:10.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.695+0000 7fb6984ad740 1 Processor -- start 2024-09-19T08:52:10.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.695+0000 7fb6984ad740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555630c2a4a0 0x555630c34f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:10.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.695+0000 7fb6984ad740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555630c2adc0 0x555630c321f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:10.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.695+0000 7fb6984ad740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555630c35680 0x555630c32730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:10.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.695+0000 7fb6984ad740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555630c38250 con 0x555630c2adc0 2024-09-19T08:52:10.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.695+0000 7fb6984ad740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555630b4b050 con 0x555630c35680 2024-09-19T08:52:10.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.695+0000 7fb6984ad740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555630c383d0 con 0x555630c2a4a0 2024-09-19T08:52:10.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.695+0000 7fb68effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555630c2adc0 0x555630c321f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:10.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.695+0000 7fb6984ad740 10 client.0 _ll_register_callbacks cb 0x555630b8b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:52:10.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.695+0000 7fb68effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555630c2adc0 0x555630c321f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49234/0 (socket says 172.21.15.164:49234) 2024-09-19T08:52:10.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.695+0000 7fb68effd640 1 -- 172.21.15.164:0/731787154 learned_addr learned my addr 172.21.15.164:0/731787154 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:10.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.695+0000 7fb68f7fe640 1 --2- 172.21.15.164:0/731787154 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555630c2a4a0 0x555630c34f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:10.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.695+0000 7fb68ffff640 1 --2- 172.21.15.164:0/731787154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555630c35680 0x555630c32730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:10.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.695+0000 7fb68effd640 1 -- 172.21.15.164:0/731787154 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555630c2a4a0 msgr2=0x555630c34f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:10.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.695+0000 7fb68effd640 1 --2- 172.21.15.164:0/731787154 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555630c2a4a0 0x555630c34f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:10.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.695+0000 7fb68effd640 1 -- 172.21.15.164:0/731787154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555630c35680 msgr2=0x555630c32730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:10.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.695+0000 7fb68effd640 1 --2- 172.21.15.164:0/731787154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555630c35680 0x555630c32730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:10.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.695+0000 7fb68effd640 1 -- 172.21.15.164:0/731787154 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556309013c0 con 0x555630c2adc0 2024-09-19T08:52:10.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.695+0000 7fb68ffff640 1 --2- 172.21.15.164:0/731787154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555630c35680 0x555630c32730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:52:10.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.695+0000 7fb68effd640 1 --2- 172.21.15.164:0/731787154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555630c2adc0 0x555630c321f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb684009a50 tx=0x7fb6840b88e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=782232b7b9d894a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:10.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.695+0000 7fb66effd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:52:10.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.696+0000 7fb66effd640 1 -- 172.21.15.164:0/731787154 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb684049040 con 0x555630c2adc0 2024-09-19T08:52:10.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.696+0000 7fb66effd640 1 -- 172.21.15.164:0/731787154 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7fb6840b8b80 con 0x555630c2adc0 2024-09-19T08:52:10.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.696+0000 7fb6984ad740 1 -- 172.21.15.164:0/731787154 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb680005500 con 0x555630c2adc0 2024-09-19T08:52:10.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.696+0000 7fb66effd640 1 -- 172.21.15.164:0/731787154 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb6840bc510 con 0x555630c2adc0 2024-09-19T08:52:10.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.696+0000 7fb6984ad740 10 client.16740 fetch_fsmap learned FSMap version 31 2024-09-19T08:52:10.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.696+0000 7fb6984ad740 1 -- 172.21.15.164:0/731787154 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555630c38250 con 0x555630c2adc0 2024-09-19T08:52:10.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.696+0000 7fb66effd640 1 -- 172.21.15.164:0/731787154 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb680005500 con 0x555630c2adc0 2024-09-19T08:52:10.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.696+0000 7fb66effd640 1 -- 172.21.15.164:0/731787154 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7fb6840bc770 con 0x555630c2adc0 2024-09-19T08:52:10.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.696+0000 7fb6984ad740 10 client.16740 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T08:52:10.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.696+0000 7fb6984ad740 10 client.16740 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:52:10.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.696+0000 7fb6984ad740 20 client.16740 populate_metadata read hostname 'smithi164' 2024-09-19T08:52:10.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.696+0000 7fb6984ad740 1 --2- 172.21.15.164:0/731787154 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555630c450a0 0x555630c474d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:10.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.696+0000 7fb6984ad740 4 client.16740 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:52:10.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.696+0000 7fb6984ad740 1 -- 172.21.15.164:0/731787154 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555630b9d850 con 0x555630c450a0 2024-09-19T08:52:10.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.696+0000 7fb68f7fe640 1 --2- 172.21.15.164:0/731787154 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555630c450a0 0x555630c474d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:10.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.697+0000 7fb68f7fe640 1 --2- 172.21.15.164:0/731787154 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555630c450a0 0x555630c474d0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7e7e2e772276af6 server_cookie=a0d25b2f217a1abd in_seq=0 out_seq=0 2024-09-19T08:52:10.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.697+0000 7fb66effd640 10 client.16740 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:52:10.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.697+0000 7fb66effd640 1 -- 172.21.15.164:0/731787154 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555630b9d850 con 0x555630c450a0 2024-09-19T08:52:10.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.697+0000 7fb66effd640 10 client.16740 handle_command_reply: tid=0 2024-09-19T08:52:10.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.697+0000 7fb6984ad740 1 client.16740 shutdown 2024-09-19T08:52:10.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.698+0000 7fb6984ad740 1 -- 172.21.15.164:0/731787154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555630c2adc0 msgr2=0x555630c321f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:10.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.698+0000 7fb6984ad740 1 --2- 172.21.15.164:0/731787154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555630c2adc0 0x555630c321f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb684009a50 tx=0x7fb6840b88e0 comp rx=0 tx=0).stop 2024-09-19T08:52:10.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.698+0000 7fb6984ad740 1 -- 172.21.15.164:0/731787154 shutdown_connections 2024-09-19T08:52:10.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.698+0000 7fb6984ad740 1 --2- 172.21.15.164:0/731787154 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555630c450a0 0x555630c474d0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:10.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.698+0000 7fb6984ad740 1 --2- 172.21.15.164:0/731787154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555630c35680 0x555630c32730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:10.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.698+0000 7fb6984ad740 1 --2- 172.21.15.164:0/731787154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555630c2adc0 0x555630c321f0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:10.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.698+0000 7fb66effd640 0 client.16740 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:52:10.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.698+0000 7fb6984ad740 1 --2- 172.21.15.164:0/731787154 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555630c2a4a0 0x555630c34f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:10.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.698+0000 7fb6984ad740 1 -- 172.21.15.164:0/731787154 >> 172.21.15.164:0/731787154 conn(0x555630c26a30 msgr2=0x555630c2c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:10.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.698+0000 7fb6984ad740 1 -- 172.21.15.164:0/731787154 shutdown_connections 2024-09-19T08:52:10.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.698+0000 7fb6984ad740 1 -- 172.21.15.164:0/731787154 wait complete. 2024-09-19T08:52:10.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.698+0000 7fb6984ad740 20 client.16740 trim_cache size 0 max 16384 2024-09-19T08:52:10.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.715+0000 7fb6984ad740 1 Processor -- start 2024-09-19T08:52:10.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.715+0000 7fb6984ad740 1 -- start start 2024-09-19T08:52:10.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.715+0000 7fb6984ad740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555630c26a30 0x555630c321b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:10.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.715+0000 7fb6984ad740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555630c2a4a0 0x555630c34160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:10.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.715+0000 7fb6984ad740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555630c2adc0 0x555630c4cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:10.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.715+0000 7fb6984ad740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb6901ad760 con 0x555630c2adc0 2024-09-19T08:52:10.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.715+0000 7fb6984ad740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb6901af2d0 con 0x555630c2a4a0 2024-09-19T08:52:10.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.715+0000 7fb6984ad740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb6901ade30 con 0x555630c26a30 2024-09-19T08:52:10.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.715+0000 7fb68ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555630c2adc0 0x555630c4cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:10.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.715+0000 7fb68ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555630c2adc0 0x555630c4cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49250/0 (socket says 172.21.15.164:49250) 2024-09-19T08:52:10.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.715+0000 7fb68ffff640 1 -- 172.21.15.164:0/1022005728 learned_addr learned my addr 172.21.15.164:0/1022005728 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:10.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.715+0000 7fb68effd640 1 --2- 172.21.15.164:0/1022005728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555630c2a4a0 0x555630c34160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:10.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.715+0000 7fb68f7fe640 1 --2- 172.21.15.164:0/1022005728 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555630c26a30 0x555630c321b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:10.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.715+0000 7fb68ffff640 1 -- 172.21.15.164:0/1022005728 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555630c26a30 msgr2=0x555630c321b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:10.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.715+0000 7fb68ffff640 1 --2- 172.21.15.164:0/1022005728 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555630c26a30 0x555630c321b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:10.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.715+0000 7fb68ffff640 1 -- 172.21.15.164:0/1022005728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555630c2a4a0 msgr2=0x555630c34160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:10.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.715+0000 7fb68ffff640 1 --2- 172.21.15.164:0/1022005728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555630c2a4a0 0x555630c34160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:10.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.715+0000 7fb68ffff640 1 -- 172.21.15.164:0/1022005728 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555630c26860 con 0x555630c2adc0 2024-09-19T08:52:10.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.716+0000 7fb68ffff640 1 --2- 172.21.15.164:0/1022005728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555630c2adc0 0x555630c4cde0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb688009e10 tx=0x7fb688009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04f2d095af31cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:10.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.716+0000 7fb6757fa640 1 -- 172.21.15.164:0/1022005728 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb688012d60 con 0x555630c2adc0 2024-09-19T08:52:10.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.716+0000 7fb6757fa640 1 -- 172.21.15.164:0/1022005728 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb688020960 con 0x555630c2adc0 2024-09-19T08:52:10.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.716+0000 7fb6757fa640 1 -- 172.21.15.164:0/1022005728 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6880055d0 con 0x555630c2adc0 2024-09-19T08:52:10.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.716+0000 7fb6984ad740 1 -- 172.21.15.164:0/1022005728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555630c2adc0 msgr2=0x555630c4cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:10.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.716+0000 7fb6984ad740 1 --2- 172.21.15.164:0/1022005728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555630c2adc0 0x555630c4cde0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb688009e10 tx=0x7fb688009a00 comp rx=0 tx=0).stop 2024-09-19T08:52:10.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.716+0000 7fb6984ad740 1 -- 172.21.15.164:0/1022005728 shutdown_connections 2024-09-19T08:52:10.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.716+0000 7fb6984ad740 1 --2- 172.21.15.164:0/1022005728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555630c2adc0 0x555630c4cde0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:10.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.716+0000 7fb6984ad740 1 --2- 172.21.15.164:0/1022005728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555630c2a4a0 0x555630c34160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:10.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.716+0000 7fb6984ad740 1 --2- 172.21.15.164:0/1022005728 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555630c26a30 0x555630c321b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:10.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.716+0000 7fb6984ad740 1 -- 172.21.15.164:0/1022005728 >> 172.21.15.164:0/1022005728 conn(0x555630c32890 msgr2=0x555630c33a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:10.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.716+0000 7fb6984ad740 1 -- 172.21.15.164:0/1022005728 shutdown_connections 2024-09-19T08:52:10.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.717+0000 7fb6984ad740 1 -- 172.21.15.164:0/1022005728 wait complete. 2024-09-19T08:52:10.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.717+0000 7fb6984ad740 1 -- start start 2024-09-19T08:52:10.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.717+0000 7fb6984ad740 1 Processor -- start 2024-09-19T08:52:10.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.717+0000 7fb6984ad740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555630c26a30 0x555630c3b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:10.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.717+0000 7fb6984ad740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555630c2a4a0 0x555630c3bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:10.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.717+0000 7fb6984ad740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555630c2adc0 0x555630caedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:10.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.717+0000 7fb6984ad740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb6901aebd0 con 0x555630c2a4a0 2024-09-19T08:52:10.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.717+0000 7fb6984ad740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb6901ae500 con 0x555630c2adc0 2024-09-19T08:52:10.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.717+0000 7fb6984ad740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb6901ab760 con 0x555630c26a30 2024-09-19T08:52:10.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.717+0000 7fb6984ad740 10 client.0 _ll_register_callbacks cb 0x7fb684005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:52:10.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.717+0000 7fb68effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555630c2a4a0 0x555630c3bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:10.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.717+0000 7fb68effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555630c2a4a0 0x555630c3bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49260/0 (socket says 172.21.15.164:49260) 2024-09-19T08:52:10.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.717+0000 7fb68effd640 1 -- 172.21.15.164:0/1792195944 learned_addr learned my addr 172.21.15.164:0/1792195944 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:10.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.718+0000 7fb68f7fe640 1 --2- 172.21.15.164:0/1792195944 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555630c26a30 0x555630c3b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:10.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.718+0000 7fb68effd640 1 -- 172.21.15.164:0/1792195944 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555630c26a30 msgr2=0x555630c3b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:10.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.718+0000 7fb68effd640 1 --2- 172.21.15.164:0/1792195944 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555630c26a30 0x555630c3b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:10.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.718+0000 7fb68effd640 1 -- 172.21.15.164:0/1792195944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555630c2adc0 msgr2=0x555630caedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:10.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.718+0000 7fb68ffff640 1 --2- 172.21.15.164:0/1792195944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555630c2adc0 0x555630caedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:10.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.718+0000 7fb68effd640 1 --2- 172.21.15.164:0/1792195944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555630c2adc0 0x555630caedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:10.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.718+0000 7fb68effd640 1 -- 172.21.15.164:0/1792195944 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555630c3c530 con 0x555630c2a4a0 2024-09-19T08:52:10.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.718+0000 7fb68ffff640 1 --2- 172.21.15.164:0/1792195944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555630c2adc0 0x555630caedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:52:10.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.718+0000 7fb68effd640 1 --2- 172.21.15.164:0/1792195944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555630c2a4a0 0x555630c3bc50 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fb6840c4c50 tx=0x7fb684009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d9073f357304e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:10.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.718+0000 7fb66effd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:52:10.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.718+0000 7fb66effd640 1 -- 172.21.15.164:0/1792195944 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6840a5020 con 0x555630c2a4a0 2024-09-19T08:52:10.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.718+0000 7fb66effd640 1 -- 172.21.15.164:0/1792195944 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7fb6840edc00 con 0x555630c2a4a0 2024-09-19T08:52:10.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.718+0000 7fb6984ad740 1 -- 172.21.15.164:0/1792195944 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb688020960 con 0x555630c2a4a0 2024-09-19T08:52:10.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.718+0000 7fb66effd640 1 -- 172.21.15.164:0/1792195944 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb6840cd050 con 0x555630c2a4a0 2024-09-19T08:52:10.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.718+0000 7fb6984ad740 10 client.16746 fetch_fsmap learned FSMap version 31 2024-09-19T08:52:10.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.718+0000 7fb6984ad740 1 -- 172.21.15.164:0/1792195944 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555630cb0450 con 0x555630c2a4a0 2024-09-19T08:52:10.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.718+0000 7fb66effd640 1 -- 172.21.15.164:0/1792195944 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb688020960 con 0x555630c2a4a0 2024-09-19T08:52:10.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.718+0000 7fb66effd640 1 -- 172.21.15.164:0/1792195944 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7fb6840c4cf0 con 0x555630c2a4a0 2024-09-19T08:52:10.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.718+0000 7fb6984ad740 10 client.16746 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T08:52:10.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.718+0000 7fb6984ad740 10 client.16746 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:52:10.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.718+0000 7fb6984ad740 20 client.16746 populate_metadata read hostname 'smithi164' 2024-09-19T08:52:10.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.719+0000 7fb6984ad740 1 --2- 172.21.15.164:0/1792195944 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555630c35680 0x555630cb2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:10.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.719+0000 7fb6984ad740 4 client.16746 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:52:10.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.719+0000 7fb6984ad740 1 -- 172.21.15.164:0/1792195944 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555630b97d60 con 0x555630c35680 2024-09-19T08:52:10.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.719+0000 7fb68f7fe640 1 --2- 172.21.15.164:0/1792195944 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555630c35680 0x555630cb2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:10.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.719+0000 7fb68f7fe640 1 --2- 172.21.15.164:0/1792195944 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555630c35680 0x555630cb2a70 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffb71bee1e3cf6e1 server_cookie=728f1c8858f9d6b6 in_seq=0 out_seq=0 2024-09-19T08:52:10.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.719+0000 7fb66effd640 10 client.16746 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:52:10.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.719+0000 7fb66effd640 1 -- 172.21.15.164:0/1792195944 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555630b97d60 con 0x555630c35680 2024-09-19T08:52:10.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.719+0000 7fb66effd640 10 client.16746 handle_command_reply: tid=0 2024-09-19T08:52:10.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.719+0000 7fb6984ad740 1 client.16746 shutdown 2024-09-19T08:52:10.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.720+0000 7fb6984ad740 1 -- 172.21.15.164:0/1792195944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555630c2a4a0 msgr2=0x555630c3bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:10.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.720+0000 7fb6984ad740 1 --2- 172.21.15.164:0/1792195944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555630c2a4a0 0x555630c3bc50 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fb6840c4c50 tx=0x7fb684009560 comp rx=0 tx=0).stop 2024-09-19T08:52:10.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.720+0000 7fb6984ad740 1 -- 172.21.15.164:0/1792195944 shutdown_connections 2024-09-19T08:52:10.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.720+0000 7fb6984ad740 1 --2- 172.21.15.164:0/1792195944 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555630c35680 0x555630cb2a70 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:10.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.720+0000 7fb6984ad740 1 --2- 172.21.15.164:0/1792195944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555630c2adc0 0x555630caedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:10.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.720+0000 7fb66effd640 0 client.16746 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:52:10.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.720+0000 7fb6984ad740 1 --2- 172.21.15.164:0/1792195944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555630c2a4a0 0x555630c3bc50 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:10.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.720+0000 7fb6984ad740 1 --2- 172.21.15.164:0/1792195944 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555630c26a30 0x555630c3b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:10.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.720+0000 7fb6984ad740 1 -- 172.21.15.164:0/1792195944 >> 172.21.15.164:0/1792195944 conn(0x555630c32890 msgr2=0x555630c34270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:10.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.720+0000 7fb6984ad740 1 -- 172.21.15.164:0/1792195944 shutdown_connections 2024-09-19T08:52:10.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.720+0000 7fb6984ad740 1 -- 172.21.15.164:0/1792195944 wait complete. 2024-09-19T08:52:10.749 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:52:10.749 INFO:teuthology.orchestra.run.smithi164.stdout: "status": "no active scrubs running", 2024-09-19T08:52:10.749 INFO:teuthology.orchestra.run.smithi164.stdout: "scrubs": {} 2024-09-19T08:52:10.749 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:52:10.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.720+0000 7fb6984ad740 20 client.16746 trim_cache size 0 max 16384 2024-09-19T08:52:10.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.720+0000 7fb6960d4640 1 -- 172.21.15.164:0/1961208381 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb6780a33b0 msgr2=0x7fb6780a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:10.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.720+0000 7fb6960d4640 1 --2- 172.21.15.164:0/1961208381 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb6780a33b0 0x7fb6780a5870 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb688009fd0 tx=0x7fb688009600 comp rx=0 tx=0).stop 2024-09-19T08:52:10.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.720+0000 7fb6960d4640 1 -- 172.21.15.164:0/1961208381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb690101420 msgr2=0x7fb690199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:10.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.720+0000 7fb6960d4640 1 --2- 172.21.15.164:0/1961208381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb690101420 0x7fb690199850 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb684009230 tx=0x7fb684004270 comp rx=0 tx=0).stop 2024-09-19T08:52:10.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.720+0000 7fb6960d4640 1 -- 172.21.15.164:0/1961208381 shutdown_connections 2024-09-19T08:52:10.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.720+0000 7fb6960d4640 1 --2- 172.21.15.164:0/1961208381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb690106a80 0x7fb6901a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:10.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.720+0000 7fb6960d4640 1 --2- 172.21.15.164:0/1961208381 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb6780a33b0 0x7fb6780a5870 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:10.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.720+0000 7fb6960d4640 1 --2- 172.21.15.164:0/1961208381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb690101420 0x7fb690199850 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:10.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.720+0000 7fb6960d4640 1 --2- 172.21.15.164:0/1961208381 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb690100a80 0x7fb690199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:10.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.720+0000 7fb6960d4640 1 -- 172.21.15.164:0/1961208381 >> 172.21.15.164:0/1961208381 conn(0x7fb6900fc7a0 msgr2=0x7fb6900fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:10.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.721+0000 7fb6960d4640 1 -- 172.21.15.164:0/1961208381 shutdown_connections 2024-09-19T08:52:10.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:10.721+0000 7fb6960d4640 1 -- 172.21.15.164:0/1961208381 wait complete. 2024-09-19T08:52:11.553 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T08:52:11.553 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:52:11.801 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:52:12.206 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:12 smithi164 ceph-mon[30928]: pgmap v196: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 528 MiB/s wr, 1.82k op/s 2024-09-19T08:52:12.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:12 smithi170 ceph-mon[35072]: pgmap v196: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 528 MiB/s wr, 1.82k op/s 2024-09-19T08:52:12.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:12 smithi173 ceph-mon[35306]: pgmap v196: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 528 MiB/s wr, 1.82k op/s 2024-09-19T08:52:12.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.530+0000 7f01c8fba640 1 -- 172.21.15.164:0/2617385908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01c411b060 msgr2=0x7f01c411d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:12.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.530+0000 7f01c8fba640 1 --2- 172.21.15.164:0/2617385908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01c411b060 0x7f01c411d450 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f01bc009be0 tx=0x7f01bc02a6d0 comp rx=0 tx=0).stop 2024-09-19T08:52:12.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.533+0000 7f01c8fba640 1 -- 172.21.15.164:0/2617385908 shutdown_connections 2024-09-19T08:52:12.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.533+0000 7f01c8fba640 1 --2- 172.21.15.164:0/2617385908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01c411b060 0x7f01c411d450 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:12.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.533+0000 7f01c8fba640 1 --2- 172.21.15.164:0/2617385908 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01c406e4c0 0x7f01c411ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:12.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.533+0000 7f01c8fba640 1 --2- 172.21.15.164:0/2617385908 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01c406dba0 0x7f01c406df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:12.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.533+0000 7f01c8fba640 1 -- 172.21.15.164:0/2617385908 >> 172.21.15.164:0/2617385908 conn(0x7f01c406cbe0 msgr2=0x7f01c406cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:12.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.534+0000 7f01c8fba640 1 -- 172.21.15.164:0/2617385908 shutdown_connections 2024-09-19T08:52:12.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.535+0000 7f01c8fba640 1 -- 172.21.15.164:0/2617385908 wait complete. 2024-09-19T08:52:12.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.536+0000 7f01c8fba640 1 Processor -- start 2024-09-19T08:52:12.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.539+0000 7f01c8fba640 1 -- start start 2024-09-19T08:52:12.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.541+0000 7f01c8fba640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01c406dba0 0x7f01c411a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:12.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.541+0000 7f01c8fba640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01c406e4c0 0x7f01c41136e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:12.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.542+0000 7f01c8fba640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01c411b060 0x7f01c4113c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:12.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.542+0000 7f01c8fba640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f01c4120bd0 con 0x7f01c411b060 2024-09-19T08:52:12.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.542+0000 7f01c8fba640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f01c4120a50 con 0x7f01c406e4c0 2024-09-19T08:52:12.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.543+0000 7f01c8fba640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f01c4120d50 con 0x7f01c406dba0 2024-09-19T08:52:12.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.546+0000 7f01c3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01c411b060 0x7f01c4113c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:12.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.546+0000 7f01c3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01c411b060 0x7f01c4113c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49266/0 (socket says 172.21.15.164:49266) 2024-09-19T08:52:12.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.546+0000 7f01c3fff640 1 -- 172.21.15.164:0/597509461 learned_addr learned my addr 172.21.15.164:0/597509461 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:12.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.548+0000 7f01c37fe640 1 --2- 172.21.15.164:0/597509461 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01c406dba0 0x7f01c411a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:12.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.549+0000 7f01c2ffd640 1 --2- 172.21.15.164:0/597509461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01c406e4c0 0x7f01c41136e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:12.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.555+0000 7f01c37fe640 1 -- 172.21.15.164:0/597509461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01c406e4c0 msgr2=0x7f01c41136e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:12.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.555+0000 7f01c37fe640 1 --2- 172.21.15.164:0/597509461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01c406e4c0 0x7f01c41136e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:12.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.555+0000 7f01c37fe640 1 -- 172.21.15.164:0/597509461 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01c411b060 msgr2=0x7f01c4113c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:12.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.555+0000 7f01c37fe640 1 --2- 172.21.15.164:0/597509461 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01c411b060 0x7f01c4113c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:12.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.555+0000 7f01c37fe640 1 -- 172.21.15.164:0/597509461 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c4114210 con 0x7f01c406dba0 2024-09-19T08:52:12.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.564+0000 7f01c37fe640 1 --2- 172.21.15.164:0/597509461 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01c406dba0 0x7f01c411a650 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f01b400e790 tx=0x7f01b400ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:12.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.573+0000 7f01c0ff9640 1 -- 172.21.15.164:0/597509461 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b400eea0 con 0x7f01c406dba0 2024-09-19T08:52:12.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.573+0000 7f01c0ff9640 1 -- 172.21.15.164:0/597509461 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01b400ccc0 con 0x7f01c406dba0 2024-09-19T08:52:12.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.573+0000 7f01c0ff9640 1 -- 172.21.15.164:0/597509461 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b4018780 con 0x7f01c406dba0 2024-09-19T08:52:12.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.573+0000 7f01c8fba640 1 -- 172.21.15.164:0/597509461 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c4114500 con 0x7f01c406dba0 2024-09-19T08:52:12.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.573+0000 7f01c8fba640 1 -- 172.21.15.164:0/597509461 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c41184a0 con 0x7f01c406dba0 2024-09-19T08:52:12.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.578+0000 7f01c8fba640 1 -- 172.21.15.164:0/597509461 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01c406f2e0 con 0x7f01c406dba0 2024-09-19T08:52:12.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.584+0000 7f01c0ff9640 1 -- 172.21.15.164:0/597509461 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f01b4010430 con 0x7f01c406dba0 2024-09-19T08:52:12.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.588+0000 7f01c0ff9640 1 --2- 172.21.15.164:0/597509461 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f01ac0a3500 0x7f01ac0a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:12.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.589+0000 7f01c0ff9640 1 -- 172.21.15.164:0/597509461 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f01b409a3f0 con 0x7f01c406dba0 2024-09-19T08:52:12.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.591+0000 7f01c0ff9640 1 -- 172.21.15.164:0/597509461 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f01b409ca50 con 0x7f01c406dba0 2024-09-19T08:52:12.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.592+0000 7f01c2ffd640 1 --2- 172.21.15.164:0/597509461 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f01ac0a3500 0x7f01ac0a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:12.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:12.604+0000 7f01c2ffd640 1 --2- 172.21.15.164:0/597509461 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f01ac0a3500 0x7f01ac0a59c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f01c4114ea0 tx=0x7f01b8008040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:13.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:13.103+0000 7f01c8fba640 1 -- 172.21.15.164:0/597509461 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01c406df80 con 0x7f01c406dba0 2024-09-19T08:52:13.106 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:52:13.106 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":31,"btime":"2024-09-19T08:52:03:050221+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:52:02.074057+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24442,24424,24449,14595]},"id":1}]} 2024-09-19T08:52:13.107 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 31 2024-09-19T08:52:13.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:13.106+0000 7f01c0ff9640 1 -- 172.21.15.164:0/597509461 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11985 (secure 0 0 0) 0x7f01c406df80 con 0x7f01c406dba0 2024-09-19T08:52:13.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:13.113+0000 7f01c8fba640 1 -- 172.21.15.164:0/597509461 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f01ac0a3500 msgr2=0x7f01ac0a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:13.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:13.113+0000 7f01c8fba640 1 --2- 172.21.15.164:0/597509461 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f01ac0a3500 0x7f01ac0a59c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f01c4114ea0 tx=0x7f01b8008040 comp rx=0 tx=0).stop 2024-09-19T08:52:13.113 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:13.113+0000 7f01c8fba640 1 -- 172.21.15.164:0/597509461 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01c406dba0 msgr2=0x7f01c411a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:13.114 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:13.113+0000 7f01c8fba640 1 --2- 172.21.15.164:0/597509461 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01c406dba0 0x7f01c411a650 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f01b400e790 tx=0x7f01b400ec50 comp rx=0 tx=0).stop 2024-09-19T08:52:13.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:13.114+0000 7f01c8fba640 1 -- 172.21.15.164:0/597509461 shutdown_connections 2024-09-19T08:52:13.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:13.114+0000 7f01c8fba640 1 --2- 172.21.15.164:0/597509461 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f01ac0a3500 0x7f01ac0a59c0 secure :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f01c4114ea0 tx=0x7f01b8008040 comp rx=0 tx=0).stop 2024-09-19T08:52:13.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:13.114+0000 7f01c8fba640 1 --2- 172.21.15.164:0/597509461 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01c411b060 0x7f01c4113c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:13.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:13.114+0000 7f01c8fba640 1 --2- 172.21.15.164:0/597509461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01c406e4c0 0x7f01c41136e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:13.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:13.114+0000 7f01c8fba640 1 --2- 172.21.15.164:0/597509461 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01c406dba0 0x7f01c411a650 secure :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f01b400e790 tx=0x7f01b400ec50 comp rx=0 tx=0).stop 2024-09-19T08:52:13.115 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:13.114+0000 7f01c8fba640 1 -- 172.21.15.164:0/597509461 >> 172.21.15.164:0/597509461 conn(0x7f01c406cbe0 msgr2=0x7f01c411b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:13.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:13.119+0000 7f01c8fba640 1 -- 172.21.15.164:0/597509461 shutdown_connections 2024-09-19T08:52:13.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:13.122+0000 7f01c8fba640 1 -- 172.21.15.164:0/597509461 wait complete. 2024-09-19T08:52:13.273 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:13 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/597509461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:52:13.624 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:13 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/597509461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:52:13.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:13 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/597509461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:52:14.186 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T08:52:14.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:14 smithi164 ceph-mon[30928]: pgmap v197: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 413 MiB/s wr, 1.44k op/s 2024-09-19T08:52:14.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:14 smithi170 ceph-mon[35072]: pgmap v197: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 413 MiB/s wr, 1.44k op/s 2024-09-19T08:52:14.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:14 smithi173 ceph-mon[35306]: pgmap v197: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 413 MiB/s wr, 1.44k op/s 2024-09-19T08:52:14.683 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:52:15.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.187+0000 7f556b6e9640 1 -- 172.21.15.164:0/3592942998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5564074eb0 msgr2=0x7f5564112680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:15.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.187+0000 7f556b6e9640 1 --2- 172.21.15.164:0/3592942998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5564074eb0 0x7f5564112680 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5554009860 tx=0x7f555402a340 comp rx=0 tx=0).stop 2024-09-19T08:52:15.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.187+0000 7f556b6e9640 1 -- 172.21.15.164:0/3592942998 shutdown_connections 2024-09-19T08:52:15.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.187+0000 7f556b6e9640 1 --2- 172.21.15.164:0/3592942998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5564112bc0 0x7f5564114fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:15.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.187+0000 7f556b6e9640 1 --2- 172.21.15.164:0/3592942998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5564074eb0 0x7f5564112680 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:15.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.187+0000 7f556b6e9640 1 --2- 172.21.15.164:0/3592942998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5564074590 0x7f5564074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:15.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.187+0000 7f556b6e9640 1 -- 172.21.15.164:0/3592942998 >> 172.21.15.164:0/3592942998 conn(0x7f55640fd8d0 msgr2=0x7f55640ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:15.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.187+0000 7f556b6e9640 1 -- 172.21.15.164:0/3592942998 shutdown_connections 2024-09-19T08:52:15.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.187+0000 7f556b6e9640 1 -- 172.21.15.164:0/3592942998 wait complete. 2024-09-19T08:52:15.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.188+0000 7f556b6e9640 1 Processor -- start 2024-09-19T08:52:15.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.188+0000 7f556b6e9640 1 -- start start 2024-09-19T08:52:15.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.188+0000 7f556b6e9640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5564074590 0x7f55641a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:15.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.188+0000 7f556b6e9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5564074eb0 0x7f55641a4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:15.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.188+0000 7f556b6e9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5564112bc0 0x7f556419e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:15.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.188+0000 7f556b6e9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5564118590 con 0x7f5564074eb0 2024-09-19T08:52:15.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.188+0000 7f556b6e9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5564118410 con 0x7f5564112bc0 2024-09-19T08:52:15.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.188+0000 7f556b6e9640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5564118710 con 0x7f5564074590 2024-09-19T08:52:15.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.188+0000 7f5568c84640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5564074eb0 0x7f55641a4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:15.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.188+0000 7f5568c84640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5564074eb0 0x7f55641a4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46732/0 (socket says 172.21.15.164:46732) 2024-09-19T08:52:15.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.188+0000 7f5568c84640 1 -- 172.21.15.164:0/3360628867 learned_addr learned my addr 172.21.15.164:0/3360628867 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:15.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.188+0000 7f5569485640 1 --2- 172.21.15.164:0/3360628867 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5564074590 0x7f55641a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:15.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.188+0000 7f5569c86640 1 --2- 172.21.15.164:0/3360628867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5564112bc0 0x7f556419e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:15.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.189+0000 7f5568c84640 1 -- 172.21.15.164:0/3360628867 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5564074590 msgr2=0x7f55641a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:15.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.189+0000 7f5568c84640 1 --2- 172.21.15.164:0/3360628867 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5564074590 0x7f55641a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:15.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.189+0000 7f5568c84640 1 -- 172.21.15.164:0/3360628867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5564112bc0 msgr2=0x7f556419e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:15.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.189+0000 7f5568c84640 1 --2- 172.21.15.164:0/3360628867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5564112bc0 0x7f556419e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:15.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.189+0000 7f5568c84640 1 -- 172.21.15.164:0/3360628867 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556419f030 con 0x7f5564074eb0 2024-09-19T08:52:15.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.189+0000 7f5569c86640 1 --2- 172.21.15.164:0/3360628867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5564112bc0 0x7f556419e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:52:15.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.189+0000 7f5569485640 1 --2- 172.21.15.164:0/3360628867 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5564074590 0x7f55641a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:52:15.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.189+0000 7f5568c84640 1 --2- 172.21.15.164:0/3360628867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5564074eb0 0x7f55641a4c80 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f5554000c00 tx=0x7f5554032440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:15.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.189+0000 7f555a7fc640 1 -- 172.21.15.164:0/3360628867 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5554037050 con 0x7f5564074eb0 2024-09-19T08:52:15.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.189+0000 7f556b6e9640 1 -- 172.21.15.164:0/3360628867 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f556419f2c0 con 0x7f5564074eb0 2024-09-19T08:52:15.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.189+0000 7f555a7fc640 1 -- 172.21.15.164:0/3360628867 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5554043070 con 0x7f5564074eb0 2024-09-19T08:52:15.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.189+0000 7f555a7fc640 1 -- 172.21.15.164:0/3360628867 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5554003b70 con 0x7f5564074eb0 2024-09-19T08:52:15.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.189+0000 7f556b6e9640 1 -- 172.21.15.164:0/3360628867 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55641ab610 con 0x7f5564074eb0 2024-09-19T08:52:15.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.192+0000 7f555a7fc640 1 -- 172.21.15.164:0/3360628867 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f5554003d10 con 0x7f5564074eb0 2024-09-19T08:52:15.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.192+0000 7f556dac2740 1 Processor -- start 2024-09-19T08:52:15.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.192+0000 7f556dac2740 1 -- start start 2024-09-19T08:52:15.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.192+0000 7f556dac2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b498e22b40 0x55b498e21090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:15.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.192+0000 7f556dac2740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b498e215d0 0x55b498e21a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:15.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.192+0000 7f556dac2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b498e2c370 0x55b498e2e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:15.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.192+0000 7f556dac2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b498d872f0 con 0x55b498e2c370 2024-09-19T08:52:15.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.192+0000 7f556dac2740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b498d2ecf0 con 0x55b498e215d0 2024-09-19T08:52:15.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.192+0000 7f556dac2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b498d832b0 con 0x55b498e22b40 2024-09-19T08:52:15.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.193+0000 7f5568c84640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b498e2c370 0x55b498e2e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:15.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.193+0000 7f5569485640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b498e215d0 0x55b498e21a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:15.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.193+0000 7f5568c84640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b498e2c370 0x55b498e2e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46748/0 (socket says 172.21.15.164:46748) 2024-09-19T08:52:15.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.193+0000 7f5568c84640 1 -- 172.21.15.164:0/2131226232 learned_addr learned my addr 172.21.15.164:0/2131226232 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:15.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.193+0000 7f5569c86640 1 --2- 172.21.15.164:0/2131226232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b498e22b40 0x55b498e21090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:15.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.193+0000 7f555a7fc640 1 --2- 172.21.15.164:0/3360628867 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f55500a33b0 0x7f55500a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:15.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.193+0000 7f5569485640 1 -- 172.21.15.164:0/2131226232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b498e22b40 msgr2=0x55b498e21090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:15.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.193+0000 7f5569485640 1 --2- 172.21.15.164:0/2131226232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b498e22b40 0x55b498e21090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:15.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.193+0000 7f5569485640 1 -- 172.21.15.164:0/2131226232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b498e2c370 msgr2=0x55b498e2e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:15.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.193+0000 7f5569485640 1 --2- 172.21.15.164:0/2131226232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b498e2c370 0x55b498e2e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:15.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.193+0000 7f5569485640 1 -- 172.21.15.164:0/2131226232 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b498d133a0 con 0x55b498e215d0 2024-09-19T08:52:15.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.193+0000 7f5569c86640 1 --2- 172.21.15.164:0/2131226232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b498e22b40 0x55b498e21090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:52:15.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.193+0000 7f5569c86640 1 --2- 172.21.15.164:0/3360628867 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f55500a33b0 0x7f55500a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:15.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.193+0000 7f555a7fc640 1 -- 172.21.15.164:0/3360628867 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f55540effc0 con 0x7f5564074eb0 2024-09-19T08:52:15.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.193+0000 7f5569485640 1 --2- 172.21.15.164:0/2131226232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b498e215d0 0x55b498e21a10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5560009fa0 tx=0x7f55600118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa9325489a9f9c9d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:15.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.193+0000 7f554f7fe640 1 -- 172.21.15.164:0/2131226232 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55600040f0 con 0x55b498e215d0 2024-09-19T08:52:15.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.193+0000 7f554f7fe640 1 -- 172.21.15.164:0/2131226232 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5560005500 con 0x55b498e215d0 2024-09-19T08:52:15.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.193+0000 7f554f7fe640 1 -- 172.21.15.164:0/2131226232 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55600035f0 con 0x55b498e215d0 2024-09-19T08:52:15.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.194+0000 7f556dac2740 1 -- 172.21.15.164:0/2131226232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b498e215d0 msgr2=0x55b498e21a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:15.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.194+0000 7f5569c86640 1 --2- 172.21.15.164:0/3360628867 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f55500a33b0 0x7f55500a5870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f555c009ba0 tx=0x7f555c004a40 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:15.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.194+0000 7f556dac2740 1 --2- 172.21.15.164:0/2131226232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b498e215d0 0x55b498e21a10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5560009fa0 tx=0x7f55600118d0 comp rx=0 tx=0).stop 2024-09-19T08:52:15.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.194+0000 7f556dac2740 1 -- 172.21.15.164:0/2131226232 shutdown_connections 2024-09-19T08:52:15.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.194+0000 7f556dac2740 1 --2- 172.21.15.164:0/2131226232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b498e2c370 0x55b498e2e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:15.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.194+0000 7f556dac2740 1 --2- 172.21.15.164:0/2131226232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b498e215d0 0x55b498e21a10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:15.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.194+0000 7f556dac2740 1 --2- 172.21.15.164:0/2131226232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b498e22b40 0x55b498e21090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:15.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.194+0000 7f556dac2740 1 -- 172.21.15.164:0/2131226232 >> 172.21.15.164:0/2131226232 conn(0x55b498dea310 msgr2=0x55b498e1dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:15.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.194+0000 7f556dac2740 1 -- 172.21.15.164:0/2131226232 shutdown_connections 2024-09-19T08:52:15.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.194+0000 7f556dac2740 1 -- 172.21.15.164:0/2131226232 wait complete. 2024-09-19T08:52:15.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.195+0000 7f556dac2740 1 -- start start 2024-09-19T08:52:15.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.195+0000 7f556dac2740 1 Processor -- start 2024-09-19T08:52:15.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.195+0000 7f556dac2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b498e215d0 0x55b498e2bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:15.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.195+0000 7f556dac2740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b498e22b40 0x55b498e29070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:15.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.195+0000 7f556dac2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b498e2c370 0x55b498e29800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:15.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.195+0000 7f556dac2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b498e2f070 con 0x55b498e215d0 2024-09-19T08:52:15.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.195+0000 7f556dac2740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b498d42060 con 0x55b498e22b40 2024-09-19T08:52:15.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.195+0000 7f556dac2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b498e2f1f0 con 0x55b498e2c370 2024-09-19T08:52:15.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.195+0000 7f556dac2740 10 client.0 _ll_register_callbacks cb 0x55b498a958c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:52:15.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.195+0000 7f5569485640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b498e215d0 0x55b498e2bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:15.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.195+0000 7f5569485640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b498e215d0 0x55b498e2bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46762/0 (socket says 172.21.15.164:46762) 2024-09-19T08:52:15.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.195+0000 7f5569485640 1 -- 172.21.15.164:0/407283138 learned_addr learned my addr 172.21.15.164:0/407283138 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:15.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.195+0000 7f5568c84640 1 --2- 172.21.15.164:0/407283138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b498e22b40 0x55b498e29070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:15.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.196+0000 7f5569c86640 1 --2- 172.21.15.164:0/407283138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b498e2c370 0x55b498e29800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:15.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.196+0000 7f5568c84640 1 -- 172.21.15.164:0/407283138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b498e2c370 msgr2=0x55b498e29800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:15.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.196+0000 7f5568c84640 1 --2- 172.21.15.164:0/407283138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b498e2c370 0x55b498e29800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:15.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.196+0000 7f5568c84640 1 -- 172.21.15.164:0/407283138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b498e215d0 msgr2=0x55b498e2bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:15.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.196+0000 7f5568c84640 1 --2- 172.21.15.164:0/407283138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b498e215d0 0x55b498e2bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:15.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.196+0000 7f5568c84640 1 -- 172.21.15.164:0/407283138 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b498af83c0 con 0x55b498e22b40 2024-09-19T08:52:15.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.196+0000 7f5569485640 1 --2- 172.21.15.164:0/407283138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b498e215d0 0x55b498e2bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:52:15.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.196+0000 7f5568c84640 1 --2- 172.21.15.164:0/407283138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b498e22b40 0x55b498e29070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f55540edd50 tx=0x7f55540f3700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af791a03b15410b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:15.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.196+0000 7f554cff9640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-19T08:52:15.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.196+0000 7f554cff9640 1 -- 172.21.15.164:0/407283138 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55540bc1c0 con 0x55b498e22b40 2024-09-19T08:52:15.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.196+0000 7f554cff9640 1 -- 172.21.15.164:0/407283138 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f55540f20d0 con 0x55b498e22b40 2024-09-19T08:52:15.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.196+0000 7f556dac2740 1 -- 172.21.15.164:0/407283138 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5560005500 con 0x55b498e22b40 2024-09-19T08:52:15.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.196+0000 7f554cff9640 1 -- 172.21.15.164:0/407283138 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f55540bbd50 con 0x55b498e22b40 2024-09-19T08:52:15.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.196+0000 7f556dac2740 10 client.26078 fetch_fsmap learned FSMap version 31 2024-09-19T08:52:15.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.196+0000 7f556dac2740 1 -- 172.21.15.164:0/407283138 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b498e3ab80 con 0x55b498e22b40 2024-09-19T08:52:15.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.196+0000 7f554cff9640 1 -- 172.21.15.164:0/407283138 <== mon.2 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5560005500 con 0x55b498e22b40 2024-09-19T08:52:15.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.197+0000 7f554cff9640 1 -- 172.21.15.164:0/407283138 <== mon.2 v2:172.21.15.170:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f55540bc650 con 0x55b498e22b40 2024-09-19T08:52:15.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.197+0000 7f556dac2740 10 client.26078 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T08:52:15.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.197+0000 7f556dac2740 10 client.26078 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-19T08:52:15.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.197+0000 7f556dac2740 20 client.26078 populate_metadata read hostname 'smithi164' 2024-09-19T08:52:15.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.197+0000 7f556dac2740 1 --2- 172.21.15.164:0/407283138 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55b498e3bf90 0x55b498e3e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:15.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.197+0000 7f556dac2740 4 client.26078 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:52:15.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.197+0000 7f556dac2740 1 -- 172.21.15.164:0/407283138 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b498d94860 con 0x55b498e3bf90 2024-09-19T08:52:15.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.197+0000 7f5569485640 1 --2- 172.21.15.164:0/407283138 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55b498e3bf90 0x55b498e3e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:15.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.197+0000 7f554cff9640 10 client.26078 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:52:15.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.197+0000 7f5569485640 1 --2- 172.21.15.164:0/407283138 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55b498e3bf90 0x55b498e3e430 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6e894805b38b5bb7 server_cookie=f732d858f96be46d in_seq=0 out_seq=0 2024-09-19T08:52:15.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.198+0000 7f554cff9640 1 -- 172.21.15.164:0/407283138 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b498d94860 con 0x55b498e3bf90 2024-09-19T08:52:15.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.198+0000 7f554cff9640 10 client.26078 handle_command_reply: tid=0 2024-09-19T08:52:15.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.198+0000 7f556dac2740 1 client.26078 shutdown 2024-09-19T08:52:15.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.198+0000 7f556dac2740 1 -- 172.21.15.164:0/407283138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b498e22b40 msgr2=0x55b498e29070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:15.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.198+0000 7f556dac2740 1 --2- 172.21.15.164:0/407283138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b498e22b40 0x55b498e29070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f55540edd50 tx=0x7f55540f3700 comp rx=0 tx=0).stop 2024-09-19T08:52:15.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.198+0000 7f556dac2740 1 -- 172.21.15.164:0/407283138 shutdown_connections 2024-09-19T08:52:15.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.198+0000 7f556dac2740 1 --2- 172.21.15.164:0/407283138 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55b498e3bf90 0x55b498e3e430 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:15.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.198+0000 7f556dac2740 1 --2- 172.21.15.164:0/407283138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b498e2c370 0x55b498e29800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:15.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.198+0000 7f554cff9640 0 client.26078 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:52:15.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.198+0000 7f556dac2740 1 --2- 172.21.15.164:0/407283138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b498e22b40 0x55b498e29070 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:15.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.198+0000 7f556dac2740 1 --2- 172.21.15.164:0/407283138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b498e215d0 0x55b498e2bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:15.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.198+0000 7f556dac2740 1 -- 172.21.15.164:0/407283138 >> 172.21.15.164:0/407283138 conn(0x55b498dea310 msgr2=0x55b498e23960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:15.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.198+0000 7f556dac2740 1 -- 172.21.15.164:0/407283138 shutdown_connections 2024-09-19T08:52:15.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.198+0000 7f556dac2740 1 -- 172.21.15.164:0/407283138 wait complete. 2024-09-19T08:52:15.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.198+0000 7f556dac2740 20 client.26078 trim_cache size 0 max 16384 2024-09-19T08:52:15.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.225+0000 7f556dac2740 1 Processor -- start 2024-09-19T08:52:15.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.225+0000 7f556dac2740 1 -- start start 2024-09-19T08:52:15.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.226+0000 7f556dac2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b498e215d0 0x55b498e22300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:15.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.226+0000 7f556dac2740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b498e22b40 0x55b498e242c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:15.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.226+0000 7f556dac2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b498e2c370 0x55b498e24800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:15.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.226+0000 7f556dac2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f55641b3d20 con 0x55b498e215d0 2024-09-19T08:52:15.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.226+0000 7f556dac2740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f55641b5890 con 0x55b498e22b40 2024-09-19T08:52:15.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.226+0000 7f556dac2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f55641b43f0 con 0x55b498e2c370 2024-09-19T08:52:15.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.226+0000 7f5569485640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b498e215d0 0x55b498e22300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:15.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.226+0000 7f5569485640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b498e215d0 0x55b498e22300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46764/0 (socket says 172.21.15.164:46764) 2024-09-19T08:52:15.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.226+0000 7f5569485640 1 -- 172.21.15.164:0/2581025251 learned_addr learned my addr 172.21.15.164:0/2581025251 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:15.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.226+0000 7f5568c84640 1 --2- 172.21.15.164:0/2581025251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b498e22b40 0x55b498e242c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:15.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.226+0000 7f5569c86640 1 --2- 172.21.15.164:0/2581025251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b498e2c370 0x55b498e24800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:15.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.226+0000 7f5569485640 1 -- 172.21.15.164:0/2581025251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b498e2c370 msgr2=0x55b498e24800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:15.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.226+0000 7f5569485640 1 --2- 172.21.15.164:0/2581025251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b498e2c370 0x55b498e24800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:15.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.226+0000 7f5569485640 1 -- 172.21.15.164:0/2581025251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b498e22b40 msgr2=0x55b498e242c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:15.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.226+0000 7f5569485640 1 --2- 172.21.15.164:0/2581025251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b498e22b40 0x55b498e242c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:15.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.226+0000 7f5569485640 1 -- 172.21.15.164:0/2581025251 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b498e1e960 con 0x55b498e215d0 2024-09-19T08:52:15.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.226+0000 7f5569c86640 1 --2- 172.21.15.164:0/2581025251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b498e2c370 0x55b498e24800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:52:15.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.226+0000 7f5568c84640 1 --2- 172.21.15.164:0/2581025251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b498e22b40 0x55b498e242c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:52:15.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.226+0000 7f5569485640 1 --2- 172.21.15.164:0/2581025251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b498e215d0 0x55b498e22300 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f5560009710 tx=0x7f55600049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eafb0958625cc040 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:15.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.227+0000 7f554effd640 1 -- 172.21.15.164:0/2581025251 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556001d040 con 0x55b498e215d0 2024-09-19T08:52:15.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.227+0000 7f554effd640 1 -- 172.21.15.164:0/2581025251 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5560003d80 con 0x55b498e215d0 2024-09-19T08:52:15.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.227+0000 7f554effd640 1 -- 172.21.15.164:0/2581025251 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556000fa40 con 0x55b498e215d0 2024-09-19T08:52:15.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.227+0000 7f556dac2740 1 -- 172.21.15.164:0/2581025251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b498e215d0 msgr2=0x55b498e22300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:15.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.227+0000 7f556dac2740 1 --2- 172.21.15.164:0/2581025251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b498e215d0 0x55b498e22300 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f5560009710 tx=0x7f55600049e0 comp rx=0 tx=0).stop 2024-09-19T08:52:15.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.228+0000 7f556dac2740 1 -- 172.21.15.164:0/2581025251 shutdown_connections 2024-09-19T08:52:15.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.228+0000 7f556dac2740 1 --2- 172.21.15.164:0/2581025251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b498e2c370 0x55b498e24800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:15.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.228+0000 7f556dac2740 1 --2- 172.21.15.164:0/2581025251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b498e22b40 0x55b498e242c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:15.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.228+0000 7f556dac2740 1 --2- 172.21.15.164:0/2581025251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b498e215d0 0x55b498e22300 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:15.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.228+0000 7f556dac2740 1 -- 172.21.15.164:0/2581025251 >> 172.21.15.164:0/2581025251 conn(0x55b498dea310 msgr2=0x55b498e18a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:15.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.228+0000 7f556dac2740 1 -- 172.21.15.164:0/2581025251 shutdown_connections 2024-09-19T08:52:15.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.228+0000 7f556dac2740 1 -- 172.21.15.164:0/2581025251 wait complete. 2024-09-19T08:52:15.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.228+0000 7f556dac2740 1 -- start start 2024-09-19T08:52:15.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.228+0000 7f556dac2740 1 Processor -- start 2024-09-19T08:52:15.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.228+0000 7f556dac2740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b498e215d0 0x55b498e2d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:15.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.228+0000 7f556dac2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b498e22b40 0x55b498e2d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:15.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.228+0000 7f556dac2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b498e2c370 0x55b498e33480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:15.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.229+0000 7f556dac2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f55641b5190 con 0x55b498e2c370 2024-09-19T08:52:15.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.229+0000 7f556dac2740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f55641b4ac0 con 0x55b498e215d0 2024-09-19T08:52:15.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.229+0000 7f556dac2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f55641b1d20 con 0x55b498e22b40 2024-09-19T08:52:15.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.229+0000 7f556dac2740 10 client.0 _ll_register_callbacks cb 0x7f55540097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:52:15.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.229+0000 7f5569c86640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b498e2c370 0x55b498e33480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:15.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.229+0000 7f5569c86640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b498e2c370 0x55b498e33480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46774/0 (socket says 172.21.15.164:46774) 2024-09-19T08:52:15.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.229+0000 7f5569c86640 1 -- 172.21.15.164:0/136822545 learned_addr learned my addr 172.21.15.164:0/136822545 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:15.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.229+0000 7f5568c84640 1 --2- 172.21.15.164:0/136822545 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b498e22b40 0x55b498e2d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:15.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.229+0000 7f5569485640 1 --2- 172.21.15.164:0/136822545 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b498e215d0 0x55b498e2d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:15.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.229+0000 7f5569c86640 1 -- 172.21.15.164:0/136822545 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b498e22b40 msgr2=0x55b498e2d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:15.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.229+0000 7f5569c86640 1 --2- 172.21.15.164:0/136822545 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b498e22b40 0x55b498e2d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:15.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.229+0000 7f5569c86640 1 -- 172.21.15.164:0/136822545 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b498e215d0 msgr2=0x55b498e2d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:15.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.229+0000 7f5569c86640 1 --2- 172.21.15.164:0/136822545 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b498e215d0 0x55b498e2d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:15.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.229+0000 7f5569c86640 1 -- 172.21.15.164:0/136822545 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b498e33ac0 con 0x55b498e2c370 2024-09-19T08:52:15.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.229+0000 7f5569485640 1 --2- 172.21.15.164:0/136822545 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b498e215d0 0x55b498e2d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:52:15.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.229+0000 7f5569c86640 1 --2- 172.21.15.164:0/136822545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b498e2c370 0x55b498e33480 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f555c00efa0 tx=0x7f555c010040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85da74a57b09767 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:15.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.229+0000 7f554cff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:52:15.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.229+0000 7f554cff9640 1 -- 172.21.15.164:0/136822545 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c023040 con 0x55b498e2c370 2024-09-19T08:52:15.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.229+0000 7f554cff9640 1 -- 172.21.15.164:0/136822545 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f555c027430 con 0x55b498e2c370 2024-09-19T08:52:15.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.229+0000 7f556dac2740 1 -- 172.21.15.164:0/136822545 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5560003d80 con 0x55b498e2c370 2024-09-19T08:52:15.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.230+0000 7f554cff9640 1 -- 172.21.15.164:0/136822545 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f555c035050 con 0x55b498e2c370 2024-09-19T08:52:15.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.230+0000 7f556dac2740 10 client.16767 fetch_fsmap learned FSMap version 31 2024-09-19T08:52:15.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.230+0000 7f556dac2740 1 -- 172.21.15.164:0/136822545 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b498e34d90 con 0x55b498e2c370 2024-09-19T08:52:15.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.230+0000 7f554cff9640 1 -- 172.21.15.164:0/136822545 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5560003d80 con 0x55b498e2c370 2024-09-19T08:52:15.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.230+0000 7f554cff9640 1 -- 172.21.15.164:0/136822545 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f555c026070 con 0x55b498e2c370 2024-09-19T08:52:15.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.230+0000 7f556dac2740 10 client.16767 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T08:52:15.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.230+0000 7f556dac2740 10 client.16767 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-19T08:52:15.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.230+0000 7f556dac2740 20 client.16767 populate_metadata read hostname 'smithi164' 2024-09-19T08:52:15.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.230+0000 7f556dac2740 1 --2- 172.21.15.164:0/136822545 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55b498ea6ff0 0x55b498ea9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:15.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.230+0000 7f556dac2740 4 client.16767 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:52:15.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.230+0000 7f556dac2740 1 -- 172.21.15.164:0/136822545 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b498d8ed70 con 0x55b498ea6ff0 2024-09-19T08:52:15.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.230+0000 7f5569485640 1 --2- 172.21.15.164:0/136822545 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55b498ea6ff0 0x55b498ea9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:15.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.231+0000 7f554cff9640 10 client.16767 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:52:15.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.231+0000 7f5569485640 1 --2- 172.21.15.164:0/136822545 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55b498ea6ff0 0x55b498ea9410 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fcd58fe55e61de52 server_cookie=89cf40dee41d365d in_seq=0 out_seq=0 2024-09-19T08:52:15.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.231+0000 7f554cff9640 1 -- 172.21.15.164:0/136822545 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b498d8ed70 con 0x55b498ea6ff0 2024-09-19T08:52:15.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.231+0000 7f554cff9640 10 client.16767 handle_command_reply: tid=0 2024-09-19T08:52:15.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.231+0000 7f556dac2740 1 client.16767 shutdown 2024-09-19T08:52:15.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.232+0000 7f556dac2740 1 -- 172.21.15.164:0/136822545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b498e2c370 msgr2=0x55b498e33480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:15.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.232+0000 7f556dac2740 1 --2- 172.21.15.164:0/136822545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b498e2c370 0x55b498e33480 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f555c00efa0 tx=0x7f555c010040 comp rx=0 tx=0).stop 2024-09-19T08:52:15.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.232+0000 7f556dac2740 1 -- 172.21.15.164:0/136822545 shutdown_connections 2024-09-19T08:52:15.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.232+0000 7f556dac2740 1 --2- 172.21.15.164:0/136822545 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55b498ea6ff0 0x55b498ea9410 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:15.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.232+0000 7f556dac2740 1 --2- 172.21.15.164:0/136822545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b498e2c370 0x55b498e33480 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:15.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.232+0000 7f556dac2740 1 --2- 172.21.15.164:0/136822545 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b498e22b40 0x55b498e2d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:15.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.232+0000 7f554cff9640 0 client.16767 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:52:15.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.232+0000 7f556dac2740 1 --2- 172.21.15.164:0/136822545 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b498e215d0 0x55b498e2d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:15.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.232+0000 7f556dac2740 1 -- 172.21.15.164:0/136822545 >> 172.21.15.164:0/136822545 conn(0x55b498dea310 msgr2=0x55b498e27900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:15.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.232+0000 7f556dac2740 1 -- 172.21.15.164:0/136822545 shutdown_connections 2024-09-19T08:52:15.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.232+0000 7f556dac2740 1 -- 172.21.15.164:0/136822545 wait complete. 2024-09-19T08:52:15.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.232+0000 7f556dac2740 20 client.16767 trim_cache size 0 max 16384 2024-09-19T08:52:15.237 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:52:15.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.232+0000 7f556b6e9640 1 -- 172.21.15.164:0/3360628867 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f55500a33b0 msgr2=0x7f55500a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:15.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.232+0000 7f556b6e9640 1 --2- 172.21.15.164:0/3360628867 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f55500a33b0 0x7f55500a5870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f555c009ba0 tx=0x7f555c004a40 comp rx=0 tx=0).stop 2024-09-19T08:52:15.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.232+0000 7f556b6e9640 1 -- 172.21.15.164:0/3360628867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5564074eb0 msgr2=0x7f55641a4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:15.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.232+0000 7f556b6e9640 1 --2- 172.21.15.164:0/3360628867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5564074eb0 0x7f55641a4c80 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f5554000c00 tx=0x7f5554032440 comp rx=0 tx=0).stop 2024-09-19T08:52:15.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.233+0000 7f556b6e9640 1 -- 172.21.15.164:0/3360628867 shutdown_connections 2024-09-19T08:52:15.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.233+0000 7f556b6e9640 1 --2- 172.21.15.164:0/3360628867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5564112bc0 0x7f556419e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:15.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.233+0000 7f556b6e9640 1 --2- 172.21.15.164:0/3360628867 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f55500a33b0 0x7f55500a5870 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:15.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.233+0000 7f556b6e9640 1 --2- 172.21.15.164:0/3360628867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5564074eb0 0x7f55641a4c80 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:15.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.233+0000 7f556b6e9640 1 --2- 172.21.15.164:0/3360628867 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5564074590 0x7f55641a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:15.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.233+0000 7f556b6e9640 1 -- 172.21.15.164:0/3360628867 >> 172.21.15.164:0/3360628867 conn(0x7f55640fd8d0 msgr2=0x7f55640ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:15.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.233+0000 7f556b6e9640 1 -- 172.21.15.164:0/3360628867 shutdown_connections 2024-09-19T08:52:15.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:15.233+0000 7f556b6e9640 1 -- 172.21.15.164:0/3360628867 wait complete. 2024-09-19T08:52:16.005 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T08:52:16.251 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:52:16.475 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:16 smithi164 ceph-mon[30928]: pgmap v198: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 306 MiB/s wr, 1.12k op/s 2024-09-19T08:52:16.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:16 smithi170 ceph-mon[35072]: pgmap v198: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 306 MiB/s wr, 1.12k op/s 2024-09-19T08:52:16.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:16 smithi173 ceph-mon[35306]: pgmap v198: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 306 MiB/s wr, 1.12k op/s 2024-09-19T08:52:16.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.703+0000 7f69d0dcd640 1 -- 172.21.15.164:0/2562710792 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69cc100a80 msgr2=0x7f69cc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:16.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.703+0000 7f69d0dcd640 1 --2- 172.21.15.164:0/2562710792 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69cc100a80 0x7f69cc100ee0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f69b4009930 tx=0x7f69b402a420 comp rx=0 tx=0).stop 2024-09-19T08:52:16.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.703+0000 7f69d0dcd640 1 -- 172.21.15.164:0/2562710792 shutdown_connections 2024-09-19T08:52:16.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.703+0000 7f69d0dcd640 1 --2- 172.21.15.164:0/2562710792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69cc101420 0x7f69cc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:16.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.703+0000 7f69d0dcd640 1 --2- 172.21.15.164:0/2562710792 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69cc100a80 0x7f69cc100ee0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:16.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.703+0000 7f69d0dcd640 1 --2- 172.21.15.164:0/2562710792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69cc106a80 0x7f69cc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:16.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.703+0000 7f69d0dcd640 1 -- 172.21.15.164:0/2562710792 >> 172.21.15.164:0/2562710792 conn(0x7f69cc0fc7a0 msgr2=0x7f69cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:16.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.703+0000 7f69d0dcd640 1 -- 172.21.15.164:0/2562710792 shutdown_connections 2024-09-19T08:52:16.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.703+0000 7f69d0dcd640 1 -- 172.21.15.164:0/2562710792 wait complete. 2024-09-19T08:52:16.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.704+0000 7f69d0dcd640 1 Processor -- start 2024-09-19T08:52:16.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.704+0000 7f69d0dcd640 1 -- start start 2024-09-19T08:52:16.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.704+0000 7f69d0dcd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69cc100a80 0x7f69cc1970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:16.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.704+0000 7f69d0dcd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69cc101420 0x7f69cc197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:16.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.704+0000 7f69d0dcd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69cc106a80 0x7f69cc197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:16.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.704+0000 7f69d0dcd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc110e80 con 0x7f69cc106a80 2024-09-19T08:52:16.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.704+0000 7f69d0dcd640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc110d00 con 0x7f69cc101420 2024-09-19T08:52:16.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.704+0000 7f69d0dcd640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc111000 con 0x7f69cc100a80 2024-09-19T08:52:16.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.704+0000 7f69cad9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69cc106a80 0x7f69cc197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:16.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.704+0000 7f69cad9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69cc106a80 0x7f69cc197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46796/0 (socket says 172.21.15.164:46796) 2024-09-19T08:52:16.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.704+0000 7f69cad9d640 1 -- 172.21.15.164:0/794977479 learned_addr learned my addr 172.21.15.164:0/794977479 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:16.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.704+0000 7f69c3fff640 1 --2- 172.21.15.164:0/794977479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69cc101420 0x7f69cc197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:16.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.704+0000 7f69ca59c640 1 --2- 172.21.15.164:0/794977479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69cc100a80 0x7f69cc1970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:16.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.705+0000 7f69cad9d640 1 -- 172.21.15.164:0/794977479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69cc100a80 msgr2=0x7f69cc1970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:16.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.705+0000 7f69cad9d640 1 --2- 172.21.15.164:0/794977479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69cc100a80 0x7f69cc1970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:16.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.705+0000 7f69cad9d640 1 -- 172.21.15.164:0/794977479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69cc101420 msgr2=0x7f69cc197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:16.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.705+0000 7f69cad9d640 1 --2- 172.21.15.164:0/794977479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69cc101420 0x7f69cc197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:16.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.705+0000 7f69cad9d640 1 -- 172.21.15.164:0/794977479 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69cc1a6950 con 0x7f69cc106a80 2024-09-19T08:52:16.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.705+0000 7f69c3fff640 1 --2- 172.21.15.164:0/794977479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69cc101420 0x7f69cc197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:52:16.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.705+0000 7f69ca59c640 1 --2- 172.21.15.164:0/794977479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69cc100a80 0x7f69cc1970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:52:16.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.705+0000 7f69cad9d640 1 --2- 172.21.15.164:0/794977479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69cc106a80 0x7f69cc197b60 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f69c400eb10 tx=0x7f69c400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:16.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.705+0000 7f69c37fe640 1 -- 172.21.15.164:0/794977479 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c4004b20 con 0x7f69cc106a80 2024-09-19T08:52:16.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.705+0000 7f69d0dcd640 1 -- 172.21.15.164:0/794977479 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69cc1a6c40 con 0x7f69cc106a80 2024-09-19T08:52:16.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.705+0000 7f69c37fe640 1 -- 172.21.15.164:0/794977479 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69c4004cc0 con 0x7f69cc106a80 2024-09-19T08:52:16.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.705+0000 7f69c37fe640 1 -- 172.21.15.164:0/794977479 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c4010760 con 0x7f69cc106a80 2024-09-19T08:52:16.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.705+0000 7f69d0dcd640 1 -- 172.21.15.164:0/794977479 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69cc1a7150 con 0x7f69cc106a80 2024-09-19T08:52:16.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.707+0000 7f69c37fe640 1 -- 172.21.15.164:0/794977479 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f69c4005020 con 0x7f69cc106a80 2024-09-19T08:52:16.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.707+0000 7f69d31a6740 1 Processor -- start 2024-09-19T08:52:16.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.708+0000 7f69d31a6740 1 -- start start 2024-09-19T08:52:16.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.708+0000 7f69d31a6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e35d89ab40 0x55e35d899090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:16.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.708+0000 7f69d31a6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e35d8995d0 0x55e35d899a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:16.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.708+0000 7f69d31a6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e35d8a4370 0x55e35d8a6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:16.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.708+0000 7f69d31a6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e35d7ff2f0 con 0x55e35d8995d0 2024-09-19T08:52:16.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.708+0000 7f69d31a6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e35d7a6cf0 con 0x55e35d89ab40 2024-09-19T08:52:16.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.708+0000 7f69d31a6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e35d7fb2b0 con 0x55e35d8a4370 2024-09-19T08:52:16.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.708+0000 7f69ca59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e35d8995d0 0x55e35d899a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:16.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.708+0000 7f69ca59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e35d8995d0 0x55e35d899a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46804/0 (socket says 172.21.15.164:46804) 2024-09-19T08:52:16.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.708+0000 7f69ca59c640 1 -- 172.21.15.164:0/664053916 learned_addr learned my addr 172.21.15.164:0/664053916 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:16.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.708+0000 7f69c3fff640 1 --2- 172.21.15.164:0/664053916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e35d89ab40 0x55e35d899090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:16.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.708+0000 7f69c37fe640 1 --2- 172.21.15.164:0/794977479 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f69b80a3430 0x7f69b80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:16.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.708+0000 7f69c3fff640 1 --2- 172.21.15.164:0/664053916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e35d8a4370 0x55e35d8a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:16.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.708+0000 7f69ca59c640 1 -- 172.21.15.164:0/664053916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e35d8a4370 msgr2=0x55e35d8a6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:16.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.708+0000 7f69ca59c640 1 --2- 172.21.15.164:0/664053916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e35d8a4370 0x55e35d8a6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:16.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.708+0000 7f69ca59c640 1 -- 172.21.15.164:0/664053916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e35d89ab40 msgr2=0x55e35d899090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:16.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.708+0000 7f69ca59c640 1 --2- 172.21.15.164:0/664053916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e35d89ab40 0x55e35d899090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:16.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.708+0000 7f69ca59c640 1 -- 172.21.15.164:0/664053916 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e35d78b3a0 con 0x55e35d8995d0 2024-09-19T08:52:16.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.708+0000 7f69c3fff640 1 --2- 172.21.15.164:0/664053916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e35d8a4370 0x55e35d8a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:52:16.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.709+0000 7f69cad9d640 1 --2- 172.21.15.164:0/794977479 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f69b80a3430 0x7f69b80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:16.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.709+0000 7f69c37fe640 1 -- 172.21.15.164:0/794977479 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f69c40d0420 con 0x7f69cc106a80 2024-09-19T08:52:16.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.709+0000 7f69ca59c640 1 --2- 172.21.15.164:0/664053916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e35d8995d0 0x55e35d899a10 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f69bc00cbf0 tx=0x7f69bc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585db198fcf80277 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:16.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.709+0000 7f69c0ff9640 1 -- 172.21.15.164:0/664053916 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc005220 con 0x55e35d8995d0 2024-09-19T08:52:16.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.709+0000 7f69c0ff9640 1 -- 172.21.15.164:0/664053916 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69bc016070 con 0x55e35d8995d0 2024-09-19T08:52:16.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.709+0000 7f69c0ff9640 1 -- 172.21.15.164:0/664053916 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc0106c0 con 0x55e35d8995d0 2024-09-19T08:52:16.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.710+0000 7f69d31a6740 1 -- 172.21.15.164:0/664053916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e35d8995d0 msgr2=0x55e35d899a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:16.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.710+0000 7f69cad9d640 1 --2- 172.21.15.164:0/794977479 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f69b80a3430 0x7f69b80a58f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f69c4098b40 tx=0x7f69c4091b70 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:16.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.710+0000 7f69d31a6740 1 --2- 172.21.15.164:0/664053916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e35d8995d0 0x55e35d899a10 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f69bc00cbf0 tx=0x7f69bc00e720 comp rx=0 tx=0).stop 2024-09-19T08:52:16.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.710+0000 7f69d31a6740 1 -- 172.21.15.164:0/664053916 shutdown_connections 2024-09-19T08:52:16.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.710+0000 7f69d31a6740 1 --2- 172.21.15.164:0/664053916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e35d8a4370 0x55e35d8a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:16.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.710+0000 7f69d31a6740 1 --2- 172.21.15.164:0/664053916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e35d8995d0 0x55e35d899a10 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:16.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.710+0000 7f69d31a6740 1 --2- 172.21.15.164:0/664053916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e35d89ab40 0x55e35d899090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:16.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.710+0000 7f69d31a6740 1 -- 172.21.15.164:0/664053916 >> 172.21.15.164:0/664053916 conn(0x55e35d862310 msgr2=0x55e35d895c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:16.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.710+0000 7f69d31a6740 1 -- 172.21.15.164:0/664053916 shutdown_connections 2024-09-19T08:52:16.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.710+0000 7f69d31a6740 1 -- 172.21.15.164:0/664053916 wait complete. 2024-09-19T08:52:16.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.710+0000 7f69d31a6740 1 -- start start 2024-09-19T08:52:16.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.710+0000 7f69d31a6740 1 Processor -- start 2024-09-19T08:52:16.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.710+0000 7f69d31a6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e35d8995d0 0x55e35d8a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:16.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.710+0000 7f69d31a6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e35d89ab40 0x55e35d8a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:16.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.710+0000 7f69d31a6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e35d8a4370 0x55e35d8a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:16.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.710+0000 7f69d31a6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e35d8a7070 con 0x55e35d8995d0 2024-09-19T08:52:16.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.710+0000 7f69d31a6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e35d7ba060 con 0x55e35d89ab40 2024-09-19T08:52:16.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.710+0000 7f69d31a6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e35d8a71f0 con 0x55e35d8a4370 2024-09-19T08:52:16.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.711+0000 7f69d31a6740 10 client.0 _ll_register_callbacks cb 0x55e35d50d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:52:16.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.711+0000 7f69c3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e35d8995d0 0x55e35d8a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:16.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.711+0000 7f69c3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e35d8995d0 0x55e35d8a3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46812/0 (socket says 172.21.15.164:46812) 2024-09-19T08:52:16.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.711+0000 7f69c3fff640 1 -- 172.21.15.164:0/2412588901 learned_addr learned my addr 172.21.15.164:0/2412588901 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:16.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.711+0000 7f69ca59c640 1 --2- 172.21.15.164:0/2412588901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e35d89ab40 0x55e35d8a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:16.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.711+0000 7f69c3fff640 1 -- 172.21.15.164:0/2412588901 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e35d8a4370 msgr2=0x55e35d8a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:16.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.711+0000 7f69c3fff640 1 --2- 172.21.15.164:0/2412588901 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e35d8a4370 0x55e35d8a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:16.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.711+0000 7f69c3fff640 1 -- 172.21.15.164:0/2412588901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e35d89ab40 msgr2=0x55e35d8a0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:16.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.711+0000 7f69c3fff640 1 --2- 172.21.15.164:0/2412588901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e35d89ab40 0x55e35d8a0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:16.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.711+0000 7f69c3fff640 1 -- 172.21.15.164:0/2412588901 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e35d5703c0 con 0x55e35d8995d0 2024-09-19T08:52:16.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.711+0000 7f69c3fff640 1 --2- 172.21.15.164:0/2412588901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e35d8995d0 0x55e35d8a3cc0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f69b402a940 tx=0x7f69b4031a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab35d9d40c26e22 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:16.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.711+0000 7f69b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:52:16.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.711+0000 7f69b1ffb640 1 -- 172.21.15.164:0/2412588901 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b4035040 con 0x55e35d8995d0 2024-09-19T08:52:16.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.711+0000 7f69b1ffb640 1 -- 172.21.15.164:0/2412588901 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f69b4004250 con 0x55e35d8995d0 2024-09-19T08:52:16.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.711+0000 7f69d31a6740 1 -- 172.21.15.164:0/2412588901 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69bc016070 con 0x55e35d8995d0 2024-09-19T08:52:16.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.711+0000 7f69b1ffb640 1 -- 172.21.15.164:0/2412588901 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f69b4004450 con 0x55e35d8995d0 2024-09-19T08:52:16.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.711+0000 7f69d31a6740 10 client.16779 fetch_fsmap learned FSMap version 31 2024-09-19T08:52:16.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.711+0000 7f69d31a6740 1 -- 172.21.15.164:0/2412588901 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e35d8b29f0 con 0x55e35d8995d0 2024-09-19T08:52:16.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.712+0000 7f69b1ffb640 1 -- 172.21.15.164:0/2412588901 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69bc016070 con 0x55e35d8995d0 2024-09-19T08:52:16.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.712+0000 7f69b1ffb640 1 -- 172.21.15.164:0/2412588901 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f69b403a040 con 0x55e35d8995d0 2024-09-19T08:52:16.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.712+0000 7f69d31a6740 10 client.16779 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T08:52:16.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.712+0000 7f69d31a6740 10 client.16779 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T08:52:16.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.712+0000 7f69d31a6740 20 client.16779 populate_metadata read hostname 'smithi164' 2024-09-19T08:52:16.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.712+0000 7f69d31a6740 1 --2- 172.21.15.164:0/2412588901 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55e35d8b3e00 0x55e35d8b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:16.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.712+0000 7f69d31a6740 4 client.16779 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:52:16.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.712+0000 7f69d31a6740 1 -- 172.21.15.164:0/2412588901 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e35d80c860 con 0x55e35d8b3e00 2024-09-19T08:52:16.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.712+0000 7f69ca59c640 1 --2- 172.21.15.164:0/2412588901 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55e35d8b3e00 0x55e35d8b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:16.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.713+0000 7f69b1ffb640 10 client.16779 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:52:16.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.713+0000 7f69ca59c640 1 --2- 172.21.15.164:0/2412588901 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55e35d8b3e00 0x55e35d8b62a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b351c2da2c5aa624 server_cookie=308215c816c058de in_seq=0 out_seq=0 2024-09-19T08:52:16.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.713+0000 7f69b1ffb640 1 -- 172.21.15.164:0/2412588901 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e35d80c860 con 0x55e35d8b3e00 2024-09-19T08:52:16.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.713+0000 7f69b1ffb640 10 client.16779 handle_command_reply: tid=0 2024-09-19T08:52:16.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.713+0000 7f69d31a6740 1 client.16779 shutdown 2024-09-19T08:52:16.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.713+0000 7f69d31a6740 1 -- 172.21.15.164:0/2412588901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e35d8995d0 msgr2=0x55e35d8a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:16.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.713+0000 7f69d31a6740 1 --2- 172.21.15.164:0/2412588901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e35d8995d0 0x55e35d8a3cc0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f69b402a940 tx=0x7f69b4031a40 comp rx=0 tx=0).stop 2024-09-19T08:52:16.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.713+0000 7f69d31a6740 1 -- 172.21.15.164:0/2412588901 shutdown_connections 2024-09-19T08:52:16.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.713+0000 7f69d31a6740 1 --2- 172.21.15.164:0/2412588901 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55e35d8b3e00 0x55e35d8b62a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:16.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.713+0000 7f69d31a6740 1 --2- 172.21.15.164:0/2412588901 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e35d8a4370 0x55e35d8a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:16.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.713+0000 7f69d31a6740 1 --2- 172.21.15.164:0/2412588901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e35d89ab40 0x55e35d8a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:16.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.713+0000 7f69b1ffb640 0 client.16779 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:52:16.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.713+0000 7f69d31a6740 1 --2- 172.21.15.164:0/2412588901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e35d8995d0 0x55e35d8a3cc0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:16.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.713+0000 7f69d31a6740 1 -- 172.21.15.164:0/2412588901 >> 172.21.15.164:0/2412588901 conn(0x55e35d862310 msgr2=0x55e35d89b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:16.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.713+0000 7f69d31a6740 1 -- 172.21.15.164:0/2412588901 shutdown_connections 2024-09-19T08:52:16.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.714+0000 7f69d31a6740 1 -- 172.21.15.164:0/2412588901 wait complete. 2024-09-19T08:52:16.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.714+0000 7f69d31a6740 20 client.16779 trim_cache size 0 max 16384 2024-09-19T08:52:16.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.726+0000 7f69d31a6740 1 Processor -- start 2024-09-19T08:52:16.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.726+0000 7f69d31a6740 1 -- start start 2024-09-19T08:52:16.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.727+0000 7f69d31a6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e35d8995d0 0x55e35d895e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:16.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.727+0000 7f69d31a6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e35d89ab40 0x55e35d8b0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:16.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.727+0000 7f69d31a6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e35d8a4370 0x55e35d8bbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:16.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.727+0000 7f69d31a6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc1af950 con 0x55e35d89ab40 2024-09-19T08:52:16.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.727+0000 7f69d31a6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc1b14c0 con 0x55e35d8a4370 2024-09-19T08:52:16.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.727+0000 7f69d31a6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc1b0020 con 0x55e35d8995d0 2024-09-19T08:52:16.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.727+0000 7f69ca59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e35d89ab40 0x55e35d8b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:16.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.727+0000 7f69ca59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e35d89ab40 0x55e35d8b0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46828/0 (socket says 172.21.15.164:46828) 2024-09-19T08:52:16.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.727+0000 7f69ca59c640 1 -- 172.21.15.164:0/1695953768 learned_addr learned my addr 172.21.15.164:0/1695953768 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:16.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.727+0000 7f69c3fff640 1 --2- 172.21.15.164:0/1695953768 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e35d8995d0 0x55e35d895e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:16.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.727+0000 7f69c3fff640 1 --2- 172.21.15.164:0/1695953768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e35d8a4370 0x55e35d8bbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:16.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.727+0000 7f69ca59c640 1 -- 172.21.15.164:0/1695953768 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e35d8995d0 msgr2=0x55e35d895e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:16.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.727+0000 7f69ca59c640 1 --2- 172.21.15.164:0/1695953768 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e35d8995d0 0x55e35d895e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:16.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.727+0000 7f69ca59c640 1 -- 172.21.15.164:0/1695953768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e35d8a4370 msgr2=0x55e35d8bbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:16.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.727+0000 7f69ca59c640 1 --2- 172.21.15.164:0/1695953768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e35d8a4370 0x55e35d8bbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:16.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.727+0000 7f69ca59c640 1 -- 172.21.15.164:0/1695953768 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e35d8992d0 con 0x55e35d89ab40 2024-09-19T08:52:16.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.727+0000 7f69c3fff640 1 --2- 172.21.15.164:0/1695953768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e35d8a4370 0x55e35d8bbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:52:16.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.728+0000 7f69ca59c640 1 --2- 172.21.15.164:0/1695953768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e35d89ab40 0x55e35d8b0310 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f69bc005720 tx=0x7f69bc00c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e94af3714a40f319 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:16.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.728+0000 7f69b3fff640 1 -- 172.21.15.164:0/1695953768 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc005300 con 0x55e35d89ab40 2024-09-19T08:52:16.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.728+0000 7f69b3fff640 1 -- 172.21.15.164:0/1695953768 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69bc016040 con 0x55e35d89ab40 2024-09-19T08:52:16.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.728+0000 7f69b3fff640 1 -- 172.21.15.164:0/1695953768 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc0256a0 con 0x55e35d89ab40 2024-09-19T08:52:16.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.728+0000 7f69d31a6740 1 -- 172.21.15.164:0/1695953768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e35d89ab40 msgr2=0x55e35d8b0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:16.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.728+0000 7f69d31a6740 1 --2- 172.21.15.164:0/1695953768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e35d89ab40 0x55e35d8b0310 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f69bc005720 tx=0x7f69bc00c510 comp rx=0 tx=0).stop 2024-09-19T08:52:16.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.728+0000 7f69d31a6740 1 -- 172.21.15.164:0/1695953768 shutdown_connections 2024-09-19T08:52:16.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.728+0000 7f69d31a6740 1 --2- 172.21.15.164:0/1695953768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e35d8a4370 0x55e35d8bbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:16.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.728+0000 7f69d31a6740 1 --2- 172.21.15.164:0/1695953768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e35d89ab40 0x55e35d8b0310 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:16.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.728+0000 7f69d31a6740 1 --2- 172.21.15.164:0/1695953768 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e35d8995d0 0x55e35d895e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:16.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.728+0000 7f69d31a6740 1 -- 172.21.15.164:0/1695953768 >> 172.21.15.164:0/1695953768 conn(0x55e35d862310 msgr2=0x55e35d895700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:16.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.729+0000 7f69d31a6740 1 -- 172.21.15.164:0/1695953768 shutdown_connections 2024-09-19T08:52:16.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.729+0000 7f69d31a6740 1 -- 172.21.15.164:0/1695953768 wait complete. 2024-09-19T08:52:16.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.729+0000 7f69d31a6740 1 -- start start 2024-09-19T08:52:16.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.729+0000 7f69d31a6740 1 Processor -- start 2024-09-19T08:52:16.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.729+0000 7f69d31a6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e35d8995d0 0x55e35d8a83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:16.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.729+0000 7f69d31a6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e35d89ab40 0x55e35d8a8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:16.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.729+0000 7f69d31a6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e35d8a4370 0x55e35d8a8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:16.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.729+0000 7f69d31a6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc1b0dc0 con 0x55e35d8995d0 2024-09-19T08:52:16.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.729+0000 7f69d31a6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc1b06f0 con 0x55e35d89ab40 2024-09-19T08:52:16.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.729+0000 7f69d31a6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc1ad950 con 0x55e35d8a4370 2024-09-19T08:52:16.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.729+0000 7f69d31a6740 10 client.0 _ll_register_callbacks cb 0x7f69b4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:52:16.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.729+0000 7f69c3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e35d8995d0 0x55e35d8a83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:16.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.729+0000 7f69c3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e35d8995d0 0x55e35d8a83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46834/0 (socket says 172.21.15.164:46834) 2024-09-19T08:52:16.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.729+0000 7f69c3fff640 1 -- 172.21.15.164:0/1688587848 learned_addr learned my addr 172.21.15.164:0/1688587848 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:16.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.729+0000 7f69ca59c640 1 --2- 172.21.15.164:0/1688587848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e35d89ab40 0x55e35d8a8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:16.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.730+0000 7f69c3fff640 1 --2- 172.21.15.164:0/1688587848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e35d8a4370 0x55e35d8a8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:16.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.730+0000 7f69c3fff640 1 -- 172.21.15.164:0/1688587848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e35d8a4370 msgr2=0x55e35d8a8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:16.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.730+0000 7f69c3fff640 1 --2- 172.21.15.164:0/1688587848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e35d8a4370 0x55e35d8a8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:16.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.730+0000 7f69c3fff640 1 -- 172.21.15.164:0/1688587848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e35d89ab40 msgr2=0x55e35d8a8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:16.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.730+0000 7f69c3fff640 1 --2- 172.21.15.164:0/1688587848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e35d89ab40 0x55e35d8a8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:16.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.730+0000 7f69c3fff640 1 -- 172.21.15.164:0/1688587848 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e35d89cb70 con 0x55e35d8995d0 2024-09-19T08:52:16.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.730+0000 7f69c3fff640 1 --2- 172.21.15.164:0/1688587848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e35d8995d0 0x55e35d8a83f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f69b4031ba0 tx=0x7f69b4009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14fc15a310248cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:16.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.730+0000 7f69b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:52:16.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.730+0000 7f69b1ffb640 1 -- 172.21.15.164:0/1688587848 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b4035030 con 0x55e35d8995d0 2024-09-19T08:52:16.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.730+0000 7f69b1ffb640 1 -- 172.21.15.164:0/1688587848 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f69b4005300 con 0x55e35d8995d0 2024-09-19T08:52:16.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.730+0000 7f69d31a6740 1 -- 172.21.15.164:0/1688587848 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69bc016040 con 0x55e35d8995d0 2024-09-19T08:52:16.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.730+0000 7f69b1ffb640 1 -- 172.21.15.164:0/1688587848 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f69b4053040 con 0x55e35d8995d0 2024-09-19T08:52:16.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.730+0000 7f69d31a6740 10 client.16785 fetch_fsmap learned FSMap version 31 2024-09-19T08:52:16.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.730+0000 7f69d31a6740 1 -- 172.21.15.164:0/1688587848 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e35d91ee50 con 0x55e35d8995d0 2024-09-19T08:52:16.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.731+0000 7f69b1ffb640 1 -- 172.21.15.164:0/1688587848 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69bc016040 con 0x55e35d8995d0 2024-09-19T08:52:16.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.731+0000 7f69b1ffb640 1 -- 172.21.15.164:0/1688587848 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f69b4056020 con 0x55e35d8995d0 2024-09-19T08:52:16.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.731+0000 7f69d31a6740 10 client.16785 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T08:52:16.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.731+0000 7f69d31a6740 10 client.16785 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T08:52:16.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.731+0000 7f69d31a6740 20 client.16785 populate_metadata read hostname 'smithi164' 2024-09-19T08:52:16.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.731+0000 7f69d31a6740 1 --2- 172.21.15.164:0/1688587848 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55e35d91f550 0x55e35d921970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:16.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.731+0000 7f69d31a6740 4 client.16785 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:52:16.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.731+0000 7f69d31a6740 1 -- 172.21.15.164:0/1688587848 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e35d806d70 con 0x55e35d91f550 2024-09-19T08:52:16.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.731+0000 7f69ca59c640 1 --2- 172.21.15.164:0/1688587848 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55e35d91f550 0x55e35d921970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:16.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.731+0000 7f69b1ffb640 10 client.16785 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:52:16.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.731+0000 7f69ca59c640 1 --2- 172.21.15.164:0/1688587848 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55e35d91f550 0x55e35d921970 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b3bd0114ac3292fb server_cookie=dc3daf2d3a7c5e0e in_seq=0 out_seq=0 2024-09-19T08:52:16.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.732+0000 7f69b1ffb640 1 -- 172.21.15.164:0/1688587848 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e35d806d70 con 0x55e35d91f550 2024-09-19T08:52:16.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.732+0000 7f69b1ffb640 10 client.16785 handle_command_reply: tid=0 2024-09-19T08:52:16.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.732+0000 7f69d31a6740 1 client.16785 shutdown 2024-09-19T08:52:16.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.732+0000 7f69d31a6740 1 -- 172.21.15.164:0/1688587848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e35d8995d0 msgr2=0x55e35d8a83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:16.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.732+0000 7f69d31a6740 1 --2- 172.21.15.164:0/1688587848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e35d8995d0 0x55e35d8a83f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f69b4031ba0 tx=0x7f69b4009290 comp rx=0 tx=0).stop 2024-09-19T08:52:16.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.732+0000 7f69d31a6740 1 -- 172.21.15.164:0/1688587848 shutdown_connections 2024-09-19T08:52:16.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.732+0000 7f69d31a6740 1 --2- 172.21.15.164:0/1688587848 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55e35d91f550 0x55e35d921970 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:16.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.732+0000 7f69b1ffb640 0 client.16785 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:52:16.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.732+0000 7f69d31a6740 1 --2- 172.21.15.164:0/1688587848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e35d8a4370 0x55e35d8a8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:16.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.732+0000 7f69d31a6740 1 --2- 172.21.15.164:0/1688587848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e35d89ab40 0x55e35d8a8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:16.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.732+0000 7f69d31a6740 1 --2- 172.21.15.164:0/1688587848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e35d8995d0 0x55e35d8a83f0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:16.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.732+0000 7f69d31a6740 1 -- 172.21.15.164:0/1688587848 >> 172.21.15.164:0/1688587848 conn(0x55e35d862310 msgr2=0x55e35d89b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:16.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.732+0000 7f69d31a6740 1 -- 172.21.15.164:0/1688587848 shutdown_connections 2024-09-19T08:52:16.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.732+0000 7f69d31a6740 1 -- 172.21.15.164:0/1688587848 wait complete. 2024-09-19T08:52:16.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.732+0000 7f69d31a6740 20 client.16785 trim_cache size 0 max 16384 2024-09-19T08:52:16.743 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:52:16.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.733+0000 7f69d0dcd640 1 -- 172.21.15.164:0/794977479 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f69b80a3430 msgr2=0x7f69b80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:16.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.733+0000 7f69d0dcd640 1 --2- 172.21.15.164:0/794977479 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f69b80a3430 0x7f69b80a58f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f69c4098b40 tx=0x7f69c4091b70 comp rx=0 tx=0).stop 2024-09-19T08:52:16.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.733+0000 7f69d0dcd640 1 -- 172.21.15.164:0/794977479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69cc106a80 msgr2=0x7f69cc197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:16.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.733+0000 7f69d0dcd640 1 --2- 172.21.15.164:0/794977479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69cc106a80 0x7f69cc197b60 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f69c400eb10 tx=0x7f69c400c2d0 comp rx=0 tx=0).stop 2024-09-19T08:52:16.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.733+0000 7f69d0dcd640 1 -- 172.21.15.164:0/794977479 shutdown_connections 2024-09-19T08:52:16.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.733+0000 7f69d0dcd640 1 --2- 172.21.15.164:0/794977479 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f69b80a3430 0x7f69b80a58f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:16.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.733+0000 7f69d0dcd640 1 --2- 172.21.15.164:0/794977479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69cc106a80 0x7f69cc197b60 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:16.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.733+0000 7f69d0dcd640 1 --2- 172.21.15.164:0/794977479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69cc101420 0x7f69cc197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:16.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.733+0000 7f69d0dcd640 1 --2- 172.21.15.164:0/794977479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69cc100a80 0x7f69cc1970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:16.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.733+0000 7f69d0dcd640 1 -- 172.21.15.164:0/794977479 >> 172.21.15.164:0/794977479 conn(0x7f69cc0fc7a0 msgr2=0x7f69cc0fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:16.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.733+0000 7f69d0dcd640 1 -- 172.21.15.164:0/794977479 shutdown_connections 2024-09-19T08:52:16.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:16.733+0000 7f69d0dcd640 1 -- 172.21.15.164:0/794977479 wait complete. 2024-09-19T08:52:17.388 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T08:52:17.634 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:52:18.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.138+0000 7f2d9a2c0640 1 -- 172.21.15.164:0/3211930678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d94106f20 msgr2=0x7f2d9410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:18.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.138+0000 7f2d9a2c0640 1 --2- 172.21.15.164:0/3211930678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d94106f20 0x7f2d9410eaa0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c009ca0 tx=0x7f2d8c02a8c0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.138+0000 7f2d9a2c0640 1 -- 172.21.15.164:0/3211930678 shutdown_connections 2024-09-19T08:52:18.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.138+0000 7f2d9a2c0640 1 --2- 172.21.15.164:0/3211930678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d94106f20 0x7f2d9410eaa0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.138+0000 7f2d9a2c0640 1 --2- 172.21.15.164:0/3211930678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d941027a0 0x7f2d941067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.138+0000 7f2d9a2c0640 1 --2- 172.21.15.164:0/3211930678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d94101df0 0x7f2d941021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.138+0000 7f2d9a2c0640 1 -- 172.21.15.164:0/3211930678 >> 172.21.15.164:0/3211930678 conn(0x7f2d940fd8b0 msgr2=0x7f2d940ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:18.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.138+0000 7f2d9a2c0640 1 -- 172.21.15.164:0/3211930678 shutdown_connections 2024-09-19T08:52:18.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.138+0000 7f2d9a2c0640 1 -- 172.21.15.164:0/3211930678 wait complete. 2024-09-19T08:52:18.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.139+0000 7f2d9a2c0640 1 Processor -- start 2024-09-19T08:52:18.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.139+0000 7f2d9a2c0640 1 -- start start 2024-09-19T08:52:18.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.139+0000 7f2d9a2c0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d94101df0 0x7f2d940744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:18.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.139+0000 7f2d9a2c0640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d941027a0 0x7f2d94074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:18.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.139+0000 7f2d9a2c0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d94106f20 0x7f2d94074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:18.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.139+0000 7f2d9a2c0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2d94111f80 con 0x7f2d94101df0 2024-09-19T08:52:18.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.139+0000 7f2d9a2c0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2d94111e00 con 0x7f2d94106f20 2024-09-19T08:52:18.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.139+0000 7f2d9a2c0640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2d94112100 con 0x7f2d941027a0 2024-09-19T08:52:18.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.139+0000 7f2d93fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d94101df0 0x7f2d940744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:18.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.139+0000 7f2d93fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d94101df0 0x7f2d940744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46856/0 (socket says 172.21.15.164:46856) 2024-09-19T08:52:18.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.139+0000 7f2d93fff640 1 -- 172.21.15.164:0/2032349973 learned_addr learned my addr 172.21.15.164:0/2032349973 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:18.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.140+0000 7f2d9885d640 1 --2- 172.21.15.164:0/2032349973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d94106f20 0x7f2d94074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:18.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.140+0000 7f2d937fe640 1 --2- 172.21.15.164:0/2032349973 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d941027a0 0x7f2d94074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:18.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.140+0000 7f2d93fff640 1 -- 172.21.15.164:0/2032349973 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d941027a0 msgr2=0x7f2d94074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:18.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.140+0000 7f2d93fff640 1 --2- 172.21.15.164:0/2032349973 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d941027a0 0x7f2d94074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.140+0000 7f2d93fff640 1 -- 172.21.15.164:0/2032349973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d94106f20 msgr2=0x7f2d94074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:18.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.140+0000 7f2d93fff640 1 --2- 172.21.15.164:0/2032349973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d94106f20 0x7f2d94074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.140+0000 7f2d93fff640 1 -- 172.21.15.164:0/2032349973 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d941a68e0 con 0x7f2d94101df0 2024-09-19T08:52:18.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.140+0000 7f2d937fe640 1 --2- 172.21.15.164:0/2032349973 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d941027a0 0x7f2d94074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:52:18.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.140+0000 7f2d93fff640 1 --2- 172.21.15.164:0/2032349973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d94101df0 0x7f2d940744f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f2d8400d950 tx=0x7f2d8400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:18.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.140+0000 7f2d917fa640 1 -- 172.21.15.164:0/2032349973 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d84017070 con 0x7f2d94101df0 2024-09-19T08:52:18.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.140+0000 7f2d9a2c0640 1 -- 172.21.15.164:0/2032349973 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d941a6bd0 con 0x7f2d94101df0 2024-09-19T08:52:18.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.140+0000 7f2d917fa640 1 -- 172.21.15.164:0/2032349973 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d84009c30 con 0x7f2d94101df0 2024-09-19T08:52:18.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.140+0000 7f2d917fa640 1 -- 172.21.15.164:0/2032349973 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8400f6b0 con 0x7f2d94101df0 2024-09-19T08:52:18.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.140+0000 7f2d9a2c0640 1 -- 172.21.15.164:0/2032349973 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d941a7110 con 0x7f2d94101df0 2024-09-19T08:52:18.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.143+0000 7f2d917fa640 1 -- 172.21.15.164:0/2032349973 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f2d840109d0 con 0x7f2d94101df0 2024-09-19T08:52:18.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.143+0000 7f2d9c699740 1 Processor -- start 2024-09-19T08:52:18.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.143+0000 7f2d9c699740 1 -- start start 2024-09-19T08:52:18.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.144+0000 7f2d9c699740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b356b00b40 0x55b356aff090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:18.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.144+0000 7f2d9c699740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b356aff5d0 0x55b356affa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:18.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.144+0000 7f2d9c699740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b356b0a370 0x55b356b0c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:18.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.144+0000 7f2d9c699740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b356a652f0 con 0x55b356b0a370 2024-09-19T08:52:18.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.144+0000 7f2d9c699740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b356a0ccf0 con 0x55b356b00b40 2024-09-19T08:52:18.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.144+0000 7f2d9c699740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b356a612b0 con 0x55b356aff5d0 2024-09-19T08:52:18.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.144+0000 7f2d937fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b356b0a370 0x55b356b0c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:18.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.144+0000 7f2d937fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b356b0a370 0x55b356b0c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46864/0 (socket says 172.21.15.164:46864) 2024-09-19T08:52:18.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.144+0000 7f2d9885d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b356b00b40 0x55b356aff090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:18.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.144+0000 7f2d937fe640 1 -- 172.21.15.164:0/428640224 learned_addr learned my addr 172.21.15.164:0/428640224 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:18.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.144+0000 7f2d93fff640 1 --2- 172.21.15.164:0/428640224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b356aff5d0 0x55b356affa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:18.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.144+0000 7f2d917fa640 1 --2- 172.21.15.164:0/2032349973 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f2d7c0a33b0 0x7f2d7c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:18.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.144+0000 7f2d9885d640 1 -- 172.21.15.164:0/428640224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b356aff5d0 msgr2=0x55b356affa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:18.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.144+0000 7f2d9885d640 1 --2- 172.21.15.164:0/428640224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b356aff5d0 0x55b356affa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.144+0000 7f2d9885d640 1 -- 172.21.15.164:0/428640224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b356b0a370 msgr2=0x55b356b0c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:18.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.144+0000 7f2d9885d640 1 --2- 172.21.15.164:0/428640224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b356b0a370 0x55b356b0c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.144+0000 7f2d9885d640 1 -- 172.21.15.164:0/428640224 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3569f13a0 con 0x55b356b00b40 2024-09-19T08:52:18.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.144+0000 7f2d917fa640 1 -- 172.21.15.164:0/2032349973 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f2d840cf930 con 0x7f2d94101df0 2024-09-19T08:52:18.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.144+0000 7f2d9885d640 1 --2- 172.21.15.164:0/2032349973 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f2d7c0a33b0 0x7f2d7c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:18.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.144+0000 7f2d9885d640 1 --2- 172.21.15.164:0/428640224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b356b00b40 0x55b356aff090 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c006010 tx=0x7f2d8c0099d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=526e6039cf26d9eb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:18.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.144+0000 7f2d9885d640 1 --2- 172.21.15.164:0/2032349973 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f2d7c0a33b0 0x7f2d7c0a5870 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x55b356b01170 tx=0x7f2d8c04e000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:18.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.144+0000 7f2d7a7fc640 1 -- 172.21.15.164:0/428640224 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c042040 con 0x55b356b00b40 2024-09-19T08:52:18.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.145+0000 7f2d7a7fc640 1 -- 172.21.15.164:0/428640224 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d8c036040 con 0x55b356b00b40 2024-09-19T08:52:18.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.145+0000 7f2d7a7fc640 1 -- 172.21.15.164:0/428640224 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c02fba0 con 0x55b356b00b40 2024-09-19T08:52:18.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.145+0000 7f2d9c699740 1 -- 172.21.15.164:0/428640224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b356b00b40 msgr2=0x55b356aff090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:18.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.145+0000 7f2d9c699740 1 --2- 172.21.15.164:0/428640224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b356b00b40 0x55b356aff090 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c006010 tx=0x7f2d8c0099d0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.145+0000 7f2d9c699740 1 -- 172.21.15.164:0/428640224 shutdown_connections 2024-09-19T08:52:18.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.145+0000 7f2d9c699740 1 --2- 172.21.15.164:0/428640224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b356b0a370 0x55b356b0c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.145+0000 7f2d9c699740 1 --2- 172.21.15.164:0/428640224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b356aff5d0 0x55b356affa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.145+0000 7f2d9c699740 1 --2- 172.21.15.164:0/428640224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b356b00b40 0x55b356aff090 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.145+0000 7f2d9c699740 1 -- 172.21.15.164:0/428640224 >> 172.21.15.164:0/428640224 conn(0x55b356ac8310 msgr2=0x55b356afbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:18.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.145+0000 7f2d9c699740 1 -- 172.21.15.164:0/428640224 shutdown_connections 2024-09-19T08:52:18.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.145+0000 7f2d9c699740 1 -- 172.21.15.164:0/428640224 wait complete. 2024-09-19T08:52:18.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.146+0000 7f2d9c699740 1 -- start start 2024-09-19T08:52:18.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.146+0000 7f2d9c699740 1 Processor -- start 2024-09-19T08:52:18.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.146+0000 7f2d9c699740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b356aff5d0 0x55b356b09de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:18.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.146+0000 7f2d9c699740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b356b00b40 0x55b356b07090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:18.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.146+0000 7f2d9c699740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b356b0a370 0x55b356b077d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:18.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.146+0000 7f2d9c699740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b356b0d070 con 0x55b356b0a370 2024-09-19T08:52:18.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.146+0000 7f2d9c699740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b356a20060 con 0x55b356b00b40 2024-09-19T08:52:18.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.146+0000 7f2d9c699740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b356b0d1f0 con 0x55b356aff5d0 2024-09-19T08:52:18.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.146+0000 7f2d9c699740 10 client.0 _ll_register_callbacks cb 0x55b3567738c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:52:18.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.146+0000 7f2d9885d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b356b0a370 0x55b356b077d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:18.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.146+0000 7f2d9885d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b356b0a370 0x55b356b077d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46868/0 (socket says 172.21.15.164:46868) 2024-09-19T08:52:18.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.146+0000 7f2d9885d640 1 -- 172.21.15.164:0/2158689203 learned_addr learned my addr 172.21.15.164:0/2158689203 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:18.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.146+0000 7f2d937fe640 1 --2- 172.21.15.164:0/2158689203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b356b00b40 0x55b356b07090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:18.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.146+0000 7f2d93fff640 1 --2- 172.21.15.164:0/2158689203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b356aff5d0 0x55b356b09de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:18.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.147+0000 7f2d9885d640 1 -- 172.21.15.164:0/2158689203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b356aff5d0 msgr2=0x55b356b09de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:18.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.147+0000 7f2d9885d640 1 --2- 172.21.15.164:0/2158689203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b356aff5d0 0x55b356b09de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.147+0000 7f2d9885d640 1 -- 172.21.15.164:0/2158689203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b356b00b40 msgr2=0x55b356b07090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:18.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.147+0000 7f2d9885d640 1 --2- 172.21.15.164:0/2158689203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b356b00b40 0x55b356b07090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.147+0000 7f2d9885d640 1 -- 172.21.15.164:0/2158689203 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3567d63c0 con 0x55b356b0a370 2024-09-19T08:52:18.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.147+0000 7f2d937fe640 1 --2- 172.21.15.164:0/2158689203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b356b00b40 0x55b356b07090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:52:18.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.147+0000 7f2d9885d640 1 --2- 172.21.15.164:0/2158689203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b356b0a370 0x55b356b077d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c02ade0 tx=0x7f2d8c02f8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2d5fe08931ee8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:18.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.147+0000 7f2d737fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:52:18.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.147+0000 7f2d737fe640 1 -- 172.21.15.164:0/2158689203 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c059070 con 0x55b356b0a370 2024-09-19T08:52:18.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.147+0000 7f2d737fe640 1 -- 172.21.15.164:0/2158689203 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f2d8c004200 con 0x55b356b0a370 2024-09-19T08:52:18.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.147+0000 7f2d9c699740 1 -- 172.21.15.164:0/2158689203 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d8c036040 con 0x55b356b0a370 2024-09-19T08:52:18.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.147+0000 7f2d737fe640 1 -- 172.21.15.164:0/2158689203 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2d8c065050 con 0x55b356b0a370 2024-09-19T08:52:18.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.147+0000 7f2d9c699740 10 client.16797 fetch_fsmap learned FSMap version 31 2024-09-19T08:52:18.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.147+0000 7f2d9c699740 1 -- 172.21.15.164:0/2158689203 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b356b0d070 con 0x55b356b0a370 2024-09-19T08:52:18.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.147+0000 7f2d737fe640 1 -- 172.21.15.164:0/2158689203 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d8c036040 con 0x55b356b0a370 2024-09-19T08:52:18.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.147+0000 7f2d737fe640 1 -- 172.21.15.164:0/2158689203 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f2d8c030a30 con 0x55b356b0a370 2024-09-19T08:52:18.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.148+0000 7f2d9c699740 10 client.16797 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T08:52:18.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.148+0000 7f2d9c699740 10 client.16797 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T08:52:18.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.148+0000 7f2d9c699740 20 client.16797 populate_metadata read hostname 'smithi164' 2024-09-19T08:52:18.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.148+0000 7f2d9c699740 1 --2- 172.21.15.164:0/2158689203 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55b356b19d20 0x55b356b1c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:18.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.148+0000 7f2d9c699740 4 client.16797 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:52:18.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.148+0000 7f2d9c699740 1 -- 172.21.15.164:0/2158689203 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b356a72860 con 0x55b356b19d20 2024-09-19T08:52:18.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.148+0000 7f2d93fff640 1 --2- 172.21.15.164:0/2158689203 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55b356b19d20 0x55b356b1c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:18.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.148+0000 7f2d93fff640 1 --2- 172.21.15.164:0/2158689203 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55b356b19d20 0x55b356b1c110 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c3a7197b105aa7bc server_cookie=b6383272ee123658 in_seq=0 out_seq=0 2024-09-19T08:52:18.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.148+0000 7f2d737fe640 10 client.16797 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:52:18.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.149+0000 7f2d737fe640 1 -- 172.21.15.164:0/2158689203 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b356a72860 con 0x55b356b19d20 2024-09-19T08:52:18.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.149+0000 7f2d737fe640 10 client.16797 handle_command_reply: tid=0 2024-09-19T08:52:18.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.149+0000 7f2d9c699740 1 client.16797 shutdown 2024-09-19T08:52:18.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.149+0000 7f2d9c699740 1 -- 172.21.15.164:0/2158689203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b356b0a370 msgr2=0x55b356b077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:18.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.149+0000 7f2d9c699740 1 --2- 172.21.15.164:0/2158689203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b356b0a370 0x55b356b077d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c02ade0 tx=0x7f2d8c02f8a0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.149+0000 7f2d9c699740 1 -- 172.21.15.164:0/2158689203 shutdown_connections 2024-09-19T08:52:18.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.149+0000 7f2d9c699740 1 --2- 172.21.15.164:0/2158689203 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55b356b19d20 0x55b356b1c110 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.149+0000 7f2d9c699740 1 --2- 172.21.15.164:0/2158689203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b356b0a370 0x55b356b077d0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.149+0000 7f2d9c699740 1 --2- 172.21.15.164:0/2158689203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b356b00b40 0x55b356b07090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.149+0000 7f2d9c699740 1 --2- 172.21.15.164:0/2158689203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b356aff5d0 0x55b356b09de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.149+0000 7f2d9c699740 1 -- 172.21.15.164:0/2158689203 >> 172.21.15.164:0/2158689203 conn(0x55b356ac8310 msgr2=0x55b356b019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:18.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.149+0000 7f2d737fe640 0 client.16797 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:52:18.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.149+0000 7f2d9c699740 1 -- 172.21.15.164:0/2158689203 shutdown_connections 2024-09-19T08:52:18.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.149+0000 7f2d9c699740 1 -- 172.21.15.164:0/2158689203 wait complete. 2024-09-19T08:52:18.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.150+0000 7f2d9c699740 20 client.16797 trim_cache size 0 max 16384 2024-09-19T08:52:18.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.163+0000 7f2d9c699740 1 Processor -- start 2024-09-19T08:52:18.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.163+0000 7f2d9c699740 1 -- start start 2024-09-19T08:52:18.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.163+0000 7f2d9c699740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b356aff5d0 0x55b356b16310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:18.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.163+0000 7f2d9c699740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b356b00b40 0x55b356b021b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:18.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.163+0000 7f2d9c699740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b356b0a370 0x55b356b026f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:18.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.163+0000 7f2d9c699740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2d941af810 con 0x55b356b0a370 2024-09-19T08:52:18.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.163+0000 7f2d9c699740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2d941b1380 con 0x55b356b00b40 2024-09-19T08:52:18.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.163+0000 7f2d9c699740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2d941afee0 con 0x55b356aff5d0 2024-09-19T08:52:18.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.163+0000 7f2d9885d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b356b0a370 0x55b356b026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:18.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.163+0000 7f2d9885d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b356b0a370 0x55b356b026f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46882/0 (socket says 172.21.15.164:46882) 2024-09-19T08:52:18.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.163+0000 7f2d9885d640 1 -- 172.21.15.164:0/1106997008 learned_addr learned my addr 172.21.15.164:0/1106997008 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:18.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.163+0000 7f2d937fe640 1 --2- 172.21.15.164:0/1106997008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b356b00b40 0x55b356b021b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:18.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.163+0000 7f2d93fff640 1 --2- 172.21.15.164:0/1106997008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b356aff5d0 0x55b356b16310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:18.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.163+0000 7f2d9885d640 1 -- 172.21.15.164:0/1106997008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b356aff5d0 msgr2=0x55b356b16310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:18.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.163+0000 7f2d9885d640 1 --2- 172.21.15.164:0/1106997008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b356aff5d0 0x55b356b16310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.163+0000 7f2d9885d640 1 -- 172.21.15.164:0/1106997008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b356b00b40 msgr2=0x55b356b021b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:18.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.163+0000 7f2d9885d640 1 --2- 172.21.15.164:0/1106997008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b356b00b40 0x55b356b021b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.163+0000 7f2d9885d640 1 -- 172.21.15.164:0/1106997008 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b356afc960 con 0x55b356b0a370 2024-09-19T08:52:18.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.163+0000 7f2d93fff640 1 --2- 172.21.15.164:0/1106997008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b356aff5d0 0x55b356b16310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:52:18.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.163+0000 7f2d937fe640 1 --2- 172.21.15.164:0/1106997008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b356b00b40 0x55b356b021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:52:18.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.163+0000 7f2d9885d640 1 --2- 172.21.15.164:0/1106997008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b356b0a370 0x55b356b026f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c02f870 tx=0x7f2d8c02f910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd14ef4d5e84a40b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:18.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.164+0000 7f2d79ffb640 1 -- 172.21.15.164:0/1106997008 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c059040 con 0x55b356b0a370 2024-09-19T08:52:18.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.164+0000 7f2d79ffb640 1 -- 172.21.15.164:0/1106997008 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d8c036030 con 0x55b356b0a370 2024-09-19T08:52:18.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.164+0000 7f2d79ffb640 1 -- 172.21.15.164:0/1106997008 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c042070 con 0x55b356b0a370 2024-09-19T08:52:18.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.164+0000 7f2d9c699740 1 -- 172.21.15.164:0/1106997008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b356b0a370 msgr2=0x55b356b026f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:18.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.164+0000 7f2d9c699740 1 --2- 172.21.15.164:0/1106997008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b356b0a370 0x55b356b026f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c02f870 tx=0x7f2d8c02f910 comp rx=0 tx=0).stop 2024-09-19T08:52:18.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.164+0000 7f2d9c699740 1 -- 172.21.15.164:0/1106997008 shutdown_connections 2024-09-19T08:52:18.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.164+0000 7f2d9c699740 1 --2- 172.21.15.164:0/1106997008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b356b0a370 0x55b356b026f0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.164+0000 7f2d9c699740 1 --2- 172.21.15.164:0/1106997008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b356b00b40 0x55b356b021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.164+0000 7f2d9c699740 1 --2- 172.21.15.164:0/1106997008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b356aff5d0 0x55b356b16310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.164+0000 7f2d9c699740 1 -- 172.21.15.164:0/1106997008 >> 172.21.15.164:0/1106997008 conn(0x55b356ac8310 msgr2=0x55b356b09110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:18.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.164+0000 7f2d9c699740 1 -- 172.21.15.164:0/1106997008 shutdown_connections 2024-09-19T08:52:18.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.164+0000 7f2d9c699740 1 -- 172.21.15.164:0/1106997008 wait complete. 2024-09-19T08:52:18.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.165+0000 7f2d9c699740 1 -- start start 2024-09-19T08:52:18.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.165+0000 7f2d9c699740 1 Processor -- start 2024-09-19T08:52:18.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.165+0000 7f2d9c699740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b356aff5d0 0x55b356b0dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:18.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.165+0000 7f2d9c699740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b356b00b40 0x55b356b0e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:18.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.165+0000 7f2d9c699740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b356b0a370 0x55b356b7c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:18.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.165+0000 7f2d9c699740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2d941b0c80 con 0x55b356b0a370 2024-09-19T08:52:18.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.165+0000 7f2d9c699740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2d941b05b0 con 0x55b356b00b40 2024-09-19T08:52:18.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.165+0000 7f2d9c699740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2d941ad810 con 0x55b356aff5d0 2024-09-19T08:52:18.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.165+0000 7f2d9c699740 10 client.0 _ll_register_callbacks cb 0x7f2d8800b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:52:18.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.165+0000 7f2d9885d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b356b0a370 0x55b356b7c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:18.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.165+0000 7f2d9885d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b356b0a370 0x55b356b7c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46888/0 (socket says 172.21.15.164:46888) 2024-09-19T08:52:18.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.165+0000 7f2d9885d640 1 -- 172.21.15.164:0/3360163250 learned_addr learned my addr 172.21.15.164:0/3360163250 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:18.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.166+0000 7f2d93fff640 1 --2- 172.21.15.164:0/3360163250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b356aff5d0 0x55b356b0dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:18.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.166+0000 7f2d937fe640 1 --2- 172.21.15.164:0/3360163250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b356b00b40 0x55b356b0e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:18.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.166+0000 7f2d9885d640 1 -- 172.21.15.164:0/3360163250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b356aff5d0 msgr2=0x55b356b0dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:18.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.166+0000 7f2d9885d640 1 --2- 172.21.15.164:0/3360163250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b356aff5d0 0x55b356b0dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.166+0000 7f2d9885d640 1 -- 172.21.15.164:0/3360163250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b356b00b40 msgr2=0x55b356b0e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:18.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.166+0000 7f2d9885d640 1 --2- 172.21.15.164:0/3360163250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b356b00b40 0x55b356b0e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.166+0000 7f2d9885d640 1 -- 172.21.15.164:0/3360163250 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b356b7cfc0 con 0x55b356b0a370 2024-09-19T08:52:18.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.166+0000 7f2d937fe640 1 --2- 172.21.15.164:0/3360163250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b356b00b40 0x55b356b0e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:52:18.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.166+0000 7f2d9885d640 1 --2- 172.21.15.164:0/3360163250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b356b0a370 0x55b356b7c920 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c02ade0 tx=0x7f2d8c02f8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c73375f0ea3ece2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:18.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.166+0000 7f2d737fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:52:18.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.166+0000 7f2d737fe640 1 -- 172.21.15.164:0/3360163250 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c042070 con 0x55b356b0a370 2024-09-19T08:52:18.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.166+0000 7f2d737fe640 1 -- 172.21.15.164:0/3360163250 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f2d8c003f80 con 0x55b356b0a370 2024-09-19T08:52:18.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.166+0000 7f2d9c699740 1 -- 172.21.15.164:0/3360163250 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d8c036030 con 0x55b356b0a370 2024-09-19T08:52:18.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.166+0000 7f2d737fe640 1 -- 172.21.15.164:0/3360163250 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2d8c059040 con 0x55b356b0a370 2024-09-19T08:52:18.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.166+0000 7f2d9c699740 10 client.16803 fetch_fsmap learned FSMap version 31 2024-09-19T08:52:18.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.166+0000 7f2d9c699740 1 -- 172.21.15.164:0/3360163250 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b356b84370 con 0x55b356b0a370 2024-09-19T08:52:18.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.166+0000 7f2d737fe640 1 -- 172.21.15.164:0/3360163250 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d8c036030 con 0x55b356b0a370 2024-09-19T08:52:18.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.167+0000 7f2d737fe640 1 -- 172.21.15.164:0/3360163250 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f2d8c004bd0 con 0x55b356b0a370 2024-09-19T08:52:18.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.167+0000 7f2d9c699740 10 client.16803 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T08:52:18.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.167+0000 7f2d9c699740 10 client.16803 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T08:52:18.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.167+0000 7f2d9c699740 20 client.16803 populate_metadata read hostname 'smithi164' 2024-09-19T08:52:18.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.167+0000 7f2d9c699740 1 --2- 172.21.15.164:0/3360163250 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55b356b84a40 0x55b356b86e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:18.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.167+0000 7f2d9c699740 4 client.16803 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:52:18.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.167+0000 7f2d9c699740 1 -- 172.21.15.164:0/3360163250 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b356a6cd70 con 0x55b356b84a40 2024-09-19T08:52:18.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.167+0000 7f2d93fff640 1 --2- 172.21.15.164:0/3360163250 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55b356b84a40 0x55b356b86e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:18.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.167+0000 7f2d93fff640 1 --2- 172.21.15.164:0/3360163250 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55b356b84a40 0x55b356b86e60 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=57d2370174eb0b9d server_cookie=7a4e224dd6259a3f in_seq=0 out_seq=0 2024-09-19T08:52:18.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.167+0000 7f2d737fe640 10 client.16803 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:52:18.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.167+0000 7f2d737fe640 1 -- 172.21.15.164:0/3360163250 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b356a6cd70 con 0x55b356b84a40 2024-09-19T08:52:18.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.167+0000 7f2d737fe640 10 client.16803 handle_command_reply: tid=0 2024-09-19T08:52:18.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.167+0000 7f2d9c699740 1 client.16803 shutdown 2024-09-19T08:52:18.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.168+0000 7f2d9c699740 1 -- 172.21.15.164:0/3360163250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b356b0a370 msgr2=0x55b356b7c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:18.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.168+0000 7f2d9c699740 1 --2- 172.21.15.164:0/3360163250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b356b0a370 0x55b356b7c920 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c02ade0 tx=0x7f2d8c02f8a0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.168+0000 7f2d9c699740 1 -- 172.21.15.164:0/3360163250 shutdown_connections 2024-09-19T08:52:18.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.168+0000 7f2d9c699740 1 --2- 172.21.15.164:0/3360163250 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55b356b84a40 0x55b356b86e60 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.168+0000 7f2d9c699740 1 --2- 172.21.15.164:0/3360163250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b356b0a370 0x55b356b7c920 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.168+0000 7f2d9c699740 1 --2- 172.21.15.164:0/3360163250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b356b00b40 0x55b356b0e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.168+0000 7f2d737fe640 0 client.16803 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:52:18.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.168+0000 7f2d9c699740 1 --2- 172.21.15.164:0/3360163250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b356aff5d0 0x55b356b0dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.168+0000 7f2d9c699740 1 -- 172.21.15.164:0/3360163250 >> 172.21.15.164:0/3360163250 conn(0x55b356ac8310 msgr2=0x55b356b05830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:18.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.168+0000 7f2d9c699740 1 -- 172.21.15.164:0/3360163250 shutdown_connections 2024-09-19T08:52:18.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.168+0000 7f2d9c699740 1 -- 172.21.15.164:0/3360163250 wait complete. 2024-09-19T08:52:18.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.168+0000 7f2d9c699740 20 client.16803 trim_cache size 0 max 16384 2024-09-19T08:52:18.180 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:52:18.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.168+0000 7f2d9a2c0640 1 -- 172.21.15.164:0/2032349973 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f2d7c0a33b0 msgr2=0x7f2d7c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:18.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.168+0000 7f2d9a2c0640 1 --2- 172.21.15.164:0/2032349973 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f2d7c0a33b0 0x7f2d7c0a5870 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x55b356b01170 tx=0x7f2d8c04e000 comp rx=0 tx=0).stop 2024-09-19T08:52:18.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.168+0000 7f2d9a2c0640 1 -- 172.21.15.164:0/2032349973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d94101df0 msgr2=0x7f2d940744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:18.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.168+0000 7f2d9a2c0640 1 --2- 172.21.15.164:0/2032349973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d94101df0 0x7f2d940744f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f2d8400d950 tx=0x7f2d8400de10 comp rx=0 tx=0).stop 2024-09-19T08:52:18.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.169+0000 7f2d9a2c0640 1 -- 172.21.15.164:0/2032349973 shutdown_connections 2024-09-19T08:52:18.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.169+0000 7f2d9a2c0640 1 --2- 172.21.15.164:0/2032349973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d94106f20 0x7f2d94074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.169+0000 7f2d9a2c0640 1 --2- 172.21.15.164:0/2032349973 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d941027a0 0x7f2d94074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.169+0000 7f2d9a2c0640 1 --2- 172.21.15.164:0/2032349973 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f2d7c0a33b0 0x7f2d7c0a5870 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.169+0000 7f2d9a2c0640 1 --2- 172.21.15.164:0/2032349973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d94101df0 0x7f2d940744f0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:18.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.169+0000 7f2d9a2c0640 1 -- 172.21.15.164:0/2032349973 >> 172.21.15.164:0/2032349973 conn(0x7f2d940fd8b0 msgr2=0x7f2d940ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:18.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.169+0000 7f2d9a2c0640 1 -- 172.21.15.164:0/2032349973 shutdown_connections 2024-09-19T08:52:18.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:18.169+0000 7f2d9a2c0640 1 -- 172.21.15.164:0/2032349973 wait complete. 2024-09-19T08:52:18.427 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:18 smithi170 ceph-mon[35072]: pgmap v199: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 250 MiB/s wr, 924 op/s 2024-09-19T08:52:18.463 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:18 smithi173 ceph-mon[35306]: pgmap v199: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 250 MiB/s wr, 924 op/s 2024-09-19T08:52:18.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:18 smithi164 ceph-mon[30928]: pgmap v199: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 250 MiB/s wr, 924 op/s 2024-09-19T08:52:20.104 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T08:52:20.359 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:52:20.479 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:20 smithi164 ceph-mon[30928]: pgmap v200: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 260 MiB/s wr, 982 op/s 2024-09-19T08:52:20.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:20 smithi170 ceph-mon[35072]: pgmap v200: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 260 MiB/s wr, 982 op/s 2024-09-19T08:52:20.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:20 smithi173 ceph-mon[35306]: pgmap v200: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 260 MiB/s wr, 982 op/s 2024-09-19T08:52:20.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.838+0000 7f2d77382640 1 -- 172.21.15.164:0/3946161835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d701036e0 msgr2=0x7f2d70107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:20.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.838+0000 7f2d77382640 1 --2- 172.21.15.164:0/3946161835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d701036e0 0x7f2d70107b90 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009930 tx=0x7f2d6002a420 comp rx=0 tx=0).stop 2024-09-19T08:52:20.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.838+0000 7f2d77382640 1 -- 172.21.15.164:0/3946161835 shutdown_connections 2024-09-19T08:52:20.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.838+0000 7f2d77382640 1 --2- 172.21.15.164:0/3946161835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d701082a0 0x7f2d7010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:20.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.838+0000 7f2d77382640 1 --2- 172.21.15.164:0/3946161835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d701036e0 0x7f2d70107b90 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:20.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.838+0000 7f2d77382640 1 --2- 172.21.15.164:0/3946161835 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d70102d30 0x7f2d70103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:20.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.838+0000 7f2d77382640 1 -- 172.21.15.164:0/3946161835 >> 172.21.15.164:0/3946161835 conn(0x7f2d700fc7d0 msgr2=0x7f2d700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:20.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.838+0000 7f2d77382640 1 -- 172.21.15.164:0/3946161835 shutdown_connections 2024-09-19T08:52:20.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.838+0000 7f2d77382640 1 -- 172.21.15.164:0/3946161835 wait complete. 2024-09-19T08:52:20.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.839+0000 7f2d77382640 1 Processor -- start 2024-09-19T08:52:20.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.839+0000 7f2d77382640 1 -- start start 2024-09-19T08:52:20.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.839+0000 7f2d77382640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d70102d30 0x7f2d70198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:20.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.839+0000 7f2d77382640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d701036e0 0x7f2d70198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:20.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.839+0000 7f2d77382640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d701082a0 0x7f2d7019ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:20.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.839+0000 7f2d77382640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2d701120a0 con 0x7f2d701082a0 2024-09-19T08:52:20.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.839+0000 7f2d77382640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2d70111f20 con 0x7f2d701036e0 2024-09-19T08:52:20.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.839+0000 7f2d77382640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2d70112220 con 0x7f2d70102d30 2024-09-19T08:52:20.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.839+0000 7f2d7591f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d701082a0 0x7f2d7019ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:20.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.839+0000 7f2d7591f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d701082a0 0x7f2d7019ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46918/0 (socket says 172.21.15.164:46918) 2024-09-19T08:52:20.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.840+0000 7f2d7591f640 1 -- 172.21.15.164:0/1078257096 learned_addr learned my addr 172.21.15.164:0/1078257096 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:20.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.840+0000 7f2d7491d640 1 --2- 172.21.15.164:0/1078257096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d701036e0 0x7f2d70198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:20.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.840+0000 7f2d7511e640 1 --2- 172.21.15.164:0/1078257096 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d70102d30 0x7f2d70198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:20.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.840+0000 7f2d7591f640 1 -- 172.21.15.164:0/1078257096 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d70102d30 msgr2=0x7f2d70198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:20.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.840+0000 7f2d7591f640 1 --2- 172.21.15.164:0/1078257096 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d70102d30 0x7f2d70198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:20.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.840+0000 7f2d7591f640 1 -- 172.21.15.164:0/1078257096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d701036e0 msgr2=0x7f2d70198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:20.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.840+0000 7f2d7591f640 1 --2- 172.21.15.164:0/1078257096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d701036e0 0x7f2d70198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:20.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.840+0000 7f2d7591f640 1 -- 172.21.15.164:0/1078257096 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7019d5b0 con 0x7f2d701082a0 2024-09-19T08:52:20.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.840+0000 7f2d7491d640 1 --2- 172.21.15.164:0/1078257096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d701036e0 0x7f2d70198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:52:20.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.840+0000 7f2d7591f640 1 --2- 172.21.15.164:0/1078257096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d701082a0 0x7f2d7019ced0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2d6800eb10 tx=0x7f2d6800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:20.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.840+0000 7f2d667fc640 1 -- 172.21.15.164:0/1078257096 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d68004b20 con 0x7f2d701082a0 2024-09-19T08:52:20.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.840+0000 7f2d667fc640 1 -- 172.21.15.164:0/1078257096 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d68004cc0 con 0x7f2d701082a0 2024-09-19T08:52:20.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.840+0000 7f2d77382640 1 -- 172.21.15.164:0/1078257096 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d7019d8a0 con 0x7f2d701082a0 2024-09-19T08:52:20.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.840+0000 7f2d667fc640 1 -- 172.21.15.164:0/1078257096 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d68010760 con 0x7f2d701082a0 2024-09-19T08:52:20.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.840+0000 7f2d77382640 1 -- 172.21.15.164:0/1078257096 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d701a5180 con 0x7f2d701082a0 2024-09-19T08:52:20.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.842+0000 7f2d667fc640 1 -- 172.21.15.164:0/1078257096 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f2d68005020 con 0x7f2d701082a0 2024-09-19T08:52:20.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.843+0000 7f2d7975b740 1 Processor -- start 2024-09-19T08:52:20.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.843+0000 7f2d7975b740 1 -- start start 2024-09-19T08:52:20.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.843+0000 7f2d7975b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b190a8b40 0x556b190a7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:20.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.843+0000 7f2d7975b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b190a75d0 0x556b190a7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:20.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.843+0000 7f2d7975b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b190b2370 0x556b190b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:20.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.843+0000 7f2d7975b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556b1900d2f0 con 0x556b190a8b40 2024-09-19T08:52:20.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.843+0000 7f2d7975b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556b18fb4cf0 con 0x556b190b2370 2024-09-19T08:52:20.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.843+0000 7f2d7975b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556b190092b0 con 0x556b190a75d0 2024-09-19T08:52:20.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.843+0000 7f2d667fc640 1 --2- 172.21.15.164:0/1078257096 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f2d5c0a33b0 0x7f2d5c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:20.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.843+0000 7f2d7491d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b190a8b40 0x556b190a7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:20.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.843+0000 7f2d7491d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b190a8b40 0x556b190a7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46924/0 (socket says 172.21.15.164:46924) 2024-09-19T08:52:20.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.843+0000 7f2d7491d640 1 -- 172.21.15.164:0/3351435613 learned_addr learned my addr 172.21.15.164:0/3351435613 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:20.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.843+0000 7f2d7511e640 1 --2- 172.21.15.164:0/3351435613 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b190a75d0 0x556b190a7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:20.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.844+0000 7f2d7591f640 1 --2- 172.21.15.164:0/1078257096 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f2d5c0a33b0 0x7f2d5c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:20.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.844+0000 7f2d7491d640 1 --2- 172.21.15.164:0/3351435613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b190b2370 0x556b190b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:20.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.844+0000 7f2d667fc640 1 -- 172.21.15.164:0/1078257096 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f2d680d03e0 con 0x7f2d701082a0 2024-09-19T08:52:20.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.844+0000 7f2d7491d640 1 -- 172.21.15.164:0/3351435613 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b190a75d0 msgr2=0x556b190a7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:20.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.844+0000 7f2d7491d640 1 --2- 172.21.15.164:0/3351435613 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b190a75d0 0x556b190a7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:20.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.844+0000 7f2d7491d640 1 -- 172.21.15.164:0/3351435613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b190b2370 msgr2=0x556b190b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:20.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.844+0000 7f2d7491d640 1 --2- 172.21.15.164:0/3351435613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b190b2370 0x556b190b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:20.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.844+0000 7f2d7491d640 1 -- 172.21.15.164:0/3351435613 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b18f993a0 con 0x556b190a8b40 2024-09-19T08:52:20.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.844+0000 7f2d7491d640 1 --2- 172.21.15.164:0/3351435613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b190a8b40 0x556b190a7090 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f2d60005c40 tx=0x7f2d60005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2200dc3ac4f23e96 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:20.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.844+0000 7f2d7591f640 1 --2- 172.21.15.164:0/1078257096 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f2d5c0a33b0 0x7f2d5c0a5870 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2d68098b00 tx=0x7f2d68091b30 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:20.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.844+0000 7f2d5b7fe640 1 -- 172.21.15.164:0/3351435613 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60035070 con 0x556b190a8b40 2024-09-19T08:52:20.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.844+0000 7f2d5b7fe640 1 -- 172.21.15.164:0/3351435613 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d60044070 con 0x556b190a8b40 2024-09-19T08:52:20.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.844+0000 7f2d5b7fe640 1 -- 172.21.15.164:0/3351435613 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d600055a0 con 0x556b190a8b40 2024-09-19T08:52:20.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.845+0000 7f2d7975b740 1 -- 172.21.15.164:0/3351435613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b190a8b40 msgr2=0x556b190a7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:20.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.845+0000 7f2d7975b740 1 --2- 172.21.15.164:0/3351435613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b190a8b40 0x556b190a7090 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f2d60005c40 tx=0x7f2d60005bd0 comp rx=0 tx=0).stop 2024-09-19T08:52:20.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.845+0000 7f2d7975b740 1 -- 172.21.15.164:0/3351435613 shutdown_connections 2024-09-19T08:52:20.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.845+0000 7f2d7975b740 1 --2- 172.21.15.164:0/3351435613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b190b2370 0x556b190b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:20.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.845+0000 7f2d7975b740 1 --2- 172.21.15.164:0/3351435613 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b190a75d0 0x556b190a7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:20.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.845+0000 7f2d7975b740 1 --2- 172.21.15.164:0/3351435613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b190a8b40 0x556b190a7090 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:20.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.845+0000 7f2d7975b740 1 -- 172.21.15.164:0/3351435613 >> 172.21.15.164:0/3351435613 conn(0x556b19070310 msgr2=0x556b190a3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:20.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.845+0000 7f2d7975b740 1 -- 172.21.15.164:0/3351435613 shutdown_connections 2024-09-19T08:52:20.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.845+0000 7f2d7975b740 1 -- 172.21.15.164:0/3351435613 wait complete. 2024-09-19T08:52:20.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.845+0000 7f2d7975b740 1 -- start start 2024-09-19T08:52:20.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.845+0000 7f2d7975b740 1 Processor -- start 2024-09-19T08:52:20.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.846+0000 7f2d7975b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b190a75d0 0x556b190b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:20.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.846+0000 7f2d7975b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b190a8b40 0x556b190af060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:20.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.846+0000 7f2d7975b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b190b2370 0x556b190af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:20.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.846+0000 7f2d7975b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556b190b5010 con 0x556b190b2370 2024-09-19T08:52:20.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.846+0000 7f2d7975b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556b18fc8060 con 0x556b190a75d0 2024-09-19T08:52:20.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.846+0000 7f2d7975b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556b190b5190 con 0x556b190a8b40 2024-09-19T08:52:20.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.846+0000 7f2d7975b740 10 client.0 _ll_register_callbacks cb 0x556b18d1b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:52:20.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.846+0000 7f2d7491d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b190b2370 0x556b190af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:20.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.846+0000 7f2d7491d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b190b2370 0x556b190af7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46928/0 (socket says 172.21.15.164:46928) 2024-09-19T08:52:20.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.846+0000 7f2d7491d640 1 -- 172.21.15.164:0/283319919 learned_addr learned my addr 172.21.15.164:0/283319919 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:20.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.846+0000 7f2d7491d640 1 --2- 172.21.15.164:0/283319919 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b190a75d0 0x556b190b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:20.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.846+0000 7f2d7511e640 1 --2- 172.21.15.164:0/283319919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b190a8b40 0x556b190af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:20.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.846+0000 7f2d7491d640 1 -- 172.21.15.164:0/283319919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b190a8b40 msgr2=0x556b190af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:20.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.846+0000 7f2d7491d640 1 --2- 172.21.15.164:0/283319919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b190a8b40 0x556b190af060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:20.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.846+0000 7f2d7491d640 1 -- 172.21.15.164:0/283319919 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b190a75d0 msgr2=0x556b190b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:20.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.846+0000 7f2d7491d640 1 --2- 172.21.15.164:0/283319919 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b190a75d0 0x556b190b1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:20.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.846+0000 7f2d7491d640 1 -- 172.21.15.164:0/283319919 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b18d7e3c0 con 0x556b190b2370 2024-09-19T08:52:20.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.846+0000 7f2d7491d640 1 --2- 172.21.15.164:0/283319919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b190b2370 0x556b190af7f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f2d60006010 tx=0x7f2d6003f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af3c3e357b421ce7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:20.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.846+0000 7f2d58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:52:20.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.847+0000 7f2d58ff9640 1 -- 172.21.15.164:0/283319919 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60035070 con 0x556b190b2370 2024-09-19T08:52:20.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.847+0000 7f2d58ff9640 1 -- 172.21.15.164:0/283319919 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f2d60004440 con 0x556b190b2370 2024-09-19T08:52:20.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.847+0000 7f2d7975b740 1 -- 172.21.15.164:0/283319919 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d60044070 con 0x556b190b2370 2024-09-19T08:52:20.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.847+0000 7f2d58ff9640 1 -- 172.21.15.164:0/283319919 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2d60031a10 con 0x556b190b2370 2024-09-19T08:52:20.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.847+0000 7f2d7975b740 10 client.16815 fetch_fsmap learned FSMap version 31 2024-09-19T08:52:20.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.847+0000 7f2d7975b740 1 -- 172.21.15.164:0/283319919 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556b190c0b00 con 0x556b190b2370 2024-09-19T08:52:20.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.847+0000 7f2d58ff9640 1 -- 172.21.15.164:0/283319919 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d60044070 con 0x556b190b2370 2024-09-19T08:52:20.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.847+0000 7f2d58ff9640 1 -- 172.21.15.164:0/283319919 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f2d600038c0 con 0x556b190b2370 2024-09-19T08:52:20.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.847+0000 7f2d7975b740 10 client.16815 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T08:52:20.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.847+0000 7f2d7975b740 10 client.16815 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T08:52:20.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.847+0000 7f2d7975b740 20 client.16815 populate_metadata read hostname 'smithi164' 2024-09-19T08:52:20.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.847+0000 7f2d7975b740 1 --2- 172.21.15.164:0/283319919 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x556b190c1e50 0x556b190c4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:20.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.847+0000 7f2d7975b740 4 client.16815 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:52:20.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.847+0000 7f2d7975b740 1 -- 172.21.15.164:0/283319919 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b1901a860 con 0x556b190c1e50 2024-09-19T08:52:20.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.847+0000 7f2d7511e640 1 --2- 172.21.15.164:0/283319919 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x556b190c1e50 0x556b190c4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:20.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.848+0000 7f2d7511e640 1 --2- 172.21.15.164:0/283319919 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x556b190c1e50 0x556b190c4240 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=414e87180fd59e86 server_cookie=2e5bc81bddeedef2 in_seq=0 out_seq=0 2024-09-19T08:52:20.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.848+0000 7f2d58ff9640 10 client.16815 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:52:20.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.849+0000 7f2d58ff9640 1 -- 172.21.15.164:0/283319919 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556b1901a860 con 0x556b190c1e50 2024-09-19T08:52:20.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.849+0000 7f2d58ff9640 10 client.16815 handle_command_reply: tid=0 2024-09-19T08:52:20.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.849+0000 7f2d7975b740 1 client.16815 shutdown 2024-09-19T08:52:20.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.849+0000 7f2d7975b740 1 -- 172.21.15.164:0/283319919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b190b2370 msgr2=0x556b190af7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:20.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.849+0000 7f2d7975b740 1 --2- 172.21.15.164:0/283319919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b190b2370 0x556b190af7f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f2d60006010 tx=0x7f2d6003f040 comp rx=0 tx=0).stop 2024-09-19T08:52:20.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.849+0000 7f2d7975b740 1 -- 172.21.15.164:0/283319919 shutdown_connections 2024-09-19T08:52:20.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.849+0000 7f2d7975b740 1 --2- 172.21.15.164:0/283319919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b190b2370 0x556b190af7f0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:20.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.849+0000 7f2d7975b740 1 --2- 172.21.15.164:0/283319919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b190a8b40 0x556b190af060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:20.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.849+0000 7f2d7975b740 1 --2- 172.21.15.164:0/283319919 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x556b190c1e50 0x556b190c4240 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:20.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.849+0000 7f2d7975b740 1 --2- 172.21.15.164:0/283319919 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b190a75d0 0x556b190b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:20.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.849+0000 7f2d7975b740 1 -- 172.21.15.164:0/283319919 >> 172.21.15.164:0/283319919 conn(0x556b19070310 msgr2=0x556b190a99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:20.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.849+0000 7f2d58ff9640 0 client.16815 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:52:20.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.849+0000 7f2d7975b740 1 -- 172.21.15.164:0/283319919 shutdown_connections 2024-09-19T08:52:20.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.849+0000 7f2d7975b740 1 -- 172.21.15.164:0/283319919 wait complete. 2024-09-19T08:52:20.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.849+0000 7f2d7975b740 20 client.16815 trim_cache size 0 max 16384 2024-09-19T08:52:20.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.862+0000 7f2d7975b740 1 Processor -- start 2024-09-19T08:52:20.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.862+0000 7f2d7975b740 1 -- start start 2024-09-19T08:52:20.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.862+0000 7f2d7975b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b190a75d0 0x556b190bdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:20.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.862+0000 7f2d7975b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b190a8b40 0x556b190aa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:20.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.862+0000 7f2d7975b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b190b2370 0x556b190aa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:20.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.862+0000 7f2d7975b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2d701ad700 con 0x556b190b2370 2024-09-19T08:52:20.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.862+0000 7f2d7975b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2d701af270 con 0x556b190a75d0 2024-09-19T08:52:20.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.862+0000 7f2d7975b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2d701addd0 con 0x556b190a8b40 2024-09-19T08:52:20.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.862+0000 7f2d7491d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b190b2370 0x556b190aa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:20.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.862+0000 7f2d7491d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b190b2370 0x556b190aa660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46934/0 (socket says 172.21.15.164:46934) 2024-09-19T08:52:20.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.862+0000 7f2d7491d640 1 -- 172.21.15.164:0/218594951 learned_addr learned my addr 172.21.15.164:0/218594951 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:20.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.863+0000 7f2d7511e640 1 --2- 172.21.15.164:0/218594951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b190a8b40 0x556b190aa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:20.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.863+0000 7f2d7491d640 1 --2- 172.21.15.164:0/218594951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b190a75d0 0x556b190bdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:20.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.863+0000 7f2d7491d640 1 -- 172.21.15.164:0/218594951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b190a8b40 msgr2=0x556b190aa120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:20.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.863+0000 7f2d7491d640 1 --2- 172.21.15.164:0/218594951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b190a8b40 0x556b190aa120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:20.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.863+0000 7f2d7491d640 1 -- 172.21.15.164:0/218594951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b190a75d0 msgr2=0x556b190bdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:20.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.863+0000 7f2d7491d640 1 --2- 172.21.15.164:0/218594951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b190a75d0 0x556b190bdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:20.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.863+0000 7f2d7491d640 1 -- 172.21.15.164:0/218594951 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b190ad5b0 con 0x556b190b2370 2024-09-19T08:52:20.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.863+0000 7f2d7491d640 1 --2- 172.21.15.164:0/218594951 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b190b2370 0x556b190aa660 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f2d60001eb0 tx=0x7f2d6002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c523dd54581935 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:20.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.863+0000 7f2d5affd640 1 -- 172.21.15.164:0/218594951 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60035030 con 0x556b190b2370 2024-09-19T08:52:20.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.863+0000 7f2d5affd640 1 -- 172.21.15.164:0/218594951 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d60050070 con 0x556b190b2370 2024-09-19T08:52:20.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.863+0000 7f2d5affd640 1 -- 172.21.15.164:0/218594951 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6003d700 con 0x556b190b2370 2024-09-19T08:52:20.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.864+0000 7f2d7975b740 1 -- 172.21.15.164:0/218594951 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b190b2370 msgr2=0x556b190aa660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:20.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.864+0000 7f2d7975b740 1 --2- 172.21.15.164:0/218594951 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b190b2370 0x556b190aa660 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f2d60001eb0 tx=0x7f2d6002a420 comp rx=0 tx=0).stop 2024-09-19T08:52:20.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.864+0000 7f2d7975b740 1 -- 172.21.15.164:0/218594951 shutdown_connections 2024-09-19T08:52:20.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.864+0000 7f2d7975b740 1 --2- 172.21.15.164:0/218594951 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b190b2370 0x556b190aa660 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:20.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.864+0000 7f2d7975b740 1 --2- 172.21.15.164:0/218594951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b190a8b40 0x556b190aa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:20.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.864+0000 7f2d7975b740 1 --2- 172.21.15.164:0/218594951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b190a75d0 0x556b190bdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:20.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.864+0000 7f2d7975b740 1 -- 172.21.15.164:0/218594951 >> 172.21.15.164:0/218594951 conn(0x556b19070310 msgr2=0x556b190bd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:20.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.864+0000 7f2d7975b740 1 -- 172.21.15.164:0/218594951 shutdown_connections 2024-09-19T08:52:20.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.864+0000 7f2d7975b740 1 -- 172.21.15.164:0/218594951 wait complete. 2024-09-19T08:52:20.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.864+0000 7f2d7975b740 1 -- start start 2024-09-19T08:52:20.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.864+0000 7f2d7975b740 1 Processor -- start 2024-09-19T08:52:20.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.865+0000 7f2d7975b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b190a75d0 0x556b190c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:20.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.865+0000 7f2d7975b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b190a8b40 0x556b190c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:20.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.865+0000 7f2d7975b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b190b2370 0x556b190c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:20.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.865+0000 7f2d7975b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2d701aeb70 con 0x556b190b2370 2024-09-19T08:52:20.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.865+0000 7f2d7975b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2d701ae4a0 con 0x556b190a75d0 2024-09-19T08:52:20.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.865+0000 7f2d7975b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2d701ab700 con 0x556b190a8b40 2024-09-19T08:52:20.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.865+0000 7f2d7975b740 10 client.0 _ll_register_callbacks cb 0x7f2d6c00b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:52:20.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.865+0000 7f2d7491d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b190b2370 0x556b190c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:20.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.865+0000 7f2d7491d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b190b2370 0x556b190c4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46938/0 (socket says 172.21.15.164:46938) 2024-09-19T08:52:20.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.865+0000 7f2d7491d640 1 -- 172.21.15.164:0/195386710 learned_addr learned my addr 172.21.15.164:0/195386710 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:20.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.865+0000 7f2d7491d640 1 --2- 172.21.15.164:0/195386710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b190a75d0 0x556b190c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:20.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.865+0000 7f2d7511e640 1 --2- 172.21.15.164:0/195386710 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b190a8b40 0x556b190c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:20.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.865+0000 7f2d7491d640 1 -- 172.21.15.164:0/195386710 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b190a8b40 msgr2=0x556b190c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:20.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.865+0000 7f2d7491d640 1 --2- 172.21.15.164:0/195386710 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b190a8b40 0x556b190c49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:20.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.865+0000 7f2d7491d640 1 -- 172.21.15.164:0/195386710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b190a75d0 msgr2=0x556b190c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:20.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.865+0000 7f2d7491d640 1 --2- 172.21.15.164:0/195386710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b190a75d0 0x556b190c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:20.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.865+0000 7f2d7491d640 1 -- 172.21.15.164:0/195386710 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b190c5730 con 0x556b190b2370 2024-09-19T08:52:20.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.865+0000 7f2d7511e640 1 --2- 172.21.15.164:0/195386710 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b190a8b40 0x556b190c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:52:20.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.865+0000 7f2d7491d640 1 --2- 172.21.15.164:0/195386710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b190b2370 0x556b190c4f00 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f2d6002a940 tx=0x7f2d6003f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf4fed79d3a37868 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:20.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.865+0000 7f2d58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:52:20.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.865+0000 7f2d58ff9640 1 -- 172.21.15.164:0/195386710 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60035030 con 0x556b190b2370 2024-09-19T08:52:20.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.865+0000 7f2d58ff9640 1 -- 172.21.15.164:0/195386710 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f2d600037f0 con 0x556b190b2370 2024-09-19T08:52:20.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.866+0000 7f2d7975b740 1 -- 172.21.15.164:0/195386710 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d60050070 con 0x556b190b2370 2024-09-19T08:52:20.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.866+0000 7f2d58ff9640 1 -- 172.21.15.164:0/195386710 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2d600039f0 con 0x556b190b2370 2024-09-19T08:52:20.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.866+0000 7f2d7975b740 10 client.16821 fetch_fsmap learned FSMap version 31 2024-09-19T08:52:20.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.866+0000 7f2d7975b740 1 -- 172.21.15.164:0/195386710 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556b1912c820 con 0x556b190b2370 2024-09-19T08:52:20.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.866+0000 7f2d58ff9640 1 -- 172.21.15.164:0/195386710 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d60050070 con 0x556b190b2370 2024-09-19T08:52:20.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.866+0000 7f2d58ff9640 1 -- 172.21.15.164:0/195386710 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f2d60070050 con 0x556b190b2370 2024-09-19T08:52:20.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.866+0000 7f2d7975b740 10 client.16821 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T08:52:20.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.866+0000 7f2d7975b740 10 client.16821 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T08:52:20.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.866+0000 7f2d7975b740 20 client.16821 populate_metadata read hostname 'smithi164' 2024-09-19T08:52:20.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.866+0000 7f2d7975b740 1 --2- 172.21.15.164:0/195386710 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x556b1912ccc0 0x556b1912f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:20.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.866+0000 7f2d7975b740 4 client.16821 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:52:20.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.866+0000 7f2d7975b740 1 -- 172.21.15.164:0/195386710 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b19014d70 con 0x556b1912ccc0 2024-09-19T08:52:20.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.866+0000 7f2d7511e640 1 --2- 172.21.15.164:0/195386710 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x556b1912ccc0 0x556b1912f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:20.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.867+0000 7f2d58ff9640 10 client.16821 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:52:20.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.867+0000 7f2d7511e640 1 --2- 172.21.15.164:0/195386710 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x556b1912ccc0 0x556b1912f0e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d14194220bc59ef6 server_cookie=4602787a805ed152 in_seq=0 out_seq=0 2024-09-19T08:52:20.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.867+0000 7f2d58ff9640 1 -- 172.21.15.164:0/195386710 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b19014d70 con 0x556b1912ccc0 2024-09-19T08:52:20.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.867+0000 7f2d58ff9640 10 client.16821 handle_command_reply: tid=0 2024-09-19T08:52:20.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.867+0000 7f2d7975b740 1 client.16821 shutdown 2024-09-19T08:52:20.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.867+0000 7f2d7975b740 1 -- 172.21.15.164:0/195386710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b190b2370 msgr2=0x556b190c4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:20.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.867+0000 7f2d7975b740 1 --2- 172.21.15.164:0/195386710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b190b2370 0x556b190c4f00 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f2d6002a940 tx=0x7f2d6003f040 comp rx=0 tx=0).stop 2024-09-19T08:52:20.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.867+0000 7f2d7975b740 1 -- 172.21.15.164:0/195386710 shutdown_connections 2024-09-19T08:52:20.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.867+0000 7f2d7975b740 1 --2- 172.21.15.164:0/195386710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b190b2370 0x556b190c4f00 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:20.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.867+0000 7f2d7975b740 1 --2- 172.21.15.164:0/195386710 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556b190a8b40 0x556b190c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:20.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.867+0000 7f2d7975b740 1 --2- 172.21.15.164:0/195386710 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x556b1912ccc0 0x556b1912f0e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:20.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.867+0000 7f2d7975b740 1 --2- 172.21.15.164:0/195386710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b190a75d0 0x556b190c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:20.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.867+0000 7f2d7975b740 1 -- 172.21.15.164:0/195386710 >> 172.21.15.164:0/195386710 conn(0x556b19070310 msgr2=0x556b190bdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:20.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.867+0000 7f2d58ff9640 0 client.16821 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:52:20.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.867+0000 7f2d7975b740 1 -- 172.21.15.164:0/195386710 shutdown_connections 2024-09-19T08:52:20.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.867+0000 7f2d7975b740 1 -- 172.21.15.164:0/195386710 wait complete. 2024-09-19T08:52:20.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.867+0000 7f2d7975b740 20 client.16821 trim_cache size 0 max 16384 2024-09-19T08:52:20.875 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:52:20.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.868+0000 7f2d77382640 1 -- 172.21.15.164:0/1078257096 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f2d5c0a33b0 msgr2=0x7f2d5c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:20.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.868+0000 7f2d77382640 1 --2- 172.21.15.164:0/1078257096 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f2d5c0a33b0 0x7f2d5c0a5870 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2d68098b00 tx=0x7f2d68091b30 comp rx=0 tx=0).stop 2024-09-19T08:52:20.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.868+0000 7f2d77382640 1 -- 172.21.15.164:0/1078257096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d701082a0 msgr2=0x7f2d7019ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:20.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.868+0000 7f2d77382640 1 --2- 172.21.15.164:0/1078257096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d701082a0 0x7f2d7019ced0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2d6800eb10 tx=0x7f2d6800c2d0 comp rx=0 tx=0).stop 2024-09-19T08:52:20.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.868+0000 7f2d77382640 1 -- 172.21.15.164:0/1078257096 shutdown_connections 2024-09-19T08:52:20.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.868+0000 7f2d77382640 1 --2- 172.21.15.164:0/1078257096 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f2d5c0a33b0 0x7f2d5c0a5870 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:20.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.868+0000 7f2d77382640 1 --2- 172.21.15.164:0/1078257096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d701082a0 0x7f2d7019ced0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:20.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.868+0000 7f2d77382640 1 --2- 172.21.15.164:0/1078257096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d701036e0 0x7f2d70198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:20.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.868+0000 7f2d77382640 1 --2- 172.21.15.164:0/1078257096 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d70102d30 0x7f2d70198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:20.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.868+0000 7f2d77382640 1 -- 172.21.15.164:0/1078257096 >> 172.21.15.164:0/1078257096 conn(0x7f2d700fc7d0 msgr2=0x7f2d70105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:20.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.868+0000 7f2d77382640 1 -- 172.21.15.164:0/1078257096 shutdown_connections 2024-09-19T08:52:20.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:20.868+0000 7f2d77382640 1 -- 172.21.15.164:0/1078257096 wait complete. 2024-09-19T08:52:21.540 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T08:52:21.784 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:52:22.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.264+0000 7fdeb5dff640 1 -- 172.21.15.164:0/977218352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdeb0102790 msgr2=0x7fdeb01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:22.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.264+0000 7fdeb5dff640 1 --2- 172.21.15.164:0/977218352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdeb0102790 0x7fdeb01067e0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fdea4009900 tx=0x7fdea402a410 comp rx=0 tx=0).stop 2024-09-19T08:52:22.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.264+0000 7fdeb5dff640 1 -- 172.21.15.164:0/977218352 shutdown_connections 2024-09-19T08:52:22.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.264+0000 7fdeb5dff640 1 --2- 172.21.15.164:0/977218352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdeb0106f10 0x7fdeb010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.264+0000 7fdeb5dff640 1 --2- 172.21.15.164:0/977218352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdeb0102790 0x7fdeb01067e0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.264+0000 7fdeb5dff640 1 --2- 172.21.15.164:0/977218352 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeb0101de0 0x7fdeb01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.264+0000 7fdeb5dff640 1 -- 172.21.15.164:0/977218352 >> 172.21.15.164:0/977218352 conn(0x7fdeb00fd8c0 msgr2=0x7fdeb00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:22.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.264+0000 7fdeb5dff640 1 -- 172.21.15.164:0/977218352 shutdown_connections 2024-09-19T08:52:22.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.264+0000 7fdeb5dff640 1 -- 172.21.15.164:0/977218352 wait complete. 2024-09-19T08:52:22.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.265+0000 7fdeb5dff640 1 Processor -- start 2024-09-19T08:52:22.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.265+0000 7fdeb5dff640 1 -- start start 2024-09-19T08:52:22.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.265+0000 7fdeb5dff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdeb0101de0 0x7fdeb01983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:22.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.265+0000 7fdeb5dff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdeb0102790 0x7fdeb01988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:22.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.265+0000 7fdeb5dff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeb0106f10 0x7fdeb019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:22.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.265+0000 7fdeb5dff640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb0111f70 con 0x7fdeb0101de0 2024-09-19T08:52:22.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.265+0000 7fdeb5dff640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb0111df0 con 0x7fdeb0106f10 2024-09-19T08:52:22.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.265+0000 7fdeb5dff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb01120f0 con 0x7fdeb0102790 2024-09-19T08:52:22.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.265+0000 7fdeaf7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdeb0101de0 0x7fdeb01983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:22.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.265+0000 7fdeaf7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdeb0101de0 0x7fdeb01983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46948/0 (socket says 172.21.15.164:46948) 2024-09-19T08:52:22.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.265+0000 7fdeaf7fe640 1 -- 172.21.15.164:0/4069460490 learned_addr learned my addr 172.21.15.164:0/4069460490 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:22.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.266+0000 7fdeaffff640 1 --2- 172.21.15.164:0/4069460490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeb0106f10 0x7fdeb019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:22.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.266+0000 7fdeaeffd640 1 --2- 172.21.15.164:0/4069460490 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdeb0102790 0x7fdeb01988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:22.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.266+0000 7fdeaf7fe640 1 -- 172.21.15.164:0/4069460490 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdeb0102790 msgr2=0x7fdeb01988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:22.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.266+0000 7fdeaf7fe640 1 --2- 172.21.15.164:0/4069460490 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdeb0102790 0x7fdeb01988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.266+0000 7fdeaf7fe640 1 -- 172.21.15.164:0/4069460490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeb0106f10 msgr2=0x7fdeb019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:22.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.266+0000 7fdeaf7fe640 1 --2- 172.21.15.164:0/4069460490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeb0106f10 0x7fdeb019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.266+0000 7fdeaf7fe640 1 -- 172.21.15.164:0/4069460490 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb019d360 con 0x7fdeb0101de0 2024-09-19T08:52:22.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.266+0000 7fdeaffff640 1 --2- 172.21.15.164:0/4069460490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeb0106f10 0x7fdeb019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:52:22.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.266+0000 7fdeaf7fe640 1 --2- 172.21.15.164:0/4069460490 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdeb0101de0 0x7fdeb01983b0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fdea000d950 tx=0x7fdea000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:22.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.266+0000 7fdeacff9640 1 -- 172.21.15.164:0/4069460490 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0017070 con 0x7fdeb0101de0 2024-09-19T08:52:22.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.266+0000 7fdeacff9640 1 -- 172.21.15.164:0/4069460490 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdea0009c30 con 0x7fdeb0101de0 2024-09-19T08:52:22.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.266+0000 7fdeb5dff640 1 -- 172.21.15.164:0/4069460490 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeb019d650 con 0x7fdeb0101de0 2024-09-19T08:52:22.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.266+0000 7fdeacff9640 1 -- 172.21.15.164:0/4069460490 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea000f6b0 con 0x7fdeb0101de0 2024-09-19T08:52:22.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.266+0000 7fdeb5dff640 1 -- 172.21.15.164:0/4069460490 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeb01a4f30 con 0x7fdeb0101de0 2024-09-19T08:52:22.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.268+0000 7fdeacff9640 1 -- 172.21.15.164:0/4069460490 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fdea00109d0 con 0x7fdeb0101de0 2024-09-19T08:52:22.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.269+0000 7fdeb81d8740 1 Processor -- start 2024-09-19T08:52:22.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.269+0000 7fdeb81d8740 1 -- start start 2024-09-19T08:52:22.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.269+0000 7fdeb81d8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559806b07b40 0x559806b06090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:22.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.269+0000 7fdeb81d8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559806b065d0 0x559806b06a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:22.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.269+0000 7fdeb81d8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559806b11370 0x559806b13760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:22.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.269+0000 7fdeb81d8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559806a6c2f0 con 0x559806b065d0 2024-09-19T08:52:22.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.269+0000 7fdeb81d8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x559806a13cf0 con 0x559806b07b40 2024-09-19T08:52:22.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.269+0000 7fdeb81d8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559806a682b0 con 0x559806b11370 2024-09-19T08:52:22.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.269+0000 7fdeaf7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559806b065d0 0x559806b06a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:22.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.269+0000 7fdeaffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559806b07b40 0x559806b06090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:22.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.269+0000 7fdeaffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559806b07b40 0x559806b06090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:44378/0 (socket says 172.21.15.164:44378) 2024-09-19T08:52:22.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.269+0000 7fdeaffff640 1 -- 172.21.15.164:0/4279576993 learned_addr learned my addr 172.21.15.164:0/4279576993 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:22.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.269+0000 7fdeaf7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559806b065d0 0x559806b06a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46958/0 (socket says 172.21.15.164:46958) 2024-09-19T08:52:22.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.269+0000 7fdeacff9640 1 --2- 172.21.15.164:0/4069460490 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fde980a33b0 0x7fde980a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:22.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.269+0000 7fdeaeffd640 1 --2- 172.21.15.164:0/4279576993 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559806b11370 0x559806b13760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:22.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.270+0000 7fdeaffff640 1 --2- 172.21.15.164:0/4069460490 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fde980a33b0 0x7fde980a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:22.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.270+0000 7fdeacff9640 1 -- 172.21.15.164:0/4069460490 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7fdea00cfa40 con 0x7fdeb0101de0 2024-09-19T08:52:22.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.270+0000 7fdeaf7fe640 1 -- 172.21.15.164:0/4279576993 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559806b11370 msgr2=0x559806b13760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:22.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.270+0000 7fdeaf7fe640 1 --2- 172.21.15.164:0/4279576993 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559806b11370 0x559806b13760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.270+0000 7fdeaf7fe640 1 -- 172.21.15.164:0/4279576993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559806b07b40 msgr2=0x559806b06090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:22.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.270+0000 7fdeaf7fe640 1 --2- 172.21.15.164:0/4279576993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559806b07b40 0x559806b06090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.270+0000 7fdeaf7fe640 1 -- 172.21.15.164:0/4279576993 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598069f83a0 con 0x559806b065d0 2024-09-19T08:52:22.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.270+0000 7fdeaf7fe640 1 --2- 172.21.15.164:0/4279576993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559806b065d0 0x559806b06a10 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fdea00cd170 tx=0x7fdea00979a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d4c3955d9773a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:22.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.270+0000 7fdeaffff640 1 --2- 172.21.15.164:0/4069460490 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fde980a33b0 0x7fde980a5870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdea8014c40 tx=0x7fdea8004ab0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:22.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.270+0000 7fde95ffb640 1 -- 172.21.15.164:0/4279576993 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0020030 con 0x559806b065d0 2024-09-19T08:52:22.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.270+0000 7fde95ffb640 1 -- 172.21.15.164:0/4279576993 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdea0009de0 con 0x559806b065d0 2024-09-19T08:52:22.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.270+0000 7fde95ffb640 1 -- 172.21.15.164:0/4279576993 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea00cd3f0 con 0x559806b065d0 2024-09-19T08:52:22.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.271+0000 7fdeb81d8740 1 -- 172.21.15.164:0/4279576993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559806b065d0 msgr2=0x559806b06a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:22.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.271+0000 7fdeb81d8740 1 --2- 172.21.15.164:0/4279576993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559806b065d0 0x559806b06a10 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fdea00cd170 tx=0x7fdea00979a0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.271+0000 7fdeb81d8740 1 -- 172.21.15.164:0/4279576993 shutdown_connections 2024-09-19T08:52:22.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.271+0000 7fdeb81d8740 1 --2- 172.21.15.164:0/4279576993 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559806b11370 0x559806b13760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.271+0000 7fdeb81d8740 1 --2- 172.21.15.164:0/4279576993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559806b065d0 0x559806b06a10 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.271+0000 7fdeb81d8740 1 --2- 172.21.15.164:0/4279576993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559806b07b40 0x559806b06090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.271+0000 7fdeb81d8740 1 -- 172.21.15.164:0/4279576993 >> 172.21.15.164:0/4279576993 conn(0x559806acf310 msgr2=0x559806b02c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:22.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.271+0000 7fdeb81d8740 1 -- 172.21.15.164:0/4279576993 shutdown_connections 2024-09-19T08:52:22.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.271+0000 7fdeb81d8740 1 -- 172.21.15.164:0/4279576993 wait complete. 2024-09-19T08:52:22.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.271+0000 7fdeb81d8740 1 -- start start 2024-09-19T08:52:22.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.272+0000 7fdeb81d8740 1 Processor -- start 2024-09-19T08:52:22.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.272+0000 7fdeb81d8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559806b065d0 0x559806b10db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:22.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.272+0000 7fdeb81d8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559806b07b40 0x559806b0e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:22.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.272+0000 7fdeb81d8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559806b11370 0x559806b0e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:22.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.272+0000 7fdeb81d8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559806b13fb0 con 0x559806b065d0 2024-09-19T08:52:22.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.272+0000 7fdeb81d8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x559806a27060 con 0x559806b11370 2024-09-19T08:52:22.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.272+0000 7fdeb81d8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559806b14130 con 0x559806b07b40 2024-09-19T08:52:22.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.272+0000 7fdeb81d8740 10 client.0 _ll_register_callbacks cb 0x55980677a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:52:22.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.272+0000 7fdeaf7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559806b065d0 0x559806b10db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:22.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.272+0000 7fdeaf7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559806b065d0 0x559806b10db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46962/0 (socket says 172.21.15.164:46962) 2024-09-19T08:52:22.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.272+0000 7fdeaf7fe640 1 -- 172.21.15.164:0/2883121180 learned_addr learned my addr 172.21.15.164:0/2883121180 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:22.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.272+0000 7fdeaffff640 1 --2- 172.21.15.164:0/2883121180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559806b11370 0x559806b0e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:22.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.272+0000 7fdeaeffd640 1 --2- 172.21.15.164:0/2883121180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559806b07b40 0x559806b0e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:22.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.272+0000 7fdeaf7fe640 1 -- 172.21.15.164:0/2883121180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559806b07b40 msgr2=0x559806b0e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:22.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.272+0000 7fdeaf7fe640 1 --2- 172.21.15.164:0/2883121180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559806b07b40 0x559806b0e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.272+0000 7fdeaf7fe640 1 -- 172.21.15.164:0/2883121180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559806b11370 msgr2=0x559806b0e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:22.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.272+0000 7fdeaf7fe640 1 --2- 172.21.15.164:0/2883121180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559806b11370 0x559806b0e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.272+0000 7fdeaf7fe640 1 -- 172.21.15.164:0/2883121180 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598067dd3c0 con 0x559806b065d0 2024-09-19T08:52:22.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.272+0000 7fdeaffff640 1 --2- 172.21.15.164:0/2883121180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559806b11370 0x559806b0e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:52:22.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.272+0000 7fdeaf7fe640 1 --2- 172.21.15.164:0/2883121180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559806b065d0 0x559806b10db0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fdea00986d0 tx=0x7fdea0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91888b9b70eb4e46 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:22.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.272+0000 7fde8effd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:52:22.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.273+0000 7fde8effd640 1 -- 172.21.15.164:0/2883121180 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0020070 con 0x559806b065d0 2024-09-19T08:52:22.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.273+0000 7fde8effd640 1 -- 172.21.15.164:0/2883121180 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7fdea009b150 con 0x559806b065d0 2024-09-19T08:52:22.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.273+0000 7fdeb81d8740 1 -- 172.21.15.164:0/2883121180 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdea0009de0 con 0x559806b065d0 2024-09-19T08:52:22.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.273+0000 7fde8effd640 1 -- 172.21.15.164:0/2883121180 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdea0097fa0 con 0x559806b065d0 2024-09-19T08:52:22.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.273+0000 7fdeb81d8740 10 client.16833 fetch_fsmap learned FSMap version 31 2024-09-19T08:52:22.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.273+0000 7fdeb81d8740 1 -- 172.21.15.164:0/2883121180 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559806b1faa0 con 0x559806b065d0 2024-09-19T08:52:22.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.273+0000 7fde8effd640 1 -- 172.21.15.164:0/2883121180 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdea0009de0 con 0x559806b065d0 2024-09-19T08:52:22.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.273+0000 7fde8effd640 1 -- 172.21.15.164:0/2883121180 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7fdea0010460 con 0x559806b065d0 2024-09-19T08:52:22.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.273+0000 7fdeb81d8740 10 client.16833 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T08:52:22.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.273+0000 7fdeb81d8740 10 client.16833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:52:22.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.273+0000 7fdeb81d8740 20 client.16833 populate_metadata read hostname 'smithi164' 2024-09-19T08:52:22.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.273+0000 7fdeb81d8740 1 --2- 172.21.15.164:0/2883121180 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x559806b20df0 0x559806b231e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:22.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.273+0000 7fdeb81d8740 4 client.16833 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:52:22.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.273+0000 7fdeb81d8740 1 -- 172.21.15.164:0/2883121180 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559806a79860 con 0x559806b20df0 2024-09-19T08:52:22.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.273+0000 7fdeaeffd640 1 --2- 172.21.15.164:0/2883121180 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x559806b20df0 0x559806b231e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:22.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.274+0000 7fde8effd640 10 client.16833 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:52:22.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.274+0000 7fdeaeffd640 1 --2- 172.21.15.164:0/2883121180 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x559806b20df0 0x559806b231e0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bad13cf712f923 server_cookie=a0694e8441a0efe3 in_seq=0 out_seq=0 2024-09-19T08:52:22.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.275+0000 7fde8effd640 1 -- 172.21.15.164:0/2883121180 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559806a79860 con 0x559806b20df0 2024-09-19T08:52:22.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.275+0000 7fde8effd640 10 client.16833 handle_command_reply: tid=0 2024-09-19T08:52:22.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.275+0000 7fdeb81d8740 1 client.16833 shutdown 2024-09-19T08:52:22.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.275+0000 7fdeb81d8740 1 -- 172.21.15.164:0/2883121180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559806b065d0 msgr2=0x559806b10db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:22.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.275+0000 7fdeb81d8740 1 --2- 172.21.15.164:0/2883121180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559806b065d0 0x559806b10db0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fdea00986d0 tx=0x7fdea0000bc0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.275+0000 7fdeb81d8740 1 -- 172.21.15.164:0/2883121180 shutdown_connections 2024-09-19T08:52:22.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.275+0000 7fdeb81d8740 1 --2- 172.21.15.164:0/2883121180 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x559806b20df0 0x559806b231e0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.275+0000 7fdeb81d8740 1 --2- 172.21.15.164:0/2883121180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559806b11370 0x559806b0e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.275+0000 7fde8effd640 0 client.16833 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:52:22.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.275+0000 7fdeb81d8740 1 --2- 172.21.15.164:0/2883121180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559806b07b40 0x559806b0e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.275+0000 7fdeb81d8740 1 --2- 172.21.15.164:0/2883121180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559806b065d0 0x559806b10db0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.275+0000 7fdeb81d8740 1 -- 172.21.15.164:0/2883121180 >> 172.21.15.164:0/2883121180 conn(0x559806acf310 msgr2=0x559806b08990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:22.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.275+0000 7fdeb81d8740 1 -- 172.21.15.164:0/2883121180 shutdown_connections 2024-09-19T08:52:22.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.275+0000 7fdeb81d8740 1 -- 172.21.15.164:0/2883121180 wait complete. 2024-09-19T08:52:22.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.276+0000 7fdeb81d8740 20 client.16833 trim_cache size 0 max 16384 2024-09-19T08:52:22.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.288+0000 7fdeb81d8740 1 Processor -- start 2024-09-19T08:52:22.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.288+0000 7fdeb81d8740 1 -- start start 2024-09-19T08:52:22.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.289+0000 7fdeb81d8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559806b065d0 0x559806b0e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:22.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.289+0000 7fdeb81d8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559806b07b40 0x559806b287b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:22.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.289+0000 7fdeb81d8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559806b11370 0x559806b28cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:22.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.289+0000 7fdeb81d8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb01ad6f0 con 0x559806b11370 2024-09-19T08:52:22.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.289+0000 7fdeb81d8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb01af260 con 0x559806b07b40 2024-09-19T08:52:22.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.289+0000 7fdeb81d8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb01addc0 con 0x559806b065d0 2024-09-19T08:52:22.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.289+0000 7fdeaffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559806b11370 0x559806b28cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:22.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.289+0000 7fdeaeffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559806b07b40 0x559806b287b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:22.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.289+0000 7fdeaffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559806b11370 0x559806b28cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46966/0 (socket says 172.21.15.164:46966) 2024-09-19T08:52:22.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.289+0000 7fdeaeffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559806b07b40 0x559806b287b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:44398/0 (socket says 172.21.15.164:44398) 2024-09-19T08:52:22.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.289+0000 7fdeaffff640 1 -- 172.21.15.164:0/1733329735 learned_addr learned my addr 172.21.15.164:0/1733329735 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:22.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.289+0000 7fdeaf7fe640 1 --2- 172.21.15.164:0/1733329735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559806b065d0 0x559806b0e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:22.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.289+0000 7fdeaffff640 1 -- 172.21.15.164:0/1733329735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559806b065d0 msgr2=0x559806b0e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:22.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.289+0000 7fdeaffff640 1 --2- 172.21.15.164:0/1733329735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559806b065d0 0x559806b0e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.289+0000 7fdeaffff640 1 -- 172.21.15.164:0/1733329735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559806b07b40 msgr2=0x559806b287b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:22.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.289+0000 7fdeaffff640 1 --2- 172.21.15.164:0/1733329735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559806b07b40 0x559806b287b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.289+0000 7fdeaffff640 1 -- 172.21.15.164:0/1733329735 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559806b09120 con 0x559806b11370 2024-09-19T08:52:22.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.289+0000 7fdeaf7fe640 1 --2- 172.21.15.164:0/1733329735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559806b065d0 0x559806b0e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:52:22.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.289+0000 7fdeaeffd640 1 --2- 172.21.15.164:0/1733329735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559806b07b40 0x559806b287b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:52:22.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.289+0000 7fdeaffff640 1 --2- 172.21.15.164:0/1733329735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559806b11370 0x559806b28cf0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fdea8014dd0 tx=0x7fdea80087c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f2fe8c9a82328d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:22.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.290+0000 7fde957fa640 1 -- 172.21.15.164:0/1733329735 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea8013d70 con 0x559806b11370 2024-09-19T08:52:22.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.290+0000 7fde957fa640 1 -- 172.21.15.164:0/1733329735 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdea8005020 con 0x559806b11370 2024-09-19T08:52:22.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.290+0000 7fde957fa640 1 -- 172.21.15.164:0/1733329735 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea8021680 con 0x559806b11370 2024-09-19T08:52:22.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.290+0000 7fdeb81d8740 1 -- 172.21.15.164:0/1733329735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559806b11370 msgr2=0x559806b28cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:22.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.290+0000 7fdeb81d8740 1 --2- 172.21.15.164:0/1733329735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559806b11370 0x559806b28cf0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fdea8014dd0 tx=0x7fdea80087c0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.290+0000 7fdeb81d8740 1 -- 172.21.15.164:0/1733329735 shutdown_connections 2024-09-19T08:52:22.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.290+0000 7fdeb81d8740 1 --2- 172.21.15.164:0/1733329735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559806b11370 0x559806b28cf0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.290+0000 7fdeb81d8740 1 --2- 172.21.15.164:0/1733329735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559806b07b40 0x559806b287b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.290+0000 7fdeb81d8740 1 --2- 172.21.15.164:0/1733329735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559806b065d0 0x559806b0e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.290+0000 7fdeb81d8740 1 -- 172.21.15.164:0/1733329735 >> 172.21.15.164:0/1733329735 conn(0x559806acf310 msgr2=0x559806b0d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:22.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.290+0000 7fdeb81d8740 1 -- 172.21.15.164:0/1733329735 shutdown_connections 2024-09-19T08:52:22.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.290+0000 7fdeb81d8740 1 -- 172.21.15.164:0/1733329735 wait complete. 2024-09-19T08:52:22.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.291+0000 7fdeb81d8740 1 -- start start 2024-09-19T08:52:22.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.291+0000 7fdeb81d8740 1 Processor -- start 2024-09-19T08:52:22.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.291+0000 7fdeb81d8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559806b065d0 0x559806b19480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:22.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.291+0000 7fdeb81d8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559806b07b40 0x559806b199c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:22.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.291+0000 7fdeb81d8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559806b11370 0x559806b14900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:22.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.291+0000 7fdeb81d8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb01aeb60 con 0x559806b11370 2024-09-19T08:52:22.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.291+0000 7fdeb81d8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb01ae490 con 0x559806b07b40 2024-09-19T08:52:22.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.291+0000 7fdeb81d8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb01ab6f0 con 0x559806b065d0 2024-09-19T08:52:22.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.291+0000 7fdeb81d8740 10 client.0 _ll_register_callbacks cb 0x7fdea8008660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:52:22.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.291+0000 7fdeaffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559806b11370 0x559806b14900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:22.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.291+0000 7fdeaffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559806b11370 0x559806b14900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46974/0 (socket says 172.21.15.164:46974) 2024-09-19T08:52:22.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.291+0000 7fdeaffff640 1 -- 172.21.15.164:0/802505881 learned_addr learned my addr 172.21.15.164:0/802505881 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:22.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.292+0000 7fdeaf7fe640 1 --2- 172.21.15.164:0/802505881 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559806b065d0 0x559806b19480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:22.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.292+0000 7fdeaeffd640 1 --2- 172.21.15.164:0/802505881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559806b07b40 0x559806b199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:22.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.292+0000 7fdeaffff640 1 -- 172.21.15.164:0/802505881 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559806b065d0 msgr2=0x559806b19480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:22.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.292+0000 7fdeaffff640 1 --2- 172.21.15.164:0/802505881 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559806b065d0 0x559806b19480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.292+0000 7fdeaffff640 1 -- 172.21.15.164:0/802505881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559806b07b40 msgr2=0x559806b199c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:22.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.292+0000 7fdeaffff640 1 --2- 172.21.15.164:0/802505881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559806b07b40 0x559806b199c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.292+0000 7fdeaffff640 1 -- 172.21.15.164:0/802505881 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559806b15130 con 0x559806b11370 2024-09-19T08:52:22.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.292+0000 7fdeaffff640 1 --2- 172.21.15.164:0/802505881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559806b11370 0x559806b14900 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fdea8005cc0 tx=0x7fdea80144e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c48f064c01941014 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:22.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.292+0000 7fde8effd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:52:22.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.292+0000 7fde8effd640 1 -- 172.21.15.164:0/802505881 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea8036060 con 0x559806b11370 2024-09-19T08:52:22.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.292+0000 7fde8effd640 1 -- 172.21.15.164:0/802505881 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7fdea8004d30 con 0x559806b11370 2024-09-19T08:52:22.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.292+0000 7fdeb81d8740 1 -- 172.21.15.164:0/802505881 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdea8005020 con 0x559806b11370 2024-09-19T08:52:22.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.292+0000 7fde8effd640 1 -- 172.21.15.164:0/802505881 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdea8008c50 con 0x559806b11370 2024-09-19T08:52:22.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.292+0000 7fdeb81d8740 10 client.16839 fetch_fsmap learned FSMap version 31 2024-09-19T08:52:22.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.292+0000 7fdeb81d8740 1 -- 172.21.15.164:0/802505881 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559806b8b260 con 0x559806b11370 2024-09-19T08:52:22.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.293+0000 7fde8effd640 1 -- 172.21.15.164:0/802505881 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdea8005020 con 0x559806b11370 2024-09-19T08:52:22.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.293+0000 7fde8effd640 1 -- 172.21.15.164:0/802505881 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7fdea8021a90 con 0x559806b11370 2024-09-19T08:52:22.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.293+0000 7fdeb81d8740 10 client.16839 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T08:52:22.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.293+0000 7fdeb81d8740 10 client.16839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:52:22.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.293+0000 7fdeb81d8740 20 client.16839 populate_metadata read hostname 'smithi164' 2024-09-19T08:52:22.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.293+0000 7fdeb81d8740 1 --2- 172.21.15.164:0/802505881 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x559806b8b960 0x559806b8dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:22.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.293+0000 7fdeb81d8740 4 client.16839 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:52:22.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.293+0000 7fdeb81d8740 1 -- 172.21.15.164:0/802505881 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559806a73d70 con 0x559806b8b960 2024-09-19T08:52:22.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.293+0000 7fdeaf7fe640 1 --2- 172.21.15.164:0/802505881 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x559806b8b960 0x559806b8dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:22.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.293+0000 7fdeaf7fe640 1 --2- 172.21.15.164:0/802505881 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x559806b8b960 0x559806b8dd80 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32a23e8fa8ee140 server_cookie=96741b55e78f4ebb in_seq=0 out_seq=0 2024-09-19T08:52:22.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.293+0000 7fde8effd640 10 client.16839 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:52:22.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.294+0000 7fde8effd640 1 -- 172.21.15.164:0/802505881 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559806a73d70 con 0x559806b8b960 2024-09-19T08:52:22.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.294+0000 7fde8effd640 10 client.16839 handle_command_reply: tid=0 2024-09-19T08:52:22.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.294+0000 7fdeb81d8740 1 client.16839 shutdown 2024-09-19T08:52:22.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.294+0000 7fdeb81d8740 1 -- 172.21.15.164:0/802505881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559806b11370 msgr2=0x559806b14900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:22.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.294+0000 7fdeb81d8740 1 --2- 172.21.15.164:0/802505881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559806b11370 0x559806b14900 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fdea8005cc0 tx=0x7fdea80144e0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.294+0000 7fdeb81d8740 1 -- 172.21.15.164:0/802505881 shutdown_connections 2024-09-19T08:52:22.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.294+0000 7fdeb81d8740 1 --2- 172.21.15.164:0/802505881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559806b11370 0x559806b14900 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.294+0000 7fdeb81d8740 1 --2- 172.21.15.164:0/802505881 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x559806b8b960 0x559806b8dd80 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.294+0000 7fdeb81d8740 1 --2- 172.21.15.164:0/802505881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559806b07b40 0x559806b199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.294+0000 7fdeb81d8740 1 --2- 172.21.15.164:0/802505881 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559806b065d0 0x559806b19480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.294+0000 7fdeb81d8740 1 -- 172.21.15.164:0/802505881 >> 172.21.15.164:0/802505881 conn(0x559806acf310 msgr2=0x559806b1c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:22.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.294+0000 7fde8effd640 0 client.16839 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:52:22.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.294+0000 7fdeb81d8740 1 -- 172.21.15.164:0/802505881 shutdown_connections 2024-09-19T08:52:22.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.294+0000 7fdeb81d8740 1 -- 172.21.15.164:0/802505881 wait complete. 2024-09-19T08:52:22.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.294+0000 7fdeb81d8740 20 client.16839 trim_cache size 0 max 16384 2024-09-19T08:52:22.299 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:52:22.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.294+0000 7fdeb5dff640 1 -- 172.21.15.164:0/4069460490 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fde980a33b0 msgr2=0x7fde980a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:22.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.294+0000 7fdeb5dff640 1 --2- 172.21.15.164:0/4069460490 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fde980a33b0 0x7fde980a5870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdea8014c40 tx=0x7fdea8004ab0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.295+0000 7fdeb5dff640 1 -- 172.21.15.164:0/4069460490 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdeb0101de0 msgr2=0x7fdeb01983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:22.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.295+0000 7fdeb5dff640 1 --2- 172.21.15.164:0/4069460490 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdeb0101de0 0x7fdeb01983b0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fdea000d950 tx=0x7fdea000de10 comp rx=0 tx=0).stop 2024-09-19T08:52:22.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.295+0000 7fdeb5dff640 1 -- 172.21.15.164:0/4069460490 shutdown_connections 2024-09-19T08:52:22.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.295+0000 7fdeb5dff640 1 --2- 172.21.15.164:0/4069460490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeb0106f10 0x7fdeb019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.295+0000 7fdeb5dff640 1 --2- 172.21.15.164:0/4069460490 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdeb0102790 0x7fdeb01988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.295+0000 7fdeb5dff640 1 --2- 172.21.15.164:0/4069460490 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fde980a33b0 0x7fde980a5870 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.295+0000 7fdeb5dff640 1 --2- 172.21.15.164:0/4069460490 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdeb0101de0 0x7fdeb01983b0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:22.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.295+0000 7fdeb5dff640 1 -- 172.21.15.164:0/4069460490 >> 172.21.15.164:0/4069460490 conn(0x7fdeb00fd8c0 msgr2=0x7fdeb00fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:22.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.295+0000 7fdeb5dff640 1 -- 172.21.15.164:0/4069460490 shutdown_connections 2024-09-19T08:52:22.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:22.295+0000 7fdeb5dff640 1 -- 172.21.15.164:0/4069460490 wait complete. 2024-09-19T08:52:22.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:22 smithi164 ceph-mon[30928]: pgmap v201: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 230 MiB/s wr, 913 op/s 2024-09-19T08:52:22.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:22 smithi170 ceph-mon[35072]: pgmap v201: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 230 MiB/s wr, 913 op/s 2024-09-19T08:52:22.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:22 smithi173 ceph-mon[35306]: pgmap v201: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 230 MiB/s wr, 913 op/s 2024-09-19T08:52:23.973 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T08:52:23.974 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T08:52:24.228 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:52:24.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:24 smithi164 ceph-mon[30928]: pgmap v202: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 200 MiB/s wr, 798 op/s 2024-09-19T08:52:24.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:24 smithi170 ceph-mon[35072]: pgmap v202: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 200 MiB/s wr, 798 op/s 2024-09-19T08:52:24.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:24 smithi173 ceph-mon[35306]: pgmap v202: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 200 MiB/s wr, 798 op/s 2024-09-19T08:52:24.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.743+0000 7fd352e02640 1 -- 172.21.15.164:0/584508052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd34c106a80 msgr2=0x7fd34c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:24.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.743+0000 7fd352e02640 1 --2- 172.21.15.164:0/584508052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd34c106a80 0x7fd34c106e60 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fd340009930 tx=0x7fd34002a420 comp rx=0 tx=0).stop 2024-09-19T08:52:24.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.743+0000 7fd352e02640 1 -- 172.21.15.164:0/584508052 shutdown_connections 2024-09-19T08:52:24.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.743+0000 7fd352e02640 1 --2- 172.21.15.164:0/584508052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd34c101420 0x7fd34c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.743+0000 7fd352e02640 1 --2- 172.21.15.164:0/584508052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd34c100a80 0x7fd34c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.743+0000 7fd352e02640 1 --2- 172.21.15.164:0/584508052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd34c106a80 0x7fd34c106e60 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.743+0000 7fd352e02640 1 -- 172.21.15.164:0/584508052 >> 172.21.15.164:0/584508052 conn(0x7fd34c0fc7a0 msgr2=0x7fd34c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:24.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.743+0000 7fd352e02640 1 -- 172.21.15.164:0/584508052 shutdown_connections 2024-09-19T08:52:24.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.743+0000 7fd352e02640 1 -- 172.21.15.164:0/584508052 wait complete. 2024-09-19T08:52:24.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.744+0000 7fd352e02640 1 Processor -- start 2024-09-19T08:52:24.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.744+0000 7fd352e02640 1 -- start start 2024-09-19T08:52:24.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.744+0000 7fd352e02640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd34c100a80 0x7fd34c102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:24.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.744+0000 7fd352e02640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd34c101420 0x7fd34c102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:24.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.744+0000 7fd352e02640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd34c106a80 0x7fd34c1030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:24.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.744+0000 7fd352e02640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c110f20 con 0x7fd34c106a80 2024-09-19T08:52:24.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.744+0000 7fd352e02640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c110da0 con 0x7fd34c100a80 2024-09-19T08:52:24.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.744+0000 7fd352e02640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1110a0 con 0x7fd34c101420 2024-09-19T08:52:24.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.744+0000 7fd35139f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd34c106a80 0x7fd34c1030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:24.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.744+0000 7fd35139f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd34c106a80 0x7fd34c1030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39866/0 (socket says 172.21.15.164:39866) 2024-09-19T08:52:24.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.744+0000 7fd35139f640 1 -- 172.21.15.164:0/2483786765 learned_addr learned my addr 172.21.15.164:0/2483786765 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:24.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.744+0000 7fd34bfff640 1 --2- 172.21.15.164:0/2483786765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd34c101420 0x7fd34c102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:24.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.744+0000 7fd350b9e640 1 --2- 172.21.15.164:0/2483786765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd34c100a80 0x7fd34c102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:24.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.744+0000 7fd35139f640 1 -- 172.21.15.164:0/2483786765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd34c101420 msgr2=0x7fd34c102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:24.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.744+0000 7fd35139f640 1 --2- 172.21.15.164:0/2483786765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd34c101420 0x7fd34c102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.744+0000 7fd35139f640 1 -- 172.21.15.164:0/2483786765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd34c100a80 msgr2=0x7fd34c102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:24.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.744+0000 7fd35139f640 1 --2- 172.21.15.164:0/2483786765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd34c100a80 0x7fd34c102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.744+0000 7fd35139f640 1 -- 172.21.15.164:0/2483786765 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34c1a6a80 con 0x7fd34c106a80 2024-09-19T08:52:24.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.745+0000 7fd35139f640 1 --2- 172.21.15.164:0/2483786765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd34c106a80 0x7fd34c1030d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fd34400b970 tx=0x7fd34400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:24.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.745+0000 7fd349ffb640 1 -- 172.21.15.164:0/2483786765 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd344019070 con 0x7fd34c106a80 2024-09-19T08:52:24.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.745+0000 7fd349ffb640 1 -- 172.21.15.164:0/2483786765 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd344014440 con 0x7fd34c106a80 2024-09-19T08:52:24.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.745+0000 7fd352e02640 1 -- 172.21.15.164:0/2483786765 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd34c1a6d70 con 0x7fd34c106a80 2024-09-19T08:52:24.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.745+0000 7fd349ffb640 1 -- 172.21.15.164:0/2483786765 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3440075d0 con 0x7fd34c106a80 2024-09-19T08:52:24.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.745+0000 7fd352e02640 1 -- 172.21.15.164:0/2483786765 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd34c1a72b0 con 0x7fd34c106a80 2024-09-19T08:52:24.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.747+0000 7fd349ffb640 1 -- 172.21.15.164:0/2483786765 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fd34401c070 con 0x7fd34c106a80 2024-09-19T08:52:24.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.747+0000 7fd3551db740 1 Processor -- start 2024-09-19T08:52:24.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.748+0000 7fd3551db740 1 -- start start 2024-09-19T08:52:24.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.748+0000 7fd349ffb640 1 --2- 172.21.15.164:0/2483786765 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd33c0a3430 0x7fd33c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:24.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.748+0000 7fd3551db740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1a6b18b60 0x55b1a6b18f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:24.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.748+0000 7fd3551db740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b1a6b19480 0x55b1a6b23800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:24.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.748+0000 7fd3551db740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a6b23d40 0x55b1a6b26130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:24.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.748+0000 7fd3551db740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b1a6a7e280 con 0x55b1a6b23d40 2024-09-19T08:52:24.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.748+0000 7fd3551db740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b1a6a25b10 con 0x55b1a6b18b60 2024-09-19T08:52:24.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.748+0000 7fd3551db740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b1a6a7a240 con 0x55b1a6b19480 2024-09-19T08:52:24.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.748+0000 7fd34bfff640 1 --2- 172.21.15.164:0/2483786765 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd33c0a3430 0x7fd33c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:24.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.748+0000 7fd349ffb640 1 -- 172.21.15.164:0/2483786765 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7fd3440cf0f0 con 0x7fd34c106a80 2024-09-19T08:52:24.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.748+0000 7fd35139f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a6b23d40 0x55b1a6b26130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:24.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.748+0000 7fd350b9e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1a6b18b60 0x55b1a6b18f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:24.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.748+0000 7fd35139f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a6b23d40 0x55b1a6b26130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39872/0 (socket says 172.21.15.164:39872) 2024-09-19T08:52:24.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.748+0000 7fd350b9e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1a6b18b60 0x55b1a6b18f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:34214/0 (socket says 172.21.15.164:34214) 2024-09-19T08:52:24.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.748+0000 7fd35139f640 1 -- 172.21.15.164:0/3147367255 learned_addr learned my addr 172.21.15.164:0/3147367255 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:24.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.748+0000 7fd34bfff640 1 --2- 172.21.15.164:0/3147367255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b1a6b19480 0x55b1a6b23800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:24.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.748+0000 7fd35139f640 1 -- 172.21.15.164:0/3147367255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b1a6b19480 msgr2=0x55b1a6b23800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:24.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.748+0000 7fd35139f640 1 --2- 172.21.15.164:0/3147367255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b1a6b19480 0x55b1a6b23800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.748+0000 7fd35139f640 1 -- 172.21.15.164:0/3147367255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1a6b18b60 msgr2=0x55b1a6b18f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:24.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.748+0000 7fd35139f640 1 --2- 172.21.15.164:0/3147367255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1a6b18b60 0x55b1a6b18f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.748+0000 7fd34bfff640 1 --2- 172.21.15.164:0/2483786765 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd33c0a3430 0x7fd33c0a58f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd34c1040e0 tx=0x7fd3380156d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:24.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.748+0000 7fd35139f640 1 -- 172.21.15.164:0/3147367255 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1a6a321f0 con 0x55b1a6b23d40 2024-09-19T08:52:24.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.749+0000 7fd35139f640 1 --2- 172.21.15.164:0/3147367255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a6b23d40 0x55b1a6b26130 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd344097a30 tx=0x7fd34409b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a8abc040a89e14 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:24.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.749+0000 7fd336ffd640 1 -- 172.21.15.164:0/3147367255 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3440b4070 con 0x55b1a6b23d40 2024-09-19T08:52:24.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.749+0000 7fd336ffd640 1 -- 172.21.15.164:0/3147367255 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3440149a0 con 0x55b1a6b23d40 2024-09-19T08:52:24.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.749+0000 7fd336ffd640 1 -- 172.21.15.164:0/3147367255 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3440a3d00 con 0x55b1a6b23d40 2024-09-19T08:52:24.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.749+0000 7fd3551db740 1 -- 172.21.15.164:0/3147367255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a6b23d40 msgr2=0x55b1a6b26130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:24.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.749+0000 7fd3551db740 1 --2- 172.21.15.164:0/3147367255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a6b23d40 0x55b1a6b26130 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd344097a30 tx=0x7fd34409b690 comp rx=0 tx=0).stop 2024-09-19T08:52:24.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.749+0000 7fd3551db740 1 -- 172.21.15.164:0/3147367255 shutdown_connections 2024-09-19T08:52:24.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.749+0000 7fd3551db740 1 --2- 172.21.15.164:0/3147367255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a6b23d40 0x55b1a6b26130 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.749+0000 7fd3551db740 1 --2- 172.21.15.164:0/3147367255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b1a6b19480 0x55b1a6b23800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.749+0000 7fd3551db740 1 --2- 172.21.15.164:0/3147367255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1a6b18b60 0x55b1a6b18f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.749+0000 7fd3551db740 1 -- 172.21.15.164:0/3147367255 >> 172.21.15.164:0/3147367255 conn(0x55b1a6b150f0 msgr2=0x55b1a6b154d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:24.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.750+0000 7fd3551db740 1 -- 172.21.15.164:0/3147367255 shutdown_connections 2024-09-19T08:52:24.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.750+0000 7fd3551db740 1 -- 172.21.15.164:0/3147367255 wait complete. 2024-09-19T08:52:24.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.750+0000 7fd3551db740 1 -- start start 2024-09-19T08:52:24.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.750+0000 7fd3551db740 1 Processor -- start 2024-09-19T08:52:24.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.750+0000 7fd3551db740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1a6b18b60 0x55b1a6b20c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:24.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.750+0000 7fd3551db740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b1a6b19480 0x55b1a6b21190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:24.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.750+0000 7fd3551db740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a6b23d40 0x55b1a6b216d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:24.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.750+0000 7fd3551db740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b1a6b26910 con 0x55b1a6b23d40 2024-09-19T08:52:24.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.750+0000 7fd3551db740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b1a6a38f10 con 0x55b1a6b18b60 2024-09-19T08:52:24.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.750+0000 7fd3551db740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b1a6b26a90 con 0x55b1a6b19480 2024-09-19T08:52:24.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.750+0000 7fd3551db740 10 client.0 _ll_register_callbacks cb 0x55b1a6a01a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:52:24.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.751+0000 7fd35139f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a6b23d40 0x55b1a6b216d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:24.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.751+0000 7fd35139f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a6b23d40 0x55b1a6b216d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39882/0 (socket says 172.21.15.164:39882) 2024-09-19T08:52:24.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.751+0000 7fd35139f640 1 -- 172.21.15.164:0/3250337485 learned_addr learned my addr 172.21.15.164:0/3250337485 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:24.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.751+0000 7fd350b9e640 1 --2- 172.21.15.164:0/3250337485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1a6b18b60 0x55b1a6b20c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:24.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.751+0000 7fd34bfff640 1 --2- 172.21.15.164:0/3250337485 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b1a6b19480 0x55b1a6b21190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:24.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.751+0000 7fd35139f640 1 -- 172.21.15.164:0/3250337485 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b1a6b19480 msgr2=0x55b1a6b21190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:24.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.751+0000 7fd35139f640 1 --2- 172.21.15.164:0/3250337485 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b1a6b19480 0x55b1a6b21190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.751+0000 7fd35139f640 1 -- 172.21.15.164:0/3250337485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1a6b18b60 msgr2=0x55b1a6b20c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:24.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.751+0000 7fd35139f640 1 --2- 172.21.15.164:0/3250337485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1a6b18b60 0x55b1a6b20c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.751+0000 7fd35139f640 1 -- 172.21.15.164:0/3250337485 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1a67ef6a0 con 0x55b1a6b23d40 2024-09-19T08:52:24.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.751+0000 7fd34bfff640 1 --2- 172.21.15.164:0/3250337485 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b1a6b19480 0x55b1a6b21190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:52:24.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.751+0000 7fd35139f640 1 --2- 172.21.15.164:0/3250337485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a6b23d40 0x55b1a6b216d0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fd3440cca40 tx=0x7fd3440979a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae9332264938f2e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:24.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.751+0000 7fd32bfff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:52:24.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.751+0000 7fd32bfff640 1 -- 172.21.15.164:0/3250337485 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3440b4040 con 0x55b1a6b23d40 2024-09-19T08:52:24.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.751+0000 7fd32bfff640 1 -- 172.21.15.164:0/3250337485 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7fd3440998d0 con 0x55b1a6b23d40 2024-09-19T08:52:24.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.751+0000 7fd3551db740 1 -- 172.21.15.164:0/3250337485 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3440149a0 con 0x55b1a6b23d40 2024-09-19T08:52:24.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.751+0000 7fd32bfff640 1 -- 172.21.15.164:0/3250337485 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd3440a3d00 con 0x55b1a6b23d40 2024-09-19T08:52:24.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.751+0000 7fd3551db740 10 client.16851 fetch_fsmap learned FSMap version 31 2024-09-19T08:52:24.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.751+0000 7fd3551db740 1 -- 172.21.15.164:0/3250337485 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b1a6b32540 con 0x55b1a6b23d40 2024-09-19T08:52:24.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.751+0000 7fd32bfff640 1 -- 172.21.15.164:0/3250337485 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3440149a0 con 0x55b1a6b23d40 2024-09-19T08:52:24.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.752+0000 7fd32bfff640 1 -- 172.21.15.164:0/3250337485 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7fd34400cbc0 con 0x55b1a6b23d40 2024-09-19T08:52:24.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.752+0000 7fd3551db740 10 client.16851 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T08:52:24.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.752+0000 7fd3551db740 10 client.16851 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:52:24.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.752+0000 7fd3551db740 20 client.16851 populate_metadata read hostname 'smithi164' 2024-09-19T08:52:24.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.752+0000 7fd3551db740 1 --2- 172.21.15.164:0/3250337485 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55b1a6b33890 0x55b1a6b35c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:24.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.752+0000 7fd3551db740 4 client.16851 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:52:24.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.752+0000 7fd3551db740 1 -- 172.21.15.164:0/3250337485 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1a6a8b7f0 con 0x55b1a6b33890 2024-09-19T08:52:24.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.752+0000 7fd350b9e640 1 --2- 172.21.15.164:0/3250337485 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55b1a6b33890 0x55b1a6b35c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:24.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.752+0000 7fd350b9e640 1 --2- 172.21.15.164:0/3250337485 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55b1a6b33890 0x55b1a6b35c80 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47ff48f8f223d36d server_cookie=38cc7c9f0ee22085 in_seq=0 out_seq=0 2024-09-19T08:52:24.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.752+0000 7fd32bfff640 10 client.16851 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:52:24.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.753+0000 7fd32bfff640 1 -- 172.21.15.164:0/3250337485 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b1a6a8b7f0 con 0x55b1a6b33890 2024-09-19T08:52:24.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.753+0000 7fd32bfff640 10 client.16851 handle_command_reply: tid=0 2024-09-19T08:52:24.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.753+0000 7fd3551db740 1 client.16851 shutdown 2024-09-19T08:52:24.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.753+0000 7fd3551db740 1 -- 172.21.15.164:0/3250337485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a6b23d40 msgr2=0x55b1a6b216d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:24.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.753+0000 7fd3551db740 1 --2- 172.21.15.164:0/3250337485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a6b23d40 0x55b1a6b216d0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fd3440cca40 tx=0x7fd3440979a0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.753+0000 7fd3551db740 1 -- 172.21.15.164:0/3250337485 shutdown_connections 2024-09-19T08:52:24.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.753+0000 7fd3551db740 1 --2- 172.21.15.164:0/3250337485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a6b23d40 0x55b1a6b216d0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.753+0000 7fd3551db740 1 --2- 172.21.15.164:0/3250337485 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b1a6b19480 0x55b1a6b21190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.753+0000 7fd3551db740 1 --2- 172.21.15.164:0/3250337485 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55b1a6b33890 0x55b1a6b35c80 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.753+0000 7fd3551db740 1 --2- 172.21.15.164:0/3250337485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1a6b18b60 0x55b1a6b20c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.753+0000 7fd3551db740 1 -- 172.21.15.164:0/3250337485 >> 172.21.15.164:0/3250337485 conn(0x55b1a6b150f0 msgr2=0x55b1a6b14270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:24.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.753+0000 7fd32bfff640 0 client.16851 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:52:24.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.753+0000 7fd3551db740 1 -- 172.21.15.164:0/3250337485 shutdown_connections 2024-09-19T08:52:24.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.753+0000 7fd3551db740 1 -- 172.21.15.164:0/3250337485 wait complete. 2024-09-19T08:52:24.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.754+0000 7fd3551db740 20 client.16851 trim_cache size 0 max 16384 2024-09-19T08:52:24.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.770+0000 7fd3551db740 1 Processor -- start 2024-09-19T08:52:24.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.770+0000 7fd3551db740 1 -- start start 2024-09-19T08:52:24.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.771+0000 7fd3551db740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b1a6b18b60 0x55b1a6b21520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:24.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.771+0000 7fd3551db740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1a6b19480 0x55b1a6b388c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:24.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.771+0000 7fd3551db740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a6b23d40 0x55b1a6b38e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:24.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.771+0000 7fd3551db740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1afa80 con 0x55b1a6b23d40 2024-09-19T08:52:24.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.771+0000 7fd3551db740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1b15f0 con 0x55b1a6b19480 2024-09-19T08:52:24.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.771+0000 7fd3551db740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1b0150 con 0x55b1a6b18b60 2024-09-19T08:52:24.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.771+0000 7fd35139f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a6b23d40 0x55b1a6b38e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:24.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.771+0000 7fd35139f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a6b23d40 0x55b1a6b38e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39890/0 (socket says 172.21.15.164:39890) 2024-09-19T08:52:24.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.771+0000 7fd35139f640 1 -- 172.21.15.164:0/3379275322 learned_addr learned my addr 172.21.15.164:0/3379275322 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:24.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.771+0000 7fd34bfff640 1 --2- 172.21.15.164:0/3379275322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1a6b19480 0x55b1a6b388c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:24.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.771+0000 7fd350b9e640 1 --2- 172.21.15.164:0/3379275322 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b1a6b18b60 0x55b1a6b21520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:24.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.771+0000 7fd35139f640 1 -- 172.21.15.164:0/3379275322 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b1a6b18b60 msgr2=0x55b1a6b21520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:24.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.771+0000 7fd35139f640 1 --2- 172.21.15.164:0/3379275322 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b1a6b18b60 0x55b1a6b21520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.771+0000 7fd35139f640 1 -- 172.21.15.164:0/3379275322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1a6b19480 msgr2=0x55b1a6b388c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:24.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.771+0000 7fd35139f640 1 --2- 172.21.15.164:0/3379275322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1a6b19480 0x55b1a6b388c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.771+0000 7fd35139f640 1 -- 172.21.15.164:0/3379275322 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1a6b19270 con 0x55b1a6b23d40 2024-09-19T08:52:24.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.771+0000 7fd350b9e640 1 --2- 172.21.15.164:0/3379275322 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b1a6b18b60 0x55b1a6b21520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:52:24.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.771+0000 7fd35139f640 1 --2- 172.21.15.164:0/3379275322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a6b23d40 0x55b1a6b38e00 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd34400bea0 tx=0x7fd344004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4efee72146209bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:24.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.771+0000 7fd3367fc640 1 -- 172.21.15.164:0/3379275322 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3440b4040 con 0x55b1a6b23d40 2024-09-19T08:52:24.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.771+0000 7fd3367fc640 1 -- 172.21.15.164:0/3379275322 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3440998d0 con 0x55b1a6b23d40 2024-09-19T08:52:24.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.771+0000 7fd3367fc640 1 -- 172.21.15.164:0/3379275322 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34409a530 con 0x55b1a6b23d40 2024-09-19T08:52:24.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.772+0000 7fd3551db740 1 -- 172.21.15.164:0/3379275322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a6b23d40 msgr2=0x55b1a6b38e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:24.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.772+0000 7fd3551db740 1 --2- 172.21.15.164:0/3379275322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a6b23d40 0x55b1a6b38e00 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd34400bea0 tx=0x7fd344004b50 comp rx=0 tx=0).stop 2024-09-19T08:52:24.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.772+0000 7fd3551db740 1 -- 172.21.15.164:0/3379275322 shutdown_connections 2024-09-19T08:52:24.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.772+0000 7fd3551db740 1 --2- 172.21.15.164:0/3379275322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a6b23d40 0x55b1a6b38e00 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.772+0000 7fd3551db740 1 --2- 172.21.15.164:0/3379275322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1a6b19480 0x55b1a6b388c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.772+0000 7fd3551db740 1 --2- 172.21.15.164:0/3379275322 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b1a6b18b60 0x55b1a6b21520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.772+0000 7fd3551db740 1 -- 172.21.15.164:0/3379275322 >> 172.21.15.164:0/3379275322 conn(0x55b1a6b150f0 msgr2=0x55b1a6b22bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:24.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.772+0000 7fd3551db740 1 -- 172.21.15.164:0/3379275322 shutdown_connections 2024-09-19T08:52:24.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.772+0000 7fd3551db740 1 -- 172.21.15.164:0/3379275322 wait complete. 2024-09-19T08:52:24.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.773+0000 7fd3551db740 1 -- start start 2024-09-19T08:52:24.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.773+0000 7fd3551db740 1 Processor -- start 2024-09-19T08:52:24.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.773+0000 7fd3551db740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a6b18b60 0x55b1a6b36630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:24.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.773+0000 7fd3551db740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1a6b19480 0x55b1a6b36b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:24.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.773+0000 7fd3551db740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b1a6b23d40 0x55b1a6b370b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:24.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.773+0000 7fd3551db740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1b0ef0 con 0x55b1a6b18b60 2024-09-19T08:52:24.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.773+0000 7fd3551db740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1b0820 con 0x55b1a6b19480 2024-09-19T08:52:24.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.773+0000 7fd3551db740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1ada80 con 0x55b1a6b23d40 2024-09-19T08:52:24.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.773+0000 7fd3551db740 10 client.0 _ll_register_callbacks cb 0x7fd3440b2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:52:24.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.773+0000 7fd350b9e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a6b18b60 0x55b1a6b36630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:24.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.773+0000 7fd350b9e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a6b18b60 0x55b1a6b36630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39900/0 (socket says 172.21.15.164:39900) 2024-09-19T08:52:24.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.773+0000 7fd350b9e640 1 -- 172.21.15.164:0/282297097 learned_addr learned my addr 172.21.15.164:0/282297097 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:24.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.774+0000 7fd34bfff640 1 --2- 172.21.15.164:0/282297097 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1a6b19480 0x55b1a6b36b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:24.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.774+0000 7fd350b9e640 1 -- 172.21.15.164:0/282297097 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b1a6b23d40 msgr2=0x55b1a6b370b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:24.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.774+0000 7fd350b9e640 1 --2- 172.21.15.164:0/282297097 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b1a6b23d40 0x55b1a6b370b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.774+0000 7fd350b9e640 1 -- 172.21.15.164:0/282297097 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1a6b19480 msgr2=0x55b1a6b36b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:24.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.774+0000 7fd350b9e640 1 --2- 172.21.15.164:0/282297097 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1a6b19480 0x55b1a6b36b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.774+0000 7fd350b9e640 1 -- 172.21.15.164:0/282297097 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1a6b9d1d0 con 0x55b1a6b18b60 2024-09-19T08:52:24.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.774+0000 7fd350b9e640 1 --2- 172.21.15.164:0/282297097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a6b18b60 0x55b1a6b36630 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fd34002a940 tx=0x7fd3400095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3c7512e363f6ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:24.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.774+0000 7fd32bfff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:52:24.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.774+0000 7fd32bfff640 1 -- 172.21.15.164:0/282297097 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd340035040 con 0x55b1a6b18b60 2024-09-19T08:52:24.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.774+0000 7fd32bfff640 1 -- 172.21.15.164:0/282297097 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7fd340004450 con 0x55b1a6b18b60 2024-09-19T08:52:24.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.774+0000 7fd3551db740 1 -- 172.21.15.164:0/282297097 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3440998d0 con 0x55b1a6b18b60 2024-09-19T08:52:24.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.774+0000 7fd32bfff640 1 -- 172.21.15.164:0/282297097 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd340058050 con 0x55b1a6b18b60 2024-09-19T08:52:24.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.774+0000 7fd3551db740 10 client.16857 fetch_fsmap learned FSMap version 31 2024-09-19T08:52:24.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.774+0000 7fd3551db740 1 -- 172.21.15.164:0/282297097 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b1a6b9e4a0 con 0x55b1a6b18b60 2024-09-19T08:52:24.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.774+0000 7fd32bfff640 1 -- 172.21.15.164:0/282297097 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3440998d0 con 0x55b1a6b18b60 2024-09-19T08:52:24.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.775+0000 7fd32bfff640 1 -- 172.21.15.164:0/282297097 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7fd340003920 con 0x55b1a6b18b60 2024-09-19T08:52:24.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.775+0000 7fd3551db740 10 client.16857 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T08:52:24.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.775+0000 7fd3551db740 10 client.16857 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:52:24.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.775+0000 7fd3551db740 20 client.16857 populate_metadata read hostname 'smithi164' 2024-09-19T08:52:24.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.775+0000 7fd3551db740 1 --2- 172.21.15.164:0/282297097 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55b1a6b9ebd0 0x55b1a6ba1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:24.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.775+0000 7fd3551db740 4 client.16857 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T08:52:24.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.775+0000 7fd3551db740 1 -- 172.21.15.164:0/282297097 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b1a6a85d00 con 0x55b1a6b9ebd0 2024-09-19T08:52:24.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.775+0000 7fd34bfff640 1 --2- 172.21.15.164:0/282297097 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55b1a6b9ebd0 0x55b1a6ba1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:24.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.775+0000 7fd32bfff640 10 client.16857 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:52:24.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.775+0000 7fd34bfff640 1 --2- 172.21.15.164:0/282297097 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55b1a6b9ebd0 0x55b1a6ba1010 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88eeeb2f576b6096 server_cookie=a5c022513b1d9f9 in_seq=0 out_seq=0 2024-09-19T08:52:24.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.776+0000 7fd32bfff640 1 -- 172.21.15.164:0/282297097 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b1a6a85d00 con 0x55b1a6b9ebd0 2024-09-19T08:52:24.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.776+0000 7fd32bfff640 10 client.16857 handle_command_reply: tid=0 2024-09-19T08:52:24.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.776+0000 7fd3551db740 1 client.16857 shutdown 2024-09-19T08:52:24.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.777+0000 7fd3551db740 1 -- 172.21.15.164:0/282297097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a6b18b60 msgr2=0x55b1a6b36630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:24.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.777+0000 7fd3551db740 1 --2- 172.21.15.164:0/282297097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a6b18b60 0x55b1a6b36630 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fd34002a940 tx=0x7fd3400095b0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.777+0000 7fd3551db740 1 -- 172.21.15.164:0/282297097 shutdown_connections 2024-09-19T08:52:24.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.777+0000 7fd3551db740 1 --2- 172.21.15.164:0/282297097 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b1a6b23d40 0x55b1a6b370b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.777+0000 7fd3551db740 1 --2- 172.21.15.164:0/282297097 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55b1a6b9ebd0 0x55b1a6ba1010 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.777+0000 7fd32bfff640 0 client.16857 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:52:24.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.777+0000 7fd3551db740 1 --2- 172.21.15.164:0/282297097 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1a6b19480 0x55b1a6b36b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.777+0000 7fd3551db740 1 --2- 172.21.15.164:0/282297097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1a6b18b60 0x55b1a6b36630 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.777+0000 7fd3551db740 1 -- 172.21.15.164:0/282297097 >> 172.21.15.164:0/282297097 conn(0x55b1a6b150f0 msgr2=0x55b1a6b1a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:24.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.777+0000 7fd3551db740 1 -- 172.21.15.164:0/282297097 shutdown_connections 2024-09-19T08:52:24.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.777+0000 7fd3551db740 1 -- 172.21.15.164:0/282297097 wait complete. 2024-09-19T08:52:24.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.777+0000 7fd3551db740 20 client.16857 trim_cache size 0 max 16384 2024-09-19T08:52:24.781 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:52:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: "return_code": 0, 2024-09-19T08:52:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_tag": "8a8ed284-00da-4814-a611-7a9ab9e8642d", 2024-09-19T08:52:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: "mode": "asynchronous" 2024-09-19T08:52:24.782 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:52:24.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.778+0000 7fd352e02640 1 -- 172.21.15.164:0/2483786765 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd33c0a3430 msgr2=0x7fd33c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:24.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.778+0000 7fd352e02640 1 --2- 172.21.15.164:0/2483786765 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd33c0a3430 0x7fd33c0a58f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd34c1040e0 tx=0x7fd3380156d0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.778+0000 7fd352e02640 1 -- 172.21.15.164:0/2483786765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd34c106a80 msgr2=0x7fd34c1030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:24.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.778+0000 7fd352e02640 1 --2- 172.21.15.164:0/2483786765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd34c106a80 0x7fd34c1030d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fd34400b970 tx=0x7fd34400bd70 comp rx=0 tx=0).stop 2024-09-19T08:52:24.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.778+0000 7fd352e02640 1 -- 172.21.15.164:0/2483786765 shutdown_connections 2024-09-19T08:52:24.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.778+0000 7fd352e02640 1 --2- 172.21.15.164:0/2483786765 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd33c0a3430 0x7fd33c0a58f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.778+0000 7fd352e02640 1 --2- 172.21.15.164:0/2483786765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd34c106a80 0x7fd34c1030d0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.778+0000 7fd352e02640 1 --2- 172.21.15.164:0/2483786765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd34c101420 0x7fd34c102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.778+0000 7fd352e02640 1 --2- 172.21.15.164:0/2483786765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd34c100a80 0x7fd34c102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:24.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.778+0000 7fd352e02640 1 -- 172.21.15.164:0/2483786765 >> 172.21.15.164:0/2483786765 conn(0x7fd34c0fc7a0 msgr2=0x7fd34c1048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:24.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.778+0000 7fd352e02640 1 -- 172.21.15.164:0/2483786765 shutdown_connections 2024-09-19T08:52:24.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:24.778+0000 7fd352e02640 1 -- 172.21.15.164:0/2483786765 wait complete. 2024-09-19T08:52:25.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:25 smithi164 ceph-mon[30928]: scrub summary: active paths [/] 2024-09-19T08:52:25.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:25 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:52:25.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:25 smithi164 ceph-mon[30928]: scrub summary: active paths [/] 2024-09-19T08:52:25.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:25 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:52:25.594 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:52:25.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:25 smithi170 ceph-mon[35072]: scrub summary: active paths [/] 2024-09-19T08:52:25.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:25 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:52:25.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:25 smithi170 ceph-mon[35072]: scrub summary: active paths [/] 2024-09-19T08:52:25.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:25 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:52:25.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:25 smithi173 ceph-mon[35306]: scrub summary: active paths [/] 2024-09-19T08:52:25.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:25 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:52:25.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:25 smithi173 ceph-mon[35306]: scrub summary: active paths [/] 2024-09-19T08:52:25.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:25 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:52:25.845 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:52:26.226 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:26 smithi164 ceph-mon[30928]: pgmap v203: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 196 MiB/s wr, 789 op/s 2024-09-19T08:52:26.226 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:26 smithi164 ceph-mon[30928]: scrub queued for path: / 2024-09-19T08:52:26.226 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:26 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:52:26.226 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:26 smithi164 ceph-mon[30928]: scrub summary: active paths [/] 2024-09-19T08:52:26.226 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:26 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:52:26.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.309+0000 7eff388ed640 1 -- 172.21.15.164:0/172574740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff34102570 msgr2=0x7eff341029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:26.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.309+0000 7eff388ed640 1 --2- 172.21.15.164:0/172574740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff34102570 0x7eff341029b0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7eff1c009900 tx=0x7eff1c02a480 comp rx=0 tx=0).stop 2024-09-19T08:52:26.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.309+0000 7eff388ed640 1 -- 172.21.15.164:0/172574740 shutdown_connections 2024-09-19T08:52:26.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.309+0000 7eff388ed640 1 --2- 172.21.15.164:0/172574740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff34102ef0 0x7eff3410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:26.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.309+0000 7eff388ed640 1 --2- 172.21.15.164:0/172574740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff34102570 0x7eff341029b0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:26.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.309+0000 7eff388ed640 1 --2- 172.21.15.164:0/172574740 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff34100e20 0x7eff34101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:26.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.309+0000 7eff388ed640 1 -- 172.21.15.164:0/172574740 >> 172.21.15.164:0/172574740 conn(0x7eff340fc7d0 msgr2=0x7eff340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:26.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.309+0000 7eff388ed640 1 -- 172.21.15.164:0/172574740 shutdown_connections 2024-09-19T08:52:26.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.309+0000 7eff388ed640 1 -- 172.21.15.164:0/172574740 wait complete. 2024-09-19T08:52:26.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.310+0000 7eff388ed640 1 Processor -- start 2024-09-19T08:52:26.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.310+0000 7eff388ed640 1 -- start start 2024-09-19T08:52:26.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.310+0000 7eff388ed640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff34100e20 0x7eff3419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:26.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.310+0000 7eff388ed640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff34102570 0x7eff3419ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:26.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.310+0000 7eff388ed640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff34102ef0 0x7eff3419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:26.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.310+0000 7eff388ed640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7eff34112060 con 0x7eff34100e20 2024-09-19T08:52:26.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.310+0000 7eff388ed640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7eff34111ee0 con 0x7eff34102570 2024-09-19T08:52:26.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.310+0000 7eff388ed640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7eff341121e0 con 0x7eff34102ef0 2024-09-19T08:52:26.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.310+0000 7eff3259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff34100e20 0x7eff3419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:26.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.310+0000 7eff3259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff34100e20 0x7eff3419a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39912/0 (socket says 172.21.15.164:39912) 2024-09-19T08:52:26.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.310+0000 7eff3259c640 1 -- 172.21.15.164:0/3144712497 learned_addr learned my addr 172.21.15.164:0/3144712497 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:26.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.310+0000 7eff2bfff640 1 --2- 172.21.15.164:0/3144712497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff34102570 0x7eff3419ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:26.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.310+0000 7eff32d9d640 1 --2- 172.21.15.164:0/3144712497 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff34102ef0 0x7eff3419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:26.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.310+0000 7eff3259c640 1 -- 172.21.15.164:0/3144712497 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff34102ef0 msgr2=0x7eff3419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:26.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.310+0000 7eff3259c640 1 --2- 172.21.15.164:0/3144712497 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff34102ef0 0x7eff3419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:26.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.311+0000 7eff3259c640 1 -- 172.21.15.164:0/3144712497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff34102570 msgr2=0x7eff3419ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:26.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.311+0000 7eff3259c640 1 --2- 172.21.15.164:0/3144712497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff34102570 0x7eff3419ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:26.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.311+0000 7eff3259c640 1 -- 172.21.15.164:0/3144712497 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff3419f6a0 con 0x7eff34100e20 2024-09-19T08:52:26.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.311+0000 7eff3259c640 1 --2- 172.21.15.164:0/3144712497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff34100e20 0x7eff3419a6f0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7eff2400d950 tx=0x7eff2400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:26.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.311+0000 7eff2b7fe640 1 -- 172.21.15.164:0/3144712497 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24017030 con 0x7eff34100e20 2024-09-19T08:52:26.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.311+0000 7eff2b7fe640 1 -- 172.21.15.164:0/3144712497 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff24004020 con 0x7eff34100e20 2024-09-19T08:52:26.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.311+0000 7eff388ed640 1 -- 172.21.15.164:0/3144712497 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff3419f990 con 0x7eff34100e20 2024-09-19T08:52:26.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.311+0000 7eff2b7fe640 1 -- 172.21.15.164:0/3144712497 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff240047a0 con 0x7eff34100e20 2024-09-19T08:52:26.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.311+0000 7eff388ed640 1 -- 172.21.15.164:0/3144712497 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff341a7270 con 0x7eff34100e20 2024-09-19T08:52:26.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.314+0000 7eff2b7fe640 1 -- 172.21.15.164:0/3144712497 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7eff240041d0 con 0x7eff34100e20 2024-09-19T08:52:26.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.314+0000 7eff3acc6740 1 Processor -- start 2024-09-19T08:52:26.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.314+0000 7eff3acc6740 1 -- start start 2024-09-19T08:52:26.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.315+0000 7eff2b7fe640 1 --2- 172.21.15.164:0/3144712497 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7eff200a33b0 0x7eff200a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:26.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.315+0000 7eff3acc6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611661f54a0 0x5611661f5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:26.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.315+0000 7eff3acc6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611661f5dc0 0x561166200140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:26.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.315+0000 7eff3acc6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561166200680 0x561166202a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:26.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.315+0000 7eff3acc6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56116615b2e0 con 0x5611661f54a0 2024-09-19T08:52:26.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.315+0000 7eff3acc6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561166102ce0 con 0x561166200680 2024-09-19T08:52:26.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.315+0000 7eff3acc6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5611661572a0 con 0x5611661f5dc0 2024-09-19T08:52:26.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.315+0000 7eff32d9d640 1 --2- 172.21.15.164:0/3144712497 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7eff200a33b0 0x7eff200a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:26.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.315+0000 7eff3259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611661f54a0 0x5611661f5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:26.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.315+0000 7eff2b7fe640 1 -- 172.21.15.164:0/3144712497 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7eff240cf6a0 con 0x7eff34100e20 2024-09-19T08:52:26.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.315+0000 7eff3259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611661f54a0 0x5611661f5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39918/0 (socket says 172.21.15.164:39918) 2024-09-19T08:52:26.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.315+0000 7eff3259c640 1 -- 172.21.15.164:0/1011170676 learned_addr learned my addr 172.21.15.164:0/1011170676 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:26.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.315+0000 7eff2bfff640 1 --2- 172.21.15.164:0/1011170676 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611661f5dc0 0x561166200140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:26.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.315+0000 7eff3259c640 1 -- 172.21.15.164:0/1011170676 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611661f5dc0 msgr2=0x561166200140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:26.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.315+0000 7eff3259c640 1 --2- 172.21.15.164:0/1011170676 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611661f5dc0 0x561166200140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:26.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.315+0000 7eff3259c640 1 -- 172.21.15.164:0/1011170676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561166200680 msgr2=0x561166202a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:26.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.315+0000 7eff3259c640 1 --2- 172.21.15.164:0/1011170676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561166200680 0x561166202a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:26.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.315+0000 7eff3259c640 1 -- 172.21.15.164:0/1011170676 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611660e7440 con 0x5611661f54a0 2024-09-19T08:52:26.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.315+0000 7eff32d9d640 1 --2- 172.21.15.164:0/3144712497 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7eff200a33b0 0x7eff200a5870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7eff2c005fd0 tx=0x7eff2c010740 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:26.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.315+0000 7eff3259c640 1 --2- 172.21.15.164:0/1011170676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611661f54a0 0x5611661f5880 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7eff24098390 tx=0x7eff2409b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=536d0b856a6c0143 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:26.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.315+0000 7eff28ff9640 1 -- 172.21.15.164:0/1011170676 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2409c0c0 con 0x5611661f54a0 2024-09-19T08:52:26.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.316+0000 7eff28ff9640 1 -- 172.21.15.164:0/1011170676 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff2406c7a0 con 0x5611661f54a0 2024-09-19T08:52:26.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.316+0000 7eff28ff9640 1 -- 172.21.15.164:0/1011170676 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24099af0 con 0x5611661f54a0 2024-09-19T08:52:26.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.316+0000 7eff3acc6740 1 -- 172.21.15.164:0/1011170676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611661f54a0 msgr2=0x5611661f5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:26.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.316+0000 7eff3acc6740 1 --2- 172.21.15.164:0/1011170676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611661f54a0 0x5611661f5880 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7eff24098390 tx=0x7eff2409b890 comp rx=0 tx=0).stop 2024-09-19T08:52:26.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.316+0000 7eff3acc6740 1 -- 172.21.15.164:0/1011170676 shutdown_connections 2024-09-19T08:52:26.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.316+0000 7eff3acc6740 1 --2- 172.21.15.164:0/1011170676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561166200680 0x561166202a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:26.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.316+0000 7eff3acc6740 1 --2- 172.21.15.164:0/1011170676 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611661f5dc0 0x561166200140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:26.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.316+0000 7eff3acc6740 1 --2- 172.21.15.164:0/1011170676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611661f54a0 0x5611661f5880 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:26.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.316+0000 7eff3acc6740 1 -- 172.21.15.164:0/1011170676 >> 172.21.15.164:0/1011170676 conn(0x5611661f1a30 msgr2=0x5611661f1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:26.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.316+0000 7eff3acc6740 1 -- 172.21.15.164:0/1011170676 shutdown_connections 2024-09-19T08:52:26.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.316+0000 7eff3acc6740 1 -- 172.21.15.164:0/1011170676 wait complete. 2024-09-19T08:52:26.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.317+0000 7eff3acc6740 1 -- start start 2024-09-19T08:52:26.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.317+0000 7eff3acc6740 1 Processor -- start 2024-09-19T08:52:26.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.317+0000 7eff3acc6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611661f54a0 0x561166200000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:26.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.317+0000 7eff3acc6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611661f5dc0 0x5611661fd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:26.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.317+0000 7eff3acc6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561166200680 0x5611661fd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:26.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.317+0000 7eff3acc6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561166203250 con 0x5611661f5dc0 2024-09-19T08:52:26.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.317+0000 7eff3acc6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561166116050 con 0x561166200680 2024-09-19T08:52:26.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.317+0000 7eff3acc6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5611662033d0 con 0x5611661f54a0 2024-09-19T08:52:26.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.317+0000 7eff3acc6740 10 client.0 _ll_register_callbacks cb 0x561166156160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:52:26.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.317+0000 7eff2bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611661f5dc0 0x5611661fd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:26.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.317+0000 7eff2bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611661f5dc0 0x5611661fd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39928/0 (socket says 172.21.15.164:39928) 2024-09-19T08:52:26.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.317+0000 7eff2bfff640 1 -- 172.21.15.164:0/2356852145 learned_addr learned my addr 172.21.15.164:0/2356852145 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:26.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.317+0000 7eff3259c640 1 --2- 172.21.15.164:0/2356852145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611661f54a0 0x561166200000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:26.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.317+0000 7eff32d9d640 1 --2- 172.21.15.164:0/2356852145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561166200680 0x5611661fd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:26.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.317+0000 7eff2bfff640 1 -- 172.21.15.164:0/2356852145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611661f54a0 msgr2=0x561166200000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:26.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.318+0000 7eff2bfff640 1 --2- 172.21.15.164:0/2356852145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611661f54a0 0x561166200000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:26.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.318+0000 7eff2bfff640 1 -- 172.21.15.164:0/2356852145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561166200680 msgr2=0x5611661fd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:26.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.318+0000 7eff2bfff640 1 --2- 172.21.15.164:0/2356852145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561166200680 0x5611661fd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:26.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.318+0000 7eff2bfff640 1 -- 172.21.15.164:0/2356852145 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561165ecc3c0 con 0x5611661f5dc0 2024-09-19T08:52:26.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.318+0000 7eff32d9d640 1 --2- 172.21.15.164:0/2356852145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561166200680 0x5611661fd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:52:26.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.318+0000 7eff2bfff640 1 --2- 172.21.15.164:0/2356852145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611661f5dc0 0x5611661fd2b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7eff1c0314e0 tx=0x7eff1c0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a147ad744c7d3227 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:26.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.318+0000 7eff19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:52:26.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.318+0000 7eff19ffb640 1 -- 172.21.15.164:0/2356852145 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c035030 con 0x5611661f5dc0 2024-09-19T08:52:26.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.318+0000 7eff19ffb640 1 -- 172.21.15.164:0/2356852145 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7eff1c003a00 con 0x5611661f5dc0 2024-09-19T08:52:26.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.318+0000 7eff3acc6740 1 -- 172.21.15.164:0/2356852145 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff2406c7a0 con 0x5611661f5dc0 2024-09-19T08:52:26.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.318+0000 7eff19ffb640 1 -- 172.21.15.164:0/2356852145 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7eff1c003c00 con 0x5611661f5dc0 2024-09-19T08:52:26.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.318+0000 7eff3acc6740 10 client.16869 fetch_fsmap learned FSMap version 31 2024-09-19T08:52:26.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.318+0000 7eff3acc6740 1 -- 172.21.15.164:0/2356852145 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56116620ec40 con 0x5611661f5dc0 2024-09-19T08:52:26.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.318+0000 7eff19ffb640 1 -- 172.21.15.164:0/2356852145 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff2406c7a0 con 0x5611661f5dc0 2024-09-19T08:52:26.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.319+0000 7eff19ffb640 1 -- 172.21.15.164:0/2356852145 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7eff1c03c4d0 con 0x5611661f5dc0 2024-09-19T08:52:26.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.319+0000 7eff3acc6740 10 client.16869 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T08:52:26.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.319+0000 7eff3acc6740 10 client.16869 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:52:26.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.319+0000 7eff3acc6740 20 client.16869 populate_metadata read hostname 'smithi164' 2024-09-19T08:52:26.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.319+0000 7eff3acc6740 1 --2- 172.21.15.164:0/2356852145 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x561166210150 0x561166212570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:26.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.319+0000 7eff3acc6740 4 client.16869 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:52:26.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.319+0000 7eff3acc6740 1 -- 172.21.15.164:0/2356852145 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561166168850 con 0x561166210150 2024-09-19T08:52:26.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.319+0000 7eff3259c640 1 --2- 172.21.15.164:0/2356852145 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x561166210150 0x561166212570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:26.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.319+0000 7eff3259c640 1 --2- 172.21.15.164:0/2356852145 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x561166210150 0x561166212570 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8813cf0b78af12f3 server_cookie=2cf44cc081e5628e in_seq=0 out_seq=0 2024-09-19T08:52:26.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.319+0000 7eff19ffb640 10 client.16869 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:52:26.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.320+0000 7eff19ffb640 1 -- 172.21.15.164:0/2356852145 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561166168850 con 0x561166210150 2024-09-19T08:52:26.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.320+0000 7eff19ffb640 10 client.16869 handle_command_reply: tid=0 2024-09-19T08:52:26.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.320+0000 7eff3acc6740 1 client.16869 shutdown 2024-09-19T08:52:26.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.320+0000 7eff3acc6740 1 -- 172.21.15.164:0/2356852145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611661f5dc0 msgr2=0x5611661fd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:26.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.320+0000 7eff3acc6740 1 --2- 172.21.15.164:0/2356852145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611661f5dc0 0x5611661fd2b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7eff1c0314e0 tx=0x7eff1c0042e0 comp rx=0 tx=0).stop 2024-09-19T08:52:26.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.320+0000 7eff3acc6740 1 -- 172.21.15.164:0/2356852145 shutdown_connections 2024-09-19T08:52:26.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.320+0000 7eff3acc6740 1 --2- 172.21.15.164:0/2356852145 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x561166210150 0x561166212570 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:26.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.320+0000 7eff19ffb640 0 client.16869 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:52:26.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.320+0000 7eff3acc6740 1 --2- 172.21.15.164:0/2356852145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561166200680 0x5611661fd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:26.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.320+0000 7eff3acc6740 1 --2- 172.21.15.164:0/2356852145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611661f5dc0 0x5611661fd2b0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:26.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.320+0000 7eff3acc6740 1 --2- 172.21.15.164:0/2356852145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611661f54a0 0x561166200000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:26.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.320+0000 7eff3acc6740 1 -- 172.21.15.164:0/2356852145 >> 172.21.15.164:0/2356852145 conn(0x5611661f1a30 msgr2=0x5611661f7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:26.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.321+0000 7eff3acc6740 1 -- 172.21.15.164:0/2356852145 shutdown_connections 2024-09-19T08:52:26.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.321+0000 7eff3acc6740 1 -- 172.21.15.164:0/2356852145 wait complete. 2024-09-19T08:52:26.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.321+0000 7eff3acc6740 20 client.16869 trim_cache size 0 max 16384 2024-09-19T08:52:26.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.338+0000 7eff3acc6740 1 Processor -- start 2024-09-19T08:52:26.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.338+0000 7eff3acc6740 1 -- start start 2024-09-19T08:52:26.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.338+0000 7eff3acc6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5611661f54a0 0x56116620c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:26.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.338+0000 7eff3acc6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611661f5dc0 0x5611661ff220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:26.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.338+0000 7eff3acc6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561166200680 0x561166217dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:26.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.338+0000 7eff3acc6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7eff341afa40 con 0x5611661f5dc0 2024-09-19T08:52:26.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.338+0000 7eff3acc6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7eff341b15b0 con 0x5611661f54a0 2024-09-19T08:52:26.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.338+0000 7eff3acc6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7eff341b0110 con 0x561166200680 2024-09-19T08:52:26.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.338+0000 7eff2bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611661f5dc0 0x5611661ff220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:26.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.338+0000 7eff2bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611661f5dc0 0x5611661ff220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39942/0 (socket says 172.21.15.164:39942) 2024-09-19T08:52:26.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.338+0000 7eff2bfff640 1 -- 172.21.15.164:0/4193870011 learned_addr learned my addr 172.21.15.164:0/4193870011 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:26.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.338+0000 7eff32d9d640 1 --2- 172.21.15.164:0/4193870011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561166200680 0x561166217dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:26.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.338+0000 7eff3259c640 1 --2- 172.21.15.164:0/4193870011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5611661f54a0 0x56116620c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:26.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.338+0000 7eff2bfff640 1 -- 172.21.15.164:0/4193870011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561166200680 msgr2=0x561166217dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:26.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.338+0000 7eff2bfff640 1 --2- 172.21.15.164:0/4193870011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561166200680 0x561166217dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:26.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.338+0000 7eff2bfff640 1 -- 172.21.15.164:0/4193870011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5611661f54a0 msgr2=0x56116620c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:26.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.338+0000 7eff2bfff640 1 --2- 172.21.15.164:0/4193870011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5611661f54a0 0x56116620c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:26.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.338+0000 7eff2bfff640 1 -- 172.21.15.164:0/4193870011 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611661f2ae0 con 0x5611661f5dc0 2024-09-19T08:52:26.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.338+0000 7eff32d9d640 1 --2- 172.21.15.164:0/4193870011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561166200680 0x561166217dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:52:26.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.338+0000 7eff3259c640 1 --2- 172.21.15.164:0/4193870011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5611661f54a0 0x56116620c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:52:26.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.338+0000 7eff2bfff640 1 --2- 172.21.15.164:0/4193870011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611661f5dc0 0x5611661ff220 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7eff1c03ca10 tx=0x7eff1c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd806b5abe27978 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:26.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.339+0000 7eff1bfff640 1 -- 172.21.15.164:0/4193870011 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c051070 con 0x5611661f5dc0 2024-09-19T08:52:26.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.339+0000 7eff1bfff640 1 -- 172.21.15.164:0/4193870011 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff1c040070 con 0x5611661f5dc0 2024-09-19T08:52:26.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.339+0000 7eff1bfff640 1 -- 172.21.15.164:0/4193870011 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c035070 con 0x5611661f5dc0 2024-09-19T08:52:26.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.339+0000 7eff3acc6740 1 -- 172.21.15.164:0/4193870011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611661f5dc0 msgr2=0x5611661ff220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:26.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.339+0000 7eff3acc6740 1 --2- 172.21.15.164:0/4193870011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611661f5dc0 0x5611661ff220 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7eff1c03ca10 tx=0x7eff1c005780 comp rx=0 tx=0).stop 2024-09-19T08:52:26.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.339+0000 7eff3acc6740 1 -- 172.21.15.164:0/4193870011 shutdown_connections 2024-09-19T08:52:26.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.339+0000 7eff3acc6740 1 --2- 172.21.15.164:0/4193870011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561166200680 0x561166217dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:26.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.339+0000 7eff3acc6740 1 --2- 172.21.15.164:0/4193870011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5611661f5dc0 0x5611661ff220 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:26.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.339+0000 7eff3acc6740 1 --2- 172.21.15.164:0/4193870011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5611661f54a0 0x56116620c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:26.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.339+0000 7eff3acc6740 1 -- 172.21.15.164:0/4193870011 >> 172.21.15.164:0/4193870011 conn(0x5611661f1a30 msgr2=0x5611661feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:26.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.339+0000 7eff3acc6740 1 -- 172.21.15.164:0/4193870011 shutdown_connections 2024-09-19T08:52:26.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.339+0000 7eff3acc6740 1 -- 172.21.15.164:0/4193870011 wait complete. 2024-09-19T08:52:26.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.340+0000 7eff3acc6740 1 -- start start 2024-09-19T08:52:26.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.340+0000 7eff3acc6740 1 Processor -- start 2024-09-19T08:52:26.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.340+0000 7eff3acc6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5611661f54a0 0x5611662046d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:26.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.340+0000 7eff3acc6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611661f5dc0 0x561166204c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:26.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.340+0000 7eff3acc6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561166200680 0x561166279da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:26.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.340+0000 7eff3acc6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7eff341b0eb0 con 0x561166200680 2024-09-19T08:52:26.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.340+0000 7eff3acc6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7eff341b07e0 con 0x5611661f54a0 2024-09-19T08:52:26.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.340+0000 7eff3acc6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7eff341ada40 con 0x5611661f5dc0 2024-09-19T08:52:26.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.340+0000 7eff3acc6740 10 client.0 _ll_register_callbacks cb 0x7eff1c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:52:26.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.340+0000 7eff32d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561166200680 0x561166279da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:26.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.340+0000 7eff32d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561166200680 0x561166279da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39952/0 (socket says 172.21.15.164:39952) 2024-09-19T08:52:26.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.340+0000 7eff32d9d640 1 -- 172.21.15.164:0/2850286413 learned_addr learned my addr 172.21.15.164:0/2850286413 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:26.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.340+0000 7eff3259c640 1 --2- 172.21.15.164:0/2850286413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5611661f54a0 0x5611662046d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:26.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.340+0000 7eff2bfff640 1 --2- 172.21.15.164:0/2850286413 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611661f5dc0 0x561166204c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:26.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.341+0000 7eff32d9d640 1 -- 172.21.15.164:0/2850286413 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611661f5dc0 msgr2=0x561166204c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:26.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.341+0000 7eff32d9d640 1 --2- 172.21.15.164:0/2850286413 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611661f5dc0 0x561166204c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:26.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.341+0000 7eff32d9d640 1 -- 172.21.15.164:0/2850286413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5611661f54a0 msgr2=0x5611662046d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:26.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.341+0000 7eff32d9d640 1 --2- 172.21.15.164:0/2850286413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5611661f54a0 0x5611662046d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:26.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.341+0000 7eff32d9d640 1 -- 172.21.15.164:0/2850286413 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611662054c0 con 0x561166200680 2024-09-19T08:52:26.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.341+0000 7eff2bfff640 1 --2- 172.21.15.164:0/2850286413 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611661f5dc0 0x561166204c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:52:26.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.341+0000 7eff3259c640 1 --2- 172.21.15.164:0/2850286413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5611661f54a0 0x5611662046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:52:26.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.341+0000 7eff32d9d640 1 --2- 172.21.15.164:0/2850286413 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561166200680 0x561166279da0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7eff2c012bc0 tx=0x7eff2c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ae308a8532ab91 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:26.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.341+0000 7eff19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:52:26.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.341+0000 7eff19ffb640 1 -- 172.21.15.164:0/2850286413 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2c003970 con 0x561166200680 2024-09-19T08:52:26.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.341+0000 7eff3acc6740 1 -- 172.21.15.164:0/2850286413 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff1c040070 con 0x561166200680 2024-09-19T08:52:26.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.341+0000 7eff19ffb640 1 -- 172.21.15.164:0/2850286413 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7eff2c003b10 con 0x561166200680 2024-09-19T08:52:26.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.341+0000 7eff19ffb640 1 -- 172.21.15.164:0/2850286413 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7eff2c02f050 con 0x561166200680 2024-09-19T08:52:26.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.341+0000 7eff3acc6740 10 client.16875 fetch_fsmap learned FSMap version 31 2024-09-19T08:52:26.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.341+0000 7eff3acc6740 1 -- 172.21.15.164:0/2850286413 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56116627b3a0 con 0x561166200680 2024-09-19T08:52:26.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.341+0000 7eff19ffb640 1 -- 172.21.15.164:0/2850286413 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff1c040070 con 0x561166200680 2024-09-19T08:52:26.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.342+0000 7eff19ffb640 1 -- 172.21.15.164:0/2850286413 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7eff2c00f040 con 0x561166200680 2024-09-19T08:52:26.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.342+0000 7eff3acc6740 10 client.16875 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T08:52:26.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.342+0000 7eff3acc6740 10 client.16875 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:52:26.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.342+0000 7eff3acc6740 20 client.16875 populate_metadata read hostname 'smithi164' 2024-09-19T08:52:26.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.342+0000 7eff3acc6740 1 --2- 172.21.15.164:0/2850286413 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x561166210150 0x56116627db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:26.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.342+0000 7eff3acc6740 4 client.16875 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:52:26.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.342+0000 7eff3acc6740 1 -- 172.21.15.164:0/2850286413 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561166162d60 con 0x561166210150 2024-09-19T08:52:26.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.342+0000 7eff3259c640 1 --2- 172.21.15.164:0/2850286413 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x561166210150 0x56116627db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:26.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.342+0000 7eff19ffb640 10 client.16875 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:52:26.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.342+0000 7eff3259c640 1 --2- 172.21.15.164:0/2850286413 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x561166210150 0x56116627db80 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9f228814abd17ec server_cookie=cef14d8a4c71ae4c in_seq=0 out_seq=0 2024-09-19T08:52:26.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.343+0000 7eff19ffb640 1 -- 172.21.15.164:0/2850286413 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561166162d60 con 0x561166210150 2024-09-19T08:52:26.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.343+0000 7eff19ffb640 10 client.16875 handle_command_reply: tid=0 2024-09-19T08:52:26.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.343+0000 7eff3acc6740 1 client.16875 shutdown 2024-09-19T08:52:26.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.343+0000 7eff3acc6740 1 -- 172.21.15.164:0/2850286413 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561166200680 msgr2=0x561166279da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:26.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.343+0000 7eff3acc6740 1 --2- 172.21.15.164:0/2850286413 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561166200680 0x561166279da0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7eff2c012bc0 tx=0x7eff2c004ab0 comp rx=0 tx=0).stop 2024-09-19T08:52:26.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.343+0000 7eff3acc6740 1 -- 172.21.15.164:0/2850286413 shutdown_connections 2024-09-19T08:52:26.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.343+0000 7eff3acc6740 1 --2- 172.21.15.164:0/2850286413 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561166200680 0x561166279da0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:26.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.343+0000 7eff3acc6740 1 --2- 172.21.15.164:0/2850286413 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611661f5dc0 0x561166204c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:26.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.343+0000 7eff3acc6740 1 --2- 172.21.15.164:0/2850286413 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x561166210150 0x56116627db80 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:26.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.343+0000 7eff3acc6740 1 --2- 172.21.15.164:0/2850286413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5611661f54a0 0x5611662046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:26.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.343+0000 7eff3acc6740 1 -- 172.21.15.164:0/2850286413 >> 172.21.15.164:0/2850286413 conn(0x5611661f1a30 msgr2=0x5611661f0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:26.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.343+0000 7eff19ffb640 0 client.16875 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:52:26.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.343+0000 7eff3acc6740 1 -- 172.21.15.164:0/2850286413 shutdown_connections 2024-09-19T08:52:26.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.343+0000 7eff3acc6740 1 -- 172.21.15.164:0/2850286413 wait complete. 2024-09-19T08:52:26.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.343+0000 7eff3acc6740 20 client.16875 trim_cache size 0 max 16384 2024-09-19T08:52:26.358 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:52:26.358 INFO:teuthology.orchestra.run.smithi164.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T08:52:26.358 INFO:teuthology.orchestra.run.smithi164.stdout: "scrubs": { 2024-09-19T08:52:26.358 INFO:teuthology.orchestra.run.smithi164.stdout: "8a8ed284-00da-4814-a611-7a9ab9e8642d": { 2024-09-19T08:52:26.358 INFO:teuthology.orchestra.run.smithi164.stdout: "path": "/", 2024-09-19T08:52:26.359 INFO:teuthology.orchestra.run.smithi164.stdout: "tag": "8a8ed284-00da-4814-a611-7a9ab9e8642d", 2024-09-19T08:52:26.359 INFO:teuthology.orchestra.run.smithi164.stdout: "options": "recursive,force" 2024-09-19T08:52:26.359 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:52:26.359 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:52:26.359 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:52:26.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.343+0000 7eff388ed640 1 -- 172.21.15.164:0/3144712497 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7eff200a33b0 msgr2=0x7eff200a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:26.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.343+0000 7eff388ed640 1 --2- 172.21.15.164:0/3144712497 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7eff200a33b0 0x7eff200a5870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7eff2c005fd0 tx=0x7eff2c010740 comp rx=0 tx=0).stop 2024-09-19T08:52:26.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.343+0000 7eff388ed640 1 -- 172.21.15.164:0/3144712497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff34100e20 msgr2=0x7eff3419a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:26.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.343+0000 7eff388ed640 1 --2- 172.21.15.164:0/3144712497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff34100e20 0x7eff3419a6f0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7eff2400d950 tx=0x7eff2400de10 comp rx=0 tx=0).stop 2024-09-19T08:52:26.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.344+0000 7eff388ed640 1 -- 172.21.15.164:0/3144712497 shutdown_connections 2024-09-19T08:52:26.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.344+0000 7eff388ed640 1 --2- 172.21.15.164:0/3144712497 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff34102ef0 0x7eff3419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:26.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.344+0000 7eff388ed640 1 --2- 172.21.15.164:0/3144712497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff34102570 0x7eff3419ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:26.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.344+0000 7eff388ed640 1 --2- 172.21.15.164:0/3144712497 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7eff200a33b0 0x7eff200a5870 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:26.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.344+0000 7eff388ed640 1 --2- 172.21.15.164:0/3144712497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff34100e20 0x7eff3419a6f0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:26.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.344+0000 7eff388ed640 1 -- 172.21.15.164:0/3144712497 >> 172.21.15.164:0/3144712497 conn(0x7eff340fc7d0 msgr2=0x7eff34105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:26.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.344+0000 7eff388ed640 1 -- 172.21.15.164:0/3144712497 shutdown_connections 2024-09-19T08:52:26.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:26.344+0000 7eff388ed640 1 -- 172.21.15.164:0/3144712497 wait complete. 2024-09-19T08:52:26.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:26 smithi170 ceph-mon[35072]: pgmap v203: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 196 MiB/s wr, 789 op/s 2024-09-19T08:52:26.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:26 smithi170 ceph-mon[35072]: scrub queued for path: / 2024-09-19T08:52:26.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:26 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:52:26.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:26 smithi170 ceph-mon[35072]: scrub summary: active paths [/] 2024-09-19T08:52:26.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:26 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:52:26.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:26 smithi173 ceph-mon[35306]: pgmap v203: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 196 MiB/s wr, 789 op/s 2024-09-19T08:52:26.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:26 smithi173 ceph-mon[35306]: scrub queued for path: / 2024-09-19T08:52:26.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:26 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:52:26.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:26 smithi173 ceph-mon[35306]: scrub summary: active paths [/] 2024-09-19T08:52:26.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:26 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:52:27.053 INFO:tasks.cephfs.filesystem:scrub status for tag:8a8ed284-00da-4814-a611-7a9ab9e8642d - {'path': '/', 'tag': '8a8ed284-00da-4814-a611-7a9ab9e8642d', 'options': 'recursive,force'} 2024-09-19T08:52:28.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:28 smithi164 ceph-mon[30928]: pgmap v204: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 133 MiB/s wr, 551 op/s 2024-09-19T08:52:28.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:28 smithi164 ceph-mon[30928]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T08:52:28.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:28 smithi170 ceph-mon[35072]: pgmap v204: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 133 MiB/s wr, 551 op/s 2024-09-19T08:52:28.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:28 smithi170 ceph-mon[35072]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T08:52:28.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:28 smithi173 ceph-mon[35306]: pgmap v204: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 133 MiB/s wr, 551 op/s 2024-09-19T08:52:28.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:28 smithi173 ceph-mon[35306]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T08:52:30.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:30 smithi164 ceph-mon[30928]: pgmap v205: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 34 MiB/s wr, 160 op/s 2024-09-19T08:52:30.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:30 smithi170 ceph-mon[35072]: pgmap v205: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 34 MiB/s wr, 160 op/s 2024-09-19T08:52:30.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:30 smithi173 ceph-mon[35306]: pgmap v205: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 34 MiB/s wr, 160 op/s 2024-09-19T08:52:32.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:32 smithi164 ceph-mon[30928]: pgmap v206: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 MiB/s wr, 76 op/s 2024-09-19T08:52:32.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:32 smithi170 ceph-mon[35072]: pgmap v206: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 MiB/s wr, 76 op/s 2024-09-19T08:52:32.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:32 smithi173 ceph-mon[35306]: pgmap v206: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 MiB/s wr, 76 op/s 2024-09-19T08:52:33.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:33 smithi164 ceph-mon[30928]: scrub summary: idle 2024-09-19T08:52:33.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:33 smithi164 ceph-mon[30928]: scrub summary: idle 2024-09-19T08:52:33.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:33 smithi170 ceph-mon[35072]: scrub summary: idle 2024-09-19T08:52:33.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:33 smithi170 ceph-mon[35072]: scrub summary: idle 2024-09-19T08:52:33.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:33 smithi173 ceph-mon[35306]: scrub summary: idle 2024-09-19T08:52:33.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:33 smithi173 ceph-mon[35306]: scrub summary: idle 2024-09-19T08:52:34.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:34 smithi164 ceph-mon[30928]: pgmap v207: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 KiB/s wr, 12 op/s 2024-09-19T08:52:34.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:34 smithi170 ceph-mon[35072]: pgmap v207: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 KiB/s wr, 12 op/s 2024-09-19T08:52:34.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:34 smithi173 ceph-mon[35306]: pgmap v207: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 KiB/s wr, 12 op/s 2024-09-19T08:52:36.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:36 smithi164 ceph-mon[30928]: pgmap v208: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.0 MiB/s wr, 82 op/s 2024-09-19T08:52:36.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:36 smithi170 ceph-mon[35072]: pgmap v208: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.0 MiB/s wr, 82 op/s 2024-09-19T08:52:36.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:36 smithi173 ceph-mon[35306]: pgmap v208: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.0 MiB/s wr, 82 op/s 2024-09-19T08:52:38.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:38 smithi164 ceph-mon[30928]: pgmap v209: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 221 op/s 2024-09-19T08:52:38.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:38 smithi170 ceph-mon[35072]: pgmap v209: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 221 op/s 2024-09-19T08:52:38.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:38 smithi173 ceph-mon[35306]: pgmap v209: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 221 op/s 2024-09-19T08:52:40.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:40 smithi164 ceph-mon[30928]: pgmap v210: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 256 op/s 2024-09-19T08:52:40.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:40 smithi170 ceph-mon[35072]: pgmap v210: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 256 op/s 2024-09-19T08:52:40.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:40 smithi173 ceph-mon[35306]: pgmap v210: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 256 op/s 2024-09-19T08:52:42.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:42 smithi164 ceph-mon[30928]: pgmap v211: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 293 op/s 2024-09-19T08:52:42.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:42 smithi170 ceph-mon[35072]: pgmap v211: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 293 op/s 2024-09-19T08:52:42.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:42 smithi173 ceph-mon[35306]: pgmap v211: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 293 op/s 2024-09-19T08:52:44.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:44 smithi164 ceph-mon[30928]: pgmap v212: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 20 MiB/s wr, 287 op/s 2024-09-19T08:52:44.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:44 smithi170 ceph-mon[35072]: pgmap v212: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 20 MiB/s wr, 287 op/s 2024-09-19T08:52:44.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:44 smithi173 ceph-mon[35306]: pgmap v212: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 20 MiB/s wr, 287 op/s 2024-09-19T08:52:46.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:46 smithi164 ceph-mon[30928]: pgmap v213: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 20 MiB/s wr, 288 op/s 2024-09-19T08:52:46.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:46 smithi170 ceph-mon[35072]: pgmap v213: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 20 MiB/s wr, 288 op/s 2024-09-19T08:52:46.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:46 smithi173 ceph-mon[35306]: pgmap v213: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 20 MiB/s wr, 288 op/s 2024-09-19T08:52:48.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:48 smithi164 ceph-mon[30928]: pgmap v214: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 220 op/s 2024-09-19T08:52:48.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:48 smithi170 ceph-mon[35072]: pgmap v214: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 220 op/s 2024-09-19T08:52:48.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:48 smithi173 ceph-mon[35306]: pgmap v214: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 220 op/s 2024-09-19T08:52:50.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:50 smithi164 ceph-mon[30928]: pgmap v215: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 5.1 MiB/s wr, 80 op/s 2024-09-19T08:52:50.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:50 smithi170 ceph-mon[35072]: pgmap v215: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 5.1 MiB/s wr, 80 op/s 2024-09-19T08:52:50.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:50 smithi173 ceph-mon[35306]: pgmap v215: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 5.1 MiB/s wr, 80 op/s 2024-09-19T08:52:52.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:52 smithi164 ceph-mon[30928]: pgmap v216: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.4 MiB/s wr, 45 op/s 2024-09-19T08:52:52.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:52 smithi170 ceph-mon[35072]: pgmap v216: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.4 MiB/s wr, 45 op/s 2024-09-19T08:52:52.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:52 smithi173 ceph-mon[35306]: pgmap v216: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.4 MiB/s wr, 45 op/s 2024-09-19T08:52:54.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:54 smithi164 ceph-mon[30928]: pgmap v217: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:52:54.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:54 smithi170 ceph-mon[35072]: pgmap v217: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:52:54.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:54 smithi173 ceph-mon[35306]: pgmap v217: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:52:56.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:56 smithi164 ceph-mon[30928]: pgmap v218: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-19T08:52:56.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:56 smithi170 ceph-mon[35072]: pgmap v218: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-19T08:52:56.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:56 smithi173 ceph-mon[35306]: pgmap v218: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-19T08:52:57.237 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:52:57.502 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:52:57.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.983+0000 7fcc0d114640 1 -- 172.21.15.164:0/1681526226 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc08106f20 msgr2=0x7fcc0810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:57.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.983+0000 7fcc0d114640 1 --2- 172.21.15.164:0/1681526226 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc08106f20 0x7fcc0810eaa0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fcc00009ca0 tx=0x7fcc0002a8c0 comp rx=0 tx=0).stop 2024-09-19T08:52:57.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.984+0000 7fcc0d114640 1 -- 172.21.15.164:0/1681526226 shutdown_connections 2024-09-19T08:52:57.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.984+0000 7fcc0d114640 1 --2- 172.21.15.164:0/1681526226 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc08106f20 0x7fcc0810eaa0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:57.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.984+0000 7fcc0d114640 1 --2- 172.21.15.164:0/1681526226 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc081027a0 0x7fcc081067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:57.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.984+0000 7fcc0d114640 1 --2- 172.21.15.164:0/1681526226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc08101df0 0x7fcc081021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:57.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.984+0000 7fcc0d114640 1 -- 172.21.15.164:0/1681526226 >> 172.21.15.164:0/1681526226 conn(0x7fcc080fd8b0 msgr2=0x7fcc080ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:57.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.984+0000 7fcc0d114640 1 -- 172.21.15.164:0/1681526226 shutdown_connections 2024-09-19T08:52:57.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.984+0000 7fcc0d114640 1 -- 172.21.15.164:0/1681526226 wait complete. 2024-09-19T08:52:57.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.984+0000 7fcc0d114640 1 Processor -- start 2024-09-19T08:52:57.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.985+0000 7fcc0d114640 1 -- start start 2024-09-19T08:52:57.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.985+0000 7fcc0d114640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc08101df0 0x7fcc0819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:57.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.985+0000 7fcc0d114640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc081027a0 0x7fcc0819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:57.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.985+0000 7fcc0d114640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc08106f20 0x7fcc0819ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:57.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.985+0000 7fcc0d114640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcc080760e0 con 0x7fcc08106f20 2024-09-19T08:52:57.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.985+0000 7fcc0d114640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcc08075f60 con 0x7fcc081027a0 2024-09-19T08:52:57.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.985+0000 7fcc0d114640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcc08076260 con 0x7fcc08101df0 2024-09-19T08:52:57.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.985+0000 7fcc0759e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc08106f20 0x7fcc0819ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:57.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.985+0000 7fcc0759e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc08106f20 0x7fcc0819ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47256/0 (socket says 172.21.15.164:47256) 2024-09-19T08:52:57.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.985+0000 7fcc0759e640 1 -- 172.21.15.164:0/472607556 learned_addr learned my addr 172.21.15.164:0/472607556 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:57.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.985+0000 7fcc0659c640 1 --2- 172.21.15.164:0/472607556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc081027a0 0x7fcc0819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:57.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.985+0000 7fcc06d9d640 1 --2- 172.21.15.164:0/472607556 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc08101df0 0x7fcc0819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:57.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.985+0000 7fcc0759e640 1 -- 172.21.15.164:0/472607556 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc08101df0 msgr2=0x7fcc0819a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:57.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.985+0000 7fcc0759e640 1 --2- 172.21.15.164:0/472607556 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc08101df0 0x7fcc0819a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:57.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.985+0000 7fcc0759e640 1 -- 172.21.15.164:0/472607556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc081027a0 msgr2=0x7fcc0819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:57.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.985+0000 7fcc0759e640 1 --2- 172.21.15.164:0/472607556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc081027a0 0x7fcc0819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:57.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.985+0000 7fcc0759e640 1 -- 172.21.15.164:0/472607556 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc0819f540 con 0x7fcc08106f20 2024-09-19T08:52:57.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.985+0000 7fcc0659c640 1 --2- 172.21.15.164:0/472607556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc081027a0 0x7fcc0819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:52:57.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.986+0000 7fcc0759e640 1 --2- 172.21.15.164:0/472607556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc08106f20 0x7fcc0819ee60 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fcc000098d0 tx=0x7fcc00003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:57.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.986+0000 7fcbeffff640 1 -- 172.21.15.164:0/472607556 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc000316d0 con 0x7fcc08106f20 2024-09-19T08:52:57.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.986+0000 7fcbeffff640 1 -- 172.21.15.164:0/472607556 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc00031870 con 0x7fcc08106f20 2024-09-19T08:52:57.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.986+0000 7fcc0d114640 1 -- 172.21.15.164:0/472607556 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc0819f7d0 con 0x7fcc08106f20 2024-09-19T08:52:57.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.986+0000 7fcbeffff640 1 -- 172.21.15.164:0/472607556 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0002fc00 con 0x7fcc08106f20 2024-09-19T08:52:57.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.986+0000 7fcc0d114640 1 -- 172.21.15.164:0/472607556 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc081a7110 con 0x7fcc08106f20 2024-09-19T08:52:57.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.988+0000 7fcbeffff640 1 -- 172.21.15.164:0/472607556 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fcc00004040 con 0x7fcc08106f20 2024-09-19T08:52:57.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.989+0000 7fcc0f4ed740 1 Processor -- start 2024-09-19T08:52:57.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.989+0000 7fcc0f4ed740 1 -- start start 2024-09-19T08:52:57.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.989+0000 7fcc0f4ed740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c379f184a0 0x55c379f18880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:57.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.989+0000 7fcc0f4ed740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c379f18dc0 0x55c379f23140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:57.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.989+0000 7fcc0f4ed740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c379f23680 0x55c379f25a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:57.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.989+0000 7fcc0f4ed740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c379e7e2e0 con 0x55c379f23680 2024-09-19T08:52:57.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.989+0000 7fcc0f4ed740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c379e25ce0 con 0x55c379f184a0 2024-09-19T08:52:57.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.989+0000 7fcc0f4ed740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c379e7a2a0 con 0x55c379f18dc0 2024-09-19T08:52:57.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.989+0000 7fcbeffff640 1 --2- 172.21.15.164:0/472607556 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fcbf00a3480 0x7fcbf00a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:57.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.989+0000 7fcc0659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c379f23680 0x55c379f25a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:57.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.989+0000 7fcc0659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c379f23680 0x55c379f25a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47258/0 (socket says 172.21.15.164:47258) 2024-09-19T08:52:57.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.989+0000 7fcc0659c640 1 -- 172.21.15.164:0/1151901925 learned_addr learned my addr 172.21.15.164:0/1151901925 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:57.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.989+0000 7fcc06d9d640 1 --2- 172.21.15.164:0/1151901925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c379f18dc0 0x55c379f23140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:57.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.989+0000 7fcc0759e640 1 --2- 172.21.15.164:0/472607556 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fcbf00a3480 0x7fcbf00a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:57.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.989+0000 7fcc0659c640 1 --2- 172.21.15.164:0/1151901925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c379f184a0 0x55c379f18880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:57.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.989+0000 7fcbeffff640 1 -- 172.21.15.164:0/472607556 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7fcc000f1200 con 0x7fcc08106f20 2024-09-19T08:52:57.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.990+0000 7fcc06d9d640 1 -- 172.21.15.164:0/1151901925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c379f184a0 msgr2=0x55c379f18880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:57.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.990+0000 7fcc06d9d640 1 --2- 172.21.15.164:0/1151901925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c379f184a0 0x55c379f18880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:57.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.990+0000 7fcc06d9d640 1 -- 172.21.15.164:0/1151901925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c379f23680 msgr2=0x55c379f25a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:57.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.990+0000 7fcc06d9d640 1 --2- 172.21.15.164:0/1151901925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c379f23680 0x55c379f25a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:57.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.990+0000 7fcc06d9d640 1 -- 172.21.15.164:0/1151901925 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c379e0a440 con 0x55c379f18dc0 2024-09-19T08:52:57.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.990+0000 7fcc0659c640 1 --2- 172.21.15.164:0/1151901925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c379f23680 0x55c379f25a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:52:57.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.990+0000 7fcc0759e640 1 --2- 172.21.15.164:0/472607556 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fcbf00a3480 0x7fcbf00a5940 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcc000bd5f0 tx=0x7fcc0008f500 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:57.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.990+0000 7fcc06d9d640 1 --2- 172.21.15.164:0/1151901925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c379f18dc0 0x55c379f23140 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800bbf0 tx=0x7fcbf80107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eb6673a578e8038 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:57.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.990+0000 7fcbed7fa640 1 -- 172.21.15.164:0/1151901925 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8004030 con 0x55c379f18dc0 2024-09-19T08:52:57.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.990+0000 7fcbed7fa640 1 -- 172.21.15.164:0/1151901925 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbf80041d0 con 0x55c379f18dc0 2024-09-19T08:52:57.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.990+0000 7fcbed7fa640 1 -- 172.21.15.164:0/1151901925 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8011640 con 0x55c379f18dc0 2024-09-19T08:52:57.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.991+0000 7fcc0f4ed740 1 -- 172.21.15.164:0/1151901925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c379f18dc0 msgr2=0x55c379f23140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:57.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.991+0000 7fcc0f4ed740 1 --2- 172.21.15.164:0/1151901925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c379f18dc0 0x55c379f23140 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800bbf0 tx=0x7fcbf80107a0 comp rx=0 tx=0).stop 2024-09-19T08:52:57.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.991+0000 7fcc0f4ed740 1 -- 172.21.15.164:0/1151901925 shutdown_connections 2024-09-19T08:52:57.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.991+0000 7fcc0f4ed740 1 --2- 172.21.15.164:0/1151901925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c379f23680 0x55c379f25a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:57.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.991+0000 7fcc0f4ed740 1 --2- 172.21.15.164:0/1151901925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c379f18dc0 0x55c379f23140 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:57.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.991+0000 7fcc0f4ed740 1 --2- 172.21.15.164:0/1151901925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c379f184a0 0x55c379f18880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:57.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.991+0000 7fcc0f4ed740 1 -- 172.21.15.164:0/1151901925 >> 172.21.15.164:0/1151901925 conn(0x55c379f14a30 msgr2=0x55c379f14e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:57.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.991+0000 7fcc0f4ed740 1 -- 172.21.15.164:0/1151901925 shutdown_connections 2024-09-19T08:52:57.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.991+0000 7fcc0f4ed740 1 -- 172.21.15.164:0/1151901925 wait complete. 2024-09-19T08:52:57.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.991+0000 7fcc0f4ed740 1 -- start start 2024-09-19T08:52:57.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.991+0000 7fcc0f4ed740 1 Processor -- start 2024-09-19T08:52:57.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.991+0000 7fcc0f4ed740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c379f184a0 0x55c379f22fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:57.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.991+0000 7fcc0f4ed740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c379f18dc0 0x55c379f20280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:57.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.991+0000 7fcc0f4ed740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c379f23680 0x55c379f207e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:57.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.991+0000 7fcc0f4ed740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c379f26250 con 0x55c379f23680 2024-09-19T08:52:57.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.991+0000 7fcc0f4ed740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c379e39050 con 0x55c379f184a0 2024-09-19T08:52:57.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.991+0000 7fcc0f4ed740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c379f263d0 con 0x55c379f18dc0 2024-09-19T08:52:57.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.992+0000 7fcc0f4ed740 10 client.0 _ll_register_callbacks cb 0x55c379e79160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:52:57.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.992+0000 7fcc0659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c379f23680 0x55c379f207e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:57.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.992+0000 7fcc06d9d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c379f18dc0 0x55c379f20280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:57.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.992+0000 7fcc0659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c379f23680 0x55c379f207e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47268/0 (socket says 172.21.15.164:47268) 2024-09-19T08:52:57.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.992+0000 7fcc0659c640 1 -- 172.21.15.164:0/808003831 learned_addr learned my addr 172.21.15.164:0/808003831 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:57.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.992+0000 7fcc0659c640 1 --2- 172.21.15.164:0/808003831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c379f184a0 0x55c379f22fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:57.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.992+0000 7fcc0659c640 1 -- 172.21.15.164:0/808003831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c379f18dc0 msgr2=0x55c379f20280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:57.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.992+0000 7fcc0659c640 1 --2- 172.21.15.164:0/808003831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c379f18dc0 0x55c379f20280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:57.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.992+0000 7fcc0659c640 1 -- 172.21.15.164:0/808003831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c379f184a0 msgr2=0x55c379f22fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:57.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.992+0000 7fcc0659c640 1 --2- 172.21.15.164:0/808003831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c379f184a0 0x55c379f22fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:57.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.992+0000 7fcc0659c640 1 -- 172.21.15.164:0/808003831 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c379bef3c0 con 0x55c379f23680 2024-09-19T08:52:57.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.992+0000 7fcc0659c640 1 --2- 172.21.15.164:0/808003831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c379f23680 0x55c379f207e0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc012e80 tx=0x7fcbfc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07bd4f281c6fb87 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:57.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.992+0000 7fcbeaff5640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:52:57.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.992+0000 7fcbeaff5640 1 -- 172.21.15.164:0/808003831 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc004020 con 0x55c379f23680 2024-09-19T08:52:57.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.992+0000 7fcbeaff5640 1 -- 172.21.15.164:0/808003831 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7fcbfc0041c0 con 0x55c379f23680 2024-09-19T08:52:57.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.992+0000 7fcc0f4ed740 1 -- 172.21.15.164:0/808003831 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbf80041d0 con 0x55c379f23680 2024-09-19T08:52:57.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.992+0000 7fcbeaff5640 1 -- 172.21.15.164:0/808003831 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcbfc02b050 con 0x55c379f23680 2024-09-19T08:52:57.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.992+0000 7fcc0f4ed740 10 client.16887 fetch_fsmap learned FSMap version 32 2024-09-19T08:52:57.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.992+0000 7fcc0f4ed740 1 -- 172.21.15.164:0/808003831 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c379f31c80 con 0x55c379f23680 2024-09-19T08:52:57.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.993+0000 7fcbeaff5640 1 -- 172.21.15.164:0/808003831 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbf80041d0 con 0x55c379f23680 2024-09-19T08:52:57.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.993+0000 7fcbeaff5640 1 -- 172.21.15.164:0/808003831 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fcbfc004830 con 0x55c379f23680 2024-09-19T08:52:57.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.993+0000 7fcc0f4ed740 10 client.16887 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:52:57.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.993+0000 7fcc0f4ed740 10 client.16887 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:52:57.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.993+0000 7fcc0f4ed740 20 client.16887 populate_metadata read hostname 'smithi164' 2024-09-19T08:52:57.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.993+0000 7fcc0f4ed740 1 --2- 172.21.15.164:0/808003831 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c379f33270 0x55c379f35660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:57.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.993+0000 7fcc0f4ed740 4 client.16887 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:52:57.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.993+0000 7fcc0f4ed740 1 -- 172.21.15.164:0/808003831 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c379e8b850 con 0x55c379f33270 2024-09-19T08:52:57.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.993+0000 7fcc06d9d640 1 --2- 172.21.15.164:0/808003831 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c379f33270 0x55c379f35660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:58.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.994+0000 7fcc06d9d640 1 --2- 172.21.15.164:0/808003831 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c379f33270 0x55c379f35660 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37d0155e9d16455d server_cookie=a899973b54cba22f in_seq=0 out_seq=0 2024-09-19T08:52:58.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.994+0000 7fcbeaff5640 10 client.16887 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:52:58.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.994+0000 7fcbeaff5640 1 -- 172.21.15.164:0/808003831 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c379e8b850 con 0x55c379f33270 2024-09-19T08:52:58.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.994+0000 7fcbeaff5640 10 client.16887 handle_command_reply: tid=0 2024-09-19T08:52:58.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.994+0000 7fcc0f4ed740 1 client.16887 shutdown 2024-09-19T08:52:58.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.995+0000 7fcc0f4ed740 1 -- 172.21.15.164:0/808003831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c379f23680 msgr2=0x55c379f207e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:58.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.995+0000 7fcc0f4ed740 1 --2- 172.21.15.164:0/808003831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c379f23680 0x55c379f207e0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc012e80 tx=0x7fcbfc004730 comp rx=0 tx=0).stop 2024-09-19T08:52:58.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.995+0000 7fcc0f4ed740 1 -- 172.21.15.164:0/808003831 shutdown_connections 2024-09-19T08:52:58.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.995+0000 7fcc0f4ed740 1 --2- 172.21.15.164:0/808003831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c379f23680 0x55c379f207e0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:58.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.995+0000 7fcc0f4ed740 1 --2- 172.21.15.164:0/808003831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c379f18dc0 0x55c379f20280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:58.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.995+0000 7fcc0f4ed740 1 --2- 172.21.15.164:0/808003831 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c379f33270 0x55c379f35660 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:58.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.995+0000 7fcc0f4ed740 1 --2- 172.21.15.164:0/808003831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c379f184a0 0x55c379f22fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:58.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.995+0000 7fcbeaff5640 0 client.16887 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:52:58.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.995+0000 7fcc0f4ed740 1 -- 172.21.15.164:0/808003831 >> 172.21.15.164:0/808003831 conn(0x55c379f14a30 msgr2=0x55c379f1a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:58.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.995+0000 7fcc0f4ed740 1 -- 172.21.15.164:0/808003831 shutdown_connections 2024-09-19T08:52:58.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.995+0000 7fcc0f4ed740 1 -- 172.21.15.164:0/808003831 wait complete. 2024-09-19T08:52:58.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:57.995+0000 7fcc0f4ed740 20 client.16887 trim_cache size 0 max 16384 2024-09-19T08:52:58.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.011+0000 7fcc0f4ed740 1 Processor -- start 2024-09-19T08:52:58.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.011+0000 7fcc0f4ed740 1 -- start start 2024-09-19T08:52:58.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.012+0000 7fcc0f4ed740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c379f184a0 0x55c379f2f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:58.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.012+0000 7fcc0f4ed740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c379f18dc0 0x55c379f221f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:58.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.012+0000 7fcc0f4ed740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c379f23680 0x55c379f3ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:58.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.012+0000 7fcc0f4ed740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcc081af960 con 0x55c379f184a0 2024-09-19T08:52:58.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.012+0000 7fcc0f4ed740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcc081b14a0 con 0x55c379f23680 2024-09-19T08:52:58.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.012+0000 7fcc0f4ed740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcc081b0030 con 0x55c379f18dc0 2024-09-19T08:52:58.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.012+0000 7fcc0659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c379f184a0 0x55c379f2f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:58.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.012+0000 7fcc0659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c379f184a0 0x55c379f2f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47276/0 (socket says 172.21.15.164:47276) 2024-09-19T08:52:58.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.012+0000 7fcc0659c640 1 -- 172.21.15.164:0/2699662433 learned_addr learned my addr 172.21.15.164:0/2699662433 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:58.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.012+0000 7fcc06d9d640 1 --2- 172.21.15.164:0/2699662433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c379f18dc0 0x55c379f221f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:58.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.012+0000 7fcc0659c640 1 -- 172.21.15.164:0/2699662433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c379f18dc0 msgr2=0x55c379f221f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:58.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.012+0000 7fcc0659c640 1 --2- 172.21.15.164:0/2699662433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c379f18dc0 0x55c379f221f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:58.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.012+0000 7fcc0659c640 1 -- 172.21.15.164:0/2699662433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c379f23680 msgr2=0x55c379f3ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:58.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.012+0000 7fcc0659c640 1 --2- 172.21.15.164:0/2699662433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c379f23680 0x55c379f3ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:58.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.012+0000 7fcc0659c640 1 -- 172.21.15.164:0/2699662433 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c379f144d0 con 0x55c379f184a0 2024-09-19T08:52:58.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.012+0000 7fcc06d9d640 1 --2- 172.21.15.164:0/2699662433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c379f18dc0 0x55c379f221f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:52:58.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.012+0000 7fcc0659c640 1 --2- 172.21.15.164:0/2699662433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c379f184a0 0x55c379f2f4b0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc000c00 tx=0x7fcbfc005c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce0b89b976d31ff6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:58.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.012+0000 7fcbecff9640 1 -- 172.21.15.164:0/2699662433 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc012660 con 0x55c379f184a0 2024-09-19T08:52:58.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.012+0000 7fcbecff9640 1 -- 172.21.15.164:0/2699662433 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbfc0120e0 con 0x55c379f184a0 2024-09-19T08:52:58.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.012+0000 7fcbecff9640 1 -- 172.21.15.164:0/2699662433 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc004fe0 con 0x55c379f184a0 2024-09-19T08:52:58.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.013+0000 7fcc0f4ed740 1 -- 172.21.15.164:0/2699662433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c379f184a0 msgr2=0x55c379f2f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:58.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.013+0000 7fcc0f4ed740 1 --2- 172.21.15.164:0/2699662433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c379f184a0 0x55c379f2f4b0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc000c00 tx=0x7fcbfc005c10 comp rx=0 tx=0).stop 2024-09-19T08:52:58.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.013+0000 7fcc0f4ed740 1 -- 172.21.15.164:0/2699662433 shutdown_connections 2024-09-19T08:52:58.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.013+0000 7fcc0f4ed740 1 --2- 172.21.15.164:0/2699662433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c379f23680 0x55c379f3ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:58.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.013+0000 7fcc0f4ed740 1 --2- 172.21.15.164:0/2699662433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c379f18dc0 0x55c379f221f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:58.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.013+0000 7fcc0f4ed740 1 --2- 172.21.15.164:0/2699662433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c379f184a0 0x55c379f2f4b0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:58.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.013+0000 7fcc0f4ed740 1 -- 172.21.15.164:0/2699662433 >> 172.21.15.164:0/2699662433 conn(0x55c379f14a30 msgr2=0x55c379f21ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:58.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.013+0000 7fcc0f4ed740 1 -- 172.21.15.164:0/2699662433 shutdown_connections 2024-09-19T08:52:58.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.013+0000 7fcc0f4ed740 1 -- 172.21.15.164:0/2699662433 wait complete. 2024-09-19T08:52:58.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.013+0000 7fcc0f4ed740 1 -- start start 2024-09-19T08:52:58.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.013+0000 7fcc0f4ed740 1 Processor -- start 2024-09-19T08:52:58.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.014+0000 7fcc0f4ed740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c379f184a0 0x55c379f2bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:58.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.014+0000 7fcc0f4ed740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c379f18dc0 0x55c379f26fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:58.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.014+0000 7fcc0f4ed740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c379f23680 0x55c379f274f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:58.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.014+0000 7fcc0f4ed740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcc081b0dd0 con 0x55c379f23680 2024-09-19T08:52:58.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.014+0000 7fcc0f4ed740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcc081b0700 con 0x55c379f18dc0 2024-09-19T08:52:58.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.014+0000 7fcc0f4ed740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcc081ad960 con 0x55c379f184a0 2024-09-19T08:52:58.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.014+0000 7fcc0f4ed740 10 client.0 _ll_register_callbacks cb 0x7fcbfc012890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:52:58.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.014+0000 7fcc0659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c379f23680 0x55c379f274f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:58.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.014+0000 7fcc0659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c379f23680 0x55c379f274f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47288/0 (socket says 172.21.15.164:47288) 2024-09-19T08:52:58.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.014+0000 7fcc0659c640 1 -- 172.21.15.164:0/891435820 learned_addr learned my addr 172.21.15.164:0/891435820 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:58.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.014+0000 7fcc06d9d640 1 --2- 172.21.15.164:0/891435820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c379f18dc0 0x55c379f26fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:58.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.014+0000 7fcc0659c640 1 -- 172.21.15.164:0/891435820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c379f184a0 msgr2=0x55c379f2bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:58.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.014+0000 7fcc0659c640 1 --2- 172.21.15.164:0/891435820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c379f184a0 0x55c379f2bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:58.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.014+0000 7fcc0659c640 1 -- 172.21.15.164:0/891435820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c379f18dc0 msgr2=0x55c379f26fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:52:58.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.014+0000 7fcc0659c640 1 --2- 172.21.15.164:0/891435820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c379f18dc0 0x55c379f26fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:58.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.014+0000 7fcc0659c640 1 -- 172.21.15.164:0/891435820 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c379f27d20 con 0x55c379f23680 2024-09-19T08:52:58.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.014+0000 7fcc06d9d640 1 --2- 172.21.15.164:0/891435820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c379f18dc0 0x55c379f26fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:52:58.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.014+0000 7fcc0659c640 1 --2- 172.21.15.164:0/891435820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c379f23680 0x55c379f274f0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc004d00 tx=0x7fcbfc004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9934ec345e63eae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:58.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.014+0000 7fcbeaff5640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:52:58.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.014+0000 7fcbeaff5640 1 -- 172.21.15.164:0/891435820 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc034040 con 0x55c379f23680 2024-09-19T08:52:58.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.014+0000 7fcbeaff5640 1 -- 172.21.15.164:0/891435820 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7fcbfc00ebd0 con 0x55c379f23680 2024-09-19T08:52:58.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.014+0000 7fcc0f4ed740 1 -- 172.21.15.164:0/891435820 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbfc0120e0 con 0x55c379f23680 2024-09-19T08:52:58.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.015+0000 7fcbeaff5640 1 -- 172.21.15.164:0/891435820 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcbfc012660 con 0x55c379f23680 2024-09-19T08:52:58.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.015+0000 7fcc0f4ed740 10 client.16893 fetch_fsmap learned FSMap version 32 2024-09-19T08:52:58.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.015+0000 7fcc0f4ed740 1 -- 172.21.15.164:0/891435820 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c379f9dbe0 con 0x55c379f23680 2024-09-19T08:52:58.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.015+0000 7fcbeaff5640 1 -- 172.21.15.164:0/891435820 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbfc0120e0 con 0x55c379f23680 2024-09-19T08:52:58.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.015+0000 7fcbeaff5640 1 -- 172.21.15.164:0/891435820 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fcbfc0243f0 con 0x55c379f23680 2024-09-19T08:52:58.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.015+0000 7fcc0f4ed740 10 client.16893 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:52:58.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.015+0000 7fcc0f4ed740 10 client.16893 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:52:58.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.015+0000 7fcc0f4ed740 20 client.16893 populate_metadata read hostname 'smithi164' 2024-09-19T08:52:58.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.015+0000 7fcc0f4ed740 1 --2- 172.21.15.164:0/891435820 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c379f9e1a0 0x55c379fa0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:58.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.015+0000 7fcc0f4ed740 4 client.16893 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:52:58.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.015+0000 7fcc0f4ed740 1 -- 172.21.15.164:0/891435820 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c379e85d60 con 0x55c379f9e1a0 2024-09-19T08:52:58.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.016+0000 7fcc06d9d640 1 --2- 172.21.15.164:0/891435820 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c379f9e1a0 0x55c379fa0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:58.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.016+0000 7fcc06d9d640 1 --2- 172.21.15.164:0/891435820 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c379f9e1a0 0x55c379fa0590 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f2eeed507c9b273 server_cookie=b1feebd566612ffc in_seq=0 out_seq=0 2024-09-19T08:52:58.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.016+0000 7fcbeaff5640 10 client.16893 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:52:58.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.016+0000 7fcbeaff5640 1 -- 172.21.15.164:0/891435820 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c379e85d60 con 0x55c379f9e1a0 2024-09-19T08:52:58.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.016+0000 7fcbeaff5640 10 client.16893 handle_command_reply: tid=0 2024-09-19T08:52:58.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.016+0000 7fcc0f4ed740 1 client.16893 shutdown 2024-09-19T08:52:58.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.016+0000 7fcc0f4ed740 1 -- 172.21.15.164:0/891435820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c379f23680 msgr2=0x55c379f274f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:58.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.016+0000 7fcc0f4ed740 1 --2- 172.21.15.164:0/891435820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c379f23680 0x55c379f274f0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc004d00 tx=0x7fcbfc004be0 comp rx=0 tx=0).stop 2024-09-19T08:52:58.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.016+0000 7fcc0f4ed740 1 -- 172.21.15.164:0/891435820 shutdown_connections 2024-09-19T08:52:58.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.016+0000 7fcc0f4ed740 1 --2- 172.21.15.164:0/891435820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c379f23680 0x55c379f274f0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:58.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.016+0000 7fcc0f4ed740 1 --2- 172.21.15.164:0/891435820 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c379f9e1a0 0x55c379fa0590 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:58.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.016+0000 7fcc0f4ed740 1 --2- 172.21.15.164:0/891435820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c379f18dc0 0x55c379f26fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:58.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.016+0000 7fcc0f4ed740 1 --2- 172.21.15.164:0/891435820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c379f184a0 0x55c379f2bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:58.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.016+0000 7fcc0f4ed740 1 -- 172.21.15.164:0/891435820 >> 172.21.15.164:0/891435820 conn(0x55c379f14a30 msgr2=0x55c379f21ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:58.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.016+0000 7fcbeaff5640 0 client.16893 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:52:58.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.017+0000 7fcc0f4ed740 1 -- 172.21.15.164:0/891435820 shutdown_connections 2024-09-19T08:52:58.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.017+0000 7fcc0f4ed740 1 -- 172.21.15.164:0/891435820 wait complete. 2024-09-19T08:52:58.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.017+0000 7fcc0f4ed740 20 client.16893 trim_cache size 0 max 16384 2024-09-19T08:52:58.024 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:52:58.024 INFO:teuthology.orchestra.run.smithi164.stdout: "status": "no active scrubs running", 2024-09-19T08:52:58.024 INFO:teuthology.orchestra.run.smithi164.stdout: "scrubs": {} 2024-09-19T08:52:58.024 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:52:58.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.017+0000 7fcc0d114640 1 -- 172.21.15.164:0/472607556 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fcbf00a3480 msgr2=0x7fcbf00a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:58.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.017+0000 7fcc0d114640 1 --2- 172.21.15.164:0/472607556 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fcbf00a3480 0x7fcbf00a5940 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcc000bd5f0 tx=0x7fcc0008f500 comp rx=0 tx=0).stop 2024-09-19T08:52:58.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.017+0000 7fcc0d114640 1 -- 172.21.15.164:0/472607556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc08106f20 msgr2=0x7fcc0819ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:58.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.017+0000 7fcc0d114640 1 --2- 172.21.15.164:0/472607556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc08106f20 0x7fcc0819ee60 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fcc000098d0 tx=0x7fcc00003f10 comp rx=0 tx=0).stop 2024-09-19T08:52:58.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.017+0000 7fcc0d114640 1 -- 172.21.15.164:0/472607556 shutdown_connections 2024-09-19T08:52:58.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.017+0000 7fcc0d114640 1 --2- 172.21.15.164:0/472607556 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fcbf00a3480 0x7fcbf00a5940 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:58.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.017+0000 7fcc0d114640 1 --2- 172.21.15.164:0/472607556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc08106f20 0x7fcc0819ee60 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:58.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.017+0000 7fcc0d114640 1 --2- 172.21.15.164:0/472607556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc081027a0 0x7fcc0819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:58.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.017+0000 7fcc0d114640 1 --2- 172.21.15.164:0/472607556 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc08101df0 0x7fcc0819a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:58.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.017+0000 7fcc0d114640 1 -- 172.21.15.164:0/472607556 >> 172.21.15.164:0/472607556 conn(0x7fcc080fd8b0 msgr2=0x7fcc080feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:58.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.017+0000 7fcc0d114640 1 -- 172.21.15.164:0/472607556 shutdown_connections 2024-09-19T08:52:58.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:58.017+0000 7fcc0d114640 1 -- 172.21.15.164:0/472607556 wait complete. 2024-09-19T08:52:58.531 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:52:58 smithi164 ceph-mon[30928]: pgmap v219: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 op/s 2024-09-19T08:52:58.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:52:58 smithi170 ceph-mon[35072]: pgmap v219: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 op/s 2024-09-19T08:52:58.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:52:58 smithi173 ceph-mon[35306]: pgmap v219: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 op/s 2024-09-19T08:52:58.843 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T08:52:58.843 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:52:59.091 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:52:59.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.579+0000 7faaa8ffa640 1 -- 172.21.15.164:0/872239657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faaa4102790 msgr2=0x7faaa41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:59.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.579+0000 7faaa8ffa640 1 --2- 172.21.15.164:0/872239657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faaa4102790 0x7faaa41067e0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7faa8c009900 tx=0x7faa8c02a410 comp rx=0 tx=0).stop 2024-09-19T08:52:59.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.579+0000 7faaa8ffa640 1 -- 172.21.15.164:0/872239657 shutdown_connections 2024-09-19T08:52:59.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.579+0000 7faaa8ffa640 1 --2- 172.21.15.164:0/872239657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faaa4106f10 0x7faaa410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:59.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.580+0000 7faaa8ffa640 1 --2- 172.21.15.164:0/872239657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faaa4102790 0x7faaa41067e0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:59.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.580+0000 7faaa8ffa640 1 --2- 172.21.15.164:0/872239657 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faaa4101de0 0x7faaa41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:59.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.580+0000 7faaa8ffa640 1 -- 172.21.15.164:0/872239657 >> 172.21.15.164:0/872239657 conn(0x7faaa40fd8c0 msgr2=0x7faaa40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:59.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.580+0000 7faaa8ffa640 1 -- 172.21.15.164:0/872239657 shutdown_connections 2024-09-19T08:52:59.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.580+0000 7faaa8ffa640 1 -- 172.21.15.164:0/872239657 wait complete. 2024-09-19T08:52:59.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.580+0000 7faaa8ffa640 1 Processor -- start 2024-09-19T08:52:59.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.580+0000 7faaa8ffa640 1 -- start start 2024-09-19T08:52:59.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.580+0000 7faaa8ffa640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faaa4101de0 0x7faaa4198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:59.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.580+0000 7faaa8ffa640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faaa4102790 0x7faaa4198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:59.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.581+0000 7faaa8ffa640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faaa4106f10 0x7faaa419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:59.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.581+0000 7faaa8ffa640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faaa4111f90 con 0x7faaa4101de0 2024-09-19T08:52:59.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.581+0000 7faaa8ffa640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7faaa4111e10 con 0x7faaa4106f10 2024-09-19T08:52:59.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.581+0000 7faaa8ffa640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faaa4112110 con 0x7faaa4102790 2024-09-19T08:52:59.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.581+0000 7faaa259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faaa4101de0 0x7faaa4198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:59.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.581+0000 7faaa259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faaa4101de0 0x7faaa4198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47314/0 (socket says 172.21.15.164:47314) 2024-09-19T08:52:59.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.581+0000 7faaa259c640 1 -- 172.21.15.164:0/3323358999 learned_addr learned my addr 172.21.15.164:0/3323358999 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:52:59.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.581+0000 7faaa1d9b640 1 --2- 172.21.15.164:0/3323358999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faaa4102790 0x7faaa4198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:59.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.581+0000 7faaa2d9d640 1 --2- 172.21.15.164:0/3323358999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faaa4106f10 0x7faaa419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:59.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.581+0000 7faaa259c640 1 -- 172.21.15.164:0/3323358999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faaa4102790 msgr2=0x7faaa4198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:59.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.581+0000 7faaa259c640 1 --2- 172.21.15.164:0/3323358999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faaa4102790 0x7faaa4198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:59.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.581+0000 7faaa259c640 1 -- 172.21.15.164:0/3323358999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faaa4106f10 msgr2=0x7faaa419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:59.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.581+0000 7faaa259c640 1 --2- 172.21.15.164:0/3323358999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faaa4106f10 0x7faaa419cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:59.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.581+0000 7faaa259c640 1 -- 172.21.15.164:0/3323358999 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa419d4c0 con 0x7faaa4101de0 2024-09-19T08:52:59.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.581+0000 7faaa2d9d640 1 --2- 172.21.15.164:0/3323358999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faaa4106f10 0x7faaa419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:52:59.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.581+0000 7faaa259c640 1 --2- 172.21.15.164:0/3323358999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faaa4101de0 0x7faaa4198510 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7faa9400b4a0 tx=0x7faa9400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:59.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.581+0000 7faa9b7fe640 1 -- 172.21.15.164:0/3323358999 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa94014070 con 0x7faaa4101de0 2024-09-19T08:52:59.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.581+0000 7faaa8ffa640 1 -- 172.21.15.164:0/3323358999 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaa419d7b0 con 0x7faaa4101de0 2024-09-19T08:52:59.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.581+0000 7faa9b7fe640 1 -- 172.21.15.164:0/3323358999 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa94010af0 con 0x7faaa4101de0 2024-09-19T08:52:59.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.581+0000 7faa9b7fe640 1 -- 172.21.15.164:0/3323358999 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa94017d10 con 0x7faaa4101de0 2024-09-19T08:52:59.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.581+0000 7faaa8ffa640 1 -- 172.21.15.164:0/3323358999 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaa41a5090 con 0x7faaa4101de0 2024-09-19T08:52:59.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.582+0000 7faaa8ffa640 1 -- 172.21.15.164:0/3323358999 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaa41021c0 con 0x7faaa4101de0 2024-09-19T08:52:59.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.583+0000 7faa9b7fe640 1 -- 172.21.15.164:0/3323358999 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7faa94017070 con 0x7faaa4101de0 2024-09-19T08:52:59.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.584+0000 7faa9b7fe640 1 --2- 172.21.15.164:0/3323358999 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7faa900a33b0 0x7faa900a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:52:59.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.584+0000 7faaa1d9b640 1 --2- 172.21.15.164:0/3323358999 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7faa900a33b0 0x7faa900a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:52:59.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.584+0000 7faa9b7fe640 1 -- 172.21.15.164:0/3323358999 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7faa940cf0e0 con 0x7faaa4101de0 2024-09-19T08:52:59.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.585+0000 7faaa1d9b640 1 --2- 172.21.15.164:0/3323358999 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7faa900a33b0 0x7faa900a5870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faaa41087c0 tx=0x7faa8c031490 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:52:59.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.588+0000 7faa9b7fe640 1 -- 172.21.15.164:0/3323358999 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7faa9409bbb0 con 0x7faaa4101de0 2024-09-19T08:52:59.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.789+0000 7faaa8ffa640 1 -- 172.21.15.164:0/3323358999 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faaa410a0f0 con 0x7faaa4101de0 2024-09-19T08:52:59.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.789+0000 7faa9b7fe640 1 -- 172.21.15.164:0/3323358999 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11984 (secure 0 0 0) 0x7faa9409b4b0 con 0x7faaa4101de0 2024-09-19T08:52:59.791 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:52:59.791 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":32,"btime":"2024-09-19T08:52:27:192804+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:52:27.074168+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:52:59.791 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 32 2024-09-19T08:52:59.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.792+0000 7faaa8ffa640 1 -- 172.21.15.164:0/3323358999 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7faa900a33b0 msgr2=0x7faa900a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:59.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.792+0000 7faaa8ffa640 1 --2- 172.21.15.164:0/3323358999 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7faa900a33b0 0x7faa900a5870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faaa41087c0 tx=0x7faa8c031490 comp rx=0 tx=0).stop 2024-09-19T08:52:59.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.792+0000 7faaa8ffa640 1 -- 172.21.15.164:0/3323358999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faaa4101de0 msgr2=0x7faaa4198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:52:59.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.792+0000 7faaa8ffa640 1 --2- 172.21.15.164:0/3323358999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faaa4101de0 0x7faaa4198510 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7faa9400b4a0 tx=0x7faa9400b960 comp rx=0 tx=0).stop 2024-09-19T08:52:59.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.792+0000 7faaa8ffa640 1 -- 172.21.15.164:0/3323358999 shutdown_connections 2024-09-19T08:52:59.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.792+0000 7faaa8ffa640 1 --2- 172.21.15.164:0/3323358999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faaa4106f10 0x7faaa419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:59.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.792+0000 7faaa8ffa640 1 --2- 172.21.15.164:0/3323358999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faaa4102790 0x7faaa4198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:59.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.792+0000 7faaa8ffa640 1 --2- 172.21.15.164:0/3323358999 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7faa900a33b0 0x7faa900a5870 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:59.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.792+0000 7faaa8ffa640 1 --2- 172.21.15.164:0/3323358999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faaa4101de0 0x7faaa4198510 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:52:59.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.792+0000 7faaa8ffa640 1 -- 172.21.15.164:0/3323358999 >> 172.21.15.164:0/3323358999 conn(0x7faaa40fd8c0 msgr2=0x7faaa40fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:52:59.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.792+0000 7faaa8ffa640 1 -- 172.21.15.164:0/3323358999 shutdown_connections 2024-09-19T08:52:59.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:52:59.792+0000 7faaa8ffa640 1 -- 172.21.15.164:0/3323358999 wait complete. 2024-09-19T08:53:00.506 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:00 smithi164 ceph-mon[30928]: pgmap v220: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:53:00.506 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:00 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3323358999' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:53:00.535 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T08:53:00.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:00 smithi170 ceph-mon[35072]: pgmap v220: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:53:00.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:00 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3323358999' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:53:00.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:00 smithi173 ceph-mon[35306]: pgmap v220: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:53:00.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:00 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3323358999' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:53:00.799 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:53:01.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.250+0000 7fa60f9a0640 1 -- 172.21.15.164:0/568161277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa608103660 msgr2=0x7fa60810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:01.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.250+0000 7fa60f9a0640 1 --2- 172.21.15.164:0/568161277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa608103660 0x7fa60810bb50 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8009900 tx=0x7fa5f802a480 comp rx=0 tx=0).stop 2024-09-19T08:53:01.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.250+0000 7fa60f9a0640 1 -- 172.21.15.164:0/568161277 shutdown_connections 2024-09-19T08:53:01.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.250+0000 7fa60f9a0640 1 --2- 172.21.15.164:0/568161277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa60810c170 0x7fa60810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:01.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.250+0000 7fa60f9a0640 1 --2- 172.21.15.164:0/568161277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa608103660 0x7fa60810bb50 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:01.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.250+0000 7fa60f9a0640 1 --2- 172.21.15.164:0/568161277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa608102d40 0x7fa608103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:01.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.250+0000 7fa60f9a0640 1 -- 172.21.15.164:0/568161277 >> 172.21.15.164:0/568161277 conn(0x7fa6080fc7a0 msgr2=0x7fa6080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:01.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.250+0000 7fa60f9a0640 1 -- 172.21.15.164:0/568161277 shutdown_connections 2024-09-19T08:53:01.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.250+0000 7fa60f9a0640 1 -- 172.21.15.164:0/568161277 wait complete. 2024-09-19T08:53:01.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.250+0000 7fa60f9a0640 1 Processor -- start 2024-09-19T08:53:01.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.250+0000 7fa60f9a0640 1 -- start start 2024-09-19T08:53:01.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.250+0000 7fa60f9a0640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa608102d40 0x7fa608198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:01.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.250+0000 7fa60f9a0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa608103660 0x7fa608198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:01.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.250+0000 7fa60f9a0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa60810c170 0x7fa60819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:01.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.250+0000 7fa60f9a0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa608111f70 con 0x7fa608103660 2024-09-19T08:53:01.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.250+0000 7fa60f9a0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa608111df0 con 0x7fa60810c170 2024-09-19T08:53:01.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.250+0000 7fa60f9a0640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa6081120f0 con 0x7fa608102d40 2024-09-19T08:53:01.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.251+0000 7fa60cf3b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa608103660 0x7fa608198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:01.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.251+0000 7fa60cf3b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa608103660 0x7fa608198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47328/0 (socket says 172.21.15.164:47328) 2024-09-19T08:53:01.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.251+0000 7fa60cf3b640 1 -- 172.21.15.164:0/1024541398 learned_addr learned my addr 172.21.15.164:0/1024541398 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:01.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.251+0000 7fa60d73c640 1 --2- 172.21.15.164:0/1024541398 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa608102d40 0x7fa608198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:01.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.251+0000 7fa60df3d640 1 --2- 172.21.15.164:0/1024541398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa60810c170 0x7fa60819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:01.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.251+0000 7fa60cf3b640 1 -- 172.21.15.164:0/1024541398 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa608102d40 msgr2=0x7fa608198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:01.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.251+0000 7fa60cf3b640 1 --2- 172.21.15.164:0/1024541398 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa608102d40 0x7fa608198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:01.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.251+0000 7fa60cf3b640 1 -- 172.21.15.164:0/1024541398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa60810c170 msgr2=0x7fa60819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:01.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.251+0000 7fa60cf3b640 1 --2- 172.21.15.164:0/1024541398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa60810c170 0x7fa60819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:01.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.251+0000 7fa60cf3b640 1 -- 172.21.15.164:0/1024541398 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa60819d3c0 con 0x7fa608103660 2024-09-19T08:53:01.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.251+0000 7fa60df3d640 1 --2- 172.21.15.164:0/1024541398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa60810c170 0x7fa60819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:53:01.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.251+0000 7fa60cf3b640 1 --2- 172.21.15.164:0/1024541398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa608103660 0x7fa608198950 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8005ab0 tx=0x7fa5f80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:01.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.251+0000 7fa5fe7fc640 1 -- 172.21.15.164:0/1024541398 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f803f030 con 0x7fa608103660 2024-09-19T08:53:01.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.251+0000 7fa5fe7fc640 1 -- 172.21.15.164:0/1024541398 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5f8003ce0 con 0x7fa608103660 2024-09-19T08:53:01.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.251+0000 7fa60f9a0640 1 -- 172.21.15.164:0/1024541398 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa60819d650 con 0x7fa608103660 2024-09-19T08:53:01.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.251+0000 7fa5fe7fc640 1 -- 172.21.15.164:0/1024541398 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8035070 con 0x7fa608103660 2024-09-19T08:53:01.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.251+0000 7fa60f9a0640 1 -- 172.21.15.164:0/1024541398 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6081a4f90 con 0x7fa608103660 2024-09-19T08:53:01.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.253+0000 7fa5fe7fc640 1 -- 172.21.15.164:0/1024541398 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fa5f8042030 con 0x7fa608103660 2024-09-19T08:53:01.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.254+0000 7fa611d79740 1 Processor -- start 2024-09-19T08:53:01.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.254+0000 7fa611d79740 1 -- start start 2024-09-19T08:53:01.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.254+0000 7fa611d79740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562387244b40 0x562387243090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:01.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.254+0000 7fa611d79740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623872435d0 0x562387243a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:01.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.254+0000 7fa611d79740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56238724e370 0x562387250760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:01.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.255+0000 7fa5fe7fc640 1 --2- 172.21.15.164:0/1024541398 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa5f40a33b0 0x7fa5f40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:01.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.255+0000 7fa611d79740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5623871a92f0 con 0x56238724e370 2024-09-19T08:53:01.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.255+0000 7fa611d79740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562387150cf0 con 0x5623872435d0 2024-09-19T08:53:01.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.255+0000 7fa611d79740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5623871a52b0 con 0x562387244b40 2024-09-19T08:53:01.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.255+0000 7fa60cf3b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56238724e370 0x562387250760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:01.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.255+0000 7fa60df3d640 1 --2- 172.21.15.164:0/1024541398 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa5f40a33b0 0x7fa5f40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:01.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.255+0000 7fa60df3d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562387244b40 0x562387243090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:01.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.255+0000 7fa5fe7fc640 1 -- 172.21.15.164:0/1024541398 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7fa5f80f0270 con 0x7fa608103660 2024-09-19T08:53:01.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.255+0000 7fa60df3d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562387244b40 0x562387243090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:42578/0 (socket says 172.21.15.164:42578) 2024-09-19T08:53:01.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.255+0000 7fa60df3d640 1 -- 172.21.15.164:0/1956583534 learned_addr learned my addr 172.21.15.164:0/1956583534 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:01.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.255+0000 7fa60d73c640 1 --2- 172.21.15.164:0/1956583534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623872435d0 0x562387243a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:01.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.255+0000 7fa60cf3b640 1 -- 172.21.15.164:0/1956583534 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562387244b40 msgr2=0x562387243090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:01.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.255+0000 7fa60cf3b640 1 --2- 172.21.15.164:0/1956583534 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562387244b40 0x562387243090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:01.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.255+0000 7fa60cf3b640 1 -- 172.21.15.164:0/1956583534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623872435d0 msgr2=0x562387243a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:01.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.255+0000 7fa60cf3b640 1 --2- 172.21.15.164:0/1956583534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623872435d0 0x562387243a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:01.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.255+0000 7fa60cf3b640 1 -- 172.21.15.164:0/1956583534 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623871353a0 con 0x56238724e370 2024-09-19T08:53:01.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.255+0000 7fa60df3d640 1 --2- 172.21.15.164:0/1024541398 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa5f40a33b0 0x7fa5f40a5870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa60000afd0 tx=0x7fa60000acc0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:01.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.255+0000 7fa60cf3b640 1 --2- 172.21.15.164:0/1956583534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56238724e370 0x562387250760 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80ed9a0 tx=0x7fa5f8087310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a0467dce689c59 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:01.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.255+0000 7fa5f37fe640 1 -- 172.21.15.164:0/1956583534 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f80dc070 con 0x56238724e370 2024-09-19T08:53:01.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.256+0000 7fa5f37fe640 1 -- 172.21.15.164:0/1956583534 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5f80bcc30 con 0x56238724e370 2024-09-19T08:53:01.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.256+0000 7fa5f37fe640 1 -- 172.21.15.164:0/1956583534 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8049040 con 0x56238724e370 2024-09-19T08:53:01.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.256+0000 7fa611d79740 1 -- 172.21.15.164:0/1956583534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56238724e370 msgr2=0x562387250760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:01.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.256+0000 7fa611d79740 1 --2- 172.21.15.164:0/1956583534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56238724e370 0x562387250760 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80ed9a0 tx=0x7fa5f8087310 comp rx=0 tx=0).stop 2024-09-19T08:53:01.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.256+0000 7fa611d79740 1 -- 172.21.15.164:0/1956583534 shutdown_connections 2024-09-19T08:53:01.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.256+0000 7fa611d79740 1 --2- 172.21.15.164:0/1956583534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56238724e370 0x562387250760 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:01.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.256+0000 7fa611d79740 1 --2- 172.21.15.164:0/1956583534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623872435d0 0x562387243a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:01.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.256+0000 7fa611d79740 1 --2- 172.21.15.164:0/1956583534 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562387244b40 0x562387243090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:01.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.256+0000 7fa611d79740 1 -- 172.21.15.164:0/1956583534 >> 172.21.15.164:0/1956583534 conn(0x56238720c310 msgr2=0x56238723fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:01.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.256+0000 7fa611d79740 1 -- 172.21.15.164:0/1956583534 shutdown_connections 2024-09-19T08:53:01.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.256+0000 7fa611d79740 1 -- 172.21.15.164:0/1956583534 wait complete. 2024-09-19T08:53:01.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.257+0000 7fa611d79740 1 -- start start 2024-09-19T08:53:01.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.257+0000 7fa611d79740 1 Processor -- start 2024-09-19T08:53:01.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.257+0000 7fa611d79740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623872435d0 0x56238724ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:01.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.257+0000 7fa611d79740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562387244b40 0x56238724b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:01.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.257+0000 7fa611d79740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56238724e370 0x56238724b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:01.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.257+0000 7fa611d79740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562387250e30 con 0x562387244b40 2024-09-19T08:53:01.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.257+0000 7fa611d79740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562387164060 con 0x5623872435d0 2024-09-19T08:53:01.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.257+0000 7fa611d79740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562387250fb0 con 0x56238724e370 2024-09-19T08:53:01.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.257+0000 7fa611d79740 10 client.0 _ll_register_callbacks cb 0x562386eb78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:01.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.257+0000 7fa60cf3b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562387244b40 0x56238724b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:01.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.257+0000 7fa60cf3b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562387244b40 0x56238724b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47354/0 (socket says 172.21.15.164:47354) 2024-09-19T08:53:01.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.257+0000 7fa60cf3b640 1 -- 172.21.15.164:0/211218132 learned_addr learned my addr 172.21.15.164:0/211218132 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:01.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.257+0000 7fa60d73c640 1 --2- 172.21.15.164:0/211218132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623872435d0 0x56238724ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:01.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.258+0000 7fa60df3d640 1 --2- 172.21.15.164:0/211218132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56238724e370 0x56238724b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:01.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.258+0000 7fa60cf3b640 1 -- 172.21.15.164:0/211218132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56238724e370 msgr2=0x56238724b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:01.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.258+0000 7fa60cf3b640 1 --2- 172.21.15.164:0/211218132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56238724e370 0x56238724b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:01.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.258+0000 7fa60cf3b640 1 -- 172.21.15.164:0/211218132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623872435d0 msgr2=0x56238724ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:01.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.258+0000 7fa60cf3b640 1 --2- 172.21.15.164:0/211218132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623872435d0 0x56238724ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:01.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.258+0000 7fa60cf3b640 1 -- 172.21.15.164:0/211218132 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562386f1a3c0 con 0x562387244b40 2024-09-19T08:53:01.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.258+0000 7fa60df3d640 1 --2- 172.21.15.164:0/211218132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56238724e370 0x56238724b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:53:01.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.258+0000 7fa60cf3b640 1 --2- 172.21.15.164:0/211218132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562387244b40 0x56238724b070 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80bc230 tx=0x7fa5f803b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83ceb0e6de82e75 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:01.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.258+0000 7fa5f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:53:01.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.258+0000 7fa5f0ff9640 1 -- 172.21.15.164:0/211218132 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8049070 con 0x562387244b40 2024-09-19T08:53:01.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.258+0000 7fa5f0ff9640 1 -- 172.21.15.164:0/211218132 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7fa5f80b8ba0 con 0x562387244b40 2024-09-19T08:53:01.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.258+0000 7fa611d79740 1 -- 172.21.15.164:0/211218132 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5f80bcc30 con 0x562387244b40 2024-09-19T08:53:01.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.259+0000 7fa5f0ff9640 1 -- 172.21.15.164:0/211218132 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa5f80dc050 con 0x562387244b40 2024-09-19T08:53:01.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.259+0000 7fa611d79740 10 client.16911 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:01.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.259+0000 7fa611d79740 1 -- 172.21.15.164:0/211218132 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562387250e30 con 0x562387244b40 2024-09-19T08:53:01.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.259+0000 7fa5f0ff9640 1 -- 172.21.15.164:0/211218132 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5f80bcc30 con 0x562387244b40 2024-09-19T08:53:01.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.259+0000 7fa5f0ff9640 1 -- 172.21.15.164:0/211218132 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fa5f80edc20 con 0x562387244b40 2024-09-19T08:53:01.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.259+0000 7fa611d79740 10 client.16911 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:01.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.259+0000 7fa611d79740 10 client.16911 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-19T08:53:01.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.259+0000 7fa611d79740 20 client.16911 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:01.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.259+0000 7fa611d79740 1 --2- 172.21.15.164:0/211218132 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x56238725db10 0x56238725ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:01.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.259+0000 7fa611d79740 4 client.16911 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:53:01.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.259+0000 7fa611d79740 1 -- 172.21.15.164:0/211218132 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623871b6860 con 0x56238725db10 2024-09-19T08:53:01.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.259+0000 7fa60d73c640 1 --2- 172.21.15.164:0/211218132 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x56238725db10 0x56238725ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:01.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.259+0000 7fa5f0ff9640 10 client.16911 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:53:01.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.259+0000 7fa60d73c640 1 --2- 172.21.15.164:0/211218132 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x56238725db10 0x56238725ff30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=34bf4b3222c70bdb server_cookie=ca50df095d85b543 in_seq=0 out_seq=0 2024-09-19T08:53:01.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.260+0000 7fa5f0ff9640 1 -- 172.21.15.164:0/211218132 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5623871b6860 con 0x56238725db10 2024-09-19T08:53:01.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.260+0000 7fa5f0ff9640 10 client.16911 handle_command_reply: tid=0 2024-09-19T08:53:01.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.260+0000 7fa611d79740 1 client.16911 shutdown 2024-09-19T08:53:01.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.260+0000 7fa611d79740 1 -- 172.21.15.164:0/211218132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562387244b40 msgr2=0x56238724b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:01.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.260+0000 7fa611d79740 1 --2- 172.21.15.164:0/211218132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562387244b40 0x56238724b070 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80bc230 tx=0x7fa5f803b690 comp rx=0 tx=0).stop 2024-09-19T08:53:01.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.260+0000 7fa611d79740 1 -- 172.21.15.164:0/211218132 shutdown_connections 2024-09-19T08:53:01.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.260+0000 7fa611d79740 1 --2- 172.21.15.164:0/211218132 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x56238725db10 0x56238725ff30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:01.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.260+0000 7fa611d79740 1 --2- 172.21.15.164:0/211218132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56238724e370 0x56238724b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:01.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.260+0000 7fa611d79740 1 --2- 172.21.15.164:0/211218132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562387244b40 0x56238724b070 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:01.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.260+0000 7fa5f0ff9640 0 client.16911 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:53:01.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.260+0000 7fa611d79740 1 --2- 172.21.15.164:0/211218132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623872435d0 0x56238724ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:01.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.260+0000 7fa611d79740 1 -- 172.21.15.164:0/211218132 >> 172.21.15.164:0/211218132 conn(0x56238720c310 msgr2=0x562387245990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:01.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.260+0000 7fa611d79740 1 -- 172.21.15.164:0/211218132 shutdown_connections 2024-09-19T08:53:01.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.260+0000 7fa611d79740 1 -- 172.21.15.164:0/211218132 wait complete. 2024-09-19T08:53:01.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.260+0000 7fa611d79740 20 client.16911 trim_cache size 0 max 16384 2024-09-19T08:53:01.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.273+0000 7fa611d79740 1 Processor -- start 2024-09-19T08:53:01.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.273+0000 7fa611d79740 1 -- start start 2024-09-19T08:53:01.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.274+0000 7fa611d79740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623872435d0 0x56238723fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:01.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.274+0000 7fa611d79740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562387244b40 0x5623872565e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:01.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.274+0000 7fa611d79740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56238724e370 0x562387256b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:01.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.274+0000 7fa611d79740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa6081ad790 con 0x5623872435d0 2024-09-19T08:53:01.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.274+0000 7fa611d79740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa6081af2d0 con 0x56238724e370 2024-09-19T08:53:01.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.274+0000 7fa611d79740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa6081ade60 con 0x562387244b40 2024-09-19T08:53:01.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.274+0000 7fa60d73c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623872435d0 0x56238723fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:01.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.274+0000 7fa60d73c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623872435d0 0x56238723fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47370/0 (socket says 172.21.15.164:47370) 2024-09-19T08:53:01.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.274+0000 7fa60d73c640 1 -- 172.21.15.164:0/2468878475 learned_addr learned my addr 172.21.15.164:0/2468878475 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:01.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.274+0000 7fa60df3d640 1 --2- 172.21.15.164:0/2468878475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56238724e370 0x562387256b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:01.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.274+0000 7fa60d73c640 1 -- 172.21.15.164:0/2468878475 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562387244b40 msgr2=0x5623872565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:01.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.274+0000 7fa60d73c640 1 --2- 172.21.15.164:0/2468878475 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562387244b40 0x5623872565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:01.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.274+0000 7fa60d73c640 1 -- 172.21.15.164:0/2468878475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56238724e370 msgr2=0x562387256b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:01.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.274+0000 7fa60d73c640 1 --2- 172.21.15.164:0/2468878475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56238724e370 0x562387256b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:01.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.274+0000 7fa60d73c640 1 -- 172.21.15.164:0/2468878475 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562387246120 con 0x5623872435d0 2024-09-19T08:53:01.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.274+0000 7fa60d73c640 1 --2- 172.21.15.164:0/2468878475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623872435d0 0x56238723fdf0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x562387260760 tx=0x7fa60400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3ad9671a449825 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:01.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.274+0000 7fa5f2ffd640 1 -- 172.21.15.164:0/2468878475 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa604003d20 con 0x5623872435d0 2024-09-19T08:53:01.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.274+0000 7fa5f2ffd640 1 -- 172.21.15.164:0/2468878475 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa604005500 con 0x5623872435d0 2024-09-19T08:53:01.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.274+0000 7fa5f2ffd640 1 -- 172.21.15.164:0/2468878475 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa604010430 con 0x5623872435d0 2024-09-19T08:53:01.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.275+0000 7fa611d79740 1 -- 172.21.15.164:0/2468878475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623872435d0 msgr2=0x56238723fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:01.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.275+0000 7fa611d79740 1 --2- 172.21.15.164:0/2468878475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623872435d0 0x56238723fdf0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x562387260760 tx=0x7fa60400cc40 comp rx=0 tx=0).stop 2024-09-19T08:53:01.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.275+0000 7fa611d79740 1 -- 172.21.15.164:0/2468878475 shutdown_connections 2024-09-19T08:53:01.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.275+0000 7fa611d79740 1 --2- 172.21.15.164:0/2468878475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56238724e370 0x562387256b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:01.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.275+0000 7fa611d79740 1 --2- 172.21.15.164:0/2468878475 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562387244b40 0x5623872565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:01.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.275+0000 7fa611d79740 1 --2- 172.21.15.164:0/2468878475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623872435d0 0x56238723fdf0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:01.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.275+0000 7fa611d79740 1 -- 172.21.15.164:0/2468878475 >> 172.21.15.164:0/2468878475 conn(0x56238720c310 msgr2=0x56238723f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:01.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.275+0000 7fa611d79740 1 -- 172.21.15.164:0/2468878475 shutdown_connections 2024-09-19T08:53:01.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.275+0000 7fa611d79740 1 -- 172.21.15.164:0/2468878475 wait complete. 2024-09-19T08:53:01.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.275+0000 7fa611d79740 1 -- start start 2024-09-19T08:53:01.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.275+0000 7fa611d79740 1 Processor -- start 2024-09-19T08:53:01.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.276+0000 7fa611d79740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623872435d0 0x562387265440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:01.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.276+0000 7fa611d79740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562387244b40 0x562387265980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:01.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.276+0000 7fa611d79740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56238724e370 0x562387265ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:01.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.276+0000 7fa611d79740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa6081aec00 con 0x56238724e370 2024-09-19T08:53:01.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.276+0000 7fa611d79740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa6081ae530 con 0x5623872435d0 2024-09-19T08:53:01.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.276+0000 7fa611d79740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa6081ab790 con 0x562387244b40 2024-09-19T08:53:01.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.276+0000 7fa611d79740 10 client.0 _ll_register_callbacks cb 0x7fa60400c350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:01.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.276+0000 7fa60df3d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56238724e370 0x562387265ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:01.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.276+0000 7fa60df3d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56238724e370 0x562387265ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47378/0 (socket says 172.21.15.164:47378) 2024-09-19T08:53:01.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.276+0000 7fa60df3d640 1 -- 172.21.15.164:0/653794787 learned_addr learned my addr 172.21.15.164:0/653794787 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:01.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.276+0000 7fa60d73c640 1 --2- 172.21.15.164:0/653794787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623872435d0 0x562387265440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:01.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.276+0000 7fa60cf3b640 1 --2- 172.21.15.164:0/653794787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562387244b40 0x562387265980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:01.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.276+0000 7fa60df3d640 1 -- 172.21.15.164:0/653794787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562387244b40 msgr2=0x562387265980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:01.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.276+0000 7fa60df3d640 1 --2- 172.21.15.164:0/653794787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562387244b40 0x562387265980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:01.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.276+0000 7fa60df3d640 1 -- 172.21.15.164:0/653794787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623872435d0 msgr2=0x562387265440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:01.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.276+0000 7fa60df3d640 1 --2- 172.21.15.164:0/653794787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623872435d0 0x562387265440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:01.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.276+0000 7fa60df3d640 1 -- 172.21.15.164:0/653794787 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623872cb7c0 con 0x56238724e370 2024-09-19T08:53:01.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.276+0000 7fa60df3d640 1 --2- 172.21.15.164:0/653794787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56238724e370 0x562387265ec0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa60000fc80 tx=0x7fa600005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63796c5d638e9829 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:01.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.276+0000 7fa5f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:53:01.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.277+0000 7fa5f0ff9640 1 -- 172.21.15.164:0/653794787 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60002e060 con 0x56238724e370 2024-09-19T08:53:01.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.277+0000 7fa611d79740 1 -- 172.21.15.164:0/653794787 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa604005500 con 0x56238724e370 2024-09-19T08:53:01.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.277+0000 7fa5f0ff9640 1 -- 172.21.15.164:0/653794787 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7fa600020bc0 con 0x56238724e370 2024-09-19T08:53:01.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.277+0000 7fa5f0ff9640 1 -- 172.21.15.164:0/653794787 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa60000fe30 con 0x56238724e370 2024-09-19T08:53:01.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.277+0000 7fa611d79740 10 client.16917 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:01.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.277+0000 7fa611d79740 1 -- 172.21.15.164:0/653794787 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5623872cc9c0 con 0x56238724e370 2024-09-19T08:53:01.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.277+0000 7fa5f0ff9640 1 -- 172.21.15.164:0/653794787 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa604005500 con 0x56238724e370 2024-09-19T08:53:01.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.277+0000 7fa5f0ff9640 1 -- 172.21.15.164:0/653794787 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fa600028070 con 0x56238724e370 2024-09-19T08:53:01.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.277+0000 7fa611d79740 10 client.16917 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:01.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.277+0000 7fa611d79740 10 client.16917 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-19T08:53:01.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.277+0000 7fa611d79740 20 client.16917 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:01.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.277+0000 7fa611d79740 1 --2- 172.21.15.164:0/653794787 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5623872cd0e0 0x5623872cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:01.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.277+0000 7fa611d79740 4 client.16917 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:53:01.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.277+0000 7fa611d79740 1 -- 172.21.15.164:0/653794787 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5623871b0d70 con 0x5623872cd0e0 2024-09-19T08:53:01.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.277+0000 7fa60d73c640 1 --2- 172.21.15.164:0/653794787 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5623872cd0e0 0x5623872cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:01.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.278+0000 7fa60d73c640 1 --2- 172.21.15.164:0/653794787 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5623872cd0e0 0x5623872cf4d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fd8bda5bdb2e08ce server_cookie=10a749686aec0471 in_seq=0 out_seq=0 2024-09-19T08:53:01.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.278+0000 7fa5f0ff9640 10 client.16917 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:53:01.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.278+0000 7fa5f0ff9640 1 -- 172.21.15.164:0/653794787 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5623871b0d70 con 0x5623872cd0e0 2024-09-19T08:53:01.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.278+0000 7fa5f0ff9640 10 client.16917 handle_command_reply: tid=0 2024-09-19T08:53:01.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.278+0000 7fa611d79740 1 client.16917 shutdown 2024-09-19T08:53:01.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.278+0000 7fa611d79740 1 -- 172.21.15.164:0/653794787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56238724e370 msgr2=0x562387265ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:01.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.278+0000 7fa611d79740 1 --2- 172.21.15.164:0/653794787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56238724e370 0x562387265ec0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa60000fc80 tx=0x7fa600005c30 comp rx=0 tx=0).stop 2024-09-19T08:53:01.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.278+0000 7fa611d79740 1 -- 172.21.15.164:0/653794787 shutdown_connections 2024-09-19T08:53:01.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.278+0000 7fa611d79740 1 --2- 172.21.15.164:0/653794787 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5623872cd0e0 0x5623872cf4d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:01.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.278+0000 7fa611d79740 1 --2- 172.21.15.164:0/653794787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56238724e370 0x562387265ec0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:01.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.278+0000 7fa611d79740 1 --2- 172.21.15.164:0/653794787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562387244b40 0x562387265980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:01.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.278+0000 7fa611d79740 1 --2- 172.21.15.164:0/653794787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623872435d0 0x562387265440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:01.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.278+0000 7fa611d79740 1 -- 172.21.15.164:0/653794787 >> 172.21.15.164:0/653794787 conn(0x56238720c310 msgr2=0x56238723e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:01.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.278+0000 7fa5f0ff9640 0 client.16917 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:53:01.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.278+0000 7fa611d79740 1 -- 172.21.15.164:0/653794787 shutdown_connections 2024-09-19T08:53:01.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.278+0000 7fa611d79740 1 -- 172.21.15.164:0/653794787 wait complete. 2024-09-19T08:53:01.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.278+0000 7fa611d79740 20 client.16917 trim_cache size 0 max 16384 2024-09-19T08:53:01.290 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:53:01.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.279+0000 7fa60f9a0640 1 -- 172.21.15.164:0/1024541398 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa5f40a33b0 msgr2=0x7fa5f40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:01.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.279+0000 7fa60f9a0640 1 --2- 172.21.15.164:0/1024541398 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa5f40a33b0 0x7fa5f40a5870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa60000afd0 tx=0x7fa60000acc0 comp rx=0 tx=0).stop 2024-09-19T08:53:01.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.279+0000 7fa60f9a0640 1 -- 172.21.15.164:0/1024541398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa608103660 msgr2=0x7fa608198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:01.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.279+0000 7fa60f9a0640 1 --2- 172.21.15.164:0/1024541398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa608103660 0x7fa608198950 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8005ab0 tx=0x7fa5f80042f0 comp rx=0 tx=0).stop 2024-09-19T08:53:01.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.279+0000 7fa60f9a0640 1 -- 172.21.15.164:0/1024541398 shutdown_connections 2024-09-19T08:53:01.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.279+0000 7fa60f9a0640 1 --2- 172.21.15.164:0/1024541398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa60810c170 0x7fa60819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:01.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.279+0000 7fa60f9a0640 1 --2- 172.21.15.164:0/1024541398 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa5f40a33b0 0x7fa5f40a5870 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:01.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.279+0000 7fa60f9a0640 1 --2- 172.21.15.164:0/1024541398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa608103660 0x7fa608198950 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:01.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.279+0000 7fa60f9a0640 1 --2- 172.21.15.164:0/1024541398 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa608102d40 0x7fa608198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:01.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.279+0000 7fa60f9a0640 1 -- 172.21.15.164:0/1024541398 >> 172.21.15.164:0/1024541398 conn(0x7fa6080fc7a0 msgr2=0x7fa6080fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:01.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.279+0000 7fa60f9a0640 1 -- 172.21.15.164:0/1024541398 shutdown_connections 2024-09-19T08:53:01.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:01.279+0000 7fa60f9a0640 1 -- 172.21.15.164:0/1024541398 wait complete. 2024-09-19T08:53:02.097 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T08:53:02.349 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:53:02.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:02 smithi164 ceph-mon[30928]: pgmap v221: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:53:02.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:02 smithi170 ceph-mon[35072]: pgmap v221: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:53:02.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:02 smithi173 ceph-mon[35306]: pgmap v221: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:53:03.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.893+0000 7fab25d33640 1 -- 172.21.15.164:0/4096489654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab20101de0 msgr2=0x7fab201021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:03.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.893+0000 7fab25d33640 1 --2- 172.21.15.164:0/4096489654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab20101de0 0x7fab201021c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fab10009900 tx=0x7fab1002a410 comp rx=0 tx=0).stop 2024-09-19T08:53:03.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.893+0000 7fab25d33640 1 -- 172.21.15.164:0/4096489654 shutdown_connections 2024-09-19T08:53:03.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.893+0000 7fab25d33640 1 --2- 172.21.15.164:0/4096489654 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab20106f10 0x7fab2010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:03.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.893+0000 7fab25d33640 1 --2- 172.21.15.164:0/4096489654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab20102790 0x7fab201067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:03.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.893+0000 7fab25d33640 1 --2- 172.21.15.164:0/4096489654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab20101de0 0x7fab201021c0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:03.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.893+0000 7fab25d33640 1 -- 172.21.15.164:0/4096489654 >> 172.21.15.164:0/4096489654 conn(0x7fab200fd8c0 msgr2=0x7fab200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:03.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.893+0000 7fab25d33640 1 -- 172.21.15.164:0/4096489654 shutdown_connections 2024-09-19T08:53:03.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.893+0000 7fab25d33640 1 -- 172.21.15.164:0/4096489654 wait complete. 2024-09-19T08:53:03.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.893+0000 7fab25d33640 1 Processor -- start 2024-09-19T08:53:03.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.894+0000 7fab25d33640 1 -- start start 2024-09-19T08:53:03.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.894+0000 7fab25d33640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab20101de0 0x7fab201983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:03.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.894+0000 7fab25d33640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab20102790 0x7fab20198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:03.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.894+0000 7fab25d33640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab20106f10 0x7fab2019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:03.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.894+0000 7fab25d33640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fab20111f70 con 0x7fab20101de0 2024-09-19T08:53:03.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.894+0000 7fab25d33640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fab20111df0 con 0x7fab20102790 2024-09-19T08:53:03.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.894+0000 7fab25d33640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fab201120f0 con 0x7fab20106f10 2024-09-19T08:53:03.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.894+0000 7fab1f7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab20101de0 0x7fab201983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:03.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.894+0000 7fab1f7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab20101de0 0x7fab201983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47388/0 (socket says 172.21.15.164:47388) 2024-09-19T08:53:03.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.894+0000 7fab1f7fe640 1 -- 172.21.15.164:0/742295454 learned_addr learned my addr 172.21.15.164:0/742295454 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:03.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.894+0000 7fab1effd640 1 --2- 172.21.15.164:0/742295454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab20102790 0x7fab20198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:03.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.894+0000 7fab1ffff640 1 --2- 172.21.15.164:0/742295454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab20106f10 0x7fab2019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:03.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.894+0000 7fab1f7fe640 1 -- 172.21.15.164:0/742295454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab20106f10 msgr2=0x7fab2019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:03.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.894+0000 7fab1f7fe640 1 --2- 172.21.15.164:0/742295454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab20106f10 0x7fab2019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:03.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.894+0000 7fab1f7fe640 1 -- 172.21.15.164:0/742295454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab20102790 msgr2=0x7fab20198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:03.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.894+0000 7fab1f7fe640 1 --2- 172.21.15.164:0/742295454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab20102790 0x7fab20198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:03.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.894+0000 7fab1f7fe640 1 -- 172.21.15.164:0/742295454 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab2019d380 con 0x7fab20101de0 2024-09-19T08:53:03.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.894+0000 7fab1ffff640 1 --2- 172.21.15.164:0/742295454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab20106f10 0x7fab2019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:53:03.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.894+0000 7fab1effd640 1 --2- 172.21.15.164:0/742295454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab20102790 0x7fab20198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:53:03.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.894+0000 7fab1f7fe640 1 --2- 172.21.15.164:0/742295454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab20101de0 0x7fab201983d0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fab10006010 tx=0x7fab100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:03.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.895+0000 7fab1cff9640 1 -- 172.21.15.164:0/742295454 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1003f030 con 0x7fab20101de0 2024-09-19T08:53:03.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.895+0000 7fab1cff9640 1 -- 172.21.15.164:0/742295454 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab10003ce0 con 0x7fab20101de0 2024-09-19T08:53:03.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.895+0000 7fab25d33640 1 -- 172.21.15.164:0/742295454 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab2019d610 con 0x7fab20101de0 2024-09-19T08:53:03.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.895+0000 7fab1cff9640 1 -- 172.21.15.164:0/742295454 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab10035070 con 0x7fab20101de0 2024-09-19T08:53:03.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.895+0000 7fab25d33640 1 -- 172.21.15.164:0/742295454 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab201a4f50 con 0x7fab20101de0 2024-09-19T08:53:03.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.897+0000 7fab1cff9640 1 -- 172.21.15.164:0/742295454 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fab10042030 con 0x7fab20101de0 2024-09-19T08:53:03.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.897+0000 7fab2810c740 1 Processor -- start 2024-09-19T08:53:03.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.897+0000 7fab2810c740 1 -- start start 2024-09-19T08:53:03.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.897+0000 7fab1cff9640 1 --2- 172.21.15.164:0/742295454 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fab080a33b0 0x7fab080a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:03.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.898+0000 7fab2810c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4bdafdb40 0x55d4bdafc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:03.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.898+0000 7fab2810c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4bdafc5d0 0x55d4bdafca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:03.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.898+0000 7fab2810c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4bdb07370 0x55d4bdb09760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:03.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.898+0000 7fab2810c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d4bda622f0 con 0x55d4bdafc5d0 2024-09-19T08:53:03.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.898+0000 7fab2810c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d4bda09cf0 con 0x55d4bdafdb40 2024-09-19T08:53:03.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.898+0000 7fab2810c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d4bda5e2b0 con 0x55d4bdb07370 2024-09-19T08:53:03.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.898+0000 7fab1ffff640 1 --2- 172.21.15.164:0/742295454 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fab080a33b0 0x7fab080a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:03.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.898+0000 7fab1cff9640 1 -- 172.21.15.164:0/742295454 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7fab100f0300 con 0x7fab20101de0 2024-09-19T08:53:03.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.898+0000 7fab1effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4bdafc5d0 0x55d4bdafca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:03.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.898+0000 7fab1effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4bdafc5d0 0x55d4bdafca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47404/0 (socket says 172.21.15.164:47404) 2024-09-19T08:53:03.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.898+0000 7fab1effd640 1 -- 172.21.15.164:0/4270626839 learned_addr learned my addr 172.21.15.164:0/4270626839 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:03.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.898+0000 7fab1f7fe640 1 --2- 172.21.15.164:0/4270626839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4bdafdb40 0x55d4bdafc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:03.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.898+0000 7fab1ffff640 1 --2- 172.21.15.164:0/4270626839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4bdb07370 0x55d4bdb09760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:03.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.898+0000 7fab1effd640 1 -- 172.21.15.164:0/4270626839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4bdb07370 msgr2=0x55d4bdb09760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:03.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.898+0000 7fab1effd640 1 --2- 172.21.15.164:0/4270626839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4bdb07370 0x55d4bdb09760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:03.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.898+0000 7fab1effd640 1 -- 172.21.15.164:0/4270626839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4bdafdb40 msgr2=0x55d4bdafc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:03.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.898+0000 7fab1effd640 1 --2- 172.21.15.164:0/4270626839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4bdafdb40 0x55d4bdafc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:03.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.898+0000 7fab1effd640 1 -- 172.21.15.164:0/4270626839 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4bd9ee3a0 con 0x55d4bdafc5d0 2024-09-19T08:53:03.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.898+0000 7fab1ffff640 1 --2- 172.21.15.164:0/742295454 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fab080a33b0 0x7fab080a5870 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fab18005cc0 tx=0x7fab1800d710 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:03.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.898+0000 7fab1effd640 1 --2- 172.21.15.164:0/4270626839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4bdafc5d0 0x55d4bdafca10 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fab1400bd30 tx=0x7fab1400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d51123195bf4897 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:03.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.898+0000 7fab05ffb640 1 -- 172.21.15.164:0/4270626839 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14015070 con 0x55d4bdafc5d0 2024-09-19T08:53:03.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.898+0000 7fab05ffb640 1 -- 172.21.15.164:0/4270626839 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab1400be20 con 0x55d4bdafc5d0 2024-09-19T08:53:03.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.898+0000 7fab05ffb640 1 -- 172.21.15.164:0/4270626839 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab140107f0 con 0x55d4bdafc5d0 2024-09-19T08:53:03.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.899+0000 7fab2810c740 1 -- 172.21.15.164:0/4270626839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4bdafc5d0 msgr2=0x55d4bdafca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:03.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.899+0000 7fab2810c740 1 --2- 172.21.15.164:0/4270626839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4bdafc5d0 0x55d4bdafca10 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fab1400bd30 tx=0x7fab1400f730 comp rx=0 tx=0).stop 2024-09-19T08:53:03.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.899+0000 7fab2810c740 1 -- 172.21.15.164:0/4270626839 shutdown_connections 2024-09-19T08:53:03.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.899+0000 7fab2810c740 1 --2- 172.21.15.164:0/4270626839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4bdb07370 0x55d4bdb09760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:03.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.899+0000 7fab2810c740 1 --2- 172.21.15.164:0/4270626839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4bdafc5d0 0x55d4bdafca10 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:03.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.899+0000 7fab2810c740 1 --2- 172.21.15.164:0/4270626839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4bdafdb40 0x55d4bdafc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:03.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.899+0000 7fab2810c740 1 -- 172.21.15.164:0/4270626839 >> 172.21.15.164:0/4270626839 conn(0x55d4bdac5310 msgr2=0x55d4bdaf8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:03.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.899+0000 7fab2810c740 1 -- 172.21.15.164:0/4270626839 shutdown_connections 2024-09-19T08:53:03.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.899+0000 7fab2810c740 1 -- 172.21.15.164:0/4270626839 wait complete. 2024-09-19T08:53:03.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.900+0000 7fab2810c740 1 -- start start 2024-09-19T08:53:03.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.900+0000 7fab2810c740 1 Processor -- start 2024-09-19T08:53:03.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.900+0000 7fab2810c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4bdafc5d0 0x55d4bdb06d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:03.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.900+0000 7fab2810c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4bdafdb40 0x55d4bdb03fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:03.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.900+0000 7fab2810c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4bdb07370 0x55d4bdb04560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:03.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.900+0000 7fab2810c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d4bdb0e480 con 0x55d4bdafdb40 2024-09-19T08:53:03.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.900+0000 7fab2810c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d4bda1d060 con 0x55d4bdb07370 2024-09-19T08:53:03.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.900+0000 7fab2810c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d4bdb0e600 con 0x55d4bdafc5d0 2024-09-19T08:53:03.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.900+0000 7fab2810c740 10 client.0 _ll_register_callbacks cb 0x55d4bd7708c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:03.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.900+0000 7fab1effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4bdafdb40 0x55d4bdb03fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:03.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.900+0000 7fab1effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4bdafdb40 0x55d4bdb03fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47420/0 (socket says 172.21.15.164:47420) 2024-09-19T08:53:03.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.900+0000 7fab1effd640 1 -- 172.21.15.164:0/3510985461 learned_addr learned my addr 172.21.15.164:0/3510985461 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:03.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.900+0000 7fab1ffff640 1 --2- 172.21.15.164:0/3510985461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4bdb07370 0x55d4bdb04560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:03.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.900+0000 7fab1f7fe640 1 --2- 172.21.15.164:0/3510985461 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4bdafc5d0 0x55d4bdb06d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:03.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.901+0000 7fab1effd640 1 -- 172.21.15.164:0/3510985461 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4bdafc5d0 msgr2=0x55d4bdb06d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:03.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.901+0000 7fab1effd640 1 --2- 172.21.15.164:0/3510985461 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4bdafc5d0 0x55d4bdb06d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:03.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.901+0000 7fab1effd640 1 -- 172.21.15.164:0/3510985461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4bdb07370 msgr2=0x55d4bdb04560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:03.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.901+0000 7fab1effd640 1 --2- 172.21.15.164:0/3510985461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4bdb07370 0x55d4bdb04560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:03.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.901+0000 7fab1effd640 1 -- 172.21.15.164:0/3510985461 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4bd7d33c0 con 0x55d4bdafdb40 2024-09-19T08:53:03.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.901+0000 7fab1effd640 1 --2- 172.21.15.164:0/3510985461 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4bdafdb40 0x55d4bdb03fc0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fab1400fc30 tx=0x7fab14004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d03e368007ea589 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:03.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.901+0000 7faafeffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:53:03.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.901+0000 7faafeffd640 1 -- 172.21.15.164:0/3510985461 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14029060 con 0x55d4bdafdb40 2024-09-19T08:53:03.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.901+0000 7faafeffd640 1 -- 172.21.15.164:0/3510985461 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7fab140035f0 con 0x55d4bdafdb40 2024-09-19T08:53:03.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.901+0000 7fab2810c740 1 -- 172.21.15.164:0/3510985461 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab1400be20 con 0x55d4bdafdb40 2024-09-19T08:53:03.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.901+0000 7faafeffd640 1 -- 172.21.15.164:0/3510985461 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fab14015050 con 0x55d4bdafdb40 2024-09-19T08:53:03.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.901+0000 7fab2810c740 10 client.16929 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:03.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.901+0000 7fab2810c740 1 -- 172.21.15.164:0/3510985461 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d4bdb19f00 con 0x55d4bdafdb40 2024-09-19T08:53:03.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.901+0000 7faafeffd640 1 -- 172.21.15.164:0/3510985461 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab1400be20 con 0x55d4bdafdb40 2024-09-19T08:53:03.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.902+0000 7faafeffd640 1 -- 172.21.15.164:0/3510985461 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fab14010970 con 0x55d4bdafdb40 2024-09-19T08:53:03.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.902+0000 7fab2810c740 10 client.16929 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:03.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.902+0000 7fab2810c740 10 client.16929 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T08:53:03.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.902+0000 7fab2810c740 20 client.16929 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:03.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.902+0000 7fab2810c740 1 --2- 172.21.15.164:0/3510985461 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55d4bdb1b250 0x55d4bdb1d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:03.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.902+0000 7fab2810c740 4 client.16929 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:53:03.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.902+0000 7fab2810c740 1 -- 172.21.15.164:0/3510985461 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4bda6f860 con 0x55d4bdb1b250 2024-09-19T08:53:03.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.902+0000 7fab1f7fe640 1 --2- 172.21.15.164:0/3510985461 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55d4bdb1b250 0x55d4bdb1d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:03.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.902+0000 7faafeffd640 10 client.16929 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:53:03.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.902+0000 7fab1f7fe640 1 --2- 172.21.15.164:0/3510985461 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55d4bdb1b250 0x55d4bdb1d640 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=eed7286d23e80fd server_cookie=35833ccd200d3472 in_seq=0 out_seq=0 2024-09-19T08:53:03.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.903+0000 7faafeffd640 1 -- 172.21.15.164:0/3510985461 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d4bda6f860 con 0x55d4bdb1b250 2024-09-19T08:53:03.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.903+0000 7faafeffd640 10 client.16929 handle_command_reply: tid=0 2024-09-19T08:53:03.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.903+0000 7fab2810c740 1 client.16929 shutdown 2024-09-19T08:53:03.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.903+0000 7fab2810c740 1 -- 172.21.15.164:0/3510985461 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4bdafdb40 msgr2=0x55d4bdb03fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:03.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.903+0000 7fab2810c740 1 --2- 172.21.15.164:0/3510985461 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4bdafdb40 0x55d4bdb03fc0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fab1400fc30 tx=0x7fab14004270 comp rx=0 tx=0).stop 2024-09-19T08:53:03.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.903+0000 7fab2810c740 1 -- 172.21.15.164:0/3510985461 shutdown_connections 2024-09-19T08:53:03.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.903+0000 7fab2810c740 1 --2- 172.21.15.164:0/3510985461 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55d4bdb1b250 0x55d4bdb1d640 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:03.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.903+0000 7fab2810c740 1 --2- 172.21.15.164:0/3510985461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4bdb07370 0x55d4bdb04560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:03.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.903+0000 7fab2810c740 1 --2- 172.21.15.164:0/3510985461 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4bdafdb40 0x55d4bdb03fc0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:03.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.903+0000 7faafeffd640 0 client.16929 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:53:03.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.903+0000 7fab2810c740 1 --2- 172.21.15.164:0/3510985461 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4bdafc5d0 0x55d4bdb06d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:03.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.903+0000 7fab2810c740 1 -- 172.21.15.164:0/3510985461 >> 172.21.15.164:0/3510985461 conn(0x55d4bdac5310 msgr2=0x55d4bdafe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:03.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.903+0000 7fab2810c740 1 -- 172.21.15.164:0/3510985461 shutdown_connections 2024-09-19T08:53:03.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.903+0000 7fab2810c740 1 -- 172.21.15.164:0/3510985461 wait complete. 2024-09-19T08:53:03.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.903+0000 7fab2810c740 20 client.16929 trim_cache size 0 max 16384 2024-09-19T08:53:03.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.916+0000 7fab2810c740 1 Processor -- start 2024-09-19T08:53:03.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.916+0000 7fab2810c740 1 -- start start 2024-09-19T08:53:03.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.916+0000 7fab2810c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4bdafc5d0 0x55d4bdb17750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:03.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.916+0000 7fab2810c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4bdafdb40 0x55d4bdb03880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:03.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.916+0000 7fab2810c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4bdb07370 0x55d4bdaf7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:03.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.916+0000 7fab2810c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fab201ab540 con 0x55d4bdb07370 2024-09-19T08:53:03.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.916+0000 7fab2810c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fab201ad080 con 0x55d4bdafdb40 2024-09-19T08:53:03.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.916+0000 7fab2810c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fab201abc10 con 0x55d4bdafc5d0 2024-09-19T08:53:03.065 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:53:03.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.916+0000 7fab1ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4bdb07370 0x55d4bdaf7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:03.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.917+0000 7fab1ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4bdb07370 0x55d4bdaf7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47426/0 (socket says 172.21.15.164:47426) 2024-09-19T08:53:03.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.917+0000 7fab1ffff640 1 -- 172.21.15.164:0/2560917373 learned_addr learned my addr 172.21.15.164:0/2560917373 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:03.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.917+0000 7fab1f7fe640 1 --2- 172.21.15.164:0/2560917373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4bdafc5d0 0x55d4bdb17750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:03.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.917+0000 7fab1effd640 1 --2- 172.21.15.164:0/2560917373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4bdafdb40 0x55d4bdb03880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:03.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.917+0000 7fab1ffff640 1 -- 172.21.15.164:0/2560917373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4bdafc5d0 msgr2=0x55d4bdb17750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:03.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.917+0000 7fab1ffff640 1 --2- 172.21.15.164:0/2560917373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4bdafc5d0 0x55d4bdb17750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:03.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.917+0000 7fab1ffff640 1 -- 172.21.15.164:0/2560917373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4bdafdb40 msgr2=0x55d4bdb03880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:03.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.917+0000 7fab1ffff640 1 --2- 172.21.15.164:0/2560917373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4bdafdb40 0x55d4bdb03880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:03.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.917+0000 7fab1ffff640 1 -- 172.21.15.164:0/2560917373 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4bdaf9960 con 0x55d4bdb07370 2024-09-19T08:53:03.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.917+0000 7fab1ffff640 1 --2- 172.21.15.164:0/2560917373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4bdb07370 0x55d4bdaf7bf0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fab1801e740 tx=0x7fab18018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c45eae0536714a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:03.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.917+0000 7fab057fa640 1 -- 172.21.15.164:0/2560917373 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab18004b20 con 0x55d4bdb07370 2024-09-19T08:53:03.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.917+0000 7fab057fa640 1 -- 172.21.15.164:0/2560917373 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab18011cf0 con 0x55d4bdb07370 2024-09-19T08:53:03.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.917+0000 7fab057fa640 1 -- 172.21.15.164:0/2560917373 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab180057d0 con 0x55d4bdb07370 2024-09-19T08:53:03.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.918+0000 7fab2810c740 1 -- 172.21.15.164:0/2560917373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4bdb07370 msgr2=0x55d4bdaf7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:03.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.918+0000 7fab2810c740 1 --2- 172.21.15.164:0/2560917373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4bdb07370 0x55d4bdaf7bf0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fab1801e740 tx=0x7fab18018000 comp rx=0 tx=0).stop 2024-09-19T08:53:03.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.918+0000 7fab2810c740 1 -- 172.21.15.164:0/2560917373 shutdown_connections 2024-09-19T08:53:03.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.918+0000 7fab2810c740 1 --2- 172.21.15.164:0/2560917373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4bdb07370 0x55d4bdaf7bf0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:03.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.918+0000 7fab2810c740 1 --2- 172.21.15.164:0/2560917373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4bdafdb40 0x55d4bdb03880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:03.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.918+0000 7fab2810c740 1 --2- 172.21.15.164:0/2560917373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4bdafc5d0 0x55d4bdb17750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:03.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.918+0000 7fab2810c740 1 -- 172.21.15.164:0/2560917373 >> 172.21.15.164:0/2560917373 conn(0x55d4bdac5310 msgr2=0x55d4bdaf74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:03.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.918+0000 7fab2810c740 1 -- 172.21.15.164:0/2560917373 shutdown_connections 2024-09-19T08:53:03.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.918+0000 7fab2810c740 1 -- 172.21.15.164:0/2560917373 wait complete. 2024-09-19T08:53:03.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.918+0000 7fab2810c740 1 -- start start 2024-09-19T08:53:03.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.918+0000 7fab2810c740 1 Processor -- start 2024-09-19T08:53:03.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.919+0000 7fab2810c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4bdafc5d0 0x55d4bdb13ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:03.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.919+0000 7fab2810c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4bdafdb40 0x55d4bdb00bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:03.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.919+0000 7fab2810c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4bdb07370 0x55d4bdb01110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:03.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.919+0000 7fab2810c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fab201ac9b0 con 0x55d4bdafdb40 2024-09-19T08:53:03.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.919+0000 7fab2810c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fab201ac2e0 con 0x55d4bdb07370 2024-09-19T08:53:03.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.919+0000 7fab2810c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fab201a9520 con 0x55d4bdafc5d0 2024-09-19T08:53:03.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.919+0000 7fab2810c740 10 client.0 _ll_register_callbacks cb 0x7fab10005750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:03.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.919+0000 7fab1effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4bdafdb40 0x55d4bdb00bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:03.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.919+0000 7fab1effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4bdafdb40 0x55d4bdb00bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47432/0 (socket says 172.21.15.164:47432) 2024-09-19T08:53:03.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.919+0000 7fab1effd640 1 -- 172.21.15.164:0/872830240 learned_addr learned my addr 172.21.15.164:0/872830240 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:03.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.919+0000 7fab1f7fe640 1 --2- 172.21.15.164:0/872830240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4bdafc5d0 0x55d4bdb13ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:03.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.919+0000 7fab1ffff640 1 --2- 172.21.15.164:0/872830240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4bdb07370 0x55d4bdb01110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:03.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.919+0000 7fab1effd640 1 -- 172.21.15.164:0/872830240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4bdafc5d0 msgr2=0x55d4bdb13ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:03.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.919+0000 7fab1effd640 1 --2- 172.21.15.164:0/872830240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4bdafc5d0 0x55d4bdb13ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:03.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.919+0000 7fab1effd640 1 -- 172.21.15.164:0/872830240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4bdb07370 msgr2=0x55d4bdb01110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:03.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.919+0000 7fab1effd640 1 --2- 172.21.15.164:0/872830240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4bdb07370 0x55d4bdb01110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:03.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.919+0000 7fab1effd640 1 -- 172.21.15.164:0/872830240 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4bdb01800 con 0x55d4bdafdb40 2024-09-19T08:53:03.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.919+0000 7fab1effd640 1 --2- 172.21.15.164:0/872830240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4bdafdb40 0x55d4bdb00bd0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x55d4bdb05600 tx=0x7fab14010f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d50256dfc52527bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:03.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.919+0000 7faafeffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:53:03.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.919+0000 7faafeffd640 1 -- 172.21.15.164:0/872830240 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14029070 con 0x55d4bdafdb40 2024-09-19T08:53:03.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.919+0000 7faafeffd640 1 -- 172.21.15.164:0/872830240 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7fab14011540 con 0x55d4bdafdb40 2024-09-19T08:53:03.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.919+0000 7fab2810c740 1 -- 172.21.15.164:0/872830240 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab18011cf0 con 0x55d4bdafdb40 2024-09-19T08:53:03.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.920+0000 7faafeffd640 1 -- 172.21.15.164:0/872830240 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fab1403c050 con 0x55d4bdafdb40 2024-09-19T08:53:03.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.920+0000 7fab2810c740 10 client.16935 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:03.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.920+0000 7fab2810c740 1 -- 172.21.15.164:0/872830240 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d4bdb85aa0 con 0x55d4bdafdb40 2024-09-19T08:53:03.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.920+0000 7faafeffd640 1 -- 172.21.15.164:0/872830240 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab18011cf0 con 0x55d4bdafdb40 2024-09-19T08:53:03.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.920+0000 7faafeffd640 1 -- 172.21.15.164:0/872830240 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fab140104f0 con 0x55d4bdafdb40 2024-09-19T08:53:03.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.920+0000 7fab2810c740 10 client.16935 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:03.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.920+0000 7fab2810c740 10 client.16935 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T08:53:03.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.920+0000 7fab2810c740 20 client.16935 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:03.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.920+0000 7fab2810c740 1 --2- 172.21.15.164:0/872830240 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55d4bdb861b0 0x55d4bdb885d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:03.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.920+0000 7fab2810c740 4 client.16935 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:53:03.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.920+0000 7fab2810c740 1 -- 172.21.15.164:0/872830240 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d4bda69d70 con 0x55d4bdb861b0 2024-09-19T08:53:03.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.920+0000 7fab1f7fe640 1 --2- 172.21.15.164:0/872830240 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55d4bdb861b0 0x55d4bdb885d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:03.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.921+0000 7faafeffd640 10 client.16935 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:53:03.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.921+0000 7fab1f7fe640 1 --2- 172.21.15.164:0/872830240 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55d4bdb861b0 0x55d4bdb885d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=217a9a8dea87bb0a server_cookie=68ca2a19bdfceb9a in_seq=0 out_seq=0 2024-09-19T08:53:03.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.921+0000 7faafeffd640 1 -- 172.21.15.164:0/872830240 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d4bda69d70 con 0x55d4bdb861b0 2024-09-19T08:53:03.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.921+0000 7faafeffd640 10 client.16935 handle_command_reply: tid=0 2024-09-19T08:53:03.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.921+0000 7fab2810c740 1 client.16935 shutdown 2024-09-19T08:53:03.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.921+0000 7fab2810c740 1 -- 172.21.15.164:0/872830240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4bdafdb40 msgr2=0x55d4bdb00bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:03.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.921+0000 7fab2810c740 1 --2- 172.21.15.164:0/872830240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4bdafdb40 0x55d4bdb00bd0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x55d4bdb05600 tx=0x7fab14010f60 comp rx=0 tx=0).stop 2024-09-19T08:53:03.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.921+0000 7fab2810c740 1 -- 172.21.15.164:0/872830240 shutdown_connections 2024-09-19T08:53:03.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.921+0000 7fab2810c740 1 --2- 172.21.15.164:0/872830240 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55d4bdb861b0 0x55d4bdb885d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:03.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.921+0000 7fab2810c740 1 --2- 172.21.15.164:0/872830240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d4bdb07370 0x55d4bdb01110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:03.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.921+0000 7fab2810c740 1 --2- 172.21.15.164:0/872830240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4bdafdb40 0x55d4bdb00bd0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:03.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.921+0000 7faafeffd640 0 client.16935 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:53:03.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.921+0000 7fab2810c740 1 --2- 172.21.15.164:0/872830240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d4bdafc5d0 0x55d4bdb13ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:03.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.921+0000 7fab2810c740 1 -- 172.21.15.164:0/872830240 >> 172.21.15.164:0/872830240 conn(0x55d4bdac5310 msgr2=0x55d4bdafe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:03.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.921+0000 7fab2810c740 1 -- 172.21.15.164:0/872830240 shutdown_connections 2024-09-19T08:53:03.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.921+0000 7fab2810c740 1 -- 172.21.15.164:0/872830240 wait complete. 2024-09-19T08:53:03.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.921+0000 7fab2810c740 20 client.16935 trim_cache size 0 max 16384 2024-09-19T08:53:03.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.922+0000 7fab25d33640 1 -- 172.21.15.164:0/742295454 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fab080a33b0 msgr2=0x7fab080a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:03.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.922+0000 7fab25d33640 1 --2- 172.21.15.164:0/742295454 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fab080a33b0 0x7fab080a5870 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fab18005cc0 tx=0x7fab1800d710 comp rx=0 tx=0).stop 2024-09-19T08:53:03.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.922+0000 7fab25d33640 1 -- 172.21.15.164:0/742295454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab20101de0 msgr2=0x7fab201983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:03.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.922+0000 7fab25d33640 1 --2- 172.21.15.164:0/742295454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab20101de0 0x7fab201983d0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fab10006010 tx=0x7fab100042f0 comp rx=0 tx=0).stop 2024-09-19T08:53:03.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.922+0000 7fab25d33640 1 -- 172.21.15.164:0/742295454 shutdown_connections 2024-09-19T08:53:03.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.922+0000 7fab25d33640 1 --2- 172.21.15.164:0/742295454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab20106f10 0x7fab2019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:03.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.922+0000 7fab25d33640 1 --2- 172.21.15.164:0/742295454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab20102790 0x7fab20198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:03.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.922+0000 7fab25d33640 1 --2- 172.21.15.164:0/742295454 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fab080a33b0 0x7fab080a5870 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:03.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.922+0000 7fab25d33640 1 --2- 172.21.15.164:0/742295454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab20101de0 0x7fab201983d0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:03.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.922+0000 7fab25d33640 1 -- 172.21.15.164:0/742295454 >> 172.21.15.164:0/742295454 conn(0x7fab200fd8c0 msgr2=0x7fab200febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:03.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.922+0000 7fab25d33640 1 -- 172.21.15.164:0/742295454 shutdown_connections 2024-09-19T08:53:03.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:02.922+0000 7fab25d33640 1 -- 172.21.15.164:0/742295454 wait complete. 2024-09-19T08:53:03.795 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T08:53:04.041 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:53:04.461 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:04 smithi164 ceph-mon[30928]: pgmap v222: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-19T08:53:04.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.546+0000 7f3378af6640 1 -- 172.21.15.164:0/182981115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3374101de0 msgr2=0x7f33741021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:04.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.546+0000 7f3378af6640 1 --2- 172.21.15.164:0/182981115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3374101de0 0x7f33741021c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f336c009ca0 tx=0x7f336c02a8c0 comp rx=0 tx=0).stop 2024-09-19T08:53:04.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.546+0000 7f3378af6640 1 -- 172.21.15.164:0/182981115 shutdown_connections 2024-09-19T08:53:04.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.546+0000 7f3378af6640 1 --2- 172.21.15.164:0/182981115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3374106f10 0x7f337410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:04.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.546+0000 7f3378af6640 1 --2- 172.21.15.164:0/182981115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3374102790 0x7f33741067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:04.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.546+0000 7f3378af6640 1 --2- 172.21.15.164:0/182981115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3374101de0 0x7f33741021c0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:04.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.546+0000 7f3378af6640 1 -- 172.21.15.164:0/182981115 >> 172.21.15.164:0/182981115 conn(0x7f33740fd8c0 msgr2=0x7f33740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:04.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.546+0000 7f3378af6640 1 -- 172.21.15.164:0/182981115 shutdown_connections 2024-09-19T08:53:04.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.546+0000 7f3378af6640 1 -- 172.21.15.164:0/182981115 wait complete. 2024-09-19T08:53:04.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.547+0000 7f3378af6640 1 Processor -- start 2024-09-19T08:53:04.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.547+0000 7f3378af6640 1 -- start start 2024-09-19T08:53:04.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.547+0000 7f3378af6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3374101de0 0x7f3374198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:04.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.547+0000 7f3378af6640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3374102790 0x7f33741989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:04.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.547+0000 7f3378af6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3374106f10 0x7f337419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:04.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.547+0000 7f3378af6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3374111f70 con 0x7f3374106f10 2024-09-19T08:53:04.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.547+0000 7f3378af6640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3374111df0 con 0x7f3374101de0 2024-09-19T08:53:04.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.547+0000 7f3378af6640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f33741120f0 con 0x7f3374102790 2024-09-19T08:53:04.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.547+0000 7f3372d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3374106f10 0x7f337419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:04.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.547+0000 7f337259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3374101de0 0x7f3374198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:04.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.547+0000 7f3372d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3374106f10 0x7f337419cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42176/0 (socket says 172.21.15.164:42176) 2024-09-19T08:53:04.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.547+0000 7f3372d9d640 1 -- 172.21.15.164:0/3959897117 learned_addr learned my addr 172.21.15.164:0/3959897117 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:04.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.547+0000 7f3371d9b640 1 --2- 172.21.15.164:0/3959897117 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3374102790 0x7f33741989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:04.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.548+0000 7f3372d9d640 1 -- 172.21.15.164:0/3959897117 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3374102790 msgr2=0x7f33741989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:04.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.548+0000 7f3372d9d640 1 --2- 172.21.15.164:0/3959897117 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3374102790 0x7f33741989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:04.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.548+0000 7f3372d9d640 1 -- 172.21.15.164:0/3959897117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3374101de0 msgr2=0x7f3374198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:04.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.548+0000 7f3372d9d640 1 --2- 172.21.15.164:0/3959897117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3374101de0 0x7f3374198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:04.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.548+0000 7f3372d9d640 1 -- 172.21.15.164:0/3959897117 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f337419d440 con 0x7f3374106f10 2024-09-19T08:53:04.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.548+0000 7f3372d9d640 1 --2- 172.21.15.164:0/3959897117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3374106f10 0x7f337419cd60 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f336400e970 tx=0x7f336400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:04.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.548+0000 7f336b7fe640 1 -- 172.21.15.164:0/3959897117 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336400ccc0 con 0x7f3374106f10 2024-09-19T08:53:04.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.548+0000 7f3378af6640 1 -- 172.21.15.164:0/3959897117 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f337419d730 con 0x7f3374106f10 2024-09-19T08:53:04.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.548+0000 7f336b7fe640 1 -- 172.21.15.164:0/3959897117 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f336400ce60 con 0x7f3374106f10 2024-09-19T08:53:04.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.548+0000 7f336b7fe640 1 -- 172.21.15.164:0/3959897117 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364010490 con 0x7f3374106f10 2024-09-19T08:53:04.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.548+0000 7f3378af6640 1 -- 172.21.15.164:0/3959897117 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33741a5010 con 0x7f3374106f10 2024-09-19T08:53:04.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.550+0000 7f336b7fe640 1 -- 172.21.15.164:0/3959897117 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f3364005020 con 0x7f3374106f10 2024-09-19T08:53:04.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.551+0000 7f337aecf740 1 Processor -- start 2024-09-19T08:53:04.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.551+0000 7f337aecf740 1 -- start start 2024-09-19T08:53:04.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.551+0000 7f337aecf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560c534eab40 0x560c534e9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:04.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.551+0000 7f337aecf740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560c534e95d0 0x560c534e9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:04.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.551+0000 7f337aecf740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c534f8780 0x560c534fab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:04.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.551+0000 7f337aecf740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560c5344f2f0 con 0x560c534f8780 2024-09-19T08:53:04.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.551+0000 7f337aecf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560c533f6cf0 con 0x560c534eab40 2024-09-19T08:53:04.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.551+0000 7f337aecf740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x560c5344b2b0 con 0x560c534e95d0 2024-09-19T08:53:04.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.551+0000 7f3371d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c534f8780 0x560c534fab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:04.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.551+0000 7f3371d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c534f8780 0x560c534fab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42190/0 (socket says 172.21.15.164:42190) 2024-09-19T08:53:04.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.551+0000 7f3371d9b640 1 -- 172.21.15.164:0/2143325115 learned_addr learned my addr 172.21.15.164:0/2143325115 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:04.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.551+0000 7f336b7fe640 1 --2- 172.21.15.164:0/3959897117 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f33600a33b0 0x7f33600a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:04.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.551+0000 7f3371d9b640 1 --2- 172.21.15.164:0/2143325115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560c534eab40 0x560c534e9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:04.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.551+0000 7f337259c640 1 --2- 172.21.15.164:0/2143325115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560c534e95d0 0x560c534e9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:04.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.552+0000 7f3372d9d640 1 --2- 172.21.15.164:0/3959897117 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f33600a33b0 0x7f33600a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:04.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.552+0000 7f3371d9b640 1 -- 172.21.15.164:0/2143325115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560c534e95d0 msgr2=0x560c534e9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:04.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.552+0000 7f3371d9b640 1 --2- 172.21.15.164:0/2143325115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560c534e95d0 0x560c534e9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:04.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.552+0000 7f3371d9b640 1 -- 172.21.15.164:0/2143325115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c534f8780 msgr2=0x560c534fab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:04.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.552+0000 7f3371d9b640 1 --2- 172.21.15.164:0/2143325115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c534f8780 0x560c534fab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:04.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.552+0000 7f3371d9b640 1 -- 172.21.15.164:0/2143325115 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c533db3a0 con 0x560c534eab40 2024-09-19T08:53:04.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.552+0000 7f336b7fe640 1 -- 172.21.15.164:0/3959897117 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f33640d0100 con 0x7f3374106f10 2024-09-19T08:53:04.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.552+0000 7f3371d9b640 1 --2- 172.21.15.164:0/2143325115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560c534eab40 0x560c534e9090 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f335c0117e0 tx=0x7f335c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=655791221a960a06 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:04.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.552+0000 7f3372d9d640 1 --2- 172.21.15.164:0/3959897117 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f33600a33b0 0x7f33600a5870 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f33640cd940 tx=0x7f3364098040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:04.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.552+0000 7f3368ff9640 1 -- 172.21.15.164:0/2143325115 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c011e20 con 0x560c534eab40 2024-09-19T08:53:04.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.552+0000 7f3368ff9640 1 -- 172.21.15.164:0/2143325115 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f335c00f7c0 con 0x560c534eab40 2024-09-19T08:53:04.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.552+0000 7f3368ff9640 1 -- 172.21.15.164:0/2143325115 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c0038e0 con 0x560c534eab40 2024-09-19T08:53:04.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.553+0000 7f337aecf740 1 -- 172.21.15.164:0/2143325115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560c534eab40 msgr2=0x560c534e9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:04.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.553+0000 7f337aecf740 1 --2- 172.21.15.164:0/2143325115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560c534eab40 0x560c534e9090 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f335c0117e0 tx=0x7f335c005f90 comp rx=0 tx=0).stop 2024-09-19T08:53:04.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.553+0000 7f337aecf740 1 -- 172.21.15.164:0/2143325115 shutdown_connections 2024-09-19T08:53:04.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.553+0000 7f337aecf740 1 --2- 172.21.15.164:0/2143325115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c534f8780 0x560c534fab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:04.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.553+0000 7f337aecf740 1 --2- 172.21.15.164:0/2143325115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560c534e95d0 0x560c534e9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:04.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.553+0000 7f337aecf740 1 --2- 172.21.15.164:0/2143325115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560c534eab40 0x560c534e9090 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:04.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.553+0000 7f337aecf740 1 -- 172.21.15.164:0/2143325115 >> 172.21.15.164:0/2143325115 conn(0x560c534b2310 msgr2=0x560c534e5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:04.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.553+0000 7f337aecf740 1 -- 172.21.15.164:0/2143325115 shutdown_connections 2024-09-19T08:53:04.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.553+0000 7f337aecf740 1 -- 172.21.15.164:0/2143325115 wait complete. 2024-09-19T08:53:04.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.553+0000 7f337aecf740 1 -- start start 2024-09-19T08:53:04.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.554+0000 7f337aecf740 1 Processor -- start 2024-09-19T08:53:04.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.554+0000 7f337aecf740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560c534e95d0 0x560c534f8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:04.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.554+0000 7f337aecf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560c534eab40 0x560c534f5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:04.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.554+0000 7f337aecf740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c534f8780 0x560c534f5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:04.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.554+0000 7f337aecf740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560c534fb480 con 0x560c534f8780 2024-09-19T08:53:04.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.554+0000 7f337aecf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560c5340a060 con 0x560c534eab40 2024-09-19T08:53:04.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.554+0000 7f337aecf740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x560c534fb600 con 0x560c534e95d0 2024-09-19T08:53:04.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.554+0000 7f337aecf740 10 client.0 _ll_register_callbacks cb 0x560c5315d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:04.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.554+0000 7f3371d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c534f8780 0x560c534f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:04.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.554+0000 7f337259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560c534eab40 0x560c534f5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:04.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.554+0000 7f3371d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c534f8780 0x560c534f5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42196/0 (socket says 172.21.15.164:42196) 2024-09-19T08:53:04.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.554+0000 7f337259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560c534eab40 0x560c534f5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:46018/0 (socket says 172.21.15.164:46018) 2024-09-19T08:53:04.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.554+0000 7f3371d9b640 1 -- 172.21.15.164:0/3911567770 learned_addr learned my addr 172.21.15.164:0/3911567770 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:04.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.554+0000 7f3371d9b640 1 --2- 172.21.15.164:0/3911567770 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560c534e95d0 0x560c534f8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:04.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.554+0000 7f337259c640 1 -- 172.21.15.164:0/3911567770 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560c534e95d0 msgr2=0x560c534f8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:04.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.554+0000 7f337259c640 1 --2- 172.21.15.164:0/3911567770 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560c534e95d0 0x560c534f8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:04.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.554+0000 7f337259c640 1 -- 172.21.15.164:0/3911567770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c534f8780 msgr2=0x560c534f5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:04.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.554+0000 7f337259c640 1 --2- 172.21.15.164:0/3911567770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c534f8780 0x560c534f5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:04.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.554+0000 7f337259c640 1 -- 172.21.15.164:0/3911567770 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c531c03c0 con 0x560c534eab40 2024-09-19T08:53:04.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.554+0000 7f3371d9b640 1 --2- 172.21.15.164:0/3911567770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c534f8780 0x560c534f5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:53:04.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.555+0000 7f337259c640 1 --2- 172.21.15.164:0/3911567770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560c534eab40 0x560c534f5420 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f336c000f80 tx=0x7f336c005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f8d211b96beb12d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:04.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.555+0000 7f3359ffb640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-19T08:53:04.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.555+0000 7f3359ffb640 1 -- 172.21.15.164:0/3911567770 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c02f610 con 0x560c534eab40 2024-09-19T08:53:04.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.555+0000 7f337aecf740 1 -- 172.21.15.164:0/3911567770 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f335c00f7c0 con 0x560c534eab40 2024-09-19T08:53:04.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.555+0000 7f3359ffb640 1 -- 172.21.15.164:0/3911567770 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f336c02f7b0 con 0x560c534eab40 2024-09-19T08:53:04.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.555+0000 7f3359ffb640 1 -- 172.21.15.164:0/3911567770 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f336c02f9b0 con 0x560c534eab40 2024-09-19T08:53:04.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.555+0000 7f337aecf740 10 client.26225 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:04.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.555+0000 7f337aecf740 1 -- 172.21.15.164:0/3911567770 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560c53506f70 con 0x560c534eab40 2024-09-19T08:53:04.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.555+0000 7f3359ffb640 1 -- 172.21.15.164:0/3911567770 <== mon.2 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f335c00f7c0 con 0x560c534eab40 2024-09-19T08:53:04.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.556+0000 7f3359ffb640 1 -- 172.21.15.164:0/3911567770 <== mon.2 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f336c045070 con 0x560c534eab40 2024-09-19T08:53:04.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.556+0000 7f337aecf740 10 client.26225 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:04.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.556+0000 7f337aecf740 10 client.26225 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T08:53:04.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.556+0000 7f337aecf740 20 client.26225 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:04.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.556+0000 7f337aecf740 1 --2- 172.21.15.164:0/3911567770 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x560c53508380 0x560c5350a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:04.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.556+0000 7f337aecf740 4 client.26225 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:53:04.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.556+0000 7f337aecf740 1 -- 172.21.15.164:0/3911567770 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c5345c860 con 0x560c53508380 2024-09-19T08:53:04.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.556+0000 7f3371d9b640 1 --2- 172.21.15.164:0/3911567770 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x560c53508380 0x560c5350a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:04.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.556+0000 7f3359ffb640 10 client.26225 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:53:04.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.556+0000 7f3371d9b640 1 --2- 172.21.15.164:0/3911567770 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x560c53508380 0x560c5350a770 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c6ac4313142178c5 server_cookie=bf3ec6d167e57e6b in_seq=0 out_seq=0 2024-09-19T08:53:04.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.557+0000 7f3359ffb640 1 -- 172.21.15.164:0/3911567770 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560c5345c860 con 0x560c53508380 2024-09-19T08:53:04.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.557+0000 7f3359ffb640 10 client.26225 handle_command_reply: tid=0 2024-09-19T08:53:04.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.557+0000 7f337aecf740 1 client.26225 shutdown 2024-09-19T08:53:04.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.557+0000 7f337aecf740 1 -- 172.21.15.164:0/3911567770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560c534eab40 msgr2=0x560c534f5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:04.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.557+0000 7f337aecf740 1 --2- 172.21.15.164:0/3911567770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560c534eab40 0x560c534f5420 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f336c000f80 tx=0x7f336c005c50 comp rx=0 tx=0).stop 2024-09-19T08:53:04.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.557+0000 7f337aecf740 1 -- 172.21.15.164:0/3911567770 shutdown_connections 2024-09-19T08:53:04.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.557+0000 7f337aecf740 1 --2- 172.21.15.164:0/3911567770 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x560c53508380 0x560c5350a770 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:04.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.557+0000 7f337aecf740 1 --2- 172.21.15.164:0/3911567770 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c534f8780 0x560c534f5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:04.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.557+0000 7f3359ffb640 0 client.26225 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:53:04.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.557+0000 7f337aecf740 1 --2- 172.21.15.164:0/3911567770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560c534eab40 0x560c534f5420 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:04.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.557+0000 7f337aecf740 1 --2- 172.21.15.164:0/3911567770 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560c534e95d0 0x560c534f8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:04.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.557+0000 7f337aecf740 1 -- 172.21.15.164:0/3911567770 >> 172.21.15.164:0/3911567770 conn(0x560c534b2310 msgr2=0x560c534eb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:04.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.557+0000 7f337aecf740 1 -- 172.21.15.164:0/3911567770 shutdown_connections 2024-09-19T08:53:04.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.557+0000 7f337aecf740 1 -- 172.21.15.164:0/3911567770 wait complete. 2024-09-19T08:53:04.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.557+0000 7f337aecf740 20 client.26225 trim_cache size 0 max 16384 2024-09-19T08:53:04.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.570+0000 7f337aecf740 1 Processor -- start 2024-09-19T08:53:04.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.570+0000 7f337aecf740 1 -- start start 2024-09-19T08:53:04.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.571+0000 7f337aecf740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560c534e95d0 0x560c53504720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:04.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.571+0000 7f337aecf740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c534eab40 0x560c534f4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:04.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.571+0000 7f337aecf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560c534f8780 0x560c53510040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:04.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.571+0000 7f337aecf740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f33741ab5d0 con 0x560c534eab40 2024-09-19T08:53:04.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.571+0000 7f337aecf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f33741ad110 con 0x560c534f8780 2024-09-19T08:53:04.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.571+0000 7f337aecf740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f33741abca0 con 0x560c534e95d0 2024-09-19T08:53:04.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.571+0000 7f337259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c534eab40 0x560c534f4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:04.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.571+0000 7f337259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c534eab40 0x560c534f4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42204/0 (socket says 172.21.15.164:42204) 2024-09-19T08:53:04.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.571+0000 7f337259c640 1 -- 172.21.15.164:0/1202769331 learned_addr learned my addr 172.21.15.164:0/1202769331 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:04.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.571+0000 7f3371d9b640 1 --2- 172.21.15.164:0/1202769331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560c534e95d0 0x560c53504720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:04.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.571+0000 7f3371d9b640 1 --2- 172.21.15.164:0/1202769331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560c534f8780 0x560c53510040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:04.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.571+0000 7f337259c640 1 -- 172.21.15.164:0/1202769331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560c534e95d0 msgr2=0x560c53504720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:04.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.571+0000 7f337259c640 1 --2- 172.21.15.164:0/1202769331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560c534e95d0 0x560c53504720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:04.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.571+0000 7f337259c640 1 -- 172.21.15.164:0/1202769331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560c534f8780 msgr2=0x560c53510040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:04.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.571+0000 7f337259c640 1 --2- 172.21.15.164:0/1202769331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560c534f8780 0x560c53510040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:04.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.571+0000 7f337259c640 1 -- 172.21.15.164:0/1202769331 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c534e5ed0 con 0x560c534eab40 2024-09-19T08:53:04.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.571+0000 7f337259c640 1 --2- 172.21.15.164:0/1202769331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c534eab40 0x560c534f4400 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f336c02ade0 tx=0x7f336c03b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a107d74b8f7e1ae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:04.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.571+0000 7f335bfff640 1 -- 172.21.15.164:0/1202769331 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c05c070 con 0x560c534eab40 2024-09-19T08:53:04.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.571+0000 7f335bfff640 1 -- 172.21.15.164:0/1202769331 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f336c036040 con 0x560c534eab40 2024-09-19T08:53:04.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.571+0000 7f335bfff640 1 -- 172.21.15.164:0/1202769331 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c003bd0 con 0x560c534eab40 2024-09-19T08:53:04.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.572+0000 7f337aecf740 1 -- 172.21.15.164:0/1202769331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c534eab40 msgr2=0x560c534f4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:04.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.572+0000 7f337aecf740 1 --2- 172.21.15.164:0/1202769331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c534eab40 0x560c534f4400 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f336c02ade0 tx=0x7f336c03b400 comp rx=0 tx=0).stop 2024-09-19T08:53:04.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.572+0000 7f337aecf740 1 -- 172.21.15.164:0/1202769331 shutdown_connections 2024-09-19T08:53:04.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.572+0000 7f337aecf740 1 --2- 172.21.15.164:0/1202769331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560c534f8780 0x560c53510040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:04.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.572+0000 7f337aecf740 1 --2- 172.21.15.164:0/1202769331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c534eab40 0x560c534f4400 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:04.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.572+0000 7f337aecf740 1 --2- 172.21.15.164:0/1202769331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560c534e95d0 0x560c53504720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:04.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.572+0000 7f337aecf740 1 -- 172.21.15.164:0/1202769331 >> 172.21.15.164:0/1202769331 conn(0x560c534b2310 msgr2=0x560c534f3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:04.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.572+0000 7f337aecf740 1 -- 172.21.15.164:0/1202769331 shutdown_connections 2024-09-19T08:53:04.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.572+0000 7f337aecf740 1 -- 172.21.15.164:0/1202769331 wait complete. 2024-09-19T08:53:04.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.573+0000 7f337aecf740 1 -- start start 2024-09-19T08:53:04.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.573+0000 7f337aecf740 1 Processor -- start 2024-09-19T08:53:04.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.573+0000 7f337aecf740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c534e95d0 0x560c53500890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:04.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.573+0000 7f337aecf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560c534eab40 0x560c53500dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:04.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.573+0000 7f337aecf740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560c534f8780 0x560c534f1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:04.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.573+0000 7f337aecf740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f33741aca40 con 0x560c534e95d0 2024-09-19T08:53:04.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.573+0000 7f337aecf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f33741ac370 con 0x560c534eab40 2024-09-19T08:53:04.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.573+0000 7f337aecf740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f33741a95d0 con 0x560c534f8780 2024-09-19T08:53:04.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.573+0000 7f337aecf740 10 client.0 _ll_register_callbacks cb 0x7f336c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:04.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.573+0000 7f3371d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c534e95d0 0x560c53500890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:04.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.573+0000 7f3371d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c534e95d0 0x560c53500890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42218/0 (socket says 172.21.15.164:42218) 2024-09-19T08:53:04.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.573+0000 7f3371d9b640 1 -- 172.21.15.164:0/2407230918 learned_addr learned my addr 172.21.15.164:0/2407230918 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:04.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.573+0000 7f337259c640 1 --2- 172.21.15.164:0/2407230918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560c534eab40 0x560c53500dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:04.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.573+0000 7f3371d9b640 1 --2- 172.21.15.164:0/2407230918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560c534f8780 0x560c534f1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:04.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.574+0000 7f337259c640 1 -- 172.21.15.164:0/2407230918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560c534f8780 msgr2=0x560c534f1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:04.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.574+0000 7f337259c640 1 --2- 172.21.15.164:0/2407230918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560c534f8780 0x560c534f1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:04.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.574+0000 7f337259c640 1 -- 172.21.15.164:0/2407230918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c534e95d0 msgr2=0x560c53500890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:04.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.574+0000 7f337259c640 1 --2- 172.21.15.164:0/2407230918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c534e95d0 0x560c53500890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:04.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.574+0000 7f337259c640 1 -- 172.21.15.164:0/2407230918 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c534f1fb0 con 0x560c534eab40 2024-09-19T08:53:04.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.574+0000 7f337259c640 1 --2- 172.21.15.164:0/2407230918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560c534eab40 0x560c53500dd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x560c534f6940 tx=0x7f336c005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3337a60648572d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:04.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.574+0000 7f3359ffb640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-19T08:53:04.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.574+0000 7f3359ffb640 1 -- 172.21.15.164:0/2407230918 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c05c070 con 0x560c534eab40 2024-09-19T08:53:04.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.574+0000 7f337aecf740 1 -- 172.21.15.164:0/2407230918 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f336c036040 con 0x560c534eab40 2024-09-19T08:53:04.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.574+0000 7f3359ffb640 1 -- 172.21.15.164:0/2407230918 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7f336c02fb50 con 0x560c534eab40 2024-09-19T08:53:04.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.574+0000 7f3359ffb640 1 -- 172.21.15.164:0/2407230918 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f336c02fd70 con 0x560c534eab40 2024-09-19T08:53:04.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.574+0000 7f337aecf740 10 client.26231 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:04.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.574+0000 7f337aecf740 1 -- 172.21.15.164:0/2407230918 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560c53572770 con 0x560c534eab40 2024-09-19T08:53:04.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.574+0000 7f3359ffb640 1 -- 172.21.15.164:0/2407230918 <== mon.2 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f336c036040 con 0x560c534eab40 2024-09-19T08:53:04.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.575+0000 7f3359ffb640 1 -- 172.21.15.164:0/2407230918 <== mon.2 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f336c056030 con 0x560c534eab40 2024-09-19T08:53:04.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.575+0000 7f337aecf740 10 client.26231 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:04.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.575+0000 7f337aecf740 10 client.26231 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T08:53:04.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.575+0000 7f337aecf740 20 client.26231 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:04.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.575+0000 7f337aecf740 1 --2- 172.21.15.164:0/2407230918 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x560c53572c40 0x560c53575030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:04.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.575+0000 7f337aecf740 4 client.26231 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:53:04.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.575+0000 7f337aecf740 1 -- 172.21.15.164:0/2407230918 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560c53456d70 con 0x560c53572c40 2024-09-19T08:53:04.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.575+0000 7f3371d9b640 1 --2- 172.21.15.164:0/2407230918 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x560c53572c40 0x560c53575030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:04.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.575+0000 7f3371d9b640 1 --2- 172.21.15.164:0/2407230918 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x560c53572c40 0x560c53575030 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=53201397714d8c28 server_cookie=ccee001506da27c4 in_seq=0 out_seq=0 2024-09-19T08:53:04.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.575+0000 7f3359ffb640 10 client.26231 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:53:04.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.575+0000 7f3359ffb640 1 -- 172.21.15.164:0/2407230918 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560c53456d70 con 0x560c53572c40 2024-09-19T08:53:04.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.575+0000 7f3359ffb640 10 client.26231 handle_command_reply: tid=0 2024-09-19T08:53:04.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.575+0000 7f337aecf740 1 client.26231 shutdown 2024-09-19T08:53:04.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.576+0000 7f337aecf740 1 -- 172.21.15.164:0/2407230918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560c534eab40 msgr2=0x560c53500dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:04.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.576+0000 7f337aecf740 1 --2- 172.21.15.164:0/2407230918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560c534eab40 0x560c53500dd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x560c534f6940 tx=0x7f336c005ef0 comp rx=0 tx=0).stop 2024-09-19T08:53:04.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.576+0000 7f337aecf740 1 -- 172.21.15.164:0/2407230918 shutdown_connections 2024-09-19T08:53:04.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.576+0000 7f337aecf740 1 --2- 172.21.15.164:0/2407230918 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x560c53572c40 0x560c53575030 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:04.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.576+0000 7f337aecf740 1 --2- 172.21.15.164:0/2407230918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560c534f8780 0x560c534f1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:04.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.576+0000 7f337aecf740 1 --2- 172.21.15.164:0/2407230918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560c534eab40 0x560c53500dd0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:04.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.576+0000 7f3359ffb640 0 client.26231 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:53:04.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.576+0000 7f337aecf740 1 --2- 172.21.15.164:0/2407230918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c534e95d0 0x560c53500890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:04.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.576+0000 7f337aecf740 1 -- 172.21.15.164:0/2407230918 >> 172.21.15.164:0/2407230918 conn(0x560c534b2310 msgr2=0x560c534eb110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:04.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.576+0000 7f337aecf740 1 -- 172.21.15.164:0/2407230918 shutdown_connections 2024-09-19T08:53:04.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.576+0000 7f337aecf740 1 -- 172.21.15.164:0/2407230918 wait complete. 2024-09-19T08:53:04.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.576+0000 7f337aecf740 20 client.26231 trim_cache size 0 max 16384 2024-09-19T08:53:04.583 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:53:04.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.576+0000 7f3378af6640 1 -- 172.21.15.164:0/3959897117 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f33600a33b0 msgr2=0x7f33600a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:04.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.576+0000 7f3378af6640 1 --2- 172.21.15.164:0/3959897117 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f33600a33b0 0x7f33600a5870 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f33640cd940 tx=0x7f3364098040 comp rx=0 tx=0).stop 2024-09-19T08:53:04.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.576+0000 7f3378af6640 1 -- 172.21.15.164:0/3959897117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3374106f10 msgr2=0x7f337419cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:04.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.576+0000 7f3378af6640 1 --2- 172.21.15.164:0/3959897117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3374106f10 0x7f337419cd60 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f336400e970 tx=0x7f336400ed70 comp rx=0 tx=0).stop 2024-09-19T08:53:04.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.576+0000 7f3378af6640 1 -- 172.21.15.164:0/3959897117 shutdown_connections 2024-09-19T08:53:04.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.576+0000 7f3378af6640 1 --2- 172.21.15.164:0/3959897117 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f33600a33b0 0x7f33600a5870 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:04.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.576+0000 7f3378af6640 1 --2- 172.21.15.164:0/3959897117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3374106f10 0x7f337419cd60 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:04.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.576+0000 7f3378af6640 1 --2- 172.21.15.164:0/3959897117 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3374102790 0x7f33741989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:04.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.577+0000 7f3378af6640 1 --2- 172.21.15.164:0/3959897117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3374101de0 0x7f3374198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:04.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.577+0000 7f3378af6640 1 -- 172.21.15.164:0/3959897117 >> 172.21.15.164:0/3959897117 conn(0x7f33740fd8c0 msgr2=0x7f33740fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:04.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.577+0000 7f3378af6640 1 -- 172.21.15.164:0/3959897117 shutdown_connections 2024-09-19T08:53:04.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:04.577+0000 7f3378af6640 1 -- 172.21.15.164:0/3959897117 wait complete. 2024-09-19T08:53:04.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:04 smithi170 ceph-mon[35072]: pgmap v222: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-19T08:53:04.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:04 smithi173 ceph-mon[35306]: pgmap v222: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-19T08:53:05.339 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T08:53:05.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:05 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:53:05.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:05 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:53:05.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:05 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:53:05.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:05 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:53:05.588 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:53:05.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:05 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:53:05.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:05 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:53:05.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:05 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:53:05.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:05 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:53:05.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:05 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:53:05.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:05 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:53:05.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:05 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:53:05.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:05 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:53:06.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.020+0000 7fea66be8640 1 -- 172.21.15.164:0/343680038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea60100d20 msgr2=0x7fea60106860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:06.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.020+0000 7fea66be8640 1 --2- 172.21.15.164:0/343680038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea60100d20 0x7fea60106860 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fea50009cd0 tx=0x7fea5002a8e0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.020+0000 7fea66be8640 1 -- 172.21.15.164:0/343680038 shutdown_connections 2024-09-19T08:53:06.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.020+0000 7fea66be8640 1 --2- 172.21.15.164:0/343680038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea60106fc0 0x7fea6010eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.020+0000 7fea66be8640 1 --2- 172.21.15.164:0/343680038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea60100d20 0x7fea60106860 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.020+0000 7fea66be8640 1 --2- 172.21.15.164:0/343680038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea60100370 0x7fea60100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.020+0000 7fea66be8640 1 -- 172.21.15.164:0/343680038 >> 172.21.15.164:0/343680038 conn(0x7fea60077210 msgr2=0x7fea600fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:06.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.020+0000 7fea66be8640 1 -- 172.21.15.164:0/343680038 shutdown_connections 2024-09-19T08:53:06.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.020+0000 7fea66be8640 1 -- 172.21.15.164:0/343680038 wait complete. 2024-09-19T08:53:06.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.020+0000 7fea66be8640 1 Processor -- start 2024-09-19T08:53:06.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.020+0000 7fea66be8640 1 -- start start 2024-09-19T08:53:06.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.021+0000 7fea66be8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea60100370 0x7fea601984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:06.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.021+0000 7fea66be8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea60100d20 0x7fea60198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:06.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.021+0000 7fea66be8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea60106fc0 0x7fea6019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:06.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.021+0000 7fea66be8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fea60111fa0 con 0x7fea60100370 2024-09-19T08:53:06.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.021+0000 7fea66be8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fea60111e20 con 0x7fea60106fc0 2024-09-19T08:53:06.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.021+0000 7fea66be8640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fea60112120 con 0x7fea60100d20 2024-09-19T08:53:06.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.021+0000 7fea64984640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea60100370 0x7fea601984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:06.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.021+0000 7fea64984640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea60100370 0x7fea601984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42228/0 (socket says 172.21.15.164:42228) 2024-09-19T08:53:06.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.021+0000 7fea64984640 1 -- 172.21.15.164:0/47506807 learned_addr learned my addr 172.21.15.164:0/47506807 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:06.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.021+0000 7fea5ffff640 1 --2- 172.21.15.164:0/47506807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea60100d20 0x7fea60198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:06.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.021+0000 7fea65185640 1 --2- 172.21.15.164:0/47506807 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea60106fc0 0x7fea6019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:06.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.021+0000 7fea64984640 1 -- 172.21.15.164:0/47506807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea60100d20 msgr2=0x7fea60198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:06.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.021+0000 7fea64984640 1 --2- 172.21.15.164:0/47506807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea60100d20 0x7fea60198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.021+0000 7fea64984640 1 -- 172.21.15.164:0/47506807 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea60106fc0 msgr2=0x7fea6019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:06.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.021+0000 7fea64984640 1 --2- 172.21.15.164:0/47506807 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea60106fc0 0x7fea6019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.021+0000 7fea64984640 1 -- 172.21.15.164:0/47506807 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea6019d490 con 0x7fea60100370 2024-09-19T08:53:06.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.021+0000 7fea65185640 1 --2- 172.21.15.164:0/47506807 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea60106fc0 0x7fea6019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:53:06.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.021+0000 7fea5ffff640 1 --2- 172.21.15.164:0/47506807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea60100d20 0x7fea60198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:53:06.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.021+0000 7fea64984640 1 --2- 172.21.15.164:0/47506807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea60100370 0x7fea601984e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fea5800d950 tx=0x7fea5800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:06.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.021+0000 7fea5dffb640 1 -- 172.21.15.164:0/47506807 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea58017030 con 0x7fea60100370 2024-09-19T08:53:06.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.021+0000 7fea66be8640 1 -- 172.21.15.164:0/47506807 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea6019d780 con 0x7fea60100370 2024-09-19T08:53:06.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.021+0000 7fea5dffb640 1 -- 172.21.15.164:0/47506807 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea58004020 con 0x7fea60100370 2024-09-19T08:53:06.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.021+0000 7fea5dffb640 1 -- 172.21.15.164:0/47506807 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea580047a0 con 0x7fea60100370 2024-09-19T08:53:06.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.022+0000 7fea66be8640 1 -- 172.21.15.164:0/47506807 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea601a5060 con 0x7fea60100370 2024-09-19T08:53:06.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.024+0000 7fea5dffb640 1 -- 172.21.15.164:0/47506807 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fea580041d0 con 0x7fea60100370 2024-09-19T08:53:06.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.024+0000 7fea68fc1740 1 Processor -- start 2024-09-19T08:53:06.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.024+0000 7fea5dffb640 1 --2- 172.21.15.164:0/47506807 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fea4c0a33b0 0x7fea4c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:06.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.024+0000 7fea68fc1740 1 -- start start 2024-09-19T08:53:06.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.024+0000 7fea65185640 1 --2- 172.21.15.164:0/47506807 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fea4c0a33b0 0x7fea4c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:06.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.024+0000 7fea5dffb640 1 -- 172.21.15.164:0/47506807 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7fea580cf7e0 con 0x7fea60100370 2024-09-19T08:53:06.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.024+0000 7fea68fc1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a938b2fb40 0x55a938b2e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:06.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.024+0000 7fea68fc1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a938b2e5d0 0x55a938b2ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:06.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.024+0000 7fea68fc1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a938b39370 0x55a938b3b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:06.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.024+0000 7fea68fc1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a938a942f0 con 0x55a938b39370 2024-09-19T08:53:06.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.024+0000 7fea68fc1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a938a3bcf0 con 0x55a938b2e5d0 2024-09-19T08:53:06.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.024+0000 7fea68fc1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a938a902b0 con 0x55a938b2fb40 2024-09-19T08:53:06.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.025+0000 7fea65185640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a938b39370 0x55a938b3b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:06.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.025+0000 7fea65185640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a938b39370 0x55a938b3b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42242/0 (socket says 172.21.15.164:42242) 2024-09-19T08:53:06.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.025+0000 7fea65185640 1 -- 172.21.15.164:0/3787099754 learned_addr learned my addr 172.21.15.164:0/3787099754 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:06.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.025+0000 7fea64984640 1 --2- 172.21.15.164:0/3787099754 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a938b2fb40 0x55a938b2e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:06.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.025+0000 7fea5ffff640 1 --2- 172.21.15.164:0/3787099754 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a938b2e5d0 0x55a938b2ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:06.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.025+0000 7fea65185640 1 --2- 172.21.15.164:0/47506807 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fea4c0a33b0 0x7fea4c0a5870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fea4800bed0 tx=0x7fea48014840 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:06.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.025+0000 7fea65185640 1 -- 172.21.15.164:0/3787099754 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a938b2fb40 msgr2=0x55a938b2e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:06.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.025+0000 7fea65185640 1 --2- 172.21.15.164:0/3787099754 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a938b2fb40 0x55a938b2e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.025+0000 7fea65185640 1 -- 172.21.15.164:0/3787099754 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a938b2e5d0 msgr2=0x55a938b2ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:06.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.025+0000 7fea65185640 1 --2- 172.21.15.164:0/3787099754 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a938b2e5d0 0x55a938b2ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.025+0000 7fea65185640 1 -- 172.21.15.164:0/3787099754 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a938a203a0 con 0x55a938b39370 2024-09-19T08:53:06.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.025+0000 7fea65185640 1 --2- 172.21.15.164:0/3787099754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a938b39370 0x55a938b3b760 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fea48014d40 tx=0x7fea4801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a5a7d72ec626c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:06.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.025+0000 7fea56ffd640 1 -- 172.21.15.164:0/3787099754 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea480142d0 con 0x55a938b39370 2024-09-19T08:53:06.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.025+0000 7fea56ffd640 1 -- 172.21.15.164:0/3787099754 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea48023070 con 0x55a938b39370 2024-09-19T08:53:06.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.025+0000 7fea56ffd640 1 -- 172.21.15.164:0/3787099754 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4801e760 con 0x55a938b39370 2024-09-19T08:53:06.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.026+0000 7fea68fc1740 1 -- 172.21.15.164:0/3787099754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a938b39370 msgr2=0x55a938b3b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:06.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.026+0000 7fea68fc1740 1 --2- 172.21.15.164:0/3787099754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a938b39370 0x55a938b3b760 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fea48014d40 tx=0x7fea4801b000 comp rx=0 tx=0).stop 2024-09-19T08:53:06.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.026+0000 7fea68fc1740 1 -- 172.21.15.164:0/3787099754 shutdown_connections 2024-09-19T08:53:06.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.026+0000 7fea68fc1740 1 --2- 172.21.15.164:0/3787099754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a938b39370 0x55a938b3b760 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.026+0000 7fea68fc1740 1 --2- 172.21.15.164:0/3787099754 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a938b2e5d0 0x55a938b2ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.026+0000 7fea68fc1740 1 --2- 172.21.15.164:0/3787099754 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a938b2fb40 0x55a938b2e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.026+0000 7fea68fc1740 1 -- 172.21.15.164:0/3787099754 >> 172.21.15.164:0/3787099754 conn(0x55a938af7310 msgr2=0x55a938b2ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:06.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.026+0000 7fea68fc1740 1 -- 172.21.15.164:0/3787099754 shutdown_connections 2024-09-19T08:53:06.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.026+0000 7fea68fc1740 1 -- 172.21.15.164:0/3787099754 wait complete. 2024-09-19T08:53:06.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.027+0000 7fea68fc1740 1 -- start start 2024-09-19T08:53:06.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.027+0000 7fea68fc1740 1 Processor -- start 2024-09-19T08:53:06.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.027+0000 7fea68fc1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a938b2e5d0 0x55a938b38d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:06.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.027+0000 7fea68fc1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a938b2fb40 0x55a938b36010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:06.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.027+0000 7fea68fc1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a938b39370 0x55a938b36600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:06.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.027+0000 7fea68fc1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a938b40420 con 0x55a938b2e5d0 2024-09-19T08:53:06.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.027+0000 7fea68fc1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a938a4f060 con 0x55a938b2fb40 2024-09-19T08:53:06.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.027+0000 7fea68fc1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a938b405a0 con 0x55a938b39370 2024-09-19T08:53:06.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.027+0000 7fea68fc1740 10 client.0 _ll_register_callbacks cb 0x55a9387a28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:06.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.027+0000 7fea64984640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a938b2e5d0 0x55a938b38d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:06.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.027+0000 7fea64984640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a938b2e5d0 0x55a938b38d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42256/0 (socket says 172.21.15.164:42256) 2024-09-19T08:53:06.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.027+0000 7fea64984640 1 -- 172.21.15.164:0/1538061986 learned_addr learned my addr 172.21.15.164:0/1538061986 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:06.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.027+0000 7fea5ffff640 1 --2- 172.21.15.164:0/1538061986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a938b2fb40 0x55a938b36010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:06.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.027+0000 7fea65185640 1 --2- 172.21.15.164:0/1538061986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a938b39370 0x55a938b36600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:06.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.028+0000 7fea64984640 1 -- 172.21.15.164:0/1538061986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a938b39370 msgr2=0x55a938b36600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:06.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.028+0000 7fea64984640 1 --2- 172.21.15.164:0/1538061986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a938b39370 0x55a938b36600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.028+0000 7fea64984640 1 -- 172.21.15.164:0/1538061986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a938b2fb40 msgr2=0x55a938b36010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:06.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.028+0000 7fea64984640 1 --2- 172.21.15.164:0/1538061986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a938b2fb40 0x55a938b36010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.028+0000 7fea64984640 1 -- 172.21.15.164:0/1538061986 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9388053c0 con 0x55a938b2e5d0 2024-09-19T08:53:06.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.028+0000 7fea64984640 1 --2- 172.21.15.164:0/1538061986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a938b2e5d0 0x55a938b38d60 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fea58097f00 tx=0x7fea5809c1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d38c8e6816f469 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:06.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.028+0000 7fea3ffff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:53:06.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.028+0000 7fea3ffff640 1 -- 172.21.15.164:0/1538061986 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea580b4060 con 0x55a938b2e5d0 2024-09-19T08:53:06.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.028+0000 7fea3ffff640 1 -- 172.21.15.164:0/1538061986 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7fea580860b0 con 0x55a938b2e5d0 2024-09-19T08:53:06.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.028+0000 7fea68fc1740 1 -- 172.21.15.164:0/1538061986 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea48023070 con 0x55a938b2e5d0 2024-09-19T08:53:06.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.028+0000 7fea3ffff640 1 -- 172.21.15.164:0/1538061986 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fea580a4710 con 0x55a938b2e5d0 2024-09-19T08:53:06.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.028+0000 7fea68fc1740 10 client.16965 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:06.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.028+0000 7fea68fc1740 1 -- 172.21.15.164:0/1538061986 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a938b4bf20 con 0x55a938b2e5d0 2024-09-19T08:53:06.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.028+0000 7fea3ffff640 1 -- 172.21.15.164:0/1538061986 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea48023070 con 0x55a938b2e5d0 2024-09-19T08:53:06.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.028+0000 7fea3ffff640 1 -- 172.21.15.164:0/1538061986 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fea580977f0 con 0x55a938b2e5d0 2024-09-19T08:53:06.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.028+0000 7fea68fc1740 10 client.16965 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:06.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.028+0000 7fea68fc1740 10 client.16965 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T08:53:06.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.028+0000 7fea68fc1740 20 client.16965 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:06.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.028+0000 7fea68fc1740 1 --2- 172.21.15.164:0/1538061986 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55a938b4d330 0x55a938b4f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:06.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.028+0000 7fea68fc1740 4 client.16965 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:53:06.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.028+0000 7fea68fc1740 1 -- 172.21.15.164:0/1538061986 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a938aa1860 con 0x55a938b4d330 2024-09-19T08:53:06.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.029+0000 7fea5ffff640 1 --2- 172.21.15.164:0/1538061986 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55a938b4d330 0x55a938b4f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:06.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.029+0000 7fea3ffff640 10 client.16965 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:53:06.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.029+0000 7fea5ffff640 1 --2- 172.21.15.164:0/1538061986 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55a938b4d330 0x55a938b4f720 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8558f5d711866979 server_cookie=6f197f8229a1e76 in_seq=0 out_seq=0 2024-09-19T08:53:06.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.030+0000 7fea3ffff640 1 -- 172.21.15.164:0/1538061986 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a938aa1860 con 0x55a938b4d330 2024-09-19T08:53:06.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.030+0000 7fea3ffff640 10 client.16965 handle_command_reply: tid=0 2024-09-19T08:53:06.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.030+0000 7fea68fc1740 1 client.16965 shutdown 2024-09-19T08:53:06.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.030+0000 7fea68fc1740 1 -- 172.21.15.164:0/1538061986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a938b2e5d0 msgr2=0x55a938b38d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:06.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.030+0000 7fea68fc1740 1 --2- 172.21.15.164:0/1538061986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a938b2e5d0 0x55a938b38d60 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fea58097f00 tx=0x7fea5809c1a0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.030+0000 7fea68fc1740 1 -- 172.21.15.164:0/1538061986 shutdown_connections 2024-09-19T08:53:06.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.030+0000 7fea68fc1740 1 --2- 172.21.15.164:0/1538061986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a938b39370 0x55a938b36600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.030+0000 7fea68fc1740 1 --2- 172.21.15.164:0/1538061986 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55a938b4d330 0x55a938b4f720 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.030+0000 7fea68fc1740 1 --2- 172.21.15.164:0/1538061986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a938b2fb40 0x55a938b36010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.030+0000 7fea68fc1740 1 --2- 172.21.15.164:0/1538061986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a938b2e5d0 0x55a938b38d60 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.030+0000 7fea3ffff640 0 client.16965 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:53:06.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.030+0000 7fea68fc1740 1 -- 172.21.15.164:0/1538061986 >> 172.21.15.164:0/1538061986 conn(0x55a938af7310 msgr2=0x55a938b30960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:06.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.030+0000 7fea68fc1740 1 -- 172.21.15.164:0/1538061986 shutdown_connections 2024-09-19T08:53:06.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.030+0000 7fea68fc1740 1 -- 172.21.15.164:0/1538061986 wait complete. 2024-09-19T08:53:06.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.030+0000 7fea68fc1740 20 client.16965 trim_cache size 0 max 16384 2024-09-19T08:53:06.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.043+0000 7fea68fc1740 1 Processor -- start 2024-09-19T08:53:06.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.043+0000 7fea68fc1740 1 -- start start 2024-09-19T08:53:06.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.043+0000 7fea68fc1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a938b2e5d0 0x55a938b29bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:06.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.043+0000 7fea68fc1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a938b2fb40 0x55a938b2a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:06.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.043+0000 7fea68fc1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a938b39370 0x55a938b31120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:06.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.043+0000 7fea68fc1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fea601ab650 con 0x55a938b2fb40 2024-09-19T08:53:06.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.043+0000 7fea68fc1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fea601ad190 con 0x55a938b2e5d0 2024-09-19T08:53:06.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.043+0000 7fea68fc1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fea601abd20 con 0x55a938b39370 2024-09-19T08:53:06.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.044+0000 7fea5ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a938b2fb40 0x55a938b2a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:06.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.044+0000 7fea5ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a938b2fb40 0x55a938b2a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42268/0 (socket says 172.21.15.164:42268) 2024-09-19T08:53:06.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.044+0000 7fea5ffff640 1 -- 172.21.15.164:0/2525020174 learned_addr learned my addr 172.21.15.164:0/2525020174 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:06.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.044+0000 7fea64984640 1 --2- 172.21.15.164:0/2525020174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a938b2e5d0 0x55a938b29bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:06.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.044+0000 7fea5ffff640 1 -- 172.21.15.164:0/2525020174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a938b39370 msgr2=0x55a938b31120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:06.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.044+0000 7fea5ffff640 1 --2- 172.21.15.164:0/2525020174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a938b39370 0x55a938b31120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.044+0000 7fea5ffff640 1 -- 172.21.15.164:0/2525020174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a938b2e5d0 msgr2=0x55a938b29bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:06.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.044+0000 7fea5ffff640 1 --2- 172.21.15.164:0/2525020174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a938b2e5d0 0x55a938b29bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.044+0000 7fea5ffff640 1 -- 172.21.15.164:0/2525020174 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a938b2ebd0 con 0x55a938b2fb40 2024-09-19T08:53:06.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.044+0000 7fea5ffff640 1 --2- 172.21.15.164:0/2525020174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a938b2fb40 0x55a938b2a130 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fea50005fe0 tx=0x7fea50009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f52dbd8bc73b3c19 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:06.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.044+0000 7fea567fc640 1 -- 172.21.15.164:0/2525020174 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50031b80 con 0x55a938b2fb40 2024-09-19T08:53:06.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.044+0000 7fea567fc640 1 -- 172.21.15.164:0/2525020174 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea5004d070 con 0x55a938b2fb40 2024-09-19T08:53:06.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.044+0000 7fea567fc640 1 -- 172.21.15.164:0/2525020174 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50004830 con 0x55a938b2fb40 2024-09-19T08:53:06.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.045+0000 7fea68fc1740 1 -- 172.21.15.164:0/2525020174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a938b2fb40 msgr2=0x55a938b2a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:06.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.045+0000 7fea68fc1740 1 --2- 172.21.15.164:0/2525020174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a938b2fb40 0x55a938b2a130 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fea50005fe0 tx=0x7fea50009360 comp rx=0 tx=0).stop 2024-09-19T08:53:06.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.045+0000 7fea68fc1740 1 -- 172.21.15.164:0/2525020174 shutdown_connections 2024-09-19T08:53:06.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.045+0000 7fea68fc1740 1 --2- 172.21.15.164:0/2525020174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a938b39370 0x55a938b31120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.045+0000 7fea68fc1740 1 --2- 172.21.15.164:0/2525020174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a938b2fb40 0x55a938b2a130 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.045+0000 7fea68fc1740 1 --2- 172.21.15.164:0/2525020174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a938b2e5d0 0x55a938b29bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.045+0000 7fea68fc1740 1 -- 172.21.15.164:0/2525020174 >> 172.21.15.164:0/2525020174 conn(0x55a938af7310 msgr2=0x55a938b294e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:06.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.045+0000 7fea68fc1740 1 -- 172.21.15.164:0/2525020174 shutdown_connections 2024-09-19T08:53:06.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.045+0000 7fea68fc1740 1 -- 172.21.15.164:0/2525020174 wait complete. 2024-09-19T08:53:06.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.045+0000 7fea68fc1740 1 -- start start 2024-09-19T08:53:06.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.046+0000 7fea68fc1740 1 Processor -- start 2024-09-19T08:53:06.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.046+0000 7fea68fc1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a938b2e5d0 0x55a938b33210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:06.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.046+0000 7fea68fc1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a938b2fb40 0x55a938b33750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:06.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.046+0000 7fea68fc1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a938b39370 0x55a938b33c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:06.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.046+0000 7fea68fc1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fea601acac0 con 0x55a938b2e5d0 2024-09-19T08:53:06.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.046+0000 7fea68fc1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fea601ac3f0 con 0x55a938b39370 2024-09-19T08:53:06.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.046+0000 7fea68fc1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fea601a9670 con 0x55a938b2fb40 2024-09-19T08:53:06.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.046+0000 7fea68fc1740 10 client.0 _ll_register_callbacks cb 0x7fea5809b870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:06.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.046+0000 7fea64984640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a938b2e5d0 0x55a938b33210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:06.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.046+0000 7fea64984640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a938b2e5d0 0x55a938b33210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42284/0 (socket says 172.21.15.164:42284) 2024-09-19T08:53:06.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.046+0000 7fea64984640 1 -- 172.21.15.164:0/1220147897 learned_addr learned my addr 172.21.15.164:0/1220147897 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:06.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.046+0000 7fea65185640 1 --2- 172.21.15.164:0/1220147897 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a938b39370 0x55a938b33c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:06.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.046+0000 7fea5ffff640 1 --2- 172.21.15.164:0/1220147897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a938b2fb40 0x55a938b33750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:06.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.047+0000 7fea64984640 1 -- 172.21.15.164:0/1220147897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a938b2fb40 msgr2=0x55a938b33750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:06.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.047+0000 7fea64984640 1 --2- 172.21.15.164:0/1220147897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a938b2fb40 0x55a938b33750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.047+0000 7fea64984640 1 -- 172.21.15.164:0/1220147897 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a938b39370 msgr2=0x55a938b33c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:06.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.047+0000 7fea64984640 1 --2- 172.21.15.164:0/1220147897 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a938b39370 0x55a938b33c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.047+0000 7fea64984640 1 -- 172.21.15.164:0/1220147897 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a938bb7120 con 0x55a938b2e5d0 2024-09-19T08:53:06.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.047+0000 7fea65185640 1 --2- 172.21.15.164:0/1220147897 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a938b39370 0x55a938b33c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:53:06.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.047+0000 7fea64984640 1 --2- 172.21.15.164:0/1220147897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a938b2e5d0 0x55a938b33210 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fea580a48d0 tx=0x7fea5809add0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b811e2d05daf0a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:06.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.047+0000 7fea3ffff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:53:06.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.047+0000 7fea3ffff640 1 -- 172.21.15.164:0/1220147897 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5809b5e0 con 0x55a938b2e5d0 2024-09-19T08:53:06.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.047+0000 7fea3ffff640 1 -- 172.21.15.164:0/1220147897 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8455+0+0 (secure 0 0 0) 0x7fea5809aef0 con 0x55a938b2e5d0 2024-09-19T08:53:06.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.047+0000 7fea68fc1740 1 -- 172.21.15.164:0/1220147897 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea5004d070 con 0x55a938b2e5d0 2024-09-19T08:53:06.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.047+0000 7fea3ffff640 1 -- 172.21.15.164:0/1220147897 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fea580979f0 con 0x55a938b2e5d0 2024-09-19T08:53:06.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.047+0000 7fea68fc1740 10 client.16971 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:06.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.047+0000 7fea68fc1740 1 -- 172.21.15.164:0/1220147897 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a938bb83a0 con 0x55a938b2e5d0 2024-09-19T08:53:06.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.047+0000 7fea3ffff640 1 -- 172.21.15.164:0/1220147897 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea5004d070 con 0x55a938b2e5d0 2024-09-19T08:53:06.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.047+0000 7fea3ffff640 1 -- 172.21.15.164:0/1220147897 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fea580a0fe0 con 0x55a938b2e5d0 2024-09-19T08:53:06.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.047+0000 7fea68fc1740 10 client.16971 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:06.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.047+0000 7fea68fc1740 10 client.16971 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T08:53:06.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.047+0000 7fea68fc1740 20 client.16971 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:06.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.048+0000 7fea68fc1740 1 --2- 172.21.15.164:0/1220147897 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55a938bb8ac0 0x55a938bbaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:06.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.048+0000 7fea68fc1740 4 client.16971 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:53:06.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.048+0000 7fea68fc1740 1 -- 172.21.15.164:0/1220147897 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a938a9bd70 con 0x55a938bb8ac0 2024-09-19T08:53:06.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.048+0000 7fea5ffff640 1 --2- 172.21.15.164:0/1220147897 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55a938bb8ac0 0x55a938bbaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:06.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.048+0000 7fea5ffff640 1 --2- 172.21.15.164:0/1220147897 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55a938bb8ac0 0x55a938bbaeb0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=904223558b3556bb server_cookie=4cc77064402afe42 in_seq=0 out_seq=0 2024-09-19T08:53:06.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.048+0000 7fea3ffff640 10 client.16971 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:53:06.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.048+0000 7fea3ffff640 1 -- 172.21.15.164:0/1220147897 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a938a9bd70 con 0x55a938bb8ac0 2024-09-19T08:53:06.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.048+0000 7fea3ffff640 10 client.16971 handle_command_reply: tid=0 2024-09-19T08:53:06.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.048+0000 7fea68fc1740 1 client.16971 shutdown 2024-09-19T08:53:06.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.049+0000 7fea68fc1740 1 -- 172.21.15.164:0/1220147897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a938b2e5d0 msgr2=0x55a938b33210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:06.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.049+0000 7fea68fc1740 1 --2- 172.21.15.164:0/1220147897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a938b2e5d0 0x55a938b33210 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fea580a48d0 tx=0x7fea5809add0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.049+0000 7fea68fc1740 1 -- 172.21.15.164:0/1220147897 shutdown_connections 2024-09-19T08:53:06.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.049+0000 7fea68fc1740 1 --2- 172.21.15.164:0/1220147897 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55a938bb8ac0 0x55a938bbaeb0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.049+0000 7fea68fc1740 1 --2- 172.21.15.164:0/1220147897 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a938b39370 0x55a938b33c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.049+0000 7fea68fc1740 1 --2- 172.21.15.164:0/1220147897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a938b2fb40 0x55a938b33750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.049+0000 7fea3ffff640 0 client.16971 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:53:06.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.049+0000 7fea68fc1740 1 --2- 172.21.15.164:0/1220147897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a938b2e5d0 0x55a938b33210 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.049+0000 7fea68fc1740 1 -- 172.21.15.164:0/1220147897 >> 172.21.15.164:0/1220147897 conn(0x55a938af7310 msgr2=0x55a938b29d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:06.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.049+0000 7fea68fc1740 1 -- 172.21.15.164:0/1220147897 shutdown_connections 2024-09-19T08:53:06.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.049+0000 7fea68fc1740 1 -- 172.21.15.164:0/1220147897 wait complete. 2024-09-19T08:53:06.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.049+0000 7fea68fc1740 20 client.16971 trim_cache size 0 max 16384 2024-09-19T08:53:06.060 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:53:06.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.049+0000 7fea66be8640 1 -- 172.21.15.164:0/47506807 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fea4c0a33b0 msgr2=0x7fea4c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:06.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.049+0000 7fea66be8640 1 --2- 172.21.15.164:0/47506807 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fea4c0a33b0 0x7fea4c0a5870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fea4800bed0 tx=0x7fea48014840 comp rx=0 tx=0).stop 2024-09-19T08:53:06.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.049+0000 7fea66be8640 1 -- 172.21.15.164:0/47506807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea60100370 msgr2=0x7fea601984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:06.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.049+0000 7fea66be8640 1 --2- 172.21.15.164:0/47506807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea60100370 0x7fea601984e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fea5800d950 tx=0x7fea5800de10 comp rx=0 tx=0).stop 2024-09-19T08:53:06.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.050+0000 7fea66be8640 1 -- 172.21.15.164:0/47506807 shutdown_connections 2024-09-19T08:53:06.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.050+0000 7fea66be8640 1 --2- 172.21.15.164:0/47506807 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea60106fc0 0x7fea6019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.050+0000 7fea66be8640 1 --2- 172.21.15.164:0/47506807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea60100d20 0x7fea60198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.050+0000 7fea66be8640 1 --2- 172.21.15.164:0/47506807 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fea4c0a33b0 0x7fea4c0a5870 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.050+0000 7fea66be8640 1 --2- 172.21.15.164:0/47506807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea60100370 0x7fea601984e0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:06.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.050+0000 7fea66be8640 1 -- 172.21.15.164:0/47506807 >> 172.21.15.164:0/47506807 conn(0x7fea60077210 msgr2=0x7fea600fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:06.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.050+0000 7fea66be8640 1 -- 172.21.15.164:0/47506807 shutdown_connections 2024-09-19T08:53:06.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:06.050+0000 7fea66be8640 1 -- 172.21.15.164:0/47506807 wait complete. 2024-09-19T08:53:06.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:06 smithi164 ceph-mon[30928]: pgmap v223: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-19T08:53:06.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:06 smithi170 ceph-mon[35072]: pgmap v223: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-19T08:53:06.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:06 smithi173 ceph-mon[35306]: pgmap v223: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-19T08:53:06.949 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T08:53:07.199 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:53:07.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.696+0000 7f104166e640 1 -- 172.21.15.164:0/2071228635 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f103c101de0 msgr2=0x7f103c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:07.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.696+0000 7f104166e640 1 --2- 172.21.15.164:0/2071228635 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f103c101de0 0x7f103c1021c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f102c009900 tx=0x7f102c02a410 comp rx=0 tx=0).stop 2024-09-19T08:53:07.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.696+0000 7f104166e640 1 -- 172.21.15.164:0/2071228635 shutdown_connections 2024-09-19T08:53:07.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.696+0000 7f104166e640 1 --2- 172.21.15.164:0/2071228635 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f103c106f10 0x7f103c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.696+0000 7f104166e640 1 --2- 172.21.15.164:0/2071228635 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f103c102790 0x7f103c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.696+0000 7f104166e640 1 --2- 172.21.15.164:0/2071228635 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f103c101de0 0x7f103c1021c0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.696+0000 7f104166e640 1 -- 172.21.15.164:0/2071228635 >> 172.21.15.164:0/2071228635 conn(0x7f103c0fd8c0 msgr2=0x7f103c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:07.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.696+0000 7f104166e640 1 -- 172.21.15.164:0/2071228635 shutdown_connections 2024-09-19T08:53:07.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.696+0000 7f104166e640 1 -- 172.21.15.164:0/2071228635 wait complete. 2024-09-19T08:53:07.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.696+0000 7f104166e640 1 Processor -- start 2024-09-19T08:53:07.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.697+0000 7f104166e640 1 -- start start 2024-09-19T08:53:07.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.697+0000 7f104166e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f103c101de0 0x7f103c1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:07.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.697+0000 7f104166e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f103c102790 0x7f103c198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:07.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.697+0000 7f104166e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f103c106f10 0x7f103c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:07.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.697+0000 7f104166e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f103c111f70 con 0x7f103c101de0 2024-09-19T08:53:07.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.697+0000 7f104166e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f103c111df0 con 0x7f103c102790 2024-09-19T08:53:07.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.697+0000 7f104166e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f103c1120f0 con 0x7f103c106f10 2024-09-19T08:53:07.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.697+0000 7f103affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f103c101de0 0x7f103c1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:07.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.697+0000 7f103affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f103c101de0 0x7f103c1984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42296/0 (socket says 172.21.15.164:42296) 2024-09-19T08:53:07.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.697+0000 7f103affd640 1 -- 172.21.15.164:0/1765098227 learned_addr learned my addr 172.21.15.164:0/1765098227 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:07.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.697+0000 7f1033fff640 1 --2- 172.21.15.164:0/1765098227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f103c102790 0x7f103c198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:07.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.697+0000 7f103affd640 1 -- 172.21.15.164:0/1765098227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f103c106f10 msgr2=0x7f103c19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:07.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.697+0000 7f103b7fe640 1 --2- 172.21.15.164:0/1765098227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f103c106f10 0x7f103c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:07.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.697+0000 7f103affd640 1 --2- 172.21.15.164:0/1765098227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f103c106f10 0x7f103c19cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.697+0000 7f103affd640 1 -- 172.21.15.164:0/1765098227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f103c102790 msgr2=0x7f103c198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:07.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.697+0000 7f103affd640 1 --2- 172.21.15.164:0/1765098227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f103c102790 0x7f103c198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.697+0000 7f103affd640 1 -- 172.21.15.164:0/1765098227 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f103c19d480 con 0x7f103c101de0 2024-09-19T08:53:07.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.697+0000 7f1033fff640 1 --2- 172.21.15.164:0/1765098227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f103c102790 0x7f103c198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:53:07.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.697+0000 7f103affd640 1 --2- 172.21.15.164:0/1765098227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f103c101de0 0x7f103c1984d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f102c006010 tx=0x7f102c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:07.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.697+0000 7f1038ff9640 1 -- 172.21.15.164:0/1765098227 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c03f030 con 0x7f103c101de0 2024-09-19T08:53:07.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.698+0000 7f1038ff9640 1 -- 172.21.15.164:0/1765098227 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f102c003ce0 con 0x7f103c101de0 2024-09-19T08:53:07.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.698+0000 7f1038ff9640 1 -- 172.21.15.164:0/1765098227 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c035070 con 0x7f103c101de0 2024-09-19T08:53:07.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.698+0000 7f104166e640 1 -- 172.21.15.164:0/1765098227 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f103c19d710 con 0x7f103c101de0 2024-09-19T08:53:07.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.698+0000 7f104166e640 1 -- 172.21.15.164:0/1765098227 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f103c1a5050 con 0x7f103c101de0 2024-09-19T08:53:07.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.700+0000 7f1038ff9640 1 -- 172.21.15.164:0/1765098227 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f102c042030 con 0x7f103c101de0 2024-09-19T08:53:07.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.700+0000 7f1043a47740 1 Processor -- start 2024-09-19T08:53:07.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.700+0000 7f1043a47740 1 -- start start 2024-09-19T08:53:07.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.701+0000 7f1043a47740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56382a3ceb40 0x56382a3cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:07.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.701+0000 7f1043a47740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56382a3cd5d0 0x56382a3cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:07.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.701+0000 7f1043a47740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56382a3d8370 0x56382a3da760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:07.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.701+0000 7f1043a47740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56382a3332f0 con 0x56382a3ceb40 2024-09-19T08:53:07.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.701+0000 7f1043a47740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56382a2dacf0 con 0x56382a3d8370 2024-09-19T08:53:07.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.701+0000 7f1043a47740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56382a32f2b0 con 0x56382a3cd5d0 2024-09-19T08:53:07.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.701+0000 7f103b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56382a3ceb40 0x56382a3cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:07.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.701+0000 7f103b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56382a3ceb40 0x56382a3cd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42306/0 (socket says 172.21.15.164:42306) 2024-09-19T08:53:07.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.701+0000 7f103b7fe640 1 -- 172.21.15.164:0/3799562090 learned_addr learned my addr 172.21.15.164:0/3799562090 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:07.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.701+0000 7f1038ff9640 1 --2- 172.21.15.164:0/1765098227 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f10280a3430 0x7f10280a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:07.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.701+0000 7f1033fff640 1 --2- 172.21.15.164:0/3799562090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56382a3d8370 0x56382a3da760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:07.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.701+0000 7f103affd640 1 --2- 172.21.15.164:0/3799562090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56382a3cd5d0 0x56382a3cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:07.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.701+0000 7f103b7fe640 1 -- 172.21.15.164:0/3799562090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56382a3cd5d0 msgr2=0x56382a3cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:07.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.701+0000 7f103b7fe640 1 --2- 172.21.15.164:0/3799562090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56382a3cd5d0 0x56382a3cda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.701+0000 7f103b7fe640 1 -- 172.21.15.164:0/3799562090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56382a3d8370 msgr2=0x56382a3da760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:07.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.701+0000 7f103b7fe640 1 --2- 172.21.15.164:0/3799562090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56382a3d8370 0x56382a3da760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.701+0000 7f103b7fe640 1 -- 172.21.15.164:0/3799562090 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56382a2bf3a0 con 0x56382a3ceb40 2024-09-19T08:53:07.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.701+0000 7f103affd640 1 --2- 172.21.15.164:0/3799562090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56382a3cd5d0 0x56382a3cda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:53:07.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.701+0000 7f103b7fe640 1 --2- 172.21.15.164:0/1765098227 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f10280a3430 0x7f10280a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:07.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.701+0000 7f103b7fe640 1 --2- 172.21.15.164:0/3799562090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56382a3ceb40 0x56382a3cd090 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f1034009eb0 tx=0x7f103400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7d513946cc649f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:07.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.701+0000 7f1038ff9640 1 -- 172.21.15.164:0/1765098227 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8499+0+0 (secure 0 0 0) 0x7f102c0f03d0 con 0x7f103c101de0 2024-09-19T08:53:07.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.702+0000 7f10317fa640 1 -- 172.21.15.164:0/3799562090 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103401a610 con 0x56382a3ceb40 2024-09-19T08:53:07.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.702+0000 7f10317fa640 1 -- 172.21.15.164:0/3799562090 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1034012e40 con 0x56382a3ceb40 2024-09-19T08:53:07.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.702+0000 7f10317fa640 1 -- 172.21.15.164:0/3799562090 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103400d8f0 con 0x56382a3ceb40 2024-09-19T08:53:07.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.702+0000 7f1043a47740 1 -- 172.21.15.164:0/3799562090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56382a3ceb40 msgr2=0x56382a3cd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:07.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.702+0000 7f103b7fe640 1 --2- 172.21.15.164:0/1765098227 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f10280a3430 0x7f10280a58f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1034020000 tx=0x7f103401d000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:07.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.702+0000 7f1043a47740 1 --2- 172.21.15.164:0/3799562090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56382a3ceb40 0x56382a3cd090 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f1034009eb0 tx=0x7f103400f730 comp rx=0 tx=0).stop 2024-09-19T08:53:07.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.703+0000 7f1043a47740 1 -- 172.21.15.164:0/3799562090 shutdown_connections 2024-09-19T08:53:07.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.703+0000 7f1043a47740 1 --2- 172.21.15.164:0/3799562090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56382a3d8370 0x56382a3da760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.703+0000 7f1043a47740 1 --2- 172.21.15.164:0/3799562090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56382a3cd5d0 0x56382a3cda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.703+0000 7f1043a47740 1 --2- 172.21.15.164:0/3799562090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56382a3ceb40 0x56382a3cd090 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.703+0000 7f1043a47740 1 -- 172.21.15.164:0/3799562090 >> 172.21.15.164:0/3799562090 conn(0x56382a396310 msgr2=0x56382a3c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:07.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.703+0000 7f1043a47740 1 -- 172.21.15.164:0/3799562090 shutdown_connections 2024-09-19T08:53:07.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.703+0000 7f1043a47740 1 -- 172.21.15.164:0/3799562090 wait complete. 2024-09-19T08:53:07.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.703+0000 7f1043a47740 1 -- start start 2024-09-19T08:53:07.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.703+0000 7f1043a47740 1 Processor -- start 2024-09-19T08:53:07.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.703+0000 7f1043a47740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56382a3cd5d0 0x56382a3d7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:07.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.703+0000 7f1043a47740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56382a3ceb40 0x56382a3d5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:07.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.703+0000 7f1043a47740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56382a3d8370 0x56382a3d5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:07.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.703+0000 7f1043a47740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56382a3db070 con 0x56382a3d8370 2024-09-19T08:53:07.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.703+0000 7f1043a47740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56382a2ee060 con 0x56382a3ceb40 2024-09-19T08:53:07.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.703+0000 7f1043a47740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56382a3db1f0 con 0x56382a3cd5d0 2024-09-19T08:53:07.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.703+0000 7f1043a47740 10 client.0 _ll_register_callbacks cb 0x56382a0418c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:07.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.703+0000 7f103b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56382a3d8370 0x56382a3d5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:07.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.703+0000 7f103b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56382a3d8370 0x56382a3d5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42314/0 (socket says 172.21.15.164:42314) 2024-09-19T08:53:07.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.703+0000 7f103b7fe640 1 -- 172.21.15.164:0/4038136858 learned_addr learned my addr 172.21.15.164:0/4038136858 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:07.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.703+0000 7f1033fff640 1 --2- 172.21.15.164:0/4038136858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56382a3ceb40 0x56382a3d5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:07.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.703+0000 7f103affd640 1 --2- 172.21.15.164:0/4038136858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56382a3cd5d0 0x56382a3d7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:07.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.704+0000 7f103b7fe640 1 -- 172.21.15.164:0/4038136858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56382a3cd5d0 msgr2=0x56382a3d7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:07.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.704+0000 7f103b7fe640 1 --2- 172.21.15.164:0/4038136858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56382a3cd5d0 0x56382a3d7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.704+0000 7f103b7fe640 1 -- 172.21.15.164:0/4038136858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56382a3ceb40 msgr2=0x56382a3d5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:07.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.704+0000 7f103b7fe640 1 --2- 172.21.15.164:0/4038136858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56382a3ceb40 0x56382a3d5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.704+0000 7f103b7fe640 1 -- 172.21.15.164:0/4038136858 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56382a0a43c0 con 0x56382a3d8370 2024-09-19T08:53:07.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.704+0000 7f103b7fe640 1 --2- 172.21.15.164:0/4038136858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56382a3d8370 0x56382a3d5610 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f103400d8a0 tx=0x7f103400d8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a801dd4784d1c3b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:07.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.704+0000 7f10227fc640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:53:07.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.704+0000 7f10227fc640 1 -- 172.21.15.164:0/4038136858 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1034034060 con 0x56382a3d8370 2024-09-19T08:53:07.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.704+0000 7f10227fc640 1 -- 172.21.15.164:0/4038136858 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8499+0+0 (secure 0 0 0) 0x7f1034004df0 con 0x56382a3d8370 2024-09-19T08:53:07.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.704+0000 7f1043a47740 1 -- 172.21.15.164:0/4038136858 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56382a340860 con 0x56382a3d8370 2024-09-19T08:53:07.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.704+0000 7f10227fc640 1 -- 172.21.15.164:0/4038136858 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f103400fea0 con 0x56382a3d8370 2024-09-19T08:53:07.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.704+0000 7f1043a47740 10 client.16983 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:07.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.704+0000 7f1043a47740 1 -- 172.21.15.164:0/4038136858 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56382a3e6a80 con 0x56382a3d8370 2024-09-19T08:53:07.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.704+0000 7f10227fc640 1 -- 172.21.15.164:0/4038136858 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56382a340860 con 0x56382a3d8370 2024-09-19T08:53:07.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.705+0000 7f10227fc640 1 -- 172.21.15.164:0/4038136858 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f1034005050 con 0x56382a3d8370 2024-09-19T08:53:07.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.705+0000 7f1043a47740 10 client.16983 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:07.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.705+0000 7f1043a47740 10 client.16983 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:53:07.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.705+0000 7f1043a47740 20 client.16983 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:07.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.705+0000 7f1043a47740 1 --2- 172.21.15.164:0/4038136858 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56382a3e7ee0 0x56382a3ea2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:07.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.705+0000 7f1043a47740 4 client.16983 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:53:07.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.705+0000 7f1043a47740 1 -- 172.21.15.164:0/4038136858 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56382a33ad70 con 0x56382a3e7ee0 2024-09-19T08:53:07.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.705+0000 7f103affd640 1 --2- 172.21.15.164:0/4038136858 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56382a3e7ee0 0x56382a3ea2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:07.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.705+0000 7f10227fc640 10 client.16983 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:07.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.705+0000 7f103affd640 1 --2- 172.21.15.164:0/4038136858 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56382a3e7ee0 0x56382a3ea2d0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f93b1819486e590 server_cookie=d685290c2682afa6 in_seq=0 out_seq=0 2024-09-19T08:53:07.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.706+0000 7f10227fc640 1 -- 172.21.15.164:0/4038136858 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56382a33ad70 con 0x56382a3e7ee0 2024-09-19T08:53:07.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.706+0000 7f10227fc640 10 client.16983 handle_command_reply: tid=0 2024-09-19T08:53:07.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.706+0000 7f1043a47740 1 client.16983 shutdown 2024-09-19T08:53:07.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.706+0000 7f1043a47740 1 -- 172.21.15.164:0/4038136858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56382a3d8370 msgr2=0x56382a3d5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:07.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.706+0000 7f1043a47740 1 --2- 172.21.15.164:0/4038136858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56382a3d8370 0x56382a3d5610 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f103400d8a0 tx=0x7f103400d8d0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.706+0000 7f1043a47740 1 -- 172.21.15.164:0/4038136858 shutdown_connections 2024-09-19T08:53:07.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.706+0000 7f1043a47740 1 --2- 172.21.15.164:0/4038136858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56382a3d8370 0x56382a3d5610 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.706+0000 7f1043a47740 1 --2- 172.21.15.164:0/4038136858 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56382a3e7ee0 0x56382a3ea2d0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.706+0000 7f1043a47740 1 --2- 172.21.15.164:0/4038136858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56382a3ceb40 0x56382a3d5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.706+0000 7f1043a47740 1 --2- 172.21.15.164:0/4038136858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56382a3cd5d0 0x56382a3d7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.753 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:53:07.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.706+0000 7f10227fc640 0 client.16983 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:07.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.706+0000 7f1043a47740 1 -- 172.21.15.164:0/4038136858 >> 172.21.15.164:0/4038136858 conn(0x56382a396310 msgr2=0x56382a3cf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:07.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.706+0000 7f1043a47740 1 -- 172.21.15.164:0/4038136858 shutdown_connections 2024-09-19T08:53:07.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.706+0000 7f1043a47740 1 -- 172.21.15.164:0/4038136858 wait complete. 2024-09-19T08:53:07.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.706+0000 7f1043a47740 20 client.16983 trim_cache size 0 max 16384 2024-09-19T08:53:07.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.719+0000 7f1043a47740 1 Processor -- start 2024-09-19T08:53:07.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.719+0000 7f1043a47740 1 -- start start 2024-09-19T08:53:07.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.719+0000 7f1043a47740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56382a3cd5d0 0x56382a3e4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:07.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.719+0000 7f1043a47740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56382a3ceb40 0x56382a3e0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:07.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.719+0000 7f1043a47740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56382a3d8370 0x56382a3e0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:07.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.719+0000 7f1043a47740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f103c1ad890 con 0x56382a3ceb40 2024-09-19T08:53:07.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.719+0000 7f1043a47740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f103c1af3d0 con 0x56382a3cd5d0 2024-09-19T08:53:07.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.719+0000 7f1043a47740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f103c1adf60 con 0x56382a3d8370 2024-09-19T08:53:07.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.720+0000 7f1033fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56382a3ceb40 0x56382a3e0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:07.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.720+0000 7f1033fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56382a3ceb40 0x56382a3e0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42320/0 (socket says 172.21.15.164:42320) 2024-09-19T08:53:07.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.720+0000 7f1033fff640 1 -- 172.21.15.164:0/2888517119 learned_addr learned my addr 172.21.15.164:0/2888517119 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:07.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.720+0000 7f103b7fe640 1 --2- 172.21.15.164:0/2888517119 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56382a3d8370 0x56382a3e0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:07.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.720+0000 7f103affd640 1 --2- 172.21.15.164:0/2888517119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56382a3cd5d0 0x56382a3e4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:07.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.720+0000 7f1033fff640 1 -- 172.21.15.164:0/2888517119 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56382a3d8370 msgr2=0x56382a3e0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:07.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.720+0000 7f1033fff640 1 --2- 172.21.15.164:0/2888517119 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56382a3d8370 0x56382a3e0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.720+0000 7f1033fff640 1 -- 172.21.15.164:0/2888517119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56382a3cd5d0 msgr2=0x56382a3e4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:07.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.720+0000 7f1033fff640 1 --2- 172.21.15.164:0/2888517119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56382a3cd5d0 0x56382a3e4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.720+0000 7f1033fff640 1 -- 172.21.15.164:0/2888517119 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56382a3ca960 con 0x56382a3ceb40 2024-09-19T08:53:07.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.720+0000 7f1033fff640 1 --2- 172.21.15.164:0/2888517119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56382a3ceb40 0x56382a3e0860 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f1024005fd0 tx=0x7f1024012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d35cd7ad9b4ac5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:07.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.720+0000 7f1030ff9640 1 -- 172.21.15.164:0/2888517119 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102400bd90 con 0x56382a3ceb40 2024-09-19T08:53:07.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.720+0000 7f1030ff9640 1 -- 172.21.15.164:0/2888517119 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1024005500 con 0x56382a3ceb40 2024-09-19T08:53:07.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.720+0000 7f1030ff9640 1 -- 172.21.15.164:0/2888517119 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102401c6b0 con 0x56382a3ceb40 2024-09-19T08:53:07.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.721+0000 7f1043a47740 1 -- 172.21.15.164:0/2888517119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56382a3ceb40 msgr2=0x56382a3e0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:07.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.721+0000 7f1043a47740 1 --2- 172.21.15.164:0/2888517119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56382a3ceb40 0x56382a3e0860 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f1024005fd0 tx=0x7f1024012820 comp rx=0 tx=0).stop 2024-09-19T08:53:07.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.721+0000 7f1043a47740 1 -- 172.21.15.164:0/2888517119 shutdown_connections 2024-09-19T08:53:07.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.721+0000 7f1043a47740 1 --2- 172.21.15.164:0/2888517119 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56382a3d8370 0x56382a3e0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.721+0000 7f1043a47740 1 --2- 172.21.15.164:0/2888517119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56382a3ceb40 0x56382a3e0860 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.721+0000 7f1043a47740 1 --2- 172.21.15.164:0/2888517119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56382a3cd5d0 0x56382a3e4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.721+0000 7f1043a47740 1 -- 172.21.15.164:0/2888517119 >> 172.21.15.164:0/2888517119 conn(0x56382a396310 msgr2=0x56382a3d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:07.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.721+0000 7f1043a47740 1 -- 172.21.15.164:0/2888517119 shutdown_connections 2024-09-19T08:53:07.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.721+0000 7f1043a47740 1 -- 172.21.15.164:0/2888517119 wait complete. 2024-09-19T08:53:07.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.721+0000 7f1043a47740 1 -- start start 2024-09-19T08:53:07.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.721+0000 7f1043a47740 1 Processor -- start 2024-09-19T08:53:07.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.721+0000 7f1043a47740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56382a3cd5d0 0x56382a3eef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:07.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.721+0000 7f1043a47740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56382a3ceb40 0x56382a3ef450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:07.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.721+0000 7f1043a47740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56382a3d8370 0x56382a454930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:07.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.721+0000 7f1043a47740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f103c1aed00 con 0x56382a3cd5d0 2024-09-19T08:53:07.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.721+0000 7f1043a47740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f103c1ae630 con 0x56382a3ceb40 2024-09-19T08:53:07.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.721+0000 7f1043a47740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f103c1ab890 con 0x56382a3d8370 2024-09-19T08:53:07.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.722+0000 7f1043a47740 10 client.0 _ll_register_callbacks cb 0x7f103401af50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:07.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.722+0000 7f103affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56382a3cd5d0 0x56382a3eef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:07.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.722+0000 7f103affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56382a3cd5d0 0x56382a3eef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42324/0 (socket says 172.21.15.164:42324) 2024-09-19T08:53:07.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.722+0000 7f103affd640 1 -- 172.21.15.164:0/645534160 learned_addr learned my addr 172.21.15.164:0/645534160 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:07.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.722+0000 7f1033fff640 1 --2- 172.21.15.164:0/645534160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56382a3ceb40 0x56382a3ef450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:07.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.722+0000 7f103b7fe640 1 --2- 172.21.15.164:0/645534160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56382a3d8370 0x56382a454930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:07.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.722+0000 7f103affd640 1 -- 172.21.15.164:0/645534160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56382a3d8370 msgr2=0x56382a454930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:07.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.722+0000 7f103affd640 1 --2- 172.21.15.164:0/645534160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56382a3d8370 0x56382a454930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.722+0000 7f103affd640 1 -- 172.21.15.164:0/645534160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56382a3ceb40 msgr2=0x56382a3ef450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:07.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.722+0000 7f103affd640 1 --2- 172.21.15.164:0/645534160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56382a3ceb40 0x56382a3ef450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.722+0000 7f103affd640 1 -- 172.21.15.164:0/645534160 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56382a454e70 con 0x56382a3cd5d0 2024-09-19T08:53:07.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.722+0000 7f103b7fe640 1 --2- 172.21.15.164:0/645534160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56382a3d8370 0x56382a454930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:53:07.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.722+0000 7f1033fff640 1 --2- 172.21.15.164:0/645534160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56382a3ceb40 0x56382a3ef450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:53:07.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.722+0000 7f103affd640 1 --2- 172.21.15.164:0/645534160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56382a3cd5d0 0x56382a3eef10 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f102c004650 tx=0x7f102c08c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=150085efd6b0de92 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:07.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.722+0000 7f10227fc640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:53:07.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.722+0000 7f10227fc640 1 -- 172.21.15.164:0/645534160 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c049040 con 0x56382a3cd5d0 2024-09-19T08:53:07.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.722+0000 7f10227fc640 1 -- 172.21.15.164:0/645534160 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8499+0+0 (secure 0 0 0) 0x7f102c08b250 con 0x56382a3cd5d0 2024-09-19T08:53:07.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.722+0000 7f1043a47740 1 -- 172.21.15.164:0/645534160 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1024005500 con 0x56382a3cd5d0 2024-09-19T08:53:07.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.722+0000 7f10227fc640 1 -- 172.21.15.164:0/645534160 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f102c03b580 con 0x56382a3cd5d0 2024-09-19T08:53:07.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.722+0000 7f1043a47740 10 client.16989 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:07.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.722+0000 7f1043a47740 1 -- 172.21.15.164:0/645534160 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56382a456150 con 0x56382a3cd5d0 2024-09-19T08:53:07.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.723+0000 7f10227fc640 1 -- 172.21.15.164:0/645534160 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1024005500 con 0x56382a3cd5d0 2024-09-19T08:53:07.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.723+0000 7f10227fc640 1 -- 172.21.15.164:0/645534160 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f102c0edd50 con 0x56382a3cd5d0 2024-09-19T08:53:07.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.723+0000 7f1043a47740 10 client.16989 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:07.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.723+0000 7f1043a47740 10 client.16989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:53:07.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.723+0000 7f1043a47740 20 client.16989 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:07.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.723+0000 7f1043a47740 1 --2- 172.21.15.164:0/645534160 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56382a456670 0x56382a458ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:07.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.723+0000 7f1043a47740 4 client.16989 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:53:07.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.723+0000 7f1043a47740 1 -- 172.21.15.164:0/645534160 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56382a2b7170 con 0x56382a456670 2024-09-19T08:53:07.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.723+0000 7f1033fff640 1 --2- 172.21.15.164:0/645534160 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56382a456670 0x56382a458ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:07.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.723+0000 7f10227fc640 10 client.16989 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:07.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.723+0000 7f1033fff640 1 --2- 172.21.15.164:0/645534160 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56382a456670 0x56382a458ad0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=175d5041810b110d server_cookie=dc11c4ac34ca9197 in_seq=0 out_seq=0 2024-09-19T08:53:07.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.724+0000 7f10227fc640 1 -- 172.21.15.164:0/645534160 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56382a2b7170 con 0x56382a456670 2024-09-19T08:53:07.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.724+0000 7f10227fc640 10 client.16989 handle_command_reply: tid=0 2024-09-19T08:53:07.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.724+0000 7f1043a47740 1 client.16989 shutdown 2024-09-19T08:53:07.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.724+0000 7f1043a47740 1 -- 172.21.15.164:0/645534160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56382a3cd5d0 msgr2=0x56382a3eef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:07.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.724+0000 7f1043a47740 1 --2- 172.21.15.164:0/645534160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56382a3cd5d0 0x56382a3eef10 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f102c004650 tx=0x7f102c08c400 comp rx=0 tx=0).stop 2024-09-19T08:53:07.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.724+0000 7f1043a47740 1 -- 172.21.15.164:0/645534160 shutdown_connections 2024-09-19T08:53:07.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.724+0000 7f1043a47740 1 --2- 172.21.15.164:0/645534160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56382a3d8370 0x56382a454930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.724+0000 7f1043a47740 1 --2- 172.21.15.164:0/645534160 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56382a456670 0x56382a458ad0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.724+0000 7f1043a47740 1 --2- 172.21.15.164:0/645534160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56382a3ceb40 0x56382a3ef450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.724+0000 7f10227fc640 0 client.16989 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:07.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.724+0000 7f1043a47740 1 --2- 172.21.15.164:0/645534160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56382a3cd5d0 0x56382a3eef10 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.724+0000 7f1043a47740 1 -- 172.21.15.164:0/645534160 >> 172.21.15.164:0/645534160 conn(0x56382a396310 msgr2=0x56382a3d38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:07.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.724+0000 7f1043a47740 1 -- 172.21.15.164:0/645534160 shutdown_connections 2024-09-19T08:53:07.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.724+0000 7f1043a47740 1 -- 172.21.15.164:0/645534160 wait complete. 2024-09-19T08:53:07.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.724+0000 7f1043a47740 20 client.16989 trim_cache size 0 max 16384 2024-09-19T08:53:07.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.724+0000 7f104166e640 1 -- 172.21.15.164:0/1765098227 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f10280a3430 msgr2=0x7f10280a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:07.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.724+0000 7f104166e640 1 --2- 172.21.15.164:0/1765098227 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f10280a3430 0x7f10280a58f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1034020000 tx=0x7f103401d000 comp rx=0 tx=0).stop 2024-09-19T08:53:07.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.724+0000 7f104166e640 1 -- 172.21.15.164:0/1765098227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f103c101de0 msgr2=0x7f103c1984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:07.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.724+0000 7f104166e640 1 --2- 172.21.15.164:0/1765098227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f103c101de0 0x7f103c1984d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f102c006010 tx=0x7f102c0042f0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.725+0000 7f104166e640 1 -- 172.21.15.164:0/1765098227 shutdown_connections 2024-09-19T08:53:07.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.725+0000 7f104166e640 1 --2- 172.21.15.164:0/1765098227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f103c106f10 0x7f103c19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.725+0000 7f104166e640 1 --2- 172.21.15.164:0/1765098227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f103c102790 0x7f103c198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.725+0000 7f104166e640 1 --2- 172.21.15.164:0/1765098227 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f10280a3430 0x7f10280a58f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.725+0000 7f104166e640 1 --2- 172.21.15.164:0/1765098227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f103c101de0 0x7f103c1984d0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:07.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.725+0000 7f104166e640 1 -- 172.21.15.164:0/1765098227 >> 172.21.15.164:0/1765098227 conn(0x7f103c0fd8c0 msgr2=0x7f103c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:07.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.725+0000 7f104166e640 1 -- 172.21.15.164:0/1765098227 shutdown_connections 2024-09-19T08:53:07.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:07.725+0000 7f104166e640 1 -- 172.21.15.164:0/1765098227 wait complete. 2024-09-19T08:53:08.469 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:08 smithi164 ceph-mon[30928]: pgmap v224: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 28 op/s 2024-09-19T08:53:08.469 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:08 smithi164 ceph-mon[30928]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T08:53:08.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:08 smithi170 ceph-mon[35072]: pgmap v224: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 28 op/s 2024-09-19T08:53:08.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:08 smithi170 ceph-mon[35072]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T08:53:08.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:08 smithi173 ceph-mon[35306]: pgmap v224: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 28 op/s 2024-09-19T08:53:08.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:08 smithi173 ceph-mon[35306]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T08:53:09.497 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T08:53:09.498 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T08:53:09.772 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:53:10.312 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:10 smithi164 ceph-mon[30928]: pgmap v226: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T08:53:10.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.303+0000 7f74c01be640 1 -- 172.21.15.164:0/2398198613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74b8100e20 msgr2=0x7f74b8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:10.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.303+0000 7f74c01be640 1 --2- 172.21.15.164:0/2398198613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74b8100e20 0x7f74b8101200 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f74b0009900 tx=0x7f74b002a480 comp rx=0 tx=0).stop 2024-09-19T08:53:10.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.303+0000 7f74c01be640 1 -- 172.21.15.164:0/2398198613 shutdown_connections 2024-09-19T08:53:10.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.303+0000 7f74c01be640 1 --2- 172.21.15.164:0/2398198613 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74b8102ef0 0x7f74b810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:10.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.303+0000 7f74c01be640 1 --2- 172.21.15.164:0/2398198613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74b8102570 0x7f74b81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:10.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.303+0000 7f74c01be640 1 --2- 172.21.15.164:0/2398198613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74b8100e20 0x7f74b8101200 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:10.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.303+0000 7f74c01be640 1 -- 172.21.15.164:0/2398198613 >> 172.21.15.164:0/2398198613 conn(0x7f74b80fc7d0 msgr2=0x7f74b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:10.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.303+0000 7f74c01be640 1 -- 172.21.15.164:0/2398198613 shutdown_connections 2024-09-19T08:53:10.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.303+0000 7f74c01be640 1 -- 172.21.15.164:0/2398198613 wait complete. 2024-09-19T08:53:10.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.304+0000 7f74c01be640 1 Processor -- start 2024-09-19T08:53:10.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.304+0000 7f74c01be640 1 -- start start 2024-09-19T08:53:10.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.304+0000 7f74c01be640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74b8100e20 0x7f74b8199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:10.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.304+0000 7f74c01be640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74b8102570 0x7f74b8199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:10.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.304+0000 7f74c01be640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74b8102ef0 0x7f74b81a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:10.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.304+0000 7f74c01be640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f74b8110fa0 con 0x7f74b8102570 2024-09-19T08:53:10.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.304+0000 7f74c01be640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f74b8110e20 con 0x7f74b8100e20 2024-09-19T08:53:10.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.304+0000 7f74c01be640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f74b8111120 con 0x7f74b8102ef0 2024-09-19T08:53:10.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.304+0000 7f74bd759640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74b8102570 0x7f74b8199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:10.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.304+0000 7f74bd759640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74b8102570 0x7f74b8199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42338/0 (socket says 172.21.15.164:42338) 2024-09-19T08:53:10.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.304+0000 7f74bd759640 1 -- 172.21.15.164:0/182691361 learned_addr learned my addr 172.21.15.164:0/182691361 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:10.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.304+0000 7f74be75b640 1 --2- 172.21.15.164:0/182691361 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74b8102ef0 0x7f74b81a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:10.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.304+0000 7f74bdf5a640 1 --2- 172.21.15.164:0/182691361 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74b8100e20 0x7f74b8199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:10.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.304+0000 7f74bd759640 1 -- 172.21.15.164:0/182691361 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74b8102ef0 msgr2=0x7f74b81a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:10.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.304+0000 7f74bd759640 1 --2- 172.21.15.164:0/182691361 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74b8102ef0 0x7f74b81a43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:10.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.304+0000 7f74bd759640 1 -- 172.21.15.164:0/182691361 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74b8100e20 msgr2=0x7f74b8199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:10.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.304+0000 7f74bd759640 1 --2- 172.21.15.164:0/182691361 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74b8100e20 0x7f74b8199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:10.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.304+0000 7f74bd759640 1 -- 172.21.15.164:0/182691361 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74b81a48f0 con 0x7f74b8102570 2024-09-19T08:53:10.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.304+0000 7f74bdf5a640 1 --2- 172.21.15.164:0/182691361 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74b8100e20 0x7f74b8199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:53:10.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.305+0000 7f74bd759640 1 --2- 172.21.15.164:0/182691361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74b8102570 0x7f74b8199870 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00eab0 tx=0x7f74ac00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:10.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.305+0000 7f74b6ffd640 1 -- 172.21.15.164:0/182691361 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac00ce40 con 0x7f74b8102570 2024-09-19T08:53:10.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.305+0000 7f74c01be640 1 -- 172.21.15.164:0/182691361 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74b81a4ac0 con 0x7f74b8102570 2024-09-19T08:53:10.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.305+0000 7f74b6ffd640 1 -- 172.21.15.164:0/182691361 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74ac011950 con 0x7f74b8102570 2024-09-19T08:53:10.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.305+0000 7f74b6ffd640 1 -- 172.21.15.164:0/182691361 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac010ae0 con 0x7f74b8102570 2024-09-19T08:53:10.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.305+0000 7f74c01be640 1 -- 172.21.15.164:0/182691361 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74b81a5050 con 0x7f74b8102570 2024-09-19T08:53:10.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.307+0000 7f74b6ffd640 1 -- 172.21.15.164:0/182691361 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f74ac011b00 con 0x7f74b8102570 2024-09-19T08:53:10.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.308+0000 7f74c2597740 1 Processor -- start 2024-09-19T08:53:10.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.308+0000 7f74c2597740 1 -- start start 2024-09-19T08:53:10.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.308+0000 7f74b6ffd640 1 --2- 172.21.15.164:0/182691361 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f74a00a33b0 0x7f74a00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:10.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.308+0000 7f74c2597740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647eccb6b60 0x5647eccb6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:10.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.308+0000 7f74c2597740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647eccb7480 0x5647eccc1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:10.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.308+0000 7f74c2597740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647eccc1d40 0x5647eccc4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:10.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.308+0000 7f74c2597740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5647ecc1c280 con 0x5647eccb6b60 2024-09-19T08:53:10.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.308+0000 7f74c2597740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5647ecbc3b10 con 0x5647eccc1d40 2024-09-19T08:53:10.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.308+0000 7f74c2597740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5647ecc18240 con 0x5647eccb7480 2024-09-19T08:53:10.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.308+0000 7f74bdf5a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647eccb6b60 0x5647eccb6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:10.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.308+0000 7f74b6ffd640 1 -- 172.21.15.164:0/182691361 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8499+0+0 (secure 0 0 0) 0x7f74ac0cfcf0 con 0x7f74b8102570 2024-09-19T08:53:10.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.308+0000 7f74bdf5a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647eccb6b60 0x5647eccb6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42352/0 (socket says 172.21.15.164:42352) 2024-09-19T08:53:10.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.308+0000 7f74bdf5a640 1 -- 172.21.15.164:0/1920975453 learned_addr learned my addr 172.21.15.164:0/1920975453 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:10.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.308+0000 7f74be75b640 1 --2- 172.21.15.164:0/182691361 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f74a00a33b0 0x7f74a00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:10.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.309+0000 7f74bd759640 1 --2- 172.21.15.164:0/1920975453 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647eccb7480 0x5647eccc1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:10.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.309+0000 7f74bdf5a640 1 -- 172.21.15.164:0/1920975453 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647eccb7480 msgr2=0x5647eccc1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:10.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.309+0000 7f74bdf5a640 1 --2- 172.21.15.164:0/1920975453 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647eccb7480 0x5647eccc1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:10.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.309+0000 7f74bdf5a640 1 -- 172.21.15.164:0/1920975453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647eccc1d40 msgr2=0x5647eccc4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:10.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.309+0000 7f74bdf5a640 1 --2- 172.21.15.164:0/1920975453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647eccc1d40 0x5647eccc4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:10.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.309+0000 7f74bdf5a640 1 -- 172.21.15.164:0/1920975453 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647ecbd01f0 con 0x5647eccb6b60 2024-09-19T08:53:10.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.309+0000 7f74bd759640 1 --2- 172.21.15.164:0/1920975453 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647eccb7480 0x5647eccc1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:53:10.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.309+0000 7f74bdf5a640 1 --2- 172.21.15.164:0/1920975453 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647eccb6b60 0x5647eccb6f40 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f74b00314e0 tx=0x7f74b0004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b88cbf75deab849a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:10.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.309+0000 7f74be75b640 1 --2- 172.21.15.164:0/182691361 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f74a00a33b0 0x7f74a00a5870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x5647eccb4470 tx=0x7f74a8009490 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:10.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.309+0000 7f749ffff640 1 -- 172.21.15.164:0/1920975453 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b0035030 con 0x5647eccb6b60 2024-09-19T08:53:10.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.309+0000 7f749ffff640 1 -- 172.21.15.164:0/1920975453 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74b0005520 con 0x5647eccb6b60 2024-09-19T08:53:10.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.309+0000 7f749ffff640 1 -- 172.21.15.164:0/1920975453 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b0003850 con 0x5647eccb6b60 2024-09-19T08:53:10.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.310+0000 7f74c2597740 1 -- 172.21.15.164:0/1920975453 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647eccb6b60 msgr2=0x5647eccb6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:10.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.310+0000 7f74c2597740 1 --2- 172.21.15.164:0/1920975453 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647eccb6b60 0x5647eccb6f40 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f74b00314e0 tx=0x7f74b0004200 comp rx=0 tx=0).stop 2024-09-19T08:53:10.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.310+0000 7f74c2597740 1 -- 172.21.15.164:0/1920975453 shutdown_connections 2024-09-19T08:53:10.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.310+0000 7f74c2597740 1 --2- 172.21.15.164:0/1920975453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647eccc1d40 0x5647eccc4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:10.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.310+0000 7f74c2597740 1 --2- 172.21.15.164:0/1920975453 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647eccb7480 0x5647eccc1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:10.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.310+0000 7f74c2597740 1 --2- 172.21.15.164:0/1920975453 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647eccb6b60 0x5647eccb6f40 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:10.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.310+0000 7f74c2597740 1 -- 172.21.15.164:0/1920975453 >> 172.21.15.164:0/1920975453 conn(0x5647eccb30f0 msgr2=0x5647eccb34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:10.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.310+0000 7f74c2597740 1 -- 172.21.15.164:0/1920975453 shutdown_connections 2024-09-19T08:53:10.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.310+0000 7f74c2597740 1 -- 172.21.15.164:0/1920975453 wait complete. 2024-09-19T08:53:10.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.310+0000 7f74c2597740 1 -- start start 2024-09-19T08:53:10.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.310+0000 7f74c2597740 1 Processor -- start 2024-09-19T08:53:10.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.311+0000 7f74c2597740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647eccb6b60 0x5647eccbebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:10.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.311+0000 7f74c2597740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647eccb7480 0x5647eccbf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:10.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.311+0000 7f74c2597740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647eccc1d40 0x5647eccbf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:10.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.311+0000 7f74c2597740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5647eccc4910 con 0x5647eccb7480 2024-09-19T08:53:10.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.311+0000 7f74c2597740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5647ecbd6f10 con 0x5647eccc1d40 2024-09-19T08:53:10.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.311+0000 7f74c2597740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5647eccc4a90 con 0x5647eccb6b60 2024-09-19T08:53:10.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.311+0000 7f74c2597740 10 client.0 _ll_register_callbacks cb 0x5647ecb9fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:10.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.311+0000 7f74bd759640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647eccb7480 0x5647eccbf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:10.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.311+0000 7f74bd759640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647eccb7480 0x5647eccbf100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42360/0 (socket says 172.21.15.164:42360) 2024-09-19T08:53:10.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.311+0000 7f74bd759640 1 -- 172.21.15.164:0/3348319952 learned_addr learned my addr 172.21.15.164:0/3348319952 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:10.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.311+0000 7f74be75b640 1 --2- 172.21.15.164:0/3348319952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647eccc1d40 0x5647eccbf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:10.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.311+0000 7f74bdf5a640 1 --2- 172.21.15.164:0/3348319952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647eccb6b60 0x5647eccbebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:10.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.311+0000 7f74bd759640 1 -- 172.21.15.164:0/3348319952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647eccb6b60 msgr2=0x5647eccbebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:10.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.311+0000 7f74bd759640 1 --2- 172.21.15.164:0/3348319952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647eccb6b60 0x5647eccbebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:10.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.311+0000 7f74bd759640 1 -- 172.21.15.164:0/3348319952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647eccc1d40 msgr2=0x5647eccbf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:10.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.311+0000 7f74bd759640 1 --2- 172.21.15.164:0/3348319952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647eccc1d40 0x5647eccbf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:10.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.311+0000 7f74bd759640 1 -- 172.21.15.164:0/3348319952 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647ec98d6a0 con 0x5647eccb7480 2024-09-19T08:53:10.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.311+0000 7f74bd759640 1 --2- 172.21.15.164:0/3348319952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647eccb7480 0x5647eccbf100 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00c700 tx=0x7f74ac0bc190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08bad8982314f1f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:10.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.311+0000 7f749d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:53:10.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.312+0000 7f749d7fa640 1 -- 172.21.15.164:0/3348319952 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac0a4bf0 con 0x5647eccb7480 2024-09-19T08:53:10.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.312+0000 7f749d7fa640 1 -- 172.21.15.164:0/3348319952 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8499+0+0 (secure 0 0 0) 0x7f74ac0cd670 con 0x5647eccb7480 2024-09-19T08:53:10.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.312+0000 7f74c2597740 1 -- 172.21.15.164:0/3348319952 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74b0005520 con 0x5647eccb7480 2024-09-19T08:53:10.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.312+0000 7f749d7fa640 1 -- 172.21.15.164:0/3348319952 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f74ac0513d0 con 0x5647eccb7480 2024-09-19T08:53:10.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.312+0000 7f74c2597740 10 client.17001 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:10.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.312+0000 7f74c2597740 1 -- 172.21.15.164:0/3348319952 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5647eccd0460 con 0x5647eccb7480 2024-09-19T08:53:10.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.312+0000 7f749d7fa640 1 -- 172.21.15.164:0/3348319952 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74b0005520 con 0x5647eccb7480 2024-09-19T08:53:10.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.312+0000 7f749d7fa640 1 -- 172.21.15.164:0/3348319952 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f74ac00f040 con 0x5647eccb7480 2024-09-19T08:53:10.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.312+0000 7f74c2597740 10 client.17001 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:10.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.312+0000 7f74c2597740 10 client.17001 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:53:10.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.312+0000 7f74c2597740 20 client.17001 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:10.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.312+0000 7f74c2597740 1 --2- 172.21.15.164:0/3348319952 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5647eccd17b0 0x5647eccd3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:10.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.312+0000 7f74c2597740 4 client.17001 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:53:10.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.312+0000 7f74c2597740 1 -- 172.21.15.164:0/3348319952 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647ecc297f0 con 0x5647eccd17b0 2024-09-19T08:53:10.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.313+0000 7f74bdf5a640 1 --2- 172.21.15.164:0/3348319952 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5647eccd17b0 0x5647eccd3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:10.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.313+0000 7f74bdf5a640 1 --2- 172.21.15.164:0/3348319952 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5647eccd17b0 0x5647eccd3c40 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb9ca86f457d40e6 server_cookie=4c629a8d89986a5a in_seq=0 out_seq=0 2024-09-19T08:53:10.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.313+0000 7f749d7fa640 10 client.17001 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:10.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.314+0000 7f749d7fa640 1 -- 172.21.15.164:0/3348319952 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5647ecc297f0 con 0x5647eccd17b0 2024-09-19T08:53:10.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.314+0000 7f749d7fa640 10 client.17001 handle_command_reply: tid=0 2024-09-19T08:53:10.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.314+0000 7f74c2597740 1 client.17001 shutdown 2024-09-19T08:53:10.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.314+0000 7f74c2597740 1 -- 172.21.15.164:0/3348319952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647eccb7480 msgr2=0x5647eccbf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:10.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.314+0000 7f74c2597740 1 --2- 172.21.15.164:0/3348319952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647eccb7480 0x5647eccbf100 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00c700 tx=0x7f74ac0bc190 comp rx=0 tx=0).stop 2024-09-19T08:53:10.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.314+0000 7f74c2597740 1 -- 172.21.15.164:0/3348319952 shutdown_connections 2024-09-19T08:53:10.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.314+0000 7f74c2597740 1 --2- 172.21.15.164:0/3348319952 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5647eccd17b0 0x5647eccd3c40 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:10.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.314+0000 7f74c2597740 1 --2- 172.21.15.164:0/3348319952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647eccc1d40 0x5647eccbf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:10.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.314+0000 7f74c2597740 1 --2- 172.21.15.164:0/3348319952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647eccb7480 0x5647eccbf100 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:10.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.314+0000 7f74c2597740 1 --2- 172.21.15.164:0/3348319952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647eccb6b60 0x5647eccbebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:10.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.314+0000 7f749d7fa640 0 client.17001 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:10.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.314+0000 7f74c2597740 1 -- 172.21.15.164:0/3348319952 >> 172.21.15.164:0/3348319952 conn(0x5647eccb30f0 msgr2=0x5647eccb8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:10.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.314+0000 7f74c2597740 1 -- 172.21.15.164:0/3348319952 shutdown_connections 2024-09-19T08:53:10.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.314+0000 7f74c2597740 1 -- 172.21.15.164:0/3348319952 wait complete. 2024-09-19T08:53:10.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.314+0000 7f74c2597740 20 client.17001 trim_cache size 0 max 16384 2024-09-19T08:53:10.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.331+0000 7f74c2597740 1 Processor -- start 2024-09-19T08:53:10.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.331+0000 7f74c2597740 1 -- start start 2024-09-19T08:53:10.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.332+0000 7f74c2597740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647eccb6b60 0x5647eccb8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:10.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.332+0000 7f74c2597740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647eccb7480 0x5647eccd9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:10.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.332+0000 7f74c2597740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647eccc1d40 0x5647eccd9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:10.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.332+0000 7f74c2597740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f74b81ad680 con 0x5647eccc1d40 2024-09-19T08:53:10.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.332+0000 7f74c2597740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f74b81af1c0 con 0x5647eccb6b60 2024-09-19T08:53:10.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.332+0000 7f74c2597740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f74b81add50 con 0x5647eccb7480 2024-09-19T08:53:10.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.332+0000 7f74be75b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647eccc1d40 0x5647eccd9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:10.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.332+0000 7f74be75b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647eccc1d40 0x5647eccd9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42362/0 (socket says 172.21.15.164:42362) 2024-09-19T08:53:10.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.332+0000 7f74be75b640 1 -- 172.21.15.164:0/2015730033 learned_addr learned my addr 172.21.15.164:0/2015730033 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:10.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.332+0000 7f74bdf5a640 1 --2- 172.21.15.164:0/2015730033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647eccb6b60 0x5647eccb8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:10.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.332+0000 7f74bd759640 1 --2- 172.21.15.164:0/2015730033 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647eccb7480 0x5647eccd9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:10.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.332+0000 7f74be75b640 1 -- 172.21.15.164:0/2015730033 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647eccb7480 msgr2=0x5647eccd9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:10.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.332+0000 7f74be75b640 1 --2- 172.21.15.164:0/2015730033 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647eccb7480 0x5647eccd9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:10.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.332+0000 7f74be75b640 1 -- 172.21.15.164:0/2015730033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647eccb6b60 msgr2=0x5647eccb8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:10.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.332+0000 7f74be75b640 1 --2- 172.21.15.164:0/2015730033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647eccb6b60 0x5647eccb8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:10.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.332+0000 7f74be75b640 1 -- 172.21.15.164:0/2015730033 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647eccb2f20 con 0x5647eccc1d40 2024-09-19T08:53:10.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.332+0000 7f74bdf5a640 1 --2- 172.21.15.164:0/2015730033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647eccb6b60 0x5647eccb8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:53:10.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.333+0000 7f74be75b640 1 --2- 172.21.15.164:0/2015730033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647eccc1d40 0x5647eccd9aa0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f74a8010a40 tx=0x7f74a8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb3fdee66f9d821 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:10.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.333+0000 7f749f7fe640 1 -- 172.21.15.164:0/2015730033 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a8026070 con 0x5647eccc1d40 2024-09-19T08:53:10.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.333+0000 7f749f7fe640 1 -- 172.21.15.164:0/2015730033 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74a8016c70 con 0x5647eccc1d40 2024-09-19T08:53:10.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.333+0000 7f749f7fe640 1 -- 172.21.15.164:0/2015730033 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a801c040 con 0x5647eccc1d40 2024-09-19T08:53:10.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.333+0000 7f74c2597740 1 -- 172.21.15.164:0/2015730033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647eccc1d40 msgr2=0x5647eccd9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:10.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.333+0000 7f74c2597740 1 --2- 172.21.15.164:0/2015730033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647eccc1d40 0x5647eccd9aa0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f74a8010a40 tx=0x7f74a8012000 comp rx=0 tx=0).stop 2024-09-19T08:53:10.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.334+0000 7f74c2597740 1 -- 172.21.15.164:0/2015730033 shutdown_connections 2024-09-19T08:53:10.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.334+0000 7f74c2597740 1 --2- 172.21.15.164:0/2015730033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647eccc1d40 0x5647eccd9aa0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:10.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.334+0000 7f74c2597740 1 --2- 172.21.15.164:0/2015730033 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647eccb7480 0x5647eccd9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:10.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.334+0000 7f74c2597740 1 --2- 172.21.15.164:0/2015730033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647eccb6b60 0x5647eccb8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:10.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.334+0000 7f74c2597740 1 -- 172.21.15.164:0/2015730033 >> 172.21.15.164:0/2015730033 conn(0x5647eccb30f0 msgr2=0x5647eccb85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:10.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.334+0000 7f74c2597740 1 -- 172.21.15.164:0/2015730033 shutdown_connections 2024-09-19T08:53:10.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.334+0000 7f74c2597740 1 -- 172.21.15.164:0/2015730033 wait complete. 2024-09-19T08:53:10.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.334+0000 7f74c2597740 1 -- start start 2024-09-19T08:53:10.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.334+0000 7f74c2597740 1 Processor -- start 2024-09-19T08:53:10.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.334+0000 7f74c2597740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647eccb6b60 0x5647eccc9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:10.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.334+0000 7f74c2597740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647eccb7480 0x5647eccca250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:10.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.334+0000 7f74c2597740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647eccc1d40 0x5647eccc6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:10.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.334+0000 7f74c2597740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f74b81aeaf0 con 0x5647eccb7480 2024-09-19T08:53:10.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.334+0000 7f74c2597740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f74b81ae420 con 0x5647eccc1d40 2024-09-19T08:53:10.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.334+0000 7f74c2597740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f74b81ab680 con 0x5647eccb6b60 2024-09-19T08:53:10.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.334+0000 7f74c2597740 10 client.0 _ll_register_callbacks cb 0x7f74ac0103d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:10.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.335+0000 7f74bd759640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647eccb7480 0x5647eccca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:10.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.335+0000 7f74bd759640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647eccb7480 0x5647eccca250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42370/0 (socket says 172.21.15.164:42370) 2024-09-19T08:53:10.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.335+0000 7f74bd759640 1 -- 172.21.15.164:0/3718392456 learned_addr learned my addr 172.21.15.164:0/3718392456 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:10.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.335+0000 7f74be75b640 1 --2- 172.21.15.164:0/3718392456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647eccc1d40 0x5647eccc6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:10.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.335+0000 7f74bdf5a640 1 --2- 172.21.15.164:0/3718392456 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647eccb6b60 0x5647eccc9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:10.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.335+0000 7f74bd759640 1 -- 172.21.15.164:0/3718392456 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647eccb6b60 msgr2=0x5647eccc9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:10.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.335+0000 7f74bd759640 1 --2- 172.21.15.164:0/3718392456 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647eccb6b60 0x5647eccc9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:10.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.335+0000 7f74bd759640 1 -- 172.21.15.164:0/3718392456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647eccc1d40 msgr2=0x5647eccc6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:10.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.335+0000 7f74bd759640 1 --2- 172.21.15.164:0/3718392456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647eccc1d40 0x5647eccc6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:10.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.335+0000 7f74bd759640 1 -- 172.21.15.164:0/3718392456 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647eccc7220 con 0x5647eccb7480 2024-09-19T08:53:10.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.335+0000 7f74bd759640 1 --2- 172.21.15.164:0/3718392456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647eccb7480 0x5647eccca250 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00ea80 tx=0x7f74ac0a4990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=122b866a865dae93 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:10.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.335+0000 7f749d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:53:10.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.335+0000 7f749d7fa640 1 -- 172.21.15.164:0/3718392456 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac051070 con 0x5647eccb7480 2024-09-19T08:53:10.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.335+0000 7f749d7fa640 1 -- 172.21.15.164:0/3718392456 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8499+0+0 (secure 0 0 0) 0x7f74ac07ab60 con 0x5647eccb7480 2024-09-19T08:53:10.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.335+0000 7f74c2597740 1 -- 172.21.15.164:0/3718392456 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74a8016c70 con 0x5647eccb7480 2024-09-19T08:53:10.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.335+0000 7f749d7fa640 1 -- 172.21.15.164:0/3718392456 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f74ac0a4bf0 con 0x5647eccb7480 2024-09-19T08:53:10.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.335+0000 7f74c2597740 10 client.17007 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:10.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.336+0000 7f74c2597740 1 -- 172.21.15.164:0/3718392456 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5647ecd3cb80 con 0x5647eccb7480 2024-09-19T08:53:10.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.336+0000 7f749d7fa640 1 -- 172.21.15.164:0/3718392456 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74a8016c70 con 0x5647eccb7480 2024-09-19T08:53:10.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.336+0000 7f749d7fa640 1 -- 172.21.15.164:0/3718392456 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f74ac0bace0 con 0x5647eccb7480 2024-09-19T08:53:10.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.336+0000 7f74c2597740 10 client.17007 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:10.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.336+0000 7f74c2597740 10 client.17007 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:53:10.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.336+0000 7f74c2597740 20 client.17007 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:10.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.336+0000 7f74c2597740 1 --2- 172.21.15.164:0/3718392456 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5647ecd3d330 0x5647ecd3f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:10.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.336+0000 7f74c2597740 4 client.17007 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T08:53:10.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.336+0000 7f74c2597740 1 -- 172.21.15.164:0/3718392456 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5647ecc23d00 con 0x5647ecd3d330 2024-09-19T08:53:10.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.336+0000 7f74bdf5a640 1 --2- 172.21.15.164:0/3718392456 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5647ecd3d330 0x5647ecd3f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:10.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.337+0000 7f74bdf5a640 1 --2- 172.21.15.164:0/3718392456 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5647ecd3d330 0x5647ecd3f750 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=648273bbcef11cb server_cookie=62b89eba0298bd46 in_seq=0 out_seq=0 2024-09-19T08:53:10.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.337+0000 7f749d7fa640 10 client.17007 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:10.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.338+0000 7f749d7fa640 1 -- 172.21.15.164:0/3718392456 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5647ecc23d00 con 0x5647ecd3d330 2024-09-19T08:53:10.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.338+0000 7f749d7fa640 10 client.17007 handle_command_reply: tid=0 2024-09-19T08:53:10.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.338+0000 7f74c2597740 1 client.17007 shutdown 2024-09-19T08:53:10.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.338+0000 7f74c2597740 1 -- 172.21.15.164:0/3718392456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647eccb7480 msgr2=0x5647eccca250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:10.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.338+0000 7f74c2597740 1 --2- 172.21.15.164:0/3718392456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647eccb7480 0x5647eccca250 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00ea80 tx=0x7f74ac0a4990 comp rx=0 tx=0).stop 2024-09-19T08:53:10.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.338+0000 7f74c2597740 1 -- 172.21.15.164:0/3718392456 shutdown_connections 2024-09-19T08:53:10.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.338+0000 7f74c2597740 1 --2- 172.21.15.164:0/3718392456 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5647ecd3d330 0x5647ecd3f750 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:10.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.338+0000 7f74c2597740 1 --2- 172.21.15.164:0/3718392456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647eccc1d40 0x5647eccc6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:10.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.338+0000 7f74c2597740 1 --2- 172.21.15.164:0/3718392456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647eccb7480 0x5647eccca250 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:10.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.338+0000 7f74c2597740 1 --2- 172.21.15.164:0/3718392456 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647eccb6b60 0x5647eccc9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:10.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.338+0000 7f74c2597740 1 -- 172.21.15.164:0/3718392456 >> 172.21.15.164:0/3718392456 conn(0x5647eccb30f0 msgr2=0x5647eccb2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:10.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.338+0000 7f749d7fa640 0 client.17007 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:10.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.338+0000 7f74c2597740 1 -- 172.21.15.164:0/3718392456 shutdown_connections 2024-09-19T08:53:10.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.338+0000 7f74c2597740 1 -- 172.21.15.164:0/3718392456 wait complete. 2024-09-19T08:53:10.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.338+0000 7f74c2597740 20 client.17007 trim_cache size 0 max 16384 2024-09-19T08:53:10.342 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:53:10.342 INFO:teuthology.orchestra.run.smithi164.stdout: "return_code": 0, 2024-09-19T08:53:10.342 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_tag": "994d7330-fa06-4544-bb9f-6d415b8b3f68", 2024-09-19T08:53:10.342 INFO:teuthology.orchestra.run.smithi164.stdout: "mode": "asynchronous" 2024-09-19T08:53:10.342 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:53:10.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.339+0000 7f74c01be640 1 -- 172.21.15.164:0/182691361 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f74a00a33b0 msgr2=0x7f74a00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:10.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.339+0000 7f74c01be640 1 --2- 172.21.15.164:0/182691361 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f74a00a33b0 0x7f74a00a5870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x5647eccb4470 tx=0x7f74a8009490 comp rx=0 tx=0).stop 2024-09-19T08:53:10.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.339+0000 7f74c01be640 1 -- 172.21.15.164:0/182691361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74b8102570 msgr2=0x7f74b8199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:10.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.339+0000 7f74c01be640 1 --2- 172.21.15.164:0/182691361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74b8102570 0x7f74b8199870 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00eab0 tx=0x7f74ac00ef70 comp rx=0 tx=0).stop 2024-09-19T08:53:10.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.339+0000 7f74c01be640 1 -- 172.21.15.164:0/182691361 shutdown_connections 2024-09-19T08:53:10.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.339+0000 7f74c01be640 1 --2- 172.21.15.164:0/182691361 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74b8102ef0 0x7f74b81a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:10.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.339+0000 7f74c01be640 1 --2- 172.21.15.164:0/182691361 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f74a00a33b0 0x7f74a00a5870 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:10.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.339+0000 7f74c01be640 1 --2- 172.21.15.164:0/182691361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74b8102570 0x7f74b8199870 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:10.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.339+0000 7f74c01be640 1 --2- 172.21.15.164:0/182691361 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74b8100e20 0x7f74b8199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:10.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.339+0000 7f74c01be640 1 -- 172.21.15.164:0/182691361 >> 172.21.15.164:0/182691361 conn(0x7f74b80fc7d0 msgr2=0x7f74b81068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:10.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.339+0000 7f74c01be640 1 -- 172.21.15.164:0/182691361 shutdown_connections 2024-09-19T08:53:10.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:10.339+0000 7f74c01be640 1 -- 172.21.15.164:0/182691361 wait complete. 2024-09-19T08:53:10.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:10 smithi170 ceph-mon[35072]: pgmap v226: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T08:53:10.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:10 smithi173 ceph-mon[35306]: pgmap v226: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T08:53:11.206 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:53:11.457 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:53:11.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:11 smithi164 ceph-mon[30928]: scrub queued for path: / 2024-09-19T08:53:11.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:11 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:53:11.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:11 smithi164 ceph-mon[30928]: scrub summary: active paths [/] 2024-09-19T08:53:11.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:11 smithi164 ceph-mon[30928]: scrub summary: active paths [/] 2024-09-19T08:53:11.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:11 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:53:11.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:11 smithi164 ceph-mon[30928]: scrub summary: active paths [/] 2024-09-19T08:53:11.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:11 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:53:11.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:11 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:53:11.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:11 smithi170 ceph-mon[35072]: scrub queued for path: / 2024-09-19T08:53:11.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:11 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:53:11.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:11 smithi170 ceph-mon[35072]: scrub summary: active paths [/] 2024-09-19T08:53:11.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:11 smithi170 ceph-mon[35072]: scrub summary: active paths [/] 2024-09-19T08:53:11.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:11 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:53:11.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:11 smithi170 ceph-mon[35072]: scrub summary: active paths [/] 2024-09-19T08:53:11.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:11 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:53:11.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:11 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:53:11.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:11 smithi173 ceph-mon[35306]: scrub queued for path: / 2024-09-19T08:53:11.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:11 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:53:11.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:11 smithi173 ceph-mon[35306]: scrub summary: active paths [/] 2024-09-19T08:53:11.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:11 smithi173 ceph-mon[35306]: scrub summary: active paths [/] 2024-09-19T08:53:11.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:11 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:53:11.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:11 smithi173 ceph-mon[35306]: scrub summary: active paths [/] 2024-09-19T08:53:11.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:11 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:53:11.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:11 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:53:11.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.932+0000 7f1791d9b640 1 -- 172.21.15.164:0/944853180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f178c106f10 msgr2=0x7f178c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:11.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.932+0000 7f1791d9b640 1 --2- 172.21.15.164:0/944853180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f178c106f10 0x7f178c10ea90 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1784009900 tx=0x7f178402a410 comp rx=0 tx=0).stop 2024-09-19T08:53:11.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.932+0000 7f1791d9b640 1 -- 172.21.15.164:0/944853180 shutdown_connections 2024-09-19T08:53:11.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.932+0000 7f1791d9b640 1 --2- 172.21.15.164:0/944853180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f178c106f10 0x7f178c10ea90 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.932+0000 7f1791d9b640 1 --2- 172.21.15.164:0/944853180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f178c102790 0x7f178c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.932+0000 7f1791d9b640 1 --2- 172.21.15.164:0/944853180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f178c101de0 0x7f178c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.932+0000 7f1791d9b640 1 -- 172.21.15.164:0/944853180 >> 172.21.15.164:0/944853180 conn(0x7f178c0fd8c0 msgr2=0x7f178c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:11.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.932+0000 7f1791d9b640 1 -- 172.21.15.164:0/944853180 shutdown_connections 2024-09-19T08:53:11.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.932+0000 7f1791d9b640 1 -- 172.21.15.164:0/944853180 wait complete. 2024-09-19T08:53:11.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.932+0000 7f1791d9b640 1 Processor -- start 2024-09-19T08:53:11.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.932+0000 7f1791d9b640 1 -- start start 2024-09-19T08:53:11.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.933+0000 7f1791d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f178c101de0 0x7f178c1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:11.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.933+0000 7f1791d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f178c102790 0x7f178c198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:11.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.933+0000 7f1791d9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f178c106f10 0x7f178c19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:11.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.933+0000 7f1791d9b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f178c111f70 con 0x7f178c102790 2024-09-19T08:53:11.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.933+0000 7f1791d9b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f178c111df0 con 0x7f178c101de0 2024-09-19T08:53:11.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.933+0000 7f1791d9b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f178c1120f0 con 0x7f178c106f10 2024-09-19T08:53:11.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.933+0000 7f178affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f178c102790 0x7f178c198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:11.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.933+0000 7f178affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f178c102790 0x7f178c198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42390/0 (socket says 172.21.15.164:42390) 2024-09-19T08:53:11.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.933+0000 7f178affd640 1 -- 172.21.15.164:0/4128381020 learned_addr learned my addr 172.21.15.164:0/4128381020 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:11.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.933+0000 7f178b7fe640 1 --2- 172.21.15.164:0/4128381020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f178c101de0 0x7f178c1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:11.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.933+0000 7f178bfff640 1 --2- 172.21.15.164:0/4128381020 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f178c106f10 0x7f178c19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:11.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.933+0000 7f178affd640 1 -- 172.21.15.164:0/4128381020 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f178c106f10 msgr2=0x7f178c19cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:11.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.933+0000 7f178affd640 1 --2- 172.21.15.164:0/4128381020 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f178c106f10 0x7f178c19cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.933+0000 7f178affd640 1 -- 172.21.15.164:0/4128381020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f178c101de0 msgr2=0x7f178c1982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:11.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.933+0000 7f178affd640 1 --2- 172.21.15.164:0/4128381020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f178c101de0 0x7f178c1982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.933+0000 7f178affd640 1 -- 172.21.15.164:0/4128381020 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f178c19d290 con 0x7f178c102790 2024-09-19T08:53:11.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.933+0000 7f178b7fe640 1 --2- 172.21.15.164:0/4128381020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f178c101de0 0x7f178c1982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:53:11.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.933+0000 7f178affd640 1 --2- 172.21.15.164:0/4128381020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f178c102790 0x7f178c198820 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f178000c920 tx=0x7f178000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:11.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.933+0000 7f1788ff9640 1 -- 172.21.15.164:0/4128381020 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780007c70 con 0x7f178c102790 2024-09-19T08:53:11.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.933+0000 7f1791d9b640 1 -- 172.21.15.164:0/4128381020 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f178c19d580 con 0x7f178c102790 2024-09-19T08:53:11.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.933+0000 7f1788ff9640 1 -- 172.21.15.164:0/4128381020 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1780007e10 con 0x7f178c102790 2024-09-19T08:53:11.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.933+0000 7f1788ff9640 1 -- 172.21.15.164:0/4128381020 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780018890 con 0x7f178c102790 2024-09-19T08:53:11.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.934+0000 7f1791d9b640 1 -- 172.21.15.164:0/4128381020 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f178c1a4e60 con 0x7f178c102790 2024-09-19T08:53:11.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.936+0000 7f1788ff9640 1 -- 172.21.15.164:0/4128381020 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f1780018a70 con 0x7f178c102790 2024-09-19T08:53:11.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.936+0000 7f1794174740 1 Processor -- start 2024-09-19T08:53:11.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.936+0000 7f1794174740 1 -- start start 2024-09-19T08:53:11.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.936+0000 7f1788ff9640 1 --2- 172.21.15.164:0/4128381020 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f17740a33b0 0x7f17740a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:11.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.936+0000 7f1794174740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623f7b0b360 0x5623f7b0b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:11.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.936+0000 7f1794174740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623f7b0bc80 0x5623f7b1a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:11.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.936+0000 7f1794174740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623f7b1aa90 0x5623f7b1ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:11.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.936+0000 7f1794174740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5623f7a712e0 con 0x5623f7b0b360 2024-09-19T08:53:11.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.936+0000 7f1794174740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5623f7a18ce0 con 0x5623f7b1aa90 2024-09-19T08:53:11.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.936+0000 7f1794174740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5623f7a6d2a0 con 0x5623f7b0bc80 2024-09-19T08:53:11.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.936+0000 7f178b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623f7b0b360 0x5623f7b0b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:11.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.936+0000 7f178b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623f7b0b360 0x5623f7b0b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42392/0 (socket says 172.21.15.164:42392) 2024-09-19T08:53:11.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.936+0000 7f178b7fe640 1 -- 172.21.15.164:0/1188075269 learned_addr learned my addr 172.21.15.164:0/1188075269 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:11.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.936+0000 7f178bfff640 1 --2- 172.21.15.164:0/4128381020 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f17740a33b0 0x7f17740a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:11.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.937+0000 7f1788ff9640 1 -- 172.21.15.164:0/4128381020 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8499+0+0 (secure 0 0 0) 0x7f17800cfe90 con 0x7f178c102790 2024-09-19T08:53:11.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.937+0000 7f178affd640 1 --2- 172.21.15.164:0/1188075269 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623f7b0bc80 0x5623f7b1a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:11.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.937+0000 7f178bfff640 1 --2- 172.21.15.164:0/1188075269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623f7b1aa90 0x5623f7b1ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:11.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.937+0000 7f178b7fe640 1 -- 172.21.15.164:0/1188075269 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623f7b0bc80 msgr2=0x5623f7b1a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:11.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.937+0000 7f178b7fe640 1 --2- 172.21.15.164:0/1188075269 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623f7b0bc80 0x5623f7b1a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.937+0000 7f178b7fe640 1 -- 172.21.15.164:0/1188075269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623f7b1aa90 msgr2=0x5623f7b1ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:11.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.937+0000 7f178b7fe640 1 --2- 172.21.15.164:0/1188075269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623f7b1aa90 0x5623f7b1ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.937+0000 7f178b7fe640 1 -- 172.21.15.164:0/1188075269 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623f79fd440 con 0x5623f7b0b360 2024-09-19T08:53:11.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.937+0000 7f178bfff640 1 --2- 172.21.15.164:0/1188075269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623f7b1aa90 0x5623f7b1ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:53:11.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.937+0000 7f178b7fe640 1 --2- 172.21.15.164:0/1188075269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623f7b0b360 0x5623f7b0b740 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f177c007f70 tx=0x7f177c00e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ced5b843a71532a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:11.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.937+0000 7f178bfff640 1 --2- 172.21.15.164:0/4128381020 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f17740a33b0 0x7f17740a5870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x5623f7b08b20 tx=0x7f17840314d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:11.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.937+0000 7f1771ffb640 1 -- 172.21.15.164:0/1188075269 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177c014070 con 0x5623f7b0b360 2024-09-19T08:53:11.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.937+0000 7f1771ffb640 1 -- 172.21.15.164:0/1188075269 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f177c004d20 con 0x5623f7b0b360 2024-09-19T08:53:11.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.937+0000 7f1771ffb640 1 -- 172.21.15.164:0/1188075269 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177c005540 con 0x5623f7b0b360 2024-09-19T08:53:11.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.938+0000 7f1794174740 1 -- 172.21.15.164:0/1188075269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623f7b0b360 msgr2=0x5623f7b0b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:11.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.938+0000 7f1794174740 1 --2- 172.21.15.164:0/1188075269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623f7b0b360 0x5623f7b0b740 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f177c007f70 tx=0x7f177c00e7e0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.938+0000 7f1794174740 1 -- 172.21.15.164:0/1188075269 shutdown_connections 2024-09-19T08:53:11.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.938+0000 7f1794174740 1 --2- 172.21.15.164:0/1188075269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623f7b1aa90 0x5623f7b1ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.938+0000 7f1794174740 1 --2- 172.21.15.164:0/1188075269 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623f7b0bc80 0x5623f7b1a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.938+0000 7f1794174740 1 --2- 172.21.15.164:0/1188075269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623f7b0b360 0x5623f7b0b740 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.938+0000 7f1794174740 1 -- 172.21.15.164:0/1188075269 >> 172.21.15.164:0/1188075269 conn(0x5623f7b07a30 msgr2=0x5623f7b07e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:11.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.938+0000 7f1794174740 1 -- 172.21.15.164:0/1188075269 shutdown_connections 2024-09-19T08:53:11.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.938+0000 7f1794174740 1 -- 172.21.15.164:0/1188075269 wait complete. 2024-09-19T08:53:11.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.938+0000 7f1794174740 1 -- start start 2024-09-19T08:53:11.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.939+0000 7f1794174740 1 Processor -- start 2024-09-19T08:53:11.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.939+0000 7f1794174740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623f7b0b360 0x5623f7b1a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:11.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.939+0000 7f1794174740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623f7b0bc80 0x5623f7b176c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:11.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.939+0000 7f1794174740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623f7b1aa90 0x5623f7b17c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:11.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.939+0000 7f1794174740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5623f7b1d660 con 0x5623f7b0bc80 2024-09-19T08:53:11.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.939+0000 7f1794174740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5623f7a2c050 con 0x5623f7b1aa90 2024-09-19T08:53:11.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.939+0000 7f1794174740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5623f7b1d7e0 con 0x5623f7b0b360 2024-09-19T08:53:11.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.939+0000 7f1794174740 10 client.0 _ll_register_callbacks cb 0x5623f7a6c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:11.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.939+0000 7f178affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623f7b0bc80 0x5623f7b176c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:11.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.939+0000 7f178affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623f7b0bc80 0x5623f7b176c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42396/0 (socket says 172.21.15.164:42396) 2024-09-19T08:53:11.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.939+0000 7f178affd640 1 -- 172.21.15.164:0/4061028672 learned_addr learned my addr 172.21.15.164:0/4061028672 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:11.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.939+0000 7f178bfff640 1 --2- 172.21.15.164:0/4061028672 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623f7b1aa90 0x5623f7b17c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:11.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.939+0000 7f178b7fe640 1 --2- 172.21.15.164:0/4061028672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623f7b0b360 0x5623f7b1a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:11.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.939+0000 7f178affd640 1 -- 172.21.15.164:0/4061028672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623f7b0b360 msgr2=0x5623f7b1a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:11.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.939+0000 7f178affd640 1 --2- 172.21.15.164:0/4061028672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623f7b0b360 0x5623f7b1a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.939+0000 7f178affd640 1 -- 172.21.15.164:0/4061028672 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623f7b1aa90 msgr2=0x5623f7b17c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:11.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.939+0000 7f178affd640 1 --2- 172.21.15.164:0/4061028672 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623f7b1aa90 0x5623f7b17c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.939+0000 7f178affd640 1 -- 172.21.15.164:0/4061028672 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623f77e23c0 con 0x5623f7b0bc80 2024-09-19T08:53:11.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.939+0000 7f178b7fe640 1 --2- 172.21.15.164:0/4061028672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623f7b0b360 0x5623f7b1a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:53:11.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.939+0000 7f178affd640 1 --2- 172.21.15.164:0/4061028672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623f7b0bc80 0x5623f7b176c0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f1780007590 tx=0x7f178009c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea67236288f58c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:11.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.939+0000 7f176affd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:53:11.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.939+0000 7f176affd640 1 -- 172.21.15.164:0/4061028672 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780018d60 con 0x5623f7b0bc80 2024-09-19T08:53:11.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.940+0000 7f176affd640 1 -- 172.21.15.164:0/4061028672 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8499+0+0 (secure 0 0 0) 0x7f17800cd870 con 0x5623f7b0bc80 2024-09-19T08:53:11.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.940+0000 7f1794174740 1 -- 172.21.15.164:0/4061028672 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f177c004d20 con 0x5623f7b0bc80 2024-09-19T08:53:11.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.940+0000 7f176affd640 1 -- 172.21.15.164:0/4061028672 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f17800b5120 con 0x5623f7b0bc80 2024-09-19T08:53:11.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.940+0000 7f1794174740 10 client.17019 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:11.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.940+0000 7f1794174740 1 -- 172.21.15.164:0/4061028672 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5623f7b29060 con 0x5623f7b0bc80 2024-09-19T08:53:11.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.940+0000 7f176affd640 1 -- 172.21.15.164:0/4061028672 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f177c004d20 con 0x5623f7b0bc80 2024-09-19T08:53:11.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.940+0000 7f176affd640 1 -- 172.21.15.164:0/4061028672 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f17800bc070 con 0x5623f7b0bc80 2024-09-19T08:53:11.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.940+0000 7f1794174740 10 client.17019 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:11.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.940+0000 7f1794174740 10 client.17019 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:53:11.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.940+0000 7f1794174740 20 client.17019 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:11.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.940+0000 7f1794174740 1 --2- 172.21.15.164:0/4061028672 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5623f7b2a3b0 0x5623f7b2c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:11.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.940+0000 7f1794174740 4 client.17019 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:53:11.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.940+0000 7f1794174740 1 -- 172.21.15.164:0/4061028672 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623f7a7e850 con 0x5623f7b2a3b0 2024-09-19T08:53:11.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.940+0000 7f178b7fe640 1 --2- 172.21.15.164:0/4061028672 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5623f7b2a3b0 0x5623f7b2c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:11.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.941+0000 7f178b7fe640 1 --2- 172.21.15.164:0/4061028672 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5623f7b2a3b0 0x5623f7b2c850 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73168283c94d0474 server_cookie=2c7cd417bc9e0639 in_seq=0 out_seq=0 2024-09-19T08:53:11.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.941+0000 7f176affd640 10 client.17019 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:11.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.941+0000 7f176affd640 1 -- 172.21.15.164:0/4061028672 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5623f7a7e850 con 0x5623f7b2a3b0 2024-09-19T08:53:11.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.941+0000 7f176affd640 10 client.17019 handle_command_reply: tid=0 2024-09-19T08:53:11.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.942+0000 7f1794174740 1 client.17019 shutdown 2024-09-19T08:53:11.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.942+0000 7f1794174740 1 -- 172.21.15.164:0/4061028672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623f7b0bc80 msgr2=0x5623f7b176c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:11.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.942+0000 7f1794174740 1 --2- 172.21.15.164:0/4061028672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623f7b0bc80 0x5623f7b176c0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f1780007590 tx=0x7f178009c180 comp rx=0 tx=0).stop 2024-09-19T08:53:11.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.942+0000 7f1794174740 1 -- 172.21.15.164:0/4061028672 shutdown_connections 2024-09-19T08:53:11.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.942+0000 7f1794174740 1 --2- 172.21.15.164:0/4061028672 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5623f7b2a3b0 0x5623f7b2c850 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.942+0000 7f1794174740 1 --2- 172.21.15.164:0/4061028672 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623f7b1aa90 0x5623f7b17c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.942+0000 7f1794174740 1 --2- 172.21.15.164:0/4061028672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623f7b0bc80 0x5623f7b176c0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.942+0000 7f1794174740 1 --2- 172.21.15.164:0/4061028672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623f7b0b360 0x5623f7b1a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.942+0000 7f176affd640 0 client.17019 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:11.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.942+0000 7f1794174740 1 -- 172.21.15.164:0/4061028672 >> 172.21.15.164:0/4061028672 conn(0x5623f7b07a30 msgr2=0x5623f7b0d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:11.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.942+0000 7f1794174740 1 -- 172.21.15.164:0/4061028672 shutdown_connections 2024-09-19T08:53:11.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.942+0000 7f1794174740 1 -- 172.21.15.164:0/4061028672 wait complete. 2024-09-19T08:53:11.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.942+0000 7f1794174740 20 client.17019 trim_cache size 0 max 16384 2024-09-19T08:53:11.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.959+0000 7f1794174740 1 Processor -- start 2024-09-19T08:53:11.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.959+0000 7f1794174740 1 -- start start 2024-09-19T08:53:11.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.959+0000 7f1794174740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623f7b0b360 0x5623f7b26900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:11.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.959+0000 7f1794174740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623f7b0bc80 0x5623f7b19630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:11.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.959+0000 7f1794174740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623f7b1aa90 0x5623f7b32170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:11.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.959+0000 7f1794174740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f178c1ab4f0 con 0x5623f7b1aa90 2024-09-19T08:53:11.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.959+0000 7f1794174740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f178c1ad030 con 0x5623f7b0b360 2024-09-19T08:53:11.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.959+0000 7f1794174740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f178c1abbc0 con 0x5623f7b0bc80 2024-09-19T08:53:11.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.959+0000 7f178bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623f7b1aa90 0x5623f7b32170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:11.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.959+0000 7f178bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623f7b1aa90 0x5623f7b32170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42412/0 (socket says 172.21.15.164:42412) 2024-09-19T08:53:11.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.959+0000 7f178bfff640 1 -- 172.21.15.164:0/1896253725 learned_addr learned my addr 172.21.15.164:0/1896253725 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:11.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.960+0000 7f178b7fe640 1 --2- 172.21.15.164:0/1896253725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623f7b0b360 0x5623f7b26900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:11.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.960+0000 7f178affd640 1 --2- 172.21.15.164:0/1896253725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623f7b0bc80 0x5623f7b19630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:11.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.960+0000 7f178bfff640 1 -- 172.21.15.164:0/1896253725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623f7b0bc80 msgr2=0x5623f7b19630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:11.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.960+0000 7f178bfff640 1 --2- 172.21.15.164:0/1896253725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623f7b0bc80 0x5623f7b19630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.960+0000 7f178bfff640 1 -- 172.21.15.164:0/1896253725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623f7b0b360 msgr2=0x5623f7b26900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:11.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.960+0000 7f178bfff640 1 --2- 172.21.15.164:0/1896253725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623f7b0b360 0x5623f7b26900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.960+0000 7f178bfff640 1 -- 172.21.15.164:0/1896253725 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623f7b074d0 con 0x5623f7b1aa90 2024-09-19T08:53:11.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.960+0000 7f178b7fe640 1 --2- 172.21.15.164:0/1896253725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623f7b0b360 0x5623f7b26900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:53:11.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.960+0000 7f178bfff640 1 --2- 172.21.15.164:0/1896253725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623f7b1aa90 0x5623f7b32170 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f1784009900 tx=0x7f1784038000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e80b2718c85006e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:11.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.960+0000 7f17717fa640 1 -- 172.21.15.164:0/1896253725 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178404d070 con 0x5623f7b1aa90 2024-09-19T08:53:11.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.960+0000 7f17717fa640 1 -- 172.21.15.164:0/1896253725 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1784005530 con 0x5623f7b1aa90 2024-09-19T08:53:11.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.960+0000 7f17717fa640 1 -- 172.21.15.164:0/1896253725 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1784041040 con 0x5623f7b1aa90 2024-09-19T08:53:11.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.961+0000 7f1794174740 1 -- 172.21.15.164:0/1896253725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623f7b1aa90 msgr2=0x5623f7b32170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:11.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.961+0000 7f1794174740 1 --2- 172.21.15.164:0/1896253725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623f7b1aa90 0x5623f7b32170 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f1784009900 tx=0x7f1784038000 comp rx=0 tx=0).stop 2024-09-19T08:53:11.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.961+0000 7f1794174740 1 -- 172.21.15.164:0/1896253725 shutdown_connections 2024-09-19T08:53:11.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.961+0000 7f1794174740 1 --2- 172.21.15.164:0/1896253725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623f7b1aa90 0x5623f7b32170 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.961+0000 7f1794174740 1 --2- 172.21.15.164:0/1896253725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623f7b0bc80 0x5623f7b19630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.961+0000 7f1794174740 1 --2- 172.21.15.164:0/1896253725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623f7b0b360 0x5623f7b26900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.961+0000 7f1794174740 1 -- 172.21.15.164:0/1896253725 >> 172.21.15.164:0/1896253725 conn(0x5623f7b07a30 msgr2=0x5623f7b18f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:11.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.961+0000 7f1794174740 1 -- 172.21.15.164:0/1896253725 shutdown_connections 2024-09-19T08:53:11.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.961+0000 7f1794174740 1 -- 172.21.15.164:0/1896253725 wait complete. 2024-09-19T08:53:11.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.961+0000 7f1794174740 1 -- start start 2024-09-19T08:53:11.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.961+0000 7f1794174740 1 Processor -- start 2024-09-19T08:53:11.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.962+0000 7f1794174740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623f7b0b360 0x5623f7b1ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:11.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.962+0000 7f1794174740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623f7b0bc80 0x5623f7b1ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:11.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.962+0000 7f1794174740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623f7b1aa90 0x5623f7b1f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:11.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.962+0000 7f1794174740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f178c1ac960 con 0x5623f7b0b360 2024-09-19T08:53:11.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.962+0000 7f1794174740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f178c1ac290 con 0x5623f7b0bc80 2024-09-19T08:53:11.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.962+0000 7f1794174740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f178c1a94f0 con 0x5623f7b1aa90 2024-09-19T08:53:11.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.962+0000 7f1794174740 10 client.0 _ll_register_callbacks cb 0x7f178000a5a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:11.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.962+0000 7f178b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623f7b0b360 0x5623f7b1ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:11.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.962+0000 7f178b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623f7b0b360 0x5623f7b1ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42420/0 (socket says 172.21.15.164:42420) 2024-09-19T08:53:11.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.962+0000 7f178b7fe640 1 -- 172.21.15.164:0/2188903778 learned_addr learned my addr 172.21.15.164:0/2188903778 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:11.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.962+0000 7f178affd640 1 --2- 172.21.15.164:0/2188903778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623f7b0bc80 0x5623f7b1ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:11.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.962+0000 7f178bfff640 1 --2- 172.21.15.164:0/2188903778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623f7b1aa90 0x5623f7b1f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:11.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.962+0000 7f178b7fe640 1 -- 172.21.15.164:0/2188903778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623f7b1aa90 msgr2=0x5623f7b1f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:11.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.962+0000 7f178b7fe640 1 --2- 172.21.15.164:0/2188903778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623f7b1aa90 0x5623f7b1f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.962+0000 7f178b7fe640 1 -- 172.21.15.164:0/2188903778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623f7b0bc80 msgr2=0x5623f7b1ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:11.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.962+0000 7f178b7fe640 1 --2- 172.21.15.164:0/2188903778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623f7b0bc80 0x5623f7b1ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.962+0000 7f178b7fe640 1 -- 172.21.15.164:0/2188903778 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623f7b94390 con 0x5623f7b0b360 2024-09-19T08:53:11.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.962+0000 7f178b7fe640 1 --2- 172.21.15.164:0/2188903778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623f7b0b360 0x5623f7b1ea20 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f177c007f70 tx=0x7f177c004bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5dcae3ddf117ab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:11.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.962+0000 7f176affd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:53:11.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.963+0000 7f176affd640 1 -- 172.21.15.164:0/2188903778 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177c02a030 con 0x5623f7b0b360 2024-09-19T08:53:11.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.963+0000 7f1794174740 1 -- 172.21.15.164:0/2188903778 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1784005530 con 0x5623f7b0b360 2024-09-19T08:53:11.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.963+0000 7f176affd640 1 -- 172.21.15.164:0/2188903778 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8499+0+0 (secure 0 0 0) 0x7f177c003970 con 0x5623f7b0b360 2024-09-19T08:53:11.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.963+0000 7f176affd640 1 -- 172.21.15.164:0/2188903778 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f177c034050 con 0x5623f7b0b360 2024-09-19T08:53:11.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.963+0000 7f1794174740 10 client.17025 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:11.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.963+0000 7f1794174740 1 -- 172.21.15.164:0/2188903778 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5623f7b95680 con 0x5623f7b0b360 2024-09-19T08:53:11.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.963+0000 7f176affd640 1 -- 172.21.15.164:0/2188903778 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1784005530 con 0x5623f7b0b360 2024-09-19T08:53:11.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.963+0000 7f176affd640 1 -- 172.21.15.164:0/2188903778 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f177c005640 con 0x5623f7b0b360 2024-09-19T08:53:11.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.963+0000 7f1794174740 10 client.17025 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:11.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.963+0000 7f1794174740 10 client.17025 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:53:11.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.963+0000 7f1794174740 20 client.17025 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:11.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.963+0000 7f1794174740 1 --2- 172.21.15.164:0/2188903778 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5623f7b95b70 0x5623f7b97f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:11.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.963+0000 7f1794174740 4 client.17025 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:53:11.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.963+0000 7f1794174740 1 -- 172.21.15.164:0/2188903778 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5623f7a78d60 con 0x5623f7b95b70 2024-09-19T08:53:11.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.963+0000 7f178affd640 1 --2- 172.21.15.164:0/2188903778 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5623f7b95b70 0x5623f7b97f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:11.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.964+0000 7f178affd640 1 --2- 172.21.15.164:0/2188903778 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5623f7b95b70 0x5623f7b97f60 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9afbb8db9c5eafd server_cookie=37134616c870c38 in_seq=0 out_seq=0 2024-09-19T08:53:11.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.964+0000 7f176affd640 10 client.17025 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:11.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.964+0000 7f176affd640 1 -- 172.21.15.164:0/2188903778 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5623f7a78d60 con 0x5623f7b95b70 2024-09-19T08:53:11.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.964+0000 7f176affd640 10 client.17025 handle_command_reply: tid=0 2024-09-19T08:53:11.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.964+0000 7f1794174740 1 client.17025 shutdown 2024-09-19T08:53:11.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.964+0000 7f1794174740 1 -- 172.21.15.164:0/2188903778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623f7b0b360 msgr2=0x5623f7b1ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:11.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.964+0000 7f1794174740 1 --2- 172.21.15.164:0/2188903778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623f7b0b360 0x5623f7b1ea20 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f177c007f70 tx=0x7f177c004bf0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.964+0000 7f1794174740 1 -- 172.21.15.164:0/2188903778 shutdown_connections 2024-09-19T08:53:11.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.964+0000 7f1794174740 1 --2- 172.21.15.164:0/2188903778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623f7b1aa90 0x5623f7b1f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.964+0000 7f1794174740 1 --2- 172.21.15.164:0/2188903778 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5623f7b95b70 0x5623f7b97f60 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.964+0000 7f1794174740 1 --2- 172.21.15.164:0/2188903778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623f7b0bc80 0x5623f7b1ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.964+0000 7f1794174740 1 --2- 172.21.15.164:0/2188903778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623f7b0b360 0x5623f7b1ea20 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.964+0000 7f1794174740 1 -- 172.21.15.164:0/2188903778 >> 172.21.15.164:0/2188903778 conn(0x5623f7b07a30 msgr2=0x5623f7b19740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:11.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.964+0000 7f176affd640 0 client.17025 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:11.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.964+0000 7f1794174740 1 -- 172.21.15.164:0/2188903778 shutdown_connections 2024-09-19T08:53:11.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.964+0000 7f1794174740 1 -- 172.21.15.164:0/2188903778 wait complete. 2024-09-19T08:53:11.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.964+0000 7f1794174740 20 client.17025 trim_cache size 0 max 16384 2024-09-19T08:53:11.973 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:53:11.973 INFO:teuthology.orchestra.run.smithi164.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T08:53:11.974 INFO:teuthology.orchestra.run.smithi164.stdout: "scrubs": { 2024-09-19T08:53:11.974 INFO:teuthology.orchestra.run.smithi164.stdout: "994d7330-fa06-4544-bb9f-6d415b8b3f68": { 2024-09-19T08:53:11.974 INFO:teuthology.orchestra.run.smithi164.stdout: "path": "/", 2024-09-19T08:53:11.974 INFO:teuthology.orchestra.run.smithi164.stdout: "tag": "994d7330-fa06-4544-bb9f-6d415b8b3f68", 2024-09-19T08:53:11.974 INFO:teuthology.orchestra.run.smithi164.stdout: "options": "recursive,force" 2024-09-19T08:53:11.974 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:53:11.974 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:53:11.974 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:53:11.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.965+0000 7f1791d9b640 1 -- 172.21.15.164:0/4128381020 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f17740a33b0 msgr2=0x7f17740a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:11.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.965+0000 7f1791d9b640 1 --2- 172.21.15.164:0/4128381020 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f17740a33b0 0x7f17740a5870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x5623f7b08b20 tx=0x7f17840314d0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.965+0000 7f1791d9b640 1 -- 172.21.15.164:0/4128381020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f178c102790 msgr2=0x7f178c198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:11.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.965+0000 7f1791d9b640 1 --2- 172.21.15.164:0/4128381020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f178c102790 0x7f178c198820 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f178000c920 tx=0x7f178000cde0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.965+0000 7f1791d9b640 1 -- 172.21.15.164:0/4128381020 shutdown_connections 2024-09-19T08:53:11.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.965+0000 7f1791d9b640 1 --2- 172.21.15.164:0/4128381020 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f178c106f10 0x7f178c19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.965+0000 7f1791d9b640 1 --2- 172.21.15.164:0/4128381020 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f17740a33b0 0x7f17740a5870 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.965+0000 7f1791d9b640 1 --2- 172.21.15.164:0/4128381020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f178c102790 0x7f178c198820 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.965+0000 7f1791d9b640 1 --2- 172.21.15.164:0/4128381020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f178c101de0 0x7f178c1982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:11.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.965+0000 7f1791d9b640 1 -- 172.21.15.164:0/4128381020 >> 172.21.15.164:0/4128381020 conn(0x7f178c0fd8c0 msgr2=0x7f178c0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:11.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.965+0000 7f1791d9b640 1 -- 172.21.15.164:0/4128381020 shutdown_connections 2024-09-19T08:53:11.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:11.965+0000 7f1791d9b640 1 -- 172.21.15.164:0/4128381020 wait complete. 2024-09-19T08:53:12.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:12 smithi164 ceph-mon[30928]: pgmap v227: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T08:53:12.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:12 smithi170 ceph-mon[35072]: pgmap v227: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T08:53:12.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:12 smithi173 ceph-mon[35306]: pgmap v227: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T08:53:12.700 INFO:tasks.cephfs.filesystem:scrub status for tag:994d7330-fa06-4544-bb9f-6d415b8b3f68 - {'path': '/', 'tag': '994d7330-fa06-4544-bb9f-6d415b8b3f68', 'options': 'recursive,force'} 2024-09-19T08:53:14.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:14 smithi170 ceph-mon[35072]: pgmap v228: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-09-19T08:53:14.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:14 smithi173 ceph-mon[35306]: pgmap v228: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-09-19T08:53:14.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:14 smithi164 ceph-mon[30928]: pgmap v228: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-09-19T08:53:16.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:15 smithi170 ceph-mon[35072]: pgmap v229: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-09-19T08:53:16.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:15 smithi170 ceph-mon[35072]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T08:53:16.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:15 smithi173 ceph-mon[35306]: pgmap v229: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-09-19T08:53:16.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:15 smithi173 ceph-mon[35306]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T08:53:16.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:15 smithi164 ceph-mon[30928]: pgmap v229: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-09-19T08:53:16.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:15 smithi164 ceph-mon[30928]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T08:53:18.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:17 smithi170 ceph-mon[35072]: pgmap v231: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.1 KiB/s wr, 15 op/s 2024-09-19T08:53:18.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:17 smithi170 ceph-mon[35072]: scrub summary: idle 2024-09-19T08:53:18.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:17 smithi173 ceph-mon[35306]: pgmap v231: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.1 KiB/s wr, 15 op/s 2024-09-19T08:53:18.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:17 smithi173 ceph-mon[35306]: scrub summary: idle 2024-09-19T08:53:18.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:17 smithi164 ceph-mon[30928]: pgmap v231: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.1 KiB/s wr, 15 op/s 2024-09-19T08:53:18.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:17 smithi164 ceph-mon[30928]: scrub summary: idle 2024-09-19T08:53:19.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:18 smithi170 ceph-mon[35072]: scrub summary: idle 2024-09-19T08:53:19.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:18 smithi173 ceph-mon[35306]: scrub summary: idle 2024-09-19T08:53:19.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:18 smithi164 ceph-mon[30928]: scrub summary: idle 2024-09-19T08:53:20.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:19 smithi170 ceph-mon[35072]: pgmap v232: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.6 KiB/s wr, 13 op/s 2024-09-19T08:53:20.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:19 smithi173 ceph-mon[35306]: pgmap v232: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.6 KiB/s wr, 13 op/s 2024-09-19T08:53:20.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:19 smithi164 ceph-mon[30928]: pgmap v232: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.6 KiB/s wr, 13 op/s 2024-09-19T08:53:20.479 INFO:tasks.quiescer.fs.[cephfs]:Total duration (51.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T08:53:20.479 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.75 2024-09-19T08:53:20.479 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:53:20.740 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:53:21.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.221+0000 7f7931a85640 1 -- 172.21.15.164:0/261198263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f792c10c1a0 msgr2=0x7f792c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:21.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.221+0000 7f7931a85640 1 --2- 172.21.15.164:0/261198263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f792c10c1a0 0x7f792c10e610 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f7924009ca0 tx=0x7f792402a950 comp rx=0 tx=0).stop 2024-09-19T08:53:21.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.221+0000 7f7931a85640 1 -- 172.21.15.164:0/261198263 shutdown_connections 2024-09-19T08:53:21.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.221+0000 7f7931a85640 1 --2- 172.21.15.164:0/261198263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f792c10c1a0 0x7f792c10e610 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:21.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.221+0000 7f7931a85640 1 --2- 172.21.15.164:0/261198263 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f792c1036e0 0x7f792c10bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:21.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.221+0000 7f7931a85640 1 --2- 172.21.15.164:0/261198263 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f792c102dc0 0x7f792c1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:21.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.221+0000 7f7931a85640 1 -- 172.21.15.164:0/261198263 >> 172.21.15.164:0/261198263 conn(0x7f792c0fc7a0 msgr2=0x7f792c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:21.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.221+0000 7f7931a85640 1 -- 172.21.15.164:0/261198263 shutdown_connections 2024-09-19T08:53:21.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.221+0000 7f7931a85640 1 -- 172.21.15.164:0/261198263 wait complete. 2024-09-19T08:53:21.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.222+0000 7f7931a85640 1 Processor -- start 2024-09-19T08:53:21.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.222+0000 7f7931a85640 1 -- start start 2024-09-19T08:53:21.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.222+0000 7f7931a85640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f792c102dc0 0x7f792c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:21.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.223+0000 7f7931a85640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f792c1036e0 0x7f792c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:21.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.223+0000 7f7931a85640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f792c10c1a0 0x7f792c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:21.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.223+0000 7f7931a85640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f792c111f70 con 0x7f792c10c1a0 2024-09-19T08:53:21.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.223+0000 7f7931a85640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f792c111df0 con 0x7f792c102dc0 2024-09-19T08:53:21.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.223+0000 7f7931a85640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f792c1120f0 con 0x7f792c1036e0 2024-09-19T08:53:21.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.223+0000 7f792bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f792c10c1a0 0x7f792c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:21.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.223+0000 7f792bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f792c10c1a0 0x7f792c19cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58190/0 (socket says 172.21.15.164:58190) 2024-09-19T08:53:21.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.223+0000 7f792bfff640 1 -- 172.21.15.164:0/1869277427 learned_addr learned my addr 172.21.15.164:0/1869277427 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:21.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.223+0000 7f792affd640 1 --2- 172.21.15.164:0/1869277427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f792c1036e0 0x7f792c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:21.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.223+0000 7f792b7fe640 1 --2- 172.21.15.164:0/1869277427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f792c102dc0 0x7f792c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:21.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.223+0000 7f792bfff640 1 -- 172.21.15.164:0/1869277427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f792c1036e0 msgr2=0x7f792c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:21.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.223+0000 7f792bfff640 1 --2- 172.21.15.164:0/1869277427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f792c1036e0 0x7f792c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:21.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.223+0000 7f792bfff640 1 -- 172.21.15.164:0/1869277427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f792c102dc0 msgr2=0x7f792c198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:21.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.223+0000 7f792bfff640 1 --2- 172.21.15.164:0/1869277427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f792c102dc0 0x7f792c198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:21.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.223+0000 7f792bfff640 1 -- 172.21.15.164:0/1869277427 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792c19d420 con 0x7f792c10c1a0 2024-09-19T08:53:21.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.223+0000 7f792bfff640 1 --2- 172.21.15.164:0/1869277427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f792c10c1a0 0x7f792c19cd40 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f792402ae70 tx=0x7f792402f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:21.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.223+0000 7f7928ff9640 1 -- 172.21.15.164:0/1869277427 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7924034040 con 0x7f792c10c1a0 2024-09-19T08:53:21.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.223+0000 7f7928ff9640 1 -- 172.21.15.164:0/1869277427 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7924030920 con 0x7f792c10c1a0 2024-09-19T08:53:21.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.223+0000 7f7928ff9640 1 -- 172.21.15.164:0/1869277427 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79240047b0 con 0x7f792c10c1a0 2024-09-19T08:53:21.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.223+0000 7f7931a85640 1 -- 172.21.15.164:0/1869277427 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f792c19d6b0 con 0x7f792c10c1a0 2024-09-19T08:53:21.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.223+0000 7f7931a85640 1 -- 172.21.15.164:0/1869277427 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f792c1a4ff0 con 0x7f792c10c1a0 2024-09-19T08:53:21.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.224+0000 7f7931a85640 1 -- 172.21.15.164:0/1869277427 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f792c06aff0 con 0x7f792c10c1a0 2024-09-19T08:53:21.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.226+0000 7f7928ff9640 1 -- 172.21.15.164:0/1869277427 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f7924037020 con 0x7f792c10c1a0 2024-09-19T08:53:21.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.226+0000 7f7928ff9640 1 --2- 172.21.15.164:0/1869277427 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f79140a33b0 0x7f79140a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:21.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.226+0000 7f792b7fe640 1 --2- 172.21.15.164:0/1869277427 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f79140a33b0 0x7f79140a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:21.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.226+0000 7f7928ff9640 1 -- 172.21.15.164:0/1869277427 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f79240f0ae0 con 0x7f792c10c1a0 2024-09-19T08:53:21.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.227+0000 7f792b7fe640 1 --2- 172.21.15.164:0/1869277427 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f79140a33b0 0x7f79140a5870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f791c005fd0 tx=0x7f791c005850 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:21.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.230+0000 7f7928ff9640 1 -- 172.21.15.164:0/1869277427 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f79240bd5b0 con 0x7f792c10c1a0 2024-09-19T08:53:21.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.433+0000 7f7931a85640 1 -- 172.21.15.164:0/1869277427 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f792c107240 con 0x7f792c10c1a0 2024-09-19T08:53:21.433 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.434+0000 7f7928ff9640 1 -- 172.21.15.164:0/1869277427 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11984 (secure 0 0 0) 0x7f79240bceb0 con 0x7f792c10c1a0 2024-09-19T08:53:21.434 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:53:21.434 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":32,"btime":"2024-09-19T08:52:27:192804+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:52:27.074168+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:53:21.434 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 32 2024-09-19T08:53:21.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.436+0000 7f7931a85640 1 -- 172.21.15.164:0/1869277427 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f79140a33b0 msgr2=0x7f79140a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:21.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.436+0000 7f7931a85640 1 --2- 172.21.15.164:0/1869277427 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f79140a33b0 0x7f79140a5870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f791c005fd0 tx=0x7f791c005850 comp rx=0 tx=0).stop 2024-09-19T08:53:21.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.436+0000 7f7931a85640 1 -- 172.21.15.164:0/1869277427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f792c10c1a0 msgr2=0x7f792c19cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:21.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.436+0000 7f7931a85640 1 --2- 172.21.15.164:0/1869277427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f792c10c1a0 0x7f792c19cd40 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f792402ae70 tx=0x7f792402f5b0 comp rx=0 tx=0).stop 2024-09-19T08:53:21.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.436+0000 7f7931a85640 1 -- 172.21.15.164:0/1869277427 shutdown_connections 2024-09-19T08:53:21.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.436+0000 7f7931a85640 1 --2- 172.21.15.164:0/1869277427 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f79140a33b0 0x7f79140a5870 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:21.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.436+0000 7f7931a85640 1 --2- 172.21.15.164:0/1869277427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f792c10c1a0 0x7f792c19cd40 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:21.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.436+0000 7f7931a85640 1 --2- 172.21.15.164:0/1869277427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f792c1036e0 0x7f792c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:21.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.436+0000 7f7931a85640 1 --2- 172.21.15.164:0/1869277427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f792c102dc0 0x7f792c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:21.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.436+0000 7f7931a85640 1 -- 172.21.15.164:0/1869277427 >> 172.21.15.164:0/1869277427 conn(0x7f792c0fc7a0 msgr2=0x7f792c0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:21.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.436+0000 7f7931a85640 1 -- 172.21.15.164:0/1869277427 shutdown_connections 2024-09-19T08:53:21.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:21.436+0000 7f7931a85640 1 -- 172.21.15.164:0/1869277427 wait complete. 2024-09-19T08:53:22.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:21 smithi170 ceph-mon[35072]: pgmap v233: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.1 KiB/s wr, 14 op/s 2024-09-19T08:53:22.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:21 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/1869277427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:53:22.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:21 smithi173 ceph-mon[35306]: pgmap v233: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.1 KiB/s wr, 14 op/s 2024-09-19T08:53:22.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:21 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/1869277427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:53:22.245 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:21 smithi164 ceph-mon[30928]: pgmap v233: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.1 KiB/s wr, 14 op/s 2024-09-19T08:53:22.246 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:21 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/1869277427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:53:22.276 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 145.75 --await-for 45.0' 2024-09-19T08:53:22.276 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 145.75 --await-for 45.0 2024-09-19T08:53:22.524 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:53:22.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:22.998+0000 7f1a1b2b8640 1 -- 172.21.15.164:0/685320832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a14103670 msgr2=0x7f1a1410bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:23.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:22.998+0000 7f1a1b2b8640 1 --2- 172.21.15.164:0/685320832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a14103670 0x7f1a1410bb60 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f1a08009900 tx=0x7f1a0802a480 comp rx=0 tx=0).stop 2024-09-19T08:53:23.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:22.998+0000 7f1a1b2b8640 1 -- 172.21.15.164:0/685320832 shutdown_connections 2024-09-19T08:53:23.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:22.998+0000 7f1a1b2b8640 1 --2- 172.21.15.164:0/685320832 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a1410c180 0x7f1a1410e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:22.998+0000 7f1a1b2b8640 1 --2- 172.21.15.164:0/685320832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a14103670 0x7f1a1410bb60 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:22.998+0000 7f1a1b2b8640 1 --2- 172.21.15.164:0/685320832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a14102d50 0x7f1a14103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:22.998+0000 7f1a1b2b8640 1 -- 172.21.15.164:0/685320832 >> 172.21.15.164:0/685320832 conn(0x7f1a140fc7d0 msgr2=0x7f1a140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:23.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:22.998+0000 7f1a1b2b8640 1 -- 172.21.15.164:0/685320832 shutdown_connections 2024-09-19T08:53:23.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:22.998+0000 7f1a1b2b8640 1 -- 172.21.15.164:0/685320832 wait complete. 2024-09-19T08:53:23.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:22.999+0000 7f1a1b2b8640 1 Processor -- start 2024-09-19T08:53:23.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:22.999+0000 7f1a1b2b8640 1 -- start start 2024-09-19T08:53:23.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:22.999+0000 7f1a1b2b8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a14102d50 0x7f1a141009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:23.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:22.999+0000 7f1a1b2b8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a14103670 0x7f1a14100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:23.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:22.999+0000 7f1a1b2b8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a1410c180 0x7f1a14101430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:23.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:22.999+0000 7f1a1b2b8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1a14111f80 con 0x7f1a14103670 2024-09-19T08:53:23.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:22.999+0000 7f1a1b2b8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1a14111e00 con 0x7f1a14102d50 2024-09-19T08:53:23.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:22.999+0000 7f1a1b2b8640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1a14112100 con 0x7f1a1410c180 2024-09-19T08:53:23.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:22.999+0000 7f1a18853640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a14103670 0x7f1a14100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:23.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:22.999+0000 7f1a18853640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a14103670 0x7f1a14100ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58218/0 (socket says 172.21.15.164:58218) 2024-09-19T08:53:23.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:22.999+0000 7f1a18853640 1 -- 172.21.15.164:0/1581081412 learned_addr learned my addr 172.21.15.164:0/1581081412 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:23.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:22.999+0000 7f1a19855640 1 --2- 172.21.15.164:0/1581081412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a1410c180 0x7f1a14101430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:23.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:22.999+0000 7f1a19054640 1 --2- 172.21.15.164:0/1581081412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a14102d50 0x7f1a141009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:23.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:22.999+0000 7f1a18853640 1 -- 172.21.15.164:0/1581081412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a1410c180 msgr2=0x7f1a14101430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:23.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:22.999+0000 7f1a18853640 1 --2- 172.21.15.164:0/1581081412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a1410c180 0x7f1a14101430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:22.999+0000 7f1a18853640 1 -- 172.21.15.164:0/1581081412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a14102d50 msgr2=0x7f1a141009b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:23.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:22.999+0000 7f1a18853640 1 --2- 172.21.15.164:0/1581081412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a14102d50 0x7f1a141009b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:22.999+0000 7f1a18853640 1 -- 172.21.15.164:0/1581081412 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a141a68a0 con 0x7f1a14103670 2024-09-19T08:53:23.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.000+0000 7f1a18853640 1 --2- 172.21.15.164:0/1581081412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a14103670 0x7f1a14100ef0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f1a08006010 tx=0x7f1a080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:23.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.000+0000 7f1a127fc640 1 -- 172.21.15.164:0/1581081412 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0803f030 con 0x7f1a14103670 2024-09-19T08:53:23.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.000+0000 7f1a1b2b8640 1 -- 172.21.15.164:0/1581081412 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a141a6b30 con 0x7f1a14103670 2024-09-19T08:53:23.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.000+0000 7f1a127fc640 1 -- 172.21.15.164:0/1581081412 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a08003ce0 con 0x7f1a14103670 2024-09-19T08:53:23.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.000+0000 7f1a127fc640 1 -- 172.21.15.164:0/1581081412 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a08035070 con 0x7f1a14103670 2024-09-19T08:53:23.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.000+0000 7f1a1b2b8640 1 -- 172.21.15.164:0/1581081412 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a141a7060 con 0x7f1a14103670 2024-09-19T08:53:23.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.002+0000 7f1a127fc640 1 -- 172.21.15.164:0/1581081412 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f1a08042030 con 0x7f1a14103670 2024-09-19T08:53:23.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.003+0000 7f1a1d691740 1 Processor -- start 2024-09-19T08:53:23.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.003+0000 7f1a1d691740 1 -- start start 2024-09-19T08:53:23.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.003+0000 7f1a1d691740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b3e96850 0x55c4b3e96c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:23.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.003+0000 7f1a1d691740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4b3e97170 0x55c4b3ea14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:23.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.003+0000 7f1a1d691740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4b3ea1a30 0x55c4b3ea3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:23.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.003+0000 7f1a1d691740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c4b3dfb160 con 0x55c4b3ea1a30 2024-09-19T08:53:23.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.003+0000 7f1a1d691740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c4b3da2720 con 0x55c4b3e97170 2024-09-19T08:53:23.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.003+0000 7f1a1d691740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c4b3df7120 con 0x55c4b3e96850 2024-09-19T08:53:23.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.003+0000 7f1a127fc640 1 --2- 172.21.15.164:0/1581081412 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f19fc0a33b0 0x7f19fc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:23.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.003+0000 7f1a18853640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4b3ea1a30 0x55c4b3ea3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:23.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.003+0000 7f1a18853640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4b3ea1a30 0x55c4b3ea3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58226/0 (socket says 172.21.15.164:58226) 2024-09-19T08:53:23.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.003+0000 7f1a18853640 1 -- 172.21.15.164:0/472882433 learned_addr learned my addr 172.21.15.164:0/472882433 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:23.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.003+0000 7f1a19855640 1 --2- 172.21.15.164:0/1581081412 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f19fc0a33b0 0x7f19fc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:23.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.003+0000 7f1a19054640 1 --2- 172.21.15.164:0/472882433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4b3e97170 0x55c4b3ea14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:23.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.003+0000 7f1a19855640 1 --2- 172.21.15.164:0/472882433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b3e96850 0x55c4b3e96c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:23.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.003+0000 7f1a127fc640 1 -- 172.21.15.164:0/1581081412 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f1a080f0190 con 0x7f1a14103670 2024-09-19T08:53:23.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.003+0000 7f1a18853640 1 -- 172.21.15.164:0/472882433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b3e96850 msgr2=0x55c4b3e96c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:23.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.003+0000 7f1a18853640 1 --2- 172.21.15.164:0/472882433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b3e96850 0x55c4b3e96c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.003+0000 7f1a18853640 1 -- 172.21.15.164:0/472882433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4b3e97170 msgr2=0x55c4b3ea14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:23.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.003+0000 7f1a18853640 1 --2- 172.21.15.164:0/472882433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4b3e97170 0x55c4b3ea14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.004+0000 7f1a18853640 1 -- 172.21.15.164:0/472882433 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4b3b6cba0 con 0x55c4b3ea1a30 2024-09-19T08:53:23.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.004+0000 7f1a19054640 1 --2- 172.21.15.164:0/472882433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4b3e97170 0x55c4b3ea14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:53:23.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.004+0000 7f1a19855640 1 --2- 172.21.15.164:0/1581081412 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f19fc0a33b0 0x7f19fc0a5870 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c0127d0 tx=0x7f1a0c00b9e0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:23.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.004+0000 7f1a18853640 1 --2- 172.21.15.164:0/472882433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4b3ea1a30 0x55c4b3ea3e20 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f1a080b8a80 tx=0x7f1a0803b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ff9a1c33bf8851 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:23.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.004+0000 7f19fb7fe640 1 -- 172.21.15.164:0/472882433 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a08049070 con 0x55c4b3ea1a30 2024-09-19T08:53:23.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.004+0000 7f19fb7fe640 1 -- 172.21.15.164:0/472882433 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a080bc620 con 0x55c4b3ea1a30 2024-09-19T08:53:23.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.004+0000 7f19fb7fe640 1 -- 172.21.15.164:0/472882433 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a080b8c90 con 0x55c4b3ea1a30 2024-09-19T08:53:23.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.005+0000 7f1a1d691740 1 -- 172.21.15.164:0/472882433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4b3ea1a30 msgr2=0x55c4b3ea3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:23.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.005+0000 7f1a1d691740 1 --2- 172.21.15.164:0/472882433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4b3ea1a30 0x55c4b3ea3e20 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f1a080b8a80 tx=0x7f1a0803b5d0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.005+0000 7f1a1d691740 1 -- 172.21.15.164:0/472882433 shutdown_connections 2024-09-19T08:53:23.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.005+0000 7f1a1d691740 1 --2- 172.21.15.164:0/472882433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4b3ea1a30 0x55c4b3ea3e20 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.005+0000 7f1a1d691740 1 --2- 172.21.15.164:0/472882433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4b3e97170 0x55c4b3ea14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.005+0000 7f1a1d691740 1 --2- 172.21.15.164:0/472882433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b3e96850 0x55c4b3e96c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.005+0000 7f1a1d691740 1 -- 172.21.15.164:0/472882433 >> 172.21.15.164:0/472882433 conn(0x55c4b3e92de0 msgr2=0x55c4b3e931c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:23.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.005+0000 7f1a1d691740 1 -- 172.21.15.164:0/472882433 shutdown_connections 2024-09-19T08:53:23.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.005+0000 7f1a1d691740 1 -- 172.21.15.164:0/472882433 wait complete. 2024-09-19T08:53:23.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.005+0000 7f1a1d691740 1 -- start start 2024-09-19T08:53:23.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.005+0000 7f1a1d691740 1 Processor -- start 2024-09-19T08:53:23.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.006+0000 7f1a1d691740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b3e96850 0x55c4b3e9e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:23.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.006+0000 7f1a1d691740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4b3e97170 0x55c4b3e9ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:23.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.006+0000 7f1a1d691740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4b3ea1a30 0x55c4b3e9f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:23.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.006+0000 7f1a1d691740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c4b3ea43c0 con 0x55c4b3e97170 2024-09-19T08:53:23.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.006+0000 7f1a1d691740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c4b3d83900 con 0x55c4b3ea1a30 2024-09-19T08:53:23.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.006+0000 7f1a1d691740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c4b3ea4540 con 0x55c4b3e96850 2024-09-19T08:53:23.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.006+0000 7f1a1d691740 10 client.0 _ll_register_callbacks cb 0x55c4b3d81780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:23.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.006+0000 7f1a18853640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4b3e97170 0x55c4b3e9ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:23.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.006+0000 7f1a18853640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4b3e97170 0x55c4b3e9ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58232/0 (socket says 172.21.15.164:58232) 2024-09-19T08:53:23.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.006+0000 7f1a18853640 1 -- 172.21.15.164:0/696787639 learned_addr learned my addr 172.21.15.164:0/696787639 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:23.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.006+0000 7f1a19054640 1 --2- 172.21.15.164:0/696787639 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b3e96850 0x55c4b3e9e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:23.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.006+0000 7f1a18853640 1 -- 172.21.15.164:0/696787639 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b3e96850 msgr2=0x55c4b3e9e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:23.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.006+0000 7f1a18853640 1 --2- 172.21.15.164:0/696787639 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b3e96850 0x55c4b3e9e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.006+0000 7f1a18853640 1 -- 172.21.15.164:0/696787639 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4b3ea1a30 msgr2=0x55c4b3e9f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:23.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.006+0000 7f1a18853640 1 --2- 172.21.15.164:0/696787639 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4b3ea1a30 0x55c4b3e9f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.006+0000 7f1a18853640 1 -- 172.21.15.164:0/696787639 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4b3dc3c50 con 0x55c4b3e97170 2024-09-19T08:53:23.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.006+0000 7f1a18853640 1 --2- 172.21.15.164:0/696787639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4b3e97170 0x55c4b3e9ee80 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f1a0803b580 tx=0x7f1a080eddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f776eac3630da1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:23.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.006+0000 7f19f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:53:23.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.006+0000 7f19f8ff9640 1 -- 172.21.15.164:0/696787639 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a08049040 con 0x55c4b3e97170 2024-09-19T08:53:23.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.006+0000 7f19f8ff9640 1 -- 172.21.15.164:0/696787639 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f1a080bc880 con 0x55c4b3e97170 2024-09-19T08:53:23.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.006+0000 7f1a1d691740 1 -- 172.21.15.164:0/696787639 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a080bc620 con 0x55c4b3e97170 2024-09-19T08:53:23.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.006+0000 7f19f8ff9640 1 -- 172.21.15.164:0/696787639 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1a080bbf80 con 0x55c4b3e97170 2024-09-19T08:53:23.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.007+0000 7f1a1d691740 10 client.17043 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:23.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.007+0000 7f1a1d691740 1 -- 172.21.15.164:0/696787639 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c4b3ea43c0 con 0x55c4b3e97170 2024-09-19T08:53:23.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.007+0000 7f19f8ff9640 1 -- 172.21.15.164:0/696787639 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a080bc620 con 0x55c4b3e97170 2024-09-19T08:53:23.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.007+0000 7f19f8ff9640 1 -- 172.21.15.164:0/696787639 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f1a080c4d50 con 0x55c4b3e97170 2024-09-19T08:53:23.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.007+0000 7f1a1d691740 10 client.17043 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:23.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.007+0000 7f1a1d691740 10 client.17043 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:53:23.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.007+0000 7f1a1d691740 20 client.17043 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:23.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.007+0000 7f1a1d691740 1 --2- 172.21.15.164:0/696787639 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c4b3eb12f0 0x55c4b3eb3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:23.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.007+0000 7f1a1d691740 4 client.17043 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:53:23.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.007+0000 7f1a1d691740 1 -- 172.21.15.164:0/696787639 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4b3e08890 con 0x55c4b3eb12f0 2024-09-19T08:53:23.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.007+0000 7f1a19054640 1 --2- 172.21.15.164:0/696787639 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c4b3eb12f0 0x55c4b3eb3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:23.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.008+0000 7f19f8ff9640 10 client.17043 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:23.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.008+0000 7f1a19054640 1 --2- 172.21.15.164:0/696787639 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c4b3eb12f0 0x55c4b3eb3710 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7170e82b91a03dd server_cookie=3e73406b8748a694 in_seq=0 out_seq=0 2024-09-19T08:53:23.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.008+0000 7f19f8ff9640 1 -- 172.21.15.164:0/696787639 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c4b3e08890 con 0x55c4b3eb12f0 2024-09-19T08:53:23.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.008+0000 7f19f8ff9640 10 client.17043 handle_command_reply: tid=0 2024-09-19T08:53:23.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.008+0000 7f1a1d691740 1 client.17043 shutdown 2024-09-19T08:53:23.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.008+0000 7f1a1d691740 1 -- 172.21.15.164:0/696787639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4b3e97170 msgr2=0x55c4b3e9ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:23.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.008+0000 7f1a1d691740 1 --2- 172.21.15.164:0/696787639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4b3e97170 0x55c4b3e9ee80 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f1a0803b580 tx=0x7f1a080eddd0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.008+0000 7f1a1d691740 1 -- 172.21.15.164:0/696787639 shutdown_connections 2024-09-19T08:53:23.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.009+0000 7f1a1d691740 1 --2- 172.21.15.164:0/696787639 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c4b3eb12f0 0x55c4b3eb3710 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.009+0000 7f1a1d691740 1 --2- 172.21.15.164:0/696787639 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4b3ea1a30 0x55c4b3e9f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.009+0000 7f1a1d691740 1 --2- 172.21.15.164:0/696787639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4b3e97170 0x55c4b3e9ee80 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.009+0000 7f1a1d691740 1 --2- 172.21.15.164:0/696787639 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b3e96850 0x55c4b3e9e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.009+0000 7f1a1d691740 1 -- 172.21.15.164:0/696787639 >> 172.21.15.164:0/696787639 conn(0x55c4b3e92de0 msgr2=0x55c4b3e98ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:23.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.009+0000 7f19f8ff9640 0 client.17043 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:23.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.009+0000 7f1a1d691740 1 -- 172.21.15.164:0/696787639 shutdown_connections 2024-09-19T08:53:23.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.009+0000 7f1a1d691740 1 -- 172.21.15.164:0/696787639 wait complete. 2024-09-19T08:53:23.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.009+0000 7f1a1d691740 20 client.17043 trim_cache size 0 max 16384 2024-09-19T08:53:23.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.025+0000 7f1a1d691740 1 Processor -- start 2024-09-19T08:53:23.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.025+0000 7f1a1d691740 1 -- start start 2024-09-19T08:53:23.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.026+0000 7f1a1d691740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4b3e96850 0x55c4b3e931c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:23.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.026+0000 7f1a1d691740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b3e97170 0x55c4b3e9f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:23.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.026+0000 7f1a1d691740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4b3ea1a30 0x55c4b3eb8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:23.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.026+0000 7f1a1d691740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1a141af9a0 con 0x55c4b3ea1a30 2024-09-19T08:53:23.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.026+0000 7f1a1d691740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1a141b14e0 con 0x55c4b3e96850 2024-09-19T08:53:23.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.026+0000 7f1a1d691740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1a141b0070 con 0x55c4b3e97170 2024-09-19T08:53:23.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.026+0000 7f1a19855640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4b3ea1a30 0x55c4b3eb8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:23.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.026+0000 7f1a19855640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4b3ea1a30 0x55c4b3eb8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58234/0 (socket says 172.21.15.164:58234) 2024-09-19T08:53:23.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.026+0000 7f1a19855640 1 -- 172.21.15.164:0/1958630252 learned_addr learned my addr 172.21.15.164:0/1958630252 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:23.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.026+0000 7f1a18853640 1 --2- 172.21.15.164:0/1958630252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b3e97170 0x55c4b3e9f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:23.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.026+0000 7f1a19855640 1 -- 172.21.15.164:0/1958630252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b3e97170 msgr2=0x55c4b3e9f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:23.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.026+0000 7f1a19855640 1 --2- 172.21.15.164:0/1958630252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b3e97170 0x55c4b3e9f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.026+0000 7f1a19855640 1 -- 172.21.15.164:0/1958630252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4b3e96850 msgr2=0x55c4b3e931c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:23.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.026+0000 7f1a19855640 1 --2- 172.21.15.164:0/1958630252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4b3e96850 0x55c4b3e931c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.026+0000 7f1a19855640 1 -- 172.21.15.164:0/1958630252 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4b3e9a2b0 con 0x55c4b3ea1a30 2024-09-19T08:53:23.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.026+0000 7f1a18853640 1 --2- 172.21.15.164:0/1958630252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b3e97170 0x55c4b3e9f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:53:23.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.026+0000 7f1a19855640 1 --2- 172.21.15.164:0/1958630252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4b3ea1a30 0x55c4b3eb8fc0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c00be00 tx=0x7f1a0c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeeeae19b418207f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:23.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.026+0000 7f19faffd640 1 -- 172.21.15.164:0/1958630252 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c024070 con 0x55c4b3ea1a30 2024-09-19T08:53:23.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.026+0000 7f19faffd640 1 -- 172.21.15.164:0/1958630252 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a0c00be30 con 0x55c4b3ea1a30 2024-09-19T08:53:23.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.026+0000 7f19faffd640 1 -- 172.21.15.164:0/1958630252 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c029820 con 0x55c4b3ea1a30 2024-09-19T08:53:23.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.027+0000 7f1a1d691740 1 -- 172.21.15.164:0/1958630252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4b3ea1a30 msgr2=0x55c4b3eb8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:23.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.027+0000 7f1a1d691740 1 --2- 172.21.15.164:0/1958630252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4b3ea1a30 0x55c4b3eb8fc0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c00be00 tx=0x7f1a0c01c000 comp rx=0 tx=0).stop 2024-09-19T08:53:23.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.027+0000 7f1a1d691740 1 -- 172.21.15.164:0/1958630252 shutdown_connections 2024-09-19T08:53:23.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.027+0000 7f1a1d691740 1 --2- 172.21.15.164:0/1958630252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4b3ea1a30 0x55c4b3eb8fc0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.027+0000 7f1a1d691740 1 --2- 172.21.15.164:0/1958630252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b3e97170 0x55c4b3e9f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.027+0000 7f1a1d691740 1 --2- 172.21.15.164:0/1958630252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4b3e96850 0x55c4b3e931c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.027+0000 7f1a1d691740 1 -- 172.21.15.164:0/1958630252 >> 172.21.15.164:0/1958630252 conn(0x55c4b3e92de0 msgr2=0x55c4b3ea09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:23.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.027+0000 7f1a1d691740 1 -- 172.21.15.164:0/1958630252 shutdown_connections 2024-09-19T08:53:23.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.027+0000 7f1a1d691740 1 -- 172.21.15.164:0/1958630252 wait complete. 2024-09-19T08:53:23.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.027+0000 7f1a1d691740 1 -- start start 2024-09-19T08:53:23.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.027+0000 7f1a1d691740 1 Processor -- start 2024-09-19T08:53:23.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.028+0000 7f1a1d691740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b3e96850 0x55c4b3eaa010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:23.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.028+0000 7f1a1d691740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4b3e97170 0x55c4b3ea5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:23.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.028+0000 7f1a1d691740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4b3ea1a30 0x55c4b3ea5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:23.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.028+0000 7f1a1d691740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1a141b0e10 con 0x55c4b3ea1a30 2024-09-19T08:53:23.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.028+0000 7f1a1d691740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1a141b0740 con 0x55c4b3e97170 2024-09-19T08:53:23.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.028+0000 7f1a1d691740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1a141ad9a0 con 0x55c4b3e96850 2024-09-19T08:53:23.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.028+0000 7f1a1d691740 10 client.0 _ll_register_callbacks cb 0x7f1a04007910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:23.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.028+0000 7f1a19855640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4b3ea1a30 0x55c4b3ea5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:23.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.028+0000 7f1a19855640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4b3ea1a30 0x55c4b3ea5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58244/0 (socket says 172.21.15.164:58244) 2024-09-19T08:53:23.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.028+0000 7f1a19855640 1 -- 172.21.15.164:0/621308453 learned_addr learned my addr 172.21.15.164:0/621308453 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:23.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.028+0000 7f1a18853640 1 --2- 172.21.15.164:0/621308453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4b3e97170 0x55c4b3ea5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:23.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.028+0000 7f1a19054640 1 --2- 172.21.15.164:0/621308453 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b3e96850 0x55c4b3eaa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:23.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.028+0000 7f1a19855640 1 -- 172.21.15.164:0/621308453 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b3e96850 msgr2=0x55c4b3eaa010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:23.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.028+0000 7f1a19855640 1 --2- 172.21.15.164:0/621308453 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b3e96850 0x55c4b3eaa010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.028+0000 7f1a19855640 1 -- 172.21.15.164:0/621308453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4b3e97170 msgr2=0x55c4b3ea5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:23.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.028+0000 7f1a19855640 1 --2- 172.21.15.164:0/621308453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4b3e97170 0x55c4b3ea5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.028+0000 7f1a19855640 1 -- 172.21.15.164:0/621308453 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4b3f1af90 con 0x55c4b3ea1a30 2024-09-19T08:53:23.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.028+0000 7f1a19855640 1 --2- 172.21.15.164:0/621308453 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4b3ea1a30 0x55c4b3ea5b20 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c00b320 tx=0x7f1a0c010f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d6ae26e72c4c1bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:23.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.028+0000 7f19f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:53:23.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.029+0000 7f19f8ff9640 1 -- 172.21.15.164:0/621308453 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c024040 con 0x55c4b3ea1a30 2024-09-19T08:53:23.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.029+0000 7f19f8ff9640 1 -- 172.21.15.164:0/621308453 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f1a0c005040 con 0x55c4b3ea1a30 2024-09-19T08:53:23.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.029+0000 7f1a1d691740 1 -- 172.21.15.164:0/621308453 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a0c00be30 con 0x55c4b3ea1a30 2024-09-19T08:53:23.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.029+0000 7f19f8ff9640 1 -- 172.21.15.164:0/621308453 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1a0c003a80 con 0x55c4b3ea1a30 2024-09-19T08:53:23.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.029+0000 7f1a1d691740 10 client.17049 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:23.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.029+0000 7f1a1d691740 1 -- 172.21.15.164:0/621308453 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c4b3f1c0a0 con 0x55c4b3ea1a30 2024-09-19T08:53:23.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.029+0000 7f19f8ff9640 1 -- 172.21.15.164:0/621308453 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a0c00be30 con 0x55c4b3ea1a30 2024-09-19T08:53:23.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.029+0000 7f19f8ff9640 1 -- 172.21.15.164:0/621308453 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f1a0c01f070 con 0x55c4b3ea1a30 2024-09-19T08:53:23.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.029+0000 7f1a1d691740 10 client.17049 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:23.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.029+0000 7f1a1d691740 10 client.17049 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:53:23.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.029+0000 7f1a1d691740 20 client.17049 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:23.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.029+0000 7f1a1d691740 1 --2- 172.21.15.164:0/621308453 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c4b3f1c790 0x55c4b3f1ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:23.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.029+0000 7f1a1d691740 4 client.17049 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.75, "await_for": 45.0}] 2024-09-19T08:53:23.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.029+0000 7f1a1d691740 1 -- 172.21.15.164:0/621308453 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.75, "await_for": 45.0}) -- 0x55c4b3e02be0 con 0x55c4b3f1c790 2024-09-19T08:53:23.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.030+0000 7f1a19054640 1 --2- 172.21.15.164:0/621308453 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c4b3f1c790 0x55c4b3f1ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:23.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.030+0000 7f19f8ff9640 10 client.17049 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:23.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.030+0000 7f1a19054640 1 --2- 172.21.15.164:0/621308453 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c4b3f1c790 0x55c4b3f1ebb0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d71443fd8aac9f1 server_cookie=bc15e5cc12eee0bc in_seq=0 out_seq=0 2024-09-19T08:53:23.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.055+0000 7f19f8ff9640 1 -- 172.21.15.164:0/621308453 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c4b3e02be0 con 0x55c4b3f1c790 2024-09-19T08:53:23.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.055+0000 7f19f8ff9640 10 client.17049 handle_command_reply: tid=0 2024-09-19T08:53:23.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.055+0000 7f1a1d691740 1 client.17049 shutdown 2024-09-19T08:53:23.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.055+0000 7f1a1d691740 1 -- 172.21.15.164:0/621308453 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4b3ea1a30 msgr2=0x55c4b3ea5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:23.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.055+0000 7f1a1d691740 1 --2- 172.21.15.164:0/621308453 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4b3ea1a30 0x55c4b3ea5b20 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c00b320 tx=0x7f1a0c010f70 comp rx=0 tx=0).stop 2024-09-19T08:53:23.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.055+0000 7f1a1d691740 1 -- 172.21.15.164:0/621308453 shutdown_connections 2024-09-19T08:53:23.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.055+0000 7f1a1d691740 1 --2- 172.21.15.164:0/621308453 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4b3ea1a30 0x55c4b3ea5b20 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.055+0000 7f1a1d691740 1 --2- 172.21.15.164:0/621308453 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c4b3f1c790 0x55c4b3f1ebb0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.055+0000 7f1a1d691740 1 --2- 172.21.15.164:0/621308453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4b3e97170 0x55c4b3ea5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.055+0000 7f1a1d691740 1 --2- 172.21.15.164:0/621308453 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b3e96850 0x55c4b3eaa010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.055+0000 7f1a1d691740 1 -- 172.21.15.164:0/621308453 >> 172.21.15.164:0/621308453 conn(0x55c4b3e92de0 msgr2=0x55c4b3e982a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:23.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.055+0000 7f19f8ff9640 0 client.17049 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:23.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.055+0000 7f1a1d691740 1 -- 172.21.15.164:0/621308453 shutdown_connections 2024-09-19T08:53:23.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.055+0000 7f1a1d691740 1 -- 172.21.15.164:0/621308453 wait complete. 2024-09-19T08:53:23.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.055+0000 7f1a1d691740 20 client.17049 trim_cache size 0 max 16384 2024-09-19T08:53:23.056 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:53:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "epoch": 32, 2024-09-19T08:53:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "leader": 24479, 2024-09-19T08:53:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "set_version": 26, 2024-09-19T08:53:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "sets": { 2024-09-19T08:53:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "d0313327": { 2024-09-19T08:53:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "version": 26, 2024-09-19T08:53:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "age_ref": 0.0, 2024-09-19T08:53:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:53:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "QUIESCED", 2024-09-19T08:53:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:53:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:53:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "timeout": 90.0, 2024-09-19T08:53:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "expiration": 145.8, 2024-09-19T08:53:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: "members": { 2024-09-19T08:53:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: "file:/": { 2024-09-19T08:53:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: "excluded": false, 2024-09-19T08:53:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:53:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "QUIESCED", 2024-09-19T08:53:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:53:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:53:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:53:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:53:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:53:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:53:23.059 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:53:23.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.056+0000 7f1a1b2b8640 1 -- 172.21.15.164:0/1581081412 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f19fc0a33b0 msgr2=0x7f19fc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:23.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.056+0000 7f1a1b2b8640 1 --2- 172.21.15.164:0/1581081412 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f19fc0a33b0 0x7f19fc0a5870 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c0127d0 tx=0x7f1a0c00b9e0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.056+0000 7f1a1b2b8640 1 -- 172.21.15.164:0/1581081412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a14103670 msgr2=0x7f1a14100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:23.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.056+0000 7f1a1b2b8640 1 --2- 172.21.15.164:0/1581081412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a14103670 0x7f1a14100ef0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f1a08006010 tx=0x7f1a080042f0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.056+0000 7f1a1b2b8640 1 -- 172.21.15.164:0/1581081412 shutdown_connections 2024-09-19T08:53:23.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.056+0000 7f1a1b2b8640 1 --2- 172.21.15.164:0/1581081412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a1410c180 0x7f1a14101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.056+0000 7f1a1b2b8640 1 --2- 172.21.15.164:0/1581081412 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f19fc0a33b0 0x7f19fc0a5870 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.056+0000 7f1a1b2b8640 1 --2- 172.21.15.164:0/1581081412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a14103670 0x7f1a14100ef0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.056+0000 7f1a1b2b8640 1 --2- 172.21.15.164:0/1581081412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a14102d50 0x7f1a141009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:23.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.056+0000 7f1a1b2b8640 1 -- 172.21.15.164:0/1581081412 >> 172.21.15.164:0/1581081412 conn(0x7f1a140fc7d0 msgr2=0x7f1a140fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:23.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.056+0000 7f1a1b2b8640 1 -- 172.21.15.164:0/1581081412 shutdown_connections 2024-09-19T08:53:23.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:23.056+0000 7f1a1b2b8640 1 -- 172.21.15.164:0/1581081412 wait complete. 2024-09-19T08:53:24.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:23 smithi164 ceph-mon[30928]: pgmap v234: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.1 KiB/s wr, 13 op/s 2024-09-19T08:53:24.066 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd0313327', quiesce took 3.6 seconds. Will release after: 22.15 2024-09-19T08:53:24.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:23 smithi170 ceph-mon[35072]: pgmap v234: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.1 KiB/s wr, 13 op/s 2024-09-19T08:53:24.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:23 smithi173 ceph-mon[35306]: pgmap v234: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.1 KiB/s wr, 13 op/s 2024-09-19T08:53:26.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:25 smithi170 ceph-mon[35072]: pgmap v235: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.1 KiB/s wr, 11 op/s 2024-09-19T08:53:26.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:25 smithi173 ceph-mon[35306]: pgmap v235: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.1 KiB/s wr, 11 op/s 2024-09-19T08:53:26.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:25 smithi164 ceph-mon[30928]: pgmap v235: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.1 KiB/s wr, 11 op/s 2024-09-19T08:53:28.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:27 smithi170 ceph-mon[35072]: pgmap v236: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.1 KiB/s wr, 9 op/s 2024-09-19T08:53:28.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:27 smithi173 ceph-mon[35306]: pgmap v236: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.1 KiB/s wr, 9 op/s 2024-09-19T08:53:28.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:27 smithi164 ceph-mon[30928]: pgmap v236: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.1 KiB/s wr, 9 op/s 2024-09-19T08:53:30.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:29 smithi173 ceph-mon[35306]: pgmap v237: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-09-19T08:53:30.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:29 smithi164 ceph-mon[30928]: pgmap v237: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-09-19T08:53:30.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:29 smithi170 ceph-mon[35072]: pgmap v237: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-09-19T08:53:32.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:31 smithi173 ceph-mon[35306]: pgmap v238: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-09-19T08:53:32.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:31 smithi164 ceph-mon[30928]: pgmap v238: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-09-19T08:53:32.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:31 smithi170 ceph-mon[35072]: pgmap v238: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-09-19T08:53:34.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:33 smithi173 ceph-mon[35306]: pgmap v239: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:53:34.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:33 smithi164 ceph-mon[30928]: pgmap v239: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:53:34.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:33 smithi170 ceph-mon[35072]: pgmap v239: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:53:36.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:35 smithi173 ceph-mon[35306]: pgmap v240: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-19T08:53:36.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:35 smithi164 ceph-mon[30928]: pgmap v240: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-19T08:53:36.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:35 smithi170 ceph-mon[35072]: pgmap v240: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-19T08:53:38.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:37 smithi173 ceph-mon[35306]: pgmap v241: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 op/s 2024-09-19T08:53:38.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:37 smithi164 ceph-mon[30928]: pgmap v241: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 op/s 2024-09-19T08:53:38.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:37 smithi170 ceph-mon[35072]: pgmap v241: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 op/s 2024-09-19T08:53:40.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:39 smithi173 ceph-mon[35306]: pgmap v242: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:53:40.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:39 smithi164 ceph-mon[30928]: pgmap v242: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:53:40.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:39 smithi170 ceph-mon[35072]: pgmap v242: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:53:42.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:41 smithi173 ceph-mon[35306]: pgmap v243: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:53:42.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:41 smithi164 ceph-mon[30928]: pgmap v243: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:53:42.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:41 smithi170 ceph-mon[35072]: pgmap v243: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:53:42.701 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:53:42.969 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:53:43.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.498+0000 7fd06ca12640 1 -- 172.21.15.164:0/422971906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd068103660 msgr2=0x7fd06810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:43.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.498+0000 7fd06ca12640 1 --2- 172.21.15.164:0/422971906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd068103660 0x7fd06810bb50 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fd05c009900 tx=0x7fd05c02a480 comp rx=0 tx=0).stop 2024-09-19T08:53:43.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.498+0000 7fd06ca12640 1 -- 172.21.15.164:0/422971906 shutdown_connections 2024-09-19T08:53:43.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.498+0000 7fd06ca12640 1 --2- 172.21.15.164:0/422971906 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd06810c170 0x7fd06810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.498+0000 7fd06ca12640 1 --2- 172.21.15.164:0/422971906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd068103660 0x7fd06810bb50 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.498+0000 7fd06ca12640 1 --2- 172.21.15.164:0/422971906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd068102d40 0x7fd068103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.498+0000 7fd06ca12640 1 -- 172.21.15.164:0/422971906 >> 172.21.15.164:0/422971906 conn(0x7fd0680fc7a0 msgr2=0x7fd0680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:43.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.498+0000 7fd06ca12640 1 -- 172.21.15.164:0/422971906 shutdown_connections 2024-09-19T08:53:43.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.498+0000 7fd06ca12640 1 -- 172.21.15.164:0/422971906 wait complete. 2024-09-19T08:53:43.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.499+0000 7fd06ca12640 1 Processor -- start 2024-09-19T08:53:43.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.499+0000 7fd06ca12640 1 -- start start 2024-09-19T08:53:43.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.499+0000 7fd06ca12640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd068102d40 0x7fd0681983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:43.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.499+0000 7fd06ca12640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd068103660 0x7fd068198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:43.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.499+0000 7fd06ca12640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd06810c170 0x7fd06819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:43.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.499+0000 7fd06ca12640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd068111f70 con 0x7fd068102d40 2024-09-19T08:53:43.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.499+0000 7fd06ca12640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd068111df0 con 0x7fd06810c170 2024-09-19T08:53:43.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.499+0000 7fd06ca12640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd0681120f0 con 0x7fd068103660 2024-09-19T08:53:43.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.499+0000 7fd06659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd068102d40 0x7fd0681983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:43.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.499+0000 7fd06659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd068102d40 0x7fd0681983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47912/0 (socket says 172.21.15.164:47912) 2024-09-19T08:53:43.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.499+0000 7fd06659c640 1 -- 172.21.15.164:0/3820256109 learned_addr learned my addr 172.21.15.164:0/3820256109 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:43.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.499+0000 7fd066d9d640 1 --2- 172.21.15.164:0/3820256109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd06810c170 0x7fd06819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:43.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.499+0000 7fd065d9b640 1 --2- 172.21.15.164:0/3820256109 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd068103660 0x7fd068198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:43.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.500+0000 7fd06659c640 1 -- 172.21.15.164:0/3820256109 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd068103660 msgr2=0x7fd068198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:43.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.500+0000 7fd06659c640 1 --2- 172.21.15.164:0/3820256109 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd068103660 0x7fd068198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.500+0000 7fd06659c640 1 -- 172.21.15.164:0/3820256109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd06810c170 msgr2=0x7fd06819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:43.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.500+0000 7fd06659c640 1 --2- 172.21.15.164:0/3820256109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd06810c170 0x7fd06819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.500+0000 7fd06659c640 1 -- 172.21.15.164:0/3820256109 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd06819d380 con 0x7fd068102d40 2024-09-19T08:53:43.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.500+0000 7fd06659c640 1 --2- 172.21.15.164:0/3820256109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd068102d40 0x7fd0681983d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fd05800c920 tx=0x7fd05800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:43.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.500+0000 7fd0577fe640 1 -- 172.21.15.164:0/3820256109 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058007c70 con 0x7fd068102d40 2024-09-19T08:53:43.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.500+0000 7fd06ca12640 1 -- 172.21.15.164:0/3820256109 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd06819d670 con 0x7fd068102d40 2024-09-19T08:53:43.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.500+0000 7fd0577fe640 1 -- 172.21.15.164:0/3820256109 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd058007e10 con 0x7fd068102d40 2024-09-19T08:53:43.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.500+0000 7fd0577fe640 1 -- 172.21.15.164:0/3820256109 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058018890 con 0x7fd068102d40 2024-09-19T08:53:43.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.500+0000 7fd06ca12640 1 -- 172.21.15.164:0/3820256109 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0681a4f50 con 0x7fd068102d40 2024-09-19T08:53:43.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.502+0000 7fd0577fe640 1 -- 172.21.15.164:0/3820256109 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fd058018a70 con 0x7fd068102d40 2024-09-19T08:53:43.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.502+0000 7fd06edeb740 1 Processor -- start 2024-09-19T08:53:43.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.503+0000 7fd06edeb740 1 -- start start 2024-09-19T08:53:43.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.503+0000 7fd06edeb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c74d159380 0x55c74d159760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:43.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.503+0000 7fd0577fe640 1 --2- 172.21.15.164:0/3820256109 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd0500a33b0 0x7fd0500a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:43.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.503+0000 7fd06edeb740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c74d159ca0 0x55c74d168550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:43.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.503+0000 7fd06edeb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c74d168a90 0x55c74d16ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:43.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.503+0000 7fd06edeb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c74d0bf2e0 con 0x55c74d168a90 2024-09-19T08:53:43.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.503+0000 7fd06edeb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c74d066ce0 con 0x55c74d159380 2024-09-19T08:53:43.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.503+0000 7fd06edeb740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c74d0bb2a0 con 0x55c74d159ca0 2024-09-19T08:53:43.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.503+0000 7fd066d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c74d168a90 0x55c74d16ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:43.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.503+0000 7fd06659c640 1 --2- 172.21.15.164:0/3820256109 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd0500a33b0 0x7fd0500a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:43.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.503+0000 7fd066d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c74d168a90 0x55c74d16ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47914/0 (socket says 172.21.15.164:47914) 2024-09-19T08:53:43.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.503+0000 7fd066d9d640 1 -- 172.21.15.164:0/4100749085 learned_addr learned my addr 172.21.15.164:0/4100749085 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:43.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.503+0000 7fd066d9d640 1 --2- 172.21.15.164:0/4100749085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c74d159380 0x55c74d159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:43.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.503+0000 7fd065d9b640 1 --2- 172.21.15.164:0/4100749085 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c74d159ca0 0x55c74d168550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:43.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.503+0000 7fd0577fe640 1 -- 172.21.15.164:0/3820256109 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7fd0580cfde0 con 0x7fd068102d40 2024-09-19T08:53:43.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.503+0000 7fd066d9d640 1 -- 172.21.15.164:0/4100749085 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c74d159ca0 msgr2=0x55c74d168550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:43.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.503+0000 7fd066d9d640 1 --2- 172.21.15.164:0/4100749085 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c74d159ca0 0x55c74d168550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.503+0000 7fd066d9d640 1 -- 172.21.15.164:0/4100749085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c74d159380 msgr2=0x55c74d159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:43.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.503+0000 7fd066d9d640 1 --2- 172.21.15.164:0/4100749085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c74d159380 0x55c74d159760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.503+0000 7fd066d9d640 1 -- 172.21.15.164:0/4100749085 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c74d04b440 con 0x55c74d168a90 2024-09-19T08:53:43.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.504+0000 7fd066d9d640 1 --2- 172.21.15.164:0/4100749085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c74d168a90 0x55c74d16ae80 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fd060012a20 tx=0x7fd0600099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b744f5e744e9f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:43.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.504+0000 7fd06659c640 1 --2- 172.21.15.164:0/3820256109 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd0500a33b0 0x7fd0500a5870 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd058098500 tx=0x7fd058000bc0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:43.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.504+0000 7fd054ff9640 1 -- 172.21.15.164:0/4100749085 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd060003970 con 0x55c74d168a90 2024-09-19T08:53:43.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.504+0000 7fd054ff9640 1 -- 172.21.15.164:0/4100749085 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd060010770 con 0x55c74d168a90 2024-09-19T08:53:43.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.504+0000 7fd054ff9640 1 -- 172.21.15.164:0/4100749085 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd060004450 con 0x55c74d168a90 2024-09-19T08:53:43.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.505+0000 7fd06edeb740 1 -- 172.21.15.164:0/4100749085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c74d168a90 msgr2=0x55c74d16ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:43.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.505+0000 7fd06edeb740 1 --2- 172.21.15.164:0/4100749085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c74d168a90 0x55c74d16ae80 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fd060012a20 tx=0x7fd0600099f0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.505+0000 7fd06edeb740 1 -- 172.21.15.164:0/4100749085 shutdown_connections 2024-09-19T08:53:43.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.505+0000 7fd06edeb740 1 --2- 172.21.15.164:0/4100749085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c74d168a90 0x55c74d16ae80 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.505+0000 7fd06edeb740 1 --2- 172.21.15.164:0/4100749085 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c74d159ca0 0x55c74d168550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.505+0000 7fd06edeb740 1 --2- 172.21.15.164:0/4100749085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c74d159380 0x55c74d159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.505+0000 7fd06edeb740 1 -- 172.21.15.164:0/4100749085 >> 172.21.15.164:0/4100749085 conn(0x55c74d155a30 msgr2=0x55c74d155e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:43.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.505+0000 7fd06edeb740 1 -- 172.21.15.164:0/4100749085 shutdown_connections 2024-09-19T08:53:43.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.505+0000 7fd06edeb740 1 -- 172.21.15.164:0/4100749085 wait complete. 2024-09-19T08:53:43.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.505+0000 7fd06edeb740 1 -- start start 2024-09-19T08:53:43.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.505+0000 7fd06edeb740 1 Processor -- start 2024-09-19T08:53:43.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.505+0000 7fd06edeb740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c74d159380 0x55c74d168460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:43.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.505+0000 7fd06edeb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c74d159ca0 0x55c74d165760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:43.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.505+0000 7fd06edeb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c74d168a90 0x55c74d165dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:43.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.505+0000 7fd06edeb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c74d16b660 con 0x55c74d159ca0 2024-09-19T08:53:43.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.505+0000 7fd06edeb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c74d07a050 con 0x55c74d168a90 2024-09-19T08:53:43.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.505+0000 7fd06edeb740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c74d16b7e0 con 0x55c74d159380 2024-09-19T08:53:43.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.505+0000 7fd06edeb740 10 client.0 _ll_register_callbacks cb 0x55c74d0ba160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:43.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.505+0000 7fd065d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c74d159ca0 0x55c74d165760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:43.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.506+0000 7fd065d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c74d159ca0 0x55c74d165760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47916/0 (socket says 172.21.15.164:47916) 2024-09-19T08:53:43.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.506+0000 7fd065d9b640 1 -- 172.21.15.164:0/742163339 learned_addr learned my addr 172.21.15.164:0/742163339 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:43.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.506+0000 7fd066d9d640 1 --2- 172.21.15.164:0/742163339 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c74d159380 0x55c74d168460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:43.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.506+0000 7fd065d9b640 1 -- 172.21.15.164:0/742163339 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c74d159380 msgr2=0x55c74d168460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:43.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.506+0000 7fd065d9b640 1 --2- 172.21.15.164:0/742163339 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c74d159380 0x55c74d168460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.506+0000 7fd066d9d640 1 --2- 172.21.15.164:0/742163339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c74d168a90 0x55c74d165dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:43.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.506+0000 7fd065d9b640 1 -- 172.21.15.164:0/742163339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c74d168a90 msgr2=0x55c74d165dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:43.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.506+0000 7fd065d9b640 1 --2- 172.21.15.164:0/742163339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c74d168a90 0x55c74d165dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.506+0000 7fd065d9b640 1 -- 172.21.15.164:0/742163339 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c74ce303c0 con 0x55c74d159ca0 2024-09-19T08:53:43.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.506+0000 7fd065d9b640 1 --2- 172.21.15.164:0/742163339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c74d159ca0 0x55c74d165760 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fd05c02a9a0 tx=0x7fd05c031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb837d075d00be62 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:43.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.506+0000 7fd04dffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:53:43.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.506+0000 7fd04dffb640 1 -- 172.21.15.164:0/742163339 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c035040 con 0x55c74d159ca0 2024-09-19T08:53:43.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.506+0000 7fd04dffb640 1 -- 172.21.15.164:0/742163339 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7fd05c004240 con 0x55c74d159ca0 2024-09-19T08:53:43.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.506+0000 7fd06edeb740 1 -- 172.21.15.164:0/742163339 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd060010770 con 0x55c74d159ca0 2024-09-19T08:53:43.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.506+0000 7fd04dffb640 1 -- 172.21.15.164:0/742163339 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd05c004440 con 0x55c74d159ca0 2024-09-19T08:53:43.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.506+0000 7fd06edeb740 10 client.17061 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:43.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.506+0000 7fd06edeb740 1 -- 172.21.15.164:0/742163339 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c74d177110 con 0x55c74d159ca0 2024-09-19T08:53:43.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.506+0000 7fd04dffb640 1 -- 172.21.15.164:0/742163339 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd060010770 con 0x55c74d159ca0 2024-09-19T08:53:43.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.507+0000 7fd04dffb640 1 -- 172.21.15.164:0/742163339 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fd05c0456f0 con 0x55c74d159ca0 2024-09-19T08:53:43.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.507+0000 7fd06edeb740 10 client.17061 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:43.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.507+0000 7fd06edeb740 10 client.17061 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:53:43.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.507+0000 7fd06edeb740 20 client.17061 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:43.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.507+0000 7fd06edeb740 1 --2- 172.21.15.164:0/742163339 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c74d1785d0 0x55c74d17aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:43.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.507+0000 7fd06edeb740 4 client.17061 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:53:43.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.507+0000 7fd06edeb740 1 -- 172.21.15.164:0/742163339 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c74d0cc850 con 0x55c74d1785d0 2024-09-19T08:53:43.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.507+0000 7fd066d9d640 1 --2- 172.21.15.164:0/742163339 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c74d1785d0 0x55c74d17aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:43.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.507+0000 7fd066d9d640 1 --2- 172.21.15.164:0/742163339 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c74d1785d0 0x55c74d17aa30 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc4911cff13ff829 server_cookie=2b21b563b79108cd in_seq=0 out_seq=0 2024-09-19T08:53:43.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.507+0000 7fd04dffb640 10 client.17061 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:43.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.508+0000 7fd04dffb640 1 -- 172.21.15.164:0/742163339 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c74d0cc850 con 0x55c74d1785d0 2024-09-19T08:53:43.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.508+0000 7fd04dffb640 10 client.17061 handle_command_reply: tid=0 2024-09-19T08:53:43.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.508+0000 7fd06edeb740 1 client.17061 shutdown 2024-09-19T08:53:43.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.508+0000 7fd06edeb740 1 -- 172.21.15.164:0/742163339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c74d159ca0 msgr2=0x55c74d165760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:43.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.508+0000 7fd06edeb740 1 --2- 172.21.15.164:0/742163339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c74d159ca0 0x55c74d165760 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fd05c02a9a0 tx=0x7fd05c031920 comp rx=0 tx=0).stop 2024-09-19T08:53:43.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.508+0000 7fd06edeb740 1 -- 172.21.15.164:0/742163339 shutdown_connections 2024-09-19T08:53:43.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.508+0000 7fd06edeb740 1 --2- 172.21.15.164:0/742163339 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c74d1785d0 0x55c74d17aa30 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.508+0000 7fd06edeb740 1 --2- 172.21.15.164:0/742163339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c74d168a90 0x55c74d165dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.508+0000 7fd06edeb740 1 --2- 172.21.15.164:0/742163339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c74d159ca0 0x55c74d165760 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.508+0000 7fd06edeb740 1 --2- 172.21.15.164:0/742163339 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c74d159380 0x55c74d168460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.508+0000 7fd04dffb640 0 client.17061 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:43.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.508+0000 7fd06edeb740 1 -- 172.21.15.164:0/742163339 >> 172.21.15.164:0/742163339 conn(0x55c74d155a30 msgr2=0x55c74d15b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:43.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.508+0000 7fd06edeb740 1 -- 172.21.15.164:0/742163339 shutdown_connections 2024-09-19T08:53:43.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.508+0000 7fd06edeb740 1 -- 172.21.15.164:0/742163339 wait complete. 2024-09-19T08:53:43.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.509+0000 7fd06edeb740 20 client.17061 trim_cache size 0 max 16384 2024-09-19T08:53:43.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.525+0000 7fd06edeb740 1 Processor -- start 2024-09-19T08:53:43.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.525+0000 7fd06edeb740 1 -- start start 2024-09-19T08:53:43.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.526+0000 7fd06edeb740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c74d159380 0x55c74d174900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:43.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.526+0000 7fd06edeb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c74d159ca0 0x55c74d180300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:43.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.526+0000 7fd06edeb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c74d168a90 0x55c74d180840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:43.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.526+0000 7fd06edeb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd0681ab5b0 con 0x55c74d159ca0 2024-09-19T08:53:43.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.526+0000 7fd06edeb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd0681ad0f0 con 0x55c74d168a90 2024-09-19T08:53:43.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.526+0000 7fd06edeb740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd0681abc80 con 0x55c74d159380 2024-09-19T08:53:43.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.526+0000 7fd065d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c74d159ca0 0x55c74d180300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:43.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.526+0000 7fd065d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c74d159ca0 0x55c74d180300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47932/0 (socket says 172.21.15.164:47932) 2024-09-19T08:53:43.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.526+0000 7fd065d9b640 1 -- 172.21.15.164:0/709493056 learned_addr learned my addr 172.21.15.164:0/709493056 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:43.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.526+0000 7fd066d9d640 1 --2- 172.21.15.164:0/709493056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c74d168a90 0x55c74d180840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:43.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.526+0000 7fd065d9b640 1 -- 172.21.15.164:0/709493056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c74d159380 msgr2=0x55c74d174900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:43.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.526+0000 7fd065d9b640 1 --2- 172.21.15.164:0/709493056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c74d159380 0x55c74d174900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.526+0000 7fd065d9b640 1 -- 172.21.15.164:0/709493056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c74d168a90 msgr2=0x55c74d180840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:43.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.526+0000 7fd065d9b640 1 --2- 172.21.15.164:0/709493056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c74d168a90 0x55c74d180840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.526+0000 7fd065d9b640 1 -- 172.21.15.164:0/709493056 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c74d166130 con 0x55c74d159ca0 2024-09-19T08:53:43.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.526+0000 7fd066d9d640 1 --2- 172.21.15.164:0/709493056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c74d168a90 0x55c74d180840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:53:43.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.526+0000 7fd065d9b640 1 --2- 172.21.15.164:0/709493056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c74d159ca0 0x55c74d180300 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fd05c006010 tx=0x7fd05c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e65e4b8557eaa1e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:43.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.526+0000 7fd04ffff640 1 -- 172.21.15.164:0/709493056 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c035030 con 0x55c74d159ca0 2024-09-19T08:53:43.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.526+0000 7fd04ffff640 1 -- 172.21.15.164:0/709493056 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd05c054050 con 0x55c74d159ca0 2024-09-19T08:53:43.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.526+0000 7fd04ffff640 1 -- 172.21.15.164:0/709493056 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c052070 con 0x55c74d159ca0 2024-09-19T08:53:43.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.527+0000 7fd06edeb740 1 -- 172.21.15.164:0/709493056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c74d159ca0 msgr2=0x55c74d180300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:43.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.527+0000 7fd06edeb740 1 --2- 172.21.15.164:0/709493056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c74d159ca0 0x55c74d180300 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fd05c006010 tx=0x7fd05c0045e0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.527+0000 7fd06edeb740 1 -- 172.21.15.164:0/709493056 shutdown_connections 2024-09-19T08:53:43.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.527+0000 7fd06edeb740 1 --2- 172.21.15.164:0/709493056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c74d168a90 0x55c74d180840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.527+0000 7fd06edeb740 1 --2- 172.21.15.164:0/709493056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c74d159ca0 0x55c74d180300 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.527+0000 7fd06edeb740 1 --2- 172.21.15.164:0/709493056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c74d159380 0x55c74d174900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.527+0000 7fd06edeb740 1 -- 172.21.15.164:0/709493056 >> 172.21.15.164:0/709493056 conn(0x55c74d155a30 msgr2=0x55c74d164720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:43.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.527+0000 7fd06edeb740 1 -- 172.21.15.164:0/709493056 shutdown_connections 2024-09-19T08:53:43.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.527+0000 7fd06edeb740 1 -- 172.21.15.164:0/709493056 wait complete. 2024-09-19T08:53:43.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.527+0000 7fd06edeb740 1 -- start start 2024-09-19T08:53:43.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.527+0000 7fd06edeb740 1 Processor -- start 2024-09-19T08:53:43.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.528+0000 7fd06edeb740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c74d159380 0x55c74d16c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:43.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.528+0000 7fd06edeb740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c74d159ca0 0x55c74d16ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:43.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.528+0000 7fd06edeb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c74d168a90 0x55c74d16d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:43.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.528+0000 7fd06edeb740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd0681aca20 con 0x55c74d159380 2024-09-19T08:53:43.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.528+0000 7fd06edeb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd0681ac350 con 0x55c74d168a90 2024-09-19T08:53:43.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.528+0000 7fd06edeb740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd0681a95b0 con 0x55c74d159ca0 2024-09-19T08:53:43.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.528+0000 7fd06edeb740 10 client.0 _ll_register_callbacks cb 0x7fd05c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:43.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.528+0000 7fd066d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c74d159380 0x55c74d16c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:43.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.528+0000 7fd066d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c74d159380 0x55c74d16c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47948/0 (socket says 172.21.15.164:47948) 2024-09-19T08:53:43.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.528+0000 7fd066d9d640 1 -- 172.21.15.164:0/3522703877 learned_addr learned my addr 172.21.15.164:0/3522703877 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:43.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.528+0000 7fd066d9d640 1 --2- 172.21.15.164:0/3522703877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c74d168a90 0x55c74d16d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:43.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.528+0000 7fd065d9b640 1 --2- 172.21.15.164:0/3522703877 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c74d159ca0 0x55c74d16ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:43.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.528+0000 7fd066d9d640 1 -- 172.21.15.164:0/3522703877 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c74d159ca0 msgr2=0x55c74d16ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:43.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.528+0000 7fd066d9d640 1 --2- 172.21.15.164:0/3522703877 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c74d159ca0 0x55c74d16ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.528+0000 7fd066d9d640 1 -- 172.21.15.164:0/3522703877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c74d168a90 msgr2=0x55c74d16d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:43.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.528+0000 7fd066d9d640 1 --2- 172.21.15.164:0/3522703877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c74d168a90 0x55c74d16d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.528+0000 7fd066d9d640 1 -- 172.21.15.164:0/3522703877 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c74d1e2470 con 0x55c74d159380 2024-09-19T08:53:43.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.528+0000 7fd066d9d640 1 --2- 172.21.15.164:0/3522703877 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c74d159380 0x55c74d16c970 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fd0600098b0 tx=0x7fd0600099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5fe3ce1b4a80733 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:43.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.528+0000 7fd04dffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:53:43.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.528+0000 7fd04dffb640 1 -- 172.21.15.164:0/3522703877 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd060023070 con 0x55c74d159380 2024-09-19T08:53:43.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.528+0000 7fd04dffb640 1 -- 172.21.15.164:0/3522703877 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7fd060003970 con 0x55c74d159380 2024-09-19T08:53:43.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.528+0000 7fd06edeb740 1 -- 172.21.15.164:0/3522703877 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd05c054050 con 0x55c74d159380 2024-09-19T08:53:43.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.529+0000 7fd04dffb640 1 -- 172.21.15.164:0/3522703877 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd060010770 con 0x55c74d159380 2024-09-19T08:53:43.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.529+0000 7fd06edeb740 10 client.17067 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:43.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.529+0000 7fd06edeb740 1 -- 172.21.15.164:0/3522703877 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c74d1e3730 con 0x55c74d159380 2024-09-19T08:53:43.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.529+0000 7fd04dffb640 1 -- 172.21.15.164:0/3522703877 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd05c054050 con 0x55c74d159380 2024-09-19T08:53:43.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.529+0000 7fd04dffb640 1 -- 172.21.15.164:0/3522703877 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fd06000f040 con 0x55c74d159380 2024-09-19T08:53:43.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.529+0000 7fd06edeb740 10 client.17067 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:43.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.529+0000 7fd06edeb740 10 client.17067 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:53:43.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.529+0000 7fd06edeb740 20 client.17067 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:43.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.529+0000 7fd06edeb740 1 --2- 172.21.15.164:0/3522703877 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c74d1e3c50 0x55c74d1e6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:43.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.529+0000 7fd06edeb740 4 client.17067 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:53:43.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.529+0000 7fd06edeb740 1 -- 172.21.15.164:0/3522703877 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c74d0c6d60 con 0x55c74d1e3c50 2024-09-19T08:53:43.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.529+0000 7fd065d9b640 1 --2- 172.21.15.164:0/3522703877 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c74d1e3c50 0x55c74d1e6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:43.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.530+0000 7fd04dffb640 10 client.17067 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:43.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.530+0000 7fd065d9b640 1 --2- 172.21.15.164:0/3522703877 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c74d1e3c50 0x55c74d1e6070 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fec1e0ed5a28649b server_cookie=f0e0f4a43e3aae45 in_seq=0 out_seq=0 2024-09-19T08:53:43.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.530+0000 7fd04dffb640 1 -- 172.21.15.164:0/3522703877 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c74d0c6d60 con 0x55c74d1e3c50 2024-09-19T08:53:43.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.530+0000 7fd04dffb640 10 client.17067 handle_command_reply: tid=0 2024-09-19T08:53:43.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.530+0000 7fd06edeb740 1 client.17067 shutdown 2024-09-19T08:53:43.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.530+0000 7fd06edeb740 1 -- 172.21.15.164:0/3522703877 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c74d159380 msgr2=0x55c74d16c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:43.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.530+0000 7fd06edeb740 1 --2- 172.21.15.164:0/3522703877 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c74d159380 0x55c74d16c970 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fd0600098b0 tx=0x7fd0600099f0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.530+0000 7fd06edeb740 1 -- 172.21.15.164:0/3522703877 shutdown_connections 2024-09-19T08:53:43.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.530+0000 7fd06edeb740 1 --2- 172.21.15.164:0/3522703877 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c74d1e3c50 0x55c74d1e6070 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.530+0000 7fd06edeb740 1 --2- 172.21.15.164:0/3522703877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c74d168a90 0x55c74d16d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.530+0000 7fd06edeb740 1 --2- 172.21.15.164:0/3522703877 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c74d159ca0 0x55c74d16ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.530+0000 7fd06edeb740 1 --2- 172.21.15.164:0/3522703877 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c74d159380 0x55c74d16c970 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.530+0000 7fd06edeb740 1 -- 172.21.15.164:0/3522703877 >> 172.21.15.164:0/3522703877 conn(0x55c74d155a30 msgr2=0x55c74d163f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:43.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.530+0000 7fd04dffb640 0 client.17067 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:43.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.530+0000 7fd06edeb740 1 -- 172.21.15.164:0/3522703877 shutdown_connections 2024-09-19T08:53:43.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.530+0000 7fd06edeb740 1 -- 172.21.15.164:0/3522703877 wait complete. 2024-09-19T08:53:43.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.530+0000 7fd06edeb740 20 client.17067 trim_cache size 0 max 16384 2024-09-19T08:53:43.550 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:53:43.550 INFO:teuthology.orchestra.run.smithi164.stdout: "status": "no active scrubs running", 2024-09-19T08:53:43.550 INFO:teuthology.orchestra.run.smithi164.stdout: "scrubs": {} 2024-09-19T08:53:43.550 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:53:43.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.531+0000 7fd06ca12640 1 -- 172.21.15.164:0/3820256109 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd0500a33b0 msgr2=0x7fd0500a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:43.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.531+0000 7fd06ca12640 1 --2- 172.21.15.164:0/3820256109 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd0500a33b0 0x7fd0500a5870 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd058098500 tx=0x7fd058000bc0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.531+0000 7fd06ca12640 1 -- 172.21.15.164:0/3820256109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd068102d40 msgr2=0x7fd0681983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:43.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.531+0000 7fd06ca12640 1 --2- 172.21.15.164:0/3820256109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd068102d40 0x7fd0681983d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fd05800c920 tx=0x7fd05800cde0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.531+0000 7fd06ca12640 1 -- 172.21.15.164:0/3820256109 shutdown_connections 2024-09-19T08:53:43.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.531+0000 7fd06ca12640 1 --2- 172.21.15.164:0/3820256109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd06810c170 0x7fd06819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.531+0000 7fd06ca12640 1 --2- 172.21.15.164:0/3820256109 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd068103660 0x7fd068198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.531+0000 7fd06ca12640 1 --2- 172.21.15.164:0/3820256109 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd0500a33b0 0x7fd0500a5870 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.531+0000 7fd06ca12640 1 --2- 172.21.15.164:0/3820256109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd068102d40 0x7fd0681983d0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:43.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.531+0000 7fd06ca12640 1 -- 172.21.15.164:0/3820256109 >> 172.21.15.164:0/3820256109 conn(0x7fd0680fc7a0 msgr2=0x7fd0680fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:43.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.531+0000 7fd06ca12640 1 -- 172.21.15.164:0/3820256109 shutdown_connections 2024-09-19T08:53:43.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:43.531+0000 7fd06ca12640 1 -- 172.21.15.164:0/3820256109 wait complete. 2024-09-19T08:53:44.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:43 smithi173 ceph-mon[35306]: pgmap v244: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:53:44.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:43 smithi164 ceph-mon[30928]: pgmap v244: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:53:44.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:43 smithi170 ceph-mon[35072]: pgmap v244: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:53:44.529 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T08:53:44.529 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:53:44.782 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:53:45.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.245+0000 7f597326e640 1 -- 172.21.15.164:0/846637426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f596c1036e0 msgr2=0x7f596c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:45.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.245+0000 7f597326e640 1 --2- 172.21.15.164:0/846637426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f596c1036e0 0x7f596c107b90 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f5960009930 tx=0x7f596002a420 comp rx=0 tx=0).stop 2024-09-19T08:53:45.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.246+0000 7f597326e640 1 -- 172.21.15.164:0/846637426 shutdown_connections 2024-09-19T08:53:45.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.246+0000 7f597326e640 1 --2- 172.21.15.164:0/846637426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f596c1082a0 0x7f596c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:45.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.246+0000 7f597326e640 1 --2- 172.21.15.164:0/846637426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f596c1036e0 0x7f596c107b90 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:45.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.246+0000 7f597326e640 1 --2- 172.21.15.164:0/846637426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f596c102d30 0x7f596c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:45.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.246+0000 7f597326e640 1 -- 172.21.15.164:0/846637426 >> 172.21.15.164:0/846637426 conn(0x7f596c0fc7d0 msgr2=0x7f596c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:45.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.246+0000 7f597326e640 1 -- 172.21.15.164:0/846637426 shutdown_connections 2024-09-19T08:53:45.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.246+0000 7f597326e640 1 -- 172.21.15.164:0/846637426 wait complete. 2024-09-19T08:53:45.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.246+0000 7f597326e640 1 Processor -- start 2024-09-19T08:53:45.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.246+0000 7f597326e640 1 -- start start 2024-09-19T08:53:45.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.247+0000 7f597326e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f596c102d30 0x7f596c1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:45.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.247+0000 7f597326e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f596c1036e0 0x7f596c198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:45.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.247+0000 7f597326e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f596c1082a0 0x7f596c19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:45.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.247+0000 7f597326e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f596c112060 con 0x7f596c102d30 2024-09-19T08:53:45.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.247+0000 7f597326e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f596c111ee0 con 0x7f596c1082a0 2024-09-19T08:53:45.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.247+0000 7f597326e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f596c1121e0 con 0x7f596c1036e0 2024-09-19T08:53:45.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.247+0000 7f597100a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f596c102d30 0x7f596c1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:45.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.247+0000 7f597100a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f596c102d30 0x7f596c1984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47970/0 (socket says 172.21.15.164:47970) 2024-09-19T08:53:45.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.247+0000 7f597100a640 1 -- 172.21.15.164:0/1596536109 learned_addr learned my addr 172.21.15.164:0/1596536109 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:45.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.247+0000 7f597180b640 1 --2- 172.21.15.164:0/1596536109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f596c1082a0 0x7f596c19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:45.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.247+0000 7f5970809640 1 --2- 172.21.15.164:0/1596536109 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f596c1036e0 0x7f596c198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:45.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.247+0000 7f597100a640 1 -- 172.21.15.164:0/1596536109 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f596c1036e0 msgr2=0x7f596c198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:45.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.247+0000 7f597100a640 1 --2- 172.21.15.164:0/1596536109 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f596c1036e0 0x7f596c198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:45.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.247+0000 7f597100a640 1 -- 172.21.15.164:0/1596536109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f596c1082a0 msgr2=0x7f596c19cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:45.277 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.247+0000 7f597100a640 1 --2- 172.21.15.164:0/1596536109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f596c1082a0 0x7f596c19cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:45.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.247+0000 7f597100a640 1 -- 172.21.15.164:0/1596536109 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f596c19d470 con 0x7f596c102d30 2024-09-19T08:53:45.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.247+0000 7f597180b640 1 --2- 172.21.15.164:0/1596536109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f596c1082a0 0x7f596c19cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:53:45.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.247+0000 7f597100a640 1 --2- 172.21.15.164:0/1596536109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f596c102d30 0x7f596c1984c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f595c00cce0 tx=0x7f595c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:45.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.247+0000 7f596a7fc640 1 -- 172.21.15.164:0/1596536109 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c00f650 con 0x7f596c102d30 2024-09-19T08:53:45.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.247+0000 7f596a7fc640 1 -- 172.21.15.164:0/1596536109 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f595c00fcd0 con 0x7f596c102d30 2024-09-19T08:53:45.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.247+0000 7f597326e640 1 -- 172.21.15.164:0/1596536109 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f596c19d760 con 0x7f596c102d30 2024-09-19T08:53:45.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.248+0000 7f596a7fc640 1 -- 172.21.15.164:0/1596536109 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c018c70 con 0x7f596c102d30 2024-09-19T08:53:45.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.248+0000 7f597326e640 1 -- 172.21.15.164:0/1596536109 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f596c1a5040 con 0x7f596c102d30 2024-09-19T08:53:45.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.248+0000 7f597326e640 1 -- 172.21.15.164:0/1596536109 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f596c103110 con 0x7f596c102d30 2024-09-19T08:53:45.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.250+0000 7f596a7fc640 1 -- 172.21.15.164:0/1596536109 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f595c00f7f0 con 0x7f596c102d30 2024-09-19T08:53:45.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.250+0000 7f596a7fc640 1 --2- 172.21.15.164:0/1596536109 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f59540a3430 0x7f59540a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:45.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.250+0000 7f5970809640 1 --2- 172.21.15.164:0/1596536109 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f59540a3430 0x7f59540a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:45.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.250+0000 7f596a7fc640 1 -- 172.21.15.164:0/1596536109 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f595c0d0490 con 0x7f596c102d30 2024-09-19T08:53:45.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.251+0000 7f5970809640 1 --2- 172.21.15.164:0/1596536109 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f59540a3430 0x7f59540a58f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5960009900 tx=0x7f5960000bc0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:45.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.253+0000 7f596a7fc640 1 -- 172.21.15.164:0/1596536109 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f595c014030 con 0x7f596c102d30 2024-09-19T08:53:45.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.457+0000 7f597326e640 1 -- 172.21.15.164:0/1596536109 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f596c199820 con 0x7f596c102d30 2024-09-19T08:53:45.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.458+0000 7f596a7fc640 1 -- 172.21.15.164:0/1596536109 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11984 (secure 0 0 0) 0x7f596c199820 con 0x7f596c102d30 2024-09-19T08:53:45.459 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:53:45.459 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":32,"btime":"2024-09-19T08:52:27:192804+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:52:27.074168+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:53:45.459 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 32 2024-09-19T08:53:45.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.460+0000 7f597326e640 1 -- 172.21.15.164:0/1596536109 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f59540a3430 msgr2=0x7f59540a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:45.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.460+0000 7f597326e640 1 --2- 172.21.15.164:0/1596536109 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f59540a3430 0x7f59540a58f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5960009900 tx=0x7f5960000bc0 comp rx=0 tx=0).stop 2024-09-19T08:53:45.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.460+0000 7f597326e640 1 -- 172.21.15.164:0/1596536109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f596c102d30 msgr2=0x7f596c1984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:45.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.460+0000 7f597326e640 1 --2- 172.21.15.164:0/1596536109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f596c102d30 0x7f596c1984c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f595c00cce0 tx=0x7f595c00a620 comp rx=0 tx=0).stop 2024-09-19T08:53:45.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.461+0000 7f597326e640 1 -- 172.21.15.164:0/1596536109 shutdown_connections 2024-09-19T08:53:45.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.461+0000 7f597326e640 1 --2- 172.21.15.164:0/1596536109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f596c1082a0 0x7f596c19cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:45.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.461+0000 7f597326e640 1 --2- 172.21.15.164:0/1596536109 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f596c1036e0 0x7f596c198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:45.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.461+0000 7f597326e640 1 --2- 172.21.15.164:0/1596536109 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f59540a3430 0x7f59540a58f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:45.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.461+0000 7f597326e640 1 --2- 172.21.15.164:0/1596536109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f596c102d30 0x7f596c1984c0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:45.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.461+0000 7f597326e640 1 -- 172.21.15.164:0/1596536109 >> 172.21.15.164:0/1596536109 conn(0x7f596c0fc7d0 msgr2=0x7f596c106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:45.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.461+0000 7f597326e640 1 -- 172.21.15.164:0/1596536109 shutdown_connections 2024-09-19T08:53:45.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:45.461+0000 7f597326e640 1 -- 172.21.15.164:0/1596536109 wait complete. 2024-09-19T08:53:46.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:45 smithi173 ceph-mon[35306]: pgmap v245: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-19T08:53:46.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:45 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/1596536109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:53:46.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:45 smithi164 ceph-mon[30928]: pgmap v245: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-19T08:53:46.195 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:45 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/1596536109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:53:46.217 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:53:46.225 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T08:53:46.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:45 smithi170 ceph-mon[35072]: pgmap v245: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-19T08:53:46.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:45 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/1596536109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:53:46.475 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:53:46.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.955+0000 7f620b55d640 1 -- 172.21.15.164:0/3232936472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6204102790 msgr2=0x7f62041067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:46.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.955+0000 7f620b55d640 1 --2- 172.21.15.164:0/3232936472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6204102790 0x7f62041067e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f61f4009900 tx=0x7f61f402a410 comp rx=0 tx=0).stop 2024-09-19T08:53:46.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.955+0000 7f620b55d640 1 -- 172.21.15.164:0/3232936472 shutdown_connections 2024-09-19T08:53:46.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.955+0000 7f620b55d640 1 --2- 172.21.15.164:0/3232936472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6204106f10 0x7f620410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:46.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.955+0000 7f620b55d640 1 --2- 172.21.15.164:0/3232936472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6204102790 0x7f62041067e0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:46.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.955+0000 7f620b55d640 1 --2- 172.21.15.164:0/3232936472 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6204101de0 0x7f62041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:46.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.955+0000 7f620b55d640 1 -- 172.21.15.164:0/3232936472 >> 172.21.15.164:0/3232936472 conn(0x7f62040fd8c0 msgr2=0x7f62040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:46.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.956+0000 7f620b55d640 1 -- 172.21.15.164:0/3232936472 shutdown_connections 2024-09-19T08:53:46.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.956+0000 7f620b55d640 1 -- 172.21.15.164:0/3232936472 wait complete. 2024-09-19T08:53:46.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.956+0000 7f620b55d640 1 Processor -- start 2024-09-19T08:53:46.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.956+0000 7f620b55d640 1 -- start start 2024-09-19T08:53:46.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.956+0000 7f620b55d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6204101de0 0x7f62040744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:46.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.956+0000 7f620b55d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6204102790 0x7f6204074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:46.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.956+0000 7f620b55d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6204106f10 0x7f6204074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:46.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.956+0000 7f620b55d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6204111f90 con 0x7f6204102790 2024-09-19T08:53:46.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.956+0000 7f620b55d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6204111e10 con 0x7f6204106f10 2024-09-19T08:53:46.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.956+0000 7f620b55d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6204112110 con 0x7f6204101de0 2024-09-19T08:53:46.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.957+0000 7f6208af8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6204102790 0x7f6204074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:46.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.957+0000 7f6208af8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6204102790 0x7f6204074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:47992/0 (socket says 172.21.15.164:47992) 2024-09-19T08:53:46.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.957+0000 7f6208af8640 1 -- 172.21.15.164:0/1157950519 learned_addr learned my addr 172.21.15.164:0/1157950519 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:46.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.957+0000 7f6209afa640 1 --2- 172.21.15.164:0/1157950519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6204106f10 0x7f6204074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:46.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.957+0000 7f62092f9640 1 --2- 172.21.15.164:0/1157950519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6204101de0 0x7f62040744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:46.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.957+0000 7f6208af8640 1 -- 172.21.15.164:0/1157950519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6204101de0 msgr2=0x7f62040744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:46.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.957+0000 7f6208af8640 1 --2- 172.21.15.164:0/1157950519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6204101de0 0x7f62040744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:46.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.957+0000 7f6208af8640 1 -- 172.21.15.164:0/1157950519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6204106f10 msgr2=0x7f6204074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:46.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.957+0000 7f6208af8640 1 --2- 172.21.15.164:0/1157950519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6204106f10 0x7f6204074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:46.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.957+0000 7f6208af8640 1 -- 172.21.15.164:0/1157950519 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62041a6950 con 0x7f6204102790 2024-09-19T08:53:46.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.957+0000 7f62092f9640 1 --2- 172.21.15.164:0/1157950519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6204101de0 0x7f62040744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:53:46.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.957+0000 7f6208af8640 1 --2- 172.21.15.164:0/1157950519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6204102790 0x7f6204074a30 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f61f4006010 tx=0x7f61f40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:46.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.957+0000 7f61fa7fc640 1 -- 172.21.15.164:0/1157950519 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f403f030 con 0x7f6204102790 2024-09-19T08:53:46.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.957+0000 7f620b55d640 1 -- 172.21.15.164:0/1157950519 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62041a6be0 con 0x7f6204102790 2024-09-19T08:53:46.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.957+0000 7f61fa7fc640 1 -- 172.21.15.164:0/1157950519 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61f4003ce0 con 0x7f6204102790 2024-09-19T08:53:46.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.957+0000 7f61fa7fc640 1 -- 172.21.15.164:0/1157950519 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f4035070 con 0x7f6204102790 2024-09-19T08:53:46.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.957+0000 7f620b55d640 1 -- 172.21.15.164:0/1157950519 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62041a70c0 con 0x7f6204102790 2024-09-19T08:53:46.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.958+0000 7f620b55d640 1 -- 172.21.15.164:0/1157950519 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62041021c0 con 0x7f6204102790 2024-09-19T08:53:46.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.960+0000 7f61fa7fc640 1 -- 172.21.15.164:0/1157950519 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f61f4042030 con 0x7f6204102790 2024-09-19T08:53:46.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.960+0000 7f61fa7fc640 1 --2- 172.21.15.164:0/1157950519 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f61f00a33b0 0x7f61f00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:46.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.960+0000 7f62092f9640 1 --2- 172.21.15.164:0/1157950519 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f61f00a33b0 0x7f61f00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:46.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.960+0000 7f61fa7fc640 1 -- 172.21.15.164:0/1157950519 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f61f40f0540 con 0x7f6204102790 2024-09-19T08:53:46.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.961+0000 7f62092f9640 1 --2- 172.21.15.164:0/1157950519 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f61f00a33b0 0x7f61f00a5870 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6200009930 tx=0x7f6200008040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:46.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:46.963+0000 7f61fa7fc640 1 -- 172.21.15.164:0/1157950519 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f61f40bd020 con 0x7f6204102790 2024-09-19T08:53:46.989 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:53:47.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:47.176+0000 7f620b55d640 1 -- 172.21.15.164:0/1157950519 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f620402d7c0 con 0x7f6204102790 2024-09-19T08:53:47.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:47.176+0000 7f61fa7fc640 1 -- 172.21.15.164:0/1157950519 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11984 (secure 0 0 0) 0x7f61f40bc920 con 0x7f6204102790 2024-09-19T08:53:47.177 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:53:47.178 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":32,"btime":"2024-09-19T08:52:27:192804+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:52:27.074168+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:53:47.178 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 32 2024-09-19T08:53:47.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:47.178+0000 7f620b55d640 1 -- 172.21.15.164:0/1157950519 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f61f00a33b0 msgr2=0x7f61f00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:47.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:47.179+0000 7f620b55d640 1 --2- 172.21.15.164:0/1157950519 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f61f00a33b0 0x7f61f00a5870 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6200009930 tx=0x7f6200008040 comp rx=0 tx=0).stop 2024-09-19T08:53:47.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:47.179+0000 7f620b55d640 1 -- 172.21.15.164:0/1157950519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6204102790 msgr2=0x7f6204074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:47.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:47.179+0000 7f620b55d640 1 --2- 172.21.15.164:0/1157950519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6204102790 0x7f6204074a30 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f61f4006010 tx=0x7f61f40042f0 comp rx=0 tx=0).stop 2024-09-19T08:53:47.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:47.179+0000 7f620b55d640 1 -- 172.21.15.164:0/1157950519 shutdown_connections 2024-09-19T08:53:47.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:47.179+0000 7f620b55d640 1 --2- 172.21.15.164:0/1157950519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6204106f10 0x7f6204074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:47.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:47.179+0000 7f620b55d640 1 --2- 172.21.15.164:0/1157950519 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f61f00a33b0 0x7f61f00a5870 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:47.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:47.179+0000 7f620b55d640 1 --2- 172.21.15.164:0/1157950519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6204102790 0x7f6204074a30 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:47.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:47.179+0000 7f620b55d640 1 --2- 172.21.15.164:0/1157950519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6204101de0 0x7f62040744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:47.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:47.179+0000 7f620b55d640 1 -- 172.21.15.164:0/1157950519 >> 172.21.15.164:0/1157950519 conn(0x7f62040fd8c0 msgr2=0x7f62040fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:47.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:47.179+0000 7f620b55d640 1 -- 172.21.15.164:0/1157950519 shutdown_connections 2024-09-19T08:53:47.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:47.179+0000 7f620b55d640 1 -- 172.21.15.164:0/1157950519 wait complete. 2024-09-19T08:53:47.928 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id d0313327 --release --await' 2024-09-19T08:53:47.928 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id d0313327 --release --await 2024-09-19T08:53:48.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.108+0000 7fd19f90d640 1 -- 172.21.15.164:0/3189965108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd198102790 msgr2=0x7fd1981067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:48.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.108+0000 7fd19f90d640 1 --2- 172.21.15.164:0/3189965108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd198102790 0x7fd1981067e0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fd194009900 tx=0x7fd19402a410 comp rx=0 tx=0).stop 2024-09-19T08:53:48.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.108+0000 7fd19f90d640 1 -- 172.21.15.164:0/3189965108 shutdown_connections 2024-09-19T08:53:48.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.108+0000 7fd19f90d640 1 --2- 172.21.15.164:0/3189965108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd198106f10 0x7fd19810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.108+0000 7fd19f90d640 1 --2- 172.21.15.164:0/3189965108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd198102790 0x7fd1981067e0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.108+0000 7fd19f90d640 1 --2- 172.21.15.164:0/3189965108 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd198101de0 0x7fd1981021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.108+0000 7fd19f90d640 1 -- 172.21.15.164:0/3189965108 >> 172.21.15.164:0/3189965108 conn(0x7fd1980fd8c0 msgr2=0x7fd1980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:48.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.108+0000 7fd19f90d640 1 -- 172.21.15.164:0/3189965108 shutdown_connections 2024-09-19T08:53:48.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.108+0000 7fd19f90d640 1 -- 172.21.15.164:0/3189965108 wait complete. 2024-09-19T08:53:48.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.108+0000 7fd19f90d640 1 Processor -- start 2024-09-19T08:53:48.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.109+0000 7fd19f90d640 1 -- start start 2024-09-19T08:53:48.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.109+0000 7fd19f90d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd198101de0 0x7fd1981984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:48.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.109+0000 7fd19f90d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd198102790 0x7fd1981989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:48.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.109+0000 7fd19f90d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd198106f10 0x7fd19819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:48.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.109+0000 7fd19f90d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd198111f70 con 0x7fd198102790 2024-09-19T08:53:48.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.109+0000 7fd19f90d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd198111df0 con 0x7fd198101de0 2024-09-19T08:53:48.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.109+0000 7fd19f90d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd1981120f0 con 0x7fd198106f10 2024-09-19T08:53:48.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.109+0000 7fd19cea8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd198102790 0x7fd1981989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:48.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.109+0000 7fd19cea8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd198102790 0x7fd1981989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48020/0 (socket says 172.21.15.164:48020) 2024-09-19T08:53:48.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.109+0000 7fd19cea8640 1 -- 172.21.15.164:0/2049640002 learned_addr learned my addr 172.21.15.164:0/2049640002 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:48.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.109+0000 7fd19d6a9640 1 --2- 172.21.15.164:0/2049640002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd198101de0 0x7fd1981984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:48.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.109+0000 7fd19deaa640 1 --2- 172.21.15.164:0/2049640002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd198106f10 0x7fd19819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:48.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.109+0000 7fd19cea8640 1 -- 172.21.15.164:0/2049640002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd198106f10 msgr2=0x7fd19819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:48.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.109+0000 7fd19cea8640 1 --2- 172.21.15.164:0/2049640002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd198106f10 0x7fd19819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.109+0000 7fd19cea8640 1 -- 172.21.15.164:0/2049640002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd198101de0 msgr2=0x7fd1981984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:48.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.109+0000 7fd19cea8640 1 --2- 172.21.15.164:0/2049640002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd198101de0 0x7fd1981984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.109+0000 7fd19cea8640 1 -- 172.21.15.164:0/2049640002 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19819d460 con 0x7fd198102790 2024-09-19T08:53:48.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.109+0000 7fd19cea8640 1 --2- 172.21.15.164:0/2049640002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd198102790 0x7fd1981989f0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd194006010 tx=0x7fd1940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:48.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.110+0000 7fd18a7fc640 1 -- 172.21.15.164:0/2049640002 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19403f030 con 0x7fd198102790 2024-09-19T08:53:48.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.110+0000 7fd19f90d640 1 -- 172.21.15.164:0/2049640002 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd19819d6f0 con 0x7fd198102790 2024-09-19T08:53:48.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.110+0000 7fd19f90d640 1 -- 172.21.15.164:0/2049640002 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1981a5030 con 0x7fd198102790 2024-09-19T08:53:48.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.110+0000 7fd18a7fc640 1 -- 172.21.15.164:0/2049640002 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd194003ce0 con 0x7fd198102790 2024-09-19T08:53:48.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.110+0000 7fd18a7fc640 1 -- 172.21.15.164:0/2049640002 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd194049060 con 0x7fd198102790 2024-09-19T08:53:48.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.112+0000 7fd18a7fc640 1 -- 172.21.15.164:0/2049640002 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fd194042030 con 0x7fd198102790 2024-09-19T08:53:48.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.113+0000 7fd1a1ce6740 1 Processor -- start 2024-09-19T08:53:48.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.113+0000 7fd1a1ce6740 1 -- start start 2024-09-19T08:53:48.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.113+0000 7fd1a1ce6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d7da4eb40 0x555d7da4d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:48.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.113+0000 7fd18a7fc640 1 --2- 172.21.15.164:0/2049640002 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd1840a33b0 0x7fd1840a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:48.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.113+0000 7fd1a1ce6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d7da4d5d0 0x555d7da4da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:48.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.113+0000 7fd1a1ce6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d7da58370 0x555d7da5a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:48.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.113+0000 7fd1a1ce6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555d7d9b32f0 con 0x555d7da4d5d0 2024-09-19T08:53:48.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.113+0000 7fd1a1ce6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555d7d95acf0 con 0x555d7da58370 2024-09-19T08:53:48.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.113+0000 7fd1a1ce6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555d7d9af2b0 con 0x555d7da4eb40 2024-09-19T08:53:48.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.113+0000 7fd18a7fc640 1 -- 172.21.15.164:0/2049640002 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7fd1940f02b0 con 0x7fd198102790 2024-09-19T08:53:48.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.113+0000 7fd19cea8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d7da4d5d0 0x555d7da4da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:48.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.113+0000 7fd19cea8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d7da4d5d0 0x555d7da4da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48028/0 (socket says 172.21.15.164:48028) 2024-09-19T08:53:48.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.113+0000 7fd19cea8640 1 -- 172.21.15.164:0/4293119017 learned_addr learned my addr 172.21.15.164:0/4293119017 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:48.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.113+0000 7fd19d6a9640 1 --2- 172.21.15.164:0/2049640002 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd1840a33b0 0x7fd1840a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:48.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.113+0000 7fd19deaa640 1 --2- 172.21.15.164:0/4293119017 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d7da58370 0x555d7da5a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:48.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.113+0000 7fd19deaa640 1 --2- 172.21.15.164:0/4293119017 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d7da4eb40 0x555d7da4d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:48.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.114+0000 7fd19cea8640 1 -- 172.21.15.164:0/4293119017 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d7da4eb40 msgr2=0x555d7da4d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:48.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.114+0000 7fd19cea8640 1 --2- 172.21.15.164:0/4293119017 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d7da4eb40 0x555d7da4d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.114+0000 7fd19cea8640 1 -- 172.21.15.164:0/4293119017 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d7da58370 msgr2=0x555d7da5a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:48.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.114+0000 7fd19cea8640 1 --2- 172.21.15.164:0/4293119017 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d7da58370 0x555d7da5a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.114+0000 7fd19cea8640 1 -- 172.21.15.164:0/4293119017 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d7d93f3a0 con 0x555d7da4d5d0 2024-09-19T08:53:48.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.114+0000 7fd19deaa640 1 --2- 172.21.15.164:0/4293119017 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d7da4eb40 0x555d7da4d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:53:48.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.114+0000 7fd19cea8640 1 --2- 172.21.15.164:0/4293119017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d7da4d5d0 0x555d7da4da10 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fd194000c00 tx=0x7fd1940b8b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0d380b6b83d80a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:48.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.114+0000 7fd19d6a9640 1 --2- 172.21.15.164:0/2049640002 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd1840a33b0 0x7fd1840a5870 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd18c005fd0 tx=0x7fd18c00b390 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:48.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.114+0000 7fd1837fe640 1 -- 172.21.15.164:0/4293119017 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd194035040 con 0x555d7da4d5d0 2024-09-19T08:53:48.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.114+0000 7fd1837fe640 1 -- 172.21.15.164:0/4293119017 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd19403b460 con 0x555d7da4d5d0 2024-09-19T08:53:48.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.114+0000 7fd1837fe640 1 -- 172.21.15.164:0/4293119017 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1940c1f50 con 0x555d7da4d5d0 2024-09-19T08:53:48.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.115+0000 7fd1a1ce6740 1 -- 172.21.15.164:0/4293119017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d7da4d5d0 msgr2=0x555d7da4da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:48.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.115+0000 7fd1a1ce6740 1 --2- 172.21.15.164:0/4293119017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d7da4d5d0 0x555d7da4da10 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fd194000c00 tx=0x7fd1940b8b10 comp rx=0 tx=0).stop 2024-09-19T08:53:48.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.115+0000 7fd1a1ce6740 1 -- 172.21.15.164:0/4293119017 shutdown_connections 2024-09-19T08:53:48.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.115+0000 7fd1a1ce6740 1 --2- 172.21.15.164:0/4293119017 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d7da58370 0x555d7da5a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.115+0000 7fd1a1ce6740 1 --2- 172.21.15.164:0/4293119017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d7da4d5d0 0x555d7da4da10 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.115+0000 7fd1a1ce6740 1 --2- 172.21.15.164:0/4293119017 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d7da4eb40 0x555d7da4d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.115+0000 7fd1a1ce6740 1 -- 172.21.15.164:0/4293119017 >> 172.21.15.164:0/4293119017 conn(0x555d7da16310 msgr2=0x555d7da49c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:48.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.115+0000 7fd1a1ce6740 1 -- 172.21.15.164:0/4293119017 shutdown_connections 2024-09-19T08:53:48.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.115+0000 7fd1a1ce6740 1 -- 172.21.15.164:0/4293119017 wait complete. 2024-09-19T08:53:48.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.115+0000 7fd1a1ce6740 1 -- start start 2024-09-19T08:53:48.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.116+0000 7fd1a1ce6740 1 Processor -- start 2024-09-19T08:53:48.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.116+0000 7fd1a1ce6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d7da4d5d0 0x555d7da57de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:48.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.116+0000 7fd1a1ce6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d7da4eb40 0x555d7da55090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:48.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.116+0000 7fd1a1ce6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d7da58370 0x555d7da557d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:48.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.116+0000 7fd1a1ce6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555d7da5f240 con 0x555d7da4d5d0 2024-09-19T08:53:48.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.116+0000 7fd1a1ce6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555d7d96e060 con 0x555d7da4eb40 2024-09-19T08:53:48.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.116+0000 7fd1a1ce6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555d7da5f3c0 con 0x555d7da58370 2024-09-19T08:53:48.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.116+0000 7fd19d6a9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d7da4d5d0 0x555d7da57de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:48.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.116+0000 7fd19d6a9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d7da4d5d0 0x555d7da57de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48032/0 (socket says 172.21.15.164:48032) 2024-09-19T08:53:48.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.116+0000 7fd19d6a9640 1 -- 172.21.15.164:0/2184134223 learned_addr learned my addr 172.21.15.164:0/2184134223 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:48.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.116+0000 7fd1a1ce6740 10 client.0 _ll_register_callbacks cb 0x555d7d6c18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:48.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.116+0000 7fd19cea8640 1 --2- 172.21.15.164:0/2184134223 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d7da4eb40 0x555d7da55090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:48.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.116+0000 7fd19deaa640 1 --2- 172.21.15.164:0/2184134223 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d7da58370 0x555d7da557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:48.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.117+0000 7fd19d6a9640 1 -- 172.21.15.164:0/2184134223 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d7da58370 msgr2=0x555d7da557d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:48.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.117+0000 7fd19d6a9640 1 --2- 172.21.15.164:0/2184134223 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d7da58370 0x555d7da557d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.117+0000 7fd19d6a9640 1 -- 172.21.15.164:0/2184134223 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d7da4eb40 msgr2=0x555d7da55090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:48.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.117+0000 7fd19d6a9640 1 --2- 172.21.15.164:0/2184134223 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d7da4eb40 0x555d7da55090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.117+0000 7fd19d6a9640 1 -- 172.21.15.164:0/2184134223 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d7d7243c0 con 0x555d7da4d5d0 2024-09-19T08:53:48.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.117+0000 7fd19d6a9640 1 --2- 172.21.15.164:0/2184134223 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d7da4d5d0 0x555d7da57de0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fd18c00b620 tx=0x7fd18c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89531b8db921b7ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:48.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.117+0000 7fd180ff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:53:48.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.117+0000 7fd180ff9640 1 -- 172.21.15.164:0/2184134223 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c005220 con 0x555d7da4d5d0 2024-09-19T08:53:48.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.117+0000 7fd1a1ce6740 1 -- 172.21.15.164:0/2184134223 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555d7d9c0860 con 0x555d7da4d5d0 2024-09-19T08:53:48.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.117+0000 7fd180ff9640 1 -- 172.21.15.164:0/2184134223 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7fd18c0053c0 con 0x555d7da4d5d0 2024-09-19T08:53:48.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.117+0000 7fd180ff9640 1 -- 172.21.15.164:0/2184134223 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd18c0055c0 con 0x555d7da4d5d0 2024-09-19T08:53:48.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.117+0000 7fd1a1ce6740 10 client.17091 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:48.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.117+0000 7fd1a1ce6740 1 -- 172.21.15.164:0/2184134223 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555d7da6acb0 con 0x555d7da4d5d0 2024-09-19T08:53:48.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.117+0000 7fd180ff9640 1 -- 172.21.15.164:0/2184134223 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555d7d9c0860 con 0x555d7da4d5d0 2024-09-19T08:53:48.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.117+0000 7fd180ff9640 1 -- 172.21.15.164:0/2184134223 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fd18c02faf0 con 0x555d7da4d5d0 2024-09-19T08:53:48.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.117+0000 7fd1a1ce6740 10 client.17091 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:48.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.117+0000 7fd1a1ce6740 10 client.17091 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-19T08:53:48.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.117+0000 7fd1a1ce6740 20 client.17091 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:48.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.117+0000 7fd1a1ce6740 1 --2- 172.21.15.164:0/2184134223 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x555d7da6c4f0 0x555d7da6e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:48.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.118+0000 7fd1a1ce6740 4 client.17091 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:53:48.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.118+0000 7fd1a1ce6740 1 -- 172.21.15.164:0/2184134223 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d7d9bad70 con 0x555d7da6c4f0 2024-09-19T08:53:48.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.118+0000 7fd19cea8640 1 --2- 172.21.15.164:0/2184134223 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x555d7da6c4f0 0x555d7da6e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:48.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.118+0000 7fd180ff9640 10 client.17091 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:53:48.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.118+0000 7fd19cea8640 1 --2- 172.21.15.164:0/2184134223 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x555d7da6c4f0 0x555d7da6e9b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=abe0b766acd1349f server_cookie=29c422b670761a65 in_seq=0 out_seq=0 2024-09-19T08:53:48.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.119+0000 7fd180ff9640 1 -- 172.21.15.164:0/2184134223 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555d7d9bad70 con 0x555d7da6c4f0 2024-09-19T08:53:48.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.119+0000 7fd180ff9640 10 client.17091 handle_command_reply: tid=0 2024-09-19T08:53:48.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.119+0000 7fd1a1ce6740 1 client.17091 shutdown 2024-09-19T08:53:48.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.119+0000 7fd1a1ce6740 1 -- 172.21.15.164:0/2184134223 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d7da4d5d0 msgr2=0x555d7da57de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:48.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.119+0000 7fd1a1ce6740 1 --2- 172.21.15.164:0/2184134223 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d7da4d5d0 0x555d7da57de0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fd18c00b620 tx=0x7fd18c004730 comp rx=0 tx=0).stop 2024-09-19T08:53:48.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.119+0000 7fd1a1ce6740 1 -- 172.21.15.164:0/2184134223 shutdown_connections 2024-09-19T08:53:48.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.119+0000 7fd1a1ce6740 1 --2- 172.21.15.164:0/2184134223 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x555d7da6c4f0 0x555d7da6e9b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.119+0000 7fd1a1ce6740 1 --2- 172.21.15.164:0/2184134223 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d7da58370 0x555d7da557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.119+0000 7fd180ff9640 0 client.17091 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:53:48.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.119+0000 7fd1a1ce6740 1 --2- 172.21.15.164:0/2184134223 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d7da4eb40 0x555d7da55090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.119+0000 7fd1a1ce6740 1 --2- 172.21.15.164:0/2184134223 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d7da4d5d0 0x555d7da57de0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.119+0000 7fd1a1ce6740 1 -- 172.21.15.164:0/2184134223 >> 172.21.15.164:0/2184134223 conn(0x555d7da16310 msgr2=0x555d7da4f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:48.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.119+0000 7fd1a1ce6740 1 -- 172.21.15.164:0/2184134223 shutdown_connections 2024-09-19T08:53:48.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.119+0000 7fd1a1ce6740 1 -- 172.21.15.164:0/2184134223 wait complete. 2024-09-19T08:53:48.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.119+0000 7fd1a1ce6740 20 client.17091 trim_cache size 0 max 16384 2024-09-19T08:53:48.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.135+0000 7fd1a1ce6740 1 Processor -- start 2024-09-19T08:53:48.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.135+0000 7fd1a1ce6740 1 -- start start 2024-09-19T08:53:48.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.136+0000 7fd1a1ce6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d7da4d5d0 0x555d7da50180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:48.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.136+0000 7fd1a1ce6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d7da4eb40 0x555d7da506c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:48.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.136+0000 7fd1a1ce6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d7da58370 0x555d7da742c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:48.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.136+0000 7fd1a1ce6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd1981ab5b0 con 0x555d7da4d5d0 2024-09-19T08:53:48.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.136+0000 7fd1a1ce6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd1981ad0f0 con 0x555d7da4eb40 2024-09-19T08:53:48.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.136+0000 7fd1a1ce6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd1981abc80 con 0x555d7da58370 2024-09-19T08:53:48.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.136+0000 7fd19d6a9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d7da4d5d0 0x555d7da50180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:48.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.136+0000 7fd19d6a9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d7da4d5d0 0x555d7da50180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48048/0 (socket says 172.21.15.164:48048) 2024-09-19T08:53:48.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.136+0000 7fd19d6a9640 1 -- 172.21.15.164:0/302421562 learned_addr learned my addr 172.21.15.164:0/302421562 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:48.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.136+0000 7fd19cea8640 1 --2- 172.21.15.164:0/302421562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d7da4eb40 0x555d7da506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:48.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.136+0000 7fd19deaa640 1 --2- 172.21.15.164:0/302421562 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d7da58370 0x555d7da742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:48.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.136+0000 7fd19d6a9640 1 -- 172.21.15.164:0/302421562 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d7da58370 msgr2=0x555d7da742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:48.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.136+0000 7fd19d6a9640 1 --2- 172.21.15.164:0/302421562 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d7da58370 0x555d7da742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.136+0000 7fd19d6a9640 1 -- 172.21.15.164:0/302421562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d7da4eb40 msgr2=0x555d7da506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:48.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.136+0000 7fd19d6a9640 1 --2- 172.21.15.164:0/302421562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d7da4eb40 0x555d7da506c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.136+0000 7fd19d6a9640 1 -- 172.21.15.164:0/302421562 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d7da50c00 con 0x555d7da4d5d0 2024-09-19T08:53:48.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.136+0000 7fd19cea8640 1 --2- 172.21.15.164:0/302421562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d7da4eb40 0x555d7da506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:53:48.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.136+0000 7fd19deaa640 1 --2- 172.21.15.164:0/302421562 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d7da58370 0x555d7da742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:53:48.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.136+0000 7fd19d6a9640 1 --2- 172.21.15.164:0/302421562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d7da4d5d0 0x555d7da50180 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd18c02fee0 tx=0x7fd18c01e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e85c02c4c0071ac7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:48.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.137+0000 7fd182ffd640 1 -- 172.21.15.164:0/302421562 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c03f070 con 0x555d7da4d5d0 2024-09-19T08:53:48.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.137+0000 7fd182ffd640 1 -- 172.21.15.164:0/302421562 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd18c036040 con 0x555d7da4d5d0 2024-09-19T08:53:48.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.137+0000 7fd182ffd640 1 -- 172.21.15.164:0/302421562 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c03a430 con 0x555d7da4d5d0 2024-09-19T08:53:48.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.137+0000 7fd1a1ce6740 1 -- 172.21.15.164:0/302421562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d7da4d5d0 msgr2=0x555d7da50180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:48.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.137+0000 7fd1a1ce6740 1 --2- 172.21.15.164:0/302421562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d7da4d5d0 0x555d7da50180 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd18c02fee0 tx=0x7fd18c01e570 comp rx=0 tx=0).stop 2024-09-19T08:53:48.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.137+0000 7fd1a1ce6740 1 -- 172.21.15.164:0/302421562 shutdown_connections 2024-09-19T08:53:48.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.137+0000 7fd1a1ce6740 1 --2- 172.21.15.164:0/302421562 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d7da58370 0x555d7da742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.137+0000 7fd1a1ce6740 1 --2- 172.21.15.164:0/302421562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d7da4eb40 0x555d7da506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.137+0000 7fd1a1ce6740 1 --2- 172.21.15.164:0/302421562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d7da4d5d0 0x555d7da50180 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.138+0000 7fd1a1ce6740 1 -- 172.21.15.164:0/302421562 >> 172.21.15.164:0/302421562 conn(0x555d7da16310 msgr2=0x555d7da48d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:48.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.138+0000 7fd1a1ce6740 1 -- 172.21.15.164:0/302421562 shutdown_connections 2024-09-19T08:53:48.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.138+0000 7fd1a1ce6740 1 -- 172.21.15.164:0/302421562 wait complete. 2024-09-19T08:53:48.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.138+0000 7fd1a1ce6740 1 -- start start 2024-09-19T08:53:48.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.138+0000 7fd1a1ce6740 1 Processor -- start 2024-09-19T08:53:48.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.138+0000 7fd1a1ce6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d7da4d5d0 0x555d7da648e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:48.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.138+0000 7fd1a1ce6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d7da4eb40 0x555d7da64e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:48.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.138+0000 7fd1a1ce6740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d7da58370 0x555d7da5fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:48.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.138+0000 7fd1a1ce6740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd1981aca20 con 0x555d7da58370 2024-09-19T08:53:48.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.138+0000 7fd1a1ce6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd1981ac350 con 0x555d7da4eb40 2024-09-19T08:53:48.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.138+0000 7fd1a1ce6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd1981a95b0 con 0x555d7da4d5d0 2024-09-19T08:53:48.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.138+0000 7fd1a1ce6740 10 client.0 _ll_register_callbacks cb 0x7fd18c004380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:48.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.139+0000 7fd19deaa640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d7da58370 0x555d7da5fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:48.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.139+0000 7fd19deaa640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d7da58370 0x555d7da5fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48058/0 (socket says 172.21.15.164:48058) 2024-09-19T08:53:48.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.139+0000 7fd19deaa640 1 -- 172.21.15.164:0/2609752510 learned_addr learned my addr 172.21.15.164:0/2609752510 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:48.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.139+0000 7fd19d6a9640 1 --2- 172.21.15.164:0/2609752510 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d7da4d5d0 0x555d7da648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:48.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.139+0000 7fd19cea8640 1 --2- 172.21.15.164:0/2609752510 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d7da4eb40 0x555d7da64e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:48.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.139+0000 7fd19deaa640 1 -- 172.21.15.164:0/2609752510 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d7da4d5d0 msgr2=0x555d7da648e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:48.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.139+0000 7fd19deaa640 1 --2- 172.21.15.164:0/2609752510 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d7da4d5d0 0x555d7da648e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.139+0000 7fd19deaa640 1 -- 172.21.15.164:0/2609752510 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d7da4eb40 msgr2=0x555d7da64e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:48.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.139+0000 7fd19deaa640 1 --2- 172.21.15.164:0/2609752510 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d7da4eb40 0x555d7da64e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.139+0000 7fd19deaa640 1 -- 172.21.15.164:0/2609752510 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d7da60560 con 0x555d7da58370 2024-09-19T08:53:48.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.139+0000 7fd19deaa640 1 --2- 172.21.15.164:0/2609752510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d7da58370 0x555d7da5fd30 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fd190011920 tx=0x7fd190011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bcc4659910dc8c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:48.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.139+0000 7fd180ff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:53:48.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.139+0000 7fd180ff9640 1 -- 172.21.15.164:0/2609752510 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd190004d20 con 0x555d7da58370 2024-09-19T08:53:48.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.139+0000 7fd180ff9640 1 -- 172.21.15.164:0/2609752510 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7fd190010ac0 con 0x555d7da58370 2024-09-19T08:53:48.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.139+0000 7fd1a1ce6740 1 -- 172.21.15.164:0/2609752510 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd18c036040 con 0x555d7da58370 2024-09-19T08:53:48.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.140+0000 7fd180ff9640 1 -- 172.21.15.164:0/2609752510 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd190010ce0 con 0x555d7da58370 2024-09-19T08:53:48.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.140+0000 7fd1a1ce6740 10 client.17097 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:48.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.140+0000 7fd1a1ce6740 1 -- 172.21.15.164:0/2609752510 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555d7dad6e90 con 0x555d7da58370 2024-09-19T08:53:48.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.140+0000 7fd180ff9640 1 -- 172.21.15.164:0/2609752510 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd18c036040 con 0x555d7da58370 2024-09-19T08:53:48.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.140+0000 7fd180ff9640 1 -- 172.21.15.164:0/2609752510 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fd19001b070 con 0x555d7da58370 2024-09-19T08:53:48.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.140+0000 7fd1a1ce6740 10 client.17097 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:48.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.140+0000 7fd1a1ce6740 10 client.17097 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-19T08:53:48.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.140+0000 7fd1a1ce6740 20 client.17097 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:48.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.140+0000 7fd1a1ce6740 1 --2- 172.21.15.164:0/2609752510 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x555d7da6c4f0 0x555d7dad9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:48.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.140+0000 7fd1a1ce6740 4 client.17097 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:53:48.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.140+0000 7fd1a1ce6740 1 -- 172.21.15.164:0/2609752510 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555d7d937170 con 0x555d7da6c4f0 2024-09-19T08:53:48.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.140+0000 7fd19d6a9640 1 --2- 172.21.15.164:0/2609752510 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x555d7da6c4f0 0x555d7dad9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:48.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.141+0000 7fd180ff9640 10 client.17097 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:53:48.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.141+0000 7fd19d6a9640 1 --2- 172.21.15.164:0/2609752510 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x555d7da6c4f0 0x555d7dad9670 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f5d6849cd5e27ab0 server_cookie=2f5040643e267cbc in_seq=0 out_seq=0 2024-09-19T08:53:48.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.141+0000 7fd180ff9640 1 -- 172.21.15.164:0/2609752510 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555d7d937170 con 0x555d7da6c4f0 2024-09-19T08:53:48.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.141+0000 7fd180ff9640 10 client.17097 handle_command_reply: tid=0 2024-09-19T08:53:48.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.141+0000 7fd1a1ce6740 1 client.17097 shutdown 2024-09-19T08:53:48.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.141+0000 7fd1a1ce6740 1 -- 172.21.15.164:0/2609752510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d7da58370 msgr2=0x555d7da5fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:48.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.141+0000 7fd1a1ce6740 1 --2- 172.21.15.164:0/2609752510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d7da58370 0x555d7da5fd30 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fd190011920 tx=0x7fd190011de0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.141+0000 7fd1a1ce6740 1 -- 172.21.15.164:0/2609752510 shutdown_connections 2024-09-19T08:53:48.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.141+0000 7fd1a1ce6740 1 --2- 172.21.15.164:0/2609752510 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x555d7da6c4f0 0x555d7dad9670 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.141+0000 7fd1a1ce6740 1 --2- 172.21.15.164:0/2609752510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d7da58370 0x555d7da5fd30 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.141+0000 7fd1a1ce6740 1 --2- 172.21.15.164:0/2609752510 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555d7da4eb40 0x555d7da64e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.141+0000 7fd1a1ce6740 1 --2- 172.21.15.164:0/2609752510 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d7da4d5d0 0x555d7da648e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.141+0000 7fd180ff9640 0 client.17097 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:53:48.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.141+0000 7fd1a1ce6740 1 -- 172.21.15.164:0/2609752510 >> 172.21.15.164:0/2609752510 conn(0x555d7da16310 msgr2=0x555d7da568f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:48.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.141+0000 7fd1a1ce6740 1 -- 172.21.15.164:0/2609752510 shutdown_connections 2024-09-19T08:53:48.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.141+0000 7fd1a1ce6740 1 -- 172.21.15.164:0/2609752510 wait complete. 2024-09-19T08:53:48.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.141+0000 7fd1a1ce6740 20 client.17097 trim_cache size 0 max 16384 2024-09-19T08:53:48.149 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:53:48.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.142+0000 7fd19f90d640 1 -- 172.21.15.164:0/2049640002 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd1840a33b0 msgr2=0x7fd1840a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:48.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.142+0000 7fd19f90d640 1 --2- 172.21.15.164:0/2049640002 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd1840a33b0 0x7fd1840a5870 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd18c005fd0 tx=0x7fd18c00b390 comp rx=0 tx=0).stop 2024-09-19T08:53:48.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.142+0000 7fd19f90d640 1 -- 172.21.15.164:0/2049640002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd198102790 msgr2=0x7fd1981989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:48.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.142+0000 7fd19f90d640 1 --2- 172.21.15.164:0/2049640002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd198102790 0x7fd1981989f0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd194006010 tx=0x7fd1940042f0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.142+0000 7fd19f90d640 1 -- 172.21.15.164:0/2049640002 shutdown_connections 2024-09-19T08:53:48.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.142+0000 7fd19f90d640 1 --2- 172.21.15.164:0/2049640002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd198106f10 0x7fd19819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.142+0000 7fd19f90d640 1 --2- 172.21.15.164:0/2049640002 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd1840a33b0 0x7fd1840a5870 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.142+0000 7fd19f90d640 1 --2- 172.21.15.164:0/2049640002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd198102790 0x7fd1981989f0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.142+0000 7fd19f90d640 1 --2- 172.21.15.164:0/2049640002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd198101de0 0x7fd1981984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:48.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.142+0000 7fd19f90d640 1 -- 172.21.15.164:0/2049640002 >> 172.21.15.164:0/2049640002 conn(0x7fd1980fd8c0 msgr2=0x7fd1980fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:48.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.142+0000 7fd19f90d640 1 -- 172.21.15.164:0/2049640002 shutdown_connections 2024-09-19T08:53:48.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:48.142+0000 7fd19f90d640 1 -- 172.21.15.164:0/2049640002 wait complete. 2024-09-19T08:53:48.153 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:47 smithi164 ceph-mon[30928]: pgmap v246: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 op/s 2024-09-19T08:53:48.153 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:47 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/1157950519' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:53:48.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:47 smithi173 ceph-mon[35306]: pgmap v246: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 op/s 2024-09-19T08:53:48.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:47 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/1157950519' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:53:48.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:47 smithi170 ceph-mon[35072]: pgmap v246: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 op/s 2024-09-19T08:53:48.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:47 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/1157950519' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:53:48.954 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:53:48.960 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T08:53:49.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.446+0000 7f1598ce1640 1 -- 172.21.15.164:0/3302954389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1594102790 msgr2=0x7f15941067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:49.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.446+0000 7f1598ce1640 1 --2- 172.21.15.164:0/3302954389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1594102790 0x7f15941067e0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f1588009900 tx=0x7f158802a410 comp rx=0 tx=0).stop 2024-09-19T08:53:49.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.446+0000 7f1598ce1640 1 -- 172.21.15.164:0/3302954389 shutdown_connections 2024-09-19T08:53:49.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.446+0000 7f1598ce1640 1 --2- 172.21.15.164:0/3302954389 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1594106f10 0x7f159410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.446+0000 7f1598ce1640 1 --2- 172.21.15.164:0/3302954389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1594102790 0x7f15941067e0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.446+0000 7f1598ce1640 1 --2- 172.21.15.164:0/3302954389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1594101de0 0x7f15941021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.446+0000 7f1598ce1640 1 -- 172.21.15.164:0/3302954389 >> 172.21.15.164:0/3302954389 conn(0x7f15940fd8c0 msgr2=0x7f15940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:49.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.446+0000 7f1598ce1640 1 -- 172.21.15.164:0/3302954389 shutdown_connections 2024-09-19T08:53:49.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.446+0000 7f1598ce1640 1 -- 172.21.15.164:0/3302954389 wait complete. 2024-09-19T08:53:49.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.447+0000 7f1598ce1640 1 Processor -- start 2024-09-19T08:53:49.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.447+0000 7f1598ce1640 1 -- start start 2024-09-19T08:53:49.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.447+0000 7f1598ce1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1594101de0 0x7f15941983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:49.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.447+0000 7f1598ce1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1594102790 0x7f15941988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:49.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.447+0000 7f1598ce1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1594106f10 0x7f159419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:49.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.447+0000 7f1598ce1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1594111fc0 con 0x7f1594106f10 2024-09-19T08:53:49.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.447+0000 7f1598ce1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1594111e40 con 0x7f1594101de0 2024-09-19T08:53:49.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.447+0000 7f1598ce1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1594112140 con 0x7f1594102790 2024-09-19T08:53:49.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.447+0000 7f1592d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1594106f10 0x7f159419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:49.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.447+0000 7f1592d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1594106f10 0x7f159419cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48072/0 (socket says 172.21.15.164:48072) 2024-09-19T08:53:49.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.447+0000 7f1592d9d640 1 -- 172.21.15.164:0/3688023146 learned_addr learned my addr 172.21.15.164:0/3688023146 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:49.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.447+0000 7f159259c640 1 --2- 172.21.15.164:0/3688023146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1594101de0 0x7f15941983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:49.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.447+0000 7f1591d9b640 1 --2- 172.21.15.164:0/3688023146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1594102790 0x7f15941988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:49.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.447+0000 7f1592d9d640 1 -- 172.21.15.164:0/3688023146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1594102790 msgr2=0x7f15941988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:49.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.447+0000 7f1592d9d640 1 --2- 172.21.15.164:0/3688023146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1594102790 0x7f15941988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.447+0000 7f1592d9d640 1 -- 172.21.15.164:0/3688023146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1594101de0 msgr2=0x7f15941983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:49.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.447+0000 7f1592d9d640 1 --2- 172.21.15.164:0/3688023146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1594101de0 0x7f15941983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.447+0000 7f1592d9d640 1 -- 172.21.15.164:0/3688023146 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f159419d350 con 0x7f1594106f10 2024-09-19T08:53:49.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.447+0000 7f1591d9b640 1 --2- 172.21.15.164:0/3688023146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1594102790 0x7f15941988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:53:49.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.447+0000 7f1592d9d640 1 --2- 172.21.15.164:0/3688023146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1594106f10 0x7f159419cc70 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f158c00cad0 tx=0x7f158c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:49.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.448+0000 7f15837fe640 1 -- 172.21.15.164:0/3688023146 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158c00ad60 con 0x7f1594106f10 2024-09-19T08:53:49.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.448+0000 7f1598ce1640 1 -- 172.21.15.164:0/3688023146 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f159419d640 con 0x7f1594106f10 2024-09-19T08:53:49.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.448+0000 7f15837fe640 1 -- 172.21.15.164:0/3688023146 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f158c005020 con 0x7f1594106f10 2024-09-19T08:53:49.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.448+0000 7f15837fe640 1 -- 172.21.15.164:0/3688023146 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158c00f6b0 con 0x7f1594106f10 2024-09-19T08:53:49.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.448+0000 7f1598ce1640 1 -- 172.21.15.164:0/3688023146 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15941a4f20 con 0x7f1594106f10 2024-09-19T08:53:49.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.450+0000 7f15837fe640 1 -- 172.21.15.164:0/3688023146 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f158c005560 con 0x7f1594106f10 2024-09-19T08:53:49.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.451+0000 7f159b0ba740 1 Processor -- start 2024-09-19T08:53:49.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.451+0000 7f159b0ba740 1 -- start start 2024-09-19T08:53:49.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.451+0000 7f15837fe640 1 --2- 172.21.15.164:0/3688023146 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f157c0a33b0 0x7f157c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:49.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.451+0000 7f159b0ba740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df3adea060 0x55df3adea440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:49.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.451+0000 7f1591d9b640 1 --2- 172.21.15.164:0/3688023146 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f157c0a33b0 0x7f157c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:49.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.451+0000 7f159b0ba740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55df3adea980 0x55df3adf4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:49.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.451+0000 7f159b0ba740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df3adf5240 0x55df3adf7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:49.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.451+0000 7f159b0ba740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55df3ad4f620 con 0x55df3adea060 2024-09-19T08:53:49.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.451+0000 7f159b0ba740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55df3acf6e50 con 0x55df3adf5240 2024-09-19T08:53:49.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.451+0000 7f159b0ba740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55df3ad4b5e0 con 0x55df3adea980 2024-09-19T08:53:49.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.451+0000 7f15837fe640 1 -- 172.21.15.164:0/3688023146 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f158c0cfd40 con 0x7f1594106f10 2024-09-19T08:53:49.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.452+0000 7f159259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df3adea060 0x55df3adea440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:49.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.452+0000 7f159259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df3adea060 0x55df3adea440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48088/0 (socket says 172.21.15.164:48088) 2024-09-19T08:53:49.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.452+0000 7f159259c640 1 -- 172.21.15.164:0/1458988979 learned_addr learned my addr 172.21.15.164:0/1458988979 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:49.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.452+0000 7f1591d9b640 1 --2- 172.21.15.164:0/1458988979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55df3adea980 0x55df3adf4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:49.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.452+0000 7f1592d9d640 1 --2- 172.21.15.164:0/1458988979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df3adf5240 0x55df3adf7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:49.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.452+0000 7f159259c640 1 -- 172.21.15.164:0/1458988979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55df3adea980 msgr2=0x55df3adf4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:49.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.452+0000 7f159259c640 1 --2- 172.21.15.164:0/1458988979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55df3adea980 0x55df3adf4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.452+0000 7f159259c640 1 -- 172.21.15.164:0/1458988979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df3adf5240 msgr2=0x55df3adf7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:49.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.452+0000 7f159259c640 1 --2- 172.21.15.164:0/1458988979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df3adf5240 0x55df3adf7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.452+0000 7f159259c640 1 -- 172.21.15.164:0/1458988979 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df3ace7d40 con 0x55df3adea060 2024-09-19T08:53:49.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.452+0000 7f1592d9d640 1 --2- 172.21.15.164:0/1458988979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df3adf5240 0x55df3adf7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:53:49.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.452+0000 7f1591d9b640 1 --2- 172.21.15.164:0/3688023146 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f157c0a33b0 0x7f157c0a5870 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1588000c00 tx=0x7f1588004160 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:49.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.452+0000 7f159259c640 1 --2- 172.21.15.164:0/1458988979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df3adea060 0x55df3adea440 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f158400ad00 tx=0x7f158400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee95c4d39b27364 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:49.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.452+0000 7f1580ff9640 1 -- 172.21.15.164:0/1458988979 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158401d070 con 0x55df3adea060 2024-09-19T08:53:49.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.452+0000 7f1580ff9640 1 -- 172.21.15.164:0/1458988979 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1584018b20 con 0x55df3adea060 2024-09-19T08:53:49.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.452+0000 7f1580ff9640 1 -- 172.21.15.164:0/1458988979 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1584010bc0 con 0x55df3adea060 2024-09-19T08:53:49.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.453+0000 7f159b0ba740 1 -- 172.21.15.164:0/1458988979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df3adea060 msgr2=0x55df3adea440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:49.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.453+0000 7f159b0ba740 1 --2- 172.21.15.164:0/1458988979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df3adea060 0x55df3adea440 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f158400ad00 tx=0x7f158400f790 comp rx=0 tx=0).stop 2024-09-19T08:53:49.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.453+0000 7f159b0ba740 1 -- 172.21.15.164:0/1458988979 shutdown_connections 2024-09-19T08:53:49.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.453+0000 7f159b0ba740 1 --2- 172.21.15.164:0/1458988979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df3adf5240 0x55df3adf7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.453+0000 7f159b0ba740 1 --2- 172.21.15.164:0/1458988979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55df3adea980 0x55df3adf4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.453+0000 7f159b0ba740 1 --2- 172.21.15.164:0/1458988979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df3adea060 0x55df3adea440 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.453+0000 7f159b0ba740 1 -- 172.21.15.164:0/1458988979 >> 172.21.15.164:0/1458988979 conn(0x55df3ade65f0 msgr2=0x55df3ade69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:49.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.453+0000 7f159b0ba740 1 -- 172.21.15.164:0/1458988979 shutdown_connections 2024-09-19T08:53:49.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.453+0000 7f159b0ba740 1 -- 172.21.15.164:0/1458988979 wait complete. 2024-09-19T08:53:49.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.453+0000 7f159b0ba740 1 -- start start 2024-09-19T08:53:49.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.453+0000 7f159b0ba740 1 Processor -- start 2024-09-19T08:53:49.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.454+0000 7f159b0ba740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df3adea060 0x55df3adf2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:49.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.454+0000 7f159b0ba740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55df3adea980 0x55df3adf2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:49.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.454+0000 7f159b0ba740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df3adf5240 0x55df3adf2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:49.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.454+0000 7f159b0ba740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55df3adf7e10 con 0x55df3adea060 2024-09-19T08:53:49.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.454+0000 7f159b0ba740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55df3acd7f10 con 0x55df3adf5240 2024-09-19T08:53:49.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.454+0000 7f159b0ba740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55df3adf7f90 con 0x55df3adea980 2024-09-19T08:53:49.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.454+0000 7f159b0ba740 10 client.0 _ll_register_callbacks cb 0x55df3acfbdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:49.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.454+0000 7f159259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df3adea060 0x55df3adf2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:49.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.454+0000 7f159259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df3adea060 0x55df3adf2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48102/0 (socket says 172.21.15.164:48102) 2024-09-19T08:53:49.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.454+0000 7f159259c640 1 -- 172.21.15.164:0/3251612373 learned_addr learned my addr 172.21.15.164:0/3251612373 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:49.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.454+0000 7f1592d9d640 1 --2- 172.21.15.164:0/3251612373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df3adf5240 0x55df3adf2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:49.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.454+0000 7f159259c640 1 -- 172.21.15.164:0/3251612373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55df3adea980 msgr2=0x55df3adf2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:49.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.454+0000 7f1591d9b640 1 --2- 172.21.15.164:0/3251612373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55df3adea980 0x55df3adf2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:49.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.454+0000 7f159259c640 1 --2- 172.21.15.164:0/3251612373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55df3adea980 0x55df3adf2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.454+0000 7f159259c640 1 -- 172.21.15.164:0/3251612373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df3adf5240 msgr2=0x55df3adf2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:49.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.454+0000 7f159259c640 1 --2- 172.21.15.164:0/3251612373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df3adf5240 0x55df3adf2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.454+0000 7f159259c640 1 -- 172.21.15.164:0/3251612373 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df3aa5e580 con 0x55df3adea060 2024-09-19T08:53:49.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.454+0000 7f1592d9d640 1 --2- 172.21.15.164:0/3251612373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df3adf5240 0x55df3adf2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:53:49.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.454+0000 7f1591d9b640 1 --2- 172.21.15.164:0/3251612373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55df3adea980 0x55df3adf2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:53:49.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.454+0000 7f159259c640 1 --2- 172.21.15.164:0/3251612373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df3adea060 0x55df3adf2120 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f158401b040 tx=0x7f1584010bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=364819d4960fefe7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:49.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.454+0000 7f1579ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:53:49.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.454+0000 7f1579ffb640 1 -- 172.21.15.164:0/3251612373 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158401d040 con 0x55df3adea060 2024-09-19T08:53:49.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.455+0000 7f1579ffb640 1 -- 172.21.15.164:0/3251612373 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f1584018440 con 0x55df3adea060 2024-09-19T08:53:49.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.455+0000 7f159b0ba740 1 -- 172.21.15.164:0/3251612373 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1584018b20 con 0x55df3adea060 2024-09-19T08:53:49.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.455+0000 7f1579ffb640 1 -- 172.21.15.164:0/3251612373 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1584031050 con 0x55df3adea060 2024-09-19T08:53:49.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.455+0000 7f159b0ba740 10 client.17109 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:49.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.455+0000 7f159b0ba740 1 -- 172.21.15.164:0/3251612373 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55df3ab28960 con 0x55df3adea060 2024-09-19T08:53:49.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.455+0000 7f1579ffb640 1 -- 172.21.15.164:0/3251612373 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1584018b20 con 0x55df3adea060 2024-09-19T08:53:49.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.455+0000 7f1579ffb640 1 -- 172.21.15.164:0/3251612373 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f1584020030 con 0x55df3adea060 2024-09-19T08:53:49.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.455+0000 7f159b0ba740 10 client.17109 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:49.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.455+0000 7f159b0ba740 10 client.17109 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:53:49.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.455+0000 7f159b0ba740 20 client.17109 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:49.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.455+0000 7f159b0ba740 1 --2- 172.21.15.164:0/3251612373 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55df3ae04cc0 0x55df3ae070b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:49.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.455+0000 7f159b0ba740 4 client.17109 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:53:49.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.455+0000 7f159b0ba740 1 -- 172.21.15.164:0/3251612373 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df3ad5cb90 con 0x55df3ae04cc0 2024-09-19T08:53:49.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.456+0000 7f1591d9b640 1 --2- 172.21.15.164:0/3251612373 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55df3ae04cc0 0x55df3ae070b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:49.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.456+0000 7f1579ffb640 10 client.17109 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:49.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.456+0000 7f1591d9b640 1 --2- 172.21.15.164:0/3251612373 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55df3ae04cc0 0x55df3ae070b0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3169fc1104a045e6 server_cookie=2e99fa7e43b19585 in_seq=0 out_seq=0 2024-09-19T08:53:49.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.457+0000 7f1579ffb640 1 -- 172.21.15.164:0/3251612373 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55df3ad5cb90 con 0x55df3ae04cc0 2024-09-19T08:53:49.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.457+0000 7f1579ffb640 10 client.17109 handle_command_reply: tid=0 2024-09-19T08:53:49.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.457+0000 7f159b0ba740 1 client.17109 shutdown 2024-09-19T08:53:49.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.457+0000 7f159b0ba740 1 -- 172.21.15.164:0/3251612373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df3adea060 msgr2=0x55df3adf2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:49.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.457+0000 7f159b0ba740 1 --2- 172.21.15.164:0/3251612373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df3adea060 0x55df3adf2120 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f158401b040 tx=0x7f1584010bd0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.457+0000 7f159b0ba740 1 -- 172.21.15.164:0/3251612373 shutdown_connections 2024-09-19T08:53:49.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.457+0000 7f159b0ba740 1 --2- 172.21.15.164:0/3251612373 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55df3ae04cc0 0x55df3ae070b0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.457+0000 7f159b0ba740 1 --2- 172.21.15.164:0/3251612373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df3adf5240 0x55df3adf2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.457+0000 7f159b0ba740 1 --2- 172.21.15.164:0/3251612373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55df3adea980 0x55df3adf2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.457+0000 7f159b0ba740 1 --2- 172.21.15.164:0/3251612373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df3adea060 0x55df3adf2120 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.457+0000 7f1579ffb640 0 client.17109 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:49.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.457+0000 7f159b0ba740 1 -- 172.21.15.164:0/3251612373 >> 172.21.15.164:0/3251612373 conn(0x55df3ade65f0 msgr2=0x55df3adec2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:49.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.457+0000 7f159b0ba740 1 -- 172.21.15.164:0/3251612373 shutdown_connections 2024-09-19T08:53:49.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.457+0000 7f159b0ba740 1 -- 172.21.15.164:0/3251612373 wait complete. 2024-09-19T08:53:49.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.457+0000 7f159b0ba740 20 client.17109 trim_cache size 0 max 16384 2024-09-19T08:53:49.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.477+0000 7f159b0ba740 1 Processor -- start 2024-09-19T08:53:49.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.477+0000 7f159b0ba740 1 -- start start 2024-09-19T08:53:49.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.477+0000 7f159b0ba740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df3adea060 0x55df3ade69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:49.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.477+0000 7f159b0ba740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55df3adea980 0x55df3ade5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:49.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.477+0000 7f159b0ba740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df3adf5240 0x55df3ae0c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:49.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.477+0000 7f159b0ba740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f15941ad800 con 0x55df3adf5240 2024-09-19T08:53:49.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.477+0000 7f159b0ba740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f15941af340 con 0x55df3adea060 2024-09-19T08:53:49.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.477+0000 7f159b0ba740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f15941aded0 con 0x55df3adea980 2024-09-19T08:53:49.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.478+0000 7f1591d9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55df3adea980 0x55df3ade5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:49.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.478+0000 7f1591d9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55df3adea980 0x55df3ade5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:38008/0 (socket says 172.21.15.164:38008) 2024-09-19T08:53:49.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.478+0000 7f159259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df3adea060 0x55df3ade69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:49.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.478+0000 7f1591d9b640 1 -- 172.21.15.164:0/521432478 learned_addr learned my addr 172.21.15.164:0/521432478 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:49.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.478+0000 7f1592d9d640 1 --2- 172.21.15.164:0/521432478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df3adf5240 0x55df3ae0c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:49.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.478+0000 7f1591d9b640 1 -- 172.21.15.164:0/521432478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df3adea060 msgr2=0x55df3ade69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:49.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.478+0000 7f1591d9b640 1 --2- 172.21.15.164:0/521432478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df3adea060 0x55df3ade69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.478+0000 7f1591d9b640 1 -- 172.21.15.164:0/521432478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df3adf5240 msgr2=0x55df3ae0c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:49.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.478+0000 7f1591d9b640 1 --2- 172.21.15.164:0/521432478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df3adf5240 0x55df3ae0c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.478+0000 7f1591d9b640 1 -- 172.21.15.164:0/521432478 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df3adf4060 con 0x55df3adea980 2024-09-19T08:53:49.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.478+0000 7f1592d9d640 1 --2- 172.21.15.164:0/521432478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df3adf5240 0x55df3ae0c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:53:49.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.478+0000 7f1591d9b640 1 --2- 172.21.15.164:0/521432478 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55df3adea980 0x55df3ade5640 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1588009230 tx=0x7f1588044000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6aaa5cad82ad484f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:49.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.479+0000 7f157bfff640 1 -- 172.21.15.164:0/521432478 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1588058070 con 0x55df3adea980 2024-09-19T08:53:49.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.479+0000 7f157bfff640 1 -- 172.21.15.164:0/521432478 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1588003f80 con 0x55df3adea980 2024-09-19T08:53:49.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.479+0000 7f157bfff640 1 -- 172.21.15.164:0/521432478 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1588035050 con 0x55df3adea980 2024-09-19T08:53:49.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.480+0000 7f159b0ba740 1 -- 172.21.15.164:0/521432478 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55df3adea980 msgr2=0x55df3ade5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:49.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.480+0000 7f159b0ba740 1 --2- 172.21.15.164:0/521432478 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55df3adea980 0x55df3ade5640 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1588009230 tx=0x7f1588044000 comp rx=0 tx=0).stop 2024-09-19T08:53:49.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.480+0000 7f159b0ba740 1 -- 172.21.15.164:0/521432478 shutdown_connections 2024-09-19T08:53:49.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.480+0000 7f159b0ba740 1 --2- 172.21.15.164:0/521432478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df3adf5240 0x55df3ae0c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.480+0000 7f159b0ba740 1 --2- 172.21.15.164:0/521432478 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55df3adea980 0x55df3ade5640 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.480+0000 7f159b0ba740 1 --2- 172.21.15.164:0/521432478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df3adea060 0x55df3ade69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.480+0000 7f159b0ba740 1 -- 172.21.15.164:0/521432478 >> 172.21.15.164:0/521432478 conn(0x55df3ade65f0 msgr2=0x55df3ade4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:49.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.480+0000 7f159b0ba740 1 -- 172.21.15.164:0/521432478 shutdown_connections 2024-09-19T08:53:49.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.480+0000 7f159b0ba740 1 -- 172.21.15.164:0/521432478 wait complete. 2024-09-19T08:53:49.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.480+0000 7f159b0ba740 1 -- start start 2024-09-19T08:53:49.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.480+0000 7f159b0ba740 1 Processor -- start 2024-09-19T08:53:49.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.481+0000 7f159b0ba740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df3adea060 0x55df3adf98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:49.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.481+0000 7f159b0ba740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55df3adea980 0x55df3adf9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:49.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.481+0000 7f159b0ba740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df3adf5240 0x55df3adfceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:49.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.481+0000 7f159b0ba740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f15941aec70 con 0x55df3adf5240 2024-09-19T08:53:49.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.481+0000 7f159b0ba740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f15941ae5a0 con 0x55df3adea060 2024-09-19T08:53:49.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.481+0000 7f159b0ba740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f15941ab800 con 0x55df3adea980 2024-09-19T08:53:49.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.481+0000 7f159b0ba740 10 client.0 _ll_register_callbacks cb 0x7f158400a960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:49.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.481+0000 7f1591d9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55df3adea980 0x55df3adf9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:49.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.481+0000 7f159259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df3adea060 0x55df3adf98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:49.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.481+0000 7f1591d9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55df3adea980 0x55df3adf9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:38020/0 (socket says 172.21.15.164:38020) 2024-09-19T08:53:49.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.481+0000 7f1591d9b640 1 -- 172.21.15.164:0/3445504976 learned_addr learned my addr 172.21.15.164:0/3445504976 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:49.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.481+0000 7f1592d9d640 1 --2- 172.21.15.164:0/3445504976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df3adf5240 0x55df3adfceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:49.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.481+0000 7f1591d9b640 1 -- 172.21.15.164:0/3445504976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df3adea060 msgr2=0x55df3adf98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:49.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.481+0000 7f1591d9b640 1 --2- 172.21.15.164:0/3445504976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df3adea060 0x55df3adf98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.481+0000 7f1591d9b640 1 -- 172.21.15.164:0/3445504976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df3adf5240 msgr2=0x55df3adfceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:49.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.482+0000 7f1591d9b640 1 --2- 172.21.15.164:0/3445504976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df3adf5240 0x55df3adfceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.482+0000 7f1591d9b640 1 -- 172.21.15.164:0/3445504976 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df3adea7b0 con 0x55df3adea980 2024-09-19T08:53:49.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.482+0000 7f1592d9d640 1 --2- 172.21.15.164:0/3445504976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df3adf5240 0x55df3adfceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:53:49.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.482+0000 7f1591d9b640 1 --2- 172.21.15.164:0/3445504976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55df3adea980 0x55df3adf9df0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1588004650 tx=0x7f15880037e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb81440cb933d4d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:49.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.482+0000 7f1579ffb640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-09-19T08:53:49.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.482+0000 7f1579ffb640 1 -- 172.21.15.164:0/3445504976 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1588058070 con 0x55df3adea980 2024-09-19T08:53:49.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.482+0000 7f1579ffb640 1 -- 172.21.15.164:0/3445504976 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f1588003cb0 con 0x55df3adea980 2024-09-19T08:53:49.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.482+0000 7f159b0ba740 1 -- 172.21.15.164:0/3445504976 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1588003f80 con 0x55df3adea980 2024-09-19T08:53:49.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.482+0000 7f1579ffb640 1 -- 172.21.15.164:0/3445504976 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f15880315b0 con 0x55df3adea980 2024-09-19T08:53:49.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.482+0000 7f159b0ba740 10 client.26284 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:49.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.483+0000 7f159b0ba740 1 -- 172.21.15.164:0/3445504976 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55df3ae6ffa0 con 0x55df3adea980 2024-09-19T08:53:49.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.483+0000 7f1579ffb640 1 -- 172.21.15.164:0/3445504976 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1588003f80 con 0x55df3adea980 2024-09-19T08:53:49.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.484+0000 7f1579ffb640 1 -- 172.21.15.164:0/3445504976 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f1584019ce0 con 0x55df3adea980 2024-09-19T08:53:49.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.484+0000 7f159b0ba740 10 client.26284 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:49.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.484+0000 7f159b0ba740 10 client.26284 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:53:49.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.484+0000 7f159b0ba740 20 client.26284 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:49.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.484+0000 7f159b0ba740 1 --2- 172.21.15.164:0/3445504976 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55df3ae706c0 0x55df3ae72ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:49.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.484+0000 7f159b0ba740 4 client.26284 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d0313327", "release": true, "await": true}] 2024-09-19T08:53:49.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.484+0000 7f159b0ba740 1 -- 172.21.15.164:0/3445504976 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d0313327", "release": true, "await": true}) -- 0x55df3ad570a0 con 0x55df3ae706c0 2024-09-19T08:53:49.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.484+0000 7f159259c640 1 --2- 172.21.15.164:0/3445504976 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55df3ae706c0 0x55df3ae72ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:49.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.484+0000 7f159259c640 1 --2- 172.21.15.164:0/3445504976 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55df3ae706c0 0x55df3ae72ae0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=952507a38f9f444a server_cookie=88d20494050f66ab in_seq=0 out_seq=0 2024-09-19T08:53:49.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.484+0000 7f1579ffb640 10 client.26284 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:49.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.488+0000 7f1579ffb640 1 -- 172.21.15.164:0/3445504976 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55df3ad570a0 con 0x55df3ae706c0 2024-09-19T08:53:49.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.488+0000 7f1579ffb640 10 client.26284 handle_command_reply: tid=0 2024-09-19T08:53:49.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.488+0000 7f159b0ba740 1 client.26284 shutdown 2024-09-19T08:53:49.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.488+0000 7f159b0ba740 1 -- 172.21.15.164:0/3445504976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55df3adea980 msgr2=0x55df3adf9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:49.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.488+0000 7f159b0ba740 1 --2- 172.21.15.164:0/3445504976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55df3adea980 0x55df3adf9df0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1588004650 tx=0x7f15880037e0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.488+0000 7f159b0ba740 1 -- 172.21.15.164:0/3445504976 shutdown_connections 2024-09-19T08:53:49.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.488+0000 7f159b0ba740 1 --2- 172.21.15.164:0/3445504976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df3adf5240 0x55df3adfceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.488+0000 7f159b0ba740 1 --2- 172.21.15.164:0/3445504976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55df3adea980 0x55df3adf9df0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.488+0000 7f159b0ba740 1 --2- 172.21.15.164:0/3445504976 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55df3ae706c0 0x55df3ae72ae0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.488+0000 7f159b0ba740 1 --2- 172.21.15.164:0/3445504976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df3adea060 0x55df3adf98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.488+0000 7f1579ffb640 0 client.26284 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:49.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.488+0000 7f159b0ba740 1 -- 172.21.15.164:0/3445504976 >> 172.21.15.164:0/3445504976 conn(0x55df3ade65f0 msgr2=0x55df3adec2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:49.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.489+0000 7f159b0ba740 1 -- 172.21.15.164:0/3445504976 shutdown_connections 2024-09-19T08:53:49.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.489+0000 7f159b0ba740 1 -- 172.21.15.164:0/3445504976 wait complete. 2024-09-19T08:53:49.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.489+0000 7f159b0ba740 20 client.26284 trim_cache size 0 max 16384 2024-09-19T08:53:49.492 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:53:49.492 INFO:teuthology.orchestra.run.smithi164.stdout: "epoch": 32, 2024-09-19T08:53:49.492 INFO:teuthology.orchestra.run.smithi164.stdout: "leader": 24479, 2024-09-19T08:53:49.492 INFO:teuthology.orchestra.run.smithi164.stdout: "set_version": 28, 2024-09-19T08:53:49.492 INFO:teuthology.orchestra.run.smithi164.stdout: "sets": { 2024-09-19T08:53:49.492 INFO:teuthology.orchestra.run.smithi164.stdout: "d0313327": { 2024-09-19T08:53:49.492 INFO:teuthology.orchestra.run.smithi164.stdout: "version": 28, 2024-09-19T08:53:49.493 INFO:teuthology.orchestra.run.smithi164.stdout: "age_ref": 0.0, 2024-09-19T08:53:49.493 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:53:49.493 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "RELEASED", 2024-09-19T08:53:49.493 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:53:49.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:53:49.493 INFO:teuthology.orchestra.run.smithi164.stdout: "timeout": 90.0, 2024-09-19T08:53:49.493 INFO:teuthology.orchestra.run.smithi164.stdout: "expiration": 145.8, 2024-09-19T08:53:49.493 INFO:teuthology.orchestra.run.smithi164.stdout: "members": { 2024-09-19T08:53:49.493 INFO:teuthology.orchestra.run.smithi164.stdout: "file:/": { 2024-09-19T08:53:49.493 INFO:teuthology.orchestra.run.smithi164.stdout: "excluded": false, 2024-09-19T08:53:49.493 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:53:49.493 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "RELEASED", 2024-09-19T08:53:49.493 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:53:49.494 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:53:49.494 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:53:49.494 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:53:49.494 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:53:49.494 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:53:49.494 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:53:49.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.489+0000 7f1598ce1640 1 -- 172.21.15.164:0/3688023146 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f157c0a33b0 msgr2=0x7f157c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:49.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.489+0000 7f1598ce1640 1 --2- 172.21.15.164:0/3688023146 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f157c0a33b0 0x7f157c0a5870 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1588000c00 tx=0x7f1588004160 comp rx=0 tx=0).stop 2024-09-19T08:53:49.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.489+0000 7f1598ce1640 1 -- 172.21.15.164:0/3688023146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1594106f10 msgr2=0x7f159419cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:49.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.489+0000 7f1598ce1640 1 --2- 172.21.15.164:0/3688023146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1594106f10 0x7f159419cc70 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f158c00cad0 tx=0x7f158c00ced0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.490+0000 7f1598ce1640 1 -- 172.21.15.164:0/3688023146 shutdown_connections 2024-09-19T08:53:49.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.490+0000 7f1598ce1640 1 --2- 172.21.15.164:0/3688023146 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f157c0a33b0 0x7f157c0a5870 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.490+0000 7f1598ce1640 1 --2- 172.21.15.164:0/3688023146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1594106f10 0x7f159419cc70 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.490+0000 7f1598ce1640 1 --2- 172.21.15.164:0/3688023146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1594102790 0x7f15941988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.490+0000 7f1598ce1640 1 --2- 172.21.15.164:0/3688023146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1594101de0 0x7f15941983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:49.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.490+0000 7f1598ce1640 1 -- 172.21.15.164:0/3688023146 >> 172.21.15.164:0/3688023146 conn(0x7f15940fd8c0 msgr2=0x7f15940febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:49.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.490+0000 7f1598ce1640 1 -- 172.21.15.164:0/3688023146 shutdown_connections 2024-09-19T08:53:49.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:49.490+0000 7f1598ce1640 1 -- 172.21.15.164:0/3688023146 wait complete. 2024-09-19T08:53:49.501 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:53:50.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:49 smithi173 ceph-mon[35306]: pgmap v247: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:53:50.177 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:49 smithi170 ceph-mon[35072]: pgmap v247: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:53:50.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:49 smithi164 ceph-mon[30928]: pgmap v247: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:53:50.361 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd0313327', total seconds elapsed: 29.9 2024-09-19T08:53:50.361 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.75 2024-09-19T08:53:50.362 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:53:50.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.841+0000 7fb84d831640 1 -- 172.21.15.164:0/3259757720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb848102ef0 msgr2=0x7fb84810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:50.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.841+0000 7fb84d831640 1 --2- 172.21.15.164:0/3259757720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb848102ef0 0x7fb84810da40 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fb840009ca0 tx=0x7fb84002a950 comp rx=0 tx=0).stop 2024-09-19T08:53:50.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.841+0000 7fb84d831640 1 -- 172.21.15.164:0/3259757720 shutdown_connections 2024-09-19T08:53:50.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.841+0000 7fb84d831640 1 --2- 172.21.15.164:0/3259757720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb848102ef0 0x7fb84810da40 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.841+0000 7fb84d831640 1 --2- 172.21.15.164:0/3259757720 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb848102570 0x7fb8481029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.841+0000 7fb84d831640 1 --2- 172.21.15.164:0/3259757720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb848100e20 0x7fb848101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.841+0000 7fb84d831640 1 -- 172.21.15.164:0/3259757720 >> 172.21.15.164:0/3259757720 conn(0x7fb8480fc7d0 msgr2=0x7fb8480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:50.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.841+0000 7fb84d831640 1 -- 172.21.15.164:0/3259757720 shutdown_connections 2024-09-19T08:53:50.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.841+0000 7fb84d831640 1 -- 172.21.15.164:0/3259757720 wait complete. 2024-09-19T08:53:50.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.841+0000 7fb84d831640 1 Processor -- start 2024-09-19T08:53:50.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.841+0000 7fb84d831640 1 -- start start 2024-09-19T08:53:50.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.842+0000 7fb84d831640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb848100e20 0x7fb8481984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:50.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.842+0000 7fb84d831640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb848102570 0x7fb848198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:50.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.842+0000 7fb84d831640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb848102ef0 0x7fb84819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:50.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.842+0000 7fb84d831640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb848110f80 con 0x7fb848100e20 2024-09-19T08:53:50.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.842+0000 7fb84d831640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb848110e00 con 0x7fb848102570 2024-09-19T08:53:50.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.842+0000 7fb84d831640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb848111100 con 0x7fb848102ef0 2024-09-19T08:53:50.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.842+0000 7fb846ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb848100e20 0x7fb8481984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:50.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.842+0000 7fb846ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb848100e20 0x7fb8481984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48130/0 (socket says 172.21.15.164:48130) 2024-09-19T08:53:50.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.842+0000 7fb846ffd640 1 -- 172.21.15.164:0/4266380785 learned_addr learned my addr 172.21.15.164:0/4266380785 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:50.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.842+0000 7fb8467fc640 1 --2- 172.21.15.164:0/4266380785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb848102570 0x7fb848198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:50.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.842+0000 7fb8477fe640 1 --2- 172.21.15.164:0/4266380785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb848102ef0 0x7fb84819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:50.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.842+0000 7fb846ffd640 1 -- 172.21.15.164:0/4266380785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb848102ef0 msgr2=0x7fb84819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:50.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.842+0000 7fb846ffd640 1 --2- 172.21.15.164:0/4266380785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb848102ef0 0x7fb84819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.842+0000 7fb846ffd640 1 -- 172.21.15.164:0/4266380785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb848102570 msgr2=0x7fb848198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:50.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.842+0000 7fb846ffd640 1 --2- 172.21.15.164:0/4266380785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb848102570 0x7fb848198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.842+0000 7fb846ffd640 1 -- 172.21.15.164:0/4266380785 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb84819d480 con 0x7fb848100e20 2024-09-19T08:53:50.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.842+0000 7fb8467fc640 1 --2- 172.21.15.164:0/4266380785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb848102570 0x7fb848198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:53:50.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.842+0000 7fb846ffd640 1 --2- 172.21.15.164:0/4266380785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb848100e20 0x7fb8481984d0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fb83800c890 tx=0x7fb83800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:50.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.843+0000 7fb84c82f640 1 -- 172.21.15.164:0/4266380785 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb838004220 con 0x7fb848100e20 2024-09-19T08:53:50.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.843+0000 7fb84c82f640 1 -- 172.21.15.164:0/4266380785 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8380043c0 con 0x7fb848100e20 2024-09-19T08:53:50.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.843+0000 7fb84d831640 1 -- 172.21.15.164:0/4266380785 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb84819d770 con 0x7fb848100e20 2024-09-19T08:53:50.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.843+0000 7fb84c82f640 1 -- 172.21.15.164:0/4266380785 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8380049e0 con 0x7fb848100e20 2024-09-19T08:53:50.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.843+0000 7fb84d831640 1 -- 172.21.15.164:0/4266380785 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8481a5050 con 0x7fb848100e20 2024-09-19T08:53:50.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.845+0000 7fb84c82f640 1 -- 172.21.15.164:0/4266380785 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fb8380079b0 con 0x7fb848100e20 2024-09-19T08:53:50.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.846+0000 7fb84c82f640 1 --2- 172.21.15.164:0/4266380785 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb8300a33b0 0x7fb8300a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:50.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.846+0000 7fb8467fc640 1 --2- 172.21.15.164:0/4266380785 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb8300a33b0 0x7fb8300a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:50.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.846+0000 7fb84c82f640 1 -- 172.21.15.164:0/4266380785 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7fb8380cfd40 con 0x7fb848100e20 2024-09-19T08:53:50.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.846+0000 7fb84fc0a740 1 Processor -- start 2024-09-19T08:53:50.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.846+0000 7fb84fc0a740 1 -- start start 2024-09-19T08:53:50.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.846+0000 7fb8467fc640 1 --2- 172.21.15.164:0/4266380785 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb8300a33b0 0x7fb8300a5870 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb83c005fd0 tx=0x7fb83c005850 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:50.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.846+0000 7fb84fc0a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c4b558b40 0x558c4b557090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:50.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.847+0000 7fb84fc0a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c4b5575d0 0x558c4b557a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:50.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.847+0000 7fb84fc0a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c4b566780 0x558c4b568b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:50.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.847+0000 7fb84fc0a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558c4b4bd2f0 con 0x558c4b566780 2024-09-19T08:53:50.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.847+0000 7fb84fc0a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558c4b464cf0 con 0x558c4b5575d0 2024-09-19T08:53:50.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.847+0000 7fb84fc0a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558c4b4b92b0 con 0x558c4b558b40 2024-09-19T08:53:50.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.847+0000 7fb8477fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c4b566780 0x558c4b568b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:50.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.847+0000 7fb8477fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c4b566780 0x558c4b568b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48134/0 (socket says 172.21.15.164:48134) 2024-09-19T08:53:50.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.847+0000 7fb8477fe640 1 -- 172.21.15.164:0/366743193 learned_addr learned my addr 172.21.15.164:0/366743193 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:50.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.847+0000 7fb8467fc640 1 --2- 172.21.15.164:0/366743193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c4b5575d0 0x558c4b557a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:50.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.847+0000 7fb8477fe640 1 -- 172.21.15.164:0/366743193 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c4b558b40 msgr2=0x558c4b557090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:50.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.847+0000 7fb846ffd640 1 --2- 172.21.15.164:0/366743193 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c4b558b40 0x558c4b557090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:50.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.847+0000 7fb8477fe640 1 --2- 172.21.15.164:0/366743193 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c4b558b40 0x558c4b557090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.847+0000 7fb8477fe640 1 -- 172.21.15.164:0/366743193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c4b5575d0 msgr2=0x558c4b557a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:50.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.847+0000 7fb8477fe640 1 --2- 172.21.15.164:0/366743193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c4b5575d0 0x558c4b557a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.847+0000 7fb8477fe640 1 -- 172.21.15.164:0/366743193 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c4b4493a0 con 0x558c4b566780 2024-09-19T08:53:50.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.847+0000 7fb8467fc640 1 --2- 172.21.15.164:0/366743193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c4b5575d0 0x558c4b557a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:53:50.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.847+0000 7fb8477fe640 1 --2- 172.21.15.164:0/366743193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c4b566780 0x558c4b568b70 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fb840035000 tx=0x7fb8400098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8b560cc14ff827 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:50.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.847+0000 7fb82dffb640 1 -- 172.21.15.164:0/366743193 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb840048070 con 0x558c4b566780 2024-09-19T08:53:50.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.848+0000 7fb82dffb640 1 -- 172.21.15.164:0/366743193 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb84002fa70 con 0x558c4b566780 2024-09-19T08:53:50.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.848+0000 7fb82dffb640 1 -- 172.21.15.164:0/366743193 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb840034040 con 0x558c4b566780 2024-09-19T08:53:50.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.848+0000 7fb84fc0a740 1 -- 172.21.15.164:0/366743193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c4b566780 msgr2=0x558c4b568b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:50.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.848+0000 7fb84fc0a740 1 --2- 172.21.15.164:0/366743193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c4b566780 0x558c4b568b70 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fb840035000 tx=0x7fb8400098d0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.848+0000 7fb84fc0a740 1 -- 172.21.15.164:0/366743193 shutdown_connections 2024-09-19T08:53:50.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.848+0000 7fb84fc0a740 1 --2- 172.21.15.164:0/366743193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c4b566780 0x558c4b568b70 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.848+0000 7fb84fc0a740 1 --2- 172.21.15.164:0/366743193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c4b5575d0 0x558c4b557a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.848+0000 7fb84fc0a740 1 --2- 172.21.15.164:0/366743193 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c4b558b40 0x558c4b557090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.848+0000 7fb84fc0a740 1 -- 172.21.15.164:0/366743193 >> 172.21.15.164:0/366743193 conn(0x558c4b520310 msgr2=0x558c4b553c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:50.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.848+0000 7fb84fc0a740 1 -- 172.21.15.164:0/366743193 shutdown_connections 2024-09-19T08:53:50.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.848+0000 7fb84fc0a740 1 -- 172.21.15.164:0/366743193 wait complete. 2024-09-19T08:53:50.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.849+0000 7fb84fc0a740 1 -- start start 2024-09-19T08:53:50.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.849+0000 7fb84fc0a740 1 Processor -- start 2024-09-19T08:53:50.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.849+0000 7fb84fc0a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c4b5575d0 0x558c4b5661f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:50.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.849+0000 7fb84fc0a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c4b558b40 0x558c4b5634a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:50.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.849+0000 7fb84fc0a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c4b566780 0x558c4b563be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:50.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.849+0000 7fb84fc0a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558c4b569420 con 0x558c4b5575d0 2024-09-19T08:53:50.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.849+0000 7fb84fc0a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558c4b478060 con 0x558c4b558b40 2024-09-19T08:53:50.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.849+0000 7fb84fc0a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558c4b5695a0 con 0x558c4b566780 2024-09-19T08:53:50.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.849+0000 7fb846ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c4b5575d0 0x558c4b5661f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:50.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.849+0000 7fb84fc0a740 10 client.0 _ll_register_callbacks cb 0x558c4b1cb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:50.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.849+0000 7fb846ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c4b5575d0 0x558c4b5661f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48150/0 (socket says 172.21.15.164:48150) 2024-09-19T08:53:50.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.849+0000 7fb846ffd640 1 -- 172.21.15.164:0/3718526098 learned_addr learned my addr 172.21.15.164:0/3718526098 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:50.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.849+0000 7fb8477fe640 1 --2- 172.21.15.164:0/3718526098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c4b566780 0x558c4b563be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:50.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.850+0000 7fb8467fc640 1 --2- 172.21.15.164:0/3718526098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c4b558b40 0x558c4b5634a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:50.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.850+0000 7fb8477fe640 1 -- 172.21.15.164:0/3718526098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c4b558b40 msgr2=0x558c4b5634a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:50.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.850+0000 7fb8477fe640 1 --2- 172.21.15.164:0/3718526098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c4b558b40 0x558c4b5634a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.850+0000 7fb8477fe640 1 -- 172.21.15.164:0/3718526098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c4b5575d0 msgr2=0x558c4b5661f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:50.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.850+0000 7fb8477fe640 1 --2- 172.21.15.164:0/3718526098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c4b5575d0 0x558c4b5661f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.850+0000 7fb8477fe640 1 -- 172.21.15.164:0/3718526098 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c4b22e3c0 con 0x558c4b566780 2024-09-19T08:53:50.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.850+0000 7fb846ffd640 1 --2- 172.21.15.164:0/3718526098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c4b5575d0 0x558c4b5661f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:53:50.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.850+0000 7fb8467fc640 1 --2- 172.21.15.164:0/3718526098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c4b558b40 0x558c4b5634a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:53:50.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.850+0000 7fb8477fe640 1 --2- 172.21.15.164:0/3718526098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c4b566780 0x558c4b563be0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb840005e30 tx=0x7fb84002f5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ca4b2706eed81d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:50.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.850+0000 7fb826ffd640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-09-19T08:53:50.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.850+0000 7fb826ffd640 1 -- 172.21.15.164:0/3718526098 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb840048030 con 0x558c4b566780 2024-09-19T08:53:50.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.850+0000 7fb84fc0a740 1 -- 172.21.15.164:0/3718526098 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb84002fa70 con 0x558c4b566780 2024-09-19T08:53:50.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.850+0000 7fb826ffd640 1 -- 172.21.15.164:0/3718526098 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7fb840040050 con 0x558c4b566780 2024-09-19T08:53:50.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.851+0000 7fb826ffd640 1 -- 172.21.15.164:0/3718526098 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb840034040 con 0x558c4b566780 2024-09-19T08:53:50.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.851+0000 7fb84fc0a740 10 client.26293 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:50.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.851+0000 7fb84fc0a740 1 -- 172.21.15.164:0/3718526098 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558c4b569420 con 0x558c4b566780 2024-09-19T08:53:50.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.851+0000 7fb826ffd640 1 -- 172.21.15.164:0/3718526098 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb84002fa70 con 0x558c4b566780 2024-09-19T08:53:50.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.851+0000 7fb826ffd640 1 -- 172.21.15.164:0/3718526098 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb84004e760 con 0x558c4b566780 2024-09-19T08:53:50.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.851+0000 7fb84fc0a740 10 client.26293 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:50.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.851+0000 7fb84fc0a740 10 client.26293 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T08:53:50.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.851+0000 7fb84fc0a740 20 client.26293 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:50.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.851+0000 7fb84fc0a740 1 --2- 172.21.15.164:0/3718526098 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x558c4b576040 0x558c4b578430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:50.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.851+0000 7fb84fc0a740 4 client.26293 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:53:50.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.851+0000 7fb84fc0a740 1 -- 172.21.15.164:0/3718526098 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c4b4ca860 con 0x558c4b576040 2024-09-19T08:53:50.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.851+0000 7fb846ffd640 1 --2- 172.21.15.164:0/3718526098 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x558c4b576040 0x558c4b578430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:50.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.852+0000 7fb826ffd640 10 client.26293 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:53:50.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.852+0000 7fb846ffd640 1 --2- 172.21.15.164:0/3718526098 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x558c4b576040 0x558c4b578430 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=21fd27455e464f1b server_cookie=cd691669a6dbda2f in_seq=0 out_seq=0 2024-09-19T08:53:50.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.852+0000 7fb826ffd640 1 -- 172.21.15.164:0/3718526098 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558c4b4ca860 con 0x558c4b576040 2024-09-19T08:53:50.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.852+0000 7fb826ffd640 10 client.26293 handle_command_reply: tid=0 2024-09-19T08:53:50.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.852+0000 7fb84fc0a740 1 client.26293 shutdown 2024-09-19T08:53:50.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.853+0000 7fb84fc0a740 1 -- 172.21.15.164:0/3718526098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c4b566780 msgr2=0x558c4b563be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:50.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.853+0000 7fb84fc0a740 1 --2- 172.21.15.164:0/3718526098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c4b566780 0x558c4b563be0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb840005e30 tx=0x7fb84002f5d0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.853+0000 7fb84fc0a740 1 -- 172.21.15.164:0/3718526098 shutdown_connections 2024-09-19T08:53:50.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.853+0000 7fb84fc0a740 1 --2- 172.21.15.164:0/3718526098 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x558c4b576040 0x558c4b578430 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.853+0000 7fb84fc0a740 1 --2- 172.21.15.164:0/3718526098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c4b566780 0x558c4b563be0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.853+0000 7fb84fc0a740 1 --2- 172.21.15.164:0/3718526098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c4b558b40 0x558c4b5634a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.853+0000 7fb84fc0a740 1 --2- 172.21.15.164:0/3718526098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c4b5575d0 0x558c4b5661f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.853+0000 7fb84fc0a740 1 -- 172.21.15.164:0/3718526098 >> 172.21.15.164:0/3718526098 conn(0x558c4b520310 msgr2=0x558c4b559960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:50.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.853+0000 7fb826ffd640 0 client.26293 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:53:50.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.853+0000 7fb84fc0a740 1 -- 172.21.15.164:0/3718526098 shutdown_connections 2024-09-19T08:53:50.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.853+0000 7fb84fc0a740 1 -- 172.21.15.164:0/3718526098 wait complete. 2024-09-19T08:53:50.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.853+0000 7fb84fc0a740 20 client.26293 trim_cache size 0 max 16384 2024-09-19T08:53:50.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.869+0000 7fb84fc0a740 1 Processor -- start 2024-09-19T08:53:50.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.869+0000 7fb84fc0a740 1 -- start start 2024-09-19T08:53:50.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.870+0000 7fb84fc0a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c4b5575d0 0x558c4b5726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:50.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.870+0000 7fb84fc0a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c4b558b40 0x558c4b565410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:50.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.870+0000 7fb84fc0a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c4b566780 0x558c4b57dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:50.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.870+0000 7fb84fc0a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb8481ab530 con 0x558c4b5575d0 2024-09-19T08:53:50.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.870+0000 7fb84fc0a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb8481ad070 con 0x558c4b566780 2024-09-19T08:53:50.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.870+0000 7fb84fc0a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb8481abc00 con 0x558c4b558b40 2024-09-19T08:53:50.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.870+0000 7fb846ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c4b5575d0 0x558c4b5726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:50.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.870+0000 7fb846ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c4b5575d0 0x558c4b5726c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48164/0 (socket says 172.21.15.164:48164) 2024-09-19T08:53:50.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.870+0000 7fb846ffd640 1 -- 172.21.15.164:0/3189023252 learned_addr learned my addr 172.21.15.164:0/3189023252 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:50.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.870+0000 7fb8467fc640 1 --2- 172.21.15.164:0/3189023252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c4b558b40 0x558c4b565410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:50.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.870+0000 7fb8477fe640 1 --2- 172.21.15.164:0/3189023252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c4b566780 0x558c4b57dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:50.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.870+0000 7fb846ffd640 1 -- 172.21.15.164:0/3189023252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c4b558b40 msgr2=0x558c4b565410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:50.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.870+0000 7fb846ffd640 1 --2- 172.21.15.164:0/3189023252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c4b558b40 0x558c4b565410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.870+0000 7fb846ffd640 1 -- 172.21.15.164:0/3189023252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c4b566780 msgr2=0x558c4b57dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:50.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.870+0000 7fb846ffd640 1 --2- 172.21.15.164:0/3189023252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c4b566780 0x558c4b57dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.870+0000 7fb846ffd640 1 -- 172.21.15.164:0/3189023252 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c4b572c00 con 0x558c4b5575d0 2024-09-19T08:53:50.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.870+0000 7fb8477fe640 1 --2- 172.21.15.164:0/3189023252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c4b566780 0x558c4b57dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:53:50.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.871+0000 7fb846ffd640 1 --2- 172.21.15.164:0/3189023252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c4b5575d0 0x558c4b5726c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fb83800a630 tx=0x7fb838005300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4d7d35957ca88c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:50.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.871+0000 7fb82d7fa640 1 -- 172.21.15.164:0/3189023252 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83801d040 con 0x558c4b5575d0 2024-09-19T08:53:50.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.871+0000 7fb82d7fa640 1 -- 172.21.15.164:0/3189023252 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8380a0410 con 0x558c4b5575d0 2024-09-19T08:53:50.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.871+0000 7fb82d7fa640 1 -- 172.21.15.164:0/3189023252 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb838007ca0 con 0x558c4b5575d0 2024-09-19T08:53:50.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.872+0000 7fb84fc0a740 1 -- 172.21.15.164:0/3189023252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c4b5575d0 msgr2=0x558c4b5726c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:50.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.872+0000 7fb84fc0a740 1 --2- 172.21.15.164:0/3189023252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c4b5575d0 0x558c4b5726c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fb83800a630 tx=0x7fb838005300 comp rx=0 tx=0).stop 2024-09-19T08:53:50.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.872+0000 7fb84fc0a740 1 -- 172.21.15.164:0/3189023252 shutdown_connections 2024-09-19T08:53:50.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.872+0000 7fb84fc0a740 1 --2- 172.21.15.164:0/3189023252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c4b566780 0x558c4b57dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.872+0000 7fb84fc0a740 1 --2- 172.21.15.164:0/3189023252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c4b558b40 0x558c4b565410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.872+0000 7fb84fc0a740 1 --2- 172.21.15.164:0/3189023252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c4b5575d0 0x558c4b5726c0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.872+0000 7fb84fc0a740 1 -- 172.21.15.164:0/3189023252 >> 172.21.15.164:0/3189023252 conn(0x558c4b520310 msgr2=0x558c4b564d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:50.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.872+0000 7fb84fc0a740 1 -- 172.21.15.164:0/3189023252 shutdown_connections 2024-09-19T08:53:50.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.872+0000 7fb84fc0a740 1 -- 172.21.15.164:0/3189023252 wait complete. 2024-09-19T08:53:50.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.872+0000 7fb84fc0a740 1 -- start start 2024-09-19T08:53:50.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.873+0000 7fb84fc0a740 1 Processor -- start 2024-09-19T08:53:50.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.873+0000 7fb84fc0a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c4b5575d0 0x558c4b56ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:50.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.873+0000 7fb84fc0a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c4b558b40 0x558c4b56a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:50.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.873+0000 7fb84fc0a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c4b566780 0x558c4b56a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:50.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.873+0000 7fb84fc0a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb8481ac9a0 con 0x558c4b558b40 2024-09-19T08:53:50.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.873+0000 7fb84fc0a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb8481ac2d0 con 0x558c4b566780 2024-09-19T08:53:50.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.873+0000 7fb84fc0a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb8481a9530 con 0x558c4b5575d0 2024-09-19T08:53:50.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.873+0000 7fb84fc0a740 10 client.0 _ll_register_callbacks cb 0x7fb840046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:50.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.873+0000 7fb8467fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c4b558b40 0x558c4b56a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:50.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.873+0000 7fb8467fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c4b558b40 0x558c4b56a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48176/0 (socket says 172.21.15.164:48176) 2024-09-19T08:53:50.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.873+0000 7fb8467fc640 1 -- 172.21.15.164:0/1080729957 learned_addr learned my addr 172.21.15.164:0/1080729957 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:50.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.873+0000 7fb8477fe640 1 --2- 172.21.15.164:0/1080729957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c4b566780 0x558c4b56a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:50.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.873+0000 7fb846ffd640 1 --2- 172.21.15.164:0/1080729957 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c4b5575d0 0x558c4b56ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:50.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.873+0000 7fb8467fc640 1 -- 172.21.15.164:0/1080729957 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c4b5575d0 msgr2=0x558c4b56ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:50.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.873+0000 7fb8467fc640 1 --2- 172.21.15.164:0/1080729957 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c4b5575d0 0x558c4b56ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.873+0000 7fb8467fc640 1 -- 172.21.15.164:0/1080729957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c4b566780 msgr2=0x558c4b56a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:50.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.873+0000 7fb8467fc640 1 --2- 172.21.15.164:0/1080729957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c4b566780 0x558c4b56a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.873+0000 7fb8467fc640 1 -- 172.21.15.164:0/1080729957 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c4b56ac00 con 0x558c4b558b40 2024-09-19T08:53:50.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.873+0000 7fb8477fe640 1 --2- 172.21.15.164:0/1080729957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c4b566780 0x558c4b56a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:53:50.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.874+0000 7fb8467fc640 1 --2- 172.21.15.164:0/1080729957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c4b558b40 0x558c4b56a100 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fb8481052d0 tx=0x7fb83c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d853c719e71d49e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:50.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.874+0000 7fb826ffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:53:50.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.874+0000 7fb826ffd640 1 -- 172.21.15.164:0/1080729957 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c004300 con 0x558c4b558b40 2024-09-19T08:53:50.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.874+0000 7fb826ffd640 1 -- 172.21.15.164:0/1080729957 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7fb83c0035f0 con 0x558c4b558b40 2024-09-19T08:53:50.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.874+0000 7fb84fc0a740 1 -- 172.21.15.164:0/1080729957 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8380a0410 con 0x558c4b558b40 2024-09-19T08:53:50.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.874+0000 7fb826ffd640 1 -- 172.21.15.164:0/1080729957 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb83c0044a0 con 0x558c4b558b40 2024-09-19T08:53:50.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.874+0000 7fb84fc0a740 10 client.17133 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:50.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.874+0000 7fb84fc0a740 1 -- 172.21.15.164:0/1080729957 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558c4b5e08f0 con 0x558c4b558b40 2024-09-19T08:53:50.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.874+0000 7fb826ffd640 1 -- 172.21.15.164:0/1080729957 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb83c003cd0 con 0x558c4b558b40 2024-09-19T08:53:50.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.874+0000 7fb826ffd640 1 -- 172.21.15.164:0/1080729957 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb83c032cc0 con 0x558c4b558b40 2024-09-19T08:53:50.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.875+0000 7fb84fc0a740 10 client.17133 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:50.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.875+0000 7fb84fc0a740 10 client.17133 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T08:53:50.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.875+0000 7fb84fc0a740 20 client.17133 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:50.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.875+0000 7fb84fc0a740 1 --2- 172.21.15.164:0/1080729957 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x558c4b5e1080 0x558c4b5e34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:50.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.875+0000 7fb84fc0a740 4 client.17133 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:53:50.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.875+0000 7fb84fc0a740 1 -- 172.21.15.164:0/1080729957 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fb8380a0410 con 0x558c4b5e1080 2024-09-19T08:53:50.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.875+0000 7fb846ffd640 1 --2- 172.21.15.164:0/1080729957 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x558c4b5e1080 0x558c4b5e34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:50.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.875+0000 7fb826ffd640 10 client.17133 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:53:50.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.875+0000 7fb846ffd640 1 --2- 172.21.15.164:0/1080729957 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x558c4b5e1080 0x558c4b5e34a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=41d1f7bf7879a452 server_cookie=cd59541791ddb86b in_seq=0 out_seq=0 2024-09-19T08:53:50.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.876+0000 7fb826ffd640 1 -- 172.21.15.164:0/1080729957 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fb8380a0410 con 0x558c4b5e1080 2024-09-19T08:53:50.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.876+0000 7fb826ffd640 10 client.17133 handle_command_reply: tid=0 2024-09-19T08:53:50.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.876+0000 7fb84fc0a740 1 client.17133 shutdown 2024-09-19T08:53:50.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.876+0000 7fb84fc0a740 1 -- 172.21.15.164:0/1080729957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c4b558b40 msgr2=0x558c4b56a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:50.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.876+0000 7fb84fc0a740 1 --2- 172.21.15.164:0/1080729957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c4b558b40 0x558c4b56a100 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fb8481052d0 tx=0x7fb83c0046c0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.876+0000 7fb84fc0a740 1 -- 172.21.15.164:0/1080729957 shutdown_connections 2024-09-19T08:53:50.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.876+0000 7fb84fc0a740 1 --2- 172.21.15.164:0/1080729957 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x558c4b5e1080 0x558c4b5e34a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.876+0000 7fb826ffd640 0 client.17133 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:53:50.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.876+0000 7fb84fc0a740 1 --2- 172.21.15.164:0/1080729957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c4b566780 0x558c4b56a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.876+0000 7fb84fc0a740 1 --2- 172.21.15.164:0/1080729957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c4b558b40 0x558c4b56a100 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.876+0000 7fb84fc0a740 1 --2- 172.21.15.164:0/1080729957 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c4b5575d0 0x558c4b56ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.876+0000 7fb84fc0a740 1 -- 172.21.15.164:0/1080729957 >> 172.21.15.164:0/1080729957 conn(0x558c4b520310 msgr2=0x558c4b565520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:50.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.876+0000 7fb84fc0a740 1 -- 172.21.15.164:0/1080729957 shutdown_connections 2024-09-19T08:53:50.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.876+0000 7fb84fc0a740 1 -- 172.21.15.164:0/1080729957 wait complete. 2024-09-19T08:53:50.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.876+0000 7fb84fc0a740 20 client.17133 trim_cache size 0 max 16384 2024-09-19T08:53:50.886 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:53:50.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.877+0000 7fb84d831640 1 -- 172.21.15.164:0/4266380785 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb8300a33b0 msgr2=0x7fb8300a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:50.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.877+0000 7fb84d831640 1 --2- 172.21.15.164:0/4266380785 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb8300a33b0 0x7fb8300a5870 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb83c005fd0 tx=0x7fb83c005850 comp rx=0 tx=0).stop 2024-09-19T08:53:50.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.877+0000 7fb84d831640 1 -- 172.21.15.164:0/4266380785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb848100e20 msgr2=0x7fb8481984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:50.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.877+0000 7fb84d831640 1 --2- 172.21.15.164:0/4266380785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb848100e20 0x7fb8481984d0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fb83800c890 tx=0x7fb83800cd50 comp rx=0 tx=0).stop 2024-09-19T08:53:50.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.877+0000 7fb84d831640 1 -- 172.21.15.164:0/4266380785 shutdown_connections 2024-09-19T08:53:50.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.877+0000 7fb84d831640 1 --2- 172.21.15.164:0/4266380785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb848102ef0 0x7fb84819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.877+0000 7fb84d831640 1 --2- 172.21.15.164:0/4266380785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb848102570 0x7fb848198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.877+0000 7fb84d831640 1 --2- 172.21.15.164:0/4266380785 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb8300a33b0 0x7fb8300a5870 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.877+0000 7fb84d831640 1 --2- 172.21.15.164:0/4266380785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb848100e20 0x7fb8481984d0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:50.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.877+0000 7fb84d831640 1 -- 172.21.15.164:0/4266380785 >> 172.21.15.164:0/4266380785 conn(0x7fb8480fc7d0 msgr2=0x7fb84806a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:50.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.877+0000 7fb84d831640 1 -- 172.21.15.164:0/4266380785 shutdown_connections 2024-09-19T08:53:50.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:50.878+0000 7fb84d831640 1 -- 172.21.15.164:0/4266380785 wait complete. 2024-09-19T08:53:51.126 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:53:51.666 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T08:53:52.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.122+0000 7f1657d8b640 1 -- 172.21.15.164:0/1211393457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f165010c180 msgr2=0x7f165010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:52.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.122+0000 7f1657d8b640 1 --2- 172.21.15.164:0/1211393457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f165010c180 0x7f165010e610 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f1648009ca0 tx=0x7f164802a950 comp rx=0 tx=0).stop 2024-09-19T08:53:52.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.122+0000 7f1657d8b640 1 -- 172.21.15.164:0/1211393457 shutdown_connections 2024-09-19T08:53:52.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.122+0000 7f1657d8b640 1 --2- 172.21.15.164:0/1211393457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f165010c180 0x7f165010e610 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:52.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.122+0000 7f1657d8b640 1 --2- 172.21.15.164:0/1211393457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1650103700 0x7f165010bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:52.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.122+0000 7f1657d8b640 1 --2- 172.21.15.164:0/1211393457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1650102de0 0x7f16501031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:52.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.122+0000 7f1657d8b640 1 -- 172.21.15.164:0/1211393457 >> 172.21.15.164:0/1211393457 conn(0x7f16500fc780 msgr2=0x7f16500feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:52.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.122+0000 7f1657d8b640 1 -- 172.21.15.164:0/1211393457 shutdown_connections 2024-09-19T08:53:52.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.122+0000 7f1657d8b640 1 -- 172.21.15.164:0/1211393457 wait complete. 2024-09-19T08:53:52.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.123+0000 7f1657d8b640 1 Processor -- start 2024-09-19T08:53:52.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.123+0000 7f1657d8b640 1 -- start start 2024-09-19T08:53:52.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.123+0000 7f1657d8b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1650102de0 0x7f16501983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:52.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.123+0000 7f1657d8b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1650103700 0x7f16501988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:52.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.123+0000 7f1657d8b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f165010c180 0x7f165019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:52.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.123+0000 7f1657d8b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1650111f30 con 0x7f1650103700 2024-09-19T08:53:52.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.123+0000 7f1657d8b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1650111db0 con 0x7f165010c180 2024-09-19T08:53:52.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.123+0000 7f1657d8b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f16501120b0 con 0x7f1650102de0 2024-09-19T08:53:52.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.123+0000 7f1655326640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1650103700 0x7f16501988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:52.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.123+0000 7f1655326640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1650103700 0x7f16501988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48196/0 (socket says 172.21.15.164:48196) 2024-09-19T08:53:52.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.123+0000 7f1655326640 1 -- 172.21.15.164:0/1868803599 learned_addr learned my addr 172.21.15.164:0/1868803599 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:52.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.123+0000 7f1655b27640 1 --2- 172.21.15.164:0/1868803599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1650102de0 0x7f16501983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:52.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.123+0000 7f1656328640 1 --2- 172.21.15.164:0/1868803599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f165010c180 0x7f165019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:52.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.123+0000 7f1655326640 1 -- 172.21.15.164:0/1868803599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1650102de0 msgr2=0x7f16501983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:52.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.123+0000 7f1655326640 1 --2- 172.21.15.164:0/1868803599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1650102de0 0x7f16501983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:52.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.123+0000 7f1655326640 1 -- 172.21.15.164:0/1868803599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f165010c180 msgr2=0x7f165019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:52.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.123+0000 7f1655326640 1 --2- 172.21.15.164:0/1868803599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f165010c180 0x7f165019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:52.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.123+0000 7f1655326640 1 -- 172.21.15.164:0/1868803599 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f165019d360 con 0x7f1650103700 2024-09-19T08:53:52.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.124+0000 7f1655326640 1 --2- 172.21.15.164:0/1868803599 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1650103700 0x7f16501988f0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f164c00c890 tx=0x7f164c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:52.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.124+0000 7f1646ffd640 1 -- 172.21.15.164:0/1868803599 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164c004220 con 0x7f1650103700 2024-09-19T08:53:52.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.124+0000 7f1657d8b640 1 -- 172.21.15.164:0/1868803599 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f165019d650 con 0x7f1650103700 2024-09-19T08:53:52.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.124+0000 7f1646ffd640 1 -- 172.21.15.164:0/1868803599 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f164c0043c0 con 0x7f1650103700 2024-09-19T08:53:52.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.124+0000 7f1646ffd640 1 -- 172.21.15.164:0/1868803599 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164c004a00 con 0x7f1650103700 2024-09-19T08:53:52.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.124+0000 7f1657d8b640 1 -- 172.21.15.164:0/1868803599 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16501a4f30 con 0x7f1650103700 2024-09-19T08:53:52.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.125+0000 7f1657d8b640 1 -- 172.21.15.164:0/1868803599 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16501031c0 con 0x7f1650103700 2024-09-19T08:53:52.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.126+0000 7f1646ffd640 1 -- 172.21.15.164:0/1868803599 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f164c0079b0 con 0x7f1650103700 2024-09-19T08:53:52.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.126+0000 7f1646ffd640 1 --2- 172.21.15.164:0/1868803599 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f163c0a33b0 0x7f163c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:52.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.127+0000 7f1655b27640 1 --2- 172.21.15.164:0/1868803599 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f163c0a33b0 0x7f163c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:52.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.127+0000 7f1646ffd640 1 -- 172.21.15.164:0/1868803599 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f164c0cf940 con 0x7f1650103700 2024-09-19T08:53:52.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.127+0000 7f1655b27640 1 --2- 172.21.15.164:0/1868803599 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f163c0a33b0 0x7f163c0a5870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f164000abe0 tx=0x7f164000a2d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:52.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.131+0000 7f1646ffd640 1 -- 172.21.15.164:0/1868803599 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f164c014030 con 0x7f1650103700 2024-09-19T08:53:52.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:51 smithi173 ceph-mon[35306]: pgmap v248: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:53:52.178 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:53:52.281 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:51 smithi164 ceph-mon[30928]: pgmap v248: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:53:52.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.351+0000 7f1657d8b640 1 -- 172.21.15.164:0/1868803599 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1650107260 con 0x7f1650103700 2024-09-19T08:53:52.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.352+0000 7f1646ffd640 1 -- 172.21.15.164:0/1868803599 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11984 (secure 0 0 0) 0x7f164c09c360 con 0x7f1650103700 2024-09-19T08:53:52.353 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:53:52.353 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":32,"btime":"2024-09-19T08:52:27:192804+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:52:27.074168+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:53:52.353 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 32 2024-09-19T08:53:52.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.354+0000 7f1657d8b640 1 -- 172.21.15.164:0/1868803599 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f163c0a33b0 msgr2=0x7f163c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:52.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.354+0000 7f1657d8b640 1 --2- 172.21.15.164:0/1868803599 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f163c0a33b0 0x7f163c0a5870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f164000abe0 tx=0x7f164000a2d0 comp rx=0 tx=0).stop 2024-09-19T08:53:52.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.354+0000 7f1657d8b640 1 -- 172.21.15.164:0/1868803599 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1650103700 msgr2=0x7f16501988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:52.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.354+0000 7f1657d8b640 1 --2- 172.21.15.164:0/1868803599 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1650103700 0x7f16501988f0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f164c00c890 tx=0x7f164c00cc90 comp rx=0 tx=0).stop 2024-09-19T08:53:52.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.354+0000 7f1657d8b640 1 -- 172.21.15.164:0/1868803599 shutdown_connections 2024-09-19T08:53:52.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.354+0000 7f1657d8b640 1 --2- 172.21.15.164:0/1868803599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f165010c180 0x7f165019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:52.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.354+0000 7f1657d8b640 1 --2- 172.21.15.164:0/1868803599 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f163c0a33b0 0x7f163c0a5870 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:52.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.354+0000 7f1657d8b640 1 --2- 172.21.15.164:0/1868803599 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1650103700 0x7f16501988f0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:52.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.354+0000 7f1657d8b640 1 --2- 172.21.15.164:0/1868803599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1650102de0 0x7f16501983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:52.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.354+0000 7f1657d8b640 1 -- 172.21.15.164:0/1868803599 >> 172.21.15.164:0/1868803599 conn(0x7f16500fc780 msgr2=0x7f16500fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:52.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.354+0000 7f1657d8b640 1 -- 172.21.15.164:0/1868803599 shutdown_connections 2024-09-19T08:53:52.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:52.354+0000 7f1657d8b640 1 -- 172.21.15.164:0/1868803599 wait complete. 2024-09-19T08:53:52.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:51 smithi170 ceph-mon[35072]: pgmap v248: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:53:53.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:52 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/1868803599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:53:53.077 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 145.75 --await-for 45.0' 2024-09-19T08:53:53.077 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 145.75 --await-for 45.0 2024-09-19T08:53:53.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.277+0000 7fb0f9286640 1 -- 172.21.15.164:0/4094705842 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0f41036e0 msgr2=0x7fb0f4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:53.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.277+0000 7fb0f9286640 1 --2- 172.21.15.164:0/4094705842 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0f41036e0 0x7fb0f4107b90 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8009930 tx=0x7fb0e802a420 comp rx=0 tx=0).stop 2024-09-19T08:53:53.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.277+0000 7fb0f9286640 1 -- 172.21.15.164:0/4094705842 shutdown_connections 2024-09-19T08:53:53.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.277+0000 7fb0f9286640 1 --2- 172.21.15.164:0/4094705842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0f41082a0 0x7fb0f410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.277+0000 7fb0f9286640 1 --2- 172.21.15.164:0/4094705842 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0f41036e0 0x7fb0f4107b90 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.277+0000 7fb0f9286640 1 --2- 172.21.15.164:0/4094705842 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0f4102d30 0x7fb0f4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.277+0000 7fb0f9286640 1 -- 172.21.15.164:0/4094705842 >> 172.21.15.164:0/4094705842 conn(0x7fb0f40fc7d0 msgr2=0x7fb0f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:53.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.277+0000 7fb0f9286640 1 -- 172.21.15.164:0/4094705842 shutdown_connections 2024-09-19T08:53:53.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.277+0000 7fb0f9286640 1 -- 172.21.15.164:0/4094705842 wait complete. 2024-09-19T08:53:53.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.278+0000 7fb0f9286640 1 Processor -- start 2024-09-19T08:53:53.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.278+0000 7fb0f9286640 1 -- start start 2024-09-19T08:53:53.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.278+0000 7fb0f9286640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0f4102d30 0x7fb0f4198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:53.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.278+0000 7fb0f9286640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0f41036e0 0x7fb0f4198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:53.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.278+0000 7fb0f9286640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0f41082a0 0x7fb0f419ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:53.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.278+0000 7fb0f9286640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f41120c0 con 0x7fb0f41036e0 2024-09-19T08:53:53.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.278+0000 7fb0f9286640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f4111f40 con 0x7fb0f41082a0 2024-09-19T08:53:53.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.278+0000 7fb0f9286640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f4112240 con 0x7fb0f4102d30 2024-09-19T08:53:53.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.278+0000 7fb0f27fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0f41036e0 0x7fb0f4198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:53.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.278+0000 7fb0f27fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0f41036e0 0x7fb0f4198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48214/0 (socket says 172.21.15.164:48214) 2024-09-19T08:53:53.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.278+0000 7fb0f27fc640 1 -- 172.21.15.164:0/1608588871 learned_addr learned my addr 172.21.15.164:0/1608588871 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:53.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.278+0000 7fb0f37fe640 1 --2- 172.21.15.164:0/1608588871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0f41082a0 0x7fb0f419ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:53.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.279+0000 7fb0f2ffd640 1 --2- 172.21.15.164:0/1608588871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0f4102d30 0x7fb0f4198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:53.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.279+0000 7fb0f27fc640 1 -- 172.21.15.164:0/1608588871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0f4102d30 msgr2=0x7fb0f4198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:53.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.279+0000 7fb0f27fc640 1 --2- 172.21.15.164:0/1608588871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0f4102d30 0x7fb0f4198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.279+0000 7fb0f27fc640 1 -- 172.21.15.164:0/1608588871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0f41082a0 msgr2=0x7fb0f419ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:53.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.279+0000 7fb0f27fc640 1 --2- 172.21.15.164:0/1608588871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0f41082a0 0x7fb0f419ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.279+0000 7fb0f27fc640 1 -- 172.21.15.164:0/1608588871 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f419d5b0 con 0x7fb0f41036e0 2024-09-19T08:53:53.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.279+0000 7fb0f37fe640 1 --2- 172.21.15.164:0/1608588871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0f41082a0 0x7fb0f419ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:53:53.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.279+0000 7fb0f2ffd640 1 --2- 172.21.15.164:0/1608588871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0f4102d30 0x7fb0f4198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:53:53.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.279+0000 7fb0f27fc640 1 --2- 172.21.15.164:0/1608588871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0f41036e0 0x7fb0f4198b40 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8005940 tx=0x7fb0e80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:53.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.279+0000 7fb0dbfff640 1 -- 172.21.15.164:0/1608588871 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e803f030 con 0x7fb0f41036e0 2024-09-19T08:53:53.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.279+0000 7fb0f9286640 1 -- 172.21.15.164:0/1608588871 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0f419d840 con 0x7fb0f41036e0 2024-09-19T08:53:53.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.279+0000 7fb0dbfff640 1 -- 172.21.15.164:0/1608588871 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0e8003ce0 con 0x7fb0f41036e0 2024-09-19T08:53:53.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.279+0000 7fb0dbfff640 1 -- 172.21.15.164:0/1608588871 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8035070 con 0x7fb0f41036e0 2024-09-19T08:53:53.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.279+0000 7fb0f9286640 1 -- 172.21.15.164:0/1608588871 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0f41a5180 con 0x7fb0f41036e0 2024-09-19T08:53:53.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.282+0000 7fb0dbfff640 1 -- 172.21.15.164:0/1608588871 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fb0e8042030 con 0x7fb0f41036e0 2024-09-19T08:53:53.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.282+0000 7fb0fb65f740 1 Processor -- start 2024-09-19T08:53:53.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.282+0000 7fb0fb65f740 1 -- start start 2024-09-19T08:53:53.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.282+0000 7fb0fb65f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564dd2278b40 0x564dd2277090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:53.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.282+0000 7fb0fb65f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564dd22775d0 0x564dd2277a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:53.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.282+0000 7fb0fb65f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd2282370 0x564dd2284760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:53.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.282+0000 7fb0fb65f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564dd21dd2f0 con 0x564dd2282370 2024-09-19T08:53:53.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.282+0000 7fb0fb65f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564dd2184cf0 con 0x564dd2278b40 2024-09-19T08:53:53.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.282+0000 7fb0fb65f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564dd21d92b0 con 0x564dd22775d0 2024-09-19T08:53:53.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.282+0000 7fb0f27fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd2282370 0x564dd2284760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:53.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.283+0000 7fb0f27fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd2282370 0x564dd2284760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48226/0 (socket says 172.21.15.164:48226) 2024-09-19T08:53:53.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.283+0000 7fb0f37fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564dd2278b40 0x564dd2277090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:53.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.283+0000 7fb0f27fc640 1 -- 172.21.15.164:0/3086159628 learned_addr learned my addr 172.21.15.164:0/3086159628 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:53.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.283+0000 7fb0f2ffd640 1 --2- 172.21.15.164:0/3086159628 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564dd22775d0 0x564dd2277a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:53.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.283+0000 7fb0dbfff640 1 --2- 172.21.15.164:0/1608588871 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb0dc0a33b0 0x7fb0dc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:53.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.283+0000 7fb0f27fc640 1 -- 172.21.15.164:0/3086159628 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564dd22775d0 msgr2=0x564dd2277a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:53.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.283+0000 7fb0f27fc640 1 --2- 172.21.15.164:0/3086159628 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564dd22775d0 0x564dd2277a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.283+0000 7fb0f27fc640 1 -- 172.21.15.164:0/3086159628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564dd2278b40 msgr2=0x564dd2277090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:53.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.283+0000 7fb0f27fc640 1 --2- 172.21.15.164:0/3086159628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564dd2278b40 0x564dd2277090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.283+0000 7fb0f27fc640 1 -- 172.21.15.164:0/3086159628 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564dd21693a0 con 0x564dd2282370 2024-09-19T08:53:53.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.283+0000 7fb0f37fe640 1 --2- 172.21.15.164:0/1608588871 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb0dc0a33b0 0x7fb0dc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:53.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.283+0000 7fb0dbfff640 1 -- 172.21.15.164:0/1608588871 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7fb0e80f0210 con 0x7fb0f41036e0 2024-09-19T08:53:53.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.283+0000 7fb0f27fc640 1 --2- 172.21.15.164:0/3086159628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd2282370 0x564dd2284760 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80ed940 tx=0x7fb0e80b19c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e874e5d90372224 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:53.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.283+0000 7fb0d97fa640 1 -- 172.21.15.164:0/3086159628 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e80dc070 con 0x564dd2282370 2024-09-19T08:53:53.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.283+0000 7fb0d97fa640 1 -- 172.21.15.164:0/3086159628 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0e8087380 con 0x564dd2282370 2024-09-19T08:53:53.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.283+0000 7fb0d97fa640 1 -- 172.21.15.164:0/3086159628 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8085430 con 0x564dd2282370 2024-09-19T08:53:53.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.284+0000 7fb0fb65f740 1 -- 172.21.15.164:0/3086159628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd2282370 msgr2=0x564dd2284760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:53.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.284+0000 7fb0fb65f740 1 --2- 172.21.15.164:0/3086159628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd2282370 0x564dd2284760 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80ed940 tx=0x7fb0e80b19c0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.284+0000 7fb0f37fe640 1 --2- 172.21.15.164:0/1608588871 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb0dc0a33b0 0x7fb0dc0a5870 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec005fd0 tx=0x7fb0ec004a40 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:53.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.284+0000 7fb0fb65f740 1 -- 172.21.15.164:0/3086159628 shutdown_connections 2024-09-19T08:53:53.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.284+0000 7fb0fb65f740 1 --2- 172.21.15.164:0/3086159628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd2282370 0x564dd2284760 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.284+0000 7fb0fb65f740 1 --2- 172.21.15.164:0/3086159628 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564dd22775d0 0x564dd2277a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.284+0000 7fb0fb65f740 1 --2- 172.21.15.164:0/3086159628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564dd2278b40 0x564dd2277090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.284+0000 7fb0fb65f740 1 -- 172.21.15.164:0/3086159628 >> 172.21.15.164:0/3086159628 conn(0x564dd2240310 msgr2=0x564dd2273c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:53.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.285+0000 7fb0fb65f740 1 -- 172.21.15.164:0/3086159628 shutdown_connections 2024-09-19T08:53:53.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.285+0000 7fb0fb65f740 1 -- 172.21.15.164:0/3086159628 wait complete. 2024-09-19T08:53:53.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.285+0000 7fb0fb65f740 1 -- start start 2024-09-19T08:53:53.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.285+0000 7fb0fb65f740 1 Processor -- start 2024-09-19T08:53:53.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.285+0000 7fb0fb65f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd22775d0 0x564dd2281ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:53.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.286+0000 7fb0fb65f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564dd2278b40 0x564dd227ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:53.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.286+0000 7fb0f2ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd22775d0 0x564dd2281ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:53.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.286+0000 7fb0f2ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd22775d0 0x564dd2281ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48236/0 (socket says 172.21.15.164:48236) 2024-09-19T08:53:53.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.286+0000 7fb0f2ffd640 1 -- 172.21.15.164:0/3706909084 learned_addr learned my addr 172.21.15.164:0/3706909084 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:53.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.286+0000 7fb0fb65f740 1 --2- 172.21.15.164:0/3706909084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564dd2282370 0x564dd227f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:53.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.286+0000 7fb0fb65f740 1 -- 172.21.15.164:0/3706909084 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564dd2284e30 con 0x564dd22775d0 2024-09-19T08:53:53.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.286+0000 7fb0fb65f740 1 -- 172.21.15.164:0/3706909084 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564dd2198060 con 0x564dd2282370 2024-09-19T08:53:53.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.286+0000 7fb0fb65f740 1 -- 172.21.15.164:0/3706909084 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564dd2284fb0 con 0x564dd2278b40 2024-09-19T08:53:53.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.286+0000 7fb0fb65f740 10 client.0 _ll_register_callbacks cb 0x564dd1eeb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:53.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.286+0000 7fb0f27fc640 1 --2- 172.21.15.164:0/3706909084 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564dd2278b40 0x564dd227ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:53.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.286+0000 7fb0f2ffd640 1 -- 172.21.15.164:0/3706909084 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564dd2278b40 msgr2=0x564dd227ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:53.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.286+0000 7fb0f2ffd640 1 --2- 172.21.15.164:0/3706909084 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564dd2278b40 0x564dd227ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.286+0000 7fb0f37fe640 1 --2- 172.21.15.164:0/3706909084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564dd2282370 0x564dd227f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:53.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.286+0000 7fb0f2ffd640 1 -- 172.21.15.164:0/3706909084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564dd2282370 msgr2=0x564dd227f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:53.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.286+0000 7fb0f2ffd640 1 --2- 172.21.15.164:0/3706909084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564dd2282370 0x564dd227f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.286+0000 7fb0f2ffd640 1 -- 172.21.15.164:0/3706909084 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564dd1f4e3c0 con 0x564dd22775d0 2024-09-19T08:53:53.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.286+0000 7fb0f27fc640 1 --2- 172.21.15.164:0/3706909084 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564dd2278b40 0x564dd227ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:53:53.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.286+0000 7fb0f37fe640 1 --2- 172.21.15.164:0/3706909084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564dd2282370 0x564dd227f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:53:53.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.286+0000 7fb0f2ffd640 1 --2- 172.21.15.164:0/3706909084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd22775d0 0x564dd2281ce0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400f4f0 tx=0x7fb0e400f9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea77e5e721740b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:53.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.286+0000 7fb0d6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:53:53.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.286+0000 7fb0d6ff5640 1 -- 172.21.15.164:0/3706909084 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4013850 con 0x564dd22775d0 2024-09-19T08:53:53.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.287+0000 7fb0fb65f740 1 -- 172.21.15.164:0/3706909084 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0e8087380 con 0x564dd22775d0 2024-09-19T08:53:53.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.287+0000 7fb0d6ff5640 1 -- 172.21.15.164:0/3706909084 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7fb0e40139f0 con 0x564dd22775d0 2024-09-19T08:53:53.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.287+0000 7fb0d6ff5640 1 -- 172.21.15.164:0/3706909084 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb0e4013bf0 con 0x564dd22775d0 2024-09-19T08:53:53.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.287+0000 7fb0fb65f740 10 client.17151 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:53.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.287+0000 7fb0fb65f740 1 -- 172.21.15.164:0/3706909084 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564dd2298fd0 con 0x564dd22775d0 2024-09-19T08:53:53.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.287+0000 7fb0d6ff5640 1 -- 172.21.15.164:0/3706909084 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0e8087380 con 0x564dd22775d0 2024-09-19T08:53:53.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.287+0000 7fb0d6ff5640 1 -- 172.21.15.164:0/3706909084 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb0e4011440 con 0x564dd22775d0 2024-09-19T08:53:53.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.287+0000 7fb0fb65f740 10 client.17151 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:53.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.287+0000 7fb0fb65f740 10 client.17151 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T08:53:53.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.287+0000 7fb0fb65f740 20 client.17151 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:53.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.288+0000 7fb0fb65f740 1 --2- 172.21.15.164:0/3706909084 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x564dd229a630 0x564dd229cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:53.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.288+0000 7fb0fb65f740 4 client.17151 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:53:53.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.288+0000 7fb0fb65f740 1 -- 172.21.15.164:0/3706909084 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564dd21ea860 con 0x564dd229a630 2024-09-19T08:53:53.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.288+0000 7fb0f27fc640 1 --2- 172.21.15.164:0/3706909084 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x564dd229a630 0x564dd229cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:53.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.288+0000 7fb0f27fc640 1 --2- 172.21.15.164:0/3706909084 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x564dd229a630 0x564dd229cac0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=55f7b58a9b81b850 server_cookie=cb3e1f1789e16442 in_seq=0 out_seq=0 2024-09-19T08:53:53.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.288+0000 7fb0d6ff5640 10 client.17151 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:53:53.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.289+0000 7fb0d6ff5640 1 -- 172.21.15.164:0/3706909084 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564dd21ea860 con 0x564dd229a630 2024-09-19T08:53:53.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.289+0000 7fb0d6ff5640 10 client.17151 handle_command_reply: tid=0 2024-09-19T08:53:53.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.289+0000 7fb0fb65f740 1 client.17151 shutdown 2024-09-19T08:53:53.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.289+0000 7fb0fb65f740 1 -- 172.21.15.164:0/3706909084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd22775d0 msgr2=0x564dd2281ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:53.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.289+0000 7fb0fb65f740 1 --2- 172.21.15.164:0/3706909084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd22775d0 0x564dd2281ce0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400f4f0 tx=0x7fb0e400f9b0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.289+0000 7fb0fb65f740 1 -- 172.21.15.164:0/3706909084 shutdown_connections 2024-09-19T08:53:53.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.289+0000 7fb0fb65f740 1 --2- 172.21.15.164:0/3706909084 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x564dd229a630 0x564dd229cac0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.289+0000 7fb0fb65f740 1 --2- 172.21.15.164:0/3706909084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564dd2282370 0x564dd227f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.289+0000 7fb0fb65f740 1 --2- 172.21.15.164:0/3706909084 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564dd2278b40 0x564dd227ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.289+0000 7fb0fb65f740 1 --2- 172.21.15.164:0/3706909084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd22775d0 0x564dd2281ce0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.289+0000 7fb0fb65f740 1 -- 172.21.15.164:0/3706909084 >> 172.21.15.164:0/3706909084 conn(0x564dd2240310 msgr2=0x564dd2279960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:53.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.289+0000 7fb0d6ff5640 0 client.17151 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:53:53.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.289+0000 7fb0fb65f740 1 -- 172.21.15.164:0/3706909084 shutdown_connections 2024-09-19T08:53:53.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.289+0000 7fb0fb65f740 1 -- 172.21.15.164:0/3706909084 wait complete. 2024-09-19T08:53:53.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.290+0000 7fb0fb65f740 20 client.17151 trim_cache size 0 max 16384 2024-09-19T08:53:53.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.303+0000 7fb0fb65f740 1 Processor -- start 2024-09-19T08:53:53.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.303+0000 7fb0fb65f740 1 -- start start 2024-09-19T08:53:53.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.303+0000 7fb0fb65f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd22775d0 0x564dd227ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:53.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.303+0000 7fb0fb65f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564dd2278b40 0x564dd2280f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:53.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.303+0000 7fb0fb65f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564dd2282370 0x564dd227a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:53.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.303+0000 7fb0fb65f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f41ab600 con 0x564dd22775d0 2024-09-19T08:53:53.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.303+0000 7fb0fb65f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f41ad140 con 0x564dd2282370 2024-09-19T08:53:53.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.303+0000 7fb0fb65f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f41abcd0 con 0x564dd2278b40 2024-09-19T08:53:53.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.303+0000 7fb0f2ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd22775d0 0x564dd227ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:53.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.303+0000 7fb0f2ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd22775d0 0x564dd227ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48248/0 (socket says 172.21.15.164:48248) 2024-09-19T08:53:53.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.303+0000 7fb0f2ffd640 1 -- 172.21.15.164:0/3738971327 learned_addr learned my addr 172.21.15.164:0/3738971327 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:53.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.303+0000 7fb0f37fe640 1 --2- 172.21.15.164:0/3738971327 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564dd2282370 0x564dd227a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:53.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.303+0000 7fb0f27fc640 1 --2- 172.21.15.164:0/3738971327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564dd2278b40 0x564dd2280f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:53.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.304+0000 7fb0f2ffd640 1 -- 172.21.15.164:0/3738971327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564dd2278b40 msgr2=0x564dd2280f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:53.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.304+0000 7fb0f2ffd640 1 --2- 172.21.15.164:0/3738971327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564dd2278b40 0x564dd2280f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.304+0000 7fb0f2ffd640 1 -- 172.21.15.164:0/3738971327 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564dd2282370 msgr2=0x564dd227a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:53.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.304+0000 7fb0f2ffd640 1 --2- 172.21.15.164:0/3738971327 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564dd2282370 0x564dd227a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.304+0000 7fb0f2ffd640 1 -- 172.21.15.164:0/3738971327 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564dd227d4e0 con 0x564dd22775d0 2024-09-19T08:53:53.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.304+0000 7fb0f2ffd640 1 --2- 172.21.15.164:0/3738971327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd22775d0 0x564dd227ded0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fb0e40118e0 tx=0x7fb0e4006c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20277dc98103534 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:53.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.304+0000 7fb0d8ff9640 1 -- 172.21.15.164:0/3738971327 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4028070 con 0x564dd22775d0 2024-09-19T08:53:53.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.304+0000 7fb0d8ff9640 1 -- 172.21.15.164:0/3738971327 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0e402a050 con 0x564dd22775d0 2024-09-19T08:53:53.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.304+0000 7fb0d8ff9640 1 -- 172.21.15.164:0/3738971327 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e40059c0 con 0x564dd22775d0 2024-09-19T08:53:53.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.305+0000 7fb0fb65f740 1 -- 172.21.15.164:0/3738971327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd22775d0 msgr2=0x564dd227ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:53.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.305+0000 7fb0fb65f740 1 --2- 172.21.15.164:0/3738971327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd22775d0 0x564dd227ded0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fb0e40118e0 tx=0x7fb0e4006c60 comp rx=0 tx=0).stop 2024-09-19T08:53:53.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.305+0000 7fb0fb65f740 1 -- 172.21.15.164:0/3738971327 shutdown_connections 2024-09-19T08:53:53.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.305+0000 7fb0fb65f740 1 --2- 172.21.15.164:0/3738971327 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564dd2282370 0x564dd227a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.305+0000 7fb0fb65f740 1 --2- 172.21.15.164:0/3738971327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564dd2278b40 0x564dd2280f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.305+0000 7fb0fb65f740 1 --2- 172.21.15.164:0/3738971327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd22775d0 0x564dd227ded0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.305+0000 7fb0fb65f740 1 -- 172.21.15.164:0/3738971327 >> 172.21.15.164:0/3738971327 conn(0x564dd2240310 msgr2=0x564dd22807f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:53.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.305+0000 7fb0fb65f740 1 -- 172.21.15.164:0/3738971327 shutdown_connections 2024-09-19T08:53:53.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.305+0000 7fb0fb65f740 1 -- 172.21.15.164:0/3738971327 wait complete. 2024-09-19T08:53:53.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.306+0000 7fb0fb65f740 1 -- start start 2024-09-19T08:53:53.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.306+0000 7fb0fb65f740 1 Processor -- start 2024-09-19T08:53:53.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.306+0000 7fb0fb65f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd22775d0 0x564dd22a2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:53.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.307+0000 7fb0fb65f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564dd2278b40 0x564dd229f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:53.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.307+0000 7fb0fb65f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564dd2282370 0x564dd229fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:53.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.307+0000 7fb0fb65f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f41aca70 con 0x564dd22775d0 2024-09-19T08:53:53.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.307+0000 7fb0f2ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd22775d0 0x564dd22a2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:53.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.307+0000 7fb0fb65f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f41ac3a0 con 0x564dd2282370 2024-09-19T08:53:53.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.307+0000 7fb0fb65f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f41a9600 con 0x564dd2278b40 2024-09-19T08:53:53.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.307+0000 7fb0f2ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd22775d0 0x564dd22a2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48250/0 (socket says 172.21.15.164:48250) 2024-09-19T08:53:53.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.307+0000 7fb0f2ffd640 1 -- 172.21.15.164:0/4185467054 learned_addr learned my addr 172.21.15.164:0/4185467054 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:53.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.307+0000 7fb0fb65f740 10 client.0 _ll_register_callbacks cb 0x7fb0e400a530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:53.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.307+0000 7fb0f27fc640 1 --2- 172.21.15.164:0/4185467054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564dd2278b40 0x564dd229f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:53.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.307+0000 7fb0f2ffd640 1 -- 172.21.15.164:0/4185467054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564dd2278b40 msgr2=0x564dd229f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:53.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.307+0000 7fb0f2ffd640 1 --2- 172.21.15.164:0/4185467054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564dd2278b40 0x564dd229f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.307+0000 7fb0f2ffd640 1 -- 172.21.15.164:0/4185467054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564dd2282370 msgr2=0x564dd229fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:53.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.307+0000 7fb0f2ffd640 1 --2- 172.21.15.164:0/4185467054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564dd2282370 0x564dd229fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.307+0000 7fb0f2ffd640 1 -- 172.21.15.164:0/4185467054 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564dd22a0490 con 0x564dd22775d0 2024-09-19T08:53:53.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.307+0000 7fb0f27fc640 1 --2- 172.21.15.164:0/4185467054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564dd2278b40 0x564dd229f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:53:53.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.307+0000 7fb0f2ffd640 1 --2- 172.21.15.164:0/4185467054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd22775d0 0x564dd22a2640 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4011d60 tx=0x7fb0e4011910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a57d29d2e6772c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:53.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.307+0000 7fb0d6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:53:53.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.308+0000 7fb0d6ff5640 1 -- 172.21.15.164:0/4185467054 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e403c060 con 0x564dd22775d0 2024-09-19T08:53:53.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.308+0000 7fb0d6ff5640 1 -- 172.21.15.164:0/4185467054 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7fb0e40059c0 con 0x564dd22775d0 2024-09-19T08:53:53.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.308+0000 7fb0fb65f740 1 -- 172.21.15.164:0/4185467054 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0e402a050 con 0x564dd22775d0 2024-09-19T08:53:53.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.308+0000 7fb0d6ff5640 1 -- 172.21.15.164:0/4185467054 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb0e4028040 con 0x564dd22775d0 2024-09-19T08:53:53.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.308+0000 7fb0fb65f740 10 client.17157 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:53.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.308+0000 7fb0fb65f740 1 -- 172.21.15.164:0/4185467054 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564dd2304ec0 con 0x564dd22775d0 2024-09-19T08:53:53.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.308+0000 7fb0d6ff5640 1 -- 172.21.15.164:0/4185467054 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0e402a050 con 0x564dd22775d0 2024-09-19T08:53:53.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.308+0000 7fb0d6ff5640 1 -- 172.21.15.164:0/4185467054 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb0e4006790 con 0x564dd22775d0 2024-09-19T08:53:53.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.309+0000 7fb0fb65f740 10 client.17157 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:53.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.309+0000 7fb0fb65f740 10 client.17157 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T08:53:53.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.309+0000 7fb0fb65f740 20 client.17157 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:53.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.309+0000 7fb0fb65f740 1 --2- 172.21.15.164:0/4185467054 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x564dd23054d0 0x564dd2307920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:53.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.309+0000 7fb0fb65f740 4 client.17157 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:53:53.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.309+0000 7fb0fb65f740 1 -- 172.21.15.164:0/4185467054 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564dd21e4d70 con 0x564dd23054d0 2024-09-19T08:53:53.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.309+0000 7fb0f27fc640 1 --2- 172.21.15.164:0/4185467054 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x564dd23054d0 0x564dd2307920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:53.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.310+0000 7fb0f27fc640 1 --2- 172.21.15.164:0/4185467054 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x564dd23054d0 0x564dd2307920 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1532d3c2230dd379 server_cookie=4b8141c0f80418d6 in_seq=0 out_seq=0 2024-09-19T08:53:53.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.310+0000 7fb0d6ff5640 10 client.17157 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:53:53.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.310+0000 7fb0d6ff5640 1 -- 172.21.15.164:0/4185467054 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564dd21e4d70 con 0x564dd23054d0 2024-09-19T08:53:53.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.310+0000 7fb0d6ff5640 10 client.17157 handle_command_reply: tid=0 2024-09-19T08:53:53.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.310+0000 7fb0fb65f740 1 client.17157 shutdown 2024-09-19T08:53:53.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.310+0000 7fb0fb65f740 1 -- 172.21.15.164:0/4185467054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd22775d0 msgr2=0x564dd22a2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:53.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.310+0000 7fb0fb65f740 1 --2- 172.21.15.164:0/4185467054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd22775d0 0x564dd22a2640 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4011d60 tx=0x7fb0e4011910 comp rx=0 tx=0).stop 2024-09-19T08:53:53.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.310+0000 7fb0fb65f740 1 -- 172.21.15.164:0/4185467054 shutdown_connections 2024-09-19T08:53:53.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.310+0000 7fb0fb65f740 1 --2- 172.21.15.164:0/4185467054 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x564dd23054d0 0x564dd2307920 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.310+0000 7fb0fb65f740 1 --2- 172.21.15.164:0/4185467054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564dd2282370 0x564dd229fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.310+0000 7fb0fb65f740 1 --2- 172.21.15.164:0/4185467054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564dd2278b40 0x564dd229f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.310+0000 7fb0fb65f740 1 --2- 172.21.15.164:0/4185467054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564dd22775d0 0x564dd22a2640 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.310+0000 7fb0fb65f740 1 -- 172.21.15.164:0/4185467054 >> 172.21.15.164:0/4185467054 conn(0x564dd2240310 msgr2=0x564dd2281010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:53.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.310+0000 7fb0d6ff5640 0 client.17157 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:53:53.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.310+0000 7fb0fb65f740 1 -- 172.21.15.164:0/4185467054 shutdown_connections 2024-09-19T08:53:53.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.310+0000 7fb0fb65f740 1 -- 172.21.15.164:0/4185467054 wait complete. 2024-09-19T08:53:53.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.311+0000 7fb0fb65f740 20 client.17157 trim_cache size 0 max 16384 2024-09-19T08:53:53.314 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:53:53.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.311+0000 7fb0f9286640 1 -- 172.21.15.164:0/1608588871 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb0dc0a33b0 msgr2=0x7fb0dc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:53.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.311+0000 7fb0f9286640 1 --2- 172.21.15.164:0/1608588871 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb0dc0a33b0 0x7fb0dc0a5870 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec005fd0 tx=0x7fb0ec004a40 comp rx=0 tx=0).stop 2024-09-19T08:53:53.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.311+0000 7fb0f9286640 1 -- 172.21.15.164:0/1608588871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0f41036e0 msgr2=0x7fb0f4198b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:53.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.311+0000 7fb0f9286640 1 --2- 172.21.15.164:0/1608588871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0f41036e0 0x7fb0f4198b40 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8005940 tx=0x7fb0e80042f0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.311+0000 7fb0f9286640 1 -- 172.21.15.164:0/1608588871 shutdown_connections 2024-09-19T08:53:53.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.311+0000 7fb0f9286640 1 --2- 172.21.15.164:0/1608588871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0f41082a0 0x7fb0f419ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.311+0000 7fb0f9286640 1 --2- 172.21.15.164:0/1608588871 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb0dc0a33b0 0x7fb0dc0a5870 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.311+0000 7fb0f9286640 1 --2- 172.21.15.164:0/1608588871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0f41036e0 0x7fb0f4198b40 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.311+0000 7fb0f9286640 1 --2- 172.21.15.164:0/1608588871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0f4102d30 0x7fb0f4198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:53.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.311+0000 7fb0f9286640 1 -- 172.21.15.164:0/1608588871 >> 172.21.15.164:0/1608588871 conn(0x7fb0f40fc7d0 msgr2=0x7fb0f4105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:53.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.312+0000 7fb0f9286640 1 -- 172.21.15.164:0/1608588871 shutdown_connections 2024-09-19T08:53:53.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:53.312+0000 7fb0f9286640 1 -- 172.21.15.164:0/1608588871 wait complete. 2024-09-19T08:53:53.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:52 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/1868803599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:53:53.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:52 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/1868803599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:53:53.588 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:53:53.956 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:53 smithi164 ceph-mon[30928]: pgmap v249: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:53:53.984 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T08:53:54.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:53 smithi170 ceph-mon[35072]: pgmap v249: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:53:54.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:53 smithi173 ceph-mon[35306]: pgmap v249: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:53:54.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.516+0000 7f11a3e05640 1 -- 172.21.15.164:0/1220177494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f119c103660 msgr2=0x7f119c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:54.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.516+0000 7f11a3e05640 1 --2- 172.21.15.164:0/1220177494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f119c103660 0x7f119c10bb50 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f1198009900 tx=0x7f119802a480 comp rx=0 tx=0).stop 2024-09-19T08:53:54.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.516+0000 7f11a3e05640 1 -- 172.21.15.164:0/1220177494 shutdown_connections 2024-09-19T08:53:54.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.516+0000 7f11a3e05640 1 --2- 172.21.15.164:0/1220177494 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f119c10c170 0x7f119c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.516+0000 7f11a3e05640 1 --2- 172.21.15.164:0/1220177494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f119c103660 0x7f119c10bb50 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.516+0000 7f11a3e05640 1 --2- 172.21.15.164:0/1220177494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f119c102d40 0x7f119c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.516+0000 7f11a3e05640 1 -- 172.21.15.164:0/1220177494 >> 172.21.15.164:0/1220177494 conn(0x7f119c0fc7a0 msgr2=0x7f119c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:54.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.516+0000 7f11a3e05640 1 -- 172.21.15.164:0/1220177494 shutdown_connections 2024-09-19T08:53:54.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.516+0000 7f11a3e05640 1 -- 172.21.15.164:0/1220177494 wait complete. 2024-09-19T08:53:54.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.516+0000 7f11a3e05640 1 Processor -- start 2024-09-19T08:53:54.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.517+0000 7f11a3e05640 1 -- start start 2024-09-19T08:53:54.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.517+0000 7f11a3e05640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f119c102d40 0x7f119c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:54.519 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.517+0000 7f11a3e05640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f119c103660 0x7f119c198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:54.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.517+0000 7f11a3e05640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f119c10c170 0x7f119c19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:54.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.517+0000 7f11a3e05640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f119c111fb0 con 0x7f119c102d40 2024-09-19T08:53:54.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.517+0000 7f11a3e05640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f119c111e30 con 0x7f119c10c170 2024-09-19T08:53:54.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.517+0000 7f11a3e05640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f119c112130 con 0x7f119c103660 2024-09-19T08:53:54.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.517+0000 7f11a1ba1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f119c102d40 0x7f119c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:54.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.517+0000 7f11a1ba1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f119c102d40 0x7f119c198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40444/0 (socket says 172.21.15.164:40444) 2024-09-19T08:53:54.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.517+0000 7f11a1ba1640 1 -- 172.21.15.164:0/1955334792 learned_addr learned my addr 172.21.15.164:0/1955334792 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:54.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.517+0000 7f11a13a0640 1 --2- 172.21.15.164:0/1955334792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f119c103660 0x7f119c198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:54.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.517+0000 7f11a23a2640 1 --2- 172.21.15.164:0/1955334792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f119c10c170 0x7f119c19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:54.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.517+0000 7f11a1ba1640 1 -- 172.21.15.164:0/1955334792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f119c103660 msgr2=0x7f119c198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:54.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.517+0000 7f11a1ba1640 1 --2- 172.21.15.164:0/1955334792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f119c103660 0x7f119c198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.517+0000 7f11a1ba1640 1 -- 172.21.15.164:0/1955334792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f119c10c170 msgr2=0x7f119c19ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:54.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.517+0000 7f11a1ba1640 1 --2- 172.21.15.164:0/1955334792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f119c10c170 0x7f119c19ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.517+0000 7f11a1ba1640 1 -- 172.21.15.164:0/1955334792 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f119c19d540 con 0x7f119c102d40 2024-09-19T08:53:54.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.517+0000 7f11a13a0640 1 --2- 172.21.15.164:0/1955334792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f119c103660 0x7f119c198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:53:54.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.517+0000 7f11a1ba1640 1 --2- 172.21.15.164:0/1955334792 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f119c102d40 0x7f119c198590 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f118c00c890 tx=0x7f118c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:54.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.518+0000 7f1192ffd640 1 -- 172.21.15.164:0/1955334792 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c004220 con 0x7f119c102d40 2024-09-19T08:53:54.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.518+0000 7f11a3e05640 1 -- 172.21.15.164:0/1955334792 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f119c19d830 con 0x7f119c102d40 2024-09-19T08:53:54.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.518+0000 7f1192ffd640 1 -- 172.21.15.164:0/1955334792 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f118c0043c0 con 0x7f119c102d40 2024-09-19T08:53:54.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.518+0000 7f11a3e05640 1 -- 172.21.15.164:0/1955334792 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f119c1008d0 con 0x7f119c102d40 2024-09-19T08:53:54.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.518+0000 7f1192ffd640 1 -- 172.21.15.164:0/1955334792 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c004a00 con 0x7f119c102d40 2024-09-19T08:53:54.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.521+0000 7f1192ffd640 1 -- 172.21.15.164:0/1955334792 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f118c0079b0 con 0x7f119c102d40 2024-09-19T08:53:54.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.521+0000 7f1192ffd640 1 --2- 172.21.15.164:0/1955334792 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f11880a77c0 0x7f11880a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:54.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.521+0000 7f11a13a0640 1 --2- 172.21.15.164:0/1955334792 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f11880a77c0 0x7f11880a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:54.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.521+0000 7f1192ffd640 1 -- 172.21.15.164:0/1955334792 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f118c0cfa90 con 0x7f119c102d40 2024-09-19T08:53:54.522 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.522+0000 7f11a61de740 1 Processor -- start 2024-09-19T08:53:54.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.522+0000 7f11a61de740 1 -- start start 2024-09-19T08:53:54.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.522+0000 7f11a13a0640 1 --2- 172.21.15.164:0/1955334792 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f11880a77c0 0x7f11880a9c80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f119c0fee00 tx=0x7f1198005a20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:54.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.522+0000 7f11a61de740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e2714e850 0x558e2714ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:54.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.522+0000 7f11a61de740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e2714f170 0x558e271594f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:54.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.522+0000 7f11a61de740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e27159a30 0x558e2715be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:54.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.522+0000 7f11a61de740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558e270b3160 con 0x558e2714f170 2024-09-19T08:53:54.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.522+0000 7f11a61de740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558e2705a720 con 0x558e27159a30 2024-09-19T08:53:54.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.522+0000 7f11a61de740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558e270af120 con 0x558e2714e850 2024-09-19T08:53:54.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.523+0000 7f11a13a0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e2714f170 0x558e271594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:54.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.523+0000 7f11a13a0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e2714f170 0x558e271594f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40460/0 (socket says 172.21.15.164:40460) 2024-09-19T08:53:54.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.523+0000 7f11a13a0640 1 -- 172.21.15.164:0/3070999082 learned_addr learned my addr 172.21.15.164:0/3070999082 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:54.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.523+0000 7f11a1ba1640 1 --2- 172.21.15.164:0/3070999082 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e2714e850 0x558e2714ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:54.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.523+0000 7f11a23a2640 1 --2- 172.21.15.164:0/3070999082 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e27159a30 0x558e2715be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:54.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.523+0000 7f11a13a0640 1 -- 172.21.15.164:0/3070999082 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e2714e850 msgr2=0x558e2714ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:54.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.523+0000 7f11a13a0640 1 --2- 172.21.15.164:0/3070999082 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e2714e850 0x558e2714ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.523+0000 7f11a13a0640 1 -- 172.21.15.164:0/3070999082 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e27159a30 msgr2=0x558e2715be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:54.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.523+0000 7f11a13a0640 1 --2- 172.21.15.164:0/3070999082 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e27159a30 0x558e2715be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.523+0000 7f11a13a0640 1 -- 172.21.15.164:0/3070999082 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e26e24ba0 con 0x558e2714f170 2024-09-19T08:53:54.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.523+0000 7f11a23a2640 1 --2- 172.21.15.164:0/3070999082 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e27159a30 0x558e2715be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:53:54.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.523+0000 7f11a13a0640 1 --2- 172.21.15.164:0/3070999082 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e2714f170 0x558e271594f0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f1198009900 tx=0x7f119803c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c5ba07e35b4e54 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:54.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.523+0000 7f1187fff640 1 -- 172.21.15.164:0/3070999082 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198035050 con 0x558e2714f170 2024-09-19T08:53:54.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.524+0000 7f1187fff640 1 -- 172.21.15.164:0/3070999082 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f119804d070 con 0x558e2714f170 2024-09-19T08:53:54.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.524+0000 7f1187fff640 1 -- 172.21.15.164:0/3070999082 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198048c70 con 0x558e2714f170 2024-09-19T08:53:54.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.524+0000 7f11a61de740 1 -- 172.21.15.164:0/3070999082 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e2714f170 msgr2=0x558e271594f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:54.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.524+0000 7f11a61de740 1 --2- 172.21.15.164:0/3070999082 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e2714f170 0x558e271594f0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f1198009900 tx=0x7f119803c000 comp rx=0 tx=0).stop 2024-09-19T08:53:54.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.524+0000 7f11a61de740 1 -- 172.21.15.164:0/3070999082 shutdown_connections 2024-09-19T08:53:54.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.524+0000 7f11a61de740 1 --2- 172.21.15.164:0/3070999082 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e27159a30 0x558e2715be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.525+0000 7f11a61de740 1 --2- 172.21.15.164:0/3070999082 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e2714f170 0x558e271594f0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.525+0000 7f11a61de740 1 --2- 172.21.15.164:0/3070999082 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e2714e850 0x558e2714ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.525+0000 7f11a61de740 1 -- 172.21.15.164:0/3070999082 >> 172.21.15.164:0/3070999082 conn(0x558e2714ade0 msgr2=0x558e2714b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:54.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.525+0000 7f11a61de740 1 -- 172.21.15.164:0/3070999082 shutdown_connections 2024-09-19T08:53:54.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.525+0000 7f11a61de740 1 -- 172.21.15.164:0/3070999082 wait complete. 2024-09-19T08:53:54.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.525+0000 7f11a61de740 1 -- start start 2024-09-19T08:53:54.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.525+0000 7f11a61de740 1 Processor -- start 2024-09-19T08:53:54.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.526+0000 7f11a61de740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e2714e850 0x558e271568b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:54.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.526+0000 7f11a61de740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e2714f170 0x558e27156df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:54.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.526+0000 7f11a61de740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e27159a30 0x558e27157330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:54.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.526+0000 7f11a61de740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558e2715c600 con 0x558e2714f170 2024-09-19T08:53:54.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.526+0000 7f11a61de740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558e2703b900 con 0x558e27159a30 2024-09-19T08:53:54.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.526+0000 7f11a61de740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558e2715c780 con 0x558e2714e850 2024-09-19T08:53:54.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.526+0000 7f11a13a0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e2714f170 0x558e27156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:54.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.526+0000 7f11a61de740 10 client.0 _ll_register_callbacks cb 0x558e27039780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:54.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.526+0000 7f11a1ba1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e2714e850 0x558e271568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:54.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.526+0000 7f11a13a0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e2714f170 0x558e27156df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40474/0 (socket says 172.21.15.164:40474) 2024-09-19T08:53:54.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.526+0000 7f11a13a0640 1 -- 172.21.15.164:0/2800427399 learned_addr learned my addr 172.21.15.164:0/2800427399 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:54.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.526+0000 7f11a23a2640 1 --2- 172.21.15.164:0/2800427399 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e27159a30 0x558e27157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:54.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.526+0000 7f11a1ba1640 1 -- 172.21.15.164:0/2800427399 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e27159a30 msgr2=0x558e27157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:54.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.526+0000 7f11a1ba1640 1 --2- 172.21.15.164:0/2800427399 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e27159a30 0x558e27157330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.526+0000 7f11a1ba1640 1 -- 172.21.15.164:0/2800427399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e2714f170 msgr2=0x558e27156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:54.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.527+0000 7f11a1ba1640 1 --2- 172.21.15.164:0/2800427399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e2714f170 0x558e27156df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.527+0000 7f11a1ba1640 1 -- 172.21.15.164:0/2800427399 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e2707bc50 con 0x558e2714e850 2024-09-19T08:53:54.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.527+0000 7f11a13a0640 1 --2- 172.21.15.164:0/2800427399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e2714f170 0x558e27156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:53:54.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.527+0000 7f11a1ba1640 1 --2- 172.21.15.164:0/2800427399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e2714e850 0x558e271568b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f118c0cd220 tx=0x7f118c09b9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=def787329dbfa2f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:54.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.527+0000 7f11857fa640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-09-19T08:53:54.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.527+0000 7f11857fa640 1 -- 172.21.15.164:0/2800427399 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c01d040 con 0x558e2714e850 2024-09-19T08:53:54.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.527+0000 7f11a61de740 1 -- 172.21.15.164:0/2800427399 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f119804d070 con 0x558e2714e850 2024-09-19T08:53:54.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.527+0000 7f11857fa640 1 -- 172.21.15.164:0/2800427399 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f118c09b200 con 0x558e2714e850 2024-09-19T08:53:54.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.527+0000 7f11857fa640 1 -- 172.21.15.164:0/2800427399 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f118c0a0160 con 0x558e2714e850 2024-09-19T08:53:54.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.527+0000 7f11a61de740 10 client.26320 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:54.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.527+0000 7f11a61de740 1 -- 172.21.15.164:0/2800427399 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558e27168240 con 0x558e2714e850 2024-09-19T08:53:54.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.528+0000 7f11857fa640 1 -- 172.21.15.164:0/2800427399 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f119804d070 con 0x558e2714e850 2024-09-19T08:53:54.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.528+0000 7f11857fa640 1 -- 172.21.15.164:0/2800427399 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f118c09ad90 con 0x558e2714e850 2024-09-19T08:53:54.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.528+0000 7f11a61de740 10 client.26320 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:54.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.528+0000 7f11a61de740 10 client.26320 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:53:54.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.528+0000 7f11a61de740 20 client.26320 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:54.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.528+0000 7f11a61de740 1 --2- 172.21.15.164:0/2800427399 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x558e27169700 0x558e2716bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:54.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.528+0000 7f11a61de740 4 client.26320 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:53:54.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.528+0000 7f11a61de740 1 -- 172.21.15.164:0/2800427399 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e270c0890 con 0x558e27169700 2024-09-19T08:53:54.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.528+0000 7f11a13a0640 1 --2- 172.21.15.164:0/2800427399 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x558e27169700 0x558e2716bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:54.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.529+0000 7f11857fa640 10 client.26320 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:54.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.529+0000 7f11a13a0640 1 --2- 172.21.15.164:0/2800427399 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x558e27169700 0x558e2716bb50 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=386faeae76ac8721 server_cookie=14ca06e1e3ebfa4e in_seq=0 out_seq=0 2024-09-19T08:53:54.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.529+0000 7f11857fa640 1 -- 172.21.15.164:0/2800427399 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e270c0890 con 0x558e27169700 2024-09-19T08:53:54.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.529+0000 7f11857fa640 10 client.26320 handle_command_reply: tid=0 2024-09-19T08:53:54.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.529+0000 7f11a61de740 1 client.26320 shutdown 2024-09-19T08:53:54.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.530+0000 7f11a61de740 1 -- 172.21.15.164:0/2800427399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e2714e850 msgr2=0x558e271568b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:54.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.530+0000 7f11a61de740 1 --2- 172.21.15.164:0/2800427399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e2714e850 0x558e271568b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f118c0cd220 tx=0x7f118c09b9b0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.530+0000 7f11a61de740 1 -- 172.21.15.164:0/2800427399 shutdown_connections 2024-09-19T08:53:54.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.530+0000 7f11a61de740 1 --2- 172.21.15.164:0/2800427399 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x558e27169700 0x558e2716bb50 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.530+0000 7f11a61de740 1 --2- 172.21.15.164:0/2800427399 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e27159a30 0x558e27157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.530+0000 7f11a61de740 1 --2- 172.21.15.164:0/2800427399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e2714f170 0x558e27156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.530+0000 7f11a61de740 1 --2- 172.21.15.164:0/2800427399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e2714e850 0x558e271568b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.530+0000 7f11857fa640 0 client.26320 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:54.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.530+0000 7f11a61de740 1 -- 172.21.15.164:0/2800427399 >> 172.21.15.164:0/2800427399 conn(0x558e2714ade0 msgr2=0x558e27150ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:54.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.530+0000 7f11a61de740 1 -- 172.21.15.164:0/2800427399 shutdown_connections 2024-09-19T08:53:54.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.530+0000 7f11a61de740 1 -- 172.21.15.164:0/2800427399 wait complete. 2024-09-19T08:53:54.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.530+0000 7f11a61de740 20 client.26320 trim_cache size 0 max 16384 2024-09-19T08:53:54.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.548+0000 7f11a61de740 1 Processor -- start 2024-09-19T08:53:54.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.548+0000 7f11a61de740 1 -- start start 2024-09-19T08:53:54.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.548+0000 7f11a61de740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e2714e850 0x558e2714b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:54.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.548+0000 7f11a61de740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e2714f170 0x558e27158820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:54.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.548+0000 7f11a61de740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e27159a30 0x558e27171420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:54.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.548+0000 7f11a61de740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f119c1ab6d0 con 0x558e2714f170 2024-09-19T08:53:54.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.548+0000 7f11a61de740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f119c1ad210 con 0x558e2714e850 2024-09-19T08:53:54.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.548+0000 7f11a61de740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f119c1abda0 con 0x558e27159a30 2024-09-19T08:53:54.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.548+0000 7f11a13a0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e2714f170 0x558e27158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:54.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.548+0000 7f11a13a0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e2714f170 0x558e27158820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40484/0 (socket says 172.21.15.164:40484) 2024-09-19T08:53:54.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.548+0000 7f11a13a0640 1 -- 172.21.15.164:0/3803438449 learned_addr learned my addr 172.21.15.164:0/3803438449 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:54.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.548+0000 7f11a1ba1640 1 --2- 172.21.15.164:0/3803438449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e2714e850 0x558e2714b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:54.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.548+0000 7f11a23a2640 1 --2- 172.21.15.164:0/3803438449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e27159a30 0x558e27171420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:54.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.548+0000 7f11a13a0640 1 -- 172.21.15.164:0/3803438449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e27159a30 msgr2=0x558e27171420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:54.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.548+0000 7f11a13a0640 1 --2- 172.21.15.164:0/3803438449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e27159a30 0x558e27171420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.549+0000 7f11a13a0640 1 -- 172.21.15.164:0/3803438449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e2714e850 msgr2=0x558e2714b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:54.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.549+0000 7f11a13a0640 1 --2- 172.21.15.164:0/3803438449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e2714e850 0x558e2714b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.549+0000 7f11a13a0640 1 -- 172.21.15.164:0/3803438449 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e27155860 con 0x558e2714f170 2024-09-19T08:53:54.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.549+0000 7f11a23a2640 1 --2- 172.21.15.164:0/3803438449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e27159a30 0x558e27171420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:53:54.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.549+0000 7f11a13a0640 1 --2- 172.21.15.164:0/3803438449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e2714f170 0x558e27158820 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f1198004650 tx=0x7f11980488b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4103313b6f5885 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:54.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.549+0000 7f11877fe640 1 -- 172.21.15.164:0/3803438449 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119803f070 con 0x558e2714f170 2024-09-19T08:53:54.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.549+0000 7f11877fe640 1 -- 172.21.15.164:0/3803438449 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f119804d050 con 0x558e2714f170 2024-09-19T08:53:54.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.549+0000 7f11877fe640 1 -- 172.21.15.164:0/3803438449 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198035040 con 0x558e2714f170 2024-09-19T08:53:54.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.550+0000 7f11a61de740 1 -- 172.21.15.164:0/3803438449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e2714f170 msgr2=0x558e27158820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:54.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.550+0000 7f11a61de740 1 --2- 172.21.15.164:0/3803438449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e2714f170 0x558e27158820 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f1198004650 tx=0x7f11980488b0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.550+0000 7f11a61de740 1 -- 172.21.15.164:0/3803438449 shutdown_connections 2024-09-19T08:53:54.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.550+0000 7f11a61de740 1 --2- 172.21.15.164:0/3803438449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e27159a30 0x558e27171420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.550+0000 7f11a61de740 1 --2- 172.21.15.164:0/3803438449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e2714f170 0x558e27158820 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.550+0000 7f11a61de740 1 --2- 172.21.15.164:0/3803438449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e2714e850 0x558e2714b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.550+0000 7f11a61de740 1 -- 172.21.15.164:0/3803438449 >> 172.21.15.164:0/3803438449 conn(0x558e2714ade0 msgr2=0x558e27158110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:54.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.550+0000 7f11a61de740 1 -- 172.21.15.164:0/3803438449 shutdown_connections 2024-09-19T08:53:54.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.550+0000 7f11a61de740 1 -- 172.21.15.164:0/3803438449 wait complete. 2024-09-19T08:53:54.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.550+0000 7f11a61de740 1 -- start start 2024-09-19T08:53:54.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.550+0000 7f11a61de740 1 Processor -- start 2024-09-19T08:53:54.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.551+0000 7f11a61de740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e2714e850 0x558e2715d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:54.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.551+0000 7f11a61de740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e2714f170 0x558e2715df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:54.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.551+0000 7f11a61de740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e27159a30 0x558e2715e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:54.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.551+0000 7f11a1ba1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e2714e850 0x558e2715d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:54.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.551+0000 7f11a61de740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f119c1acb40 con 0x558e27159a30 2024-09-19T08:53:54.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.551+0000 7f11a61de740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f119c1ac470 con 0x558e2714e850 2024-09-19T08:53:54.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.551+0000 7f11a1ba1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e2714e850 0x558e2715d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:33084/0 (socket says 172.21.15.164:33084) 2024-09-19T08:53:54.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.551+0000 7f11a61de740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f119c1a96d0 con 0x558e2714f170 2024-09-19T08:53:54.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.551+0000 7f11a1ba1640 1 -- 172.21.15.164:0/2945365355 learned_addr learned my addr 172.21.15.164:0/2945365355 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:54.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.551+0000 7f11a61de740 10 client.0 _ll_register_callbacks cb 0x7f118c0058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:54.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.551+0000 7f11a23a2640 1 --2- 172.21.15.164:0/2945365355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e27159a30 0x558e2715e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:54.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.551+0000 7f11a13a0640 1 --2- 172.21.15.164:0/2945365355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e2714f170 0x558e2715df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:54.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.551+0000 7f11a1ba1640 1 -- 172.21.15.164:0/2945365355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e2714f170 msgr2=0x558e2715df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:54.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.551+0000 7f11a1ba1640 1 --2- 172.21.15.164:0/2945365355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e2714f170 0x558e2715df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.551+0000 7f11a1ba1640 1 -- 172.21.15.164:0/2945365355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e27159a30 msgr2=0x558e2715e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:54.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.551+0000 7f11a1ba1640 1 --2- 172.21.15.164:0/2945365355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e27159a30 0x558e2715e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.551+0000 7f11a1ba1640 1 -- 172.21.15.164:0/2945365355 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e271d3690 con 0x558e2714e850 2024-09-19T08:53:54.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.551+0000 7f11a13a0640 1 --2- 172.21.15.164:0/2945365355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e2714f170 0x558e2715df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:53:54.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.552+0000 7f11a1ba1640 1 --2- 172.21.15.164:0/2945365355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e2714e850 0x558e2715d9c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f118c0cd820 tx=0x7f118c005300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3996c094ca6563 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:54.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.552+0000 7f11857fa640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-19T08:53:54.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.552+0000 7f11857fa640 1 -- 172.21.15.164:0/2945365355 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c07f060 con 0x558e2714e850 2024-09-19T08:53:54.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.552+0000 7f11a61de740 1 -- 172.21.15.164:0/2945365355 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f119804d050 con 0x558e2714e850 2024-09-19T08:53:54.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.552+0000 7f11857fa640 1 -- 172.21.15.164:0/2945365355 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f118c09b200 con 0x558e2714e850 2024-09-19T08:53:54.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.552+0000 7f11857fa640 1 -- 172.21.15.164:0/2945365355 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f118c01d070 con 0x558e2714e850 2024-09-19T08:53:54.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.552+0000 7f11a61de740 10 client.26384 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:54.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.552+0000 7f11a61de740 1 -- 172.21.15.164:0/2945365355 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558e271d45d0 con 0x558e2714e850 2024-09-19T08:53:54.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.552+0000 7f11857fa640 1 -- 172.21.15.164:0/2945365355 <== mon.2 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f119804d050 con 0x558e2714e850 2024-09-19T08:53:54.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.553+0000 7f11857fa640 1 -- 172.21.15.164:0/2945365355 <== mon.2 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f118c086120 con 0x558e2714e850 2024-09-19T08:53:54.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.553+0000 7f11a61de740 10 client.26384 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:54.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.553+0000 7f11a61de740 10 client.26384 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:53:54.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.553+0000 7f11a61de740 20 client.26384 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:54.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.553+0000 7f11a61de740 1 --2- 172.21.15.164:0/2945365355 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x558e271d4b80 0x558e271d6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:54.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.553+0000 7f11a61de740 4 client.26384 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.75, "await_for": 45.0}] 2024-09-19T08:53:54.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.553+0000 7f11a61de740 1 -- 172.21.15.164:0/2945365355 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.75, "await_for": 45.0}) -- 0x558e270babe0 con 0x558e271d4b80 2024-09-19T08:53:54.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.553+0000 7f11a13a0640 1 --2- 172.21.15.164:0/2945365355 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x558e271d4b80 0x558e271d6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:54.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.553+0000 7f11857fa640 10 client.26384 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:54.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.553+0000 7f11a13a0640 1 --2- 172.21.15.164:0/2945365355 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x558e271d4b80 0x558e271d6fa0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c241a386bdaa7020 server_cookie=77ede10964eeb9c in_seq=0 out_seq=0 2024-09-19T08:53:54.559 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:53:54.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.569+0000 7f11857fa640 1 -- 172.21.15.164:0/2945365355 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558e270babe0 con 0x558e271d4b80 2024-09-19T08:53:54.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.569+0000 7f11857fa640 10 client.26384 handle_command_reply: tid=0 2024-09-19T08:53:54.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.569+0000 7f11a61de740 1 client.26384 shutdown 2024-09-19T08:53:54.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.570+0000 7f11a61de740 1 -- 172.21.15.164:0/2945365355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e2714e850 msgr2=0x558e2715d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:54.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.570+0000 7f11a61de740 1 --2- 172.21.15.164:0/2945365355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e2714e850 0x558e2715d9c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f118c0cd820 tx=0x7f118c005300 comp rx=0 tx=0).stop 2024-09-19T08:53:54.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.570+0000 7f11a61de740 1 -- 172.21.15.164:0/2945365355 shutdown_connections 2024-09-19T08:53:54.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.570+0000 7f11a61de740 1 --2- 172.21.15.164:0/2945365355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e27159a30 0x558e2715e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.570+0000 7f11a61de740 1 --2- 172.21.15.164:0/2945365355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e2714f170 0x558e2715df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.570+0000 7f11a61de740 1 --2- 172.21.15.164:0/2945365355 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x558e271d4b80 0x558e271d6fa0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.570+0000 7f11a61de740 1 --2- 172.21.15.164:0/2945365355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e2714e850 0x558e2715d9c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.570+0000 7f11a61de740 1 -- 172.21.15.164:0/2945365355 >> 172.21.15.164:0/2945365355 conn(0x558e2714ade0 msgr2=0x558e27158930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:54.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.570+0000 7f11857fa640 0 client.26384 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:54.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.570+0000 7f11a61de740 1 -- 172.21.15.164:0/2945365355 shutdown_connections 2024-09-19T08:53:54.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.570+0000 7f11a61de740 1 -- 172.21.15.164:0/2945365355 wait complete. 2024-09-19T08:53:54.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.570+0000 7f11a61de740 20 client.26384 trim_cache size 0 max 16384 2024-09-19T08:53:54.572 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:53:54.572 INFO:teuthology.orchestra.run.smithi164.stdout: "epoch": 32, 2024-09-19T08:53:54.572 INFO:teuthology.orchestra.run.smithi164.stdout: "leader": 24479, 2024-09-19T08:53:54.572 INFO:teuthology.orchestra.run.smithi164.stdout: "set_version": 30, 2024-09-19T08:53:54.572 INFO:teuthology.orchestra.run.smithi164.stdout: "sets": { 2024-09-19T08:53:54.572 INFO:teuthology.orchestra.run.smithi164.stdout: "1e7fd7ac": { 2024-09-19T08:53:54.572 INFO:teuthology.orchestra.run.smithi164.stdout: "version": 30, 2024-09-19T08:53:54.572 INFO:teuthology.orchestra.run.smithi164.stdout: "age_ref": 0.0, 2024-09-19T08:53:54.572 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:53:54.572 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "QUIESCED", 2024-09-19T08:53:54.573 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:53:54.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:53:54.573 INFO:teuthology.orchestra.run.smithi164.stdout: "timeout": 90.0, 2024-09-19T08:53:54.573 INFO:teuthology.orchestra.run.smithi164.stdout: "expiration": 145.8, 2024-09-19T08:53:54.573 INFO:teuthology.orchestra.run.smithi164.stdout: "members": { 2024-09-19T08:53:54.573 INFO:teuthology.orchestra.run.smithi164.stdout: "file:/": { 2024-09-19T08:53:54.573 INFO:teuthology.orchestra.run.smithi164.stdout: "excluded": false, 2024-09-19T08:53:54.573 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:53:54.573 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "QUIESCED", 2024-09-19T08:53:54.573 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:53:54.573 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:53:54.573 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:53:54.573 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:53:54.574 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:53:54.574 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:53:54.574 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:53:54.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.571+0000 7f11a3e05640 1 -- 172.21.15.164:0/1955334792 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f11880a77c0 msgr2=0x7f11880a9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:54.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.571+0000 7f11a3e05640 1 --2- 172.21.15.164:0/1955334792 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f11880a77c0 0x7f11880a9c80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f119c0fee00 tx=0x7f1198005a20 comp rx=0 tx=0).stop 2024-09-19T08:53:54.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.571+0000 7f11a3e05640 1 -- 172.21.15.164:0/1955334792 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f119c102d40 msgr2=0x7f119c198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:54.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.571+0000 7f11a3e05640 1 --2- 172.21.15.164:0/1955334792 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f119c102d40 0x7f119c198590 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f118c00c890 tx=0x7f118c00cd50 comp rx=0 tx=0).stop 2024-09-19T08:53:54.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.571+0000 7f11a3e05640 1 -- 172.21.15.164:0/1955334792 shutdown_connections 2024-09-19T08:53:54.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.571+0000 7f11a3e05640 1 --2- 172.21.15.164:0/1955334792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f119c10c170 0x7f119c19ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.571+0000 7f11a3e05640 1 --2- 172.21.15.164:0/1955334792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f119c103660 0x7f119c198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.571+0000 7f11a3e05640 1 --2- 172.21.15.164:0/1955334792 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f11880a77c0 0x7f11880a9c80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.571+0000 7f11a3e05640 1 --2- 172.21.15.164:0/1955334792 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f119c102d40 0x7f119c198590 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:54.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.571+0000 7f11a3e05640 1 -- 172.21.15.164:0/1955334792 >> 172.21.15.164:0/1955334792 conn(0x7f119c0fc7a0 msgr2=0x7f119c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:54.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.571+0000 7f11a3e05640 1 -- 172.21.15.164:0/1955334792 shutdown_connections 2024-09-19T08:53:54.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:54.571+0000 7f11a3e05640 1 -- 172.21.15.164:0/1955334792 wait complete. 2024-09-19T08:53:55.184 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1e7fd7ac', quiesce took 4.8 seconds. Will release after: 20.95 2024-09-19T08:53:55.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.658+0000 7f751fa24640 1 -- 172.21.15.164:0/11479647 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7518102790 msgr2=0x7f75181067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:55.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.658+0000 7f751fa24640 1 --2- 172.21.15.164:0/11479647 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7518102790 0x7f75181067e0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f7508009900 tx=0x7f750802a410 comp rx=0 tx=0).stop 2024-09-19T08:53:55.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.658+0000 7f751fa24640 1 -- 172.21.15.164:0/11479647 shutdown_connections 2024-09-19T08:53:55.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.658+0000 7f751fa24640 1 --2- 172.21.15.164:0/11479647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7518106f10 0x7f751810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:55.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.658+0000 7f751fa24640 1 --2- 172.21.15.164:0/11479647 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7518102790 0x7f75181067e0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:55.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.658+0000 7f751fa24640 1 --2- 172.21.15.164:0/11479647 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7518101de0 0x7f75181021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:55.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.658+0000 7f751fa24640 1 -- 172.21.15.164:0/11479647 >> 172.21.15.164:0/11479647 conn(0x7f75180fd8c0 msgr2=0x7f75180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:55.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.658+0000 7f751fa24640 1 -- 172.21.15.164:0/11479647 shutdown_connections 2024-09-19T08:53:55.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.658+0000 7f751fa24640 1 -- 172.21.15.164:0/11479647 wait complete. 2024-09-19T08:53:55.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.659+0000 7f751fa24640 1 Processor -- start 2024-09-19T08:53:55.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.659+0000 7f751fa24640 1 -- start start 2024-09-19T08:53:55.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.659+0000 7f751fa24640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7518101de0 0x7f7518198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:55.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.659+0000 7f751fa24640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7518102790 0x7f7518198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:55.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.659+0000 7f751fa24640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7518106f10 0x7f751819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:55.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.659+0000 7f751fa24640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7518111f70 con 0x7f7518102790 2024-09-19T08:53:55.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.659+0000 7f751fa24640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7518111df0 con 0x7f7518106f10 2024-09-19T08:53:55.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.659+0000 7f751fa24640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f75181120f0 con 0x7f7518101de0 2024-09-19T08:53:55.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.659+0000 7f751cfbf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7518102790 0x7f7518198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:55.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.659+0000 7f751cfbf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7518102790 0x7f7518198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40504/0 (socket says 172.21.15.164:40504) 2024-09-19T08:53:55.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.659+0000 7f751cfbf640 1 -- 172.21.15.164:0/3847529923 learned_addr learned my addr 172.21.15.164:0/3847529923 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:55.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.659+0000 7f751dfc1640 1 --2- 172.21.15.164:0/3847529923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7518106f10 0x7f751819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:55.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.659+0000 7f751d7c0640 1 --2- 172.21.15.164:0/3847529923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7518101de0 0x7f7518198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:55.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.660+0000 7f751cfbf640 1 -- 172.21.15.164:0/3847529923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7518101de0 msgr2=0x7f7518198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:55.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.660+0000 7f751cfbf640 1 --2- 172.21.15.164:0/3847529923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7518101de0 0x7f7518198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:55.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.660+0000 7f751cfbf640 1 -- 172.21.15.164:0/3847529923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7518106f10 msgr2=0x7f751819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:55.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.660+0000 7f751cfbf640 1 --2- 172.21.15.164:0/3847529923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7518106f10 0x7f751819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:55.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.660+0000 7f751cfbf640 1 -- 172.21.15.164:0/3847529923 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f751819d400 con 0x7f7518102790 2024-09-19T08:53:55.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.660+0000 7f751cfbf640 1 --2- 172.21.15.164:0/3847529923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7518102790 0x7f7518198990 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f7508006010 tx=0x7f75080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:55.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.660+0000 7f750e7fc640 1 -- 172.21.15.164:0/3847529923 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750803f030 con 0x7f7518102790 2024-09-19T08:53:55.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.660+0000 7f751fa24640 1 -- 172.21.15.164:0/3847529923 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f751819d690 con 0x7f7518102790 2024-09-19T08:53:55.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.660+0000 7f750e7fc640 1 -- 172.21.15.164:0/3847529923 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7508003ce0 con 0x7f7518102790 2024-09-19T08:53:55.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.660+0000 7f750e7fc640 1 -- 172.21.15.164:0/3847529923 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7508035070 con 0x7f7518102790 2024-09-19T08:53:55.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.660+0000 7f751fa24640 1 -- 172.21.15.164:0/3847529923 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75181a4fd0 con 0x7f7518102790 2024-09-19T08:53:55.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.662+0000 7f750e7fc640 1 -- 172.21.15.164:0/3847529923 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f7508042030 con 0x7f7518102790 2024-09-19T08:53:55.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.663+0000 7f7521dfd740 1 Processor -- start 2024-09-19T08:53:55.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.663+0000 7f7521dfd740 1 -- start start 2024-09-19T08:53:55.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.663+0000 7f7521dfd740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6747a3b40 0x55b6747a2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:55.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.663+0000 7f7521dfd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6747a25d0 0x55b6747a2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:55.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.663+0000 7f7521dfd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6747ad370 0x55b6747af760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:55.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.663+0000 7f7521dfd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b6747082f0 con 0x55b6747ad370 2024-09-19T08:53:55.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.663+0000 7f7521dfd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b6746afcf0 con 0x55b6747a25d0 2024-09-19T08:53:55.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.663+0000 7f7521dfd740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b6747042b0 con 0x55b6747a3b40 2024-09-19T08:53:55.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.663+0000 7f750e7fc640 1 --2- 172.21.15.164:0/3847529923 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f75040a33b0 0x7f75040a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:55.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.663+0000 7f751cfbf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6747ad370 0x55b6747af760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:55.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.663+0000 7f751cfbf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6747ad370 0x55b6747af760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40506/0 (socket says 172.21.15.164:40506) 2024-09-19T08:53:55.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.663+0000 7f751cfbf640 1 -- 172.21.15.164:0/3103846837 learned_addr learned my addr 172.21.15.164:0/3103846837 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:55.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.663+0000 7f751d7c0640 1 --2- 172.21.15.164:0/3103846837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6747a25d0 0x55b6747a2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:55.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.663+0000 7f751dfc1640 1 --2- 172.21.15.164:0/3847529923 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f75040a33b0 0x7f75040a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:55.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.663+0000 7f751dfc1640 1 --2- 172.21.15.164:0/3103846837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6747a3b40 0x55b6747a2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:55.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.664+0000 7f750e7fc640 1 -- 172.21.15.164:0/3847529923 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f75080f0260 con 0x7f7518102790 2024-09-19T08:53:55.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.664+0000 7f751cfbf640 1 -- 172.21.15.164:0/3103846837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6747a3b40 msgr2=0x55b6747a2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:55.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.664+0000 7f751cfbf640 1 --2- 172.21.15.164:0/3103846837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6747a3b40 0x55b6747a2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:55.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.664+0000 7f751cfbf640 1 -- 172.21.15.164:0/3103846837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6747a25d0 msgr2=0x55b6747a2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:55.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.664+0000 7f751cfbf640 1 --2- 172.21.15.164:0/3103846837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6747a25d0 0x55b6747a2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:55.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.664+0000 7f751cfbf640 1 -- 172.21.15.164:0/3103846837 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6746943a0 con 0x55b6747ad370 2024-09-19T08:53:55.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.664+0000 7f751d7c0640 1 --2- 172.21.15.164:0/3103846837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6747a25d0 0x55b6747a2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:53:55.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.664+0000 7f751dfc1640 1 --2- 172.21.15.164:0/3847529923 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f75040a33b0 0x7f75040a5870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7510009fd0 tx=0x7f7510009d80 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:55.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.664+0000 7f751cfbf640 1 --2- 172.21.15.164:0/3103846837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6747ad370 0x55b6747af760 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f75080edaa0 tx=0x7f750802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b73d5ce35e5dd99 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:55.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.664+0000 7f75037fe640 1 -- 172.21.15.164:0/3103846837 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75080bc230 con 0x55b6747ad370 2024-09-19T08:53:55.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.664+0000 7f75037fe640 1 -- 172.21.15.164:0/3103846837 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75080c09f0 con 0x55b6747ad370 2024-09-19T08:53:55.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.664+0000 7f75037fe640 1 -- 172.21.15.164:0/3103846837 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750803bcb0 con 0x55b6747ad370 2024-09-19T08:53:55.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.665+0000 7f7521dfd740 1 -- 172.21.15.164:0/3103846837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6747ad370 msgr2=0x55b6747af760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:55.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.665+0000 7f7521dfd740 1 --2- 172.21.15.164:0/3103846837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6747ad370 0x55b6747af760 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f75080edaa0 tx=0x7f750802a410 comp rx=0 tx=0).stop 2024-09-19T08:53:55.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.665+0000 7f7521dfd740 1 -- 172.21.15.164:0/3103846837 shutdown_connections 2024-09-19T08:53:55.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.665+0000 7f7521dfd740 1 --2- 172.21.15.164:0/3103846837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6747ad370 0x55b6747af760 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:55.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.665+0000 7f7521dfd740 1 --2- 172.21.15.164:0/3103846837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6747a25d0 0x55b6747a2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:55.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.665+0000 7f7521dfd740 1 --2- 172.21.15.164:0/3103846837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6747a3b40 0x55b6747a2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:55.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.665+0000 7f7521dfd740 1 -- 172.21.15.164:0/3103846837 >> 172.21.15.164:0/3103846837 conn(0x55b67476b310 msgr2=0x55b67479ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:55.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.665+0000 7f7521dfd740 1 -- 172.21.15.164:0/3103846837 shutdown_connections 2024-09-19T08:53:55.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.665+0000 7f7521dfd740 1 -- 172.21.15.164:0/3103846837 wait complete. 2024-09-19T08:53:55.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.665+0000 7f7521dfd740 1 -- start start 2024-09-19T08:53:55.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.665+0000 7f7521dfd740 1 Processor -- start 2024-09-19T08:53:55.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.666+0000 7f7521dfd740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6747a25d0 0x55b6747acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:55.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.666+0000 7f7521dfd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6747a3b40 0x55b6747aa060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:55.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.666+0000 7f7521dfd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6747ad370 0x55b6747aa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:55.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.666+0000 7f7521dfd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b6747afe30 con 0x55b6747a3b40 2024-09-19T08:53:55.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.666+0000 7f7521dfd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b6746c3060 con 0x55b6747ad370 2024-09-19T08:53:55.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.666+0000 7f7521dfd740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b6747affb0 con 0x55b6747a25d0 2024-09-19T08:53:55.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.666+0000 7f7521dfd740 10 client.0 _ll_register_callbacks cb 0x55b6744168c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:55.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.666+0000 7f751cfbf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6747a3b40 0x55b6747aa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:55.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.666+0000 7f751cfbf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6747a3b40 0x55b6747aa060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40510/0 (socket says 172.21.15.164:40510) 2024-09-19T08:53:55.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.666+0000 7f751cfbf640 1 -- 172.21.15.164:0/3826360277 learned_addr learned my addr 172.21.15.164:0/3826360277 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:55.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.666+0000 7f751dfc1640 1 --2- 172.21.15.164:0/3826360277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6747ad370 0x55b6747aa7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:55.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.666+0000 7f751d7c0640 1 --2- 172.21.15.164:0/3826360277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6747a25d0 0x55b6747acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:55.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.666+0000 7f751cfbf640 1 -- 172.21.15.164:0/3826360277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6747a25d0 msgr2=0x55b6747acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:55.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.666+0000 7f751cfbf640 1 --2- 172.21.15.164:0/3826360277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6747a25d0 0x55b6747acdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:55.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.666+0000 7f751cfbf640 1 -- 172.21.15.164:0/3826360277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6747ad370 msgr2=0x55b6747aa7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:55.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.666+0000 7f751cfbf640 1 --2- 172.21.15.164:0/3826360277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6747ad370 0x55b6747aa7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:55.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.666+0000 7f751cfbf640 1 -- 172.21.15.164:0/3826360277 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6744793c0 con 0x55b6747a3b40 2024-09-19T08:53:55.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.666+0000 7f751cfbf640 1 --2- 172.21.15.164:0/3826360277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6747a3b40 0x55b6747aa060 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f75080c4d80 tx=0x7f75080f01f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbbb71f54f82b149 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:55.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.666+0000 7f7500ff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:53:55.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.666+0000 7f7500ff9640 1 -- 172.21.15.164:0/3826360277 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75080ca060 con 0x55b6747a3b40 2024-09-19T08:53:55.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.666+0000 7f7500ff9640 1 -- 172.21.15.164:0/3826360277 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f7508049c40 con 0x55b6747a3b40 2024-09-19T08:53:55.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.667+0000 7f7521dfd740 1 -- 172.21.15.164:0/3826360277 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75080c09f0 con 0x55b6747a3b40 2024-09-19T08:53:55.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.667+0000 7f7500ff9640 1 -- 172.21.15.164:0/3826360277 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f75080da070 con 0x55b6747a3b40 2024-09-19T08:53:55.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.667+0000 7f7521dfd740 10 client.17187 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:55.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.667+0000 7f7521dfd740 1 -- 172.21.15.164:0/3826360277 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b6747afe30 con 0x55b6747a3b40 2024-09-19T08:53:55.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.667+0000 7f7500ff9640 1 -- 172.21.15.164:0/3826360277 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75080c09f0 con 0x55b6747a3b40 2024-09-19T08:53:55.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.667+0000 7f7500ff9640 1 -- 172.21.15.164:0/3826360277 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f75080c4eb0 con 0x55b6747a3b40 2024-09-19T08:53:55.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.667+0000 7f7521dfd740 10 client.17187 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:55.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.667+0000 7f7521dfd740 10 client.17187 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T08:53:55.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.667+0000 7f7521dfd740 20 client.17187 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:55.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.667+0000 7f7521dfd740 1 --2- 172.21.15.164:0/3826360277 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55b6747bc9e0 0x55b6747bedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:55.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.667+0000 7f7521dfd740 4 client.17187 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:53:55.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.667+0000 7f7521dfd740 1 -- 172.21.15.164:0/3826360277 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b674715860 con 0x55b6747bc9e0 2024-09-19T08:53:55.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.668+0000 7f751d7c0640 1 --2- 172.21.15.164:0/3826360277 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55b6747bc9e0 0x55b6747bedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:55.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.669+0000 7f751d7c0640 1 --2- 172.21.15.164:0/3826360277 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55b6747bc9e0 0x55b6747bedd0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f202f167766588cc server_cookie=67808f90b4a223ae in_seq=0 out_seq=0 2024-09-19T08:53:55.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.669+0000 7f7500ff9640 10 client.17187 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:53:55.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.669+0000 7f7500ff9640 1 -- 172.21.15.164:0/3826360277 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b674715860 con 0x55b6747bc9e0 2024-09-19T08:53:55.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.669+0000 7f7500ff9640 10 client.17187 handle_command_reply: tid=0 2024-09-19T08:53:55.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.669+0000 7f7521dfd740 1 client.17187 shutdown 2024-09-19T08:53:55.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.669+0000 7f7521dfd740 1 -- 172.21.15.164:0/3826360277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6747a3b40 msgr2=0x55b6747aa060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:55.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.669+0000 7f7521dfd740 1 --2- 172.21.15.164:0/3826360277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6747a3b40 0x55b6747aa060 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f75080c4d80 tx=0x7f75080f01f0 comp rx=0 tx=0).stop 2024-09-19T08:53:55.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.669+0000 7f7521dfd740 1 -- 172.21.15.164:0/3826360277 shutdown_connections 2024-09-19T08:53:55.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.669+0000 7f7521dfd740 1 --2- 172.21.15.164:0/3826360277 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55b6747bc9e0 0x55b6747bedd0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:55.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.669+0000 7f7521dfd740 1 --2- 172.21.15.164:0/3826360277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6747ad370 0x55b6747aa7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:55.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.669+0000 7f7521dfd740 1 --2- 172.21.15.164:0/3826360277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6747a3b40 0x55b6747aa060 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:55.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.669+0000 7f7521dfd740 1 --2- 172.21.15.164:0/3826360277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6747a25d0 0x55b6747acdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:55.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.669+0000 7f7521dfd740 1 -- 172.21.15.164:0/3826360277 >> 172.21.15.164:0/3826360277 conn(0x55b67476b310 msgr2=0x55b6747a4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:55.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.670+0000 7f7500ff9640 0 client.17187 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:53:55.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.670+0000 7f7521dfd740 1 -- 172.21.15.164:0/3826360277 shutdown_connections 2024-09-19T08:53:55.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.670+0000 7f7521dfd740 1 -- 172.21.15.164:0/3826360277 wait complete. 2024-09-19T08:53:55.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.670+0000 7f7521dfd740 20 client.17187 trim_cache size 0 max 16384 2024-09-19T08:53:55.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.682+0000 7f7521dfd740 1 Processor -- start 2024-09-19T08:53:55.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.682+0000 7f7521dfd740 1 -- start start 2024-09-19T08:53:55.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.683+0000 7f7521dfd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6747a25d0 0x55b6747aa030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:55.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.683+0000 7f7521dfd740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6747a3b40 0x55b6747c4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:55.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.683+0000 7f7521dfd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6747ad370 0x55b6747c4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:55.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.683+0000 7f7521dfd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f75181ad7b0 con 0x55b6747ad370 2024-09-19T08:53:55.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.683+0000 7f7521dfd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f75181af2f0 con 0x55b6747a25d0 2024-09-19T08:53:55.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.683+0000 7f7521dfd740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f75181ade80 con 0x55b6747a3b40 2024-09-19T08:53:55.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.683+0000 7f751dfc1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6747ad370 0x55b6747c4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:55.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.683+0000 7f751dfc1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6747ad370 0x55b6747c4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40514/0 (socket says 172.21.15.164:40514) 2024-09-19T08:53:55.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.683+0000 7f751dfc1640 1 -- 172.21.15.164:0/277978923 learned_addr learned my addr 172.21.15.164:0/277978923 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:55.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.683+0000 7f751d7c0640 1 --2- 172.21.15.164:0/277978923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6747a25d0 0x55b6747aa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:55.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.683+0000 7f751dfc1640 1 -- 172.21.15.164:0/277978923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6747a3b40 msgr2=0x55b6747c4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:55.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.683+0000 7f751dfc1640 1 --2- 172.21.15.164:0/277978923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6747a3b40 0x55b6747c4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:55.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.683+0000 7f751dfc1640 1 -- 172.21.15.164:0/277978923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6747a25d0 msgr2=0x55b6747aa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:55.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.683+0000 7f751dfc1640 1 --2- 172.21.15.164:0/277978923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6747a25d0 0x55b6747aa030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:55.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.683+0000 7f751dfc1640 1 -- 172.21.15.164:0/277978923 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6747a85d0 con 0x55b6747ad370 2024-09-19T08:53:55.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.683+0000 7f751d7c0640 1 --2- 172.21.15.164:0/277978923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6747a25d0 0x55b6747aa030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:53:55.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.683+0000 7f751dfc1640 1 --2- 172.21.15.164:0/277978923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6747ad370 0x55b6747c4ad0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f7510012750 tx=0x7f751001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5acc25a1b6462a90 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:55.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.683+0000 7f7502ffd640 1 -- 172.21.15.164:0/277978923 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751001eaa0 con 0x55b6747ad370 2024-09-19T08:53:55.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.683+0000 7f7502ffd640 1 -- 172.21.15.164:0/277978923 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f751001ec40 con 0x55b6747ad370 2024-09-19T08:53:55.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.683+0000 7f7502ffd640 1 -- 172.21.15.164:0/277978923 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7510020400 con 0x55b6747ad370 2024-09-19T08:53:55.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.684+0000 7f7521dfd740 1 -- 172.21.15.164:0/277978923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6747ad370 msgr2=0x55b6747c4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:55.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.684+0000 7f7521dfd740 1 --2- 172.21.15.164:0/277978923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6747ad370 0x55b6747c4ad0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f7510012750 tx=0x7f751001f000 comp rx=0 tx=0).stop 2024-09-19T08:53:55.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.684+0000 7f7521dfd740 1 -- 172.21.15.164:0/277978923 shutdown_connections 2024-09-19T08:53:55.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.684+0000 7f7521dfd740 1 --2- 172.21.15.164:0/277978923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6747ad370 0x55b6747c4ad0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:55.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.684+0000 7f7521dfd740 1 --2- 172.21.15.164:0/277978923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6747a3b40 0x55b6747c4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:55.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.684+0000 7f7521dfd740 1 --2- 172.21.15.164:0/277978923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6747a25d0 0x55b6747aa030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:55.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.684+0000 7f7521dfd740 1 -- 172.21.15.164:0/277978923 >> 172.21.15.164:0/277978923 conn(0x55b67476b310 msgr2=0x55b6747a9920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:55.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.684+0000 7f7521dfd740 1 -- 172.21.15.164:0/277978923 shutdown_connections 2024-09-19T08:53:55.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.684+0000 7f7521dfd740 1 -- 172.21.15.164:0/277978923 wait complete. 2024-09-19T08:53:55.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.684+0000 7f7521dfd740 1 -- start start 2024-09-19T08:53:55.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.684+0000 7f7521dfd740 1 Processor -- start 2024-09-19T08:53:55.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.685+0000 7f7521dfd740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6747a25d0 0x55b6747b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:55.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.685+0000 7f7521dfd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6747a3b40 0x55b6747a6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:55.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.685+0000 7f7521dfd740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6747ad370 0x55b6747a72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:55.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.685+0000 7f7521dfd740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f75181aec20 con 0x55b6747a25d0 2024-09-19T08:53:55.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.685+0000 7f7521dfd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f75181ae550 con 0x55b6747a3b40 2024-09-19T08:53:55.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.685+0000 7f7521dfd740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f75181ab7b0 con 0x55b6747ad370 2024-09-19T08:53:55.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.685+0000 7f7521dfd740 10 client.0 _ll_register_callbacks cb 0x7f751400b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:55.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.685+0000 7f751d7c0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6747a25d0 0x55b6747b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:55.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.685+0000 7f751d7c0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6747a25d0 0x55b6747b58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40528/0 (socket says 172.21.15.164:40528) 2024-09-19T08:53:55.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.685+0000 7f751d7c0640 1 -- 172.21.15.164:0/4077677869 learned_addr learned my addr 172.21.15.164:0/4077677869 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:55.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.685+0000 7f751cfbf640 1 --2- 172.21.15.164:0/4077677869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6747a3b40 0x55b6747a6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:55.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.685+0000 7f751dfc1640 1 --2- 172.21.15.164:0/4077677869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6747ad370 0x55b6747a72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:55.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.685+0000 7f751d7c0640 1 -- 172.21.15.164:0/4077677869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6747ad370 msgr2=0x55b6747a72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:55.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.685+0000 7f751d7c0640 1 --2- 172.21.15.164:0/4077677869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6747ad370 0x55b6747a72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:55.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.685+0000 7f751d7c0640 1 -- 172.21.15.164:0/4077677869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6747a3b40 msgr2=0x55b6747a6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:55.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.685+0000 7f751d7c0640 1 --2- 172.21.15.164:0/4077677869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6747a3b40 0x55b6747a6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:55.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.685+0000 7f751d7c0640 1 -- 172.21.15.164:0/4077677869 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6747a7a70 con 0x55b6747a25d0 2024-09-19T08:53:55.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.685+0000 7f751cfbf640 1 --2- 172.21.15.164:0/4077677869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6747a3b40 0x55b6747a6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:53:55.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.686+0000 7f751d7c0640 1 --2- 172.21.15.164:0/4077677869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6747a25d0 0x55b6747b58b0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f7514000c00 tx=0x7f751400bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6f09bbd41777f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:55.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.686+0000 7f7500ff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:53:55.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.686+0000 7f7500ff9640 1 -- 172.21.15.164:0/4077677869 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7514005020 con 0x55b6747a25d0 2024-09-19T08:53:55.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.686+0000 7f7500ff9640 1 -- 172.21.15.164:0/4077677869 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f75140051c0 con 0x55b6747a25d0 2024-09-19T08:53:55.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.686+0000 7f7521dfd740 1 -- 172.21.15.164:0/4077677869 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f751001ec40 con 0x55b6747a25d0 2024-09-19T08:53:55.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.686+0000 7f7500ff9640 1 -- 172.21.15.164:0/4077677869 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f75140053c0 con 0x55b6747a25d0 2024-09-19T08:53:55.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.686+0000 7f7521dfd740 10 client.17193 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:55.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.686+0000 7f7521dfd740 1 -- 172.21.15.164:0/4077677869 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b6748274d0 con 0x55b6747a25d0 2024-09-19T08:53:55.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.686+0000 7f7500ff9640 1 -- 172.21.15.164:0/4077677869 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f751001ec40 con 0x55b6747a25d0 2024-09-19T08:53:55.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.686+0000 7f7500ff9640 1 -- 172.21.15.164:0/4077677869 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f75140116a0 con 0x55b6747a25d0 2024-09-19T08:53:55.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.686+0000 7f7521dfd740 10 client.17193 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:55.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.686+0000 7f7521dfd740 10 client.17193 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T08:53:55.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.686+0000 7f7521dfd740 20 client.17193 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:55.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.686+0000 7f7521dfd740 1 --2- 172.21.15.164:0/4077677869 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55b674827c60 0x55b67482a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:55.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.686+0000 7f7521dfd740 4 client.17193 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:53:55.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.686+0000 7f7521dfd740 1 -- 172.21.15.164:0/4077677869 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b67470fd70 con 0x55b674827c60 2024-09-19T08:53:55.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.687+0000 7f751cfbf640 1 --2- 172.21.15.164:0/4077677869 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55b674827c60 0x55b67482a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:55.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.687+0000 7f751cfbf640 1 --2- 172.21.15.164:0/4077677869 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55b674827c60 0x55b67482a080 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=361e837f421cfc2b server_cookie=6cba68d7d18a8e36 in_seq=0 out_seq=0 2024-09-19T08:53:55.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.687+0000 7f7500ff9640 10 client.17193 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:53:55.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.687+0000 7f7500ff9640 1 -- 172.21.15.164:0/4077677869 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b67470fd70 con 0x55b674827c60 2024-09-19T08:53:55.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.687+0000 7f7500ff9640 10 client.17193 handle_command_reply: tid=0 2024-09-19T08:53:55.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.687+0000 7f7521dfd740 1 client.17193 shutdown 2024-09-19T08:53:55.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.687+0000 7f7521dfd740 1 -- 172.21.15.164:0/4077677869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6747a25d0 msgr2=0x55b6747b58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:55.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.687+0000 7f7521dfd740 1 --2- 172.21.15.164:0/4077677869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6747a25d0 0x55b6747b58b0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f7514000c00 tx=0x7f751400bd80 comp rx=0 tx=0).stop 2024-09-19T08:53:55.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.687+0000 7f7521dfd740 1 -- 172.21.15.164:0/4077677869 shutdown_connections 2024-09-19T08:53:55.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.687+0000 7f7521dfd740 1 --2- 172.21.15.164:0/4077677869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6747ad370 0x55b6747a72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:55.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.687+0000 7f7521dfd740 1 --2- 172.21.15.164:0/4077677869 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55b674827c60 0x55b67482a080 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:55.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.688+0000 7f7521dfd740 1 --2- 172.21.15.164:0/4077677869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6747a3b40 0x55b6747a6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:55.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.688+0000 7f7521dfd740 1 --2- 172.21.15.164:0/4077677869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6747a25d0 0x55b6747b58b0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:55.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.688+0000 7f7500ff9640 0 client.17193 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:53:55.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.688+0000 7f7521dfd740 1 -- 172.21.15.164:0/4077677869 >> 172.21.15.164:0/4077677869 conn(0x55b67476b310 msgr2=0x55b6747a8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:55.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.688+0000 7f7521dfd740 1 -- 172.21.15.164:0/4077677869 shutdown_connections 2024-09-19T08:53:55.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.688+0000 7f7521dfd740 1 -- 172.21.15.164:0/4077677869 wait complete. 2024-09-19T08:53:55.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.688+0000 7f7521dfd740 20 client.17193 trim_cache size 0 max 16384 2024-09-19T08:53:55.694 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:53:55.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.688+0000 7f751fa24640 1 -- 172.21.15.164:0/3847529923 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f75040a33b0 msgr2=0x7f75040a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:55.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.688+0000 7f751fa24640 1 --2- 172.21.15.164:0/3847529923 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f75040a33b0 0x7f75040a5870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7510009fd0 tx=0x7f7510009d80 comp rx=0 tx=0).stop 2024-09-19T08:53:55.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.688+0000 7f751fa24640 1 -- 172.21.15.164:0/3847529923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7518102790 msgr2=0x7f7518198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:55.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.688+0000 7f751fa24640 1 --2- 172.21.15.164:0/3847529923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7518102790 0x7f7518198990 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f7508006010 tx=0x7f75080042f0 comp rx=0 tx=0).stop 2024-09-19T08:53:55.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.688+0000 7f751fa24640 1 -- 172.21.15.164:0/3847529923 shutdown_connections 2024-09-19T08:53:55.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.688+0000 7f751fa24640 1 --2- 172.21.15.164:0/3847529923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7518106f10 0x7f751819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:55.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.688+0000 7f751fa24640 1 --2- 172.21.15.164:0/3847529923 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f75040a33b0 0x7f75040a5870 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:55.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.688+0000 7f751fa24640 1 --2- 172.21.15.164:0/3847529923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7518102790 0x7f7518198990 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:55.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.688+0000 7f751fa24640 1 --2- 172.21.15.164:0/3847529923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7518101de0 0x7f7518198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:55.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.688+0000 7f751fa24640 1 -- 172.21.15.164:0/3847529923 >> 172.21.15.164:0/3847529923 conn(0x7f75180fd8c0 msgr2=0x7f75180fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:55.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.688+0000 7f751fa24640 1 -- 172.21.15.164:0/3847529923 shutdown_connections 2024-09-19T08:53:55.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:55.688+0000 7f751fa24640 1 -- 172.21.15.164:0/3847529923 wait complete. 2024-09-19T08:53:56.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:55 smithi164 ceph-mon[30928]: pgmap v250: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T08:53:56.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:55 smithi170 ceph-mon[35072]: pgmap v250: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T08:53:56.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:55 smithi173 ceph-mon[35306]: pgmap v250: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T08:53:56.486 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T08:53:56.733 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:53:57.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.200+0000 7f801e05b640 1 -- 172.21.15.164:0/147468383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8018106f20 msgr2=0x7f801810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:57.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.200+0000 7f801e05b640 1 --2- 172.21.15.164:0/147468383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8018106f20 0x7f801810eaa0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f8010009900 tx=0x7f801002a410 comp rx=0 tx=0).stop 2024-09-19T08:53:57.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.200+0000 7f801e05b640 1 -- 172.21.15.164:0/147468383 shutdown_connections 2024-09-19T08:53:57.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.200+0000 7f801e05b640 1 --2- 172.21.15.164:0/147468383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8018106f20 0x7f801810eaa0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.200+0000 7f801e05b640 1 --2- 172.21.15.164:0/147468383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80181027a0 0x7f80181067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.200+0000 7f801e05b640 1 --2- 172.21.15.164:0/147468383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8018101df0 0x7f80181021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.200+0000 7f801e05b640 1 -- 172.21.15.164:0/147468383 >> 172.21.15.164:0/147468383 conn(0x7f80180fd8b0 msgr2=0x7f80180ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:57.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.201+0000 7f801e05b640 1 -- 172.21.15.164:0/147468383 shutdown_connections 2024-09-19T08:53:57.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.201+0000 7f801e05b640 1 -- 172.21.15.164:0/147468383 wait complete. 2024-09-19T08:53:57.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.201+0000 7f801e05b640 1 Processor -- start 2024-09-19T08:53:57.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.201+0000 7f801e05b640 1 -- start start 2024-09-19T08:53:57.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.201+0000 7f801e05b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8018101df0 0x7f80181984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:57.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.201+0000 7f801e05b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80181027a0 0x7f80181989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:57.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.201+0000 7f801e05b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8018106f20 0x7f801819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:57.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.201+0000 7f801e05b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8018111f80 con 0x7f8018106f20 2024-09-19T08:53:57.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.201+0000 7f801e05b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8018111e00 con 0x7f8018101df0 2024-09-19T08:53:57.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.201+0000 7f801e05b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8018112100 con 0x7f80181027a0 2024-09-19T08:53:57.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.202+0000 7f8017fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8018106f20 0x7f801819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:57.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.202+0000 7f8017fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8018106f20 0x7f801819cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40546/0 (socket says 172.21.15.164:40546) 2024-09-19T08:53:57.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.202+0000 7f8017fff640 1 -- 172.21.15.164:0/1157676045 learned_addr learned my addr 172.21.15.164:0/1157676045 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:57.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.202+0000 7f80177fe640 1 --2- 172.21.15.164:0/1157676045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8018101df0 0x7f80181984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:57.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.202+0000 7f8017fff640 1 -- 172.21.15.164:0/1157676045 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80181027a0 msgr2=0x7f80181989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:57.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.202+0000 7f8016ffd640 1 --2- 172.21.15.164:0/1157676045 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80181027a0 0x7f80181989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:57.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.202+0000 7f8017fff640 1 --2- 172.21.15.164:0/1157676045 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80181027a0 0x7f80181989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.202+0000 7f8017fff640 1 -- 172.21.15.164:0/1157676045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8018101df0 msgr2=0x7f80181984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:57.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.202+0000 7f8017fff640 1 --2- 172.21.15.164:0/1157676045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8018101df0 0x7f80181984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.202+0000 7f8017fff640 1 -- 172.21.15.164:0/1157676045 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f801819d450 con 0x7f8018106f20 2024-09-19T08:53:57.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.202+0000 7f80177fe640 1 --2- 172.21.15.164:0/1157676045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8018101df0 0x7f80181984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:53:57.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.202+0000 7f8017fff640 1 --2- 172.21.15.164:0/1157676045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8018106f20 0x7f801819cd70 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f8010009230 tx=0x7f80100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:57.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.202+0000 7f8014ff9640 1 -- 172.21.15.164:0/1157676045 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801003f030 con 0x7f8018106f20 2024-09-19T08:53:57.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.202+0000 7f801e05b640 1 -- 172.21.15.164:0/1157676045 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f801819d6e0 con 0x7f8018106f20 2024-09-19T08:53:57.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.202+0000 7f8014ff9640 1 -- 172.21.15.164:0/1157676045 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8010003ce0 con 0x7f8018106f20 2024-09-19T08:53:57.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.202+0000 7f8014ff9640 1 -- 172.21.15.164:0/1157676045 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8010035070 con 0x7f8018106f20 2024-09-19T08:53:57.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.202+0000 7f801e05b640 1 -- 172.21.15.164:0/1157676045 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80181a5020 con 0x7f8018106f20 2024-09-19T08:53:57.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.204+0000 7f8014ff9640 1 -- 172.21.15.164:0/1157676045 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f8010042030 con 0x7f8018106f20 2024-09-19T08:53:57.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.205+0000 7f8020434740 1 Processor -- start 2024-09-19T08:53:57.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.205+0000 7f8020434740 1 -- start start 2024-09-19T08:53:57.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.205+0000 7f8014ff9640 1 --2- 172.21.15.164:0/1157676045 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f80000a33b0 0x7f80000a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:57.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.205+0000 7f8020434740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647d45e2b40 0x5647d45e1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:57.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.205+0000 7f8020434740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647d45e15d0 0x5647d45e1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:57.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.205+0000 7f8020434740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647d45f0780 0x5647d45f2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:57.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.205+0000 7f8020434740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5647d45472f0 con 0x5647d45e2b40 2024-09-19T08:53:57.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.205+0000 7f8020434740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5647d44eecf0 con 0x5647d45f0780 2024-09-19T08:53:57.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.205+0000 7f8020434740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5647d45432b0 con 0x5647d45e15d0 2024-09-19T08:53:57.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.205+0000 7f8014ff9640 1 -- 172.21.15.164:0/1157676045 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f80100f06d0 con 0x7f8018106f20 2024-09-19T08:53:57.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.205+0000 7f8016ffd640 1 --2- 172.21.15.164:0/1157676045 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f80000a33b0 0x7f80000a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:57.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.205+0000 7f80177fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647d45e2b40 0x5647d45e1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:57.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.205+0000 7f80177fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647d45e2b40 0x5647d45e1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40560/0 (socket says 172.21.15.164:40560) 2024-09-19T08:53:57.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.205+0000 7f80177fe640 1 -- 172.21.15.164:0/3358434650 learned_addr learned my addr 172.21.15.164:0/3358434650 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:57.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.205+0000 7f8016ffd640 1 --2- 172.21.15.164:0/3358434650 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647d45e15d0 0x5647d45e1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:57.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.206+0000 7f80177fe640 1 -- 172.21.15.164:0/3358434650 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647d45e15d0 msgr2=0x5647d45e1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:57.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.206+0000 7f80177fe640 1 --2- 172.21.15.164:0/3358434650 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647d45e15d0 0x5647d45e1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.206+0000 7f80177fe640 1 -- 172.21.15.164:0/3358434650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647d45f0780 msgr2=0x5647d45f2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:57.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.206+0000 7f8017fff640 1 --2- 172.21.15.164:0/3358434650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647d45f0780 0x5647d45f2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:57.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.206+0000 7f80177fe640 1 --2- 172.21.15.164:0/3358434650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647d45f0780 0x5647d45f2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.206+0000 7f80177fe640 1 -- 172.21.15.164:0/3358434650 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647d44d33a0 con 0x5647d45e2b40 2024-09-19T08:53:57.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.206+0000 7f8017fff640 1 --2- 172.21.15.164:0/3358434650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647d45f0780 0x5647d45f2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:53:57.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.206+0000 7f8016ffd640 1 --2- 172.21.15.164:0/1157676045 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f80000a33b0 0x7f80000a5870 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x5647d45deac0 tx=0x7f800c00b450 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:57.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.206+0000 7f80177fe640 1 --2- 172.21.15.164:0/3358434650 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647d45e2b40 0x5647d45e1090 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f8008008e70 tx=0x7f800800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df7fafc371dd4ec1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:57.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.206+0000 7f7ffdffb640 1 -- 172.21.15.164:0/3358434650 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800800f830 con 0x5647d45e2b40 2024-09-19T08:53:57.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.206+0000 7f7ffdffb640 1 -- 172.21.15.164:0/3358434650 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8008018480 con 0x5647d45e2b40 2024-09-19T08:53:57.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.206+0000 7f7ffdffb640 1 -- 172.21.15.164:0/3358434650 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800800e5b0 con 0x5647d45e2b40 2024-09-19T08:53:57.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.207+0000 7f8020434740 1 -- 172.21.15.164:0/3358434650 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647d45e2b40 msgr2=0x5647d45e1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:57.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.207+0000 7f8020434740 1 --2- 172.21.15.164:0/3358434650 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647d45e2b40 0x5647d45e1090 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f8008008e70 tx=0x7f800800d710 comp rx=0 tx=0).stop 2024-09-19T08:53:57.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.207+0000 7f8020434740 1 -- 172.21.15.164:0/3358434650 shutdown_connections 2024-09-19T08:53:57.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.207+0000 7f8020434740 1 --2- 172.21.15.164:0/3358434650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647d45f0780 0x5647d45f2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.207+0000 7f8020434740 1 --2- 172.21.15.164:0/3358434650 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647d45e15d0 0x5647d45e1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.207+0000 7f8020434740 1 --2- 172.21.15.164:0/3358434650 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647d45e2b40 0x5647d45e1090 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.207+0000 7f8020434740 1 -- 172.21.15.164:0/3358434650 >> 172.21.15.164:0/3358434650 conn(0x5647d45aa310 msgr2=0x5647d45ddc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:57.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.207+0000 7f8020434740 1 -- 172.21.15.164:0/3358434650 shutdown_connections 2024-09-19T08:53:57.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.207+0000 7f8020434740 1 -- 172.21.15.164:0/3358434650 wait complete. 2024-09-19T08:53:57.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.207+0000 7f8020434740 1 -- start start 2024-09-19T08:53:57.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.207+0000 7f8020434740 1 Processor -- start 2024-09-19T08:53:57.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.208+0000 7f8020434740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647d45e15d0 0x5647d45f01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:57.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.208+0000 7f8020434740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647d45e2b40 0x5647d45ed450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:57.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.208+0000 7f8020434740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647d45f0780 0x5647d45eda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:57.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.208+0000 7f8020434740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5647d45f3480 con 0x5647d45e2b40 2024-09-19T08:53:57.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.208+0000 7f8020434740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5647d4502060 con 0x5647d45e15d0 2024-09-19T08:53:57.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.208+0000 7f8020434740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5647d45f3600 con 0x5647d45f0780 2024-09-19T08:53:57.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.208+0000 7f8020434740 10 client.0 _ll_register_callbacks cb 0x5647d42558c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:57.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.208+0000 7f8016ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647d45e2b40 0x5647d45ed450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:57.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.208+0000 7f8016ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647d45e2b40 0x5647d45ed450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40576/0 (socket says 172.21.15.164:40576) 2024-09-19T08:53:57.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.208+0000 7f8016ffd640 1 -- 172.21.15.164:0/3783420291 learned_addr learned my addr 172.21.15.164:0/3783420291 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:57.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.208+0000 7f80177fe640 1 --2- 172.21.15.164:0/3783420291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647d45e15d0 0x5647d45f01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:57.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.208+0000 7f8017fff640 1 --2- 172.21.15.164:0/3783420291 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647d45f0780 0x5647d45eda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:57.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.208+0000 7f8016ffd640 1 -- 172.21.15.164:0/3783420291 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647d45f0780 msgr2=0x5647d45eda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:57.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.208+0000 7f8016ffd640 1 --2- 172.21.15.164:0/3783420291 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647d45f0780 0x5647d45eda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.208+0000 7f8016ffd640 1 -- 172.21.15.164:0/3783420291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647d45e15d0 msgr2=0x5647d45f01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:57.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.208+0000 7f8016ffd640 1 --2- 172.21.15.164:0/3783420291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647d45e15d0 0x5647d45f01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.208+0000 7f8016ffd640 1 -- 172.21.15.164:0/3783420291 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647d42b83c0 con 0x5647d45e2b40 2024-09-19T08:53:57.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.208+0000 7f80177fe640 1 --2- 172.21.15.164:0/3783420291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647d45e15d0 0x5647d45f01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:53:57.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.208+0000 7f8016ffd640 1 --2- 172.21.15.164:0/3783420291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647d45e2b40 0x5647d45ed450 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f800c012a30 tx=0x7f800c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f78f8379ffc73f56 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:57.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.208+0000 7f7ff6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:53:57.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.208+0000 7f7ff6ffd640 1 -- 172.21.15.164:0/3783420291 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c023070 con 0x5647d45e2b40 2024-09-19T08:53:57.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.208+0000 7f8020434740 1 -- 172.21.15.164:0/3783420291 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8008018480 con 0x5647d45e2b40 2024-09-19T08:53:57.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.208+0000 7f7ff6ffd640 1 -- 172.21.15.164:0/3783420291 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f800c011d50 con 0x5647d45e2b40 2024-09-19T08:53:57.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.208+0000 7f7ff6ffd640 1 -- 172.21.15.164:0/3783420291 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f800c02f050 con 0x5647d45e2b40 2024-09-19T08:53:57.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.208+0000 7f8020434740 10 client.17205 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:57.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.209+0000 7f8020434740 1 -- 172.21.15.164:0/3783420291 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5647d45fef20 con 0x5647d45e2b40 2024-09-19T08:53:57.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.209+0000 7f7ff6ffd640 1 -- 172.21.15.164:0/3783420291 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8008018480 con 0x5647d45e2b40 2024-09-19T08:53:57.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.209+0000 7f7ff6ffd640 1 -- 172.21.15.164:0/3783420291 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f800c0105b0 con 0x5647d45e2b40 2024-09-19T08:53:57.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.209+0000 7f8020434740 10 client.17205 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:57.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.209+0000 7f8020434740 10 client.17205 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:53:57.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.209+0000 7f8020434740 20 client.17205 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:57.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.209+0000 7f8020434740 1 --2- 172.21.15.164:0/3783420291 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5647d4600380 0x5647d4602820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:57.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.209+0000 7f8020434740 4 client.17205 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:53:57.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.209+0000 7f8020434740 1 -- 172.21.15.164:0/3783420291 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647d4554860 con 0x5647d4600380 2024-09-19T08:53:57.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.209+0000 7f80177fe640 1 --2- 172.21.15.164:0/3783420291 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5647d4600380 0x5647d4602820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:57.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.210+0000 7f80177fe640 1 --2- 172.21.15.164:0/3783420291 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5647d4600380 0x5647d4602820 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe8ab1200cbeac21 server_cookie=86a3aeacadb5191 in_seq=0 out_seq=0 2024-09-19T08:53:57.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.210+0000 7f7ff6ffd640 10 client.17205 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:57.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.210+0000 7f7ff6ffd640 1 -- 172.21.15.164:0/3783420291 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5647d4554860 con 0x5647d4600380 2024-09-19T08:53:57.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.210+0000 7f7ff6ffd640 10 client.17205 handle_command_reply: tid=0 2024-09-19T08:53:57.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.210+0000 7f8020434740 1 client.17205 shutdown 2024-09-19T08:53:57.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.211+0000 7f8020434740 1 -- 172.21.15.164:0/3783420291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647d45e2b40 msgr2=0x5647d45ed450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:57.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.211+0000 7f8020434740 1 --2- 172.21.15.164:0/3783420291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647d45e2b40 0x5647d45ed450 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f800c012a30 tx=0x7f800c0046c0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.211+0000 7f8020434740 1 -- 172.21.15.164:0/3783420291 shutdown_connections 2024-09-19T08:53:57.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.211+0000 7f8020434740 1 --2- 172.21.15.164:0/3783420291 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647d45f0780 0x5647d45eda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.211+0000 7f8020434740 1 --2- 172.21.15.164:0/3783420291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647d45e2b40 0x5647d45ed450 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.211+0000 7f8020434740 1 --2- 172.21.15.164:0/3783420291 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5647d4600380 0x5647d4602820 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.211+0000 7f8020434740 1 --2- 172.21.15.164:0/3783420291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647d45e15d0 0x5647d45f01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.211+0000 7f7ff6ffd640 0 client.17205 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:57.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.211+0000 7f8020434740 1 -- 172.21.15.164:0/3783420291 >> 172.21.15.164:0/3783420291 conn(0x5647d45aa310 msgr2=0x5647d45e3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:57.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.211+0000 7f8020434740 1 -- 172.21.15.164:0/3783420291 shutdown_connections 2024-09-19T08:53:57.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.211+0000 7f8020434740 1 -- 172.21.15.164:0/3783420291 wait complete. 2024-09-19T08:53:57.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.211+0000 7f8020434740 20 client.17205 trim_cache size 0 max 16384 2024-09-19T08:53:57.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.224+0000 7f8020434740 1 Processor -- start 2024-09-19T08:53:57.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.224+0000 7f8020434740 1 -- start start 2024-09-19T08:53:57.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.224+0000 7f8020434740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647d45e15d0 0x5647d45dde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:57.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.224+0000 7f8020434740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647d45e2b40 0x5647d45fc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:57.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.224+0000 7f8020434740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647d45f0780 0x5647d46080f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:57.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.224+0000 7f8020434740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f80181ab560 con 0x5647d45f0780 2024-09-19T08:53:57.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.224+0000 7f8020434740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f80181ad0a0 con 0x5647d45e2b40 2024-09-19T08:53:57.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.224+0000 7f8020434740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f80181abc30 con 0x5647d45e15d0 2024-09-19T08:53:57.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.224+0000 7f8017fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647d45f0780 0x5647d46080f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:57.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.224+0000 7f8017fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647d45f0780 0x5647d46080f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40586/0 (socket says 172.21.15.164:40586) 2024-09-19T08:53:57.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.224+0000 7f8017fff640 1 -- 172.21.15.164:0/911023015 learned_addr learned my addr 172.21.15.164:0/911023015 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:57.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.224+0000 7f8016ffd640 1 --2- 172.21.15.164:0/911023015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647d45e2b40 0x5647d45fc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:57.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.224+0000 7f80177fe640 1 --2- 172.21.15.164:0/911023015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647d45e15d0 0x5647d45dde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:57.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.224+0000 7f8017fff640 1 -- 172.21.15.164:0/911023015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647d45e15d0 msgr2=0x5647d45dde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:57.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.224+0000 7f8017fff640 1 --2- 172.21.15.164:0/911023015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647d45e15d0 0x5647d45dde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.224+0000 7f8017fff640 1 -- 172.21.15.164:0/911023015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647d45e2b40 msgr2=0x5647d45fc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:57.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.224+0000 7f8017fff640 1 --2- 172.21.15.164:0/911023015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647d45e2b40 0x5647d45fc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.224+0000 7f8017fff640 1 -- 172.21.15.164:0/911023015 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647d45e4120 con 0x5647d45f0780 2024-09-19T08:53:57.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.224+0000 7f80177fe640 1 --2- 172.21.15.164:0/911023015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647d45e15d0 0x5647d45dde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:53:57.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.224+0000 7f8016ffd640 1 --2- 172.21.15.164:0/911023015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647d45e2b40 0x5647d45fc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:53:57.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.224+0000 7f8017fff640 1 --2- 172.21.15.164:0/911023015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647d45f0780 0x5647d46080f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f8010009a20 tx=0x7f801003b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9fb17b09fdfac3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:57.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.225+0000 7f7ffd7fa640 1 -- 172.21.15.164:0/911023015 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8010049050 con 0x5647d45f0780 2024-09-19T08:53:57.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.225+0000 7f7ffd7fa640 1 -- 172.21.15.164:0/911023015 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80100bc990 con 0x5647d45f0780 2024-09-19T08:53:57.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.225+0000 7f7ffd7fa640 1 -- 172.21.15.164:0/911023015 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80100ee080 con 0x5647d45f0780 2024-09-19T08:53:57.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.225+0000 7f8020434740 1 -- 172.21.15.164:0/911023015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647d45f0780 msgr2=0x5647d46080f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:57.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.225+0000 7f8020434740 1 --2- 172.21.15.164:0/911023015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647d45f0780 0x5647d46080f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f8010009a20 tx=0x7f801003b790 comp rx=0 tx=0).stop 2024-09-19T08:53:57.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.225+0000 7f8020434740 1 -- 172.21.15.164:0/911023015 shutdown_connections 2024-09-19T08:53:57.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.225+0000 7f8020434740 1 --2- 172.21.15.164:0/911023015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647d45f0780 0x5647d46080f0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.225+0000 7f8020434740 1 --2- 172.21.15.164:0/911023015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647d45e2b40 0x5647d45fc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.225+0000 7f8020434740 1 --2- 172.21.15.164:0/911023015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647d45e15d0 0x5647d45dde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.225+0000 7f8020434740 1 -- 172.21.15.164:0/911023015 >> 172.21.15.164:0/911023015 conn(0x5647d45aa310 msgr2=0x5647d45dd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:57.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.225+0000 7f8020434740 1 -- 172.21.15.164:0/911023015 shutdown_connections 2024-09-19T08:53:57.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.225+0000 7f8020434740 1 -- 172.21.15.164:0/911023015 wait complete. 2024-09-19T08:53:57.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.226+0000 7f8020434740 1 -- start start 2024-09-19T08:53:57.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.226+0000 7f8020434740 1 Processor -- start 2024-09-19T08:53:57.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.226+0000 7f8020434740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647d45e15d0 0x5647d45f9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:57.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.226+0000 7f8020434740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647d45e2b40 0x5647d45ea390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:57.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.226+0000 7f8020434740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647d45f0780 0x5647d45eab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:57.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.226+0000 7f8020434740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f80181ac9d0 con 0x5647d45f0780 2024-09-19T08:53:57.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.226+0000 7f8020434740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f80181ac300 con 0x5647d45e2b40 2024-09-19T08:53:57.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.226+0000 7f8020434740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f80181a9560 con 0x5647d45e15d0 2024-09-19T08:53:57.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.226+0000 7f8020434740 10 client.0 _ll_register_callbacks cb 0x7f800c012730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:57.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.226+0000 7f8017fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647d45f0780 0x5647d45eab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:57.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.226+0000 7f8017fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647d45f0780 0x5647d45eab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40600/0 (socket says 172.21.15.164:40600) 2024-09-19T08:53:57.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.226+0000 7f8017fff640 1 -- 172.21.15.164:0/2716470064 learned_addr learned my addr 172.21.15.164:0/2716470064 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:57.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.226+0000 7f80177fe640 1 --2- 172.21.15.164:0/2716470064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647d45e15d0 0x5647d45f9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:57.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.227+0000 7f8016ffd640 1 --2- 172.21.15.164:0/2716470064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647d45e2b40 0x5647d45ea390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:57.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.227+0000 7f8017fff640 1 -- 172.21.15.164:0/2716470064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647d45e15d0 msgr2=0x5647d45f9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:57.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.227+0000 7f8017fff640 1 --2- 172.21.15.164:0/2716470064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647d45e15d0 0x5647d45f9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.227+0000 7f8017fff640 1 -- 172.21.15.164:0/2716470064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647d45e2b40 msgr2=0x5647d45ea390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:57.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.227+0000 7f8017fff640 1 --2- 172.21.15.164:0/2716470064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647d45e2b40 0x5647d45ea390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.227+0000 7f8017fff640 1 -- 172.21.15.164:0/2716470064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647d466a0c0 con 0x5647d45f0780 2024-09-19T08:53:57.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.227+0000 7f8016ffd640 1 --2- 172.21.15.164:0/2716470064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647d45e2b40 0x5647d45ea390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:53:57.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.227+0000 7f8017fff640 1 --2- 172.21.15.164:0/2716470064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647d45f0780 0x5647d45eab20 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f80100b9360 tx=0x7f801002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f2d48f52fc030 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:57.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.227+0000 7f7ff6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:53:57.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.227+0000 7f7ff6ffd640 1 -- 172.21.15.164:0/2716470064 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80100d9020 con 0x5647d45f0780 2024-09-19T08:53:57.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.227+0000 7f7ff6ffd640 1 -- 172.21.15.164:0/2716470064 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f80100db2c0 con 0x5647d45f0780 2024-09-19T08:53:57.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.227+0000 7f8020434740 1 -- 172.21.15.164:0/2716470064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80100bc990 con 0x5647d45f0780 2024-09-19T08:53:57.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.227+0000 7f7ff6ffd640 1 -- 172.21.15.164:0/2716470064 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8010049020 con 0x5647d45f0780 2024-09-19T08:53:57.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.227+0000 7f8020434740 10 client.17211 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:57.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.227+0000 7f8020434740 1 -- 172.21.15.164:0/2716470064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5647d466b1f0 con 0x5647d45f0780 2024-09-19T08:53:57.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.227+0000 7f7ff6ffd640 1 -- 172.21.15.164:0/2716470064 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80100bc990 con 0x5647d45f0780 2024-09-19T08:53:57.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.228+0000 7f7ff6ffd640 1 -- 172.21.15.164:0/2716470064 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f8010031bd0 con 0x5647d45f0780 2024-09-19T08:53:57.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.228+0000 7f8020434740 10 client.17211 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:57.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.228+0000 7f8020434740 10 client.17211 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:53:57.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.228+0000 7f8020434740 20 client.17211 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:57.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.228+0000 7f8020434740 1 --2- 172.21.15.164:0/2716470064 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5647d466b910 0x5647d466dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:57.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.228+0000 7f8020434740 4 client.17211 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:53:57.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.228+0000 7f8020434740 1 -- 172.21.15.164:0/2716470064 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5647d454ed70 con 0x5647d466b910 2024-09-19T08:53:57.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.228+0000 7f80177fe640 1 --2- 172.21.15.164:0/2716470064 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5647d466b910 0x5647d466dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:57.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.228+0000 7f7ff6ffd640 10 client.17211 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:57.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.228+0000 7f80177fe640 1 --2- 172.21.15.164:0/2716470064 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5647d466b910 0x5647d466dd30 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdd4b9fb3aad4bb4 server_cookie=c5fa9499f47d1e15 in_seq=0 out_seq=0 2024-09-19T08:53:57.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.228+0000 7f7ff6ffd640 1 -- 172.21.15.164:0/2716470064 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5647d454ed70 con 0x5647d466b910 2024-09-19T08:53:57.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.228+0000 7f7ff6ffd640 10 client.17211 handle_command_reply: tid=0 2024-09-19T08:53:57.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.228+0000 7f8020434740 1 client.17211 shutdown 2024-09-19T08:53:57.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.229+0000 7f8020434740 1 -- 172.21.15.164:0/2716470064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647d45f0780 msgr2=0x5647d45eab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:57.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.229+0000 7f8020434740 1 --2- 172.21.15.164:0/2716470064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647d45f0780 0x5647d45eab20 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f80100b9360 tx=0x7f801002a410 comp rx=0 tx=0).stop 2024-09-19T08:53:57.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.229+0000 7f8020434740 1 -- 172.21.15.164:0/2716470064 shutdown_connections 2024-09-19T08:53:57.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.229+0000 7f8020434740 1 --2- 172.21.15.164:0/2716470064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5647d45f0780 0x5647d45eab20 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.229+0000 7f8020434740 1 --2- 172.21.15.164:0/2716470064 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5647d466b910 0x5647d466dd30 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.229+0000 7f8020434740 1 --2- 172.21.15.164:0/2716470064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647d45e2b40 0x5647d45ea390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.229+0000 7f7ff6ffd640 0 client.17211 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:57.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.229+0000 7f8020434740 1 --2- 172.21.15.164:0/2716470064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5647d45e15d0 0x5647d45f9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.229+0000 7f8020434740 1 -- 172.21.15.164:0/2716470064 >> 172.21.15.164:0/2716470064 conn(0x5647d45aa310 msgr2=0x5647d45dc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:57.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.229+0000 7f8020434740 1 -- 172.21.15.164:0/2716470064 shutdown_connections 2024-09-19T08:53:57.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.229+0000 7f8020434740 1 -- 172.21.15.164:0/2716470064 wait complete. 2024-09-19T08:53:57.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.229+0000 7f8020434740 20 client.17211 trim_cache size 0 max 16384 2024-09-19T08:53:57.241 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:53:57.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.229+0000 7f801e05b640 1 -- 172.21.15.164:0/1157676045 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f80000a33b0 msgr2=0x7f80000a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:57.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.229+0000 7f801e05b640 1 --2- 172.21.15.164:0/1157676045 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f80000a33b0 0x7f80000a5870 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x5647d45deac0 tx=0x7f800c00b450 comp rx=0 tx=0).stop 2024-09-19T08:53:57.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.229+0000 7f801e05b640 1 -- 172.21.15.164:0/1157676045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8018106f20 msgr2=0x7f801819cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:57.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.229+0000 7f801e05b640 1 --2- 172.21.15.164:0/1157676045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8018106f20 0x7f801819cd70 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f8010009230 tx=0x7f80100042f0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.230+0000 7f801e05b640 1 -- 172.21.15.164:0/1157676045 shutdown_connections 2024-09-19T08:53:57.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.230+0000 7f801e05b640 1 --2- 172.21.15.164:0/1157676045 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f80000a33b0 0x7f80000a5870 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.230+0000 7f801e05b640 1 --2- 172.21.15.164:0/1157676045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8018106f20 0x7f801819cd70 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.230+0000 7f801e05b640 1 --2- 172.21.15.164:0/1157676045 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80181027a0 0x7f80181989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.230+0000 7f801e05b640 1 --2- 172.21.15.164:0/1157676045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8018101df0 0x7f80181984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:57.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.230+0000 7f801e05b640 1 -- 172.21.15.164:0/1157676045 >> 172.21.15.164:0/1157676045 conn(0x7f80180fd8b0 msgr2=0x7f80180febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:57.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.230+0000 7f801e05b640 1 -- 172.21.15.164:0/1157676045 shutdown_connections 2024-09-19T08:53:57.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:57.230+0000 7f801e05b640 1 -- 172.21.15.164:0/1157676045 wait complete. 2024-09-19T08:53:58.283 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:57 smithi164 ceph-mon[30928]: pgmap v251: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 op/s 2024-09-19T08:53:58.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:57 smithi170 ceph-mon[35072]: pgmap v251: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 op/s 2024-09-19T08:53:58.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:57 smithi173 ceph-mon[35306]: pgmap v251: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 op/s 2024-09-19T08:53:58.945 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T08:53:58.945 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T08:53:59.203 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:53:59.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.669+0000 7f6f6d4da640 1 -- 172.21.15.164:0/1992308861 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f681036e0 msgr2=0x7f6f68107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:59.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.669+0000 7f6f6d4da640 1 --2- 172.21.15.164:0/1992308861 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f681036e0 0x7f6f68107b90 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c009930 tx=0x7f6f5c02a420 comp rx=0 tx=0).stop 2024-09-19T08:53:59.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.669+0000 7f6f6d4da640 1 -- 172.21.15.164:0/1992308861 shutdown_connections 2024-09-19T08:53:59.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.669+0000 7f6f6d4da640 1 --2- 172.21.15.164:0/1992308861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f681082a0 0x7f6f6810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.669+0000 7f6f6d4da640 1 --2- 172.21.15.164:0/1992308861 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f681036e0 0x7f6f68107b90 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.669+0000 7f6f6d4da640 1 --2- 172.21.15.164:0/1992308861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f68102d30 0x7f6f68103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.669+0000 7f6f6d4da640 1 -- 172.21.15.164:0/1992308861 >> 172.21.15.164:0/1992308861 conn(0x7f6f680fc7d0 msgr2=0x7f6f680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:59.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.669+0000 7f6f6d4da640 1 -- 172.21.15.164:0/1992308861 shutdown_connections 2024-09-19T08:53:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.669+0000 7f6f6d4da640 1 -- 172.21.15.164:0/1992308861 wait complete. 2024-09-19T08:53:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.669+0000 7f6f6d4da640 1 Processor -- start 2024-09-19T08:53:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.669+0000 7f6f6d4da640 1 -- start start 2024-09-19T08:53:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.670+0000 7f6f6d4da640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f68102d30 0x7f6f681985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.670+0000 7f6f6d4da640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f681036e0 0x7f6f68198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.670+0000 7f6f6d4da640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f681082a0 0x7f6f6819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.670+0000 7f6f6d4da640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6f681120c0 con 0x7f6f681082a0 2024-09-19T08:53:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.670+0000 7f6f6d4da640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6f68111f40 con 0x7f6f681036e0 2024-09-19T08:53:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.670+0000 7f6f6d4da640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6f68112240 con 0x7f6f68102d30 2024-09-19T08:53:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.670+0000 7f6f677fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f681082a0 0x7f6f6819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.670+0000 7f6f677fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f681082a0 0x7f6f6819ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40618/0 (socket says 172.21.15.164:40618) 2024-09-19T08:53:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.670+0000 7f6f677fe640 1 -- 172.21.15.164:0/3352455423 learned_addr learned my addr 172.21.15.164:0/3352455423 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:59.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.670+0000 7f6f667fc640 1 --2- 172.21.15.164:0/3352455423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f681036e0 0x7f6f68198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:59.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.670+0000 7f6f66ffd640 1 --2- 172.21.15.164:0/3352455423 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f68102d30 0x7f6f681985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:59.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.670+0000 7f6f677fe640 1 -- 172.21.15.164:0/3352455423 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f68102d30 msgr2=0x7f6f681985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:59.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.670+0000 7f6f677fe640 1 --2- 172.21.15.164:0/3352455423 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f68102d30 0x7f6f681985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.670+0000 7f6f677fe640 1 -- 172.21.15.164:0/3352455423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f681036e0 msgr2=0x7f6f68198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:59.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.670+0000 7f6f677fe640 1 --2- 172.21.15.164:0/3352455423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f681036e0 0x7f6f68198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.670+0000 7f6f677fe640 1 -- 172.21.15.164:0/3352455423 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f6819d590 con 0x7f6f681082a0 2024-09-19T08:53:59.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.670+0000 7f6f66ffd640 1 --2- 172.21.15.164:0/3352455423 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f68102d30 0x7f6f681985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:53:59.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.670+0000 7f6f677fe640 1 --2- 172.21.15.164:0/3352455423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f681082a0 0x7f6f6819ceb0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f6f6000c970 tx=0x7f6f6000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:59.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.670+0000 7f6f4ffff640 1 -- 172.21.15.164:0/3352455423 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60007bb0 con 0x7f6f681082a0 2024-09-19T08:53:59.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.671+0000 7f6f6d4da640 1 -- 172.21.15.164:0/3352455423 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f6819d880 con 0x7f6f681082a0 2024-09-19T08:53:59.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.671+0000 7f6f4ffff640 1 -- 172.21.15.164:0/3352455423 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f60007d50 con 0x7f6f681082a0 2024-09-19T08:53:59.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.671+0000 7f6f4ffff640 1 -- 172.21.15.164:0/3352455423 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60018800 con 0x7f6f681082a0 2024-09-19T08:53:59.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.671+0000 7f6f6d4da640 1 -- 172.21.15.164:0/3352455423 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f681a5160 con 0x7f6f681082a0 2024-09-19T08:53:59.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.673+0000 7f6f4ffff640 1 -- 172.21.15.164:0/3352455423 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f6f6000fa50 con 0x7f6f681082a0 2024-09-19T08:53:59.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.673+0000 7f6f6f8b3740 1 Processor -- start 2024-09-19T08:53:59.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.673+0000 7f6f6f8b3740 1 -- start start 2024-09-19T08:53:59.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.673+0000 7f6f6f8b3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561382b78f70 0x561382b79350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:59.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.673+0000 7f6f6f8b3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561382b79890 0x561382b83c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:59.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.673+0000 7f6f6f8b3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561382b84150 0x561382b86540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:59.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.673+0000 7f6f6f8b3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561382ada280 con 0x561382b79890 2024-09-19T08:53:59.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.673+0000 7f6f6f8b3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561382a81b10 con 0x561382b84150 2024-09-19T08:53:59.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.673+0000 7f6f6f8b3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x561382ad6240 con 0x561382b78f70 2024-09-19T08:53:59.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.674+0000 7f6f66ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561382b79890 0x561382b83c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:59.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.674+0000 7f6f4ffff640 1 --2- 172.21.15.164:0/3352455423 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f6f500a33b0 0x7f6f500a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:59.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.674+0000 7f6f66ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561382b79890 0x561382b83c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40628/0 (socket says 172.21.15.164:40628) 2024-09-19T08:53:59.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.674+0000 7f6f66ffd640 1 -- 172.21.15.164:0/1235689907 learned_addr learned my addr 172.21.15.164:0/1235689907 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:59.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.674+0000 7f6f667fc640 1 --2- 172.21.15.164:0/1235689907 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561382b78f70 0x561382b79350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:59.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.674+0000 7f6f677fe640 1 --2- 172.21.15.164:0/3352455423 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f6f500a33b0 0x7f6f500a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:59.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.674+0000 7f6f667fc640 1 --2- 172.21.15.164:0/1235689907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561382b84150 0x561382b86540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:59.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.674+0000 7f6f66ffd640 1 -- 172.21.15.164:0/1235689907 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561382b78f70 msgr2=0x561382b79350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:59.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.674+0000 7f6f66ffd640 1 --2- 172.21.15.164:0/1235689907 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561382b78f70 0x561382b79350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.674+0000 7f6f66ffd640 1 -- 172.21.15.164:0/1235689907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561382b84150 msgr2=0x561382b86540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:59.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.674+0000 7f6f66ffd640 1 --2- 172.21.15.164:0/1235689907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561382b84150 0x561382b86540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.674+0000 7f6f66ffd640 1 -- 172.21.15.164:0/1235689907 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561382a8e1f0 con 0x561382b79890 2024-09-19T08:53:59.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.674+0000 7f6f667fc640 1 --2- 172.21.15.164:0/1235689907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561382b84150 0x561382b86540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:53:59.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.674+0000 7f6f4ffff640 1 -- 172.21.15.164:0/3352455423 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f6f600cfde0 con 0x7f6f681082a0 2024-09-19T08:53:59.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.674+0000 7f6f66ffd640 1 --2- 172.21.15.164:0/1235689907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561382b79890 0x561382b83c10 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f6f58007fb0 tx=0x7f6f5800c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a29ddc41cad6b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:59.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.674+0000 7f6f677fe640 1 --2- 172.21.15.164:0/3352455423 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f6f500a33b0 0x7f6f500a5870 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6f6009c090 tx=0x7f6f6006de80 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:59.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.674+0000 7f6f4d7fa640 1 -- 172.21.15.164:0/1235689907 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f58014070 con 0x561382b79890 2024-09-19T08:53:59.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.674+0000 7f6f4d7fa640 1 -- 172.21.15.164:0/1235689907 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f5800fd30 con 0x561382b79890 2024-09-19T08:53:59.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.674+0000 7f6f4d7fa640 1 -- 172.21.15.164:0/1235689907 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5800ed70 con 0x561382b79890 2024-09-19T08:53:59.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.675+0000 7f6f6f8b3740 1 -- 172.21.15.164:0/1235689907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561382b79890 msgr2=0x561382b83c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:59.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.675+0000 7f6f6f8b3740 1 --2- 172.21.15.164:0/1235689907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561382b79890 0x561382b83c10 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f6f58007fb0 tx=0x7f6f5800c750 comp rx=0 tx=0).stop 2024-09-19T08:53:59.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.675+0000 7f6f6f8b3740 1 -- 172.21.15.164:0/1235689907 shutdown_connections 2024-09-19T08:53:59.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.675+0000 7f6f6f8b3740 1 --2- 172.21.15.164:0/1235689907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561382b84150 0x561382b86540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.675+0000 7f6f6f8b3740 1 --2- 172.21.15.164:0/1235689907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561382b79890 0x561382b83c10 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.675+0000 7f6f6f8b3740 1 --2- 172.21.15.164:0/1235689907 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561382b78f70 0x561382b79350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.675+0000 7f6f6f8b3740 1 -- 172.21.15.164:0/1235689907 >> 172.21.15.164:0/1235689907 conn(0x561382b75500 msgr2=0x561382b758e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:59.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.675+0000 7f6f6f8b3740 1 -- 172.21.15.164:0/1235689907 shutdown_connections 2024-09-19T08:53:59.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.675+0000 7f6f6f8b3740 1 -- 172.21.15.164:0/1235689907 wait complete. 2024-09-19T08:53:59.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.676+0000 7f6f6f8b3740 1 -- start start 2024-09-19T08:53:59.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.676+0000 7f6f6f8b3740 1 Processor -- start 2024-09-19T08:53:59.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.676+0000 7f6f6f8b3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561382b78f70 0x561382b80f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:59.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.676+0000 7f6f6f8b3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561382b79890 0x561382b814b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:59.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.676+0000 7f6f6f8b3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561382b84150 0x561382b819f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:59.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.676+0000 7f6f6f8b3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561382b86d20 con 0x561382b84150 2024-09-19T08:53:59.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.676+0000 7f6f6f8b3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561382a94f10 con 0x561382b78f70 2024-09-19T08:53:59.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.676+0000 7f6f6f8b3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x561382b86ea0 con 0x561382b79890 2024-09-19T08:53:59.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.676+0000 7f6f6f8b3740 10 client.0 _ll_register_callbacks cb 0x561382a5da40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:59.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.676+0000 7f6f667fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561382b84150 0x561382b819f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:59.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.676+0000 7f6f667fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561382b84150 0x561382b819f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40634/0 (socket says 172.21.15.164:40634) 2024-09-19T08:53:59.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.676+0000 7f6f667fc640 1 -- 172.21.15.164:0/3866920131 learned_addr learned my addr 172.21.15.164:0/3866920131 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:59.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.676+0000 7f6f667fc640 1 --2- 172.21.15.164:0/3866920131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561382b78f70 0x561382b80f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:59.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.676+0000 7f6f66ffd640 1 --2- 172.21.15.164:0/3866920131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561382b79890 0x561382b814b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:59.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.676+0000 7f6f667fc640 1 -- 172.21.15.164:0/3866920131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561382b79890 msgr2=0x561382b814b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:59.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.676+0000 7f6f667fc640 1 --2- 172.21.15.164:0/3866920131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561382b79890 0x561382b814b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.676+0000 7f6f667fc640 1 -- 172.21.15.164:0/3866920131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561382b78f70 msgr2=0x561382b80f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:59.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.676+0000 7f6f667fc640 1 --2- 172.21.15.164:0/3866920131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561382b78f70 0x561382b80f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.676+0000 7f6f667fc640 1 -- 172.21.15.164:0/3866920131 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56138284b6a0 con 0x561382b84150 2024-09-19T08:53:59.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.677+0000 7f6f667fc640 1 --2- 172.21.15.164:0/3866920131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561382b84150 0x561382b819f0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c005940 tx=0x7f6f5c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf00ffd259360f2c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:59.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.677+0000 7f6f4aff5640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:53:59.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.677+0000 7f6f4aff5640 1 -- 172.21.15.164:0/3866920131 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c035040 con 0x561382b84150 2024-09-19T08:53:59.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.677+0000 7f6f4aff5640 1 -- 172.21.15.164:0/3866920131 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f6f5c0035f0 con 0x561382b84150 2024-09-19T08:53:59.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.677+0000 7f6f6f8b3740 1 -- 172.21.15.164:0/3866920131 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f5800fd30 con 0x561382b84150 2024-09-19T08:53:59.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.677+0000 7f6f4aff5640 1 -- 172.21.15.164:0/3866920131 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6f5c0037f0 con 0x561382b84150 2024-09-19T08:53:59.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.677+0000 7f6f6f8b3740 10 client.17223 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:59.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.677+0000 7f6f6f8b3740 1 -- 172.21.15.164:0/3866920131 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561382b927e0 con 0x561382b84150 2024-09-19T08:53:59.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.677+0000 7f6f4aff5640 1 -- 172.21.15.164:0/3866920131 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f5800fd30 con 0x561382b84150 2024-09-19T08:53:59.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.677+0000 7f6f4aff5640 1 -- 172.21.15.164:0/3866920131 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f6f5c03a040 con 0x561382b84150 2024-09-19T08:53:59.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.677+0000 7f6f6f8b3740 10 client.17223 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:59.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.677+0000 7f6f6f8b3740 10 client.17223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:53:59.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.677+0000 7f6f6f8b3740 20 client.17223 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:59.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.677+0000 7f6f6f8b3740 1 --2- 172.21.15.164:0/3866920131 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x561382b94020 0x561382b96450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:59.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.677+0000 7f6f6f8b3740 4 client.17223 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:53:59.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.677+0000 7f6f6f8b3740 1 -- 172.21.15.164:0/3866920131 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561382ae77f0 con 0x561382b94020 2024-09-19T08:53:59.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.678+0000 7f6f66ffd640 1 --2- 172.21.15.164:0/3866920131 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x561382b94020 0x561382b96450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:59.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.678+0000 7f6f66ffd640 1 --2- 172.21.15.164:0/3866920131 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x561382b94020 0x561382b96450 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ebe3d01bc3816f4 server_cookie=e083d5d15c8979d8 in_seq=0 out_seq=0 2024-09-19T08:53:59.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.678+0000 7f6f4aff5640 10 client.17223 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:59.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.679+0000 7f6f4aff5640 1 -- 172.21.15.164:0/3866920131 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561382ae77f0 con 0x561382b94020 2024-09-19T08:53:59.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.679+0000 7f6f4aff5640 10 client.17223 handle_command_reply: tid=0 2024-09-19T08:53:59.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.679+0000 7f6f6f8b3740 1 client.17223 shutdown 2024-09-19T08:53:59.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.679+0000 7f6f6f8b3740 1 -- 172.21.15.164:0/3866920131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561382b84150 msgr2=0x561382b819f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:59.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.679+0000 7f6f6f8b3740 1 --2- 172.21.15.164:0/3866920131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561382b84150 0x561382b819f0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c005940 tx=0x7f6f5c009260 comp rx=0 tx=0).stop 2024-09-19T08:53:59.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.679+0000 7f6f6f8b3740 1 -- 172.21.15.164:0/3866920131 shutdown_connections 2024-09-19T08:53:59.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.679+0000 7f6f6f8b3740 1 --2- 172.21.15.164:0/3866920131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561382b84150 0x561382b819f0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.679+0000 7f6f6f8b3740 1 --2- 172.21.15.164:0/3866920131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561382b79890 0x561382b814b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.679+0000 7f6f6f8b3740 1 --2- 172.21.15.164:0/3866920131 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x561382b94020 0x561382b96450 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.679+0000 7f6f6f8b3740 1 --2- 172.21.15.164:0/3866920131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561382b78f70 0x561382b80f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.679+0000 7f6f6f8b3740 1 -- 172.21.15.164:0/3866920131 >> 172.21.15.164:0/3866920131 conn(0x561382b75500 msgr2=0x561382b70270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:59.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.679+0000 7f6f4aff5640 0 client.17223 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:59.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.679+0000 7f6f6f8b3740 1 -- 172.21.15.164:0/3866920131 shutdown_connections 2024-09-19T08:53:59.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.679+0000 7f6f6f8b3740 1 -- 172.21.15.164:0/3866920131 wait complete. 2024-09-19T08:53:59.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.679+0000 7f6f6f8b3740 20 client.17223 trim_cache size 0 max 16384 2024-09-19T08:53:59.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.696+0000 7f6f6f8b3740 1 Processor -- start 2024-09-19T08:53:59.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.696+0000 7f6f6f8b3740 1 -- start start 2024-09-19T08:53:59.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.696+0000 7f6f6f8b3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561382b78f70 0x561382b7b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:59.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.696+0000 7f6f6f8b3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561382b79890 0x561382b70160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:59.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.696+0000 7f6f6f8b3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561382b84150 0x561382b9bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:59.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.696+0000 7f6f6f8b3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6f681ab500 con 0x561382b84150 2024-09-19T08:53:59.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.696+0000 7f6f6f8b3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6f681ad040 con 0x561382b79890 2024-09-19T08:53:59.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.696+0000 7f6f6f8b3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6f681abbd0 con 0x561382b78f70 2024-09-19T08:53:59.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.696+0000 7f6f667fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561382b84150 0x561382b9bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:59.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.697+0000 7f6f667fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561382b84150 0x561382b9bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40644/0 (socket says 172.21.15.164:40644) 2024-09-19T08:53:59.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.697+0000 7f6f667fc640 1 -- 172.21.15.164:0/1695000635 learned_addr learned my addr 172.21.15.164:0/1695000635 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:59.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.697+0000 7f6f66ffd640 1 --2- 172.21.15.164:0/1695000635 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561382b79890 0x561382b70160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:59.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.697+0000 7f6f667fc640 1 --2- 172.21.15.164:0/1695000635 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561382b78f70 0x561382b7b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:59.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.697+0000 7f6f667fc640 1 -- 172.21.15.164:0/1695000635 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561382b78f70 msgr2=0x561382b7b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:59.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.697+0000 7f6f667fc640 1 --2- 172.21.15.164:0/1695000635 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561382b78f70 0x561382b7b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.697+0000 7f6f667fc640 1 -- 172.21.15.164:0/1695000635 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561382b79890 msgr2=0x561382b70160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:59.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.697+0000 7f6f667fc640 1 --2- 172.21.15.164:0/1695000635 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561382b79890 0x561382b70160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.697+0000 7f6f667fc640 1 -- 172.21.15.164:0/1695000635 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561382b6c830 con 0x561382b84150 2024-09-19T08:53:59.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.697+0000 7f6f667fc640 1 --2- 172.21.15.164:0/1695000635 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561382b84150 0x561382b9bd60 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c005910 tx=0x7f6f5c02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3948ff27e9abf79f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:59.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.697+0000 7f6f4cff9640 1 -- 172.21.15.164:0/1695000635 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c035070 con 0x561382b84150 2024-09-19T08:53:59.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.697+0000 7f6f4cff9640 1 -- 172.21.15.164:0/1695000635 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f5c03d930 con 0x561382b84150 2024-09-19T08:53:59.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.697+0000 7f6f4cff9640 1 -- 172.21.15.164:0/1695000635 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c003da0 con 0x561382b84150 2024-09-19T08:53:59.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.698+0000 7f6f6f8b3740 1 -- 172.21.15.164:0/1695000635 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561382b84150 msgr2=0x561382b9bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:59.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.698+0000 7f6f6f8b3740 1 --2- 172.21.15.164:0/1695000635 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561382b84150 0x561382b9bd60 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c005910 tx=0x7f6f5c02a420 comp rx=0 tx=0).stop 2024-09-19T08:53:59.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.698+0000 7f6f6f8b3740 1 -- 172.21.15.164:0/1695000635 shutdown_connections 2024-09-19T08:53:59.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.698+0000 7f6f6f8b3740 1 --2- 172.21.15.164:0/1695000635 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561382b84150 0x561382b9bd60 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.698+0000 7f6f6f8b3740 1 --2- 172.21.15.164:0/1695000635 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561382b79890 0x561382b70160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.698+0000 7f6f6f8b3740 1 --2- 172.21.15.164:0/1695000635 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561382b78f70 0x561382b7b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.698+0000 7f6f6f8b3740 1 -- 172.21.15.164:0/1695000635 >> 172.21.15.164:0/1695000635 conn(0x561382b75500 msgr2=0x561382b6fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:59.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.698+0000 7f6f6f8b3740 1 -- 172.21.15.164:0/1695000635 shutdown_connections 2024-09-19T08:53:59.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.698+0000 7f6f6f8b3740 1 -- 172.21.15.164:0/1695000635 wait complete. 2024-09-19T08:53:59.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.698+0000 7f6f6f8b3740 1 -- start start 2024-09-19T08:53:59.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.698+0000 7f6f6f8b3740 1 Processor -- start 2024-09-19T08:53:59.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.698+0000 7f6f6f8b3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561382b78f70 0x561382b8c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:59.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.698+0000 7f6f6f8b3740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561382b79890 0x561382b7ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:59.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.698+0000 7f6f6f8b3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561382b84150 0x561382b7e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:59.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.698+0000 7f6f6f8b3740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6f681ac970 con 0x561382b79890 2024-09-19T08:53:59.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.698+0000 7f6f6f8b3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6f681ac2a0 con 0x561382b78f70 2024-09-19T08:53:59.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.698+0000 7f6f6f8b3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6f681a9520 con 0x561382b84150 2024-09-19T08:53:59.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.698+0000 7f6f6f8b3740 10 client.0 _ll_register_callbacks cb 0x7f6f5c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:53:59.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.698+0000 7f6f66ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561382b79890 0x561382b7ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:59.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.698+0000 7f6f66ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561382b79890 0x561382b7ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40646/0 (socket says 172.21.15.164:40646) 2024-09-19T08:53:59.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.698+0000 7f6f66ffd640 1 -- 172.21.15.164:0/3427128535 learned_addr learned my addr 172.21.15.164:0/3427128535 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:53:59.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.698+0000 7f6f667fc640 1 --2- 172.21.15.164:0/3427128535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561382b78f70 0x561382b8c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:59.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.699+0000 7f6f66ffd640 1 -- 172.21.15.164:0/3427128535 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561382b84150 msgr2=0x561382b7e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:59.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.699+0000 7f6f66ffd640 1 --2- 172.21.15.164:0/3427128535 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561382b84150 0x561382b7e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.699+0000 7f6f66ffd640 1 -- 172.21.15.164:0/3427128535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561382b78f70 msgr2=0x561382b8c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:53:59.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.699+0000 7f6f66ffd640 1 --2- 172.21.15.164:0/3427128535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561382b78f70 0x561382b8c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.699+0000 7f6f66ffd640 1 -- 172.21.15.164:0/3427128535 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561382b8a420 con 0x561382b79890 2024-09-19T08:53:59.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.699+0000 7f6f66ffd640 1 --2- 172.21.15.164:0/3427128535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561382b79890 0x561382b7ddd0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f6f58007fb0 tx=0x7f6f58007300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b951ab0a5c35ca5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:53:59.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.699+0000 7f6f4aff5640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:53:59.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.699+0000 7f6f4aff5640 1 -- 172.21.15.164:0/3427128535 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f58014040 con 0x561382b79890 2024-09-19T08:53:59.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.699+0000 7f6f4aff5640 1 -- 172.21.15.164:0/3427128535 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f6f5800e9c0 con 0x561382b79890 2024-09-19T08:53:59.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.699+0000 7f6f6f8b3740 1 -- 172.21.15.164:0/3427128535 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f5c03d930 con 0x561382b79890 2024-09-19T08:53:59.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.699+0000 7f6f4aff5640 1 -- 172.21.15.164:0/3427128535 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6f5800e7b0 con 0x561382b79890 2024-09-19T08:53:59.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.699+0000 7f6f6f8b3740 10 client.17229 fetch_fsmap learned FSMap version 32 2024-09-19T08:53:59.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.699+0000 7f6f6f8b3740 1 -- 172.21.15.164:0/3427128535 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561382bfef10 con 0x561382b79890 2024-09-19T08:53:59.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.699+0000 7f6f4aff5640 1 -- 172.21.15.164:0/3427128535 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f5c03d930 con 0x561382b79890 2024-09-19T08:53:59.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.700+0000 7f6f4aff5640 1 -- 172.21.15.164:0/3427128535 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f6f580194e0 con 0x561382b79890 2024-09-19T08:53:59.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.700+0000 7f6f6f8b3740 10 client.17229 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:53:59.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.700+0000 7f6f6f8b3740 10 client.17229 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:53:59.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.700+0000 7f6f6f8b3740 20 client.17229 populate_metadata read hostname 'smithi164' 2024-09-19T08:53:59.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.700+0000 7f6f6f8b3740 1 --2- 172.21.15.164:0/3427128535 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x561382bff380 0x561382c017a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:53:59.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.700+0000 7f6f6f8b3740 4 client.17229 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T08:53:59.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.700+0000 7f6f6f8b3740 1 -- 172.21.15.164:0/3427128535 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561382ae1d00 con 0x561382bff380 2024-09-19T08:53:59.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.700+0000 7f6f667fc640 1 --2- 172.21.15.164:0/3427128535 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x561382bff380 0x561382c017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:53:59.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.700+0000 7f6f4aff5640 10 client.17229 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:59.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.700+0000 7f6f667fc640 1 --2- 172.21.15.164:0/3427128535 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x561382bff380 0x561382c017a0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=add14b2957f8b5fb server_cookie=1966ecdd82f62c3c in_seq=0 out_seq=0 2024-09-19T08:53:59.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.701+0000 7f6f4aff5640 1 -- 172.21.15.164:0/3427128535 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561382ae1d00 con 0x561382bff380 2024-09-19T08:53:59.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.701+0000 7f6f4aff5640 10 client.17229 handle_command_reply: tid=0 2024-09-19T08:53:59.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.701+0000 7f6f6f8b3740 1 client.17229 shutdown 2024-09-19T08:53:59.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.702+0000 7f6f6f8b3740 1 -- 172.21.15.164:0/3427128535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561382b79890 msgr2=0x561382b7ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:59.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.702+0000 7f6f6f8b3740 1 --2- 172.21.15.164:0/3427128535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561382b79890 0x561382b7ddd0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f6f58007fb0 tx=0x7f6f58007300 comp rx=0 tx=0).stop 2024-09-19T08:53:59.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.702+0000 7f6f6f8b3740 1 -- 172.21.15.164:0/3427128535 shutdown_connections 2024-09-19T08:53:59.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.702+0000 7f6f6f8b3740 1 --2- 172.21.15.164:0/3427128535 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561382b84150 0x561382b7e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.702+0000 7f6f6f8b3740 1 --2- 172.21.15.164:0/3427128535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561382b79890 0x561382b7ddd0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.702+0000 7f6f6f8b3740 1 --2- 172.21.15.164:0/3427128535 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x561382bff380 0x561382c017a0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.702+0000 7f6f6f8b3740 1 --2- 172.21.15.164:0/3427128535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561382b78f70 0x561382b8c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.702+0000 7f6f6f8b3740 1 -- 172.21.15.164:0/3427128535 >> 172.21.15.164:0/3427128535 conn(0x561382b75500 msgr2=0x561382b70270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:59.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.702+0000 7f6f4aff5640 0 client.17229 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:53:59.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.702+0000 7f6f6f8b3740 1 -- 172.21.15.164:0/3427128535 shutdown_connections 2024-09-19T08:53:59.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.703+0000 7f6f6f8b3740 1 -- 172.21.15.164:0/3427128535 wait complete. 2024-09-19T08:53:59.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.703+0000 7f6f6f8b3740 20 client.17229 trim_cache size 0 max 16384 2024-09-19T08:53:59.721 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:53:59.722 INFO:teuthology.orchestra.run.smithi164.stdout: "return_code": 0, 2024-09-19T08:53:59.722 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_tag": "79f876fe-21e9-4a8d-8ab0-5e1265f85164", 2024-09-19T08:53:59.722 INFO:teuthology.orchestra.run.smithi164.stdout: "mode": "asynchronous" 2024-09-19T08:53:59.722 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:53:59.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.703+0000 7f6f6d4da640 1 -- 172.21.15.164:0/3352455423 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f6f500a33b0 msgr2=0x7f6f500a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:59.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.704+0000 7f6f6d4da640 1 --2- 172.21.15.164:0/3352455423 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f6f500a33b0 0x7f6f500a5870 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6f6009c090 tx=0x7f6f6006de80 comp rx=0 tx=0).stop 2024-09-19T08:53:59.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.704+0000 7f6f6d4da640 1 -- 172.21.15.164:0/3352455423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f681082a0 msgr2=0x7f6f6819ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:53:59.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.704+0000 7f6f6d4da640 1 --2- 172.21.15.164:0/3352455423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f681082a0 0x7f6f6819ceb0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f6f6000c970 tx=0x7f6f6000ce30 comp rx=0 tx=0).stop 2024-09-19T08:53:59.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.704+0000 7f6f6d4da640 1 -- 172.21.15.164:0/3352455423 shutdown_connections 2024-09-19T08:53:59.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.704+0000 7f6f6d4da640 1 --2- 172.21.15.164:0/3352455423 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f6f500a33b0 0x7f6f500a5870 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.704+0000 7f6f6d4da640 1 --2- 172.21.15.164:0/3352455423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f681082a0 0x7f6f6819ceb0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.704+0000 7f6f6d4da640 1 --2- 172.21.15.164:0/3352455423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f681036e0 0x7f6f68198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.704+0000 7f6f6d4da640 1 --2- 172.21.15.164:0/3352455423 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f68102d30 0x7f6f681985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:53:59.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.704+0000 7f6f6d4da640 1 -- 172.21.15.164:0/3352455423 >> 172.21.15.164:0/3352455423 conn(0x7f6f680fc7d0 msgr2=0x7f6f68105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:53:59.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.704+0000 7f6f6d4da640 1 -- 172.21.15.164:0/3352455423 shutdown_connections 2024-09-19T08:53:59.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:53:59.704+0000 7f6f6d4da640 1 -- 172.21.15.164:0/3352455423 wait complete. 2024-09-19T08:54:00.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:59 smithi164 ceph-mon[30928]: pgmap v252: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:54:00.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:59 smithi164 ceph-mon[30928]: scrub summary: active paths [/] 2024-09-19T08:54:00.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:59 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:54:00.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:59 smithi164 ceph-mon[30928]: scrub summary: active paths [/] 2024-09-19T08:54:00.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:53:59 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:54:00.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:59 smithi170 ceph-mon[35072]: pgmap v252: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:54:00.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:59 smithi170 ceph-mon[35072]: scrub summary: active paths [/] 2024-09-19T08:54:00.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:59 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:54:00.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:59 smithi170 ceph-mon[35072]: scrub summary: active paths [/] 2024-09-19T08:54:00.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:53:59 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:54:00.410 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:54:00.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:59 smithi173 ceph-mon[35306]: pgmap v252: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:54:00.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:59 smithi173 ceph-mon[35306]: scrub summary: active paths [/] 2024-09-19T08:54:00.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:59 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:54:00.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:59 smithi173 ceph-mon[35306]: scrub summary: active paths [/] 2024-09-19T08:54:00.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:53:59 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:54:00.663 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:54:01.078 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:00 smithi164 ceph-mon[30928]: scrub queued for path: / 2024-09-19T08:54:01.078 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:00 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:54:01.078 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:00 smithi164 ceph-mon[30928]: scrub summary: active paths [/] 2024-09-19T08:54:01.078 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:00 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:54:01.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.160+0000 7f5748f07640 1 -- 172.21.15.164:0/978763213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5744100e20 msgr2=0x7f5744101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:01.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.160+0000 7f5748f07640 1 --2- 172.21.15.164:0/978763213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5744100e20 0x7f5744101200 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f5734009900 tx=0x7f573402a480 comp rx=0 tx=0).stop 2024-09-19T08:54:01.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.160+0000 7f5748f07640 1 -- 172.21.15.164:0/978763213 shutdown_connections 2024-09-19T08:54:01.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.160+0000 7f5748f07640 1 --2- 172.21.15.164:0/978763213 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5744102ef0 0x7f574410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:01.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.160+0000 7f5748f07640 1 --2- 172.21.15.164:0/978763213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5744102570 0x7f57441029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:01.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.160+0000 7f5748f07640 1 --2- 172.21.15.164:0/978763213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5744100e20 0x7f5744101200 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:01.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.160+0000 7f5748f07640 1 -- 172.21.15.164:0/978763213 >> 172.21.15.164:0/978763213 conn(0x7f57440fc7d0 msgr2=0x7f57440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:01.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.160+0000 7f5748f07640 1 -- 172.21.15.164:0/978763213 shutdown_connections 2024-09-19T08:54:01.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.160+0000 7f5748f07640 1 -- 172.21.15.164:0/978763213 wait complete. 2024-09-19T08:54:01.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.161+0000 7f5748f07640 1 Processor -- start 2024-09-19T08:54:01.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.161+0000 7f5748f07640 1 -- start start 2024-09-19T08:54:01.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.161+0000 7f5748f07640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5744100e20 0x7f57441993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:01.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.161+0000 7f5748f07640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5744102570 0x7f57441998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:01.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.161+0000 7f5748f07640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5744102ef0 0x7f57441a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:01.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.161+0000 7f5748f07640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5744110fa0 con 0x7f5744102570 2024-09-19T08:54:01.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.161+0000 7f5748f07640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5744110e20 con 0x7f5744100e20 2024-09-19T08:54:01.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.161+0000 7f5748f07640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5744111120 con 0x7f5744102ef0 2024-09-19T08:54:01.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.161+0000 7f573bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5744102570 0x7f57441998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:01.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.161+0000 7f573bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5744102570 0x7f57441998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40660/0 (socket says 172.21.15.164:40660) 2024-09-19T08:54:01.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.161+0000 7f573bfff640 1 -- 172.21.15.164:0/2158508995 learned_addr learned my addr 172.21.15.164:0/2158508995 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:01.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.162+0000 7f574259c640 1 --2- 172.21.15.164:0/2158508995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5744100e20 0x7f57441993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:01.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.162+0000 7f5742d9d640 1 --2- 172.21.15.164:0/2158508995 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5744102ef0 0x7f57441a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:01.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.162+0000 7f573bfff640 1 -- 172.21.15.164:0/2158508995 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5744102ef0 msgr2=0x7f57441a4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:01.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.162+0000 7f573bfff640 1 --2- 172.21.15.164:0/2158508995 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5744102ef0 0x7f57441a4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:01.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.162+0000 7f573bfff640 1 -- 172.21.15.164:0/2158508995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5744100e20 msgr2=0x7f57441993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:01.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.162+0000 7f573bfff640 1 --2- 172.21.15.164:0/2158508995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5744100e20 0x7f57441993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:01.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.162+0000 7f573bfff640 1 -- 172.21.15.164:0/2158508995 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57441a4970 con 0x7f5744102570 2024-09-19T08:54:01.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.162+0000 7f573bfff640 1 --2- 172.21.15.164:0/2158508995 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5744102570 0x7f57441998f0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f572c00d950 tx=0x7f572c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:01.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.162+0000 7f573b7fe640 1 -- 172.21.15.164:0/2158508995 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c017070 con 0x7f5744102570 2024-09-19T08:54:01.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.162+0000 7f573b7fe640 1 -- 172.21.15.164:0/2158508995 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f572c009c30 con 0x7f5744102570 2024-09-19T08:54:01.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.162+0000 7f5748f07640 1 -- 172.21.15.164:0/2158508995 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57441a4b40 con 0x7f5744102570 2024-09-19T08:54:01.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.162+0000 7f573b7fe640 1 -- 172.21.15.164:0/2158508995 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c00f6b0 con 0x7f5744102570 2024-09-19T08:54:01.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.162+0000 7f5748f07640 1 -- 172.21.15.164:0/2158508995 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57441a5050 con 0x7f5744102570 2024-09-19T08:54:01.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.164+0000 7f573b7fe640 1 -- 172.21.15.164:0/2158508995 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f572c0109d0 con 0x7f5744102570 2024-09-19T08:54:01.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.165+0000 7f574b2e0740 1 Processor -- start 2024-09-19T08:54:01.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.165+0000 7f574b2e0740 1 -- start start 2024-09-19T08:54:01.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.165+0000 7f574b2e0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f702ca53a0 0x55f702ca5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:01.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.165+0000 7f574b2e0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f702ca5cc0 0x55f702cb4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:01.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.165+0000 7f574b2e0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f702cb4a90 0x55f702cb6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:01.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.165+0000 7f574b2e0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f702c0b2e0 con 0x55f702cb4a90 2024-09-19T08:54:01.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.165+0000 7f574b2e0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f702bb2ce0 con 0x55f702ca53a0 2024-09-19T08:54:01.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.165+0000 7f574b2e0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f702c072a0 con 0x55f702ca5cc0 2024-09-19T08:54:01.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.165+0000 7f573b7fe640 1 --2- 172.21.15.164:0/2158508995 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f57300a33b0 0x7f57300a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:01.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.165+0000 7f573bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f702cb4a90 0x55f702cb6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:01.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.165+0000 7f574259c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f702ca5cc0 0x55f702cb4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:01.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.165+0000 7f574259c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f702ca5cc0 0x55f702cb4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:36320/0 (socket says 172.21.15.164:36320) 2024-09-19T08:54:01.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.165+0000 7f574259c640 1 -- 172.21.15.164:0/2676005245 learned_addr learned my addr 172.21.15.164:0/2676005245 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:01.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.165+0000 7f5742d9d640 1 --2- 172.21.15.164:0/2158508995 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f57300a33b0 0x7f57300a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:01.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.165+0000 7f5742d9d640 1 --2- 172.21.15.164:0/2676005245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f702ca53a0 0x55f702ca5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:01.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.165+0000 7f573b7fe640 1 -- 172.21.15.164:0/2158508995 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f572c0cf930 con 0x7f5744102570 2024-09-19T08:54:01.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.166+0000 7f573bfff640 1 -- 172.21.15.164:0/2676005245 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f702ca5cc0 msgr2=0x55f702cb4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:01.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.166+0000 7f573bfff640 1 --2- 172.21.15.164:0/2676005245 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f702ca5cc0 0x55f702cb4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:01.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.166+0000 7f573bfff640 1 -- 172.21.15.164:0/2676005245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f702ca53a0 msgr2=0x55f702ca5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:01.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.166+0000 7f573bfff640 1 --2- 172.21.15.164:0/2676005245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f702ca53a0 0x55f702ca5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:01.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.166+0000 7f573bfff640 1 -- 172.21.15.164:0/2676005245 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f702b97440 con 0x55f702cb4a90 2024-09-19T08:54:01.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.166+0000 7f573bfff640 1 --2- 172.21.15.164:0/2676005245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f702cb4a90 0x55f702cb6e80 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f572c0cd0c0 tx=0x7f572c010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65137748adac4461 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:01.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.166+0000 7f5742d9d640 1 --2- 172.21.15.164:0/2158508995 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f57300a33b0 0x7f57300a5870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f573c0148c0 tx=0x7f573c001df0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:01.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.166+0000 7f5738ff9640 1 -- 172.21.15.164:0/2676005245 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c020070 con 0x55f702cb4a90 2024-09-19T08:54:01.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.166+0000 7f5738ff9640 1 -- 172.21.15.164:0/2676005245 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f572c09c350 con 0x55f702cb4a90 2024-09-19T08:54:01.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.166+0000 7f5738ff9640 1 -- 172.21.15.164:0/2676005245 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c0982c0 con 0x55f702cb4a90 2024-09-19T08:54:01.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.167+0000 7f574b2e0740 1 -- 172.21.15.164:0/2676005245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f702cb4a90 msgr2=0x55f702cb6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:01.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.167+0000 7f574b2e0740 1 --2- 172.21.15.164:0/2676005245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f702cb4a90 0x55f702cb6e80 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f572c0cd0c0 tx=0x7f572c010d70 comp rx=0 tx=0).stop 2024-09-19T08:54:01.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.167+0000 7f574b2e0740 1 -- 172.21.15.164:0/2676005245 shutdown_connections 2024-09-19T08:54:01.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.167+0000 7f574b2e0740 1 --2- 172.21.15.164:0/2676005245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f702cb4a90 0x55f702cb6e80 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:01.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.167+0000 7f574b2e0740 1 --2- 172.21.15.164:0/2676005245 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f702ca5cc0 0x55f702cb4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:01.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.167+0000 7f574b2e0740 1 --2- 172.21.15.164:0/2676005245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f702ca53a0 0x55f702ca5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:01.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.167+0000 7f574b2e0740 1 -- 172.21.15.164:0/2676005245 >> 172.21.15.164:0/2676005245 conn(0x55f702ca1a30 msgr2=0x55f702ca1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:01.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.167+0000 7f574b2e0740 1 -- 172.21.15.164:0/2676005245 shutdown_connections 2024-09-19T08:54:01.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.167+0000 7f574b2e0740 1 -- 172.21.15.164:0/2676005245 wait complete. 2024-09-19T08:54:01.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.167+0000 7f574b2e0740 1 -- start start 2024-09-19T08:54:01.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.167+0000 7f574b2e0740 1 Processor -- start 2024-09-19T08:54:01.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.167+0000 7f574b2e0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f702ca53a0 0x55f702cb44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:01.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.168+0000 7f574b2e0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f702ca5cc0 0x55f702cb1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:01.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.168+0000 7f574b2e0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f702cb4a90 0x55f702cb1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:01.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.168+0000 7f574b2e0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f702cb7600 con 0x55f702ca5cc0 2024-09-19T08:54:01.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.168+0000 7f574b2e0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f702bc6050 con 0x55f702cb4a90 2024-09-19T08:54:01.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.168+0000 7f574b2e0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f702cb7780 con 0x55f702ca53a0 2024-09-19T08:54:01.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.168+0000 7f574b2e0740 10 client.0 _ll_register_callbacks cb 0x55f702c06160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:54:01.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.168+0000 7f573bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f702ca5cc0 0x55f702cb1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:01.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.168+0000 7f573bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f702ca5cc0 0x55f702cb1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40678/0 (socket says 172.21.15.164:40678) 2024-09-19T08:54:01.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.168+0000 7f573bfff640 1 -- 172.21.15.164:0/2312856783 learned_addr learned my addr 172.21.15.164:0/2312856783 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:01.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.168+0000 7f5742d9d640 1 --2- 172.21.15.164:0/2312856783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f702cb4a90 0x55f702cb1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:01.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.168+0000 7f574259c640 1 --2- 172.21.15.164:0/2312856783 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f702ca53a0 0x55f702cb44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:01.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.168+0000 7f573bfff640 1 -- 172.21.15.164:0/2312856783 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f702ca53a0 msgr2=0x55f702cb44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:01.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.168+0000 7f573bfff640 1 --2- 172.21.15.164:0/2312856783 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f702ca53a0 0x55f702cb44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:01.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.168+0000 7f573bfff640 1 -- 172.21.15.164:0/2312856783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f702cb4a90 msgr2=0x55f702cb1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:01.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.168+0000 7f573bfff640 1 --2- 172.21.15.164:0/2312856783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f702cb4a90 0x55f702cb1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:01.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.168+0000 7f573bfff640 1 -- 172.21.15.164:0/2312856783 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f70297c3c0 con 0x55f702ca5cc0 2024-09-19T08:54:01.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.168+0000 7f573bfff640 1 --2- 172.21.15.164:0/2312856783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f702ca5cc0 0x55f702cb1790 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f572c098620 tx=0x7f572c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e536f0415ed6730a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:01.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.168+0000 7f5729ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:54:01.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.168+0000 7f5729ffb640 1 -- 172.21.15.164:0/2312856783 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c020070 con 0x55f702ca5cc0 2024-09-19T08:54:01.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.168+0000 7f5729ffb640 1 -- 172.21.15.164:0/2312856783 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f572c086110 con 0x55f702ca5cc0 2024-09-19T08:54:01.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.168+0000 7f574b2e0740 1 -- 172.21.15.164:0/2312856783 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f572c09c350 con 0x55f702ca5cc0 2024-09-19T08:54:01.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.168+0000 7f5729ffb640 1 -- 172.21.15.164:0/2312856783 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f572c009de0 con 0x55f702ca5cc0 2024-09-19T08:54:01.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.168+0000 7f574b2e0740 10 client.17241 fetch_fsmap learned FSMap version 32 2024-09-19T08:54:01.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.168+0000 7f574b2e0740 1 -- 172.21.15.164:0/2312856783 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f702cc3110 con 0x55f702ca5cc0 2024-09-19T08:54:01.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.169+0000 7f5729ffb640 1 -- 172.21.15.164:0/2312856783 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f572c0b4e50 con 0x55f702ca5cc0 2024-09-19T08:54:01.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.169+0000 7f5729ffb640 1 -- 172.21.15.164:0/2312856783 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f572c0bd240 con 0x55f702ca5cc0 2024-09-19T08:54:01.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.169+0000 7f574b2e0740 10 client.17241 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:54:01.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.169+0000 7f574b2e0740 10 client.17241 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:54:01.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.169+0000 7f574b2e0740 20 client.17241 populate_metadata read hostname 'smithi164' 2024-09-19T08:54:01.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.169+0000 7f574b2e0740 1 --2- 172.21.15.164:0/2312856783 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55f702cc4460 0x55f702cc6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:01.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.169+0000 7f574b2e0740 4 client.17241 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:54:01.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.169+0000 7f574b2e0740 1 -- 172.21.15.164:0/2312856783 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f702c18850 con 0x55f702cc4460 2024-09-19T08:54:01.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.169+0000 7f574259c640 1 --2- 172.21.15.164:0/2312856783 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55f702cc4460 0x55f702cc6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:01.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.169+0000 7f574259c640 1 --2- 172.21.15.164:0/2312856783 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55f702cc4460 0x55f702cc6850 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50376a480b68e9b1 server_cookie=95479543c8155653 in_seq=0 out_seq=0 2024-09-19T08:54:01.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.169+0000 7f5729ffb640 10 client.17241 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:54:01.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.170+0000 7f5729ffb640 1 -- 172.21.15.164:0/2312856783 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f702c18850 con 0x55f702cc4460 2024-09-19T08:54:01.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.170+0000 7f5729ffb640 10 client.17241 handle_command_reply: tid=0 2024-09-19T08:54:01.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.170+0000 7f574b2e0740 1 client.17241 shutdown 2024-09-19T08:54:01.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.170+0000 7f574b2e0740 1 -- 172.21.15.164:0/2312856783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f702ca5cc0 msgr2=0x55f702cb1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:01.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.170+0000 7f574b2e0740 1 --2- 172.21.15.164:0/2312856783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f702ca5cc0 0x55f702cb1790 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f572c098620 tx=0x7f572c005f90 comp rx=0 tx=0).stop 2024-09-19T08:54:01.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.170+0000 7f574b2e0740 1 -- 172.21.15.164:0/2312856783 shutdown_connections 2024-09-19T08:54:01.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.170+0000 7f574b2e0740 1 --2- 172.21.15.164:0/2312856783 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55f702cc4460 0x55f702cc6850 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:01.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.170+0000 7f574b2e0740 1 --2- 172.21.15.164:0/2312856783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f702cb4a90 0x55f702cb1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:01.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.170+0000 7f574b2e0740 1 --2- 172.21.15.164:0/2312856783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f702ca5cc0 0x55f702cb1790 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:01.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.170+0000 7f5729ffb640 0 client.17241 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:54:01.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.170+0000 7f574b2e0740 1 --2- 172.21.15.164:0/2312856783 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f702ca53a0 0x55f702cb44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:01.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.170+0000 7f574b2e0740 1 -- 172.21.15.164:0/2312856783 >> 172.21.15.164:0/2312856783 conn(0x55f702ca1a30 msgr2=0x55f702ca7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:01.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.170+0000 7f574b2e0740 1 -- 172.21.15.164:0/2312856783 shutdown_connections 2024-09-19T08:54:01.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.170+0000 7f574b2e0740 1 -- 172.21.15.164:0/2312856783 wait complete. 2024-09-19T08:54:01.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.170+0000 7f574b2e0740 20 client.17241 trim_cache size 0 max 16384 2024-09-19T08:54:01.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.187+0000 7f574b2e0740 1 Processor -- start 2024-09-19T08:54:01.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.187+0000 7f574b2e0740 1 -- start start 2024-09-19T08:54:01.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.187+0000 7f574b2e0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f702ca53a0 0x55f702cae9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:01.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.187+0000 7f574b2e0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f702ca5cc0 0x55f702caef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:01.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.187+0000 7f574b2e0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f702cb4a90 0x55f702cac4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:01.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.187+0000 7f574b2e0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f57441ab660 con 0x55f702ca5cc0 2024-09-19T08:54:01.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.187+0000 7f574b2e0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f57441ad1a0 con 0x55f702ca53a0 2024-09-19T08:54:01.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.187+0000 7f574b2e0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f57441abd30 con 0x55f702cb4a90 2024-09-19T08:54:01.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.187+0000 7f573bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f702ca5cc0 0x55f702caef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:01.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.187+0000 7f573bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f702ca5cc0 0x55f702caef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40680/0 (socket says 172.21.15.164:40680) 2024-09-19T08:54:01.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.187+0000 7f573bfff640 1 -- 172.21.15.164:0/3002234773 learned_addr learned my addr 172.21.15.164:0/3002234773 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:01.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.188+0000 7f5742d9d640 1 --2- 172.21.15.164:0/3002234773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f702cb4a90 0x55f702cac4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:01.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.188+0000 7f574259c640 1 --2- 172.21.15.164:0/3002234773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f702ca53a0 0x55f702cae9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:01.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.188+0000 7f573bfff640 1 -- 172.21.15.164:0/3002234773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f702cb4a90 msgr2=0x55f702cac4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:01.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.188+0000 7f573bfff640 1 --2- 172.21.15.164:0/3002234773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f702cb4a90 0x55f702cac4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:01.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.188+0000 7f573bfff640 1 -- 172.21.15.164:0/3002234773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f702ca53a0 msgr2=0x55f702cae9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:01.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.188+0000 7f573bfff640 1 --2- 172.21.15.164:0/3002234773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f702ca53a0 0x55f702cae9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:01.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.188+0000 7f573bfff640 1 -- 172.21.15.164:0/3002234773 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f702cb4850 con 0x55f702ca5cc0 2024-09-19T08:54:01.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.188+0000 7f573bfff640 1 --2- 172.21.15.164:0/3002234773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f702ca5cc0 0x55f702caef30 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f572c09c500 tx=0x7f572c09a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a55f0af970198f71 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:01.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.188+0000 7f572bfff640 1 -- 172.21.15.164:0/3002234773 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c020070 con 0x55f702ca5cc0 2024-09-19T08:54:01.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.188+0000 7f572bfff640 1 -- 172.21.15.164:0/3002234773 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f572c09c350 con 0x55f702ca5cc0 2024-09-19T08:54:01.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.188+0000 7f572bfff640 1 -- 172.21.15.164:0/3002234773 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c0b1030 con 0x55f702ca5cc0 2024-09-19T08:54:01.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.189+0000 7f574b2e0740 1 -- 172.21.15.164:0/3002234773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f702ca5cc0 msgr2=0x55f702caef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:01.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.189+0000 7f574b2e0740 1 --2- 172.21.15.164:0/3002234773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f702ca5cc0 0x55f702caef30 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f572c09c500 tx=0x7f572c09a160 comp rx=0 tx=0).stop 2024-09-19T08:54:01.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.189+0000 7f574b2e0740 1 -- 172.21.15.164:0/3002234773 shutdown_connections 2024-09-19T08:54:01.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.189+0000 7f574b2e0740 1 --2- 172.21.15.164:0/3002234773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f702cb4a90 0x55f702cac4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:01.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.189+0000 7f574b2e0740 1 --2- 172.21.15.164:0/3002234773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f702ca5cc0 0x55f702caef30 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:01.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.189+0000 7f574b2e0740 1 --2- 172.21.15.164:0/3002234773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f702ca53a0 0x55f702cae9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:01.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.189+0000 7f574b2e0740 1 -- 172.21.15.164:0/3002234773 >> 172.21.15.164:0/3002234773 conn(0x55f702ca1a30 msgr2=0x55f702cb0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:01.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.189+0000 7f574b2e0740 1 -- 172.21.15.164:0/3002234773 shutdown_connections 2024-09-19T08:54:01.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.189+0000 7f574b2e0740 1 -- 172.21.15.164:0/3002234773 wait complete. 2024-09-19T08:54:01.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.189+0000 7f574b2e0740 1 -- start start 2024-09-19T08:54:01.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.189+0000 7f574b2e0740 1 Processor -- start 2024-09-19T08:54:01.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.189+0000 7f574b2e0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f702ca53a0 0x55f702cc9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:01.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.189+0000 7f574b2e0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f702ca5cc0 0x55f702cca130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:01.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.189+0000 7f574b2e0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f702cb4a90 0x55f702cc7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:01.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.189+0000 7f574b2e0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f57441acad0 con 0x55f702ca53a0 2024-09-19T08:54:01.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.189+0000 7f574b2e0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f57441ac400 con 0x55f702ca5cc0 2024-09-19T08:54:01.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.189+0000 7f574b2e0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f57441a9680 con 0x55f702cb4a90 2024-09-19T08:54:01.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.190+0000 7f574b2e0740 10 client.0 _ll_register_callbacks cb 0x7f572c084f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:54:01.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.190+0000 7f574259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f702ca53a0 0x55f702cc9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:01.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.190+0000 7f574259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f702ca53a0 0x55f702cc9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40684/0 (socket says 172.21.15.164:40684) 2024-09-19T08:54:01.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.190+0000 7f574259c640 1 -- 172.21.15.164:0/1461908351 learned_addr learned my addr 172.21.15.164:0/1461908351 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:01.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.190+0000 7f573bfff640 1 --2- 172.21.15.164:0/1461908351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f702ca5cc0 0x55f702cca130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:01.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.190+0000 7f5742d9d640 1 --2- 172.21.15.164:0/1461908351 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f702cb4a90 0x55f702cc7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:01.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.190+0000 7f574259c640 1 -- 172.21.15.164:0/1461908351 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f702cb4a90 msgr2=0x55f702cc7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:01.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.190+0000 7f574259c640 1 --2- 172.21.15.164:0/1461908351 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f702cb4a90 0x55f702cc7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:01.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.190+0000 7f574259c640 1 -- 172.21.15.164:0/1461908351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f702ca5cc0 msgr2=0x55f702cca130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:01.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.190+0000 7f574259c640 1 --2- 172.21.15.164:0/1461908351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f702ca5cc0 0x55f702cca130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:01.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.190+0000 7f574259c640 1 -- 172.21.15.164:0/1461908351 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f702cc79c0 con 0x55f702ca53a0 2024-09-19T08:54:01.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.190+0000 7f5742d9d640 1 --2- 172.21.15.164:0/1461908351 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f702cb4a90 0x55f702cc7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:54:01.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.190+0000 7f574259c640 1 --2- 172.21.15.164:0/1461908351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f702ca53a0 0x55f702cc9bf0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f5734005940 tx=0x7f5734004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c00993066cb731 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:01.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.190+0000 7f5729ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:54:01.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.190+0000 7f5729ffb640 1 -- 172.21.15.164:0/1461908351 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5734035040 con 0x55f702ca53a0 2024-09-19T08:54:01.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.190+0000 7f5729ffb640 1 -- 172.21.15.164:0/1461908351 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8455+0+0 (secure 0 0 0) 0x7f5734005590 con 0x55f702ca53a0 2024-09-19T08:54:01.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.190+0000 7f574b2e0740 1 -- 172.21.15.164:0/1461908351 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f572c09c350 con 0x55f702ca53a0 2024-09-19T08:54:01.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.190+0000 7f5729ffb640 1 -- 172.21.15.164:0/1461908351 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5734004750 con 0x55f702ca53a0 2024-09-19T08:54:01.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.190+0000 7f574b2e0740 10 client.17247 fetch_fsmap learned FSMap version 32 2024-09-19T08:54:01.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.190+0000 7f574b2e0740 1 -- 172.21.15.164:0/1461908351 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f702d2ea50 con 0x55f702ca53a0 2024-09-19T08:54:01.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.191+0000 7f5729ffb640 1 -- 172.21.15.164:0/1461908351 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f572c09c350 con 0x55f702ca53a0 2024-09-19T08:54:01.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.191+0000 7f5729ffb640 1 -- 172.21.15.164:0/1461908351 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f5734003c90 con 0x55f702ca53a0 2024-09-19T08:54:01.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.191+0000 7f574b2e0740 10 client.17247 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:54:01.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.191+0000 7f574b2e0740 10 client.17247 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:54:01.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.191+0000 7f574b2e0740 20 client.17247 populate_metadata read hostname 'smithi164' 2024-09-19T08:54:01.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.191+0000 7f574b2e0740 1 --2- 172.21.15.164:0/1461908351 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55f702d2f170 0x55f702d31590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:01.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.191+0000 7f574b2e0740 4 client.17247 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:54:01.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.191+0000 7f574b2e0740 1 -- 172.21.15.164:0/1461908351 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f702c12d60 con 0x55f702d2f170 2024-09-19T08:54:01.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.191+0000 7f573bfff640 1 --2- 172.21.15.164:0/1461908351 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55f702d2f170 0x55f702d31590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:01.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.191+0000 7f5729ffb640 10 client.17247 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:54:01.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.191+0000 7f573bfff640 1 --2- 172.21.15.164:0/1461908351 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55f702d2f170 0x55f702d31590 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9b715539ed99618 server_cookie=58e1f97bd3fdf39d in_seq=0 out_seq=0 2024-09-19T08:54:01.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.192+0000 7f5729ffb640 1 -- 172.21.15.164:0/1461908351 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f702c12d60 con 0x55f702d2f170 2024-09-19T08:54:01.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.192+0000 7f5729ffb640 10 client.17247 handle_command_reply: tid=0 2024-09-19T08:54:01.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.192+0000 7f574b2e0740 1 client.17247 shutdown 2024-09-19T08:54:01.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.192+0000 7f574b2e0740 1 -- 172.21.15.164:0/1461908351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f702ca53a0 msgr2=0x55f702cc9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:01.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.192+0000 7f574b2e0740 1 --2- 172.21.15.164:0/1461908351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f702ca53a0 0x55f702cc9bf0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f5734005940 tx=0x7f5734004530 comp rx=0 tx=0).stop 2024-09-19T08:54:01.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.192+0000 7f574b2e0740 1 -- 172.21.15.164:0/1461908351 shutdown_connections 2024-09-19T08:54:01.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.192+0000 7f574b2e0740 1 --2- 172.21.15.164:0/1461908351 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f702cb4a90 0x55f702cc7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:01.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.192+0000 7f574b2e0740 1 --2- 172.21.15.164:0/1461908351 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55f702d2f170 0x55f702d31590 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:01.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.192+0000 7f574b2e0740 1 --2- 172.21.15.164:0/1461908351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f702ca5cc0 0x55f702cca130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:01.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.192+0000 7f574b2e0740 1 --2- 172.21.15.164:0/1461908351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f702ca53a0 0x55f702cc9bf0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:01.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.192+0000 7f574b2e0740 1 -- 172.21.15.164:0/1461908351 >> 172.21.15.164:0/1461908351 conn(0x55f702ca1a30 msgr2=0x55f702cac4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:01.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.192+0000 7f5729ffb640 0 client.17247 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:54:01.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.192+0000 7f574b2e0740 1 -- 172.21.15.164:0/1461908351 shutdown_connections 2024-09-19T08:54:01.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.192+0000 7f574b2e0740 1 -- 172.21.15.164:0/1461908351 wait complete. 2024-09-19T08:54:01.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.192+0000 7f574b2e0740 20 client.17247 trim_cache size 0 max 16384 2024-09-19T08:54:01.203 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:54:01.203 INFO:teuthology.orchestra.run.smithi164.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T08:54:01.203 INFO:teuthology.orchestra.run.smithi164.stdout: "scrubs": { 2024-09-19T08:54:01.203 INFO:teuthology.orchestra.run.smithi164.stdout: "79f876fe-21e9-4a8d-8ab0-5e1265f85164": { 2024-09-19T08:54:01.203 INFO:teuthology.orchestra.run.smithi164.stdout: "path": "/", 2024-09-19T08:54:01.203 INFO:teuthology.orchestra.run.smithi164.stdout: "tag": "79f876fe-21e9-4a8d-8ab0-5e1265f85164", 2024-09-19T08:54:01.203 INFO:teuthology.orchestra.run.smithi164.stdout: "options": "recursive,force" 2024-09-19T08:54:01.203 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:54:01.203 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:54:01.203 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:54:01.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.193+0000 7f5748f07640 1 -- 172.21.15.164:0/2158508995 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f57300a33b0 msgr2=0x7f57300a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:01.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.193+0000 7f5748f07640 1 --2- 172.21.15.164:0/2158508995 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f57300a33b0 0x7f57300a5870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f573c0148c0 tx=0x7f573c001df0 comp rx=0 tx=0).stop 2024-09-19T08:54:01.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.193+0000 7f5748f07640 1 -- 172.21.15.164:0/2158508995 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5744102570 msgr2=0x7f57441998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:01.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.193+0000 7f5748f07640 1 --2- 172.21.15.164:0/2158508995 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5744102570 0x7f57441998f0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f572c00d950 tx=0x7f572c00de10 comp rx=0 tx=0).stop 2024-09-19T08:54:01.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.193+0000 7f5748f07640 1 -- 172.21.15.164:0/2158508995 shutdown_connections 2024-09-19T08:54:01.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.193+0000 7f5748f07640 1 --2- 172.21.15.164:0/2158508995 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5744102ef0 0x7f57441a4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:01.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.193+0000 7f5748f07640 1 --2- 172.21.15.164:0/2158508995 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f57300a33b0 0x7f57300a5870 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:01.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.193+0000 7f5748f07640 1 --2- 172.21.15.164:0/2158508995 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5744102570 0x7f57441998f0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:01.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.193+0000 7f5748f07640 1 --2- 172.21.15.164:0/2158508995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5744100e20 0x7f57441993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:01.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.193+0000 7f5748f07640 1 -- 172.21.15.164:0/2158508995 >> 172.21.15.164:0/2158508995 conn(0x7f57440fc7d0 msgr2=0x7f57440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:01.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.193+0000 7f5748f07640 1 -- 172.21.15.164:0/2158508995 shutdown_connections 2024-09-19T08:54:01.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:01.193+0000 7f5748f07640 1 -- 172.21.15.164:0/2158508995 wait complete. 2024-09-19T08:54:01.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:00 smithi170 ceph-mon[35072]: scrub queued for path: / 2024-09-19T08:54:01.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:00 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:54:01.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:00 smithi170 ceph-mon[35072]: scrub summary: active paths [/] 2024-09-19T08:54:01.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:00 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:54:01.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:00 smithi173 ceph-mon[35306]: scrub queued for path: / 2024-09-19T08:54:01.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:00 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:54:01.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:00 smithi173 ceph-mon[35306]: scrub summary: active paths [/] 2024-09-19T08:54:01.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:00 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:54:02.023 INFO:tasks.cephfs.filesystem:scrub status for tag:79f876fe-21e9-4a8d-8ab0-5e1265f85164 - {'path': '/', 'tag': '79f876fe-21e9-4a8d-8ab0-5e1265f85164', 'options': 'recursive,force'} 2024-09-19T08:54:02.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:01 smithi164 ceph-mon[30928]: pgmap v253: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:54:02.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:01 smithi170 ceph-mon[35072]: pgmap v253: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:54:02.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:01 smithi173 ceph-mon[35306]: pgmap v253: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:54:04.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:03 smithi164 ceph-mon[30928]: pgmap v254: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-19T08:54:04.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:03 smithi170 ceph-mon[35072]: pgmap v254: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-19T08:54:04.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:03 smithi173 ceph-mon[35306]: pgmap v254: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-19T08:54:06.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:05 smithi164 ceph-mon[30928]: pgmap v255: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-19T08:54:06.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:05 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:54:06.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:05 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:06.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:05 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:54:06.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:05 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:54:06.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:05 smithi170 ceph-mon[35072]: pgmap v255: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-19T08:54:06.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:05 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:54:06.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:05 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:06.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:05 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:54:06.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:05 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:54:06.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:05 smithi173 ceph-mon[35306]: pgmap v255: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-19T08:54:06.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:05 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:54:06.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:05 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:54:06.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:05 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:54:06.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:05 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:54:08.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:07 smithi164 ceph-mon[30928]: pgmap v256: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-09-19T08:54:08.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:07 smithi164 ceph-mon[30928]: scrub summary: idle 2024-09-19T08:54:08.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:07 smithi164 ceph-mon[30928]: scrub summary: idle 2024-09-19T08:54:08.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:07 smithi170 ceph-mon[35072]: pgmap v256: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-09-19T08:54:08.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:07 smithi170 ceph-mon[35072]: scrub summary: idle 2024-09-19T08:54:08.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:07 smithi170 ceph-mon[35072]: scrub summary: idle 2024-09-19T08:54:08.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:07 smithi173 ceph-mon[35306]: pgmap v256: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-09-19T08:54:08.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:07 smithi173 ceph-mon[35306]: scrub summary: idle 2024-09-19T08:54:08.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:07 smithi173 ceph-mon[35306]: scrub summary: idle 2024-09-19T08:54:09.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:08 smithi164 ceph-mon[30928]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T08:54:09.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:08 smithi170 ceph-mon[35072]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T08:54:09.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:08 smithi173 ceph-mon[35306]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T08:54:10.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:09 smithi164 ceph-mon[30928]: pgmap v258: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 54 KiB/s wr, 34 op/s 2024-09-19T08:54:10.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:09 smithi170 ceph-mon[35072]: pgmap v258: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 54 KiB/s wr, 34 op/s 2024-09-19T08:54:10.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:09 smithi173 ceph-mon[35306]: pgmap v258: 97 pgs: 97 active+clean; 51 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 54 KiB/s wr, 34 op/s 2024-09-19T08:54:12.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:11 smithi164 ceph-mon[30928]: pgmap v259: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 44 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 54 KiB/s wr, 34 op/s 2024-09-19T08:54:12.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:11 smithi170 ceph-mon[35072]: pgmap v259: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 44 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 54 KiB/s wr, 34 op/s 2024-09-19T08:54:12.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:11 smithi173 ceph-mon[35306]: pgmap v259: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 44 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 54 KiB/s wr, 34 op/s 2024-09-19T08:54:14.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:13 smithi164 ceph-mon[30928]: pgmap v260: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 43 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 KiB/s wr, 16 op/s 2024-09-19T08:54:14.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:13 smithi170 ceph-mon[35072]: pgmap v260: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 43 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 KiB/s wr, 16 op/s 2024-09-19T08:54:14.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:13 smithi173 ceph-mon[35306]: pgmap v260: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 43 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 KiB/s wr, 16 op/s 2024-09-19T08:54:16.135 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:54:16.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:15 smithi164 ceph-mon[30928]: pgmap v261: 97 pgs: 97 active+clean; 41 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 27 KiB/s wr, 14 op/s 2024-09-19T08:54:16.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:15 smithi170 ceph-mon[35072]: pgmap v261: 97 pgs: 97 active+clean; 41 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 27 KiB/s wr, 14 op/s 2024-09-19T08:54:16.395 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:54:16.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:15 smithi173 ceph-mon[35306]: pgmap v261: 97 pgs: 97 active+clean; 41 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 27 KiB/s wr, 14 op/s 2024-09-19T08:54:16.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.894+0000 7f27e42e1640 1 -- 172.21.15.164:0/1394754779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27dc101de0 msgr2=0x7f27dc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:16.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.894+0000 7f27e42e1640 1 --2- 172.21.15.164:0/1394754779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27dc101de0 0x7f27dc1021c0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f27cc009900 tx=0x7f27cc02a410 comp rx=0 tx=0).stop 2024-09-19T08:54:16.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.894+0000 7f27e42e1640 1 -- 172.21.15.164:0/1394754779 shutdown_connections 2024-09-19T08:54:16.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.894+0000 7f27e42e1640 1 --2- 172.21.15.164:0/1394754779 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27dc106f10 0x7f27dc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:16.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.894+0000 7f27e42e1640 1 --2- 172.21.15.164:0/1394754779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27dc102790 0x7f27dc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:16.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.894+0000 7f27e42e1640 1 --2- 172.21.15.164:0/1394754779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27dc101de0 0x7f27dc1021c0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:16.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.894+0000 7f27e42e1640 1 -- 172.21.15.164:0/1394754779 >> 172.21.15.164:0/1394754779 conn(0x7f27dc0fd8c0 msgr2=0x7f27dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:16.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.894+0000 7f27e42e1640 1 -- 172.21.15.164:0/1394754779 shutdown_connections 2024-09-19T08:54:16.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.894+0000 7f27e42e1640 1 -- 172.21.15.164:0/1394754779 wait complete. 2024-09-19T08:54:16.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.895+0000 7f27e42e1640 1 Processor -- start 2024-09-19T08:54:16.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.895+0000 7f27e42e1640 1 -- start start 2024-09-19T08:54:16.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.895+0000 7f27e42e1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27dc101de0 0x7f27dc198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:16.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.895+0000 7f27e42e1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27dc102790 0x7f27dc1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:16.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.895+0000 7f27e42e1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27dc106f10 0x7f27dc19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:16.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.895+0000 7f27e42e1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f27dc111f90 con 0x7f27dc106f10 2024-09-19T08:54:16.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.895+0000 7f27e42e1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f27dc111e10 con 0x7f27dc101de0 2024-09-19T08:54:16.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.895+0000 7f27e42e1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f27dc112110 con 0x7f27dc102790 2024-09-19T08:54:16.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.895+0000 7f27e287e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27dc106f10 0x7f27dc19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:16.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.895+0000 7f27e287e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27dc106f10 0x7f27dc19cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41276/0 (socket says 172.21.15.164:41276) 2024-09-19T08:54:16.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.895+0000 7f27e287e640 1 -- 172.21.15.164:0/3416394923 learned_addr learned my addr 172.21.15.164:0/3416394923 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:16.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.896+0000 7f27e187c640 1 --2- 172.21.15.164:0/3416394923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27dc102790 0x7f27dc1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:16.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.896+0000 7f27e207d640 1 --2- 172.21.15.164:0/3416394923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27dc101de0 0x7f27dc198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:16.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.896+0000 7f27e287e640 1 -- 172.21.15.164:0/3416394923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27dc102790 msgr2=0x7f27dc1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:16.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.896+0000 7f27e287e640 1 --2- 172.21.15.164:0/3416394923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27dc102790 0x7f27dc1989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:16.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.896+0000 7f27e287e640 1 -- 172.21.15.164:0/3416394923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27dc101de0 msgr2=0x7f27dc198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:16.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.896+0000 7f27e287e640 1 --2- 172.21.15.164:0/3416394923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27dc101de0 0x7f27dc198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:16.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.896+0000 7f27e287e640 1 -- 172.21.15.164:0/3416394923 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27dc19d430 con 0x7f27dc106f10 2024-09-19T08:54:16.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.896+0000 7f27e187c640 1 --2- 172.21.15.164:0/3416394923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27dc102790 0x7f27dc1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:54:16.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.896+0000 7f27e287e640 1 --2- 172.21.15.164:0/3416394923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27dc106f10 0x7f27dc19cd50 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f27d400b970 tx=0x7f27d400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:16.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.896+0000 7f27d37fe640 1 -- 172.21.15.164:0/3416394923 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d4019070 con 0x7f27dc106f10 2024-09-19T08:54:16.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.896+0000 7f27e42e1640 1 -- 172.21.15.164:0/3416394923 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27dc19d720 con 0x7f27dc106f10 2024-09-19T08:54:16.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.896+0000 7f27d37fe640 1 -- 172.21.15.164:0/3416394923 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27d400cb40 con 0x7f27dc106f10 2024-09-19T08:54:16.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.896+0000 7f27d37fe640 1 -- 172.21.15.164:0/3416394923 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d4007610 con 0x7f27dc106f10 2024-09-19T08:54:16.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.896+0000 7f27e42e1640 1 -- 172.21.15.164:0/3416394923 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27dc1a5000 con 0x7f27dc106f10 2024-09-19T08:54:16.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.897+0000 7f27e42e1640 1 -- 172.21.15.164:0/3416394923 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27dc1021c0 con 0x7f27dc106f10 2024-09-19T08:54:16.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.898+0000 7f27d37fe640 1 -- 172.21.15.164:0/3416394923 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f27d401c070 con 0x7f27dc106f10 2024-09-19T08:54:16.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.899+0000 7f27d37fe640 1 --2- 172.21.15.164:0/3416394923 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f27c80a33b0 0x7f27c80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:16.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.899+0000 7f27e207d640 1 --2- 172.21.15.164:0/3416394923 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f27c80a33b0 0x7f27c80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:16.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.899+0000 7f27d37fe640 1 -- 172.21.15.164:0/3416394923 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7f27d40cf240 con 0x7f27dc106f10 2024-09-19T08:54:16.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.899+0000 7f27e207d640 1 --2- 172.21.15.164:0/3416394923 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f27c80a33b0 0x7f27c80a5870 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f27cc005ab0 tx=0x7f27cc005a20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:16.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:16.903+0000 7f27d37fe640 1 -- 172.21.15.164:0/3416394923 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f27d409bd40 con 0x7f27dc106f10 2024-09-19T08:54:17.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:17.105+0000 7f27e42e1640 1 -- 172.21.15.164:0/3416394923 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27dc02d7c0 con 0x7f27dc106f10 2024-09-19T08:54:17.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:17.106+0000 7f27d37fe640 1 -- 172.21.15.164:0/3416394923 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11984 (secure 0 0 0) 0x7f27d4097030 con 0x7f27dc106f10 2024-09-19T08:54:17.106 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:54:17.106 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":32,"btime":"2024-09-19T08:52:27:192804+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:52:27.074168+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:54:17.106 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 32 2024-09-19T08:54:17.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:17.108+0000 7f27e42e1640 1 -- 172.21.15.164:0/3416394923 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f27c80a33b0 msgr2=0x7f27c80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:17.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:17.108+0000 7f27e42e1640 1 --2- 172.21.15.164:0/3416394923 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f27c80a33b0 0x7f27c80a5870 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f27cc005ab0 tx=0x7f27cc005a20 comp rx=0 tx=0).stop 2024-09-19T08:54:17.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:17.108+0000 7f27e42e1640 1 -- 172.21.15.164:0/3416394923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27dc106f10 msgr2=0x7f27dc19cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:17.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:17.108+0000 7f27e42e1640 1 --2- 172.21.15.164:0/3416394923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27dc106f10 0x7f27dc19cd50 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f27d400b970 tx=0x7f27d400be30 comp rx=0 tx=0).stop 2024-09-19T08:54:17.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:17.108+0000 7f27e42e1640 1 -- 172.21.15.164:0/3416394923 shutdown_connections 2024-09-19T08:54:17.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:17.108+0000 7f27e42e1640 1 --2- 172.21.15.164:0/3416394923 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f27c80a33b0 0x7f27c80a5870 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:17.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:17.108+0000 7f27e42e1640 1 --2- 172.21.15.164:0/3416394923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27dc106f10 0x7f27dc19cd50 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:17.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:17.108+0000 7f27e42e1640 1 --2- 172.21.15.164:0/3416394923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27dc102790 0x7f27dc1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:17.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:17.108+0000 7f27e42e1640 1 --2- 172.21.15.164:0/3416394923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27dc101de0 0x7f27dc198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:17.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:17.108+0000 7f27e42e1640 1 -- 172.21.15.164:0/3416394923 >> 172.21.15.164:0/3416394923 conn(0x7f27dc0fd8c0 msgr2=0x7f27dc0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:17.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:17.108+0000 7f27e42e1640 1 -- 172.21.15.164:0/3416394923 shutdown_connections 2024-09-19T08:54:17.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:17.108+0000 7f27e42e1640 1 -- 172.21.15.164:0/3416394923 wait complete. 2024-09-19T08:54:18.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:17 smithi164 ceph-mon[30928]: pgmap v262: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T08:54:18.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:17 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3416394923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:54:18.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:17 smithi170 ceph-mon[35072]: pgmap v262: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T08:54:18.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:17 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3416394923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:54:18.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:17 smithi173 ceph-mon[35306]: pgmap v262: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T08:54:18.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:17 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3416394923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:54:18.645 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id 1e7fd7ac --release --await' 2024-09-19T08:54:18.646 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id 1e7fd7ac --release --await 2024-09-19T08:54:18.900 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:54:19.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.360+0000 7f56951ef640 1 -- 172.21.15.164:0/3047256996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5690102ef0 msgr2=0x7f569010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:19.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.360+0000 7f56951ef640 1 --2- 172.21.15.164:0/3047256996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5690102ef0 0x7f569010da40 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f5688009ca0 tx=0x7f568802a950 comp rx=0 tx=0).stop 2024-09-19T08:54:19.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.361+0000 7f56951ef640 1 -- 172.21.15.164:0/3047256996 shutdown_connections 2024-09-19T08:54:19.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.361+0000 7f56951ef640 1 --2- 172.21.15.164:0/3047256996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5690102ef0 0x7f569010da40 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.361+0000 7f56951ef640 1 --2- 172.21.15.164:0/3047256996 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5690102570 0x7f56901029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.361+0000 7f56951ef640 1 --2- 172.21.15.164:0/3047256996 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5690100e20 0x7f5690101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.361+0000 7f56951ef640 1 -- 172.21.15.164:0/3047256996 >> 172.21.15.164:0/3047256996 conn(0x7f56900fc7d0 msgr2=0x7f56900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:19.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.361+0000 7f56951ef640 1 -- 172.21.15.164:0/3047256996 shutdown_connections 2024-09-19T08:54:19.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.361+0000 7f56951ef640 1 -- 172.21.15.164:0/3047256996 wait complete. 2024-09-19T08:54:19.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.361+0000 7f56951ef640 1 Processor -- start 2024-09-19T08:54:19.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.362+0000 7f56951ef640 1 -- start start 2024-09-19T08:54:19.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.362+0000 7f56951ef640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5690100e20 0x7f56901992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:19.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.362+0000 7f56951ef640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5690102570 0x7f5690199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:19.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.362+0000 7f56951ef640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5690102ef0 0x7f56901a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:19.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.362+0000 7f56951ef640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5690110f80 con 0x7f5690102ef0 2024-09-19T08:54:19.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.362+0000 7f56951ef640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5690110e00 con 0x7f5690100e20 2024-09-19T08:54:19.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.362+0000 7f56951ef640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5690111100 con 0x7f5690102570 2024-09-19T08:54:19.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.362+0000 7f568f59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5690102ef0 0x7f56901a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:19.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.362+0000 7f568f59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5690102ef0 0x7f56901a4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41298/0 (socket says 172.21.15.164:41298) 2024-09-19T08:54:19.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.362+0000 7f568f59e640 1 -- 172.21.15.164:0/2542543174 learned_addr learned my addr 172.21.15.164:0/2542543174 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:19.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.362+0000 7f568ed9d640 1 --2- 172.21.15.164:0/2542543174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5690100e20 0x7f56901992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:19.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.362+0000 7f568e59c640 1 --2- 172.21.15.164:0/2542543174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5690102570 0x7f5690199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:19.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.362+0000 7f568f59e640 1 -- 172.21.15.164:0/2542543174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5690102570 msgr2=0x7f5690199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:19.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.362+0000 7f568f59e640 1 --2- 172.21.15.164:0/2542543174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5690102570 0x7f5690199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.362+0000 7f568f59e640 1 -- 172.21.15.164:0/2542543174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5690100e20 msgr2=0x7f56901992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:19.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.362+0000 7f568f59e640 1 --2- 172.21.15.164:0/2542543174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5690100e20 0x7f56901992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.362+0000 7f568f59e640 1 -- 172.21.15.164:0/2542543174 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56901a48b0 con 0x7f5690102ef0 2024-09-19T08:54:19.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.363+0000 7f568f59e640 1 --2- 172.21.15.164:0/2542543174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5690102ef0 0x7f56901a4370 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f568802ae70 tx=0x7f568802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:19.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.363+0000 7f5677fff640 1 -- 172.21.15.164:0/2542543174 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5688034040 con 0x7f5690102ef0 2024-09-19T08:54:19.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.363+0000 7f5677fff640 1 -- 172.21.15.164:0/2542543174 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5688030920 con 0x7f5690102ef0 2024-09-19T08:54:19.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.363+0000 7f56951ef640 1 -- 172.21.15.164:0/2542543174 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56901a4a80 con 0x7f5690102ef0 2024-09-19T08:54:19.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.363+0000 7f5677fff640 1 -- 172.21.15.164:0/2542543174 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5688045480 con 0x7f5690102ef0 2024-09-19T08:54:19.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.363+0000 7f56951ef640 1 -- 172.21.15.164:0/2542543174 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56901a4ed0 con 0x7f5690102ef0 2024-09-19T08:54:19.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.365+0000 7f5677fff640 1 -- 172.21.15.164:0/2542543174 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f5688037020 con 0x7f5690102ef0 2024-09-19T08:54:19.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.365+0000 7f56975c8740 1 Processor -- start 2024-09-19T08:54:19.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.365+0000 7f56975c8740 1 -- start start 2024-09-19T08:54:19.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.366+0000 7f56975c8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605921c1060 0x5605921c1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:19.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.366+0000 7f56975c8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605921c1980 0x5605921cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:19.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.366+0000 7f56975c8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605921cc240 0x5605921ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:19.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.366+0000 7f56975c8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560592126620 con 0x5605921cc240 2024-09-19T08:54:19.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.366+0000 7f56975c8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5605920cde50 con 0x5605921c1060 2024-09-19T08:54:19.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.366+0000 7f56975c8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5605921225e0 con 0x5605921c1980 2024-09-19T08:54:19.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.366+0000 7f568e59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605921cc240 0x5605921ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:19.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.366+0000 7f568e59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605921cc240 0x5605921ce630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41310/0 (socket says 172.21.15.164:41310) 2024-09-19T08:54:19.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.366+0000 7f568e59c640 1 -- 172.21.15.164:0/4042298377 learned_addr learned my addr 172.21.15.164:0/4042298377 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:19.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.366+0000 7f568e59c640 1 --2- 172.21.15.164:0/4042298377 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605921c1060 0x5605921c1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:19.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.366+0000 7f568ed9d640 1 --2- 172.21.15.164:0/4042298377 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605921c1980 0x5605921cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:19.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.366+0000 7f5677fff640 1 --2- 172.21.15.164:0/2542543174 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f56780a33b0 0x7f56780a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:19.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.366+0000 7f568e59c640 1 -- 172.21.15.164:0/4042298377 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605921c1980 msgr2=0x5605921cbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:19.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.366+0000 7f568e59c640 1 --2- 172.21.15.164:0/4042298377 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605921c1980 0x5605921cbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.366+0000 7f568e59c640 1 -- 172.21.15.164:0/4042298377 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605921c1060 msgr2=0x5605921c1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:19.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.366+0000 7f568e59c640 1 --2- 172.21.15.164:0/4042298377 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605921c1060 0x5605921c1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.366+0000 7f568e59c640 1 -- 172.21.15.164:0/4042298377 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605920bed40 con 0x5605921cc240 2024-09-19T08:54:19.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.366+0000 7f568f59e640 1 --2- 172.21.15.164:0/2542543174 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f56780a33b0 0x7f56780a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:19.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.366+0000 7f568e59c640 1 --2- 172.21.15.164:0/4042298377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605921cc240 0x5605921ce630 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f5684013580 tx=0x7f568400c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc8caf29be6cc83 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:19.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.366+0000 7f5677fff640 1 -- 172.21.15.164:0/2542543174 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7f56880f0b70 con 0x7f5690102ef0 2024-09-19T08:54:19.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.367+0000 7f56757fa640 1 -- 172.21.15.164:0/4042298377 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684005020 con 0x5605921cc240 2024-09-19T08:54:19.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.367+0000 7f56757fa640 1 -- 172.21.15.164:0/4042298377 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56840047a0 con 0x5605921cc240 2024-09-19T08:54:19.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.367+0000 7f56757fa640 1 -- 172.21.15.164:0/4042298377 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568400f6c0 con 0x5605921cc240 2024-09-19T08:54:19.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.367+0000 7f56975c8740 1 -- 172.21.15.164:0/4042298377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605921cc240 msgr2=0x5605921ce630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:19.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.367+0000 7f568f59e640 1 --2- 172.21.15.164:0/2542543174 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f56780a33b0 0x7f56780a5870 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f56880b9370 tx=0x7f568808cb30 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:19.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.367+0000 7f56975c8740 1 --2- 172.21.15.164:0/4042298377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605921cc240 0x5605921ce630 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f5684013580 tx=0x7f568400c610 comp rx=0 tx=0).stop 2024-09-19T08:54:19.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.367+0000 7f56975c8740 1 -- 172.21.15.164:0/4042298377 shutdown_connections 2024-09-19T08:54:19.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.367+0000 7f56975c8740 1 --2- 172.21.15.164:0/4042298377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605921cc240 0x5605921ce630 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.367+0000 7f56975c8740 1 --2- 172.21.15.164:0/4042298377 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605921c1980 0x5605921cbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.367+0000 7f56975c8740 1 --2- 172.21.15.164:0/4042298377 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605921c1060 0x5605921c1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.367+0000 7f56975c8740 1 -- 172.21.15.164:0/4042298377 >> 172.21.15.164:0/4042298377 conn(0x5605921bd5f0 msgr2=0x5605921bd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:19.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.367+0000 7f56975c8740 1 -- 172.21.15.164:0/4042298377 shutdown_connections 2024-09-19T08:54:19.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.367+0000 7f56975c8740 1 -- 172.21.15.164:0/4042298377 wait complete. 2024-09-19T08:54:19.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.368+0000 7f56975c8740 1 -- start start 2024-09-19T08:54:19.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.368+0000 7f56975c8740 1 Processor -- start 2024-09-19T08:54:19.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.368+0000 7f56975c8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605921c1060 0x5605921c9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:19.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.368+0000 7f56975c8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605921c1980 0x5605921c9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:19.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.368+0000 7f56975c8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605921cc240 0x5605921c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:19.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.368+0000 7f56975c8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5605921cee10 con 0x5605921cc240 2024-09-19T08:54:19.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.368+0000 7f56975c8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5605920aef10 con 0x5605921c1980 2024-09-19T08:54:19.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.368+0000 7f56975c8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5605921cef90 con 0x5605921c1060 2024-09-19T08:54:19.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.368+0000 7f56975c8740 10 client.0 _ll_register_callbacks cb 0x5605920d2db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:54:19.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.368+0000 7f568e59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605921cc240 0x5605921c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:19.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.368+0000 7f568e59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605921cc240 0x5605921c9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41326/0 (socket says 172.21.15.164:41326) 2024-09-19T08:54:19.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.368+0000 7f568e59c640 1 -- 172.21.15.164:0/1013311070 learned_addr learned my addr 172.21.15.164:0/1013311070 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:19.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.368+0000 7f568e59c640 1 --2- 172.21.15.164:0/1013311070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605921c1060 0x5605921c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:19.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.368+0000 7f568ed9d640 1 --2- 172.21.15.164:0/1013311070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605921c1980 0x5605921c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:19.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.368+0000 7f568e59c640 1 -- 172.21.15.164:0/1013311070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605921c1060 msgr2=0x5605921c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:19.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.368+0000 7f568e59c640 1 --2- 172.21.15.164:0/1013311070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605921c1060 0x5605921c9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.368+0000 7f568e59c640 1 -- 172.21.15.164:0/1013311070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605921c1980 msgr2=0x5605921c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:19.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.368+0000 7f568e59c640 1 --2- 172.21.15.164:0/1013311070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605921c1980 0x5605921c9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.369+0000 7f568e59c640 1 -- 172.21.15.164:0/1013311070 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560591e35580 con 0x5605921cc240 2024-09-19T08:54:19.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.369+0000 7f568e59c640 1 --2- 172.21.15.164:0/1013311070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605921cc240 0x5605921c9bd0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f568401b040 tx=0x7f56840056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37613973c281b3a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:19.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.369+0000 7f5672ff5640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:54:19.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.369+0000 7f5672ff5640 1 -- 172.21.15.164:0/1013311070 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684005210 con 0x5605921cc240 2024-09-19T08:54:19.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.369+0000 7f5672ff5640 1 -- 172.21.15.164:0/1013311070 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7f56840053b0 con 0x5605921cc240 2024-09-19T08:54:19.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.369+0000 7f56975c8740 1 -- 172.21.15.164:0/1013311070 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56840047a0 con 0x5605921cc240 2024-09-19T08:54:19.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.369+0000 7f5672ff5640 1 -- 172.21.15.164:0/1013311070 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5684035050 con 0x5605921cc240 2024-09-19T08:54:19.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.369+0000 7f56975c8740 10 client.17265 fetch_fsmap learned FSMap version 32 2024-09-19T08:54:19.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.369+0000 7f56975c8740 1 -- 172.21.15.164:0/1013311070 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560591eff960 con 0x5605921cc240 2024-09-19T08:54:19.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.369+0000 7f5672ff5640 1 -- 172.21.15.164:0/1013311070 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56840047a0 con 0x5605921cc240 2024-09-19T08:54:19.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.369+0000 7f5672ff5640 1 -- 172.21.15.164:0/1013311070 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f568401e700 con 0x5605921cc240 2024-09-19T08:54:19.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.369+0000 7f56975c8740 10 client.17265 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:54:19.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.369+0000 7f56975c8740 10 client.17265 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:54:19.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.369+0000 7f56975c8740 20 client.17265 populate_metadata read hostname 'smithi164' 2024-09-19T08:54:19.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.369+0000 7f56975c8740 1 --2- 172.21.15.164:0/1013311070 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5605921dbd90 0x5605921de180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:19.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.369+0000 7f56975c8740 4 client.17265 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:54:19.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.369+0000 7f56975c8740 1 -- 172.21.15.164:0/1013311070 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560592133b90 con 0x5605921dbd90 2024-09-19T08:54:19.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.370+0000 7f568ed9d640 1 --2- 172.21.15.164:0/1013311070 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5605921dbd90 0x5605921de180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:19.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.370+0000 7f568ed9d640 1 --2- 172.21.15.164:0/1013311070 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5605921dbd90 0x5605921de180 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eba59e845e0163a7 server_cookie=87682a01719d3f5f in_seq=0 out_seq=0 2024-09-19T08:54:19.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.370+0000 7f5672ff5640 10 client.17265 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:54:19.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.371+0000 7f5672ff5640 1 -- 172.21.15.164:0/1013311070 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560592133b90 con 0x5605921dbd90 2024-09-19T08:54:19.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.371+0000 7f5672ff5640 10 client.17265 handle_command_reply: tid=0 2024-09-19T08:54:19.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.371+0000 7f56975c8740 1 client.17265 shutdown 2024-09-19T08:54:19.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.371+0000 7f56975c8740 1 -- 172.21.15.164:0/1013311070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605921cc240 msgr2=0x5605921c9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:19.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.371+0000 7f56975c8740 1 --2- 172.21.15.164:0/1013311070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605921cc240 0x5605921c9bd0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f568401b040 tx=0x7f56840056d0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.371+0000 7f56975c8740 1 -- 172.21.15.164:0/1013311070 shutdown_connections 2024-09-19T08:54:19.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.371+0000 7f56975c8740 1 --2- 172.21.15.164:0/1013311070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605921cc240 0x5605921c9bd0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.371+0000 7f56975c8740 1 --2- 172.21.15.164:0/1013311070 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5605921dbd90 0x5605921de180 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.371+0000 7f56975c8740 1 --2- 172.21.15.164:0/1013311070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605921c1980 0x5605921c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.371+0000 7f56975c8740 1 --2- 172.21.15.164:0/1013311070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605921c1060 0x5605921c9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.371+0000 7f5672ff5640 0 client.17265 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:54:19.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.371+0000 7f56975c8740 1 -- 172.21.15.164:0/1013311070 >> 172.21.15.164:0/1013311070 conn(0x5605921bd5f0 msgr2=0x5605921c32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:19.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.371+0000 7f56975c8740 1 -- 172.21.15.164:0/1013311070 shutdown_connections 2024-09-19T08:54:19.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.371+0000 7f56975c8740 1 -- 172.21.15.164:0/1013311070 wait complete. 2024-09-19T08:54:19.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.371+0000 7f56975c8740 20 client.17265 trim_cache size 0 max 16384 2024-09-19T08:54:19.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.388+0000 7f56975c8740 1 Processor -- start 2024-09-19T08:54:19.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.388+0000 7f56975c8740 1 -- start start 2024-09-19T08:54:19.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.388+0000 7f56975c8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605921c1060 0x5605921bbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:19.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.388+0000 7f56975c8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605921c1980 0x5605921c31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:19.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.388+0000 7f56975c8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605921cc240 0x5605921e3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:19.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.388+0000 7f56975c8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f56901ad780 con 0x5605921c1060 2024-09-19T08:54:19.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.388+0000 7f56975c8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f56901af2c0 con 0x5605921cc240 2024-09-19T08:54:19.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.388+0000 7f56975c8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f56901ade50 con 0x5605921c1980 2024-09-19T08:54:19.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.389+0000 7f568e59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605921c1060 0x5605921bbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:19.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.389+0000 7f568e59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605921c1060 0x5605921bbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41334/0 (socket says 172.21.15.164:41334) 2024-09-19T08:54:19.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.389+0000 7f568e59c640 1 -- 172.21.15.164:0/2190289930 learned_addr learned my addr 172.21.15.164:0/2190289930 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:19.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.389+0000 7f568ed9d640 1 --2- 172.21.15.164:0/2190289930 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605921c1980 0x5605921c31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:19.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.389+0000 7f568e59c640 1 --2- 172.21.15.164:0/2190289930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605921cc240 0x5605921e3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:19.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.389+0000 7f568e59c640 1 -- 172.21.15.164:0/2190289930 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605921c1980 msgr2=0x5605921c31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:19.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.389+0000 7f568e59c640 1 --2- 172.21.15.164:0/2190289930 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605921c1980 0x5605921c31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.389+0000 7f568e59c640 1 -- 172.21.15.164:0/2190289930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605921cc240 msgr2=0x5605921e3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:19.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.389+0000 7f568e59c640 1 --2- 172.21.15.164:0/2190289930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605921cc240 0x5605921e3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.389+0000 7f568e59c640 1 -- 172.21.15.164:0/2190289930 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605921c4d30 con 0x5605921c1060 2024-09-19T08:54:19.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.389+0000 7f568e59c640 1 --2- 172.21.15.164:0/2190289930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605921c1060 0x5605921bbf30 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f5684004750 tx=0x7f5684004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4622942f77d472c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:19.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.389+0000 7f5674ff9640 1 -- 172.21.15.164:0/2190289930 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684004e50 con 0x5605921c1060 2024-09-19T08:54:19.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.389+0000 7f5674ff9640 1 -- 172.21.15.164:0/2190289930 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5684004780 con 0x5605921c1060 2024-09-19T08:54:19.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.389+0000 7f5674ff9640 1 -- 172.21.15.164:0/2190289930 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684013020 con 0x5605921c1060 2024-09-19T08:54:19.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.390+0000 7f56975c8740 1 -- 172.21.15.164:0/2190289930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605921c1060 msgr2=0x5605921bbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:19.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.390+0000 7f56975c8740 1 --2- 172.21.15.164:0/2190289930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605921c1060 0x5605921bbf30 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f5684004750 tx=0x7f5684004680 comp rx=0 tx=0).stop 2024-09-19T08:54:19.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.390+0000 7f56975c8740 1 -- 172.21.15.164:0/2190289930 shutdown_connections 2024-09-19T08:54:19.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.390+0000 7f56975c8740 1 --2- 172.21.15.164:0/2190289930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605921cc240 0x5605921e3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.390+0000 7f56975c8740 1 --2- 172.21.15.164:0/2190289930 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605921c1980 0x5605921c31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.390+0000 7f56975c8740 1 --2- 172.21.15.164:0/2190289930 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605921c1060 0x5605921bbf30 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.390+0000 7f56975c8740 1 -- 172.21.15.164:0/2190289930 >> 172.21.15.164:0/2190289930 conn(0x5605921bd5f0 msgr2=0x5605921c2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:19.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.390+0000 7f56975c8740 1 -- 172.21.15.164:0/2190289930 shutdown_connections 2024-09-19T08:54:19.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.390+0000 7f56975c8740 1 -- 172.21.15.164:0/2190289930 wait complete. 2024-09-19T08:54:19.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.390+0000 7f56975c8740 1 -- start start 2024-09-19T08:54:19.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.391+0000 7f56975c8740 1 Processor -- start 2024-09-19T08:54:19.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.391+0000 7f56975c8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605921c1060 0x5605921d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:19.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.391+0000 7f56975c8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605921c1980 0x5605921d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:19.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.391+0000 7f56975c8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605921cc240 0x560592245960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:19.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.391+0000 7f56975c8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f56901aebf0 con 0x5605921cc240 2024-09-19T08:54:19.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.391+0000 7f56975c8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f56901ae520 con 0x5605921c1060 2024-09-19T08:54:19.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.391+0000 7f56975c8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f56901ab780 con 0x5605921c1980 2024-09-19T08:54:19.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.391+0000 7f56975c8740 10 client.0 _ll_register_callbacks cb 0x7f5684004da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:54:19.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.391+0000 7f568e59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605921cc240 0x560592245960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:19.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.391+0000 7f568e59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605921cc240 0x560592245960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41338/0 (socket says 172.21.15.164:41338) 2024-09-19T08:54:19.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.391+0000 7f568e59c640 1 -- 172.21.15.164:0/4180137545 learned_addr learned my addr 172.21.15.164:0/4180137545 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:19.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.391+0000 7f568e59c640 1 --2- 172.21.15.164:0/4180137545 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605921c1060 0x5605921d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:19.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.391+0000 7f568ed9d640 1 --2- 172.21.15.164:0/4180137545 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605921c1980 0x5605921d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:19.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.391+0000 7f568e59c640 1 -- 172.21.15.164:0/4180137545 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605921c1980 msgr2=0x5605921d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:19.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.391+0000 7f568e59c640 1 --2- 172.21.15.164:0/4180137545 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605921c1980 0x5605921d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.391+0000 7f568e59c640 1 -- 172.21.15.164:0/4180137545 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605921c1060 msgr2=0x5605921d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:19.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.391+0000 7f568e59c640 1 --2- 172.21.15.164:0/4180137545 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605921c1060 0x5605921d03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.391+0000 7f568e59c640 1 -- 172.21.15.164:0/4180137545 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560592245ea0 con 0x5605921cc240 2024-09-19T08:54:19.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.391+0000 7f568e59c640 1 --2- 172.21.15.164:0/4180137545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605921cc240 0x560592245960 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f568400c7a0 tx=0x7f5684030040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d2bffa9328d6c23 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:19.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.391+0000 7f5672ff5640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:54:19.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.392+0000 7f5672ff5640 1 -- 172.21.15.164:0/4180137545 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684013280 con 0x5605921cc240 2024-09-19T08:54:19.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.392+0000 7f5672ff5640 1 -- 172.21.15.164:0/4180137545 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7f568401e430 con 0x5605921cc240 2024-09-19T08:54:19.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.392+0000 7f56975c8740 1 -- 172.21.15.164:0/4180137545 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5684004780 con 0x5605921cc240 2024-09-19T08:54:19.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.392+0000 7f5672ff5640 1 -- 172.21.15.164:0/4180137545 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f568401e630 con 0x5605921cc240 2024-09-19T08:54:19.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.392+0000 7f56975c8740 10 client.17271 fetch_fsmap learned FSMap version 32 2024-09-19T08:54:19.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.392+0000 7f56975c8740 1 -- 172.21.15.164:0/4180137545 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5605922470c0 con 0x5605921cc240 2024-09-19T08:54:19.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.392+0000 7f5672ff5640 1 -- 172.21.15.164:0/4180137545 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5684004780 con 0x5605921cc240 2024-09-19T08:54:19.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.392+0000 7f5672ff5640 1 -- 172.21.15.164:0/4180137545 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f568404f030 con 0x5605921cc240 2024-09-19T08:54:19.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.392+0000 7f56975c8740 10 client.17271 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:54:19.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.392+0000 7f56975c8740 10 client.17271 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:54:19.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.392+0000 7f56975c8740 20 client.17271 populate_metadata read hostname 'smithi164' 2024-09-19T08:54:19.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.392+0000 7f56975c8740 1 --2- 172.21.15.164:0/4180137545 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x560592247730 0x560592249b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:19.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.392+0000 7f56975c8740 4 client.17271 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1e7fd7ac", "release": true, "await": true}] 2024-09-19T08:54:19.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.392+0000 7f56975c8740 1 -- 172.21.15.164:0/4180137545 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1e7fd7ac", "release": true, "await": true}) -- 0x5605921bcde0 con 0x560592247730 2024-09-19T08:54:19.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.392+0000 7f568ed9d640 1 --2- 172.21.15.164:0/4180137545 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x560592247730 0x560592249b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:19.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.393+0000 7f5672ff5640 10 client.17271 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:54:19.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.393+0000 7f568ed9d640 1 --2- 172.21.15.164:0/4180137545 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x560592247730 0x560592249b20 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd84ce0a5dd651d6 server_cookie=536a67848fbc15f6 in_seq=0 out_seq=0 2024-09-19T08:54:19.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.396+0000 7f5672ff5640 1 -- 172.21.15.164:0/4180137545 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5605921bcde0 con 0x560592247730 2024-09-19T08:54:19.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.396+0000 7f5672ff5640 10 client.17271 handle_command_reply: tid=0 2024-09-19T08:54:19.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.396+0000 7f56975c8740 1 client.17271 shutdown 2024-09-19T08:54:19.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.397+0000 7f56975c8740 1 -- 172.21.15.164:0/4180137545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605921cc240 msgr2=0x560592245960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:19.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.397+0000 7f56975c8740 1 --2- 172.21.15.164:0/4180137545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605921cc240 0x560592245960 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f568400c7a0 tx=0x7f5684030040 comp rx=0 tx=0).stop 2024-09-19T08:54:19.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.397+0000 7f56975c8740 1 -- 172.21.15.164:0/4180137545 shutdown_connections 2024-09-19T08:54:19.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.397+0000 7f56975c8740 1 --2- 172.21.15.164:0/4180137545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605921cc240 0x560592245960 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.397+0000 7f56975c8740 1 --2- 172.21.15.164:0/4180137545 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605921c1980 0x5605921d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.397+0000 7f56975c8740 1 --2- 172.21.15.164:0/4180137545 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x560592247730 0x560592249b20 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.397+0000 7f56975c8740 1 --2- 172.21.15.164:0/4180137545 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605921c1060 0x5605921d03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.397+0000 7f5672ff5640 0 client.17271 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:54:19.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.397+0000 7f56975c8740 1 -- 172.21.15.164:0/4180137545 >> 172.21.15.164:0/4180137545 conn(0x5605921bd5f0 msgr2=0x5605921c32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:19.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.397+0000 7f56975c8740 1 -- 172.21.15.164:0/4180137545 shutdown_connections 2024-09-19T08:54:19.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.397+0000 7f56975c8740 1 -- 172.21.15.164:0/4180137545 wait complete. 2024-09-19T08:54:19.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.397+0000 7f56975c8740 20 client.17271 trim_cache size 0 max 16384 2024-09-19T08:54:19.399 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:54:19.399 INFO:teuthology.orchestra.run.smithi164.stdout: "epoch": 32, 2024-09-19T08:54:19.399 INFO:teuthology.orchestra.run.smithi164.stdout: "leader": 24479, 2024-09-19T08:54:19.399 INFO:teuthology.orchestra.run.smithi164.stdout: "set_version": 32, 2024-09-19T08:54:19.399 INFO:teuthology.orchestra.run.smithi164.stdout: "sets": { 2024-09-19T08:54:19.399 INFO:teuthology.orchestra.run.smithi164.stdout: "1e7fd7ac": { 2024-09-19T08:54:19.399 INFO:teuthology.orchestra.run.smithi164.stdout: "version": 32, 2024-09-19T08:54:19.399 INFO:teuthology.orchestra.run.smithi164.stdout: "age_ref": 0.0, 2024-09-19T08:54:19.399 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:54:19.400 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "RELEASED", 2024-09-19T08:54:19.400 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:54:19.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:54:19.400 INFO:teuthology.orchestra.run.smithi164.stdout: "timeout": 90.0, 2024-09-19T08:54:19.400 INFO:teuthology.orchestra.run.smithi164.stdout: "expiration": 145.8, 2024-09-19T08:54:19.400 INFO:teuthology.orchestra.run.smithi164.stdout: "members": { 2024-09-19T08:54:19.400 INFO:teuthology.orchestra.run.smithi164.stdout: "file:/": { 2024-09-19T08:54:19.400 INFO:teuthology.orchestra.run.smithi164.stdout: "excluded": false, 2024-09-19T08:54:19.400 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:54:19.400 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "RELEASED", 2024-09-19T08:54:19.400 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:54:19.400 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:54:19.401 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:54:19.401 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:54:19.401 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:54:19.401 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:54:19.401 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:54:19.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.397+0000 7f56951ef640 1 -- 172.21.15.164:0/2542543174 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f56780a33b0 msgr2=0x7f56780a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:19.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.397+0000 7f56951ef640 1 --2- 172.21.15.164:0/2542543174 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f56780a33b0 0x7f56780a5870 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f56880b9370 tx=0x7f568808cb30 comp rx=0 tx=0).stop 2024-09-19T08:54:19.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.397+0000 7f56951ef640 1 -- 172.21.15.164:0/2542543174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5690102ef0 msgr2=0x7f56901a4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:19.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.397+0000 7f56951ef640 1 --2- 172.21.15.164:0/2542543174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5690102ef0 0x7f56901a4370 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f568802ae70 tx=0x7f568802f5d0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.397+0000 7f56951ef640 1 -- 172.21.15.164:0/2542543174 shutdown_connections 2024-09-19T08:54:19.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.397+0000 7f56951ef640 1 --2- 172.21.15.164:0/2542543174 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f56780a33b0 0x7f56780a5870 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.397+0000 7f56951ef640 1 --2- 172.21.15.164:0/2542543174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5690102ef0 0x7f56901a4370 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.397+0000 7f56951ef640 1 --2- 172.21.15.164:0/2542543174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5690102570 0x7f5690199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.397+0000 7f56951ef640 1 --2- 172.21.15.164:0/2542543174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5690100e20 0x7f56901992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:19.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.397+0000 7f56951ef640 1 -- 172.21.15.164:0/2542543174 >> 172.21.15.164:0/2542543174 conn(0x7f56900fc7d0 msgr2=0x7f5690105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:19.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.397+0000 7f56951ef640 1 -- 172.21.15.164:0/2542543174 shutdown_connections 2024-09-19T08:54:19.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:19.397+0000 7f56951ef640 1 -- 172.21.15.164:0/2542543174 wait complete. 2024-09-19T08:54:20.137 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1e7fd7ac', total seconds elapsed: 29.8 2024-09-19T08:54:20.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:19 smithi164 ceph-mon[30928]: pgmap v263: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 196 B/s wr, 9 op/s 2024-09-19T08:54:20.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:19 smithi164 ceph-mon[30928]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T08:54:20.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:19 smithi170 ceph-mon[35072]: pgmap v263: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 196 B/s wr, 9 op/s 2024-09-19T08:54:20.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:19 smithi170 ceph-mon[35072]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T08:54:20.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:19 smithi173 ceph-mon[35306]: pgmap v263: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 196 B/s wr, 9 op/s 2024-09-19T08:54:20.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:19 smithi173 ceph-mon[35306]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T08:54:22.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:21 smithi164 ceph-mon[30928]: pgmap v265: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T08:54:22.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:21 smithi170 ceph-mon[35072]: pgmap v265: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T08:54:22.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:21 smithi173 ceph-mon[35306]: pgmap v265: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T08:54:24.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:23 smithi164 ceph-mon[30928]: pgmap v266: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T08:54:24.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:23 smithi170 ceph-mon[35072]: pgmap v266: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T08:54:24.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:23 smithi173 ceph-mon[35306]: pgmap v266: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T08:54:26.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:25 smithi164 ceph-mon[30928]: pgmap v267: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:54:26.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:25 smithi170 ceph-mon[35072]: pgmap v267: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:54:26.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:25 smithi173 ceph-mon[35306]: pgmap v267: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:54:28.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:27 smithi164 ceph-mon[30928]: pgmap v268: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:54:28.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:27 smithi170 ceph-mon[35072]: pgmap v268: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:54:28.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:27 smithi173 ceph-mon[35306]: pgmap v268: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:54:30.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:29 smithi164 ceph-mon[30928]: pgmap v269: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:54:30.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:29 smithi170 ceph-mon[35072]: pgmap v269: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:54:30.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:29 smithi173 ceph-mon[35306]: pgmap v269: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:54:32.024 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:54:32.281 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:31 smithi164 ceph-mon[30928]: pgmap v270: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-09-19T08:54:32.287 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:54:32.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:31 smithi170 ceph-mon[35072]: pgmap v270: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-09-19T08:54:32.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:31 smithi173 ceph-mon[35306]: pgmap v270: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-09-19T08:54:32.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.728+0000 7fbb8285f640 1 -- 172.21.15.164:0/1082212726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb7c101dd0 msgr2=0x7fbb7c1021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:32.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.728+0000 7fbb8285f640 1 --2- 172.21.15.164:0/1082212726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb7c101dd0 0x7fbb7c1021b0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009900 tx=0x7fbb6c02a410 comp rx=0 tx=0).stop 2024-09-19T08:54:32.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.728+0000 7fbb8285f640 1 -- 172.21.15.164:0/1082212726 shutdown_connections 2024-09-19T08:54:32.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.728+0000 7fbb8285f640 1 --2- 172.21.15.164:0/1082212726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb7c106f00 0x7fbb7c10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.728+0000 7fbb8285f640 1 --2- 172.21.15.164:0/1082212726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb7c102780 0x7fbb7c1067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.728+0000 7fbb8285f640 1 --2- 172.21.15.164:0/1082212726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb7c101dd0 0x7fbb7c1021b0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.728+0000 7fbb8285f640 1 -- 172.21.15.164:0/1082212726 >> 172.21.15.164:0/1082212726 conn(0x7fbb7c0fd850 msgr2=0x7fbb7c0ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:32.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.728+0000 7fbb8285f640 1 -- 172.21.15.164:0/1082212726 shutdown_connections 2024-09-19T08:54:32.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.728+0000 7fbb8285f640 1 -- 172.21.15.164:0/1082212726 wait complete. 2024-09-19T08:54:32.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.729+0000 7fbb8285f640 1 Processor -- start 2024-09-19T08:54:32.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.729+0000 7fbb8285f640 1 -- start start 2024-09-19T08:54:32.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.729+0000 7fbb8285f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb7c101dd0 0x7fbb7c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:32.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.729+0000 7fbb8285f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb7c102780 0x7fbb7c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:32.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.729+0000 7fbb8285f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb7c106f00 0x7fbb7c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:32.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.729+0000 7fbb8285f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c111f60 con 0x7fbb7c102780 2024-09-19T08:54:32.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.729+0000 7fbb8285f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c111de0 con 0x7fbb7c106f00 2024-09-19T08:54:32.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.729+0000 7fbb8285f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c1120e0 con 0x7fbb7c101dd0 2024-09-19T08:54:32.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.730+0000 7fbb7b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb7c102780 0x7fbb7c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:32.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.730+0000 7fbb7b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb7c102780 0x7fbb7c198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:34046/0 (socket says 172.21.15.164:34046) 2024-09-19T08:54:32.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.730+0000 7fbb7b7fe640 1 -- 172.21.15.164:0/2006438443 learned_addr learned my addr 172.21.15.164:0/2006438443 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:32.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.730+0000 7fbb80dfc640 1 --2- 172.21.15.164:0/2006438443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb7c106f00 0x7fbb7c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:32.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.730+0000 7fbb7bfff640 1 --2- 172.21.15.164:0/2006438443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb7c101dd0 0x7fbb7c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:32.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.730+0000 7fbb7b7fe640 1 -- 172.21.15.164:0/2006438443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb7c101dd0 msgr2=0x7fbb7c198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:32.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.730+0000 7fbb7b7fe640 1 --2- 172.21.15.164:0/2006438443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb7c101dd0 0x7fbb7c198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.730+0000 7fbb7b7fe640 1 -- 172.21.15.164:0/2006438443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb7c106f00 msgr2=0x7fbb7c19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:32.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.730+0000 7fbb7b7fe640 1 --2- 172.21.15.164:0/2006438443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb7c106f00 0x7fbb7c19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.730+0000 7fbb7b7fe640 1 -- 172.21.15.164:0/2006438443 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb7c19d3f0 con 0x7fbb7c102780 2024-09-19T08:54:32.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.730+0000 7fbb80dfc640 1 --2- 172.21.15.164:0/2006438443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb7c106f00 0x7fbb7c19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:54:32.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.730+0000 7fbb7b7fe640 1 --2- 172.21.15.164:0/2006438443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb7c102780 0x7fbb7c198980 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fbb7000d770 tx=0x7fbb7000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:32.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.730+0000 7fbb797fa640 1 -- 172.21.15.164:0/2006438443 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb70017070 con 0x7fbb7c102780 2024-09-19T08:54:32.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.730+0000 7fbb797fa640 1 -- 172.21.15.164:0/2006438443 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb70005500 con 0x7fbb7c102780 2024-09-19T08:54:32.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.730+0000 7fbb8285f640 1 -- 172.21.15.164:0/2006438443 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb7c19d6e0 con 0x7fbb7c102780 2024-09-19T08:54:32.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.730+0000 7fbb797fa640 1 -- 172.21.15.164:0/2006438443 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb70004e00 con 0x7fbb7c102780 2024-09-19T08:54:32.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.730+0000 7fbb8285f640 1 -- 172.21.15.164:0/2006438443 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb7c1a4fc0 con 0x7fbb7c102780 2024-09-19T08:54:32.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.733+0000 7fbb797fa640 1 -- 172.21.15.164:0/2006438443 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fbb70004fa0 con 0x7fbb7c102780 2024-09-19T08:54:32.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.733+0000 7fbb84c38740 1 Processor -- start 2024-09-19T08:54:32.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.733+0000 7fbb84c38740 1 -- start start 2024-09-19T08:54:32.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.733+0000 7fbb84c38740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590d6790360 0x5590d6790740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:32.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.733+0000 7fbb84c38740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590d6790c80 0x5590d679f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:32.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.733+0000 7fbb84c38740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5590d679fa90 0x5590d67a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:32.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.733+0000 7fbb84c38740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5590d66f62e0 con 0x5590d679fa90 2024-09-19T08:54:32.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.733+0000 7fbb84c38740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5590d669dce0 con 0x5590d6790c80 2024-09-19T08:54:32.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.733+0000 7fbb84c38740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5590d66f22a0 con 0x5590d6790360 2024-09-19T08:54:32.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.733+0000 7fbb7b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5590d679fa90 0x5590d67a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:32.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.733+0000 7fbb7b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5590d679fa90 0x5590d67a1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:34056/0 (socket says 172.21.15.164:34056) 2024-09-19T08:54:32.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.733+0000 7fbb7b7fe640 1 -- 172.21.15.164:0/64860173 learned_addr learned my addr 172.21.15.164:0/64860173 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:32.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.733+0000 7fbb80dfc640 1 --2- 172.21.15.164:0/64860173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590d6790360 0x5590d6790740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:32.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.733+0000 7fbb797fa640 1 --2- 172.21.15.164:0/2006438443 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fbb640a33b0 0x7fbb640a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:32.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.733+0000 7fbb7bfff640 1 --2- 172.21.15.164:0/64860173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590d6790c80 0x5590d679f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:32.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.734+0000 7fbb80dfc640 1 --2- 172.21.15.164:0/2006438443 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fbb640a33b0 0x7fbb640a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:32.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.734+0000 7fbb797fa640 1 -- 172.21.15.164:0/2006438443 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7fbb700cf810 con 0x7fbb7c102780 2024-09-19T08:54:32.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.734+0000 7fbb7b7fe640 1 -- 172.21.15.164:0/64860173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590d6790360 msgr2=0x5590d6790740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:32.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.734+0000 7fbb7b7fe640 1 --2- 172.21.15.164:0/64860173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590d6790360 0x5590d6790740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.734+0000 7fbb7b7fe640 1 -- 172.21.15.164:0/64860173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590d6790c80 msgr2=0x5590d679f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:32.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.734+0000 7fbb7b7fe640 1 --2- 172.21.15.164:0/64860173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590d6790c80 0x5590d679f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.734+0000 7fbb7b7fe640 1 -- 172.21.15.164:0/64860173 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590d6682440 con 0x5590d679fa90 2024-09-19T08:54:32.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.734+0000 7fbb7b7fe640 1 --2- 172.21.15.164:0/64860173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5590d679fa90 0x5590d67a1e80 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fbb700ccf10 tx=0x7fbb70097f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a782257604ccef1f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:32.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.734+0000 7fbb80dfc640 1 --2- 172.21.15.164:0/2006438443 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fbb640a33b0 0x7fbb640a5870 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbb74005fd0 tx=0x7fbb74004ab0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:32.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.734+0000 7fbb627fc640 1 -- 172.21.15.164:0/64860173 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb70020050 con 0x5590d679fa90 2024-09-19T08:54:32.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.734+0000 7fbb627fc640 1 -- 172.21.15.164:0/64860173 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb700046c0 con 0x5590d679fa90 2024-09-19T08:54:32.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.734+0000 7fbb627fc640 1 -- 172.21.15.164:0/64860173 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb70099e30 con 0x5590d679fa90 2024-09-19T08:54:32.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.735+0000 7fbb84c38740 1 -- 172.21.15.164:0/64860173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5590d679fa90 msgr2=0x5590d67a1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:32.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.735+0000 7fbb84c38740 1 --2- 172.21.15.164:0/64860173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5590d679fa90 0x5590d67a1e80 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fbb700ccf10 tx=0x7fbb70097f00 comp rx=0 tx=0).stop 2024-09-19T08:54:32.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.735+0000 7fbb84c38740 1 -- 172.21.15.164:0/64860173 shutdown_connections 2024-09-19T08:54:32.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.735+0000 7fbb84c38740 1 --2- 172.21.15.164:0/64860173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5590d679fa90 0x5590d67a1e80 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.735+0000 7fbb84c38740 1 --2- 172.21.15.164:0/64860173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590d6790c80 0x5590d679f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.735+0000 7fbb84c38740 1 --2- 172.21.15.164:0/64860173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590d6790360 0x5590d6790740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.735+0000 7fbb84c38740 1 -- 172.21.15.164:0/64860173 >> 172.21.15.164:0/64860173 conn(0x5590d678ca30 msgr2=0x5590d678ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:32.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.735+0000 7fbb84c38740 1 -- 172.21.15.164:0/64860173 shutdown_connections 2024-09-19T08:54:32.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.735+0000 7fbb84c38740 1 -- 172.21.15.164:0/64860173 wait complete. 2024-09-19T08:54:32.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.735+0000 7fbb84c38740 1 -- start start 2024-09-19T08:54:32.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.735+0000 7fbb84c38740 1 Processor -- start 2024-09-19T08:54:32.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.736+0000 7fbb84c38740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5590d6790360 0x5590d679f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:32.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.736+0000 7fbb84c38740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590d6790c80 0x5590d679c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:32.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.736+0000 7fbb84c38740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590d679fa90 0x5590d679cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:32.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.736+0000 7fbb84c38740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5590d67a25a0 con 0x5590d6790360 2024-09-19T08:54:32.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.736+0000 7fbb84c38740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5590d66b1050 con 0x5590d6790c80 2024-09-19T08:54:32.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.736+0000 7fbb84c38740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5590d67a2720 con 0x5590d679fa90 2024-09-19T08:54:32.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.736+0000 7fbb84c38740 10 client.0 _ll_register_callbacks cb 0x5590d66f1160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:54:32.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.736+0000 7fbb7bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5590d6790360 0x5590d679f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:32.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.736+0000 7fbb7bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5590d6790360 0x5590d679f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:34062/0 (socket says 172.21.15.164:34062) 2024-09-19T08:54:32.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.736+0000 7fbb7bfff640 1 -- 172.21.15.164:0/2446550785 learned_addr learned my addr 172.21.15.164:0/2446550785 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:32.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.736+0000 7fbb7b7fe640 1 --2- 172.21.15.164:0/2446550785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590d6790c80 0x5590d679c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:32.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.736+0000 7fbb80dfc640 1 --2- 172.21.15.164:0/2446550785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590d679fa90 0x5590d679cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:32.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.736+0000 7fbb7bfff640 1 -- 172.21.15.164:0/2446550785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590d679fa90 msgr2=0x5590d679cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:32.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.736+0000 7fbb7bfff640 1 --2- 172.21.15.164:0/2446550785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590d679fa90 0x5590d679cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.736+0000 7fbb7bfff640 1 -- 172.21.15.164:0/2446550785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590d6790c80 msgr2=0x5590d679c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:32.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.736+0000 7fbb7bfff640 1 --2- 172.21.15.164:0/2446550785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590d6790c80 0x5590d679c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.736+0000 7fbb7bfff640 1 -- 172.21.15.164:0/2446550785 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590d64673c0 con 0x5590d6790360 2024-09-19T08:54:32.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.736+0000 7fbb7b7fe640 1 --2- 172.21.15.164:0/2446550785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590d6790c80 0x5590d679c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:54:32.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.736+0000 7fbb7bfff640 1 --2- 172.21.15.164:0/2446550785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5590d6790360 0x5590d679f400 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009230 tx=0x7fbb6c009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de34a848ad597485 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:32.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.736+0000 7fbb5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:54:32.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.736+0000 7fbb5b7fe640 1 -- 172.21.15.164:0/2446550785 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c040070 con 0x5590d6790360 2024-09-19T08:54:32.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.737+0000 7fbb5b7fe640 1 -- 172.21.15.164:0/2446550785 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7fbb6c004240 con 0x5590d6790360 2024-09-19T08:54:32.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.737+0000 7fbb84c38740 1 -- 172.21.15.164:0/2446550785 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5590d6703850 con 0x5590d6790360 2024-09-19T08:54:32.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.737+0000 7fbb5b7fe640 1 -- 172.21.15.164:0/2446550785 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbb6c035040 con 0x5590d6790360 2024-09-19T08:54:32.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.737+0000 7fbb84c38740 10 client.17283 fetch_fsmap learned FSMap version 32 2024-09-19T08:54:32.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.737+0000 7fbb84c38740 1 -- 172.21.15.164:0/2446550785 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5590d67adfb0 con 0x5590d6790360 2024-09-19T08:54:32.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.737+0000 7fbb5b7fe640 1 -- 172.21.15.164:0/2446550785 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5590d6703850 con 0x5590d6790360 2024-09-19T08:54:32.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.737+0000 7fbb5b7fe640 1 -- 172.21.15.164:0/2446550785 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fbb6c004960 con 0x5590d6790360 2024-09-19T08:54:32.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.737+0000 7fbb84c38740 10 client.17283 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:54:32.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.737+0000 7fbb84c38740 10 client.17283 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:54:32.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.737+0000 7fbb84c38740 20 client.17283 populate_metadata read hostname 'smithi164' 2024-09-19T08:54:32.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.737+0000 7fbb84c38740 1 --2- 172.21.15.164:0/2446550785 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5590d67af470 0x5590d67b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:32.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.737+0000 7fbb84c38740 4 client.17283 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:54:32.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.737+0000 7fbb84c38740 1 -- 172.21.15.164:0/2446550785 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590d66fdd60 con 0x5590d67af470 2024-09-19T08:54:32.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.737+0000 7fbb7b7fe640 1 --2- 172.21.15.164:0/2446550785 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5590d67af470 0x5590d67b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:32.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.738+0000 7fbb5b7fe640 10 client.17283 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:54:32.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.738+0000 7fbb7b7fe640 1 --2- 172.21.15.164:0/2446550785 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5590d67af470 0x5590d67b1860 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67971d5845b7082 server_cookie=a2ed1f676f9f7d15 in_seq=0 out_seq=0 2024-09-19T08:54:32.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.738+0000 7fbb5b7fe640 1 -- 172.21.15.164:0/2446550785 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5590d66fdd60 con 0x5590d67af470 2024-09-19T08:54:32.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.738+0000 7fbb5b7fe640 10 client.17283 handle_command_reply: tid=0 2024-09-19T08:54:32.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.739+0000 7fbb84c38740 1 client.17283 shutdown 2024-09-19T08:54:32.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.739+0000 7fbb84c38740 1 -- 172.21.15.164:0/2446550785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5590d6790360 msgr2=0x5590d679f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:32.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.739+0000 7fbb84c38740 1 --2- 172.21.15.164:0/2446550785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5590d6790360 0x5590d679f400 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009230 tx=0x7fbb6c009650 comp rx=0 tx=0).stop 2024-09-19T08:54:32.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.739+0000 7fbb84c38740 1 -- 172.21.15.164:0/2446550785 shutdown_connections 2024-09-19T08:54:32.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.739+0000 7fbb84c38740 1 --2- 172.21.15.164:0/2446550785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590d679fa90 0x5590d679cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.739+0000 7fbb84c38740 1 --2- 172.21.15.164:0/2446550785 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5590d67af470 0x5590d67b1860 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.739+0000 7fbb84c38740 1 --2- 172.21.15.164:0/2446550785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590d6790c80 0x5590d679c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.739+0000 7fbb84c38740 1 --2- 172.21.15.164:0/2446550785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5590d6790360 0x5590d679f400 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.739+0000 7fbb5b7fe640 0 client.17283 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:54:32.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.739+0000 7fbb84c38740 1 -- 172.21.15.164:0/2446550785 >> 172.21.15.164:0/2446550785 conn(0x5590d678ca30 msgr2=0x5590d6792790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:32.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.739+0000 7fbb84c38740 1 -- 172.21.15.164:0/2446550785 shutdown_connections 2024-09-19T08:54:32.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.739+0000 7fbb84c38740 1 -- 172.21.15.164:0/2446550785 wait complete. 2024-09-19T08:54:32.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.739+0000 7fbb84c38740 20 client.17283 trim_cache size 0 max 16384 2024-09-19T08:54:32.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.756+0000 7fbb84c38740 1 Processor -- start 2024-09-19T08:54:32.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.756+0000 7fbb84c38740 1 -- start start 2024-09-19T08:54:32.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.756+0000 7fbb84c38740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590d6790360 0x5590d67ab840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:32.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.756+0000 7fbb84c38740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5590d6790c80 0x5590d67b7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:32.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.756+0000 7fbb84c38740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590d679fa90 0x5590d67b7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:32.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.756+0000 7fbb84c38740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c1ab640 con 0x5590d6790c80 2024-09-19T08:54:32.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.756+0000 7fbb84c38740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c1ad180 con 0x5590d6790360 2024-09-19T08:54:32.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.756+0000 7fbb84c38740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c1abd10 con 0x5590d679fa90 2024-09-19T08:54:32.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.756+0000 7fbb7b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5590d6790c80 0x5590d67b7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:32.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.756+0000 7fbb7b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5590d6790c80 0x5590d67b7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:34070/0 (socket says 172.21.15.164:34070) 2024-09-19T08:54:32.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.756+0000 7fbb7b7fe640 1 -- 172.21.15.164:0/1310788035 learned_addr learned my addr 172.21.15.164:0/1310788035 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:32.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.756+0000 7fbb7bfff640 1 --2- 172.21.15.164:0/1310788035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590d6790360 0x5590d67ab840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:32.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.756+0000 7fbb80dfc640 1 --2- 172.21.15.164:0/1310788035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590d679fa90 0x5590d67b7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:32.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.756+0000 7fbb7b7fe640 1 -- 172.21.15.164:0/1310788035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590d679fa90 msgr2=0x5590d67b7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:32.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.756+0000 7fbb7b7fe640 1 --2- 172.21.15.164:0/1310788035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590d679fa90 0x5590d67b7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.756+0000 7fbb7b7fe640 1 -- 172.21.15.164:0/1310788035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590d6790360 msgr2=0x5590d67ab840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:32.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.756+0000 7fbb7b7fe640 1 --2- 172.21.15.164:0/1310788035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590d6790360 0x5590d67ab840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.756+0000 7fbb7b7fe640 1 -- 172.21.15.164:0/1310788035 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590d679b660 con 0x5590d6790c80 2024-09-19T08:54:32.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.756+0000 7fbb80dfc640 1 --2- 172.21.15.164:0/1310788035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590d679fa90 0x5590d67b7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:54:32.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.756+0000 7fbb7b7fe640 1 --2- 172.21.15.164:0/1310788035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5590d6790c80 0x5590d67b7130 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fbb70009c70 tx=0x7fbb70097e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e52a693af235a8b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:32.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.757+0000 7fbb61ffb640 1 -- 172.21.15.164:0/1310788035 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7009ce60 con 0x5590d6790c80 2024-09-19T08:54:32.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.757+0000 7fbb61ffb640 1 -- 172.21.15.164:0/1310788035 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb700a47a0 con 0x5590d6790c80 2024-09-19T08:54:32.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.757+0000 7fbb61ffb640 1 -- 172.21.15.164:0/1310788035 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb70020040 con 0x5590d6790c80 2024-09-19T08:54:32.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.757+0000 7fbb84c38740 1 -- 172.21.15.164:0/1310788035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5590d6790c80 msgr2=0x5590d67b7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:32.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.757+0000 7fbb84c38740 1 --2- 172.21.15.164:0/1310788035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5590d6790c80 0x5590d67b7130 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fbb70009c70 tx=0x7fbb70097e50 comp rx=0 tx=0).stop 2024-09-19T08:54:32.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.757+0000 7fbb84c38740 1 -- 172.21.15.164:0/1310788035 shutdown_connections 2024-09-19T08:54:32.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.757+0000 7fbb84c38740 1 --2- 172.21.15.164:0/1310788035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590d679fa90 0x5590d67b7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.757+0000 7fbb84c38740 1 --2- 172.21.15.164:0/1310788035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5590d6790c80 0x5590d67b7130 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.757+0000 7fbb84c38740 1 --2- 172.21.15.164:0/1310788035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590d6790360 0x5590d67ab840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.757+0000 7fbb84c38740 1 -- 172.21.15.164:0/1310788035 >> 172.21.15.164:0/1310788035 conn(0x5590d678ca30 msgr2=0x5590d6797d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:32.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.757+0000 7fbb84c38740 1 -- 172.21.15.164:0/1310788035 shutdown_connections 2024-09-19T08:54:32.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.757+0000 7fbb84c38740 1 -- 172.21.15.164:0/1310788035 wait complete. 2024-09-19T08:54:32.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.758+0000 7fbb84c38740 1 -- start start 2024-09-19T08:54:32.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.758+0000 7fbb84c38740 1 Processor -- start 2024-09-19T08:54:32.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.758+0000 7fbb84c38740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5590d6790360 0x5590d6799730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:32.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.758+0000 7fbb84c38740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590d6790c80 0x5590d6799c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:32.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.758+0000 7fbb84c38740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590d679fa90 0x5590d679a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:32.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.758+0000 7fbb84c38740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c1acab0 con 0x5590d6790360 2024-09-19T08:54:32.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.758+0000 7fbb84c38740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c1ac3e0 con 0x5590d6790c80 2024-09-19T08:54:32.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.758+0000 7fbb84c38740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c1a9640 con 0x5590d679fa90 2024-09-19T08:54:32.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.758+0000 7fbb84c38740 10 client.0 _ll_register_callbacks cb 0x7fbb6c031400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:54:32.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.758+0000 7fbb7bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5590d6790360 0x5590d6799730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:32.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.759+0000 7fbb7b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590d6790c80 0x5590d6799c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:32.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.759+0000 7fbb7b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590d6790c80 0x5590d6799c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:52428/0 (socket says 172.21.15.164:52428) 2024-09-19T08:54:32.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.759+0000 7fbb7b7fe640 1 -- 172.21.15.164:0/2156844204 learned_addr learned my addr 172.21.15.164:0/2156844204 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:32.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.759+0000 7fbb80dfc640 1 --2- 172.21.15.164:0/2156844204 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590d679fa90 0x5590d679a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:32.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.759+0000 7fbb7bfff640 1 -- 172.21.15.164:0/2156844204 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590d679fa90 msgr2=0x5590d679a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:32.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.759+0000 7fbb7bfff640 1 --2- 172.21.15.164:0/2156844204 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590d679fa90 0x5590d679a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.759+0000 7fbb7bfff640 1 -- 172.21.15.164:0/2156844204 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590d6790c80 msgr2=0x5590d6799c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:32.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.759+0000 7fbb7bfff640 1 --2- 172.21.15.164:0/2156844204 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590d6790c80 0x5590d6799c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.759+0000 7fbb7bfff640 1 -- 172.21.15.164:0/2156844204 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590d679cf80 con 0x5590d6790360 2024-09-19T08:54:32.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.759+0000 7fbb80dfc640 1 --2- 172.21.15.164:0/2156844204 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590d679fa90 0x5590d679a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:54:32.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.759+0000 7fbb7b7fe640 1 --2- 172.21.15.164:0/2156844204 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590d6790c80 0x5590d6799c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:54:32.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.759+0000 7fbb7bfff640 1 --2- 172.21.15.164:0/2156844204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5590d6790360 0x5590d6799730 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c02a930 tx=0x7fbb6c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa9e97b999b67c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:32.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.759+0000 7fbb5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:54:32.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.759+0000 7fbb5b7fe640 1 -- 172.21.15.164:0/2156844204 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c05b060 con 0x5590d6790360 2024-09-19T08:54:32.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.759+0000 7fbb5b7fe640 1 -- 172.21.15.164:0/2156844204 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7fbb6c0048d0 con 0x5590d6790360 2024-09-19T08:54:32.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.759+0000 7fbb84c38740 1 -- 172.21.15.164:0/2156844204 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb700a47a0 con 0x5590d6790360 2024-09-19T08:54:32.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.760+0000 7fbb5b7fe640 1 -- 172.21.15.164:0/2156844204 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbb6c040070 con 0x5590d6790360 2024-09-19T08:54:32.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.760+0000 7fbb84c38740 10 client.17289 fetch_fsmap learned FSMap version 32 2024-09-19T08:54:32.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.760+0000 7fbb84c38740 1 -- 172.21.15.164:0/2156844204 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5590d681a170 con 0x5590d6790360 2024-09-19T08:54:32.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.760+0000 7fbb5b7fe640 1 -- 172.21.15.164:0/2156844204 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb700a47a0 con 0x5590d6790360 2024-09-19T08:54:32.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.760+0000 7fbb5b7fe640 1 -- 172.21.15.164:0/2156844204 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fbb6c060470 con 0x5590d6790360 2024-09-19T08:54:32.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.760+0000 7fbb84c38740 10 client.17289 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:54:32.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.760+0000 7fbb84c38740 10 client.17289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:54:32.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.760+0000 7fbb84c38740 20 client.17289 populate_metadata read hostname 'smithi164' 2024-09-19T08:54:32.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.760+0000 7fbb84c38740 1 --2- 172.21.15.164:0/2156844204 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5590d681a890 0x5590d681cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:32.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.760+0000 7fbb84c38740 4 client.17289 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:54:32.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.760+0000 7fbb84c38740 1 -- 172.21.15.164:0/2156844204 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5590d667a210 con 0x5590d681a890 2024-09-19T08:54:32.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.760+0000 7fbb7b7fe640 1 --2- 172.21.15.164:0/2156844204 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5590d681a890 0x5590d681cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:32.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.761+0000 7fbb7b7fe640 1 --2- 172.21.15.164:0/2156844204 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5590d681a890 0x5590d681cc80 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a19de0e881d612e server_cookie=bcb428443cb1800e in_seq=0 out_seq=0 2024-09-19T08:54:32.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.761+0000 7fbb5b7fe640 10 client.17289 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:54:32.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.761+0000 7fbb5b7fe640 1 -- 172.21.15.164:0/2156844204 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5590d667a210 con 0x5590d681a890 2024-09-19T08:54:32.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.761+0000 7fbb5b7fe640 10 client.17289 handle_command_reply: tid=0 2024-09-19T08:54:32.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.761+0000 7fbb84c38740 1 client.17289 shutdown 2024-09-19T08:54:32.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.761+0000 7fbb84c38740 1 -- 172.21.15.164:0/2156844204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5590d6790360 msgr2=0x5590d6799730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:32.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.761+0000 7fbb84c38740 1 --2- 172.21.15.164:0/2156844204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5590d6790360 0x5590d6799730 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c02a930 tx=0x7fbb6c009260 comp rx=0 tx=0).stop 2024-09-19T08:54:32.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.761+0000 7fbb84c38740 1 -- 172.21.15.164:0/2156844204 shutdown_connections 2024-09-19T08:54:32.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.761+0000 7fbb84c38740 1 --2- 172.21.15.164:0/2156844204 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590d679fa90 0x5590d679a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.761+0000 7fbb84c38740 1 --2- 172.21.15.164:0/2156844204 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5590d681a890 0x5590d681cc80 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.761+0000 7fbb5b7fe640 0 client.17289 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:54:32.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.761+0000 7fbb84c38740 1 --2- 172.21.15.164:0/2156844204 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590d6790c80 0x5590d6799c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.761+0000 7fbb84c38740 1 --2- 172.21.15.164:0/2156844204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5590d6790360 0x5590d6799730 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.761+0000 7fbb84c38740 1 -- 172.21.15.164:0/2156844204 >> 172.21.15.164:0/2156844204 conn(0x5590d678ca30 msgr2=0x5590d678b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:32.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.761+0000 7fbb84c38740 1 -- 172.21.15.164:0/2156844204 shutdown_connections 2024-09-19T08:54:32.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.761+0000 7fbb84c38740 1 -- 172.21.15.164:0/2156844204 wait complete. 2024-09-19T08:54:32.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.761+0000 7fbb84c38740 20 client.17289 trim_cache size 0 max 16384 2024-09-19T08:54:32.778 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:54:32.778 INFO:teuthology.orchestra.run.smithi164.stdout: "status": "no active scrubs running", 2024-09-19T08:54:32.778 INFO:teuthology.orchestra.run.smithi164.stdout: "scrubs": {} 2024-09-19T08:54:32.778 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:54:32.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.762+0000 7fbb8285f640 1 -- 172.21.15.164:0/2006438443 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fbb640a33b0 msgr2=0x7fbb640a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:32.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.762+0000 7fbb8285f640 1 --2- 172.21.15.164:0/2006438443 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fbb640a33b0 0x7fbb640a5870 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbb74005fd0 tx=0x7fbb74004ab0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.762+0000 7fbb8285f640 1 -- 172.21.15.164:0/2006438443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb7c102780 msgr2=0x7fbb7c198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:32.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.762+0000 7fbb8285f640 1 --2- 172.21.15.164:0/2006438443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb7c102780 0x7fbb7c198980 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fbb7000d770 tx=0x7fbb7000dc30 comp rx=0 tx=0).stop 2024-09-19T08:54:32.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.762+0000 7fbb8285f640 1 -- 172.21.15.164:0/2006438443 shutdown_connections 2024-09-19T08:54:32.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.762+0000 7fbb8285f640 1 --2- 172.21.15.164:0/2006438443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb7c106f00 0x7fbb7c19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.762+0000 7fbb8285f640 1 --2- 172.21.15.164:0/2006438443 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fbb640a33b0 0x7fbb640a5870 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.762+0000 7fbb8285f640 1 --2- 172.21.15.164:0/2006438443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb7c102780 0x7fbb7c198980 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.762+0000 7fbb8285f640 1 --2- 172.21.15.164:0/2006438443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb7c101dd0 0x7fbb7c198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:32.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.762+0000 7fbb8285f640 1 -- 172.21.15.164:0/2006438443 >> 172.21.15.164:0/2006438443 conn(0x7fbb7c0fd850 msgr2=0x7fbb7c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:32.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.762+0000 7fbb8285f640 1 -- 172.21.15.164:0/2006438443 shutdown_connections 2024-09-19T08:54:32.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:32.762+0000 7fbb8285f640 1 -- 172.21.15.164:0/2006438443 wait complete. 2024-09-19T08:54:33.532 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T08:54:33.532 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:54:33.782 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:54:34.189 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:33 smithi164 ceph-mon[30928]: pgmap v271: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:54:34.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.269+0000 7fa959b1d640 1 -- 172.21.15.164:0/3633031013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa954105ad0 msgr2=0x7fa954105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:34.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.269+0000 7fa959b1d640 1 --2- 172.21.15.164:0/3633031013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa954105ad0 0x7fa954105eb0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fa94c009ca0 tx=0x7fa94c02a8c0 comp rx=0 tx=0).stop 2024-09-19T08:54:34.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.269+0000 7fa959b1d640 1 -- 172.21.15.164:0/3633031013 shutdown_connections 2024-09-19T08:54:34.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.269+0000 7fa959b1d640 1 --2- 172.21.15.164:0/3633031013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa954101ba0 0x7fa95410d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:34.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.269+0000 7fa959b1d640 1 --2- 172.21.15.164:0/3633031013 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa954101200 0x7fa954101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:34.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.269+0000 7fa959b1d640 1 --2- 172.21.15.164:0/3633031013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa954105ad0 0x7fa954105eb0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:34.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.269+0000 7fa959b1d640 1 -- 172.21.15.164:0/3633031013 >> 172.21.15.164:0/3633031013 conn(0x7fa954077210 msgr2=0x7fa9540fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:34.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.269+0000 7fa959b1d640 1 -- 172.21.15.164:0/3633031013 shutdown_connections 2024-09-19T08:54:34.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.269+0000 7fa959b1d640 1 -- 172.21.15.164:0/3633031013 wait complete. 2024-09-19T08:54:34.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.270+0000 7fa959b1d640 1 Processor -- start 2024-09-19T08:54:34.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.270+0000 7fa959b1d640 1 -- start start 2024-09-19T08:54:34.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.271+0000 7fa959b1d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa954101200 0x7fa954199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:34.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.271+0000 7fa959b1d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa954101ba0 0x7fa954199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:34.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.271+0000 7fa959b1d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa954105ad0 0x7fa954199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:34.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.271+0000 7fa959b1d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa954110eb0 con 0x7fa954105ad0 2024-09-19T08:54:34.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.271+0000 7fa959b1d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa954110d30 con 0x7fa954101ba0 2024-09-19T08:54:34.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.271+0000 7fa959b1d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa954111030 con 0x7fa954101200 2024-09-19T08:54:34.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.271+0000 7fa953fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa954105ad0 0x7fa954199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:34.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.271+0000 7fa953fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa954105ad0 0x7fa954199de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:34994/0 (socket says 172.21.15.164:34994) 2024-09-19T08:54:34.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.271+0000 7fa953fff640 1 -- 172.21.15.164:0/1837380538 learned_addr learned my addr 172.21.15.164:0/1837380538 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:34.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.271+0000 7fa952ffd640 1 --2- 172.21.15.164:0/1837380538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa954101ba0 0x7fa954199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:34.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.271+0000 7fa9537fe640 1 --2- 172.21.15.164:0/1837380538 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa954101200 0x7fa954199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:34.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.271+0000 7fa953fff640 1 -- 172.21.15.164:0/1837380538 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa954101200 msgr2=0x7fa954199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:34.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.271+0000 7fa953fff640 1 --2- 172.21.15.164:0/1837380538 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa954101200 0x7fa954199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:34.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.271+0000 7fa953fff640 1 -- 172.21.15.164:0/1837380538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa954101ba0 msgr2=0x7fa954199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:34.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.271+0000 7fa953fff640 1 --2- 172.21.15.164:0/1837380538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa954101ba0 0x7fa954199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:34.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.271+0000 7fa953fff640 1 -- 172.21.15.164:0/1837380538 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9541a4720 con 0x7fa954105ad0 2024-09-19T08:54:34.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.271+0000 7fa952ffd640 1 --2- 172.21.15.164:0/1837380538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa954101ba0 0x7fa954199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:54:34.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.271+0000 7fa953fff640 1 --2- 172.21.15.164:0/1837380538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa954105ad0 0x7fa954199de0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fa94400d720 tx=0x7fa94400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:34.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.271+0000 7fa950ff9640 1 -- 172.21.15.164:0/1837380538 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa944017070 con 0x7fa954105ad0 2024-09-19T08:54:34.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.271+0000 7fa950ff9640 1 -- 172.21.15.164:0/1837380538 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa94400de30 con 0x7fa954105ad0 2024-09-19T08:54:34.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.271+0000 7fa959b1d640 1 -- 172.21.15.164:0/1837380538 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9541a4a10 con 0x7fa954105ad0 2024-09-19T08:54:34.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.271+0000 7fa950ff9640 1 -- 172.21.15.164:0/1837380538 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa944004d90 con 0x7fa954105ad0 2024-09-19T08:54:34.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.271+0000 7fa959b1d640 1 -- 172.21.15.164:0/1837380538 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9541a4f00 con 0x7fa954105ad0 2024-09-19T08:54:34.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.272+0000 7fa959b1d640 1 -- 172.21.15.164:0/1837380538 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa954101660 con 0x7fa954105ad0 2024-09-19T08:54:34.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.273+0000 7fa950ff9640 1 -- 172.21.15.164:0/1837380538 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fa944004f30 con 0x7fa954105ad0 2024-09-19T08:54:34.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.274+0000 7fa950ff9640 1 --2- 172.21.15.164:0/1837380538 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa9400a33b0 0x7fa9400a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:34.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.274+0000 7fa950ff9640 1 -- 172.21.15.164:0/1837380538 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7fa9440cf660 con 0x7fa954105ad0 2024-09-19T08:54:34.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.274+0000 7fa9537fe640 1 --2- 172.21.15.164:0/1837380538 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa9400a33b0 0x7fa9400a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:34.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.275+0000 7fa9537fe640 1 --2- 172.21.15.164:0/1837380538 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa9400a33b0 0x7fa9400a5870 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa94c000f80 tx=0x7fa94c031000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:34.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.279+0000 7fa950ff9640 1 -- 172.21.15.164:0/1837380538 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa94409c080 con 0x7fa954105ad0 2024-09-19T08:54:34.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:33 smithi170 ceph-mon[35072]: pgmap v271: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:54:34.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:33 smithi173 ceph-mon[35306]: pgmap v271: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:54:34.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.480+0000 7fa959b1d640 1 -- 172.21.15.164:0/1837380538 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa954109010 con 0x7fa954105ad0 2024-09-19T08:54:34.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.480+0000 7fa950ff9640 1 -- 172.21.15.164:0/1837380538 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11984 (secure 0 0 0) 0x7fa94409b980 con 0x7fa954105ad0 2024-09-19T08:54:34.480 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:54:34.480 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":32,"btime":"2024-09-19T08:52:27:192804+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:52:27.074168+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:54:34.481 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 32 2024-09-19T08:54:34.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.482+0000 7fa959b1d640 1 -- 172.21.15.164:0/1837380538 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa9400a33b0 msgr2=0x7fa9400a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:34.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.482+0000 7fa959b1d640 1 --2- 172.21.15.164:0/1837380538 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa9400a33b0 0x7fa9400a5870 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa94c000f80 tx=0x7fa94c031000 comp rx=0 tx=0).stop 2024-09-19T08:54:34.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.482+0000 7fa959b1d640 1 -- 172.21.15.164:0/1837380538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa954105ad0 msgr2=0x7fa954199de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:34.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.482+0000 7fa959b1d640 1 --2- 172.21.15.164:0/1837380538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa954105ad0 0x7fa954199de0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fa94400d720 tx=0x7fa94400dbe0 comp rx=0 tx=0).stop 2024-09-19T08:54:34.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.482+0000 7fa959b1d640 1 -- 172.21.15.164:0/1837380538 shutdown_connections 2024-09-19T08:54:34.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.482+0000 7fa959b1d640 1 --2- 172.21.15.164:0/1837380538 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa9400a33b0 0x7fa9400a5870 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:34.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.482+0000 7fa959b1d640 1 --2- 172.21.15.164:0/1837380538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa954105ad0 0x7fa954199de0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:34.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.482+0000 7fa959b1d640 1 --2- 172.21.15.164:0/1837380538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa954101ba0 0x7fa954199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:34.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.483+0000 7fa959b1d640 1 --2- 172.21.15.164:0/1837380538 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa954101200 0x7fa954199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:34.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.483+0000 7fa959b1d640 1 -- 172.21.15.164:0/1837380538 >> 172.21.15.164:0/1837380538 conn(0x7fa954077210 msgr2=0x7fa954103ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:34.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.483+0000 7fa959b1d640 1 -- 172.21.15.164:0/1837380538 shutdown_connections 2024-09-19T08:54:34.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:34.483+0000 7fa959b1d640 1 -- 172.21.15.164:0/1837380538 wait complete. 2024-09-19T08:54:35.143 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:34 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/1837380538' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:54:35.171 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T08:54:35.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:34 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/1837380538' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:54:35.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:34 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/1837380538' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:54:35.420 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:54:35.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.888+0000 7f5f3b7df640 1 -- 172.21.15.164:0/946212102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f34074590 msgr2=0x7f5f34074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:35.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.888+0000 7f5f3b7df640 1 --2- 172.21.15.164:0/946212102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f34074590 0x7f5f34074970 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f5f30009900 tx=0x7f5f3002a480 comp rx=0 tx=0).stop 2024-09-19T08:54:35.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.888+0000 7f5f3b7df640 1 -- 172.21.15.164:0/946212102 shutdown_connections 2024-09-19T08:54:35.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.888+0000 7f5f3b7df640 1 --2- 172.21.15.164:0/946212102 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f34112b90 0x7f5f34114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:35.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.888+0000 7f5f3b7df640 1 --2- 172.21.15.164:0/946212102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f34074eb0 0x7f5f34112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:35.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.888+0000 7f5f3b7df640 1 --2- 172.21.15.164:0/946212102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f34074590 0x7f5f34074970 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:35.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.888+0000 7f5f3b7df640 1 -- 172.21.15.164:0/946212102 >> 172.21.15.164:0/946212102 conn(0x7f5f340fd8c0 msgr2=0x7f5f340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:35.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.889+0000 7f5f3b7df640 1 -- 172.21.15.164:0/946212102 shutdown_connections 2024-09-19T08:54:35.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.889+0000 7f5f3b7df640 1 -- 172.21.15.164:0/946212102 wait complete. 2024-09-19T08:54:35.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.889+0000 7f5f3b7df640 1 Processor -- start 2024-09-19T08:54:35.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.889+0000 7f5f3b7df640 1 -- start start 2024-09-19T08:54:35.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.889+0000 7f5f3b7df640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f34074590 0x7f5f341a4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:35.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.889+0000 7f5f3b7df640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f34074eb0 0x7f5f341a4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:35.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.889+0000 7f5f3b7df640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f34112b90 0x7f5f3419ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:35.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.889+0000 7f5f3b7df640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5f34118650 con 0x7f5f34074590 2024-09-19T08:54:35.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.889+0000 7f5f3b7df640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341184d0 con 0x7f5f34112b90 2024-09-19T08:54:35.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.889+0000 7f5f3b7df640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341187d0 con 0x7f5f34074eb0 2024-09-19T08:54:35.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.889+0000 7f5f3957b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f34074590 0x7f5f341a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:35.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.889+0000 7f5f3957b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f34074590 0x7f5f341a4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35006/0 (socket says 172.21.15.164:35006) 2024-09-19T08:54:35.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.889+0000 7f5f3957b640 1 -- 172.21.15.164:0/1255230382 learned_addr learned my addr 172.21.15.164:0/1255230382 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:35.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.890+0000 7f5f38d7a640 1 --2- 172.21.15.164:0/1255230382 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f34074eb0 0x7f5f341a4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:35.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.890+0000 7f5f39d7c640 1 --2- 172.21.15.164:0/1255230382 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f34112b90 0x7f5f3419ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:35.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.890+0000 7f5f3957b640 1 -- 172.21.15.164:0/1255230382 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f34074eb0 msgr2=0x7f5f341a4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:35.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.890+0000 7f5f3957b640 1 --2- 172.21.15.164:0/1255230382 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f34074eb0 0x7f5f341a4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:35.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.890+0000 7f5f3957b640 1 -- 172.21.15.164:0/1255230382 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f34112b90 msgr2=0x7f5f3419ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:35.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.890+0000 7f5f3957b640 1 --2- 172.21.15.164:0/1255230382 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f34112b90 0x7f5f3419ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:35.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.890+0000 7f5f3957b640 1 -- 172.21.15.164:0/1255230382 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f3419f150 con 0x7f5f34074590 2024-09-19T08:54:35.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.890+0000 7f5f39d7c640 1 --2- 172.21.15.164:0/1255230382 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f34112b90 0x7f5f3419ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:54:35.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.890+0000 7f5f3957b640 1 --2- 172.21.15.164:0/1255230382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f34074590 0x7f5f341a4860 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f5f30005ab0 tx=0x7f5f300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:35.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.890+0000 7f5f2a7fc640 1 -- 172.21.15.164:0/1255230382 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3003f030 con 0x7f5f34074590 2024-09-19T08:54:35.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.890+0000 7f5f2a7fc640 1 -- 172.21.15.164:0/1255230382 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f30003ce0 con 0x7f5f34074590 2024-09-19T08:54:35.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.890+0000 7f5f3b7df640 1 -- 172.21.15.164:0/1255230382 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f3419f3e0 con 0x7f5f34074590 2024-09-19T08:54:35.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.890+0000 7f5f2a7fc640 1 -- 172.21.15.164:0/1255230382 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f30035070 con 0x7f5f34074590 2024-09-19T08:54:35.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.890+0000 7f5f3b7df640 1 -- 172.21.15.164:0/1255230382 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f341ab730 con 0x7f5f34074590 2024-09-19T08:54:35.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.892+0000 7f5f2a7fc640 1 -- 172.21.15.164:0/1255230382 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f5f30042030 con 0x7f5f34074590 2024-09-19T08:54:35.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.893+0000 7f5f3dbb8740 1 Processor -- start 2024-09-19T08:54:35.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.893+0000 7f5f3dbb8740 1 -- start start 2024-09-19T08:54:35.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.893+0000 7f5f3dbb8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628f61cfb40 0x5628f61ce090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:35.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.893+0000 7f5f3dbb8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628f61ce5d0 0x5628f61cea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:35.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.893+0000 7f5f3dbb8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628f61d9370 0x5628f61db760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:35.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.893+0000 7f5f3dbb8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5628f61342f0 con 0x5628f61ce5d0 2024-09-19T08:54:35.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.893+0000 7f5f3dbb8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5628f60dbcf0 con 0x5628f61cfb40 2024-09-19T08:54:35.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.893+0000 7f5f3dbb8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5628f61302b0 con 0x5628f61d9370 2024-09-19T08:54:35.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.893+0000 7f5f3957b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628f61ce5d0 0x5628f61cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:35.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.893+0000 7f5f3957b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628f61ce5d0 0x5628f61cea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35016/0 (socket says 172.21.15.164:35016) 2024-09-19T08:54:35.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.893+0000 7f5f3957b640 1 -- 172.21.15.164:0/3932003694 learned_addr learned my addr 172.21.15.164:0/3932003694 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:35.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.893+0000 7f5f39d7c640 1 --2- 172.21.15.164:0/3932003694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628f61cfb40 0x5628f61ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:35.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.893+0000 7f5f38d7a640 1 --2- 172.21.15.164:0/3932003694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628f61d9370 0x5628f61db760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:35.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.893+0000 7f5f2a7fc640 1 --2- 172.21.15.164:0/1255230382 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f5f200a33b0 0x7f5f200a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:35.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.893+0000 7f5f2a7fc640 1 -- 172.21.15.164:0/1255230382 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7f5f300f0680 con 0x7f5f34074590 2024-09-19T08:54:35.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.893+0000 7f5f3957b640 1 -- 172.21.15.164:0/3932003694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628f61d9370 msgr2=0x5628f61db760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:35.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.893+0000 7f5f3957b640 1 --2- 172.21.15.164:0/3932003694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628f61d9370 0x5628f61db760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:35.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.893+0000 7f5f3957b640 1 -- 172.21.15.164:0/3932003694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628f61cfb40 msgr2=0x5628f61ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:35.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.893+0000 7f5f3957b640 1 --2- 172.21.15.164:0/3932003694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628f61cfb40 0x5628f61ce090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:35.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.893+0000 7f5f3957b640 1 -- 172.21.15.164:0/3932003694 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628f60c03a0 con 0x5628f61ce5d0 2024-09-19T08:54:35.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.893+0000 7f5f39d7c640 1 --2- 172.21.15.164:0/3932003694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628f61cfb40 0x5628f61ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:54:35.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.893+0000 7f5f38d7a640 1 --2- 172.21.15.164:0/3932003694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628f61d9370 0x5628f61db760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:54:35.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.894+0000 7f5f39d7c640 1 --2- 172.21.15.164:0/1255230382 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f5f200a33b0 0x7f5f200a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:35.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.894+0000 7f5f3957b640 1 --2- 172.21.15.164:0/3932003694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628f61ce5d0 0x5628f61cea10 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f5f300edd80 tx=0x7f5f3002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d61a22a943551a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:35.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.894+0000 7f5f1f7fe640 1 -- 172.21.15.164:0/3932003694 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f30049070 con 0x5628f61ce5d0 2024-09-19T08:54:35.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.894+0000 7f5f1f7fe640 1 -- 172.21.15.164:0/3932003694 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f300bd0a0 con 0x5628f61ce5d0 2024-09-19T08:54:35.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.894+0000 7f5f1f7fe640 1 -- 172.21.15.164:0/3932003694 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f300c5610 con 0x5628f61ce5d0 2024-09-19T08:54:35.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.895+0000 7f5f3dbb8740 1 -- 172.21.15.164:0/3932003694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628f61ce5d0 msgr2=0x5628f61cea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:35.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.895+0000 7f5f39d7c640 1 --2- 172.21.15.164:0/1255230382 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f5f200a33b0 0x7f5f200a5870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5f34076600 tx=0x7f5f2c004b50 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:35.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.895+0000 7f5f3dbb8740 1 --2- 172.21.15.164:0/3932003694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628f61ce5d0 0x5628f61cea10 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f5f300edd80 tx=0x7f5f3002a480 comp rx=0 tx=0).stop 2024-09-19T08:54:35.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.895+0000 7f5f3dbb8740 1 -- 172.21.15.164:0/3932003694 shutdown_connections 2024-09-19T08:54:35.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.895+0000 7f5f3dbb8740 1 --2- 172.21.15.164:0/3932003694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628f61d9370 0x5628f61db760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:35.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.895+0000 7f5f3dbb8740 1 --2- 172.21.15.164:0/3932003694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628f61ce5d0 0x5628f61cea10 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:35.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.895+0000 7f5f3dbb8740 1 --2- 172.21.15.164:0/3932003694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628f61cfb40 0x5628f61ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:35.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.895+0000 7f5f3dbb8740 1 -- 172.21.15.164:0/3932003694 >> 172.21.15.164:0/3932003694 conn(0x5628f6197310 msgr2=0x5628f61cac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:35.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.895+0000 7f5f3dbb8740 1 -- 172.21.15.164:0/3932003694 shutdown_connections 2024-09-19T08:54:35.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.895+0000 7f5f3dbb8740 1 -- 172.21.15.164:0/3932003694 wait complete. 2024-09-19T08:54:35.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.895+0000 7f5f3dbb8740 1 -- start start 2024-09-19T08:54:35.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.895+0000 7f5f3dbb8740 1 Processor -- start 2024-09-19T08:54:35.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.895+0000 7f5f3dbb8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628f61ce5d0 0x5628f61d8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:35.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.896+0000 7f5f3dbb8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628f61cfb40 0x5628f61d60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:35.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.896+0000 7f5f3dbb8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628f61d9370 0x5628f61d6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:35.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.896+0000 7f5f3dbb8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5628f61dbe30 con 0x5628f61d9370 2024-09-19T08:54:35.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.896+0000 7f5f3dbb8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5628f60ef060 con 0x5628f61cfb40 2024-09-19T08:54:35.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.896+0000 7f5f3dbb8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5628f61dbfb0 con 0x5628f61ce5d0 2024-09-19T08:54:35.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.896+0000 7f5f3dbb8740 10 client.0 _ll_register_callbacks cb 0x5628f5e428c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:54:35.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.896+0000 7f5f39d7c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628f61d9370 0x5628f61d6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:35.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.896+0000 7f5f39d7c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628f61d9370 0x5628f61d6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35030/0 (socket says 172.21.15.164:35030) 2024-09-19T08:54:35.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.896+0000 7f5f39d7c640 1 -- 172.21.15.164:0/2168243313 learned_addr learned my addr 172.21.15.164:0/2168243313 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:35.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.896+0000 7f5f3957b640 1 --2- 172.21.15.164:0/2168243313 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628f61ce5d0 0x5628f61d8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:35.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.896+0000 7f5f38d7a640 1 --2- 172.21.15.164:0/2168243313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628f61cfb40 0x5628f61d60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:35.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.896+0000 7f5f39d7c640 1 -- 172.21.15.164:0/2168243313 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628f61ce5d0 msgr2=0x5628f61d8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:35.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.896+0000 7f5f39d7c640 1 --2- 172.21.15.164:0/2168243313 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628f61ce5d0 0x5628f61d8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:35.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.896+0000 7f5f39d7c640 1 -- 172.21.15.164:0/2168243313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628f61cfb40 msgr2=0x5628f61d60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:35.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.896+0000 7f5f39d7c640 1 --2- 172.21.15.164:0/2168243313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628f61cfb40 0x5628f61d60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:35.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.896+0000 7f5f39d7c640 1 -- 172.21.15.164:0/2168243313 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628f5ea53c0 con 0x5628f61d9370 2024-09-19T08:54:35.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.896+0000 7f5f39d7c640 1 --2- 172.21.15.164:0/2168243313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628f61d9370 0x5628f61d6860 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c011ef0 tx=0x7f5f2c007740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94273eec688d9960 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:35.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.896+0000 7f5f1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:54:35.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.896+0000 7f5f1cff9640 1 -- 172.21.15.164:0/2168243313 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c021070 con 0x5628f61d9370 2024-09-19T08:54:35.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.896+0000 7f5f3dbb8740 1 -- 172.21.15.164:0/2168243313 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f300bd0a0 con 0x5628f61d9370 2024-09-19T08:54:35.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.896+0000 7f5f1cff9640 1 -- 172.21.15.164:0/2168243313 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7f5f2c01d840 con 0x5628f61d9370 2024-09-19T08:54:35.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.896+0000 7f5f1cff9640 1 -- 172.21.15.164:0/2168243313 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5f2c02d050 con 0x5628f61d9370 2024-09-19T08:54:35.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.896+0000 7f5f3dbb8740 10 client.17307 fetch_fsmap learned FSMap version 32 2024-09-19T08:54:35.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.896+0000 7f5f3dbb8740 1 -- 172.21.15.164:0/2168243313 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5628f61e78f0 con 0x5628f61d9370 2024-09-19T08:54:35.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.897+0000 7f5f1cff9640 1 -- 172.21.15.164:0/2168243313 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f300bd0a0 con 0x5628f61d9370 2024-09-19T08:54:35.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.897+0000 7f5f1cff9640 1 -- 172.21.15.164:0/2168243313 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f5f2c01b970 con 0x5628f61d9370 2024-09-19T08:54:35.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.897+0000 7f5f3dbb8740 10 client.17307 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:54:35.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.897+0000 7f5f3dbb8740 10 client.17307 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-19T08:54:35.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.897+0000 7f5f3dbb8740 20 client.17307 populate_metadata read hostname 'smithi164' 2024-09-19T08:54:35.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.897+0000 7f5f3dbb8740 1 --2- 172.21.15.164:0/2168243313 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5628f61e8da0 0x5628f61eb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:35.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.897+0000 7f5f3dbb8740 4 client.17307 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:54:35.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.897+0000 7f5f3dbb8740 1 -- 172.21.15.164:0/2168243313 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628f6141860 con 0x5628f61e8da0 2024-09-19T08:54:35.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.897+0000 7f5f3957b640 1 --2- 172.21.15.164:0/2168243313 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5628f61e8da0 0x5628f61eb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:35.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.897+0000 7f5f3957b640 1 --2- 172.21.15.164:0/2168243313 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5628f61e8da0 0x5628f61eb1e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b01a8958fbf7008e server_cookie=923d868441a1115a in_seq=0 out_seq=0 2024-09-19T08:54:35.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.897+0000 7f5f1cff9640 10 client.17307 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:54:35.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.898+0000 7f5f1cff9640 1 -- 172.21.15.164:0/2168243313 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5628f6141860 con 0x5628f61e8da0 2024-09-19T08:54:35.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.898+0000 7f5f1cff9640 10 client.17307 handle_command_reply: tid=0 2024-09-19T08:54:35.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.898+0000 7f5f3dbb8740 1 client.17307 shutdown 2024-09-19T08:54:35.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.898+0000 7f5f3dbb8740 1 -- 172.21.15.164:0/2168243313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628f61d9370 msgr2=0x5628f61d6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:35.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.898+0000 7f5f3dbb8740 1 --2- 172.21.15.164:0/2168243313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628f61d9370 0x5628f61d6860 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c011ef0 tx=0x7f5f2c007740 comp rx=0 tx=0).stop 2024-09-19T08:54:35.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.898+0000 7f5f3dbb8740 1 -- 172.21.15.164:0/2168243313 shutdown_connections 2024-09-19T08:54:35.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.898+0000 7f5f3dbb8740 1 --2- 172.21.15.164:0/2168243313 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5628f61e8da0 0x5628f61eb1e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:35.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.898+0000 7f5f3dbb8740 1 --2- 172.21.15.164:0/2168243313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628f61d9370 0x5628f61d6860 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:35.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.898+0000 7f5f3dbb8740 1 --2- 172.21.15.164:0/2168243313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628f61cfb40 0x5628f61d60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:35.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.898+0000 7f5f3dbb8740 1 --2- 172.21.15.164:0/2168243313 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628f61ce5d0 0x5628f61d8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:35.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.898+0000 7f5f3dbb8740 1 -- 172.21.15.164:0/2168243313 >> 172.21.15.164:0/2168243313 conn(0x5628f6197310 msgr2=0x5628f61d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:35.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.898+0000 7f5f1cff9640 0 client.17307 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:54:35.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.898+0000 7f5f3dbb8740 1 -- 172.21.15.164:0/2168243313 shutdown_connections 2024-09-19T08:54:35.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.898+0000 7f5f3dbb8740 1 -- 172.21.15.164:0/2168243313 wait complete. 2024-09-19T08:54:35.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.898+0000 7f5f3dbb8740 20 client.17307 trim_cache size 0 max 16384 2024-09-19T08:54:35.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.911+0000 7f5f3dbb8740 1 Processor -- start 2024-09-19T08:54:35.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.911+0000 7f5f3dbb8740 1 -- start start 2024-09-19T08:54:35.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.911+0000 7f5f3dbb8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628f61ce5d0 0x5628f61cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:35.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.911+0000 7f5f3dbb8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628f61cfb40 0x5628f61d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:35.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.911+0000 7f5f3dbb8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628f61d9370 0x5628f61d1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:35.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.911+0000 7f5f3dbb8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341b3f70 con 0x5628f61ce5d0 2024-09-19T08:54:35.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.911+0000 7f5f3dbb8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341b5ab0 con 0x5628f61cfb40 2024-09-19T08:54:35.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.911+0000 7f5f3dbb8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341b4640 con 0x5628f61d9370 2024-09-19T08:54:35.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.911+0000 7f5f3957b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628f61ce5d0 0x5628f61cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:35.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.911+0000 7f5f3957b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628f61ce5d0 0x5628f61cae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35036/0 (socket says 172.21.15.164:35036) 2024-09-19T08:54:35.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.911+0000 7f5f3957b640 1 -- 172.21.15.164:0/3972362245 learned_addr learned my addr 172.21.15.164:0/3972362245 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:35.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.912+0000 7f5f38d7a640 1 --2- 172.21.15.164:0/3972362245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628f61cfb40 0x5628f61d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:35.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.912+0000 7f5f3957b640 1 -- 172.21.15.164:0/3972362245 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628f61d9370 msgr2=0x5628f61d1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:35.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.912+0000 7f5f3957b640 1 --2- 172.21.15.164:0/3972362245 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628f61d9370 0x5628f61d1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:35.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.912+0000 7f5f3957b640 1 -- 172.21.15.164:0/3972362245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628f61cfb40 msgr2=0x5628f61d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:35.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.912+0000 7f5f3957b640 1 --2- 172.21.15.164:0/3972362245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628f61cfb40 0x5628f61d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:35.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.912+0000 7f5f3957b640 1 -- 172.21.15.164:0/3972362245 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628f61d4620 con 0x5628f61ce5d0 2024-09-19T08:54:35.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.912+0000 7f5f38d7a640 1 --2- 172.21.15.164:0/3972362245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628f61cfb40 0x5628f61d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:54:35.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.912+0000 7f5f3957b640 1 --2- 172.21.15.164:0/3972362245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628f61ce5d0 0x5628f61cae10 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f5f300bc6a0 tx=0x7f5f300f6550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3871bef20597c526 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:35.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.912+0000 7f5f1effd640 1 -- 172.21.15.164:0/3972362245 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f30049040 con 0x5628f61ce5d0 2024-09-19T08:54:35.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.912+0000 7f5f1effd640 1 -- 172.21.15.164:0/3972362245 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f300bd0a0 con 0x5628f61ce5d0 2024-09-19T08:54:35.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.912+0000 7f5f1effd640 1 -- 172.21.15.164:0/3972362245 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3003be40 con 0x5628f61ce5d0 2024-09-19T08:54:35.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.913+0000 7f5f3dbb8740 1 -- 172.21.15.164:0/3972362245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628f61ce5d0 msgr2=0x5628f61cae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:35.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.913+0000 7f5f3dbb8740 1 --2- 172.21.15.164:0/3972362245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628f61ce5d0 0x5628f61cae10 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f5f300bc6a0 tx=0x7f5f300f6550 comp rx=0 tx=0).stop 2024-09-19T08:54:35.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.913+0000 7f5f3dbb8740 1 -- 172.21.15.164:0/3972362245 shutdown_connections 2024-09-19T08:54:35.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.913+0000 7f5f3dbb8740 1 --2- 172.21.15.164:0/3972362245 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628f61d9370 0x5628f61d1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:35.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.913+0000 7f5f3dbb8740 1 --2- 172.21.15.164:0/3972362245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628f61cfb40 0x5628f61d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:35.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.913+0000 7f5f3dbb8740 1 --2- 172.21.15.164:0/3972362245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628f61ce5d0 0x5628f61cae10 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:35.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.913+0000 7f5f3dbb8740 1 -- 172.21.15.164:0/3972362245 >> 172.21.15.164:0/3972362245 conn(0x5628f6197310 msgr2=0x5628f61ca700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:35.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.913+0000 7f5f3dbb8740 1 -- 172.21.15.164:0/3972362245 shutdown_connections 2024-09-19T08:54:35.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.913+0000 7f5f3dbb8740 1 -- 172.21.15.164:0/3972362245 wait complete. 2024-09-19T08:54:35.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.913+0000 7f5f3dbb8740 1 -- start start 2024-09-19T08:54:35.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.913+0000 7f5f3dbb8740 1 Processor -- start 2024-09-19T08:54:35.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.914+0000 7f5f3dbb8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628f61ce5d0 0x5628f61e1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:35.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.914+0000 7f5f3dbb8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628f61cfb40 0x5628f61d2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:35.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.914+0000 7f5f3dbb8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628f61d9370 0x5628f61d3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:35.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.914+0000 7f5f3dbb8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341b53e0 con 0x5628f61d9370 2024-09-19T08:54:35.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.914+0000 7f5f3dbb8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341b4d10 con 0x5628f61cfb40 2024-09-19T08:54:35.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.914+0000 7f5f3dbb8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341b1f70 con 0x5628f61ce5d0 2024-09-19T08:54:35.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.914+0000 7f5f3dbb8740 10 client.0 _ll_register_callbacks cb 0x7f5f2c007af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:54:35.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.914+0000 7f5f39d7c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628f61d9370 0x5628f61d3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:35.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.914+0000 7f5f39d7c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628f61d9370 0x5628f61d3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35038/0 (socket says 172.21.15.164:35038) 2024-09-19T08:54:35.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.914+0000 7f5f39d7c640 1 -- 172.21.15.164:0/1973377205 learned_addr learned my addr 172.21.15.164:0/1973377205 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:35.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.914+0000 7f5f38d7a640 1 --2- 172.21.15.164:0/1973377205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628f61cfb40 0x5628f61d2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:35.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.914+0000 7f5f3957b640 1 --2- 172.21.15.164:0/1973377205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628f61ce5d0 0x5628f61e1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:35.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.914+0000 7f5f39d7c640 1 -- 172.21.15.164:0/1973377205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628f61ce5d0 msgr2=0x5628f61e1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:35.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.914+0000 7f5f39d7c640 1 --2- 172.21.15.164:0/1973377205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628f61ce5d0 0x5628f61e1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:35.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.914+0000 7f5f39d7c640 1 -- 172.21.15.164:0/1973377205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628f61cfb40 msgr2=0x5628f61d2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:35.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.914+0000 7f5f39d7c640 1 --2- 172.21.15.164:0/1973377205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628f61cfb40 0x5628f61d2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:35.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.914+0000 7f5f39d7c640 1 -- 172.21.15.164:0/1973377205 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628f61d3590 con 0x5628f61d9370 2024-09-19T08:54:35.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.914+0000 7f5f39d7c640 1 --2- 172.21.15.164:0/1973377205 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628f61d9370 0x5628f61d3050 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00fed0 tx=0x7f5f2c007440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84521051be14bf54 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:35.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.914+0000 7f5f1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:54:35.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.914+0000 7f5f1cff9640 1 -- 172.21.15.164:0/1973377205 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c036060 con 0x5628f61d9370 2024-09-19T08:54:35.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.914+0000 7f5f1cff9640 1 -- 172.21.15.164:0/1973377205 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7f5f2c01b5c0 con 0x5628f61d9370 2024-09-19T08:54:35.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.914+0000 7f5f3dbb8740 1 -- 172.21.15.164:0/1973377205 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f300bd0a0 con 0x5628f61d9370 2024-09-19T08:54:35.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.915+0000 7f5f1cff9640 1 -- 172.21.15.164:0/1973377205 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5f2c021070 con 0x5628f61d9370 2024-09-19T08:54:35.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.915+0000 7f5f3dbb8740 10 client.17313 fetch_fsmap learned FSMap version 32 2024-09-19T08:54:35.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.915+0000 7f5f3dbb8740 1 -- 172.21.15.164:0/1973377205 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5628f6253700 con 0x5628f61d9370 2024-09-19T08:54:35.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.915+0000 7f5f1cff9640 1 -- 172.21.15.164:0/1973377205 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f300bd0a0 con 0x5628f61d9370 2024-09-19T08:54:35.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.915+0000 7f5f1cff9640 1 -- 172.21.15.164:0/1973377205 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f5f2c01dc90 con 0x5628f61d9370 2024-09-19T08:54:35.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.915+0000 7f5f3dbb8740 10 client.17313 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:54:35.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.915+0000 7f5f3dbb8740 10 client.17313 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-19T08:54:35.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.915+0000 7f5f3dbb8740 20 client.17313 populate_metadata read hostname 'smithi164' 2024-09-19T08:54:35.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.915+0000 7f5f3dbb8740 1 --2- 172.21.15.164:0/1973377205 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5628f6253e80 0x5628f62562a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:35.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.915+0000 7f5f3dbb8740 4 client.17313 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:54:35.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.915+0000 7f5f3dbb8740 1 -- 172.21.15.164:0/1973377205 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5628f613bd70 con 0x5628f6253e80 2024-09-19T08:54:35.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.915+0000 7f5f3957b640 1 --2- 172.21.15.164:0/1973377205 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5628f6253e80 0x5628f62562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:35.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.916+0000 7f5f1cff9640 10 client.17313 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:54:35.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.916+0000 7f5f3957b640 1 --2- 172.21.15.164:0/1973377205 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5628f6253e80 0x5628f62562a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d8426c46809ece32 server_cookie=a642679a8a38b9e4 in_seq=0 out_seq=0 2024-09-19T08:54:35.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.916+0000 7f5f1cff9640 1 -- 172.21.15.164:0/1973377205 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5628f613bd70 con 0x5628f6253e80 2024-09-19T08:54:35.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.916+0000 7f5f1cff9640 10 client.17313 handle_command_reply: tid=0 2024-09-19T08:54:35.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.916+0000 7f5f3dbb8740 1 client.17313 shutdown 2024-09-19T08:54:35.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.916+0000 7f5f3dbb8740 1 -- 172.21.15.164:0/1973377205 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628f61d9370 msgr2=0x5628f61d3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:35.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.916+0000 7f5f3dbb8740 1 --2- 172.21.15.164:0/1973377205 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628f61d9370 0x5628f61d3050 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00fed0 tx=0x7f5f2c007440 comp rx=0 tx=0).stop 2024-09-19T08:54:35.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.916+0000 7f5f3dbb8740 1 -- 172.21.15.164:0/1973377205 shutdown_connections 2024-09-19T08:54:35.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.916+0000 7f5f3dbb8740 1 --2- 172.21.15.164:0/1973377205 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x5628f6253e80 0x5628f62562a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:35.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.916+0000 7f5f3dbb8740 1 --2- 172.21.15.164:0/1973377205 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628f61d9370 0x5628f61d3050 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:35.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.916+0000 7f5f3dbb8740 1 --2- 172.21.15.164:0/1973377205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628f61cfb40 0x5628f61d2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:35.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.916+0000 7f5f3dbb8740 1 --2- 172.21.15.164:0/1973377205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628f61ce5d0 0x5628f61e1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:35.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.916+0000 7f5f3dbb8740 1 -- 172.21.15.164:0/1973377205 >> 172.21.15.164:0/1973377205 conn(0x5628f6197310 msgr2=0x5628f61d7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:35.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.916+0000 7f5f1cff9640 0 client.17313 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:54:35.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.916+0000 7f5f3dbb8740 1 -- 172.21.15.164:0/1973377205 shutdown_connections 2024-09-19T08:54:35.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.916+0000 7f5f3dbb8740 1 -- 172.21.15.164:0/1973377205 wait complete. 2024-09-19T08:54:35.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.916+0000 7f5f3dbb8740 20 client.17313 trim_cache size 0 max 16384 2024-09-19T08:54:35.927 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:54:35.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.917+0000 7f5f3b7df640 1 -- 172.21.15.164:0/1255230382 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f5f200a33b0 msgr2=0x7f5f200a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:35.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.917+0000 7f5f3b7df640 1 --2- 172.21.15.164:0/1255230382 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f5f200a33b0 0x7f5f200a5870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5f34076600 tx=0x7f5f2c004b50 comp rx=0 tx=0).stop 2024-09-19T08:54:35.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.917+0000 7f5f3b7df640 1 -- 172.21.15.164:0/1255230382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f34074590 msgr2=0x7f5f341a4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:35.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.917+0000 7f5f3b7df640 1 --2- 172.21.15.164:0/1255230382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f34074590 0x7f5f341a4860 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f5f30005ab0 tx=0x7f5f300042f0 comp rx=0 tx=0).stop 2024-09-19T08:54:35.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.917+0000 7f5f3b7df640 1 -- 172.21.15.164:0/1255230382 shutdown_connections 2024-09-19T08:54:35.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.917+0000 7f5f3b7df640 1 --2- 172.21.15.164:0/1255230382 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f34112b90 0x7f5f3419ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:35.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.917+0000 7f5f3b7df640 1 --2- 172.21.15.164:0/1255230382 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f34074eb0 0x7f5f341a4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:35.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.917+0000 7f5f3b7df640 1 --2- 172.21.15.164:0/1255230382 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f5f200a33b0 0x7f5f200a5870 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:35.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.917+0000 7f5f3b7df640 1 --2- 172.21.15.164:0/1255230382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f34074590 0x7f5f341a4860 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:35.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.917+0000 7f5f3b7df640 1 -- 172.21.15.164:0/1255230382 >> 172.21.15.164:0/1255230382 conn(0x7f5f340fd8c0 msgr2=0x7f5f340ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:35.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.917+0000 7f5f3b7df640 1 -- 172.21.15.164:0/1255230382 shutdown_connections 2024-09-19T08:54:35.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:35.917+0000 7f5f3b7df640 1 -- 172.21.15.164:0/1255230382 wait complete. 2024-09-19T08:54:36.276 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:35 smithi164 ceph-mon[30928]: pgmap v272: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T08:54:36.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:35 smithi170 ceph-mon[35072]: pgmap v272: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T08:54:36.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:35 smithi173 ceph-mon[35306]: pgmap v272: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T08:54:36.628 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T08:54:36.874 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:54:37.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.378+0000 7fd3e24f0640 1 -- 172.21.15.164:0/3355133391 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3dc102570 msgr2=0x7fd3dc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:37.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.378+0000 7fd3e24f0640 1 --2- 172.21.15.164:0/3355133391 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3dc102570 0x7fd3dc1029b0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4009900 tx=0x7fd3c402a480 comp rx=0 tx=0).stop 2024-09-19T08:54:37.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.378+0000 7fd3e24f0640 1 -- 172.21.15.164:0/3355133391 shutdown_connections 2024-09-19T08:54:37.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.378+0000 7fd3e24f0640 1 --2- 172.21.15.164:0/3355133391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3dc102ef0 0x7fd3dc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:37.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.378+0000 7fd3e24f0640 1 --2- 172.21.15.164:0/3355133391 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3dc102570 0x7fd3dc1029b0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:37.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.378+0000 7fd3e24f0640 1 --2- 172.21.15.164:0/3355133391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3dc100e20 0x7fd3dc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:37.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.378+0000 7fd3e24f0640 1 -- 172.21.15.164:0/3355133391 >> 172.21.15.164:0/3355133391 conn(0x7fd3dc0fc7d0 msgr2=0x7fd3dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:37.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.378+0000 7fd3e24f0640 1 -- 172.21.15.164:0/3355133391 shutdown_connections 2024-09-19T08:54:37.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.378+0000 7fd3e24f0640 1 -- 172.21.15.164:0/3355133391 wait complete. 2024-09-19T08:54:37.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.379+0000 7fd3e24f0640 1 Processor -- start 2024-09-19T08:54:37.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.379+0000 7fd3e24f0640 1 -- start start 2024-09-19T08:54:37.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.379+0000 7fd3e24f0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3dc100e20 0x7fd3dc1973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:37.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.379+0000 7fd3e24f0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3dc102570 0x7fd3dc197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:37.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.379+0000 7fd3e24f0640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3dc102ef0 0x7fd3dc19bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:37.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.379+0000 7fd3e24f0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc110f80 con 0x7fd3dc102570 2024-09-19T08:54:37.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.379+0000 7fd3e24f0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc110e00 con 0x7fd3dc100e20 2024-09-19T08:54:37.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.379+0000 7fd3e24f0640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc111100 con 0x7fd3dc102ef0 2024-09-19T08:54:37.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.379+0000 7fd3d3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3dc102570 0x7fd3dc197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:37.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.379+0000 7fd3d3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3dc102570 0x7fd3dc197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35060/0 (socket says 172.21.15.164:35060) 2024-09-19T08:54:37.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.379+0000 7fd3d3fff640 1 -- 172.21.15.164:0/1341387307 learned_addr learned my addr 172.21.15.164:0/1341387307 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:37.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.379+0000 7fd3dbfff640 1 --2- 172.21.15.164:0/1341387307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3dc100e20 0x7fd3dc1973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:37.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.379+0000 7fd3e0a8d640 1 --2- 172.21.15.164:0/1341387307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3dc102ef0 0x7fd3dc19bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:37.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.380+0000 7fd3d3fff640 1 -- 172.21.15.164:0/1341387307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3dc102ef0 msgr2=0x7fd3dc19bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:37.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.380+0000 7fd3d3fff640 1 --2- 172.21.15.164:0/1341387307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3dc102ef0 0x7fd3dc19bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:37.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.380+0000 7fd3d3fff640 1 -- 172.21.15.164:0/1341387307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3dc100e20 msgr2=0x7fd3dc1973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:37.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.380+0000 7fd3d3fff640 1 --2- 172.21.15.164:0/1341387307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3dc100e20 0x7fd3dc1973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:37.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.380+0000 7fd3d3fff640 1 -- 172.21.15.164:0/1341387307 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3dc19c390 con 0x7fd3dc102570 2024-09-19T08:54:37.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.380+0000 7fd3d3fff640 1 --2- 172.21.15.164:0/1341387307 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3dc102570 0x7fd3dc197920 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4006010 tx=0x7fd3c4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:37.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.380+0000 7fd3d9ffb640 1 -- 172.21.15.164:0/1341387307 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c403f070 con 0x7fd3dc102570 2024-09-19T08:54:37.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.380+0000 7fd3d9ffb640 1 -- 172.21.15.164:0/1341387307 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3c4031c30 con 0x7fd3dc102570 2024-09-19T08:54:37.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.380+0000 7fd3e24f0640 1 -- 172.21.15.164:0/1341387307 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3dc19c620 con 0x7fd3dc102570 2024-09-19T08:54:37.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.380+0000 7fd3d9ffb640 1 -- 172.21.15.164:0/1341387307 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4035030 con 0x7fd3dc102570 2024-09-19T08:54:37.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.380+0000 7fd3e24f0640 1 -- 172.21.15.164:0/1341387307 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3dc1a3f60 con 0x7fd3dc102570 2024-09-19T08:54:37.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.382+0000 7fd3d9ffb640 1 -- 172.21.15.164:0/1341387307 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fd3c4042070 con 0x7fd3dc102570 2024-09-19T08:54:37.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.382+0000 7fd3e48c9740 1 Processor -- start 2024-09-19T08:54:37.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.383+0000 7fd3e48c9740 1 -- start start 2024-09-19T08:54:37.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.383+0000 7fd3e48c9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9ab79fb40 0x55f9ab79e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:37.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.383+0000 7fd3e48c9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9ab79e5d0 0x55f9ab79ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:37.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.383+0000 7fd3e48c9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9ab7ab580 0x55f9ab7ad970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:37.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.383+0000 7fd3e48c9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f9ab7042f0 con 0x55f9ab79fb40 2024-09-19T08:54:37.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.383+0000 7fd3e48c9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f9ab6abcf0 con 0x55f9ab7ab580 2024-09-19T08:54:37.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.383+0000 7fd3e48c9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f9ab7002b0 con 0x55f9ab79e5d0 2024-09-19T08:54:37.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.383+0000 7fd3e0a8d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9ab79fb40 0x55f9ab79e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:37.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.383+0000 7fd3d9ffb640 1 --2- 172.21.15.164:0/1341387307 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd3c80a3430 0x7fd3c80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:37.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.383+0000 7fd3e0a8d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9ab79fb40 0x55f9ab79e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35072/0 (socket says 172.21.15.164:35072) 2024-09-19T08:54:37.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.383+0000 7fd3e0a8d640 1 -- 172.21.15.164:0/1338485938 learned_addr learned my addr 172.21.15.164:0/1338485938 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:37.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.383+0000 7fd3dbfff640 1 --2- 172.21.15.164:0/1338485938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9ab79e5d0 0x55f9ab79ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:37.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.383+0000 7fd3e0a8d640 1 --2- 172.21.15.164:0/1341387307 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd3c80a3430 0x7fd3c80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:37.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.383+0000 7fd3d9ffb640 1 -- 172.21.15.164:0/1341387307 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7fd3c40f01a0 con 0x7fd3dc102570 2024-09-19T08:54:37.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.383+0000 7fd3d3fff640 1 --2- 172.21.15.164:0/1338485938 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9ab7ab580 0x55f9ab7ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:37.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.383+0000 7fd3e0a8d640 1 -- 172.21.15.164:0/1338485938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9ab79e5d0 msgr2=0x55f9ab79ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:37.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.383+0000 7fd3e0a8d640 1 --2- 172.21.15.164:0/1338485938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9ab79e5d0 0x55f9ab79ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:37.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.383+0000 7fd3e0a8d640 1 -- 172.21.15.164:0/1338485938 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9ab7ab580 msgr2=0x55f9ab7ad970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:37.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.383+0000 7fd3e0a8d640 1 --2- 172.21.15.164:0/1338485938 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9ab7ab580 0x55f9ab7ad970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:37.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.383+0000 7fd3e0a8d640 1 -- 172.21.15.164:0/1338485938 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9ab6903a0 con 0x55f9ab79fb40 2024-09-19T08:54:37.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.383+0000 7fd3d3fff640 1 --2- 172.21.15.164:0/1338485938 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9ab7ab580 0x55f9ab7ad970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:54:37.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.383+0000 7fd3e0a8d640 1 --2- 172.21.15.164:0/1338485938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9ab79fb40 0x55f9ab79e090 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4014960 tx=0x7fd3d4014e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9eda2fcd91f38d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:37.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.384+0000 7fd3d27fc640 1 -- 172.21.15.164:0/1338485938 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4020ae0 con 0x55f9ab79fb40 2024-09-19T08:54:37.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.384+0000 7fd3d27fc640 1 -- 172.21.15.164:0/1338485938 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3d4020c80 con 0x55f9ab79fb40 2024-09-19T08:54:37.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.384+0000 7fd3d27fc640 1 -- 172.21.15.164:0/1338485938 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d40285f0 con 0x55f9ab79fb40 2024-09-19T08:54:37.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.384+0000 7fd3e48c9740 1 -- 172.21.15.164:0/1338485938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9ab79fb40 msgr2=0x55f9ab79e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:37.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.384+0000 7fd3e0a8d640 1 --2- 172.21.15.164:0/1341387307 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd3c80a3430 0x7fd3c80a58f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4029000 tx=0x7fd3d4026000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:37.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.384+0000 7fd3e48c9740 1 --2- 172.21.15.164:0/1338485938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9ab79fb40 0x55f9ab79e090 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4014960 tx=0x7fd3d4014e20 comp rx=0 tx=0).stop 2024-09-19T08:54:37.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.384+0000 7fd3e48c9740 1 -- 172.21.15.164:0/1338485938 shutdown_connections 2024-09-19T08:54:37.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.384+0000 7fd3e48c9740 1 --2- 172.21.15.164:0/1338485938 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9ab7ab580 0x55f9ab7ad970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:37.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.384+0000 7fd3e48c9740 1 --2- 172.21.15.164:0/1338485938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9ab79e5d0 0x55f9ab79ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:37.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.384+0000 7fd3e48c9740 1 --2- 172.21.15.164:0/1338485938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9ab79fb40 0x55f9ab79e090 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:37.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.384+0000 7fd3e48c9740 1 -- 172.21.15.164:0/1338485938 >> 172.21.15.164:0/1338485938 conn(0x55f9ab767310 msgr2=0x55f9ab79ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:37.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.385+0000 7fd3e48c9740 1 -- 172.21.15.164:0/1338485938 shutdown_connections 2024-09-19T08:54:37.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.385+0000 7fd3e48c9740 1 -- 172.21.15.164:0/1338485938 wait complete. 2024-09-19T08:54:37.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.385+0000 7fd3e48c9740 1 -- start start 2024-09-19T08:54:37.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.385+0000 7fd3e48c9740 1 Processor -- start 2024-09-19T08:54:37.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.385+0000 7fd3e48c9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9ab79e5d0 0x55f9ab7a66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:37.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.385+0000 7fd3e48c9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9ab79fb40 0x55f9ab7a6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:37.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.385+0000 7fd3e48c9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9ab7ab580 0x55f9ab7a7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:37.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.385+0000 7fd3e48c9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f9ab7a1550 con 0x55f9ab79e5d0 2024-09-19T08:54:37.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.385+0000 7fd3e48c9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f9ab6bf060 con 0x55f9ab7ab580 2024-09-19T08:54:37.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.385+0000 7fd3e48c9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f9ab7a16d0 con 0x55f9ab79fb40 2024-09-19T08:54:37.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.385+0000 7fd3e48c9740 10 client.0 _ll_register_callbacks cb 0x55f9ab4128c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:54:37.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.385+0000 7fd3dbfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9ab79e5d0 0x55f9ab7a66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:37.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.386+0000 7fd3dbfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9ab79e5d0 0x55f9ab7a66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35084/0 (socket says 172.21.15.164:35084) 2024-09-19T08:54:37.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.386+0000 7fd3dbfff640 1 -- 172.21.15.164:0/267045451 learned_addr learned my addr 172.21.15.164:0/267045451 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:37.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.386+0000 7fd3e0a8d640 1 --2- 172.21.15.164:0/267045451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9ab7ab580 0x55f9ab7a7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:37.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.386+0000 7fd3d3fff640 1 --2- 172.21.15.164:0/267045451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9ab79fb40 0x55f9ab7a6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:37.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.386+0000 7fd3dbfff640 1 -- 172.21.15.164:0/267045451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9ab79fb40 msgr2=0x55f9ab7a6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:37.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.386+0000 7fd3dbfff640 1 --2- 172.21.15.164:0/267045451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9ab79fb40 0x55f9ab7a6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:37.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.386+0000 7fd3dbfff640 1 -- 172.21.15.164:0/267045451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9ab7ab580 msgr2=0x55f9ab7a7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:37.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.386+0000 7fd3dbfff640 1 --2- 172.21.15.164:0/267045451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9ab7ab580 0x55f9ab7a7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:37.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.386+0000 7fd3dbfff640 1 -- 172.21.15.164:0/267045451 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9ab4753c0 con 0x55f9ab79e5d0 2024-09-19T08:54:37.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.386+0000 7fd3e0a8d640 1 --2- 172.21.15.164:0/267045451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9ab7ab580 0x55f9ab7a7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:54:37.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.386+0000 7fd3dbfff640 1 --2- 172.21.15.164:0/267045451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9ab79e5d0 0x55f9ab7a66d0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc0114c0 tx=0x7fd3cc011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8543c0de3149ae69 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:37.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.386+0000 7fd3bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:54:37.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.386+0000 7fd3bb7fe640 1 -- 172.21.15.164:0/267045451 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc004020 con 0x55f9ab79e5d0 2024-09-19T08:54:37.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.386+0000 7fd3bb7fe640 1 -- 172.21.15.164:0/267045451 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7fd3cc0041c0 con 0x55f9ab79e5d0 2024-09-19T08:54:37.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.386+0000 7fd3e48c9740 1 -- 172.21.15.164:0/267045451 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3d4020c80 con 0x55f9ab79e5d0 2024-09-19T08:54:37.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.386+0000 7fd3bb7fe640 1 -- 172.21.15.164:0/267045451 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd3cc0043c0 con 0x55f9ab79e5d0 2024-09-19T08:54:37.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.386+0000 7fd3e48c9740 10 client.17325 fetch_fsmap learned FSMap version 32 2024-09-19T08:54:37.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.386+0000 7fd3e48c9740 1 -- 172.21.15.164:0/267045451 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f9ab7b7b10 con 0x55f9ab79e5d0 2024-09-19T08:54:37.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.386+0000 7fd3bb7fe640 1 -- 172.21.15.164:0/267045451 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3d4020c80 con 0x55f9ab79e5d0 2024-09-19T08:54:37.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.387+0000 7fd3bb7fe640 1 -- 172.21.15.164:0/267045451 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fd3cc003b20 con 0x55f9ab79e5d0 2024-09-19T08:54:37.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.387+0000 7fd3e48c9740 10 client.17325 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:54:37.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.387+0000 7fd3e48c9740 10 client.17325 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T08:54:37.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.387+0000 7fd3e48c9740 20 client.17325 populate_metadata read hostname 'smithi164' 2024-09-19T08:54:37.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.387+0000 7fd3e48c9740 1 --2- 172.21.15.164:0/267045451 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55f9ab7b8fd0 0x55f9ab7bb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:37.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.387+0000 7fd3e48c9740 4 client.17325 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:54:37.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.387+0000 7fd3e48c9740 1 -- 172.21.15.164:0/267045451 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9ab711860 con 0x55f9ab7b8fd0 2024-09-19T08:54:37.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.387+0000 7fd3d3fff640 1 --2- 172.21.15.164:0/267045451 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55f9ab7b8fd0 0x55f9ab7bb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:37.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.387+0000 7fd3bb7fe640 10 client.17325 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:54:37.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.387+0000 7fd3d3fff640 1 --2- 172.21.15.164:0/267045451 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55f9ab7b8fd0 0x55f9ab7bb420 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b0ad954a65950974 server_cookie=c49c4154cab03a in_seq=0 out_seq=0 2024-09-19T08:54:37.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.388+0000 7fd3bb7fe640 1 -- 172.21.15.164:0/267045451 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f9ab711860 con 0x55f9ab7b8fd0 2024-09-19T08:54:37.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.388+0000 7fd3bb7fe640 10 client.17325 handle_command_reply: tid=0 2024-09-19T08:54:37.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.388+0000 7fd3e48c9740 1 client.17325 shutdown 2024-09-19T08:54:37.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.388+0000 7fd3e48c9740 1 -- 172.21.15.164:0/267045451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9ab79e5d0 msgr2=0x55f9ab7a66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:37.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.388+0000 7fd3e48c9740 1 --2- 172.21.15.164:0/267045451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9ab79e5d0 0x55f9ab7a66d0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc0114c0 tx=0x7fd3cc011980 comp rx=0 tx=0).stop 2024-09-19T08:54:37.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.388+0000 7fd3e48c9740 1 -- 172.21.15.164:0/267045451 shutdown_connections 2024-09-19T08:54:37.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.388+0000 7fd3e48c9740 1 --2- 172.21.15.164:0/267045451 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55f9ab7b8fd0 0x55f9ab7bb420 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:37.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.388+0000 7fd3e48c9740 1 --2- 172.21.15.164:0/267045451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9ab7ab580 0x55f9ab7a7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:37.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.388+0000 7fd3e48c9740 1 --2- 172.21.15.164:0/267045451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9ab79fb40 0x55f9ab7a6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:37.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.388+0000 7fd3bb7fe640 0 client.17325 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:54:37.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.388+0000 7fd3e48c9740 1 --2- 172.21.15.164:0/267045451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9ab79e5d0 0x55f9ab7a66d0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:37.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.388+0000 7fd3e48c9740 1 -- 172.21.15.164:0/267045451 >> 172.21.15.164:0/267045451 conn(0x55f9ab767310 msgr2=0x55f9ab7a0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:37.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.388+0000 7fd3e48c9740 1 -- 172.21.15.164:0/267045451 shutdown_connections 2024-09-19T08:54:37.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.388+0000 7fd3e48c9740 1 -- 172.21.15.164:0/267045451 wait complete. 2024-09-19T08:54:37.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.389+0000 7fd3e48c9740 20 client.17325 trim_cache size 0 max 16384 2024-09-19T08:54:37.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.401+0000 7fd3e48c9740 1 Processor -- start 2024-09-19T08:54:37.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.401+0000 7fd3e48c9740 1 -- start start 2024-09-19T08:54:37.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.402+0000 7fd3e48c9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9ab79e5d0 0x55f9ab79ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:37.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.402+0000 7fd3e48c9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9ab79fb40 0x55f9ab7c0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:37.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.402+0000 7fd3e48c9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9ab7ab580 0x55f9ab7c1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:37.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.402+0000 7fd3e48c9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc1ab4f0 con 0x55f9ab79fb40 2024-09-19T08:54:37.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.402+0000 7fd3e48c9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc1ad030 con 0x55f9ab7ab580 2024-09-19T08:54:37.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.402+0000 7fd3e48c9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc1abbc0 con 0x55f9ab79e5d0 2024-09-19T08:54:37.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.402+0000 7fd3d3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9ab79fb40 0x55f9ab7c0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:37.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.402+0000 7fd3d3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9ab79fb40 0x55f9ab7c0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35094/0 (socket says 172.21.15.164:35094) 2024-09-19T08:54:37.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.402+0000 7fd3d3fff640 1 -- 172.21.15.164:0/3194424311 learned_addr learned my addr 172.21.15.164:0/3194424311 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:37.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.402+0000 7fd3dbfff640 1 --2- 172.21.15.164:0/3194424311 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9ab79e5d0 0x55f9ab79ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:37.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.402+0000 7fd3d3fff640 1 -- 172.21.15.164:0/3194424311 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9ab79e5d0 msgr2=0x55f9ab79ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:37.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.402+0000 7fd3e0a8d640 1 --2- 172.21.15.164:0/3194424311 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9ab7ab580 0x55f9ab7c1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:37.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.402+0000 7fd3d3fff640 1 --2- 172.21.15.164:0/3194424311 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9ab79e5d0 0x55f9ab79ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:37.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.402+0000 7fd3d3fff640 1 -- 172.21.15.164:0/3194424311 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9ab7ab580 msgr2=0x55f9ab7c1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:37.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.402+0000 7fd3d3fff640 1 --2- 172.21.15.164:0/3194424311 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9ab7ab580 0x55f9ab7c1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:37.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.402+0000 7fd3d3fff640 1 -- 172.21.15.164:0/3194424311 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9ab7a7a50 con 0x55f9ab79fb40 2024-09-19T08:54:37.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.402+0000 7fd3dbfff640 1 --2- 172.21.15.164:0/3194424311 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9ab79e5d0 0x55f9ab79ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:54:37.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.402+0000 7fd3e0a8d640 1 --2- 172.21.15.164:0/3194424311 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9ab7ab580 0x55f9ab7c1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:54:37.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.402+0000 7fd3d3fff640 1 --2- 172.21.15.164:0/3194424311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9ab79fb40 0x55f9ab7c0cf0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4000c00 tx=0x7fd3c408c0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca32b082631e41d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:37.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.402+0000 7fd3d1ffb640 1 -- 172.21.15.164:0/3194424311 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4049040 con 0x55f9ab79fb40 2024-09-19T08:54:37.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.402+0000 7fd3d1ffb640 1 -- 172.21.15.164:0/3194424311 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3c4031de0 con 0x55f9ab79fb40 2024-09-19T08:54:37.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.402+0000 7fd3d1ffb640 1 -- 172.21.15.164:0/3194424311 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c40edbb0 con 0x55f9ab79fb40 2024-09-19T08:54:37.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.403+0000 7fd3e48c9740 1 -- 172.21.15.164:0/3194424311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9ab79fb40 msgr2=0x55f9ab7c0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:37.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.403+0000 7fd3e48c9740 1 --2- 172.21.15.164:0/3194424311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9ab79fb40 0x55f9ab7c0cf0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4000c00 tx=0x7fd3c408c0e0 comp rx=0 tx=0).stop 2024-09-19T08:54:37.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.403+0000 7fd3e48c9740 1 -- 172.21.15.164:0/3194424311 shutdown_connections 2024-09-19T08:54:37.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.403+0000 7fd3e48c9740 1 --2- 172.21.15.164:0/3194424311 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9ab7ab580 0x55f9ab7c1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:37.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.403+0000 7fd3e48c9740 1 --2- 172.21.15.164:0/3194424311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9ab79fb40 0x55f9ab7c0cf0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:37.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.403+0000 7fd3e48c9740 1 --2- 172.21.15.164:0/3194424311 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9ab79e5d0 0x55f9ab79ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:37.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.403+0000 7fd3e48c9740 1 -- 172.21.15.164:0/3194424311 >> 172.21.15.164:0/3194424311 conn(0x55f9ab767310 msgr2=0x55f9ab79a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:37.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.403+0000 7fd3e48c9740 1 -- 172.21.15.164:0/3194424311 shutdown_connections 2024-09-19T08:54:37.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.403+0000 7fd3e48c9740 1 -- 172.21.15.164:0/3194424311 wait complete. 2024-09-19T08:54:37.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.403+0000 7fd3e48c9740 1 -- start start 2024-09-19T08:54:37.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.404+0000 7fd3e48c9740 1 Processor -- start 2024-09-19T08:54:37.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.404+0000 7fd3e48c9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9ab79e5d0 0x55f9ab7b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:37.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.404+0000 7fd3e48c9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9ab79fb40 0x55f9ab7b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:37.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.404+0000 7fd3e48c9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9ab7ab580 0x55f9ab7b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:37.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.404+0000 7fd3e48c9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc1ac960 con 0x55f9ab79e5d0 2024-09-19T08:54:37.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.404+0000 7fd3e48c9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc1ac290 con 0x55f9ab79fb40 2024-09-19T08:54:37.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.404+0000 7fd3e48c9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc1a94f0 con 0x55f9ab7ab580 2024-09-19T08:54:37.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.404+0000 7fd3e48c9740 10 client.0 _ll_register_callbacks cb 0x7fd3cc00bc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:54:37.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.404+0000 7fd3dbfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9ab79e5d0 0x55f9ab7b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:37.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.404+0000 7fd3dbfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9ab79e5d0 0x55f9ab7b0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35100/0 (socket says 172.21.15.164:35100) 2024-09-19T08:54:37.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.404+0000 7fd3dbfff640 1 -- 172.21.15.164:0/3165360967 learned_addr learned my addr 172.21.15.164:0/3165360967 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:37.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.404+0000 7fd3e0a8d640 1 --2- 172.21.15.164:0/3165360967 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9ab7ab580 0x55f9ab7b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:37.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.404+0000 7fd3d3fff640 1 --2- 172.21.15.164:0/3165360967 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9ab79fb40 0x55f9ab7b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:37.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.404+0000 7fd3dbfff640 1 -- 172.21.15.164:0/3165360967 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9ab7ab580 msgr2=0x55f9ab7b0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:37.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.404+0000 7fd3dbfff640 1 --2- 172.21.15.164:0/3165360967 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9ab7ab580 0x55f9ab7b0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:37.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.404+0000 7fd3dbfff640 1 -- 172.21.15.164:0/3165360967 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9ab79fb40 msgr2=0x55f9ab7b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:37.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.404+0000 7fd3dbfff640 1 --2- 172.21.15.164:0/3165360967 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9ab79fb40 0x55f9ab7b06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:37.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.404+0000 7fd3dbfff640 1 -- 172.21.15.164:0/3165360967 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9ab822f10 con 0x55f9ab79e5d0 2024-09-19T08:54:37.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.404+0000 7fd3dbfff640 1 --2- 172.21.15.164:0/3165360967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9ab79e5d0 0x55f9ab7b0190 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc00b2b0 tx=0x7fd3cc004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea4a6f05d096c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:37.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.404+0000 7fd3bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:54:37.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.405+0000 7fd3bb7fe640 1 -- 172.21.15.164:0/3165360967 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc01ab30 con 0x55f9ab79e5d0 2024-09-19T08:54:37.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.405+0000 7fd3bb7fe640 1 -- 172.21.15.164:0/3165360967 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7fd3cc01acd0 con 0x55f9ab79e5d0 2024-09-19T08:54:37.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.405+0000 7fd3e48c9740 1 -- 172.21.15.164:0/3165360967 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3c4031de0 con 0x55f9ab79e5d0 2024-09-19T08:54:37.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.405+0000 7fd3bb7fe640 1 -- 172.21.15.164:0/3165360967 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd3cc005040 con 0x55f9ab79e5d0 2024-09-19T08:54:37.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.405+0000 7fd3e48c9740 10 client.17331 fetch_fsmap learned FSMap version 32 2024-09-19T08:54:37.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.405+0000 7fd3e48c9740 1 -- 172.21.15.164:0/3165360967 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f9ab824140 con 0x55f9ab79e5d0 2024-09-19T08:54:37.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.405+0000 7fd3bb7fe640 1 -- 172.21.15.164:0/3165360967 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3c4031de0 con 0x55f9ab79e5d0 2024-09-19T08:54:37.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.405+0000 7fd3bb7fe640 1 -- 172.21.15.164:0/3165360967 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fd3cc0035f0 con 0x55f9ab79e5d0 2024-09-19T08:54:37.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.405+0000 7fd3e48c9740 10 client.17331 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:54:37.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.405+0000 7fd3e48c9740 10 client.17331 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T08:54:37.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.405+0000 7fd3e48c9740 20 client.17331 populate_metadata read hostname 'smithi164' 2024-09-19T08:54:37.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.405+0000 7fd3e48c9740 1 --2- 172.21.15.164:0/3165360967 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55f9ab824860 0x55f9ab826c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:37.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.405+0000 7fd3e48c9740 4 client.17331 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:54:37.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.405+0000 7fd3e48c9740 1 -- 172.21.15.164:0/3165360967 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f9ab70bd70 con 0x55f9ab824860 2024-09-19T08:54:37.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.406+0000 7fd3d3fff640 1 --2- 172.21.15.164:0/3165360967 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55f9ab824860 0x55f9ab826c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:37.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.406+0000 7fd3d3fff640 1 --2- 172.21.15.164:0/3165360967 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55f9ab824860 0x55f9ab826c50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=96b7e6352bf8521c server_cookie=fe7c6b1bbaca2145 in_seq=0 out_seq=0 2024-09-19T08:54:37.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.406+0000 7fd3bb7fe640 10 client.17331 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:54:37.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.406+0000 7fd3bb7fe640 1 -- 172.21.15.164:0/3165360967 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f9ab70bd70 con 0x55f9ab824860 2024-09-19T08:54:37.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.406+0000 7fd3bb7fe640 10 client.17331 handle_command_reply: tid=0 2024-09-19T08:54:37.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.406+0000 7fd3e48c9740 1 client.17331 shutdown 2024-09-19T08:54:37.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.406+0000 7fd3e48c9740 1 -- 172.21.15.164:0/3165360967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9ab79e5d0 msgr2=0x55f9ab7b0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:37.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.406+0000 7fd3e48c9740 1 --2- 172.21.15.164:0/3165360967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9ab79e5d0 0x55f9ab7b0190 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc00b2b0 tx=0x7fd3cc004320 comp rx=0 tx=0).stop 2024-09-19T08:54:37.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.406+0000 7fd3e48c9740 1 -- 172.21.15.164:0/3165360967 shutdown_connections 2024-09-19T08:54:37.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.406+0000 7fd3e48c9740 1 --2- 172.21.15.164:0/3165360967 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55f9ab824860 0x55f9ab826c50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:37.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.406+0000 7fd3e48c9740 1 --2- 172.21.15.164:0/3165360967 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f9ab7ab580 0x55f9ab7b0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:37.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.406+0000 7fd3e48c9740 1 --2- 172.21.15.164:0/3165360967 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9ab79fb40 0x55f9ab7b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:37.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.406+0000 7fd3e48c9740 1 --2- 172.21.15.164:0/3165360967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9ab79e5d0 0x55f9ab7b0190 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:37.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.406+0000 7fd3e48c9740 1 -- 172.21.15.164:0/3165360967 >> 172.21.15.164:0/3165360967 conn(0x55f9ab767310 msgr2=0x55f9ab7a0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:37.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.406+0000 7fd3bb7fe640 0 client.17331 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:54:37.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.406+0000 7fd3e48c9740 1 -- 172.21.15.164:0/3165360967 shutdown_connections 2024-09-19T08:54:37.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.406+0000 7fd3e48c9740 1 -- 172.21.15.164:0/3165360967 wait complete. 2024-09-19T08:54:37.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.407+0000 7fd3e48c9740 20 client.17331 trim_cache size 0 max 16384 2024-09-19T08:54:37.419 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:54:37.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.407+0000 7fd3e24f0640 1 -- 172.21.15.164:0/1341387307 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd3c80a3430 msgr2=0x7fd3c80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:37.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.407+0000 7fd3e24f0640 1 --2- 172.21.15.164:0/1341387307 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd3c80a3430 0x7fd3c80a58f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4029000 tx=0x7fd3d4026000 comp rx=0 tx=0).stop 2024-09-19T08:54:37.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.407+0000 7fd3e24f0640 1 -- 172.21.15.164:0/1341387307 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3dc102570 msgr2=0x7fd3dc197920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:37.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.407+0000 7fd3e24f0640 1 --2- 172.21.15.164:0/1341387307 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3dc102570 0x7fd3dc197920 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4006010 tx=0x7fd3c4003c10 comp rx=0 tx=0).stop 2024-09-19T08:54:37.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.407+0000 7fd3e24f0640 1 -- 172.21.15.164:0/1341387307 shutdown_connections 2024-09-19T08:54:37.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.407+0000 7fd3e24f0640 1 --2- 172.21.15.164:0/1341387307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3dc102ef0 0x7fd3dc19bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:37.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.407+0000 7fd3e24f0640 1 --2- 172.21.15.164:0/1341387307 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd3c80a3430 0x7fd3c80a58f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:37.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.407+0000 7fd3e24f0640 1 --2- 172.21.15.164:0/1341387307 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3dc102570 0x7fd3dc197920 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:37.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.407+0000 7fd3e24f0640 1 --2- 172.21.15.164:0/1341387307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3dc100e20 0x7fd3dc1973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:37.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.407+0000 7fd3e24f0640 1 -- 172.21.15.164:0/1341387307 >> 172.21.15.164:0/1341387307 conn(0x7fd3dc0fc7d0 msgr2=0x7fd3dc1068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:37.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.407+0000 7fd3e24f0640 1 -- 172.21.15.164:0/1341387307 shutdown_connections 2024-09-19T08:54:37.421 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:37.407+0000 7fd3e24f0640 1 -- 172.21.15.164:0/1341387307 wait complete. 2024-09-19T08:54:38.154 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T08:54:38.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:37 smithi164 ceph-mon[30928]: pgmap v273: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T08:54:38.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:37 smithi170 ceph-mon[35072]: pgmap v273: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T08:54:38.402 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:54:38.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:37 smithi173 ceph-mon[35306]: pgmap v273: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T08:54:38.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.912+0000 7fa2a8b4a640 1 -- 172.21.15.164:0/2352197850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2a41082a0 msgr2=0x7fa2a410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:38.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.912+0000 7fa2a8b4a640 1 --2- 172.21.15.164:0/2352197850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2a41082a0 0x7fa2a410ec40 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fa29c009930 tx=0x7fa29c02a420 comp rx=0 tx=0).stop 2024-09-19T08:54:38.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.913+0000 7fa2a8b4a640 1 -- 172.21.15.164:0/2352197850 shutdown_connections 2024-09-19T08:54:38.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.913+0000 7fa2a8b4a640 1 --2- 172.21.15.164:0/2352197850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2a41082a0 0x7fa2a410ec40 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.913+0000 7fa2a8b4a640 1 --2- 172.21.15.164:0/2352197850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2a41036e0 0x7fa2a4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.913+0000 7fa2a8b4a640 1 --2- 172.21.15.164:0/2352197850 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2a4102d30 0x7fa2a4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.913+0000 7fa2a8b4a640 1 -- 172.21.15.164:0/2352197850 >> 172.21.15.164:0/2352197850 conn(0x7fa2a40fc7d0 msgr2=0x7fa2a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:38.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.913+0000 7fa2a8b4a640 1 -- 172.21.15.164:0/2352197850 shutdown_connections 2024-09-19T08:54:38.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.913+0000 7fa2a8b4a640 1 -- 172.21.15.164:0/2352197850 wait complete. 2024-09-19T08:54:38.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.913+0000 7fa2a8b4a640 1 Processor -- start 2024-09-19T08:54:38.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.913+0000 7fa2a8b4a640 1 -- start start 2024-09-19T08:54:38.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.914+0000 7fa2a8b4a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2a4102d30 0x7fa2a4198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:38.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.914+0000 7fa2a8b4a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2a41036e0 0x7fa2a4198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:38.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.914+0000 7fa2a8b4a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2a41082a0 0x7fa2a419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:38.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.914+0000 7fa2a8b4a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a41120a0 con 0x7fa2a41082a0 2024-09-19T08:54:38.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.914+0000 7fa2a8b4a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a4111f20 con 0x7fa2a41036e0 2024-09-19T08:54:38.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.914+0000 7fa2a8b4a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a4112220 con 0x7fa2a4102d30 2024-09-19T08:54:38.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.914+0000 7fa2a2d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2a41082a0 0x7fa2a419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:38.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.914+0000 7fa2a2d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2a41082a0 0x7fa2a419ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35116/0 (socket says 172.21.15.164:35116) 2024-09-19T08:54:38.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.914+0000 7fa2a2d9d640 1 -- 172.21.15.164:0/2108044271 learned_addr learned my addr 172.21.15.164:0/2108044271 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:38.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.914+0000 7fa2a259c640 1 --2- 172.21.15.164:0/2108044271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2a4102d30 0x7fa2a4198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:38.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.914+0000 7fa2a1d9b640 1 --2- 172.21.15.164:0/2108044271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2a41036e0 0x7fa2a4198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:38.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.914+0000 7fa2a2d9d640 1 -- 172.21.15.164:0/2108044271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2a4102d30 msgr2=0x7fa2a4198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:38.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.914+0000 7fa2a2d9d640 1 --2- 172.21.15.164:0/2108044271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2a4102d30 0x7fa2a4198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.914+0000 7fa2a2d9d640 1 -- 172.21.15.164:0/2108044271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2a41036e0 msgr2=0x7fa2a4198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:38.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.914+0000 7fa2a2d9d640 1 --2- 172.21.15.164:0/2108044271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2a41036e0 0x7fa2a4198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.914+0000 7fa2a2d9d640 1 -- 172.21.15.164:0/2108044271 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a419d500 con 0x7fa2a41082a0 2024-09-19T08:54:38.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.914+0000 7fa2a2d9d640 1 --2- 172.21.15.164:0/2108044271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2a41082a0 0x7fa2a419ce20 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fa29c005ab0 tx=0x7fa29c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:38.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.914+0000 7fa29b7fe640 1 -- 172.21.15.164:0/2108044271 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c03f030 con 0x7fa2a41082a0 2024-09-19T08:54:38.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.915+0000 7fa29b7fe640 1 -- 172.21.15.164:0/2108044271 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa29c003ce0 con 0x7fa2a41082a0 2024-09-19T08:54:38.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.915+0000 7fa2a8b4a640 1 -- 172.21.15.164:0/2108044271 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a419d790 con 0x7fa2a41082a0 2024-09-19T08:54:38.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.915+0000 7fa29b7fe640 1 -- 172.21.15.164:0/2108044271 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c035070 con 0x7fa2a41082a0 2024-09-19T08:54:38.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.915+0000 7fa2a8b4a640 1 -- 172.21.15.164:0/2108044271 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a41a50d0 con 0x7fa2a41082a0 2024-09-19T08:54:38.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.917+0000 7fa29b7fe640 1 -- 172.21.15.164:0/2108044271 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fa29c042030 con 0x7fa2a41082a0 2024-09-19T08:54:38.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.917+0000 7fa2aaf23740 1 Processor -- start 2024-09-19T08:54:38.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.918+0000 7fa2aaf23740 1 -- start start 2024-09-19T08:54:38.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.918+0000 7fa29b7fe640 1 --2- 172.21.15.164:0/2108044271 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa2900a3430 0x7fa2900a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:38.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.918+0000 7fa2aaf23740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56470fd33b40 0x56470fd32090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:38.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.918+0000 7fa2aaf23740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56470fd325d0 0x56470fd32a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:38.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.918+0000 7fa2aaf23740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56470fd3d370 0x56470fd3f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:38.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.918+0000 7fa2aaf23740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56470fc982f0 con 0x56470fd33b40 2024-09-19T08:54:38.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.918+0000 7fa2aaf23740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56470fc3fcf0 con 0x56470fd3d370 2024-09-19T08:54:38.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.918+0000 7fa2aaf23740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56470fc942b0 con 0x56470fd325d0 2024-09-19T08:54:38.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.918+0000 7fa2a1d9b640 1 --2- 172.21.15.164:0/2108044271 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa2900a3430 0x7fa2900a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:38.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.918+0000 7fa29b7fe640 1 -- 172.21.15.164:0/2108044271 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7fa29c0f00b0 con 0x7fa2a41082a0 2024-09-19T08:54:38.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.918+0000 7fa2a259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56470fd33b40 0x56470fd32090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:38.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.918+0000 7fa2a259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56470fd33b40 0x56470fd32090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35128/0 (socket says 172.21.15.164:35128) 2024-09-19T08:54:38.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.918+0000 7fa2a259c640 1 -- 172.21.15.164:0/129434220 learned_addr learned my addr 172.21.15.164:0/129434220 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:38.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.918+0000 7fa2a1d9b640 1 --2- 172.21.15.164:0/129434220 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56470fd325d0 0x56470fd32a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:38.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.918+0000 7fa2a2d9d640 1 --2- 172.21.15.164:0/129434220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56470fd3d370 0x56470fd3f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:38.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.918+0000 7fa2a259c640 1 -- 172.21.15.164:0/129434220 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56470fd325d0 msgr2=0x56470fd32a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:38.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.918+0000 7fa2a259c640 1 --2- 172.21.15.164:0/129434220 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56470fd325d0 0x56470fd32a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.918+0000 7fa2a259c640 1 -- 172.21.15.164:0/129434220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56470fd3d370 msgr2=0x56470fd3f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:38.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.918+0000 7fa2a259c640 1 --2- 172.21.15.164:0/129434220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56470fd3d370 0x56470fd3f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.918+0000 7fa2a259c640 1 -- 172.21.15.164:0/129434220 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56470fc243a0 con 0x56470fd33b40 2024-09-19T08:54:38.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.918+0000 7fa2a2d9d640 1 --2- 172.21.15.164:0/129434220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56470fd3d370 0x56470fd3f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:54:38.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.918+0000 7fa2a1d9b640 1 --2- 172.21.15.164:0/2108044271 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa2900a3430 0x7fa2900a58f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa28c00bfd0 tx=0x7fa28c00b6f0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:38.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.918+0000 7fa2a259c640 1 --2- 172.21.15.164:0/129434220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56470fd33b40 0x56470fd32090 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fa2940103f0 tx=0x7fa2940108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f146933ddbc3138c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:38.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.919+0000 7fa298ff9640 1 -- 172.21.15.164:0/129434220 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa294016070 con 0x56470fd33b40 2024-09-19T08:54:38.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.919+0000 7fa298ff9640 1 -- 172.21.15.164:0/129434220 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa294012bf0 con 0x56470fd33b40 2024-09-19T08:54:38.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.919+0000 7fa298ff9640 1 -- 172.21.15.164:0/129434220 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29401bd40 con 0x56470fd33b40 2024-09-19T08:54:38.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.919+0000 7fa2aaf23740 1 -- 172.21.15.164:0/129434220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56470fd33b40 msgr2=0x56470fd32090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:38.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.919+0000 7fa2aaf23740 1 --2- 172.21.15.164:0/129434220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56470fd33b40 0x56470fd32090 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fa2940103f0 tx=0x7fa2940108b0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.919+0000 7fa2aaf23740 1 -- 172.21.15.164:0/129434220 shutdown_connections 2024-09-19T08:54:38.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.919+0000 7fa2aaf23740 1 --2- 172.21.15.164:0/129434220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56470fd3d370 0x56470fd3f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.919+0000 7fa2aaf23740 1 --2- 172.21.15.164:0/129434220 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56470fd325d0 0x56470fd32a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.919+0000 7fa2aaf23740 1 --2- 172.21.15.164:0/129434220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56470fd33b40 0x56470fd32090 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.919+0000 7fa2aaf23740 1 -- 172.21.15.164:0/129434220 >> 172.21.15.164:0/129434220 conn(0x56470fcfb310 msgr2=0x56470fd2ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:38.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.919+0000 7fa2aaf23740 1 -- 172.21.15.164:0/129434220 shutdown_connections 2024-09-19T08:54:38.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.920+0000 7fa2aaf23740 1 -- 172.21.15.164:0/129434220 wait complete. 2024-09-19T08:54:38.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.920+0000 7fa2aaf23740 1 -- start start 2024-09-19T08:54:38.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.920+0000 7fa2aaf23740 1 Processor -- start 2024-09-19T08:54:38.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.920+0000 7fa2aaf23740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56470fd325d0 0x56470fd3cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:38.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.920+0000 7fa2aaf23740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56470fd33b40 0x56470fd3a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:38.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.920+0000 7fa2aaf23740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56470fd3d370 0x56470fd3a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:38.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.920+0000 7fa2aaf23740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56470fd40070 con 0x56470fd3d370 2024-09-19T08:54:38.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.920+0000 7fa2aaf23740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56470fc53060 con 0x56470fd33b40 2024-09-19T08:54:38.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.920+0000 7fa2aaf23740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56470fd401f0 con 0x56470fd325d0 2024-09-19T08:54:38.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.920+0000 7fa2aaf23740 10 client.0 _ll_register_callbacks cb 0x56470f9a68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:54:38.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.920+0000 7fa2a2d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56470fd3d370 0x56470fd3a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:38.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.920+0000 7fa2a2d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56470fd3d370 0x56470fd3a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35132/0 (socket says 172.21.15.164:35132) 2024-09-19T08:54:38.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.920+0000 7fa2a2d9d640 1 -- 172.21.15.164:0/4284127163 learned_addr learned my addr 172.21.15.164:0/4284127163 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:38.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.920+0000 7fa2a259c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56470fd325d0 0x56470fd3cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:38.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.920+0000 7fa2a1d9b640 1 --2- 172.21.15.164:0/4284127163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56470fd33b40 0x56470fd3a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:38.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.921+0000 7fa2a259c640 1 -- 172.21.15.164:0/4284127163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56470fd33b40 msgr2=0x56470fd3a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:38.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.921+0000 7fa2a259c640 1 --2- 172.21.15.164:0/4284127163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56470fd33b40 0x56470fd3a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.921+0000 7fa2a259c640 1 -- 172.21.15.164:0/4284127163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56470fd3d370 msgr2=0x56470fd3a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:38.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.921+0000 7fa2a259c640 1 --2- 172.21.15.164:0/4284127163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56470fd3d370 0x56470fd3a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.921+0000 7fa2a259c640 1 -- 172.21.15.164:0/4284127163 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56470fa093c0 con 0x56470fd325d0 2024-09-19T08:54:38.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.921+0000 7fa2a2d9d640 1 --2- 172.21.15.164:0/4284127163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56470fd3d370 0x56470fd3a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:54:38.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.921+0000 7fa2a259c640 1 --2- 172.21.15.164:0/4284127163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56470fd325d0 0x56470fd3cd90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa294006010 tx=0x7fa29401b9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62f3e3a467fe6726 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:38.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.921+0000 7fa289ffb640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-09-19T08:54:38.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.921+0000 7fa289ffb640 1 -- 172.21.15.164:0/4284127163 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29402a060 con 0x56470fd325d0 2024-09-19T08:54:38.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.921+0000 7fa2aaf23740 1 -- 172.21.15.164:0/4284127163 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa294012bf0 con 0x56470fd325d0 2024-09-19T08:54:38.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.921+0000 7fa289ffb640 1 -- 172.21.15.164:0/4284127163 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7fa29401bac0 con 0x56470fd325d0 2024-09-19T08:54:38.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.921+0000 7fa289ffb640 1 -- 172.21.15.164:0/4284127163 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa294016050 con 0x56470fd325d0 2024-09-19T08:54:38.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.921+0000 7fa2aaf23740 10 client.26449 fetch_fsmap learned FSMap version 32 2024-09-19T08:54:38.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.921+0000 7fa2aaf23740 1 -- 172.21.15.164:0/4284127163 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56470fd4bab0 con 0x56470fd325d0 2024-09-19T08:54:38.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.921+0000 7fa289ffb640 1 -- 172.21.15.164:0/4284127163 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa294012bf0 con 0x56470fd325d0 2024-09-19T08:54:38.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.922+0000 7fa289ffb640 1 -- 172.21.15.164:0/4284127163 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fa29400f040 con 0x56470fd325d0 2024-09-19T08:54:38.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.922+0000 7fa2aaf23740 10 client.26449 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:54:38.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.922+0000 7fa2aaf23740 10 client.26449 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T08:54:38.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.922+0000 7fa2aaf23740 20 client.26449 populate_metadata read hostname 'smithi164' 2024-09-19T08:54:38.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.922+0000 7fa2aaf23740 1 --2- 172.21.15.164:0/4284127163 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x56470fd4ce60 0x56470fd4f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:38.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.922+0000 7fa2aaf23740 4 client.26449 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:54:38.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.922+0000 7fa2aaf23740 1 -- 172.21.15.164:0/4284127163 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56470fca5860 con 0x56470fd4ce60 2024-09-19T08:54:38.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.922+0000 7fa2a1d9b640 1 --2- 172.21.15.164:0/4284127163 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x56470fd4ce60 0x56470fd4f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:38.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.922+0000 7fa2a1d9b640 1 --2- 172.21.15.164:0/4284127163 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x56470fd4ce60 0x56470fd4f250 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e9d6020f1aee0754 server_cookie=74320c5bb2cc6607 in_seq=0 out_seq=0 2024-09-19T08:54:38.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.922+0000 7fa289ffb640 10 client.26449 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:54:38.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.923+0000 7fa289ffb640 1 -- 172.21.15.164:0/4284127163 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56470fca5860 con 0x56470fd4ce60 2024-09-19T08:54:38.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.923+0000 7fa289ffb640 10 client.26449 handle_command_reply: tid=0 2024-09-19T08:54:38.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.923+0000 7fa2aaf23740 1 client.26449 shutdown 2024-09-19T08:54:38.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.923+0000 7fa2aaf23740 1 -- 172.21.15.164:0/4284127163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56470fd325d0 msgr2=0x56470fd3cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:38.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.923+0000 7fa2aaf23740 1 --2- 172.21.15.164:0/4284127163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56470fd325d0 0x56470fd3cd90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa294006010 tx=0x7fa29401b9b0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.923+0000 7fa2aaf23740 1 -- 172.21.15.164:0/4284127163 shutdown_connections 2024-09-19T08:54:38.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.923+0000 7fa2aaf23740 1 --2- 172.21.15.164:0/4284127163 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x56470fd4ce60 0x56470fd4f250 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.923+0000 7fa2aaf23740 1 --2- 172.21.15.164:0/4284127163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56470fd3d370 0x56470fd3a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.923+0000 7fa2aaf23740 1 --2- 172.21.15.164:0/4284127163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56470fd33b40 0x56470fd3a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.923+0000 7fa2aaf23740 1 --2- 172.21.15.164:0/4284127163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56470fd325d0 0x56470fd3cd90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.923+0000 7fa289ffb640 0 client.26449 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:54:38.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.923+0000 7fa2aaf23740 1 -- 172.21.15.164:0/4284127163 >> 172.21.15.164:0/4284127163 conn(0x56470fcfb310 msgr2=0x56470fd34960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:38.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.923+0000 7fa2aaf23740 1 -- 172.21.15.164:0/4284127163 shutdown_connections 2024-09-19T08:54:38.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.923+0000 7fa2aaf23740 1 -- 172.21.15.164:0/4284127163 wait complete. 2024-09-19T08:54:38.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.923+0000 7fa2aaf23740 20 client.26449 trim_cache size 0 max 16384 2024-09-19T08:54:38.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.937+0000 7fa2aaf23740 1 Processor -- start 2024-09-19T08:54:38.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.937+0000 7fa2aaf23740 1 -- start start 2024-09-19T08:54:38.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.937+0000 7fa2aaf23740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56470fd325d0 0x56470fd39920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:38.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.937+0000 7fa2aaf23740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56470fd33b40 0x56470fd49310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:38.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.937+0000 7fa2aaf23740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56470fd3d370 0x56470fd54820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:38.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.937+0000 7fa2aaf23740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a41ad7e0 con 0x56470fd33b40 2024-09-19T08:54:38.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.937+0000 7fa2aaf23740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a41af320 con 0x56470fd3d370 2024-09-19T08:54:38.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.937+0000 7fa2aaf23740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a41adeb0 con 0x56470fd325d0 2024-09-19T08:54:38.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.937+0000 7fa2a1d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56470fd33b40 0x56470fd49310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:38.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.937+0000 7fa2a1d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56470fd33b40 0x56470fd49310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35134/0 (socket says 172.21.15.164:35134) 2024-09-19T08:54:38.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.937+0000 7fa2a1d9b640 1 -- 172.21.15.164:0/2268371237 learned_addr learned my addr 172.21.15.164:0/2268371237 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:38.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.937+0000 7fa2a2d9d640 1 --2- 172.21.15.164:0/2268371237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56470fd3d370 0x56470fd54820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:38.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.937+0000 7fa2a259c640 1 --2- 172.21.15.164:0/2268371237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56470fd325d0 0x56470fd39920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:38.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.937+0000 7fa2a1d9b640 1 -- 172.21.15.164:0/2268371237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56470fd325d0 msgr2=0x56470fd39920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:38.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.937+0000 7fa2a1d9b640 1 --2- 172.21.15.164:0/2268371237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56470fd325d0 0x56470fd39920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.937+0000 7fa2a1d9b640 1 -- 172.21.15.164:0/2268371237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56470fd3d370 msgr2=0x56470fd54820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:38.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.937+0000 7fa2a1d9b640 1 --2- 172.21.15.164:0/2268371237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56470fd3d370 0x56470fd54820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.937+0000 7fa2a1d9b640 1 -- 172.21.15.164:0/2268371237 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56470fd2e510 con 0x56470fd33b40 2024-09-19T08:54:38.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.937+0000 7fa2a2d9d640 1 --2- 172.21.15.164:0/2268371237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56470fd3d370 0x56470fd54820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:54:38.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.938+0000 7fa2a1d9b640 1 --2- 172.21.15.164:0/2268371237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56470fd33b40 0x56470fd49310 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fa2a406a370 tx=0x7fa28c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa56b2b79a9f2b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:38.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.938+0000 7fa28bfff640 1 -- 172.21.15.164:0/2268371237 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c0292d0 con 0x56470fd33b40 2024-09-19T08:54:38.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.938+0000 7fa28bfff640 1 -- 172.21.15.164:0/2268371237 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa28c02e070 con 0x56470fd33b40 2024-09-19T08:54:38.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.938+0000 7fa28bfff640 1 -- 172.21.15.164:0/2268371237 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c0226b0 con 0x56470fd33b40 2024-09-19T08:54:38.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.938+0000 7fa2aaf23740 1 -- 172.21.15.164:0/2268371237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56470fd33b40 msgr2=0x56470fd49310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:38.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.938+0000 7fa2aaf23740 1 --2- 172.21.15.164:0/2268371237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56470fd33b40 0x56470fd49310 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fa2a406a370 tx=0x7fa28c004730 comp rx=0 tx=0).stop 2024-09-19T08:54:38.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.938+0000 7fa2aaf23740 1 -- 172.21.15.164:0/2268371237 shutdown_connections 2024-09-19T08:54:38.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.938+0000 7fa2aaf23740 1 --2- 172.21.15.164:0/2268371237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56470fd3d370 0x56470fd54820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.938+0000 7fa2aaf23740 1 --2- 172.21.15.164:0/2268371237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56470fd33b40 0x56470fd49310 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.938+0000 7fa2aaf23740 1 --2- 172.21.15.164:0/2268371237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56470fd325d0 0x56470fd39920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.938+0000 7fa2aaf23740 1 -- 172.21.15.164:0/2268371237 >> 172.21.15.164:0/2268371237 conn(0x56470fcfb310 msgr2=0x56470fd34930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:38.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.938+0000 7fa2aaf23740 1 -- 172.21.15.164:0/2268371237 shutdown_connections 2024-09-19T08:54:38.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.939+0000 7fa2aaf23740 1 -- 172.21.15.164:0/2268371237 wait complete. 2024-09-19T08:54:38.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.939+0000 7fa2aaf23740 1 -- start start 2024-09-19T08:54:38.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.939+0000 7fa2aaf23740 1 Processor -- start 2024-09-19T08:54:38.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.939+0000 7fa2aaf23740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56470fd325d0 0x56470fd45a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:38.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.939+0000 7fa2aaf23740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56470fd33b40 0x56470fd36c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:38.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.939+0000 7fa2aaf23740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56470fd3d370 0x56470fd37140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:38.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.939+0000 7fa2aaf23740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a41aec50 con 0x56470fd33b40 2024-09-19T08:54:38.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.939+0000 7fa2aaf23740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a41ae580 con 0x56470fd325d0 2024-09-19T08:54:38.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.939+0000 7fa2aaf23740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a41ab7e0 con 0x56470fd3d370 2024-09-19T08:54:38.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.939+0000 7fa2aaf23740 10 client.0 _ll_register_callbacks cb 0x7fa28c0123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:54:38.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.939+0000 7fa2a1d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56470fd33b40 0x56470fd36c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:38.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.939+0000 7fa2a1d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56470fd33b40 0x56470fd36c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35136/0 (socket says 172.21.15.164:35136) 2024-09-19T08:54:38.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.939+0000 7fa2a1d9b640 1 -- 172.21.15.164:0/171095464 learned_addr learned my addr 172.21.15.164:0/171095464 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:38.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.939+0000 7fa2a259c640 1 --2- 172.21.15.164:0/171095464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56470fd325d0 0x56470fd45a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:38.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.939+0000 7fa2a2d9d640 1 --2- 172.21.15.164:0/171095464 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56470fd3d370 0x56470fd37140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:38.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.940+0000 7fa2a1d9b640 1 -- 172.21.15.164:0/171095464 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56470fd3d370 msgr2=0x56470fd37140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:38.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.940+0000 7fa2a1d9b640 1 --2- 172.21.15.164:0/171095464 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56470fd3d370 0x56470fd37140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.940+0000 7fa2a1d9b640 1 -- 172.21.15.164:0/171095464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56470fd325d0 msgr2=0x56470fd45a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:38.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.940+0000 7fa2a1d9b640 1 --2- 172.21.15.164:0/171095464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56470fd325d0 0x56470fd45a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.940+0000 7fa2a1d9b640 1 -- 172.21.15.164:0/171095464 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56470fd377e0 con 0x56470fd33b40 2024-09-19T08:54:38.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.940+0000 7fa2a2d9d640 1 --2- 172.21.15.164:0/171095464 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56470fd3d370 0x56470fd37140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:54:38.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.940+0000 7fa2a1d9b640 1 --2- 172.21.15.164:0/171095464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56470fd33b40 0x56470fd36c00 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fa28c029ea0 tx=0x7fa28c029dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90eecd67182574fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:38.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.940+0000 7fa289ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:54:38.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.940+0000 7fa289ffb640 1 -- 172.21.15.164:0/171095464 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c029390 con 0x56470fd33b40 2024-09-19T08:54:38.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.940+0000 7fa289ffb640 1 -- 172.21.15.164:0/171095464 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7fa28c029530 con 0x56470fd33b40 2024-09-19T08:54:38.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.940+0000 7fa2aaf23740 1 -- 172.21.15.164:0/171095464 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa28c02e070 con 0x56470fd33b40 2024-09-19T08:54:38.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.940+0000 7fa289ffb640 1 -- 172.21.15.164:0/171095464 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa28c029730 con 0x56470fd33b40 2024-09-19T08:54:38.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.940+0000 7fa2aaf23740 10 client.17349 fetch_fsmap learned FSMap version 32 2024-09-19T08:54:38.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.940+0000 7fa2aaf23740 1 -- 172.21.15.164:0/171095464 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56470fdb7680 con 0x56470fd33b40 2024-09-19T08:54:38.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.940+0000 7fa289ffb640 1 -- 172.21.15.164:0/171095464 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa28c02e070 con 0x56470fd33b40 2024-09-19T08:54:38.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.940+0000 7fa289ffb640 1 -- 172.21.15.164:0/171095464 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fa28c034460 con 0x56470fd33b40 2024-09-19T08:54:38.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.941+0000 7fa2aaf23740 10 client.17349 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:54:38.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.941+0000 7fa2aaf23740 10 client.17349 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T08:54:38.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.941+0000 7fa2aaf23740 20 client.17349 populate_metadata read hostname 'smithi164' 2024-09-19T08:54:38.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.941+0000 7fa2aaf23740 1 --2- 172.21.15.164:0/171095464 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x56470fdb7ce0 0x56470fdba0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:38.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.941+0000 7fa2aaf23740 4 client.17349 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:54:38.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.941+0000 7fa2aaf23740 1 -- 172.21.15.164:0/171095464 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56470fc9fd70 con 0x56470fdb7ce0 2024-09-19T08:54:38.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.941+0000 7fa2a259c640 1 --2- 172.21.15.164:0/171095464 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x56470fdb7ce0 0x56470fdba0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:38.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.941+0000 7fa289ffb640 10 client.17349 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:54:38.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.941+0000 7fa2a259c640 1 --2- 172.21.15.164:0/171095464 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x56470fdb7ce0 0x56470fdba0d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ff7496e48102e7a server_cookie=3e15bfd61374fa7d in_seq=0 out_seq=0 2024-09-19T08:54:38.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.941+0000 7fa289ffb640 1 -- 172.21.15.164:0/171095464 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56470fc9fd70 con 0x56470fdb7ce0 2024-09-19T08:54:38.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.941+0000 7fa289ffb640 10 client.17349 handle_command_reply: tid=0 2024-09-19T08:54:38.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.941+0000 7fa2aaf23740 1 client.17349 shutdown 2024-09-19T08:54:38.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.941+0000 7fa2aaf23740 1 -- 172.21.15.164:0/171095464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56470fd33b40 msgr2=0x56470fd36c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:38.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.941+0000 7fa2aaf23740 1 --2- 172.21.15.164:0/171095464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56470fd33b40 0x56470fd36c00 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fa28c029ea0 tx=0x7fa28c029dc0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.941+0000 7fa2aaf23740 1 -- 172.21.15.164:0/171095464 shutdown_connections 2024-09-19T08:54:38.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.941+0000 7fa2aaf23740 1 --2- 172.21.15.164:0/171095464 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x56470fdb7ce0 0x56470fdba0d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.941+0000 7fa2aaf23740 1 --2- 172.21.15.164:0/171095464 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56470fd3d370 0x56470fd37140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.941+0000 7fa2aaf23740 1 --2- 172.21.15.164:0/171095464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56470fd33b40 0x56470fd36c00 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.941+0000 7fa2aaf23740 1 --2- 172.21.15.164:0/171095464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56470fd325d0 0x56470fd45a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.941+0000 7fa2aaf23740 1 -- 172.21.15.164:0/171095464 >> 172.21.15.164:0/171095464 conn(0x56470fcfb310 msgr2=0x56470fd34110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:38.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.941+0000 7fa289ffb640 0 client.17349 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:54:38.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.942+0000 7fa2aaf23740 1 -- 172.21.15.164:0/171095464 shutdown_connections 2024-09-19T08:54:38.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.942+0000 7fa2aaf23740 1 -- 172.21.15.164:0/171095464 wait complete. 2024-09-19T08:54:38.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.942+0000 7fa2aaf23740 20 client.17349 trim_cache size 0 max 16384 2024-09-19T08:54:38.958 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:54:38.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.942+0000 7fa2a8b4a640 1 -- 172.21.15.164:0/2108044271 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa2900a3430 msgr2=0x7fa2900a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:38.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.942+0000 7fa2a8b4a640 1 --2- 172.21.15.164:0/2108044271 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa2900a3430 0x7fa2900a58f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa28c00bfd0 tx=0x7fa28c00b6f0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.942+0000 7fa2a8b4a640 1 -- 172.21.15.164:0/2108044271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2a41082a0 msgr2=0x7fa2a419ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:38.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.942+0000 7fa2a8b4a640 1 --2- 172.21.15.164:0/2108044271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2a41082a0 0x7fa2a419ce20 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fa29c005ab0 tx=0x7fa29c0042f0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.942+0000 7fa2a8b4a640 1 -- 172.21.15.164:0/2108044271 shutdown_connections 2024-09-19T08:54:38.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.942+0000 7fa2a8b4a640 1 --2- 172.21.15.164:0/2108044271 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa2900a3430 0x7fa2900a58f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.942+0000 7fa2a8b4a640 1 --2- 172.21.15.164:0/2108044271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2a41082a0 0x7fa2a419ce20 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.942+0000 7fa2a8b4a640 1 --2- 172.21.15.164:0/2108044271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2a41036e0 0x7fa2a4198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.942+0000 7fa2a8b4a640 1 --2- 172.21.15.164:0/2108044271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2a4102d30 0x7fa2a4198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:38.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.942+0000 7fa2a8b4a640 1 -- 172.21.15.164:0/2108044271 >> 172.21.15.164:0/2108044271 conn(0x7fa2a40fc7d0 msgr2=0x7fa2a4105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:38.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.942+0000 7fa2a8b4a640 1 -- 172.21.15.164:0/2108044271 shutdown_connections 2024-09-19T08:54:38.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:38.942+0000 7fa2a8b4a640 1 -- 172.21.15.164:0/2108044271 wait complete. 2024-09-19T08:54:39.802 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T08:54:40.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:40 smithi164 ceph-mon[30928]: pgmap v274: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:54:40.055 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:54:40.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:40 smithi170 ceph-mon[35072]: pgmap v274: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:54:40.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:40 smithi173 ceph-mon[35306]: pgmap v274: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:54:40.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.570+0000 7f49eaf39640 1 -- 172.21.15.164:0/100654425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49e4103740 msgr2=0x7f49e4107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:40.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.570+0000 7f49eaf39640 1 --2- 172.21.15.164:0/100654425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49e4103740 0x7f49e4107b80 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009930 tx=0x7f49d002a420 comp rx=0 tx=0).stop 2024-09-19T08:54:40.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.570+0000 7f49eaf39640 1 -- 172.21.15.164:0/100654425 shutdown_connections 2024-09-19T08:54:40.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.570+0000 7f49eaf39640 1 --2- 172.21.15.164:0/100654425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49e4108290 0x7f49e410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.570+0000 7f49eaf39640 1 --2- 172.21.15.164:0/100654425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49e4103740 0x7f49e4107b80 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.570+0000 7f49eaf39640 1 --2- 172.21.15.164:0/100654425 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49e4102d90 0x7f49e4103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.570+0000 7f49eaf39640 1 -- 172.21.15.164:0/100654425 >> 172.21.15.164:0/100654425 conn(0x7f49e40fc7d0 msgr2=0x7f49e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:40.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.570+0000 7f49eaf39640 1 -- 172.21.15.164:0/100654425 shutdown_connections 2024-09-19T08:54:40.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.570+0000 7f49eaf39640 1 -- 172.21.15.164:0/100654425 wait complete. 2024-09-19T08:54:40.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.570+0000 7f49eaf39640 1 Processor -- start 2024-09-19T08:54:40.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.571+0000 7f49eaf39640 1 -- start start 2024-09-19T08:54:40.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.571+0000 7f49eaf39640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49e4102d90 0x7f49e4198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:40.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.571+0000 7f49eaf39640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49e4103740 0x7f49e4198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:40.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.571+0000 7f49eaf39640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49e4108290 0x7f49e419cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:40.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.571+0000 7f49eaf39640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f49e4112090 con 0x7f49e4103740 2024-09-19T08:54:40.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.571+0000 7f49eaf39640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f49e4111f10 con 0x7f49e4108290 2024-09-19T08:54:40.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.571+0000 7f49eaf39640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f49e4112210 con 0x7f49e4102d90 2024-09-19T08:54:40.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.571+0000 7f49e3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49e4103740 0x7f49e4198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:40.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.571+0000 7f49e3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49e4103740 0x7f49e4198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35154/0 (socket says 172.21.15.164:35154) 2024-09-19T08:54:40.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.571+0000 7f49e3fff640 1 -- 172.21.15.164:0/3027117914 learned_addr learned my addr 172.21.15.164:0/3027117914 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:40.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.571+0000 7f49e94d6640 1 --2- 172.21.15.164:0/3027117914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49e4108290 0x7f49e419cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:40.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.571+0000 7f49e8cd5640 1 --2- 172.21.15.164:0/3027117914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49e4102d90 0x7f49e4198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:40.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.571+0000 7f49e3fff640 1 -- 172.21.15.164:0/3027117914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49e4102d90 msgr2=0x7f49e4198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:40.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.571+0000 7f49e3fff640 1 --2- 172.21.15.164:0/3027117914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49e4102d90 0x7f49e4198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.571+0000 7f49e3fff640 1 -- 172.21.15.164:0/3027117914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49e4108290 msgr2=0x7f49e419cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:40.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.571+0000 7f49e3fff640 1 --2- 172.21.15.164:0/3027117914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49e4108290 0x7f49e419cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.571+0000 7f49e3fff640 1 -- 172.21.15.164:0/3027117914 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e419d620 con 0x7f49e4103740 2024-09-19T08:54:40.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.571+0000 7f49e8cd5640 1 --2- 172.21.15.164:0/3027117914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49e4102d90 0x7f49e4198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:54:40.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.572+0000 7f49e3fff640 1 --2- 172.21.15.164:0/3027117914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49e4103740 0x7f49e4198bb0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f49d0005940 tx=0x7f49d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:40.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.572+0000 7f49e1ffb640 1 -- 172.21.15.164:0/3027117914 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d003f030 con 0x7f49e4103740 2024-09-19T08:54:40.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.572+0000 7f49e1ffb640 1 -- 172.21.15.164:0/3027117914 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49d0003ce0 con 0x7f49e4103740 2024-09-19T08:54:40.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.572+0000 7f49eaf39640 1 -- 172.21.15.164:0/3027117914 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49e419d8b0 con 0x7f49e4103740 2024-09-19T08:54:40.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.572+0000 7f49e1ffb640 1 -- 172.21.15.164:0/3027117914 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0035070 con 0x7f49e4103740 2024-09-19T08:54:40.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.572+0000 7f49eaf39640 1 -- 172.21.15.164:0/3027117914 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49e41a51f0 con 0x7f49e4103740 2024-09-19T08:54:40.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.574+0000 7f49e1ffb640 1 -- 172.21.15.164:0/3027117914 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f49d0042030 con 0x7f49e4103740 2024-09-19T08:54:40.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.575+0000 7f49ed312740 1 Processor -- start 2024-09-19T08:54:40.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.575+0000 7f49ed312740 1 -- start start 2024-09-19T08:54:40.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.575+0000 7f49ed312740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd2c34b40 0x55cdd2c33090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:40.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.575+0000 7f49ed312740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd2c335d0 0x55cdd2c33a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:40.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.575+0000 7f49ed312740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd2c42780 0x55cdd2c44b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:40.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.575+0000 7f49ed312740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cdd2b992f0 con 0x55cdd2c42780 2024-09-19T08:54:40.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.575+0000 7f49ed312740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55cdd2b40cf0 con 0x55cdd2c335d0 2024-09-19T08:54:40.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.575+0000 7f49ed312740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55cdd2b952b0 con 0x55cdd2c34b40 2024-09-19T08:54:40.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.575+0000 7f49e3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd2c42780 0x55cdd2c44b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:40.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.575+0000 7f49e3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd2c42780 0x55cdd2c44b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35170/0 (socket says 172.21.15.164:35170) 2024-09-19T08:54:40.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.575+0000 7f49e3fff640 1 -- 172.21.15.164:0/3685480537 learned_addr learned my addr 172.21.15.164:0/3685480537 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:40.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.576+0000 7f49e1ffb640 1 --2- 172.21.15.164:0/3027117914 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f49d40a33b0 0x7f49d40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:40.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.576+0000 7f49e94d6640 1 --2- 172.21.15.164:0/3685480537 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd2c34b40 0x55cdd2c33090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:40.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.576+0000 7f49e94d6640 1 --2- 172.21.15.164:0/3027117914 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f49d40a33b0 0x7f49d40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:40.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.576+0000 7f49e8cd5640 1 --2- 172.21.15.164:0/3685480537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd2c335d0 0x55cdd2c33a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:40.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.576+0000 7f49e3fff640 1 -- 172.21.15.164:0/3685480537 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd2c34b40 msgr2=0x55cdd2c33090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:40.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.576+0000 7f49e3fff640 1 --2- 172.21.15.164:0/3685480537 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd2c34b40 0x55cdd2c33090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.576+0000 7f49e3fff640 1 -- 172.21.15.164:0/3685480537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd2c335d0 msgr2=0x55cdd2c33a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:40.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.576+0000 7f49e3fff640 1 --2- 172.21.15.164:0/3685480537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd2c335d0 0x55cdd2c33a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.576+0000 7f49e3fff640 1 -- 172.21.15.164:0/3685480537 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdd2b253a0 con 0x55cdd2c42780 2024-09-19T08:54:40.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.576+0000 7f49e3fff640 1 --2- 172.21.15.164:0/3685480537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd2c42780 0x55cdd2c44b70 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f49d00bc220 tx=0x7f49d0086f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca7a284d0a2debe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:40.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.576+0000 7f49e1ffb640 1 -- 172.21.15.164:0/3027117914 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7f49d00f0260 con 0x7f49e4103740 2024-09-19T08:54:40.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.576+0000 7f49e94d6640 1 --2- 172.21.15.164:0/3027117914 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f49d40a33b0 0x7f49d40a5870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f49dc009990 tx=0x7f49dc0096a0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:40.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.576+0000 7f49ceffd640 1 -- 172.21.15.164:0/3685480537 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0049070 con 0x55cdd2c42780 2024-09-19T08:54:40.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.576+0000 7f49ceffd640 1 -- 172.21.15.164:0/3685480537 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49d00bcc20 con 0x55cdd2c42780 2024-09-19T08:54:40.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.577+0000 7f49ceffd640 1 -- 172.21.15.164:0/3685480537 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d00bc520 con 0x55cdd2c42780 2024-09-19T08:54:40.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.577+0000 7f49ed312740 1 -- 172.21.15.164:0/3685480537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd2c42780 msgr2=0x55cdd2c44b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:40.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.577+0000 7f49ed312740 1 --2- 172.21.15.164:0/3685480537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd2c42780 0x55cdd2c44b70 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f49d00bc220 tx=0x7f49d0086f10 comp rx=0 tx=0).stop 2024-09-19T08:54:40.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.577+0000 7f49ed312740 1 -- 172.21.15.164:0/3685480537 shutdown_connections 2024-09-19T08:54:40.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.577+0000 7f49ed312740 1 --2- 172.21.15.164:0/3685480537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd2c42780 0x55cdd2c44b70 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.577+0000 7f49ed312740 1 --2- 172.21.15.164:0/3685480537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd2c335d0 0x55cdd2c33a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.577+0000 7f49ed312740 1 --2- 172.21.15.164:0/3685480537 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd2c34b40 0x55cdd2c33090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.577+0000 7f49ed312740 1 -- 172.21.15.164:0/3685480537 >> 172.21.15.164:0/3685480537 conn(0x55cdd2bfc310 msgr2=0x55cdd2c2fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:40.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.577+0000 7f49ed312740 1 -- 172.21.15.164:0/3685480537 shutdown_connections 2024-09-19T08:54:40.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.577+0000 7f49ed312740 1 -- 172.21.15.164:0/3685480537 wait complete. 2024-09-19T08:54:40.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.578+0000 7f49ed312740 1 -- start start 2024-09-19T08:54:40.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.578+0000 7f49ed312740 1 Processor -- start 2024-09-19T08:54:40.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.578+0000 7f49ed312740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd2c335d0 0x55cdd2c42130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:40.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.578+0000 7f49ed312740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd2c34b40 0x55cdd2c3f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:40.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.578+0000 7f49ed312740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd2c42780 0x55cdd2c3f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:40.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.578+0000 7f49ed312740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cdd2c45240 con 0x55cdd2c34b40 2024-09-19T08:54:40.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.578+0000 7f49ed312740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55cdd2b54060 con 0x55cdd2c42780 2024-09-19T08:54:40.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.578+0000 7f49ed312740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55cdd2c453c0 con 0x55cdd2c335d0 2024-09-19T08:54:40.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.578+0000 7f49e3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd2c34b40 0x55cdd2c3f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:40.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.578+0000 7f49e3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd2c34b40 0x55cdd2c3f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35182/0 (socket says 172.21.15.164:35182) 2024-09-19T08:54:40.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.579+0000 7f49e3fff640 1 -- 172.21.15.164:0/3298846735 learned_addr learned my addr 172.21.15.164:0/3298846735 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:40.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.579+0000 7f49ed312740 10 client.0 _ll_register_callbacks cb 0x55cdd28a78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:54:40.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.579+0000 7f49e94d6640 1 --2- 172.21.15.164:0/3298846735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd2c42780 0x55cdd2c3f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:40.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.579+0000 7f49e8cd5640 1 --2- 172.21.15.164:0/3298846735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd2c335d0 0x55cdd2c42130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:40.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.579+0000 7f49e3fff640 1 -- 172.21.15.164:0/3298846735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd2c335d0 msgr2=0x55cdd2c42130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:40.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.579+0000 7f49e3fff640 1 --2- 172.21.15.164:0/3298846735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd2c335d0 0x55cdd2c42130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.579+0000 7f49e3fff640 1 -- 172.21.15.164:0/3298846735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd2c42780 msgr2=0x55cdd2c3f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:40.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.579+0000 7f49e3fff640 1 --2- 172.21.15.164:0/3298846735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd2c42780 0x55cdd2c3f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.579+0000 7f49e3fff640 1 -- 172.21.15.164:0/3298846735 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdd290a3c0 con 0x55cdd2c34b40 2024-09-19T08:54:40.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.579+0000 7f49e8cd5640 1 --2- 172.21.15.164:0/3298846735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd2c335d0 0x55cdd2c42130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:54:40.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.579+0000 7f49e3fff640 1 --2- 172.21.15.164:0/3298846735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd2c34b40 0x55cdd2c3f3e0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f49d00b8ef0 tx=0x7f49d002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab735cbe17d0c5fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:40.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.579+0000 7f49c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:54:40.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.579+0000 7f49c3fff640 1 -- 172.21.15.164:0/3298846735 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0049040 con 0x55cdd2c34b40 2024-09-19T08:54:40.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.579+0000 7f49c3fff640 1 -- 172.21.15.164:0/3298846735 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7f49d00b8b90 con 0x55cdd2c34b40 2024-09-19T08:54:40.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.579+0000 7f49ed312740 1 -- 172.21.15.164:0/3298846735 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49d00bcc20 con 0x55cdd2c34b40 2024-09-19T08:54:40.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.579+0000 7f49c3fff640 1 -- 172.21.15.164:0/3298846735 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f49d00b87c0 con 0x55cdd2c34b40 2024-09-19T08:54:40.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.579+0000 7f49ed312740 10 client.17361 fetch_fsmap learned FSMap version 32 2024-09-19T08:54:40.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.579+0000 7f49ed312740 1 -- 172.21.15.164:0/3298846735 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cdd2c45240 con 0x55cdd2c34b40 2024-09-19T08:54:40.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.580+0000 7f49c3fff640 1 -- 172.21.15.164:0/3298846735 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49d00bcc20 con 0x55cdd2c34b40 2024-09-19T08:54:40.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.580+0000 7f49c3fff640 1 -- 172.21.15.164:0/3298846735 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f49d00b8270 con 0x55cdd2c34b40 2024-09-19T08:54:40.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.580+0000 7f49ed312740 10 client.17361 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:54:40.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.580+0000 7f49ed312740 10 client.17361 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T08:54:40.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.580+0000 7f49ed312740 20 client.17361 populate_metadata read hostname 'smithi164' 2024-09-19T08:54:40.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.580+0000 7f49ed312740 1 --2- 172.21.15.164:0/3298846735 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55cdd2c51f00 0x55cdd2c54340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:40.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.580+0000 7f49ed312740 4 client.17361 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:54:40.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.580+0000 7f49ed312740 1 -- 172.21.15.164:0/3298846735 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cdd2ba6860 con 0x55cdd2c51f00 2024-09-19T08:54:40.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.580+0000 7f49e8cd5640 1 --2- 172.21.15.164:0/3298846735 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55cdd2c51f00 0x55cdd2c54340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:40.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.581+0000 7f49e8cd5640 1 --2- 172.21.15.164:0/3298846735 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55cdd2c51f00 0x55cdd2c54340 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=37cbc799680bfb67 server_cookie=cd5d33600e3e5ce4 in_seq=0 out_seq=0 2024-09-19T08:54:40.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.581+0000 7f49c3fff640 10 client.17361 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:54:40.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.581+0000 7f49c3fff640 1 -- 172.21.15.164:0/3298846735 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cdd2ba6860 con 0x55cdd2c51f00 2024-09-19T08:54:40.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.581+0000 7f49c3fff640 10 client.17361 handle_command_reply: tid=0 2024-09-19T08:54:40.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.581+0000 7f49ed312740 1 client.17361 shutdown 2024-09-19T08:54:40.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.581+0000 7f49ed312740 1 -- 172.21.15.164:0/3298846735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd2c34b40 msgr2=0x55cdd2c3f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:40.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.581+0000 7f49ed312740 1 --2- 172.21.15.164:0/3298846735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd2c34b40 0x55cdd2c3f3e0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f49d00b8ef0 tx=0x7f49d002a420 comp rx=0 tx=0).stop 2024-09-19T08:54:40.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.581+0000 7f49ed312740 1 -- 172.21.15.164:0/3298846735 shutdown_connections 2024-09-19T08:54:40.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.581+0000 7f49ed312740 1 --2- 172.21.15.164:0/3298846735 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55cdd2c51f00 0x55cdd2c54340 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.582+0000 7f49ed312740 1 --2- 172.21.15.164:0/3298846735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd2c42780 0x55cdd2c3f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.582+0000 7f49ed312740 1 --2- 172.21.15.164:0/3298846735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd2c34b40 0x55cdd2c3f3e0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.582+0000 7f49c3fff640 0 client.17361 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:54:40.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.582+0000 7f49ed312740 1 --2- 172.21.15.164:0/3298846735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd2c335d0 0x55cdd2c42130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.582+0000 7f49ed312740 1 -- 172.21.15.164:0/3298846735 >> 172.21.15.164:0/3298846735 conn(0x55cdd2bfc310 msgr2=0x55cdd2c35960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:40.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.582+0000 7f49ed312740 1 -- 172.21.15.164:0/3298846735 shutdown_connections 2024-09-19T08:54:40.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.582+0000 7f49ed312740 1 -- 172.21.15.164:0/3298846735 wait complete. 2024-09-19T08:54:40.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.582+0000 7f49ed312740 20 client.17361 trim_cache size 0 max 16384 2024-09-19T08:54:40.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.595+0000 7f49ed312740 1 Processor -- start 2024-09-19T08:54:40.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.595+0000 7f49ed312740 1 -- start start 2024-09-19T08:54:40.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.595+0000 7f49ed312740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd2c335d0 0x55cdd2c4e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:40.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.595+0000 7f49ed312740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd2c34b40 0x55cdd2c59c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:40.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.595+0000 7f49ed312740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd2c42780 0x55cdd2c5a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:40.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.595+0000 7f49ed312740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f49e41ab5a0 con 0x55cdd2c42780 2024-09-19T08:54:40.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.595+0000 7f49ed312740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f49e41ad0e0 con 0x55cdd2c335d0 2024-09-19T08:54:40.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.595+0000 7f49ed312740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f49e41abc70 con 0x55cdd2c34b40 2024-09-19T08:54:40.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.595+0000 7f49e8cd5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd2c335d0 0x55cdd2c4e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:40.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.595+0000 7f49e8cd5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd2c335d0 0x55cdd2c4e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:52416/0 (socket says 172.21.15.164:52416) 2024-09-19T08:54:40.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.595+0000 7f49e8cd5640 1 -- 172.21.15.164:0/274509580 learned_addr learned my addr 172.21.15.164:0/274509580 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:40.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.596+0000 7f49e94d6640 1 --2- 172.21.15.164:0/274509580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd2c42780 0x55cdd2c5a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:40.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.596+0000 7f49e3fff640 1 --2- 172.21.15.164:0/274509580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd2c34b40 0x55cdd2c59c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:40.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.596+0000 7f49e8cd5640 1 -- 172.21.15.164:0/274509580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd2c34b40 msgr2=0x55cdd2c59c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:40.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.596+0000 7f49e8cd5640 1 --2- 172.21.15.164:0/274509580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd2c34b40 0x55cdd2c59c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.596+0000 7f49e8cd5640 1 -- 172.21.15.164:0/274509580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd2c42780 msgr2=0x55cdd2c5a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:40.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.596+0000 7f49e8cd5640 1 --2- 172.21.15.164:0/274509580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd2c42780 0x55cdd2c5a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.596+0000 7f49e8cd5640 1 -- 172.21.15.164:0/274509580 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdd2c2aa30 con 0x55cdd2c335d0 2024-09-19T08:54:40.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.596+0000 7f49e8cd5640 1 --2- 172.21.15.164:0/274509580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd2c335d0 0x55cdd2c4e4a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f49d8000c00 tx=0x7f49d8005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a049bad00feb21bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:40.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.596+0000 7f49ce7fc640 1 -- 172.21.15.164:0/274509580 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d801a330 con 0x55cdd2c335d0 2024-09-19T08:54:40.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.596+0000 7f49ce7fc640 1 -- 172.21.15.164:0/274509580 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49d801e030 con 0x55cdd2c335d0 2024-09-19T08:54:40.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.596+0000 7f49ce7fc640 1 -- 172.21.15.164:0/274509580 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d8011840 con 0x55cdd2c335d0 2024-09-19T08:54:40.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.597+0000 7f49ed312740 1 -- 172.21.15.164:0/274509580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd2c335d0 msgr2=0x55cdd2c4e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:40.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.597+0000 7f49ed312740 1 --2- 172.21.15.164:0/274509580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd2c335d0 0x55cdd2c4e4a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f49d8000c00 tx=0x7f49d8005c50 comp rx=0 tx=0).stop 2024-09-19T08:54:40.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.597+0000 7f49ed312740 1 -- 172.21.15.164:0/274509580 shutdown_connections 2024-09-19T08:54:40.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.597+0000 7f49ed312740 1 --2- 172.21.15.164:0/274509580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd2c42780 0x55cdd2c5a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.597+0000 7f49ed312740 1 --2- 172.21.15.164:0/274509580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd2c34b40 0x55cdd2c59c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.597+0000 7f49ed312740 1 --2- 172.21.15.164:0/274509580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd2c335d0 0x55cdd2c4e4a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.597+0000 7f49ed312740 1 -- 172.21.15.164:0/274509580 >> 172.21.15.164:0/274509580 conn(0x55cdd2bfc310 msgr2=0x55cdd2c3e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:40.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.597+0000 7f49ed312740 1 -- 172.21.15.164:0/274509580 shutdown_connections 2024-09-19T08:54:40.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.597+0000 7f49ed312740 1 -- 172.21.15.164:0/274509580 wait complete. 2024-09-19T08:54:40.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.598+0000 7f49ed312740 1 -- start start 2024-09-19T08:54:40.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.598+0000 7f49ed312740 1 Processor -- start 2024-09-19T08:54:40.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.598+0000 7f49ed312740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd2c335d0 0x55cdd2c4a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:40.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.598+0000 7f49ed312740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd2c34b40 0x55cdd2c4ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:40.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.598+0000 7f49ed312740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd2c42780 0x55cdd2c475e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:40.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.598+0000 7f49ed312740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f49e41aca10 con 0x55cdd2c335d0 2024-09-19T08:54:40.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.598+0000 7f49ed312740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f49e41ac340 con 0x55cdd2c34b40 2024-09-19T08:54:40.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.598+0000 7f49ed312740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f49e41a95a0 con 0x55cdd2c42780 2024-09-19T08:54:40.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.598+0000 7f49ed312740 10 client.0 _ll_register_callbacks cb 0x7f49dc009eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:54:40.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.598+0000 7f49e8cd5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd2c335d0 0x55cdd2c4a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:40.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.598+0000 7f49e8cd5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd2c335d0 0x55cdd2c4a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35202/0 (socket says 172.21.15.164:35202) 2024-09-19T08:54:40.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.598+0000 7f49e8cd5640 1 -- 172.21.15.164:0/1902494267 learned_addr learned my addr 172.21.15.164:0/1902494267 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:40.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.598+0000 7f49e94d6640 1 --2- 172.21.15.164:0/1902494267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd2c42780 0x55cdd2c475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:40.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.598+0000 7f49e3fff640 1 --2- 172.21.15.164:0/1902494267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd2c34b40 0x55cdd2c4ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:40.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.599+0000 7f49e8cd5640 1 -- 172.21.15.164:0/1902494267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd2c42780 msgr2=0x55cdd2c475e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:40.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.599+0000 7f49e8cd5640 1 --2- 172.21.15.164:0/1902494267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd2c42780 0x55cdd2c475e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.599+0000 7f49e8cd5640 1 -- 172.21.15.164:0/1902494267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd2c34b40 msgr2=0x55cdd2c4ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:40.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.599+0000 7f49e8cd5640 1 --2- 172.21.15.164:0/1902494267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd2c34b40 0x55cdd2c4ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.599+0000 7f49e8cd5640 1 -- 172.21.15.164:0/1902494267 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdd2c4b140 con 0x55cdd2c335d0 2024-09-19T08:54:40.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.599+0000 7f49e8cd5640 1 --2- 172.21.15.164:0/1902494267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd2c335d0 0x55cdd2c4a610 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f49d8005d90 tx=0x7f49d8005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c333e9b7fe7b1d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:40.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.599+0000 7f49c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:54:40.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.599+0000 7f49c3fff640 1 -- 172.21.15.164:0/1902494267 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d8031060 con 0x55cdd2c335d0 2024-09-19T08:54:40.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.599+0000 7f49c3fff640 1 -- 172.21.15.164:0/1902494267 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7f49d8013dc0 con 0x55cdd2c335d0 2024-09-19T08:54:40.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.599+0000 7f49ed312740 1 -- 172.21.15.164:0/1902494267 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49d801e030 con 0x55cdd2c335d0 2024-09-19T08:54:40.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.599+0000 7f49c3fff640 1 -- 172.21.15.164:0/1902494267 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f49d801a330 con 0x55cdd2c335d0 2024-09-19T08:54:40.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.599+0000 7f49ed312740 10 client.17367 fetch_fsmap learned FSMap version 32 2024-09-19T08:54:40.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.599+0000 7f49ed312740 1 -- 172.21.15.164:0/1902494267 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cdd2cbd100 con 0x55cdd2c335d0 2024-09-19T08:54:40.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.600+0000 7f49c3fff640 1 -- 172.21.15.164:0/1902494267 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49d801a9b0 con 0x55cdd2c335d0 2024-09-19T08:54:40.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.600+0000 7f49c3fff640 1 -- 172.21.15.164:0/1902494267 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f49d8011a80 con 0x55cdd2c335d0 2024-09-19T08:54:40.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.600+0000 7f49ed312740 10 client.17367 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:54:40.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.600+0000 7f49ed312740 10 client.17367 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T08:54:40.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.600+0000 7f49ed312740 20 client.17367 populate_metadata read hostname 'smithi164' 2024-09-19T08:54:40.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.600+0000 7f49ed312740 1 --2- 172.21.15.164:0/1902494267 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55cdd2c51f00 0x55cdd2cbf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:40.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.600+0000 7f49ed312740 4 client.17367 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:54:40.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.600+0000 7f49ed312740 1 -- 172.21.15.164:0/1902494267 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f49d801e030 con 0x55cdd2c51f00 2024-09-19T08:54:40.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.600+0000 7f49e3fff640 1 --2- 172.21.15.164:0/1902494267 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55cdd2c51f00 0x55cdd2cbf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:40.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.601+0000 7f49c3fff640 10 client.17367 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:54:40.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.601+0000 7f49e3fff640 1 --2- 172.21.15.164:0/1902494267 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55cdd2c51f00 0x55cdd2cbf7d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=39d62367cc0532b0 server_cookie=6ae803b7a0f9af72 in_seq=0 out_seq=0 2024-09-19T08:54:40.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.601+0000 7f49c3fff640 1 -- 172.21.15.164:0/1902494267 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f49d801e030 con 0x55cdd2c51f00 2024-09-19T08:54:40.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.601+0000 7f49c3fff640 10 client.17367 handle_command_reply: tid=0 2024-09-19T08:54:40.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.601+0000 7f49ed312740 1 client.17367 shutdown 2024-09-19T08:54:40.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.601+0000 7f49ed312740 1 -- 172.21.15.164:0/1902494267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd2c335d0 msgr2=0x55cdd2c4a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:40.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.601+0000 7f49ed312740 1 --2- 172.21.15.164:0/1902494267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd2c335d0 0x55cdd2c4a610 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f49d8005d90 tx=0x7f49d8005dc0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.601+0000 7f49ed312740 1 -- 172.21.15.164:0/1902494267 shutdown_connections 2024-09-19T08:54:40.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.601+0000 7f49ed312740 1 --2- 172.21.15.164:0/1902494267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cdd2c42780 0x55cdd2c475e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.601+0000 7f49ed312740 1 --2- 172.21.15.164:0/1902494267 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55cdd2c51f00 0x55cdd2cbf7d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.601+0000 7f49ed312740 1 --2- 172.21.15.164:0/1902494267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdd2c34b40 0x55cdd2c4ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.601+0000 7f49ed312740 1 --2- 172.21.15.164:0/1902494267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdd2c335d0 0x55cdd2c4a610 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.601+0000 7f49ed312740 1 -- 172.21.15.164:0/1902494267 >> 172.21.15.164:0/1902494267 conn(0x55cdd2bfc310 msgr2=0x55cdd2c40c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:40.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.601+0000 7f49c3fff640 0 client.17367 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:54:40.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.601+0000 7f49ed312740 1 -- 172.21.15.164:0/1902494267 shutdown_connections 2024-09-19T08:54:40.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.601+0000 7f49ed312740 1 -- 172.21.15.164:0/1902494267 wait complete. 2024-09-19T08:54:40.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.602+0000 7f49ed312740 20 client.17367 trim_cache size 0 max 16384 2024-09-19T08:54:40.606 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:54:40.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.602+0000 7f49eaf39640 1 -- 172.21.15.164:0/3027117914 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f49d40a33b0 msgr2=0x7f49d40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:40.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.602+0000 7f49eaf39640 1 --2- 172.21.15.164:0/3027117914 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f49d40a33b0 0x7f49d40a5870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f49dc009990 tx=0x7f49dc0096a0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.602+0000 7f49eaf39640 1 -- 172.21.15.164:0/3027117914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49e4103740 msgr2=0x7f49e4198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:40.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.602+0000 7f49eaf39640 1 --2- 172.21.15.164:0/3027117914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49e4103740 0x7f49e4198bb0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f49d0005940 tx=0x7f49d00042f0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.602+0000 7f49eaf39640 1 -- 172.21.15.164:0/3027117914 shutdown_connections 2024-09-19T08:54:40.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.602+0000 7f49eaf39640 1 --2- 172.21.15.164:0/3027117914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49e4108290 0x7f49e419cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.602+0000 7f49eaf39640 1 --2- 172.21.15.164:0/3027117914 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f49d40a33b0 0x7f49d40a5870 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.602+0000 7f49eaf39640 1 --2- 172.21.15.164:0/3027117914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49e4103740 0x7f49e4198bb0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.602+0000 7f49eaf39640 1 --2- 172.21.15.164:0/3027117914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49e4102d90 0x7f49e4198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:40.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.602+0000 7f49eaf39640 1 -- 172.21.15.164:0/3027117914 >> 172.21.15.164:0/3027117914 conn(0x7f49e40fc7d0 msgr2=0x7f49e4105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:40.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.603+0000 7f49eaf39640 1 -- 172.21.15.164:0/3027117914 shutdown_connections 2024-09-19T08:54:40.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:40.603+0000 7f49eaf39640 1 -- 172.21.15.164:0/3027117914 wait complete. 2024-09-19T08:54:41.376 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T08:54:41.624 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:54:42.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.125+0000 7f6a8c023640 1 -- 172.21.15.164:0/3360634347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a84100a60 msgr2=0x7f6a84100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:42.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.125+0000 7f6a8c023640 1 --2- 172.21.15.164:0/3360634347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a84100a60 0x7f6a84100ec0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f6a74009930 tx=0x7f6a7402a420 comp rx=0 tx=0).stop 2024-09-19T08:54:42.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.126+0000 7f6a8c023640 1 -- 172.21.15.164:0/3360634347 shutdown_connections 2024-09-19T08:54:42.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.126+0000 7f6a8c023640 1 --2- 172.21.15.164:0/3360634347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a84101400 0x7f6a8410d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.126+0000 7f6a8c023640 1 --2- 172.21.15.164:0/3360634347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a84100a60 0x7f6a84100ec0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.126+0000 7f6a8c023640 1 --2- 172.21.15.164:0/3360634347 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a84106a60 0x7f6a84106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.126+0000 7f6a8c023640 1 -- 172.21.15.164:0/3360634347 >> 172.21.15.164:0/3360634347 conn(0x7f6a840fc780 msgr2=0x7f6a840feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:42.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.126+0000 7f6a8c023640 1 -- 172.21.15.164:0/3360634347 shutdown_connections 2024-09-19T08:54:42.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.126+0000 7f6a8c023640 1 -- 172.21.15.164:0/3360634347 wait complete. 2024-09-19T08:54:42.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.126+0000 7f6a8c023640 1 Processor -- start 2024-09-19T08:54:42.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.126+0000 7f6a8c023640 1 -- start start 2024-09-19T08:54:42.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.126+0000 7f6a8c023640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a84100a60 0x7f6a840fced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:42.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.126+0000 7f6a8c023640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a84101400 0x7f6a840fd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:42.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.126+0000 7f6a8c023640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a84106a60 0x7f6a840fd950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:42.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.126+0000 7f6a8c023640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6a84110e80 con 0x7f6a84100a60 2024-09-19T08:54:42.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.126+0000 7f6a8c023640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6a84110d00 con 0x7f6a84101400 2024-09-19T08:54:42.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.126+0000 7f6a8c023640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6a84111000 con 0x7f6a84106a60 2024-09-19T08:54:42.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.127+0000 7f6a89dbf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a84100a60 0x7f6a840fced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:42.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.127+0000 7f6a89dbf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a84100a60 0x7f6a840fced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35218/0 (socket says 172.21.15.164:35218) 2024-09-19T08:54:42.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.127+0000 7f6a89dbf640 1 -- 172.21.15.164:0/500302605 learned_addr learned my addr 172.21.15.164:0/500302605 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:42.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.127+0000 7f6a8a5c0640 1 --2- 172.21.15.164:0/500302605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a84106a60 0x7f6a840fd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:42.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.127+0000 7f6a895be640 1 --2- 172.21.15.164:0/500302605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a84101400 0x7f6a840fd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:42.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.127+0000 7f6a89dbf640 1 -- 172.21.15.164:0/500302605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a84106a60 msgr2=0x7f6a840fd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:42.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.127+0000 7f6a89dbf640 1 --2- 172.21.15.164:0/500302605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a84106a60 0x7f6a840fd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.127+0000 7f6a89dbf640 1 -- 172.21.15.164:0/500302605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a84101400 msgr2=0x7f6a840fd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:42.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.127+0000 7f6a89dbf640 1 --2- 172.21.15.164:0/500302605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a84101400 0x7f6a840fd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.127+0000 7f6a89dbf640 1 -- 172.21.15.164:0/500302605 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a841a69c0 con 0x7f6a84100a60 2024-09-19T08:54:42.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.127+0000 7f6a895be640 1 --2- 172.21.15.164:0/500302605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a84101400 0x7f6a840fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:54:42.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.127+0000 7f6a89dbf640 1 --2- 172.21.15.164:0/500302605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a84100a60 0x7f6a840fced0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f6a8000b4a0 tx=0x7f6a8000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:42.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.127+0000 7f6a7affd640 1 -- 172.21.15.164:0/500302605 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a80014070 con 0x7f6a84100a60 2024-09-19T08:54:42.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.127+0000 7f6a7affd640 1 -- 172.21.15.164:0/500302605 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a80010af0 con 0x7f6a84100a60 2024-09-19T08:54:42.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.127+0000 7f6a8c023640 1 -- 172.21.15.164:0/500302605 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a841a6cb0 con 0x7f6a84100a60 2024-09-19T08:54:42.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.127+0000 7f6a7affd640 1 -- 172.21.15.164:0/500302605 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a80017d10 con 0x7f6a84100a60 2024-09-19T08:54:42.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.127+0000 7f6a8c023640 1 -- 172.21.15.164:0/500302605 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a841a71a0 con 0x7f6a84100a60 2024-09-19T08:54:42.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.129+0000 7f6a7affd640 1 -- 172.21.15.164:0/500302605 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f6a80017070 con 0x7f6a84100a60 2024-09-19T08:54:42.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.130+0000 7f6a8e3fc740 1 Processor -- start 2024-09-19T08:54:42.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.130+0000 7f6a8e3fc740 1 -- start start 2024-09-19T08:54:42.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.130+0000 7f6a7affd640 1 --2- 172.21.15.164:0/500302605 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f6a700a33b0 0x7f6a700a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:42.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.130+0000 7f6a8e3fc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ada1efb40 0x562ada1ee090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:42.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.130+0000 7f6a8e3fc740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ada1ee5d0 0x562ada1eea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:42.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.130+0000 7f6a8e3fc740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562ada1f9370 0x562ada1fb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:42.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.130+0000 7f6a8e3fc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562ada1542f0 con 0x562ada1efb40 2024-09-19T08:54:42.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.130+0000 7f6a8e3fc740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562ada0fbcf0 con 0x562ada1ee5d0 2024-09-19T08:54:42.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.130+0000 7f6a8e3fc740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562ada1502b0 con 0x562ada1f9370 2024-09-19T08:54:42.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.130+0000 7f6a8a5c0640 1 --2- 172.21.15.164:0/500302605 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f6a700a33b0 0x7f6a700a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:42.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.130+0000 7f6a89dbf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ada1efb40 0x562ada1ee090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:42.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.130+0000 7f6a89dbf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ada1efb40 0x562ada1ee090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35224/0 (socket says 172.21.15.164:35224) 2024-09-19T08:54:42.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.130+0000 7f6a89dbf640 1 -- 172.21.15.164:0/3262211645 learned_addr learned my addr 172.21.15.164:0/3262211645 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:42.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.130+0000 7f6a7affd640 1 -- 172.21.15.164:0/500302605 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7f6a800cf050 con 0x7f6a84100a60 2024-09-19T08:54:42.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.131+0000 7f6a8a5c0640 1 --2- 172.21.15.164:0/3262211645 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562ada1f9370 0x562ada1fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:42.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.131+0000 7f6a895be640 1 --2- 172.21.15.164:0/3262211645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ada1ee5d0 0x562ada1eea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:42.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.131+0000 7f6a89dbf640 1 -- 172.21.15.164:0/3262211645 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562ada1f9370 msgr2=0x562ada1fb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:42.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.131+0000 7f6a89dbf640 1 --2- 172.21.15.164:0/3262211645 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562ada1f9370 0x562ada1fb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.131+0000 7f6a89dbf640 1 -- 172.21.15.164:0/3262211645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ada1ee5d0 msgr2=0x562ada1eea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:42.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.131+0000 7f6a89dbf640 1 --2- 172.21.15.164:0/3262211645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ada1ee5d0 0x562ada1eea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.131+0000 7f6a89dbf640 1 -- 172.21.15.164:0/3262211645 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ada0e03a0 con 0x562ada1efb40 2024-09-19T08:54:42.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.131+0000 7f6a8a5c0640 1 --2- 172.21.15.164:0/500302605 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f6a700a33b0 0x7f6a700a5870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c005c40 tx=0x7f6a7c0129e0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:42.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.131+0000 7f6a89dbf640 1 --2- 172.21.15.164:0/3262211645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ada1efb40 0x562ada1ee090 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f6a800a3bc0 tx=0x7f6a800047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d24f440b5acf1df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:42.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.131+0000 7f6a6ffff640 1 -- 172.21.15.164:0/3262211645 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8001e070 con 0x562ada1efb40 2024-09-19T08:54:42.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.131+0000 7f6a6ffff640 1 -- 172.21.15.164:0/3262211645 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a8009bb80 con 0x562ada1efb40 2024-09-19T08:54:42.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.131+0000 7f6a6ffff640 1 -- 172.21.15.164:0/3262211645 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a80097af0 con 0x562ada1efb40 2024-09-19T08:54:42.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.132+0000 7f6a8e3fc740 1 -- 172.21.15.164:0/3262211645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ada1efb40 msgr2=0x562ada1ee090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:42.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.132+0000 7f6a8e3fc740 1 --2- 172.21.15.164:0/3262211645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ada1efb40 0x562ada1ee090 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f6a800a3bc0 tx=0x7f6a800047d0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.132+0000 7f6a8e3fc740 1 -- 172.21.15.164:0/3262211645 shutdown_connections 2024-09-19T08:54:42.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.132+0000 7f6a8e3fc740 1 --2- 172.21.15.164:0/3262211645 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562ada1f9370 0x562ada1fb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.132+0000 7f6a8e3fc740 1 --2- 172.21.15.164:0/3262211645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ada1ee5d0 0x562ada1eea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.132+0000 7f6a8e3fc740 1 --2- 172.21.15.164:0/3262211645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ada1efb40 0x562ada1ee090 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.132+0000 7f6a8e3fc740 1 -- 172.21.15.164:0/3262211645 >> 172.21.15.164:0/3262211645 conn(0x562ada1b7310 msgr2=0x562ada1eac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:42.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.132+0000 7f6a8e3fc740 1 -- 172.21.15.164:0/3262211645 shutdown_connections 2024-09-19T08:54:42.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.132+0000 7f6a8e3fc740 1 -- 172.21.15.164:0/3262211645 wait complete. 2024-09-19T08:54:42.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.132+0000 7f6a8e3fc740 1 -- start start 2024-09-19T08:54:42.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.132+0000 7f6a8e3fc740 1 Processor -- start 2024-09-19T08:54:42.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.133+0000 7f6a8e3fc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ada1ee5d0 0x562ada1f8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:42.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.133+0000 7f6a8e3fc740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ada1efb40 0x562ada1f6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:42.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.133+0000 7f6a8e3fc740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562ada1f9370 0x562ada1f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:42.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.133+0000 7f6a8e3fc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562ada1fc070 con 0x562ada1ee5d0 2024-09-19T08:54:42.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.133+0000 7f6a8e3fc740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562ada10f060 con 0x562ada1efb40 2024-09-19T08:54:42.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.133+0000 7f6a8e3fc740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562ada1fc1f0 con 0x562ada1f9370 2024-09-19T08:54:42.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.133+0000 7f6a89dbf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ada1ee5d0 0x562ada1f8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:42.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.133+0000 7f6a8e3fc740 10 client.0 _ll_register_callbacks cb 0x562ad9e628c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:54:42.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.133+0000 7f6a89dbf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ada1ee5d0 0x562ada1f8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35236/0 (socket says 172.21.15.164:35236) 2024-09-19T08:54:42.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.133+0000 7f6a89dbf640 1 -- 172.21.15.164:0/777724918 learned_addr learned my addr 172.21.15.164:0/777724918 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:42.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.133+0000 7f6a895be640 1 --2- 172.21.15.164:0/777724918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ada1efb40 0x562ada1f6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:42.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.133+0000 7f6a8a5c0640 1 --2- 172.21.15.164:0/777724918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562ada1f9370 0x562ada1f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:42.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.133+0000 7f6a89dbf640 1 -- 172.21.15.164:0/777724918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562ada1f9370 msgr2=0x562ada1f65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:42.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.133+0000 7f6a89dbf640 1 --2- 172.21.15.164:0/777724918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562ada1f9370 0x562ada1f65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.133+0000 7f6a89dbf640 1 -- 172.21.15.164:0/777724918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ada1efb40 msgr2=0x562ada1f6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:42.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.133+0000 7f6a89dbf640 1 --2- 172.21.15.164:0/777724918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ada1efb40 0x562ada1f6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.133+0000 7f6a89dbf640 1 -- 172.21.15.164:0/777724918 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ad9ec53c0 con 0x562ada1ee5d0 2024-09-19T08:54:42.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.133+0000 7f6a89dbf640 1 --2- 172.21.15.164:0/777724918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ada1ee5d0 0x562ada1f8d50 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009420 tx=0x7f6a800ccc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd52abf327e67057 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:42.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.133+0000 7f6a6d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:54:42.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.134+0000 7f6a6d7fa640 1 -- 172.21.15.164:0/777724918 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8001e070 con 0x562ada1ee5d0 2024-09-19T08:54:42.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.134+0000 7f6a6d7fa640 1 -- 172.21.15.164:0/777724918 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7f6a80097af0 con 0x562ada1ee5d0 2024-09-19T08:54:42.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.134+0000 7f6a8e3fc740 1 -- 172.21.15.164:0/777724918 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a8009bb80 con 0x562ada1ee5d0 2024-09-19T08:54:42.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.134+0000 7f6a6d7fa640 1 -- 172.21.15.164:0/777724918 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6a8009b480 con 0x562ada1ee5d0 2024-09-19T08:54:42.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.134+0000 7f6a8e3fc740 10 client.17379 fetch_fsmap learned FSMap version 32 2024-09-19T08:54:42.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.134+0000 7f6a8e3fc740 1 -- 172.21.15.164:0/777724918 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562ada207af0 con 0x562ada1ee5d0 2024-09-19T08:54:42.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.134+0000 7f6a6d7fa640 1 -- 172.21.15.164:0/777724918 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a8009bb80 con 0x562ada1ee5d0 2024-09-19T08:54:42.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.134+0000 7f6a6d7fa640 1 -- 172.21.15.164:0/777724918 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f6a800a3c60 con 0x562ada1ee5d0 2024-09-19T08:54:42.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.134+0000 7f6a8e3fc740 10 client.17379 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:54:42.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.134+0000 7f6a8e3fc740 10 client.17379 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:54:42.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.134+0000 7f6a8e3fc740 20 client.17379 populate_metadata read hostname 'smithi164' 2024-09-19T08:54:42.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.134+0000 7f6a8e3fc740 1 --2- 172.21.15.164:0/777724918 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x562ada208e40 0x562ada20b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:42.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.134+0000 7f6a8e3fc740 4 client.17379 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:54:42.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.134+0000 7f6a8e3fc740 1 -- 172.21.15.164:0/777724918 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ada161860 con 0x562ada208e40 2024-09-19T08:54:42.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.135+0000 7f6a895be640 1 --2- 172.21.15.164:0/777724918 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x562ada208e40 0x562ada20b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:42.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.135+0000 7f6a895be640 1 --2- 172.21.15.164:0/777724918 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x562ada208e40 0x562ada20b230 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6cf5193eeaf197d server_cookie=a30420f23c11170 in_seq=0 out_seq=0 2024-09-19T08:54:42.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.135+0000 7f6a6d7fa640 10 client.17379 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:54:42.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.136+0000 7f6a6d7fa640 1 -- 172.21.15.164:0/777724918 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562ada161860 con 0x562ada208e40 2024-09-19T08:54:42.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.136+0000 7f6a6d7fa640 10 client.17379 handle_command_reply: tid=0 2024-09-19T08:54:42.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.136+0000 7f6a8e3fc740 1 client.17379 shutdown 2024-09-19T08:54:42.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.136+0000 7f6a8e3fc740 1 -- 172.21.15.164:0/777724918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ada1ee5d0 msgr2=0x562ada1f8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:42.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.136+0000 7f6a8e3fc740 1 --2- 172.21.15.164:0/777724918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ada1ee5d0 0x562ada1f8d50 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009420 tx=0x7f6a800ccc90 comp rx=0 tx=0).stop 2024-09-19T08:54:42.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.136+0000 7f6a8e3fc740 1 -- 172.21.15.164:0/777724918 shutdown_connections 2024-09-19T08:54:42.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.136+0000 7f6a8e3fc740 1 --2- 172.21.15.164:0/777724918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562ada1f9370 0x562ada1f65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.136+0000 7f6a8e3fc740 1 --2- 172.21.15.164:0/777724918 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x562ada208e40 0x562ada20b230 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.136+0000 7f6a8e3fc740 1 --2- 172.21.15.164:0/777724918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ada1efb40 0x562ada1f6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.136+0000 7f6a8e3fc740 1 --2- 172.21.15.164:0/777724918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ada1ee5d0 0x562ada1f8d50 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.136+0000 7f6a8e3fc740 1 -- 172.21.15.164:0/777724918 >> 172.21.15.164:0/777724918 conn(0x562ada1b7310 msgr2=0x562ada1f0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:42.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.136+0000 7f6a6d7fa640 0 client.17379 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:54:42.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.136+0000 7f6a8e3fc740 1 -- 172.21.15.164:0/777724918 shutdown_connections 2024-09-19T08:54:42.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.136+0000 7f6a8e3fc740 1 -- 172.21.15.164:0/777724918 wait complete. 2024-09-19T08:54:42.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.136+0000 7f6a8e3fc740 20 client.17379 trim_cache size 0 max 16384 2024-09-19T08:54:42.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.149+0000 7f6a8e3fc740 1 Processor -- start 2024-09-19T08:54:42.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.149+0000 7f6a8e3fc740 1 -- start start 2024-09-19T08:54:42.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.149+0000 7f6a8e3fc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ada1ee5d0 0x562ada1f1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:42.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.149+0000 7f6a8e3fc740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562ada1efb40 0x562ada1f1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:42.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.149+0000 7f6a8e3fc740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ada1f9370 0x562ada20e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:42.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.149+0000 7f6a8e3fc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6a841afb10 con 0x562ada1ee5d0 2024-09-19T08:54:42.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.149+0000 7f6a8e3fc740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6a841b1650 con 0x562ada1f9370 2024-09-19T08:54:42.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.149+0000 7f6a8e3fc740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6a841b01e0 con 0x562ada1efb40 2024-09-19T08:54:42.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.149+0000 7f6a89dbf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ada1ee5d0 0x562ada1f1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:42.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.149+0000 7f6a89dbf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ada1ee5d0 0x562ada1f1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35248/0 (socket says 172.21.15.164:35248) 2024-09-19T08:54:42.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.149+0000 7f6a89dbf640 1 -- 172.21.15.164:0/2159262587 learned_addr learned my addr 172.21.15.164:0/2159262587 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:42.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.149+0000 7f6a8a5c0640 1 --2- 172.21.15.164:0/2159262587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ada1f9370 0x562ada20e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:42.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.149+0000 7f6a895be640 1 --2- 172.21.15.164:0/2159262587 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562ada1efb40 0x562ada1f1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:42.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.150+0000 7f6a89dbf640 1 -- 172.21.15.164:0/2159262587 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562ada1efb40 msgr2=0x562ada1f1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:42.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.150+0000 7f6a89dbf640 1 --2- 172.21.15.164:0/2159262587 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562ada1efb40 0x562ada1f1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.150+0000 7f6a89dbf640 1 -- 172.21.15.164:0/2159262587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ada1f9370 msgr2=0x562ada20e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:42.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.150+0000 7f6a89dbf640 1 --2- 172.21.15.164:0/2159262587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ada1f9370 0x562ada20e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.150+0000 7f6a89dbf640 1 -- 172.21.15.164:0/2159262587 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ada1f6940 con 0x562ada1ee5d0 2024-09-19T08:54:42.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.150+0000 7f6a8a5c0640 1 --2- 172.21.15.164:0/2159262587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ada1f9370 0x562ada20e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:54:42.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.150+0000 7f6a89dbf640 1 --2- 172.21.15.164:0/2159262587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ada1ee5d0 0x562ada1f1120 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f6a800b1040 tx=0x7f6a80017730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63cc9e2d0e186e9f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:42.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.150+0000 7f6a6f7fe640 1 -- 172.21.15.164:0/2159262587 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8001e070 con 0x562ada1ee5d0 2024-09-19T08:54:42.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.150+0000 7f6a6f7fe640 1 -- 172.21.15.164:0/2159262587 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a800a40f0 con 0x562ada1ee5d0 2024-09-19T08:54:42.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.150+0000 7f6a6f7fe640 1 -- 172.21.15.164:0/2159262587 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a80097af0 con 0x562ada1ee5d0 2024-09-19T08:54:42.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.150+0000 7f6a8e3fc740 1 -- 172.21.15.164:0/2159262587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ada1ee5d0 msgr2=0x562ada1f1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:42.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.150+0000 7f6a8e3fc740 1 --2- 172.21.15.164:0/2159262587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ada1ee5d0 0x562ada1f1120 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f6a800b1040 tx=0x7f6a80017730 comp rx=0 tx=0).stop 2024-09-19T08:54:42.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.151+0000 7f6a8e3fc740 1 -- 172.21.15.164:0/2159262587 shutdown_connections 2024-09-19T08:54:42.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.151+0000 7f6a8e3fc740 1 --2- 172.21.15.164:0/2159262587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ada1f9370 0x562ada20e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.151+0000 7f6a8e3fc740 1 --2- 172.21.15.164:0/2159262587 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562ada1efb40 0x562ada1f1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.151+0000 7f6a8e3fc740 1 --2- 172.21.15.164:0/2159262587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ada1ee5d0 0x562ada1f1120 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.151+0000 7f6a8e3fc740 1 -- 172.21.15.164:0/2159262587 >> 172.21.15.164:0/2159262587 conn(0x562ada1b7310 msgr2=0x562ada1f50f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:42.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.151+0000 7f6a8e3fc740 1 -- 172.21.15.164:0/2159262587 shutdown_connections 2024-09-19T08:54:42.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.151+0000 7f6a8e3fc740 1 -- 172.21.15.164:0/2159262587 wait complete. 2024-09-19T08:54:42.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.151+0000 7f6a8e3fc740 1 -- start start 2024-09-19T08:54:42.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.151+0000 7f6a8e3fc740 1 Processor -- start 2024-09-19T08:54:42.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.151+0000 7f6a8e3fc740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ada1ee5d0 0x562ada20e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:42.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.151+0000 7f6a8e3fc740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ada1efb40 0x562ada20ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:42.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.151+0000 7f6a8e3fc740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562ada1f9370 0x562ada20bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:42.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.151+0000 7f6a8e3fc740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6a841b0f80 con 0x562ada1efb40 2024-09-19T08:54:42.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.151+0000 7f6a8e3fc740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6a841b08b0 con 0x562ada1ee5d0 2024-09-19T08:54:42.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.151+0000 7f6a8e3fc740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6a841adb10 con 0x562ada1f9370 2024-09-19T08:54:42.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.152+0000 7f6a8e3fc740 10 client.0 _ll_register_callbacks cb 0x7f6a8009b250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:54:42.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.152+0000 7f6a895be640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ada1efb40 0x562ada20ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:42.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.152+0000 7f6a895be640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ada1efb40 0x562ada20ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35258/0 (socket says 172.21.15.164:35258) 2024-09-19T08:54:42.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.152+0000 7f6a895be640 1 -- 172.21.15.164:0/553855846 learned_addr learned my addr 172.21.15.164:0/553855846 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:42.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.152+0000 7f6a89dbf640 1 --2- 172.21.15.164:0/553855846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ada1ee5d0 0x562ada20e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:42.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.152+0000 7f6a895be640 1 -- 172.21.15.164:0/553855846 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562ada1f9370 msgr2=0x562ada20bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:42.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.152+0000 7f6a8a5c0640 1 --2- 172.21.15.164:0/553855846 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562ada1f9370 0x562ada20bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:42.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.152+0000 7f6a895be640 1 --2- 172.21.15.164:0/553855846 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562ada1f9370 0x562ada20bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.152+0000 7f6a895be640 1 -- 172.21.15.164:0/553855846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ada1ee5d0 msgr2=0x562ada20e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:42.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.152+0000 7f6a895be640 1 --2- 172.21.15.164:0/553855846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ada1ee5d0 0x562ada20e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.152+0000 7f6a895be640 1 -- 172.21.15.164:0/553855846 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ada20c510 con 0x562ada1efb40 2024-09-19T08:54:42.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.152+0000 7f6a895be640 1 --2- 172.21.15.164:0/553855846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ada1efb40 0x562ada20ba60 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f6a74005ab0 tx=0x7f6a740041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0a52c1a158554b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:42.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.152+0000 7f6a6d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:54:42.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.152+0000 7f6a6d7fa640 1 -- 172.21.15.164:0/553855846 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a74047070 con 0x562ada1efb40 2024-09-19T08:54:42.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.152+0000 7f6a6d7fa640 1 -- 172.21.15.164:0/553855846 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7f6a74005240 con 0x562ada1efb40 2024-09-19T08:54:42.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.152+0000 7f6a8e3fc740 1 -- 172.21.15.164:0/553855846 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a800a40f0 con 0x562ada1efb40 2024-09-19T08:54:42.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.152+0000 7f6a6d7fa640 1 -- 172.21.15.164:0/553855846 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6a74050050 con 0x562ada1efb40 2024-09-19T08:54:42.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.152+0000 7f6a8e3fc740 10 client.17385 fetch_fsmap learned FSMap version 32 2024-09-19T08:54:42.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.152+0000 7f6a8e3fc740 1 -- 172.21.15.164:0/553855846 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562ada273570 con 0x562ada1efb40 2024-09-19T08:54:42.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.153+0000 7f6a6d7fa640 1 -- 172.21.15.164:0/553855846 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a800a40f0 con 0x562ada1efb40 2024-09-19T08:54:42.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.153+0000 7f6a6d7fa640 1 -- 172.21.15.164:0/553855846 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f6a7404a020 con 0x562ada1efb40 2024-09-19T08:54:42.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.153+0000 7f6a8e3fc740 10 client.17385 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:54:42.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.153+0000 7f6a8e3fc740 10 client.17385 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:54:42.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.153+0000 7f6a8e3fc740 20 client.17385 populate_metadata read hostname 'smithi164' 2024-09-19T08:54:42.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.153+0000 7f6a8e3fc740 1 --2- 172.21.15.164:0/553855846 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x562ada273d40 0x562ada276160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:42.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.153+0000 7f6a8e3fc740 4 client.17385 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:54:42.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.153+0000 7f6a8e3fc740 1 -- 172.21.15.164:0/553855846 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562ada15bd70 con 0x562ada273d40 2024-09-19T08:54:42.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.153+0000 7f6a89dbf640 1 --2- 172.21.15.164:0/553855846 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x562ada273d40 0x562ada276160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:42.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.154+0000 7f6a89dbf640 1 --2- 172.21.15.164:0/553855846 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x562ada273d40 0x562ada276160 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=738722df0be9f035 server_cookie=f41fb8844c9af059 in_seq=0 out_seq=0 2024-09-19T08:54:42.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.154+0000 7f6a6d7fa640 10 client.17385 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:54:42.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.154+0000 7f6a6d7fa640 1 -- 172.21.15.164:0/553855846 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562ada15bd70 con 0x562ada273d40 2024-09-19T08:54:42.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.154+0000 7f6a6d7fa640 10 client.17385 handle_command_reply: tid=0 2024-09-19T08:54:42.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.154+0000 7f6a8e3fc740 1 client.17385 shutdown 2024-09-19T08:54:42.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.154+0000 7f6a8e3fc740 1 -- 172.21.15.164:0/553855846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ada1efb40 msgr2=0x562ada20ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:42.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.154+0000 7f6a8e3fc740 1 --2- 172.21.15.164:0/553855846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ada1efb40 0x562ada20ba60 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f6a74005ab0 tx=0x7f6a740041a0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.154+0000 7f6a8e3fc740 1 -- 172.21.15.164:0/553855846 shutdown_connections 2024-09-19T08:54:42.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.154+0000 7f6a8e3fc740 1 --2- 172.21.15.164:0/553855846 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562ada1f9370 0x562ada20bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.154+0000 7f6a8e3fc740 1 --2- 172.21.15.164:0/553855846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ada1efb40 0x562ada20ba60 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.154+0000 7f6a8e3fc740 1 --2- 172.21.15.164:0/553855846 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x562ada273d40 0x562ada276160 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.154+0000 7f6a8e3fc740 1 --2- 172.21.15.164:0/553855846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ada1ee5d0 0x562ada20e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.154+0000 7f6a6d7fa640 0 client.17385 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:54:42.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.154+0000 7f6a8e3fc740 1 -- 172.21.15.164:0/553855846 >> 172.21.15.164:0/553855846 conn(0x562ada1b7310 msgr2=0x562ada1e94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:42.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.154+0000 7f6a8e3fc740 1 -- 172.21.15.164:0/553855846 shutdown_connections 2024-09-19T08:54:42.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.154+0000 7f6a8e3fc740 1 -- 172.21.15.164:0/553855846 wait complete. 2024-09-19T08:54:42.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.154+0000 7f6a8e3fc740 20 client.17385 trim_cache size 0 max 16384 2024-09-19T08:54:42.161 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:54:42.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.155+0000 7f6a8c023640 1 -- 172.21.15.164:0/500302605 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f6a700a33b0 msgr2=0x7f6a700a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:42.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.155+0000 7f6a8c023640 1 --2- 172.21.15.164:0/500302605 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f6a700a33b0 0x7f6a700a5870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c005c40 tx=0x7f6a7c0129e0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.155+0000 7f6a8c023640 1 -- 172.21.15.164:0/500302605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a84100a60 msgr2=0x7f6a840fced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:42.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.155+0000 7f6a8c023640 1 --2- 172.21.15.164:0/500302605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a84100a60 0x7f6a840fced0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f6a8000b4a0 tx=0x7f6a8000b960 comp rx=0 tx=0).stop 2024-09-19T08:54:42.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.155+0000 7f6a8c023640 1 -- 172.21.15.164:0/500302605 shutdown_connections 2024-09-19T08:54:42.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.155+0000 7f6a8c023640 1 --2- 172.21.15.164:0/500302605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a84106a60 0x7f6a840fd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.155+0000 7f6a8c023640 1 --2- 172.21.15.164:0/500302605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a84101400 0x7f6a840fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.155+0000 7f6a8c023640 1 --2- 172.21.15.164:0/500302605 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f6a700a33b0 0x7f6a700a5870 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.155+0000 7f6a8c023640 1 --2- 172.21.15.164:0/500302605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a84100a60 0x7f6a840fced0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:42.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.155+0000 7f6a8c023640 1 -- 172.21.15.164:0/500302605 >> 172.21.15.164:0/500302605 conn(0x7f6a840fc780 msgr2=0x7f6a8406aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:42.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.155+0000 7f6a8c023640 1 -- 172.21.15.164:0/500302605 shutdown_connections 2024-09-19T08:54:42.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:42.155+0000 7f6a8c023640 1 -- 172.21.15.164:0/500302605 wait complete. 2024-09-19T08:54:42.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:42 smithi164 ceph-mon[30928]: pgmap v275: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:54:42.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:42 smithi170 ceph-mon[35072]: pgmap v275: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:54:42.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:42 smithi173 ceph-mon[35306]: pgmap v275: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:54:43.902 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T08:54:43.903 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T08:54:44.166 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:54:44.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:44 smithi164 ceph-mon[30928]: pgmap v276: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:54:44.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:44 smithi170 ceph-mon[35072]: pgmap v276: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:54:44.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:44 smithi173 ceph-mon[35306]: pgmap v276: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:54:44.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.636+0000 7fa747b3d640 1 -- 172.21.15.164:0/3989841894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa740101420 msgr2=0x7fa74010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:44.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.636+0000 7fa747b3d640 1 --2- 172.21.15.164:0/3989841894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa740101420 0x7fa74010d950 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fa738009cd0 tx=0x7fa73802a8e0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.636+0000 7fa747b3d640 1 -- 172.21.15.164:0/3989841894 shutdown_connections 2024-09-19T08:54:44.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.636+0000 7fa747b3d640 1 --2- 172.21.15.164:0/3989841894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa740101420 0x7fa74010d950 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.636+0000 7fa747b3d640 1 --2- 172.21.15.164:0/3989841894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa740100a80 0x7fa740100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.636+0000 7fa747b3d640 1 --2- 172.21.15.164:0/3989841894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa740106a80 0x7fa740106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.636+0000 7fa747b3d640 1 -- 172.21.15.164:0/3989841894 >> 172.21.15.164:0/3989841894 conn(0x7fa7400fc7a0 msgr2=0x7fa7400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:44.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.636+0000 7fa747b3d640 1 -- 172.21.15.164:0/3989841894 shutdown_connections 2024-09-19T08:54:44.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.636+0000 7fa747b3d640 1 -- 172.21.15.164:0/3989841894 wait complete. 2024-09-19T08:54:44.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.637+0000 7fa747b3d640 1 Processor -- start 2024-09-19T08:54:44.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.637+0000 7fa747b3d640 1 -- start start 2024-09-19T08:54:44.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.637+0000 7fa747b3d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa740100a80 0x7fa740199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:44.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.637+0000 7fa747b3d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa740101420 0x7fa740199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:44.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.637+0000 7fa747b3d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa740106a80 0x7fa7401a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:44.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.637+0000 7fa747b3d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa740110ed0 con 0x7fa740101420 2024-09-19T08:54:44.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.637+0000 7fa747b3d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa740110d50 con 0x7fa740106a80 2024-09-19T08:54:44.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.637+0000 7fa747b3d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa740111050 con 0x7fa740100a80 2024-09-19T08:54:44.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.637+0000 7fa7450d8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa740101420 0x7fa740199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:44.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.637+0000 7fa7450d8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa740101420 0x7fa740199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44902/0 (socket says 172.21.15.164:44902) 2024-09-19T08:54:44.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.637+0000 7fa7450d8640 1 -- 172.21.15.164:0/305410094 learned_addr learned my addr 172.21.15.164:0/305410094 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:44.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.637+0000 7fa7460da640 1 --2- 172.21.15.164:0/305410094 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa740106a80 0x7fa7401a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:44.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.637+0000 7fa7458d9640 1 --2- 172.21.15.164:0/305410094 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa740100a80 0x7fa740199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:44.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.638+0000 7fa7450d8640 1 -- 172.21.15.164:0/305410094 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa740100a80 msgr2=0x7fa740199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:44.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.638+0000 7fa7450d8640 1 --2- 172.21.15.164:0/305410094 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa740100a80 0x7fa740199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.638+0000 7fa7450d8640 1 -- 172.21.15.164:0/305410094 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa740106a80 msgr2=0x7fa7401a41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:44.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.638+0000 7fa7450d8640 1 --2- 172.21.15.164:0/305410094 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa740106a80 0x7fa7401a41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.638+0000 7fa7450d8640 1 -- 172.21.15.164:0/305410094 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7401a4700 con 0x7fa740101420 2024-09-19T08:54:44.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.638+0000 7fa7450d8640 1 --2- 172.21.15.164:0/305410094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa740101420 0x7fa740199680 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fa73000c920 tx=0x7fa73000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:44.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.638+0000 7fa736ffd640 1 -- 172.21.15.164:0/305410094 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa730007bb0 con 0x7fa740101420 2024-09-19T08:54:44.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.638+0000 7fa747b3d640 1 -- 172.21.15.164:0/305410094 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7401a48d0 con 0x7fa740101420 2024-09-19T08:54:44.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.638+0000 7fa736ffd640 1 -- 172.21.15.164:0/305410094 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa730007d50 con 0x7fa740101420 2024-09-19T08:54:44.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.638+0000 7fa736ffd640 1 -- 172.21.15.164:0/305410094 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7300187d0 con 0x7fa740101420 2024-09-19T08:54:44.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.638+0000 7fa747b3d640 1 -- 172.21.15.164:0/305410094 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7401a4de0 con 0x7fa740101420 2024-09-19T08:54:44.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.640+0000 7fa736ffd640 1 -- 172.21.15.164:0/305410094 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fa73000f450 con 0x7fa740101420 2024-09-19T08:54:44.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.640+0000 7fa749f16740 1 Processor -- start 2024-09-19T08:54:44.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.641+0000 7fa749f16740 1 -- start start 2024-09-19T08:54:44.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.641+0000 7fa749f16740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cc4ddbb60 0x562cc4ddbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:44.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.641+0000 7fa749f16740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cc4ddc480 0x562cc4de6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:44.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.641+0000 7fa749f16740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cc4de6d40 0x562cc4de9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:44.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.641+0000 7fa749f16740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562cc4d41280 con 0x562cc4ddc480 2024-09-19T08:54:44.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.641+0000 7fa749f16740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562cc4ce8b10 con 0x562cc4ddbb60 2024-09-19T08:54:44.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.641+0000 7fa749f16740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562cc4d3d240 con 0x562cc4de6d40 2024-09-19T08:54:44.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.641+0000 7fa736ffd640 1 --2- 172.21.15.164:0/305410094 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa72c0a33b0 0x7fa72c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:44.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.641+0000 7fa7458d9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cc4ddc480 0x562cc4de6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:44.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.641+0000 7fa7458d9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cc4ddc480 0x562cc4de6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44916/0 (socket says 172.21.15.164:44916) 2024-09-19T08:54:44.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.641+0000 7fa7458d9640 1 -- 172.21.15.164:0/1446661064 learned_addr learned my addr 172.21.15.164:0/1446661064 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:44.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.641+0000 7fa7450d8640 1 --2- 172.21.15.164:0/1446661064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cc4de6d40 0x562cc4de9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:44.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.641+0000 7fa7460da640 1 --2- 172.21.15.164:0/305410094 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa72c0a33b0 0x7fa72c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:44.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.641+0000 7fa736ffd640 1 -- 172.21.15.164:0/305410094 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7fa7300cfde0 con 0x7fa740101420 2024-09-19T08:54:44.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.641+0000 7fa7460da640 1 --2- 172.21.15.164:0/1446661064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cc4ddbb60 0x562cc4ddbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:44.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.641+0000 7fa7458d9640 1 -- 172.21.15.164:0/1446661064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cc4de6d40 msgr2=0x562cc4de9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:44.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.641+0000 7fa7458d9640 1 --2- 172.21.15.164:0/1446661064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cc4de6d40 0x562cc4de9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.641+0000 7fa7458d9640 1 -- 172.21.15.164:0/1446661064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cc4ddbb60 msgr2=0x562cc4ddbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:44.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.641+0000 7fa7458d9640 1 --2- 172.21.15.164:0/1446661064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cc4ddbb60 0x562cc4ddbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.641+0000 7fa7458d9640 1 -- 172.21.15.164:0/1446661064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cc4cf51f0 con 0x562cc4ddc480 2024-09-19T08:54:44.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.642+0000 7fa7458d9640 1 --2- 172.21.15.164:0/1446661064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cc4ddc480 0x562cc4de6800 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00ac90 tx=0x7fa73c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab18248e4fb1f7bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:44.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.642+0000 7fa7460da640 1 --2- 172.21.15.164:0/305410094 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa72c0a33b0 0x7fa72c0a5870 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa738006010 tx=0x7fa738009970 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:44.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.642+0000 7fa72bfff640 1 -- 172.21.15.164:0/1446661064 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c01d070 con 0x562cc4ddc480 2024-09-19T08:54:44.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.642+0000 7fa72bfff640 1 -- 172.21.15.164:0/1446661064 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa73c018b20 con 0x562cc4ddc480 2024-09-19T08:54:44.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.642+0000 7fa72bfff640 1 -- 172.21.15.164:0/1446661064 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c010b40 con 0x562cc4ddc480 2024-09-19T08:54:44.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.643+0000 7fa749f16740 1 -- 172.21.15.164:0/1446661064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cc4ddc480 msgr2=0x562cc4de6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:44.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.643+0000 7fa749f16740 1 --2- 172.21.15.164:0/1446661064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cc4ddc480 0x562cc4de6800 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00ac90 tx=0x7fa73c00f790 comp rx=0 tx=0).stop 2024-09-19T08:54:44.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.643+0000 7fa749f16740 1 -- 172.21.15.164:0/1446661064 shutdown_connections 2024-09-19T08:54:44.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.643+0000 7fa749f16740 1 --2- 172.21.15.164:0/1446661064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cc4de6d40 0x562cc4de9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.643+0000 7fa749f16740 1 --2- 172.21.15.164:0/1446661064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cc4ddc480 0x562cc4de6800 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.643+0000 7fa749f16740 1 --2- 172.21.15.164:0/1446661064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cc4ddbb60 0x562cc4ddbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.643+0000 7fa749f16740 1 -- 172.21.15.164:0/1446661064 >> 172.21.15.164:0/1446661064 conn(0x562cc4dd80f0 msgr2=0x562cc4dd84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:44.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.643+0000 7fa749f16740 1 -- 172.21.15.164:0/1446661064 shutdown_connections 2024-09-19T08:54:44.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.643+0000 7fa749f16740 1 -- 172.21.15.164:0/1446661064 wait complete. 2024-09-19T08:54:44.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.643+0000 7fa749f16740 1 -- start start 2024-09-19T08:54:44.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.643+0000 7fa749f16740 1 Processor -- start 2024-09-19T08:54:44.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.644+0000 7fa749f16740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cc4ddbb60 0x562cc4de3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:44.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.644+0000 7fa749f16740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cc4ddc480 0x562cc4de4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:44.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.644+0000 7fa749f16740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cc4de6d40 0x562cc4de4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:44.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.644+0000 7fa749f16740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562cc4de9910 con 0x562cc4de6d40 2024-09-19T08:54:44.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.644+0000 7fa749f16740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562cc4cfbf10 con 0x562cc4ddbb60 2024-09-19T08:54:44.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.644+0000 7fa749f16740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562cc4de9a90 con 0x562cc4ddc480 2024-09-19T08:54:44.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.644+0000 7fa749f16740 10 client.0 _ll_register_callbacks cb 0x562cc4cc4a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:54:44.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.644+0000 7fa7460da640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cc4de6d40 0x562cc4de4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:44.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.644+0000 7fa7460da640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cc4de6d40 0x562cc4de4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44932/0 (socket says 172.21.15.164:44932) 2024-09-19T08:54:44.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.644+0000 7fa7460da640 1 -- 172.21.15.164:0/2370636448 learned_addr learned my addr 172.21.15.164:0/2370636448 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:44.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.644+0000 7fa7458d9640 1 --2- 172.21.15.164:0/2370636448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cc4ddbb60 0x562cc4de3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:44.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.644+0000 7fa7450d8640 1 --2- 172.21.15.164:0/2370636448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cc4ddc480 0x562cc4de4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:44.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.644+0000 7fa7460da640 1 -- 172.21.15.164:0/2370636448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cc4ddc480 msgr2=0x562cc4de4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:44.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.644+0000 7fa7460da640 1 --2- 172.21.15.164:0/2370636448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cc4ddc480 0x562cc4de4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.644+0000 7fa7460da640 1 -- 172.21.15.164:0/2370636448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cc4ddbb60 msgr2=0x562cc4de3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:44.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.644+0000 7fa7460da640 1 --2- 172.21.15.164:0/2370636448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cc4ddbb60 0x562cc4de3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.644+0000 7fa7460da640 1 -- 172.21.15.164:0/2370636448 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cc4ab26a0 con 0x562cc4de6d40 2024-09-19T08:54:44.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.644+0000 7fa7450d8640 1 --2- 172.21.15.164:0/2370636448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cc4ddc480 0x562cc4de4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:54:44.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.644+0000 7fa7460da640 1 --2- 172.21.15.164:0/2370636448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cc4de6d40 0x562cc4de4660 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fa738003ba0 tx=0x7fa7380095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7218ef703dbd854d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:44.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.644+0000 7fa7297fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:54:44.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.645+0000 7fa7297fa640 1 -- 172.21.15.164:0/2370636448 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7380305a0 con 0x562cc4de6d40 2024-09-19T08:54:44.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.645+0000 7fa7297fa640 1 -- 172.21.15.164:0/2370636448 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7fa738030740 con 0x562cc4de6d40 2024-09-19T08:54:44.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.645+0000 7fa749f16740 1 -- 172.21.15.164:0/2370636448 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa73c018b20 con 0x562cc4de6d40 2024-09-19T08:54:44.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.645+0000 7fa7297fa640 1 -- 172.21.15.164:0/2370636448 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa738030980 con 0x562cc4de6d40 2024-09-19T08:54:44.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.645+0000 7fa749f16740 10 client.17397 fetch_fsmap learned FSMap version 32 2024-09-19T08:54:44.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.645+0000 7fa749f16740 1 -- 172.21.15.164:0/2370636448 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562cc4df5450 con 0x562cc4de6d40 2024-09-19T08:54:44.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.645+0000 7fa7297fa640 1 -- 172.21.15.164:0/2370636448 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa73c018b20 con 0x562cc4de6d40 2024-09-19T08:54:44.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.645+0000 7fa7297fa640 1 -- 172.21.15.164:0/2370636448 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fa738030070 con 0x562cc4de6d40 2024-09-19T08:54:44.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.645+0000 7fa749f16740 10 client.17397 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:54:44.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.645+0000 7fa749f16740 10 client.17397 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:54:44.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.645+0000 7fa749f16740 20 client.17397 populate_metadata read hostname 'smithi164' 2024-09-19T08:54:44.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.645+0000 7fa749f16740 1 --2- 172.21.15.164:0/2370636448 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x562cc4df6860 0x562cc4df8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:44.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.645+0000 7fa749f16740 4 client.17397 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:54:44.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.645+0000 7fa749f16740 1 -- 172.21.15.164:0/2370636448 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cc4d4e7f0 con 0x562cc4df6860 2024-09-19T08:54:44.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.646+0000 7fa7458d9640 1 --2- 172.21.15.164:0/2370636448 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x562cc4df6860 0x562cc4df8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:44.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.646+0000 7fa7458d9640 1 --2- 172.21.15.164:0/2370636448 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x562cc4df6860 0x562cc4df8cb0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32e357a10b8f6f09 server_cookie=21fc3e6466d7db10 in_seq=0 out_seq=0 2024-09-19T08:54:44.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.646+0000 7fa7297fa640 10 client.17397 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:54:44.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.647+0000 7fa7297fa640 1 -- 172.21.15.164:0/2370636448 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562cc4d4e7f0 con 0x562cc4df6860 2024-09-19T08:54:44.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.647+0000 7fa7297fa640 10 client.17397 handle_command_reply: tid=0 2024-09-19T08:54:44.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.647+0000 7fa749f16740 1 client.17397 shutdown 2024-09-19T08:54:44.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.647+0000 7fa749f16740 1 -- 172.21.15.164:0/2370636448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cc4de6d40 msgr2=0x562cc4de4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:44.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.647+0000 7fa749f16740 1 --2- 172.21.15.164:0/2370636448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cc4de6d40 0x562cc4de4660 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fa738003ba0 tx=0x7fa7380095c0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.647+0000 7fa749f16740 1 -- 172.21.15.164:0/2370636448 shutdown_connections 2024-09-19T08:54:44.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.647+0000 7fa749f16740 1 --2- 172.21.15.164:0/2370636448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cc4de6d40 0x562cc4de4660 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.647+0000 7fa749f16740 1 --2- 172.21.15.164:0/2370636448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cc4ddc480 0x562cc4de4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.647+0000 7fa749f16740 1 --2- 172.21.15.164:0/2370636448 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x562cc4df6860 0x562cc4df8cb0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.647+0000 7fa7297fa640 0 client.17397 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:54:44.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.647+0000 7fa749f16740 1 --2- 172.21.15.164:0/2370636448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cc4ddbb60 0x562cc4de3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.647+0000 7fa749f16740 1 -- 172.21.15.164:0/2370636448 >> 172.21.15.164:0/2370636448 conn(0x562cc4dd80f0 msgr2=0x562cc4ddde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:44.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.647+0000 7fa749f16740 1 -- 172.21.15.164:0/2370636448 shutdown_connections 2024-09-19T08:54:44.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.647+0000 7fa749f16740 1 -- 172.21.15.164:0/2370636448 wait complete. 2024-09-19T08:54:44.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.647+0000 7fa749f16740 20 client.17397 trim_cache size 0 max 16384 2024-09-19T08:54:44.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.664+0000 7fa749f16740 1 Processor -- start 2024-09-19T08:54:44.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.664+0000 7fa749f16740 1 -- start start 2024-09-19T08:54:44.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.664+0000 7fa749f16740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cc4ddbb60 0x562cc4dde610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:44.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.665+0000 7fa749f16740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cc4ddc480 0x562cc4de5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:44.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.665+0000 7fa749f16740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cc4de6d40 0x562cc4dfe580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:44.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.665+0000 7fa749f16740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa7401ad6a0 con 0x562cc4ddbb60 2024-09-19T08:54:44.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.665+0000 7fa749f16740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa7401af1e0 con 0x562cc4ddc480 2024-09-19T08:54:44.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.665+0000 7fa749f16740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa7401add70 con 0x562cc4de6d40 2024-09-19T08:54:44.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.665+0000 7fa7458d9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cc4ddbb60 0x562cc4dde610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:44.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.665+0000 7fa7458d9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cc4ddbb60 0x562cc4dde610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44938/0 (socket says 172.21.15.164:44938) 2024-09-19T08:54:44.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.665+0000 7fa7458d9640 1 -- 172.21.15.164:0/1737511583 learned_addr learned my addr 172.21.15.164:0/1737511583 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:44.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.665+0000 7fa7458d9640 1 -- 172.21.15.164:0/1737511583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cc4de6d40 msgr2=0x562cc4dfe580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:44.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.665+0000 7fa7458d9640 1 --2- 172.21.15.164:0/1737511583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cc4de6d40 0x562cc4dfe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.665+0000 7fa7458d9640 1 -- 172.21.15.164:0/1737511583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cc4ddc480 msgr2=0x562cc4de5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:44.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.665+0000 7fa7458d9640 1 --2- 172.21.15.164:0/1737511583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cc4ddc480 0x562cc4de5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.665+0000 7fa7458d9640 1 -- 172.21.15.164:0/1737511583 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cc4dd7f20 con 0x562cc4ddbb60 2024-09-19T08:54:44.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.665+0000 7fa7458d9640 1 --2- 172.21.15.164:0/1737511583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cc4ddbb60 0x562cc4dde610 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa73c010730 tx=0x7fa73c0106c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b2cdf5cf145d778 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:44.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.665+0000 7fa72b7fe640 1 -- 172.21.15.164:0/1737511583 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c031070 con 0x562cc4ddbb60 2024-09-19T08:54:44.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.665+0000 7fa72b7fe640 1 -- 172.21.15.164:0/1737511583 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa73c018aa0 con 0x562cc4ddbb60 2024-09-19T08:54:44.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.665+0000 7fa72b7fe640 1 -- 172.21.15.164:0/1737511583 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c01d040 con 0x562cc4ddbb60 2024-09-19T08:54:44.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.666+0000 7fa749f16740 1 -- 172.21.15.164:0/1737511583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cc4ddbb60 msgr2=0x562cc4dde610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:44.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.666+0000 7fa749f16740 1 --2- 172.21.15.164:0/1737511583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cc4ddbb60 0x562cc4dde610 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa73c010730 tx=0x7fa73c0106c0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.666+0000 7fa749f16740 1 -- 172.21.15.164:0/1737511583 shutdown_connections 2024-09-19T08:54:44.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.666+0000 7fa749f16740 1 --2- 172.21.15.164:0/1737511583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cc4de6d40 0x562cc4dfe580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.666+0000 7fa749f16740 1 --2- 172.21.15.164:0/1737511583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cc4ddc480 0x562cc4de5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.666+0000 7fa749f16740 1 --2- 172.21.15.164:0/1737511583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cc4ddbb60 0x562cc4dde610 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.666+0000 7fa749f16740 1 -- 172.21.15.164:0/1737511583 >> 172.21.15.164:0/1737511583 conn(0x562cc4dd80f0 msgr2=0x562cc4de5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:44.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.666+0000 7fa749f16740 1 -- 172.21.15.164:0/1737511583 shutdown_connections 2024-09-19T08:54:44.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.666+0000 7fa749f16740 1 -- 172.21.15.164:0/1737511583 wait complete. 2024-09-19T08:54:44.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.666+0000 7fa749f16740 1 -- start start 2024-09-19T08:54:44.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.666+0000 7fa749f16740 1 Processor -- start 2024-09-19T08:54:44.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.667+0000 7fa749f16740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cc4ddbb60 0x562cc4def520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:44.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.667+0000 7fa749f16740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cc4ddc480 0x562cc4dea8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:44.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.667+0000 7fa749f16740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cc4de6d40 0x562cc4deae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:44.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.667+0000 7fa749f16740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa7401aeb10 con 0x562cc4ddc480 2024-09-19T08:54:44.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.667+0000 7fa749f16740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa7401ae440 con 0x562cc4ddbb60 2024-09-19T08:54:44.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.667+0000 7fa749f16740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa7401ab6a0 con 0x562cc4de6d40 2024-09-19T08:54:44.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.667+0000 7fa749f16740 10 client.0 _ll_register_callbacks cb 0x7fa738031690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:54:44.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.667+0000 7fa7450d8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cc4ddc480 0x562cc4dea8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:44.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.667+0000 7fa7450d8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cc4ddc480 0x562cc4dea8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44952/0 (socket says 172.21.15.164:44952) 2024-09-19T08:54:44.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.667+0000 7fa7450d8640 1 -- 172.21.15.164:0/431815944 learned_addr learned my addr 172.21.15.164:0/431815944 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:44.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.667+0000 7fa7458d9640 1 --2- 172.21.15.164:0/431815944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cc4ddbb60 0x562cc4def520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:44.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.667+0000 7fa7460da640 1 --2- 172.21.15.164:0/431815944 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cc4de6d40 0x562cc4deae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:44.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.667+0000 7fa7450d8640 1 -- 172.21.15.164:0/431815944 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cc4de6d40 msgr2=0x562cc4deae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:44.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.667+0000 7fa7450d8640 1 --2- 172.21.15.164:0/431815944 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cc4de6d40 0x562cc4deae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.667+0000 7fa7450d8640 1 -- 172.21.15.164:0/431815944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cc4ddbb60 msgr2=0x562cc4def520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:44.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.667+0000 7fa7450d8640 1 --2- 172.21.15.164:0/431815944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cc4ddbb60 0x562cc4def520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.667+0000 7fa7450d8640 1 -- 172.21.15.164:0/431815944 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cc4deb690 con 0x562cc4ddc480 2024-09-19T08:54:44.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.667+0000 7fa7450d8640 1 --2- 172.21.15.164:0/431815944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cc4ddc480 0x562cc4dea8a0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fa73009c030 tx=0x7fa73006ad90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4fed1f7f756f2a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:44.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.667+0000 7fa7297fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:54:44.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.667+0000 7fa7297fa640 1 -- 172.21.15.164:0/431815944 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73009c0a0 con 0x562cc4ddc480 2024-09-19T08:54:44.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.668+0000 7fa7297fa640 1 -- 172.21.15.164:0/431815944 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7fa73000f740 con 0x562cc4ddc480 2024-09-19T08:54:44.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.668+0000 7fa749f16740 1 -- 172.21.15.164:0/431815944 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa73c018aa0 con 0x562cc4ddc480 2024-09-19T08:54:44.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.668+0000 7fa7297fa640 1 -- 172.21.15.164:0/431815944 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa7300a0450 con 0x562cc4ddc480 2024-09-19T08:54:44.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.668+0000 7fa749f16740 10 client.17403 fetch_fsmap learned FSMap version 32 2024-09-19T08:54:44.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.668+0000 7fa749f16740 1 -- 172.21.15.164:0/431815944 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562cc4e616d0 con 0x562cc4ddc480 2024-09-19T08:54:44.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.668+0000 7fa7297fa640 1 -- 172.21.15.164:0/431815944 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa73c018aa0 con 0x562cc4ddc480 2024-09-19T08:54:44.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.668+0000 7fa7297fa640 1 -- 172.21.15.164:0/431815944 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fa7300bace0 con 0x562cc4ddc480 2024-09-19T08:54:44.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.668+0000 7fa749f16740 10 client.17403 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:54:44.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.668+0000 7fa749f16740 10 client.17403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:54:44.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.668+0000 7fa749f16740 20 client.17403 populate_metadata read hostname 'smithi164' 2024-09-19T08:54:44.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.668+0000 7fa749f16740 1 --2- 172.21.15.164:0/431815944 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x562cc4e61e50 0x562cc4e64270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:44.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.668+0000 7fa749f16740 4 client.17403 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T08:54:44.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.668+0000 7fa749f16740 1 -- 172.21.15.164:0/431815944 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562cc4d48d00 con 0x562cc4e61e50 2024-09-19T08:54:44.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.668+0000 7fa7458d9640 1 --2- 172.21.15.164:0/431815944 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x562cc4e61e50 0x562cc4e64270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:44.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.669+0000 7fa7297fa640 10 client.17403 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:54:44.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.669+0000 7fa7458d9640 1 --2- 172.21.15.164:0/431815944 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x562cc4e61e50 0x562cc4e64270 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eecae2f4d8e72ab server_cookie=9c9db2e9f825e1d5 in_seq=0 out_seq=0 2024-09-19T08:54:44.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.670+0000 7fa7297fa640 1 -- 172.21.15.164:0/431815944 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562cc4d48d00 con 0x562cc4e61e50 2024-09-19T08:54:44.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.670+0000 7fa7297fa640 10 client.17403 handle_command_reply: tid=0 2024-09-19T08:54:44.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.670+0000 7fa749f16740 1 client.17403 shutdown 2024-09-19T08:54:44.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.670+0000 7fa749f16740 1 -- 172.21.15.164:0/431815944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cc4ddc480 msgr2=0x562cc4dea8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:44.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.670+0000 7fa749f16740 1 --2- 172.21.15.164:0/431815944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cc4ddc480 0x562cc4dea8a0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fa73009c030 tx=0x7fa73006ad90 comp rx=0 tx=0).stop 2024-09-19T08:54:44.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.670+0000 7fa749f16740 1 -- 172.21.15.164:0/431815944 shutdown_connections 2024-09-19T08:54:44.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.670+0000 7fa749f16740 1 --2- 172.21.15.164:0/431815944 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cc4de6d40 0x562cc4deae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.670+0000 7fa749f16740 1 --2- 172.21.15.164:0/431815944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cc4ddc480 0x562cc4dea8a0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.670+0000 7fa749f16740 1 --2- 172.21.15.164:0/431815944 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x562cc4e61e50 0x562cc4e64270 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.670+0000 7fa749f16740 1 --2- 172.21.15.164:0/431815944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cc4ddbb60 0x562cc4def520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.670+0000 7fa7297fa640 0 client.17403 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:54:44.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.670+0000 7fa749f16740 1 -- 172.21.15.164:0/431815944 >> 172.21.15.164:0/431815944 conn(0x562cc4dd80f0 msgr2=0x562cc4de43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:44.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.670+0000 7fa749f16740 1 -- 172.21.15.164:0/431815944 shutdown_connections 2024-09-19T08:54:44.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.670+0000 7fa749f16740 1 -- 172.21.15.164:0/431815944 wait complete. 2024-09-19T08:54:44.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.670+0000 7fa749f16740 20 client.17403 trim_cache size 0 max 16384 2024-09-19T08:54:44.676 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:54:44.676 INFO:teuthology.orchestra.run.smithi164.stdout: "return_code": 0, 2024-09-19T08:54:44.676 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_tag": "bb9378c3-ce84-4284-b336-7c22d63d9748", 2024-09-19T08:54:44.676 INFO:teuthology.orchestra.run.smithi164.stdout: "mode": "asynchronous" 2024-09-19T08:54:44.676 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:54:44.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.671+0000 7fa747b3d640 1 -- 172.21.15.164:0/305410094 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa72c0a33b0 msgr2=0x7fa72c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:44.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.671+0000 7fa747b3d640 1 --2- 172.21.15.164:0/305410094 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa72c0a33b0 0x7fa72c0a5870 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa738006010 tx=0x7fa738009970 comp rx=0 tx=0).stop 2024-09-19T08:54:44.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.671+0000 7fa747b3d640 1 -- 172.21.15.164:0/305410094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa740101420 msgr2=0x7fa740199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:44.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.671+0000 7fa747b3d640 1 --2- 172.21.15.164:0/305410094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa740101420 0x7fa740199680 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fa73000c920 tx=0x7fa73000cd20 comp rx=0 tx=0).stop 2024-09-19T08:54:44.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.671+0000 7fa747b3d640 1 -- 172.21.15.164:0/305410094 shutdown_connections 2024-09-19T08:54:44.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.671+0000 7fa747b3d640 1 --2- 172.21.15.164:0/305410094 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa740106a80 0x7fa7401a41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.671+0000 7fa747b3d640 1 --2- 172.21.15.164:0/305410094 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa72c0a33b0 0x7fa72c0a5870 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.671+0000 7fa747b3d640 1 --2- 172.21.15.164:0/305410094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa740101420 0x7fa740199680 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.671+0000 7fa747b3d640 1 --2- 172.21.15.164:0/305410094 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa740100a80 0x7fa740199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:44.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.671+0000 7fa747b3d640 1 -- 172.21.15.164:0/305410094 >> 172.21.15.164:0/305410094 conn(0x7fa7400fc7a0 msgr2=0x7fa7400fcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:44.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.671+0000 7fa747b3d640 1 -- 172.21.15.164:0/305410094 shutdown_connections 2024-09-19T08:54:44.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:44.671+0000 7fa747b3d640 1 -- 172.21.15.164:0/305410094 wait complete. 2024-09-19T08:54:45.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:45 smithi164 ceph-mon[30928]: scrub summary: active paths [/] 2024-09-19T08:54:45.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:45 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:54:45.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:45 smithi164 ceph-mon[30928]: scrub summary: active paths [/] 2024-09-19T08:54:45.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:45 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:54:45.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:45 smithi170 ceph-mon[35072]: scrub summary: active paths [/] 2024-09-19T08:54:45.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:45 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:54:45.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:45 smithi170 ceph-mon[35072]: scrub summary: active paths [/] 2024-09-19T08:54:45.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:45 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:54:45.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:45 smithi173 ceph-mon[35306]: scrub summary: active paths [/] 2024-09-19T08:54:45.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:45 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:54:45.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:45 smithi173 ceph-mon[35306]: scrub summary: active paths [/] 2024-09-19T08:54:45.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:45 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:54:45.504 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:54:45.751 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:54:46.153 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:46 smithi164 ceph-mon[30928]: pgmap v277: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T08:54:46.153 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:46 smithi164 ceph-mon[30928]: scrub queued for path: / 2024-09-19T08:54:46.153 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:46 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:54:46.154 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:46 smithi164 ceph-mon[30928]: scrub summary: active paths [/] 2024-09-19T08:54:46.154 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:46 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:54:46.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.235+0000 7f6b2509c640 1 -- 172.21.15.164:0/1138513262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b201027a0 msgr2=0x7f6b201067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:46.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.235+0000 7f6b2509c640 1 --2- 172.21.15.164:0/1138513262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b201027a0 0x7f6b201067f0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f6b08009900 tx=0x7f6b0802a410 comp rx=0 tx=0).stop 2024-09-19T08:54:46.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.235+0000 7f6b2509c640 1 -- 172.21.15.164:0/1138513262 shutdown_connections 2024-09-19T08:54:46.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.235+0000 7f6b2509c640 1 --2- 172.21.15.164:0/1138513262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b20106f20 0x7f6b2010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:46.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.235+0000 7f6b2509c640 1 --2- 172.21.15.164:0/1138513262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b201027a0 0x7f6b201067f0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:46.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.235+0000 7f6b2509c640 1 --2- 172.21.15.164:0/1138513262 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b20101df0 0x7f6b201021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:46.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.235+0000 7f6b2509c640 1 -- 172.21.15.164:0/1138513262 >> 172.21.15.164:0/1138513262 conn(0x7f6b200fd8b0 msgr2=0x7f6b200ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:46.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.235+0000 7f6b2509c640 1 -- 172.21.15.164:0/1138513262 shutdown_connections 2024-09-19T08:54:46.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.235+0000 7f6b2509c640 1 -- 172.21.15.164:0/1138513262 wait complete. 2024-09-19T08:54:46.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.236+0000 7f6b2509c640 1 Processor -- start 2024-09-19T08:54:46.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.236+0000 7f6b2509c640 1 -- start start 2024-09-19T08:54:46.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.236+0000 7f6b2509c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b20101df0 0x7f6b20198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:46.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.236+0000 7f6b2509c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b201027a0 0x7f6b201988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:46.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.236+0000 7f6b2509c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b20106f20 0x7f6b2019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:46.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.236+0000 7f6b2509c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6b20111f80 con 0x7f6b201027a0 2024-09-19T08:54:46.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.236+0000 7f6b2509c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6b20111e00 con 0x7f6b20101df0 2024-09-19T08:54:46.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.236+0000 7f6b2509c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6b20112100 con 0x7f6b20106f20 2024-09-19T08:54:46.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.236+0000 7f6b17fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b201027a0 0x7f6b201988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:46.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.236+0000 7f6b17fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b201027a0 0x7f6b201988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44956/0 (socket says 172.21.15.164:44956) 2024-09-19T08:54:46.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.236+0000 7f6b17fff640 1 -- 172.21.15.164:0/1290154626 learned_addr learned my addr 172.21.15.164:0/1290154626 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:46.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.237+0000 7f6b1ed9d640 1 --2- 172.21.15.164:0/1290154626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b20101df0 0x7f6b20198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:46.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.237+0000 7f6b1f59e640 1 --2- 172.21.15.164:0/1290154626 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b20106f20 0x7f6b2019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:46.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.237+0000 7f6b17fff640 1 -- 172.21.15.164:0/1290154626 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b20106f20 msgr2=0x7f6b2019cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:46.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.237+0000 7f6b17fff640 1 --2- 172.21.15.164:0/1290154626 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b20106f20 0x7f6b2019cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:46.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.237+0000 7f6b17fff640 1 -- 172.21.15.164:0/1290154626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b20101df0 msgr2=0x7f6b20198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:46.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.237+0000 7f6b17fff640 1 --2- 172.21.15.164:0/1290154626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b20101df0 0x7f6b20198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:46.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.237+0000 7f6b17fff640 1 -- 172.21.15.164:0/1290154626 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b2019d310 con 0x7f6b201027a0 2024-09-19T08:54:46.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.237+0000 7f6b17fff640 1 --2- 172.21.15.164:0/1290154626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b201027a0 0x7f6b201988a0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f6b08005ab0 tx=0x7f6b080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:46.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.237+0000 7f6b1cd99640 1 -- 172.21.15.164:0/1290154626 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0803f030 con 0x7f6b201027a0 2024-09-19T08:54:46.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.237+0000 7f6b2509c640 1 -- 172.21.15.164:0/1290154626 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b2019d5a0 con 0x7f6b201027a0 2024-09-19T08:54:46.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.237+0000 7f6b1cd99640 1 -- 172.21.15.164:0/1290154626 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b08003ce0 con 0x7f6b201027a0 2024-09-19T08:54:46.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.237+0000 7f6b1cd99640 1 -- 172.21.15.164:0/1290154626 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b08035070 con 0x7f6b201027a0 2024-09-19T08:54:46.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.237+0000 7f6b2509c640 1 -- 172.21.15.164:0/1290154626 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b201a4ee0 con 0x7f6b201027a0 2024-09-19T08:54:46.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.239+0000 7f6b1cd99640 1 -- 172.21.15.164:0/1290154626 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f6b08042030 con 0x7f6b201027a0 2024-09-19T08:54:46.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.240+0000 7f6b27475740 1 Processor -- start 2024-09-19T08:54:46.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.240+0000 7f6b27475740 1 -- start start 2024-09-19T08:54:46.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.240+0000 7f6b27475740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd68aa4a0 0x555cd68aa880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:46.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.240+0000 7f6b27475740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555cd68aadc0 0x555cd68b5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:46.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.240+0000 7f6b27475740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cd68b5680 0x555cd68b7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:46.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.241+0000 7f6b27475740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555cd68102e0 con 0x555cd68aadc0 2024-09-19T08:54:46.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.241+0000 7f6b27475740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555cd67b7ce0 con 0x555cd68b5680 2024-09-19T08:54:46.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.241+0000 7f6b27475740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555cd680c2a0 con 0x555cd68aa4a0 2024-09-19T08:54:46.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.241+0000 7f6b1cd99640 1 --2- 172.21.15.164:0/1290154626 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f6b0c0a33b0 0x7f6b0c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:46.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.241+0000 7f6b1ed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555cd68aadc0 0x555cd68b5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:46.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.241+0000 7f6b1ed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555cd68aadc0 0x555cd68b5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44958/0 (socket says 172.21.15.164:44958) 2024-09-19T08:54:46.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.241+0000 7f6b1f59e640 1 --2- 172.21.15.164:0/1290154626 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f6b0c0a33b0 0x7f6b0c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:46.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.241+0000 7f6b1ed9d640 1 -- 172.21.15.164:0/3192414888 learned_addr learned my addr 172.21.15.164:0/3192414888 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:46.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.241+0000 7f6b1cd99640 1 -- 172.21.15.164:0/1290154626 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7f6b080f06e0 con 0x7f6b201027a0 2024-09-19T08:54:46.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.241+0000 7f6b17fff640 1 --2- 172.21.15.164:0/3192414888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cd68b5680 0x555cd68b7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:46.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.241+0000 7f6b1f59e640 1 --2- 172.21.15.164:0/3192414888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd68aa4a0 0x555cd68aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:46.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.241+0000 7f6b1ed9d640 1 -- 172.21.15.164:0/3192414888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd68aa4a0 msgr2=0x555cd68aa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:46.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.241+0000 7f6b1ed9d640 1 --2- 172.21.15.164:0/3192414888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd68aa4a0 0x555cd68aa880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:46.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.241+0000 7f6b1ed9d640 1 -- 172.21.15.164:0/3192414888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cd68b5680 msgr2=0x555cd68b7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:46.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.241+0000 7f6b1ed9d640 1 --2- 172.21.15.164:0/3192414888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cd68b5680 0x555cd68b7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:46.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.241+0000 7f6b1ed9d640 1 -- 172.21.15.164:0/3192414888 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cd679c440 con 0x555cd68aadc0 2024-09-19T08:54:46.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.241+0000 7f6b17fff640 1 --2- 172.21.15.164:0/3192414888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cd68b5680 0x555cd68b7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:54:46.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.241+0000 7f6b1f59e640 1 --2- 172.21.15.164:0/1290154626 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f6b0c0a33b0 0x7f6b0c0a5870 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800ae20 tx=0x7f6b180117b0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:46.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.241+0000 7f6b1ed9d640 1 --2- 172.21.15.164:0/3192414888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555cd68aadc0 0x555cd68b5140 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f6b1000cbf0 tx=0x7f6b1000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98586fdb4ad00fc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:46.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.241+0000 7f6b157fa640 1 -- 172.21.15.164:0/3192414888 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b10010630 con 0x555cd68aadc0 2024-09-19T08:54:46.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.241+0000 7f6b157fa640 1 -- 172.21.15.164:0/3192414888 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b10010cb0 con 0x555cd68aadc0 2024-09-19T08:54:46.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.241+0000 7f6b157fa640 1 -- 172.21.15.164:0/3192414888 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b10019ba0 con 0x555cd68aadc0 2024-09-19T08:54:46.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.242+0000 7f6b27475740 1 -- 172.21.15.164:0/3192414888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555cd68aadc0 msgr2=0x555cd68b5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:46.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.242+0000 7f6b27475740 1 --2- 172.21.15.164:0/3192414888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555cd68aadc0 0x555cd68b5140 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f6b1000cbf0 tx=0x7f6b1000e720 comp rx=0 tx=0).stop 2024-09-19T08:54:46.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.242+0000 7f6b27475740 1 -- 172.21.15.164:0/3192414888 shutdown_connections 2024-09-19T08:54:46.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.242+0000 7f6b27475740 1 --2- 172.21.15.164:0/3192414888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cd68b5680 0x555cd68b7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:46.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.242+0000 7f6b27475740 1 --2- 172.21.15.164:0/3192414888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555cd68aadc0 0x555cd68b5140 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:46.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.242+0000 7f6b27475740 1 --2- 172.21.15.164:0/3192414888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd68aa4a0 0x555cd68aa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:46.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.242+0000 7f6b27475740 1 -- 172.21.15.164:0/3192414888 >> 172.21.15.164:0/3192414888 conn(0x555cd68a6a30 msgr2=0x555cd68a6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:46.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.242+0000 7f6b27475740 1 -- 172.21.15.164:0/3192414888 shutdown_connections 2024-09-19T08:54:46.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.242+0000 7f6b27475740 1 -- 172.21.15.164:0/3192414888 wait complete. 2024-09-19T08:54:46.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.243+0000 7f6b27475740 1 -- start start 2024-09-19T08:54:46.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.243+0000 7f6b27475740 1 Processor -- start 2024-09-19T08:54:46.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.243+0000 7f6b27475740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555cd68aa4a0 0x555cd68b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:46.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.243+0000 7f6b27475740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd68aadc0 0x555cd68b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:46.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.243+0000 7f6b27475740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cd68b5680 0x555cd68b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:46.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.243+0000 7f6b27475740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555cd68b8250 con 0x555cd68aa4a0 2024-09-19T08:54:46.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.243+0000 7f6b27475740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555cd67cb050 con 0x555cd68b5680 2024-09-19T08:54:46.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.243+0000 7f6b27475740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555cd68b83d0 con 0x555cd68aadc0 2024-09-19T08:54:46.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.243+0000 7f6b27475740 10 client.0 _ll_register_callbacks cb 0x555cd680b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:54:46.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.243+0000 7f6b1ed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555cd68aa4a0 0x555cd68b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:46.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.243+0000 7f6b1ed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555cd68aa4a0 0x555cd68b4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44968/0 (socket says 172.21.15.164:44968) 2024-09-19T08:54:46.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.243+0000 7f6b1ed9d640 1 -- 172.21.15.164:0/3732806885 learned_addr learned my addr 172.21.15.164:0/3732806885 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:46.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.243+0000 7f6b17fff640 1 --2- 172.21.15.164:0/3732806885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd68aadc0 0x555cd68b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:46.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.243+0000 7f6b1f59e640 1 --2- 172.21.15.164:0/3732806885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cd68b5680 0x555cd68b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:46.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.243+0000 7f6b1ed9d640 1 -- 172.21.15.164:0/3732806885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd68aadc0 msgr2=0x555cd68b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:46.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.243+0000 7f6b1ed9d640 1 --2- 172.21.15.164:0/3732806885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd68aadc0 0x555cd68b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:46.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.243+0000 7f6b1ed9d640 1 -- 172.21.15.164:0/3732806885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cd68b5680 msgr2=0x555cd68b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:46.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.243+0000 7f6b1ed9d640 1 --2- 172.21.15.164:0/3732806885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cd68b5680 0x555cd68b26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:46.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.243+0000 7f6b1ed9d640 1 -- 172.21.15.164:0/3732806885 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cd65813c0 con 0x555cd68aa4a0 2024-09-19T08:54:46.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.243+0000 7f6b17fff640 1 --2- 172.21.15.164:0/3732806885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd68aadc0 0x555cd68b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:54:46.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.243+0000 7f6b1ed9d640 1 --2- 172.21.15.164:0/3732806885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555cd68aa4a0 0x555cd68b4f00 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f6b10005d20 tx=0x7f6b10019820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2794db881d5eb936 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:46.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.243+0000 7f6b067fc640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:54:46.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.244+0000 7f6b067fc640 1 -- 172.21.15.164:0/3732806885 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1002a060 con 0x555cd68aa4a0 2024-09-19T08:54:46.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.244+0000 7f6b067fc640 1 -- 172.21.15.164:0/3732806885 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7f6b10005040 con 0x555cd68aa4a0 2024-09-19T08:54:46.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.244+0000 7f6b27475740 1 -- 172.21.15.164:0/3732806885 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b10010cb0 con 0x555cd68aa4a0 2024-09-19T08:54:46.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.244+0000 7f6b067fc640 1 -- 172.21.15.164:0/3732806885 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6b10005240 con 0x555cd68aa4a0 2024-09-19T08:54:46.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.244+0000 7f6b27475740 10 client.17415 fetch_fsmap learned FSMap version 32 2024-09-19T08:54:46.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.244+0000 7f6b27475740 1 -- 172.21.15.164:0/3732806885 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555cd68c3ad0 con 0x555cd68aa4a0 2024-09-19T08:54:46.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.244+0000 7f6b067fc640 1 -- 172.21.15.164:0/3732806885 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b10010cb0 con 0x555cd68aa4a0 2024-09-19T08:54:46.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.244+0000 7f6b067fc640 1 -- 172.21.15.164:0/3732806885 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f6b100046c0 con 0x555cd68aa4a0 2024-09-19T08:54:46.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.244+0000 7f6b27475740 10 client.17415 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:54:46.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.244+0000 7f6b27475740 10 client.17415 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:54:46.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.244+0000 7f6b27475740 20 client.17415 populate_metadata read hostname 'smithi164' 2024-09-19T08:54:46.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.244+0000 7f6b27475740 1 --2- 172.21.15.164:0/3732806885 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555cd68c4ee0 0x555cd68c72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:46.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.244+0000 7f6b27475740 4 client.17415 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:54:46.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.244+0000 7f6b27475740 1 -- 172.21.15.164:0/3732806885 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cd681d850 con 0x555cd68c4ee0 2024-09-19T08:54:46.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.245+0000 7f6b17fff640 1 --2- 172.21.15.164:0/3732806885 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555cd68c4ee0 0x555cd68c72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:46.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.245+0000 7f6b067fc640 10 client.17415 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:54:46.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.245+0000 7f6b17fff640 1 --2- 172.21.15.164:0/3732806885 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555cd68c4ee0 0x555cd68c72d0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62311b79034288f1 server_cookie=e401c070cdfeee08 in_seq=0 out_seq=0 2024-09-19T08:54:46.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.246+0000 7f6b067fc640 1 -- 172.21.15.164:0/3732806885 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555cd681d850 con 0x555cd68c4ee0 2024-09-19T08:54:46.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.246+0000 7f6b067fc640 10 client.17415 handle_command_reply: tid=0 2024-09-19T08:54:46.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.246+0000 7f6b27475740 1 client.17415 shutdown 2024-09-19T08:54:46.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.246+0000 7f6b27475740 1 -- 172.21.15.164:0/3732806885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555cd68aa4a0 msgr2=0x555cd68b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:46.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.246+0000 7f6b27475740 1 --2- 172.21.15.164:0/3732806885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555cd68aa4a0 0x555cd68b4f00 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f6b10005d20 tx=0x7f6b10019820 comp rx=0 tx=0).stop 2024-09-19T08:54:46.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.246+0000 7f6b27475740 1 -- 172.21.15.164:0/3732806885 shutdown_connections 2024-09-19T08:54:46.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.246+0000 7f6b27475740 1 --2- 172.21.15.164:0/3732806885 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555cd68c4ee0 0x555cd68c72d0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:46.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.246+0000 7f6b27475740 1 --2- 172.21.15.164:0/3732806885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cd68b5680 0x555cd68b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:46.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.246+0000 7f6b27475740 1 --2- 172.21.15.164:0/3732806885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd68aadc0 0x555cd68b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:46.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.246+0000 7f6b067fc640 0 client.17415 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:54:46.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.246+0000 7f6b27475740 1 --2- 172.21.15.164:0/3732806885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555cd68aa4a0 0x555cd68b4f00 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:46.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.246+0000 7f6b27475740 1 -- 172.21.15.164:0/3732806885 >> 172.21.15.164:0/3732806885 conn(0x555cd68a6a30 msgr2=0x555cd68ac710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:46.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.246+0000 7f6b27475740 1 -- 172.21.15.164:0/3732806885 shutdown_connections 2024-09-19T08:54:46.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.246+0000 7f6b27475740 1 -- 172.21.15.164:0/3732806885 wait complete. 2024-09-19T08:54:46.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.246+0000 7f6b27475740 20 client.17415 trim_cache size 0 max 16384 2024-09-19T08:54:46.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.263+0000 7f6b27475740 1 Processor -- start 2024-09-19T08:54:46.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.263+0000 7f6b27475740 1 -- start start 2024-09-19T08:54:46.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.263+0000 7f6b27475740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555cd68aa4a0 0x555cd68c14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:46.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.263+0000 7f6b27475740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cd68aadc0 0x555cd68cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:46.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.263+0000 7f6b27475740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd68b5680 0x555cd68cce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:46.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.263+0000 7f6b27475740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6b201ad6a0 con 0x555cd68aa4a0 2024-09-19T08:54:46.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.263+0000 7f6b27475740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6b201af1e0 con 0x555cd68aadc0 2024-09-19T08:54:46.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.263+0000 7f6b27475740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6b201add70 con 0x555cd68b5680 2024-09-19T08:54:46.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.263+0000 7f6b1ed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555cd68aa4a0 0x555cd68c14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:46.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.263+0000 7f6b1ed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555cd68aa4a0 0x555cd68c14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44976/0 (socket says 172.21.15.164:44976) 2024-09-19T08:54:46.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.263+0000 7f6b1ed9d640 1 -- 172.21.15.164:0/3791269375 learned_addr learned my addr 172.21.15.164:0/3791269375 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:46.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.263+0000 7f6b1ed9d640 1 -- 172.21.15.164:0/3791269375 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd68b5680 msgr2=0x555cd68cce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:46.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.263+0000 7f6b1ed9d640 1 --2- 172.21.15.164:0/3791269375 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd68b5680 0x555cd68cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:46.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.263+0000 7f6b1ed9d640 1 -- 172.21.15.164:0/3791269375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cd68aadc0 msgr2=0x555cd68cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:46.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.263+0000 7f6b17fff640 1 --2- 172.21.15.164:0/3791269375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cd68aadc0 0x555cd68cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:46.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.263+0000 7f6b1ed9d640 1 --2- 172.21.15.164:0/3791269375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cd68aadc0 0x555cd68cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:46.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.264+0000 7f6b1ed9d640 1 -- 172.21.15.164:0/3791269375 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cd68b2f70 con 0x555cd68aa4a0 2024-09-19T08:54:46.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.264+0000 7f6b1ed9d640 1 --2- 172.21.15.164:0/3791269375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555cd68aa4a0 0x555cd68c14b0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f6b10005ab0 tx=0x7f6b10005f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191edbaf0d566901 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:46.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.264+0000 7f6b14ff9640 1 -- 172.21.15.164:0/3791269375 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1001e070 con 0x555cd68aa4a0 2024-09-19T08:54:46.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.264+0000 7f6b14ff9640 1 -- 172.21.15.164:0/3791269375 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b10005040 con 0x555cd68aa4a0 2024-09-19T08:54:46.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.264+0000 7f6b14ff9640 1 -- 172.21.15.164:0/3791269375 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b10004660 con 0x555cd68aa4a0 2024-09-19T08:54:46.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.264+0000 7f6b27475740 1 -- 172.21.15.164:0/3791269375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555cd68aa4a0 msgr2=0x555cd68c14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:46.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.264+0000 7f6b27475740 1 --2- 172.21.15.164:0/3791269375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555cd68aa4a0 0x555cd68c14b0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f6b10005ab0 tx=0x7f6b10005f20 comp rx=0 tx=0).stop 2024-09-19T08:54:46.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.265+0000 7f6b27475740 1 -- 172.21.15.164:0/3791269375 shutdown_connections 2024-09-19T08:54:46.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.265+0000 7f6b27475740 1 --2- 172.21.15.164:0/3791269375 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd68b5680 0x555cd68cce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:46.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.265+0000 7f6b27475740 1 --2- 172.21.15.164:0/3791269375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cd68aadc0 0x555cd68cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:46.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.265+0000 7f6b27475740 1 --2- 172.21.15.164:0/3791269375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555cd68aa4a0 0x555cd68c14b0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:46.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.265+0000 7f6b27475740 1 -- 172.21.15.164:0/3791269375 >> 172.21.15.164:0/3791269375 conn(0x555cd68a6a30 msgr2=0x555cd68b12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:46.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.265+0000 7f6b27475740 1 -- 172.21.15.164:0/3791269375 shutdown_connections 2024-09-19T08:54:46.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.265+0000 7f6b27475740 1 -- 172.21.15.164:0/3791269375 wait complete. 2024-09-19T08:54:46.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.265+0000 7f6b27475740 1 -- start start 2024-09-19T08:54:46.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.265+0000 7f6b27475740 1 Processor -- start 2024-09-19T08:54:46.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.265+0000 7f6b27475740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd68aa4a0 0x555cd6927cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:46.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.265+0000 7f6b27475740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555cd68aadc0 0x555cd6928200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:46.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.265+0000 7f6b27475740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cd68b5680 0x555cd68bac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:46.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.265+0000 7f6b27475740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6b201aeb10 con 0x555cd68aadc0 2024-09-19T08:54:46.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.265+0000 7f6b27475740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6b201ae440 con 0x555cd68b5680 2024-09-19T08:54:46.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.265+0000 7f6b27475740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6b201ab6a0 con 0x555cd68aa4a0 2024-09-19T08:54:46.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.265+0000 7f6b27475740 10 client.0 _ll_register_callbacks cb 0x7f6b10014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:54:46.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.265+0000 7f6b17fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555cd68aadc0 0x555cd6928200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:46.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.265+0000 7f6b17fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555cd68aadc0 0x555cd6928200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44986/0 (socket says 172.21.15.164:44986) 2024-09-19T08:54:46.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.265+0000 7f6b17fff640 1 -- 172.21.15.164:0/1294107927 learned_addr learned my addr 172.21.15.164:0/1294107927 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:54:46.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.266+0000 7f6b1ed9d640 1 --2- 172.21.15.164:0/1294107927 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd68aa4a0 0x555cd6927cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:46.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.266+0000 7f6b17fff640 1 -- 172.21.15.164:0/1294107927 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd68aa4a0 msgr2=0x555cd6927cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:46.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.266+0000 7f6b17fff640 1 --2- 172.21.15.164:0/1294107927 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd68aa4a0 0x555cd6927cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:46.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.266+0000 7f6b1f59e640 1 --2- 172.21.15.164:0/1294107927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cd68b5680 0x555cd68bac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:46.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.266+0000 7f6b17fff640 1 -- 172.21.15.164:0/1294107927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cd68b5680 msgr2=0x555cd68bac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:54:46.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.266+0000 7f6b17fff640 1 --2- 172.21.15.164:0/1294107927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cd68b5680 0x555cd68bac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:46.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.266+0000 7f6b17fff640 1 -- 172.21.15.164:0/1294107927 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cd68bb410 con 0x555cd68aadc0 2024-09-19T08:54:46.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.266+0000 7f6b17fff640 1 --2- 172.21.15.164:0/1294107927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555cd68aadc0 0x555cd6928200 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f6b080b9370 tx=0x7f6b0803b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd2ce042a5c9b77 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:54:46.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.266+0000 7f6b067fc640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:54:46.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.266+0000 7f6b067fc640 1 -- 172.21.15.164:0/1294107927 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b08049040 con 0x555cd68aadc0 2024-09-19T08:54:46.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.266+0000 7f6b067fc640 1 -- 172.21.15.164:0/1294107927 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8455+0+0 (secure 0 0 0) 0x7f6b0803bd70 con 0x555cd68aadc0 2024-09-19T08:54:46.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.266+0000 7f6b27475740 1 -- 172.21.15.164:0/1294107927 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b10005040 con 0x555cd68aadc0 2024-09-19T08:54:46.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.266+0000 7f6b067fc640 1 -- 172.21.15.164:0/1294107927 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6b080bc9a0 con 0x555cd68aadc0 2024-09-19T08:54:46.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.266+0000 7f6b27475740 10 client.17421 fetch_fsmap learned FSMap version 32 2024-09-19T08:54:46.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.266+0000 7f6b27475740 1 -- 172.21.15.164:0/1294107927 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555cd692f490 con 0x555cd68aadc0 2024-09-19T08:54:46.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.266+0000 7f6b067fc640 1 -- 172.21.15.164:0/1294107927 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b10005040 con 0x555cd68aadc0 2024-09-19T08:54:46.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.266+0000 7f6b067fc640 1 -- 172.21.15.164:0/1294107927 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f6b0803b460 con 0x555cd68aadc0 2024-09-19T08:54:46.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.267+0000 7f6b27475740 10 client.17421 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:54:46.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.267+0000 7f6b27475740 10 client.17421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:54:46.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.267+0000 7f6b27475740 20 client.17421 populate_metadata read hostname 'smithi164' 2024-09-19T08:54:46.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.267+0000 7f6b27475740 1 --2- 172.21.15.164:0/1294107927 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555cd692fbc0 0x555cd6931fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:54:46.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.267+0000 7f6b27475740 4 client.17421 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:54:46.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.267+0000 7f6b27475740 1 -- 172.21.15.164:0/1294107927 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555cd6817d60 con 0x555cd692fbc0 2024-09-19T08:54:46.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.267+0000 7f6b1ed9d640 1 --2- 172.21.15.164:0/1294107927 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555cd692fbc0 0x555cd6931fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:54:46.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.267+0000 7f6b067fc640 10 client.17421 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:54:46.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.267+0000 7f6b1ed9d640 1 --2- 172.21.15.164:0/1294107927 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555cd692fbc0 0x555cd6931fe0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3431d29015274676 server_cookie=f36deabe9ce92266 in_seq=0 out_seq=0 2024-09-19T08:54:46.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.267+0000 7f6b067fc640 1 -- 172.21.15.164:0/1294107927 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555cd6817d60 con 0x555cd692fbc0 2024-09-19T08:54:46.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.267+0000 7f6b067fc640 10 client.17421 handle_command_reply: tid=0 2024-09-19T08:54:46.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.267+0000 7f6b27475740 1 client.17421 shutdown 2024-09-19T08:54:46.314 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:54:46.314 INFO:teuthology.orchestra.run.smithi164.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T08:54:46.314 INFO:teuthology.orchestra.run.smithi164.stdout: "scrubs": { 2024-09-19T08:54:46.314 INFO:teuthology.orchestra.run.smithi164.stdout: "bb9378c3-ce84-4284-b336-7c22d63d9748": { 2024-09-19T08:54:46.314 INFO:teuthology.orchestra.run.smithi164.stdout: "path": "/", 2024-09-19T08:54:46.314 INFO:teuthology.orchestra.run.smithi164.stdout: "tag": "bb9378c3-ce84-4284-b336-7c22d63d9748", 2024-09-19T08:54:46.314 INFO:teuthology.orchestra.run.smithi164.stdout: "options": "recursive,force" 2024-09-19T08:54:46.314 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:54:46.314 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:54:46.315 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:54:46.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.268+0000 7f6b27475740 1 -- 172.21.15.164:0/1294107927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555cd68aadc0 msgr2=0x555cd6928200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:46.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.268+0000 7f6b27475740 1 --2- 172.21.15.164:0/1294107927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555cd68aadc0 0x555cd6928200 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f6b080b9370 tx=0x7f6b0803b800 comp rx=0 tx=0).stop 2024-09-19T08:54:46.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.268+0000 7f6b27475740 1 -- 172.21.15.164:0/1294107927 shutdown_connections 2024-09-19T08:54:46.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.268+0000 7f6b27475740 1 --2- 172.21.15.164:0/1294107927 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555cd692fbc0 0x555cd6931fe0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:46.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.268+0000 7f6b27475740 1 --2- 172.21.15.164:0/1294107927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cd68b5680 0x555cd68bac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:46.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.268+0000 7f6b27475740 1 --2- 172.21.15.164:0/1294107927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555cd68aadc0 0x555cd6928200 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:46.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.268+0000 7f6b27475740 1 --2- 172.21.15.164:0/1294107927 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd68aa4a0 0x555cd6927cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:46.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.268+0000 7f6b067fc640 0 client.17421 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:54:46.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.268+0000 7f6b27475740 1 -- 172.21.15.164:0/1294107927 >> 172.21.15.164:0/1294107927 conn(0x555cd68a6a30 msgr2=0x555cd68a5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:46.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.268+0000 7f6b27475740 1 -- 172.21.15.164:0/1294107927 shutdown_connections 2024-09-19T08:54:46.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.268+0000 7f6b27475740 1 -- 172.21.15.164:0/1294107927 wait complete. 2024-09-19T08:54:46.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.268+0000 7f6b27475740 20 client.17421 trim_cache size 0 max 16384 2024-09-19T08:54:46.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.268+0000 7f6b2509c640 1 -- 172.21.15.164:0/1290154626 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f6b0c0a33b0 msgr2=0x7f6b0c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:46.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.268+0000 7f6b2509c640 1 --2- 172.21.15.164:0/1290154626 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f6b0c0a33b0 0x7f6b0c0a5870 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800ae20 tx=0x7f6b180117b0 comp rx=0 tx=0).stop 2024-09-19T08:54:46.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.268+0000 7f6b2509c640 1 -- 172.21.15.164:0/1290154626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b201027a0 msgr2=0x7f6b201988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:54:46.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.268+0000 7f6b2509c640 1 --2- 172.21.15.164:0/1290154626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b201027a0 0x7f6b201988a0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f6b08005ab0 tx=0x7f6b080042f0 comp rx=0 tx=0).stop 2024-09-19T08:54:46.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.268+0000 7f6b2509c640 1 -- 172.21.15.164:0/1290154626 shutdown_connections 2024-09-19T08:54:46.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.268+0000 7f6b2509c640 1 --2- 172.21.15.164:0/1290154626 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b20106f20 0x7f6b2019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:46.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.268+0000 7f6b2509c640 1 --2- 172.21.15.164:0/1290154626 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f6b0c0a33b0 0x7f6b0c0a5870 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:46.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.268+0000 7f6b2509c640 1 --2- 172.21.15.164:0/1290154626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b201027a0 0x7f6b201988a0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:46.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.268+0000 7f6b2509c640 1 --2- 172.21.15.164:0/1290154626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b20101df0 0x7f6b20198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:54:46.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.268+0000 7f6b2509c640 1 -- 172.21.15.164:0/1290154626 >> 172.21.15.164:0/1290154626 conn(0x7f6b200fd8b0 msgr2=0x7f6b200ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:54:46.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.268+0000 7f6b2509c640 1 -- 172.21.15.164:0/1290154626 shutdown_connections 2024-09-19T08:54:46.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:54:46.269+0000 7f6b2509c640 1 -- 172.21.15.164:0/1290154626 wait complete. 2024-09-19T08:54:46.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:46 smithi170 ceph-mon[35072]: pgmap v277: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T08:54:46.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:46 smithi170 ceph-mon[35072]: scrub queued for path: / 2024-09-19T08:54:46.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:46 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:54:46.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:46 smithi170 ceph-mon[35072]: scrub summary: active paths [/] 2024-09-19T08:54:46.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:46 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:54:46.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:46 smithi173 ceph-mon[35306]: pgmap v277: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T08:54:46.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:46 smithi173 ceph-mon[35306]: scrub queued for path: / 2024-09-19T08:54:46.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:46 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:54:46.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:46 smithi173 ceph-mon[35306]: scrub summary: active paths [/] 2024-09-19T08:54:46.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:46 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:54:47.229 INFO:tasks.cephfs.filesystem:scrub status for tag:bb9378c3-ce84-4284-b336-7c22d63d9748 - {'path': '/', 'tag': 'bb9378c3-ce84-4284-b336-7c22d63d9748', 'options': 'recursive,force'} 2024-09-19T08:54:48.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:48 smithi164 ceph-mon[30928]: pgmap v278: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-09-19T08:54:48.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:48 smithi170 ceph-mon[35072]: pgmap v278: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-09-19T08:54:48.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:48 smithi173 ceph-mon[35306]: pgmap v278: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-09-19T08:54:50.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:50 smithi164 ceph-mon[30928]: pgmap v279: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-09-19T08:54:50.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:50 smithi170 ceph-mon[35072]: pgmap v279: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-09-19T08:54:50.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:50 smithi173 ceph-mon[35306]: pgmap v279: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-09-19T08:54:52.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:52 smithi164 ceph-mon[30928]: pgmap v280: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 14 op/s 2024-09-19T08:54:52.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:52 smithi170 ceph-mon[35072]: pgmap v280: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 14 op/s 2024-09-19T08:54:52.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:52 smithi173 ceph-mon[35306]: pgmap v280: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 14 op/s 2024-09-19T08:54:53.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:53 smithi164 ceph-mon[30928]: scrub summary: idle 2024-09-19T08:54:53.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:53 smithi164 ceph-mon[30928]: scrub summary: idle 2024-09-19T08:54:53.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:53 smithi170 ceph-mon[35072]: scrub summary: idle 2024-09-19T08:54:53.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:53 smithi170 ceph-mon[35072]: scrub summary: idle 2024-09-19T08:54:53.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:53 smithi173 ceph-mon[35306]: scrub summary: idle 2024-09-19T08:54:53.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:53 smithi173 ceph-mon[35306]: scrub summary: idle 2024-09-19T08:54:54.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:54 smithi164 ceph-mon[30928]: pgmap v281: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 11 op/s 2024-09-19T08:54:54.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:54 smithi170 ceph-mon[35072]: pgmap v281: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 11 op/s 2024-09-19T08:54:54.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:54 smithi173 ceph-mon[35306]: pgmap v281: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 11 op/s 2024-09-19T08:54:56.339 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:56 smithi164 ceph-mon[30928]: pgmap v282: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 13 op/s 2024-09-19T08:54:56.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:56 smithi170 ceph-mon[35072]: pgmap v282: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 13 op/s 2024-09-19T08:54:56.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:56 smithi173 ceph-mon[35306]: pgmap v282: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 13 op/s 2024-09-19T08:54:58.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:54:58 smithi170 ceph-mon[35072]: pgmap v283: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 12 op/s 2024-09-19T08:54:58.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:54:58 smithi173 ceph-mon[35306]: pgmap v283: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 12 op/s 2024-09-19T08:54:58.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:54:58 smithi164 ceph-mon[30928]: pgmap v283: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 12 op/s 2024-09-19T08:55:00.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:00 smithi170 ceph-mon[35072]: pgmap v284: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 10 op/s 2024-09-19T08:55:00.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:00 smithi173 ceph-mon[35306]: pgmap v284: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 10 op/s 2024-09-19T08:55:00.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:00 smithi164 ceph-mon[30928]: pgmap v284: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 10 op/s 2024-09-19T08:55:02.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:02 smithi170 ceph-mon[35072]: pgmap v285: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 14 op/s 2024-09-19T08:55:02.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:02 smithi173 ceph-mon[35306]: pgmap v285: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 14 op/s 2024-09-19T08:55:02.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:02 smithi164 ceph-mon[30928]: pgmap v285: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 14 op/s 2024-09-19T08:55:04.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:04 smithi170 ceph-mon[35072]: pgmap v286: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-19T08:55:04.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:04 smithi173 ceph-mon[35306]: pgmap v286: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-19T08:55:04.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:04 smithi164 ceph-mon[30928]: pgmap v286: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-19T08:55:06.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:06 smithi170 ceph-mon[35072]: pgmap v287: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 28 op/s 2024-09-19T08:55:06.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:06 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:55:06.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:06 smithi173 ceph-mon[35306]: pgmap v287: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 28 op/s 2024-09-19T08:55:06.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:06 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:55:06.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:06 smithi164 ceph-mon[30928]: pgmap v287: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 28 op/s 2024-09-19T08:55:06.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:06 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:55:07.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:07 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:55:07.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:07 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:55:07.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:07 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:55:07.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:07 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:55:07.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:07 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:55:07.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:07 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:55:07.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:07 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:55:07.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:07 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:55:07.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:07 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:55:08.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:08 smithi170 ceph-mon[35072]: pgmap v288: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-09-19T08:55:08.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:08 smithi170 ceph-mon[35072]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T08:55:08.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:08 smithi173 ceph-mon[35306]: pgmap v288: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-09-19T08:55:08.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:08 smithi173 ceph-mon[35306]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T08:55:08.432 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:08 smithi164 ceph-mon[30928]: pgmap v288: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-09-19T08:55:08.432 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:08 smithi164 ceph-mon[30928]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T08:55:10.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:10 smithi170 ceph-mon[35072]: pgmap v290: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T08:55:10.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:10 smithi173 ceph-mon[35306]: pgmap v290: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T08:55:10.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:10 smithi164 ceph-mon[30928]: pgmap v290: 97 pgs: 97 active+clean; 41 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T08:55:12.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:12 smithi170 ceph-mon[35072]: pgmap v291: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 34 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T08:55:12.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:12 smithi173 ceph-mon[35306]: pgmap v291: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 34 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T08:55:12.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:12 smithi164 ceph-mon[30928]: pgmap v291: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 34 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T08:55:14.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:14 smithi170 ceph-mon[35072]: pgmap v292: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 33 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-09-19T08:55:14.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:14 smithi173 ceph-mon[35306]: pgmap v292: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 33 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-09-19T08:55:14.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:14 smithi164 ceph-mon[30928]: pgmap v292: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 33 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-09-19T08:55:16.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:16 smithi170 ceph-mon[35072]: pgmap v293: 97 pgs: 97 active+clean; 31 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T08:55:16.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:16 smithi173 ceph-mon[35306]: pgmap v293: 97 pgs: 97 active+clean; 31 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T08:55:16.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:16 smithi164 ceph-mon[30928]: pgmap v293: 97 pgs: 97 active+clean; 31 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T08:55:17.230 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:55:17.489 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:55:18.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.099+0000 7f031f609640 1 -- 172.21.15.164:0/2095279869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f031810c160 msgr2=0x7f031810e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:18.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.099+0000 7f031f609640 1 --2- 172.21.15.164:0/2095279869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f031810c160 0x7f031810e600 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f0310009ca0 tx=0x7f031002a950 comp rx=0 tx=0).stop 2024-09-19T08:55:18.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.099+0000 7f031f609640 1 -- 172.21.15.164:0/2095279869 shutdown_connections 2024-09-19T08:55:18.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.099+0000 7f031f609640 1 --2- 172.21.15.164:0/2095279869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f031810c160 0x7f031810e600 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.099+0000 7f031f609640 1 --2- 172.21.15.164:0/2095279869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0318103650 0x7f031810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.099+0000 7f031f609640 1 --2- 172.21.15.164:0/2095279869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0318102d30 0x7f0318103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.099+0000 7f031f609640 1 -- 172.21.15.164:0/2095279869 >> 172.21.15.164:0/2095279869 conn(0x7f03180fc7d0 msgr2=0x7f03180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:18.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.099+0000 7f031f609640 1 -- 172.21.15.164:0/2095279869 shutdown_connections 2024-09-19T08:55:18.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.099+0000 7f031f609640 1 -- 172.21.15.164:0/2095279869 wait complete. 2024-09-19T08:55:18.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.100+0000 7f031f609640 1 Processor -- start 2024-09-19T08:55:18.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.100+0000 7f031f609640 1 -- start start 2024-09-19T08:55:18.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.100+0000 7f031f609640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0318102d30 0x7f0318100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:18.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.100+0000 7f031f609640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0318103650 0x7f0318100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:18.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.100+0000 7f031f609640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f031810c160 0x7f0318101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:18.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.100+0000 7f031f609640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0318111fa0 con 0x7f031810c160 2024-09-19T08:55:18.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.100+0000 7f031f609640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0318111e20 con 0x7f0318102d30 2024-09-19T08:55:18.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.100+0000 7f031f609640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0318112120 con 0x7f0318103650 2024-09-19T08:55:18.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.100+0000 7f031dba6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f031810c160 0x7f0318101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:18.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.100+0000 7f031cba4640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0318103650 0x7f0318100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:18.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.100+0000 7f031d3a5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0318102d30 0x7f0318100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:18.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.100+0000 7f031dba6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f031810c160 0x7f0318101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45318/0 (socket says 172.21.15.164:45318) 2024-09-19T08:55:18.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.100+0000 7f031cba4640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0318103650 0x7f0318100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:43768/0 (socket says 172.21.15.164:43768) 2024-09-19T08:55:18.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.100+0000 7f031dba6640 1 -- 172.21.15.164:0/2561633329 learned_addr learned my addr 172.21.15.164:0/2561633329 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:18.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.100+0000 7f031dba6640 1 -- 172.21.15.164:0/2561633329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0318103650 msgr2=0x7f0318100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:18.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.100+0000 7f031dba6640 1 --2- 172.21.15.164:0/2561633329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0318103650 0x7f0318100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.101+0000 7f031dba6640 1 -- 172.21.15.164:0/2561633329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0318102d30 msgr2=0x7f0318100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:18.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.101+0000 7f031dba6640 1 --2- 172.21.15.164:0/2561633329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0318102d30 0x7f0318100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.101+0000 7f031dba6640 1 -- 172.21.15.164:0/2561633329 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03181a6900 con 0x7f031810c160 2024-09-19T08:55:18.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.101+0000 7f031dba6640 1 --2- 172.21.15.164:0/2561633329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f031810c160 0x7f0318101410 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f0310006010 tx=0x7f031002f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:18.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.105+0000 7f030e7fc640 1 -- 172.21.15.164:0/2561633329 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310034040 con 0x7f031810c160 2024-09-19T08:55:18.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.105+0000 7f031f609640 1 -- 172.21.15.164:0/2561633329 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03181a6b90 con 0x7f031810c160 2024-09-19T08:55:18.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.105+0000 7f031f609640 1 -- 172.21.15.164:0/2561633329 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03181a70c0 con 0x7f031810c160 2024-09-19T08:55:18.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.105+0000 7f030e7fc640 1 -- 172.21.15.164:0/2561633329 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0310003e00 con 0x7f031810c160 2024-09-19T08:55:18.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.105+0000 7f030e7fc640 1 -- 172.21.15.164:0/2561633329 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310045480 con 0x7f031810c160 2024-09-19T08:55:18.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.107+0000 7f030e7fc640 1 -- 172.21.15.164:0/2561633329 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f0310037020 con 0x7f031810c160 2024-09-19T08:55:18.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.108+0000 7f030e7fc640 1 --2- 172.21.15.164:0/2561633329 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f03040a33b0 0x7f03040a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:18.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.108+0000 7f030e7fc640 1 -- 172.21.15.164:0/2561633329 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7f03100f0b60 con 0x7f031810c160 2024-09-19T08:55:18.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.108+0000 7f031d3a5640 1 --2- 172.21.15.164:0/2561633329 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f03040a33b0 0x7f03040a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:18.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.108+0000 7f031d3a5640 1 --2- 172.21.15.164:0/2561633329 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f03040a33b0 0x7f03040a5870 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f03140058c0 tx=0x7f031400a2d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:18.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.109+0000 7f03219e2740 1 Processor -- start 2024-09-19T08:55:18.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.109+0000 7f03219e2740 1 -- start start 2024-09-19T08:55:18.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.109+0000 7f03219e2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5633c62074a0 0x5633c6207880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:18.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.109+0000 7f03219e2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633c6207dc0 0x5633c6212140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:18.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.109+0000 7f03219e2740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633c6212680 0x5633c6214a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:18.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.109+0000 7f03219e2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5633c616d2e0 con 0x5633c6207dc0 2024-09-19T08:55:18.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.109+0000 7f03219e2740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5633c6114ce0 con 0x5633c6212680 2024-09-19T08:55:18.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.109+0000 7f03219e2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5633c61692a0 con 0x5633c62074a0 2024-09-19T08:55:18.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.109+0000 7f031cba4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633c6207dc0 0x5633c6212140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:18.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.109+0000 7f031cba4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633c6207dc0 0x5633c6212140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45330/0 (socket says 172.21.15.164:45330) 2024-09-19T08:55:18.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.109+0000 7f031cba4640 1 -- 172.21.15.164:0/2357505396 learned_addr learned my addr 172.21.15.164:0/2357505396 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:18.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.109+0000 7f031dba6640 1 --2- 172.21.15.164:0/2357505396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633c6212680 0x5633c6214a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:18.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.109+0000 7f031d3a5640 1 --2- 172.21.15.164:0/2357505396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5633c62074a0 0x5633c6207880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:18.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.110+0000 7f031cba4640 1 -- 172.21.15.164:0/2357505396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5633c62074a0 msgr2=0x5633c6207880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:18.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.110+0000 7f031cba4640 1 --2- 172.21.15.164:0/2357505396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5633c62074a0 0x5633c6207880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.110+0000 7f031cba4640 1 -- 172.21.15.164:0/2357505396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633c6212680 msgr2=0x5633c6214a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:18.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.110+0000 7f031cba4640 1 --2- 172.21.15.164:0/2357505396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633c6212680 0x5633c6214a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.110+0000 7f031cba4640 1 -- 172.21.15.164:0/2357505396 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633c60f9440 con 0x5633c6207dc0 2024-09-19T08:55:18.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.110+0000 7f031cba4640 1 --2- 172.21.15.164:0/2357505396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633c6207dc0 0x5633c6212140 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f0308009a20 tx=0x7f0308009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188baca591b8f32b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:18.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.110+0000 7f03037fe640 1 -- 172.21.15.164:0/2357505396 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0308017070 con 0x5633c6207dc0 2024-09-19T08:55:18.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.110+0000 7f03037fe640 1 -- 172.21.15.164:0/2357505396 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0308013470 con 0x5633c6207dc0 2024-09-19T08:55:18.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.110+0000 7f03037fe640 1 -- 172.21.15.164:0/2357505396 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0308012630 con 0x5633c6207dc0 2024-09-19T08:55:18.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.111+0000 7f03219e2740 1 -- 172.21.15.164:0/2357505396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633c6207dc0 msgr2=0x5633c6212140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:18.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.111+0000 7f03219e2740 1 --2- 172.21.15.164:0/2357505396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633c6207dc0 0x5633c6212140 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f0308009a20 tx=0x7f0308009ee0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.111+0000 7f03219e2740 1 -- 172.21.15.164:0/2357505396 shutdown_connections 2024-09-19T08:55:18.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.111+0000 7f03219e2740 1 --2- 172.21.15.164:0/2357505396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633c6212680 0x5633c6214a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.111+0000 7f03219e2740 1 --2- 172.21.15.164:0/2357505396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633c6207dc0 0x5633c6212140 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.111+0000 7f03219e2740 1 --2- 172.21.15.164:0/2357505396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5633c62074a0 0x5633c6207880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.111+0000 7f03219e2740 1 -- 172.21.15.164:0/2357505396 >> 172.21.15.164:0/2357505396 conn(0x5633c6203a30 msgr2=0x5633c6203e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:18.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.111+0000 7f03219e2740 1 -- 172.21.15.164:0/2357505396 shutdown_connections 2024-09-19T08:55:18.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.111+0000 7f03219e2740 1 -- 172.21.15.164:0/2357505396 wait complete. 2024-09-19T08:55:18.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.111+0000 7f03219e2740 1 -- start start 2024-09-19T08:55:18.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.111+0000 7f03219e2740 1 Processor -- start 2024-09-19T08:55:18.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.112+0000 7f03219e2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633c62074a0 0x5633c6211f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:18.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.112+0000 7f03219e2740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633c6207dc0 0x5633c620f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:18.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.112+0000 7f03219e2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5633c6212680 0x5633c620f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:18.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.112+0000 7f03219e2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5633c6215250 con 0x5633c62074a0 2024-09-19T08:55:18.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.112+0000 7f03219e2740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5633c6128050 con 0x5633c6207dc0 2024-09-19T08:55:18.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.112+0000 7f03219e2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5633c62153d0 con 0x5633c6212680 2024-09-19T08:55:18.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.112+0000 7f03219e2740 10 client.0 _ll_register_callbacks cb 0x5633c6168160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:55:18.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.112+0000 7f031d3a5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633c62074a0 0x5633c6211f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:18.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.112+0000 7f031d3a5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633c62074a0 0x5633c6211f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45340/0 (socket says 172.21.15.164:45340) 2024-09-19T08:55:18.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.112+0000 7f031d3a5640 1 -- 172.21.15.164:0/1594796278 learned_addr learned my addr 172.21.15.164:0/1594796278 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:18.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.112+0000 7f031dba6640 1 --2- 172.21.15.164:0/1594796278 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5633c6212680 0x5633c620f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:18.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.112+0000 7f031cba4640 1 --2- 172.21.15.164:0/1594796278 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633c6207dc0 0x5633c620f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:18.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.112+0000 7f031d3a5640 1 -- 172.21.15.164:0/1594796278 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5633c6212680 msgr2=0x5633c620f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:18.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.112+0000 7f031d3a5640 1 --2- 172.21.15.164:0/1594796278 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5633c6212680 0x5633c620f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.112+0000 7f031d3a5640 1 -- 172.21.15.164:0/1594796278 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633c6207dc0 msgr2=0x5633c620f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:18.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.112+0000 7f031d3a5640 1 --2- 172.21.15.164:0/1594796278 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633c6207dc0 0x5633c620f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.112+0000 7f031d3a5640 1 -- 172.21.15.164:0/1594796278 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633c5ede3c0 con 0x5633c62074a0 2024-09-19T08:55:18.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.112+0000 7f031dba6640 1 --2- 172.21.15.164:0/1594796278 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5633c6212680 0x5633c620f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:55:18.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.113+0000 7f031d3a5640 1 --2- 172.21.15.164:0/1594796278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633c62074a0 0x5633c6211f00 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f031401de30 tx=0x7f0314004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c47a8b0580b805e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:18.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.113+0000 7f0300ff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:55:18.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.113+0000 7f0300ff9640 1 -- 172.21.15.164:0/1594796278 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031402a060 con 0x5633c62074a0 2024-09-19T08:55:18.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.113+0000 7f0300ff9640 1 -- 172.21.15.164:0/1594796278 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7f031401b440 con 0x5633c62074a0 2024-09-19T08:55:18.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.113+0000 7f03219e2740 1 -- 172.21.15.164:0/1594796278 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0308013470 con 0x5633c62074a0 2024-09-19T08:55:18.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.113+0000 7f0300ff9640 1 -- 172.21.15.164:0/1594796278 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0314016070 con 0x5633c62074a0 2024-09-19T08:55:18.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.113+0000 7f03219e2740 10 client.17433 fetch_fsmap learned FSMap version 32 2024-09-19T08:55:18.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.113+0000 7f03219e2740 1 -- 172.21.15.164:0/1594796278 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5633c6220c20 con 0x5633c62074a0 2024-09-19T08:55:18.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.113+0000 7f0300ff9640 1 -- 172.21.15.164:0/1594796278 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0308013470 con 0x5633c62074a0 2024-09-19T08:55:18.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.113+0000 7f0300ff9640 1 -- 172.21.15.164:0/1594796278 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f0314024030 con 0x5633c62074a0 2024-09-19T08:55:18.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.113+0000 7f03219e2740 10 client.17433 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:55:18.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.113+0000 7f03219e2740 10 client.17433 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:55:18.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.113+0000 7f03219e2740 20 client.17433 populate_metadata read hostname 'smithi164' 2024-09-19T08:55:18.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.113+0000 7f03219e2740 1 --2- 172.21.15.164:0/1594796278 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5633c62220e0 0x5633c6224580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:18.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.113+0000 7f03219e2740 4 client.17433 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:55:18.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.113+0000 7f03219e2740 1 -- 172.21.15.164:0/1594796278 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633c617a850 con 0x5633c62220e0 2024-09-19T08:55:18.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.114+0000 7f031cba4640 1 --2- 172.21.15.164:0/1594796278 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5633c62220e0 0x5633c6224580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:18.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.114+0000 7f0300ff9640 10 client.17433 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:55:18.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.114+0000 7f031cba4640 1 --2- 172.21.15.164:0/1594796278 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5633c62220e0 0x5633c6224580 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bfed4379e426ef8 server_cookie=fbf3700ee6b9e5e9 in_seq=0 out_seq=0 2024-09-19T08:55:18.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.115+0000 7f0300ff9640 1 -- 172.21.15.164:0/1594796278 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5633c617a850 con 0x5633c62220e0 2024-09-19T08:55:18.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.115+0000 7f0300ff9640 10 client.17433 handle_command_reply: tid=0 2024-09-19T08:55:18.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.115+0000 7f03219e2740 1 client.17433 shutdown 2024-09-19T08:55:18.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.115+0000 7f03219e2740 1 -- 172.21.15.164:0/1594796278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633c62074a0 msgr2=0x5633c6211f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:18.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.115+0000 7f03219e2740 1 --2- 172.21.15.164:0/1594796278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633c62074a0 0x5633c6211f00 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f031401de30 tx=0x7f0314004730 comp rx=0 tx=0).stop 2024-09-19T08:55:18.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.115+0000 7f03219e2740 1 -- 172.21.15.164:0/1594796278 shutdown_connections 2024-09-19T08:55:18.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.115+0000 7f03219e2740 1 --2- 172.21.15.164:0/1594796278 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5633c6212680 0x5633c620f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.115+0000 7f03219e2740 1 --2- 172.21.15.164:0/1594796278 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5633c62220e0 0x5633c6224580 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.115+0000 7f03219e2740 1 --2- 172.21.15.164:0/1594796278 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633c6207dc0 0x5633c620f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.115+0000 7f03219e2740 1 --2- 172.21.15.164:0/1594796278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633c62074a0 0x5633c6211f00 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.115+0000 7f03219e2740 1 -- 172.21.15.164:0/1594796278 >> 172.21.15.164:0/1594796278 conn(0x5633c6203a30 msgr2=0x5633c6209710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:18.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.115+0000 7f0300ff9640 0 client.17433 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:55:18.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.115+0000 7f03219e2740 1 -- 172.21.15.164:0/1594796278 shutdown_connections 2024-09-19T08:55:18.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.115+0000 7f03219e2740 1 -- 172.21.15.164:0/1594796278 wait complete. 2024-09-19T08:55:18.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.115+0000 7f03219e2740 20 client.17433 trim_cache size 0 max 16384 2024-09-19T08:55:18.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.132+0000 7f03219e2740 1 Processor -- start 2024-09-19T08:55:18.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.132+0000 7f03219e2740 1 -- start start 2024-09-19T08:55:18.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.132+0000 7f03219e2740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633c6203a30 0x5633c621e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:18.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.132+0000 7f03219e2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633c62074a0 0x5633c620da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:18.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.132+0000 7f03219e2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5633c6207dc0 0x5633c6202a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:18.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.132+0000 7f03219e2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc00e6f0 con 0x5633c62074a0 2024-09-19T08:55:18.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.132+0000 7f03219e2740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc010230 con 0x5633c6203a30 2024-09-19T08:55:18.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.132+0000 7f03219e2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc00edc0 con 0x5633c6207dc0 2024-09-19T08:55:18.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.133+0000 7f031cba4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633c62074a0 0x5633c620da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:18.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.133+0000 7f031cba4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633c62074a0 0x5633c620da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45350/0 (socket says 172.21.15.164:45350) 2024-09-19T08:55:18.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.133+0000 7f031cba4640 1 -- 172.21.15.164:0/444129297 learned_addr learned my addr 172.21.15.164:0/444129297 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:18.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.133+0000 7f031d3a5640 1 --2- 172.21.15.164:0/444129297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633c6203a30 0x5633c621e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:18.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.133+0000 7f031dba6640 1 --2- 172.21.15.164:0/444129297 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5633c6207dc0 0x5633c6202a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:18.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.133+0000 7f031cba4640 1 -- 172.21.15.164:0/444129297 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5633c6207dc0 msgr2=0x5633c6202a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:18.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.133+0000 7f031cba4640 1 --2- 172.21.15.164:0/444129297 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5633c6207dc0 0x5633c6202a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.133+0000 7f031cba4640 1 -- 172.21.15.164:0/444129297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633c6203a30 msgr2=0x5633c621e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:18.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.133+0000 7f031cba4640 1 --2- 172.21.15.164:0/444129297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633c6203a30 0x5633c621e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.133+0000 7f031cba4640 1 -- 172.21.15.164:0/444129297 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633c620e160 con 0x5633c62074a0 2024-09-19T08:55:18.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.133+0000 7f031cba4640 1 --2- 172.21.15.164:0/444129297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633c62074a0 0x5633c620da80 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f0308009940 tx=0x7f0308009060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb4fc757f847be8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:18.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.133+0000 7f0302ffd640 1 -- 172.21.15.164:0/444129297 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0308017040 con 0x5633c62074a0 2024-09-19T08:55:18.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.133+0000 7f0302ffd640 1 -- 172.21.15.164:0/444129297 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0308013950 con 0x5633c62074a0 2024-09-19T08:55:18.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.133+0000 7f0302ffd640 1 -- 172.21.15.164:0/444129297 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03080266a0 con 0x5633c62074a0 2024-09-19T08:55:18.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.134+0000 7f03219e2740 1 -- 172.21.15.164:0/444129297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633c62074a0 msgr2=0x5633c620da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:18.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.134+0000 7f03219e2740 1 --2- 172.21.15.164:0/444129297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633c62074a0 0x5633c620da80 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f0308009940 tx=0x7f0308009060 comp rx=0 tx=0).stop 2024-09-19T08:55:18.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.134+0000 7f03219e2740 1 -- 172.21.15.164:0/444129297 shutdown_connections 2024-09-19T08:55:18.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.134+0000 7f03219e2740 1 --2- 172.21.15.164:0/444129297 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5633c6207dc0 0x5633c6202a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.134+0000 7f03219e2740 1 --2- 172.21.15.164:0/444129297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633c62074a0 0x5633c620da80 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.134+0000 7f03219e2740 1 --2- 172.21.15.164:0/444129297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633c6203a30 0x5633c621e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.134+0000 7f03219e2740 1 -- 172.21.15.164:0/444129297 >> 172.21.15.164:0/444129297 conn(0x5633c620f850 msgr2=0x5633c6202350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:18.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.134+0000 7f03219e2740 1 -- 172.21.15.164:0/444129297 shutdown_connections 2024-09-19T08:55:18.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.134+0000 7f03219e2740 1 -- 172.21.15.164:0/444129297 wait complete. 2024-09-19T08:55:18.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.134+0000 7f03219e2740 1 -- start start 2024-09-19T08:55:18.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.135+0000 7f03219e2740 1 Processor -- start 2024-09-19T08:55:18.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.135+0000 7f03219e2740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633c6203a30 0x5633c62167e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:18.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.135+0000 7f03219e2740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633c62074a0 0x5633c6216d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:18.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.135+0000 7f03219e2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5633c6207dc0 0x5633c628be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:18.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.135+0000 7f03219e2740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc00fb60 con 0x5633c6203a30 2024-09-19T08:55:18.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.135+0000 7f03219e2740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc00f490 con 0x5633c62074a0 2024-09-19T08:55:18.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.135+0000 7f03219e2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc00c6f0 con 0x5633c6207dc0 2024-09-19T08:55:18.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.135+0000 7f03219e2740 10 client.0 _ll_register_callbacks cb 0x7f031400fb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:55:18.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.135+0000 7f031d3a5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633c6203a30 0x5633c62167e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:18.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.135+0000 7f031d3a5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633c6203a30 0x5633c62167e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45358/0 (socket says 172.21.15.164:45358) 2024-09-19T08:55:18.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.135+0000 7f031d3a5640 1 -- 172.21.15.164:0/3630374224 learned_addr learned my addr 172.21.15.164:0/3630374224 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:18.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.135+0000 7f031cba4640 1 --2- 172.21.15.164:0/3630374224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633c62074a0 0x5633c6216d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:18.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.135+0000 7f031dba6640 1 --2- 172.21.15.164:0/3630374224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5633c6207dc0 0x5633c628be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:18.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.135+0000 7f031d3a5640 1 -- 172.21.15.164:0/3630374224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5633c6207dc0 msgr2=0x5633c628be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:18.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.135+0000 7f031d3a5640 1 --2- 172.21.15.164:0/3630374224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5633c6207dc0 0x5633c628be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.135+0000 7f031d3a5640 1 -- 172.21.15.164:0/3630374224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633c62074a0 msgr2=0x5633c6216d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:18.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.135+0000 7f031d3a5640 1 --2- 172.21.15.164:0/3630374224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633c62074a0 0x5633c6216d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.135+0000 7f031d3a5640 1 -- 172.21.15.164:0/3630374224 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633c628c360 con 0x5633c6203a30 2024-09-19T08:55:18.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.136+0000 7f031d3a5640 1 --2- 172.21.15.164:0/3630374224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633c6203a30 0x5633c62167e0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f031401d850 tx=0x7f031400a8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df238051aaf08314 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:18.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.136+0000 7f0300ff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:55:18.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.136+0000 7f0300ff9640 1 -- 172.21.15.164:0/3630374224 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031403c020 con 0x5633c6203a30 2024-09-19T08:55:18.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.136+0000 7f0300ff9640 1 -- 172.21.15.164:0/3630374224 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7f03140255f0 con 0x5633c6203a30 2024-09-19T08:55:18.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.136+0000 7f03219e2740 1 -- 172.21.15.164:0/3630374224 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0308013950 con 0x5633c6203a30 2024-09-19T08:55:18.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.136+0000 7f0300ff9640 1 -- 172.21.15.164:0/3630374224 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f031402a020 con 0x5633c6203a30 2024-09-19T08:55:18.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.136+0000 7f03219e2740 10 client.17439 fetch_fsmap learned FSMap version 32 2024-09-19T08:55:18.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.136+0000 7f03219e2740 1 -- 172.21.15.164:0/3630374224 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5633c628d510 con 0x5633c6203a30 2024-09-19T08:55:18.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.136+0000 7f0300ff9640 1 -- 172.21.15.164:0/3630374224 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0308013950 con 0x5633c6203a30 2024-09-19T08:55:18.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.136+0000 7f0300ff9640 1 -- 172.21.15.164:0/3630374224 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f0314037050 con 0x5633c6203a30 2024-09-19T08:55:18.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.136+0000 7f03219e2740 10 client.17439 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:55:18.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.136+0000 7f03219e2740 10 client.17439 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:55:18.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.136+0000 7f03219e2740 20 client.17439 populate_metadata read hostname 'smithi164' 2024-09-19T08:55:18.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.136+0000 7f03219e2740 1 --2- 172.21.15.164:0/3630374224 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5633c6212680 0x5633c628fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:18.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.136+0000 7f03219e2740 4 client.17439 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:55:18.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.136+0000 7f03219e2740 1 -- 172.21.15.164:0/3630374224 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5633c6174d60 con 0x5633c6212680 2024-09-19T08:55:18.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.137+0000 7f031cba4640 1 --2- 172.21.15.164:0/3630374224 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5633c6212680 0x5633c628fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:18.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.137+0000 7f0300ff9640 10 client.17439 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:55:18.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.137+0000 7f031cba4640 1 --2- 172.21.15.164:0/3630374224 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5633c6212680 0x5633c628fd20 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81df10952ff95ad4 server_cookie=edaf65ae75bf75e2 in_seq=0 out_seq=0 2024-09-19T08:55:18.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.137+0000 7f0300ff9640 1 -- 172.21.15.164:0/3630374224 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5633c6174d60 con 0x5633c6212680 2024-09-19T08:55:18.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.137+0000 7f0300ff9640 10 client.17439 handle_command_reply: tid=0 2024-09-19T08:55:18.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.137+0000 7f03219e2740 1 client.17439 shutdown 2024-09-19T08:55:18.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.137+0000 7f03219e2740 1 -- 172.21.15.164:0/3630374224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633c6203a30 msgr2=0x5633c62167e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:18.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.137+0000 7f03219e2740 1 --2- 172.21.15.164:0/3630374224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633c6203a30 0x5633c62167e0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f031401d850 tx=0x7f031400a8a0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.137+0000 7f03219e2740 1 -- 172.21.15.164:0/3630374224 shutdown_connections 2024-09-19T08:55:18.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.137+0000 7f03219e2740 1 --2- 172.21.15.164:0/3630374224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5633c6207dc0 0x5633c628be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.137+0000 7f03219e2740 1 --2- 172.21.15.164:0/3630374224 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5633c6212680 0x5633c628fd20 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.137+0000 7f0300ff9640 0 client.17439 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:55:18.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.137+0000 7f03219e2740 1 --2- 172.21.15.164:0/3630374224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633c62074a0 0x5633c6216d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.137+0000 7f03219e2740 1 --2- 172.21.15.164:0/3630374224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633c6203a30 0x5633c62167e0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.137+0000 7f03219e2740 1 -- 172.21.15.164:0/3630374224 >> 172.21.15.164:0/3630374224 conn(0x5633c620f850 msgr2=0x5633c6209710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:18.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.138+0000 7f03219e2740 1 -- 172.21.15.164:0/3630374224 shutdown_connections 2024-09-19T08:55:18.222 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:55:18.222 INFO:teuthology.orchestra.run.smithi164.stdout: "status": "no active scrubs running", 2024-09-19T08:55:18.222 INFO:teuthology.orchestra.run.smithi164.stdout: "scrubs": {} 2024-09-19T08:55:18.223 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:55:18.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.138+0000 7f03219e2740 1 -- 172.21.15.164:0/3630374224 wait complete. 2024-09-19T08:55:18.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.138+0000 7f03219e2740 20 client.17439 trim_cache size 0 max 16384 2024-09-19T08:55:18.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.138+0000 7f031f609640 1 -- 172.21.15.164:0/2561633329 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f03040a33b0 msgr2=0x7f03040a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:18.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.138+0000 7f031f609640 1 --2- 172.21.15.164:0/2561633329 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f03040a33b0 0x7f03040a5870 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f03140058c0 tx=0x7f031400a2d0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.138+0000 7f031f609640 1 -- 172.21.15.164:0/2561633329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f031810c160 msgr2=0x7f0318101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:18.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.138+0000 7f031f609640 1 --2- 172.21.15.164:0/2561633329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f031810c160 0x7f0318101410 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f0310006010 tx=0x7f031002f710 comp rx=0 tx=0).stop 2024-09-19T08:55:18.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.138+0000 7f031f609640 1 -- 172.21.15.164:0/2561633329 shutdown_connections 2024-09-19T08:55:18.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.138+0000 7f031f609640 1 --2- 172.21.15.164:0/2561633329 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f03040a33b0 0x7f03040a5870 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.138+0000 7f031f609640 1 --2- 172.21.15.164:0/2561633329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f031810c160 0x7f0318101410 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.138+0000 7f031f609640 1 --2- 172.21.15.164:0/2561633329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0318103650 0x7f0318100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.138+0000 7f031f609640 1 --2- 172.21.15.164:0/2561633329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0318102d30 0x7f0318100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:18.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.138+0000 7f031f609640 1 -- 172.21.15.164:0/2561633329 >> 172.21.15.164:0/2561633329 conn(0x7f03180fc7d0 msgr2=0x7f03180fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:18.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.139+0000 7f031f609640 1 -- 172.21.15.164:0/2561633329 shutdown_connections 2024-09-19T08:55:18.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:18.139+0000 7f031f609640 1 -- 172.21.15.164:0/2561633329 wait complete. 2024-09-19T08:55:18.281 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:18 smithi164 ceph-mon[30928]: pgmap v294: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T08:55:18.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:18 smithi170 ceph-mon[35072]: pgmap v294: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T08:55:18.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:18 smithi173 ceph-mon[35306]: pgmap v294: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T08:55:18.946 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T08:55:18.947 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:55:19.204 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:55:19.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.687+0000 7f8703f44640 1 -- 172.21.15.164:0/1701686931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86fc101de0 msgr2=0x7f86fc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:19.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.687+0000 7f8703f44640 1 --2- 172.21.15.164:0/1701686931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86fc101de0 0x7f86fc1021c0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f86f8009900 tx=0x7f86f802a410 comp rx=0 tx=0).stop 2024-09-19T08:55:19.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.687+0000 7f8703f44640 1 -- 172.21.15.164:0/1701686931 shutdown_connections 2024-09-19T08:55:19.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.687+0000 7f8703f44640 1 --2- 172.21.15.164:0/1701686931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86fc106f10 0x7f86fc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:19.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.687+0000 7f8703f44640 1 --2- 172.21.15.164:0/1701686931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86fc102790 0x7f86fc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:19.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.687+0000 7f8703f44640 1 --2- 172.21.15.164:0/1701686931 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86fc101de0 0x7f86fc1021c0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:19.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.687+0000 7f8703f44640 1 -- 172.21.15.164:0/1701686931 >> 172.21.15.164:0/1701686931 conn(0x7f86fc0fd8c0 msgr2=0x7f86fc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:19.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.687+0000 7f8703f44640 1 -- 172.21.15.164:0/1701686931 shutdown_connections 2024-09-19T08:55:19.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.687+0000 7f8703f44640 1 -- 172.21.15.164:0/1701686931 wait complete. 2024-09-19T08:55:19.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.688+0000 7f8703f44640 1 Processor -- start 2024-09-19T08:55:19.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.688+0000 7f8703f44640 1 -- start start 2024-09-19T08:55:19.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.688+0000 7f8703f44640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86fc101de0 0x7f86fc198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:19.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.688+0000 7f8703f44640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86fc102790 0x7f86fc198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:19.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.688+0000 7f8703f44640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86fc106f10 0x7f86fc19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:19.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.688+0000 7f8703f44640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc111f70 con 0x7f86fc106f10 2024-09-19T08:55:19.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.688+0000 7f8703f44640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc111df0 con 0x7f86fc101de0 2024-09-19T08:55:19.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.688+0000 7f8703f44640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1120f0 con 0x7f86fc102790 2024-09-19T08:55:19.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.688+0000 7f87024e1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86fc106f10 0x7f86fc19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:19.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.688+0000 7f87024e1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86fc106f10 0x7f86fc19cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45388/0 (socket says 172.21.15.164:45388) 2024-09-19T08:55:19.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.688+0000 7f87024e1640 1 -- 172.21.15.164:0/2662236220 learned_addr learned my addr 172.21.15.164:0/2662236220 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:19.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.688+0000 7f8701ce0640 1 --2- 172.21.15.164:0/2662236220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86fc101de0 0x7f86fc198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:19.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.688+0000 7f87014df640 1 --2- 172.21.15.164:0/2662236220 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86fc102790 0x7f86fc198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:19.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.688+0000 7f87024e1640 1 -- 172.21.15.164:0/2662236220 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86fc102790 msgr2=0x7f86fc198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:19.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.688+0000 7f87024e1640 1 --2- 172.21.15.164:0/2662236220 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86fc102790 0x7f86fc198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:19.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.688+0000 7f87024e1640 1 -- 172.21.15.164:0/2662236220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86fc101de0 msgr2=0x7f86fc198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:19.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.688+0000 7f87024e1640 1 --2- 172.21.15.164:0/2662236220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86fc101de0 0x7f86fc198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:19.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.688+0000 7f87024e1640 1 -- 172.21.15.164:0/2662236220 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86fc19d3e0 con 0x7f86fc106f10 2024-09-19T08:55:19.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.689+0000 7f87024e1640 1 --2- 172.21.15.164:0/2662236220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86fc106f10 0x7f86fc19cd00 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f86f4009fd0 tx=0x7f86f400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:19.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.689+0000 7f86f2ffd640 1 -- 172.21.15.164:0/2662236220 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f4017070 con 0x7f86fc106f10 2024-09-19T08:55:19.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.689+0000 7f8703f44640 1 -- 172.21.15.164:0/2662236220 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86fc19d6d0 con 0x7f86fc106f10 2024-09-19T08:55:19.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.689+0000 7f86f2ffd640 1 -- 172.21.15.164:0/2662236220 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86f4003e50 con 0x7f86fc106f10 2024-09-19T08:55:19.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.689+0000 7f86f2ffd640 1 -- 172.21.15.164:0/2662236220 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f40106c0 con 0x7f86fc106f10 2024-09-19T08:55:19.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.689+0000 7f8703f44640 1 -- 172.21.15.164:0/2662236220 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86fc1a4fb0 con 0x7f86fc106f10 2024-09-19T08:55:19.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.690+0000 7f8703f44640 1 -- 172.21.15.164:0/2662236220 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86fc1021c0 con 0x7f86fc106f10 2024-09-19T08:55:19.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.691+0000 7f86f2ffd640 1 -- 172.21.15.164:0/2662236220 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f86f4003970 con 0x7f86fc106f10 2024-09-19T08:55:19.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.692+0000 7f86f2ffd640 1 --2- 172.21.15.164:0/2662236220 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f86e80a33b0 0x7f86e80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:19.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.692+0000 7f86f2ffd640 1 -- 172.21.15.164:0/2662236220 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7f86f40cf9a0 con 0x7f86fc106f10 2024-09-19T08:55:19.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.692+0000 7f8701ce0640 1 --2- 172.21.15.164:0/2662236220 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f86e80a33b0 0x7f86e80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:19.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.692+0000 7f8701ce0640 1 --2- 172.21.15.164:0/2662236220 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f86e80a33b0 0x7f86e80a5870 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f86f8009230 tx=0x7f86f80058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:19.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.696+0000 7f86f2ffd640 1 -- 172.21.15.164:0/2662236220 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f86f409bd40 con 0x7f86fc106f10 2024-09-19T08:55:19.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.897+0000 7f8703f44640 1 -- 172.21.15.164:0/2662236220 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86fc10a150 con 0x7f86fc106f10 2024-09-19T08:55:19.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.897+0000 7f86f2ffd640 1 -- 172.21.15.164:0/2662236220 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11984 (secure 0 0 0) 0x7f86f40a0040 con 0x7f86fc106f10 2024-09-19T08:55:19.897 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:55:19.897 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":32,"btime":"2024-09-19T08:52:27:192804+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:52:27.074168+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:55:19.898 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 32 2024-09-19T08:55:19.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.900+0000 7f8703f44640 1 -- 172.21.15.164:0/2662236220 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f86e80a33b0 msgr2=0x7f86e80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:19.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.900+0000 7f8703f44640 1 --2- 172.21.15.164:0/2662236220 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f86e80a33b0 0x7f86e80a5870 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f86f8009230 tx=0x7f86f80058d0 comp rx=0 tx=0).stop 2024-09-19T08:55:19.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.900+0000 7f8703f44640 1 -- 172.21.15.164:0/2662236220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86fc106f10 msgr2=0x7f86fc19cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:19.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.900+0000 7f8703f44640 1 --2- 172.21.15.164:0/2662236220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86fc106f10 0x7f86fc19cd00 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f86f4009fd0 tx=0x7f86f400b260 comp rx=0 tx=0).stop 2024-09-19T08:55:19.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.900+0000 7f8703f44640 1 -- 172.21.15.164:0/2662236220 shutdown_connections 2024-09-19T08:55:19.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.900+0000 7f8703f44640 1 --2- 172.21.15.164:0/2662236220 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f86e80a33b0 0x7f86e80a5870 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:19.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.900+0000 7f8703f44640 1 --2- 172.21.15.164:0/2662236220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86fc106f10 0x7f86fc19cd00 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:19.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.900+0000 7f8703f44640 1 --2- 172.21.15.164:0/2662236220 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86fc102790 0x7f86fc198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:19.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.900+0000 7f8703f44640 1 --2- 172.21.15.164:0/2662236220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86fc101de0 0x7f86fc198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:19.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.900+0000 7f8703f44640 1 -- 172.21.15.164:0/2662236220 >> 172.21.15.164:0/2662236220 conn(0x7f86fc0fd8c0 msgr2=0x7f86fc0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:19.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.900+0000 7f8703f44640 1 -- 172.21.15.164:0/2662236220 shutdown_connections 2024-09-19T08:55:19.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:19.900+0000 7f8703f44640 1 -- 172.21.15.164:0/2662236220 wait complete. 2024-09-19T08:55:20.216 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:20 smithi164 ceph-mon[30928]: pgmap v295: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 198 B/s wr, 10 op/s 2024-09-19T08:55:20.216 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:20 smithi164 ceph-mon[30928]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T08:55:20.216 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:20 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2662236220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:55:20.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:20 smithi170 ceph-mon[35072]: pgmap v295: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 198 B/s wr, 10 op/s 2024-09-19T08:55:20.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:20 smithi170 ceph-mon[35072]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T08:55:20.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:20 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2662236220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:55:20.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:20 smithi173 ceph-mon[35306]: pgmap v295: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 198 B/s wr, 10 op/s 2024-09-19T08:55:20.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:20 smithi173 ceph-mon[35306]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T08:55:20.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:20 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2662236220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:55:20.553 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T08:55:20.803 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:55:21.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.336+0000 7fd68d06a640 1 -- 172.21.15.164:0/2980311494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd688102790 msgr2=0x7fd6881067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:21.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.336+0000 7fd68d06a640 1 --2- 172.21.15.164:0/2980311494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd688102790 0x7fd6881067e0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009900 tx=0x7fd67c02a410 comp rx=0 tx=0).stop 2024-09-19T08:55:21.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.336+0000 7fd68d06a640 1 -- 172.21.15.164:0/2980311494 shutdown_connections 2024-09-19T08:55:21.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.336+0000 7fd68d06a640 1 --2- 172.21.15.164:0/2980311494 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd688106f10 0x7fd68810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:21.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.336+0000 7fd68d06a640 1 --2- 172.21.15.164:0/2980311494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd688102790 0x7fd6881067e0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:21.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.336+0000 7fd68d06a640 1 --2- 172.21.15.164:0/2980311494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd688101de0 0x7fd6881021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:21.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.336+0000 7fd68d06a640 1 -- 172.21.15.164:0/2980311494 >> 172.21.15.164:0/2980311494 conn(0x7fd6880fd8c0 msgr2=0x7fd6880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:21.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.336+0000 7fd68d06a640 1 -- 172.21.15.164:0/2980311494 shutdown_connections 2024-09-19T08:55:21.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.336+0000 7fd68d06a640 1 -- 172.21.15.164:0/2980311494 wait complete. 2024-09-19T08:55:21.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.337+0000 7fd68d06a640 1 Processor -- start 2024-09-19T08:55:21.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.337+0000 7fd68d06a640 1 -- start start 2024-09-19T08:55:21.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.337+0000 7fd68d06a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd688101de0 0x7fd6880744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:21.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.337+0000 7fd68d06a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd688102790 0x7fd688074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:21.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.337+0000 7fd68d06a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd688106f10 0x7fd688074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:21.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.337+0000 7fd68d06a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd688112010 con 0x7fd688101de0 2024-09-19T08:55:21.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.337+0000 7fd68d06a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd688111e90 con 0x7fd688102790 2024-09-19T08:55:21.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.337+0000 7fd68d06a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd688112190 con 0x7fd688106f10 2024-09-19T08:55:21.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.338+0000 7fd686d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd688101de0 0x7fd6880744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:21.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.338+0000 7fd686d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd688101de0 0x7fd6880744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45416/0 (socket says 172.21.15.164:45416) 2024-09-19T08:55:21.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.338+0000 7fd686d9d640 1 -- 172.21.15.164:0/3003502573 learned_addr learned my addr 172.21.15.164:0/3003502573 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:21.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.338+0000 7fd68759e640 1 --2- 172.21.15.164:0/3003502573 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd688106f10 0x7fd688074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:21.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.338+0000 7fd68659c640 1 --2- 172.21.15.164:0/3003502573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd688102790 0x7fd688074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:21.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.338+0000 7fd686d9d640 1 -- 172.21.15.164:0/3003502573 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd688106f10 msgr2=0x7fd688074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:21.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.338+0000 7fd686d9d640 1 --2- 172.21.15.164:0/3003502573 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd688106f10 0x7fd688074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:21.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.338+0000 7fd686d9d640 1 -- 172.21.15.164:0/3003502573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd688102790 msgr2=0x7fd688074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:21.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.338+0000 7fd686d9d640 1 --2- 172.21.15.164:0/3003502573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd688102790 0x7fd688074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:21.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.338+0000 7fd686d9d640 1 -- 172.21.15.164:0/3003502573 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6881a6a70 con 0x7fd688101de0 2024-09-19T08:55:21.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.338+0000 7fd68759e640 1 --2- 172.21.15.164:0/3003502573 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd688106f10 0x7fd688074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:55:21.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.338+0000 7fd686d9d640 1 --2- 172.21.15.164:0/3003502573 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd688101de0 0x7fd6880744f0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fd67800b4a0 tx=0x7fd67800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:21.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.338+0000 7fd66ffff640 1 -- 172.21.15.164:0/3003502573 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd678014070 con 0x7fd688101de0 2024-09-19T08:55:21.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.338+0000 7fd66ffff640 1 -- 172.21.15.164:0/3003502573 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd678010af0 con 0x7fd688101de0 2024-09-19T08:55:21.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.338+0000 7fd68d06a640 1 -- 172.21.15.164:0/3003502573 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6881a6d60 con 0x7fd688101de0 2024-09-19T08:55:21.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.338+0000 7fd66ffff640 1 -- 172.21.15.164:0/3003502573 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd678017d10 con 0x7fd688101de0 2024-09-19T08:55:21.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.338+0000 7fd68d06a640 1 -- 172.21.15.164:0/3003502573 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6881a7240 con 0x7fd688101de0 2024-09-19T08:55:21.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.340+0000 7fd66ffff640 1 -- 172.21.15.164:0/3003502573 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fd678017070 con 0x7fd688101de0 2024-09-19T08:55:21.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.341+0000 7fd66ffff640 1 --2- 172.21.15.164:0/3003502573 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd6700a33b0 0x7fd6700a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:21.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.341+0000 7fd68f443740 1 Processor -- start 2024-09-19T08:55:21.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.341+0000 7fd68f443740 1 -- start start 2024-09-19T08:55:21.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.341+0000 7fd66ffff640 1 -- 172.21.15.164:0/3003502573 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fd6780ceff0 con 0x7fd688101de0 2024-09-19T08:55:21.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.341+0000 7fd68659c640 1 --2- 172.21.15.164:0/3003502573 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd6700a33b0 0x7fd6700a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:21.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.341+0000 7fd68f443740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a8d548b40 0x562a8d547090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:21.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.341+0000 7fd68f443740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a8d5475d0 0x562a8d547a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:21.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.341+0000 7fd68f443740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a8d552370 0x562a8d554760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:21.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.341+0000 7fd68f443740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562a8d4ad2f0 con 0x562a8d548b40 2024-09-19T08:55:21.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.341+0000 7fd68f443740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562a8d454cf0 con 0x562a8d5475d0 2024-09-19T08:55:21.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.341+0000 7fd68f443740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562a8d4a92b0 con 0x562a8d552370 2024-09-19T08:55:21.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.342+0000 7fd686d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a8d548b40 0x562a8d547090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:21.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.342+0000 7fd686d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a8d548b40 0x562a8d547090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45424/0 (socket says 172.21.15.164:45424) 2024-09-19T08:55:21.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.342+0000 7fd686d9d640 1 -- 172.21.15.164:0/1826267900 learned_addr learned my addr 172.21.15.164:0/1826267900 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:21.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.342+0000 7fd68659c640 1 --2- 172.21.15.164:0/3003502573 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd6700a33b0 0x7fd6700a5870 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd688075f80 tx=0x7fd67c031490 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:21.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.342+0000 7fd68659c640 1 --2- 172.21.15.164:0/1826267900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a8d5475d0 0x562a8d547a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:21.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.342+0000 7fd686d9d640 1 -- 172.21.15.164:0/1826267900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a8d552370 msgr2=0x562a8d554760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:21.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.342+0000 7fd686d9d640 1 --2- 172.21.15.164:0/1826267900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a8d552370 0x562a8d554760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:21.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.342+0000 7fd686d9d640 1 -- 172.21.15.164:0/1826267900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a8d5475d0 msgr2=0x562a8d547a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:21.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.342+0000 7fd686d9d640 1 --2- 172.21.15.164:0/1826267900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a8d5475d0 0x562a8d547a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:21.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.342+0000 7fd686d9d640 1 -- 172.21.15.164:0/1826267900 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a8d4393a0 con 0x562a8d548b40 2024-09-19T08:55:21.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.342+0000 7fd686d9d640 1 --2- 172.21.15.164:0/1826267900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a8d548b40 0x562a8d547090 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fd67809b3b0 tx=0x7fd6780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca51a0e440f52ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:21.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.342+0000 7fd66d7fa640 1 -- 172.21.15.164:0/1826267900 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6780b4070 con 0x562a8d548b40 2024-09-19T08:55:21.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.342+0000 7fd66d7fa640 1 -- 172.21.15.164:0/1826267900 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd67809f7d0 con 0x562a8d548b40 2024-09-19T08:55:21.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.342+0000 7fd66d7fa640 1 -- 172.21.15.164:0/1826267900 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67801e070 con 0x562a8d548b40 2024-09-19T08:55:21.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.343+0000 7fd68f443740 1 -- 172.21.15.164:0/1826267900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a8d548b40 msgr2=0x562a8d547090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:21.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.343+0000 7fd68f443740 1 --2- 172.21.15.164:0/1826267900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a8d548b40 0x562a8d547090 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fd67809b3b0 tx=0x7fd6780046c0 comp rx=0 tx=0).stop 2024-09-19T08:55:21.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.343+0000 7fd68f443740 1 -- 172.21.15.164:0/1826267900 shutdown_connections 2024-09-19T08:55:21.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.343+0000 7fd68f443740 1 --2- 172.21.15.164:0/1826267900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a8d552370 0x562a8d554760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:21.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.343+0000 7fd68f443740 1 --2- 172.21.15.164:0/1826267900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a8d5475d0 0x562a8d547a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:21.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.343+0000 7fd68f443740 1 --2- 172.21.15.164:0/1826267900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a8d548b40 0x562a8d547090 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:21.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.343+0000 7fd68f443740 1 -- 172.21.15.164:0/1826267900 >> 172.21.15.164:0/1826267900 conn(0x562a8d510310 msgr2=0x562a8d543c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:21.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.343+0000 7fd68f443740 1 -- 172.21.15.164:0/1826267900 shutdown_connections 2024-09-19T08:55:21.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.343+0000 7fd68f443740 1 -- 172.21.15.164:0/1826267900 wait complete. 2024-09-19T08:55:21.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.343+0000 7fd68f443740 1 -- start start 2024-09-19T08:55:21.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.343+0000 7fd68f443740 1 Processor -- start 2024-09-19T08:55:21.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.344+0000 7fd68f443740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a8d5475d0 0x562a8d551da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:21.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.344+0000 7fd68f443740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a8d548b40 0x562a8d54f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:21.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.344+0000 7fd68f443740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a8d552370 0x562a8d54f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:21.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.344+0000 7fd68f443740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562a8d555070 con 0x562a8d552370 2024-09-19T08:55:21.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.344+0000 7fd68f443740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562a8d468060 con 0x562a8d5475d0 2024-09-19T08:55:21.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.344+0000 7fd68f443740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562a8d5551f0 con 0x562a8d548b40 2024-09-19T08:55:21.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.344+0000 7fd68f443740 10 client.0 _ll_register_callbacks cb 0x562a8d1bb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:55:21.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.344+0000 7fd68759e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a8d552370 0x562a8d54f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:21.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.344+0000 7fd68759e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a8d552370 0x562a8d54f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45428/0 (socket says 172.21.15.164:45428) 2024-09-19T08:55:21.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.344+0000 7fd68759e640 1 -- 172.21.15.164:0/2281956034 learned_addr learned my addr 172.21.15.164:0/2281956034 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:21.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.344+0000 7fd686d9d640 1 --2- 172.21.15.164:0/2281956034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a8d5475d0 0x562a8d551da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:21.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.344+0000 7fd68759e640 1 -- 172.21.15.164:0/2281956034 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a8d548b40 msgr2=0x562a8d54f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:21.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.344+0000 7fd68659c640 1 --2- 172.21.15.164:0/2281956034 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a8d548b40 0x562a8d54f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:21.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.344+0000 7fd68759e640 1 --2- 172.21.15.164:0/2281956034 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a8d548b40 0x562a8d54f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:21.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.344+0000 7fd68759e640 1 -- 172.21.15.164:0/2281956034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a8d5475d0 msgr2=0x562a8d551da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:21.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.344+0000 7fd68759e640 1 --2- 172.21.15.164:0/2281956034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a8d5475d0 0x562a8d551da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:21.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.344+0000 7fd68759e640 1 -- 172.21.15.164:0/2281956034 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a8d21e3c0 con 0x562a8d552370 2024-09-19T08:55:21.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.344+0000 7fd68759e640 1 --2- 172.21.15.164:0/2281956034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a8d552370 0x562a8d54f7e0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fd68000c9f0 tx=0x7fd68000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79044c3a2e41bcca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:21.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.344+0000 7fd66aff5640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:55:21.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.344+0000 7fd66aff5640 1 -- 172.21.15.164:0/2281956034 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68001a060 con 0x562a8d552370 2024-09-19T08:55:21.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.344+0000 7fd66aff5640 1 -- 172.21.15.164:0/2281956034 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fd6800079b0 con 0x562a8d552370 2024-09-19T08:55:21.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.344+0000 7fd68f443740 1 -- 172.21.15.164:0/2281956034 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd67809f7d0 con 0x562a8d552370 2024-09-19T08:55:21.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.345+0000 7fd66aff5640 1 -- 172.21.15.164:0/2281956034 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd680007bb0 con 0x562a8d552370 2024-09-19T08:55:21.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.345+0000 7fd68f443740 10 client.17457 fetch_fsmap learned FSMap version 32 2024-09-19T08:55:21.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.345+0000 7fd68f443740 1 -- 172.21.15.164:0/2281956034 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562a8d560bb0 con 0x562a8d552370 2024-09-19T08:55:21.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.345+0000 7fd66aff5640 1 -- 172.21.15.164:0/2281956034 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd67809f7d0 con 0x562a8d552370 2024-09-19T08:55:21.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.345+0000 7fd66aff5640 1 -- 172.21.15.164:0/2281956034 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fd68001d370 con 0x562a8d552370 2024-09-19T08:55:21.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.345+0000 7fd68f443740 10 client.17457 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:55:21.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.345+0000 7fd68f443740 10 client.17457 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-19T08:55:21.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.345+0000 7fd68f443740 20 client.17457 populate_metadata read hostname 'smithi164' 2024-09-19T08:55:21.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.345+0000 7fd68f443740 1 --2- 172.21.15.164:0/2281956034 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x562a8d562070 0x562a8d564460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:21.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.345+0000 7fd68f443740 4 client.17457 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:55:21.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.345+0000 7fd68f443740 1 -- 172.21.15.164:0/2281956034 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a8d4ba860 con 0x562a8d562070 2024-09-19T08:55:21.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.345+0000 7fd686d9d640 1 --2- 172.21.15.164:0/2281956034 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x562a8d562070 0x562a8d564460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:21.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.345+0000 7fd66aff5640 10 client.17457 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:55:21.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.345+0000 7fd686d9d640 1 --2- 172.21.15.164:0/2281956034 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x562a8d562070 0x562a8d564460 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bfccc818bb997428 server_cookie=cbd6347f08cb8b49 in_seq=0 out_seq=0 2024-09-19T08:55:21.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.346+0000 7fd66aff5640 1 -- 172.21.15.164:0/2281956034 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562a8d4ba860 con 0x562a8d562070 2024-09-19T08:55:21.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.346+0000 7fd66aff5640 10 client.17457 handle_command_reply: tid=0 2024-09-19T08:55:21.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.346+0000 7fd68f443740 1 client.17457 shutdown 2024-09-19T08:55:21.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.346+0000 7fd68f443740 1 -- 172.21.15.164:0/2281956034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a8d552370 msgr2=0x562a8d54f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:21.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.346+0000 7fd68f443740 1 --2- 172.21.15.164:0/2281956034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a8d552370 0x562a8d54f7e0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fd68000c9f0 tx=0x7fd68000ceb0 comp rx=0 tx=0).stop 2024-09-19T08:55:21.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.346+0000 7fd68f443740 1 -- 172.21.15.164:0/2281956034 shutdown_connections 2024-09-19T08:55:21.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.346+0000 7fd68f443740 1 --2- 172.21.15.164:0/2281956034 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x562a8d562070 0x562a8d564460 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:21.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.346+0000 7fd68f443740 1 --2- 172.21.15.164:0/2281956034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a8d552370 0x562a8d54f7e0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:21.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.346+0000 7fd68f443740 1 --2- 172.21.15.164:0/2281956034 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a8d548b40 0x562a8d54f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:21.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.346+0000 7fd66aff5640 0 client.17457 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:55:21.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.346+0000 7fd68f443740 1 --2- 172.21.15.164:0/2281956034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a8d5475d0 0x562a8d551da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:21.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.346+0000 7fd68f443740 1 -- 172.21.15.164:0/2281956034 >> 172.21.15.164:0/2281956034 conn(0x562a8d510310 msgr2=0x562a8d549990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:21.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.346+0000 7fd68f443740 1 -- 172.21.15.164:0/2281956034 shutdown_connections 2024-09-19T08:55:21.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.346+0000 7fd68f443740 1 -- 172.21.15.164:0/2281956034 wait complete. 2024-09-19T08:55:21.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.346+0000 7fd68f443740 20 client.17457 trim_cache size 0 max 16384 2024-09-19T08:55:21.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.359+0000 7fd68f443740 1 Processor -- start 2024-09-19T08:55:21.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.359+0000 7fd68f443740 1 -- start start 2024-09-19T08:55:21.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.359+0000 7fd68f443740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a8d5475d0 0x562a8d543e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:21.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.360+0000 7fd68f443740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a8d548b40 0x562a8d54e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:21.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.360+0000 7fd68f443740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a8d552370 0x562a8d569d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:21.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.360+0000 7fd68f443740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881afb80 con 0x562a8d548b40 2024-09-19T08:55:21.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.360+0000 7fd68f443740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881b16c0 con 0x562a8d5475d0 2024-09-19T08:55:21.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.360+0000 7fd68f443740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881b0250 con 0x562a8d552370 2024-09-19T08:55:21.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.360+0000 7fd68659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a8d548b40 0x562a8d54e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:21.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.360+0000 7fd68659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a8d548b40 0x562a8d54e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45438/0 (socket says 172.21.15.164:45438) 2024-09-19T08:55:21.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.360+0000 7fd68659c640 1 -- 172.21.15.164:0/1723436644 learned_addr learned my addr 172.21.15.164:0/1723436644 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:21.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.360+0000 7fd686d9d640 1 --2- 172.21.15.164:0/1723436644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a8d5475d0 0x562a8d543e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:21.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.360+0000 7fd68759e640 1 --2- 172.21.15.164:0/1723436644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a8d552370 0x562a8d569d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:21.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.360+0000 7fd68659c640 1 -- 172.21.15.164:0/1723436644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a8d552370 msgr2=0x562a8d569d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:21.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.360+0000 7fd68659c640 1 --2- 172.21.15.164:0/1723436644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a8d552370 0x562a8d569d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:21.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.360+0000 7fd68659c640 1 -- 172.21.15.164:0/1723436644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a8d5475d0 msgr2=0x562a8d543e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:21.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.360+0000 7fd68659c640 1 --2- 172.21.15.164:0/1723436644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a8d5475d0 0x562a8d543e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:21.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.360+0000 7fd68659c640 1 -- 172.21.15.164:0/1723436644 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a8d54fdc0 con 0x562a8d548b40 2024-09-19T08:55:21.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.360+0000 7fd68659c640 1 --2- 172.21.15.164:0/1723436644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a8d548b40 0x562a8d54e910 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fd67c033000 tx=0x7fd67c03c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd29d39fb566f92 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:21.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.360+0000 7fd66cff9640 1 -- 172.21.15.164:0/1723436644 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c04d070 con 0x562a8d548b40 2024-09-19T08:55:21.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.360+0000 7fd66cff9640 1 -- 172.21.15.164:0/1723436644 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd67c004380 con 0x562a8d548b40 2024-09-19T08:55:21.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.360+0000 7fd66cff9640 1 -- 172.21.15.164:0/1723436644 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c035050 con 0x562a8d548b40 2024-09-19T08:55:21.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.361+0000 7fd68f443740 1 -- 172.21.15.164:0/1723436644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a8d548b40 msgr2=0x562a8d54e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:21.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.361+0000 7fd68f443740 1 --2- 172.21.15.164:0/1723436644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a8d548b40 0x562a8d54e910 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fd67c033000 tx=0x7fd67c03c000 comp rx=0 tx=0).stop 2024-09-19T08:55:21.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.361+0000 7fd68f443740 1 -- 172.21.15.164:0/1723436644 shutdown_connections 2024-09-19T08:55:21.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.361+0000 7fd68f443740 1 --2- 172.21.15.164:0/1723436644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a8d552370 0x562a8d569d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:21.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.361+0000 7fd68f443740 1 --2- 172.21.15.164:0/1723436644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a8d548b40 0x562a8d54e910 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:21.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.361+0000 7fd68f443740 1 --2- 172.21.15.164:0/1723436644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a8d5475d0 0x562a8d543e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:21.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.361+0000 7fd68f443740 1 -- 172.21.15.164:0/1723436644 >> 172.21.15.164:0/1723436644 conn(0x562a8d510310 msgr2=0x562a8d543700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:21.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.361+0000 7fd68f443740 1 -- 172.21.15.164:0/1723436644 shutdown_connections 2024-09-19T08:55:21.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.361+0000 7fd68f443740 1 -- 172.21.15.164:0/1723436644 wait complete. 2024-09-19T08:55:21.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.361+0000 7fd68f443740 1 -- start start 2024-09-19T08:55:21.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.361+0000 7fd68f443740 1 Processor -- start 2024-09-19T08:55:21.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.362+0000 7fd68f443740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a8d5475d0 0x562a8d55aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:21.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.362+0000 7fd68f443740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a8d548b40 0x562a8d555e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:21.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.362+0000 7fd68f443740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a8d552370 0x562a8d556340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:21.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.362+0000 7fd68f443740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881b0ff0 con 0x562a8d548b40 2024-09-19T08:55:21.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.362+0000 7fd68f443740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881b0920 con 0x562a8d552370 2024-09-19T08:55:21.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.362+0000 7fd68f443740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881adb80 con 0x562a8d5475d0 2024-09-19T08:55:21.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.362+0000 7fd68f443740 10 client.0 _ll_register_callbacks cb 0x7fd6800096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:55:21.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.362+0000 7fd68659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a8d548b40 0x562a8d555e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:21.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.362+0000 7fd68659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a8d548b40 0x562a8d555e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45454/0 (socket says 172.21.15.164:45454) 2024-09-19T08:55:21.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.362+0000 7fd68659c640 1 -- 172.21.15.164:0/2445371519 learned_addr learned my addr 172.21.15.164:0/2445371519 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:21.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.362+0000 7fd686d9d640 1 --2- 172.21.15.164:0/2445371519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a8d5475d0 0x562a8d55aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:21.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.362+0000 7fd68759e640 1 --2- 172.21.15.164:0/2445371519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a8d552370 0x562a8d556340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:21.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.362+0000 7fd68659c640 1 -- 172.21.15.164:0/2445371519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a8d5475d0 msgr2=0x562a8d55aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:21.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.362+0000 7fd68659c640 1 --2- 172.21.15.164:0/2445371519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a8d5475d0 0x562a8d55aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:21.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.362+0000 7fd68659c640 1 -- 172.21.15.164:0/2445371519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a8d552370 msgr2=0x562a8d556340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:21.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.362+0000 7fd68659c640 1 --2- 172.21.15.164:0/2445371519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a8d552370 0x562a8d556340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:21.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.362+0000 7fd68659c640 1 -- 172.21.15.164:0/2445371519 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a8d556b70 con 0x562a8d548b40 2024-09-19T08:55:21.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.362+0000 7fd68759e640 1 --2- 172.21.15.164:0/2445371519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a8d552370 0x562a8d556340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:55:21.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.362+0000 7fd68659c640 1 --2- 172.21.15.164:0/2445371519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a8d548b40 0x562a8d555e00 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0096c0 tx=0x7fd67c049460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=758801f892547c22 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:21.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.362+0000 7fd66aff5640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:55:21.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.362+0000 7fd66aff5640 1 -- 172.21.15.164:0/2445371519 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c060060 con 0x562a8d548b40 2024-09-19T08:55:21.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.363+0000 7fd66aff5640 1 -- 172.21.15.164:0/2445371519 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fd67c004180 con 0x562a8d548b40 2024-09-19T08:55:21.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.363+0000 7fd68f443740 1 -- 172.21.15.164:0/2445371519 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd67c004380 con 0x562a8d548b40 2024-09-19T08:55:21.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.363+0000 7fd66aff5640 1 -- 172.21.15.164:0/2445371519 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd67c04d070 con 0x562a8d548b40 2024-09-19T08:55:21.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.363+0000 7fd68f443740 10 client.17463 fetch_fsmap learned FSMap version 32 2024-09-19T08:55:21.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.363+0000 7fd68f443740 1 -- 172.21.15.164:0/2445371519 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562a8d5ccc00 con 0x562a8d548b40 2024-09-19T08:55:21.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.363+0000 7fd66aff5640 1 -- 172.21.15.164:0/2445371519 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd67c004380 con 0x562a8d548b40 2024-09-19T08:55:21.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.363+0000 7fd66aff5640 1 -- 172.21.15.164:0/2445371519 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fd67c065470 con 0x562a8d548b40 2024-09-19T08:55:21.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.363+0000 7fd68f443740 10 client.17463 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:55:21.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.363+0000 7fd68f443740 10 client.17463 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-19T08:55:21.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.363+0000 7fd68f443740 20 client.17463 populate_metadata read hostname 'smithi164' 2024-09-19T08:55:21.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.363+0000 7fd68f443740 1 --2- 172.21.15.164:0/2445371519 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x562a8d5cd320 0x562a8d5cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:21.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.363+0000 7fd68f443740 4 client.17463 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:55:21.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.363+0000 7fd68f443740 1 -- 172.21.15.164:0/2445371519 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562a8d4b4d70 con 0x562a8d5cd320 2024-09-19T08:55:21.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.363+0000 7fd686d9d640 1 --2- 172.21.15.164:0/2445371519 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x562a8d5cd320 0x562a8d5cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:21.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.364+0000 7fd66aff5640 10 client.17463 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:55:21.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.364+0000 7fd686d9d640 1 --2- 172.21.15.164:0/2445371519 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x562a8d5cd320 0x562a8d5cf740 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c11219d612775064 server_cookie=d83c5ac1e3e6f05f in_seq=0 out_seq=0 2024-09-19T08:55:21.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.364+0000 7fd66aff5640 1 -- 172.21.15.164:0/2445371519 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562a8d4b4d70 con 0x562a8d5cd320 2024-09-19T08:55:21.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.364+0000 7fd66aff5640 10 client.17463 handle_command_reply: tid=0 2024-09-19T08:55:21.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.364+0000 7fd68f443740 1 client.17463 shutdown 2024-09-19T08:55:21.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.364+0000 7fd68f443740 1 -- 172.21.15.164:0/2445371519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a8d548b40 msgr2=0x562a8d555e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:21.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.364+0000 7fd68f443740 1 --2- 172.21.15.164:0/2445371519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a8d548b40 0x562a8d555e00 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0096c0 tx=0x7fd67c049460 comp rx=0 tx=0).stop 2024-09-19T08:55:21.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.364+0000 7fd68f443740 1 -- 172.21.15.164:0/2445371519 shutdown_connections 2024-09-19T08:55:21.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.364+0000 7fd68f443740 1 --2- 172.21.15.164:0/2445371519 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x562a8d5cd320 0x562a8d5cf740 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:21.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.364+0000 7fd68f443740 1 --2- 172.21.15.164:0/2445371519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a8d552370 0x562a8d556340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:21.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.364+0000 7fd68f443740 1 --2- 172.21.15.164:0/2445371519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a8d548b40 0x562a8d555e00 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:21.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.364+0000 7fd68f443740 1 --2- 172.21.15.164:0/2445371519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a8d5475d0 0x562a8d55aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:21.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.364+0000 7fd68f443740 1 -- 172.21.15.164:0/2445371519 >> 172.21.15.164:0/2445371519 conn(0x562a8d510310 msgr2=0x562a8d5508b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:21.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.364+0000 7fd66aff5640 0 client.17463 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:55:21.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.364+0000 7fd68f443740 1 -- 172.21.15.164:0/2445371519 shutdown_connections 2024-09-19T08:55:21.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.364+0000 7fd68f443740 1 -- 172.21.15.164:0/2445371519 wait complete. 2024-09-19T08:55:21.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.364+0000 7fd68f443740 20 client.17463 trim_cache size 0 max 16384 2024-09-19T08:55:21.380 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:55:21.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.365+0000 7fd68d06a640 1 -- 172.21.15.164:0/3003502573 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd6700a33b0 msgr2=0x7fd6700a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:21.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.365+0000 7fd68d06a640 1 --2- 172.21.15.164:0/3003502573 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd6700a33b0 0x7fd6700a5870 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd688075f80 tx=0x7fd67c031490 comp rx=0 tx=0).stop 2024-09-19T08:55:21.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.365+0000 7fd68d06a640 1 -- 172.21.15.164:0/3003502573 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd688101de0 msgr2=0x7fd6880744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:21.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.365+0000 7fd68d06a640 1 --2- 172.21.15.164:0/3003502573 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd688101de0 0x7fd6880744f0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fd67800b4a0 tx=0x7fd67800b960 comp rx=0 tx=0).stop 2024-09-19T08:55:21.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.365+0000 7fd68d06a640 1 -- 172.21.15.164:0/3003502573 shutdown_connections 2024-09-19T08:55:21.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.365+0000 7fd68d06a640 1 --2- 172.21.15.164:0/3003502573 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd688106f10 0x7fd688074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:21.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.365+0000 7fd68d06a640 1 --2- 172.21.15.164:0/3003502573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd688102790 0x7fd688074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:21.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.365+0000 7fd68d06a640 1 --2- 172.21.15.164:0/3003502573 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd6700a33b0 0x7fd6700a5870 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:21.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.365+0000 7fd68d06a640 1 --2- 172.21.15.164:0/3003502573 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd688101de0 0x7fd6880744f0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:21.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.365+0000 7fd68d06a640 1 -- 172.21.15.164:0/3003502573 >> 172.21.15.164:0/3003502573 conn(0x7fd6880fd8c0 msgr2=0x7fd6880fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:21.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.365+0000 7fd68d06a640 1 -- 172.21.15.164:0/3003502573 shutdown_connections 2024-09-19T08:55:21.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:21.365+0000 7fd68d06a640 1 -- 172.21.15.164:0/3003502573 wait complete. 2024-09-19T08:55:22.021 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T08:55:22.266 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:55:22.281 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:22 smithi164 ceph-mon[30928]: pgmap v297: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T08:55:22.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:22 smithi170 ceph-mon[35072]: pgmap v297: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T08:55:22.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:22 smithi173 ceph-mon[35306]: pgmap v297: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T08:55:22.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.737+0000 7fa5949d0640 1 -- 172.21.15.164:0/3958418793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa590106f00 msgr2=0x7fa59010ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:22.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.737+0000 7fa5949d0640 1 --2- 172.21.15.164:0/3958418793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa590106f00 0x7fa59010ea80 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fa588009ca0 tx=0x7fa58802a8c0 comp rx=0 tx=0).stop 2024-09-19T08:55:22.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.737+0000 7fa5949d0640 1 -- 172.21.15.164:0/3958418793 shutdown_connections 2024-09-19T08:55:22.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.737+0000 7fa5949d0640 1 --2- 172.21.15.164:0/3958418793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa590106f00 0x7fa59010ea80 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:22.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.737+0000 7fa5949d0640 1 --2- 172.21.15.164:0/3958418793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa590102780 0x7fa5901067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:22.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.737+0000 7fa5949d0640 1 --2- 172.21.15.164:0/3958418793 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa590101dd0 0x7fa5901021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:22.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.737+0000 7fa5949d0640 1 -- 172.21.15.164:0/3958418793 >> 172.21.15.164:0/3958418793 conn(0x7fa5900fd8b0 msgr2=0x7fa5900ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:22.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.737+0000 7fa5949d0640 1 -- 172.21.15.164:0/3958418793 shutdown_connections 2024-09-19T08:55:22.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.737+0000 7fa5949d0640 1 -- 172.21.15.164:0/3958418793 wait complete. 2024-09-19T08:55:22.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.737+0000 7fa5949d0640 1 Processor -- start 2024-09-19T08:55:22.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.738+0000 7fa5949d0640 1 -- start start 2024-09-19T08:55:22.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.738+0000 7fa5949d0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa590101dd0 0x7fa59019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:22.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.738+0000 7fa5949d0640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa590102780 0x7fa59019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:22.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.738+0000 7fa5949d0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa590106f00 0x7fa59019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:22.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.738+0000 7fa5949d0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa590111f60 con 0x7fa590106f00 2024-09-19T08:55:22.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.738+0000 7fa5949d0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa590111de0 con 0x7fa590101dd0 2024-09-19T08:55:22.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.738+0000 7fa5949d0640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa5901120e0 con 0x7fa590102780 2024-09-19T08:55:22.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.738+0000 7fa58ed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa590106f00 0x7fa59019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:22.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.738+0000 7fa58ed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa590106f00 0x7fa59019eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45470/0 (socket says 172.21.15.164:45470) 2024-09-19T08:55:22.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.738+0000 7fa58ed9d640 1 -- 172.21.15.164:0/907949560 learned_addr learned my addr 172.21.15.164:0/907949560 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:22.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.738+0000 7fa58e59c640 1 --2- 172.21.15.164:0/907949560 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa590101dd0 0x7fa59019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:22.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.738+0000 7fa58ed9d640 1 -- 172.21.15.164:0/907949560 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa590102780 msgr2=0x7fa59019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:22.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.738+0000 7fa58dd9b640 1 --2- 172.21.15.164:0/907949560 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa590102780 0x7fa59019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:22.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.738+0000 7fa58ed9d640 1 --2- 172.21.15.164:0/907949560 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa590102780 0x7fa59019ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:22.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.738+0000 7fa58ed9d640 1 -- 172.21.15.164:0/907949560 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa590101dd0 msgr2=0x7fa59019a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:22.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.738+0000 7fa58ed9d640 1 --2- 172.21.15.164:0/907949560 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa590101dd0 0x7fa59019a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:22.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.738+0000 7fa58ed9d640 1 -- 172.21.15.164:0/907949560 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa59019f5d0 con 0x7fa590106f00 2024-09-19T08:55:22.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.738+0000 7fa58e59c640 1 --2- 172.21.15.164:0/907949560 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa590101dd0 0x7fa59019a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:55:22.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.738+0000 7fa58ed9d640 1 --2- 172.21.15.164:0/907949560 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa590106f00 0x7fa59019eef0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fa588031a60 tx=0x7fa588031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:22.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.739+0000 7fa57f7fe640 1 -- 172.21.15.164:0/907949560 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa588003c30 con 0x7fa590106f00 2024-09-19T08:55:22.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.739+0000 7fa57f7fe640 1 -- 172.21.15.164:0/907949560 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa588003dd0 con 0x7fa590106f00 2024-09-19T08:55:22.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.739+0000 7fa5949d0640 1 -- 172.21.15.164:0/907949560 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa59019f860 con 0x7fa590106f00 2024-09-19T08:55:22.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.739+0000 7fa57f7fe640 1 -- 172.21.15.164:0/907949560 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5880047e0 con 0x7fa590106f00 2024-09-19T08:55:22.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.739+0000 7fa5949d0640 1 -- 172.21.15.164:0/907949560 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5901a71a0 con 0x7fa590106f00 2024-09-19T08:55:22.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.741+0000 7fa57f7fe640 1 -- 172.21.15.164:0/907949560 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fa58802f790 con 0x7fa590106f00 2024-09-19T08:55:22.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.741+0000 7fa596da9740 1 Processor -- start 2024-09-19T08:55:22.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.741+0000 7fa596da9740 1 -- start start 2024-09-19T08:55:22.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.742+0000 7fa57f7fe640 1 --2- 172.21.15.164:0/907949560 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa5780a33b0 0x7fa5780a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:22.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.742+0000 7fa596da9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564adbf62b40 0x564adbf61090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:22.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.742+0000 7fa596da9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564adbf615d0 0x564adbf61a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:22.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.742+0000 7fa596da9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564adbf6c370 0x564adbf6e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:22.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.742+0000 7fa596da9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564adbec72f0 con 0x564adbf615d0 2024-09-19T08:55:22.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.742+0000 7fa596da9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564adbe6ecf0 con 0x564adbf62b40 2024-09-19T08:55:22.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.742+0000 7fa596da9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564adbec32b0 con 0x564adbf6c370 2024-09-19T08:55:22.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.742+0000 7fa58dd9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564adbf615d0 0x564adbf61a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:22.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.742+0000 7fa57f7fe640 1 -- 172.21.15.164:0/907949560 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fa5880f1110 con 0x7fa590106f00 2024-09-19T08:55:22.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.742+0000 7fa58dd9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564adbf615d0 0x564adbf61a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45476/0 (socket says 172.21.15.164:45476) 2024-09-19T08:55:22.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.742+0000 7fa58dd9b640 1 -- 172.21.15.164:0/1042182341 learned_addr learned my addr 172.21.15.164:0/1042182341 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:22.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.742+0000 7fa58dd9b640 1 --2- 172.21.15.164:0/907949560 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa5780a33b0 0x7fa5780a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:22.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.742+0000 7fa58e59c640 1 --2- 172.21.15.164:0/1042182341 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564adbf62b40 0x564adbf61090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:22.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.742+0000 7fa58ed9d640 1 --2- 172.21.15.164:0/1042182341 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564adbf6c370 0x564adbf6e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:22.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.742+0000 7fa58dd9b640 1 -- 172.21.15.164:0/1042182341 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564adbf6c370 msgr2=0x564adbf6e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:22.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.742+0000 7fa58dd9b640 1 --2- 172.21.15.164:0/1042182341 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564adbf6c370 0x564adbf6e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:22.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.742+0000 7fa58dd9b640 1 -- 172.21.15.164:0/1042182341 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564adbf62b40 msgr2=0x564adbf61090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:22.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.742+0000 7fa58dd9b640 1 --2- 172.21.15.164:0/1042182341 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564adbf62b40 0x564adbf61090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:22.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.742+0000 7fa58dd9b640 1 -- 172.21.15.164:0/1042182341 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564adbe533a0 con 0x564adbf615d0 2024-09-19T08:55:22.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.742+0000 7fa58e59c640 1 --2- 172.21.15.164:0/1042182341 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564adbf62b40 0x564adbf61090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:55:22.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.742+0000 7fa58dd9b640 1 --2- 172.21.15.164:0/1042182341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564adbf615d0 0x564adbf61a10 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fa5840129b0 tx=0x7fa584012e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98f57398511cdc06 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:22.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.742+0000 7fa58dd9b640 1 --2- 172.21.15.164:0/907949560 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa5780a33b0 0x7fa5780a5870 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa58400d710 tx=0x7fa584009000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:22.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.742+0000 7fa57cff9640 1 -- 172.21.15.164:0/1042182341 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa584020070 con 0x564adbf615d0 2024-09-19T08:55:22.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.743+0000 7fa57cff9640 1 -- 172.21.15.164:0/1042182341 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa58400f7c0 con 0x564adbf615d0 2024-09-19T08:55:22.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.743+0000 7fa57cff9640 1 -- 172.21.15.164:0/1042182341 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58401cad0 con 0x564adbf615d0 2024-09-19T08:55:22.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.743+0000 7fa596da9740 1 -- 172.21.15.164:0/1042182341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564adbf615d0 msgr2=0x564adbf61a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:22.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.743+0000 7fa596da9740 1 --2- 172.21.15.164:0/1042182341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564adbf615d0 0x564adbf61a10 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fa5840129b0 tx=0x7fa584012e70 comp rx=0 tx=0).stop 2024-09-19T08:55:22.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.743+0000 7fa596da9740 1 -- 172.21.15.164:0/1042182341 shutdown_connections 2024-09-19T08:55:22.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.743+0000 7fa596da9740 1 --2- 172.21.15.164:0/1042182341 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564adbf6c370 0x564adbf6e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:22.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.743+0000 7fa596da9740 1 --2- 172.21.15.164:0/1042182341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564adbf615d0 0x564adbf61a10 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:22.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.743+0000 7fa596da9740 1 --2- 172.21.15.164:0/1042182341 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564adbf62b40 0x564adbf61090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:22.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.743+0000 7fa596da9740 1 -- 172.21.15.164:0/1042182341 >> 172.21.15.164:0/1042182341 conn(0x564adbf2a310 msgr2=0x564adbf5dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:22.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.743+0000 7fa596da9740 1 -- 172.21.15.164:0/1042182341 shutdown_connections 2024-09-19T08:55:22.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.743+0000 7fa596da9740 1 -- 172.21.15.164:0/1042182341 wait complete. 2024-09-19T08:55:22.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.744+0000 7fa596da9740 1 -- start start 2024-09-19T08:55:22.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.744+0000 7fa596da9740 1 Processor -- start 2024-09-19T08:55:22.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.744+0000 7fa596da9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564adbf615d0 0x564adbf6bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:22.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.744+0000 7fa596da9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564adbf62b40 0x564adbf69080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:22.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.744+0000 7fa596da9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564adbf6c370 0x564adbf69810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:22.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.744+0000 7fa596da9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564adbf6f010 con 0x564adbf615d0 2024-09-19T08:55:22.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.744+0000 7fa596da9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564adbe82060 con 0x564adbf6c370 2024-09-19T08:55:22.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.744+0000 7fa596da9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564adbf6f190 con 0x564adbf62b40 2024-09-19T08:55:22.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.744+0000 7fa58e59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564adbf615d0 0x564adbf6bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:22.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.744+0000 7fa596da9740 10 client.0 _ll_register_callbacks cb 0x564adbbd58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:55:22.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.744+0000 7fa58e59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564adbf615d0 0x564adbf6bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45490/0 (socket says 172.21.15.164:45490) 2024-09-19T08:55:22.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.744+0000 7fa58e59c640 1 -- 172.21.15.164:0/623764600 learned_addr learned my addr 172.21.15.164:0/623764600 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:22.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.744+0000 7fa58dd9b640 1 --2- 172.21.15.164:0/623764600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564adbf62b40 0x564adbf69080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:22.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.744+0000 7fa58ed9d640 1 --2- 172.21.15.164:0/623764600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564adbf6c370 0x564adbf69810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:22.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.744+0000 7fa58e59c640 1 -- 172.21.15.164:0/623764600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564adbf62b40 msgr2=0x564adbf69080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:22.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.744+0000 7fa58e59c640 1 --2- 172.21.15.164:0/623764600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564adbf62b40 0x564adbf69080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:22.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.744+0000 7fa58e59c640 1 -- 172.21.15.164:0/623764600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564adbf6c370 msgr2=0x564adbf69810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:22.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.744+0000 7fa58e59c640 1 --2- 172.21.15.164:0/623764600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564adbf6c370 0x564adbf69810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:22.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.744+0000 7fa58e59c640 1 -- 172.21.15.164:0/623764600 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564adbc383c0 con 0x564adbf615d0 2024-09-19T08:55:22.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.744+0000 7fa58ed9d640 1 --2- 172.21.15.164:0/623764600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564adbf6c370 0x564adbf69810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:55:22.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.745+0000 7fa58e59c640 1 --2- 172.21.15.164:0/623764600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564adbf615d0 0x564adbf6bdd0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fa580010a70 tx=0x7fa580010f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa7869f4e52a0ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:22.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.745+0000 7fa575ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:55:22.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.745+0000 7fa575ffb640 1 -- 172.21.15.164:0/623764600 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58000eda0 con 0x564adbf615d0 2024-09-19T08:55:22.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.745+0000 7fa575ffb640 1 -- 172.21.15.164:0/623764600 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fa58001c480 con 0x564adbf615d0 2024-09-19T08:55:22.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.745+0000 7fa596da9740 1 -- 172.21.15.164:0/623764600 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa58400f7c0 con 0x564adbf615d0 2024-09-19T08:55:22.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.745+0000 7fa575ffb640 1 -- 172.21.15.164:0/623764600 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa58001c680 con 0x564adbf615d0 2024-09-19T08:55:22.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.745+0000 7fa596da9740 10 client.17475 fetch_fsmap learned FSMap version 32 2024-09-19T08:55:22.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.745+0000 7fa596da9740 1 -- 172.21.15.164:0/623764600 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564adbf7ab90 con 0x564adbf615d0 2024-09-19T08:55:22.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.745+0000 7fa575ffb640 1 -- 172.21.15.164:0/623764600 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa58400f7c0 con 0x564adbf615d0 2024-09-19T08:55:22.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.745+0000 7fa575ffb640 1 -- 172.21.15.164:0/623764600 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fa5800259d0 con 0x564adbf615d0 2024-09-19T08:55:22.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.745+0000 7fa596da9740 10 client.17475 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:55:22.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.745+0000 7fa596da9740 10 client.17475 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T08:55:22.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.745+0000 7fa596da9740 20 client.17475 populate_metadata read hostname 'smithi164' 2024-09-19T08:55:22.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.745+0000 7fa596da9740 1 --2- 172.21.15.164:0/623764600 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x564adbf7bfa0 0x564adbf7e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:22.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.745+0000 7fa596da9740 4 client.17475 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:55:22.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.745+0000 7fa596da9740 1 -- 172.21.15.164:0/623764600 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564adbed4860 con 0x564adbf7bfa0 2024-09-19T08:55:22.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.746+0000 7fa58dd9b640 1 --2- 172.21.15.164:0/623764600 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x564adbf7bfa0 0x564adbf7e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:22.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.746+0000 7fa575ffb640 10 client.17475 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:55:22.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.746+0000 7fa58dd9b640 1 --2- 172.21.15.164:0/623764600 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x564adbf7bfa0 0x564adbf7e390 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2b5b0c1b9f844238 server_cookie=34ae4f466cf4b737 in_seq=0 out_seq=0 2024-09-19T08:55:22.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.747+0000 7fa575ffb640 1 -- 172.21.15.164:0/623764600 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564adbed4860 con 0x564adbf7bfa0 2024-09-19T08:55:22.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.747+0000 7fa575ffb640 10 client.17475 handle_command_reply: tid=0 2024-09-19T08:55:22.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.747+0000 7fa596da9740 1 client.17475 shutdown 2024-09-19T08:55:22.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.747+0000 7fa596da9740 1 -- 172.21.15.164:0/623764600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564adbf615d0 msgr2=0x564adbf6bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:22.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.747+0000 7fa596da9740 1 --2- 172.21.15.164:0/623764600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564adbf615d0 0x564adbf6bdd0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fa580010a70 tx=0x7fa580010f30 comp rx=0 tx=0).stop 2024-09-19T08:55:22.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.747+0000 7fa596da9740 1 -- 172.21.15.164:0/623764600 shutdown_connections 2024-09-19T08:55:22.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.747+0000 7fa596da9740 1 --2- 172.21.15.164:0/623764600 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x564adbf7bfa0 0x564adbf7e390 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:22.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.747+0000 7fa575ffb640 0 client.17475 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:55:22.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.747+0000 7fa596da9740 1 --2- 172.21.15.164:0/623764600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564adbf6c370 0x564adbf69810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:22.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.747+0000 7fa596da9740 1 --2- 172.21.15.164:0/623764600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564adbf62b40 0x564adbf69080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:22.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.747+0000 7fa596da9740 1 --2- 172.21.15.164:0/623764600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564adbf615d0 0x564adbf6bdd0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:22.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.747+0000 7fa596da9740 1 -- 172.21.15.164:0/623764600 >> 172.21.15.164:0/623764600 conn(0x564adbf2a310 msgr2=0x564adbf63930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:22.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.747+0000 7fa596da9740 1 -- 172.21.15.164:0/623764600 shutdown_connections 2024-09-19T08:55:22.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.747+0000 7fa596da9740 1 -- 172.21.15.164:0/623764600 wait complete. 2024-09-19T08:55:22.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.747+0000 7fa596da9740 20 client.17475 trim_cache size 0 max 16384 2024-09-19T08:55:22.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.760+0000 7fa596da9740 1 Processor -- start 2024-09-19T08:55:22.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.760+0000 7fa596da9740 1 -- start start 2024-09-19T08:55:22.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.760+0000 7fa596da9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564adbf615d0 0x564adbf62300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:22.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.760+0000 7fa596da9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564adbf62b40 0x564adbf5d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:22.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.760+0000 7fa596da9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564adbf6c370 0x564adbf5dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:22.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.760+0000 7fa596da9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa5901af990 con 0x564adbf6c370 2024-09-19T08:55:22.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.760+0000 7fa596da9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa5901b14d0 con 0x564adbf615d0 2024-09-19T08:55:22.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.760+0000 7fa596da9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa5901b0060 con 0x564adbf62b40 2024-09-19T08:55:22.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.760+0000 7fa58ed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564adbf6c370 0x564adbf5dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:22.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.760+0000 7fa58ed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564adbf6c370 0x564adbf5dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45502/0 (socket says 172.21.15.164:45502) 2024-09-19T08:55:22.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.760+0000 7fa58ed9d640 1 -- 172.21.15.164:0/1576224734 learned_addr learned my addr 172.21.15.164:0/1576224734 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:22.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.760+0000 7fa58e59c640 1 --2- 172.21.15.164:0/1576224734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564adbf615d0 0x564adbf62300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:22.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.760+0000 7fa58dd9b640 1 --2- 172.21.15.164:0/1576224734 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564adbf62b40 0x564adbf5d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:22.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.760+0000 7fa58ed9d640 1 -- 172.21.15.164:0/1576224734 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564adbf62b40 msgr2=0x564adbf5d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:22.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.760+0000 7fa58ed9d640 1 --2- 172.21.15.164:0/1576224734 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564adbf62b40 0x564adbf5d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:22.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.760+0000 7fa58ed9d640 1 -- 172.21.15.164:0/1576224734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564adbf615d0 msgr2=0x564adbf62300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:22.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.760+0000 7fa58ed9d640 1 --2- 172.21.15.164:0/1576224734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564adbf615d0 0x564adbf62300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:22.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.760+0000 7fa58ed9d640 1 -- 172.21.15.164:0/1576224734 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564adbf69b80 con 0x564adbf6c370 2024-09-19T08:55:22.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.760+0000 7fa58dd9b640 1 --2- 172.21.15.164:0/1576224734 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564adbf62b40 0x564adbf5d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:55:22.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.760+0000 7fa58ed9d640 1 --2- 172.21.15.164:0/1576224734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564adbf6c370 0x564adbf5dc40 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fa5880b9940 tx=0x7fa588004050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6d2f0ff6a82cce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:22.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.761+0000 7fa577fff640 1 -- 172.21.15.164:0/1576224734 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5880bdbe0 con 0x564adbf6c370 2024-09-19T08:55:22.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.761+0000 7fa577fff640 1 -- 172.21.15.164:0/1576224734 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa588036040 con 0x564adbf6c370 2024-09-19T08:55:22.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.761+0000 7fa577fff640 1 -- 172.21.15.164:0/1576224734 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5880eeac0 con 0x564adbf6c370 2024-09-19T08:55:22.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.761+0000 7fa596da9740 1 -- 172.21.15.164:0/1576224734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564adbf6c370 msgr2=0x564adbf5dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:22.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.761+0000 7fa596da9740 1 --2- 172.21.15.164:0/1576224734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564adbf6c370 0x564adbf5dc40 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fa5880b9940 tx=0x7fa588004050 comp rx=0 tx=0).stop 2024-09-19T08:55:22.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.761+0000 7fa596da9740 1 -- 172.21.15.164:0/1576224734 shutdown_connections 2024-09-19T08:55:22.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.761+0000 7fa596da9740 1 --2- 172.21.15.164:0/1576224734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564adbf6c370 0x564adbf5dc40 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:22.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.761+0000 7fa596da9740 1 --2- 172.21.15.164:0/1576224734 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564adbf62b40 0x564adbf5d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:22.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.761+0000 7fa596da9740 1 --2- 172.21.15.164:0/1576224734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564adbf615d0 0x564adbf62300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:22.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.761+0000 7fa596da9740 1 -- 172.21.15.164:0/1576224734 >> 172.21.15.164:0/1576224734 conn(0x564adbf2a310 msgr2=0x564adbf58a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:22.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.761+0000 7fa596da9740 1 -- 172.21.15.164:0/1576224734 shutdown_connections 2024-09-19T08:55:22.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.761+0000 7fa596da9740 1 -- 172.21.15.164:0/1576224734 wait complete. 2024-09-19T08:55:22.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.761+0000 7fa596da9740 1 -- start start 2024-09-19T08:55:22.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.762+0000 7fa596da9740 1 Processor -- start 2024-09-19T08:55:22.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.762+0000 7fa596da9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564adbf615d0 0x564adbf74bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:22.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.762+0000 7fa596da9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564adbf62b40 0x564adbf65dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:22.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.762+0000 7fa596da9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564adbf6c370 0x564adbf66310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:22.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.762+0000 7fa596da9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa5901b0e00 con 0x564adbf62b40 2024-09-19T08:55:22.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.762+0000 7fa596da9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa5901b0730 con 0x564adbf615d0 2024-09-19T08:55:22.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.762+0000 7fa596da9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa5901ad990 con 0x564adbf6c370 2024-09-19T08:55:22.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.762+0000 7fa596da9740 10 client.0 _ll_register_callbacks cb 0x7fa58000c490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:55:22.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.762+0000 7fa58dd9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564adbf62b40 0x564adbf65dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:22.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.762+0000 7fa58dd9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564adbf62b40 0x564adbf65dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45518/0 (socket says 172.21.15.164:45518) 2024-09-19T08:55:22.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.762+0000 7fa58dd9b640 1 -- 172.21.15.164:0/922982728 learned_addr learned my addr 172.21.15.164:0/922982728 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:22.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.762+0000 7fa58e59c640 1 --2- 172.21.15.164:0/922982728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564adbf615d0 0x564adbf74bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:22.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.762+0000 7fa58ed9d640 1 --2- 172.21.15.164:0/922982728 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564adbf6c370 0x564adbf66310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:22.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.762+0000 7fa58dd9b640 1 -- 172.21.15.164:0/922982728 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564adbf6c370 msgr2=0x564adbf66310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:22.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.762+0000 7fa58dd9b640 1 --2- 172.21.15.164:0/922982728 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564adbf6c370 0x564adbf66310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:22.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.762+0000 7fa58dd9b640 1 -- 172.21.15.164:0/922982728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564adbf615d0 msgr2=0x564adbf74bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:22.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.762+0000 7fa58dd9b640 1 --2- 172.21.15.164:0/922982728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564adbf615d0 0x564adbf74bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:22.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.762+0000 7fa58dd9b640 1 -- 172.21.15.164:0/922982728 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564adbf66b40 con 0x564adbf62b40 2024-09-19T08:55:22.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.762+0000 7fa58dd9b640 1 --2- 172.21.15.164:0/922982728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564adbf62b40 0x564adbf65dd0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fa58401e010 tx=0x7fa58400ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=579af8471fd979fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:22.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.762+0000 7fa575ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:55:22.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.763+0000 7fa575ffb640 1 -- 172.21.15.164:0/922982728 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa584036030 con 0x564adbf62b40 2024-09-19T08:55:22.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.763+0000 7fa575ffb640 1 -- 172.21.15.164:0/922982728 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fa58400f7c0 con 0x564adbf62b40 2024-09-19T08:55:22.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.763+0000 7fa596da9740 1 -- 172.21.15.164:0/922982728 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa588036040 con 0x564adbf62b40 2024-09-19T08:55:22.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.763+0000 7fa575ffb640 1 -- 172.21.15.164:0/922982728 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa584020040 con 0x564adbf62b40 2024-09-19T08:55:22.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.763+0000 7fa596da9740 10 client.17481 fetch_fsmap learned FSMap version 32 2024-09-19T08:55:22.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.763+0000 7fa596da9740 1 -- 172.21.15.164:0/922982728 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564adbfe6b00 con 0x564adbf62b40 2024-09-19T08:55:22.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.763+0000 7fa575ffb640 1 -- 172.21.15.164:0/922982728 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa588036040 con 0x564adbf62b40 2024-09-19T08:55:22.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.763+0000 7fa575ffb640 1 -- 172.21.15.164:0/922982728 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fa58403d4d0 con 0x564adbf62b40 2024-09-19T08:55:22.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.763+0000 7fa596da9740 10 client.17481 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:55:22.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.763+0000 7fa596da9740 10 client.17481 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T08:55:22.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.763+0000 7fa596da9740 20 client.17481 populate_metadata read hostname 'smithi164' 2024-09-19T08:55:22.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.763+0000 7fa596da9740 1 --2- 172.21.15.164:0/922982728 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x564adbf7bfa0 0x564adbfe8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:22.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.763+0000 7fa596da9740 4 client.17481 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:55:22.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.763+0000 7fa596da9740 1 -- 172.21.15.164:0/922982728 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564adbeced70 con 0x564adbf7bfa0 2024-09-19T08:55:22.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.764+0000 7fa58e59c640 1 --2- 172.21.15.164:0/922982728 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x564adbf7bfa0 0x564adbfe8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:22.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.764+0000 7fa575ffb640 10 client.17481 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:55:22.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.764+0000 7fa58e59c640 1 --2- 172.21.15.164:0/922982728 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x564adbf7bfa0 0x564adbfe8f90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=87b2a23ba2a9ae6f server_cookie=a8f011652e4e70ac in_seq=0 out_seq=0 2024-09-19T08:55:22.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.764+0000 7fa575ffb640 1 -- 172.21.15.164:0/922982728 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564adbeced70 con 0x564adbf7bfa0 2024-09-19T08:55:22.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.764+0000 7fa575ffb640 10 client.17481 handle_command_reply: tid=0 2024-09-19T08:55:22.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.764+0000 7fa596da9740 1 client.17481 shutdown 2024-09-19T08:55:22.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.764+0000 7fa596da9740 1 -- 172.21.15.164:0/922982728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564adbf62b40 msgr2=0x564adbf65dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:22.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.764+0000 7fa596da9740 1 --2- 172.21.15.164:0/922982728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564adbf62b40 0x564adbf65dd0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fa58401e010 tx=0x7fa58400ff30 comp rx=0 tx=0).stop 2024-09-19T08:55:22.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.764+0000 7fa596da9740 1 -- 172.21.15.164:0/922982728 shutdown_connections 2024-09-19T08:55:22.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.764+0000 7fa596da9740 1 --2- 172.21.15.164:0/922982728 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x564adbf7bfa0 0x564adbfe8f90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:22.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.764+0000 7fa596da9740 1 --2- 172.21.15.164:0/922982728 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564adbf6c370 0x564adbf66310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:22.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.764+0000 7fa596da9740 1 --2- 172.21.15.164:0/922982728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564adbf62b40 0x564adbf65dd0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:22.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.764+0000 7fa596da9740 1 --2- 172.21.15.164:0/922982728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564adbf615d0 0x564adbf74bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:22.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.764+0000 7fa596da9740 1 -- 172.21.15.164:0/922982728 >> 172.21.15.164:0/922982728 conn(0x564adbf2a310 msgr2=0x564adbf62170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:22.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.764+0000 7fa575ffb640 0 client.17481 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:55:22.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.764+0000 7fa596da9740 1 -- 172.21.15.164:0/922982728 shutdown_connections 2024-09-19T08:55:22.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.764+0000 7fa596da9740 1 -- 172.21.15.164:0/922982728 wait complete. 2024-09-19T08:55:22.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.765+0000 7fa596da9740 20 client.17481 trim_cache size 0 max 16384 2024-09-19T08:55:22.772 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:55:22.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.765+0000 7fa5949d0640 1 -- 172.21.15.164:0/907949560 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa5780a33b0 msgr2=0x7fa5780a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:22.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.765+0000 7fa5949d0640 1 --2- 172.21.15.164:0/907949560 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa5780a33b0 0x7fa5780a5870 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa58400d710 tx=0x7fa584009000 comp rx=0 tx=0).stop 2024-09-19T08:55:22.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.765+0000 7fa5949d0640 1 -- 172.21.15.164:0/907949560 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa590106f00 msgr2=0x7fa59019eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:22.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.765+0000 7fa5949d0640 1 --2- 172.21.15.164:0/907949560 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa590106f00 0x7fa59019eef0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fa588031a60 tx=0x7fa588031a90 comp rx=0 tx=0).stop 2024-09-19T08:55:22.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.765+0000 7fa5949d0640 1 -- 172.21.15.164:0/907949560 shutdown_connections 2024-09-19T08:55:22.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.765+0000 7fa5949d0640 1 --2- 172.21.15.164:0/907949560 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa5780a33b0 0x7fa5780a5870 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:22.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.765+0000 7fa5949d0640 1 --2- 172.21.15.164:0/907949560 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa590106f00 0x7fa59019eef0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:22.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.765+0000 7fa5949d0640 1 --2- 172.21.15.164:0/907949560 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa590102780 0x7fa59019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:22.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.765+0000 7fa5949d0640 1 --2- 172.21.15.164:0/907949560 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa590101dd0 0x7fa59019a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:22.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.765+0000 7fa5949d0640 1 -- 172.21.15.164:0/907949560 >> 172.21.15.164:0/907949560 conn(0x7fa5900fd8b0 msgr2=0x7fa5900ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:22.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.765+0000 7fa5949d0640 1 -- 172.21.15.164:0/907949560 shutdown_connections 2024-09-19T08:55:22.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:22.765+0000 7fa5949d0640 1 -- 172.21.15.164:0/907949560 wait complete. 2024-09-19T08:55:23.556 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T08:55:23.802 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:55:24.175 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:24 smithi164 ceph-mon[30928]: pgmap v298: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T08:55:24.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.255+0000 7f457db29640 1 -- 172.21.15.164:0/748669849 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4578101de0 msgr2=0x7f45781021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:24.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.255+0000 7f457db29640 1 --2- 172.21.15.164:0/748669849 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4578101de0 0x7f45781021c0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f4568009900 tx=0x7f456802a410 comp rx=0 tx=0).stop 2024-09-19T08:55:24.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.255+0000 7f457db29640 1 -- 172.21.15.164:0/748669849 shutdown_connections 2024-09-19T08:55:24.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.255+0000 7f457db29640 1 --2- 172.21.15.164:0/748669849 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4578106f10 0x7f457810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:24.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.255+0000 7f457db29640 1 --2- 172.21.15.164:0/748669849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4578102790 0x7f45781067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:24.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.255+0000 7f457db29640 1 --2- 172.21.15.164:0/748669849 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4578101de0 0x7f45781021c0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:24.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.255+0000 7f457db29640 1 -- 172.21.15.164:0/748669849 >> 172.21.15.164:0/748669849 conn(0x7f45780fd8c0 msgr2=0x7f45780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:24.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.255+0000 7f457db29640 1 -- 172.21.15.164:0/748669849 shutdown_connections 2024-09-19T08:55:24.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.255+0000 7f457db29640 1 -- 172.21.15.164:0/748669849 wait complete. 2024-09-19T08:55:24.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.256+0000 7f457db29640 1 Processor -- start 2024-09-19T08:55:24.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.256+0000 7f457db29640 1 -- start start 2024-09-19T08:55:24.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.256+0000 7f457db29640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4578101de0 0x7f4578198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:24.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.256+0000 7f457db29640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4578102790 0x7f4578198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:24.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.256+0000 7f457db29640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4578106f10 0x7f457819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:24.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.256+0000 7f457db29640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4578111f80 con 0x7f4578102790 2024-09-19T08:55:24.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.256+0000 7f457db29640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4578111e00 con 0x7f4578106f10 2024-09-19T08:55:24.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.256+0000 7f457db29640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4578112100 con 0x7f4578101de0 2024-09-19T08:55:24.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.256+0000 7f4576ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4578102790 0x7f4578198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:24.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.256+0000 7f4576ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4578102790 0x7f4578198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40696/0 (socket says 172.21.15.164:40696) 2024-09-19T08:55:24.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.256+0000 7f4576ffd640 1 -- 172.21.15.164:0/294789474 learned_addr learned my addr 172.21.15.164:0/294789474 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:24.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.257+0000 7f4577fff640 1 --2- 172.21.15.164:0/294789474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4578106f10 0x7f457819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:24.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.257+0000 7f45777fe640 1 --2- 172.21.15.164:0/294789474 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4578101de0 0x7f4578198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:24.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.257+0000 7f4576ffd640 1 -- 172.21.15.164:0/294789474 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4578101de0 msgr2=0x7f4578198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:24.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.257+0000 7f4576ffd640 1 --2- 172.21.15.164:0/294789474 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4578101de0 0x7f4578198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:24.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.257+0000 7f4576ffd640 1 -- 172.21.15.164:0/294789474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4578106f10 msgr2=0x7f457819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:24.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.257+0000 7f4576ffd640 1 --2- 172.21.15.164:0/294789474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4578106f10 0x7f457819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:24.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.257+0000 7f4576ffd640 1 -- 172.21.15.164:0/294789474 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f457819d3e0 con 0x7f4578102790 2024-09-19T08:55:24.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.257+0000 7f4576ffd640 1 --2- 172.21.15.164:0/294789474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4578102790 0x7f4578198970 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f456c00d950 tx=0x7f456c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:24.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.257+0000 7f4574ff9640 1 -- 172.21.15.164:0/294789474 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456c017030 con 0x7f4578102790 2024-09-19T08:55:24.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.257+0000 7f457db29640 1 -- 172.21.15.164:0/294789474 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f457819d6d0 con 0x7f4578102790 2024-09-19T08:55:24.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.257+0000 7f4574ff9640 1 -- 172.21.15.164:0/294789474 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f456c004020 con 0x7f4578102790 2024-09-19T08:55:24.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.257+0000 7f4574ff9640 1 -- 172.21.15.164:0/294789474 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456c0047a0 con 0x7f4578102790 2024-09-19T08:55:24.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.257+0000 7f457db29640 1 -- 172.21.15.164:0/294789474 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45781a4fb0 con 0x7f4578102790 2024-09-19T08:55:24.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.259+0000 7f4574ff9640 1 -- 172.21.15.164:0/294789474 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f456c0041d0 con 0x7f4578102790 2024-09-19T08:55:24.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.260+0000 7f457ff02740 1 Processor -- start 2024-09-19T08:55:24.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.260+0000 7f457ff02740 1 -- start start 2024-09-19T08:55:24.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.260+0000 7f457ff02740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec1f483b40 0x55ec1f482090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:24.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.260+0000 7f457ff02740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ec1f4825d0 0x55ec1f482a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:24.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.260+0000 7f457ff02740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ec1f48d370 0x55ec1f48f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:24.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.260+0000 7f457ff02740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ec1f3e82f0 con 0x55ec1f48d370 2024-09-19T08:55:24.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.260+0000 7f457ff02740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ec1f38fcf0 con 0x55ec1f483b40 2024-09-19T08:55:24.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.260+0000 7f457ff02740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ec1f3e42b0 con 0x55ec1f4825d0 2024-09-19T08:55:24.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.260+0000 7f4576ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ec1f48d370 0x55ec1f48f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:24.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.260+0000 7f4574ff9640 1 --2- 172.21.15.164:0/294789474 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f45600a33b0 0x7f45600a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:24.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.260+0000 7f4576ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ec1f48d370 0x55ec1f48f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40710/0 (socket says 172.21.15.164:40710) 2024-09-19T08:55:24.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.260+0000 7f4576ffd640 1 -- 172.21.15.164:0/2971468364 learned_addr learned my addr 172.21.15.164:0/2971468364 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:24.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.260+0000 7f45777fe640 1 --2- 172.21.15.164:0/2971468364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ec1f4825d0 0x55ec1f482a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:24.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.260+0000 7f4577fff640 1 --2- 172.21.15.164:0/2971468364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec1f483b40 0x55ec1f482090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:24.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.261+0000 7f4577fff640 1 --2- 172.21.15.164:0/294789474 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f45600a33b0 0x7f45600a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:24.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.261+0000 7f4574ff9640 1 -- 172.21.15.164:0/294789474 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f456c0cf670 con 0x7f4578102790 2024-09-19T08:55:24.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.261+0000 7f4576ffd640 1 -- 172.21.15.164:0/2971468364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ec1f4825d0 msgr2=0x55ec1f482a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:24.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.261+0000 7f4576ffd640 1 --2- 172.21.15.164:0/2971468364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ec1f4825d0 0x55ec1f482a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:24.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.261+0000 7f4576ffd640 1 -- 172.21.15.164:0/2971468364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec1f483b40 msgr2=0x55ec1f482090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:24.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.261+0000 7f4576ffd640 1 --2- 172.21.15.164:0/2971468364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec1f483b40 0x55ec1f482090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:24.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.261+0000 7f4576ffd640 1 -- 172.21.15.164:0/2971468364 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec1f3743a0 con 0x55ec1f48d370 2024-09-19T08:55:24.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.261+0000 7f45777fe640 1 --2- 172.21.15.164:0/2971468364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ec1f4825d0 0x55ec1f482a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:55:24.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.261+0000 7f4577fff640 1 --2- 172.21.15.164:0/2971468364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec1f483b40 0x55ec1f482090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:55:24.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.261+0000 7f4576ffd640 1 --2- 172.21.15.164:0/2971468364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ec1f48d370 0x55ec1f48f760 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f456c069f70 tx=0x7f456c004540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9436912ec49b94 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:24.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.261+0000 7f455dffb640 1 -- 172.21.15.164:0/2971468364 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456c0860e0 con 0x55ec1f48d370 2024-09-19T08:55:24.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.261+0000 7f4577fff640 1 --2- 172.21.15.164:0/294789474 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f45600a33b0 0x7f45600a5870 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4570005fd0 tx=0x7f4570004ab0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:24.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.261+0000 7f455dffb640 1 -- 172.21.15.164:0/2971468364 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f456c09c030 con 0x55ec1f48d370 2024-09-19T08:55:24.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.261+0000 7f455dffb640 1 -- 172.21.15.164:0/2971468364 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456c0cd020 con 0x55ec1f48d370 2024-09-19T08:55:24.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.262+0000 7f457ff02740 1 -- 172.21.15.164:0/2971468364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ec1f48d370 msgr2=0x55ec1f48f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:24.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.262+0000 7f457ff02740 1 --2- 172.21.15.164:0/2971468364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ec1f48d370 0x55ec1f48f760 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f456c069f70 tx=0x7f456c004540 comp rx=0 tx=0).stop 2024-09-19T08:55:24.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.262+0000 7f457ff02740 1 -- 172.21.15.164:0/2971468364 shutdown_connections 2024-09-19T08:55:24.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.262+0000 7f457ff02740 1 --2- 172.21.15.164:0/2971468364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ec1f48d370 0x55ec1f48f760 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:24.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.262+0000 7f457ff02740 1 --2- 172.21.15.164:0/2971468364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ec1f4825d0 0x55ec1f482a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:24.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.262+0000 7f457ff02740 1 --2- 172.21.15.164:0/2971468364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec1f483b40 0x55ec1f482090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:24.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.262+0000 7f457ff02740 1 -- 172.21.15.164:0/2971468364 >> 172.21.15.164:0/2971468364 conn(0x55ec1f44b310 msgr2=0x55ec1f47ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:24.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.262+0000 7f457ff02740 1 -- 172.21.15.164:0/2971468364 shutdown_connections 2024-09-19T08:55:24.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.262+0000 7f457ff02740 1 -- 172.21.15.164:0/2971468364 wait complete. 2024-09-19T08:55:24.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.262+0000 7f457ff02740 1 -- start start 2024-09-19T08:55:24.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.262+0000 7f457ff02740 1 Processor -- start 2024-09-19T08:55:24.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.263+0000 7f457ff02740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ec1f4825d0 0x55ec1f48cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:24.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.263+0000 7f457ff02740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ec1f483b40 0x55ec1f48a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:24.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.263+0000 7f457ff02740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec1f48d370 0x55ec1f48a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:24.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.263+0000 7f457ff02740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ec1f490070 con 0x55ec1f4825d0 2024-09-19T08:55:24.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.263+0000 7f457ff02740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ec1f3a3060 con 0x55ec1f48d370 2024-09-19T08:55:24.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.263+0000 7f457ff02740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ec1f4901f0 con 0x55ec1f483b40 2024-09-19T08:55:24.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.263+0000 7f457ff02740 10 client.0 _ll_register_callbacks cb 0x55ec1f0f68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:55:24.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.263+0000 7f45777fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ec1f4825d0 0x55ec1f48cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:24.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.263+0000 7f45777fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ec1f4825d0 0x55ec1f48cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40714/0 (socket says 172.21.15.164:40714) 2024-09-19T08:55:24.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.263+0000 7f45777fe640 1 -- 172.21.15.164:0/1764506288 learned_addr learned my addr 172.21.15.164:0/1764506288 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:24.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.263+0000 7f4576ffd640 1 --2- 172.21.15.164:0/1764506288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ec1f483b40 0x55ec1f48a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:24.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.263+0000 7f4577fff640 1 --2- 172.21.15.164:0/1764506288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec1f48d370 0x55ec1f48a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:24.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.263+0000 7f45777fe640 1 -- 172.21.15.164:0/1764506288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ec1f483b40 msgr2=0x55ec1f48a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:24.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.263+0000 7f45777fe640 1 --2- 172.21.15.164:0/1764506288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ec1f483b40 0x55ec1f48a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:24.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.263+0000 7f45777fe640 1 -- 172.21.15.164:0/1764506288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec1f48d370 msgr2=0x55ec1f48a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:24.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.263+0000 7f45777fe640 1 --2- 172.21.15.164:0/1764506288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec1f48d370 0x55ec1f48a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:24.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.263+0000 7f45777fe640 1 -- 172.21.15.164:0/1764506288 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec1f1593c0 con 0x55ec1f4825d0 2024-09-19T08:55:24.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.263+0000 7f4577fff640 1 --2- 172.21.15.164:0/1764506288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec1f48d370 0x55ec1f48a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:55:24.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.263+0000 7f45777fe640 1 --2- 172.21.15.164:0/1764506288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ec1f4825d0 0x55ec1f48cd90 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f4568009230 tx=0x7f4568005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b522b4711e60c2a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:24.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.263+0000 7f4556ffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:55:24.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.264+0000 7f4556ffd640 1 -- 172.21.15.164:0/1764506288 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4568035040 con 0x55ec1f4825d0 2024-09-19T08:55:24.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.264+0000 7f4556ffd640 1 -- 172.21.15.164:0/1764506288 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f45680046c0 con 0x55ec1f4825d0 2024-09-19T08:55:24.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.264+0000 7f457ff02740 1 -- 172.21.15.164:0/1764506288 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f456c09c030 con 0x55ec1f4825d0 2024-09-19T08:55:24.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.264+0000 7f4556ffd640 1 -- 172.21.15.164:0/1764506288 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f45680048c0 con 0x55ec1f4825d0 2024-09-19T08:55:24.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.264+0000 7f457ff02740 10 client.17493 fetch_fsmap learned FSMap version 32 2024-09-19T08:55:24.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.264+0000 7f457ff02740 1 -- 172.21.15.164:0/1764506288 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ec1f49bb40 con 0x55ec1f4825d0 2024-09-19T08:55:24.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.264+0000 7f4556ffd640 1 -- 172.21.15.164:0/1764506288 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f456c09c030 con 0x55ec1f4825d0 2024-09-19T08:55:24.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.264+0000 7f4556ffd640 1 -- 172.21.15.164:0/1764506288 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f45680315b0 con 0x55ec1f4825d0 2024-09-19T08:55:24.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.264+0000 7f457ff02740 10 client.17493 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:55:24.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.264+0000 7f457ff02740 10 client.17493 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T08:55:24.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.264+0000 7f457ff02740 20 client.17493 populate_metadata read hostname 'smithi164' 2024-09-19T08:55:24.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.264+0000 7f457ff02740 1 --2- 172.21.15.164:0/1764506288 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55ec1f49cf50 0x55ec1f49f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:24.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.264+0000 7f457ff02740 4 client.17493 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:55:24.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.264+0000 7f457ff02740 1 -- 172.21.15.164:0/1764506288 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec1f3f5860 con 0x55ec1f49cf50 2024-09-19T08:55:24.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.264+0000 7f4576ffd640 1 --2- 172.21.15.164:0/1764506288 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55ec1f49cf50 0x55ec1f49f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:24.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.265+0000 7f4576ffd640 1 --2- 172.21.15.164:0/1764506288 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55ec1f49cf50 0x55ec1f49f340 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2061053581b6867a server_cookie=2a81a32e1ffa6432 in_seq=0 out_seq=0 2024-09-19T08:55:24.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.265+0000 7f4556ffd640 10 client.17493 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:55:24.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.266+0000 7f4556ffd640 1 -- 172.21.15.164:0/1764506288 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ec1f3f5860 con 0x55ec1f49cf50 2024-09-19T08:55:24.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.266+0000 7f4556ffd640 10 client.17493 handle_command_reply: tid=0 2024-09-19T08:55:24.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.266+0000 7f457ff02740 1 client.17493 shutdown 2024-09-19T08:55:24.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.266+0000 7f457ff02740 1 -- 172.21.15.164:0/1764506288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ec1f4825d0 msgr2=0x55ec1f48cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:24.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.266+0000 7f457ff02740 1 --2- 172.21.15.164:0/1764506288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ec1f4825d0 0x55ec1f48cd90 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f4568009230 tx=0x7f4568005040 comp rx=0 tx=0).stop 2024-09-19T08:55:24.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.266+0000 7f457ff02740 1 -- 172.21.15.164:0/1764506288 shutdown_connections 2024-09-19T08:55:24.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.266+0000 7f457ff02740 1 --2- 172.21.15.164:0/1764506288 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55ec1f49cf50 0x55ec1f49f340 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:24.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.266+0000 7f457ff02740 1 --2- 172.21.15.164:0/1764506288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec1f48d370 0x55ec1f48a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:24.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.266+0000 7f457ff02740 1 --2- 172.21.15.164:0/1764506288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ec1f483b40 0x55ec1f48a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:24.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.266+0000 7f457ff02740 1 --2- 172.21.15.164:0/1764506288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ec1f4825d0 0x55ec1f48cd90 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:24.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.266+0000 7f4556ffd640 0 client.17493 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:55:24.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.266+0000 7f457ff02740 1 -- 172.21.15.164:0/1764506288 >> 172.21.15.164:0/1764506288 conn(0x55ec1f44b310 msgr2=0x55ec1f484990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:24.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.266+0000 7f457ff02740 1 -- 172.21.15.164:0/1764506288 shutdown_connections 2024-09-19T08:55:24.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.266+0000 7f457ff02740 1 -- 172.21.15.164:0/1764506288 wait complete. 2024-09-19T08:55:24.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.266+0000 7f457ff02740 20 client.17493 trim_cache size 0 max 16384 2024-09-19T08:55:24.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.279+0000 7f457ff02740 1 Processor -- start 2024-09-19T08:55:24.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.279+0000 7f457ff02740 1 -- start start 2024-09-19T08:55:24.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.279+0000 7f457ff02740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec1f4825d0 0x55ec1f485180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:24.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.279+0000 7f457ff02740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ec1f483b40 0x55ec1f4856c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:24.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.279+0000 7f457ff02740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ec1f48d370 0x55ec1f4a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:24.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.279+0000 7f457ff02740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f45781ad7a0 con 0x55ec1f48d370 2024-09-19T08:55:24.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.279+0000 7f457ff02740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f45781af2e0 con 0x55ec1f4825d0 2024-09-19T08:55:24.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.279+0000 7f457ff02740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f45781ade70 con 0x55ec1f483b40 2024-09-19T08:55:24.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.279+0000 7f4577fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ec1f48d370 0x55ec1f4a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:24.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.279+0000 7f4577fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ec1f48d370 0x55ec1f4a4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40726/0 (socket says 172.21.15.164:40726) 2024-09-19T08:55:24.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.279+0000 7f4577fff640 1 -- 172.21.15.164:0/70691499 learned_addr learned my addr 172.21.15.164:0/70691499 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:24.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.279+0000 7f4576ffd640 1 --2- 172.21.15.164:0/70691499 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ec1f483b40 0x55ec1f4856c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:24.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.279+0000 7f45777fe640 1 --2- 172.21.15.164:0/70691499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec1f4825d0 0x55ec1f485180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:24.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.279+0000 7f4577fff640 1 -- 172.21.15.164:0/70691499 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ec1f483b40 msgr2=0x55ec1f4856c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:24.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.279+0000 7f4577fff640 1 --2- 172.21.15.164:0/70691499 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ec1f483b40 0x55ec1f4856c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:24.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.279+0000 7f4577fff640 1 -- 172.21.15.164:0/70691499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec1f4825d0 msgr2=0x55ec1f485180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:24.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.279+0000 7f4577fff640 1 --2- 172.21.15.164:0/70691499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec1f4825d0 0x55ec1f485180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:24.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.279+0000 7f4577fff640 1 -- 172.21.15.164:0/70691499 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec1f48d100 con 0x55ec1f48d370 2024-09-19T08:55:24.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.279+0000 7f45777fe640 1 --2- 172.21.15.164:0/70691499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec1f4825d0 0x55ec1f485180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:55:24.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.279+0000 7f4577fff640 1 --2- 172.21.15.164:0/70691499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ec1f48d370 0x55ec1f4a4bf0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f45700147a0 tx=0x7f4570009ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b7a701c18fb84a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:24.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.280+0000 7f455d7fa640 1 -- 172.21.15.164:0/70691499 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457001a070 con 0x55ec1f48d370 2024-09-19T08:55:24.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.280+0000 7f455d7fa640 1 -- 172.21.15.164:0/70691499 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4570025030 con 0x55ec1f48d370 2024-09-19T08:55:24.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.280+0000 7f455d7fa640 1 -- 172.21.15.164:0/70691499 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457001eb80 con 0x55ec1f48d370 2024-09-19T08:55:24.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.280+0000 7f457ff02740 1 -- 172.21.15.164:0/70691499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ec1f48d370 msgr2=0x55ec1f4a4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:24.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.280+0000 7f457ff02740 1 --2- 172.21.15.164:0/70691499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ec1f48d370 0x55ec1f4a4bf0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f45700147a0 tx=0x7f4570009ab0 comp rx=0 tx=0).stop 2024-09-19T08:55:24.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.280+0000 7f457ff02740 1 -- 172.21.15.164:0/70691499 shutdown_connections 2024-09-19T08:55:24.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.280+0000 7f457ff02740 1 --2- 172.21.15.164:0/70691499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ec1f48d370 0x55ec1f4a4bf0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:24.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.280+0000 7f457ff02740 1 --2- 172.21.15.164:0/70691499 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ec1f483b40 0x55ec1f4856c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:24.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.280+0000 7f457ff02740 1 --2- 172.21.15.164:0/70691499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec1f4825d0 0x55ec1f485180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:24.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.280+0000 7f457ff02740 1 -- 172.21.15.164:0/70691499 >> 172.21.15.164:0/70691499 conn(0x55ec1f44b310 msgr2=0x55ec1f47db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:24.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.280+0000 7f457ff02740 1 -- 172.21.15.164:0/70691499 shutdown_connections 2024-09-19T08:55:24.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.280+0000 7f457ff02740 1 -- 172.21.15.164:0/70691499 wait complete. 2024-09-19T08:55:24.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.281+0000 7f457ff02740 1 -- start start 2024-09-19T08:55:24.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.281+0000 7f457ff02740 1 Processor -- start 2024-09-19T08:55:24.302 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:55:24.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.281+0000 7f457ff02740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ec1f4825d0 0x55ec1f495480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:24.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.281+0000 7f457ff02740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec1f483b40 0x55ec1f4959c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:24.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.281+0000 7f457ff02740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ec1f48d370 0x55ec1f492450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:24.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.281+0000 7f457ff02740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f45781aec10 con 0x55ec1f4825d0 2024-09-19T08:55:24.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.281+0000 7f457ff02740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f45781ae540 con 0x55ec1f483b40 2024-09-19T08:55:24.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.281+0000 7f457ff02740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f45781ab7a0 con 0x55ec1f48d370 2024-09-19T08:55:24.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.281+0000 7f457ff02740 10 client.0 _ll_register_callbacks cb 0x7f4568009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:55:24.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.281+0000 7f45777fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ec1f4825d0 0x55ec1f495480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:24.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.281+0000 7f45777fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ec1f4825d0 0x55ec1f495480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40738/0 (socket says 172.21.15.164:40738) 2024-09-19T08:55:24.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.281+0000 7f45777fe640 1 -- 172.21.15.164:0/2935898406 learned_addr learned my addr 172.21.15.164:0/2935898406 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:24.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.281+0000 7f4577fff640 1 --2- 172.21.15.164:0/2935898406 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ec1f48d370 0x55ec1f492450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:24.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.281+0000 7f4576ffd640 1 --2- 172.21.15.164:0/2935898406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec1f483b40 0x55ec1f4959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:24.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.281+0000 7f45777fe640 1 -- 172.21.15.164:0/2935898406 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ec1f48d370 msgr2=0x55ec1f492450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:24.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.281+0000 7f45777fe640 1 --2- 172.21.15.164:0/2935898406 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ec1f48d370 0x55ec1f492450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:24.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.281+0000 7f45777fe640 1 -- 172.21.15.164:0/2935898406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec1f483b40 msgr2=0x55ec1f4959c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:24.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.281+0000 7f45777fe640 1 --2- 172.21.15.164:0/2935898406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec1f483b40 0x55ec1f4959c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:24.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.282+0000 7f45777fe640 1 -- 172.21.15.164:0/2935898406 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec1f492990 con 0x55ec1f4825d0 2024-09-19T08:55:24.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.282+0000 7f45777fe640 1 --2- 172.21.15.164:0/2935898406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ec1f4825d0 0x55ec1f495480 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f456802aa00 tx=0x7f4568005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2fe81ac79faceb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:24.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.282+0000 7f4556ffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:55:24.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.282+0000 7f4556ffd640 1 -- 172.21.15.164:0/2935898406 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4568035050 con 0x55ec1f4825d0 2024-09-19T08:55:24.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.282+0000 7f457ff02740 1 -- 172.21.15.164:0/2935898406 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4570025030 con 0x55ec1f4825d0 2024-09-19T08:55:24.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.282+0000 7f4556ffd640 1 -- 172.21.15.164:0/2935898406 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f45680046c0 con 0x55ec1f4825d0 2024-09-19T08:55:24.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.282+0000 7f4556ffd640 1 -- 172.21.15.164:0/2935898406 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f45680317c0 con 0x55ec1f4825d0 2024-09-19T08:55:24.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.282+0000 7f457ff02740 10 client.17499 fetch_fsmap learned FSMap version 32 2024-09-19T08:55:24.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.282+0000 7f457ff02740 1 -- 172.21.15.164:0/2935898406 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ec1f508280 con 0x55ec1f4825d0 2024-09-19T08:55:24.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.282+0000 7f4556ffd640 1 -- 172.21.15.164:0/2935898406 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4570025030 con 0x55ec1f4825d0 2024-09-19T08:55:24.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.282+0000 7f4556ffd640 1 -- 172.21.15.164:0/2935898406 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f4568003c40 con 0x55ec1f4825d0 2024-09-19T08:55:24.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.282+0000 7f457ff02740 10 client.17499 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:55:24.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.282+0000 7f457ff02740 10 client.17499 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T08:55:24.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.282+0000 7f457ff02740 20 client.17499 populate_metadata read hostname 'smithi164' 2024-09-19T08:55:24.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.282+0000 7f457ff02740 1 --2- 172.21.15.164:0/2935898406 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55ec1f5089d0 0x55ec1f50adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:24.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.282+0000 7f457ff02740 4 client.17499 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:55:24.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.282+0000 7f457ff02740 1 -- 172.21.15.164:0/2935898406 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ec1f4837b0 con 0x55ec1f5089d0 2024-09-19T08:55:24.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.283+0000 7f4576ffd640 1 --2- 172.21.15.164:0/2935898406 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55ec1f5089d0 0x55ec1f50adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:24.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.283+0000 7f4556ffd640 10 client.17499 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:55:24.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.283+0000 7f4576ffd640 1 --2- 172.21.15.164:0/2935898406 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55ec1f5089d0 0x55ec1f50adf0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8bf11b9dd4d630c3 server_cookie=83e1ff7ac512e26 in_seq=0 out_seq=0 2024-09-19T08:55:24.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.283+0000 7f4556ffd640 1 -- 172.21.15.164:0/2935898406 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ec1f4837b0 con 0x55ec1f5089d0 2024-09-19T08:55:24.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.283+0000 7f4556ffd640 10 client.17499 handle_command_reply: tid=0 2024-09-19T08:55:24.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.283+0000 7f457ff02740 1 client.17499 shutdown 2024-09-19T08:55:24.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.283+0000 7f457ff02740 1 -- 172.21.15.164:0/2935898406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ec1f4825d0 msgr2=0x55ec1f495480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:24.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.283+0000 7f457ff02740 1 --2- 172.21.15.164:0/2935898406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ec1f4825d0 0x55ec1f495480 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f456802aa00 tx=0x7f4568005730 comp rx=0 tx=0).stop 2024-09-19T08:55:24.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.283+0000 7f457ff02740 1 -- 172.21.15.164:0/2935898406 shutdown_connections 2024-09-19T08:55:24.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.283+0000 7f457ff02740 1 --2- 172.21.15.164:0/2935898406 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x55ec1f5089d0 0x55ec1f50adf0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:24.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.283+0000 7f457ff02740 1 --2- 172.21.15.164:0/2935898406 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ec1f48d370 0x55ec1f492450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:24.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.283+0000 7f457ff02740 1 --2- 172.21.15.164:0/2935898406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec1f483b40 0x55ec1f4959c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:24.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.283+0000 7f4556ffd640 0 client.17499 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:55:24.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.283+0000 7f457ff02740 1 --2- 172.21.15.164:0/2935898406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ec1f4825d0 0x55ec1f495480 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:24.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.283+0000 7f457ff02740 1 -- 172.21.15.164:0/2935898406 >> 172.21.15.164:0/2935898406 conn(0x55ec1f44b310 msgr2=0x55ec1f48b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:24.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.284+0000 7f457ff02740 1 -- 172.21.15.164:0/2935898406 shutdown_connections 2024-09-19T08:55:24.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.284+0000 7f457ff02740 1 -- 172.21.15.164:0/2935898406 wait complete. 2024-09-19T08:55:24.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.284+0000 7f457ff02740 20 client.17499 trim_cache size 0 max 16384 2024-09-19T08:55:24.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.284+0000 7f457db29640 1 -- 172.21.15.164:0/294789474 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f45600a33b0 msgr2=0x7f45600a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:24.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.284+0000 7f457db29640 1 --2- 172.21.15.164:0/294789474 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f45600a33b0 0x7f45600a5870 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4570005fd0 tx=0x7f4570004ab0 comp rx=0 tx=0).stop 2024-09-19T08:55:24.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.284+0000 7f457db29640 1 -- 172.21.15.164:0/294789474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4578102790 msgr2=0x7f4578198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:24.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.284+0000 7f457db29640 1 --2- 172.21.15.164:0/294789474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4578102790 0x7f4578198970 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f456c00d950 tx=0x7f456c00de10 comp rx=0 tx=0).stop 2024-09-19T08:55:24.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.284+0000 7f457db29640 1 -- 172.21.15.164:0/294789474 shutdown_connections 2024-09-19T08:55:24.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.284+0000 7f457db29640 1 --2- 172.21.15.164:0/294789474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4578106f10 0x7f457819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:24.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.284+0000 7f457db29640 1 --2- 172.21.15.164:0/294789474 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f45600a33b0 0x7f45600a5870 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:24.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.284+0000 7f457db29640 1 --2- 172.21.15.164:0/294789474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4578102790 0x7f4578198970 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:24.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.284+0000 7f457db29640 1 --2- 172.21.15.164:0/294789474 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4578101de0 0x7f4578198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:24.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.284+0000 7f457db29640 1 -- 172.21.15.164:0/294789474 >> 172.21.15.164:0/294789474 conn(0x7f45780fd8c0 msgr2=0x7f45780feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:24.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.284+0000 7f457db29640 1 -- 172.21.15.164:0/294789474 shutdown_connections 2024-09-19T08:55:24.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:24.284+0000 7f457db29640 1 -- 172.21.15.164:0/294789474 wait complete. 2024-09-19T08:55:24.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:24 smithi170 ceph-mon[35072]: pgmap v298: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T08:55:24.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:24 smithi173 ceph-mon[35306]: pgmap v298: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T08:55:24.997 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T08:55:25.243 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:55:25.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.730+0000 7f39ba391640 1 -- 172.21.15.164:0/635179240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39b410c180 msgr2=0x7f39b410e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:25.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.730+0000 7f39ba391640 1 --2- 172.21.15.164:0/635179240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39b410c180 0x7f39b410e600 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f39ac009ca0 tx=0x7f39ac02a950 comp rx=0 tx=0).stop 2024-09-19T08:55:25.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.730+0000 7f39ba391640 1 -- 172.21.15.164:0/635179240 shutdown_connections 2024-09-19T08:55:25.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.730+0000 7f39ba391640 1 --2- 172.21.15.164:0/635179240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39b410c180 0x7f39b410e600 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:25.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.730+0000 7f39ba391640 1 --2- 172.21.15.164:0/635179240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39b41036c0 0x7f39b410bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:25.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.730+0000 7f39ba391640 1 --2- 172.21.15.164:0/635179240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39b4102da0 0x7f39b4103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:25.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.730+0000 7f39ba391640 1 -- 172.21.15.164:0/635179240 >> 172.21.15.164:0/635179240 conn(0x7f39b40fc7a0 msgr2=0x7f39b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:25.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.730+0000 7f39ba391640 1 -- 172.21.15.164:0/635179240 shutdown_connections 2024-09-19T08:55:25.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.730+0000 7f39ba391640 1 -- 172.21.15.164:0/635179240 wait complete. 2024-09-19T08:55:25.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.731+0000 7f39ba391640 1 Processor -- start 2024-09-19T08:55:25.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.731+0000 7f39ba391640 1 -- start start 2024-09-19T08:55:25.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.731+0000 7f39ba391640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39b4102da0 0x7f39b41983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:25.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.731+0000 7f39ba391640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39b41036c0 0x7f39b4198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:25.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.731+0000 7f39ba391640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39b410c180 0x7f39b419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:25.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.731+0000 7f39ba391640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f39b4111f60 con 0x7f39b4102da0 2024-09-19T08:55:25.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.731+0000 7f39ba391640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f39b4111de0 con 0x7f39b410c180 2024-09-19T08:55:25.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.731+0000 7f39ba391640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f39b41120e0 con 0x7f39b41036c0 2024-09-19T08:55:25.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.731+0000 7f39b3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39b4102da0 0x7f39b41983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:25.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.731+0000 7f39b3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39b4102da0 0x7f39b41983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40760/0 (socket says 172.21.15.164:40760) 2024-09-19T08:55:25.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.731+0000 7f39b3fff640 1 -- 172.21.15.164:0/3154497036 learned_addr learned my addr 172.21.15.164:0/3154497036 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:25.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.731+0000 7f39b892e640 1 --2- 172.21.15.164:0/3154497036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39b410c180 0x7f39b419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:25.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.732+0000 7f39b37fe640 1 --2- 172.21.15.164:0/3154497036 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39b41036c0 0x7f39b4198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:25.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.732+0000 7f39b3fff640 1 -- 172.21.15.164:0/3154497036 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39b41036c0 msgr2=0x7f39b4198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:25.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.732+0000 7f39b3fff640 1 --2- 172.21.15.164:0/3154497036 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39b41036c0 0x7f39b4198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:25.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.732+0000 7f39b3fff640 1 -- 172.21.15.164:0/3154497036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39b410c180 msgr2=0x7f39b419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:25.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.732+0000 7f39b3fff640 1 --2- 172.21.15.164:0/3154497036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39b410c180 0x7f39b419ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:25.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.732+0000 7f39b3fff640 1 -- 172.21.15.164:0/3154497036 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b419d390 con 0x7f39b4102da0 2024-09-19T08:55:25.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.732+0000 7f39b37fe640 1 --2- 172.21.15.164:0/3154497036 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39b41036c0 0x7f39b4198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:55:25.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.732+0000 7f39b3fff640 1 --2- 172.21.15.164:0/3154497036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39b4102da0 0x7f39b41983e0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f39a400d770 tx=0x7f39a400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:25.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.732+0000 7f39b17fa640 1 -- 172.21.15.164:0/3154497036 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a4017070 con 0x7f39b4102da0 2024-09-19T08:55:25.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.732+0000 7f39b17fa640 1 -- 172.21.15.164:0/3154497036 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39a4005500 con 0x7f39b4102da0 2024-09-19T08:55:25.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.732+0000 7f39b17fa640 1 -- 172.21.15.164:0/3154497036 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a4004e00 con 0x7f39b4102da0 2024-09-19T08:55:25.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.732+0000 7f39ba391640 1 -- 172.21.15.164:0/3154497036 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39b419d680 con 0x7f39b4102da0 2024-09-19T08:55:25.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.732+0000 7f39ba391640 1 -- 172.21.15.164:0/3154497036 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39b41a4f60 con 0x7f39b4102da0 2024-09-19T08:55:25.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.734+0000 7f39b17fa640 1 -- 172.21.15.164:0/3154497036 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f39a4004fa0 con 0x7f39b4102da0 2024-09-19T08:55:25.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.735+0000 7f39bc76a740 1 Processor -- start 2024-09-19T08:55:25.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.735+0000 7f39bc76a740 1 -- start start 2024-09-19T08:55:25.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.735+0000 7f39bc76a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be50256b40 0x55be50255090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:25.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.735+0000 7f39bc76a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be502555d0 0x55be50255a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:25.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.735+0000 7f39bc76a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be50260370 0x55be50262760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:25.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.735+0000 7f39bc76a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55be501bb2f0 con 0x55be50260370 2024-09-19T08:55:25.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.735+0000 7f39bc76a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55be50162cf0 con 0x55be502555d0 2024-09-19T08:55:25.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.735+0000 7f39bc76a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55be501b72b0 con 0x55be50256b40 2024-09-19T08:55:25.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.735+0000 7f39b17fa640 1 --2- 172.21.15.164:0/3154497036 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f399c0a33b0 0x7f399c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:25.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.735+0000 7f39b37fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be50260370 0x55be50262760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:25.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.735+0000 7f39b37fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be50260370 0x55be50262760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40770/0 (socket says 172.21.15.164:40770) 2024-09-19T08:55:25.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.735+0000 7f39b892e640 1 --2- 172.21.15.164:0/3154497036 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f399c0a33b0 0x7f399c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:25.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.735+0000 7f39b37fe640 1 -- 172.21.15.164:0/4220118871 learned_addr learned my addr 172.21.15.164:0/4220118871 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:25.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.735+0000 7f39b3fff640 1 --2- 172.21.15.164:0/4220118871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be502555d0 0x55be50255a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:25.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.735+0000 7f39b892e640 1 --2- 172.21.15.164:0/4220118871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be50256b40 0x55be50255090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:25.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.735+0000 7f39b17fa640 1 -- 172.21.15.164:0/3154497036 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f39a40cf760 con 0x7f39b4102da0 2024-09-19T08:55:25.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.735+0000 7f39b37fe640 1 -- 172.21.15.164:0/4220118871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be50256b40 msgr2=0x55be50255090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:25.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.735+0000 7f39b37fe640 1 --2- 172.21.15.164:0/4220118871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be50256b40 0x55be50255090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:25.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.735+0000 7f39b37fe640 1 -- 172.21.15.164:0/4220118871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be502555d0 msgr2=0x55be50255a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:25.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.735+0000 7f39b37fe640 1 --2- 172.21.15.164:0/4220118871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be502555d0 0x55be50255a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:25.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.735+0000 7f39b37fe640 1 -- 172.21.15.164:0/4220118871 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be501473a0 con 0x55be50260370 2024-09-19T08:55:25.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.736+0000 7f39b892e640 1 --2- 172.21.15.164:0/3154497036 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f399c0a33b0 0x7f399c0a5870 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f39ac005e30 tx=0x7f39ac02fe10 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:25.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.736+0000 7f39b37fe640 1 --2- 172.21.15.164:0/4220118871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be50260370 0x55be50262760 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f39a800bcb0 tx=0x7f39a800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657a5452ee848954 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:25.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.736+0000 7f399a7fc640 1 -- 172.21.15.164:0/4220118871 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a8015070 con 0x55be50260370 2024-09-19T08:55:25.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.736+0000 7f399a7fc640 1 -- 172.21.15.164:0/4220118871 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39a8011920 con 0x55be50260370 2024-09-19T08:55:25.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.736+0000 7f399a7fc640 1 -- 172.21.15.164:0/4220118871 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a8010910 con 0x55be50260370 2024-09-19T08:55:25.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.736+0000 7f39bc76a740 1 -- 172.21.15.164:0/4220118871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be50260370 msgr2=0x55be50262760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:25.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.736+0000 7f39bc76a740 1 --2- 172.21.15.164:0/4220118871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be50260370 0x55be50262760 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f39a800bcb0 tx=0x7f39a800f730 comp rx=0 tx=0).stop 2024-09-19T08:55:25.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.737+0000 7f39bc76a740 1 -- 172.21.15.164:0/4220118871 shutdown_connections 2024-09-19T08:55:25.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.737+0000 7f39bc76a740 1 --2- 172.21.15.164:0/4220118871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be50260370 0x55be50262760 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:25.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.737+0000 7f39bc76a740 1 --2- 172.21.15.164:0/4220118871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be502555d0 0x55be50255a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:25.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.737+0000 7f39bc76a740 1 --2- 172.21.15.164:0/4220118871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be50256b40 0x55be50255090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:25.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.737+0000 7f39bc76a740 1 -- 172.21.15.164:0/4220118871 >> 172.21.15.164:0/4220118871 conn(0x55be5021e310 msgr2=0x55be50251c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:25.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.737+0000 7f39bc76a740 1 -- 172.21.15.164:0/4220118871 shutdown_connections 2024-09-19T08:55:25.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.737+0000 7f39bc76a740 1 -- 172.21.15.164:0/4220118871 wait complete. 2024-09-19T08:55:25.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.737+0000 7f39bc76a740 1 -- start start 2024-09-19T08:55:25.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.737+0000 7f39bc76a740 1 Processor -- start 2024-09-19T08:55:25.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.738+0000 7f39bc76a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be502555d0 0x55be5025fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:25.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.738+0000 7f39bc76a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be50256b40 0x55be5025cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:25.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.738+0000 7f39bc76a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be50260370 0x55be5025d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:25.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.738+0000 7f39bc76a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55be50263070 con 0x55be50260370 2024-09-19T08:55:25.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.738+0000 7f39bc76a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55be50176060 con 0x55be50256b40 2024-09-19T08:55:25.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.738+0000 7f39bc76a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55be502631f0 con 0x55be502555d0 2024-09-19T08:55:25.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.738+0000 7f39bc76a740 10 client.0 _ll_register_callbacks cb 0x55be4fec98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:55:25.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.738+0000 7f39b892e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be50260370 0x55be5025d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:25.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.738+0000 7f39b892e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be50260370 0x55be5025d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40784/0 (socket says 172.21.15.164:40784) 2024-09-19T08:55:25.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.738+0000 7f39b892e640 1 -- 172.21.15.164:0/335074027 learned_addr learned my addr 172.21.15.164:0/335074027 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:25.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.738+0000 7f39b37fe640 1 --2- 172.21.15.164:0/335074027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be50256b40 0x55be5025cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:25.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.738+0000 7f39b3fff640 1 --2- 172.21.15.164:0/335074027 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be502555d0 0x55be5025fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:25.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.738+0000 7f39b892e640 1 -- 172.21.15.164:0/335074027 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be502555d0 msgr2=0x55be5025fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:25.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.738+0000 7f39b892e640 1 --2- 172.21.15.164:0/335074027 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be502555d0 0x55be5025fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:25.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.738+0000 7f39b892e640 1 -- 172.21.15.164:0/335074027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be50256b40 msgr2=0x55be5025cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:25.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.738+0000 7f39b892e640 1 --2- 172.21.15.164:0/335074027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be50256b40 0x55be5025cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:25.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.738+0000 7f39b892e640 1 -- 172.21.15.164:0/335074027 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be4ff2c3c0 con 0x55be50260370 2024-09-19T08:55:25.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.738+0000 7f39b3fff640 1 --2- 172.21.15.164:0/335074027 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be502555d0 0x55be5025fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:55:25.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.738+0000 7f39b37fe640 1 --2- 172.21.15.164:0/335074027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be50256b40 0x55be5025cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:55:25.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.738+0000 7f39b892e640 1 --2- 172.21.15.164:0/335074027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be50260370 0x55be5025d570 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f39ac009580 tx=0x7f39ac02a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9318cc5635bdb6ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:25.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.738+0000 7f39937fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:55:25.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.738+0000 7f39937fe640 1 -- 172.21.15.164:0/335074027 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ac034040 con 0x55be50260370 2024-09-19T08:55:25.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.738+0000 7f39937fe640 1 -- 172.21.15.164:0/335074027 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f39ac004040 con 0x55be50260370 2024-09-19T08:55:25.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.738+0000 7f39bc76a740 1 -- 172.21.15.164:0/335074027 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39a8011920 con 0x55be50260370 2024-09-19T08:55:25.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.738+0000 7f39937fe640 1 -- 172.21.15.164:0/335074027 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f39ac004240 con 0x55be50260370 2024-09-19T08:55:25.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.739+0000 7f39bc76a740 10 client.17511 fetch_fsmap learned FSMap version 32 2024-09-19T08:55:25.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.739+0000 7f39bc76a740 1 -- 172.21.15.164:0/335074027 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55be5026eac0 con 0x55be50260370 2024-09-19T08:55:25.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.739+0000 7f39937fe640 1 -- 172.21.15.164:0/335074027 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39a8011920 con 0x55be50260370 2024-09-19T08:55:25.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.739+0000 7f39937fe640 1 -- 172.21.15.164:0/335074027 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f39ac048040 con 0x55be50260370 2024-09-19T08:55:25.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.739+0000 7f39bc76a740 10 client.17511 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:55:25.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.739+0000 7f39bc76a740 10 client.17511 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T08:55:25.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.739+0000 7f39bc76a740 20 client.17511 populate_metadata read hostname 'smithi164' 2024-09-19T08:55:25.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.739+0000 7f39bc76a740 1 --2- 172.21.15.164:0/335074027 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55be5026ff80 0x55be502723d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:25.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.739+0000 7f39bc76a740 4 client.17511 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:55:25.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.739+0000 7f39bc76a740 1 -- 172.21.15.164:0/335074027 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be501c8860 con 0x55be5026ff80 2024-09-19T08:55:25.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.739+0000 7f39b3fff640 1 --2- 172.21.15.164:0/335074027 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55be5026ff80 0x55be502723d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:25.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.740+0000 7f39b3fff640 1 --2- 172.21.15.164:0/335074027 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55be5026ff80 0x55be502723d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3cd05174ade72c80 server_cookie=3be98b86d3e92ac7 in_seq=0 out_seq=0 2024-09-19T08:55:25.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.740+0000 7f39937fe640 10 client.17511 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:55:25.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.741+0000 7f39937fe640 1 -- 172.21.15.164:0/335074027 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55be501c8860 con 0x55be5026ff80 2024-09-19T08:55:25.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.741+0000 7f39937fe640 10 client.17511 handle_command_reply: tid=0 2024-09-19T08:55:25.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.741+0000 7f39bc76a740 1 client.17511 shutdown 2024-09-19T08:55:25.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.741+0000 7f39bc76a740 1 -- 172.21.15.164:0/335074027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be50260370 msgr2=0x55be5025d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:25.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.741+0000 7f39bc76a740 1 --2- 172.21.15.164:0/335074027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be50260370 0x55be5025d570 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f39ac009580 tx=0x7f39ac02a950 comp rx=0 tx=0).stop 2024-09-19T08:55:25.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.741+0000 7f39bc76a740 1 -- 172.21.15.164:0/335074027 shutdown_connections 2024-09-19T08:55:25.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.741+0000 7f39bc76a740 1 --2- 172.21.15.164:0/335074027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be50260370 0x55be5025d570 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:25.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.741+0000 7f39bc76a740 1 --2- 172.21.15.164:0/335074027 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55be5026ff80 0x55be502723d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:25.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.741+0000 7f39bc76a740 1 --2- 172.21.15.164:0/335074027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be50256b40 0x55be5025cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:25.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.741+0000 7f39bc76a740 1 --2- 172.21.15.164:0/335074027 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be502555d0 0x55be5025fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:25.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.741+0000 7f39bc76a740 1 -- 172.21.15.164:0/335074027 >> 172.21.15.164:0/335074027 conn(0x55be5021e310 msgr2=0x55be50257960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:25.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.741+0000 7f39937fe640 0 client.17511 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:55:25.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.741+0000 7f39bc76a740 1 -- 172.21.15.164:0/335074027 shutdown_connections 2024-09-19T08:55:25.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.741+0000 7f39bc76a740 1 -- 172.21.15.164:0/335074027 wait complete. 2024-09-19T08:55:25.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.741+0000 7f39bc76a740 20 client.17511 trim_cache size 0 max 16384 2024-09-19T08:55:25.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.754+0000 7f39bc76a740 1 Processor -- start 2024-09-19T08:55:25.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.754+0000 7f39bc76a740 1 -- start start 2024-09-19T08:55:25.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.754+0000 7f39bc76a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be502555d0 0x55be5026c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:25.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.754+0000 7f39bc76a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be50256b40 0x55be50258120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:25.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.755+0000 7f39bc76a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be50260370 0x55be50258660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:25.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.755+0000 7f39bc76a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f39b41ad730 con 0x55be502555d0 2024-09-19T08:55:25.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.755+0000 7f39bc76a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f39b41af270 con 0x55be50260370 2024-09-19T08:55:25.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.755+0000 7f39bc76a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f39b41ade00 con 0x55be50256b40 2024-09-19T08:55:25.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.755+0000 7f39b3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be502555d0 0x55be5026c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:25.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.755+0000 7f39b3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be502555d0 0x55be5026c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40788/0 (socket says 172.21.15.164:40788) 2024-09-19T08:55:25.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.755+0000 7f39b3fff640 1 -- 172.21.15.164:0/3381146533 learned_addr learned my addr 172.21.15.164:0/3381146533 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:25.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.755+0000 7f39b37fe640 1 --2- 172.21.15.164:0/3381146533 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be50256b40 0x55be50258120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:25.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.755+0000 7f39b3fff640 1 -- 172.21.15.164:0/3381146533 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be50256b40 msgr2=0x55be50258120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:25.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.755+0000 7f39b3fff640 1 --2- 172.21.15.164:0/3381146533 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be50256b40 0x55be50258120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:25.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.755+0000 7f39b3fff640 1 -- 172.21.15.164:0/3381146533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be50260370 msgr2=0x55be50258660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:25.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.755+0000 7f39b3fff640 1 --2- 172.21.15.164:0/3381146533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be50260370 0x55be50258660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:25.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.755+0000 7f39b3fff640 1 -- 172.21.15.164:0/3381146533 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be50251ed0 con 0x55be502555d0 2024-09-19T08:55:25.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.755+0000 7f39b3fff640 1 --2- 172.21.15.164:0/3381146533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be502555d0 0x55be5026c310 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f39a409ba10 tx=0x7f39a406d800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5df12d39920e65 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:25.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.755+0000 7f3999ffb640 1 -- 172.21.15.164:0/3381146533 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a4020040 con 0x55be502555d0 2024-09-19T08:55:25.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.755+0000 7f3999ffb640 1 -- 172.21.15.164:0/3381146533 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39a409fe30 con 0x55be502555d0 2024-09-19T08:55:25.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.755+0000 7f3999ffb640 1 -- 172.21.15.164:0/3381146533 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a40cd170 con 0x55be502555d0 2024-09-19T08:55:25.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.756+0000 7f39bc76a740 1 -- 172.21.15.164:0/3381146533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be502555d0 msgr2=0x55be5026c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:25.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.756+0000 7f39bc76a740 1 --2- 172.21.15.164:0/3381146533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be502555d0 0x55be5026c310 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f39a409ba10 tx=0x7f39a406d800 comp rx=0 tx=0).stop 2024-09-19T08:55:25.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.756+0000 7f39bc76a740 1 -- 172.21.15.164:0/3381146533 shutdown_connections 2024-09-19T08:55:25.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.756+0000 7f39bc76a740 1 --2- 172.21.15.164:0/3381146533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be50260370 0x55be50258660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:25.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.756+0000 7f39bc76a740 1 --2- 172.21.15.164:0/3381146533 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be50256b40 0x55be50258120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:25.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.756+0000 7f39bc76a740 1 --2- 172.21.15.164:0/3381146533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be502555d0 0x55be5026c310 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:25.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.756+0000 7f39bc76a740 1 -- 172.21.15.164:0/3381146533 >> 172.21.15.164:0/3381146533 conn(0x55be5021e310 msgr2=0x55be5025cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:25.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.756+0000 7f39bc76a740 1 -- 172.21.15.164:0/3381146533 shutdown_connections 2024-09-19T08:55:25.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.756+0000 7f39bc76a740 1 -- 172.21.15.164:0/3381146533 wait complete. 2024-09-19T08:55:25.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.756+0000 7f39bc76a740 1 -- start start 2024-09-19T08:55:25.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.757+0000 7f39bc76a740 1 Processor -- start 2024-09-19T08:55:25.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.757+0000 7f39bc76a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be502555d0 0x55be50259240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:25.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.757+0000 7f39bc76a740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be50256b40 0x55be50259780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:25.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.757+0000 7f39bc76a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be50260370 0x55be50259d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:25.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.757+0000 7f39bc76a740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f39b41aeba0 con 0x55be50256b40 2024-09-19T08:55:25.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.757+0000 7f39bc76a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f39b41ae4d0 con 0x55be502555d0 2024-09-19T08:55:25.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.757+0000 7f39bc76a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f39b41ab730 con 0x55be50260370 2024-09-19T08:55:25.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.757+0000 7f39bc76a740 10 client.0 _ll_register_callbacks cb 0x7f39ac009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:55:25.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.757+0000 7f39b37fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be50256b40 0x55be50259780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:25.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.757+0000 7f39b37fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be50256b40 0x55be50259780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40800/0 (socket says 172.21.15.164:40800) 2024-09-19T08:55:25.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.757+0000 7f39b37fe640 1 -- 172.21.15.164:0/262003135 learned_addr learned my addr 172.21.15.164:0/262003135 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:25.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.757+0000 7f39b892e640 1 --2- 172.21.15.164:0/262003135 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be50260370 0x55be50259d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:25.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.757+0000 7f39b3fff640 1 --2- 172.21.15.164:0/262003135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be502555d0 0x55be50259240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:25.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.757+0000 7f39b37fe640 1 -- 172.21.15.164:0/262003135 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be50260370 msgr2=0x55be50259d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:25.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.757+0000 7f39b37fe640 1 --2- 172.21.15.164:0/262003135 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be50260370 0x55be50259d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:25.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.757+0000 7f39b37fe640 1 -- 172.21.15.164:0/262003135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be502555d0 msgr2=0x55be50259240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:25.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.757+0000 7f39b37fe640 1 --2- 172.21.15.164:0/262003135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be502555d0 0x55be50259240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:25.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.757+0000 7f39b37fe640 1 -- 172.21.15.164:0/262003135 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be5025a560 con 0x55be50256b40 2024-09-19T08:55:25.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.757+0000 7f39b3fff640 1 --2- 172.21.15.164:0/262003135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be502555d0 0x55be50259240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:55:25.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.758+0000 7f39b37fe640 1 --2- 172.21.15.164:0/262003135 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be50256b40 0x55be50259780 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x55be50256070 tx=0x7f39a800b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9be36cd470e488 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:25.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.758+0000 7f39937fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:55:25.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.758+0000 7f39937fe640 1 -- 172.21.15.164:0/262003135 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a8028060 con 0x55be50256b40 2024-09-19T08:55:25.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.758+0000 7f39937fe640 1 -- 172.21.15.164:0/262003135 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f39a8005030 con 0x55be50256b40 2024-09-19T08:55:25.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.758+0000 7f39bc76a740 1 -- 172.21.15.164:0/262003135 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39a409fe30 con 0x55be50256b40 2024-09-19T08:55:25.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.758+0000 7f39937fe640 1 -- 172.21.15.164:0/262003135 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f39a8015050 con 0x55be50256b40 2024-09-19T08:55:25.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.758+0000 7f39bc76a740 10 client.17517 fetch_fsmap learned FSMap version 32 2024-09-19T08:55:25.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.758+0000 7f39bc76a740 1 -- 172.21.15.164:0/262003135 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55be502df2d0 con 0x55be50256b40 2024-09-19T08:55:25.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.758+0000 7f39937fe640 1 -- 172.21.15.164:0/262003135 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39a409fe30 con 0x55be50256b40 2024-09-19T08:55:25.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.758+0000 7f39937fe640 1 -- 172.21.15.164:0/262003135 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f39a80035f0 con 0x55be50256b40 2024-09-19T08:55:25.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.758+0000 7f39bc76a740 10 client.17517 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:55:25.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.758+0000 7f39bc76a740 10 client.17517 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T08:55:25.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.758+0000 7f39bc76a740 20 client.17517 populate_metadata read hostname 'smithi164' 2024-09-19T08:55:25.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.758+0000 7f39bc76a740 1 --2- 172.21.15.164:0/262003135 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55be502df710 0x55be502e1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:25.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.758+0000 7f39bc76a740 4 client.17517 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:55:25.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.758+0000 7f39bc76a740 1 -- 172.21.15.164:0/262003135 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55be501c2d70 con 0x55be502df710 2024-09-19T08:55:25.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.759+0000 7f39b3fff640 1 --2- 172.21.15.164:0/262003135 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55be502df710 0x55be502e1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:25.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.759+0000 7f39b3fff640 1 --2- 172.21.15.164:0/262003135 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55be502df710 0x55be502e1b30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e0d942882bddd46 server_cookie=544a6a79f2009253 in_seq=0 out_seq=0 2024-09-19T08:55:25.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.759+0000 7f39937fe640 10 client.17517 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:55:25.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.759+0000 7f39937fe640 1 -- 172.21.15.164:0/262003135 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55be501c2d70 con 0x55be502df710 2024-09-19T08:55:25.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.759+0000 7f39937fe640 10 client.17517 handle_command_reply: tid=0 2024-09-19T08:55:25.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.759+0000 7f39bc76a740 1 client.17517 shutdown 2024-09-19T08:55:25.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.760+0000 7f39bc76a740 1 -- 172.21.15.164:0/262003135 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be50256b40 msgr2=0x55be50259780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:25.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.760+0000 7f39bc76a740 1 --2- 172.21.15.164:0/262003135 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be50256b40 0x55be50259780 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x55be50256070 tx=0x7f39a800b9f0 comp rx=0 tx=0).stop 2024-09-19T08:55:25.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.760+0000 7f39bc76a740 1 -- 172.21.15.164:0/262003135 shutdown_connections 2024-09-19T08:55:25.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.760+0000 7f39bc76a740 1 --2- 172.21.15.164:0/262003135 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be50260370 0x55be50259d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:25.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.760+0000 7f39bc76a740 1 --2- 172.21.15.164:0/262003135 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be50256b40 0x55be50259780 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:25.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.760+0000 7f39bc76a740 1 --2- 172.21.15.164:0/262003135 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55be502df710 0x55be502e1b30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:25.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.760+0000 7f39bc76a740 1 --2- 172.21.15.164:0/262003135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be502555d0 0x55be50259240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:25.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.760+0000 7f39937fe640 0 client.17517 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:55:25.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.760+0000 7f39bc76a740 1 -- 172.21.15.164:0/262003135 >> 172.21.15.164:0/262003135 conn(0x55be5021e310 msgr2=0x55be5025e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:25.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.760+0000 7f39bc76a740 1 -- 172.21.15.164:0/262003135 shutdown_connections 2024-09-19T08:55:25.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.760+0000 7f39bc76a740 1 -- 172.21.15.164:0/262003135 wait complete. 2024-09-19T08:55:25.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.760+0000 7f39bc76a740 20 client.17517 trim_cache size 0 max 16384 2024-09-19T08:55:25.784 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:55:25.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.760+0000 7f39ba391640 1 -- 172.21.15.164:0/3154497036 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f399c0a33b0 msgr2=0x7f399c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:25.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.760+0000 7f39ba391640 1 --2- 172.21.15.164:0/3154497036 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f399c0a33b0 0x7f399c0a5870 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f39ac005e30 tx=0x7f39ac02fe10 comp rx=0 tx=0).stop 2024-09-19T08:55:25.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.760+0000 7f39ba391640 1 -- 172.21.15.164:0/3154497036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39b4102da0 msgr2=0x7f39b41983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:25.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.760+0000 7f39ba391640 1 --2- 172.21.15.164:0/3154497036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39b4102da0 0x7f39b41983e0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f39a400d770 tx=0x7f39a400dc30 comp rx=0 tx=0).stop 2024-09-19T08:55:25.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.760+0000 7f39ba391640 1 -- 172.21.15.164:0/3154497036 shutdown_connections 2024-09-19T08:55:25.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.760+0000 7f39ba391640 1 --2- 172.21.15.164:0/3154497036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39b410c180 0x7f39b419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:25.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.760+0000 7f39ba391640 1 --2- 172.21.15.164:0/3154497036 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39b41036c0 0x7f39b4198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:25.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.760+0000 7f39ba391640 1 --2- 172.21.15.164:0/3154497036 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f399c0a33b0 0x7f399c0a5870 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:25.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.760+0000 7f39ba391640 1 --2- 172.21.15.164:0/3154497036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39b4102da0 0x7f39b41983e0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:25.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.760+0000 7f39ba391640 1 -- 172.21.15.164:0/3154497036 >> 172.21.15.164:0/3154497036 conn(0x7f39b40fc7a0 msgr2=0x7f39b40fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:25.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.761+0000 7f39ba391640 1 -- 172.21.15.164:0/3154497036 shutdown_connections 2024-09-19T08:55:25.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:25.761+0000 7f39ba391640 1 -- 172.21.15.164:0/3154497036 wait complete. 2024-09-19T08:55:26.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:26 smithi173 ceph-mon[35306]: pgmap v299: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:55:26.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:26 smithi164 ceph-mon[30928]: pgmap v299: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:55:26.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:26 smithi170 ceph-mon[35072]: pgmap v299: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:55:26.649 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T08:55:26.894 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:55:27.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.365+0000 7fb38d3fc640 1 -- 172.21.15.164:0/666092040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb388101420 msgr2=0x7fb38810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:27.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.365+0000 7fb38d3fc640 1 --2- 172.21.15.164:0/666092040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb388101420 0x7fb38810d950 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fb378009930 tx=0x7fb37802a420 comp rx=0 tx=0).stop 2024-09-19T08:55:27.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.365+0000 7fb38d3fc640 1 -- 172.21.15.164:0/666092040 shutdown_connections 2024-09-19T08:55:27.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.365+0000 7fb38d3fc640 1 --2- 172.21.15.164:0/666092040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb388101420 0x7fb38810d950 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.365+0000 7fb38d3fc640 1 --2- 172.21.15.164:0/666092040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb388100a80 0x7fb388100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.365+0000 7fb38d3fc640 1 --2- 172.21.15.164:0/666092040 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb388106a80 0x7fb388106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.365+0000 7fb38d3fc640 1 -- 172.21.15.164:0/666092040 >> 172.21.15.164:0/666092040 conn(0x7fb3880fc7a0 msgr2=0x7fb3880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:27.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.365+0000 7fb38d3fc640 1 -- 172.21.15.164:0/666092040 shutdown_connections 2024-09-19T08:55:27.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.365+0000 7fb38d3fc640 1 -- 172.21.15.164:0/666092040 wait complete. 2024-09-19T08:55:27.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.366+0000 7fb38d3fc640 1 Processor -- start 2024-09-19T08:55:27.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.366+0000 7fb38d3fc640 1 -- start start 2024-09-19T08:55:27.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.366+0000 7fb38d3fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb388100a80 0x7fb388198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:27.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.366+0000 7fb38d3fc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb388101420 0x7fb3881988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:27.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.366+0000 7fb38d3fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb388106a80 0x7fb38819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:27.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.366+0000 7fb38d3fc640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb388110e80 con 0x7fb388106a80 2024-09-19T08:55:27.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.366+0000 7fb38d3fc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb388110d00 con 0x7fb388100a80 2024-09-19T08:55:27.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.366+0000 7fb38d3fc640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb388111000 con 0x7fb388101420 2024-09-19T08:55:27.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.366+0000 7fb3877fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb388106a80 0x7fb38819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:27.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.366+0000 7fb386ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb388100a80 0x7fb388198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:27.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.366+0000 7fb3877fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb388106a80 0x7fb38819cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40818/0 (socket says 172.21.15.164:40818) 2024-09-19T08:55:27.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.366+0000 7fb3877fe640 1 -- 172.21.15.164:0/3624823279 learned_addr learned my addr 172.21.15.164:0/3624823279 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:27.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.366+0000 7fb3867fc640 1 --2- 172.21.15.164:0/3624823279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb388101420 0x7fb3881988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:27.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.366+0000 7fb3877fe640 1 -- 172.21.15.164:0/3624823279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb388101420 msgr2=0x7fb3881988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:27.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.366+0000 7fb3877fe640 1 --2- 172.21.15.164:0/3624823279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb388101420 0x7fb3881988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.366+0000 7fb3877fe640 1 -- 172.21.15.164:0/3624823279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb388100a80 msgr2=0x7fb388198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:27.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.366+0000 7fb3877fe640 1 --2- 172.21.15.164:0/3624823279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb388100a80 0x7fb388198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.366+0000 7fb3877fe640 1 -- 172.21.15.164:0/3624823279 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb38819d310 con 0x7fb388106a80 2024-09-19T08:55:27.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.367+0000 7fb3877fe640 1 --2- 172.21.15.164:0/3624823279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb388106a80 0x7fb38819cc30 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fb378005ab0 tx=0x7fb3780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:27.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.367+0000 7fb37ffff640 1 -- 172.21.15.164:0/3624823279 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37803f030 con 0x7fb388106a80 2024-09-19T08:55:27.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.367+0000 7fb37ffff640 1 -- 172.21.15.164:0/3624823279 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb378003ce0 con 0x7fb388106a80 2024-09-19T08:55:27.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.367+0000 7fb38d3fc640 1 -- 172.21.15.164:0/3624823279 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb38819d5a0 con 0x7fb388106a80 2024-09-19T08:55:27.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.367+0000 7fb37ffff640 1 -- 172.21.15.164:0/3624823279 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb378035070 con 0x7fb388106a80 2024-09-19T08:55:27.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.367+0000 7fb38d3fc640 1 -- 172.21.15.164:0/3624823279 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3881a4ee0 con 0x7fb388106a80 2024-09-19T08:55:27.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.369+0000 7fb37ffff640 1 -- 172.21.15.164:0/3624823279 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fb378042030 con 0x7fb388106a80 2024-09-19T08:55:27.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.370+0000 7fb38f7d5740 1 Processor -- start 2024-09-19T08:55:27.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.370+0000 7fb38f7d5740 1 -- start start 2024-09-19T08:55:27.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.370+0000 7fb37ffff640 1 --2- 172.21.15.164:0/3624823279 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb3740a33b0 0x7fb3740a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:27.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.370+0000 7fb3867fc640 1 --2- 172.21.15.164:0/3624823279 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb3740a33b0 0x7fb3740a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:27.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.370+0000 7fb37ffff640 1 -- 172.21.15.164:0/3624823279 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fb3780f0550 con 0x7fb388106a80 2024-09-19T08:55:27.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.370+0000 7fb38f7d5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cb9353b40 0x561cb9352090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:27.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.370+0000 7fb38f7d5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561cb93525d0 0x561cb9352a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:27.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.370+0000 7fb38f7d5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb935d370 0x561cb935f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:27.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.370+0000 7fb38f7d5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561cb92b82f0 con 0x561cb935d370 2024-09-19T08:55:27.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.370+0000 7fb38f7d5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561cb925fcf0 con 0x561cb9353b40 2024-09-19T08:55:27.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.370+0000 7fb38f7d5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x561cb92b42b0 con 0x561cb93525d0 2024-09-19T08:55:27.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.370+0000 7fb3877fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb935d370 0x561cb935f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:27.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.370+0000 7fb3877fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb935d370 0x561cb935f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40832/0 (socket says 172.21.15.164:40832) 2024-09-19T08:55:27.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.370+0000 7fb3877fe640 1 -- 172.21.15.164:0/3124241840 learned_addr learned my addr 172.21.15.164:0/3124241840 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:27.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.370+0000 7fb386ffd640 1 --2- 172.21.15.164:0/3124241840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cb9353b40 0x561cb9352090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:27.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.371+0000 7fb3867fc640 1 --2- 172.21.15.164:0/3124241840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561cb93525d0 0x561cb9352a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:27.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.371+0000 7fb3877fe640 1 -- 172.21.15.164:0/3124241840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561cb93525d0 msgr2=0x561cb9352a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:27.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.371+0000 7fb3877fe640 1 --2- 172.21.15.164:0/3124241840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561cb93525d0 0x561cb9352a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.371+0000 7fb3877fe640 1 -- 172.21.15.164:0/3124241840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cb9353b40 msgr2=0x561cb9352090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:27.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.371+0000 7fb3877fe640 1 --2- 172.21.15.164:0/3124241840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cb9353b40 0x561cb9352090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.371+0000 7fb3877fe640 1 -- 172.21.15.164:0/3124241840 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cb92443a0 con 0x561cb935d370 2024-09-19T08:55:27.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.371+0000 7fb3867fc640 1 --2- 172.21.15.164:0/3124241840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561cb93525d0 0x561cb9352a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:55:27.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.371+0000 7fb3867fc640 1 --2- 172.21.15.164:0/3624823279 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb3740a33b0 0x7fb3740a5870 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb38806b870 tx=0x7fb37000f3c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:27.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.371+0000 7fb3877fe640 1 --2- 172.21.15.164:0/3124241840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb935d370 0x561cb935f760 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fb3780c4fb0 tx=0x7fb37803b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99bc0a8329bebdc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:27.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.371+0000 7fb37d7fa640 1 -- 172.21.15.164:0/3124241840 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb378049070 con 0x561cb935d370 2024-09-19T08:55:27.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.371+0000 7fb37d7fa640 1 -- 172.21.15.164:0/3124241840 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3780bcf70 con 0x561cb935d370 2024-09-19T08:55:27.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.371+0000 7fb37d7fa640 1 -- 172.21.15.164:0/3124241840 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3780edf60 con 0x561cb935d370 2024-09-19T08:55:27.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.372+0000 7fb38f7d5740 1 -- 172.21.15.164:0/3124241840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb935d370 msgr2=0x561cb935f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:27.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.372+0000 7fb38f7d5740 1 --2- 172.21.15.164:0/3124241840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb935d370 0x561cb935f760 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fb3780c4fb0 tx=0x7fb37803b460 comp rx=0 tx=0).stop 2024-09-19T08:55:27.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.372+0000 7fb38f7d5740 1 -- 172.21.15.164:0/3124241840 shutdown_connections 2024-09-19T08:55:27.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.372+0000 7fb38f7d5740 1 --2- 172.21.15.164:0/3124241840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb935d370 0x561cb935f760 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.372+0000 7fb38f7d5740 1 --2- 172.21.15.164:0/3124241840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561cb93525d0 0x561cb9352a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.372+0000 7fb38f7d5740 1 --2- 172.21.15.164:0/3124241840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cb9353b40 0x561cb9352090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.372+0000 7fb38f7d5740 1 -- 172.21.15.164:0/3124241840 >> 172.21.15.164:0/3124241840 conn(0x561cb931b310 msgr2=0x561cb934ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:27.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.372+0000 7fb38f7d5740 1 -- 172.21.15.164:0/3124241840 shutdown_connections 2024-09-19T08:55:27.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.372+0000 7fb38f7d5740 1 -- 172.21.15.164:0/3124241840 wait complete. 2024-09-19T08:55:27.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.372+0000 7fb38f7d5740 1 -- start start 2024-09-19T08:55:27.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.372+0000 7fb38f7d5740 1 Processor -- start 2024-09-19T08:55:27.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.373+0000 7fb38f7d5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb93525d0 0x561cb935cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:27.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.373+0000 7fb38f7d5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561cb9353b40 0x561cb935a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:27.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.373+0000 7fb38f7d5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cb935d370 0x561cb935a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:27.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.373+0000 7fb38f7d5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561cb92b82f0 con 0x561cb93525d0 2024-09-19T08:55:27.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.373+0000 7fb38f7d5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561cb9273060 con 0x561cb935d370 2024-09-19T08:55:27.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.373+0000 7fb38f7d5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x561cb935fe30 con 0x561cb9353b40 2024-09-19T08:55:27.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.373+0000 7fb38f7d5740 10 client.0 _ll_register_callbacks cb 0x561cb8fc68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:55:27.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.373+0000 7fb386ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb93525d0 0x561cb935cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:27.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.373+0000 7fb386ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb93525d0 0x561cb935cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40836/0 (socket says 172.21.15.164:40836) 2024-09-19T08:55:27.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.373+0000 7fb386ffd640 1 -- 172.21.15.164:0/2389728062 learned_addr learned my addr 172.21.15.164:0/2389728062 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:27.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.373+0000 7fb3877fe640 1 --2- 172.21.15.164:0/2389728062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cb935d370 0x561cb935a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:27.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.373+0000 7fb386ffd640 1 -- 172.21.15.164:0/2389728062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561cb9353b40 msgr2=0x561cb935a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:27.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.373+0000 7fb3867fc640 1 --2- 172.21.15.164:0/2389728062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561cb9353b40 0x561cb935a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:27.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.373+0000 7fb386ffd640 1 --2- 172.21.15.164:0/2389728062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561cb9353b40 0x561cb935a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.373+0000 7fb386ffd640 1 -- 172.21.15.164:0/2389728062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cb935d370 msgr2=0x561cb935a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:27.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.373+0000 7fb386ffd640 1 --2- 172.21.15.164:0/2389728062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cb935d370 0x561cb935a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.373+0000 7fb386ffd640 1 -- 172.21.15.164:0/2389728062 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cb90293c0 con 0x561cb93525d0 2024-09-19T08:55:27.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.373+0000 7fb386ffd640 1 --2- 172.21.15.164:0/2389728062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb93525d0 0x561cb935cd80 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fb380010b70 tx=0x7fb38000f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60c789b0154f5bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:27.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.373+0000 7fb36e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:55:27.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.373+0000 7fb36e7fc640 1 -- 172.21.15.164:0/2389728062 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb380004b20 con 0x561cb93525d0 2024-09-19T08:55:27.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.373+0000 7fb36e7fc640 1 -- 172.21.15.164:0/2389728062 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fb380004cc0 con 0x561cb93525d0 2024-09-19T08:55:27.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.373+0000 7fb38f7d5740 1 -- 172.21.15.164:0/2389728062 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3780bcf70 con 0x561cb93525d0 2024-09-19T08:55:27.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.374+0000 7fb36e7fc640 1 -- 172.21.15.164:0/2389728062 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb38000e5c0 con 0x561cb93525d0 2024-09-19T08:55:27.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.374+0000 7fb38f7d5740 10 client.17529 fetch_fsmap learned FSMap version 32 2024-09-19T08:55:27.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.374+0000 7fb38f7d5740 1 -- 172.21.15.164:0/2389728062 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561cb936b7e0 con 0x561cb93525d0 2024-09-19T08:55:27.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.374+0000 7fb36e7fc640 1 -- 172.21.15.164:0/2389728062 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3780bcf70 con 0x561cb93525d0 2024-09-19T08:55:27.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.374+0000 7fb36e7fc640 1 -- 172.21.15.164:0/2389728062 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb380019900 con 0x561cb93525d0 2024-09-19T08:55:27.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.374+0000 7fb38f7d5740 10 client.17529 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:55:27.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.374+0000 7fb38f7d5740 10 client.17529 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:55:27.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.374+0000 7fb38f7d5740 20 client.17529 populate_metadata read hostname 'smithi164' 2024-09-19T08:55:27.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.374+0000 7fb38f7d5740 1 --2- 172.21.15.164:0/2389728062 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x561cb936cd90 0x561cb936f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:27.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.374+0000 7fb38f7d5740 4 client.17529 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:55:27.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.374+0000 7fb38f7d5740 1 -- 172.21.15.164:0/2389728062 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cb92c5860 con 0x561cb936cd90 2024-09-19T08:55:27.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.374+0000 7fb3867fc640 1 --2- 172.21.15.164:0/2389728062 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x561cb936cd90 0x561cb936f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:27.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.375+0000 7fb3867fc640 1 --2- 172.21.15.164:0/2389728062 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x561cb936cd90 0x561cb936f1f0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd71aec20f4d2d4d server_cookie=b946c390c48a8627 in_seq=0 out_seq=0 2024-09-19T08:55:27.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.375+0000 7fb36e7fc640 10 client.17529 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:55:27.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.375+0000 7fb36e7fc640 1 -- 172.21.15.164:0/2389728062 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561cb92c5860 con 0x561cb936cd90 2024-09-19T08:55:27.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.375+0000 7fb36e7fc640 10 client.17529 handle_command_reply: tid=0 2024-09-19T08:55:27.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.375+0000 7fb38f7d5740 1 client.17529 shutdown 2024-09-19T08:55:27.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.375+0000 7fb38f7d5740 1 -- 172.21.15.164:0/2389728062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb93525d0 msgr2=0x561cb935cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:27.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.375+0000 7fb38f7d5740 1 --2- 172.21.15.164:0/2389728062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb93525d0 0x561cb935cd80 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fb380010b70 tx=0x7fb38000f7f0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.375+0000 7fb38f7d5740 1 -- 172.21.15.164:0/2389728062 shutdown_connections 2024-09-19T08:55:27.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.375+0000 7fb38f7d5740 1 --2- 172.21.15.164:0/2389728062 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x561cb936cd90 0x561cb936f1f0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.375+0000 7fb38f7d5740 1 --2- 172.21.15.164:0/2389728062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cb935d370 0x561cb935a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.375+0000 7fb38f7d5740 1 --2- 172.21.15.164:0/2389728062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561cb9353b40 0x561cb935a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.375+0000 7fb38f7d5740 1 --2- 172.21.15.164:0/2389728062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb93525d0 0x561cb935cd80 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.375+0000 7fb38f7d5740 1 -- 172.21.15.164:0/2389728062 >> 172.21.15.164:0/2389728062 conn(0x561cb931b310 msgr2=0x561cb9354990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:27.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.375+0000 7fb36e7fc640 0 client.17529 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:55:27.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.376+0000 7fb38f7d5740 1 -- 172.21.15.164:0/2389728062 shutdown_connections 2024-09-19T08:55:27.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.376+0000 7fb38f7d5740 1 -- 172.21.15.164:0/2389728062 wait complete. 2024-09-19T08:55:27.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.376+0000 7fb38f7d5740 20 client.17529 trim_cache size 0 max 16384 2024-09-19T08:55:27.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.388+0000 7fb38f7d5740 1 Processor -- start 2024-09-19T08:55:27.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.388+0000 7fb38f7d5740 1 -- start start 2024-09-19T08:55:27.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.389+0000 7fb38f7d5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cb93525d0 0x561cb9368f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:27.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.389+0000 7fb38f7d5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561cb9353b40 0x561cb93598f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:27.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.389+0000 7fb38f7d5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb935d370 0x561cb934e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:27.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.389+0000 7fb38f7d5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb3881ad710 con 0x561cb935d370 2024-09-19T08:55:27.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.389+0000 7fb38f7d5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb3881af250 con 0x561cb93525d0 2024-09-19T08:55:27.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.389+0000 7fb38f7d5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb3881adde0 con 0x561cb9353b40 2024-09-19T08:55:27.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.389+0000 7fb3877fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb935d370 0x561cb934e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:27.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.389+0000 7fb3877fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb935d370 0x561cb934e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40838/0 (socket says 172.21.15.164:40838) 2024-09-19T08:55:27.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.389+0000 7fb3877fe640 1 -- 172.21.15.164:0/2054680329 learned_addr learned my addr 172.21.15.164:0/2054680329 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:27.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.389+0000 7fb386ffd640 1 --2- 172.21.15.164:0/2054680329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cb93525d0 0x561cb9368f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:27.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.389+0000 7fb3867fc640 1 --2- 172.21.15.164:0/2054680329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561cb9353b40 0x561cb93598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:27.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.389+0000 7fb3877fe640 1 -- 172.21.15.164:0/2054680329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561cb9353b40 msgr2=0x561cb93598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:27.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.389+0000 7fb3877fe640 1 --2- 172.21.15.164:0/2054680329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561cb9353b40 0x561cb93598f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.389+0000 7fb3877fe640 1 -- 172.21.15.164:0/2054680329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cb93525d0 msgr2=0x561cb9368f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:27.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.389+0000 7fb3877fe640 1 --2- 172.21.15.164:0/2054680329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cb93525d0 0x561cb9368f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.389+0000 7fb3877fe640 1 -- 172.21.15.164:0/2054680329 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cb935afd0 con 0x561cb935d370 2024-09-19T08:55:27.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.389+0000 7fb3867fc640 1 --2- 172.21.15.164:0/2054680329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561cb9353b40 0x561cb93598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:55:27.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.389+0000 7fb3877fe640 1 --2- 172.21.15.164:0/2054680329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb935d370 0x561cb934e700 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fb378006010 tx=0x7fb37803b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b8f525eeea5860 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:27.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.390+0000 7fb37cff9640 1 -- 172.21.15.164:0/2054680329 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3780da070 con 0x561cb935d370 2024-09-19T08:55:27.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.390+0000 7fb37cff9640 1 -- 172.21.15.164:0/2054680329 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3780bcf70 con 0x561cb935d370 2024-09-19T08:55:27.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.390+0000 7fb37cff9640 1 -- 172.21.15.164:0/2054680329 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb378049040 con 0x561cb935d370 2024-09-19T08:55:27.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.390+0000 7fb38f7d5740 1 -- 172.21.15.164:0/2054680329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb935d370 msgr2=0x561cb934e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:27.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.390+0000 7fb38f7d5740 1 --2- 172.21.15.164:0/2054680329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb935d370 0x561cb934e700 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fb378006010 tx=0x7fb37803b600 comp rx=0 tx=0).stop 2024-09-19T08:55:27.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.390+0000 7fb38f7d5740 1 -- 172.21.15.164:0/2054680329 shutdown_connections 2024-09-19T08:55:27.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.390+0000 7fb38f7d5740 1 --2- 172.21.15.164:0/2054680329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb935d370 0x561cb934e700 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.390+0000 7fb38f7d5740 1 --2- 172.21.15.164:0/2054680329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561cb9353b40 0x561cb93598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.390+0000 7fb38f7d5740 1 --2- 172.21.15.164:0/2054680329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cb93525d0 0x561cb9368f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.390+0000 7fb38f7d5740 1 -- 172.21.15.164:0/2054680329 >> 172.21.15.164:0/2054680329 conn(0x561cb931b310 msgr2=0x561cb93552c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:27.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.390+0000 7fb38f7d5740 1 -- 172.21.15.164:0/2054680329 shutdown_connections 2024-09-19T08:55:27.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.390+0000 7fb38f7d5740 1 -- 172.21.15.164:0/2054680329 wait complete. 2024-09-19T08:55:27.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.391+0000 7fb38f7d5740 1 -- start start 2024-09-19T08:55:27.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.391+0000 7fb38f7d5740 1 Processor -- start 2024-09-19T08:55:27.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.391+0000 7fb38f7d5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cb93525d0 0x561cb9360620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:27.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.391+0000 7fb38f7d5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561cb9353b40 0x561cb9360b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:27.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.391+0000 7fb38f7d5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb935d370 0x561cb93d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:27.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.391+0000 7fb38f7d5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb3881aeb80 con 0x561cb935d370 2024-09-19T08:55:27.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.391+0000 7fb38f7d5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb3881ae4b0 con 0x561cb93525d0 2024-09-19T08:55:27.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.391+0000 7fb38f7d5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb3881ab710 con 0x561cb9353b40 2024-09-19T08:55:27.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.391+0000 7fb38f7d5740 10 client.0 _ll_register_callbacks cb 0x7fb38000aa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:55:27.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.391+0000 7fb3877fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb935d370 0x561cb93d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:27.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.391+0000 7fb3877fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb935d370 0x561cb93d5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40854/0 (socket says 172.21.15.164:40854) 2024-09-19T08:55:27.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.391+0000 7fb386ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cb93525d0 0x561cb9360620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:27.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.391+0000 7fb3877fe640 1 -- 172.21.15.164:0/3152956852 learned_addr learned my addr 172.21.15.164:0/3152956852 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:27.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.391+0000 7fb3867fc640 1 --2- 172.21.15.164:0/3152956852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561cb9353b40 0x561cb9360b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:27.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.391+0000 7fb3877fe640 1 -- 172.21.15.164:0/3152956852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561cb9353b40 msgr2=0x561cb9360b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:27.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.391+0000 7fb3877fe640 1 --2- 172.21.15.164:0/3152956852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561cb9353b40 0x561cb9360b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.391+0000 7fb3877fe640 1 -- 172.21.15.164:0/3152956852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cb93525d0 msgr2=0x561cb9360620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:27.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.391+0000 7fb3877fe640 1 --2- 172.21.15.164:0/3152956852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cb93525d0 0x561cb9360620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.391+0000 7fb3877fe640 1 -- 172.21.15.164:0/3152956852 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cb93d5fc0 con 0x561cb935d370 2024-09-19T08:55:27.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.392+0000 7fb3877fe640 1 --2- 172.21.15.164:0/3152956852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb935d370 0x561cb93d5a80 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fb378009900 tx=0x7fb3780ee190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0cc322d38b6de42 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:27.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.392+0000 7fb36e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:55:27.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.392+0000 7fb36e7fc640 1 -- 172.21.15.164:0/3152956852 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3780da070 con 0x561cb935d370 2024-09-19T08:55:27.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.392+0000 7fb36e7fc640 1 -- 172.21.15.164:0/3152956852 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fb3780b8ee0 con 0x561cb935d370 2024-09-19T08:55:27.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.392+0000 7fb38f7d5740 1 -- 172.21.15.164:0/3152956852 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3780bcf70 con 0x561cb935d370 2024-09-19T08:55:27.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.392+0000 7fb36e7fc640 1 -- 172.21.15.164:0/3152956852 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb3780d4070 con 0x561cb935d370 2024-09-19T08:55:27.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.392+0000 7fb38f7d5740 10 client.17535 fetch_fsmap learned FSMap version 32 2024-09-19T08:55:27.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.392+0000 7fb38f7d5740 1 -- 172.21.15.164:0/3152956852 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561cb93d7270 con 0x561cb935d370 2024-09-19T08:55:27.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.392+0000 7fb36e7fc640 1 -- 172.21.15.164:0/3152956852 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3780bcf70 con 0x561cb935d370 2024-09-19T08:55:27.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.392+0000 7fb36e7fc640 1 -- 172.21.15.164:0/3152956852 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb3780a2080 con 0x561cb935d370 2024-09-19T08:55:27.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.392+0000 7fb38f7d5740 10 client.17535 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:55:27.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.392+0000 7fb38f7d5740 10 client.17535 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:55:27.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.392+0000 7fb38f7d5740 20 client.17535 populate_metadata read hostname 'smithi164' 2024-09-19T08:55:27.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.392+0000 7fb38f7d5740 1 --2- 172.21.15.164:0/3152956852 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x561cb93d7600 0x561cb93d99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:27.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.392+0000 7fb38f7d5740 4 client.17535 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:55:27.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.392+0000 7fb38f7d5740 1 -- 172.21.15.164:0/3152956852 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561cb92bfd70 con 0x561cb93d7600 2024-09-19T08:55:27.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.393+0000 7fb386ffd640 1 --2- 172.21.15.164:0/3152956852 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x561cb93d7600 0x561cb93d99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:27.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.393+0000 7fb386ffd640 1 --2- 172.21.15.164:0/3152956852 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x561cb93d7600 0x561cb93d99f0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94000288c94f39f5 server_cookie=93e5d555dc9d101c in_seq=0 out_seq=0 2024-09-19T08:55:27.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.393+0000 7fb36e7fc640 10 client.17535 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:55:27.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.393+0000 7fb36e7fc640 1 -- 172.21.15.164:0/3152956852 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561cb92bfd70 con 0x561cb93d7600 2024-09-19T08:55:27.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.393+0000 7fb36e7fc640 10 client.17535 handle_command_reply: tid=0 2024-09-19T08:55:27.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.393+0000 7fb38f7d5740 1 client.17535 shutdown 2024-09-19T08:55:27.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.393+0000 7fb38f7d5740 1 -- 172.21.15.164:0/3152956852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb935d370 msgr2=0x561cb93d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:27.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.393+0000 7fb38f7d5740 1 --2- 172.21.15.164:0/3152956852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb935d370 0x561cb93d5a80 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fb378009900 tx=0x7fb3780ee190 comp rx=0 tx=0).stop 2024-09-19T08:55:27.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.393+0000 7fb38f7d5740 1 -- 172.21.15.164:0/3152956852 shutdown_connections 2024-09-19T08:55:27.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.393+0000 7fb38f7d5740 1 --2- 172.21.15.164:0/3152956852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561cb935d370 0x561cb93d5a80 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.393+0000 7fb38f7d5740 1 --2- 172.21.15.164:0/3152956852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561cb9353b40 0x561cb9360b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.393+0000 7fb38f7d5740 1 --2- 172.21.15.164:0/3152956852 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x561cb93d7600 0x561cb93d99f0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.393+0000 7fb38f7d5740 1 --2- 172.21.15.164:0/3152956852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cb93525d0 0x561cb9360620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.393+0000 7fb38f7d5740 1 -- 172.21.15.164:0/3152956852 >> 172.21.15.164:0/3152956852 conn(0x561cb931b310 msgr2=0x561cb934e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:27.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.393+0000 7fb36e7fc640 0 client.17535 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:55:27.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.394+0000 7fb38f7d5740 1 -- 172.21.15.164:0/3152956852 shutdown_connections 2024-09-19T08:55:27.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.394+0000 7fb38f7d5740 1 -- 172.21.15.164:0/3152956852 wait complete. 2024-09-19T08:55:27.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.394+0000 7fb38f7d5740 20 client.17535 trim_cache size 0 max 16384 2024-09-19T08:55:27.402 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:55:27.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.394+0000 7fb38d3fc640 1 -- 172.21.15.164:0/3624823279 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb3740a33b0 msgr2=0x7fb3740a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:27.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.394+0000 7fb38d3fc640 1 --2- 172.21.15.164:0/3624823279 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb3740a33b0 0x7fb3740a5870 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb38806b870 tx=0x7fb37000f3c0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.394+0000 7fb38d3fc640 1 -- 172.21.15.164:0/3624823279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb388106a80 msgr2=0x7fb38819cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:27.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.394+0000 7fb38d3fc640 1 --2- 172.21.15.164:0/3624823279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb388106a80 0x7fb38819cc30 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fb378005ab0 tx=0x7fb3780042f0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.394+0000 7fb38d3fc640 1 -- 172.21.15.164:0/3624823279 shutdown_connections 2024-09-19T08:55:27.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.394+0000 7fb38d3fc640 1 --2- 172.21.15.164:0/3624823279 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb3740a33b0 0x7fb3740a5870 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.394+0000 7fb38d3fc640 1 --2- 172.21.15.164:0/3624823279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb388106a80 0x7fb38819cc30 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.394+0000 7fb38d3fc640 1 --2- 172.21.15.164:0/3624823279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb388101420 0x7fb3881988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.394+0000 7fb38d3fc640 1 --2- 172.21.15.164:0/3624823279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb388100a80 0x7fb388198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:27.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.394+0000 7fb38d3fc640 1 -- 172.21.15.164:0/3624823279 >> 172.21.15.164:0/3624823279 conn(0x7fb3880fc7a0 msgr2=0x7fb388104790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:27.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.394+0000 7fb38d3fc640 1 -- 172.21.15.164:0/3624823279 shutdown_connections 2024-09-19T08:55:27.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:27.394+0000 7fb38d3fc640 1 -- 172.21.15.164:0/3624823279 wait complete. 2024-09-19T08:55:28.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:28 smithi173 ceph-mon[35306]: pgmap v300: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:55:28.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:28 smithi164 ceph-mon[30928]: pgmap v300: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:55:28.624 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:28 smithi170 ceph-mon[35072]: pgmap v300: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:55:29.101 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T08:55:29.101 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T08:55:29.370 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:55:29.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.860+0000 7fc2acd33640 1 -- 172.21.15.164:0/3467385542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2a8102d30 msgr2=0x7fc2a8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:29.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.860+0000 7fc2acd33640 1 --2- 172.21.15.164:0/3467385542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2a8102d30 0x7fc2a8103110 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fc298009cd0 tx=0x7fc29802a8e0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.860+0000 7fc2acd33640 1 -- 172.21.15.164:0/3467385542 shutdown_connections 2024-09-19T08:55:29.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.860+0000 7fc2acd33640 1 --2- 172.21.15.164:0/3467385542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2a81082a0 0x7fc2a810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.860+0000 7fc2acd33640 1 --2- 172.21.15.164:0/3467385542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2a81036e0 0x7fc2a8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.860+0000 7fc2acd33640 1 --2- 172.21.15.164:0/3467385542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2a8102d30 0x7fc2a8103110 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.860+0000 7fc2acd33640 1 -- 172.21.15.164:0/3467385542 >> 172.21.15.164:0/3467385542 conn(0x7fc2a80fc7d0 msgr2=0x7fc2a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:29.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.860+0000 7fc2acd33640 1 -- 172.21.15.164:0/3467385542 shutdown_connections 2024-09-19T08:55:29.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.860+0000 7fc2acd33640 1 -- 172.21.15.164:0/3467385542 wait complete. 2024-09-19T08:55:29.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.861+0000 7fc2acd33640 1 Processor -- start 2024-09-19T08:55:29.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.861+0000 7fc2acd33640 1 -- start start 2024-09-19T08:55:29.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.861+0000 7fc2acd33640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2a8102d30 0x7fc2a81985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:29.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.861+0000 7fc2acd33640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2a81036e0 0x7fc2a8198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:29.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.861+0000 7fc2acd33640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2a81082a0 0x7fc2a819cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:29.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.861+0000 7fc2acd33640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a81120a0 con 0x7fc2a81036e0 2024-09-19T08:55:29.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.861+0000 7fc2acd33640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a8111f20 con 0x7fc2a8102d30 2024-09-19T08:55:29.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.861+0000 7fc2acd33640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a8112220 con 0x7fc2a81082a0 2024-09-19T08:55:29.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.861+0000 7fc2a5d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2a81036e0 0x7fc2a8198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:29.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.861+0000 7fc2a5d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2a81036e0 0x7fc2a8198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40862/0 (socket says 172.21.15.164:40862) 2024-09-19T08:55:29.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.861+0000 7fc2a5d9b640 1 -- 172.21.15.164:0/2943113361 learned_addr learned my addr 172.21.15.164:0/2943113361 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:29.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.861+0000 7fc2a6d9d640 1 --2- 172.21.15.164:0/2943113361 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2a81082a0 0x7fc2a819cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:29.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.861+0000 7fc2a659c640 1 --2- 172.21.15.164:0/2943113361 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2a8102d30 0x7fc2a81985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:29.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.861+0000 7fc2a5d9b640 1 -- 172.21.15.164:0/2943113361 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2a81082a0 msgr2=0x7fc2a819cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:29.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.861+0000 7fc2a5d9b640 1 --2- 172.21.15.164:0/2943113361 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2a81082a0 0x7fc2a819cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.861+0000 7fc2a5d9b640 1 -- 172.21.15.164:0/2943113361 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2a8102d30 msgr2=0x7fc2a81985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:29.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.861+0000 7fc2a5d9b640 1 --2- 172.21.15.164:0/2943113361 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2a8102d30 0x7fc2a81985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.861+0000 7fc2a5d9b640 1 -- 172.21.15.164:0/2943113361 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a819d5a0 con 0x7fc2a81036e0 2024-09-19T08:55:29.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.861+0000 7fc2a659c640 1 --2- 172.21.15.164:0/2943113361 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2a8102d30 0x7fc2a81985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:55:29.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.861+0000 7fc2a5d9b640 1 --2- 172.21.15.164:0/2943113361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2a81036e0 0x7fc2a8198b30 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00d740 tx=0x7fc29c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:29.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.862+0000 7fc2977fe640 1 -- 172.21.15.164:0/2943113361 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c017070 con 0x7fc2a81036e0 2024-09-19T08:55:29.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.862+0000 7fc2acd33640 1 -- 172.21.15.164:0/2943113361 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2a819d890 con 0x7fc2a81036e0 2024-09-19T08:55:29.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.862+0000 7fc2977fe640 1 -- 172.21.15.164:0/2943113361 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc29c00de50 con 0x7fc2a81036e0 2024-09-19T08:55:29.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.862+0000 7fc2977fe640 1 -- 172.21.15.164:0/2943113361 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c00f6b0 con 0x7fc2a81036e0 2024-09-19T08:55:29.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.862+0000 7fc2acd33640 1 -- 172.21.15.164:0/2943113361 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2a81a5170 con 0x7fc2a81036e0 2024-09-19T08:55:29.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.864+0000 7fc2977fe640 1 -- 172.21.15.164:0/2943113361 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fc29c01a070 con 0x7fc2a81036e0 2024-09-19T08:55:29.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.864+0000 7fc2af10c740 1 Processor -- start 2024-09-19T08:55:29.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.864+0000 7fc2af10c740 1 -- start start 2024-09-19T08:55:29.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.865+0000 7fc2af10c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dcc0d9fb60 0x55dcc0d9ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:29.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.865+0000 7fc2af10c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dcc0da0480 0x55dcc0daa800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:29.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.865+0000 7fc2af10c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcc0daad40 0x55dcc0dad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:29.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.865+0000 7fc2af10c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55dcc0d05280 con 0x55dcc0da0480 2024-09-19T08:55:29.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.865+0000 7fc2af10c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55dcc0cacb10 con 0x55dcc0daad40 2024-09-19T08:55:29.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.865+0000 7fc2af10c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55dcc0d01240 con 0x55dcc0d9fb60 2024-09-19T08:55:29.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.865+0000 7fc2a659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dcc0da0480 0x55dcc0daa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:29.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.865+0000 7fc2a659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dcc0da0480 0x55dcc0daa800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40868/0 (socket says 172.21.15.164:40868) 2024-09-19T08:55:29.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.865+0000 7fc2a659c640 1 -- 172.21.15.164:0/2592022651 learned_addr learned my addr 172.21.15.164:0/2592022651 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:29.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.865+0000 7fc2977fe640 1 --2- 172.21.15.164:0/2943113361 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fc2900a33b0 0x7fc2900a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:29.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.865+0000 7fc2a6d9d640 1 --2- 172.21.15.164:0/2592022651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dcc0d9fb60 0x55dcc0d9ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:29.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.865+0000 7fc2a5d9b640 1 --2- 172.21.15.164:0/2592022651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcc0daad40 0x55dcc0dad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:29.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.865+0000 7fc2a659c640 1 -- 172.21.15.164:0/2592022651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dcc0d9fb60 msgr2=0x55dcc0d9ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:29.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.865+0000 7fc2a659c640 1 --2- 172.21.15.164:0/2592022651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dcc0d9fb60 0x55dcc0d9ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.865+0000 7fc2a659c640 1 -- 172.21.15.164:0/2592022651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcc0daad40 msgr2=0x55dcc0dad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:29.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.865+0000 7fc2a659c640 1 --2- 172.21.15.164:0/2592022651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcc0daad40 0x55dcc0dad130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.865+0000 7fc2a659c640 1 -- 172.21.15.164:0/2592022651 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcc0cb91f0 con 0x55dcc0da0480 2024-09-19T08:55:29.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.865+0000 7fc2a6d9d640 1 --2- 172.21.15.164:0/2592022651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dcc0d9fb60 0x55dcc0d9ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:55:29.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.865+0000 7fc2a5d9b640 1 --2- 172.21.15.164:0/2592022651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcc0daad40 0x55dcc0dad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:55:29.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.865+0000 7fc2a6d9d640 1 --2- 172.21.15.164:0/2943113361 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fc2900a33b0 0x7fc2900a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:29.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.865+0000 7fc2977fe640 1 -- 172.21.15.164:0/2943113361 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fc29c0cf6e0 con 0x7fc2a81036e0 2024-09-19T08:55:29.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.865+0000 7fc2a659c640 1 --2- 172.21.15.164:0/2592022651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dcc0da0480 0x55dcc0daa800 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fc298035000 tx=0x7fc2980315e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c9f1b91b15c79 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:29.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.866+0000 7fc294ff9640 1 -- 172.21.15.164:0/2592022651 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc298031ac0 con 0x55dcc0da0480 2024-09-19T08:55:29.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.866+0000 7fc294ff9640 1 -- 172.21.15.164:0/2592022651 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc298036040 con 0x55dcc0da0480 2024-09-19T08:55:29.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.866+0000 7fc294ff9640 1 -- 172.21.15.164:0/2592022651 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc298004810 con 0x55dcc0da0480 2024-09-19T08:55:29.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.866+0000 7fc2a6d9d640 1 --2- 172.21.15.164:0/2943113361 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fc2900a33b0 0x7fc2900a5870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0005fd0 tx=0x7fc2a00046c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:29.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.866+0000 7fc2af10c740 1 -- 172.21.15.164:0/2592022651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dcc0da0480 msgr2=0x55dcc0daa800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:29.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.866+0000 7fc2af10c740 1 --2- 172.21.15.164:0/2592022651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dcc0da0480 0x55dcc0daa800 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fc298035000 tx=0x7fc2980315e0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.866+0000 7fc2af10c740 1 -- 172.21.15.164:0/2592022651 shutdown_connections 2024-09-19T08:55:29.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.866+0000 7fc2af10c740 1 --2- 172.21.15.164:0/2592022651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcc0daad40 0x55dcc0dad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.866+0000 7fc2af10c740 1 --2- 172.21.15.164:0/2592022651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dcc0da0480 0x55dcc0daa800 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.866+0000 7fc2af10c740 1 --2- 172.21.15.164:0/2592022651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dcc0d9fb60 0x55dcc0d9ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.866+0000 7fc2af10c740 1 -- 172.21.15.164:0/2592022651 >> 172.21.15.164:0/2592022651 conn(0x55dcc0d9c0f0 msgr2=0x55dcc0d9c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:29.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.866+0000 7fc2af10c740 1 -- 172.21.15.164:0/2592022651 shutdown_connections 2024-09-19T08:55:29.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.866+0000 7fc2af10c740 1 -- 172.21.15.164:0/2592022651 wait complete. 2024-09-19T08:55:29.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.867+0000 7fc2af10c740 1 -- start start 2024-09-19T08:55:29.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.867+0000 7fc2af10c740 1 Processor -- start 2024-09-19T08:55:29.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.867+0000 7fc2af10c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dcc0d9fb60 0x55dcc0da7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:29.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.867+0000 7fc2af10c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcc0da0480 0x55dcc0da8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:29.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.867+0000 7fc2af10c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dcc0daad40 0x55dcc0da8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:29.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.867+0000 7fc2af10c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55dcc0dad910 con 0x55dcc0daad40 2024-09-19T08:55:29.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.867+0000 7fc2af10c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55dcc0cbff10 con 0x55dcc0da0480 2024-09-19T08:55:29.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.867+0000 7fc2af10c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55dcc0dada90 con 0x55dcc0d9fb60 2024-09-19T08:55:29.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.867+0000 7fc2af10c740 10 client.0 _ll_register_callbacks cb 0x55dcc0c88a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:55:29.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.867+0000 7fc2a6d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dcc0daad40 0x55dcc0da8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:29.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.867+0000 7fc2a6d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dcc0daad40 0x55dcc0da8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40884/0 (socket says 172.21.15.164:40884) 2024-09-19T08:55:29.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.867+0000 7fc2a6d9d640 1 -- 172.21.15.164:0/1792007508 learned_addr learned my addr 172.21.15.164:0/1792007508 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:29.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.867+0000 7fc2a5d9b640 1 --2- 172.21.15.164:0/1792007508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcc0da0480 0x55dcc0da8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:29.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.867+0000 7fc2a659c640 1 --2- 172.21.15.164:0/1792007508 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dcc0d9fb60 0x55dcc0da7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:29.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.867+0000 7fc2a6d9d640 1 -- 172.21.15.164:0/1792007508 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dcc0d9fb60 msgr2=0x55dcc0da7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:29.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.867+0000 7fc2a6d9d640 1 --2- 172.21.15.164:0/1792007508 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dcc0d9fb60 0x55dcc0da7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.868+0000 7fc2a6d9d640 1 -- 172.21.15.164:0/1792007508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcc0da0480 msgr2=0x55dcc0da8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:29.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.868+0000 7fc2a6d9d640 1 --2- 172.21.15.164:0/1792007508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcc0da0480 0x55dcc0da8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.868+0000 7fc2a6d9d640 1 -- 172.21.15.164:0/1792007508 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcc0a766a0 con 0x55dcc0daad40 2024-09-19T08:55:29.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.868+0000 7fc2a6d9d640 1 --2- 172.21.15.164:0/1792007508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dcc0daad40 0x55dcc0da8660 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fc2a001cad0 tx=0x7fc2a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ec40d6837c4396 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:29.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.868+0000 7fc28dffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:55:29.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.868+0000 7fc28dffb640 1 -- 172.21.15.164:0/1792007508 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a001ae20 con 0x55dcc0daad40 2024-09-19T08:55:29.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.868+0000 7fc28dffb640 1 -- 172.21.15.164:0/1792007508 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fc2a00243e0 con 0x55dcc0daad40 2024-09-19T08:55:29.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.868+0000 7fc2af10c740 1 -- 172.21.15.164:0/1792007508 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc298036040 con 0x55dcc0daad40 2024-09-19T08:55:29.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.868+0000 7fc28dffb640 1 -- 172.21.15.164:0/1792007508 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc2a00245e0 con 0x55dcc0daad40 2024-09-19T08:55:29.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.868+0000 7fc2af10c740 10 client.17547 fetch_fsmap learned FSMap version 32 2024-09-19T08:55:29.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.868+0000 7fc2af10c740 1 -- 172.21.15.164:0/1792007508 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dcc0db94b0 con 0x55dcc0daad40 2024-09-19T08:55:29.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.868+0000 7fc28dffb640 1 -- 172.21.15.164:0/1792007508 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc298036040 con 0x55dcc0daad40 2024-09-19T08:55:29.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.868+0000 7fc28dffb640 1 -- 172.21.15.164:0/1792007508 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fc2a0023070 con 0x55dcc0daad40 2024-09-19T08:55:29.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.868+0000 7fc2af10c740 10 client.17547 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:55:29.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.868+0000 7fc2af10c740 10 client.17547 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:55:29.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.868+0000 7fc2af10c740 20 client.17547 populate_metadata read hostname 'smithi164' 2024-09-19T08:55:29.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.868+0000 7fc2af10c740 1 --2- 172.21.15.164:0/1792007508 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55dcc0dba970 0x55dcc0dbcde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:29.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.868+0000 7fc2af10c740 4 client.17547 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:55:29.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.868+0000 7fc2af10c740 1 -- 172.21.15.164:0/1792007508 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcc0d127f0 con 0x55dcc0dba970 2024-09-19T08:55:29.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.869+0000 7fc2a659c640 1 --2- 172.21.15.164:0/1792007508 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55dcc0dba970 0x55dcc0dbcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:29.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.869+0000 7fc2a659c640 1 --2- 172.21.15.164:0/1792007508 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55dcc0dba970 0x55dcc0dbcde0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11d1a8a36fe6bdb1 server_cookie=4be5152daa0fbc19 in_seq=0 out_seq=0 2024-09-19T08:55:29.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.869+0000 7fc28dffb640 10 client.17547 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:55:29.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.870+0000 7fc28dffb640 1 -- 172.21.15.164:0/1792007508 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dcc0d127f0 con 0x55dcc0dba970 2024-09-19T08:55:29.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.870+0000 7fc28dffb640 10 client.17547 handle_command_reply: tid=0 2024-09-19T08:55:29.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.870+0000 7fc2af10c740 1 client.17547 shutdown 2024-09-19T08:55:29.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.870+0000 7fc2af10c740 1 -- 172.21.15.164:0/1792007508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dcc0daad40 msgr2=0x55dcc0da8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:29.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.870+0000 7fc2af10c740 1 --2- 172.21.15.164:0/1792007508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dcc0daad40 0x55dcc0da8660 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fc2a001cad0 tx=0x7fc2a0004730 comp rx=0 tx=0).stop 2024-09-19T08:55:29.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.870+0000 7fc2af10c740 1 -- 172.21.15.164:0/1792007508 shutdown_connections 2024-09-19T08:55:29.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.870+0000 7fc2af10c740 1 --2- 172.21.15.164:0/1792007508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dcc0daad40 0x55dcc0da8660 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.870+0000 7fc2af10c740 1 --2- 172.21.15.164:0/1792007508 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55dcc0dba970 0x55dcc0dbcde0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.870+0000 7fc2af10c740 1 --2- 172.21.15.164:0/1792007508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcc0da0480 0x55dcc0da8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.870+0000 7fc2af10c740 1 --2- 172.21.15.164:0/1792007508 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dcc0d9fb60 0x55dcc0da7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.870+0000 7fc28dffb640 0 client.17547 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:55:29.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.870+0000 7fc2af10c740 1 -- 172.21.15.164:0/1792007508 >> 172.21.15.164:0/1792007508 conn(0x55dcc0d9c0f0 msgr2=0x55dcc0d9b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:29.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.870+0000 7fc2af10c740 1 -- 172.21.15.164:0/1792007508 shutdown_connections 2024-09-19T08:55:29.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.870+0000 7fc2af10c740 1 -- 172.21.15.164:0/1792007508 wait complete. 2024-09-19T08:55:29.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.870+0000 7fc2af10c740 20 client.17547 trim_cache size 0 max 16384 2024-09-19T08:55:29.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.888+0000 7fc2af10c740 1 Processor -- start 2024-09-19T08:55:29.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.888+0000 7fc2af10c740 1 -- start start 2024-09-19T08:55:29.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.888+0000 7fc2af10c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcc0d9fb60 0x55dcc0da9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:29.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.888+0000 7fc2af10c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dcc0da0480 0x55dcc0dc2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:29.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.888+0000 7fc2af10c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dcc0daad40 0x55dcc0dc2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:29.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.888+0000 7fc2af10c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a81ad8d0 con 0x55dcc0da0480 2024-09-19T08:55:29.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.888+0000 7fc2af10c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a81af410 con 0x55dcc0d9fb60 2024-09-19T08:55:29.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.888+0000 7fc2af10c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a81adfa0 con 0x55dcc0daad40 2024-09-19T08:55:29.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.888+0000 7fc2a5d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dcc0da0480 0x55dcc0dc2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:29.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.888+0000 7fc2a5d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dcc0da0480 0x55dcc0dc2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40890/0 (socket says 172.21.15.164:40890) 2024-09-19T08:55:29.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.888+0000 7fc2a5d9b640 1 -- 172.21.15.164:0/362970544 learned_addr learned my addr 172.21.15.164:0/362970544 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:29.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.889+0000 7fc2a659c640 1 --2- 172.21.15.164:0/362970544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcc0d9fb60 0x55dcc0da9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:29.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.889+0000 7fc2a5d9b640 1 -- 172.21.15.164:0/362970544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dcc0daad40 msgr2=0x55dcc0dc2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:29.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.889+0000 7fc2a6d9d640 1 --2- 172.21.15.164:0/362970544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dcc0daad40 0x55dcc0dc2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:29.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.889+0000 7fc2a5d9b640 1 --2- 172.21.15.164:0/362970544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dcc0daad40 0x55dcc0dc2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.889+0000 7fc2a5d9b640 1 -- 172.21.15.164:0/362970544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcc0d9fb60 msgr2=0x55dcc0da9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:29.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.889+0000 7fc2a5d9b640 1 --2- 172.21.15.164:0/362970544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcc0d9fb60 0x55dcc0da9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.889+0000 7fc2a5d9b640 1 -- 172.21.15.164:0/362970544 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcc0d9d1b0 con 0x55dcc0da0480 2024-09-19T08:55:29.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.889+0000 7fc2a6d9d640 1 --2- 172.21.15.164:0/362970544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dcc0daad40 0x55dcc0dc2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:55:29.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.889+0000 7fc2a659c640 1 --2- 172.21.15.164:0/362970544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcc0d9fb60 0x55dcc0da9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:55:29.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.889+0000 7fc2a5d9b640 1 --2- 172.21.15.164:0/362970544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dcc0da0480 0x55dcc0dc2700 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fc29c009610 tx=0x7fc29c0d1bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347d8568e78e047e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:29.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.889+0000 7fc28ffff640 1 -- 172.21.15.164:0/362970544 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c021040 con 0x55dcc0da0480 2024-09-19T08:55:29.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.889+0000 7fc28ffff640 1 -- 172.21.15.164:0/362970544 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc29c09fdb0 con 0x55dcc0da0480 2024-09-19T08:55:29.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.889+0000 7fc28ffff640 1 -- 172.21.15.164:0/362970544 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c0cfb90 con 0x55dcc0da0480 2024-09-19T08:55:29.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.890+0000 7fc2af10c740 1 -- 172.21.15.164:0/362970544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dcc0da0480 msgr2=0x55dcc0dc2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:29.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.890+0000 7fc2af10c740 1 --2- 172.21.15.164:0/362970544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dcc0da0480 0x55dcc0dc2700 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fc29c009610 tx=0x7fc29c0d1bf0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.890+0000 7fc2af10c740 1 -- 172.21.15.164:0/362970544 shutdown_connections 2024-09-19T08:55:29.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.890+0000 7fc2af10c740 1 --2- 172.21.15.164:0/362970544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dcc0daad40 0x55dcc0dc2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.890+0000 7fc2af10c740 1 --2- 172.21.15.164:0/362970544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dcc0da0480 0x55dcc0dc2700 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.890+0000 7fc2af10c740 1 --2- 172.21.15.164:0/362970544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcc0d9fb60 0x55dcc0da9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.890+0000 7fc2af10c740 1 -- 172.21.15.164:0/362970544 >> 172.21.15.164:0/362970544 conn(0x55dcc0d9c0f0 msgr2=0x55dcc0da9330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:29.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.890+0000 7fc2af10c740 1 -- 172.21.15.164:0/362970544 shutdown_connections 2024-09-19T08:55:29.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.890+0000 7fc2af10c740 1 -- 172.21.15.164:0/362970544 wait complete. 2024-09-19T08:55:29.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.890+0000 7fc2af10c740 1 -- start start 2024-09-19T08:55:29.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.890+0000 7fc2af10c740 1 Processor -- start 2024-09-19T08:55:29.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.890+0000 7fc2af10c740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dcc0d9fb60 0x55dcc0daf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:29.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.891+0000 7fc2af10c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcc0da0480 0x55dcc0dafb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:29.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.891+0000 7fc2af10c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dcc0daad40 0x55dcc0e1d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:29.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.891+0000 7fc2af10c740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a81aed40 con 0x55dcc0d9fb60 2024-09-19T08:55:29.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.891+0000 7fc2af10c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a81ae670 con 0x55dcc0da0480 2024-09-19T08:55:29.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.891+0000 7fc2af10c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a81ab8d0 con 0x55dcc0daad40 2024-09-19T08:55:29.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.891+0000 7fc2af10c740 10 client.0 _ll_register_callbacks cb 0x7fc2a000a7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:55:29.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.891+0000 7fc2a659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dcc0d9fb60 0x55dcc0daf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:29.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.891+0000 7fc2a659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dcc0d9fb60 0x55dcc0daf650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40896/0 (socket says 172.21.15.164:40896) 2024-09-19T08:55:29.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.891+0000 7fc2a659c640 1 -- 172.21.15.164:0/1701384277 learned_addr learned my addr 172.21.15.164:0/1701384277 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:29.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.891+0000 7fc2a5d9b640 1 --2- 172.21.15.164:0/1701384277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcc0da0480 0x55dcc0dafb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:29.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.891+0000 7fc2a6d9d640 1 --2- 172.21.15.164:0/1701384277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dcc0daad40 0x55dcc0e1d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:29.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.891+0000 7fc2a659c640 1 -- 172.21.15.164:0/1701384277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dcc0daad40 msgr2=0x55dcc0e1d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:29.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.891+0000 7fc2a659c640 1 --2- 172.21.15.164:0/1701384277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dcc0daad40 0x55dcc0e1d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.891+0000 7fc2a659c640 1 -- 172.21.15.164:0/1701384277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcc0da0480 msgr2=0x55dcc0dafb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:29.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.891+0000 7fc2a659c640 1 --2- 172.21.15.164:0/1701384277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcc0da0480 0x55dcc0dafb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.891+0000 7fc2a659c640 1 -- 172.21.15.164:0/1701384277 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcc0e1dea0 con 0x55dcc0d9fb60 2024-09-19T08:55:29.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.891+0000 7fc2a6d9d640 1 --2- 172.21.15.164:0/1701384277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dcc0daad40 0x55dcc0e1d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:55:29.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.891+0000 7fc2a659c640 1 --2- 172.21.15.164:0/1701384277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dcc0d9fb60 0x55dcc0daf650 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fc29802a920 tx=0x7fc298005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d263949fc5ed389 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:29.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.891+0000 7fc28dffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:55:29.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.891+0000 7fc28dffb640 1 -- 172.21.15.164:0/1701384277 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc298031ad0 con 0x55dcc0d9fb60 2024-09-19T08:55:29.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.891+0000 7fc28dffb640 1 -- 172.21.15.164:0/1701384277 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fc298031c70 con 0x55dcc0d9fb60 2024-09-19T08:55:29.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.891+0000 7fc2af10c740 1 -- 172.21.15.164:0/1701384277 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc29c09fdb0 con 0x55dcc0d9fb60 2024-09-19T08:55:29.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.891+0000 7fc28dffb640 1 -- 172.21.15.164:0/1701384277 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc298066050 con 0x55dcc0d9fb60 2024-09-19T08:55:29.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.891+0000 7fc2af10c740 10 client.17553 fetch_fsmap learned FSMap version 32 2024-09-19T08:55:29.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.891+0000 7fc2af10c740 1 -- 172.21.15.164:0/1701384277 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dcc0e26200 con 0x55dcc0d9fb60 2024-09-19T08:55:29.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.892+0000 7fc28dffb640 1 -- 172.21.15.164:0/1701384277 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc29c09fdb0 con 0x55dcc0d9fb60 2024-09-19T08:55:29.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.892+0000 7fc28dffb640 1 -- 172.21.15.164:0/1701384277 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fc298030070 con 0x55dcc0d9fb60 2024-09-19T08:55:29.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.892+0000 7fc2af10c740 10 client.17553 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:55:29.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.892+0000 7fc2af10c740 10 client.17553 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:55:29.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.892+0000 7fc2af10c740 20 client.17553 populate_metadata read hostname 'smithi164' 2024-09-19T08:55:29.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.892+0000 7fc2af10c740 1 --2- 172.21.15.164:0/1701384277 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55dcc0e265b0 0x55dcc0e289a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:29.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.892+0000 7fc2af10c740 4 client.17553 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T08:55:29.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.892+0000 7fc2af10c740 1 -- 172.21.15.164:0/1701384277 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55dcc0d0cd00 con 0x55dcc0e265b0 2024-09-19T08:55:29.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.892+0000 7fc2a5d9b640 1 --2- 172.21.15.164:0/1701384277 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55dcc0e265b0 0x55dcc0e289a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:29.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.892+0000 7fc28dffb640 10 client.17553 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:55:29.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.892+0000 7fc2a5d9b640 1 --2- 172.21.15.164:0/1701384277 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55dcc0e265b0 0x55dcc0e289a0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a78c53f73d8dfa9 server_cookie=6db81724eb10587c in_seq=0 out_seq=0 2024-09-19T08:55:29.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.894+0000 7fc28dffb640 1 -- 172.21.15.164:0/1701384277 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dcc0d0cd00 con 0x55dcc0e265b0 2024-09-19T08:55:29.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.894+0000 7fc28dffb640 10 client.17553 handle_command_reply: tid=0 2024-09-19T08:55:29.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.894+0000 7fc2af10c740 1 client.17553 shutdown 2024-09-19T08:55:29.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.894+0000 7fc2af10c740 1 -- 172.21.15.164:0/1701384277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dcc0d9fb60 msgr2=0x55dcc0daf650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:29.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.894+0000 7fc2af10c740 1 --2- 172.21.15.164:0/1701384277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dcc0d9fb60 0x55dcc0daf650 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fc29802a920 tx=0x7fc298005c30 comp rx=0 tx=0).stop 2024-09-19T08:55:29.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.894+0000 7fc2af10c740 1 -- 172.21.15.164:0/1701384277 shutdown_connections 2024-09-19T08:55:29.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.894+0000 7fc2af10c740 1 --2- 172.21.15.164:0/1701384277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dcc0daad40 0x55dcc0e1d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.894+0000 7fc2af10c740 1 --2- 172.21.15.164:0/1701384277 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55dcc0e265b0 0x55dcc0e289a0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.894+0000 7fc2af10c740 1 --2- 172.21.15.164:0/1701384277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcc0da0480 0x55dcc0dafb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.894+0000 7fc2af10c740 1 --2- 172.21.15.164:0/1701384277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dcc0d9fb60 0x55dcc0daf650 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.894+0000 7fc2af10c740 1 -- 172.21.15.164:0/1701384277 >> 172.21.15.164:0/1701384277 conn(0x55dcc0d9c0f0 msgr2=0x55dcc0da83c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:29.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.894+0000 7fc28dffb640 0 client.17553 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:55:29.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.894+0000 7fc2af10c740 1 -- 172.21.15.164:0/1701384277 shutdown_connections 2024-09-19T08:55:29.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.894+0000 7fc2af10c740 1 -- 172.21.15.164:0/1701384277 wait complete. 2024-09-19T08:55:29.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.894+0000 7fc2af10c740 20 client.17553 trim_cache size 0 max 16384 2024-09-19T08:55:29.899 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:55:29.899 INFO:teuthology.orchestra.run.smithi164.stdout: "return_code": 0, 2024-09-19T08:55:29.899 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_tag": "6ba73cfe-7ee8-4660-8615-e0a2f8887979", 2024-09-19T08:55:29.899 INFO:teuthology.orchestra.run.smithi164.stdout: "mode": "asynchronous" 2024-09-19T08:55:29.899 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:55:29.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.895+0000 7fc2acd33640 1 -- 172.21.15.164:0/2943113361 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fc2900a33b0 msgr2=0x7fc2900a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:29.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.895+0000 7fc2acd33640 1 --2- 172.21.15.164:0/2943113361 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fc2900a33b0 0x7fc2900a5870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0005fd0 tx=0x7fc2a00046c0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.895+0000 7fc2acd33640 1 -- 172.21.15.164:0/2943113361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2a81036e0 msgr2=0x7fc2a8198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:29.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.895+0000 7fc2acd33640 1 --2- 172.21.15.164:0/2943113361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2a81036e0 0x7fc2a8198b30 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00d740 tx=0x7fc29c00dc00 comp rx=0 tx=0).stop 2024-09-19T08:55:29.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.895+0000 7fc2acd33640 1 -- 172.21.15.164:0/2943113361 shutdown_connections 2024-09-19T08:55:29.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.895+0000 7fc2acd33640 1 --2- 172.21.15.164:0/2943113361 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2a81082a0 0x7fc2a819cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.895+0000 7fc2acd33640 1 --2- 172.21.15.164:0/2943113361 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fc2900a33b0 0x7fc2900a5870 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.895+0000 7fc2acd33640 1 --2- 172.21.15.164:0/2943113361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2a81036e0 0x7fc2a8198b30 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.895+0000 7fc2acd33640 1 --2- 172.21.15.164:0/2943113361 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2a8102d30 0x7fc2a81985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:29.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.895+0000 7fc2acd33640 1 -- 172.21.15.164:0/2943113361 >> 172.21.15.164:0/2943113361 conn(0x7fc2a80fc7d0 msgr2=0x7fc2a8105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:29.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.895+0000 7fc2acd33640 1 -- 172.21.15.164:0/2943113361 shutdown_connections 2024-09-19T08:55:29.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:29.895+0000 7fc2acd33640 1 -- 172.21.15.164:0/2943113361 wait complete. 2024-09-19T08:55:30.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:30 smithi164 ceph-mon[30928]: pgmap v301: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:55:30.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:30 smithi164 ceph-mon[30928]: scrub summary: active paths [/] 2024-09-19T08:55:30.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:30 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:55:30.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:30 smithi164 ceph-mon[30928]: scrub summary: active paths [/] 2024-09-19T08:55:30.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:30 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:55:30.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:30 smithi173 ceph-mon[35306]: pgmap v301: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:55:30.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:30 smithi173 ceph-mon[35306]: scrub summary: active paths [/] 2024-09-19T08:55:30.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:30 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:55:30.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:30 smithi173 ceph-mon[35306]: scrub summary: active paths [/] 2024-09-19T08:55:30.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:30 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:55:30.552 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:55:30.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:30 smithi170 ceph-mon[35072]: pgmap v301: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:55:30.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:30 smithi170 ceph-mon[35072]: scrub summary: active paths [/] 2024-09-19T08:55:30.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:30 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:55:30.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:30 smithi170 ceph-mon[35072]: scrub summary: active paths [/] 2024-09-19T08:55:30.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:30 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:55:30.799 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:55:31.225 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:31 smithi164 ceph-mon[30928]: scrub queued for path: / 2024-09-19T08:55:31.225 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:31 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:55:31.225 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:31 smithi164 ceph-mon[30928]: scrub summary: active paths [/] 2024-09-19T08:55:31.225 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:31 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:55:31.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.307+0000 7f1a406d7640 1 -- 172.21.15.164:0/19020362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a381008e0 msgr2=0x7f1a38100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:31.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.307+0000 7f1a406d7640 1 --2- 172.21.15.164:0/19020362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a381008e0 0x7f1a38100cc0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f1a28009ca0 tx=0x7f1a2802a8c0 comp rx=0 tx=0).stop 2024-09-19T08:55:31.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.307+0000 7f1a406d7640 1 -- 172.21.15.164:0/19020362 shutdown_connections 2024-09-19T08:55:31.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.307+0000 7f1a406d7640 1 --2- 172.21.15.164:0/19020362 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a38108180 0x7f1a3810eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:31.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.307+0000 7f1a406d7640 1 --2- 172.21.15.164:0/19020362 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a38101290 0x7f1a38107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:31.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.307+0000 7f1a406d7640 1 --2- 172.21.15.164:0/19020362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a381008e0 0x7f1a38100cc0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:31.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.307+0000 7f1a406d7640 1 -- 172.21.15.164:0/19020362 >> 172.21.15.164:0/19020362 conn(0x7f1a38077210 msgr2=0x7f1a38077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:31.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.307+0000 7f1a406d7640 1 -- 172.21.15.164:0/19020362 shutdown_connections 2024-09-19T08:55:31.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.307+0000 7f1a406d7640 1 -- 172.21.15.164:0/19020362 wait complete. 2024-09-19T08:55:31.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.308+0000 7f1a406d7640 1 Processor -- start 2024-09-19T08:55:31.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.308+0000 7f1a406d7640 1 -- start start 2024-09-19T08:55:31.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.308+0000 7f1a406d7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a381008e0 0x7f1a3819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:31.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.308+0000 7f1a406d7640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a38101290 0x7f1a3819ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:31.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.308+0000 7f1a406d7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a38108180 0x7f1a3819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:31.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.308+0000 7f1a406d7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1a38112070 con 0x7f1a381008e0 2024-09-19T08:55:31.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.308+0000 7f1a406d7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1a38111ef0 con 0x7f1a38108180 2024-09-19T08:55:31.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.308+0000 7f1a406d7640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1a381121f0 con 0x7f1a38101290 2024-09-19T08:55:31.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.309+0000 7f1a3e473640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a381008e0 0x7f1a3819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:31.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.309+0000 7f1a3e473640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a381008e0 0x7f1a3819a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40916/0 (socket says 172.21.15.164:40916) 2024-09-19T08:55:31.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.309+0000 7f1a3e473640 1 -- 172.21.15.164:0/432122997 learned_addr learned my addr 172.21.15.164:0/432122997 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:31.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.309+0000 7f1a3dc72640 1 --2- 172.21.15.164:0/432122997 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a38101290 0x7f1a3819ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:31.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.309+0000 7f1a3e473640 1 -- 172.21.15.164:0/432122997 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a38101290 msgr2=0x7f1a3819ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:31.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.309+0000 7f1a3e473640 1 --2- 172.21.15.164:0/432122997 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a38101290 0x7f1a3819ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:31.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.309+0000 7f1a3ec74640 1 --2- 172.21.15.164:0/432122997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a38108180 0x7f1a3819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:31.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.309+0000 7f1a3e473640 1 -- 172.21.15.164:0/432122997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a38108180 msgr2=0x7f1a3819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:31.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.309+0000 7f1a3e473640 1 --2- 172.21.15.164:0/432122997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a38108180 0x7f1a3819efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:31.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.309+0000 7f1a3e473640 1 -- 172.21.15.164:0/432122997 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3819f6a0 con 0x7f1a381008e0 2024-09-19T08:55:31.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.309+0000 7f1a3e473640 1 --2- 172.21.15.164:0/432122997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a381008e0 0x7f1a3819a6f0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f1a280097b0 tx=0x7f1a28031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:31.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.309+0000 7f1a377fe640 1 -- 172.21.15.164:0/432122997 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2802fba0 con 0x7f1a381008e0 2024-09-19T08:55:31.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.309+0000 7f1a377fe640 1 -- 172.21.15.164:0/432122997 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a2802fd40 con 0x7f1a381008e0 2024-09-19T08:55:31.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.309+0000 7f1a406d7640 1 -- 172.21.15.164:0/432122997 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a3819f930 con 0x7f1a381008e0 2024-09-19T08:55:31.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.309+0000 7f1a377fe640 1 -- 172.21.15.164:0/432122997 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2803b930 con 0x7f1a381008e0 2024-09-19T08:55:31.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.309+0000 7f1a406d7640 1 -- 172.21.15.164:0/432122997 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a381a7270 con 0x7f1a381008e0 2024-09-19T08:55:31.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.312+0000 7f1a377fe640 1 -- 172.21.15.164:0/432122997 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f1a28030cc0 con 0x7f1a381008e0 2024-09-19T08:55:31.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.312+0000 7f1a42ab0740 1 Processor -- start 2024-09-19T08:55:31.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.312+0000 7f1a42ab0740 1 -- start start 2024-09-19T08:55:31.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.313+0000 7f1a42ab0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56218ed474a0 0x56218ed47880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:31.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.313+0000 7f1a42ab0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56218ed47dc0 0x56218ed52140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:31.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.313+0000 7f1a42ab0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56218ed52680 0x56218ed54a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:31.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.313+0000 7f1a42ab0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56218ecad2e0 con 0x56218ed474a0 2024-09-19T08:55:31.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.313+0000 7f1a42ab0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56218ec54ce0 con 0x56218ed52680 2024-09-19T08:55:31.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.313+0000 7f1a42ab0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56218eca92a0 con 0x56218ed47dc0 2024-09-19T08:55:31.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.313+0000 7f1a3ec74640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56218ed474a0 0x56218ed47880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:31.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.313+0000 7f1a3ec74640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56218ed474a0 0x56218ed47880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40924/0 (socket says 172.21.15.164:40924) 2024-09-19T08:55:31.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.313+0000 7f1a3ec74640 1 -- 172.21.15.164:0/951978014 learned_addr learned my addr 172.21.15.164:0/951978014 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:31.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.313+0000 7f1a3e473640 1 --2- 172.21.15.164:0/951978014 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56218ed47dc0 0x56218ed52140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:31.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.313+0000 7f1a3dc72640 1 --2- 172.21.15.164:0/951978014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56218ed52680 0x56218ed54a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:31.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.313+0000 7f1a377fe640 1 --2- 172.21.15.164:0/432122997 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f1a200a3430 0x7f1a200a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:31.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.313+0000 7f1a3ec74640 1 -- 172.21.15.164:0/951978014 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56218ed47dc0 msgr2=0x56218ed52140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:31.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.313+0000 7f1a3ec74640 1 --2- 172.21.15.164:0/951978014 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56218ed47dc0 0x56218ed52140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:31.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.313+0000 7f1a3ec74640 1 -- 172.21.15.164:0/951978014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56218ed52680 msgr2=0x56218ed54a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:31.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.313+0000 7f1a3ec74640 1 --2- 172.21.15.164:0/951978014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56218ed52680 0x56218ed54a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:31.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.313+0000 7f1a3ec74640 1 -- 172.21.15.164:0/951978014 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56218ec39440 con 0x56218ed474a0 2024-09-19T08:55:31.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.313+0000 7f1a377fe640 1 -- 172.21.15.164:0/432122997 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f1a280f1320 con 0x7f1a381008e0 2024-09-19T08:55:31.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.313+0000 7f1a3ec74640 1 --2- 172.21.15.164:0/432122997 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f1a200a3430 0x7f1a200a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:31.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.313+0000 7f1a3ec74640 1 --2- 172.21.15.164:0/951978014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56218ed474a0 0x56218ed47880 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f1a3000ba90 tx=0x7f1a3000bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dbe36bcceffdffa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:31.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.314+0000 7f1a34ff9640 1 -- 172.21.15.164:0/951978014 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a30019570 con 0x56218ed474a0 2024-09-19T08:55:31.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.314+0000 7f1a34ff9640 1 -- 172.21.15.164:0/951978014 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a3000fb10 con 0x56218ed474a0 2024-09-19T08:55:31.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.314+0000 7f1a34ff9640 1 -- 172.21.15.164:0/951978014 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a30018a80 con 0x56218ed474a0 2024-09-19T08:55:31.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.314+0000 7f1a42ab0740 1 -- 172.21.15.164:0/951978014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56218ed474a0 msgr2=0x56218ed47880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:31.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.314+0000 7f1a3ec74640 1 --2- 172.21.15.164:0/432122997 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f1a200a3430 0x7f1a200a58f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1a3000dfd0 tx=0x7f1a30020000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:31.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.314+0000 7f1a42ab0740 1 --2- 172.21.15.164:0/951978014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56218ed474a0 0x56218ed47880 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f1a3000ba90 tx=0x7f1a3000bf50 comp rx=0 tx=0).stop 2024-09-19T08:55:31.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.315+0000 7f1a42ab0740 1 -- 172.21.15.164:0/951978014 shutdown_connections 2024-09-19T08:55:31.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.315+0000 7f1a42ab0740 1 --2- 172.21.15.164:0/951978014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56218ed52680 0x56218ed54a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:31.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.315+0000 7f1a42ab0740 1 --2- 172.21.15.164:0/951978014 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56218ed47dc0 0x56218ed52140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:31.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.315+0000 7f1a42ab0740 1 --2- 172.21.15.164:0/951978014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56218ed474a0 0x56218ed47880 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:31.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.315+0000 7f1a42ab0740 1 -- 172.21.15.164:0/951978014 >> 172.21.15.164:0/951978014 conn(0x56218ed43a30 msgr2=0x56218ed43e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:31.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.315+0000 7f1a42ab0740 1 -- 172.21.15.164:0/951978014 shutdown_connections 2024-09-19T08:55:31.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.315+0000 7f1a42ab0740 1 -- 172.21.15.164:0/951978014 wait complete. 2024-09-19T08:55:31.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.315+0000 7f1a42ab0740 1 -- start start 2024-09-19T08:55:31.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.315+0000 7f1a42ab0740 1 Processor -- start 2024-09-19T08:55:31.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.315+0000 7f1a42ab0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56218ed474a0 0x56218ed51f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:31.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.315+0000 7f1a42ab0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56218ed47dc0 0x56218ed4f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:31.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.315+0000 7f1a42ab0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56218ed52680 0x56218ed4f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:31.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.315+0000 7f1a42ab0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56218ed55250 con 0x56218ed47dc0 2024-09-19T08:55:31.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.315+0000 7f1a42ab0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56218ec68050 con 0x56218ed474a0 2024-09-19T08:55:31.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.315+0000 7f1a42ab0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56218ed553d0 con 0x56218ed52680 2024-09-19T08:55:31.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.315+0000 7f1a42ab0740 10 client.0 _ll_register_callbacks cb 0x56218eca8160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:55:31.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.315+0000 7f1a3dc72640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56218ed47dc0 0x56218ed4f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:31.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.315+0000 7f1a3dc72640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56218ed47dc0 0x56218ed4f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40934/0 (socket says 172.21.15.164:40934) 2024-09-19T08:55:31.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.315+0000 7f1a3dc72640 1 -- 172.21.15.164:0/1555297620 learned_addr learned my addr 172.21.15.164:0/1555297620 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:31.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.316+0000 7f1a3e473640 1 --2- 172.21.15.164:0/1555297620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56218ed474a0 0x56218ed51f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:31.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.316+0000 7f1a3ec74640 1 --2- 172.21.15.164:0/1555297620 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56218ed52680 0x56218ed4f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:31.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.316+0000 7f1a3dc72640 1 -- 172.21.15.164:0/1555297620 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56218ed52680 msgr2=0x56218ed4f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:31.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.316+0000 7f1a3dc72640 1 --2- 172.21.15.164:0/1555297620 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56218ed52680 0x56218ed4f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:31.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.316+0000 7f1a3dc72640 1 -- 172.21.15.164:0/1555297620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56218ed474a0 msgr2=0x56218ed51f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:31.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.316+0000 7f1a3dc72640 1 --2- 172.21.15.164:0/1555297620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56218ed474a0 0x56218ed51f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:31.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.316+0000 7f1a3dc72640 1 -- 172.21.15.164:0/1555297620 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56218ea1e3c0 con 0x56218ed47dc0 2024-09-19T08:55:31.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.316+0000 7f1a3dc72640 1 --2- 172.21.15.164:0/1555297620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56218ed47dc0 0x56218ed4f1e0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c010720 tx=0x7f1a2c010be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=333293efae53098b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:31.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.316+0000 7f1a1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:55:31.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.316+0000 7f1a1dffb640 1 -- 172.21.15.164:0/1555297620 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c00cb10 con 0x56218ed47dc0 2024-09-19T08:55:31.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.316+0000 7f1a1dffb640 1 -- 172.21.15.164:0/1555297620 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f1a2c00ccb0 con 0x56218ed47dc0 2024-09-19T08:55:31.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.316+0000 7f1a42ab0740 1 -- 172.21.15.164:0/1555297620 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a3000fb10 con 0x56218ed47dc0 2024-09-19T08:55:31.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.316+0000 7f1a1dffb640 1 -- 172.21.15.164:0/1555297620 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1a2c005020 con 0x56218ed47dc0 2024-09-19T08:55:31.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.316+0000 7f1a42ab0740 10 client.17565 fetch_fsmap learned FSMap version 32 2024-09-19T08:55:31.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.316+0000 7f1a42ab0740 1 -- 172.21.15.164:0/1555297620 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56218ed60bd0 con 0x56218ed47dc0 2024-09-19T08:55:31.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.317+0000 7f1a1dffb640 1 -- 172.21.15.164:0/1555297620 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a3000fb10 con 0x56218ed47dc0 2024-09-19T08:55:31.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.317+0000 7f1a1dffb640 1 -- 172.21.15.164:0/1555297620 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f1a2c011440 con 0x56218ed47dc0 2024-09-19T08:55:31.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.317+0000 7f1a42ab0740 10 client.17565 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:55:31.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.317+0000 7f1a42ab0740 10 client.17565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:55:31.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.317+0000 7f1a42ab0740 20 client.17565 populate_metadata read hostname 'smithi164' 2024-09-19T08:55:31.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.317+0000 7f1a42ab0740 1 --2- 172.21.15.164:0/1555297620 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56218ed620e0 0x56218ed64510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:31.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.317+0000 7f1a42ab0740 4 client.17565 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:55:31.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.317+0000 7f1a42ab0740 1 -- 172.21.15.164:0/1555297620 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56218ecba850 con 0x56218ed620e0 2024-09-19T08:55:31.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.317+0000 7f1a3e473640 1 --2- 172.21.15.164:0/1555297620 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56218ed620e0 0x56218ed64510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:31.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.317+0000 7f1a3e473640 1 --2- 172.21.15.164:0/1555297620 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56218ed620e0 0x56218ed64510 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afc308169b869b7b server_cookie=a0be2d2f481acea8 in_seq=0 out_seq=0 2024-09-19T08:55:31.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.317+0000 7f1a1dffb640 10 client.17565 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:55:31.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.318+0000 7f1a1dffb640 1 -- 172.21.15.164:0/1555297620 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56218ecba850 con 0x56218ed620e0 2024-09-19T08:55:31.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.318+0000 7f1a1dffb640 10 client.17565 handle_command_reply: tid=0 2024-09-19T08:55:31.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.318+0000 7f1a42ab0740 1 client.17565 shutdown 2024-09-19T08:55:31.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.318+0000 7f1a42ab0740 1 -- 172.21.15.164:0/1555297620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56218ed47dc0 msgr2=0x56218ed4f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:31.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.318+0000 7f1a42ab0740 1 --2- 172.21.15.164:0/1555297620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56218ed47dc0 0x56218ed4f1e0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c010720 tx=0x7f1a2c010be0 comp rx=0 tx=0).stop 2024-09-19T08:55:31.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.318+0000 7f1a42ab0740 1 -- 172.21.15.164:0/1555297620 shutdown_connections 2024-09-19T08:55:31.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.318+0000 7f1a42ab0740 1 --2- 172.21.15.164:0/1555297620 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56218ed52680 0x56218ed4f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:31.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.318+0000 7f1a42ab0740 1 --2- 172.21.15.164:0/1555297620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56218ed47dc0 0x56218ed4f1e0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:31.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.318+0000 7f1a42ab0740 1 --2- 172.21.15.164:0/1555297620 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56218ed620e0 0x56218ed64510 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:31.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.318+0000 7f1a42ab0740 1 --2- 172.21.15.164:0/1555297620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56218ed474a0 0x56218ed51f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:31.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.318+0000 7f1a42ab0740 1 -- 172.21.15.164:0/1555297620 >> 172.21.15.164:0/1555297620 conn(0x56218ed43a30 msgr2=0x56218ed49710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:31.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.318+0000 7f1a1dffb640 0 client.17565 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:55:31.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.318+0000 7f1a42ab0740 1 -- 172.21.15.164:0/1555297620 shutdown_connections 2024-09-19T08:55:31.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.318+0000 7f1a42ab0740 1 -- 172.21.15.164:0/1555297620 wait complete. 2024-09-19T08:55:31.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.319+0000 7f1a42ab0740 20 client.17565 trim_cache size 0 max 16384 2024-09-19T08:55:31.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.336+0000 7f1a42ab0740 1 Processor -- start 2024-09-19T08:55:31.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.336+0000 7f1a42ab0740 1 -- start start 2024-09-19T08:55:31.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.336+0000 7f1a42ab0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56218ed43a30 0x56218ed5e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:31.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.336+0000 7f1a42ab0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56218ed474a0 0x56218ed69e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:31.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.336+0000 7f1a42ab0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56218ed47dc0 0x56218ed6a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:31.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.336+0000 7f1a42ab0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1a381af8f0 con 0x56218ed43a30 2024-09-19T08:55:31.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.336+0000 7f1a42ab0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1a381b1430 con 0x56218ed474a0 2024-09-19T08:55:31.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.336+0000 7f1a42ab0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1a381affc0 con 0x56218ed47dc0 2024-09-19T08:55:31.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.336+0000 7f1a3e473640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56218ed43a30 0x56218ed5e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:31.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.336+0000 7f1a3e473640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56218ed43a30 0x56218ed5e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40948/0 (socket says 172.21.15.164:40948) 2024-09-19T08:55:31.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.336+0000 7f1a3e473640 1 -- 172.21.15.164:0/1483468489 learned_addr learned my addr 172.21.15.164:0/1483468489 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:31.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.336+0000 7f1a3dc72640 1 --2- 172.21.15.164:0/1483468489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56218ed474a0 0x56218ed69e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:31.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.336+0000 7f1a3ec74640 1 --2- 172.21.15.164:0/1483468489 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56218ed47dc0 0x56218ed6a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:31.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.336+0000 7f1a3e473640 1 -- 172.21.15.164:0/1483468489 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56218ed47dc0 msgr2=0x56218ed6a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:31.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.336+0000 7f1a3e473640 1 --2- 172.21.15.164:0/1483468489 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56218ed47dc0 0x56218ed6a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:31.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.336+0000 7f1a3e473640 1 -- 172.21.15.164:0/1483468489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56218ed474a0 msgr2=0x56218ed69e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:31.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.336+0000 7f1a3e473640 1 --2- 172.21.15.164:0/1483468489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56218ed474a0 0x56218ed69e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:31.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.336+0000 7f1a3e473640 1 -- 172.21.15.164:0/1483468489 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56218ed4d730 con 0x56218ed43a30 2024-09-19T08:55:31.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.336+0000 7f1a3dc72640 1 --2- 172.21.15.164:0/1483468489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56218ed474a0 0x56218ed69e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:55:31.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.336+0000 7f1a3ec74640 1 --2- 172.21.15.164:0/1483468489 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56218ed47dc0 0x56218ed6a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:55:31.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.336+0000 7f1a3e473640 1 --2- 172.21.15.164:0/1483468489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56218ed43a30 0x56218ed5e4b0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f1a280bd6e0 tx=0x7f1a280f3830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25ff9040c2dbae7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:31.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.336+0000 7f1a1ffff640 1 -- 172.21.15.164:0/1483468489 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a280bde50 con 0x56218ed43a30 2024-09-19T08:55:31.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.337+0000 7f1a1ffff640 1 -- 172.21.15.164:0/1483468489 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a28036040 con 0x56218ed43a30 2024-09-19T08:55:31.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.337+0000 7f1a1ffff640 1 -- 172.21.15.164:0/1483468489 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a280b9dc0 con 0x56218ed43a30 2024-09-19T08:55:31.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.337+0000 7f1a42ab0740 1 -- 172.21.15.164:0/1483468489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56218ed43a30 msgr2=0x56218ed5e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:31.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.337+0000 7f1a42ab0740 1 --2- 172.21.15.164:0/1483468489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56218ed43a30 0x56218ed5e4b0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f1a280bd6e0 tx=0x7f1a280f3830 comp rx=0 tx=0).stop 2024-09-19T08:55:31.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.337+0000 7f1a42ab0740 1 -- 172.21.15.164:0/1483468489 shutdown_connections 2024-09-19T08:55:31.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.337+0000 7f1a42ab0740 1 --2- 172.21.15.164:0/1483468489 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56218ed47dc0 0x56218ed6a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:31.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.337+0000 7f1a42ab0740 1 --2- 172.21.15.164:0/1483468489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56218ed474a0 0x56218ed69e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:31.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.337+0000 7f1a42ab0740 1 --2- 172.21.15.164:0/1483468489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56218ed43a30 0x56218ed5e4b0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:31.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.337+0000 7f1a42ab0740 1 -- 172.21.15.164:0/1483468489 >> 172.21.15.164:0/1483468489 conn(0x56218ed4f880 msgr2=0x56218ed4dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:31.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.337+0000 7f1a42ab0740 1 -- 172.21.15.164:0/1483468489 shutdown_connections 2024-09-19T08:55:31.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.337+0000 7f1a42ab0740 1 -- 172.21.15.164:0/1483468489 wait complete. 2024-09-19T08:55:31.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.337+0000 7f1a42ab0740 1 -- start start 2024-09-19T08:55:31.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.337+0000 7f1a42ab0740 1 Processor -- start 2024-09-19T08:55:31.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.338+0000 7f1a42ab0740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56218ed43a30 0x56218ed56b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:31.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.338+0000 7f1a42ab0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56218ed474a0 0x56218ed570a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:31.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.338+0000 7f1a42ab0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56218ed47dc0 0x56218edcbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:31.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.338+0000 7f1a42ab0740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1a381b0d60 con 0x56218ed43a30 2024-09-19T08:55:31.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.338+0000 7f1a42ab0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1a381b0690 con 0x56218ed47dc0 2024-09-19T08:55:31.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.338+0000 7f1a42ab0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1a381ad8f0 con 0x56218ed474a0 2024-09-19T08:55:31.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.338+0000 7f1a42ab0740 10 client.0 _ll_register_callbacks cb 0x7f1a2c00e350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:55:31.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.338+0000 7f1a3e473640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56218ed43a30 0x56218ed56b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:31.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.338+0000 7f1a3e473640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56218ed43a30 0x56218ed56b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40950/0 (socket says 172.21.15.164:40950) 2024-09-19T08:55:31.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.338+0000 7f1a3e473640 1 -- 172.21.15.164:0/2808450887 learned_addr learned my addr 172.21.15.164:0/2808450887 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:31.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.338+0000 7f1a3dc72640 1 --2- 172.21.15.164:0/2808450887 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56218ed474a0 0x56218ed570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:31.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.338+0000 7f1a3ec74640 1 --2- 172.21.15.164:0/2808450887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56218ed47dc0 0x56218edcbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:31.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.338+0000 7f1a3e473640 1 -- 172.21.15.164:0/2808450887 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56218ed474a0 msgr2=0x56218ed570a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:31.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.338+0000 7f1a3e473640 1 --2- 172.21.15.164:0/2808450887 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56218ed474a0 0x56218ed570a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:31.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.338+0000 7f1a3e473640 1 -- 172.21.15.164:0/2808450887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56218ed47dc0 msgr2=0x56218edcbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:31.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.338+0000 7f1a3e473640 1 --2- 172.21.15.164:0/2808450887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56218ed47dc0 0x56218edcbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:31.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.338+0000 7f1a3e473640 1 -- 172.21.15.164:0/2808450887 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56218edcc3e0 con 0x56218ed43a30 2024-09-19T08:55:31.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.338+0000 7f1a3dc72640 1 --2- 172.21.15.164:0/2808450887 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56218ed474a0 0x56218ed570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:55:31.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.338+0000 7f1a3ec74640 1 --2- 172.21.15.164:0/2808450887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56218ed47dc0 0x56218edcbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:55:31.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.338+0000 7f1a3e473640 1 --2- 172.21.15.164:0/2808450887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56218ed43a30 0x56218ed56b60 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f1a28030010 tx=0x7f1a280bd820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec90ebb68abfbc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:31.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.338+0000 7f1a1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:55:31.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.339+0000 7f1a1dffb640 1 -- 172.21.15.164:0/2808450887 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a280c63c0 con 0x56218ed43a30 2024-09-19T08:55:31.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.339+0000 7f1a1dffb640 1 -- 172.21.15.164:0/2808450887 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f1a280eeca0 con 0x56218ed43a30 2024-09-19T08:55:31.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.339+0000 7f1a42ab0740 1 -- 172.21.15.164:0/2808450887 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a28036040 con 0x56218ed43a30 2024-09-19T08:55:31.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.339+0000 7f1a1dffb640 1 -- 172.21.15.164:0/2808450887 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1a280eeea0 con 0x56218ed43a30 2024-09-19T08:55:31.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.339+0000 7f1a42ab0740 10 client.17571 fetch_fsmap learned FSMap version 32 2024-09-19T08:55:31.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.339+0000 7f1a42ab0740 1 -- 172.21.15.164:0/2808450887 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56218edcd640 con 0x56218ed43a30 2024-09-19T08:55:31.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.339+0000 7f1a1dffb640 1 -- 172.21.15.164:0/2808450887 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a28036040 con 0x56218ed43a30 2024-09-19T08:55:31.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.339+0000 7f1a1dffb640 1 -- 172.21.15.164:0/2808450887 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f1a28004b40 con 0x56218ed43a30 2024-09-19T08:55:31.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.339+0000 7f1a42ab0740 10 client.17571 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:55:31.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.339+0000 7f1a42ab0740 10 client.17571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:55:31.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.339+0000 7f1a42ab0740 20 client.17571 populate_metadata read hostname 'smithi164' 2024-09-19T08:55:31.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.339+0000 7f1a42ab0740 1 --2- 172.21.15.164:0/2808450887 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56218ed52680 0x56218edcfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:31.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.339+0000 7f1a42ab0740 4 client.17571 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:55:31.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.339+0000 7f1a42ab0740 1 -- 172.21.15.164:0/2808450887 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56218ecb4d60 con 0x56218ed52680 2024-09-19T08:55:31.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.339+0000 7f1a3dc72640 1 --2- 172.21.15.164:0/2808450887 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56218ed52680 0x56218edcfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:31.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.340+0000 7f1a1dffb640 10 client.17571 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:55:31.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.340+0000 7f1a3dc72640 1 --2- 172.21.15.164:0/2808450887 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56218ed52680 0x56218edcfd40 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8dfe33e020334bf server_cookie=5b48e0272dc66627 in_seq=0 out_seq=0 2024-09-19T08:55:31.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.340+0000 7f1a1dffb640 1 -- 172.21.15.164:0/2808450887 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56218ecb4d60 con 0x56218ed52680 2024-09-19T08:55:31.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.340+0000 7f1a1dffb640 10 client.17571 handle_command_reply: tid=0 2024-09-19T08:55:31.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.340+0000 7f1a42ab0740 1 client.17571 shutdown 2024-09-19T08:55:31.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.340+0000 7f1a42ab0740 1 -- 172.21.15.164:0/2808450887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56218ed43a30 msgr2=0x56218ed56b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:31.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.340+0000 7f1a42ab0740 1 --2- 172.21.15.164:0/2808450887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56218ed43a30 0x56218ed56b60 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f1a28030010 tx=0x7f1a280bd820 comp rx=0 tx=0).stop 2024-09-19T08:55:31.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.340+0000 7f1a42ab0740 1 -- 172.21.15.164:0/2808450887 shutdown_connections 2024-09-19T08:55:31.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.340+0000 7f1a42ab0740 1 --2- 172.21.15.164:0/2808450887 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56218ed52680 0x56218edcfd40 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:31.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.341+0000 7f1a42ab0740 1 --2- 172.21.15.164:0/2808450887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56218ed47dc0 0x56218edcbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:31.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.341+0000 7f1a42ab0740 1 --2- 172.21.15.164:0/2808450887 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56218ed474a0 0x56218ed570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:31.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.341+0000 7f1a1dffb640 0 client.17571 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:55:31.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.341+0000 7f1a42ab0740 1 --2- 172.21.15.164:0/2808450887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56218ed43a30 0x56218ed56b60 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:31.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.341+0000 7f1a42ab0740 1 -- 172.21.15.164:0/2808450887 >> 172.21.15.164:0/2808450887 conn(0x56218ed4f880 msgr2=0x56218ed51260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:31.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.341+0000 7f1a42ab0740 1 -- 172.21.15.164:0/2808450887 shutdown_connections 2024-09-19T08:55:31.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.341+0000 7f1a42ab0740 1 -- 172.21.15.164:0/2808450887 wait complete. 2024-09-19T08:55:31.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.341+0000 7f1a42ab0740 20 client.17571 trim_cache size 0 max 16384 2024-09-19T08:55:31.355 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:55:31.355 INFO:teuthology.orchestra.run.smithi164.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T08:55:31.356 INFO:teuthology.orchestra.run.smithi164.stdout: "scrubs": { 2024-09-19T08:55:31.356 INFO:teuthology.orchestra.run.smithi164.stdout: "6ba73cfe-7ee8-4660-8615-e0a2f8887979": { 2024-09-19T08:55:31.356 INFO:teuthology.orchestra.run.smithi164.stdout: "path": "/", 2024-09-19T08:55:31.356 INFO:teuthology.orchestra.run.smithi164.stdout: "tag": "6ba73cfe-7ee8-4660-8615-e0a2f8887979", 2024-09-19T08:55:31.356 INFO:teuthology.orchestra.run.smithi164.stdout: "options": "recursive,force" 2024-09-19T08:55:31.356 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:55:31.356 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:55:31.356 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:55:31.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.341+0000 7f1a406d7640 1 -- 172.21.15.164:0/432122997 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f1a200a3430 msgr2=0x7f1a200a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:31.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.341+0000 7f1a406d7640 1 --2- 172.21.15.164:0/432122997 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f1a200a3430 0x7f1a200a58f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1a3000dfd0 tx=0x7f1a30020000 comp rx=0 tx=0).stop 2024-09-19T08:55:31.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.341+0000 7f1a406d7640 1 -- 172.21.15.164:0/432122997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a381008e0 msgr2=0x7f1a3819a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:31.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.341+0000 7f1a406d7640 1 --2- 172.21.15.164:0/432122997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a381008e0 0x7f1a3819a6f0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f1a280097b0 tx=0x7f1a28031b10 comp rx=0 tx=0).stop 2024-09-19T08:55:31.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.341+0000 7f1a406d7640 1 -- 172.21.15.164:0/432122997 shutdown_connections 2024-09-19T08:55:31.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.341+0000 7f1a406d7640 1 --2- 172.21.15.164:0/432122997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a38108180 0x7f1a3819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:31.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.341+0000 7f1a406d7640 1 --2- 172.21.15.164:0/432122997 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a38101290 0x7f1a3819ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:31.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.341+0000 7f1a406d7640 1 --2- 172.21.15.164:0/432122997 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f1a200a3430 0x7f1a200a58f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:31.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.341+0000 7f1a406d7640 1 --2- 172.21.15.164:0/432122997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a381008e0 0x7f1a3819a6f0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:31.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.341+0000 7f1a406d7640 1 -- 172.21.15.164:0/432122997 >> 172.21.15.164:0/432122997 conn(0x7f1a38077210 msgr2=0x7f1a38102370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:31.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.341+0000 7f1a406d7640 1 -- 172.21.15.164:0/432122997 shutdown_connections 2024-09-19T08:55:31.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:31.341+0000 7f1a406d7640 1 -- 172.21.15.164:0/432122997 wait complete. 2024-09-19T08:55:31.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:31 smithi173 ceph-mon[35306]: scrub queued for path: / 2024-09-19T08:55:31.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:31 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:55:31.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:31 smithi173 ceph-mon[35306]: scrub summary: active paths [/] 2024-09-19T08:55:31.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:31 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:55:31.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:31 smithi170 ceph-mon[35072]: scrub queued for path: / 2024-09-19T08:55:31.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:31 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:55:31.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:31 smithi170 ceph-mon[35072]: scrub summary: active paths [/] 2024-09-19T08:55:31.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:31 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:55:32.143 INFO:tasks.cephfs.filesystem:scrub status for tag:6ba73cfe-7ee8-4660-8615-e0a2f8887979 - {'path': '/', 'tag': '6ba73cfe-7ee8-4660-8615-e0a2f8887979', 'options': 'recursive,force'} 2024-09-19T08:55:32.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:32 smithi173 ceph-mon[35306]: pgmap v302: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-09-19T08:55:32.531 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:32 smithi164 ceph-mon[30928]: pgmap v302: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-09-19T08:55:32.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:32 smithi170 ceph-mon[35072]: pgmap v302: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-09-19T08:55:34.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:34 smithi173 ceph-mon[35306]: pgmap v303: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-19T08:55:34.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:34 smithi164 ceph-mon[30928]: pgmap v303: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-19T08:55:34.624 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:34 smithi170 ceph-mon[35072]: pgmap v303: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-19T08:55:36.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:36 smithi173 ceph-mon[35306]: pgmap v304: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 12 op/s 2024-09-19T08:55:36.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:36 smithi164 ceph-mon[30928]: pgmap v304: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 12 op/s 2024-09-19T08:55:36.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:36 smithi170 ceph-mon[35072]: pgmap v304: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 12 op/s 2024-09-19T08:55:38.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:38 smithi173 ceph-mon[35306]: pgmap v305: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 12 op/s 2024-09-19T08:55:38.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:38 smithi173 ceph-mon[35306]: scrub summary: idle 2024-09-19T08:55:38.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:38 smithi173 ceph-mon[35306]: scrub summary: idle 2024-09-19T08:55:38.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:38 smithi164 ceph-mon[30928]: pgmap v305: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 12 op/s 2024-09-19T08:55:38.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:38 smithi164 ceph-mon[30928]: scrub summary: idle 2024-09-19T08:55:38.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:38 smithi164 ceph-mon[30928]: scrub summary: idle 2024-09-19T08:55:38.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:38 smithi170 ceph-mon[35072]: pgmap v305: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 12 op/s 2024-09-19T08:55:38.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:38 smithi170 ceph-mon[35072]: scrub summary: idle 2024-09-19T08:55:38.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:38 smithi170 ceph-mon[35072]: scrub summary: idle 2024-09-19T08:55:40.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:40 smithi173 ceph-mon[35306]: pgmap v306: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 11 op/s 2024-09-19T08:55:40.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:40 smithi164 ceph-mon[30928]: pgmap v306: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 11 op/s 2024-09-19T08:55:40.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:40 smithi170 ceph-mon[35072]: pgmap v306: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 11 op/s 2024-09-19T08:55:42.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:42 smithi173 ceph-mon[35306]: pgmap v307: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 14 op/s 2024-09-19T08:55:42.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:42 smithi164 ceph-mon[30928]: pgmap v307: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 14 op/s 2024-09-19T08:55:42.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:42 smithi170 ceph-mon[35072]: pgmap v307: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 14 op/s 2024-09-19T08:55:44.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:44 smithi173 ceph-mon[35306]: pgmap v308: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 11 op/s 2024-09-19T08:55:44.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:44 smithi164 ceph-mon[30928]: pgmap v308: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 11 op/s 2024-09-19T08:55:44.624 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:44 smithi170 ceph-mon[35072]: pgmap v308: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 11 op/s 2024-09-19T08:55:46.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:46 smithi164 ceph-mon[30928]: pgmap v309: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 13 op/s 2024-09-19T08:55:46.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:46 smithi170 ceph-mon[35072]: pgmap v309: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 13 op/s 2024-09-19T08:55:46.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:46 smithi173 ceph-mon[35306]: pgmap v309: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 13 op/s 2024-09-19T08:55:48.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:48 smithi164 ceph-mon[30928]: pgmap v310: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T08:55:48.624 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:48 smithi170 ceph-mon[35072]: pgmap v310: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T08:55:48.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:48 smithi173 ceph-mon[35306]: pgmap v310: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T08:55:50.427 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:50 smithi170 ceph-mon[35072]: pgmap v311: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:55:50.463 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:50 smithi173 ceph-mon[35306]: pgmap v311: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:55:50.531 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:50 smithi164 ceph-mon[30928]: pgmap v311: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:55:52.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:52 smithi164 ceph-mon[30928]: pgmap v312: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:55:52.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:52 smithi170 ceph-mon[35072]: pgmap v312: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:55:52.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:52 smithi173 ceph-mon[35306]: pgmap v312: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:55:54.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:54 smithi164 ceph-mon[30928]: pgmap v313: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:55:54.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:54 smithi170 ceph-mon[35072]: pgmap v313: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:55:54.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:54 smithi173 ceph-mon[35306]: pgmap v313: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:55:55.781 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T08:55:55.781 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-19T08:55:55.782 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:55:56.047 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:55:56.465 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:56 smithi164 ceph-mon[30928]: pgmap v314: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T08:55:56.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.549+0000 7f31a17a9640 1 -- 172.21.15.164:0/244337316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f319c103780 msgr2=0x7f319c107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:56.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.549+0000 7f31a17a9640 1 --2- 172.21.15.164:0/244337316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f319c103780 0x7f319c107bc0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f3184009930 tx=0x7f318402a420 comp rx=0 tx=0).stop 2024-09-19T08:55:56.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.549+0000 7f31a17a9640 1 -- 172.21.15.164:0/244337316 shutdown_connections 2024-09-19T08:55:56.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.549+0000 7f31a17a9640 1 --2- 172.21.15.164:0/244337316 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f319c108300 0x7f319c10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:56.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.549+0000 7f31a17a9640 1 --2- 172.21.15.164:0/244337316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f319c103780 0x7f319c107bc0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:56.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.549+0000 7f31a17a9640 1 --2- 172.21.15.164:0/244337316 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f319c102dd0 0x7f319c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:56.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.549+0000 7f31a17a9640 1 -- 172.21.15.164:0/244337316 >> 172.21.15.164:0/244337316 conn(0x7f319c0fc7d0 msgr2=0x7f319c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:56.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.549+0000 7f31a17a9640 1 -- 172.21.15.164:0/244337316 shutdown_connections 2024-09-19T08:55:56.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.549+0000 7f31a17a9640 1 -- 172.21.15.164:0/244337316 wait complete. 2024-09-19T08:55:56.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.549+0000 7f31a17a9640 1 Processor -- start 2024-09-19T08:55:56.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.550+0000 7f31a17a9640 1 -- start start 2024-09-19T08:55:56.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.550+0000 7f31a17a9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f319c102dd0 0x7f319c100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:56.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.550+0000 7f31a17a9640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f319c103780 0x7f319c100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:56.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.550+0000 7f31a17a9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f319c108300 0x7f319c1014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:56.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.550+0000 7f31a17a9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f319c112070 con 0x7f319c102dd0 2024-09-19T08:55:56.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.550+0000 7f31a17a9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f319c111ef0 con 0x7f319c108300 2024-09-19T08:55:56.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.550+0000 7f31a17a9640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1121f0 con 0x7f319c103780 2024-09-19T08:55:56.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.550+0000 7f319affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f319c102dd0 0x7f319c100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:56.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.550+0000 7f319affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f319c102dd0 0x7f319c100a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43696/0 (socket says 172.21.15.164:43696) 2024-09-19T08:55:56.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.550+0000 7f319affd640 1 -- 172.21.15.164:0/1341842984 learned_addr learned my addr 172.21.15.164:0/1341842984 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:56.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.550+0000 7f319b7fe640 1 --2- 172.21.15.164:0/1341842984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f319c108300 0x7f319c1014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:56.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.550+0000 7f319a7fc640 1 --2- 172.21.15.164:0/1341842984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f319c103780 0x7f319c100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:56.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.550+0000 7f319affd640 1 -- 172.21.15.164:0/1341842984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f319c103780 msgr2=0x7f319c100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:56.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.550+0000 7f319affd640 1 --2- 172.21.15.164:0/1341842984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f319c103780 0x7f319c100f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:56.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.550+0000 7f319affd640 1 -- 172.21.15.164:0/1341842984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f319c108300 msgr2=0x7f319c1014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:56.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.550+0000 7f319affd640 1 --2- 172.21.15.164:0/1341842984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f319c108300 0x7f319c1014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:56.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.550+0000 7f319affd640 1 -- 172.21.15.164:0/1341842984 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f319c1a6a50 con 0x7f319c102dd0 2024-09-19T08:55:56.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.551+0000 7f319affd640 1 --2- 172.21.15.164:0/1341842984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f319c102dd0 0x7f319c100a30 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f318c00d770 tx=0x7f318c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:56.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.551+0000 7f3193fff640 1 -- 172.21.15.164:0/1341842984 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c017070 con 0x7f319c102dd0 2024-09-19T08:55:56.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.551+0000 7f31a17a9640 1 -- 172.21.15.164:0/1341842984 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f319c1a6d40 con 0x7f319c102dd0 2024-09-19T08:55:56.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.551+0000 7f3193fff640 1 -- 172.21.15.164:0/1341842984 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f318c005500 con 0x7f319c102dd0 2024-09-19T08:55:56.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.551+0000 7f3193fff640 1 -- 172.21.15.164:0/1341842984 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c004e00 con 0x7f319c102dd0 2024-09-19T08:55:56.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.551+0000 7f31a17a9640 1 -- 172.21.15.164:0/1341842984 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f319c1a71a0 con 0x7f319c102dd0 2024-09-19T08:55:56.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.552+0000 7f31a17a9640 1 -- 172.21.15.164:0/1341842984 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f319c1031b0 con 0x7f319c102dd0 2024-09-19T08:55:56.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.554+0000 7f3193fff640 1 -- 172.21.15.164:0/1341842984 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f318c010460 con 0x7f319c102dd0 2024-09-19T08:55:56.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.555+0000 7f3193fff640 1 --2- 172.21.15.164:0/1341842984 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f31880a33b0 0x7f31880a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:56.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.555+0000 7f319a7fc640 1 --2- 172.21.15.164:0/1341842984 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f31880a33b0 0x7f31880a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:56.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.555+0000 7f3193fff640 1 -- 172.21.15.164:0/1341842984 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f318c0cf6f0 con 0x7f319c102dd0 2024-09-19T08:55:56.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.555+0000 7f319a7fc640 1 --2- 172.21.15.164:0/1341842984 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f31880a33b0 0x7f31880a5870 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3184005940 tx=0x7f31840058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:56.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.560+0000 7f3193fff640 1 -- 172.21.15.164:0/1341842984 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f318c09bac0 con 0x7f319c102dd0 2024-09-19T08:55:56.624 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:56 smithi170 ceph-mon[35072]: pgmap v314: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T08:55:56.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:56 smithi173 ceph-mon[35306]: pgmap v314: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T08:55:56.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.769+0000 7f31a17a9640 1 -- 172.21.15.164:0/1341842984 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f319c10a290 con 0x7f319c102dd0 2024-09-19T08:55:56.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.770+0000 7f3193fff640 1 -- 172.21.15.164:0/1341842984 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11984 (secure 0 0 0) 0x7f318c09fdc0 con 0x7f319c102dd0 2024-09-19T08:55:56.771 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:55:56.771 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":32,"btime":"2024-09-19T08:52:27:192804+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:52:27.074168+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:55:56.771 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 32 2024-09-19T08:55:56.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.773+0000 7f31a17a9640 1 -- 172.21.15.164:0/1341842984 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f31880a33b0 msgr2=0x7f31880a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:56.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.773+0000 7f31a17a9640 1 --2- 172.21.15.164:0/1341842984 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f31880a33b0 0x7f31880a5870 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3184005940 tx=0x7f31840058d0 comp rx=0 tx=0).stop 2024-09-19T08:55:56.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.773+0000 7f31a17a9640 1 -- 172.21.15.164:0/1341842984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f319c102dd0 msgr2=0x7f319c100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:56.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.773+0000 7f31a17a9640 1 --2- 172.21.15.164:0/1341842984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f319c102dd0 0x7f319c100a30 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f318c00d770 tx=0x7f318c00dc30 comp rx=0 tx=0).stop 2024-09-19T08:55:56.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.773+0000 7f31a17a9640 1 -- 172.21.15.164:0/1341842984 shutdown_connections 2024-09-19T08:55:56.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.773+0000 7f31a17a9640 1 --2- 172.21.15.164:0/1341842984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f319c108300 0x7f319c1014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:56.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.773+0000 7f31a17a9640 1 --2- 172.21.15.164:0/1341842984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f319c103780 0x7f319c100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:56.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.773+0000 7f31a17a9640 1 --2- 172.21.15.164:0/1341842984 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f31880a33b0 0x7f31880a5870 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:56.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.773+0000 7f31a17a9640 1 --2- 172.21.15.164:0/1341842984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f319c102dd0 0x7f319c100a30 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:56.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.773+0000 7f31a17a9640 1 -- 172.21.15.164:0/1341842984 >> 172.21.15.164:0/1341842984 conn(0x7f319c0fc7d0 msgr2=0x7f319c105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:56.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.774+0000 7f31a17a9640 1 -- 172.21.15.164:0/1341842984 shutdown_connections 2024-09-19T08:55:56.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:56.774+0000 7f31a17a9640 1 -- 172.21.15.164:0/1341842984 wait complete. 2024-09-19T08:55:57.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:57 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/1341842984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:55:57.574 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-19T08:55:57.574 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-19T08:55:57.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:57 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/1341842984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:55:57.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:57 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/1341842984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:55:57.827 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:55:58.225 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:55:58 smithi164 ceph-mon[30928]: pgmap v315: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:55:58.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.316+0000 7fe936d2d640 1 -- 172.21.15.164:0/2398449373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe930102ef0 msgr2=0x7fe93010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:58.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.316+0000 7fe936d2d640 1 --2- 172.21.15.164:0/2398449373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe930102ef0 0x7fe93010da40 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fe928009ca0 tx=0x7fe92802a950 comp rx=0 tx=0).stop 2024-09-19T08:55:58.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.316+0000 7fe936d2d640 1 -- 172.21.15.164:0/2398449373 shutdown_connections 2024-09-19T08:55:58.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.316+0000 7fe936d2d640 1 --2- 172.21.15.164:0/2398449373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe930102ef0 0x7fe93010da40 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:58.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.316+0000 7fe936d2d640 1 --2- 172.21.15.164:0/2398449373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe930102570 0x7fe9301029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:58.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.316+0000 7fe936d2d640 1 --2- 172.21.15.164:0/2398449373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe930100e20 0x7fe930101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:58.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.316+0000 7fe936d2d640 1 -- 172.21.15.164:0/2398449373 >> 172.21.15.164:0/2398449373 conn(0x7fe9300fc7d0 msgr2=0x7fe9300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:58.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.316+0000 7fe936d2d640 1 -- 172.21.15.164:0/2398449373 shutdown_connections 2024-09-19T08:55:58.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.316+0000 7fe936d2d640 1 -- 172.21.15.164:0/2398449373 wait complete. 2024-09-19T08:55:58.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.317+0000 7fe936d2d640 1 Processor -- start 2024-09-19T08:55:58.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.317+0000 7fe936d2d640 1 -- start start 2024-09-19T08:55:58.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.317+0000 7fe936d2d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe930100e20 0x7fe930199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:58.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.317+0000 7fe936d2d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe930102570 0x7fe930199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:58.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.317+0000 7fe936d2d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe930102ef0 0x7fe9301a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:58.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.317+0000 7fe936d2d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe930110f80 con 0x7fe930102570 2024-09-19T08:55:58.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.317+0000 7fe936d2d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe930110e00 con 0x7fe930100e20 2024-09-19T08:55:58.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.317+0000 7fe936d2d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe930111100 con 0x7fe930102ef0 2024-09-19T08:55:58.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.317+0000 7fe92ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe930102570 0x7fe930199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:58.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.317+0000 7fe92ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe930102570 0x7fe930199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43718/0 (socket says 172.21.15.164:43718) 2024-09-19T08:55:58.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.317+0000 7fe92ffff640 1 -- 172.21.15.164:0/1595429575 learned_addr learned my addr 172.21.15.164:0/1595429575 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:58.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.317+0000 7fe934ac9640 1 --2- 172.21.15.164:0/1595429575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe930100e20 0x7fe930199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:58.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.317+0000 7fe92ffff640 1 -- 172.21.15.164:0/1595429575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe930102ef0 msgr2=0x7fe9301a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:58.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.317+0000 7fe9352ca640 1 --2- 172.21.15.164:0/1595429575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe930102ef0 0x7fe9301a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:58.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.317+0000 7fe92ffff640 1 --2- 172.21.15.164:0/1595429575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe930102ef0 0x7fe9301a43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:58.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.318+0000 7fe92ffff640 1 -- 172.21.15.164:0/1595429575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe930100e20 msgr2=0x7fe930199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:58.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.318+0000 7fe92ffff640 1 --2- 172.21.15.164:0/1595429575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe930100e20 0x7fe930199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:58.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.318+0000 7fe92ffff640 1 -- 172.21.15.164:0/1595429575 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9301a48f0 con 0x7fe930102570 2024-09-19T08:55:58.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.318+0000 7fe92ffff640 1 --2- 172.21.15.164:0/1595429575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe930102570 0x7fe930199870 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00d950 tx=0x7fe91c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:58.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.318+0000 7fe92dffb640 1 -- 172.21.15.164:0/1595429575 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c017070 con 0x7fe930102570 2024-09-19T08:55:58.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.318+0000 7fe936d2d640 1 -- 172.21.15.164:0/1595429575 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9301a4ac0 con 0x7fe930102570 2024-09-19T08:55:58.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.318+0000 7fe92dffb640 1 -- 172.21.15.164:0/1595429575 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe91c009c30 con 0x7fe930102570 2024-09-19T08:55:58.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.318+0000 7fe92dffb640 1 -- 172.21.15.164:0/1595429575 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c00f6b0 con 0x7fe930102570 2024-09-19T08:55:58.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.318+0000 7fe936d2d640 1 -- 172.21.15.164:0/1595429575 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9301a5000 con 0x7fe930102570 2024-09-19T08:55:58.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.321+0000 7fe92dffb640 1 -- 172.21.15.164:0/1595429575 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fe91c0109d0 con 0x7fe930102570 2024-09-19T08:55:58.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.321+0000 7fe939106740 1 Processor -- start 2024-09-19T08:55:58.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.321+0000 7fe939106740 1 -- start start 2024-09-19T08:55:58.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.321+0000 7fe939106740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e544a760 0x55d8e544ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:58.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.321+0000 7fe939106740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e544b080 0x55d8e5455400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:58.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.321+0000 7fe939106740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e5455940 0x55d8e5457d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:58.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.321+0000 7fe92dffb640 1 --2- 172.21.15.164:0/1595429575 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe9200a33b0 0x7fe9200a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:58.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.321+0000 7fe939106740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d8e53b02a0 con 0x55d8e544b080 2024-09-19T08:55:58.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.321+0000 7fe939106740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d8e5357b10 con 0x55d8e5455940 2024-09-19T08:55:58.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.321+0000 7fe939106740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d8e53ac260 con 0x55d8e544a760 2024-09-19T08:55:58.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.322+0000 7fe934ac9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e544b080 0x55d8e5455400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:58.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.322+0000 7fe934ac9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e544b080 0x55d8e5455400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43734/0 (socket says 172.21.15.164:43734) 2024-09-19T08:55:58.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.322+0000 7fe934ac9640 1 -- 172.21.15.164:0/547943877 learned_addr learned my addr 172.21.15.164:0/547943877 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:58.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.322+0000 7fe9352ca640 1 --2- 172.21.15.164:0/1595429575 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe9200a33b0 0x7fe9200a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:58.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.322+0000 7fe9352ca640 1 --2- 172.21.15.164:0/547943877 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e544a760 0x55d8e544ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:58.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.322+0000 7fe92dffb640 1 -- 172.21.15.164:0/1595429575 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fe91c0cf880 con 0x7fe930102570 2024-09-19T08:55:58.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.322+0000 7fe92ffff640 1 --2- 172.21.15.164:0/547943877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e5455940 0x55d8e5457d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:58.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.322+0000 7fe934ac9640 1 -- 172.21.15.164:0/547943877 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e544a760 msgr2=0x55d8e544ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:58.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.322+0000 7fe934ac9640 1 --2- 172.21.15.164:0/547943877 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e544a760 0x55d8e544ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:58.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.322+0000 7fe934ac9640 1 -- 172.21.15.164:0/547943877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e5455940 msgr2=0x55d8e5457d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:58.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.322+0000 7fe934ac9640 1 --2- 172.21.15.164:0/547943877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e5455940 0x55d8e5457d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:58.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.322+0000 7fe934ac9640 1 -- 172.21.15.164:0/547943877 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8e5121b90 con 0x55d8e544b080 2024-09-19T08:55:58.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.322+0000 7fe92ffff640 1 --2- 172.21.15.164:0/547943877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e5455940 0x55d8e5457d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:55:58.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.322+0000 7fe934ac9640 1 --2- 172.21.15.164:0/547943877 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e544b080 0x55d8e5455400 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fe92400b760 tx=0x7fe92400bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b689844922746087 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:58.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.322+0000 7fe9352ca640 1 --2- 172.21.15.164:0/1595429575 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe9200a33b0 0x7fe9200a5870 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe928005e30 tx=0x7fe928005d80 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:58.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.322+0000 7fe91affd640 1 -- 172.21.15.164:0/547943877 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924003dd0 con 0x55d8e544b080 2024-09-19T08:55:58.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.322+0000 7fe91affd640 1 -- 172.21.15.164:0/547943877 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe924015070 con 0x55d8e544b080 2024-09-19T08:55:58.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.322+0000 7fe91affd640 1 -- 172.21.15.164:0/547943877 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924004e60 con 0x55d8e544b080 2024-09-19T08:55:58.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.323+0000 7fe939106740 1 -- 172.21.15.164:0/547943877 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e544b080 msgr2=0x55d8e5455400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:58.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.323+0000 7fe939106740 1 --2- 172.21.15.164:0/547943877 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e544b080 0x55d8e5455400 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fe92400b760 tx=0x7fe92400bc20 comp rx=0 tx=0).stop 2024-09-19T08:55:58.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.323+0000 7fe939106740 1 -- 172.21.15.164:0/547943877 shutdown_connections 2024-09-19T08:55:58.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.323+0000 7fe939106740 1 --2- 172.21.15.164:0/547943877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e5455940 0x55d8e5457d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:58.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.323+0000 7fe939106740 1 --2- 172.21.15.164:0/547943877 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e544b080 0x55d8e5455400 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:58.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.323+0000 7fe939106740 1 --2- 172.21.15.164:0/547943877 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e544a760 0x55d8e544ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:58.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.323+0000 7fe939106740 1 -- 172.21.15.164:0/547943877 >> 172.21.15.164:0/547943877 conn(0x55d8e5446cf0 msgr2=0x55d8e54470d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:58.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.323+0000 7fe939106740 1 -- 172.21.15.164:0/547943877 shutdown_connections 2024-09-19T08:55:58.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.323+0000 7fe939106740 1 -- 172.21.15.164:0/547943877 wait complete. 2024-09-19T08:55:58.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.323+0000 7fe939106740 1 -- start start 2024-09-19T08:55:58.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.324+0000 7fe939106740 1 Processor -- start 2024-09-19T08:55:58.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.324+0000 7fe939106740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e544a760 0x55d8e54527d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:58.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.324+0000 7fe939106740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e544b080 0x55d8e5452d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:58.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.324+0000 7fe939106740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e5455940 0x55d8e5453250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:58.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.324+0000 7fe939106740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d8e5458510 con 0x55d8e544b080 2024-09-19T08:55:58.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.324+0000 7fe939106740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d8e53386e0 con 0x55d8e5455940 2024-09-19T08:55:58.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.324+0000 7fe939106740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d8e5458690 con 0x55d8e544a760 2024-09-19T08:55:58.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.324+0000 7fe939106740 10 client.0 _ll_register_callbacks cb 0x55d8e535ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:55:58.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.324+0000 7fe92ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e544b080 0x55d8e5452d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:58.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.324+0000 7fe92ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e544b080 0x55d8e5452d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43736/0 (socket says 172.21.15.164:43736) 2024-09-19T08:55:58.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.324+0000 7fe92ffff640 1 -- 172.21.15.164:0/4133761621 learned_addr learned my addr 172.21.15.164:0/4133761621 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:58.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.324+0000 7fe934ac9640 1 --2- 172.21.15.164:0/4133761621 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e544a760 0x55d8e54527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:58.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.324+0000 7fe9352ca640 1 --2- 172.21.15.164:0/4133761621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e5455940 0x55d8e5453250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:58.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.324+0000 7fe92ffff640 1 -- 172.21.15.164:0/4133761621 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e544a760 msgr2=0x55d8e54527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:58.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.324+0000 7fe92ffff640 1 --2- 172.21.15.164:0/4133761621 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e544a760 0x55d8e54527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:58.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.324+0000 7fe92ffff640 1 -- 172.21.15.164:0/4133761621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e5455940 msgr2=0x55d8e5453250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:58.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.324+0000 7fe92ffff640 1 --2- 172.21.15.164:0/4133761621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e5455940 0x55d8e5453250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:58.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.324+0000 7fe92ffff640 1 -- 172.21.15.164:0/4133761621 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8e5461a10 con 0x55d8e544b080 2024-09-19T08:55:58.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.324+0000 7fe934ac9640 1 --2- 172.21.15.164:0/4133761621 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e544a760 0x55d8e54527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:55:58.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.325+0000 7fe92ffff640 1 --2- 172.21.15.164:0/4133761621 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e544b080 0x55d8e5452d10 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fe91c009610 tx=0x7fe91c09baa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528bab5b5510b638 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:58.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.325+0000 7fe90ffff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:55:58.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.325+0000 7fe90ffff640 1 -- 172.21.15.164:0/4133761621 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c020070 con 0x55d8e544b080 2024-09-19T08:55:58.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.325+0000 7fe90ffff640 1 -- 172.21.15.164:0/4133761621 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fe91c0aba10 con 0x55d8e544b080 2024-09-19T08:55:58.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.325+0000 7fe939106740 1 -- 172.21.15.164:0/4133761621 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe924015070 con 0x55d8e544b080 2024-09-19T08:55:58.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.325+0000 7fe90ffff640 1 -- 172.21.15.164:0/4133761621 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe91c09ff50 con 0x55d8e544b080 2024-09-19T08:55:58.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.325+0000 7fe939106740 10 client.17589 fetch_fsmap learned FSMap version 32 2024-09-19T08:55:58.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.325+0000 7fe939106740 1 -- 172.21.15.164:0/4133761621 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d8e54642f0 con 0x55d8e544b080 2024-09-19T08:55:58.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.325+0000 7fe90ffff640 1 -- 172.21.15.164:0/4133761621 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe924015070 con 0x55d8e544b080 2024-09-19T08:55:58.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.325+0000 7fe90ffff640 1 -- 172.21.15.164:0/4133761621 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe91c09aff0 con 0x55d8e544b080 2024-09-19T08:55:58.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.325+0000 7fe939106740 10 client.17589 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:55:58.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.325+0000 7fe939106740 10 client.17589 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:55:58.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.325+0000 7fe939106740 20 client.17589 populate_metadata read hostname 'smithi164' 2024-09-19T08:55:58.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.325+0000 7fe939106740 1 --2- 172.21.15.164:0/4133761621 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55d8e5465700 0x55d8e5467b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:58.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.325+0000 7fe939106740 4 client.17589 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:55:58.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.325+0000 7fe939106740 1 -- 172.21.15.164:0/4133761621 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8e53bd9d0 con 0x55d8e5465700 2024-09-19T08:55:58.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.326+0000 7fe934ac9640 1 --2- 172.21.15.164:0/4133761621 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55d8e5465700 0x55d8e5467b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:58.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.326+0000 7fe90ffff640 10 client.17589 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:55:58.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.326+0000 7fe934ac9640 1 --2- 172.21.15.164:0/4133761621 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55d8e5465700 0x55d8e5467b10 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b781b639ebc12430 server_cookie=d1d32b67b643ea22 in_seq=0 out_seq=0 2024-09-19T08:55:58.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.327+0000 7fe90ffff640 1 -- 172.21.15.164:0/4133761621 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d8e53bd9d0 con 0x55d8e5465700 2024-09-19T08:55:58.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.327+0000 7fe90ffff640 10 client.17589 handle_command_reply: tid=0 2024-09-19T08:55:58.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.327+0000 7fe939106740 1 client.17589 shutdown 2024-09-19T08:55:58.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.327+0000 7fe939106740 1 -- 172.21.15.164:0/4133761621 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e544b080 msgr2=0x55d8e5452d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:58.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.327+0000 7fe939106740 1 --2- 172.21.15.164:0/4133761621 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e544b080 0x55d8e5452d10 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fe91c009610 tx=0x7fe91c09baa0 comp rx=0 tx=0).stop 2024-09-19T08:55:58.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.327+0000 7fe939106740 1 -- 172.21.15.164:0/4133761621 shutdown_connections 2024-09-19T08:55:58.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.327+0000 7fe939106740 1 --2- 172.21.15.164:0/4133761621 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55d8e5465700 0x55d8e5467b10 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:58.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.327+0000 7fe939106740 1 --2- 172.21.15.164:0/4133761621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e5455940 0x55d8e5453250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:58.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.327+0000 7fe939106740 1 --2- 172.21.15.164:0/4133761621 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e544b080 0x55d8e5452d10 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:58.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.327+0000 7fe939106740 1 --2- 172.21.15.164:0/4133761621 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e544a760 0x55d8e54527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:58.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.327+0000 7fe90ffff640 0 client.17589 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:55:58.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.327+0000 7fe939106740 1 -- 172.21.15.164:0/4133761621 >> 172.21.15.164:0/4133761621 conn(0x55d8e5446cf0 msgr2=0x55d8e544c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:58.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.327+0000 7fe939106740 1 -- 172.21.15.164:0/4133761621 shutdown_connections 2024-09-19T08:55:58.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.327+0000 7fe939106740 1 -- 172.21.15.164:0/4133761621 wait complete. 2024-09-19T08:55:58.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.327+0000 7fe939106740 20 client.17589 trim_cache size 0 max 16384 2024-09-19T08:55:58.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.344+0000 7fe939106740 1 Processor -- start 2024-09-19T08:55:58.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.344+0000 7fe939106740 1 -- start start 2024-09-19T08:55:58.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.344+0000 7fe939106740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e544a760 0x55d8e544c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:58.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.344+0000 7fe939106740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e544b080 0x55d8e546d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:58.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.344+0000 7fe939106740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e5455940 0x55d8e546d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:58.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.344+0000 7fe939106740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe9301ad730 con 0x55d8e544b080 2024-09-19T08:55:58.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.344+0000 7fe939106740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe9301af270 con 0x55d8e544a760 2024-09-19T08:55:58.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.344+0000 7fe939106740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe9301ade00 con 0x55d8e5455940 2024-09-19T08:55:58.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.345+0000 7fe92ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e544b080 0x55d8e546d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:58.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.345+0000 7fe92ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e544b080 0x55d8e546d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43742/0 (socket says 172.21.15.164:43742) 2024-09-19T08:55:58.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.345+0000 7fe92ffff640 1 -- 172.21.15.164:0/3201353829 learned_addr learned my addr 172.21.15.164:0/3201353829 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:58.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.345+0000 7fe9352ca640 1 --2- 172.21.15.164:0/3201353829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e5455940 0x55d8e546d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:58.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.345+0000 7fe934ac9640 1 --2- 172.21.15.164:0/3201353829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e544a760 0x55d8e544c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:58.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.345+0000 7fe92ffff640 1 -- 172.21.15.164:0/3201353829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e5455940 msgr2=0x55d8e546d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:58.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.345+0000 7fe92ffff640 1 --2- 172.21.15.164:0/3201353829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e5455940 0x55d8e546d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:58.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.345+0000 7fe92ffff640 1 -- 172.21.15.164:0/3201353829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e544a760 msgr2=0x55d8e544c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:58.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.345+0000 7fe92ffff640 1 --2- 172.21.15.164:0/3201353829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e544a760 0x55d8e544c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:58.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.345+0000 7fe92ffff640 1 -- 172.21.15.164:0/3201353829 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8e5446b20 con 0x55d8e544b080 2024-09-19T08:55:58.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.345+0000 7fe934ac9640 1 --2- 172.21.15.164:0/3201353829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e544a760 0x55d8e544c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:55:58.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.345+0000 7fe92ffff640 1 --2- 172.21.15.164:0/3201353829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e544b080 0x55d8e546d3e0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0a42e0 tx=0x7fe91c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb475bb2a690eea1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:58.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.345+0000 7fe91a7fc640 1 -- 172.21.15.164:0/3201353829 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c020070 con 0x55d8e544b080 2024-09-19T08:55:58.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.345+0000 7fe91a7fc640 1 -- 172.21.15.164:0/3201353829 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe91c0a4810 con 0x55d8e544b080 2024-09-19T08:55:58.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.345+0000 7fe91a7fc640 1 -- 172.21.15.164:0/3201353829 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c0ab9c0 con 0x55d8e544b080 2024-09-19T08:55:58.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.346+0000 7fe939106740 1 -- 172.21.15.164:0/3201353829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e544b080 msgr2=0x55d8e546d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:58.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.346+0000 7fe939106740 1 --2- 172.21.15.164:0/3201353829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e544b080 0x55d8e546d3e0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0a42e0 tx=0x7fe91c005880 comp rx=0 tx=0).stop 2024-09-19T08:55:58.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.346+0000 7fe939106740 1 -- 172.21.15.164:0/3201353829 shutdown_connections 2024-09-19T08:55:58.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.346+0000 7fe939106740 1 --2- 172.21.15.164:0/3201353829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e5455940 0x55d8e546d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:58.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.346+0000 7fe939106740 1 --2- 172.21.15.164:0/3201353829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e544b080 0x55d8e546d3e0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:58.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.346+0000 7fe939106740 1 --2- 172.21.15.164:0/3201353829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e544a760 0x55d8e544c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:58.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.346+0000 7fe939106740 1 -- 172.21.15.164:0/3201353829 >> 172.21.15.164:0/3201353829 conn(0x55d8e5446cf0 msgr2=0x55d8e544c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:58.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.346+0000 7fe939106740 1 -- 172.21.15.164:0/3201353829 shutdown_connections 2024-09-19T08:55:58.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.346+0000 7fe939106740 1 -- 172.21.15.164:0/3201353829 wait complete. 2024-09-19T08:55:58.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.346+0000 7fe939106740 1 -- start start 2024-09-19T08:55:58.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.346+0000 7fe939106740 1 Processor -- start 2024-09-19T08:55:58.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.347+0000 7fe939106740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e544a760 0x55d8e545a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:58.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.347+0000 7fe939106740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e544b080 0x55d8e545a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:58.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.347+0000 7fe939106740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e5455940 0x55d8e54c8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:58.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.347+0000 7fe939106740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe9301aeba0 con 0x55d8e544b080 2024-09-19T08:55:58.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.347+0000 7fe939106740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe9301ae4d0 con 0x55d8e544a760 2024-09-19T08:55:58.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.347+0000 7fe939106740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe9301ab730 con 0x55d8e5455940 2024-09-19T08:55:58.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.347+0000 7fe939106740 10 client.0 _ll_register_callbacks cb 0x7fe91c09b7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:55:58.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.347+0000 7fe92ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e544b080 0x55d8e545a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:58.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.347+0000 7fe92ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e544b080 0x55d8e545a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43750/0 (socket says 172.21.15.164:43750) 2024-09-19T08:55:58.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.347+0000 7fe92ffff640 1 -- 172.21.15.164:0/2432263718 learned_addr learned my addr 172.21.15.164:0/2432263718 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:55:58.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.347+0000 7fe934ac9640 1 --2- 172.21.15.164:0/2432263718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e544a760 0x55d8e545a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:58.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.347+0000 7fe9352ca640 1 --2- 172.21.15.164:0/2432263718 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e5455940 0x55d8e54c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:58.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.347+0000 7fe92ffff640 1 -- 172.21.15.164:0/2432263718 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e5455940 msgr2=0x55d8e54c8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:58.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.347+0000 7fe92ffff640 1 --2- 172.21.15.164:0/2432263718 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e5455940 0x55d8e54c8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:58.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.347+0000 7fe92ffff640 1 -- 172.21.15.164:0/2432263718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e544a760 msgr2=0x55d8e545a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:55:58.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.347+0000 7fe92ffff640 1 --2- 172.21.15.164:0/2432263718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e544a760 0x55d8e545a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:58.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.347+0000 7fe92ffff640 1 -- 172.21.15.164:0/2432263718 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8e5442260 con 0x55d8e544b080 2024-09-19T08:55:58.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.347+0000 7fe934ac9640 1 --2- 172.21.15.164:0/2432263718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e544a760 0x55d8e545a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:55:58.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.347+0000 7fe92ffff640 1 --2- 172.21.15.164:0/2432263718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e544b080 0x55d8e545a770 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fe91c09b8a0 tx=0x7fe91c005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df32d6040839a03d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:55:58.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.347+0000 7fe90ffff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:55:58.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.348+0000 7fe90ffff640 1 -- 172.21.15.164:0/2432263718 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c020070 con 0x55d8e544b080 2024-09-19T08:55:58.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.348+0000 7fe90ffff640 1 -- 172.21.15.164:0/2432263718 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fe91c0abbc0 con 0x55d8e544b080 2024-09-19T08:55:58.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.348+0000 7fe939106740 1 -- 172.21.15.164:0/2432263718 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe91c0a4810 con 0x55d8e544b080 2024-09-19T08:55:58.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.348+0000 7fe90ffff640 1 -- 172.21.15.164:0/2432263718 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe91c097e40 con 0x55d8e544b080 2024-09-19T08:55:58.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.348+0000 7fe939106740 10 client.17595 fetch_fsmap learned FSMap version 32 2024-09-19T08:55:58.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.348+0000 7fe939106740 1 -- 172.21.15.164:0/2432263718 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d8e54d0c60 con 0x55d8e544b080 2024-09-19T08:55:58.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.348+0000 7fe90ffff640 1 -- 172.21.15.164:0/2432263718 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe91c0a4810 con 0x55d8e544b080 2024-09-19T08:55:58.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.348+0000 7fe90ffff640 1 -- 172.21.15.164:0/2432263718 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe91c0a10e0 con 0x55d8e544b080 2024-09-19T08:55:58.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.348+0000 7fe939106740 10 client.17595 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:55:58.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.348+0000 7fe939106740 10 client.17595 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:55:58.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.348+0000 7fe939106740 20 client.17595 populate_metadata read hostname 'smithi164' 2024-09-19T08:55:58.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.348+0000 7fe939106740 1 --2- 172.21.15.164:0/2432263718 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55d8e54d11f0 0x55d8e54d35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:55:58.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.348+0000 7fe939106740 4 client.17595 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-19T08:55:58.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.348+0000 7fe939106740 1 -- 172.21.15.164:0/2432263718 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55d8e53b7d20 con 0x55d8e54d11f0 2024-09-19T08:55:58.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.348+0000 7fe934ac9640 1 --2- 172.21.15.164:0/2432263718 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55d8e54d11f0 0x55d8e54d35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:55:58.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.349+0000 7fe90ffff640 10 client.17595 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:55:58.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.349+0000 7fe934ac9640 1 --2- 172.21.15.164:0/2432263718 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55d8e54d11f0 0x55d8e54d35e0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=782482566941b3fc server_cookie=5647acc81e23a8e9 in_seq=0 out_seq=0 2024-09-19T08:55:58.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.364+0000 7fe90ffff640 1 -- 172.21.15.164:0/2432263718 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d8e53b7d20 con 0x55d8e54d11f0 2024-09-19T08:55:58.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.364+0000 7fe90ffff640 10 client.17595 handle_command_reply: tid=0 2024-09-19T08:55:58.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.364+0000 7fe939106740 1 client.17595 shutdown 2024-09-19T08:55:58.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.364+0000 7fe939106740 1 -- 172.21.15.164:0/2432263718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e544b080 msgr2=0x55d8e545a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:58.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.364+0000 7fe939106740 1 --2- 172.21.15.164:0/2432263718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e544b080 0x55d8e545a770 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fe91c09b8a0 tx=0x7fe91c005f70 comp rx=0 tx=0).stop 2024-09-19T08:55:58.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.364+0000 7fe939106740 1 -- 172.21.15.164:0/2432263718 shutdown_connections 2024-09-19T08:55:58.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.364+0000 7fe939106740 1 --2- 172.21.15.164:0/2432263718 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8e5455940 0x55d8e54c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:58.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.364+0000 7fe939106740 1 --2- 172.21.15.164:0/2432263718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d8e544b080 0x55d8e545a770 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:58.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.364+0000 7fe939106740 1 --2- 172.21.15.164:0/2432263718 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55d8e54d11f0 0x55d8e54d35e0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:58.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.365+0000 7fe90ffff640 0 client.17595 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:55:58.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.365+0000 7fe939106740 1 --2- 172.21.15.164:0/2432263718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8e544a760 0x55d8e545a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:58.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.365+0000 7fe939106740 1 -- 172.21.15.164:0/2432263718 >> 172.21.15.164:0/2432263718 conn(0x55d8e5446cf0 msgr2=0x55d8e544c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:58.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.365+0000 7fe939106740 1 -- 172.21.15.164:0/2432263718 shutdown_connections 2024-09-19T08:55:58.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.365+0000 7fe939106740 1 -- 172.21.15.164:0/2432263718 wait complete. 2024-09-19T08:55:58.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.365+0000 7fe939106740 20 client.17595 trim_cache size 0 max 16384 2024-09-19T08:55:58.367 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:55:58.367 INFO:teuthology.orchestra.run.smithi164.stdout: "epoch": 32, 2024-09-19T08:55:58.367 INFO:teuthology.orchestra.run.smithi164.stdout: "leader": 24479, 2024-09-19T08:55:58.367 INFO:teuthology.orchestra.run.smithi164.stdout: "set_version": 34, 2024-09-19T08:55:58.367 INFO:teuthology.orchestra.run.smithi164.stdout: "sets": { 2024-09-19T08:55:58.367 INFO:teuthology.orchestra.run.smithi164.stdout: "60e31c5c": { 2024-09-19T08:55:58.367 INFO:teuthology.orchestra.run.smithi164.stdout: "version": 34, 2024-09-19T08:55:58.367 INFO:teuthology.orchestra.run.smithi164.stdout: "age_ref": 0.0, 2024-09-19T08:55:58.367 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:55:58.367 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "QUIESCED", 2024-09-19T08:55:58.368 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:55:58.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:55:58.368 INFO:teuthology.orchestra.run.smithi164.stdout: "timeout": 90.0, 2024-09-19T08:55:58.368 INFO:teuthology.orchestra.run.smithi164.stdout: "expiration": 150.0, 2024-09-19T08:55:58.368 INFO:teuthology.orchestra.run.smithi164.stdout: "members": { 2024-09-19T08:55:58.368 INFO:teuthology.orchestra.run.smithi164.stdout: "file:/": { 2024-09-19T08:55:58.368 INFO:teuthology.orchestra.run.smithi164.stdout: "excluded": false, 2024-09-19T08:55:58.368 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:55:58.368 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "QUIESCED", 2024-09-19T08:55:58.368 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:55:58.368 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:55:58.368 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:55:58.368 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:55:58.369 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:55:58.369 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:55:58.369 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:55:58.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.365+0000 7fe936d2d640 1 -- 172.21.15.164:0/1595429575 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe9200a33b0 msgr2=0x7fe9200a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:58.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.365+0000 7fe936d2d640 1 --2- 172.21.15.164:0/1595429575 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe9200a33b0 0x7fe9200a5870 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe928005e30 tx=0x7fe928005d80 comp rx=0 tx=0).stop 2024-09-19T08:55:58.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.365+0000 7fe936d2d640 1 -- 172.21.15.164:0/1595429575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe930102570 msgr2=0x7fe930199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:55:58.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.365+0000 7fe936d2d640 1 --2- 172.21.15.164:0/1595429575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe930102570 0x7fe930199870 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00d950 tx=0x7fe91c00de10 comp rx=0 tx=0).stop 2024-09-19T08:55:58.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.366+0000 7fe936d2d640 1 -- 172.21.15.164:0/1595429575 shutdown_connections 2024-09-19T08:55:58.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.366+0000 7fe936d2d640 1 --2- 172.21.15.164:0/1595429575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe930102ef0 0x7fe9301a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:58.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.366+0000 7fe936d2d640 1 --2- 172.21.15.164:0/1595429575 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe9200a33b0 0x7fe9200a5870 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:58.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.366+0000 7fe936d2d640 1 --2- 172.21.15.164:0/1595429575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe930102570 0x7fe930199870 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:58.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.366+0000 7fe936d2d640 1 --2- 172.21.15.164:0/1595429575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe930100e20 0x7fe930199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:55:58.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.366+0000 7fe936d2d640 1 -- 172.21.15.164:0/1595429575 >> 172.21.15.164:0/1595429575 conn(0x7fe9300fc7d0 msgr2=0x7fe930106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:55:58.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.366+0000 7fe936d2d640 1 -- 172.21.15.164:0/1595429575 shutdown_connections 2024-09-19T08:55:58.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:55:58.366+0000 7fe936d2d640 1 -- 172.21.15.164:0/1595429575 wait complete. 2024-09-19T08:55:58.624 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:55:58 smithi170 ceph-mon[35072]: pgmap v315: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:55:58.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:55:58 smithi173 ceph-mon[35306]: pgmap v315: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:55:59.084 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '60e31c5c', quiesce took 3.3 seconds. Will release after: 26.7 2024-09-19T08:56:00.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:00 smithi164 ceph-mon[30928]: pgmap v316: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:56:00.624 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:00 smithi170 ceph-mon[35072]: pgmap v316: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:56:00.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:00 smithi173 ceph-mon[35306]: pgmap v316: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:56:02.144 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:56:02.408 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:56:02.529 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:02 smithi164 ceph-mon[30928]: pgmap v317: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:56:02.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:02 smithi170 ceph-mon[35072]: pgmap v317: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:56:02.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:02 smithi173 ceph-mon[35306]: pgmap v317: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:56:02.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.915+0000 7fd796485640 1 -- 172.21.15.164:0/1514364854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd790102790 msgr2=0x7fd7901067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:02.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.915+0000 7fd796485640 1 --2- 172.21.15.164:0/1514364854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd790102790 0x7fd7901067e0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fd778009900 tx=0x7fd77802a410 comp rx=0 tx=0).stop 2024-09-19T08:56:02.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.915+0000 7fd796485640 1 -- 172.21.15.164:0/1514364854 shutdown_connections 2024-09-19T08:56:02.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.915+0000 7fd796485640 1 --2- 172.21.15.164:0/1514364854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd790106f10 0x7fd79010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:02.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.915+0000 7fd796485640 1 --2- 172.21.15.164:0/1514364854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd790102790 0x7fd7901067e0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:02.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.915+0000 7fd796485640 1 --2- 172.21.15.164:0/1514364854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd790101de0 0x7fd7901021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:02.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.915+0000 7fd796485640 1 -- 172.21.15.164:0/1514364854 >> 172.21.15.164:0/1514364854 conn(0x7fd7900fd8c0 msgr2=0x7fd7900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:02.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.915+0000 7fd796485640 1 -- 172.21.15.164:0/1514364854 shutdown_connections 2024-09-19T08:56:02.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.915+0000 7fd796485640 1 -- 172.21.15.164:0/1514364854 wait complete. 2024-09-19T08:56:02.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.915+0000 7fd796485640 1 Processor -- start 2024-09-19T08:56:02.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.916+0000 7fd796485640 1 -- start start 2024-09-19T08:56:02.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.916+0000 7fd796485640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd790101de0 0x7fd790198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:02.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.916+0000 7fd796485640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd790102790 0x7fd7901989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:02.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.916+0000 7fd796485640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd790106f10 0x7fd79019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:02.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.916+0000 7fd796485640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd790111f90 con 0x7fd790102790 2024-09-19T08:56:02.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.916+0000 7fd796485640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd790111e10 con 0x7fd790101de0 2024-09-19T08:56:02.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.916+0000 7fd796485640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd790112110 con 0x7fd790106f10 2024-09-19T08:56:02.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.916+0000 7fd787fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd790102790 0x7fd7901989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:02.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.916+0000 7fd787fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd790102790 0x7fd7901989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43778/0 (socket says 172.21.15.164:43778) 2024-09-19T08:56:02.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.916+0000 7fd787fff640 1 -- 172.21.15.164:0/1942952678 learned_addr learned my addr 172.21.15.164:0/1942952678 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:02.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.916+0000 7fd78ffff640 1 --2- 172.21.15.164:0/1942952678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd790101de0 0x7fd790198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:02.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.916+0000 7fd794a22640 1 --2- 172.21.15.164:0/1942952678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd790106f10 0x7fd79019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:02.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.916+0000 7fd787fff640 1 -- 172.21.15.164:0/1942952678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd790106f10 msgr2=0x7fd79019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:02.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.916+0000 7fd787fff640 1 --2- 172.21.15.164:0/1942952678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd790106f10 0x7fd79019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:02.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.916+0000 7fd787fff640 1 -- 172.21.15.164:0/1942952678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd790101de0 msgr2=0x7fd790198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:02.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.916+0000 7fd787fff640 1 --2- 172.21.15.164:0/1942952678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd790101de0 0x7fd790198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:02.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.916+0000 7fd787fff640 1 -- 172.21.15.164:0/1942952678 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd79019d440 con 0x7fd790102790 2024-09-19T08:56:02.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.916+0000 7fd787fff640 1 --2- 172.21.15.164:0/1942952678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd790102790 0x7fd7901989d0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fd778006010 tx=0x7fd7780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:02.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.917+0000 7fd78dffb640 1 -- 172.21.15.164:0/1942952678 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77803f030 con 0x7fd790102790 2024-09-19T08:56:02.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.917+0000 7fd78dffb640 1 -- 172.21.15.164:0/1942952678 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd778003ce0 con 0x7fd790102790 2024-09-19T08:56:02.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.917+0000 7fd796485640 1 -- 172.21.15.164:0/1942952678 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd79019d6d0 con 0x7fd790102790 2024-09-19T08:56:02.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.917+0000 7fd78dffb640 1 -- 172.21.15.164:0/1942952678 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd778035070 con 0x7fd790102790 2024-09-19T08:56:02.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.917+0000 7fd796485640 1 -- 172.21.15.164:0/1942952678 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7901a5010 con 0x7fd790102790 2024-09-19T08:56:02.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.919+0000 7fd78dffb640 1 -- 172.21.15.164:0/1942952678 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fd778042030 con 0x7fd790102790 2024-09-19T08:56:02.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.919+0000 7fd79885e740 1 Processor -- start 2024-09-19T08:56:02.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.919+0000 7fd79885e740 1 -- start start 2024-09-19T08:56:02.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.919+0000 7fd79885e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558821ca8360 0x558821ca8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:02.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.919+0000 7fd79885e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821ca8c80 0x558821cb7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:02.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.919+0000 7fd79885e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558821cb7a90 0x558821cb9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:02.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.919+0000 7fd79885e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558821c0e2e0 con 0x558821ca8360 2024-09-19T08:56:02.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.919+0000 7fd79885e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558821bb5ce0 con 0x558821ca8c80 2024-09-19T08:56:02.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.919+0000 7fd79885e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558821c0a2a0 con 0x558821cb7a90 2024-09-19T08:56:02.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.920+0000 7fd794a22640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558821ca8360 0x558821ca8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:02.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.920+0000 7fd78dffb640 1 --2- 172.21.15.164:0/1942952678 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd77c0a33b0 0x7fd77c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:02.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.920+0000 7fd794a22640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558821ca8360 0x558821ca8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43782/0 (socket says 172.21.15.164:43782) 2024-09-19T08:56:02.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.920+0000 7fd794a22640 1 -- 172.21.15.164:0/3693674179 learned_addr learned my addr 172.21.15.164:0/3693674179 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:02.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.920+0000 7fd787fff640 1 --2- 172.21.15.164:0/3693674179 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558821cb7a90 0x558821cb9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:02.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.920+0000 7fd78ffff640 1 --2- 172.21.15.164:0/3693674179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821ca8c80 0x558821cb7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:02.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.920+0000 7fd794a22640 1 --2- 172.21.15.164:0/1942952678 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd77c0a33b0 0x7fd77c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:02.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.920+0000 7fd78dffb640 1 -- 172.21.15.164:0/1942952678 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fd7780f0100 con 0x7fd790102790 2024-09-19T08:56:02.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.920+0000 7fd794a22640 1 -- 172.21.15.164:0/3693674179 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558821cb7a90 msgr2=0x558821cb9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:02.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.920+0000 7fd794a22640 1 --2- 172.21.15.164:0/3693674179 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558821cb7a90 0x558821cb9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:02.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.920+0000 7fd794a22640 1 -- 172.21.15.164:0/3693674179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821ca8c80 msgr2=0x558821cb7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:02.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.920+0000 7fd794a22640 1 --2- 172.21.15.164:0/3693674179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821ca8c80 0x558821cb7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:02.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.920+0000 7fd794a22640 1 -- 172.21.15.164:0/3693674179 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558821b9a440 con 0x558821ca8360 2024-09-19T08:56:02.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.920+0000 7fd794a22640 1 --2- 172.21.15.164:0/3693674179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558821ca8360 0x558821ca8740 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fd7880149c0 tx=0x7fd788014e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab050cd50529c0f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:02.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.920+0000 7fd7867fc640 1 -- 172.21.15.164:0/3693674179 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78801eb40 con 0x558821ca8360 2024-09-19T08:56:02.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.920+0000 7fd7867fc640 1 -- 172.21.15.164:0/3693674179 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd788023070 con 0x558821ca8360 2024-09-19T08:56:02.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.920+0000 7fd7867fc640 1 -- 172.21.15.164:0/3693674179 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78800ce20 con 0x558821ca8360 2024-09-19T08:56:02.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.921+0000 7fd794a22640 1 --2- 172.21.15.164:0/1942952678 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd77c0a33b0 0x7fd77c0a5870 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd788022000 tx=0x7fd788020000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:02.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.921+0000 7fd79885e740 1 -- 172.21.15.164:0/3693674179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558821ca8360 msgr2=0x558821ca8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:02.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.921+0000 7fd79885e740 1 --2- 172.21.15.164:0/3693674179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558821ca8360 0x558821ca8740 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fd7880149c0 tx=0x7fd788014e80 comp rx=0 tx=0).stop 2024-09-19T08:56:02.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.921+0000 7fd79885e740 1 -- 172.21.15.164:0/3693674179 shutdown_connections 2024-09-19T08:56:02.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.921+0000 7fd79885e740 1 --2- 172.21.15.164:0/3693674179 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558821cb7a90 0x558821cb9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:02.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.921+0000 7fd79885e740 1 --2- 172.21.15.164:0/3693674179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821ca8c80 0x558821cb7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:02.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.921+0000 7fd79885e740 1 --2- 172.21.15.164:0/3693674179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558821ca8360 0x558821ca8740 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:02.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.921+0000 7fd79885e740 1 -- 172.21.15.164:0/3693674179 >> 172.21.15.164:0/3693674179 conn(0x558821ca4a30 msgr2=0x558821ca4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:02.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.921+0000 7fd79885e740 1 -- 172.21.15.164:0/3693674179 shutdown_connections 2024-09-19T08:56:02.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.921+0000 7fd79885e740 1 -- 172.21.15.164:0/3693674179 wait complete. 2024-09-19T08:56:02.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.922+0000 7fd79885e740 1 -- start start 2024-09-19T08:56:02.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.922+0000 7fd79885e740 1 Processor -- start 2024-09-19T08:56:02.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.922+0000 7fd79885e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821ca8360 0x558821cb7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:02.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.922+0000 7fd79885e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558821ca8c80 0x558821cb46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:02.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.922+0000 7fd79885e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558821cb7a90 0x558821cb4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:02.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.922+0000 7fd79885e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558821cba660 con 0x558821ca8c80 2024-09-19T08:56:02.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.922+0000 7fd79885e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558821bc9050 con 0x558821ca8360 2024-09-19T08:56:02.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.922+0000 7fd79885e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558821cba7e0 con 0x558821cb7a90 2024-09-19T08:56:02.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.922+0000 7fd79885e740 10 client.0 _ll_register_callbacks cb 0x558821c09160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:56:02.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.922+0000 7fd787fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558821ca8c80 0x558821cb46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:02.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.922+0000 7fd78ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821ca8360 0x558821cb7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:02.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.922+0000 7fd787fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558821ca8c80 0x558821cb46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43788/0 (socket says 172.21.15.164:43788) 2024-09-19T08:56:02.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.922+0000 7fd787fff640 1 -- 172.21.15.164:0/2184172729 learned_addr learned my addr 172.21.15.164:0/2184172729 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:02.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.922+0000 7fd78ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821ca8360 0x558821cb7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:40348/0 (socket says 172.21.15.164:40348) 2024-09-19T08:56:02.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.922+0000 7fd794a22640 1 --2- 172.21.15.164:0/2184172729 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558821cb7a90 0x558821cb4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:02.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.923+0000 7fd787fff640 1 -- 172.21.15.164:0/2184172729 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558821cb7a90 msgr2=0x558821cb4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:02.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.923+0000 7fd787fff640 1 --2- 172.21.15.164:0/2184172729 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558821cb7a90 0x558821cb4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:02.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.923+0000 7fd787fff640 1 -- 172.21.15.164:0/2184172729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821ca8360 msgr2=0x558821cb7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:02.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.923+0000 7fd787fff640 1 --2- 172.21.15.164:0/2184172729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821ca8360 0x558821cb7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:02.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.923+0000 7fd787fff640 1 -- 172.21.15.164:0/2184172729 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55882197f3c0 con 0x558821ca8c80 2024-09-19T08:56:02.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.923+0000 7fd78ffff640 1 --2- 172.21.15.164:0/2184172729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821ca8360 0x558821cb7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:56:02.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.923+0000 7fd787fff640 1 --2- 172.21.15.164:0/2184172729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558821ca8c80 0x558821cb46d0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fd7780ed8f0 tx=0x7fd77808b110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab464b42fad7c73e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:02.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.923+0000 7fd76f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:56:02.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.923+0000 7fd76f7fe640 1 -- 172.21.15.164:0/2184172729 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd778049070 con 0x558821ca8c80 2024-09-19T08:56:02.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.923+0000 7fd76f7fe640 1 -- 172.21.15.164:0/2184172729 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fd77803bc40 con 0x558821ca8c80 2024-09-19T08:56:02.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.923+0000 7fd79885e740 1 -- 172.21.15.164:0/2184172729 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd788023070 con 0x558821ca8c80 2024-09-19T08:56:02.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.923+0000 7fd76f7fe640 1 -- 172.21.15.164:0/2184172729 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd7780bbf60 con 0x558821ca8c80 2024-09-19T08:56:02.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.923+0000 7fd79885e740 10 client.17607 fetch_fsmap learned FSMap version 32 2024-09-19T08:56:02.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.923+0000 7fd79885e740 1 -- 172.21.15.164:0/2184172729 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558821cc6090 con 0x558821ca8c80 2024-09-19T08:56:02.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.923+0000 7fd76f7fe640 1 -- 172.21.15.164:0/2184172729 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd788023070 con 0x558821ca8c80 2024-09-19T08:56:02.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.923+0000 7fd76f7fe640 1 -- 172.21.15.164:0/2184172729 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fd778086dd0 con 0x558821ca8c80 2024-09-19T08:56:02.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.923+0000 7fd79885e740 10 client.17607 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:56:02.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.923+0000 7fd79885e740 10 client.17607 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:56:02.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.924+0000 7fd79885e740 20 client.17607 populate_metadata read hostname 'smithi164' 2024-09-19T08:56:02.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.924+0000 7fd79885e740 1 --2- 172.21.15.164:0/2184172729 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x558821cc74f0 0x558821cc98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:02.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.924+0000 7fd79885e740 4 client.17607 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:56:02.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.924+0000 7fd79885e740 1 -- 172.21.15.164:0/2184172729 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558821c1b850 con 0x558821cc74f0 2024-09-19T08:56:02.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.924+0000 7fd78ffff640 1 --2- 172.21.15.164:0/2184172729 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x558821cc74f0 0x558821cc98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:02.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.924+0000 7fd76f7fe640 10 client.17607 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:56:02.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.924+0000 7fd78ffff640 1 --2- 172.21.15.164:0/2184172729 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x558821cc74f0 0x558821cc98e0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c2c9954775aee7a server_cookie=5070bfcaf2bdaaca in_seq=0 out_seq=0 2024-09-19T08:56:02.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.925+0000 7fd76f7fe640 1 -- 172.21.15.164:0/2184172729 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558821c1b850 con 0x558821cc74f0 2024-09-19T08:56:02.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.925+0000 7fd76f7fe640 10 client.17607 handle_command_reply: tid=0 2024-09-19T08:56:02.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.925+0000 7fd79885e740 1 client.17607 shutdown 2024-09-19T08:56:02.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.925+0000 7fd79885e740 1 -- 172.21.15.164:0/2184172729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558821ca8c80 msgr2=0x558821cb46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:02.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.925+0000 7fd79885e740 1 --2- 172.21.15.164:0/2184172729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558821ca8c80 0x558821cb46d0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fd7780ed8f0 tx=0x7fd77808b110 comp rx=0 tx=0).stop 2024-09-19T08:56:02.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.925+0000 7fd79885e740 1 -- 172.21.15.164:0/2184172729 shutdown_connections 2024-09-19T08:56:02.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.925+0000 7fd79885e740 1 --2- 172.21.15.164:0/2184172729 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558821cb7a90 0x558821cb4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:02.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.925+0000 7fd79885e740 1 --2- 172.21.15.164:0/2184172729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558821ca8c80 0x558821cb46d0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:02.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.925+0000 7fd79885e740 1 --2- 172.21.15.164:0/2184172729 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x558821cc74f0 0x558821cc98e0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:02.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.925+0000 7fd79885e740 1 --2- 172.21.15.164:0/2184172729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821ca8360 0x558821cb7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:02.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.925+0000 7fd76f7fe640 0 client.17607 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:56:02.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.925+0000 7fd79885e740 1 -- 172.21.15.164:0/2184172729 >> 172.21.15.164:0/2184172729 conn(0x558821ca4a30 msgr2=0x558821caa790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:02.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.925+0000 7fd79885e740 1 -- 172.21.15.164:0/2184172729 shutdown_connections 2024-09-19T08:56:02.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.925+0000 7fd79885e740 1 -- 172.21.15.164:0/2184172729 wait complete. 2024-09-19T08:56:02.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.925+0000 7fd79885e740 20 client.17607 trim_cache size 0 max 16384 2024-09-19T08:56:02.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.942+0000 7fd79885e740 1 Processor -- start 2024-09-19T08:56:02.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.942+0000 7fd79885e740 1 -- start start 2024-09-19T08:56:02.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.942+0000 7fd79885e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558821ca8360 0x558821cc3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:02.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.943+0000 7fd79885e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558821ca8c80 0x558821ccf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:02.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.943+0000 7fd79885e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821cb7a90 0x558821ccf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:02.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.943+0000 7fd79885e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd7901ab510 con 0x558821ca8360 2024-09-19T08:56:02.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.943+0000 7fd79885e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd7901ad050 con 0x558821cb7a90 2024-09-19T08:56:02.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.943+0000 7fd79885e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd7901abbe0 con 0x558821ca8c80 2024-09-19T08:56:02.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.943+0000 7fd78ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558821ca8360 0x558821cc3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:02.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.943+0000 7fd78ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558821ca8360 0x558821cc3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43800/0 (socket says 172.21.15.164:43800) 2024-09-19T08:56:02.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.943+0000 7fd78ffff640 1 -- 172.21.15.164:0/626142156 learned_addr learned my addr 172.21.15.164:0/626142156 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:02.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.943+0000 7fd787fff640 1 --2- 172.21.15.164:0/626142156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558821ca8c80 0x558821ccf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:02.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.943+0000 7fd78ffff640 1 -- 172.21.15.164:0/626142156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558821ca8c80 msgr2=0x558821ccf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:02.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.943+0000 7fd78ffff640 1 --2- 172.21.15.164:0/626142156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558821ca8c80 0x558821ccf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:02.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.943+0000 7fd78ffff640 1 -- 172.21.15.164:0/626142156 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821cb7a90 msgr2=0x558821ccf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:02.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.943+0000 7fd78ffff640 1 --2- 172.21.15.164:0/626142156 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821cb7a90 0x558821ccf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:02.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.943+0000 7fd78ffff640 1 -- 172.21.15.164:0/626142156 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558821ca8a70 con 0x558821ca8360 2024-09-19T08:56:02.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.943+0000 7fd78ffff640 1 --2- 172.21.15.164:0/626142156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558821ca8360 0x558821cc3900 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fd780009f30 tx=0x7fd7800047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e793702a6307eb76 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:02.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.943+0000 7fd785ffb640 1 -- 172.21.15.164:0/626142156 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78001f490 con 0x558821ca8360 2024-09-19T08:56:02.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.943+0000 7fd785ffb640 1 -- 172.21.15.164:0/626142156 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd78001fb10 con 0x558821ca8360 2024-09-19T08:56:02.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.943+0000 7fd785ffb640 1 -- 172.21.15.164:0/626142156 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78001e330 con 0x558821ca8360 2024-09-19T08:56:02.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.944+0000 7fd79885e740 1 -- 172.21.15.164:0/626142156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558821ca8360 msgr2=0x558821cc3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:02.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.944+0000 7fd79885e740 1 --2- 172.21.15.164:0/626142156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558821ca8360 0x558821cc3900 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fd780009f30 tx=0x7fd7800047d0 comp rx=0 tx=0).stop 2024-09-19T08:56:02.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.944+0000 7fd79885e740 1 -- 172.21.15.164:0/626142156 shutdown_connections 2024-09-19T08:56:02.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.944+0000 7fd79885e740 1 --2- 172.21.15.164:0/626142156 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821cb7a90 0x558821ccf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:02.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.944+0000 7fd79885e740 1 --2- 172.21.15.164:0/626142156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558821ca8c80 0x558821ccf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:02.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.944+0000 7fd79885e740 1 --2- 172.21.15.164:0/626142156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558821ca8360 0x558821cc3900 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:02.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.944+0000 7fd79885e740 1 -- 172.21.15.164:0/626142156 >> 172.21.15.164:0/626142156 conn(0x558821ca4a30 msgr2=0x558821cafd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:02.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.944+0000 7fd79885e740 1 -- 172.21.15.164:0/626142156 shutdown_connections 2024-09-19T08:56:02.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.944+0000 7fd79885e740 1 -- 172.21.15.164:0/626142156 wait complete. 2024-09-19T08:56:02.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.944+0000 7fd79885e740 1 -- start start 2024-09-19T08:56:02.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.945+0000 7fd79885e740 1 Processor -- start 2024-09-19T08:56:02.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.945+0000 7fd79885e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821ca8360 0x558821cbba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:02.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.945+0000 7fd79885e740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558821ca8c80 0x558821cbbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:02.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.945+0000 7fd79885e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558821cb7a90 0x558821cbc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:02.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.945+0000 7fd79885e740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd7901ac980 con 0x558821ca8c80 2024-09-19T08:56:02.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.945+0000 7fd79885e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd7901ac2b0 con 0x558821ca8360 2024-09-19T08:56:02.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.945+0000 7fd79885e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd7901a9530 con 0x558821cb7a90 2024-09-19T08:56:02.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.945+0000 7fd79885e740 10 client.0 _ll_register_callbacks cb 0x7fd778005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:56:02.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.945+0000 7fd787fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558821ca8c80 0x558821cbbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:02.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.945+0000 7fd787fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558821ca8c80 0x558821cbbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:43804/0 (socket says 172.21.15.164:43804) 2024-09-19T08:56:02.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.945+0000 7fd787fff640 1 -- 172.21.15.164:0/2321682772 learned_addr learned my addr 172.21.15.164:0/2321682772 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:02.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.945+0000 7fd78ffff640 1 --2- 172.21.15.164:0/2321682772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821ca8360 0x558821cbba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:02.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.945+0000 7fd794a22640 1 --2- 172.21.15.164:0/2321682772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558821cb7a90 0x558821cbc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:02.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.945+0000 7fd787fff640 1 -- 172.21.15.164:0/2321682772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558821cb7a90 msgr2=0x558821cbc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:02.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.945+0000 7fd787fff640 1 --2- 172.21.15.164:0/2321682772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558821cb7a90 0x558821cbc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:02.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.945+0000 7fd787fff640 1 -- 172.21.15.164:0/2321682772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821ca8360 msgr2=0x558821cbba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:02.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.945+0000 7fd787fff640 1 --2- 172.21.15.164:0/2321682772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821ca8360 0x558821cbba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:02.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.945+0000 7fd787fff640 1 -- 172.21.15.164:0/2321682772 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558821d31380 con 0x558821ca8c80 2024-09-19T08:56:02.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.945+0000 7fd787fff640 1 --2- 172.21.15.164:0/2321682772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558821ca8c80 0x558821cbbf70 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fd77803d040 tx=0x7fd77803be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582fd1f7f645fabe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:02.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.945+0000 7fd76f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:56:02.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.946+0000 7fd76f7fe640 1 -- 172.21.15.164:0/2321682772 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd778049070 con 0x558821ca8c80 2024-09-19T08:56:02.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.946+0000 7fd76f7fe640 1 -- 172.21.15.164:0/2321682772 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fd778031bd0 con 0x558821ca8c80 2024-09-19T08:56:02.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.946+0000 7fd79885e740 1 -- 172.21.15.164:0/2321682772 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd78001fb10 con 0x558821ca8c80 2024-09-19T08:56:02.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.946+0000 7fd76f7fe640 1 -- 172.21.15.164:0/2321682772 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd7780044e0 con 0x558821ca8c80 2024-09-19T08:56:02.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.946+0000 7fd79885e740 10 client.17613 fetch_fsmap learned FSMap version 32 2024-09-19T08:56:02.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.946+0000 7fd79885e740 1 -- 172.21.15.164:0/2321682772 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558821d32670 con 0x558821ca8c80 2024-09-19T08:56:02.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.946+0000 7fd76f7fe640 1 -- 172.21.15.164:0/2321682772 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd78001fb10 con 0x558821ca8c80 2024-09-19T08:56:02.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.946+0000 7fd76f7fe640 1 -- 172.21.15.164:0/2321682772 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fd7780d5200 con 0x558821ca8c80 2024-09-19T08:56:02.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.946+0000 7fd79885e740 10 client.17613 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:56:02.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.946+0000 7fd79885e740 10 client.17613 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:56:02.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.946+0000 7fd79885e740 20 client.17613 populate_metadata read hostname 'smithi164' 2024-09-19T08:56:02.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.946+0000 7fd79885e740 1 --2- 172.21.15.164:0/2321682772 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x558821d32bd0 0x558821d34ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:02.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.946+0000 7fd79885e740 4 client.17613 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:56:02.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.946+0000 7fd79885e740 1 -- 172.21.15.164:0/2321682772 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558821c15d60 con 0x558821d32bd0 2024-09-19T08:56:02.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.946+0000 7fd78ffff640 1 --2- 172.21.15.164:0/2321682772 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x558821d32bd0 0x558821d34ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:02.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.947+0000 7fd78ffff640 1 --2- 172.21.15.164:0/2321682772 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x558821d32bd0 0x558821d34ff0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc8d2cbe5eb7cce0 server_cookie=c775157511d66fe7 in_seq=0 out_seq=0 2024-09-19T08:56:02.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.947+0000 7fd76f7fe640 10 client.17613 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:56:02.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.947+0000 7fd76f7fe640 1 -- 172.21.15.164:0/2321682772 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558821c15d60 con 0x558821d32bd0 2024-09-19T08:56:02.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.947+0000 7fd76f7fe640 10 client.17613 handle_command_reply: tid=0 2024-09-19T08:56:02.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.947+0000 7fd79885e740 1 client.17613 shutdown 2024-09-19T08:56:02.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.947+0000 7fd79885e740 1 -- 172.21.15.164:0/2321682772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558821ca8c80 msgr2=0x558821cbbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:02.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.947+0000 7fd79885e740 1 --2- 172.21.15.164:0/2321682772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558821ca8c80 0x558821cbbf70 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fd77803d040 tx=0x7fd77803be70 comp rx=0 tx=0).stop 2024-09-19T08:56:02.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.947+0000 7fd79885e740 1 -- 172.21.15.164:0/2321682772 shutdown_connections 2024-09-19T08:56:02.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.947+0000 7fd79885e740 1 --2- 172.21.15.164:0/2321682772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558821cb7a90 0x558821cbc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:02.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.947+0000 7fd79885e740 1 --2- 172.21.15.164:0/2321682772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558821ca8c80 0x558821cbbf70 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:02.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.947+0000 7fd79885e740 1 --2- 172.21.15.164:0/2321682772 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x558821d32bd0 0x558821d34ff0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:02.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.947+0000 7fd79885e740 1 --2- 172.21.15.164:0/2321682772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821ca8360 0x558821cbba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:02.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.947+0000 7fd76f7fe640 0 client.17613 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:56:02.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.947+0000 7fd79885e740 1 -- 172.21.15.164:0/2321682772 >> 172.21.15.164:0/2321682772 conn(0x558821ca4a30 msgr2=0x558821cb5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:02.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.947+0000 7fd79885e740 1 -- 172.21.15.164:0/2321682772 shutdown_connections 2024-09-19T08:56:02.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.947+0000 7fd79885e740 1 -- 172.21.15.164:0/2321682772 wait complete. 2024-09-19T08:56:02.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.947+0000 7fd79885e740 20 client.17613 trim_cache size 0 max 16384 2024-09-19T08:56:02.957 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:56:02.957 INFO:teuthology.orchestra.run.smithi164.stdout: "status": "no active scrubs running", 2024-09-19T08:56:02.957 INFO:teuthology.orchestra.run.smithi164.stdout: "scrubs": {} 2024-09-19T08:56:02.957 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:56:02.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.948+0000 7fd796485640 1 -- 172.21.15.164:0/1942952678 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd77c0a33b0 msgr2=0x7fd77c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:02.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.948+0000 7fd796485640 1 --2- 172.21.15.164:0/1942952678 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd77c0a33b0 0x7fd77c0a5870 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd788022000 tx=0x7fd788020000 comp rx=0 tx=0).stop 2024-09-19T08:56:02.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.948+0000 7fd796485640 1 -- 172.21.15.164:0/1942952678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd790102790 msgr2=0x7fd7901989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:02.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.948+0000 7fd796485640 1 --2- 172.21.15.164:0/1942952678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd790102790 0x7fd7901989d0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fd778006010 tx=0x7fd7780042f0 comp rx=0 tx=0).stop 2024-09-19T08:56:02.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.948+0000 7fd796485640 1 -- 172.21.15.164:0/1942952678 shutdown_connections 2024-09-19T08:56:02.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.948+0000 7fd796485640 1 --2- 172.21.15.164:0/1942952678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd790106f10 0x7fd79019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:02.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.948+0000 7fd796485640 1 --2- 172.21.15.164:0/1942952678 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd77c0a33b0 0x7fd77c0a5870 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:02.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.948+0000 7fd796485640 1 --2- 172.21.15.164:0/1942952678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd790102790 0x7fd7901989d0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:02.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.948+0000 7fd796485640 1 --2- 172.21.15.164:0/1942952678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd790101de0 0x7fd790198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:02.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.948+0000 7fd796485640 1 -- 172.21.15.164:0/1942952678 >> 172.21.15.164:0/1942952678 conn(0x7fd7900fd8c0 msgr2=0x7fd7900fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:02.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.948+0000 7fd796485640 1 -- 172.21.15.164:0/1942952678 shutdown_connections 2024-09-19T08:56:02.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:02.948+0000 7fd796485640 1 -- 172.21.15.164:0/1942952678 wait complete. 2024-09-19T08:56:03.821 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T08:56:03.822 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:56:04.072 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:56:04.492 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:04 smithi164 ceph-mon[30928]: pgmap v318: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-19T08:56:04.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.573+0000 7fc04995c640 1 -- 172.21.15.164:0/3870561848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc044102570 msgr2=0x7fc0441029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:04.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.573+0000 7fc04995c640 1 --2- 172.21.15.164:0/3870561848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc044102570 0x7fc0441029b0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009900 tx=0x7fc02c02a480 comp rx=0 tx=0).stop 2024-09-19T08:56:04.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.573+0000 7fc04995c640 1 -- 172.21.15.164:0/3870561848 shutdown_connections 2024-09-19T08:56:04.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.573+0000 7fc04995c640 1 --2- 172.21.15.164:0/3870561848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc044102ef0 0x7fc04410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:04.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.573+0000 7fc04995c640 1 --2- 172.21.15.164:0/3870561848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc044102570 0x7fc0441029b0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:04.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.573+0000 7fc04995c640 1 --2- 172.21.15.164:0/3870561848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc044100e20 0x7fc044101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:04.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.573+0000 7fc04995c640 1 -- 172.21.15.164:0/3870561848 >> 172.21.15.164:0/3870561848 conn(0x7fc0440fc7d0 msgr2=0x7fc0440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:04.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.573+0000 7fc04995c640 1 -- 172.21.15.164:0/3870561848 shutdown_connections 2024-09-19T08:56:04.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.573+0000 7fc04995c640 1 -- 172.21.15.164:0/3870561848 wait complete. 2024-09-19T08:56:04.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.573+0000 7fc04995c640 1 Processor -- start 2024-09-19T08:56:04.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.574+0000 7fc04995c640 1 -- start start 2024-09-19T08:56:04.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.574+0000 7fc04995c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc044100e20 0x7fc044197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:04.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.574+0000 7fc04995c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc044102570 0x7fc044197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:04.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.574+0000 7fc04995c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc044102ef0 0x7fc04419bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:04.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.574+0000 7fc04995c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc044110f80 con 0x7fc044102570 2024-09-19T08:56:04.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.574+0000 7fc04995c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc044110e00 con 0x7fc044102ef0 2024-09-19T08:56:04.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.574+0000 7fc04995c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc044111100 con 0x7fc044100e20 2024-09-19T08:56:04.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.574+0000 7fc0427fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc044102570 0x7fc044197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:04.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.574+0000 7fc0427fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc044102570 0x7fc044197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50014/0 (socket says 172.21.15.164:50014) 2024-09-19T08:56:04.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.574+0000 7fc0427fc640 1 -- 172.21.15.164:0/3554877250 learned_addr learned my addr 172.21.15.164:0/3554877250 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:04.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.574+0000 7fc042ffd640 1 --2- 172.21.15.164:0/3554877250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc044100e20 0x7fc044197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:04.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.574+0000 7fc0437fe640 1 --2- 172.21.15.164:0/3554877250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc044102ef0 0x7fc04419bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:04.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.574+0000 7fc0427fc640 1 -- 172.21.15.164:0/3554877250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc044100e20 msgr2=0x7fc044197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:04.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.574+0000 7fc0427fc640 1 --2- 172.21.15.164:0/3554877250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc044100e20 0x7fc044197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:04.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.574+0000 7fc0427fc640 1 -- 172.21.15.164:0/3554877250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc044102ef0 msgr2=0x7fc04419bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:04.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.574+0000 7fc0427fc640 1 --2- 172.21.15.164:0/3554877250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc044102ef0 0x7fc04419bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:04.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.574+0000 7fc0427fc640 1 -- 172.21.15.164:0/3554877250 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04419c3d0 con 0x7fc044102570 2024-09-19T08:56:04.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.575+0000 7fc0427fc640 1 --2- 172.21.15.164:0/3554877250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc044102570 0x7fc044197960 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fc02c006010 tx=0x7fc02c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:04.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.575+0000 7fc04895a640 1 -- 172.21.15.164:0/3554877250 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c03f070 con 0x7fc044102570 2024-09-19T08:56:04.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.575+0000 7fc04995c640 1 -- 172.21.15.164:0/3554877250 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc04419c660 con 0x7fc044102570 2024-09-19T08:56:04.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.575+0000 7fc04895a640 1 -- 172.21.15.164:0/3554877250 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc02c031c30 con 0x7fc044102570 2024-09-19T08:56:04.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.575+0000 7fc04895a640 1 -- 172.21.15.164:0/3554877250 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c035030 con 0x7fc044102570 2024-09-19T08:56:04.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.575+0000 7fc04995c640 1 -- 172.21.15.164:0/3554877250 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0441a3fa0 con 0x7fc044102570 2024-09-19T08:56:04.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.576+0000 7fc04995c640 1 -- 172.21.15.164:0/3554877250 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc044101200 con 0x7fc044102570 2024-09-19T08:56:04.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.577+0000 7fc04895a640 1 -- 172.21.15.164:0/3554877250 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fc02c042070 con 0x7fc044102570 2024-09-19T08:56:04.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.578+0000 7fc04895a640 1 --2- 172.21.15.164:0/3554877250 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fc0300a3430 0x7fc0300a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:04.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.578+0000 7fc042ffd640 1 --2- 172.21.15.164:0/3554877250 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fc0300a3430 0x7fc0300a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:04.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.578+0000 7fc04895a640 1 -- 172.21.15.164:0/3554877250 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7fc02c0f04c0 con 0x7fc044102570 2024-09-19T08:56:04.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.578+0000 7fc042ffd640 1 --2- 172.21.15.164:0/3554877250 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fc0300a3430 0x7fc0300a58f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc034005fd0 tx=0x7fc034009680 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:04.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.582+0000 7fc04895a640 1 -- 172.21.15.164:0/3554877250 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc02c031de0 con 0x7fc044102570 2024-09-19T08:56:04.624 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:04 smithi170 ceph-mon[35072]: pgmap v318: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-19T08:56:04.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:04 smithi173 ceph-mon[35306]: pgmap v318: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-19T08:56:04.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.782+0000 7fc04995c640 1 -- 172.21.15.164:0/3554877250 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc044109090 con 0x7fc044102570 2024-09-19T08:56:04.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.783+0000 7fc04895a640 1 -- 172.21.15.164:0/3554877250 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11984 (secure 0 0 0) 0x7fc02c0bcee0 con 0x7fc044102570 2024-09-19T08:56:04.783 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:56:04.783 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":32,"btime":"2024-09-19T08:52:27:192804+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:52:27.074168+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:56:04.784 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 32 2024-09-19T08:56:04.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.785+0000 7fc04995c640 1 -- 172.21.15.164:0/3554877250 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fc0300a3430 msgr2=0x7fc0300a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:04.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.785+0000 7fc04995c640 1 --2- 172.21.15.164:0/3554877250 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fc0300a3430 0x7fc0300a58f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc034005fd0 tx=0x7fc034009680 comp rx=0 tx=0).stop 2024-09-19T08:56:04.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.785+0000 7fc04995c640 1 -- 172.21.15.164:0/3554877250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc044102570 msgr2=0x7fc044197960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:04.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.785+0000 7fc04995c640 1 --2- 172.21.15.164:0/3554877250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc044102570 0x7fc044197960 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fc02c006010 tx=0x7fc02c003c10 comp rx=0 tx=0).stop 2024-09-19T08:56:04.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.785+0000 7fc04995c640 1 -- 172.21.15.164:0/3554877250 shutdown_connections 2024-09-19T08:56:04.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.785+0000 7fc04995c640 1 --2- 172.21.15.164:0/3554877250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc044102ef0 0x7fc04419bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:04.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.785+0000 7fc04995c640 1 --2- 172.21.15.164:0/3554877250 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fc0300a3430 0x7fc0300a58f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:04.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.785+0000 7fc04995c640 1 --2- 172.21.15.164:0/3554877250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc044102570 0x7fc044197960 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:04.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.785+0000 7fc04995c640 1 --2- 172.21.15.164:0/3554877250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc044100e20 0x7fc044197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:04.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.785+0000 7fc04995c640 1 -- 172.21.15.164:0/3554877250 >> 172.21.15.164:0/3554877250 conn(0x7fc0440fc7d0 msgr2=0x7fc044105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:04.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.785+0000 7fc04995c640 1 -- 172.21.15.164:0/3554877250 shutdown_connections 2024-09-19T08:56:04.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:04.785+0000 7fc04995c640 1 -- 172.21.15.164:0/3554877250 wait complete. 2024-09-19T08:56:05.447 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:05 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3554877250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:56:05.476 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T08:56:05.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:05 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3554877250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:56:05.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:05 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3554877250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:56:05.723 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:56:06.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.191+0000 7f3f32e94640 1 -- 172.21.15.164:0/99936484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f2c101df0 msgr2=0x7f3f2c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:06.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.191+0000 7f3f32e94640 1 --2- 172.21.15.164:0/99936484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f2c101df0 0x7f3f2c1021d0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f3f20009900 tx=0x7f3f2002a410 comp rx=0 tx=0).stop 2024-09-19T08:56:06.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.191+0000 7f3f32e94640 1 -- 172.21.15.164:0/99936484 shutdown_connections 2024-09-19T08:56:06.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.191+0000 7f3f32e94640 1 --2- 172.21.15.164:0/99936484 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f2c106f20 0x7f3f2c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.191+0000 7f3f32e94640 1 --2- 172.21.15.164:0/99936484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f2c1027a0 0x7f3f2c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.191+0000 7f3f32e94640 1 --2- 172.21.15.164:0/99936484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f2c101df0 0x7f3f2c1021d0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.191+0000 7f3f32e94640 1 -- 172.21.15.164:0/99936484 >> 172.21.15.164:0/99936484 conn(0x7f3f2c0fd8f0 msgr2=0x7f3f2c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:06.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.191+0000 7f3f32e94640 1 -- 172.21.15.164:0/99936484 shutdown_connections 2024-09-19T08:56:06.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.191+0000 7f3f32e94640 1 -- 172.21.15.164:0/99936484 wait complete. 2024-09-19T08:56:06.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.192+0000 7f3f32e94640 1 Processor -- start 2024-09-19T08:56:06.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.192+0000 7f3f32e94640 1 -- start start 2024-09-19T08:56:06.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.192+0000 7f3f32e94640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f2c101df0 0x7f3f2c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:06.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.192+0000 7f3f32e94640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f2c1027a0 0x7f3f2c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:06.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.192+0000 7f3f32e94640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f2c106f20 0x7f3f2c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:06.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.192+0000 7f3f32e94640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c111f80 con 0x7f3f2c106f20 2024-09-19T08:56:06.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.192+0000 7f3f32e94640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c111e00 con 0x7f3f2c1027a0 2024-09-19T08:56:06.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.192+0000 7f3f32e94640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c112100 con 0x7f3f2c101df0 2024-09-19T08:56:06.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.192+0000 7f3f31431640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f2c106f20 0x7f3f2c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:06.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.193+0000 7f3f31431640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f2c106f20 0x7f3f2c19cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50036/0 (socket says 172.21.15.164:50036) 2024-09-19T08:56:06.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.193+0000 7f3f31431640 1 -- 172.21.15.164:0/936847187 learned_addr learned my addr 172.21.15.164:0/936847187 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:06.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.193+0000 7f3f2bfff640 1 --2- 172.21.15.164:0/936847187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f2c1027a0 0x7f3f2c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:06.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.193+0000 7f3f30c30640 1 --2- 172.21.15.164:0/936847187 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f2c101df0 0x7f3f2c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:06.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.193+0000 7f3f2bfff640 1 -- 172.21.15.164:0/936847187 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f2c101df0 msgr2=0x7f3f2c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:06.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.193+0000 7f3f2bfff640 1 --2- 172.21.15.164:0/936847187 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f2c101df0 0x7f3f2c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.193+0000 7f3f2bfff640 1 -- 172.21.15.164:0/936847187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f2c106f20 msgr2=0x7f3f2c19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:06.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.193+0000 7f3f2bfff640 1 --2- 172.21.15.164:0/936847187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f2c106f20 0x7f3f2c19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.193+0000 7f3f2bfff640 1 -- 172.21.15.164:0/936847187 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f2c19d410 con 0x7f3f2c1027a0 2024-09-19T08:56:06.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.193+0000 7f3f31431640 1 --2- 172.21.15.164:0/936847187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f2c106f20 0x7f3f2c19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:56:06.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.193+0000 7f3f2bfff640 1 --2- 172.21.15.164:0/936847187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f2c1027a0 0x7f3f2c1989a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3f1800e9f0 tx=0x7f3f1800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:06.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.193+0000 7f3f29ffb640 1 -- 172.21.15.164:0/936847187 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1800ccc0 con 0x7f3f2c1027a0 2024-09-19T08:56:06.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.193+0000 7f3f32e94640 1 -- 172.21.15.164:0/936847187 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f2c19d700 con 0x7f3f2c1027a0 2024-09-19T08:56:06.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.193+0000 7f3f29ffb640 1 -- 172.21.15.164:0/936847187 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f18010910 con 0x7f3f2c1027a0 2024-09-19T08:56:06.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.193+0000 7f3f29ffb640 1 -- 172.21.15.164:0/936847187 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f18018960 con 0x7f3f2c1027a0 2024-09-19T08:56:06.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.194+0000 7f3f32e94640 1 -- 172.21.15.164:0/936847187 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f2c1a4fe0 con 0x7f3f2c1027a0 2024-09-19T08:56:06.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.196+0000 7f3f29ffb640 1 -- 172.21.15.164:0/936847187 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f3f18010ac0 con 0x7f3f2c1027a0 2024-09-19T08:56:06.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.196+0000 7f3f3526d740 1 Processor -- start 2024-09-19T08:56:06.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.196+0000 7f3f3526d740 1 -- start start 2024-09-19T08:56:06.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.197+0000 7f3f3526d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e7e5700b40 0x55e7e56ff090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:06.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.197+0000 7f3f3526d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7e56ff5d0 0x55e7e56ffa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:06.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.197+0000 7f3f3526d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e570e780 0x55e7e5710b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:06.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.197+0000 7f3f3526d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e7e56652f0 con 0x55e7e570e780 2024-09-19T08:56:06.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.197+0000 7f3f3526d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e7e560ccf0 con 0x55e7e56ff5d0 2024-09-19T08:56:06.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.197+0000 7f3f3526d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e7e56612b0 con 0x55e7e5700b40 2024-09-19T08:56:06.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.197+0000 7f3f29ffb640 1 --2- 172.21.15.164:0/936847187 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3f1c0a33b0 0x7f3f1c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:06.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.197+0000 7f3f30c30640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7e56ff5d0 0x55e7e56ffa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:06.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.197+0000 7f3f2bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e570e780 0x55e7e5710b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:06.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.197+0000 7f3f31431640 1 --2- 172.21.15.164:0/936847187 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3f1c0a33b0 0x7f3f1c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:06.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.197+0000 7f3f30c30640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7e56ff5d0 0x55e7e56ffa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:50094/0 (socket says 172.21.15.164:50094) 2024-09-19T08:56:06.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.197+0000 7f3f30c30640 1 -- 172.21.15.164:0/1845281726 learned_addr learned my addr 172.21.15.164:0/1845281726 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:06.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.197+0000 7f3f29ffb640 1 -- 172.21.15.164:0/936847187 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f3f180d00d0 con 0x7f3f2c1027a0 2024-09-19T08:56:06.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.197+0000 7f3f31431640 1 --2- 172.21.15.164:0/1845281726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e7e5700b40 0x55e7e56ff090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:06.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.197+0000 7f3f2bfff640 1 -- 172.21.15.164:0/1845281726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e7e5700b40 msgr2=0x55e7e56ff090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:06.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.197+0000 7f3f2bfff640 1 --2- 172.21.15.164:0/1845281726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e7e5700b40 0x55e7e56ff090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.197+0000 7f3f2bfff640 1 -- 172.21.15.164:0/1845281726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7e56ff5d0 msgr2=0x55e7e56ffa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:06.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.197+0000 7f3f2bfff640 1 --2- 172.21.15.164:0/1845281726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7e56ff5d0 0x55e7e56ffa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.197+0000 7f3f2bfff640 1 -- 172.21.15.164:0/1845281726 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7e55f13a0 con 0x55e7e570e780 2024-09-19T08:56:06.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.197+0000 7f3f30c30640 1 --2- 172.21.15.164:0/1845281726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7e56ff5d0 0x55e7e56ffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:56:06.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.198+0000 7f3f31431640 1 --2- 172.21.15.164:0/936847187 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3f1c0a33b0 0x7f3f1c0a5870 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3f240049d0 tx=0x7f3f240116d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:06.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.198+0000 7f3f2bfff640 1 --2- 172.21.15.164:0/1845281726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e570e780 0x55e7e5710b70 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f3f180cd800 tx=0x7f3f1800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a55fd2e456a058ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:06.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.198+0000 7f3f16ffd640 1 -- 172.21.15.164:0/1845281726 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f180caee0 con 0x55e7e570e780 2024-09-19T08:56:06.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.198+0000 7f3f16ffd640 1 -- 172.21.15.164:0/1845281726 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f1809c440 con 0x55e7e570e780 2024-09-19T08:56:06.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.198+0000 7f3f16ffd640 1 -- 172.21.15.164:0/1845281726 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f18098ab0 con 0x55e7e570e780 2024-09-19T08:56:06.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.199+0000 7f3f3526d740 1 -- 172.21.15.164:0/1845281726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e570e780 msgr2=0x55e7e5710b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:06.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.199+0000 7f3f3526d740 1 --2- 172.21.15.164:0/1845281726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e570e780 0x55e7e5710b70 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f3f180cd800 tx=0x7f3f1800cf60 comp rx=0 tx=0).stop 2024-09-19T08:56:06.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.199+0000 7f3f3526d740 1 -- 172.21.15.164:0/1845281726 shutdown_connections 2024-09-19T08:56:06.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.199+0000 7f3f3526d740 1 --2- 172.21.15.164:0/1845281726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e570e780 0x55e7e5710b70 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.199+0000 7f3f3526d740 1 --2- 172.21.15.164:0/1845281726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7e56ff5d0 0x55e7e56ffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.199+0000 7f3f3526d740 1 --2- 172.21.15.164:0/1845281726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e7e5700b40 0x55e7e56ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.199+0000 7f3f3526d740 1 -- 172.21.15.164:0/1845281726 >> 172.21.15.164:0/1845281726 conn(0x55e7e56c8310 msgr2=0x55e7e56fbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:06.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.199+0000 7f3f3526d740 1 -- 172.21.15.164:0/1845281726 shutdown_connections 2024-09-19T08:56:06.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.199+0000 7f3f3526d740 1 -- 172.21.15.164:0/1845281726 wait complete. 2024-09-19T08:56:06.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.199+0000 7f3f3526d740 1 -- start start 2024-09-19T08:56:06.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.199+0000 7f3f3526d740 1 Processor -- start 2024-09-19T08:56:06.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.200+0000 7f3f3526d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e56ff5d0 0x55e7e570e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:06.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.200+0000 7f3f3526d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7e5700b40 0x55e7e570b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:06.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.200+0000 7f3f3526d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e7e570e780 0x55e7e570b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:06.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.200+0000 7f3f3526d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e7e5711480 con 0x55e7e56ff5d0 2024-09-19T08:56:06.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.200+0000 7f3f30c30640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e56ff5d0 0x55e7e570e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:06.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.200+0000 7f3f30c30640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e56ff5d0 0x55e7e570e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50058/0 (socket says 172.21.15.164:50058) 2024-09-19T08:56:06.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.200+0000 7f3f30c30640 1 -- 172.21.15.164:0/172887508 learned_addr learned my addr 172.21.15.164:0/172887508 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:06.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.200+0000 7f3f3526d740 1 -- 172.21.15.164:0/172887508 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e7e5620060 con 0x55e7e5700b40 2024-09-19T08:56:06.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.200+0000 7f3f31431640 1 --2- 172.21.15.164:0/172887508 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e7e570e780 0x55e7e570b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:06.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.200+0000 7f3f2bfff640 1 --2- 172.21.15.164:0/172887508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7e5700b40 0x55e7e570b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:06.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.200+0000 7f3f3526d740 1 -- 172.21.15.164:0/172887508 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e7e5711600 con 0x55e7e570e780 2024-09-19T08:56:06.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.200+0000 7f3f3526d740 10 client.0 _ll_register_callbacks cb 0x55e7e53738c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:56:06.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.200+0000 7f3f30c30640 1 -- 172.21.15.164:0/172887508 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e7e570e780 msgr2=0x55e7e570b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:06.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.200+0000 7f3f30c30640 1 --2- 172.21.15.164:0/172887508 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e7e570e780 0x55e7e570b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.200+0000 7f3f30c30640 1 -- 172.21.15.164:0/172887508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7e5700b40 msgr2=0x55e7e570b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:06.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.200+0000 7f3f30c30640 1 --2- 172.21.15.164:0/172887508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7e5700b40 0x55e7e570b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.200+0000 7f3f30c30640 1 -- 172.21.15.164:0/172887508 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7e53d63c0 con 0x55e7e56ff5d0 2024-09-19T08:56:06.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.200+0000 7f3f3526d740 1 -- 172.21.15.164:0/172887508 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f1809c440 con 0x55e7e56ff5d0 2024-09-19T08:56:06.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.200+0000 7f3f2bfff640 1 --2- 172.21.15.164:0/172887508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7e5700b40 0x55e7e570b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:56:06.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.201+0000 7f3f30c30640 1 --2- 172.21.15.164:0/172887508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e56ff5d0 0x55e7e570e150 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f3f20005940 tx=0x7f3f200317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e082c4d1d0a406d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:06.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.201+0000 7f3f0bfff640 10 client.17631 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:56:06.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.201+0000 7f3f0bfff640 1 -- 172.21.15.164:0/172887508 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f20035040 con 0x55e7e56ff5d0 2024-09-19T08:56:06.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.201+0000 7f3f0bfff640 1 -- 172.21.15.164:0/172887508 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f3f20004260 con 0x55e7e56ff5d0 2024-09-19T08:56:06.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.201+0000 7f3f0bfff640 1 -- 172.21.15.164:0/172887508 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3f200044a0 con 0x55e7e56ff5d0 2024-09-19T08:56:06.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.201+0000 7f3f3526d740 10 client.17631 fetch_fsmap learned FSMap version 32 2024-09-19T08:56:06.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.201+0000 7f3f3526d740 1 -- 172.21.15.164:0/172887508 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55e7e571cea0 con 0x55e7e56ff5d0 2024-09-19T08:56:06.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.204+0000 7f3f0bfff640 1 -- 172.21.15.164:0/172887508 <== mon.0 v2:172.21.15.164:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f3f20005040 con 0x55e7e56ff5d0 2024-09-19T08:56:06.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.204+0000 7f3f3526d740 10 client.17631 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:56:06.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.204+0000 7f3f3526d740 10 client.17631 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-19T08:56:06.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.204+0000 7f3f3526d740 20 client.17631 populate_metadata read hostname 'smithi164' 2024-09-19T08:56:06.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.204+0000 7f3f3526d740 1 --2- 172.21.15.164:0/172887508 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55e7e571e2c0 0x55e7e57206b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:06.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.204+0000 7f3f3526d740 4 client.17631 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:56:06.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.204+0000 7f3f3526d740 1 -- 172.21.15.164:0/172887508 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7e5672860 con 0x55e7e571e2c0 2024-09-19T08:56:06.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.204+0000 7f3f2bfff640 1 --2- 172.21.15.164:0/172887508 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55e7e571e2c0 0x55e7e57206b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:06.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.205+0000 7f3f2bfff640 1 --2- 172.21.15.164:0/172887508 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55e7e571e2c0 0x55e7e57206b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=421c268803dc05b9 server_cookie=ffcb35076ce4032f in_seq=0 out_seq=0 2024-09-19T08:56:06.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.205+0000 7f3f0bfff640 10 client.17631 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:56:06.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.206+0000 7f3f0bfff640 1 -- 172.21.15.164:0/172887508 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e7e5672860 con 0x55e7e571e2c0 2024-09-19T08:56:06.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.206+0000 7f3f0bfff640 10 client.17631 handle_command_reply: tid=0 2024-09-19T08:56:06.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.206+0000 7f3f3526d740 1 client.17631 shutdown 2024-09-19T08:56:06.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.206+0000 7f3f3526d740 1 -- 172.21.15.164:0/172887508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e56ff5d0 msgr2=0x55e7e570e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:06.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.206+0000 7f3f3526d740 1 --2- 172.21.15.164:0/172887508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e56ff5d0 0x55e7e570e150 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f3f20005940 tx=0x7f3f200317e0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.206+0000 7f3f3526d740 1 -- 172.21.15.164:0/172887508 shutdown_connections 2024-09-19T08:56:06.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.206+0000 7f3f3526d740 1 --2- 172.21.15.164:0/172887508 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55e7e571e2c0 0x55e7e57206b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.206+0000 7f3f0bfff640 0 client.17631 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:56:06.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.206+0000 7f3f3526d740 1 --2- 172.21.15.164:0/172887508 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e7e570e780 0x55e7e570b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.206+0000 7f3f3526d740 1 --2- 172.21.15.164:0/172887508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7e5700b40 0x55e7e570b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.206+0000 7f3f3526d740 1 --2- 172.21.15.164:0/172887508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e56ff5d0 0x55e7e570e150 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.206+0000 7f3f3526d740 1 -- 172.21.15.164:0/172887508 >> 172.21.15.164:0/172887508 conn(0x55e7e56c8310 msgr2=0x55e7e5701990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:06.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.206+0000 7f3f3526d740 1 -- 172.21.15.164:0/172887508 shutdown_connections 2024-09-19T08:56:06.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.206+0000 7f3f3526d740 1 -- 172.21.15.164:0/172887508 wait complete. 2024-09-19T08:56:06.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.206+0000 7f3f3526d740 20 client.17631 trim_cache size 0 max 16384 2024-09-19T08:56:06.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.219+0000 7f3f3526d740 1 Processor -- start 2024-09-19T08:56:06.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.219+0000 7f3f3526d740 1 -- start start 2024-09-19T08:56:06.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.219+0000 7f3f3526d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e56ff5d0 0x55e7e570b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:06.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.219+0000 7f3f3526d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e7e5700b40 0x55e7e571a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:06.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.219+0000 7f3f3526d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7e570e780 0x55e7e5725f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:06.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.220+0000 7f3f3526d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c1ab5b0 con 0x55e7e56ff5d0 2024-09-19T08:56:06.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.220+0000 7f3f3526d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c1ad0f0 con 0x55e7e570e780 2024-09-19T08:56:06.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.220+0000 7f3f3526d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c1abc80 con 0x55e7e5700b40 2024-09-19T08:56:06.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.220+0000 7f3f30c30640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e56ff5d0 0x55e7e570b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:06.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.220+0000 7f3f30c30640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e56ff5d0 0x55e7e570b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50064/0 (socket says 172.21.15.164:50064) 2024-09-19T08:56:06.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.220+0000 7f3f30c30640 1 -- 172.21.15.164:0/1448747747 learned_addr learned my addr 172.21.15.164:0/1448747747 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:06.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.220+0000 7f3f2bfff640 1 --2- 172.21.15.164:0/1448747747 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e7e5700b40 0x55e7e571a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:06.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.220+0000 7f3f31431640 1 --2- 172.21.15.164:0/1448747747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7e570e780 0x55e7e5725f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:06.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.220+0000 7f3f30c30640 1 -- 172.21.15.164:0/1448747747 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e7e5700b40 msgr2=0x55e7e571a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:06.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.220+0000 7f3f30c30640 1 --2- 172.21.15.164:0/1448747747 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e7e5700b40 0x55e7e571a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.220+0000 7f3f30c30640 1 -- 172.21.15.164:0/1448747747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7e570e780 msgr2=0x55e7e5725f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:06.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.220+0000 7f3f30c30640 1 --2- 172.21.15.164:0/1448747747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7e570e780 0x55e7e5725f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.220+0000 7f3f30c30640 1 -- 172.21.15.164:0/1448747747 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7e57114d0 con 0x55e7e56ff5d0 2024-09-19T08:56:06.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.220+0000 7f3f30c30640 1 --2- 172.21.15.164:0/1448747747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e56ff5d0 0x55e7e570b3d0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f3f20009850 tx=0x7f3f2003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519747fbff921d22 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:06.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.220+0000 7f3f167fc640 1 -- 172.21.15.164:0/1448747747 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f20035030 con 0x55e7e56ff5d0 2024-09-19T08:56:06.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.220+0000 7f3f167fc640 1 -- 172.21.15.164:0/1448747747 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f1809c440 con 0x55e7e56ff5d0 2024-09-19T08:56:06.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.220+0000 7f3f167fc640 1 -- 172.21.15.164:0/1448747747 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f20046470 con 0x55e7e56ff5d0 2024-09-19T08:56:06.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.221+0000 7f3f3526d740 1 -- 172.21.15.164:0/1448747747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e56ff5d0 msgr2=0x55e7e570b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:06.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.221+0000 7f3f3526d740 1 --2- 172.21.15.164:0/1448747747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e56ff5d0 0x55e7e570b3d0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f3f20009850 tx=0x7f3f2003f000 comp rx=0 tx=0).stop 2024-09-19T08:56:06.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.221+0000 7f3f3526d740 1 -- 172.21.15.164:0/1448747747 shutdown_connections 2024-09-19T08:56:06.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.221+0000 7f3f3526d740 1 --2- 172.21.15.164:0/1448747747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7e570e780 0x55e7e5725f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.221+0000 7f3f3526d740 1 --2- 172.21.15.164:0/1448747747 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e7e5700b40 0x55e7e571a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.221+0000 7f3f3526d740 1 --2- 172.21.15.164:0/1448747747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e56ff5d0 0x55e7e570b3d0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.221+0000 7f3f3526d740 1 -- 172.21.15.164:0/1448747747 >> 172.21.15.164:0/1448747747 conn(0x55e7e56c8310 msgr2=0x55e7e570acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:06.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.221+0000 7f3f3526d740 1 -- 172.21.15.164:0/1448747747 shutdown_connections 2024-09-19T08:56:06.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.221+0000 7f3f3526d740 1 -- 172.21.15.164:0/1448747747 wait complete. 2024-09-19T08:56:06.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.222+0000 7f3f3526d740 1 -- start start 2024-09-19T08:56:06.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.222+0000 7f3f3526d740 1 Processor -- start 2024-09-19T08:56:06.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.222+0000 7f3f3526d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e7e56ff5d0 0x55e7e5711850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:06.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.222+0000 7f3f3526d740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e5700b40 0x55e7e5711d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:06.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.222+0000 7f3f3526d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7e570e780 0x55e7e57122d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:06.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.222+0000 7f3f3526d740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c1aca20 con 0x55e7e5700b40 2024-09-19T08:56:06.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.222+0000 7f3f3526d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c1ac350 con 0x55e7e570e780 2024-09-19T08:56:06.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.222+0000 7f3f3526d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c1a95b0 con 0x55e7e56ff5d0 2024-09-19T08:56:06.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.222+0000 7f3f3526d740 10 client.0 _ll_register_callbacks cb 0x7f3f20009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:56:06.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.222+0000 7f3f2bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e5700b40 0x55e7e5711d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:06.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.222+0000 7f3f2bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e5700b40 0x55e7e5711d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50072/0 (socket says 172.21.15.164:50072) 2024-09-19T08:56:06.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.222+0000 7f3f2bfff640 1 -- 172.21.15.164:0/3888577057 learned_addr learned my addr 172.21.15.164:0/3888577057 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:06.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.222+0000 7f3f30c30640 1 --2- 172.21.15.164:0/3888577057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e7e56ff5d0 0x55e7e5711850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:06.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.222+0000 7f3f31431640 1 --2- 172.21.15.164:0/3888577057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7e570e780 0x55e7e57122d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:06.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.223+0000 7f3f2bfff640 1 -- 172.21.15.164:0/3888577057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e7e56ff5d0 msgr2=0x55e7e5711850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:06.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.223+0000 7f3f2bfff640 1 --2- 172.21.15.164:0/3888577057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e7e56ff5d0 0x55e7e5711850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.223+0000 7f3f2bfff640 1 -- 172.21.15.164:0/3888577057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7e570e780 msgr2=0x55e7e57122d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:06.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.223+0000 7f3f2bfff640 1 --2- 172.21.15.164:0/3888577057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7e570e780 0x55e7e57122d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.223+0000 7f3f2bfff640 1 -- 172.21.15.164:0/3888577057 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7e5787140 con 0x55e7e5700b40 2024-09-19T08:56:06.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.223+0000 7f3f2bfff640 1 --2- 172.21.15.164:0/3888577057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e5700b40 0x55e7e5711d90 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f3f18010f50 tx=0x7f3f1800cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cacf44512fe292ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:06.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.223+0000 7f3f0bfff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:56:06.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.223+0000 7f3f0bfff640 1 -- 172.21.15.164:0/3888577057 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f180cdc80 con 0x55e7e5700b40 2024-09-19T08:56:06.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.223+0000 7f3f3526d740 1 -- 172.21.15.164:0/3888577057 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f1809c440 con 0x55e7e5700b40 2024-09-19T08:56:06.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.223+0000 7f3f0bfff640 1 -- 172.21.15.164:0/3888577057 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8455+0+0 (secure 0 0 0) 0x7f3f180a4b70 con 0x55e7e5700b40 2024-09-19T08:56:06.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.223+0000 7f3f0bfff640 1 -- 172.21.15.164:0/3888577057 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3f180caee0 con 0x55e7e5700b40 2024-09-19T08:56:06.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.223+0000 7f3f3526d740 10 client.17637 fetch_fsmap learned FSMap version 32 2024-09-19T08:56:06.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.223+0000 7f3f3526d740 1 -- 172.21.15.164:0/3888577057 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e7e5788420 con 0x55e7e5700b40 2024-09-19T08:56:06.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.223+0000 7f3f0bfff640 1 -- 172.21.15.164:0/3888577057 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f1809c440 con 0x55e7e5700b40 2024-09-19T08:56:06.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.224+0000 7f3f0bfff640 1 -- 172.21.15.164:0/3888577057 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f3f1801d940 con 0x55e7e5700b40 2024-09-19T08:56:06.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.224+0000 7f3f3526d740 10 client.17637 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:56:06.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.224+0000 7f3f3526d740 10 client.17637 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-19T08:56:06.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.224+0000 7f3f3526d740 20 client.17637 populate_metadata read hostname 'smithi164' 2024-09-19T08:56:06.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.224+0000 7f3f3526d740 1 --2- 172.21.15.164:0/3888577057 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55e7e571e2c0 0x55e7e578a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:06.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.224+0000 7f3f3526d740 4 client.17637 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:56:06.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.224+0000 7f3f3526d740 1 -- 172.21.15.164:0/3888577057 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e7e566cd70 con 0x55e7e571e2c0 2024-09-19T08:56:06.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.224+0000 7f3f30c30640 1 --2- 172.21.15.164:0/3888577057 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55e7e571e2c0 0x55e7e578a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:06.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.224+0000 7f3f0bfff640 10 client.17637 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:56:06.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.224+0000 7f3f30c30640 1 --2- 172.21.15.164:0/3888577057 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55e7e571e2c0 0x55e7e578a760 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b73aff62a7c044c2 server_cookie=a87729bbaccfbcb1 in_seq=0 out_seq=0 2024-09-19T08:56:06.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.225+0000 7f3f0bfff640 1 -- 172.21.15.164:0/3888577057 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e7e566cd70 con 0x55e7e571e2c0 2024-09-19T08:56:06.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.225+0000 7f3f0bfff640 10 client.17637 handle_command_reply: tid=0 2024-09-19T08:56:06.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.225+0000 7f3f3526d740 1 client.17637 shutdown 2024-09-19T08:56:06.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.225+0000 7f3f3526d740 1 -- 172.21.15.164:0/3888577057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e5700b40 msgr2=0x55e7e5711d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:06.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.225+0000 7f3f3526d740 1 --2- 172.21.15.164:0/3888577057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e5700b40 0x55e7e5711d90 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f3f18010f50 tx=0x7f3f1800cea0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.225+0000 7f3f3526d740 1 -- 172.21.15.164:0/3888577057 shutdown_connections 2024-09-19T08:56:06.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.225+0000 7f3f3526d740 1 --2- 172.21.15.164:0/3888577057 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55e7e571e2c0 0x55e7e578a760 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.225+0000 7f3f0bfff640 0 client.17637 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:56:06.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.225+0000 7f3f3526d740 1 --2- 172.21.15.164:0/3888577057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7e570e780 0x55e7e57122d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.225+0000 7f3f3526d740 1 --2- 172.21.15.164:0/3888577057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7e5700b40 0x55e7e5711d90 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.225+0000 7f3f3526d740 1 --2- 172.21.15.164:0/3888577057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e7e56ff5d0 0x55e7e5711850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.225+0000 7f3f3526d740 1 -- 172.21.15.164:0/3888577057 >> 172.21.15.164:0/3888577057 conn(0x55e7e56c8310 msgr2=0x55e7e56fa4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:06.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.225+0000 7f3f3526d740 1 -- 172.21.15.164:0/3888577057 shutdown_connections 2024-09-19T08:56:06.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.225+0000 7f3f3526d740 1 -- 172.21.15.164:0/3888577057 wait complete. 2024-09-19T08:56:06.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.225+0000 7f3f3526d740 20 client.17637 trim_cache size 0 max 16384 2024-09-19T08:56:06.232 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:56:06.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.226+0000 7f3f32e94640 1 -- 172.21.15.164:0/936847187 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3f1c0a33b0 msgr2=0x7f3f1c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:06.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.226+0000 7f3f32e94640 1 --2- 172.21.15.164:0/936847187 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3f1c0a33b0 0x7f3f1c0a5870 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3f240049d0 tx=0x7f3f240116d0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.226+0000 7f3f32e94640 1 -- 172.21.15.164:0/936847187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f2c1027a0 msgr2=0x7f3f2c1989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:06.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.226+0000 7f3f32e94640 1 --2- 172.21.15.164:0/936847187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f2c1027a0 0x7f3f2c1989a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3f1800e9f0 tx=0x7f3f1800eeb0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.226+0000 7f3f32e94640 1 -- 172.21.15.164:0/936847187 shutdown_connections 2024-09-19T08:56:06.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.226+0000 7f3f32e94640 1 --2- 172.21.15.164:0/936847187 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f3f1c0a33b0 0x7f3f1c0a5870 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.226+0000 7f3f32e94640 1 --2- 172.21.15.164:0/936847187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f2c106f20 0x7f3f2c19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.226+0000 7f3f32e94640 1 --2- 172.21.15.164:0/936847187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f2c1027a0 0x7f3f2c1989a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.226+0000 7f3f32e94640 1 --2- 172.21.15.164:0/936847187 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f2c101df0 0x7f3f2c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:06.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.226+0000 7f3f32e94640 1 -- 172.21.15.164:0/936847187 >> 172.21.15.164:0/936847187 conn(0x7f3f2c0fd8f0 msgr2=0x7f3f2c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:06.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.226+0000 7f3f32e94640 1 -- 172.21.15.164:0/936847187 shutdown_connections 2024-09-19T08:56:06.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:06.226+0000 7f3f32e94640 1 -- 172.21.15.164:0/936847187 wait complete. 2024-09-19T08:56:06.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:06 smithi164 ceph-mon[30928]: pgmap v319: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 28 op/s 2024-09-19T08:56:06.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:06 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:56:06.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:06 smithi170 ceph-mon[35072]: pgmap v319: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 28 op/s 2024-09-19T08:56:06.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:06 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:56:06.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:06 smithi173 ceph-mon[35306]: pgmap v319: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 28 op/s 2024-09-19T08:56:06.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:06 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:56:06.998 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T08:56:07.971 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:56:08.409 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:08 smithi164 ceph-mon[30928]: pgmap v320: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-09-19T08:56:08.409 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:08 smithi164 ceph-mon[30928]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T08:56:08.410 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:08 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:56:08.410 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:08 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:56:08.410 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:08 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:56:08.410 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:08 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:56:08.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.547+0000 7f4c8759e640 1 -- 172.21.15.164:0/3598513299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c881012a0 msgr2=0x7f4c88105770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:08.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.547+0000 7f4c8759e640 1 --2- 172.21.15.164:0/3598513299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c881012a0 0x7f4c88105770 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4c8001acb0 tx=0x7f4c8003b9b0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.547+0000 7f4c8559a640 1 -- 172.21.15.164:0/3598513299 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c80047a90 con 0x7f4c881012a0 2024-09-19T08:56:08.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.548+0000 7f4c8759e640 1 -- 172.21.15.164:0/3598513299 shutdown_connections 2024-09-19T08:56:08.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.548+0000 7f4c8759e640 1 --2- 172.21.15.164:0/3598513299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c88105d90 0x7f4c88106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.548+0000 7f4c8759e640 1 --2- 172.21.15.164:0/3598513299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c881012a0 0x7f4c88105770 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.548+0000 7f4c8759e640 1 --2- 172.21.15.164:0/3598513299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c88100980 0x7f4c88100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.548+0000 7f4c8759e640 1 -- 172.21.15.164:0/3598513299 >> 172.21.15.164:0/3598513299 conn(0x7f4c880fc610 msgr2=0x7f4c880fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:08.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.548+0000 7f4c8759e640 1 -- 172.21.15.164:0/3598513299 shutdown_connections 2024-09-19T08:56:08.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.548+0000 7f4c8759e640 1 -- 172.21.15.164:0/3598513299 wait complete. 2024-09-19T08:56:08.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.549+0000 7f4c8759e640 1 Processor -- start 2024-09-19T08:56:08.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.549+0000 7f4c8759e640 1 -- start start 2024-09-19T08:56:08.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.549+0000 7f4c8759e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c88100980 0x7f4c88072dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:08.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.549+0000 7f4c8759e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c881012a0 0x7f4c88073300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:08.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.550+0000 7f4c8759e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c88105d90 0x7f4c8810cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:08.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.550+0000 7f4c8759e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4c88109720 con 0x7f4c881012a0 2024-09-19T08:56:08.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.550+0000 7f4c8759e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4c881095a0 con 0x7f4c88105d90 2024-09-19T08:56:08.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.550+0000 7f4c8759e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4c881098a0 con 0x7f4c88100980 2024-09-19T08:56:08.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.550+0000 7f4c85d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c881012a0 0x7f4c88073300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:08.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.550+0000 7f4c85d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c881012a0 0x7f4c88073300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50086/0 (socket says 172.21.15.164:50086) 2024-09-19T08:56:08.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.550+0000 7f4c85d9b640 1 -- 172.21.15.164:0/291053676 learned_addr learned my addr 172.21.15.164:0/291053676 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:08.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.551+0000 7f4c86d9d640 1 --2- 172.21.15.164:0/291053676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c88105d90 0x7f4c8810cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:08.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.551+0000 7f4c85d9b640 1 -- 172.21.15.164:0/291053676 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c88100980 msgr2=0x7f4c88072dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:08.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.551+0000 7f4c85d9b640 1 --2- 172.21.15.164:0/291053676 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c88100980 0x7f4c88072dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.551+0000 7f4c85d9b640 1 -- 172.21.15.164:0/291053676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c88105d90 msgr2=0x7f4c8810cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:08.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.551+0000 7f4c85d9b640 1 --2- 172.21.15.164:0/291053676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c88105d90 0x7f4c8810cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.551+0000 7f4c85d9b640 1 -- 172.21.15.164:0/291053676 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c88115380 con 0x7f4c881012a0 2024-09-19T08:56:08.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.552+0000 7f4c85d9b640 1 --2- 172.21.15.164:0/291053676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c881012a0 0x7f4c88073300 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f4c8001a630 tx=0x7f4c80047b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:08.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.552+0000 7f4c777fe640 1 -- 172.21.15.164:0/291053676 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c80093070 con 0x7f4c881012a0 2024-09-19T08:56:08.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.552+0000 7f4c777fe640 1 -- 172.21.15.164:0/291053676 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c80040820 con 0x7f4c881012a0 2024-09-19T08:56:08.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.552+0000 7f4c777fe640 1 -- 172.21.15.164:0/291053676 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c80044040 con 0x7f4c881012a0 2024-09-19T08:56:08.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.554+0000 7f4c8759e640 1 -- 172.21.15.164:0/291053676 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c8810d450 con 0x7f4c881012a0 2024-09-19T08:56:08.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.555+0000 7f4c8759e640 1 -- 172.21.15.164:0/291053676 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c8810d8e0 con 0x7f4c881012a0 2024-09-19T08:56:08.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.558+0000 7f4c8f1b8740 1 Processor -- start 2024-09-19T08:56:08.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.559+0000 7f4c8f1b8740 1 -- start start 2024-09-19T08:56:08.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.560+0000 7f4c8f1b8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee795c6d20 0x55ee795c5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:08.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.560+0000 7f4c8f1b8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee795c57b0 0x55ee795c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:08.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.560+0000 7f4c8f1b8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee795d0550 0x55ee795d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:08.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.560+0000 7f4c8f1b8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ee7952b2f0 con 0x55ee795c57b0 2024-09-19T08:56:08.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.560+0000 7f4c8f1b8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ee794d2cf0 con 0x55ee795c6d20 2024-09-19T08:56:08.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.560+0000 7f4c8f1b8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ee795272b0 con 0x55ee795d0550 2024-09-19T08:56:08.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.561+0000 7f4c86d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee795c6d20 0x55ee795c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:08.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.561+0000 7f4c85d9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee795d0550 0x55ee795d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:08.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.561+0000 7f4c85d9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee795d0550 0x55ee795d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:42110/0 (socket says 172.21.15.164:42110) 2024-09-19T08:56:08.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.561+0000 7f4c86d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee795c6d20 0x55ee795c5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:50148/0 (socket says 172.21.15.164:50148) 2024-09-19T08:56:08.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.561+0000 7f4c85d9b640 1 -- 172.21.15.164:0/3966181153 learned_addr learned my addr 172.21.15.164:0/3966181153 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:08.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.562+0000 7f4c85d9b640 1 -- 172.21.15.164:0/3966181153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee795c6d20 msgr2=0x55ee795c5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:08.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.562+0000 7f4c85d9b640 1 --2- 172.21.15.164:0/3966181153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee795c6d20 0x55ee795c5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.562+0000 7f4c85d9b640 1 -- 172.21.15.164:0/3966181153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee795c57b0 msgr2=0x55ee795c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:08.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.562+0000 7f4c85d9b640 1 --2- 172.21.15.164:0/3966181153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee795c57b0 0x55ee795c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.562+0000 7f4c85d9b640 1 -- 172.21.15.164:0/3966181153 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee794b73a0 con 0x55ee795d0550 2024-09-19T08:56:08.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.562+0000 7f4c86d9d640 1 --2- 172.21.15.164:0/3966181153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee795c6d20 0x55ee795c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-19T08:56:08.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.562+0000 7f4c85d9b640 1 --2- 172.21.15.164:0/3966181153 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee795d0550 0x55ee795d2940 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4c8003b9f0 tx=0x7f4c80004f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67235a73e7833392 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:08.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.563+0000 7f4c74ff9640 1 -- 172.21.15.164:0/3966181153 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c800af020 con 0x55ee795d0550 2024-09-19T08:56:08.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.563+0000 7f4c74ff9640 1 -- 172.21.15.164:0/3966181153 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c80097c60 con 0x55ee795d0550 2024-09-19T08:56:08.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.563+0000 7f4c74ff9640 1 -- 172.21.15.164:0/3966181153 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c800056c0 con 0x55ee795d0550 2024-09-19T08:56:08.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.565+0000 7f4c8f1b8740 1 -- 172.21.15.164:0/3966181153 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee795d0550 msgr2=0x55ee795d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:08.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.565+0000 7f4c8f1b8740 1 --2- 172.21.15.164:0/3966181153 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee795d0550 0x55ee795d2940 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4c8003b9f0 tx=0x7f4c80004f10 comp rx=0 tx=0).stop 2024-09-19T08:56:08.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.565+0000 7f4c777fe640 1 -- 172.21.15.164:0/291053676 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f4c800b2020 con 0x7f4c881012a0 2024-09-19T08:56:08.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.566+0000 7f4c777fe640 1 --2- 172.21.15.164:0/291053676 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f4c700a35c0 0x7f4c700a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:08.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.567+0000 7f4c777fe640 1 -- 172.21.15.164:0/291053676 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7f4c8014a2a0 con 0x7f4c881012a0 2024-09-19T08:56:08.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.572+0000 7f4c86d9d640 1 --2- 172.21.15.164:0/291053676 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f4c700a35c0 0x7f4c700a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:08.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.572+0000 7f4c8f1b8740 1 -- 172.21.15.164:0/3966181153 shutdown_connections 2024-09-19T08:56:08.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.572+0000 7f4c8f1b8740 1 --2- 172.21.15.164:0/3966181153 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee795d0550 0x55ee795d2940 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.572+0000 7f4c8f1b8740 1 --2- 172.21.15.164:0/3966181153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee795c57b0 0x55ee795c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.572+0000 7f4c8f1b8740 1 --2- 172.21.15.164:0/3966181153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee795c6d20 0x55ee795c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.572+0000 7f4c8f1b8740 1 -- 172.21.15.164:0/3966181153 >> 172.21.15.164:0/3966181153 conn(0x55ee7958e310 msgr2=0x55ee795c1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:08.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.573+0000 7f4c8f1b8740 1 -- 172.21.15.164:0/3966181153 shutdown_connections 2024-09-19T08:56:08.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.573+0000 7f4c8f1b8740 1 -- 172.21.15.164:0/3966181153 wait complete. 2024-09-19T08:56:08.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.573+0000 7f4c8f1b8740 1 -- start start 2024-09-19T08:56:08.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.573+0000 7f4c8f1b8740 1 Processor -- start 2024-09-19T08:56:08.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.574+0000 7f4c8f1b8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee795c57b0 0x55ee795cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:08.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.574+0000 7f4c8f1b8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee795c6d20 0x55ee795cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:08.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.574+0000 7f4c8f1b8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee795d0550 0x55ee795cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:08.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.574+0000 7f4c8f1b8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ee795272b0 con 0x55ee795c57b0 2024-09-19T08:56:08.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.574+0000 7f4c8f1b8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ee794e6060 con 0x55ee795c6d20 2024-09-19T08:56:08.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.574+0000 7f4c8f1b8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ee795d3010 con 0x55ee795d0550 2024-09-19T08:56:08.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.574+0000 7f4c8f1b8740 10 client.0 _ll_register_callbacks cb 0x55ee792398c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:56:08.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.574+0000 7f4c8659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee795c57b0 0x55ee795cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:08.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.575+0000 7f4c8659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee795c57b0 0x55ee795cfe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50104/0 (socket says 172.21.15.164:50104) 2024-09-19T08:56:08.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.575+0000 7f4c8659c640 1 -- 172.21.15.164:0/1422393404 learned_addr learned my addr 172.21.15.164:0/1422393404 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:08.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.575+0000 7f4c85d9b640 1 --2- 172.21.15.164:0/1422393404 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee795c6d20 0x55ee795cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:08.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.575+0000 7f4c86d9d640 1 --2- 172.21.15.164:0/291053676 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f4c700a35c0 0x7f4c700a5a80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4c78009c40 tx=0x7f4c78005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:08.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.575+0000 7f4c8659c640 1 -- 172.21.15.164:0/1422393404 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee795d0550 msgr2=0x55ee795cd620 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:56:08.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.575+0000 7f4c8659c640 1 --2- 172.21.15.164:0/1422393404 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee795d0550 0x55ee795cd620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.575+0000 7f4c8659c640 1 -- 172.21.15.164:0/1422393404 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee795c6d20 msgr2=0x55ee795cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:08.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.575+0000 7f4c8659c640 1 --2- 172.21.15.164:0/1422393404 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee795c6d20 0x55ee795cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.575+0000 7f4c8659c640 1 -- 172.21.15.164:0/1422393404 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee7929c3c0 con 0x55ee795c57b0 2024-09-19T08:56:08.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.576+0000 7f4c8659c640 1 --2- 172.21.15.164:0/1422393404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee795c57b0 0x55ee795cfe30 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c009e40 tx=0x7f4c7c00b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7688d89c4a687492 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:08.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.576+0000 7f4c6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:56:08.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.576+0000 7f4c6dffb640 1 -- 172.21.15.164:0/1422393404 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c015020 con 0x55ee795c57b0 2024-09-19T08:56:08.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.576+0000 7f4c8f1b8740 1 -- 172.21.15.164:0/1422393404 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c80097c60 con 0x55ee795c57b0 2024-09-19T08:56:08.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.576+0000 7f4c6dffb640 1 -- 172.21.15.164:0/1422393404 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7f4c7c011420 con 0x55ee795c57b0 2024-09-19T08:56:08.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.576+0000 7f4c6dffb640 1 -- 172.21.15.164:0/1422393404 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4c7c020050 con 0x55ee795c57b0 2024-09-19T08:56:08.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.576+0000 7f4c8f1b8740 10 client.17646 fetch_fsmap learned FSMap version 32 2024-09-19T08:56:08.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.577+0000 7f4c8f1b8740 1 -- 172.21.15.164:0/1422393404 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ee795de740 con 0x55ee795c57b0 2024-09-19T08:56:08.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.577+0000 7f4c6dffb640 1 -- 172.21.15.164:0/1422393404 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c80097c60 con 0x55ee795c57b0 2024-09-19T08:56:08.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.577+0000 7f4c6dffb640 1 -- 172.21.15.164:0/1422393404 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f4c7c019890 con 0x55ee795c57b0 2024-09-19T08:56:08.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.577+0000 7f4c8f1b8740 10 client.17646 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:56:08.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.577+0000 7f4c8f1b8740 10 client.17646 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T08:56:08.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.577+0000 7f4c8f1b8740 20 client.17646 populate_metadata read hostname 'smithi164' 2024-09-19T08:56:08.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.577+0000 7f4c8f1b8740 1 --2- 172.21.15.164:0/1422393404 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55ee795e0020 0x55ee795e24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:08.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.577+0000 7f4c8f1b8740 4 client.17646 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:56:08.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.577+0000 7f4c8f1b8740 1 -- 172.21.15.164:0/1422393404 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee79538860 con 0x55ee795e0020 2024-09-19T08:56:08.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.578+0000 7f4c85d9b640 1 --2- 172.21.15.164:0/1422393404 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55ee795e0020 0x55ee795e24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:08.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.579+0000 7f4c85d9b640 1 --2- 172.21.15.164:0/1422393404 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55ee795e0020 0x55ee795e24e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=da03d748fc4a0638 server_cookie=90e6c665fa3deba9 in_seq=0 out_seq=0 2024-09-19T08:56:08.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.579+0000 7f4c6dffb640 10 client.17646 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:56:08.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.583+0000 7f4c6dffb640 1 -- 172.21.15.164:0/1422393404 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ee79538860 con 0x55ee795e0020 2024-09-19T08:56:08.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.583+0000 7f4c6dffb640 10 client.17646 handle_command_reply: tid=0 2024-09-19T08:56:08.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.584+0000 7f4c8f1b8740 1 client.17646 shutdown 2024-09-19T08:56:08.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.584+0000 7f4c8f1b8740 1 -- 172.21.15.164:0/1422393404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee795c57b0 msgr2=0x55ee795cfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:08.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.584+0000 7f4c8f1b8740 1 --2- 172.21.15.164:0/1422393404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee795c57b0 0x55ee795cfe30 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c009e40 tx=0x7f4c7c00b4b0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.584+0000 7f4c8f1b8740 1 -- 172.21.15.164:0/1422393404 shutdown_connections 2024-09-19T08:56:08.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.584+0000 7f4c8f1b8740 1 --2- 172.21.15.164:0/1422393404 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55ee795e0020 0x55ee795e24e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.584+0000 7f4c8f1b8740 1 --2- 172.21.15.164:0/1422393404 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee795d0550 0x55ee795cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.584+0000 7f4c8f1b8740 1 --2- 172.21.15.164:0/1422393404 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee795c6d20 0x55ee795cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.584+0000 7f4c8f1b8740 1 --2- 172.21.15.164:0/1422393404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee795c57b0 0x55ee795cfe30 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.584+0000 7f4c8f1b8740 1 -- 172.21.15.164:0/1422393404 >> 172.21.15.164:0/1422393404 conn(0x55ee7958e310 msgr2=0x55ee795c7b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:08.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.584+0000 7f4c6dffb640 0 client.17646 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:56:08.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.585+0000 7f4c8f1b8740 1 -- 172.21.15.164:0/1422393404 shutdown_connections 2024-09-19T08:56:08.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.585+0000 7f4c8f1b8740 1 -- 172.21.15.164:0/1422393404 wait complete. 2024-09-19T08:56:08.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.585+0000 7f4c8f1b8740 20 client.17646 trim_cache size 0 max 16384 2024-09-19T08:56:08.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.612+0000 7f4c8f1b8740 1 Processor -- start 2024-09-19T08:56:08.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.614+0000 7f4c8f1b8740 1 -- start start 2024-09-19T08:56:08.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.614+0000 7f4c8f1b8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee795c57b0 0x55ee795c1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:08.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.614+0000 7f4c8f1b8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee795c6d20 0x55ee795db810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:08.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.615+0000 7f4c8f1b8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee795d0550 0x55ee795dbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:08.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.615+0000 7f4c8f1b8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8811de60 con 0x55ee795c6d20 2024-09-19T08:56:08.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.615+0000 7f4c8f1b8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8811f9a0 con 0x55ee795c57b0 2024-09-19T08:56:08.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.615+0000 7f4c8f1b8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8811e530 con 0x55ee795d0550 2024-09-19T08:56:08.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.615+0000 7f4c85d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee795c6d20 0x55ee795db810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:08.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.615+0000 7f4c85d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee795c6d20 0x55ee795db810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50116/0 (socket says 172.21.15.164:50116) 2024-09-19T08:56:08.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.615+0000 7f4c85d9b640 1 -- 172.21.15.164:0/1731514820 learned_addr learned my addr 172.21.15.164:0/1731514820 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:08.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.615+0000 7f4c85d9b640 1 -- 172.21.15.164:0/1731514820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee795d0550 msgr2=0x55ee795dbd50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:56:08.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.615+0000 7f4c85d9b640 1 --2- 172.21.15.164:0/1731514820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee795d0550 0x55ee795dbd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.615+0000 7f4c85d9b640 1 -- 172.21.15.164:0/1731514820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee795c57b0 msgr2=0x55ee795c1ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:56:08.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.615+0000 7f4c85d9b640 1 --2- 172.21.15.164:0/1731514820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee795c57b0 0x55ee795c1ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.615+0000 7f4c85d9b640 1 -- 172.21.15.164:0/1731514820 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee795c8300 con 0x55ee795c6d20 2024-09-19T08:56:08.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.616+0000 7f4c85d9b640 1 --2- 172.21.15.164:0/1731514820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee795c6d20 0x55ee795db810 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f4c8001a3b0 tx=0x7f4c8011d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be924d7df409350 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:08.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.617+0000 7f4c6ffff640 1 -- 172.21.15.164:0/1731514820 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c800af040 con 0x55ee795c6d20 2024-09-19T08:56:08.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.617+0000 7f4c6ffff640 1 -- 172.21.15.164:0/1731514820 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c80040af0 con 0x55ee795c6d20 2024-09-19T08:56:08.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.617+0000 7f4c6ffff640 1 -- 172.21.15.164:0/1731514820 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8011f440 con 0x55ee795c6d20 2024-09-19T08:56:08.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.618+0000 7f4c8f1b8740 1 -- 172.21.15.164:0/1731514820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee795c6d20 msgr2=0x55ee795db810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:08.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.618+0000 7f4c8f1b8740 1 --2- 172.21.15.164:0/1731514820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee795c6d20 0x55ee795db810 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f4c8001a3b0 tx=0x7f4c8011d040 comp rx=0 tx=0).stop 2024-09-19T08:56:08.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.618+0000 7f4c8f1b8740 1 -- 172.21.15.164:0/1731514820 shutdown_connections 2024-09-19T08:56:08.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.618+0000 7f4c8f1b8740 1 --2- 172.21.15.164:0/1731514820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee795d0550 0x55ee795dbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.618+0000 7f4c8f1b8740 1 --2- 172.21.15.164:0/1731514820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee795c6d20 0x55ee795db810 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.618+0000 7f4c8f1b8740 1 --2- 172.21.15.164:0/1731514820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee795c57b0 0x55ee795c1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.618+0000 7f4c8f1b8740 1 -- 172.21.15.164:0/1731514820 >> 172.21.15.164:0/1731514820 conn(0x55ee7958e310 msgr2=0x55ee795c18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:08.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.622+0000 7f4c8f1b8740 1 -- 172.21.15.164:0/1731514820 shutdown_connections 2024-09-19T08:56:08.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.623+0000 7f4c8f1b8740 1 -- 172.21.15.164:0/1731514820 wait complete. 2024-09-19T08:56:08.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.623+0000 7f4c8f1b8740 1 -- start start 2024-09-19T08:56:08.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.623+0000 7f4c8f1b8740 1 Processor -- start 2024-09-19T08:56:08.624 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:08 smithi170 ceph-mon[35072]: pgmap v320: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-09-19T08:56:08.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:08 smithi170 ceph-mon[35072]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T08:56:08.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:08 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:56:08.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:08 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:56:08.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:08 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:56:08.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:08 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:56:08.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.624+0000 7f4c8f1b8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee795c57b0 0x55ee795d5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:08.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.624+0000 7f4c8f1b8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee795c6d20 0x55ee795d5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:08.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.624+0000 7f4c8f1b8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee795d0550 0x55ee795c9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:08.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.625+0000 7f4c8f1b8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8811f2d0 con 0x55ee795d0550 2024-09-19T08:56:08.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.625+0000 7f4c8f1b8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8811ec00 con 0x55ee795c57b0 2024-09-19T08:56:08.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.625+0000 7f4c8f1b8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8811be60 con 0x55ee795c6d20 2024-09-19T08:56:08.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.625+0000 7f4c8f1b8740 10 client.0 _ll_register_callbacks cb 0x7f4c7c006ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:56:08.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.626+0000 7f4c86d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee795d0550 0x55ee795c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:08.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.626+0000 7f4c86d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee795d0550 0x55ee795c9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50130/0 (socket says 172.21.15.164:50130) 2024-09-19T08:56:08.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.626+0000 7f4c86d9d640 1 -- 172.21.15.164:0/2019417004 learned_addr learned my addr 172.21.15.164:0/2019417004 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:08.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.627+0000 7f4c86d9d640 1 -- 172.21.15.164:0/2019417004 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee795c6d20 msgr2=0x55ee795d5eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:56:08.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.627+0000 7f4c86d9d640 1 --2- 172.21.15.164:0/2019417004 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee795c6d20 0x55ee795d5eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.627+0000 7f4c86d9d640 1 -- 172.21.15.164:0/2019417004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee795c57b0 msgr2=0x55ee795d5970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:56:08.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.627+0000 7f4c86d9d640 1 --2- 172.21.15.164:0/2019417004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee795c57b0 0x55ee795d5970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.627+0000 7f4c86d9d640 1 -- 172.21.15.164:0/2019417004 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee795c9990 con 0x55ee795d0550 2024-09-19T08:56:08.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.627+0000 7f4c86d9d640 1 --2- 172.21.15.164:0/2019417004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee795d0550 0x55ee795c9450 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f4c78011c60 tx=0x7f4c780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766f7d9d4983a9d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:08.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.629+0000 7f4c6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:56:08.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.629+0000 7f4c6dffb640 1 -- 172.21.15.164:0/2019417004 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7802b060 con 0x55ee795d0550 2024-09-19T08:56:08.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.629+0000 7f4c6dffb640 1 -- 172.21.15.164:0/2019417004 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7f4c78011c90 con 0x55ee795d0550 2024-09-19T08:56:08.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.630+0000 7f4c8f1b8740 1 -- 172.21.15.164:0/2019417004 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c80040af0 con 0x55ee795d0550 2024-09-19T08:56:08.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.630+0000 7f4c6dffb640 1 -- 172.21.15.164:0/2019417004 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4c7801ee50 con 0x55ee795d0550 2024-09-19T08:56:08.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.632+0000 7f4c8f1b8740 10 client.17652 fetch_fsmap learned FSMap version 32 2024-09-19T08:56:08.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.632+0000 7f4c8f1b8740 1 -- 172.21.15.164:0/2019417004 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ee7964a4a0 con 0x55ee795d0550 2024-09-19T08:56:08.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.634+0000 7f4c6dffb640 1 -- 172.21.15.164:0/2019417004 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c78011e90 con 0x55ee795d0550 2024-09-19T08:56:08.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.634+0000 7f4c6dffb640 1 -- 172.21.15.164:0/2019417004 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f4c7801a040 con 0x55ee795d0550 2024-09-19T08:56:08.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.637+0000 7f4c8f1b8740 10 client.17652 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:56:08.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.637+0000 7f4c8f1b8740 10 client.17652 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T08:56:08.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.637+0000 7f4c8f1b8740 20 client.17652 populate_metadata read hostname 'smithi164' 2024-09-19T08:56:08.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.637+0000 7f4c8f1b8740 1 --2- 172.21.15.164:0/2019417004 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55ee7964ab80 0x55ee7964cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:08.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.637+0000 7f4c8f1b8740 4 client.17652 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:56:08.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.637+0000 7f4c8f1b8740 1 -- 172.21.15.164:0/2019417004 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f4c80040af0 con 0x55ee7964ab80 2024-09-19T08:56:08.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.638+0000 7f4c8659c640 1 --2- 172.21.15.164:0/2019417004 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55ee7964ab80 0x55ee7964cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:08.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.641+0000 7f4c8659c640 1 --2- 172.21.15.164:0/2019417004 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55ee7964ab80 0x55ee7964cfa0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fec42ced87b69e43 server_cookie=1a6a8d8fa93df860 in_seq=0 out_seq=0 2024-09-19T08:56:08.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.641+0000 7f4c6dffb640 10 client.17652 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:56:08.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.641+0000 7f4c6dffb640 1 -- 172.21.15.164:0/2019417004 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f4c80040af0 con 0x55ee7964ab80 2024-09-19T08:56:08.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.641+0000 7f4c6dffb640 10 client.17652 handle_command_reply: tid=0 2024-09-19T08:56:08.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.642+0000 7f4c8f1b8740 1 client.17652 shutdown 2024-09-19T08:56:08.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.645+0000 7f4c8f1b8740 1 -- 172.21.15.164:0/2019417004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee795d0550 msgr2=0x55ee795c9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:08.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.645+0000 7f4c8f1b8740 1 --2- 172.21.15.164:0/2019417004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee795d0550 0x55ee795c9450 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f4c78011c60 tx=0x7f4c780046c0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.646+0000 7f4c8f1b8740 1 -- 172.21.15.164:0/2019417004 shutdown_connections 2024-09-19T08:56:08.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.646+0000 7f4c8f1b8740 1 --2- 172.21.15.164:0/2019417004 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55ee7964ab80 0x55ee7964cfa0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.646+0000 7f4c8f1b8740 1 --2- 172.21.15.164:0/2019417004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee795d0550 0x55ee795c9450 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.646+0000 7f4c8f1b8740 1 --2- 172.21.15.164:0/2019417004 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee795c6d20 0x55ee795d5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.646+0000 7f4c6dffb640 0 client.17652 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:56:08.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.646+0000 7f4c8f1b8740 1 --2- 172.21.15.164:0/2019417004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee795c57b0 0x55ee795d5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.646+0000 7f4c8f1b8740 1 -- 172.21.15.164:0/2019417004 >> 172.21.15.164:0/2019417004 conn(0x55ee7958e310 msgr2=0x55ee795ce940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:08.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.647+0000 7f4c8f1b8740 1 -- 172.21.15.164:0/2019417004 shutdown_connections 2024-09-19T08:56:08.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.648+0000 7f4c8f1b8740 1 -- 172.21.15.164:0/2019417004 wait complete. 2024-09-19T08:56:08.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.648+0000 7f4c8f1b8740 20 client.17652 trim_cache size 0 max 16384 2024-09-19T08:56:08.649 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:56:08.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.651+0000 7f4c8759e640 1 -- 172.21.15.164:0/291053676 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f4c700a35c0 msgr2=0x7f4c700a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:08.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.651+0000 7f4c8759e640 1 --2- 172.21.15.164:0/291053676 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f4c700a35c0 0x7f4c700a5a80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4c78009c40 tx=0x7f4c78005f90 comp rx=0 tx=0).stop 2024-09-19T08:56:08.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.651+0000 7f4c8759e640 1 -- 172.21.15.164:0/291053676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c881012a0 msgr2=0x7f4c88073300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:08.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.651+0000 7f4c8759e640 1 --2- 172.21.15.164:0/291053676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c881012a0 0x7f4c88073300 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f4c8001a630 tx=0x7f4c80047b10 comp rx=0 tx=0).stop 2024-09-19T08:56:08.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.652+0000 7f4c8759e640 1 -- 172.21.15.164:0/291053676 shutdown_connections 2024-09-19T08:56:08.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.652+0000 7f4c8759e640 1 --2- 172.21.15.164:0/291053676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c88105d90 0x7f4c8810cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.652+0000 7f4c8759e640 1 --2- 172.21.15.164:0/291053676 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f4c700a35c0 0x7f4c700a5a80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.652+0000 7f4c8759e640 1 --2- 172.21.15.164:0/291053676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c881012a0 0x7f4c88073300 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.652+0000 7f4c8759e640 1 --2- 172.21.15.164:0/291053676 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c88100980 0x7f4c88072dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:08.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.652+0000 7f4c8759e640 1 -- 172.21.15.164:0/291053676 >> 172.21.15.164:0/291053676 conn(0x7f4c880fc610 msgr2=0x7f4c8806bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:08.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.656+0000 7f4c8759e640 1 -- 172.21.15.164:0/291053676 shutdown_connections 2024-09-19T08:56:08.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:08.656+0000 7f4c8759e640 1 -- 172.21.15.164:0/291053676 wait complete. 2024-09-19T08:56:08.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:08 smithi173 ceph-mon[35306]: pgmap v320: 97 pgs: 97 active+clean; 31 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-09-19T08:56:08.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:08 smithi173 ceph-mon[35306]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T08:56:08.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:08 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:56:08.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:08 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:56:08.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:08 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:56:08.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:08 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:56:09.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:09 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:09.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:09 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:09.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:09 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:09.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:09 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:09.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:09 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:09.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:09 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:09.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:09 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:09.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:09 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:09.400 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T08:56:09.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:09 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:09.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:09 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:09.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:09 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:09.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:09 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:09.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:09 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:09.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:09 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:09.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:09 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:09.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:09 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:09.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:09 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:09.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:09 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:09.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:09 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:09.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:09 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:09.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:09 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:09.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:09 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:09.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:09 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:09.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:09 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:09.695 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:56:10.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.193+0000 7f716b610640 1 -- 172.21.15.164:0/2110119042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7164100e20 msgr2=0x7f7164101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:10.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.193+0000 7f716b610640 1 --2- 172.21.15.164:0/2110119042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7164100e20 0x7f7164101200 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f7160009ca0 tx=0x7f716002a950 comp rx=0 tx=0).stop 2024-09-19T08:56:10.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.193+0000 7f716b610640 1 -- 172.21.15.164:0/2110119042 shutdown_connections 2024-09-19T08:56:10.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.193+0000 7f716b610640 1 --2- 172.21.15.164:0/2110119042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7164102ef0 0x7f716410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:10.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.193+0000 7f716b610640 1 --2- 172.21.15.164:0/2110119042 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7164102570 0x7f71641029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:10.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.193+0000 7f716b610640 1 --2- 172.21.15.164:0/2110119042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7164100e20 0x7f7164101200 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:10.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.193+0000 7f716b610640 1 -- 172.21.15.164:0/2110119042 >> 172.21.15.164:0/2110119042 conn(0x7f71640fc7d0 msgr2=0x7f71640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:10.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.193+0000 7f716b610640 1 -- 172.21.15.164:0/2110119042 shutdown_connections 2024-09-19T08:56:10.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.193+0000 7f716b610640 1 -- 172.21.15.164:0/2110119042 wait complete. 2024-09-19T08:56:10.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.193+0000 7f716b610640 1 Processor -- start 2024-09-19T08:56:10.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.194+0000 7f716b610640 1 -- start start 2024-09-19T08:56:10.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.194+0000 7f716b610640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7164100e20 0x7f71641993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:10.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.194+0000 7f716b610640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7164102570 0x7f7164199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:10.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.194+0000 7f716b610640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7164102ef0 0x7f71641a4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:10.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.194+0000 7f716b610640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7164110f80 con 0x7f7164102570 2024-09-19T08:56:10.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.194+0000 7f716b610640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7164110e00 con 0x7f7164102ef0 2024-09-19T08:56:10.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.194+0000 7f716b610640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7164111100 con 0x7f7164100e20 2024-09-19T08:56:10.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.194+0000 7f7168bab640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7164102570 0x7f7164199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:10.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.194+0000 7f7168bab640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7164102570 0x7f7164199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50148/0 (socket says 172.21.15.164:50148) 2024-09-19T08:56:10.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.194+0000 7f7168bab640 1 -- 172.21.15.164:0/2290619701 learned_addr learned my addr 172.21.15.164:0/2290619701 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:10.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.194+0000 7f7169bad640 1 --2- 172.21.15.164:0/2290619701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7164102ef0 0x7f71641a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:10.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.194+0000 7f71693ac640 1 --2- 172.21.15.164:0/2290619701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7164100e20 0x7f71641993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:10.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.194+0000 7f7168bab640 1 -- 172.21.15.164:0/2290619701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7164100e20 msgr2=0x7f71641993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:10.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.194+0000 7f7168bab640 1 --2- 172.21.15.164:0/2290619701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7164100e20 0x7f71641993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:10.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.194+0000 7f7168bab640 1 -- 172.21.15.164:0/2290619701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7164102ef0 msgr2=0x7f71641a4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:10.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.194+0000 7f7168bab640 1 --2- 172.21.15.164:0/2290619701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7164102ef0 0x7f71641a4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:10.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.194+0000 7f7168bab640 1 -- 172.21.15.164:0/2290619701 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71641a4990 con 0x7f7164102570 2024-09-19T08:56:10.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.194+0000 7f7168bab640 1 --2- 172.21.15.164:0/2290619701 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7164102570 0x7f7164199910 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f715400c970 tx=0x7f715400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:10.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.195+0000 7f715a7fc640 1 -- 172.21.15.164:0/2290619701 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7154007bb0 con 0x7f7164102570 2024-09-19T08:56:10.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.195+0000 7f716b610640 1 -- 172.21.15.164:0/2290619701 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71641a4b60 con 0x7f7164102570 2024-09-19T08:56:10.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.195+0000 7f715a7fc640 1 -- 172.21.15.164:0/2290619701 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7154007d50 con 0x7f7164102570 2024-09-19T08:56:10.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.195+0000 7f715a7fc640 1 -- 172.21.15.164:0/2290619701 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715400f450 con 0x7f7164102570 2024-09-19T08:56:10.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.195+0000 7f716b610640 1 -- 172.21.15.164:0/2290619701 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71641a5070 con 0x7f7164102570 2024-09-19T08:56:10.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.197+0000 7f715a7fc640 1 -- 172.21.15.164:0/2290619701 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f7154005020 con 0x7f7164102570 2024-09-19T08:56:10.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.197+0000 7f716d9e9740 1 Processor -- start 2024-09-19T08:56:10.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.197+0000 7f716d9e9740 1 -- start start 2024-09-19T08:56:10.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.198+0000 7f716d9e9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ea5af6b40 0x558ea5af5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:10.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.198+0000 7f716d9e9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ea5af55d0 0x558ea5af5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:10.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.198+0000 7f716d9e9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ea5b04780 0x558ea5b06b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:10.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.198+0000 7f716d9e9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558ea5a5b2f0 con 0x558ea5b04780 2024-09-19T08:56:10.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.198+0000 7f716d9e9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558ea5a02cf0 con 0x558ea5af55d0 2024-09-19T08:56:10.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.198+0000 7f716d9e9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558ea5a572b0 con 0x558ea5af6b40 2024-09-19T08:56:10.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.198+0000 7f715a7fc640 1 --2- 172.21.15.164:0/2290619701 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f71500a33b0 0x7f71500a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:10.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.198+0000 7f7168bab640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ea5b04780 0x558ea5b06b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:10.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.198+0000 7f7168bab640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ea5b04780 0x558ea5b06b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50164/0 (socket says 172.21.15.164:50164) 2024-09-19T08:56:10.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.198+0000 7f7168bab640 1 -- 172.21.15.164:0/1893550478 learned_addr learned my addr 172.21.15.164:0/1893550478 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:10.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.198+0000 7f7169bad640 1 --2- 172.21.15.164:0/2290619701 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f71500a33b0 0x7f71500a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:10.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.198+0000 7f71693ac640 1 --2- 172.21.15.164:0/1893550478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ea5af55d0 0x558ea5af5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:10.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.198+0000 7f7169bad640 1 --2- 172.21.15.164:0/1893550478 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ea5af6b40 0x558ea5af5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:10.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.198+0000 7f715a7fc640 1 -- 172.21.15.164:0/2290619701 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7f71540cfb00 con 0x7f7164102570 2024-09-19T08:56:10.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.198+0000 7f7168bab640 1 -- 172.21.15.164:0/1893550478 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ea5af6b40 msgr2=0x558ea5af5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:10.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.198+0000 7f7168bab640 1 --2- 172.21.15.164:0/1893550478 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ea5af6b40 0x558ea5af5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:10.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.198+0000 7f7168bab640 1 -- 172.21.15.164:0/1893550478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ea5af55d0 msgr2=0x558ea5af5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:10.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.198+0000 7f7168bab640 1 --2- 172.21.15.164:0/1893550478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ea5af55d0 0x558ea5af5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:10.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.198+0000 7f7168bab640 1 -- 172.21.15.164:0/1893550478 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ea59e73a0 con 0x558ea5b04780 2024-09-19T08:56:10.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.198+0000 7f71693ac640 1 --2- 172.21.15.164:0/1893550478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ea5af55d0 0x558ea5af5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:56:10.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.198+0000 7f7168bab640 1 --2- 172.21.15.164:0/1893550478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ea5b04780 0x558ea5b06b70 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f71540987c0 tx=0x7f7154004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c64ba4f207aff7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:10.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.198+0000 7f7169bad640 1 --2- 172.21.15.164:0/2290619701 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f71500a33b0 0x7f71500a5870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f715c010510 tx=0x7f715c00b750 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:10.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.199+0000 7f714f7fe640 1 -- 172.21.15.164:0/1893550478 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715401d070 con 0x558ea5b04780 2024-09-19T08:56:10.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.199+0000 7f714f7fe640 1 -- 172.21.15.164:0/1893550478 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f715409bdf0 con 0x558ea5b04780 2024-09-19T08:56:10.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.199+0000 7f714f7fe640 1 -- 172.21.15.164:0/1893550478 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7154098460 con 0x558ea5b04780 2024-09-19T08:56:10.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.199+0000 7f716d9e9740 1 -- 172.21.15.164:0/1893550478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ea5b04780 msgr2=0x558ea5b06b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:10.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.199+0000 7f716d9e9740 1 --2- 172.21.15.164:0/1893550478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ea5b04780 0x558ea5b06b70 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f71540987c0 tx=0x7f7154004a20 comp rx=0 tx=0).stop 2024-09-19T08:56:10.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.199+0000 7f716d9e9740 1 -- 172.21.15.164:0/1893550478 shutdown_connections 2024-09-19T08:56:10.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.199+0000 7f716d9e9740 1 --2- 172.21.15.164:0/1893550478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ea5b04780 0x558ea5b06b70 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:10.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.199+0000 7f716d9e9740 1 --2- 172.21.15.164:0/1893550478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ea5af55d0 0x558ea5af5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:10.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.199+0000 7f716d9e9740 1 --2- 172.21.15.164:0/1893550478 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ea5af6b40 0x558ea5af5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:10.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.199+0000 7f716d9e9740 1 -- 172.21.15.164:0/1893550478 >> 172.21.15.164:0/1893550478 conn(0x558ea5abe310 msgr2=0x558ea5af1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:10.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.199+0000 7f716d9e9740 1 -- 172.21.15.164:0/1893550478 shutdown_connections 2024-09-19T08:56:10.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.199+0000 7f716d9e9740 1 -- 172.21.15.164:0/1893550478 wait complete. 2024-09-19T08:56:10.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.200+0000 7f716d9e9740 1 -- start start 2024-09-19T08:56:10.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.200+0000 7f716d9e9740 1 Processor -- start 2024-09-19T08:56:10.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.200+0000 7f716d9e9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ea5af55d0 0x558ea5b041c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:10.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.200+0000 7f716d9e9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ea5af6b40 0x558ea5b01470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:10.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.200+0000 7f716d9e9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ea5b04780 0x558ea5b01c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:10.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.200+0000 7f716d9e9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558ea5b07480 con 0x558ea5b04780 2024-09-19T08:56:10.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.200+0000 7f716d9e9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558ea5a16060 con 0x558ea5af6b40 2024-09-19T08:56:10.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.200+0000 7f716d9e9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558ea5b07600 con 0x558ea5af55d0 2024-09-19T08:56:10.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.200+0000 7f716d9e9740 10 client.0 _ll_register_callbacks cb 0x558ea57698c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:56:10.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.200+0000 7f7169bad640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ea5b04780 0x558ea5b01c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:10.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.200+0000 7f7169bad640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ea5b04780 0x558ea5b01c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50180/0 (socket says 172.21.15.164:50180) 2024-09-19T08:56:10.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.200+0000 7f7169bad640 1 -- 172.21.15.164:0/3147558102 learned_addr learned my addr 172.21.15.164:0/3147558102 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:10.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.201+0000 7f71693ac640 1 --2- 172.21.15.164:0/3147558102 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ea5af55d0 0x558ea5b041c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:10.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.201+0000 7f7168bab640 1 --2- 172.21.15.164:0/3147558102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ea5af6b40 0x558ea5b01470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:10.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.201+0000 7f7169bad640 1 -- 172.21.15.164:0/3147558102 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ea5af55d0 msgr2=0x558ea5b041c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:10.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.201+0000 7f7169bad640 1 --2- 172.21.15.164:0/3147558102 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ea5af55d0 0x558ea5b041c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:10.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.201+0000 7f7169bad640 1 -- 172.21.15.164:0/3147558102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ea5af6b40 msgr2=0x558ea5b01470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:10.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.201+0000 7f7169bad640 1 --2- 172.21.15.164:0/3147558102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ea5af6b40 0x558ea5b01470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:10.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.201+0000 7f7169bad640 1 -- 172.21.15.164:0/3147558102 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ea57cc3c0 con 0x558ea5b04780 2024-09-19T08:56:10.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.201+0000 7f7168bab640 1 --2- 172.21.15.164:0/3147558102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ea5af6b40 0x558ea5b01470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:56:10.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.201+0000 7f7169bad640 1 --2- 172.21.15.164:0/3147558102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ea5b04780 0x558ea5b01c00 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f715c011010 tx=0x7f715c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab910292cc7bbd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:10.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.201+0000 7f714cff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:56:10.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.201+0000 7f714cff9640 1 -- 172.21.15.164:0/3147558102 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c025070 con 0x558ea5b04780 2024-09-19T08:56:10.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.201+0000 7f716d9e9740 1 -- 172.21.15.164:0/3147558102 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f715409bdf0 con 0x558ea5b04780 2024-09-19T08:56:10.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.201+0000 7f714cff9640 1 -- 172.21.15.164:0/3147558102 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7f715c01e9b0 con 0x558ea5b04780 2024-09-19T08:56:10.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.201+0000 7f714cff9640 1 -- 172.21.15.164:0/3147558102 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f715c01ebb0 con 0x558ea5b04780 2024-09-19T08:56:10.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.201+0000 7f716d9e9740 10 client.17664 fetch_fsmap learned FSMap version 32 2024-09-19T08:56:10.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.201+0000 7f716d9e9740 1 -- 172.21.15.164:0/3147558102 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558ea5b13020 con 0x558ea5b04780 2024-09-19T08:56:10.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.201+0000 7f714cff9640 1 -- 172.21.15.164:0/3147558102 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f715409bdf0 con 0x558ea5b04780 2024-09-19T08:56:10.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.202+0000 7f714cff9640 1 -- 172.21.15.164:0/3147558102 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f715c01c040 con 0x558ea5b04780 2024-09-19T08:56:10.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.202+0000 7f716d9e9740 10 client.17664 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:56:10.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.202+0000 7f716d9e9740 10 client.17664 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T08:56:10.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.202+0000 7f716d9e9740 20 client.17664 populate_metadata read hostname 'smithi164' 2024-09-19T08:56:10.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.202+0000 7f716d9e9740 1 --2- 172.21.15.164:0/3147558102 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x558ea5b14430 0x558ea5b16880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:10.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.202+0000 7f716d9e9740 4 client.17664 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:56:10.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.202+0000 7f716d9e9740 1 -- 172.21.15.164:0/3147558102 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ea5a68860 con 0x558ea5b14430 2024-09-19T08:56:10.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.202+0000 7f71693ac640 1 --2- 172.21.15.164:0/3147558102 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x558ea5b14430 0x558ea5b16880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:10.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.202+0000 7f714cff9640 10 client.17664 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:56:10.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.202+0000 7f71693ac640 1 --2- 172.21.15.164:0/3147558102 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x558ea5b14430 0x558ea5b16880 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a8ff46adfaf832d6 server_cookie=5aa625223908c70e in_seq=0 out_seq=0 2024-09-19T08:56:10.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.203+0000 7f714cff9640 1 -- 172.21.15.164:0/3147558102 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558ea5a68860 con 0x558ea5b14430 2024-09-19T08:56:10.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.203+0000 7f714cff9640 10 client.17664 handle_command_reply: tid=0 2024-09-19T08:56:10.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.203+0000 7f716d9e9740 1 client.17664 shutdown 2024-09-19T08:56:10.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.203+0000 7f716d9e9740 1 -- 172.21.15.164:0/3147558102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ea5b04780 msgr2=0x558ea5b01c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:10.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.203+0000 7f716d9e9740 1 --2- 172.21.15.164:0/3147558102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ea5b04780 0x558ea5b01c00 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f715c011010 tx=0x7f715c004730 comp rx=0 tx=0).stop 2024-09-19T08:56:10.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.203+0000 7f716d9e9740 1 -- 172.21.15.164:0/3147558102 shutdown_connections 2024-09-19T08:56:10.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.203+0000 7f716d9e9740 1 --2- 172.21.15.164:0/3147558102 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x558ea5b14430 0x558ea5b16880 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:10.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.203+0000 7f716d9e9740 1 --2- 172.21.15.164:0/3147558102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ea5b04780 0x558ea5b01c00 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:10.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.203+0000 7f716d9e9740 1 --2- 172.21.15.164:0/3147558102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ea5af6b40 0x558ea5b01470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:10.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.203+0000 7f714cff9640 0 client.17664 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:56:10.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.203+0000 7f716d9e9740 1 --2- 172.21.15.164:0/3147558102 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ea5af55d0 0x558ea5b041c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:10.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.203+0000 7f716d9e9740 1 -- 172.21.15.164:0/3147558102 >> 172.21.15.164:0/3147558102 conn(0x558ea5abe310 msgr2=0x558ea5af7990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:10.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.203+0000 7f716d9e9740 1 -- 172.21.15.164:0/3147558102 shutdown_connections 2024-09-19T08:56:10.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.203+0000 7f716d9e9740 1 -- 172.21.15.164:0/3147558102 wait complete. 2024-09-19T08:56:10.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.204+0000 7f716d9e9740 20 client.17664 trim_cache size 0 max 16384 2024-09-19T08:56:10.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.217+0000 7f716d9e9740 1 Processor -- start 2024-09-19T08:56:10.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.217+0000 7f716d9e9740 1 -- start start 2024-09-19T08:56:10.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.217+0000 7f716d9e9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ea5af55d0 0x558ea5af1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:10.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.217+0000 7f716d9e9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ea5af6b40 0x558ea5b1c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:10.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.217+0000 7f716d9e9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ea5b04780 0x558ea5b1c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:10.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.217+0000 7f716d9e9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f71641ab640 con 0x558ea5b04780 2024-09-19T08:56:10.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.217+0000 7f716d9e9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f71641ad180 con 0x558ea5af6b40 2024-09-19T08:56:10.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.217+0000 7f716d9e9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f71641abd10 con 0x558ea5af55d0 2024-09-19T08:56:10.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.217+0000 7f7169bad640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ea5b04780 0x558ea5b1c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:10.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.217+0000 7f7168bab640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ea5af6b40 0x558ea5b1c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:10.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.217+0000 7f71693ac640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ea5af55d0 0x558ea5af1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:10.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.217+0000 7f7169bad640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ea5b04780 0x558ea5b1c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50192/0 (socket says 172.21.15.164:50192) 2024-09-19T08:56:10.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.217+0000 7f7169bad640 1 -- 172.21.15.164:0/3746187912 learned_addr learned my addr 172.21.15.164:0/3746187912 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:10.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.218+0000 7f7169bad640 1 -- 172.21.15.164:0/3746187912 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ea5af55d0 msgr2=0x558ea5af1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:10.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.218+0000 7f7169bad640 1 --2- 172.21.15.164:0/3746187912 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ea5af55d0 0x558ea5af1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:10.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.218+0000 7f7169bad640 1 -- 172.21.15.164:0/3746187912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ea5af6b40 msgr2=0x558ea5b1c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:10.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.218+0000 7f7169bad640 1 --2- 172.21.15.164:0/3746187912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ea5af6b40 0x558ea5b1c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:10.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.218+0000 7f7169bad640 1 -- 172.21.15.164:0/3746187912 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ea5af2960 con 0x558ea5b04780 2024-09-19T08:56:10.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.218+0000 7f7168bab640 1 --2- 172.21.15.164:0/3746187912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ea5af6b40 0x558ea5b1c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:56:10.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.218+0000 7f7169bad640 1 --2- 172.21.15.164:0/3746187912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ea5b04780 0x558ea5b1c6e0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f715c0295b0 tx=0x7f715c01e3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36faf2504ec5b89 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:10.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.218+0000 7f714effd640 1 -- 172.21.15.164:0/3746187912 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c025070 con 0x558ea5b04780 2024-09-19T08:56:10.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.218+0000 7f714effd640 1 -- 172.21.15.164:0/3746187912 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f715c01e7b0 con 0x558ea5b04780 2024-09-19T08:56:10.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.218+0000 7f714effd640 1 -- 172.21.15.164:0/3746187912 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c0035f0 con 0x558ea5b04780 2024-09-19T08:56:10.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.219+0000 7f716d9e9740 1 -- 172.21.15.164:0/3746187912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ea5b04780 msgr2=0x558ea5b1c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:10.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.219+0000 7f716d9e9740 1 --2- 172.21.15.164:0/3746187912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ea5b04780 0x558ea5b1c6e0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f715c0295b0 tx=0x7f715c01e3e0 comp rx=0 tx=0).stop 2024-09-19T08:56:10.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.219+0000 7f716d9e9740 1 -- 172.21.15.164:0/3746187912 shutdown_connections 2024-09-19T08:56:10.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.219+0000 7f716d9e9740 1 --2- 172.21.15.164:0/3746187912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ea5b04780 0x558ea5b1c6e0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:10.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.219+0000 7f716d9e9740 1 --2- 172.21.15.164:0/3746187912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ea5af6b40 0x558ea5b1c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:10.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.219+0000 7f716d9e9740 1 --2- 172.21.15.164:0/3746187912 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ea5af55d0 0x558ea5af1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:10.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.219+0000 7f716d9e9740 1 -- 172.21.15.164:0/3746187912 >> 172.21.15.164:0/3746187912 conn(0x558ea5abe310 msgr2=0x558ea5af1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:10.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.219+0000 7f716d9e9740 1 -- 172.21.15.164:0/3746187912 shutdown_connections 2024-09-19T08:56:10.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.219+0000 7f716d9e9740 1 -- 172.21.15.164:0/3746187912 wait complete. 2024-09-19T08:56:10.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.220+0000 7f716d9e9740 1 -- start start 2024-09-19T08:56:10.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.220+0000 7f716d9e9740 1 Processor -- start 2024-09-19T08:56:10.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.220+0000 7f716d9e9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ea5af55d0 0x558ea5b08d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:10.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.220+0000 7f716d9e9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ea5af6b40 0x558ea5b09250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:10.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.220+0000 7f716d9e9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ea5b04780 0x558ea5b7e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:10.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.220+0000 7f716d9e9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f71641acab0 con 0x558ea5b04780 2024-09-19T08:56:10.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.220+0000 7f716d9e9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f71641ac3e0 con 0x558ea5af55d0 2024-09-19T08:56:10.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.220+0000 7f716d9e9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f71641a9640 con 0x558ea5af6b40 2024-09-19T08:56:10.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.220+0000 7f716d9e9740 10 client.0 _ll_register_callbacks cb 0x7f715c00b900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:56:10.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.220+0000 7f7169bad640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ea5b04780 0x558ea5b7e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:10.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.220+0000 7f71693ac640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ea5af55d0 0x558ea5b08d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:10.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.220+0000 7f7169bad640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ea5b04780 0x558ea5b7e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50194/0 (socket says 172.21.15.164:50194) 2024-09-19T08:56:10.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.220+0000 7f7169bad640 1 -- 172.21.15.164:0/392205501 learned_addr learned my addr 172.21.15.164:0/392205501 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:10.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.220+0000 7f71693ac640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ea5af55d0 0x558ea5b08d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:50206/0 (socket says 172.21.15.164:50206) 2024-09-19T08:56:10.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.220+0000 7f7168bab640 1 --2- 172.21.15.164:0/392205501 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ea5af6b40 0x558ea5b09250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:10.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.221+0000 7f71693ac640 1 -- 172.21.15.164:0/392205501 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ea5af6b40 msgr2=0x558ea5b09250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:10.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.221+0000 7f71693ac640 1 --2- 172.21.15.164:0/392205501 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ea5af6b40 0x558ea5b09250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:10.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.221+0000 7f71693ac640 1 -- 172.21.15.164:0/392205501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ea5b04780 msgr2=0x558ea5b7e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:10.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.221+0000 7f71693ac640 1 --2- 172.21.15.164:0/392205501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ea5b04780 0x558ea5b7e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:10.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.221+0000 7f71693ac640 1 -- 172.21.15.164:0/392205501 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ea5b7e6b0 con 0x558ea5af55d0 2024-09-19T08:56:10.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.221+0000 7f7169bad640 1 --2- 172.21.15.164:0/392205501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ea5b04780 0x558ea5b7e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:56:10.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.221+0000 7f71693ac640 1 --2- 172.21.15.164:0/392205501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ea5af55d0 0x558ea5b08d10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7160000f80 tx=0x7f71600095d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdab75d430deeebc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:10.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.221+0000 7f714cff9640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-19T08:56:10.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.221+0000 7f714cff9640 1 -- 172.21.15.164:0/392205501 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716004b070 con 0x558ea5af55d0 2024-09-19T08:56:10.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.221+0000 7f714cff9640 1 -- 172.21.15.164:0/392205501 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7f716002f5a0 con 0x558ea5af55d0 2024-09-19T08:56:10.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.221+0000 7f716d9e9740 1 -- 172.21.15.164:0/392205501 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f715c01e7b0 con 0x558ea5af55d0 2024-09-19T08:56:10.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.221+0000 7f714cff9640 1 -- 172.21.15.164:0/392205501 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7160057050 con 0x558ea5af55d0 2024-09-19T08:56:10.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.221+0000 7f716d9e9740 10 client.26756 fetch_fsmap learned FSMap version 32 2024-09-19T08:56:10.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.221+0000 7f716d9e9740 1 -- 172.21.15.164:0/392205501 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558ea5b7f9f0 con 0x558ea5af55d0 2024-09-19T08:56:10.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.221+0000 7f714cff9640 1 -- 172.21.15.164:0/392205501 <== mon.2 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f715c01e7b0 con 0x558ea5af55d0 2024-09-19T08:56:10.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.222+0000 7f714cff9640 1 -- 172.21.15.164:0/392205501 <== mon.2 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f7160004560 con 0x558ea5af55d0 2024-09-19T08:56:10.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.222+0000 7f716d9e9740 10 client.26756 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:56:10.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.222+0000 7f716d9e9740 10 client.26756 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T08:56:10.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.222+0000 7f716d9e9740 20 client.26756 populate_metadata read hostname 'smithi164' 2024-09-19T08:56:10.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.222+0000 7f716d9e9740 1 --2- 172.21.15.164:0/392205501 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x558ea5b80110 0x558ea5b82530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:10.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.222+0000 7f716d9e9740 4 client.26756 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:56:10.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.222+0000 7f716d9e9740 1 -- 172.21.15.164:0/392205501 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ea5a62d70 con 0x558ea5b80110 2024-09-19T08:56:10.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.222+0000 7f7168bab640 1 --2- 172.21.15.164:0/392205501 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x558ea5b80110 0x558ea5b82530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:10.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.222+0000 7f7168bab640 1 --2- 172.21.15.164:0/392205501 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x558ea5b80110 0x558ea5b82530 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=79252e2e0b62a7ce server_cookie=b96031b8cf42d5d1 in_seq=0 out_seq=0 2024-09-19T08:56:10.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.222+0000 7f714cff9640 10 client.26756 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:56:10.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.223+0000 7f714cff9640 1 -- 172.21.15.164:0/392205501 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ea5a62d70 con 0x558ea5b80110 2024-09-19T08:56:10.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.223+0000 7f714cff9640 10 client.26756 handle_command_reply: tid=0 2024-09-19T08:56:10.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.223+0000 7f716d9e9740 1 client.26756 shutdown 2024-09-19T08:56:10.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.223+0000 7f716d9e9740 1 -- 172.21.15.164:0/392205501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ea5af55d0 msgr2=0x558ea5b08d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:10.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.223+0000 7f716d9e9740 1 --2- 172.21.15.164:0/392205501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ea5af55d0 0x558ea5b08d10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7160000f80 tx=0x7f71600095d0 comp rx=0 tx=0).stop 2024-09-19T08:56:10.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.223+0000 7f716d9e9740 1 -- 172.21.15.164:0/392205501 shutdown_connections 2024-09-19T08:56:10.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.223+0000 7f716d9e9740 1 --2- 172.21.15.164:0/392205501 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x558ea5b80110 0x558ea5b82530 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:10.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.223+0000 7f716d9e9740 1 --2- 172.21.15.164:0/392205501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ea5b04780 0x558ea5b7e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:10.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.223+0000 7f714cff9640 0 client.26756 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:56:10.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.223+0000 7f716d9e9740 1 --2- 172.21.15.164:0/392205501 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ea5af6b40 0x558ea5b09250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:10.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.223+0000 7f716d9e9740 1 --2- 172.21.15.164:0/392205501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ea5af55d0 0x558ea5b08d10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:10.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.223+0000 7f716d9e9740 1 -- 172.21.15.164:0/392205501 >> 172.21.15.164:0/392205501 conn(0x558ea5abe310 msgr2=0x558ea5b02cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:10.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.223+0000 7f716d9e9740 1 -- 172.21.15.164:0/392205501 shutdown_connections 2024-09-19T08:56:10.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.223+0000 7f716d9e9740 1 -- 172.21.15.164:0/392205501 wait complete. 2024-09-19T08:56:10.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.223+0000 7f716d9e9740 20 client.26756 trim_cache size 0 max 16384 2024-09-19T08:56:10.239 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:56:10.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.223+0000 7f716b610640 1 -- 172.21.15.164:0/2290619701 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f71500a33b0 msgr2=0x7f71500a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:10.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.223+0000 7f716b610640 1 --2- 172.21.15.164:0/2290619701 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f71500a33b0 0x7f71500a5870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f715c010510 tx=0x7f715c00b750 comp rx=0 tx=0).stop 2024-09-19T08:56:10.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.223+0000 7f716b610640 1 -- 172.21.15.164:0/2290619701 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7164102570 msgr2=0x7f7164199910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:10.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.223+0000 7f716b610640 1 --2- 172.21.15.164:0/2290619701 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7164102570 0x7f7164199910 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f715400c970 tx=0x7f715400ce30 comp rx=0 tx=0).stop 2024-09-19T08:56:10.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.224+0000 7f716b610640 1 -- 172.21.15.164:0/2290619701 shutdown_connections 2024-09-19T08:56:10.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.224+0000 7f716b610640 1 --2- 172.21.15.164:0/2290619701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7164102ef0 0x7f71641a4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:10.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.224+0000 7f716b610640 1 --2- 172.21.15.164:0/2290619701 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f71500a33b0 0x7f71500a5870 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:10.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.224+0000 7f716b610640 1 --2- 172.21.15.164:0/2290619701 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7164102570 0x7f7164199910 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:10.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.224+0000 7f716b610640 1 --2- 172.21.15.164:0/2290619701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7164100e20 0x7f71641993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:10.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.224+0000 7f716b610640 1 -- 172.21.15.164:0/2290619701 >> 172.21.15.164:0/2290619701 conn(0x7f71640fc7d0 msgr2=0x7f71640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:10.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.224+0000 7f716b610640 1 -- 172.21.15.164:0/2290619701 shutdown_connections 2024-09-19T08:56:10.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:10.224+0000 7f716b610640 1 -- 172.21.15.164:0/2290619701 wait complete. 2024-09-19T08:56:10.520 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:10 smithi164 ceph-mon[30928]: pgmap v322: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 30 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T08:56:10.520 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:10 smithi164 ceph-mon[30928]: Adjusting osd_memory_target on smithi173 to 176.0M 2024-09-19T08:56:10.520 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:10 smithi164 ceph-mon[30928]: Unable to set osd_memory_target on smithi173 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-19T08:56:10.521 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:10 smithi164 ceph-mon[30928]: Adjusting osd_memory_target on smithi170 to 176.0M 2024-09-19T08:56:10.521 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:10 smithi164 ceph-mon[30928]: Unable to set osd_memory_target on smithi170 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-19T08:56:10.521 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:10 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:56:10.521 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:10 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:56:10.521 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:10 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:10.521 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:10 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:10.521 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:10 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:10.521 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:10 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:10.521 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:10 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:10.521 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:10 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:56:10.521 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:10 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:56:10.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:10 smithi170 ceph-mon[35072]: pgmap v322: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 30 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T08:56:10.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:10 smithi170 ceph-mon[35072]: Adjusting osd_memory_target on smithi173 to 176.0M 2024-09-19T08:56:10.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:10 smithi170 ceph-mon[35072]: Unable to set osd_memory_target on smithi173 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-19T08:56:10.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:10 smithi170 ceph-mon[35072]: Adjusting osd_memory_target on smithi170 to 176.0M 2024-09-19T08:56:10.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:10 smithi170 ceph-mon[35072]: Unable to set osd_memory_target on smithi170 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-19T08:56:10.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:10 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:56:10.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:10 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:56:10.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:10 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:10.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:10 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:10.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:10 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:10.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:10 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:10.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:10 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:10.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:10 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:56:10.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:10 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:56:10.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:10 smithi173 ceph-mon[35306]: pgmap v322: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 30 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T08:56:10.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:10 smithi173 ceph-mon[35306]: Adjusting osd_memory_target on smithi173 to 176.0M 2024-09-19T08:56:10.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:10 smithi173 ceph-mon[35306]: Unable to set osd_memory_target on smithi173 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-19T08:56:10.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:10 smithi173 ceph-mon[35306]: Adjusting osd_memory_target on smithi170 to 176.0M 2024-09-19T08:56:10.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:10 smithi173 ceph-mon[35306]: Unable to set osd_memory_target on smithi170 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-19T08:56:10.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:10 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:56:10.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:10 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:56:10.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:10 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:10.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:10 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:10.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:10 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:10.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:10 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T08:56:10.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:10 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:56:10.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:10 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:56:10.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:10 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:56:11.070 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T08:56:11.323 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:56:11.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:11 smithi164 ceph-mon[30928]: Adjusting osd_memory_target on smithi164 to 176.0M 2024-09-19T08:56:11.444 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:11 smithi164 ceph-mon[30928]: Unable to set osd_memory_target on smithi164 to 184581017: error parsing value: Value '184581017' is below minimum 939524096 2024-09-19T08:56:11.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:11 smithi170 ceph-mon[35072]: Adjusting osd_memory_target on smithi164 to 176.0M 2024-09-19T08:56:11.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:11 smithi170 ceph-mon[35072]: Unable to set osd_memory_target on smithi164 to 184581017: error parsing value: Value '184581017' is below minimum 939524096 2024-09-19T08:56:11.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:11 smithi173 ceph-mon[35306]: Adjusting osd_memory_target on smithi164 to 176.0M 2024-09-19T08:56:11.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:11 smithi173 ceph-mon[35306]: Unable to set osd_memory_target on smithi164 to 184581017: error parsing value: Value '184581017' is below minimum 939524096 2024-09-19T08:56:11.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.806+0000 7f9ad86fc640 1 -- 172.21.15.164:0/1253923356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ad0106a80 msgr2=0x7f9ad0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:11.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.806+0000 7f9ad86fc640 1 --2- 172.21.15.164:0/1253923356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ad0106a80 0x7f9ad0106e60 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f9acc009cd0 tx=0x7f9acc02a8e0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.806+0000 7f9ad86fc640 1 -- 172.21.15.164:0/1253923356 shutdown_connections 2024-09-19T08:56:11.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.806+0000 7f9ad86fc640 1 --2- 172.21.15.164:0/1253923356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ad0101420 0x7f9ad010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.806+0000 7f9ad86fc640 1 --2- 172.21.15.164:0/1253923356 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ad0100a80 0x7f9ad0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.806+0000 7f9ad86fc640 1 --2- 172.21.15.164:0/1253923356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ad0106a80 0x7f9ad0106e60 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.806+0000 7f9ad86fc640 1 -- 172.21.15.164:0/1253923356 >> 172.21.15.164:0/1253923356 conn(0x7f9ad00fc7a0 msgr2=0x7f9ad00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:11.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.806+0000 7f9ad86fc640 1 -- 172.21.15.164:0/1253923356 shutdown_connections 2024-09-19T08:56:11.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.806+0000 7f9ad86fc640 1 -- 172.21.15.164:0/1253923356 wait complete. 2024-09-19T08:56:11.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.806+0000 7f9ad86fc640 1 Processor -- start 2024-09-19T08:56:11.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.806+0000 7f9ad86fc640 1 -- start start 2024-09-19T08:56:11.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.806+0000 7f9ad86fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ad0100a80 0x7f9ad0197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:11.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.806+0000 7f9ad86fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ad0101420 0x7f9ad0197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:11.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.806+0000 7f9ad86fc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ad0106a80 0x7f9ad0197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:11.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.807+0000 7f9ad86fc640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad0110e80 con 0x7f9ad0101420 2024-09-19T08:56:11.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.807+0000 7f9ad86fc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad0110d00 con 0x7f9ad0100a80 2024-09-19T08:56:11.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.807+0000 7f9ad86fc640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad0111000 con 0x7f9ad0106a80 2024-09-19T08:56:11.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.807+0000 7f9ad5c97640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ad0101420 0x7f9ad0197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:11.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.807+0000 7f9ad5c97640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ad0101420 0x7f9ad0197640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50218/0 (socket says 172.21.15.164:50218) 2024-09-19T08:56:11.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.807+0000 7f9ad5c97640 1 -- 172.21.15.164:0/4047202273 learned_addr learned my addr 172.21.15.164:0/4047202273 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:11.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.807+0000 7f9ad6498640 1 --2- 172.21.15.164:0/4047202273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ad0100a80 0x7f9ad0197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:11.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.807+0000 7f9ad5c97640 1 -- 172.21.15.164:0/4047202273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ad0106a80 msgr2=0x7f9ad0197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:11.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.807+0000 7f9ad5c97640 1 --2- 172.21.15.164:0/4047202273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ad0106a80 0x7f9ad0197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.807+0000 7f9ad5c97640 1 -- 172.21.15.164:0/4047202273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ad0100a80 msgr2=0x7f9ad0197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:11.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.807+0000 7f9ad5c97640 1 --2- 172.21.15.164:0/4047202273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ad0100a80 0x7f9ad0197100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.807+0000 7f9ad5c97640 1 -- 172.21.15.164:0/4047202273 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad01a6970 con 0x7f9ad0101420 2024-09-19T08:56:11.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.807+0000 7f9ad5c97640 1 --2- 172.21.15.164:0/4047202273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ad0101420 0x7f9ad0197640 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000d770 tx=0x7f9ac000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:11.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.807+0000 7f9ac77fe640 1 -- 172.21.15.164:0/4047202273 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0017070 con 0x7f9ad0101420 2024-09-19T08:56:11.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.807+0000 7f9ac77fe640 1 -- 172.21.15.164:0/4047202273 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ac0005500 con 0x7f9ad0101420 2024-09-19T08:56:11.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.807+0000 7f9ad86fc640 1 -- 172.21.15.164:0/4047202273 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ad01a6c60 con 0x7f9ad0101420 2024-09-19T08:56:11.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.807+0000 7f9ac77fe640 1 -- 172.21.15.164:0/4047202273 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0004e00 con 0x7f9ad0101420 2024-09-19T08:56:11.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.807+0000 7f9ad86fc640 1 -- 172.21.15.164:0/4047202273 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ad01a7150 con 0x7f9ad0101420 2024-09-19T08:56:11.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.810+0000 7f9ac77fe640 1 -- 172.21.15.164:0/4047202273 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f9ac0004fa0 con 0x7f9ad0101420 2024-09-19T08:56:11.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.810+0000 7f9adaad5740 1 Processor -- start 2024-09-19T08:56:11.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.810+0000 7f9adaad5740 1 -- start start 2024-09-19T08:56:11.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.810+0000 7f9adaad5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644e1418b40 0x5644e1417090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:11.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.810+0000 7f9adaad5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644e14175d0 0x5644e1417a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:11.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.810+0000 7f9adaad5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5644e1422370 0x5644e1424760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:11.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.810+0000 7f9adaad5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5644e137d2f0 con 0x5644e14175d0 2024-09-19T08:56:11.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.810+0000 7f9adaad5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5644e1324cf0 con 0x5644e1418b40 2024-09-19T08:56:11.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.810+0000 7f9adaad5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5644e13792b0 con 0x5644e1422370 2024-09-19T08:56:11.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.811+0000 7f9ad6498640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644e14175d0 0x5644e1417a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:11.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.811+0000 7f9ad6498640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644e14175d0 0x5644e1417a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50224/0 (socket says 172.21.15.164:50224) 2024-09-19T08:56:11.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.811+0000 7f9ad6498640 1 -- 172.21.15.164:0/3478531246 learned_addr learned my addr 172.21.15.164:0/3478531246 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:11.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.811+0000 7f9ad6c99640 1 --2- 172.21.15.164:0/3478531246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644e1418b40 0x5644e1417090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:11.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.811+0000 7f9ad5c97640 1 --2- 172.21.15.164:0/3478531246 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5644e1422370 0x5644e1424760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:11.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.811+0000 7f9ac77fe640 1 --2- 172.21.15.164:0/4047202273 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f9abc0a33b0 0x7f9abc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:11.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.811+0000 7f9ad6498640 1 -- 172.21.15.164:0/3478531246 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5644e1422370 msgr2=0x5644e1424760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:11.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.811+0000 7f9ad6498640 1 --2- 172.21.15.164:0/3478531246 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5644e1422370 0x5644e1424760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.811+0000 7f9ad6498640 1 -- 172.21.15.164:0/3478531246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644e1418b40 msgr2=0x5644e1417090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:11.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.811+0000 7f9ad6498640 1 --2- 172.21.15.164:0/3478531246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644e1418b40 0x5644e1417090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.811+0000 7f9ad6498640 1 -- 172.21.15.164:0/3478531246 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644e13093a0 con 0x5644e14175d0 2024-09-19T08:56:11.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.811+0000 7f9ad6c99640 1 --2- 172.21.15.164:0/3478531246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644e1418b40 0x5644e1417090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:56:11.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.811+0000 7f9ad5c97640 1 --2- 172.21.15.164:0/3478531246 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5644e1422370 0x5644e1424760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:56:11.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.811+0000 7f9ac77fe640 1 -- 172.21.15.164:0/4047202273 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7f9ac00cf840 con 0x7f9ad0101420 2024-09-19T08:56:11.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.811+0000 7f9ad6c99640 1 --2- 172.21.15.164:0/4047202273 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f9abc0a33b0 0x7f9abc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:11.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.811+0000 7f9ad6498640 1 --2- 172.21.15.164:0/3478531246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644e14175d0 0x5644e1417a10 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f9acc000f80 tx=0x7f9acc009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3618446a0f7889 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:11.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.811+0000 7f9ac4ff9640 1 -- 172.21.15.164:0/3478531246 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc0318c0 con 0x5644e14175d0 2024-09-19T08:56:11.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.811+0000 7f9ac4ff9640 1 -- 172.21.15.164:0/3478531246 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9acc036040 con 0x5644e14175d0 2024-09-19T08:56:11.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.811+0000 7f9ac4ff9640 1 -- 172.21.15.164:0/3478531246 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc004620 con 0x5644e14175d0 2024-09-19T08:56:11.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.812+0000 7f9adaad5740 1 -- 172.21.15.164:0/3478531246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644e14175d0 msgr2=0x5644e1417a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:11.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.812+0000 7f9ad6c99640 1 --2- 172.21.15.164:0/4047202273 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f9abc0a33b0 0x7f9abc0a5870 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8005fd0 tx=0x7f9ac80046c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:11.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.812+0000 7f9adaad5740 1 --2- 172.21.15.164:0/3478531246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644e14175d0 0x5644e1417a10 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f9acc000f80 tx=0x7f9acc009a20 comp rx=0 tx=0).stop 2024-09-19T08:56:11.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.812+0000 7f9adaad5740 1 -- 172.21.15.164:0/3478531246 shutdown_connections 2024-09-19T08:56:11.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.812+0000 7f9adaad5740 1 --2- 172.21.15.164:0/3478531246 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5644e1422370 0x5644e1424760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.812+0000 7f9adaad5740 1 --2- 172.21.15.164:0/3478531246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644e14175d0 0x5644e1417a10 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.812+0000 7f9adaad5740 1 --2- 172.21.15.164:0/3478531246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644e1418b40 0x5644e1417090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.812+0000 7f9adaad5740 1 -- 172.21.15.164:0/3478531246 >> 172.21.15.164:0/3478531246 conn(0x5644e13e0310 msgr2=0x5644e1413c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:11.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.812+0000 7f9adaad5740 1 -- 172.21.15.164:0/3478531246 shutdown_connections 2024-09-19T08:56:11.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.812+0000 7f9adaad5740 1 -- 172.21.15.164:0/3478531246 wait complete. 2024-09-19T08:56:11.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.813+0000 7f9adaad5740 1 -- start start 2024-09-19T08:56:11.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.813+0000 7f9adaad5740 1 Processor -- start 2024-09-19T08:56:11.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.813+0000 7f9adaad5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5644e14175d0 0x5644e1421d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:11.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.813+0000 7f9adaad5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644e1418b40 0x5644e141efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:11.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.813+0000 7f9adaad5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644e1422370 0x5644e141f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:11.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.813+0000 7f9adaad5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5644e1425010 con 0x5644e1418b40 2024-09-19T08:56:11.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.813+0000 7f9adaad5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5644e1338060 con 0x5644e1422370 2024-09-19T08:56:11.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.813+0000 7f9adaad5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5644e1425190 con 0x5644e14175d0 2024-09-19T08:56:11.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.813+0000 7f9adaad5740 10 client.0 _ll_register_callbacks cb 0x5644e108b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:56:11.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.813+0000 7f9ad5c97640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644e1418b40 0x5644e141efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:11.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.813+0000 7f9ad5c97640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644e1418b40 0x5644e141efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50238/0 (socket says 172.21.15.164:50238) 2024-09-19T08:56:11.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.813+0000 7f9ad5c97640 1 -- 172.21.15.164:0/3409477859 learned_addr learned my addr 172.21.15.164:0/3409477859 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:11.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.813+0000 7f9ad6498640 1 --2- 172.21.15.164:0/3409477859 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5644e14175d0 0x5644e1421d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:11.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.813+0000 7f9ad6c99640 1 --2- 172.21.15.164:0/3409477859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644e1422370 0x5644e141f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:11.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.813+0000 7f9ad5c97640 1 -- 172.21.15.164:0/3409477859 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5644e14175d0 msgr2=0x5644e1421d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:11.849 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.813+0000 7f9ad5c97640 1 --2- 172.21.15.164:0/3409477859 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5644e14175d0 0x5644e1421d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.813+0000 7f9ad5c97640 1 -- 172.21.15.164:0/3409477859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644e1422370 msgr2=0x5644e141f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:11.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.813+0000 7f9ad5c97640 1 --2- 172.21.15.164:0/3409477859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644e1422370 0x5644e141f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.813+0000 7f9ad5c97640 1 -- 172.21.15.164:0/3409477859 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644e10ee3c0 con 0x5644e1418b40 2024-09-19T08:56:11.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.813+0000 7f9ad6498640 1 --2- 172.21.15.164:0/3409477859 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5644e14175d0 0x5644e1421d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:56:11.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.813+0000 7f9ad5c97640 1 --2- 172.21.15.164:0/3409477859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644e1418b40 0x5644e141efc0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f9ac00cd000 tx=0x7f9ac0097dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce632007620e7cdc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:11.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.813+0000 7f9ab9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:56:11.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.814+0000 7f9ab9ffb640 1 -- 172.21.15.164:0/3409477859 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0020070 con 0x5644e1418b40 2024-09-19T08:56:11.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.814+0000 7f9ab9ffb640 1 -- 172.21.15.164:0/3409477859 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7f9ac00cd220 con 0x5644e1418b40 2024-09-19T08:56:11.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.814+0000 7f9adaad5740 1 -- 172.21.15.164:0/3409477859 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9acc036040 con 0x5644e1418b40 2024-09-19T08:56:11.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.814+0000 7f9ab9ffb640 1 -- 172.21.15.164:0/3409477859 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9ac009bb30 con 0x5644e1418b40 2024-09-19T08:56:11.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.814+0000 7f9adaad5740 10 client.17682 fetch_fsmap learned FSMap version 32 2024-09-19T08:56:11.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.814+0000 7f9adaad5740 1 -- 172.21.15.164:0/3409477859 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5644e14309d0 con 0x5644e1418b40 2024-09-19T08:56:11.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.814+0000 7f9ab9ffb640 1 -- 172.21.15.164:0/3409477859 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9acc036040 con 0x5644e1418b40 2024-09-19T08:56:11.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.814+0000 7f9ab9ffb640 1 -- 172.21.15.164:0/3409477859 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f9ac00860b0 con 0x5644e1418b40 2024-09-19T08:56:11.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.814+0000 7f9adaad5740 10 client.17682 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:56:11.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.814+0000 7f9adaad5740 10 client.17682 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T08:56:11.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.814+0000 7f9adaad5740 20 client.17682 populate_metadata read hostname 'smithi164' 2024-09-19T08:56:11.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.814+0000 7f9adaad5740 1 --2- 172.21.15.164:0/3409477859 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x5644e1431e90 0x5644e14342e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:11.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.814+0000 7f9adaad5740 4 client.17682 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:56:11.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.814+0000 7f9adaad5740 1 -- 172.21.15.164:0/3409477859 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644e138a860 con 0x5644e1431e90 2024-09-19T08:56:11.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.815+0000 7f9ad6498640 1 --2- 172.21.15.164:0/3409477859 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x5644e1431e90 0x5644e14342e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:11.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.815+0000 7f9ad6498640 1 --2- 172.21.15.164:0/3409477859 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x5644e1431e90 0x5644e14342e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c5bf3480812649f9 server_cookie=10f4572fef3f16eb in_seq=0 out_seq=0 2024-09-19T08:56:11.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.815+0000 7f9ab9ffb640 10 client.17682 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:56:11.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.816+0000 7f9ab9ffb640 1 -- 172.21.15.164:0/3409477859 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5644e138a860 con 0x5644e1431e90 2024-09-19T08:56:11.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.816+0000 7f9ab9ffb640 10 client.17682 handle_command_reply: tid=0 2024-09-19T08:56:11.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.816+0000 7f9adaad5740 1 client.17682 shutdown 2024-09-19T08:56:11.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.816+0000 7f9adaad5740 1 -- 172.21.15.164:0/3409477859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644e1418b40 msgr2=0x5644e141efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:11.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.816+0000 7f9adaad5740 1 --2- 172.21.15.164:0/3409477859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644e1418b40 0x5644e141efc0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f9ac00cd000 tx=0x7f9ac0097dd0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.816+0000 7f9adaad5740 1 -- 172.21.15.164:0/3409477859 shutdown_connections 2024-09-19T08:56:11.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.816+0000 7f9adaad5740 1 --2- 172.21.15.164:0/3409477859 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x5644e1431e90 0x5644e14342e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.816+0000 7f9adaad5740 1 --2- 172.21.15.164:0/3409477859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644e1422370 0x5644e141f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.816+0000 7f9adaad5740 1 --2- 172.21.15.164:0/3409477859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644e1418b40 0x5644e141efc0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.816+0000 7f9adaad5740 1 --2- 172.21.15.164:0/3409477859 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5644e14175d0 0x5644e1421d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.816+0000 7f9ab9ffb640 0 client.17682 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:56:11.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.816+0000 7f9adaad5740 1 -- 172.21.15.164:0/3409477859 >> 172.21.15.164:0/3409477859 conn(0x5644e13e0310 msgr2=0x5644e1419930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:11.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.816+0000 7f9adaad5740 1 -- 172.21.15.164:0/3409477859 shutdown_connections 2024-09-19T08:56:11.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.816+0000 7f9adaad5740 1 -- 172.21.15.164:0/3409477859 wait complete. 2024-09-19T08:56:11.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.816+0000 7f9adaad5740 20 client.17682 trim_cache size 0 max 16384 2024-09-19T08:56:11.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.829+0000 7f9adaad5740 1 Processor -- start 2024-09-19T08:56:11.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.829+0000 7f9adaad5740 1 -- start start 2024-09-19T08:56:11.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.829+0000 7f9adaad5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644e14175d0 0x5644e141a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:11.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.829+0000 7f9adaad5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644e1418b40 0x5644e141a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:11.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.829+0000 7f9adaad5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5644e1422370 0x5644e1439bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:11.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.829+0000 7f9adaad5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad01afa10 con 0x5644e14175d0 2024-09-19T08:56:11.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.829+0000 7f9adaad5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad01b1550 con 0x5644e1418b40 2024-09-19T08:56:11.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.829+0000 7f9adaad5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad01b00e0 con 0x5644e1422370 2024-09-19T08:56:11.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.829+0000 7f9ad6498640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644e14175d0 0x5644e141a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:11.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.829+0000 7f9ad6498640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644e14175d0 0x5644e141a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50250/0 (socket says 172.21.15.164:50250) 2024-09-19T08:56:11.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.829+0000 7f9ad6498640 1 -- 172.21.15.164:0/4221692693 learned_addr learned my addr 172.21.15.164:0/4221692693 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:11.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.829+0000 7f9ad5c97640 1 --2- 172.21.15.164:0/4221692693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644e1418b40 0x5644e141a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:11.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.829+0000 7f9ad6498640 1 -- 172.21.15.164:0/4221692693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5644e1422370 msgr2=0x5644e1439bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:11.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.829+0000 7f9ad6498640 1 --2- 172.21.15.164:0/4221692693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5644e1422370 0x5644e1439bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.829+0000 7f9ad6498640 1 -- 172.21.15.164:0/4221692693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644e1418b40 msgr2=0x5644e141a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:11.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.829+0000 7f9ad6498640 1 --2- 172.21.15.164:0/4221692693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644e1418b40 0x5644e141a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.829+0000 7f9ad6498640 1 -- 172.21.15.164:0/4221692693 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644e1412310 con 0x5644e14175d0 2024-09-19T08:56:11.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.829+0000 7f9ad5c97640 1 --2- 172.21.15.164:0/4221692693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644e1418b40 0x5644e141a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:56:11.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.829+0000 7f9ad6498640 1 --2- 172.21.15.164:0/4221692693 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644e14175d0 0x5644e141a120 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f9acc004af0 tx=0x7f9acc005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=108821bdf09e4d88 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:11.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.829+0000 7f9abbfff640 1 -- 172.21.15.164:0/4221692693 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc031670 con 0x5644e14175d0 2024-09-19T08:56:11.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.829+0000 7f9abbfff640 1 -- 172.21.15.164:0/4221692693 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9acc036040 con 0x5644e14175d0 2024-09-19T08:56:11.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.829+0000 7f9abbfff640 1 -- 172.21.15.164:0/4221692693 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc0044e0 con 0x5644e14175d0 2024-09-19T08:56:11.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.830+0000 7f9adaad5740 1 -- 172.21.15.164:0/4221692693 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644e14175d0 msgr2=0x5644e141a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:11.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.830+0000 7f9adaad5740 1 --2- 172.21.15.164:0/4221692693 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644e14175d0 0x5644e141a120 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f9acc004af0 tx=0x7f9acc005d80 comp rx=0 tx=0).stop 2024-09-19T08:56:11.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.830+0000 7f9adaad5740 1 -- 172.21.15.164:0/4221692693 shutdown_connections 2024-09-19T08:56:11.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.830+0000 7f9adaad5740 1 --2- 172.21.15.164:0/4221692693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5644e1422370 0x5644e1439bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.830+0000 7f9adaad5740 1 --2- 172.21.15.164:0/4221692693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644e1418b40 0x5644e141a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.830+0000 7f9adaad5740 1 --2- 172.21.15.164:0/4221692693 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644e14175d0 0x5644e141a120 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.830+0000 7f9adaad5740 1 -- 172.21.15.164:0/4221692693 >> 172.21.15.164:0/4221692693 conn(0x5644e13e0310 msgr2=0x5644e1421040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:11.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.830+0000 7f9adaad5740 1 -- 172.21.15.164:0/4221692693 shutdown_connections 2024-09-19T08:56:11.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.830+0000 7f9adaad5740 1 -- 172.21.15.164:0/4221692693 wait complete. 2024-09-19T08:56:11.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.830+0000 7f9adaad5740 1 -- start start 2024-09-19T08:56:11.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.830+0000 7f9adaad5740 1 Processor -- start 2024-09-19T08:56:11.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.831+0000 7f9adaad5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644e14175d0 0x5644e142a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:11.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.831+0000 7f9adaad5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5644e1418b40 0x5644e142a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:11.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.831+0000 7f9adaad5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644e1422370 0x5644e14273f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:11.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.831+0000 7f9adaad5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad01b0e80 con 0x5644e1422370 2024-09-19T08:56:11.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.831+0000 7f9adaad5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad01b07b0 con 0x5644e14175d0 2024-09-19T08:56:11.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.831+0000 7f9adaad5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad01ada10 con 0x5644e1418b40 2024-09-19T08:56:11.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.831+0000 7f9adaad5740 10 client.0 _ll_register_callbacks cb 0x7f9ac000d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:56:11.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.831+0000 7f9ad6c99640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644e1422370 0x5644e14273f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:11.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.831+0000 7f9ad6c99640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644e1422370 0x5644e14273f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50266/0 (socket says 172.21.15.164:50266) 2024-09-19T08:56:11.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.831+0000 7f9ad6c99640 1 -- 172.21.15.164:0/3552902860 learned_addr learned my addr 172.21.15.164:0/3552902860 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:11.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.831+0000 7f9ad6498640 1 --2- 172.21.15.164:0/3552902860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644e14175d0 0x5644e142a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:11.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.831+0000 7f9ad6c99640 1 -- 172.21.15.164:0/3552902860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5644e1418b40 msgr2=0x5644e142a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:11.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.831+0000 7f9ad5c97640 1 --2- 172.21.15.164:0/3552902860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5644e1418b40 0x5644e142a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:11.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.831+0000 7f9ad6c99640 1 --2- 172.21.15.164:0/3552902860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5644e1418b40 0x5644e142a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.831+0000 7f9ad6c99640 1 -- 172.21.15.164:0/3552902860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644e14175d0 msgr2=0x5644e142a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:11.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.831+0000 7f9ad6c99640 1 --2- 172.21.15.164:0/3552902860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644e14175d0 0x5644e142a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.831+0000 7f9ad6c99640 1 -- 172.21.15.164:0/3552902860 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644e1413510 con 0x5644e1422370 2024-09-19T08:56:11.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.831+0000 7f9ad6498640 1 --2- 172.21.15.164:0/3552902860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644e14175d0 0x5644e142a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:56:11.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.831+0000 7f9ad5c97640 1 --2- 172.21.15.164:0/3552902860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5644e1418b40 0x5644e142a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:56:11.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.831+0000 7f9ad6c99640 1 --2- 172.21.15.164:0/3552902860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644e1422370 0x5644e14273f0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8004650 tx=0x7f9ac8012a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a9de7585f42b27 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:11.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.831+0000 7f9ab9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:56:11.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.831+0000 7f9ab9ffb640 1 -- 172.21.15.164:0/3552902860 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac8023070 con 0x5644e1422370 2024-09-19T08:56:11.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.831+0000 7f9ab9ffb640 1 -- 172.21.15.164:0/3552902860 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7f9ac801f8f0 con 0x5644e1422370 2024-09-19T08:56:11.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.831+0000 7f9adaad5740 1 -- 172.21.15.164:0/3552902860 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9acc036040 con 0x5644e1422370 2024-09-19T08:56:11.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.831+0000 7f9ab9ffb640 1 -- 172.21.15.164:0/3552902860 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9ac801faf0 con 0x5644e1422370 2024-09-19T08:56:11.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.831+0000 7f9adaad5740 10 client.17688 fetch_fsmap learned FSMap version 32 2024-09-19T08:56:11.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.831+0000 7f9adaad5740 1 -- 172.21.15.164:0/3552902860 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5644e149cee0 con 0x5644e1422370 2024-09-19T08:56:11.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.832+0000 7f9ab9ffb640 1 -- 172.21.15.164:0/3552902860 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9acc036040 con 0x5644e1422370 2024-09-19T08:56:11.858 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:56:11.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.832+0000 7f9ab9ffb640 1 -- 172.21.15.164:0/3552902860 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f9ac80273b0 con 0x5644e1422370 2024-09-19T08:56:11.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.832+0000 7f9adaad5740 10 client.17688 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:56:11.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.832+0000 7f9adaad5740 10 client.17688 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T08:56:11.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.832+0000 7f9adaad5740 20 client.17688 populate_metadata read hostname 'smithi164' 2024-09-19T08:56:11.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.832+0000 7f9adaad5740 1 --2- 172.21.15.164:0/3552902860 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x5644e149d690 0x5644e149fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:11.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.832+0000 7f9adaad5740 4 client.17688 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:56:11.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.832+0000 7f9adaad5740 1 -- 172.21.15.164:0/3552902860 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5644e1384d70 con 0x5644e149d690 2024-09-19T08:56:11.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.832+0000 7f9ad6498640 1 --2- 172.21.15.164:0/3552902860 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x5644e149d690 0x5644e149fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:11.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.832+0000 7f9ad6498640 1 --2- 172.21.15.164:0/3552902860 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x5644e149d690 0x5644e149fab0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=15bbeabbf170664a server_cookie=a1b8d645d51633aa in_seq=0 out_seq=0 2024-09-19T08:56:11.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.832+0000 7f9ab9ffb640 10 client.17688 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:56:11.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.833+0000 7f9ab9ffb640 1 -- 172.21.15.164:0/3552902860 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5644e1384d70 con 0x5644e149d690 2024-09-19T08:56:11.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.833+0000 7f9ab9ffb640 10 client.17688 handle_command_reply: tid=0 2024-09-19T08:56:11.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.833+0000 7f9adaad5740 1 client.17688 shutdown 2024-09-19T08:56:11.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.833+0000 7f9adaad5740 1 -- 172.21.15.164:0/3552902860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644e1422370 msgr2=0x5644e14273f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:11.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.833+0000 7f9adaad5740 1 --2- 172.21.15.164:0/3552902860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644e1422370 0x5644e14273f0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8004650 tx=0x7f9ac8012a80 comp rx=0 tx=0).stop 2024-09-19T08:56:11.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.833+0000 7f9adaad5740 1 -- 172.21.15.164:0/3552902860 shutdown_connections 2024-09-19T08:56:11.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.833+0000 7f9adaad5740 1 --2- 172.21.15.164:0/3552902860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644e1422370 0x5644e14273f0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.833+0000 7f9adaad5740 1 --2- 172.21.15.164:0/3552902860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5644e1418b40 0x5644e142a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.833+0000 7f9adaad5740 1 --2- 172.21.15.164:0/3552902860 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x5644e149d690 0x5644e149fab0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.833+0000 7f9adaad5740 1 --2- 172.21.15.164:0/3552902860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644e14175d0 0x5644e142a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.833+0000 7f9ab9ffb640 0 client.17688 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:56:11.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.833+0000 7f9adaad5740 1 -- 172.21.15.164:0/3552902860 >> 172.21.15.164:0/3552902860 conn(0x5644e13e0310 msgr2=0x5644e14124e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:11.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.833+0000 7f9adaad5740 1 -- 172.21.15.164:0/3552902860 shutdown_connections 2024-09-19T08:56:11.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.833+0000 7f9adaad5740 1 -- 172.21.15.164:0/3552902860 wait complete. 2024-09-19T08:56:11.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.833+0000 7f9adaad5740 20 client.17688 trim_cache size 0 max 16384 2024-09-19T08:56:11.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.834+0000 7f9ad86fc640 1 -- 172.21.15.164:0/4047202273 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f9abc0a33b0 msgr2=0x7f9abc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:11.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.834+0000 7f9ad86fc640 1 --2- 172.21.15.164:0/4047202273 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f9abc0a33b0 0x7f9abc0a5870 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8005fd0 tx=0x7f9ac80046c0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.834+0000 7f9ad86fc640 1 -- 172.21.15.164:0/4047202273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ad0101420 msgr2=0x7f9ad0197640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:11.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.834+0000 7f9ad86fc640 1 --2- 172.21.15.164:0/4047202273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ad0101420 0x7f9ad0197640 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000d770 tx=0x7f9ac000dc30 comp rx=0 tx=0).stop 2024-09-19T08:56:11.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.834+0000 7f9ad86fc640 1 -- 172.21.15.164:0/4047202273 shutdown_connections 2024-09-19T08:56:11.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.834+0000 7f9ad86fc640 1 --2- 172.21.15.164:0/4047202273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ad0106a80 0x7f9ad0197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.834+0000 7f9ad86fc640 1 --2- 172.21.15.164:0/4047202273 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f9abc0a33b0 0x7f9abc0a5870 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.834+0000 7f9ad86fc640 1 --2- 172.21.15.164:0/4047202273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ad0101420 0x7f9ad0197640 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.834+0000 7f9ad86fc640 1 --2- 172.21.15.164:0/4047202273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ad0100a80 0x7f9ad0197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:11.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.834+0000 7f9ad86fc640 1 -- 172.21.15.164:0/4047202273 >> 172.21.15.164:0/4047202273 conn(0x7f9ad00fc7a0 msgr2=0x7f9ad00fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:11.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.834+0000 7f9ad86fc640 1 -- 172.21.15.164:0/4047202273 shutdown_connections 2024-09-19T08:56:11.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:11.834+0000 7f9ad86fc640 1 -- 172.21.15.164:0/4047202273 wait complete. 2024-09-19T08:56:12.475 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:12 smithi164 ceph-mon[30928]: pgmap v323: 97 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 81 active+clean; 23 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T08:56:12.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:12 smithi170 ceph-mon[35072]: pgmap v323: 97 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 81 active+clean; 23 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T08:56:12.653 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T08:56:12.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:12 smithi173 ceph-mon[35306]: pgmap v323: 97 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 81 active+clean; 23 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-19T08:56:12.903 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:56:13.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.381+0000 7efe81c1f640 1 -- 172.21.15.164:0/4080919828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe7c103700 msgr2=0x7efe7c10bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:13.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.381+0000 7efe81c1f640 1 --2- 172.21.15.164:0/4080919828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe7c103700 0x7efe7c10bbf0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7efe70009900 tx=0x7efe7002a480 comp rx=0 tx=0).stop 2024-09-19T08:56:13.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.381+0000 7efe81c1f640 1 -- 172.21.15.164:0/4080919828 shutdown_connections 2024-09-19T08:56:13.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.381+0000 7efe81c1f640 1 --2- 172.21.15.164:0/4080919828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe7c10c180 0x7efe7c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:13.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.381+0000 7efe81c1f640 1 --2- 172.21.15.164:0/4080919828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe7c103700 0x7efe7c10bbf0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:13.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.381+0000 7efe81c1f640 1 --2- 172.21.15.164:0/4080919828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe7c102de0 0x7efe7c1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:13.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.381+0000 7efe81c1f640 1 -- 172.21.15.164:0/4080919828 >> 172.21.15.164:0/4080919828 conn(0x7efe7c0fc7a0 msgr2=0x7efe7c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:13.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.381+0000 7efe81c1f640 1 -- 172.21.15.164:0/4080919828 shutdown_connections 2024-09-19T08:56:13.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.381+0000 7efe81c1f640 1 -- 172.21.15.164:0/4080919828 wait complete. 2024-09-19T08:56:13.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.382+0000 7efe81c1f640 1 Processor -- start 2024-09-19T08:56:13.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.382+0000 7efe81c1f640 1 -- start start 2024-09-19T08:56:13.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.382+0000 7efe81c1f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe7c102de0 0x7efe7c1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:13.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.382+0000 7efe81c1f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe7c103700 0x7efe7c198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:13.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.382+0000 7efe81c1f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe7c10c180 0x7efe7c19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:13.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.382+0000 7efe81c1f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efe7c111fb0 con 0x7efe7c10c180 2024-09-19T08:56:13.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.382+0000 7efe81c1f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7efe7c111e30 con 0x7efe7c102de0 2024-09-19T08:56:13.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.382+0000 7efe81c1f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efe7c112130 con 0x7efe7c103700 2024-09-19T08:56:13.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.383+0000 7efe7bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe7c10c180 0x7efe7c19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:13.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.383+0000 7efe7bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe7c10c180 0x7efe7c19cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50294/0 (socket says 172.21.15.164:50294) 2024-09-19T08:56:13.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.383+0000 7efe7bfff640 1 -- 172.21.15.164:0/2526259040 learned_addr learned my addr 172.21.15.164:0/2526259040 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:13.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.383+0000 7efe7affd640 1 --2- 172.21.15.164:0/2526259040 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe7c103700 0x7efe7c198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:13.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.383+0000 7efe7b7fe640 1 --2- 172.21.15.164:0/2526259040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe7c102de0 0x7efe7c1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:13.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.383+0000 7efe7bfff640 1 -- 172.21.15.164:0/2526259040 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe7c103700 msgr2=0x7efe7c198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:13.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.383+0000 7efe7bfff640 1 --2- 172.21.15.164:0/2526259040 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe7c103700 0x7efe7c198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:13.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.383+0000 7efe7bfff640 1 -- 172.21.15.164:0/2526259040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe7c102de0 msgr2=0x7efe7c1984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:13.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.383+0000 7efe7bfff640 1 --2- 172.21.15.164:0/2526259040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe7c102de0 0x7efe7c1984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:13.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.383+0000 7efe7bfff640 1 -- 172.21.15.164:0/2526259040 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe7c19d470 con 0x7efe7c10c180 2024-09-19T08:56:13.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.383+0000 7efe7b7fe640 1 --2- 172.21.15.164:0/2526259040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe7c102de0 0x7efe7c1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:56:13.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.383+0000 7efe7bfff640 1 --2- 172.21.15.164:0/2526259040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe7c10c180 0x7efe7c19cd90 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7efe7400eaa0 tx=0x7efe7400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:13.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.383+0000 7efe78ff9640 1 -- 172.21.15.164:0/2526259040 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7400cdb0 con 0x7efe7c10c180 2024-09-19T08:56:13.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.383+0000 7efe78ff9640 1 -- 172.21.15.164:0/2526259040 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe74005020 con 0x7efe7c10c180 2024-09-19T08:56:13.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.383+0000 7efe78ff9640 1 -- 172.21.15.164:0/2526259040 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe740116b0 con 0x7efe7c10c180 2024-09-19T08:56:13.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.383+0000 7efe81c1f640 1 -- 172.21.15.164:0/2526259040 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe7c19d760 con 0x7efe7c10c180 2024-09-19T08:56:13.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.383+0000 7efe81c1f640 1 -- 172.21.15.164:0/2526259040 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe7c1a5040 con 0x7efe7c10c180 2024-09-19T08:56:13.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.386+0000 7efe78ff9640 1 -- 172.21.15.164:0/2526259040 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7efe74005560 con 0x7efe7c10c180 2024-09-19T08:56:13.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.386+0000 7efe83ff8740 1 Processor -- start 2024-09-19T08:56:13.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.386+0000 7efe83ff8740 1 -- start start 2024-09-19T08:56:13.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.386+0000 7efe83ff8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5593ced88b40 0x5593ced87090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:13.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.386+0000 7efe78ff9640 1 --2- 172.21.15.164:0/2526259040 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7efe640a33b0 0x7efe640a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:13.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.386+0000 7efe83ff8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593ced875d0 0x5593ced87a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:13.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.386+0000 7efe83ff8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593ced96780 0x5593ced98b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:13.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.386+0000 7efe83ff8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5593ceced2f0 con 0x5593ced96780 2024-09-19T08:56:13.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.386+0000 7efe83ff8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5593cec94cf0 con 0x5593ced88b40 2024-09-19T08:56:13.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.386+0000 7efe83ff8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5593cece92b0 con 0x5593ced875d0 2024-09-19T08:56:13.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.387+0000 7efe7b7fe640 1 --2- 172.21.15.164:0/2526259040 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7efe640a33b0 0x7efe640a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:13.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.387+0000 7efe7bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593ced96780 0x5593ced98b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:13.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.387+0000 7efe7bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593ced96780 0x5593ced98b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50300/0 (socket says 172.21.15.164:50300) 2024-09-19T08:56:13.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.387+0000 7efe7bfff640 1 -- 172.21.15.164:0/3736496542 learned_addr learned my addr 172.21.15.164:0/3736496542 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:13.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.387+0000 7efe7affd640 1 --2- 172.21.15.164:0/3736496542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5593ced88b40 0x5593ced87090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:13.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.387+0000 7efe78ff9640 1 -- 172.21.15.164:0/2526259040 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7efe740cff90 con 0x7efe7c10c180 2024-09-19T08:56:13.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.387+0000 7efe7affd640 1 --2- 172.21.15.164:0/3736496542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593ced875d0 0x5593ced87a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:13.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.387+0000 7efe7bfff640 1 -- 172.21.15.164:0/3736496542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593ced875d0 msgr2=0x5593ced87a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:13.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.387+0000 7efe7bfff640 1 --2- 172.21.15.164:0/3736496542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593ced875d0 0x5593ced87a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:13.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.387+0000 7efe7bfff640 1 -- 172.21.15.164:0/3736496542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5593ced88b40 msgr2=0x5593ced87090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:13.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.387+0000 7efe7bfff640 1 --2- 172.21.15.164:0/3736496542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5593ced88b40 0x5593ced87090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:13.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.387+0000 7efe7bfff640 1 -- 172.21.15.164:0/3736496542 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593cec793a0 con 0x5593ced96780 2024-09-19T08:56:13.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.387+0000 7efe7affd640 1 --2- 172.21.15.164:0/3736496542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593ced875d0 0x5593ced87a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:56:13.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.387+0000 7efe7b7fe640 1 --2- 172.21.15.164:0/2526259040 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7efe640a33b0 0x7efe640a5870 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efe6c009930 tx=0x7efe6c008040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:13.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.387+0000 7efe7bfff640 1 --2- 172.21.15.164:0/3736496542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593ced96780 0x5593ced98b70 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7efe74009270 tx=0x7efe740054d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f44d96e652c1fd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:13.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.387+0000 7efe61ffb640 1 -- 172.21.15.164:0/3736496542 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7409c280 con 0x5593ced96780 2024-09-19T08:56:13.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.387+0000 7efe61ffb640 1 -- 172.21.15.164:0/3736496542 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe740a0630 con 0x5593ced96780 2024-09-19T08:56:13.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.387+0000 7efe61ffb640 1 -- 172.21.15.164:0/3736496542 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7409c4e0 con 0x5593ced96780 2024-09-19T08:56:13.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.388+0000 7efe83ff8740 1 -- 172.21.15.164:0/3736496542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593ced96780 msgr2=0x5593ced98b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:13.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.388+0000 7efe83ff8740 1 --2- 172.21.15.164:0/3736496542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593ced96780 0x5593ced98b70 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7efe74009270 tx=0x7efe740054d0 comp rx=0 tx=0).stop 2024-09-19T08:56:13.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.388+0000 7efe83ff8740 1 -- 172.21.15.164:0/3736496542 shutdown_connections 2024-09-19T08:56:13.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.388+0000 7efe83ff8740 1 --2- 172.21.15.164:0/3736496542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593ced96780 0x5593ced98b70 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:13.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.388+0000 7efe83ff8740 1 --2- 172.21.15.164:0/3736496542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593ced875d0 0x5593ced87a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:13.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.388+0000 7efe83ff8740 1 --2- 172.21.15.164:0/3736496542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5593ced88b40 0x5593ced87090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:13.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.388+0000 7efe83ff8740 1 -- 172.21.15.164:0/3736496542 >> 172.21.15.164:0/3736496542 conn(0x5593ced50310 msgr2=0x5593ced83c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:13.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.388+0000 7efe83ff8740 1 -- 172.21.15.164:0/3736496542 shutdown_connections 2024-09-19T08:56:13.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.388+0000 7efe83ff8740 1 -- 172.21.15.164:0/3736496542 wait complete. 2024-09-19T08:56:13.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.388+0000 7efe83ff8740 1 -- start start 2024-09-19T08:56:13.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.389+0000 7efe83ff8740 1 Processor -- start 2024-09-19T08:56:13.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.389+0000 7efe83ff8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5593ced875d0 0x5593ced961d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:13.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.389+0000 7efe83ff8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593ced88b40 0x5593ced93480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:13.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.389+0000 7efe83ff8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593ced96780 0x5593ced93c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:13.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.389+0000 7efe83ff8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5593ced99480 con 0x5593ced88b40 2024-09-19T08:56:13.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.389+0000 7efe83ff8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5593ceca8060 con 0x5593ced875d0 2024-09-19T08:56:13.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.389+0000 7efe83ff8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5593ced99600 con 0x5593ced96780 2024-09-19T08:56:13.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.389+0000 7efe83ff8740 10 client.0 _ll_register_callbacks cb 0x5593ce9fb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:56:13.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.389+0000 7efe7affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593ced88b40 0x5593ced93480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:13.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.389+0000 7efe7affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593ced88b40 0x5593ced93480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50314/0 (socket says 172.21.15.164:50314) 2024-09-19T08:56:13.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.389+0000 7efe7b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5593ced875d0 0x5593ced961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:13.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.389+0000 7efe7affd640 1 -- 172.21.15.164:0/2770156760 learned_addr learned my addr 172.21.15.164:0/2770156760 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:13.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.389+0000 7efe7bfff640 1 --2- 172.21.15.164:0/2770156760 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593ced96780 0x5593ced93c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:13.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.389+0000 7efe7affd640 1 -- 172.21.15.164:0/2770156760 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593ced96780 msgr2=0x5593ced93c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:13.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.389+0000 7efe7affd640 1 --2- 172.21.15.164:0/2770156760 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593ced96780 0x5593ced93c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:13.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.389+0000 7efe7affd640 1 -- 172.21.15.164:0/2770156760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5593ced875d0 msgr2=0x5593ced961d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:13.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.389+0000 7efe7affd640 1 --2- 172.21.15.164:0/2770156760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5593ced875d0 0x5593ced961d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:13.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.389+0000 7efe7affd640 1 -- 172.21.15.164:0/2770156760 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593cea5e3c0 con 0x5593ced88b40 2024-09-19T08:56:13.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.390+0000 7efe7affd640 1 --2- 172.21.15.164:0/2770156760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593ced88b40 0x5593ced93480 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7efe70009900 tx=0x7efe70031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19155f09afcff246 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:13.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.390+0000 7efe5affd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:56:13.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.390+0000 7efe5affd640 1 -- 172.21.15.164:0/2770156760 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe70035040 con 0x5593ced88b40 2024-09-19T08:56:13.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.390+0000 7efe83ff8740 1 -- 172.21.15.164:0/2770156760 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe740a0630 con 0x5593ced88b40 2024-09-19T08:56:13.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.390+0000 7efe5affd640 1 -- 172.21.15.164:0/2770156760 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7efe70004240 con 0x5593ced88b40 2024-09-19T08:56:13.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.390+0000 7efe5affd640 1 -- 172.21.15.164:0/2770156760 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efe70031a30 con 0x5593ced88b40 2024-09-19T08:56:13.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.390+0000 7efe83ff8740 10 client.17700 fetch_fsmap learned FSMap version 32 2024-09-19T08:56:13.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.390+0000 7efe83ff8740 1 -- 172.21.15.164:0/2770156760 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5593ceda4f00 con 0x5593ced88b40 2024-09-19T08:56:13.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.390+0000 7efe5affd640 1 -- 172.21.15.164:0/2770156760 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe740a0630 con 0x5593ced88b40 2024-09-19T08:56:13.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.390+0000 7efe5affd640 1 -- 172.21.15.164:0/2770156760 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7efe700486e0 con 0x5593ced88b40 2024-09-19T08:56:13.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.390+0000 7efe83ff8740 10 client.17700 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:56:13.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.390+0000 7efe83ff8740 10 client.17700 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:56:13.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.390+0000 7efe83ff8740 20 client.17700 populate_metadata read hostname 'smithi164' 2024-09-19T08:56:13.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.390+0000 7efe83ff8740 1 --2- 172.21.15.164:0/2770156760 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5593ceda63c0 0x5593ceda87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:13.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.390+0000 7efe83ff8740 4 client.17700 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:56:13.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.390+0000 7efe83ff8740 1 -- 172.21.15.164:0/2770156760 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593cecfa860 con 0x5593ceda63c0 2024-09-19T08:56:13.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.391+0000 7efe7b7fe640 1 --2- 172.21.15.164:0/2770156760 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5593ceda63c0 0x5593ceda87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:13.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.391+0000 7efe5affd640 10 client.17700 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:56:13.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.391+0000 7efe7b7fe640 1 --2- 172.21.15.164:0/2770156760 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5593ceda63c0 0x5593ceda87d0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb9bc97173212388 server_cookie=aa6cca1e6ee2c452 in_seq=0 out_seq=0 2024-09-19T08:56:13.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.392+0000 7efe5affd640 1 -- 172.21.15.164:0/2770156760 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5593cecfa860 con 0x5593ceda63c0 2024-09-19T08:56:13.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.392+0000 7efe5affd640 10 client.17700 handle_command_reply: tid=0 2024-09-19T08:56:13.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.392+0000 7efe83ff8740 1 client.17700 shutdown 2024-09-19T08:56:13.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.392+0000 7efe83ff8740 1 -- 172.21.15.164:0/2770156760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593ced88b40 msgr2=0x5593ced93480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:13.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.392+0000 7efe83ff8740 1 --2- 172.21.15.164:0/2770156760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593ced88b40 0x5593ced93480 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7efe70009900 tx=0x7efe70031630 comp rx=0 tx=0).stop 2024-09-19T08:56:13.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.392+0000 7efe83ff8740 1 -- 172.21.15.164:0/2770156760 shutdown_connections 2024-09-19T08:56:13.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.392+0000 7efe83ff8740 1 --2- 172.21.15.164:0/2770156760 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593ced96780 0x5593ced93c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:13.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.392+0000 7efe83ff8740 1 --2- 172.21.15.164:0/2770156760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593ced88b40 0x5593ced93480 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:13.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.392+0000 7efe83ff8740 1 --2- 172.21.15.164:0/2770156760 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5593ceda63c0 0x5593ceda87d0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:13.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.392+0000 7efe5affd640 0 client.17700 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:56:13.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.392+0000 7efe83ff8740 1 --2- 172.21.15.164:0/2770156760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5593ced875d0 0x5593ced961d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:13.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.392+0000 7efe83ff8740 1 -- 172.21.15.164:0/2770156760 >> 172.21.15.164:0/2770156760 conn(0x5593ced50310 msgr2=0x5593ced89990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:13.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.392+0000 7efe83ff8740 1 -- 172.21.15.164:0/2770156760 shutdown_connections 2024-09-19T08:56:13.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.392+0000 7efe83ff8740 1 -- 172.21.15.164:0/2770156760 wait complete. 2024-09-19T08:56:13.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.392+0000 7efe83ff8740 20 client.17700 trim_cache size 0 max 16384 2024-09-19T08:56:13.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.405+0000 7efe83ff8740 1 Processor -- start 2024-09-19T08:56:13.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.405+0000 7efe83ff8740 1 -- start start 2024-09-19T08:56:13.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.405+0000 7efe83ff8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593ced875d0 0x5593ced83e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:13.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.405+0000 7efe83ff8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5593ced88b40 0x5593ced98b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:13.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.405+0000 7efe83ff8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593ced96780 0x5593ced92d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:13.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.405+0000 7efe83ff8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efe7c1ab6e0 con 0x5593ced96780 2024-09-19T08:56:13.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.405+0000 7efe83ff8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7efe7c1ad220 con 0x5593ced88b40 2024-09-19T08:56:13.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.405+0000 7efe83ff8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efe7c1abdb0 con 0x5593ced875d0 2024-09-19T08:56:13.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.405+0000 7efe7bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593ced96780 0x5593ced92d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:13.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.405+0000 7efe7bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593ced96780 0x5593ced92d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50320/0 (socket says 172.21.15.164:50320) 2024-09-19T08:56:13.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.405+0000 7efe7bfff640 1 -- 172.21.15.164:0/3209360225 learned_addr learned my addr 172.21.15.164:0/3209360225 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:13.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.405+0000 7efe7affd640 1 --2- 172.21.15.164:0/3209360225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5593ced88b40 0x5593ced98b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:13.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.405+0000 7efe7b7fe640 1 --2- 172.21.15.164:0/3209360225 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593ced875d0 0x5593ced83e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:13.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.406+0000 7efe7bfff640 1 -- 172.21.15.164:0/3209360225 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593ced875d0 msgr2=0x5593ced83e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:13.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.406+0000 7efe7bfff640 1 --2- 172.21.15.164:0/3209360225 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593ced875d0 0x5593ced83e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:13.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.406+0000 7efe7bfff640 1 -- 172.21.15.164:0/3209360225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5593ced88b40 msgr2=0x5593ced98b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:13.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.406+0000 7efe7bfff640 1 --2- 172.21.15.164:0/3209360225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5593ced88b40 0x5593ced98b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:13.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.406+0000 7efe7bfff640 1 -- 172.21.15.164:0/3209360225 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593ced93f80 con 0x5593ced96780 2024-09-19T08:56:13.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.406+0000 7efe7affd640 1 --2- 172.21.15.164:0/3209360225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5593ced88b40 0x5593ced98b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:56:13.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.406+0000 7efe7bfff640 1 --2- 172.21.15.164:0/3209360225 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593ced96780 0x5593ced92d40 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7efe740cd7a0 tx=0x7efe74015ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1573c802f2d1b186 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:13.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.406+0000 7efe617fa640 1 -- 172.21.15.164:0/3209360225 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe740a4ef0 con 0x5593ced96780 2024-09-19T08:56:13.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.406+0000 7efe617fa640 1 -- 172.21.15.164:0/3209360225 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe7409a580 con 0x5593ced96780 2024-09-19T08:56:13.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.406+0000 7efe617fa640 1 -- 172.21.15.164:0/3209360225 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe740bc150 con 0x5593ced96780 2024-09-19T08:56:13.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.406+0000 7efe83ff8740 1 -- 172.21.15.164:0/3209360225 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593ced96780 msgr2=0x5593ced92d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:13.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.406+0000 7efe83ff8740 1 --2- 172.21.15.164:0/3209360225 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593ced96780 0x5593ced92d40 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7efe740cd7a0 tx=0x7efe74015ea0 comp rx=0 tx=0).stop 2024-09-19T08:56:13.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.406+0000 7efe83ff8740 1 -- 172.21.15.164:0/3209360225 shutdown_connections 2024-09-19T08:56:13.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.406+0000 7efe83ff8740 1 --2- 172.21.15.164:0/3209360225 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593ced96780 0x5593ced92d40 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:13.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.407+0000 7efe83ff8740 1 --2- 172.21.15.164:0/3209360225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5593ced88b40 0x5593ced98b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:13.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.407+0000 7efe83ff8740 1 --2- 172.21.15.164:0/3209360225 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593ced875d0 0x5593ced83e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:13.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.407+0000 7efe83ff8740 1 -- 172.21.15.164:0/3209360225 >> 172.21.15.164:0/3209360225 conn(0x5593ced50310 msgr2=0x5593ced83700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:13.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.407+0000 7efe83ff8740 1 -- 172.21.15.164:0/3209360225 shutdown_connections 2024-09-19T08:56:13.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.407+0000 7efe83ff8740 1 -- 172.21.15.164:0/3209360225 wait complete. 2024-09-19T08:56:13.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.407+0000 7efe83ff8740 1 -- start start 2024-09-19T08:56:13.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.407+0000 7efe83ff8740 1 Processor -- start 2024-09-19T08:56:13.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.407+0000 7efe83ff8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5593ced875d0 0x5593ced9f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:13.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.407+0000 7efe83ff8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593ced88b40 0x5593ced90210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:13.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.407+0000 7efe83ff8740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593ced96780 0x5593ced90750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:13.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.407+0000 7efe83ff8740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efe7c1acb50 con 0x5593ced96780 2024-09-19T08:56:13.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.407+0000 7efe83ff8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7efe7c1ac480 con 0x5593ced875d0 2024-09-19T08:56:13.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.407+0000 7efe83ff8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efe7c1a9700 con 0x5593ced88b40 2024-09-19T08:56:13.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.407+0000 7efe83ff8740 10 client.0 _ll_register_callbacks cb 0x7efe70009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:56:13.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.407+0000 7efe7bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593ced96780 0x5593ced90750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:13.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.408+0000 7efe7bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593ced96780 0x5593ced90750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:50322/0 (socket says 172.21.15.164:50322) 2024-09-19T08:56:13.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.408+0000 7efe7bfff640 1 -- 172.21.15.164:0/1731315831 learned_addr learned my addr 172.21.15.164:0/1731315831 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:13.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.408+0000 7efe7b7fe640 1 --2- 172.21.15.164:0/1731315831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5593ced875d0 0x5593ced9f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:13.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.408+0000 7efe7affd640 1 --2- 172.21.15.164:0/1731315831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593ced88b40 0x5593ced90210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:13.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.408+0000 7efe7bfff640 1 -- 172.21.15.164:0/1731315831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593ced88b40 msgr2=0x5593ced90210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:13.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.408+0000 7efe7bfff640 1 --2- 172.21.15.164:0/1731315831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593ced88b40 0x5593ced90210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:13.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.408+0000 7efe7bfff640 1 -- 172.21.15.164:0/1731315831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5593ced875d0 msgr2=0x5593ced9f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:13.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.408+0000 7efe7bfff640 1 --2- 172.21.15.164:0/1731315831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5593ced875d0 0x5593ced9f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:13.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.408+0000 7efe7bfff640 1 -- 172.21.15.164:0/1731315831 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593ced90fa0 con 0x5593ced96780 2024-09-19T08:56:13.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.408+0000 7efe7b7fe640 1 --2- 172.21.15.164:0/1731315831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5593ced875d0 0x5593ced9f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:56:13.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.408+0000 7efe7bfff640 1 --2- 172.21.15.164:0/1731315831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593ced96780 0x5593ced90750 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7efe74098690 tx=0x7efe7409c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18eb78d5b324151 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:13.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.408+0000 7efe5affd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:56:13.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.408+0000 7efe5affd640 1 -- 172.21.15.164:0/1731315831 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7407ae60 con 0x5593ced96780 2024-09-19T08:56:13.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.408+0000 7efe5affd640 1 -- 172.21.15.164:0/1731315831 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7efe7409c4e0 con 0x5593ced96780 2024-09-19T08:56:13.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.408+0000 7efe83ff8740 1 -- 172.21.15.164:0/1731315831 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe7409a580 con 0x5593ced96780 2024-09-19T08:56:13.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.408+0000 7efe5affd640 1 -- 172.21.15.164:0/1731315831 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efe7409c760 con 0x5593ced96780 2024-09-19T08:56:13.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.408+0000 7efe83ff8740 10 client.17706 fetch_fsmap learned FSMap version 32 2024-09-19T08:56:13.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.408+0000 7efe83ff8740 1 -- 172.21.15.164:0/1731315831 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5593cee110e0 con 0x5593ced96780 2024-09-19T08:56:13.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.408+0000 7efe5affd640 1 -- 172.21.15.164:0/1731315831 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe7409a580 con 0x5593ced96780 2024-09-19T08:56:13.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.408+0000 7efe5affd640 1 -- 172.21.15.164:0/1731315831 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7efe7400f040 con 0x5593ced96780 2024-09-19T08:56:13.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.409+0000 7efe83ff8740 10 client.17706 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:56:13.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.409+0000 7efe83ff8740 10 client.17706 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:56:13.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.409+0000 7efe83ff8740 20 client.17706 populate_metadata read hostname 'smithi164' 2024-09-19T08:56:13.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.409+0000 7efe83ff8740 1 --2- 172.21.15.164:0/1731315831 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5593ceda63c0 0x5593cee132c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:13.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.409+0000 7efe83ff8740 4 client.17706 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:56:13.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.409+0000 7efe83ff8740 1 -- 172.21.15.164:0/1731315831 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5593cecf4d70 con 0x5593ceda63c0 2024-09-19T08:56:13.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.409+0000 7efe7b7fe640 1 --2- 172.21.15.164:0/1731315831 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5593ceda63c0 0x5593cee132c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:13.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.409+0000 7efe5affd640 10 client.17706 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:56:13.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.409+0000 7efe7b7fe640 1 --2- 172.21.15.164:0/1731315831 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5593ceda63c0 0x5593cee132c0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94989d730cb1c035 server_cookie=7d762a81bbea9884 in_seq=0 out_seq=0 2024-09-19T08:56:13.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.409+0000 7efe5affd640 1 -- 172.21.15.164:0/1731315831 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5593cecf4d70 con 0x5593ceda63c0 2024-09-19T08:56:13.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.409+0000 7efe5affd640 10 client.17706 handle_command_reply: tid=0 2024-09-19T08:56:13.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.409+0000 7efe83ff8740 1 client.17706 shutdown 2024-09-19T08:56:13.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.410+0000 7efe83ff8740 1 -- 172.21.15.164:0/1731315831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593ced96780 msgr2=0x5593ced90750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:13.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.410+0000 7efe83ff8740 1 --2- 172.21.15.164:0/1731315831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593ced96780 0x5593ced90750 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7efe74098690 tx=0x7efe7409c300 comp rx=0 tx=0).stop 2024-09-19T08:56:13.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.410+0000 7efe83ff8740 1 -- 172.21.15.164:0/1731315831 shutdown_connections 2024-09-19T08:56:13.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.410+0000 7efe83ff8740 1 --2- 172.21.15.164:0/1731315831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5593ced96780 0x5593ced90750 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:13.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.410+0000 7efe83ff8740 1 --2- 172.21.15.164:0/1731315831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593ced88b40 0x5593ced90210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:13.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.410+0000 7efe83ff8740 1 --2- 172.21.15.164:0/1731315831 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5593ceda63c0 0x5593cee132c0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:13.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.410+0000 7efe83ff8740 1 --2- 172.21.15.164:0/1731315831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5593ced875d0 0x5593ced9f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:13.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.410+0000 7efe83ff8740 1 -- 172.21.15.164:0/1731315831 >> 172.21.15.164:0/1731315831 conn(0x5593ced50310 msgr2=0x5593ced95500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:13.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.410+0000 7efe5affd640 0 client.17706 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:56:13.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.410+0000 7efe83ff8740 1 -- 172.21.15.164:0/1731315831 shutdown_connections 2024-09-19T08:56:13.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.410+0000 7efe83ff8740 1 -- 172.21.15.164:0/1731315831 wait complete. 2024-09-19T08:56:13.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.410+0000 7efe83ff8740 20 client.17706 trim_cache size 0 max 16384 2024-09-19T08:56:13.416 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:56:13.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.410+0000 7efe81c1f640 1 -- 172.21.15.164:0/2526259040 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7efe640a33b0 msgr2=0x7efe640a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:13.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.410+0000 7efe81c1f640 1 --2- 172.21.15.164:0/2526259040 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7efe640a33b0 0x7efe640a5870 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efe6c009930 tx=0x7efe6c008040 comp rx=0 tx=0).stop 2024-09-19T08:56:13.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.410+0000 7efe81c1f640 1 -- 172.21.15.164:0/2526259040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe7c10c180 msgr2=0x7efe7c19cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:13.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.410+0000 7efe81c1f640 1 --2- 172.21.15.164:0/2526259040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe7c10c180 0x7efe7c19cd90 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7efe7400eaa0 tx=0x7efe7400ef60 comp rx=0 tx=0).stop 2024-09-19T08:56:13.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.410+0000 7efe81c1f640 1 -- 172.21.15.164:0/2526259040 shutdown_connections 2024-09-19T08:56:13.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.410+0000 7efe81c1f640 1 --2- 172.21.15.164:0/2526259040 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7efe640a33b0 0x7efe640a5870 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:13.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.410+0000 7efe81c1f640 1 --2- 172.21.15.164:0/2526259040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe7c10c180 0x7efe7c19cd90 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:13.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.410+0000 7efe81c1f640 1 --2- 172.21.15.164:0/2526259040 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe7c103700 0x7efe7c198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:13.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.410+0000 7efe81c1f640 1 --2- 172.21.15.164:0/2526259040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe7c102de0 0x7efe7c1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:13.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.410+0000 7efe81c1f640 1 -- 172.21.15.164:0/2526259040 >> 172.21.15.164:0/2526259040 conn(0x7efe7c0fc7a0 msgr2=0x7efe7c0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:13.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.411+0000 7efe81c1f640 1 -- 172.21.15.164:0/2526259040 shutdown_connections 2024-09-19T08:56:13.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:13.411+0000 7efe81c1f640 1 -- 172.21.15.164:0/2526259040 wait complete. 2024-09-19T08:56:14.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:14 smithi164 ceph-mon[30928]: pgmap v324: 97 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 81 active+clean; 22 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-09-19T08:56:14.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:14 smithi170 ceph-mon[35072]: pgmap v324: 97 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 81 active+clean; 22 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-09-19T08:56:14.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:14 smithi173 ceph-mon[35306]: pgmap v324: 97 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 81 active+clean; 22 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-09-19T08:56:15.178 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T08:56:15.178 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T08:56:15.442 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:56:16.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.030+0000 7fe8f4b4e640 1 -- 172.21.15.164:0/3887184774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8f0100a80 msgr2=0x7fe8f0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:16.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.030+0000 7fe8f4b4e640 1 --2- 172.21.15.164:0/3887184774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8f0100a80 0x7fe8f0100ee0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4009930 tx=0x7fe8e402a420 comp rx=0 tx=0).stop 2024-09-19T08:56:16.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.030+0000 7fe8f4b4e640 1 -- 172.21.15.164:0/3887184774 shutdown_connections 2024-09-19T08:56:16.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.030+0000 7fe8f4b4e640 1 --2- 172.21.15.164:0/3887184774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8f0101420 0x7fe8f010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:16.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.030+0000 7fe8f4b4e640 1 --2- 172.21.15.164:0/3887184774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8f0100a80 0x7fe8f0100ee0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:16.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.030+0000 7fe8f4b4e640 1 --2- 172.21.15.164:0/3887184774 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8f0106a80 0x7fe8f0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:16.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.030+0000 7fe8f4b4e640 1 -- 172.21.15.164:0/3887184774 >> 172.21.15.164:0/3887184774 conn(0x7fe8f00fc7a0 msgr2=0x7fe8f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:16.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.030+0000 7fe8f4b4e640 1 -- 172.21.15.164:0/3887184774 shutdown_connections 2024-09-19T08:56:16.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.030+0000 7fe8f4b4e640 1 -- 172.21.15.164:0/3887184774 wait complete. 2024-09-19T08:56:16.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.031+0000 7fe8f4b4e640 1 Processor -- start 2024-09-19T08:56:16.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.031+0000 7fe8f4b4e640 1 -- start start 2024-09-19T08:56:16.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.032+0000 7fe8f4b4e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8f0100a80 0x7fe8f01983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:16.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.032+0000 7fe8f4b4e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8f0101420 0x7fe8f01988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:16.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.032+0000 7fe8f4b4e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8f0106a80 0x7fe8f019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:16.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.032+0000 7fe8f4b4e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f0111f60 con 0x7fe8f0100a80 2024-09-19T08:56:16.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.032+0000 7fe8f4b4e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f0111de0 con 0x7fe8f0106a80 2024-09-19T08:56:16.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.032+0000 7fe8f4b4e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f01120e0 con 0x7fe8f0101420 2024-09-19T08:56:16.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.032+0000 7fe8ee59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8f0100a80 0x7fe8f01983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:16.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.032+0000 7fe8ee59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8f0100a80 0x7fe8f01983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38636/0 (socket says 172.21.15.164:38636) 2024-09-19T08:56:16.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.032+0000 7fe8ee59c640 1 -- 172.21.15.164:0/480088036 learned_addr learned my addr 172.21.15.164:0/480088036 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:16.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.032+0000 7fe8eed9d640 1 --2- 172.21.15.164:0/480088036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8f0106a80 0x7fe8f019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:16.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.032+0000 7fe8edd9b640 1 --2- 172.21.15.164:0/480088036 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8f0101420 0x7fe8f01988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:16.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.032+0000 7fe8ee59c640 1 -- 172.21.15.164:0/480088036 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8f0101420 msgr2=0x7fe8f01988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:16.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.032+0000 7fe8ee59c640 1 --2- 172.21.15.164:0/480088036 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8f0101420 0x7fe8f01988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:16.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.032+0000 7fe8ee59c640 1 -- 172.21.15.164:0/480088036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8f0106a80 msgr2=0x7fe8f019cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:16.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.032+0000 7fe8ee59c640 1 --2- 172.21.15.164:0/480088036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8f0106a80 0x7fe8f019cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:16.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.032+0000 7fe8ee59c640 1 -- 172.21.15.164:0/480088036 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f019d350 con 0x7fe8f0100a80 2024-09-19T08:56:16.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.032+0000 7fe8ee59c640 1 --2- 172.21.15.164:0/480088036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8f0100a80 0x7fe8f01983a0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000c920 tx=0x7fe8e000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:16.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.032+0000 7fe8df7fe640 1 -- 172.21.15.164:0/480088036 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0007c70 con 0x7fe8f0100a80 2024-09-19T08:56:16.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.032+0000 7fe8f4b4e640 1 -- 172.21.15.164:0/480088036 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f019d640 con 0x7fe8f0100a80 2024-09-19T08:56:16.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.032+0000 7fe8df7fe640 1 -- 172.21.15.164:0/480088036 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8e0007e10 con 0x7fe8f0100a80 2024-09-19T08:56:16.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.032+0000 7fe8df7fe640 1 -- 172.21.15.164:0/480088036 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0018890 con 0x7fe8f0100a80 2024-09-19T08:56:16.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.032+0000 7fe8f4b4e640 1 -- 172.21.15.164:0/480088036 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f01a4f20 con 0x7fe8f0100a80 2024-09-19T08:56:16.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.035+0000 7fe8df7fe640 1 -- 172.21.15.164:0/480088036 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fe8e0018a50 con 0x7fe8f0100a80 2024-09-19T08:56:16.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.035+0000 7fe8f6f27740 1 Processor -- start 2024-09-19T08:56:16.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.035+0000 7fe8f6f27740 1 -- start start 2024-09-19T08:56:16.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.035+0000 7fe8f6f27740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b7116f0b60 0x55b7116f0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:16.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.035+0000 7fe8f6f27740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7116f1480 0x55b7116fb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:16.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.035+0000 7fe8f6f27740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7116fbd40 0x55b7116fe130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:16.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.035+0000 7fe8f6f27740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b711656280 con 0x55b7116f0b60 2024-09-19T08:56:16.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.035+0000 7fe8f6f27740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b7115fdb10 con 0x55b7116fbd40 2024-09-19T08:56:16.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.035+0000 7fe8f6f27740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b711652240 con 0x55b7116f1480 2024-09-19T08:56:16.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.035+0000 7fe8eed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b7116f0b60 0x55b7116f0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:16.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.035+0000 7fe8eed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b7116f0b60 0x55b7116f0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38648/0 (socket says 172.21.15.164:38648) 2024-09-19T08:56:16.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.035+0000 7fe8eed9d640 1 -- 172.21.15.164:0/1839393470 learned_addr learned my addr 172.21.15.164:0/1839393470 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:16.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.036+0000 7fe8ee59c640 1 --2- 172.21.15.164:0/1839393470 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7116f1480 0x55b7116fb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:16.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.036+0000 7fe8edd9b640 1 --2- 172.21.15.164:0/1839393470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7116fbd40 0x55b7116fe130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:16.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.036+0000 7fe8eed9d640 1 -- 172.21.15.164:0/1839393470 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7116f1480 msgr2=0x55b7116fb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:16.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.036+0000 7fe8eed9d640 1 --2- 172.21.15.164:0/1839393470 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7116f1480 0x55b7116fb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:16.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.036+0000 7fe8eed9d640 1 -- 172.21.15.164:0/1839393470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7116fbd40 msgr2=0x55b7116fe130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:16.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.036+0000 7fe8eed9d640 1 --2- 172.21.15.164:0/1839393470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7116fbd40 0x55b7116fe130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:16.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.036+0000 7fe8eed9d640 1 -- 172.21.15.164:0/1839393470 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b71160a1f0 con 0x55b7116f0b60 2024-09-19T08:56:16.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.036+0000 7fe8df7fe640 1 --2- 172.21.15.164:0/480088036 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe8d80a33b0 0x7fe8d80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:16.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.036+0000 7fe8ee59c640 1 --2- 172.21.15.164:0/480088036 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe8d80a33b0 0x7fe8d80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:16.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.036+0000 7fe8eed9d640 1 --2- 172.21.15.164:0/1839393470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b7116f0b60 0x55b7116f0f40 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8010370 tx=0x7fe8e8010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc6185ee5be13ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:16.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.036+0000 7fe8df7fe640 1 -- 172.21.15.164:0/480088036 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7fe8e00cfdd0 con 0x7fe8f0100a80 2024-09-19T08:56:16.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.036+0000 7fe8dcff9640 1 -- 172.21.15.164:0/1839393470 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8016070 con 0x55b7116f0b60 2024-09-19T08:56:16.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.036+0000 7fe8dcff9640 1 -- 172.21.15.164:0/1839393470 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8e8004d20 con 0x55b7116f0b60 2024-09-19T08:56:16.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.036+0000 7fe8dcff9640 1 -- 172.21.15.164:0/1839393470 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e80053e0 con 0x55b7116f0b60 2024-09-19T08:56:16.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.037+0000 7fe8ee59c640 1 --2- 172.21.15.164:0/480088036 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe8d80a33b0 0x7fe8d80a5870 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe8e00cd5e0 tx=0x7fe8e004b040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:16.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.037+0000 7fe8f6f27740 1 -- 172.21.15.164:0/1839393470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b7116f0b60 msgr2=0x55b7116f0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:16.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.037+0000 7fe8f6f27740 1 --2- 172.21.15.164:0/1839393470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b7116f0b60 0x55b7116f0f40 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8010370 tx=0x7fe8e8010830 comp rx=0 tx=0).stop 2024-09-19T08:56:16.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.037+0000 7fe8f6f27740 1 -- 172.21.15.164:0/1839393470 shutdown_connections 2024-09-19T08:56:16.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.037+0000 7fe8f6f27740 1 --2- 172.21.15.164:0/1839393470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7116fbd40 0x55b7116fe130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:16.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.037+0000 7fe8f6f27740 1 --2- 172.21.15.164:0/1839393470 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7116f1480 0x55b7116fb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:16.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.037+0000 7fe8f6f27740 1 --2- 172.21.15.164:0/1839393470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b7116f0b60 0x55b7116f0f40 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:16.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.037+0000 7fe8f6f27740 1 -- 172.21.15.164:0/1839393470 >> 172.21.15.164:0/1839393470 conn(0x55b7116ed0f0 msgr2=0x55b7116ed4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:16.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.037+0000 7fe8f6f27740 1 -- 172.21.15.164:0/1839393470 shutdown_connections 2024-09-19T08:56:16.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.037+0000 7fe8f6f27740 1 -- 172.21.15.164:0/1839393470 wait complete. 2024-09-19T08:56:16.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.037+0000 7fe8f6f27740 1 -- start start 2024-09-19T08:56:16.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.038+0000 7fe8f6f27740 1 Processor -- start 2024-09-19T08:56:16.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.038+0000 7fe8f6f27740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7116f0b60 0x55b7116f8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:16.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.038+0000 7fe8f6f27740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b7116f1480 0x55b7116f9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:16.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.038+0000 7fe8f6f27740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7116fbd40 0x55b7116f9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:16.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.038+0000 7fe8f6f27740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b7116fe910 con 0x55b7116f1480 2024-09-19T08:56:16.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.038+0000 7fe8f6f27740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b711610f10 con 0x55b7116f0b60 2024-09-19T08:56:16.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.038+0000 7fe8f6f27740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b7116fea90 con 0x55b7116fbd40 2024-09-19T08:56:16.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.038+0000 7fe8f6f27740 10 client.0 _ll_register_callbacks cb 0x55b7115d9a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:56:16.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.038+0000 7fe8edd9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b7116f1480 0x55b7116f9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:16.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.038+0000 7fe8edd9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b7116f1480 0x55b7116f9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38650/0 (socket says 172.21.15.164:38650) 2024-09-19T08:56:16.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.038+0000 7fe8edd9b640 1 -- 172.21.15.164:0/2595685717 learned_addr learned my addr 172.21.15.164:0/2595685717 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:16.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.038+0000 7fe8eed9d640 1 --2- 172.21.15.164:0/2595685717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7116f0b60 0x55b7116f8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:16.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.038+0000 7fe8eed9d640 1 --2- 172.21.15.164:0/2595685717 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7116fbd40 0x55b7116f9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:16.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.038+0000 7fe8edd9b640 1 -- 172.21.15.164:0/2595685717 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7116fbd40 msgr2=0x55b7116f9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:16.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.038+0000 7fe8edd9b640 1 --2- 172.21.15.164:0/2595685717 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7116fbd40 0x55b7116f9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:16.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.038+0000 7fe8edd9b640 1 -- 172.21.15.164:0/2595685717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7116f0b60 msgr2=0x55b7116f8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:16.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.038+0000 7fe8edd9b640 1 --2- 172.21.15.164:0/2595685717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7116f0b60 0x55b7116f8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:16.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.038+0000 7fe8edd9b640 1 -- 172.21.15.164:0/2595685717 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7113c76a0 con 0x55b7116f1480 2024-09-19T08:56:16.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.038+0000 7fe8eed9d640 1 --2- 172.21.15.164:0/2595685717 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7116fbd40 0x55b7116f9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:56:16.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.038+0000 7fe8edd9b640 1 --2- 172.21.15.164:0/2595685717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b7116f1480 0x55b7116f9100 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fe8e402a940 tx=0x7fe8e4031860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d7f8a256a90965 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:16.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.038+0000 7fe8d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:56:16.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.039+0000 7fe8d5ffb640 1 -- 172.21.15.164:0/2595685717 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4035070 con 0x55b7116f1480 2024-09-19T08:56:16.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.039+0000 7fe8f6f27740 1 -- 172.21.15.164:0/2595685717 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8e8004d20 con 0x55b7116f1480 2024-09-19T08:56:16.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.039+0000 7fe8d5ffb640 1 -- 172.21.15.164:0/2595685717 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7fe8e4004240 con 0x55b7116f1480 2024-09-19T08:56:16.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.039+0000 7fe8d5ffb640 1 -- 172.21.15.164:0/2595685717 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe8e4004440 con 0x55b7116f1480 2024-09-19T08:56:16.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.039+0000 7fe8f6f27740 10 client.17718 fetch_fsmap learned FSMap version 32 2024-09-19T08:56:16.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.039+0000 7fe8f6f27740 1 -- 172.21.15.164:0/2595685717 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b71170a510 con 0x55b7116f1480 2024-09-19T08:56:16.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.039+0000 7fe8d5ffb640 1 -- 172.21.15.164:0/2595685717 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8e8004d20 con 0x55b7116f1480 2024-09-19T08:56:16.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.039+0000 7fe8d5ffb640 1 -- 172.21.15.164:0/2595685717 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe8e4004b80 con 0x55b7116f1480 2024-09-19T08:56:16.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.039+0000 7fe8f6f27740 10 client.17718 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:56:16.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.039+0000 7fe8f6f27740 10 client.17718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:56:16.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.039+0000 7fe8f6f27740 20 client.17718 populate_metadata read hostname 'smithi164' 2024-09-19T08:56:16.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.039+0000 7fe8f6f27740 1 --2- 172.21.15.164:0/2595685717 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55b71170b920 0x55b71170dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:16.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.039+0000 7fe8f6f27740 4 client.17718 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:56:16.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.039+0000 7fe8f6f27740 1 -- 172.21.15.164:0/2595685717 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7116637f0 con 0x55b71170b920 2024-09-19T08:56:16.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.040+0000 7fe8eed9d640 1 --2- 172.21.15.164:0/2595685717 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55b71170b920 0x55b71170dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:16.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.040+0000 7fe8eed9d640 1 --2- 172.21.15.164:0/2595685717 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55b71170b920 0x55b71170dd70 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b76a7354fdcc9ee6 server_cookie=313f388f96387dcb in_seq=0 out_seq=0 2024-09-19T08:56:16.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.040+0000 7fe8d5ffb640 10 client.17718 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:56:16.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.041+0000 7fe8d5ffb640 1 -- 172.21.15.164:0/2595685717 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b7116637f0 con 0x55b71170b920 2024-09-19T08:56:16.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.041+0000 7fe8d5ffb640 10 client.17718 handle_command_reply: tid=0 2024-09-19T08:56:16.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.041+0000 7fe8f6f27740 1 client.17718 shutdown 2024-09-19T08:56:16.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.041+0000 7fe8f6f27740 1 -- 172.21.15.164:0/2595685717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b7116f1480 msgr2=0x55b7116f9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:16.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.041+0000 7fe8f6f27740 1 --2- 172.21.15.164:0/2595685717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b7116f1480 0x55b7116f9100 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fe8e402a940 tx=0x7fe8e4031860 comp rx=0 tx=0).stop 2024-09-19T08:56:16.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.041+0000 7fe8f6f27740 1 -- 172.21.15.164:0/2595685717 shutdown_connections 2024-09-19T08:56:16.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.041+0000 7fe8f6f27740 1 --2- 172.21.15.164:0/2595685717 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7116fbd40 0x55b7116f9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:16.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.041+0000 7fe8f6f27740 1 --2- 172.21.15.164:0/2595685717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b7116f1480 0x55b7116f9100 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:16.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.041+0000 7fe8f6f27740 1 --2- 172.21.15.164:0/2595685717 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55b71170b920 0x55b71170dd70 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:16.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.041+0000 7fe8f6f27740 1 --2- 172.21.15.164:0/2595685717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7116f0b60 0x55b7116f8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:16.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.041+0000 7fe8d5ffb640 0 client.17718 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:56:16.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.041+0000 7fe8f6f27740 1 -- 172.21.15.164:0/2595685717 >> 172.21.15.164:0/2595685717 conn(0x55b7116ed0f0 msgr2=0x55b7116ec270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:16.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.041+0000 7fe8f6f27740 1 -- 172.21.15.164:0/2595685717 shutdown_connections 2024-09-19T08:56:16.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.041+0000 7fe8f6f27740 1 -- 172.21.15.164:0/2595685717 wait complete. 2024-09-19T08:56:16.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.041+0000 7fe8f6f27740 20 client.17718 trim_cache size 0 max 16384 2024-09-19T08:56:16.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.058+0000 7fe8f6f27740 1 Processor -- start 2024-09-19T08:56:16.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.058+0000 7fe8f6f27740 1 -- start start 2024-09-19T08:56:16.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.058+0000 7fe8f6f27740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7116f0b60 0x55b7116faa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:16.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.058+0000 7fe8f6f27740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7116f1480 0x55b711713640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:16.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.058+0000 7fe8f6f27740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b7116fbd40 0x55b711713b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:16.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.058+0000 7fe8f6f27740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f01ad7c0 con 0x55b7116fbd40 2024-09-19T08:56:16.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.058+0000 7fe8f6f27740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f01af300 con 0x55b7116f1480 2024-09-19T08:56:16.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.058+0000 7fe8f6f27740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f01ade90 con 0x55b7116f0b60 2024-09-19T08:56:16.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.058+0000 7fe8eed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b7116fbd40 0x55b711713b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:16.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.058+0000 7fe8eed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b7116fbd40 0x55b711713b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38660/0 (socket says 172.21.15.164:38660) 2024-09-19T08:56:16.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.058+0000 7fe8eed9d640 1 -- 172.21.15.164:0/2671501595 learned_addr learned my addr 172.21.15.164:0/2671501595 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:16.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.058+0000 7fe8eed9d640 1 --2- 172.21.15.164:0/2671501595 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7116f0b60 0x55b7116faa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:16.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.058+0000 7fe8edd9b640 1 --2- 172.21.15.164:0/2671501595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7116f1480 0x55b711713640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:16.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.058+0000 7fe8eed9d640 1 -- 172.21.15.164:0/2671501595 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7116f0b60 msgr2=0x55b7116faa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:16.347 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.058+0000 7fe8eed9d640 1 --2- 172.21.15.164:0/2671501595 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7116f0b60 0x55b7116faa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:16.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.058+0000 7fe8eed9d640 1 -- 172.21.15.164:0/2671501595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7116f1480 msgr2=0x55b711713640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:16.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.058+0000 7fe8eed9d640 1 --2- 172.21.15.164:0/2671501595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7116f1480 0x55b711713640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:16.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.058+0000 7fe8eed9d640 1 -- 172.21.15.164:0/2671501595 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7116e8830 con 0x55b7116fbd40 2024-09-19T08:56:16.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.058+0000 7fe8eed9d640 1 --2- 172.21.15.164:0/2671501595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b7116fbd40 0x55b711713b80 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8000f80 tx=0x7fe8e8012f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a3033eb3c93db2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:16.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.058+0000 7fe8d7fff640 1 -- 172.21.15.164:0/2671501595 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8016040 con 0x55b7116fbd40 2024-09-19T08:56:16.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.058+0000 7fe8d7fff640 1 -- 172.21.15.164:0/2671501595 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8e8003b70 con 0x55b7116fbd40 2024-09-19T08:56:16.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.059+0000 7fe8d7fff640 1 -- 172.21.15.164:0/2671501595 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e80043d0 con 0x55b7116fbd40 2024-09-19T08:56:16.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.059+0000 7fe8f6f27740 1 -- 172.21.15.164:0/2671501595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b7116fbd40 msgr2=0x55b711713b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:16.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.059+0000 7fe8f6f27740 1 --2- 172.21.15.164:0/2671501595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b7116fbd40 0x55b711713b80 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8000f80 tx=0x7fe8e8012f60 comp rx=0 tx=0).stop 2024-09-19T08:56:16.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.059+0000 7fe8f6f27740 1 -- 172.21.15.164:0/2671501595 shutdown_connections 2024-09-19T08:56:16.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.059+0000 7fe8f6f27740 1 --2- 172.21.15.164:0/2671501595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b7116fbd40 0x55b711713b80 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:16.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.059+0000 7fe8f6f27740 1 --2- 172.21.15.164:0/2671501595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7116f1480 0x55b711713640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:16.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.059+0000 7fe8f6f27740 1 --2- 172.21.15.164:0/2671501595 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7116f0b60 0x55b7116faa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:16.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.059+0000 7fe8f6f27740 1 -- 172.21.15.164:0/2671501595 >> 172.21.15.164:0/2671501595 conn(0x55b7116ed0f0 msgr2=0x55b7116fa310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:16.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.059+0000 7fe8f6f27740 1 -- 172.21.15.164:0/2671501595 shutdown_connections 2024-09-19T08:56:16.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.059+0000 7fe8f6f27740 1 -- 172.21.15.164:0/2671501595 wait complete. 2024-09-19T08:56:16.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.059+0000 7fe8f6f27740 1 -- start start 2024-09-19T08:56:16.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.060+0000 7fe8f6f27740 1 Processor -- start 2024-09-19T08:56:16.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.060+0000 7fe8f6f27740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7116f0b60 0x55b7117042f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:16.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.060+0000 7fe8f6f27740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7116f1480 0x55b7116f5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:16.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.060+0000 7fe8f6f27740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b7116fbd40 0x55b7116f5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:16.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.060+0000 7fe8f6f27740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f01aec30 con 0x55b7116fbd40 2024-09-19T08:56:16.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.060+0000 7fe8f6f27740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f01ae560 con 0x55b7116f0b60 2024-09-19T08:56:16.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.060+0000 7fe8f6f27740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f01ab7c0 con 0x55b7116f1480 2024-09-19T08:56:16.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.060+0000 7fe8f6f27740 10 client.0 _ll_register_callbacks cb 0x7fe8e4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:56:16.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.060+0000 7fe8eed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b7116fbd40 0x55b7116f5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:16.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.060+0000 7fe8eed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b7116fbd40 0x55b7116f5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38668/0 (socket says 172.21.15.164:38668) 2024-09-19T08:56:16.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.060+0000 7fe8eed9d640 1 -- 172.21.15.164:0/1934642762 learned_addr learned my addr 172.21.15.164:0/1934642762 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:16.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.060+0000 7fe8edd9b640 1 --2- 172.21.15.164:0/1934642762 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7116f1480 0x55b7116f5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:16.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.060+0000 7fe8eed9d640 1 --2- 172.21.15.164:0/1934642762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7116f0b60 0x55b7117042f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:16.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.060+0000 7fe8eed9d640 1 -- 172.21.15.164:0/1934642762 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7116f1480 msgr2=0x55b7116f5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:16.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.060+0000 7fe8eed9d640 1 --2- 172.21.15.164:0/1934642762 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7116f1480 0x55b7116f5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:16.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.060+0000 7fe8eed9d640 1 -- 172.21.15.164:0/1934642762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7116f0b60 msgr2=0x55b7117042f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:16.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.060+0000 7fe8eed9d640 1 --2- 172.21.15.164:0/1934642762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7116f0b60 0x55b7117042f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:16.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.060+0000 7fe8eed9d640 1 -- 172.21.15.164:0/1934642762 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7116f6480 con 0x55b7116fbd40 2024-09-19T08:56:16.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.060+0000 7fe8eed9d640 1 --2- 172.21.15.164:0/1934642762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b7116fbd40 0x55b7116f5d70 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009ac0 tx=0x7fe8e8012dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66819e36756f50a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:16.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.060+0000 7fe8d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:56:16.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.061+0000 7fe8d5ffb640 1 -- 172.21.15.164:0/1934642762 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8016070 con 0x55b7116fbd40 2024-09-19T08:56:16.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.061+0000 7fe8d5ffb640 1 -- 172.21.15.164:0/1934642762 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7fe8e80129e0 con 0x55b7116fbd40 2024-09-19T08:56:16.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.061+0000 7fe8f6f27740 1 -- 172.21.15.164:0/1934642762 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8e8003b70 con 0x55b7116fbd40 2024-09-19T08:56:16.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.061+0000 7fe8d5ffb640 1 -- 172.21.15.164:0/1934642762 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe8e803a040 con 0x55b7116fbd40 2024-09-19T08:56:16.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.061+0000 7fe8f6f27740 10 client.17724 fetch_fsmap learned FSMap version 32 2024-09-19T08:56:16.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.061+0000 7fe8f6f27740 1 -- 172.21.15.164:0/1934642762 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b711776590 con 0x55b7116fbd40 2024-09-19T08:56:16.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.061+0000 7fe8d5ffb640 1 -- 172.21.15.164:0/1934642762 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8e8003b70 con 0x55b7116fbd40 2024-09-19T08:56:16.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.061+0000 7fe8d5ffb640 1 -- 172.21.15.164:0/1934642762 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe8e800f040 con 0x55b7116fbd40 2024-09-19T08:56:16.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.061+0000 7fe8f6f27740 10 client.17724 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:56:16.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.061+0000 7fe8f6f27740 10 client.17724 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:56:16.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.061+0000 7fe8f6f27740 20 client.17724 populate_metadata read hostname 'smithi164' 2024-09-19T08:56:16.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.061+0000 7fe8f6f27740 1 --2- 172.21.15.164:0/1934642762 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55b711776a00 0x55b711778df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:16.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.061+0000 7fe8f6f27740 4 client.17724 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T08:56:16.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.061+0000 7fe8f6f27740 1 -- 172.21.15.164:0/1934642762 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b71165dd00 con 0x55b711776a00 2024-09-19T08:56:16.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.061+0000 7fe8edd9b640 1 --2- 172.21.15.164:0/1934642762 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55b711776a00 0x55b711778df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:16.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.062+0000 7fe8edd9b640 1 --2- 172.21.15.164:0/1934642762 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55b711776a00 0x55b711778df0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ce3054fc5594179 server_cookie=3d199dcac5273fa7 in_seq=0 out_seq=0 2024-09-19T08:56:16.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.062+0000 7fe8d5ffb640 10 client.17724 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:56:16.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.063+0000 7fe8d5ffb640 1 -- 172.21.15.164:0/1934642762 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b71165dd00 con 0x55b711776a00 2024-09-19T08:56:16.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.063+0000 7fe8d5ffb640 10 client.17724 handle_command_reply: tid=0 2024-09-19T08:56:16.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.063+0000 7fe8f6f27740 1 client.17724 shutdown 2024-09-19T08:56:16.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.063+0000 7fe8f6f27740 1 -- 172.21.15.164:0/1934642762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b7116fbd40 msgr2=0x55b7116f5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:16.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.063+0000 7fe8f6f27740 1 --2- 172.21.15.164:0/1934642762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b7116fbd40 0x55b7116f5d70 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009ac0 tx=0x7fe8e8012dc0 comp rx=0 tx=0).stop 2024-09-19T08:56:16.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.063+0000 7fe8f6f27740 1 -- 172.21.15.164:0/1934642762 shutdown_connections 2024-09-19T08:56:16.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.063+0000 7fe8f6f27740 1 --2- 172.21.15.164:0/1934642762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b7116fbd40 0x55b7116f5d70 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:16.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.063+0000 7fe8f6f27740 1 --2- 172.21.15.164:0/1934642762 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7116f1480 0x55b7116f5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:16.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.063+0000 7fe8f6f27740 1 --2- 172.21.15.164:0/1934642762 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55b711776a00 0x55b711778df0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:16.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.063+0000 7fe8f6f27740 1 --2- 172.21.15.164:0/1934642762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7116f0b60 0x55b7117042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:16.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.063+0000 7fe8f6f27740 1 -- 172.21.15.164:0/1934642762 >> 172.21.15.164:0/1934642762 conn(0x55b7116ed0f0 msgr2=0x55b7116ec270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:16.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.063+0000 7fe8d5ffb640 0 client.17724 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:56:16.353 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:56:16.353 INFO:teuthology.orchestra.run.smithi164.stdout: "return_code": 0, 2024-09-19T08:56:16.353 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_tag": "d8e9d411-2fc5-47d2-bc80-c6b4c8777f12", 2024-09-19T08:56:16.353 INFO:teuthology.orchestra.run.smithi164.stdout: "mode": "asynchronous" 2024-09-19T08:56:16.353 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:56:16.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.063+0000 7fe8f6f27740 1 -- 172.21.15.164:0/1934642762 shutdown_connections 2024-09-19T08:56:16.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.063+0000 7fe8f6f27740 1 -- 172.21.15.164:0/1934642762 wait complete. 2024-09-19T08:56:16.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.063+0000 7fe8f6f27740 20 client.17724 trim_cache size 0 max 16384 2024-09-19T08:56:16.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.064+0000 7fe8f4b4e640 1 -- 172.21.15.164:0/480088036 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe8d80a33b0 msgr2=0x7fe8d80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:16.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.064+0000 7fe8f4b4e640 1 --2- 172.21.15.164:0/480088036 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe8d80a33b0 0x7fe8d80a5870 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe8e00cd5e0 tx=0x7fe8e004b040 comp rx=0 tx=0).stop 2024-09-19T08:56:16.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.064+0000 7fe8f4b4e640 1 -- 172.21.15.164:0/480088036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8f0100a80 msgr2=0x7fe8f01983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:16.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.064+0000 7fe8f4b4e640 1 --2- 172.21.15.164:0/480088036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8f0100a80 0x7fe8f01983a0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000c920 tx=0x7fe8e000cde0 comp rx=0 tx=0).stop 2024-09-19T08:56:16.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.064+0000 7fe8f4b4e640 1 -- 172.21.15.164:0/480088036 shutdown_connections 2024-09-19T08:56:16.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.064+0000 7fe8f4b4e640 1 --2- 172.21.15.164:0/480088036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8f0106a80 0x7fe8f019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:16.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.064+0000 7fe8f4b4e640 1 --2- 172.21.15.164:0/480088036 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8f0101420 0x7fe8f01988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:16.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.064+0000 7fe8f4b4e640 1 --2- 172.21.15.164:0/480088036 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe8d80a33b0 0x7fe8d80a5870 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:16.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.064+0000 7fe8f4b4e640 1 --2- 172.21.15.164:0/480088036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8f0100a80 0x7fe8f01983a0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:16.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.064+0000 7fe8f4b4e640 1 -- 172.21.15.164:0/480088036 >> 172.21.15.164:0/480088036 conn(0x7fe8f00fc7a0 msgr2=0x7fe8f00fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:16.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.064+0000 7fe8f4b4e640 1 -- 172.21.15.164:0/480088036 shutdown_connections 2024-09-19T08:56:16.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:16.064+0000 7fe8f4b4e640 1 -- 172.21.15.164:0/480088036 wait complete. 2024-09-19T08:56:16.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:16 smithi164 ceph-mon[30928]: pgmap v325: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T08:56:16.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:16 smithi170 ceph-mon[35072]: pgmap v325: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T08:56:16.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:16 smithi173 ceph-mon[35306]: pgmap v325: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T08:56:17.165 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:56:17.417 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:56:17.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:17 smithi164 ceph-mon[30928]: scrub queued for path: / 2024-09-19T08:56:17.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:17 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:56:17.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:17 smithi164 ceph-mon[30928]: scrub summary: active paths [/] 2024-09-19T08:56:17.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:17 smithi164 ceph-mon[30928]: scrub summary: active paths [/] 2024-09-19T08:56:17.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:17 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:56:17.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:17 smithi164 ceph-mon[30928]: scrub summary: active paths [/] 2024-09-19T08:56:17.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:17 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:56:17.533 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:17 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:56:17.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:17 smithi170 ceph-mon[35072]: scrub queued for path: / 2024-09-19T08:56:17.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:17 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:56:17.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:17 smithi170 ceph-mon[35072]: scrub summary: active paths [/] 2024-09-19T08:56:17.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:17 smithi170 ceph-mon[35072]: scrub summary: active paths [/] 2024-09-19T08:56:17.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:17 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:56:17.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:17 smithi170 ceph-mon[35072]: scrub summary: active paths [/] 2024-09-19T08:56:17.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:17 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:56:17.626 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:17 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:56:17.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:17 smithi173 ceph-mon[35306]: scrub queued for path: / 2024-09-19T08:56:17.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:17 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:56:17.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:17 smithi173 ceph-mon[35306]: scrub summary: active paths [/] 2024-09-19T08:56:17.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:17 smithi173 ceph-mon[35306]: scrub summary: active paths [/] 2024-09-19T08:56:17.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:17 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:56:17.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:17 smithi173 ceph-mon[35306]: scrub summary: active paths [/] 2024-09-19T08:56:17.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:17 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:56:17.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:17 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:56:18.508 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:18 smithi164 ceph-mon[30928]: pgmap v326: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T08:56:18.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:18 smithi170 ceph-mon[35072]: pgmap v326: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T08:56:18.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:18 smithi173 ceph-mon[35306]: pgmap v326: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T08:56:18.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.847+0000 7feab70bd640 1 -- 172.21.15.164:0/2074002841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab01082a0 msgr2=0x7feab010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:18.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.847+0000 7feab70bd640 1 --2- 172.21.15.164:0/2074002841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab01082a0 0x7feab010ec40 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7feaa8009930 tx=0x7feaa802a420 comp rx=0 tx=0).stop 2024-09-19T08:56:18.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.847+0000 7feab70bd640 1 -- 172.21.15.164:0/2074002841 shutdown_connections 2024-09-19T08:56:18.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.847+0000 7feab70bd640 1 --2- 172.21.15.164:0/2074002841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab01082a0 0x7feab010ec40 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:18.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.847+0000 7feab70bd640 1 --2- 172.21.15.164:0/2074002841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feab01036e0 0x7feab0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:18.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.847+0000 7feab70bd640 1 --2- 172.21.15.164:0/2074002841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feab0102d30 0x7feab0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:18.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.847+0000 7feab70bd640 1 -- 172.21.15.164:0/2074002841 >> 172.21.15.164:0/2074002841 conn(0x7feab00fc7d0 msgr2=0x7feab00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:18.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.847+0000 7feab70bd640 1 -- 172.21.15.164:0/2074002841 shutdown_connections 2024-09-19T08:56:18.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.847+0000 7feab70bd640 1 -- 172.21.15.164:0/2074002841 wait complete. 2024-09-19T08:56:18.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.848+0000 7feab70bd640 1 Processor -- start 2024-09-19T08:56:18.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.848+0000 7feab70bd640 1 -- start start 2024-09-19T08:56:18.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.848+0000 7feab70bd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feab0102d30 0x7feab0198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:18.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.848+0000 7feab70bd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feab01036e0 0x7feab0198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:18.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.848+0000 7feab70bd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab01082a0 0x7feab019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:18.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.848+0000 7feab70bd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feab0112100 con 0x7feab01082a0 2024-09-19T08:56:18.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.848+0000 7feab70bd640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7feab0111f80 con 0x7feab0102d30 2024-09-19T08:56:18.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.848+0000 7feab70bd640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feab0112280 con 0x7feab01036e0 2024-09-19T08:56:18.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.848+0000 7feab565a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab01082a0 0x7feab019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:18.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.848+0000 7feab565a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab01082a0 0x7feab019ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38690/0 (socket says 172.21.15.164:38690) 2024-09-19T08:56:18.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.848+0000 7feab565a640 1 -- 172.21.15.164:0/2997905700 learned_addr learned my addr 172.21.15.164:0/2997905700 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:18.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.848+0000 7feaaffff640 1 --2- 172.21.15.164:0/2997905700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feab01036e0 0x7feab0198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:18.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.848+0000 7feab4e59640 1 --2- 172.21.15.164:0/2997905700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feab0102d30 0x7feab0198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:18.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.848+0000 7feab565a640 1 -- 172.21.15.164:0/2997905700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feab01036e0 msgr2=0x7feab0198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:18.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.848+0000 7feab565a640 1 --2- 172.21.15.164:0/2997905700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feab01036e0 0x7feab0198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:18.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.848+0000 7feab565a640 1 -- 172.21.15.164:0/2997905700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feab0102d30 msgr2=0x7feab0198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:18.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.848+0000 7feab565a640 1 --2- 172.21.15.164:0/2997905700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feab0102d30 0x7feab0198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:18.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.848+0000 7feab565a640 1 -- 172.21.15.164:0/2997905700 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feab019d540 con 0x7feab01082a0 2024-09-19T08:56:18.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.848+0000 7feab4e59640 1 --2- 172.21.15.164:0/2997905700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feab0102d30 0x7feab0198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:56:18.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.849+0000 7feab565a640 1 --2- 172.21.15.164:0/2997905700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab01082a0 0x7feab019ce60 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7feaa8005ab0 tx=0x7feaa80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:18.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.849+0000 7feaadffb640 1 -- 172.21.15.164:0/2997905700 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa803f030 con 0x7feab01082a0 2024-09-19T08:56:18.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.849+0000 7feab70bd640 1 -- 172.21.15.164:0/2997905700 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feab019d7d0 con 0x7feab01082a0 2024-09-19T08:56:18.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.849+0000 7feaadffb640 1 -- 172.21.15.164:0/2997905700 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaa8003ce0 con 0x7feab01082a0 2024-09-19T08:56:18.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.849+0000 7feaadffb640 1 -- 172.21.15.164:0/2997905700 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa8035070 con 0x7feab01082a0 2024-09-19T08:56:18.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.849+0000 7feab70bd640 1 -- 172.21.15.164:0/2997905700 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feab01a5110 con 0x7feab01082a0 2024-09-19T08:56:18.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.851+0000 7feaadffb640 1 -- 172.21.15.164:0/2997905700 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7feaa8042030 con 0x7feab01082a0 2024-09-19T08:56:18.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.852+0000 7feab9496740 1 Processor -- start 2024-09-19T08:56:18.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.852+0000 7feab9496740 1 -- start start 2024-09-19T08:56:18.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.852+0000 7feaadffb640 1 --2- 172.21.15.164:0/2997905700 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fea9c0a33b0 0x7fea9c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:18.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.852+0000 7feab9496740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e5e4a74a0 0x556e5e4a7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:18.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.852+0000 7feaaffff640 1 --2- 172.21.15.164:0/2997905700 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fea9c0a33b0 0x7fea9c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:18.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.852+0000 7feab9496740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556e5e4a7dc0 0x556e5e4b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:18.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.852+0000 7feaadffb640 1 -- 172.21.15.164:0/2997905700 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7feaa80f02b0 con 0x7feab01082a0 2024-09-19T08:56:18.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.852+0000 7feab9496740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e5e4b2680 0x556e5e4b4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:18.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.852+0000 7feab9496740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556e5e40d2e0 con 0x556e5e4b2680 2024-09-19T08:56:18.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.852+0000 7feab9496740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556e5e3b4ce0 con 0x556e5e4a74a0 2024-09-19T08:56:18.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.852+0000 7feab9496740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556e5e4092a0 con 0x556e5e4a7dc0 2024-09-19T08:56:18.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.852+0000 7feab4e59640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e5e4a74a0 0x556e5e4a7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:18.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.853+0000 7feab4e59640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e5e4a74a0 0x556e5e4a7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.164:42220/0 (socket says 172.21.15.164:42220) 2024-09-19T08:56:18.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.853+0000 7feab4e59640 1 -- 172.21.15.164:0/3143388142 learned_addr learned my addr 172.21.15.164:0/3143388142 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:18.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.853+0000 7feab565a640 1 --2- 172.21.15.164:0/3143388142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e5e4b2680 0x556e5e4b4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:18.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.853+0000 7feaaffff640 1 --2- 172.21.15.164:0/3143388142 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556e5e4a7dc0 0x556e5e4b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:18.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.853+0000 7feab565a640 1 -- 172.21.15.164:0/3143388142 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556e5e4a7dc0 msgr2=0x556e5e4b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:18.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.853+0000 7feab565a640 1 --2- 172.21.15.164:0/3143388142 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556e5e4a7dc0 0x556e5e4b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:18.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.853+0000 7feaaffff640 1 --2- 172.21.15.164:0/2997905700 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fea9c0a33b0 0x7fea9c0a5870 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7feab00fede0 tx=0x7fea98013880 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:18.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.853+0000 7feab565a640 1 -- 172.21.15.164:0/3143388142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e5e4a74a0 msgr2=0x556e5e4a7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:18.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.853+0000 7feab565a640 1 --2- 172.21.15.164:0/3143388142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e5e4a74a0 0x556e5e4a7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:18.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.853+0000 7feab565a640 1 -- 172.21.15.164:0/3143388142 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e5e399440 con 0x556e5e4b2680 2024-09-19T08:56:18.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.853+0000 7feab4e59640 1 --2- 172.21.15.164:0/3143388142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e5e4a74a0 0x556e5e4a7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:56:18.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.853+0000 7feab565a640 1 --2- 172.21.15.164:0/3143388142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e5e4b2680 0x556e5e4b4a70 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7feaa80eda10 tx=0x7feaa8003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c895105e427a3368 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:18.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.853+0000 7feaa6ffd640 1 -- 172.21.15.164:0/3143388142 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa8049040 con 0x556e5e4b2680 2024-09-19T08:56:18.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.853+0000 7feaa6ffd640 1 -- 172.21.15.164:0/3143388142 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaa80bcca0 con 0x556e5e4b2680 2024-09-19T08:56:18.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.853+0000 7feaa6ffd640 1 -- 172.21.15.164:0/3143388142 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa80b8c10 con 0x556e5e4b2680 2024-09-19T08:56:18.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.854+0000 7feab9496740 1 -- 172.21.15.164:0/3143388142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e5e4b2680 msgr2=0x556e5e4b4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:18.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.854+0000 7feab9496740 1 --2- 172.21.15.164:0/3143388142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e5e4b2680 0x556e5e4b4a70 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7feaa80eda10 tx=0x7feaa8003f50 comp rx=0 tx=0).stop 2024-09-19T08:56:18.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.854+0000 7feab9496740 1 -- 172.21.15.164:0/3143388142 shutdown_connections 2024-09-19T08:56:18.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.854+0000 7feab9496740 1 --2- 172.21.15.164:0/3143388142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e5e4b2680 0x556e5e4b4a70 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:18.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.854+0000 7feab9496740 1 --2- 172.21.15.164:0/3143388142 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556e5e4a7dc0 0x556e5e4b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:18.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.854+0000 7feab9496740 1 --2- 172.21.15.164:0/3143388142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e5e4a74a0 0x556e5e4a7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:18.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.854+0000 7feab9496740 1 -- 172.21.15.164:0/3143388142 >> 172.21.15.164:0/3143388142 conn(0x556e5e4a3a30 msgr2=0x556e5e4a3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:18.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.854+0000 7feab9496740 1 -- 172.21.15.164:0/3143388142 shutdown_connections 2024-09-19T08:56:18.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.854+0000 7feab9496740 1 -- 172.21.15.164:0/3143388142 wait complete. 2024-09-19T08:56:18.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.855+0000 7feab9496740 1 -- start start 2024-09-19T08:56:18.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.855+0000 7feab9496740 1 Processor -- start 2024-09-19T08:56:18.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.855+0000 7feab9496740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e5e4a74a0 0x556e5e4b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:18.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.855+0000 7feab9496740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556e5e4a7dc0 0x556e5e4af250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:18.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.855+0000 7feab9496740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e5e4b2680 0x556e5e4af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:18.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.855+0000 7feab9496740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556e5e4b5010 con 0x556e5e4b2680 2024-09-19T08:56:18.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.855+0000 7feab9496740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556e5e3c8050 con 0x556e5e4a74a0 2024-09-19T08:56:18.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.855+0000 7feab9496740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556e5e4b5190 con 0x556e5e4a7dc0 2024-09-19T08:56:18.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.855+0000 7feab9496740 10 client.0 _ll_register_callbacks cb 0x556e5e408160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:56:18.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.856+0000 7feab565a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e5e4b2680 0x556e5e4af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:18.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.856+0000 7feab565a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e5e4b2680 0x556e5e4af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38698/0 (socket says 172.21.15.164:38698) 2024-09-19T08:56:18.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.856+0000 7feab565a640 1 -- 172.21.15.164:0/518579405 learned_addr learned my addr 172.21.15.164:0/518579405 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:18.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.856+0000 7feab4e59640 1 --2- 172.21.15.164:0/518579405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e5e4a74a0 0x556e5e4b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:18.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.856+0000 7feaaffff640 1 --2- 172.21.15.164:0/518579405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556e5e4a7dc0 0x556e5e4af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:18.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.856+0000 7feab565a640 1 -- 172.21.15.164:0/518579405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556e5e4a7dc0 msgr2=0x556e5e4af250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:18.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.856+0000 7feab565a640 1 --2- 172.21.15.164:0/518579405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556e5e4a7dc0 0x556e5e4af250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:18.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.856+0000 7feab565a640 1 -- 172.21.15.164:0/518579405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e5e4a74a0 msgr2=0x556e5e4b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:18.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.856+0000 7feab565a640 1 --2- 172.21.15.164:0/518579405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e5e4a74a0 0x556e5e4b1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:18.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.856+0000 7feab565a640 1 -- 172.21.15.164:0/518579405 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e5e17e3c0 con 0x556e5e4b2680 2024-09-19T08:56:18.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.856+0000 7feab565a640 1 --2- 172.21.15.164:0/518579405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e5e4b2680 0x556e5e4af790 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7feaa80bce50 tx=0x7feaa80edf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a674c199dd0ce8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:18.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.856+0000 7fea8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:56:18.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.856+0000 7fea8ffff640 1 -- 172.21.15.164:0/518579405 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa80d9070 con 0x556e5e4b2680 2024-09-19T08:56:18.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.856+0000 7fea8ffff640 1 -- 172.21.15.164:0/518579405 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7feaa803bc10 con 0x556e5e4b2680 2024-09-19T08:56:18.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.856+0000 7feab9496740 1 -- 172.21.15.164:0/518579405 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaa80bcca0 con 0x556e5e4b2680 2024-09-19T08:56:18.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.856+0000 7fea8ffff640 1 -- 172.21.15.164:0/518579405 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feaa8049050 con 0x556e5e4b2680 2024-09-19T08:56:18.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.856+0000 7feab9496740 10 client.17736 fetch_fsmap learned FSMap version 32 2024-09-19T08:56:18.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.856+0000 7feab9496740 1 -- 172.21.15.164:0/518579405 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556e5e4b5010 con 0x556e5e4b2680 2024-09-19T08:56:18.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.857+0000 7fea8ffff640 1 -- 172.21.15.164:0/518579405 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaa80bcca0 con 0x556e5e4b2680 2024-09-19T08:56:18.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.857+0000 7fea8ffff640 1 -- 172.21.15.164:0/518579405 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7feaa80c4d80 con 0x556e5e4b2680 2024-09-19T08:56:18.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.857+0000 7feab9496740 10 client.17736 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:56:18.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.857+0000 7feab9496740 10 client.17736 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:56:18.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.857+0000 7feab9496740 20 client.17736 populate_metadata read hostname 'smithi164' 2024-09-19T08:56:18.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.857+0000 7feab9496740 1 --2- 172.21.15.164:0/518579405 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556e5e4c1be0 0x556e5e4c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:18.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.857+0000 7feab9496740 4 client.17736 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:56:18.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.857+0000 7feab9496740 1 -- 172.21.15.164:0/518579405 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e5e41a850 con 0x556e5e4c1be0 2024-09-19T08:56:18.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.857+0000 7feab4e59640 1 --2- 172.21.15.164:0/518579405 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556e5e4c1be0 0x556e5e4c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:18.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.858+0000 7fea8ffff640 10 client.17736 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:56:18.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.858+0000 7feab4e59640 1 --2- 172.21.15.164:0/518579405 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556e5e4c1be0 0x556e5e4c3fd0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f847a0d46c28f29 server_cookie=3bbab5654ecc2068 in_seq=0 out_seq=0 2024-09-19T08:56:18.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.858+0000 7fea8ffff640 1 -- 172.21.15.164:0/518579405 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556e5e41a850 con 0x556e5e4c1be0 2024-09-19T08:56:18.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.858+0000 7fea8ffff640 10 client.17736 handle_command_reply: tid=0 2024-09-19T08:56:18.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.858+0000 7feab9496740 1 client.17736 shutdown 2024-09-19T08:56:18.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.858+0000 7feab9496740 1 -- 172.21.15.164:0/518579405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e5e4b2680 msgr2=0x556e5e4af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:18.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.858+0000 7feab9496740 1 --2- 172.21.15.164:0/518579405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e5e4b2680 0x556e5e4af790 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7feaa80bce50 tx=0x7feaa80edf40 comp rx=0 tx=0).stop 2024-09-19T08:56:18.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.858+0000 7feab9496740 1 -- 172.21.15.164:0/518579405 shutdown_connections 2024-09-19T08:56:18.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.858+0000 7feab9496740 1 --2- 172.21.15.164:0/518579405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e5e4b2680 0x556e5e4af790 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:18.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.858+0000 7feab9496740 1 --2- 172.21.15.164:0/518579405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556e5e4a7dc0 0x556e5e4af250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:18.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.858+0000 7feab9496740 1 --2- 172.21.15.164:0/518579405 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556e5e4c1be0 0x556e5e4c3fd0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:18.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.858+0000 7feab9496740 1 --2- 172.21.15.164:0/518579405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e5e4a74a0 0x556e5e4b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:18.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.858+0000 7feab9496740 1 -- 172.21.15.164:0/518579405 >> 172.21.15.164:0/518579405 conn(0x556e5e4a3a30 msgr2=0x556e5e4a9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:18.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.859+0000 7fea8ffff640 0 client.17736 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:56:18.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.859+0000 7feab9496740 1 -- 172.21.15.164:0/518579405 shutdown_connections 2024-09-19T08:56:18.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.859+0000 7feab9496740 1 -- 172.21.15.164:0/518579405 wait complete. 2024-09-19T08:56:18.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.859+0000 7feab9496740 20 client.17736 trim_cache size 0 max 16384 2024-09-19T08:56:18.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.875+0000 7feab9496740 1 Processor -- start 2024-09-19T08:56:18.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.875+0000 7feab9496740 1 -- start start 2024-09-19T08:56:18.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.876+0000 7feab9496740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e5e4a74a0 0x556e5e4be1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:18.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.876+0000 7feab9496740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e5e4a7dc0 0x556e5e4ba730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:18.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.876+0000 7feab9496740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556e5e4b2680 0x556e5e4bac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:18.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.876+0000 7feab9496740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feab01ad7e0 con 0x556e5e4a7dc0 2024-09-19T08:56:18.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.876+0000 7feab9496740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7feab01af320 con 0x556e5e4a74a0 2024-09-19T08:56:18.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.876+0000 7feab9496740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feab01adeb0 con 0x556e5e4b2680 2024-09-19T08:56:18.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.876+0000 7feaaffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e5e4a7dc0 0x556e5e4ba730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:18.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.876+0000 7feaaffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e5e4a7dc0 0x556e5e4ba730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38710/0 (socket says 172.21.15.164:38710) 2024-09-19T08:56:18.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.876+0000 7feaaffff640 1 -- 172.21.15.164:0/1761493044 learned_addr learned my addr 172.21.15.164:0/1761493044 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:18.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.876+0000 7feab4e59640 1 --2- 172.21.15.164:0/1761493044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e5e4a74a0 0x556e5e4be1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:18.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.876+0000 7feaaffff640 1 -- 172.21.15.164:0/1761493044 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556e5e4b2680 msgr2=0x556e5e4bac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:18.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.876+0000 7feaaffff640 1 --2- 172.21.15.164:0/1761493044 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556e5e4b2680 0x556e5e4bac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:18.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.876+0000 7feaaffff640 1 -- 172.21.15.164:0/1761493044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e5e4a74a0 msgr2=0x556e5e4be1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:18.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.876+0000 7feaaffff640 1 --2- 172.21.15.164:0/1761493044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e5e4a74a0 0x556e5e4be1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:18.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.876+0000 7feaaffff640 1 -- 172.21.15.164:0/1761493044 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e5e4a72d0 con 0x556e5e4a7dc0 2024-09-19T08:56:18.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.876+0000 7feab4e59640 1 --2- 172.21.15.164:0/1761493044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e5e4a74a0 0x556e5e4be1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:56:18.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.876+0000 7feaaffff640 1 --2- 172.21.15.164:0/1761493044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e5e4a7dc0 0x556e5e4ba730 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7feab006a370 tx=0x7fea9801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96ad06fc18c5bc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:18.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.876+0000 7feaa67fc640 1 -- 172.21.15.164:0/1761493044 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea98005020 con 0x556e5e4a7dc0 2024-09-19T08:56:18.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.876+0000 7feaa67fc640 1 -- 172.21.15.164:0/1761493044 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea980047a0 con 0x556e5e4a7dc0 2024-09-19T08:56:18.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.876+0000 7feaa67fc640 1 -- 172.21.15.164:0/1761493044 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9801f6b0 con 0x556e5e4a7dc0 2024-09-19T08:56:18.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.877+0000 7feab9496740 1 -- 172.21.15.164:0/1761493044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e5e4a7dc0 msgr2=0x556e5e4ba730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:18.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.877+0000 7feab9496740 1 --2- 172.21.15.164:0/1761493044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e5e4a7dc0 0x556e5e4ba730 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7feab006a370 tx=0x7fea9801d000 comp rx=0 tx=0).stop 2024-09-19T08:56:18.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.877+0000 7feab9496740 1 -- 172.21.15.164:0/1761493044 shutdown_connections 2024-09-19T08:56:18.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.877+0000 7feab9496740 1 --2- 172.21.15.164:0/1761493044 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556e5e4b2680 0x556e5e4bac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:18.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.877+0000 7feab9496740 1 --2- 172.21.15.164:0/1761493044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e5e4a7dc0 0x556e5e4ba730 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:18.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.877+0000 7feab9496740 1 --2- 172.21.15.164:0/1761493044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e5e4a74a0 0x556e5e4be1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:18.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.877+0000 7feab9496740 1 -- 172.21.15.164:0/1761493044 >> 172.21.15.164:0/1761493044 conn(0x556e5e4a3a30 msgr2=0x556e5e4ae340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:18.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.877+0000 7feab9496740 1 -- 172.21.15.164:0/1761493044 shutdown_connections 2024-09-19T08:56:18.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.877+0000 7feab9496740 1 -- 172.21.15.164:0/1761493044 wait complete. 2024-09-19T08:56:18.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.877+0000 7feab9496740 1 -- start start 2024-09-19T08:56:18.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.878+0000 7feab9496740 1 Processor -- start 2024-09-19T08:56:18.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.878+0000 7feab9496740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e5e4a74a0 0x556e5e4aec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:18.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.878+0000 7feab9496740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556e5e4a7dc0 0x556e5e4af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:18.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.878+0000 7feab9496740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e5e4b2680 0x556e5e528010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:18.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.878+0000 7feab9496740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feab01aec50 con 0x556e5e4b2680 2024-09-19T08:56:18.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.878+0000 7feab9496740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7feab01ae580 con 0x556e5e4a74a0 2024-09-19T08:56:18.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.878+0000 7feab9496740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feab01ab7e0 con 0x556e5e4a7dc0 2024-09-19T08:56:18.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.878+0000 7feab9496740 10 client.0 _ll_register_callbacks cb 0x7feaa0009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:56:18.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.878+0000 7feab565a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e5e4b2680 0x556e5e528010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:18.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.878+0000 7feab565a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e5e4b2680 0x556e5e528010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38714/0 (socket says 172.21.15.164:38714) 2024-09-19T08:56:18.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.878+0000 7feab565a640 1 -- 172.21.15.164:0/3220960528 learned_addr learned my addr 172.21.15.164:0/3220960528 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:18.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.878+0000 7feab4e59640 1 --2- 172.21.15.164:0/3220960528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e5e4a74a0 0x556e5e4aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:18.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.878+0000 7feaaffff640 1 --2- 172.21.15.164:0/3220960528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556e5e4a7dc0 0x556e5e4af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:18.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.878+0000 7feab565a640 1 -- 172.21.15.164:0/3220960528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556e5e4a7dc0 msgr2=0x556e5e4af1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:18.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.878+0000 7feab565a640 1 --2- 172.21.15.164:0/3220960528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556e5e4a7dc0 0x556e5e4af1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:18.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.878+0000 7feab565a640 1 -- 172.21.15.164:0/3220960528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e5e4a74a0 msgr2=0x556e5e4aec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:18.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.878+0000 7feab565a640 1 --2- 172.21.15.164:0/3220960528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e5e4a74a0 0x556e5e4aec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:18.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.878+0000 7feab565a640 1 -- 172.21.15.164:0/3220960528 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e5e5286b0 con 0x556e5e4b2680 2024-09-19T08:56:18.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.878+0000 7feab565a640 1 --2- 172.21.15.164:0/3220960528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e5e4b2680 0x556e5e528010 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7feaa8031550 tx=0x7feaa803b570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1550673f4e13ae5f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:18.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.879+0000 7fea8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:56:18.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.879+0000 7fea8ffff640 1 -- 172.21.15.164:0/3220960528 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa80d9070 con 0x556e5e4b2680 2024-09-19T08:56:18.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.879+0000 7fea8ffff640 1 -- 172.21.15.164:0/3220960528 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7feaa80bb9f0 con 0x556e5e4b2680 2024-09-19T08:56:18.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.879+0000 7feab9496740 1 -- 172.21.15.164:0/3220960528 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea980047a0 con 0x556e5e4b2680 2024-09-19T08:56:18.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.879+0000 7fea8ffff640 1 -- 172.21.15.164:0/3220960528 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feaa80c09a0 con 0x556e5e4b2680 2024-09-19T08:56:18.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.879+0000 7feab9496740 10 client.17742 fetch_fsmap learned FSMap version 32 2024-09-19T08:56:18.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.879+0000 7feab9496740 1 -- 172.21.15.164:0/3220960528 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556e5e5286b0 con 0x556e5e4b2680 2024-09-19T08:56:18.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.879+0000 7fea8ffff640 1 -- 172.21.15.164:0/3220960528 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea980047a0 con 0x556e5e4b2680 2024-09-19T08:56:18.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.879+0000 7fea8ffff640 1 -- 172.21.15.164:0/3220960528 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7feaa80d5380 con 0x556e5e4b2680 2024-09-19T08:56:18.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.879+0000 7feab9496740 10 client.17742 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:56:18.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.879+0000 7feab9496740 10 client.17742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:56:18.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.879+0000 7feab9496740 20 client.17742 populate_metadata read hostname 'smithi164' 2024-09-19T08:56:18.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.879+0000 7feab9496740 1 --2- 172.21.15.164:0/3220960528 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556e5e530e70 0x556e5e533290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:18.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.879+0000 7feab9496740 4 client.17742 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:56:18.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.879+0000 7feab9496740 1 -- 172.21.15.164:0/3220960528 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556e5e414d60 con 0x556e5e530e70 2024-09-19T08:56:18.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.879+0000 7feab4e59640 1 --2- 172.21.15.164:0/3220960528 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556e5e530e70 0x556e5e533290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:18.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.880+0000 7feab4e59640 1 --2- 172.21.15.164:0/3220960528 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556e5e530e70 0x556e5e533290 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37e73128f6fced7e server_cookie=9c8bdf8253d0cf58 in_seq=0 out_seq=0 2024-09-19T08:56:18.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.880+0000 7fea8ffff640 10 client.17742 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:56:18.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.880+0000 7fea8ffff640 1 -- 172.21.15.164:0/3220960528 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556e5e414d60 con 0x556e5e530e70 2024-09-19T08:56:18.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.880+0000 7fea8ffff640 10 client.17742 handle_command_reply: tid=0 2024-09-19T08:56:18.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.880+0000 7feab9496740 1 client.17742 shutdown 2024-09-19T08:56:18.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.880+0000 7feab9496740 1 -- 172.21.15.164:0/3220960528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e5e4b2680 msgr2=0x556e5e528010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:18.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.880+0000 7feab9496740 1 --2- 172.21.15.164:0/3220960528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e5e4b2680 0x556e5e528010 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7feaa8031550 tx=0x7feaa803b570 comp rx=0 tx=0).stop 2024-09-19T08:56:18.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.880+0000 7feab9496740 1 -- 172.21.15.164:0/3220960528 shutdown_connections 2024-09-19T08:56:18.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.880+0000 7feab9496740 1 --2- 172.21.15.164:0/3220960528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e5e4b2680 0x556e5e528010 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:18.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.880+0000 7feab9496740 1 --2- 172.21.15.164:0/3220960528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556e5e4a7dc0 0x556e5e4af1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:18.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.880+0000 7feab9496740 1 --2- 172.21.15.164:0/3220960528 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x556e5e530e70 0x556e5e533290 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:18.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.880+0000 7feab9496740 1 --2- 172.21.15.164:0/3220960528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e5e4a74a0 0x556e5e4aec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:18.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.880+0000 7feab9496740 1 -- 172.21.15.164:0/3220960528 >> 172.21.15.164:0/3220960528 conn(0x556e5e4a3a30 msgr2=0x556e5e4b0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:18.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.880+0000 7fea8ffff640 0 client.17742 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:56:18.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.880+0000 7feab9496740 1 -- 172.21.15.164:0/3220960528 shutdown_connections 2024-09-19T08:56:18.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.880+0000 7feab9496740 1 -- 172.21.15.164:0/3220960528 wait complete. 2024-09-19T08:56:18.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.880+0000 7feab9496740 20 client.17742 trim_cache size 0 max 16384 2024-09-19T08:56:18.887 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:56:18.887 INFO:teuthology.orchestra.run.smithi164.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T08:56:18.887 INFO:teuthology.orchestra.run.smithi164.stdout: "scrubs": { 2024-09-19T08:56:18.887 INFO:teuthology.orchestra.run.smithi164.stdout: "d8e9d411-2fc5-47d2-bc80-c6b4c8777f12": { 2024-09-19T08:56:18.887 INFO:teuthology.orchestra.run.smithi164.stdout: "path": "/", 2024-09-19T08:56:18.887 INFO:teuthology.orchestra.run.smithi164.stdout: "tag": "d8e9d411-2fc5-47d2-bc80-c6b4c8777f12", 2024-09-19T08:56:18.887 INFO:teuthology.orchestra.run.smithi164.stdout: "options": "recursive,force" 2024-09-19T08:56:18.887 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:56:18.888 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:56:18.888 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:56:18.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.881+0000 7feab70bd640 1 -- 172.21.15.164:0/2997905700 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fea9c0a33b0 msgr2=0x7fea9c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:18.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.881+0000 7feab70bd640 1 --2- 172.21.15.164:0/2997905700 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fea9c0a33b0 0x7fea9c0a5870 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7feab00fede0 tx=0x7fea98013880 comp rx=0 tx=0).stop 2024-09-19T08:56:18.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.881+0000 7feab70bd640 1 -- 172.21.15.164:0/2997905700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab01082a0 msgr2=0x7feab019ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:18.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.881+0000 7feab70bd640 1 --2- 172.21.15.164:0/2997905700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab01082a0 0x7feab019ce60 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7feaa8005ab0 tx=0x7feaa80042f0 comp rx=0 tx=0).stop 2024-09-19T08:56:18.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.881+0000 7feab70bd640 1 -- 172.21.15.164:0/2997905700 shutdown_connections 2024-09-19T08:56:18.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.881+0000 7feab70bd640 1 --2- 172.21.15.164:0/2997905700 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fea9c0a33b0 0x7fea9c0a5870 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:18.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.881+0000 7feab70bd640 1 --2- 172.21.15.164:0/2997905700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab01082a0 0x7feab019ce60 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:18.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.881+0000 7feab70bd640 1 --2- 172.21.15.164:0/2997905700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feab01036e0 0x7feab0198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:18.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.881+0000 7feab70bd640 1 --2- 172.21.15.164:0/2997905700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feab0102d30 0x7feab0198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:18.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.881+0000 7feab70bd640 1 -- 172.21.15.164:0/2997905700 >> 172.21.15.164:0/2997905700 conn(0x7feab00fc7d0 msgr2=0x7feab0105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:18.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.881+0000 7feab70bd640 1 -- 172.21.15.164:0/2997905700 shutdown_connections 2024-09-19T08:56:18.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:18.881+0000 7feab70bd640 1 -- 172.21.15.164:0/2997905700 wait complete. 2024-09-19T08:56:19.677 INFO:tasks.cephfs.filesystem:scrub status for tag:d8e9d411-2fc5-47d2-bc80-c6b4c8777f12 - {'path': '/', 'tag': 'd8e9d411-2fc5-47d2-bc80-c6b4c8777f12', 'options': 'recursive,force'} 2024-09-19T08:56:20.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:20 smithi164 ceph-mon[30928]: pgmap v327: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 183 B/s wr, 9 op/s 2024-09-19T08:56:20.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:20 smithi164 ceph-mon[30928]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T08:56:20.624 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:20 smithi170 ceph-mon[35072]: pgmap v327: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 183 B/s wr, 9 op/s 2024-09-19T08:56:20.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:20 smithi170 ceph-mon[35072]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T08:56:20.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:20 smithi173 ceph-mon[35306]: pgmap v327: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 183 B/s wr, 9 op/s 2024-09-19T08:56:20.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:20 smithi173 ceph-mon[35306]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T08:56:22.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:22 smithi164 ceph-mon[30928]: pgmap v329: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 102 B/s wr, 13 op/s 2024-09-19T08:56:22.624 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:22 smithi170 ceph-mon[35072]: pgmap v329: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 102 B/s wr, 13 op/s 2024-09-19T08:56:22.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:22 smithi173 ceph-mon[35306]: pgmap v329: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 102 B/s wr, 13 op/s 2024-09-19T08:56:23.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:23 smithi164 ceph-mon[30928]: scrub summary: idle 2024-09-19T08:56:23.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:23 smithi164 ceph-mon[30928]: scrub summary: idle 2024-09-19T08:56:23.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:23 smithi170 ceph-mon[35072]: scrub summary: idle 2024-09-19T08:56:23.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:23 smithi170 ceph-mon[35072]: scrub summary: idle 2024-09-19T08:56:23.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:23 smithi173 ceph-mon[35306]: scrub summary: idle 2024-09-19T08:56:23.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:23 smithi173 ceph-mon[35306]: scrub summary: idle 2024-09-19T08:56:24.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:24 smithi164 ceph-mon[30928]: pgmap v330: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 102 B/s wr, 13 op/s 2024-09-19T08:56:24.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:24 smithi170 ceph-mon[35072]: pgmap v330: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 102 B/s wr, 13 op/s 2024-09-19T08:56:24.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:24 smithi173 ceph-mon[35306]: pgmap v330: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 102 B/s wr, 13 op/s 2024-09-19T08:56:25.785 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:56:26.046 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:56:26.462 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:26 smithi164 ceph-mon[30928]: pgmap v331: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 13 op/s 2024-09-19T08:56:26.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.543+0000 7ff67095e640 1 -- 172.21.15.164:0/1448406397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff66c102570 msgr2=0x7ff66c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:26.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.543+0000 7ff67095e640 1 --2- 172.21.15.164:0/1448406397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff66c102570 0x7ff66c1029b0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7ff654009900 tx=0x7ff65402a480 comp rx=0 tx=0).stop 2024-09-19T08:56:26.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.543+0000 7ff67095e640 1 -- 172.21.15.164:0/1448406397 shutdown_connections 2024-09-19T08:56:26.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.543+0000 7ff67095e640 1 --2- 172.21.15.164:0/1448406397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff66c102ef0 0x7ff66c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:26.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.543+0000 7ff67095e640 1 --2- 172.21.15.164:0/1448406397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff66c102570 0x7ff66c1029b0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:26.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.543+0000 7ff67095e640 1 --2- 172.21.15.164:0/1448406397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff66c100e20 0x7ff66c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:26.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.543+0000 7ff67095e640 1 -- 172.21.15.164:0/1448406397 >> 172.21.15.164:0/1448406397 conn(0x7ff66c0fc7d0 msgr2=0x7ff66c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:26.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.543+0000 7ff67095e640 1 -- 172.21.15.164:0/1448406397 shutdown_connections 2024-09-19T08:56:26.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.543+0000 7ff67095e640 1 -- 172.21.15.164:0/1448406397 wait complete. 2024-09-19T08:56:26.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.543+0000 7ff67095e640 1 Processor -- start 2024-09-19T08:56:26.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.543+0000 7ff67095e640 1 -- start start 2024-09-19T08:56:26.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.544+0000 7ff67095e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff66c100e20 0x7ff66c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:26.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.544+0000 7ff67095e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff66c102570 0x7ff66c198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:26.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.544+0000 7ff67095e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff66c102ef0 0x7ff66c19cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:26.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.544+0000 7ff67095e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c111020 con 0x7ff66c102ef0 2024-09-19T08:56:26.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.544+0000 7ff67095e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c110ea0 con 0x7ff66c102570 2024-09-19T08:56:26.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.544+0000 7ff67095e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c1111a0 con 0x7ff66c100e20 2024-09-19T08:56:26.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.544+0000 7ff66ad9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff66c102ef0 0x7ff66c19cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:26.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.544+0000 7ff66ad9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff66c102ef0 0x7ff66c19cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:54318/0 (socket says 172.21.15.164:54318) 2024-09-19T08:56:26.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.544+0000 7ff66ad9d640 1 -- 172.21.15.164:0/3117422603 learned_addr learned my addr 172.21.15.164:0/3117422603 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:26.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.544+0000 7ff663fff640 1 --2- 172.21.15.164:0/3117422603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff66c102570 0x7ff66c198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:26.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.544+0000 7ff66a59c640 1 --2- 172.21.15.164:0/3117422603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff66c100e20 0x7ff66c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:26.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.544+0000 7ff66ad9d640 1 -- 172.21.15.164:0/3117422603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff66c100e20 msgr2=0x7ff66c198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:26.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.544+0000 7ff66ad9d640 1 --2- 172.21.15.164:0/3117422603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff66c100e20 0x7ff66c198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:26.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.544+0000 7ff66ad9d640 1 -- 172.21.15.164:0/3117422603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff66c102570 msgr2=0x7ff66c198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:26.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.544+0000 7ff66ad9d640 1 --2- 172.21.15.164:0/3117422603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff66c102570 0x7ff66c198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:26.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.544+0000 7ff66ad9d640 1 -- 172.21.15.164:0/3117422603 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff66c19d6b0 con 0x7ff66c102ef0 2024-09-19T08:56:26.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.544+0000 7ff66a59c640 1 --2- 172.21.15.164:0/3117422603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff66c100e20 0x7ff66c198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:56:26.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.544+0000 7ff66ad9d640 1 --2- 172.21.15.164:0/3117422603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff66c102ef0 0x7ff66c19cfd0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7ff66400b970 tx=0x7ff66400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:26.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.544+0000 7ff6637fe640 1 -- 172.21.15.164:0/3117422603 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff664019070 con 0x7ff66c102ef0 2024-09-19T08:56:26.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.544+0000 7ff6637fe640 1 -- 172.21.15.164:0/3117422603 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff66400cb40 con 0x7ff66c102ef0 2024-09-19T08:56:26.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.544+0000 7ff67095e640 1 -- 172.21.15.164:0/3117422603 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff66c19d9a0 con 0x7ff66c102ef0 2024-09-19T08:56:26.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.544+0000 7ff6637fe640 1 -- 172.21.15.164:0/3117422603 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff664007650 con 0x7ff66c102ef0 2024-09-19T08:56:26.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.544+0000 7ff67095e640 1 -- 172.21.15.164:0/3117422603 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff66c1a5280 con 0x7ff66c102ef0 2024-09-19T08:56:26.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.545+0000 7ff67095e640 1 -- 172.21.15.164:0/3117422603 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff66c101280 con 0x7ff66c102ef0 2024-09-19T08:56:26.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.546+0000 7ff6637fe640 1 -- 172.21.15.164:0/3117422603 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7ff66401c070 con 0x7ff66c102ef0 2024-09-19T08:56:26.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.547+0000 7ff6637fe640 1 --2- 172.21.15.164:0/3117422603 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff6580a33b0 0x7ff6580a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:26.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.547+0000 7ff66a59c640 1 --2- 172.21.15.164:0/3117422603 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff6580a33b0 0x7ff6580a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:26.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.547+0000 7ff6637fe640 1 -- 172.21.15.164:0/3117422603 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7ff6640cf2d0 con 0x7ff66c102ef0 2024-09-19T08:56:26.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.548+0000 7ff66a59c640 1 --2- 172.21.15.164:0/3117422603 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff6580a33b0 0x7ff6580a5870 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff65c006fd0 tx=0x7ff65c008040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:26.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.551+0000 7ff6637fe640 1 -- 172.21.15.164:0/3117422603 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff66409bda0 con 0x7ff66c102ef0 2024-09-19T08:56:26.624 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:26 smithi170 ceph-mon[35072]: pgmap v331: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 13 op/s 2024-09-19T08:56:26.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:26 smithi173 ceph-mon[35306]: pgmap v331: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 13 op/s 2024-09-19T08:56:26.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.752+0000 7ff67095e640 1 -- 172.21.15.164:0/3117422603 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff66c10a0a0 con 0x7ff66c102ef0 2024-09-19T08:56:26.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.753+0000 7ff6637fe640 1 -- 172.21.15.164:0/3117422603 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11984 (secure 0 0 0) 0x7ff66409b6a0 con 0x7ff66c102ef0 2024-09-19T08:56:26.753 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:56:26.753 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":32,"btime":"2024-09-19T08:52:27:192804+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:52:27.074168+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:56:26.754 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 32 2024-09-19T08:56:26.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.755+0000 7ff67095e640 1 -- 172.21.15.164:0/3117422603 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff6580a33b0 msgr2=0x7ff6580a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:26.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.755+0000 7ff67095e640 1 --2- 172.21.15.164:0/3117422603 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff6580a33b0 0x7ff6580a5870 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff65c006fd0 tx=0x7ff65c008040 comp rx=0 tx=0).stop 2024-09-19T08:56:26.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.755+0000 7ff67095e640 1 -- 172.21.15.164:0/3117422603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff66c102ef0 msgr2=0x7ff66c19cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:26.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.755+0000 7ff67095e640 1 --2- 172.21.15.164:0/3117422603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff66c102ef0 0x7ff66c19cfd0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7ff66400b970 tx=0x7ff66400be30 comp rx=0 tx=0).stop 2024-09-19T08:56:26.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.755+0000 7ff67095e640 1 -- 172.21.15.164:0/3117422603 shutdown_connections 2024-09-19T08:56:26.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.755+0000 7ff67095e640 1 --2- 172.21.15.164:0/3117422603 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff6580a33b0 0x7ff6580a5870 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:26.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.755+0000 7ff67095e640 1 --2- 172.21.15.164:0/3117422603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff66c102ef0 0x7ff66c19cfd0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:26.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.755+0000 7ff67095e640 1 --2- 172.21.15.164:0/3117422603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff66c102570 0x7ff66c198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:26.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.755+0000 7ff67095e640 1 --2- 172.21.15.164:0/3117422603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff66c100e20 0x7ff66c198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:26.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.755+0000 7ff67095e640 1 -- 172.21.15.164:0/3117422603 >> 172.21.15.164:0/3117422603 conn(0x7ff66c0fc7d0 msgr2=0x7ff66c06a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:26.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.755+0000 7ff67095e640 1 -- 172.21.15.164:0/3117422603 shutdown_connections 2024-09-19T08:56:26.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:26.755+0000 7ff67095e640 1 -- 172.21.15.164:0/3117422603 wait complete. 2024-09-19T08:56:27.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:27 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3117422603' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:56:27.563 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id 60e31c5c --release --await' 2024-09-19T08:56:27.563 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id 60e31c5c --release --await 2024-09-19T08:56:27.624 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:27 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3117422603' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:56:27.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:27 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3117422603' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:56:27.812 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:56:28.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.292+0000 7fb71d4a8640 1 -- 172.21.15.164:0/2745539288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb718106fc0 msgr2=0x7fb71810eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:28.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.292+0000 7fb71d4a8640 1 --2- 172.21.15.164:0/2745539288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb718106fc0 0x7fb71810eaf0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fb710009930 tx=0x7fb71002a420 comp rx=0 tx=0).stop 2024-09-19T08:56:28.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.293+0000 7fb71d4a8640 1 -- 172.21.15.164:0/2745539288 shutdown_connections 2024-09-19T08:56:28.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.293+0000 7fb71d4a8640 1 --2- 172.21.15.164:0/2745539288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb718106fc0 0x7fb71810eaf0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:28.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.293+0000 7fb71d4a8640 1 --2- 172.21.15.164:0/2745539288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb718100d20 0x7fb718106860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:28.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.293+0000 7fb71d4a8640 1 --2- 172.21.15.164:0/2745539288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb718100370 0x7fb718100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:28.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.293+0000 7fb71d4a8640 1 -- 172.21.15.164:0/2745539288 >> 172.21.15.164:0/2745539288 conn(0x7fb718077210 msgr2=0x7fb7180fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:28.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.293+0000 7fb71d4a8640 1 -- 172.21.15.164:0/2745539288 shutdown_connections 2024-09-19T08:56:28.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.293+0000 7fb71d4a8640 1 -- 172.21.15.164:0/2745539288 wait complete. 2024-09-19T08:56:28.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.293+0000 7fb71d4a8640 1 Processor -- start 2024-09-19T08:56:28.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.293+0000 7fb71d4a8640 1 -- start start 2024-09-19T08:56:28.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.293+0000 7fb71d4a8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb718100370 0x7fb718198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:28.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.294+0000 7fb71d4a8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb718100d20 0x7fb7181989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:28.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.294+0000 7fb71d4a8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb718106fc0 0x7fb71819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:28.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.294+0000 7fb71d4a8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb718111fa0 con 0x7fb718100d20 2024-09-19T08:56:28.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.294+0000 7fb71d4a8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb718111e20 con 0x7fb718100370 2024-09-19T08:56:28.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.294+0000 7fb71d4a8640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb718112120 con 0x7fb718106fc0 2024-09-19T08:56:28.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.294+0000 7fb7167fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb718100d20 0x7fb7181989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:28.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.294+0000 7fb7167fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb718100d20 0x7fb7181989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:54336/0 (socket says 172.21.15.164:54336) 2024-09-19T08:56:28.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.294+0000 7fb7167fc640 1 -- 172.21.15.164:0/322247643 learned_addr learned my addr 172.21.15.164:0/322247643 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:28.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.294+0000 7fb716ffd640 1 --2- 172.21.15.164:0/322247643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb718100370 0x7fb718198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:28.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.294+0000 7fb7177fe640 1 --2- 172.21.15.164:0/322247643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb718106fc0 0x7fb71819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:28.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.294+0000 7fb7167fc640 1 -- 172.21.15.164:0/322247643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb718106fc0 msgr2=0x7fb71819cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:28.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.294+0000 7fb7167fc640 1 --2- 172.21.15.164:0/322247643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb718106fc0 0x7fb71819cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:28.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.294+0000 7fb7167fc640 1 -- 172.21.15.164:0/322247643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb718100370 msgr2=0x7fb718198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:28.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.294+0000 7fb7167fc640 1 --2- 172.21.15.164:0/322247643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb718100370 0x7fb718198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:28.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.294+0000 7fb7167fc640 1 -- 172.21.15.164:0/322247643 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71819d430 con 0x7fb718100d20 2024-09-19T08:56:28.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.294+0000 7fb716ffd640 1 --2- 172.21.15.164:0/322247643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb718100370 0x7fb718198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:56:28.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.294+0000 7fb7167fc640 1 --2- 172.21.15.164:0/322247643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb718100d20 0x7fb7181989c0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fb70c00c970 tx=0x7fb70c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:28.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.294+0000 7fb6fffff640 1 -- 172.21.15.164:0/322247643 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c007bb0 con 0x7fb718100d20 2024-09-19T08:56:28.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.294+0000 7fb71d4a8640 1 -- 172.21.15.164:0/322247643 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb71819d720 con 0x7fb718100d20 2024-09-19T08:56:28.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.294+0000 7fb6fffff640 1 -- 172.21.15.164:0/322247643 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb70c007d50 con 0x7fb718100d20 2024-09-19T08:56:28.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.294+0000 7fb6fffff640 1 -- 172.21.15.164:0/322247643 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c0187a0 con 0x7fb718100d20 2024-09-19T08:56:28.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.294+0000 7fb71d4a8640 1 -- 172.21.15.164:0/322247643 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7181a5000 con 0x7fb718100d20 2024-09-19T08:56:28.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.297+0000 7fb6fffff640 1 -- 172.21.15.164:0/322247643 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fb70c00f450 con 0x7fb718100d20 2024-09-19T08:56:28.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.297+0000 7fb71f881740 1 Processor -- start 2024-09-19T08:56:28.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.297+0000 7fb71f881740 1 -- start start 2024-09-19T08:56:28.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.298+0000 7fb71f881740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56334f672060 0x56334f672440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:28.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.298+0000 7fb71f881740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56334f672980 0x56334f67cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:28.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.298+0000 7fb71f881740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334f67d240 0x56334f67f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:28.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.298+0000 7fb71f881740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56334f5d7620 con 0x56334f672980 2024-09-19T08:56:28.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.298+0000 7fb71f881740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56334f57ee50 con 0x56334f67d240 2024-09-19T08:56:28.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.298+0000 7fb71f881740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56334f5d35e0 con 0x56334f672060 2024-09-19T08:56:28.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.298+0000 7fb716ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56334f672980 0x56334f67cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:28.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.298+0000 7fb716ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56334f672980 0x56334f67cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:54344/0 (socket says 172.21.15.164:54344) 2024-09-19T08:56:28.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.298+0000 7fb716ffd640 1 -- 172.21.15.164:0/3034102697 learned_addr learned my addr 172.21.15.164:0/3034102697 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:28.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.298+0000 7fb7177fe640 1 --2- 172.21.15.164:0/3034102697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56334f672060 0x56334f672440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:28.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.298+0000 7fb6fffff640 1 --2- 172.21.15.164:0/322247643 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb7000a33b0 0x7fb7000a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:28.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.298+0000 7fb7167fc640 1 --2- 172.21.15.164:0/3034102697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334f67d240 0x56334f67f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:28.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.298+0000 7fb716ffd640 1 -- 172.21.15.164:0/3034102697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56334f672060 msgr2=0x56334f672440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:28.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.298+0000 7fb716ffd640 1 --2- 172.21.15.164:0/3034102697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56334f672060 0x56334f672440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:28.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.298+0000 7fb7177fe640 1 --2- 172.21.15.164:0/322247643 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb7000a33b0 0x7fb7000a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:28.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.298+0000 7fb716ffd640 1 -- 172.21.15.164:0/3034102697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334f67d240 msgr2=0x56334f67f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:28.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.298+0000 7fb716ffd640 1 --2- 172.21.15.164:0/3034102697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334f67d240 0x56334f67f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:28.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.298+0000 7fb716ffd640 1 -- 172.21.15.164:0/3034102697 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56334f56fd40 con 0x56334f672980 2024-09-19T08:56:28.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.298+0000 7fb6fffff640 1 -- 172.21.15.164:0/322247643 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7fb70c0cfeb0 con 0x7fb718100d20 2024-09-19T08:56:28.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.298+0000 7fb716ffd640 1 --2- 172.21.15.164:0/3034102697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56334f672980 0x56334f67cd00 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fb708009f20 tx=0x7fb70800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcdc3cd30acd0107 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:28.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.299+0000 7fb6fd7fa640 1 -- 172.21.15.164:0/3034102697 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb708003b70 con 0x56334f672980 2024-09-19T08:56:28.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.299+0000 7fb6fd7fa640 1 -- 172.21.15.164:0/3034102697 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb708003d10 con 0x56334f672980 2024-09-19T08:56:28.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.299+0000 7fb6fd7fa640 1 -- 172.21.15.164:0/3034102697 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7080119a0 con 0x56334f672980 2024-09-19T08:56:28.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.299+0000 7fb71f881740 1 -- 172.21.15.164:0/3034102697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56334f672980 msgr2=0x56334f67cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:28.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.299+0000 7fb7177fe640 1 --2- 172.21.15.164:0/322247643 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb7000a33b0 0x7fb7000a5870 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb710009900 tx=0x7fb71002a420 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:28.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.299+0000 7fb71f881740 1 --2- 172.21.15.164:0/3034102697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56334f672980 0x56334f67cd00 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fb708009f20 tx=0x7fb70800f7f0 comp rx=0 tx=0).stop 2024-09-19T08:56:28.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.299+0000 7fb71f881740 1 -- 172.21.15.164:0/3034102697 shutdown_connections 2024-09-19T08:56:28.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.299+0000 7fb71f881740 1 --2- 172.21.15.164:0/3034102697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334f67d240 0x56334f67f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:28.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.299+0000 7fb71f881740 1 --2- 172.21.15.164:0/3034102697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56334f672980 0x56334f67cd00 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:28.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.299+0000 7fb71f881740 1 --2- 172.21.15.164:0/3034102697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56334f672060 0x56334f672440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:28.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.299+0000 7fb71f881740 1 -- 172.21.15.164:0/3034102697 >> 172.21.15.164:0/3034102697 conn(0x56334f66e5f0 msgr2=0x56334f66e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:28.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.299+0000 7fb71f881740 1 -- 172.21.15.164:0/3034102697 shutdown_connections 2024-09-19T08:56:28.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.299+0000 7fb71f881740 1 -- 172.21.15.164:0/3034102697 wait complete. 2024-09-19T08:56:28.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.300+0000 7fb71f881740 1 -- start start 2024-09-19T08:56:28.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.300+0000 7fb71f881740 1 Processor -- start 2024-09-19T08:56:28.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.300+0000 7fb71f881740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334f672060 0x56334f67a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:28.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.300+0000 7fb71f881740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56334f672980 0x56334f67a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:28.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.300+0000 7fb71f881740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56334f67d240 0x56334f67abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:28.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.300+0000 7fb71f881740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56334f67fe10 con 0x56334f672980 2024-09-19T08:56:28.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.300+0000 7fb71f881740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56334f55ff10 con 0x56334f672060 2024-09-19T08:56:28.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.300+0000 7fb71f881740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56334f67ff90 con 0x56334f67d240 2024-09-19T08:56:28.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.300+0000 7fb71f881740 10 client.0 _ll_register_callbacks cb 0x56334f583db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:56:28.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.300+0000 7fb7167fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56334f672980 0x56334f67a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:28.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.300+0000 7fb7167fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56334f672980 0x56334f67a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:54350/0 (socket says 172.21.15.164:54350) 2024-09-19T08:56:28.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.300+0000 7fb7167fc640 1 -- 172.21.15.164:0/4020605676 learned_addr learned my addr 172.21.15.164:0/4020605676 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:28.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.300+0000 7fb716ffd640 1 --2- 172.21.15.164:0/4020605676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334f672060 0x56334f67a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:28.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.300+0000 7fb7177fe640 1 --2- 172.21.15.164:0/4020605676 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56334f67d240 0x56334f67abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:28.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.300+0000 7fb7167fc640 1 -- 172.21.15.164:0/4020605676 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56334f67d240 msgr2=0x56334f67abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:28.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.300+0000 7fb7167fc640 1 --2- 172.21.15.164:0/4020605676 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56334f67d240 0x56334f67abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:28.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.300+0000 7fb7167fc640 1 -- 172.21.15.164:0/4020605676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334f672060 msgr2=0x56334f67a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:28.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.300+0000 7fb7167fc640 1 --2- 172.21.15.164:0/4020605676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334f672060 0x56334f67a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:28.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.300+0000 7fb7167fc640 1 -- 172.21.15.164:0/4020605676 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56334f2e6580 con 0x56334f672980 2024-09-19T08:56:28.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.300+0000 7fb7177fe640 1 --2- 172.21.15.164:0/4020605676 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56334f67d240 0x56334f67abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:56:28.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.301+0000 7fb7167fc640 1 --2- 172.21.15.164:0/4020605676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56334f672980 0x56334f67a6b0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fb70c09c210 tx=0x7fb70c09bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29695e50e3f3dc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:28.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.301+0000 7fb6faff5640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:56:28.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.301+0000 7fb6faff5640 1 -- 172.21.15.164:0/4020605676 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c0a4e40 con 0x56334f672980 2024-09-19T08:56:28.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.301+0000 7fb6faff5640 1 -- 172.21.15.164:0/4020605676 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7fb70c00f740 con 0x56334f672980 2024-09-19T08:56:28.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.301+0000 7fb71f881740 1 -- 172.21.15.164:0/4020605676 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb708003d10 con 0x56334f672980 2024-09-19T08:56:28.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.301+0000 7fb6faff5640 1 -- 172.21.15.164:0/4020605676 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb70c0988f0 con 0x56334f672980 2024-09-19T08:56:28.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.301+0000 7fb71f881740 10 client.17760 fetch_fsmap learned FSMap version 32 2024-09-19T08:56:28.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.301+0000 7fb71f881740 1 -- 172.21.15.164:0/4020605676 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56334f3b0960 con 0x56334f672980 2024-09-19T08:56:28.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.301+0000 7fb6faff5640 1 -- 172.21.15.164:0/4020605676 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb708003d10 con 0x56334f672980 2024-09-19T08:56:28.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.301+0000 7fb6faff5640 1 -- 172.21.15.164:0/4020605676 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb70c020030 con 0x56334f672980 2024-09-19T08:56:28.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.301+0000 7fb71f881740 10 client.17760 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:56:28.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.301+0000 7fb71f881740 10 client.17760 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:56:28.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.301+0000 7fb71f881740 20 client.17760 populate_metadata read hostname 'smithi164' 2024-09-19T08:56:28.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.301+0000 7fb71f881740 1 --2- 172.21.15.164:0/4020605676 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56334f68ce00 0x56334f68f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:28.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.301+0000 7fb71f881740 4 client.17760 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:56:28.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.301+0000 7fb71f881740 1 -- 172.21.15.164:0/4020605676 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56334f5e4b90 con 0x56334f68ce00 2024-09-19T08:56:28.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.302+0000 7fb716ffd640 1 --2- 172.21.15.164:0/4020605676 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56334f68ce00 0x56334f68f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:28.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.302+0000 7fb6faff5640 10 client.17760 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:56:28.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.302+0000 7fb716ffd640 1 --2- 172.21.15.164:0/4020605676 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56334f68ce00 0x56334f68f1f0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14dacdd7fe955f7b server_cookie=abd20b9526a75581 in_seq=0 out_seq=0 2024-09-19T08:56:28.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.303+0000 7fb6faff5640 1 -- 172.21.15.164:0/4020605676 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56334f5e4b90 con 0x56334f68ce00 2024-09-19T08:56:28.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.303+0000 7fb6faff5640 10 client.17760 handle_command_reply: tid=0 2024-09-19T08:56:28.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.303+0000 7fb71f881740 1 client.17760 shutdown 2024-09-19T08:56:28.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.303+0000 7fb71f881740 1 -- 172.21.15.164:0/4020605676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56334f672980 msgr2=0x56334f67a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:28.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.303+0000 7fb71f881740 1 --2- 172.21.15.164:0/4020605676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56334f672980 0x56334f67a6b0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fb70c09c210 tx=0x7fb70c09bbe0 comp rx=0 tx=0).stop 2024-09-19T08:56:28.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.303+0000 7fb71f881740 1 -- 172.21.15.164:0/4020605676 shutdown_connections 2024-09-19T08:56:28.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.303+0000 7fb71f881740 1 --2- 172.21.15.164:0/4020605676 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56334f67d240 0x56334f67abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:28.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.303+0000 7fb71f881740 1 --2- 172.21.15.164:0/4020605676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56334f672980 0x56334f67a6b0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:28.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.303+0000 7fb71f881740 1 --2- 172.21.15.164:0/4020605676 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56334f68ce00 0x56334f68f1f0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:28.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.303+0000 7fb71f881740 1 --2- 172.21.15.164:0/4020605676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334f672060 0x56334f67a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:28.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.303+0000 7fb71f881740 1 -- 172.21.15.164:0/4020605676 >> 172.21.15.164:0/4020605676 conn(0x56334f66e5f0 msgr2=0x56334f6742d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:28.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.303+0000 7fb6faff5640 0 client.17760 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:56:28.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.303+0000 7fb71f881740 1 -- 172.21.15.164:0/4020605676 shutdown_connections 2024-09-19T08:56:28.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.303+0000 7fb71f881740 1 -- 172.21.15.164:0/4020605676 wait complete. 2024-09-19T08:56:28.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.303+0000 7fb71f881740 20 client.17760 trim_cache size 0 max 16384 2024-09-19T08:56:28.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.320+0000 7fb71f881740 1 Processor -- start 2024-09-19T08:56:28.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.320+0000 7fb71f881740 1 -- start start 2024-09-19T08:56:28.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.320+0000 7fb71f881740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56334f672060 0x56334f678a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:28.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.320+0000 7fb71f881740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56334f672980 0x56334f6741c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:28.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.320+0000 7fb71f881740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334f67d240 0x56334f6949b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:28.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.320+0000 7fb71f881740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb7181ad850 con 0x56334f672060 2024-09-19T08:56:28.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.320+0000 7fb71f881740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb7181af390 con 0x56334f67d240 2024-09-19T08:56:28.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.320+0000 7fb71f881740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb7181adf20 con 0x56334f672980 2024-09-19T08:56:28.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.320+0000 7fb716ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56334f672060 0x56334f678a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:28.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.320+0000 7fb716ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56334f672060 0x56334f678a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:54366/0 (socket says 172.21.15.164:54366) 2024-09-19T08:56:28.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.321+0000 7fb716ffd640 1 -- 172.21.15.164:0/2455592185 learned_addr learned my addr 172.21.15.164:0/2455592185 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:28.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.321+0000 7fb7167fc640 1 --2- 172.21.15.164:0/2455592185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56334f672980 0x56334f6741c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:28.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.321+0000 7fb7177fe640 1 --2- 172.21.15.164:0/2455592185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334f67d240 0x56334f6949b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:28.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.321+0000 7fb716ffd640 1 -- 172.21.15.164:0/2455592185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56334f672980 msgr2=0x56334f6741c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:28.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.321+0000 7fb716ffd640 1 --2- 172.21.15.164:0/2455592185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56334f672980 0x56334f6741c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:28.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.321+0000 7fb716ffd640 1 -- 172.21.15.164:0/2455592185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334f67d240 msgr2=0x56334f6949b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:28.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.321+0000 7fb716ffd640 1 --2- 172.21.15.164:0/2455592185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334f67d240 0x56334f6949b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:28.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.321+0000 7fb716ffd640 1 -- 172.21.15.164:0/2455592185 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56334f67c0b0 con 0x56334f672060 2024-09-19T08:56:28.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.321+0000 7fb7167fc640 1 --2- 172.21.15.164:0/2455592185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56334f672980 0x56334f6741c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:56:28.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.321+0000 7fb716ffd640 1 --2- 172.21.15.164:0/2455592185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56334f672060 0x56334f678a40 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fb7080114b0 tx=0x7fb708009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e27acaee4d6ce5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:28.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.321+0000 7fb6fcff9640 1 -- 172.21.15.164:0/2455592185 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb708004c00 con 0x56334f672060 2024-09-19T08:56:28.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.321+0000 7fb6fcff9640 1 -- 172.21.15.164:0/2455592185 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb708004da0 con 0x56334f672060 2024-09-19T08:56:28.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.321+0000 7fb6fcff9640 1 -- 172.21.15.164:0/2455592185 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7080053c0 con 0x56334f672060 2024-09-19T08:56:28.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.322+0000 7fb71f881740 1 -- 172.21.15.164:0/2455592185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56334f672060 msgr2=0x56334f678a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:28.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.322+0000 7fb71f881740 1 --2- 172.21.15.164:0/2455592185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56334f672060 0x56334f678a40 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fb7080114b0 tx=0x7fb708009810 comp rx=0 tx=0).stop 2024-09-19T08:56:28.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.322+0000 7fb71f881740 1 -- 172.21.15.164:0/2455592185 shutdown_connections 2024-09-19T08:56:28.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.322+0000 7fb71f881740 1 --2- 172.21.15.164:0/2455592185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334f67d240 0x56334f6949b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:28.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.322+0000 7fb71f881740 1 --2- 172.21.15.164:0/2455592185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56334f672980 0x56334f6741c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:28.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.322+0000 7fb71f881740 1 --2- 172.21.15.164:0/2455592185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56334f672060 0x56334f678a40 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:28.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.322+0000 7fb71f881740 1 -- 172.21.15.164:0/2455592185 >> 172.21.15.164:0/2455592185 conn(0x56334f66e5f0 msgr2=0x56334f673ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:28.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.322+0000 7fb71f881740 1 -- 172.21.15.164:0/2455592185 shutdown_connections 2024-09-19T08:56:28.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.322+0000 7fb71f881740 1 -- 172.21.15.164:0/2455592185 wait complete. 2024-09-19T08:56:28.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.322+0000 7fb71f881740 1 -- start start 2024-09-19T08:56:28.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.322+0000 7fb71f881740 1 Processor -- start 2024-09-19T08:56:28.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.323+0000 7fb71f881740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334f672060 0x56334f6f0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:28.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.323+0000 7fb71f881740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56334f672980 0x56334f6f11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:28.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.323+0000 7fb71f881740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56334f67d240 0x56334f6817a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:28.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.323+0000 7fb71f881740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb7181aecc0 con 0x56334f67d240 2024-09-19T08:56:28.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.323+0000 7fb71f881740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb7181ae5f0 con 0x56334f672060 2024-09-19T08:56:28.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.323+0000 7fb71f881740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb7181ab850 con 0x56334f672980 2024-09-19T08:56:28.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.323+0000 7fb71f881740 10 client.0 _ll_register_callbacks cb 0x7fb70c00a480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:56:28.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.323+0000 7fb7177fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56334f67d240 0x56334f6817a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:28.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.323+0000 7fb7177fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56334f67d240 0x56334f6817a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:54380/0 (socket says 172.21.15.164:54380) 2024-09-19T08:56:28.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.323+0000 7fb7177fe640 1 -- 172.21.15.164:0/1846223201 learned_addr learned my addr 172.21.15.164:0/1846223201 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:28.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.323+0000 7fb7167fc640 1 --2- 172.21.15.164:0/1846223201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56334f672980 0x56334f6f11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:28.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.323+0000 7fb716ffd640 1 --2- 172.21.15.164:0/1846223201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334f672060 0x56334f6f0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:28.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.323+0000 7fb7177fe640 1 -- 172.21.15.164:0/1846223201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56334f672980 msgr2=0x56334f6f11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:28.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.323+0000 7fb7177fe640 1 --2- 172.21.15.164:0/1846223201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56334f672980 0x56334f6f11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:28.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.323+0000 7fb7177fe640 1 -- 172.21.15.164:0/1846223201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334f672060 msgr2=0x56334f6f0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:28.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.323+0000 7fb7177fe640 1 --2- 172.21.15.164:0/1846223201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334f672060 0x56334f6f0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:28.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.323+0000 7fb7177fe640 1 -- 172.21.15.164:0/1846223201 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56334f679120 con 0x56334f67d240 2024-09-19T08:56:28.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.323+0000 7fb716ffd640 1 --2- 172.21.15.164:0/1846223201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334f672060 0x56334f6f0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:56:28.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.323+0000 7fb7177fe640 1 --2- 172.21.15.164:0/1846223201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56334f67d240 0x56334f6817a0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fb71002a940 tx=0x7fb710009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec9873a66983f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:28.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.323+0000 7fb6faff5640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:56:28.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.323+0000 7fb6faff5640 1 -- 172.21.15.164:0/1846223201 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71004e070 con 0x56334f67d240 2024-09-19T08:56:28.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.323+0000 7fb6faff5640 1 -- 172.21.15.164:0/1846223201 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7fb7100046c0 con 0x56334f67d240 2024-09-19T08:56:28.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.323+0000 7fb71f881740 1 -- 172.21.15.164:0/1846223201 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb708004da0 con 0x56334f67d240 2024-09-19T08:56:28.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.324+0000 7fb6faff5640 1 -- 172.21.15.164:0/1846223201 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb71005a050 con 0x56334f67d240 2024-09-19T08:56:28.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.324+0000 7fb71f881740 10 client.17766 fetch_fsmap learned FSMap version 32 2024-09-19T08:56:28.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.324+0000 7fb71f881740 1 -- 172.21.15.164:0/1846223201 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56334f6727b0 con 0x56334f67d240 2024-09-19T08:56:28.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.324+0000 7fb6faff5640 1 -- 172.21.15.164:0/1846223201 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb708004da0 con 0x56334f67d240 2024-09-19T08:56:28.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.324+0000 7fb6faff5640 1 -- 172.21.15.164:0/1846223201 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb71003a040 con 0x56334f67d240 2024-09-19T08:56:28.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.324+0000 7fb71f881740 10 client.17766 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:56:28.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.324+0000 7fb71f881740 10 client.17766 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:56:28.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.324+0000 7fb71f881740 20 client.17766 populate_metadata read hostname 'smithi164' 2024-09-19T08:56:28.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.324+0000 7fb71f881740 1 --2- 172.21.15.164:0/1846223201 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56334f6f8890 0x56334f6facb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:28.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.324+0000 7fb71f881740 4 client.17766 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "60e31c5c", "release": true, "await": true}] 2024-09-19T08:56:28.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.324+0000 7fb71f881740 1 -- 172.21.15.164:0/1846223201 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "quiesce db", "set_id": "60e31c5c", "release": true, "await": true}) -- 0x56334f5df0a0 con 0x56334f6f8890 2024-09-19T08:56:28.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.324+0000 7fb716ffd640 1 --2- 172.21.15.164:0/1846223201 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56334f6f8890 0x56334f6facb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:28.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.325+0000 7fb716ffd640 1 --2- 172.21.15.164:0/1846223201 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56334f6f8890 0x56334f6facb0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=353d35cf363b7664 server_cookie=f109b7afa5d1b09c in_seq=0 out_seq=0 2024-09-19T08:56:28.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.325+0000 7fb6faff5640 10 client.17766 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:56:28.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.328+0000 7fb6faff5640 1 -- 172.21.15.164:0/1846223201 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56334f5df0a0 con 0x56334f6f8890 2024-09-19T08:56:28.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.328+0000 7fb6faff5640 10 client.17766 handle_command_reply: tid=0 2024-09-19T08:56:28.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.328+0000 7fb71f881740 1 client.17766 shutdown 2024-09-19T08:56:28.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.328+0000 7fb71f881740 1 -- 172.21.15.164:0/1846223201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56334f67d240 msgr2=0x56334f6817a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:28.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.328+0000 7fb71f881740 1 --2- 172.21.15.164:0/1846223201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56334f67d240 0x56334f6817a0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fb71002a940 tx=0x7fb710009260 comp rx=0 tx=0).stop 2024-09-19T08:56:28.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.328+0000 7fb71f881740 1 -- 172.21.15.164:0/1846223201 shutdown_connections 2024-09-19T08:56:28.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.328+0000 7fb71f881740 1 --2- 172.21.15.164:0/1846223201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56334f67d240 0x56334f6817a0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:28.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.328+0000 7fb71f881740 1 --2- 172.21.15.164:0/1846223201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56334f672980 0x56334f6f11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:28.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.328+0000 7fb71f881740 1 --2- 172.21.15.164:0/1846223201 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x56334f6f8890 0x56334f6facb0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:28.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.328+0000 7fb6faff5640 0 client.17766 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:56:28.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.328+0000 7fb71f881740 1 --2- 172.21.15.164:0/1846223201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334f672060 0x56334f6f0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:28.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.328+0000 7fb71f881740 1 -- 172.21.15.164:0/1846223201 >> 172.21.15.164:0/1846223201 conn(0x56334f66e5f0 msgr2=0x56334f66d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:28.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.328+0000 7fb71f881740 1 -- 172.21.15.164:0/1846223201 shutdown_connections 2024-09-19T08:56:28.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.328+0000 7fb71f881740 1 -- 172.21.15.164:0/1846223201 wait complete. 2024-09-19T08:56:28.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.328+0000 7fb71f881740 20 client.17766 trim_cache size 0 max 16384 2024-09-19T08:56:28.332 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:56:28.332 INFO:teuthology.orchestra.run.smithi164.stdout: "epoch": 32, 2024-09-19T08:56:28.332 INFO:teuthology.orchestra.run.smithi164.stdout: "leader": 24479, 2024-09-19T08:56:28.332 INFO:teuthology.orchestra.run.smithi164.stdout: "set_version": 36, 2024-09-19T08:56:28.332 INFO:teuthology.orchestra.run.smithi164.stdout: "sets": { 2024-09-19T08:56:28.332 INFO:teuthology.orchestra.run.smithi164.stdout: "60e31c5c": { 2024-09-19T08:56:28.332 INFO:teuthology.orchestra.run.smithi164.stdout: "version": 36, 2024-09-19T08:56:28.333 INFO:teuthology.orchestra.run.smithi164.stdout: "age_ref": 0.0, 2024-09-19T08:56:28.333 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:56:28.333 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "RELEASED", 2024-09-19T08:56:28.333 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:56:28.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:56:28.333 INFO:teuthology.orchestra.run.smithi164.stdout: "timeout": 90.0, 2024-09-19T08:56:28.333 INFO:teuthology.orchestra.run.smithi164.stdout: "expiration": 150.0, 2024-09-19T08:56:28.333 INFO:teuthology.orchestra.run.smithi164.stdout: "members": { 2024-09-19T08:56:28.333 INFO:teuthology.orchestra.run.smithi164.stdout: "file:/": { 2024-09-19T08:56:28.333 INFO:teuthology.orchestra.run.smithi164.stdout: "excluded": false, 2024-09-19T08:56:28.333 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:56:28.333 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "RELEASED", 2024-09-19T08:56:28.333 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:56:28.334 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:56:28.334 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:56:28.334 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:56:28.334 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:56:28.334 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:56:28.334 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:56:28.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.329+0000 7fb71d4a8640 1 -- 172.21.15.164:0/322247643 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb7000a33b0 msgr2=0x7fb7000a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:28.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.329+0000 7fb71d4a8640 1 --2- 172.21.15.164:0/322247643 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb7000a33b0 0x7fb7000a5870 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb710009900 tx=0x7fb71002a420 comp rx=0 tx=0).stop 2024-09-19T08:56:28.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.329+0000 7fb71d4a8640 1 -- 172.21.15.164:0/322247643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb718100d20 msgr2=0x7fb7181989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:28.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.329+0000 7fb71d4a8640 1 --2- 172.21.15.164:0/322247643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb718100d20 0x7fb7181989c0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fb70c00c970 tx=0x7fb70c00ce30 comp rx=0 tx=0).stop 2024-09-19T08:56:28.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.329+0000 7fb71d4a8640 1 -- 172.21.15.164:0/322247643 shutdown_connections 2024-09-19T08:56:28.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.329+0000 7fb71d4a8640 1 --2- 172.21.15.164:0/322247643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb718106fc0 0x7fb71819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:28.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.329+0000 7fb71d4a8640 1 --2- 172.21.15.164:0/322247643 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb7000a33b0 0x7fb7000a5870 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:28.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.329+0000 7fb71d4a8640 1 --2- 172.21.15.164:0/322247643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb718100d20 0x7fb7181989c0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:28.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.329+0000 7fb71d4a8640 1 --2- 172.21.15.164:0/322247643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb718100370 0x7fb718198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:28.335 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.329+0000 7fb71d4a8640 1 -- 172.21.15.164:0/322247643 >> 172.21.15.164:0/322247643 conn(0x7fb718077210 msgr2=0x7fb7180fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:28.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.329+0000 7fb71d4a8640 1 -- 172.21.15.164:0/322247643 shutdown_connections 2024-09-19T08:56:28.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:28.329+0000 7fb71d4a8640 1 -- 172.21.15.164:0/322247643 wait complete. 2024-09-19T08:56:28.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:28 smithi164 ceph-mon[30928]: pgmap v332: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 13 op/s 2024-09-19T08:56:28.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:28 smithi170 ceph-mon[35072]: pgmap v332: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 13 op/s 2024-09-19T08:56:28.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:28 smithi173 ceph-mon[35306]: pgmap v332: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 13 op/s 2024-09-19T08:56:29.122 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '60e31c5c', total seconds elapsed: 33.3 2024-09-19T08:56:29.123 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-19T08:56:29.123 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:56:29.371 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:56:29.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.843+0000 7f56f2a62640 1 -- 172.21.15.164:0/1968448304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56ec112640 msgr2=0x7f56ec112a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:29.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.843+0000 7f56f2a62640 1 --2- 172.21.15.164:0/1968448304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56ec112640 0x7f56ec112a20 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f56dc009900 tx=0x7f56dc02a410 comp rx=0 tx=0).stop 2024-09-19T08:56:29.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.843+0000 7f56f2a62640 1 -- 172.21.15.164:0/1968448304 shutdown_connections 2024-09-19T08:56:29.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.843+0000 7f56f2a62640 1 --2- 172.21.15.164:0/1968448304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56ec10cf30 0x7f56ec119410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:29.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.843+0000 7f56f2a62640 1 --2- 172.21.15.164:0/1968448304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56ec10c590 0x7f56ec10c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:29.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.843+0000 7f56f2a62640 1 --2- 172.21.15.164:0/1968448304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56ec112640 0x7f56ec112a20 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:29.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.843+0000 7f56f2a62640 1 -- 172.21.15.164:0/1968448304 >> 172.21.15.164:0/1968448304 conn(0x7f56ec06cdc0 msgr2=0x7f56ec06d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:29.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.843+0000 7f56f2a62640 1 -- 172.21.15.164:0/1968448304 shutdown_connections 2024-09-19T08:56:29.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.843+0000 7f56f2a62640 1 -- 172.21.15.164:0/1968448304 wait complete. 2024-09-19T08:56:29.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.843+0000 7f56f2a62640 1 Processor -- start 2024-09-19T08:56:29.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.843+0000 7f56f2a62640 1 -- start start 2024-09-19T08:56:29.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.844+0000 7f56f2a62640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56ec10c590 0x7f56ec1a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:29.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.844+0000 7f56f2a62640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56ec10cf30 0x7f56ec1a3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:29.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.844+0000 7f56f2a62640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56ec112640 0x7f56ec1a76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:29.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.844+0000 7f56f2a62640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec11c9a0 con 0x7f56ec112640 2024-09-19T08:56:29.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.844+0000 7f56f2a62640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec11c820 con 0x7f56ec10c590 2024-09-19T08:56:29.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.844+0000 7f56f2a62640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec11cb20 con 0x7f56ec10cf30 2024-09-19T08:56:29.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.844+0000 7f56f0fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56ec112640 0x7f56ec1a76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:29.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.844+0000 7f56f0fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56ec112640 0x7f56ec1a76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:54406/0 (socket says 172.21.15.164:54406) 2024-09-19T08:56:29.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.844+0000 7f56f0fff640 1 -- 172.21.15.164:0/836900591 learned_addr learned my addr 172.21.15.164:0/836900591 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:29.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.844+0000 7f56ebfff640 1 --2- 172.21.15.164:0/836900591 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56ec10c590 0x7f56ec1a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:29.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.844+0000 7f56eb7fe640 1 --2- 172.21.15.164:0/836900591 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56ec10cf30 0x7f56ec1a3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:29.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.844+0000 7f56f0fff640 1 -- 172.21.15.164:0/836900591 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56ec10cf30 msgr2=0x7f56ec1a3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:29.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.844+0000 7f56f0fff640 1 --2- 172.21.15.164:0/836900591 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56ec10cf30 0x7f56ec1a3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:29.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.844+0000 7f56f0fff640 1 -- 172.21.15.164:0/836900591 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56ec10c590 msgr2=0x7f56ec1a2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:29.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.844+0000 7f56f0fff640 1 --2- 172.21.15.164:0/836900591 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56ec10c590 0x7f56ec1a2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:29.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.844+0000 7f56f0fff640 1 -- 172.21.15.164:0/836900591 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56ec1a7dc0 con 0x7f56ec112640 2024-09-19T08:56:29.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.844+0000 7f56f0fff640 1 --2- 172.21.15.164:0/836900591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56ec112640 0x7f56ec1a76e0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f56e400eaa0 tx=0x7f56e400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:29.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.845+0000 7f56e97fa640 1 -- 172.21.15.164:0/836900591 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e400cdb0 con 0x7f56ec112640 2024-09-19T08:56:29.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.845+0000 7f56e97fa640 1 -- 172.21.15.164:0/836900591 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56e4005020 con 0x7f56ec112640 2024-09-19T08:56:29.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.845+0000 7f56f2a62640 1 -- 172.21.15.164:0/836900591 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56ec1a80b0 con 0x7f56ec112640 2024-09-19T08:56:29.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.845+0000 7f56e97fa640 1 -- 172.21.15.164:0/836900591 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e40116b0 con 0x7f56ec112640 2024-09-19T08:56:29.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.845+0000 7f56f2a62640 1 -- 172.21.15.164:0/836900591 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56ec1af990 con 0x7f56ec112640 2024-09-19T08:56:29.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.846+0000 7f56f2a62640 1 -- 172.21.15.164:0/836900591 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56ec10c9f0 con 0x7f56ec112640 2024-09-19T08:56:29.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.847+0000 7f56e97fa640 1 -- 172.21.15.164:0/836900591 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f56e4005560 con 0x7f56ec112640 2024-09-19T08:56:29.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.847+0000 7f56e97fa640 1 --2- 172.21.15.164:0/836900591 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f56d40a33b0 0x7f56d40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:29.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.848+0000 7f56ebfff640 1 --2- 172.21.15.164:0/836900591 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f56d40a33b0 0x7f56d40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:29.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.848+0000 7f56e97fa640 1 -- 172.21.15.164:0/836900591 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f56e40d0380 con 0x7f56ec112640 2024-09-19T08:56:29.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.848+0000 7f56ebfff640 1 --2- 172.21.15.164:0/836900591 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f56d40a33b0 0x7f56d40a5870 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f56dc009230 tx=0x7f56dc0058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:29.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:29.851+0000 7f56e97fa640 1 -- 172.21.15.164:0/836900591 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f56e4016030 con 0x7f56ec112640 2024-09-19T08:56:30.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:30.053+0000 7f56f2a62640 1 -- 172.21.15.164:0/836900591 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56ec02d7c0 con 0x7f56ec112640 2024-09-19T08:56:30.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:30.054+0000 7f56e97fa640 1 -- 172.21.15.164:0/836900591 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11984 (secure 0 0 0) 0x7f56e4098020 con 0x7f56ec112640 2024-09-19T08:56:30.053 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:56:30.054 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":32,"btime":"2024-09-19T08:52:27:192804+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:52:27.074168+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:56:30.054 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 32 2024-09-19T08:56:30.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:30.056+0000 7f56f2a62640 1 -- 172.21.15.164:0/836900591 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f56d40a33b0 msgr2=0x7f56d40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:30.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:30.056+0000 7f56f2a62640 1 --2- 172.21.15.164:0/836900591 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f56d40a33b0 0x7f56d40a5870 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f56dc009230 tx=0x7f56dc0058d0 comp rx=0 tx=0).stop 2024-09-19T08:56:30.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:30.056+0000 7f56f2a62640 1 -- 172.21.15.164:0/836900591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56ec112640 msgr2=0x7f56ec1a76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:30.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:30.056+0000 7f56f2a62640 1 --2- 172.21.15.164:0/836900591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56ec112640 0x7f56ec1a76e0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f56e400eaa0 tx=0x7f56e400ef60 comp rx=0 tx=0).stop 2024-09-19T08:56:30.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:30.056+0000 7f56f2a62640 1 -- 172.21.15.164:0/836900591 shutdown_connections 2024-09-19T08:56:30.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:30.056+0000 7f56f2a62640 1 --2- 172.21.15.164:0/836900591 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f56d40a33b0 0x7f56d40a5870 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:30.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:30.056+0000 7f56f2a62640 1 --2- 172.21.15.164:0/836900591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56ec112640 0x7f56ec1a76e0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:30.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:30.056+0000 7f56f2a62640 1 --2- 172.21.15.164:0/836900591 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56ec10cf30 0x7f56ec1a3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:30.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:30.056+0000 7f56f2a62640 1 --2- 172.21.15.164:0/836900591 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56ec10c590 0x7f56ec1a2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:30.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:30.056+0000 7f56f2a62640 1 -- 172.21.15.164:0/836900591 >> 172.21.15.164:0/836900591 conn(0x7f56ec06cdc0 msgr2=0x7f56ec10af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:30.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:30.056+0000 7f56f2a62640 1 -- 172.21.15.164:0/836900591 shutdown_connections 2024-09-19T08:56:30.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:30.056+0000 7f56f2a62640 1 -- 172.21.15.164:0/836900591 wait complete. 2024-09-19T08:56:30.524 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:30 smithi164 ceph-mon[30928]: pgmap v333: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 13 op/s 2024-09-19T08:56:30.524 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:30 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/836900591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:56:30.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:30 smithi170 ceph-mon[35072]: pgmap v333: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 13 op/s 2024-09-19T08:56:30.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:30 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/836900591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:56:30.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:30 smithi173 ceph-mon[35306]: pgmap v333: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 13 op/s 2024-09-19T08:56:30.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:30 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/836900591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:56:30.971 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-19T08:56:30.971 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-19T08:56:31.219 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:56:31.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.705+0000 7f7e4e0a7640 1 -- 172.21.15.164:0/609705044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e48102ef0 msgr2=0x7f7e4810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:31.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.705+0000 7f7e4e0a7640 1 --2- 172.21.15.164:0/609705044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e48102ef0 0x7f7e4810da40 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f7e40009900 tx=0x7f7e4002a480 comp rx=0 tx=0).stop 2024-09-19T08:56:31.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.706+0000 7f7e4e0a7640 1 -- 172.21.15.164:0/609705044 shutdown_connections 2024-09-19T08:56:31.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.706+0000 7f7e4e0a7640 1 --2- 172.21.15.164:0/609705044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e48102ef0 0x7f7e4810da40 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.706+0000 7f7e4e0a7640 1 --2- 172.21.15.164:0/609705044 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e48102570 0x7f7e481029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.706+0000 7f7e4e0a7640 1 --2- 172.21.15.164:0/609705044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e48100e20 0x7f7e48101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.706+0000 7f7e4e0a7640 1 -- 172.21.15.164:0/609705044 >> 172.21.15.164:0/609705044 conn(0x7f7e480fc7d0 msgr2=0x7f7e480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:31.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.706+0000 7f7e4e0a7640 1 -- 172.21.15.164:0/609705044 shutdown_connections 2024-09-19T08:56:31.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.706+0000 7f7e4e0a7640 1 -- 172.21.15.164:0/609705044 wait complete. 2024-09-19T08:56:31.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.706+0000 7f7e4e0a7640 1 Processor -- start 2024-09-19T08:56:31.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.706+0000 7f7e4e0a7640 1 -- start start 2024-09-19T08:56:31.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.707+0000 7f7e4e0a7640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e48100e20 0x7f7e481983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:31.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.707+0000 7f7e4e0a7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e48102570 0x7f7e481988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:31.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.707+0000 7f7e4e0a7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e48102ef0 0x7f7e4819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:31.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.707+0000 7f7e4e0a7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7e48110f80 con 0x7f7e48102ef0 2024-09-19T08:56:31.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.707+0000 7f7e4e0a7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7e48110e00 con 0x7f7e48102570 2024-09-19T08:56:31.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.707+0000 7f7e4e0a7640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7e48111100 con 0x7f7e48100e20 2024-09-19T08:56:31.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.707+0000 7f7e47fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e48102ef0 0x7f7e4819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:31.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.707+0000 7f7e47fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e48102ef0 0x7f7e4819cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:54432/0 (socket says 172.21.15.164:54432) 2024-09-19T08:56:31.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.707+0000 7f7e47fff640 1 -- 172.21.15.164:0/708333254 learned_addr learned my addr 172.21.15.164:0/708333254 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:31.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.707+0000 7f7e46ffd640 1 --2- 172.21.15.164:0/708333254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e48102570 0x7f7e481988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:31.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.707+0000 7f7e477fe640 1 --2- 172.21.15.164:0/708333254 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e48100e20 0x7f7e481983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:31.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.707+0000 7f7e47fff640 1 -- 172.21.15.164:0/708333254 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e48100e20 msgr2=0x7f7e481983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:31.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.707+0000 7f7e47fff640 1 --2- 172.21.15.164:0/708333254 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e48100e20 0x7f7e481983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.707+0000 7f7e47fff640 1 -- 172.21.15.164:0/708333254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e48102570 msgr2=0x7f7e481988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:31.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.707+0000 7f7e47fff640 1 --2- 172.21.15.164:0/708333254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e48102570 0x7f7e481988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.707+0000 7f7e47fff640 1 -- 172.21.15.164:0/708333254 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e4819d360 con 0x7f7e48102ef0 2024-09-19T08:56:31.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.707+0000 7f7e477fe640 1 --2- 172.21.15.164:0/708333254 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e48100e20 0x7f7e481983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:56:31.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.707+0000 7f7e47fff640 1 --2- 172.21.15.164:0/708333254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e48102ef0 0x7f7e4819cc80 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f7e40005ab0 tx=0x7f7e40003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:31.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.707+0000 7f7e44ff9640 1 -- 172.21.15.164:0/708333254 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4003f070 con 0x7f7e48102ef0 2024-09-19T08:56:31.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.707+0000 7f7e44ff9640 1 -- 172.21.15.164:0/708333254 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e40031e30 con 0x7f7e48102ef0 2024-09-19T08:56:31.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.707+0000 7f7e4e0a7640 1 -- 172.21.15.164:0/708333254 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e4819d5f0 con 0x7f7e48102ef0 2024-09-19T08:56:31.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.707+0000 7f7e44ff9640 1 -- 172.21.15.164:0/708333254 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e40035030 con 0x7f7e48102ef0 2024-09-19T08:56:31.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.707+0000 7f7e4e0a7640 1 -- 172.21.15.164:0/708333254 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e481a4f30 con 0x7f7e48102ef0 2024-09-19T08:56:31.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.710+0000 7f7e50480740 1 Processor -- start 2024-09-19T08:56:31.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.710+0000 7f7e44ff9640 1 -- 172.21.15.164:0/708333254 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f7e40042070 con 0x7f7e48102ef0 2024-09-19T08:56:31.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.710+0000 7f7e50480740 1 -- start start 2024-09-19T08:56:31.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.711+0000 7f7e50480740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6a31f2760 0x55b6a31f2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:31.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.711+0000 7f7e50480740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6a31f3080 0x55b6a31fd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:31.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.711+0000 7f7e50480740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a31fd940 0x55b6a31ffd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:31.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.711+0000 7f7e50480740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b6a31582a0 con 0x55b6a31f2760 2024-09-19T08:56:31.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.711+0000 7f7e50480740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b6a30ffb10 con 0x55b6a31fd940 2024-09-19T08:56:31.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.711+0000 7f7e50480740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b6a3154260 con 0x55b6a31f3080 2024-09-19T08:56:31.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.711+0000 7f7e46ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6a31f2760 0x55b6a31f2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:31.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.711+0000 7f7e46ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6a31f2760 0x55b6a31f2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:54436/0 (socket says 172.21.15.164:54436) 2024-09-19T08:56:31.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.711+0000 7f7e46ffd640 1 -- 172.21.15.164:0/4110981578 learned_addr learned my addr 172.21.15.164:0/4110981578 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:31.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.711+0000 7f7e477fe640 1 --2- 172.21.15.164:0/4110981578 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6a31f3080 0x55b6a31fd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:31.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.711+0000 7f7e46ffd640 1 -- 172.21.15.164:0/4110981578 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6a31f3080 msgr2=0x55b6a31fd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:31.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.711+0000 7f7e46ffd640 1 --2- 172.21.15.164:0/4110981578 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6a31f3080 0x55b6a31fd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.711+0000 7f7e46ffd640 1 -- 172.21.15.164:0/4110981578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a31fd940 msgr2=0x55b6a31ffd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:31.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.711+0000 7f7e46ffd640 1 --2- 172.21.15.164:0/4110981578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a31fd940 0x55b6a31ffd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.711+0000 7f7e46ffd640 1 -- 172.21.15.164:0/4110981578 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6a2ec9b90 con 0x55b6a31f2760 2024-09-19T08:56:31.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.711+0000 7f7e44ff9640 1 --2- 172.21.15.164:0/708333254 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f7e300a33b0 0x7f7e300a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:31.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.711+0000 7f7e46ffd640 1 --2- 172.21.15.164:0/4110981578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6a31f2760 0x55b6a31f2b40 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c00cca0 tx=0x7f7e3c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4cb3d37a33f183 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:31.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.711+0000 7f7e477fe640 1 --2- 172.21.15.164:0/708333254 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f7e300a33b0 0x7f7e300a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:31.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.711+0000 7f7e44ff9640 1 -- 172.21.15.164:0/708333254 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f7e400f0650 con 0x7f7e48102ef0 2024-09-19T08:56:31.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.711+0000 7f7e2dffb640 1 -- 172.21.15.164:0/4110981578 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c016070 con 0x55b6a31f2760 2024-09-19T08:56:31.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.711+0000 7f7e2dffb640 1 -- 172.21.15.164:0/4110981578 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e3c0047a0 con 0x55b6a31f2760 2024-09-19T08:56:31.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.711+0000 7f7e2dffb640 1 -- 172.21.15.164:0/4110981578 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c0128c0 con 0x55b6a31f2760 2024-09-19T08:56:31.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.712+0000 7f7e50480740 1 -- 172.21.15.164:0/4110981578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6a31f2760 msgr2=0x55b6a31f2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:31.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.712+0000 7f7e477fe640 1 --2- 172.21.15.164:0/708333254 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f7e300a33b0 0x7f7e300a5870 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x55b6a31efb50 tx=0x7f7e38008510 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:31.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.712+0000 7f7e50480740 1 --2- 172.21.15.164:0/4110981578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6a31f2760 0x55b6a31f2b40 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c00cca0 tx=0x7f7e3c010740 comp rx=0 tx=0).stop 2024-09-19T08:56:31.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.712+0000 7f7e50480740 1 -- 172.21.15.164:0/4110981578 shutdown_connections 2024-09-19T08:56:31.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.712+0000 7f7e50480740 1 --2- 172.21.15.164:0/4110981578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a31fd940 0x55b6a31ffd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.712+0000 7f7e50480740 1 --2- 172.21.15.164:0/4110981578 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6a31f3080 0x55b6a31fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.712+0000 7f7e50480740 1 --2- 172.21.15.164:0/4110981578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6a31f2760 0x55b6a31f2b40 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.712+0000 7f7e50480740 1 -- 172.21.15.164:0/4110981578 >> 172.21.15.164:0/4110981578 conn(0x55b6a31eecf0 msgr2=0x55b6a31ef0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:31.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.712+0000 7f7e50480740 1 -- 172.21.15.164:0/4110981578 shutdown_connections 2024-09-19T08:56:31.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.712+0000 7f7e50480740 1 -- 172.21.15.164:0/4110981578 wait complete. 2024-09-19T08:56:31.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.713+0000 7f7e50480740 1 -- start start 2024-09-19T08:56:31.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.713+0000 7f7e50480740 1 Processor -- start 2024-09-19T08:56:31.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.713+0000 7f7e50480740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6a31f2760 0x55b6a31fa900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:31.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.713+0000 7f7e50480740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6a31f3080 0x55b6a31fae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:31.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.713+0000 7f7e50480740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a31fd940 0x55b6a31fb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:31.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.713+0000 7f7e50480740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b6a3204830 con 0x55b6a31f3080 2024-09-19T08:56:31.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.713+0000 7f7e50480740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b6a30e06e0 con 0x55b6a31fd940 2024-09-19T08:56:31.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.713+0000 7f7e50480740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b6a32049b0 con 0x55b6a31f2760 2024-09-19T08:56:31.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.713+0000 7f7e50480740 10 client.0 _ll_register_callbacks cb 0x55b6a3104a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:56:31.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.713+0000 7f7e46ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6a31f3080 0x55b6a31fae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:31.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.713+0000 7f7e46ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6a31f3080 0x55b6a31fae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:54438/0 (socket says 172.21.15.164:54438) 2024-09-19T08:56:31.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.713+0000 7f7e46ffd640 1 -- 172.21.15.164:0/3779817820 learned_addr learned my addr 172.21.15.164:0/3779817820 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:31.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.713+0000 7f7e47fff640 1 --2- 172.21.15.164:0/3779817820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a31fd940 0x55b6a31fb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:31.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.713+0000 7f7e477fe640 1 --2- 172.21.15.164:0/3779817820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6a31f2760 0x55b6a31fa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:31.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.713+0000 7f7e46ffd640 1 -- 172.21.15.164:0/3779817820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6a31f2760 msgr2=0x55b6a31fa900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:31.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.713+0000 7f7e46ffd640 1 --2- 172.21.15.164:0/3779817820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6a31f2760 0x55b6a31fa900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.713+0000 7f7e46ffd640 1 -- 172.21.15.164:0/3779817820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a31fd940 msgr2=0x55b6a31fb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:31.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.713+0000 7f7e46ffd640 1 --2- 172.21.15.164:0/3779817820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a31fd940 0x55b6a31fb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.714+0000 7f7e46ffd640 1 -- 172.21.15.164:0/3779817820 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6a320ddf0 con 0x55b6a31f3080 2024-09-19T08:56:31.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.714+0000 7f7e477fe640 1 --2- 172.21.15.164:0/3779817820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6a31f2760 0x55b6a31fa900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:56:31.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.714+0000 7f7e47fff640 1 --2- 172.21.15.164:0/3779817820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a31fd940 0x55b6a31fb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:56:31.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.714+0000 7f7e46ffd640 1 --2- 172.21.15.164:0/3779817820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6a31f3080 0x55b6a31fae40 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c005890 tx=0x7f7e3c005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b68e95dfbbfe31f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:31.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.714+0000 7f7e26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:56:31.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.714+0000 7f7e26ffd640 1 -- 172.21.15.164:0/3779817820 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c016040 con 0x55b6a31f3080 2024-09-19T08:56:31.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.714+0000 7f7e26ffd640 1 -- 172.21.15.164:0/3779817820 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f7e3c004050 con 0x55b6a31f3080 2024-09-19T08:56:31.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.714+0000 7f7e50480740 1 -- 172.21.15.164:0/3779817820 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e3c0047a0 con 0x55b6a31f3080 2024-09-19T08:56:31.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.714+0000 7f7e26ffd640 1 -- 172.21.15.164:0/3779817820 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7e3c00c920 con 0x55b6a31f3080 2024-09-19T08:56:31.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.714+0000 7f7e50480740 10 client.17784 fetch_fsmap learned FSMap version 32 2024-09-19T08:56:31.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.714+0000 7f7e50480740 1 -- 172.21.15.164:0/3779817820 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b6a32107c0 con 0x55b6a31f3080 2024-09-19T08:56:31.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.714+0000 7f7e26ffd640 1 -- 172.21.15.164:0/3779817820 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e3c0047a0 con 0x55b6a31f3080 2024-09-19T08:56:31.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.714+0000 7f7e26ffd640 1 -- 172.21.15.164:0/3779817820 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f7e3c0035f0 con 0x55b6a31f3080 2024-09-19T08:56:31.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.714+0000 7f7e50480740 10 client.17784 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:56:31.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.714+0000 7f7e50480740 10 client.17784 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:56:31.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.714+0000 7f7e50480740 20 client.17784 populate_metadata read hostname 'smithi164' 2024-09-19T08:56:31.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.714+0000 7f7e50480740 1 --2- 172.21.15.164:0/3779817820 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55b6a3211bd0 0x55b6a3213fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:31.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.714+0000 7f7e50480740 4 client.17784 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:56:31.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.714+0000 7f7e50480740 1 -- 172.21.15.164:0/3779817820 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6a31659d0 con 0x55b6a3211bd0 2024-09-19T08:56:31.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.715+0000 7f7e477fe640 1 --2- 172.21.15.164:0/3779817820 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55b6a3211bd0 0x55b6a3213fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:31.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.715+0000 7f7e477fe640 1 --2- 172.21.15.164:0/3779817820 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55b6a3211bd0 0x55b6a3213fc0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c310748dce82d53 server_cookie=812e7f9422bbecd5 in_seq=0 out_seq=0 2024-09-19T08:56:31.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.715+0000 7f7e26ffd640 10 client.17784 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:56:31.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.716+0000 7f7e26ffd640 1 -- 172.21.15.164:0/3779817820 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b6a31659d0 con 0x55b6a3211bd0 2024-09-19T08:56:31.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.716+0000 7f7e26ffd640 10 client.17784 handle_command_reply: tid=0 2024-09-19T08:56:31.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.716+0000 7f7e50480740 1 client.17784 shutdown 2024-09-19T08:56:31.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.716+0000 7f7e50480740 1 -- 172.21.15.164:0/3779817820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6a31f3080 msgr2=0x55b6a31fae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:31.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.716+0000 7f7e50480740 1 --2- 172.21.15.164:0/3779817820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6a31f3080 0x55b6a31fae40 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c005890 tx=0x7f7e3c005250 comp rx=0 tx=0).stop 2024-09-19T08:56:31.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.716+0000 7f7e50480740 1 -- 172.21.15.164:0/3779817820 shutdown_connections 2024-09-19T08:56:31.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.716+0000 7f7e50480740 1 --2- 172.21.15.164:0/3779817820 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55b6a3211bd0 0x55b6a3213fc0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.716+0000 7f7e50480740 1 --2- 172.21.15.164:0/3779817820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a31fd940 0x55b6a31fb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.716+0000 7f7e50480740 1 --2- 172.21.15.164:0/3779817820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6a31f3080 0x55b6a31fae40 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.716+0000 7f7e50480740 1 --2- 172.21.15.164:0/3779817820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6a31f2760 0x55b6a31fa900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.716+0000 7f7e26ffd640 0 client.17784 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:56:31.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.716+0000 7f7e50480740 1 -- 172.21.15.164:0/3779817820 >> 172.21.15.164:0/3779817820 conn(0x55b6a31eecf0 msgr2=0x55b6a31f4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:31.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.716+0000 7f7e50480740 1 -- 172.21.15.164:0/3779817820 shutdown_connections 2024-09-19T08:56:31.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.716+0000 7f7e50480740 1 -- 172.21.15.164:0/3779817820 wait complete. 2024-09-19T08:56:31.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.716+0000 7f7e50480740 20 client.17784 trim_cache size 0 max 16384 2024-09-19T08:56:31.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.733+0000 7f7e50480740 1 Processor -- start 2024-09-19T08:56:31.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.733+0000 7f7e50480740 1 -- start start 2024-09-19T08:56:31.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.733+0000 7f7e50480740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6a31eecf0 0x55b6a31f9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:31.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.733+0000 7f7e50480740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6a31f2760 0x55b6a31faf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:31.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.733+0000 7f7e50480740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a31f3080 0x55b6a31f48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:31.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.733+0000 7f7e50480740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7e481ab370 con 0x55b6a31f2760 2024-09-19T08:56:31.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.733+0000 7f7e50480740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7e481aceb0 con 0x55b6a31f3080 2024-09-19T08:56:31.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.733+0000 7f7e50480740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7e481aba40 con 0x55b6a31eecf0 2024-09-19T08:56:31.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.733+0000 7f7e46ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6a31f2760 0x55b6a31faf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:31.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.733+0000 7f7e46ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6a31f2760 0x55b6a31faf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:54450/0 (socket says 172.21.15.164:54450) 2024-09-19T08:56:31.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.734+0000 7f7e46ffd640 1 -- 172.21.15.164:0/3119890323 learned_addr learned my addr 172.21.15.164:0/3119890323 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:31.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.734+0000 7f7e47fff640 1 --2- 172.21.15.164:0/3119890323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a31f3080 0x55b6a31f48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:31.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.734+0000 7f7e46ffd640 1 -- 172.21.15.164:0/3119890323 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6a31eecf0 msgr2=0x55b6a31f9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:31.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.734+0000 7f7e477fe640 1 --2- 172.21.15.164:0/3119890323 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6a31eecf0 0x55b6a31f9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:31.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.734+0000 7f7e46ffd640 1 --2- 172.21.15.164:0/3119890323 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6a31eecf0 0x55b6a31f9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.734+0000 7f7e46ffd640 1 -- 172.21.15.164:0/3119890323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a31f3080 msgr2=0x55b6a31f48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:31.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.734+0000 7f7e46ffd640 1 --2- 172.21.15.164:0/3119890323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a31f3080 0x55b6a31f48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.734+0000 7f7e46ffd640 1 -- 172.21.15.164:0/3119890323 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6a31efda0 con 0x55b6a31f2760 2024-09-19T08:56:31.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.734+0000 7f7e47fff640 1 --2- 172.21.15.164:0/3119890323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a31f3080 0x55b6a31f48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:56:31.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.734+0000 7f7e46ffd640 1 --2- 172.21.15.164:0/3119890323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6a31f2760 0x55b6a31faf30 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c003b60 tx=0x7f7e3c005650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89750fd822f474fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:31.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.734+0000 7f7e2d7fa640 1 -- 172.21.15.164:0/3119890323 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c035030 con 0x55b6a31f2760 2024-09-19T08:56:31.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.734+0000 7f7e2d7fa640 1 -- 172.21.15.164:0/3119890323 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e3c01e070 con 0x55b6a31f2760 2024-09-19T08:56:31.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.734+0000 7f7e2d7fa640 1 -- 172.21.15.164:0/3119890323 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c016040 con 0x55b6a31f2760 2024-09-19T08:56:31.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.735+0000 7f7e50480740 1 -- 172.21.15.164:0/3119890323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6a31f2760 msgr2=0x55b6a31faf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:31.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.735+0000 7f7e50480740 1 --2- 172.21.15.164:0/3119890323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6a31f2760 0x55b6a31faf30 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c003b60 tx=0x7f7e3c005650 comp rx=0 tx=0).stop 2024-09-19T08:56:31.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.735+0000 7f7e50480740 1 -- 172.21.15.164:0/3119890323 shutdown_connections 2024-09-19T08:56:31.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.735+0000 7f7e50480740 1 --2- 172.21.15.164:0/3119890323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a31f3080 0x55b6a31f48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.735+0000 7f7e50480740 1 --2- 172.21.15.164:0/3119890323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6a31f2760 0x55b6a31faf30 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.735+0000 7f7e50480740 1 --2- 172.21.15.164:0/3119890323 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6a31eecf0 0x55b6a31f9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.735+0000 7f7e50480740 1 -- 172.21.15.164:0/3119890323 >> 172.21.15.164:0/3119890323 conn(0x55b6a31fb4e0 msgr2=0x55b6a31f41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:31.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.735+0000 7f7e50480740 1 -- 172.21.15.164:0/3119890323 shutdown_connections 2024-09-19T08:56:31.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.735+0000 7f7e50480740 1 -- 172.21.15.164:0/3119890323 wait complete. 2024-09-19T08:56:31.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.735+0000 7f7e50480740 1 -- start start 2024-09-19T08:56:31.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.736+0000 7f7e50480740 1 Processor -- start 2024-09-19T08:56:31.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.736+0000 7f7e50480740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6a31eecf0 0x55b6a3209c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:31.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.736+0000 7f7e50480740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6a31f2760 0x55b6a320a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:31.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.736+0000 7f7e50480740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a31f3080 0x55b6a3206c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:31.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.736+0000 7f7e50480740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7e481ac7e0 con 0x55b6a31eecf0 2024-09-19T08:56:31.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.736+0000 7f7e50480740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7e481ac110 con 0x55b6a31f3080 2024-09-19T08:56:31.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.736+0000 7f7e50480740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7e481a9390 con 0x55b6a31f2760 2024-09-19T08:56:31.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.736+0000 7f7e477fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6a31eecf0 0x55b6a3209c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:31.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.736+0000 7f7e477fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6a31eecf0 0x55b6a3209c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:54452/0 (socket says 172.21.15.164:54452) 2024-09-19T08:56:31.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.736+0000 7f7e477fe640 1 -- 172.21.15.164:0/1457580275 learned_addr learned my addr 172.21.15.164:0/1457580275 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:31.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.736+0000 7f7e50480740 10 client.0 _ll_register_callbacks cb 0x7f7e3c004c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:56:31.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.736+0000 7f7e46ffd640 1 --2- 172.21.15.164:0/1457580275 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6a31f2760 0x55b6a320a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:31.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.736+0000 7f7e47fff640 1 --2- 172.21.15.164:0/1457580275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a31f3080 0x55b6a3206c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:31.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.736+0000 7f7e477fe640 1 -- 172.21.15.164:0/1457580275 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6a31f2760 msgr2=0x55b6a320a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:31.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.736+0000 7f7e477fe640 1 --2- 172.21.15.164:0/1457580275 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6a31f2760 0x55b6a320a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.736+0000 7f7e477fe640 1 -- 172.21.15.164:0/1457580275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a31f3080 msgr2=0x55b6a3206c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:31.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.736+0000 7f7e477fe640 1 --2- 172.21.15.164:0/1457580275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a31f3080 0x55b6a3206c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.736+0000 7f7e477fe640 1 -- 172.21.15.164:0/1457580275 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6a320dac0 con 0x55b6a31eecf0 2024-09-19T08:56:31.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.736+0000 7f7e47fff640 1 --2- 172.21.15.164:0/1457580275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a31f3080 0x55b6a3206c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:56:31.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.737+0000 7f7e477fe640 1 --2- 172.21.15.164:0/1457580275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6a31eecf0 0x55b6a3209c30 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x55b6a3214580 tx=0x7f7e38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf982c7240f043d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:31.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.737+0000 7f7e26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:56:31.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.737+0000 7f7e26ffd640 1 -- 172.21.15.164:0/1457580275 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3802a070 con 0x55b6a31eecf0 2024-09-19T08:56:31.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.737+0000 7f7e50480740 1 -- 172.21.15.164:0/1457580275 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e3c01e070 con 0x55b6a31eecf0 2024-09-19T08:56:31.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.737+0000 7f7e26ffd640 1 -- 172.21.15.164:0/1457580275 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f7e38005050 con 0x55b6a31eecf0 2024-09-19T08:56:31.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.737+0000 7f7e26ffd640 1 -- 172.21.15.164:0/1457580275 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7e38036050 con 0x55b6a31eecf0 2024-09-19T08:56:31.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.737+0000 7f7e50480740 10 client.17790 fetch_fsmap learned FSMap version 32 2024-09-19T08:56:31.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.737+0000 7f7e50480740 1 -- 172.21.15.164:0/1457580275 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b6a31ea260 con 0x55b6a31eecf0 2024-09-19T08:56:31.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.737+0000 7f7e26ffd640 1 -- 172.21.15.164:0/1457580275 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e3c01e070 con 0x55b6a31eecf0 2024-09-19T08:56:31.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.737+0000 7f7e26ffd640 1 -- 172.21.15.164:0/1457580275 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f7e380266a0 con 0x55b6a31eecf0 2024-09-19T08:56:31.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.737+0000 7f7e50480740 10 client.17790 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:56:31.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.737+0000 7f7e50480740 10 client.17790 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:56:31.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.737+0000 7f7e50480740 20 client.17790 populate_metadata read hostname 'smithi164' 2024-09-19T08:56:31.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.737+0000 7f7e50480740 1 --2- 172.21.15.164:0/1457580275 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55b6a31fd940 0x55b6a327f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:31.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.737+0000 7f7e50480740 4 client.17790 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-19T08:56:31.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.737+0000 7f7e50480740 1 -- 172.21.15.164:0/1457580275 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55b6a315fd20 con 0x55b6a31fd940 2024-09-19T08:56:31.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.738+0000 7f7e46ffd640 1 --2- 172.21.15.164:0/1457580275 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55b6a31fd940 0x55b6a327f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:31.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.738+0000 7f7e26ffd640 10 client.17790 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:56:31.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.738+0000 7f7e46ffd640 1 --2- 172.21.15.164:0/1457580275 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55b6a31fd940 0x55b6a327f1e0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e0e2a5a6fc6fb54 server_cookie=650cb006b3bfe546 in_seq=0 out_seq=0 2024-09-19T08:56:31.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.754+0000 7f7e26ffd640 1 -- 172.21.15.164:0/1457580275 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b6a315fd20 con 0x55b6a31fd940 2024-09-19T08:56:31.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.754+0000 7f7e26ffd640 10 client.17790 handle_command_reply: tid=0 2024-09-19T08:56:31.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.754+0000 7f7e50480740 1 client.17790 shutdown 2024-09-19T08:56:31.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.754+0000 7f7e50480740 1 -- 172.21.15.164:0/1457580275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6a31eecf0 msgr2=0x55b6a3209c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:31.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.754+0000 7f7e50480740 1 --2- 172.21.15.164:0/1457580275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6a31eecf0 0x55b6a3209c30 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x55b6a3214580 tx=0x7f7e38004a40 comp rx=0 tx=0).stop 2024-09-19T08:56:31.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.754+0000 7f7e50480740 1 -- 172.21.15.164:0/1457580275 shutdown_connections 2024-09-19T08:56:31.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.754+0000 7f7e50480740 1 --2- 172.21.15.164:0/1457580275 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55b6a31fd940 0x55b6a327f1e0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.754+0000 7f7e50480740 1 --2- 172.21.15.164:0/1457580275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a31f3080 0x55b6a3206c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.754+0000 7f7e50480740 1 --2- 172.21.15.164:0/1457580275 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b6a31f2760 0x55b6a320a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.754+0000 7f7e50480740 1 --2- 172.21.15.164:0/1457580275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b6a31eecf0 0x55b6a3209c30 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.754+0000 7f7e50480740 1 -- 172.21.15.164:0/1457580275 >> 172.21.15.164:0/1457580275 conn(0x55b6a31fb4e0 msgr2=0x55b6a320d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:31.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.754+0000 7f7e26ffd640 0 client.17790 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:56:31.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.755+0000 7f7e50480740 1 -- 172.21.15.164:0/1457580275 shutdown_connections 2024-09-19T08:56:31.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.755+0000 7f7e50480740 1 -- 172.21.15.164:0/1457580275 wait complete. 2024-09-19T08:56:31.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.755+0000 7f7e50480740 20 client.17790 trim_cache size 0 max 16384 2024-09-19T08:56:31.755 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:56:31.755 INFO:teuthology.orchestra.run.smithi164.stdout: "epoch": 32, 2024-09-19T08:56:31.756 INFO:teuthology.orchestra.run.smithi164.stdout: "leader": 24479, 2024-09-19T08:56:31.756 INFO:teuthology.orchestra.run.smithi164.stdout: "set_version": 38, 2024-09-19T08:56:31.756 INFO:teuthology.orchestra.run.smithi164.stdout: "sets": { 2024-09-19T08:56:31.756 INFO:teuthology.orchestra.run.smithi164.stdout: "e5bd5b7c": { 2024-09-19T08:56:31.756 INFO:teuthology.orchestra.run.smithi164.stdout: "version": 38, 2024-09-19T08:56:31.756 INFO:teuthology.orchestra.run.smithi164.stdout: "age_ref": 0.0, 2024-09-19T08:56:31.756 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:56:31.756 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "QUIESCED", 2024-09-19T08:56:31.756 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:56:31.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:56:31.756 INFO:teuthology.orchestra.run.smithi164.stdout: "timeout": 90.0, 2024-09-19T08:56:31.756 INFO:teuthology.orchestra.run.smithi164.stdout: "expiration": 150.0, 2024-09-19T08:56:31.756 INFO:teuthology.orchestra.run.smithi164.stdout: "members": { 2024-09-19T08:56:31.757 INFO:teuthology.orchestra.run.smithi164.stdout: "file:/": { 2024-09-19T08:56:31.757 INFO:teuthology.orchestra.run.smithi164.stdout: "excluded": false, 2024-09-19T08:56:31.757 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:56:31.757 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "QUIESCED", 2024-09-19T08:56:31.757 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:56:31.757 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:56:31.757 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:56:31.757 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:56:31.757 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:56:31.757 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:56:31.757 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:56:31.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.755+0000 7f7e4e0a7640 1 -- 172.21.15.164:0/708333254 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f7e300a33b0 msgr2=0x7f7e300a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:31.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.755+0000 7f7e4e0a7640 1 --2- 172.21.15.164:0/708333254 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f7e300a33b0 0x7f7e300a5870 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x55b6a31efb50 tx=0x7f7e38008510 comp rx=0 tx=0).stop 2024-09-19T08:56:31.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.755+0000 7f7e4e0a7640 1 -- 172.21.15.164:0/708333254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e48102ef0 msgr2=0x7f7e4819cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:31.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.755+0000 7f7e4e0a7640 1 --2- 172.21.15.164:0/708333254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e48102ef0 0x7f7e4819cc80 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f7e40005ab0 tx=0x7f7e40003c30 comp rx=0 tx=0).stop 2024-09-19T08:56:31.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.755+0000 7f7e4e0a7640 1 -- 172.21.15.164:0/708333254 shutdown_connections 2024-09-19T08:56:31.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.755+0000 7f7e4e0a7640 1 --2- 172.21.15.164:0/708333254 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f7e300a33b0 0x7f7e300a5870 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.755+0000 7f7e4e0a7640 1 --2- 172.21.15.164:0/708333254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e48102ef0 0x7f7e4819cc80 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.755+0000 7f7e4e0a7640 1 --2- 172.21.15.164:0/708333254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e48102570 0x7f7e481988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.755+0000 7f7e4e0a7640 1 --2- 172.21.15.164:0/708333254 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e48100e20 0x7f7e481983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:31.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.755+0000 7f7e4e0a7640 1 -- 172.21.15.164:0/708333254 >> 172.21.15.164:0/708333254 conn(0x7f7e480fc7d0 msgr2=0x7f7e4806a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:31.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.755+0000 7f7e4e0a7640 1 -- 172.21.15.164:0/708333254 shutdown_connections 2024-09-19T08:56:31.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:31.755+0000 7f7e4e0a7640 1 -- 172.21.15.164:0/708333254 wait complete. 2024-09-19T08:56:32.463 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:32 smithi164 ceph-mon[30928]: pgmap v334: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:56:32.493 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e5bd5b7c', quiesce took 3.4 seconds. Will release after: 26.6 2024-09-19T08:56:32.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:32 smithi170 ceph-mon[35072]: pgmap v334: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:56:32.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:32 smithi173 ceph-mon[35306]: pgmap v334: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:56:34.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:34 smithi164 ceph-mon[30928]: pgmap v335: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-19T08:56:34.624 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:34 smithi170 ceph-mon[35072]: pgmap v335: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-19T08:56:34.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:34 smithi173 ceph-mon[35306]: pgmap v335: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-19T08:56:36.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:36 smithi164 ceph-mon[30928]: pgmap v336: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T08:56:36.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:36 smithi170 ceph-mon[35072]: pgmap v336: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T08:56:36.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:36 smithi173 ceph-mon[35306]: pgmap v336: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T08:56:38.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:38 smithi164 ceph-mon[30928]: pgmap v337: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:56:38.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:38 smithi170 ceph-mon[35072]: pgmap v337: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:56:38.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:38 smithi173 ceph-mon[35306]: pgmap v337: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:56:40.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:40 smithi164 ceph-mon[30928]: pgmap v338: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:56:40.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:40 smithi170 ceph-mon[35072]: pgmap v338: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:56:40.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:40 smithi173 ceph-mon[35306]: pgmap v338: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:56:42.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:42 smithi164 ceph-mon[30928]: pgmap v339: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:56:42.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:42 smithi170 ceph-mon[35072]: pgmap v339: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:56:42.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:42 smithi173 ceph-mon[35306]: pgmap v339: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:56:44.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:44 smithi164 ceph-mon[30928]: pgmap v340: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:56:44.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:44 smithi170 ceph-mon[35072]: pgmap v340: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:56:44.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:44 smithi173 ceph-mon[35306]: pgmap v340: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:56:46.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:46 smithi170 ceph-mon[35072]: pgmap v341: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T08:56:46.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:46 smithi173 ceph-mon[35306]: pgmap v341: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T08:56:46.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:46 smithi164 ceph-mon[30928]: pgmap v341: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T08:56:48.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:48 smithi170 ceph-mon[35072]: pgmap v342: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:56:48.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:48 smithi173 ceph-mon[35306]: pgmap v342: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:56:48.781 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:48 smithi164 ceph-mon[30928]: pgmap v342: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:56:49.678 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:56:49.939 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:56:50.331 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:50 smithi164 ceph-mon[30928]: pgmap v343: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:56:50.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.413+0000 7f5671f9d640 1 -- 172.21.15.164:0/131053779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f566c106a80 msgr2=0x7f566c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:50.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.413+0000 7f5671f9d640 1 --2- 172.21.15.164:0/131053779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f566c106a80 0x7f566c106e60 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f565c009930 tx=0x7f565c02a420 comp rx=0 tx=0).stop 2024-09-19T08:56:50.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.413+0000 7f5671f9d640 1 -- 172.21.15.164:0/131053779 shutdown_connections 2024-09-19T08:56:50.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.413+0000 7f5671f9d640 1 --2- 172.21.15.164:0/131053779 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f566c101420 0x7f566c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.413+0000 7f5671f9d640 1 --2- 172.21.15.164:0/131053779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f566c100a80 0x7f566c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.413+0000 7f5671f9d640 1 --2- 172.21.15.164:0/131053779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f566c106a80 0x7f566c106e60 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.413+0000 7f5671f9d640 1 -- 172.21.15.164:0/131053779 >> 172.21.15.164:0/131053779 conn(0x7f566c0fc7a0 msgr2=0x7f566c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:50.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.413+0000 7f5671f9d640 1 -- 172.21.15.164:0/131053779 shutdown_connections 2024-09-19T08:56:50.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.413+0000 7f5671f9d640 1 -- 172.21.15.164:0/131053779 wait complete. 2024-09-19T08:56:50.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.414+0000 7f5671f9d640 1 Processor -- start 2024-09-19T08:56:50.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.414+0000 7f5671f9d640 1 -- start start 2024-09-19T08:56:50.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.414+0000 7f5671f9d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f566c100a80 0x7f566c199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:50.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.414+0000 7f5671f9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f566c101420 0x7f566c1998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:50.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.414+0000 7f5671f9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f566c106a80 0x7f566c1a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:50.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.414+0000 7f5671f9d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f566c110e80 con 0x7f566c106a80 2024-09-19T08:56:50.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.414+0000 7f5671f9d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f566c110d00 con 0x7f566c101420 2024-09-19T08:56:50.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.414+0000 7f5671f9d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f566c111000 con 0x7f566c100a80 2024-09-19T08:56:50.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.414+0000 7f566bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f566c106a80 0x7f566c1a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:50.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.414+0000 7f566bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f566c106a80 0x7f566c1a43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52494/0 (socket says 172.21.15.164:52494) 2024-09-19T08:56:50.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.414+0000 7f566bfff640 1 -- 172.21.15.164:0/800023015 learned_addr learned my addr 172.21.15.164:0/800023015 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:50.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.414+0000 7f566b7fe640 1 --2- 172.21.15.164:0/800023015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f566c100a80 0x7f566c199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:50.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.414+0000 7f5663fff640 1 --2- 172.21.15.164:0/800023015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f566c101420 0x7f566c1998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:50.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.415+0000 7f566bfff640 1 -- 172.21.15.164:0/800023015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f566c100a80 msgr2=0x7f566c199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:50.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.415+0000 7f566bfff640 1 --2- 172.21.15.164:0/800023015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f566c100a80 0x7f566c199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.415+0000 7f566bfff640 1 -- 172.21.15.164:0/800023015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f566c101420 msgr2=0x7f566c1998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:50.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.415+0000 7f566bfff640 1 --2- 172.21.15.164:0/800023015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f566c101420 0x7f566c1998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.415+0000 7f566bfff640 1 -- 172.21.15.164:0/800023015 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566c1a4930 con 0x7f566c106a80 2024-09-19T08:56:50.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.415+0000 7f566bfff640 1 --2- 172.21.15.164:0/800023015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f566c106a80 0x7f566c1a43f0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f566400cad0 tx=0x7f566400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:50.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.415+0000 7f56697fa640 1 -- 172.21.15.164:0/800023015 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566400ae20 con 0x7f566c106a80 2024-09-19T08:56:50.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.415+0000 7f5671f9d640 1 -- 172.21.15.164:0/800023015 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f566c1a4b00 con 0x7f566c106a80 2024-09-19T08:56:50.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.415+0000 7f56697fa640 1 -- 172.21.15.164:0/800023015 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5664005020 con 0x7f566c106a80 2024-09-19T08:56:50.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.415+0000 7f56697fa640 1 -- 172.21.15.164:0/800023015 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566400f6f0 con 0x7f566c106a80 2024-09-19T08:56:50.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.415+0000 7f5671f9d640 1 -- 172.21.15.164:0/800023015 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f566c1a5010 con 0x7f566c106a80 2024-09-19T08:56:50.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.417+0000 7f56697fa640 1 -- 172.21.15.164:0/800023015 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f56640051d0 con 0x7f566c106a80 2024-09-19T08:56:50.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.418+0000 7f5674376740 1 Processor -- start 2024-09-19T08:56:50.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.418+0000 7f5674376740 1 -- start start 2024-09-19T08:56:50.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.418+0000 7f5674376740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c57e0dd4a0 0x55c57e0dd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:50.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.418+0000 7f5674376740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c57e0dddc0 0x55c57e0e8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:50.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.418+0000 7f56697fa640 1 --2- 172.21.15.164:0/800023015 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f56580a33b0 0x7f56580a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:50.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.418+0000 7f5674376740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c57e0e8680 0x55c57e0eaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:50.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.418+0000 7f5674376740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c57e0432e0 con 0x55c57e0dd4a0 2024-09-19T08:56:50.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.418+0000 7f5674376740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c57dfeace0 con 0x55c57e0dddc0 2024-09-19T08:56:50.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.418+0000 7f5674376740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c57e03f2a0 con 0x55c57e0e8680 2024-09-19T08:56:50.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.418+0000 7f5663fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c57e0dd4a0 0x55c57e0dd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:50.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.418+0000 7f5663fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c57e0dd4a0 0x55c57e0dd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52500/0 (socket says 172.21.15.164:52500) 2024-09-19T08:56:50.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.418+0000 7f5663fff640 1 -- 172.21.15.164:0/1485652151 learned_addr learned my addr 172.21.15.164:0/1485652151 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:50.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.418+0000 7f56697fa640 1 -- 172.21.15.164:0/800023015 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f56640cfe00 con 0x7f566c106a80 2024-09-19T08:56:50.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.418+0000 7f5663fff640 1 --2- 172.21.15.164:0/800023015 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f56580a33b0 0x7f56580a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:50.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.418+0000 7f566b7fe640 1 --2- 172.21.15.164:0/1485652151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c57e0dddc0 0x55c57e0e8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:50.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.418+0000 7f566bfff640 1 --2- 172.21.15.164:0/1485652151 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c57e0e8680 0x55c57e0eaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:50.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.418+0000 7f5663fff640 1 -- 172.21.15.164:0/1485652151 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c57e0e8680 msgr2=0x55c57e0eaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:50.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.418+0000 7f5663fff640 1 --2- 172.21.15.164:0/1485652151 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c57e0e8680 0x55c57e0eaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.418+0000 7f5663fff640 1 -- 172.21.15.164:0/1485652151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c57e0dddc0 msgr2=0x55c57e0e8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:50.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.418+0000 7f5663fff640 1 --2- 172.21.15.164:0/1485652151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c57e0dddc0 0x55c57e0e8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.418+0000 7f5663fff640 1 -- 172.21.15.164:0/1485652151 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c57dfcf440 con 0x55c57e0dd4a0 2024-09-19T08:56:50.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.419+0000 7f5663fff640 1 --2- 172.21.15.164:0/1485652151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c57e0dd4a0 0x55c57e0dd880 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f5654014c80 tx=0x7f56540117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb510f55404d627 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:50.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.419+0000 7f5663fff640 1 --2- 172.21.15.164:0/800023015 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f56580a33b0 0x7f56580a5870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5654011cb0 tx=0x7f5654016000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:50.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.419+0000 7f5661ffb640 1 -- 172.21.15.164:0/1485652151 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5654014620 con 0x55c57e0dd4a0 2024-09-19T08:56:50.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.419+0000 7f5661ffb640 1 -- 172.21.15.164:0/1485652151 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5654023070 con 0x55c57e0dd4a0 2024-09-19T08:56:50.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.419+0000 7f5661ffb640 1 -- 172.21.15.164:0/1485652151 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56540035f0 con 0x55c57e0dd4a0 2024-09-19T08:56:50.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.419+0000 7f5674376740 1 -- 172.21.15.164:0/1485652151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c57e0dd4a0 msgr2=0x55c57e0dd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:50.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.419+0000 7f5674376740 1 --2- 172.21.15.164:0/1485652151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c57e0dd4a0 0x55c57e0dd880 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f5654014c80 tx=0x7f56540117b0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.420+0000 7f5674376740 1 -- 172.21.15.164:0/1485652151 shutdown_connections 2024-09-19T08:56:50.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.420+0000 7f5674376740 1 --2- 172.21.15.164:0/1485652151 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c57e0e8680 0x55c57e0eaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.420+0000 7f5674376740 1 --2- 172.21.15.164:0/1485652151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c57e0dddc0 0x55c57e0e8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.420+0000 7f5674376740 1 --2- 172.21.15.164:0/1485652151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c57e0dd4a0 0x55c57e0dd880 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.420+0000 7f5674376740 1 -- 172.21.15.164:0/1485652151 >> 172.21.15.164:0/1485652151 conn(0x55c57e0d9a30 msgr2=0x55c57e0d9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:50.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.420+0000 7f5674376740 1 -- 172.21.15.164:0/1485652151 shutdown_connections 2024-09-19T08:56:50.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.420+0000 7f5674376740 1 -- 172.21.15.164:0/1485652151 wait complete. 2024-09-19T08:56:50.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.420+0000 7f5674376740 1 -- start start 2024-09-19T08:56:50.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.420+0000 7f5674376740 1 Processor -- start 2024-09-19T08:56:50.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.420+0000 7f5674376740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c57e0dd4a0 0x55c57e0e8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:50.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.420+0000 7f5674376740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c57e0dddc0 0x55c57e0e5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:50.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.420+0000 7f5674376740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c57e0e8680 0x55c57e0e58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:50.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.420+0000 7f5674376740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c57e0eb250 con 0x55c57e0dd4a0 2024-09-19T08:56:50.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.420+0000 7f5674376740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c57dffe050 con 0x55c57e0e8680 2024-09-19T08:56:50.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.420+0000 7f5674376740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c57e0eb3d0 con 0x55c57e0dddc0 2024-09-19T08:56:50.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.420+0000 7f5674376740 10 client.0 _ll_register_callbacks cb 0x55c57e03e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:56:50.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.421+0000 7f566b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c57e0dd4a0 0x55c57e0e8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:50.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.421+0000 7f566b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c57e0dd4a0 0x55c57e0e8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52508/0 (socket says 172.21.15.164:52508) 2024-09-19T08:56:50.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.421+0000 7f566b7fe640 1 -- 172.21.15.164:0/4207528623 learned_addr learned my addr 172.21.15.164:0/4207528623 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:50.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.421+0000 7f5663fff640 1 --2- 172.21.15.164:0/4207528623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c57e0dddc0 0x55c57e0e5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:50.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.421+0000 7f566bfff640 1 --2- 172.21.15.164:0/4207528623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c57e0e8680 0x55c57e0e58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:50.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.421+0000 7f566b7fe640 1 -- 172.21.15.164:0/4207528623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c57e0dddc0 msgr2=0x55c57e0e5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:50.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.421+0000 7f566b7fe640 1 --2- 172.21.15.164:0/4207528623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c57e0dddc0 0x55c57e0e5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.421+0000 7f566b7fe640 1 -- 172.21.15.164:0/4207528623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c57e0e8680 msgr2=0x55c57e0e58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:50.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.421+0000 7f566b7fe640 1 --2- 172.21.15.164:0/4207528623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c57e0e8680 0x55c57e0e58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.421+0000 7f566b7fe640 1 -- 172.21.15.164:0/4207528623 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c57ddb43c0 con 0x55c57e0dd4a0 2024-09-19T08:56:50.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.421+0000 7f566b7fe640 1 --2- 172.21.15.164:0/4207528623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c57e0dd4a0 0x55c57e0e8050 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f565c009930 tx=0x7f565c031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a6ab9ce4056dc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:50.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.421+0000 7f564affd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:56:50.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.421+0000 7f564affd640 1 -- 172.21.15.164:0/4207528623 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c035040 con 0x55c57e0dd4a0 2024-09-19T08:56:50.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.421+0000 7f564affd640 1 -- 172.21.15.164:0/4207528623 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f565c004240 con 0x55c57e0dd4a0 2024-09-19T08:56:50.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.421+0000 7f5674376740 1 -- 172.21.15.164:0/4207528623 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5654023070 con 0x55c57e0dd4a0 2024-09-19T08:56:50.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.421+0000 7f564affd640 1 -- 172.21.15.164:0/4207528623 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f565c004440 con 0x55c57e0dd4a0 2024-09-19T08:56:50.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.421+0000 7f5674376740 10 client.17802 fetch_fsmap learned FSMap version 32 2024-09-19T08:56:50.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.421+0000 7f5674376740 1 -- 172.21.15.164:0/4207528623 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c57e0f6c80 con 0x55c57e0dd4a0 2024-09-19T08:56:50.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.421+0000 7f564affd640 1 -- 172.21.15.164:0/4207528623 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5654023070 con 0x55c57e0dd4a0 2024-09-19T08:56:50.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.422+0000 7f564affd640 1 -- 172.21.15.164:0/4207528623 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f565c004b80 con 0x55c57e0dd4a0 2024-09-19T08:56:50.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.422+0000 7f5674376740 10 client.17802 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:56:50.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.422+0000 7f5674376740 10 client.17802 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:56:50.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.422+0000 7f5674376740 20 client.17802 populate_metadata read hostname 'smithi164' 2024-09-19T08:56:50.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.422+0000 7f5674376740 1 --2- 172.21.15.164:0/4207528623 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c57e0f80e0 0x55c57e0fa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:50.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.422+0000 7f5674376740 4 client.17802 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:56:50.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.422+0000 7f5674376740 1 -- 172.21.15.164:0/4207528623 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c57e050850 con 0x55c57e0f80e0 2024-09-19T08:56:50.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.422+0000 7f5663fff640 1 --2- 172.21.15.164:0/4207528623 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c57e0f80e0 0x55c57e0fa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:50.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.423+0000 7f5663fff640 1 --2- 172.21.15.164:0/4207528623 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c57e0f80e0 0x55c57e0fa520 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1654b144cb31e581 server_cookie=30d0c83ef2d2d495 in_seq=0 out_seq=0 2024-09-19T08:56:50.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.423+0000 7f564affd640 10 client.17802 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:56:50.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.423+0000 7f564affd640 1 -- 172.21.15.164:0/4207528623 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c57e050850 con 0x55c57e0f80e0 2024-09-19T08:56:50.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.423+0000 7f564affd640 10 client.17802 handle_command_reply: tid=0 2024-09-19T08:56:50.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.423+0000 7f5674376740 1 client.17802 shutdown 2024-09-19T08:56:50.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.423+0000 7f5674376740 1 -- 172.21.15.164:0/4207528623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c57e0dd4a0 msgr2=0x55c57e0e8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:50.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.423+0000 7f5674376740 1 --2- 172.21.15.164:0/4207528623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c57e0dd4a0 0x55c57e0e8050 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f565c009930 tx=0x7f565c031920 comp rx=0 tx=0).stop 2024-09-19T08:56:50.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.423+0000 7f5674376740 1 -- 172.21.15.164:0/4207528623 shutdown_connections 2024-09-19T08:56:50.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.423+0000 7f5674376740 1 --2- 172.21.15.164:0/4207528623 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c57e0f80e0 0x55c57e0fa520 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.423+0000 7f5674376740 1 --2- 172.21.15.164:0/4207528623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c57e0e8680 0x55c57e0e58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.423+0000 7f5674376740 1 --2- 172.21.15.164:0/4207528623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c57e0dddc0 0x55c57e0e5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.423+0000 7f5674376740 1 --2- 172.21.15.164:0/4207528623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c57e0dd4a0 0x55c57e0e8050 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.423+0000 7f564affd640 0 client.17802 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:56:50.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.423+0000 7f5674376740 1 -- 172.21.15.164:0/4207528623 >> 172.21.15.164:0/4207528623 conn(0x55c57e0d9a30 msgr2=0x55c57e0d8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:50.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.424+0000 7f5674376740 1 -- 172.21.15.164:0/4207528623 shutdown_connections 2024-09-19T08:56:50.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.424+0000 7f5674376740 1 -- 172.21.15.164:0/4207528623 wait complete. 2024-09-19T08:56:50.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.424+0000 7f5674376740 20 client.17802 trim_cache size 0 max 16384 2024-09-19T08:56:50.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.441+0000 7f5674376740 1 Processor -- start 2024-09-19T08:56:50.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.441+0000 7f5674376740 1 -- start start 2024-09-19T08:56:50.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.441+0000 7f5674376740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c57e0dd4a0 0x55c57e0ffdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:50.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.441+0000 7f5674376740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c57e0dddc0 0x55c57e100330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:50.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.441+0000 7f5674376740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c57e0e8680 0x55c57e0e2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:50.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.441+0000 7f5674376740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f566c1ad790 con 0x55c57e0dd4a0 2024-09-19T08:56:50.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.441+0000 7f5674376740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f566c1af2d0 con 0x55c57e0dddc0 2024-09-19T08:56:50.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.441+0000 7f5674376740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f566c1ade60 con 0x55c57e0e8680 2024-09-19T08:56:50.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.441+0000 7f566b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c57e0dd4a0 0x55c57e0ffdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:50.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.441+0000 7f566b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c57e0dd4a0 0x55c57e0ffdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52514/0 (socket says 172.21.15.164:52514) 2024-09-19T08:56:50.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.441+0000 7f566b7fe640 1 -- 172.21.15.164:0/3205462222 learned_addr learned my addr 172.21.15.164:0/3205462222 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:50.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.441+0000 7f5663fff640 1 --2- 172.21.15.164:0/3205462222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c57e0dddc0 0x55c57e100330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:50.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.441+0000 7f566b7fe640 1 -- 172.21.15.164:0/3205462222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c57e0e8680 msgr2=0x55c57e0e2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:50.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.441+0000 7f566b7fe640 1 --2- 172.21.15.164:0/3205462222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c57e0e8680 0x55c57e0e2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.441+0000 7f566b7fe640 1 -- 172.21.15.164:0/3205462222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c57e0dddc0 msgr2=0x55c57e100330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:50.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.441+0000 7f566b7fe640 1 --2- 172.21.15.164:0/3205462222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c57e0dddc0 0x55c57e100330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.441+0000 7f566b7fe640 1 -- 172.21.15.164:0/3205462222 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c57e0e42b0 con 0x55c57e0dd4a0 2024-09-19T08:56:50.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.442+0000 7f566b7fe640 1 --2- 172.21.15.164:0/3205462222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c57e0dd4a0 0x55c57e0ffdf0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f565c000c00 tx=0x7f565c0098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b0258232200bee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:50.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.442+0000 7f56617fa640 1 -- 172.21.15.164:0/3205462222 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c035030 con 0x55c57e0dd4a0 2024-09-19T08:56:50.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.442+0000 7f56617fa640 1 -- 172.21.15.164:0/3205462222 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f565c003890 con 0x55c57e0dd4a0 2024-09-19T08:56:50.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.442+0000 7f56617fa640 1 -- 172.21.15.164:0/3205462222 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c004c90 con 0x55c57e0dd4a0 2024-09-19T08:56:50.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.443+0000 7f5674376740 1 -- 172.21.15.164:0/3205462222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c57e0dd4a0 msgr2=0x55c57e0ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:50.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.443+0000 7f5674376740 1 --2- 172.21.15.164:0/3205462222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c57e0dd4a0 0x55c57e0ffdf0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f565c000c00 tx=0x7f565c0098b0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.443+0000 7f5674376740 1 -- 172.21.15.164:0/3205462222 shutdown_connections 2024-09-19T08:56:50.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.443+0000 7f5674376740 1 --2- 172.21.15.164:0/3205462222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c57e0e8680 0x55c57e0e2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.443+0000 7f5674376740 1 --2- 172.21.15.164:0/3205462222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c57e0dddc0 0x55c57e100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.443+0000 7f5674376740 1 --2- 172.21.15.164:0/3205462222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c57e0dd4a0 0x55c57e0ffdf0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.443+0000 7f5674376740 1 -- 172.21.15.164:0/3205462222 >> 172.21.15.164:0/3205462222 conn(0x55c57e0d9a30 msgr2=0x55c57e0d8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:50.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.443+0000 7f5674376740 1 -- 172.21.15.164:0/3205462222 shutdown_connections 2024-09-19T08:56:50.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.443+0000 7f5674376740 1 -- 172.21.15.164:0/3205462222 wait complete. 2024-09-19T08:56:50.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.443+0000 7f5674376740 1 -- start start 2024-09-19T08:56:50.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.443+0000 7f5674376740 1 Processor -- start 2024-09-19T08:56:50.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.443+0000 7f5674376740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c57e0dd4a0 0x55c57e0eead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:50.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.443+0000 7f5674376740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c57e0dddc0 0x55c57e0ef010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:50.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.443+0000 7f5674376740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c57e0e8680 0x55c57e161dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:50.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.443+0000 7f5674376740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f566c1aec00 con 0x55c57e0dd4a0 2024-09-19T08:56:50.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.443+0000 7f5674376740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f566c1ae530 con 0x55c57e0dddc0 2024-09-19T08:56:50.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.443+0000 7f5674376740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f566c1ab790 con 0x55c57e0e8680 2024-09-19T08:56:50.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.444+0000 7f5674376740 10 client.0 _ll_register_callbacks cb 0x7f565c0314d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:56:50.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.444+0000 7f566b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c57e0dd4a0 0x55c57e0eead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:50.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.444+0000 7f566b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c57e0dd4a0 0x55c57e0eead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52530/0 (socket says 172.21.15.164:52530) 2024-09-19T08:56:50.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.444+0000 7f566b7fe640 1 -- 172.21.15.164:0/3134988147 learned_addr learned my addr 172.21.15.164:0/3134988147 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:50.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.444+0000 7f566bfff640 1 --2- 172.21.15.164:0/3134988147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c57e0e8680 0x55c57e161dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:50.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.444+0000 7f5663fff640 1 --2- 172.21.15.164:0/3134988147 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c57e0dddc0 0x55c57e0ef010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:50.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.444+0000 7f566b7fe640 1 -- 172.21.15.164:0/3134988147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c57e0e8680 msgr2=0x55c57e161dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:50.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.444+0000 7f566b7fe640 1 --2- 172.21.15.164:0/3134988147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c57e0e8680 0x55c57e161dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.444+0000 7f566b7fe640 1 -- 172.21.15.164:0/3134988147 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c57e0dddc0 msgr2=0x55c57e0ef010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:50.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.444+0000 7f566b7fe640 1 --2- 172.21.15.164:0/3134988147 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c57e0dddc0 0x55c57e0ef010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.444+0000 7f566b7fe640 1 -- 172.21.15.164:0/3134988147 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c57e0e5c60 con 0x55c57e0dd4a0 2024-09-19T08:56:50.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.444+0000 7f566b7fe640 1 --2- 172.21.15.164:0/3134988147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c57e0dd4a0 0x55c57e0eead0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f565c0163f0 tx=0x7f565c003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d0b010e06b8d77d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:50.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.444+0000 7f564affd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:56:50.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.444+0000 7f564affd640 1 -- 172.21.15.164:0/3134988147 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c035070 con 0x55c57e0dd4a0 2024-09-19T08:56:50.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.444+0000 7f564affd640 1 -- 172.21.15.164:0/3134988147 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f565c005410 con 0x55c57e0dd4a0 2024-09-19T08:56:50.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.444+0000 7f5674376740 1 -- 172.21.15.164:0/3134988147 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f565c003890 con 0x55c57e0dd4a0 2024-09-19T08:56:50.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.445+0000 7f564affd640 1 -- 172.21.15.164:0/3134988147 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f565c065050 con 0x55c57e0dd4a0 2024-09-19T08:56:50.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.445+0000 7f5674376740 10 client.17808 fetch_fsmap learned FSMap version 32 2024-09-19T08:56:50.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.445+0000 7f5674376740 1 -- 172.21.15.164:0/3134988147 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c57e0ddbf0 con 0x55c57e0dd4a0 2024-09-19T08:56:50.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.445+0000 7f564affd640 1 -- 172.21.15.164:0/3134988147 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f565c003890 con 0x55c57e0dd4a0 2024-09-19T08:56:50.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.445+0000 7f564affd640 1 -- 172.21.15.164:0/3134988147 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f565c045590 con 0x55c57e0dd4a0 2024-09-19T08:56:50.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.445+0000 7f5674376740 10 client.17808 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:56:50.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.445+0000 7f5674376740 10 client.17808 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:56:50.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.445+0000 7f5674376740 20 client.17808 populate_metadata read hostname 'smithi164' 2024-09-19T08:56:50.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.445+0000 7f5674376740 1 --2- 172.21.15.164:0/3134988147 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c57e163840 0x55c57e165c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:50.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.445+0000 7f5674376740 4 client.17808 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:56:50.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.445+0000 7f5674376740 1 -- 172.21.15.164:0/3134988147 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c57e04ad60 con 0x55c57e163840 2024-09-19T08:56:50.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.445+0000 7f5663fff640 1 --2- 172.21.15.164:0/3134988147 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c57e163840 0x55c57e165c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:50.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.446+0000 7f564affd640 10 client.17808 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:56:50.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.446+0000 7f5663fff640 1 --2- 172.21.15.164:0/3134988147 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c57e163840 0x55c57e165c30 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e730619495f82d1 server_cookie=f22760d2a5f17edf in_seq=0 out_seq=0 2024-09-19T08:56:50.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.446+0000 7f564affd640 1 -- 172.21.15.164:0/3134988147 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c57e04ad60 con 0x55c57e163840 2024-09-19T08:56:50.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.446+0000 7f564affd640 10 client.17808 handle_command_reply: tid=0 2024-09-19T08:56:50.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.446+0000 7f5674376740 1 client.17808 shutdown 2024-09-19T08:56:50.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.446+0000 7f5674376740 1 -- 172.21.15.164:0/3134988147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c57e0dd4a0 msgr2=0x55c57e0eead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:50.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.446+0000 7f5674376740 1 --2- 172.21.15.164:0/3134988147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c57e0dd4a0 0x55c57e0eead0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f565c0163f0 tx=0x7f565c003ae0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.446+0000 7f5674376740 1 -- 172.21.15.164:0/3134988147 shutdown_connections 2024-09-19T08:56:50.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.446+0000 7f5674376740 1 --2- 172.21.15.164:0/3134988147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c57e0e8680 0x55c57e161dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.446+0000 7f5674376740 1 --2- 172.21.15.164:0/3134988147 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c57e163840 0x55c57e165c30 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.446+0000 7f5674376740 1 --2- 172.21.15.164:0/3134988147 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c57e0dddc0 0x55c57e0ef010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.446+0000 7f5674376740 1 --2- 172.21.15.164:0/3134988147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c57e0dd4a0 0x55c57e0eead0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.446+0000 7f5674376740 1 -- 172.21.15.164:0/3134988147 >> 172.21.15.164:0/3134988147 conn(0x55c57e0d9a30 msgr2=0x55c57e0e6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:50.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.446+0000 7f564affd640 0 client.17808 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:56:50.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.446+0000 7f5674376740 1 -- 172.21.15.164:0/3134988147 shutdown_connections 2024-09-19T08:56:50.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.446+0000 7f5674376740 1 -- 172.21.15.164:0/3134988147 wait complete. 2024-09-19T08:56:50.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.446+0000 7f5674376740 20 client.17808 trim_cache size 0 max 16384 2024-09-19T08:56:50.458 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:56:50.458 INFO:teuthology.orchestra.run.smithi164.stdout: "status": "no active scrubs running", 2024-09-19T08:56:50.458 INFO:teuthology.orchestra.run.smithi164.stdout: "scrubs": {} 2024-09-19T08:56:50.458 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:56:50.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.447+0000 7f5671f9d640 1 -- 172.21.15.164:0/800023015 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f56580a33b0 msgr2=0x7f56580a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:50.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.447+0000 7f5671f9d640 1 --2- 172.21.15.164:0/800023015 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f56580a33b0 0x7f56580a5870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5654011cb0 tx=0x7f5654016000 comp rx=0 tx=0).stop 2024-09-19T08:56:50.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.447+0000 7f5671f9d640 1 -- 172.21.15.164:0/800023015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f566c106a80 msgr2=0x7f566c1a43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:50.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.447+0000 7f5671f9d640 1 --2- 172.21.15.164:0/800023015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f566c106a80 0x7f566c1a43f0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f566400cad0 tx=0x7f566400cf90 comp rx=0 tx=0).stop 2024-09-19T08:56:50.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.447+0000 7f5671f9d640 1 -- 172.21.15.164:0/800023015 shutdown_connections 2024-09-19T08:56:50.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.447+0000 7f5671f9d640 1 --2- 172.21.15.164:0/800023015 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f56580a33b0 0x7f56580a5870 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.447+0000 7f5671f9d640 1 --2- 172.21.15.164:0/800023015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f566c106a80 0x7f566c1a43f0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.447+0000 7f5671f9d640 1 --2- 172.21.15.164:0/800023015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f566c101420 0x7f566c1998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.447+0000 7f5671f9d640 1 --2- 172.21.15.164:0/800023015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f566c100a80 0x7f566c199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:50.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.447+0000 7f5671f9d640 1 -- 172.21.15.164:0/800023015 >> 172.21.15.164:0/800023015 conn(0x7f566c0fc7a0 msgr2=0x7f566c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:50.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.447+0000 7f5671f9d640 1 -- 172.21.15.164:0/800023015 shutdown_connections 2024-09-19T08:56:50.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:50.447+0000 7f5671f9d640 1 -- 172.21.15.164:0/800023015 wait complete. 2024-09-19T08:56:50.624 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:50 smithi170 ceph-mon[35072]: pgmap v343: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:56:50.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:50 smithi173 ceph-mon[35306]: pgmap v343: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:56:51.331 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T08:56:51.332 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:56:51.587 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:56:52.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.037+0000 7f1200260640 1 -- 172.21.15.164:0/3747611992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11f81027a0 msgr2=0x7f11f81067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:52.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.037+0000 7f1200260640 1 --2- 172.21.15.164:0/3747611992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11f81027a0 0x7f11f81067f0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009900 tx=0x7f11e802a410 comp rx=0 tx=0).stop 2024-09-19T08:56:52.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.037+0000 7f1200260640 1 -- 172.21.15.164:0/3747611992 shutdown_connections 2024-09-19T08:56:52.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.037+0000 7f1200260640 1 --2- 172.21.15.164:0/3747611992 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11f8106f20 0x7f11f810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:52.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.037+0000 7f1200260640 1 --2- 172.21.15.164:0/3747611992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11f81027a0 0x7f11f81067f0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:52.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.037+0000 7f1200260640 1 --2- 172.21.15.164:0/3747611992 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11f8101df0 0x7f11f81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:52.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.037+0000 7f1200260640 1 -- 172.21.15.164:0/3747611992 >> 172.21.15.164:0/3747611992 conn(0x7f11f80fd8b0 msgr2=0x7f11f80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:52.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.037+0000 7f1200260640 1 -- 172.21.15.164:0/3747611992 shutdown_connections 2024-09-19T08:56:52.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.037+0000 7f1200260640 1 -- 172.21.15.164:0/3747611992 wait complete. 2024-09-19T08:56:52.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.038+0000 7f1200260640 1 Processor -- start 2024-09-19T08:56:52.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.038+0000 7f1200260640 1 -- start start 2024-09-19T08:56:52.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.038+0000 7f1200260640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11f8101df0 0x7f11f80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:52.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.038+0000 7f1200260640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11f81027a0 0x7f11f8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:52.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.038+0000 7f1200260640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11f8106f20 0x7f11f8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:52.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.038+0000 7f1200260640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f11f8111f80 con 0x7f11f8106f20 2024-09-19T08:56:52.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.038+0000 7f1200260640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f11f8111e00 con 0x7f11f81027a0 2024-09-19T08:56:52.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.038+0000 7f1200260640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f11f8112100 con 0x7f11f8101df0 2024-09-19T08:56:52.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.038+0000 7f11fe7fd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11f8106f20 0x7f11f8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:52.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.038+0000 7f11fe7fd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11f8106f20 0x7f11f8074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:52550/0 (socket says 172.21.15.164:52550) 2024-09-19T08:56:52.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.038+0000 7f11fe7fd640 1 -- 172.21.15.164:0/3028626940 learned_addr learned my addr 172.21.15.164:0/3028626940 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:52.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.038+0000 7f11fdffc640 1 --2- 172.21.15.164:0/3028626940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11f8101df0 0x7f11f80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:52.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.038+0000 7f11fd7fb640 1 --2- 172.21.15.164:0/3028626940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11f81027a0 0x7f11f8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:52.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.038+0000 7f11fe7fd640 1 -- 172.21.15.164:0/3028626940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11f8101df0 msgr2=0x7f11f80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:52.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.038+0000 7f11fe7fd640 1 --2- 172.21.15.164:0/3028626940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11f8101df0 0x7f11f80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:52.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.038+0000 7f11fe7fd640 1 -- 172.21.15.164:0/3028626940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11f81027a0 msgr2=0x7f11f8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:52.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.038+0000 7f11fe7fd640 1 --2- 172.21.15.164:0/3028626940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11f81027a0 0x7f11f8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:52.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.038+0000 7f11fe7fd640 1 -- 172.21.15.164:0/3028626940 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11f81a68a0 con 0x7f11f8106f20 2024-09-19T08:56:52.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.038+0000 7f11fe7fd640 1 --2- 172.21.15.164:0/3028626940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11f8106f20 0x7f11f8074f70 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f11f000c970 tx=0x7f11f000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:52.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.039+0000 7f11eeffd640 1 -- 172.21.15.164:0/3028626940 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f0007bb0 con 0x7f11f8106f20 2024-09-19T08:56:52.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.039+0000 7f11eeffd640 1 -- 172.21.15.164:0/3028626940 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11f0007d50 con 0x7f11f8106f20 2024-09-19T08:56:52.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.039+0000 7f1200260640 1 -- 172.21.15.164:0/3028626940 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11f81a6b90 con 0x7f11f8106f20 2024-09-19T08:56:52.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.039+0000 7f11eeffd640 1 -- 172.21.15.164:0/3028626940 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f0018800 con 0x7f11f8106f20 2024-09-19T08:56:52.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.039+0000 7f1200260640 1 -- 172.21.15.164:0/3028626940 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11f81a7080 con 0x7f11f8106f20 2024-09-19T08:56:52.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.040+0000 7f1200260640 1 -- 172.21.15.164:0/3028626940 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11f81021d0 con 0x7f11f8106f20 2024-09-19T08:56:52.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.041+0000 7f11eeffd640 1 -- 172.21.15.164:0/3028626940 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f11f000fa50 con 0x7f11f8106f20 2024-09-19T08:56:52.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.041+0000 7f11eeffd640 1 --2- 172.21.15.164:0/3028626940 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f11e40a33b0 0x7f11e40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:52.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.042+0000 7f11fdffc640 1 --2- 172.21.15.164:0/3028626940 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f11e40a33b0 0x7f11e40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:52.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.042+0000 7f11eeffd640 1 -- 172.21.15.164:0/3028626940 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f11f00cfec0 con 0x7f11f8106f20 2024-09-19T08:56:52.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.042+0000 7f11fdffc640 1 --2- 172.21.15.164:0/3028626940 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f11e40a33b0 0x7f11e40a5870 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f11f4005fd0 tx=0x7f11f4009680 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:52.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.045+0000 7f11eeffd640 1 -- 172.21.15.164:0/3028626940 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f11f0014030 con 0x7f11f8106f20 2024-09-19T08:56:52.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.248+0000 7f1200260640 1 -- 172.21.15.164:0/3028626940 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11f802d7c0 con 0x7f11f8106f20 2024-09-19T08:56:52.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.249+0000 7f11eeffd640 1 -- 172.21.15.164:0/3028626940 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11984 (secure 0 0 0) 0x7f11f00bae20 con 0x7f11f8106f20 2024-09-19T08:56:52.249 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:56:52.250 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":32,"btime":"2024-09-19T08:52:27:192804+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:52:27.074168+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:56:52.250 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 32 2024-09-19T08:56:52.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.251+0000 7f1200260640 1 -- 172.21.15.164:0/3028626940 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f11e40a33b0 msgr2=0x7f11e40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:52.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.251+0000 7f1200260640 1 --2- 172.21.15.164:0/3028626940 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f11e40a33b0 0x7f11e40a5870 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f11f4005fd0 tx=0x7f11f4009680 comp rx=0 tx=0).stop 2024-09-19T08:56:52.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.251+0000 7f1200260640 1 -- 172.21.15.164:0/3028626940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11f8106f20 msgr2=0x7f11f8074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:52.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.251+0000 7f1200260640 1 --2- 172.21.15.164:0/3028626940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11f8106f20 0x7f11f8074f70 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f11f000c970 tx=0x7f11f000ce30 comp rx=0 tx=0).stop 2024-09-19T08:56:52.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.251+0000 7f1200260640 1 -- 172.21.15.164:0/3028626940 shutdown_connections 2024-09-19T08:56:52.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.251+0000 7f1200260640 1 --2- 172.21.15.164:0/3028626940 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f11e40a33b0 0x7f11e40a5870 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:52.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.251+0000 7f1200260640 1 --2- 172.21.15.164:0/3028626940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11f8106f20 0x7f11f8074f70 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:52.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.251+0000 7f1200260640 1 --2- 172.21.15.164:0/3028626940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11f81027a0 0x7f11f8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:52.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.251+0000 7f1200260640 1 --2- 172.21.15.164:0/3028626940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11f8101df0 0x7f11f80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:52.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.251+0000 7f1200260640 1 -- 172.21.15.164:0/3028626940 >> 172.21.15.164:0/3028626940 conn(0x7f11f80fd8b0 msgr2=0x7f11f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:52.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.251+0000 7f1200260640 1 -- 172.21.15.164:0/3028626940 shutdown_connections 2024-09-19T08:56:52.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:52.251+0000 7f1200260640 1 -- 172.21.15.164:0/3028626940 wait complete. 2024-09-19T08:56:52.531 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:52 smithi164 ceph-mon[30928]: pgmap v344: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:56:52.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:52 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3028626940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:56:52.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:52 smithi170 ceph-mon[35072]: pgmap v344: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:56:52.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:52 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3028626940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:56:52.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:52 smithi173 ceph-mon[35306]: pgmap v344: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:56:52.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:52 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3028626940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:56:52.961 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T08:56:53.207 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:56:53.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.688+0000 7f89d2bd8640 1 -- 172.21.15.164:0/62102980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89cc100a80 msgr2=0x7f89cc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:53.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.688+0000 7f89d2bd8640 1 --2- 172.21.15.164:0/62102980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89cc100a80 0x7f89cc100ee0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f89b8009930 tx=0x7f89b802a420 comp rx=0 tx=0).stop 2024-09-19T08:56:53.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.688+0000 7f89d2bd8640 1 -- 172.21.15.164:0/62102980 shutdown_connections 2024-09-19T08:56:53.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.688+0000 7f89d2bd8640 1 --2- 172.21.15.164:0/62102980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89cc101420 0x7f89cc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:53.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.688+0000 7f89d2bd8640 1 --2- 172.21.15.164:0/62102980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89cc100a80 0x7f89cc100ee0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:53.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.688+0000 7f89d2bd8640 1 --2- 172.21.15.164:0/62102980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89cc106a80 0x7f89cc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:53.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.688+0000 7f89d2bd8640 1 -- 172.21.15.164:0/62102980 >> 172.21.15.164:0/62102980 conn(0x7f89cc0fc7a0 msgr2=0x7f89cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:53.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.688+0000 7f89d2bd8640 1 -- 172.21.15.164:0/62102980 shutdown_connections 2024-09-19T08:56:53.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.688+0000 7f89d2bd8640 1 -- 172.21.15.164:0/62102980 wait complete. 2024-09-19T08:56:53.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.689+0000 7f89d2bd8640 1 Processor -- start 2024-09-19T08:56:53.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.689+0000 7f89d2bd8640 1 -- start start 2024-09-19T08:56:53.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.689+0000 7f89d2bd8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89cc100a80 0x7f89cc198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:53.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.689+0000 7f89d2bd8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89cc101420 0x7f89cc198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:53.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.689+0000 7f89d2bd8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89cc106a80 0x7f89cc19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:53.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.689+0000 7f89d2bd8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc110e80 con 0x7f89cc101420 2024-09-19T08:56:53.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.689+0000 7f89d2bd8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc110d00 con 0x7f89cc106a80 2024-09-19T08:56:53.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.689+0000 7f89d2bd8640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc111000 con 0x7f89cc100a80 2024-09-19T08:56:53.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.689+0000 7f89cbfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89cc101420 0x7f89cc198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:53.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.689+0000 7f89cbfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89cc101420 0x7f89cc198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44208/0 (socket says 172.21.15.164:44208) 2024-09-19T08:56:53.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.689+0000 7f89cbfff640 1 -- 172.21.15.164:0/3063270757 learned_addr learned my addr 172.21.15.164:0/3063270757 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:53.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.689+0000 7f89d0974640 1 --2- 172.21.15.164:0/3063270757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89cc100a80 0x7f89cc198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:53.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.689+0000 7f89d1175640 1 --2- 172.21.15.164:0/3063270757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89cc106a80 0x7f89cc19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:53.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.689+0000 7f89cbfff640 1 -- 172.21.15.164:0/3063270757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89cc100a80 msgr2=0x7f89cc198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:53.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.689+0000 7f89cbfff640 1 --2- 172.21.15.164:0/3063270757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89cc100a80 0x7f89cc198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:53.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.689+0000 7f89cbfff640 1 -- 172.21.15.164:0/3063270757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89cc106a80 msgr2=0x7f89cc19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:53.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.689+0000 7f89cbfff640 1 --2- 172.21.15.164:0/3063270757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89cc106a80 0x7f89cc19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:53.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.689+0000 7f89cbfff640 1 -- 172.21.15.164:0/3063270757 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89cc19d3e0 con 0x7f89cc101420 2024-09-19T08:56:53.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.690+0000 7f89cbfff640 1 --2- 172.21.15.164:0/3063270757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89cc101420 0x7f89cc198970 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f89b8009230 tx=0x7f89b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:53.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.690+0000 7f89c9ffb640 1 -- 172.21.15.164:0/3063270757 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b803f030 con 0x7f89cc101420 2024-09-19T08:56:53.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.690+0000 7f89d2bd8640 1 -- 172.21.15.164:0/3063270757 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89cc19d670 con 0x7f89cc101420 2024-09-19T08:56:53.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.690+0000 7f89c9ffb640 1 -- 172.21.15.164:0/3063270757 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89b8003ad0 con 0x7f89cc101420 2024-09-19T08:56:53.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.690+0000 7f89c9ffb640 1 -- 172.21.15.164:0/3063270757 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8035070 con 0x7f89cc101420 2024-09-19T08:56:53.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.690+0000 7f89d2bd8640 1 -- 172.21.15.164:0/3063270757 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89cc1a4fb0 con 0x7f89cc101420 2024-09-19T08:56:53.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.692+0000 7f89c9ffb640 1 -- 172.21.15.164:0/3063270757 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f89b8042030 con 0x7f89cc101420 2024-09-19T08:56:53.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.693+0000 7f89d4fb1740 1 Processor -- start 2024-09-19T08:56:53.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.693+0000 7f89d4fb1740 1 -- start start 2024-09-19T08:56:53.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.693+0000 7f89d4fb1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e290afb40 0x555e290ae090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:53.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.693+0000 7f89c9ffb640 1 --2- 172.21.15.164:0/3063270757 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f89bc0a33b0 0x7f89bc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:53.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.693+0000 7f89d4fb1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555e290ae5d0 0x555e290aea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:53.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.693+0000 7f89d4fb1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e290b9370 0x555e290bb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:53.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.693+0000 7f89d4fb1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555e290142f0 con 0x555e290ae5d0 2024-09-19T08:56:53.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.693+0000 7f89d4fb1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555e28fbbcf0 con 0x555e290afb40 2024-09-19T08:56:53.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.693+0000 7f89d4fb1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555e290102b0 con 0x555e290b9370 2024-09-19T08:56:53.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.693+0000 7f89cbfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555e290ae5d0 0x555e290aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:53.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.693+0000 7f89d0974640 1 --2- 172.21.15.164:0/3063270757 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f89bc0a33b0 0x7f89bc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:53.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.693+0000 7f89cbfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555e290ae5d0 0x555e290aea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44220/0 (socket says 172.21.15.164:44220) 2024-09-19T08:56:53.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.693+0000 7f89cbfff640 1 -- 172.21.15.164:0/3386705157 learned_addr learned my addr 172.21.15.164:0/3386705157 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:53.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.693+0000 7f89c9ffb640 1 -- 172.21.15.164:0/3063270757 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f89b80f0120 con 0x7f89cc101420 2024-09-19T08:56:53.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.693+0000 7f89d1175640 1 --2- 172.21.15.164:0/3386705157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e290afb40 0x555e290ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:53.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.693+0000 7f89d1175640 1 --2- 172.21.15.164:0/3386705157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e290b9370 0x555e290bb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:53.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.693+0000 7f89cbfff640 1 -- 172.21.15.164:0/3386705157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e290b9370 msgr2=0x555e290bb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:53.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.693+0000 7f89cbfff640 1 --2- 172.21.15.164:0/3386705157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e290b9370 0x555e290bb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:53.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.694+0000 7f89cbfff640 1 -- 172.21.15.164:0/3386705157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e290afb40 msgr2=0x555e290ae090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:53.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.694+0000 7f89cbfff640 1 --2- 172.21.15.164:0/3386705157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e290afb40 0x555e290ae090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:53.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.694+0000 7f89cbfff640 1 -- 172.21.15.164:0/3386705157 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e28fa03a0 con 0x555e290ae5d0 2024-09-19T08:56:53.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.694+0000 7f89d0974640 1 --2- 172.21.15.164:0/3063270757 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f89bc0a33b0 0x7f89bc0a5870 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f89c000bb60 tx=0x7f89c000b390 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:53.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.694+0000 7f89cbfff640 1 --2- 172.21.15.164:0/3386705157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555e290ae5d0 0x555e290aea10 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f89b80b8ec0 tx=0x7f89b803b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c28f05e76fc9c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:53.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.694+0000 7f89b6ffd640 1 -- 172.21.15.164:0/3386705157 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8049040 con 0x555e290ae5d0 2024-09-19T08:56:53.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.694+0000 7f89b6ffd640 1 -- 172.21.15.164:0/3386705157 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89b80bc4f0 con 0x555e290ae5d0 2024-09-19T08:56:53.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.694+0000 7f89b6ffd640 1 -- 172.21.15.164:0/3386705157 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b80edb30 con 0x555e290ae5d0 2024-09-19T08:56:53.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.695+0000 7f89d4fb1740 1 -- 172.21.15.164:0/3386705157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555e290ae5d0 msgr2=0x555e290aea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:53.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.695+0000 7f89d4fb1740 1 --2- 172.21.15.164:0/3386705157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555e290ae5d0 0x555e290aea10 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f89b80b8ec0 tx=0x7f89b803b000 comp rx=0 tx=0).stop 2024-09-19T08:56:53.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.695+0000 7f89d4fb1740 1 -- 172.21.15.164:0/3386705157 shutdown_connections 2024-09-19T08:56:53.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.695+0000 7f89d4fb1740 1 --2- 172.21.15.164:0/3386705157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e290b9370 0x555e290bb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:53.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.695+0000 7f89d4fb1740 1 --2- 172.21.15.164:0/3386705157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555e290ae5d0 0x555e290aea10 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:53.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.695+0000 7f89d4fb1740 1 --2- 172.21.15.164:0/3386705157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e290afb40 0x555e290ae090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:53.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.695+0000 7f89d4fb1740 1 -- 172.21.15.164:0/3386705157 >> 172.21.15.164:0/3386705157 conn(0x555e29077310 msgr2=0x555e290aac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:53.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.695+0000 7f89d4fb1740 1 -- 172.21.15.164:0/3386705157 shutdown_connections 2024-09-19T08:56:53.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.695+0000 7f89d4fb1740 1 -- 172.21.15.164:0/3386705157 wait complete. 2024-09-19T08:56:53.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.695+0000 7f89d4fb1740 1 -- start start 2024-09-19T08:56:53.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.695+0000 7f89d4fb1740 1 Processor -- start 2024-09-19T08:56:53.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.696+0000 7f89d4fb1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e290ae5d0 0x555e290b8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:53.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.696+0000 7f89d4fb1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e290afb40 0x555e290b60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:53.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.696+0000 7f89d4fb1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555e290b9370 0x555e290b6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:53.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.696+0000 7f89d4fb1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555e290bbe30 con 0x555e290b9370 2024-09-19T08:56:53.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.696+0000 7f89d4fb1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555e28fcf060 con 0x555e290afb40 2024-09-19T08:56:53.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.696+0000 7f89d4fb1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555e290bbfb0 con 0x555e290ae5d0 2024-09-19T08:56:53.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.696+0000 7f89d4fb1740 10 client.0 _ll_register_callbacks cb 0x555e28d228c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:56:53.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.696+0000 7f89d1175640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555e290b9370 0x555e290b6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:53.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.696+0000 7f89d1175640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555e290b9370 0x555e290b6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44224/0 (socket says 172.21.15.164:44224) 2024-09-19T08:56:53.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.696+0000 7f89d1175640 1 -- 172.21.15.164:0/49192520 learned_addr learned my addr 172.21.15.164:0/49192520 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:53.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.696+0000 7f89d1175640 1 -- 172.21.15.164:0/49192520 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e290ae5d0 msgr2=0x555e290b8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:53.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.696+0000 7f89d1175640 1 --2- 172.21.15.164:0/49192520 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e290ae5d0 0x555e290b8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:53.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.696+0000 7f89cbfff640 1 --2- 172.21.15.164:0/49192520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e290afb40 0x555e290b60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:53.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.696+0000 7f89d1175640 1 -- 172.21.15.164:0/49192520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e290afb40 msgr2=0x555e290b60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:53.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.696+0000 7f89d1175640 1 --2- 172.21.15.164:0/49192520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e290afb40 0x555e290b60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:53.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.696+0000 7f89d1175640 1 -- 172.21.15.164:0/49192520 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e28d853c0 con 0x555e290b9370 2024-09-19T08:56:53.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.696+0000 7f89cbfff640 1 --2- 172.21.15.164:0/49192520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e290afb40 0x555e290b60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:56:53.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.696+0000 7f89d1175640 1 --2- 172.21.15.164:0/49192520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555e290b9370 0x555e290b6840 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f89c4012c50 tx=0x7f89c400c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=442589b2e8517198 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:53.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.696+0000 7f89abfff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:56:53.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.696+0000 7f89abfff640 1 -- 172.21.15.164:0/49192520 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c4004d20 con 0x555e290b9370 2024-09-19T08:56:53.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.696+0000 7f89abfff640 1 -- 172.21.15.164:0/49192520 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f89c4005020 con 0x555e290b9370 2024-09-19T08:56:53.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.696+0000 7f89d4fb1740 1 -- 172.21.15.164:0/49192520 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89b80bc4f0 con 0x555e290b9370 2024-09-19T08:56:53.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.696+0000 7f89abfff640 1 -- 172.21.15.164:0/49192520 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f89c4005220 con 0x555e290b9370 2024-09-19T08:56:53.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.697+0000 7f89d4fb1740 10 client.17826 fetch_fsmap learned FSMap version 32 2024-09-19T08:56:53.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.697+0000 7f89d4fb1740 1 -- 172.21.15.164:0/49192520 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555e290c7970 con 0x555e290b9370 2024-09-19T08:56:53.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.697+0000 7f89abfff640 1 -- 172.21.15.164:0/49192520 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89b80bc4f0 con 0x555e290b9370 2024-09-19T08:56:53.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.697+0000 7f89abfff640 1 -- 172.21.15.164:0/49192520 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f89c401c070 con 0x555e290b9370 2024-09-19T08:56:53.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.697+0000 7f89d4fb1740 10 client.17826 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:56:53.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.697+0000 7f89d4fb1740 10 client.17826 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-19T08:56:53.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.697+0000 7f89d4fb1740 20 client.17826 populate_metadata read hostname 'smithi164' 2024-09-19T08:56:53.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.697+0000 7f89d4fb1740 1 --2- 172.21.15.164:0/49192520 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x555e290c8ed0 0x555e290cb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:53.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.697+0000 7f89d4fb1740 4 client.17826 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:56:53.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.697+0000 7f89d4fb1740 1 -- 172.21.15.164:0/49192520 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e29021860 con 0x555e290c8ed0 2024-09-19T08:56:53.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.697+0000 7f89d0974640 1 --2- 172.21.15.164:0/49192520 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x555e290c8ed0 0x555e290cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:53.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.697+0000 7f89abfff640 10 client.17826 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:56:53.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.697+0000 7f89d0974640 1 --2- 172.21.15.164:0/49192520 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x555e290c8ed0 0x555e290cb370 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3a267dcc8180c18f server_cookie=43eaa7f4b5c8bb7 in_seq=0 out_seq=0 2024-09-19T08:56:53.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.698+0000 7f89abfff640 1 -- 172.21.15.164:0/49192520 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555e29021860 con 0x555e290c8ed0 2024-09-19T08:56:53.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.698+0000 7f89abfff640 10 client.17826 handle_command_reply: tid=0 2024-09-19T08:56:53.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.698+0000 7f89d4fb1740 1 client.17826 shutdown 2024-09-19T08:56:53.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.698+0000 7f89d4fb1740 1 -- 172.21.15.164:0/49192520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555e290b9370 msgr2=0x555e290b6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:53.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.698+0000 7f89d4fb1740 1 --2- 172.21.15.164:0/49192520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555e290b9370 0x555e290b6840 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f89c4012c50 tx=0x7f89c400c700 comp rx=0 tx=0).stop 2024-09-19T08:56:53.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.698+0000 7f89d4fb1740 1 -- 172.21.15.164:0/49192520 shutdown_connections 2024-09-19T08:56:53.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.698+0000 7f89d4fb1740 1 --2- 172.21.15.164:0/49192520 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x555e290c8ed0 0x555e290cb370 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:53.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.698+0000 7f89d4fb1740 1 --2- 172.21.15.164:0/49192520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555e290b9370 0x555e290b6840 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:53.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.698+0000 7f89d4fb1740 1 --2- 172.21.15.164:0/49192520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e290afb40 0x555e290b60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:53.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.698+0000 7f89d4fb1740 1 --2- 172.21.15.164:0/49192520 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e290ae5d0 0x555e290b8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:53.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.698+0000 7f89abfff640 0 client.17826 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:56:53.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.698+0000 7f89d4fb1740 1 -- 172.21.15.164:0/49192520 >> 172.21.15.164:0/49192520 conn(0x555e29077310 msgr2=0x555e290b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:53.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.698+0000 7f89d4fb1740 1 -- 172.21.15.164:0/49192520 shutdown_connections 2024-09-19T08:56:53.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.698+0000 7f89d4fb1740 1 -- 172.21.15.164:0/49192520 wait complete. 2024-09-19T08:56:53.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.698+0000 7f89d4fb1740 20 client.17826 trim_cache size 0 max 16384 2024-09-19T08:56:53.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.712+0000 7f89d4fb1740 1 Processor -- start 2024-09-19T08:56:53.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.712+0000 7f89d4fb1740 1 -- start start 2024-09-19T08:56:53.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.712+0000 7f89d4fb1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555e290ae5d0 0x555e290b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:53.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.712+0000 7f89d4fb1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e290afb40 0x555e290b1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:53.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.712+0000 7f89d4fb1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e290b9370 0x555e290b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:53.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.712+0000 7f89d4fb1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc1ad720 con 0x555e290ae5d0 2024-09-19T08:56:53.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.712+0000 7f89d4fb1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc1af260 con 0x555e290afb40 2024-09-19T08:56:53.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.712+0000 7f89d4fb1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc1addf0 con 0x555e290b9370 2024-09-19T08:56:53.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.712+0000 7f89d0974640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555e290ae5d0 0x555e290b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:53.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.712+0000 7f89d0974640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555e290ae5d0 0x555e290b5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44238/0 (socket says 172.21.15.164:44238) 2024-09-19T08:56:53.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.712+0000 7f89d0974640 1 -- 172.21.15.164:0/2365059032 learned_addr learned my addr 172.21.15.164:0/2365059032 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:53.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.712+0000 7f89d1175640 1 --2- 172.21.15.164:0/2365059032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e290b9370 0x555e290b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:53.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.712+0000 7f89cbfff640 1 --2- 172.21.15.164:0/2365059032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e290afb40 0x555e290b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:53.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.713+0000 7f89d0974640 1 -- 172.21.15.164:0/2365059032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e290b9370 msgr2=0x555e290b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:53.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.713+0000 7f89d0974640 1 --2- 172.21.15.164:0/2365059032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e290b9370 0x555e290b16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:53.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.713+0000 7f89d0974640 1 -- 172.21.15.164:0/2365059032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e290afb40 msgr2=0x555e290b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:53.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.713+0000 7f89d0974640 1 --2- 172.21.15.164:0/2365059032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e290afb40 0x555e290b1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:53.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.713+0000 7f89d0974640 1 -- 172.21.15.164:0/2365059032 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e290a9310 con 0x555e290ae5d0 2024-09-19T08:56:53.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.713+0000 7f89d0974640 1 --2- 172.21.15.164:0/2365059032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555e290ae5d0 0x555e290b5970 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x555e290cbbb0 tx=0x7f89c0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ebfd0402b04ac58 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:53.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.713+0000 7f89b67fc640 1 -- 172.21.15.164:0/2365059032 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c001e620 con 0x555e290ae5d0 2024-09-19T08:56:53.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.713+0000 7f89b67fc640 1 -- 172.21.15.164:0/2365059032 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89c001eca0 con 0x555e290ae5d0 2024-09-19T08:56:53.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.713+0000 7f89b67fc640 1 -- 172.21.15.164:0/2365059032 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c0025b90 con 0x555e290ae5d0 2024-09-19T08:56:53.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.714+0000 7f89d4fb1740 1 -- 172.21.15.164:0/2365059032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555e290ae5d0 msgr2=0x555e290b5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:53.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.714+0000 7f89d4fb1740 1 --2- 172.21.15.164:0/2365059032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555e290ae5d0 0x555e290b5970 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x555e290cbbb0 tx=0x7f89c0013000 comp rx=0 tx=0).stop 2024-09-19T08:56:53.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.714+0000 7f89d4fb1740 1 -- 172.21.15.164:0/2365059032 shutdown_connections 2024-09-19T08:56:53.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.714+0000 7f89d4fb1740 1 --2- 172.21.15.164:0/2365059032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e290b9370 0x555e290b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:53.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.714+0000 7f89d4fb1740 1 --2- 172.21.15.164:0/2365059032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e290afb40 0x555e290b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:53.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.714+0000 7f89d4fb1740 1 --2- 172.21.15.164:0/2365059032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555e290ae5d0 0x555e290b5970 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:53.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.714+0000 7f89d4fb1740 1 -- 172.21.15.164:0/2365059032 >> 172.21.15.164:0/2365059032 conn(0x555e29077310 msgr2=0x555e290b51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:53.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.714+0000 7f89d4fb1740 1 -- 172.21.15.164:0/2365059032 shutdown_connections 2024-09-19T08:56:53.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.714+0000 7f89d4fb1740 1 -- 172.21.15.164:0/2365059032 wait complete. 2024-09-19T08:56:53.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.714+0000 7f89d4fb1740 1 -- start start 2024-09-19T08:56:53.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.714+0000 7f89d4fb1740 1 Processor -- start 2024-09-19T08:56:53.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.715+0000 7f89d4fb1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555e290ae5d0 0x555e290bcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:53.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.715+0000 7f89d4fb1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e290afb40 0x555e290bd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:53.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.715+0000 7f89d4fb1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e290b9370 0x555e290bd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:53.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.715+0000 7f89d4fb1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc1aeb90 con 0x555e290ae5d0 2024-09-19T08:56:53.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.715+0000 7f89d4fb1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc1ae4c0 con 0x555e290afb40 2024-09-19T08:56:53.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.715+0000 7f89d0974640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555e290ae5d0 0x555e290bcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:53.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.715+0000 7f89d0974640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555e290ae5d0 0x555e290bcee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44254/0 (socket says 172.21.15.164:44254) 2024-09-19T08:56:53.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.715+0000 7f89d0974640 1 -- 172.21.15.164:0/2804954869 learned_addr learned my addr 172.21.15.164:0/2804954869 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:53.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.715+0000 7f89d4fb1740 1 -- 172.21.15.164:0/2804954869 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc1ab720 con 0x555e290b9370 2024-09-19T08:56:53.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.715+0000 7f89d4fb1740 10 client.0 _ll_register_callbacks cb 0x7f89c4012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:56:53.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.715+0000 7f89cbfff640 1 --2- 172.21.15.164:0/2804954869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e290afb40 0x555e290bd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:53.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.715+0000 7f89d0974640 1 -- 172.21.15.164:0/2804954869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e290b9370 msgr2=0x555e290bd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:53.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.715+0000 7f89d0974640 1 --2- 172.21.15.164:0/2804954869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e290b9370 0x555e290bd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:53.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.715+0000 7f89d0974640 1 -- 172.21.15.164:0/2804954869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e290afb40 msgr2=0x555e290bd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:53.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.715+0000 7f89d0974640 1 --2- 172.21.15.164:0/2804954869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e290afb40 0x555e290bd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:53.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.715+0000 7f89d0974640 1 -- 172.21.15.164:0/2804954869 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e290aa510 con 0x555e290ae5d0 2024-09-19T08:56:53.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.715+0000 7f89cbfff640 1 --2- 172.21.15.164:0/2804954869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e290afb40 0x555e290bd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:56:53.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.715+0000 7f89d0974640 1 --2- 172.21.15.164:0/2804954869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555e290ae5d0 0x555e290bcee0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f89c0010860 tx=0x7f89c0010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0238c785b4ecc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:53.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.715+0000 7f89abfff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:56:53.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.715+0000 7f89abfff640 1 -- 172.21.15.164:0/2804954869 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c003a060 con 0x555e290ae5d0 2024-09-19T08:56:53.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.715+0000 7f89abfff640 1 -- 172.21.15.164:0/2804954869 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f89c00257d0 con 0x555e290ae5d0 2024-09-19T08:56:53.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.715+0000 7f89d4fb1740 1 -- 172.21.15.164:0/2804954869 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89c001eca0 con 0x555e290ae5d0 2024-09-19T08:56:53.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.716+0000 7f89abfff640 1 -- 172.21.15.164:0/2804954869 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f89c00259d0 con 0x555e290ae5d0 2024-09-19T08:56:53.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.716+0000 7f89d4fb1740 10 client.17832 fetch_fsmap learned FSMap version 32 2024-09-19T08:56:53.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.716+0000 7f89d4fb1740 1 -- 172.21.15.164:0/2804954869 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555e29133c80 con 0x555e290ae5d0 2024-09-19T08:56:53.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.716+0000 7f89abfff640 1 -- 172.21.15.164:0/2804954869 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89c001eca0 con 0x555e290ae5d0 2024-09-19T08:56:53.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.716+0000 7f89abfff640 1 -- 172.21.15.164:0/2804954869 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f89c001e750 con 0x555e290ae5d0 2024-09-19T08:56:53.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.716+0000 7f89d4fb1740 10 client.17832 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:56:53.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.716+0000 7f89d4fb1740 10 client.17832 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-19T08:56:53.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.716+0000 7f89d4fb1740 20 client.17832 populate_metadata read hostname 'smithi164' 2024-09-19T08:56:53.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.716+0000 7f89d4fb1740 1 --2- 172.21.15.164:0/2804954869 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x555e291342e0 0x555e29136740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:53.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.716+0000 7f89d4fb1740 4 client.17832 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:56:53.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.716+0000 7f89d4fb1740 1 -- 172.21.15.164:0/2804954869 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555e2901bd70 con 0x555e291342e0 2024-09-19T08:56:53.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.716+0000 7f89cbfff640 1 --2- 172.21.15.164:0/2804954869 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x555e291342e0 0x555e29136740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:53.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.717+0000 7f89cbfff640 1 --2- 172.21.15.164:0/2804954869 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x555e291342e0 0x555e29136740 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6a31c5f5aab18ea3 server_cookie=e805c73f06a49558 in_seq=0 out_seq=0 2024-09-19T08:56:53.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.717+0000 7f89abfff640 10 client.17832 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:56:53.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.717+0000 7f89abfff640 1 -- 172.21.15.164:0/2804954869 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555e2901bd70 con 0x555e291342e0 2024-09-19T08:56:53.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.717+0000 7f89abfff640 10 client.17832 handle_command_reply: tid=0 2024-09-19T08:56:53.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.717+0000 7f89d4fb1740 1 client.17832 shutdown 2024-09-19T08:56:53.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.717+0000 7f89d4fb1740 1 -- 172.21.15.164:0/2804954869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555e290ae5d0 msgr2=0x555e290bcee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:53.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.717+0000 7f89d4fb1740 1 --2- 172.21.15.164:0/2804954869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555e290ae5d0 0x555e290bcee0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f89c0010860 tx=0x7f89c0010ce0 comp rx=0 tx=0).stop 2024-09-19T08:56:53.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.717+0000 7f89d4fb1740 1 -- 172.21.15.164:0/2804954869 shutdown_connections 2024-09-19T08:56:53.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.717+0000 7f89d4fb1740 1 --2- 172.21.15.164:0/2804954869 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x555e291342e0 0x555e29136740 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:53.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.717+0000 7f89d4fb1740 1 --2- 172.21.15.164:0/2804954869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e290b9370 0x555e290bd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:53.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.717+0000 7f89d4fb1740 1 --2- 172.21.15.164:0/2804954869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e290afb40 0x555e290bd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:53.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.717+0000 7f89d4fb1740 1 --2- 172.21.15.164:0/2804954869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555e290ae5d0 0x555e290bcee0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:53.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.717+0000 7f89d4fb1740 1 -- 172.21.15.164:0/2804954869 >> 172.21.15.164:0/2804954869 conn(0x555e29077310 msgr2=0x555e290a94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:53.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.717+0000 7f89abfff640 0 client.17832 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:56:53.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.717+0000 7f89d4fb1740 1 -- 172.21.15.164:0/2804954869 shutdown_connections 2024-09-19T08:56:53.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.717+0000 7f89d4fb1740 1 -- 172.21.15.164:0/2804954869 wait complete. 2024-09-19T08:56:53.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.717+0000 7f89d4fb1740 20 client.17832 trim_cache size 0 max 16384 2024-09-19T08:56:53.732 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:56:53.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.718+0000 7f89d2bd8640 1 -- 172.21.15.164:0/3063270757 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f89bc0a33b0 msgr2=0x7f89bc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:53.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.718+0000 7f89d2bd8640 1 --2- 172.21.15.164:0/3063270757 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f89bc0a33b0 0x7f89bc0a5870 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f89c000bb60 tx=0x7f89c000b390 comp rx=0 tx=0).stop 2024-09-19T08:56:53.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.718+0000 7f89d2bd8640 1 -- 172.21.15.164:0/3063270757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89cc101420 msgr2=0x7f89cc198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:53.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.718+0000 7f89d2bd8640 1 --2- 172.21.15.164:0/3063270757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89cc101420 0x7f89cc198970 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f89b8009230 tx=0x7f89b8004270 comp rx=0 tx=0).stop 2024-09-19T08:56:53.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.718+0000 7f89d2bd8640 1 -- 172.21.15.164:0/3063270757 shutdown_connections 2024-09-19T08:56:53.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.718+0000 7f89d2bd8640 1 --2- 172.21.15.164:0/3063270757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89cc106a80 0x7f89cc19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:53.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.718+0000 7f89d2bd8640 1 --2- 172.21.15.164:0/3063270757 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f89bc0a33b0 0x7f89bc0a5870 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:53.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.718+0000 7f89d2bd8640 1 --2- 172.21.15.164:0/3063270757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89cc101420 0x7f89cc198970 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:53.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.718+0000 7f89d2bd8640 1 --2- 172.21.15.164:0/3063270757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89cc100a80 0x7f89cc198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:53.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.718+0000 7f89d2bd8640 1 -- 172.21.15.164:0/3063270757 >> 172.21.15.164:0/3063270757 conn(0x7f89cc0fc7a0 msgr2=0x7f89cc06aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:53.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.718+0000 7f89d2bd8640 1 -- 172.21.15.164:0/3063270757 shutdown_connections 2024-09-19T08:56:53.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:53.718+0000 7f89d2bd8640 1 -- 172.21.15.164:0/3063270757 wait complete. 2024-09-19T08:56:54.447 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T08:56:54.624 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:54 smithi170 ceph-mon[35072]: pgmap v345: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:56:54.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:54 smithi173 ceph-mon[35306]: pgmap v345: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:56:54.702 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:56:54.781 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:54 smithi164 ceph-mon[30928]: pgmap v345: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:56:55.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.166+0000 7f5fb28db640 1 -- 172.21.15.164:0/3219962393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fac102790 msgr2=0x7f5fac1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:55.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.166+0000 7f5fb28db640 1 --2- 172.21.15.164:0/3219962393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fac102790 0x7f5fac1067e0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0009900 tx=0x7f5fa002a410 comp rx=0 tx=0).stop 2024-09-19T08:56:55.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.166+0000 7f5fb28db640 1 -- 172.21.15.164:0/3219962393 shutdown_connections 2024-09-19T08:56:55.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.166+0000 7f5fb28db640 1 --2- 172.21.15.164:0/3219962393 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fac106f10 0x7f5fac10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:55.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.166+0000 7f5fb28db640 1 --2- 172.21.15.164:0/3219962393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fac102790 0x7f5fac1067e0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:55.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.166+0000 7f5fb28db640 1 --2- 172.21.15.164:0/3219962393 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fac101de0 0x7f5fac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:55.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.166+0000 7f5fb28db640 1 -- 172.21.15.164:0/3219962393 >> 172.21.15.164:0/3219962393 conn(0x7f5fac0fd8c0 msgr2=0x7f5fac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:55.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.167+0000 7f5fb28db640 1 -- 172.21.15.164:0/3219962393 shutdown_connections 2024-09-19T08:56:55.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.167+0000 7f5fb28db640 1 -- 172.21.15.164:0/3219962393 wait complete. 2024-09-19T08:56:55.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.167+0000 7f5fb28db640 1 Processor -- start 2024-09-19T08:56:55.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.167+0000 7f5fb28db640 1 -- start start 2024-09-19T08:56:55.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.167+0000 7f5fb28db640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fac101de0 0x7f5fac198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:55.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.167+0000 7f5fb28db640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fac102790 0x7f5fac1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:55.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.168+0000 7f5fb28db640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fac106f10 0x7f5fac19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:55.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.168+0000 7f5fb28db640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac111f70 con 0x7f5fac101de0 2024-09-19T08:56:55.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.168+0000 7f5fb28db640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac111df0 con 0x7f5fac106f10 2024-09-19T08:56:55.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.168+0000 7f5fb28db640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac1120f0 con 0x7f5fac102790 2024-09-19T08:56:55.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.168+0000 7f5fabfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fac101de0 0x7f5fac198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:55.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.168+0000 7f5fabfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fac101de0 0x7f5fac198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44262/0 (socket says 172.21.15.164:44262) 2024-09-19T08:56:55.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.168+0000 7f5fabfff640 1 -- 172.21.15.164:0/3646811570 learned_addr learned my addr 172.21.15.164:0/3646811570 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:55.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.168+0000 7f5fab7fe640 1 --2- 172.21.15.164:0/3646811570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fac102790 0x7f5fac1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:55.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.168+0000 7f5fabfff640 1 -- 172.21.15.164:0/3646811570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fac102790 msgr2=0x7f5fac1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:55.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.168+0000 7f5fabfff640 1 --2- 172.21.15.164:0/3646811570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fac102790 0x7f5fac1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:55.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.168+0000 7f5fabfff640 1 -- 172.21.15.164:0/3646811570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fac106f10 msgr2=0x7f5fac19cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:55.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.168+0000 7f5fb0e78640 1 --2- 172.21.15.164:0/3646811570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fac106f10 0x7f5fac19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:55.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.168+0000 7f5fabfff640 1 --2- 172.21.15.164:0/3646811570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fac106f10 0x7f5fac19cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:55.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.168+0000 7f5fabfff640 1 -- 172.21.15.164:0/3646811570 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fac19d310 con 0x7f5fac101de0 2024-09-19T08:56:55.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.168+0000 7f5fb0e78640 1 --2- 172.21.15.164:0/3646811570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fac106f10 0x7f5fac19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:56:55.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.168+0000 7f5fabfff640 1 --2- 172.21.15.164:0/3646811570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fac101de0 0x7f5fac198360 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c00c920 tx=0x7f5f9c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:55.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.168+0000 7f5fa97fa640 1 -- 172.21.15.164:0/3646811570 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c007c70 con 0x7f5fac101de0 2024-09-19T08:56:55.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.168+0000 7f5fa97fa640 1 -- 172.21.15.164:0/3646811570 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f9c007e10 con 0x7f5fac101de0 2024-09-19T08:56:55.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.169+0000 7f5fb28db640 1 -- 172.21.15.164:0/3646811570 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fac19d600 con 0x7f5fac101de0 2024-09-19T08:56:55.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.169+0000 7f5fa97fa640 1 -- 172.21.15.164:0/3646811570 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c018890 con 0x7f5fac101de0 2024-09-19T08:56:55.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.169+0000 7f5fb28db640 1 -- 172.21.15.164:0/3646811570 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fac1a4ee0 con 0x7f5fac101de0 2024-09-19T08:56:55.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.171+0000 7f5fa97fa640 1 -- 172.21.15.164:0/3646811570 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f5f9c018a70 con 0x7f5fac101de0 2024-09-19T08:56:55.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.171+0000 7f5fb4cb4740 1 Processor -- start 2024-09-19T08:56:55.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.171+0000 7f5fb4cb4740 1 -- start start 2024-09-19T08:56:55.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.172+0000 7f5fb4cb4740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf56015b40 0x55bf56014090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:55.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.172+0000 7f5fb4cb4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf560145d0 0x55bf56014a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:55.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.172+0000 7f5fb4cb4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf56023780 0x55bf56025b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:55.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.172+0000 7f5fb4cb4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bf55f7a2f0 con 0x55bf56023780 2024-09-19T08:56:55.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.172+0000 7f5fb4cb4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55bf55f21cf0 con 0x55bf560145d0 2024-09-19T08:56:55.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.172+0000 7f5fb4cb4740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55bf55f762b0 con 0x55bf56015b40 2024-09-19T08:56:55.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.172+0000 7f5fa97fa640 1 --2- 172.21.15.164:0/3646811570 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f5f940a3430 0x7f5f940a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:55.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.172+0000 7f5fab7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf56023780 0x55bf56025b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:55.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.172+0000 7f5fab7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf56023780 0x55bf56025b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44276/0 (socket says 172.21.15.164:44276) 2024-09-19T08:56:55.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.172+0000 7f5fab7fe640 1 -- 172.21.15.164:0/2435971485 learned_addr learned my addr 172.21.15.164:0/2435971485 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:55.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.172+0000 7f5fb0e78640 1 --2- 172.21.15.164:0/2435971485 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf56015b40 0x55bf56014090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:55.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.172+0000 7f5fabfff640 1 --2- 172.21.15.164:0/2435971485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf560145d0 0x55bf56014a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:55.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.172+0000 7f5fb0e78640 1 --2- 172.21.15.164:0/3646811570 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f5f940a3430 0x7f5f940a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:55.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.172+0000 7f5fab7fe640 1 -- 172.21.15.164:0/2435971485 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf56015b40 msgr2=0x55bf56014090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:55.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.172+0000 7f5fab7fe640 1 --2- 172.21.15.164:0/2435971485 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf56015b40 0x55bf56014090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:55.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.172+0000 7f5fab7fe640 1 -- 172.21.15.164:0/2435971485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf560145d0 msgr2=0x55bf56014a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:55.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.172+0000 7f5fa97fa640 1 -- 172.21.15.164:0/3646811570 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f5f9c0cfda0 con 0x7f5fac101de0 2024-09-19T08:56:55.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.172+0000 7f5fab7fe640 1 --2- 172.21.15.164:0/2435971485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf560145d0 0x55bf56014a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:55.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.172+0000 7f5fab7fe640 1 -- 172.21.15.164:0/2435971485 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf55f063a0 con 0x55bf56023780 2024-09-19T08:56:55.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.172+0000 7f5fabfff640 1 --2- 172.21.15.164:0/2435971485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf560145d0 0x55bf56014a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:56:55.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.172+0000 7f5fb0e78640 1 --2- 172.21.15.164:0/3646811570 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f5f940a3430 0x7f5f940a58f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4005c40 tx=0x7f5fa4009200 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:55.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.172+0000 7f5fab7fe640 1 --2- 172.21.15.164:0/2435971485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf56023780 0x55bf56025b70 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0009900 tx=0x7f5fa0031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e240229c387615 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:55.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.173+0000 7f5f927fc640 1 -- 172.21.15.164:0/2435971485 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0040070 con 0x55bf56023780 2024-09-19T08:56:55.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.173+0000 7f5f927fc640 1 -- 172.21.15.164:0/2435971485 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fa0005020 con 0x55bf56023780 2024-09-19T08:56:55.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.173+0000 7f5f927fc640 1 -- 172.21.15.164:0/2435971485 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0035040 con 0x55bf56023780 2024-09-19T08:56:55.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.173+0000 7f5fb4cb4740 1 -- 172.21.15.164:0/2435971485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf56023780 msgr2=0x55bf56025b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:55.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.173+0000 7f5fb4cb4740 1 --2- 172.21.15.164:0/2435971485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf56023780 0x55bf56025b70 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0009900 tx=0x7f5fa0031630 comp rx=0 tx=0).stop 2024-09-19T08:56:55.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.173+0000 7f5fb4cb4740 1 -- 172.21.15.164:0/2435971485 shutdown_connections 2024-09-19T08:56:55.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.173+0000 7f5fb4cb4740 1 --2- 172.21.15.164:0/2435971485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf56023780 0x55bf56025b70 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:55.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.173+0000 7f5fb4cb4740 1 --2- 172.21.15.164:0/2435971485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf560145d0 0x55bf56014a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:55.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.173+0000 7f5fb4cb4740 1 --2- 172.21.15.164:0/2435971485 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf56015b40 0x55bf56014090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:55.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.173+0000 7f5fb4cb4740 1 -- 172.21.15.164:0/2435971485 >> 172.21.15.164:0/2435971485 conn(0x55bf55fdd310 msgr2=0x55bf56010c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:55.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.174+0000 7f5fb4cb4740 1 -- 172.21.15.164:0/2435971485 shutdown_connections 2024-09-19T08:56:55.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.174+0000 7f5fb4cb4740 1 -- 172.21.15.164:0/2435971485 wait complete. 2024-09-19T08:56:55.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.174+0000 7f5fb4cb4740 1 -- start start 2024-09-19T08:56:55.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.174+0000 7f5fb4cb4740 1 Processor -- start 2024-09-19T08:56:55.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.174+0000 7f5fb4cb4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf560145d0 0x55bf560230e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:55.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.174+0000 7f5fb4cb4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf56015b40 0x55bf56020390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:55.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.174+0000 7f5fb4cb4740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf56023780 0x55bf560208f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:55.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.174+0000 7f5fb4cb4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bf56026420 con 0x55bf56015b40 2024-09-19T08:56:55.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.174+0000 7f5fb4cb4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55bf55f35060 con 0x55bf560145d0 2024-09-19T08:56:55.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.174+0000 7f5fb4cb4740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55bf560265a0 con 0x55bf56023780 2024-09-19T08:56:55.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.174+0000 7f5fb4cb4740 10 client.0 _ll_register_callbacks cb 0x55bf55c888c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:56:55.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.174+0000 7f5fab7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf56015b40 0x55bf56020390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:55.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.175+0000 7f5fab7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf56015b40 0x55bf56020390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44286/0 (socket says 172.21.15.164:44286) 2024-09-19T08:56:55.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.175+0000 7f5fabfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf560145d0 0x55bf560230e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:55.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.175+0000 7f5fab7fe640 1 -- 172.21.15.164:0/745607332 learned_addr learned my addr 172.21.15.164:0/745607332 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:55.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.175+0000 7f5fb0e78640 1 --2- 172.21.15.164:0/745607332 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf56023780 0x55bf560208f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:55.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.175+0000 7f5fab7fe640 1 -- 172.21.15.164:0/745607332 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf56023780 msgr2=0x55bf560208f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:55.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.175+0000 7f5fab7fe640 1 --2- 172.21.15.164:0/745607332 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf56023780 0x55bf560208f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:55.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.175+0000 7f5fab7fe640 1 -- 172.21.15.164:0/745607332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf560145d0 msgr2=0x55bf560230e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:55.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.175+0000 7f5fab7fe640 1 --2- 172.21.15.164:0/745607332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf560145d0 0x55bf560230e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:55.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.175+0000 7f5fab7fe640 1 -- 172.21.15.164:0/745607332 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf55ceb3c0 con 0x55bf56015b40 2024-09-19T08:56:55.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.175+0000 7f5fab7fe640 1 --2- 172.21.15.164:0/745607332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf56015b40 0x55bf56020390 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f5fa002a450 tx=0x7f5fa0005400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38ee7ff484990 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:55.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.175+0000 7f5f8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:56:55.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.175+0000 7f5f8b7fe640 1 -- 172.21.15.164:0/745607332 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0040030 con 0x55bf56015b40 2024-09-19T08:56:55.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.175+0000 7f5f8b7fe640 1 -- 172.21.15.164:0/745607332 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f5fa0004940 con 0x55bf56015b40 2024-09-19T08:56:55.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.175+0000 7f5fb4cb4740 1 -- 172.21.15.164:0/745607332 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fa0005020 con 0x55bf56015b40 2024-09-19T08:56:55.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.175+0000 7f5f8b7fe640 1 -- 172.21.15.164:0/745607332 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5fa0035070 con 0x55bf56015b40 2024-09-19T08:56:55.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.175+0000 7f5fb4cb4740 10 client.17844 fetch_fsmap learned FSMap version 32 2024-09-19T08:56:55.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.175+0000 7f5fb4cb4740 1 -- 172.21.15.164:0/745607332 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bf56031e30 con 0x55bf56015b40 2024-09-19T08:56:55.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.176+0000 7f5f8b7fe640 1 -- 172.21.15.164:0/745607332 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fa0005020 con 0x55bf56015b40 2024-09-19T08:56:55.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.176+0000 7f5f8b7fe640 1 -- 172.21.15.164:0/745607332 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f5fa0057c10 con 0x55bf56015b40 2024-09-19T08:56:55.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.176+0000 7f5fb4cb4740 10 client.17844 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:56:55.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.176+0000 7f5fb4cb4740 10 client.17844 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T08:56:55.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.176+0000 7f5fb4cb4740 20 client.17844 populate_metadata read hostname 'smithi164' 2024-09-19T08:56:55.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.176+0000 7f5fb4cb4740 1 --2- 172.21.15.164:0/745607332 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55bf56033180 0x55bf56035570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:55.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.176+0000 7f5fb4cb4740 4 client.17844 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:56:55.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.176+0000 7f5fb4cb4740 1 -- 172.21.15.164:0/745607332 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf55f87860 con 0x55bf56033180 2024-09-19T08:56:55.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.176+0000 7f5fabfff640 1 --2- 172.21.15.164:0/745607332 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55bf56033180 0x55bf56035570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:55.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.177+0000 7f5f8b7fe640 10 client.17844 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:56:55.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.177+0000 7f5fabfff640 1 --2- 172.21.15.164:0/745607332 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55bf56033180 0x55bf56035570 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cb01278bc595318b server_cookie=b1b067a6220aab70 in_seq=0 out_seq=0 2024-09-19T08:56:55.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.177+0000 7f5f8b7fe640 1 -- 172.21.15.164:0/745607332 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf55f87860 con 0x55bf56033180 2024-09-19T08:56:55.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.177+0000 7f5f8b7fe640 10 client.17844 handle_command_reply: tid=0 2024-09-19T08:56:55.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.177+0000 7f5fb4cb4740 1 client.17844 shutdown 2024-09-19T08:56:55.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.177+0000 7f5fb4cb4740 1 -- 172.21.15.164:0/745607332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf56015b40 msgr2=0x55bf56020390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:55.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.177+0000 7f5fb4cb4740 1 --2- 172.21.15.164:0/745607332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf56015b40 0x55bf56020390 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f5fa002a450 tx=0x7f5fa0005400 comp rx=0 tx=0).stop 2024-09-19T08:56:55.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.178+0000 7f5fb4cb4740 1 -- 172.21.15.164:0/745607332 shutdown_connections 2024-09-19T08:56:55.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.178+0000 7f5fb4cb4740 1 --2- 172.21.15.164:0/745607332 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55bf56033180 0x55bf56035570 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:55.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.178+0000 7f5fb4cb4740 1 --2- 172.21.15.164:0/745607332 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf56023780 0x55bf560208f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:55.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.178+0000 7f5fb4cb4740 1 --2- 172.21.15.164:0/745607332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf56015b40 0x55bf56020390 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:55.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.178+0000 7f5f8b7fe640 0 client.17844 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:56:55.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.178+0000 7f5fb4cb4740 1 --2- 172.21.15.164:0/745607332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf560145d0 0x55bf560230e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:55.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.178+0000 7f5fb4cb4740 1 -- 172.21.15.164:0/745607332 >> 172.21.15.164:0/745607332 conn(0x55bf55fdd310 msgr2=0x55bf56016960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:55.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.178+0000 7f5fb4cb4740 1 -- 172.21.15.164:0/745607332 shutdown_connections 2024-09-19T08:56:55.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.178+0000 7f5fb4cb4740 1 -- 172.21.15.164:0/745607332 wait complete. 2024-09-19T08:56:55.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.178+0000 7f5fb4cb4740 20 client.17844 trim_cache size 0 max 16384 2024-09-19T08:56:55.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.190+0000 7f5fb4cb4740 1 Processor -- start 2024-09-19T08:56:55.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.190+0000 7f5fb4cb4740 1 -- start start 2024-09-19T08:56:55.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.190+0000 7f5fb4cb4740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf560145d0 0x55bf5601fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:55.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.191+0000 7f5fb4cb4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf56015b40 0x55bf56038590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:55.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.191+0000 7f5fb4cb4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf56023780 0x55bf56038ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:55.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.191+0000 7f5fb4cb4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac1ab480 con 0x55bf56015b40 2024-09-19T08:56:55.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.191+0000 7f5fb4cb4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac1acfc0 con 0x55bf56023780 2024-09-19T08:56:55.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.191+0000 7f5fb4cb4740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac1abb50 con 0x55bf560145d0 2024-09-19T08:56:55.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.191+0000 7f5fab7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf56015b40 0x55bf56038590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:55.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.191+0000 7f5fab7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf56015b40 0x55bf56038590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44288/0 (socket says 172.21.15.164:44288) 2024-09-19T08:56:55.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.191+0000 7f5fab7fe640 1 -- 172.21.15.164:0/3754584398 learned_addr learned my addr 172.21.15.164:0/3754584398 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:55.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.191+0000 7f5fabfff640 1 --2- 172.21.15.164:0/3754584398 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf560145d0 0x55bf5601fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:55.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.191+0000 7f5fab7fe640 1 -- 172.21.15.164:0/3754584398 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf560145d0 msgr2=0x55bf5601fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:55.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.191+0000 7f5fab7fe640 1 --2- 172.21.15.164:0/3754584398 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf560145d0 0x55bf5601fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:55.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.191+0000 7f5fb0e78640 1 --2- 172.21.15.164:0/3754584398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf56023780 0x55bf56038ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:55.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.191+0000 7f5fab7fe640 1 -- 172.21.15.164:0/3754584398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf56023780 msgr2=0x55bf56038ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:55.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.191+0000 7f5fab7fe640 1 --2- 172.21.15.164:0/3754584398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf56023780 0x55bf56038ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:55.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.191+0000 7f5fab7fe640 1 -- 172.21.15.164:0/3754584398 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf56023410 con 0x55bf56015b40 2024-09-19T08:56:55.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.191+0000 7f5fab7fe640 1 --2- 172.21.15.164:0/3754584398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf56015b40 0x55bf56038590 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0006010 tx=0x7f5fa0005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337815bfd3d84198 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:55.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.191+0000 7f5f91ffb640 1 -- 172.21.15.164:0/3754584398 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0040070 con 0x55bf56015b40 2024-09-19T08:56:55.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.191+0000 7f5f91ffb640 1 -- 172.21.15.164:0/3754584398 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fa003c7b0 con 0x55bf56015b40 2024-09-19T08:56:55.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.191+0000 7f5f91ffb640 1 -- 172.21.15.164:0/3754584398 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0004d80 con 0x55bf56015b40 2024-09-19T08:56:55.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.192+0000 7f5fb4cb4740 1 -- 172.21.15.164:0/3754584398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf56015b40 msgr2=0x55bf56038590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:55.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.192+0000 7f5fb4cb4740 1 --2- 172.21.15.164:0/3754584398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf56015b40 0x55bf56038590 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0006010 tx=0x7f5fa0005780 comp rx=0 tx=0).stop 2024-09-19T08:56:55.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.192+0000 7f5fb4cb4740 1 -- 172.21.15.164:0/3754584398 shutdown_connections 2024-09-19T08:56:55.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.192+0000 7f5fb4cb4740 1 --2- 172.21.15.164:0/3754584398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf56023780 0x55bf56038ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:55.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.192+0000 7f5fb4cb4740 1 --2- 172.21.15.164:0/3754584398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf56015b40 0x55bf56038590 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:55.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.192+0000 7f5fb4cb4740 1 --2- 172.21.15.164:0/3754584398 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf560145d0 0x55bf5601fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:55.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.192+0000 7f5fb4cb4740 1 -- 172.21.15.164:0/3754584398 >> 172.21.15.164:0/3754584398 conn(0x55bf55fdd310 msgr2=0x55bf5601f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:55.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.192+0000 7f5fb4cb4740 1 -- 172.21.15.164:0/3754584398 shutdown_connections 2024-09-19T08:56:55.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.192+0000 7f5fb4cb4740 1 -- 172.21.15.164:0/3754584398 wait complete. 2024-09-19T08:56:55.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.192+0000 7f5fb4cb4740 1 -- start start 2024-09-19T08:56:55.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.192+0000 7f5fb4cb4740 1 Processor -- start 2024-09-19T08:56:55.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.193+0000 7f5fb4cb4740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf560145d0 0x55bf56036a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:55.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.193+0000 7f5fb4cb4740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf56015b40 0x55bf56036fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:55.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.193+0000 7f5fb4cb4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf56023780 0x55bf5609cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:55.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.193+0000 7f5fb4cb4740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac1ac8f0 con 0x55bf560145d0 2024-09-19T08:56:55.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.193+0000 7f5fb4cb4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac1ac220 con 0x55bf56023780 2024-09-19T08:56:55.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.193+0000 7f5fb4cb4740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac1a9480 con 0x55bf56015b40 2024-09-19T08:56:55.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.193+0000 7f5fb4cb4740 10 client.0 _ll_register_callbacks cb 0x7f5fa4009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:56:55.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.193+0000 7f5fabfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf560145d0 0x55bf56036a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:55.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.193+0000 7f5fabfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf560145d0 0x55bf56036a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44294/0 (socket says 172.21.15.164:44294) 2024-09-19T08:56:55.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.193+0000 7f5fabfff640 1 -- 172.21.15.164:0/1437847070 learned_addr learned my addr 172.21.15.164:0/1437847070 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:55.199 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.193+0000 7f5fab7fe640 1 --2- 172.21.15.164:0/1437847070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf56015b40 0x55bf56036fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:55.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.193+0000 7f5fb0e78640 1 --2- 172.21.15.164:0/1437847070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf56023780 0x55bf5609cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:55.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.193+0000 7f5fabfff640 1 -- 172.21.15.164:0/1437847070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf56015b40 msgr2=0x55bf56036fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:55.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.193+0000 7f5fabfff640 1 --2- 172.21.15.164:0/1437847070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf56015b40 0x55bf56036fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:55.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.193+0000 7f5fabfff640 1 -- 172.21.15.164:0/1437847070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf56023780 msgr2=0x55bf5609cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:55.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.193+0000 7f5fabfff640 1 --2- 172.21.15.164:0/1437847070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf56023780 0x55bf5609cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:55.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.193+0000 7f5fabfff640 1 -- 172.21.15.164:0/1437847070 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf5609d000 con 0x55bf560145d0 2024-09-19T08:56:55.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.193+0000 7f5fb0e78640 1 --2- 172.21.15.164:0/1437847070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf56023780 0x55bf5609cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:56:55.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.193+0000 7f5fabfff640 1 --2- 172.21.15.164:0/1437847070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf560145d0 0x55bf56036a60 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c0cdb30 tx=0x7f5f9c0bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c6e15934cdd786c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:55.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.193+0000 7f5f8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:56:55.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.193+0000 7f5f8b7fe640 1 -- 172.21.15.164:0/1437847070 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c0bc0b0 con 0x55bf560145d0 2024-09-19T08:56:55.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.193+0000 7f5f8b7fe640 1 -- 172.21.15.164:0/1437847070 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f5f9c00fb30 con 0x55bf560145d0 2024-09-19T08:56:55.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.193+0000 7f5fb4cb4740 1 -- 172.21.15.164:0/1437847070 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fa003c7b0 con 0x55bf560145d0 2024-09-19T08:56:55.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.194+0000 7f5f8b7fe640 1 -- 172.21.15.164:0/1437847070 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5f9c0a0120 con 0x55bf560145d0 2024-09-19T08:56:55.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.194+0000 7f5fb4cb4740 10 client.17850 fetch_fsmap learned FSMap version 32 2024-09-19T08:56:55.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.194+0000 7f5fb4cb4740 1 -- 172.21.15.164:0/1437847070 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bf5609e360 con 0x55bf560145d0 2024-09-19T08:56:55.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.194+0000 7f5f8b7fe640 1 -- 172.21.15.164:0/1437847070 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fa003c7b0 con 0x55bf560145d0 2024-09-19T08:56:55.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.194+0000 7f5f8b7fe640 1 -- 172.21.15.164:0/1437847070 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f5f9c098730 con 0x55bf560145d0 2024-09-19T08:56:55.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.194+0000 7f5fb4cb4740 10 client.17850 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:56:55.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.194+0000 7f5fb4cb4740 10 client.17850 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T08:56:55.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.194+0000 7f5fb4cb4740 20 client.17850 populate_metadata read hostname 'smithi164' 2024-09-19T08:56:55.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.194+0000 7f5fb4cb4740 1 --2- 172.21.15.164:0/1437847070 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55bf56033180 0x55bf560a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:55.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.194+0000 7f5fb4cb4740 4 client.17850 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:56:55.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.194+0000 7f5fb4cb4740 1 -- 172.21.15.164:0/1437847070 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf55f81d70 con 0x55bf56033180 2024-09-19T08:56:55.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.194+0000 7f5fab7fe640 1 --2- 172.21.15.164:0/1437847070 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55bf56033180 0x55bf560a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:55.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.195+0000 7f5f8b7fe640 10 client.17850 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:56:55.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.195+0000 7f5fab7fe640 1 --2- 172.21.15.164:0/1437847070 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55bf56033180 0x55bf560a0a20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=22225f23749f0f6d server_cookie=84f75efc851efe6c in_seq=0 out_seq=0 2024-09-19T08:56:55.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.195+0000 7f5f8b7fe640 1 -- 172.21.15.164:0/1437847070 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf55f81d70 con 0x55bf56033180 2024-09-19T08:56:55.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.195+0000 7f5f8b7fe640 10 client.17850 handle_command_reply: tid=0 2024-09-19T08:56:55.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.195+0000 7f5fb4cb4740 1 client.17850 shutdown 2024-09-19T08:56:55.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.195+0000 7f5fb4cb4740 1 -- 172.21.15.164:0/1437847070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf560145d0 msgr2=0x55bf56036a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:55.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.195+0000 7f5fb4cb4740 1 --2- 172.21.15.164:0/1437847070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf560145d0 0x55bf56036a60 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c0cdb30 tx=0x7f5f9c0bc040 comp rx=0 tx=0).stop 2024-09-19T08:56:55.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.195+0000 7f5fb4cb4740 1 -- 172.21.15.164:0/1437847070 shutdown_connections 2024-09-19T08:56:55.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.195+0000 7f5fb4cb4740 1 --2- 172.21.15.164:0/1437847070 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x55bf56033180 0x55bf560a0a20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:55.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.195+0000 7f5fb4cb4740 1 --2- 172.21.15.164:0/1437847070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf56023780 0x55bf5609cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:55.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.195+0000 7f5fb4cb4740 1 --2- 172.21.15.164:0/1437847070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf56015b40 0x55bf56036fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:55.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.195+0000 7f5fb4cb4740 1 --2- 172.21.15.164:0/1437847070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf560145d0 0x55bf56036a60 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:55.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.195+0000 7f5fb4cb4740 1 -- 172.21.15.164:0/1437847070 >> 172.21.15.164:0/1437847070 conn(0x55bf55fdd310 msgr2=0x55bf5600f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:55.202 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.195+0000 7f5f8b7fe640 0 client.17850 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:56:55.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.195+0000 7f5fb4cb4740 1 -- 172.21.15.164:0/1437847070 shutdown_connections 2024-09-19T08:56:55.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.195+0000 7f5fb4cb4740 1 -- 172.21.15.164:0/1437847070 wait complete. 2024-09-19T08:56:55.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.195+0000 7f5fb4cb4740 20 client.17850 trim_cache size 0 max 16384 2024-09-19T08:56:55.203 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:56:55.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.196+0000 7f5fb28db640 1 -- 172.21.15.164:0/3646811570 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f5f940a3430 msgr2=0x7f5f940a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:55.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.196+0000 7f5fb28db640 1 --2- 172.21.15.164:0/3646811570 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f5f940a3430 0x7f5f940a58f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4005c40 tx=0x7f5fa4009200 comp rx=0 tx=0).stop 2024-09-19T08:56:55.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.196+0000 7f5fb28db640 1 -- 172.21.15.164:0/3646811570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fac101de0 msgr2=0x7f5fac198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:55.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.196+0000 7f5fb28db640 1 --2- 172.21.15.164:0/3646811570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fac101de0 0x7f5fac198360 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c00c920 tx=0x7f5f9c00cde0 comp rx=0 tx=0).stop 2024-09-19T08:56:55.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.196+0000 7f5fb28db640 1 -- 172.21.15.164:0/3646811570 shutdown_connections 2024-09-19T08:56:55.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.196+0000 7f5fb28db640 1 --2- 172.21.15.164:0/3646811570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fac106f10 0x7f5fac19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:55.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.196+0000 7f5fb28db640 1 --2- 172.21.15.164:0/3646811570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fac102790 0x7f5fac1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:55.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.196+0000 7f5fb28db640 1 --2- 172.21.15.164:0/3646811570 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f5f940a3430 0x7f5f940a58f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:55.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.196+0000 7f5fb28db640 1 --2- 172.21.15.164:0/3646811570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5fac101de0 0x7f5fac198360 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:55.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.196+0000 7f5fb28db640 1 -- 172.21.15.164:0/3646811570 >> 172.21.15.164:0/3646811570 conn(0x7f5fac0fd8c0 msgr2=0x7f5fac0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:55.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.196+0000 7f5fb28db640 1 -- 172.21.15.164:0/3646811570 shutdown_connections 2024-09-19T08:56:55.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:55.196+0000 7f5fb28db640 1 -- 172.21.15.164:0/3646811570 wait complete. 2024-09-19T08:56:56.036 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T08:56:56.281 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:56:56.549 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:56 smithi164 ceph-mon[30928]: pgmap v346: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T08:56:56.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:56 smithi170 ceph-mon[35072]: pgmap v346: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T08:56:56.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:56 smithi173 ceph-mon[35306]: pgmap v346: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T08:56:56.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.766+0000 7fbe08fb8640 1 -- 172.21.15.164:0/2806550610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe041008e0 msgr2=0x7fbe04100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:56.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.766+0000 7fbe08fb8640 1 --2- 172.21.15.164:0/2806550610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe041008e0 0x7fbe04100cc0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4009930 tx=0x7fbdf402a420 comp rx=0 tx=0).stop 2024-09-19T08:56:56.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.766+0000 7fbe08fb8640 1 -- 172.21.15.164:0/2806550610 shutdown_connections 2024-09-19T08:56:56.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.766+0000 7fbe08fb8640 1 --2- 172.21.15.164:0/2806550610 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe04108160 0x7fbe0410eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.766+0000 7fbe08fb8640 1 --2- 172.21.15.164:0/2806550610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe04101290 0x7fbe04107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.766+0000 7fbe08fb8640 1 --2- 172.21.15.164:0/2806550610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe041008e0 0x7fbe04100cc0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.766+0000 7fbe08fb8640 1 -- 172.21.15.164:0/2806550610 >> 172.21.15.164:0/2806550610 conn(0x7fbe04077210 msgr2=0x7fbe04077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:56.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.766+0000 7fbe08fb8640 1 -- 172.21.15.164:0/2806550610 shutdown_connections 2024-09-19T08:56:56.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.766+0000 7fbe08fb8640 1 -- 172.21.15.164:0/2806550610 wait complete. 2024-09-19T08:56:56.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.767+0000 7fbe08fb8640 1 Processor -- start 2024-09-19T08:56:56.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.767+0000 7fbe08fb8640 1 -- start start 2024-09-19T08:56:56.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.767+0000 7fbe08fb8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe041008e0 0x7fbe04198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:56.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.767+0000 7fbe08fb8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe04101290 0x7fbe041989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:56.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.767+0000 7fbe08fb8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe04108160 0x7fbe0419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:56.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.767+0000 7fbe08fb8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbe04111fd0 con 0x7fbe04101290 2024-09-19T08:56:56.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.767+0000 7fbe08fb8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbe04111e50 con 0x7fbe04108160 2024-09-19T08:56:56.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.767+0000 7fbe08fb8640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbe04112150 con 0x7fbe041008e0 2024-09-19T08:56:56.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.767+0000 7fbe01d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe04101290 0x7fbe041989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:56.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.767+0000 7fbe01d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe04101290 0x7fbe041989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44300/0 (socket says 172.21.15.164:44300) 2024-09-19T08:56:56.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.767+0000 7fbe01d9b640 1 -- 172.21.15.164:0/1414979288 learned_addr learned my addr 172.21.15.164:0/1414979288 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:56.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.768+0000 7fbe02d9d640 1 --2- 172.21.15.164:0/1414979288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe04108160 0x7fbe0419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:56.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.768+0000 7fbe0259c640 1 --2- 172.21.15.164:0/1414979288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe041008e0 0x7fbe04198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:56.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.768+0000 7fbe01d9b640 1 -- 172.21.15.164:0/1414979288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe041008e0 msgr2=0x7fbe04198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:56.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.768+0000 7fbe01d9b640 1 --2- 172.21.15.164:0/1414979288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe041008e0 0x7fbe04198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.768+0000 7fbe01d9b640 1 -- 172.21.15.164:0/1414979288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe04108160 msgr2=0x7fbe0419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:56.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.768+0000 7fbe01d9b640 1 --2- 172.21.15.164:0/1414979288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe04108160 0x7fbe0419cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.768+0000 7fbe01d9b640 1 -- 172.21.15.164:0/1414979288 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe0419d430 con 0x7fbe04101290 2024-09-19T08:56:56.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.768+0000 7fbe02d9d640 1 --2- 172.21.15.164:0/1414979288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe04108160 0x7fbe0419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:56:56.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.768+0000 7fbe01d9b640 1 --2- 172.21.15.164:0/1414979288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe04101290 0x7fbe041989c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fbdf800d750 tx=0x7fbdf800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:56.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.768+0000 7fbdf37fe640 1 -- 172.21.15.164:0/1414979288 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8017070 con 0x7fbe04101290 2024-09-19T08:56:56.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.768+0000 7fbdf37fe640 1 -- 172.21.15.164:0/1414979288 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdf8009c30 con 0x7fbe04101290 2024-09-19T08:56:56.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.768+0000 7fbe08fb8640 1 -- 172.21.15.164:0/1414979288 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe0419d720 con 0x7fbe04101290 2024-09-19T08:56:56.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.768+0000 7fbdf37fe640 1 -- 172.21.15.164:0/1414979288 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf800f620 con 0x7fbe04101290 2024-09-19T08:56:56.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.768+0000 7fbe08fb8640 1 -- 172.21.15.164:0/1414979288 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe041a5000 con 0x7fbe04101290 2024-09-19T08:56:56.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.770+0000 7fbdf37fe640 1 -- 172.21.15.164:0/1414979288 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fbdf801a070 con 0x7fbe04101290 2024-09-19T08:56:56.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.771+0000 7fbe0b391740 1 Processor -- start 2024-09-19T08:56:56.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.771+0000 7fbe0b391740 1 -- start start 2024-09-19T08:56:56.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.771+0000 7fbe0b391740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587bd245b40 0x5587bd244090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:56.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.771+0000 7fbe0b391740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5587bd2445d0 0x5587bd244a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:56.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.771+0000 7fbe0b391740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587bd253780 0x5587bd255b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:56.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.771+0000 7fbe0b391740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5587bd1aa2f0 con 0x5587bd253780 2024-09-19T08:56:56.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.771+0000 7fbe0b391740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5587bd151cf0 con 0x5587bd245b40 2024-09-19T08:56:56.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.771+0000 7fbe0b391740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5587bd1a62b0 con 0x5587bd2445d0 2024-09-19T08:56:56.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.771+0000 7fbe01d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587bd253780 0x5587bd255b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:56.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.771+0000 7fbdf37fe640 1 --2- 172.21.15.164:0/1414979288 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fbdec0a33b0 0x7fbdec0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:56.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.771+0000 7fbe01d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587bd253780 0x5587bd255b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44308/0 (socket says 172.21.15.164:44308) 2024-09-19T08:56:56.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.771+0000 7fbe01d9b640 1 -- 172.21.15.164:0/439721256 learned_addr learned my addr 172.21.15.164:0/439721256 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:56.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.771+0000 7fbe02d9d640 1 --2- 172.21.15.164:0/439721256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587bd245b40 0x5587bd244090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:56.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.771+0000 7fbe0259c640 1 --2- 172.21.15.164:0/439721256 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5587bd2445d0 0x5587bd244a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:56.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.772+0000 7fbe02d9d640 1 --2- 172.21.15.164:0/1414979288 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fbdec0a33b0 0x7fbdec0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:56.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.772+0000 7fbdf37fe640 1 -- 172.21.15.164:0/1414979288 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7fbdf80cf770 con 0x7fbe04101290 2024-09-19T08:56:56.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.772+0000 7fbe01d9b640 1 -- 172.21.15.164:0/439721256 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5587bd2445d0 msgr2=0x5587bd244a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:56.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.772+0000 7fbe01d9b640 1 --2- 172.21.15.164:0/439721256 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5587bd2445d0 0x5587bd244a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.772+0000 7fbe01d9b640 1 -- 172.21.15.164:0/439721256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587bd245b40 msgr2=0x5587bd244090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:56.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.772+0000 7fbe01d9b640 1 --2- 172.21.15.164:0/439721256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587bd245b40 0x5587bd244090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.772+0000 7fbe01d9b640 1 -- 172.21.15.164:0/439721256 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587bd1363a0 con 0x5587bd253780 2024-09-19T08:56:56.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.772+0000 7fbe0259c640 1 --2- 172.21.15.164:0/439721256 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5587bd2445d0 0x5587bd244a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:56:56.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.772+0000 7fbe01d9b640 1 --2- 172.21.15.164:0/439721256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587bd253780 0x5587bd255b70 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fbdf809f980 tx=0x7fbdf8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c02bffc410b7bf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:56.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.772+0000 7fbe02d9d640 1 --2- 172.21.15.164:0/1414979288 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fbdec0a33b0 0x7fbdec0a5870 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc007310 tx=0x7fbdfc0072a0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:56.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.772+0000 7fbdf0ff9640 1 -- 172.21.15.164:0/439721256 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8021070 con 0x5587bd253780 2024-09-19T08:56:56.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.772+0000 7fbdf0ff9640 1 -- 172.21.15.164:0/439721256 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdf809c190 con 0x5587bd253780 2024-09-19T08:56:56.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.772+0000 7fbdf0ff9640 1 -- 172.21.15.164:0/439721256 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8098100 con 0x5587bd253780 2024-09-19T08:56:56.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.773+0000 7fbe0b391740 1 -- 172.21.15.164:0/439721256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587bd253780 msgr2=0x5587bd255b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:56.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.773+0000 7fbe0b391740 1 --2- 172.21.15.164:0/439721256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587bd253780 0x5587bd255b70 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fbdf809f980 tx=0x7fbdf8000bc0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.773+0000 7fbe0b391740 1 -- 172.21.15.164:0/439721256 shutdown_connections 2024-09-19T08:56:56.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.773+0000 7fbe0b391740 1 --2- 172.21.15.164:0/439721256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587bd253780 0x5587bd255b70 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.773+0000 7fbe0b391740 1 --2- 172.21.15.164:0/439721256 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5587bd2445d0 0x5587bd244a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.773+0000 7fbe0b391740 1 --2- 172.21.15.164:0/439721256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587bd245b40 0x5587bd244090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.773+0000 7fbe0b391740 1 -- 172.21.15.164:0/439721256 >> 172.21.15.164:0/439721256 conn(0x5587bd20d310 msgr2=0x5587bd240c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:56.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.773+0000 7fbe0b391740 1 -- 172.21.15.164:0/439721256 shutdown_connections 2024-09-19T08:56:56.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.773+0000 7fbe0b391740 1 -- 172.21.15.164:0/439721256 wait complete. 2024-09-19T08:56:56.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.773+0000 7fbe0b391740 1 -- start start 2024-09-19T08:56:56.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.773+0000 7fbe0b391740 1 Processor -- start 2024-09-19T08:56:56.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.774+0000 7fbe0b391740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5587bd2445d0 0x5587bd2531c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:56.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.774+0000 7fbe0b391740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587bd245b40 0x5587bd250470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:56.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.774+0000 7fbe0b391740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587bd253780 0x5587bd250c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:56.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.774+0000 7fbe0b391740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5587bd256420 con 0x5587bd245b40 2024-09-19T08:56:56.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.774+0000 7fbe0b391740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5587bd165060 con 0x5587bd253780 2024-09-19T08:56:56.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.774+0000 7fbe0b391740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5587bd2565a0 con 0x5587bd2445d0 2024-09-19T08:56:56.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.774+0000 7fbe0b391740 10 client.0 _ll_register_callbacks cb 0x5587bceb88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:56:56.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.774+0000 7fbe01d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587bd245b40 0x5587bd250470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:56.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.774+0000 7fbe01d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587bd245b40 0x5587bd250470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44320/0 (socket says 172.21.15.164:44320) 2024-09-19T08:56:56.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.774+0000 7fbe01d9b640 1 -- 172.21.15.164:0/2156559235 learned_addr learned my addr 172.21.15.164:0/2156559235 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:56.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.774+0000 7fbe02d9d640 1 --2- 172.21.15.164:0/2156559235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587bd253780 0x5587bd250c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:56.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.774+0000 7fbe0259c640 1 --2- 172.21.15.164:0/2156559235 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5587bd2445d0 0x5587bd2531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:56.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.774+0000 7fbe01d9b640 1 -- 172.21.15.164:0/2156559235 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5587bd2445d0 msgr2=0x5587bd2531c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:56.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.774+0000 7fbe01d9b640 1 --2- 172.21.15.164:0/2156559235 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5587bd2445d0 0x5587bd2531c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.774+0000 7fbe01d9b640 1 -- 172.21.15.164:0/2156559235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587bd253780 msgr2=0x5587bd250c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:56.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.774+0000 7fbe01d9b640 1 --2- 172.21.15.164:0/2156559235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587bd253780 0x5587bd250c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.774+0000 7fbe01d9b640 1 -- 172.21.15.164:0/2156559235 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587bcf1b3c0 con 0x5587bd245b40 2024-09-19T08:56:56.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.774+0000 7fbe0259c640 1 --2- 172.21.15.164:0/2156559235 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5587bd2445d0 0x5587bd2531c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:56:56.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.774+0000 7fbe01d9b640 1 --2- 172.21.15.164:0/2156559235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587bd245b40 0x5587bd250470 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fbdf80cd500 tx=0x7fbdf80cd430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c66347ee2a5c21 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:56.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.774+0000 7fbde9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:56:56.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.774+0000 7fbde9ffb640 1 -- 172.21.15.164:0/2156559235 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8021040 con 0x5587bd245b40 2024-09-19T08:56:56.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.774+0000 7fbde9ffb640 1 -- 172.21.15.164:0/2156559235 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7fbdf809f9b0 con 0x5587bd245b40 2024-09-19T08:56:56.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.774+0000 7fbe0b391740 1 -- 172.21.15.164:0/2156559235 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdf809c190 con 0x5587bd245b40 2024-09-19T08:56:56.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.775+0000 7fbde9ffb640 1 -- 172.21.15.164:0/2156559235 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbdf8009de0 con 0x5587bd245b40 2024-09-19T08:56:56.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.775+0000 7fbe0b391740 10 client.17862 fetch_fsmap learned FSMap version 32 2024-09-19T08:56:56.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.775+0000 7fbe0b391740 1 -- 172.21.15.164:0/2156559235 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5587bd261f10 con 0x5587bd245b40 2024-09-19T08:56:56.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.775+0000 7fbde9ffb640 1 -- 172.21.15.164:0/2156559235 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdf809c190 con 0x5587bd245b40 2024-09-19T08:56:56.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.775+0000 7fbde9ffb640 1 -- 172.21.15.164:0/2156559235 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fbdf809a950 con 0x5587bd245b40 2024-09-19T08:56:56.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.775+0000 7fbe0b391740 10 client.17862 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:56:56.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.775+0000 7fbe0b391740 10 client.17862 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T08:56:56.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.775+0000 7fbe0b391740 20 client.17862 populate_metadata read hostname 'smithi164' 2024-09-19T08:56:56.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.775+0000 7fbe0b391740 1 --2- 172.21.15.164:0/2156559235 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x5587bd263260 0x5587bd265650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:56.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.775+0000 7fbe0b391740 4 client.17862 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:56:56.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.775+0000 7fbe0b391740 1 -- 172.21.15.164:0/2156559235 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587bd1b7860 con 0x5587bd263260 2024-09-19T08:56:56.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.775+0000 7fbe0259c640 1 --2- 172.21.15.164:0/2156559235 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x5587bd263260 0x5587bd265650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:56.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.776+0000 7fbde9ffb640 10 client.17862 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:56:56.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.776+0000 7fbe0259c640 1 --2- 172.21.15.164:0/2156559235 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x5587bd263260 0x5587bd265650 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4b75349ac9cd9828 server_cookie=3e224c5d02fe6cea in_seq=0 out_seq=0 2024-09-19T08:56:56.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.777+0000 7fbde9ffb640 1 -- 172.21.15.164:0/2156559235 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5587bd1b7860 con 0x5587bd263260 2024-09-19T08:56:56.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.777+0000 7fbde9ffb640 10 client.17862 handle_command_reply: tid=0 2024-09-19T08:56:56.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.777+0000 7fbe0b391740 1 client.17862 shutdown 2024-09-19T08:56:56.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.777+0000 7fbe0b391740 1 -- 172.21.15.164:0/2156559235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587bd245b40 msgr2=0x5587bd250470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:56.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.777+0000 7fbe0b391740 1 --2- 172.21.15.164:0/2156559235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587bd245b40 0x5587bd250470 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fbdf80cd500 tx=0x7fbdf80cd430 comp rx=0 tx=0).stop 2024-09-19T08:56:56.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.777+0000 7fbe0b391740 1 -- 172.21.15.164:0/2156559235 shutdown_connections 2024-09-19T08:56:56.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.777+0000 7fbe0b391740 1 --2- 172.21.15.164:0/2156559235 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x5587bd263260 0x5587bd265650 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.777+0000 7fbe0b391740 1 --2- 172.21.15.164:0/2156559235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587bd253780 0x5587bd250c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.777+0000 7fbe0b391740 1 --2- 172.21.15.164:0/2156559235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587bd245b40 0x5587bd250470 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.777+0000 7fbe0b391740 1 --2- 172.21.15.164:0/2156559235 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5587bd2445d0 0x5587bd2531c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.777+0000 7fbe0b391740 1 -- 172.21.15.164:0/2156559235 >> 172.21.15.164:0/2156559235 conn(0x5587bd20d310 msgr2=0x5587bd246990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:56.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.777+0000 7fbde9ffb640 0 client.17862 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:56:56.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.777+0000 7fbe0b391740 1 -- 172.21.15.164:0/2156559235 shutdown_connections 2024-09-19T08:56:56.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.777+0000 7fbe0b391740 1 -- 172.21.15.164:0/2156559235 wait complete. 2024-09-19T08:56:56.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.777+0000 7fbe0b391740 20 client.17862 trim_cache size 0 max 16384 2024-09-19T08:56:56.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.790+0000 7fbe0b391740 1 Processor -- start 2024-09-19T08:56:56.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.790+0000 7fbe0b391740 1 -- start start 2024-09-19T08:56:56.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.790+0000 7fbe0b391740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5587bd2445d0 0x5587bd2523e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:56.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.790+0000 7fbe0b391740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587bd245b40 0x5587bd24d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:56.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.791+0000 7fbe0b391740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587bd253780 0x5587bd24dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:56.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.791+0000 7fbe0b391740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbe041ab570 con 0x5587bd245b40 2024-09-19T08:56:56.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.791+0000 7fbe0b391740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbe041ad0b0 con 0x5587bd253780 2024-09-19T08:56:56.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.791+0000 7fbe0b391740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbe041abc40 con 0x5587bd2445d0 2024-09-19T08:56:56.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.791+0000 7fbe01d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587bd245b40 0x5587bd24d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:56.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.791+0000 7fbe01d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587bd245b40 0x5587bd24d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44322/0 (socket says 172.21.15.164:44322) 2024-09-19T08:56:56.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.791+0000 7fbe01d9b640 1 -- 172.21.15.164:0/35164759 learned_addr learned my addr 172.21.15.164:0/35164759 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:56.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.791+0000 7fbe0259c640 1 --2- 172.21.15.164:0/35164759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5587bd2445d0 0x5587bd2523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:56.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.791+0000 7fbe02d9d640 1 --2- 172.21.15.164:0/35164759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587bd253780 0x5587bd24dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:56.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.791+0000 7fbe01d9b640 1 -- 172.21.15.164:0/35164759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5587bd2445d0 msgr2=0x5587bd2523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:56.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.791+0000 7fbe01d9b640 1 --2- 172.21.15.164:0/35164759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5587bd2445d0 0x5587bd2523e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.791+0000 7fbe01d9b640 1 -- 172.21.15.164:0/35164759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587bd253780 msgr2=0x5587bd24dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:56.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.791+0000 7fbe01d9b640 1 --2- 172.21.15.164:0/35164759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587bd253780 0x5587bd24dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.791+0000 7fbe01d9b640 1 -- 172.21.15.164:0/35164759 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587bd246fc0 con 0x5587bd245b40 2024-09-19T08:56:56.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.791+0000 7fbe01d9b640 1 --2- 172.21.15.164:0/35164759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587bd245b40 0x5587bd24d6d0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fbdf800d720 tx=0x7fbdf809ae30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd74bc8400391b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:56.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.791+0000 7fbdebfff640 1 -- 172.21.15.164:0/35164759 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8021070 con 0x5587bd245b40 2024-09-19T08:56:56.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.791+0000 7fbdebfff640 1 -- 172.21.15.164:0/35164759 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdf80b3070 con 0x5587bd245b40 2024-09-19T08:56:56.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.791+0000 7fbdebfff640 1 -- 172.21.15.164:0/35164759 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8086250 con 0x5587bd245b40 2024-09-19T08:56:56.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.792+0000 7fbe0b391740 1 -- 172.21.15.164:0/35164759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587bd245b40 msgr2=0x5587bd24d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:56.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.792+0000 7fbe0b391740 1 --2- 172.21.15.164:0/35164759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587bd245b40 0x5587bd24d6d0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fbdf800d720 tx=0x7fbdf809ae30 comp rx=0 tx=0).stop 2024-09-19T08:56:56.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.792+0000 7fbe0b391740 1 -- 172.21.15.164:0/35164759 shutdown_connections 2024-09-19T08:56:56.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.792+0000 7fbe0b391740 1 --2- 172.21.15.164:0/35164759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587bd253780 0x5587bd24dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.792+0000 7fbe0b391740 1 --2- 172.21.15.164:0/35164759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587bd245b40 0x5587bd24d6d0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.792+0000 7fbe0b391740 1 --2- 172.21.15.164:0/35164759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5587bd2445d0 0x5587bd2523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.792+0000 7fbe0b391740 1 -- 172.21.15.164:0/35164759 >> 172.21.15.164:0/35164759 conn(0x5587bd20d310 msgr2=0x5587bd251cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:56.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.792+0000 7fbe0b391740 1 -- 172.21.15.164:0/35164759 shutdown_connections 2024-09-19T08:56:56.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.792+0000 7fbe0b391740 1 -- 172.21.15.164:0/35164759 wait complete. 2024-09-19T08:56:56.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.792+0000 7fbe0b391740 1 -- start start 2024-09-19T08:56:56.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.793+0000 7fbe0b391740 1 Processor -- start 2024-09-19T08:56:56.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.793+0000 7fbe0b391740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587bd2445d0 0x5587bd266360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:56.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.793+0000 7fbe0b391740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5587bd245b40 0x5587bd2668a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:56.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.793+0000 7fbe0b391740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587bd253780 0x5587bd2ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:56.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.793+0000 7fbe0b391740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbe041ac9e0 con 0x5587bd253780 2024-09-19T08:56:56.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.793+0000 7fbe0b391740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbe041ac310 con 0x5587bd2445d0 2024-09-19T08:56:56.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.793+0000 7fbe0b391740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbe041a9570 con 0x5587bd245b40 2024-09-19T08:56:56.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.793+0000 7fbe0b391740 10 client.0 _ll_register_callbacks cb 0x7fbdfc007db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:56:56.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.793+0000 7fbe01d9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5587bd245b40 0x5587bd2668a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:56.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.793+0000 7fbe02d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587bd253780 0x5587bd2ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:56.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.793+0000 7fbe01d9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5587bd245b40 0x5587bd2668a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:37552/0 (socket says 172.21.15.164:37552) 2024-09-19T08:56:56.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.793+0000 7fbe02d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587bd253780 0x5587bd2ccba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44330/0 (socket says 172.21.15.164:44330) 2024-09-19T08:56:56.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.793+0000 7fbe01d9b640 1 -- 172.21.15.164:0/641024673 learned_addr learned my addr 172.21.15.164:0/641024673 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:56.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.793+0000 7fbe0259c640 1 --2- 172.21.15.164:0/641024673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587bd2445d0 0x5587bd266360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:56.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.793+0000 7fbe02d9d640 1 -- 172.21.15.164:0/641024673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5587bd245b40 msgr2=0x5587bd2668a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:56.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.793+0000 7fbe02d9d640 1 --2- 172.21.15.164:0/641024673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5587bd245b40 0x5587bd2668a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.793+0000 7fbe02d9d640 1 -- 172.21.15.164:0/641024673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587bd2445d0 msgr2=0x5587bd266360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:56.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.793+0000 7fbe02d9d640 1 --2- 172.21.15.164:0/641024673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587bd2445d0 0x5587bd266360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.793+0000 7fbe02d9d640 1 -- 172.21.15.164:0/641024673 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587bd267180 con 0x5587bd253780 2024-09-19T08:56:56.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.793+0000 7fbe0259c640 1 --2- 172.21.15.164:0/641024673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587bd2445d0 0x5587bd266360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:56:56.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.794+0000 7fbe02d9d640 1 --2- 172.21.15.164:0/641024673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587bd253780 0x5587bd2ccba0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x5587bd245250 tx=0x7fbdfc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ec8dede502da65 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:56.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.794+0000 7fbde9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:56:56.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.794+0000 7fbde9ffb640 1 -- 172.21.15.164:0/641024673 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc004b20 con 0x5587bd253780 2024-09-19T08:56:56.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.794+0000 7fbde9ffb640 1 -- 172.21.15.164:0/641024673 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7fbdfc004cc0 con 0x5587bd253780 2024-09-19T08:56:56.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.794+0000 7fbe0b391740 1 -- 172.21.15.164:0/641024673 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdf80b3070 con 0x5587bd253780 2024-09-19T08:56:56.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.794+0000 7fbde9ffb640 1 -- 172.21.15.164:0/641024673 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbdfc00eb50 con 0x5587bd253780 2024-09-19T08:56:56.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.794+0000 7fbe0b391740 10 client.17868 fetch_fsmap learned FSMap version 32 2024-09-19T08:56:56.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.794+0000 7fbe0b391740 1 -- 172.21.15.164:0/641024673 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5587bd2ce1c0 con 0x5587bd253780 2024-09-19T08:56:56.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.794+0000 7fbde9ffb640 1 -- 172.21.15.164:0/641024673 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdfc00ecf0 con 0x5587bd253780 2024-09-19T08:56:56.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.794+0000 7fbde9ffb640 1 -- 172.21.15.164:0/641024673 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fbdfc026070 con 0x5587bd253780 2024-09-19T08:56:56.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.794+0000 7fbe0b391740 10 client.17868 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:56:56.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.794+0000 7fbe0b391740 10 client.17868 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T08:56:56.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.794+0000 7fbe0b391740 20 client.17868 populate_metadata read hostname 'smithi164' 2024-09-19T08:56:56.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.794+0000 7fbe0b391740 1 --2- 172.21.15.164:0/641024673 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x5587bd2ce940 0x5587bd2d0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:56.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.794+0000 7fbe0b391740 4 client.17868 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:56:56.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.794+0000 7fbe0b391740 1 -- 172.21.15.164:0/641024673 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fbdf80b3070 con 0x5587bd2ce940 2024-09-19T08:56:56.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.795+0000 7fbe0259c640 1 --2- 172.21.15.164:0/641024673 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x5587bd2ce940 0x5587bd2d0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:56.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.795+0000 7fbe0259c640 1 --2- 172.21.15.164:0/641024673 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x5587bd2ce940 0x5587bd2d0d60 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ac8f0d391520bbab server_cookie=3c86c5e3f639a14a in_seq=0 out_seq=0 2024-09-19T08:56:56.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.795+0000 7fbde9ffb640 10 client.17868 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:56:56.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.795+0000 7fbde9ffb640 1 -- 172.21.15.164:0/641024673 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fbdf80b3070 con 0x5587bd2ce940 2024-09-19T08:56:56.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.795+0000 7fbde9ffb640 10 client.17868 handle_command_reply: tid=0 2024-09-19T08:56:56.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.795+0000 7fbe0b391740 1 client.17868 shutdown 2024-09-19T08:56:56.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.795+0000 7fbe0b391740 1 -- 172.21.15.164:0/641024673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587bd253780 msgr2=0x5587bd2ccba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:56.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.795+0000 7fbe0b391740 1 --2- 172.21.15.164:0/641024673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587bd253780 0x5587bd2ccba0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x5587bd245250 tx=0x7fbdfc004a40 comp rx=0 tx=0).stop 2024-09-19T08:56:56.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.795+0000 7fbe0b391740 1 -- 172.21.15.164:0/641024673 shutdown_connections 2024-09-19T08:56:56.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.795+0000 7fbe0b391740 1 --2- 172.21.15.164:0/641024673 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x5587bd2ce940 0x5587bd2d0d60 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.795+0000 7fbe0b391740 1 --2- 172.21.15.164:0/641024673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587bd253780 0x5587bd2ccba0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.795+0000 7fbe0b391740 1 --2- 172.21.15.164:0/641024673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5587bd245b40 0x5587bd2668a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.795+0000 7fbe0b391740 1 --2- 172.21.15.164:0/641024673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587bd2445d0 0x5587bd266360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.796+0000 7fbe0b391740 1 -- 172.21.15.164:0/641024673 >> 172.21.15.164:0/641024673 conn(0x5587bd20d310 msgr2=0x5587bd251cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:56.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.796+0000 7fbde9ffb640 0 client.17868 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:56:56.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.796+0000 7fbe0b391740 1 -- 172.21.15.164:0/641024673 shutdown_connections 2024-09-19T08:56:56.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.796+0000 7fbe0b391740 1 -- 172.21.15.164:0/641024673 wait complete. 2024-09-19T08:56:56.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.796+0000 7fbe0b391740 20 client.17868 trim_cache size 0 max 16384 2024-09-19T08:56:56.802 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:56:56.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.796+0000 7fbe08fb8640 1 -- 172.21.15.164:0/1414979288 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fbdec0a33b0 msgr2=0x7fbdec0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:56.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.796+0000 7fbe08fb8640 1 --2- 172.21.15.164:0/1414979288 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fbdec0a33b0 0x7fbdec0a5870 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc007310 tx=0x7fbdfc0072a0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.796+0000 7fbe08fb8640 1 -- 172.21.15.164:0/1414979288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe04101290 msgr2=0x7fbe041989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:56.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.796+0000 7fbe08fb8640 1 --2- 172.21.15.164:0/1414979288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe04101290 0x7fbe041989c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fbdf800d750 tx=0x7fbdf800dc10 comp rx=0 tx=0).stop 2024-09-19T08:56:56.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.796+0000 7fbe08fb8640 1 -- 172.21.15.164:0/1414979288 shutdown_connections 2024-09-19T08:56:56.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.796+0000 7fbe08fb8640 1 --2- 172.21.15.164:0/1414979288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe04108160 0x7fbe0419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.796+0000 7fbe08fb8640 1 --2- 172.21.15.164:0/1414979288 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fbdec0a33b0 0x7fbdec0a5870 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.796+0000 7fbe08fb8640 1 --2- 172.21.15.164:0/1414979288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe04101290 0x7fbe041989c0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.796+0000 7fbe08fb8640 1 --2- 172.21.15.164:0/1414979288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe041008e0 0x7fbe04198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:56.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.796+0000 7fbe08fb8640 1 -- 172.21.15.164:0/1414979288 >> 172.21.15.164:0/1414979288 conn(0x7fbe04077210 msgr2=0x7fbe040ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:56.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.796+0000 7fbe08fb8640 1 -- 172.21.15.164:0/1414979288 shutdown_connections 2024-09-19T08:56:56.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:56.796+0000 7fbe08fb8640 1 -- 172.21.15.164:0/1414979288 wait complete. 2024-09-19T08:56:57.625 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T08:56:57.879 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:56:58.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.352+0000 7fb82375b640 1 -- 172.21.15.164:0/1593798845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb81c106f10 msgr2=0x7fb81c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:58.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.352+0000 7fb82375b640 1 --2- 172.21.15.164:0/1593798845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb81c106f10 0x7fb81c10ea90 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fb814009ca0 tx=0x7fb81402a8c0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.352+0000 7fb82375b640 1 -- 172.21.15.164:0/1593798845 shutdown_connections 2024-09-19T08:56:58.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.352+0000 7fb82375b640 1 --2- 172.21.15.164:0/1593798845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb81c106f10 0x7fb81c10ea90 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.352+0000 7fb82375b640 1 --2- 172.21.15.164:0/1593798845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb81c102790 0x7fb81c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.352+0000 7fb82375b640 1 --2- 172.21.15.164:0/1593798845 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb81c101de0 0x7fb81c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.352+0000 7fb82375b640 1 -- 172.21.15.164:0/1593798845 >> 172.21.15.164:0/1593798845 conn(0x7fb81c0fd8c0 msgr2=0x7fb81c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:58.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.352+0000 7fb82375b640 1 -- 172.21.15.164:0/1593798845 shutdown_connections 2024-09-19T08:56:58.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.352+0000 7fb82375b640 1 -- 172.21.15.164:0/1593798845 wait complete. 2024-09-19T08:56:58.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.352+0000 7fb82375b640 1 Processor -- start 2024-09-19T08:56:58.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.352+0000 7fb82375b640 1 -- start start 2024-09-19T08:56:58.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.353+0000 7fb82375b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb81c101de0 0x7fb81c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:58.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.353+0000 7fb82375b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb81c102790 0x7fb81c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:58.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.353+0000 7fb82375b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb81c106f10 0x7fb81c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:58.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.353+0000 7fb82375b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c111f90 con 0x7fb81c101de0 2024-09-19T08:56:58.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.353+0000 7fb82375b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c111e10 con 0x7fb81c106f10 2024-09-19T08:56:58.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.353+0000 7fb82375b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c112110 con 0x7fb81c102790 2024-09-19T08:56:58.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.353+0000 7fb8214f7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb81c101de0 0x7fb81c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:58.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.353+0000 7fb8214f7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb81c101de0 0x7fb81c198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44346/0 (socket says 172.21.15.164:44346) 2024-09-19T08:56:58.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.353+0000 7fb8214f7640 1 -- 172.21.15.164:0/2239942232 learned_addr learned my addr 172.21.15.164:0/2239942232 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:58.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.353+0000 7fb821cf8640 1 --2- 172.21.15.164:0/2239942232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb81c106f10 0x7fb81c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:58.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.353+0000 7fb820cf6640 1 --2- 172.21.15.164:0/2239942232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb81c102790 0x7fb81c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:58.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.353+0000 7fb8214f7640 1 -- 172.21.15.164:0/2239942232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb81c102790 msgr2=0x7fb81c1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:58.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.353+0000 7fb8214f7640 1 --2- 172.21.15.164:0/2239942232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb81c102790 0x7fb81c1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.353+0000 7fb8214f7640 1 -- 172.21.15.164:0/2239942232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb81c106f10 msgr2=0x7fb81c19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:58.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.353+0000 7fb8214f7640 1 --2- 172.21.15.164:0/2239942232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb81c106f10 0x7fb81c19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.353+0000 7fb8214f7640 1 -- 172.21.15.164:0/2239942232 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb81c19d410 con 0x7fb81c101de0 2024-09-19T08:56:58.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.353+0000 7fb820cf6640 1 --2- 172.21.15.164:0/2239942232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb81c102790 0x7fb81c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:56:58.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.353+0000 7fb8214f7640 1 --2- 172.21.15.164:0/2239942232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb81c101de0 0x7fb81c198460 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fb81000b4a0 tx=0x7fb81000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:58.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.353+0000 7fb80e7fc640 1 -- 172.21.15.164:0/2239942232 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb810014070 con 0x7fb81c101de0 2024-09-19T08:56:58.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.353+0000 7fb80e7fc640 1 -- 172.21.15.164:0/2239942232 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb810010af0 con 0x7fb81c101de0 2024-09-19T08:56:58.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.353+0000 7fb82375b640 1 -- 172.21.15.164:0/2239942232 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb81c19d700 con 0x7fb81c101de0 2024-09-19T08:56:58.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.353+0000 7fb80e7fc640 1 -- 172.21.15.164:0/2239942232 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb810017d10 con 0x7fb81c101de0 2024-09-19T08:56:58.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.354+0000 7fb82375b640 1 -- 172.21.15.164:0/2239942232 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb81c1a4fe0 con 0x7fb81c101de0 2024-09-19T08:56:58.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.356+0000 7fb80e7fc640 1 -- 172.21.15.164:0/2239942232 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fb810017070 con 0x7fb81c101de0 2024-09-19T08:56:58.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.356+0000 7fb825b34740 1 Processor -- start 2024-09-19T08:56:58.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.356+0000 7fb825b34740 1 -- start start 2024-09-19T08:56:58.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.356+0000 7fb80e7fc640 1 --2- 172.21.15.164:0/2239942232 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb8080a33b0 0x7fb8080a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:58.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.356+0000 7fb825b34740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf528d5b40 0x55bf528d4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:58.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.356+0000 7fb825b34740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf528d45d0 0x55bf528d4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:58.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.356+0000 7fb825b34740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf528df370 0x55bf528e1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:58.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.357+0000 7fb825b34740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bf5283a2f0 con 0x55bf528d5b40 2024-09-19T08:56:58.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.357+0000 7fb825b34740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55bf527e1cf0 con 0x55bf528df370 2024-09-19T08:56:58.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.357+0000 7fb825b34740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55bf528362b0 con 0x55bf528d45d0 2024-09-19T08:56:58.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.357+0000 7fb821cf8640 1 --2- 172.21.15.164:0/2239942232 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb8080a33b0 0x7fb8080a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:58.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.357+0000 7fb80e7fc640 1 -- 172.21.15.164:0/2239942232 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7fb8100cf120 con 0x7fb81c101de0 2024-09-19T08:56:58.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.357+0000 7fb8214f7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf528d5b40 0x55bf528d4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:58.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.357+0000 7fb8214f7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf528d5b40 0x55bf528d4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44358/0 (socket says 172.21.15.164:44358) 2024-09-19T08:56:58.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.357+0000 7fb8214f7640 1 -- 172.21.15.164:0/465183838 learned_addr learned my addr 172.21.15.164:0/465183838 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:58.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.357+0000 7fb821cf8640 1 --2- 172.21.15.164:0/465183838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf528df370 0x55bf528e1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:58.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.357+0000 7fb820cf6640 1 --2- 172.21.15.164:0/465183838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf528d45d0 0x55bf528d4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:58.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.357+0000 7fb8214f7640 1 -- 172.21.15.164:0/465183838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf528d45d0 msgr2=0x55bf528d4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:58.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.357+0000 7fb8214f7640 1 --2- 172.21.15.164:0/465183838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf528d45d0 0x55bf528d4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.357+0000 7fb8214f7640 1 -- 172.21.15.164:0/465183838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf528df370 msgr2=0x55bf528e1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:58.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.357+0000 7fb8214f7640 1 --2- 172.21.15.164:0/465183838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf528df370 0x55bf528e1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.357+0000 7fb8214f7640 1 -- 172.21.15.164:0/465183838 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf527c63a0 con 0x55bf528d5b40 2024-09-19T08:56:58.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.357+0000 7fb821cf8640 1 --2- 172.21.15.164:0/2239942232 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb8080a33b0 0x7fb8080a5870 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb814005fe0 tx=0x7fb814031ce0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:58.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.357+0000 7fb8214f7640 1 --2- 172.21.15.164:0/465183838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf528d5b40 0x55bf528d4090 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fb810097950 tx=0x7fb8100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed7a6c204af9344 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:58.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.357+0000 7fb8077fe640 1 -- 172.21.15.164:0/465183838 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81001e040 con 0x55bf528d5b40 2024-09-19T08:56:58.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.357+0000 7fb8077fe640 1 -- 172.21.15.164:0/465183838 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb81009bbf0 con 0x55bf528d5b40 2024-09-19T08:56:58.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.357+0000 7fb8077fe640 1 -- 172.21.15.164:0/465183838 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb810098b40 con 0x55bf528d5b40 2024-09-19T08:56:58.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.358+0000 7fb825b34740 1 -- 172.21.15.164:0/465183838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf528d5b40 msgr2=0x55bf528d4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:58.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.358+0000 7fb825b34740 1 --2- 172.21.15.164:0/465183838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf528d5b40 0x55bf528d4090 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fb810097950 tx=0x7fb8100046c0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.358+0000 7fb825b34740 1 -- 172.21.15.164:0/465183838 shutdown_connections 2024-09-19T08:56:58.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.358+0000 7fb825b34740 1 --2- 172.21.15.164:0/465183838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf528df370 0x55bf528e1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.358+0000 7fb825b34740 1 --2- 172.21.15.164:0/465183838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf528d45d0 0x55bf528d4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.358+0000 7fb825b34740 1 --2- 172.21.15.164:0/465183838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf528d5b40 0x55bf528d4090 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.358+0000 7fb825b34740 1 -- 172.21.15.164:0/465183838 >> 172.21.15.164:0/465183838 conn(0x55bf5289d310 msgr2=0x55bf528d0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:58.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.358+0000 7fb825b34740 1 -- 172.21.15.164:0/465183838 shutdown_connections 2024-09-19T08:56:58.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.358+0000 7fb825b34740 1 -- 172.21.15.164:0/465183838 wait complete. 2024-09-19T08:56:58.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.359+0000 7fb825b34740 1 -- start start 2024-09-19T08:56:58.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.359+0000 7fb825b34740 1 Processor -- start 2024-09-19T08:56:58.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.359+0000 7fb825b34740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf528d45d0 0x55bf528dedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:58.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.359+0000 7fb825b34740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf528d5b40 0x55bf528dc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:58.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.359+0000 7fb825b34740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf528df370 0x55bf528dc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:58.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.359+0000 7fb825b34740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bf528e2070 con 0x55bf528d5b40 2024-09-19T08:56:58.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.359+0000 7fb825b34740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55bf527f5060 con 0x55bf528df370 2024-09-19T08:56:58.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.359+0000 7fb825b34740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55bf528e21f0 con 0x55bf528d45d0 2024-09-19T08:56:58.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.359+0000 7fb825b34740 10 client.0 _ll_register_callbacks cb 0x55bf525488c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:56:58.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.359+0000 7fb820cf6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf528d5b40 0x55bf528dc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:58.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.359+0000 7fb820cf6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf528d5b40 0x55bf528dc080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44368/0 (socket says 172.21.15.164:44368) 2024-09-19T08:56:58.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.359+0000 7fb820cf6640 1 -- 172.21.15.164:0/554652532 learned_addr learned my addr 172.21.15.164:0/554652532 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:58.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.359+0000 7fb821cf8640 1 --2- 172.21.15.164:0/554652532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf528df370 0x55bf528dc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:58.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.359+0000 7fb8214f7640 1 --2- 172.21.15.164:0/554652532 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf528d45d0 0x55bf528dedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:58.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.359+0000 7fb820cf6640 1 -- 172.21.15.164:0/554652532 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf528d45d0 msgr2=0x55bf528dedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:58.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.359+0000 7fb820cf6640 1 --2- 172.21.15.164:0/554652532 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf528d45d0 0x55bf528dedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.359+0000 7fb820cf6640 1 -- 172.21.15.164:0/554652532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf528df370 msgr2=0x55bf528dc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:58.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.359+0000 7fb820cf6640 1 --2- 172.21.15.164:0/554652532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf528df370 0x55bf528dc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.359+0000 7fb820cf6640 1 -- 172.21.15.164:0/554652532 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf525ab3c0 con 0x55bf528d5b40 2024-09-19T08:56:58.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.360+0000 7fb820cf6640 1 --2- 172.21.15.164:0/554652532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf528d5b40 0x55bf528dc080 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fb818010670 tx=0x7fb818010b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b85cc2b2c3f9f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:58.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.360+0000 7fb804ff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:56:58.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.360+0000 7fb804ff9640 1 -- 172.21.15.164:0/554652532 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb818004220 con 0x55bf528d5b40 2024-09-19T08:56:58.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.360+0000 7fb804ff9640 1 -- 172.21.15.164:0/554652532 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7fb8180043c0 con 0x55bf528d5b40 2024-09-19T08:56:58.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.360+0000 7fb825b34740 1 -- 172.21.15.164:0/554652532 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb81009bbf0 con 0x55bf528d5b40 2024-09-19T08:56:58.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.360+0000 7fb804ff9640 1 -- 172.21.15.164:0/554652532 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb818023050 con 0x55bf528d5b40 2024-09-19T08:56:58.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.360+0000 7fb825b34740 10 client.17880 fetch_fsmap learned FSMap version 32 2024-09-19T08:56:58.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.360+0000 7fb825b34740 1 -- 172.21.15.164:0/554652532 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bf528edbf0 con 0x55bf528d5b40 2024-09-19T08:56:58.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.360+0000 7fb804ff9640 1 -- 172.21.15.164:0/554652532 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb81009bbf0 con 0x55bf528d5b40 2024-09-19T08:56:58.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.360+0000 7fb804ff9640 1 -- 172.21.15.164:0/554652532 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb81800f040 con 0x55bf528d5b40 2024-09-19T08:56:58.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.360+0000 7fb825b34740 10 client.17880 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:56:58.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.360+0000 7fb825b34740 10 client.17880 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T08:56:58.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.360+0000 7fb825b34740 20 client.17880 populate_metadata read hostname 'smithi164' 2024-09-19T08:56:58.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.360+0000 7fb825b34740 1 --2- 172.21.15.164:0/554652532 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55bf528ef0a0 0x55bf528f14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:58.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.360+0000 7fb825b34740 4 client.17880 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:56:58.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.360+0000 7fb825b34740 1 -- 172.21.15.164:0/554652532 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf52847860 con 0x55bf528ef0a0 2024-09-19T08:56:58.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.361+0000 7fb8214f7640 1 --2- 172.21.15.164:0/554652532 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55bf528ef0a0 0x55bf528f14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:58.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.361+0000 7fb804ff9640 10 client.17880 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:56:58.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.361+0000 7fb8214f7640 1 --2- 172.21.15.164:0/554652532 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55bf528ef0a0 0x55bf528f14e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=62622d0ead711eda server_cookie=fe6437a2d1745532 in_seq=0 out_seq=0 2024-09-19T08:56:58.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.362+0000 7fb804ff9640 1 -- 172.21.15.164:0/554652532 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf52847860 con 0x55bf528ef0a0 2024-09-19T08:56:58.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.362+0000 7fb804ff9640 10 client.17880 handle_command_reply: tid=0 2024-09-19T08:56:58.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.362+0000 7fb825b34740 1 client.17880 shutdown 2024-09-19T08:56:58.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.362+0000 7fb825b34740 1 -- 172.21.15.164:0/554652532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf528d5b40 msgr2=0x55bf528dc080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:58.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.362+0000 7fb825b34740 1 --2- 172.21.15.164:0/554652532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf528d5b40 0x55bf528dc080 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fb818010670 tx=0x7fb818010b30 comp rx=0 tx=0).stop 2024-09-19T08:56:58.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.362+0000 7fb825b34740 1 -- 172.21.15.164:0/554652532 shutdown_connections 2024-09-19T08:56:58.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.362+0000 7fb825b34740 1 --2- 172.21.15.164:0/554652532 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55bf528ef0a0 0x55bf528f14e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.362+0000 7fb825b34740 1 --2- 172.21.15.164:0/554652532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf528df370 0x55bf528dc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.362+0000 7fb825b34740 1 --2- 172.21.15.164:0/554652532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf528d5b40 0x55bf528dc080 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.362+0000 7fb804ff9640 0 client.17880 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:56:58.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.362+0000 7fb825b34740 1 --2- 172.21.15.164:0/554652532 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf528d45d0 0x55bf528dedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.362+0000 7fb825b34740 1 -- 172.21.15.164:0/554652532 >> 172.21.15.164:0/554652532 conn(0x55bf5289d310 msgr2=0x55bf528d6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:58.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.363+0000 7fb825b34740 1 -- 172.21.15.164:0/554652532 shutdown_connections 2024-09-19T08:56:58.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.363+0000 7fb825b34740 1 -- 172.21.15.164:0/554652532 wait complete. 2024-09-19T08:56:58.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.363+0000 7fb825b34740 20 client.17880 trim_cache size 0 max 16384 2024-09-19T08:56:58.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.375+0000 7fb825b34740 1 Processor -- start 2024-09-19T08:56:58.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.375+0000 7fb825b34740 1 -- start start 2024-09-19T08:56:58.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.376+0000 7fb825b34740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf528d45d0 0x55bf528e7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:58.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.376+0000 7fb825b34740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf528d5b40 0x55bf528e7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:58.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.376+0000 7fb825b34740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf528df370 0x55bf528d7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:58.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.376+0000 7fb825b34740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c1ad850 con 0x55bf528d5b40 2024-09-19T08:56:58.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.376+0000 7fb825b34740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c1af390 con 0x55bf528d45d0 2024-09-19T08:56:58.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.376+0000 7fb825b34740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c1adf20 con 0x55bf528df370 2024-09-19T08:56:58.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.376+0000 7fb820cf6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf528d5b40 0x55bf528e7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:58.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.376+0000 7fb820cf6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf528d5b40 0x55bf528e7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44374/0 (socket says 172.21.15.164:44374) 2024-09-19T08:56:58.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.376+0000 7fb820cf6640 1 -- 172.21.15.164:0/746686684 learned_addr learned my addr 172.21.15.164:0/746686684 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:58.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.376+0000 7fb820cf6640 1 -- 172.21.15.164:0/746686684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf528df370 msgr2=0x55bf528d7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:58.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.376+0000 7fb820cf6640 1 --2- 172.21.15.164:0/746686684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf528df370 0x55bf528d7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.376+0000 7fb820cf6640 1 -- 172.21.15.164:0/746686684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf528d45d0 msgr2=0x55bf528e7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:58.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.376+0000 7fb820cf6640 1 --2- 172.21.15.164:0/746686684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf528d45d0 0x55bf528e7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.376+0000 7fb820cf6640 1 -- 172.21.15.164:0/746686684 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf528d1960 con 0x55bf528d5b40 2024-09-19T08:56:58.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.376+0000 7fb820cf6640 1 --2- 172.21.15.164:0/746686684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf528d5b40 0x55bf528e7da0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fb818005600 tx=0x7fb818005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92db42c2268cf442 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:58.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.376+0000 7fb806ffd640 1 -- 172.21.15.164:0/746686684 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81802c070 con 0x55bf528d5b40 2024-09-19T08:56:58.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.376+0000 7fb806ffd640 1 -- 172.21.15.164:0/746686684 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8180095a0 con 0x55bf528d5b40 2024-09-19T08:56:58.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.376+0000 7fb806ffd640 1 -- 172.21.15.164:0/746686684 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb818004eb0 con 0x55bf528d5b40 2024-09-19T08:56:58.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.377+0000 7fb825b34740 1 -- 172.21.15.164:0/746686684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf528d5b40 msgr2=0x55bf528e7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:58.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.377+0000 7fb825b34740 1 --2- 172.21.15.164:0/746686684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf528d5b40 0x55bf528e7da0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fb818005600 tx=0x7fb818005850 comp rx=0 tx=0).stop 2024-09-19T08:56:58.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.377+0000 7fb825b34740 1 -- 172.21.15.164:0/746686684 shutdown_connections 2024-09-19T08:56:58.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.377+0000 7fb825b34740 1 --2- 172.21.15.164:0/746686684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf528df370 0x55bf528d7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.377+0000 7fb825b34740 1 --2- 172.21.15.164:0/746686684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf528d5b40 0x55bf528e7da0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.377+0000 7fb825b34740 1 --2- 172.21.15.164:0/746686684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf528d45d0 0x55bf528e7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.377+0000 7fb825b34740 1 -- 172.21.15.164:0/746686684 >> 172.21.15.164:0/746686684 conn(0x55bf5289d310 msgr2=0x55bf528db170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:58.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.377+0000 7fb825b34740 1 -- 172.21.15.164:0/746686684 shutdown_connections 2024-09-19T08:56:58.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.377+0000 7fb825b34740 1 -- 172.21.15.164:0/746686684 wait complete. 2024-09-19T08:56:58.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.377+0000 7fb825b34740 1 -- start start 2024-09-19T08:56:58.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.377+0000 7fb825b34740 1 Processor -- start 2024-09-19T08:56:58.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.378+0000 7fb825b34740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf528d45d0 0x55bf528e2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:58.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.378+0000 7fb825b34740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf528d5b40 0x55bf528e2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:58.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.378+0000 7fb825b34740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf528df370 0x55bf528e2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:58.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.378+0000 7fb825b34740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c1aecc0 con 0x55bf528d45d0 2024-09-19T08:56:58.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.378+0000 7fb825b34740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c1ae5f0 con 0x55bf528df370 2024-09-19T08:56:58.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.378+0000 7fb825b34740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c1ab850 con 0x55bf528d5b40 2024-09-19T08:56:58.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.378+0000 7fb825b34740 10 client.0 _ll_register_callbacks cb 0x7fb818009480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:56:58.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.378+0000 7fb8214f7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf528d45d0 0x55bf528e2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:58.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.378+0000 7fb8214f7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf528d45d0 0x55bf528e2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44378/0 (socket says 172.21.15.164:44378) 2024-09-19T08:56:58.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.378+0000 7fb8214f7640 1 -- 172.21.15.164:0/281110806 learned_addr learned my addr 172.21.15.164:0/281110806 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:58.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.378+0000 7fb821cf8640 1 --2- 172.21.15.164:0/281110806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf528df370 0x55bf528e2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:58.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.378+0000 7fb820cf6640 1 --2- 172.21.15.164:0/281110806 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf528d5b40 0x55bf528e2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:58.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.378+0000 7fb8214f7640 1 -- 172.21.15.164:0/281110806 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf528d5b40 msgr2=0x55bf528e2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:58.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.378+0000 7fb8214f7640 1 --2- 172.21.15.164:0/281110806 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf528d5b40 0x55bf528e2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.378+0000 7fb8214f7640 1 -- 172.21.15.164:0/281110806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf528df370 msgr2=0x55bf528e2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:58.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.378+0000 7fb8214f7640 1 --2- 172.21.15.164:0/281110806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf528df370 0x55bf528e2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.378+0000 7fb8214f7640 1 -- 172.21.15.164:0/281110806 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf529615c0 con 0x55bf528d45d0 2024-09-19T08:56:58.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.378+0000 7fb8214f7640 1 --2- 172.21.15.164:0/281110806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf528d45d0 0x55bf528e2440 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fb81009b480 tx=0x7fb81009af10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19fd5177a3ea5100 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:58.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.378+0000 7fb804ff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:56:58.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.378+0000 7fb804ff9640 1 -- 172.21.15.164:0/281110806 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81001e040 con 0x55bf528d45d0 2024-09-19T08:56:58.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.378+0000 7fb804ff9640 1 -- 172.21.15.164:0/281110806 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7fb810099340 con 0x55bf528d45d0 2024-09-19T08:56:58.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.378+0000 7fb825b34740 1 -- 172.21.15.164:0/281110806 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8180095a0 con 0x55bf528d45d0 2024-09-19T08:56:58.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.379+0000 7fb804ff9640 1 -- 172.21.15.164:0/281110806 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb8100a38d0 con 0x55bf528d45d0 2024-09-19T08:56:58.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.379+0000 7fb825b34740 10 client.17886 fetch_fsmap learned FSMap version 32 2024-09-19T08:56:58.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.379+0000 7fb825b34740 1 -- 172.21.15.164:0/281110806 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bf529628c0 con 0x55bf528d45d0 2024-09-19T08:56:58.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.379+0000 7fb804ff9640 1 -- 172.21.15.164:0/281110806 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8180095a0 con 0x55bf528d45d0 2024-09-19T08:56:58.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.379+0000 7fb804ff9640 1 -- 172.21.15.164:0/281110806 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb810097b60 con 0x55bf528d45d0 2024-09-19T08:56:58.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.379+0000 7fb825b34740 10 client.17886 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:56:58.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.379+0000 7fb825b34740 10 client.17886 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T08:56:58.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.379+0000 7fb825b34740 20 client.17886 populate_metadata read hostname 'smithi164' 2024-09-19T08:56:58.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.379+0000 7fb825b34740 1 --2- 172.21.15.164:0/281110806 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55bf52962b50 0x55bf52964f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:58.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.379+0000 7fb825b34740 4 client.17886 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:56:58.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.379+0000 7fb825b34740 1 -- 172.21.15.164:0/281110806 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf52841d70 con 0x55bf52962b50 2024-09-19T08:56:58.407 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:56:58.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.379+0000 7fb820cf6640 1 --2- 172.21.15.164:0/281110806 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55bf52962b50 0x55bf52964f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:58.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.380+0000 7fb820cf6640 1 --2- 172.21.15.164:0/281110806 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55bf52962b50 0x55bf52964f40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c6ddf94112b9ac44 server_cookie=b204185c8da92407 in_seq=0 out_seq=0 2024-09-19T08:56:58.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.380+0000 7fb804ff9640 10 client.17886 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:56:58.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.380+0000 7fb804ff9640 1 -- 172.21.15.164:0/281110806 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf52841d70 con 0x55bf52962b50 2024-09-19T08:56:58.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.380+0000 7fb804ff9640 10 client.17886 handle_command_reply: tid=0 2024-09-19T08:56:58.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.380+0000 7fb825b34740 1 client.17886 shutdown 2024-09-19T08:56:58.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.380+0000 7fb825b34740 1 -- 172.21.15.164:0/281110806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf528d45d0 msgr2=0x55bf528e2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:58.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.380+0000 7fb825b34740 1 --2- 172.21.15.164:0/281110806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf528d45d0 0x55bf528e2440 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fb81009b480 tx=0x7fb81009af10 comp rx=0 tx=0).stop 2024-09-19T08:56:58.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.380+0000 7fb825b34740 1 -- 172.21.15.164:0/281110806 shutdown_connections 2024-09-19T08:56:58.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.380+0000 7fb825b34740 1 --2- 172.21.15.164:0/281110806 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55bf52962b50 0x55bf52964f40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.380+0000 7fb825b34740 1 --2- 172.21.15.164:0/281110806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bf528df370 0x55bf528e2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.380+0000 7fb825b34740 1 --2- 172.21.15.164:0/281110806 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf528d5b40 0x55bf528e2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.380+0000 7fb804ff9640 0 client.17886 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:56:58.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.380+0000 7fb825b34740 1 --2- 172.21.15.164:0/281110806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf528d45d0 0x55bf528e2440 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.380+0000 7fb825b34740 1 -- 172.21.15.164:0/281110806 >> 172.21.15.164:0/281110806 conn(0x55bf5289d310 msgr2=0x55bf528cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:58.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.380+0000 7fb825b34740 1 -- 172.21.15.164:0/281110806 shutdown_connections 2024-09-19T08:56:58.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.380+0000 7fb825b34740 1 -- 172.21.15.164:0/281110806 wait complete. 2024-09-19T08:56:58.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.380+0000 7fb825b34740 20 client.17886 trim_cache size 0 max 16384 2024-09-19T08:56:58.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.381+0000 7fb82375b640 1 -- 172.21.15.164:0/2239942232 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb8080a33b0 msgr2=0x7fb8080a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:58.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.381+0000 7fb82375b640 1 --2- 172.21.15.164:0/2239942232 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb8080a33b0 0x7fb8080a5870 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb814005fe0 tx=0x7fb814031ce0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.381+0000 7fb82375b640 1 -- 172.21.15.164:0/2239942232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb81c101de0 msgr2=0x7fb81c198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:58.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.381+0000 7fb82375b640 1 --2- 172.21.15.164:0/2239942232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb81c101de0 0x7fb81c198460 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fb81000b4a0 tx=0x7fb81000b960 comp rx=0 tx=0).stop 2024-09-19T08:56:58.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.381+0000 7fb82375b640 1 -- 172.21.15.164:0/2239942232 shutdown_connections 2024-09-19T08:56:58.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.381+0000 7fb82375b640 1 --2- 172.21.15.164:0/2239942232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb81c106f10 0x7fb81c19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.381+0000 7fb82375b640 1 --2- 172.21.15.164:0/2239942232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb81c102790 0x7fb81c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.381+0000 7fb82375b640 1 --2- 172.21.15.164:0/2239942232 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb8080a33b0 0x7fb8080a5870 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.381+0000 7fb82375b640 1 --2- 172.21.15.164:0/2239942232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb81c101de0 0x7fb81c198460 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:58.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.381+0000 7fb82375b640 1 -- 172.21.15.164:0/2239942232 >> 172.21.15.164:0/2239942232 conn(0x7fb81c0fd8c0 msgr2=0x7fb81c0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:58.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.381+0000 7fb82375b640 1 -- 172.21.15.164:0/2239942232 shutdown_connections 2024-09-19T08:56:58.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:58.381+0000 7fb82375b640 1 -- 172.21.15.164:0/2239942232 wait complete. 2024-09-19T08:56:58.531 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:56:58 smithi164 ceph-mon[30928]: pgmap v347: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:56:58.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:56:58 smithi170 ceph-mon[35072]: pgmap v347: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:56:58.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:56:58 smithi173 ceph-mon[35306]: pgmap v347: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:56:59.044 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T08:56:59.093 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:56:59.305 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:56:59.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.775+0000 7f8d1b7cc640 1 -- 172.21.15.164:0/3087554451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d14101420 msgr2=0x7f8d1410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:59.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.775+0000 7f8d1b7cc640 1 --2- 172.21.15.164:0/3087554451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d14101420 0x7f8d1410d950 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c009cd0 tx=0x7f8d0c02a8e0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.775+0000 7f8d1b7cc640 1 -- 172.21.15.164:0/3087554451 shutdown_connections 2024-09-19T08:56:59.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.775+0000 7f8d1b7cc640 1 --2- 172.21.15.164:0/3087554451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d14101420 0x7f8d1410d950 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.775+0000 7f8d1b7cc640 1 --2- 172.21.15.164:0/3087554451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d14100a80 0x7f8d14100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.775+0000 7f8d1b7cc640 1 --2- 172.21.15.164:0/3087554451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d14106a80 0x7f8d14106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.776+0000 7f8d1b7cc640 1 -- 172.21.15.164:0/3087554451 >> 172.21.15.164:0/3087554451 conn(0x7f8d140fc7a0 msgr2=0x7f8d140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:59.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.776+0000 7f8d1b7cc640 1 -- 172.21.15.164:0/3087554451 shutdown_connections 2024-09-19T08:56:59.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.776+0000 7f8d1b7cc640 1 -- 172.21.15.164:0/3087554451 wait complete. 2024-09-19T08:56:59.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.776+0000 7f8d1b7cc640 1 Processor -- start 2024-09-19T08:56:59.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.776+0000 7f8d1b7cc640 1 -- start start 2024-09-19T08:56:59.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.776+0000 7f8d1b7cc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d14100a80 0x7f8d141983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:59.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.776+0000 7f8d1b7cc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d14101420 0x7f8d14198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:59.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.776+0000 7f8d1b7cc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d14106a80 0x7f8d1419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:59.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.776+0000 7f8d1b7cc640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8d14110ea0 con 0x7f8d14106a80 2024-09-19T08:56:59.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.776+0000 7f8d1b7cc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8d14110d20 con 0x7f8d14100a80 2024-09-19T08:56:59.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.776+0000 7f8d1b7cc640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8d14111020 con 0x7f8d14101420 2024-09-19T08:56:59.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.777+0000 7f8d19d69640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d14106a80 0x7f8d1419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:59.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.777+0000 7f8d19d69640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d14106a80 0x7f8d1419ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44396/0 (socket says 172.21.15.164:44396) 2024-09-19T08:56:59.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.777+0000 7f8d19d69640 1 -- 172.21.15.164:0/496827405 learned_addr learned my addr 172.21.15.164:0/496827405 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:59.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.777+0000 7f8d19568640 1 --2- 172.21.15.164:0/496827405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d14100a80 0x7f8d141983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:59.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.777+0000 7f8d19d69640 1 -- 172.21.15.164:0/496827405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d14101420 msgr2=0x7f8d14198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:59.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.777+0000 7f8d19d69640 1 --2- 172.21.15.164:0/496827405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d14101420 0x7f8d14198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.777+0000 7f8d19d69640 1 -- 172.21.15.164:0/496827405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d14100a80 msgr2=0x7f8d141983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:59.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.777+0000 7f8d19d69640 1 --2- 172.21.15.164:0/496827405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d14100a80 0x7f8d141983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.777+0000 7f8d19d69640 1 -- 172.21.15.164:0/496827405 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d1419d3a0 con 0x7f8d14106a80 2024-09-19T08:56:59.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.777+0000 7f8d19568640 1 --2- 172.21.15.164:0/496827405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d14100a80 0x7f8d141983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:56:59.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.777+0000 7f8d19d69640 1 --2- 172.21.15.164:0/496827405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d14106a80 0x7f8d1419ccc0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c006010 tx=0x7f8d0c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:59.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.777+0000 7f8d0a7fc640 1 -- 172.21.15.164:0/496827405 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c004450 con 0x7f8d14106a80 2024-09-19T08:56:59.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.777+0000 7f8d0a7fc640 1 -- 172.21.15.164:0/496827405 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d0c003e10 con 0x7f8d14106a80 2024-09-19T08:56:59.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.777+0000 7f8d1b7cc640 1 -- 172.21.15.164:0/496827405 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d1419d630 con 0x7f8d14106a80 2024-09-19T08:56:59.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.777+0000 7f8d0a7fc640 1 -- 172.21.15.164:0/496827405 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c0466e0 con 0x7f8d14106a80 2024-09-19T08:56:59.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.777+0000 7f8d1b7cc640 1 -- 172.21.15.164:0/496827405 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d141a4f70 con 0x7f8d14106a80 2024-09-19T08:56:59.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.780+0000 7f8d0a7fc640 1 -- 172.21.15.164:0/496827405 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f8d0c0305a0 con 0x7f8d14106a80 2024-09-19T08:56:59.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.780+0000 7f8d1dba5740 1 Processor -- start 2024-09-19T08:56:59.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.780+0000 7f8d1dba5740 1 -- start start 2024-09-19T08:56:59.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.781+0000 7f8d1dba5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555804634b40 0x555804633090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:59.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.781+0000 7f8d1dba5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558046335d0 0x555804633a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:59.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.781+0000 7f8d1dba5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55580463e370 0x555804640760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:59.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.781+0000 7f8d1dba5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5558045992f0 con 0x55580463e370 2024-09-19T08:56:59.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.781+0000 7f8d1dba5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555804540cf0 con 0x555804634b40 2024-09-19T08:56:59.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.781+0000 7f8d1dba5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5558045952b0 con 0x5558046335d0 2024-09-19T08:56:59.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.781+0000 7f8d18d67640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55580463e370 0x555804640760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:59.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.781+0000 7f8d18d67640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55580463e370 0x555804640760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44406/0 (socket says 172.21.15.164:44406) 2024-09-19T08:56:59.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.781+0000 7f8d18d67640 1 -- 172.21.15.164:0/1521003148 learned_addr learned my addr 172.21.15.164:0/1521003148 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:59.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.781+0000 7f8d0a7fc640 1 --2- 172.21.15.164:0/496827405 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f8d000a33b0 0x7f8d000a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:59.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.781+0000 7f8d19568640 1 --2- 172.21.15.164:0/1521003148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558046335d0 0x555804633a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:59.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.781+0000 7f8d18d67640 1 --2- 172.21.15.164:0/1521003148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555804634b40 0x555804633090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:59.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.781+0000 7f8d19d69640 1 --2- 172.21.15.164:0/496827405 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f8d000a33b0 0x7f8d000a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:59.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.781+0000 7f8d18d67640 1 -- 172.21.15.164:0/1521003148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558046335d0 msgr2=0x555804633a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:59.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.781+0000 7f8d18d67640 1 --2- 172.21.15.164:0/1521003148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558046335d0 0x555804633a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.781+0000 7f8d18d67640 1 -- 172.21.15.164:0/1521003148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555804634b40 msgr2=0x555804633090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:59.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.781+0000 7f8d18d67640 1 --2- 172.21.15.164:0/1521003148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555804634b40 0x555804633090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.781+0000 7f8d18d67640 1 -- 172.21.15.164:0/1521003148 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558045253a0 con 0x55580463e370 2024-09-19T08:56:59.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.781+0000 7f8d0a7fc640 1 -- 172.21.15.164:0/496827405 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f8d0c02e030 con 0x7f8d14106a80 2024-09-19T08:56:59.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.781+0000 7f8d18d67640 1 --2- 172.21.15.164:0/1521003148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55580463e370 0x555804640760 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f8d04013480 tx=0x7f8d0400c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=679e1828da1167bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:59.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.782+0000 7f8d19d69640 1 --2- 172.21.15.164:0/496827405 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f8d000a33b0 0x7f8d000a5870 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0ef3c0 tx=0x7f8d0c08f7e0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:59.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.782+0000 7f8cff7fe640 1 -- 172.21.15.164:0/1521003148 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d040115e0 con 0x55580463e370 2024-09-19T08:56:59.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.782+0000 7f8cff7fe640 1 -- 172.21.15.164:0/1521003148 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d04011c60 con 0x55580463e370 2024-09-19T08:56:59.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.782+0000 7f8cff7fe640 1 -- 172.21.15.164:0/1521003148 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d04010d30 con 0x55580463e370 2024-09-19T08:56:59.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.782+0000 7f8d1dba5740 1 -- 172.21.15.164:0/1521003148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55580463e370 msgr2=0x555804640760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:59.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.782+0000 7f8d1dba5740 1 --2- 172.21.15.164:0/1521003148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55580463e370 0x555804640760 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f8d04013480 tx=0x7f8d0400c4f0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.782+0000 7f8d1dba5740 1 -- 172.21.15.164:0/1521003148 shutdown_connections 2024-09-19T08:56:59.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.782+0000 7f8d1dba5740 1 --2- 172.21.15.164:0/1521003148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55580463e370 0x555804640760 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.782+0000 7f8d1dba5740 1 --2- 172.21.15.164:0/1521003148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558046335d0 0x555804633a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.782+0000 7f8d1dba5740 1 --2- 172.21.15.164:0/1521003148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555804634b40 0x555804633090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.782+0000 7f8d1dba5740 1 -- 172.21.15.164:0/1521003148 >> 172.21.15.164:0/1521003148 conn(0x5558045fc310 msgr2=0x55580462fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:59.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.782+0000 7f8d1dba5740 1 -- 172.21.15.164:0/1521003148 shutdown_connections 2024-09-19T08:56:59.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.782+0000 7f8d1dba5740 1 -- 172.21.15.164:0/1521003148 wait complete. 2024-09-19T08:56:59.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.783+0000 7f8d1dba5740 1 -- start start 2024-09-19T08:56:59.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.783+0000 7f8d1dba5740 1 Processor -- start 2024-09-19T08:56:59.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.783+0000 7f8d1dba5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558046335d0 0x55580463dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:59.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.783+0000 7f8d1dba5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555804634b40 0x55580463b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:59.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.783+0000 7f8d1dba5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55580463e370 0x55580463b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:59.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.783+0000 7f8d1dba5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555804641070 con 0x555804634b40 2024-09-19T08:56:59.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.783+0000 7f8d1dba5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555804554060 con 0x55580463e370 2024-09-19T08:56:59.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.783+0000 7f8d1dba5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5558046411f0 con 0x5558046335d0 2024-09-19T08:56:59.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.783+0000 7f8d1dba5740 10 client.0 _ll_register_callbacks cb 0x5558042a78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:56:59.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.783+0000 7f8d19568640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555804634b40 0x55580463b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:59.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.783+0000 7f8d19568640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555804634b40 0x55580463b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44422/0 (socket says 172.21.15.164:44422) 2024-09-19T08:56:59.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.783+0000 7f8d19568640 1 -- 172.21.15.164:0/2990459596 learned_addr learned my addr 172.21.15.164:0/2990459596 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:59.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.784+0000 7f8d18d67640 1 --2- 172.21.15.164:0/2990459596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55580463e370 0x55580463b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:59.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.784+0000 7f8d18d67640 1 --2- 172.21.15.164:0/2990459596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558046335d0 0x55580463dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:59.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.784+0000 7f8d19568640 1 -- 172.21.15.164:0/2990459596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558046335d0 msgr2=0x55580463dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:59.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.784+0000 7f8d19568640 1 --2- 172.21.15.164:0/2990459596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558046335d0 0x55580463dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.784+0000 7f8d19568640 1 -- 172.21.15.164:0/2990459596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55580463e370 msgr2=0x55580463b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:59.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.784+0000 7f8d19568640 1 --2- 172.21.15.164:0/2990459596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55580463e370 0x55580463b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.784+0000 7f8d19568640 1 -- 172.21.15.164:0/2990459596 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55580430a3c0 con 0x555804634b40 2024-09-19T08:56:59.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.784+0000 7f8d18d67640 1 --2- 172.21.15.164:0/2990459596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558046335d0 0x55580463dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:56:59.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.784+0000 7f8d19568640 1 --2- 172.21.15.164:0/2990459596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555804634b40 0x55580463b010 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f8d1000e500 tx=0x7f8d1000e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb42fb9c083c1ed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:59.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.784+0000 7f8cfcff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:56:59.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.784+0000 7f8cfcff9640 1 -- 172.21.15.164:0/2990459596 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d10017020 con 0x555804634b40 2024-09-19T08:56:59.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.784+0000 7f8cfcff9640 1 -- 172.21.15.164:0/2990459596 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f8d10013660 con 0x555804634b40 2024-09-19T08:56:59.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.784+0000 7f8d1dba5740 1 -- 172.21.15.164:0/2990459596 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d04011c60 con 0x555804634b40 2024-09-19T08:56:59.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.784+0000 7f8cfcff9640 1 -- 172.21.15.164:0/2990459596 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8d10022050 con 0x555804634b40 2024-09-19T08:56:59.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.784+0000 7f8d1dba5740 10 client.17898 fetch_fsmap learned FSMap version 32 2024-09-19T08:56:59.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.784+0000 7f8d1dba5740 1 -- 172.21.15.164:0/2990459596 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55580464cb70 con 0x555804634b40 2024-09-19T08:56:59.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.784+0000 7f8cfcff9640 1 -- 172.21.15.164:0/2990459596 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d04011c60 con 0x555804634b40 2024-09-19T08:56:59.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.785+0000 7f8cfcff9640 1 -- 172.21.15.164:0/2990459596 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f8d10012b20 con 0x555804634b40 2024-09-19T08:56:59.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.785+0000 7f8d1dba5740 10 client.17898 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:56:59.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.785+0000 7f8d1dba5740 10 client.17898 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:56:59.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.785+0000 7f8d1dba5740 20 client.17898 populate_metadata read hostname 'smithi164' 2024-09-19T08:56:59.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.785+0000 7f8d1dba5740 1 --2- 172.21.15.164:0/2990459596 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55580464e080 0x5558046504c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:59.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.785+0000 7f8d1dba5740 4 client.17898 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:56:59.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.785+0000 7f8d1dba5740 1 -- 172.21.15.164:0/2990459596 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558045a6860 con 0x55580464e080 2024-09-19T08:56:59.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.785+0000 7f8d18d67640 1 --2- 172.21.15.164:0/2990459596 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55580464e080 0x5558046504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:59.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.785+0000 7f8cfcff9640 10 client.17898 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:56:59.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.785+0000 7f8d18d67640 1 --2- 172.21.15.164:0/2990459596 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55580464e080 0x5558046504c0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4247a28450dfd0bd server_cookie=c8948f5a80501de5 in_seq=0 out_seq=0 2024-09-19T08:56:59.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.786+0000 7f8cfcff9640 1 -- 172.21.15.164:0/2990459596 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5558045a6860 con 0x55580464e080 2024-09-19T08:56:59.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.786+0000 7f8cfcff9640 10 client.17898 handle_command_reply: tid=0 2024-09-19T08:56:59.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.786+0000 7f8d1dba5740 1 client.17898 shutdown 2024-09-19T08:56:59.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.786+0000 7f8d1dba5740 1 -- 172.21.15.164:0/2990459596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555804634b40 msgr2=0x55580463b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:59.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.786+0000 7f8d1dba5740 1 --2- 172.21.15.164:0/2990459596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555804634b40 0x55580463b010 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f8d1000e500 tx=0x7f8d1000e9c0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.786+0000 7f8d1dba5740 1 -- 172.21.15.164:0/2990459596 shutdown_connections 2024-09-19T08:56:59.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.786+0000 7f8d1dba5740 1 --2- 172.21.15.164:0/2990459596 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55580464e080 0x5558046504c0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.786+0000 7f8d1dba5740 1 --2- 172.21.15.164:0/2990459596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55580463e370 0x55580463b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.786+0000 7f8d1dba5740 1 --2- 172.21.15.164:0/2990459596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555804634b40 0x55580463b010 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.786+0000 7f8d1dba5740 1 --2- 172.21.15.164:0/2990459596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558046335d0 0x55580463dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.786+0000 7f8d1dba5740 1 -- 172.21.15.164:0/2990459596 >> 172.21.15.164:0/2990459596 conn(0x5558045fc310 msgr2=0x555804635960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:59.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.786+0000 7f8cfcff9640 0 client.17898 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:56:59.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.786+0000 7f8d1dba5740 1 -- 172.21.15.164:0/2990459596 shutdown_connections 2024-09-19T08:56:59.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.786+0000 7f8d1dba5740 1 -- 172.21.15.164:0/2990459596 wait complete. 2024-09-19T08:56:59.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.787+0000 7f8d1dba5740 20 client.17898 trim_cache size 0 max 16384 2024-09-19T08:56:59.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.799+0000 7f8d1dba5740 1 Processor -- start 2024-09-19T08:56:59.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.800+0000 7f8d1dba5740 1 -- start start 2024-09-19T08:56:59.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.800+0000 7f8d1dba5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558046335d0 0x55580463afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:59.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.800+0000 7f8d1dba5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555804634b40 0x55580464a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:59.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.800+0000 7f8d1dba5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55580463e370 0x555804646860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:59.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.800+0000 7f8d1dba5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141ad6a0 con 0x555804634b40 2024-09-19T08:56:59.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.800+0000 7f8d1dba5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141af1e0 con 0x55580463e370 2024-09-19T08:56:59.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.800+0000 7f8d1dba5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141add70 con 0x5558046335d0 2024-09-19T08:56:59.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.800+0000 7f8d19568640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555804634b40 0x55580464a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:59.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.800+0000 7f8d19568640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555804634b40 0x55580464a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44436/0 (socket says 172.21.15.164:44436) 2024-09-19T08:56:59.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.800+0000 7f8d19568640 1 -- 172.21.15.164:0/601280251 learned_addr learned my addr 172.21.15.164:0/601280251 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:59.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.800+0000 7f8d18d67640 1 --2- 172.21.15.164:0/601280251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55580463e370 0x555804646860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:59.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.800+0000 7f8d19568640 1 -- 172.21.15.164:0/601280251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558046335d0 msgr2=0x55580463afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:59.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.800+0000 7f8d19568640 1 --2- 172.21.15.164:0/601280251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558046335d0 0x55580463afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.800+0000 7f8d19568640 1 -- 172.21.15.164:0/601280251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55580463e370 msgr2=0x555804646860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:59.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.800+0000 7f8d19568640 1 --2- 172.21.15.164:0/601280251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55580463e370 0x555804646860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.800+0000 7f8d19568640 1 -- 172.21.15.164:0/601280251 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555804630960 con 0x555804634b40 2024-09-19T08:56:59.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.800+0000 7f8d18d67640 1 --2- 172.21.15.164:0/601280251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55580463e370 0x555804646860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:56:59.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.800+0000 7f8d19568640 1 --2- 172.21.15.164:0/601280251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555804634b40 0x55580464a310 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f8d10001eb0 tx=0x7f8d1000e490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bbbb1b2b0306541 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:59.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.800+0000 7f8cfeffd640 1 -- 172.21.15.164:0/601280251 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d10017070 con 0x555804634b40 2024-09-19T08:56:59.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.800+0000 7f8cfeffd640 1 -- 172.21.15.164:0/601280251 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d1001b6c0 con 0x555804634b40 2024-09-19T08:56:59.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.800+0000 7f8cfeffd640 1 -- 172.21.15.164:0/601280251 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d10013e30 con 0x555804634b40 2024-09-19T08:56:59.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.801+0000 7f8d1dba5740 1 -- 172.21.15.164:0/601280251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555804634b40 msgr2=0x55580464a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:59.819 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.801+0000 7f8d1dba5740 1 --2- 172.21.15.164:0/601280251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555804634b40 0x55580464a310 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f8d10001eb0 tx=0x7f8d1000e490 comp rx=0 tx=0).stop 2024-09-19T08:56:59.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.801+0000 7f8d1dba5740 1 -- 172.21.15.164:0/601280251 shutdown_connections 2024-09-19T08:56:59.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.801+0000 7f8d1dba5740 1 --2- 172.21.15.164:0/601280251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55580463e370 0x555804646860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.801+0000 7f8d1dba5740 1 --2- 172.21.15.164:0/601280251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555804634b40 0x55580464a310 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.801+0000 7f8d1dba5740 1 --2- 172.21.15.164:0/601280251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558046335d0 0x55580463afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.801+0000 7f8d1dba5740 1 -- 172.21.15.164:0/601280251 >> 172.21.15.164:0/601280251 conn(0x5558045fc310 msgr2=0x55580463a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:59.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.801+0000 7f8d1dba5740 1 -- 172.21.15.164:0/601280251 shutdown_connections 2024-09-19T08:56:59.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.801+0000 7f8d1dba5740 1 -- 172.21.15.164:0/601280251 wait complete. 2024-09-19T08:56:59.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.801+0000 7f8d1dba5740 1 -- start start 2024-09-19T08:56:59.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.802+0000 7f8d1dba5740 1 Processor -- start 2024-09-19T08:56:59.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.802+0000 7f8d1dba5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558046335d0 0x555804654ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:59.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.802+0000 7f8d1dba5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555804634b40 0x5558046553e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:59.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.802+0000 7f8d1dba5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55580463e370 0x5558046bad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:59.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.802+0000 7f8d1dba5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141aeb10 con 0x555804634b40 2024-09-19T08:56:59.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.802+0000 7f8d1dba5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141ae440 con 0x55580463e370 2024-09-19T08:56:59.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.802+0000 7f8d1dba5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141ab6a0 con 0x5558046335d0 2024-09-19T08:56:59.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.802+0000 7f8d1dba5740 10 client.0 _ll_register_callbacks cb 0x7f8d1000c1f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:56:59.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.802+0000 7f8d19568640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555804634b40 0x5558046553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:59.821 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.802+0000 7f8d19568640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555804634b40 0x5558046553e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44442/0 (socket says 172.21.15.164:44442) 2024-09-19T08:56:59.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.802+0000 7f8d19568640 1 -- 172.21.15.164:0/4097599215 learned_addr learned my addr 172.21.15.164:0/4097599215 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:56:59.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.802+0000 7f8d18d67640 1 --2- 172.21.15.164:0/4097599215 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558046335d0 0x555804654ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:59.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.802+0000 7f8d18d67640 1 --2- 172.21.15.164:0/4097599215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55580463e370 0x5558046bad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:59.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.802+0000 7f8d19568640 1 -- 172.21.15.164:0/4097599215 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558046335d0 msgr2=0x555804654ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:59.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.802+0000 7f8d19568640 1 --2- 172.21.15.164:0/4097599215 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558046335d0 0x555804654ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.802+0000 7f8d19568640 1 -- 172.21.15.164:0/4097599215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55580463e370 msgr2=0x5558046bad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:56:59.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.802+0000 7f8d19568640 1 --2- 172.21.15.164:0/4097599215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55580463e370 0x5558046bad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.803+0000 7f8d19568640 1 -- 172.21.15.164:0/4097599215 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555804655cc0 con 0x555804634b40 2024-09-19T08:56:59.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.803+0000 7f8d19568640 1 --2- 172.21.15.164:0/4097599215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555804634b40 0x5558046553e0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f8d10010010 tx=0x7f8d10013910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddbb7c2d08c05ff0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:56:59.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.803+0000 7f8cfcff9640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:56:59.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.803+0000 7f8cfcff9640 1 -- 172.21.15.164:0/4097599215 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d10017040 con 0x555804634b40 2024-09-19T08:56:59.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.803+0000 7f8cfcff9640 1 -- 172.21.15.164:0/4097599215 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f8d100059c0 con 0x555804634b40 2024-09-19T08:56:59.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.803+0000 7f8d1dba5740 1 -- 172.21.15.164:0/4097599215 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d1001b6c0 con 0x555804634b40 2024-09-19T08:56:59.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.803+0000 7f8cfcff9640 1 -- 172.21.15.164:0/4097599215 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8d10005bc0 con 0x555804634b40 2024-09-19T08:56:59.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.803+0000 7f8d1dba5740 10 client.17904 fetch_fsmap learned FSMap version 32 2024-09-19T08:56:59.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.803+0000 7f8d1dba5740 1 -- 172.21.15.164:0/4097599215 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5558046bc3e0 con 0x555804634b40 2024-09-19T08:56:59.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.803+0000 7f8cfcff9640 1 -- 172.21.15.164:0/4097599215 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d1001b6c0 con 0x555804634b40 2024-09-19T08:56:59.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.804+0000 7f8cfcff9640 1 -- 172.21.15.164:0/4097599215 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f8d10006270 con 0x555804634b40 2024-09-19T08:56:59.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.804+0000 7f8d1dba5740 10 client.17904 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:56:59.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.804+0000 7f8d1dba5740 10 client.17904 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:56:59.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.804+0000 7f8d1dba5740 20 client.17904 populate_metadata read hostname 'smithi164' 2024-09-19T08:56:59.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.804+0000 7f8d1dba5740 1 --2- 172.21.15.164:0/4097599215 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5558046bcad0 0x5558046beef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:56:59.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.804+0000 7f8d1dba5740 4 client.17904 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:56:59.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.804+0000 7f8d1dba5740 1 -- 172.21.15.164:0/4097599215 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5558045a0d70 con 0x5558046bcad0 2024-09-19T08:56:59.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.804+0000 7f8d18d67640 1 --2- 172.21.15.164:0/4097599215 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5558046bcad0 0x5558046beef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:56:59.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.804+0000 7f8d18d67640 1 --2- 172.21.15.164:0/4097599215 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5558046bcad0 0x5558046beef0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b8b4158b8b3d9d2 server_cookie=fb1570b58ce348a6 in_seq=0 out_seq=0 2024-09-19T08:56:59.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.804+0000 7f8cfcff9640 10 client.17904 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:56:59.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.804+0000 7f8cfcff9640 1 -- 172.21.15.164:0/4097599215 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5558045a0d70 con 0x5558046bcad0 2024-09-19T08:56:59.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.804+0000 7f8cfcff9640 10 client.17904 handle_command_reply: tid=0 2024-09-19T08:56:59.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.804+0000 7f8d1dba5740 1 client.17904 shutdown 2024-09-19T08:56:59.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.805+0000 7f8d1dba5740 1 -- 172.21.15.164:0/4097599215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555804634b40 msgr2=0x5558046553e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:59.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.805+0000 7f8d1dba5740 1 --2- 172.21.15.164:0/4097599215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555804634b40 0x5558046553e0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f8d10010010 tx=0x7f8d10013910 comp rx=0 tx=0).stop 2024-09-19T08:56:59.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.805+0000 7f8d1dba5740 1 -- 172.21.15.164:0/4097599215 shutdown_connections 2024-09-19T08:56:59.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.805+0000 7f8d1dba5740 1 --2- 172.21.15.164:0/4097599215 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5558046bcad0 0x5558046beef0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.805+0000 7f8d1dba5740 1 --2- 172.21.15.164:0/4097599215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55580463e370 0x5558046bad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.805+0000 7f8cfcff9640 0 client.17904 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:56:59.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.805+0000 7f8d1dba5740 1 --2- 172.21.15.164:0/4097599215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555804634b40 0x5558046553e0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.805+0000 7f8d1dba5740 1 --2- 172.21.15.164:0/4097599215 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558046335d0 0x555804654ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.805+0000 7f8d1dba5740 1 -- 172.21.15.164:0/4097599215 >> 172.21.15.164:0/4097599215 conn(0x5558045fc310 msgr2=0x55580462e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:59.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.805+0000 7f8d1dba5740 1 -- 172.21.15.164:0/4097599215 shutdown_connections 2024-09-19T08:56:59.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.805+0000 7f8d1dba5740 1 -- 172.21.15.164:0/4097599215 wait complete. 2024-09-19T08:56:59.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.805+0000 7f8d1dba5740 20 client.17904 trim_cache size 0 max 16384 2024-09-19T08:56:59.826 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:56:59.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.805+0000 7f8d1b7cc640 1 -- 172.21.15.164:0/496827405 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f8d000a33b0 msgr2=0x7f8d000a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:59.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.805+0000 7f8d1b7cc640 1 --2- 172.21.15.164:0/496827405 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f8d000a33b0 0x7f8d000a5870 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0ef3c0 tx=0x7f8d0c08f7e0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.806+0000 7f8d1b7cc640 1 -- 172.21.15.164:0/496827405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d14106a80 msgr2=0x7f8d1419ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:56:59.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.806+0000 7f8d1b7cc640 1 --2- 172.21.15.164:0/496827405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d14106a80 0x7f8d1419ccc0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c006010 tx=0x7f8d0c003c20 comp rx=0 tx=0).stop 2024-09-19T08:56:59.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.806+0000 7f8d1b7cc640 1 -- 172.21.15.164:0/496827405 shutdown_connections 2024-09-19T08:56:59.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.806+0000 7f8d1b7cc640 1 --2- 172.21.15.164:0/496827405 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f8d000a33b0 0x7f8d000a5870 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.806+0000 7f8d1b7cc640 1 --2- 172.21.15.164:0/496827405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d14106a80 0x7f8d1419ccc0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.806+0000 7f8d1b7cc640 1 --2- 172.21.15.164:0/496827405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d14101420 0x7f8d14198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.806+0000 7f8d1b7cc640 1 --2- 172.21.15.164:0/496827405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d14100a80 0x7f8d141983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:56:59.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.806+0000 7f8d1b7cc640 1 -- 172.21.15.164:0/496827405 >> 172.21.15.164:0/496827405 conn(0x7f8d140fc7a0 msgr2=0x7f8d1406a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:56:59.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.806+0000 7f8d1b7cc640 1 -- 172.21.15.164:0/496827405 shutdown_connections 2024-09-19T08:56:59.827 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:56:59.806+0000 7f8d1b7cc640 1 -- 172.21.15.164:0/496827405 wait complete. 2024-09-19T08:56:59.828 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:57:00.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:00 smithi170 ceph-mon[35072]: pgmap v348: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:57:00.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:00 smithi173 ceph-mon[35306]: pgmap v348: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:57:00.663 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:00 smithi164 ceph-mon[30928]: pgmap v348: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:57:01.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.076+0000 7f2af3a0a640 1 -- 172.21.15.164:0/3195661734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2aec101430 msgr2=0x7f2aec10d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:01.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.076+0000 7f2af3a0a640 1 --2- 172.21.15.164:0/3195661734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2aec101430 0x7f2aec10d960 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4009930 tx=0x7f2ae402a420 comp rx=0 tx=0).stop 2024-09-19T08:57:01.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.077+0000 7f2af3a0a640 1 -- 172.21.15.164:0/3195661734 shutdown_connections 2024-09-19T08:57:01.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.077+0000 7f2af3a0a640 1 --2- 172.21.15.164:0/3195661734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2aec101430 0x7f2aec10d960 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:01.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.077+0000 7f2af3a0a640 1 --2- 172.21.15.164:0/3195661734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aec100a90 0x7f2aec100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:01.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.077+0000 7f2af3a0a640 1 --2- 172.21.15.164:0/3195661734 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2aec106a90 0x7f2aec106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:01.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.077+0000 7f2af3a0a640 1 -- 172.21.15.164:0/3195661734 >> 172.21.15.164:0/3195661734 conn(0x7f2aec0fc7d0 msgr2=0x7f2aec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:01.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.077+0000 7f2af3a0a640 1 -- 172.21.15.164:0/3195661734 shutdown_connections 2024-09-19T08:57:01.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.077+0000 7f2af3a0a640 1 -- 172.21.15.164:0/3195661734 wait complete. 2024-09-19T08:57:01.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.077+0000 7f2af3a0a640 1 Processor -- start 2024-09-19T08:57:01.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.077+0000 7f2af3a0a640 1 -- start start 2024-09-19T08:57:01.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.078+0000 7f2af3a0a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2aec100a90 0x7f2aec1992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:01.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.078+0000 7f2af3a0a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2aec101430 0x7f2aec199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:01.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.078+0000 7f2af3a0a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aec106a90 0x7f2aec1a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:01.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.078+0000 7f2af3a0a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec110eb0 con 0x7f2aec100a90 2024-09-19T08:57:01.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.078+0000 7f2af3a0a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec110d30 con 0x7f2aec106a90 2024-09-19T08:57:01.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.078+0000 7f2af3a0a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec111030 con 0x7f2aec101430 2024-09-19T08:57:01.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.078+0000 7f2af17a6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2aec100a90 0x7f2aec1992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:01.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.078+0000 7f2af17a6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2aec100a90 0x7f2aec1992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44474/0 (socket says 172.21.15.164:44474) 2024-09-19T08:57:01.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.078+0000 7f2af17a6640 1 -- 172.21.15.164:0/4034246900 learned_addr learned my addr 172.21.15.164:0/4034246900 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:01.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.078+0000 7f2af1fa7640 1 --2- 172.21.15.164:0/4034246900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aec106a90 0x7f2aec1a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:01.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.078+0000 7f2af0fa5640 1 --2- 172.21.15.164:0/4034246900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2aec101430 0x7f2aec199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:01.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.078+0000 7f2af17a6640 1 -- 172.21.15.164:0/4034246900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2aec101430 msgr2=0x7f2aec199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:01.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.078+0000 7f2af17a6640 1 --2- 172.21.15.164:0/4034246900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2aec101430 0x7f2aec199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:01.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.078+0000 7f2af17a6640 1 -- 172.21.15.164:0/4034246900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aec106a90 msgr2=0x7f2aec1a4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:01.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.078+0000 7f2af17a6640 1 --2- 172.21.15.164:0/4034246900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aec106a90 0x7f2aec1a4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:01.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.078+0000 7f2af17a6640 1 -- 172.21.15.164:0/4034246900 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aec1a48a0 con 0x7f2aec100a90 2024-09-19T08:57:01.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.079+0000 7f2af17a6640 1 --2- 172.21.15.164:0/4034246900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2aec100a90 0x7f2aec1992e0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f2adc00ba50 tx=0x7f2adc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:01.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.079+0000 7f2aea7fc640 1 -- 172.21.15.164:0/4034246900 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc019070 con 0x7f2aec100a90 2024-09-19T08:57:01.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.079+0000 7f2aea7fc640 1 -- 172.21.15.164:0/4034246900 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2adc00cbe0 con 0x7f2aec100a90 2024-09-19T08:57:01.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.079+0000 7f2af3a0a640 1 -- 172.21.15.164:0/4034246900 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aec1a4a70 con 0x7f2aec100a90 2024-09-19T08:57:01.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.079+0000 7f2aea7fc640 1 -- 172.21.15.164:0/4034246900 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc0076e0 con 0x7f2aec100a90 2024-09-19T08:57:01.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.079+0000 7f2af3a0a640 1 -- 172.21.15.164:0/4034246900 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aec1a4f30 con 0x7f2aec100a90 2024-09-19T08:57:01.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.080+0000 7f2af3a0a640 1 -- 172.21.15.164:0/4034246900 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aec100ef0 con 0x7f2aec100a90 2024-09-19T08:57:01.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.081+0000 7f2aea7fc640 1 -- 172.21.15.164:0/4034246900 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f2adc01c070 con 0x7f2aec100a90 2024-09-19T08:57:01.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.082+0000 7f2aea7fc640 1 --2- 172.21.15.164:0/4034246900 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f2ad40a33b0 0x7f2ad40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:01.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.082+0000 7f2af0fa5640 1 --2- 172.21.15.164:0/4034246900 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f2ad40a33b0 0x7f2ad40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:01.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.082+0000 7f2aea7fc640 1 -- 172.21.15.164:0/4034246900 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f2adc0cf230 con 0x7f2aec100a90 2024-09-19T08:57:01.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.082+0000 7f2af0fa5640 1 --2- 172.21.15.164:0/4034246900 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f2ad40a33b0 0x7f2ad40a5870 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2aec06b2e0 tx=0x7f2ae000b590 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:01.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.086+0000 7f2aea7fc640 1 -- 172.21.15.164:0/4034246900 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2adc09b6b0 con 0x7f2aec100a90 2024-09-19T08:57:01.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.289+0000 7f2af3a0a640 1 -- 172.21.15.164:0/4034246900 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2aec1a5290 con 0x7f2aec100a90 2024-09-19T08:57:01.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.290+0000 7f2aea7fc640 1 -- 172.21.15.164:0/4034246900 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11984 (secure 0 0 0) 0x7f2adc09f9b0 con 0x7f2aec100a90 2024-09-19T08:57:01.290 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:57:01.290 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":32,"btime":"2024-09-19T08:52:27:192804+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:52:27.074168+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[14595,24449,24424,24442,24479]},"id":1}]} 2024-09-19T08:57:01.290 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 32 2024-09-19T08:57:01.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.292+0000 7f2af3a0a640 1 -- 172.21.15.164:0/4034246900 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f2ad40a33b0 msgr2=0x7f2ad40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:01.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.292+0000 7f2af3a0a640 1 --2- 172.21.15.164:0/4034246900 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f2ad40a33b0 0x7f2ad40a5870 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2aec06b2e0 tx=0x7f2ae000b590 comp rx=0 tx=0).stop 2024-09-19T08:57:01.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.292+0000 7f2af3a0a640 1 -- 172.21.15.164:0/4034246900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2aec100a90 msgr2=0x7f2aec1992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:01.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.292+0000 7f2af3a0a640 1 --2- 172.21.15.164:0/4034246900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2aec100a90 0x7f2aec1992e0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f2adc00ba50 tx=0x7f2adc00bf10 comp rx=0 tx=0).stop 2024-09-19T08:57:01.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.292+0000 7f2af3a0a640 1 -- 172.21.15.164:0/4034246900 shutdown_connections 2024-09-19T08:57:01.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.292+0000 7f2af3a0a640 1 --2- 172.21.15.164:0/4034246900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aec106a90 0x7f2aec1a4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:01.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.292+0000 7f2af3a0a640 1 --2- 172.21.15.164:0/4034246900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2aec101430 0x7f2aec199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:01.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.292+0000 7f2af3a0a640 1 --2- 172.21.15.164:0/4034246900 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f2ad40a33b0 0x7f2ad40a5870 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:01.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.292+0000 7f2af3a0a640 1 --2- 172.21.15.164:0/4034246900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2aec100a90 0x7f2aec1992e0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:01.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.292+0000 7f2af3a0a640 1 -- 172.21.15.164:0/4034246900 >> 172.21.15.164:0/4034246900 conn(0x7f2aec0fc7d0 msgr2=0x7f2aec0fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:01.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.292+0000 7f2af3a0a640 1 -- 172.21.15.164:0/4034246900 shutdown_connections 2024-09-19T08:57:01.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:01.292+0000 7f2af3a0a640 1 -- 172.21.15.164:0/4034246900 wait complete. 2024-09-19T08:57:01.519 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T08:57:01.539 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T08:57:01.565 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:01 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/4034246900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:57:01.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:01 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/4034246900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:57:01.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:01 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/4034246900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:57:02.010 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id e5bd5b7c --release --await' 2024-09-19T08:57:02.010 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id e5bd5b7c --release --await 2024-09-19T08:57:02.123 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:57:02.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:02 smithi164 ceph-mon[30928]: pgmap v349: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:57:02.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:02 smithi170 ceph-mon[35072]: pgmap v349: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:57:02.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.622+0000 7f991bb46640 1 -- 172.21.15.164:0/1379435414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9914102d30 msgr2=0x7f9914103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:02.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.622+0000 7f991bb46640 1 --2- 172.21.15.164:0/1379435414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9914102d30 0x7f9914103110 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f9904009cd0 tx=0x7f990402a8e0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.622+0000 7f991bb46640 1 -- 172.21.15.164:0/1379435414 shutdown_connections 2024-09-19T08:57:02.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.622+0000 7f991bb46640 1 --2- 172.21.15.164:0/1379435414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99141082a0 0x7f991410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.622+0000 7f991bb46640 1 --2- 172.21.15.164:0/1379435414 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99141036e0 0x7f9914107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.622+0000 7f991bb46640 1 --2- 172.21.15.164:0/1379435414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9914102d30 0x7f9914103110 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.623+0000 7f991bb46640 1 -- 172.21.15.164:0/1379435414 >> 172.21.15.164:0/1379435414 conn(0x7f99140fc7d0 msgr2=0x7f99140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:02.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.623+0000 7f991bb46640 1 -- 172.21.15.164:0/1379435414 shutdown_connections 2024-09-19T08:57:02.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.623+0000 7f991bb46640 1 -- 172.21.15.164:0/1379435414 wait complete. 2024-09-19T08:57:02.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.623+0000 7f991bb46640 1 Processor -- start 2024-09-19T08:57:02.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.623+0000 7f991bb46640 1 -- start start 2024-09-19T08:57:02.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.623+0000 7f991bb46640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9914102d30 0x7f9914198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:02.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.623+0000 7f991bb46640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99141036e0 0x7f9914198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:02.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.623+0000 7f991bb46640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99141082a0 0x7f991419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:02.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.623+0000 7f991bb46640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9914112100 con 0x7f99141082a0 2024-09-19T08:57:02.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.623+0000 7f991bb46640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9914111f80 con 0x7f99141036e0 2024-09-19T08:57:02.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.623+0000 7f991bb46640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9914112280 con 0x7f9914102d30 2024-09-19T08:57:02.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.624+0000 7f991a0e3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99141082a0 0x7f991419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:02.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.624+0000 7f991a0e3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99141082a0 0x7f991419ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44492/0 (socket says 172.21.15.164:44492) 2024-09-19T08:57:02.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.624+0000 7f991a0e3640 1 -- 172.21.15.164:0/4099711897 learned_addr learned my addr 172.21.15.164:0/4099711897 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:02.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.624+0000 7f99198e2640 1 --2- 172.21.15.164:0/4099711897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9914102d30 0x7f9914198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:02.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.624+0000 7f99190e1640 1 --2- 172.21.15.164:0/4099711897 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99141036e0 0x7f9914198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:02.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.624+0000 7f991a0e3640 1 -- 172.21.15.164:0/4099711897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9914102d30 msgr2=0x7f9914198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:02.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.624+0000 7f991a0e3640 1 --2- 172.21.15.164:0/4099711897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9914102d30 0x7f9914198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.624+0000 7f991a0e3640 1 -- 172.21.15.164:0/4099711897 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99141036e0 msgr2=0x7f9914198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:02.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.624+0000 7f991a0e3640 1 --2- 172.21.15.164:0/4099711897 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99141036e0 0x7f9914198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.624+0000 7f991a0e3640 1 -- 172.21.15.164:0/4099711897 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991419d530 con 0x7f99141082a0 2024-09-19T08:57:02.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.624+0000 7f99198e2640 1 --2- 172.21.15.164:0/4099711897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9914102d30 0x7f9914198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:57:02.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.624+0000 7f991a0e3640 1 --2- 172.21.15.164:0/4099711897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99141082a0 0x7f991419ce50 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f990c009fd0 tx=0x7f990c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:02.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.624+0000 7f9912ffd640 1 -- 172.21.15.164:0/4099711897 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c011830 con 0x7f99141082a0 2024-09-19T08:57:02.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.624+0000 7f9912ffd640 1 -- 172.21.15.164:0/4099711897 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f990c016070 con 0x7f99141082a0 2024-09-19T08:57:02.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.624+0000 7f991bb46640 1 -- 172.21.15.164:0/4099711897 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f991419d820 con 0x7f99141082a0 2024-09-19T08:57:02.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.624+0000 7f9912ffd640 1 -- 172.21.15.164:0/4099711897 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c00f2e0 con 0x7f99141082a0 2024-09-19T08:57:02.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.624+0000 7f991bb46640 1 -- 172.21.15.164:0/4099711897 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99141a5100 con 0x7f99141082a0 2024-09-19T08:57:02.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.627+0000 7f9912ffd640 1 -- 172.21.15.164:0/4099711897 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f990c0119d0 con 0x7f99141082a0 2024-09-19T08:57:02.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.627+0000 7f991df1f740 1 Processor -- start 2024-09-19T08:57:02.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.627+0000 7f991df1f740 1 -- start start 2024-09-19T08:57:02.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.627+0000 7f991df1f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c18eb43b60 0x55c18eb43f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:02.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.627+0000 7f991df1f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c18eb44480 0x55c18eb4e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:02.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.627+0000 7f991df1f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c18eb4ed40 0x55c18eb51130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:02.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.627+0000 7f991df1f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c18eaa9280 con 0x55c18eb4ed40 2024-09-19T08:57:02.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.627+0000 7f991df1f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c18ea50b10 con 0x55c18eb44480 2024-09-19T08:57:02.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.627+0000 7f991df1f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c18eaa5240 con 0x55c18eb43b60 2024-09-19T08:57:02.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.627+0000 7f9912ffd640 1 --2- 172.21.15.164:0/4099711897 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f98fc0a33b0 0x7f98fc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:02.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.627+0000 7f99190e1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c18eb4ed40 0x55c18eb51130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:02.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.627+0000 7f99190e1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c18eb4ed40 0x55c18eb51130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44498/0 (socket says 172.21.15.164:44498) 2024-09-19T08:57:02.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.627+0000 7f99190e1640 1 -- 172.21.15.164:0/1882412116 learned_addr learned my addr 172.21.15.164:0/1882412116 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:02.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.628+0000 7f991a0e3640 1 --2- 172.21.15.164:0/4099711897 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f98fc0a33b0 0x7f98fc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:02.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.628+0000 7f99190e1640 1 --2- 172.21.15.164:0/1882412116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c18eb43b60 0x55c18eb43f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:02.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.628+0000 7f99198e2640 1 --2- 172.21.15.164:0/1882412116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c18eb44480 0x55c18eb4e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:02.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.628+0000 7f9912ffd640 1 -- 172.21.15.164:0/4099711897 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f990c0cf360 con 0x7f99141082a0 2024-09-19T08:57:02.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.628+0000 7f99190e1640 1 -- 172.21.15.164:0/1882412116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c18eb43b60 msgr2=0x55c18eb43f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:02.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.628+0000 7f99190e1640 1 --2- 172.21.15.164:0/1882412116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c18eb43b60 0x55c18eb43f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.628+0000 7f99190e1640 1 -- 172.21.15.164:0/1882412116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c18eb44480 msgr2=0x55c18eb4e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:02.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.628+0000 7f99190e1640 1 --2- 172.21.15.164:0/1882412116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c18eb44480 0x55c18eb4e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.628+0000 7f99190e1640 1 -- 172.21.15.164:0/1882412116 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c18ea5d1f0 con 0x55c18eb4ed40 2024-09-19T08:57:02.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.628+0000 7f99190e1640 1 --2- 172.21.15.164:0/1882412116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c18eb4ed40 0x55c18eb51130 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f9908005e10 tx=0x7f990800b620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e5dff0cc0e89f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:02.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.628+0000 7f991a0e3640 1 --2- 172.21.15.164:0/4099711897 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f98fc0a33b0 0x7f98fc0a5870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f990c097b90 tx=0x7f990c062040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:02.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.628+0000 7f98fbfff640 1 -- 172.21.15.164:0/1882412116 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9908018030 con 0x55c18eb4ed40 2024-09-19T08:57:02.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.628+0000 7f98fbfff640 1 -- 172.21.15.164:0/1882412116 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9908010c00 con 0x55c18eb4ed40 2024-09-19T08:57:02.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.628+0000 7f98fbfff640 1 -- 172.21.15.164:0/1882412116 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990800eb70 con 0x55c18eb4ed40 2024-09-19T08:57:02.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.629+0000 7f991df1f740 1 -- 172.21.15.164:0/1882412116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c18eb4ed40 msgr2=0x55c18eb51130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:02.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.629+0000 7f991df1f740 1 --2- 172.21.15.164:0/1882412116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c18eb4ed40 0x55c18eb51130 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f9908005e10 tx=0x7f990800b620 comp rx=0 tx=0).stop 2024-09-19T08:57:02.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.629+0000 7f991df1f740 1 -- 172.21.15.164:0/1882412116 shutdown_connections 2024-09-19T08:57:02.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.629+0000 7f991df1f740 1 --2- 172.21.15.164:0/1882412116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c18eb4ed40 0x55c18eb51130 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.629+0000 7f991df1f740 1 --2- 172.21.15.164:0/1882412116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c18eb44480 0x55c18eb4e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.629+0000 7f991df1f740 1 --2- 172.21.15.164:0/1882412116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c18eb43b60 0x55c18eb43f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.629+0000 7f991df1f740 1 -- 172.21.15.164:0/1882412116 >> 172.21.15.164:0/1882412116 conn(0x55c18eb400f0 msgr2=0x55c18eb404d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:02.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.629+0000 7f991df1f740 1 -- 172.21.15.164:0/1882412116 shutdown_connections 2024-09-19T08:57:02.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.629+0000 7f991df1f740 1 -- 172.21.15.164:0/1882412116 wait complete. 2024-09-19T08:57:02.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.629+0000 7f991df1f740 1 -- start start 2024-09-19T08:57:02.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.629+0000 7f991df1f740 1 Processor -- start 2024-09-19T08:57:02.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.630+0000 7f991df1f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c18eb43b60 0x55c18eb4bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:02.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.630+0000 7f991df1f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c18eb44480 0x55c18eb4c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:02.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.630+0000 7f991df1f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c18eb4ed40 0x55c18eb4c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:02.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.630+0000 7f991df1f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c18eb51910 con 0x55c18eb44480 2024-09-19T08:57:02.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.630+0000 7f991df1f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c18ea63f10 con 0x55c18eb4ed40 2024-09-19T08:57:02.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.630+0000 7f991df1f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c18eb51a90 con 0x55c18eb43b60 2024-09-19T08:57:02.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.630+0000 7f991df1f740 10 client.0 _ll_register_callbacks cb 0x55c18ea2ca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:57:02.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.630+0000 7f99198e2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c18eb44480 0x55c18eb4c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:02.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.630+0000 7f99198e2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c18eb44480 0x55c18eb4c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44514/0 (socket says 172.21.15.164:44514) 2024-09-19T08:57:02.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.630+0000 7f99198e2640 1 -- 172.21.15.164:0/219807240 learned_addr learned my addr 172.21.15.164:0/219807240 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:02.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.630+0000 7f99190e1640 1 --2- 172.21.15.164:0/219807240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c18eb43b60 0x55c18eb4bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:02.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.630+0000 7f99190e1640 1 --2- 172.21.15.164:0/219807240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c18eb4ed40 0x55c18eb4c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:02.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.630+0000 7f99198e2640 1 -- 172.21.15.164:0/219807240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c18eb43b60 msgr2=0x55c18eb4bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:02.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.630+0000 7f99198e2640 1 --2- 172.21.15.164:0/219807240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c18eb43b60 0x55c18eb4bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.630+0000 7f99198e2640 1 -- 172.21.15.164:0/219807240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c18eb4ed40 msgr2=0x55c18eb4c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:02.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.630+0000 7f99198e2640 1 --2- 172.21.15.164:0/219807240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c18eb4ed40 0x55c18eb4c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.630+0000 7f99198e2640 1 -- 172.21.15.164:0/219807240 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c18e81a6a0 con 0x55c18eb44480 2024-09-19T08:57:02.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.630+0000 7f99190e1640 1 --2- 172.21.15.164:0/219807240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c18eb43b60 0x55c18eb4bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:57:02.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.630+0000 7f99198e2640 1 --2- 172.21.15.164:0/219807240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c18eb44480 0x55c18eb4c180 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f9904035000 tx=0x7f9904031bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ee0b351c8d1c81 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:02.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.630+0000 7f98f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:57:02.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.631+0000 7f98f97fa640 1 -- 172.21.15.164:0/219807240 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904049060 con 0x55c18eb44480 2024-09-19T08:57:02.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.631+0000 7f98f97fa640 1 -- 172.21.15.164:0/219807240 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f9904003b80 con 0x55c18eb44480 2024-09-19T08:57:02.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.631+0000 7f991df1f740 1 -- 172.21.15.164:0/219807240 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9908010c00 con 0x55c18eb44480 2024-09-19T08:57:02.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.631+0000 7f98f97fa640 1 -- 172.21.15.164:0/219807240 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9904003d80 con 0x55c18eb44480 2024-09-19T08:57:02.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.631+0000 7f991df1f740 10 client.17922 fetch_fsmap learned FSMap version 32 2024-09-19T08:57:02.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.631+0000 7f991df1f740 1 -- 172.21.15.164:0/219807240 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c18eb5d540 con 0x55c18eb44480 2024-09-19T08:57:02.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.631+0000 7f98f97fa640 1 -- 172.21.15.164:0/219807240 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9908010c00 con 0x55c18eb44480 2024-09-19T08:57:02.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.631+0000 7f98f97fa640 1 -- 172.21.15.164:0/219807240 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f9904043070 con 0x55c18eb44480 2024-09-19T08:57:02.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.631+0000 7f991df1f740 10 client.17922 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:57:02.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.631+0000 7f991df1f740 10 client.17922 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:57:02.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.631+0000 7f991df1f740 20 client.17922 populate_metadata read hostname 'smithi164' 2024-09-19T08:57:02.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.631+0000 7f991df1f740 1 --2- 172.21.15.164:0/219807240 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c18eb5eaa0 0x55c18eb60ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:02.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.631+0000 7f991df1f740 4 client.17922 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:57:02.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.631+0000 7f991df1f740 1 -- 172.21.15.164:0/219807240 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c18eab67f0 con 0x55c18eb5eaa0 2024-09-19T08:57:02.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.631+0000 7f99190e1640 1 --2- 172.21.15.164:0/219807240 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c18eb5eaa0 0x55c18eb60ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:02.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.632+0000 7f98f97fa640 10 client.17922 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:57:02.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.632+0000 7f99190e1640 1 --2- 172.21.15.164:0/219807240 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c18eb5eaa0 0x55c18eb60ee0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f22fd3d27b47057 server_cookie=304f92c52f1bf0cc in_seq=0 out_seq=0 2024-09-19T08:57:02.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.632+0000 7f98f97fa640 1 -- 172.21.15.164:0/219807240 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c18eab67f0 con 0x55c18eb5eaa0 2024-09-19T08:57:02.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.632+0000 7f98f97fa640 10 client.17922 handle_command_reply: tid=0 2024-09-19T08:57:02.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.633+0000 7f991df1f740 1 client.17922 shutdown 2024-09-19T08:57:02.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.633+0000 7f991df1f740 1 -- 172.21.15.164:0/219807240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c18eb44480 msgr2=0x55c18eb4c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:02.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.633+0000 7f991df1f740 1 --2- 172.21.15.164:0/219807240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c18eb44480 0x55c18eb4c180 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f9904035000 tx=0x7f9904031bc0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.633+0000 7f991df1f740 1 -- 172.21.15.164:0/219807240 shutdown_connections 2024-09-19T08:57:02.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.633+0000 7f991df1f740 1 --2- 172.21.15.164:0/219807240 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c18eb5eaa0 0x55c18eb60ee0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.633+0000 7f991df1f740 1 --2- 172.21.15.164:0/219807240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c18eb4ed40 0x55c18eb4c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.633+0000 7f991df1f740 1 --2- 172.21.15.164:0/219807240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c18eb44480 0x55c18eb4c180 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.633+0000 7f991df1f740 1 --2- 172.21.15.164:0/219807240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c18eb43b60 0x55c18eb4bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.633+0000 7f98f97fa640 0 client.17922 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:57:02.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.633+0000 7f991df1f740 1 -- 172.21.15.164:0/219807240 >> 172.21.15.164:0/219807240 conn(0x55c18eb400f0 msgr2=0x55c18eb45dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:02.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.633+0000 7f991df1f740 1 -- 172.21.15.164:0/219807240 shutdown_connections 2024-09-19T08:57:02.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.633+0000 7f991df1f740 1 -- 172.21.15.164:0/219807240 wait complete. 2024-09-19T08:57:02.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.633+0000 7f991df1f740 20 client.17922 trim_cache size 0 max 16384 2024-09-19T08:57:02.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.655+0000 7f991df1f740 1 Processor -- start 2024-09-19T08:57:02.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.656+0000 7f991df1f740 1 -- start start 2024-09-19T08:57:02.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.656+0000 7f991df1f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c18eb43b60 0x55c18eb4aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:02.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.656+0000 7f991df1f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c18eb44480 0x55c18eb667b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:02.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.656+0000 7f991df1f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c18eb4ed40 0x55c18eb66cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:02.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.656+0000 7f991df1f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f99141ad710 con 0x55c18eb4ed40 2024-09-19T08:57:02.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.656+0000 7f991df1f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f99141af250 con 0x55c18eb44480 2024-09-19T08:57:02.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.656+0000 7f991df1f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f99141adde0 con 0x55c18eb43b60 2024-09-19T08:57:02.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.656+0000 7f99190e1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c18eb4ed40 0x55c18eb66cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:02.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.656+0000 7f99190e1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c18eb4ed40 0x55c18eb66cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44528/0 (socket says 172.21.15.164:44528) 2024-09-19T08:57:02.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.656+0000 7f99190e1640 1 -- 172.21.15.164:0/3961995180 learned_addr learned my addr 172.21.15.164:0/3961995180 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:02.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.656+0000 7f99190e1640 1 --2- 172.21.15.164:0/3961995180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c18eb43b60 0x55c18eb4aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:02.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.656+0000 7f99198e2640 1 --2- 172.21.15.164:0/3961995180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c18eb44480 0x55c18eb667b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:02.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.656+0000 7f99190e1640 1 -- 172.21.15.164:0/3961995180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c18eb43b60 msgr2=0x55c18eb4aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:02.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.656+0000 7f99190e1640 1 --2- 172.21.15.164:0/3961995180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c18eb43b60 0x55c18eb4aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.656+0000 7f99190e1640 1 -- 172.21.15.164:0/3961995180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c18eb44480 msgr2=0x55c18eb667b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:02.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.656+0000 7f99190e1640 1 --2- 172.21.15.164:0/3961995180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c18eb44480 0x55c18eb667b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.656+0000 7f99190e1640 1 -- 172.21.15.164:0/3961995180 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c18eb61560 con 0x55c18eb4ed40 2024-09-19T08:57:02.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.656+0000 7f99198e2640 1 --2- 172.21.15.164:0/3961995180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c18eb44480 0x55c18eb667b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:57:02.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.657+0000 7f99190e1640 1 --2- 172.21.15.164:0/3961995180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c18eb4ed40 0x55c18eb66cf0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f990800beb0 tx=0x7f990800b620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9263722bb42da4c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:02.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.657+0000 7f98fb7fe640 1 -- 172.21.15.164:0/3961995180 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9908018040 con 0x55c18eb4ed40 2024-09-19T08:57:02.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.657+0000 7f98fb7fe640 1 -- 172.21.15.164:0/3961995180 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9908019040 con 0x55c18eb4ed40 2024-09-19T08:57:02.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.657+0000 7f98fb7fe640 1 -- 172.21.15.164:0/3961995180 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990800e880 con 0x55c18eb4ed40 2024-09-19T08:57:02.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.658+0000 7f991df1f740 1 -- 172.21.15.164:0/3961995180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c18eb4ed40 msgr2=0x55c18eb66cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:02.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.658+0000 7f991df1f740 1 --2- 172.21.15.164:0/3961995180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c18eb4ed40 0x55c18eb66cf0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f990800beb0 tx=0x7f990800b620 comp rx=0 tx=0).stop 2024-09-19T08:57:02.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.658+0000 7f991df1f740 1 -- 172.21.15.164:0/3961995180 shutdown_connections 2024-09-19T08:57:02.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.658+0000 7f991df1f740 1 --2- 172.21.15.164:0/3961995180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c18eb4ed40 0x55c18eb66cf0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.658+0000 7f991df1f740 1 --2- 172.21.15.164:0/3961995180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c18eb44480 0x55c18eb667b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.658+0000 7f991df1f740 1 --2- 172.21.15.164:0/3961995180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c18eb43b60 0x55c18eb4aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.658+0000 7f991df1f740 1 -- 172.21.15.164:0/3961995180 >> 172.21.15.164:0/3961995180 conn(0x55c18eb400f0 msgr2=0x55c18eb4a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:02.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.658+0000 7f991df1f740 1 -- 172.21.15.164:0/3961995180 shutdown_connections 2024-09-19T08:57:02.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.658+0000 7f991df1f740 1 -- 172.21.15.164:0/3961995180 wait complete. 2024-09-19T08:57:02.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.658+0000 7f991df1f740 1 -- start start 2024-09-19T08:57:02.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.659+0000 7f991df1f740 1 Processor -- start 2024-09-19T08:57:02.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:02 smithi173 ceph-mon[35306]: pgmap v349: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:57:02.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.659+0000 7f991df1f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c18eb43b60 0x55c18eb531f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:02.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.659+0000 7f991df1f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c18eb44480 0x55c18eb53730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:02.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.659+0000 7f991df1f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c18eb4ed40 0x55c18ebc8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:02.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.659+0000 7f991df1f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f99141aeb80 con 0x55c18eb43b60 2024-09-19T08:57:02.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.659+0000 7f991df1f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f99141ae4b0 con 0x55c18eb44480 2024-09-19T08:57:02.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.659+0000 7f991df1f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f99141ab710 con 0x55c18eb4ed40 2024-09-19T08:57:02.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.659+0000 7f991df1f740 10 client.0 _ll_register_callbacks cb 0x7f99040318c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:57:02.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.659+0000 7f99190e1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c18eb43b60 0x55c18eb531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:02.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.659+0000 7f99190e1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c18eb43b60 0x55c18eb531f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44544/0 (socket says 172.21.15.164:44544) 2024-09-19T08:57:02.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.659+0000 7f99190e1640 1 -- 172.21.15.164:0/2641042857 learned_addr learned my addr 172.21.15.164:0/2641042857 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:02.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.659+0000 7f99198e2640 1 --2- 172.21.15.164:0/2641042857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c18eb44480 0x55c18eb53730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:02.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.659+0000 7f99190e1640 1 --2- 172.21.15.164:0/2641042857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c18eb4ed40 0x55c18ebc8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:02.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.660+0000 7f99190e1640 1 -- 172.21.15.164:0/2641042857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c18eb4ed40 msgr2=0x55c18ebc8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:02.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.660+0000 7f99190e1640 1 --2- 172.21.15.164:0/2641042857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c18eb4ed40 0x55c18ebc8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.660+0000 7f99190e1640 1 -- 172.21.15.164:0/2641042857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c18eb44480 msgr2=0x55c18eb53730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:02.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.660+0000 7f99190e1640 1 --2- 172.21.15.164:0/2641042857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c18eb44480 0x55c18eb53730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.660+0000 7f99190e1640 1 -- 172.21.15.164:0/2641042857 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c18ebc8d70 con 0x55c18eb43b60 2024-09-19T08:57:02.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.660+0000 7f99198e2640 1 --2- 172.21.15.164:0/2641042857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c18eb44480 0x55c18eb53730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:57:02.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.660+0000 7f99190e1640 1 --2- 172.21.15.164:0/2641042857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c18eb43b60 0x55c18eb531f0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f9908012e90 tx=0x7f99080109c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e14d28e19c59244 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:02.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.660+0000 7f98f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:57:02.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.660+0000 7f98f97fa640 1 -- 172.21.15.164:0/2641042857 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9908018070 con 0x55c18eb43b60 2024-09-19T08:57:02.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.660+0000 7f98f97fa640 1 -- 172.21.15.164:0/2641042857 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7f9908010db0 con 0x55c18eb43b60 2024-09-19T08:57:02.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.660+0000 7f991df1f740 1 -- 172.21.15.164:0/2641042857 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9908019040 con 0x55c18eb43b60 2024-09-19T08:57:02.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.660+0000 7f98f97fa640 1 -- 172.21.15.164:0/2641042857 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9908049050 con 0x55c18eb43b60 2024-09-19T08:57:02.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.660+0000 7f991df1f740 10 client.17928 fetch_fsmap learned FSMap version 32 2024-09-19T08:57:02.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.660+0000 7f991df1f740 1 -- 172.21.15.164:0/2641042857 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c18ebca070 con 0x55c18eb43b60 2024-09-19T08:57:02.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.661+0000 7f98f97fa640 1 -- 172.21.15.164:0/2641042857 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9908037020 con 0x55c18eb43b60 2024-09-19T08:57:02.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.661+0000 7f98f97fa640 1 -- 172.21.15.164:0/2641042857 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f99080035f0 con 0x55c18eb43b60 2024-09-19T08:57:02.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.661+0000 7f991df1f740 10 client.17928 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:57:02.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.661+0000 7f991df1f740 10 client.17928 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:57:02.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.661+0000 7f991df1f740 20 client.17928 populate_metadata read hostname 'smithi164' 2024-09-19T08:57:02.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.661+0000 7f991df1f740 1 --2- 172.21.15.164:0/2641042857 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c18ebca490 0x55c18ebcc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:02.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.661+0000 7f991df1f740 4 client.17928 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T08:57:02.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.661+0000 7f991df1f740 1 -- 172.21.15.164:0/2641042857 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c18eab0d00 con 0x55c18ebca490 2024-09-19T08:57:02.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.661+0000 7f99198e2640 1 --2- 172.21.15.164:0/2641042857 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c18ebca490 0x55c18ebcc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:02.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.661+0000 7f98f97fa640 10 client.17928 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:57:02.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.662+0000 7f99198e2640 1 --2- 172.21.15.164:0/2641042857 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c18ebca490 0x55c18ebcc880 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe6d5bcd819a8904 server_cookie=ba1b72e5d7bdb493 in_seq=0 out_seq=0 2024-09-19T08:57:02.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.663+0000 7f98f97fa640 1 -- 172.21.15.164:0/2641042857 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c18eab0d00 con 0x55c18ebca490 2024-09-19T08:57:02.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.663+0000 7f98f97fa640 10 client.17928 handle_command_reply: tid=0 2024-09-19T08:57:02.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.663+0000 7f991df1f740 1 client.17928 shutdown 2024-09-19T08:57:02.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.663+0000 7f991df1f740 1 -- 172.21.15.164:0/2641042857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c18eb43b60 msgr2=0x55c18eb531f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:02.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.663+0000 7f991df1f740 1 --2- 172.21.15.164:0/2641042857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c18eb43b60 0x55c18eb531f0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f9908012e90 tx=0x7f99080109c0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.663+0000 7f991df1f740 1 -- 172.21.15.164:0/2641042857 shutdown_connections 2024-09-19T08:57:02.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.663+0000 7f991df1f740 1 --2- 172.21.15.164:0/2641042857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c18eb4ed40 0x55c18ebc8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.663+0000 7f991df1f740 1 --2- 172.21.15.164:0/2641042857 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55c18ebca490 0x55c18ebcc880 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.663+0000 7f991df1f740 1 --2- 172.21.15.164:0/2641042857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c18eb44480 0x55c18eb53730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.663+0000 7f991df1f740 1 --2- 172.21.15.164:0/2641042857 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c18eb43b60 0x55c18eb531f0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.663+0000 7f98f97fa640 0 client.17928 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:57:02.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.663+0000 7f991df1f740 1 -- 172.21.15.164:0/2641042857 >> 172.21.15.164:0/2641042857 conn(0x55c18eb400f0 msgr2=0x55c18eb3f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:02.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.663+0000 7f991df1f740 1 -- 172.21.15.164:0/2641042857 shutdown_connections 2024-09-19T08:57:02.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.663+0000 7f991df1f740 1 -- 172.21.15.164:0/2641042857 wait complete. 2024-09-19T08:57:02.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.663+0000 7f991df1f740 20 client.17928 trim_cache size 0 max 16384 2024-09-19T08:57:02.668 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:57:02.668 INFO:teuthology.orchestra.run.smithi164.stdout: "return_code": 0, 2024-09-19T08:57:02.669 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_tag": "b75f8ed1-f2a9-473a-aec5-fc9da3aad776", 2024-09-19T08:57:02.669 INFO:teuthology.orchestra.run.smithi164.stdout: "mode": "asynchronous" 2024-09-19T08:57:02.669 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:57:02.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.664+0000 7f991bb46640 1 -- 172.21.15.164:0/4099711897 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f98fc0a33b0 msgr2=0x7f98fc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:02.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.664+0000 7f991bb46640 1 --2- 172.21.15.164:0/4099711897 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f98fc0a33b0 0x7f98fc0a5870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f990c097b90 tx=0x7f990c062040 comp rx=0 tx=0).stop 2024-09-19T08:57:02.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.664+0000 7f991bb46640 1 -- 172.21.15.164:0/4099711897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99141082a0 msgr2=0x7f991419ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:02.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.664+0000 7f991bb46640 1 --2- 172.21.15.164:0/4099711897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99141082a0 0x7f991419ce50 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f990c009fd0 tx=0x7f990c00b8e0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.664+0000 7f991bb46640 1 -- 172.21.15.164:0/4099711897 shutdown_connections 2024-09-19T08:57:02.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.664+0000 7f991bb46640 1 --2- 172.21.15.164:0/4099711897 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f98fc0a33b0 0x7f98fc0a5870 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.664+0000 7f991bb46640 1 --2- 172.21.15.164:0/4099711897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99141082a0 0x7f991419ce50 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.664+0000 7f991bb46640 1 --2- 172.21.15.164:0/4099711897 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99141036e0 0x7f9914198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.664+0000 7f991bb46640 1 --2- 172.21.15.164:0/4099711897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9914102d30 0x7f9914198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:02.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.664+0000 7f991bb46640 1 -- 172.21.15.164:0/4099711897 >> 172.21.15.164:0/4099711897 conn(0x7f99140fc7d0 msgr2=0x7f9914105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:02.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.665+0000 7f991bb46640 1 -- 172.21.15.164:0/4099711897 shutdown_connections 2024-09-19T08:57:02.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:02.665+0000 7f991bb46640 1 -- 172.21.15.164:0/4099711897 wait complete. 2024-09-19T08:57:02.875 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:57:03.386 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:57:03.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:03 smithi170 ceph-mon[35072]: scrub queued for path: / 2024-09-19T08:57:03.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:03 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:57:03.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:03 smithi170 ceph-mon[35072]: scrub summary: active paths [/] 2024-09-19T08:57:03.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:03 smithi170 ceph-mon[35072]: scrub summary: active paths [/] 2024-09-19T08:57:03.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:03 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:57:03.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:03 smithi170 ceph-mon[35072]: scrub summary: active paths [/] 2024-09-19T08:57:03.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:03 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:57:03.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:03 smithi170 ceph-mon[35072]: scrub summary: idle+waiting paths [/] 2024-09-19T08:57:03.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:03 smithi173 ceph-mon[35306]: scrub queued for path: / 2024-09-19T08:57:03.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:03 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:57:03.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:03 smithi173 ceph-mon[35306]: scrub summary: active paths [/] 2024-09-19T08:57:03.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:03 smithi173 ceph-mon[35306]: scrub summary: active paths [/] 2024-09-19T08:57:03.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:03 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:57:03.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:03 smithi173 ceph-mon[35306]: scrub summary: active paths [/] 2024-09-19T08:57:03.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:03 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:57:03.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:03 smithi173 ceph-mon[35306]: scrub summary: idle+waiting paths [/] 2024-09-19T08:57:03.666 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:03 smithi164 ceph-mon[30928]: scrub queued for path: / 2024-09-19T08:57:03.666 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:03 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:57:03.666 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:03 smithi164 ceph-mon[30928]: scrub summary: active paths [/] 2024-09-19T08:57:03.667 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:03 smithi164 ceph-mon[30928]: scrub summary: active paths [/] 2024-09-19T08:57:03.667 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:03 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:57:03.667 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:03 smithi164 ceph-mon[30928]: scrub summary: active paths [/] 2024-09-19T08:57:03.667 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:03 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:57:03.667 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:03 smithi164 ceph-mon[30928]: scrub summary: idle+waiting paths [/] 2024-09-19T08:57:03.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.891+0000 7fd3fc04b640 1 -- 172.21.15.164:0/3673370611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3f4106f10 msgr2=0x7fd3f410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:03.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.891+0000 7fd3fc04b640 1 --2- 172.21.15.164:0/3673370611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3f4106f10 0x7fd3f410ea90 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec009ca0 tx=0x7fd3ec02a8c0 comp rx=0 tx=0).stop 2024-09-19T08:57:03.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.891+0000 7fd3fc04b640 1 -- 172.21.15.164:0/3673370611 shutdown_connections 2024-09-19T08:57:03.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.891+0000 7fd3fc04b640 1 --2- 172.21.15.164:0/3673370611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3f4106f10 0x7fd3f410ea90 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:03.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.891+0000 7fd3fc04b640 1 --2- 172.21.15.164:0/3673370611 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3f4102790 0x7fd3f41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:03.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.891+0000 7fd3fc04b640 1 --2- 172.21.15.164:0/3673370611 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3f4101de0 0x7fd3f41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:03.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.891+0000 7fd3fc04b640 1 -- 172.21.15.164:0/3673370611 >> 172.21.15.164:0/3673370611 conn(0x7fd3f40fd8c0 msgr2=0x7fd3f40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:03.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.891+0000 7fd3fc04b640 1 -- 172.21.15.164:0/3673370611 shutdown_connections 2024-09-19T08:57:03.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.891+0000 7fd3fc04b640 1 -- 172.21.15.164:0/3673370611 wait complete. 2024-09-19T08:57:03.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.892+0000 7fd3fc04b640 1 Processor -- start 2024-09-19T08:57:03.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.892+0000 7fd3fc04b640 1 -- start start 2024-09-19T08:57:03.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.892+0000 7fd3fc04b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3f4101de0 0x7fd3f40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:03.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.892+0000 7fd3fc04b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3f4102790 0x7fd3f4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:03.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.892+0000 7fd3fc04b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3f4106f10 0x7fd3f4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:03.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.892+0000 7fd3fc04b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f4111f70 con 0x7fd3f4101de0 2024-09-19T08:57:03.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.892+0000 7fd3fc04b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f4111df0 con 0x7fd3f4106f10 2024-09-19T08:57:03.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.892+0000 7fd3fc04b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41120f0 con 0x7fd3f4102790 2024-09-19T08:57:03.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.892+0000 7fd3f9de7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3f4101de0 0x7fd3f40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:03.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.892+0000 7fd3f9de7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3f4101de0 0x7fd3f40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57280/0 (socket says 172.21.15.164:57280) 2024-09-19T08:57:03.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.892+0000 7fd3f9de7640 1 -- 172.21.15.164:0/3864622528 learned_addr learned my addr 172.21.15.164:0/3864622528 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:03.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.892+0000 7fd3f95e6640 1 --2- 172.21.15.164:0/3864622528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3f4102790 0x7fd3f4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:03.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.892+0000 7fd3fa5e8640 1 --2- 172.21.15.164:0/3864622528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3f4106f10 0x7fd3f4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:03.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.892+0000 7fd3f9de7640 1 -- 172.21.15.164:0/3864622528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3f4102790 msgr2=0x7fd3f4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:03.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.892+0000 7fd3f9de7640 1 --2- 172.21.15.164:0/3864622528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3f4102790 0x7fd3f4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:03.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.892+0000 7fd3f9de7640 1 -- 172.21.15.164:0/3864622528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3f4106f10 msgr2=0x7fd3f4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:03.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.892+0000 7fd3f9de7640 1 --2- 172.21.15.164:0/3864622528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3f4106f10 0x7fd3f4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:03.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.892+0000 7fd3f9de7640 1 -- 172.21.15.164:0/3864622528 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f41a68d0 con 0x7fd3f4101de0 2024-09-19T08:57:03.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.892+0000 7fd3f95e6640 1 --2- 172.21.15.164:0/3864622528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3f4102790 0x7fd3f4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:57:03.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.892+0000 7fd3fa5e8640 1 --2- 172.21.15.164:0/3864622528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3f4106f10 0x7fd3f4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:57:03.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.893+0000 7fd3f9de7640 1 --2- 172.21.15.164:0/3864622528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3f4101de0 0x7fd3f40744f0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800d950 tx=0x7fd3e800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:03.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.893+0000 7fd3e6ffd640 1 -- 172.21.15.164:0/3864622528 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8017030 con 0x7fd3f4101de0 2024-09-19T08:57:03.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.893+0000 7fd3fc04b640 1 -- 172.21.15.164:0/3864622528 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3f41a6bc0 con 0x7fd3f4101de0 2024-09-19T08:57:03.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.893+0000 7fd3e6ffd640 1 -- 172.21.15.164:0/3864622528 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3e8004020 con 0x7fd3f4101de0 2024-09-19T08:57:03.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.893+0000 7fd3e6ffd640 1 -- 172.21.15.164:0/3864622528 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e80047a0 con 0x7fd3f4101de0 2024-09-19T08:57:03.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.893+0000 7fd3fc04b640 1 -- 172.21.15.164:0/3864622528 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3f41a7100 con 0x7fd3f4101de0 2024-09-19T08:57:03.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.895+0000 7fd3e6ffd640 1 -- 172.21.15.164:0/3864622528 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fd3e80041d0 con 0x7fd3f4101de0 2024-09-19T08:57:03.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.895+0000 7fd3fe424740 1 Processor -- start 2024-09-19T08:57:03.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.895+0000 7fd3fe424740 1 -- start start 2024-09-19T08:57:03.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.896+0000 7fd3fe424740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eadd462060 0x55eadd462440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:03.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.896+0000 7fd3fe424740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eadd462980 0x55eadd46cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:03.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.896+0000 7fd3fe424740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eadd46d240 0x55eadd46f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:03.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.896+0000 7fd3fe424740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55eadd3c7620 con 0x55eadd46d240 2024-09-19T08:57:03.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.896+0000 7fd3fe424740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55eadd36ee50 con 0x55eadd462980 2024-09-19T08:57:03.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.896+0000 7fd3fe424740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55eadd3c35e0 con 0x55eadd462060 2024-09-19T08:57:03.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.896+0000 7fd3f95e6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eadd46d240 0x55eadd46f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:03.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.896+0000 7fd3f95e6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eadd46d240 0x55eadd46f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57284/0 (socket says 172.21.15.164:57284) 2024-09-19T08:57:03.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.896+0000 7fd3f95e6640 1 -- 172.21.15.164:0/494209138 learned_addr learned my addr 172.21.15.164:0/494209138 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:03.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.896+0000 7fd3fa5e8640 1 --2- 172.21.15.164:0/494209138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eadd462060 0x55eadd462440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:03.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.896+0000 7fd3f9de7640 1 --2- 172.21.15.164:0/494209138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eadd462980 0x55eadd46cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:03.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.896+0000 7fd3f95e6640 1 -- 172.21.15.164:0/494209138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eadd462060 msgr2=0x55eadd462440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:03.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.896+0000 7fd3f95e6640 1 --2- 172.21.15.164:0/494209138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eadd462060 0x55eadd462440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:03.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.896+0000 7fd3f95e6640 1 -- 172.21.15.164:0/494209138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eadd462980 msgr2=0x55eadd46cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:03.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.896+0000 7fd3f95e6640 1 --2- 172.21.15.164:0/494209138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eadd462980 0x55eadd46cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:03.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.896+0000 7fd3f95e6640 1 -- 172.21.15.164:0/494209138 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eadd35fd40 con 0x55eadd46d240 2024-09-19T08:57:03.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.896+0000 7fd3e6ffd640 1 --2- 172.21.15.164:0/3864622528 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd3e00a33b0 0x7fd3e00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:03.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.897+0000 7fd3fa5e8640 1 --2- 172.21.15.164:0/3864622528 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd3e00a33b0 0x7fd3e00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:03.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.897+0000 7fd3f95e6640 1 --2- 172.21.15.164:0/494209138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eadd46d240 0x55eadd46f630 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fd3f000bcb0 tx=0x7fd3f000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9915e75af5bad9e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:03.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.897+0000 7fd3e6ffd640 1 -- 172.21.15.164:0/3864622528 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7fd3e80cf7a0 con 0x7fd3f4101de0 2024-09-19T08:57:03.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.897+0000 7fd3dffff640 1 -- 172.21.15.164:0/494209138 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0015070 con 0x55eadd46d240 2024-09-19T08:57:03.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.897+0000 7fd3dffff640 1 -- 172.21.15.164:0/494209138 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3f0011920 con 0x55eadd46d240 2024-09-19T08:57:03.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.897+0000 7fd3dffff640 1 -- 172.21.15.164:0/494209138 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f00109a0 con 0x55eadd46d240 2024-09-19T08:57:03.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.898+0000 7fd3fe424740 1 -- 172.21.15.164:0/494209138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eadd46d240 msgr2=0x55eadd46f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:03.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.898+0000 7fd3fa5e8640 1 --2- 172.21.15.164:0/3864622528 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd3e00a33b0 0x7fd3e00a5870 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec004af0 tx=0x7fd3ec005d80 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:03.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.898+0000 7fd3fe424740 1 --2- 172.21.15.164:0/494209138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eadd46d240 0x55eadd46f630 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fd3f000bcb0 tx=0x7fd3f000f730 comp rx=0 tx=0).stop 2024-09-19T08:57:03.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.898+0000 7fd3fe424740 1 -- 172.21.15.164:0/494209138 shutdown_connections 2024-09-19T08:57:03.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.898+0000 7fd3fe424740 1 --2- 172.21.15.164:0/494209138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eadd46d240 0x55eadd46f630 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:03.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.898+0000 7fd3fe424740 1 --2- 172.21.15.164:0/494209138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eadd462980 0x55eadd46cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:03.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.898+0000 7fd3fe424740 1 --2- 172.21.15.164:0/494209138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eadd462060 0x55eadd462440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:03.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.898+0000 7fd3fe424740 1 -- 172.21.15.164:0/494209138 >> 172.21.15.164:0/494209138 conn(0x55eadd45e5f0 msgr2=0x55eadd45e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:03.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.898+0000 7fd3fe424740 1 -- 172.21.15.164:0/494209138 shutdown_connections 2024-09-19T08:57:03.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.898+0000 7fd3fe424740 1 -- 172.21.15.164:0/494209138 wait complete. 2024-09-19T08:57:03.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.898+0000 7fd3fe424740 1 -- start start 2024-09-19T08:57:03.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.898+0000 7fd3fe424740 1 Processor -- start 2024-09-19T08:57:03.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.899+0000 7fd3fe424740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eadd462060 0x55eadd46a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:03.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.899+0000 7fd3fe424740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eadd462980 0x55eadd46a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:03.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.899+0000 7fd3fe424740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eadd46d240 0x55eadd46abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:03.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.899+0000 7fd3fe424740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55eadd46fe10 con 0x55eadd462980 2024-09-19T08:57:03.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.899+0000 7fd3fe424740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55eadd34ff10 con 0x55eadd46d240 2024-09-19T08:57:03.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.899+0000 7fd3fe424740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55eadd46ff90 con 0x55eadd462060 2024-09-19T08:57:03.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.899+0000 7fd3f95e6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eadd462980 0x55eadd46a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:03.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.899+0000 7fd3fe424740 10 client.0 _ll_register_callbacks cb 0x55eadd373db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:57:03.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.899+0000 7fd3f95e6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eadd462980 0x55eadd46a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57292/0 (socket says 172.21.15.164:57292) 2024-09-19T08:57:03.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.899+0000 7fd3f95e6640 1 -- 172.21.15.164:0/273483016 learned_addr learned my addr 172.21.15.164:0/273483016 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:03.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.899+0000 7fd3f9de7640 1 --2- 172.21.15.164:0/273483016 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eadd462060 0x55eadd46a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:03.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.899+0000 7fd3fa5e8640 1 --2- 172.21.15.164:0/273483016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eadd46d240 0x55eadd46abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:03.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.899+0000 7fd3f95e6640 1 -- 172.21.15.164:0/273483016 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eadd462060 msgr2=0x55eadd46a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:03.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.899+0000 7fd3f95e6640 1 --2- 172.21.15.164:0/273483016 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eadd462060 0x55eadd46a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:03.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.900+0000 7fd3f95e6640 1 -- 172.21.15.164:0/273483016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eadd46d240 msgr2=0x55eadd46abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:03.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.900+0000 7fd3f95e6640 1 --2- 172.21.15.164:0/273483016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eadd46d240 0x55eadd46abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:03.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.900+0000 7fd3f95e6640 1 -- 172.21.15.164:0/273483016 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eadd0d6580 con 0x55eadd462980 2024-09-19T08:57:03.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.900+0000 7fd3f95e6640 1 --2- 172.21.15.164:0/273483016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eadd462980 0x55eadd46a670 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0005880 tx=0x7fd3f0010440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475306d81a81d232 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:03.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.900+0000 7fd3dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:57:03.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.900+0000 7fd3dd7fa640 1 -- 172.21.15.164:0/273483016 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0015070 con 0x55eadd462980 2024-09-19T08:57:03.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.900+0000 7fd3fe424740 1 -- 172.21.15.164:0/273483016 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3f0011920 con 0x55eadd462980 2024-09-19T08:57:03.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.900+0000 7fd3dd7fa640 1 -- 172.21.15.164:0/273483016 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7fd3f0011650 con 0x55eadd462980 2024-09-19T08:57:03.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.900+0000 7fd3dd7fa640 1 -- 172.21.15.164:0/273483016 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd3f000b8c0 con 0x55eadd462980 2024-09-19T08:57:03.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.900+0000 7fd3fe424740 10 client.17940 fetch_fsmap learned FSMap version 32 2024-09-19T08:57:03.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.900+0000 7fd3fe424740 1 -- 172.21.15.164:0/273483016 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55eadd1a0960 con 0x55eadd462980 2024-09-19T08:57:03.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.900+0000 7fd3dd7fa640 1 -- 172.21.15.164:0/273483016 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3f0011920 con 0x55eadd462980 2024-09-19T08:57:03.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.901+0000 7fd3dd7fa640 1 -- 172.21.15.164:0/273483016 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fd3f0010800 con 0x55eadd462980 2024-09-19T08:57:03.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.901+0000 7fd3fe424740 10 client.17940 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:57:03.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.901+0000 7fd3fe424740 10 client.17940 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:57:03.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.901+0000 7fd3fe424740 20 client.17940 populate_metadata read hostname 'smithi164' 2024-09-19T08:57:03.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.901+0000 7fd3fe424740 1 --2- 172.21.15.164:0/273483016 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55eadd47cd60 0x55eadd47f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:03.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.901+0000 7fd3fe424740 4 client.17940 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:57:03.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.901+0000 7fd3fe424740 1 -- 172.21.15.164:0/273483016 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eadd3d4b90 con 0x55eadd47cd60 2024-09-19T08:57:03.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.901+0000 7fd3f9de7640 1 --2- 172.21.15.164:0/273483016 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55eadd47cd60 0x55eadd47f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:03.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.901+0000 7fd3f9de7640 1 --2- 172.21.15.164:0/273483016 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55eadd47cd60 0x55eadd47f150 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f02e39eb151f37d server_cookie=eb067f2c42f809ee in_seq=0 out_seq=0 2024-09-19T08:57:03.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.901+0000 7fd3dd7fa640 10 client.17940 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:57:03.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.902+0000 7fd3dd7fa640 1 -- 172.21.15.164:0/273483016 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eadd3d4b90 con 0x55eadd47cd60 2024-09-19T08:57:03.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.902+0000 7fd3dd7fa640 10 client.17940 handle_command_reply: tid=0 2024-09-19T08:57:03.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.902+0000 7fd3fe424740 1 client.17940 shutdown 2024-09-19T08:57:03.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.902+0000 7fd3fe424740 1 -- 172.21.15.164:0/273483016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eadd462980 msgr2=0x55eadd46a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:03.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.902+0000 7fd3fe424740 1 --2- 172.21.15.164:0/273483016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eadd462980 0x55eadd46a670 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0005880 tx=0x7fd3f0010440 comp rx=0 tx=0).stop 2024-09-19T08:57:03.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.902+0000 7fd3fe424740 1 -- 172.21.15.164:0/273483016 shutdown_connections 2024-09-19T08:57:03.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.902+0000 7fd3fe424740 1 --2- 172.21.15.164:0/273483016 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55eadd47cd60 0x55eadd47f150 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:03.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.902+0000 7fd3fe424740 1 --2- 172.21.15.164:0/273483016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eadd46d240 0x55eadd46abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:03.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.902+0000 7fd3dd7fa640 0 client.17940 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:57:03.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.902+0000 7fd3fe424740 1 --2- 172.21.15.164:0/273483016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eadd462980 0x55eadd46a670 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:03.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.902+0000 7fd3fe424740 1 --2- 172.21.15.164:0/273483016 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eadd462060 0x55eadd46a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:03.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.902+0000 7fd3fe424740 1 -- 172.21.15.164:0/273483016 >> 172.21.15.164:0/273483016 conn(0x55eadd45e5f0 msgr2=0x55eadd4642d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:03.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.903+0000 7fd3fe424740 1 -- 172.21.15.164:0/273483016 shutdown_connections 2024-09-19T08:57:03.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.903+0000 7fd3fe424740 1 -- 172.21.15.164:0/273483016 wait complete. 2024-09-19T08:57:03.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.903+0000 7fd3fe424740 20 client.17940 trim_cache size 0 max 16384 2024-09-19T08:57:03.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.920+0000 7fd3fe424740 1 Processor -- start 2024-09-19T08:57:03.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.920+0000 7fd3fe424740 1 -- start start 2024-09-19T08:57:03.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.920+0000 7fd3fe424740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eadd462060 0x55eadd4641c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:03.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.920+0000 7fd3fe424740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eadd462980 0x55eadd484960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:03.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.920+0000 7fd3fe424740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eadd46d240 0x55eadd484ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:03.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.920+0000 7fd3fe424740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41af9f0 con 0x55eadd462980 2024-09-19T08:57:03.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.920+0000 7fd3fe424740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41b1530 con 0x55eadd46d240 2024-09-19T08:57:03.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.920+0000 7fd3fe424740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41b00c0 con 0x55eadd462060 2024-09-19T08:57:03.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.921+0000 7fd3f95e6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eadd462980 0x55eadd484960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:03.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.921+0000 7fd3f95e6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eadd462980 0x55eadd484960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57294/0 (socket says 172.21.15.164:57294) 2024-09-19T08:57:03.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.921+0000 7fd3f95e6640 1 -- 172.21.15.164:0/4049976986 learned_addr learned my addr 172.21.15.164:0/4049976986 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:03.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.921+0000 7fd3fa5e8640 1 --2- 172.21.15.164:0/4049976986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eadd46d240 0x55eadd484ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:03.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.921+0000 7fd3f9de7640 1 --2- 172.21.15.164:0/4049976986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eadd462060 0x55eadd4641c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:03.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.921+0000 7fd3f95e6640 1 -- 172.21.15.164:0/4049976986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eadd462060 msgr2=0x55eadd4641c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:03.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.921+0000 7fd3f95e6640 1 --2- 172.21.15.164:0/4049976986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eadd462060 0x55eadd4641c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:03.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.921+0000 7fd3f95e6640 1 -- 172.21.15.164:0/4049976986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eadd46d240 msgr2=0x55eadd484ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:03.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.921+0000 7fd3f95e6640 1 --2- 172.21.15.164:0/4049976986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eadd46d240 0x55eadd484ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:03.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.921+0000 7fd3f95e6640 1 -- 172.21.15.164:0/4049976986 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eadd46caa0 con 0x55eadd462980 2024-09-19T08:57:03.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.921+0000 7fd3fa5e8640 1 --2- 172.21.15.164:0/4049976986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eadd46d240 0x55eadd484ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:57:03.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.921+0000 7fd3f95e6640 1 --2- 172.21.15.164:0/4049976986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eadd462980 0x55eadd484960 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0010410 tx=0x7fd3f000fd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6cf995853b7bcd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:03.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.922+0000 7fd3df7fe640 1 -- 172.21.15.164:0/4049976986 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0031030 con 0x55eadd462980 2024-09-19T08:57:03.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.922+0000 7fd3df7fe640 1 -- 172.21.15.164:0/4049976986 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3f0010e90 con 0x55eadd462980 2024-09-19T08:57:03.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.922+0000 7fd3df7fe640 1 -- 172.21.15.164:0/4049976986 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0011b40 con 0x55eadd462980 2024-09-19T08:57:03.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.922+0000 7fd3fe424740 1 -- 172.21.15.164:0/4049976986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eadd462980 msgr2=0x55eadd484960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:03.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.922+0000 7fd3fe424740 1 --2- 172.21.15.164:0/4049976986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eadd462980 0x55eadd484960 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0010410 tx=0x7fd3f000fd60 comp rx=0 tx=0).stop 2024-09-19T08:57:03.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.922+0000 7fd3fe424740 1 -- 172.21.15.164:0/4049976986 shutdown_connections 2024-09-19T08:57:03.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.922+0000 7fd3fe424740 1 --2- 172.21.15.164:0/4049976986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eadd46d240 0x55eadd484ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:03.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.922+0000 7fd3fe424740 1 --2- 172.21.15.164:0/4049976986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eadd462980 0x55eadd484960 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:03.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.922+0000 7fd3fe424740 1 --2- 172.21.15.164:0/4049976986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eadd462060 0x55eadd4641c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:03.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.922+0000 7fd3fe424740 1 -- 172.21.15.164:0/4049976986 >> 172.21.15.164:0/4049976986 conn(0x55eadd45e5f0 msgr2=0x55eadd463ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:03.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.923+0000 7fd3fe424740 1 -- 172.21.15.164:0/4049976986 shutdown_connections 2024-09-19T08:57:03.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.923+0000 7fd3fe424740 1 -- 172.21.15.164:0/4049976986 wait complete. 2024-09-19T08:57:03.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.923+0000 7fd3fe424740 1 -- start start 2024-09-19T08:57:03.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.923+0000 7fd3fe424740 1 Processor -- start 2024-09-19T08:57:03.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.923+0000 7fd3fe424740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eadd462060 0x55eadd4751e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:03.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.923+0000 7fd3fe424740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eadd46d240 0x55eadd475720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:03.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.923+0000 7fd3fe424740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eadd47cd60 0x55eadd4721b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:03.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.923+0000 7fd3fe424740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41b0e60 con 0x55eadd46d240 2024-09-19T08:57:03.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.924+0000 7fd3fe424740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41b0790 con 0x55eadd462060 2024-09-19T08:57:03.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.924+0000 7fd3fe424740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41ad9f0 con 0x55eadd47cd60 2024-09-19T08:57:03.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.924+0000 7fd3fe424740 10 client.0 _ll_register_callbacks cb 0x7fd3f000bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:57:03.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.924+0000 7fd3f95e6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eadd46d240 0x55eadd475720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:03.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.924+0000 7fd3f95e6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eadd46d240 0x55eadd475720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57310/0 (socket says 172.21.15.164:57310) 2024-09-19T08:57:03.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.924+0000 7fd3f95e6640 1 -- 172.21.15.164:0/2886555171 learned_addr learned my addr 172.21.15.164:0/2886555171 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:03.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.924+0000 7fd3fa5e8640 1 --2- 172.21.15.164:0/2886555171 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eadd47cd60 0x55eadd4721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:03.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.924+0000 7fd3f9de7640 1 --2- 172.21.15.164:0/2886555171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eadd462060 0x55eadd4751e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:03.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.924+0000 7fd3f95e6640 1 -- 172.21.15.164:0/2886555171 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eadd47cd60 msgr2=0x55eadd4721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:03.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.924+0000 7fd3f95e6640 1 --2- 172.21.15.164:0/2886555171 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eadd47cd60 0x55eadd4721b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:03.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.924+0000 7fd3f95e6640 1 -- 172.21.15.164:0/2886555171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eadd462060 msgr2=0x55eadd4751e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:03.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.924+0000 7fd3f95e6640 1 --2- 172.21.15.164:0/2886555171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eadd462060 0x55eadd4751e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:03.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.924+0000 7fd3f95e6640 1 -- 172.21.15.164:0/2886555171 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eadd4726f0 con 0x55eadd46d240 2024-09-19T08:57:03.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.924+0000 7fd3f95e6640 1 --2- 172.21.15.164:0/2886555171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eadd46d240 0x55eadd475720 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0010c20 tx=0x7fd3f000b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed98bd21d6f3196 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:03.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.924+0000 7fd3dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:57:03.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.924+0000 7fd3dd7fa640 1 -- 172.21.15.164:0/2886555171 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0031040 con 0x55eadd46d240 2024-09-19T08:57:03.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.924+0000 7fd3dd7fa640 1 -- 172.21.15.164:0/2886555171 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7fd3f000b760 con 0x55eadd46d240 2024-09-19T08:57:03.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.924+0000 7fd3fe424740 1 -- 172.21.15.164:0/2886555171 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3f0010e90 con 0x55eadd46d240 2024-09-19T08:57:03.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.925+0000 7fd3dd7fa640 1 -- 172.21.15.164:0/2886555171 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd3f0010690 con 0x55eadd46d240 2024-09-19T08:57:03.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.925+0000 7fd3fe424740 10 client.17946 fetch_fsmap learned FSMap version 32 2024-09-19T08:57:03.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.925+0000 7fd3fe424740 1 -- 172.21.15.164:0/2886555171 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55eadd4f0820 con 0x55eadd46d240 2024-09-19T08:57:03.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.925+0000 7fd3dd7fa640 1 -- 172.21.15.164:0/2886555171 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3f0010e90 con 0x55eadd46d240 2024-09-19T08:57:03.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.925+0000 7fd3dd7fa640 1 -- 172.21.15.164:0/2886555171 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fd3f002d440 con 0x55eadd46d240 2024-09-19T08:57:03.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.925+0000 7fd3fe424740 10 client.17946 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:57:03.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.925+0000 7fd3fe424740 10 client.17946 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:57:03.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.925+0000 7fd3fe424740 20 client.17946 populate_metadata read hostname 'smithi164' 2024-09-19T08:57:03.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.925+0000 7fd3fe424740 1 --2- 172.21.15.164:0/2886555171 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55eadd4f0e00 0x55eadd4f3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:03.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.925+0000 7fd3fe424740 4 client.17946 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "e5bd5b7c", "release": true, "await": true}] 2024-09-19T08:57:03.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.925+0000 7fd3fe424740 1 -- 172.21.15.164:0/2886555171 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "quiesce db", "set_id": "e5bd5b7c", "release": true, "await": true}) -- 0x55eadd3cf0a0 con 0x55eadd4f0e00 2024-09-19T08:57:03.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.925+0000 7fd3f9de7640 1 --2- 172.21.15.164:0/2886555171 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55eadd4f0e00 0x55eadd4f3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:03.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.926+0000 7fd3f9de7640 1 --2- 172.21.15.164:0/2886555171 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55eadd4f0e00 0x55eadd4f3220 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb1cc30dce266d0 server_cookie=3efb304cbe5527d in_seq=0 out_seq=0 2024-09-19T08:57:03.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.926+0000 7fd3dd7fa640 10 client.17946 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:57:03.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.929+0000 7fd3dd7fa640 1 -- 172.21.15.164:0/2886555171 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55eadd3cf0a0 con 0x55eadd4f0e00 2024-09-19T08:57:03.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.929+0000 7fd3dd7fa640 10 client.17946 handle_command_reply: tid=0 2024-09-19T08:57:03.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.929+0000 7fd3fe424740 1 client.17946 shutdown 2024-09-19T08:57:03.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.929+0000 7fd3fe424740 1 -- 172.21.15.164:0/2886555171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eadd46d240 msgr2=0x55eadd475720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:03.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.929+0000 7fd3fe424740 1 --2- 172.21.15.164:0/2886555171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eadd46d240 0x55eadd475720 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0010c20 tx=0x7fd3f000b420 comp rx=0 tx=0).stop 2024-09-19T08:57:03.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.929+0000 7fd3fe424740 1 -- 172.21.15.164:0/2886555171 shutdown_connections 2024-09-19T08:57:03.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.929+0000 7fd3fe424740 1 --2- 172.21.15.164:0/2886555171 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eadd47cd60 0x55eadd4721b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:03.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.929+0000 7fd3fe424740 1 --2- 172.21.15.164:0/2886555171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eadd46d240 0x55eadd475720 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:03.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.929+0000 7fd3fe424740 1 --2- 172.21.15.164:0/2886555171 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55eadd4f0e00 0x55eadd4f3220 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:03.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.930+0000 7fd3fe424740 1 --2- 172.21.15.164:0/2886555171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eadd462060 0x55eadd4751e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:03.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.930+0000 7fd3fe424740 1 -- 172.21.15.164:0/2886555171 >> 172.21.15.164:0/2886555171 conn(0x55eadd45e5f0 msgr2=0x55eadd4642d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:03.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.930+0000 7fd3dd7fa640 0 client.17946 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:57:03.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.930+0000 7fd3fe424740 1 -- 172.21.15.164:0/2886555171 shutdown_connections 2024-09-19T08:57:03.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.930+0000 7fd3fe424740 1 -- 172.21.15.164:0/2886555171 wait complete. 2024-09-19T08:57:03.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.930+0000 7fd3fe424740 20 client.17946 trim_cache size 0 max 16384 2024-09-19T08:57:03.936 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:57:03.936 INFO:teuthology.orchestra.run.smithi164.stdout: "epoch": 32, 2024-09-19T08:57:03.936 INFO:teuthology.orchestra.run.smithi164.stdout: "leader": 24479, 2024-09-19T08:57:03.936 INFO:teuthology.orchestra.run.smithi164.stdout: "set_version": 40, 2024-09-19T08:57:03.936 INFO:teuthology.orchestra.run.smithi164.stdout: "sets": { 2024-09-19T08:57:03.936 INFO:teuthology.orchestra.run.smithi164.stdout: "e5bd5b7c": { 2024-09-19T08:57:03.936 INFO:teuthology.orchestra.run.smithi164.stdout: "version": 40, 2024-09-19T08:57:03.936 INFO:teuthology.orchestra.run.smithi164.stdout: "age_ref": 0.0, 2024-09-19T08:57:03.937 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:57:03.937 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "RELEASED", 2024-09-19T08:57:03.937 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:57:03.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:03.937 INFO:teuthology.orchestra.run.smithi164.stdout: "timeout": 90.0, 2024-09-19T08:57:03.937 INFO:teuthology.orchestra.run.smithi164.stdout: "expiration": 150.0, 2024-09-19T08:57:03.937 INFO:teuthology.orchestra.run.smithi164.stdout: "members": { 2024-09-19T08:57:03.937 INFO:teuthology.orchestra.run.smithi164.stdout: "file:/": { 2024-09-19T08:57:03.937 INFO:teuthology.orchestra.run.smithi164.stdout: "excluded": false, 2024-09-19T08:57:03.937 INFO:teuthology.orchestra.run.smithi164.stdout: "state": { 2024-09-19T08:57:03.937 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "RELEASED", 2024-09-19T08:57:03.937 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0 2024-09-19T08:57:03.937 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:03.937 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:03.938 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:03.938 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:03.938 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:03.938 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:57:03.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.930+0000 7fd3fc04b640 1 -- 172.21.15.164:0/3864622528 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd3e00a33b0 msgr2=0x7fd3e00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:03.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.930+0000 7fd3fc04b640 1 --2- 172.21.15.164:0/3864622528 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd3e00a33b0 0x7fd3e00a5870 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec004af0 tx=0x7fd3ec005d80 comp rx=0 tx=0).stop 2024-09-19T08:57:03.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.931+0000 7fd3fc04b640 1 -- 172.21.15.164:0/3864622528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3f4101de0 msgr2=0x7fd3f40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:03.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.931+0000 7fd3fc04b640 1 --2- 172.21.15.164:0/3864622528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3f4101de0 0x7fd3f40744f0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800d950 tx=0x7fd3e800de10 comp rx=0 tx=0).stop 2024-09-19T08:57:03.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.931+0000 7fd3fc04b640 1 -- 172.21.15.164:0/3864622528 shutdown_connections 2024-09-19T08:57:03.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.931+0000 7fd3fc04b640 1 --2- 172.21.15.164:0/3864622528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3f4106f10 0x7fd3f4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:03.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.931+0000 7fd3fc04b640 1 --2- 172.21.15.164:0/3864622528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3f4102790 0x7fd3f4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:03.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.931+0000 7fd3fc04b640 1 --2- 172.21.15.164:0/3864622528 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd3e00a33b0 0x7fd3e00a5870 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:03.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.931+0000 7fd3fc04b640 1 --2- 172.21.15.164:0/3864622528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3f4101de0 0x7fd3f40744f0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:03.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.931+0000 7fd3fc04b640 1 -- 172.21.15.164:0/3864622528 >> 172.21.15.164:0/3864622528 conn(0x7fd3f40fd8c0 msgr2=0x7fd3f40fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:03.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.931+0000 7fd3fc04b640 1 -- 172.21.15.164:0/3864622528 shutdown_connections 2024-09-19T08:57:03.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:03.931+0000 7fd3fc04b640 1 -- 172.21.15.164:0/3864622528 wait complete. 2024-09-19T08:57:03.940 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:57:04.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:04 smithi170 ceph-mon[35072]: pgmap v350: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-09-19T08:57:04.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:04 smithi173 ceph-mon[35306]: pgmap v350: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-09-19T08:57:04.670 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:04 smithi164 ceph-mon[30928]: pgmap v350: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-09-19T08:57:04.829 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'e5bd5b7c', total seconds elapsed: 35.7 2024-09-19T08:57:05.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.052+0000 7fe4fe25c640 1 -- 172.21.15.164:0/4200371425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4f8106f20 msgr2=0x7fe4f810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:05.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.052+0000 7fe4fe25c640 1 --2- 172.21.15.164:0/4200371425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4f8106f20 0x7fe4f810eaa0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0009ca0 tx=0x7fe4f002a8c0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.053+0000 7fe4fe25c640 1 -- 172.21.15.164:0/4200371425 shutdown_connections 2024-09-19T08:57:05.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.053+0000 7fe4fe25c640 1 --2- 172.21.15.164:0/4200371425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4f8106f20 0x7fe4f810eaa0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.053+0000 7fe4fe25c640 1 --2- 172.21.15.164:0/4200371425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4f81027a0 0x7fe4f81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.053+0000 7fe4fe25c640 1 --2- 172.21.15.164:0/4200371425 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4f8101df0 0x7fe4f81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.053+0000 7fe4fe25c640 1 -- 172.21.15.164:0/4200371425 >> 172.21.15.164:0/4200371425 conn(0x7fe4f80fd8b0 msgr2=0x7fe4f80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:05.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.053+0000 7fe4fe25c640 1 -- 172.21.15.164:0/4200371425 shutdown_connections 2024-09-19T08:57:05.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.053+0000 7fe4fe25c640 1 -- 172.21.15.164:0/4200371425 wait complete. 2024-09-19T08:57:05.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.053+0000 7fe4fe25c640 1 Processor -- start 2024-09-19T08:57:05.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.053+0000 7fe4fe25c640 1 -- start start 2024-09-19T08:57:05.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.054+0000 7fe4fe25c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4f8101df0 0x7fe4f8198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:05.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.054+0000 7fe4fe25c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4f81027a0 0x7fe4f8198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:05.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.054+0000 7fe4fe25c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4f8106f20 0x7fe4f819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:05.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.054+0000 7fe4fe25c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f8111f80 con 0x7fe4f8101df0 2024-09-19T08:57:05.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.054+0000 7fe4fe25c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f8111e00 con 0x7fe4f81027a0 2024-09-19T08:57:05.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.054+0000 7fe4fe25c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f8112100 con 0x7fe4f8106f20 2024-09-19T08:57:05.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.054+0000 7fe4f77fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4f8101df0 0x7fe4f8198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:05.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.054+0000 7fe4f77fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4f8101df0 0x7fe4f8198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57342/0 (socket says 172.21.15.164:57342) 2024-09-19T08:57:05.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.054+0000 7fe4f77fe640 1 -- 172.21.15.164:0/152895669 learned_addr learned my addr 172.21.15.164:0/152895669 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:05.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.054+0000 7fe4eeffd640 1 --2- 172.21.15.164:0/152895669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4f81027a0 0x7fe4f8198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:05.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.054+0000 7fe4f7fff640 1 --2- 172.21.15.164:0/152895669 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4f8106f20 0x7fe4f819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:05.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.054+0000 7fe4f77fe640 1 -- 172.21.15.164:0/152895669 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4f8106f20 msgr2=0x7fe4f819cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:05.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.054+0000 7fe4f77fe640 1 --2- 172.21.15.164:0/152895669 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4f8106f20 0x7fe4f819cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.054+0000 7fe4f77fe640 1 -- 172.21.15.164:0/152895669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4f81027a0 msgr2=0x7fe4f8198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:05.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.054+0000 7fe4f77fe640 1 --2- 172.21.15.164:0/152895669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4f81027a0 0x7fe4f8198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.054+0000 7fe4f77fe640 1 -- 172.21.15.164:0/152895669 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f819d2d0 con 0x7fe4f8101df0 2024-09-19T08:57:05.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.054+0000 7fe4f77fe640 1 --2- 172.21.15.164:0/152895669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4f8101df0 0x7fe4f8198320 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800d950 tx=0x7fe4e800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:05.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.054+0000 7fe4f57fa640 1 -- 172.21.15.164:0/152895669 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8017070 con 0x7fe4f8101df0 2024-09-19T08:57:05.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.054+0000 7fe4f57fa640 1 -- 172.21.15.164:0/152895669 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e800de80 con 0x7fe4f8101df0 2024-09-19T08:57:05.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.054+0000 7fe4fe25c640 1 -- 172.21.15.164:0/152895669 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4f819d5c0 con 0x7fe4f8101df0 2024-09-19T08:57:05.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.054+0000 7fe4f57fa640 1 -- 172.21.15.164:0/152895669 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e800f620 con 0x7fe4f8101df0 2024-09-19T08:57:05.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.054+0000 7fe4fe25c640 1 -- 172.21.15.164:0/152895669 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4f81a4ea0 con 0x7fe4f8101df0 2024-09-19T08:57:05.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.057+0000 7fe4f57fa640 1 -- 172.21.15.164:0/152895669 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fe4e8009c30 con 0x7fe4f8101df0 2024-09-19T08:57:05.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.057+0000 7fe500635740 1 Processor -- start 2024-09-19T08:57:05.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.057+0000 7fe500635740 1 -- start start 2024-09-19T08:57:05.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.057+0000 7fe500635740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55737771e4a0 0x55737771e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:05.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.057+0000 7fe500635740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55737771edc0 0x557377729140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:05.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.057+0000 7fe500635740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557377729680 0x55737772ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:05.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.057+0000 7fe500635740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5573776842e0 con 0x557377729680 2024-09-19T08:57:05.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.057+0000 7fe500635740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55737762bce0 con 0x55737771edc0 2024-09-19T08:57:05.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.057+0000 7fe500635740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5573776802a0 con 0x55737771e4a0 2024-09-19T08:57:05.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.057+0000 7fe4eeffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557377729680 0x55737772ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:05.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.057+0000 7fe4eeffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557377729680 0x55737772ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57358/0 (socket says 172.21.15.164:57358) 2024-09-19T08:57:05.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.057+0000 7fe4eeffd640 1 -- 172.21.15.164:0/360573652 learned_addr learned my addr 172.21.15.164:0/360573652 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:05.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.057+0000 7fe4f7fff640 1 --2- 172.21.15.164:0/360573652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55737771e4a0 0x55737771e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:05.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.057+0000 7fe4f77fe640 1 --2- 172.21.15.164:0/360573652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55737771edc0 0x557377729140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:05.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.058+0000 7fe4eeffd640 1 -- 172.21.15.164:0/360573652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55737771e4a0 msgr2=0x55737771e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:05.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.058+0000 7fe4eeffd640 1 --2- 172.21.15.164:0/360573652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55737771e4a0 0x55737771e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.058+0000 7fe4eeffd640 1 -- 172.21.15.164:0/360573652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55737771edc0 msgr2=0x557377729140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:05.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.058+0000 7fe4eeffd640 1 --2- 172.21.15.164:0/360573652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55737771edc0 0x557377729140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.058+0000 7fe4eeffd640 1 -- 172.21.15.164:0/360573652 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557377610440 con 0x557377729680 2024-09-19T08:57:05.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.058+0000 7fe4f57fa640 1 --2- 172.21.15.164:0/152895669 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe4e40a33b0 0x7fe4e40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:05.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.058+0000 7fe4f7fff640 1 --2- 172.21.15.164:0/360573652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55737771e4a0 0x55737771e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:57:05.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.058+0000 7fe4f77fe640 1 --2- 172.21.15.164:0/360573652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55737771edc0 0x557377729140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:57:05.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.058+0000 7fe4eeffd640 1 --2- 172.21.15.164:0/360573652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557377729680 0x55737772ba70 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fe4e000ac20 tx=0x7fe4e000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d38e2848e5df643 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:05.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.058+0000 7fe4f7fff640 1 --2- 172.21.15.164:0/152895669 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe4e40a33b0 0x7fe4e40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:05.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.058+0000 7fe4f57fa640 1 -- 172.21.15.164:0/152895669 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7fe4e80cf9f0 con 0x7fe4f8101df0 2024-09-19T08:57:05.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.058+0000 7fe4edffb640 1 -- 172.21.15.164:0/360573652 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e001d070 con 0x557377729680 2024-09-19T08:57:05.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.058+0000 7fe4edffb640 1 -- 172.21.15.164:0/360573652 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e0018b20 con 0x557377729680 2024-09-19T08:57:05.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.058+0000 7fe4edffb640 1 -- 172.21.15.164:0/360573652 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0010b20 con 0x557377729680 2024-09-19T08:57:05.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.059+0000 7fe500635740 1 -- 172.21.15.164:0/360573652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557377729680 msgr2=0x55737772ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:05.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.059+0000 7fe4f7fff640 1 --2- 172.21.15.164:0/152895669 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe4e40a33b0 0x7fe4e40a5870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0009430 tx=0x7fe4f0005ef0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:05.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.059+0000 7fe500635740 1 --2- 172.21.15.164:0/360573652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557377729680 0x55737772ba70 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fe4e000ac20 tx=0x7fe4e000f790 comp rx=0 tx=0).stop 2024-09-19T08:57:05.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.059+0000 7fe500635740 1 -- 172.21.15.164:0/360573652 shutdown_connections 2024-09-19T08:57:05.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.059+0000 7fe500635740 1 --2- 172.21.15.164:0/360573652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557377729680 0x55737772ba70 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.059+0000 7fe500635740 1 --2- 172.21.15.164:0/360573652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55737771edc0 0x557377729140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.059+0000 7fe500635740 1 --2- 172.21.15.164:0/360573652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55737771e4a0 0x55737771e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.059+0000 7fe500635740 1 -- 172.21.15.164:0/360573652 >> 172.21.15.164:0/360573652 conn(0x55737771aa30 msgr2=0x55737771ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:05.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.059+0000 7fe500635740 1 -- 172.21.15.164:0/360573652 shutdown_connections 2024-09-19T08:57:05.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.059+0000 7fe500635740 1 -- 172.21.15.164:0/360573652 wait complete. 2024-09-19T08:57:05.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.059+0000 7fe500635740 1 -- start start 2024-09-19T08:57:05.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.059+0000 7fe500635740 1 Processor -- start 2024-09-19T08:57:05.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.060+0000 7fe500635740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55737771e4a0 0x557377728fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:05.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.060+0000 7fe500635740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55737771edc0 0x557377726250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:05.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.060+0000 7fe500635740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557377729680 0x557377726790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:05.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.060+0000 7fe500635740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55737772c250 con 0x557377729680 2024-09-19T08:57:05.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.060+0000 7fe500635740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55737763f050 con 0x55737771e4a0 2024-09-19T08:57:05.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.060+0000 7fe500635740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55737772c3d0 con 0x55737771edc0 2024-09-19T08:57:05.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.060+0000 7fe500635740 10 client.0 _ll_register_callbacks cb 0x55737767f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:57:05.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.060+0000 7fe4f7fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557377729680 0x557377726790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:05.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.060+0000 7fe4f7fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557377729680 0x557377726790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57368/0 (socket says 172.21.15.164:57368) 2024-09-19T08:57:05.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.060+0000 7fe4f7fff640 1 -- 172.21.15.164:0/4137964012 learned_addr learned my addr 172.21.15.164:0/4137964012 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:05.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.060+0000 7fe4f77fe640 1 --2- 172.21.15.164:0/4137964012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55737771e4a0 0x557377728fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:05.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.060+0000 7fe4eeffd640 1 --2- 172.21.15.164:0/4137964012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55737771edc0 0x557377726250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:05.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.060+0000 7fe4f7fff640 1 -- 172.21.15.164:0/4137964012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55737771edc0 msgr2=0x557377726250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:05.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.060+0000 7fe4f7fff640 1 --2- 172.21.15.164:0/4137964012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55737771edc0 0x557377726250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.060+0000 7fe4f7fff640 1 -- 172.21.15.164:0/4137964012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55737771e4a0 msgr2=0x557377728fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:05.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.060+0000 7fe4f7fff640 1 --2- 172.21.15.164:0/4137964012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55737771e4a0 0x557377728fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.060+0000 7fe4f7fff640 1 -- 172.21.15.164:0/4137964012 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573773f53c0 con 0x557377729680 2024-09-19T08:57:05.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.060+0000 7fe4f77fe640 1 --2- 172.21.15.164:0/4137964012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55737771e4a0 0x557377728fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:57:05.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.060+0000 7fe4f7fff640 1 --2- 172.21.15.164:0/4137964012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557377729680 0x557377726790 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0003ca0 tx=0x7fe4f00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d3b4c938b6bf6f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:05.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.060+0000 7fe4d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:57:05.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.061+0000 7fe4d6ffd640 1 -- 172.21.15.164:0/4137964012 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f004e070 con 0x557377729680 2024-09-19T08:57:05.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.061+0000 7fe500635740 1 -- 172.21.15.164:0/4137964012 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4e0018b20 con 0x557377729680 2024-09-19T08:57:05.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.061+0000 7fe4d6ffd640 1 -- 172.21.15.164:0/4137964012 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7fe4f00045c0 con 0x557377729680 2024-09-19T08:57:05.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.061+0000 7fe4d6ffd640 1 -- 172.21.15.164:0/4137964012 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe4f005a050 con 0x557377729680 2024-09-19T08:57:05.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.061+0000 7fe500635740 10 client.17958 fetch_fsmap learned FSMap version 32 2024-09-19T08:57:05.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.061+0000 7fe500635740 1 -- 172.21.15.164:0/4137964012 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557377737bd0 con 0x557377729680 2024-09-19T08:57:05.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.061+0000 7fe4d6ffd640 1 -- 172.21.15.164:0/4137964012 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e0018b20 con 0x557377729680 2024-09-19T08:57:05.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.061+0000 7fe4d6ffd640 1 -- 172.21.15.164:0/4137964012 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe4f0043040 con 0x557377729680 2024-09-19T08:57:05.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.061+0000 7fe500635740 10 client.17958 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:57:05.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.061+0000 7fe500635740 10 client.17958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:57:05.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.061+0000 7fe500635740 20 client.17958 populate_metadata read hostname 'smithi164' 2024-09-19T08:57:05.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.061+0000 7fe500635740 1 --2- 172.21.15.164:0/4137964012 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x557377738fe0 0x55737773b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:05.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.061+0000 7fe500635740 4 client.17958 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:57:05.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.061+0000 7fe500635740 1 -- 172.21.15.164:0/4137964012 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557377691850 con 0x557377738fe0 2024-09-19T08:57:05.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.062+0000 7fe4f77fe640 1 --2- 172.21.15.164:0/4137964012 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x557377738fe0 0x55737773b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:05.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.062+0000 7fe4f77fe640 1 --2- 172.21.15.164:0/4137964012 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x557377738fe0 0x55737773b4a0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68627d7012aa45b1 server_cookie=72bf5a0896641de7 in_seq=0 out_seq=0 2024-09-19T08:57:05.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.062+0000 7fe4d6ffd640 10 client.17958 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:57:05.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.063+0000 7fe4d6ffd640 1 -- 172.21.15.164:0/4137964012 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557377691850 con 0x557377738fe0 2024-09-19T08:57:05.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.063+0000 7fe4d6ffd640 10 client.17958 handle_command_reply: tid=0 2024-09-19T08:57:05.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.063+0000 7fe500635740 1 client.17958 shutdown 2024-09-19T08:57:05.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.063+0000 7fe500635740 1 -- 172.21.15.164:0/4137964012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557377729680 msgr2=0x557377726790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:05.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.063+0000 7fe500635740 1 --2- 172.21.15.164:0/4137964012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557377729680 0x557377726790 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0003ca0 tx=0x7fe4f00095c0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.063+0000 7fe500635740 1 -- 172.21.15.164:0/4137964012 shutdown_connections 2024-09-19T08:57:05.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.063+0000 7fe500635740 1 --2- 172.21.15.164:0/4137964012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557377729680 0x557377726790 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.063+0000 7fe500635740 1 --2- 172.21.15.164:0/4137964012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55737771edc0 0x557377726250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.063+0000 7fe500635740 1 --2- 172.21.15.164:0/4137964012 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x557377738fe0 0x55737773b4a0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.063+0000 7fe500635740 1 --2- 172.21.15.164:0/4137964012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55737771e4a0 0x557377728fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.063+0000 7fe500635740 1 -- 172.21.15.164:0/4137964012 >> 172.21.15.164:0/4137964012 conn(0x55737771aa30 msgr2=0x557377720710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:05.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.063+0000 7fe4d6ffd640 0 client.17958 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:57:05.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.063+0000 7fe500635740 1 -- 172.21.15.164:0/4137964012 shutdown_connections 2024-09-19T08:57:05.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.063+0000 7fe500635740 1 -- 172.21.15.164:0/4137964012 wait complete. 2024-09-19T08:57:05.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.063+0000 7fe500635740 20 client.17958 trim_cache size 0 max 16384 2024-09-19T08:57:05.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.080+0000 7fe500635740 1 Processor -- start 2024-09-19T08:57:05.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.080+0000 7fe500635740 1 -- start start 2024-09-19T08:57:05.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.080+0000 7fe500635740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55737771e4a0 0x5573777354b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:05.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.080+0000 7fe500635740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55737771edc0 0x5573777217f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:05.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.080+0000 7fe500635740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557377729680 0x557377740d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:05.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.080+0000 7fe500635740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f81ad670 con 0x55737771e4a0 2024-09-19T08:57:05.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.080+0000 7fe500635740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f81af1b0 con 0x55737771edc0 2024-09-19T08:57:05.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.080+0000 7fe500635740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f81add40 con 0x557377729680 2024-09-19T08:57:05.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.080+0000 7fe4f77fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55737771e4a0 0x5573777354b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:05.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.080+0000 7fe4f77fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55737771e4a0 0x5573777354b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57378/0 (socket says 172.21.15.164:57378) 2024-09-19T08:57:05.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.080+0000 7fe4f77fe640 1 -- 172.21.15.164:0/3213260572 learned_addr learned my addr 172.21.15.164:0/3213260572 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:05.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.080+0000 7fe4f7fff640 1 --2- 172.21.15.164:0/3213260572 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557377729680 0x557377740d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:05.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.080+0000 7fe4f77fe640 1 -- 172.21.15.164:0/3213260572 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557377729680 msgr2=0x557377740d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:05.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.080+0000 7fe4f77fe640 1 --2- 172.21.15.164:0/3213260572 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557377729680 0x557377740d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.080+0000 7fe4f77fe640 1 -- 172.21.15.164:0/3213260572 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55737771edc0 msgr2=0x5573777217f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:05.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.080+0000 7fe4f77fe640 1 --2- 172.21.15.164:0/3213260572 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55737771edc0 0x5573777217f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.081+0000 7fe4f77fe640 1 -- 172.21.15.164:0/3213260572 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55737771ebf0 con 0x55737771e4a0 2024-09-19T08:57:05.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.081+0000 7fe4f77fe640 1 --2- 172.21.15.164:0/3213260572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55737771e4a0 0x5573777354b0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fe4e809b9b0 tx=0x7fe4e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794fa4ea526b6ff7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:05.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.081+0000 7fe4ed7fa640 1 -- 172.21.15.164:0/3213260572 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8020040 con 0x55737771e4a0 2024-09-19T08:57:05.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.081+0000 7fe4ed7fa640 1 -- 172.21.15.164:0/3213260572 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e80a4920 con 0x55737771e4a0 2024-09-19T08:57:05.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.081+0000 7fe4ed7fa640 1 -- 172.21.15.164:0/3213260572 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e809b360 con 0x55737771e4a0 2024-09-19T08:57:05.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.081+0000 7fe500635740 1 -- 172.21.15.164:0/3213260572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55737771e4a0 msgr2=0x5573777354b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:05.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.081+0000 7fe500635740 1 --2- 172.21.15.164:0/3213260572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55737771e4a0 0x5573777354b0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fe4e809b9b0 tx=0x7fe4e8005880 comp rx=0 tx=0).stop 2024-09-19T08:57:05.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.081+0000 7fe500635740 1 -- 172.21.15.164:0/3213260572 shutdown_connections 2024-09-19T08:57:05.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.081+0000 7fe500635740 1 --2- 172.21.15.164:0/3213260572 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557377729680 0x557377740d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.081+0000 7fe500635740 1 --2- 172.21.15.164:0/3213260572 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55737771edc0 0x5573777217f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.081+0000 7fe500635740 1 --2- 172.21.15.164:0/3213260572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55737771e4a0 0x5573777354b0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.081+0000 7fe500635740 1 -- 172.21.15.164:0/3213260572 >> 172.21.15.164:0/3213260572 conn(0x55737771aa30 msgr2=0x5573777210e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:05.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.082+0000 7fe500635740 1 -- 172.21.15.164:0/3213260572 shutdown_connections 2024-09-19T08:57:05.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.082+0000 7fe500635740 1 -- 172.21.15.164:0/3213260572 wait complete. 2024-09-19T08:57:05.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.082+0000 7fe500635740 1 -- start start 2024-09-19T08:57:05.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.082+0000 7fe500635740 1 Processor -- start 2024-09-19T08:57:05.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.082+0000 7fe500635740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55737771e4a0 0x557377731cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:05.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.082+0000 7fe500635740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55737771edc0 0x55737772cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:05.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.082+0000 7fe500635740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557377729680 0x55737772d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:05.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.082+0000 7fe500635740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f81aeae0 con 0x55737771e4a0 2024-09-19T08:57:05.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.082+0000 7fe500635740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f81ae410 con 0x55737771edc0 2024-09-19T08:57:05.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.082+0000 7fe500635740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f81ab670 con 0x557377729680 2024-09-19T08:57:05.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.082+0000 7fe500635740 10 client.0 _ll_register_callbacks cb 0x7fe4f0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:57:05.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.082+0000 7fe4f77fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55737771e4a0 0x557377731cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:05.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.082+0000 7fe4f77fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55737771e4a0 0x557377731cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57386/0 (socket says 172.21.15.164:57386) 2024-09-19T08:57:05.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.082+0000 7fe4f77fe640 1 -- 172.21.15.164:0/529577083 learned_addr learned my addr 172.21.15.164:0/529577083 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:05.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.082+0000 7fe4eeffd640 1 --2- 172.21.15.164:0/529577083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55737771edc0 0x55737772cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:05.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.083+0000 7fe4f7fff640 1 --2- 172.21.15.164:0/529577083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557377729680 0x55737772d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:05.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.083+0000 7fe4f77fe640 1 -- 172.21.15.164:0/529577083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557377729680 msgr2=0x55737772d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:05.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.083+0000 7fe4f77fe640 1 --2- 172.21.15.164:0/529577083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557377729680 0x55737772d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.083+0000 7fe4f77fe640 1 -- 172.21.15.164:0/529577083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55737771edc0 msgr2=0x55737772cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:05.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.083+0000 7fe4f77fe640 1 --2- 172.21.15.164:0/529577083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55737771edc0 0x55737772cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.083+0000 7fe4f77fe640 1 -- 172.21.15.164:0/529577083 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55737772dd60 con 0x55737771e4a0 2024-09-19T08:57:05.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.083+0000 7fe4f77fe640 1 --2- 172.21.15.164:0/529577083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55737771e4a0 0x557377731cc0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8098680 tx=0x7fe4e80a01e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=622d8ce830472003 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:05.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.083+0000 7fe4d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:57:05.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.083+0000 7fe4d6ffd640 1 -- 172.21.15.164:0/529577083 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8020050 con 0x55737771e4a0 2024-09-19T08:57:05.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.083+0000 7fe4d6ffd640 1 -- 172.21.15.164:0/529577083 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8455+0+0 (secure 0 0 0) 0x7fe4e8098320 con 0x55737771e4a0 2024-09-19T08:57:05.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.083+0000 7fe500635740 1 -- 172.21.15.164:0/529577083 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4e80a4920 con 0x55737771e4a0 2024-09-19T08:57:05.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.083+0000 7fe4d6ffd640 1 -- 172.21.15.164:0/529577083 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe4e809b100 con 0x55737771e4a0 2024-09-19T08:57:05.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.083+0000 7fe500635740 10 client.17964 fetch_fsmap learned FSMap version 32 2024-09-19T08:57:05.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.083+0000 7fe500635740 1 -- 172.21.15.164:0/529577083 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5573777a3cc0 con 0x55737771e4a0 2024-09-19T08:57:05.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.083+0000 7fe4d6ffd640 1 -- 172.21.15.164:0/529577083 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e80a4920 con 0x55737771e4a0 2024-09-19T08:57:05.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.084+0000 7fe4d6ffd640 1 -- 172.21.15.164:0/529577083 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe4e809ada0 con 0x55737771e4a0 2024-09-19T08:57:05.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.084+0000 7fe500635740 10 client.17964 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T08:57:05.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.084+0000 7fe500635740 10 client.17964 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:57:05.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.084+0000 7fe500635740 20 client.17964 populate_metadata read hostname 'smithi164' 2024-09-19T08:57:05.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.084+0000 7fe500635740 1 --2- 172.21.15.164:0/529577083 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5573777a4390 0x5573777a67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:05.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.084+0000 7fe500635740 4 client.17964 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:57:05.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.084+0000 7fe500635740 1 -- 172.21.15.164:0/529577083 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55737768bd60 con 0x5573777a4390 2024-09-19T08:57:05.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.084+0000 7fe4eeffd640 1 --2- 172.21.15.164:0/529577083 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5573777a4390 0x5573777a67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:05.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.084+0000 7fe4d6ffd640 10 client.17964 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:57:05.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.084+0000 7fe4eeffd640 1 --2- 172.21.15.164:0/529577083 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5573777a4390 0x5573777a67b0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4f9149025895ea1 server_cookie=a943e35d2afb0ab2 in_seq=0 out_seq=0 2024-09-19T08:57:05.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.085+0000 7fe4d6ffd640 1 -- 172.21.15.164:0/529577083 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55737768bd60 con 0x5573777a4390 2024-09-19T08:57:05.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.085+0000 7fe4d6ffd640 10 client.17964 handle_command_reply: tid=0 2024-09-19T08:57:05.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.085+0000 7fe500635740 1 client.17964 shutdown 2024-09-19T08:57:05.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.085+0000 7fe500635740 1 -- 172.21.15.164:0/529577083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55737771e4a0 msgr2=0x557377731cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:05.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.085+0000 7fe500635740 1 --2- 172.21.15.164:0/529577083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55737771e4a0 0x557377731cc0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8098680 tx=0x7fe4e80a01e0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.085+0000 7fe500635740 1 -- 172.21.15.164:0/529577083 shutdown_connections 2024-09-19T08:57:05.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.085+0000 7fe500635740 1 --2- 172.21.15.164:0/529577083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557377729680 0x55737772d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.085+0000 7fe500635740 1 --2- 172.21.15.164:0/529577083 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5573777a4390 0x5573777a67b0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.085+0000 7fe500635740 1 --2- 172.21.15.164:0/529577083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55737771edc0 0x55737772cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.085+0000 7fe500635740 1 --2- 172.21.15.164:0/529577083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55737771e4a0 0x557377731cc0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.085+0000 7fe500635740 1 -- 172.21.15.164:0/529577083 >> 172.21.15.164:0/529577083 conn(0x55737771aa30 msgr2=0x557377720710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:05.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.085+0000 7fe4d6ffd640 0 client.17964 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:57:05.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.085+0000 7fe500635740 1 -- 172.21.15.164:0/529577083 shutdown_connections 2024-09-19T08:57:05.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.085+0000 7fe500635740 1 -- 172.21.15.164:0/529577083 wait complete. 2024-09-19T08:57:05.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.085+0000 7fe500635740 20 client.17964 trim_cache size 0 max 16384 2024-09-19T08:57:05.093 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:57:05.093 INFO:teuthology.orchestra.run.smithi164.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T08:57:05.093 INFO:teuthology.orchestra.run.smithi164.stdout: "scrubs": { 2024-09-19T08:57:05.093 INFO:teuthology.orchestra.run.smithi164.stdout: "b75f8ed1-f2a9-473a-aec5-fc9da3aad776": { 2024-09-19T08:57:05.093 INFO:teuthology.orchestra.run.smithi164.stdout: "path": "/", 2024-09-19T08:57:05.093 INFO:teuthology.orchestra.run.smithi164.stdout: "tag": "b75f8ed1-f2a9-473a-aec5-fc9da3aad776", 2024-09-19T08:57:05.094 INFO:teuthology.orchestra.run.smithi164.stdout: "options": "recursive,force" 2024-09-19T08:57:05.094 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:05.094 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:05.094 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:57:05.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.085+0000 7fe4fe25c640 1 -- 172.21.15.164:0/152895669 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe4e40a33b0 msgr2=0x7fe4e40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:05.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.085+0000 7fe4fe25c640 1 --2- 172.21.15.164:0/152895669 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe4e40a33b0 0x7fe4e40a5870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0009430 tx=0x7fe4f0005ef0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.085+0000 7fe4fe25c640 1 -- 172.21.15.164:0/152895669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4f8101df0 msgr2=0x7fe4f8198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:05.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.085+0000 7fe4fe25c640 1 --2- 172.21.15.164:0/152895669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4f8101df0 0x7fe4f8198320 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800d950 tx=0x7fe4e800dd50 comp rx=0 tx=0).stop 2024-09-19T08:57:05.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.086+0000 7fe4fe25c640 1 -- 172.21.15.164:0/152895669 shutdown_connections 2024-09-19T08:57:05.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.086+0000 7fe4fe25c640 1 --2- 172.21.15.164:0/152895669 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4f8106f20 0x7fe4f819cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.086+0000 7fe4fe25c640 1 --2- 172.21.15.164:0/152895669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4f81027a0 0x7fe4f8198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.086+0000 7fe4fe25c640 1 --2- 172.21.15.164:0/152895669 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fe4e40a33b0 0x7fe4e40a5870 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.086+0000 7fe4fe25c640 1 --2- 172.21.15.164:0/152895669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4f8101df0 0x7fe4f8198320 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:05.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.086+0000 7fe4fe25c640 1 -- 172.21.15.164:0/152895669 >> 172.21.15.164:0/152895669 conn(0x7fe4f80fd8b0 msgr2=0x7fe4f80fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:05.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.086+0000 7fe4fe25c640 1 -- 172.21.15.164:0/152895669 shutdown_connections 2024-09-19T08:57:05.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:05.086+0000 7fe4fe25c640 1 -- 172.21.15.164:0/152895669 wait complete. 2024-09-19T08:57:05.809 INFO:tasks.cephfs.filesystem:scrub status for tag:b75f8ed1-f2a9-473a-aec5-fc9da3aad776 - {'path': '/', 'tag': 'b75f8ed1-f2a9-473a-aec5-fc9da3aad776', 'options': 'recursive,force'} 2024-09-19T08:57:06.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:06 smithi170 ceph-mon[35072]: pgmap v351: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 34 KiB/s wr, 25 op/s 2024-09-19T08:57:06.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:06 smithi173 ceph-mon[35306]: pgmap v351: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 34 KiB/s wr, 25 op/s 2024-09-19T08:57:06.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:06 smithi164 ceph-mon[30928]: pgmap v351: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 34 KiB/s wr, 25 op/s 2024-09-19T08:57:08.624 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:08 smithi170 ceph-mon[35072]: pgmap v352: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-09-19T08:57:08.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:08 smithi170 ceph-mon[35072]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T08:57:08.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:08 smithi173 ceph-mon[35306]: pgmap v352: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-09-19T08:57:08.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:08 smithi173 ceph-mon[35306]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T08:57:08.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:08 smithi164 ceph-mon[30928]: pgmap v352: 97 pgs: 97 active+clean; 21 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-09-19T08:57:08.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:08 smithi164 ceph-mon[30928]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T08:57:10.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:10 smithi170 ceph-mon[35072]: pgmap v354: 97 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 89 active+clean; 20 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 53 KiB/s wr, 34 op/s 2024-09-19T08:57:10.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:10 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:57:10.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:10 smithi173 ceph-mon[35306]: pgmap v354: 97 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 89 active+clean; 20 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 53 KiB/s wr, 34 op/s 2024-09-19T08:57:10.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:10 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:57:10.781 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:10 smithi164 ceph-mon[30928]: pgmap v354: 97 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 89 active+clean; 20 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 53 KiB/s wr, 34 op/s 2024-09-19T08:57:10.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:10 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:57:11.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:11 smithi164 ceph-mon[30928]: pgmap v355: 97 pgs: 12 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 81 active+clean; 13 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 53 KiB/s wr, 34 op/s 2024-09-19T08:57:11.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:11 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:57:11.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:11 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:57:11.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:11 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:57:11.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:11 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:57:11.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:11 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:57:11.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:11 smithi170 ceph-mon[35072]: pgmap v355: 97 pgs: 12 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 81 active+clean; 13 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 53 KiB/s wr, 34 op/s 2024-09-19T08:57:11.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:11 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:57:11.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:11 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:57:11.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:11 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:57:11.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:11 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:57:11.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:11 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:57:11.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:11 smithi173 ceph-mon[35306]: pgmap v355: 97 pgs: 12 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 81 active+clean; 13 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 53 KiB/s wr, 34 op/s 2024-09-19T08:57:11.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:11 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:57:11.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:11 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:57:11.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:11 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:57:11.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:11 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:57:11.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:11 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:57:12.122 INFO:tasks.workunit.client.0.smithi164.stdout:stopping iogen 2024-09-19T08:57:12.123 INFO:tasks.workunit.client.0.smithi164.stderr:+ echo 'stopping iogen' 2024-09-19T08:57:12.123 INFO:tasks.workunit.client.0.smithi164.stderr:+ ./iogen -k 2024-09-19T08:57:12.129 INFO:tasks.workunit.client.0.smithi164.stdout:OK 2024-09-19T08:57:12.129 INFO:tasks.workunit.client.0.smithi164.stderr:+ echo OK 2024-09-19T08:57:12.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:12 smithi164 ceph-mon[30928]: scrub summary: idle 2024-09-19T08:57:12.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:12 smithi170 ceph-mon[35072]: scrub summary: idle 2024-09-19T08:57:12.910 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:12 smithi173 ceph-mon[35306]: scrub summary: idle 2024-09-19T08:57:13.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:13 smithi164 ceph-mon[30928]: scrub summary: idle 2024-09-19T08:57:13.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:13 smithi164 ceph-mon[30928]: pgmap v356: 97 pgs: 12 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 81 active+clean; 12 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 41 KiB/s wr, 22 op/s 2024-09-19T08:57:13.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:13 smithi170 ceph-mon[35072]: scrub summary: idle 2024-09-19T08:57:13.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:13 smithi170 ceph-mon[35072]: pgmap v356: 97 pgs: 12 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 81 active+clean; 12 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 41 KiB/s wr, 22 op/s 2024-09-19T08:57:13.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:13 smithi173 ceph-mon[35306]: scrub summary: idle 2024-09-19T08:57:13.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:13 smithi173 ceph-mon[35306]: pgmap v356: 97 pgs: 12 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 81 active+clean; 12 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 41 KiB/s wr, 22 op/s 2024-09-19T08:57:14.150 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-19T08:57:14.150 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-19T08:57:14.267 INFO:tasks.workunit:Stopping ['suites/iogen.sh'] on client.0... 2024-09-19T08:57:14.267 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-19T08:57:14.967 DEBUG:teuthology.parallel:result is None 2024-09-19T08:57:14.967 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-19T08:57:15.006 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-19T08:57:15.006 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-19T08:57:15.016 INFO:tasks.quiescer:joining Quiescers 2024-09-19T08:57:15.016 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-19T08:57:15.018 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:57:15.294 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:57:16.176 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:15 smithi164 ceph-mon[30928]: pgmap v357: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 34 KiB/s wr, 18 op/s 2024-09-19T08:57:16.176 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:15 smithi164 ceph-mon[30928]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T08:57:16.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.259+0000 7f05b8a28640 1 -- 172.21.15.164:0/67431347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05b4100680 msgr2=0x7f05b4100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:16.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.259+0000 7f05b8a28640 1 --2- 172.21.15.164:0/67431347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05b4100680 0x7f05b4100ae0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f059c009900 tx=0x7f059c02a410 comp rx=0 tx=0).stop 2024-09-19T08:57:16.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.259+0000 7f05b8a28640 1 -- 172.21.15.164:0/67431347 shutdown_connections 2024-09-19T08:57:16.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.259+0000 7f05b8a28640 1 --2- 172.21.15.164:0/67431347 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05b4101020 0x7f05b410d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:16.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.259+0000 7f05b8a28640 1 --2- 172.21.15.164:0/67431347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05b4100680 0x7f05b4100ae0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:16.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.259+0000 7f05b8a28640 1 --2- 172.21.15.164:0/67431347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b4105220 0x7f05b4105600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:16.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.259+0000 7f05b8a28640 1 -- 172.21.15.164:0/67431347 >> 172.21.15.164:0/67431347 conn(0x7f05b4077210 msgr2=0x7f05b4077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:16.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.259+0000 7f05b8a28640 1 -- 172.21.15.164:0/67431347 shutdown_connections 2024-09-19T08:57:16.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.259+0000 7f05b8a28640 1 -- 172.21.15.164:0/67431347 wait complete. 2024-09-19T08:57:16.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.260+0000 7f05b8a28640 1 Processor -- start 2024-09-19T08:57:16.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.260+0000 7f05b8a28640 1 -- start start 2024-09-19T08:57:16.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.260+0000 7f05b8a28640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05b4100680 0x7f05b4199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:16.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.260+0000 7f05b8a28640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b4101020 0x7f05b4199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:16.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.260+0000 7f05b8a28640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05b4105220 0x7f05b41a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:16.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.260+0000 7f05b8a28640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f05b4110f90 con 0x7f05b4100680 2024-09-19T08:57:16.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.260+0000 7f05b8a28640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f05b4110e10 con 0x7f05b4101020 2024-09-19T08:57:16.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.260+0000 7f05b8a28640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f05b4111110 con 0x7f05b4105220 2024-09-19T08:57:16.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.260+0000 7f05b259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05b4100680 0x7f05b4199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:16.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.260+0000 7f05b259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05b4100680 0x7f05b4199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51186/0 (socket says 172.21.15.164:51186) 2024-09-19T08:57:16.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.260+0000 7f05b259c640 1 -- 172.21.15.164:0/2453173991 learned_addr learned my addr 172.21.15.164:0/2453173991 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:16.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.260+0000 7f05b1d9b640 1 --2- 172.21.15.164:0/2453173991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b4101020 0x7f05b4199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:16.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.261+0000 7f05b259c640 1 -- 172.21.15.164:0/2453173991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05b4105220 msgr2=0x7f05b41a43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:16.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.261+0000 7f05b259c640 1 --2- 172.21.15.164:0/2453173991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05b4105220 0x7f05b41a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:16.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.261+0000 7f05b259c640 1 -- 172.21.15.164:0/2453173991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b4101020 msgr2=0x7f05b4199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:16.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.261+0000 7f05b259c640 1 --2- 172.21.15.164:0/2453173991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b4101020 0x7f05b4199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:16.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.261+0000 7f05b259c640 1 -- 172.21.15.164:0/2453173991 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b41a48e0 con 0x7f05b4100680 2024-09-19T08:57:16.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.261+0000 7f05b259c640 1 --2- 172.21.15.164:0/2453173991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05b4100680 0x7f05b4199320 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f05a400d950 tx=0x7f05a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:16.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.261+0000 7f05ab7fe640 1 -- 172.21.15.164:0/2453173991 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a4017030 con 0x7f05b4100680 2024-09-19T08:57:16.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.261+0000 7f05ab7fe640 1 -- 172.21.15.164:0/2453173991 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05a4004020 con 0x7f05b4100680 2024-09-19T08:57:16.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.261+0000 7f05b8a28640 1 -- 172.21.15.164:0/2453173991 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05b41a4ab0 con 0x7f05b4100680 2024-09-19T08:57:16.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.261+0000 7f05ab7fe640 1 -- 172.21.15.164:0/2453173991 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a40047a0 con 0x7f05b4100680 2024-09-19T08:57:16.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.261+0000 7f05b8a28640 1 -- 172.21.15.164:0/2453173991 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05b41a4f70 con 0x7f05b4100680 2024-09-19T08:57:16.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.262+0000 7f05b8a28640 1 -- 172.21.15.164:0/2453173991 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05b4100ae0 con 0x7f05b4100680 2024-09-19T08:57:16.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.263+0000 7f05ab7fe640 1 -- 172.21.15.164:0/2453173991 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f05a40041d0 con 0x7f05b4100680 2024-09-19T08:57:16.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.264+0000 7f05ab7fe640 1 --2- 172.21.15.164:0/2453173991 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f05a00a33b0 0x7f05a00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:16.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.264+0000 7f05b1d9b640 1 --2- 172.21.15.164:0/2453173991 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f05a00a33b0 0x7f05a00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:16.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.264+0000 7f05ab7fe640 1 -- 172.21.15.164:0/2453173991 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7f05a40cf630 con 0x7f05b4100680 2024-09-19T08:57:16.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.264+0000 7f05b1d9b640 1 --2- 172.21.15.164:0/2453173991 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f05a00a33b0 0x7f05a00a5870 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f059c009230 tx=0x7f059c0058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:16.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.268+0000 7f05ab7fe640 1 -- 172.21.15.164:0/2453173991 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f05a409c050 con 0x7f05b4100680 2024-09-19T08:57:16.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:15 smithi170 ceph-mon[35072]: pgmap v357: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 34 KiB/s wr, 18 op/s 2024-09-19T08:57:16.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:15 smithi170 ceph-mon[35072]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T08:57:16.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:15 smithi173 ceph-mon[35306]: pgmap v357: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 34 KiB/s wr, 18 op/s 2024-09-19T08:57:16.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:15 smithi173 ceph-mon[35306]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T08:57:16.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.468+0000 7f05b8a28640 1 -- 172.21.15.164:0/2453173991 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05b4108730 con 0x7f05b4100680 2024-09-19T08:57:16.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.469+0000 7f05ab7fe640 1 -- 172.21.15.164:0/2453173991 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11985 (secure 0 0 0) 0x7f05a409b950 con 0x7f05b4100680 2024-09-19T08:57:16.469 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:57:16.469 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":33,"btime":"2024-09-19T08:57:15:914589+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:57:15.844000+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24442,24424,24449,14595]},"id":1}]} 2024-09-19T08:57:16.469 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 33 2024-09-19T08:57:16.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.471+0000 7f05b8a28640 1 -- 172.21.15.164:0/2453173991 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f05a00a33b0 msgr2=0x7f05a00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:16.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.471+0000 7f05b8a28640 1 --2- 172.21.15.164:0/2453173991 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f05a00a33b0 0x7f05a00a5870 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f059c009230 tx=0x7f059c0058d0 comp rx=0 tx=0).stop 2024-09-19T08:57:16.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.471+0000 7f05b8a28640 1 -- 172.21.15.164:0/2453173991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05b4100680 msgr2=0x7f05b4199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:16.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.471+0000 7f05b8a28640 1 --2- 172.21.15.164:0/2453173991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05b4100680 0x7f05b4199320 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f05a400d950 tx=0x7f05a400de10 comp rx=0 tx=0).stop 2024-09-19T08:57:16.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.472+0000 7f05b8a28640 1 -- 172.21.15.164:0/2453173991 shutdown_connections 2024-09-19T08:57:16.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.472+0000 7f05b8a28640 1 --2- 172.21.15.164:0/2453173991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05b4105220 0x7f05b41a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:16.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.472+0000 7f05b8a28640 1 --2- 172.21.15.164:0/2453173991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b4101020 0x7f05b4199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:16.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.472+0000 7f05b8a28640 1 --2- 172.21.15.164:0/2453173991 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f05a00a33b0 0x7f05a00a5870 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:16.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.472+0000 7f05b8a28640 1 --2- 172.21.15.164:0/2453173991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05b4100680 0x7f05b4199320 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:16.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.472+0000 7f05b8a28640 1 -- 172.21.15.164:0/2453173991 >> 172.21.15.164:0/2453173991 conn(0x7f05b4077210 msgr2=0x7f05b40fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:16.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.472+0000 7f05b8a28640 1 -- 172.21.15.164:0/2453173991 shutdown_connections 2024-09-19T08:57:16.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:16.472+0000 7f05b8a28640 1 -- 172.21.15.164:0/2453173991 wait complete. 2024-09-19T08:57:17.206 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:16 smithi164 ceph-mon[30928]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T08:57:17.206 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:16 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2453173991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:57:17.252 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --cancel --all' 2024-09-19T08:57:17.252 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --cancel --all 2024-09-19T08:57:17.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:16 smithi170 ceph-mon[35072]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T08:57:17.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:16 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2453173991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:57:17.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:16 smithi173 ceph-mon[35306]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T08:57:17.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:16 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2453173991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:57:17.633 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:57:18.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:17 smithi164 ceph-mon[30928]: pgmap v359: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-09-19T08:57:18.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:17 smithi170 ceph-mon[35072]: pgmap v359: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-09-19T08:57:18.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:17 smithi173 ceph-mon[35306]: pgmap v359: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-09-19T08:57:19.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.302+0000 7f0afaf52640 1 -- 172.21.15.164:0/2228561317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0af4101420 msgr2=0x7f0af410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:19.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.302+0000 7f0afaf52640 1 --2- 172.21.15.164:0/2228561317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0af4101420 0x7f0af410d950 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f0aec009cd0 tx=0x7f0aec02a8e0 comp rx=0 tx=0).stop 2024-09-19T08:57:19.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.302+0000 7f0afaf52640 1 -- 172.21.15.164:0/2228561317 shutdown_connections 2024-09-19T08:57:19.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.302+0000 7f0afaf52640 1 --2- 172.21.15.164:0/2228561317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0af4101420 0x7f0af410d950 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:19.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.302+0000 7f0afaf52640 1 --2- 172.21.15.164:0/2228561317 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0af4100a80 0x7f0af4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:19.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.302+0000 7f0afaf52640 1 --2- 172.21.15.164:0/2228561317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0af4106a80 0x7f0af4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:19.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.302+0000 7f0afaf52640 1 -- 172.21.15.164:0/2228561317 >> 172.21.15.164:0/2228561317 conn(0x7f0af40fc7a0 msgr2=0x7f0af40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:19.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.302+0000 7f0afaf52640 1 -- 172.21.15.164:0/2228561317 shutdown_connections 2024-09-19T08:57:19.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.302+0000 7f0afaf52640 1 -- 172.21.15.164:0/2228561317 wait complete. 2024-09-19T08:57:19.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.303+0000 7f0afaf52640 1 Processor -- start 2024-09-19T08:57:19.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.303+0000 7f0afaf52640 1 -- start start 2024-09-19T08:57:19.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.303+0000 7f0afaf52640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0af4100a80 0x7f0af41972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:19.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.303+0000 7f0afaf52640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0af4101420 0x7f0af4197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:19.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.303+0000 7f0afaf52640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0af4106a80 0x7f0af419bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:19.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.303+0000 7f0afaf52640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0af4110e80 con 0x7f0af4101420 2024-09-19T08:57:19.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.303+0000 7f0afaf52640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0af4110d00 con 0x7f0af4106a80 2024-09-19T08:57:19.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.303+0000 7f0afaf52640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0af4111000 con 0x7f0af4100a80 2024-09-19T08:57:19.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.303+0000 7f0af3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0af4101420 0x7f0af4197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:19.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.303+0000 7f0af3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0af4101420 0x7f0af4197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51204/0 (socket says 172.21.15.164:51204) 2024-09-19T08:57:19.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.303+0000 7f0af3fff640 1 -- 172.21.15.164:0/2901731396 learned_addr learned my addr 172.21.15.164:0/2901731396 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:19.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.303+0000 7f0af94ef640 1 --2- 172.21.15.164:0/2901731396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0af4106a80 0x7f0af419bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:19.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.303+0000 7f0af8cee640 1 --2- 172.21.15.164:0/2901731396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0af4100a80 0x7f0af41972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:19.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.303+0000 7f0af3fff640 1 -- 172.21.15.164:0/2901731396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0af4100a80 msgr2=0x7f0af41972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:19.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.303+0000 7f0af3fff640 1 --2- 172.21.15.164:0/2901731396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0af4100a80 0x7f0af41972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:19.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.303+0000 7f0af3fff640 1 -- 172.21.15.164:0/2901731396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0af4106a80 msgr2=0x7f0af419bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:19.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.303+0000 7f0af3fff640 1 --2- 172.21.15.164:0/2901731396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0af4106a80 0x7f0af419bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:19.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.303+0000 7f0af3fff640 1 -- 172.21.15.164:0/2901731396 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af419c290 con 0x7f0af4101420 2024-09-19T08:57:19.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.303+0000 7f0af8cee640 1 --2- 172.21.15.164:0/2901731396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0af4100a80 0x7f0af41972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:57:19.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.304+0000 7f0af3fff640 1 --2- 172.21.15.164:0/2901731396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0af4101420 0x7f0af4197820 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f0ae000d770 tx=0x7f0ae000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:19.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.304+0000 7f0af1ffb640 1 -- 172.21.15.164:0/2901731396 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0017070 con 0x7f0af4101420 2024-09-19T08:57:19.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.304+0000 7f0afaf52640 1 -- 172.21.15.164:0/2901731396 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0af419c580 con 0x7f0af4101420 2024-09-19T08:57:19.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.304+0000 7f0af1ffb640 1 -- 172.21.15.164:0/2901731396 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ae0005500 con 0x7f0af4101420 2024-09-19T08:57:19.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.304+0000 7f0af1ffb640 1 -- 172.21.15.164:0/2901731396 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0004e00 con 0x7f0af4101420 2024-09-19T08:57:19.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.304+0000 7f0afaf52640 1 -- 172.21.15.164:0/2901731396 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0af41a3e60 con 0x7f0af4101420 2024-09-19T08:57:19.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.306+0000 7f0af1ffb640 1 -- 172.21.15.164:0/2901731396 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f0ae0004fa0 con 0x7f0af4101420 2024-09-19T08:57:19.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.307+0000 7f0afd32b740 1 Processor -- start 2024-09-19T08:57:19.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.307+0000 7f0afd32b740 1 -- start start 2024-09-19T08:57:19.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.307+0000 7f0afd32b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc730f5a00 0x55cc730f5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:19.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.307+0000 7f0afd32b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc730f6320 0x55cc731006a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:19.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.307+0000 7f0afd32b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cc73100be0 0x55cc73102fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:19.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.307+0000 7f0afd32b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cc7305a200 con 0x55cc730f6320 2024-09-19T08:57:19.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.307+0000 7f0afd32b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55cc73000dc0 con 0x55cc73100be0 2024-09-19T08:57:19.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.307+0000 7f0afd32b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55cc730561c0 con 0x55cc730f5a00 2024-09-19T08:57:19.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.307+0000 7f0af1ffb640 1 --2- 172.21.15.164:0/2901731396 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f0ae40a33b0 0x7f0ae40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:19.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.307+0000 7f0af8cee640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc730f6320 0x55cc731006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:19.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.307+0000 7f0af8cee640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc730f6320 0x55cc731006a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51216/0 (socket says 172.21.15.164:51216) 2024-09-19T08:57:19.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.307+0000 7f0af8cee640 1 -- 172.21.15.164:0/4185464833 learned_addr learned my addr 172.21.15.164:0/4185464833 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:19.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.307+0000 7f0af3fff640 1 --2- 172.21.15.164:0/4185464833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cc73100be0 0x55cc73102fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:19.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.307+0000 7f0af94ef640 1 --2- 172.21.15.164:0/4185464833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc730f5a00 0x55cc730f5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:19.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.307+0000 7f0af94ef640 1 --2- 172.21.15.164:0/2901731396 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f0ae40a33b0 0x7f0ae40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:19.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.307+0000 7f0af8cee640 1 -- 172.21.15.164:0/4185464833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc730f5a00 msgr2=0x55cc730f5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:19.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.307+0000 7f0af1ffb640 1 -- 172.21.15.164:0/2901731396 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7f0ae00cf770 con 0x7f0af4101420 2024-09-19T08:57:19.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.307+0000 7f0af8cee640 1 --2- 172.21.15.164:0/4185464833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc730f5a00 0x55cc730f5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:19.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.307+0000 7f0af8cee640 1 -- 172.21.15.164:0/4185464833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cc73100be0 msgr2=0x55cc73102fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:19.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.307+0000 7f0af8cee640 1 --2- 172.21.15.164:0/4185464833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cc73100be0 0x55cc73102fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:19.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.307+0000 7f0af8cee640 1 -- 172.21.15.164:0/4185464833 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc72dcb620 con 0x55cc730f6320 2024-09-19T08:57:19.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.308+0000 7f0af8cee640 1 --2- 172.21.15.164:0/4185464833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc730f6320 0x55cc731006a0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8009cd0 tx=0x7f0ae80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce528c85e203900 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:19.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.308+0000 7f0af94ef640 1 --2- 172.21.15.164:0/2901731396 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f0ae40a33b0 0x7f0ae40a5870 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0aec004ab0 tx=0x7f0aec004a40 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:19.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.308+0000 7f0adeffd640 1 -- 172.21.15.164:0/4185464833 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8012450 con 0x55cc730f6320 2024-09-19T08:57:19.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.308+0000 7f0adeffd640 1 -- 172.21.15.164:0/4185464833 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ae8012ad0 con 0x55cc730f6320 2024-09-19T08:57:19.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.308+0000 7f0adeffd640 1 -- 172.21.15.164:0/4185464833 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae800fca0 con 0x55cc730f6320 2024-09-19T08:57:19.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.309+0000 7f0afd32b740 1 -- 172.21.15.164:0/4185464833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc730f6320 msgr2=0x55cc731006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:19.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.309+0000 7f0afd32b740 1 --2- 172.21.15.164:0/4185464833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc730f6320 0x55cc731006a0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8009cd0 tx=0x7f0ae80117b0 comp rx=0 tx=0).stop 2024-09-19T08:57:19.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.309+0000 7f0afd32b740 1 -- 172.21.15.164:0/4185464833 shutdown_connections 2024-09-19T08:57:19.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.309+0000 7f0afd32b740 1 --2- 172.21.15.164:0/4185464833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cc73100be0 0x55cc73102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:19.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.309+0000 7f0afd32b740 1 --2- 172.21.15.164:0/4185464833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc730f6320 0x55cc731006a0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:19.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.309+0000 7f0afd32b740 1 --2- 172.21.15.164:0/4185464833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc730f5a00 0x55cc730f5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:19.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.309+0000 7f0afd32b740 1 -- 172.21.15.164:0/4185464833 >> 172.21.15.164:0/4185464833 conn(0x55cc730f1f90 msgr2=0x55cc730f2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:19.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.309+0000 7f0afd32b740 1 -- 172.21.15.164:0/4185464833 shutdown_connections 2024-09-19T08:57:19.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.309+0000 7f0afd32b740 1 -- 172.21.15.164:0/4185464833 wait complete. 2024-09-19T08:57:19.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.309+0000 7f0afd32b740 1 -- start start 2024-09-19T08:57:19.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.309+0000 7f0afd32b740 1 Processor -- start 2024-09-19T08:57:19.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.309+0000 7f0afd32b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc730f5a00 0x55cc731004f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:19.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.309+0000 7f0afd32b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc730f6320 0x55cc730fd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:19.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.309+0000 7f0afd32b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cc73100be0 0x55cc730fdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:19.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.309+0000 7f0afd32b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cc731037b0 con 0x55cc730f5a00 2024-09-19T08:57:19.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.309+0000 7f0afd32b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55cc72ff4c50 con 0x55cc73100be0 2024-09-19T08:57:19.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.309+0000 7f0afd32b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55cc73103930 con 0x55cc730f6320 2024-09-19T08:57:19.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.310+0000 7f0afd32b740 10 client.0 _ll_register_callbacks cb 0x55cc73006ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:57:19.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.310+0000 7f0af8cee640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc730f5a00 0x55cc731004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:19.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.310+0000 7f0af8cee640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc730f5a00 0x55cc731004f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51228/0 (socket says 172.21.15.164:51228) 2024-09-19T08:57:19.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.310+0000 7f0af8cee640 1 -- 172.21.15.164:0/3404321617 learned_addr learned my addr 172.21.15.164:0/3404321617 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:19.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.310+0000 7f0af94ef640 1 --2- 172.21.15.164:0/3404321617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cc73100be0 0x55cc730fdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:19.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.310+0000 7f0af3fff640 1 --2- 172.21.15.164:0/3404321617 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc730f6320 0x55cc730fd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:19.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.310+0000 7f0af8cee640 1 -- 172.21.15.164:0/3404321617 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc730f6320 msgr2=0x55cc730fd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:19.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.310+0000 7f0af8cee640 1 --2- 172.21.15.164:0/3404321617 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc730f6320 0x55cc730fd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:19.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.310+0000 7f0af8cee640 1 -- 172.21.15.164:0/3404321617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cc73100be0 msgr2=0x55cc730fdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:19.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.310+0000 7f0af8cee640 1 --2- 172.21.15.164:0/3404321617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cc73100be0 0x55cc730fdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:19.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.310+0000 7f0af8cee640 1 -- 172.21.15.164:0/3404321617 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc730fe4e0 con 0x55cc730f5a00 2024-09-19T08:57:19.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.310+0000 7f0af3fff640 1 --2- 172.21.15.164:0/3404321617 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc730f6320 0x55cc730fd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:57:19.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.310+0000 7f0af8cee640 1 --2- 172.21.15.164:0/3404321617 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc730f5a00 0x55cc731004f0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800f8c0 tx=0x7f0ae8005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bfaf8dba3e4f25c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:19.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.310+0000 7f0ad3fff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:57:19.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.310+0000 7f0ad3fff640 1 -- 172.21.15.164:0/3404321617 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae800f440 con 0x55cc730f5a00 2024-09-19T08:57:19.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.310+0000 7f0ad3fff640 1 -- 172.21.15.164:0/3404321617 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7f0ae800fa20 con 0x55cc730f5a00 2024-09-19T08:57:19.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.310+0000 7f0afd32b740 1 -- 172.21.15.164:0/3404321617 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ae8012ad0 con 0x55cc730f5a00 2024-09-19T08:57:19.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.311+0000 7f0ad3fff640 1 -- 172.21.15.164:0/3404321617 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0ae800fc20 con 0x55cc730f5a00 2024-09-19T08:57:19.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.311+0000 7f0afd32b740 10 client.17982 fetch_fsmap learned FSMap version 33 2024-09-19T08:57:19.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.311+0000 7f0afd32b740 1 -- 172.21.15.164:0/3404321617 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55cc7310f300 con 0x55cc730f5a00 2024-09-19T08:57:19.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.311+0000 7f0ad3fff640 1 -- 172.21.15.164:0/3404321617 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ae8012ad0 con 0x55cc730f5a00 2024-09-19T08:57:19.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.311+0000 7f0ad3fff640 1 -- 172.21.15.164:0/3404321617 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 33) ==== 8366+0+0 (secure 0 0 0) 0x7f0ae8005350 con 0x55cc730f5a00 2024-09-19T08:57:19.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.311+0000 7f0afd32b740 10 client.17982 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T08:57:19.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.311+0000 7f0afd32b740 10 client.17982 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:57:19.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.311+0000 7f0afd32b740 20 client.17982 populate_metadata read hostname 'smithi164' 2024-09-19T08:57:19.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.311+0000 7f0afd32b740 1 --2- 172.21.15.164:0/3404321617 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55cc73110760 0x55cc73112b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:19.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.311+0000 7f0afd32b740 4 client.17982 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:57:19.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.311+0000 7f0afd32b740 1 -- 172.21.15.164:0/3404321617 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc73067770 con 0x55cc73110760 2024-09-19T08:57:19.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.311+0000 7f0af3fff640 1 --2- 172.21.15.164:0/3404321617 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55cc73110760 0x55cc73112b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:19.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.312+0000 7f0af3fff640 1 --2- 172.21.15.164:0/3404321617 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55cc73110760 0x55cc73112b50 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eacfcbfcb576ef7b server_cookie=b8433006a5024139 in_seq=0 out_seq=0 2024-09-19T08:57:19.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.312+0000 7f0ad3fff640 10 client.17982 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:57:19.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.312+0000 7f0ad3fff640 1 -- 172.21.15.164:0/3404321617 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cc73067770 con 0x55cc73110760 2024-09-19T08:57:19.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.312+0000 7f0ad3fff640 10 client.17982 handle_command_reply: tid=0 2024-09-19T08:57:19.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.312+0000 7f0afd32b740 1 client.17982 shutdown 2024-09-19T08:57:19.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.313+0000 7f0afd32b740 1 -- 172.21.15.164:0/3404321617 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc730f5a00 msgr2=0x55cc731004f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:19.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.313+0000 7f0afd32b740 1 --2- 172.21.15.164:0/3404321617 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc730f5a00 0x55cc731004f0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800f8c0 tx=0x7f0ae8005710 comp rx=0 tx=0).stop 2024-09-19T08:57:19.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.313+0000 7f0afd32b740 1 -- 172.21.15.164:0/3404321617 shutdown_connections 2024-09-19T08:57:19.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.313+0000 7f0afd32b740 1 --2- 172.21.15.164:0/3404321617 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55cc73110760 0x55cc73112b50 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:19.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.313+0000 7f0afd32b740 1 --2- 172.21.15.164:0/3404321617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cc73100be0 0x55cc730fdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:19.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.313+0000 7f0ad3fff640 0 client.17982 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:57:19.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.313+0000 7f0afd32b740 1 --2- 172.21.15.164:0/3404321617 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc730f6320 0x55cc730fd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:19.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.313+0000 7f0afd32b740 1 --2- 172.21.15.164:0/3404321617 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc730f5a00 0x55cc731004f0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:19.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.313+0000 7f0afd32b740 1 -- 172.21.15.164:0/3404321617 >> 172.21.15.164:0/3404321617 conn(0x55cc730f1f90 msgr2=0x55cc730f11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:19.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.313+0000 7f0afd32b740 1 -- 172.21.15.164:0/3404321617 shutdown_connections 2024-09-19T08:57:19.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.313+0000 7f0afd32b740 1 -- 172.21.15.164:0/3404321617 wait complete. 2024-09-19T08:57:19.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.313+0000 7f0afd32b740 20 client.17982 trim_cache size 0 max 16384 2024-09-19T08:57:19.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.329+0000 7f0afd32b740 1 Processor -- start 2024-09-19T08:57:19.330 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.329+0000 7f0afd32b740 1 -- start start 2024-09-19T08:57:19.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.330+0000 7f0afd32b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc730f5a00 0x55cc7310ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:19.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.330+0000 7f0afd32b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc730f6320 0x55cc731180f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:19.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.330+0000 7f0afd32b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cc73100be0 0x55cc73118630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:19.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.330+0000 7f0afd32b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0af41ab540 con 0x55cc730f6320 2024-09-19T08:57:19.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.330+0000 7f0afd32b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0af41ad0b0 con 0x55cc73100be0 2024-09-19T08:57:19.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.330+0000 7f0afd32b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0af41abc10 con 0x55cc730f5a00 2024-09-19T08:57:19.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.330+0000 7f0af3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc730f6320 0x55cc731180f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:19.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.330+0000 7f0af3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc730f6320 0x55cc731180f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51238/0 (socket says 172.21.15.164:51238) 2024-09-19T08:57:19.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.330+0000 7f0af3fff640 1 -- 172.21.15.164:0/1578764640 learned_addr learned my addr 172.21.15.164:0/1578764640 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:19.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.330+0000 7f0af8cee640 1 --2- 172.21.15.164:0/1578764640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc730f5a00 0x55cc7310ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:19.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.330+0000 7f0af94ef640 1 --2- 172.21.15.164:0/1578764640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cc73100be0 0x55cc73118630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:19.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.330+0000 7f0af3fff640 1 -- 172.21.15.164:0/1578764640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc730f5a00 msgr2=0x55cc7310ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:19.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.330+0000 7f0af3fff640 1 --2- 172.21.15.164:0/1578764640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc730f5a00 0x55cc7310ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:19.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.330+0000 7f0af3fff640 1 -- 172.21.15.164:0/1578764640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cc73100be0 msgr2=0x55cc73118630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:19.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.330+0000 7f0af3fff640 1 --2- 172.21.15.164:0/1578764640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cc73100be0 0x55cc73118630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:19.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.330+0000 7f0af3fff640 1 -- 172.21.15.164:0/1578764640 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc73112ec0 con 0x55cc730f6320 2024-09-19T08:57:19.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.330+0000 7f0af3fff640 1 --2- 172.21.15.164:0/1578764640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc730f6320 0x55cc731180f0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f0ae009b9c0 tx=0x7f0ae00d1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99871d428509e5bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:19.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.330+0000 7f0ade7fc640 1 -- 172.21.15.164:0/1578764640 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae00b4070 con 0x55cc730f6320 2024-09-19T08:57:19.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.330+0000 7f0ade7fc640 1 -- 172.21.15.164:0/1578764640 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ae009fde0 con 0x55cc730f6320 2024-09-19T08:57:19.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.330+0000 7f0ade7fc640 1 -- 172.21.15.164:0/1578764640 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae00860b0 con 0x55cc730f6320 2024-09-19T08:57:19.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.331+0000 7f0afd32b740 1 -- 172.21.15.164:0/1578764640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc730f6320 msgr2=0x55cc731180f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:19.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.331+0000 7f0afd32b740 1 --2- 172.21.15.164:0/1578764640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc730f6320 0x55cc731180f0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f0ae009b9c0 tx=0x7f0ae00d1040 comp rx=0 tx=0).stop 2024-09-19T08:57:19.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.331+0000 7f0afd32b740 1 -- 172.21.15.164:0/1578764640 shutdown_connections 2024-09-19T08:57:19.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.331+0000 7f0afd32b740 1 --2- 172.21.15.164:0/1578764640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cc73100be0 0x55cc73118630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:19.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.331+0000 7f0afd32b740 1 --2- 172.21.15.164:0/1578764640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc730f6320 0x55cc731180f0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:19.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.331+0000 7f0afd32b740 1 --2- 172.21.15.164:0/1578764640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc730f5a00 0x55cc7310ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:19.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.331+0000 7f0afd32b740 1 -- 172.21.15.164:0/1578764640 >> 172.21.15.164:0/1578764640 conn(0x55cc730f1f90 msgr2=0x55cc730fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:19.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.331+0000 7f0afd32b740 1 -- 172.21.15.164:0/1578764640 shutdown_connections 2024-09-19T08:57:19.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.331+0000 7f0afd32b740 1 -- 172.21.15.164:0/1578764640 wait complete. 2024-09-19T08:57:19.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.331+0000 7f0afd32b740 1 -- start start 2024-09-19T08:57:19.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.331+0000 7f0afd32b740 1 Processor -- start 2024-09-19T08:57:19.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.332+0000 7f0afd32b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc730f5a00 0x55cc73108f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:19.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.332+0000 7f0afd32b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cc730f6320 0x55cc73109460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:19.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.332+0000 7f0afd32b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc73100be0 0x55cc73106470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:19.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.332+0000 7f0afd32b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0af41ac9e0 con 0x55cc730f5a00 2024-09-19T08:57:19.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.332+0000 7f0afd32b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0af41ac2e0 con 0x55cc730f6320 2024-09-19T08:57:19.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.332+0000 7f0afd32b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0af41a9540 con 0x55cc73100be0 2024-09-19T08:57:19.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.332+0000 7f0afd32b740 10 client.0 _ll_register_callbacks cb 0x7f0ae8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:57:19.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.332+0000 7f0af8cee640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc730f5a00 0x55cc73108f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:19.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.332+0000 7f0af8cee640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc730f5a00 0x55cc73108f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51252/0 (socket says 172.21.15.164:51252) 2024-09-19T08:57:19.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.332+0000 7f0af8cee640 1 -- 172.21.15.164:0/4174050467 learned_addr learned my addr 172.21.15.164:0/4174050467 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:19.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.332+0000 7f0af94ef640 1 --2- 172.21.15.164:0/4174050467 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc73100be0 0x55cc73106470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:19.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.332+0000 7f0af3fff640 1 --2- 172.21.15.164:0/4174050467 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cc730f6320 0x55cc73109460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:19.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.332+0000 7f0af8cee640 1 -- 172.21.15.164:0/4174050467 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc73100be0 msgr2=0x55cc73106470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:19.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.332+0000 7f0af8cee640 1 --2- 172.21.15.164:0/4174050467 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc73100be0 0x55cc73106470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:19.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.332+0000 7f0af8cee640 1 -- 172.21.15.164:0/4174050467 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cc730f6320 msgr2=0x55cc73109460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:19.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.332+0000 7f0af8cee640 1 --2- 172.21.15.164:0/4174050467 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cc730f6320 0x55cc73109460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:19.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.332+0000 7f0af8cee640 1 -- 172.21.15.164:0/4174050467 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc73106ca0 con 0x55cc730f5a00 2024-09-19T08:57:19.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.332+0000 7f0af94ef640 1 --2- 172.21.15.164:0/4174050467 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc73100be0 0x55cc73106470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:57:19.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.333+0000 7f0af8cee640 1 --2- 172.21.15.164:0/4174050467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc730f5a00 0x55cc73108f20 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800a010 tx=0x7f0ae8004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4b1b7646c905 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:19.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.333+0000 7f0ad3fff640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:57:19.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.333+0000 7f0ad3fff640 1 -- 172.21.15.164:0/4174050467 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8041060 con 0x55cc730f5a00 2024-09-19T08:57:19.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.333+0000 7f0ad3fff640 1 -- 172.21.15.164:0/4174050467 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7f0ae800f630 con 0x55cc730f5a00 2024-09-19T08:57:19.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.333+0000 7f0afd32b740 1 -- 172.21.15.164:0/4174050467 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ae009fde0 con 0x55cc730f5a00 2024-09-19T08:57:19.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.333+0000 7f0ad3fff640 1 -- 172.21.15.164:0/4174050467 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0ae8038040 con 0x55cc730f5a00 2024-09-19T08:57:19.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.333+0000 7f0afd32b740 10 client.17988 fetch_fsmap learned FSMap version 33 2024-09-19T08:57:19.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.333+0000 7f0afd32b740 1 -- 172.21.15.164:0/4174050467 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55cc7317ad10 con 0x55cc730f5a00 2024-09-19T08:57:19.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.333+0000 7f0ad3fff640 1 -- 172.21.15.164:0/4174050467 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ae009fde0 con 0x55cc730f5a00 2024-09-19T08:57:19.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.333+0000 7f0ad3fff640 1 -- 172.21.15.164:0/4174050467 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 33) ==== 8366+0+0 (secure 0 0 0) 0x7f0ae8004c30 con 0x55cc730f5a00 2024-09-19T08:57:19.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.333+0000 7f0afd32b740 10 client.17988 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T08:57:19.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.333+0000 7f0afd32b740 10 client.17988 resolve_mds: validated gid 24479 aka daemon mds.b 2024-09-19T08:57:19.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.333+0000 7f0afd32b740 20 client.17988 populate_metadata read hostname 'smithi164' 2024-09-19T08:57:19.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.333+0000 7f0afd32b740 1 --2- 172.21.15.164:0/4174050467 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55cc7317b3e0 0x55cc7317d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:19.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.333+0000 7f0afd32b740 4 client.17988 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-19T08:57:19.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.333+0000 7f0afd32b740 1 -- 172.21.15.164:0/4174050467 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55cc73061c80 con 0x55cc7317b3e0 2024-09-19T08:57:19.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.334+0000 7f0af3fff640 1 --2- 172.21.15.164:0/4174050467 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55cc7317b3e0 0x55cc7317d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:19.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.334+0000 7f0ad3fff640 10 client.17988 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:57:19.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.334+0000 7f0af3fff640 1 --2- 172.21.15.164:0/4174050467 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55cc7317b3e0 0x55cc7317d7d0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79be41d38cb006c2 server_cookie=9852ea38f5d54744 in_seq=0 out_seq=0 2024-09-19T08:57:19.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.334+0000 7f0ad3fff640 1 -- 172.21.15.164:0/4174050467 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55cc73061c80 con 0x55cc7317b3e0 2024-09-19T08:57:19.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.334+0000 7f0ad3fff640 10 client.17988 handle_command_reply: tid=0 2024-09-19T08:57:19.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.334+0000 7f0afd32b740 1 client.17988 shutdown 2024-09-19T08:57:19.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.334+0000 7f0afd32b740 1 -- 172.21.15.164:0/4174050467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc730f5a00 msgr2=0x55cc73108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:19.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.334+0000 7f0afd32b740 1 --2- 172.21.15.164:0/4174050467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc730f5a00 0x55cc73108f20 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800a010 tx=0x7f0ae8004b40 comp rx=0 tx=0).stop 2024-09-19T08:57:19.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.335+0000 7f0afd32b740 1 -- 172.21.15.164:0/4174050467 shutdown_connections 2024-09-19T08:57:19.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.335+0000 7f0afd32b740 1 --2- 172.21.15.164:0/4174050467 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc73100be0 0x55cc73106470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:19.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.335+0000 7f0afd32b740 1 --2- 172.21.15.164:0/4174050467 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x55cc7317b3e0 0x55cc7317d7d0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:19.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.335+0000 7f0afd32b740 1 --2- 172.21.15.164:0/4174050467 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cc730f6320 0x55cc73109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:19.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.335+0000 7f0afd32b740 1 --2- 172.21.15.164:0/4174050467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cc730f5a00 0x55cc73108f20 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:19.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.335+0000 7f0ad3fff640 0 client.17988 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:57:19.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.335+0000 7f0afd32b740 1 -- 172.21.15.164:0/4174050467 >> 172.21.15.164:0/4174050467 conn(0x55cc730f1f90 msgr2=0x55cc730f09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:19.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.335+0000 7f0afd32b740 1 -- 172.21.15.164:0/4174050467 shutdown_connections 2024-09-19T08:57:19.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.335+0000 7f0afd32b740 1 -- 172.21.15.164:0/4174050467 wait complete. 2024-09-19T08:57:19.341 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.335+0000 7f0afd32b740 20 client.17988 trim_cache size 0 max 16384 2024-09-19T08:57:19.341 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:57:19.341 INFO:teuthology.orchestra.run.smithi164.stdout: "epoch": 33, 2024-09-19T08:57:19.341 INFO:teuthology.orchestra.run.smithi164.stdout: "leader": 24479, 2024-09-19T08:57:19.341 INFO:teuthology.orchestra.run.smithi164.stdout: "set_version": 40, 2024-09-19T08:57:19.341 INFO:teuthology.orchestra.run.smithi164.stdout: "sets": {} 2024-09-19T08:57:19.341 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:57:19.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.336+0000 7f0afaf52640 1 -- 172.21.15.164:0/2901731396 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f0ae40a33b0 msgr2=0x7f0ae40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:19.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.336+0000 7f0afaf52640 1 --2- 172.21.15.164:0/2901731396 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f0ae40a33b0 0x7f0ae40a5870 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0aec004ab0 tx=0x7f0aec004a40 comp rx=0 tx=0).stop 2024-09-19T08:57:19.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.336+0000 7f0afaf52640 1 -- 172.21.15.164:0/2901731396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0af4101420 msgr2=0x7f0af4197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:19.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.336+0000 7f0afaf52640 1 --2- 172.21.15.164:0/2901731396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0af4101420 0x7f0af4197820 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f0ae000d770 tx=0x7f0ae000dc30 comp rx=0 tx=0).stop 2024-09-19T08:57:19.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.336+0000 7f0afaf52640 1 -- 172.21.15.164:0/2901731396 shutdown_connections 2024-09-19T08:57:19.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.336+0000 7f0afaf52640 1 --2- 172.21.15.164:0/2901731396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0af4106a80 0x7f0af419bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:19.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.336+0000 7f0afaf52640 1 --2- 172.21.15.164:0/2901731396 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f0ae40a33b0 0x7f0ae40a5870 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:19.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.336+0000 7f0afaf52640 1 --2- 172.21.15.164:0/2901731396 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0af4101420 0x7f0af4197820 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:19.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.336+0000 7f0afaf52640 1 --2- 172.21.15.164:0/2901731396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0af4100a80 0x7f0af41972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:19.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.336+0000 7f0afaf52640 1 -- 172.21.15.164:0/2901731396 >> 172.21.15.164:0/2901731396 conn(0x7f0af40fc7a0 msgr2=0x7f0af40fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:19.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.336+0000 7f0afaf52640 1 -- 172.21.15.164:0/2901731396 shutdown_connections 2024-09-19T08:57:19.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:19.336+0000 7f0afaf52640 1 -- 172.21.15.164:0/2901731396 wait complete. 2024-09-19T08:57:20.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:19 smithi164 ceph-mon[30928]: pgmap v360: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-19T08:57:20.213 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:19 smithi173 ceph-mon[35306]: pgmap v360: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-19T08:57:20.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:19 smithi170 ceph-mon[35072]: pgmap v360: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-19T08:57:21.184 INFO:tasks.quiescer:done joining Quiescers 2024-09-19T08:57:21.184 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-19T08:57:21.195 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-19T08:57:21.195 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T08:57:21.195 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi164.front.sepia.ceph.com 2024-09-19T08:57:21.195 DEBUG:teuthology.orchestra.run.smithi164:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-19T08:57:21.247 DEBUG:teuthology.orchestra.run.smithi164:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-19T08:57:21.277 DEBUG:teuthology.orchestra.run.smithi164:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-19T08:57:21.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.438+0000 7fcf3e1fd640 1 -- 172.21.15.164:0/3199701477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf381009f0 msgr2=0x7fcf38100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:21.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.438+0000 7fcf3e1fd640 1 --2- 172.21.15.164:0/3199701477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf381009f0 0x7fcf38100e50 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c009930 tx=0x7fcf2c02a420 comp rx=0 tx=0).stop 2024-09-19T08:57:21.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.438+0000 7fcf3e1fd640 1 -- 172.21.15.164:0/3199701477 shutdown_connections 2024-09-19T08:57:21.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.438+0000 7fcf3e1fd640 1 --2- 172.21.15.164:0/3199701477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf38101390 0x7fcf3810d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:21.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.438+0000 7fcf3e1fd640 1 --2- 172.21.15.164:0/3199701477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf381009f0 0x7fcf38100e50 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:21.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.438+0000 7fcf3e1fd640 1 --2- 172.21.15.164:0/3199701477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf38106a00 0x7fcf38106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:21.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.438+0000 7fcf3e1fd640 1 -- 172.21.15.164:0/3199701477 >> 172.21.15.164:0/3199701477 conn(0x7fcf380fc6b0 msgr2=0x7fcf380fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:21.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.438+0000 7fcf3e1fd640 1 -- 172.21.15.164:0/3199701477 shutdown_connections 2024-09-19T08:57:21.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.438+0000 7fcf3e1fd640 1 -- 172.21.15.164:0/3199701477 wait complete. 2024-09-19T08:57:21.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.438+0000 7fcf3e1fd640 1 Processor -- start 2024-09-19T08:57:21.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.438+0000 7fcf3e1fd640 1 -- start start 2024-09-19T08:57:21.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.439+0000 7fcf3e1fd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf381009f0 0x7fcf3819b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:21.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.439+0000 7fcf3e1fd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf38101390 0x7fcf3819b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:21.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.439+0000 7fcf3e1fd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf38106a00 0x7fcf381a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:21.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.439+0000 7fcf3e1fd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3806b200 con 0x7fcf38106a00 2024-09-19T08:57:21.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.439+0000 7fcf3e1fd640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3806b080 con 0x7fcf381009f0 2024-09-19T08:57:21.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.439+0000 7fcf3e1fd640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3810ff00 con 0x7fcf38101390 2024-09-19T08:57:21.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.439+0000 7fcf37fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf38106a00 0x7fcf381a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:21.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.439+0000 7fcf37fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf38106a00 0x7fcf381a62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51266/0 (socket says 172.21.15.164:51266) 2024-09-19T08:57:21.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.439+0000 7fcf37fff640 1 -- 172.21.15.164:0/2983380568 learned_addr learned my addr 172.21.15.164:0/2983380568 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:21.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.439+0000 7fcf377fe640 1 --2- 172.21.15.164:0/2983380568 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf381009f0 0x7fcf3819b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:21.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.439+0000 7fcf37fff640 1 -- 172.21.15.164:0/2983380568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf38101390 msgr2=0x7fcf3819b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:21.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.439+0000 7fcf37fff640 1 --2- 172.21.15.164:0/2983380568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf38101390 0x7fcf3819b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:21.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.439+0000 7fcf37fff640 1 -- 172.21.15.164:0/2983380568 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf381009f0 msgr2=0x7fcf3819b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:21.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.439+0000 7fcf37fff640 1 --2- 172.21.15.164:0/2983380568 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf381009f0 0x7fcf3819b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:21.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.439+0000 7fcf37fff640 1 -- 172.21.15.164:0/2983380568 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf381a6820 con 0x7fcf38106a00 2024-09-19T08:57:21.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.439+0000 7fcf377fe640 1 --2- 172.21.15.164:0/2983380568 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf381009f0 0x7fcf3819b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:57:21.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.439+0000 7fcf37fff640 1 --2- 172.21.15.164:0/2983380568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf38106a00 0x7fcf381a62b0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fcf3000cad0 tx=0x7fcf3000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:21.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.439+0000 7fcf34ff9640 1 -- 172.21.15.164:0/2983380568 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3000ae20 con 0x7fcf38106a00 2024-09-19T08:57:21.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.439+0000 7fcf34ff9640 1 -- 172.21.15.164:0/2983380568 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf30005020 con 0x7fcf38106a00 2024-09-19T08:57:21.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.439+0000 7fcf3e1fd640 1 -- 172.21.15.164:0/2983380568 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf381a6b10 con 0x7fcf38106a00 2024-09-19T08:57:21.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.439+0000 7fcf34ff9640 1 -- 172.21.15.164:0/2983380568 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3000f6f0 con 0x7fcf38106a00 2024-09-19T08:57:21.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.439+0000 7fcf3e1fd640 1 -- 172.21.15.164:0/2983380568 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf381a7010 con 0x7fcf38106a00 2024-09-19T08:57:21.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.441+0000 7fcf3e1fd640 1 -- 172.21.15.164:0/2983380568 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf38100e50 con 0x7fcf38106a00 2024-09-19T08:57:21.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.442+0000 7fcf34ff9640 1 -- 172.21.15.164:0/2983380568 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fcf300051d0 con 0x7fcf38106a00 2024-09-19T08:57:21.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.442+0000 7fcf34ff9640 1 --2- 172.21.15.164:0/2983380568 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fcf200a3460 0x7fcf200a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:21.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.442+0000 7fcf377fe640 1 --2- 172.21.15.164:0/2983380568 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fcf200a3460 0x7fcf200a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:21.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.442+0000 7fcf34ff9640 1 -- 172.21.15.164:0/2983380568 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7fcf300cfe50 con 0x7fcf38106a00 2024-09-19T08:57:21.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.443+0000 7fcf377fe640 1 --2- 172.21.15.164:0/2983380568 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fcf200a3460 0x7fcf200a5920 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcf28005e10 tx=0x7fcf28005da0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:21.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.447+0000 7fcf34ff9640 1 -- 172.21.15.164:0/2983380568 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcf30014030 con 0x7fcf38106a00 2024-09-19T08:57:21.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.633+0000 7fcf3e1fd640 1 -- 172.21.15.164:0/2983380568 --> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fcf38062630 con 0x7fcf200a3460 2024-09-19T08:57:21.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.641+0000 7fcf34ff9640 1 -- 172.21.15.164:0/2983380568 <== mgr.15201 v2:172.21.15.164:6800/173486514 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7fcf38062630 con 0x7fcf200a3460 2024-09-19T08:57:21.641 INFO:teuthology.orchestra.run.smithi164.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-19T00:00:00", "created": "2024-09-19T08:45:56", "first": "2024-09-19T08:46:00", "last": "2024-09-19T08:57:00", "last_pruned": "2024-09-19T08:57:00", "created_count": 12, "pruned_count": 6, "active": true} 2024-09-19T08:57:21.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.643+0000 7fcf3e1fd640 1 -- 172.21.15.164:0/2983380568 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fcf200a3460 msgr2=0x7fcf200a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:21.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.643+0000 7fcf3e1fd640 1 --2- 172.21.15.164:0/2983380568 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fcf200a3460 0x7fcf200a5920 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcf28005e10 tx=0x7fcf28005da0 comp rx=0 tx=0).stop 2024-09-19T08:57:21.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.643+0000 7fcf3e1fd640 1 -- 172.21.15.164:0/2983380568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf38106a00 msgr2=0x7fcf381a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:21.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.643+0000 7fcf3e1fd640 1 --2- 172.21.15.164:0/2983380568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf38106a00 0x7fcf381a62b0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fcf3000cad0 tx=0x7fcf3000cf90 comp rx=0 tx=0).stop 2024-09-19T08:57:21.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.643+0000 7fcf3e1fd640 1 -- 172.21.15.164:0/2983380568 shutdown_connections 2024-09-19T08:57:21.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.643+0000 7fcf3e1fd640 1 --2- 172.21.15.164:0/2983380568 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fcf200a3460 0x7fcf200a5920 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:21.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.643+0000 7fcf3e1fd640 1 --2- 172.21.15.164:0/2983380568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf38106a00 0x7fcf381a62b0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:21.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.643+0000 7fcf3e1fd640 1 --2- 172.21.15.164:0/2983380568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf38101390 0x7fcf3819b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:21.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.643+0000 7fcf3e1fd640 1 --2- 172.21.15.164:0/2983380568 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf381009f0 0x7fcf3819b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:21.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.643+0000 7fcf3e1fd640 1 -- 172.21.15.164:0/2983380568 >> 172.21.15.164:0/2983380568 conn(0x7fcf380fc6b0 msgr2=0x7fcf38110a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:21.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.643+0000 7fcf3e1fd640 1 -- 172.21.15.164:0/2983380568 shutdown_connections 2024-09-19T08:57:21.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.643+0000 7fcf3e1fd640 1 -- 172.21.15.164:0/2983380568 wait complete. 2024-09-19T08:57:21.653 DEBUG:teuthology.orchestra.run.smithi164:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-19T08:57:21.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.756+0000 7f33db9c6640 1 -- 172.21.15.164:0/2489982157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33d41009d0 msgr2=0x7f33d4100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:21.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.756+0000 7f33db9c6640 1 --2- 172.21.15.164:0/2489982157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33d41009d0 0x7f33d4100e30 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f33c8009930 tx=0x7f33c802a420 comp rx=0 tx=0).stop 2024-09-19T08:57:21.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.756+0000 7f33db9c6640 1 -- 172.21.15.164:0/2489982157 shutdown_connections 2024-09-19T08:57:21.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.756+0000 7f33db9c6640 1 --2- 172.21.15.164:0/2489982157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33d4101370 0x7f33d410d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:21.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.756+0000 7f33db9c6640 1 --2- 172.21.15.164:0/2489982157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33d41009d0 0x7f33d4100e30 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:21.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.756+0000 7f33db9c6640 1 --2- 172.21.15.164:0/2489982157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33d41069e0 0x7f33d4106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:21.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.756+0000 7f33db9c6640 1 -- 172.21.15.164:0/2489982157 >> 172.21.15.164:0/2489982157 conn(0x7f33d40fc6f0 msgr2=0x7f33d40feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:21.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.756+0000 7f33db9c6640 1 -- 172.21.15.164:0/2489982157 shutdown_connections 2024-09-19T08:57:21.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.756+0000 7f33db9c6640 1 -- 172.21.15.164:0/2489982157 wait complete. 2024-09-19T08:57:21.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.757+0000 7f33db9c6640 1 Processor -- start 2024-09-19T08:57:21.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.757+0000 7f33db9c6640 1 -- start start 2024-09-19T08:57:21.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.757+0000 7f33db9c6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33d41009d0 0x7f33d4199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:21.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.757+0000 7f33db9c6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33d4101370 0x7f33d4199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:21.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.757+0000 7f33db9c6640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33d41069e0 0x7f33d41a41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:21.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.757+0000 7f33db9c6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f33d4110e20 con 0x7f33d4101370 2024-09-19T08:57:21.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.757+0000 7f33db9c6640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f33d4110ca0 con 0x7f33d41009d0 2024-09-19T08:57:21.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.757+0000 7f33db9c6640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f33d4110fa0 con 0x7f33d41069e0 2024-09-19T08:57:21.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.757+0000 7f33d8f61640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33d4101370 0x7f33d4199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:21.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.757+0000 7f33d8f61640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33d4101370 0x7f33d4199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:51284/0 (socket says 172.21.15.164:51284) 2024-09-19T08:57:21.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.757+0000 7f33d8f61640 1 -- 172.21.15.164:0/1477920293 learned_addr learned my addr 172.21.15.164:0/1477920293 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:21.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.757+0000 7f33d9762640 1 --2- 172.21.15.164:0/1477920293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33d41009d0 0x7f33d4199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:21.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.757+0000 7f33d9f63640 1 --2- 172.21.15.164:0/1477920293 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33d41069e0 0x7f33d41a41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:21.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.757+0000 7f33d8f61640 1 -- 172.21.15.164:0/1477920293 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33d41069e0 msgr2=0x7f33d41a41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:21.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.757+0000 7f33d8f61640 1 --2- 172.21.15.164:0/1477920293 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33d41069e0 0x7f33d41a41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:21.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.757+0000 7f33d8f61640 1 -- 172.21.15.164:0/1477920293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33d41009d0 msgr2=0x7f33d4199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:21.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.757+0000 7f33d8f61640 1 --2- 172.21.15.164:0/1477920293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33d41009d0 0x7f33d4199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:21.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.757+0000 7f33d8f61640 1 -- 172.21.15.164:0/1477920293 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d41a4710 con 0x7f33d4101370 2024-09-19T08:57:21.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.757+0000 7f33d9762640 1 --2- 172.21.15.164:0/1477920293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33d41009d0 0x7f33d4199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:57:21.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.757+0000 7f33d9f63640 1 --2- 172.21.15.164:0/1477920293 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33d41069e0 0x7f33d41a41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:57:21.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.757+0000 7f33d8f61640 1 --2- 172.21.15.164:0/1477920293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33d4101370 0x7f33d4199690 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f33c8009a50 tx=0x7f33c8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:21.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.757+0000 7f33d27fc640 1 -- 172.21.15.164:0/1477920293 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c803f030 con 0x7f33d4101370 2024-09-19T08:57:21.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.758+0000 7f33db9c6640 1 -- 172.21.15.164:0/1477920293 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33d41a48e0 con 0x7f33d4101370 2024-09-19T08:57:21.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.758+0000 7f33d27fc640 1 -- 172.21.15.164:0/1477920293 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33c8003ce0 con 0x7f33d4101370 2024-09-19T08:57:21.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.758+0000 7f33d27fc640 1 -- 172.21.15.164:0/1477920293 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c8035070 con 0x7f33d4101370 2024-09-19T08:57:21.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.758+0000 7f33db9c6640 1 -- 172.21.15.164:0/1477920293 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33d41a4d40 con 0x7f33d4101370 2024-09-19T08:57:21.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.758+0000 7f33db9c6640 1 -- 172.21.15.164:0/1477920293 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33d4102110 con 0x7f33d4101370 2024-09-19T08:57:21.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.760+0000 7f33d27fc640 1 -- 172.21.15.164:0/1477920293 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f33c8042030 con 0x7f33d4101370 2024-09-19T08:57:21.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.760+0000 7f33d27fc640 1 --2- 172.21.15.164:0/1477920293 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f33bc0a3460 0x7f33bc0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:21.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.760+0000 7f33d9762640 1 --2- 172.21.15.164:0/1477920293 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f33bc0a3460 0x7f33bc0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:21.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.760+0000 7f33d27fc640 1 -- 172.21.15.164:0/1477920293 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7f33c80f0780 con 0x7f33d4101370 2024-09-19T08:57:21.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.761+0000 7f33d9762640 1 --2- 172.21.15.164:0/1477920293 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f33bc0a3460 0x7f33bc0a5920 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f33cc005d90 tx=0x7f33cc005d20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:21.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.763+0000 7f33d27fc640 1 -- 172.21.15.164:0/1477920293 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f33c80bd1a0 con 0x7f33d4101370 2024-09-19T08:57:21.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.956+0000 7f33db9c6640 1 -- 172.21.15.164:0/1477920293 --> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f33d40fd460 con 0x7f33bc0a3460 2024-09-19T08:57:21.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.963+0000 7f33d27fc640 1 -- 172.21.15.164:0/1477920293 <== mgr.15201 v2:172.21.15.164:6800/173486514 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f33d40fd460 con 0x7f33bc0a3460 2024-09-19T08:57:21.976 INFO:teuthology.orchestra.run.smithi164.stdout:/ 1m 6m3h 2024-09-19T08:57:21.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.966+0000 7f33db9c6640 1 -- 172.21.15.164:0/1477920293 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f33bc0a3460 msgr2=0x7f33bc0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:21.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.966+0000 7f33db9c6640 1 --2- 172.21.15.164:0/1477920293 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f33bc0a3460 0x7f33bc0a5920 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f33cc005d90 tx=0x7f33cc005d20 comp rx=0 tx=0).stop 2024-09-19T08:57:21.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.966+0000 7f33db9c6640 1 -- 172.21.15.164:0/1477920293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33d4101370 msgr2=0x7f33d4199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:21.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.966+0000 7f33db9c6640 1 --2- 172.21.15.164:0/1477920293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33d4101370 0x7f33d4199690 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f33c8009a50 tx=0x7f33c8004270 comp rx=0 tx=0).stop 2024-09-19T08:57:21.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.966+0000 7f33db9c6640 1 -- 172.21.15.164:0/1477920293 shutdown_connections 2024-09-19T08:57:21.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.966+0000 7f33db9c6640 1 --2- 172.21.15.164:0/1477920293 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33d41069e0 0x7f33d41a41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:21.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.966+0000 7f33db9c6640 1 --2- 172.21.15.164:0/1477920293 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f33bc0a3460 0x7f33bc0a5920 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:21.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.966+0000 7f33db9c6640 1 --2- 172.21.15.164:0/1477920293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33d4101370 0x7f33d4199690 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:21.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.966+0000 7f33db9c6640 1 --2- 172.21.15.164:0/1477920293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33d41009d0 0x7f33d4199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:21.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.966+0000 7f33db9c6640 1 -- 172.21.15.164:0/1477920293 >> 172.21.15.164:0/1477920293 conn(0x7f33d40fc6f0 msgr2=0x7f33d40fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:21.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.966+0000 7f33db9c6640 1 -- 172.21.15.164:0/1477920293 shutdown_connections 2024-09-19T08:57:21.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:21.966+0000 7f33db9c6640 1 -- 172.21.15.164:0/1477920293 wait complete. 2024-09-19T08:57:21.979 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-19T08:57:22.244 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-19T08:57:22.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:21 smithi164 ceph-mon[30928]: pgmap v361: 97 pgs: 97 active+clean; 11 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 27 KiB/s wr, 213 op/s 2024-09-19T08:57:22.378 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:21 smithi170 ceph-mon[35072]: pgmap v361: 97 pgs: 97 active+clean; 11 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 27 KiB/s wr, 213 op/s 2024-09-19T08:57:22.413 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:21 smithi173 ceph-mon[35306]: pgmap v361: 97 pgs: 97 active+clean; 11 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 27 KiB/s wr, 213 op/s 2024-09-19T08:57:23.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:22 smithi164 ceph-mon[30928]: from='client.17994 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:57:23.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:22 smithi164 ceph-mon[30928]: from='client.18000 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:57:23.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:22 smithi170 ceph-mon[35072]: from='client.17994 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:57:23.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:22 smithi170 ceph-mon[35072]: from='client.18000 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:57:23.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:22 smithi173 ceph-mon[35306]: from='client.17994 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:57:23.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:22 smithi173 ceph-mon[35306]: from='client.18000 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T08:57:24.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:23 smithi164 ceph-mon[30928]: pgmap v362: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 27 KiB/s wr, 248 op/s 2024-09-19T08:57:24.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:23 smithi170 ceph-mon[35072]: pgmap v362: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 27 KiB/s wr, 248 op/s 2024-09-19T08:57:24.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:23 smithi173 ceph-mon[35306]: pgmap v362: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 27 KiB/s wr, 248 op/s 2024-09-19T08:57:26.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:25 smithi164 ceph-mon[30928]: pgmap v363: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 KiB/s wr, 271 op/s 2024-09-19T08:57:26.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:25 smithi170 ceph-mon[35072]: pgmap v363: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 KiB/s wr, 271 op/s 2024-09-19T08:57:26.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:25 smithi173 ceph-mon[35306]: pgmap v363: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 KiB/s wr, 271 op/s 2024-09-19T08:57:28.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:27 smithi164 ceph-mon[30928]: pgmap v364: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.4 KiB/s wr, 240 op/s 2024-09-19T08:57:28.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:27 smithi170 ceph-mon[35072]: pgmap v364: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.4 KiB/s wr, 240 op/s 2024-09-19T08:57:28.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:27 smithi173 ceph-mon[35306]: pgmap v364: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.4 KiB/s wr, 240 op/s 2024-09-19T08:57:30.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:29 smithi164 ceph-mon[30928]: pgmap v365: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.7 KiB/s wr, 228 op/s 2024-09-19T08:57:30.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:29 smithi170 ceph-mon[35072]: pgmap v365: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.7 KiB/s wr, 228 op/s 2024-09-19T08:57:30.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:29 smithi173 ceph-mon[35306]: pgmap v365: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.7 KiB/s wr, 228 op/s 2024-09-19T08:57:32.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:31 smithi164 ceph-mon[30928]: pgmap v366: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.7 KiB/s wr, 219 op/s 2024-09-19T08:57:32.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:31 smithi170 ceph-mon[35072]: pgmap v366: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.7 KiB/s wr, 219 op/s 2024-09-19T08:57:32.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:31 smithi173 ceph-mon[35306]: pgmap v366: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.7 KiB/s wr, 219 op/s 2024-09-19T08:57:34.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:33 smithi164 ceph-mon[30928]: pgmap v367: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.8 KiB/s wr, 61 op/s 2024-09-19T08:57:34.377 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:33 smithi170 ceph-mon[35072]: pgmap v367: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.8 KiB/s wr, 61 op/s 2024-09-19T08:57:34.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:33 smithi173 ceph-mon[35306]: pgmap v367: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.8 KiB/s wr, 61 op/s 2024-09-19T08:57:35.810 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T08:57:36.070 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:57:36.190 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:35 smithi164 ceph-mon[30928]: pgmap v368: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.8 KiB/s wr, 34 op/s 2024-09-19T08:57:36.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:35 smithi170 ceph-mon[35072]: pgmap v368: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.8 KiB/s wr, 34 op/s 2024-09-19T08:57:36.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:35 smithi173 ceph-mon[35306]: pgmap v368: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.8 KiB/s wr, 34 op/s 2024-09-19T08:57:37.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.650+0000 7fdb06897640 1 -- 172.21.15.164:0/3070179312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb00100a80 msgr2=0x7fdb00100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:37.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.650+0000 7fdb06897640 1 --2- 172.21.15.164:0/3070179312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb00100a80 0x7fdb00100ee0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009930 tx=0x7fdae802a420 comp rx=0 tx=0).stop 2024-09-19T08:57:37.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.650+0000 7fdb06897640 1 -- 172.21.15.164:0/3070179312 shutdown_connections 2024-09-19T08:57:37.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.650+0000 7fdb06897640 1 --2- 172.21.15.164:0/3070179312 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb00101420 0x7fdb0010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.650+0000 7fdb06897640 1 --2- 172.21.15.164:0/3070179312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb00100a80 0x7fdb00100ee0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.650+0000 7fdb06897640 1 --2- 172.21.15.164:0/3070179312 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb00106a80 0x7fdb00106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.650+0000 7fdb06897640 1 -- 172.21.15.164:0/3070179312 >> 172.21.15.164:0/3070179312 conn(0x7fdb000fc7a0 msgr2=0x7fdb000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:37.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.650+0000 7fdb06897640 1 -- 172.21.15.164:0/3070179312 shutdown_connections 2024-09-19T08:57:37.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.650+0000 7fdb06897640 1 -- 172.21.15.164:0/3070179312 wait complete. 2024-09-19T08:57:37.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.650+0000 7fdb06897640 1 Processor -- start 2024-09-19T08:57:37.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.650+0000 7fdb06897640 1 -- start start 2024-09-19T08:57:37.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.651+0000 7fdb06897640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb00100a80 0x7fdb00199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:37.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.651+0000 7fdb06897640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb00101420 0x7fdb00199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:37.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.651+0000 7fdb06897640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb00106a80 0x7fdb001a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:37.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.651+0000 7fdb06897640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdb00110f20 con 0x7fdb00100a80 2024-09-19T08:57:37.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.651+0000 7fdb06897640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdb00110da0 con 0x7fdb00106a80 2024-09-19T08:57:37.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.651+0000 7fdb06897640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdb001110a0 con 0x7fdb00101420 2024-09-19T08:57:37.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.651+0000 7fdafffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb00100a80 0x7fdb00199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:37.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.651+0000 7fdafffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb00100a80 0x7fdb00199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40878/0 (socket says 172.21.15.164:40878) 2024-09-19T08:57:37.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.651+0000 7fdafffff640 1 -- 172.21.15.164:0/4093545383 learned_addr learned my addr 172.21.15.164:0/4093545383 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:37.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.651+0000 7fdaf7fff640 1 --2- 172.21.15.164:0/4093545383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb00101420 0x7fdb00199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:37.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.651+0000 7fdafffff640 1 -- 172.21.15.164:0/4093545383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb00101420 msgr2=0x7fdb00199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:37.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.651+0000 7fdb04e34640 1 --2- 172.21.15.164:0/4093545383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb00106a80 0x7fdb001a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:37.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.651+0000 7fdafffff640 1 --2- 172.21.15.164:0/4093545383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb00101420 0x7fdb00199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.651+0000 7fdafffff640 1 -- 172.21.15.164:0/4093545383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb00106a80 msgr2=0x7fdb001a44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:37.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.651+0000 7fdafffff640 1 --2- 172.21.15.164:0/4093545383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb00106a80 0x7fdb001a44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.651+0000 7fdafffff640 1 -- 172.21.15.164:0/4093545383 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb001a49f0 con 0x7fdb00100a80 2024-09-19T08:57:37.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.651+0000 7fdaf7fff640 1 --2- 172.21.15.164:0/4093545383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb00101420 0x7fdb00199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:57:37.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.651+0000 7fdb04e34640 1 --2- 172.21.15.164:0/4093545383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb00106a80 0x7fdb001a44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:57:37.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.651+0000 7fdafffff640 1 --2- 172.21.15.164:0/4093545383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb00100a80 0x7fdb00199430 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000b4a0 tx=0x7fdaf000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:37.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.651+0000 7fdafdffb640 1 -- 172.21.15.164:0/4093545383 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0014070 con 0x7fdb00100a80 2024-09-19T08:57:37.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.651+0000 7fdafdffb640 1 -- 172.21.15.164:0/4093545383 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaf0010af0 con 0x7fdb00100a80 2024-09-19T08:57:37.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.651+0000 7fdb06897640 1 -- 172.21.15.164:0/4093545383 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb001a4bc0 con 0x7fdb00100a80 2024-09-19T08:57:37.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.651+0000 7fdafdffb640 1 -- 172.21.15.164:0/4093545383 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0017d10 con 0x7fdb00100a80 2024-09-19T08:57:37.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.651+0000 7fdb06897640 1 -- 172.21.15.164:0/4093545383 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb001a50d0 con 0x7fdb00100a80 2024-09-19T08:57:37.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.654+0000 7fdafdffb640 1 -- 172.21.15.164:0/4093545383 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fdaf0017070 con 0x7fdb00100a80 2024-09-19T08:57:37.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.654+0000 7fdb08c70740 1 Processor -- start 2024-09-19T08:57:37.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.654+0000 7fdb08c70740 1 -- start start 2024-09-19T08:57:37.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.654+0000 7fdb08c70740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd70d24a0 0x555bd70d2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:37.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.654+0000 7fdafdffb640 1 --2- 172.21.15.164:0/4093545383 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fdaec0a33b0 0x7fdaec0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:37.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.654+0000 7fdb08c70740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bd70d2dc0 0x555bd70dd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:37.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.654+0000 7fdb08c70740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bd70dd680 0x555bd70dfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:37.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.655+0000 7fdb08c70740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555bd70382e0 con 0x555bd70d24a0 2024-09-19T08:57:37.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.655+0000 7fdb08c70740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555bd6fdfce0 con 0x555bd70dd680 2024-09-19T08:57:37.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.655+0000 7fdb08c70740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555bd70342a0 con 0x555bd70d2dc0 2024-09-19T08:57:37.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.655+0000 7fdb04e34640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd70d24a0 0x555bd70d2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:37.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.655+0000 7fdb04e34640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd70d24a0 0x555bd70d2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40880/0 (socket says 172.21.15.164:40880) 2024-09-19T08:57:37.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.655+0000 7fdb04e34640 1 -- 172.21.15.164:0/2258914697 learned_addr learned my addr 172.21.15.164:0/2258914697 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:37.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.655+0000 7fdafffff640 1 --2- 172.21.15.164:0/4093545383 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fdaec0a33b0 0x7fdaec0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:37.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.655+0000 7fdafdffb640 1 -- 172.21.15.164:0/4093545383 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7fdaf00ceff0 con 0x7fdb00100a80 2024-09-19T08:57:37.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.655+0000 7fdaf7fff640 1 --2- 172.21.15.164:0/2258914697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bd70d2dc0 0x555bd70dd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:37.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.655+0000 7fdb04e34640 1 --2- 172.21.15.164:0/2258914697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bd70dd680 0x555bd70dfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:37.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.655+0000 7fdb04e34640 1 -- 172.21.15.164:0/2258914697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bd70d2dc0 msgr2=0x555bd70dd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:37.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.655+0000 7fdb04e34640 1 --2- 172.21.15.164:0/2258914697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bd70d2dc0 0x555bd70dd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.655+0000 7fdb04e34640 1 -- 172.21.15.164:0/2258914697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bd70dd680 msgr2=0x555bd70dfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:37.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.655+0000 7fdb04e34640 1 --2- 172.21.15.164:0/2258914697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bd70dd680 0x555bd70dfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.655+0000 7fdb04e34640 1 -- 172.21.15.164:0/2258914697 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bd6fc4440 con 0x555bd70d24a0 2024-09-19T08:57:37.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.655+0000 7fdafffff640 1 --2- 172.21.15.164:0/4093545383 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fdaec0a33b0 0x7fdaec0a5870 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00ccd80 tx=0x7fdaf00908b0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:37.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.655+0000 7fdb04e34640 1 --2- 172.21.15.164:0/2258914697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd70d24a0 0x555bd70d2880 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80138e0 tx=0x7fdaf8007970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d375427220ed93 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:37.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.655+0000 7fdaf67fc640 1 -- 172.21.15.164:0/2258914697 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf800ebf0 con 0x555bd70d24a0 2024-09-19T08:57:37.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.655+0000 7fdaf67fc640 1 -- 172.21.15.164:0/2258914697 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaf800ed90 con 0x555bd70d24a0 2024-09-19T08:57:37.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.655+0000 7fdaf67fc640 1 -- 172.21.15.164:0/2258914697 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8010620 con 0x555bd70d24a0 2024-09-19T08:57:37.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.656+0000 7fdb08c70740 1 -- 172.21.15.164:0/2258914697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd70d24a0 msgr2=0x555bd70d2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:37.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.656+0000 7fdb08c70740 1 --2- 172.21.15.164:0/2258914697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd70d24a0 0x555bd70d2880 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80138e0 tx=0x7fdaf8007970 comp rx=0 tx=0).stop 2024-09-19T08:57:37.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.656+0000 7fdb08c70740 1 -- 172.21.15.164:0/2258914697 shutdown_connections 2024-09-19T08:57:37.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.656+0000 7fdb08c70740 1 --2- 172.21.15.164:0/2258914697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bd70dd680 0x555bd70dfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.656+0000 7fdb08c70740 1 --2- 172.21.15.164:0/2258914697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bd70d2dc0 0x555bd70dd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.656+0000 7fdb08c70740 1 --2- 172.21.15.164:0/2258914697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd70d24a0 0x555bd70d2880 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.656+0000 7fdb08c70740 1 -- 172.21.15.164:0/2258914697 >> 172.21.15.164:0/2258914697 conn(0x555bd70cea30 msgr2=0x555bd70cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:37.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.656+0000 7fdb08c70740 1 -- 172.21.15.164:0/2258914697 shutdown_connections 2024-09-19T08:57:37.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.656+0000 7fdb08c70740 1 -- 172.21.15.164:0/2258914697 wait complete. 2024-09-19T08:57:37.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.657+0000 7fdb08c70740 1 -- start start 2024-09-19T08:57:37.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.657+0000 7fdb08c70740 1 Processor -- start 2024-09-19T08:57:37.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.657+0000 7fdb08c70740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bd70d24a0 0x555bd70dcf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:37.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.657+0000 7fdb08c70740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bd70d2dc0 0x555bd70da220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:37.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.657+0000 7fdb08c70740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd70dd680 0x555bd70da760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:37.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.657+0000 7fdb08c70740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555bd70e0250 con 0x555bd70dd680 2024-09-19T08:57:37.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.657+0000 7fdb08c70740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555bd6ff3050 con 0x555bd70d24a0 2024-09-19T08:57:37.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.657+0000 7fdb08c70740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555bd70e03d0 con 0x555bd70d2dc0 2024-09-19T08:57:37.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.657+0000 7fdb08c70740 10 client.0 _ll_register_callbacks cb 0x555bd7033160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:57:37.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.657+0000 7fdb04e34640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd70dd680 0x555bd70da760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:37.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.657+0000 7fdb04e34640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd70dd680 0x555bd70da760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40896/0 (socket says 172.21.15.164:40896) 2024-09-19T08:57:37.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.657+0000 7fdb04e34640 1 -- 172.21.15.164:0/367144309 learned_addr learned my addr 172.21.15.164:0/367144309 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:37.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.658+0000 7fdb04e34640 1 --2- 172.21.15.164:0/367144309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bd70d24a0 0x555bd70dcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:37.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.658+0000 7fdaf7fff640 1 --2- 172.21.15.164:0/367144309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bd70d2dc0 0x555bd70da220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:37.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.658+0000 7fdb04e34640 1 -- 172.21.15.164:0/367144309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bd70d2dc0 msgr2=0x555bd70da220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:37.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.658+0000 7fdb04e34640 1 --2- 172.21.15.164:0/367144309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bd70d2dc0 0x555bd70da220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.658+0000 7fdb04e34640 1 -- 172.21.15.164:0/367144309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bd70d24a0 msgr2=0x555bd70dcf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:37.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.658+0000 7fdb04e34640 1 --2- 172.21.15.164:0/367144309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bd70d24a0 0x555bd70dcf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.658+0000 7fdb04e34640 1 -- 172.21.15.164:0/367144309 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bd6da93c0 con 0x555bd70dd680 2024-09-19T08:57:37.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.658+0000 7fdb04e34640 1 --2- 172.21.15.164:0/367144309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd70dd680 0x555bd70da760 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8011e30 tx=0x7fdaf80078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fccb89e5ba6a63ed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:37.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.658+0000 7fdadf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:57:37.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.658+0000 7fdadf7fe640 1 -- 172.21.15.164:0/367144309 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf80115d0 con 0x555bd70dd680 2024-09-19T08:57:37.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.658+0000 7fdadf7fe640 1 -- 172.21.15.164:0/367144309 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7fdaf8011770 con 0x555bd70dd680 2024-09-19T08:57:37.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.658+0000 7fdb08c70740 1 -- 172.21.15.164:0/367144309 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaf800ed90 con 0x555bd70dd680 2024-09-19T08:57:37.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.658+0000 7fdadf7fe640 1 -- 172.21.15.164:0/367144309 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdaf8031050 con 0x555bd70dd680 2024-09-19T08:57:37.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.658+0000 7fdb08c70740 10 client.18012 fetch_fsmap learned FSMap version 33 2024-09-19T08:57:37.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.658+0000 7fdb08c70740 1 -- 172.21.15.164:0/367144309 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555bd70ebbc0 con 0x555bd70dd680 2024-09-19T08:57:37.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.658+0000 7fdadf7fe640 1 -- 172.21.15.164:0/367144309 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaf800ed90 con 0x555bd70dd680 2024-09-19T08:57:37.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.659+0000 7fdadf7fe640 1 -- 172.21.15.164:0/367144309 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 33) ==== 8366+0+0 (secure 0 0 0) 0x7fdaf80103c0 con 0x555bd70dd680 2024-09-19T08:57:37.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.659+0000 7fdb08c70740 10 client.18012 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T08:57:37.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.659+0000 7fdb08c70740 10 client.18012 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:57:37.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.659+0000 7fdb08c70740 20 client.18012 populate_metadata read hostname 'smithi164' 2024-09-19T08:57:37.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.659+0000 7fdb08c70740 1 --2- 172.21.15.164:0/367144309 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555bd70ed020 0x555bd70ef410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:37.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.659+0000 7fdb08c70740 4 client.18012 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:57:37.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.659+0000 7fdb08c70740 1 -- 172.21.15.164:0/367144309 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bd7045850 con 0x555bd70ed020 2024-09-19T08:57:37.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.659+0000 7fdaf7fff640 1 --2- 172.21.15.164:0/367144309 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555bd70ed020 0x555bd70ef410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:37.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.660+0000 7fdaf7fff640 1 --2- 172.21.15.164:0/367144309 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555bd70ed020 0x555bd70ef410 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d502535ec3592172 server_cookie=a801772e506bb6d1 in_seq=0 out_seq=0 2024-09-19T08:57:37.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.660+0000 7fdadf7fe640 10 client.18012 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:57:37.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.660+0000 7fdadf7fe640 1 -- 172.21.15.164:0/367144309 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555bd7045850 con 0x555bd70ed020 2024-09-19T08:57:37.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.660+0000 7fdadf7fe640 10 client.18012 handle_command_reply: tid=0 2024-09-19T08:57:37.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.660+0000 7fdb08c70740 1 client.18012 shutdown 2024-09-19T08:57:37.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.660+0000 7fdb08c70740 1 -- 172.21.15.164:0/367144309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd70dd680 msgr2=0x555bd70da760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:37.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.660+0000 7fdb08c70740 1 --2- 172.21.15.164:0/367144309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd70dd680 0x555bd70da760 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8011e30 tx=0x7fdaf80078d0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.660+0000 7fdb08c70740 1 -- 172.21.15.164:0/367144309 shutdown_connections 2024-09-19T08:57:37.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.660+0000 7fdb08c70740 1 --2- 172.21.15.164:0/367144309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd70dd680 0x555bd70da760 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.660+0000 7fdb08c70740 1 --2- 172.21.15.164:0/367144309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bd70d2dc0 0x555bd70da220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.660+0000 7fdb08c70740 1 --2- 172.21.15.164:0/367144309 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555bd70ed020 0x555bd70ef410 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.661+0000 7fdb08c70740 1 --2- 172.21.15.164:0/367144309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bd70d24a0 0x555bd70dcf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.661+0000 7fdadf7fe640 0 client.18012 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:57:37.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.661+0000 7fdb08c70740 1 -- 172.21.15.164:0/367144309 >> 172.21.15.164:0/367144309 conn(0x555bd70cea30 msgr2=0x555bd70d4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:37.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.661+0000 7fdb08c70740 1 -- 172.21.15.164:0/367144309 shutdown_connections 2024-09-19T08:57:37.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.661+0000 7fdb08c70740 1 -- 172.21.15.164:0/367144309 wait complete. 2024-09-19T08:57:37.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.661+0000 7fdb08c70740 20 client.18012 trim_cache size 0 max 16384 2024-09-19T08:57:37.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.678+0000 7fdb08c70740 1 Processor -- start 2024-09-19T08:57:37.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.678+0000 7fdb08c70740 1 -- start start 2024-09-19T08:57:37.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.678+0000 7fdb08c70740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bd70d24a0 0x555bd70d9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:37.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.678+0000 7fdb08c70740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd70d2dc0 0x555bd70f2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:37.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.678+0000 7fdb08c70740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bd70dd680 0x555bd70f2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:37.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.678+0000 7fdb08c70740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdb001ad9e0 con 0x555bd70d2dc0 2024-09-19T08:57:37.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.678+0000 7fdb08c70740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdb001af550 con 0x555bd70dd680 2024-09-19T08:57:37.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.678+0000 7fdb08c70740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdb001ae0b0 con 0x555bd70d24a0 2024-09-19T08:57:37.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.678+0000 7fdaf7fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd70d2dc0 0x555bd70f2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:37.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.678+0000 7fdaf7fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd70d2dc0 0x555bd70f2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40900/0 (socket says 172.21.15.164:40900) 2024-09-19T08:57:37.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.678+0000 7fdaf7fff640 1 -- 172.21.15.164:0/2969555837 learned_addr learned my addr 172.21.15.164:0/2969555837 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:37.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.678+0000 7fdb04e34640 1 --2- 172.21.15.164:0/2969555837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bd70dd680 0x555bd70f2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:37.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.678+0000 7fdb04e34640 1 --2- 172.21.15.164:0/2969555837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bd70d24a0 0x555bd70d9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:37.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.678+0000 7fdaf7fff640 1 -- 172.21.15.164:0/2969555837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bd70d24a0 msgr2=0x555bd70d9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:37.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.678+0000 7fdaf7fff640 1 --2- 172.21.15.164:0/2969555837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bd70d24a0 0x555bd70d9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.678+0000 7fdaf7fff640 1 -- 172.21.15.164:0/2969555837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bd70dd680 msgr2=0x555bd70f2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:37.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.678+0000 7fdaf7fff640 1 --2- 172.21.15.164:0/2969555837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bd70dd680 0x555bd70f2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.678+0000 7fdaf7fff640 1 -- 172.21.15.164:0/2969555837 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bd70daa90 con 0x555bd70d2dc0 2024-09-19T08:57:37.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.678+0000 7fdaf7fff640 1 --2- 172.21.15.164:0/2969555837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd70d2dc0 0x555bd70f2820 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fdae80092a0 tx=0x7fdae802ab60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d0388d2f217df4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:37.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.679+0000 7fdaf5ffb640 1 -- 172.21.15.164:0/2969555837 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8035040 con 0x555bd70d2dc0 2024-09-19T08:57:37.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.679+0000 7fdaf5ffb640 1 -- 172.21.15.164:0/2969555837 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdae8005520 con 0x555bd70d2dc0 2024-09-19T08:57:37.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.679+0000 7fdaf5ffb640 1 -- 172.21.15.164:0/2969555837 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8004b90 con 0x555bd70d2dc0 2024-09-19T08:57:37.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.679+0000 7fdb08c70740 1 -- 172.21.15.164:0/2969555837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd70d2dc0 msgr2=0x555bd70f2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:37.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.679+0000 7fdb08c70740 1 --2- 172.21.15.164:0/2969555837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd70d2dc0 0x555bd70f2820 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fdae80092a0 tx=0x7fdae802ab60 comp rx=0 tx=0).stop 2024-09-19T08:57:37.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.679+0000 7fdb08c70740 1 -- 172.21.15.164:0/2969555837 shutdown_connections 2024-09-19T08:57:37.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.679+0000 7fdb08c70740 1 --2- 172.21.15.164:0/2969555837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bd70dd680 0x555bd70f2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.679+0000 7fdb08c70740 1 --2- 172.21.15.164:0/2969555837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd70d2dc0 0x555bd70f2820 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.679+0000 7fdb08c70740 1 --2- 172.21.15.164:0/2969555837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bd70d24a0 0x555bd70d9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.679+0000 7fdb08c70740 1 -- 172.21.15.164:0/2969555837 >> 172.21.15.164:0/2969555837 conn(0x555bd70cea30 msgr2=0x555bd70cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:37.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.679+0000 7fdb08c70740 1 -- 172.21.15.164:0/2969555837 shutdown_connections 2024-09-19T08:57:37.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.679+0000 7fdb08c70740 1 -- 172.21.15.164:0/2969555837 wait complete. 2024-09-19T08:57:37.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.680+0000 7fdb08c70740 1 -- start start 2024-09-19T08:57:37.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.680+0000 7fdb08c70740 1 Processor -- start 2024-09-19T08:57:37.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.680+0000 7fdb08c70740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd70d24a0 0x555bd714ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:37.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.680+0000 7fdb08c70740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bd70d2dc0 0x555bd714f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:37.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.680+0000 7fdb08c70740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bd70dd680 0x555bd71532c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:37.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.680+0000 7fdb08c70740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdb001aee80 con 0x555bd70d24a0 2024-09-19T08:57:37.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.680+0000 7fdb08c70740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdb001ae780 con 0x555bd70dd680 2024-09-19T08:57:37.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.680+0000 7fdb08c70740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdb001ab9e0 con 0x555bd70d2dc0 2024-09-19T08:57:37.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.680+0000 7fdb08c70740 10 client.0 _ll_register_callbacks cb 0x7fdaf8013740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:57:37.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.680+0000 7fdb04e34640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd70d24a0 0x555bd714ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:37.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.680+0000 7fdb04e34640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd70d24a0 0x555bd714ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40914/0 (socket says 172.21.15.164:40914) 2024-09-19T08:57:37.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.680+0000 7fdb04e34640 1 -- 172.21.15.164:0/45892108 learned_addr learned my addr 172.21.15.164:0/45892108 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:37.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.680+0000 7fdaf7fff640 1 --2- 172.21.15.164:0/45892108 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bd70d2dc0 0x555bd714f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:37.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.680+0000 7fdb04e34640 1 --2- 172.21.15.164:0/45892108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bd70dd680 0x555bd71532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:37.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.680+0000 7fdb04e34640 1 -- 172.21.15.164:0/45892108 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bd70d2dc0 msgr2=0x555bd714f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:37.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.680+0000 7fdb04e34640 1 --2- 172.21.15.164:0/45892108 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bd70d2dc0 0x555bd714f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.681+0000 7fdb04e34640 1 -- 172.21.15.164:0/45892108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bd70dd680 msgr2=0x555bd71532c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:37.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.681+0000 7fdb04e34640 1 --2- 172.21.15.164:0/45892108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bd70dd680 0x555bd71532c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.681+0000 7fdb04e34640 1 -- 172.21.15.164:0/45892108 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bd7153960 con 0x555bd70d24a0 2024-09-19T08:57:37.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.681+0000 7fdb04e34640 1 --2- 172.21.15.164:0/45892108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd70d24a0 0x555bd714ecb0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8004a80 tx=0x7fdaf80078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c484ca3cc1e827b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:37.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.681+0000 7fdadf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:57:37.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.681+0000 7fdadf7fe640 1 -- 172.21.15.164:0/45892108 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8011b10 con 0x555bd70d24a0 2024-09-19T08:57:37.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.681+0000 7fdadf7fe640 1 -- 172.21.15.164:0/45892108 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7fdaf80103c0 con 0x555bd70d24a0 2024-09-19T08:57:37.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.681+0000 7fdb08c70740 1 -- 172.21.15.164:0/45892108 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdae8005520 con 0x555bd70d24a0 2024-09-19T08:57:37.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.681+0000 7fdadf7fe640 1 -- 172.21.15.164:0/45892108 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdaf80105c0 con 0x555bd70d24a0 2024-09-19T08:57:37.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.681+0000 7fdb08c70740 10 client.18018 fetch_fsmap learned FSMap version 33 2024-09-19T08:57:37.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.681+0000 7fdb08c70740 1 -- 172.21.15.164:0/45892108 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555bd715bf00 con 0x555bd70d24a0 2024-09-19T08:57:37.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.681+0000 7fdadf7fe640 1 -- 172.21.15.164:0/45892108 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdae8005520 con 0x555bd70d24a0 2024-09-19T08:57:37.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.681+0000 7fdadf7fe640 1 -- 172.21.15.164:0/45892108 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 33) ==== 8366+0+0 (secure 0 0 0) 0x7fdaf800ed20 con 0x555bd70d24a0 2024-09-19T08:57:37.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.681+0000 7fdb08c70740 10 client.18018 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T08:57:37.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.681+0000 7fdb08c70740 10 client.18018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:57:37.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.681+0000 7fdb08c70740 20 client.18018 populate_metadata read hostname 'smithi164' 2024-09-19T08:57:37.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.681+0000 7fdb08c70740 1 --2- 172.21.15.164:0/45892108 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555bd715c650 0x555bd715ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:37.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.682+0000 7fdb08c70740 4 client.18018 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T08:57:37.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.682+0000 7fdb08c70740 1 -- 172.21.15.164:0/45892108 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555bd703fd60 con 0x555bd715c650 2024-09-19T08:57:37.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.682+0000 7fdaf7fff640 1 --2- 172.21.15.164:0/45892108 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555bd715c650 0x555bd715ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:37.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.682+0000 7fdadf7fe640 10 client.18018 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:57:37.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.682+0000 7fdaf7fff640 1 --2- 172.21.15.164:0/45892108 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555bd715c650 0x555bd715ea70 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd89703d1e2dc07b server_cookie=5903cfbcae69258a in_seq=0 out_seq=0 2024-09-19T08:57:37.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.682+0000 7fdadf7fe640 1 -- 172.21.15.164:0/45892108 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555bd703fd60 con 0x555bd715c650 2024-09-19T08:57:37.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.682+0000 7fdadf7fe640 10 client.18018 handle_command_reply: tid=0 2024-09-19T08:57:37.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.682+0000 7fdb08c70740 1 client.18018 shutdown 2024-09-19T08:57:37.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.683+0000 7fdb08c70740 1 -- 172.21.15.164:0/45892108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd70d24a0 msgr2=0x555bd714ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:37.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.683+0000 7fdb08c70740 1 --2- 172.21.15.164:0/45892108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd70d24a0 0x555bd714ecb0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8004a80 tx=0x7fdaf80078d0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.683+0000 7fdb08c70740 1 -- 172.21.15.164:0/45892108 shutdown_connections 2024-09-19T08:57:37.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.683+0000 7fdb08c70740 1 --2- 172.21.15.164:0/45892108 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x555bd715c650 0x555bd715ea70 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.683+0000 7fdb08c70740 1 --2- 172.21.15.164:0/45892108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bd70dd680 0x555bd71532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.683+0000 7fdb08c70740 1 --2- 172.21.15.164:0/45892108 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bd70d2dc0 0x555bd714f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.683+0000 7fdb08c70740 1 --2- 172.21.15.164:0/45892108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555bd70d24a0 0x555bd714ecb0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.683+0000 7fdb08c70740 1 -- 172.21.15.164:0/45892108 >> 172.21.15.164:0/45892108 conn(0x555bd70cea30 msgr2=0x555bd70dba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:37.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.683+0000 7fdadf7fe640 0 client.18018 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:57:37.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.683+0000 7fdb08c70740 1 -- 172.21.15.164:0/45892108 shutdown_connections 2024-09-19T08:57:37.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.683+0000 7fdb08c70740 1 -- 172.21.15.164:0/45892108 wait complete. 2024-09-19T08:57:37.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.683+0000 7fdb08c70740 20 client.18018 trim_cache size 0 max 16384 2024-09-19T08:57:37.692 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:57:37.692 INFO:teuthology.orchestra.run.smithi164.stdout: "status": "no active scrubs running", 2024-09-19T08:57:37.692 INFO:teuthology.orchestra.run.smithi164.stdout: "scrubs": {} 2024-09-19T08:57:37.692 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:57:37.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.683+0000 7fdb06897640 1 -- 172.21.15.164:0/4093545383 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fdaec0a33b0 msgr2=0x7fdaec0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:37.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.683+0000 7fdb06897640 1 --2- 172.21.15.164:0/4093545383 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fdaec0a33b0 0x7fdaec0a5870 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00ccd80 tx=0x7fdaf00908b0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.683+0000 7fdb06897640 1 -- 172.21.15.164:0/4093545383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb00100a80 msgr2=0x7fdb00199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:37.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.683+0000 7fdb06897640 1 --2- 172.21.15.164:0/4093545383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb00100a80 0x7fdb00199430 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000b4a0 tx=0x7fdaf000b960 comp rx=0 tx=0).stop 2024-09-19T08:57:37.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.683+0000 7fdb06897640 1 -- 172.21.15.164:0/4093545383 shutdown_connections 2024-09-19T08:57:37.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.683+0000 7fdb06897640 1 --2- 172.21.15.164:0/4093545383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb00106a80 0x7fdb001a44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.683+0000 7fdb06897640 1 --2- 172.21.15.164:0/4093545383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb00101420 0x7fdb00199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.683+0000 7fdb06897640 1 --2- 172.21.15.164:0/4093545383 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fdaec0a33b0 0x7fdaec0a5870 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.683+0000 7fdb06897640 1 --2- 172.21.15.164:0/4093545383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb00100a80 0x7fdb00199430 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:37.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.683+0000 7fdb06897640 1 -- 172.21.15.164:0/4093545383 >> 172.21.15.164:0/4093545383 conn(0x7fdb000fc7a0 msgr2=0x7fdb000fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:37.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.683+0000 7fdb06897640 1 -- 172.21.15.164:0/4093545383 shutdown_connections 2024-09-19T08:57:37.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:37.684+0000 7fdb06897640 1 -- 172.21.15.164:0/4093545383 wait complete. 2024-09-19T08:57:38.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:37 smithi164 ceph-mon[30928]: pgmap v369: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 341 B/s wr, 12 op/s 2024-09-19T08:57:38.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:37 smithi170 ceph-mon[35072]: pgmap v369: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 341 B/s wr, 12 op/s 2024-09-19T08:57:38.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:37 smithi173 ceph-mon[35306]: pgmap v369: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 341 B/s wr, 12 op/s 2024-09-19T08:57:38.566 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T08:57:38.567 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:57:38.819 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:57:39.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.334+0000 7ffa159a6640 1 -- 172.21.15.164:0/773499998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa10102d30 msgr2=0x7ffa10103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:39.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.334+0000 7ffa159a6640 1 --2- 172.21.15.164:0/773499998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa10102d30 0x7ffa10103110 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009cd0 tx=0x7ffa0002a8e0 comp rx=0 tx=0).stop 2024-09-19T08:57:39.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.334+0000 7ffa159a6640 1 -- 172.21.15.164:0/773499998 shutdown_connections 2024-09-19T08:57:39.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.334+0000 7ffa159a6640 1 --2- 172.21.15.164:0/773499998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa101082a0 0x7ffa1010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:39.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.334+0000 7ffa159a6640 1 --2- 172.21.15.164:0/773499998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa101036e0 0x7ffa10107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:39.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.334+0000 7ffa159a6640 1 --2- 172.21.15.164:0/773499998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa10102d30 0x7ffa10103110 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:39.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.334+0000 7ffa159a6640 1 -- 172.21.15.164:0/773499998 >> 172.21.15.164:0/773499998 conn(0x7ffa100fc7d0 msgr2=0x7ffa100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:39.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.334+0000 7ffa159a6640 1 -- 172.21.15.164:0/773499998 shutdown_connections 2024-09-19T08:57:39.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.334+0000 7ffa159a6640 1 -- 172.21.15.164:0/773499998 wait complete. 2024-09-19T08:57:39.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.334+0000 7ffa159a6640 1 Processor -- start 2024-09-19T08:57:39.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.335+0000 7ffa159a6640 1 -- start start 2024-09-19T08:57:39.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.335+0000 7ffa159a6640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa10102d30 0x7ffa101985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:39.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.335+0000 7ffa159a6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa101036e0 0x7ffa10198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:39.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.335+0000 7ffa159a6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa101082a0 0x7ffa1019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:39.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.335+0000 7ffa159a6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffa101120a0 con 0x7ffa101036e0 2024-09-19T08:57:39.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.335+0000 7ffa159a6640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ffa10111f20 con 0x7ffa101082a0 2024-09-19T08:57:39.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.335+0000 7ffa159a6640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ffa10112220 con 0x7ffa10102d30 2024-09-19T08:57:39.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.335+0000 7ffa0e7fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa101036e0 0x7ffa10198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:39.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.335+0000 7ffa0e7fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa101036e0 0x7ffa10198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40928/0 (socket says 172.21.15.164:40928) 2024-09-19T08:57:39.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.335+0000 7ffa0e7fc640 1 -- 172.21.15.164:0/920456473 learned_addr learned my addr 172.21.15.164:0/920456473 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:39.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.335+0000 7ffa0effd640 1 --2- 172.21.15.164:0/920456473 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa10102d30 0x7ffa101985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:39.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.335+0000 7ffa0f7fe640 1 --2- 172.21.15.164:0/920456473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa101082a0 0x7ffa1019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:39.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.335+0000 7ffa0e7fc640 1 -- 172.21.15.164:0/920456473 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa10102d30 msgr2=0x7ffa101985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:39.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.335+0000 7ffa0e7fc640 1 --2- 172.21.15.164:0/920456473 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa10102d30 0x7ffa101985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:39.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.335+0000 7ffa0e7fc640 1 -- 172.21.15.164:0/920456473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa101082a0 msgr2=0x7ffa1019ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:39.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.335+0000 7ffa0e7fc640 1 --2- 172.21.15.164:0/920456473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa101082a0 0x7ffa1019ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:39.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.335+0000 7ffa0e7fc640 1 -- 172.21.15.164:0/920456473 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa1019d570 con 0x7ffa101036e0 2024-09-19T08:57:39.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.336+0000 7ffa0e7fc640 1 --2- 172.21.15.164:0/920456473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa101036e0 0x7ffa10198b00 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7ff9f800d740 tx=0x7ff9f800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:39.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.336+0000 7ffa149a4640 1 -- 172.21.15.164:0/920456473 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8017070 con 0x7ffa101036e0 2024-09-19T08:57:39.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.336+0000 7ffa159a6640 1 -- 172.21.15.164:0/920456473 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa1019d860 con 0x7ffa101036e0 2024-09-19T08:57:39.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.336+0000 7ffa149a4640 1 -- 172.21.15.164:0/920456473 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9f800de50 con 0x7ffa101036e0 2024-09-19T08:57:39.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.336+0000 7ffa149a4640 1 -- 172.21.15.164:0/920456473 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f800f6b0 con 0x7ffa101036e0 2024-09-19T08:57:39.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.336+0000 7ffa159a6640 1 -- 172.21.15.164:0/920456473 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa101a5140 con 0x7ffa101036e0 2024-09-19T08:57:39.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.337+0000 7ffa159a6640 1 -- 172.21.15.164:0/920456473 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa1006a850 con 0x7ffa101036e0 2024-09-19T08:57:39.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.338+0000 7ffa149a4640 1 -- 172.21.15.164:0/920456473 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7ff9f801a070 con 0x7ffa101036e0 2024-09-19T08:57:39.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.339+0000 7ffa149a4640 1 --2- 172.21.15.164:0/920456473 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff9fc0a33b0 0x7ff9fc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:39.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.339+0000 7ffa0effd640 1 --2- 172.21.15.164:0/920456473 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff9fc0a33b0 0x7ff9fc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:39.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.339+0000 7ffa149a4640 1 -- 172.21.15.164:0/920456473 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7ff9f80cf620 con 0x7ffa101036e0 2024-09-19T08:57:39.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.339+0000 7ffa0effd640 1 --2- 172.21.15.164:0/920456473 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff9fc0a33b0 0x7ff9fc0a5870 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ffa00006010 tx=0x7ffa00009360 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:39.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.342+0000 7ffa149a4640 1 -- 172.21.15.164:0/920456473 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff9f809b9f0 con 0x7ffa101036e0 2024-09-19T08:57:39.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.544+0000 7ffa159a6640 1 -- 172.21.15.164:0/920456473 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa1010a2b0 con 0x7ffa101036e0 2024-09-19T08:57:39.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.545+0000 7ffa149a4640 1 -- 172.21.15.164:0/920456473 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11985 (secure 0 0 0) 0x7ff9f809fcf0 con 0x7ffa101036e0 2024-09-19T08:57:39.545 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:57:39.545 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":33,"btime":"2024-09-19T08:57:15:914589+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:57:15.844000+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24442,24424,24449,14595]},"id":1}]} 2024-09-19T08:57:39.545 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 33 2024-09-19T08:57:39.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.547+0000 7ffa159a6640 1 -- 172.21.15.164:0/920456473 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff9fc0a33b0 msgr2=0x7ff9fc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:39.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.547+0000 7ffa159a6640 1 --2- 172.21.15.164:0/920456473 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff9fc0a33b0 0x7ff9fc0a5870 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ffa00006010 tx=0x7ffa00009360 comp rx=0 tx=0).stop 2024-09-19T08:57:39.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.547+0000 7ffa159a6640 1 -- 172.21.15.164:0/920456473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa101036e0 msgr2=0x7ffa10198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:39.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.547+0000 7ffa159a6640 1 --2- 172.21.15.164:0/920456473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa101036e0 0x7ffa10198b00 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7ff9f800d740 tx=0x7ff9f800dc00 comp rx=0 tx=0).stop 2024-09-19T08:57:39.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.547+0000 7ffa159a6640 1 -- 172.21.15.164:0/920456473 shutdown_connections 2024-09-19T08:57:39.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.547+0000 7ffa159a6640 1 --2- 172.21.15.164:0/920456473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa101082a0 0x7ffa1019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:39.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.547+0000 7ffa159a6640 1 --2- 172.21.15.164:0/920456473 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff9fc0a33b0 0x7ff9fc0a5870 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:39.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.547+0000 7ffa159a6640 1 --2- 172.21.15.164:0/920456473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa101036e0 0x7ffa10198b00 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:39.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.547+0000 7ffa159a6640 1 --2- 172.21.15.164:0/920456473 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa10102d30 0x7ffa101985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:39.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.547+0000 7ffa159a6640 1 -- 172.21.15.164:0/920456473 >> 172.21.15.164:0/920456473 conn(0x7ffa100fc7d0 msgr2=0x7ffa10105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:39.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.547+0000 7ffa159a6640 1 -- 172.21.15.164:0/920456473 shutdown_connections 2024-09-19T08:57:39.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:39.547+0000 7ffa159a6640 1 -- 172.21.15.164:0/920456473 wait complete. 2024-09-19T08:57:40.219 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:39 smithi164 ceph-mon[30928]: pgmap v370: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 8 op/s 2024-09-19T08:57:40.219 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:39 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/920456473' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:57:40.246 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T08:57:40.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:39 smithi170 ceph-mon[35072]: pgmap v370: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 8 op/s 2024-09-19T08:57:40.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:39 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/920456473' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:57:40.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:39 smithi173 ceph-mon[35306]: pgmap v370: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 8 op/s 2024-09-19T08:57:40.697 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:39 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/920456473' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:57:40.701 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:57:41.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.027+0000 7f8ac907c640 1 -- 172.21.15.164:0/3451995917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ac4101420 msgr2=0x7f8ac410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:41.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.027+0000 7f8ac907c640 1 --2- 172.21.15.164:0/3451995917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ac4101420 0x7f8ac410d950 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009cd0 tx=0x7f8abc02a8e0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.027+0000 7f8ac907c640 1 -- 172.21.15.164:0/3451995917 shutdown_connections 2024-09-19T08:57:41.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.027+0000 7f8ac907c640 1 --2- 172.21.15.164:0/3451995917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ac4101420 0x7f8ac410d950 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.027+0000 7f8ac907c640 1 --2- 172.21.15.164:0/3451995917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ac4100a80 0x7f8ac4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.027+0000 7f8ac907c640 1 --2- 172.21.15.164:0/3451995917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ac4106a80 0x7f8ac4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.027+0000 7f8ac907c640 1 -- 172.21.15.164:0/3451995917 >> 172.21.15.164:0/3451995917 conn(0x7f8ac40fc7a0 msgr2=0x7f8ac40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:41.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.027+0000 7f8ac907c640 1 -- 172.21.15.164:0/3451995917 shutdown_connections 2024-09-19T08:57:41.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.027+0000 7f8ac907c640 1 -- 172.21.15.164:0/3451995917 wait complete. 2024-09-19T08:57:41.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.027+0000 7f8ac907c640 1 Processor -- start 2024-09-19T08:57:41.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.027+0000 7f8ac907c640 1 -- start start 2024-09-19T08:57:41.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.028+0000 7f8ac907c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ac4100a80 0x7f8ac4199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:41.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.028+0000 7f8ac907c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ac4101420 0x7f8ac4199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:41.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.028+0000 7f8ac907c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ac4106a80 0x7f8ac41a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:41.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.028+0000 7f8ac907c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac4110ea0 con 0x7f8ac4100a80 2024-09-19T08:57:41.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.028+0000 7f8ac907c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac4110d20 con 0x7f8ac4106a80 2024-09-19T08:57:41.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.028+0000 7f8ac907c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac4111020 con 0x7f8ac4101420 2024-09-19T08:57:41.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.028+0000 7f8ac2d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ac4100a80 0x7f8ac4199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:41.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.028+0000 7f8ac2d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ac4100a80 0x7f8ac4199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40946/0 (socket says 172.21.15.164:40946) 2024-09-19T08:57:41.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.028+0000 7f8ac2d9d640 1 -- 172.21.15.164:0/3847188874 learned_addr learned my addr 172.21.15.164:0/3847188874 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:41.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.028+0000 7f8abbfff640 1 --2- 172.21.15.164:0/3847188874 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ac4101420 0x7f8ac4199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:41.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.028+0000 7f8ac359e640 1 --2- 172.21.15.164:0/3847188874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ac4106a80 0x7f8ac41a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:41.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.028+0000 7f8ac2d9d640 1 -- 172.21.15.164:0/3847188874 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ac4101420 msgr2=0x7f8ac4199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:41.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.028+0000 7f8ac2d9d640 1 --2- 172.21.15.164:0/3847188874 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ac4101420 0x7f8ac4199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.028+0000 7f8ac2d9d640 1 -- 172.21.15.164:0/3847188874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ac4106a80 msgr2=0x7f8ac41a42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:41.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.028+0000 7f8ac2d9d640 1 --2- 172.21.15.164:0/3847188874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ac4106a80 0x7f8ac41a42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.028+0000 7f8ac2d9d640 1 -- 172.21.15.164:0/3847188874 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ac41a4810 con 0x7f8ac4100a80 2024-09-19T08:57:41.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.028+0000 7f8ac2d9d640 1 --2- 172.21.15.164:0/3847188874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ac4100a80 0x7f8ac4199250 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f8ab400b4a0 tx=0x7f8ab400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:41.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.028+0000 7f8ac0d99640 1 -- 172.21.15.164:0/3847188874 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4014070 con 0x7f8ac4100a80 2024-09-19T08:57:41.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.028+0000 7f8ac0d99640 1 -- 172.21.15.164:0/3847188874 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ab4010af0 con 0x7f8ac4100a80 2024-09-19T08:57:41.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.028+0000 7f8ac907c640 1 -- 172.21.15.164:0/3847188874 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ac41a49e0 con 0x7f8ac4100a80 2024-09-19T08:57:41.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.028+0000 7f8ac0d99640 1 -- 172.21.15.164:0/3847188874 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4017d10 con 0x7f8ac4100a80 2024-09-19T08:57:41.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.029+0000 7f8ac907c640 1 -- 172.21.15.164:0/3847188874 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ac41a4ea0 con 0x7f8ac4100a80 2024-09-19T08:57:41.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.031+0000 7f8ac0d99640 1 -- 172.21.15.164:0/3847188874 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f8ab4017070 con 0x7f8ac4100a80 2024-09-19T08:57:41.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.031+0000 7f8acb455740 1 Processor -- start 2024-09-19T08:57:41.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.031+0000 7f8acb455740 1 -- start start 2024-09-19T08:57:41.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.031+0000 7f8ac0d99640 1 --2- 172.21.15.164:0/3847188874 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f8ab00a33b0 0x7f8ab00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:41.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.031+0000 7f8ac359e640 1 --2- 172.21.15.164:0/3847188874 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f8ab00a33b0 0x7f8ab00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:41.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.031+0000 7f8ac0d99640 1 -- 172.21.15.164:0/3847188874 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7f8ab40cf050 con 0x7f8ac4100a80 2024-09-19T08:57:41.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.031+0000 7f8acb455740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55572fa47b40 0x55572fa46090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:41.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.032+0000 7f8acb455740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572fa465d0 0x55572fa46a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:41.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.032+0000 7f8acb455740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55572fa55780 0x55572fa57b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:41.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.032+0000 7f8acb455740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55572f9ac2f0 con 0x55572fa465d0 2024-09-19T08:57:41.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.032+0000 7f8acb455740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55572f953cf0 con 0x55572fa47b40 2024-09-19T08:57:41.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.032+0000 7f8acb455740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55572f9a82b0 con 0x55572fa55780 2024-09-19T08:57:41.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.032+0000 7f8abbfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572fa465d0 0x55572fa46a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:41.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.032+0000 7f8abbfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572fa465d0 0x55572fa46a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40950/0 (socket says 172.21.15.164:40950) 2024-09-19T08:57:41.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.032+0000 7f8abbfff640 1 -- 172.21.15.164:0/1408516083 learned_addr learned my addr 172.21.15.164:0/1408516083 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:41.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.032+0000 7f8ac2d9d640 1 --2- 172.21.15.164:0/1408516083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55572fa47b40 0x55572fa46090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:41.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.032+0000 7f8ac359e640 1 --2- 172.21.15.164:0/3847188874 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f8ab00a33b0 0x7f8ab00a5870 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009a60 tx=0x7f8abc009890 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:41.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.032+0000 7f8ac359e640 1 --2- 172.21.15.164:0/1408516083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55572fa55780 0x55572fa57b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:41.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.032+0000 7f8abbfff640 1 -- 172.21.15.164:0/1408516083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55572fa55780 msgr2=0x55572fa57b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:41.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.032+0000 7f8abbfff640 1 --2- 172.21.15.164:0/1408516083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55572fa55780 0x55572fa57b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.032+0000 7f8abbfff640 1 -- 172.21.15.164:0/1408516083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55572fa47b40 msgr2=0x55572fa46090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:41.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.032+0000 7f8abbfff640 1 --2- 172.21.15.164:0/1408516083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55572fa47b40 0x55572fa46090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.032+0000 7f8abbfff640 1 -- 172.21.15.164:0/1408516083 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55572f9383a0 con 0x55572fa465d0 2024-09-19T08:57:41.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.032+0000 7f8ac359e640 1 --2- 172.21.15.164:0/1408516083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55572fa55780 0x55572fa57b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:57:41.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.032+0000 7f8abbfff640 1 --2- 172.21.15.164:0/1408516083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572fa465d0 0x55572fa46a10 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00cc40 tx=0x7f8aac00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e877bfaa79e391 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:41.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.032+0000 7f8ab97fa640 1 -- 172.21.15.164:0/1408516083 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac014070 con 0x55572fa465d0 2024-09-19T08:57:41.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.032+0000 7f8ab97fa640 1 -- 172.21.15.164:0/1408516083 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8aac0047a0 con 0x55572fa465d0 2024-09-19T08:57:41.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.032+0000 7f8ab97fa640 1 -- 172.21.15.164:0/1408516083 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac010740 con 0x55572fa465d0 2024-09-19T08:57:41.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.033+0000 7f8acb455740 1 -- 172.21.15.164:0/1408516083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572fa465d0 msgr2=0x55572fa46a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:41.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.033+0000 7f8acb455740 1 --2- 172.21.15.164:0/1408516083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572fa465d0 0x55572fa46a10 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00cc40 tx=0x7f8aac00e720 comp rx=0 tx=0).stop 2024-09-19T08:57:41.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.033+0000 7f8acb455740 1 -- 172.21.15.164:0/1408516083 shutdown_connections 2024-09-19T08:57:41.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.033+0000 7f8acb455740 1 --2- 172.21.15.164:0/1408516083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55572fa55780 0x55572fa57b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.033+0000 7f8acb455740 1 --2- 172.21.15.164:0/1408516083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572fa465d0 0x55572fa46a10 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.033+0000 7f8acb455740 1 --2- 172.21.15.164:0/1408516083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55572fa47b40 0x55572fa46090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.033+0000 7f8acb455740 1 -- 172.21.15.164:0/1408516083 >> 172.21.15.164:0/1408516083 conn(0x55572fa0f310 msgr2=0x55572fa42c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:41.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.033+0000 7f8acb455740 1 -- 172.21.15.164:0/1408516083 shutdown_connections 2024-09-19T08:57:41.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.033+0000 7f8acb455740 1 -- 172.21.15.164:0/1408516083 wait complete. 2024-09-19T08:57:41.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.034+0000 7f8acb455740 1 -- start start 2024-09-19T08:57:41.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.034+0000 7f8acb455740 1 Processor -- start 2024-09-19T08:57:41.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.034+0000 7f8acb455740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55572fa465d0 0x55572fa55130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:41.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.034+0000 7f8acb455740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55572fa47b40 0x55572fa523e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:41.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.034+0000 7f8acb455740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572fa55780 0x55572fa52980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:41.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.034+0000 7f8acb455740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55572fa58480 con 0x55572fa55780 2024-09-19T08:57:41.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.034+0000 7f8acb455740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55572f967060 con 0x55572fa465d0 2024-09-19T08:57:41.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.034+0000 7f8acb455740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55572fa58600 con 0x55572fa47b40 2024-09-19T08:57:41.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.034+0000 7f8acb455740 10 client.0 _ll_register_callbacks cb 0x55572f6ba8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:57:41.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.034+0000 7f8ac359e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572fa55780 0x55572fa52980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:41.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.034+0000 7f8ac359e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572fa55780 0x55572fa52980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40964/0 (socket says 172.21.15.164:40964) 2024-09-19T08:57:41.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.034+0000 7f8ac359e640 1 -- 172.21.15.164:0/811794634 learned_addr learned my addr 172.21.15.164:0/811794634 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:41.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.034+0000 7f8ac2d9d640 1 --2- 172.21.15.164:0/811794634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55572fa465d0 0x55572fa55130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:41.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.034+0000 7f8abbfff640 1 --2- 172.21.15.164:0/811794634 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55572fa47b40 0x55572fa523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:41.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.034+0000 7f8ac359e640 1 -- 172.21.15.164:0/811794634 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55572fa47b40 msgr2=0x55572fa523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:41.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.034+0000 7f8ac359e640 1 --2- 172.21.15.164:0/811794634 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55572fa47b40 0x55572fa523e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.034+0000 7f8ac359e640 1 -- 172.21.15.164:0/811794634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55572fa465d0 msgr2=0x55572fa55130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:41.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.034+0000 7f8ac359e640 1 --2- 172.21.15.164:0/811794634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55572fa465d0 0x55572fa55130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.034+0000 7f8ac359e640 1 -- 172.21.15.164:0/811794634 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55572f71d3c0 con 0x55572fa55780 2024-09-19T08:57:41.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.034+0000 7f8abbfff640 1 --2- 172.21.15.164:0/811794634 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55572fa47b40 0x55572fa523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:57:41.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.034+0000 7f8ac2d9d640 1 --2- 172.21.15.164:0/811794634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55572fa465d0 0x55572fa55130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:57:41.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.035+0000 7f8ac359e640 1 --2- 172.21.15.164:0/811794634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572fa55780 0x55572fa52980 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f8abc035000 tx=0x7f8abc02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdfcb407db2f8618 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:41.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.035+0000 7f8aaa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:57:41.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.035+0000 7f8aaa7fc640 1 -- 172.21.15.164:0/811794634 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8abc041070 con 0x55572fa55780 2024-09-19T08:57:41.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.035+0000 7f8aaa7fc640 1 -- 172.21.15.164:0/811794634 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7f8abc004800 con 0x55572fa55780 2024-09-19T08:57:41.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.035+0000 7f8acb455740 1 -- 172.21.15.164:0/811794634 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8aac0047a0 con 0x55572fa55780 2024-09-19T08:57:41.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.035+0000 7f8aaa7fc640 1 -- 172.21.15.164:0/811794634 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8abc0305a0 con 0x55572fa55780 2024-09-19T08:57:41.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.035+0000 7f8acb455740 10 client.18036 fetch_fsmap learned FSMap version 33 2024-09-19T08:57:41.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.035+0000 7f8acb455740 1 -- 172.21.15.164:0/811794634 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55572fa63ea0 con 0x55572fa55780 2024-09-19T08:57:41.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.035+0000 7f8aaa7fc640 1 -- 172.21.15.164:0/811794634 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8aac0047a0 con 0x55572fa55780 2024-09-19T08:57:41.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.035+0000 7f8aaa7fc640 1 -- 172.21.15.164:0/811794634 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 33) ==== 8366+0+0 (secure 0 0 0) 0x7f8abc030070 con 0x55572fa55780 2024-09-19T08:57:41.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.035+0000 7f8acb455740 10 client.18036 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T08:57:41.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.035+0000 7f8acb455740 10 client.18036 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-19T08:57:41.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.035+0000 7f8acb455740 20 client.18036 populate_metadata read hostname 'smithi164' 2024-09-19T08:57:41.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.035+0000 7f8acb455740 1 --2- 172.21.15.164:0/811794634 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55572fa653b0 0x55572fa67820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:41.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.035+0000 7f8acb455740 4 client.18036 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:57:41.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.035+0000 7f8acb455740 1 -- 172.21.15.164:0/811794634 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55572f9b9860 con 0x55572fa653b0 2024-09-19T08:57:41.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.036+0000 7f8ac2d9d640 1 --2- 172.21.15.164:0/811794634 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55572fa653b0 0x55572fa67820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:41.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.036+0000 7f8aaa7fc640 10 client.18036 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:57:41.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.036+0000 7f8ac2d9d640 1 --2- 172.21.15.164:0/811794634 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55572fa653b0 0x55572fa67820 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=71b6f43c7ddf4be0 server_cookie=c61ad8fa00ce01f6 in_seq=0 out_seq=0 2024-09-19T08:57:41.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.036+0000 7f8aaa7fc640 1 -- 172.21.15.164:0/811794634 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55572f9b9860 con 0x55572fa653b0 2024-09-19T08:57:41.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.036+0000 7f8aaa7fc640 10 client.18036 handle_command_reply: tid=0 2024-09-19T08:57:41.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.036+0000 7f8acb455740 1 client.18036 shutdown 2024-09-19T08:57:41.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.037+0000 7f8acb455740 1 -- 172.21.15.164:0/811794634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572fa55780 msgr2=0x55572fa52980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:41.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.037+0000 7f8acb455740 1 --2- 172.21.15.164:0/811794634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572fa55780 0x55572fa52980 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f8abc035000 tx=0x7f8abc02a8e0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.037+0000 7f8acb455740 1 -- 172.21.15.164:0/811794634 shutdown_connections 2024-09-19T08:57:41.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.037+0000 7f8acb455740 1 --2- 172.21.15.164:0/811794634 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55572fa653b0 0x55572fa67820 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.057 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.037+0000 7f8acb455740 1 --2- 172.21.15.164:0/811794634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572fa55780 0x55572fa52980 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.037+0000 7f8acb455740 1 --2- 172.21.15.164:0/811794634 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55572fa47b40 0x55572fa523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.037+0000 7f8acb455740 1 --2- 172.21.15.164:0/811794634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55572fa465d0 0x55572fa55130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.037+0000 7f8aaa7fc640 0 client.18036 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:57:41.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.037+0000 7f8acb455740 1 -- 172.21.15.164:0/811794634 >> 172.21.15.164:0/811794634 conn(0x55572fa0f310 msgr2=0x55572fa48930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:41.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.037+0000 7f8acb455740 1 -- 172.21.15.164:0/811794634 shutdown_connections 2024-09-19T08:57:41.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.037+0000 7f8acb455740 1 -- 172.21.15.164:0/811794634 wait complete. 2024-09-19T08:57:41.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.037+0000 7f8acb455740 20 client.18036 trim_cache size 0 max 16384 2024-09-19T08:57:41.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.050+0000 7f8acb455740 1 Processor -- start 2024-09-19T08:57:41.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.050+0000 7f8acb455740 1 -- start start 2024-09-19T08:57:41.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.050+0000 7f8acb455740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572fa465d0 0x55572fa42e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:41.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.050+0000 7f8acb455740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55572fa47b40 0x55572fa61780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:41.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.050+0000 7f8acb455740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55572fa55780 0x55572fa6d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:41.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.050+0000 7f8acb455740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac41ab540 con 0x55572fa465d0 2024-09-19T08:57:41.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.050+0000 7f8acb455740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac41ad0b0 con 0x55572fa47b40 2024-09-19T08:57:41.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.050+0000 7f8acb455740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac41abc10 con 0x55572fa55780 2024-09-19T08:57:41.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.050+0000 7f8ac2d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572fa465d0 0x55572fa42e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:41.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.050+0000 7f8ac2d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572fa465d0 0x55572fa42e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40966/0 (socket says 172.21.15.164:40966) 2024-09-19T08:57:41.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.050+0000 7f8ac2d9d640 1 -- 172.21.15.164:0/385089297 learned_addr learned my addr 172.21.15.164:0/385089297 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:41.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.050+0000 7f8ac2d9d640 1 -- 172.21.15.164:0/385089297 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55572fa55780 msgr2=0x55572fa6d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:41.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.050+0000 7f8ac2d9d640 1 --2- 172.21.15.164:0/385089297 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55572fa55780 0x55572fa6d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.050+0000 7f8ac2d9d640 1 -- 172.21.15.164:0/385089297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55572fa47b40 msgr2=0x55572fa61780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:41.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.050+0000 7f8abbfff640 1 --2- 172.21.15.164:0/385089297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55572fa47b40 0x55572fa61780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:41.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.050+0000 7f8ac2d9d640 1 --2- 172.21.15.164:0/385089297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55572fa47b40 0x55572fa61780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.050+0000 7f8ac2d9d640 1 -- 172.21.15.164:0/385089297 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55572fa54d50 con 0x55572fa465d0 2024-09-19T08:57:41.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.051+0000 7f8abbfff640 1 --2- 172.21.15.164:0/385089297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55572fa47b40 0x55572fa61780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:57:41.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.051+0000 7f8ac2d9d640 1 --2- 172.21.15.164:0/385089297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572fa465d0 0x55572fa42e10 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f8ab409b410 tx=0x7f8ab4012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=194c9395ffa8c6f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:41.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.051+0000 7f8ab8ff9640 1 -- 172.21.15.164:0/385089297 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab40b3070 con 0x55572fa465d0 2024-09-19T08:57:41.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.051+0000 7f8ab8ff9640 1 -- 172.21.15.164:0/385089297 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ab4099780 con 0x55572fa465d0 2024-09-19T08:57:41.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.051+0000 7f8ab8ff9640 1 -- 172.21.15.164:0/385089297 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab401e040 con 0x55572fa465d0 2024-09-19T08:57:41.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.051+0000 7f8acb455740 1 -- 172.21.15.164:0/385089297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572fa465d0 msgr2=0x55572fa42e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:41.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.051+0000 7f8acb455740 1 --2- 172.21.15.164:0/385089297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572fa465d0 0x55572fa42e10 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f8ab409b410 tx=0x7f8ab4012040 comp rx=0 tx=0).stop 2024-09-19T08:57:41.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.052+0000 7f8acb455740 1 -- 172.21.15.164:0/385089297 shutdown_connections 2024-09-19T08:57:41.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.052+0000 7f8acb455740 1 --2- 172.21.15.164:0/385089297 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55572fa55780 0x55572fa6d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.052+0000 7f8acb455740 1 --2- 172.21.15.164:0/385089297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55572fa47b40 0x55572fa61780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.052+0000 7f8acb455740 1 --2- 172.21.15.164:0/385089297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572fa465d0 0x55572fa42e10 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.052+0000 7f8acb455740 1 -- 172.21.15.164:0/385089297 >> 172.21.15.164:0/385089297 conn(0x55572fa0f310 msgr2=0x55572fa42700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:41.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.052+0000 7f8acb455740 1 -- 172.21.15.164:0/385089297 shutdown_connections 2024-09-19T08:57:41.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.052+0000 7f8acb455740 1 -- 172.21.15.164:0/385089297 wait complete. 2024-09-19T08:57:41.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.052+0000 7f8acb455740 1 -- start start 2024-09-19T08:57:41.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.052+0000 7f8acb455740 1 Processor -- start 2024-09-19T08:57:41.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.052+0000 7f8acb455740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55572fa465d0 0x55572fa4f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:41.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.052+0000 7f8acb455740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572fa47b40 0x55572fa4fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:41.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.052+0000 7f8acb455740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55572fa55780 0x55572fa50070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:41.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.052+0000 7f8acb455740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac41ac9e0 con 0x55572fa47b40 2024-09-19T08:57:41.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.052+0000 7f8acb455740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac41ac2e0 con 0x55572fa55780 2024-09-19T08:57:41.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.052+0000 7f8acb455740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac41a9540 con 0x55572fa465d0 2024-09-19T08:57:41.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.052+0000 7f8acb455740 10 client.0 _ll_register_callbacks cb 0x7f8abc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:57:41.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.052+0000 7f8abbfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572fa47b40 0x55572fa4fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:41.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.052+0000 7f8abbfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572fa47b40 0x55572fa4fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40968/0 (socket says 172.21.15.164:40968) 2024-09-19T08:57:41.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.052+0000 7f8abbfff640 1 -- 172.21.15.164:0/3340447832 learned_addr learned my addr 172.21.15.164:0/3340447832 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:41.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.053+0000 7f8ac2d9d640 1 --2- 172.21.15.164:0/3340447832 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55572fa465d0 0x55572fa4f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:41.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.053+0000 7f8ac359e640 1 --2- 172.21.15.164:0/3340447832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55572fa55780 0x55572fa50070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:41.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.053+0000 7f8abbfff640 1 -- 172.21.15.164:0/3340447832 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55572fa465d0 msgr2=0x55572fa4f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:41.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.053+0000 7f8abbfff640 1 --2- 172.21.15.164:0/3340447832 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55572fa465d0 0x55572fa4f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.053+0000 7f8abbfff640 1 -- 172.21.15.164:0/3340447832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55572fa55780 msgr2=0x55572fa50070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:41.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.053+0000 7f8abbfff640 1 --2- 172.21.15.164:0/3340447832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55572fa55780 0x55572fa50070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.053+0000 7f8abbfff640 1 -- 172.21.15.164:0/3340447832 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55572facf2c0 con 0x55572fa47b40 2024-09-19T08:57:41.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.053+0000 7f8ac359e640 1 --2- 172.21.15.164:0/3340447832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55572fa55780 0x55572fa50070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:57:41.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.053+0000 7f8ac2d9d640 1 --2- 172.21.15.164:0/3340447832 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55572fa465d0 0x55572fa4f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:57:41.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.053+0000 7f8abbfff640 1 --2- 172.21.15.164:0/3340447832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572fa47b40 0x55572fa4fb30 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00cc40 tx=0x7f8aac010ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a362392b7da87ddb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:41.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.053+0000 7f8aaa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:57:41.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.053+0000 7f8aaa7fc640 1 -- 172.21.15.164:0/3340447832 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac014040 con 0x55572fa47b40 2024-09-19T08:57:41.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.053+0000 7f8aaa7fc640 1 -- 172.21.15.164:0/3340447832 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7f8aac005070 con 0x55572fa47b40 2024-09-19T08:57:41.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.053+0000 7f8acb455740 1 -- 172.21.15.164:0/3340447832 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ab4099780 con 0x55572fa47b40 2024-09-19T08:57:41.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.053+0000 7f8aaa7fc640 1 -- 172.21.15.164:0/3340447832 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8aac005270 con 0x55572fa47b40 2024-09-19T08:57:41.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.053+0000 7f8acb455740 10 client.18042 fetch_fsmap learned FSMap version 33 2024-09-19T08:57:41.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.053+0000 7f8acb455740 1 -- 172.21.15.164:0/3340447832 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55572fad0510 con 0x55572fa47b40 2024-09-19T08:57:41.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.053+0000 7f8aaa7fc640 1 -- 172.21.15.164:0/3340447832 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ab4099780 con 0x55572fa47b40 2024-09-19T08:57:41.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.054+0000 7f8aaa7fc640 1 -- 172.21.15.164:0/3340447832 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 33) ==== 8366+0+0 (secure 0 0 0) 0x7f8aac023430 con 0x55572fa47b40 2024-09-19T08:57:41.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.054+0000 7f8acb455740 10 client.18042 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T08:57:41.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.054+0000 7f8acb455740 10 client.18042 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-19T08:57:41.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.054+0000 7f8acb455740 20 client.18042 populate_metadata read hostname 'smithi164' 2024-09-19T08:57:41.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.054+0000 7f8acb455740 1 --2- 172.21.15.164:0/3340447832 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55572fad0a00 0x55572fad2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:41.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.054+0000 7f8acb455740 4 client.18042 mds_command: new command op to 14595 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:57:41.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.054+0000 7f8acb455740 1 -- 172.21.15.164:0/3340447832 --> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55572f9b3d70 con 0x55572fad0a00 2024-09-19T08:57:41.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.054+0000 7f8ac2d9d640 1 --2- 172.21.15.164:0/3340447832 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55572fad0a00 0x55572fad2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:41.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.054+0000 7f8aaa7fc640 10 client.18042 ms_handle_connect on v2:172.21.15.164:6840/2221659234 2024-09-19T08:57:41.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.054+0000 7f8ac2d9d640 1 --2- 172.21.15.164:0/3340447832 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55572fad0a00 0x55572fad2df0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fc9b3218ef831b79 server_cookie=a86acccf55b26b5 in_seq=0 out_seq=0 2024-09-19T08:57:41.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.054+0000 7f8aaa7fc640 1 -- 172.21.15.164:0/3340447832 <== mds.4 v2:172.21.15.164:6840/2221659234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55572f9b3d70 con 0x55572fad0a00 2024-09-19T08:57:41.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.054+0000 7f8aaa7fc640 10 client.18042 handle_command_reply: tid=0 2024-09-19T08:57:41.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.054+0000 7f8acb455740 1 client.18042 shutdown 2024-09-19T08:57:41.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.054+0000 7f8acb455740 1 -- 172.21.15.164:0/3340447832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572fa47b40 msgr2=0x55572fa4fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:41.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.054+0000 7f8acb455740 1 --2- 172.21.15.164:0/3340447832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572fa47b40 0x55572fa4fb30 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00cc40 tx=0x7f8aac010ad0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.054+0000 7f8acb455740 1 -- 172.21.15.164:0/3340447832 shutdown_connections 2024-09-19T08:57:41.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.054+0000 7f8acb455740 1 --2- 172.21.15.164:0/3340447832 >> [v2:172.21.15.164:6840/2221659234,v1:172.21.15.164:6841/2221659234] conn(0x55572fad0a00 0x55572fad2df0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.054+0000 7f8acb455740 1 --2- 172.21.15.164:0/3340447832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55572fa55780 0x55572fa50070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.054+0000 7f8acb455740 1 --2- 172.21.15.164:0/3340447832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55572fa47b40 0x55572fa4fb30 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.054+0000 7f8aaa7fc640 0 client.18042 ms_handle_reset on v2:172.21.15.164:6840/2221659234 2024-09-19T08:57:41.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.054+0000 7f8acb455740 1 --2- 172.21.15.164:0/3340447832 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55572fa465d0 0x55572fa4f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.054+0000 7f8acb455740 1 -- 172.21.15.164:0/3340447832 >> 172.21.15.164:0/3340447832 conn(0x55572fa0f310 msgr2=0x55572fa48110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:41.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.055+0000 7f8acb455740 1 -- 172.21.15.164:0/3340447832 shutdown_connections 2024-09-19T08:57:41.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.055+0000 7f8acb455740 1 -- 172.21.15.164:0/3340447832 wait complete. 2024-09-19T08:57:41.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.055+0000 7f8acb455740 20 client.18042 trim_cache size 0 max 16384 2024-09-19T08:57:41.072 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:57:41.072 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.055+0000 7f8ac907c640 1 -- 172.21.15.164:0/3847188874 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f8ab00a33b0 msgr2=0x7f8ab00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:41.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.055+0000 7f8ac907c640 1 --2- 172.21.15.164:0/3847188874 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f8ab00a33b0 0x7f8ab00a5870 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009a60 tx=0x7f8abc009890 comp rx=0 tx=0).stop 2024-09-19T08:57:41.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.055+0000 7f8ac907c640 1 -- 172.21.15.164:0/3847188874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ac4100a80 msgr2=0x7f8ac4199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:41.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.055+0000 7f8ac907c640 1 --2- 172.21.15.164:0/3847188874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ac4100a80 0x7f8ac4199250 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f8ab400b4a0 tx=0x7f8ab400b960 comp rx=0 tx=0).stop 2024-09-19T08:57:41.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.055+0000 7f8ac907c640 1 -- 172.21.15.164:0/3847188874 shutdown_connections 2024-09-19T08:57:41.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.055+0000 7f8ac907c640 1 --2- 172.21.15.164:0/3847188874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ac4106a80 0x7f8ac41a42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.055+0000 7f8ac907c640 1 --2- 172.21.15.164:0/3847188874 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ac4101420 0x7f8ac4199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.055+0000 7f8ac907c640 1 --2- 172.21.15.164:0/3847188874 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f8ab00a33b0 0x7f8ab00a5870 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.055+0000 7f8ac907c640 1 --2- 172.21.15.164:0/3847188874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ac4100a80 0x7f8ac4199250 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:41.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.055+0000 7f8ac907c640 1 -- 172.21.15.164:0/3847188874 >> 172.21.15.164:0/3847188874 conn(0x7f8ac40fc7a0 msgr2=0x7f8ac40fcd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:41.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.055+0000 7f8ac907c640 1 -- 172.21.15.164:0/3847188874 shutdown_connections 2024-09-19T08:57:41.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:41.055+0000 7f8ac907c640 1 -- 172.21.15.164:0/3847188874 wait complete. 2024-09-19T08:57:41.949 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T08:57:42.198 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:57:42.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:41 smithi164 ceph-mon[30928]: pgmap v371: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-19T08:57:42.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:41 smithi170 ceph-mon[35072]: pgmap v371: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-19T08:57:42.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:41 smithi173 ceph-mon[35306]: pgmap v371: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-19T08:57:42.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.662+0000 7fa7798e8640 1 -- 172.21.15.164:0/3920405210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa774102570 msgr2=0x7fa7741029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:42.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.662+0000 7fa7798e8640 1 --2- 172.21.15.164:0/3920405210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa774102570 0x7fa7741029b0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fa768009900 tx=0x7fa76802a480 comp rx=0 tx=0).stop 2024-09-19T08:57:42.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.662+0000 7fa7798e8640 1 -- 172.21.15.164:0/3920405210 shutdown_connections 2024-09-19T08:57:42.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.662+0000 7fa7798e8640 1 --2- 172.21.15.164:0/3920405210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa774102ef0 0x7fa77410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:42.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.662+0000 7fa7798e8640 1 --2- 172.21.15.164:0/3920405210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa774102570 0x7fa7741029b0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:42.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.662+0000 7fa7798e8640 1 --2- 172.21.15.164:0/3920405210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa774100e20 0x7fa774101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:42.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.662+0000 7fa7798e8640 1 -- 172.21.15.164:0/3920405210 >> 172.21.15.164:0/3920405210 conn(0x7fa7740fc7d0 msgr2=0x7fa7740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:42.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.662+0000 7fa7798e8640 1 -- 172.21.15.164:0/3920405210 shutdown_connections 2024-09-19T08:57:42.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.662+0000 7fa7798e8640 1 -- 172.21.15.164:0/3920405210 wait complete. 2024-09-19T08:57:42.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.663+0000 7fa7798e8640 1 Processor -- start 2024-09-19T08:57:42.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.663+0000 7fa7798e8640 1 -- start start 2024-09-19T08:57:42.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.664+0000 7fa7798e8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa774100e20 0x7fa77419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:42.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.664+0000 7fa7798e8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa774102570 0x7fa77419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:42.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.664+0000 7fa7798e8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa774102ef0 0x7fa77419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:42.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.664+0000 7fa7798e8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa774110f80 con 0x7fa774102ef0 2024-09-19T08:57:42.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.664+0000 7fa7798e8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa774110e00 con 0x7fa774100e20 2024-09-19T08:57:42.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.664+0000 7fa7798e8640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa774111100 con 0x7fa774102570 2024-09-19T08:57:42.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.664+0000 7fa7737fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa774102ef0 0x7fa77419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:42.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.664+0000 7fa7737fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa774102ef0 0x7fa77419ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:40990/0 (socket says 172.21.15.164:40990) 2024-09-19T08:57:42.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.664+0000 7fa7737fe640 1 -- 172.21.15.164:0/3275939839 learned_addr learned my addr 172.21.15.164:0/3275939839 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:42.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.664+0000 7fa772ffd640 1 --2- 172.21.15.164:0/3275939839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa774100e20 0x7fa77419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:42.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.664+0000 7fa7727fc640 1 --2- 172.21.15.164:0/3275939839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa774102570 0x7fa77419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:42.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.664+0000 7fa7737fe640 1 -- 172.21.15.164:0/3275939839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa774102570 msgr2=0x7fa77419abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:42.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.664+0000 7fa7737fe640 1 --2- 172.21.15.164:0/3275939839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa774102570 0x7fa77419abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:42.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.664+0000 7fa7737fe640 1 -- 172.21.15.164:0/3275939839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa774100e20 msgr2=0x7fa77419a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:42.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.664+0000 7fa7737fe640 1 --2- 172.21.15.164:0/3275939839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa774100e20 0x7fa77419a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:42.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.664+0000 7fa7737fe640 1 -- 172.21.15.164:0/3275939839 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77419f660 con 0x7fa774102ef0 2024-09-19T08:57:42.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.664+0000 7fa7727fc640 1 --2- 172.21.15.164:0/3275939839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa774102570 0x7fa77419abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:57:42.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.664+0000 7fa7737fe640 1 --2- 172.21.15.164:0/3275939839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa774102ef0 0x7fa77419ef80 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00d750 tx=0x7fa76c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:42.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.665+0000 7fa7788e6640 1 -- 172.21.15.164:0/3275939839 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c017070 con 0x7fa774102ef0 2024-09-19T08:57:42.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.665+0000 7fa7798e8640 1 -- 172.21.15.164:0/3275939839 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa77419f950 con 0x7fa774102ef0 2024-09-19T08:57:42.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.665+0000 7fa7788e6640 1 -- 172.21.15.164:0/3275939839 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa76c005500 con 0x7fa774102ef0 2024-09-19T08:57:42.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.665+0000 7fa7788e6640 1 -- 172.21.15.164:0/3275939839 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c004d90 con 0x7fa774102ef0 2024-09-19T08:57:42.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.665+0000 7fa7798e8640 1 -- 172.21.15.164:0/3275939839 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7741a7230 con 0x7fa774102ef0 2024-09-19T08:57:42.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.667+0000 7fa7788e6640 1 -- 172.21.15.164:0/3275939839 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fa76c004f30 con 0x7fa774102ef0 2024-09-19T08:57:42.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.667+0000 7fa77bcc1740 1 Processor -- start 2024-09-19T08:57:42.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.667+0000 7fa77bcc1740 1 -- start start 2024-09-19T08:57:42.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.667+0000 7fa77bcc1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564aa3f2b40 0x5564aa3f1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:42.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.667+0000 7fa77bcc1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5564aa3f15d0 0x5564aa3f1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:42.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.667+0000 7fa77bcc1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564aa3fc370 0x5564aa3fe760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:42.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.667+0000 7fa7788e6640 1 --2- 172.21.15.164:0/3275939839 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa75c0a33b0 0x7fa75c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:42.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.667+0000 7fa77bcc1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5564aa3572f0 con 0x5564aa3f2b40 2024-09-19T08:57:42.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.667+0000 7fa77bcc1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5564aa2fecf0 con 0x5564aa3fc370 2024-09-19T08:57:42.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.667+0000 7fa77bcc1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5564aa3532b0 con 0x5564aa3f15d0 2024-09-19T08:57:42.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.667+0000 7fa7727fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564aa3f2b40 0x5564aa3f1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:42.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.667+0000 7fa7727fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564aa3f2b40 0x5564aa3f1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41000/0 (socket says 172.21.15.164:41000) 2024-09-19T08:57:42.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.668+0000 7fa7727fc640 1 -- 172.21.15.164:0/3527143570 learned_addr learned my addr 172.21.15.164:0/3527143570 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:42.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.668+0000 7fa7737fe640 1 --2- 172.21.15.164:0/3275939839 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa75c0a33b0 0x7fa75c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:42.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.668+0000 7fa7788e6640 1 -- 172.21.15.164:0/3275939839 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7fa76c0cf780 con 0x7fa774102ef0 2024-09-19T08:57:42.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.668+0000 7fa7727fc640 1 --2- 172.21.15.164:0/3527143570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564aa3fc370 0x5564aa3fe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:42.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.668+0000 7fa772ffd640 1 --2- 172.21.15.164:0/3527143570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5564aa3f15d0 0x5564aa3f1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:42.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.668+0000 7fa7727fc640 1 -- 172.21.15.164:0/3527143570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5564aa3f15d0 msgr2=0x5564aa3f1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:42.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.668+0000 7fa7727fc640 1 --2- 172.21.15.164:0/3527143570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5564aa3f15d0 0x5564aa3f1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:42.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.668+0000 7fa7727fc640 1 -- 172.21.15.164:0/3527143570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564aa3fc370 msgr2=0x5564aa3fe760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:42.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.668+0000 7fa7727fc640 1 --2- 172.21.15.164:0/3527143570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564aa3fc370 0x5564aa3fe760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:42.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.668+0000 7fa7727fc640 1 -- 172.21.15.164:0/3527143570 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564aa2e33a0 con 0x5564aa3f2b40 2024-09-19T08:57:42.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.668+0000 7fa7727fc640 1 --2- 172.21.15.164:0/3527143570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564aa3f2b40 0x5564aa3f1090 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fa768006010 tx=0x7fa768031480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581f9727a37c8a8e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:42.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.668+0000 7fa7737fe640 1 --2- 172.21.15.164:0/3275939839 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa75c0a33b0 0x7fa75c0a5870 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0ccf10 tx=0x7fa76c090ed0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:42.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.668+0000 7fa759ffb640 1 -- 172.21.15.164:0/3527143570 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa768035070 con 0x5564aa3f2b40 2024-09-19T08:57:42.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.668+0000 7fa759ffb640 1 -- 172.21.15.164:0/3527143570 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa768048850 con 0x5564aa3f2b40 2024-09-19T08:57:42.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.668+0000 7fa759ffb640 1 -- 172.21.15.164:0/3527143570 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa768004950 con 0x5564aa3f2b40 2024-09-19T08:57:42.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.669+0000 7fa77bcc1740 1 -- 172.21.15.164:0/3527143570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564aa3f2b40 msgr2=0x5564aa3f1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:42.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.669+0000 7fa77bcc1740 1 --2- 172.21.15.164:0/3527143570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564aa3f2b40 0x5564aa3f1090 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fa768006010 tx=0x7fa768031480 comp rx=0 tx=0).stop 2024-09-19T08:57:42.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.669+0000 7fa77bcc1740 1 -- 172.21.15.164:0/3527143570 shutdown_connections 2024-09-19T08:57:42.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.669+0000 7fa77bcc1740 1 --2- 172.21.15.164:0/3527143570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564aa3fc370 0x5564aa3fe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:42.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.669+0000 7fa77bcc1740 1 --2- 172.21.15.164:0/3527143570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5564aa3f15d0 0x5564aa3f1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:42.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.669+0000 7fa77bcc1740 1 --2- 172.21.15.164:0/3527143570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564aa3f2b40 0x5564aa3f1090 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:42.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.669+0000 7fa77bcc1740 1 -- 172.21.15.164:0/3527143570 >> 172.21.15.164:0/3527143570 conn(0x5564aa3ba310 msgr2=0x5564aa3edc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:42.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.669+0000 7fa77bcc1740 1 -- 172.21.15.164:0/3527143570 shutdown_connections 2024-09-19T08:57:42.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.669+0000 7fa77bcc1740 1 -- 172.21.15.164:0/3527143570 wait complete. 2024-09-19T08:57:42.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.669+0000 7fa77bcc1740 1 -- start start 2024-09-19T08:57:42.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.670+0000 7fa77bcc1740 1 Processor -- start 2024-09-19T08:57:42.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.670+0000 7fa77bcc1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564aa3f15d0 0x5564aa3fbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:42.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.670+0000 7fa77bcc1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564aa3f2b40 0x5564aa3f8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:42.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.670+0000 7fa77bcc1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5564aa3fc370 0x5564aa3f95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:42.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.670+0000 7fa77bcc1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5564aa3ff010 con 0x5564aa3f2b40 2024-09-19T08:57:42.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.670+0000 7fa77bcc1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5564aa312060 con 0x5564aa3f15d0 2024-09-19T08:57:42.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.670+0000 7fa77bcc1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5564aa3ff190 con 0x5564aa3fc370 2024-09-19T08:57:42.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.670+0000 7fa77bcc1740 10 client.0 _ll_register_callbacks cb 0x5564aa0658c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:57:42.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.670+0000 7fa772ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564aa3f2b40 0x5564aa3f8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:42.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.670+0000 7fa772ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564aa3f2b40 0x5564aa3f8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41014/0 (socket says 172.21.15.164:41014) 2024-09-19T08:57:42.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.670+0000 7fa772ffd640 1 -- 172.21.15.164:0/224106670 learned_addr learned my addr 172.21.15.164:0/224106670 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:42.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.670+0000 7fa7727fc640 1 --2- 172.21.15.164:0/224106670 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5564aa3fc370 0x5564aa3f95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:42.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.670+0000 7fa7727fc640 1 --2- 172.21.15.164:0/224106670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564aa3f15d0 0x5564aa3fbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:42.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.670+0000 7fa772ffd640 1 -- 172.21.15.164:0/224106670 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5564aa3fc370 msgr2=0x5564aa3f95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:42.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.670+0000 7fa772ffd640 1 --2- 172.21.15.164:0/224106670 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5564aa3fc370 0x5564aa3f95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:42.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.670+0000 7fa772ffd640 1 -- 172.21.15.164:0/224106670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564aa3f15d0 msgr2=0x5564aa3fbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:42.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.670+0000 7fa772ffd640 1 --2- 172.21.15.164:0/224106670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564aa3f15d0 0x5564aa3fbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:42.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.670+0000 7fa772ffd640 1 -- 172.21.15.164:0/224106670 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564aa0c83c0 con 0x5564aa3f2b40 2024-09-19T08:57:42.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.670+0000 7fa772ffd640 1 --2- 172.21.15.164:0/224106670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564aa3f2b40 0x5564aa3f8fe0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fa7640129a0 tx=0x7fa764012e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc59a444d1214bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:42.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.670+0000 7fa752ffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:57:42.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.670+0000 7fa752ffd640 1 -- 172.21.15.164:0/224106670 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76400f9e0 con 0x5564aa3f2b40 2024-09-19T08:57:42.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.671+0000 7fa752ffd640 1 -- 172.21.15.164:0/224106670 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7fa76400fb80 con 0x5564aa3f2b40 2024-09-19T08:57:42.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.671+0000 7fa77bcc1740 1 -- 172.21.15.164:0/224106670 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa768048850 con 0x5564aa3f2b40 2024-09-19T08:57:42.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.671+0000 7fa752ffd640 1 -- 172.21.15.164:0/224106670 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa76400fd80 con 0x5564aa3f2b40 2024-09-19T08:57:42.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.671+0000 7fa77bcc1740 10 client.18054 fetch_fsmap learned FSMap version 33 2024-09-19T08:57:42.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.671+0000 7fa77bcc1740 1 -- 172.21.15.164:0/224106670 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5564aa40aad0 con 0x5564aa3f2b40 2024-09-19T08:57:42.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.671+0000 7fa752ffd640 1 -- 172.21.15.164:0/224106670 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa768048850 con 0x5564aa3f2b40 2024-09-19T08:57:42.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.671+0000 7fa752ffd640 1 -- 172.21.15.164:0/224106670 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 33) ==== 8366+0+0 (secure 0 0 0) 0x7fa764010430 con 0x5564aa3f2b40 2024-09-19T08:57:42.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.671+0000 7fa77bcc1740 10 client.18054 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T08:57:42.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.671+0000 7fa77bcc1740 10 client.18054 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T08:57:42.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.671+0000 7fa77bcc1740 20 client.18054 populate_metadata read hostname 'smithi164' 2024-09-19T08:57:42.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.671+0000 7fa77bcc1740 1 --2- 172.21.15.164:0/224106670 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x5564aa40bfe0 0x5564aa40e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:42.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.671+0000 7fa77bcc1740 4 client.18054 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:57:42.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.671+0000 7fa77bcc1740 1 -- 172.21.15.164:0/224106670 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564aa364860 con 0x5564aa40bfe0 2024-09-19T08:57:42.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.671+0000 7fa7727fc640 1 --2- 172.21.15.164:0/224106670 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x5564aa40bfe0 0x5564aa40e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:42.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.672+0000 7fa752ffd640 10 client.18054 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:57:42.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.672+0000 7fa7727fc640 1 --2- 172.21.15.164:0/224106670 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x5564aa40bfe0 0x5564aa40e3f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fdf3801c82f5745d server_cookie=350b62c4a8d5d402 in_seq=0 out_seq=0 2024-09-19T08:57:42.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.672+0000 7fa752ffd640 1 -- 172.21.15.164:0/224106670 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5564aa364860 con 0x5564aa40bfe0 2024-09-19T08:57:42.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.673+0000 7fa752ffd640 10 client.18054 handle_command_reply: tid=0 2024-09-19T08:57:42.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.673+0000 7fa77bcc1740 1 client.18054 shutdown 2024-09-19T08:57:42.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.673+0000 7fa77bcc1740 1 -- 172.21.15.164:0/224106670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564aa3f2b40 msgr2=0x5564aa3f8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:42.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.673+0000 7fa77bcc1740 1 --2- 172.21.15.164:0/224106670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564aa3f2b40 0x5564aa3f8fe0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fa7640129a0 tx=0x7fa764012e60 comp rx=0 tx=0).stop 2024-09-19T08:57:42.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.673+0000 7fa77bcc1740 1 -- 172.21.15.164:0/224106670 shutdown_connections 2024-09-19T08:57:42.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.673+0000 7fa77bcc1740 1 --2- 172.21.15.164:0/224106670 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x5564aa40bfe0 0x5564aa40e3f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:42.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.673+0000 7fa77bcc1740 1 --2- 172.21.15.164:0/224106670 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5564aa3fc370 0x5564aa3f95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:42.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.673+0000 7fa77bcc1740 1 --2- 172.21.15.164:0/224106670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564aa3f2b40 0x5564aa3f8fe0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:42.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.673+0000 7fa77bcc1740 1 --2- 172.21.15.164:0/224106670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564aa3f15d0 0x5564aa3fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:42.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.673+0000 7fa77bcc1740 1 -- 172.21.15.164:0/224106670 >> 172.21.15.164:0/224106670 conn(0x5564aa3ba310 msgr2=0x5564aa3f3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:42.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.673+0000 7fa752ffd640 0 client.18054 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:57:42.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.673+0000 7fa77bcc1740 1 -- 172.21.15.164:0/224106670 shutdown_connections 2024-09-19T08:57:42.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.673+0000 7fa77bcc1740 1 -- 172.21.15.164:0/224106670 wait complete. 2024-09-19T08:57:42.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.673+0000 7fa77bcc1740 20 client.18054 trim_cache size 0 max 16384 2024-09-19T08:57:42.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.686+0000 7fa77bcc1740 1 Processor -- start 2024-09-19T08:57:42.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.686+0000 7fa77bcc1740 1 -- start start 2024-09-19T08:57:42.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.686+0000 7fa77bcc1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564aa3f15d0 0x5564aa3f78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:42.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.686+0000 7fa77bcc1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564aa3f2b40 0x5564aa3f4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:42.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.686+0000 7fa77bcc1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5564aa3fc370 0x5564aa3f4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:42.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.686+0000 7fa77bcc1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa7741ab560 con 0x5564aa3f2b40 2024-09-19T08:57:42.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.686+0000 7fa77bcc1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa7741ad0d0 con 0x5564aa3f15d0 2024-09-19T08:57:42.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.686+0000 7fa77bcc1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa7741abc30 con 0x5564aa3fc370 2024-09-19T08:57:42.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.686+0000 7fa772ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564aa3f2b40 0x5564aa3f4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:42.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.686+0000 7fa772ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564aa3f2b40 0x5564aa3f4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41026/0 (socket says 172.21.15.164:41026) 2024-09-19T08:57:42.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.686+0000 7fa772ffd640 1 -- 172.21.15.164:0/4142009403 learned_addr learned my addr 172.21.15.164:0/4142009403 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:42.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.686+0000 7fa7727fc640 1 --2- 172.21.15.164:0/4142009403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564aa3f15d0 0x5564aa3f78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:42.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.686+0000 7fa7727fc640 1 --2- 172.21.15.164:0/4142009403 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5564aa3fc370 0x5564aa3f4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:42.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.686+0000 7fa772ffd640 1 -- 172.21.15.164:0/4142009403 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5564aa3fc370 msgr2=0x5564aa3f4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:42.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.686+0000 7fa772ffd640 1 --2- 172.21.15.164:0/4142009403 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5564aa3fc370 0x5564aa3f4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:42.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.686+0000 7fa772ffd640 1 -- 172.21.15.164:0/4142009403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564aa3f15d0 msgr2=0x5564aa3f78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:42.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.686+0000 7fa772ffd640 1 --2- 172.21.15.164:0/4142009403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564aa3f15d0 0x5564aa3f78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:42.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.686+0000 7fa772ffd640 1 -- 172.21.15.164:0/4142009403 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564aa3f7f90 con 0x5564aa3f2b40 2024-09-19T08:57:42.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.686+0000 7fa7727fc640 1 --2- 172.21.15.164:0/4142009403 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5564aa3fc370 0x5564aa3f4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:57:42.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.687+0000 7fa772ffd640 1 --2- 172.21.15.164:0/4142009403 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564aa3f2b40 0x5564aa3f4120 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fa764012420 tx=0x7fa764009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975a4313706cc4ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:42.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.687+0000 7fa7597fa640 1 -- 172.21.15.164:0/4142009403 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa764029070 con 0x5564aa3f2b40 2024-09-19T08:57:42.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.687+0000 7fa7597fa640 1 -- 172.21.15.164:0/4142009403 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7640124e0 con 0x5564aa3f2b40 2024-09-19T08:57:42.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.687+0000 7fa7597fa640 1 -- 172.21.15.164:0/4142009403 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa764003b90 con 0x5564aa3f2b40 2024-09-19T08:57:42.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.687+0000 7fa77bcc1740 1 -- 172.21.15.164:0/4142009403 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564aa3f2b40 msgr2=0x5564aa3f4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:42.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.687+0000 7fa77bcc1740 1 --2- 172.21.15.164:0/4142009403 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564aa3f2b40 0x5564aa3f4120 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fa764012420 tx=0x7fa764009570 comp rx=0 tx=0).stop 2024-09-19T08:57:42.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.687+0000 7fa77bcc1740 1 -- 172.21.15.164:0/4142009403 shutdown_connections 2024-09-19T08:57:42.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.687+0000 7fa77bcc1740 1 --2- 172.21.15.164:0/4142009403 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5564aa3fc370 0x5564aa3f4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:42.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.687+0000 7fa77bcc1740 1 --2- 172.21.15.164:0/4142009403 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564aa3f2b40 0x5564aa3f4120 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:42.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.687+0000 7fa77bcc1740 1 --2- 172.21.15.164:0/4142009403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564aa3f15d0 0x5564aa3f78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:42.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.687+0000 7fa77bcc1740 1 -- 172.21.15.164:0/4142009403 >> 172.21.15.164:0/4142009403 conn(0x5564aa3ba310 msgr2=0x5564aa3fb060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:42.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.688+0000 7fa77bcc1740 1 -- 172.21.15.164:0/4142009403 shutdown_connections 2024-09-19T08:57:42.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.688+0000 7fa77bcc1740 1 -- 172.21.15.164:0/4142009403 wait complete. 2024-09-19T08:57:42.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.688+0000 7fa77bcc1740 1 -- start start 2024-09-19T08:57:42.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.688+0000 7fa77bcc1740 1 Processor -- start 2024-09-19T08:57:42.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.688+0000 7fa77bcc1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564aa3f15d0 0x5564aa400510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:42.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.688+0000 7fa77bcc1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5564aa3f2b40 0x5564aa400a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:42.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.688+0000 7fa77bcc1740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564aa3fc370 0x5564aa475cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:42.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.688+0000 7fa77bcc1740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa7741aca00 con 0x5564aa3fc370 2024-09-19T08:57:42.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.688+0000 7fa77bcc1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa7741ac300 con 0x5564aa3f15d0 2024-09-19T08:57:42.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.688+0000 7fa77bcc1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa7741a95a0 con 0x5564aa3f2b40 2024-09-19T08:57:42.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.688+0000 7fa77bcc1740 10 client.0 _ll_register_callbacks cb 0x7fa764009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:57:42.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.688+0000 7fa7727fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564aa3fc370 0x5564aa475cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:42.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.689+0000 7fa7727fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564aa3fc370 0x5564aa475cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:41040/0 (socket says 172.21.15.164:41040) 2024-09-19T08:57:42.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.689+0000 7fa7727fc640 1 -- 172.21.15.164:0/856818214 learned_addr learned my addr 172.21.15.164:0/856818214 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:42.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.689+0000 7fa7727fc640 1 --2- 172.21.15.164:0/856818214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564aa3f15d0 0x5564aa400510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:42.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.689+0000 7fa772ffd640 1 --2- 172.21.15.164:0/856818214 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5564aa3f2b40 0x5564aa400a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:42.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.689+0000 7fa7727fc640 1 -- 172.21.15.164:0/856818214 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5564aa3f2b40 msgr2=0x5564aa400a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:42.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.689+0000 7fa7727fc640 1 --2- 172.21.15.164:0/856818214 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5564aa3f2b40 0x5564aa400a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:42.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.689+0000 7fa7727fc640 1 -- 172.21.15.164:0/856818214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564aa3f15d0 msgr2=0x5564aa400510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:42.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.689+0000 7fa7727fc640 1 --2- 172.21.15.164:0/856818214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564aa3f15d0 0x5564aa400510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:42.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.689+0000 7fa7727fc640 1 -- 172.21.15.164:0/856818214 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564aa401350 con 0x5564aa3fc370 2024-09-19T08:57:42.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.689+0000 7fa7727fc640 1 --2- 172.21.15.164:0/856818214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564aa3fc370 0x5564aa475cd0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fa768005ec0 tx=0x7fa76803f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ce72211c173cf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:42.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.689+0000 7fa752ffd640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:57:42.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.689+0000 7fa752ffd640 1 -- 172.21.15.164:0/856818214 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa768035040 con 0x5564aa3fc370 2024-09-19T08:57:42.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.689+0000 7fa752ffd640 1 -- 172.21.15.164:0/856818214 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7fa768031540 con 0x5564aa3fc370 2024-09-19T08:57:42.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.689+0000 7fa77bcc1740 1 -- 172.21.15.164:0/856818214 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7640124e0 con 0x5564aa3fc370 2024-09-19T08:57:42.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.689+0000 7fa752ffd640 1 -- 172.21.15.164:0/856818214 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa768063080 con 0x5564aa3fc370 2024-09-19T08:57:42.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.689+0000 7fa77bcc1740 10 client.18060 fetch_fsmap learned FSMap version 33 2024-09-19T08:57:42.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.689+0000 7fa77bcc1740 1 -- 172.21.15.164:0/856818214 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5564aa477340 con 0x5564aa3fc370 2024-09-19T08:57:42.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.689+0000 7fa752ffd640 1 -- 172.21.15.164:0/856818214 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7640124e0 con 0x5564aa3fc370 2024-09-19T08:57:42.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.690+0000 7fa752ffd640 1 -- 172.21.15.164:0/856818214 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 33) ==== 8366+0+0 (secure 0 0 0) 0x7fa7680397d0 con 0x5564aa3fc370 2024-09-19T08:57:42.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.690+0000 7fa77bcc1740 10 client.18060 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T08:57:42.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.690+0000 7fa77bcc1740 10 client.18060 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T08:57:42.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.690+0000 7fa77bcc1740 20 client.18060 populate_metadata read hostname 'smithi164' 2024-09-19T08:57:42.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.690+0000 7fa77bcc1740 1 --2- 172.21.15.164:0/856818214 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x5564aa477620 0x5564aa479a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:42.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.690+0000 7fa77bcc1740 4 client.18060 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:57:42.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.690+0000 7fa77bcc1740 1 -- 172.21.15.164:0/856818214 --> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5564aa35ed70 con 0x5564aa477620 2024-09-19T08:57:42.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.690+0000 7fa772ffd640 1 --2- 172.21.15.164:0/856818214 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x5564aa477620 0x5564aa479a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:42.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.690+0000 7fa772ffd640 1 --2- 172.21.15.164:0/856818214 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x5564aa477620 0x5564aa479a10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9353d98dfe92efb6 server_cookie=cd737da57cb16592 in_seq=0 out_seq=0 2024-09-19T08:57:42.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.690+0000 7fa752ffd640 10 client.18060 ms_handle_connect on v2:172.21.15.173:6836/2431090203 2024-09-19T08:57:42.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.691+0000 7fa752ffd640 1 -- 172.21.15.164:0/856818214 <== mds.3 v2:172.21.15.173:6836/2431090203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5564aa35ed70 con 0x5564aa477620 2024-09-19T08:57:42.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.691+0000 7fa752ffd640 10 client.18060 handle_command_reply: tid=0 2024-09-19T08:57:42.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.691+0000 7fa77bcc1740 1 client.18060 shutdown 2024-09-19T08:57:42.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.691+0000 7fa77bcc1740 1 -- 172.21.15.164:0/856818214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564aa3fc370 msgr2=0x5564aa475cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:42.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.691+0000 7fa77bcc1740 1 --2- 172.21.15.164:0/856818214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564aa3fc370 0x5564aa475cd0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fa768005ec0 tx=0x7fa76803f040 comp rx=0 tx=0).stop 2024-09-19T08:57:42.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.691+0000 7fa77bcc1740 1 -- 172.21.15.164:0/856818214 shutdown_connections 2024-09-19T08:57:42.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.691+0000 7fa77bcc1740 1 --2- 172.21.15.164:0/856818214 >> [v2:172.21.15.173:6836/2431090203,v1:172.21.15.173:6837/2431090203] conn(0x5564aa477620 0x5564aa479a10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:42.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.691+0000 7fa77bcc1740 1 --2- 172.21.15.164:0/856818214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564aa3fc370 0x5564aa475cd0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:42.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.691+0000 7fa77bcc1740 1 --2- 172.21.15.164:0/856818214 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5564aa3f2b40 0x5564aa400a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:42.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.691+0000 7fa77bcc1740 1 --2- 172.21.15.164:0/856818214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564aa3f15d0 0x5564aa400510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:42.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.691+0000 7fa752ffd640 0 client.18060 ms_handle_reset on v2:172.21.15.173:6836/2431090203 2024-09-19T08:57:42.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.691+0000 7fa77bcc1740 1 -- 172.21.15.164:0/856818214 >> 172.21.15.164:0/856818214 conn(0x5564aa3ba310 msgr2=0x5564aa3ec4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:42.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.691+0000 7fa77bcc1740 1 -- 172.21.15.164:0/856818214 shutdown_connections 2024-09-19T08:57:42.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.691+0000 7fa77bcc1740 1 -- 172.21.15.164:0/856818214 wait complete. 2024-09-19T08:57:42.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.691+0000 7fa77bcc1740 20 client.18060 trim_cache size 0 max 16384 2024-09-19T08:57:42.697 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:57:42.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.691+0000 7fa7798e8640 1 -- 172.21.15.164:0/3275939839 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa75c0a33b0 msgr2=0x7fa75c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:42.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.691+0000 7fa7798e8640 1 --2- 172.21.15.164:0/3275939839 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa75c0a33b0 0x7fa75c0a5870 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0ccf10 tx=0x7fa76c090ed0 comp rx=0 tx=0).stop 2024-09-19T08:57:42.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.691+0000 7fa7798e8640 1 -- 172.21.15.164:0/3275939839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa774102ef0 msgr2=0x7fa77419ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:42.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.691+0000 7fa7798e8640 1 --2- 172.21.15.164:0/3275939839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa774102ef0 0x7fa77419ef80 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00d750 tx=0x7fa76c00dc10 comp rx=0 tx=0).stop 2024-09-19T08:57:42.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.692+0000 7fa7798e8640 1 -- 172.21.15.164:0/3275939839 shutdown_connections 2024-09-19T08:57:42.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.692+0000 7fa7798e8640 1 --2- 172.21.15.164:0/3275939839 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa75c0a33b0 0x7fa75c0a5870 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:42.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.692+0000 7fa7798e8640 1 --2- 172.21.15.164:0/3275939839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa774102ef0 0x7fa77419ef80 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:42.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.692+0000 7fa7798e8640 1 --2- 172.21.15.164:0/3275939839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa774102570 0x7fa77419abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:42.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.692+0000 7fa7798e8640 1 --2- 172.21.15.164:0/3275939839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa774100e20 0x7fa77419a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:42.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.692+0000 7fa7798e8640 1 -- 172.21.15.164:0/3275939839 >> 172.21.15.164:0/3275939839 conn(0x7fa7740fc7d0 msgr2=0x7fa77406a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:42.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.692+0000 7fa7798e8640 1 -- 172.21.15.164:0/3275939839 shutdown_connections 2024-09-19T08:57:42.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:42.692+0000 7fa7798e8640 1 -- 172.21.15.164:0/3275939839 wait complete. 2024-09-19T08:57:43.433 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T08:57:43.682 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:57:44.115 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:43 smithi164 ceph-mon[30928]: pgmap v372: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-19T08:57:44.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.129+0000 7f7780842640 1 -- 172.21.15.164:0/3540888794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f777c102790 msgr2=0x7f777c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:44.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.129+0000 7f7780842640 1 --2- 172.21.15.164:0/3540888794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f777c102790 0x7f777c1067e0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f7770009900 tx=0x7f777002a410 comp rx=0 tx=0).stop 2024-09-19T08:57:44.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.129+0000 7f7780842640 1 -- 172.21.15.164:0/3540888794 shutdown_connections 2024-09-19T08:57:44.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.129+0000 7f7780842640 1 --2- 172.21.15.164:0/3540888794 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f777c106f10 0x7f777c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.129+0000 7f7780842640 1 --2- 172.21.15.164:0/3540888794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f777c102790 0x7f777c1067e0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.129+0000 7f7780842640 1 --2- 172.21.15.164:0/3540888794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f777c101de0 0x7f777c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.129+0000 7f7780842640 1 -- 172.21.15.164:0/3540888794 >> 172.21.15.164:0/3540888794 conn(0x7f777c0fd8c0 msgr2=0x7f777c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:44.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.130+0000 7f7780842640 1 -- 172.21.15.164:0/3540888794 shutdown_connections 2024-09-19T08:57:44.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.130+0000 7f7780842640 1 -- 172.21.15.164:0/3540888794 wait complete. 2024-09-19T08:57:44.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.130+0000 7f7780842640 1 Processor -- start 2024-09-19T08:57:44.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.130+0000 7f7780842640 1 -- start start 2024-09-19T08:57:44.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.131+0000 7f7780842640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f777c101de0 0x7f777c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:44.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.131+0000 7f7780842640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f777c102790 0x7f777c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:44.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.131+0000 7f7780842640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f777c106f10 0x7f777c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:44.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.131+0000 7f7780842640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f777c111f70 con 0x7f777c102790 2024-09-19T08:57:44.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.131+0000 7f7780842640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f777c111df0 con 0x7f777c106f10 2024-09-19T08:57:44.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.131+0000 7f7780842640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f777c1120f0 con 0x7f777c101de0 2024-09-19T08:57:44.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.131+0000 7f7779d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f777c102790 0x7f777c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:44.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.131+0000 7f7779d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f777c102790 0x7f777c1989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60506/0 (socket says 172.21.15.164:60506) 2024-09-19T08:57:44.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.131+0000 7f7779d9b640 1 -- 172.21.15.164:0/1627610902 learned_addr learned my addr 172.21.15.164:0/1627610902 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:44.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.131+0000 7f777a59c640 1 --2- 172.21.15.164:0/1627610902 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f777c101de0 0x7f777c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:44.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.131+0000 7f777ad9d640 1 --2- 172.21.15.164:0/1627610902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f777c106f10 0x7f777c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:44.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.131+0000 7f7779d9b640 1 -- 172.21.15.164:0/1627610902 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f777c101de0 msgr2=0x7f777c198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:44.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.131+0000 7f7779d9b640 1 --2- 172.21.15.164:0/1627610902 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f777c101de0 0x7f777c198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.131+0000 7f7779d9b640 1 -- 172.21.15.164:0/1627610902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f777c106f10 msgr2=0x7f777c19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:44.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.131+0000 7f7779d9b640 1 --2- 172.21.15.164:0/1627610902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f777c106f10 0x7f777c19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.131+0000 7f7779d9b640 1 -- 172.21.15.164:0/1627610902 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f777c19d440 con 0x7f777c102790 2024-09-19T08:57:44.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.131+0000 7f7779d9b640 1 --2- 172.21.15.164:0/1627610902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f777c102790 0x7f777c1989d0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f7770006010 tx=0x7f77700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:44.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.131+0000 7f776b7fe640 1 -- 172.21.15.164:0/1627610902 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777003f030 con 0x7f777c102790 2024-09-19T08:57:44.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.131+0000 7f7780842640 1 -- 172.21.15.164:0/1627610902 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f777c19d6d0 con 0x7f777c102790 2024-09-19T08:57:44.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.131+0000 7f776b7fe640 1 -- 172.21.15.164:0/1627610902 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7770003ce0 con 0x7f777c102790 2024-09-19T08:57:44.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.132+0000 7f776b7fe640 1 -- 172.21.15.164:0/1627610902 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7770035070 con 0x7f777c102790 2024-09-19T08:57:44.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.132+0000 7f7780842640 1 -- 172.21.15.164:0/1627610902 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f777c1a5010 con 0x7f777c102790 2024-09-19T08:57:44.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.134+0000 7f776b7fe640 1 -- 172.21.15.164:0/1627610902 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f7770042030 con 0x7f777c102790 2024-09-19T08:57:44.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.134+0000 7f7782c1b740 1 Processor -- start 2024-09-19T08:57:44.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.134+0000 7f7782c1b740 1 -- start start 2024-09-19T08:57:44.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.135+0000 7f7782c1b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e962aab40 0x561e962a9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:44.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.135+0000 7f7782c1b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e962a95d0 0x561e962a9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:44.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.135+0000 7f7782c1b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e962b4370 0x561e962b6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:44.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.135+0000 7f7782c1b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561e9620f2f0 con 0x561e962a95d0 2024-09-19T08:57:44.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.135+0000 7f7782c1b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561e961b6cf0 con 0x561e962aab40 2024-09-19T08:57:44.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.135+0000 7f7782c1b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x561e9620b2b0 con 0x561e962b4370 2024-09-19T08:57:44.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.135+0000 7f776b7fe640 1 --2- 172.21.15.164:0/1627610902 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f77640a33b0 0x7f77640a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:44.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.135+0000 7f777a59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e962a95d0 0x561e962a9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:44.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.135+0000 7f777a59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e962a95d0 0x561e962a9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60510/0 (socket says 172.21.15.164:60510) 2024-09-19T08:57:44.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.135+0000 7f777a59c640 1 -- 172.21.15.164:0/2436350808 learned_addr learned my addr 172.21.15.164:0/2436350808 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:44.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.135+0000 7f777ad9d640 1 --2- 172.21.15.164:0/2436350808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e962aab40 0x561e962a9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:44.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.135+0000 7f777ad9d640 1 --2- 172.21.15.164:0/1627610902 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f77640a33b0 0x7f77640a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:44.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.135+0000 7f7779d9b640 1 --2- 172.21.15.164:0/2436350808 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e962b4370 0x561e962b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:44.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.135+0000 7f777a59c640 1 -- 172.21.15.164:0/2436350808 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e962b4370 msgr2=0x561e962b6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:44.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.135+0000 7f776b7fe640 1 -- 172.21.15.164:0/1627610902 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7f77700f0270 con 0x7f777c102790 2024-09-19T08:57:44.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.135+0000 7f777a59c640 1 --2- 172.21.15.164:0/2436350808 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e962b4370 0x561e962b6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.135+0000 7f777a59c640 1 -- 172.21.15.164:0/2436350808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e962aab40 msgr2=0x561e962a9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:44.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.135+0000 7f777a59c640 1 --2- 172.21.15.164:0/2436350808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e962aab40 0x561e962a9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.135+0000 7f777a59c640 1 -- 172.21.15.164:0/2436350808 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e9619b3a0 con 0x561e962a95d0 2024-09-19T08:57:44.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.135+0000 7f7779d9b640 1 --2- 172.21.15.164:0/2436350808 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e962b4370 0x561e962b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:57:44.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.135+0000 7f777ad9d640 1 --2- 172.21.15.164:0/2436350808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e962aab40 0x561e962a9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:57:44.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.135+0000 7f777a59c640 1 --2- 172.21.15.164:0/2436350808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e962a95d0 0x561e962a9a10 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f776c009cf0 tx=0x7f776c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b02b6a95276894 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:44.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.135+0000 7f777ad9d640 1 --2- 172.21.15.164:0/1627610902 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f77640a33b0 0x7f77640a5870 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7774009fd0 tx=0x7f7774009600 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:44.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.136+0000 7f7768ff9640 1 -- 172.21.15.164:0/2436350808 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c004230 con 0x561e962a95d0 2024-09-19T08:57:44.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.136+0000 7f7768ff9640 1 -- 172.21.15.164:0/2436350808 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f776c0043d0 con 0x561e962a95d0 2024-09-19T08:57:44.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.136+0000 7f7768ff9640 1 -- 172.21.15.164:0/2436350808 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c019960 con 0x561e962a95d0 2024-09-19T08:57:44.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.136+0000 7f7782c1b740 1 -- 172.21.15.164:0/2436350808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e962a95d0 msgr2=0x561e962a9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:44.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.136+0000 7f7782c1b740 1 --2- 172.21.15.164:0/2436350808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e962a95d0 0x561e962a9a10 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f776c009cf0 tx=0x7f776c011810 comp rx=0 tx=0).stop 2024-09-19T08:57:44.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.136+0000 7f7782c1b740 1 -- 172.21.15.164:0/2436350808 shutdown_connections 2024-09-19T08:57:44.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.136+0000 7f7782c1b740 1 --2- 172.21.15.164:0/2436350808 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e962b4370 0x561e962b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.136+0000 7f7782c1b740 1 --2- 172.21.15.164:0/2436350808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e962a95d0 0x561e962a9a10 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.136+0000 7f7782c1b740 1 --2- 172.21.15.164:0/2436350808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e962aab40 0x561e962a9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.136+0000 7f7782c1b740 1 -- 172.21.15.164:0/2436350808 >> 172.21.15.164:0/2436350808 conn(0x561e96272310 msgr2=0x561e962a5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:44.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.137+0000 7f7782c1b740 1 -- 172.21.15.164:0/2436350808 shutdown_connections 2024-09-19T08:57:44.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.137+0000 7f7782c1b740 1 -- 172.21.15.164:0/2436350808 wait complete. 2024-09-19T08:57:44.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.137+0000 7f7782c1b740 1 -- start start 2024-09-19T08:57:44.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.137+0000 7f7782c1b740 1 Processor -- start 2024-09-19T08:57:44.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.137+0000 7f7782c1b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e962a95d0 0x561e962b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:44.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.137+0000 7f7782c1b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e962aab40 0x561e962b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:44.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.137+0000 7f7782c1b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e962b4370 0x561e962b1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:44.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.137+0000 7f7782c1b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561e962b7070 con 0x561e962b4370 2024-09-19T08:57:44.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.137+0000 7f7782c1b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561e961ca060 con 0x561e962aab40 2024-09-19T08:57:44.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.137+0000 7f7782c1b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x561e962b71f0 con 0x561e962a95d0 2024-09-19T08:57:44.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.137+0000 7f7782c1b740 10 client.0 _ll_register_callbacks cb 0x561e95f1d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:57:44.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.137+0000 7f777ad9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e962b4370 0x561e962b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:44.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.137+0000 7f777ad9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e962b4370 0x561e962b1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60518/0 (socket says 172.21.15.164:60518) 2024-09-19T08:57:44.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.137+0000 7f777ad9d640 1 -- 172.21.15.164:0/1509501305 learned_addr learned my addr 172.21.15.164:0/1509501305 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:44.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.138+0000 7f777a59c640 1 --2- 172.21.15.164:0/1509501305 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e962a95d0 0x561e962b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:44.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.138+0000 7f7779d9b640 1 --2- 172.21.15.164:0/1509501305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e962aab40 0x561e962b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:44.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.138+0000 7f777ad9d640 1 -- 172.21.15.164:0/1509501305 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e962a95d0 msgr2=0x561e962b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:44.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.138+0000 7f777ad9d640 1 --2- 172.21.15.164:0/1509501305 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e962a95d0 0x561e962b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.138+0000 7f777ad9d640 1 -- 172.21.15.164:0/1509501305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e962aab40 msgr2=0x561e962b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:44.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.138+0000 7f777ad9d640 1 --2- 172.21.15.164:0/1509501305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e962aab40 0x561e962b0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.138+0000 7f777ad9d640 1 -- 172.21.15.164:0/1509501305 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e95f803c0 con 0x561e962b4370 2024-09-19T08:57:44.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.138+0000 7f777ad9d640 1 --2- 172.21.15.164:0/1509501305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e962b4370 0x561e962b1530 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f7774003c10 tx=0x7f7774004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d5e7ff858f9610 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:44.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.138+0000 7f7761ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:57:44.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.138+0000 7f7761ffb640 1 -- 172.21.15.164:0/1509501305 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7774024030 con 0x561e962b4370 2024-09-19T08:57:44.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.138+0000 7f7761ffb640 1 -- 172.21.15.164:0/1509501305 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7f77740045b0 con 0x561e962b4370 2024-09-19T08:57:44.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.138+0000 7f7782c1b740 1 -- 172.21.15.164:0/1509501305 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f776c0043d0 con 0x561e962b4370 2024-09-19T08:57:44.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.138+0000 7f7761ffb640 1 -- 172.21.15.164:0/1509501305 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7774003e50 con 0x561e962b4370 2024-09-19T08:57:44.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.138+0000 7f7782c1b740 10 client.18072 fetch_fsmap learned FSMap version 33 2024-09-19T08:57:44.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.138+0000 7f7782c1b740 1 -- 172.21.15.164:0/1509501305 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561e962c2a70 con 0x561e962b4370 2024-09-19T08:57:44.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.138+0000 7f7761ffb640 1 -- 172.21.15.164:0/1509501305 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f776c0043d0 con 0x561e962b4370 2024-09-19T08:57:44.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.139+0000 7f7761ffb640 1 -- 172.21.15.164:0/1509501305 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 33) ==== 8366+0+0 (secure 0 0 0) 0x7f777401e040 con 0x561e962b4370 2024-09-19T08:57:44.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.139+0000 7f7782c1b740 10 client.18072 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T08:57:44.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.139+0000 7f7782c1b740 10 client.18072 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T08:57:44.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.139+0000 7f7782c1b740 20 client.18072 populate_metadata read hostname 'smithi164' 2024-09-19T08:57:44.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.139+0000 7f7782c1b740 1 --2- 172.21.15.164:0/1509501305 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x561e962c3e80 0x561e962c6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:44.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.139+0000 7f7782c1b740 4 client.18072 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:57:44.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.139+0000 7f7782c1b740 1 -- 172.21.15.164:0/1509501305 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e9621c860 con 0x561e962c3e80 2024-09-19T08:57:44.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.139+0000 7f777a59c640 1 --2- 172.21.15.164:0/1509501305 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x561e962c3e80 0x561e962c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:44.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.139+0000 7f777a59c640 1 --2- 172.21.15.164:0/1509501305 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x561e962c3e80 0x561e962c6320 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e0bd0f723591859d server_cookie=1fa289f2d1939b58 in_seq=0 out_seq=0 2024-09-19T08:57:44.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.139+0000 7f7761ffb640 10 client.18072 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:57:44.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.140+0000 7f7761ffb640 1 -- 172.21.15.164:0/1509501305 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561e9621c860 con 0x561e962c3e80 2024-09-19T08:57:44.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.140+0000 7f7761ffb640 10 client.18072 handle_command_reply: tid=0 2024-09-19T08:57:44.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.140+0000 7f7782c1b740 1 client.18072 shutdown 2024-09-19T08:57:44.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.140+0000 7f7782c1b740 1 -- 172.21.15.164:0/1509501305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e962b4370 msgr2=0x561e962b1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:44.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.140+0000 7f7782c1b740 1 --2- 172.21.15.164:0/1509501305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e962b4370 0x561e962b1530 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f7774003c10 tx=0x7f7774004ab0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.140+0000 7f7782c1b740 1 -- 172.21.15.164:0/1509501305 shutdown_connections 2024-09-19T08:57:44.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.140+0000 7f7782c1b740 1 --2- 172.21.15.164:0/1509501305 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x561e962c3e80 0x561e962c6320 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.140+0000 7f7782c1b740 1 --2- 172.21.15.164:0/1509501305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e962b4370 0x561e962b1530 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.140+0000 7f7782c1b740 1 --2- 172.21.15.164:0/1509501305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e962aab40 0x561e962b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.140+0000 7f7782c1b740 1 --2- 172.21.15.164:0/1509501305 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e962a95d0 0x561e962b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.140+0000 7f7761ffb640 0 client.18072 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:57:44.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.140+0000 7f7782c1b740 1 -- 172.21.15.164:0/1509501305 >> 172.21.15.164:0/1509501305 conn(0x561e96272310 msgr2=0x561e962ab930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:44.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.141+0000 7f7782c1b740 1 -- 172.21.15.164:0/1509501305 shutdown_connections 2024-09-19T08:57:44.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.141+0000 7f7782c1b740 1 -- 172.21.15.164:0/1509501305 wait complete. 2024-09-19T08:57:44.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.141+0000 7f7782c1b740 20 client.18072 trim_cache size 0 max 16384 2024-09-19T08:57:44.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.153+0000 7f7782c1b740 1 Processor -- start 2024-09-19T08:57:44.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.153+0000 7f7782c1b740 1 -- start start 2024-09-19T08:57:44.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.153+0000 7f7782c1b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e962a95d0 0x561e962b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:44.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.153+0000 7f7782c1b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e962aab40 0x561e962ac120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:44.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.153+0000 7f7782c1b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e962b4370 0x561e962ac660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:44.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.153+0000 7f7782c1b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f777c1ad7b0 con 0x561e962b4370 2024-09-19T08:57:44.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.154+0000 7f7782c1b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f777c1af320 con 0x561e962a95d0 2024-09-19T08:57:44.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.154+0000 7f7782c1b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f777c1ade80 con 0x561e962aab40 2024-09-19T08:57:44.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.154+0000 7f777ad9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e962b4370 0x561e962ac660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:44.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.154+0000 7f777ad9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e962b4370 0x561e962ac660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60534/0 (socket says 172.21.15.164:60534) 2024-09-19T08:57:44.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.154+0000 7f777ad9d640 1 -- 172.21.15.164:0/1452070328 learned_addr learned my addr 172.21.15.164:0/1452070328 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:44.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.154+0000 7f777a59c640 1 --2- 172.21.15.164:0/1452070328 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e962a95d0 0x561e962b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:44.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.154+0000 7f777ad9d640 1 -- 172.21.15.164:0/1452070328 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e962aab40 msgr2=0x561e962ac120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:44.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.154+0000 7f777ad9d640 1 --2- 172.21.15.164:0/1452070328 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e962aab40 0x561e962ac120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.154+0000 7f777ad9d640 1 -- 172.21.15.164:0/1452070328 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e962a95d0 msgr2=0x561e962b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:44.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.154+0000 7f777ad9d640 1 --2- 172.21.15.164:0/1452070328 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e962a95d0 0x561e962b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.154+0000 7f777ad9d640 1 -- 172.21.15.164:0/1452070328 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e962aff60 con 0x561e962b4370 2024-09-19T08:57:44.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.154+0000 7f777a59c640 1 --2- 172.21.15.164:0/1452070328 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e962a95d0 0x561e962b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:57:44.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.154+0000 7f777ad9d640 1 --2- 172.21.15.164:0/1452070328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e962b4370 0x561e962ac660 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f7774020490 tx=0x7f77740096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35aab4edddeed125 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:44.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.154+0000 7f7763fff640 1 -- 172.21.15.164:0/1452070328 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7774037020 con 0x561e962b4370 2024-09-19T08:57:44.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.154+0000 7f7763fff640 1 -- 172.21.15.164:0/1452070328 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77740048e0 con 0x561e962b4370 2024-09-19T08:57:44.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.154+0000 7f7763fff640 1 -- 172.21.15.164:0/1452070328 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7774005050 con 0x561e962b4370 2024-09-19T08:57:44.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.155+0000 7f7782c1b740 1 -- 172.21.15.164:0/1452070328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e962b4370 msgr2=0x561e962ac660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:44.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.155+0000 7f7782c1b740 1 --2- 172.21.15.164:0/1452070328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e962b4370 0x561e962ac660 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f7774020490 tx=0x7f77740096f0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.155+0000 7f7782c1b740 1 -- 172.21.15.164:0/1452070328 shutdown_connections 2024-09-19T08:57:44.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.155+0000 7f7782c1b740 1 --2- 172.21.15.164:0/1452070328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e962b4370 0x561e962ac660 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.155+0000 7f7782c1b740 1 --2- 172.21.15.164:0/1452070328 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e962aab40 0x561e962ac120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.155+0000 7f7782c1b740 1 --2- 172.21.15.164:0/1452070328 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e962a95d0 0x561e962b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.155+0000 7f7782c1b740 1 -- 172.21.15.164:0/1452070328 >> 172.21.15.164:0/1452070328 conn(0x561e96272310 msgr2=0x561e962b2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:44.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.155+0000 7f7782c1b740 1 -- 172.21.15.164:0/1452070328 shutdown_connections 2024-09-19T08:57:44.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.155+0000 7f7782c1b740 1 -- 172.21.15.164:0/1452070328 wait complete. 2024-09-19T08:57:44.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.155+0000 7f7782c1b740 1 -- start start 2024-09-19T08:57:44.161 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.155+0000 7f7782c1b740 1 Processor -- start 2024-09-19T08:57:44.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.156+0000 7f7782c1b740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e962a95d0 0x561e962b81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:44.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.156+0000 7f7782c1b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e962aab40 0x561e962b8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:44.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.156+0000 7f7782c1b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e962b4370 0x561e962b8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:44.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.156+0000 7f7782c1b740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f777c1aec50 con 0x561e962a95d0 2024-09-19T08:57:44.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.156+0000 7f7782c1b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f777c1ae550 con 0x561e962b4370 2024-09-19T08:57:44.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.156+0000 7f7782c1b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f777c1ab7b0 con 0x561e962aab40 2024-09-19T08:57:44.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.156+0000 7f7782c1b740 10 client.0 _ll_register_callbacks cb 0x7f7774012640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:57:44.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.156+0000 7f777a59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e962a95d0 0x561e962b81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:44.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.156+0000 7f777a59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e962a95d0 0x561e962b81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60550/0 (socket says 172.21.15.164:60550) 2024-09-19T08:57:44.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.156+0000 7f777a59c640 1 -- 172.21.15.164:0/3279372750 learned_addr learned my addr 172.21.15.164:0/3279372750 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:44.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.156+0000 7f777ad9d640 1 --2- 172.21.15.164:0/3279372750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e962b4370 0x561e962b8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:44.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.156+0000 7f777a59c640 1 -- 172.21.15.164:0/3279372750 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e962aab40 msgr2=0x561e962b8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:44.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.156+0000 7f777a59c640 1 --2- 172.21.15.164:0/3279372750 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e962aab40 0x561e962b8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.156+0000 7f777a59c640 1 -- 172.21.15.164:0/3279372750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e962b4370 msgr2=0x561e962b8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:44.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.156+0000 7f777a59c640 1 --2- 172.21.15.164:0/3279372750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e962b4370 0x561e962b8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.156+0000 7f777a59c640 1 -- 172.21.15.164:0/3279372750 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e9632dc10 con 0x561e962a95d0 2024-09-19T08:57:44.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.156+0000 7f777ad9d640 1 --2- 172.21.15.164:0/3279372750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e962b4370 0x561e962b8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:57:44.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.156+0000 7f777a59c640 1 --2- 172.21.15.164:0/3279372750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e962a95d0 0x561e962b81e0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f776c000c00 tx=0x7f776c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e23cb4fa79f698 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:44.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.156+0000 7f7761ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:57:44.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.156+0000 7f7761ffb640 1 -- 172.21.15.164:0/3279372750 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c005250 con 0x561e962a95d0 2024-09-19T08:57:44.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.156+0000 7f7761ffb640 1 -- 172.21.15.164:0/3279372750 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7f776c0053f0 con 0x561e962a95d0 2024-09-19T08:57:44.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.156+0000 7f7782c1b740 1 -- 172.21.15.164:0/3279372750 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77740048e0 con 0x561e962a95d0 2024-09-19T08:57:44.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.157+0000 7f7761ffb640 1 -- 172.21.15.164:0/3279372750 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f776c039050 con 0x561e962a95d0 2024-09-19T08:57:44.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.157+0000 7f7782c1b740 10 client.18078 fetch_fsmap learned FSMap version 33 2024-09-19T08:57:44.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.157+0000 7f7782c1b740 1 -- 172.21.15.164:0/3279372750 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561e9632ee10 con 0x561e962a95d0 2024-09-19T08:57:44.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.157+0000 7f7761ffb640 1 -- 172.21.15.164:0/3279372750 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77740048e0 con 0x561e962a95d0 2024-09-19T08:57:44.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.157+0000 7f7761ffb640 1 -- 172.21.15.164:0/3279372750 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 33) ==== 8366+0+0 (secure 0 0 0) 0x7f776c0197a0 con 0x561e962a95d0 2024-09-19T08:57:44.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.157+0000 7f7782c1b740 10 client.18078 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T08:57:44.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.157+0000 7f7782c1b740 10 client.18078 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T08:57:44.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.157+0000 7f7782c1b740 20 client.18078 populate_metadata read hostname 'smithi164' 2024-09-19T08:57:44.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.157+0000 7f7782c1b740 1 --2- 172.21.15.164:0/3279372750 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x561e9632f3f0 0x561e96331800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:44.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.157+0000 7f7782c1b740 4 client.18078 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:57:44.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.157+0000 7f7782c1b740 1 -- 172.21.15.164:0/3279372750 --> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561e96216d70 con 0x561e9632f3f0 2024-09-19T08:57:44.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.157+0000 7f7779d9b640 1 --2- 172.21.15.164:0/3279372750 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x561e9632f3f0 0x561e96331800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:44.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.158+0000 7f7779d9b640 1 --2- 172.21.15.164:0/3279372750 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x561e9632f3f0 0x561e96331800 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c6a5312ef3eea0d6 server_cookie=ca6e55f0f9e28acc in_seq=0 out_seq=0 2024-09-19T08:57:44.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.158+0000 7f7761ffb640 10 client.18078 ms_handle_connect on v2:172.21.15.173:6838/1089658493 2024-09-19T08:57:44.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.158+0000 7f7761ffb640 1 -- 172.21.15.164:0/3279372750 <== mds.2 v2:172.21.15.173:6838/1089658493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561e96216d70 con 0x561e9632f3f0 2024-09-19T08:57:44.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.158+0000 7f7761ffb640 10 client.18078 handle_command_reply: tid=0 2024-09-19T08:57:44.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.158+0000 7f7782c1b740 1 client.18078 shutdown 2024-09-19T08:57:44.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.158+0000 7f7782c1b740 1 -- 172.21.15.164:0/3279372750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e962a95d0 msgr2=0x561e962b81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:44.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.158+0000 7f7782c1b740 1 --2- 172.21.15.164:0/3279372750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e962a95d0 0x561e962b81e0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f776c000c00 tx=0x7f776c009310 comp rx=0 tx=0).stop 2024-09-19T08:57:44.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.158+0000 7f7782c1b740 1 -- 172.21.15.164:0/3279372750 shutdown_connections 2024-09-19T08:57:44.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.158+0000 7f7782c1b740 1 --2- 172.21.15.164:0/3279372750 >> [v2:172.21.15.173:6838/1089658493,v1:172.21.15.173:6839/1089658493] conn(0x561e9632f3f0 0x561e96331800 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.158+0000 7f7782c1b740 1 --2- 172.21.15.164:0/3279372750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e962b4370 0x561e962b8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.158+0000 7f7782c1b740 1 --2- 172.21.15.164:0/3279372750 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561e962aab40 0x561e962b8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.158+0000 7f7782c1b740 1 --2- 172.21.15.164:0/3279372750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e962a95d0 0x561e962b81e0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.158+0000 7f7782c1b740 1 -- 172.21.15.164:0/3279372750 >> 172.21.15.164:0/3279372750 conn(0x561e96272310 msgr2=0x561e962bf300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:44.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.158+0000 7f7761ffb640 0 client.18078 ms_handle_reset on v2:172.21.15.173:6838/1089658493 2024-09-19T08:57:44.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.158+0000 7f7782c1b740 1 -- 172.21.15.164:0/3279372750 shutdown_connections 2024-09-19T08:57:44.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.158+0000 7f7782c1b740 1 -- 172.21.15.164:0/3279372750 wait complete. 2024-09-19T08:57:44.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.158+0000 7f7782c1b740 20 client.18078 trim_cache size 0 max 16384 2024-09-19T08:57:44.166 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:57:44.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.159+0000 7f7780842640 1 -- 172.21.15.164:0/1627610902 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f77640a33b0 msgr2=0x7f77640a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:44.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.159+0000 7f7780842640 1 --2- 172.21.15.164:0/1627610902 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f77640a33b0 0x7f77640a5870 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7774009fd0 tx=0x7f7774009600 comp rx=0 tx=0).stop 2024-09-19T08:57:44.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.159+0000 7f7780842640 1 -- 172.21.15.164:0/1627610902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f777c102790 msgr2=0x7f777c1989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:44.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.159+0000 7f7780842640 1 --2- 172.21.15.164:0/1627610902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f777c102790 0x7f777c1989d0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f7770006010 tx=0x7f77700042f0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.159+0000 7f7780842640 1 -- 172.21.15.164:0/1627610902 shutdown_connections 2024-09-19T08:57:44.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.159+0000 7f7780842640 1 --2- 172.21.15.164:0/1627610902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f777c106f10 0x7f777c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.159+0000 7f7780842640 1 --2- 172.21.15.164:0/1627610902 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f77640a33b0 0x7f77640a5870 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.159+0000 7f7780842640 1 --2- 172.21.15.164:0/1627610902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f777c102790 0x7f777c1989d0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.159+0000 7f7780842640 1 --2- 172.21.15.164:0/1627610902 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f777c101de0 0x7f777c198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:44.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.159+0000 7f7780842640 1 -- 172.21.15.164:0/1627610902 >> 172.21.15.164:0/1627610902 conn(0x7f777c0fd8c0 msgr2=0x7f777c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:44.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.159+0000 7f7780842640 1 -- 172.21.15.164:0/1627610902 shutdown_connections 2024-09-19T08:57:44.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:44.159+0000 7f7780842640 1 -- 172.21.15.164:0/1627610902 wait complete. 2024-09-19T08:57:44.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:43 smithi170 ceph-mon[35072]: pgmap v372: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-19T08:57:44.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:43 smithi173 ceph-mon[35306]: pgmap v372: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-19T08:57:44.934 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T08:57:45.183 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:57:45.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.663+0000 7f9217d4a640 1 -- 172.21.15.164:0/2827525493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9210103670 msgr2=0x7f921010bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:45.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.663+0000 7f9217d4a640 1 --2- 172.21.15.164:0/2827525493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9210103670 0x7f921010bb60 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f9204009900 tx=0x7f920402a480 comp rx=0 tx=0).stop 2024-09-19T08:57:45.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.663+0000 7f9217d4a640 1 -- 172.21.15.164:0/2827525493 shutdown_connections 2024-09-19T08:57:45.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.663+0000 7f9217d4a640 1 --2- 172.21.15.164:0/2827525493 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f921010c180 0x7f921010e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.663+0000 7f9217d4a640 1 --2- 172.21.15.164:0/2827525493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9210103670 0x7f921010bb60 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.663+0000 7f9217d4a640 1 --2- 172.21.15.164:0/2827525493 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9210102d50 0x7f9210103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.663+0000 7f9217d4a640 1 -- 172.21.15.164:0/2827525493 >> 172.21.15.164:0/2827525493 conn(0x7f92100fc7d0 msgr2=0x7f92100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:45.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.663+0000 7f9217d4a640 1 -- 172.21.15.164:0/2827525493 shutdown_connections 2024-09-19T08:57:45.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.663+0000 7f9217d4a640 1 -- 172.21.15.164:0/2827525493 wait complete. 2024-09-19T08:57:45.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.664+0000 7f9217d4a640 1 Processor -- start 2024-09-19T08:57:45.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.664+0000 7f9217d4a640 1 -- start start 2024-09-19T08:57:45.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.664+0000 7f9217d4a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9210102d50 0x7f9210198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:45.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.664+0000 7f9217d4a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9210103670 0x7f92101989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:45.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.664+0000 7f9217d4a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f921010c180 0x7f921019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:45.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.664+0000 7f9217d4a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9210111f80 con 0x7f9210103670 2024-09-19T08:57:45.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.664+0000 7f9217d4a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9210111e00 con 0x7f921010c180 2024-09-19T08:57:45.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.664+0000 7f9217d4a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9210112100 con 0x7f9210102d50 2024-09-19T08:57:45.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.664+0000 7f92152e5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9210103670 0x7f92101989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:45.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.664+0000 7f92152e5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9210103670 0x7f92101989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60568/0 (socket says 172.21.15.164:60568) 2024-09-19T08:57:45.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.664+0000 7f92152e5640 1 -- 172.21.15.164:0/1854777148 learned_addr learned my addr 172.21.15.164:0/1854777148 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:45.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.664+0000 7f9215ae6640 1 --2- 172.21.15.164:0/1854777148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9210102d50 0x7f9210198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:45.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.664+0000 7f92162e7640 1 --2- 172.21.15.164:0/1854777148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f921010c180 0x7f921019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:45.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.664+0000 7f92152e5640 1 -- 172.21.15.164:0/1854777148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9210102d50 msgr2=0x7f9210198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:45.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.664+0000 7f92152e5640 1 --2- 172.21.15.164:0/1854777148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9210102d50 0x7f9210198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.664+0000 7f92152e5640 1 -- 172.21.15.164:0/1854777148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f921010c180 msgr2=0x7f921019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:45.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.664+0000 7f92152e5640 1 --2- 172.21.15.164:0/1854777148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f921010c180 0x7f921019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.664+0000 7f92152e5640 1 -- 172.21.15.164:0/1854777148 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f921019d410 con 0x7f9210103670 2024-09-19T08:57:45.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.665+0000 7f92152e5640 1 --2- 172.21.15.164:0/1854777148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9210103670 0x7f92101989a0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f9204006010 tx=0x7f92040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:45.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.665+0000 7f920effd640 1 -- 172.21.15.164:0/1854777148 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920403f030 con 0x7f9210103670 2024-09-19T08:57:45.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.665+0000 7f920effd640 1 -- 172.21.15.164:0/1854777148 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9204003ce0 con 0x7f9210103670 2024-09-19T08:57:45.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.665+0000 7f920effd640 1 -- 172.21.15.164:0/1854777148 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9204035070 con 0x7f9210103670 2024-09-19T08:57:45.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.665+0000 7f9217d4a640 1 -- 172.21.15.164:0/1854777148 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f921019d6a0 con 0x7f9210103670 2024-09-19T08:57:45.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.665+0000 7f9217d4a640 1 -- 172.21.15.164:0/1854777148 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92101a4fe0 con 0x7f9210103670 2024-09-19T08:57:45.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.667+0000 7f920effd640 1 -- 172.21.15.164:0/1854777148 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f9204042030 con 0x7f9210103670 2024-09-19T08:57:45.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.667+0000 7f921a123740 1 Processor -- start 2024-09-19T08:57:45.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.668+0000 7f921a123740 1 -- start start 2024-09-19T08:57:45.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.668+0000 7f921a123740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae4b0dfb40 0x55ae4b0de090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:45.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.668+0000 7f921a123740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae4b0de5d0 0x55ae4b0dea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:45.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.668+0000 7f920effd640 1 --2- 172.21.15.164:0/1854777148 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f91f80a33b0 0x7f91f80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:45.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.668+0000 7f921a123740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4b0e9370 0x55ae4b0eb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:45.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.668+0000 7f921a123740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ae4b0442f0 con 0x55ae4b0dfb40 2024-09-19T08:57:45.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.668+0000 7f921a123740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ae4afebcf0 con 0x55ae4b0e9370 2024-09-19T08:57:45.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.668+0000 7f921a123740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ae4b0402b0 con 0x55ae4b0de5d0 2024-09-19T08:57:45.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.668+0000 7f92162e7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae4b0dfb40 0x55ae4b0de090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:45.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.668+0000 7f9215ae6640 1 --2- 172.21.15.164:0/1854777148 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f91f80a33b0 0x7f91f80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:45.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.668+0000 7f92162e7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae4b0dfb40 0x55ae4b0de090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60570/0 (socket says 172.21.15.164:60570) 2024-09-19T08:57:45.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.668+0000 7f92162e7640 1 -- 172.21.15.164:0/625515860 learned_addr learned my addr 172.21.15.164:0/625515860 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:45.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.668+0000 7f920effd640 1 -- 172.21.15.164:0/1854777148 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7f92040efdb0 con 0x7f9210103670 2024-09-19T08:57:45.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.668+0000 7f92152e5640 1 --2- 172.21.15.164:0/625515860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae4b0de5d0 0x55ae4b0dea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:45.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.668+0000 7f92162e7640 1 --2- 172.21.15.164:0/625515860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4b0e9370 0x55ae4b0eb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:45.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.668+0000 7f92162e7640 1 -- 172.21.15.164:0/625515860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae4b0de5d0 msgr2=0x55ae4b0dea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:45.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.668+0000 7f92162e7640 1 --2- 172.21.15.164:0/625515860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae4b0de5d0 0x55ae4b0dea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.668+0000 7f92162e7640 1 -- 172.21.15.164:0/625515860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4b0e9370 msgr2=0x55ae4b0eb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:45.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.668+0000 7f92162e7640 1 --2- 172.21.15.164:0/625515860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4b0e9370 0x55ae4b0eb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.668+0000 7f92162e7640 1 -- 172.21.15.164:0/625515860 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae4afd03a0 con 0x55ae4b0dfb40 2024-09-19T08:57:45.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.668+0000 7f92152e5640 1 --2- 172.21.15.164:0/625515860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae4b0de5d0 0x55ae4b0dea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:57:45.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.668+0000 7f9215ae6640 1 --2- 172.21.15.164:0/1854777148 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f91f80a33b0 0x7f91f80a5870 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9200005c40 tx=0x7f920000a590 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:45.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.668+0000 7f92162e7640 1 --2- 172.21.15.164:0/625515860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae4b0dfb40 0x55ae4b0de090 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f9208004650 tx=0x7f9208004350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd724ffae1fbf2c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:45.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.669+0000 7f91f7fff640 1 -- 172.21.15.164:0/625515860 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9208003c70 con 0x55ae4b0dfb40 2024-09-19T08:57:45.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.669+0000 7f91f7fff640 1 -- 172.21.15.164:0/625515860 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9208003e10 con 0x55ae4b0dfb40 2024-09-19T08:57:45.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.669+0000 7f91f7fff640 1 -- 172.21.15.164:0/625515860 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9208011440 con 0x55ae4b0dfb40 2024-09-19T08:57:45.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.669+0000 7f921a123740 1 -- 172.21.15.164:0/625515860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae4b0dfb40 msgr2=0x55ae4b0de090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:45.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.669+0000 7f921a123740 1 --2- 172.21.15.164:0/625515860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae4b0dfb40 0x55ae4b0de090 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f9208004650 tx=0x7f9208004350 comp rx=0 tx=0).stop 2024-09-19T08:57:45.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.669+0000 7f921a123740 1 -- 172.21.15.164:0/625515860 shutdown_connections 2024-09-19T08:57:45.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.669+0000 7f921a123740 1 --2- 172.21.15.164:0/625515860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4b0e9370 0x55ae4b0eb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.669+0000 7f921a123740 1 --2- 172.21.15.164:0/625515860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae4b0de5d0 0x55ae4b0dea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.669+0000 7f921a123740 1 --2- 172.21.15.164:0/625515860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae4b0dfb40 0x55ae4b0de090 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.669+0000 7f921a123740 1 -- 172.21.15.164:0/625515860 >> 172.21.15.164:0/625515860 conn(0x55ae4b0a7310 msgr2=0x55ae4b0dac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:45.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.669+0000 7f921a123740 1 -- 172.21.15.164:0/625515860 shutdown_connections 2024-09-19T08:57:45.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.669+0000 7f921a123740 1 -- 172.21.15.164:0/625515860 wait complete. 2024-09-19T08:57:45.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.670+0000 7f921a123740 1 -- start start 2024-09-19T08:57:45.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.670+0000 7f921a123740 1 Processor -- start 2024-09-19T08:57:45.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.670+0000 7f921a123740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae4b0de5d0 0x55ae4b0e8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:45.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.670+0000 7f921a123740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4b0dfb40 0x55ae4b0e5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:45.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.670+0000 7f921a123740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae4b0e9370 0x55ae4b0e6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:45.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.670+0000 7f921a123740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ae4b0ec010 con 0x55ae4b0de5d0 2024-09-19T08:57:45.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.670+0000 7f921a123740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ae4afff060 con 0x55ae4b0dfb40 2024-09-19T08:57:45.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.670+0000 7f921a123740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ae4b0ec190 con 0x55ae4b0e9370 2024-09-19T08:57:45.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.670+0000 7f921a123740 10 client.0 _ll_register_callbacks cb 0x55ae4ad528c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:57:45.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.670+0000 7f9215ae6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae4b0de5d0 0x55ae4b0e8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:45.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.670+0000 7f9215ae6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae4b0de5d0 0x55ae4b0e8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60576/0 (socket says 172.21.15.164:60576) 2024-09-19T08:57:45.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.670+0000 7f9215ae6640 1 -- 172.21.15.164:0/810353122 learned_addr learned my addr 172.21.15.164:0/810353122 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:45.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.671+0000 7f92152e5640 1 --2- 172.21.15.164:0/810353122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4b0dfb40 0x55ae4b0e5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:45.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.671+0000 7f9215ae6640 1 -- 172.21.15.164:0/810353122 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae4b0e9370 msgr2=0x55ae4b0e6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:45.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.671+0000 7f9215ae6640 1 --2- 172.21.15.164:0/810353122 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae4b0e9370 0x55ae4b0e6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.671+0000 7f9215ae6640 1 -- 172.21.15.164:0/810353122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4b0dfb40 msgr2=0x55ae4b0e5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:45.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.671+0000 7f9215ae6640 1 --2- 172.21.15.164:0/810353122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4b0dfb40 0x55ae4b0e5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.671+0000 7f9215ae6640 1 -- 172.21.15.164:0/810353122 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae4adb53c0 con 0x55ae4b0de5d0 2024-09-19T08:57:45.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.671+0000 7f9215ae6640 1 --2- 172.21.15.164:0/810353122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae4b0de5d0 0x55ae4b0e8cf0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f9200010e30 tx=0x7f9200004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b514ea278be0acda server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:45.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.671+0000 7f91f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:57:45.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.671+0000 7f91f57fa640 1 -- 172.21.15.164:0/810353122 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9200016070 con 0x55ae4b0de5d0 2024-09-19T08:57:45.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.671+0000 7f91f57fa640 1 -- 172.21.15.164:0/810353122 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7f9200027680 con 0x55ae4b0de5d0 2024-09-19T08:57:45.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.671+0000 7f921a123740 1 -- 172.21.15.164:0/810353122 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9208003e10 con 0x55ae4b0de5d0 2024-09-19T08:57:45.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.671+0000 7f91f57fa640 1 -- 172.21.15.164:0/810353122 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f920000aad0 con 0x55ae4b0de5d0 2024-09-19T08:57:45.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.671+0000 7f921a123740 10 client.18090 fetch_fsmap learned FSMap version 33 2024-09-19T08:57:45.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.671+0000 7f921a123740 1 -- 172.21.15.164:0/810353122 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ae4b0f7aa0 con 0x55ae4b0de5d0 2024-09-19T08:57:45.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.671+0000 7f91f57fa640 1 -- 172.21.15.164:0/810353122 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9208003e10 con 0x55ae4b0de5d0 2024-09-19T08:57:45.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.671+0000 7f91f57fa640 1 -- 172.21.15.164:0/810353122 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 33) ==== 8366+0+0 (secure 0 0 0) 0x7f920001cb20 con 0x55ae4b0de5d0 2024-09-19T08:57:45.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.672+0000 7f921a123740 10 client.18090 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T08:57:45.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.672+0000 7f921a123740 10 client.18090 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T08:57:45.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.672+0000 7f921a123740 20 client.18090 populate_metadata read hostname 'smithi164' 2024-09-19T08:57:45.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.672+0000 7f921a123740 1 --2- 172.21.15.164:0/810353122 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55ae4b0f8f60 0x55ae4b0fb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:45.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.672+0000 7f921a123740 4 client.18090 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:57:45.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.672+0000 7f921a123740 1 -- 172.21.15.164:0/810353122 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae4b051860 con 0x55ae4b0f8f60 2024-09-19T08:57:45.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.672+0000 7f92152e5640 1 --2- 172.21.15.164:0/810353122 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55ae4b0f8f60 0x55ae4b0fb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:45.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.672+0000 7f92152e5640 1 --2- 172.21.15.164:0/810353122 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55ae4b0f8f60 0x55ae4b0fb420 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ffff964549cad951 server_cookie=e83bcd823b76aded in_seq=0 out_seq=0 2024-09-19T08:57:45.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.672+0000 7f91f57fa640 10 client.18090 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:57:45.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.673+0000 7f91f57fa640 1 -- 172.21.15.164:0/810353122 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ae4b051860 con 0x55ae4b0f8f60 2024-09-19T08:57:45.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.673+0000 7f91f57fa640 10 client.18090 handle_command_reply: tid=0 2024-09-19T08:57:45.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.673+0000 7f921a123740 1 client.18090 shutdown 2024-09-19T08:57:45.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.673+0000 7f921a123740 1 -- 172.21.15.164:0/810353122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae4b0de5d0 msgr2=0x55ae4b0e8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:45.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.673+0000 7f921a123740 1 --2- 172.21.15.164:0/810353122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae4b0de5d0 0x55ae4b0e8cf0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f9200010e30 tx=0x7f9200004ab0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.673+0000 7f921a123740 1 -- 172.21.15.164:0/810353122 shutdown_connections 2024-09-19T08:57:45.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.673+0000 7f921a123740 1 --2- 172.21.15.164:0/810353122 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae4b0e9370 0x55ae4b0e6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.673+0000 7f921a123740 1 --2- 172.21.15.164:0/810353122 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55ae4b0f8f60 0x55ae4b0fb420 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.673+0000 7f921a123740 1 --2- 172.21.15.164:0/810353122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4b0dfb40 0x55ae4b0e5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.673+0000 7f91f57fa640 0 client.18090 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:57:45.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.673+0000 7f921a123740 1 --2- 172.21.15.164:0/810353122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae4b0de5d0 0x55ae4b0e8cf0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.673+0000 7f921a123740 1 -- 172.21.15.164:0/810353122 >> 172.21.15.164:0/810353122 conn(0x55ae4b0a7310 msgr2=0x55ae4b0e0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:45.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.673+0000 7f921a123740 1 -- 172.21.15.164:0/810353122 shutdown_connections 2024-09-19T08:57:45.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.673+0000 7f921a123740 1 -- 172.21.15.164:0/810353122 wait complete. 2024-09-19T08:57:45.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.673+0000 7f921a123740 20 client.18090 trim_cache size 0 max 16384 2024-09-19T08:57:45.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.686+0000 7f921a123740 1 Processor -- start 2024-09-19T08:57:45.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.686+0000 7f921a123740 1 -- start start 2024-09-19T08:57:45.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.686+0000 7f921a123740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4b0de5d0 0x55ae4b0f52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:45.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.686+0000 7f921a123740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae4b0dfb40 0x55ae4b100d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:45.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.686+0000 7f921a123740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae4b0e9370 0x55ae4b101280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:45.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.686+0000 7f921a123740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f92101ad830 con 0x55ae4b0e9370 2024-09-19T08:57:45.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.686+0000 7f921a123740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f92101af3a0 con 0x55ae4b0de5d0 2024-09-19T08:57:45.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.686+0000 7f921a123740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f92101adf00 con 0x55ae4b0dfb40 2024-09-19T08:57:45.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.686+0000 7f92162e7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae4b0e9370 0x55ae4b101280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:45.702 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.686+0000 7f92162e7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae4b0e9370 0x55ae4b101280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60586/0 (socket says 172.21.15.164:60586) 2024-09-19T08:57:45.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.686+0000 7f92162e7640 1 -- 172.21.15.164:0/1183958880 learned_addr learned my addr 172.21.15.164:0/1183958880 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:45.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.687+0000 7f92152e5640 1 --2- 172.21.15.164:0/1183958880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae4b0dfb40 0x55ae4b100d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:45.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.687+0000 7f9215ae6640 1 --2- 172.21.15.164:0/1183958880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4b0de5d0 0x55ae4b0f52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:45.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.687+0000 7f92162e7640 1 -- 172.21.15.164:0/1183958880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae4b0dfb40 msgr2=0x55ae4b100d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:45.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.687+0000 7f92162e7640 1 --2- 172.21.15.164:0/1183958880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae4b0dfb40 0x55ae4b100d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.687+0000 7f92162e7640 1 -- 172.21.15.164:0/1183958880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4b0de5d0 msgr2=0x55ae4b0f52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:45.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.687+0000 7f92162e7640 1 --2- 172.21.15.164:0/1183958880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4b0de5d0 0x55ae4b0f52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.687+0000 7f92162e7640 1 -- 172.21.15.164:0/1183958880 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae4b0db960 con 0x55ae4b0e9370 2024-09-19T08:57:45.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.687+0000 7f9215ae6640 1 --2- 172.21.15.164:0/1183958880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4b0de5d0 0x55ae4b0f52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:57:45.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.687+0000 7f92162e7640 1 --2- 172.21.15.164:0/1183958880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae4b0e9370 0x55ae4b101280 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f9208003a80 tx=0x7f9208003c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=266a82029b5186ed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:45.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.687+0000 7f91f77fe640 1 -- 172.21.15.164:0/1183958880 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9208023020 con 0x55ae4b0e9370 2024-09-19T08:57:45.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.687+0000 7f91f77fe640 1 -- 172.21.15.164:0/1183958880 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9208005530 con 0x55ae4b0e9370 2024-09-19T08:57:45.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.687+0000 7f91f77fe640 1 -- 172.21.15.164:0/1183958880 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920801f6a0 con 0x55ae4b0e9370 2024-09-19T08:57:45.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.688+0000 7f921a123740 1 -- 172.21.15.164:0/1183958880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae4b0e9370 msgr2=0x55ae4b101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:45.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.688+0000 7f921a123740 1 --2- 172.21.15.164:0/1183958880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae4b0e9370 0x55ae4b101280 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f9208003a80 tx=0x7f9208003c70 comp rx=0 tx=0).stop 2024-09-19T08:57:45.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.688+0000 7f921a123740 1 -- 172.21.15.164:0/1183958880 shutdown_connections 2024-09-19T08:57:45.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.688+0000 7f921a123740 1 --2- 172.21.15.164:0/1183958880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae4b0e9370 0x55ae4b101280 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.688+0000 7f921a123740 1 --2- 172.21.15.164:0/1183958880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae4b0dfb40 0x55ae4b100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.688+0000 7f921a123740 1 --2- 172.21.15.164:0/1183958880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4b0de5d0 0x55ae4b0f52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.688+0000 7f921a123740 1 -- 172.21.15.164:0/1183958880 >> 172.21.15.164:0/1183958880 conn(0x55ae4b0a7310 msgr2=0x55ae4b0e1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:45.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.688+0000 7f921a123740 1 -- 172.21.15.164:0/1183958880 shutdown_connections 2024-09-19T08:57:45.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.688+0000 7f921a123740 1 -- 172.21.15.164:0/1183958880 wait complete. 2024-09-19T08:57:45.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.688+0000 7f921a123740 1 -- start start 2024-09-19T08:57:45.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.688+0000 7f921a123740 1 Processor -- start 2024-09-19T08:57:45.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.689+0000 7f921a123740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae4b0de5d0 0x55ae4b0ed340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:45.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.689+0000 7f921a123740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae4b0dfb40 0x55ae4b0ed880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:45.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.689+0000 7f921a123740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4b0e9370 0x55ae4b0eddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:45.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.689+0000 7f921a123740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f92101aecd0 con 0x55ae4b0de5d0 2024-09-19T08:57:45.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.689+0000 7f921a123740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f92101ae5d0 con 0x55ae4b0e9370 2024-09-19T08:57:45.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.689+0000 7f921a123740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f92101ab830 con 0x55ae4b0dfb40 2024-09-19T08:57:45.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.689+0000 7f921a123740 10 client.0 _ll_register_callbacks cb 0x7f9200010ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:57:45.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.689+0000 7f9215ae6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae4b0de5d0 0x55ae4b0ed340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:45.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.689+0000 7f9215ae6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae4b0de5d0 0x55ae4b0ed340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60594/0 (socket says 172.21.15.164:60594) 2024-09-19T08:57:45.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.689+0000 7f9215ae6640 1 -- 172.21.15.164:0/3648217012 learned_addr learned my addr 172.21.15.164:0/3648217012 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:45.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.689+0000 7f92162e7640 1 --2- 172.21.15.164:0/3648217012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4b0e9370 0x55ae4b0eddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:45.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.689+0000 7f92152e5640 1 --2- 172.21.15.164:0/3648217012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae4b0dfb40 0x55ae4b0ed880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:45.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.689+0000 7f9215ae6640 1 -- 172.21.15.164:0/3648217012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae4b0dfb40 msgr2=0x55ae4b0ed880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:45.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.689+0000 7f9215ae6640 1 --2- 172.21.15.164:0/3648217012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae4b0dfb40 0x55ae4b0ed880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.689+0000 7f9215ae6640 1 -- 172.21.15.164:0/3648217012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4b0e9370 msgr2=0x55ae4b0eddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:45.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.689+0000 7f9215ae6640 1 --2- 172.21.15.164:0/3648217012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4b0e9370 0x55ae4b0eddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.706 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.689+0000 7f9215ae6640 1 -- 172.21.15.164:0/3648217012 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae4b0da4f0 con 0x55ae4b0de5d0 2024-09-19T08:57:45.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.689+0000 7f9215ae6640 1 --2- 172.21.15.164:0/3648217012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae4b0de5d0 0x55ae4b0ed340 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f9200010e30 tx=0x7f9200019be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6b42abce23106a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:45.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.689+0000 7f91f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:57:45.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.690+0000 7f91f57fa640 1 -- 172.21.15.164:0/3648217012 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9200034040 con 0x55ae4b0de5d0 2024-09-19T08:57:45.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.690+0000 7f91f57fa640 1 -- 172.21.15.164:0/3648217012 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7f9200019e00 con 0x55ae4b0de5d0 2024-09-19T08:57:45.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.690+0000 7f921a123740 1 -- 172.21.15.164:0/3648217012 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9208005530 con 0x55ae4b0de5d0 2024-09-19T08:57:45.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.690+0000 7f91f57fa640 1 -- 172.21.15.164:0/3648217012 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f920001c420 con 0x55ae4b0de5d0 2024-09-19T08:57:45.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.690+0000 7f921a123740 10 client.18096 fetch_fsmap learned FSMap version 33 2024-09-19T08:57:45.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.690+0000 7f921a123740 1 -- 172.21.15.164:0/3648217012 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ae4b163fe0 con 0x55ae4b0de5d0 2024-09-19T08:57:45.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.690+0000 7f91f57fa640 1 -- 172.21.15.164:0/3648217012 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9208005530 con 0x55ae4b0de5d0 2024-09-19T08:57:45.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.690+0000 7f91f57fa640 1 -- 172.21.15.164:0/3648217012 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 33) ==== 8366+0+0 (secure 0 0 0) 0x7f9200037030 con 0x55ae4b0de5d0 2024-09-19T08:57:45.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.690+0000 7f921a123740 10 client.18096 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T08:57:45.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.690+0000 7f921a123740 10 client.18096 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T08:57:45.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.690+0000 7f921a123740 20 client.18096 populate_metadata read hostname 'smithi164' 2024-09-19T08:57:45.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.690+0000 7f921a123740 1 --2- 172.21.15.164:0/3648217012 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55ae4b1647b0 0x55ae4b166bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:45.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.690+0000 7f921a123740 4 client.18096 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:57:45.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.690+0000 7f921a123740 1 -- 172.21.15.164:0/3648217012 --> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ae4b04bd70 con 0x55ae4b1647b0 2024-09-19T08:57:45.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.691+0000 7f92152e5640 1 --2- 172.21.15.164:0/3648217012 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55ae4b1647b0 0x55ae4b166bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:45.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.691+0000 7f92152e5640 1 --2- 172.21.15.164:0/3648217012 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55ae4b1647b0 0x55ae4b166bd0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=359331584ecd4756 server_cookie=741edb18db0adf95 in_seq=0 out_seq=0 2024-09-19T08:57:45.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.691+0000 7f91f57fa640 10 client.18096 ms_handle_connect on v2:172.21.15.170:6836/2350302392 2024-09-19T08:57:45.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.691+0000 7f91f57fa640 1 -- 172.21.15.164:0/3648217012 <== mds.1 v2:172.21.15.170:6836/2350302392 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ae4b04bd70 con 0x55ae4b1647b0 2024-09-19T08:57:45.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.691+0000 7f91f57fa640 10 client.18096 handle_command_reply: tid=0 2024-09-19T08:57:45.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.691+0000 7f921a123740 1 client.18096 shutdown 2024-09-19T08:57:45.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.691+0000 7f921a123740 1 -- 172.21.15.164:0/3648217012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae4b0de5d0 msgr2=0x55ae4b0ed340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:45.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.691+0000 7f921a123740 1 --2- 172.21.15.164:0/3648217012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae4b0de5d0 0x55ae4b0ed340 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f9200010e30 tx=0x7f9200019be0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.691+0000 7f921a123740 1 -- 172.21.15.164:0/3648217012 shutdown_connections 2024-09-19T08:57:45.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.691+0000 7f921a123740 1 --2- 172.21.15.164:0/3648217012 >> [v2:172.21.15.170:6836/2350302392,v1:172.21.15.170:6837/2350302392] conn(0x55ae4b1647b0 0x55ae4b166bd0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.691+0000 7f921a123740 1 --2- 172.21.15.164:0/3648217012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4b0e9370 0x55ae4b0eddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.691+0000 7f921a123740 1 --2- 172.21.15.164:0/3648217012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae4b0dfb40 0x55ae4b0ed880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.691+0000 7f921a123740 1 --2- 172.21.15.164:0/3648217012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae4b0de5d0 0x55ae4b0ed340 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.691+0000 7f91f57fa640 0 client.18096 ms_handle_reset on v2:172.21.15.170:6836/2350302392 2024-09-19T08:57:45.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.691+0000 7f921a123740 1 -- 172.21.15.164:0/3648217012 >> 172.21.15.164:0/3648217012 conn(0x55ae4b0a7310 msgr2=0x55ae4b0d94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:45.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.691+0000 7f921a123740 1 -- 172.21.15.164:0/3648217012 shutdown_connections 2024-09-19T08:57:45.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.691+0000 7f921a123740 1 -- 172.21.15.164:0/3648217012 wait complete. 2024-09-19T08:57:45.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.692+0000 7f921a123740 20 client.18096 trim_cache size 0 max 16384 2024-09-19T08:57:45.716 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:57:45.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.692+0000 7f9217d4a640 1 -- 172.21.15.164:0/1854777148 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f91f80a33b0 msgr2=0x7f91f80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:45.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.692+0000 7f9217d4a640 1 --2- 172.21.15.164:0/1854777148 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f91f80a33b0 0x7f91f80a5870 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9200005c40 tx=0x7f920000a590 comp rx=0 tx=0).stop 2024-09-19T08:57:45.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.692+0000 7f9217d4a640 1 -- 172.21.15.164:0/1854777148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9210103670 msgr2=0x7f92101989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:45.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.692+0000 7f9217d4a640 1 --2- 172.21.15.164:0/1854777148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9210103670 0x7f92101989a0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f9204006010 tx=0x7f92040042f0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.692+0000 7f9217d4a640 1 -- 172.21.15.164:0/1854777148 shutdown_connections 2024-09-19T08:57:45.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.692+0000 7f9217d4a640 1 --2- 172.21.15.164:0/1854777148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f921010c180 0x7f921019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.692+0000 7f9217d4a640 1 --2- 172.21.15.164:0/1854777148 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f91f80a33b0 0x7f91f80a5870 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.692+0000 7f9217d4a640 1 --2- 172.21.15.164:0/1854777148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9210103670 0x7f92101989a0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.692+0000 7f9217d4a640 1 --2- 172.21.15.164:0/1854777148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9210102d50 0x7f9210198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:45.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.692+0000 7f9217d4a640 1 -- 172.21.15.164:0/1854777148 >> 172.21.15.164:0/1854777148 conn(0x7f92100fc7d0 msgr2=0x7f92100fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:45.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.692+0000 7f9217d4a640 1 -- 172.21.15.164:0/1854777148 shutdown_connections 2024-09-19T08:57:45.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:45.692+0000 7f9217d4a640 1 -- 172.21.15.164:0/1854777148 wait complete. 2024-09-19T08:57:46.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:45 smithi164 ceph-mon[30928]: pgmap v373: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-19T08:57:46.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:45 smithi170 ceph-mon[35072]: pgmap v373: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-19T08:57:46.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:45 smithi173 ceph-mon[35306]: pgmap v373: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-19T08:57:46.419 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T08:57:46.669 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:57:47.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.104+0000 7fd1a0878640 1 -- 172.21.15.164:0/3388472283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd19c102d30 msgr2=0x7fd19c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:47.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.104+0000 7fd1a0878640 1 --2- 172.21.15.164:0/3388472283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd19c102d30 0x7fd19c103110 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fd18c009ca0 tx=0x7fd18c02a950 comp rx=0 tx=0).stop 2024-09-19T08:57:47.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.104+0000 7fd1a0878640 1 -- 172.21.15.164:0/3388472283 shutdown_connections 2024-09-19T08:57:47.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.104+0000 7fd1a0878640 1 --2- 172.21.15.164:0/3388472283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd19c10c160 0x7fd19c10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.104+0000 7fd1a0878640 1 --2- 172.21.15.164:0/3388472283 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd19c103650 0x7fd19c10bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.104+0000 7fd1a0878640 1 --2- 172.21.15.164:0/3388472283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd19c102d30 0x7fd19c103110 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.104+0000 7fd1a0878640 1 -- 172.21.15.164:0/3388472283 >> 172.21.15.164:0/3388472283 conn(0x7fd19c0fc7d0 msgr2=0x7fd19c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:47.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.104+0000 7fd1a0878640 1 -- 172.21.15.164:0/3388472283 shutdown_connections 2024-09-19T08:57:47.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.104+0000 7fd1a0878640 1 -- 172.21.15.164:0/3388472283 wait complete. 2024-09-19T08:57:47.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.104+0000 7fd1a0878640 1 Processor -- start 2024-09-19T08:57:47.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.105+0000 7fd1a0878640 1 -- start start 2024-09-19T08:57:47.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.105+0000 7fd1a0878640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd19c102d30 0x7fd19c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:47.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.105+0000 7fd1a0878640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd19c103650 0x7fd19c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:47.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.105+0000 7fd1a0878640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd19c10c160 0x7fd19c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:47.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.105+0000 7fd1a0878640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c111f60 con 0x7fd19c103650 2024-09-19T08:57:47.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.105+0000 7fd1a0878640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c111de0 con 0x7fd19c10c160 2024-09-19T08:57:47.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.105+0000 7fd1a0878640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1120e0 con 0x7fd19c102d30 2024-09-19T08:57:47.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.105+0000 7fd199d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd19c103650 0x7fd19c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:47.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.105+0000 7fd199d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd19c103650 0x7fd19c198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60616/0 (socket says 172.21.15.164:60616) 2024-09-19T08:57:47.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.105+0000 7fd199d9b640 1 -- 172.21.15.164:0/2314753187 learned_addr learned my addr 172.21.15.164:0/2314753187 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:47.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.105+0000 7fd19a59c640 1 --2- 172.21.15.164:0/2314753187 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd19c102d30 0x7fd19c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:47.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.105+0000 7fd19ad9d640 1 --2- 172.21.15.164:0/2314753187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd19c10c160 0x7fd19c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:47.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.105+0000 7fd199d9b640 1 -- 172.21.15.164:0/2314753187 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd19c102d30 msgr2=0x7fd19c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:47.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.105+0000 7fd199d9b640 1 --2- 172.21.15.164:0/2314753187 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd19c102d30 0x7fd19c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.105+0000 7fd199d9b640 1 -- 172.21.15.164:0/2314753187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd19c10c160 msgr2=0x7fd19c19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:47.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.105+0000 7fd199d9b640 1 --2- 172.21.15.164:0/2314753187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd19c10c160 0x7fd19c19cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.105+0000 7fd199d9b640 1 -- 172.21.15.164:0/2314753187 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19c19d380 con 0x7fd19c103650 2024-09-19T08:57:47.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.105+0000 7fd199d9b640 1 --2- 172.21.15.164:0/2314753187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd19c103650 0x7fd19c198910 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fd18400c970 tx=0x7fd18400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:47.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.105+0000 7fd1937fe640 1 -- 172.21.15.164:0/2314753187 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd184007bb0 con 0x7fd19c103650 2024-09-19T08:57:47.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.106+0000 7fd1a0878640 1 -- 172.21.15.164:0/2314753187 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd19c19d670 con 0x7fd19c103650 2024-09-19T08:57:47.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.106+0000 7fd1937fe640 1 -- 172.21.15.164:0/2314753187 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd184007d50 con 0x7fd19c103650 2024-09-19T08:57:47.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.106+0000 7fd1937fe640 1 -- 172.21.15.164:0/2314753187 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd184018820 con 0x7fd19c103650 2024-09-19T08:57:47.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.106+0000 7fd1a0878640 1 -- 172.21.15.164:0/2314753187 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd19c1a4f50 con 0x7fd19c103650 2024-09-19T08:57:47.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.108+0000 7fd1937fe640 1 -- 172.21.15.164:0/2314753187 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fd184018a80 con 0x7fd19c103650 2024-09-19T08:57:47.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.108+0000 7fd1a2c51740 1 Processor -- start 2024-09-19T08:57:47.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.108+0000 7fd1a2c51740 1 -- start start 2024-09-19T08:57:47.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.108+0000 7fd1a2c51740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5580b5939b40 0x5580b5938090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:47.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.108+0000 7fd1a2c51740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5580b59385d0 0x5580b5938a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:47.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.108+0000 7fd1a2c51740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580b5943370 0x5580b5945760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:47.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.108+0000 7fd1a2c51740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5580b589e2f0 con 0x5580b59385d0 2024-09-19T08:57:47.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.108+0000 7fd1a2c51740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5580b5845cf0 con 0x5580b5943370 2024-09-19T08:57:47.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.108+0000 7fd1a2c51740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5580b589a2b0 con 0x5580b5939b40 2024-09-19T08:57:47.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.109+0000 7fd1937fe640 1 --2- 172.21.15.164:0/2314753187 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd1880a33b0 0x7fd1880a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:47.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.109+0000 7fd19a59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5580b59385d0 0x5580b5938a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:47.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.109+0000 7fd19a59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5580b59385d0 0x5580b5938a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60620/0 (socket says 172.21.15.164:60620) 2024-09-19T08:57:47.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.109+0000 7fd19a59c640 1 -- 172.21.15.164:0/4017326250 learned_addr learned my addr 172.21.15.164:0/4017326250 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:47.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.109+0000 7fd199d9b640 1 --2- 172.21.15.164:0/4017326250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580b5943370 0x5580b5945760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:47.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.109+0000 7fd19ad9d640 1 --2- 172.21.15.164:0/2314753187 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd1880a33b0 0x7fd1880a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:47.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.109+0000 7fd19ad9d640 1 --2- 172.21.15.164:0/4017326250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5580b5939b40 0x5580b5938090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:47.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.109+0000 7fd1937fe640 1 -- 172.21.15.164:0/2314753187 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7fd1840cfd90 con 0x7fd19c103650 2024-09-19T08:57:47.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.109+0000 7fd19a59c640 1 -- 172.21.15.164:0/4017326250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5580b5939b40 msgr2=0x5580b5938090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:47.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.109+0000 7fd19a59c640 1 --2- 172.21.15.164:0/4017326250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5580b5939b40 0x5580b5938090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.109+0000 7fd19a59c640 1 -- 172.21.15.164:0/4017326250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580b5943370 msgr2=0x5580b5945760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:47.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.109+0000 7fd19a59c640 1 --2- 172.21.15.164:0/4017326250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580b5943370 0x5580b5945760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.109+0000 7fd19a59c640 1 -- 172.21.15.164:0/4017326250 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580b582a3a0 con 0x5580b59385d0 2024-09-19T08:57:47.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.109+0000 7fd199d9b640 1 --2- 172.21.15.164:0/4017326250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580b5943370 0x5580b5945760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:57:47.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.109+0000 7fd19ad9d640 1 --2- 172.21.15.164:0/2314753187 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd1880a33b0 0x7fd1880a5870 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd194013720 tx=0x7fd19400e930 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:47.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.109+0000 7fd19a59c640 1 --2- 172.21.15.164:0/4017326250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5580b59385d0 0x5580b5938a10 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fd18c035000 tx=0x7fd18c009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83f800420dfc6ed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:47.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.109+0000 7fd190ff9640 1 -- 172.21.15.164:0/4017326250 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c048070 con 0x5580b59385d0 2024-09-19T08:57:47.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.110+0000 7fd190ff9640 1 -- 172.21.15.164:0/4017326250 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd18c02fa00 con 0x5580b59385d0 2024-09-19T08:57:47.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.110+0000 7fd190ff9640 1 -- 172.21.15.164:0/4017326250 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c034040 con 0x5580b59385d0 2024-09-19T08:57:47.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.110+0000 7fd1a2c51740 1 -- 172.21.15.164:0/4017326250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5580b59385d0 msgr2=0x5580b5938a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:47.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.110+0000 7fd1a2c51740 1 --2- 172.21.15.164:0/4017326250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5580b59385d0 0x5580b5938a10 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fd18c035000 tx=0x7fd18c009960 comp rx=0 tx=0).stop 2024-09-19T08:57:47.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.110+0000 7fd1a2c51740 1 -- 172.21.15.164:0/4017326250 shutdown_connections 2024-09-19T08:57:47.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.110+0000 7fd1a2c51740 1 --2- 172.21.15.164:0/4017326250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580b5943370 0x5580b5945760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.110+0000 7fd1a2c51740 1 --2- 172.21.15.164:0/4017326250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5580b59385d0 0x5580b5938a10 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.110+0000 7fd1a2c51740 1 --2- 172.21.15.164:0/4017326250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5580b5939b40 0x5580b5938090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.110+0000 7fd1a2c51740 1 -- 172.21.15.164:0/4017326250 >> 172.21.15.164:0/4017326250 conn(0x5580b5901310 msgr2=0x5580b5934c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:47.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.110+0000 7fd1a2c51740 1 -- 172.21.15.164:0/4017326250 shutdown_connections 2024-09-19T08:57:47.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.110+0000 7fd1a2c51740 1 -- 172.21.15.164:0/4017326250 wait complete. 2024-09-19T08:57:47.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.111+0000 7fd1a2c51740 1 -- start start 2024-09-19T08:57:47.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.111+0000 7fd1a2c51740 1 Processor -- start 2024-09-19T08:57:47.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.111+0000 7fd1a2c51740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580b59385d0 0x5580b5942d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:47.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.111+0000 7fd1a2c51740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5580b5939b40 0x5580b593ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:47.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.111+0000 7fd1a2c51740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5580b5943370 0x5580b59405d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:47.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.111+0000 7fd1a2c51740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5580b5946010 con 0x5580b5943370 2024-09-19T08:57:47.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.111+0000 7fd1a2c51740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5580b5859060 con 0x5580b59385d0 2024-09-19T08:57:47.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.111+0000 7fd1a2c51740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5580b5946190 con 0x5580b5939b40 2024-09-19T08:57:47.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.111+0000 7fd1a2c51740 10 client.0 _ll_register_callbacks cb 0x5580b55ac8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:57:47.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.111+0000 7fd199d9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5580b5939b40 0x5580b593ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:47.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.111+0000 7fd199d9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5580b5939b40 0x5580b593ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:34182/0 (socket says 172.21.15.164:34182) 2024-09-19T08:57:47.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.111+0000 7fd199d9b640 1 -- 172.21.15.164:0/916010745 learned_addr learned my addr 172.21.15.164:0/916010745 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:47.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.111+0000 7fd19a59c640 1 --2- 172.21.15.164:0/916010745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580b59385d0 0x5580b5942d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:47.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.111+0000 7fd19ad9d640 1 --2- 172.21.15.164:0/916010745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5580b5943370 0x5580b59405d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:47.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.112+0000 7fd199d9b640 1 -- 172.21.15.164:0/916010745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580b59385d0 msgr2=0x5580b5942d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:47.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.112+0000 7fd199d9b640 1 --2- 172.21.15.164:0/916010745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580b59385d0 0x5580b5942d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.112+0000 7fd199d9b640 1 -- 172.21.15.164:0/916010745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5580b5943370 msgr2=0x5580b59405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:47.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.112+0000 7fd199d9b640 1 --2- 172.21.15.164:0/916010745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5580b5943370 0x5580b59405d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.112+0000 7fd199d9b640 1 -- 172.21.15.164:0/916010745 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580b560f3c0 con 0x5580b5939b40 2024-09-19T08:57:47.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.112+0000 7fd199d9b640 1 --2- 172.21.15.164:0/916010745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5580b5939b40 0x5580b593ffe0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd1840cd580 tx=0x7fd18406bcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58ad5034d225afe1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:47.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.112+0000 7fd181ffb640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-09-19T08:57:47.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.112+0000 7fd181ffb640 1 -- 172.21.15.164:0/916010745 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18409c0b0 con 0x5580b5939b40 2024-09-19T08:57:47.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.112+0000 7fd181ffb640 1 -- 172.21.15.164:0/916010745 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7fd18400fac0 con 0x5580b5939b40 2024-09-19T08:57:47.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.112+0000 7fd1a2c51740 1 -- 172.21.15.164:0/916010745 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd18c02fa00 con 0x5580b5939b40 2024-09-19T08:57:47.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.112+0000 7fd181ffb640 1 -- 172.21.15.164:0/916010745 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd1840a0460 con 0x5580b5939b40 2024-09-19T08:57:47.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.112+0000 7fd1a2c51740 10 client.26923 fetch_fsmap learned FSMap version 33 2024-09-19T08:57:47.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.112+0000 7fd1a2c51740 1 -- 172.21.15.164:0/916010745 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5580b5951ad0 con 0x5580b5939b40 2024-09-19T08:57:47.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.112+0000 7fd181ffb640 1 -- 172.21.15.164:0/916010745 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd18c02fa00 con 0x5580b5939b40 2024-09-19T08:57:47.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.113+0000 7fd181ffb640 1 -- 172.21.15.164:0/916010745 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 33) ==== 8366+0+0 (secure 0 0 0) 0x7fd1840a00b0 con 0x5580b5939b40 2024-09-19T08:57:47.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.113+0000 7fd1a2c51740 10 client.26923 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T08:57:47.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.113+0000 7fd1a2c51740 10 client.26923 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:57:47.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.113+0000 7fd1a2c51740 20 client.26923 populate_metadata read hostname 'smithi164' 2024-09-19T08:57:47.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.113+0000 7fd1a2c51740 1 --2- 172.21.15.164:0/916010745 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5580b5952f90 0x5580b59553d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:47.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.113+0000 7fd1a2c51740 4 client.26923 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:57:47.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.113+0000 7fd1a2c51740 1 -- 172.21.15.164:0/916010745 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580b58ab860 con 0x5580b5952f90 2024-09-19T08:57:47.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.113+0000 7fd19a59c640 1 --2- 172.21.15.164:0/916010745 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5580b5952f90 0x5580b59553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:47.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.113+0000 7fd181ffb640 10 client.26923 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:57:47.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.113+0000 7fd19a59c640 1 --2- 172.21.15.164:0/916010745 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5580b5952f90 0x5580b59553d0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49a9bccca6ae8ac5 server_cookie=689b045a900b4534 in_seq=0 out_seq=0 2024-09-19T08:57:47.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.114+0000 7fd181ffb640 1 -- 172.21.15.164:0/916010745 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5580b58ab860 con 0x5580b5952f90 2024-09-19T08:57:47.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.114+0000 7fd181ffb640 10 client.26923 handle_command_reply: tid=0 2024-09-19T08:57:47.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.114+0000 7fd1a2c51740 1 client.26923 shutdown 2024-09-19T08:57:47.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.114+0000 7fd1a2c51740 1 -- 172.21.15.164:0/916010745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5580b5939b40 msgr2=0x5580b593ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:47.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.114+0000 7fd1a2c51740 1 --2- 172.21.15.164:0/916010745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5580b5939b40 0x5580b593ffe0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd1840cd580 tx=0x7fd18406bcd0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.114+0000 7fd1a2c51740 1 -- 172.21.15.164:0/916010745 shutdown_connections 2024-09-19T08:57:47.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.114+0000 7fd1a2c51740 1 --2- 172.21.15.164:0/916010745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5580b5943370 0x5580b59405d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.114+0000 7fd1a2c51740 1 --2- 172.21.15.164:0/916010745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5580b5939b40 0x5580b593ffe0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.114+0000 7fd1a2c51740 1 --2- 172.21.15.164:0/916010745 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5580b5952f90 0x5580b59553d0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.114+0000 7fd1a2c51740 1 --2- 172.21.15.164:0/916010745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580b59385d0 0x5580b5942d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.114+0000 7fd181ffb640 0 client.26923 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:57:47.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.114+0000 7fd1a2c51740 1 -- 172.21.15.164:0/916010745 >> 172.21.15.164:0/916010745 conn(0x5580b5901310 msgr2=0x5580b593a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:47.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.114+0000 7fd1a2c51740 1 -- 172.21.15.164:0/916010745 shutdown_connections 2024-09-19T08:57:47.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.114+0000 7fd1a2c51740 1 -- 172.21.15.164:0/916010745 wait complete. 2024-09-19T08:57:47.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.114+0000 7fd1a2c51740 20 client.26923 trim_cache size 0 max 16384 2024-09-19T08:57:47.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.127+0000 7fd1a2c51740 1 Processor -- start 2024-09-19T08:57:47.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.127+0000 7fd1a2c51740 1 -- start start 2024-09-19T08:57:47.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.128+0000 7fd1a2c51740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5580b59385d0 0x5580b5934e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:47.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.128+0000 7fd1a2c51740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580b5939b40 0x5580b594f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:47.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.128+0000 7fd1a2c51740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5580b5943370 0x5580b593f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:47.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.128+0000 7fd1a2c51740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1ad620 con 0x5580b5943370 2024-09-19T08:57:47.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.128+0000 7fd1a2c51740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1af190 con 0x5580b5939b40 2024-09-19T08:57:47.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.128+0000 7fd1a2c51740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1adcf0 con 0x5580b59385d0 2024-09-19T08:57:47.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.128+0000 7fd19ad9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5580b5943370 0x5580b593f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:47.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.128+0000 7fd19ad9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5580b5943370 0x5580b593f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60640/0 (socket says 172.21.15.164:60640) 2024-09-19T08:57:47.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.128+0000 7fd19ad9d640 1 -- 172.21.15.164:0/616259598 learned_addr learned my addr 172.21.15.164:0/616259598 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:47.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.128+0000 7fd19a59c640 1 --2- 172.21.15.164:0/616259598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5580b59385d0 0x5580b5934e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:47.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.128+0000 7fd19ad9d640 1 -- 172.21.15.164:0/616259598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5580b59385d0 msgr2=0x5580b5934e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:47.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.128+0000 7fd19ad9d640 1 --2- 172.21.15.164:0/616259598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5580b59385d0 0x5580b5934e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.128+0000 7fd19ad9d640 1 -- 172.21.15.164:0/616259598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580b5939b40 msgr2=0x5580b594f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:47.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.128+0000 7fd19ad9d640 1 --2- 172.21.15.164:0/616259598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580b5939b40 0x5580b594f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.128+0000 7fd19ad9d640 1 -- 172.21.15.164:0/616259598 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580b593ef90 con 0x5580b5943370 2024-09-19T08:57:47.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.128+0000 7fd19ad9d640 1 --2- 172.21.15.164:0/616259598 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5580b5943370 0x5580b593f8a0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fd194010900 tx=0x7fd1940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c7bcab1b4821cbc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:47.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.128+0000 7fd183fff640 1 -- 172.21.15.164:0/616259598 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd194021b40 con 0x5580b5943370 2024-09-19T08:57:47.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.128+0000 7fd183fff640 1 -- 172.21.15.164:0/616259598 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd194010e80 con 0x5580b5943370 2024-09-19T08:57:47.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.129+0000 7fd183fff640 1 -- 172.21.15.164:0/616259598 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd194004c30 con 0x5580b5943370 2024-09-19T08:57:47.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.129+0000 7fd1a2c51740 1 -- 172.21.15.164:0/616259598 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5580b5943370 msgr2=0x5580b593f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:47.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.129+0000 7fd1a2c51740 1 --2- 172.21.15.164:0/616259598 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5580b5943370 0x5580b593f8a0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fd194010900 tx=0x7fd1940046c0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.129+0000 7fd1a2c51740 1 -- 172.21.15.164:0/616259598 shutdown_connections 2024-09-19T08:57:47.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.129+0000 7fd1a2c51740 1 --2- 172.21.15.164:0/616259598 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5580b5943370 0x5580b593f8a0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.129+0000 7fd1a2c51740 1 --2- 172.21.15.164:0/616259598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580b5939b40 0x5580b594f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.129+0000 7fd1a2c51740 1 --2- 172.21.15.164:0/616259598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5580b59385d0 0x5580b5934e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.129+0000 7fd1a2c51740 1 -- 172.21.15.164:0/616259598 >> 172.21.15.164:0/616259598 conn(0x5580b5901310 msgr2=0x5580b5934700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:47.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.129+0000 7fd1a2c51740 1 -- 172.21.15.164:0/616259598 shutdown_connections 2024-09-19T08:57:47.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.129+0000 7fd1a2c51740 1 -- 172.21.15.164:0/616259598 wait complete. 2024-09-19T08:57:47.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.129+0000 7fd1a2c51740 1 -- start start 2024-09-19T08:57:47.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.130+0000 7fd1a2c51740 1 Processor -- start 2024-09-19T08:57:47.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.130+0000 7fd1a2c51740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5580b59385d0 0x5580b59471a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:47.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.130+0000 7fd1a2c51740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5580b5939b40 0x5580b59476e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:47.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.130+0000 7fd1a2c51740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580b5943370 0x5580b5947c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:47.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.130+0000 7fd1a2c51740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1aeac0 con 0x5580b59385d0 2024-09-19T08:57:47.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.130+0000 7fd1a2c51740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1ae3c0 con 0x5580b5943370 2024-09-19T08:57:47.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.130+0000 7fd1a2c51740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1ab620 con 0x5580b5939b40 2024-09-19T08:57:47.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.130+0000 7fd1a2c51740 10 client.0 _ll_register_callbacks cb 0x7fd18409bcc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:57:47.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.130+0000 7fd19a59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5580b59385d0 0x5580b59471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:47.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.130+0000 7fd19a59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5580b59385d0 0x5580b59471a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60644/0 (socket says 172.21.15.164:60644) 2024-09-19T08:57:47.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.130+0000 7fd19a59c640 1 -- 172.21.15.164:0/2636082544 learned_addr learned my addr 172.21.15.164:0/2636082544 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:47.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.130+0000 7fd199d9b640 1 --2- 172.21.15.164:0/2636082544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5580b5939b40 0x5580b59476e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:47.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.130+0000 7fd19ad9d640 1 --2- 172.21.15.164:0/2636082544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580b5943370 0x5580b5947c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:47.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.130+0000 7fd19a59c640 1 -- 172.21.15.164:0/2636082544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5580b5939b40 msgr2=0x5580b59476e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:47.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.130+0000 7fd19a59c640 1 --2- 172.21.15.164:0/2636082544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5580b5939b40 0x5580b59476e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.130+0000 7fd19a59c640 1 -- 172.21.15.164:0/2636082544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580b5943370 msgr2=0x5580b5947c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:57:47.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.130+0000 7fd19a59c640 1 --2- 172.21.15.164:0/2636082544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580b5943370 0x5580b5947c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.130+0000 7fd19a59c640 1 -- 172.21.15.164:0/2636082544 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580b59bcca0 con 0x5580b59385d0 2024-09-19T08:57:47.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.130+0000 7fd19a59c640 1 --2- 172.21.15.164:0/2636082544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5580b59385d0 0x5580b59471a0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fd18c035000 tx=0x7fd18c02b0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5b6d7907754be9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:47.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.130+0000 7fd181ffb640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-09-19T08:57:47.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.131+0000 7fd181ffb640 1 -- 172.21.15.164:0/2636082544 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c048040 con 0x5580b59385d0 2024-09-19T08:57:47.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.131+0000 7fd181ffb640 1 -- 172.21.15.164:0/2636082544 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7fd18c02fe10 con 0x5580b59385d0 2024-09-19T08:57:47.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.131+0000 7fd1a2c51740 1 -- 172.21.15.164:0/2636082544 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5580b58a5d70 con 0x5580b59385d0 2024-09-19T08:57:47.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.131+0000 7fd181ffb640 1 -- 172.21.15.164:0/2636082544 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd18c034040 con 0x5580b59385d0 2024-09-19T08:57:47.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.131+0000 7fd1a2c51740 10 client.18114 fetch_fsmap learned FSMap version 33 2024-09-19T08:57:47.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.131+0000 7fd1a2c51740 1 -- 172.21.15.164:0/2636082544 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5580b59bdf60 con 0x5580b59385d0 2024-09-19T08:57:47.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.131+0000 7fd181ffb640 1 -- 172.21.15.164:0/2636082544 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5580b58a5d70 con 0x5580b59385d0 2024-09-19T08:57:47.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.132+0000 7fd181ffb640 1 -- 172.21.15.164:0/2636082544 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 33) ==== 8366+0+0 (secure 0 0 0) 0x7fd18c02f3a0 con 0x5580b59385d0 2024-09-19T08:57:47.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.132+0000 7fd1a2c51740 10 client.18114 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T08:57:47.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.132+0000 7fd1a2c51740 10 client.18114 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T08:57:47.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.132+0000 7fd1a2c51740 20 client.18114 populate_metadata read hostname 'smithi164' 2024-09-19T08:57:47.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.132+0000 7fd1a2c51740 1 --2- 172.21.15.164:0/2636082544 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5580b59be1f0 0x5580b59c05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:47.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.132+0000 7fd1a2c51740 4 client.18114 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T08:57:47.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.132+0000 7fd1a2c51740 1 -- 172.21.15.164:0/2636082544 --> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5580b5822170 con 0x5580b59be1f0 2024-09-19T08:57:47.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.132+0000 7fd199d9b640 1 --2- 172.21.15.164:0/2636082544 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5580b59be1f0 0x5580b59c05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:47.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.133+0000 7fd199d9b640 1 --2- 172.21.15.164:0/2636082544 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5580b59be1f0 0x5580b59c05e0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=566a2a32ce49aafb server_cookie=9fda8cf80b5258d9 in_seq=0 out_seq=0 2024-09-19T08:57:47.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.133+0000 7fd181ffb640 10 client.18114 ms_handle_connect on v2:172.21.15.170:6838/1854887138 2024-09-19T08:57:47.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.133+0000 7fd181ffb640 1 -- 172.21.15.164:0/2636082544 <== mds.0 v2:172.21.15.170:6838/1854887138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5580b5822170 con 0x5580b59be1f0 2024-09-19T08:57:47.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.133+0000 7fd181ffb640 10 client.18114 handle_command_reply: tid=0 2024-09-19T08:57:47.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.133+0000 7fd1a2c51740 1 client.18114 shutdown 2024-09-19T08:57:47.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.133+0000 7fd1a2c51740 1 -- 172.21.15.164:0/2636082544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5580b59385d0 msgr2=0x5580b59471a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:47.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.133+0000 7fd1a2c51740 1 --2- 172.21.15.164:0/2636082544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5580b59385d0 0x5580b59471a0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fd18c035000 tx=0x7fd18c02b0e0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.133+0000 7fd1a2c51740 1 -- 172.21.15.164:0/2636082544 shutdown_connections 2024-09-19T08:57:47.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.133+0000 7fd1a2c51740 1 --2- 172.21.15.164:0/2636082544 >> [v2:172.21.15.170:6838/1854887138,v1:172.21.15.170:6839/1854887138] conn(0x5580b59be1f0 0x5580b59c05e0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.133+0000 7fd1a2c51740 1 --2- 172.21.15.164:0/2636082544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580b5943370 0x5580b5947c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.133+0000 7fd1a2c51740 1 --2- 172.21.15.164:0/2636082544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5580b5939b40 0x5580b59476e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.133+0000 7fd1a2c51740 1 --2- 172.21.15.164:0/2636082544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5580b59385d0 0x5580b59471a0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.133+0000 7fd1a2c51740 1 -- 172.21.15.164:0/2636082544 >> 172.21.15.164:0/2636082544 conn(0x5580b5901310 msgr2=0x5580b59334e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:47.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.133+0000 7fd181ffb640 0 client.18114 ms_handle_reset on v2:172.21.15.170:6838/1854887138 2024-09-19T08:57:47.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.133+0000 7fd1a2c51740 1 -- 172.21.15.164:0/2636082544 shutdown_connections 2024-09-19T08:57:47.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.133+0000 7fd1a2c51740 1 -- 172.21.15.164:0/2636082544 wait complete. 2024-09-19T08:57:47.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.133+0000 7fd1a2c51740 20 client.18114 trim_cache size 0 max 16384 2024-09-19T08:57:47.145 INFO:teuthology.orchestra.run.smithi164.stdout:[] 2024-09-19T08:57:47.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.134+0000 7fd1a0878640 1 -- 172.21.15.164:0/2314753187 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd1880a33b0 msgr2=0x7fd1880a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:47.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.134+0000 7fd1a0878640 1 --2- 172.21.15.164:0/2314753187 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd1880a33b0 0x7fd1880a5870 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd194013720 tx=0x7fd19400e930 comp rx=0 tx=0).stop 2024-09-19T08:57:47.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.134+0000 7fd1a0878640 1 -- 172.21.15.164:0/2314753187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd19c103650 msgr2=0x7fd19c198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:47.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.134+0000 7fd1a0878640 1 --2- 172.21.15.164:0/2314753187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd19c103650 0x7fd19c198910 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fd18400c970 tx=0x7fd18400ce30 comp rx=0 tx=0).stop 2024-09-19T08:57:47.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.134+0000 7fd1a0878640 1 -- 172.21.15.164:0/2314753187 shutdown_connections 2024-09-19T08:57:47.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.134+0000 7fd1a0878640 1 --2- 172.21.15.164:0/2314753187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd19c10c160 0x7fd19c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.134+0000 7fd1a0878640 1 --2- 172.21.15.164:0/2314753187 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd1880a33b0 0x7fd1880a5870 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.134+0000 7fd1a0878640 1 --2- 172.21.15.164:0/2314753187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd19c103650 0x7fd19c198910 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.134+0000 7fd1a0878640 1 --2- 172.21.15.164:0/2314753187 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd19c102d30 0x7fd19c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.134+0000 7fd1a0878640 1 -- 172.21.15.164:0/2314753187 >> 172.21.15.164:0/2314753187 conn(0x7fd19c0fc7d0 msgr2=0x7fd19c0fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:47.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.134+0000 7fd1a0878640 1 -- 172.21.15.164:0/2314753187 shutdown_connections 2024-09-19T08:57:47.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.134+0000 7fd1a0878640 1 -- 172.21.15.164:0/2314753187 wait complete. 2024-09-19T08:57:47.875 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-19T08:57:47.876 INFO:tasks.fwd_scrub:done joining 2024-09-19T08:57:47.876 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-19T08:57:47.887 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:57:47.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.990+0000 7f7b5e02c640 1 -- 172.21.15.164:0/1723030241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b58106990 msgr2=0x7f7b58106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:47.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.990+0000 7f7b5e02c640 1 --2- 172.21.15.164:0/1723030241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b58106990 0x7f7b58106d70 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f7b48009930 tx=0x7f7b4802a420 comp rx=0 tx=0).stop 2024-09-19T08:57:47.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.990+0000 7f7b5e02c640 1 -- 172.21.15.164:0/1723030241 shutdown_connections 2024-09-19T08:57:47.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.990+0000 7f7b5e02c640 1 --2- 172.21.15.164:0/1723030241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b58101320 0x7f7b5810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.990+0000 7f7b5e02c640 1 --2- 172.21.15.164:0/1723030241 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b58100980 0x7f7b58100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.990+0000 7f7b5e02c640 1 --2- 172.21.15.164:0/1723030241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b58106990 0x7f7b58106d70 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.990+0000 7f7b5e02c640 1 -- 172.21.15.164:0/1723030241 >> 172.21.15.164:0/1723030241 conn(0x7f7b58077a10 msgr2=0x7f7b580fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:47.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.990+0000 7f7b5e02c640 1 -- 172.21.15.164:0/1723030241 shutdown_connections 2024-09-19T08:57:47.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.990+0000 7f7b5e02c640 1 -- 172.21.15.164:0/1723030241 wait complete. 2024-09-19T08:57:47.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.990+0000 7f7b5e02c640 1 Processor -- start 2024-09-19T08:57:47.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.990+0000 7f7b5e02c640 1 -- start start 2024-09-19T08:57:47.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.991+0000 7f7b5e02c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b58100980 0x7f7b58197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:47.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.991+0000 7f7b5e02c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b58101320 0x7f7b581978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:47.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.991+0000 7f7b5e02c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b58106990 0x7f7b5819bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:47.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.991+0000 7f7b5e02c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7b58110dd0 con 0x7f7b58106990 2024-09-19T08:57:47.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.991+0000 7f7b5e02c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7b58110c50 con 0x7f7b58100980 2024-09-19T08:57:47.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.991+0000 7f7b5e02c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7b58110f50 con 0x7f7b58101320 2024-09-19T08:57:47.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.991+0000 7f7b57fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b58106990 0x7f7b5819bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:47.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.991+0000 7f7b57fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b58106990 0x7f7b5819bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60664/0 (socket says 172.21.15.164:60664) 2024-09-19T08:57:47.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.991+0000 7f7b57fff640 1 -- 172.21.15.164:0/147126080 learned_addr learned my addr 172.21.15.164:0/147126080 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:47.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.991+0000 7f7b577fe640 1 --2- 172.21.15.164:0/147126080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b58100980 0x7f7b58197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:47.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.991+0000 7f7b4effd640 1 --2- 172.21.15.164:0/147126080 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b58101320 0x7f7b581978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:47.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.991+0000 7f7b57fff640 1 -- 172.21.15.164:0/147126080 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b58101320 msgr2=0x7f7b581978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:47.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.991+0000 7f7b57fff640 1 --2- 172.21.15.164:0/147126080 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b58101320 0x7f7b581978b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.991+0000 7f7b57fff640 1 -- 172.21.15.164:0/147126080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b58100980 msgr2=0x7f7b58197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:47.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.991+0000 7f7b57fff640 1 --2- 172.21.15.164:0/147126080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b58100980 0x7f7b58197370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:47.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.991+0000 7f7b57fff640 1 -- 172.21.15.164:0/147126080 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b5819c360 con 0x7f7b58106990 2024-09-19T08:57:47.995 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.991+0000 7f7b57fff640 1 --2- 172.21.15.164:0/147126080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b58106990 0x7f7b5819bc30 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000eaa0 tx=0x7f7b5000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:47.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.991+0000 7f7b557fa640 1 -- 172.21.15.164:0/147126080 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5000cdb0 con 0x7f7b58106990 2024-09-19T08:57:47.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.992+0000 7f7b5e02c640 1 -- 172.21.15.164:0/147126080 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b5819c650 con 0x7f7b58106990 2024-09-19T08:57:47.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.992+0000 7f7b557fa640 1 -- 172.21.15.164:0/147126080 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b50005020 con 0x7f7b58106990 2024-09-19T08:57:47.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.992+0000 7f7b557fa640 1 -- 172.21.15.164:0/147126080 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b500116b0 con 0x7f7b58106990 2024-09-19T08:57:47.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.992+0000 7f7b5e02c640 1 -- 172.21.15.164:0/147126080 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b581a3ee0 con 0x7f7b58106990 2024-09-19T08:57:47.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.993+0000 7f7b5e02c640 1 -- 172.21.15.164:0/147126080 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b58100de0 con 0x7f7b58106990 2024-09-19T08:57:47.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.994+0000 7f7b557fa640 1 -- 172.21.15.164:0/147126080 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f7b50005560 con 0x7f7b58106990 2024-09-19T08:57:47.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.995+0000 7f7b557fa640 1 --2- 172.21.15.164:0/147126080 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f7b440a33c0 0x7f7b440a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:47.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.995+0000 7f7b557fa640 1 -- 172.21.15.164:0/147126080 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7f7b500cff90 con 0x7f7b58106990 2024-09-19T08:57:47.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.995+0000 7f7b577fe640 1 --2- 172.21.15.164:0/147126080 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f7b440a33c0 0x7f7b440a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:47.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.995+0000 7f7b577fe640 1 --2- 172.21.15.164:0/147126080 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f7b440a33c0 0x7f7b440a5880 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7b48009900 tx=0x7f7b480058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:47.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:47.999+0000 7f7b557fa640 1 -- 172.21.15.164:0/147126080 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7b50016030 con 0x7f7b58106990 2024-09-19T08:57:48.168 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.168+0000 7f7b5e02c640 1 -- 172.21.15.164:0/147126080 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7b58108960 con 0x7f7b58106990 2024-09-19T08:57:48.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.169+0000 7f7b557fa640 1 -- 172.21.15.164:0/147126080 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+23176 (secure 0 0 0) 0x7f7b5009c950 con 0x7f7b58106990 2024-09-19T08:57:48.170 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:57:48.170 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":95,"fsid":"1e4b0d74-7662-11ef-bceb-c7b262605968","created":"2024-09-19T08:36:09.564509+0000","modified":"2024-09-19T08:57:14.910986+0000","last_up_change":"2024-09-19T08:43:01.260852+0000","last_in_change":"2024-09-19T08:42:37.492464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:39:13.961749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:43:49.613471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T08:43:51.570675+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":94,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33b11f37-6ac5-41fc-8cfd-1b08c7eef688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6803","nonce":663038243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6805","nonce":663038243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6809","nonce":663038243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6807","nonce":663038243}]},"public_addr":"172.21.15.164:6803/663038243","cluster_addr":"172.21.15.164:6805/663038243","heartbeat_back_addr":"172.21.15.164:6809/663038243","heartbeat_front_addr":"172.21.15.164:6807/663038243","state":["exists","up"]},{"osd":1,"uuid":"6de64b0a-46e2-4a98-8005-a326a64bc87d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6801","nonce":2167066371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6803","nonce":2167066371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6807","nonce":2167066371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6805","nonce":2167066371}]},"public_addr":"172.21.15.170:6801/2167066371","cluster_addr":"172.21.15.170:6803/2167066371","heartbeat_back_addr":"172.21.15.170:6807/2167066371","heartbeat_front_addr":"172.21.15.170:6805/2167066371","state":["exists","up"]},{"osd":2,"uuid":"d4158cc7-9822-42a7-9601-d65089907584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6801","nonce":1646561048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6803","nonce":1646561048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6807","nonce":1646561048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6805","nonce":1646561048}]},"public_addr":"172.21.15.173:6801/1646561048","cluster_addr":"172.21.15.173:6803/1646561048","heartbeat_back_addr":"172.21.15.173:6807/1646561048","heartbeat_front_addr":"172.21.15.173:6805/1646561048","state":["exists","up"]},{"osd":3,"uuid":"0cf3f7f1-3a8c-42ce-88a9-9bea52e02221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6811","nonce":406795560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6813","nonce":406795560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6817","nonce":406795560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6815","nonce":406795560}]},"public_addr":"172.21.15.164:6811/406795560","cluster_addr":"172.21.15.164:6813/406795560","heartbeat_back_addr":"172.21.15.164:6817/406795560","heartbeat_front_addr":"172.21.15.164:6815/406795560","state":["exists","up"]},{"osd":4,"uuid":"6dca048a-2860-4a02-8d8e-450b30bc1a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6809","nonce":4120832165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6811","nonce":4120832165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6815","nonce":4120832165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6813","nonce":4120832165}]},"public_addr":"172.21.15.170:6809/4120832165","cluster_addr":"172.21.15.170:6811/4120832165","heartbeat_back_addr":"172.21.15.170:6815/4120832165","heartbeat_front_addr":"172.21.15.170:6813/4120832165","state":["exists","up"]},{"osd":5,"uuid":"3822521f-69d6-4141-a83f-5fc9cecc7b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6809","nonce":2914653552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6811","nonce":2914653552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6815","nonce":2914653552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6813","nonce":2914653552}]},"public_addr":"172.21.15.173:6809/2914653552","cluster_addr":"172.21.15.173:6811/2914653552","heartbeat_back_addr":"172.21.15.173:6815/2914653552","heartbeat_front_addr":"172.21.15.173:6813/2914653552","state":["exists","up"]},{"osd":6,"uuid":"0f5a965e-2eb2-4c24-b089-577e0639c160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6819","nonce":1138341393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6821","nonce":1138341393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6825","nonce":1138341393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6823","nonce":1138341393}]},"public_addr":"172.21.15.164:6819/1138341393","cluster_addr":"172.21.15.164:6821/1138341393","heartbeat_back_addr":"172.21.15.164:6825/1138341393","heartbeat_front_addr":"172.21.15.164:6823/1138341393","state":["exists","up"]},{"osd":7,"uuid":"f16cb2c3-639b-46ae-a740-1bd28ca1863c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6817","nonce":2124160360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6819","nonce":2124160360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6823","nonce":2124160360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6821","nonce":2124160360}]},"public_addr":"172.21.15.170:6817/2124160360","cluster_addr":"172.21.15.170:6819/2124160360","heartbeat_back_addr":"172.21.15.170:6823/2124160360","heartbeat_front_addr":"172.21.15.170:6821/2124160360","state":["exists","up"]},{"osd":8,"uuid":"161cff01-46ad-48e3-b999-a5473a2ad618","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6817","nonce":2285175061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6819","nonce":2285175061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6823","nonce":2285175061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6821","nonce":2285175061}]},"public_addr":"172.21.15.173:6817/2285175061","cluster_addr":"172.21.15.173:6819/2285175061","heartbeat_back_addr":"172.21.15.173:6823/2285175061","heartbeat_front_addr":"172.21.15.173:6821/2285175061","state":["exists","up"]},{"osd":9,"uuid":"994666e8-8fbd-4231-ac67-81a79aaf7c2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6827","nonce":3918847952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6829","nonce":3918847952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6833","nonce":3918847952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6831","nonce":3918847952}]},"public_addr":"172.21.15.164:6827/3918847952","cluster_addr":"172.21.15.164:6829/3918847952","heartbeat_back_addr":"172.21.15.164:6833/3918847952","heartbeat_front_addr":"172.21.15.164:6831/3918847952","state":["exists","up"]},{"osd":10,"uuid":"ff5e9099-cdb4-4905-b888-5b67320ef76b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6825","nonce":2707848227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6827","nonce":2707848227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6831","nonce":2707848227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6829","nonce":2707848227}]},"public_addr":"172.21.15.170:6825/2707848227","cluster_addr":"172.21.15.170:6827/2707848227","heartbeat_back_addr":"172.21.15.170:6831/2707848227","heartbeat_front_addr":"172.21.15.170:6829/2707848227","state":["exists","up"]},{"osd":11,"uuid":"4d93088f-e791-4938-a207-12f15b56abd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6825","nonce":2831924738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6827","nonce":2831924738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6831","nonce":2831924738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6829","nonce":2831924738}]},"public_addr":"172.21.15.173:6825/2831924738","cluster_addr":"172.21.15.173:6827/2831924738","heartbeat_back_addr":"172.21.15.173:6831/2831924738","heartbeat_front_addr":"172.21.15.173:6829/2831924738","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:21.787352+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:46.012323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:39:11.353040+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:39:37.015124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:02.063379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:27.892865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:53.372041+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:41:17.855284+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:41:43.389882+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:08.956895+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:34.069779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:59.303897+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.164:0/3424843982":"2024-09-20T08:45:47.960147+0000","172.21.15.164:0/1369265436":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1807120560":"2024-09-20T08:45:47.960147+0000","172.21.15.164:6801/1094864383":"2024-09-20T08:45:47.960147+0000","172.21.15.164:0/1905394497":"2024-09-20T08:36:33.856642+0000","172.21.15.164:6800/2805828009":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/2770797546":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1837164919":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6800/1173822158":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6800/1094864383":"2024-09-20T08:45:47.960147+0000","172.21.15.164:6801/1173822158":"2024-09-20T08:37:01.663065+0000","172.21.15.164:0/339032808":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6801/2805828009":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/2087067799":"2024-09-20T08:45:47.960147+0000","172.21.15.164:0/4018459143":"2024-09-20T08:37:01.663065+0000","172.21.15.164:0/2727626049":"2024-09-20T08:45:47.960147+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:57:48.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.172+0000 7f7b5e02c640 1 -- 172.21.15.164:0/147126080 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f7b440a33c0 msgr2=0x7f7b440a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:48.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.172+0000 7f7b5e02c640 1 --2- 172.21.15.164:0/147126080 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f7b440a33c0 0x7f7b440a5880 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7b48009900 tx=0x7f7b480058d0 comp rx=0 tx=0).stop 2024-09-19T08:57:48.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.172+0000 7f7b5e02c640 1 -- 172.21.15.164:0/147126080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b58106990 msgr2=0x7f7b5819bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:48.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.172+0000 7f7b5e02c640 1 --2- 172.21.15.164:0/147126080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b58106990 0x7f7b5819bc30 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000eaa0 tx=0x7f7b5000ef60 comp rx=0 tx=0).stop 2024-09-19T08:57:48.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.172+0000 7f7b5e02c640 1 -- 172.21.15.164:0/147126080 shutdown_connections 2024-09-19T08:57:48.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.172+0000 7f7b5e02c640 1 --2- 172.21.15.164:0/147126080 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f7b440a33c0 0x7f7b440a5880 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:48.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.172+0000 7f7b5e02c640 1 --2- 172.21.15.164:0/147126080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b58106990 0x7f7b5819bc30 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:48.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.172+0000 7f7b5e02c640 1 --2- 172.21.15.164:0/147126080 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b58101320 0x7f7b581978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:48.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.172+0000 7f7b5e02c640 1 --2- 172.21.15.164:0/147126080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b58100980 0x7f7b58197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:48.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.172+0000 7f7b5e02c640 1 -- 172.21.15.164:0/147126080 >> 172.21.15.164:0/147126080 conn(0x7f7b58077a10 msgr2=0x7f7b581039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:48.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.172+0000 7f7b5e02c640 1 -- 172.21.15.164:0/147126080 shutdown_connections 2024-09-19T08:57:48.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.172+0000 7f7b5e02c640 1 -- 172.21.15.164:0/147126080 wait complete. 2024-09-19T08:57:48.182 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:47 smithi164 ceph-mon[30928]: pgmap v374: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:57:48.184 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T08:39:13.961749+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T08:43:49.613471+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T08:43:51.570675+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 94, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T08:57:48.184 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T08:57:48.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.293+0000 7fb3fed86640 1 -- 172.21.15.164:0/3483610481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3f8106990 msgr2=0x7fb3f8106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:48.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.293+0000 7fb3fed86640 1 --2- 172.21.15.164:0/3483610481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3f8106990 0x7fb3f8106d70 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec009930 tx=0x7fb3ec02a420 comp rx=0 tx=0).stop 2024-09-19T08:57:48.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.293+0000 7fb3fed86640 1 -- 172.21.15.164:0/3483610481 shutdown_connections 2024-09-19T08:57:48.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.293+0000 7fb3fed86640 1 --2- 172.21.15.164:0/3483610481 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3f8101320 0x7fb3f810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:48.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.293+0000 7fb3fed86640 1 --2- 172.21.15.164:0/3483610481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3f8100980 0x7fb3f8100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:48.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.293+0000 7fb3fed86640 1 --2- 172.21.15.164:0/3483610481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3f8106990 0x7fb3f8106d70 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:48.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.293+0000 7fb3fed86640 1 -- 172.21.15.164:0/3483610481 >> 172.21.15.164:0/3483610481 conn(0x7fb3f80779d0 msgr2=0x7fb3f80fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:48.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.293+0000 7fb3fed86640 1 -- 172.21.15.164:0/3483610481 shutdown_connections 2024-09-19T08:57:48.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.293+0000 7fb3fed86640 1 -- 172.21.15.164:0/3483610481 wait complete. 2024-09-19T08:57:48.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.293+0000 7fb3fed86640 1 Processor -- start 2024-09-19T08:57:48.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.293+0000 7fb3fed86640 1 -- start start 2024-09-19T08:57:48.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.293+0000 7fb3fed86640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3f8100980 0x7fb3f8197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:48.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.293+0000 7fb3fed86640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3f8101320 0x7fb3f81978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:48.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.293+0000 7fb3fed86640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3f8106990 0x7fb3f819bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:48.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.294+0000 7fb3fed86640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f8110dd0 con 0x7fb3f8100980 2024-09-19T08:57:48.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.294+0000 7fb3fed86640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f8110c50 con 0x7fb3f8106990 2024-09-19T08:57:48.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.294+0000 7fb3fed86640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f8110f50 con 0x7fb3f8101320 2024-09-19T08:57:48.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.294+0000 7fb3fcb22640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3f8100980 0x7fb3f8197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:48.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.294+0000 7fb3fcb22640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3f8100980 0x7fb3f8197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60684/0 (socket says 172.21.15.164:60684) 2024-09-19T08:57:48.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.294+0000 7fb3fcb22640 1 -- 172.21.15.164:0/3406641652 learned_addr learned my addr 172.21.15.164:0/3406641652 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:48.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.294+0000 7fb3f7fff640 1 --2- 172.21.15.164:0/3406641652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3f8101320 0x7fb3f81978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:48.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.294+0000 7fb3fd323640 1 --2- 172.21.15.164:0/3406641652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3f8106990 0x7fb3f819bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:48.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.294+0000 7fb3fcb22640 1 -- 172.21.15.164:0/3406641652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3f8101320 msgr2=0x7fb3f81978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:48.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.294+0000 7fb3fcb22640 1 --2- 172.21.15.164:0/3406641652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3f8101320 0x7fb3f81978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:48.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.294+0000 7fb3fcb22640 1 -- 172.21.15.164:0/3406641652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3f8106990 msgr2=0x7fb3f819bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:48.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.294+0000 7fb3fcb22640 1 --2- 172.21.15.164:0/3406641652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3f8106990 0x7fb3f819bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:48.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.294+0000 7fb3fcb22640 1 -- 172.21.15.164:0/3406641652 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f819c350 con 0x7fb3f8100980 2024-09-19T08:57:48.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.294+0000 7fb3fd323640 1 --2- 172.21.15.164:0/3406641652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3f8106990 0x7fb3f819bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:57:48.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.294+0000 7fb3f7fff640 1 --2- 172.21.15.164:0/3406641652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3f8101320 0x7fb3f81978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:57:48.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.294+0000 7fb3fcb22640 1 --2- 172.21.15.164:0/3406641652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3f8100980 0x7fb3f8197360 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec009230 tx=0x7fb3ec004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:48.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.294+0000 7fb3f5ffb640 1 -- 172.21.15.164:0/3406641652 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec03f030 con 0x7fb3f8100980 2024-09-19T08:57:48.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.294+0000 7fb3f5ffb640 1 -- 172.21.15.164:0/3406641652 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3ec003ad0 con 0x7fb3f8100980 2024-09-19T08:57:48.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.294+0000 7fb3fed86640 1 -- 172.21.15.164:0/3406641652 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3f819c5e0 con 0x7fb3f8100980 2024-09-19T08:57:48.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.294+0000 7fb3f5ffb640 1 -- 172.21.15.164:0/3406641652 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec035070 con 0x7fb3f8100980 2024-09-19T08:57:48.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.294+0000 7fb3fed86640 1 -- 172.21.15.164:0/3406641652 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3f81a3ed0 con 0x7fb3f8100980 2024-09-19T08:57:48.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.295+0000 7fb3fed86640 1 -- 172.21.15.164:0/3406641652 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3f8100de0 con 0x7fb3f8100980 2024-09-19T08:57:48.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.297+0000 7fb3f5ffb640 1 -- 172.21.15.164:0/3406641652 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fb3ec042030 con 0x7fb3f8100980 2024-09-19T08:57:48.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.298+0000 7fb3f5ffb640 1 --2- 172.21.15.164:0/3406641652 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb3e80a3440 0x7fb3e80a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:48.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.298+0000 7fb3f7fff640 1 --2- 172.21.15.164:0/3406641652 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb3e80a3440 0x7fb3e80a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:48.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.298+0000 7fb3f5ffb640 1 -- 172.21.15.164:0/3406641652 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7fb3ec0f05e0 con 0x7fb3f8100980 2024-09-19T08:57:48.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.298+0000 7fb3f7fff640 1 --2- 172.21.15.164:0/3406641652 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb3e80a3440 0x7fb3e80a5900 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb3e40058c0 tx=0x7fb3e4005850 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:48.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.301+0000 7fb3f5ffb640 1 -- 172.21.15.164:0/3406641652 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb3ec0bd000 con 0x7fb3f8100980 2024-09-19T08:57:48.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:47 smithi170 ceph-mon[35072]: pgmap v374: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:57:48.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:47 smithi173 ceph-mon[35306]: pgmap v374: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:57:48.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.468+0000 7fb3fed86640 1 -- 172.21.15.164:0/3406641652 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb3f8108ef0 con 0x7fb3f8100980 2024-09-19T08:57:48.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.469+0000 7fb3f5ffb640 1 -- 172.21.15.164:0/3406641652 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7fb3ec0bc900 con 0x7fb3f8100980 2024-09-19T08:57:48.474 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-09-19T08:57:48.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.471+0000 7fb3fed86640 1 -- 172.21.15.164:0/3406641652 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb3e80a3440 msgr2=0x7fb3e80a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:48.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.471+0000 7fb3fed86640 1 --2- 172.21.15.164:0/3406641652 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb3e80a3440 0x7fb3e80a5900 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb3e40058c0 tx=0x7fb3e4005850 comp rx=0 tx=0).stop 2024-09-19T08:57:48.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.471+0000 7fb3fed86640 1 -- 172.21.15.164:0/3406641652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3f8100980 msgr2=0x7fb3f8197360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:48.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.471+0000 7fb3fed86640 1 --2- 172.21.15.164:0/3406641652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3f8100980 0x7fb3f8197360 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec009230 tx=0x7fb3ec004270 comp rx=0 tx=0).stop 2024-09-19T08:57:48.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.471+0000 7fb3fed86640 1 -- 172.21.15.164:0/3406641652 shutdown_connections 2024-09-19T08:57:48.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.471+0000 7fb3fed86640 1 --2- 172.21.15.164:0/3406641652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3f8106990 0x7fb3f819bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:48.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.471+0000 7fb3fed86640 1 --2- 172.21.15.164:0/3406641652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3f8101320 0x7fb3f81978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:48.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.471+0000 7fb3fed86640 1 --2- 172.21.15.164:0/3406641652 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb3e80a3440 0x7fb3e80a5900 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:48.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.471+0000 7fb3fed86640 1 --2- 172.21.15.164:0/3406641652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3f8100980 0x7fb3f8197360 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:48.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.471+0000 7fb3fed86640 1 -- 172.21.15.164:0/3406641652 >> 172.21.15.164:0/3406641652 conn(0x7fb3f80779d0 msgr2=0x7fb3f8103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:48.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.471+0000 7fb3fed86640 1 -- 172.21.15.164:0/3406641652 shutdown_connections 2024-09-19T08:57:48.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.471+0000 7fb3fed86640 1 -- 172.21.15.164:0/3406641652 wait complete. 2024-09-19T08:57:48.482 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T08:57:48.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.586+0000 7ffa6d4ac640 1 -- 172.21.15.164:0/2175469902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa68101a80 msgr2=0x7ffa68101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:48.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.586+0000 7ffa6d4ac640 1 --2- 172.21.15.164:0/2175469902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa68101a80 0x7ffa68101ee0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c009cd0 tx=0x7ffa5c02a8e0 comp rx=0 tx=0).stop 2024-09-19T08:57:48.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.586+0000 7ffa6d4ac640 1 -- 172.21.15.164:0/2175469902 shutdown_connections 2024-09-19T08:57:48.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.586+0000 7ffa6d4ac640 1 --2- 172.21.15.164:0/2175469902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa68102420 0x7ffa6810e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:48.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.586+0000 7ffa6d4ac640 1 --2- 172.21.15.164:0/2175469902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa68101a80 0x7ffa68101ee0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:48.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.586+0000 7ffa6d4ac640 1 --2- 172.21.15.164:0/2175469902 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa68107a90 0x7ffa68107e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:48.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.586+0000 7ffa6d4ac640 1 -- 172.21.15.164:0/2175469902 >> 172.21.15.164:0/2175469902 conn(0x7ffa680fd7c0 msgr2=0x7ffa680ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:48.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.586+0000 7ffa6d4ac640 1 -- 172.21.15.164:0/2175469902 shutdown_connections 2024-09-19T08:57:48.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.586+0000 7ffa6d4ac640 1 -- 172.21.15.164:0/2175469902 wait complete. 2024-09-19T08:57:48.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.587+0000 7ffa6d4ac640 1 Processor -- start 2024-09-19T08:57:48.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.587+0000 7ffa6d4ac640 1 -- start start 2024-09-19T08:57:48.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.587+0000 7ffa6d4ac640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa68101a80 0x7ffa68198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:48.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.587+0000 7ffa6d4ac640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa68102420 0x7ffa681989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:48.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.587+0000 7ffa6d4ac640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa68107a90 0x7ffa6819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:48.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.587+0000 7ffa6d4ac640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffa68111ed0 con 0x7ffa68102420 2024-09-19T08:57:48.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.587+0000 7ffa6d4ac640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ffa68111d50 con 0x7ffa68107a90 2024-09-19T08:57:48.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.587+0000 7ffa6d4ac640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ffa68112050 con 0x7ffa68101a80 2024-09-19T08:57:48.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.587+0000 7ffa667fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa68102420 0x7ffa681989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:48.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.587+0000 7ffa667fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa68102420 0x7ffa681989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60710/0 (socket says 172.21.15.164:60710) 2024-09-19T08:57:48.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.587+0000 7ffa667fc640 1 -- 172.21.15.164:0/3634992813 learned_addr learned my addr 172.21.15.164:0/3634992813 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:48.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.587+0000 7ffa66ffd640 1 --2- 172.21.15.164:0/3634992813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa68101a80 0x7ffa68198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:48.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.588+0000 7ffa677fe640 1 --2- 172.21.15.164:0/3634992813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa68107a90 0x7ffa6819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:48.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.588+0000 7ffa667fc640 1 -- 172.21.15.164:0/3634992813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa68101a80 msgr2=0x7ffa68198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:48.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.588+0000 7ffa667fc640 1 --2- 172.21.15.164:0/3634992813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa68101a80 0x7ffa68198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:48.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.588+0000 7ffa667fc640 1 -- 172.21.15.164:0/3634992813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa68107a90 msgr2=0x7ffa6819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:48.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.588+0000 7ffa667fc640 1 --2- 172.21.15.164:0/3634992813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa68107a90 0x7ffa6819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:48.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.588+0000 7ffa667fc640 1 -- 172.21.15.164:0/3634992813 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa6819d450 con 0x7ffa68102420 2024-09-19T08:57:48.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.588+0000 7ffa677fe640 1 --2- 172.21.15.164:0/3634992813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa68107a90 0x7ffa6819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:57:48.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.588+0000 7ffa667fc640 1 --2- 172.21.15.164:0/3634992813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa68102420 0x7ffa681989a0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c02ae00 tx=0x7ffa5c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:48.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.588+0000 7ffa4ffff640 1 -- 172.21.15.164:0/3634992813 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c004040 con 0x7ffa68102420 2024-09-19T08:57:48.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.588+0000 7ffa6d4ac640 1 -- 172.21.15.164:0/3634992813 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa6819d6e0 con 0x7ffa68102420 2024-09-19T08:57:48.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.588+0000 7ffa4ffff640 1 -- 172.21.15.164:0/3634992813 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa5c0046c0 con 0x7ffa68102420 2024-09-19T08:57:48.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.588+0000 7ffa4ffff640 1 -- 172.21.15.164:0/3634992813 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c046480 con 0x7ffa68102420 2024-09-19T08:57:48.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.588+0000 7ffa6d4ac640 1 -- 172.21.15.164:0/3634992813 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa681a4fd0 con 0x7ffa68102420 2024-09-19T08:57:48.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.589+0000 7ffa6d4ac640 1 -- 172.21.15.164:0/3634992813 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa68101ee0 con 0x7ffa68102420 2024-09-19T08:57:48.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.590+0000 7ffa4ffff640 1 -- 172.21.15.164:0/3634992813 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7ffa5c0305a0 con 0x7ffa68102420 2024-09-19T08:57:48.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.591+0000 7ffa4ffff640 1 --2- 172.21.15.164:0/3634992813 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ffa500a33c0 0x7ffa500a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:48.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.591+0000 7ffa66ffd640 1 --2- 172.21.15.164:0/3634992813 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ffa500a33c0 0x7ffa500a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:48.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.591+0000 7ffa4ffff640 1 -- 172.21.15.164:0/3634992813 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7ffa5c02e030 con 0x7ffa68102420 2024-09-19T08:57:48.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.592+0000 7ffa66ffd640 1 --2- 172.21.15.164:0/3634992813 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ffa500a33c0 0x7ffa500a5880 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ffa60005fd0 tx=0x7ffa6000b390 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:48.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.595+0000 7ffa4ffff640 1 -- 172.21.15.164:0/3634992813 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ffa5c04b060 con 0x7ffa68102420 2024-09-19T08:57:48.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.761+0000 7ffa6d4ac640 1 -- 172.21.15.164:0/3634992813 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ffa6802d7c0 con 0x7ffa68102420 2024-09-19T08:57:48.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.761+0000 7ffa4ffff640 1 -- 172.21.15.164:0/3634992813 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7ffa5c030890 con 0x7ffa68102420 2024-09-19T08:57:48.762 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-09-19T08:57:48.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.763+0000 7ffa6d4ac640 1 -- 172.21.15.164:0/3634992813 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ffa500a33c0 msgr2=0x7ffa500a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:48.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.763+0000 7ffa6d4ac640 1 --2- 172.21.15.164:0/3634992813 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ffa500a33c0 0x7ffa500a5880 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ffa60005fd0 tx=0x7ffa6000b390 comp rx=0 tx=0).stop 2024-09-19T08:57:48.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.763+0000 7ffa6d4ac640 1 -- 172.21.15.164:0/3634992813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa68102420 msgr2=0x7ffa681989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:48.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.763+0000 7ffa6d4ac640 1 --2- 172.21.15.164:0/3634992813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa68102420 0x7ffa681989a0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c02ae00 tx=0x7ffa5c003970 comp rx=0 tx=0).stop 2024-09-19T08:57:48.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.764+0000 7ffa6d4ac640 1 -- 172.21.15.164:0/3634992813 shutdown_connections 2024-09-19T08:57:48.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.764+0000 7ffa6d4ac640 1 --2- 172.21.15.164:0/3634992813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa68107a90 0x7ffa6819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:48.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.764+0000 7ffa6d4ac640 1 --2- 172.21.15.164:0/3634992813 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ffa500a33c0 0x7ffa500a5880 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:48.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.764+0000 7ffa6d4ac640 1 --2- 172.21.15.164:0/3634992813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa68102420 0x7ffa681989a0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:48.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.764+0000 7ffa6d4ac640 1 --2- 172.21.15.164:0/3634992813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa68101a80 0x7ffa68198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:48.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.764+0000 7ffa6d4ac640 1 -- 172.21.15.164:0/3634992813 >> 172.21.15.164:0/3634992813 conn(0x7ffa680fd7c0 msgr2=0x7ffa680fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:48.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.764+0000 7ffa6d4ac640 1 -- 172.21.15.164:0/3634992813 shutdown_connections 2024-09-19T08:57:48.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.764+0000 7ffa6d4ac640 1 -- 172.21.15.164:0/3634992813 wait complete. 2024-09-19T08:57:48.775 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T08:57:48.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.877+0000 7f1f620e4640 1 -- 172.21.15.164:0/2232408708 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f5c1013c0 msgr2=0x7f1f5c10d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:48.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.877+0000 7f1f620e4640 1 --2- 172.21.15.164:0/2232408708 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f5c1013c0 0x7f1f5c10d8f0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f1f54009cd0 tx=0x7f1f5402a8e0 comp rx=0 tx=0).stop 2024-09-19T08:57:48.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.877+0000 7f1f620e4640 1 -- 172.21.15.164:0/2232408708 shutdown_connections 2024-09-19T08:57:48.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.877+0000 7f1f620e4640 1 --2- 172.21.15.164:0/2232408708 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f5c1013c0 0x7f1f5c10d8f0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:48.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.877+0000 7f1f620e4640 1 --2- 172.21.15.164:0/2232408708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f5c100a20 0x7f1f5c100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:48.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.877+0000 7f1f620e4640 1 --2- 172.21.15.164:0/2232408708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f5c106a30 0x7f1f5c106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:48.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.877+0000 7f1f620e4640 1 -- 172.21.15.164:0/2232408708 >> 172.21.15.164:0/2232408708 conn(0x7f1f5c0fc740 msgr2=0x7f1f5c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:48.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.877+0000 7f1f620e4640 1 -- 172.21.15.164:0/2232408708 shutdown_connections 2024-09-19T08:57:48.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.877+0000 7f1f620e4640 1 -- 172.21.15.164:0/2232408708 wait complete. 2024-09-19T08:57:48.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.878+0000 7f1f620e4640 1 Processor -- start 2024-09-19T08:57:48.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.878+0000 7f1f620e4640 1 -- start start 2024-09-19T08:57:48.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.878+0000 7f1f620e4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f5c100a20 0x7f1f5c1991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:48.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.878+0000 7f1f620e4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f5c1013c0 0x7f1f5c199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:48.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.878+0000 7f1f620e4640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f5c106a30 0x7f1f5c1a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:48.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.878+0000 7f1f620e4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5c110e70 con 0x7f1f5c100a20 2024-09-19T08:57:48.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.878+0000 7f1f620e4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5c110cf0 con 0x7f1f5c1013c0 2024-09-19T08:57:48.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.878+0000 7f1f620e4640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5c110ff0 con 0x7f1f5c106a30 2024-09-19T08:57:48.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.878+0000 7f1f5b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f5c100a20 0x7f1f5c1991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:48.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.878+0000 7f1f5b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f5c100a20 0x7f1f5c1991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60726/0 (socket says 172.21.15.164:60726) 2024-09-19T08:57:48.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.878+0000 7f1f5b7fe640 1 -- 172.21.15.164:0/3271228841 learned_addr learned my addr 172.21.15.164:0/3271228841 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:48.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.878+0000 7f1f5affd640 1 --2- 172.21.15.164:0/3271228841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f5c1013c0 0x7f1f5c199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:48.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.878+0000 7f1f5bfff640 1 --2- 172.21.15.164:0/3271228841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f5c106a30 0x7f1f5c1a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:48.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.879+0000 7f1f5b7fe640 1 -- 172.21.15.164:0/3271228841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f5c106a30 msgr2=0x7f1f5c1a4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:48.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.879+0000 7f1f5b7fe640 1 --2- 172.21.15.164:0/3271228841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f5c106a30 0x7f1f5c1a4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:48.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.879+0000 7f1f5b7fe640 1 -- 172.21.15.164:0/3271228841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f5c1013c0 msgr2=0x7f1f5c199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:48.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.879+0000 7f1f5b7fe640 1 --2- 172.21.15.164:0/3271228841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f5c1013c0 0x7f1f5c199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:48.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.879+0000 7f1f5b7fe640 1 -- 172.21.15.164:0/3271228841 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f5c1a47b0 con 0x7f1f5c100a20 2024-09-19T08:57:48.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.879+0000 7f1f5b7fe640 1 --2- 172.21.15.164:0/3271228841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f5c100a20 0x7f1f5c1991f0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00c890 tx=0x7f1f4c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:48.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.879+0000 7f1f58ff9640 1 -- 172.21.15.164:0/3271228841 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c004220 con 0x7f1f5c100a20 2024-09-19T08:57:48.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.879+0000 7f1f58ff9640 1 -- 172.21.15.164:0/3271228841 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f4c0043c0 con 0x7f1f5c100a20 2024-09-19T08:57:48.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.879+0000 7f1f58ff9640 1 -- 172.21.15.164:0/3271228841 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c004a00 con 0x7f1f5c100a20 2024-09-19T08:57:48.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.879+0000 7f1f620e4640 1 -- 172.21.15.164:0/3271228841 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f5c1a4980 con 0x7f1f5c100a20 2024-09-19T08:57:48.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.879+0000 7f1f620e4640 1 -- 172.21.15.164:0/3271228841 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f5c1a4ea0 con 0x7f1f5c100a20 2024-09-19T08:57:48.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.880+0000 7f1f620e4640 1 -- 172.21.15.164:0/3271228841 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f5c100e80 con 0x7f1f5c100a20 2024-09-19T08:57:48.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.881+0000 7f1f58ff9640 1 -- 172.21.15.164:0/3271228841 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f1f4c0079b0 con 0x7f1f5c100a20 2024-09-19T08:57:48.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.882+0000 7f1f58ff9640 1 --2- 172.21.15.164:0/3271228841 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f1f440a33c0 0x7f1f440a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:48.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.882+0000 7f1f5affd640 1 --2- 172.21.15.164:0/3271228841 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f1f440a33c0 0x7f1f440a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:48.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.882+0000 7f1f58ff9640 1 -- 172.21.15.164:0/3271228841 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7f1f4c0cf880 con 0x7f1f5c100a20 2024-09-19T08:57:48.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.882+0000 7f1f5affd640 1 --2- 172.21.15.164:0/3271228841 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f1f440a33c0 0x7f1f440a5880 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c06aef0 tx=0x7f1f5000a2d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:48.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:48.886+0000 7f1f58ff9640 1 -- 172.21.15.164:0/3271228841 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1f4c014030 con 0x7f1f5c100a20 2024-09-19T08:57:49.031 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:48 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/147126080' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:57:49.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:48 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3406641652' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T08:57:49.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:48 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3634992813' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T08:57:49.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.055+0000 7f1f620e4640 1 -- 172.21.15.164:0/3271228841 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1f5c1a5100 con 0x7f1f5c100a20 2024-09-19T08:57:49.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.055+0000 7f1f58ff9640 1 -- 172.21.15.164:0/3271228841 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7f1f4c09c2a0 con 0x7f1f5c100a20 2024-09-19T08:57:49.055 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-09-19T08:57:49.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.057+0000 7f1f620e4640 1 -- 172.21.15.164:0/3271228841 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f1f440a33c0 msgr2=0x7f1f440a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:49.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.057+0000 7f1f620e4640 1 --2- 172.21.15.164:0/3271228841 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f1f440a33c0 0x7f1f440a5880 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c06aef0 tx=0x7f1f5000a2d0 comp rx=0 tx=0).stop 2024-09-19T08:57:49.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.057+0000 7f1f620e4640 1 -- 172.21.15.164:0/3271228841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f5c100a20 msgr2=0x7f1f5c1991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:49.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.057+0000 7f1f620e4640 1 --2- 172.21.15.164:0/3271228841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f5c100a20 0x7f1f5c1991f0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00c890 tx=0x7f1f4c00cc90 comp rx=0 tx=0).stop 2024-09-19T08:57:49.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.057+0000 7f1f620e4640 1 -- 172.21.15.164:0/3271228841 shutdown_connections 2024-09-19T08:57:49.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.057+0000 7f1f620e4640 1 --2- 172.21.15.164:0/3271228841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f5c106a30 0x7f1f5c1a4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:49.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.057+0000 7f1f620e4640 1 --2- 172.21.15.164:0/3271228841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f5c1013c0 0x7f1f5c199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:49.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.057+0000 7f1f620e4640 1 --2- 172.21.15.164:0/3271228841 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f1f440a33c0 0x7f1f440a5880 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:49.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.057+0000 7f1f620e4640 1 --2- 172.21.15.164:0/3271228841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f5c100a20 0x7f1f5c1991f0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:49.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.057+0000 7f1f620e4640 1 -- 172.21.15.164:0/3271228841 >> 172.21.15.164:0/3271228841 conn(0x7f1f5c0fc740 msgr2=0x7f1f5c0fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:49.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.057+0000 7f1f620e4640 1 -- 172.21.15.164:0/3271228841 shutdown_connections 2024-09-19T08:57:49.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.057+0000 7f1f620e4640 1 -- 172.21.15.164:0/3271228841 wait complete. 2024-09-19T08:57:49.069 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-19T08:57:49.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.168+0000 7fb1ea5f6640 1 -- 172.21.15.164:0/2448373188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1e410e360 msgr2=0x7fb1e4110750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:49.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.168+0000 7fb1ea5f6640 1 --2- 172.21.15.164:0/2448373188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1e410e360 0x7fb1e4110750 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc009ca0 tx=0x7fb1dc02a950 comp rx=0 tx=0).stop 2024-09-19T08:57:49.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.168+0000 7fb1ea5f6640 1 -- 172.21.15.164:0/2448373188 shutdown_connections 2024-09-19T08:57:49.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.168+0000 7fb1ea5f6640 1 --2- 172.21.15.164:0/2448373188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1e410e360 0x7fb1e4110750 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:49.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.168+0000 7fb1ea5f6640 1 --2- 172.21.15.164:0/2448373188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1e4074ec0 0x7fb1e410de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:49.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.168+0000 7fb1ea5f6640 1 --2- 172.21.15.164:0/2448373188 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1e40745a0 0x7fb1e4074980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:49.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.168+0000 7fb1ea5f6640 1 -- 172.21.15.164:0/2448373188 >> 172.21.15.164:0/2448373188 conn(0x7fb1e40fd7c0 msgr2=0x7fb1e40ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:49.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.168+0000 7fb1ea5f6640 1 -- 172.21.15.164:0/2448373188 shutdown_connections 2024-09-19T08:57:49.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.168+0000 7fb1ea5f6640 1 -- 172.21.15.164:0/2448373188 wait complete. 2024-09-19T08:57:49.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.169+0000 7fb1ea5f6640 1 Processor -- start 2024-09-19T08:57:49.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.169+0000 7fb1ea5f6640 1 -- start start 2024-09-19T08:57:49.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.169+0000 7fb1ea5f6640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1e40745a0 0x7fb1e419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:49.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.169+0000 7fb1ea5f6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1e4074ec0 0x7fb1e419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:49.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.169+0000 7fb1ea5f6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1e410e360 0x7fb1e419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:49.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.169+0000 7fb1ea5f6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e4114090 con 0x7fb1e410e360 2024-09-19T08:57:49.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.169+0000 7fb1ea5f6640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e4113f10 con 0x7fb1e4074ec0 2024-09-19T08:57:49.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.169+0000 7fb1ea5f6640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e4114210 con 0x7fb1e40745a0 2024-09-19T08:57:49.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.169+0000 7fb1e8b93640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1e410e360 0x7fb1e419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:49.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.169+0000 7fb1e8b93640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1e410e360 0x7fb1e419ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60746/0 (socket says 172.21.15.164:60746) 2024-09-19T08:57:49.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.169+0000 7fb1e8b93640 1 -- 172.21.15.164:0/1145203784 learned_addr learned my addr 172.21.15.164:0/1145203784 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:57:49.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.169+0000 7fb1e3fff640 1 --2- 172.21.15.164:0/1145203784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1e40745a0 0x7fb1e419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:49.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.169+0000 7fb1e37fe640 1 --2- 172.21.15.164:0/1145203784 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1e4074ec0 0x7fb1e419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:49.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.170+0000 7fb1e8b93640 1 -- 172.21.15.164:0/1145203784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1e40745a0 msgr2=0x7fb1e419a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:49.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.170+0000 7fb1e8b93640 1 --2- 172.21.15.164:0/1145203784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1e40745a0 0x7fb1e419a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:49.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.170+0000 7fb1e8b93640 1 -- 172.21.15.164:0/1145203784 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1e4074ec0 msgr2=0x7fb1e419aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:49.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.170+0000 7fb1e8b93640 1 --2- 172.21.15.164:0/1145203784 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1e4074ec0 0x7fb1e419aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:49.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.170+0000 7fb1e8b93640 1 -- 172.21.15.164:0/1145203784 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e419f550 con 0x7fb1e410e360 2024-09-19T08:57:49.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.170+0000 7fb1e37fe640 1 --2- 172.21.15.164:0/1145203784 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1e4074ec0 0x7fb1e419aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:57:49.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.170+0000 7fb1e8b93640 1 --2- 172.21.15.164:0/1145203784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1e410e360 0x7fb1e419ee20 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc006010 tx=0x7fb1dc02f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:49.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.170+0000 7fb1e17fa640 1 -- 172.21.15.164:0/1145203784 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc034040 con 0x7fb1e410e360 2024-09-19T08:57:49.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.170+0000 7fb1e17fa640 1 -- 172.21.15.164:0/1145203784 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1dc003e00 con 0x7fb1e410e360 2024-09-19T08:57:49.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.170+0000 7fb1ea5f6640 1 -- 172.21.15.164:0/1145203784 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1e419f7e0 con 0x7fb1e410e360 2024-09-19T08:57:49.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.170+0000 7fb1e17fa640 1 -- 172.21.15.164:0/1145203784 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc045480 con 0x7fb1e410e360 2024-09-19T08:57:49.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.170+0000 7fb1ea5f6640 1 -- 172.21.15.164:0/1145203784 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1e41a70d0 con 0x7fb1e410e360 2024-09-19T08:57:49.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.171+0000 7fb1ea5f6640 1 -- 172.21.15.164:0/1145203784 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1e4074980 con 0x7fb1e410e360 2024-09-19T08:57:49.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.172+0000 7fb1e17fa640 1 -- 172.21.15.164:0/1145203784 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fb1dc037020 con 0x7fb1e410e360 2024-09-19T08:57:49.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.173+0000 7fb1e17fa640 1 --2- 172.21.15.164:0/1145203784 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb1cc0a33c0 0x7fb1cc0a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:57:49.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.173+0000 7fb1e3fff640 1 --2- 172.21.15.164:0/1145203784 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb1cc0a33c0 0x7fb1cc0a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:57:49.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.173+0000 7fb1e17fa640 1 -- 172.21.15.164:0/1145203784 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7fb1dc0f0d10 con 0x7fb1e410e360 2024-09-19T08:57:49.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.173+0000 7fb1e3fff640 1 --2- 172.21.15.164:0/1145203784 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb1cc0a33c0 0x7fb1cc0a5880 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400abe0 tx=0x7fb1d400a2d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:57:49.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.177+0000 7fb1e17fa640 1 -- 172.21.15.164:0/1145203784 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb1dc0bd840 con 0x7fb1e410e360 2024-09-19T08:57:49.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:48 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/147126080' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:57:49.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:48 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3406641652' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T08:57:49.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:48 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3634992813' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T08:57:49.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.390+0000 7fb1ea5f6640 1 -- 172.21.15.164:0/1145203784 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fb1e41a72a0 con 0x7fb1e410e360 2024-09-19T08:57:49.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.398+0000 7fb1e17fa640 1 -- 172.21.15.164:0/1145203784 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+742879 (secure 0 0 0) 0x7fb1dc0bd140 con 0x7fb1e410e360 2024-09-19T08:57:49.409 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:57:49.409 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:57:49.409 INFO:teuthology.orchestra.run.smithi164.stdout: "epoch": 26, 2024-09-19T08:57:49.409 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.409 INFO:teuthology.orchestra.run.smithi164.stdout: "active_gid": 15201, 2024-09-19T08:57:49.409 INFO:teuthology.orchestra.run.smithi164.stdout: "active_name": "x", 2024-09-19T08:57:49.410 INFO:teuthology.orchestra.run.smithi164.stdout: "active_addrs": { 2024-09-19T08:57:49.410 INFO:teuthology.orchestra.run.smithi164.stdout: "addrvec": [ 2024-09-19T08:57:49.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.410 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v2", 2024-09-19T08:57:49.410 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "172.21.15.164:6800", 2024-09-19T08:57:49.410 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 173486514 2024-09-19T08:57:49.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.410 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v1", 2024-09-19T08:57:49.410 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "172.21.15.164:6801", 2024-09-19T08:57:49.410 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 173486514 2024-09-19T08:57:49.410 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.410 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-19T08:57:49.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.411 INFO:teuthology.orchestra.run.smithi164.stdout: "active_addr": "172.21.15.164:6801/173486514", 2024-09-19T08:57:49.411 INFO:teuthology.orchestra.run.smithi164.stdout: "active_change": "2024-09-19T08:45:47.960351+0000", 2024-09-19T08:57:49.411 INFO:teuthology.orchestra.run.smithi164.stdout: "active_mgr_features": 4540701547738038271, 2024-09-19T08:57:49.411 INFO:teuthology.orchestra.run.smithi164.stdout: "available": true, 2024-09-19T08:57:49.411 INFO:teuthology.orchestra.run.smithi164.stdout: "standbys": [ 2024-09-19T08:57:49.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.411 INFO:teuthology.orchestra.run.smithi164.stdout: "gid": 24793, 2024-09-19T08:57:49.411 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "z", 2024-09-19T08:57:49.411 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr_features": 4540701547738038271, 2024-09-19T08:57:49.411 INFO:teuthology.orchestra.run.smithi164.stdout: "available_modules": [ 2024-09-19T08:57:49.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.411 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "alerts", 2024-09-19T08:57:49.412 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.412 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.412 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.412 INFO:teuthology.orchestra.run.smithi164.stdout: "interval": { 2024-09-19T08:57:49.412 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "interval", 2024-09-19T08:57:49.412 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:49.412 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.412 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.412 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "60", 2024-09-19T08:57:49.412 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.412 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.412 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.413 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T08:57:49.413 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.413 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.413 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.413 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.413 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.413 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.413 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.413 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.413 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.413 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.413 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.414 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.414 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.414 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.414 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.414 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.414 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.414 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.414 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.414 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.414 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.414 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.414 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.415 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.415 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.415 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.415 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.415 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.415 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.415 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.415 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.415 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.415 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.415 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.416 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.416 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.416 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.416 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.416 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.416 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.416 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.416 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.416 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.416 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.416 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.417 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.417 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.417 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.417 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.417 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.417 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.417 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.417 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.417 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.417 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.417 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.417 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.418 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.418 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.418 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.418 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.418 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.418 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.418 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.418 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.418 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.418 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.418 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.419 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.419 INFO:teuthology.orchestra.run.smithi164.stdout: "smtp_destination": { 2024-09-19T08:57:49.419 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "smtp_destination", 2024-09-19T08:57:49.419 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.419 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.419 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.419 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.419 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.419 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.419 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.419 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Email address to send alerts to", 2024-09-19T08:57:49.419 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.420 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.420 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.420 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.420 INFO:teuthology.orchestra.run.smithi164.stdout: "smtp_from_name": { 2024-09-19T08:57:49.420 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "smtp_from_name", 2024-09-19T08:57:49.420 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.420 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.420 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.420 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "Ceph", 2024-09-19T08:57:49.420 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.420 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.420 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.420 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Email From: name", 2024-09-19T08:57:49.421 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.421 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.421 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.421 INFO:teuthology.orchestra.run.smithi164.stdout: "smtp_host": { 2024-09-19T08:57:49.421 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "smtp_host", 2024-09-19T08:57:49.421 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.421 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.421 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.421 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.421 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.422 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.422 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.422 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "SMTP server", 2024-09-19T08:57:49.422 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.422 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.422 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.422 INFO:teuthology.orchestra.run.smithi164.stdout: "smtp_password": { 2024-09-19T08:57:49.422 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "smtp_password", 2024-09-19T08:57:49.422 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.422 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.422 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.422 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.423 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.423 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.423 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.423 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Password to authenticate with", 2024-09-19T08:57:49.423 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.423 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.423 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.423 INFO:teuthology.orchestra.run.smithi164.stdout: "smtp_port": { 2024-09-19T08:57:49.423 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "smtp_port", 2024-09-19T08:57:49.423 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.423 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.423 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.424 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "465", 2024-09-19T08:57:49.424 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.424 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.424 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.424 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "SMTP port", 2024-09-19T08:57:49.424 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.424 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.424 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.424 INFO:teuthology.orchestra.run.smithi164.stdout: "smtp_sender": { 2024-09-19T08:57:49.424 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "smtp_sender", 2024-09-19T08:57:49.424 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.425 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.425 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.425 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.425 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.425 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.425 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.425 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "SMTP envelope sender", 2024-09-19T08:57:49.425 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.425 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.425 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.426 INFO:teuthology.orchestra.run.smithi164.stdout: "smtp_ssl": { 2024-09-19T08:57:49.426 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "smtp_ssl", 2024-09-19T08:57:49.426 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.426 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.426 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.426 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.426 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.426 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.426 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.426 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T08:57:49.426 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.426 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.426 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.427 INFO:teuthology.orchestra.run.smithi164.stdout: "smtp_user": { 2024-09-19T08:57:49.427 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "smtp_user", 2024-09-19T08:57:49.427 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.427 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.427 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.427 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.427 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.427 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.427 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.427 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "User to authenticate as", 2024-09-19T08:57:49.427 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.427 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.428 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.428 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.428 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.428 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "balancer", 2024-09-19T08:57:49.428 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.428 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.428 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.428 INFO:teuthology.orchestra.run.smithi164.stdout: "active": { 2024-09-19T08:57:49.428 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "active", 2024-09-19T08:57:49.428 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.428 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.429 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.429 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.429 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.429 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.429 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.429 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T08:57:49.429 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.429 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.429 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.429 INFO:teuthology.orchestra.run.smithi164.stdout: "begin_time": { 2024-09-19T08:57:49.429 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "begin_time", 2024-09-19T08:57:49.429 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.430 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.430 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.430 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "0000", 2024-09-19T08:57:49.430 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.430 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.430 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.430 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T08:57:49.430 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T08:57:49.430 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.430 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.430 INFO:teuthology.orchestra.run.smithi164.stdout: "begin_weekday": { 2024-09-19T08:57:49.430 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "begin_weekday", 2024-09-19T08:57:49.431 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "uint", 2024-09-19T08:57:49.431 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.431 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.431 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "0", 2024-09-19T08:57:49.431 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "0", 2024-09-19T08:57:49.431 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "6", 2024-09-19T08:57:49.431 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.431 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T08:57:49.431 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T08:57:49.431 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.431 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.431 INFO:teuthology.orchestra.run.smithi164.stdout: "crush_compat_max_iterations": { 2024-09-19T08:57:49.432 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "crush_compat_max_iterations", 2024-09-19T08:57:49.432 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "uint", 2024-09-19T08:57:49.432 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.432 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.432 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "25", 2024-09-19T08:57:49.432 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "1", 2024-09-19T08:57:49.432 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "250", 2024-09-19T08:57:49.432 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.432 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T08:57:49.432 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.432 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.432 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.433 INFO:teuthology.orchestra.run.smithi164.stdout: "crush_compat_metrics": { 2024-09-19T08:57:49.433 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "crush_compat_metrics", 2024-09-19T08:57:49.433 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.433 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.433 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.433 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T08:57:49.433 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.433 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.433 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.433 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T08:57:49.433 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T08:57:49.433 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.433 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.434 INFO:teuthology.orchestra.run.smithi164.stdout: "crush_compat_step": { 2024-09-19T08:57:49.434 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "crush_compat_step", 2024-09-19T08:57:49.434 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "float", 2024-09-19T08:57:49.434 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.434 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.434 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "0.5", 2024-09-19T08:57:49.434 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "0.001", 2024-09-19T08:57:49.434 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "0.999", 2024-09-19T08:57:49.434 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.434 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "aggressiveness of optimization", 2024-09-19T08:57:49.434 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T08:57:49.434 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.435 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.435 INFO:teuthology.orchestra.run.smithi164.stdout: "end_time": { 2024-09-19T08:57:49.435 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "end_time", 2024-09-19T08:57:49.435 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.435 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.435 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.435 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "2359", 2024-09-19T08:57:49.435 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.435 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.435 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.435 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T08:57:49.435 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T08:57:49.436 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.436 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.436 INFO:teuthology.orchestra.run.smithi164.stdout: "end_weekday": { 2024-09-19T08:57:49.436 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "end_weekday", 2024-09-19T08:57:49.436 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "uint", 2024-09-19T08:57:49.436 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.436 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.436 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "0", 2024-09-19T08:57:49.436 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "0", 2024-09-19T08:57:49.436 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "6", 2024-09-19T08:57:49.436 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.436 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T08:57:49.437 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T08:57:49.437 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.437 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.437 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.437 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.437 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.437 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.437 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.437 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.437 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.438 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.438 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.438 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.438 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.438 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.438 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.438 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.438 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.438 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.438 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.438 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.438 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.438 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.439 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.439 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.439 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.439 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.439 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.439 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.439 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.439 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.439 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.439 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.439 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.439 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.440 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.440 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.440 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.440 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.440 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.440 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.440 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.440 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.440 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.440 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.440 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.441 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.441 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.441 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.441 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.441 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.441 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.441 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.441 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.441 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.441 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.441 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.441 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.442 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.442 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.442 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.442 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.442 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.442 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.442 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.442 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.442 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.442 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.442 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.442 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.443 INFO:teuthology.orchestra.run.smithi164.stdout: "min_score": { 2024-09-19T08:57:49.443 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "min_score", 2024-09-19T08:57:49.443 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "float", 2024-09-19T08:57:49.443 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.443 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.443 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "0", 2024-09-19T08:57:49.443 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.443 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.443 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.443 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T08:57:49.443 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.443 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.443 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.444 INFO:teuthology.orchestra.run.smithi164.stdout: "mode": { 2024-09-19T08:57:49.444 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "mode", 2024-09-19T08:57:49.444 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.444 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.444 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.444 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "upmap", 2024-09-19T08:57:49.444 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.444 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.444 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.444 INFO:teuthology.orchestra.run.smithi164.stdout: "crush-compat", 2024-09-19T08:57:49.444 INFO:teuthology.orchestra.run.smithi164.stdout: "none", 2024-09-19T08:57:49.444 INFO:teuthology.orchestra.run.smithi164.stdout: "read", 2024-09-19T08:57:49.445 INFO:teuthology.orchestra.run.smithi164.stdout: "upmap", 2024-09-19T08:57:49.445 INFO:teuthology.orchestra.run.smithi164.stdout: "upmap-read" 2024-09-19T08:57:49.445 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.445 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Balancer mode", 2024-09-19T08:57:49.445 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.445 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.445 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.445 INFO:teuthology.orchestra.run.smithi164.stdout: "pool_ids": { 2024-09-19T08:57:49.445 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "pool_ids", 2024-09-19T08:57:49.445 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.445 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.445 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.446 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.446 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.446 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.446 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.446 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T08:57:49.446 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.446 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.446 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.446 INFO:teuthology.orchestra.run.smithi164.stdout: "sleep_interval": { 2024-09-19T08:57:49.446 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "sleep_interval", 2024-09-19T08:57:49.446 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:49.446 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.447 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.447 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "60", 2024-09-19T08:57:49.447 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.447 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.447 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.447 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T08:57:49.447 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.447 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.447 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.447 INFO:teuthology.orchestra.run.smithi164.stdout: "upmap_max_deviation": { 2024-09-19T08:57:49.447 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "upmap_max_deviation", 2024-09-19T08:57:49.447 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.447 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.448 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.448 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "5", 2024-09-19T08:57:49.448 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "1", 2024-09-19T08:57:49.448 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.448 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.448 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T08:57:49.448 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T08:57:49.448 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.448 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.448 INFO:teuthology.orchestra.run.smithi164.stdout: "upmap_max_optimizations": { 2024-09-19T08:57:49.448 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "upmap_max_optimizations", 2024-09-19T08:57:49.448 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "uint", 2024-09-19T08:57:49.449 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.449 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.449 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "10", 2024-09-19T08:57:49.449 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.449 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.449 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.449 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T08:57:49.449 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.449 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.449 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.449 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.449 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.450 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "cephadm", 2024-09-19T08:57:49.450 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.450 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.450 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.450 INFO:teuthology.orchestra.run.smithi164.stdout: "agent_down_multiplier": { 2024-09-19T08:57:49.450 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "agent_down_multiplier", 2024-09-19T08:57:49.450 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "float", 2024-09-19T08:57:49.450 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.450 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.450 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "3.0", 2024-09-19T08:57:49.450 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.450 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.450 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.451 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T08:57:49.451 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.451 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.451 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.451 INFO:teuthology.orchestra.run.smithi164.stdout: "agent_refresh_rate": { 2024-09-19T08:57:49.451 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "agent_refresh_rate", 2024-09-19T08:57:49.451 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:49.451 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.451 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.451 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "20", 2024-09-19T08:57:49.451 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.451 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.452 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.452 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T08:57:49.452 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.452 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.452 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.452 INFO:teuthology.orchestra.run.smithi164.stdout: "agent_starting_port": { 2024-09-19T08:57:49.452 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "agent_starting_port", 2024-09-19T08:57:49.452 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.452 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.452 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.452 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "4721", 2024-09-19T08:57:49.452 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.453 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.453 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.453 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T08:57:49.453 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.453 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.453 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.453 INFO:teuthology.orchestra.run.smithi164.stdout: "allow_ptrace": { 2024-09-19T08:57:49.453 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "allow_ptrace", 2024-09-19T08:57:49.453 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.453 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.454 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.454 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.454 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.454 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.454 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.454 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T08:57:49.454 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T08:57:49.454 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.454 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.454 INFO:teuthology.orchestra.run.smithi164.stdout: "autotune_interval": { 2024-09-19T08:57:49.454 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "autotune_interval", 2024-09-19T08:57:49.454 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:49.454 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.455 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.455 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "600", 2024-09-19T08:57:49.455 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.455 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.455 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.455 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T08:57:49.455 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.455 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.455 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.455 INFO:teuthology.orchestra.run.smithi164.stdout: "autotune_memory_target_ratio": { 2024-09-19T08:57:49.455 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T08:57:49.455 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "float", 2024-09-19T08:57:49.456 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.456 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.456 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "0.7", 2024-09-19T08:57:49.456 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.456 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.456 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.456 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T08:57:49.456 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.456 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.456 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.456 INFO:teuthology.orchestra.run.smithi164.stdout: "cephadm_log_destination": { 2024-09-19T08:57:49.457 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "cephadm_log_destination", 2024-09-19T08:57:49.457 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.457 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.457 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.457 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.457 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.457 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.457 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.457 INFO:teuthology.orchestra.run.smithi164.stdout: "file", 2024-09-19T08:57:49.457 INFO:teuthology.orchestra.run.smithi164.stdout: "file,syslog", 2024-09-19T08:57:49.457 INFO:teuthology.orchestra.run.smithi164.stdout: "syslog" 2024-09-19T08:57:49.457 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.457 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T08:57:49.457 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.458 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.458 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.458 INFO:teuthology.orchestra.run.smithi164.stdout: "cgroups_split": { 2024-09-19T08:57:49.458 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "cgroups_split", 2024-09-19T08:57:49.458 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.458 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.458 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.458 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.458 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.458 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.458 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.458 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T08:57:49.459 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.459 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.459 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.459 INFO:teuthology.orchestra.run.smithi164.stdout: "config_checks_enabled": { 2024-09-19T08:57:49.459 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "config_checks_enabled", 2024-09-19T08:57:49.459 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.459 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.459 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.459 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.459 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.459 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.459 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.460 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T08:57:49.460 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.460 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.460 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.460 INFO:teuthology.orchestra.run.smithi164.stdout: "config_dashboard": { 2024-09-19T08:57:49.460 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "config_dashboard", 2024-09-19T08:57:49.460 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.460 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.460 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.460 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.460 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.460 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.460 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.461 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T08:57:49.461 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.461 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.461 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.461 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_alertmanager": { 2024-09-19T08:57:49.461 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_alertmanager", 2024-09-19T08:57:49.461 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.461 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.461 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.461 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T08:57:49.461 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.461 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.462 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.462 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Prometheus container image", 2024-09-19T08:57:49.462 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.462 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.462 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.462 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_base": { 2024-09-19T08:57:49.462 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_base", 2024-09-19T08:57:49.462 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.462 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.462 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.462 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T08:57:49.462 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.463 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.463 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.463 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Container image name, without the tag", 2024-09-19T08:57:49.463 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.463 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.463 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.463 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_elasticsearch": { 2024-09-19T08:57:49.463 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_elasticsearch", 2024-09-19T08:57:49.463 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.463 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.463 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.464 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T08:57:49.464 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.464 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.464 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.464 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "elasticsearch container image", 2024-09-19T08:57:49.464 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.464 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.464 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.464 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_grafana": { 2024-09-19T08:57:49.464 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_grafana", 2024-09-19T08:57:49.464 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.465 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.465 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.465 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T08:57:49.465 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.465 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.465 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.465 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Prometheus container image", 2024-09-19T08:57:49.465 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.465 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.465 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.465 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_haproxy": { 2024-09-19T08:57:49.465 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_haproxy", 2024-09-19T08:57:49.465 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.466 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.466 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.466 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T08:57:49.466 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.466 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.466 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.466 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "HAproxy container image", 2024-09-19T08:57:49.466 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.466 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.466 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.466 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_jaeger_agent": { 2024-09-19T08:57:49.467 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_jaeger_agent", 2024-09-19T08:57:49.467 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.467 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.467 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.467 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T08:57:49.467 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.467 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.467 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.467 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Jaeger agent container image", 2024-09-19T08:57:49.468 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.468 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.468 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.468 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_jaeger_collector": { 2024-09-19T08:57:49.468 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_jaeger_collector", 2024-09-19T08:57:49.468 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.468 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.468 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.468 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T08:57:49.468 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.468 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.469 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.469 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Jaeger collector container image", 2024-09-19T08:57:49.469 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.469 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.469 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.469 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_jaeger_query": { 2024-09-19T08:57:49.469 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_jaeger_query", 2024-09-19T08:57:49.469 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.469 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.469 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.469 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T08:57:49.469 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.470 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.470 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.470 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Jaeger query container image", 2024-09-19T08:57:49.470 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.470 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.470 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.470 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_keepalived": { 2024-09-19T08:57:49.470 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_keepalived", 2024-09-19T08:57:49.470 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.470 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.470 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.470 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T08:57:49.471 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.471 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.471 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.471 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Keepalived container image", 2024-09-19T08:57:49.471 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.471 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.471 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.471 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_loki": { 2024-09-19T08:57:49.471 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_loki", 2024-09-19T08:57:49.471 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.471 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.472 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.472 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T08:57:49.472 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.472 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.472 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.472 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Loki container image", 2024-09-19T08:57:49.472 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.472 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.472 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.472 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_node_exporter": { 2024-09-19T08:57:49.472 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_node_exporter", 2024-09-19T08:57:49.472 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.473 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.473 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.473 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T08:57:49.473 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.473 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.473 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.473 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Prometheus container image", 2024-09-19T08:57:49.473 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.473 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.473 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.473 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_nvmeof": { 2024-09-19T08:57:49.473 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_nvmeof", 2024-09-19T08:57:49.473 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.474 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.474 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.474 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T08:57:49.474 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.474 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.474 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.474 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Nvme-of container image", 2024-09-19T08:57:49.474 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.474 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.474 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.474 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_prometheus": { 2024-09-19T08:57:49.474 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_prometheus", 2024-09-19T08:57:49.475 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.475 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.475 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.475 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T08:57:49.475 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.475 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.475 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.475 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Prometheus container image", 2024-09-19T08:57:49.475 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.475 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.475 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.475 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_promtail": { 2024-09-19T08:57:49.476 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_promtail", 2024-09-19T08:57:49.476 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.476 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.476 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.476 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T08:57:49.476 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.476 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.476 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.476 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Promtail container image", 2024-09-19T08:57:49.476 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.476 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.476 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.476 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_samba": { 2024-09-19T08:57:49.477 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_samba", 2024-09-19T08:57:49.477 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.477 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.477 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.477 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T08:57:49.477 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.477 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.477 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.477 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Samba/SMB container image", 2024-09-19T08:57:49.477 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.477 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.477 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.478 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_snmp_gateway": { 2024-09-19T08:57:49.478 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_snmp_gateway", 2024-09-19T08:57:49.478 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.478 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.478 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.478 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T08:57:49.478 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.478 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.478 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.478 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "SNMP Gateway container image", 2024-09-19T08:57:49.478 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.478 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.478 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.479 INFO:teuthology.orchestra.run.smithi164.stdout: "container_init": { 2024-09-19T08:57:49.479 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_init", 2024-09-19T08:57:49.479 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.479 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.479 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.479 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.479 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.479 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.479 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.479 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T08:57:49.479 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.480 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.480 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.480 INFO:teuthology.orchestra.run.smithi164.stdout: "daemon_cache_timeout": { 2024-09-19T08:57:49.480 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "daemon_cache_timeout", 2024-09-19T08:57:49.480 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:49.480 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.480 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.480 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "600", 2024-09-19T08:57:49.480 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.480 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.480 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.481 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T08:57:49.481 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.481 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.481 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.481 INFO:teuthology.orchestra.run.smithi164.stdout: "default_cephadm_command_timeout": { 2024-09-19T08:57:49.481 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T08:57:49.481 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.481 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.481 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.481 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "900", 2024-09-19T08:57:49.481 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.481 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.482 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.482 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T08:57:49.482 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.482 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.482 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.482 INFO:teuthology.orchestra.run.smithi164.stdout: "default_registry": { 2024-09-19T08:57:49.482 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "default_registry", 2024-09-19T08:57:49.482 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.482 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.482 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.482 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "docker.io", 2024-09-19T08:57:49.482 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.483 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.483 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.483 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T08:57:49.483 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.483 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.483 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.483 INFO:teuthology.orchestra.run.smithi164.stdout: "device_cache_timeout": { 2024-09-19T08:57:49.483 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "device_cache_timeout", 2024-09-19T08:57:49.483 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:49.483 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.483 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.483 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "1800", 2024-09-19T08:57:49.484 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.484 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.484 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.484 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "seconds to cache device inventory", 2024-09-19T08:57:49.484 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.484 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.484 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.484 INFO:teuthology.orchestra.run.smithi164.stdout: "device_enhanced_scan": { 2024-09-19T08:57:49.484 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "device_enhanced_scan", 2024-09-19T08:57:49.484 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.484 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.484 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.485 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.485 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.485 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.485 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.485 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T08:57:49.485 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.485 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.485 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.485 INFO:teuthology.orchestra.run.smithi164.stdout: "facts_cache_timeout": { 2024-09-19T08:57:49.485 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "facts_cache_timeout", 2024-09-19T08:57:49.485 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:49.485 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.486 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.486 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "60", 2024-09-19T08:57:49.486 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.486 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.486 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.486 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "seconds to cache host facts data", 2024-09-19T08:57:49.486 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.486 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.486 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.486 INFO:teuthology.orchestra.run.smithi164.stdout: "grafana_dashboards_path": { 2024-09-19T08:57:49.486 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "grafana_dashboards_path", 2024-09-19T08:57:49.486 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.487 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.487 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.487 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T08:57:49.487 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.487 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.487 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.487 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T08:57:49.487 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.487 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.487 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.487 INFO:teuthology.orchestra.run.smithi164.stdout: "host_check_interval": { 2024-09-19T08:57:49.487 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "host_check_interval", 2024-09-19T08:57:49.488 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:49.488 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.488 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.488 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "600", 2024-09-19T08:57:49.488 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.488 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.488 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.488 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "how frequently to perform a host check", 2024-09-19T08:57:49.488 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.488 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.488 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.488 INFO:teuthology.orchestra.run.smithi164.stdout: "hw_monitoring": { 2024-09-19T08:57:49.489 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "hw_monitoring", 2024-09-19T08:57:49.489 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.489 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.489 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.489 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.489 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.489 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.489 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.489 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T08:57:49.489 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.489 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.489 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.490 INFO:teuthology.orchestra.run.smithi164.stdout: "inventory_list_all": { 2024-09-19T08:57:49.490 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "inventory_list_all", 2024-09-19T08:57:49.490 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.490 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.490 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.490 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.490 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.490 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.490 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.490 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T08:57:49.493 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.493 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.493 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.493 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.493 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.493 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.493 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.493 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.493 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.494 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.494 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.494 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.494 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.494 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.494 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.494 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.494 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.494 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.494 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.494 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.494 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.494 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.495 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.495 INFO:teuthology.orchestra.run.smithi164.stdout: "log_refresh_metadata": { 2024-09-19T08:57:49.495 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_refresh_metadata", 2024-09-19T08:57:49.495 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.495 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.495 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.495 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.495 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.495 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.495 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.495 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T08:57:49.496 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.496 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.496 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.496 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.496 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.496 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.496 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.496 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.496 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.496 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.496 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.496 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.497 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T08:57:49.497 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.497 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.497 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.497 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.497 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.497 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.497 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.497 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.497 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.497 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.498 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.498 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.498 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.498 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.498 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.498 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.498 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.498 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.498 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.498 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.498 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.498 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.498 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.499 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.499 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.499 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.499 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.499 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.499 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.499 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.499 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.499 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.499 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.499 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.499 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.499 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.500 INFO:teuthology.orchestra.run.smithi164.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T08:57:49.500 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T08:57:49.500 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.500 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.500 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.500 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.500 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.500 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.500 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.500 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T08:57:49.500 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.501 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.501 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.501 INFO:teuthology.orchestra.run.smithi164.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T08:57:49.501 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T08:57:49.501 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.501 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.501 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.501 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "*", 2024-09-19T08:57:49.501 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.501 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.501 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.501 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T08:57:49.501 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.502 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.502 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.502 INFO:teuthology.orchestra.run.smithi164.stdout: "max_count_per_host": { 2024-09-19T08:57:49.502 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "max_count_per_host", 2024-09-19T08:57:49.502 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.502 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.502 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.502 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "10", 2024-09-19T08:57:49.502 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.502 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.502 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.502 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "max number of daemons per service per host", 2024-09-19T08:57:49.503 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.503 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.503 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.503 INFO:teuthology.orchestra.run.smithi164.stdout: "max_osd_draining_count": { 2024-09-19T08:57:49.503 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "max_osd_draining_count", 2024-09-19T08:57:49.503 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.503 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.503 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.503 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "10", 2024-09-19T08:57:49.503 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.503 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.503 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.504 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T08:57:49.504 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.504 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.504 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.504 INFO:teuthology.orchestra.run.smithi164.stdout: "migration_current": { 2024-09-19T08:57:49.504 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "migration_current", 2024-09-19T08:57:49.504 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.504 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.504 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.504 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.504 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.504 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.505 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.505 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "internal - do not modify", 2024-09-19T08:57:49.505 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.505 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.505 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.505 INFO:teuthology.orchestra.run.smithi164.stdout: "mode": { 2024-09-19T08:57:49.505 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "mode", 2024-09-19T08:57:49.505 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.505 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.505 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.505 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "root", 2024-09-19T08:57:49.505 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.506 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.506 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.506 INFO:teuthology.orchestra.run.smithi164.stdout: "cephadm-package", 2024-09-19T08:57:49.506 INFO:teuthology.orchestra.run.smithi164.stdout: "root" 2024-09-19T08:57:49.506 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.506 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T08:57:49.506 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.506 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.506 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.506 INFO:teuthology.orchestra.run.smithi164.stdout: "oob_default_addr": { 2024-09-19T08:57:49.506 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "oob_default_addr", 2024-09-19T08:57:49.506 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.507 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.507 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.507 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "169.254.1.1", 2024-09-19T08:57:49.507 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.507 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.507 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.507 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T08:57:49.507 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.507 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.507 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.507 INFO:teuthology.orchestra.run.smithi164.stdout: "prometheus_alerts_path": { 2024-09-19T08:57:49.507 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "prometheus_alerts_path", 2024-09-19T08:57:49.507 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.508 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.508 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.508 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T08:57:49.508 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.508 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.508 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.508 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T08:57:49.508 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.508 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.508 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.508 INFO:teuthology.orchestra.run.smithi164.stdout: "registry_insecure": { 2024-09-19T08:57:49.509 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "registry_insecure", 2024-09-19T08:57:49.509 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.509 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.509 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.509 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.509 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.509 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.509 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.509 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T08:57:49.509 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.509 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.509 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.510 INFO:teuthology.orchestra.run.smithi164.stdout: "registry_password": { 2024-09-19T08:57:49.510 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "registry_password", 2024-09-19T08:57:49.510 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.510 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.510 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.510 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.510 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.510 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.510 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.510 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T08:57:49.510 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.510 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.511 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.511 INFO:teuthology.orchestra.run.smithi164.stdout: "registry_url": { 2024-09-19T08:57:49.511 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "registry_url", 2024-09-19T08:57:49.511 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.511 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.511 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.511 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.511 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.511 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.511 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.511 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T08:57:49.511 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.512 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.512 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.512 INFO:teuthology.orchestra.run.smithi164.stdout: "registry_username": { 2024-09-19T08:57:49.512 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "registry_username", 2024-09-19T08:57:49.512 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.512 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.512 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.512 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.512 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.512 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.512 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.513 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T08:57:49.513 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.513 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.513 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.513 INFO:teuthology.orchestra.run.smithi164.stdout: "secure_monitoring_stack": { 2024-09-19T08:57:49.513 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "secure_monitoring_stack", 2024-09-19T08:57:49.513 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.513 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.513 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.513 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.513 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.513 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.514 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.514 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T08:57:49.514 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.514 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.514 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.514 INFO:teuthology.orchestra.run.smithi164.stdout: "service_discovery_port": { 2024-09-19T08:57:49.514 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "service_discovery_port", 2024-09-19T08:57:49.514 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.514 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.514 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.514 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "8765", 2024-09-19T08:57:49.514 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.515 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.515 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.515 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "cephadm service discovery port", 2024-09-19T08:57:49.515 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.515 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.515 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.515 INFO:teuthology.orchestra.run.smithi164.stdout: "ssh_config_file": { 2024-09-19T08:57:49.515 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "ssh_config_file", 2024-09-19T08:57:49.515 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.515 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.515 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.515 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.515 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.516 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.516 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.516 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T08:57:49.516 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.516 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.516 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.516 INFO:teuthology.orchestra.run.smithi164.stdout: "use_agent": { 2024-09-19T08:57:49.516 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "use_agent", 2024-09-19T08:57:49.516 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.516 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.516 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.516 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.517 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.517 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.517 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.517 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T08:57:49.517 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.517 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.517 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.517 INFO:teuthology.orchestra.run.smithi164.stdout: "use_repo_digest": { 2024-09-19T08:57:49.517 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "use_repo_digest", 2024-09-19T08:57:49.517 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.517 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.517 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.518 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.518 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.518 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.518 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.518 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T08:57:49.518 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.518 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.518 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.518 INFO:teuthology.orchestra.run.smithi164.stdout: "warn_on_failed_host_check": { 2024-09-19T08:57:49.518 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "warn_on_failed_host_check", 2024-09-19T08:57:49.518 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.518 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.519 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.519 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.519 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.519 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.519 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.519 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T08:57:49.519 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.519 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.519 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.519 INFO:teuthology.orchestra.run.smithi164.stdout: "warn_on_stray_daemons": { 2024-09-19T08:57:49.519 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "warn_on_stray_daemons", 2024-09-19T08:57:49.519 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.519 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.520 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.520 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.520 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.520 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.520 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.520 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T08:57:49.520 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.520 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.520 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.520 INFO:teuthology.orchestra.run.smithi164.stdout: "warn_on_stray_hosts": { 2024-09-19T08:57:49.520 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "warn_on_stray_hosts", 2024-09-19T08:57:49.520 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T08:57:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.522 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "crash", 2024-09-19T08:57:49.522 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.522 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.522 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.522 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.522 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.522 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.522 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.522 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.522 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.522 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.523 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.523 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.523 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.523 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.523 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.523 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.523 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.523 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.523 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.523 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.523 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.523 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.523 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.524 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.524 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.524 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.524 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.524 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.524 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.524 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.524 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.524 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.524 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.524 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.524 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.525 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.525 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.525 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.525 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.525 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.525 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.525 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.525 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.525 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.525 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.525 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.525 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.526 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.526 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.526 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.526 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.526 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.526 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.526 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.526 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.526 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.526 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.526 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.526 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.527 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.527 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.527 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.527 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.527 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.527 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.527 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.527 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.527 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.527 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.527 INFO:teuthology.orchestra.run.smithi164.stdout: "retain_interval": { 2024-09-19T08:57:49.527 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "retain_interval", 2024-09-19T08:57:49.528 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:49.528 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.528 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.528 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "31536000", 2024-09-19T08:57:49.528 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.528 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.528 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.528 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T08:57:49.528 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.528 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.528 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.528 INFO:teuthology.orchestra.run.smithi164.stdout: "warn_recent_interval": { 2024-09-19T08:57:49.529 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "warn_recent_interval", 2024-09-19T08:57:49.529 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:49.529 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.529 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.529 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "1209600", 2024-09-19T08:57:49.529 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.529 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.529 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.529 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T08:57:49.529 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.529 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.529 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.529 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.529 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.530 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "dashboard", 2024-09-19T08:57:49.530 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.530 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.530 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.530 INFO:teuthology.orchestra.run.smithi164.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T08:57:49.530 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T08:57:49.530 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.530 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.530 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.530 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "10", 2024-09-19T08:57:49.530 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.531 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.531 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.531 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.531 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.531 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.531 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.531 INFO:teuthology.orchestra.run.smithi164.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T08:57:49.531 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T08:57:49.531 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.531 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.531 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.531 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.532 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.532 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.532 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.532 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.532 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.532 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.532 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.532 INFO:teuthology.orchestra.run.smithi164.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T08:57:49.532 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T08:57:49.532 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.532 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.533 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.533 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.533 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.533 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.533 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.533 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.533 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.533 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.533 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.533 INFO:teuthology.orchestra.run.smithi164.stdout: "AUDIT_API_ENABLED": { 2024-09-19T08:57:49.533 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T08:57:49.533 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.534 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.534 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.534 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.534 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.534 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.534 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.534 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.534 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.534 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.534 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.534 INFO:teuthology.orchestra.run.smithi164.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T08:57:49.534 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T08:57:49.534 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.535 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.535 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.535 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.535 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.535 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.535 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.535 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.535 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.535 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.535 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.535 INFO:teuthology.orchestra.run.smithi164.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T08:57:49.535 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T08:57:49.536 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.536 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.536 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.536 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.536 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.536 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.536 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.536 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.536 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.536 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.536 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.537 INFO:teuthology.orchestra.run.smithi164.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T08:57:49.537 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T08:57:49.537 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.537 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.537 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.537 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.537 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.537 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.537 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.537 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.537 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.538 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.538 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.538 INFO:teuthology.orchestra.run.smithi164.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T08:57:49.538 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T08:57:49.538 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.538 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.538 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.538 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.538 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.538 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.538 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.538 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.539 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.539 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.539 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.539 INFO:teuthology.orchestra.run.smithi164.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T08:57:49.539 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T08:57:49.539 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.539 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.539 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.539 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.539 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.539 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.539 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.540 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.540 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.540 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.540 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.540 INFO:teuthology.orchestra.run.smithi164.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T08:57:49.540 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T08:57:49.540 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.540 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.540 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.540 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.540 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.541 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.541 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.541 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.541 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.541 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.541 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.541 INFO:teuthology.orchestra.run.smithi164.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T08:57:49.541 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T08:57:49.541 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.541 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.541 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.541 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.542 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.542 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.542 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.542 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.542 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.542 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.542 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.542 INFO:teuthology.orchestra.run.smithi164.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T08:57:49.542 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T08:57:49.542 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.542 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.542 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.542 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.543 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.543 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.543 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.543 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.543 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.543 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.543 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.543 INFO:teuthology.orchestra.run.smithi164.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T08:57:49.543 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T08:57:49.543 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.543 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.543 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.544 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.544 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.544 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.544 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.544 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.544 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.544 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.544 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.544 INFO:teuthology.orchestra.run.smithi164.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T08:57:49.544 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T08:57:49.544 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.544 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.545 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.545 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.545 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.545 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.545 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.545 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.545 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.545 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.545 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.545 INFO:teuthology.orchestra.run.smithi164.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T08:57:49.545 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T08:57:49.545 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.546 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.546 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.546 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "admin", 2024-09-19T08:57:49.546 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.546 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.546 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.546 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.546 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.546 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.546 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.546 INFO:teuthology.orchestra.run.smithi164.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T08:57:49.546 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T08:57:49.546 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.547 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.547 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.547 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.547 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.547 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.547 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.547 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.547 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.547 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.547 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.547 INFO:teuthology.orchestra.run.smithi164.stdout: "GRAFANA_API_URL": { 2024-09-19T08:57:49.547 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "GRAFANA_API_URL", 2024-09-19T08:57:49.548 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.548 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.548 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.548 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.548 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.548 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.548 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.548 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.548 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.548 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.548 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.548 INFO:teuthology.orchestra.run.smithi164.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T08:57:49.549 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T08:57:49.549 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.549 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.549 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.549 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "admin", 2024-09-19T08:57:49.549 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.549 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.549 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.549 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.549 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.549 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.549 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.550 INFO:teuthology.orchestra.run.smithi164.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T08:57:49.550 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T08:57:49.550 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.550 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.550 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.550 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.550 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.550 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.550 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.550 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.550 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.550 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.550 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.551 INFO:teuthology.orchestra.run.smithi164.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T08:57:49.551 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T08:57:49.551 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.551 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.551 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.551 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.551 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.551 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.551 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.551 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.551 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.551 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.552 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.552 INFO:teuthology.orchestra.run.smithi164.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T08:57:49.552 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T08:57:49.552 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.552 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.552 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.552 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.552 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.552 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.552 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.552 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.553 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.553 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.553 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.553 INFO:teuthology.orchestra.run.smithi164.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T08:57:49.553 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T08:57:49.553 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.553 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.553 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.553 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.553 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.553 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.553 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.554 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.554 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.554 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.554 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.554 INFO:teuthology.orchestra.run.smithi164.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T08:57:49.554 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T08:57:49.554 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.554 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.554 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.554 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.554 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.554 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.554 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.555 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.555 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.555 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.555 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.555 INFO:teuthology.orchestra.run.smithi164.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T08:57:49.555 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T08:57:49.555 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.555 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.555 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.555 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.555 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.555 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.556 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.556 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.556 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.556 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.556 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.556 INFO:teuthology.orchestra.run.smithi164.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T08:57:49.556 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T08:57:49.556 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.556 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.556 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.556 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.556 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.557 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.557 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.557 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.557 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.557 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.557 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.557 INFO:teuthology.orchestra.run.smithi164.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T08:57:49.557 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T08:57:49.557 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.557 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.557 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.557 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.558 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.558 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.558 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.558 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.558 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.558 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.558 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.558 INFO:teuthology.orchestra.run.smithi164.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T08:57:49.558 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T08:57:49.558 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.558 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.558 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.558 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.559 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.559 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.559 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.559 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.559 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.559 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.559 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.559 INFO:teuthology.orchestra.run.smithi164.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T08:57:49.559 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T08:57:49.559 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.559 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.559 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.560 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.560 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.560 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.560 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.560 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.560 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.560 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.560 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.560 INFO:teuthology.orchestra.run.smithi164.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T08:57:49.560 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T08:57:49.560 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.560 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.561 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.561 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.561 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.561 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.561 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.561 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.561 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.561 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.561 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.561 INFO:teuthology.orchestra.run.smithi164.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T08:57:49.561 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T08:57:49.561 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.561 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.562 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.562 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.562 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.562 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.562 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.562 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.562 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.562 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.562 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.562 INFO:teuthology.orchestra.run.smithi164.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T08:57:49.562 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T08:57:49.562 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.563 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.563 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.563 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.563 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.563 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.563 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.563 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.563 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.563 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.563 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.563 INFO:teuthology.orchestra.run.smithi164.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T08:57:49.563 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T08:57:49.564 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.564 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.564 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.564 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.564 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.564 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.564 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.564 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.564 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.564 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.564 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.565 INFO:teuthology.orchestra.run.smithi164.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T08:57:49.565 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T08:57:49.565 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.565 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.565 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.565 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T08:57:49.565 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.565 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.565 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.565 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.565 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.565 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.565 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.566 INFO:teuthology.orchestra.run.smithi164.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T08:57:49.566 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T08:57:49.566 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.566 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.566 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.566 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "10", 2024-09-19T08:57:49.566 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.566 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.566 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.566 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.566 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.566 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.567 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.567 INFO:teuthology.orchestra.run.smithi164.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T08:57:49.567 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T08:57:49.567 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.567 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.567 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.567 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "8", 2024-09-19T08:57:49.567 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.567 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.567 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.567 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.567 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.568 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.568 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.568 INFO:teuthology.orchestra.run.smithi164.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T08:57:49.568 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T08:57:49.568 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.568 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.568 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.568 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "45", 2024-09-19T08:57:49.568 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.568 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.568 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.568 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.569 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.569 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.569 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.569 INFO:teuthology.orchestra.run.smithi164.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T08:57:49.569 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T08:57:49.569 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.569 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.569 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.569 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.569 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.569 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.569 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.570 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.570 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.570 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.570 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.570 INFO:teuthology.orchestra.run.smithi164.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T08:57:49.570 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T08:57:49.570 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.570 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.570 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.570 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "admin", 2024-09-19T08:57:49.570 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.570 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.571 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.571 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.571 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.571 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.571 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.571 INFO:teuthology.orchestra.run.smithi164.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T08:57:49.571 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T08:57:49.571 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.571 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.571 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.571 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.571 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.572 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.572 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.572 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.572 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.572 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.572 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.572 INFO:teuthology.orchestra.run.smithi164.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T08:57:49.572 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T08:57:49.572 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.572 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.572 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.572 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.573 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.573 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.573 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.573 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.573 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.573 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.573 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.573 INFO:teuthology.orchestra.run.smithi164.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T08:57:49.573 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T08:57:49.573 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.573 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.573 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.574 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "0", 2024-09-19T08:57:49.574 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.574 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.574 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.574 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.574 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.574 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.574 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.574 INFO:teuthology.orchestra.run.smithi164.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T08:57:49.574 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T08:57:49.574 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.574 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.575 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.575 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "10", 2024-09-19T08:57:49.575 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.575 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.575 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.575 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.575 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.575 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.575 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.575 INFO:teuthology.orchestra.run.smithi164.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T08:57:49.575 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T08:57:49.575 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.576 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.576 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.576 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "5", 2024-09-19T08:57:49.576 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.576 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.576 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.576 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.576 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.576 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.576 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.576 INFO:teuthology.orchestra.run.smithi164.stdout: "cross_origin_url": { 2024-09-19T08:57:49.576 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "cross_origin_url", 2024-09-19T08:57:49.577 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.577 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.577 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.577 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.577 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.577 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.577 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.577 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.577 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.577 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.577 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.577 INFO:teuthology.orchestra.run.smithi164.stdout: "crt_file": { 2024-09-19T08:57:49.578 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "crt_file", 2024-09-19T08:57:49.578 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.578 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.578 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.578 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.578 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.578 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.578 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.578 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.578 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.578 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.578 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.578 INFO:teuthology.orchestra.run.smithi164.stdout: "debug": { 2024-09-19T08:57:49.579 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "debug", 2024-09-19T08:57:49.579 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.579 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.579 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.579 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.579 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.579 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.579 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.579 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Enable/disable debug options", 2024-09-19T08:57:49.579 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.579 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.579 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.580 INFO:teuthology.orchestra.run.smithi164.stdout: "jwt_token_ttl": { 2024-09-19T08:57:49.580 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "jwt_token_ttl", 2024-09-19T08:57:49.580 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.580 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.580 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.580 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "28800", 2024-09-19T08:57:49.580 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.580 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.580 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.580 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.580 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.580 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.581 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.581 INFO:teuthology.orchestra.run.smithi164.stdout: "key_file": { 2024-09-19T08:57:49.581 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "key_file", 2024-09-19T08:57:49.581 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.581 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.581 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.581 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.581 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.581 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.581 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.581 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.581 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.582 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.582 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.582 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.582 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.582 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.582 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.582 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.582 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.582 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.582 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.582 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.582 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.583 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.583 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.583 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.583 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.583 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.583 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.583 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.583 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.583 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.583 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.583 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.583 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.584 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.584 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.584 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.584 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.584 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.584 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.584 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.584 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.584 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.584 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.584 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.584 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.585 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.585 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.585 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.585 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.585 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.585 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.585 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.585 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.585 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.585 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.585 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.585 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.585 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.585 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.586 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.586 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.586 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.586 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.586 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.586 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.586 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.586 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.586 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.586 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.586 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.586 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.587 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.587 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.587 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.587 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.587 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.587 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.587 INFO:teuthology.orchestra.run.smithi164.stdout: "motd": { 2024-09-19T08:57:49.587 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "motd", 2024-09-19T08:57:49.587 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.587 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.587 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.587 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.588 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.588 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.588 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.588 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "The message of the day", 2024-09-19T08:57:49.588 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.588 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.588 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.588 INFO:teuthology.orchestra.run.smithi164.stdout: "redirect_resolve_ip_addr": { 2024-09-19T08:57:49.588 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T08:57:49.588 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.588 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.588 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.588 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.589 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.589 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.589 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.589 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.589 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.589 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.589 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.589 INFO:teuthology.orchestra.run.smithi164.stdout: "server_addr": { 2024-09-19T08:57:49.589 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "server_addr", 2024-09-19T08:57:49.589 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.589 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.589 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.590 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "::", 2024-09-19T08:57:49.590 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.590 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.590 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.590 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.590 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.590 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.590 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.590 INFO:teuthology.orchestra.run.smithi164.stdout: "server_port": { 2024-09-19T08:57:49.590 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "server_port", 2024-09-19T08:57:49.590 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.590 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.591 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.591 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "8080", 2024-09-19T08:57:49.591 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.591 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.591 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.591 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.591 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.591 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.591 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.591 INFO:teuthology.orchestra.run.smithi164.stdout: "ssl": { 2024-09-19T08:57:49.591 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "ssl", 2024-09-19T08:57:49.591 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.592 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.592 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.592 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.592 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.592 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.592 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.592 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.592 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.592 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.592 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.592 INFO:teuthology.orchestra.run.smithi164.stdout: "ssl_server_port": { 2024-09-19T08:57:49.592 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "ssl_server_port", 2024-09-19T08:57:49.593 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.593 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.593 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.593 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "8443", 2024-09-19T08:57:49.593 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.593 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.593 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.593 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.593 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.593 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.593 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.594 INFO:teuthology.orchestra.run.smithi164.stdout: "standby_behaviour": { 2024-09-19T08:57:49.594 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "standby_behaviour", 2024-09-19T08:57:49.594 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.594 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.594 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.594 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "redirect", 2024-09-19T08:57:49.594 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.594 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.594 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.594 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.594 INFO:teuthology.orchestra.run.smithi164.stdout: "redirect" 2024-09-19T08:57:49.594 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.594 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.595 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.595 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.595 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.595 INFO:teuthology.orchestra.run.smithi164.stdout: "standby_error_status_code": { 2024-09-19T08:57:49.595 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "standby_error_status_code", 2024-09-19T08:57:49.595 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.595 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.595 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.595 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "500", 2024-09-19T08:57:49.595 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "400", 2024-09-19T08:57:49.595 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "599", 2024-09-19T08:57:49.595 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.595 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.596 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.596 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.596 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.596 INFO:teuthology.orchestra.run.smithi164.stdout: "url_prefix": { 2024-09-19T08:57:49.596 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "url_prefix", 2024-09-19T08:57:49.596 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.596 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.596 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.596 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.596 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.596 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.596 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.597 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.597 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.597 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.597 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.597 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.597 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.597 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "devicehealth", 2024-09-19T08:57:49.597 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.597 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.597 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.597 INFO:teuthology.orchestra.run.smithi164.stdout: "enable_monitoring": { 2024-09-19T08:57:49.598 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "enable_monitoring", 2024-09-19T08:57:49.598 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.598 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.598 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.598 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.598 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.598 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.598 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.598 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "monitor device health metrics", 2024-09-19T08:57:49.598 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.598 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.598 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.599 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.599 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.599 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.599 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.599 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.599 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.599 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.599 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.599 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.599 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.599 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.599 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.599 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.600 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.600 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.600 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.600 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.600 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.600 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.600 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.600 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.600 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.600 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.600 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.600 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.600 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.601 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.601 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.601 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.601 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.601 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.601 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.601 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.601 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.601 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.601 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.601 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.601 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.602 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.602 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.602 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.602 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.602 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.602 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.602 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.602 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.602 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.602 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.602 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.602 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.602 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.603 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.603 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.603 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.603 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.603 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.603 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.603 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.603 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.603 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.603 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.603 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.603 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.603 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.604 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.604 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.604 INFO:teuthology.orchestra.run.smithi164.stdout: "mark_out_threshold": { 2024-09-19T08:57:49.604 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "mark_out_threshold", 2024-09-19T08:57:49.604 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:49.604 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.604 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.604 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "2419200", 2024-09-19T08:57:49.604 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.604 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.604 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.604 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T08:57:49.605 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.605 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.605 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.605 INFO:teuthology.orchestra.run.smithi164.stdout: "pool_name": { 2024-09-19T08:57:49.605 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "pool_name", 2024-09-19T08:57:49.605 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.605 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.605 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.605 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "device_health_metrics", 2024-09-19T08:57:49.605 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.605 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.605 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.606 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T08:57:49.606 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.606 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.606 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.606 INFO:teuthology.orchestra.run.smithi164.stdout: "retention_period": { 2024-09-19T08:57:49.606 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "retention_period", 2024-09-19T08:57:49.606 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:49.606 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.606 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.606 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "15552000", 2024-09-19T08:57:49.606 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.606 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.607 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.607 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "how long to retain device health metrics", 2024-09-19T08:57:49.607 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.607 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.607 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.607 INFO:teuthology.orchestra.run.smithi164.stdout: "scrape_frequency": { 2024-09-19T08:57:49.607 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "scrape_frequency", 2024-09-19T08:57:49.607 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:49.607 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.607 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.607 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "86400", 2024-09-19T08:57:49.607 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.608 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.608 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.608 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T08:57:49.608 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.608 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.608 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.608 INFO:teuthology.orchestra.run.smithi164.stdout: "self_heal": { 2024-09-19T08:57:49.608 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "self_heal", 2024-09-19T08:57:49.608 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.608 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.608 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.609 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.609 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.609 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.609 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.609 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T08:57:49.609 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.609 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.609 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.609 INFO:teuthology.orchestra.run.smithi164.stdout: "sleep_interval": { 2024-09-19T08:57:49.609 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "sleep_interval", 2024-09-19T08:57:49.609 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:49.609 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.610 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.610 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "600", 2024-09-19T08:57:49.610 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.610 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.610 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.610 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T08:57:49.610 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.610 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.610 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.610 INFO:teuthology.orchestra.run.smithi164.stdout: "warn_threshold": { 2024-09-19T08:57:49.610 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "warn_threshold", 2024-09-19T08:57:49.610 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:49.611 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.611 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.611 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "7257600", 2024-09-19T08:57:49.611 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.611 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.611 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.611 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T08:57:49.611 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.611 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.611 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.611 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.611 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.612 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "diskprediction_local", 2024-09-19T08:57:49.612 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.612 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.612 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.612 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.612 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.612 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.612 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.612 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.612 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.612 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.612 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.613 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.613 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.613 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.613 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.613 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.613 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.613 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.613 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.613 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.613 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.613 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.613 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.614 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.614 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.614 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.614 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.614 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.614 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.614 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.614 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.614 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.614 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.614 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.614 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.615 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.615 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.615 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.615 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.615 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.615 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.615 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.615 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.615 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.615 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.615 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.615 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.615 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.616 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.616 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.616 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.616 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.616 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.616 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.616 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.616 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.616 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.616 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.616 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.616 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.617 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.617 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.617 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.617 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.617 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.617 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.617 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.617 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.617 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.617 INFO:teuthology.orchestra.run.smithi164.stdout: "predict_interval": { 2024-09-19T08:57:49.617 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "predict_interval", 2024-09-19T08:57:49.617 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.618 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.618 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.618 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "86400", 2024-09-19T08:57:49.618 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.618 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.618 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.618 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.618 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.618 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.618 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.618 INFO:teuthology.orchestra.run.smithi164.stdout: "predictor_model": { 2024-09-19T08:57:49.618 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "predictor_model", 2024-09-19T08:57:49.618 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.619 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.619 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.619 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "prophetstor", 2024-09-19T08:57:49.619 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.619 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.619 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.619 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.619 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.619 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.619 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.619 INFO:teuthology.orchestra.run.smithi164.stdout: "sleep_interval": { 2024-09-19T08:57:49.619 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "sleep_interval", 2024-09-19T08:57:49.620 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.620 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.620 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.620 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "600", 2024-09-19T08:57:49.620 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.620 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.620 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.620 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.620 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.620 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.620 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.620 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.620 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.621 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "influx", 2024-09-19T08:57:49.621 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": false, 2024-09-19T08:57:49.621 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "influxdb python module not found", 2024-09-19T08:57:49.621 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.621 INFO:teuthology.orchestra.run.smithi164.stdout: "batch_size": { 2024-09-19T08:57:49.621 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "batch_size", 2024-09-19T08:57:49.621 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.621 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.621 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.621 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "5000", 2024-09-19T08:57:49.621 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.621 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.622 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.622 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T08:57:49.622 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.622 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.622 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.622 INFO:teuthology.orchestra.run.smithi164.stdout: "database": { 2024-09-19T08:57:49.622 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "database", 2024-09-19T08:57:49.622 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.622 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.622 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.622 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "ceph", 2024-09-19T08:57:49.623 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.623 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.623 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.623 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T08:57:49.623 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.623 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.623 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.623 INFO:teuthology.orchestra.run.smithi164.stdout: "hostname": { 2024-09-19T08:57:49.623 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "hostname", 2024-09-19T08:57:49.623 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.623 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.623 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.623 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.624 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.624 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.624 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.624 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "InfluxDB server hostname", 2024-09-19T08:57:49.624 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.624 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.624 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.624 INFO:teuthology.orchestra.run.smithi164.stdout: "interval": { 2024-09-19T08:57:49.624 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "interval", 2024-09-19T08:57:49.624 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:49.624 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.625 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.625 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "30", 2024-09-19T08:57:49.625 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "5", 2024-09-19T08:57:49.625 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.625 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.625 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T08:57:49.625 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.625 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.625 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.625 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.625 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.625 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.625 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.626 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.626 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.626 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.626 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.626 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.626 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.626 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.626 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.626 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.626 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.626 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.626 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.626 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.627 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.627 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.627 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.627 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.627 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.627 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.627 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.627 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.627 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.627 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.627 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.627 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.628 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.628 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.628 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.628 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.628 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.628 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.628 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.628 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.628 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.628 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.628 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.628 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.629 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.629 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.629 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.629 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.629 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.629 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.629 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.629 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.629 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.629 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.629 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.629 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.630 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.630 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.630 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.630 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.630 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.630 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.630 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.630 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.630 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.630 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.630 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.630 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.630 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.631 INFO:teuthology.orchestra.run.smithi164.stdout: "password": { 2024-09-19T08:57:49.631 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "password", 2024-09-19T08:57:49.631 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.631 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.631 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.631 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.631 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.631 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.631 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.631 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "password of InfluxDB server user", 2024-09-19T08:57:49.631 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.631 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.631 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.632 INFO:teuthology.orchestra.run.smithi164.stdout: "port": { 2024-09-19T08:57:49.632 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "port", 2024-09-19T08:57:49.632 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.632 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.632 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.632 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "8086", 2024-09-19T08:57:49.632 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.632 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.632 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.632 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "InfluxDB server port", 2024-09-19T08:57:49.632 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.632 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.632 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.633 INFO:teuthology.orchestra.run.smithi164.stdout: "ssl": { 2024-09-19T08:57:49.633 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "ssl", 2024-09-19T08:57:49.633 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.633 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.633 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.633 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "false", 2024-09-19T08:57:49.633 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.633 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.633 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.633 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T08:57:49.633 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.633 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.634 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.634 INFO:teuthology.orchestra.run.smithi164.stdout: "threads": { 2024-09-19T08:57:49.634 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "threads", 2024-09-19T08:57:49.634 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.634 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.634 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.634 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "5", 2024-09-19T08:57:49.634 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "1", 2024-09-19T08:57:49.634 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "32", 2024-09-19T08:57:49.634 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.634 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T08:57:49.634 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.635 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.635 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.635 INFO:teuthology.orchestra.run.smithi164.stdout: "username": { 2024-09-19T08:57:49.635 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "username", 2024-09-19T08:57:49.635 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.635 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.635 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.635 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.635 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.635 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.635 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.635 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "username of InfluxDB server user", 2024-09-19T08:57:49.636 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.636 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.636 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.636 INFO:teuthology.orchestra.run.smithi164.stdout: "verify_ssl": { 2024-09-19T08:57:49.636 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "verify_ssl", 2024-09-19T08:57:49.636 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.636 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.636 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.636 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "true", 2024-09-19T08:57:49.636 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.636 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.637 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.637 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T08:57:49.637 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.637 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.637 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.637 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.637 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.637 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "insights", 2024-09-19T08:57:49.637 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.637 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.638 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.638 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.638 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.638 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.638 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.638 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.638 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.638 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.638 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.638 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.638 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.638 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.638 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.639 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.639 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.639 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.639 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.639 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.639 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.639 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.639 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.639 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.639 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.639 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.639 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.640 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.640 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.640 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.640 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.640 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.640 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.640 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.640 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.640 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.640 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.640 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.641 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.641 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.641 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.641 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.641 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.641 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.641 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.641 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.641 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.641 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.641 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.641 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.641 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.642 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.642 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.642 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.642 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.642 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.642 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.642 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.642 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.642 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.642 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.642 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.642 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.643 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.643 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.643 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.643 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.643 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.643 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.643 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.643 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.643 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "iostat", 2024-09-19T08:57:49.643 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.643 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.644 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.644 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.644 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.644 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.644 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.644 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.644 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.644 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.644 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.644 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.644 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.644 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.644 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.645 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.645 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.645 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.645 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.645 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.645 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.645 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.645 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.645 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.645 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.645 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.645 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.646 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.646 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.646 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.646 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.646 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.646 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.646 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.646 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.646 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.646 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.646 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.646 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.646 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.647 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.647 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.647 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.647 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.647 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.647 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.647 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.647 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.647 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.647 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.647 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.647 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.647 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.648 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.648 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.648 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.648 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.648 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.648 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.648 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.648 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.648 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.648 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.648 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.648 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.649 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.649 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.649 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.649 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.649 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.649 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.649 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "k8sevents", 2024-09-19T08:57:49.649 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.649 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.649 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.649 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_event_retention_days": { 2024-09-19T08:57:49.650 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "ceph_event_retention_days", 2024-09-19T08:57:49.650 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.650 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.650 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.650 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "7", 2024-09-19T08:57:49.650 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.650 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.650 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.650 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T08:57:49.650 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.650 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.650 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.651 INFO:teuthology.orchestra.run.smithi164.stdout: "config_check_secs": { 2024-09-19T08:57:49.651 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "config_check_secs", 2024-09-19T08:57:49.651 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.651 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.651 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.651 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "10", 2024-09-19T08:57:49.651 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "10", 2024-09-19T08:57:49.651 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.651 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.651 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T08:57:49.651 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.651 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.651 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.652 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.652 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.652 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.652 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.652 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.652 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.652 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.652 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.652 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.652 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.652 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.653 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.653 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.653 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.653 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.653 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.653 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.653 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.653 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.653 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.653 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.653 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.653 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.654 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.654 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.654 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.654 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.654 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.654 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.654 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.654 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.654 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.654 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.654 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.654 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.655 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.655 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.655 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.655 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.655 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.655 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.655 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.655 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.655 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.655 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.655 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.655 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.655 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.656 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.656 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.656 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.656 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.656 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.656 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.656 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.656 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.656 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.656 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.656 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.656 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.657 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.657 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.657 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.657 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.657 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.657 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.657 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.657 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.657 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "localpool", 2024-09-19T08:57:49.657 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.657 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.658 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.658 INFO:teuthology.orchestra.run.smithi164.stdout: "failure_domain": { 2024-09-19T08:57:49.658 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "failure_domain", 2024-09-19T08:57:49.658 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.658 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.658 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.658 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "host", 2024-09-19T08:57:49.658 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.658 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.658 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.658 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "failure domain for any created local pool", 2024-09-19T08:57:49.658 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T08:57:49.658 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.658 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.659 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.659 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.659 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.659 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.659 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.659 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.659 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.659 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.659 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.659 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.659 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.659 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.660 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.660 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.660 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.660 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.660 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.660 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.660 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.660 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.660 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.660 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.660 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.660 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.661 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.661 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.661 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.661 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.661 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.661 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.661 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.661 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.661 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.661 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.661 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.661 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.662 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.662 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.662 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.662 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.662 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.662 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.662 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.662 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.662 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.662 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.662 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.662 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.662 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.662 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.663 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.663 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.663 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.663 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.663 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.663 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.663 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.663 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.663 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.663 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.663 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.663 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.664 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.664 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.664 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.664 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.664 INFO:teuthology.orchestra.run.smithi164.stdout: "min_size": { 2024-09-19T08:57:49.664 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "min_size", 2024-09-19T08:57:49.664 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.664 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.664 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.664 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.664 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.664 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.665 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.665 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "default min_size for any created local pool", 2024-09-19T08:57:49.665 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T08:57:49.665 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.665 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.665 INFO:teuthology.orchestra.run.smithi164.stdout: "num_rep": { 2024-09-19T08:57:49.665 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "num_rep", 2024-09-19T08:57:49.665 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.665 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.665 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.665 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "3", 2024-09-19T08:57:49.665 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.666 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.666 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.666 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "default replica count for any created local pool", 2024-09-19T08:57:49.666 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.666 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.666 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.666 INFO:teuthology.orchestra.run.smithi164.stdout: "pg_num": { 2024-09-19T08:57:49.666 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "pg_num", 2024-09-19T08:57:49.666 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.666 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.666 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.667 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "128", 2024-09-19T08:57:49.667 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.667 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.667 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.667 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T08:57:49.667 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.667 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.667 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.667 INFO:teuthology.orchestra.run.smithi164.stdout: "prefix": { 2024-09-19T08:57:49.667 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "prefix", 2024-09-19T08:57:49.667 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.667 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.668 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.668 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.668 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.668 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.668 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.668 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "name prefix for any created local pool", 2024-09-19T08:57:49.668 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.668 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.668 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.668 INFO:teuthology.orchestra.run.smithi164.stdout: "subtree": { 2024-09-19T08:57:49.668 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "subtree", 2024-09-19T08:57:49.668 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.669 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.669 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.669 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "rack", 2024-09-19T08:57:49.669 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.669 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.669 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.669 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T08:57:49.669 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T08:57:49.669 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.669 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.670 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.670 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.670 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "mds_autoscaler", 2024-09-19T08:57:49.670 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.670 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.670 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.670 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.670 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.671 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.671 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.671 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.671 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.671 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.671 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.671 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.671 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.671 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.671 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.672 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.672 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.672 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.672 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.672 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.672 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.672 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.672 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.672 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.672 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.673 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.673 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.673 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.673 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.673 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.673 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.673 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.673 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.673 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.673 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.674 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.674 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.674 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.674 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.674 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.674 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.674 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.674 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.674 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.674 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.674 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.674 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.675 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.675 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.675 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.675 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.675 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.675 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.675 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.675 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.675 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.675 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.676 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.676 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.676 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.676 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.676 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.676 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.676 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.676 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.676 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.676 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.676 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.677 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.677 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.677 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.677 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "mirroring", 2024-09-19T08:57:49.677 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.677 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.677 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.677 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.677 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.678 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.678 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.678 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.678 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.678 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.678 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.678 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.678 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.678 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.678 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.678 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.679 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.679 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.679 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.679 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.679 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.679 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.679 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.679 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.679 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.679 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.679 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.680 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.680 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.680 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.680 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.680 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.680 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.680 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.680 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.680 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.680 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.681 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.681 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.681 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.681 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.681 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.681 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.681 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.681 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.681 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.681 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.681 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.682 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.682 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.682 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.682 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.682 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.682 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.682 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.682 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.682 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.682 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.683 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.683 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.683 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.683 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.683 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.683 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.683 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.683 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.683 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.683 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.684 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.684 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.684 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.684 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "nfs", 2024-09-19T08:57:49.684 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.684 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.684 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.684 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.684 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.685 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.685 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.685 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.685 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.685 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.685 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.685 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.685 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.685 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.685 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.686 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.686 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.686 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.686 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.686 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.686 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.686 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.686 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.686 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.686 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.687 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.687 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.687 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.687 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.687 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.687 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.687 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.687 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.687 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.687 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.687 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.688 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.688 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.688 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.688 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.688 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.688 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.688 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.688 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.688 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.688 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.688 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.689 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.689 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.689 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.689 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.689 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.689 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.689 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.689 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.689 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.689 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.690 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.690 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.690 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.690 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.690 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.690 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.690 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.690 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.690 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.690 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.690 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.690 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.691 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.691 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.691 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "orchestrator", 2024-09-19T08:57:49.691 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.691 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.691 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.691 INFO:teuthology.orchestra.run.smithi164.stdout: "fail_fs": { 2024-09-19T08:57:49.691 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "fail_fs", 2024-09-19T08:57:49.691 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.692 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.692 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.692 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.692 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.692 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.692 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.692 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T08:57:49.692 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.692 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.692 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.692 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.693 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.693 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.693 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.693 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.693 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.693 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.693 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.693 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.693 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.693 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.693 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.694 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.694 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.694 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.694 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.694 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.694 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.694 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.694 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.694 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.694 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.694 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.695 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.695 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.695 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.695 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.695 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.695 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.695 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.695 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.695 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.695 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.696 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.696 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.696 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.696 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.696 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.696 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.696 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.696 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.696 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.696 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.696 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.697 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.697 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.697 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.697 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.697 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.697 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.697 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.697 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.697 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.697 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.697 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.698 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.698 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.698 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.698 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.698 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.698 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.698 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.698 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.698 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.698 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.699 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.699 INFO:teuthology.orchestra.run.smithi164.stdout: "orchestrator": { 2024-09-19T08:57:49.699 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "orchestrator", 2024-09-19T08:57:49.699 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.699 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.699 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.699 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.699 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.699 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.699 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.700 INFO:teuthology.orchestra.run.smithi164.stdout: "cephadm", 2024-09-19T08:57:49.700 INFO:teuthology.orchestra.run.smithi164.stdout: "rook", 2024-09-19T08:57:49.700 INFO:teuthology.orchestra.run.smithi164.stdout: "test_orchestrator" 2024-09-19T08:57:49.700 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.700 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Orchestrator backend", 2024-09-19T08:57:49.700 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.700 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.700 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.700 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.700 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.701 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "osd_perf_query", 2024-09-19T08:57:49.701 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.701 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.701 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.701 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.701 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.701 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.701 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.701 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.702 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.702 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.702 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.702 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.702 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.702 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.702 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.702 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.702 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.702 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.702 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.703 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.703 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.703 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.703 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.703 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.703 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.703 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.703 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.703 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.703 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.704 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.704 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.704 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.704 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.704 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.704 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.704 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.704 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.704 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.704 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.705 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.705 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.705 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.705 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.705 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.705 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.705 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.705 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.705 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.705 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.705 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.706 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.706 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.706 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.706 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.706 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.706 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.706 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.706 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.706 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.706 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.706 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.707 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.707 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.707 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.707 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.707 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.707 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.707 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.707 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.707 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.707 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.708 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "osd_support", 2024-09-19T08:57:49.708 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.708 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.708 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.708 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.708 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.708 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.708 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.708 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.709 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.709 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.709 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.709 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.709 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.709 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.709 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.709 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.709 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.709 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.709 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.710 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.710 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.710 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.710 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.710 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.710 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.710 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.710 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.710 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.710 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.710 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.711 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.711 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.711 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.711 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.711 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.711 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.711 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.711 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.711 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.712 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.712 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.712 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.712 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.712 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.712 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.712 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.712 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.712 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.712 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.712 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.712 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.713 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.713 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.713 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.713 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.713 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.713 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.713 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.713 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.713 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.713 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.714 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.714 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.714 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.714 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.714 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.714 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.714 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.714 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.714 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.714 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.715 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "pg_autoscaler", 2024-09-19T08:57:49.715 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.715 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.715 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.715 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.715 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.715 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.715 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.715 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.716 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.716 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.716 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.716 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.716 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.716 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.716 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.716 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.716 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.716 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.716 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.717 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.717 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.717 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.717 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.717 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.717 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.717 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.717 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.717 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.717 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.717 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.718 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.718 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.718 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.718 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.718 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.718 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.718 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.718 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.718 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.719 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.719 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.719 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.719 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.719 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.719 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.719 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.719 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.719 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.719 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.719 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.719 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.720 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.720 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.720 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.720 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.720 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.720 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.720 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.720 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.720 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.720 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.721 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.721 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.721 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.721 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.721 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.721 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.721 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.721 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.721 INFO:teuthology.orchestra.run.smithi164.stdout: "sleep_interval": { 2024-09-19T08:57:49.721 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "sleep_interval", 2024-09-19T08:57:49.722 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:49.722 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.722 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.722 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "60", 2024-09-19T08:57:49.722 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.722 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.722 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.722 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.722 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.722 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.722 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.723 INFO:teuthology.orchestra.run.smithi164.stdout: "threshold": { 2024-09-19T08:57:49.723 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "threshold", 2024-09-19T08:57:49.723 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "float", 2024-09-19T08:57:49.723 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.723 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.723 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "3.0", 2024-09-19T08:57:49.723 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "1.0", 2024-09-19T08:57:49.723 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.723 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.723 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "scaling threshold", 2024-09-19T08:57:49.723 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T08:57:49.723 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.724 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.724 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.724 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.724 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "progress", 2024-09-19T08:57:49.724 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.724 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.724 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.724 INFO:teuthology.orchestra.run.smithi164.stdout: "allow_pg_recovery_event": { 2024-09-19T08:57:49.725 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "allow_pg_recovery_event", 2024-09-19T08:57:49.725 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.725 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.725 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.725 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.725 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.725 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.725 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.725 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T08:57:49.725 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.725 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.726 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.726 INFO:teuthology.orchestra.run.smithi164.stdout: "enabled": { 2024-09-19T08:57:49.726 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "enabled", 2024-09-19T08:57:49.726 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.726 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.726 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.726 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.726 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.726 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.726 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.726 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.727 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.727 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.727 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.727 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.727 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.727 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.727 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.727 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.727 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.727 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.728 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.728 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.728 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.728 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.728 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.728 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.728 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.728 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.728 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.728 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.728 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.729 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.729 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.729 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.729 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.729 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.729 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.729 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.729 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.729 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.729 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.730 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.730 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.730 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.730 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.730 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.730 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.730 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.730 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.730 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.731 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.731 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.731 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.731 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.731 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.731 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.731 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.731 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.731 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.731 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.731 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.732 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.732 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.732 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.732 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.732 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.732 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.732 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.732 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.732 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.732 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.733 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.733 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.733 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.733 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.733 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.733 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.733 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.733 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.733 INFO:teuthology.orchestra.run.smithi164.stdout: "max_completed_events": { 2024-09-19T08:57:49.733 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "max_completed_events", 2024-09-19T08:57:49.734 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.734 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.734 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.734 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "50", 2024-09-19T08:57:49.734 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.734 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.734 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.734 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "number of past completed events to remember", 2024-09-19T08:57:49.734 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.734 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.735 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.735 INFO:teuthology.orchestra.run.smithi164.stdout: "sleep_interval": { 2024-09-19T08:57:49.735 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "sleep_interval", 2024-09-19T08:57:49.735 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:49.735 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.735 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.735 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "5", 2024-09-19T08:57:49.735 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.735 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.735 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.736 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "how long the module is going to sleep", 2024-09-19T08:57:49.736 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.736 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.736 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.736 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.736 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.737 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "prometheus", 2024-09-19T08:57:49.737 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.737 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.737 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.737 INFO:teuthology.orchestra.run.smithi164.stdout: "cache": { 2024-09-19T08:57:49.737 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "cache", 2024-09-19T08:57:49.737 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.737 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.737 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.737 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.738 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.738 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.738 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.738 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.738 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.738 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.738 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.738 INFO:teuthology.orchestra.run.smithi164.stdout: "exclude_perf_counters": { 2024-09-19T08:57:49.738 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "exclude_perf_counters", 2024-09-19T08:57:49.738 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.739 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.739 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.739 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.739 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.739 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.739 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.739 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T08:57:49.739 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T08:57:49.739 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.739 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.740 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.740 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.740 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.740 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.740 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.740 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.740 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.740 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.740 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.740 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.741 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.741 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.741 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.741 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.741 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.741 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.741 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.741 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.741 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.741 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.741 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.742 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.742 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.742 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.742 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.742 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.742 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.742 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.742 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.742 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.742 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.743 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.743 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.743 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.743 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.743 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.743 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.743 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.743 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.743 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.743 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.743 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.743 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.744 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.744 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.744 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.744 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.744 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.744 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.744 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.744 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.744 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.744 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.744 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.745 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.745 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.745 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.745 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.745 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.745 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.745 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.745 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.745 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.745 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.745 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.746 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.746 INFO:teuthology.orchestra.run.smithi164.stdout: "rbd_stats_pools": { 2024-09-19T08:57:49.746 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rbd_stats_pools", 2024-09-19T08:57:49.746 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.746 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.746 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.746 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.746 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.746 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.746 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.747 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.747 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.747 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.747 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.747 INFO:teuthology.orchestra.run.smithi164.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T08:57:49.747 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T08:57:49.747 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.747 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.747 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.747 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "300", 2024-09-19T08:57:49.748 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.748 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.748 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.748 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.748 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.748 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.748 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.748 INFO:teuthology.orchestra.run.smithi164.stdout: "scrape_interval": { 2024-09-19T08:57:49.748 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "scrape_interval", 2024-09-19T08:57:49.748 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "float", 2024-09-19T08:57:49.749 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.749 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.749 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "15.0", 2024-09-19T08:57:49.749 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.749 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.749 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.749 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.749 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.749 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.749 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.750 INFO:teuthology.orchestra.run.smithi164.stdout: "server_addr": { 2024-09-19T08:57:49.750 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "server_addr", 2024-09-19T08:57:49.750 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.750 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.750 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.750 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "::", 2024-09-19T08:57:49.750 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.750 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.750 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.750 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T08:57:49.750 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.751 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.751 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.751 INFO:teuthology.orchestra.run.smithi164.stdout: "server_port": { 2024-09-19T08:57:49.751 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "server_port", 2024-09-19T08:57:49.751 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.751 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.751 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.751 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "9283", 2024-09-19T08:57:49.751 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.751 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.752 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.752 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T08:57:49.752 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.752 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.752 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.752 INFO:teuthology.orchestra.run.smithi164.stdout: "stale_cache_strategy": { 2024-09-19T08:57:49.752 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "stale_cache_strategy", 2024-09-19T08:57:49.752 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.753 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.753 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.753 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "log", 2024-09-19T08:57:49.753 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.753 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.753 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.753 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.753 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.753 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.753 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.753 INFO:teuthology.orchestra.run.smithi164.stdout: "standby_behaviour": { 2024-09-19T08:57:49.754 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "standby_behaviour", 2024-09-19T08:57:49.754 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.754 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.754 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.754 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "default", 2024-09-19T08:57:49.754 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.754 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.754 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.754 INFO:teuthology.orchestra.run.smithi164.stdout: "default", 2024-09-19T08:57:49.754 INFO:teuthology.orchestra.run.smithi164.stdout: "error" 2024-09-19T08:57:49.754 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.754 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.754 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.755 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.755 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.755 INFO:teuthology.orchestra.run.smithi164.stdout: "standby_error_status_code": { 2024-09-19T08:57:49.755 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "standby_error_status_code", 2024-09-19T08:57:49.755 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.755 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.755 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.755 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "500", 2024-09-19T08:57:49.755 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "400", 2024-09-19T08:57:49.755 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "599", 2024-09-19T08:57:49.756 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.756 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.756 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.756 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.756 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.756 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.756 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.756 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rbd_support", 2024-09-19T08:57:49.756 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.757 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.757 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.757 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.757 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.757 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.757 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.757 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.757 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.757 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.757 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.757 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.758 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.758 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.758 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.758 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.758 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.758 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.758 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.758 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.758 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.758 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.758 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.759 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.759 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.759 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.759 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.759 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.759 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.759 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.759 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.759 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.759 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.759 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.760 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.760 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.760 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.760 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.760 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.760 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.760 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.760 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.760 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.760 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.761 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.761 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.761 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.761 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.761 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.761 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.761 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.761 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.761 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.761 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.761 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.762 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.762 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.762 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.762 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.762 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.762 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.762 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.762 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.762 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.762 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.763 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.763 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.763 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.763 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.763 INFO:teuthology.orchestra.run.smithi164.stdout: "max_concurrent_snap_create": { 2024-09-19T08:57:49.763 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "max_concurrent_snap_create", 2024-09-19T08:57:49.763 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.763 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.763 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.763 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "10", 2024-09-19T08:57:49.763 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.764 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.764 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.764 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.764 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.764 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.764 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.764 INFO:teuthology.orchestra.run.smithi164.stdout: "mirror_snapshot_schedule": { 2024-09-19T08:57:49.764 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T08:57:49.764 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.765 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.765 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.765 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.765 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.765 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.765 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.765 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.765 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.765 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.765 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.765 INFO:teuthology.orchestra.run.smithi164.stdout: "trash_purge_schedule": { 2024-09-19T08:57:49.766 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "trash_purge_schedule", 2024-09-19T08:57:49.766 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.766 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.766 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.766 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.766 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.766 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.766 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.766 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.766 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.766 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.767 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.767 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.767 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.767 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "restful", 2024-09-19T08:57:49.767 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.767 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.767 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.767 INFO:teuthology.orchestra.run.smithi164.stdout: "enable_auth": { 2024-09-19T08:57:49.767 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "enable_auth", 2024-09-19T08:57:49.768 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.768 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.768 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.768 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.768 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.768 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.768 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.768 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.768 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.768 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.768 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.769 INFO:teuthology.orchestra.run.smithi164.stdout: "key_file": { 2024-09-19T08:57:49.769 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "key_file", 2024-09-19T08:57:49.769 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.769 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.769 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.769 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.769 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.769 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.769 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.769 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.769 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.770 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.770 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.770 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.770 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.770 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.771 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.771 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.771 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.771 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.771 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.771 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.771 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.771 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.771 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.771 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.771 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.771 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.772 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.772 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.772 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.772 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.772 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.772 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.772 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.772 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.772 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.772 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.773 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.773 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.773 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.773 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.773 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.773 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.773 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.773 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.773 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.773 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.774 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.774 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.774 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.774 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.774 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.774 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.774 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.774 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.774 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.774 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.774 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.775 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.775 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.775 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.775 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.775 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.775 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.775 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.775 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.775 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.775 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.775 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.776 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.776 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.776 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.776 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.776 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.776 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.776 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.776 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.776 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.776 INFO:teuthology.orchestra.run.smithi164.stdout: "server_addr": { 2024-09-19T08:57:49.777 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "server_addr", 2024-09-19T08:57:49.777 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.777 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.777 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.777 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.777 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.777 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.777 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.777 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.777 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.777 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.778 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.778 INFO:teuthology.orchestra.run.smithi164.stdout: "server_port": { 2024-09-19T08:57:49.778 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "server_port", 2024-09-19T08:57:49.778 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.778 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.778 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.778 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.778 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.778 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.778 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.779 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.779 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.779 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.779 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.779 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.779 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.779 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rgw", 2024-09-19T08:57:49.779 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.779 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.779 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.780 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.780 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.780 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.780 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.780 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.780 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.780 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.780 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.780 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.780 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.780 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.781 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.781 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.781 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.781 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.781 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.781 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.781 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.781 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.781 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.781 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.782 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.782 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.782 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.782 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.782 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.782 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.782 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.782 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.782 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.782 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.782 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.782 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.783 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.783 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.783 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.783 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.783 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.783 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.783 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.783 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.783 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.783 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.784 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.784 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.784 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.784 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.784 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.784 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.784 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.784 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.784 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.784 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.785 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.785 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.785 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.785 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.785 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.785 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.785 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.785 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.785 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.785 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.785 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.785 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.786 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.786 INFO:teuthology.orchestra.run.smithi164.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T08:57:49.786 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T08:57:49.786 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.786 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.786 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.786 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "5", 2024-09-19T08:57:49.786 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.786 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.786 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.786 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T08:57:49.787 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.787 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.787 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.787 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.787 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.787 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rook", 2024-09-19T08:57:49.787 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.787 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.788 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.788 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.788 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.788 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.788 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.788 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.788 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.788 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.788 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.788 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.788 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.789 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.789 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.789 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.789 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.789 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.789 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.789 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.789 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.789 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.789 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.789 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.790 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.790 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.790 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.790 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.790 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.790 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.790 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.790 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.790 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.790 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.791 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.791 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.791 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.791 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.791 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.791 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.791 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.791 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.791 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.791 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.791 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.792 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.792 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.792 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.792 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.792 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.792 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.792 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.792 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.792 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.792 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.792 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.793 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.793 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.793 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.793 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.793 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.793 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.793 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.793 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.793 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.793 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.794 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.794 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.794 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.794 INFO:teuthology.orchestra.run.smithi164.stdout: "storage_class": { 2024-09-19T08:57:49.794 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "storage_class", 2024-09-19T08:57:49.794 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.794 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.794 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.794 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "local", 2024-09-19T08:57:49.794 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.795 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.795 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.795 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T08:57:49.795 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.795 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.795 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.795 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.795 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.795 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "selftest", 2024-09-19T08:57:49.796 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.796 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.796 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.796 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.796 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.796 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.796 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.796 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.796 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.796 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.796 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.797 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.797 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.797 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.797 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.797 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.797 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.797 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.797 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.797 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.797 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.797 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.798 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.798 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.798 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.798 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.798 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.798 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.798 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.798 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.798 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.798 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.798 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.799 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.799 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.799 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.799 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.799 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.799 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.799 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.799 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.799 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.799 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.800 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.800 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.800 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.800 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.800 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.800 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.800 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.800 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.800 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.800 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.800 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.801 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.801 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.801 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.801 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.801 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.801 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.801 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.801 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.801 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.801 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.802 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.802 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.802 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.802 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.802 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.802 INFO:teuthology.orchestra.run.smithi164.stdout: "roption1": { 2024-09-19T08:57:49.802 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "roption1", 2024-09-19T08:57:49.802 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.802 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.802 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.802 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.803 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.803 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.803 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.803 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.803 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.803 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.803 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.803 INFO:teuthology.orchestra.run.smithi164.stdout: "roption2": { 2024-09-19T08:57:49.803 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "roption2", 2024-09-19T08:57:49.803 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.804 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.804 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.804 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "xyz", 2024-09-19T08:57:49.804 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.804 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.804 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.804 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.804 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.804 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.804 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.805 INFO:teuthology.orchestra.run.smithi164.stdout: "rwoption1": { 2024-09-19T08:57:49.805 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rwoption1", 2024-09-19T08:57:49.805 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.805 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.805 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.805 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.805 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.805 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.805 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.805 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.806 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.806 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.806 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.806 INFO:teuthology.orchestra.run.smithi164.stdout: "rwoption2": { 2024-09-19T08:57:49.806 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rwoption2", 2024-09-19T08:57:49.806 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.806 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.806 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.806 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.806 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.806 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.807 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.807 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.807 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.807 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.807 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.807 INFO:teuthology.orchestra.run.smithi164.stdout: "rwoption3": { 2024-09-19T08:57:49.807 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rwoption3", 2024-09-19T08:57:49.807 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "float", 2024-09-19T08:57:49.807 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.807 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.808 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.808 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.808 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.808 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.808 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.808 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.808 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.808 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.808 INFO:teuthology.orchestra.run.smithi164.stdout: "rwoption4": { 2024-09-19T08:57:49.808 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rwoption4", 2024-09-19T08:57:49.808 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.809 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.809 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.809 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.809 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.809 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.809 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.809 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.809 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.809 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.809 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.810 INFO:teuthology.orchestra.run.smithi164.stdout: "rwoption5": { 2024-09-19T08:57:49.810 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rwoption5", 2024-09-19T08:57:49.810 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.810 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.810 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.810 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.810 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.810 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.810 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.810 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.810 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.811 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.811 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.811 INFO:teuthology.orchestra.run.smithi164.stdout: "rwoption6": { 2024-09-19T08:57:49.811 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rwoption6", 2024-09-19T08:57:49.811 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.811 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.811 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.811 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.811 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.811 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.812 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.812 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.812 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.812 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.812 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.812 INFO:teuthology.orchestra.run.smithi164.stdout: "rwoption7": { 2024-09-19T08:57:49.812 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rwoption7", 2024-09-19T08:57:49.812 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.812 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.812 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.813 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.813 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "1", 2024-09-19T08:57:49.813 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "42", 2024-09-19T08:57:49.813 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.813 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.813 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.813 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.813 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.813 INFO:teuthology.orchestra.run.smithi164.stdout: "testkey": { 2024-09-19T08:57:49.813 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "testkey", 2024-09-19T08:57:49.813 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.814 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.814 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.814 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.814 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.814 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.814 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.814 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.814 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.814 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.814 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.815 INFO:teuthology.orchestra.run.smithi164.stdout: "testlkey": { 2024-09-19T08:57:49.815 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "testlkey", 2024-09-19T08:57:49.815 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.815 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.815 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.815 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.815 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.815 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.815 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.815 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.815 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.816 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.816 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.816 INFO:teuthology.orchestra.run.smithi164.stdout: "testnewline": { 2024-09-19T08:57:49.816 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "testnewline", 2024-09-19T08:57:49.816 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.816 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.816 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.816 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.816 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.816 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.817 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.817 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.817 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.817 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.817 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.817 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.817 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.817 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "snap_schedule", 2024-09-19T08:57:49.817 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.817 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.818 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.818 INFO:teuthology.orchestra.run.smithi164.stdout: "allow_m_granularity": { 2024-09-19T08:57:49.818 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "allow_m_granularity", 2024-09-19T08:57:49.818 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.818 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.818 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.818 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.818 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.818 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.818 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.818 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T08:57:49.819 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.819 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.819 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.819 INFO:teuthology.orchestra.run.smithi164.stdout: "dump_on_update": { 2024-09-19T08:57:49.819 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "dump_on_update", 2024-09-19T08:57:49.819 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.819 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.819 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.819 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.819 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.820 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.820 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.820 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "dump database to debug log on update", 2024-09-19T08:57:49.820 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.820 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.820 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.820 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.820 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.820 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.820 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.820 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.821 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.821 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.821 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.821 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.821 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.821 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.821 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.821 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.821 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.821 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.821 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.821 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.822 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.822 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.822 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.822 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.822 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.822 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.822 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.822 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.822 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.823 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.823 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.823 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.823 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.823 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.823 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.823 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.823 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.823 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.823 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.824 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.824 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.824 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.824 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.824 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.824 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.824 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.824 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.824 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.824 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.824 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.825 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.825 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.825 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.825 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.825 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.825 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.825 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.825 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.825 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.826 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.826 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.826 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.826 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.826 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.826 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.826 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.826 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.826 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.826 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.826 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.827 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.827 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "stats", 2024-09-19T08:57:49.827 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.827 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.827 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.827 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.827 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.827 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.828 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.828 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.828 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.828 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.828 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.828 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.828 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.828 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.828 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.828 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.829 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.829 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.829 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.829 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.829 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.829 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.829 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.829 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.829 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.829 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.829 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.830 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.830 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.830 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.830 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.830 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.830 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.830 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.830 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.830 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.830 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.831 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.831 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.831 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.831 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.831 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.831 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.831 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.831 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.831 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.831 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.831 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.831 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.832 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.832 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.832 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.832 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.832 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.832 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.832 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.832 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.832 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.833 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.833 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.833 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.833 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.833 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.833 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.833 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.833 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.833 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.833 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.833 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.833 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.834 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.834 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "status", 2024-09-19T08:57:49.834 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.834 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.834 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.834 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.834 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.834 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.834 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.835 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.835 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.835 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.835 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.835 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.835 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.835 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.835 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.835 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.835 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.835 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.835 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.836 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.836 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.836 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.836 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.836 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.836 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.836 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.836 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.836 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.836 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.837 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.837 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.837 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.837 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.837 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.837 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.837 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.837 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.837 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.837 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.838 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.838 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.838 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.838 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.838 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.838 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.838 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.838 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.838 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.838 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.838 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.839 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.839 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.839 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.839 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.839 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.839 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.839 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.839 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.839 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.839 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.840 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.840 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.840 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.840 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.840 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.840 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.840 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.840 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.840 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.840 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.841 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.841 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "telegraf", 2024-09-19T08:57:49.841 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.841 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.841 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.841 INFO:teuthology.orchestra.run.smithi164.stdout: "address": { 2024-09-19T08:57:49.841 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "address", 2024-09-19T08:57:49.841 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.841 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.842 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.842 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T08:57:49.842 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.842 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.842 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.842 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.842 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.842 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.842 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.842 INFO:teuthology.orchestra.run.smithi164.stdout: "interval": { 2024-09-19T08:57:49.842 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "interval", 2024-09-19T08:57:49.843 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:49.843 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.843 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.843 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "15", 2024-09-19T08:57:49.843 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.843 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.843 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.843 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.843 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.843 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.843 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.844 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.844 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.844 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.844 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.844 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.844 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.844 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.844 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.844 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.844 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.845 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.845 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.845 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.845 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.845 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.845 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.845 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.845 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.845 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.845 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.846 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.846 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.846 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.846 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.846 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.846 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.846 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.846 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.846 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.846 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.846 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.847 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.847 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.847 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.847 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.847 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.847 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.847 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.847 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.847 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.847 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.847 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.848 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.848 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.848 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.848 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.848 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.848 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.848 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.848 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.848 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.848 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.849 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.849 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.849 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.849 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.849 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.849 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.849 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.849 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.849 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.849 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.849 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.850 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.850 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.850 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.850 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.850 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.850 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "telemetry", 2024-09-19T08:57:49.850 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.850 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.850 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.850 INFO:teuthology.orchestra.run.smithi164.stdout: "channel_basic": { 2024-09-19T08:57:49.851 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "channel_basic", 2024-09-19T08:57:49.851 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.851 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.851 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.851 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.851 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.851 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.851 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.851 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T08:57:49.851 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.851 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.852 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.852 INFO:teuthology.orchestra.run.smithi164.stdout: "channel_crash": { 2024-09-19T08:57:49.852 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "channel_crash", 2024-09-19T08:57:49.852 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.852 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.852 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.852 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.852 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.852 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.852 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.852 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T08:57:49.853 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.853 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.853 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.853 INFO:teuthology.orchestra.run.smithi164.stdout: "channel_device": { 2024-09-19T08:57:49.853 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "channel_device", 2024-09-19T08:57:49.853 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.853 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.853 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.853 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.853 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.854 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.854 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.854 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T08:57:49.854 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.854 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.854 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.854 INFO:teuthology.orchestra.run.smithi164.stdout: "channel_ident": { 2024-09-19T08:57:49.854 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "channel_ident", 2024-09-19T08:57:49.854 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.854 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.855 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.855 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.855 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.855 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.855 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.855 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T08:57:49.855 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.855 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.855 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.855 INFO:teuthology.orchestra.run.smithi164.stdout: "channel_perf": { 2024-09-19T08:57:49.855 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "channel_perf", 2024-09-19T08:57:49.856 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.856 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.856 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.856 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.856 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.856 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.856 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.856 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T08:57:49.856 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.856 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.856 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.857 INFO:teuthology.orchestra.run.smithi164.stdout: "contact": { 2024-09-19T08:57:49.857 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "contact", 2024-09-19T08:57:49.857 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.857 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.857 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.857 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.857 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.857 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.857 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.858 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.858 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.858 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.858 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.858 INFO:teuthology.orchestra.run.smithi164.stdout: "description": { 2024-09-19T08:57:49.858 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "description", 2024-09-19T08:57:49.858 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.858 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.858 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.858 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.858 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.859 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.859 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.859 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.859 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.859 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.859 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.859 INFO:teuthology.orchestra.run.smithi164.stdout: "device_url": { 2024-09-19T08:57:49.859 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "device_url", 2024-09-19T08:57:49.859 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.859 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.860 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.860 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T08:57:49.860 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.860 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.860 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.860 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.860 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.860 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.860 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.860 INFO:teuthology.orchestra.run.smithi164.stdout: "enabled": { 2024-09-19T08:57:49.861 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "enabled", 2024-09-19T08:57:49.861 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.861 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.861 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.861 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.861 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.861 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.861 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.861 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.861 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.861 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.862 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.862 INFO:teuthology.orchestra.run.smithi164.stdout: "interval": { 2024-09-19T08:57:49.862 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "interval", 2024-09-19T08:57:49.862 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.862 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.862 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.862 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "24", 2024-09-19T08:57:49.862 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "8", 2024-09-19T08:57:49.862 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.862 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.863 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.863 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.863 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.863 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.863 INFO:teuthology.orchestra.run.smithi164.stdout: "last_opt_revision": { 2024-09-19T08:57:49.863 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "last_opt_revision", 2024-09-19T08:57:49.863 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.863 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.863 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.863 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "1", 2024-09-19T08:57:49.864 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.864 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.864 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.864 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.864 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.864 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.864 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.864 INFO:teuthology.orchestra.run.smithi164.stdout: "leaderboard": { 2024-09-19T08:57:49.864 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "leaderboard", 2024-09-19T08:57:49.864 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.864 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.865 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.865 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.865 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.865 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.865 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.865 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.865 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.865 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.865 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.865 INFO:teuthology.orchestra.run.smithi164.stdout: "leaderboard_description": { 2024-09-19T08:57:49.866 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "leaderboard_description", 2024-09-19T08:57:49.866 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.866 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.866 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.866 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.866 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.866 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.866 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.866 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.866 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.867 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.867 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.867 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.867 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.867 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.867 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.867 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.867 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.867 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.867 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.868 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.868 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.868 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.868 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.868 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.868 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.868 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.868 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.868 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.868 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.868 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.869 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.869 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.869 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.869 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.869 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.869 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.869 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.869 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.869 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.869 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.870 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.870 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.870 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.870 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.870 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.870 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.870 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.870 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.870 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.870 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.871 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.871 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.871 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.871 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.871 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.871 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.871 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.871 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.871 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.871 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.871 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.872 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.872 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.872 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.872 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.872 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.872 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.872 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.872 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.872 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.873 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.873 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.873 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.873 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.873 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.873 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.873 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.873 INFO:teuthology.orchestra.run.smithi164.stdout: "organization": { 2024-09-19T08:57:49.873 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "organization", 2024-09-19T08:57:49.873 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.873 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.874 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.874 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.874 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.874 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.874 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.874 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.874 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.874 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.874 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.875 INFO:teuthology.orchestra.run.smithi164.stdout: "proxy": { 2024-09-19T08:57:49.875 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "proxy", 2024-09-19T08:57:49.875 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.875 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.875 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.875 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.875 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.875 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.875 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.875 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.875 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.876 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.876 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.876 INFO:teuthology.orchestra.run.smithi164.stdout: "url": { 2024-09-19T08:57:49.876 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "url", 2024-09-19T08:57:49.876 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.876 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.876 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.876 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T08:57:49.876 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.876 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.877 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.877 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.877 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.877 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.877 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.877 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.877 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.877 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "test_orchestrator", 2024-09-19T08:57:49.877 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.878 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.878 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.878 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.878 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.878 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.878 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.878 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.878 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.878 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.878 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.878 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.878 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.879 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.879 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.879 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.879 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.879 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.879 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.879 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.879 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.879 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.879 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.880 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.880 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.880 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.880 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.880 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.880 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.880 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.880 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.880 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.880 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.880 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.881 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.881 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.881 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.881 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.881 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.881 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.881 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.881 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.881 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.881 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.882 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.882 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.882 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.882 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.882 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.882 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.882 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.882 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.882 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.882 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.882 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.882 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.883 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.883 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.883 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.883 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.883 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.883 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.883 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.883 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.883 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.883 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.883 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.884 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.884 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.884 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.884 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.884 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "volumes", 2024-09-19T08:57:49.884 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.884 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.884 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.884 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.885 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.885 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.885 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.885 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.885 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.885 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.885 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.885 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.885 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.885 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.885 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.886 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.886 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.886 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.886 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.886 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.886 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.886 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.886 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.886 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.886 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.887 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.887 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.887 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.887 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.887 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.887 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.887 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.887 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.887 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.887 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.887 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.888 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.888 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.888 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.888 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.888 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.888 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.888 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.888 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.888 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.888 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.888 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.889 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.889 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.889 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.889 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.889 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.889 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.889 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.889 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.889 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.889 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.890 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.890 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.890 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.890 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.890 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.890 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.890 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.890 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.890 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.890 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.890 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.890 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.891 INFO:teuthology.orchestra.run.smithi164.stdout: "max_concurrent_clones": { 2024-09-19T08:57:49.891 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "max_concurrent_clones", 2024-09-19T08:57:49.891 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.891 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.891 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.891 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "4", 2024-09-19T08:57:49.891 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.891 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.891 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.891 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T08:57:49.892 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.892 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.892 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.892 INFO:teuthology.orchestra.run.smithi164.stdout: "periodic_async_work": { 2024-09-19T08:57:49.892 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "periodic_async_work", 2024-09-19T08:57:49.892 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.892 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.892 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.892 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.893 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.893 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.893 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.893 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Periodically check for async work", 2024-09-19T08:57:49.893 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.893 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.893 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.893 INFO:teuthology.orchestra.run.smithi164.stdout: "snapshot_clone_delay": { 2024-09-19T08:57:49.893 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "snapshot_clone_delay", 2024-09-19T08:57:49.893 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.894 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.894 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.894 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "0", 2024-09-19T08:57:49.894 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.894 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.894 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.894 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T08:57:49.894 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.894 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.894 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.895 INFO:teuthology.orchestra.run.smithi164.stdout: "snapshot_clone_no_wait": { 2024-09-19T08:57:49.895 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T08:57:49.895 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.895 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.895 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.895 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.895 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.895 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.895 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.895 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T08:57:49.895 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.896 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.896 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.896 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.896 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.896 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "zabbix", 2024-09-19T08:57:49.896 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.896 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.896 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.896 INFO:teuthology.orchestra.run.smithi164.stdout: "discovery_interval": { 2024-09-19T08:57:49.897 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "discovery_interval", 2024-09-19T08:57:49.897 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "uint", 2024-09-19T08:57:49.897 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.897 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.897 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "100", 2024-09-19T08:57:49.897 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.897 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.897 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.897 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.897 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.898 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.898 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.898 INFO:teuthology.orchestra.run.smithi164.stdout: "identifier": { 2024-09-19T08:57:49.898 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "identifier", 2024-09-19T08:57:49.898 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.898 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.898 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.898 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.898 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.898 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.898 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.899 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.899 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.899 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.899 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.899 INFO:teuthology.orchestra.run.smithi164.stdout: "interval": { 2024-09-19T08:57:49.899 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "interval", 2024-09-19T08:57:49.899 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:49.899 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.899 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.899 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "60", 2024-09-19T08:57:49.899 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.900 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.900 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.900 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.900 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.900 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.900 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.900 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.900 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.900 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.901 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.901 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.901 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.901 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.901 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.901 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.901 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.901 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.901 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.901 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.901 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.901 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.902 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.902 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.902 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.902 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.902 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.902 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.902 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.902 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.902 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.903 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.903 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.903 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.903 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.903 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.903 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.903 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.903 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.903 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.903 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.904 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.904 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.904 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.904 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.904 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.904 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.904 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.904 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.904 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.904 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.905 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.905 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.905 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.905 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.905 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.905 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.905 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.905 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.905 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.905 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.905 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.906 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.906 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.906 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.906 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.906 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.906 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.906 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.906 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.906 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.906 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.906 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.907 INFO:teuthology.orchestra.run.smithi164.stdout: "zabbix_host": { 2024-09-19T08:57:49.907 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "zabbix_host", 2024-09-19T08:57:49.907 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.907 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.907 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.907 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.907 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.907 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.907 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.908 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.908 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.908 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.908 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.908 INFO:teuthology.orchestra.run.smithi164.stdout: "zabbix_port": { 2024-09-19T08:57:49.908 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "zabbix_port", 2024-09-19T08:57:49.908 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.908 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.908 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.908 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "10051", 2024-09-19T08:57:49.908 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.909 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.909 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.909 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.909 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.909 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.909 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.909 INFO:teuthology.orchestra.run.smithi164.stdout: "zabbix_sender": { 2024-09-19T08:57:49.909 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "zabbix_sender", 2024-09-19T08:57:49.910 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.910 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.910 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.910 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T08:57:49.910 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.910 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.910 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.910 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.910 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.910 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.910 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.911 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.911 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.911 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.911 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-19T08:57:49.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.911 INFO:teuthology.orchestra.run.smithi164.stdout: "gid": 24830, 2024-09-19T08:57:49.911 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "y", 2024-09-19T08:57:49.911 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr_features": 4540701547738038271, 2024-09-19T08:57:49.911 INFO:teuthology.orchestra.run.smithi164.stdout: "available_modules": [ 2024-09-19T08:57:49.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.912 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "alerts", 2024-09-19T08:57:49.912 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.912 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.912 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.912 INFO:teuthology.orchestra.run.smithi164.stdout: "interval": { 2024-09-19T08:57:49.912 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "interval", 2024-09-19T08:57:49.912 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:49.912 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.912 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.912 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "60", 2024-09-19T08:57:49.913 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.913 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.913 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.913 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T08:57:49.913 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.913 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.913 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.913 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.913 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.914 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.914 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.914 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.914 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.914 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.914 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.914 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.914 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.914 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.914 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.914 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.915 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.915 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.915 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.915 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.915 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.915 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.915 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.915 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.915 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.915 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.916 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.916 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.916 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.916 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.916 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.916 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.916 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.916 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.916 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.916 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.917 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.917 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.917 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.917 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.917 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.917 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.917 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.917 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.917 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.917 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.918 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.918 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.918 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.918 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.918 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.918 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.918 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.918 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.918 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.918 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.918 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.919 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.919 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.919 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.919 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.919 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.919 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.919 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.919 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.919 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.919 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.919 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.920 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.920 INFO:teuthology.orchestra.run.smithi164.stdout: "smtp_destination": { 2024-09-19T08:57:49.920 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "smtp_destination", 2024-09-19T08:57:49.920 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.920 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.920 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.920 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.920 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.920 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.921 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.921 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Email address to send alerts to", 2024-09-19T08:57:49.921 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.921 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.921 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.921 INFO:teuthology.orchestra.run.smithi164.stdout: "smtp_from_name": { 2024-09-19T08:57:49.921 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "smtp_from_name", 2024-09-19T08:57:49.921 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.921 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.921 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.921 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "Ceph", 2024-09-19T08:57:49.922 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.922 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.922 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.922 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Email From: name", 2024-09-19T08:57:49.922 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.922 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.922 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.922 INFO:teuthology.orchestra.run.smithi164.stdout: "smtp_host": { 2024-09-19T08:57:49.922 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "smtp_host", 2024-09-19T08:57:49.922 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.923 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.923 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.923 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.923 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.923 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.923 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.923 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "SMTP server", 2024-09-19T08:57:49.923 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.923 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.923 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.924 INFO:teuthology.orchestra.run.smithi164.stdout: "smtp_password": { 2024-09-19T08:57:49.924 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "smtp_password", 2024-09-19T08:57:49.924 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.924 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.924 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.924 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.924 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.924 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.924 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.924 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Password to authenticate with", 2024-09-19T08:57:49.924 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.925 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.925 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.925 INFO:teuthology.orchestra.run.smithi164.stdout: "smtp_port": { 2024-09-19T08:57:49.925 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "smtp_port", 2024-09-19T08:57:49.925 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.925 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.925 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.925 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "465", 2024-09-19T08:57:49.925 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.925 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.926 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.926 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "SMTP port", 2024-09-19T08:57:49.926 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.926 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.926 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.926 INFO:teuthology.orchestra.run.smithi164.stdout: "smtp_sender": { 2024-09-19T08:57:49.926 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "smtp_sender", 2024-09-19T08:57:49.926 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.926 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.926 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.927 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.927 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.927 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.927 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.927 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "SMTP envelope sender", 2024-09-19T08:57:49.927 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.927 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.927 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.927 INFO:teuthology.orchestra.run.smithi164.stdout: "smtp_ssl": { 2024-09-19T08:57:49.927 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "smtp_ssl", 2024-09-19T08:57:49.928 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.928 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.928 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.928 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.928 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.928 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.928 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.928 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T08:57:49.928 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.929 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.929 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.929 INFO:teuthology.orchestra.run.smithi164.stdout: "smtp_user": { 2024-09-19T08:57:49.929 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "smtp_user", 2024-09-19T08:57:49.929 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.929 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.929 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.929 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.929 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.930 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.930 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.930 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "User to authenticate as", 2024-09-19T08:57:49.930 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.930 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.930 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.930 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.930 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.931 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "balancer", 2024-09-19T08:57:49.931 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.931 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.931 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.931 INFO:teuthology.orchestra.run.smithi164.stdout: "active": { 2024-09-19T08:57:49.931 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "active", 2024-09-19T08:57:49.931 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.931 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.931 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.931 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.931 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.931 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.932 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.932 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T08:57:49.932 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.932 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.932 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.932 INFO:teuthology.orchestra.run.smithi164.stdout: "begin_time": { 2024-09-19T08:57:49.932 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "begin_time", 2024-09-19T08:57:49.932 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.932 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.932 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.932 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "0000", 2024-09-19T08:57:49.933 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.933 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.933 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.933 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T08:57:49.933 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T08:57:49.933 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.933 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.933 INFO:teuthology.orchestra.run.smithi164.stdout: "begin_weekday": { 2024-09-19T08:57:49.933 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "begin_weekday", 2024-09-19T08:57:49.933 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "uint", 2024-09-19T08:57:49.933 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.934 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.934 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "0", 2024-09-19T08:57:49.934 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "0", 2024-09-19T08:57:49.934 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "6", 2024-09-19T08:57:49.934 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.934 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T08:57:49.934 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T08:57:49.934 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.934 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.934 INFO:teuthology.orchestra.run.smithi164.stdout: "crush_compat_max_iterations": { 2024-09-19T08:57:49.935 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "crush_compat_max_iterations", 2024-09-19T08:57:49.935 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "uint", 2024-09-19T08:57:49.935 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.935 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.935 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "25", 2024-09-19T08:57:49.935 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "1", 2024-09-19T08:57:49.935 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "250", 2024-09-19T08:57:49.935 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.935 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T08:57:49.935 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.935 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.935 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.936 INFO:teuthology.orchestra.run.smithi164.stdout: "crush_compat_metrics": { 2024-09-19T08:57:49.936 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "crush_compat_metrics", 2024-09-19T08:57:49.936 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.936 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.936 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.936 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T08:57:49.936 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.936 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.936 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.937 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T08:57:49.937 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T08:57:49.937 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.937 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.937 INFO:teuthology.orchestra.run.smithi164.stdout: "crush_compat_step": { 2024-09-19T08:57:49.937 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "crush_compat_step", 2024-09-19T08:57:49.937 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "float", 2024-09-19T08:57:49.937 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.937 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.938 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "0.5", 2024-09-19T08:57:49.938 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "0.001", 2024-09-19T08:57:49.938 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "0.999", 2024-09-19T08:57:49.938 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.938 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "aggressiveness of optimization", 2024-09-19T08:57:49.938 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T08:57:49.938 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.938 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.938 INFO:teuthology.orchestra.run.smithi164.stdout: "end_time": { 2024-09-19T08:57:49.938 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "end_time", 2024-09-19T08:57:49.938 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.939 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.939 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.939 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "2359", 2024-09-19T08:57:49.939 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.939 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.939 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.939 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T08:57:49.939 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T08:57:49.939 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.939 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.940 INFO:teuthology.orchestra.run.smithi164.stdout: "end_weekday": { 2024-09-19T08:57:49.940 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "end_weekday", 2024-09-19T08:57:49.940 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "uint", 2024-09-19T08:57:49.940 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.940 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.940 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "0", 2024-09-19T08:57:49.940 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "0", 2024-09-19T08:57:49.940 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "6", 2024-09-19T08:57:49.940 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.940 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T08:57:49.940 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T08:57:49.941 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.941 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.941 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:49.941 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:49.941 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.941 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.941 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.941 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.941 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.941 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.942 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.942 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.942 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.942 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.942 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.942 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.942 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.942 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.942 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.942 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.942 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.942 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.943 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:49.943 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:49.943 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.943 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.943 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.943 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.943 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.943 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.943 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.944 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.944 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.944 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.944 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.944 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:49.944 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:49.944 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.944 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.944 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.944 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:49.945 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.945 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.945 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.945 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:49.945 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:49.945 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:49.945 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:49.945 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:49.945 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:49.945 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.945 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.946 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.946 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.946 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.946 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:49.946 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:49.946 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.946 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.946 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.946 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.946 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.947 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.947 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.947 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:49.947 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.947 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.947 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.947 INFO:teuthology.orchestra.run.smithi164.stdout: "min_score": { 2024-09-19T08:57:49.947 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "min_score", 2024-09-19T08:57:49.947 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "float", 2024-09-19T08:57:49.947 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.948 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.948 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "0", 2024-09-19T08:57:49.948 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.948 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.948 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.948 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T08:57:49.948 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.948 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.948 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.949 INFO:teuthology.orchestra.run.smithi164.stdout: "mode": { 2024-09-19T08:57:49.949 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "mode", 2024-09-19T08:57:49.949 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.949 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.949 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.949 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "upmap", 2024-09-19T08:57:49.949 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.949 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.949 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.949 INFO:teuthology.orchestra.run.smithi164.stdout: "crush-compat", 2024-09-19T08:57:49.949 INFO:teuthology.orchestra.run.smithi164.stdout: "none", 2024-09-19T08:57:49.950 INFO:teuthology.orchestra.run.smithi164.stdout: "read", 2024-09-19T08:57:49.950 INFO:teuthology.orchestra.run.smithi164.stdout: "upmap", 2024-09-19T08:57:49.950 INFO:teuthology.orchestra.run.smithi164.stdout: "upmap-read" 2024-09-19T08:57:49.950 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.950 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Balancer mode", 2024-09-19T08:57:49.950 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.950 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.950 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.950 INFO:teuthology.orchestra.run.smithi164.stdout: "pool_ids": { 2024-09-19T08:57:49.951 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "pool_ids", 2024-09-19T08:57:49.951 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.951 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.951 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.951 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.951 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.951 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.951 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.951 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T08:57:49.951 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.952 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.952 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.952 INFO:teuthology.orchestra.run.smithi164.stdout: "sleep_interval": { 2024-09-19T08:57:49.952 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "sleep_interval", 2024-09-19T08:57:49.952 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:49.952 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.952 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.952 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "60", 2024-09-19T08:57:49.952 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.952 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.953 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.953 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T08:57:49.953 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.953 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.953 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.953 INFO:teuthology.orchestra.run.smithi164.stdout: "upmap_max_deviation": { 2024-09-19T08:57:49.953 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "upmap_max_deviation", 2024-09-19T08:57:49.953 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.953 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.954 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.954 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "5", 2024-09-19T08:57:49.954 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "1", 2024-09-19T08:57:49.954 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.954 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.954 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T08:57:49.954 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T08:57:49.954 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.954 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.954 INFO:teuthology.orchestra.run.smithi164.stdout: "upmap_max_optimizations": { 2024-09-19T08:57:49.954 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "upmap_max_optimizations", 2024-09-19T08:57:49.955 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "uint", 2024-09-19T08:57:49.955 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.955 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.955 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "10", 2024-09-19T08:57:49.955 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.955 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.955 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.955 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T08:57:49.955 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.955 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.955 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.956 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.956 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:49.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:49.956 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "cephadm", 2024-09-19T08:57:49.956 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:49.956 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:49.956 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:49.956 INFO:teuthology.orchestra.run.smithi164.stdout: "agent_down_multiplier": { 2024-09-19T08:57:49.956 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "agent_down_multiplier", 2024-09-19T08:57:49.956 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "float", 2024-09-19T08:57:49.957 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.957 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.957 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "3.0", 2024-09-19T08:57:49.957 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.957 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.957 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.957 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T08:57:49.957 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.957 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.957 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.957 INFO:teuthology.orchestra.run.smithi164.stdout: "agent_refresh_rate": { 2024-09-19T08:57:49.958 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "agent_refresh_rate", 2024-09-19T08:57:49.958 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:49.958 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.958 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.958 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "20", 2024-09-19T08:57:49.958 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.958 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.958 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.958 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T08:57:49.958 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.958 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.959 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.959 INFO:teuthology.orchestra.run.smithi164.stdout: "agent_starting_port": { 2024-09-19T08:57:49.959 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "agent_starting_port", 2024-09-19T08:57:49.959 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.959 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.959 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.959 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "4721", 2024-09-19T08:57:49.959 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.959 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.960 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.960 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T08:57:49.960 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.960 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.960 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.960 INFO:teuthology.orchestra.run.smithi164.stdout: "allow_ptrace": { 2024-09-19T08:57:49.960 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "allow_ptrace", 2024-09-19T08:57:49.960 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.960 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.960 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.960 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.961 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.961 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.961 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.961 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T08:57:49.961 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T08:57:49.961 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.961 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.961 INFO:teuthology.orchestra.run.smithi164.stdout: "autotune_interval": { 2024-09-19T08:57:49.961 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "autotune_interval", 2024-09-19T08:57:49.961 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:49.962 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.962 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.962 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "600", 2024-09-19T08:57:49.962 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.962 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.962 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.962 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T08:57:49.962 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.962 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.962 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.962 INFO:teuthology.orchestra.run.smithi164.stdout: "autotune_memory_target_ratio": { 2024-09-19T08:57:49.963 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T08:57:49.963 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "float", 2024-09-19T08:57:49.963 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.963 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.963 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "0.7", 2024-09-19T08:57:49.963 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.963 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.963 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.963 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T08:57:49.963 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.963 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.964 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.964 INFO:teuthology.orchestra.run.smithi164.stdout: "cephadm_log_destination": { 2024-09-19T08:57:49.964 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "cephadm_log_destination", 2024-09-19T08:57:49.964 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.964 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.964 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.964 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:49.964 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.964 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.965 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:49.965 INFO:teuthology.orchestra.run.smithi164.stdout: "file", 2024-09-19T08:57:49.965 INFO:teuthology.orchestra.run.smithi164.stdout: "file,syslog", 2024-09-19T08:57:49.965 INFO:teuthology.orchestra.run.smithi164.stdout: "syslog" 2024-09-19T08:57:49.965 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:49.965 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T08:57:49.965 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.965 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.965 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.965 INFO:teuthology.orchestra.run.smithi164.stdout: "cgroups_split": { 2024-09-19T08:57:49.966 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "cgroups_split", 2024-09-19T08:57:49.966 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.966 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.966 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.966 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.966 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.966 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.966 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.966 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T08:57:49.966 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.967 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.967 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.967 INFO:teuthology.orchestra.run.smithi164.stdout: "config_checks_enabled": { 2024-09-19T08:57:49.967 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "config_checks_enabled", 2024-09-19T08:57:49.967 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.967 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.967 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.967 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.967 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.967 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.968 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.968 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T08:57:49.968 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.968 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.968 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.968 INFO:teuthology.orchestra.run.smithi164.stdout: "config_dashboard": { 2024-09-19T08:57:49.968 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "config_dashboard", 2024-09-19T08:57:49.968 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.968 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.969 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.969 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.969 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.969 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.969 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.969 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T08:57:49.969 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.969 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.969 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.969 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_alertmanager": { 2024-09-19T08:57:49.970 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_alertmanager", 2024-09-19T08:57:49.970 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.970 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.970 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.970 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T08:57:49.970 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.970 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.970 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.970 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Prometheus container image", 2024-09-19T08:57:49.970 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.970 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.971 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.971 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_base": { 2024-09-19T08:57:49.971 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_base", 2024-09-19T08:57:49.971 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.971 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.971 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:49.971 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T08:57:49.971 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.971 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.971 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.972 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Container image name, without the tag", 2024-09-19T08:57:49.972 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.972 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.972 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.972 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_elasticsearch": { 2024-09-19T08:57:49.972 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_elasticsearch", 2024-09-19T08:57:49.972 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.972 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.972 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.972 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T08:57:49.972 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.973 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.973 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.973 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "elasticsearch container image", 2024-09-19T08:57:49.973 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.973 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.973 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.973 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_grafana": { 2024-09-19T08:57:49.973 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_grafana", 2024-09-19T08:57:49.973 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.973 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.973 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.974 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T08:57:49.974 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.974 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.974 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.974 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Prometheus container image", 2024-09-19T08:57:49.974 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.974 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.974 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.974 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_haproxy": { 2024-09-19T08:57:49.974 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_haproxy", 2024-09-19T08:57:49.974 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.975 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.975 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.975 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T08:57:49.975 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.975 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.975 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.975 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "HAproxy container image", 2024-09-19T08:57:49.975 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.975 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.975 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.976 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_jaeger_agent": { 2024-09-19T08:57:49.976 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_jaeger_agent", 2024-09-19T08:57:49.976 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.976 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.976 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.976 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T08:57:49.976 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.976 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.976 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.976 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Jaeger agent container image", 2024-09-19T08:57:49.976 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.976 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.977 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.977 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_jaeger_collector": { 2024-09-19T08:57:49.977 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_jaeger_collector", 2024-09-19T08:57:49.977 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.977 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.977 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.977 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T08:57:49.977 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.977 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.977 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.978 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Jaeger collector container image", 2024-09-19T08:57:49.978 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.978 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.978 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.978 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_jaeger_query": { 2024-09-19T08:57:49.978 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_jaeger_query", 2024-09-19T08:57:49.978 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.978 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.978 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.978 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T08:57:49.979 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.979 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.979 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.979 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Jaeger query container image", 2024-09-19T08:57:49.979 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.979 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.979 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.979 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_keepalived": { 2024-09-19T08:57:49.979 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_keepalived", 2024-09-19T08:57:49.979 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.980 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.980 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.980 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T08:57:49.980 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.980 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.980 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.980 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Keepalived container image", 2024-09-19T08:57:49.980 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.980 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.980 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.981 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_loki": { 2024-09-19T08:57:49.981 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_loki", 2024-09-19T08:57:49.981 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.981 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.981 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.981 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T08:57:49.981 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.981 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.981 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.981 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Loki container image", 2024-09-19T08:57:49.982 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.982 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.982 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.982 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_node_exporter": { 2024-09-19T08:57:49.982 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_node_exporter", 2024-09-19T08:57:49.982 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.982 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.982 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.982 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T08:57:49.982 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.982 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.983 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.983 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Prometheus container image", 2024-09-19T08:57:49.983 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.983 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.983 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.983 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_nvmeof": { 2024-09-19T08:57:49.983 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_nvmeof", 2024-09-19T08:57:49.983 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.983 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.983 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.984 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T08:57:49.984 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.984 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.984 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.984 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Nvme-of container image", 2024-09-19T08:57:49.984 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.984 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.984 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.984 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_prometheus": { 2024-09-19T08:57:49.984 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_prometheus", 2024-09-19T08:57:49.984 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.985 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.985 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.985 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T08:57:49.985 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.985 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.985 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.985 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Prometheus container image", 2024-09-19T08:57:49.985 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.985 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.985 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.986 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_promtail": { 2024-09-19T08:57:49.986 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_promtail", 2024-09-19T08:57:49.986 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.986 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.986 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.986 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T08:57:49.986 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.986 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.986 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.986 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Promtail container image", 2024-09-19T08:57:49.986 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.987 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.987 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.987 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_samba": { 2024-09-19T08:57:49.987 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_samba", 2024-09-19T08:57:49.987 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.987 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.987 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.987 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T08:57:49.987 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.987 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.988 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.988 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Samba/SMB container image", 2024-09-19T08:57:49.988 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.988 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.988 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.988 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_snmp_gateway": { 2024-09-19T08:57:49.988 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_snmp_gateway", 2024-09-19T08:57:49.988 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.988 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.988 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.989 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T08:57:49.989 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.989 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.989 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.989 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "SNMP Gateway container image", 2024-09-19T08:57:49.989 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.989 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.989 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.989 INFO:teuthology.orchestra.run.smithi164.stdout: "container_init": { 2024-09-19T08:57:49.989 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_init", 2024-09-19T08:57:49.989 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.990 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.990 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.990 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:49.990 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.990 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.990 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.990 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T08:57:49.990 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.990 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.990 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.991 INFO:teuthology.orchestra.run.smithi164.stdout: "daemon_cache_timeout": { 2024-09-19T08:57:49.991 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "daemon_cache_timeout", 2024-09-19T08:57:49.991 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:49.991 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.991 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.991 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "600", 2024-09-19T08:57:49.991 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.991 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.991 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.991 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T08:57:49.991 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.992 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.992 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.992 INFO:teuthology.orchestra.run.smithi164.stdout: "default_cephadm_command_timeout": { 2024-09-19T08:57:49.992 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T08:57:49.992 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:49.992 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.992 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.992 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "900", 2024-09-19T08:57:49.992 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.992 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.993 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.993 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T08:57:49.993 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.993 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.993 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.993 INFO:teuthology.orchestra.run.smithi164.stdout: "default_registry": { 2024-09-19T08:57:49.993 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "default_registry", 2024-09-19T08:57:49.993 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.993 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.994 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.994 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "docker.io", 2024-09-19T08:57:49.994 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.994 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.994 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.994 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T08:57:49.994 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.994 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.994 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.994 INFO:teuthology.orchestra.run.smithi164.stdout: "device_cache_timeout": { 2024-09-19T08:57:49.995 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "device_cache_timeout", 2024-09-19T08:57:49.995 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:49.995 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.995 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.995 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "1800", 2024-09-19T08:57:49.995 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.995 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.995 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.995 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "seconds to cache device inventory", 2024-09-19T08:57:49.995 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.995 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.996 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.996 INFO:teuthology.orchestra.run.smithi164.stdout: "device_enhanced_scan": { 2024-09-19T08:57:49.996 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "device_enhanced_scan", 2024-09-19T08:57:49.996 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:49.996 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.996 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.996 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:49.996 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.996 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.996 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.997 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T08:57:49.997 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.997 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.997 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.997 INFO:teuthology.orchestra.run.smithi164.stdout: "facts_cache_timeout": { 2024-09-19T08:57:49.997 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "facts_cache_timeout", 2024-09-19T08:57:49.997 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:49.997 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.997 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.997 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "60", 2024-09-19T08:57:49.998 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.998 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.998 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.998 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "seconds to cache host facts data", 2024-09-19T08:57:49.998 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.998 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.998 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:49.998 INFO:teuthology.orchestra.run.smithi164.stdout: "grafana_dashboards_path": { 2024-09-19T08:57:49.998 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "grafana_dashboards_path", 2024-09-19T08:57:49.998 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:49.999 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:49.999 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:49.999 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T08:57:49.999 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:49.999 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:49.999 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:49.999 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T08:57:49.999 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:49.999 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:49.999 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:49.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.000 INFO:teuthology.orchestra.run.smithi164.stdout: "host_check_interval": { 2024-09-19T08:57:50.000 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "host_check_interval", 2024-09-19T08:57:50.000 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:50.000 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.000 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.000 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "600", 2024-09-19T08:57:50.000 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.000 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.000 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.000 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "how frequently to perform a host check", 2024-09-19T08:57:50.001 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.001 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.001 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.001 INFO:teuthology.orchestra.run.smithi164.stdout: "hw_monitoring": { 2024-09-19T08:57:50.001 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "hw_monitoring", 2024-09-19T08:57:50.001 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.001 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.001 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.001 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.001 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.001 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.002 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.002 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T08:57:50.002 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.002 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.002 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.002 INFO:teuthology.orchestra.run.smithi164.stdout: "inventory_list_all": { 2024-09-19T08:57:50.002 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "inventory_list_all", 2024-09-19T08:57:50.002 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.002 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.002 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.003 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.003 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.003 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.003 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.003 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T08:57:50.003 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.003 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.003 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.003 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.003 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.004 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.004 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.004 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.004 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.004 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.004 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.004 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.004 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.004 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.004 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.004 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.005 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.005 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.005 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.005 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.005 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.005 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.005 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.005 INFO:teuthology.orchestra.run.smithi164.stdout: "log_refresh_metadata": { 2024-09-19T08:57:50.005 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_refresh_metadata", 2024-09-19T08:57:50.005 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.006 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.006 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.006 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.006 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.006 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.006 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.006 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T08:57:50.006 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.006 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.006 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.007 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.007 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.007 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.007 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.007 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.007 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.007 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.007 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.007 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.007 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T08:57:50.007 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.008 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.008 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.008 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.008 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.008 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.008 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.008 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.008 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.008 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.009 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.009 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.009 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.009 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.009 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.009 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.009 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.009 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.009 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.009 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.009 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.010 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.010 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.010 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.010 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.010 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.010 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.010 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.010 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.010 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.010 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.011 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.011 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.011 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.011 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.011 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.011 INFO:teuthology.orchestra.run.smithi164.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T08:57:50.011 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T08:57:50.011 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.011 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.012 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.012 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.012 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.012 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.012 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.012 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T08:57:50.012 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.012 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.012 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.012 INFO:teuthology.orchestra.run.smithi164.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T08:57:50.013 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T08:57:50.013 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.013 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.013 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.013 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "*", 2024-09-19T08:57:50.013 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.013 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.013 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.013 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T08:57:50.013 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.014 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.014 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.014 INFO:teuthology.orchestra.run.smithi164.stdout: "max_count_per_host": { 2024-09-19T08:57:50.014 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "max_count_per_host", 2024-09-19T08:57:50.014 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.014 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.014 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.014 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "10", 2024-09-19T08:57:50.014 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.014 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.015 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.015 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "max number of daemons per service per host", 2024-09-19T08:57:50.015 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.015 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.015 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.015 INFO:teuthology.orchestra.run.smithi164.stdout: "max_osd_draining_count": { 2024-09-19T08:57:50.015 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "max_osd_draining_count", 2024-09-19T08:57:50.015 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.015 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.015 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.016 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "10", 2024-09-19T08:57:50.016 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.016 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.016 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.016 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T08:57:50.016 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.016 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.016 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.016 INFO:teuthology.orchestra.run.smithi164.stdout: "migration_current": { 2024-09-19T08:57:50.017 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "migration_current", 2024-09-19T08:57:50.017 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.017 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.017 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.017 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.017 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.017 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.017 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.017 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "internal - do not modify", 2024-09-19T08:57:50.017 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.017 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.018 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.018 INFO:teuthology.orchestra.run.smithi164.stdout: "mode": { 2024-09-19T08:57:50.018 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "mode", 2024-09-19T08:57:50.018 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.018 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.018 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.018 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "root", 2024-09-19T08:57:50.018 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.018 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.018 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.019 INFO:teuthology.orchestra.run.smithi164.stdout: "cephadm-package", 2024-09-19T08:57:50.019 INFO:teuthology.orchestra.run.smithi164.stdout: "root" 2024-09-19T08:57:50.019 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.019 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T08:57:50.019 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.019 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.019 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.019 INFO:teuthology.orchestra.run.smithi164.stdout: "oob_default_addr": { 2024-09-19T08:57:50.019 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "oob_default_addr", 2024-09-19T08:57:50.019 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.020 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.020 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.020 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "169.254.1.1", 2024-09-19T08:57:50.020 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.020 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.020 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.020 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T08:57:50.020 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.020 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.020 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.020 INFO:teuthology.orchestra.run.smithi164.stdout: "prometheus_alerts_path": { 2024-09-19T08:57:50.021 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "prometheus_alerts_path", 2024-09-19T08:57:50.021 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.021 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.021 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.021 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T08:57:50.021 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.021 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.021 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.021 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T08:57:50.021 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.021 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.022 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.022 INFO:teuthology.orchestra.run.smithi164.stdout: "registry_insecure": { 2024-09-19T08:57:50.022 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "registry_insecure", 2024-09-19T08:57:50.022 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.022 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.022 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.022 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.022 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.022 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.022 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.023 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T08:57:50.023 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.023 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.023 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.023 INFO:teuthology.orchestra.run.smithi164.stdout: "registry_password": { 2024-09-19T08:57:50.023 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "registry_password", 2024-09-19T08:57:50.023 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.023 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.023 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.023 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.023 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.024 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.024 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.024 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T08:57:50.024 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.024 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.024 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.024 INFO:teuthology.orchestra.run.smithi164.stdout: "registry_url": { 2024-09-19T08:57:50.024 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "registry_url", 2024-09-19T08:57:50.024 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.024 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.025 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.025 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.025 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.025 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.025 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.025 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T08:57:50.025 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.025 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.025 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.026 INFO:teuthology.orchestra.run.smithi164.stdout: "registry_username": { 2024-09-19T08:57:50.026 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "registry_username", 2024-09-19T08:57:50.026 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.026 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.026 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.026 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.026 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.026 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.026 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.026 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T08:57:50.026 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.026 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.027 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.027 INFO:teuthology.orchestra.run.smithi164.stdout: "secure_monitoring_stack": { 2024-09-19T08:57:50.027 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "secure_monitoring_stack", 2024-09-19T08:57:50.027 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.027 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.027 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.027 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.027 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.027 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.028 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.028 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T08:57:50.028 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.028 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.028 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.028 INFO:teuthology.orchestra.run.smithi164.stdout: "service_discovery_port": { 2024-09-19T08:57:50.028 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "service_discovery_port", 2024-09-19T08:57:50.028 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.028 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.029 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.029 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "8765", 2024-09-19T08:57:50.029 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.029 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.029 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.029 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "cephadm service discovery port", 2024-09-19T08:57:50.029 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.029 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.029 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.029 INFO:teuthology.orchestra.run.smithi164.stdout: "ssh_config_file": { 2024-09-19T08:57:50.029 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "ssh_config_file", 2024-09-19T08:57:50.030 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.030 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.030 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.030 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.030 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.030 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.030 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.030 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T08:57:50.030 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.030 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.030 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.031 INFO:teuthology.orchestra.run.smithi164.stdout: "use_agent": { 2024-09-19T08:57:50.031 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "use_agent", 2024-09-19T08:57:50.031 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.031 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.031 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.031 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.031 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.031 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.031 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.031 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T08:57:50.032 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.032 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.032 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.032 INFO:teuthology.orchestra.run.smithi164.stdout: "use_repo_digest": { 2024-09-19T08:57:50.032 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "use_repo_digest", 2024-09-19T08:57:50.032 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.032 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.032 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.032 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.032 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.032 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.033 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.033 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T08:57:50.033 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.033 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.033 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.033 INFO:teuthology.orchestra.run.smithi164.stdout: "warn_on_failed_host_check": { 2024-09-19T08:57:50.033 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "warn_on_failed_host_check", 2024-09-19T08:57:50.033 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.033 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.034 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.034 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.034 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.034 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.034 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.034 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T08:57:50.034 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.034 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.034 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.034 INFO:teuthology.orchestra.run.smithi164.stdout: "warn_on_stray_daemons": { 2024-09-19T08:57:50.035 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "warn_on_stray_daemons", 2024-09-19T08:57:50.035 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.035 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.035 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.035 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.035 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.035 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.035 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.035 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T08:57:50.035 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.035 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.036 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.036 INFO:teuthology.orchestra.run.smithi164.stdout: "warn_on_stray_hosts": { 2024-09-19T08:57:50.036 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "warn_on_stray_hosts", 2024-09-19T08:57:50.036 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.036 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.036 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.036 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.036 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.036 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.037 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.037 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T08:57:50.037 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.037 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.037 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.037 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.037 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.037 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "crash", 2024-09-19T08:57:50.038 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.038 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.038 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.038 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.038 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.038 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.038 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.038 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.038 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.038 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.038 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.039 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.039 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.039 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.039 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.039 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.039 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.039 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.039 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.039 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.039 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.039 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.040 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.040 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.040 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.040 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.040 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.040 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.040 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.040 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.040 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.040 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.041 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.041 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.041 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.041 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.041 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.041 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.041 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.041 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.042 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.042 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.042 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.042 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.042 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.042 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.042 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.042 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.042 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.042 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.042 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.042 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.043 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.043 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.043 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.043 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.043 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.043 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.043 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.043 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.043 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.043 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.044 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.044 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.044 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.044 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.044 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.044 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.044 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.044 INFO:teuthology.orchestra.run.smithi164.stdout: "retain_interval": { 2024-09-19T08:57:50.044 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "retain_interval", 2024-09-19T08:57:50.044 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:50.045 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.045 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.045 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "31536000", 2024-09-19T08:57:50.045 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.045 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.045 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.045 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T08:57:50.045 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.045 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.045 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.046 INFO:teuthology.orchestra.run.smithi164.stdout: "warn_recent_interval": { 2024-09-19T08:57:50.046 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "warn_recent_interval", 2024-09-19T08:57:50.046 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:50.046 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.046 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.046 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "1209600", 2024-09-19T08:57:50.046 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.046 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.046 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.046 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T08:57:50.047 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.047 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.047 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.047 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.047 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.047 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "dashboard", 2024-09-19T08:57:50.047 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.047 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.047 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.048 INFO:teuthology.orchestra.run.smithi164.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T08:57:50.048 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T08:57:50.048 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.048 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.048 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.048 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "10", 2024-09-19T08:57:50.048 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.048 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.048 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.048 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.048 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.049 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.049 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.049 INFO:teuthology.orchestra.run.smithi164.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T08:57:50.049 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T08:57:50.049 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.049 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.049 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.049 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.049 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.049 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.049 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.050 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.050 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.050 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.050 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.050 INFO:teuthology.orchestra.run.smithi164.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T08:57:50.050 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T08:57:50.050 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.050 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.050 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.051 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.051 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.051 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.051 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.051 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.051 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.051 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.051 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.051 INFO:teuthology.orchestra.run.smithi164.stdout: "AUDIT_API_ENABLED": { 2024-09-19T08:57:50.051 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T08:57:50.051 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.052 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.052 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.052 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.052 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.052 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.052 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.052 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.052 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.052 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.052 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.053 INFO:teuthology.orchestra.run.smithi164.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T08:57:50.053 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T08:57:50.053 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.053 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.053 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.053 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.053 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.053 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.053 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.053 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.053 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.054 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.054 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.054 INFO:teuthology.orchestra.run.smithi164.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T08:57:50.054 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T08:57:50.054 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.054 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.054 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.054 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.054 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.054 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.055 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.055 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.055 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.055 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.055 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.055 INFO:teuthology.orchestra.run.smithi164.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T08:57:50.055 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T08:57:50.055 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.055 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.055 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.055 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.056 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.056 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.056 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.056 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.056 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.056 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.056 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.056 INFO:teuthology.orchestra.run.smithi164.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T08:57:50.056 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T08:57:50.056 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.057 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.057 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.057 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.057 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.057 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.057 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.057 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.057 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.057 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.057 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.058 INFO:teuthology.orchestra.run.smithi164.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T08:57:50.058 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T08:57:50.058 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.058 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.058 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.058 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.058 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.058 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.058 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.058 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.058 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.059 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.059 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.059 INFO:teuthology.orchestra.run.smithi164.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T08:57:50.059 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T08:57:50.059 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.059 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.059 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.059 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.059 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.059 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.060 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.060 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.060 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.060 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.060 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.060 INFO:teuthology.orchestra.run.smithi164.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T08:57:50.060 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T08:57:50.060 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.060 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.060 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.061 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.061 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.061 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.061 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.061 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.061 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.061 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.061 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.061 INFO:teuthology.orchestra.run.smithi164.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T08:57:50.061 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T08:57:50.062 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.062 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.062 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.062 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.062 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.062 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.062 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.062 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.062 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.062 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.062 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.063 INFO:teuthology.orchestra.run.smithi164.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T08:57:50.063 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T08:57:50.063 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.063 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.063 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.063 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.063 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.063 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.063 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.063 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.064 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.064 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.064 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.064 INFO:teuthology.orchestra.run.smithi164.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T08:57:50.064 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T08:57:50.064 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.064 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.064 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.064 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.064 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.065 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.065 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.065 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.065 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.065 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.065 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.065 INFO:teuthology.orchestra.run.smithi164.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T08:57:50.065 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T08:57:50.065 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.065 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.066 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.066 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "admin", 2024-09-19T08:57:50.066 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.066 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.066 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.066 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.066 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.066 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.066 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.066 INFO:teuthology.orchestra.run.smithi164.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T08:57:50.067 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T08:57:50.067 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.067 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.067 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.067 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.067 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.067 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.067 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.067 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.067 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.067 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.068 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.068 INFO:teuthology.orchestra.run.smithi164.stdout: "GRAFANA_API_URL": { 2024-09-19T08:57:50.068 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "GRAFANA_API_URL", 2024-09-19T08:57:50.068 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.068 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.068 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.068 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.068 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.068 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.068 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.068 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.069 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.069 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.069 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.069 INFO:teuthology.orchestra.run.smithi164.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T08:57:50.069 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T08:57:50.069 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.069 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.069 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.069 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "admin", 2024-09-19T08:57:50.069 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.070 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.070 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.070 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.070 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.070 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.070 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.070 INFO:teuthology.orchestra.run.smithi164.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T08:57:50.070 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T08:57:50.070 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.070 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.070 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.071 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.071 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.071 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.071 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.071 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.071 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.071 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.071 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.071 INFO:teuthology.orchestra.run.smithi164.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T08:57:50.072 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T08:57:50.072 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.072 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.072 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.072 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.072 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.072 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.072 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.072 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.072 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.072 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.072 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.073 INFO:teuthology.orchestra.run.smithi164.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T08:57:50.073 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T08:57:50.073 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.073 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.073 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.073 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.073 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.073 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.073 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.074 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.074 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.074 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.074 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.074 INFO:teuthology.orchestra.run.smithi164.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T08:57:50.074 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T08:57:50.074 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.074 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.074 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.074 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.074 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.075 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.075 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.075 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.075 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.075 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.075 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.075 INFO:teuthology.orchestra.run.smithi164.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T08:57:50.075 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T08:57:50.075 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.075 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.075 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.076 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.076 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.076 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.076 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.076 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.076 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.076 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.076 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.076 INFO:teuthology.orchestra.run.smithi164.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T08:57:50.076 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T08:57:50.077 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.077 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.077 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.077 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.077 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.077 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.077 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.077 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.077 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.077 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.077 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.078 INFO:teuthology.orchestra.run.smithi164.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T08:57:50.078 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T08:57:50.078 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.078 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.078 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.078 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.078 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.078 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.078 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.078 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.078 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.079 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.079 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.079 INFO:teuthology.orchestra.run.smithi164.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T08:57:50.079 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T08:57:50.079 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.079 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.079 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.079 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.079 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.079 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.080 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.080 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.080 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.080 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.080 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.080 INFO:teuthology.orchestra.run.smithi164.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T08:57:50.080 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T08:57:50.080 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.080 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.080 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.080 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.081 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.081 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.081 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.081 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.081 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.081 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.081 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.081 INFO:teuthology.orchestra.run.smithi164.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T08:57:50.081 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T08:57:50.081 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.082 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.082 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.082 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.082 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.082 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.082 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.082 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.082 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.082 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.082 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.082 INFO:teuthology.orchestra.run.smithi164.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T08:57:50.083 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T08:57:50.083 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.083 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.083 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.083 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.083 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.083 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.083 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.083 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.083 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.083 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.083 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.084 INFO:teuthology.orchestra.run.smithi164.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T08:57:50.084 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T08:57:50.084 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.084 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.084 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.084 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.084 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.084 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.084 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.085 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.085 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.085 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.085 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.085 INFO:teuthology.orchestra.run.smithi164.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T08:57:50.085 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T08:57:50.085 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.085 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.085 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.086 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.086 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.086 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.086 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.086 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.086 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.086 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.086 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.086 INFO:teuthology.orchestra.run.smithi164.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T08:57:50.086 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T08:57:50.087 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.087 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.087 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.087 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.087 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.087 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.087 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.087 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.087 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.087 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.087 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.088 INFO:teuthology.orchestra.run.smithi164.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T08:57:50.088 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T08:57:50.088 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.088 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.088 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.088 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T08:57:50.088 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.088 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.088 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.088 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.088 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.089 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.089 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.089 INFO:teuthology.orchestra.run.smithi164.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T08:57:50.089 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T08:57:50.089 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.089 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.089 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.089 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "10", 2024-09-19T08:57:50.089 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.090 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.090 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.090 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.090 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.090 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.090 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.090 INFO:teuthology.orchestra.run.smithi164.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T08:57:50.090 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T08:57:50.090 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.090 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.091 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.091 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "8", 2024-09-19T08:57:50.091 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.091 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.091 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.091 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.091 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.091 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.091 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.091 INFO:teuthology.orchestra.run.smithi164.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T08:57:50.092 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T08:57:50.092 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.092 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.092 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.092 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "45", 2024-09-19T08:57:50.092 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.092 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.092 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.092 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.092 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.092 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.093 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.093 INFO:teuthology.orchestra.run.smithi164.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T08:57:50.093 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T08:57:50.093 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.093 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.093 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.093 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.093 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.093 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.093 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.094 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.094 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.094 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.094 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.094 INFO:teuthology.orchestra.run.smithi164.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T08:57:50.094 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T08:57:50.094 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.094 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.094 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.094 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "admin", 2024-09-19T08:57:50.095 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.095 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.095 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.095 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.095 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.095 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.095 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.095 INFO:teuthology.orchestra.run.smithi164.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T08:57:50.095 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T08:57:50.095 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.095 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.096 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.096 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.096 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.096 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.096 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.096 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.096 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.096 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.096 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.097 INFO:teuthology.orchestra.run.smithi164.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T08:57:50.097 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T08:57:50.097 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.097 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.097 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.097 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.097 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.097 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.097 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.097 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.097 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.098 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.098 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.098 INFO:teuthology.orchestra.run.smithi164.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T08:57:50.098 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T08:57:50.098 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.098 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.098 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.098 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "0", 2024-09-19T08:57:50.098 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.098 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.099 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.099 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.099 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.099 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.099 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.099 INFO:teuthology.orchestra.run.smithi164.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T08:57:50.099 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T08:57:50.099 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.099 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.099 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.099 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "10", 2024-09-19T08:57:50.100 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.100 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.100 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.100 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.100 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.100 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.100 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.100 INFO:teuthology.orchestra.run.smithi164.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T08:57:50.100 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T08:57:50.100 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.101 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.101 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.101 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "5", 2024-09-19T08:57:50.101 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.101 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.101 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.101 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.101 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.101 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.101 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.102 INFO:teuthology.orchestra.run.smithi164.stdout: "cross_origin_url": { 2024-09-19T08:57:50.102 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "cross_origin_url", 2024-09-19T08:57:50.102 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.102 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.102 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.102 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.102 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.102 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.102 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.102 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.102 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.102 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.103 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.103 INFO:teuthology.orchestra.run.smithi164.stdout: "crt_file": { 2024-09-19T08:57:50.103 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "crt_file", 2024-09-19T08:57:50.103 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.103 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.103 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.103 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.103 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.103 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.103 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.103 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.104 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.104 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.105 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.105 INFO:teuthology.orchestra.run.smithi164.stdout: "debug": { 2024-09-19T08:57:50.105 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "debug", 2024-09-19T08:57:50.105 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.105 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.105 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.105 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.106 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.106 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.106 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.106 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Enable/disable debug options", 2024-09-19T08:57:50.106 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.106 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.106 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.106 INFO:teuthology.orchestra.run.smithi164.stdout: "jwt_token_ttl": { 2024-09-19T08:57:50.106 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "jwt_token_ttl", 2024-09-19T08:57:50.106 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.107 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.107 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.107 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "28800", 2024-09-19T08:57:50.107 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.107 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.107 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.107 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.107 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.107 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.107 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.107 INFO:teuthology.orchestra.run.smithi164.stdout: "key_file": { 2024-09-19T08:57:50.108 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "key_file", 2024-09-19T08:57:50.108 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.108 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.108 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.108 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.108 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.108 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.108 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.108 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.108 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.108 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.109 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.109 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.109 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.109 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.109 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.109 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.109 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.109 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.109 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.109 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.109 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.110 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.110 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.110 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.110 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.110 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.110 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.110 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.110 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.110 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.110 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.110 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.111 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.111 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.111 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.111 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.111 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.111 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.111 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.111 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.111 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.111 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.111 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.112 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.112 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.112 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.112 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.112 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.112 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.112 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.112 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.112 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.112 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.112 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.113 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.113 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.113 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.113 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.113 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.113 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.113 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.113 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.113 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.113 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.114 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.114 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.114 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.114 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.114 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.114 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.114 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.114 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.114 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.114 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.114 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.115 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.115 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.115 INFO:teuthology.orchestra.run.smithi164.stdout: "motd": { 2024-09-19T08:57:50.115 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "motd", 2024-09-19T08:57:50.115 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.115 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.115 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.115 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.115 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "The message of the day", 2024-09-19T08:57:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: "redirect_resolve_ip_addr": { 2024-09-19T08:57:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T08:57:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.118 INFO:teuthology.orchestra.run.smithi164.stdout: "server_addr": { 2024-09-19T08:57:50.118 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "server_addr", 2024-09-19T08:57:50.118 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.118 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.118 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.118 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "::", 2024-09-19T08:57:50.118 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.118 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.118 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.118 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.118 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: "server_port": { 2024-09-19T08:57:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "server_port", 2024-09-19T08:57:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "8080", 2024-09-19T08:57:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.120 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.120 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.120 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.120 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.120 INFO:teuthology.orchestra.run.smithi164.stdout: "ssl": { 2024-09-19T08:57:50.120 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "ssl", 2024-09-19T08:57:50.120 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.120 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.120 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.120 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.120 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.121 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.121 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.121 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.121 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.121 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.121 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.121 INFO:teuthology.orchestra.run.smithi164.stdout: "ssl_server_port": { 2024-09-19T08:57:50.121 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "ssl_server_port", 2024-09-19T08:57:50.121 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.122 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.122 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.122 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "8443", 2024-09-19T08:57:50.122 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.122 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.122 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.122 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.122 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.122 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.122 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.122 INFO:teuthology.orchestra.run.smithi164.stdout: "standby_behaviour": { 2024-09-19T08:57:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "standby_behaviour", 2024-09-19T08:57:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "redirect", 2024-09-19T08:57:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: "redirect" 2024-09-19T08:57:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: "standby_error_status_code": { 2024-09-19T08:57:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "standby_error_status_code", 2024-09-19T08:57:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "500", 2024-09-19T08:57:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "400", 2024-09-19T08:57:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "599", 2024-09-19T08:57:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: "url_prefix": { 2024-09-19T08:57:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "url_prefix", 2024-09-19T08:57:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "devicehealth", 2024-09-19T08:57:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: "enable_monitoring": { 2024-09-19T08:57:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "enable_monitoring", 2024-09-19T08:57:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "monitor device health metrics", 2024-09-19T08:57:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.132 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.132 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.132 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.132 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.132 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.132 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.132 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.132 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.132 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.132 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.132 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.132 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.133 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.133 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.133 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.133 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.133 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.133 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.133 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.133 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.133 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.133 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.134 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.134 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.134 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.134 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.134 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.134 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.134 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.134 INFO:teuthology.orchestra.run.smithi164.stdout: "mark_out_threshold": { 2024-09-19T08:57:50.134 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "mark_out_threshold", 2024-09-19T08:57:50.134 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:50.134 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.135 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.135 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "2419200", 2024-09-19T08:57:50.135 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.135 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.135 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.135 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T08:57:50.135 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.135 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.135 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.135 INFO:teuthology.orchestra.run.smithi164.stdout: "pool_name": { 2024-09-19T08:57:50.136 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "pool_name", 2024-09-19T08:57:50.136 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.136 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.136 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.136 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "device_health_metrics", 2024-09-19T08:57:50.136 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.136 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.136 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.136 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T08:57:50.136 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.136 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.137 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.137 INFO:teuthology.orchestra.run.smithi164.stdout: "retention_period": { 2024-09-19T08:57:50.137 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "retention_period", 2024-09-19T08:57:50.137 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:50.137 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.137 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.137 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "15552000", 2024-09-19T08:57:50.137 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.137 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.137 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.137 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "how long to retain device health metrics", 2024-09-19T08:57:50.138 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.138 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.138 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.138 INFO:teuthology.orchestra.run.smithi164.stdout: "scrape_frequency": { 2024-09-19T08:57:50.138 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "scrape_frequency", 2024-09-19T08:57:50.138 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:50.138 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.138 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.138 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "86400", 2024-09-19T08:57:50.139 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.139 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.139 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.139 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T08:57:50.139 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.139 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.139 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.139 INFO:teuthology.orchestra.run.smithi164.stdout: "self_heal": { 2024-09-19T08:57:50.139 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "self_heal", 2024-09-19T08:57:50.140 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.140 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.140 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.140 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.140 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.140 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.140 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.140 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T08:57:50.140 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.140 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.141 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.141 INFO:teuthology.orchestra.run.smithi164.stdout: "sleep_interval": { 2024-09-19T08:57:50.141 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "sleep_interval", 2024-09-19T08:57:50.141 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:50.141 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.141 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.141 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "600", 2024-09-19T08:57:50.141 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.141 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.141 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.142 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T08:57:50.142 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.142 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.142 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.142 INFO:teuthology.orchestra.run.smithi164.stdout: "warn_threshold": { 2024-09-19T08:57:50.142 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "warn_threshold", 2024-09-19T08:57:50.142 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:50.142 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.142 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.142 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "7257600", 2024-09-19T08:57:50.143 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.143 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.143 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.143 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T08:57:50.143 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.143 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.143 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.143 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.143 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.144 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "diskprediction_local", 2024-09-19T08:57:50.144 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.144 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.144 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.144 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.144 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.144 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.144 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.144 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.144 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.144 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.145 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.145 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.145 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.145 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.145 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.145 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.145 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.145 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.145 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.145 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.145 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.146 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.146 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.146 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.146 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.146 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.146 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.146 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.146 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.146 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.147 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.147 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.147 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.147 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.147 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.147 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.147 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.147 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.147 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.147 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.147 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.148 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.148 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.148 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.148 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.148 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.148 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.148 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.148 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.148 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.148 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.149 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.149 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.149 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.149 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.149 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.149 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.149 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.149 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.149 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.149 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.149 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.150 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.150 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.150 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.150 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.150 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.150 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.150 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.150 INFO:teuthology.orchestra.run.smithi164.stdout: "predict_interval": { 2024-09-19T08:57:50.150 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "predict_interval", 2024-09-19T08:57:50.150 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.151 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.151 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.151 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "86400", 2024-09-19T08:57:50.151 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.151 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.151 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.151 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.151 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.151 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.151 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.152 INFO:teuthology.orchestra.run.smithi164.stdout: "predictor_model": { 2024-09-19T08:57:50.152 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "predictor_model", 2024-09-19T08:57:50.152 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.152 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.152 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.152 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "prophetstor", 2024-09-19T08:57:50.152 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.152 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.152 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.152 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.152 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.153 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.153 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.153 INFO:teuthology.orchestra.run.smithi164.stdout: "sleep_interval": { 2024-09-19T08:57:50.153 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "sleep_interval", 2024-09-19T08:57:50.153 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.153 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.153 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.153 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "600", 2024-09-19T08:57:50.153 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.153 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.154 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.154 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.154 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.154 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.154 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.154 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.154 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.154 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "influx", 2024-09-19T08:57:50.154 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": false, 2024-09-19T08:57:50.154 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "influxdb python module not found", 2024-09-19T08:57:50.155 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.155 INFO:teuthology.orchestra.run.smithi164.stdout: "batch_size": { 2024-09-19T08:57:50.155 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "batch_size", 2024-09-19T08:57:50.155 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.155 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.155 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.155 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "5000", 2024-09-19T08:57:50.155 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.155 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.155 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.156 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T08:57:50.156 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.156 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.156 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.156 INFO:teuthology.orchestra.run.smithi164.stdout: "database": { 2024-09-19T08:57:50.156 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "database", 2024-09-19T08:57:50.156 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.156 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.156 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.157 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "ceph", 2024-09-19T08:57:50.157 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.157 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.157 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.157 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T08:57:50.157 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.157 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.157 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.157 INFO:teuthology.orchestra.run.smithi164.stdout: "hostname": { 2024-09-19T08:57:50.157 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "hostname", 2024-09-19T08:57:50.158 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.158 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.158 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.158 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.158 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.158 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.158 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.158 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "InfluxDB server hostname", 2024-09-19T08:57:50.158 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.158 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.158 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.159 INFO:teuthology.orchestra.run.smithi164.stdout: "interval": { 2024-09-19T08:57:50.159 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "interval", 2024-09-19T08:57:50.159 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:50.159 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.159 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.159 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "30", 2024-09-19T08:57:50.159 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "5", 2024-09-19T08:57:50.159 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.159 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.159 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T08:57:50.160 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.160 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.160 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.160 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.160 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.160 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.160 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.160 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.160 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.161 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.161 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.161 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.161 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.161 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.161 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.161 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.161 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.161 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.161 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.161 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.162 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.162 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.162 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.162 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.162 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.162 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.162 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.162 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.162 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.162 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: "password": { 2024-09-19T08:57:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "password", 2024-09-19T08:57:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "password of InfluxDB server user", 2024-09-19T08:57:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: "port": { 2024-09-19T08:57:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "port", 2024-09-19T08:57:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "8086", 2024-09-19T08:57:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "InfluxDB server port", 2024-09-19T08:57:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: "ssl": { 2024-09-19T08:57:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "ssl", 2024-09-19T08:57:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "false", 2024-09-19T08:57:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T08:57:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: "threads": { 2024-09-19T08:57:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "threads", 2024-09-19T08:57:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "5", 2024-09-19T08:57:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "1", 2024-09-19T08:57:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "32", 2024-09-19T08:57:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T08:57:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: "username": { 2024-09-19T08:57:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "username", 2024-09-19T08:57:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "username of InfluxDB server user", 2024-09-19T08:57:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.173 INFO:teuthology.orchestra.run.smithi164.stdout: "verify_ssl": { 2024-09-19T08:57:50.173 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "verify_ssl", 2024-09-19T08:57:50.173 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.173 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.173 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.173 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "true", 2024-09-19T08:57:50.173 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.173 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.173 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.173 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T08:57:50.173 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.173 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.174 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.174 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.174 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.174 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "insights", 2024-09-19T08:57:50.174 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.174 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.174 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.174 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.174 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.174 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.174 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.177 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.177 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.177 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.177 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.177 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.177 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.177 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.177 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.177 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.177 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.177 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.179 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.179 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.179 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.179 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.179 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.179 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.179 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.179 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.179 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.179 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.179 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "iostat", 2024-09-19T08:57:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.184 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.184 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.184 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.184 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.184 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.184 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.184 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.184 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.184 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.184 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.184 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.185 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.185 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.185 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.185 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.185 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.185 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.185 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.185 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.185 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.185 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.185 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.186 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.186 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.186 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.186 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.186 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.186 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.186 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.186 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.186 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.186 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.186 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.186 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.187 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.187 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "k8sevents", 2024-09-19T08:57:50.187 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.187 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.187 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.187 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_event_retention_days": { 2024-09-19T08:57:50.187 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "ceph_event_retention_days", 2024-09-19T08:57:50.187 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.187 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.187 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.188 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "7", 2024-09-19T08:57:50.188 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.188 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.188 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.188 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T08:57:50.188 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.188 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.188 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.188 INFO:teuthology.orchestra.run.smithi164.stdout: "config_check_secs": { 2024-09-19T08:57:50.188 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "config_check_secs", 2024-09-19T08:57:50.189 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.189 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.189 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.189 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "10", 2024-09-19T08:57:50.189 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "10", 2024-09-19T08:57:50.189 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.189 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.189 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T08:57:50.189 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.189 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.189 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.190 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.190 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.190 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.190 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.190 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.190 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.190 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.190 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.190 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.190 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.190 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.191 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.191 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.191 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.191 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.191 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.191 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.191 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.191 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.191 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.191 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.192 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.192 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.192 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.192 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.192 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.192 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.192 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.192 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.192 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.192 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.192 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.193 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.193 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.193 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.193 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.193 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.193 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.193 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.193 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.193 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.193 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.193 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.194 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.194 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.194 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.194 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.194 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.194 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.194 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.194 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.194 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.194 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.195 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.195 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.195 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.195 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.195 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.195 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.195 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.195 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.195 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.195 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.195 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.195 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.196 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.196 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.196 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.196 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "localpool", 2024-09-19T08:57:50.196 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.196 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.196 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.196 INFO:teuthology.orchestra.run.smithi164.stdout: "failure_domain": { 2024-09-19T08:57:50.196 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "failure_domain", 2024-09-19T08:57:50.197 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.197 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.197 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.197 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "host", 2024-09-19T08:57:50.197 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.197 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.197 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.197 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "failure domain for any created local pool", 2024-09-19T08:57:50.197 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T08:57:50.197 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.197 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.198 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.198 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.198 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.198 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.198 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.198 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.198 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.198 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.198 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.198 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.198 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.199 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.199 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.199 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.199 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.199 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.199 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.199 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.199 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.199 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.199 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.199 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.200 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.200 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.200 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.200 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.200 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.200 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.200 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.200 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.200 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.200 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.200 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.201 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.201 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.201 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.201 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.201 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.201 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.201 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.201 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.201 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.201 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.201 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.202 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.202 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.202 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.202 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.202 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.202 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.202 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.202 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.202 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.202 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.203 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.203 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.203 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.203 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.203 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.203 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.203 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.203 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.203 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.203 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.203 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.203 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.204 INFO:teuthology.orchestra.run.smithi164.stdout: "min_size": { 2024-09-19T08:57:50.204 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "min_size", 2024-09-19T08:57:50.204 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.204 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.204 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.204 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.204 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.204 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.204 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.204 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "default min_size for any created local pool", 2024-09-19T08:57:50.205 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T08:57:50.205 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.205 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.205 INFO:teuthology.orchestra.run.smithi164.stdout: "num_rep": { 2024-09-19T08:57:50.205 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "num_rep", 2024-09-19T08:57:50.205 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.205 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.206 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.206 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "3", 2024-09-19T08:57:50.206 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.206 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.206 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.206 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "default replica count for any created local pool", 2024-09-19T08:57:50.206 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.206 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.206 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.206 INFO:teuthology.orchestra.run.smithi164.stdout: "pg_num": { 2024-09-19T08:57:50.207 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "pg_num", 2024-09-19T08:57:50.207 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.207 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.207 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.207 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "128", 2024-09-19T08:57:50.207 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.207 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.207 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.207 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T08:57:50.207 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.207 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.208 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.208 INFO:teuthology.orchestra.run.smithi164.stdout: "prefix": { 2024-09-19T08:57:50.208 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "prefix", 2024-09-19T08:57:50.208 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.208 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.208 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.208 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.208 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.208 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.208 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.209 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "name prefix for any created local pool", 2024-09-19T08:57:50.209 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.209 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.209 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.209 INFO:teuthology.orchestra.run.smithi164.stdout: "subtree": { 2024-09-19T08:57:50.209 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "subtree", 2024-09-19T08:57:50.209 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.209 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.209 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.209 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "rack", 2024-09-19T08:57:50.209 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.210 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.210 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.210 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T08:57:50.210 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T08:57:50.210 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.210 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.210 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.210 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.210 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "mds_autoscaler", 2024-09-19T08:57:50.211 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.211 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.211 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.211 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.211 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.211 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.211 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.211 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.211 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.211 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.212 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.212 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.212 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.212 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.212 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.212 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.212 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.212 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.212 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.212 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.212 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.213 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.213 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.213 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.213 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.213 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.213 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.213 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.213 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.213 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.213 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.214 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.214 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.214 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.214 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.214 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.214 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.214 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.214 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.214 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.214 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.215 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.215 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.215 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.215 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.215 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.215 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.215 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.215 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.215 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.215 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.215 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.215 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.216 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.216 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.216 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.216 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.216 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.216 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.216 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.216 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.216 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.217 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.217 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.217 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.217 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.217 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.217 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.217 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.217 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.217 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.218 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "mirroring", 2024-09-19T08:57:50.218 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.218 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.218 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.218 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.218 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.218 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.218 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.218 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.218 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.218 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.218 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.218 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.219 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.219 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.219 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.219 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.219 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.219 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.219 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.219 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.219 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.219 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.219 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.220 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.220 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.220 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.220 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.220 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.220 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.220 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.220 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.220 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.220 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.221 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.221 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.221 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.221 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.221 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.221 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.221 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.221 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.221 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.221 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.222 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.222 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.222 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.222 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.222 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.222 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.222 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.222 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.222 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.222 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.222 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.223 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.223 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.223 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.223 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.223 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.223 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.223 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.223 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.223 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.223 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.224 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.224 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.224 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.224 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.224 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.224 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.224 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.224 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "nfs", 2024-09-19T08:57:50.224 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.225 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.225 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.225 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.225 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.225 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.225 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.225 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.225 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.225 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.225 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.226 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.226 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.226 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.226 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.226 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.226 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.226 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.226 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.226 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.226 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.226 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.227 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.227 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.227 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.227 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.227 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.227 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.227 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.227 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.227 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.227 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.227 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.228 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.228 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.228 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.228 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.228 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.228 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.228 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.228 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.228 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.229 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.229 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.229 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.229 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.229 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.229 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.229 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.229 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.229 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.229 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.229 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.230 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.230 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.230 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.230 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.230 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.230 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.230 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.230 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.230 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.230 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.231 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.231 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.231 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.231 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.231 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.231 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.231 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.231 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.231 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "orchestrator", 2024-09-19T08:57:50.232 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.232 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.232 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.232 INFO:teuthology.orchestra.run.smithi164.stdout: "fail_fs": { 2024-09-19T08:57:50.232 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "fail_fs", 2024-09-19T08:57:50.232 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.232 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.232 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.232 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.232 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.232 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.233 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.233 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T08:57:50.233 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.233 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.233 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.233 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.233 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.233 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.233 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.233 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.234 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.234 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.234 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.234 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.234 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.234 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.234 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.234 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.234 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.234 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.234 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.234 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.235 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.235 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.235 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.235 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.235 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.235 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.235 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.235 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.235 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.235 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.236 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.236 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.236 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.236 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.236 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.236 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.236 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.236 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.236 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.236 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.237 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.237 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.237 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.237 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.237 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.237 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.237 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.237 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.237 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.237 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.237 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.238 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.238 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.238 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.238 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.238 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.238 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.238 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.238 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.238 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.238 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.238 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.239 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.239 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.239 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.239 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.239 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.239 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.239 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.239 INFO:teuthology.orchestra.run.smithi164.stdout: "orchestrator": { 2024-09-19T08:57:50.239 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "orchestrator", 2024-09-19T08:57:50.239 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.240 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.240 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.240 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.240 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.240 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.240 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.240 INFO:teuthology.orchestra.run.smithi164.stdout: "cephadm", 2024-09-19T08:57:50.240 INFO:teuthology.orchestra.run.smithi164.stdout: "rook", 2024-09-19T08:57:50.240 INFO:teuthology.orchestra.run.smithi164.stdout: "test_orchestrator" 2024-09-19T08:57:50.240 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.240 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Orchestrator backend", 2024-09-19T08:57:50.241 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.241 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.241 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.241 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.241 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.241 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "osd_perf_query", 2024-09-19T08:57:50.241 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.241 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.241 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.242 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.242 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.242 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.242 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.242 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.242 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.242 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.242 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.242 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.242 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.242 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.243 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.243 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.243 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.243 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.243 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.243 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.243 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.243 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.243 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.243 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.243 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.244 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.244 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.244 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.244 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.244 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.244 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.244 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.244 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.244 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.244 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.244 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.245 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.245 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.245 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.245 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.245 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.245 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.245 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.245 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.245 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.245 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.246 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.246 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.246 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.246 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.246 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.246 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.246 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.246 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.246 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.246 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.247 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.247 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.247 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.247 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.247 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.247 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.247 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.247 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.247 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.247 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.247 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.248 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.248 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.248 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.248 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.248 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "osd_support", 2024-09-19T08:57:50.248 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.248 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.248 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.248 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.248 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.249 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.249 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.249 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.249 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.249 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.249 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.249 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.249 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.249 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.249 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.250 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.250 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.250 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.250 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.250 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.250 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.250 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.250 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.250 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.250 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.251 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.251 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.251 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.251 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.251 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.251 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.251 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.251 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.251 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.251 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.251 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.252 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.252 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.252 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.252 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.252 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.252 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.252 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.252 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.252 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.252 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.253 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.253 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.253 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.253 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.253 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.253 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.253 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.253 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.253 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.253 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.254 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.254 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.254 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.254 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.254 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.254 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.254 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.254 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.254 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.254 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.254 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.255 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.255 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.255 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.255 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.255 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "pg_autoscaler", 2024-09-19T08:57:50.255 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.255 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.255 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.255 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.255 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.256 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.256 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.256 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.256 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.256 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.256 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.256 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.256 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.256 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.256 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.256 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.256 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.257 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.257 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.257 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.257 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.257 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.257 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.257 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.257 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.257 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.258 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.258 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.258 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.258 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.258 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.258 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.258 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.258 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.258 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.258 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.258 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.259 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.259 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.259 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.259 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.259 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.259 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.259 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.259 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.259 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.259 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.259 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.259 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.260 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.260 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.260 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.260 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.260 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.260 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.260 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.260 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.260 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.260 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.260 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.260 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.261 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.261 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.261 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.261 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.261 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.261 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.261 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.261 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.261 INFO:teuthology.orchestra.run.smithi164.stdout: "sleep_interval": { 2024-09-19T08:57:50.261 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "sleep_interval", 2024-09-19T08:57:50.261 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:50.262 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.262 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.262 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "60", 2024-09-19T08:57:50.262 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.262 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.262 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.262 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.262 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.262 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.262 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.262 INFO:teuthology.orchestra.run.smithi164.stdout: "threshold": { 2024-09-19T08:57:50.263 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "threshold", 2024-09-19T08:57:50.263 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "float", 2024-09-19T08:57:50.263 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.263 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.263 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "3.0", 2024-09-19T08:57:50.263 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "1.0", 2024-09-19T08:57:50.263 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.263 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.263 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "scaling threshold", 2024-09-19T08:57:50.263 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T08:57:50.263 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.263 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.264 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.264 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.264 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "progress", 2024-09-19T08:57:50.264 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.264 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.264 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.264 INFO:teuthology.orchestra.run.smithi164.stdout: "allow_pg_recovery_event": { 2024-09-19T08:57:50.264 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "allow_pg_recovery_event", 2024-09-19T08:57:50.264 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.265 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.265 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.265 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.265 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.265 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.265 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.265 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T08:57:50.265 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.265 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.265 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.266 INFO:teuthology.orchestra.run.smithi164.stdout: "enabled": { 2024-09-19T08:57:50.266 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "enabled", 2024-09-19T08:57:50.266 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.266 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.266 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.266 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.266 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.266 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.266 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.266 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.266 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.267 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.267 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.267 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.267 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.267 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.267 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.267 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.267 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.267 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.267 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.267 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.268 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.268 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.268 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.268 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.268 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.268 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.268 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.268 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.268 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.268 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.269 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.269 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.269 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.269 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.269 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.269 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.269 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.269 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.269 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.269 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.269 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.270 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.270 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.270 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.270 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.270 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.270 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.270 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.270 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.270 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.270 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.271 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.271 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.271 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.271 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.271 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.271 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.271 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.271 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.271 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.271 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.271 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.271 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.272 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.272 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.272 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.272 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.272 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.272 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.272 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.272 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.272 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.272 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.272 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.273 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.273 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.273 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.273 INFO:teuthology.orchestra.run.smithi164.stdout: "max_completed_events": { 2024-09-19T08:57:50.273 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "max_completed_events", 2024-09-19T08:57:50.273 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.273 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.273 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.273 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "50", 2024-09-19T08:57:50.273 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.274 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.274 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.274 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "number of past completed events to remember", 2024-09-19T08:57:50.274 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.274 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.274 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.274 INFO:teuthology.orchestra.run.smithi164.stdout: "sleep_interval": { 2024-09-19T08:57:50.274 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "sleep_interval", 2024-09-19T08:57:50.274 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:50.274 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.274 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.275 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "5", 2024-09-19T08:57:50.275 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.275 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.275 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.275 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "how long the module is going to sleep", 2024-09-19T08:57:50.275 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.275 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.275 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.275 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.275 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.276 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "prometheus", 2024-09-19T08:57:50.276 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.276 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.276 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.276 INFO:teuthology.orchestra.run.smithi164.stdout: "cache": { 2024-09-19T08:57:50.276 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "cache", 2024-09-19T08:57:50.276 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.276 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.276 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.276 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.277 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.277 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.277 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.277 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.277 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.277 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.277 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.277 INFO:teuthology.orchestra.run.smithi164.stdout: "exclude_perf_counters": { 2024-09-19T08:57:50.277 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "exclude_perf_counters", 2024-09-19T08:57:50.277 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.278 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.278 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.278 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.278 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.278 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.278 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.278 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T08:57:50.278 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T08:57:50.278 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.278 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.279 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.279 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.279 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.279 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.279 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.279 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.279 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.279 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.279 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.279 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.279 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.280 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.280 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.280 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.280 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.280 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.280 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.280 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.280 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.280 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.280 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.281 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.281 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.281 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.281 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.281 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.281 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.281 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.281 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.281 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.281 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.282 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.282 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.282 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.282 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.282 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.282 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.282 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.282 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.282 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.282 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.282 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.283 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.283 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.283 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.283 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.283 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.283 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.283 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.283 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.283 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.283 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.283 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.284 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.284 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.284 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.284 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.284 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.284 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.284 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.284 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.284 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.284 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.284 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.285 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.285 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.285 INFO:teuthology.orchestra.run.smithi164.stdout: "rbd_stats_pools": { 2024-09-19T08:57:50.285 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rbd_stats_pools", 2024-09-19T08:57:50.285 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.285 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.285 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.285 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.285 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.285 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.286 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.286 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.286 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.286 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.286 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.286 INFO:teuthology.orchestra.run.smithi164.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T08:57:50.286 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T08:57:50.286 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.286 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.287 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.287 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "300", 2024-09-19T08:57:50.287 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.287 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.287 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.287 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.287 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.287 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.287 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.287 INFO:teuthology.orchestra.run.smithi164.stdout: "scrape_interval": { 2024-09-19T08:57:50.288 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "scrape_interval", 2024-09-19T08:57:50.288 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "float", 2024-09-19T08:57:50.288 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.288 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.288 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "15.0", 2024-09-19T08:57:50.288 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.288 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.288 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.288 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.288 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.288 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.289 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.289 INFO:teuthology.orchestra.run.smithi164.stdout: "server_addr": { 2024-09-19T08:57:50.289 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "server_addr", 2024-09-19T08:57:50.289 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.289 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.289 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.289 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "::", 2024-09-19T08:57:50.289 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.289 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.289 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.289 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T08:57:50.290 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.290 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.290 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.290 INFO:teuthology.orchestra.run.smithi164.stdout: "server_port": { 2024-09-19T08:57:50.290 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "server_port", 2024-09-19T08:57:50.290 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.290 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.290 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.290 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "9283", 2024-09-19T08:57:50.291 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.291 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.291 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.291 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T08:57:50.291 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.291 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.291 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.291 INFO:teuthology.orchestra.run.smithi164.stdout: "stale_cache_strategy": { 2024-09-19T08:57:50.291 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "stale_cache_strategy", 2024-09-19T08:57:50.292 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.292 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.292 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.292 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "log", 2024-09-19T08:57:50.292 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.292 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.292 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.292 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.292 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.292 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.292 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.293 INFO:teuthology.orchestra.run.smithi164.stdout: "standby_behaviour": { 2024-09-19T08:57:50.293 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "standby_behaviour", 2024-09-19T08:57:50.293 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.293 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.293 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.293 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "default", 2024-09-19T08:57:50.293 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.293 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.293 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.293 INFO:teuthology.orchestra.run.smithi164.stdout: "default", 2024-09-19T08:57:50.293 INFO:teuthology.orchestra.run.smithi164.stdout: "error" 2024-09-19T08:57:50.294 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.294 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.294 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.294 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.294 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.294 INFO:teuthology.orchestra.run.smithi164.stdout: "standby_error_status_code": { 2024-09-19T08:57:50.294 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "standby_error_status_code", 2024-09-19T08:57:50.294 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.294 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.294 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.294 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "500", 2024-09-19T08:57:50.294 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "400", 2024-09-19T08:57:50.295 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "599", 2024-09-19T08:57:50.295 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.295 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.295 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.295 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.295 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.295 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.295 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.295 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rbd_support", 2024-09-19T08:57:50.295 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.296 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.296 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.296 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.296 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.296 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.296 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.296 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.296 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.296 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.296 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.296 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.297 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.297 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.297 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.297 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.297 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.297 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.297 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.297 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.297 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.297 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.297 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.298 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.298 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.298 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.298 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.298 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.298 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.298 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.298 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.298 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.298 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.298 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.298 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.299 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.299 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.299 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.299 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.299 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.299 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.299 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.299 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.299 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.299 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.299 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.300 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.300 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.300 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.300 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.300 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.300 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.300 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.300 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.300 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.300 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.300 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.300 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.301 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.301 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.301 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.301 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.301 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.301 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.301 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.301 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.301 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.301 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.302 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.302 INFO:teuthology.orchestra.run.smithi164.stdout: "max_concurrent_snap_create": { 2024-09-19T08:57:50.302 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "max_concurrent_snap_create", 2024-09-19T08:57:50.302 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.302 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.302 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.302 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "10", 2024-09-19T08:57:50.302 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.302 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.302 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.302 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.303 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.303 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.303 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.303 INFO:teuthology.orchestra.run.smithi164.stdout: "mirror_snapshot_schedule": { 2024-09-19T08:57:50.303 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T08:57:50.303 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.303 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.303 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.303 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.303 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.304 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.304 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.304 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.304 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.304 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.304 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.304 INFO:teuthology.orchestra.run.smithi164.stdout: "trash_purge_schedule": { 2024-09-19T08:57:50.304 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "trash_purge_schedule", 2024-09-19T08:57:50.304 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.304 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.304 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.305 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.309 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.309 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.309 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.309 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.309 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.309 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.310 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.310 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.310 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.310 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "restful", 2024-09-19T08:57:50.310 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.310 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.310 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.310 INFO:teuthology.orchestra.run.smithi164.stdout: "enable_auth": { 2024-09-19T08:57:50.310 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "enable_auth", 2024-09-19T08:57:50.310 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.310 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.310 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.311 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.311 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.311 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.311 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.311 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.311 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.311 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.311 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.311 INFO:teuthology.orchestra.run.smithi164.stdout: "key_file": { 2024-09-19T08:57:50.311 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "key_file", 2024-09-19T08:57:50.311 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.311 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.312 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.312 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.312 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.312 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.312 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.312 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.312 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.312 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.312 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.312 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.312 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.313 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.313 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.313 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.313 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.313 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.313 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.313 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.313 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.313 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.313 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.313 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.313 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.313 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.314 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.314 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.314 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.314 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.314 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.314 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.314 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.314 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.314 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.314 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.314 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.314 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.314 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.315 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.315 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.315 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.315 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.315 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.315 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.315 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.315 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.315 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.315 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.315 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.315 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.315 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.316 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.316 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.316 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.316 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.316 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.316 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.316 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.316 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.316 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.316 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.316 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.316 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.317 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.317 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.317 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.317 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.317 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.317 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.317 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.317 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.317 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.317 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.317 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.317 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.317 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.318 INFO:teuthology.orchestra.run.smithi164.stdout: "server_addr": { 2024-09-19T08:57:50.318 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "server_addr", 2024-09-19T08:57:50.318 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.318 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.318 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.318 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.318 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.318 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.318 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.318 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.318 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.318 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.319 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.319 INFO:teuthology.orchestra.run.smithi164.stdout: "server_port": { 2024-09-19T08:57:50.319 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "server_port", 2024-09-19T08:57:50.319 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.319 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.319 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.319 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.319 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.319 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.319 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.319 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.319 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.319 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.320 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.320 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.320 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.320 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rgw", 2024-09-19T08:57:50.320 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.320 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.320 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.320 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.320 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.320 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.320 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.320 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.321 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.321 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.321 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.321 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.321 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.321 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.321 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.321 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.321 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.321 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.321 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.321 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.321 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.321 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.322 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.322 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.322 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.322 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.322 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.322 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.322 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.322 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.322 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.322 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.322 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.322 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.322 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.323 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.323 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.323 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.323 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.323 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.323 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.323 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.323 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.323 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.323 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.323 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.323 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.324 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.324 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.324 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.324 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.324 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.324 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.324 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.324 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.324 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.324 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.324 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.324 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.325 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.325 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.325 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.325 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.325 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.325 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.325 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.325 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.325 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.325 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.325 INFO:teuthology.orchestra.run.smithi164.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T08:57:50.325 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T08:57:50.325 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.326 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.326 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.326 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "5", 2024-09-19T08:57:50.326 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.326 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.326 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.326 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T08:57:50.326 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.326 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.326 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.326 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.326 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.327 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rook", 2024-09-19T08:57:50.327 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.327 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.327 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.327 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.327 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.327 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.327 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.327 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.327 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.327 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.327 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.328 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.328 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.328 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.328 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.328 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.328 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.328 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.328 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.328 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.328 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.328 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.328 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.329 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.329 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.329 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.329 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.329 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.329 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.329 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.329 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.329 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.329 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.329 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.329 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.329 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.330 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.330 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.330 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.330 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.330 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.330 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.330 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.330 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.330 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.330 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.330 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.330 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.331 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.331 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.331 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.331 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.331 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.331 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.331 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.331 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.331 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.331 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.331 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.331 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.331 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.332 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.332 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.332 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.332 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.332 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.332 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.332 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.332 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.332 INFO:teuthology.orchestra.run.smithi164.stdout: "storage_class": { 2024-09-19T08:57:50.332 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "storage_class", 2024-09-19T08:57:50.332 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.332 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.333 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.333 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "local", 2024-09-19T08:57:50.333 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.333 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.333 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.333 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T08:57:50.333 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.333 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.333 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.333 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.333 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.333 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "selftest", 2024-09-19T08:57:50.334 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.334 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.334 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.334 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.334 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.334 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.334 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.334 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.334 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.334 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.334 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.334 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.334 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.335 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.335 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.335 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.335 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.335 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.335 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.335 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.335 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.335 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.335 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.335 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.335 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.336 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.336 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.336 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.336 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.336 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.336 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.336 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.336 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.336 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.336 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.336 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.336 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.336 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.337 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.337 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.337 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.337 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.337 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.337 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.337 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.337 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.337 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.337 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.337 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.337 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.337 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.338 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.338 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.338 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.338 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.338 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.338 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.338 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.338 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.338 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.338 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.338 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.338 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.338 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.339 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.339 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.339 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.339 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.339 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.339 INFO:teuthology.orchestra.run.smithi164.stdout: "roption1": { 2024-09-19T08:57:50.339 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "roption1", 2024-09-19T08:57:50.339 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.339 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.339 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.339 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.339 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.339 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.340 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.340 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.340 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.340 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.340 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.340 INFO:teuthology.orchestra.run.smithi164.stdout: "roption2": { 2024-09-19T08:57:50.340 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "roption2", 2024-09-19T08:57:50.340 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.340 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.340 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.340 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "xyz", 2024-09-19T08:57:50.341 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.341 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.341 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.341 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.341 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.341 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.341 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.341 INFO:teuthology.orchestra.run.smithi164.stdout: "rwoption1": { 2024-09-19T08:57:50.341 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rwoption1", 2024-09-19T08:57:50.341 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.341 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.341 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.341 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: "rwoption2": { 2024-09-19T08:57:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rwoption2", 2024-09-19T08:57:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: "rwoption3": { 2024-09-19T08:57:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rwoption3", 2024-09-19T08:57:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "float", 2024-09-19T08:57:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: "rwoption4": { 2024-09-19T08:57:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rwoption4", 2024-09-19T08:57:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: "rwoption5": { 2024-09-19T08:57:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rwoption5", 2024-09-19T08:57:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: "rwoption6": { 2024-09-19T08:57:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rwoption6", 2024-09-19T08:57:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: "rwoption7": { 2024-09-19T08:57:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rwoption7", 2024-09-19T08:57:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "1", 2024-09-19T08:57:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "42", 2024-09-19T08:57:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: "testkey": { 2024-09-19T08:57:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "testkey", 2024-09-19T08:57:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: "testlkey": { 2024-09-19T08:57:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "testlkey", 2024-09-19T08:57:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: "testnewline": { 2024-09-19T08:57:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "testnewline", 2024-09-19T08:57:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "snap_schedule", 2024-09-19T08:57:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: "allow_m_granularity": { 2024-09-19T08:57:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "allow_m_granularity", 2024-09-19T08:57:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T08:57:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: "dump_on_update": { 2024-09-19T08:57:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "dump_on_update", 2024-09-19T08:57:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "dump database to debug log on update", 2024-09-19T08:57:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "stats", 2024-09-19T08:57:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.363 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.363 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.363 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.363 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.363 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.363 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.363 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.363 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.363 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.363 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.363 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.363 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.363 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.363 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.364 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.364 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.364 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.364 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.364 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.364 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.364 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.364 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.364 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.364 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.364 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.364 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "status", 2024-09-19T08:57:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "telegraf", 2024-09-19T08:57:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: "address": { 2024-09-19T08:57:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "address", 2024-09-19T08:57:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T08:57:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: "interval": { 2024-09-19T08:57:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "interval", 2024-09-19T08:57:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "15", 2024-09-19T08:57:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "telemetry", 2024-09-19T08:57:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: "channel_basic": { 2024-09-19T08:57:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "channel_basic", 2024-09-19T08:57:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T08:57:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: "channel_crash": { 2024-09-19T08:57:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "channel_crash", 2024-09-19T08:57:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T08:57:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: "channel_device": { 2024-09-19T08:57:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "channel_device", 2024-09-19T08:57:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T08:57:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: "channel_ident": { 2024-09-19T08:57:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "channel_ident", 2024-09-19T08:57:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T08:57:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: "channel_perf": { 2024-09-19T08:57:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "channel_perf", 2024-09-19T08:57:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T08:57:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: "contact": { 2024-09-19T08:57:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "contact", 2024-09-19T08:57:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: "description": { 2024-09-19T08:57:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "description", 2024-09-19T08:57:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: "device_url": { 2024-09-19T08:57:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "device_url", 2024-09-19T08:57:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T08:57:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: "enabled": { 2024-09-19T08:57:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "enabled", 2024-09-19T08:57:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: "interval": { 2024-09-19T08:57:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "interval", 2024-09-19T08:57:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "24", 2024-09-19T08:57:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "8", 2024-09-19T08:57:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: "last_opt_revision": { 2024-09-19T08:57:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "last_opt_revision", 2024-09-19T08:57:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "1", 2024-09-19T08:57:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: "leaderboard": { 2024-09-19T08:57:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "leaderboard", 2024-09-19T08:57:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: "leaderboard_description": { 2024-09-19T08:57:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "leaderboard_description", 2024-09-19T08:57:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.396 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.396 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.396 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.396 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.396 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.396 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.396 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.396 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.396 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.396 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.396 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.396 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.397 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.397 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.397 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.397 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.397 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.397 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.397 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.397 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.397 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.397 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.397 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.397 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.397 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.398 INFO:teuthology.orchestra.run.smithi164.stdout: "organization": { 2024-09-19T08:57:50.398 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "organization", 2024-09-19T08:57:50.398 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.398 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.398 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.398 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.398 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.398 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.398 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.398 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.398 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.398 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.398 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.399 INFO:teuthology.orchestra.run.smithi164.stdout: "proxy": { 2024-09-19T08:57:50.399 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "proxy", 2024-09-19T08:57:50.399 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.399 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.399 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.399 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.399 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.399 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.399 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.399 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.399 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.399 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.399 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.400 INFO:teuthology.orchestra.run.smithi164.stdout: "url": { 2024-09-19T08:57:50.400 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "url", 2024-09-19T08:57:50.400 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.400 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.400 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.400 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T08:57:50.400 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.400 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.400 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.400 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.400 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.400 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.400 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.401 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.401 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.401 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "test_orchestrator", 2024-09-19T08:57:50.401 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.401 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.401 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.401 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.401 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.401 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.401 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.401 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.402 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.402 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.402 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.402 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.402 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.402 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.402 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.402 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.402 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.402 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.402 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.402 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.402 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.402 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.403 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.403 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.403 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.403 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.403 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.403 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.403 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.403 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.403 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.403 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.403 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.403 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.403 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.404 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.404 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.404 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.404 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.404 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.404 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.404 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.404 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.404 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.404 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.404 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.405 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.405 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.405 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.405 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.405 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.405 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.405 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.405 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.405 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.405 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.405 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.405 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.405 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.406 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.406 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.406 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.406 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.406 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.406 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.406 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.406 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.406 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.406 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.406 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.406 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.407 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "volumes", 2024-09-19T08:57:50.407 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.407 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.407 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.407 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.407 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.407 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.407 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.407 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.407 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.407 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.407 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.407 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.407 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.408 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.408 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.408 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.408 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.408 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.408 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.408 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.408 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.408 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.408 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.408 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.408 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.409 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.409 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.409 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.409 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.409 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.409 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.409 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.409 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.409 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.409 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.409 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.409 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.409 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.410 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.410 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.410 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.410 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.410 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.410 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.410 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.410 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.410 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.410 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.410 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.410 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.410 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.410 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.411 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.411 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.411 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.411 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.411 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.411 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.411 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.411 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.411 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.411 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.411 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.411 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.411 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.412 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.412 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.412 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.412 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.412 INFO:teuthology.orchestra.run.smithi164.stdout: "max_concurrent_clones": { 2024-09-19T08:57:50.412 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "max_concurrent_clones", 2024-09-19T08:57:50.412 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.412 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.412 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.412 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "4", 2024-09-19T08:57:50.412 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.413 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.413 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.413 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T08:57:50.413 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.413 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.413 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.413 INFO:teuthology.orchestra.run.smithi164.stdout: "periodic_async_work": { 2024-09-19T08:57:50.413 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "periodic_async_work", 2024-09-19T08:57:50.413 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.413 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.413 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.413 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.413 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.414 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.414 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.414 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Periodically check for async work", 2024-09-19T08:57:50.414 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.414 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.414 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.414 INFO:teuthology.orchestra.run.smithi164.stdout: "snapshot_clone_delay": { 2024-09-19T08:57:50.414 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "snapshot_clone_delay", 2024-09-19T08:57:50.414 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.414 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.414 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.414 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "0", 2024-09-19T08:57:50.414 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.415 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.415 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.415 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T08:57:50.415 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.415 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.415 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.415 INFO:teuthology.orchestra.run.smithi164.stdout: "snapshot_clone_no_wait": { 2024-09-19T08:57:50.415 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T08:57:50.415 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.415 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.415 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.415 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.416 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.416 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.416 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.416 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T08:57:50.416 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.416 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.416 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.416 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.416 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.416 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "zabbix", 2024-09-19T08:57:50.416 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.417 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.417 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.417 INFO:teuthology.orchestra.run.smithi164.stdout: "discovery_interval": { 2024-09-19T08:57:50.417 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "discovery_interval", 2024-09-19T08:57:50.417 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "uint", 2024-09-19T08:57:50.417 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.417 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.417 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "100", 2024-09-19T08:57:50.417 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.417 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.417 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.417 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.417 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.418 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.418 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.418 INFO:teuthology.orchestra.run.smithi164.stdout: "identifier": { 2024-09-19T08:57:50.418 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "identifier", 2024-09-19T08:57:50.418 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.418 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.418 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.418 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.418 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.418 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.418 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.418 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.418 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.419 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.419 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.419 INFO:teuthology.orchestra.run.smithi164.stdout: "interval": { 2024-09-19T08:57:50.419 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "interval", 2024-09-19T08:57:50.419 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:50.419 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.419 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.419 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "60", 2024-09-19T08:57:50.419 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.419 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.419 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.419 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.419 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.419 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.420 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.420 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.420 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.420 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.420 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.420 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.420 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.420 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.420 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.420 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.420 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.420 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.420 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.421 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.421 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.421 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.421 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.421 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.421 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.421 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.421 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.421 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.421 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.421 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.421 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.421 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.422 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.422 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.422 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.422 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.422 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.422 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.422 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.422 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.422 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.422 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.422 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.422 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.422 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.423 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.423 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.423 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.423 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.423 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.423 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.423 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.423 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.423 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.423 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.423 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.423 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.423 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.423 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.424 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.424 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.424 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.424 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.424 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.424 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.424 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.424 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.424 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.424 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.424 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.424 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.425 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.425 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.425 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.425 INFO:teuthology.orchestra.run.smithi164.stdout: "zabbix_host": { 2024-09-19T08:57:50.425 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "zabbix_host", 2024-09-19T08:57:50.425 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.425 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.425 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.425 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.425 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.425 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.425 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.425 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.426 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.426 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.426 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.426 INFO:teuthology.orchestra.run.smithi164.stdout: "zabbix_port": { 2024-09-19T08:57:50.426 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "zabbix_port", 2024-09-19T08:57:50.426 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.426 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.426 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.426 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "10051", 2024-09-19T08:57:50.426 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.426 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.426 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.427 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.427 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.427 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.427 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.427 INFO:teuthology.orchestra.run.smithi164.stdout: "zabbix_sender": { 2024-09-19T08:57:50.427 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "zabbix_sender", 2024-09-19T08:57:50.427 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.427 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.427 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.427 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T08:57:50.427 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.427 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.427 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.428 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.428 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.428 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.428 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.428 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.428 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.428 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.428 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-19T08:57:50.428 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.428 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.428 INFO:teuthology.orchestra.run.smithi164.stdout: "modules": [ 2024-09-19T08:57:50.428 INFO:teuthology.orchestra.run.smithi164.stdout: "cephadm", 2024-09-19T08:57:50.429 INFO:teuthology.orchestra.run.smithi164.stdout: "dashboard", 2024-09-19T08:57:50.429 INFO:teuthology.orchestra.run.smithi164.stdout: "iostat", 2024-09-19T08:57:50.429 INFO:teuthology.orchestra.run.smithi164.stdout: "nfs", 2024-09-19T08:57:50.429 INFO:teuthology.orchestra.run.smithi164.stdout: "restful", 2024-09-19T08:57:50.429 INFO:teuthology.orchestra.run.smithi164.stdout: "snap_schedule" 2024-09-19T08:57:50.429 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.429 INFO:teuthology.orchestra.run.smithi164.stdout: "available_modules": [ 2024-09-19T08:57:50.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.429 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "alerts", 2024-09-19T08:57:50.429 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.429 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.429 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.429 INFO:teuthology.orchestra.run.smithi164.stdout: "interval": { 2024-09-19T08:57:50.430 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "interval", 2024-09-19T08:57:50.430 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:50.430 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.430 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.430 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "60", 2024-09-19T08:57:50.430 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.430 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.430 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.430 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T08:57:50.430 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.430 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.430 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.430 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.431 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.431 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.431 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.431 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.431 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.431 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.431 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.431 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.431 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.431 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.431 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.431 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.431 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.432 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.432 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.432 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.432 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.432 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.432 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.432 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.432 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.432 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.432 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.432 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.432 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.433 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.433 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.433 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.433 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.433 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.433 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.433 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.433 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.433 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.433 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.433 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.433 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.434 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.434 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.434 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.434 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.434 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.434 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.434 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.434 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.434 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.434 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.434 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.434 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.434 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.434 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.435 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.435 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.435 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.435 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.435 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.435 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.435 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.435 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.435 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.435 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.435 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.435 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.435 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.436 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.436 INFO:teuthology.orchestra.run.smithi164.stdout: "smtp_destination": { 2024-09-19T08:57:50.436 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "smtp_destination", 2024-09-19T08:57:50.436 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.436 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.436 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.436 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.436 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.436 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.436 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.436 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Email address to send alerts to", 2024-09-19T08:57:50.436 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.436 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.437 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.437 INFO:teuthology.orchestra.run.smithi164.stdout: "smtp_from_name": { 2024-09-19T08:57:50.437 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "smtp_from_name", 2024-09-19T08:57:50.437 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.437 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.437 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.437 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "Ceph", 2024-09-19T08:57:50.437 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.437 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.437 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.437 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Email From: name", 2024-09-19T08:57:50.437 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.438 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.438 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.438 INFO:teuthology.orchestra.run.smithi164.stdout: "smtp_host": { 2024-09-19T08:57:50.438 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "smtp_host", 2024-09-19T08:57:50.438 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.438 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.438 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.438 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.438 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.438 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.438 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.438 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "SMTP server", 2024-09-19T08:57:50.438 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.439 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.439 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.439 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.439 INFO:teuthology.orchestra.run.smithi164.stdout: "smtp_password": { 2024-09-19T08:57:50.439 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "smtp_password", 2024-09-19T08:57:50.439 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.439 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.439 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.439 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.439 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.439 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.439 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.439 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Password to authenticate with", 2024-09-19T08:57:50.439 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.440 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.440 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.440 INFO:teuthology.orchestra.run.smithi164.stdout: "smtp_port": { 2024-09-19T08:57:50.440 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "smtp_port", 2024-09-19T08:57:50.440 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.440 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.440 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.440 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "465", 2024-09-19T08:57:50.440 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.440 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.440 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.440 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "SMTP port", 2024-09-19T08:57:50.440 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.441 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.441 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.441 INFO:teuthology.orchestra.run.smithi164.stdout: "smtp_sender": { 2024-09-19T08:57:50.441 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "smtp_sender", 2024-09-19T08:57:50.441 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.441 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.441 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.441 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.441 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.441 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.441 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.441 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "SMTP envelope sender", 2024-09-19T08:57:50.442 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.442 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.442 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.442 INFO:teuthology.orchestra.run.smithi164.stdout: "smtp_ssl": { 2024-09-19T08:57:50.442 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "smtp_ssl", 2024-09-19T08:57:50.442 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.442 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.442 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.442 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.442 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.442 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.442 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.442 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T08:57:50.443 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.443 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.443 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.443 INFO:teuthology.orchestra.run.smithi164.stdout: "smtp_user": { 2024-09-19T08:57:50.443 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "smtp_user", 2024-09-19T08:57:50.443 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.443 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.443 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.443 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.443 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.443 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.443 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.443 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "User to authenticate as", 2024-09-19T08:57:50.444 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.444 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.444 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.444 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.444 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.444 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "balancer", 2024-09-19T08:57:50.444 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.444 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.444 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.444 INFO:teuthology.orchestra.run.smithi164.stdout: "active": { 2024-09-19T08:57:50.444 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "active", 2024-09-19T08:57:50.445 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.445 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.445 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.445 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.445 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.445 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.445 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.445 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T08:57:50.445 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.445 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.445 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.445 INFO:teuthology.orchestra.run.smithi164.stdout: "begin_time": { 2024-09-19T08:57:50.445 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "begin_time", 2024-09-19T08:57:50.446 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.446 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.446 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.446 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "0000", 2024-09-19T08:57:50.446 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.446 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.446 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.446 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T08:57:50.446 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T08:57:50.446 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.446 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.446 INFO:teuthology.orchestra.run.smithi164.stdout: "begin_weekday": { 2024-09-19T08:57:50.447 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "begin_weekday", 2024-09-19T08:57:50.447 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "uint", 2024-09-19T08:57:50.447 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.447 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.447 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "0", 2024-09-19T08:57:50.447 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "0", 2024-09-19T08:57:50.447 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "6", 2024-09-19T08:57:50.447 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.447 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T08:57:50.447 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T08:57:50.447 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.447 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.447 INFO:teuthology.orchestra.run.smithi164.stdout: "crush_compat_max_iterations": { 2024-09-19T08:57:50.448 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "crush_compat_max_iterations", 2024-09-19T08:57:50.448 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "uint", 2024-09-19T08:57:50.448 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.448 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.448 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "25", 2024-09-19T08:57:50.448 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "1", 2024-09-19T08:57:50.448 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "250", 2024-09-19T08:57:50.448 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.448 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T08:57:50.448 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.448 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.448 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.449 INFO:teuthology.orchestra.run.smithi164.stdout: "crush_compat_metrics": { 2024-09-19T08:57:50.449 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "crush_compat_metrics", 2024-09-19T08:57:50.449 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.449 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.449 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.449 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T08:57:50.449 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.449 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.449 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.449 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T08:57:50.449 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T08:57:50.449 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.449 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.450 INFO:teuthology.orchestra.run.smithi164.stdout: "crush_compat_step": { 2024-09-19T08:57:50.450 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "crush_compat_step", 2024-09-19T08:57:50.450 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "float", 2024-09-19T08:57:50.450 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.450 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.450 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "0.5", 2024-09-19T08:57:50.450 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "0.001", 2024-09-19T08:57:50.450 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "0.999", 2024-09-19T08:57:50.450 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.450 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "aggressiveness of optimization", 2024-09-19T08:57:50.450 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T08:57:50.450 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.450 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.451 INFO:teuthology.orchestra.run.smithi164.stdout: "end_time": { 2024-09-19T08:57:50.451 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "end_time", 2024-09-19T08:57:50.451 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.451 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.451 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.451 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "2359", 2024-09-19T08:57:50.451 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.451 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.451 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.451 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T08:57:50.451 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T08:57:50.451 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.451 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.452 INFO:teuthology.orchestra.run.smithi164.stdout: "end_weekday": { 2024-09-19T08:57:50.452 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "end_weekday", 2024-09-19T08:57:50.452 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "uint", 2024-09-19T08:57:50.452 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.452 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.452 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "0", 2024-09-19T08:57:50.452 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "0", 2024-09-19T08:57:50.452 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "6", 2024-09-19T08:57:50.452 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.452 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T08:57:50.452 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T08:57:50.452 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.452 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.453 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.453 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.453 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.453 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.453 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.453 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.453 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.453 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.453 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.453 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.453 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.453 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.454 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.454 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.454 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.454 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.454 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.454 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.454 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.454 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.454 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.454 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.454 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.454 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.454 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.455 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.455 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.455 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.455 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.455 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.455 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.455 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.455 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.455 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.455 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.455 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.455 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.455 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.456 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.456 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.456 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.456 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.456 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.456 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.456 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.456 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.456 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.456 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.456 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.456 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.456 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.457 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.457 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.457 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.457 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.457 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.457 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.457 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.457 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.457 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.457 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.457 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.457 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.457 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.458 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.458 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.458 INFO:teuthology.orchestra.run.smithi164.stdout: "min_score": { 2024-09-19T08:57:50.458 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "min_score", 2024-09-19T08:57:50.458 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "float", 2024-09-19T08:57:50.458 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.458 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.458 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "0", 2024-09-19T08:57:50.458 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.458 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.458 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.458 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T08:57:50.458 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.459 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.459 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.459 INFO:teuthology.orchestra.run.smithi164.stdout: "mode": { 2024-09-19T08:57:50.459 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "mode", 2024-09-19T08:57:50.459 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.459 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.459 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.459 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "upmap", 2024-09-19T08:57:50.459 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.459 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.459 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.459 INFO:teuthology.orchestra.run.smithi164.stdout: "crush-compat", 2024-09-19T08:57:50.459 INFO:teuthology.orchestra.run.smithi164.stdout: "none", 2024-09-19T08:57:50.460 INFO:teuthology.orchestra.run.smithi164.stdout: "read", 2024-09-19T08:57:50.460 INFO:teuthology.orchestra.run.smithi164.stdout: "upmap", 2024-09-19T08:57:50.460 INFO:teuthology.orchestra.run.smithi164.stdout: "upmap-read" 2024-09-19T08:57:50.460 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.460 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Balancer mode", 2024-09-19T08:57:50.460 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.460 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.460 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.460 INFO:teuthology.orchestra.run.smithi164.stdout: "pool_ids": { 2024-09-19T08:57:50.460 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "pool_ids", 2024-09-19T08:57:50.460 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.460 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.461 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.461 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.461 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.461 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.461 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.461 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T08:57:50.461 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.461 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.461 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.461 INFO:teuthology.orchestra.run.smithi164.stdout: "sleep_interval": { 2024-09-19T08:57:50.461 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "sleep_interval", 2024-09-19T08:57:50.461 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:50.462 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.462 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.462 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "60", 2024-09-19T08:57:50.462 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.462 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.462 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.462 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T08:57:50.462 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.462 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.462 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.462 INFO:teuthology.orchestra.run.smithi164.stdout: "upmap_max_deviation": { 2024-09-19T08:57:50.462 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "upmap_max_deviation", 2024-09-19T08:57:50.462 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.463 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.463 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.463 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "5", 2024-09-19T08:57:50.463 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "1", 2024-09-19T08:57:50.463 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.463 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.463 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T08:57:50.463 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T08:57:50.463 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.463 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.463 INFO:teuthology.orchestra.run.smithi164.stdout: "upmap_max_optimizations": { 2024-09-19T08:57:50.463 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "upmap_max_optimizations", 2024-09-19T08:57:50.464 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "uint", 2024-09-19T08:57:50.464 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.464 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.464 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "10", 2024-09-19T08:57:50.464 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.464 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.464 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.464 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T08:57:50.464 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.464 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.464 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.464 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.464 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.465 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "cephadm", 2024-09-19T08:57:50.465 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.465 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.465 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.465 INFO:teuthology.orchestra.run.smithi164.stdout: "agent_down_multiplier": { 2024-09-19T08:57:50.465 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "agent_down_multiplier", 2024-09-19T08:57:50.465 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "float", 2024-09-19T08:57:50.465 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.465 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.465 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "3.0", 2024-09-19T08:57:50.465 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.465 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.466 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.466 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T08:57:50.466 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.466 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.466 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.466 INFO:teuthology.orchestra.run.smithi164.stdout: "agent_refresh_rate": { 2024-09-19T08:57:50.466 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "agent_refresh_rate", 2024-09-19T08:57:50.466 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:50.466 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.466 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.466 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "20", 2024-09-19T08:57:50.466 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.466 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.467 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.467 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T08:57:50.467 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.467 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.467 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.467 INFO:teuthology.orchestra.run.smithi164.stdout: "agent_starting_port": { 2024-09-19T08:57:50.467 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "agent_starting_port", 2024-09-19T08:57:50.467 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.467 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.467 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.467 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "4721", 2024-09-19T08:57:50.467 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.468 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.468 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.468 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T08:57:50.468 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.468 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.468 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.468 INFO:teuthology.orchestra.run.smithi164.stdout: "allow_ptrace": { 2024-09-19T08:57:50.468 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "allow_ptrace", 2024-09-19T08:57:50.468 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.468 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.468 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.468 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.468 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.469 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.469 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.469 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T08:57:50.469 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T08:57:50.469 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.469 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.469 INFO:teuthology.orchestra.run.smithi164.stdout: "autotune_interval": { 2024-09-19T08:57:50.469 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "autotune_interval", 2024-09-19T08:57:50.469 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:50.469 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.469 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.469 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "600", 2024-09-19T08:57:50.470 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.470 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.470 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.470 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T08:57:50.470 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.470 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.470 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.470 INFO:teuthology.orchestra.run.smithi164.stdout: "autotune_memory_target_ratio": { 2024-09-19T08:57:50.470 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T08:57:50.470 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "float", 2024-09-19T08:57:50.470 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.470 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.471 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "0.7", 2024-09-19T08:57:50.471 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.471 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.471 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.471 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T08:57:50.471 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.471 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.471 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.471 INFO:teuthology.orchestra.run.smithi164.stdout: "cephadm_log_destination": { 2024-09-19T08:57:50.471 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "cephadm_log_destination", 2024-09-19T08:57:50.471 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.471 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.471 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.472 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.472 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.472 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.472 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.472 INFO:teuthology.orchestra.run.smithi164.stdout: "file", 2024-09-19T08:57:50.472 INFO:teuthology.orchestra.run.smithi164.stdout: "file,syslog", 2024-09-19T08:57:50.472 INFO:teuthology.orchestra.run.smithi164.stdout: "syslog" 2024-09-19T08:57:50.472 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.472 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T08:57:50.472 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.472 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.472 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.472 INFO:teuthology.orchestra.run.smithi164.stdout: "cgroups_split": { 2024-09-19T08:57:50.473 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "cgroups_split", 2024-09-19T08:57:50.473 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.473 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.473 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.473 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.473 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.473 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.473 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.473 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T08:57:50.473 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.473 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.473 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.473 INFO:teuthology.orchestra.run.smithi164.stdout: "config_checks_enabled": { 2024-09-19T08:57:50.474 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "config_checks_enabled", 2024-09-19T08:57:50.474 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.474 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.474 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.474 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.474 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.474 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.474 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.474 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T08:57:50.474 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.474 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.474 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.474 INFO:teuthology.orchestra.run.smithi164.stdout: "config_dashboard": { 2024-09-19T08:57:50.475 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "config_dashboard", 2024-09-19T08:57:50.475 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.475 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.475 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.475 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.475 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.475 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.475 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.475 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T08:57:50.475 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.475 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.475 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.476 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_alertmanager": { 2024-09-19T08:57:50.476 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_alertmanager", 2024-09-19T08:57:50.476 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.476 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.476 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.476 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T08:57:50.476 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.476 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.476 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.476 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Prometheus container image", 2024-09-19T08:57:50.476 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.476 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.476 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.477 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_base": { 2024-09-19T08:57:50.477 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_base", 2024-09-19T08:57:50.477 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.477 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.477 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.477 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T08:57:50.477 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.477 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.477 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.477 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Container image name, without the tag", 2024-09-19T08:57:50.477 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.477 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.477 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.478 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_elasticsearch": { 2024-09-19T08:57:50.478 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_elasticsearch", 2024-09-19T08:57:50.478 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.478 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.478 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.478 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T08:57:50.478 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.478 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.478 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.478 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "elasticsearch container image", 2024-09-19T08:57:50.478 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.478 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.479 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.479 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_grafana": { 2024-09-19T08:57:50.479 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_grafana", 2024-09-19T08:57:50.479 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.479 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.479 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.479 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T08:57:50.479 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.479 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.479 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.479 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Prometheus container image", 2024-09-19T08:57:50.479 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.479 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.480 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.480 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_haproxy": { 2024-09-19T08:57:50.480 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_haproxy", 2024-09-19T08:57:50.480 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.480 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.480 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.480 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T08:57:50.480 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.480 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.480 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.480 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "HAproxy container image", 2024-09-19T08:57:50.480 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.481 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.481 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.481 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_jaeger_agent": { 2024-09-19T08:57:50.481 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_jaeger_agent", 2024-09-19T08:57:50.481 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.481 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.481 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.481 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T08:57:50.481 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.481 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.481 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.481 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Jaeger agent container image", 2024-09-19T08:57:50.482 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.482 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.482 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.482 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_jaeger_collector": { 2024-09-19T08:57:50.482 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_jaeger_collector", 2024-09-19T08:57:50.482 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.482 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.482 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.482 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T08:57:50.482 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.482 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.482 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.482 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Jaeger collector container image", 2024-09-19T08:57:50.483 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.483 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.483 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.483 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_jaeger_query": { 2024-09-19T08:57:50.483 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_jaeger_query", 2024-09-19T08:57:50.483 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.483 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.483 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.483 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T08:57:50.483 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.483 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.483 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.483 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Jaeger query container image", 2024-09-19T08:57:50.484 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.484 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.484 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.484 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_keepalived": { 2024-09-19T08:57:50.484 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_keepalived", 2024-09-19T08:57:50.484 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.484 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.484 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.484 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T08:57:50.484 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.484 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.484 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.485 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Keepalived container image", 2024-09-19T08:57:50.485 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.485 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.485 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.485 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_loki": { 2024-09-19T08:57:50.485 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_loki", 2024-09-19T08:57:50.485 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.485 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.485 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.485 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T08:57:50.485 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.485 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.486 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.486 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Loki container image", 2024-09-19T08:57:50.486 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.486 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.486 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.486 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_node_exporter": { 2024-09-19T08:57:50.486 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_node_exporter", 2024-09-19T08:57:50.486 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.486 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.486 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.486 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T08:57:50.486 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.486 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.487 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.487 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Prometheus container image", 2024-09-19T08:57:50.487 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.487 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.487 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.487 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_nvmeof": { 2024-09-19T08:57:50.487 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_nvmeof", 2024-09-19T08:57:50.487 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.487 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.487 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.487 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T08:57:50.487 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.487 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.488 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.488 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Nvme-of container image", 2024-09-19T08:57:50.488 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.488 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.488 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.488 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_prometheus": { 2024-09-19T08:57:50.488 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_prometheus", 2024-09-19T08:57:50.488 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.488 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.488 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.488 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T08:57:50.488 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.489 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.489 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.489 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Prometheus container image", 2024-09-19T08:57:50.489 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.489 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.489 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.489 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_promtail": { 2024-09-19T08:57:50.489 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_promtail", 2024-09-19T08:57:50.489 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.489 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.489 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.489 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T08:57:50.489 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.490 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.490 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.490 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Promtail container image", 2024-09-19T08:57:50.490 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.490 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.490 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.490 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_samba": { 2024-09-19T08:57:50.490 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_samba", 2024-09-19T08:57:50.490 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.490 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.490 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.490 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T08:57:50.491 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.491 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.491 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.491 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Samba/SMB container image", 2024-09-19T08:57:50.491 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.491 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.491 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.491 INFO:teuthology.orchestra.run.smithi164.stdout: "container_image_snmp_gateway": { 2024-09-19T08:57:50.491 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_image_snmp_gateway", 2024-09-19T08:57:50.491 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.491 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.491 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.491 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T08:57:50.492 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.492 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.492 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.492 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "SNMP Gateway container image", 2024-09-19T08:57:50.492 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.492 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.492 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.492 INFO:teuthology.orchestra.run.smithi164.stdout: "container_init": { 2024-09-19T08:57:50.492 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "container_init", 2024-09-19T08:57:50.492 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.492 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.492 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.493 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.493 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.493 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.493 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.493 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T08:57:50.493 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.493 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.493 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.493 INFO:teuthology.orchestra.run.smithi164.stdout: "daemon_cache_timeout": { 2024-09-19T08:57:50.493 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "daemon_cache_timeout", 2024-09-19T08:57:50.493 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:50.493 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.494 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.494 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "600", 2024-09-19T08:57:50.494 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.494 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.494 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.494 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T08:57:50.494 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.494 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.494 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.494 INFO:teuthology.orchestra.run.smithi164.stdout: "default_cephadm_command_timeout": { 2024-09-19T08:57:50.494 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T08:57:50.495 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.495 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.495 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.495 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "900", 2024-09-19T08:57:50.495 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.495 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.495 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.495 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T08:57:50.495 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.495 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.495 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.495 INFO:teuthology.orchestra.run.smithi164.stdout: "default_registry": { 2024-09-19T08:57:50.496 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "default_registry", 2024-09-19T08:57:50.496 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.496 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.496 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.496 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "docker.io", 2024-09-19T08:57:50.496 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.496 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.496 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.496 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T08:57:50.496 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.496 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.496 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.497 INFO:teuthology.orchestra.run.smithi164.stdout: "device_cache_timeout": { 2024-09-19T08:57:50.497 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "device_cache_timeout", 2024-09-19T08:57:50.497 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:50.497 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.497 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.497 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "1800", 2024-09-19T08:57:50.497 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.497 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.497 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.497 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "seconds to cache device inventory", 2024-09-19T08:57:50.497 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.497 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.497 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.498 INFO:teuthology.orchestra.run.smithi164.stdout: "device_enhanced_scan": { 2024-09-19T08:57:50.498 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "device_enhanced_scan", 2024-09-19T08:57:50.498 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.498 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.498 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.498 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.498 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.498 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.498 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.498 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T08:57:50.498 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.498 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.498 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.499 INFO:teuthology.orchestra.run.smithi164.stdout: "facts_cache_timeout": { 2024-09-19T08:57:50.499 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "facts_cache_timeout", 2024-09-19T08:57:50.499 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:50.499 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.499 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.499 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "60", 2024-09-19T08:57:50.499 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.499 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.499 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.499 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "seconds to cache host facts data", 2024-09-19T08:57:50.499 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.499 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.499 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.500 INFO:teuthology.orchestra.run.smithi164.stdout: "grafana_dashboards_path": { 2024-09-19T08:57:50.500 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "grafana_dashboards_path", 2024-09-19T08:57:50.500 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.500 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.500 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.500 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T08:57:50.500 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.500 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.500 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.500 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T08:57:50.500 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.500 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.500 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.501 INFO:teuthology.orchestra.run.smithi164.stdout: "host_check_interval": { 2024-09-19T08:57:50.501 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "host_check_interval", 2024-09-19T08:57:50.501 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:50.501 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.501 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.501 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "600", 2024-09-19T08:57:50.501 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.501 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.501 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.501 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "how frequently to perform a host check", 2024-09-19T08:57:50.501 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.501 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.502 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.502 INFO:teuthology.orchestra.run.smithi164.stdout: "hw_monitoring": { 2024-09-19T08:57:50.502 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "hw_monitoring", 2024-09-19T08:57:50.502 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.502 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.502 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.502 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.502 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.502 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.502 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.502 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T08:57:50.502 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.502 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.503 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.503 INFO:teuthology.orchestra.run.smithi164.stdout: "inventory_list_all": { 2024-09-19T08:57:50.503 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "inventory_list_all", 2024-09-19T08:57:50.503 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.503 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.503 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.503 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.503 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.503 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.503 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.503 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T08:57:50.503 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.503 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.504 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.504 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.504 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.504 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.504 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.504 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.504 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.504 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.504 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.504 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.504 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.504 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.505 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.505 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.505 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.505 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.505 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.505 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.505 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.505 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.505 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.505 INFO:teuthology.orchestra.run.smithi164.stdout: "log_refresh_metadata": { 2024-09-19T08:57:50.505 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_refresh_metadata", 2024-09-19T08:57:50.505 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.506 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.506 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.506 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.506 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.506 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.506 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.506 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T08:57:50.506 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.506 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.506 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.506 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.506 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.507 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.507 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.507 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.507 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.507 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.507 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.507 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.507 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T08:57:50.507 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.507 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.507 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.507 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.507 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.508 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.508 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.508 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.508 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.508 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.508 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.508 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.508 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.508 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.508 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.508 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.508 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.508 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.509 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.509 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.509 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.509 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.509 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.509 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.509 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.509 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.509 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.509 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.509 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.509 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.509 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.510 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.510 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.510 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.510 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.513 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.513 INFO:teuthology.orchestra.run.smithi164.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T08:57:50.513 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T08:57:50.513 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.513 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.513 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.513 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.513 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.514 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.514 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.514 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T08:57:50.514 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.514 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.514 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.514 INFO:teuthology.orchestra.run.smithi164.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T08:57:50.514 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T08:57:50.514 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.514 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.514 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.514 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "*", 2024-09-19T08:57:50.514 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.515 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.515 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.515 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T08:57:50.515 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.515 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.515 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.515 INFO:teuthology.orchestra.run.smithi164.stdout: "max_count_per_host": { 2024-09-19T08:57:50.515 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "max_count_per_host", 2024-09-19T08:57:50.515 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.515 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.515 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.515 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "10", 2024-09-19T08:57:50.515 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.516 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.516 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.516 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "max number of daemons per service per host", 2024-09-19T08:57:50.516 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.516 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.516 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.516 INFO:teuthology.orchestra.run.smithi164.stdout: "max_osd_draining_count": { 2024-09-19T08:57:50.516 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "max_osd_draining_count", 2024-09-19T08:57:50.516 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.516 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.516 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.516 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "10", 2024-09-19T08:57:50.517 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.517 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.517 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.517 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T08:57:50.517 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.517 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.517 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.517 INFO:teuthology.orchestra.run.smithi164.stdout: "migration_current": { 2024-09-19T08:57:50.517 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "migration_current", 2024-09-19T08:57:50.517 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.517 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.517 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.517 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.518 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.518 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.518 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.518 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "internal - do not modify", 2024-09-19T08:57:50.518 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.518 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.518 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.518 INFO:teuthology.orchestra.run.smithi164.stdout: "mode": { 2024-09-19T08:57:50.518 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "mode", 2024-09-19T08:57:50.518 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.518 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.518 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.518 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "root", 2024-09-19T08:57:50.519 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.519 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.519 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.519 INFO:teuthology.orchestra.run.smithi164.stdout: "cephadm-package", 2024-09-19T08:57:50.519 INFO:teuthology.orchestra.run.smithi164.stdout: "root" 2024-09-19T08:57:50.519 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.519 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T08:57:50.519 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.519 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.519 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.519 INFO:teuthology.orchestra.run.smithi164.stdout: "oob_default_addr": { 2024-09-19T08:57:50.519 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "oob_default_addr", 2024-09-19T08:57:50.520 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.520 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.520 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.520 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "169.254.1.1", 2024-09-19T08:57:50.520 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.520 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.520 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.520 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T08:57:50.520 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.520 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.520 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.520 INFO:teuthology.orchestra.run.smithi164.stdout: "prometheus_alerts_path": { 2024-09-19T08:57:50.520 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "prometheus_alerts_path", 2024-09-19T08:57:50.521 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.521 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.521 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.521 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T08:57:50.521 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.521 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.521 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.521 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T08:57:50.521 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.521 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.521 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.521 INFO:teuthology.orchestra.run.smithi164.stdout: "registry_insecure": { 2024-09-19T08:57:50.522 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "registry_insecure", 2024-09-19T08:57:50.522 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.522 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.522 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.522 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.522 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.522 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.522 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.522 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T08:57:50.522 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.522 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.522 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.522 INFO:teuthology.orchestra.run.smithi164.stdout: "registry_password": { 2024-09-19T08:57:50.523 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "registry_password", 2024-09-19T08:57:50.523 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.523 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.523 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.523 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.523 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.523 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.523 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.523 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T08:57:50.523 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.523 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.523 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.524 INFO:teuthology.orchestra.run.smithi164.stdout: "registry_url": { 2024-09-19T08:57:50.524 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "registry_url", 2024-09-19T08:57:50.524 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.524 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.524 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.524 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.524 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.524 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.524 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.524 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T08:57:50.524 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.524 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.524 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.525 INFO:teuthology.orchestra.run.smithi164.stdout: "registry_username": { 2024-09-19T08:57:50.525 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "registry_username", 2024-09-19T08:57:50.525 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.525 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.525 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.525 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.525 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.525 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.525 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.525 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T08:57:50.525 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.525 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.526 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.526 INFO:teuthology.orchestra.run.smithi164.stdout: "secure_monitoring_stack": { 2024-09-19T08:57:50.526 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "secure_monitoring_stack", 2024-09-19T08:57:50.526 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.526 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.526 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.526 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.526 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.526 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.526 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.526 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T08:57:50.526 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.526 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.527 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.527 INFO:teuthology.orchestra.run.smithi164.stdout: "service_discovery_port": { 2024-09-19T08:57:50.527 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "service_discovery_port", 2024-09-19T08:57:50.527 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.527 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.527 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.527 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "8765", 2024-09-19T08:57:50.527 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.527 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.527 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.527 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "cephadm service discovery port", 2024-09-19T08:57:50.527 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.527 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.528 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.528 INFO:teuthology.orchestra.run.smithi164.stdout: "ssh_config_file": { 2024-09-19T08:57:50.528 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "ssh_config_file", 2024-09-19T08:57:50.528 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.528 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.528 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.528 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.528 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.528 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.528 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.528 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T08:57:50.528 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.529 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.529 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.529 INFO:teuthology.orchestra.run.smithi164.stdout: "use_agent": { 2024-09-19T08:57:50.529 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "use_agent", 2024-09-19T08:57:50.529 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.529 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.529 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.529 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.529 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.529 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.529 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.529 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T08:57:50.530 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.530 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.530 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.530 INFO:teuthology.orchestra.run.smithi164.stdout: "use_repo_digest": { 2024-09-19T08:57:50.530 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "use_repo_digest", 2024-09-19T08:57:50.530 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.530 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.530 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.530 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.530 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.530 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.530 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.530 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T08:57:50.531 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.531 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.531 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.531 INFO:teuthology.orchestra.run.smithi164.stdout: "warn_on_failed_host_check": { 2024-09-19T08:57:50.531 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "warn_on_failed_host_check", 2024-09-19T08:57:50.531 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.531 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.531 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.531 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.531 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.531 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.531 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.531 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T08:57:50.532 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.532 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.532 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.532 INFO:teuthology.orchestra.run.smithi164.stdout: "warn_on_stray_daemons": { 2024-09-19T08:57:50.532 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "warn_on_stray_daemons", 2024-09-19T08:57:50.532 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.532 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.532 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.532 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.532 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.532 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.532 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.532 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T08:57:50.533 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.533 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.533 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.533 INFO:teuthology.orchestra.run.smithi164.stdout: "warn_on_stray_hosts": { 2024-09-19T08:57:50.533 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "warn_on_stray_hosts", 2024-09-19T08:57:50.533 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.533 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.533 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.533 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.533 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.533 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.533 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.534 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T08:57:50.534 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.534 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.534 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.534 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.534 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.534 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "crash", 2024-09-19T08:57:50.534 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.534 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.534 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.534 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.534 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.535 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.535 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.535 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.535 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.535 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.535 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.535 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.535 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.535 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.535 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.535 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.535 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.535 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.535 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.536 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.536 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.536 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.536 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.536 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.536 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.536 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.536 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.536 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.536 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.536 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.536 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.536 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.537 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.537 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.537 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.537 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.537 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.537 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.537 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.537 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.537 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.537 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.537 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.537 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.538 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.538 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.538 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.538 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.538 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.538 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.538 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.538 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.538 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.538 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.538 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.538 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.538 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.539 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.539 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.539 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.539 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.539 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.539 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.539 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.539 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.539 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.539 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.539 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.539 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.540 INFO:teuthology.orchestra.run.smithi164.stdout: "retain_interval": { 2024-09-19T08:57:50.540 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "retain_interval", 2024-09-19T08:57:50.540 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:50.540 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.540 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.540 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "31536000", 2024-09-19T08:57:50.540 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.540 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.540 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.540 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T08:57:50.540 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.540 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.540 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.541 INFO:teuthology.orchestra.run.smithi164.stdout: "warn_recent_interval": { 2024-09-19T08:57:50.541 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "warn_recent_interval", 2024-09-19T08:57:50.541 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:50.541 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.541 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.541 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "1209600", 2024-09-19T08:57:50.541 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.541 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.541 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.541 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T08:57:50.541 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.541 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.542 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.542 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.542 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.542 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "dashboard", 2024-09-19T08:57:50.542 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.542 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.542 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.542 INFO:teuthology.orchestra.run.smithi164.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T08:57:50.542 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T08:57:50.542 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.543 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.543 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.543 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "10", 2024-09-19T08:57:50.543 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.543 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.543 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.543 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.543 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.543 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.543 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.543 INFO:teuthology.orchestra.run.smithi164.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T08:57:50.543 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T08:57:50.543 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.544 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.544 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.544 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.544 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.544 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.544 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.544 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.544 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.544 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.544 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.544 INFO:teuthology.orchestra.run.smithi164.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T08:57:50.544 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T08:57:50.545 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.545 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.545 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.545 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.545 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.545 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.545 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.545 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.545 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.545 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.545 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.545 INFO:teuthology.orchestra.run.smithi164.stdout: "AUDIT_API_ENABLED": { 2024-09-19T08:57:50.546 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T08:57:50.546 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.546 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.546 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.546 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.546 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.546 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.546 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.546 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.546 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.546 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.546 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.546 INFO:teuthology.orchestra.run.smithi164.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T08:57:50.547 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T08:57:50.547 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.547 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.547 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.547 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.547 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.547 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.547 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.547 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.547 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.547 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.547 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.548 INFO:teuthology.orchestra.run.smithi164.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T08:57:50.548 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T08:57:50.548 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.548 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.548 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.548 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.548 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.548 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.548 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.548 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.548 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.548 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.548 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.549 INFO:teuthology.orchestra.run.smithi164.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T08:57:50.549 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T08:57:50.549 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.549 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.549 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.549 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.549 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.549 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.549 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.549 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.549 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.549 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.549 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.550 INFO:teuthology.orchestra.run.smithi164.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T08:57:50.550 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T08:57:50.550 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.550 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.550 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.550 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.550 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.550 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.550 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.550 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.550 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.550 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.550 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.551 INFO:teuthology.orchestra.run.smithi164.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T08:57:50.551 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T08:57:50.551 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.551 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.551 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.551 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.551 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.551 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.551 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.551 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.551 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.551 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.551 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.552 INFO:teuthology.orchestra.run.smithi164.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T08:57:50.552 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T08:57:50.552 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.552 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.552 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.552 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.552 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.552 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.552 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.552 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.552 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.552 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.552 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.553 INFO:teuthology.orchestra.run.smithi164.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T08:57:50.553 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T08:57:50.553 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.553 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.553 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.553 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.553 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.553 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.553 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.553 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.553 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.553 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.553 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.554 INFO:teuthology.orchestra.run.smithi164.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T08:57:50.554 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T08:57:50.554 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.554 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.554 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.554 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.554 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.554 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.554 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.554 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.554 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.554 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.554 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.555 INFO:teuthology.orchestra.run.smithi164.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T08:57:50.555 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T08:57:50.555 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.555 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.555 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.555 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.555 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.555 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.555 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.555 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.555 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.555 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.555 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.556 INFO:teuthology.orchestra.run.smithi164.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T08:57:50.556 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T08:57:50.556 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.556 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.556 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.556 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.556 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.556 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.556 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.556 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.556 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.556 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.557 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.557 INFO:teuthology.orchestra.run.smithi164.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T08:57:50.557 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T08:57:50.557 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.557 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.557 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.557 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "admin", 2024-09-19T08:57:50.557 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.557 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.557 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.557 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.557 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.558 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.558 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.558 INFO:teuthology.orchestra.run.smithi164.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T08:57:50.558 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T08:57:50.558 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.558 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.558 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.558 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.558 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.558 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.558 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.558 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.558 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.559 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.559 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.559 INFO:teuthology.orchestra.run.smithi164.stdout: "GRAFANA_API_URL": { 2024-09-19T08:57:50.559 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "GRAFANA_API_URL", 2024-09-19T08:57:50.559 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.559 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.559 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.559 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.559 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.559 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.559 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.559 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.559 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.560 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.560 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.560 INFO:teuthology.orchestra.run.smithi164.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T08:57:50.560 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T08:57:50.560 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.560 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.560 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.560 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "admin", 2024-09-19T08:57:50.560 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.560 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.560 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.560 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.561 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.561 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.561 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.561 INFO:teuthology.orchestra.run.smithi164.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T08:57:50.561 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T08:57:50.561 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.561 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.561 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.561 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.561 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.561 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.561 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.561 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.562 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.562 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.562 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.562 INFO:teuthology.orchestra.run.smithi164.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T08:57:50.562 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T08:57:50.562 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.562 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.562 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.562 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.562 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.562 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.562 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.563 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.563 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.563 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.563 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.563 INFO:teuthology.orchestra.run.smithi164.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T08:57:50.563 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T08:57:50.563 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.563 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.563 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.563 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.563 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.563 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.563 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.564 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.564 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.564 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.564 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.564 INFO:teuthology.orchestra.run.smithi164.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T08:57:50.564 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T08:57:50.564 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.564 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.564 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.564 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.564 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.564 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.564 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.565 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.565 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.565 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.565 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.565 INFO:teuthology.orchestra.run.smithi164.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T08:57:50.565 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T08:57:50.565 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.565 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.565 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.565 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.565 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.565 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.566 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.566 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.566 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.566 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.566 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.566 INFO:teuthology.orchestra.run.smithi164.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T08:57:50.566 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T08:57:50.566 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.566 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.566 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.566 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.566 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.566 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.567 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.567 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.567 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.567 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.567 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.567 INFO:teuthology.orchestra.run.smithi164.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T08:57:50.567 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T08:57:50.567 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.567 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.567 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.567 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.567 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.567 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.568 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.568 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.568 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.568 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.568 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.568 INFO:teuthology.orchestra.run.smithi164.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T08:57:50.568 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T08:57:50.568 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.568 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.568 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.568 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.568 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.569 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.569 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.569 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.569 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.569 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.569 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.569 INFO:teuthology.orchestra.run.smithi164.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T08:57:50.569 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T08:57:50.569 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.569 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.569 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.569 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.569 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.570 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.570 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.570 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.570 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.570 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.570 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.570 INFO:teuthology.orchestra.run.smithi164.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T08:57:50.570 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T08:57:50.570 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.570 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.570 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.570 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.571 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.571 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.571 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.571 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.571 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.571 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.571 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.571 INFO:teuthology.orchestra.run.smithi164.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T08:57:50.571 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T08:57:50.571 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.571 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.571 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.571 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.572 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.572 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.572 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.572 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.572 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.572 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.572 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.572 INFO:teuthology.orchestra.run.smithi164.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T08:57:50.572 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T08:57:50.572 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.572 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.572 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.572 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.573 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.573 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.573 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.573 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.573 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.573 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.573 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.573 INFO:teuthology.orchestra.run.smithi164.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T08:57:50.573 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T08:57:50.573 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.573 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.573 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.574 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.574 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.574 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.574 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.574 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.574 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.574 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.574 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.574 INFO:teuthology.orchestra.run.smithi164.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T08:57:50.574 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T08:57:50.574 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.574 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.574 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.575 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.575 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.575 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.575 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.575 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.575 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.575 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.575 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.575 INFO:teuthology.orchestra.run.smithi164.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T08:57:50.575 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T08:57:50.575 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.575 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.576 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.576 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T08:57:50.576 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.576 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.576 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.576 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.576 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.576 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.576 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.576 INFO:teuthology.orchestra.run.smithi164.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T08:57:50.576 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T08:57:50.576 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.577 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.577 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.577 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "10", 2024-09-19T08:57:50.577 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.577 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.577 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.577 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.577 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.577 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.577 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.577 INFO:teuthology.orchestra.run.smithi164.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T08:57:50.577 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T08:57:50.577 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.578 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.578 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.578 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "8", 2024-09-19T08:57:50.578 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.578 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.578 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.578 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.578 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.578 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.578 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.578 INFO:teuthology.orchestra.run.smithi164.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T08:57:50.578 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T08:57:50.579 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.579 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.579 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.579 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "45", 2024-09-19T08:57:50.579 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.579 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.579 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.579 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.579 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.579 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.579 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.579 INFO:teuthology.orchestra.run.smithi164.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T08:57:50.580 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T08:57:50.580 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.580 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.580 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.580 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.580 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.580 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.580 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.580 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.580 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.580 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.580 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.581 INFO:teuthology.orchestra.run.smithi164.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T08:57:50.581 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T08:57:50.581 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.581 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.581 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.581 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "admin", 2024-09-19T08:57:50.581 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.581 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.581 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.581 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.581 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.581 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.581 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.582 INFO:teuthology.orchestra.run.smithi164.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T08:57:50.582 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T08:57:50.582 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.582 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.582 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.582 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.582 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.582 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.582 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.582 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.582 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.582 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.582 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.583 INFO:teuthology.orchestra.run.smithi164.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T08:57:50.583 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T08:57:50.583 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.583 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.583 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.583 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.583 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.583 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.583 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.583 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.583 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.583 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.583 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.584 INFO:teuthology.orchestra.run.smithi164.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T08:57:50.584 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T08:57:50.584 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.584 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.584 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.584 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "0", 2024-09-19T08:57:50.584 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.584 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.584 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.584 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.584 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.584 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.584 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.585 INFO:teuthology.orchestra.run.smithi164.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T08:57:50.585 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T08:57:50.585 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.585 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.585 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.585 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "10", 2024-09-19T08:57:50.585 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.585 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.585 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.585 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.585 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.585 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.585 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.586 INFO:teuthology.orchestra.run.smithi164.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T08:57:50.586 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T08:57:50.586 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.586 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.586 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.586 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "5", 2024-09-19T08:57:50.586 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.586 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.586 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.586 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.586 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.586 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.586 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.587 INFO:teuthology.orchestra.run.smithi164.stdout: "cross_origin_url": { 2024-09-19T08:57:50.587 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "cross_origin_url", 2024-09-19T08:57:50.587 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.587 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.587 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.587 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.587 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.587 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.587 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.587 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.587 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.587 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.588 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.588 INFO:teuthology.orchestra.run.smithi164.stdout: "crt_file": { 2024-09-19T08:57:50.588 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "crt_file", 2024-09-19T08:57:50.588 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.588 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.588 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.588 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.588 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.588 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.588 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.588 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.588 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.589 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.589 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.589 INFO:teuthology.orchestra.run.smithi164.stdout: "debug": { 2024-09-19T08:57:50.589 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "debug", 2024-09-19T08:57:50.589 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.589 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.589 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.589 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.589 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.589 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.589 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.589 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Enable/disable debug options", 2024-09-19T08:57:50.590 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.590 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.590 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.590 INFO:teuthology.orchestra.run.smithi164.stdout: "jwt_token_ttl": { 2024-09-19T08:57:50.590 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "jwt_token_ttl", 2024-09-19T08:57:50.590 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.590 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.590 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.590 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "28800", 2024-09-19T08:57:50.590 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.590 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.590 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.590 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.591 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.591 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.591 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.591 INFO:teuthology.orchestra.run.smithi164.stdout: "key_file": { 2024-09-19T08:57:50.591 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "key_file", 2024-09-19T08:57:50.591 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.591 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.591 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.591 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.591 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.591 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.591 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.592 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.592 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.592 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.592 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.592 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.592 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.592 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.592 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.592 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.592 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.592 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.592 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.593 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.593 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.593 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.593 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.593 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.593 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.593 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.593 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.593 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.593 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.593 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.593 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.593 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.594 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.594 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.594 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.594 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.594 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.594 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.594 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.594 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.594 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.594 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.594 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.594 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.595 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.595 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.595 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.595 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.595 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.595 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.595 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.595 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.595 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.595 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.595 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.595 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.595 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.596 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.596 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.596 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.596 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.596 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.596 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.596 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.596 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.596 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.596 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.596 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.596 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.596 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.597 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.597 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.597 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.597 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.597 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.597 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.597 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.597 INFO:teuthology.orchestra.run.smithi164.stdout: "motd": { 2024-09-19T08:57:50.597 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "motd", 2024-09-19T08:57:50.597 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.597 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.597 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "The message of the day", 2024-09-19T08:57:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: "redirect_resolve_ip_addr": { 2024-09-19T08:57:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T08:57:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: "server_addr": { 2024-09-19T08:57:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "server_addr", 2024-09-19T08:57:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "::", 2024-09-19T08:57:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: "server_port": { 2024-09-19T08:57:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "server_port", 2024-09-19T08:57:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "8080", 2024-09-19T08:57:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: "ssl": { 2024-09-19T08:57:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "ssl", 2024-09-19T08:57:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: "ssl_server_port": { 2024-09-19T08:57:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "ssl_server_port", 2024-09-19T08:57:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "8443", 2024-09-19T08:57:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: "standby_behaviour": { 2024-09-19T08:57:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "standby_behaviour", 2024-09-19T08:57:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "redirect", 2024-09-19T08:57:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: "redirect" 2024-09-19T08:57:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: "standby_error_status_code": { 2024-09-19T08:57:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "standby_error_status_code", 2024-09-19T08:57:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "500", 2024-09-19T08:57:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "400", 2024-09-19T08:57:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "599", 2024-09-19T08:57:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: "url_prefix": { 2024-09-19T08:57:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "url_prefix", 2024-09-19T08:57:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "devicehealth", 2024-09-19T08:57:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: "enable_monitoring": { 2024-09-19T08:57:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "enable_monitoring", 2024-09-19T08:57:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "monitor device health metrics", 2024-09-19T08:57:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: "mark_out_threshold": { 2024-09-19T08:57:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "mark_out_threshold", 2024-09-19T08:57:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "2419200", 2024-09-19T08:57:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T08:57:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: "pool_name": { 2024-09-19T08:57:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "pool_name", 2024-09-19T08:57:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "device_health_metrics", 2024-09-19T08:57:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T08:57:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: "retention_period": { 2024-09-19T08:57:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "retention_period", 2024-09-19T08:57:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "15552000", 2024-09-19T08:57:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "how long to retain device health metrics", 2024-09-19T08:57:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: "scrape_frequency": { 2024-09-19T08:57:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "scrape_frequency", 2024-09-19T08:57:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "86400", 2024-09-19T08:57:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T08:57:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: "self_heal": { 2024-09-19T08:57:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "self_heal", 2024-09-19T08:57:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T08:57:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: "sleep_interval": { 2024-09-19T08:57:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "sleep_interval", 2024-09-19T08:57:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "600", 2024-09-19T08:57:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T08:57:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: "warn_threshold": { 2024-09-19T08:57:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "warn_threshold", 2024-09-19T08:57:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "7257600", 2024-09-19T08:57:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T08:57:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "diskprediction_local", 2024-09-19T08:57:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: "predict_interval": { 2024-09-19T08:57:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "predict_interval", 2024-09-19T08:57:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "86400", 2024-09-19T08:57:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "predictor_model": { 2024-09-19T08:57:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "predictor_model", 2024-09-19T08:57:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "prophetstor", 2024-09-19T08:57:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "sleep_interval": { 2024-09-19T08:57:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "sleep_interval", 2024-09-19T08:57:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "600", 2024-09-19T08:57:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "influx", 2024-09-19T08:57:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": false, 2024-09-19T08:57:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "influxdb python module not found", 2024-09-19T08:57:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: "batch_size": { 2024-09-19T08:57:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "batch_size", 2024-09-19T08:57:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "5000", 2024-09-19T08:57:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T08:57:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: "database": { 2024-09-19T08:57:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "database", 2024-09-19T08:57:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "ceph", 2024-09-19T08:57:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T08:57:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "hostname": { 2024-09-19T08:57:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "hostname", 2024-09-19T08:57:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "InfluxDB server hostname", 2024-09-19T08:57:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "interval": { 2024-09-19T08:57:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "interval", 2024-09-19T08:57:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "30", 2024-09-19T08:57:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "5", 2024-09-19T08:57:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T08:57:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: "password": { 2024-09-19T08:57:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "password", 2024-09-19T08:57:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "password of InfluxDB server user", 2024-09-19T08:57:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.641 INFO:teuthology.orchestra.run.smithi164.stdout: "port": { 2024-09-19T08:57:50.641 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "port", 2024-09-19T08:57:50.641 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.641 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.641 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.641 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "8086", 2024-09-19T08:57:50.641 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.641 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.641 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.641 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "InfluxDB server port", 2024-09-19T08:57:50.641 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.641 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.641 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: "ssl": { 2024-09-19T08:57:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "ssl", 2024-09-19T08:57:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "false", 2024-09-19T08:57:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T08:57:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: "threads": { 2024-09-19T08:57:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "threads", 2024-09-19T08:57:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "5", 2024-09-19T08:57:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "1", 2024-09-19T08:57:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "32", 2024-09-19T08:57:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T08:57:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.644 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.644 INFO:teuthology.orchestra.run.smithi164.stdout: "username": { 2024-09-19T08:57:50.644 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "username", 2024-09-19T08:57:50.644 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.644 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.644 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.644 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.644 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.644 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.644 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.644 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "username of InfluxDB server user", 2024-09-19T08:57:50.644 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: "verify_ssl": { 2024-09-19T08:57:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "verify_ssl", 2024-09-19T08:57:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "true", 2024-09-19T08:57:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T08:57:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "insights", 2024-09-19T08:57:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "iostat", 2024-09-19T08:57:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "k8sevents", 2024-09-19T08:57:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_event_retention_days": { 2024-09-19T08:57:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "ceph_event_retention_days", 2024-09-19T08:57:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "7", 2024-09-19T08:57:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T08:57:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: "config_check_secs": { 2024-09-19T08:57:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "config_check_secs", 2024-09-19T08:57:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "10", 2024-09-19T08:57:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "10", 2024-09-19T08:57:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T08:57:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "localpool", 2024-09-19T08:57:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: "failure_domain": { 2024-09-19T08:57:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "failure_domain", 2024-09-19T08:57:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "host", 2024-09-19T08:57:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "failure domain for any created local pool", 2024-09-19T08:57:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T08:57:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.672 INFO:teuthology.orchestra.run.smithi164.stdout: "min_size": { 2024-09-19T08:57:50.672 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "min_size", 2024-09-19T08:57:50.672 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.672 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.672 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.672 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.672 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.672 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.672 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.672 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "default min_size for any created local pool", 2024-09-19T08:57:50.672 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T08:57:50.672 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.672 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.673 INFO:teuthology.orchestra.run.smithi164.stdout: "num_rep": { 2024-09-19T08:57:50.673 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "num_rep", 2024-09-19T08:57:50.673 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.673 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.673 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.673 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "3", 2024-09-19T08:57:50.673 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.673 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.673 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.673 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "default replica count for any created local pool", 2024-09-19T08:57:50.673 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.673 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.673 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.674 INFO:teuthology.orchestra.run.smithi164.stdout: "pg_num": { 2024-09-19T08:57:50.674 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "pg_num", 2024-09-19T08:57:50.674 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.674 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.674 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.674 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "128", 2024-09-19T08:57:50.674 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.674 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.674 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.674 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T08:57:50.674 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.674 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.674 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.675 INFO:teuthology.orchestra.run.smithi164.stdout: "prefix": { 2024-09-19T08:57:50.675 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "prefix", 2024-09-19T08:57:50.675 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.675 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.675 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.675 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.675 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.675 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.675 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.675 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "name prefix for any created local pool", 2024-09-19T08:57:50.675 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.675 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.676 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.676 INFO:teuthology.orchestra.run.smithi164.stdout: "subtree": { 2024-09-19T08:57:50.676 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "subtree", 2024-09-19T08:57:50.676 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.676 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.676 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.676 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "rack", 2024-09-19T08:57:50.676 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.676 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.676 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.676 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T08:57:50.676 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T08:57:50.676 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.677 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.677 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.677 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.677 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "mds_autoscaler", 2024-09-19T08:57:50.677 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.677 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.677 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.677 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.677 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.677 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.677 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.678 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.678 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.678 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.678 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.678 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.678 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.678 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.678 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.678 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.678 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.678 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.678 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.678 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.678 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.679 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.679 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.679 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.679 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.679 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.679 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.679 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.679 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.679 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.679 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.679 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.679 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.680 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.680 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.680 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.680 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.680 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.680 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.680 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.680 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.680 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.680 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.680 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.680 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.680 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.681 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.681 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.681 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.681 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.681 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.681 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.681 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.681 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.681 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.681 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.681 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.681 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.682 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.682 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.682 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.682 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.682 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.682 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.682 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.682 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.682 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.682 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.682 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.682 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.682 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.683 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "mirroring", 2024-09-19T08:57:50.683 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.683 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.683 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.683 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.683 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.683 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.683 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.683 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.683 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.683 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.683 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.683 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.684 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.684 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.684 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.684 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.684 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.684 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.684 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.684 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.684 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.684 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.684 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.684 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.685 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.685 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.685 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.685 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.685 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.685 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.685 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.685 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.685 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.685 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.685 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.685 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.685 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.686 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.686 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.686 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.686 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.686 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.686 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.686 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.686 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.686 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.686 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.686 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.686 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.686 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.686 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.687 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.687 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.687 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.687 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.687 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.687 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.687 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.687 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.687 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.687 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.687 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.687 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.687 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.688 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.688 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.688 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.688 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.688 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.688 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.688 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.688 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "nfs", 2024-09-19T08:57:50.688 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.688 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.688 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.689 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.689 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.689 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.689 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.689 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.689 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.689 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.689 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.689 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.689 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.689 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.689 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.689 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.689 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.690 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.690 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.690 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.690 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.690 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.690 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.690 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.690 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.690 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.690 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.690 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.690 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.690 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.691 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.691 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.691 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.691 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.691 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.691 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.691 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.691 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.691 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.691 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.691 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.691 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.691 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.692 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.692 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.692 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.692 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.692 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.692 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.692 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.692 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.692 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.692 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.692 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.692 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.692 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.693 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.693 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.693 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.693 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.693 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.693 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.693 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.693 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.693 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.693 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.693 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.693 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.693 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.694 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.694 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.694 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "orchestrator", 2024-09-19T08:57:50.694 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.694 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.694 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.694 INFO:teuthology.orchestra.run.smithi164.stdout: "fail_fs": { 2024-09-19T08:57:50.694 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "fail_fs", 2024-09-19T08:57:50.694 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.694 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.694 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.694 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.695 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.695 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.695 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.695 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T08:57:50.695 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.695 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.695 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.695 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.695 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.695 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.695 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.695 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.696 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.696 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.696 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.696 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.696 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.696 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.696 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.696 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.696 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.696 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.696 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.696 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.696 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.697 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.697 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.697 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.697 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.697 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.697 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.697 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.697 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.697 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.697 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.697 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.697 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.698 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.698 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.698 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.698 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.698 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.698 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.698 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.698 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.698 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.698 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.698 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.698 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.698 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.699 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.699 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.699 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.699 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.699 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.699 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.699 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.699 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.699 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.699 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.699 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.699 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.700 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.700 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.700 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.700 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.700 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.700 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.700 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.700 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.700 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.700 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.700 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.700 INFO:teuthology.orchestra.run.smithi164.stdout: "orchestrator": { 2024-09-19T08:57:50.700 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "orchestrator", 2024-09-19T08:57:50.701 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.701 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.701 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.701 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.701 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.701 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.701 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.701 INFO:teuthology.orchestra.run.smithi164.stdout: "cephadm", 2024-09-19T08:57:50.701 INFO:teuthology.orchestra.run.smithi164.stdout: "rook", 2024-09-19T08:57:50.701 INFO:teuthology.orchestra.run.smithi164.stdout: "test_orchestrator" 2024-09-19T08:57:50.701 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.701 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Orchestrator backend", 2024-09-19T08:57:50.701 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.702 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.702 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.702 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.702 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.702 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "osd_perf_query", 2024-09-19T08:57:50.702 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.702 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.702 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.702 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.702 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.702 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.702 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.703 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.703 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.703 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.703 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.703 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.703 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.703 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.703 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.703 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.703 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.703 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.703 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.703 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.703 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.704 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.704 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.704 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.704 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.704 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.704 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.704 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.704 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.704 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.704 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.704 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.704 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.705 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.705 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.705 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.705 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.705 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.705 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.705 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.705 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.705 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.705 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.705 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.705 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.706 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.706 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.706 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.706 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.706 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.706 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.706 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.706 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.706 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.706 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.706 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.706 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.706 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.707 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.707 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.707 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.707 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.707 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.707 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.707 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.707 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.707 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.707 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.707 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.707 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.707 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.708 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "osd_support", 2024-09-19T08:57:50.708 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.708 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.708 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.708 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.708 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.708 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.708 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.708 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.708 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.708 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.708 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.709 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.709 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.709 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.709 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.709 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.709 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.709 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.709 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.709 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.709 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.709 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.709 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.709 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.710 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.710 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.710 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.710 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.710 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.710 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.710 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.710 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.710 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.710 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.710 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.710 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.711 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.711 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.711 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.711 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.711 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.711 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.711 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.711 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.711 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.711 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.711 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.711 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.711 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.711 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.712 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.712 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.712 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.712 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.712 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.712 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.712 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.712 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.712 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.712 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.712 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.712 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.713 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.713 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.713 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.713 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.713 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.717 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.717 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.717 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.717 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.717 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "pg_autoscaler", 2024-09-19T08:57:50.717 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.717 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.717 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.717 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.718 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.718 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.718 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.718 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.718 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.718 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.718 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.718 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.718 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.718 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.718 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.718 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.718 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.719 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.719 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.719 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.719 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.719 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.719 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.719 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.719 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.719 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.719 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.719 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.719 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.719 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.720 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.720 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.720 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.720 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.720 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.720 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.720 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.720 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.720 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.720 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.720 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.720 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.721 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.721 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.721 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.721 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.721 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.721 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.721 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.721 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.721 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.721 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.721 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.721 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.721 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.722 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.722 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.722 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.722 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.722 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.722 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.722 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.722 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.722 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.722 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.722 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.722 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.722 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.723 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.723 INFO:teuthology.orchestra.run.smithi164.stdout: "sleep_interval": { 2024-09-19T08:57:50.723 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "sleep_interval", 2024-09-19T08:57:50.723 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:50.723 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.723 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.723 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "60", 2024-09-19T08:57:50.723 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.723 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.723 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.723 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.723 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.724 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.724 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.724 INFO:teuthology.orchestra.run.smithi164.stdout: "threshold": { 2024-09-19T08:57:50.724 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "threshold", 2024-09-19T08:57:50.724 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "float", 2024-09-19T08:57:50.724 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.724 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.724 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "3.0", 2024-09-19T08:57:50.724 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "1.0", 2024-09-19T08:57:50.724 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.724 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.724 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "scaling threshold", 2024-09-19T08:57:50.724 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T08:57:50.725 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.725 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.725 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.725 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.725 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "progress", 2024-09-19T08:57:50.725 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.725 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.725 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.725 INFO:teuthology.orchestra.run.smithi164.stdout: "allow_pg_recovery_event": { 2024-09-19T08:57:50.725 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "allow_pg_recovery_event", 2024-09-19T08:57:50.725 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.725 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.726 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.726 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.726 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.726 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.726 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.726 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T08:57:50.726 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.726 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.726 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.726 INFO:teuthology.orchestra.run.smithi164.stdout: "enabled": { 2024-09-19T08:57:50.726 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "enabled", 2024-09-19T08:57:50.726 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.726 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.727 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.727 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.727 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.727 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.727 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.727 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.727 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.727 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.727 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.727 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.727 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.727 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.728 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.728 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.728 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.728 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.728 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.728 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.728 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.728 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.728 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.728 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.728 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.728 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.728 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.728 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.729 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.729 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.729 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.729 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.729 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.729 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.729 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.729 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.729 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.729 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.729 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.729 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.729 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.730 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.730 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.730 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.730 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.730 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.730 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.730 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.730 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.730 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.730 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.730 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.730 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.730 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.731 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.731 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.731 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.731 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.731 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.731 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.731 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.731 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.731 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.731 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.731 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.731 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.732 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.732 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.732 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.732 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.732 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.732 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.732 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.732 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.732 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.732 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.732 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.732 INFO:teuthology.orchestra.run.smithi164.stdout: "max_completed_events": { 2024-09-19T08:57:50.733 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "max_completed_events", 2024-09-19T08:57:50.733 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.733 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.733 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.733 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "50", 2024-09-19T08:57:50.733 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.733 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.733 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.733 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "number of past completed events to remember", 2024-09-19T08:57:50.733 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.733 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.733 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.733 INFO:teuthology.orchestra.run.smithi164.stdout: "sleep_interval": { 2024-09-19T08:57:50.734 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "sleep_interval", 2024-09-19T08:57:50.734 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:50.734 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.734 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.734 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "5", 2024-09-19T08:57:50.734 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.734 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.734 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.734 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "how long the module is going to sleep", 2024-09-19T08:57:50.734 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.734 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.734 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.734 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.735 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.735 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "prometheus", 2024-09-19T08:57:50.735 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.735 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.735 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.735 INFO:teuthology.orchestra.run.smithi164.stdout: "cache": { 2024-09-19T08:57:50.735 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "cache", 2024-09-19T08:57:50.735 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.735 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.735 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.735 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.736 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.736 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.736 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.736 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.736 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.736 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.736 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.736 INFO:teuthology.orchestra.run.smithi164.stdout: "exclude_perf_counters": { 2024-09-19T08:57:50.736 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "exclude_perf_counters", 2024-09-19T08:57:50.736 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.736 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.736 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.736 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.737 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.737 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.737 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.737 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T08:57:50.737 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T08:57:50.737 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.737 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.737 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.737 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.737 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.737 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.737 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.738 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.738 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.738 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.738 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.738 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.738 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.738 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.738 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.738 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.738 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.738 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.738 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.738 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.738 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.739 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.739 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.739 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.739 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.739 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.739 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.739 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.739 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.739 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.739 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.739 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.739 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.740 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.740 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.740 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.740 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.740 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.740 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.740 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.740 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.740 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.740 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.740 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.740 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.741 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.741 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.741 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.741 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.741 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.741 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.741 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.741 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.741 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.741 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.741 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.741 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.741 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.742 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.742 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.742 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.742 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.742 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.742 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.742 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.742 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.742 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.742 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.742 INFO:teuthology.orchestra.run.smithi164.stdout: "rbd_stats_pools": { 2024-09-19T08:57:50.742 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rbd_stats_pools", 2024-09-19T08:57:50.743 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.743 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.743 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.743 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.743 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.743 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.743 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.743 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.743 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.743 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.743 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.743 INFO:teuthology.orchestra.run.smithi164.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T08:57:50.743 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T08:57:50.744 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.744 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.744 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.744 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "300", 2024-09-19T08:57:50.744 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.744 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.744 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.744 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.744 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.744 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.744 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.744 INFO:teuthology.orchestra.run.smithi164.stdout: "scrape_interval": { 2024-09-19T08:57:50.745 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "scrape_interval", 2024-09-19T08:57:50.745 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "float", 2024-09-19T08:57:50.745 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.745 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.745 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "15.0", 2024-09-19T08:57:50.745 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.745 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.745 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.745 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.745 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.745 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.745 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.745 INFO:teuthology.orchestra.run.smithi164.stdout: "server_addr": { 2024-09-19T08:57:50.746 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "server_addr", 2024-09-19T08:57:50.746 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.746 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.746 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.746 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "::", 2024-09-19T08:57:50.746 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.746 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.746 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.746 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T08:57:50.746 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.746 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.746 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.746 INFO:teuthology.orchestra.run.smithi164.stdout: "server_port": { 2024-09-19T08:57:50.747 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "server_port", 2024-09-19T08:57:50.747 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.747 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.747 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.747 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "9283", 2024-09-19T08:57:50.747 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.747 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.747 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.747 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T08:57:50.747 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.747 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.747 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.747 INFO:teuthology.orchestra.run.smithi164.stdout: "stale_cache_strategy": { 2024-09-19T08:57:50.748 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "stale_cache_strategy", 2024-09-19T08:57:50.748 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.748 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.748 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.748 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "log", 2024-09-19T08:57:50.748 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.748 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.748 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.748 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.748 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.748 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.748 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.749 INFO:teuthology.orchestra.run.smithi164.stdout: "standby_behaviour": { 2024-09-19T08:57:50.749 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "standby_behaviour", 2024-09-19T08:57:50.749 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.749 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.749 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.749 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "default", 2024-09-19T08:57:50.749 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.749 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.749 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.749 INFO:teuthology.orchestra.run.smithi164.stdout: "default", 2024-09-19T08:57:50.749 INFO:teuthology.orchestra.run.smithi164.stdout: "error" 2024-09-19T08:57:50.749 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.749 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.750 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.750 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.750 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.750 INFO:teuthology.orchestra.run.smithi164.stdout: "standby_error_status_code": { 2024-09-19T08:57:50.750 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "standby_error_status_code", 2024-09-19T08:57:50.750 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.750 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.750 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.750 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "500", 2024-09-19T08:57:50.750 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "400", 2024-09-19T08:57:50.750 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "599", 2024-09-19T08:57:50.750 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.750 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.751 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.751 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.751 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.751 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.751 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.751 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rbd_support", 2024-09-19T08:57:50.751 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.751 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.751 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.751 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.751 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.752 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.752 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.752 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.752 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.752 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.752 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.752 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.752 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.752 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.752 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.752 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.752 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.752 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.753 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.753 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.753 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.753 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.753 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.753 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.753 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.753 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.753 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.753 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.753 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.753 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.753 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.754 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.754 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.754 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.754 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.754 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.754 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.754 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.754 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.754 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.754 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.754 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.754 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.755 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.755 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.755 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.755 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.755 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.755 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.755 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.755 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.755 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.755 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.755 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.755 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.755 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.756 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.756 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.756 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.756 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.756 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.756 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.756 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.756 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.756 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.756 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.756 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.756 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.757 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.757 INFO:teuthology.orchestra.run.smithi164.stdout: "max_concurrent_snap_create": { 2024-09-19T08:57:50.757 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "max_concurrent_snap_create", 2024-09-19T08:57:50.757 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.757 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.757 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.757 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "10", 2024-09-19T08:57:50.757 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.757 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.757 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.757 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.757 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.757 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.758 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.758 INFO:teuthology.orchestra.run.smithi164.stdout: "mirror_snapshot_schedule": { 2024-09-19T08:57:50.758 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T08:57:50.758 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.758 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.758 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.758 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.758 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.758 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.758 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.758 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.758 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.758 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.759 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.759 INFO:teuthology.orchestra.run.smithi164.stdout: "trash_purge_schedule": { 2024-09-19T08:57:50.759 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "trash_purge_schedule", 2024-09-19T08:57:50.759 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.759 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.759 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.759 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.759 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.759 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.759 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.759 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.759 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.760 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.760 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.760 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.760 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.760 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "restful", 2024-09-19T08:57:50.760 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.760 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.760 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.760 INFO:teuthology.orchestra.run.smithi164.stdout: "enable_auth": { 2024-09-19T08:57:50.760 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "enable_auth", 2024-09-19T08:57:50.760 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.760 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.761 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.761 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.761 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.761 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.761 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.761 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.761 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.761 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.761 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.761 INFO:teuthology.orchestra.run.smithi164.stdout: "key_file": { 2024-09-19T08:57:50.761 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "key_file", 2024-09-19T08:57:50.761 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.761 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.762 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.762 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.762 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.762 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.762 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.762 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.762 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.762 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.762 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.762 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.762 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.762 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.763 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.763 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.763 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.763 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.763 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.763 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.763 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.763 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.763 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.763 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.763 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.763 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.763 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.763 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.764 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.764 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.764 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.764 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.764 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.764 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.764 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.764 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.764 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.764 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.764 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.764 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.765 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.765 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.765 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.765 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.765 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.765 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.765 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.765 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.765 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.765 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.765 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.765 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.765 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.766 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.766 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.766 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.766 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.766 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.766 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.766 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.766 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.766 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.766 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.766 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.766 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.767 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.767 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.767 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.767 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.767 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.767 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.767 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.767 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.767 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.767 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.767 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.767 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.767 INFO:teuthology.orchestra.run.smithi164.stdout: "server_addr": { 2024-09-19T08:57:50.768 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "server_addr", 2024-09-19T08:57:50.768 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.768 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.768 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.768 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.768 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.768 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.768 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.768 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.768 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.768 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.768 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.769 INFO:teuthology.orchestra.run.smithi164.stdout: "server_port": { 2024-09-19T08:57:50.769 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "server_port", 2024-09-19T08:57:50.769 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.769 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.769 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.769 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.769 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.769 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.769 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.769 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.769 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.769 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.769 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rgw", 2024-09-19T08:57:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T08:57:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T08:57:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "5", 2024-09-19T08:57:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T08:57:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rook", 2024-09-19T08:57:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: "storage_class": { 2024-09-19T08:57:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "storage_class", 2024-09-19T08:57:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "local", 2024-09-19T08:57:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T08:57:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "selftest", 2024-09-19T08:57:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.785 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.785 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.785 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.785 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.785 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.785 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.785 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.785 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.785 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.785 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.785 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.785 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.786 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.786 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.786 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.786 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.786 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.786 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.786 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.786 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.786 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.786 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.786 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.786 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.786 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.787 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.787 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.787 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.787 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.787 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.787 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.787 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.787 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.787 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.787 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.787 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.787 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.787 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.788 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.788 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.788 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.788 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.788 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.788 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.788 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.788 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.788 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.788 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.788 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.788 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.789 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.789 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.789 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.789 INFO:teuthology.orchestra.run.smithi164.stdout: "roption1": { 2024-09-19T08:57:50.789 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "roption1", 2024-09-19T08:57:50.789 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.789 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.789 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.789 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.789 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.789 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.789 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.790 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.790 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.790 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.790 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.790 INFO:teuthology.orchestra.run.smithi164.stdout: "roption2": { 2024-09-19T08:57:50.790 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "roption2", 2024-09-19T08:57:50.790 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.790 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.790 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.790 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "xyz", 2024-09-19T08:57:50.790 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.790 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.791 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.791 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.791 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.791 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.791 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.791 INFO:teuthology.orchestra.run.smithi164.stdout: "rwoption1": { 2024-09-19T08:57:50.791 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rwoption1", 2024-09-19T08:57:50.791 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.791 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.791 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.791 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.791 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.792 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.792 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.792 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.792 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.792 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.792 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.792 INFO:teuthology.orchestra.run.smithi164.stdout: "rwoption2": { 2024-09-19T08:57:50.792 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rwoption2", 2024-09-19T08:57:50.792 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.792 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.792 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.792 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.793 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.793 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.793 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.793 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.793 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.793 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.793 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.793 INFO:teuthology.orchestra.run.smithi164.stdout: "rwoption3": { 2024-09-19T08:57:50.793 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rwoption3", 2024-09-19T08:57:50.793 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "float", 2024-09-19T08:57:50.793 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.793 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.793 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.794 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.794 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.794 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.794 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.794 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.794 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.794 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.794 INFO:teuthology.orchestra.run.smithi164.stdout: "rwoption4": { 2024-09-19T08:57:50.794 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rwoption4", 2024-09-19T08:57:50.794 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.794 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.794 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.795 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.795 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.795 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.795 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.795 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.795 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.795 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.795 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.795 INFO:teuthology.orchestra.run.smithi164.stdout: "rwoption5": { 2024-09-19T08:57:50.795 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rwoption5", 2024-09-19T08:57:50.795 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.795 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.795 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.796 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.796 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.796 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.796 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.796 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.796 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.796 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.796 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.796 INFO:teuthology.orchestra.run.smithi164.stdout: "rwoption6": { 2024-09-19T08:57:50.796 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rwoption6", 2024-09-19T08:57:50.796 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.796 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.796 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.797 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.797 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.797 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.797 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.797 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.797 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.797 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.797 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.797 INFO:teuthology.orchestra.run.smithi164.stdout: "rwoption7": { 2024-09-19T08:57:50.797 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rwoption7", 2024-09-19T08:57:50.797 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.797 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.798 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.798 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.798 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "1", 2024-09-19T08:57:50.798 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "42", 2024-09-19T08:57:50.798 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.798 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.798 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.798 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.798 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.798 INFO:teuthology.orchestra.run.smithi164.stdout: "testkey": { 2024-09-19T08:57:50.798 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "testkey", 2024-09-19T08:57:50.798 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.798 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.799 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.799 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.799 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.799 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.799 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.799 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.799 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.799 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.799 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.799 INFO:teuthology.orchestra.run.smithi164.stdout: "testlkey": { 2024-09-19T08:57:50.799 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "testlkey", 2024-09-19T08:57:50.799 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.799 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.800 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.800 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.800 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.800 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.800 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.800 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.800 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.800 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.800 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.800 INFO:teuthology.orchestra.run.smithi164.stdout: "testnewline": { 2024-09-19T08:57:50.800 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "testnewline", 2024-09-19T08:57:50.800 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.801 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.801 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.801 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.801 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.801 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.801 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.801 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.801 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.801 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.801 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.801 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.801 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.802 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "snap_schedule", 2024-09-19T08:57:50.802 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.802 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.802 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.802 INFO:teuthology.orchestra.run.smithi164.stdout: "allow_m_granularity": { 2024-09-19T08:57:50.802 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "allow_m_granularity", 2024-09-19T08:57:50.802 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.802 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.802 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.802 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.802 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.802 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.802 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.803 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T08:57:50.803 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.803 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.803 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.803 INFO:teuthology.orchestra.run.smithi164.stdout: "dump_on_update": { 2024-09-19T08:57:50.803 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "dump_on_update", 2024-09-19T08:57:50.803 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.803 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.803 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.803 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.803 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.803 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.804 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.804 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "dump database to debug log on update", 2024-09-19T08:57:50.804 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.804 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.804 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.804 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.804 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.804 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.804 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.804 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.804 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.804 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.805 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.805 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.805 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.805 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.805 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.805 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.805 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.805 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.805 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.805 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.805 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.805 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.805 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.806 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.806 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.806 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.806 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.806 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.806 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.806 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.806 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.806 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.806 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.806 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.806 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.806 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.807 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.807 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.807 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.807 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.807 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.807 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.807 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.807 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.807 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.807 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.807 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.807 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.807 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.808 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.808 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.808 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.808 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.808 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.808 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.808 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.808 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.808 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.808 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.808 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.808 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.809 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.809 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.809 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.809 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.809 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.809 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.809 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.809 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.809 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.809 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.809 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "stats", 2024-09-19T08:57:50.809 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.810 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.810 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.810 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.810 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.810 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.810 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.810 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.810 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.810 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.810 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.810 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.810 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.810 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.810 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.811 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.811 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.811 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.811 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.811 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.811 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.811 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.811 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.811 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.811 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.811 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.811 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.812 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.812 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.812 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.812 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.812 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.812 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.812 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.812 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.812 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.812 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.812 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.812 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.812 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.813 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.813 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.813 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.813 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.813 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.813 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.813 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.813 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.813 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.813 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.813 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.813 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.813 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.813 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.814 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.814 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.814 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.814 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.814 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.814 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.814 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.814 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.814 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.814 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.814 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.814 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.814 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.815 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.815 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.815 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.815 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.815 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "status", 2024-09-19T08:57:50.815 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.815 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.815 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.815 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.815 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.815 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.816 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.816 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.816 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.816 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.816 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.816 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.816 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.816 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.816 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.816 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.816 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.816 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.816 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.816 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.817 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.817 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.817 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.817 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.817 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.817 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.817 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.817 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.817 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.817 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.817 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.817 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.818 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.818 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.818 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.818 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.818 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.818 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.818 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.818 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.818 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.818 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.818 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.818 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.818 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.819 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.819 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.819 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.819 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.819 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.819 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.819 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.819 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.819 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.819 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.819 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.819 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.819 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.820 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.820 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.820 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.820 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.820 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.820 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.820 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.820 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.820 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.820 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.820 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.820 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.820 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.821 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "telegraf", 2024-09-19T08:57:50.821 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.821 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.821 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.821 INFO:teuthology.orchestra.run.smithi164.stdout: "address": { 2024-09-19T08:57:50.821 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "address", 2024-09-19T08:57:50.821 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.821 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.821 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.821 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T08:57:50.821 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.822 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.822 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.822 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.822 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.822 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.822 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.822 INFO:teuthology.orchestra.run.smithi164.stdout: "interval": { 2024-09-19T08:57:50.822 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "interval", 2024-09-19T08:57:50.822 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:50.822 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.822 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.822 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "15", 2024-09-19T08:57:50.822 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.823 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.823 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.823 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.823 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.823 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.823 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.823 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.823 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.823 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.823 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.823 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.823 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.824 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.824 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.824 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.824 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.824 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.824 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.824 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.824 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.824 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.824 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.824 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.824 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.824 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.825 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.825 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.825 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.825 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.825 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.825 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.825 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.825 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.825 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.825 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.825 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.825 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.825 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.826 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.826 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.826 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.826 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.826 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.826 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.826 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.826 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.826 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.826 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.826 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.826 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.826 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.827 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.827 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.827 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.827 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.827 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.827 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.827 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.827 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.827 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.827 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.827 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.827 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.828 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.828 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.828 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.828 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.828 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.828 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.828 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.828 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.828 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.828 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.828 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.828 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "telemetry", 2024-09-19T08:57:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: "channel_basic": { 2024-09-19T08:57:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "channel_basic", 2024-09-19T08:57:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T08:57:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: "channel_crash": { 2024-09-19T08:57:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "channel_crash", 2024-09-19T08:57:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T08:57:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.831 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.831 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.831 INFO:teuthology.orchestra.run.smithi164.stdout: "channel_device": { 2024-09-19T08:57:50.831 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "channel_device", 2024-09-19T08:57:50.831 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.831 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.831 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.831 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.831 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.831 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.831 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.831 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T08:57:50.832 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.832 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.832 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.832 INFO:teuthology.orchestra.run.smithi164.stdout: "channel_ident": { 2024-09-19T08:57:50.832 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "channel_ident", 2024-09-19T08:57:50.832 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.832 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.832 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.832 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.832 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.832 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.832 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.833 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T08:57:50.833 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.833 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.833 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.833 INFO:teuthology.orchestra.run.smithi164.stdout: "channel_perf": { 2024-09-19T08:57:50.833 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "channel_perf", 2024-09-19T08:57:50.833 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.833 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.833 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.833 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.833 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.833 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.834 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.834 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T08:57:50.834 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.834 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.834 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.834 INFO:teuthology.orchestra.run.smithi164.stdout: "contact": { 2024-09-19T08:57:50.834 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "contact", 2024-09-19T08:57:50.834 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.834 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.834 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.834 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.834 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.834 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.835 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.835 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.835 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.835 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.835 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.835 INFO:teuthology.orchestra.run.smithi164.stdout: "description": { 2024-09-19T08:57:50.835 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "description", 2024-09-19T08:57:50.835 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.835 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.835 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.835 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.835 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.835 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.836 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.836 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.836 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.836 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.836 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.836 INFO:teuthology.orchestra.run.smithi164.stdout: "device_url": { 2024-09-19T08:57:50.836 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "device_url", 2024-09-19T08:57:50.836 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.836 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.836 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.836 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T08:57:50.836 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.837 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.837 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.837 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.837 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.837 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.837 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.837 INFO:teuthology.orchestra.run.smithi164.stdout: "enabled": { 2024-09-19T08:57:50.837 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "enabled", 2024-09-19T08:57:50.837 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.837 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.837 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.837 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.838 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.838 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.838 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.838 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.838 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.838 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.838 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.838 INFO:teuthology.orchestra.run.smithi164.stdout: "interval": { 2024-09-19T08:57:50.838 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "interval", 2024-09-19T08:57:50.838 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.838 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.838 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.838 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "24", 2024-09-19T08:57:50.839 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "8", 2024-09-19T08:57:50.839 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.839 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.839 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.839 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.839 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.839 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.839 INFO:teuthology.orchestra.run.smithi164.stdout: "last_opt_revision": { 2024-09-19T08:57:50.839 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "last_opt_revision", 2024-09-19T08:57:50.839 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.839 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.839 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.840 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "1", 2024-09-19T08:57:50.840 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.840 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.840 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.840 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.840 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.840 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.840 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.840 INFO:teuthology.orchestra.run.smithi164.stdout: "leaderboard": { 2024-09-19T08:57:50.840 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "leaderboard", 2024-09-19T08:57:50.840 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.840 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.841 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.841 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.841 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.841 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.841 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.841 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.841 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.841 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.841 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.841 INFO:teuthology.orchestra.run.smithi164.stdout: "leaderboard_description": { 2024-09-19T08:57:50.841 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "leaderboard_description", 2024-09-19T08:57:50.841 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.841 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.842 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.842 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.842 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.842 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.842 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.842 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.842 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.842 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.842 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.842 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.842 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.842 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.843 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.843 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.843 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.843 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.843 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.843 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.843 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.843 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.843 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.843 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.843 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.843 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.843 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.843 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.844 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.844 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.844 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.844 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.844 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.844 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.844 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.844 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.844 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.844 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.844 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.844 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.845 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.845 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.845 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.845 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.845 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.845 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.845 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.845 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.845 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.845 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.845 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.845 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.845 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.846 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.846 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.846 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.846 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.846 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.846 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.846 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.846 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.846 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.846 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.846 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.846 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.847 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.847 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.847 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.847 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.847 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.847 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.847 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.847 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.847 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.847 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.847 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.847 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.847 INFO:teuthology.orchestra.run.smithi164.stdout: "organization": { 2024-09-19T08:57:50.848 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "organization", 2024-09-19T08:57:50.848 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.848 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.848 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.848 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.848 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.848 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.848 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.848 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.848 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.848 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.848 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.848 INFO:teuthology.orchestra.run.smithi164.stdout: "proxy": { 2024-09-19T08:57:50.849 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "proxy", 2024-09-19T08:57:50.849 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.849 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.849 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.849 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.849 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.849 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.849 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.849 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.849 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.849 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.849 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.850 INFO:teuthology.orchestra.run.smithi164.stdout: "url": { 2024-09-19T08:57:50.850 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "url", 2024-09-19T08:57:50.850 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.850 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.850 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.850 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T08:57:50.850 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.850 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.850 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.850 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.850 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.850 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.850 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.851 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.851 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.851 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "test_orchestrator", 2024-09-19T08:57:50.851 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.851 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.851 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.851 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.851 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.851 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.851 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.851 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.851 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.852 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.852 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.852 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.852 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.852 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.852 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.852 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.852 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.852 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.852 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.852 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.852 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.852 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.853 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.853 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.853 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.853 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.853 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.853 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.853 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.853 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.853 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.853 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.853 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.853 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.853 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.854 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.854 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.854 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.854 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.854 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.854 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.854 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.854 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.854 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.854 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.854 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.854 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.854 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.855 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.855 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.855 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.855 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.855 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.855 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.855 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.855 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.855 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.855 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.855 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.855 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.856 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.856 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.856 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.856 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.856 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.856 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.856 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.856 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.856 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.856 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.856 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.857 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "volumes", 2024-09-19T08:57:50.857 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.857 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.857 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.857 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.857 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.857 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.857 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.857 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.857 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.857 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.857 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.857 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.857 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.858 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.858 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.858 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.858 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.858 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.858 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.858 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.858 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.858 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.858 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.858 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.858 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.859 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.859 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.859 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.859 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.859 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.859 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.859 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.859 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.859 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.859 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.859 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.859 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.860 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.860 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.860 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.860 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.860 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.860 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.860 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.860 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.860 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.860 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.860 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.860 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.860 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.861 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.861 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.861 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.861 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.861 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.861 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.861 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.861 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.861 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.861 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.861 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.861 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.861 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.862 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.862 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.862 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.862 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.862 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.862 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.862 INFO:teuthology.orchestra.run.smithi164.stdout: "max_concurrent_clones": { 2024-09-19T08:57:50.862 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "max_concurrent_clones", 2024-09-19T08:57:50.862 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.862 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.862 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.862 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "4", 2024-09-19T08:57:50.863 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.863 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.863 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.863 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T08:57:50.863 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.863 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.863 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.863 INFO:teuthology.orchestra.run.smithi164.stdout: "periodic_async_work": { 2024-09-19T08:57:50.863 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "periodic_async_work", 2024-09-19T08:57:50.863 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.863 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.863 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.864 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.864 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.864 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.864 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.864 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Periodically check for async work", 2024-09-19T08:57:50.864 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.864 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.864 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.864 INFO:teuthology.orchestra.run.smithi164.stdout: "snapshot_clone_delay": { 2024-09-19T08:57:50.864 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "snapshot_clone_delay", 2024-09-19T08:57:50.864 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.864 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.865 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.865 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "0", 2024-09-19T08:57:50.865 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.865 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.865 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.865 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T08:57:50.865 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.865 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.865 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.865 INFO:teuthology.orchestra.run.smithi164.stdout: "snapshot_clone_no_wait": { 2024-09-19T08:57:50.865 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T08:57:50.865 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.865 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.866 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.866 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "True", 2024-09-19T08:57:50.866 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.866 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.866 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.866 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T08:57:50.866 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.866 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.866 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.866 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.866 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.867 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "zabbix", 2024-09-19T08:57:50.867 INFO:teuthology.orchestra.run.smithi164.stdout: "can_run": true, 2024-09-19T08:57:50.867 INFO:teuthology.orchestra.run.smithi164.stdout: "error_string": "", 2024-09-19T08:57:50.867 INFO:teuthology.orchestra.run.smithi164.stdout: "module_options": { 2024-09-19T08:57:50.867 INFO:teuthology.orchestra.run.smithi164.stdout: "discovery_interval": { 2024-09-19T08:57:50.867 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "discovery_interval", 2024-09-19T08:57:50.867 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "uint", 2024-09-19T08:57:50.867 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.867 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.867 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "100", 2024-09-19T08:57:50.867 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.867 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.867 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.867 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.868 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.868 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.868 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.868 INFO:teuthology.orchestra.run.smithi164.stdout: "identifier": { 2024-09-19T08:57:50.868 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "identifier", 2024-09-19T08:57:50.868 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.868 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.868 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.868 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.868 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.868 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.868 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.869 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.869 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.869 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.869 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.869 INFO:teuthology.orchestra.run.smithi164.stdout: "interval": { 2024-09-19T08:57:50.869 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "interval", 2024-09-19T08:57:50.869 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "secs", 2024-09-19T08:57:50.869 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.869 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.869 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "60", 2024-09-19T08:57:50.869 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.869 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.870 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.870 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.870 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.870 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.870 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.870 INFO:teuthology.orchestra.run.smithi164.stdout: "log_level": { 2024-09-19T08:57:50.870 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_level", 2024-09-19T08:57:50.870 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.870 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.870 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.870 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.870 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.871 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.871 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.871 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.871 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.871 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.871 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.871 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.871 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.871 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.871 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.871 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.871 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.871 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.872 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster": { 2024-09-19T08:57:50.872 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster", 2024-09-19T08:57:50.872 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.872 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.872 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.872 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.872 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.872 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.872 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.872 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.872 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.872 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.872 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.873 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_cluster_level": { 2024-09-19T08:57:50.873 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_cluster_level", 2024-09-19T08:57:50.873 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.873 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.873 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.873 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "info", 2024-09-19T08:57:50.873 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.873 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.873 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [ 2024-09-19T08:57:50.873 INFO:teuthology.orchestra.run.smithi164.stdout: "", 2024-09-19T08:57:50.873 INFO:teuthology.orchestra.run.smithi164.stdout: "critical", 2024-09-19T08:57:50.873 INFO:teuthology.orchestra.run.smithi164.stdout: "debug", 2024-09-19T08:57:50.874 INFO:teuthology.orchestra.run.smithi164.stdout: "error", 2024-09-19T08:57:50.874 INFO:teuthology.orchestra.run.smithi164.stdout: "info", 2024-09-19T08:57:50.874 INFO:teuthology.orchestra.run.smithi164.stdout: "warning" 2024-09-19T08:57:50.874 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.874 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.874 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.874 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.874 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.874 INFO:teuthology.orchestra.run.smithi164.stdout: "log_to_file": { 2024-09-19T08:57:50.874 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "log_to_file", 2024-09-19T08:57:50.874 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "bool", 2024-09-19T08:57:50.874 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.874 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 1, 2024-09-19T08:57:50.875 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "False", 2024-09-19T08:57:50.875 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.875 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.875 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.875 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.875 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.875 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.875 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.875 INFO:teuthology.orchestra.run.smithi164.stdout: "zabbix_host": { 2024-09-19T08:57:50.875 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "zabbix_host", 2024-09-19T08:57:50.875 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.875 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.876 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.876 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "", 2024-09-19T08:57:50.876 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.876 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.876 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.876 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.876 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.876 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.876 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.876 INFO:teuthology.orchestra.run.smithi164.stdout: "zabbix_port": { 2024-09-19T08:57:50.876 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "zabbix_port", 2024-09-19T08:57:50.876 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "int", 2024-09-19T08:57:50.876 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.877 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.877 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "10051", 2024-09-19T08:57:50.877 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.877 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.877 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.877 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.877 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.877 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.877 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.877 INFO:teuthology.orchestra.run.smithi164.stdout: "zabbix_sender": { 2024-09-19T08:57:50.877 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "zabbix_sender", 2024-09-19T08:57:50.877 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "str", 2024-09-19T08:57:50.877 INFO:teuthology.orchestra.run.smithi164.stdout: "level": "advanced", 2024-09-19T08:57:50.878 INFO:teuthology.orchestra.run.smithi164.stdout: "flags": 0, 2024-09-19T08:57:50.878 INFO:teuthology.orchestra.run.smithi164.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T08:57:50.878 INFO:teuthology.orchestra.run.smithi164.stdout: "min": "", 2024-09-19T08:57:50.878 INFO:teuthology.orchestra.run.smithi164.stdout: "max": "", 2024-09-19T08:57:50.878 INFO:teuthology.orchestra.run.smithi164.stdout: "enum_allowed": [], 2024-09-19T08:57:50.878 INFO:teuthology.orchestra.run.smithi164.stdout: "desc": "", 2024-09-19T08:57:50.878 INFO:teuthology.orchestra.run.smithi164.stdout: "long_desc": "", 2024-09-19T08:57:50.878 INFO:teuthology.orchestra.run.smithi164.stdout: "tags": [], 2024-09-19T08:57:50.878 INFO:teuthology.orchestra.run.smithi164.stdout: "see_also": [] 2024-09-19T08:57:50.878 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.878 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.878 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.878 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.879 INFO:teuthology.orchestra.run.smithi164.stdout: "services": { 2024-09-19T08:57:50.879 INFO:teuthology.orchestra.run.smithi164.stdout: "dashboard": "https://172.21.15.164:8443/" 2024-09-19T08:57:50.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.879 INFO:teuthology.orchestra.run.smithi164.stdout: "always_on_modules": { 2024-09-19T08:57:50.879 INFO:teuthology.orchestra.run.smithi164.stdout: "octopus": [ 2024-09-19T08:57:50.879 INFO:teuthology.orchestra.run.smithi164.stdout: "balancer", 2024-09-19T08:57:50.879 INFO:teuthology.orchestra.run.smithi164.stdout: "crash", 2024-09-19T08:57:50.879 INFO:teuthology.orchestra.run.smithi164.stdout: "devicehealth", 2024-09-19T08:57:50.879 INFO:teuthology.orchestra.run.smithi164.stdout: "orchestrator", 2024-09-19T08:57:50.879 INFO:teuthology.orchestra.run.smithi164.stdout: "pg_autoscaler", 2024-09-19T08:57:50.879 INFO:teuthology.orchestra.run.smithi164.stdout: "progress", 2024-09-19T08:57:50.879 INFO:teuthology.orchestra.run.smithi164.stdout: "rbd_support", 2024-09-19T08:57:50.879 INFO:teuthology.orchestra.run.smithi164.stdout: "status", 2024-09-19T08:57:50.879 INFO:teuthology.orchestra.run.smithi164.stdout: "telemetry", 2024-09-19T08:57:50.880 INFO:teuthology.orchestra.run.smithi164.stdout: "volumes" 2024-09-19T08:57:50.880 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.880 INFO:teuthology.orchestra.run.smithi164.stdout: "pacific": [ 2024-09-19T08:57:50.880 INFO:teuthology.orchestra.run.smithi164.stdout: "balancer", 2024-09-19T08:57:50.880 INFO:teuthology.orchestra.run.smithi164.stdout: "crash", 2024-09-19T08:57:50.880 INFO:teuthology.orchestra.run.smithi164.stdout: "devicehealth", 2024-09-19T08:57:50.880 INFO:teuthology.orchestra.run.smithi164.stdout: "orchestrator", 2024-09-19T08:57:50.880 INFO:teuthology.orchestra.run.smithi164.stdout: "pg_autoscaler", 2024-09-19T08:57:50.880 INFO:teuthology.orchestra.run.smithi164.stdout: "progress", 2024-09-19T08:57:50.880 INFO:teuthology.orchestra.run.smithi164.stdout: "rbd_support", 2024-09-19T08:57:50.880 INFO:teuthology.orchestra.run.smithi164.stdout: "status", 2024-09-19T08:57:50.880 INFO:teuthology.orchestra.run.smithi164.stdout: "telemetry", 2024-09-19T08:57:50.880 INFO:teuthology.orchestra.run.smithi164.stdout: "volumes" 2024-09-19T08:57:50.880 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.881 INFO:teuthology.orchestra.run.smithi164.stdout: "quincy": [ 2024-09-19T08:57:50.881 INFO:teuthology.orchestra.run.smithi164.stdout: "balancer", 2024-09-19T08:57:50.881 INFO:teuthology.orchestra.run.smithi164.stdout: "crash", 2024-09-19T08:57:50.881 INFO:teuthology.orchestra.run.smithi164.stdout: "devicehealth", 2024-09-19T08:57:50.881 INFO:teuthology.orchestra.run.smithi164.stdout: "orchestrator", 2024-09-19T08:57:50.881 INFO:teuthology.orchestra.run.smithi164.stdout: "pg_autoscaler", 2024-09-19T08:57:50.881 INFO:teuthology.orchestra.run.smithi164.stdout: "progress", 2024-09-19T08:57:50.881 INFO:teuthology.orchestra.run.smithi164.stdout: "rbd_support", 2024-09-19T08:57:50.881 INFO:teuthology.orchestra.run.smithi164.stdout: "status", 2024-09-19T08:57:50.881 INFO:teuthology.orchestra.run.smithi164.stdout: "telemetry", 2024-09-19T08:57:50.881 INFO:teuthology.orchestra.run.smithi164.stdout: "volumes" 2024-09-19T08:57:50.881 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.881 INFO:teuthology.orchestra.run.smithi164.stdout: "reef": [ 2024-09-19T08:57:50.881 INFO:teuthology.orchestra.run.smithi164.stdout: "balancer", 2024-09-19T08:57:50.882 INFO:teuthology.orchestra.run.smithi164.stdout: "crash", 2024-09-19T08:57:50.882 INFO:teuthology.orchestra.run.smithi164.stdout: "devicehealth", 2024-09-19T08:57:50.882 INFO:teuthology.orchestra.run.smithi164.stdout: "orchestrator", 2024-09-19T08:57:50.882 INFO:teuthology.orchestra.run.smithi164.stdout: "pg_autoscaler", 2024-09-19T08:57:50.882 INFO:teuthology.orchestra.run.smithi164.stdout: "progress", 2024-09-19T08:57:50.882 INFO:teuthology.orchestra.run.smithi164.stdout: "rbd_support", 2024-09-19T08:57:50.882 INFO:teuthology.orchestra.run.smithi164.stdout: "status", 2024-09-19T08:57:50.882 INFO:teuthology.orchestra.run.smithi164.stdout: "telemetry", 2024-09-19T08:57:50.882 INFO:teuthology.orchestra.run.smithi164.stdout: "volumes" 2024-09-19T08:57:50.882 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-19T08:57:50.882 INFO:teuthology.orchestra.run.smithi164.stdout: "squid": [ 2024-09-19T08:57:50.882 INFO:teuthology.orchestra.run.smithi164.stdout: "balancer", 2024-09-19T08:57:50.882 INFO:teuthology.orchestra.run.smithi164.stdout: "crash", 2024-09-19T08:57:50.882 INFO:teuthology.orchestra.run.smithi164.stdout: "devicehealth", 2024-09-19T08:57:50.883 INFO:teuthology.orchestra.run.smithi164.stdout: "orchestrator", 2024-09-19T08:57:50.883 INFO:teuthology.orchestra.run.smithi164.stdout: "pg_autoscaler", 2024-09-19T08:57:50.883 INFO:teuthology.orchestra.run.smithi164.stdout: "progress", 2024-09-19T08:57:50.883 INFO:teuthology.orchestra.run.smithi164.stdout: "rbd_support", 2024-09-19T08:57:50.883 INFO:teuthology.orchestra.run.smithi164.stdout: "status", 2024-09-19T08:57:50.883 INFO:teuthology.orchestra.run.smithi164.stdout: "telemetry", 2024-09-19T08:57:50.883 INFO:teuthology.orchestra.run.smithi164.stdout: "volumes" 2024-09-19T08:57:50.883 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-19T08:57:50.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.883 INFO:teuthology.orchestra.run.smithi164.stdout: "last_failure_osd_epoch": 83, 2024-09-19T08:57:50.883 INFO:teuthology.orchestra.run.smithi164.stdout: "active_clients": [ 2024-09-19T08:57:50.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.883 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "devicehealth", 2024-09-19T08:57:50.883 INFO:teuthology.orchestra.run.smithi164.stdout: "addrvec": [ 2024-09-19T08:57:50.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.884 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v2", 2024-09-19T08:57:50.884 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "172.21.15.164:0", 2024-09-19T08:57:50.884 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 2502233909 2024-09-19T08:57:50.884 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.884 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-19T08:57:50.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.884 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "libcephsqlite", 2024-09-19T08:57:50.884 INFO:teuthology.orchestra.run.smithi164.stdout: "addrvec": [ 2024-09-19T08:57:50.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.884 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v2", 2024-09-19T08:57:50.885 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "172.21.15.164:0", 2024-09-19T08:57:50.885 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 210168717 2024-09-19T08:57:50.885 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.885 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-19T08:57:50.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.885 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "rbd_support", 2024-09-19T08:57:50.885 INFO:teuthology.orchestra.run.smithi164.stdout: "addrvec": [ 2024-09-19T08:57:50.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.885 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v2", 2024-09-19T08:57:50.885 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "172.21.15.164:0", 2024-09-19T08:57:50.885 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 4192282312 2024-09-19T08:57:50.885 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.885 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-19T08:57:50.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.886 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "snap_schedule", 2024-09-19T08:57:50.886 INFO:teuthology.orchestra.run.smithi164.stdout: "addrvec": [ 2024-09-19T08:57:50.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.886 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v2", 2024-09-19T08:57:50.886 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "172.21.15.164:0", 2024-09-19T08:57:50.886 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 1945848040 2024-09-19T08:57:50.886 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.886 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-19T08:57:50.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.886 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "snap_schedule", 2024-09-19T08:57:50.886 INFO:teuthology.orchestra.run.smithi164.stdout: "addrvec": [ 2024-09-19T08:57:50.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.887 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v2", 2024-09-19T08:57:50.887 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "172.21.15.164:0", 2024-09-19T08:57:50.887 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 2205731912 2024-09-19T08:57:50.887 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.887 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-19T08:57:50.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:50.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.887 INFO:teuthology.orchestra.run.smithi164.stdout: "name": "volumes", 2024-09-19T08:57:50.887 INFO:teuthology.orchestra.run.smithi164.stdout: "addrvec": [ 2024-09-19T08:57:50.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-19T08:57:50.887 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v2", 2024-09-19T08:57:50.887 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "172.21.15.164:0", 2024-09-19T08:57:50.887 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 1931944677 2024-09-19T08:57:50.888 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.888 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-19T08:57:50.888 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:50.888 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-19T08:57:50.888 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:57:50.889 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:49 smithi170 ceph-mon[35072]: pgmap v375: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:57:50.889 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:49 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3271228841' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T08:57:50.889 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:49 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/1145203784' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T08:57:50.890 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:48 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/147126080' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:57:50.890 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:48 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3406641652' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T08:57:50.890 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:48 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3634992813' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T08:57:50.890 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:49 smithi173 ceph-mon[35306]: pgmap v375: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:57:50.890 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:49 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3271228841' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T08:57:50.890 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:49 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/1145203784' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T08:57:50.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.403+0000 7fb1ea5f6640 1 -- 172.21.15.164:0/1145203784 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb1cc0a33c0 msgr2=0x7fb1cc0a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:50.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.403+0000 7fb1ea5f6640 1 --2- 172.21.15.164:0/1145203784 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb1cc0a33c0 0x7fb1cc0a5880 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400abe0 tx=0x7fb1d400a2d0 comp rx=0 tx=0).stop 2024-09-19T08:57:50.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.403+0000 7fb1ea5f6640 1 -- 172.21.15.164:0/1145203784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1e410e360 msgr2=0x7fb1e419ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:57:50.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.403+0000 7fb1ea5f6640 1 --2- 172.21.15.164:0/1145203784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1e410e360 0x7fb1e419ee20 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc006010 tx=0x7fb1dc02f710 comp rx=0 tx=0).stop 2024-09-19T08:57:50.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.404+0000 7fb1ea5f6640 1 -- 172.21.15.164:0/1145203784 shutdown_connections 2024-09-19T08:57:50.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.404+0000 7fb1ea5f6640 1 --2- 172.21.15.164:0/1145203784 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fb1cc0a33c0 0x7fb1cc0a5880 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:50.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.404+0000 7fb1ea5f6640 1 --2- 172.21.15.164:0/1145203784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1e410e360 0x7fb1e419ee20 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:50.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.404+0000 7fb1ea5f6640 1 --2- 172.21.15.164:0/1145203784 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1e4074ec0 0x7fb1e419aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:50.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.404+0000 7fb1ea5f6640 1 --2- 172.21.15.164:0/1145203784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1e40745a0 0x7fb1e419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:57:50.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.404+0000 7fb1ea5f6640 1 -- 172.21.15.164:0/1145203784 >> 172.21.15.164:0/1145203784 conn(0x7fb1e40fd7c0 msgr2=0x7fb1e40fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:57:50.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.404+0000 7fb1ea5f6640 1 -- 172.21.15.164:0/1145203784 shutdown_connections 2024-09-19T08:57:50.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:57:49.404+0000 7fb1ea5f6640 1 -- 172.21.15.164:0/1145203784 wait complete. 2024-09-19T08:57:50.951 DEBUG:tasks.check_counter:Getting stats from a 2024-09-19T08:57:50.951 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-19T08:57:50.952 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:49 smithi164 ceph-mon[30928]: pgmap v375: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:57:50.952 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:49 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3271228841' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T08:57:50.952 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:49 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/1145203784' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T08:57:51.211 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:57:51.742 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:57:51.742 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T08:57:51.742 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 34, 2024-09-19T08:57:51.742 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 182, 2024-09-19T08:57:51.742 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 10996, 2024-09-19T08:57:51.742 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 362654, 2024-09-19T08:57:51.742 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 12, 2024-09-19T08:57:51.742 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 2, 2024-09-19T08:57:51.742 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.023383662, 2024-09-19T08:57:51.743 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.010724998, 2024-09-19T08:57:51.743 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.007270182, 2024-09-19T08:57:51.743 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.000595049, 2024-09-19T08:57:51.743 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:57:51.743 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 181, 2024-09-19T08:57:51.743 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.036063702, 2024-09-19T08:57:51.743 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000199246 2024-09-19T08:57:51.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.743 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:57:51.743 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 19, 2024-09-19T08:57:51.743 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000003206, 2024-09-19T08:57:51.743 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000168 2024-09-19T08:57:51.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.743 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 5824, 2024-09-19T08:57:51.744 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 352608 2024-09-19T08:57:51.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.744 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T08:57:51.744 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 1574, 2024-09-19T08:57:51.744 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 1345, 2024-09-19T08:57:51.744 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 2116193, 2024-09-19T08:57:51.744 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 574575, 2024-09-19T08:57:51.744 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 9, 2024-09-19T08:57:51.744 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 7, 2024-09-19T08:57:51.744 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.204801407, 2024-09-19T08:57:51.744 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.072710317, 2024-09-19T08:57:51.744 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.171651307, 2024-09-19T08:57:51.745 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.005511269, 2024-09-19T08:57:51.745 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:57:51.745 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1344, 2024-09-19T08:57:51.745 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.150445683, 2024-09-19T08:57:51.745 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000111938 2024-09-19T08:57:51.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.745 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:57:51.745 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2210, 2024-09-19T08:57:51.745 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000615284, 2024-09-19T08:57:51.745 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000278 2024-09-19T08:57:51.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.745 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 1676368, 2024-09-19T08:57:51.746 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 160960 2024-09-19T08:57:51.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.746 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T08:57:51.746 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 189, 2024-09-19T08:57:51.746 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 222, 2024-09-19T08:57:51.746 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 126214, 2024-09-19T08:57:51.746 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 690265, 2024-09-19T08:57:51.746 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 47, 2024-09-19T08:57:51.746 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 4, 2024-09-19T08:57:51.746 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.049990602, 2024-09-19T08:57:51.746 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.007352302, 2024-09-19T08:57:51.746 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.041675003, 2024-09-19T08:57:51.746 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.000368379, 2024-09-19T08:57:51.747 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:57:51.747 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 219, 2024-09-19T08:57:51.747 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.080435121, 2024-09-19T08:57:51.747 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000367283 2024-09-19T08:57:51.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.747 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:57:51.747 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 287, 2024-09-19T08:57:51.747 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000078998, 2024-09-19T08:57:51.747 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000275 2024-09-19T08:57:51.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.747 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 2816, 2024-09-19T08:57:51.747 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 129200 2024-09-19T08:57:51.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.747 INFO:teuthology.orchestra.run.smithi164.stdout: "cct": { 2024-09-19T08:57:51.748 INFO:teuthology.orchestra.run.smithi164.stdout: "total_workers": 1, 2024-09-19T08:57:51.748 INFO:teuthology.orchestra.run.smithi164.stdout: "unhealthy_workers": 0 2024-09-19T08:57:51.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.748 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-MDSRank": { 2024-09-19T08:57:51.748 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-09-19T08:57:51.748 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-09-19T08:57:51.748 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 49, 2024-09-19T08:57:51.748 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.001986779, 2024-09-19T08:57:51.748 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000040546 2024-09-19T08:57:51.748 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:51.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.748 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-PurgeQueue": { 2024-09-19T08:57:51.748 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-09-19T08:57:51.748 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-09-19T08:57:51.749 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2, 2024-09-19T08:57:51.749 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000008840, 2024-09-19T08:57:51.749 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000004420 2024-09-19T08:57:51.749 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:51.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.749 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": { 2024-09-19T08:57:51.749 INFO:teuthology.orchestra.run.smithi164.stdout: "request": 8, 2024-09-19T08:57:51.749 INFO:teuthology.orchestra.run.smithi164.stdout: "reply": 6, 2024-09-19T08:57:51.749 INFO:teuthology.orchestra.run.smithi164.stdout: "reply_latency": { 2024-09-19T08:57:51.749 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 6, 2024-09-19T08:57:51.749 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.002515943, 2024-09-19T08:57:51.749 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000419323 2024-09-19T08:57:51.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.750 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_reply": 0, 2024-09-19T08:57:51.750 INFO:teuthology.orchestra.run.smithi164.stdout: "forward": 2, 2024-09-19T08:57:51.750 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_complete": 0, 2024-09-19T08:57:51.750 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_keys": 0, 2024-09-19T08:57:51.750 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_commit": 11, 2024-09-19T08:57:51.750 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_split": 0, 2024-09-19T08:57:51.750 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_merge": 0, 2024-09-19T08:57:51.750 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes": 16, 2024-09-19T08:57:51.750 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_top": 0, 2024-09-19T08:57:51.750 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_bottom": 16, 2024-09-19T08:57:51.750 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pin_tail": 0, 2024-09-19T08:57:51.750 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pinned": 13, 2024-09-19T08:57:51.750 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_expired": 8, 2024-09-19T08:57:51.751 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_with_caps": 2, 2024-09-19T08:57:51.751 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": 2, 2024-09-19T08:57:51.751 INFO:teuthology.orchestra.run.smithi164.stdout: "subtrees": 3, 2024-09-19T08:57:51.751 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse": 42, 2024-09-19T08:57:51.751 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_hit": 35, 2024-09-19T08:57:51.751 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_forward": 2, 2024-09-19T08:57:51.751 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_discover": 5, 2024-09-19T08:57:51.751 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_dir_fetch": 0, 2024-09-19T08:57:51.751 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_remote_ino": 0, 2024-09-19T08:57:51.751 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_lock": 0, 2024-09-19T08:57:51.751 INFO:teuthology.orchestra.run.smithi164.stdout: "load_cent": 1000, 2024-09-19T08:57:51.751 INFO:teuthology.orchestra.run.smithi164.stdout: "q": 1, 2024-09-19T08:57:51.751 INFO:teuthology.orchestra.run.smithi164.stdout: "exported": 0, 2024-09-19T08:57:51.752 INFO:teuthology.orchestra.run.smithi164.stdout: "exported_inodes": 0, 2024-09-19T08:57:51.752 INFO:teuthology.orchestra.run.smithi164.stdout: "imported": 0, 2024-09-19T08:57:51.752 INFO:teuthology.orchestra.run.smithi164.stdout: "imported_inodes": 0, 2024-09-19T08:57:51.752 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_dir_fetch": 0, 2024-09-19T08:57:51.752 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_backtrace_fetch": 0, 2024-09-19T08:57:51.752 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_peer_discover": 0, 2024-09-19T08:57:51.752 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rfiles": 22, 2024-09-19T08:57:51.753 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rbytes": 10738142113, 2024-09-19T08:57:51.753 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rsnaps": 6, 2024-09-19T08:57:51.753 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T08:57:51.753 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_set_tag": 0, 2024-09-19T08:57:51.753 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T08:57:51.753 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_inotable_repaired": 0, 2024-09-19T08:57:51.753 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_inodes": 0, 2024-09-19T08:57:51.753 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T08:57:51.753 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T08:57:51.753 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_file_inodes": 0, 2024-09-19T08:57:51.753 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_inode_file_caps": 0, 2024-09-19T08:57:51.753 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_revoke": 6, 2024-09-19T08:57:51.753 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_grant": 0, 2024-09-19T08:57:51.754 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T08:57:51.754 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T08:57:51.754 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T08:57:51.754 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps": 6, 2024-09-19T08:57:51.754 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps_dirty": 0, 2024-09-19T08:57:51.754 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_cap_release": 0, 2024-09-19T08:57:51.754 INFO:teuthology.orchestra.run.smithi164.stdout: "process_request_cap_release": 0 2024-09-19T08:57:51.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.754 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_cache": { 2024-09-19T08:57:51.754 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update": 0, 2024-09-19T08:57:51.754 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update_receipt": 17, 2024-09-19T08:57:51.754 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_try_discover": 11, 2024-09-19T08:57:51.754 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_send_discover": 13, 2024-09-19T08:57:51.754 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_handle_discover": 0, 2024-09-19T08:57:51.755 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays": 0, 2024-09-19T08:57:51.755 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_delayed": 0, 2024-09-19T08:57:51.755 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_enqueuing": 0, 2024-09-19T08:57:51.755 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_created": 0, 2024-09-19T08:57:51.755 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_enqueued": 0, 2024-09-19T08:57:51.755 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_reintegrated": 0, 2024-09-19T08:57:51.755 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_migrated": 0, 2024-09-19T08:57:51.755 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_processing": 0, 2024-09-19T08:57:51.755 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_enqueued": 0, 2024-09-19T08:57:51.755 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_prioritized": 0, 2024-09-19T08:57:51.755 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_started": 0, 2024-09-19T08:57:51.755 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_completed": 0, 2024-09-19T08:57:51.755 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_path": 10, 2024-09-19T08:57:51.755 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_inode": 86, 2024-09-19T08:57:51.756 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T08:57:51.756 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_exportdir": 0, 2024-09-19T08:57:51.756 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_flush": 0, 2024-09-19T08:57:51.756 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragmentdir": 0, 2024-09-19T08:57:51.756 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragstats": 0, 2024-09-19T08:57:51.756 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_inodestats": 0 2024-09-19T08:57:51.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.756 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_log": { 2024-09-19T08:57:51.756 INFO:teuthology.orchestra.run.smithi164.stdout: "evlrg": 0, 2024-09-19T08:57:51.756 INFO:teuthology.orchestra.run.smithi164.stdout: "evadd": 7, 2024-09-19T08:57:51.756 INFO:teuthology.orchestra.run.smithi164.stdout: "evex": 0, 2024-09-19T08:57:51.756 INFO:teuthology.orchestra.run.smithi164.stdout: "evtrm": 0, 2024-09-19T08:57:51.756 INFO:teuthology.orchestra.run.smithi164.stdout: "ev": 7, 2024-09-19T08:57:51.757 INFO:teuthology.orchestra.run.smithi164.stdout: "evexg": 0, 2024-09-19T08:57:51.757 INFO:teuthology.orchestra.run.smithi164.stdout: "evexd": 0, 2024-09-19T08:57:51.757 INFO:teuthology.orchestra.run.smithi164.stdout: "segadd": 2, 2024-09-19T08:57:51.757 INFO:teuthology.orchestra.run.smithi164.stdout: "segex": 0, 2024-09-19T08:57:51.757 INFO:teuthology.orchestra.run.smithi164.stdout: "segtrm": 0, 2024-09-19T08:57:51.757 INFO:teuthology.orchestra.run.smithi164.stdout: "seg": 2, 2024-09-19T08:57:51.757 INFO:teuthology.orchestra.run.smithi164.stdout: "segmjr": 2, 2024-09-19T08:57:51.757 INFO:teuthology.orchestra.run.smithi164.stdout: "segexg": 0, 2024-09-19T08:57:51.757 INFO:teuthology.orchestra.run.smithi164.stdout: "segexd": 0, 2024-09-19T08:57:51.757 INFO:teuthology.orchestra.run.smithi164.stdout: "expos": 4194304, 2024-09-19T08:57:51.757 INFO:teuthology.orchestra.run.smithi164.stdout: "wrpos": 4196268, 2024-09-19T08:57:51.757 INFO:teuthology.orchestra.run.smithi164.stdout: "rdpos": 0, 2024-09-19T08:57:51.758 INFO:teuthology.orchestra.run.smithi164.stdout: "jlat": { 2024-09-19T08:57:51.758 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 6, 2024-09-19T08:57:51.758 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.144457903, 2024-09-19T08:57:51.758 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.024076317 2024-09-19T08:57:51.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.758 INFO:teuthology.orchestra.run.smithi164.stdout: "replayed": 0 2024-09-19T08:57:51.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.758 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_mem": { 2024-09-19T08:57:51.758 INFO:teuthology.orchestra.run.smithi164.stdout: "ino": 20, 2024-09-19T08:57:51.758 INFO:teuthology.orchestra.run.smithi164.stdout: "ino+": 24, 2024-09-19T08:57:51.758 INFO:teuthology.orchestra.run.smithi164.stdout: "ino-": 4, 2024-09-19T08:57:51.758 INFO:teuthology.orchestra.run.smithi164.stdout: "dir": 18, 2024-09-19T08:57:51.758 INFO:teuthology.orchestra.run.smithi164.stdout: "dir+": 25, 2024-09-19T08:57:51.759 INFO:teuthology.orchestra.run.smithi164.stdout: "dir-": 7, 2024-09-19T08:57:51.759 INFO:teuthology.orchestra.run.smithi164.stdout: "dn": 16, 2024-09-19T08:57:51.759 INFO:teuthology.orchestra.run.smithi164.stdout: "dn+": 24, 2024-09-19T08:57:51.759 INFO:teuthology.orchestra.run.smithi164.stdout: "dn-": 8, 2024-09-19T08:57:51.759 INFO:teuthology.orchestra.run.smithi164.stdout: "cap": 2, 2024-09-19T08:57:51.759 INFO:teuthology.orchestra.run.smithi164.stdout: "cap+": 6, 2024-09-19T08:57:51.759 INFO:teuthology.orchestra.run.smithi164.stdout: "cap-": 4, 2024-09-19T08:57:51.759 INFO:teuthology.orchestra.run.smithi164.stdout: "rss": 51848, 2024-09-19T08:57:51.759 INFO:teuthology.orchestra.run.smithi164.stdout: "heap": 223520 2024-09-19T08:57:51.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.759 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_server": { 2024-09-19T08:57:51.759 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_client_request": 11, 2024-09-19T08:57:51.759 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_server_request": 1, 2024-09-19T08:57:51.760 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_request": 8, 2024-09-19T08:57:51.760 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_session": 18, 2024-09-19T08:57:51.760 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_peer_request": 3, 2024-09-19T08:57:51.760 INFO:teuthology.orchestra.run.smithi164.stdout: "req_create_latency": { 2024-09-19T08:57:51.760 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:51.760 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:51.760 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:51.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.760 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getattr_latency": { 2024-09-19T08:57:51.760 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2, 2024-09-19T08:57:51.760 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.001759442, 2024-09-19T08:57:51.760 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000879721 2024-09-19T08:57:51.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.761 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getfilelock_latency": { 2024-09-19T08:57:51.761 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:51.761 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:51.761 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:51.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.761 INFO:teuthology.orchestra.run.smithi164.stdout: "req_link_latency": { 2024-09-19T08:57:51.761 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:51.761 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:51.761 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:51.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.761 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookup_latency": { 2024-09-19T08:57:51.761 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2, 2024-09-19T08:57:51.761 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000512728, 2024-09-19T08:57:51.761 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000256364 2024-09-19T08:57:51.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.762 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookuphash_latency": { 2024-09-19T08:57:51.762 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:51.762 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:51.762 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:51.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.762 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupino_latency": { 2024-09-19T08:57:51.762 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2, 2024-09-19T08:57:51.762 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000243773, 2024-09-19T08:57:51.762 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000121886 2024-09-19T08:57:51.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.762 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupname_latency": { 2024-09-19T08:57:51.763 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:51.763 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:51.763 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:51.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.763 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupparent_latency": { 2024-09-19T08:57:51.763 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:51.763 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:51.763 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:51.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.763 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupsnap_latency": { 2024-09-19T08:57:51.763 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:51.763 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:51.763 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:51.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.764 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lssnap_latency": { 2024-09-19T08:57:51.764 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:51.764 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:51.764 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:51.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.764 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mkdir_latency": { 2024-09-19T08:57:51.764 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:51.764 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:51.764 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:51.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.764 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mknod_latency": { 2024-09-19T08:57:51.764 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:51.764 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:51.765 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:51.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.765 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mksnap_latency": { 2024-09-19T08:57:51.765 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:51.765 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:51.765 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:51.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.765 INFO:teuthology.orchestra.run.smithi164.stdout: "req_open_latency": { 2024-09-19T08:57:51.765 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:51.765 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:51.765 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:51.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.766 INFO:teuthology.orchestra.run.smithi164.stdout: "req_readdir_latency": { 2024-09-19T08:57:51.766 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:51.766 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:51.766 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:51.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.766 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rename_latency": { 2024-09-19T08:57:51.766 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:51.766 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:51.766 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:51.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.766 INFO:teuthology.orchestra.run.smithi164.stdout: "req_renamesnap_latency": { 2024-09-19T08:57:51.766 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:51.766 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:51.767 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:51.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.767 INFO:teuthology.orchestra.run.smithi164.stdout: "req_snapdiff_latency": { 2024-09-19T08:57:51.767 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:51.767 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:51.767 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:51.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.767 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmdir_latency": { 2024-09-19T08:57:51.767 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:51.767 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:51.767 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:51.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.767 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmsnap_latency": { 2024-09-19T08:57:51.768 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:51.768 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:51.768 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:51.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.768 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmxattr_latency": { 2024-09-19T08:57:51.768 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:51.768 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:51.768 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:51.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.768 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setattr_latency": { 2024-09-19T08:57:51.768 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:51.768 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:51.768 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:51.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.769 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setdirlayout_latency": { 2024-09-19T08:57:51.769 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:51.769 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:51.769 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:51.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.769 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setfilelock_latency": { 2024-09-19T08:57:51.769 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:51.769 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:51.769 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:51.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.769 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setlayout_latency": { 2024-09-19T08:57:51.769 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:51.769 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:51.770 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:51.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.770 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setxattr_latency": { 2024-09-19T08:57:51.770 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:51.770 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:51.770 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:51.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.770 INFO:teuthology.orchestra.run.smithi164.stdout: "req_symlink_latency": { 2024-09-19T08:57:51.770 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:51.770 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:51.770 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:51.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.770 INFO:teuthology.orchestra.run.smithi164.stdout: "req_unlink_latency": { 2024-09-19T08:57:51.771 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:51.771 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:51.771 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:51.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.771 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_revoke_eviction": 0, 2024-09-19T08:57:51.771 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_acquisition_throttle": 0, 2024-09-19T08:57:51.771 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getvxattr_latency": { 2024-09-19T08:57:51.771 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:51.771 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:51.771 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:51.771 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:51.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.771 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_sessions": { 2024-09-19T08:57:51.771 INFO:teuthology.orchestra.run.smithi164.stdout: "session_count": 1, 2024-09-19T08:57:51.772 INFO:teuthology.orchestra.run.smithi164.stdout: "session_add": 3, 2024-09-19T08:57:51.772 INFO:teuthology.orchestra.run.smithi164.stdout: "session_remove": 2, 2024-09-19T08:57:51.772 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_open": 1, 2024-09-19T08:57:51.772 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_stale": 0, 2024-09-19T08:57:51.772 INFO:teuthology.orchestra.run.smithi164.stdout: "total_load": 5, 2024-09-19T08:57:51.772 INFO:teuthology.orchestra.run.smithi164.stdout: "average_load": 1, 2024-09-19T08:57:51.772 INFO:teuthology.orchestra.run.smithi164.stdout: "avg_session_uptime": 699, 2024-09-19T08:57:51.772 INFO:teuthology.orchestra.run.smithi164.stdout: "mdthresh_evicted": 0 2024-09-19T08:57:51.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.772 INFO:teuthology.orchestra.run.smithi164.stdout: "mempool": { 2024-09-19T08:57:51.772 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_bytes": 0, 2024-09-19T08:57:51.772 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_items": 0, 2024-09-19T08:57:51.773 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T08:57:51.773 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_items": 0, 2024-09-19T08:57:51.773 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T08:57:51.773 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_items": 0, 2024-09-19T08:57:51.773 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T08:57:51.773 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T08:57:51.773 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T08:57:51.773 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T08:57:51.773 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T08:57:51.773 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_items": 0, 2024-09-19T08:57:51.773 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T08:57:51.774 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T08:57:51.774 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_bytes": 0, 2024-09-19T08:57:51.774 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_items": 0, 2024-09-19T08:57:51.774 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_bytes": 0, 2024-09-19T08:57:51.774 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_items": 0, 2024-09-19T08:57:51.774 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T08:57:51.774 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T08:57:51.774 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T08:57:51.774 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T08:57:51.774 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T08:57:51.775 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_items": 0, 2024-09-19T08:57:51.775 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_bytes": 0, 2024-09-19T08:57:51.775 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_items": 0, 2024-09-19T08:57:51.775 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T08:57:51.775 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T08:57:51.775 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_bytes": 0, 2024-09-19T08:57:51.775 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_items": 0, 2024-09-19T08:57:51.775 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_bytes": 0, 2024-09-19T08:57:51.775 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_items": 0, 2024-09-19T08:57:51.775 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T08:57:51.776 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_items": 0, 2024-09-19T08:57:51.776 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T08:57:51.776 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_items": 0, 2024-09-19T08:57:51.776 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_bytes": 294327, 2024-09-19T08:57:51.776 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_items": 96, 2024-09-19T08:57:51.776 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_bytes": 0, 2024-09-19T08:57:51.776 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_items": 0, 2024-09-19T08:57:51.776 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_bytes": 0, 2024-09-19T08:57:51.777 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_items": 0, 2024-09-19T08:57:51.777 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_bytes": 0, 2024-09-19T08:57:51.777 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_items": 0, 2024-09-19T08:57:51.777 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_bytes": 0, 2024-09-19T08:57:51.777 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_items": 0, 2024-09-19T08:57:51.777 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_bytes": 7936, 2024-09-19T08:57:51.777 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_items": 175, 2024-09-19T08:57:51.777 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T08:57:51.777 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_items": 0, 2024-09-19T08:57:51.777 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_bytes": 0, 2024-09-19T08:57:51.777 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_items": 0, 2024-09-19T08:57:51.777 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_bytes": 82037, 2024-09-19T08:57:51.777 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_items": 339, 2024-09-19T08:57:51.777 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_bytes": 0, 2024-09-19T08:57:51.778 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_items": 0, 2024-09-19T08:57:51.778 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_bytes": 0, 2024-09-19T08:57:51.778 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_items": 0 2024-09-19T08:57:51.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.778 INFO:teuthology.orchestra.run.smithi164.stdout: "objecter": { 2024-09-19T08:57:51.778 INFO:teuthology.orchestra.run.smithi164.stdout: "op_active": 0, 2024-09-19T08:57:51.778 INFO:teuthology.orchestra.run.smithi164.stdout: "op_laggy": 0, 2024-09-19T08:57:51.778 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send": 46, 2024-09-19T08:57:51.778 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send_bytes": 12707, 2024-09-19T08:57:51.778 INFO:teuthology.orchestra.run.smithi164.stdout: "op_resend": 33, 2024-09-19T08:57:51.778 INFO:teuthology.orchestra.run.smithi164.stdout: "op_reply": 46, 2024-09-19T08:57:51.778 INFO:teuthology.orchestra.run.smithi164.stdout: "op_latency": { 2024-09-19T08:57:51.778 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 46, 2024-09-19T08:57:51.779 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.411993080, 2024-09-19T08:57:51.779 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.008956371 2024-09-19T08:57:51.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.779 INFO:teuthology.orchestra.run.smithi164.stdout: "op_inflight": 0, 2024-09-19T08:57:51.779 INFO:teuthology.orchestra.run.smithi164.stdout: "oplen_avg": { 2024-09-19T08:57:51.779 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 46, 2024-09-19T08:57:51.779 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 71 2024-09-19T08:57:51.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.779 INFO:teuthology.orchestra.run.smithi164.stdout: "op": 46, 2024-09-19T08:57:51.779 INFO:teuthology.orchestra.run.smithi164.stdout: "op_r": 0, 2024-09-19T08:57:51.779 INFO:teuthology.orchestra.run.smithi164.stdout: "op_w": 46, 2024-09-19T08:57:51.779 INFO:teuthology.orchestra.run.smithi164.stdout: "op_rmw": 0, 2024-09-19T08:57:51.779 INFO:teuthology.orchestra.run.smithi164.stdout: "op_pg": 0, 2024-09-19T08:57:51.779 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_stat": 0, 2024-09-19T08:57:51.780 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_create": 10, 2024-09-19T08:57:51.780 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_read": 0, 2024-09-19T08:57:51.780 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_write": 6, 2024-09-19T08:57:51.780 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writefull": 8, 2024-09-19T08:57:51.780 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writesame": 0, 2024-09-19T08:57:51.780 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_append": 0, 2024-09-19T08:57:51.780 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_zero": 0, 2024-09-19T08:57:51.780 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_truncate": 0, 2024-09-19T08:57:51.780 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_delete": 6, 2024-09-19T08:57:51.780 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_mapext": 0, 2024-09-19T08:57:51.780 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_sparse_read": 0, 2024-09-19T08:57:51.780 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_clonerange": 0, 2024-09-19T08:57:51.780 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_getxattr": 0, 2024-09-19T08:57:51.781 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_setxattr": 20, 2024-09-19T08:57:51.781 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_cmpxattr": 0, 2024-09-19T08:57:51.781 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_rmxattr": 0, 2024-09-19T08:57:51.781 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_resetxattrs": 0, 2024-09-19T08:57:51.781 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_call": 0, 2024-09-19T08:57:51.781 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_watch": 0, 2024-09-19T08:57:51.781 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_notify": 0, 2024-09-19T08:57:51.781 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T08:57:51.781 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls": 0, 2024-09-19T08:57:51.781 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls_filter": 0, 2024-09-19T08:57:51.781 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_other": 0, 2024-09-19T08:57:51.781 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_active": 0, 2024-09-19T08:57:51.781 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_send": 0, 2024-09-19T08:57:51.781 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_resend": 0, 2024-09-19T08:57:51.782 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_ping": 0, 2024-09-19T08:57:51.782 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_active": 0, 2024-09-19T08:57:51.782 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_send": 0, 2024-09-19T08:57:51.782 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_resend": 0, 2024-09-19T08:57:51.782 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_active": 0, 2024-09-19T08:57:51.782 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_send": 0, 2024-09-19T08:57:51.782 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_resend": 0, 2024-09-19T08:57:51.782 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_active": 0, 2024-09-19T08:57:51.782 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_send": 0, 2024-09-19T08:57:51.782 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_resend": 0, 2024-09-19T08:57:51.782 INFO:teuthology.orchestra.run.smithi164.stdout: "command_active": 0, 2024-09-19T08:57:51.782 INFO:teuthology.orchestra.run.smithi164.stdout: "command_send": 0, 2024-09-19T08:57:51.782 INFO:teuthology.orchestra.run.smithi164.stdout: "command_resend": 0, 2024-09-19T08:57:51.783 INFO:teuthology.orchestra.run.smithi164.stdout: "map_epoch": 95, 2024-09-19T08:57:51.783 INFO:teuthology.orchestra.run.smithi164.stdout: "map_full": 0, 2024-09-19T08:57:51.783 INFO:teuthology.orchestra.run.smithi164.stdout: "map_inc": 13, 2024-09-19T08:57:51.783 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_sessions": 10, 2024-09-19T08:57:51.783 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_open": 10, 2024-09-19T08:57:51.783 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_close": 0, 2024-09-19T08:57:51.783 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_laggy": 0, 2024-09-19T08:57:51.783 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_wr": 20, 2024-09-19T08:57:51.783 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_rd": 0, 2024-09-19T08:57:51.783 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_del": 1 2024-09-19T08:57:51.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.784 INFO:teuthology.orchestra.run.smithi164.stdout: "oft": { 2024-09-19T08:57:51.784 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_objs": 1, 2024-09-19T08:57:51.784 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_kv_pairs": 0, 2024-09-19T08:57:51.784 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_updates": 0, 2024-09-19T08:57:51.784 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_removes": 0 2024-09-19T08:57:51.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.784 INFO:teuthology.orchestra.run.smithi164.stdout: "osd-slow-ops": { 2024-09-19T08:57:51.784 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_ops_count": 86 2024-09-19T08:57:51.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.784 INFO:teuthology.orchestra.run.smithi164.stdout: "purge_queue": { 2024-09-19T08:57:51.784 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops": 0, 2024-09-19T08:57:51.784 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T08:57:51.784 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing": 0, 2024-09-19T08:57:51.785 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_high_water": 0, 2024-09-19T08:57:51.785 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed_ops": 0, 2024-09-19T08:57:51.785 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed": 0, 2024-09-19T08:57:51.785 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_item_in_journal": 0 2024-09-19T08:57:51.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.785 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T08:57:51.785 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-09-19T08:57:51.785 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-09-19T08:57:51.785 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-09-19T08:57:51.785 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 1793, 2024-09-19T08:57:51.785 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 2072601, 2024-09-19T08:57:51.785 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-09-19T08:57:51.785 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 1793, 2024-09-19T08:57:51.785 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-09-19T08:57:51.786 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-09-19T08:57:51.786 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 1793, 2024-09-19T08:57:51.786 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 2072601, 2024-09-19T08:57:51.786 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-09-19T08:57:51.786 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:51.786 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:51.786 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:51.786 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:51.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.786 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_bytes": { 2024-09-19T08:57:51.786 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-09-19T08:57:51.786 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-09-19T08:57:51.786 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-09-19T08:57:51.787 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-09-19T08:57:51.787 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-09-19T08:57:51.787 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-09-19T08:57:51.787 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-09-19T08:57:51.787 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 46, 2024-09-19T08:57:51.787 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 12707, 2024-09-19T08:57:51.787 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 46, 2024-09-19T08:57:51.787 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 12707, 2024-09-19T08:57:51.787 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-09-19T08:57:51.787 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:51.787 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:51.787 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:51.787 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:51.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.788 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_ops": { 2024-09-19T08:57:51.788 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-09-19T08:57:51.788 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 1024, 2024-09-19T08:57:51.788 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-09-19T08:57:51.788 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-09-19T08:57:51.788 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-09-19T08:57:51.788 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-09-19T08:57:51.788 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-09-19T08:57:51.788 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 46, 2024-09-19T08:57:51.788 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 46, 2024-09-19T08:57:51.788 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 46, 2024-09-19T08:57:51.788 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 46, 2024-09-19T08:57:51.789 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-09-19T08:57:51.789 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:51.789 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:51.789 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:51.789 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:51.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.789 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle": { 2024-09-19T08:57:51.789 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-09-19T08:57:51.789 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-09-19T08:57:51.789 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-09-19T08:57:51.789 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-09-19T08:57:51.789 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-09-19T08:57:51.789 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-09-19T08:57:51.790 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-09-19T08:57:51.790 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-09-19T08:57:51.790 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-09-19T08:57:51.790 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 0, 2024-09-19T08:57:51.790 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 0, 2024-09-19T08:57:51.790 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-09-19T08:57:51.790 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:51.790 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:51.790 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:51.790 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:51.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:51.790 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle-0x556eed3ec960": { 2024-09-19T08:57:51.790 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-09-19T08:57:51.790 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-09-19T08:57:51.791 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-09-19T08:57:51.791 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 7, 2024-09-19T08:57:51.791 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 1964, 2024-09-19T08:57:51.791 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-09-19T08:57:51.791 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 7, 2024-09-19T08:57:51.791 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-09-19T08:57:51.791 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-09-19T08:57:51.791 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 6, 2024-09-19T08:57:51.791 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 1964, 2024-09-19T08:57:51.791 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-09-19T08:57:51.791 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:51.791 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:51.791 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:51.792 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:51.792 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:51.792 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:57:52.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:51 smithi164 ceph-mon[30928]: pgmap v376: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:57:52.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:51 smithi170 ceph-mon[35072]: pgmap v376: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:57:52.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:51 smithi173 ceph-mon[35306]: pgmap v376: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:57:52.689 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-19T08:57:52.689 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-19T08:57:52.689 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-09-19T08:57:52.689 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=17 2024-09-19T08:57:52.690 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=6 2024-09-19T08:57:52.690 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-09-19T08:57:52.690 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T08:57:52.690 DEBUG:tasks.check_counter:Getting stats from d 2024-09-19T08:57:52.690 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-19T08:57:52.943 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:57:53.449 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:57:53.449 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T08:57:53.449 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 0, 2024-09-19T08:57:53.449 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 2, 2024-09-19T08:57:53.449 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 0, 2024-09-19T08:57:53.449 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 0, 2024-09-19T08:57:53.449 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 4, 2024-09-19T08:57:53.450 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 0, 2024-09-19T08:57:53.450 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.001145214, 2024-09-19T08:57:53.450 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.000000000, 2024-09-19T08:57:53.450 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.000222029, 2024-09-19T08:57:53.450 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T08:57:53.450 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:57:53.450 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:53.450 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:53.450 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:53.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:53.450 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:57:53.450 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:53.451 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:53.451 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:53.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:53.451 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T08:57:53.451 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T08:57:53.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:53.451 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T08:57:53.451 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 249, 2024-09-19T08:57:53.451 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 236, 2024-09-19T08:57:53.451 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 1595904, 2024-09-19T08:57:53.451 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 144512, 2024-09-19T08:57:53.451 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 2, 2024-09-19T08:57:53.451 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 1, 2024-09-19T08:57:53.452 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.052232062, 2024-09-19T08:57:53.452 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.019270597, 2024-09-19T08:57:53.452 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.046645836, 2024-09-19T08:57:53.452 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T08:57:53.452 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:57:53.452 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 236, 2024-09-19T08:57:53.452 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.023103569, 2024-09-19T08:57:53.452 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000097896 2024-09-19T08:57:53.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:53.452 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:57:53.452 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:53.452 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:53.453 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:53.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:53.453 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 1595904, 2024-09-19T08:57:53.453 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 144512 2024-09-19T08:57:53.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:53.453 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T08:57:53.453 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 2, 2024-09-19T08:57:53.453 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 179, 2024-09-19T08:57:53.453 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 320, 2024-09-19T08:57:53.453 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 188736, 2024-09-19T08:57:53.453 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 5, 2024-09-19T08:57:53.453 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 3, 2024-09-19T08:57:53.453 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.017273373, 2024-09-19T08:57:53.454 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.012412792, 2024-09-19T08:57:53.454 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.001672597, 2024-09-19T08:57:53.454 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T08:57:53.454 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:57:53.454 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 176, 2024-09-19T08:57:53.454 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.021011505, 2024-09-19T08:57:53.454 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000119383 2024-09-19T08:57:53.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:53.454 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:57:53.454 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:53.454 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:53.454 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:53.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:53.455 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T08:57:53.455 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 188736 2024-09-19T08:57:53.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:53.455 INFO:teuthology.orchestra.run.smithi164.stdout: "cct": { 2024-09-19T08:57:53.455 INFO:teuthology.orchestra.run.smithi164.stdout: "total_workers": 0, 2024-09-19T08:57:53.455 INFO:teuthology.orchestra.run.smithi164.stdout: "unhealthy_workers": 0 2024-09-19T08:57:53.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:53.455 INFO:teuthology.orchestra.run.smithi164.stdout: "mempool": { 2024-09-19T08:57:53.455 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_bytes": 0, 2024-09-19T08:57:53.455 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_items": 0, 2024-09-19T08:57:53.455 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T08:57:53.455 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_items": 0, 2024-09-19T08:57:53.456 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T08:57:53.456 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_items": 0, 2024-09-19T08:57:53.456 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T08:57:53.456 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T08:57:53.456 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T08:57:53.456 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T08:57:53.456 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T08:57:53.456 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_items": 0, 2024-09-19T08:57:53.456 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T08:57:53.457 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T08:57:53.457 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_bytes": 0, 2024-09-19T08:57:53.457 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_items": 0, 2024-09-19T08:57:53.457 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_bytes": 0, 2024-09-19T08:57:53.457 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_items": 0, 2024-09-19T08:57:53.457 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T08:57:53.457 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T08:57:53.457 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T08:57:53.457 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T08:57:53.457 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T08:57:53.457 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_items": 0, 2024-09-19T08:57:53.457 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_bytes": 0, 2024-09-19T08:57:53.457 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_items": 0, 2024-09-19T08:57:53.458 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T08:57:53.458 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T08:57:53.458 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_bytes": 0, 2024-09-19T08:57:53.458 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_items": 0, 2024-09-19T08:57:53.458 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_bytes": 0, 2024-09-19T08:57:53.458 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_items": 0, 2024-09-19T08:57:53.458 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T08:57:53.458 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_items": 0, 2024-09-19T08:57:53.458 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T08:57:53.458 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_items": 0, 2024-09-19T08:57:53.458 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_bytes": 148995, 2024-09-19T08:57:53.458 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_items": 31, 2024-09-19T08:57:53.458 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_bytes": 0, 2024-09-19T08:57:53.459 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_items": 0, 2024-09-19T08:57:53.459 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_bytes": 0, 2024-09-19T08:57:53.459 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_items": 0, 2024-09-19T08:57:53.459 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_bytes": 0, 2024-09-19T08:57:53.459 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_items": 0, 2024-09-19T08:57:53.459 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_bytes": 0, 2024-09-19T08:57:53.459 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_items": 0, 2024-09-19T08:57:53.459 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_bytes": 0, 2024-09-19T08:57:53.459 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_items": 0, 2024-09-19T08:57:53.459 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T08:57:53.459 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_items": 0, 2024-09-19T08:57:53.459 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_bytes": 0, 2024-09-19T08:57:53.459 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_items": 0, 2024-09-19T08:57:53.460 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_bytes": 928, 2024-09-19T08:57:53.460 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_items": 2, 2024-09-19T08:57:53.460 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_bytes": 0, 2024-09-19T08:57:53.460 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_items": 0, 2024-09-19T08:57:53.460 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_bytes": 0, 2024-09-19T08:57:53.460 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_items": 0 2024-09-19T08:57:53.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:53.460 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T08:57:53.460 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-09-19T08:57:53.460 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-09-19T08:57:53.460 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-09-19T08:57:53.460 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 247, 2024-09-19T08:57:53.460 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 1558304, 2024-09-19T08:57:53.461 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-09-19T08:57:53.461 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 247, 2024-09-19T08:57:53.461 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-09-19T08:57:53.461 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-09-19T08:57:53.461 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 247, 2024-09-19T08:57:53.461 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 1558304, 2024-09-19T08:57:53.461 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-09-19T08:57:53.461 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:53.461 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:53.461 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:53.462 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:53.462 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:53.462 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:57:54.146 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-19T08:57:54.146 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-19T08:57:54.146 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-19T08:57:54.146 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-19T08:57:54.146 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-19T08:57:54.146 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-19T08:57:54.147 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-19T08:57:54.147 DEBUG:tasks.check_counter:Getting stats from g 2024-09-19T08:57:54.147 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-19T08:57:54.281 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:53 smithi164 ceph-mon[30928]: pgmap v377: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:57:54.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:53 smithi170 ceph-mon[35072]: pgmap v377: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:57:54.394 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:57:54.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:53 smithi173 ceph-mon[35306]: pgmap v377: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:57:54.931 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:57:54.931 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T08:57:54.931 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 1091, 2024-09-19T08:57:54.931 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 1074, 2024-09-19T08:57:54.931 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 1933920, 2024-09-19T08:57:54.931 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 407292, 2024-09-19T08:57:54.931 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 5, 2024-09-19T08:57:54.932 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 1, 2024-09-19T08:57:54.932 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.172495482, 2024-09-19T08:57:54.932 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.061840013, 2024-09-19T08:57:54.932 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.081002390, 2024-09-19T08:57:54.932 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.032444311, 2024-09-19T08:57:54.932 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:57:54.932 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1073, 2024-09-19T08:57:54.932 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.170490594, 2024-09-19T08:57:54.932 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000158891 2024-09-19T08:57:54.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.932 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:57:54.932 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.932 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.933 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.933 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 1656384, 2024-09-19T08:57:54.933 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 144624 2024-09-19T08:57:54.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.933 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T08:57:54.933 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 1556, 2024-09-19T08:57:54.933 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 1731, 2024-09-19T08:57:54.933 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 397848, 2024-09-19T08:57:54.933 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 969429, 2024-09-19T08:57:54.933 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 7, 2024-09-19T08:57:54.933 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 5, 2024-09-19T08:57:54.933 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.141009220, 2024-09-19T08:57:54.934 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.041830634, 2024-09-19T08:57:54.934 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.035113735, 2024-09-19T08:57:54.934 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.054443741, 2024-09-19T08:57:54.934 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:57:54.934 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1728, 2024-09-19T08:57:54.934 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.161146363, 2024-09-19T08:57:54.934 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000093255 2024-09-19T08:57:54.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.934 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:57:54.934 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.934 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.934 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.935 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T08:57:54.935 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 475200 2024-09-19T08:57:54.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.935 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T08:57:54.935 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 8, 2024-09-19T08:57:54.935 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 8, 2024-09-19T08:57:54.935 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 4653, 2024-09-19T08:57:54.935 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 1732, 2024-09-19T08:57:54.935 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 5, 2024-09-19T08:57:54.935 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 4, 2024-09-19T08:57:54.935 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.003031752, 2024-09-19T08:57:54.935 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.000096985, 2024-09-19T08:57:54.936 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.002812411, 2024-09-19T08:57:54.936 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.000070625, 2024-09-19T08:57:54.936 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:57:54.936 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 7, 2024-09-19T08:57:54.936 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.005828786, 2024-09-19T08:57:54.936 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000832683 2024-09-19T08:57:54.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.936 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:57:54.936 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.936 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.936 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.936 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-19T08:57:54.937 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T08:57:54.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.937 INFO:teuthology.orchestra.run.smithi164.stdout: "cct": { 2024-09-19T08:57:54.937 INFO:teuthology.orchestra.run.smithi164.stdout: "total_workers": 1, 2024-09-19T08:57:54.937 INFO:teuthology.orchestra.run.smithi164.stdout: "unhealthy_workers": 0 2024-09-19T08:57:54.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.937 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-MDSRank": { 2024-09-19T08:57:54.937 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-09-19T08:57:54.937 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-09-19T08:57:54.937 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 3939, 2024-09-19T08:57:54.937 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.113787619, 2024-09-19T08:57:54.937 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000028887 2024-09-19T08:57:54.937 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:54.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.938 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-PurgeQueue": { 2024-09-19T08:57:54.938 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-09-19T08:57:54.938 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-09-19T08:57:54.938 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.938 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.938 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.938 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:54.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.938 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": { 2024-09-19T08:57:54.938 INFO:teuthology.orchestra.run.smithi164.stdout: "request": 0, 2024-09-19T08:57:54.938 INFO:teuthology.orchestra.run.smithi164.stdout: "reply": 0, 2024-09-19T08:57:54.938 INFO:teuthology.orchestra.run.smithi164.stdout: "reply_latency": { 2024-09-19T08:57:54.939 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.939 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.939 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.939 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_reply": 0, 2024-09-19T08:57:54.939 INFO:teuthology.orchestra.run.smithi164.stdout: "forward": 0, 2024-09-19T08:57:54.939 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_complete": 0, 2024-09-19T08:57:54.939 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_keys": 0, 2024-09-19T08:57:54.939 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_commit": 0, 2024-09-19T08:57:54.939 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_split": 0, 2024-09-19T08:57:54.939 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_merge": 0, 2024-09-19T08:57:54.939 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes": 0, 2024-09-19T08:57:54.939 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_top": 0, 2024-09-19T08:57:54.940 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_bottom": 0, 2024-09-19T08:57:54.940 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pin_tail": 0, 2024-09-19T08:57:54.940 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pinned": 0, 2024-09-19T08:57:54.940 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_expired": 0, 2024-09-19T08:57:54.940 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_with_caps": 0, 2024-09-19T08:57:54.940 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": 0, 2024-09-19T08:57:54.940 INFO:teuthology.orchestra.run.smithi164.stdout: "subtrees": 1, 2024-09-19T08:57:54.940 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse": 0, 2024-09-19T08:57:54.940 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_hit": 0, 2024-09-19T08:57:54.940 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_forward": 0, 2024-09-19T08:57:54.940 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_discover": 0, 2024-09-19T08:57:54.940 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_dir_fetch": 0, 2024-09-19T08:57:54.940 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_remote_ino": 0, 2024-09-19T08:57:54.941 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_lock": 0, 2024-09-19T08:57:54.941 INFO:teuthology.orchestra.run.smithi164.stdout: "load_cent": 0, 2024-09-19T08:57:54.941 INFO:teuthology.orchestra.run.smithi164.stdout: "q": 0, 2024-09-19T08:57:54.941 INFO:teuthology.orchestra.run.smithi164.stdout: "exported": 0, 2024-09-19T08:57:54.941 INFO:teuthology.orchestra.run.smithi164.stdout: "exported_inodes": 0, 2024-09-19T08:57:54.941 INFO:teuthology.orchestra.run.smithi164.stdout: "imported": 0, 2024-09-19T08:57:54.941 INFO:teuthology.orchestra.run.smithi164.stdout: "imported_inodes": 0, 2024-09-19T08:57:54.941 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_dir_fetch": 0, 2024-09-19T08:57:54.941 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_backtrace_fetch": 0, 2024-09-19T08:57:54.941 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_peer_discover": 0, 2024-09-19T08:57:54.941 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rfiles": 0, 2024-09-19T08:57:54.941 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rbytes": 0, 2024-09-19T08:57:54.941 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rsnaps": 0, 2024-09-19T08:57:54.942 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T08:57:54.942 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_set_tag": 0, 2024-09-19T08:57:54.942 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T08:57:54.942 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_inotable_repaired": 0, 2024-09-19T08:57:54.942 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_inodes": 0, 2024-09-19T08:57:54.942 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T08:57:54.942 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T08:57:54.942 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_file_inodes": 0, 2024-09-19T08:57:54.942 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_inode_file_caps": 0, 2024-09-19T08:57:54.942 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T08:57:54.942 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_grant": 0, 2024-09-19T08:57:54.942 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T08:57:54.942 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T08:57:54.942 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T08:57:54.943 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps": 0, 2024-09-19T08:57:54.943 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps_dirty": 0, 2024-09-19T08:57:54.943 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_cap_release": 0, 2024-09-19T08:57:54.943 INFO:teuthology.orchestra.run.smithi164.stdout: "process_request_cap_release": 0 2024-09-19T08:57:54.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.943 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_cache": { 2024-09-19T08:57:54.943 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update": 0, 2024-09-19T08:57:54.943 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update_receipt": 0, 2024-09-19T08:57:54.943 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_try_discover": 0, 2024-09-19T08:57:54.943 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_send_discover": 0, 2024-09-19T08:57:54.943 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_handle_discover": 0, 2024-09-19T08:57:54.943 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays": 0, 2024-09-19T08:57:54.943 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_delayed": 0, 2024-09-19T08:57:54.944 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_enqueuing": 0, 2024-09-19T08:57:54.944 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_created": 0, 2024-09-19T08:57:54.944 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_enqueued": 0, 2024-09-19T08:57:54.944 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_reintegrated": 0, 2024-09-19T08:57:54.944 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_migrated": 0, 2024-09-19T08:57:54.944 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_processing": 0, 2024-09-19T08:57:54.944 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_enqueued": 0, 2024-09-19T08:57:54.944 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_prioritized": 0, 2024-09-19T08:57:54.944 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_started": 0, 2024-09-19T08:57:54.944 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_completed": 0, 2024-09-19T08:57:54.944 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_path": 0, 2024-09-19T08:57:54.944 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_inode": 0, 2024-09-19T08:57:54.944 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T08:57:54.945 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_exportdir": 0, 2024-09-19T08:57:54.945 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_flush": 0, 2024-09-19T08:57:54.945 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragmentdir": 0, 2024-09-19T08:57:54.945 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragstats": 0, 2024-09-19T08:57:54.945 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_inodestats": 0 2024-09-19T08:57:54.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.945 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_log": { 2024-09-19T08:57:54.945 INFO:teuthology.orchestra.run.smithi164.stdout: "evlrg": 0, 2024-09-19T08:57:54.945 INFO:teuthology.orchestra.run.smithi164.stdout: "evadd": 0, 2024-09-19T08:57:54.945 INFO:teuthology.orchestra.run.smithi164.stdout: "evex": 0, 2024-09-19T08:57:54.945 INFO:teuthology.orchestra.run.smithi164.stdout: "evtrm": 0, 2024-09-19T08:57:54.945 INFO:teuthology.orchestra.run.smithi164.stdout: "ev": 7, 2024-09-19T08:57:54.945 INFO:teuthology.orchestra.run.smithi164.stdout: "evexg": 0, 2024-09-19T08:57:54.946 INFO:teuthology.orchestra.run.smithi164.stdout: "evexd": 0, 2024-09-19T08:57:54.946 INFO:teuthology.orchestra.run.smithi164.stdout: "segadd": 0, 2024-09-19T08:57:54.946 INFO:teuthology.orchestra.run.smithi164.stdout: "segex": 0, 2024-09-19T08:57:54.946 INFO:teuthology.orchestra.run.smithi164.stdout: "segtrm": 0, 2024-09-19T08:57:54.946 INFO:teuthology.orchestra.run.smithi164.stdout: "seg": 2, 2024-09-19T08:57:54.946 INFO:teuthology.orchestra.run.smithi164.stdout: "segmjr": 2, 2024-09-19T08:57:54.946 INFO:teuthology.orchestra.run.smithi164.stdout: "segexg": 0, 2024-09-19T08:57:54.946 INFO:teuthology.orchestra.run.smithi164.stdout: "segexd": 0, 2024-09-19T08:57:54.946 INFO:teuthology.orchestra.run.smithi164.stdout: "expos": 4194304, 2024-09-19T08:57:54.946 INFO:teuthology.orchestra.run.smithi164.stdout: "wrpos": 4196268, 2024-09-19T08:57:54.946 INFO:teuthology.orchestra.run.smithi164.stdout: "rdpos": 4196127, 2024-09-19T08:57:54.946 INFO:teuthology.orchestra.run.smithi164.stdout: "jlat": { 2024-09-19T08:57:54.946 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.946 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.947 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.947 INFO:teuthology.orchestra.run.smithi164.stdout: "replayed": 7 2024-09-19T08:57:54.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.947 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_mem": { 2024-09-19T08:57:54.947 INFO:teuthology.orchestra.run.smithi164.stdout: "ino": 3, 2024-09-19T08:57:54.947 INFO:teuthology.orchestra.run.smithi164.stdout: "ino+": 3, 2024-09-19T08:57:54.947 INFO:teuthology.orchestra.run.smithi164.stdout: "ino-": 0, 2024-09-19T08:57:54.947 INFO:teuthology.orchestra.run.smithi164.stdout: "dir": 1, 2024-09-19T08:57:54.947 INFO:teuthology.orchestra.run.smithi164.stdout: "dir+": 1, 2024-09-19T08:57:54.948 INFO:teuthology.orchestra.run.smithi164.stdout: "dir-": 0, 2024-09-19T08:57:54.948 INFO:teuthology.orchestra.run.smithi164.stdout: "dn": 0, 2024-09-19T08:57:54.948 INFO:teuthology.orchestra.run.smithi164.stdout: "dn+": 0, 2024-09-19T08:57:54.948 INFO:teuthology.orchestra.run.smithi164.stdout: "dn-": 0, 2024-09-19T08:57:54.948 INFO:teuthology.orchestra.run.smithi164.stdout: "cap": 0, 2024-09-19T08:57:54.948 INFO:teuthology.orchestra.run.smithi164.stdout: "cap+": 0, 2024-09-19T08:57:54.948 INFO:teuthology.orchestra.run.smithi164.stdout: "cap-": 0, 2024-09-19T08:57:54.948 INFO:teuthology.orchestra.run.smithi164.stdout: "rss": 49236, 2024-09-19T08:57:54.948 INFO:teuthology.orchestra.run.smithi164.stdout: "heap": 215328 2024-09-19T08:57:54.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.948 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_server": { 2024-09-19T08:57:54.948 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_client_request": 0, 2024-09-19T08:57:54.949 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_server_request": 0, 2024-09-19T08:57:54.949 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_request": 0, 2024-09-19T08:57:54.949 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_session": 0, 2024-09-19T08:57:54.949 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_peer_request": 0, 2024-09-19T08:57:54.949 INFO:teuthology.orchestra.run.smithi164.stdout: "req_create_latency": { 2024-09-19T08:57:54.949 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.949 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.949 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.949 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getattr_latency": { 2024-09-19T08:57:54.949 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.949 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.949 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.950 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getfilelock_latency": { 2024-09-19T08:57:54.950 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.950 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.950 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.950 INFO:teuthology.orchestra.run.smithi164.stdout: "req_link_latency": { 2024-09-19T08:57:54.950 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.950 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.950 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.950 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookup_latency": { 2024-09-19T08:57:54.950 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.950 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.951 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.951 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookuphash_latency": { 2024-09-19T08:57:54.951 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.951 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.951 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.951 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupino_latency": { 2024-09-19T08:57:54.951 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.951 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.951 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.951 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupname_latency": { 2024-09-19T08:57:54.952 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.952 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.952 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.952 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupparent_latency": { 2024-09-19T08:57:54.952 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.952 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.952 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.952 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupsnap_latency": { 2024-09-19T08:57:54.952 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.952 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.952 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.953 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lssnap_latency": { 2024-09-19T08:57:54.953 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.953 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.953 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.953 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mkdir_latency": { 2024-09-19T08:57:54.953 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.954 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.954 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.954 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mknod_latency": { 2024-09-19T08:57:54.954 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.954 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.954 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.954 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mksnap_latency": { 2024-09-19T08:57:54.955 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.955 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.955 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.955 INFO:teuthology.orchestra.run.smithi164.stdout: "req_open_latency": { 2024-09-19T08:57:54.955 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.955 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.955 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.955 INFO:teuthology.orchestra.run.smithi164.stdout: "req_readdir_latency": { 2024-09-19T08:57:54.955 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.955 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.955 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.956 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rename_latency": { 2024-09-19T08:57:54.956 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.956 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.956 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.956 INFO:teuthology.orchestra.run.smithi164.stdout: "req_renamesnap_latency": { 2024-09-19T08:57:54.956 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.956 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.956 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.956 INFO:teuthology.orchestra.run.smithi164.stdout: "req_snapdiff_latency": { 2024-09-19T08:57:54.956 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.957 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.957 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.957 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmdir_latency": { 2024-09-19T08:57:54.957 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.957 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.957 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.957 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmsnap_latency": { 2024-09-19T08:57:54.957 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.957 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.957 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.958 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmxattr_latency": { 2024-09-19T08:57:54.958 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.958 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.958 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.958 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setattr_latency": { 2024-09-19T08:57:54.958 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.958 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.958 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.958 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setdirlayout_latency": { 2024-09-19T08:57:54.958 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.958 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.959 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.959 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setfilelock_latency": { 2024-09-19T08:57:54.959 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.959 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.959 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.959 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setlayout_latency": { 2024-09-19T08:57:54.959 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.959 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.959 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.959 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setxattr_latency": { 2024-09-19T08:57:54.959 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.960 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.960 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.960 INFO:teuthology.orchestra.run.smithi164.stdout: "req_symlink_latency": { 2024-09-19T08:57:54.960 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.960 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.960 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.960 INFO:teuthology.orchestra.run.smithi164.stdout: "req_unlink_latency": { 2024-09-19T08:57:54.960 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.960 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.960 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.961 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_revoke_eviction": 0, 2024-09-19T08:57:54.961 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_acquisition_throttle": 0, 2024-09-19T08:57:54.961 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getvxattr_latency": { 2024-09-19T08:57:54.961 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.961 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.961 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.961 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:54.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.961 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_sessions": { 2024-09-19T08:57:54.961 INFO:teuthology.orchestra.run.smithi164.stdout: "session_count": 1, 2024-09-19T08:57:54.961 INFO:teuthology.orchestra.run.smithi164.stdout: "session_add": 3, 2024-09-19T08:57:54.961 INFO:teuthology.orchestra.run.smithi164.stdout: "session_remove": 2, 2024-09-19T08:57:54.962 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_open": 3, 2024-09-19T08:57:54.962 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_stale": 0, 2024-09-19T08:57:54.962 INFO:teuthology.orchestra.run.smithi164.stdout: "total_load": 0, 2024-09-19T08:57:54.962 INFO:teuthology.orchestra.run.smithi164.stdout: "average_load": 0, 2024-09-19T08:57:54.962 INFO:teuthology.orchestra.run.smithi164.stdout: "avg_session_uptime": 3716, 2024-09-19T08:57:54.962 INFO:teuthology.orchestra.run.smithi164.stdout: "mdthresh_evicted": 0 2024-09-19T08:57:54.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.962 INFO:teuthology.orchestra.run.smithi164.stdout: "mempool": { 2024-09-19T08:57:54.962 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_bytes": 0, 2024-09-19T08:57:54.962 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_items": 0, 2024-09-19T08:57:54.962 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T08:57:54.962 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_items": 0, 2024-09-19T08:57:54.962 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T08:57:54.962 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_items": 0, 2024-09-19T08:57:54.963 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T08:57:54.963 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T08:57:54.963 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T08:57:54.963 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T08:57:54.963 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T08:57:54.963 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_items": 0, 2024-09-19T08:57:54.963 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T08:57:54.963 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T08:57:54.963 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_bytes": 0, 2024-09-19T08:57:54.963 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_items": 0, 2024-09-19T08:57:54.963 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_bytes": 0, 2024-09-19T08:57:54.963 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_items": 0, 2024-09-19T08:57:54.963 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T08:57:54.964 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T08:57:54.964 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T08:57:54.964 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T08:57:54.964 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T08:57:54.964 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_items": 0, 2024-09-19T08:57:54.964 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_bytes": 0, 2024-09-19T08:57:54.964 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_items": 0, 2024-09-19T08:57:54.964 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T08:57:54.964 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T08:57:54.964 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_bytes": 0, 2024-09-19T08:57:54.964 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_items": 0, 2024-09-19T08:57:54.964 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_bytes": 0, 2024-09-19T08:57:54.964 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_items": 0, 2024-09-19T08:57:54.965 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T08:57:54.965 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_items": 0, 2024-09-19T08:57:54.965 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T08:57:54.965 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_items": 0, 2024-09-19T08:57:54.965 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_bytes": 197507, 2024-09-19T08:57:54.965 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_items": 57, 2024-09-19T08:57:54.965 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_bytes": 0, 2024-09-19T08:57:54.965 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_items": 0, 2024-09-19T08:57:54.965 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_bytes": 0, 2024-09-19T08:57:54.965 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_items": 0, 2024-09-19T08:57:54.965 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_bytes": 0, 2024-09-19T08:57:54.965 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_items": 0, 2024-09-19T08:57:54.966 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_bytes": 0, 2024-09-19T08:57:54.966 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_items": 0, 2024-09-19T08:57:54.966 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_bytes": 7936, 2024-09-19T08:57:54.966 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_items": 175, 2024-09-19T08:57:54.966 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T08:57:54.966 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_items": 0, 2024-09-19T08:57:54.966 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_bytes": 0, 2024-09-19T08:57:54.966 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_items": 0, 2024-09-19T08:57:54.966 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_bytes": 8512, 2024-09-19T08:57:54.966 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_items": 13, 2024-09-19T08:57:54.966 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_bytes": 0, 2024-09-19T08:57:54.966 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_items": 0, 2024-09-19T08:57:54.966 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_bytes": 0, 2024-09-19T08:57:54.966 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_items": 0 2024-09-19T08:57:54.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.967 INFO:teuthology.orchestra.run.smithi164.stdout: "objecter": { 2024-09-19T08:57:54.967 INFO:teuthology.orchestra.run.smithi164.stdout: "op_active": 0, 2024-09-19T08:57:54.967 INFO:teuthology.orchestra.run.smithi164.stdout: "op_laggy": 0, 2024-09-19T08:57:54.967 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send": 2384, 2024-09-19T08:57:54.967 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send_bytes": 21, 2024-09-19T08:57:54.967 INFO:teuthology.orchestra.run.smithi164.stdout: "op_resend": 3, 2024-09-19T08:57:54.967 INFO:teuthology.orchestra.run.smithi164.stdout: "op_reply": 2384, 2024-09-19T08:57:54.967 INFO:teuthology.orchestra.run.smithi164.stdout: "op_latency": { 2024-09-19T08:57:54.967 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2384, 2024-09-19T08:57:54.967 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 4.207929782, 2024-09-19T08:57:54.967 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.001765071 2024-09-19T08:57:54.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.968 INFO:teuthology.orchestra.run.smithi164.stdout: "op_inflight": 0, 2024-09-19T08:57:54.968 INFO:teuthology.orchestra.run.smithi164.stdout: "oplen_avg": { 2024-09-19T08:57:54.968 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2384, 2024-09-19T08:57:54.968 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 2385 2024-09-19T08:57:54.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.968 INFO:teuthology.orchestra.run.smithi164.stdout: "op": 2384, 2024-09-19T08:57:54.968 INFO:teuthology.orchestra.run.smithi164.stdout: "op_r": 2384, 2024-09-19T08:57:54.968 INFO:teuthology.orchestra.run.smithi164.stdout: "op_w": 0, 2024-09-19T08:57:54.968 INFO:teuthology.orchestra.run.smithi164.stdout: "op_rmw": 0, 2024-09-19T08:57:54.968 INFO:teuthology.orchestra.run.smithi164.stdout: "op_pg": 0, 2024-09-19T08:57:54.968 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_stat": 1549, 2024-09-19T08:57:54.968 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_create": 0, 2024-09-19T08:57:54.968 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_read": 833, 2024-09-19T08:57:54.969 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_write": 0, 2024-09-19T08:57:54.969 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writefull": 0, 2024-09-19T08:57:54.969 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writesame": 0, 2024-09-19T08:57:54.969 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_append": 0, 2024-09-19T08:57:54.969 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_zero": 0, 2024-09-19T08:57:54.969 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_truncate": 0, 2024-09-19T08:57:54.969 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_delete": 0, 2024-09-19T08:57:54.969 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_mapext": 0, 2024-09-19T08:57:54.969 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_sparse_read": 0, 2024-09-19T08:57:54.969 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_clonerange": 0, 2024-09-19T08:57:54.969 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_getxattr": 1, 2024-09-19T08:57:54.969 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_setxattr": 0, 2024-09-19T08:57:54.969 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_cmpxattr": 0, 2024-09-19T08:57:54.970 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_rmxattr": 0, 2024-09-19T08:57:54.970 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_resetxattrs": 0, 2024-09-19T08:57:54.970 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_call": 0, 2024-09-19T08:57:54.970 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_watch": 0, 2024-09-19T08:57:54.970 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_notify": 0, 2024-09-19T08:57:54.970 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T08:57:54.970 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls": 0, 2024-09-19T08:57:54.970 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls_filter": 0, 2024-09-19T08:57:54.970 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_other": 0, 2024-09-19T08:57:54.970 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_active": 0, 2024-09-19T08:57:54.970 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_send": 0, 2024-09-19T08:57:54.970 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_resend": 0, 2024-09-19T08:57:54.970 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_ping": 0, 2024-09-19T08:57:54.971 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_active": 0, 2024-09-19T08:57:54.971 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_send": 0, 2024-09-19T08:57:54.971 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_resend": 0, 2024-09-19T08:57:54.971 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_active": 0, 2024-09-19T08:57:54.971 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_send": 0, 2024-09-19T08:57:54.971 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_resend": 0, 2024-09-19T08:57:54.971 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_active": 0, 2024-09-19T08:57:54.971 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_send": 0, 2024-09-19T08:57:54.971 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_resend": 0, 2024-09-19T08:57:54.971 INFO:teuthology.orchestra.run.smithi164.stdout: "command_active": 0, 2024-09-19T08:57:54.971 INFO:teuthology.orchestra.run.smithi164.stdout: "command_send": 0, 2024-09-19T08:57:54.971 INFO:teuthology.orchestra.run.smithi164.stdout: "command_resend": 0, 2024-09-19T08:57:54.971 INFO:teuthology.orchestra.run.smithi164.stdout: "map_epoch": 95, 2024-09-19T08:57:54.971 INFO:teuthology.orchestra.run.smithi164.stdout: "map_full": 0, 2024-09-19T08:57:54.972 INFO:teuthology.orchestra.run.smithi164.stdout: "map_inc": 13, 2024-09-19T08:57:54.972 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_sessions": 6, 2024-09-19T08:57:54.972 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_open": 6, 2024-09-19T08:57:54.972 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_close": 0, 2024-09-19T08:57:54.972 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_laggy": 0, 2024-09-19T08:57:54.972 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_wr": 0, 2024-09-19T08:57:54.972 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_rd": 2, 2024-09-19T08:57:54.972 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_del": 0 2024-09-19T08:57:54.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.972 INFO:teuthology.orchestra.run.smithi164.stdout: "oft": { 2024-09-19T08:57:54.972 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_objs": 0, 2024-09-19T08:57:54.972 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_kv_pairs": 0, 2024-09-19T08:57:54.973 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_updates": 0, 2024-09-19T08:57:54.973 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_removes": 0 2024-09-19T08:57:54.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.973 INFO:teuthology.orchestra.run.smithi164.stdout: "osd-slow-ops": { 2024-09-19T08:57:54.973 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_ops_count": 0 2024-09-19T08:57:54.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.973 INFO:teuthology.orchestra.run.smithi164.stdout: "purge_queue": { 2024-09-19T08:57:54.973 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops": 0, 2024-09-19T08:57:54.973 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T08:57:54.973 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing": 0, 2024-09-19T08:57:54.973 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_high_water": 0, 2024-09-19T08:57:54.973 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed_ops": 0, 2024-09-19T08:57:54.973 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed": 0, 2024-09-19T08:57:54.974 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_item_in_journal": 0 2024-09-19T08:57:54.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.974 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T08:57:54.974 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-09-19T08:57:54.974 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-09-19T08:57:54.974 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-09-19T08:57:54.974 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 2651, 2024-09-19T08:57:54.974 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 2081225, 2024-09-19T08:57:54.974 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-09-19T08:57:54.974 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 2651, 2024-09-19T08:57:54.974 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-09-19T08:57:54.974 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-09-19T08:57:54.974 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 2651, 2024-09-19T08:57:54.975 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 2081225, 2024-09-19T08:57:54.975 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-09-19T08:57:54.975 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.975 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.975 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.975 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:54.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.975 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_bytes": { 2024-09-19T08:57:54.975 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-09-19T08:57:54.975 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-09-19T08:57:54.975 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-09-19T08:57:54.975 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-09-19T08:57:54.975 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-09-19T08:57:54.976 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-09-19T08:57:54.976 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-09-19T08:57:54.976 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 2384, 2024-09-19T08:57:54.976 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 1969, 2024-09-19T08:57:54.976 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 2384, 2024-09-19T08:57:54.976 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 1969, 2024-09-19T08:57:54.976 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-09-19T08:57:54.976 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.976 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.976 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.976 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:54.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.976 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_ops": { 2024-09-19T08:57:54.977 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-09-19T08:57:54.977 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 1024, 2024-09-19T08:57:54.977 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-09-19T08:57:54.977 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-09-19T08:57:54.977 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-09-19T08:57:54.977 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-09-19T08:57:54.977 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-09-19T08:57:54.977 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 2384, 2024-09-19T08:57:54.977 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 2384, 2024-09-19T08:57:54.977 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 2384, 2024-09-19T08:57:54.977 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 2384, 2024-09-19T08:57:54.977 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-09-19T08:57:54.977 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.978 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.978 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.978 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:54.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.978 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle": { 2024-09-19T08:57:54.978 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-09-19T08:57:54.978 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-09-19T08:57:54.978 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-09-19T08:57:54.978 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-09-19T08:57:54.978 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-09-19T08:57:54.978 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-09-19T08:57:54.978 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-09-19T08:57:54.978 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-09-19T08:57:54.979 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-09-19T08:57:54.979 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 0, 2024-09-19T08:57:54.979 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 0, 2024-09-19T08:57:54.979 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-09-19T08:57:54.979 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.979 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.979 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.979 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:54.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:54.979 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle-0x555f29eeeb40": { 2024-09-19T08:57:54.979 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-09-19T08:57:54.979 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-09-19T08:57:54.979 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-09-19T08:57:54.980 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-09-19T08:57:54.980 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-09-19T08:57:54.980 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-09-19T08:57:54.980 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-09-19T08:57:54.980 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-09-19T08:57:54.980 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-09-19T08:57:54.980 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 0, 2024-09-19T08:57:54.980 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 0, 2024-09-19T08:57:54.980 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-09-19T08:57:54.980 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:54.980 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:54.980 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:54.980 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:54.981 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:54.981 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:57:55.670 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-19T08:57:55.670 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-19T08:57:55.670 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-09-19T08:57:55.670 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-09-19T08:57:55.670 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-09-19T08:57:55.670 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-09-19T08:57:55.670 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T08:57:55.670 DEBUG:tasks.check_counter:Getting stats from j 2024-09-19T08:57:55.671 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-19T08:57:55.920 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:57:56.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:55 smithi164 ceph-mon[30928]: pgmap v378: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T08:57:56.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:55 smithi170 ceph-mon[35072]: pgmap v378: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T08:57:56.406 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-19T08:57:56.406 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T08:57:56.406 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 4, 2024-09-19T08:57:56.406 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 4, 2024-09-19T08:57:56.406 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 3024, 2024-09-19T08:57:56.406 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 448, 2024-09-19T08:57:56.406 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 4, 2024-09-19T08:57:56.406 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 0, 2024-09-19T08:57:56.406 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.003011984, 2024-09-19T08:57:56.406 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.000034844, 2024-09-19T08:57:56.407 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.002089626, 2024-09-19T08:57:56.407 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T08:57:56.407 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:57:56.407 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 3, 2024-09-19T08:57:56.407 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.003917824, 2024-09-19T08:57:56.407 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.001305941 2024-09-19T08:57:56.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:56.407 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:57:56.407 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:56.407 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:56.407 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:56.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:56.407 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-19T08:57:56.408 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T08:57:56.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:56.408 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T08:57:56.408 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 246, 2024-09-19T08:57:56.408 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 235, 2024-09-19T08:57:56.408 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 1593152, 2024-09-19T08:57:56.408 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 144704, 2024-09-19T08:57:56.408 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 2, 2024-09-19T08:57:56.408 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 1, 2024-09-19T08:57:56.408 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.056061310, 2024-09-19T08:57:56.408 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.018476644, 2024-09-19T08:57:56.408 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.051752027, 2024-09-19T08:57:56.409 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T08:57:56.409 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:57:56.409 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 234, 2024-09-19T08:57:56.409 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.023489931, 2024-09-19T08:57:56.409 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000100384 2024-09-19T08:57:56.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:56.409 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:57:56.409 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:56.409 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:56.409 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:56.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:56.409 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 1593152, 2024-09-19T08:57:56.409 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 144704 2024-09-19T08:57:56.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:56.410 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T08:57:56.410 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 2, 2024-09-19T08:57:56.410 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 181, 2024-09-19T08:57:56.410 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 320, 2024-09-19T08:57:56.410 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 189088, 2024-09-19T08:57:56.410 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 5, 2024-09-19T08:57:56.410 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 3, 2024-09-19T08:57:56.410 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.016280575, 2024-09-19T08:57:56.410 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.011959783, 2024-09-19T08:57:56.410 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.001943029, 2024-09-19T08:57:56.410 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T08:57:56.410 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:57:56.411 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 178, 2024-09-19T08:57:56.411 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.041104282, 2024-09-19T08:57:56.411 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000230922 2024-09-19T08:57:56.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:56.411 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:57:56.411 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:56.411 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:56.411 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:56.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:56.411 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T08:57:56.411 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 189088 2024-09-19T08:57:56.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:56.411 INFO:teuthology.orchestra.run.smithi164.stdout: "cct": { 2024-09-19T08:57:56.412 INFO:teuthology.orchestra.run.smithi164.stdout: "total_workers": 0, 2024-09-19T08:57:56.412 INFO:teuthology.orchestra.run.smithi164.stdout: "unhealthy_workers": 0 2024-09-19T08:57:56.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:56.412 INFO:teuthology.orchestra.run.smithi164.stdout: "mempool": { 2024-09-19T08:57:56.412 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_bytes": 0, 2024-09-19T08:57:56.412 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_items": 0, 2024-09-19T08:57:56.412 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T08:57:56.412 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_items": 0, 2024-09-19T08:57:56.412 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T08:57:56.412 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_items": 0, 2024-09-19T08:57:56.412 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T08:57:56.412 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T08:57:56.413 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T08:57:56.413 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T08:57:56.413 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T08:57:56.413 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_items": 0, 2024-09-19T08:57:56.413 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T08:57:56.413 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T08:57:56.413 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_bytes": 0, 2024-09-19T08:57:56.413 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_items": 0, 2024-09-19T08:57:56.413 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_bytes": 0, 2024-09-19T08:57:56.413 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_items": 0, 2024-09-19T08:57:56.413 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T08:57:56.413 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T08:57:56.413 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T08:57:56.414 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T08:57:56.414 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T08:57:56.414 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_items": 0, 2024-09-19T08:57:56.414 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_bytes": 0, 2024-09-19T08:57:56.414 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_items": 0, 2024-09-19T08:57:56.414 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T08:57:56.414 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T08:57:56.414 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_bytes": 0, 2024-09-19T08:57:56.414 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_items": 0, 2024-09-19T08:57:56.415 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_bytes": 0, 2024-09-19T08:57:56.415 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_items": 0, 2024-09-19T08:57:56.415 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T08:57:56.415 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_items": 0, 2024-09-19T08:57:56.415 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T08:57:56.415 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_items": 0, 2024-09-19T08:57:56.415 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_bytes": 149017, 2024-09-19T08:57:56.415 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_items": 31, 2024-09-19T08:57:56.415 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_bytes": 0, 2024-09-19T08:57:56.415 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_items": 0, 2024-09-19T08:57:56.415 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_bytes": 0, 2024-09-19T08:57:56.415 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_items": 0, 2024-09-19T08:57:56.415 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_bytes": 0, 2024-09-19T08:57:56.416 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_items": 0, 2024-09-19T08:57:56.416 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_bytes": 0, 2024-09-19T08:57:56.416 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_items": 0, 2024-09-19T08:57:56.416 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_bytes": 0, 2024-09-19T08:57:56.416 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_items": 0, 2024-09-19T08:57:56.416 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T08:57:56.416 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_items": 0, 2024-09-19T08:57:56.416 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_bytes": 0, 2024-09-19T08:57:56.416 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_items": 0, 2024-09-19T08:57:56.416 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_bytes": 928, 2024-09-19T08:57:56.416 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_items": 2, 2024-09-19T08:57:56.416 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_bytes": 0, 2024-09-19T08:57:56.416 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_items": 0, 2024-09-19T08:57:56.417 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_bytes": 0, 2024-09-19T08:57:56.417 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_items": 0 2024-09-19T08:57:56.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-19T08:57:56.417 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T08:57:56.417 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-09-19T08:57:56.417 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-09-19T08:57:56.417 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-09-19T08:57:56.417 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 248, 2024-09-19T08:57:56.417 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 1558434, 2024-09-19T08:57:56.417 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-09-19T08:57:56.417 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 248, 2024-09-19T08:57:56.417 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-09-19T08:57:56.417 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-09-19T08:57:56.418 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 248, 2024-09-19T08:57:56.418 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 1558434, 2024-09-19T08:57:56.418 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-09-19T08:57:56.418 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-09-19T08:57:56.418 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-09-19T08:57:56.418 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-09-19T08:57:56.418 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:56.418 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-19T08:57:56.418 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-19T08:57:56.419 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:55 smithi173 ceph-mon[35306]: pgmap v378: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T08:57:57.029 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-19T08:57:57.030 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-19T08:57:57.030 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-19T08:57:57.030 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-19T08:57:57.030 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-19T08:57:57.030 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-19T08:57:57.030 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-19T08:57:57.030 DEBUG:tasks.check_counter:Getting stats from b 2024-09-19T08:57:57.031 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-19T08:57:57.289 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T08:57:57.835 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-19T08:57:57.835 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T08:57:57.836 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 4069, 2024-09-19T08:57:57.836 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 4304, 2024-09-19T08:57:57.836 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 2804271, 2024-09-19T08:57:57.836 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 101211373, 2024-09-19T08:57:57.836 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 15, 2024-09-19T08:57:57.836 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 5, 2024-09-19T08:57:57.836 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.808216844, 2024-09-19T08:57:57.836 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.483412959, 2024-09-19T08:57:57.836 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.173075359, 2024-09-19T08:57:57.836 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.152084585, 2024-09-19T08:57:57.836 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:57:57.836 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 4302, 2024-09-19T08:57:57.837 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 1.108124794, 2024-09-19T08:57:57.837 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000257583 2024-09-19T08:57:57.837 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.837 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:57:57.837 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2505, 2024-09-19T08:57:57.837 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000569720, 2024-09-19T08:57:57.837 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000227 2024-09-19T08:57:57.837 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.837 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 1803824, 2024-09-19T08:57:57.837 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 552768 2024-09-19T08:57:57.837 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.837 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T08:57:57.837 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 152264, 2024-09-19T08:57:57.838 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 154182, 2024-09-19T08:57:57.838 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 60328643, 2024-09-19T08:57:57.838 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 196853499, 2024-09-19T08:57:57.838 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 17, 2024-09-19T08:57:57.838 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 7, 2024-09-19T08:57:57.838 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 8.655400395, 2024-09-19T08:57:57.838 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 3.993683464, 2024-09-19T08:57:57.838 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 12.266630742, 2024-09-19T08:57:57.838 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.320452791, 2024-09-19T08:57:57.838 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:57:57.838 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 154184, 2024-09-19T08:57:57.838 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 44.311771138, 2024-09-19T08:57:57.838 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000287395 2024-09-19T08:57:57.839 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.839 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:57:57.839 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 226144, 2024-09-19T08:57:57.839 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.053936877, 2024-09-19T08:57:57.839 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000238 2024-09-19T08:57:57.839 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.839 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 59203536, 2024-09-19T08:57:57.839 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 60047680 2024-09-19T08:57:57.839 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.839 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T08:57:57.839 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 4709, 2024-09-19T08:57:57.839 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 5112, 2024-09-19T08:57:57.839 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 1435246, 2024-09-19T08:57:57.840 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 157506756, 2024-09-19T08:57:57.840 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 166, 2024-09-19T08:57:57.840 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 9, 2024-09-19T08:57:57.840 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 1.029356233, 2024-09-19T08:57:57.840 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.596775542, 2024-09-19T08:57:57.840 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.220897164, 2024-09-19T08:57:57.840 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.219761622, 2024-09-19T08:57:57.840 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:57:57.840 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 5107, 2024-09-19T08:57:57.840 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.979588868, 2024-09-19T08:57:57.840 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000191812 2024-09-19T08:57:57.840 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.841 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:57:57.841 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 4255, 2024-09-19T08:57:57.841 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000872488, 2024-09-19T08:57:57.841 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000205 2024-09-19T08:57:57.841 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.841 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 42368, 2024-09-19T08:57:57.841 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 1216 2024-09-19T08:57:57.841 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.841 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2024-09-19T08:57:57.841 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 1, 2024-09-19T08:57:57.841 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2024-09-19T08:57:57.841 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.842 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-MDSRank": { 2024-09-19T08:57:57.842 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-09-19T08:57:57.842 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-09-19T08:57:57.842 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 3585, 2024-09-19T08:57:57.842 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 64.377766627, 2024-09-19T08:57:57.842 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.017957536 2024-09-19T08:57:57.842 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:57:57.842 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.842 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-PurgeQueue": { 2024-09-19T08:57:57.842 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-09-19T08:57:57.842 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-09-19T08:57:57.843 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2460, 2024-09-19T08:57:57.843 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.073489112, 2024-09-19T08:57:57.843 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000029873 2024-09-19T08:57:57.843 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:57:57.843 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.843 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": { 2024-09-19T08:57:57.843 INFO:teuthology.orchestra.run.smithi170.stdout: "request": 339, 2024-09-19T08:57:57.843 INFO:teuthology.orchestra.run.smithi170.stdout: "reply": 338, 2024-09-19T08:57:57.843 INFO:teuthology.orchestra.run.smithi170.stdout: "reply_latency": { 2024-09-19T08:57:57.843 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 338, 2024-09-19T08:57:57.843 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 6.112550506, 2024-09-19T08:57:57.843 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.018084468 2024-09-19T08:57:57.843 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.844 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_reply": 0, 2024-09-19T08:57:57.844 INFO:teuthology.orchestra.run.smithi170.stdout: "forward": 1, 2024-09-19T08:57:57.844 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_complete": 1, 2024-09-19T08:57:57.844 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_keys": 0, 2024-09-19T08:57:57.844 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_commit": 22, 2024-09-19T08:57:57.844 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_split": 0, 2024-09-19T08:57:57.844 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_merge": 0, 2024-09-19T08:57:57.844 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes": 120, 2024-09-19T08:57:57.844 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_top": 19, 2024-09-19T08:57:57.844 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_bottom": 90, 2024-09-19T08:57:57.844 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pin_tail": 11, 2024-09-19T08:57:57.844 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pinned": 92, 2024-09-19T08:57:57.844 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_expired": 6, 2024-09-19T08:57:57.844 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_with_caps": 2, 2024-09-19T08:57:57.845 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": 2, 2024-09-19T08:57:57.845 INFO:teuthology.orchestra.run.smithi170.stdout: "subtrees": 3, 2024-09-19T08:57:57.845 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse": 539, 2024-09-19T08:57:57.845 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_hit": 519, 2024-09-19T08:57:57.845 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_forward": 0, 2024-09-19T08:57:57.845 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_discover": 0, 2024-09-19T08:57:57.845 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_dir_fetch": 0, 2024-09-19T08:57:57.845 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_remote_ino": 0, 2024-09-19T08:57:57.845 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_lock": 0, 2024-09-19T08:57:57.845 INFO:teuthology.orchestra.run.smithi170.stdout: "load_cent": 0, 2024-09-19T08:57:57.845 INFO:teuthology.orchestra.run.smithi170.stdout: "q": 0, 2024-09-19T08:57:57.845 INFO:teuthology.orchestra.run.smithi170.stdout: "exported": 2, 2024-09-19T08:57:57.845 INFO:teuthology.orchestra.run.smithi170.stdout: "exported_inodes": 5, 2024-09-19T08:57:57.846 INFO:teuthology.orchestra.run.smithi170.stdout: "imported": 1, 2024-09-19T08:57:57.846 INFO:teuthology.orchestra.run.smithi170.stdout: "imported_inodes": 4, 2024-09-19T08:57:57.846 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_dir_fetch": 0, 2024-09-19T08:57:57.846 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_backtrace_fetch": 395, 2024-09-19T08:57:57.846 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_peer_discover": 0, 2024-09-19T08:57:57.846 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rfiles": 4, 2024-09-19T08:57:57.847 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rbytes": 234, 2024-09-19T08:57:57.847 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rsnaps": 6, 2024-09-19T08:57:57.847 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_fetch": 395, 2024-09-19T08:57:57.847 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_set_tag": 0, 2024-09-19T08:57:57.847 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T08:57:57.848 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_inotable_repaired": 0, 2024-09-19T08:57:57.848 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_inodes": 146, 2024-09-19T08:57:57.848 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_base_inodes": 15, 2024-09-19T08:57:57.848 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dirfrag_rstats": 131, 2024-09-19T08:57:57.848 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_file_inodes": 249, 2024-09-19T08:57:57.848 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_inode_file_caps": 2, 2024-09-19T08:57:57.848 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_revoke": 174, 2024-09-19T08:57:57.848 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_grant": 2973, 2024-09-19T08:57:57.848 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T08:57:57.848 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T08:57:57.848 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T08:57:57.848 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps": 146066, 2024-09-19T08:57:57.848 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps_dirty": 145911, 2024-09-19T08:57:57.849 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_cap_release": 6, 2024-09-19T08:57:57.849 INFO:teuthology.orchestra.run.smithi170.stdout: "process_request_cap_release": 49 2024-09-19T08:57:57.849 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.849 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_cache": { 2024-09-19T08:57:57.849 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update": 19, 2024-09-19T08:57:57.849 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update_receipt": 0, 2024-09-19T08:57:57.849 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_try_discover": 0, 2024-09-19T08:57:57.849 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_send_discover": 0, 2024-09-19T08:57:57.849 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_handle_discover": 47, 2024-09-19T08:57:57.849 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays": 4, 2024-09-19T08:57:57.850 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_delayed": 0, 2024-09-19T08:57:57.850 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_enqueuing": 0, 2024-09-19T08:57:57.850 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_created": 20, 2024-09-19T08:57:57.850 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_enqueued": 16, 2024-09-19T08:57:57.850 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_reintegrated": 0, 2024-09-19T08:57:57.850 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_migrated": 0, 2024-09-19T08:57:57.850 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_processing": 0, 2024-09-19T08:57:57.850 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_enqueued": 0, 2024-09-19T08:57:57.850 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_prioritized": 0, 2024-09-19T08:57:57.850 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_started": 0, 2024-09-19T08:57:57.851 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_completed": 0, 2024-09-19T08:57:57.851 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_path": 10, 2024-09-19T08:57:57.851 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_inode": 285, 2024-09-19T08:57:57.851 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_enqueue_scrub": 15, 2024-09-19T08:57:57.851 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_exportdir": 2, 2024-09-19T08:57:57.851 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_flush": 0, 2024-09-19T08:57:57.851 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragmentdir": 0, 2024-09-19T08:57:57.851 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragstats": 0, 2024-09-19T08:57:57.851 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_inodestats": 0 2024-09-19T08:57:57.851 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.851 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_log": { 2024-09-19T08:57:57.851 INFO:teuthology.orchestra.run.smithi170.stdout: "evlrg": 0, 2024-09-19T08:57:57.852 INFO:teuthology.orchestra.run.smithi170.stdout: "evadd": 146457, 2024-09-19T08:57:57.852 INFO:teuthology.orchestra.run.smithi170.stdout: "evex": 61451, 2024-09-19T08:57:57.852 INFO:teuthology.orchestra.run.smithi170.stdout: "evtrm": 61451, 2024-09-19T08:57:57.852 INFO:teuthology.orchestra.run.smithi170.stdout: "ev": 85006, 2024-09-19T08:57:57.852 INFO:teuthology.orchestra.run.smithi170.stdout: "evexg": 0, 2024-09-19T08:57:57.852 INFO:teuthology.orchestra.run.smithi170.stdout: "evexd": 12290, 2024-09-19T08:57:57.852 INFO:teuthology.orchestra.run.smithi170.stdout: "segadd": 206, 2024-09-19T08:57:57.852 INFO:teuthology.orchestra.run.smithi170.stdout: "segex": 78, 2024-09-19T08:57:57.852 INFO:teuthology.orchestra.run.smithi170.stdout: "segtrm": 78, 2024-09-19T08:57:57.852 INFO:teuthology.orchestra.run.smithi170.stdout: "seg": 128, 2024-09-19T08:57:57.852 INFO:teuthology.orchestra.run.smithi170.stdout: "segmjr": 7, 2024-09-19T08:57:57.852 INFO:teuthology.orchestra.run.smithi170.stdout: "segexg": 0, 2024-09-19T08:57:57.852 INFO:teuthology.orchestra.run.smithi170.stdout: "segexd": 18, 2024-09-19T08:57:57.853 INFO:teuthology.orchestra.run.smithi170.stdout: "expos": 159266902, 2024-09-19T08:57:57.854 INFO:teuthology.orchestra.run.smithi170.stdout: "wrpos": 471643212, 2024-09-19T08:57:57.854 INFO:teuthology.orchestra.run.smithi170.stdout: "rdpos": 0, 2024-09-19T08:57:57.854 INFO:teuthology.orchestra.run.smithi170.stdout: "jlat": { 2024-09-19T08:57:57.854 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 4501, 2024-09-19T08:57:57.854 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 593.802795410, 2024-09-19T08:57:57.854 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.131926859 2024-09-19T08:57:57.854 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.854 INFO:teuthology.orchestra.run.smithi170.stdout: "replayed": 0 2024-09-19T08:57:57.854 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.854 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_mem": { 2024-09-19T08:57:57.854 INFO:teuthology.orchestra.run.smithi170.stdout: "ino": 93, 2024-09-19T08:57:57.854 INFO:teuthology.orchestra.run.smithi170.stdout: "ino+": 129, 2024-09-19T08:57:57.855 INFO:teuthology.orchestra.run.smithi170.stdout: "ino-": 36, 2024-09-19T08:57:57.855 INFO:teuthology.orchestra.run.smithi170.stdout: "dir": 21, 2024-09-19T08:57:57.855 INFO:teuthology.orchestra.run.smithi170.stdout: "dir+": 22, 2024-09-19T08:57:57.855 INFO:teuthology.orchestra.run.smithi170.stdout: "dir-": 1, 2024-09-19T08:57:57.855 INFO:teuthology.orchestra.run.smithi170.stdout: "dn": 120, 2024-09-19T08:57:57.855 INFO:teuthology.orchestra.run.smithi170.stdout: "dn+": 142, 2024-09-19T08:57:57.855 INFO:teuthology.orchestra.run.smithi170.stdout: "dn-": 22, 2024-09-19T08:57:57.855 INFO:teuthology.orchestra.run.smithi170.stdout: "cap": 2, 2024-09-19T08:57:57.855 INFO:teuthology.orchestra.run.smithi170.stdout: "cap+": 52, 2024-09-19T08:57:57.855 INFO:teuthology.orchestra.run.smithi170.stdout: "cap-": 50, 2024-09-19T08:57:57.855 INFO:teuthology.orchestra.run.smithi170.stdout: "rss": 74608, 2024-09-19T08:57:57.855 INFO:teuthology.orchestra.run.smithi170.stdout: "heap": 231712 2024-09-19T08:57:57.855 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.855 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_server": { 2024-09-19T08:57:57.856 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_client_request": 469, 2024-09-19T08:57:57.856 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_server_request": 4, 2024-09-19T08:57:57.856 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_request": 339, 2024-09-19T08:57:57.856 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_session": 33, 2024-09-19T08:57:57.856 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_peer_request": 16, 2024-09-19T08:57:57.856 INFO:teuthology.orchestra.run.smithi170.stdout: "req_create_latency": { 2024-09-19T08:57:57.856 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 22, 2024-09-19T08:57:57.856 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.022405252, 2024-09-19T08:57:57.856 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001018420 2024-09-19T08:57:57.856 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.856 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getattr_latency": { 2024-09-19T08:57:57.856 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 144, 2024-09-19T08:57:57.856 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 3.236076542, 2024-09-19T08:57:57.857 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.022472753 2024-09-19T08:57:57.857 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.857 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getfilelock_latency": { 2024-09-19T08:57:57.857 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:57.857 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:57.857 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:57.857 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.857 INFO:teuthology.orchestra.run.smithi170.stdout: "req_link_latency": { 2024-09-19T08:57:57.857 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:57.857 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:57.857 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:57.857 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.858 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookup_latency": { 2024-09-19T08:57:57.858 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 96, 2024-09-19T08:57:57.858 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.892859587, 2024-09-19T08:57:57.858 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.009300620 2024-09-19T08:57:57.858 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.858 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookuphash_latency": { 2024-09-19T08:57:57.858 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:57.858 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:57.858 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:57.858 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.858 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupino_latency": { 2024-09-19T08:57:57.858 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:57.858 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:57.858 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:57.859 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.859 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupname_latency": { 2024-09-19T08:57:57.859 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:57.859 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:57.859 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:57.859 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.859 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupparent_latency": { 2024-09-19T08:57:57.859 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:57.859 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:57.859 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:57.859 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.859 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupsnap_latency": { 2024-09-19T08:57:57.859 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:57.859 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:57.860 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:57.860 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.860 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lssnap_latency": { 2024-09-19T08:57:57.860 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 12, 2024-09-19T08:57:57.860 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.002776235, 2024-09-19T08:57:57.860 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000231352 2024-09-19T08:57:57.860 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.860 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mkdir_latency": { 2024-09-19T08:57:57.860 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 10, 2024-09-19T08:57:57.860 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.014748352, 2024-09-19T08:57:57.860 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001474835 2024-09-19T08:57:57.860 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.861 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mknod_latency": { 2024-09-19T08:57:57.861 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:57.861 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:57.861 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:57.861 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.861 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mksnap_latency": { 2024-09-19T08:57:57.861 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 12, 2024-09-19T08:57:57.861 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 1.792388501, 2024-09-19T08:57:57.861 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.149365708 2024-09-19T08:57:57.861 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.861 INFO:teuthology.orchestra.run.smithi170.stdout: "req_open_latency": { 2024-09-19T08:57:57.861 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2, 2024-09-19T08:57:57.861 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000597690, 2024-09-19T08:57:57.862 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000298845 2024-09-19T08:57:57.862 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.862 INFO:teuthology.orchestra.run.smithi170.stdout: "req_readdir_latency": { 2024-09-19T08:57:57.862 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 4, 2024-09-19T08:57:57.862 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.014727365, 2024-09-19T08:57:57.862 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.003681841 2024-09-19T08:57:57.862 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.862 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rename_latency": { 2024-09-19T08:57:57.862 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2, 2024-09-19T08:57:57.862 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.002123997, 2024-09-19T08:57:57.862 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001061998 2024-09-19T08:57:57.862 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.862 INFO:teuthology.orchestra.run.smithi170.stdout: "req_renamesnap_latency": { 2024-09-19T08:57:57.862 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:57.863 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:57.863 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:57.863 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.863 INFO:teuthology.orchestra.run.smithi170.stdout: "req_snapdiff_latency": { 2024-09-19T08:57:57.863 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:57.863 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:57.863 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:57.863 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.863 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmdir_latency": { 2024-09-19T08:57:57.863 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 4, 2024-09-19T08:57:57.863 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.043608571, 2024-09-19T08:57:57.863 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.010902142 2024-09-19T08:57:57.863 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.864 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmsnap_latency": { 2024-09-19T08:57:57.864 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 6, 2024-09-19T08:57:57.864 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.050672061, 2024-09-19T08:57:57.864 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.008445343 2024-09-19T08:57:57.864 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.864 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmxattr_latency": { 2024-09-19T08:57:57.864 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:57.864 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:57.864 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:57.864 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.864 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setattr_latency": { 2024-09-19T08:57:57.864 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1, 2024-09-19T08:57:57.864 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000752540, 2024-09-19T08:57:57.865 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000752540 2024-09-19T08:57:57.865 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.865 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setdirlayout_latency": { 2024-09-19T08:57:57.865 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:57.865 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:57.865 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:57.865 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.865 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setfilelock_latency": { 2024-09-19T08:57:57.865 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:57.865 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:57.865 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:57.865 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.865 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setlayout_latency": { 2024-09-19T08:57:57.865 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:57.866 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:57.866 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:57.866 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.866 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setxattr_latency": { 2024-09-19T08:57:57.866 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 7, 2024-09-19T08:57:57.866 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.019108298, 2024-09-19T08:57:57.866 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002729756 2024-09-19T08:57:57.866 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.866 INFO:teuthology.orchestra.run.smithi170.stdout: "req_symlink_latency": { 2024-09-19T08:57:57.867 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:57.867 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:57.867 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:57.867 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.867 INFO:teuthology.orchestra.run.smithi170.stdout: "req_unlink_latency": { 2024-09-19T08:57:57.867 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 16, 2024-09-19T08:57:57.867 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.019705515, 2024-09-19T08:57:57.867 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001231594 2024-09-19T08:57:57.867 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.868 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_revoke_eviction": 0, 2024-09-19T08:57:57.868 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_acquisition_throttle": 0, 2024-09-19T08:57:57.868 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getvxattr_latency": { 2024-09-19T08:57:57.868 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:57.868 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:57.868 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:57.868 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:57:57.868 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.869 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_sessions": { 2024-09-19T08:57:57.869 INFO:teuthology.orchestra.run.smithi170.stdout: "session_count": 2, 2024-09-19T08:57:57.869 INFO:teuthology.orchestra.run.smithi170.stdout: "session_add": 5, 2024-09-19T08:57:57.869 INFO:teuthology.orchestra.run.smithi170.stdout: "session_remove": 3, 2024-09-19T08:57:57.869 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_open": 2, 2024-09-19T08:57:57.869 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_stale": 0, 2024-09-19T08:57:57.869 INFO:teuthology.orchestra.run.smithi170.stdout: "total_load": 66, 2024-09-19T08:57:57.869 INFO:teuthology.orchestra.run.smithi170.stdout: "average_load": 33, 2024-09-19T08:57:57.869 INFO:teuthology.orchestra.run.smithi170.stdout: "avg_session_uptime": 723, 2024-09-19T08:57:57.870 INFO:teuthology.orchestra.run.smithi170.stdout: "mdthresh_evicted": 0 2024-09-19T08:57:57.874 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.874 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2024-09-19T08:57:57.874 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2024-09-19T08:57:57.874 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2024-09-19T08:57:57.874 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T08:57:57.874 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2024-09-19T08:57:57.874 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T08:57:57.874 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2024-09-19T08:57:57.874 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T08:57:57.875 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T08:57:57.875 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T08:57:57.875 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T08:57:57.875 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T08:57:57.875 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2024-09-19T08:57:57.875 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T08:57:57.875 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T08:57:57.875 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2024-09-19T08:57:57.875 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2024-09-19T08:57:57.875 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2024-09-19T08:57:57.875 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2024-09-19T08:57:57.876 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T08:57:57.876 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T08:57:57.876 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T08:57:57.876 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T08:57:57.876 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T08:57:57.876 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2024-09-19T08:57:57.876 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2024-09-19T08:57:57.876 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2024-09-19T08:57:57.876 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T08:57:57.877 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T08:57:57.877 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2024-09-19T08:57:57.877 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2024-09-19T08:57:57.877 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2024-09-19T08:57:57.877 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2024-09-19T08:57:57.877 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T08:57:57.877 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2024-09-19T08:57:57.877 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T08:57:57.877 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2024-09-19T08:57:57.877 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 327733, 2024-09-19T08:57:57.877 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 133, 2024-09-19T08:57:57.877 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 0, 2024-09-19T08:57:57.877 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 0, 2024-09-19T08:57:57.877 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2024-09-19T08:57:57.878 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2024-09-19T08:57:57.878 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2024-09-19T08:57:57.878 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2024-09-19T08:57:57.878 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2024-09-19T08:57:57.878 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2024-09-19T08:57:57.878 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 7936, 2024-09-19T08:57:57.878 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 175, 2024-09-19T08:57:57.878 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T08:57:57.878 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2024-09-19T08:57:57.878 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2024-09-19T08:57:57.878 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2024-09-19T08:57:57.878 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 365945, 2024-09-19T08:57:57.878 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 2705, 2024-09-19T08:57:57.879 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2024-09-19T08:57:57.879 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2024-09-19T08:57:57.879 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2024-09-19T08:57:57.879 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2024-09-19T08:57:57.879 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.879 INFO:teuthology.orchestra.run.smithi170.stdout: "objecter": { 2024-09-19T08:57:57.879 INFO:teuthology.orchestra.run.smithi170.stdout: "op_active": 0, 2024-09-19T08:57:57.879 INFO:teuthology.orchestra.run.smithi170.stdout: "op_laggy": 0, 2024-09-19T08:57:57.879 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send": 8499, 2024-09-19T08:57:57.879 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send_bytes": 467599561, 2024-09-19T08:57:57.879 INFO:teuthology.orchestra.run.smithi170.stdout: "op_resend": 31, 2024-09-19T08:57:57.879 INFO:teuthology.orchestra.run.smithi170.stdout: "op_reply": 8499, 2024-09-19T08:57:57.879 INFO:teuthology.orchestra.run.smithi170.stdout: "op_latency": { 2024-09-19T08:57:57.879 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 8499, 2024-09-19T08:57:57.880 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 556.874016451, 2024-09-19T08:57:57.880 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.065522298 2024-09-19T08:57:57.880 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.880 INFO:teuthology.orchestra.run.smithi170.stdout: "op_inflight": 0, 2024-09-19T08:57:57.880 INFO:teuthology.orchestra.run.smithi170.stdout: "oplen_avg": { 2024-09-19T08:57:57.880 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 8499, 2024-09-19T08:57:57.880 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 8631 2024-09-19T08:57:57.880 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.880 INFO:teuthology.orchestra.run.smithi170.stdout: "op": 8499, 2024-09-19T08:57:57.880 INFO:teuthology.orchestra.run.smithi170.stdout: "op_r": 427, 2024-09-19T08:57:57.880 INFO:teuthology.orchestra.run.smithi170.stdout: "op_w": 8072, 2024-09-19T08:57:57.880 INFO:teuthology.orchestra.run.smithi170.stdout: "op_rmw": 0, 2024-09-19T08:57:57.880 INFO:teuthology.orchestra.run.smithi170.stdout: "op_pg": 0, 2024-09-19T08:57:57.881 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_stat": 0, 2024-09-19T08:57:57.881 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_create": 44, 2024-09-19T08:57:57.881 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_read": 16, 2024-09-19T08:57:57.881 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_write": 4502, 2024-09-19T08:57:57.881 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writefull": 48, 2024-09-19T08:57:57.881 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writesame": 0, 2024-09-19T08:57:57.881 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_append": 0, 2024-09-19T08:57:57.881 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_zero": 0, 2024-09-19T08:57:57.881 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_truncate": 0, 2024-09-19T08:57:57.881 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_delete": 3235, 2024-09-19T08:57:57.881 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_mapext": 0, 2024-09-19T08:57:57.881 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_sparse_read": 0, 2024-09-19T08:57:57.881 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_clonerange": 0, 2024-09-19T08:57:57.881 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_getxattr": 411, 2024-09-19T08:57:57.882 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_setxattr": 88, 2024-09-19T08:57:57.882 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_cmpxattr": 0, 2024-09-19T08:57:57.882 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_rmxattr": 0, 2024-09-19T08:57:57.882 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_resetxattrs": 0, 2024-09-19T08:57:57.882 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_call": 0, 2024-09-19T08:57:57.882 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_watch": 0, 2024-09-19T08:57:57.882 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_notify": 0, 2024-09-19T08:57:57.882 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T08:57:57.882 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls": 0, 2024-09-19T08:57:57.882 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls_filter": 0, 2024-09-19T08:57:57.882 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_other": 0, 2024-09-19T08:57:57.882 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_active": 0, 2024-09-19T08:57:57.882 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_send": 0, 2024-09-19T08:57:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_resend": 0, 2024-09-19T08:57:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_ping": 0, 2024-09-19T08:57:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_active": 0, 2024-09-19T08:57:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_send": 0, 2024-09-19T08:57:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_resend": 0, 2024-09-19T08:57:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_active": 0, 2024-09-19T08:57:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_send": 0, 2024-09-19T08:57:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_resend": 0, 2024-09-19T08:57:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_active": 0, 2024-09-19T08:57:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_send": 0, 2024-09-19T08:57:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_resend": 0, 2024-09-19T08:57:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: "command_active": 0, 2024-09-19T08:57:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: "command_send": 0, 2024-09-19T08:57:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: "command_resend": 0, 2024-09-19T08:57:57.884 INFO:teuthology.orchestra.run.smithi170.stdout: "map_epoch": 95, 2024-09-19T08:57:57.884 INFO:teuthology.orchestra.run.smithi170.stdout: "map_full": 0, 2024-09-19T08:57:57.884 INFO:teuthology.orchestra.run.smithi170.stdout: "map_inc": 15, 2024-09-19T08:57:57.884 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_sessions": 12, 2024-09-19T08:57:57.884 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_open": 12, 2024-09-19T08:57:57.884 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_close": 0, 2024-09-19T08:57:57.884 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_laggy": 0, 2024-09-19T08:57:57.884 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_wr": 268, 2024-09-19T08:57:57.884 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_rd": 2, 2024-09-19T08:57:57.884 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_del": 17 2024-09-19T08:57:57.884 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.884 INFO:teuthology.orchestra.run.smithi170.stdout: "oft": { 2024-09-19T08:57:57.884 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_objs": 1, 2024-09-19T08:57:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_kv_pairs": 0, 2024-09-19T08:57:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_updates": 0, 2024-09-19T08:57:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_removes": 0 2024-09-19T08:57:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: "osd-slow-ops": { 2024-09-19T08:57:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_ops_count": 260 2024-09-19T08:57:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: "purge_queue": { 2024-09-19T08:57:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops": 0, 2024-09-19T08:57:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops_high_water": 518, 2024-09-19T08:57:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing": 0, 2024-09-19T08:57:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_high_water": 6, 2024-09-19T08:57:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed_ops": 2576, 2024-09-19T08:57:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed": 16, 2024-09-19T08:57:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_item_in_journal": 0 2024-09-19T08:57:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T08:57:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-09-19T08:57:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-09-19T08:57:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-09-19T08:57:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 161038, 2024-09-19T08:57:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 42655171, 2024-09-19T08:57:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-09-19T08:57:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 161038, 2024-09-19T08:57:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-09-19T08:57:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-09-19T08:57:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 161038, 2024-09-19T08:57:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 42655171, 2024-09-19T08:57:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-09-19T08:57:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:57:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_bytes": { 2024-09-19T08:57:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-09-19T08:57:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-09-19T08:57:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-09-19T08:57:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-09-19T08:57:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-09-19T08:57:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-09-19T08:57:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-09-19T08:57:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 8499, 2024-09-19T08:57:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 467601929, 2024-09-19T08:57:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 8499, 2024-09-19T08:57:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 467601929, 2024-09-19T08:57:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-09-19T08:57:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:57:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_ops": { 2024-09-19T08:57:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-09-19T08:57:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 1024, 2024-09-19T08:57:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-09-19T08:57:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-09-19T08:57:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-09-19T08:57:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-09-19T08:57:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-09-19T08:57:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 8499, 2024-09-19T08:57:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 8499, 2024-09-19T08:57:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 8499, 2024-09-19T08:57:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 8499, 2024-09-19T08:57:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-09-19T08:57:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:57:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle": { 2024-09-19T08:57:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-09-19T08:57:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-09-19T08:57:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-09-19T08:57:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 16, 2024-09-19T08:57:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 2384, 2024-09-19T08:57:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-09-19T08:57:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 16, 2024-09-19T08:57:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-09-19T08:57:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-09-19T08:57:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 1, 2024-09-19T08:57:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 2384, 2024-09-19T08:57:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-09-19T08:57:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:57:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle-0x5564470fa960": { 2024-09-19T08:57:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-09-19T08:57:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-09-19T08:57:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-09-19T08:57:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 146457, 2024-09-19T08:57:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 467448908, 2024-09-19T08:57:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-09-19T08:57:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 146457, 2024-09-19T08:57:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-09-19T08:57:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-09-19T08:57:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 4501, 2024-09-19T08:57:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 467448908, 2024-09-19T08:57:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-09-19T08:57:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:57:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:57:57.893 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-19T08:57:58.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:57:57 smithi164 ceph-mon[30928]: pgmap v379: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:57:58.300 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:57:57 smithi170 ceph-mon[35072]: pgmap v379: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:57:58.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:57:57 smithi173 ceph-mon[35306]: pgmap v379: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:57:58.728 INFO:tasks.check_counter:Daemon mds.b mds.exported=2 2024-09-19T08:57:58.728 INFO:tasks.check_counter:Daemon mds.b mds.imported=1 2024-09-19T08:57:58.728 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=19 2024-09-19T08:57:58.728 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-09-19T08:57:58.728 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-19T08:57:58.728 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=12 2024-09-19T08:57:58.728 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=6 2024-09-19T08:57:58.728 DEBUG:tasks.check_counter:Getting stats from e 2024-09-19T08:57:58.729 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-19T08:57:58.980 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T08:57:59.532 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-19T08:57:59.532 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T08:57:59.532 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 1719, 2024-09-19T08:57:59.532 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 1699, 2024-09-19T08:57:59.532 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 127979296, 2024-09-19T08:57:59.532 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 607498, 2024-09-19T08:57:59.532 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 7, 2024-09-19T08:57:59.533 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 3, 2024-09-19T08:57:59.533 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.253227031, 2024-09-19T08:57:59.533 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.068873715, 2024-09-19T08:57:59.533 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.144795125, 2024-09-19T08:57:59.533 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.042049940, 2024-09-19T08:57:59.533 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:57:59.533 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1698, 2024-09-19T08:57:59.533 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.080712311, 2024-09-19T08:57:59.533 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000047533 2024-09-19T08:57:59.533 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.533 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:57:59.533 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.534 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.534 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.534 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.534 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 1675728, 2024-09-19T08:57:59.534 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 147984 2024-09-19T08:57:59.534 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.534 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T08:57:59.534 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 419, 2024-09-19T08:57:59.534 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 595, 2024-09-19T08:57:59.534 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 138565928, 2024-09-19T08:57:59.534 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 607576, 2024-09-19T08:57:59.534 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 9, 2024-09-19T08:57:59.534 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 7, 2024-09-19T08:57:59.535 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.131428454, 2024-09-19T08:57:59.535 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.015155478, 2024-09-19T08:57:59.535 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.098380748, 2024-09-19T08:57:59.535 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.012101318, 2024-09-19T08:57:59.535 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:57:59.535 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 593, 2024-09-19T08:57:59.535 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.041379271, 2024-09-19T08:57:59.535 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000069779 2024-09-19T08:57:59.535 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.535 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:57:59.535 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.535 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.536 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.536 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.536 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-19T08:57:59.536 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 476176 2024-09-19T08:57:59.536 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.536 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T08:57:59.536 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 1571, 2024-09-19T08:57:59.536 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 1573, 2024-09-19T08:57:59.536 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 203644143, 2024-09-19T08:57:59.536 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 499585, 2024-09-19T08:57:59.536 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 6, 2024-09-19T08:57:59.536 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 5, 2024-09-19T08:57:59.537 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.288040400, 2024-09-19T08:57:59.537 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.029302490, 2024-09-19T08:57:59.537 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.198324280, 2024-09-19T08:57:59.537 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.047631853, 2024-09-19T08:57:59.537 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:57:59.537 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1571, 2024-09-19T08:57:59.537 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.086730536, 2024-09-19T08:57:59.537 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000055207 2024-09-19T08:57:59.537 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.537 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:57:59.537 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.537 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.537 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.538 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.538 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T08:57:59.538 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T08:57:59.538 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.538 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2024-09-19T08:57:59.538 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 1, 2024-09-19T08:57:59.538 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2024-09-19T08:57:59.538 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.538 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-MDSRank": { 2024-09-19T08:57:59.538 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-09-19T08:57:59.538 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-09-19T08:57:59.538 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 5192, 2024-09-19T08:57:59.538 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.136068569, 2024-09-19T08:57:59.539 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000026207 2024-09-19T08:57:59.539 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:57:59.539 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.539 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-PurgeQueue": { 2024-09-19T08:57:59.539 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-09-19T08:57:59.539 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-09-19T08:57:59.539 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.539 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.539 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.539 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:57:59.539 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.539 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": { 2024-09-19T08:57:59.539 INFO:teuthology.orchestra.run.smithi170.stdout: "request": 0, 2024-09-19T08:57:59.540 INFO:teuthology.orchestra.run.smithi170.stdout: "reply": 0, 2024-09-19T08:57:59.540 INFO:teuthology.orchestra.run.smithi170.stdout: "reply_latency": { 2024-09-19T08:57:59.540 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.540 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.540 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.540 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.540 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_reply": 0, 2024-09-19T08:57:59.540 INFO:teuthology.orchestra.run.smithi170.stdout: "forward": 0, 2024-09-19T08:57:59.540 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_complete": 0, 2024-09-19T08:57:59.540 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_keys": 0, 2024-09-19T08:57:59.540 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_commit": 0, 2024-09-19T08:57:59.540 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_split": 0, 2024-09-19T08:57:59.541 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_merge": 0, 2024-09-19T08:57:59.541 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes": 124, 2024-09-19T08:57:59.541 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_top": 14, 2024-09-19T08:57:59.541 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_bottom": 110, 2024-09-19T08:57:59.541 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pin_tail": 0, 2024-09-19T08:57:59.541 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pinned": 105, 2024-09-19T08:57:59.541 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_expired": 0, 2024-09-19T08:57:59.541 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_with_caps": 0, 2024-09-19T08:57:59.541 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": 0, 2024-09-19T08:57:59.541 INFO:teuthology.orchestra.run.smithi170.stdout: "subtrees": 3, 2024-09-19T08:57:59.541 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse": 0, 2024-09-19T08:57:59.541 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_hit": 0, 2024-09-19T08:57:59.542 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_forward": 0, 2024-09-19T08:57:59.542 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_discover": 0, 2024-09-19T08:57:59.542 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_dir_fetch": 0, 2024-09-19T08:57:59.542 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_remote_ino": 0, 2024-09-19T08:57:59.542 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_lock": 0, 2024-09-19T08:57:59.542 INFO:teuthology.orchestra.run.smithi170.stdout: "load_cent": 0, 2024-09-19T08:57:59.542 INFO:teuthology.orchestra.run.smithi170.stdout: "q": 0, 2024-09-19T08:57:59.542 INFO:teuthology.orchestra.run.smithi170.stdout: "exported": 0, 2024-09-19T08:57:59.542 INFO:teuthology.orchestra.run.smithi170.stdout: "exported_inodes": 0, 2024-09-19T08:57:59.542 INFO:teuthology.orchestra.run.smithi170.stdout: "imported": 0, 2024-09-19T08:57:59.542 INFO:teuthology.orchestra.run.smithi170.stdout: "imported_inodes": 0, 2024-09-19T08:57:59.543 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_dir_fetch": 0, 2024-09-19T08:57:59.543 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_backtrace_fetch": 0, 2024-09-19T08:57:59.543 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_peer_discover": 0, 2024-09-19T08:57:59.543 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rfiles": 4, 2024-09-19T08:57:59.543 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rbytes": 234, 2024-09-19T08:57:59.543 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rsnaps": 6, 2024-09-19T08:57:59.543 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T08:57:59.543 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_set_tag": 0, 2024-09-19T08:57:59.543 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T08:57:59.543 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_inotable_repaired": 0, 2024-09-19T08:57:59.543 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_inodes": 0, 2024-09-19T08:57:59.543 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T08:57:59.543 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T08:57:59.544 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_file_inodes": 0, 2024-09-19T08:57:59.544 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_inode_file_caps": 0, 2024-09-19T08:57:59.544 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T08:57:59.544 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_grant": 0, 2024-09-19T08:57:59.544 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T08:57:59.544 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T08:57:59.544 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T08:57:59.544 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps": 0, 2024-09-19T08:57:59.544 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps_dirty": 0, 2024-09-19T08:57:59.544 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_cap_release": 0, 2024-09-19T08:57:59.544 INFO:teuthology.orchestra.run.smithi170.stdout: "process_request_cap_release": 0 2024-09-19T08:57:59.544 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.544 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_cache": { 2024-09-19T08:57:59.545 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update": 0, 2024-09-19T08:57:59.545 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update_receipt": 0, 2024-09-19T08:57:59.545 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_try_discover": 0, 2024-09-19T08:57:59.545 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_send_discover": 0, 2024-09-19T08:57:59.545 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_handle_discover": 0, 2024-09-19T08:57:59.545 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays": 0, 2024-09-19T08:57:59.545 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_delayed": 0, 2024-09-19T08:57:59.545 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_enqueuing": 0, 2024-09-19T08:57:59.545 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_created": 0, 2024-09-19T08:57:59.545 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_enqueued": 0, 2024-09-19T08:57:59.545 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_reintegrated": 0, 2024-09-19T08:57:59.545 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_migrated": 0, 2024-09-19T08:57:59.545 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_processing": 0, 2024-09-19T08:57:59.546 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_enqueued": 0, 2024-09-19T08:57:59.546 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_prioritized": 0, 2024-09-19T08:57:59.546 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_started": 0, 2024-09-19T08:57:59.546 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_completed": 0, 2024-09-19T08:57:59.546 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_path": 0, 2024-09-19T08:57:59.546 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_inode": 0, 2024-09-19T08:57:59.546 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T08:57:59.546 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_exportdir": 0, 2024-09-19T08:57:59.546 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_flush": 0, 2024-09-19T08:57:59.546 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragmentdir": 0, 2024-09-19T08:57:59.546 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragstats": 0, 2024-09-19T08:57:59.546 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_inodestats": 0 2024-09-19T08:57:59.546 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.547 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_log": { 2024-09-19T08:57:59.547 INFO:teuthology.orchestra.run.smithi170.stdout: "evlrg": 0, 2024-09-19T08:57:59.547 INFO:teuthology.orchestra.run.smithi170.stdout: "evadd": 0, 2024-09-19T08:57:59.547 INFO:teuthology.orchestra.run.smithi170.stdout: "evex": 0, 2024-09-19T08:57:59.547 INFO:teuthology.orchestra.run.smithi170.stdout: "evtrm": 0, 2024-09-19T08:57:59.547 INFO:teuthology.orchestra.run.smithi170.stdout: "ev": 146457, 2024-09-19T08:57:59.547 INFO:teuthology.orchestra.run.smithi170.stdout: "evexg": 0, 2024-09-19T08:57:59.547 INFO:teuthology.orchestra.run.smithi170.stdout: "evexd": 0, 2024-09-19T08:57:59.547 INFO:teuthology.orchestra.run.smithi170.stdout: "segadd": 0, 2024-09-19T08:57:59.547 INFO:teuthology.orchestra.run.smithi170.stdout: "segex": 0, 2024-09-19T08:57:59.547 INFO:teuthology.orchestra.run.smithi170.stdout: "segtrm": 0, 2024-09-19T08:57:59.547 INFO:teuthology.orchestra.run.smithi170.stdout: "seg": 146, 2024-09-19T08:57:59.547 INFO:teuthology.orchestra.run.smithi170.stdout: "segmjr": 13, 2024-09-19T08:57:59.548 INFO:teuthology.orchestra.run.smithi170.stdout: "segexg": 0, 2024-09-19T08:57:59.548 INFO:teuthology.orchestra.run.smithi170.stdout: "segexd": 0, 2024-09-19T08:57:59.548 INFO:teuthology.orchestra.run.smithi170.stdout: "expos": 159266902, 2024-09-19T08:57:59.548 INFO:teuthology.orchestra.run.smithi170.stdout: "wrpos": 471643212, 2024-09-19T08:57:59.548 INFO:teuthology.orchestra.run.smithi170.stdout: "rdpos": 471640607, 2024-09-19T08:57:59.548 INFO:teuthology.orchestra.run.smithi170.stdout: "jlat": { 2024-09-19T08:57:59.548 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.548 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.548 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.548 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.548 INFO:teuthology.orchestra.run.smithi170.stdout: "replayed": 146457 2024-09-19T08:57:59.548 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.548 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_mem": { 2024-09-19T08:57:59.549 INFO:teuthology.orchestra.run.smithi170.stdout: "ino": 84, 2024-09-19T08:57:59.549 INFO:teuthology.orchestra.run.smithi170.stdout: "ino+": 104, 2024-09-19T08:57:59.549 INFO:teuthology.orchestra.run.smithi170.stdout: "ino-": 20, 2024-09-19T08:57:59.549 INFO:teuthology.orchestra.run.smithi170.stdout: "dir": 12, 2024-09-19T08:57:59.549 INFO:teuthology.orchestra.run.smithi170.stdout: "dir+": 13, 2024-09-19T08:57:59.549 INFO:teuthology.orchestra.run.smithi170.stdout: "dir-": 1, 2024-09-19T08:57:59.549 INFO:teuthology.orchestra.run.smithi170.stdout: "dn": 124, 2024-09-19T08:57:59.549 INFO:teuthology.orchestra.run.smithi170.stdout: "dn+": 129, 2024-09-19T08:57:59.549 INFO:teuthology.orchestra.run.smithi170.stdout: "dn-": 5, 2024-09-19T08:57:59.549 INFO:teuthology.orchestra.run.smithi170.stdout: "cap": 0, 2024-09-19T08:57:59.549 INFO:teuthology.orchestra.run.smithi170.stdout: "cap+": 0, 2024-09-19T08:57:59.549 INFO:teuthology.orchestra.run.smithi170.stdout: "cap-": 0, 2024-09-19T08:57:59.549 INFO:teuthology.orchestra.run.smithi170.stdout: "rss": 54364, 2024-09-19T08:57:59.550 INFO:teuthology.orchestra.run.smithi170.stdout: "heap": 215328 2024-09-19T08:57:59.550 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.550 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_server": { 2024-09-19T08:57:59.550 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_client_request": 0, 2024-09-19T08:57:59.550 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_server_request": 0, 2024-09-19T08:57:59.550 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_request": 0, 2024-09-19T08:57:59.550 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_session": 0, 2024-09-19T08:57:59.550 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_peer_request": 0, 2024-09-19T08:57:59.550 INFO:teuthology.orchestra.run.smithi170.stdout: "req_create_latency": { 2024-09-19T08:57:59.550 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.550 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.550 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.550 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.550 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getattr_latency": { 2024-09-19T08:57:59.551 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.551 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.551 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.551 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.551 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getfilelock_latency": { 2024-09-19T08:57:59.551 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.551 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.551 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.551 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.551 INFO:teuthology.orchestra.run.smithi170.stdout: "req_link_latency": { 2024-09-19T08:57:59.551 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.551 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.551 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.552 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.552 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookup_latency": { 2024-09-19T08:57:59.552 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.552 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.552 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.552 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.552 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookuphash_latency": { 2024-09-19T08:57:59.552 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.552 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.552 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.552 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.552 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupino_latency": { 2024-09-19T08:57:59.552 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.553 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.553 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.553 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupname_latency": { 2024-09-19T08:57:59.553 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.553 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.553 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.553 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupparent_latency": { 2024-09-19T08:57:59.553 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.553 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.553 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.554 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupsnap_latency": { 2024-09-19T08:57:59.554 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.554 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.554 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.554 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lssnap_latency": { 2024-09-19T08:57:59.554 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.554 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.554 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.555 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mkdir_latency": { 2024-09-19T08:57:59.555 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.555 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.555 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.555 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.555 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mknod_latency": { 2024-09-19T08:57:59.555 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.556 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.556 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.556 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mksnap_latency": { 2024-09-19T08:57:59.556 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.556 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.556 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.556 INFO:teuthology.orchestra.run.smithi170.stdout: "req_open_latency": { 2024-09-19T08:57:59.556 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.556 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.556 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.557 INFO:teuthology.orchestra.run.smithi170.stdout: "req_readdir_latency": { 2024-09-19T08:57:59.557 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.557 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.557 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.557 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rename_latency": { 2024-09-19T08:57:59.557 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.557 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.557 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.557 INFO:teuthology.orchestra.run.smithi170.stdout: "req_renamesnap_latency": { 2024-09-19T08:57:59.558 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.558 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.558 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.558 INFO:teuthology.orchestra.run.smithi170.stdout: "req_snapdiff_latency": { 2024-09-19T08:57:59.558 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.558 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.558 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.558 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmdir_latency": { 2024-09-19T08:57:59.558 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.558 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.558 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.559 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmsnap_latency": { 2024-09-19T08:57:59.559 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.559 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.559 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.559 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmxattr_latency": { 2024-09-19T08:57:59.559 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.559 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.559 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.559 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setattr_latency": { 2024-09-19T08:57:59.559 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.560 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.560 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.560 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setdirlayout_latency": { 2024-09-19T08:57:59.560 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.560 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.560 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.560 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setfilelock_latency": { 2024-09-19T08:57:59.560 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.560 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.560 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.561 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setlayout_latency": { 2024-09-19T08:57:59.561 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.561 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.561 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.561 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setxattr_latency": { 2024-09-19T08:57:59.561 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.561 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.561 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.561 INFO:teuthology.orchestra.run.smithi170.stdout: "req_symlink_latency": { 2024-09-19T08:57:59.561 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.562 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.562 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.562 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.562 INFO:teuthology.orchestra.run.smithi170.stdout: "req_unlink_latency": { 2024-09-19T08:57:59.562 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.562 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.562 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.562 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.562 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_revoke_eviction": 0, 2024-09-19T08:57:59.562 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_acquisition_throttle": 0, 2024-09-19T08:57:59.562 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getvxattr_latency": { 2024-09-19T08:57:59.562 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.562 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.563 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.563 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:57:59.563 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.563 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_sessions": { 2024-09-19T08:57:59.563 INFO:teuthology.orchestra.run.smithi170.stdout: "session_count": 2, 2024-09-19T08:57:59.563 INFO:teuthology.orchestra.run.smithi170.stdout: "session_add": 5, 2024-09-19T08:57:59.563 INFO:teuthology.orchestra.run.smithi170.stdout: "session_remove": 3, 2024-09-19T08:57:59.563 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_open": 4, 2024-09-19T08:57:59.563 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_stale": 0, 2024-09-19T08:57:59.563 INFO:teuthology.orchestra.run.smithi170.stdout: "total_load": 0, 2024-09-19T08:57:59.563 INFO:teuthology.orchestra.run.smithi170.stdout: "average_load": 0, 2024-09-19T08:57:59.563 INFO:teuthology.orchestra.run.smithi170.stdout: "avg_session_uptime": 3185, 2024-09-19T08:57:59.564 INFO:teuthology.orchestra.run.smithi170.stdout: "mdthresh_evicted": 0 2024-09-19T08:57:59.564 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.564 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2024-09-19T08:57:59.564 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2024-09-19T08:57:59.564 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2024-09-19T08:57:59.564 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T08:57:59.564 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2024-09-19T08:57:59.564 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T08:57:59.564 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2024-09-19T08:57:59.564 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T08:57:59.564 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T08:57:59.564 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T08:57:59.564 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T08:57:59.565 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T08:57:59.565 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2024-09-19T08:57:59.565 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T08:57:59.565 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T08:57:59.565 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2024-09-19T08:57:59.565 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2024-09-19T08:57:59.565 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2024-09-19T08:57:59.565 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2024-09-19T08:57:59.565 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T08:57:59.565 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T08:57:59.565 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T08:57:59.565 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T08:57:59.566 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T08:57:59.566 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2024-09-19T08:57:59.566 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2024-09-19T08:57:59.566 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2024-09-19T08:57:59.566 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T08:57:59.566 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T08:57:59.566 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2024-09-19T08:57:59.566 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2024-09-19T08:57:59.566 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2024-09-19T08:57:59.566 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2024-09-19T08:57:59.566 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T08:57:59.566 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2024-09-19T08:57:59.566 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T08:57:59.567 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2024-09-19T08:57:59.567 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 241832, 2024-09-19T08:57:59.567 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 164, 2024-09-19T08:57:59.567 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 0, 2024-09-19T08:57:59.567 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 0, 2024-09-19T08:57:59.567 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2024-09-19T08:57:59.567 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2024-09-19T08:57:59.567 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2024-09-19T08:57:59.567 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2024-09-19T08:57:59.567 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2024-09-19T08:57:59.567 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2024-09-19T08:57:59.567 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 7936, 2024-09-19T08:57:59.568 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 175, 2024-09-19T08:57:59.568 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T08:57:59.568 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2024-09-19T08:57:59.568 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2024-09-19T08:57:59.568 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2024-09-19T08:57:59.568 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 342168, 2024-09-19T08:57:59.568 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 2893, 2024-09-19T08:57:59.568 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2024-09-19T08:57:59.568 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2024-09-19T08:57:59.568 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2024-09-19T08:57:59.568 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2024-09-19T08:57:59.568 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.568 INFO:teuthology.orchestra.run.smithi170.stdout: "objecter": { 2024-09-19T08:57:59.569 INFO:teuthology.orchestra.run.smithi170.stdout: "op_active": 0, 2024-09-19T08:57:59.569 INFO:teuthology.orchestra.run.smithi170.stdout: "op_laggy": 0, 2024-09-19T08:57:59.569 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send": 3429, 2024-09-19T08:57:59.569 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send_bytes": 26, 2024-09-19T08:57:59.569 INFO:teuthology.orchestra.run.smithi170.stdout: "op_resend": 4, 2024-09-19T08:57:59.569 INFO:teuthology.orchestra.run.smithi170.stdout: "op_reply": 3429, 2024-09-19T08:57:59.569 INFO:teuthology.orchestra.run.smithi170.stdout: "op_latency": { 2024-09-19T08:57:59.569 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 3429, 2024-09-19T08:57:59.569 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 18.018580076, 2024-09-19T08:57:59.569 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.005254762 2024-09-19T08:57:59.569 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.569 INFO:teuthology.orchestra.run.smithi170.stdout: "op_inflight": 0, 2024-09-19T08:57:59.569 INFO:teuthology.orchestra.run.smithi170.stdout: "oplen_avg": { 2024-09-19T08:57:59.570 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 3429, 2024-09-19T08:57:59.570 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 3430 2024-09-19T08:57:59.570 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.570 INFO:teuthology.orchestra.run.smithi170.stdout: "op": 3429, 2024-09-19T08:57:59.570 INFO:teuthology.orchestra.run.smithi170.stdout: "op_r": 3429, 2024-09-19T08:57:59.570 INFO:teuthology.orchestra.run.smithi170.stdout: "op_w": 0, 2024-09-19T08:57:59.570 INFO:teuthology.orchestra.run.smithi170.stdout: "op_rmw": 0, 2024-09-19T08:57:59.570 INFO:teuthology.orchestra.run.smithi170.stdout: "op_pg": 0, 2024-09-19T08:57:59.570 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_stat": 2271, 2024-09-19T08:57:59.570 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_create": 0, 2024-09-19T08:57:59.570 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_read": 1155, 2024-09-19T08:57:59.570 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_write": 0, 2024-09-19T08:57:59.570 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writefull": 0, 2024-09-19T08:57:59.570 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writesame": 0, 2024-09-19T08:57:59.571 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_append": 0, 2024-09-19T08:57:59.571 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_zero": 0, 2024-09-19T08:57:59.571 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_truncate": 0, 2024-09-19T08:57:59.571 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_delete": 0, 2024-09-19T08:57:59.571 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_mapext": 0, 2024-09-19T08:57:59.571 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_sparse_read": 0, 2024-09-19T08:57:59.571 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_clonerange": 0, 2024-09-19T08:57:59.571 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_getxattr": 2, 2024-09-19T08:57:59.571 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_setxattr": 0, 2024-09-19T08:57:59.571 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_cmpxattr": 0, 2024-09-19T08:57:59.571 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_rmxattr": 0, 2024-09-19T08:57:59.571 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_resetxattrs": 0, 2024-09-19T08:57:59.571 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_call": 0, 2024-09-19T08:57:59.572 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_watch": 0, 2024-09-19T08:57:59.572 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_notify": 0, 2024-09-19T08:57:59.572 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T08:57:59.572 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls": 0, 2024-09-19T08:57:59.572 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls_filter": 0, 2024-09-19T08:57:59.572 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_other": 0, 2024-09-19T08:57:59.572 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_active": 0, 2024-09-19T08:57:59.572 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_send": 0, 2024-09-19T08:57:59.572 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_resend": 0, 2024-09-19T08:57:59.572 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_ping": 0, 2024-09-19T08:57:59.572 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_active": 0, 2024-09-19T08:57:59.572 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_send": 0, 2024-09-19T08:57:59.572 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_resend": 0, 2024-09-19T08:57:59.573 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_active": 0, 2024-09-19T08:57:59.573 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_send": 0, 2024-09-19T08:57:59.573 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_resend": 0, 2024-09-19T08:57:59.573 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_active": 0, 2024-09-19T08:57:59.573 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_send": 0, 2024-09-19T08:57:59.573 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_resend": 0, 2024-09-19T08:57:59.573 INFO:teuthology.orchestra.run.smithi170.stdout: "command_active": 0, 2024-09-19T08:57:59.573 INFO:teuthology.orchestra.run.smithi170.stdout: "command_send": 0, 2024-09-19T08:57:59.573 INFO:teuthology.orchestra.run.smithi170.stdout: "command_resend": 0, 2024-09-19T08:57:59.573 INFO:teuthology.orchestra.run.smithi170.stdout: "map_epoch": 95, 2024-09-19T08:57:59.573 INFO:teuthology.orchestra.run.smithi170.stdout: "map_full": 0, 2024-09-19T08:57:59.573 INFO:teuthology.orchestra.run.smithi170.stdout: "map_inc": 13, 2024-09-19T08:57:59.573 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_sessions": 11, 2024-09-19T08:57:59.574 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_open": 11, 2024-09-19T08:57:59.574 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_close": 0, 2024-09-19T08:57:59.574 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_laggy": 0, 2024-09-19T08:57:59.574 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_wr": 0, 2024-09-19T08:57:59.574 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_rd": 2, 2024-09-19T08:57:59.574 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_del": 0 2024-09-19T08:57:59.574 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.574 INFO:teuthology.orchestra.run.smithi170.stdout: "oft": { 2024-09-19T08:57:59.574 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_objs": 0, 2024-09-19T08:57:59.574 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_kv_pairs": 0, 2024-09-19T08:57:59.574 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_updates": 0, 2024-09-19T08:57:59.574 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_removes": 0 2024-09-19T08:57:59.574 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.575 INFO:teuthology.orchestra.run.smithi170.stdout: "osd-slow-ops": { 2024-09-19T08:57:59.575 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_ops_count": 0 2024-09-19T08:57:59.575 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.575 INFO:teuthology.orchestra.run.smithi170.stdout: "purge_queue": { 2024-09-19T08:57:59.575 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops": 0, 2024-09-19T08:57:59.575 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T08:57:59.575 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing": 0, 2024-09-19T08:57:59.575 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_high_water": 0, 2024-09-19T08:57:59.575 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed_ops": 0, 2024-09-19T08:57:59.575 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed": 0, 2024-09-19T08:57:59.575 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_item_in_journal": 0 2024-09-19T08:57:59.575 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.575 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T08:57:59.576 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-09-19T08:57:59.576 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-09-19T08:57:59.576 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-09-19T08:57:59.576 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 3705, 2024-09-19T08:57:59.576 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 469838878, 2024-09-19T08:57:59.576 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-09-19T08:57:59.576 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 3705, 2024-09-19T08:57:59.576 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-09-19T08:57:59.576 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-09-19T08:57:59.576 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 3705, 2024-09-19T08:57:59.576 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 469838878, 2024-09-19T08:57:59.576 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-09-19T08:57:59.577 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.577 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.577 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.577 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:57:59.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.577 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_bytes": { 2024-09-19T08:57:59.577 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-09-19T08:57:59.577 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-09-19T08:57:59.577 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-09-19T08:57:59.577 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-09-19T08:57:59.577 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-09-19T08:57:59.577 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-09-19T08:57:59.577 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-09-19T08:57:59.578 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 3429, 2024-09-19T08:57:59.578 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 467567101, 2024-09-19T08:57:59.578 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 3429, 2024-09-19T08:57:59.578 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 467567101, 2024-09-19T08:57:59.578 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-09-19T08:57:59.578 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.578 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.578 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.578 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:57:59.578 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.578 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_ops": { 2024-09-19T08:57:59.578 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-09-19T08:57:59.578 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 1024, 2024-09-19T08:57:59.579 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-09-19T08:57:59.579 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-09-19T08:57:59.579 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-09-19T08:57:59.579 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-09-19T08:57:59.579 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-09-19T08:57:59.579 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 3429, 2024-09-19T08:57:59.579 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 3429, 2024-09-19T08:57:59.579 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 3429, 2024-09-19T08:57:59.579 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 3429, 2024-09-19T08:57:59.579 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-09-19T08:57:59.579 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.579 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.579 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.580 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:57:59.580 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.580 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle": { 2024-09-19T08:57:59.580 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-09-19T08:57:59.580 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-09-19T08:57:59.580 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-09-19T08:57:59.580 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-09-19T08:57:59.580 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-09-19T08:57:59.580 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-09-19T08:57:59.580 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-09-19T08:57:59.580 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-09-19T08:57:59.580 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-09-19T08:57:59.580 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 0, 2024-09-19T08:57:59.581 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 0, 2024-09-19T08:57:59.581 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-09-19T08:57:59.581 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.581 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.581 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.581 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:57:59.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:57:59.581 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle-0x559f2af66b40": { 2024-09-19T08:57:59.581 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-09-19T08:57:59.581 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-09-19T08:57:59.581 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-09-19T08:57:59.581 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-09-19T08:57:59.581 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-09-19T08:57:59.581 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-09-19T08:57:59.582 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-09-19T08:57:59.582 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-09-19T08:57:59.582 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-09-19T08:57:59.582 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 0, 2024-09-19T08:57:59.582 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 0, 2024-09-19T08:57:59.582 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-09-19T08:57:59.582 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:57:59.582 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:57:59.582 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:57:59.582 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:57:59.582 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:57:59.582 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-19T08:58:00.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:00 smithi170 ceph-mon[35072]: pgmap v380: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:58:00.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:00 smithi164 ceph-mon[30928]: pgmap v380: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:58:00.323 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-19T08:58:00.323 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-19T08:58:00.323 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-09-19T08:58:00.324 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-09-19T08:58:00.324 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-09-19T08:58:00.324 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-19T08:58:00.324 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T08:58:00.324 DEBUG:tasks.check_counter:Getting stats from h 2024-09-19T08:58:00.324 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-19T08:58:00.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:00 smithi173 ceph-mon[35306]: pgmap v380: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T08:58:00.570 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T08:58:01.115 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-19T08:58:01.115 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T08:58:01.115 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 1442, 2024-09-19T08:58:01.115 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 1170, 2024-09-19T08:58:01.115 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 492866, 2024-09-19T08:58:01.115 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 589391, 2024-09-19T08:58:01.115 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 18, 2024-09-19T08:58:01.116 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 4, 2024-09-19T08:58:01.116 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.129445181, 2024-09-19T08:58:01.116 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.053983246, 2024-09-19T08:58:01.116 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.078559864, 2024-09-19T08:58:01.116 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.006375792, 2024-09-19T08:58:01.116 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:58:01.116 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1169, 2024-09-19T08:58:01.116 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.075115418, 2024-09-19T08:58:01.116 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000064256 2024-09-19T08:58:01.116 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.116 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:58:01.117 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2363, 2024-09-19T08:58:01.117 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000540635, 2024-09-19T08:58:01.117 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000228 2024-09-19T08:58:01.117 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.117 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-19T08:58:01.117 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T08:58:01.117 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.117 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T08:58:01.117 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 410, 2024-09-19T08:58:01.117 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 531, 2024-09-19T08:58:01.117 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 1760301, 2024-09-19T08:58:01.118 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 603497, 2024-09-19T08:58:01.118 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 12, 2024-09-19T08:58:01.118 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 5, 2024-09-19T08:58:01.118 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.066927630, 2024-09-19T08:58:01.118 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.027362774, 2024-09-19T08:58:01.118 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.056073352, 2024-09-19T08:58:01.119 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.000430047, 2024-09-19T08:58:01.119 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:58:01.119 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 530, 2024-09-19T08:58:01.119 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.049913227, 2024-09-19T08:58:01.119 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000094175 2024-09-19T08:58:01.119 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.120 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:58:01.120 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 130, 2024-09-19T08:58:01.120 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000026958, 2024-09-19T08:58:01.120 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000207 2024-09-19T08:58:01.120 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.121 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 1699760, 2024-09-19T08:58:01.121 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 515632 2024-09-19T08:58:01.121 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.121 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T08:58:01.122 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 145, 2024-09-19T08:58:01.122 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 222, 2024-09-19T08:58:01.122 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 77985, 2024-09-19T08:58:01.122 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 540095, 2024-09-19T08:58:01.122 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 38, 2024-09-19T08:58:01.122 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 6, 2024-09-19T08:58:01.123 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.029303714, 2024-09-19T08:58:01.123 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.007080197, 2024-09-19T08:58:01.123 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.020672790, 2024-09-19T08:58:01.123 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.000783578, 2024-09-19T08:58:01.123 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:58:01.124 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 219, 2024-09-19T08:58:01.124 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.020039079, 2024-09-19T08:58:01.124 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000091502 2024-09-19T08:58:01.124 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.124 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:58:01.124 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 229, 2024-09-19T08:58:01.125 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000052680, 2024-09-19T08:58:01.125 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000230 2024-09-19T08:58:01.125 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.125 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 7280, 2024-09-19T08:58:01.125 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 150016 2024-09-19T08:58:01.125 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.125 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2024-09-19T08:58:01.126 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 1, 2024-09-19T08:58:01.126 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2024-09-19T08:58:01.126 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.126 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-MDSRank": { 2024-09-19T08:58:01.126 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-09-19T08:58:01.126 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-09-19T08:58:01.126 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 99, 2024-09-19T08:58:01.126 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.007997285, 2024-09-19T08:58:01.126 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000080780 2024-09-19T08:58:01.126 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:58:01.126 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.127 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-PurgeQueue": { 2024-09-19T08:58:01.127 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-09-19T08:58:01.127 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-09-19T08:58:01.127 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 17, 2024-09-19T08:58:01.127 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000439556, 2024-09-19T08:58:01.127 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000025856 2024-09-19T08:58:01.127 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:58:01.127 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.127 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": { 2024-09-19T08:58:01.128 INFO:teuthology.orchestra.run.smithi170.stdout: "request": 11, 2024-09-19T08:58:01.128 INFO:teuthology.orchestra.run.smithi170.stdout: "reply": 8, 2024-09-19T08:58:01.128 INFO:teuthology.orchestra.run.smithi170.stdout: "reply_latency": { 2024-09-19T08:58:01.128 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 8, 2024-09-19T08:58:01.128 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.006021008, 2024-09-19T08:58:01.128 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000752626 2024-09-19T08:58:01.128 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.128 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_reply": 0, 2024-09-19T08:58:01.128 INFO:teuthology.orchestra.run.smithi170.stdout: "forward": 3, 2024-09-19T08:58:01.128 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_complete": 0, 2024-09-19T08:58:01.129 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_keys": 0, 2024-09-19T08:58:01.129 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_commit": 11, 2024-09-19T08:58:01.129 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_split": 0, 2024-09-19T08:58:01.129 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_merge": 0, 2024-09-19T08:58:01.129 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes": 17, 2024-09-19T08:58:01.129 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_top": 2, 2024-09-19T08:58:01.129 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_bottom": 2, 2024-09-19T08:58:01.130 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pin_tail": 13, 2024-09-19T08:58:01.130 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pinned": 13, 2024-09-19T08:58:01.130 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_expired": 14, 2024-09-19T08:58:01.130 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_with_caps": 1, 2024-09-19T08:58:01.130 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": 1, 2024-09-19T08:58:01.130 INFO:teuthology.orchestra.run.smithi170.stdout: "subtrees": 4, 2024-09-19T08:58:01.130 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse": 50, 2024-09-19T08:58:01.130 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_hit": 38, 2024-09-19T08:58:01.130 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_forward": 0, 2024-09-19T08:58:01.130 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_discover": 6, 2024-09-19T08:58:01.130 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_dir_fetch": 0, 2024-09-19T08:58:01.130 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_remote_ino": 0, 2024-09-19T08:58:01.131 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_lock": 0, 2024-09-19T08:58:01.131 INFO:teuthology.orchestra.run.smithi170.stdout: "load_cent": 0, 2024-09-19T08:58:01.131 INFO:teuthology.orchestra.run.smithi170.stdout: "q": 0, 2024-09-19T08:58:01.131 INFO:teuthology.orchestra.run.smithi170.stdout: "exported": 1, 2024-09-19T08:58:01.131 INFO:teuthology.orchestra.run.smithi170.stdout: "exported_inodes": 4, 2024-09-19T08:58:01.131 INFO:teuthology.orchestra.run.smithi170.stdout: "imported": 2, 2024-09-19T08:58:01.131 INFO:teuthology.orchestra.run.smithi170.stdout: "imported_inodes": 5, 2024-09-19T08:58:01.131 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_dir_fetch": 0, 2024-09-19T08:58:01.131 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_backtrace_fetch": 36, 2024-09-19T08:58:01.131 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_peer_discover": 0, 2024-09-19T08:58:01.131 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rfiles": 22, 2024-09-19T08:58:01.131 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rbytes": 10738142113, 2024-09-19T08:58:01.131 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rsnaps": 6, 2024-09-19T08:58:01.132 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_fetch": 36, 2024-09-19T08:58:01.132 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_set_tag": 0, 2024-09-19T08:58:01.132 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T08:58:01.132 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_inotable_repaired": 0, 2024-09-19T08:58:01.132 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_inodes": 9, 2024-09-19T08:58:01.132 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T08:58:01.132 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dirfrag_rstats": 9, 2024-09-19T08:58:01.132 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_file_inodes": 27, 2024-09-19T08:58:01.132 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_inode_file_caps": 0, 2024-09-19T08:58:01.133 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_revoke": 2, 2024-09-19T08:58:01.133 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_grant": 3, 2024-09-19T08:58:01.133 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T08:58:01.133 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T08:58:01.133 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T08:58:01.133 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps": 2, 2024-09-19T08:58:01.133 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps_dirty": 0, 2024-09-19T08:58:01.133 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_cap_release": 1, 2024-09-19T08:58:01.133 INFO:teuthology.orchestra.run.smithi170.stdout: "process_request_cap_release": 5 2024-09-19T08:58:01.133 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.133 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_cache": { 2024-09-19T08:58:01.133 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update": 3, 2024-09-19T08:58:01.133 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update_receipt": 12, 2024-09-19T08:58:01.133 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_try_discover": 8, 2024-09-19T08:58:01.134 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_send_discover": 11, 2024-09-19T08:58:01.134 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_handle_discover": 3, 2024-09-19T08:58:01.134 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays": 0, 2024-09-19T08:58:01.134 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_delayed": 0, 2024-09-19T08:58:01.134 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_enqueuing": 0, 2024-09-19T08:58:01.134 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_created": 2, 2024-09-19T08:58:01.134 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_enqueued": 2, 2024-09-19T08:58:01.134 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_reintegrated": 0, 2024-09-19T08:58:01.134 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_migrated": 0, 2024-09-19T08:58:01.134 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_processing": 0, 2024-09-19T08:58:01.134 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_enqueued": 0, 2024-09-19T08:58:01.135 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_prioritized": 0, 2024-09-19T08:58:01.135 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_started": 0, 2024-09-19T08:58:01.135 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_completed": 0, 2024-09-19T08:58:01.135 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_path": 10, 2024-09-19T08:58:01.135 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_inode": 116, 2024-09-19T08:58:01.135 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T08:58:01.135 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_exportdir": 1, 2024-09-19T08:58:01.135 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_flush": 0, 2024-09-19T08:58:01.135 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragmentdir": 0, 2024-09-19T08:58:01.135 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragstats": 0, 2024-09-19T08:58:01.135 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_inodestats": 0 2024-09-19T08:58:01.135 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.135 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_log": { 2024-09-19T08:58:01.135 INFO:teuthology.orchestra.run.smithi170.stdout: "evlrg": 0, 2024-09-19T08:58:01.136 INFO:teuthology.orchestra.run.smithi170.stdout: "evadd": 21, 2024-09-19T08:58:01.136 INFO:teuthology.orchestra.run.smithi170.stdout: "evex": 0, 2024-09-19T08:58:01.136 INFO:teuthology.orchestra.run.smithi170.stdout: "evtrm": 0, 2024-09-19T08:58:01.136 INFO:teuthology.orchestra.run.smithi170.stdout: "ev": 21, 2024-09-19T08:58:01.136 INFO:teuthology.orchestra.run.smithi170.stdout: "evexg": 0, 2024-09-19T08:58:01.136 INFO:teuthology.orchestra.run.smithi170.stdout: "evexd": 0, 2024-09-19T08:58:01.136 INFO:teuthology.orchestra.run.smithi170.stdout: "segadd": 2, 2024-09-19T08:58:01.136 INFO:teuthology.orchestra.run.smithi170.stdout: "segex": 0, 2024-09-19T08:58:01.137 INFO:teuthology.orchestra.run.smithi170.stdout: "segtrm": 0, 2024-09-19T08:58:01.140 INFO:teuthology.orchestra.run.smithi170.stdout: "seg": 2, 2024-09-19T08:58:01.140 INFO:teuthology.orchestra.run.smithi170.stdout: "segmjr": 2, 2024-09-19T08:58:01.140 INFO:teuthology.orchestra.run.smithi170.stdout: "segexg": 0, 2024-09-19T08:58:01.140 INFO:teuthology.orchestra.run.smithi170.stdout: "segexd": 0, 2024-09-19T08:58:01.141 INFO:teuthology.orchestra.run.smithi170.stdout: "expos": 4194304, 2024-09-19T08:58:01.141 INFO:teuthology.orchestra.run.smithi170.stdout: "wrpos": 4228481, 2024-09-19T08:58:01.141 INFO:teuthology.orchestra.run.smithi170.stdout: "rdpos": 0, 2024-09-19T08:58:01.141 INFO:teuthology.orchestra.run.smithi170.stdout: "jlat": { 2024-09-19T08:58:01.141 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 16, 2024-09-19T08:58:01.141 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.079895932, 2024-09-19T08:58:01.141 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.004993495 2024-09-19T08:58:01.141 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.142 INFO:teuthology.orchestra.run.smithi170.stdout: "replayed": 0 2024-09-19T08:58:01.142 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.142 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_mem": { 2024-09-19T08:58:01.142 INFO:teuthology.orchestra.run.smithi170.stdout: "ino": 20, 2024-09-19T08:58:01.142 INFO:teuthology.orchestra.run.smithi170.stdout: "ino+": 30, 2024-09-19T08:58:01.142 INFO:teuthology.orchestra.run.smithi170.stdout: "ino-": 10, 2024-09-19T08:58:01.142 INFO:teuthology.orchestra.run.smithi170.stdout: "dir": 18, 2024-09-19T08:58:01.142 INFO:teuthology.orchestra.run.smithi170.stdout: "dir+": 25, 2024-09-19T08:58:01.142 INFO:teuthology.orchestra.run.smithi170.stdout: "dir-": 7, 2024-09-19T08:58:01.142 INFO:teuthology.orchestra.run.smithi170.stdout: "dn": 17, 2024-09-19T08:58:01.142 INFO:teuthology.orchestra.run.smithi170.stdout: "dn+": 33, 2024-09-19T08:58:01.143 INFO:teuthology.orchestra.run.smithi170.stdout: "dn-": 16, 2024-09-19T08:58:01.143 INFO:teuthology.orchestra.run.smithi170.stdout: "cap": 1, 2024-09-19T08:58:01.143 INFO:teuthology.orchestra.run.smithi170.stdout: "cap+": 5, 2024-09-19T08:58:01.143 INFO:teuthology.orchestra.run.smithi170.stdout: "cap-": 4, 2024-09-19T08:58:01.143 INFO:teuthology.orchestra.run.smithi170.stdout: "rss": 52296, 2024-09-19T08:58:01.143 INFO:teuthology.orchestra.run.smithi170.stdout: "heap": 223520 2024-09-19T08:58:01.143 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.143 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_server": { 2024-09-19T08:58:01.143 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_client_request": 16, 2024-09-19T08:58:01.143 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_server_request": 1, 2024-09-19T08:58:01.143 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_request": 11, 2024-09-19T08:58:01.143 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_session": 19, 2024-09-19T08:58:01.144 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_peer_request": 5, 2024-09-19T08:58:01.144 INFO:teuthology.orchestra.run.smithi170.stdout: "req_create_latency": { 2024-09-19T08:58:01.144 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:01.144 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:01.144 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:01.144 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.144 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getattr_latency": { 2024-09-19T08:58:01.144 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 4, 2024-09-19T08:58:01.144 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000947927, 2024-09-19T08:58:01.144 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000236981 2024-09-19T08:58:01.144 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.144 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getfilelock_latency": { 2024-09-19T08:58:01.144 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:01.145 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:01.145 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:01.145 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.145 INFO:teuthology.orchestra.run.smithi170.stdout: "req_link_latency": { 2024-09-19T08:58:01.145 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:01.145 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:01.145 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:01.145 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.145 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookup_latency": { 2024-09-19T08:58:01.145 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1, 2024-09-19T08:58:01.145 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000373257, 2024-09-19T08:58:01.146 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000373257 2024-09-19T08:58:01.146 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.146 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookuphash_latency": { 2024-09-19T08:58:01.146 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:01.146 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:01.146 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:01.146 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.146 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupino_latency": { 2024-09-19T08:58:01.146 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:01.146 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:01.146 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:01.146 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.147 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupname_latency": { 2024-09-19T08:58:01.147 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:01.147 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:01.147 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:01.147 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.147 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupparent_latency": { 2024-09-19T08:58:01.147 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:01.147 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:01.147 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:01.147 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.148 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupsnap_latency": { 2024-09-19T08:58:01.148 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:01.148 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:01.148 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:01.148 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.148 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lssnap_latency": { 2024-09-19T08:58:01.148 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:01.148 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:01.148 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:01.148 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.149 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mkdir_latency": { 2024-09-19T08:58:01.149 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:01.149 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:01.149 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:01.149 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.149 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mknod_latency": { 2024-09-19T08:58:01.149 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:01.149 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:01.149 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:01.149 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.149 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mksnap_latency": { 2024-09-19T08:58:01.149 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:01.149 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:01.150 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:01.150 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.150 INFO:teuthology.orchestra.run.smithi170.stdout: "req_open_latency": { 2024-09-19T08:58:01.150 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:01.150 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:01.150 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:01.150 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.150 INFO:teuthology.orchestra.run.smithi170.stdout: "req_readdir_latency": { 2024-09-19T08:58:01.150 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1, 2024-09-19T08:58:01.150 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000522435, 2024-09-19T08:58:01.150 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000522435 2024-09-19T08:58:01.150 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.150 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rename_latency": { 2024-09-19T08:58:01.150 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:01.151 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:01.151 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:01.151 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.151 INFO:teuthology.orchestra.run.smithi170.stdout: "req_renamesnap_latency": { 2024-09-19T08:58:01.151 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:01.151 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:01.151 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:01.151 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.151 INFO:teuthology.orchestra.run.smithi170.stdout: "req_snapdiff_latency": { 2024-09-19T08:58:01.151 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:01.151 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:01.151 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:01.151 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.152 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmdir_latency": { 2024-09-19T08:58:01.152 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:01.152 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:01.152 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:01.152 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.152 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmsnap_latency": { 2024-09-19T08:58:01.152 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:01.152 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:01.152 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:01.152 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.152 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmxattr_latency": { 2024-09-19T08:58:01.152 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:01.153 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:01.153 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:01.153 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.153 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setattr_latency": { 2024-09-19T08:58:01.153 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:01.153 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:01.153 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:01.153 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.153 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setdirlayout_latency": { 2024-09-19T08:58:01.153 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:01.153 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:01.153 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:01.153 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.154 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setfilelock_latency": { 2024-09-19T08:58:01.154 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:01.154 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:01.154 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:01.154 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.154 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setlayout_latency": { 2024-09-19T08:58:01.154 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:01.154 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:01.154 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:01.155 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.155 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setxattr_latency": { 2024-09-19T08:58:01.155 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:01.155 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:01.155 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:01.155 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.155 INFO:teuthology.orchestra.run.smithi170.stdout: "req_symlink_latency": { 2024-09-19T08:58:01.155 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:01.155 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:01.155 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:01.155 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.155 INFO:teuthology.orchestra.run.smithi170.stdout: "req_unlink_latency": { 2024-09-19T08:58:01.155 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2, 2024-09-19T08:58:01.156 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.004177389, 2024-09-19T08:58:01.156 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002088694 2024-09-19T08:58:01.156 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.156 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_revoke_eviction": 0, 2024-09-19T08:58:01.156 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_acquisition_throttle": 0, 2024-09-19T08:58:01.156 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getvxattr_latency": { 2024-09-19T08:58:01.156 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:01.156 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:01.156 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:01.157 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:58:01.157 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.157 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_sessions": { 2024-09-19T08:58:01.157 INFO:teuthology.orchestra.run.smithi170.stdout: "session_count": 2, 2024-09-19T08:58:01.157 INFO:teuthology.orchestra.run.smithi170.stdout: "session_add": 4, 2024-09-19T08:58:01.157 INFO:teuthology.orchestra.run.smithi170.stdout: "session_remove": 2, 2024-09-19T08:58:01.158 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_open": 2, 2024-09-19T08:58:01.158 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_stale": 0, 2024-09-19T08:58:01.158 INFO:teuthology.orchestra.run.smithi170.stdout: "total_load": 4, 2024-09-19T08:58:01.158 INFO:teuthology.orchestra.run.smithi170.stdout: "average_load": 2, 2024-09-19T08:58:01.158 INFO:teuthology.orchestra.run.smithi170.stdout: "avg_session_uptime": 383, 2024-09-19T08:58:01.158 INFO:teuthology.orchestra.run.smithi170.stdout: "mdthresh_evicted": 0 2024-09-19T08:58:01.158 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.159 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2024-09-19T08:58:01.159 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2024-09-19T08:58:01.159 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2024-09-19T08:58:01.159 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T08:58:01.159 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2024-09-19T08:58:01.159 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T08:58:01.159 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2024-09-19T08:58:01.159 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T08:58:01.159 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T08:58:01.159 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T08:58:01.160 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T08:58:01.160 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T08:58:01.160 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2024-09-19T08:58:01.160 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T08:58:01.160 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T08:58:01.160 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2024-09-19T08:58:01.161 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2024-09-19T08:58:01.161 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2024-09-19T08:58:01.161 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2024-09-19T08:58:01.161 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T08:58:01.161 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T08:58:01.161 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T08:58:01.161 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T08:58:01.161 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T08:58:01.161 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2024-09-19T08:58:01.161 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2024-09-19T08:58:01.161 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2024-09-19T08:58:01.162 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T08:58:01.162 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T08:58:01.162 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2024-09-19T08:58:01.162 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2024-09-19T08:58:01.162 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2024-09-19T08:58:01.162 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2024-09-19T08:58:01.162 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T08:58:01.162 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2024-09-19T08:58:01.162 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T08:58:01.163 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2024-09-19T08:58:01.163 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 262359, 2024-09-19T08:58:01.163 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 87, 2024-09-19T08:58:01.163 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 0, 2024-09-19T08:58:01.163 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 0, 2024-09-19T08:58:01.163 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2024-09-19T08:58:01.163 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2024-09-19T08:58:01.163 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2024-09-19T08:58:01.163 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2024-09-19T08:58:01.164 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2024-09-19T08:58:01.164 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2024-09-19T08:58:01.164 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 7936, 2024-09-19T08:58:01.164 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 175, 2024-09-19T08:58:01.164 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T08:58:01.164 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2024-09-19T08:58:01.164 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2024-09-19T08:58:01.164 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2024-09-19T08:58:01.165 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 83557, 2024-09-19T08:58:01.165 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 366, 2024-09-19T08:58:01.165 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2024-09-19T08:58:01.165 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2024-09-19T08:58:01.165 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2024-09-19T08:58:01.165 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2024-09-19T08:58:01.165 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.166 INFO:teuthology.orchestra.run.smithi170.stdout: "objecter": { 2024-09-19T08:58:01.166 INFO:teuthology.orchestra.run.smithi170.stdout: "op_active": 0, 2024-09-19T08:58:01.166 INFO:teuthology.orchestra.run.smithi170.stdout: "op_laggy": 0, 2024-09-19T08:58:01.166 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send": 105, 2024-09-19T08:58:01.166 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send_bytes": 45847, 2024-09-19T08:58:01.166 INFO:teuthology.orchestra.run.smithi170.stdout: "op_resend": 28, 2024-09-19T08:58:01.166 INFO:teuthology.orchestra.run.smithi170.stdout: "op_reply": 105, 2024-09-19T08:58:01.166 INFO:teuthology.orchestra.run.smithi170.stdout: "op_latency": { 2024-09-19T08:58:01.167 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 105, 2024-09-19T08:58:01.167 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.446989632, 2024-09-19T08:58:01.167 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.004257044 2024-09-19T08:58:01.167 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.167 INFO:teuthology.orchestra.run.smithi170.stdout: "op_inflight": 0, 2024-09-19T08:58:01.167 INFO:teuthology.orchestra.run.smithi170.stdout: "oplen_avg": { 2024-09-19T08:58:01.167 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 105, 2024-09-19T08:58:01.167 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 129 2024-09-19T08:58:01.167 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.167 INFO:teuthology.orchestra.run.smithi170.stdout: "op": 105, 2024-09-19T08:58:01.168 INFO:teuthology.orchestra.run.smithi170.stdout: "op_r": 37, 2024-09-19T08:58:01.168 INFO:teuthology.orchestra.run.smithi170.stdout: "op_w": 68, 2024-09-19T08:58:01.168 INFO:teuthology.orchestra.run.smithi170.stdout: "op_rmw": 0, 2024-09-19T08:58:01.168 INFO:teuthology.orchestra.run.smithi170.stdout: "op_pg": 0, 2024-09-19T08:58:01.168 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_stat": 0, 2024-09-19T08:58:01.168 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_create": 10, 2024-09-19T08:58:01.168 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_read": 1, 2024-09-19T08:58:01.168 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_write": 17, 2024-09-19T08:58:01.168 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writefull": 12, 2024-09-19T08:58:01.169 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writesame": 0, 2024-09-19T08:58:01.169 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_append": 0, 2024-09-19T08:58:01.169 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_zero": 0, 2024-09-19T08:58:01.169 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_truncate": 0, 2024-09-19T08:58:01.169 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_delete": 14, 2024-09-19T08:58:01.169 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_mapext": 0, 2024-09-19T08:58:01.169 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_sparse_read": 0, 2024-09-19T08:58:01.169 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_clonerange": 0, 2024-09-19T08:58:01.169 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_getxattr": 36, 2024-09-19T08:58:01.169 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_setxattr": 20, 2024-09-19T08:58:01.169 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_cmpxattr": 0, 2024-09-19T08:58:01.169 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_rmxattr": 0, 2024-09-19T08:58:01.170 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_resetxattrs": 0, 2024-09-19T08:58:01.170 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_call": 0, 2024-09-19T08:58:01.170 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_watch": 0, 2024-09-19T08:58:01.170 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_notify": 0, 2024-09-19T08:58:01.170 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T08:58:01.170 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls": 0, 2024-09-19T08:58:01.170 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls_filter": 0, 2024-09-19T08:58:01.170 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_other": 0, 2024-09-19T08:58:01.170 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_active": 0, 2024-09-19T08:58:01.170 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_send": 0, 2024-09-19T08:58:01.170 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_resend": 0, 2024-09-19T08:58:01.170 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_ping": 0, 2024-09-19T08:58:01.170 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_active": 0, 2024-09-19T08:58:01.171 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_send": 0, 2024-09-19T08:58:01.171 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_resend": 0, 2024-09-19T08:58:01.171 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_active": 0, 2024-09-19T08:58:01.171 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_send": 0, 2024-09-19T08:58:01.171 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_resend": 0, 2024-09-19T08:58:01.171 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_active": 0, 2024-09-19T08:58:01.171 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_send": 0, 2024-09-19T08:58:01.171 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_resend": 0, 2024-09-19T08:58:01.171 INFO:teuthology.orchestra.run.smithi170.stdout: "command_active": 0, 2024-09-19T08:58:01.171 INFO:teuthology.orchestra.run.smithi170.stdout: "command_send": 0, 2024-09-19T08:58:01.171 INFO:teuthology.orchestra.run.smithi170.stdout: "command_resend": 0, 2024-09-19T08:58:01.171 INFO:teuthology.orchestra.run.smithi170.stdout: "map_epoch": 95, 2024-09-19T08:58:01.171 INFO:teuthology.orchestra.run.smithi170.stdout: "map_full": 0, 2024-09-19T08:58:01.172 INFO:teuthology.orchestra.run.smithi170.stdout: "map_inc": 15, 2024-09-19T08:58:01.172 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_sessions": 9, 2024-09-19T08:58:01.172 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_open": 9, 2024-09-19T08:58:01.172 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_close": 0, 2024-09-19T08:58:01.172 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_laggy": 0, 2024-09-19T08:58:01.172 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_wr": 17, 2024-09-19T08:58:01.172 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_rd": 0, 2024-09-19T08:58:01.172 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_del": 2 2024-09-19T08:58:01.172 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.172 INFO:teuthology.orchestra.run.smithi170.stdout: "oft": { 2024-09-19T08:58:01.172 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_objs": 1, 2024-09-19T08:58:01.172 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_kv_pairs": 0, 2024-09-19T08:58:01.172 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_updates": 0, 2024-09-19T08:58:01.173 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_removes": 0 2024-09-19T08:58:01.173 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.173 INFO:teuthology.orchestra.run.smithi170.stdout: "osd-slow-ops": { 2024-09-19T08:58:01.173 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_ops_count": 116 2024-09-19T08:58:01.173 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.173 INFO:teuthology.orchestra.run.smithi170.stdout: "purge_queue": { 2024-09-19T08:58:01.173 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops": 0, 2024-09-19T08:58:01.173 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops_high_water": 2, 2024-09-19T08:58:01.173 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing": 0, 2024-09-19T08:58:01.173 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_high_water": 2, 2024-09-19T08:58:01.173 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed_ops": 2, 2024-09-19T08:58:01.173 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed": 2, 2024-09-19T08:58:01.174 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_item_in_journal": 0 2024-09-19T08:58:01.174 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.174 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T08:58:01.174 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-09-19T08:58:01.174 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-09-19T08:58:01.174 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-09-19T08:58:01.174 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 1993, 2024-09-19T08:58:01.174 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 2131122, 2024-09-19T08:58:01.174 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-09-19T08:58:01.174 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 1993, 2024-09-19T08:58:01.174 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-09-19T08:58:01.174 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-09-19T08:58:01.174 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 1993, 2024-09-19T08:58:01.175 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 2131122, 2024-09-19T08:58:01.175 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-09-19T08:58:01.175 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:01.175 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:01.175 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:01.175 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:58:01.175 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.175 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_bytes": { 2024-09-19T08:58:01.175 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-09-19T08:58:01.175 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-09-19T08:58:01.175 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-09-19T08:58:01.176 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-09-19T08:58:01.176 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-09-19T08:58:01.176 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-09-19T08:58:01.176 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-09-19T08:58:01.176 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 105, 2024-09-19T08:58:01.176 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 46145, 2024-09-19T08:58:01.176 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 105, 2024-09-19T08:58:01.176 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 46145, 2024-09-19T08:58:01.176 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-09-19T08:58:01.176 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:01.176 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:01.177 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:01.177 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:58:01.177 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.177 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_ops": { 2024-09-19T08:58:01.177 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-09-19T08:58:01.177 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 1024, 2024-09-19T08:58:01.177 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-09-19T08:58:01.177 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-09-19T08:58:01.177 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-09-19T08:58:01.177 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-09-19T08:58:01.177 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-09-19T08:58:01.177 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 105, 2024-09-19T08:58:01.178 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 105, 2024-09-19T08:58:01.178 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 105, 2024-09-19T08:58:01.178 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 105, 2024-09-19T08:58:01.178 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-09-19T08:58:01.178 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:01.178 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:01.178 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:01.178 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:58:01.178 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.178 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle": { 2024-09-19T08:58:01.178 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-09-19T08:58:01.178 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-09-19T08:58:01.178 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-09-19T08:58:01.179 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 2, 2024-09-19T08:58:01.179 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 298, 2024-09-19T08:58:01.179 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-09-19T08:58:01.179 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 2, 2024-09-19T08:58:01.179 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-09-19T08:58:01.179 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-09-19T08:58:01.179 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 1, 2024-09-19T08:58:01.179 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 298, 2024-09-19T08:58:01.179 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-09-19T08:58:01.179 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:01.179 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:01.179 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:01.180 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:58:01.180 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:01.180 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle-0x55a6d76b2960": { 2024-09-19T08:58:01.180 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-09-19T08:58:01.180 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-09-19T08:58:01.180 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-09-19T08:58:01.180 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 21, 2024-09-19T08:58:01.180 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 34177, 2024-09-19T08:58:01.180 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-09-19T08:58:01.180 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 21, 2024-09-19T08:58:01.180 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-09-19T08:58:01.180 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-09-19T08:58:01.181 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 16, 2024-09-19T08:58:01.181 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 34177, 2024-09-19T08:58:01.181 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-09-19T08:58:01.181 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:01.181 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:01.181 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:01.181 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:58:01.181 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:58:01.181 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-19T08:58:01.948 INFO:tasks.check_counter:Daemon mds.h mds.exported=1 2024-09-19T08:58:01.948 INFO:tasks.check_counter:Daemon mds.h mds.imported=2 2024-09-19T08:58:01.948 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=3 2024-09-19T08:58:01.948 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=12 2024-09-19T08:58:01.948 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-09-19T08:58:01.948 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-19T08:58:01.949 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T08:58:01.949 DEBUG:tasks.check_counter:Getting stats from k 2024-09-19T08:58:01.949 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-19T08:58:02.124 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:02 smithi170 ceph-mon[35072]: pgmap v381: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:58:02.197 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T08:58:02.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:02 smithi164 ceph-mon[30928]: pgmap v381: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:58:02.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:02 smithi173 ceph-mon[35306]: pgmap v381: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T08:58:02.736 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-19T08:58:02.737 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T08:58:02.737 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 1117, 2024-09-19T08:58:02.737 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 1095, 2024-09-19T08:58:02.737 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 1960640, 2024-09-19T08:58:02.737 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 416358, 2024-09-19T08:58:02.737 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 5, 2024-09-19T08:58:02.737 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 1, 2024-09-19T08:58:02.737 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.145019292, 2024-09-19T08:58:02.737 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.053324840, 2024-09-19T08:58:02.737 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.072003419, 2024-09-19T08:58:02.738 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.026274477, 2024-09-19T08:58:02.738 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:58:02.738 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1095, 2024-09-19T08:58:02.738 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.070128860, 2024-09-19T08:58:02.738 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000064044 2024-09-19T08:58:02.738 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.738 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:58:02.738 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.738 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.738 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.738 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.738 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 1679408, 2024-09-19T08:58:02.739 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 150192 2024-09-19T08:58:02.739 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.739 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T08:58:02.739 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 5, 2024-09-19T08:58:02.739 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 185, 2024-09-19T08:58:02.739 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 1710, 2024-09-19T08:58:02.740 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 477102, 2024-09-19T08:58:02.740 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 7, 2024-09-19T08:58:02.740 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 5, 2024-09-19T08:58:02.740 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.011934287, 2024-09-19T08:58:02.740 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.007547491, 2024-09-19T08:58:02.740 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.002170636, 2024-09-19T08:58:02.740 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.000184777, 2024-09-19T08:58:02.740 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:58:02.740 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 182, 2024-09-19T08:58:02.740 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.015379152, 2024-09-19T08:58:02.740 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000084500 2024-09-19T08:58:02.741 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.741 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:58:02.741 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.741 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.741 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.741 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.741 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T08:58:02.741 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 476080 2024-09-19T08:58:02.741 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.741 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T08:58:02.741 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 1596, 2024-09-19T08:58:02.741 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 1598, 2024-09-19T08:58:02.741 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 408422, 2024-09-19T08:58:02.742 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 507529, 2024-09-19T08:58:02.742 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 5, 2024-09-19T08:58:02.742 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 4, 2024-09-19T08:58:02.742 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.106289602, 2024-09-19T08:58:02.742 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.027552185, 2024-09-19T08:58:02.742 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.028619855, 2024-09-19T08:58:02.742 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.042397672, 2024-09-19T08:58:02.742 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:58:02.742 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1596, 2024-09-19T08:58:02.742 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.056130803, 2024-09-19T08:58:02.742 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000035169 2024-09-19T08:58:02.742 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.743 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:58:02.743 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.743 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.743 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.743 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.743 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T08:58:02.743 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T08:58:02.743 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.743 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2024-09-19T08:58:02.743 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 1, 2024-09-19T08:58:02.743 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2024-09-19T08:58:02.743 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.743 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-MDSRank": { 2024-09-19T08:58:02.744 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-09-19T08:58:02.744 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-09-19T08:58:02.744 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 4050, 2024-09-19T08:58:02.744 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.104319495, 2024-09-19T08:58:02.744 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000025757 2024-09-19T08:58:02.745 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:58:02.745 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.745 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-PurgeQueue": { 2024-09-19T08:58:02.745 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-09-19T08:58:02.745 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-09-19T08:58:02.745 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.745 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.745 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.746 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:58:02.746 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.746 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": { 2024-09-19T08:58:02.746 INFO:teuthology.orchestra.run.smithi170.stdout: "request": 0, 2024-09-19T08:58:02.746 INFO:teuthology.orchestra.run.smithi170.stdout: "reply": 0, 2024-09-19T08:58:02.746 INFO:teuthology.orchestra.run.smithi170.stdout: "reply_latency": { 2024-09-19T08:58:02.746 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.746 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.746 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.746 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.746 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_reply": 0, 2024-09-19T08:58:02.747 INFO:teuthology.orchestra.run.smithi170.stdout: "forward": 0, 2024-09-19T08:58:02.747 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_complete": 0, 2024-09-19T08:58:02.747 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_keys": 0, 2024-09-19T08:58:02.747 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_commit": 0, 2024-09-19T08:58:02.747 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_split": 0, 2024-09-19T08:58:02.747 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_merge": 0, 2024-09-19T08:58:02.747 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes": 0, 2024-09-19T08:58:02.747 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_top": 0, 2024-09-19T08:58:02.747 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_bottom": 0, 2024-09-19T08:58:02.748 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pin_tail": 0, 2024-09-19T08:58:02.748 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pinned": 0, 2024-09-19T08:58:02.748 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_expired": 0, 2024-09-19T08:58:02.748 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_with_caps": 0, 2024-09-19T08:58:02.748 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": 0, 2024-09-19T08:58:02.748 INFO:teuthology.orchestra.run.smithi170.stdout: "subtrees": 1, 2024-09-19T08:58:02.748 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse": 0, 2024-09-19T08:58:02.748 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_hit": 0, 2024-09-19T08:58:02.748 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_forward": 0, 2024-09-19T08:58:02.748 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_discover": 0, 2024-09-19T08:58:02.749 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_dir_fetch": 0, 2024-09-19T08:58:02.749 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_remote_ino": 0, 2024-09-19T08:58:02.749 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_lock": 0, 2024-09-19T08:58:02.749 INFO:teuthology.orchestra.run.smithi170.stdout: "load_cent": 0, 2024-09-19T08:58:02.749 INFO:teuthology.orchestra.run.smithi170.stdout: "q": 0, 2024-09-19T08:58:02.749 INFO:teuthology.orchestra.run.smithi170.stdout: "exported": 0, 2024-09-19T08:58:02.749 INFO:teuthology.orchestra.run.smithi170.stdout: "exported_inodes": 0, 2024-09-19T08:58:02.750 INFO:teuthology.orchestra.run.smithi170.stdout: "imported": 0, 2024-09-19T08:58:02.750 INFO:teuthology.orchestra.run.smithi170.stdout: "imported_inodes": 0, 2024-09-19T08:58:02.750 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_dir_fetch": 0, 2024-09-19T08:58:02.750 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_backtrace_fetch": 0, 2024-09-19T08:58:02.750 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_peer_discover": 0, 2024-09-19T08:58:02.750 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rfiles": 0, 2024-09-19T08:58:02.750 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rbytes": 0, 2024-09-19T08:58:02.750 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rsnaps": 0, 2024-09-19T08:58:02.750 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T08:58:02.750 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_set_tag": 0, 2024-09-19T08:58:02.750 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T08:58:02.750 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_inotable_repaired": 0, 2024-09-19T08:58:02.750 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_inodes": 0, 2024-09-19T08:58:02.751 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T08:58:02.751 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T08:58:02.751 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_file_inodes": 0, 2024-09-19T08:58:02.751 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_inode_file_caps": 0, 2024-09-19T08:58:02.751 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T08:58:02.751 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_grant": 0, 2024-09-19T08:58:02.751 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T08:58:02.751 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T08:58:02.751 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T08:58:02.752 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps": 0, 2024-09-19T08:58:02.752 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps_dirty": 0, 2024-09-19T08:58:02.752 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_cap_release": 0, 2024-09-19T08:58:02.752 INFO:teuthology.orchestra.run.smithi170.stdout: "process_request_cap_release": 0 2024-09-19T08:58:02.752 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.752 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_cache": { 2024-09-19T08:58:02.752 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update": 0, 2024-09-19T08:58:02.752 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update_receipt": 0, 2024-09-19T08:58:02.752 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_try_discover": 0, 2024-09-19T08:58:02.752 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_send_discover": 0, 2024-09-19T08:58:02.752 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_handle_discover": 0, 2024-09-19T08:58:02.752 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays": 0, 2024-09-19T08:58:02.752 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_delayed": 0, 2024-09-19T08:58:02.753 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_enqueuing": 0, 2024-09-19T08:58:02.753 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_created": 0, 2024-09-19T08:58:02.753 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_enqueued": 0, 2024-09-19T08:58:02.753 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_reintegrated": 0, 2024-09-19T08:58:02.753 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_migrated": 0, 2024-09-19T08:58:02.753 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_processing": 0, 2024-09-19T08:58:02.753 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_enqueued": 0, 2024-09-19T08:58:02.753 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_prioritized": 0, 2024-09-19T08:58:02.753 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_started": 0, 2024-09-19T08:58:02.753 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_completed": 0, 2024-09-19T08:58:02.753 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_path": 0, 2024-09-19T08:58:02.753 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_inode": 0, 2024-09-19T08:58:02.754 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T08:58:02.754 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_exportdir": 0, 2024-09-19T08:58:02.754 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_flush": 0, 2024-09-19T08:58:02.754 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragmentdir": 0, 2024-09-19T08:58:02.754 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragstats": 0, 2024-09-19T08:58:02.754 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_inodestats": 0 2024-09-19T08:58:02.754 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.754 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_log": { 2024-09-19T08:58:02.754 INFO:teuthology.orchestra.run.smithi170.stdout: "evlrg": 0, 2024-09-19T08:58:02.755 INFO:teuthology.orchestra.run.smithi170.stdout: "evadd": 0, 2024-09-19T08:58:02.755 INFO:teuthology.orchestra.run.smithi170.stdout: "evex": 0, 2024-09-19T08:58:02.755 INFO:teuthology.orchestra.run.smithi170.stdout: "evtrm": 0, 2024-09-19T08:58:02.755 INFO:teuthology.orchestra.run.smithi170.stdout: "ev": 8, 2024-09-19T08:58:02.755 INFO:teuthology.orchestra.run.smithi170.stdout: "evexg": 0, 2024-09-19T08:58:02.755 INFO:teuthology.orchestra.run.smithi170.stdout: "evexd": 0, 2024-09-19T08:58:02.756 INFO:teuthology.orchestra.run.smithi170.stdout: "segadd": 0, 2024-09-19T08:58:02.756 INFO:teuthology.orchestra.run.smithi170.stdout: "segex": 0, 2024-09-19T08:58:02.756 INFO:teuthology.orchestra.run.smithi170.stdout: "segtrm": 0, 2024-09-19T08:58:02.756 INFO:teuthology.orchestra.run.smithi170.stdout: "seg": 2, 2024-09-19T08:58:02.756 INFO:teuthology.orchestra.run.smithi170.stdout: "segmjr": 2, 2024-09-19T08:58:02.756 INFO:teuthology.orchestra.run.smithi170.stdout: "segexg": 0, 2024-09-19T08:58:02.756 INFO:teuthology.orchestra.run.smithi170.stdout: "segexd": 0, 2024-09-19T08:58:02.756 INFO:teuthology.orchestra.run.smithi170.stdout: "expos": 4194304, 2024-09-19T08:58:02.756 INFO:teuthology.orchestra.run.smithi170.stdout: "wrpos": 4196661, 2024-09-19T08:58:02.756 INFO:teuthology.orchestra.run.smithi170.stdout: "rdpos": 4196353, 2024-09-19T08:58:02.756 INFO:teuthology.orchestra.run.smithi170.stdout: "jlat": { 2024-09-19T08:58:02.756 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.757 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.757 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.757 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.757 INFO:teuthology.orchestra.run.smithi170.stdout: "replayed": 8 2024-09-19T08:58:02.757 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.757 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_mem": { 2024-09-19T08:58:02.757 INFO:teuthology.orchestra.run.smithi170.stdout: "ino": 3, 2024-09-19T08:58:02.757 INFO:teuthology.orchestra.run.smithi170.stdout: "ino+": 3, 2024-09-19T08:58:02.757 INFO:teuthology.orchestra.run.smithi170.stdout: "ino-": 0, 2024-09-19T08:58:02.757 INFO:teuthology.orchestra.run.smithi170.stdout: "dir": 1, 2024-09-19T08:58:02.757 INFO:teuthology.orchestra.run.smithi170.stdout: "dir+": 1, 2024-09-19T08:58:02.757 INFO:teuthology.orchestra.run.smithi170.stdout: "dir-": 0, 2024-09-19T08:58:02.757 INFO:teuthology.orchestra.run.smithi170.stdout: "dn": 0, 2024-09-19T08:58:02.758 INFO:teuthology.orchestra.run.smithi170.stdout: "dn+": 0, 2024-09-19T08:58:02.758 INFO:teuthology.orchestra.run.smithi170.stdout: "dn-": 0, 2024-09-19T08:58:02.758 INFO:teuthology.orchestra.run.smithi170.stdout: "cap": 0, 2024-09-19T08:58:02.758 INFO:teuthology.orchestra.run.smithi170.stdout: "cap+": 0, 2024-09-19T08:58:02.758 INFO:teuthology.orchestra.run.smithi170.stdout: "cap-": 0, 2024-09-19T08:58:02.758 INFO:teuthology.orchestra.run.smithi170.stdout: "rss": 51532, 2024-09-19T08:58:02.758 INFO:teuthology.orchestra.run.smithi170.stdout: "heap": 215328 2024-09-19T08:58:02.758 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.758 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_server": { 2024-09-19T08:58:02.758 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_client_request": 0, 2024-09-19T08:58:02.758 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_server_request": 0, 2024-09-19T08:58:02.758 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_request": 0, 2024-09-19T08:58:02.758 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_session": 0, 2024-09-19T08:58:02.759 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_peer_request": 0, 2024-09-19T08:58:02.759 INFO:teuthology.orchestra.run.smithi170.stdout: "req_create_latency": { 2024-09-19T08:58:02.759 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.759 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.759 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.759 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.759 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getattr_latency": { 2024-09-19T08:58:02.759 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.759 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.759 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.759 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.759 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getfilelock_latency": { 2024-09-19T08:58:02.759 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.760 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.760 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.760 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.760 INFO:teuthology.orchestra.run.smithi170.stdout: "req_link_latency": { 2024-09-19T08:58:02.760 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.760 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.760 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.760 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.760 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookup_latency": { 2024-09-19T08:58:02.760 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.760 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.760 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.760 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.761 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookuphash_latency": { 2024-09-19T08:58:02.761 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.761 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.761 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.761 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.761 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupino_latency": { 2024-09-19T08:58:02.761 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.761 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.761 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.761 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.761 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupname_latency": { 2024-09-19T08:58:02.761 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.761 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.762 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.762 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.762 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupparent_latency": { 2024-09-19T08:58:02.762 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.762 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.762 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.762 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.762 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupsnap_latency": { 2024-09-19T08:58:02.762 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.762 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.762 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.762 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.762 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lssnap_latency": { 2024-09-19T08:58:02.763 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.763 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.763 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.763 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.763 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mkdir_latency": { 2024-09-19T08:58:02.763 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.764 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.764 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.764 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.764 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mknod_latency": { 2024-09-19T08:58:02.764 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.764 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.764 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.764 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.764 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mksnap_latency": { 2024-09-19T08:58:02.764 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.764 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.765 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.765 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.765 INFO:teuthology.orchestra.run.smithi170.stdout: "req_open_latency": { 2024-09-19T08:58:02.765 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.765 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.765 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.765 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.765 INFO:teuthology.orchestra.run.smithi170.stdout: "req_readdir_latency": { 2024-09-19T08:58:02.766 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.766 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.766 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.766 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.766 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rename_latency": { 2024-09-19T08:58:02.766 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.766 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.766 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.766 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.766 INFO:teuthology.orchestra.run.smithi170.stdout: "req_renamesnap_latency": { 2024-09-19T08:58:02.767 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.767 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.767 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.767 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.767 INFO:teuthology.orchestra.run.smithi170.stdout: "req_snapdiff_latency": { 2024-09-19T08:58:02.767 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.767 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.767 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.768 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.768 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmdir_latency": { 2024-09-19T08:58:02.768 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.768 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.768 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.768 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.768 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmsnap_latency": { 2024-09-19T08:58:02.768 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.768 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.768 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.768 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.768 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmxattr_latency": { 2024-09-19T08:58:02.768 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.769 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.769 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.769 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.769 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setattr_latency": { 2024-09-19T08:58:02.769 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.769 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.769 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.769 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.769 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setdirlayout_latency": { 2024-09-19T08:58:02.769 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.769 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.769 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.770 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.770 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setfilelock_latency": { 2024-09-19T08:58:02.770 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.770 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.770 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.770 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.770 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setlayout_latency": { 2024-09-19T08:58:02.770 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.770 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.770 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.770 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.770 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setxattr_latency": { 2024-09-19T08:58:02.771 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.771 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.771 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.771 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.771 INFO:teuthology.orchestra.run.smithi170.stdout: "req_symlink_latency": { 2024-09-19T08:58:02.771 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.771 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.771 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.771 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.771 INFO:teuthology.orchestra.run.smithi170.stdout: "req_unlink_latency": { 2024-09-19T08:58:02.771 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.771 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.771 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.772 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.772 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_revoke_eviction": 0, 2024-09-19T08:58:02.772 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_acquisition_throttle": 0, 2024-09-19T08:58:02.772 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getvxattr_latency": { 2024-09-19T08:58:02.772 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.772 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.772 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.772 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:58:02.772 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.772 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_sessions": { 2024-09-19T08:58:02.772 INFO:teuthology.orchestra.run.smithi170.stdout: "session_count": 2, 2024-09-19T08:58:02.772 INFO:teuthology.orchestra.run.smithi170.stdout: "session_add": 4, 2024-09-19T08:58:02.772 INFO:teuthology.orchestra.run.smithi170.stdout: "session_remove": 2, 2024-09-19T08:58:02.773 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_open": 2, 2024-09-19T08:58:02.773 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_stale": 0, 2024-09-19T08:58:02.773 INFO:teuthology.orchestra.run.smithi170.stdout: "total_load": 0, 2024-09-19T08:58:02.773 INFO:teuthology.orchestra.run.smithi170.stdout: "average_load": 0, 2024-09-19T08:58:02.773 INFO:teuthology.orchestra.run.smithi170.stdout: "avg_session_uptime": 3186, 2024-09-19T08:58:02.773 INFO:teuthology.orchestra.run.smithi170.stdout: "mdthresh_evicted": 0 2024-09-19T08:58:02.773 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.773 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2024-09-19T08:58:02.773 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2024-09-19T08:58:02.773 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2024-09-19T08:58:02.773 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T08:58:02.773 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2024-09-19T08:58:02.774 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T08:58:02.774 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2024-09-19T08:58:02.774 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T08:58:02.774 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T08:58:02.774 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T08:58:02.774 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T08:58:02.774 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T08:58:02.774 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2024-09-19T08:58:02.774 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T08:58:02.774 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T08:58:02.774 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2024-09-19T08:58:02.774 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2024-09-19T08:58:02.774 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2024-09-19T08:58:02.775 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2024-09-19T08:58:02.775 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T08:58:02.775 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T08:58:02.775 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T08:58:02.775 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T08:58:02.775 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T08:58:02.775 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2024-09-19T08:58:02.775 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2024-09-19T08:58:02.775 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2024-09-19T08:58:02.776 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T08:58:02.776 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T08:58:02.776 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2024-09-19T08:58:02.776 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2024-09-19T08:58:02.776 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2024-09-19T08:58:02.776 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2024-09-19T08:58:02.776 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T08:58:02.776 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2024-09-19T08:58:02.776 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T08:58:02.776 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2024-09-19T08:58:02.776 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 197441, 2024-09-19T08:58:02.776 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 54, 2024-09-19T08:58:02.777 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 0, 2024-09-19T08:58:02.777 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 0, 2024-09-19T08:58:02.777 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2024-09-19T08:58:02.777 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2024-09-19T08:58:02.777 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2024-09-19T08:58:02.777 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2024-09-19T08:58:02.777 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2024-09-19T08:58:02.777 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2024-09-19T08:58:02.777 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 7936, 2024-09-19T08:58:02.777 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 175, 2024-09-19T08:58:02.777 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T08:58:02.777 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2024-09-19T08:58:02.777 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2024-09-19T08:58:02.778 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2024-09-19T08:58:02.778 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 8512, 2024-09-19T08:58:02.778 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 13, 2024-09-19T08:58:02.778 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2024-09-19T08:58:02.778 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2024-09-19T08:58:02.778 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2024-09-19T08:58:02.778 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2024-09-19T08:58:02.778 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.778 INFO:teuthology.orchestra.run.smithi170.stdout: "objecter": { 2024-09-19T08:58:02.778 INFO:teuthology.orchestra.run.smithi170.stdout: "op_active": 0, 2024-09-19T08:58:02.778 INFO:teuthology.orchestra.run.smithi170.stdout: "op_laggy": 0, 2024-09-19T08:58:02.778 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send": 2436, 2024-09-19T08:58:02.778 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send_bytes": 21, 2024-09-19T08:58:02.779 INFO:teuthology.orchestra.run.smithi170.stdout: "op_resend": 3, 2024-09-19T08:58:02.779 INFO:teuthology.orchestra.run.smithi170.stdout: "op_reply": 2436, 2024-09-19T08:58:02.779 INFO:teuthology.orchestra.run.smithi170.stdout: "op_latency": { 2024-09-19T08:58:02.779 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2436, 2024-09-19T08:58:02.779 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 1.992953760, 2024-09-19T08:58:02.779 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000818125 2024-09-19T08:58:02.779 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.779 INFO:teuthology.orchestra.run.smithi170.stdout: "op_inflight": 0, 2024-09-19T08:58:02.779 INFO:teuthology.orchestra.run.smithi170.stdout: "oplen_avg": { 2024-09-19T08:58:02.779 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2436, 2024-09-19T08:58:02.780 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 2437 2024-09-19T08:58:02.780 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.780 INFO:teuthology.orchestra.run.smithi170.stdout: "op": 2436, 2024-09-19T08:58:02.780 INFO:teuthology.orchestra.run.smithi170.stdout: "op_r": 2436, 2024-09-19T08:58:02.780 INFO:teuthology.orchestra.run.smithi170.stdout: "op_w": 0, 2024-09-19T08:58:02.780 INFO:teuthology.orchestra.run.smithi170.stdout: "op_rmw": 0, 2024-09-19T08:58:02.780 INFO:teuthology.orchestra.run.smithi170.stdout: "op_pg": 0, 2024-09-19T08:58:02.780 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_stat": 1587, 2024-09-19T08:58:02.780 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_create": 0, 2024-09-19T08:58:02.780 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_read": 847, 2024-09-19T08:58:02.780 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_write": 0, 2024-09-19T08:58:02.780 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writefull": 0, 2024-09-19T08:58:02.781 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writesame": 0, 2024-09-19T08:58:02.781 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_append": 0, 2024-09-19T08:58:02.781 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_zero": 0, 2024-09-19T08:58:02.781 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_truncate": 0, 2024-09-19T08:58:02.781 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_delete": 0, 2024-09-19T08:58:02.781 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_mapext": 0, 2024-09-19T08:58:02.781 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_sparse_read": 0, 2024-09-19T08:58:02.781 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_clonerange": 0, 2024-09-19T08:58:02.781 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_getxattr": 1, 2024-09-19T08:58:02.781 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_setxattr": 0, 2024-09-19T08:58:02.781 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_cmpxattr": 0, 2024-09-19T08:58:02.781 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_rmxattr": 0, 2024-09-19T08:58:02.781 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_resetxattrs": 0, 2024-09-19T08:58:02.782 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_call": 0, 2024-09-19T08:58:02.782 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_watch": 0, 2024-09-19T08:58:02.782 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_notify": 0, 2024-09-19T08:58:02.782 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T08:58:02.782 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls": 0, 2024-09-19T08:58:02.782 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls_filter": 0, 2024-09-19T08:58:02.782 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_other": 0, 2024-09-19T08:58:02.782 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_active": 0, 2024-09-19T08:58:02.782 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_send": 0, 2024-09-19T08:58:02.782 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_resend": 0, 2024-09-19T08:58:02.782 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_ping": 0, 2024-09-19T08:58:02.782 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_active": 0, 2024-09-19T08:58:02.782 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_send": 0, 2024-09-19T08:58:02.782 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_resend": 0, 2024-09-19T08:58:02.783 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_active": 0, 2024-09-19T08:58:02.783 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_send": 0, 2024-09-19T08:58:02.783 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_resend": 0, 2024-09-19T08:58:02.783 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_active": 0, 2024-09-19T08:58:02.783 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_send": 0, 2024-09-19T08:58:02.783 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_resend": 0, 2024-09-19T08:58:02.783 INFO:teuthology.orchestra.run.smithi170.stdout: "command_active": 0, 2024-09-19T08:58:02.783 INFO:teuthology.orchestra.run.smithi170.stdout: "command_send": 0, 2024-09-19T08:58:02.783 INFO:teuthology.orchestra.run.smithi170.stdout: "command_resend": 0, 2024-09-19T08:58:02.783 INFO:teuthology.orchestra.run.smithi170.stdout: "map_epoch": 95, 2024-09-19T08:58:02.784 INFO:teuthology.orchestra.run.smithi170.stdout: "map_full": 0, 2024-09-19T08:58:02.784 INFO:teuthology.orchestra.run.smithi170.stdout: "map_inc": 13, 2024-09-19T08:58:02.784 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_sessions": 6, 2024-09-19T08:58:02.784 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_open": 6, 2024-09-19T08:58:02.784 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_close": 0, 2024-09-19T08:58:02.784 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_laggy": 0, 2024-09-19T08:58:02.784 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_wr": 0, 2024-09-19T08:58:02.784 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_rd": 2, 2024-09-19T08:58:02.784 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_del": 0 2024-09-19T08:58:02.784 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.784 INFO:teuthology.orchestra.run.smithi170.stdout: "oft": { 2024-09-19T08:58:02.784 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_objs": 0, 2024-09-19T08:58:02.785 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_kv_pairs": 0, 2024-09-19T08:58:02.785 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_updates": 0, 2024-09-19T08:58:02.785 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_removes": 0 2024-09-19T08:58:02.785 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.785 INFO:teuthology.orchestra.run.smithi170.stdout: "osd-slow-ops": { 2024-09-19T08:58:02.785 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_ops_count": 0 2024-09-19T08:58:02.785 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.785 INFO:teuthology.orchestra.run.smithi170.stdout: "purge_queue": { 2024-09-19T08:58:02.785 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops": 0, 2024-09-19T08:58:02.785 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T08:58:02.785 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing": 0, 2024-09-19T08:58:02.786 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_high_water": 0, 2024-09-19T08:58:02.786 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed_ops": 0, 2024-09-19T08:58:02.786 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed": 0, 2024-09-19T08:58:02.786 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_item_in_journal": 0 2024-09-19T08:58:02.786 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.786 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T08:58:02.786 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-09-19T08:58:02.787 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-09-19T08:58:02.787 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-09-19T08:58:02.787 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 2714, 2024-09-19T08:58:02.787 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 2109363, 2024-09-19T08:58:02.787 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-09-19T08:58:02.787 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 2714, 2024-09-19T08:58:02.787 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-09-19T08:58:02.787 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-09-19T08:58:02.787 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 2714, 2024-09-19T08:58:02.787 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 2109363, 2024-09-19T08:58:02.787 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-09-19T08:58:02.787 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.787 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.788 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.788 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:58:02.788 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.788 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_bytes": { 2024-09-19T08:58:02.788 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-09-19T08:58:02.788 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-09-19T08:58:02.788 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-09-19T08:58:02.788 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-09-19T08:58:02.788 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-09-19T08:58:02.788 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-09-19T08:58:02.788 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-09-19T08:58:02.788 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 2436, 2024-09-19T08:58:02.788 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 2362, 2024-09-19T08:58:02.789 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 2436, 2024-09-19T08:58:02.789 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 2362, 2024-09-19T08:58:02.789 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-09-19T08:58:02.789 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.789 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.789 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.789 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:58:02.789 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.789 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_ops": { 2024-09-19T08:58:02.790 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-09-19T08:58:02.790 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 1024, 2024-09-19T08:58:02.790 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-09-19T08:58:02.790 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-09-19T08:58:02.790 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-09-19T08:58:02.790 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-09-19T08:58:02.790 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-09-19T08:58:02.790 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 2436, 2024-09-19T08:58:02.790 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 2436, 2024-09-19T08:58:02.790 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 2436, 2024-09-19T08:58:02.790 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 2436, 2024-09-19T08:58:02.790 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-09-19T08:58:02.790 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.791 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.791 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.791 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:58:02.791 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.791 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle": { 2024-09-19T08:58:02.791 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-09-19T08:58:02.791 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-09-19T08:58:02.791 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-09-19T08:58:02.791 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-09-19T08:58:02.791 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-09-19T08:58:02.791 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-09-19T08:58:02.791 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-09-19T08:58:02.791 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-09-19T08:58:02.792 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-09-19T08:58:02.792 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 0, 2024-09-19T08:58:02.792 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 0, 2024-09-19T08:58:02.792 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-09-19T08:58:02.792 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.792 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.792 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.792 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:58:02.792 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-19T08:58:02.792 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle-0x55adb2352b40": { 2024-09-19T08:58:02.792 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-09-19T08:58:02.792 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-09-19T08:58:02.793 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-09-19T08:58:02.793 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-09-19T08:58:02.793 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-09-19T08:58:02.793 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-09-19T08:58:02.793 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-09-19T08:58:02.793 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-09-19T08:58:02.793 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-09-19T08:58:02.793 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 0, 2024-09-19T08:58:02.793 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 0, 2024-09-19T08:58:02.793 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-09-19T08:58:02.793 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-09-19T08:58:02.793 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-09-19T08:58:02.794 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-09-19T08:58:02.794 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:58:02.794 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-19T08:58:02.794 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-19T08:58:03.417 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-19T08:58:03.417 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-19T08:58:03.417 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-09-19T08:58:03.417 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-09-19T08:58:03.417 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-09-19T08:58:03.417 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-09-19T08:58:03.417 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T08:58:03.417 DEBUG:tasks.check_counter:Getting stats from c 2024-09-19T08:58:03.418 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-19T08:58:03.676 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:58:04.093 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:04 smithi173 ceph-mon[35306]: pgmap v382: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:58:04.195 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-19T08:58:04.196 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T08:58:04.196 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 71, 2024-09-19T08:58:04.196 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 226, 2024-09-19T08:58:04.196 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 14384, 2024-09-19T08:58:04.196 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 784997, 2024-09-19T08:58:04.196 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 49, 2024-09-19T08:58:04.196 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 1, 2024-09-19T08:58:04.196 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.030580454, 2024-09-19T08:58:04.196 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.007929785, 2024-09-19T08:58:04.196 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.015971826, 2024-09-19T08:58:04.196 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.000718182, 2024-09-19T08:58:04.197 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:58:04.197 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 223, 2024-09-19T08:58:04.197 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.036466751, 2024-09-19T08:58:04.197 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000163528 2024-09-19T08:58:04.197 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.197 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:58:04.197 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 54, 2024-09-19T08:58:04.197 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000006245, 2024-09-19T08:58:04.197 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000115 2024-09-19T08:58:04.197 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.197 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 2368, 2024-09-19T08:58:04.197 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 356272 2024-09-19T08:58:04.197 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.197 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T08:58:04.198 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 1293, 2024-09-19T08:58:04.198 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 1095, 2024-09-19T08:58:04.198 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 450379, 2024-09-19T08:58:04.198 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 558856, 2024-09-19T08:58:04.198 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 10, 2024-09-19T08:58:04.198 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 5, 2024-09-19T08:58:04.198 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.108823530, 2024-09-19T08:58:04.198 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.041298389, 2024-09-19T08:58:04.198 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.074054744, 2024-09-19T08:58:04.198 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.005418009, 2024-09-19T08:58:04.198 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:58:04.198 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 1094, 2024-09-19T08:58:04.198 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.061407026, 2024-09-19T08:58:04.199 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000056130 2024-09-19T08:58:04.199 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.199 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:58:04.199 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 2257, 2024-09-19T08:58:04.199 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000492419, 2024-09-19T08:58:04.199 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000218 2024-09-19T08:58:04.199 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.199 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 5072, 2024-09-19T08:58:04.199 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 131392 2024-09-19T08:58:04.199 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.199 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T08:58:04.199 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 438, 2024-09-19T08:58:04.199 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 418, 2024-09-19T08:58:04.200 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 1799306, 2024-09-19T08:58:04.200 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 299235, 2024-09-19T08:58:04.200 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 9, 2024-09-19T08:58:04.200 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 6, 2024-09-19T08:58:04.200 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.058210699, 2024-09-19T08:58:04.200 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.020649310, 2024-09-19T08:58:04.200 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.053998769, 2024-09-19T08:58:04.200 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.000255842, 2024-09-19T08:58:04.200 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:58:04.200 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 417, 2024-09-19T08:58:04.201 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.031210873, 2024-09-19T08:58:04.201 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000074846 2024-09-19T08:58:04.201 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.201 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:58:04.201 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 237, 2024-09-19T08:58:04.201 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000050633, 2024-09-19T08:58:04.201 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000213 2024-09-19T08:58:04.201 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.201 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 1683088, 2024-09-19T08:58:04.201 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 152144 2024-09-19T08:58:04.201 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.201 INFO:teuthology.orchestra.run.smithi173.stdout: "cct": { 2024-09-19T08:58:04.201 INFO:teuthology.orchestra.run.smithi173.stdout: "total_workers": 1, 2024-09-19T08:58:04.202 INFO:teuthology.orchestra.run.smithi173.stdout: "unhealthy_workers": 0 2024-09-19T08:58:04.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.202 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-MDSRank": { 2024-09-19T08:58:04.202 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2024-09-19T08:58:04.202 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2024-09-19T08:58:04.202 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 49, 2024-09-19T08:58:04.202 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.001992992, 2024-09-19T08:58:04.202 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000040673 2024-09-19T08:58:04.202 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:04.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.202 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-PurgeQueue": { 2024-09-19T08:58:04.202 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2024-09-19T08:58:04.203 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2024-09-19T08:58:04.203 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 2, 2024-09-19T08:58:04.203 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000012485, 2024-09-19T08:58:04.203 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000006242 2024-09-19T08:58:04.203 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:04.203 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.203 INFO:teuthology.orchestra.run.smithi173.stdout: "mds": { 2024-09-19T08:58:04.203 INFO:teuthology.orchestra.run.smithi173.stdout: "request": 0, 2024-09-19T08:58:04.203 INFO:teuthology.orchestra.run.smithi173.stdout: "reply": 0, 2024-09-19T08:58:04.203 INFO:teuthology.orchestra.run.smithi173.stdout: "reply_latency": { 2024-09-19T08:58:04.204 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.204 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.204 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.204 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.204 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_reply": 0, 2024-09-19T08:58:04.204 INFO:teuthology.orchestra.run.smithi173.stdout: "forward": 0, 2024-09-19T08:58:04.204 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_complete": 0, 2024-09-19T08:58:04.204 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_keys": 0, 2024-09-19T08:58:04.204 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_commit": 11, 2024-09-19T08:58:04.204 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_split": 0, 2024-09-19T08:58:04.204 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_merge": 0, 2024-09-19T08:58:04.204 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes": 16, 2024-09-19T08:58:04.204 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_top": 0, 2024-09-19T08:58:04.205 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_bottom": 16, 2024-09-19T08:58:04.205 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pin_tail": 0, 2024-09-19T08:58:04.205 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pinned": 13, 2024-09-19T08:58:04.205 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_expired": 8, 2024-09-19T08:58:04.205 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_with_caps": 0, 2024-09-19T08:58:04.205 INFO:teuthology.orchestra.run.smithi173.stdout: "caps": 0, 2024-09-19T08:58:04.205 INFO:teuthology.orchestra.run.smithi173.stdout: "subtrees": 3, 2024-09-19T08:58:04.205 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse": 32, 2024-09-19T08:58:04.205 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_hit": 27, 2024-09-19T08:58:04.205 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_forward": 0, 2024-09-19T08:58:04.205 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_discover": 5, 2024-09-19T08:58:04.205 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_dir_fetch": 0, 2024-09-19T08:58:04.205 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_remote_ino": 0, 2024-09-19T08:58:04.206 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_lock": 0, 2024-09-19T08:58:04.206 INFO:teuthology.orchestra.run.smithi173.stdout: "load_cent": 1000, 2024-09-19T08:58:04.206 INFO:teuthology.orchestra.run.smithi173.stdout: "q": 1, 2024-09-19T08:58:04.206 INFO:teuthology.orchestra.run.smithi173.stdout: "exported": 0, 2024-09-19T08:58:04.206 INFO:teuthology.orchestra.run.smithi173.stdout: "exported_inodes": 0, 2024-09-19T08:58:04.206 INFO:teuthology.orchestra.run.smithi173.stdout: "imported": 0, 2024-09-19T08:58:04.206 INFO:teuthology.orchestra.run.smithi173.stdout: "imported_inodes": 0, 2024-09-19T08:58:04.206 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_dir_fetch": 0, 2024-09-19T08:58:04.206 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_backtrace_fetch": 0, 2024-09-19T08:58:04.206 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_peer_discover": 0, 2024-09-19T08:58:04.206 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rfiles": 22, 2024-09-19T08:58:04.206 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rbytes": 10738142113, 2024-09-19T08:58:04.206 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rsnaps": 6, 2024-09-19T08:58:04.207 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T08:58:04.207 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_set_tag": 0, 2024-09-19T08:58:04.207 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T08:58:04.207 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_inotable_repaired": 0, 2024-09-19T08:58:04.208 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_inodes": 0, 2024-09-19T08:58:04.208 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T08:58:04.208 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T08:58:04.208 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_file_inodes": 0, 2024-09-19T08:58:04.208 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_inode_file_caps": 0, 2024-09-19T08:58:04.208 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T08:58:04.208 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_grant": 0, 2024-09-19T08:58:04.208 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T08:58:04.209 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T08:58:04.209 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T08:58:04.209 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps": 0, 2024-09-19T08:58:04.209 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps_dirty": 0, 2024-09-19T08:58:04.209 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_cap_release": 0, 2024-09-19T08:58:04.209 INFO:teuthology.orchestra.run.smithi173.stdout: "process_request_cap_release": 0 2024-09-19T08:58:04.209 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.209 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_cache": { 2024-09-19T08:58:04.209 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update": 0, 2024-09-19T08:58:04.209 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update_receipt": 17, 2024-09-19T08:58:04.209 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_try_discover": 11, 2024-09-19T08:58:04.209 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_send_discover": 13, 2024-09-19T08:58:04.210 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_handle_discover": 0, 2024-09-19T08:58:04.210 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays": 0, 2024-09-19T08:58:04.210 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_delayed": 0, 2024-09-19T08:58:04.210 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_enqueuing": 0, 2024-09-19T08:58:04.210 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_created": 0, 2024-09-19T08:58:04.210 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_enqueued": 0, 2024-09-19T08:58:04.210 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_reintegrated": 0, 2024-09-19T08:58:04.210 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_migrated": 0, 2024-09-19T08:58:04.210 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_processing": 0, 2024-09-19T08:58:04.210 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_enqueued": 0, 2024-09-19T08:58:04.210 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_prioritized": 0, 2024-09-19T08:58:04.210 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_started": 0, 2024-09-19T08:58:04.210 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_completed": 0, 2024-09-19T08:58:04.211 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_quiesce_path": 10, 2024-09-19T08:58:04.211 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_quiesce_inode": 86, 2024-09-19T08:58:04.211 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T08:58:04.211 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_exportdir": 0, 2024-09-19T08:58:04.211 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_flush": 0, 2024-09-19T08:58:04.211 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragmentdir": 0, 2024-09-19T08:58:04.211 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragstats": 0, 2024-09-19T08:58:04.211 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_inodestats": 0 2024-09-19T08:58:04.211 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.211 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_log": { 2024-09-19T08:58:04.211 INFO:teuthology.orchestra.run.smithi173.stdout: "evlrg": 0, 2024-09-19T08:58:04.211 INFO:teuthology.orchestra.run.smithi173.stdout: "evadd": 8, 2024-09-19T08:58:04.211 INFO:teuthology.orchestra.run.smithi173.stdout: "evex": 0, 2024-09-19T08:58:04.212 INFO:teuthology.orchestra.run.smithi173.stdout: "evtrm": 0, 2024-09-19T08:58:04.212 INFO:teuthology.orchestra.run.smithi173.stdout: "ev": 8, 2024-09-19T08:58:04.212 INFO:teuthology.orchestra.run.smithi173.stdout: "evexg": 0, 2024-09-19T08:58:04.212 INFO:teuthology.orchestra.run.smithi173.stdout: "evexd": 0, 2024-09-19T08:58:04.212 INFO:teuthology.orchestra.run.smithi173.stdout: "segadd": 2, 2024-09-19T08:58:04.212 INFO:teuthology.orchestra.run.smithi173.stdout: "segex": 0, 2024-09-19T08:58:04.212 INFO:teuthology.orchestra.run.smithi173.stdout: "segtrm": 0, 2024-09-19T08:58:04.212 INFO:teuthology.orchestra.run.smithi173.stdout: "seg": 2, 2024-09-19T08:58:04.212 INFO:teuthology.orchestra.run.smithi173.stdout: "segmjr": 2, 2024-09-19T08:58:04.212 INFO:teuthology.orchestra.run.smithi173.stdout: "segexg": 0, 2024-09-19T08:58:04.212 INFO:teuthology.orchestra.run.smithi173.stdout: "segexd": 0, 2024-09-19T08:58:04.212 INFO:teuthology.orchestra.run.smithi173.stdout: "expos": 4194304, 2024-09-19T08:58:04.212 INFO:teuthology.orchestra.run.smithi173.stdout: "wrpos": 4196661, 2024-09-19T08:58:04.213 INFO:teuthology.orchestra.run.smithi173.stdout: "rdpos": 0, 2024-09-19T08:58:04.213 INFO:teuthology.orchestra.run.smithi173.stdout: "jlat": { 2024-09-19T08:58:04.213 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 7, 2024-09-19T08:58:04.213 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.022241813, 2024-09-19T08:58:04.213 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.003177401 2024-09-19T08:58:04.213 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.213 INFO:teuthology.orchestra.run.smithi173.stdout: "replayed": 0 2024-09-19T08:58:04.213 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.213 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_mem": { 2024-09-19T08:58:04.213 INFO:teuthology.orchestra.run.smithi173.stdout: "ino": 20, 2024-09-19T08:58:04.213 INFO:teuthology.orchestra.run.smithi173.stdout: "ino+": 24, 2024-09-19T08:58:04.213 INFO:teuthology.orchestra.run.smithi173.stdout: "ino-": 4, 2024-09-19T08:58:04.213 INFO:teuthology.orchestra.run.smithi173.stdout: "dir": 18, 2024-09-19T08:58:04.214 INFO:teuthology.orchestra.run.smithi173.stdout: "dir+": 25, 2024-09-19T08:58:04.214 INFO:teuthology.orchestra.run.smithi173.stdout: "dir-": 7, 2024-09-19T08:58:04.214 INFO:teuthology.orchestra.run.smithi173.stdout: "dn": 16, 2024-09-19T08:58:04.214 INFO:teuthology.orchestra.run.smithi173.stdout: "dn+": 24, 2024-09-19T08:58:04.214 INFO:teuthology.orchestra.run.smithi173.stdout: "dn-": 8, 2024-09-19T08:58:04.214 INFO:teuthology.orchestra.run.smithi173.stdout: "cap": 0, 2024-09-19T08:58:04.214 INFO:teuthology.orchestra.run.smithi173.stdout: "cap+": 0, 2024-09-19T08:58:04.214 INFO:teuthology.orchestra.run.smithi173.stdout: "cap-": 0, 2024-09-19T08:58:04.214 INFO:teuthology.orchestra.run.smithi173.stdout: "rss": 51888, 2024-09-19T08:58:04.214 INFO:teuthology.orchestra.run.smithi173.stdout: "heap": 223520 2024-09-19T08:58:04.214 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.214 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_server": { 2024-09-19T08:58:04.214 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_client_request": 0, 2024-09-19T08:58:04.214 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_server_request": 1, 2024-09-19T08:58:04.215 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_request": 0, 2024-09-19T08:58:04.215 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_session": 19, 2024-09-19T08:58:04.215 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_peer_request": 2, 2024-09-19T08:58:04.215 INFO:teuthology.orchestra.run.smithi173.stdout: "req_create_latency": { 2024-09-19T08:58:04.215 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.215 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.215 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.215 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getattr_latency": { 2024-09-19T08:58:04.215 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.215 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.215 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.215 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.215 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getfilelock_latency": { 2024-09-19T08:58:04.216 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.216 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.216 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.216 INFO:teuthology.orchestra.run.smithi173.stdout: "req_link_latency": { 2024-09-19T08:58:04.216 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.216 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.216 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.216 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookup_latency": { 2024-09-19T08:58:04.216 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.216 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.216 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.216 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.217 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookuphash_latency": { 2024-09-19T08:58:04.217 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.217 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.217 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.217 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.217 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupino_latency": { 2024-09-19T08:58:04.217 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.217 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.217 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.217 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.217 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupname_latency": { 2024-09-19T08:58:04.217 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.217 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.217 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.218 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupparent_latency": { 2024-09-19T08:58:04.218 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.218 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.218 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.218 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupsnap_latency": { 2024-09-19T08:58:04.218 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.218 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.218 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.218 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.218 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lssnap_latency": { 2024-09-19T08:58:04.218 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.219 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.219 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.219 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.219 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mkdir_latency": { 2024-09-19T08:58:04.219 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.220 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.220 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.220 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.220 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mknod_latency": { 2024-09-19T08:58:04.220 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.220 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.220 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.220 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.220 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mksnap_latency": { 2024-09-19T08:58:04.220 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.220 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.220 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.221 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.221 INFO:teuthology.orchestra.run.smithi173.stdout: "req_open_latency": { 2024-09-19T08:58:04.221 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.221 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.221 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.221 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.221 INFO:teuthology.orchestra.run.smithi173.stdout: "req_readdir_latency": { 2024-09-19T08:58:04.221 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.221 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.221 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.221 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.221 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rename_latency": { 2024-09-19T08:58:04.221 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.222 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.222 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.222 INFO:teuthology.orchestra.run.smithi173.stdout: "req_renamesnap_latency": { 2024-09-19T08:58:04.222 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.222 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.222 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.222 INFO:teuthology.orchestra.run.smithi173.stdout: "req_snapdiff_latency": { 2024-09-19T08:58:04.222 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.222 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.222 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.222 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.223 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmdir_latency": { 2024-09-19T08:58:04.223 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.223 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.223 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.223 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmsnap_latency": { 2024-09-19T08:58:04.223 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.223 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.223 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.223 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.223 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmxattr_latency": { 2024-09-19T08:58:04.223 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.223 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.224 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.224 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setattr_latency": { 2024-09-19T08:58:04.224 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.224 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.224 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.224 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setdirlayout_latency": { 2024-09-19T08:58:04.224 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.224 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.224 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.224 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.225 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setfilelock_latency": { 2024-09-19T08:58:04.225 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.225 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.225 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.225 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.225 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setlayout_latency": { 2024-09-19T08:58:04.225 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.225 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.225 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.225 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.225 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setxattr_latency": { 2024-09-19T08:58:04.225 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.226 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.226 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.226 INFO:teuthology.orchestra.run.smithi173.stdout: "req_symlink_latency": { 2024-09-19T08:58:04.226 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.226 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.226 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.226 INFO:teuthology.orchestra.run.smithi173.stdout: "req_unlink_latency": { 2024-09-19T08:58:04.226 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.226 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.226 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.226 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.226 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_revoke_eviction": 0, 2024-09-19T08:58:04.227 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_acquisition_throttle": 0, 2024-09-19T08:58:04.227 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getvxattr_latency": { 2024-09-19T08:58:04.227 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.227 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.227 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.227 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:04.227 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.227 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_sessions": { 2024-09-19T08:58:04.227 INFO:teuthology.orchestra.run.smithi173.stdout: "session_count": 2, 2024-09-19T08:58:04.227 INFO:teuthology.orchestra.run.smithi173.stdout: "session_add": 4, 2024-09-19T08:58:04.227 INFO:teuthology.orchestra.run.smithi173.stdout: "session_remove": 2, 2024-09-19T08:58:04.227 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_open": 2, 2024-09-19T08:58:04.227 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_stale": 0, 2024-09-19T08:58:04.228 INFO:teuthology.orchestra.run.smithi173.stdout: "total_load": 0, 2024-09-19T08:58:04.228 INFO:teuthology.orchestra.run.smithi173.stdout: "average_load": 0, 2024-09-19T08:58:04.228 INFO:teuthology.orchestra.run.smithi173.stdout: "avg_session_uptime": 387, 2024-09-19T08:58:04.228 INFO:teuthology.orchestra.run.smithi173.stdout: "mdthresh_evicted": 0 2024-09-19T08:58:04.228 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.228 INFO:teuthology.orchestra.run.smithi173.stdout: "mempool": { 2024-09-19T08:58:04.228 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_bytes": 0, 2024-09-19T08:58:04.228 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_items": 0, 2024-09-19T08:58:04.228 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T08:58:04.228 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_items": 0, 2024-09-19T08:58:04.228 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T08:58:04.228 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_items": 0, 2024-09-19T08:58:04.228 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T08:58:04.229 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T08:58:04.229 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T08:58:04.229 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T08:58:04.229 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T08:58:04.229 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_items": 0, 2024-09-19T08:58:04.229 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T08:58:04.229 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T08:58:04.229 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_bytes": 0, 2024-09-19T08:58:04.229 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_items": 0, 2024-09-19T08:58:04.229 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_bytes": 0, 2024-09-19T08:58:04.229 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_items": 0, 2024-09-19T08:58:04.229 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T08:58:04.229 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T08:58:04.230 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T08:58:04.230 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T08:58:04.230 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T08:58:04.230 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_items": 0, 2024-09-19T08:58:04.230 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_bytes": 0, 2024-09-19T08:58:04.230 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_items": 0, 2024-09-19T08:58:04.230 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T08:58:04.230 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T08:58:04.230 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_bytes": 0, 2024-09-19T08:58:04.231 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_items": 0, 2024-09-19T08:58:04.231 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_bytes": 0, 2024-09-19T08:58:04.231 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_items": 0, 2024-09-19T08:58:04.231 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T08:58:04.231 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_items": 0, 2024-09-19T08:58:04.231 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T08:58:04.231 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_items": 0, 2024-09-19T08:58:04.231 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_bytes": 270455, 2024-09-19T08:58:04.231 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_items": 91, 2024-09-19T08:58:04.231 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_bytes": 0, 2024-09-19T08:58:04.231 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_items": 0, 2024-09-19T08:58:04.232 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_bytes": 0, 2024-09-19T08:58:04.232 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_items": 0, 2024-09-19T08:58:04.232 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_bytes": 0, 2024-09-19T08:58:04.232 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_items": 0, 2024-09-19T08:58:04.232 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_bytes": 0, 2024-09-19T08:58:04.232 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_items": 0, 2024-09-19T08:58:04.232 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_bytes": 7936, 2024-09-19T08:58:04.232 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_items": 175, 2024-09-19T08:58:04.232 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T08:58:04.232 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_items": 0, 2024-09-19T08:58:04.232 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_bytes": 0, 2024-09-19T08:58:04.232 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_items": 0, 2024-09-19T08:58:04.232 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_bytes": 81229, 2024-09-19T08:58:04.233 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_items": 326, 2024-09-19T08:58:04.233 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_bytes": 0, 2024-09-19T08:58:04.233 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_items": 0, 2024-09-19T08:58:04.233 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_bytes": 0, 2024-09-19T08:58:04.233 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_items": 0 2024-09-19T08:58:04.233 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.233 INFO:teuthology.orchestra.run.smithi173.stdout: "objecter": { 2024-09-19T08:58:04.233 INFO:teuthology.orchestra.run.smithi173.stdout: "op_active": 0, 2024-09-19T08:58:04.233 INFO:teuthology.orchestra.run.smithi173.stdout: "op_laggy": 0, 2024-09-19T08:58:04.233 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send": 45, 2024-09-19T08:58:04.233 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send_bytes": 12839, 2024-09-19T08:58:04.233 INFO:teuthology.orchestra.run.smithi173.stdout: "op_resend": 30, 2024-09-19T08:58:04.233 INFO:teuthology.orchestra.run.smithi173.stdout: "op_reply": 45, 2024-09-19T08:58:04.233 INFO:teuthology.orchestra.run.smithi173.stdout: "op_latency": { 2024-09-19T08:58:04.234 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 45, 2024-09-19T08:58:04.234 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.311992617, 2024-09-19T08:58:04.234 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.006933169 2024-09-19T08:58:04.234 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.234 INFO:teuthology.orchestra.run.smithi173.stdout: "op_inflight": 0, 2024-09-19T08:58:04.234 INFO:teuthology.orchestra.run.smithi173.stdout: "oplen_avg": { 2024-09-19T08:58:04.234 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 45, 2024-09-19T08:58:04.234 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 67 2024-09-19T08:58:04.234 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.234 INFO:teuthology.orchestra.run.smithi173.stdout: "op": 45, 2024-09-19T08:58:04.234 INFO:teuthology.orchestra.run.smithi173.stdout: "op_r": 0, 2024-09-19T08:58:04.234 INFO:teuthology.orchestra.run.smithi173.stdout: "op_w": 45, 2024-09-19T08:58:04.234 INFO:teuthology.orchestra.run.smithi173.stdout: "op_rmw": 0, 2024-09-19T08:58:04.235 INFO:teuthology.orchestra.run.smithi173.stdout: "op_pg": 0, 2024-09-19T08:58:04.235 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_stat": 0, 2024-09-19T08:58:04.235 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_create": 10, 2024-09-19T08:58:04.235 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_read": 0, 2024-09-19T08:58:04.235 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_write": 7, 2024-09-19T08:58:04.235 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writefull": 9, 2024-09-19T08:58:04.235 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writesame": 0, 2024-09-19T08:58:04.235 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_append": 0, 2024-09-19T08:58:04.235 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_zero": 0, 2024-09-19T08:58:04.235 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_truncate": 0, 2024-09-19T08:58:04.235 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_delete": 6, 2024-09-19T08:58:04.235 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_mapext": 0, 2024-09-19T08:58:04.235 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_sparse_read": 0, 2024-09-19T08:58:04.235 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_clonerange": 0, 2024-09-19T08:58:04.236 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_getxattr": 0, 2024-09-19T08:58:04.236 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_setxattr": 20, 2024-09-19T08:58:04.236 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_cmpxattr": 0, 2024-09-19T08:58:04.236 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_rmxattr": 0, 2024-09-19T08:58:04.236 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_resetxattrs": 0, 2024-09-19T08:58:04.236 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_call": 0, 2024-09-19T08:58:04.236 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_watch": 0, 2024-09-19T08:58:04.236 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_notify": 0, 2024-09-19T08:58:04.236 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T08:58:04.236 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls": 0, 2024-09-19T08:58:04.236 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls_filter": 0, 2024-09-19T08:58:04.236 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_other": 0, 2024-09-19T08:58:04.236 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_active": 0, 2024-09-19T08:58:04.236 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_send": 0, 2024-09-19T08:58:04.237 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_resend": 0, 2024-09-19T08:58:04.237 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_ping": 0, 2024-09-19T08:58:04.237 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_active": 0, 2024-09-19T08:58:04.237 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_send": 0, 2024-09-19T08:58:04.237 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_resend": 0, 2024-09-19T08:58:04.237 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_active": 0, 2024-09-19T08:58:04.237 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_send": 0, 2024-09-19T08:58:04.237 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_resend": 0, 2024-09-19T08:58:04.237 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_active": 0, 2024-09-19T08:58:04.237 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_send": 0, 2024-09-19T08:58:04.237 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_resend": 0, 2024-09-19T08:58:04.237 INFO:teuthology.orchestra.run.smithi173.stdout: "command_active": 0, 2024-09-19T08:58:04.237 INFO:teuthology.orchestra.run.smithi173.stdout: "command_send": 0, 2024-09-19T08:58:04.237 INFO:teuthology.orchestra.run.smithi173.stdout: "command_resend": 0, 2024-09-19T08:58:04.238 INFO:teuthology.orchestra.run.smithi173.stdout: "map_epoch": 95, 2024-09-19T08:58:04.238 INFO:teuthology.orchestra.run.smithi173.stdout: "map_full": 0, 2024-09-19T08:58:04.238 INFO:teuthology.orchestra.run.smithi173.stdout: "map_inc": 13, 2024-09-19T08:58:04.238 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_sessions": 9, 2024-09-19T08:58:04.238 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_open": 9, 2024-09-19T08:58:04.238 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_close": 0, 2024-09-19T08:58:04.238 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_laggy": 0, 2024-09-19T08:58:04.238 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_wr": 14, 2024-09-19T08:58:04.238 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_rd": 0, 2024-09-19T08:58:04.238 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_del": 1 2024-09-19T08:58:04.238 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.238 INFO:teuthology.orchestra.run.smithi173.stdout: "oft": { 2024-09-19T08:58:04.238 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_objs": 1, 2024-09-19T08:58:04.239 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_kv_pairs": 0, 2024-09-19T08:58:04.239 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_updates": 0, 2024-09-19T08:58:04.239 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_removes": 0 2024-09-19T08:58:04.239 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.239 INFO:teuthology.orchestra.run.smithi173.stdout: "osd-slow-ops": { 2024-09-19T08:58:04.239 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_ops_count": 86 2024-09-19T08:58:04.239 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.239 INFO:teuthology.orchestra.run.smithi173.stdout: "purge_queue": { 2024-09-19T08:58:04.239 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops": 0, 2024-09-19T08:58:04.239 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T08:58:04.239 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing": 0, 2024-09-19T08:58:04.239 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_high_water": 0, 2024-09-19T08:58:04.239 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed_ops": 0, 2024-09-19T08:58:04.240 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed": 0, 2024-09-19T08:58:04.240 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_item_in_journal": 0 2024-09-19T08:58:04.240 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.240 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T08:58:04.240 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-09-19T08:58:04.240 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-09-19T08:58:04.240 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-09-19T08:58:04.240 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 1798, 2024-09-19T08:58:04.240 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 2083164, 2024-09-19T08:58:04.240 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-09-19T08:58:04.240 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 1798, 2024-09-19T08:58:04.240 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-09-19T08:58:04.240 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-09-19T08:58:04.241 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 1798, 2024-09-19T08:58:04.241 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 2083164, 2024-09-19T08:58:04.241 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-09-19T08:58:04.241 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.241 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.241 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.241 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:04.241 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.241 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_bytes": { 2024-09-19T08:58:04.241 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-09-19T08:58:04.241 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-09-19T08:58:04.241 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-09-19T08:58:04.241 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-09-19T08:58:04.241 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-09-19T08:58:04.242 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-09-19T08:58:04.242 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-09-19T08:58:04.242 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 45, 2024-09-19T08:58:04.242 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 12839, 2024-09-19T08:58:04.242 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 45, 2024-09-19T08:58:04.242 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 12839, 2024-09-19T08:58:04.242 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-09-19T08:58:04.242 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.242 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.242 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.242 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:04.242 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.242 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_ops": { 2024-09-19T08:58:04.243 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-09-19T08:58:04.243 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 1024, 2024-09-19T08:58:04.243 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-09-19T08:58:04.243 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-09-19T08:58:04.243 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-09-19T08:58:04.243 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-09-19T08:58:04.243 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-09-19T08:58:04.243 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 45, 2024-09-19T08:58:04.243 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 45, 2024-09-19T08:58:04.243 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 45, 2024-09-19T08:58:04.243 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 45, 2024-09-19T08:58:04.243 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-09-19T08:58:04.243 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.244 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.244 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.244 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:04.244 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.244 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle": { 2024-09-19T08:58:04.244 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-09-19T08:58:04.244 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2024-09-19T08:58:04.244 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-09-19T08:58:04.244 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-09-19T08:58:04.244 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-09-19T08:58:04.244 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-09-19T08:58:04.244 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-09-19T08:58:04.244 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-09-19T08:58:04.245 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-09-19T08:58:04.245 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 0, 2024-09-19T08:58:04.245 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 0, 2024-09-19T08:58:04.245 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-09-19T08:58:04.245 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.245 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.245 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.245 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:04.245 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:04.245 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle-0x555919f20960": { 2024-09-19T08:58:04.245 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-09-19T08:58:04.245 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2024-09-19T08:58:04.245 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-09-19T08:58:04.245 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 8, 2024-09-19T08:58:04.246 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 2357, 2024-09-19T08:58:04.246 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-09-19T08:58:04.246 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 8, 2024-09-19T08:58:04.246 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-09-19T08:58:04.246 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-09-19T08:58:04.246 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 7, 2024-09-19T08:58:04.246 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 2357, 2024-09-19T08:58:04.246 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-09-19T08:58:04.246 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:04.246 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:04.246 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:04.246 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:04.246 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:04.247 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-19T08:58:04.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:04 smithi164 ceph-mon[30928]: pgmap v382: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:58:04.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:04 smithi170 ceph-mon[35072]: pgmap v382: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T08:58:05.055 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-19T08:58:05.055 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-19T08:58:05.055 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-09-19T08:58:05.055 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=17 2024-09-19T08:58:05.055 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-09-19T08:58:05.055 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-19T08:58:05.055 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T08:58:05.055 DEBUG:tasks.check_counter:Getting stats from f 2024-09-19T08:58:05.056 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-19T08:58:05.311 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:58:05.818 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-19T08:58:05.818 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T08:58:05.818 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 2446, 2024-09-19T08:58:05.818 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 2448, 2024-09-19T08:58:05.818 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 725556, 2024-09-19T08:58:05.818 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 777835, 2024-09-19T08:58:05.818 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 6, 2024-09-19T08:58:05.818 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 2, 2024-09-19T08:58:05.819 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.214531032, 2024-09-19T08:58:05.819 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.068281221, 2024-09-19T08:58:05.819 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.057811452, 2024-09-19T08:58:05.819 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.072512357, 2024-09-19T08:58:05.819 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:58:05.819 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 2446, 2024-09-19T08:58:05.819 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.095453799, 2024-09-19T08:58:05.819 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000039024 2024-09-19T08:58:05.819 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.819 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:58:05.819 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.819 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.819 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.820 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.820 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T08:58:05.820 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T08:58:05.820 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.820 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T08:58:05.820 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 280, 2024-09-19T08:58:05.820 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 258, 2024-09-19T08:58:05.820 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 1679420, 2024-09-19T08:58:05.820 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 149610, 2024-09-19T08:58:05.820 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 3, 2024-09-19T08:58:05.820 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 2, 2024-09-19T08:58:05.820 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.051868120, 2024-09-19T08:58:05.820 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.017323021, 2024-09-19T08:58:05.821 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.050394586, 2024-09-19T08:58:05.821 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.000086636, 2024-09-19T08:58:05.821 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:58:05.821 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 258, 2024-09-19T08:58:05.821 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.019492503, 2024-09-19T08:58:05.821 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000075552 2024-09-19T08:58:05.821 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.821 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:58:05.821 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.821 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.821 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.821 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.821 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 1678864, 2024-09-19T08:58:05.822 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 148912 2024-09-19T08:58:05.822 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.822 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T08:58:05.822 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 3, 2024-09-19T08:58:05.822 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 182, 2024-09-19T08:58:05.822 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 588, 2024-09-19T08:58:05.822 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 479454, 2024-09-19T08:58:05.822 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 6, 2024-09-19T08:58:05.822 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 4, 2024-09-19T08:58:05.822 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.010759895, 2024-09-19T08:58:05.822 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.007183850, 2024-09-19T08:58:05.822 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.001740042, 2024-09-19T08:58:05.822 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.000032909, 2024-09-19T08:58:05.823 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:58:05.823 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 179, 2024-09-19T08:58:05.823 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.017723052, 2024-09-19T08:58:05.823 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000099011 2024-09-19T08:58:05.823 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.823 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:58:05.823 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.823 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.823 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.823 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.823 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T08:58:05.823 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 479136 2024-09-19T08:58:05.823 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.824 INFO:teuthology.orchestra.run.smithi173.stdout: "cct": { 2024-09-19T08:58:05.824 INFO:teuthology.orchestra.run.smithi173.stdout: "total_workers": 1, 2024-09-19T08:58:05.824 INFO:teuthology.orchestra.run.smithi173.stdout: "unhealthy_workers": 0 2024-09-19T08:58:05.824 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.824 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-MDSRank": { 2024-09-19T08:58:05.824 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2024-09-19T08:58:05.824 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2024-09-19T08:58:05.824 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 4119, 2024-09-19T08:58:05.824 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.113459425, 2024-09-19T08:58:05.824 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000027545 2024-09-19T08:58:05.824 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:05.824 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.825 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-PurgeQueue": { 2024-09-19T08:58:05.825 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2024-09-19T08:58:05.825 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2024-09-19T08:58:05.825 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.825 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.825 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.825 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:05.825 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.825 INFO:teuthology.orchestra.run.smithi173.stdout: "mds": { 2024-09-19T08:58:05.825 INFO:teuthology.orchestra.run.smithi173.stdout: "request": 0, 2024-09-19T08:58:05.825 INFO:teuthology.orchestra.run.smithi173.stdout: "reply": 0, 2024-09-19T08:58:05.825 INFO:teuthology.orchestra.run.smithi173.stdout: "reply_latency": { 2024-09-19T08:58:05.825 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.825 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.826 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.826 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.826 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_reply": 0, 2024-09-19T08:58:05.826 INFO:teuthology.orchestra.run.smithi173.stdout: "forward": 0, 2024-09-19T08:58:05.826 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_complete": 0, 2024-09-19T08:58:05.826 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_keys": 0, 2024-09-19T08:58:05.826 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_commit": 0, 2024-09-19T08:58:05.826 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_split": 0, 2024-09-19T08:58:05.826 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_merge": 0, 2024-09-19T08:58:05.826 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes": 9, 2024-09-19T08:58:05.826 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_top": 2, 2024-09-19T08:58:05.826 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_bottom": 7, 2024-09-19T08:58:05.826 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pin_tail": 0, 2024-09-19T08:58:05.827 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pinned": 6, 2024-09-19T08:58:05.827 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_expired": 0, 2024-09-19T08:58:05.827 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_with_caps": 0, 2024-09-19T08:58:05.827 INFO:teuthology.orchestra.run.smithi173.stdout: "caps": 0, 2024-09-19T08:58:05.827 INFO:teuthology.orchestra.run.smithi173.stdout: "subtrees": 3, 2024-09-19T08:58:05.827 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse": 0, 2024-09-19T08:58:05.827 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_hit": 0, 2024-09-19T08:58:05.827 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_forward": 0, 2024-09-19T08:58:05.827 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_discover": 0, 2024-09-19T08:58:05.827 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_dir_fetch": 0, 2024-09-19T08:58:05.827 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_remote_ino": 0, 2024-09-19T08:58:05.827 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_lock": 0, 2024-09-19T08:58:05.827 INFO:teuthology.orchestra.run.smithi173.stdout: "load_cent": 0, 2024-09-19T08:58:05.828 INFO:teuthology.orchestra.run.smithi173.stdout: "q": 0, 2024-09-19T08:58:05.828 INFO:teuthology.orchestra.run.smithi173.stdout: "exported": 0, 2024-09-19T08:58:05.828 INFO:teuthology.orchestra.run.smithi173.stdout: "exported_inodes": 0, 2024-09-19T08:58:05.828 INFO:teuthology.orchestra.run.smithi173.stdout: "imported": 0, 2024-09-19T08:58:05.828 INFO:teuthology.orchestra.run.smithi173.stdout: "imported_inodes": 0, 2024-09-19T08:58:05.828 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_dir_fetch": 0, 2024-09-19T08:58:05.828 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_backtrace_fetch": 0, 2024-09-19T08:58:05.828 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_peer_discover": 0, 2024-09-19T08:58:05.828 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rfiles": 0, 2024-09-19T08:58:05.828 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rbytes": 0, 2024-09-19T08:58:05.828 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rsnaps": 0, 2024-09-19T08:58:05.828 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T08:58:05.828 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_set_tag": 0, 2024-09-19T08:58:05.829 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T08:58:05.829 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_inotable_repaired": 0, 2024-09-19T08:58:05.829 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_inodes": 0, 2024-09-19T08:58:05.829 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T08:58:05.829 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T08:58:05.829 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_file_inodes": 0, 2024-09-19T08:58:05.829 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_inode_file_caps": 0, 2024-09-19T08:58:05.829 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T08:58:05.829 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_grant": 0, 2024-09-19T08:58:05.829 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T08:58:05.829 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T08:58:05.829 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T08:58:05.829 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps": 0, 2024-09-19T08:58:05.829 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps_dirty": 0, 2024-09-19T08:58:05.830 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_cap_release": 0, 2024-09-19T08:58:05.830 INFO:teuthology.orchestra.run.smithi173.stdout: "process_request_cap_release": 0 2024-09-19T08:58:05.830 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.830 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_cache": { 2024-09-19T08:58:05.830 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update": 0, 2024-09-19T08:58:05.830 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update_receipt": 0, 2024-09-19T08:58:05.830 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_try_discover": 0, 2024-09-19T08:58:05.830 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_send_discover": 0, 2024-09-19T08:58:05.830 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_handle_discover": 0, 2024-09-19T08:58:05.830 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays": 0, 2024-09-19T08:58:05.830 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_delayed": 0, 2024-09-19T08:58:05.830 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_enqueuing": 0, 2024-09-19T08:58:05.830 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_created": 0, 2024-09-19T08:58:05.831 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_enqueued": 0, 2024-09-19T08:58:05.831 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_reintegrated": 0, 2024-09-19T08:58:05.831 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_migrated": 0, 2024-09-19T08:58:05.831 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_processing": 0, 2024-09-19T08:58:05.831 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_enqueued": 0, 2024-09-19T08:58:05.831 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_prioritized": 0, 2024-09-19T08:58:05.831 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_started": 0, 2024-09-19T08:58:05.831 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_completed": 0, 2024-09-19T08:58:05.831 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_quiesce_path": 0, 2024-09-19T08:58:05.831 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_quiesce_inode": 0, 2024-09-19T08:58:05.831 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T08:58:05.831 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_exportdir": 0, 2024-09-19T08:58:05.831 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_flush": 0, 2024-09-19T08:58:05.832 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragmentdir": 0, 2024-09-19T08:58:05.832 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragstats": 0, 2024-09-19T08:58:05.832 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_inodestats": 0 2024-09-19T08:58:05.832 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.832 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_log": { 2024-09-19T08:58:05.832 INFO:teuthology.orchestra.run.smithi173.stdout: "evlrg": 0, 2024-09-19T08:58:05.832 INFO:teuthology.orchestra.run.smithi173.stdout: "evadd": 0, 2024-09-19T08:58:05.832 INFO:teuthology.orchestra.run.smithi173.stdout: "evex": 0, 2024-09-19T08:58:05.832 INFO:teuthology.orchestra.run.smithi173.stdout: "evtrm": 0, 2024-09-19T08:58:05.832 INFO:teuthology.orchestra.run.smithi173.stdout: "ev": 21, 2024-09-19T08:58:05.832 INFO:teuthology.orchestra.run.smithi173.stdout: "evexg": 0, 2024-09-19T08:58:05.832 INFO:teuthology.orchestra.run.smithi173.stdout: "evexd": 0, 2024-09-19T08:58:05.832 INFO:teuthology.orchestra.run.smithi173.stdout: "segadd": 0, 2024-09-19T08:58:05.833 INFO:teuthology.orchestra.run.smithi173.stdout: "segex": 0, 2024-09-19T08:58:05.833 INFO:teuthology.orchestra.run.smithi173.stdout: "segtrm": 0, 2024-09-19T08:58:05.833 INFO:teuthology.orchestra.run.smithi173.stdout: "seg": 2, 2024-09-19T08:58:05.833 INFO:teuthology.orchestra.run.smithi173.stdout: "segmjr": 2, 2024-09-19T08:58:05.833 INFO:teuthology.orchestra.run.smithi173.stdout: "segexg": 0, 2024-09-19T08:58:05.833 INFO:teuthology.orchestra.run.smithi173.stdout: "segexd": 0, 2024-09-19T08:58:05.833 INFO:teuthology.orchestra.run.smithi173.stdout: "expos": 4194304, 2024-09-19T08:58:05.833 INFO:teuthology.orchestra.run.smithi173.stdout: "wrpos": 4228481, 2024-09-19T08:58:05.833 INFO:teuthology.orchestra.run.smithi173.stdout: "rdpos": 4225876, 2024-09-19T08:58:05.833 INFO:teuthology.orchestra.run.smithi173.stdout: "jlat": { 2024-09-19T08:58:05.833 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.833 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.833 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.834 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.834 INFO:teuthology.orchestra.run.smithi173.stdout: "replayed": 21 2024-09-19T08:58:05.834 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.834 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_mem": { 2024-09-19T08:58:05.834 INFO:teuthology.orchestra.run.smithi173.stdout: "ino": 9, 2024-09-19T08:58:05.834 INFO:teuthology.orchestra.run.smithi173.stdout: "ino+": 21, 2024-09-19T08:58:05.834 INFO:teuthology.orchestra.run.smithi173.stdout: "ino-": 12, 2024-09-19T08:58:05.834 INFO:teuthology.orchestra.run.smithi173.stdout: "dir": 7, 2024-09-19T08:58:05.834 INFO:teuthology.orchestra.run.smithi173.stdout: "dir+": 15, 2024-09-19T08:58:05.834 INFO:teuthology.orchestra.run.smithi173.stdout: "dir-": 8, 2024-09-19T08:58:05.834 INFO:teuthology.orchestra.run.smithi173.stdout: "dn": 9, 2024-09-19T08:58:05.834 INFO:teuthology.orchestra.run.smithi173.stdout: "dn+": 21, 2024-09-19T08:58:05.834 INFO:teuthology.orchestra.run.smithi173.stdout: "dn-": 12, 2024-09-19T08:58:05.834 INFO:teuthology.orchestra.run.smithi173.stdout: "cap": 0, 2024-09-19T08:58:05.835 INFO:teuthology.orchestra.run.smithi173.stdout: "cap+": 0, 2024-09-19T08:58:05.835 INFO:teuthology.orchestra.run.smithi173.stdout: "cap-": 0, 2024-09-19T08:58:05.835 INFO:teuthology.orchestra.run.smithi173.stdout: "rss": 49548, 2024-09-19T08:58:05.835 INFO:teuthology.orchestra.run.smithi173.stdout: "heap": 215328 2024-09-19T08:58:05.835 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.835 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_server": { 2024-09-19T08:58:05.835 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_client_request": 0, 2024-09-19T08:58:05.835 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_server_request": 0, 2024-09-19T08:58:05.835 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_request": 0, 2024-09-19T08:58:05.835 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_session": 0, 2024-09-19T08:58:05.835 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_peer_request": 0, 2024-09-19T08:58:05.835 INFO:teuthology.orchestra.run.smithi173.stdout: "req_create_latency": { 2024-09-19T08:58:05.835 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.836 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.836 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.836 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.836 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getattr_latency": { 2024-09-19T08:58:05.836 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.836 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.836 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.836 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.836 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getfilelock_latency": { 2024-09-19T08:58:05.836 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.836 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.836 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.836 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.836 INFO:teuthology.orchestra.run.smithi173.stdout: "req_link_latency": { 2024-09-19T08:58:05.837 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.837 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.837 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.837 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.837 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookup_latency": { 2024-09-19T08:58:05.837 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.837 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.837 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.837 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.837 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookuphash_latency": { 2024-09-19T08:58:05.837 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.837 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.837 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.838 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.838 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupino_latency": { 2024-09-19T08:58:05.838 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.838 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.838 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.838 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.838 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupname_latency": { 2024-09-19T08:58:05.838 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.838 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.838 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.838 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.838 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupparent_latency": { 2024-09-19T08:58:05.838 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.839 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.839 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.839 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.839 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupsnap_latency": { 2024-09-19T08:58:05.839 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.839 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.839 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.839 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.839 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lssnap_latency": { 2024-09-19T08:58:05.839 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.840 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.840 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.840 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.840 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mkdir_latency": { 2024-09-19T08:58:05.840 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.840 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.841 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.841 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.841 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mknod_latency": { 2024-09-19T08:58:05.841 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.841 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.841 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.841 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.841 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mksnap_latency": { 2024-09-19T08:58:05.841 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.841 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.841 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.841 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.842 INFO:teuthology.orchestra.run.smithi173.stdout: "req_open_latency": { 2024-09-19T08:58:05.842 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.842 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.842 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.842 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.842 INFO:teuthology.orchestra.run.smithi173.stdout: "req_readdir_latency": { 2024-09-19T08:58:05.842 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.842 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.842 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.842 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.842 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rename_latency": { 2024-09-19T08:58:05.842 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.843 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.843 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.843 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.843 INFO:teuthology.orchestra.run.smithi173.stdout: "req_renamesnap_latency": { 2024-09-19T08:58:05.843 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.843 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.843 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.843 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.843 INFO:teuthology.orchestra.run.smithi173.stdout: "req_snapdiff_latency": { 2024-09-19T08:58:05.843 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.843 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.843 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.843 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.843 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmdir_latency": { 2024-09-19T08:58:05.844 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.844 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.844 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.844 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.844 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmsnap_latency": { 2024-09-19T08:58:05.844 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.844 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.844 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.844 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.844 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmxattr_latency": { 2024-09-19T08:58:05.844 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.844 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.845 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.845 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.845 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setattr_latency": { 2024-09-19T08:58:05.845 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.845 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.845 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.845 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.845 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setdirlayout_latency": { 2024-09-19T08:58:05.845 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.845 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.845 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.845 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.845 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setfilelock_latency": { 2024-09-19T08:58:05.846 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.846 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.846 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.846 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.846 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setlayout_latency": { 2024-09-19T08:58:05.846 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.846 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.846 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.846 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.846 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setxattr_latency": { 2024-09-19T08:58:05.846 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.846 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.846 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.847 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.847 INFO:teuthology.orchestra.run.smithi173.stdout: "req_symlink_latency": { 2024-09-19T08:58:05.847 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.847 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.847 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.847 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.847 INFO:teuthology.orchestra.run.smithi173.stdout: "req_unlink_latency": { 2024-09-19T08:58:05.847 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.847 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.847 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.847 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.847 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_revoke_eviction": 0, 2024-09-19T08:58:05.847 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_acquisition_throttle": 0, 2024-09-19T08:58:05.848 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getvxattr_latency": { 2024-09-19T08:58:05.848 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.848 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.848 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.848 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:05.848 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.848 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_sessions": { 2024-09-19T08:58:05.848 INFO:teuthology.orchestra.run.smithi173.stdout: "session_count": 2, 2024-09-19T08:58:05.848 INFO:teuthology.orchestra.run.smithi173.stdout: "session_add": 4, 2024-09-19T08:58:05.848 INFO:teuthology.orchestra.run.smithi173.stdout: "session_remove": 2, 2024-09-19T08:58:05.848 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_open": 2, 2024-09-19T08:58:05.848 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_stale": 0, 2024-09-19T08:58:05.848 INFO:teuthology.orchestra.run.smithi173.stdout: "total_load": 0, 2024-09-19T08:58:05.849 INFO:teuthology.orchestra.run.smithi173.stdout: "average_load": 0, 2024-09-19T08:58:05.849 INFO:teuthology.orchestra.run.smithi173.stdout: "avg_session_uptime": 3183, 2024-09-19T08:58:05.849 INFO:teuthology.orchestra.run.smithi173.stdout: "mdthresh_evicted": 0 2024-09-19T08:58:05.849 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.849 INFO:teuthology.orchestra.run.smithi173.stdout: "mempool": { 2024-09-19T08:58:05.849 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_bytes": 0, 2024-09-19T08:58:05.849 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_items": 0, 2024-09-19T08:58:05.849 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T08:58:05.849 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_items": 0, 2024-09-19T08:58:05.849 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T08:58:05.849 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_items": 0, 2024-09-19T08:58:05.849 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T08:58:05.849 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T08:58:05.850 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T08:58:05.850 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T08:58:05.850 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T08:58:05.850 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_items": 0, 2024-09-19T08:58:05.850 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T08:58:05.850 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T08:58:05.850 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_bytes": 0, 2024-09-19T08:58:05.850 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_items": 0, 2024-09-19T08:58:05.850 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_bytes": 0, 2024-09-19T08:58:05.850 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_items": 0, 2024-09-19T08:58:05.850 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T08:58:05.850 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T08:58:05.850 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T08:58:05.851 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T08:58:05.851 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T08:58:05.851 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_items": 0, 2024-09-19T08:58:05.851 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_bytes": 0, 2024-09-19T08:58:05.851 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_items": 0, 2024-09-19T08:58:05.851 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T08:58:05.851 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T08:58:05.851 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_bytes": 0, 2024-09-19T08:58:05.851 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_items": 0, 2024-09-19T08:58:05.851 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_bytes": 0, 2024-09-19T08:58:05.851 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_items": 0, 2024-09-19T08:58:05.851 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T08:58:05.851 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_items": 0, 2024-09-19T08:58:05.851 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T08:58:05.852 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_items": 0, 2024-09-19T08:58:05.852 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_bytes": 181313, 2024-09-19T08:58:05.852 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_items": 47, 2024-09-19T08:58:05.852 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_bytes": 0, 2024-09-19T08:58:05.852 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_items": 0, 2024-09-19T08:58:05.852 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_bytes": 0, 2024-09-19T08:58:05.852 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_items": 0, 2024-09-19T08:58:05.852 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_bytes": 0, 2024-09-19T08:58:05.852 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_items": 0, 2024-09-19T08:58:05.852 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_bytes": 0, 2024-09-19T08:58:05.852 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_items": 0, 2024-09-19T08:58:05.852 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_bytes": 7936, 2024-09-19T08:58:05.852 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_items": 175, 2024-09-19T08:58:05.853 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T08:58:05.853 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_items": 0, 2024-09-19T08:58:05.853 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_bytes": 0, 2024-09-19T08:58:05.853 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_items": 0, 2024-09-19T08:58:05.853 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_bytes": 37381, 2024-09-19T08:58:05.853 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_items": 130, 2024-09-19T08:58:05.853 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_bytes": 0, 2024-09-19T08:58:05.853 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_items": 0, 2024-09-19T08:58:05.853 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_bytes": 0, 2024-09-19T08:58:05.853 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_items": 0 2024-09-19T08:58:05.853 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.853 INFO:teuthology.orchestra.run.smithi173.stdout: "objecter": { 2024-09-19T08:58:05.853 INFO:teuthology.orchestra.run.smithi173.stdout: "op_active": 0, 2024-09-19T08:58:05.854 INFO:teuthology.orchestra.run.smithi173.stdout: "op_laggy": 0, 2024-09-19T08:58:05.854 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send": 2449, 2024-09-19T08:58:05.854 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send_bytes": 21, 2024-09-19T08:58:05.854 INFO:teuthology.orchestra.run.smithi173.stdout: "op_resend": 3, 2024-09-19T08:58:05.854 INFO:teuthology.orchestra.run.smithi173.stdout: "op_reply": 2449, 2024-09-19T08:58:05.854 INFO:teuthology.orchestra.run.smithi173.stdout: "op_latency": { 2024-09-19T08:58:05.854 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 2449, 2024-09-19T08:58:05.854 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 1.601962286, 2024-09-19T08:58:05.854 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000654129 2024-09-19T08:58:05.854 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.854 INFO:teuthology.orchestra.run.smithi173.stdout: "op_inflight": 0, 2024-09-19T08:58:05.854 INFO:teuthology.orchestra.run.smithi173.stdout: "oplen_avg": { 2024-09-19T08:58:05.854 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 2449, 2024-09-19T08:58:05.855 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 2450 2024-09-19T08:58:05.855 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.855 INFO:teuthology.orchestra.run.smithi173.stdout: "op": 2449, 2024-09-19T08:58:05.855 INFO:teuthology.orchestra.run.smithi173.stdout: "op_r": 2449, 2024-09-19T08:58:05.855 INFO:teuthology.orchestra.run.smithi173.stdout: "op_w": 0, 2024-09-19T08:58:05.855 INFO:teuthology.orchestra.run.smithi173.stdout: "op_rmw": 0, 2024-09-19T08:58:05.855 INFO:teuthology.orchestra.run.smithi173.stdout: "op_pg": 0, 2024-09-19T08:58:05.855 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_stat": 1593, 2024-09-19T08:58:05.855 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_create": 0, 2024-09-19T08:58:05.855 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_read": 854, 2024-09-19T08:58:05.855 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_write": 0, 2024-09-19T08:58:05.855 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writefull": 0, 2024-09-19T08:58:05.855 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writesame": 0, 2024-09-19T08:58:05.856 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_append": 0, 2024-09-19T08:58:05.856 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_zero": 0, 2024-09-19T08:58:05.856 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_truncate": 0, 2024-09-19T08:58:05.856 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_delete": 0, 2024-09-19T08:58:05.856 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_mapext": 0, 2024-09-19T08:58:05.856 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_sparse_read": 0, 2024-09-19T08:58:05.856 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_clonerange": 0, 2024-09-19T08:58:05.856 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_getxattr": 1, 2024-09-19T08:58:05.856 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_setxattr": 0, 2024-09-19T08:58:05.856 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_cmpxattr": 0, 2024-09-19T08:58:05.856 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_rmxattr": 0, 2024-09-19T08:58:05.856 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_resetxattrs": 0, 2024-09-19T08:58:05.856 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_call": 0, 2024-09-19T08:58:05.857 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_watch": 0, 2024-09-19T08:58:05.857 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_notify": 0, 2024-09-19T08:58:05.857 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T08:58:05.857 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls": 0, 2024-09-19T08:58:05.857 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls_filter": 0, 2024-09-19T08:58:05.857 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_other": 0, 2024-09-19T08:58:05.857 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_active": 0, 2024-09-19T08:58:05.857 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_send": 0, 2024-09-19T08:58:05.857 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_resend": 0, 2024-09-19T08:58:05.857 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_ping": 0, 2024-09-19T08:58:05.857 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_active": 0, 2024-09-19T08:58:05.857 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_send": 0, 2024-09-19T08:58:05.857 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_resend": 0, 2024-09-19T08:58:05.857 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_active": 0, 2024-09-19T08:58:05.858 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_send": 0, 2024-09-19T08:58:05.858 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_resend": 0, 2024-09-19T08:58:05.858 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_active": 0, 2024-09-19T08:58:05.858 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_send": 0, 2024-09-19T08:58:05.858 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_resend": 0, 2024-09-19T08:58:05.858 INFO:teuthology.orchestra.run.smithi173.stdout: "command_active": 0, 2024-09-19T08:58:05.858 INFO:teuthology.orchestra.run.smithi173.stdout: "command_send": 0, 2024-09-19T08:58:05.858 INFO:teuthology.orchestra.run.smithi173.stdout: "command_resend": 0, 2024-09-19T08:58:05.858 INFO:teuthology.orchestra.run.smithi173.stdout: "map_epoch": 95, 2024-09-19T08:58:05.858 INFO:teuthology.orchestra.run.smithi173.stdout: "map_full": 0, 2024-09-19T08:58:05.858 INFO:teuthology.orchestra.run.smithi173.stdout: "map_inc": 13, 2024-09-19T08:58:05.858 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_sessions": 4, 2024-09-19T08:58:05.858 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_open": 4, 2024-09-19T08:58:05.859 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_close": 0, 2024-09-19T08:58:05.859 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_laggy": 0, 2024-09-19T08:58:05.859 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_wr": 0, 2024-09-19T08:58:05.859 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_rd": 2, 2024-09-19T08:58:05.859 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_del": 0 2024-09-19T08:58:05.859 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.859 INFO:teuthology.orchestra.run.smithi173.stdout: "oft": { 2024-09-19T08:58:05.859 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_objs": 0, 2024-09-19T08:58:05.859 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_kv_pairs": 0, 2024-09-19T08:58:05.859 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_updates": 0, 2024-09-19T08:58:05.859 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_removes": 0 2024-09-19T08:58:05.859 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.859 INFO:teuthology.orchestra.run.smithi173.stdout: "osd-slow-ops": { 2024-09-19T08:58:05.859 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_ops_count": 0 2024-09-19T08:58:05.860 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.860 INFO:teuthology.orchestra.run.smithi173.stdout: "purge_queue": { 2024-09-19T08:58:05.860 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops": 0, 2024-09-19T08:58:05.860 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T08:58:05.860 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing": 0, 2024-09-19T08:58:05.860 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_high_water": 0, 2024-09-19T08:58:05.860 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed_ops": 0, 2024-09-19T08:58:05.860 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed": 0, 2024-09-19T08:58:05.860 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_item_in_journal": 0 2024-09-19T08:58:05.860 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.860 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T08:58:05.860 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-09-19T08:58:05.860 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-09-19T08:58:05.861 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-09-19T08:58:05.861 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 2725, 2024-09-19T08:58:05.861 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 2143269, 2024-09-19T08:58:05.861 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-09-19T08:58:05.861 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 2725, 2024-09-19T08:58:05.861 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-09-19T08:58:05.861 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-09-19T08:58:05.861 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 2725, 2024-09-19T08:58:05.861 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 2143269, 2024-09-19T08:58:05.861 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-09-19T08:58:05.861 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.861 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.861 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.861 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:05.862 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.862 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_bytes": { 2024-09-19T08:58:05.862 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-09-19T08:58:05.862 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-09-19T08:58:05.862 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-09-19T08:58:05.862 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-09-19T08:58:05.862 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-09-19T08:58:05.862 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-09-19T08:58:05.862 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-09-19T08:58:05.862 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 2449, 2024-09-19T08:58:05.862 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 34182, 2024-09-19T08:58:05.862 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 2449, 2024-09-19T08:58:05.862 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 34182, 2024-09-19T08:58:05.863 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-09-19T08:58:05.863 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.863 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.863 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.863 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:05.863 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.863 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_ops": { 2024-09-19T08:58:05.863 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-09-19T08:58:05.863 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 1024, 2024-09-19T08:58:05.863 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-09-19T08:58:05.863 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-09-19T08:58:05.863 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-09-19T08:58:05.863 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-09-19T08:58:05.864 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-09-19T08:58:05.864 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 2449, 2024-09-19T08:58:05.864 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 2449, 2024-09-19T08:58:05.864 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 2449, 2024-09-19T08:58:05.864 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 2449, 2024-09-19T08:58:05.864 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-09-19T08:58:05.864 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.864 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.864 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.864 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:05.864 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.864 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle": { 2024-09-19T08:58:05.864 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-09-19T08:58:05.865 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2024-09-19T08:58:05.865 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-09-19T08:58:05.865 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-09-19T08:58:05.865 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-09-19T08:58:05.865 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-09-19T08:58:05.865 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-09-19T08:58:05.865 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-09-19T08:58:05.865 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-09-19T08:58:05.865 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 0, 2024-09-19T08:58:05.865 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 0, 2024-09-19T08:58:05.865 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-09-19T08:58:05.865 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.865 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.865 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.866 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:05.866 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:05.866 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle-0x5563c09aeb40": { 2024-09-19T08:58:05.866 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-09-19T08:58:05.866 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2024-09-19T08:58:05.866 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-09-19T08:58:05.866 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-09-19T08:58:05.866 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-09-19T08:58:05.866 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-09-19T08:58:05.866 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-09-19T08:58:05.866 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-09-19T08:58:05.866 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-09-19T08:58:05.866 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 0, 2024-09-19T08:58:05.867 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 0, 2024-09-19T08:58:05.867 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-09-19T08:58:05.867 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:05.867 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:05.867 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:05.867 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:05.867 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:05.867 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-19T08:58:06.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:06 smithi164 ceph-mon[30928]: pgmap v383: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-09-19T08:58:06.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:06 smithi170 ceph-mon[35072]: pgmap v383: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-09-19T08:58:06.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:06 smithi173 ceph-mon[35306]: pgmap v383: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-09-19T08:58:06.624 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-19T08:58:06.624 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-19T08:58:06.624 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-19T08:58:06.624 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-09-19T08:58:06.624 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-09-19T08:58:06.624 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-19T08:58:06.624 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T08:58:06.624 DEBUG:tasks.check_counter:Getting stats from i 2024-09-19T08:58:06.624 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-19T08:58:06.871 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:58:07.369 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-19T08:58:07.369 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T08:58:07.370 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 1293, 2024-09-19T08:58:07.370 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 1047, 2024-09-19T08:58:07.370 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 445817, 2024-09-19T08:58:07.370 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 270381, 2024-09-19T08:58:07.370 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 10, 2024-09-19T08:58:07.370 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 4, 2024-09-19T08:58:07.370 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.103436177, 2024-09-19T08:58:07.370 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.037888522, 2024-09-19T08:58:07.370 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.067976404, 2024-09-19T08:58:07.370 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.005443216, 2024-09-19T08:58:07.370 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:58:07.370 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 1045, 2024-09-19T08:58:07.371 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.062019574, 2024-09-19T08:58:07.371 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000059348 2024-09-19T08:58:07.371 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.371 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:58:07.371 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 2219, 2024-09-19T08:58:07.371 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000461489, 2024-09-19T08:58:07.371 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000207 2024-09-19T08:58:07.371 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.371 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T08:58:07.371 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T08:58:07.371 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.371 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T08:58:07.371 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 118, 2024-09-19T08:58:07.372 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 297, 2024-09-19T08:58:07.372 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 67283, 2024-09-19T08:58:07.372 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 987462, 2024-09-19T08:58:07.372 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 47, 2024-09-19T08:58:07.372 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 8, 2024-09-19T08:58:07.372 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.033398475, 2024-09-19T08:58:07.372 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.011150854, 2024-09-19T08:58:07.372 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.019554195, 2024-09-19T08:58:07.372 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.000352529, 2024-09-19T08:58:07.372 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:58:07.372 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 292, 2024-09-19T08:58:07.372 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.028569120, 2024-09-19T08:58:07.373 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000097839 2024-09-19T08:58:07.373 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.373 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:58:07.373 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 164, 2024-09-19T08:58:07.373 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000043530, 2024-09-19T08:58:07.373 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000265 2024-09-19T08:58:07.373 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.373 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T08:58:07.373 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 478784 2024-09-19T08:58:07.373 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.373 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T08:58:07.374 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 379, 2024-09-19T08:58:07.374 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 351, 2024-09-19T08:58:07.374 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 1752866, 2024-09-19T08:58:07.374 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 212438, 2024-09-19T08:58:07.374 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 7, 2024-09-19T08:58:07.374 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 3, 2024-09-19T08:58:07.374 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.055112080, 2024-09-19T08:58:07.374 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.018801217, 2024-09-19T08:58:07.374 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.051433106, 2024-09-19T08:58:07.374 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.000228248, 2024-09-19T08:58:07.374 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:58:07.374 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 353, 2024-09-19T08:58:07.374 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.030825973, 2024-09-19T08:58:07.375 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000087325 2024-09-19T08:58:07.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.375 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:58:07.375 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 116, 2024-09-19T08:58:07.375 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000027233, 2024-09-19T08:58:07.375 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000234 2024-09-19T08:58:07.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.375 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 1694368, 2024-09-19T08:58:07.375 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 152768 2024-09-19T08:58:07.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.375 INFO:teuthology.orchestra.run.smithi173.stdout: "cct": { 2024-09-19T08:58:07.375 INFO:teuthology.orchestra.run.smithi173.stdout: "total_workers": 1, 2024-09-19T08:58:07.375 INFO:teuthology.orchestra.run.smithi173.stdout: "unhealthy_workers": 0 2024-09-19T08:58:07.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.376 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-MDSRank": { 2024-09-19T08:58:07.376 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2024-09-19T08:58:07.376 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2024-09-19T08:58:07.376 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 35, 2024-09-19T08:58:07.376 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.002590372, 2024-09-19T08:58:07.376 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000074010 2024-09-19T08:58:07.376 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:07.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.376 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-PurgeQueue": { 2024-09-19T08:58:07.376 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2024-09-19T08:58:07.377 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2024-09-19T08:58:07.377 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 2, 2024-09-19T08:58:07.377 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000016846, 2024-09-19T08:58:07.377 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000008423 2024-09-19T08:58:07.377 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:07.377 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.377 INFO:teuthology.orchestra.run.smithi173.stdout: "mds": { 2024-09-19T08:58:07.377 INFO:teuthology.orchestra.run.smithi173.stdout: "request": 0, 2024-09-19T08:58:07.377 INFO:teuthology.orchestra.run.smithi173.stdout: "reply": 0, 2024-09-19T08:58:07.377 INFO:teuthology.orchestra.run.smithi173.stdout: "reply_latency": { 2024-09-19T08:58:07.377 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.378 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.378 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.378 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.378 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_reply": 0, 2024-09-19T08:58:07.378 INFO:teuthology.orchestra.run.smithi173.stdout: "forward": 0, 2024-09-19T08:58:07.378 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_complete": 0, 2024-09-19T08:58:07.378 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_keys": 0, 2024-09-19T08:58:07.378 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_commit": 11, 2024-09-19T08:58:07.378 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_split": 0, 2024-09-19T08:58:07.379 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_merge": 0, 2024-09-19T08:58:07.379 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes": 16, 2024-09-19T08:58:07.379 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_top": 0, 2024-09-19T08:58:07.379 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_bottom": 16, 2024-09-19T08:58:07.379 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pin_tail": 0, 2024-09-19T08:58:07.379 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pinned": 13, 2024-09-19T08:58:07.379 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_expired": 8, 2024-09-19T08:58:07.379 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_with_caps": 0, 2024-09-19T08:58:07.379 INFO:teuthology.orchestra.run.smithi173.stdout: "caps": 0, 2024-09-19T08:58:07.379 INFO:teuthology.orchestra.run.smithi173.stdout: "subtrees": 3, 2024-09-19T08:58:07.380 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse": 32, 2024-09-19T08:58:07.380 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_hit": 27, 2024-09-19T08:58:07.380 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_forward": 0, 2024-09-19T08:58:07.380 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_discover": 5, 2024-09-19T08:58:07.380 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_dir_fetch": 0, 2024-09-19T08:58:07.380 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_remote_ino": 0, 2024-09-19T08:58:07.381 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_lock": 0, 2024-09-19T08:58:07.381 INFO:teuthology.orchestra.run.smithi173.stdout: "load_cent": 0, 2024-09-19T08:58:07.381 INFO:teuthology.orchestra.run.smithi173.stdout: "q": 0, 2024-09-19T08:58:07.381 INFO:teuthology.orchestra.run.smithi173.stdout: "exported": 0, 2024-09-19T08:58:07.381 INFO:teuthology.orchestra.run.smithi173.stdout: "exported_inodes": 0, 2024-09-19T08:58:07.381 INFO:teuthology.orchestra.run.smithi173.stdout: "imported": 0, 2024-09-19T08:58:07.381 INFO:teuthology.orchestra.run.smithi173.stdout: "imported_inodes": 0, 2024-09-19T08:58:07.381 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_dir_fetch": 0, 2024-09-19T08:58:07.381 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_backtrace_fetch": 0, 2024-09-19T08:58:07.381 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_peer_discover": 0, 2024-09-19T08:58:07.381 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rfiles": 22, 2024-09-19T08:58:07.381 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rbytes": 10738142113, 2024-09-19T08:58:07.382 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rsnaps": 6, 2024-09-19T08:58:07.382 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T08:58:07.382 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_set_tag": 0, 2024-09-19T08:58:07.382 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T08:58:07.382 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_inotable_repaired": 0, 2024-09-19T08:58:07.382 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_inodes": 0, 2024-09-19T08:58:07.382 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T08:58:07.382 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T08:58:07.382 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_file_inodes": 0, 2024-09-19T08:58:07.382 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_inode_file_caps": 0, 2024-09-19T08:58:07.382 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T08:58:07.382 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_grant": 0, 2024-09-19T08:58:07.383 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T08:58:07.383 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T08:58:07.383 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T08:58:07.383 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps": 0, 2024-09-19T08:58:07.383 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps_dirty": 0, 2024-09-19T08:58:07.383 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_cap_release": 0, 2024-09-19T08:58:07.383 INFO:teuthology.orchestra.run.smithi173.stdout: "process_request_cap_release": 0 2024-09-19T08:58:07.383 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.383 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_cache": { 2024-09-19T08:58:07.384 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update": 0, 2024-09-19T08:58:07.384 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update_receipt": 17, 2024-09-19T08:58:07.384 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_try_discover": 11, 2024-09-19T08:58:07.384 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_send_discover": 13, 2024-09-19T08:58:07.384 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_handle_discover": 0, 2024-09-19T08:58:07.384 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays": 0, 2024-09-19T08:58:07.384 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_delayed": 0, 2024-09-19T08:58:07.384 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_enqueuing": 0, 2024-09-19T08:58:07.384 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_created": 0, 2024-09-19T08:58:07.384 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_enqueued": 0, 2024-09-19T08:58:07.384 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_reintegrated": 0, 2024-09-19T08:58:07.385 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_migrated": 0, 2024-09-19T08:58:07.385 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_processing": 0, 2024-09-19T08:58:07.385 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_enqueued": 0, 2024-09-19T08:58:07.385 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_prioritized": 0, 2024-09-19T08:58:07.385 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_started": 0, 2024-09-19T08:58:07.385 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_completed": 0, 2024-09-19T08:58:07.385 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_quiesce_path": 10, 2024-09-19T08:58:07.385 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_quiesce_inode": 86, 2024-09-19T08:58:07.385 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T08:58:07.385 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_exportdir": 0, 2024-09-19T08:58:07.385 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_flush": 0, 2024-09-19T08:58:07.385 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragmentdir": 0, 2024-09-19T08:58:07.386 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragstats": 0, 2024-09-19T08:58:07.386 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_inodestats": 0 2024-09-19T08:58:07.386 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.386 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_log": { 2024-09-19T08:58:07.386 INFO:teuthology.orchestra.run.smithi173.stdout: "evlrg": 0, 2024-09-19T08:58:07.386 INFO:teuthology.orchestra.run.smithi173.stdout: "evadd": 2, 2024-09-19T08:58:07.386 INFO:teuthology.orchestra.run.smithi173.stdout: "evex": 0, 2024-09-19T08:58:07.386 INFO:teuthology.orchestra.run.smithi173.stdout: "evtrm": 0, 2024-09-19T08:58:07.386 INFO:teuthology.orchestra.run.smithi173.stdout: "ev": 2, 2024-09-19T08:58:07.386 INFO:teuthology.orchestra.run.smithi173.stdout: "evexg": 0, 2024-09-19T08:58:07.386 INFO:teuthology.orchestra.run.smithi173.stdout: "evexd": 0, 2024-09-19T08:58:07.386 INFO:teuthology.orchestra.run.smithi173.stdout: "segadd": 2, 2024-09-19T08:58:07.386 INFO:teuthology.orchestra.run.smithi173.stdout: "segex": 0, 2024-09-19T08:58:07.387 INFO:teuthology.orchestra.run.smithi173.stdout: "segtrm": 0, 2024-09-19T08:58:07.387 INFO:teuthology.orchestra.run.smithi173.stdout: "seg": 2, 2024-09-19T08:58:07.387 INFO:teuthology.orchestra.run.smithi173.stdout: "segmjr": 2, 2024-09-19T08:58:07.387 INFO:teuthology.orchestra.run.smithi173.stdout: "segexg": 0, 2024-09-19T08:58:07.387 INFO:teuthology.orchestra.run.smithi173.stdout: "segexd": 0, 2024-09-19T08:58:07.387 INFO:teuthology.orchestra.run.smithi173.stdout: "expos": 4194304, 2024-09-19T08:58:07.387 INFO:teuthology.orchestra.run.smithi173.stdout: "wrpos": 4194892, 2024-09-19T08:58:07.387 INFO:teuthology.orchestra.run.smithi173.stdout: "rdpos": 0, 2024-09-19T08:58:07.387 INFO:teuthology.orchestra.run.smithi173.stdout: "jlat": { 2024-09-19T08:58:07.387 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 1, 2024-09-19T08:58:07.387 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.001271473, 2024-09-19T08:58:07.387 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.001271473 2024-09-19T08:58:07.387 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.388 INFO:teuthology.orchestra.run.smithi173.stdout: "replayed": 0 2024-09-19T08:58:07.388 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.388 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_mem": { 2024-09-19T08:58:07.388 INFO:teuthology.orchestra.run.smithi173.stdout: "ino": 20, 2024-09-19T08:58:07.388 INFO:teuthology.orchestra.run.smithi173.stdout: "ino+": 24, 2024-09-19T08:58:07.388 INFO:teuthology.orchestra.run.smithi173.stdout: "ino-": 4, 2024-09-19T08:58:07.388 INFO:teuthology.orchestra.run.smithi173.stdout: "dir": 18, 2024-09-19T08:58:07.388 INFO:teuthology.orchestra.run.smithi173.stdout: "dir+": 25, 2024-09-19T08:58:07.388 INFO:teuthology.orchestra.run.smithi173.stdout: "dir-": 7, 2024-09-19T08:58:07.388 INFO:teuthology.orchestra.run.smithi173.stdout: "dn": 16, 2024-09-19T08:58:07.388 INFO:teuthology.orchestra.run.smithi173.stdout: "dn+": 24, 2024-09-19T08:58:07.388 INFO:teuthology.orchestra.run.smithi173.stdout: "dn-": 8, 2024-09-19T08:58:07.389 INFO:teuthology.orchestra.run.smithi173.stdout: "cap": 0, 2024-09-19T08:58:07.389 INFO:teuthology.orchestra.run.smithi173.stdout: "cap+": 0, 2024-09-19T08:58:07.389 INFO:teuthology.orchestra.run.smithi173.stdout: "cap-": 0, 2024-09-19T08:58:07.389 INFO:teuthology.orchestra.run.smithi173.stdout: "rss": 51844, 2024-09-19T08:58:07.389 INFO:teuthology.orchestra.run.smithi173.stdout: "heap": 223520 2024-09-19T08:58:07.389 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.389 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_server": { 2024-09-19T08:58:07.389 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_client_request": 0, 2024-09-19T08:58:07.389 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_server_request": 1, 2024-09-19T08:58:07.389 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_request": 0, 2024-09-19T08:58:07.390 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_session": 0, 2024-09-19T08:58:07.390 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_peer_request": 2, 2024-09-19T08:58:07.390 INFO:teuthology.orchestra.run.smithi173.stdout: "req_create_latency": { 2024-09-19T08:58:07.390 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.390 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.390 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.390 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.390 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getattr_latency": { 2024-09-19T08:58:07.390 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.390 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.390 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.391 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.391 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getfilelock_latency": { 2024-09-19T08:58:07.391 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.391 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.391 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.391 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.391 INFO:teuthology.orchestra.run.smithi173.stdout: "req_link_latency": { 2024-09-19T08:58:07.391 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.391 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.391 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.391 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.391 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookup_latency": { 2024-09-19T08:58:07.391 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.392 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.392 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.392 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.392 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookuphash_latency": { 2024-09-19T08:58:07.392 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.392 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.392 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.392 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.392 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupino_latency": { 2024-09-19T08:58:07.392 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.392 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.393 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.393 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.393 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupname_latency": { 2024-09-19T08:58:07.393 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.393 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.393 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.393 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.393 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupparent_latency": { 2024-09-19T08:58:07.393 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.393 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.394 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.394 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.394 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupsnap_latency": { 2024-09-19T08:58:07.394 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.394 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.394 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.394 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.395 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lssnap_latency": { 2024-09-19T08:58:07.395 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.395 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.395 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.395 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.395 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mkdir_latency": { 2024-09-19T08:58:07.396 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.396 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.396 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.396 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.397 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mknod_latency": { 2024-09-19T08:58:07.397 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.397 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.397 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.397 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.397 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mksnap_latency": { 2024-09-19T08:58:07.397 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.397 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.397 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.397 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.397 INFO:teuthology.orchestra.run.smithi173.stdout: "req_open_latency": { 2024-09-19T08:58:07.397 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.398 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.398 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.398 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.398 INFO:teuthology.orchestra.run.smithi173.stdout: "req_readdir_latency": { 2024-09-19T08:58:07.398 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.398 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.398 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.398 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.398 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rename_latency": { 2024-09-19T08:58:07.398 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.398 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.399 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.399 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.399 INFO:teuthology.orchestra.run.smithi173.stdout: "req_renamesnap_latency": { 2024-09-19T08:58:07.399 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.399 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.399 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.399 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.400 INFO:teuthology.orchestra.run.smithi173.stdout: "req_snapdiff_latency": { 2024-09-19T08:58:07.400 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.400 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.400 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.400 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.400 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmdir_latency": { 2024-09-19T08:58:07.400 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.400 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.400 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.400 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.400 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmsnap_latency": { 2024-09-19T08:58:07.400 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.401 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.401 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.401 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.401 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmxattr_latency": { 2024-09-19T08:58:07.401 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.401 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.401 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.401 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.401 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setattr_latency": { 2024-09-19T08:58:07.401 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.401 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.401 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.401 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.402 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setdirlayout_latency": { 2024-09-19T08:58:07.402 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.402 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.402 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.402 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.402 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setfilelock_latency": { 2024-09-19T08:58:07.402 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.402 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.402 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.402 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.403 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setlayout_latency": { 2024-09-19T08:58:07.403 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.403 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.403 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.403 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.403 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setxattr_latency": { 2024-09-19T08:58:07.403 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.403 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.403 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.403 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.403 INFO:teuthology.orchestra.run.smithi173.stdout: "req_symlink_latency": { 2024-09-19T08:58:07.403 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.404 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.404 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.404 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.404 INFO:teuthology.orchestra.run.smithi173.stdout: "req_unlink_latency": { 2024-09-19T08:58:07.404 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.404 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.404 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.404 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.404 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_revoke_eviction": 0, 2024-09-19T08:58:07.405 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_acquisition_throttle": 0, 2024-09-19T08:58:07.405 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getvxattr_latency": { 2024-09-19T08:58:07.405 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.405 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.405 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.405 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:07.405 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.405 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_sessions": { 2024-09-19T08:58:07.405 INFO:teuthology.orchestra.run.smithi173.stdout: "session_count": 0, 2024-09-19T08:58:07.406 INFO:teuthology.orchestra.run.smithi173.stdout: "session_add": 0, 2024-09-19T08:58:07.406 INFO:teuthology.orchestra.run.smithi173.stdout: "session_remove": 0, 2024-09-19T08:58:07.406 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_open": 0, 2024-09-19T08:58:07.406 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_stale": 0, 2024-09-19T08:58:07.406 INFO:teuthology.orchestra.run.smithi173.stdout: "total_load": 0, 2024-09-19T08:58:07.406 INFO:teuthology.orchestra.run.smithi173.stdout: "average_load": 0, 2024-09-19T08:58:07.406 INFO:teuthology.orchestra.run.smithi173.stdout: "avg_session_uptime": 0, 2024-09-19T08:58:07.406 INFO:teuthology.orchestra.run.smithi173.stdout: "mdthresh_evicted": 0 2024-09-19T08:58:07.406 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.406 INFO:teuthology.orchestra.run.smithi173.stdout: "mempool": { 2024-09-19T08:58:07.406 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_bytes": 0, 2024-09-19T08:58:07.406 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_items": 0, 2024-09-19T08:58:07.406 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T08:58:07.407 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_items": 0, 2024-09-19T08:58:07.407 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T08:58:07.407 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_items": 0, 2024-09-19T08:58:07.407 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T08:58:07.407 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T08:58:07.407 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T08:58:07.407 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T08:58:07.407 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T08:58:07.407 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_items": 0, 2024-09-19T08:58:07.407 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T08:58:07.407 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T08:58:07.407 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_bytes": 0, 2024-09-19T08:58:07.407 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_items": 0, 2024-09-19T08:58:07.408 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_bytes": 0, 2024-09-19T08:58:07.408 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_items": 0, 2024-09-19T08:58:07.408 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T08:58:07.408 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T08:58:07.408 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T08:58:07.408 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T08:58:07.408 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T08:58:07.408 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_items": 0, 2024-09-19T08:58:07.408 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_bytes": 0, 2024-09-19T08:58:07.408 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_items": 0, 2024-09-19T08:58:07.408 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T08:58:07.408 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T08:58:07.409 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_bytes": 0, 2024-09-19T08:58:07.409 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_items": 0, 2024-09-19T08:58:07.409 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_bytes": 0, 2024-09-19T08:58:07.409 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_items": 0, 2024-09-19T08:58:07.409 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T08:58:07.409 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_items": 0, 2024-09-19T08:58:07.409 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T08:58:07.409 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_items": 0, 2024-09-19T08:58:07.409 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_bytes": 246343, 2024-09-19T08:58:07.409 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_items": 84, 2024-09-19T08:58:07.409 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_bytes": 0, 2024-09-19T08:58:07.409 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_items": 0, 2024-09-19T08:58:07.409 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_bytes": 0, 2024-09-19T08:58:07.410 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_items": 0, 2024-09-19T08:58:07.410 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_bytes": 0, 2024-09-19T08:58:07.410 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_items": 0, 2024-09-19T08:58:07.410 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_bytes": 0, 2024-09-19T08:58:07.410 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_items": 0, 2024-09-19T08:58:07.410 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_bytes": 7936, 2024-09-19T08:58:07.410 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_items": 175, 2024-09-19T08:58:07.410 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T08:58:07.410 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_items": 0, 2024-09-19T08:58:07.410 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_bytes": 0, 2024-09-19T08:58:07.410 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_items": 0, 2024-09-19T08:58:07.411 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_bytes": 81229, 2024-09-19T08:58:07.411 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_items": 326, 2024-09-19T08:58:07.411 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_bytes": 0, 2024-09-19T08:58:07.411 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_items": 0, 2024-09-19T08:58:07.411 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_bytes": 0, 2024-09-19T08:58:07.411 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_items": 0 2024-09-19T08:58:07.411 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.411 INFO:teuthology.orchestra.run.smithi173.stdout: "objecter": { 2024-09-19T08:58:07.411 INFO:teuthology.orchestra.run.smithi173.stdout: "op_active": 0, 2024-09-19T08:58:07.411 INFO:teuthology.orchestra.run.smithi173.stdout: "op_laggy": 0, 2024-09-19T08:58:07.411 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send": 35, 2024-09-19T08:58:07.412 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send_bytes": 10710, 2024-09-19T08:58:07.412 INFO:teuthology.orchestra.run.smithi173.stdout: "op_resend": 30, 2024-09-19T08:58:07.412 INFO:teuthology.orchestra.run.smithi173.stdout: "op_reply": 35, 2024-09-19T08:58:07.412 INFO:teuthology.orchestra.run.smithi173.stdout: "op_latency": { 2024-09-19T08:58:07.412 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 35, 2024-09-19T08:58:07.412 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 5.185877753, 2024-09-19T08:58:07.412 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.148167935 2024-09-19T08:58:07.412 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.412 INFO:teuthology.orchestra.run.smithi173.stdout: "op_inflight": 0, 2024-09-19T08:58:07.412 INFO:teuthology.orchestra.run.smithi173.stdout: "oplen_avg": { 2024-09-19T08:58:07.412 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 35, 2024-09-19T08:58:07.413 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 57 2024-09-19T08:58:07.413 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.413 INFO:teuthology.orchestra.run.smithi173.stdout: "op": 35, 2024-09-19T08:58:07.413 INFO:teuthology.orchestra.run.smithi173.stdout: "op_r": 0, 2024-09-19T08:58:07.413 INFO:teuthology.orchestra.run.smithi173.stdout: "op_w": 35, 2024-09-19T08:58:07.413 INFO:teuthology.orchestra.run.smithi173.stdout: "op_rmw": 0, 2024-09-19T08:58:07.413 INFO:teuthology.orchestra.run.smithi173.stdout: "op_pg": 0, 2024-09-19T08:58:07.413 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_stat": 0, 2024-09-19T08:58:07.413 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_create": 10, 2024-09-19T08:58:07.413 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_read": 0, 2024-09-19T08:58:07.413 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_write": 1, 2024-09-19T08:58:07.413 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writefull": 5, 2024-09-19T08:58:07.413 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writesame": 0, 2024-09-19T08:58:07.414 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_append": 0, 2024-09-19T08:58:07.414 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_zero": 0, 2024-09-19T08:58:07.414 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_truncate": 0, 2024-09-19T08:58:07.414 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_delete": 6, 2024-09-19T08:58:07.414 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_mapext": 0, 2024-09-19T08:58:07.414 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_sparse_read": 0, 2024-09-19T08:58:07.414 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_clonerange": 0, 2024-09-19T08:58:07.414 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_getxattr": 0, 2024-09-19T08:58:07.414 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_setxattr": 20, 2024-09-19T08:58:07.414 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_cmpxattr": 0, 2024-09-19T08:58:07.414 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_rmxattr": 0, 2024-09-19T08:58:07.414 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_resetxattrs": 0, 2024-09-19T08:58:07.414 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_call": 0, 2024-09-19T08:58:07.415 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_watch": 0, 2024-09-19T08:58:07.415 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_notify": 0, 2024-09-19T08:58:07.415 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T08:58:07.415 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls": 0, 2024-09-19T08:58:07.415 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls_filter": 0, 2024-09-19T08:58:07.415 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_other": 0, 2024-09-19T08:58:07.415 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_active": 0, 2024-09-19T08:58:07.415 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_send": 0, 2024-09-19T08:58:07.415 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_resend": 0, 2024-09-19T08:58:07.415 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_ping": 0, 2024-09-19T08:58:07.415 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_active": 0, 2024-09-19T08:58:07.415 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_send": 0, 2024-09-19T08:58:07.416 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_resend": 0, 2024-09-19T08:58:07.416 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_active": 0, 2024-09-19T08:58:07.416 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_send": 0, 2024-09-19T08:58:07.416 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_resend": 0, 2024-09-19T08:58:07.416 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_active": 0, 2024-09-19T08:58:07.416 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_send": 0, 2024-09-19T08:58:07.416 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_resend": 0, 2024-09-19T08:58:07.416 INFO:teuthology.orchestra.run.smithi173.stdout: "command_active": 0, 2024-09-19T08:58:07.416 INFO:teuthology.orchestra.run.smithi173.stdout: "command_send": 0, 2024-09-19T08:58:07.416 INFO:teuthology.orchestra.run.smithi173.stdout: "command_resend": 0, 2024-09-19T08:58:07.416 INFO:teuthology.orchestra.run.smithi173.stdout: "map_epoch": 95, 2024-09-19T08:58:07.416 INFO:teuthology.orchestra.run.smithi173.stdout: "map_full": 0, 2024-09-19T08:58:07.417 INFO:teuthology.orchestra.run.smithi173.stdout: "map_inc": 14, 2024-09-19T08:58:07.417 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_sessions": 9, 2024-09-19T08:58:07.417 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_open": 9, 2024-09-19T08:58:07.417 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_close": 0, 2024-09-19T08:58:07.417 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_laggy": 0, 2024-09-19T08:58:07.417 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_wr": 14, 2024-09-19T08:58:07.417 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_rd": 0, 2024-09-19T08:58:07.417 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_del": 1 2024-09-19T08:58:07.417 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.417 INFO:teuthology.orchestra.run.smithi173.stdout: "oft": { 2024-09-19T08:58:07.417 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_objs": 1, 2024-09-19T08:58:07.417 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_kv_pairs": 0, 2024-09-19T08:58:07.417 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_updates": 0, 2024-09-19T08:58:07.418 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_removes": 0 2024-09-19T08:58:07.418 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.418 INFO:teuthology.orchestra.run.smithi173.stdout: "osd-slow-ops": { 2024-09-19T08:58:07.418 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_ops_count": 86 2024-09-19T08:58:07.418 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.418 INFO:teuthology.orchestra.run.smithi173.stdout: "purge_queue": { 2024-09-19T08:58:07.419 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops": 0, 2024-09-19T08:58:07.419 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T08:58:07.419 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing": 0, 2024-09-19T08:58:07.419 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_high_water": 0, 2024-09-19T08:58:07.419 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed_ops": 0, 2024-09-19T08:58:07.419 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed": 0, 2024-09-19T08:58:07.419 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_item_in_journal": 0 2024-09-19T08:58:07.419 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.419 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T08:58:07.419 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-09-19T08:58:07.420 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-09-19T08:58:07.420 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-09-19T08:58:07.420 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 1786, 2024-09-19T08:58:07.420 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 2086766, 2024-09-19T08:58:07.420 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-09-19T08:58:07.420 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 1786, 2024-09-19T08:58:07.420 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-09-19T08:58:07.420 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-09-19T08:58:07.420 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 1786, 2024-09-19T08:58:07.420 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 2086766, 2024-09-19T08:58:07.420 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-09-19T08:58:07.420 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.420 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.421 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.421 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:07.421 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.421 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_bytes": { 2024-09-19T08:58:07.421 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-09-19T08:58:07.421 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-09-19T08:58:07.421 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-09-19T08:58:07.421 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-09-19T08:58:07.421 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-09-19T08:58:07.421 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-09-19T08:58:07.421 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-09-19T08:58:07.421 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 35, 2024-09-19T08:58:07.421 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 10710, 2024-09-19T08:58:07.422 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 35, 2024-09-19T08:58:07.422 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 10710, 2024-09-19T08:58:07.422 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-09-19T08:58:07.422 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.422 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.422 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.422 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:07.422 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.422 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_ops": { 2024-09-19T08:58:07.422 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-09-19T08:58:07.422 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 1024, 2024-09-19T08:58:07.422 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-09-19T08:58:07.423 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-09-19T08:58:07.423 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-09-19T08:58:07.423 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-09-19T08:58:07.423 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-09-19T08:58:07.423 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 35, 2024-09-19T08:58:07.423 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 35, 2024-09-19T08:58:07.423 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 35, 2024-09-19T08:58:07.423 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 35, 2024-09-19T08:58:07.424 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-09-19T08:58:07.424 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.424 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.424 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.424 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:07.424 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.424 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle": { 2024-09-19T08:58:07.424 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-09-19T08:58:07.424 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2024-09-19T08:58:07.424 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-09-19T08:58:07.424 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-09-19T08:58:07.424 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-09-19T08:58:07.425 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-09-19T08:58:07.425 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-09-19T08:58:07.425 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-09-19T08:58:07.425 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-09-19T08:58:07.425 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 0, 2024-09-19T08:58:07.425 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 0, 2024-09-19T08:58:07.425 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-09-19T08:58:07.426 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.426 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.426 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.426 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:07.426 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:07.426 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle-0x555584526960": { 2024-09-19T08:58:07.426 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-09-19T08:58:07.426 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2024-09-19T08:58:07.426 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-09-19T08:58:07.426 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 2, 2024-09-19T08:58:07.426 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 588, 2024-09-19T08:58:07.426 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-09-19T08:58:07.427 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 2, 2024-09-19T08:58:07.427 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-09-19T08:58:07.427 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-09-19T08:58:07.427 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 1, 2024-09-19T08:58:07.427 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 588, 2024-09-19T08:58:07.427 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-09-19T08:58:07.427 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:07.427 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:07.427 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:07.428 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:07.428 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:07.428 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-19T08:58:08.167 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-19T08:58:08.167 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-19T08:58:08.167 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-09-19T08:58:08.167 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=17 2024-09-19T08:58:08.167 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-19T08:58:08.168 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-19T08:58:08.168 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T08:58:08.168 DEBUG:tasks.check_counter:Getting stats from l 2024-09-19T08:58:08.168 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-19T08:58:08.281 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:08 smithi164 ceph-mon[30928]: pgmap v384: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-19T08:58:08.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:08 smithi170 ceph-mon[35072]: pgmap v384: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-19T08:58:08.413 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:08 smithi173 ceph-mon[35306]: pgmap v384: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-19T08:58:08.596 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T08:58:09.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:09 smithi164 ceph-mon[30928]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T08:58:09.322 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-19T08:58:09.323 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T08:58:09.323 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 830, 2024-09-19T08:58:09.323 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 832, 2024-09-19T08:58:09.323 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 218083, 2024-09-19T08:58:09.323 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 264003, 2024-09-19T08:58:09.323 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 6, 2024-09-19T08:58:09.323 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 2, 2024-09-19T08:58:09.323 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.080986651, 2024-09-19T08:58:09.323 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.022919767, 2024-09-19T08:58:09.323 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.023424494, 2024-09-19T08:58:09.323 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.026394674, 2024-09-19T08:58:09.324 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:58:09.324 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 830, 2024-09-19T08:58:09.324 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.033352157, 2024-09-19T08:58:09.324 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000040183 2024-09-19T08:58:09.324 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.324 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:58:09.324 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.324 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.324 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.324 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.324 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T08:58:09.324 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T08:58:09.325 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.325 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T08:58:09.325 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 285, 2024-09-19T08:58:09.325 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 263, 2024-09-19T08:58:09.325 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 1681918, 2024-09-19T08:58:09.325 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 152290, 2024-09-19T08:58:09.325 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 4, 2024-09-19T08:58:09.325 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 3, 2024-09-19T08:58:09.325 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.049824201, 2024-09-19T08:58:09.325 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.017468833, 2024-09-19T08:58:09.325 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.047345798, 2024-09-19T08:58:09.325 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.000074104, 2024-09-19T08:58:09.326 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:58:09.326 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 263, 2024-09-19T08:58:09.326 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.024544278, 2024-09-19T08:58:09.326 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000093324 2024-09-19T08:58:09.326 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.326 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:58:09.326 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.326 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.326 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.326 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.326 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 1680816, 2024-09-19T08:58:09.326 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 151648 2024-09-19T08:58:09.326 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.327 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T08:58:09.327 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 828, 2024-09-19T08:58:09.327 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 1011, 2024-09-19T08:58:09.327 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 277856, 2024-09-19T08:58:09.327 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 744124, 2024-09-19T08:58:09.327 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 6, 2024-09-19T08:58:09.327 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 4, 2024-09-19T08:58:09.327 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.111320417, 2024-09-19T08:58:09.327 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.037005341, 2024-09-19T08:58:09.327 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.029121291, 2024-09-19T08:58:09.327 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.034004781, 2024-09-19T08:58:09.327 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T08:58:09.328 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 1008, 2024-09-19T08:58:09.328 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.062687253, 2024-09-19T08:58:09.328 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000062189 2024-09-19T08:58:09.328 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.328 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-09-19T08:58:09.328 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.328 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.328 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.328 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.328 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T08:58:09.328 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 481456 2024-09-19T08:58:09.328 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.329 INFO:teuthology.orchestra.run.smithi173.stdout: "cct": { 2024-09-19T08:58:09.329 INFO:teuthology.orchestra.run.smithi173.stdout: "total_workers": 1, 2024-09-19T08:58:09.329 INFO:teuthology.orchestra.run.smithi173.stdout: "unhealthy_workers": 0 2024-09-19T08:58:09.329 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.329 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-MDSRank": { 2024-09-19T08:58:09.329 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2024-09-19T08:58:09.329 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2024-09-19T08:58:09.329 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 3309, 2024-09-19T08:58:09.329 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.106088472, 2024-09-19T08:58:09.329 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000032060 2024-09-19T08:58:09.329 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:09.329 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.329 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-PurgeQueue": { 2024-09-19T08:58:09.330 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2024-09-19T08:58:09.330 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2024-09-19T08:58:09.330 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.330 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.330 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.330 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:09.330 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.330 INFO:teuthology.orchestra.run.smithi173.stdout: "mds": { 2024-09-19T08:58:09.330 INFO:teuthology.orchestra.run.smithi173.stdout: "request": 0, 2024-09-19T08:58:09.330 INFO:teuthology.orchestra.run.smithi173.stdout: "reply": 0, 2024-09-19T08:58:09.330 INFO:teuthology.orchestra.run.smithi173.stdout: "reply_latency": { 2024-09-19T08:58:09.331 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.331 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.331 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.331 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.331 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_reply": 0, 2024-09-19T08:58:09.331 INFO:teuthology.orchestra.run.smithi173.stdout: "forward": 0, 2024-09-19T08:58:09.331 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_complete": 0, 2024-09-19T08:58:09.331 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_keys": 0, 2024-09-19T08:58:09.331 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_commit": 0, 2024-09-19T08:58:09.331 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_split": 0, 2024-09-19T08:58:09.331 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_merge": 0, 2024-09-19T08:58:09.331 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes": 0, 2024-09-19T08:58:09.331 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_top": 0, 2024-09-19T08:58:09.332 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_bottom": 0, 2024-09-19T08:58:09.332 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pin_tail": 0, 2024-09-19T08:58:09.332 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pinned": 0, 2024-09-19T08:58:09.332 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_expired": 0, 2024-09-19T08:58:09.332 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_with_caps": 0, 2024-09-19T08:58:09.332 INFO:teuthology.orchestra.run.smithi173.stdout: "caps": 0, 2024-09-19T08:58:09.332 INFO:teuthology.orchestra.run.smithi173.stdout: "subtrees": 1, 2024-09-19T08:58:09.332 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse": 0, 2024-09-19T08:58:09.332 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_hit": 0, 2024-09-19T08:58:09.332 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_forward": 0, 2024-09-19T08:58:09.332 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_discover": 0, 2024-09-19T08:58:09.332 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_dir_fetch": 0, 2024-09-19T08:58:09.332 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_remote_ino": 0, 2024-09-19T08:58:09.333 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_lock": 0, 2024-09-19T08:58:09.333 INFO:teuthology.orchestra.run.smithi173.stdout: "load_cent": 0, 2024-09-19T08:58:09.333 INFO:teuthology.orchestra.run.smithi173.stdout: "q": 0, 2024-09-19T08:58:09.333 INFO:teuthology.orchestra.run.smithi173.stdout: "exported": 0, 2024-09-19T08:58:09.333 INFO:teuthology.orchestra.run.smithi173.stdout: "exported_inodes": 0, 2024-09-19T08:58:09.333 INFO:teuthology.orchestra.run.smithi173.stdout: "imported": 0, 2024-09-19T08:58:09.333 INFO:teuthology.orchestra.run.smithi173.stdout: "imported_inodes": 0, 2024-09-19T08:58:09.333 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_dir_fetch": 0, 2024-09-19T08:58:09.333 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_backtrace_fetch": 0, 2024-09-19T08:58:09.333 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_peer_discover": 0, 2024-09-19T08:58:09.333 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rfiles": 0, 2024-09-19T08:58:09.334 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rbytes": 0, 2024-09-19T08:58:09.334 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rsnaps": 0, 2024-09-19T08:58:09.334 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T08:58:09.334 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_set_tag": 0, 2024-09-19T08:58:09.334 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T08:58:09.334 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_inotable_repaired": 0, 2024-09-19T08:58:09.334 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_inodes": 0, 2024-09-19T08:58:09.334 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T08:58:09.334 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T08:58:09.334 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_file_inodes": 0, 2024-09-19T08:58:09.334 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_inode_file_caps": 0, 2024-09-19T08:58:09.335 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T08:58:09.335 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_grant": 0, 2024-09-19T08:58:09.335 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T08:58:09.335 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T08:58:09.335 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T08:58:09.335 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps": 0, 2024-09-19T08:58:09.335 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps_dirty": 0, 2024-09-19T08:58:09.335 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_cap_release": 0, 2024-09-19T08:58:09.335 INFO:teuthology.orchestra.run.smithi173.stdout: "process_request_cap_release": 0 2024-09-19T08:58:09.335 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.335 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_cache": { 2024-09-19T08:58:09.335 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update": 0, 2024-09-19T08:58:09.335 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update_receipt": 0, 2024-09-19T08:58:09.335 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_try_discover": 0, 2024-09-19T08:58:09.336 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_send_discover": 0, 2024-09-19T08:58:09.336 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_handle_discover": 0, 2024-09-19T08:58:09.336 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays": 0, 2024-09-19T08:58:09.336 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_delayed": 0, 2024-09-19T08:58:09.336 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_enqueuing": 0, 2024-09-19T08:58:09.336 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_created": 0, 2024-09-19T08:58:09.336 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_enqueued": 0, 2024-09-19T08:58:09.336 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_reintegrated": 0, 2024-09-19T08:58:09.336 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_migrated": 0, 2024-09-19T08:58:09.336 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_processing": 0, 2024-09-19T08:58:09.336 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_enqueued": 0, 2024-09-19T08:58:09.336 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_prioritized": 0, 2024-09-19T08:58:09.336 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_started": 0, 2024-09-19T08:58:09.337 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_completed": 0, 2024-09-19T08:58:09.337 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_quiesce_path": 0, 2024-09-19T08:58:09.337 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_quiesce_inode": 0, 2024-09-19T08:58:09.337 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T08:58:09.337 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_exportdir": 0, 2024-09-19T08:58:09.337 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_flush": 0, 2024-09-19T08:58:09.337 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragmentdir": 0, 2024-09-19T08:58:09.337 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragstats": 0, 2024-09-19T08:58:09.337 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_inodestats": 0 2024-09-19T08:58:09.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.337 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_log": { 2024-09-19T08:58:09.337 INFO:teuthology.orchestra.run.smithi173.stdout: "evlrg": 0, 2024-09-19T08:58:09.337 INFO:teuthology.orchestra.run.smithi173.stdout: "evadd": 0, 2024-09-19T08:58:09.338 INFO:teuthology.orchestra.run.smithi173.stdout: "evex": 0, 2024-09-19T08:58:09.338 INFO:teuthology.orchestra.run.smithi173.stdout: "evtrm": 0, 2024-09-19T08:58:09.338 INFO:teuthology.orchestra.run.smithi173.stdout: "ev": 2, 2024-09-19T08:58:09.338 INFO:teuthology.orchestra.run.smithi173.stdout: "evexg": 0, 2024-09-19T08:58:09.338 INFO:teuthology.orchestra.run.smithi173.stdout: "evexd": 0, 2024-09-19T08:58:09.338 INFO:teuthology.orchestra.run.smithi173.stdout: "segadd": 0, 2024-09-19T08:58:09.338 INFO:teuthology.orchestra.run.smithi173.stdout: "segex": 0, 2024-09-19T08:58:09.338 INFO:teuthology.orchestra.run.smithi173.stdout: "segtrm": 0, 2024-09-19T08:58:09.338 INFO:teuthology.orchestra.run.smithi173.stdout: "seg": 2, 2024-09-19T08:58:09.338 INFO:teuthology.orchestra.run.smithi173.stdout: "segmjr": 2, 2024-09-19T08:58:09.338 INFO:teuthology.orchestra.run.smithi173.stdout: "segexg": 0, 2024-09-19T08:58:09.338 INFO:teuthology.orchestra.run.smithi173.stdout: "segexd": 0, 2024-09-19T08:58:09.338 INFO:teuthology.orchestra.run.smithi173.stdout: "expos": 4194304, 2024-09-19T08:58:09.339 INFO:teuthology.orchestra.run.smithi173.stdout: "wrpos": 4194892, 2024-09-19T08:58:09.339 INFO:teuthology.orchestra.run.smithi173.stdout: "rdpos": 4194352, 2024-09-19T08:58:09.339 INFO:teuthology.orchestra.run.smithi173.stdout: "jlat": { 2024-09-19T08:58:09.339 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.339 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.339 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.339 INFO:teuthology.orchestra.run.smithi173.stdout: "replayed": 2 2024-09-19T08:58:09.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.339 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_mem": { 2024-09-19T08:58:09.339 INFO:teuthology.orchestra.run.smithi173.stdout: "ino": 3, 2024-09-19T08:58:09.339 INFO:teuthology.orchestra.run.smithi173.stdout: "ino+": 3, 2024-09-19T08:58:09.339 INFO:teuthology.orchestra.run.smithi173.stdout: "ino-": 0, 2024-09-19T08:58:09.339 INFO:teuthology.orchestra.run.smithi173.stdout: "dir": 1, 2024-09-19T08:58:09.340 INFO:teuthology.orchestra.run.smithi173.stdout: "dir+": 1, 2024-09-19T08:58:09.340 INFO:teuthology.orchestra.run.smithi173.stdout: "dir-": 0, 2024-09-19T08:58:09.340 INFO:teuthology.orchestra.run.smithi173.stdout: "dn": 0, 2024-09-19T08:58:09.340 INFO:teuthology.orchestra.run.smithi173.stdout: "dn+": 0, 2024-09-19T08:58:09.340 INFO:teuthology.orchestra.run.smithi173.stdout: "dn-": 0, 2024-09-19T08:58:09.340 INFO:teuthology.orchestra.run.smithi173.stdout: "cap": 0, 2024-09-19T08:58:09.340 INFO:teuthology.orchestra.run.smithi173.stdout: "cap+": 0, 2024-09-19T08:58:09.340 INFO:teuthology.orchestra.run.smithi173.stdout: "cap-": 0, 2024-09-19T08:58:09.340 INFO:teuthology.orchestra.run.smithi173.stdout: "rss": 48408, 2024-09-19T08:58:09.340 INFO:teuthology.orchestra.run.smithi173.stdout: "heap": 215328 2024-09-19T08:58:09.340 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.340 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_server": { 2024-09-19T08:58:09.341 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_client_request": 0, 2024-09-19T08:58:09.341 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_server_request": 0, 2024-09-19T08:58:09.341 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_request": 0, 2024-09-19T08:58:09.341 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_session": 0, 2024-09-19T08:58:09.341 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_peer_request": 0, 2024-09-19T08:58:09.341 INFO:teuthology.orchestra.run.smithi173.stdout: "req_create_latency": { 2024-09-19T08:58:09.341 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.341 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.341 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.341 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.341 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getattr_latency": { 2024-09-19T08:58:09.341 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.341 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.342 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.342 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.342 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getfilelock_latency": { 2024-09-19T08:58:09.342 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.342 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.342 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.342 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.342 INFO:teuthology.orchestra.run.smithi173.stdout: "req_link_latency": { 2024-09-19T08:58:09.342 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.343 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.343 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.343 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.343 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookup_latency": { 2024-09-19T08:58:09.343 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.343 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.343 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.343 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.343 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookuphash_latency": { 2024-09-19T08:58:09.343 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.343 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.343 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.344 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.344 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupino_latency": { 2024-09-19T08:58:09.344 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.344 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.344 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.344 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.344 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupname_latency": { 2024-09-19T08:58:09.344 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.344 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.344 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.344 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.344 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupparent_latency": { 2024-09-19T08:58:09.344 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.345 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.345 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.345 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.345 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupsnap_latency": { 2024-09-19T08:58:09.345 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.345 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.345 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.345 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.345 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lssnap_latency": { 2024-09-19T08:58:09.345 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.345 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.346 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.346 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.346 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mkdir_latency": { 2024-09-19T08:58:09.346 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.346 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.346 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.346 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.346 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mknod_latency": { 2024-09-19T08:58:09.346 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.346 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.346 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.346 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.346 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mksnap_latency": { 2024-09-19T08:58:09.347 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.347 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.347 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.347 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.347 INFO:teuthology.orchestra.run.smithi173.stdout: "req_open_latency": { 2024-09-19T08:58:09.347 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.347 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.347 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.347 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.347 INFO:teuthology.orchestra.run.smithi173.stdout: "req_readdir_latency": { 2024-09-19T08:58:09.347 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.347 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.348 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.348 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rename_latency": { 2024-09-19T08:58:09.348 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.348 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.348 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.348 INFO:teuthology.orchestra.run.smithi173.stdout: "req_renamesnap_latency": { 2024-09-19T08:58:09.348 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.348 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.348 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.348 INFO:teuthology.orchestra.run.smithi173.stdout: "req_snapdiff_latency": { 2024-09-19T08:58:09.349 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.349 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.349 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.349 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.349 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmdir_latency": { 2024-09-19T08:58:09.349 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.349 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.349 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.349 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.349 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmsnap_latency": { 2024-09-19T08:58:09.349 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.349 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.349 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.350 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.350 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmxattr_latency": { 2024-09-19T08:58:09.350 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.350 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.350 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.350 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.350 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setattr_latency": { 2024-09-19T08:58:09.350 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.350 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.350 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.350 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.350 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setdirlayout_latency": { 2024-09-19T08:58:09.350 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.351 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.351 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.351 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.351 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setfilelock_latency": { 2024-09-19T08:58:09.351 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.351 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.351 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.351 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.352 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setlayout_latency": { 2024-09-19T08:58:09.352 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.352 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.352 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.352 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.352 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setxattr_latency": { 2024-09-19T08:58:09.352 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.352 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.352 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.352 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.352 INFO:teuthology.orchestra.run.smithi173.stdout: "req_symlink_latency": { 2024-09-19T08:58:09.352 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.353 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.353 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.353 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.353 INFO:teuthology.orchestra.run.smithi173.stdout: "req_unlink_latency": { 2024-09-19T08:58:09.353 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.353 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.353 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.353 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.353 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_revoke_eviction": 0, 2024-09-19T08:58:09.353 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_acquisition_throttle": 0, 2024-09-19T08:58:09.353 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getvxattr_latency": { 2024-09-19T08:58:09.353 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.353 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.354 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.354 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:09.354 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.354 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_sessions": { 2024-09-19T08:58:09.354 INFO:teuthology.orchestra.run.smithi173.stdout: "session_count": 0, 2024-09-19T08:58:09.354 INFO:teuthology.orchestra.run.smithi173.stdout: "session_add": 0, 2024-09-19T08:58:09.354 INFO:teuthology.orchestra.run.smithi173.stdout: "session_remove": 0, 2024-09-19T08:58:09.354 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_open": 0, 2024-09-19T08:58:09.354 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_stale": 0, 2024-09-19T08:58:09.354 INFO:teuthology.orchestra.run.smithi173.stdout: "total_load": 0, 2024-09-19T08:58:09.354 INFO:teuthology.orchestra.run.smithi173.stdout: "average_load": 0, 2024-09-19T08:58:09.354 INFO:teuthology.orchestra.run.smithi173.stdout: "avg_session_uptime": 0, 2024-09-19T08:58:09.355 INFO:teuthology.orchestra.run.smithi173.stdout: "mdthresh_evicted": 0 2024-09-19T08:58:09.355 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.355 INFO:teuthology.orchestra.run.smithi173.stdout: "mempool": { 2024-09-19T08:58:09.355 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_bytes": 0, 2024-09-19T08:58:09.355 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_items": 0, 2024-09-19T08:58:09.355 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T08:58:09.355 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_items": 0, 2024-09-19T08:58:09.355 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T08:58:09.355 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_items": 0, 2024-09-19T08:58:09.355 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T08:58:09.355 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T08:58:09.355 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T08:58:09.355 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T08:58:09.355 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T08:58:09.356 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_items": 0, 2024-09-19T08:58:09.356 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T08:58:09.356 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T08:58:09.356 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_bytes": 0, 2024-09-19T08:58:09.356 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_items": 0, 2024-09-19T08:58:09.356 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_bytes": 0, 2024-09-19T08:58:09.356 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_items": 0, 2024-09-19T08:58:09.356 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T08:58:09.356 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T08:58:09.356 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T08:58:09.356 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T08:58:09.356 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T08:58:09.356 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_items": 0, 2024-09-19T08:58:09.357 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_bytes": 0, 2024-09-19T08:58:09.357 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_items": 0, 2024-09-19T08:58:09.357 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T08:58:09.357 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T08:58:09.357 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_bytes": 0, 2024-09-19T08:58:09.357 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_items": 0, 2024-09-19T08:58:09.357 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_bytes": 0, 2024-09-19T08:58:09.357 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_items": 0, 2024-09-19T08:58:09.357 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T08:58:09.357 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_items": 0, 2024-09-19T08:58:09.357 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T08:58:09.357 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_items": 0, 2024-09-19T08:58:09.357 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_bytes": 189401, 2024-09-19T08:58:09.358 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_items": 51, 2024-09-19T08:58:09.358 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_bytes": 0, 2024-09-19T08:58:09.358 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_items": 0, 2024-09-19T08:58:09.358 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_bytes": 0, 2024-09-19T08:58:09.358 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_items": 0, 2024-09-19T08:58:09.358 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_bytes": 0, 2024-09-19T08:58:09.358 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_items": 0, 2024-09-19T08:58:09.358 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_bytes": 0, 2024-09-19T08:58:09.358 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_items": 0, 2024-09-19T08:58:09.358 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_bytes": 7936, 2024-09-19T08:58:09.358 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_items": 175, 2024-09-19T08:58:09.358 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T08:58:09.358 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_items": 0, 2024-09-19T08:58:09.359 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_bytes": 0, 2024-09-19T08:58:09.359 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_items": 0, 2024-09-19T08:58:09.359 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_bytes": 8512, 2024-09-19T08:58:09.359 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_items": 13, 2024-09-19T08:58:09.359 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_bytes": 0, 2024-09-19T08:58:09.359 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_items": 0, 2024-09-19T08:58:09.359 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_bytes": 0, 2024-09-19T08:58:09.359 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_items": 0 2024-09-19T08:58:09.359 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.359 INFO:teuthology.orchestra.run.smithi173.stdout: "objecter": { 2024-09-19T08:58:09.359 INFO:teuthology.orchestra.run.smithi173.stdout: "op_active": 0, 2024-09-19T08:58:09.359 INFO:teuthology.orchestra.run.smithi173.stdout: "op_laggy": 0, 2024-09-19T08:58:09.359 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send": 1658, 2024-09-19T08:58:09.360 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send_bytes": 21, 2024-09-19T08:58:09.360 INFO:teuthology.orchestra.run.smithi173.stdout: "op_resend": 3, 2024-09-19T08:58:09.360 INFO:teuthology.orchestra.run.smithi173.stdout: "op_reply": 1658, 2024-09-19T08:58:09.360 INFO:teuthology.orchestra.run.smithi173.stdout: "op_latency": { 2024-09-19T08:58:09.360 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 1658, 2024-09-19T08:58:09.360 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 18.931551498, 2024-09-19T08:58:09.360 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.011418306 2024-09-19T08:58:09.360 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.360 INFO:teuthology.orchestra.run.smithi173.stdout: "op_inflight": 0, 2024-09-19T08:58:09.360 INFO:teuthology.orchestra.run.smithi173.stdout: "oplen_avg": { 2024-09-19T08:58:09.360 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 1658, 2024-09-19T08:58:09.360 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 1659 2024-09-19T08:58:09.361 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.361 INFO:teuthology.orchestra.run.smithi173.stdout: "op": 1658, 2024-09-19T08:58:09.361 INFO:teuthology.orchestra.run.smithi173.stdout: "op_r": 1658, 2024-09-19T08:58:09.361 INFO:teuthology.orchestra.run.smithi173.stdout: "op_w": 0, 2024-09-19T08:58:09.361 INFO:teuthology.orchestra.run.smithi173.stdout: "op_rmw": 0, 2024-09-19T08:58:09.361 INFO:teuthology.orchestra.run.smithi173.stdout: "op_pg": 0, 2024-09-19T08:58:09.361 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_stat": 826, 2024-09-19T08:58:09.361 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_create": 0, 2024-09-19T08:58:09.361 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_read": 830, 2024-09-19T08:58:09.361 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_write": 0, 2024-09-19T08:58:09.361 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writefull": 0, 2024-09-19T08:58:09.361 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writesame": 0, 2024-09-19T08:58:09.361 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_append": 0, 2024-09-19T08:58:09.362 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_zero": 0, 2024-09-19T08:58:09.362 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_truncate": 0, 2024-09-19T08:58:09.362 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_delete": 0, 2024-09-19T08:58:09.362 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_mapext": 0, 2024-09-19T08:58:09.362 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_sparse_read": 0, 2024-09-19T08:58:09.362 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_clonerange": 0, 2024-09-19T08:58:09.362 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_getxattr": 1, 2024-09-19T08:58:09.362 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_setxattr": 0, 2024-09-19T08:58:09.362 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_cmpxattr": 0, 2024-09-19T08:58:09.362 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_rmxattr": 0, 2024-09-19T08:58:09.362 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_resetxattrs": 0, 2024-09-19T08:58:09.362 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_call": 0, 2024-09-19T08:58:09.363 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_watch": 0, 2024-09-19T08:58:09.363 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_notify": 0, 2024-09-19T08:58:09.363 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T08:58:09.363 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls": 0, 2024-09-19T08:58:09.363 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls_filter": 0, 2024-09-19T08:58:09.363 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_other": 0, 2024-09-19T08:58:09.363 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_active": 0, 2024-09-19T08:58:09.363 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_send": 0, 2024-09-19T08:58:09.363 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_resend": 0, 2024-09-19T08:58:09.363 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_ping": 0, 2024-09-19T08:58:09.363 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_active": 0, 2024-09-19T08:58:09.363 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_send": 0, 2024-09-19T08:58:09.363 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_resend": 0, 2024-09-19T08:58:09.364 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_active": 0, 2024-09-19T08:58:09.364 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_send": 0, 2024-09-19T08:58:09.364 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_resend": 0, 2024-09-19T08:58:09.364 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_active": 0, 2024-09-19T08:58:09.364 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_send": 0, 2024-09-19T08:58:09.364 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_resend": 0, 2024-09-19T08:58:09.364 INFO:teuthology.orchestra.run.smithi173.stdout: "command_active": 0, 2024-09-19T08:58:09.364 INFO:teuthology.orchestra.run.smithi173.stdout: "command_send": 0, 2024-09-19T08:58:09.364 INFO:teuthology.orchestra.run.smithi173.stdout: "command_resend": 0, 2024-09-19T08:58:09.364 INFO:teuthology.orchestra.run.smithi173.stdout: "map_epoch": 96, 2024-09-19T08:58:09.364 INFO:teuthology.orchestra.run.smithi173.stdout: "map_full": 0, 2024-09-19T08:58:09.364 INFO:teuthology.orchestra.run.smithi173.stdout: "map_inc": 14, 2024-09-19T08:58:09.364 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_sessions": 5, 2024-09-19T08:58:09.365 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_open": 5, 2024-09-19T08:58:09.365 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_close": 0, 2024-09-19T08:58:09.365 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_laggy": 0, 2024-09-19T08:58:09.365 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_wr": 0, 2024-09-19T08:58:09.365 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_rd": 2, 2024-09-19T08:58:09.365 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_del": 0 2024-09-19T08:58:09.365 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.365 INFO:teuthology.orchestra.run.smithi173.stdout: "oft": { 2024-09-19T08:58:09.365 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_objs": 0, 2024-09-19T08:58:09.365 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_kv_pairs": 0, 2024-09-19T08:58:09.365 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_updates": 0, 2024-09-19T08:58:09.365 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_removes": 0 2024-09-19T08:58:09.365 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.366 INFO:teuthology.orchestra.run.smithi173.stdout: "osd-slow-ops": { 2024-09-19T08:58:09.366 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_ops_count": 0 2024-09-19T08:58:09.366 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.366 INFO:teuthology.orchestra.run.smithi173.stdout: "purge_queue": { 2024-09-19T08:58:09.366 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops": 0, 2024-09-19T08:58:09.366 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T08:58:09.366 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing": 0, 2024-09-19T08:58:09.366 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_high_water": 0, 2024-09-19T08:58:09.366 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed_ops": 0, 2024-09-19T08:58:09.366 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed": 0, 2024-09-19T08:58:09.366 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_item_in_journal": 0 2024-09-19T08:58:09.366 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.366 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T08:58:09.367 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-09-19T08:58:09.367 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-09-19T08:58:09.367 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-09-19T08:58:09.367 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 1939, 2024-09-19T08:58:09.367 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 1986043, 2024-09-19T08:58:09.367 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-09-19T08:58:09.367 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 1939, 2024-09-19T08:58:09.367 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-09-19T08:58:09.367 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-09-19T08:58:09.367 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 1939, 2024-09-19T08:58:09.367 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 1986043, 2024-09-19T08:58:09.367 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-09-19T08:58:09.367 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.367 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.368 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.368 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:09.368 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.368 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_bytes": { 2024-09-19T08:58:09.368 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-09-19T08:58:09.368 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-09-19T08:58:09.368 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-09-19T08:58:09.368 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-09-19T08:58:09.368 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-09-19T08:58:09.368 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-09-19T08:58:09.368 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-09-19T08:58:09.368 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 1658, 2024-09-19T08:58:09.368 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 593, 2024-09-19T08:58:09.369 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 1658, 2024-09-19T08:58:09.369 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 593, 2024-09-19T08:58:09.369 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-09-19T08:58:09.369 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.369 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:09 smithi173 ceph-mon[35306]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T08:58:09.369 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.370 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.370 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:09.370 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.370 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_ops": { 2024-09-19T08:58:09.370 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-09-19T08:58:09.370 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 1024, 2024-09-19T08:58:09.370 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-09-19T08:58:09.370 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-09-19T08:58:09.370 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-09-19T08:58:09.370 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-09-19T08:58:09.370 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-09-19T08:58:09.370 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 1658, 2024-09-19T08:58:09.370 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 1658, 2024-09-19T08:58:09.371 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 1658, 2024-09-19T08:58:09.371 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 1658, 2024-09-19T08:58:09.371 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-09-19T08:58:09.371 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.371 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.371 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.371 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:09.371 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.371 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle": { 2024-09-19T08:58:09.371 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-09-19T08:58:09.371 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2024-09-19T08:58:09.371 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-09-19T08:58:09.371 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-09-19T08:58:09.372 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-09-19T08:58:09.372 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-09-19T08:58:09.372 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-09-19T08:58:09.372 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-09-19T08:58:09.372 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-09-19T08:58:09.372 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 0, 2024-09-19T08:58:09.372 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 0, 2024-09-19T08:58:09.372 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-09-19T08:58:09.372 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.372 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.372 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.372 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:09.373 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-19T08:58:09.373 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle-0x5615f5814b40": { 2024-09-19T08:58:09.373 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-09-19T08:58:09.373 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2024-09-19T08:58:09.373 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-09-19T08:58:09.373 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-09-19T08:58:09.373 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-09-19T08:58:09.373 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-09-19T08:58:09.373 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-09-19T08:58:09.373 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-09-19T08:58:09.373 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-09-19T08:58:09.373 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 0, 2024-09-19T08:58:09.373 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 0, 2024-09-19T08:58:09.373 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-09-19T08:58:09.374 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-09-19T08:58:09.374 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-09-19T08:58:09.374 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-09-19T08:58:09.374 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:09.374 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-19T08:58:09.374 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-19T08:58:09.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:09 smithi170 ceph-mon[35072]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T08:58:10.065 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-19T08:58:10.065 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-19T08:58:10.065 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-09-19T08:58:10.065 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-09-19T08:58:10.065 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-09-19T08:58:10.065 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-09-19T08:58:10.065 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T08:58:10.065 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-19T08:58:10.077 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-19T08:58:10.077 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-19T08:58:10.077 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:58:10.106 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-19T08:58:10.107 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:58:10.166 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-19T08:58:10.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:10 smithi164 ceph-mon[30928]: pgmap v386: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T08:58:10.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:10 smithi170 ceph-mon[35072]: pgmap v386: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T08:58:10.410 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:10 smithi173 ceph-mon[35306]: pgmap v386: 97 pgs: 97 active+clean; 11 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T08:58:10.461 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:58:11.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.080+0000 7f96a1d31640 1 -- 172.21.15.164:0/435409447 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f969c1082e0 msgr2=0x7f969c10ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:11.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.080+0000 7f96a1d31640 1 --2- 172.21.15.164:0/435409447 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f969c1082e0 0x7f969c10ec50 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f9694009cd0 tx=0x7f969402a8e0 comp rx=0 tx=0).stop 2024-09-19T08:58:11.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.080+0000 7f96a1d31640 1 -- 172.21.15.164:0/435409447 shutdown_connections 2024-09-19T08:58:11.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.080+0000 7f96a1d31640 1 --2- 172.21.15.164:0/435409447 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f969c1082e0 0x7f969c10ec50 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:11.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.080+0000 7f96a1d31640 1 --2- 172.21.15.164:0/435409447 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f969c103760 0x7f969c107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:11.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.080+0000 7f96a1d31640 1 --2- 172.21.15.164:0/435409447 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f969c102db0 0x7f969c103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:11.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.081+0000 7f96a1d31640 1 -- 172.21.15.164:0/435409447 >> 172.21.15.164:0/435409447 conn(0x7f969c0fc7d0 msgr2=0x7f969c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:58:11.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.081+0000 7f96a1d31640 1 -- 172.21.15.164:0/435409447 shutdown_connections 2024-09-19T08:58:11.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.081+0000 7f96a1d31640 1 -- 172.21.15.164:0/435409447 wait complete. 2024-09-19T08:58:11.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.081+0000 7f96a1d31640 1 Processor -- start 2024-09-19T08:58:11.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.082+0000 7f96a1d31640 1 -- start start 2024-09-19T08:58:11.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.082+0000 7f96a1d31640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f969c102db0 0x7f969c198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:11.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.082+0000 7f96a1d31640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f969c103760 0x7f969c198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:11.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.082+0000 7f96a1d31640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f969c1082e0 0x7f969c19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:11.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.082+0000 7f96a1d31640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f969c1120b0 con 0x7f969c103760 2024-09-19T08:58:11.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.082+0000 7f96a1d31640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f969c111f30 con 0x7f969c102db0 2024-09-19T08:58:11.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.082+0000 7f96a1d31640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f969c112230 con 0x7f969c1082e0 2024-09-19T08:58:11.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.082+0000 7f969affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f969c103760 0x7f969c198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:11.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.082+0000 7f969affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f969c103760 0x7f969c198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60974/0 (socket says 172.21.15.164:60974) 2024-09-19T08:58:11.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.082+0000 7f969affd640 1 -- 172.21.15.164:0/1951465075 learned_addr learned my addr 172.21.15.164:0/1951465075 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:58:11.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.082+0000 7f969b7fe640 1 --2- 172.21.15.164:0/1951465075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f969c102db0 0x7f969c198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:11.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.082+0000 7f969bfff640 1 --2- 172.21.15.164:0/1951465075 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f969c1082e0 0x7f969c19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:11.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.082+0000 7f969b7fe640 1 -- 172.21.15.164:0/1951465075 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f969c1082e0 msgr2=0x7f969c19ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:11.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.082+0000 7f969b7fe640 1 --2- 172.21.15.164:0/1951465075 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f969c1082e0 0x7f969c19ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:11.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.082+0000 7f969b7fe640 1 -- 172.21.15.164:0/1951465075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f969c103760 msgr2=0x7f969c198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:11.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.082+0000 7f969b7fe640 1 --2- 172.21.15.164:0/1951465075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f969c103760 0x7f969c198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:11.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.082+0000 7f969b7fe640 1 -- 172.21.15.164:0/1951465075 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f969c19d500 con 0x7f969c102db0 2024-09-19T08:58:11.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.082+0000 7f969affd640 1 --2- 172.21.15.164:0/1951465075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f969c103760 0x7f969c198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:58:11.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.083+0000 7f969b7fe640 1 --2- 172.21.15.164:0/1951465075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f969c102db0 0x7f969c198550 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f968c00e9f0 tx=0x7f968c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:58:11.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.083+0000 7f9698ff9640 1 -- 172.21.15.164:0/1951465075 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c00ccc0 con 0x7f969c102db0 2024-09-19T08:58:11.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.083+0000 7f96a1d31640 1 -- 172.21.15.164:0/1951465075 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f969c19d7f0 con 0x7f969c102db0 2024-09-19T08:58:11.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.083+0000 7f9698ff9640 1 -- 172.21.15.164:0/1951465075 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f968c010910 con 0x7f969c102db0 2024-09-19T08:58:11.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.083+0000 7f9698ff9640 1 -- 172.21.15.164:0/1951465075 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c018960 con 0x7f969c102db0 2024-09-19T08:58:11.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.083+0000 7f96a1d31640 1 -- 172.21.15.164:0/1951465075 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f969c1a50d0 con 0x7f969c102db0 2024-09-19T08:58:11.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.084+0000 7f96a1d31640 1 -- 172.21.15.164:0/1951465075 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f969c103190 con 0x7f969c102db0 2024-09-19T08:58:11.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.085+0000 7f9698ff9640 1 -- 172.21.15.164:0/1951465075 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f968c010ac0 con 0x7f969c102db0 2024-09-19T08:58:11.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.085+0000 7f9698ff9640 1 --2- 172.21.15.164:0/1951465075 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f96840a33b0 0x7f96840a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:11.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.086+0000 7f969affd640 1 --2- 172.21.15.164:0/1951465075 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f96840a33b0 0x7f96840a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:11.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.086+0000 7f9698ff9640 1 -- 172.21.15.164:0/1951465075 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8499+0+0 (secure 0 0 0) 0x7f968c0d00a0 con 0x7f969c102db0 2024-09-19T08:58:11.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.086+0000 7f969affd640 1 --2- 172.21.15.164:0/1951465075 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f96840a33b0 0x7f96840a5870 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f969c0fed80 tx=0x7f9690005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:58:11.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.090+0000 7f9698ff9640 1 -- 172.21.15.164:0/1951465075 <== mon.2 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f968c09c440 con 0x7f969c102db0 2024-09-19T08:58:11.110 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:11 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:58:11.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.282+0000 7f96a1d31640 1 -- 172.21.15.164:0/1951465075 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f969c10a2c0 con 0x7f969c102db0 2024-09-19T08:58:11.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.282+0000 7f9698ff9640 1 -- 172.21.15.164:0/1951465075 <== mon.2 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v96) ==== 81+0+961 (secure 0 0 0) 0x7f968c0a0740 con 0x7f969c102db0 2024-09-19T08:58:11.283 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.164:0/3424843982 2024-09-20T08:45:47.960147+0000 2024-09-19T08:58:11.283 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.164:0/1369265436 2024-09-20T08:36:33.856642+0000 2024-09-19T08:58:11.283 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.164:0/1807120560 2024-09-20T08:45:47.960147+0000 2024-09-19T08:58:11.284 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.164:6801/1094864383 2024-09-20T08:45:47.960147+0000 2024-09-19T08:58:11.284 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.164:0/1905394497 2024-09-20T08:36:33.856642+0000 2024-09-19T08:58:11.284 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.164:6800/2805828009 2024-09-20T08:36:33.856642+0000 2024-09-19T08:58:11.284 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.164:0/2770797546 2024-09-20T08:36:33.856642+0000 2024-09-19T08:58:11.284 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.164:0/1837164919 2024-09-20T08:37:01.663065+0000 2024-09-19T08:58:11.284 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.164:6800/1173822158 2024-09-20T08:37:01.663065+0000 2024-09-19T08:58:11.284 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.164:6800/1094864383 2024-09-20T08:45:47.960147+0000 2024-09-19T08:58:11.284 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.164:6801/1173822158 2024-09-20T08:37:01.663065+0000 2024-09-19T08:58:11.284 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.164:0/339032808 2024-09-20T08:37:01.663065+0000 2024-09-19T08:58:11.284 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.164:6801/2805828009 2024-09-20T08:36:33.856642+0000 2024-09-19T08:58:11.284 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.164:0/2087067799 2024-09-20T08:45:47.960147+0000 2024-09-19T08:58:11.284 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.164:0/4018459143 2024-09-20T08:37:01.663065+0000 2024-09-19T08:58:11.284 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.164:0/2727626049 2024-09-20T08:45:47.960147+0000 2024-09-19T08:58:11.285 INFO:teuthology.orchestra.run.smithi164.stderr:listed 16 entries 2024-09-19T08:58:11.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.285+0000 7f96a1d31640 1 -- 172.21.15.164:0/1951465075 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f96840a33b0 msgr2=0x7f96840a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:11.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.285+0000 7f96a1d31640 1 --2- 172.21.15.164:0/1951465075 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f96840a33b0 0x7f96840a5870 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f969c0fed80 tx=0x7f9690005f90 comp rx=0 tx=0).stop 2024-09-19T08:58:11.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.285+0000 7f96a1d31640 1 -- 172.21.15.164:0/1951465075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f969c102db0 msgr2=0x7f969c198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:11.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.285+0000 7f96a1d31640 1 --2- 172.21.15.164:0/1951465075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f969c102db0 0x7f969c198550 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f968c00e9f0 tx=0x7f968c00eeb0 comp rx=0 tx=0).stop 2024-09-19T08:58:11.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.285+0000 7f96a1d31640 1 -- 172.21.15.164:0/1951465075 shutdown_connections 2024-09-19T08:58:11.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.285+0000 7f96a1d31640 1 --2- 172.21.15.164:0/1951465075 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f969c1082e0 0x7f969c19ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:11.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.285+0000 7f96a1d31640 1 --2- 172.21.15.164:0/1951465075 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f96840a33b0 0x7f96840a5870 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:11.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.285+0000 7f96a1d31640 1 --2- 172.21.15.164:0/1951465075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f969c103760 0x7f969c198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:11.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.285+0000 7f96a1d31640 1 --2- 172.21.15.164:0/1951465075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f969c102db0 0x7f969c198550 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:11.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.285+0000 7f96a1d31640 1 -- 172.21.15.164:0/1951465075 >> 172.21.15.164:0/1951465075 conn(0x7f969c0fc7d0 msgr2=0x7f969c106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:58:11.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.286+0000 7f96a1d31640 1 -- 172.21.15.164:0/1951465075 shutdown_connections 2024-09-19T08:58:11.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:11.286+0000 7f96a1d31640 1 -- 172.21.15.164:0/1951465075 wait complete. 2024-09-19T08:58:11.374 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:11 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:58:11.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:11 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T08:58:12.118 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-19T08:58:12.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:58:12.119 DEBUG:teuthology.orchestra.run.smithi164:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-19T08:58:12.201 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi164.front.sepia.ceph.com 2024-09-19T08:58:12.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:58:12.201 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T08:58:12.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:58:12.263 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-09-19T08:58:12.299 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T08:58:12.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:58:12.300 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:58:12.361 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:12 smithi164 ceph-mon[30928]: pgmap v387: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T08:58:12.362 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:12 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:58:12.362 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:12 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:58:12.362 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:12 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:58:12.362 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:12 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:58:12.362 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:12 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/1951465075' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T08:58:12.362 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:12 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:58:12.362 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:12 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:58:12.362 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:12 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:58:12.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:58:12.366 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link delete ceph-brx 2024-09-19T08:58:12.472 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-19T08:58:12.654 INFO:tasks.cephadm:Teardown begin 2024-09-19T08:58:12.655 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T08:58:12.655 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:58:12.656 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:12 smithi170 ceph-mon[35072]: pgmap v387: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T08:58:12.656 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:12 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:58:12.656 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:12 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:58:12.656 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:12 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:58:12.656 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:12 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:58:12.656 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:12 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/1951465075' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T08:58:12.656 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:12 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:58:12.657 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:12 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:58:12.657 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:12 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:58:12.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:12 smithi173 ceph-mon[35306]: pgmap v387: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T08:58:12.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:12 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:58:12.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:12 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:58:12.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:12 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:58:12.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:12 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:58:12.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:12 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/1951465075' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T08:58:12.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:12 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T08:58:12.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:12 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T08:58:12.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:12 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' 2024-09-19T08:58:12.910 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:58:13.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.593+0000 7ff11189a640 1 -- 172.21.15.164:0/1809567994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff10c102ef0 msgr2=0x7ff10c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:13.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.593+0000 7ff11189a640 1 --2- 172.21.15.164:0/1809567994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff10c102ef0 0x7ff10c10da40 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7ff104009980 tx=0x7ff10402c6c0 comp rx=0 tx=0).stop 2024-09-19T08:58:13.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.593+0000 7ff11189a640 1 -- 172.21.15.164:0/1809567994 shutdown_connections 2024-09-19T08:58:13.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.593+0000 7ff11189a640 1 --2- 172.21.15.164:0/1809567994 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff10c102ef0 0x7ff10c10da40 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:13.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.593+0000 7ff11189a640 1 --2- 172.21.15.164:0/1809567994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff10c102570 0x7ff10c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:13.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.593+0000 7ff11189a640 1 --2- 172.21.15.164:0/1809567994 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff10c100e20 0x7ff10c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:13.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.593+0000 7ff11189a640 1 -- 172.21.15.164:0/1809567994 >> 172.21.15.164:0/1809567994 conn(0x7ff10c0fc7d0 msgr2=0x7ff10c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:58:13.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.593+0000 7ff11189a640 1 -- 172.21.15.164:0/1809567994 shutdown_connections 2024-09-19T08:58:13.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.593+0000 7ff11189a640 1 -- 172.21.15.164:0/1809567994 wait complete. 2024-09-19T08:58:13.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.594+0000 7ff11189a640 1 Processor -- start 2024-09-19T08:58:13.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.594+0000 7ff11189a640 1 -- start start 2024-09-19T08:58:13.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.594+0000 7ff11189a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff10c100e20 0x7ff10c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:13.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.594+0000 7ff11189a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff10c102570 0x7ff10c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:13.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.594+0000 7ff11189a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff10c102ef0 0x7ff10c19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:13.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.594+0000 7ff11189a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff10c10fe80 con 0x7ff10c100e20 2024-09-19T08:58:13.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.594+0000 7ff11189a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff10c10fd00 con 0x7ff10c102570 2024-09-19T08:58:13.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.594+0000 7ff11189a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff10c110000 con 0x7ff10c102ef0 2024-09-19T08:58:13.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.594+0000 7ff10affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff10c100e20 0x7ff10c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:13.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.594+0000 7ff10affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff10c100e20 0x7ff10c19a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39490/0 (socket says 172.21.15.164:39490) 2024-09-19T08:58:13.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.594+0000 7ff10affd640 1 -- 172.21.15.164:0/3878591960 learned_addr learned my addr 172.21.15.164:0/3878591960 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:58:13.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.594+0000 7ff10a7fc640 1 --2- 172.21.15.164:0/3878591960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff10c102570 0x7ff10c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:13.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.594+0000 7ff10b7fe640 1 --2- 172.21.15.164:0/3878591960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff10c102ef0 0x7ff10c19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:13.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.594+0000 7ff10affd640 1 -- 172.21.15.164:0/3878591960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff10c102ef0 msgr2=0x7ff10c19eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:13.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.594+0000 7ff10affd640 1 --2- 172.21.15.164:0/3878591960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff10c102ef0 0x7ff10c19eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:13.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.594+0000 7ff10affd640 1 -- 172.21.15.164:0/3878591960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff10c102570 msgr2=0x7ff10c19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:13.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.594+0000 7ff10affd640 1 --2- 172.21.15.164:0/3878591960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff10c102570 0x7ff10c19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:13.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.595+0000 7ff10affd640 1 -- 172.21.15.164:0/3878591960 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff10c19f5a0 con 0x7ff10c100e20 2024-09-19T08:58:13.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.595+0000 7ff10b7fe640 1 --2- 172.21.15.164:0/3878591960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff10c102ef0 0x7ff10c19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:58:13.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.595+0000 7ff10affd640 1 --2- 172.21.15.164:0/3878591960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff10c100e20 0x7ff10c19a5f0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7ff10000d770 tx=0x7ff10000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:58:13.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.595+0000 7ff110898640 1 -- 172.21.15.164:0/3878591960 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff100017070 con 0x7ff10c100e20 2024-09-19T08:58:13.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.595+0000 7ff11189a640 1 -- 172.21.15.164:0/3878591960 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff10c19f890 con 0x7ff10c100e20 2024-09-19T08:58:13.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.595+0000 7ff110898640 1 -- 172.21.15.164:0/3878591960 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff100005500 con 0x7ff10c100e20 2024-09-19T08:58:13.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.595+0000 7ff110898640 1 -- 172.21.15.164:0/3878591960 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff100004e00 con 0x7ff10c100e20 2024-09-19T08:58:13.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.595+0000 7ff11189a640 1 -- 172.21.15.164:0/3878591960 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff10c1a7170 con 0x7ff10c100e20 2024-09-19T08:58:13.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.596+0000 7ff11189a640 1 -- 172.21.15.164:0/3878591960 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff10c101200 con 0x7ff10c100e20 2024-09-19T08:58:13.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.597+0000 7ff110898640 1 -- 172.21.15.164:0/3878591960 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7ff100004fa0 con 0x7ff10c100e20 2024-09-19T08:58:13.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.598+0000 7ff110898640 1 --2- 172.21.15.164:0/3878591960 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff0f40a33b0 0x7ff0f40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:13.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.598+0000 7ff10a7fc640 1 --2- 172.21.15.164:0/3878591960 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff0f40a33b0 0x7ff0f40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:13.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.598+0000 7ff110898640 1 -- 172.21.15.164:0/3878591960 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8499+0+0 (secure 0 0 0) 0x7ff1000cfc70 con 0x7ff10c100e20 2024-09-19T08:58:13.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.598+0000 7ff10a7fc640 1 --2- 172.21.15.164:0/3878591960 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff0f40a33b0 0x7ff0f40a5870 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc007350 tx=0x7ff0fc0072e0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:58:13.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.601+0000 7ff110898640 1 -- 172.21.15.164:0/3878591960 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff10009c660 con 0x7ff10c100e20 2024-09-19T08:58:13.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.803+0000 7ff11189a640 1 -- 172.21.15.164:0/3878591960 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff10c1a7340 con 0x7ff10c100e20 2024-09-19T08:58:13.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.804+0000 7ff110898640 1 -- 172.21.15.164:0/3878591960 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11985 (secure 0 0 0) 0x7ff10009bf60 con 0x7ff10c100e20 2024-09-19T08:58:13.803 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:58:13.804 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":33,"btime":"2024-09-19T08:57:15:914589+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:57:15.844000+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24442,24424,24449,14595]},"id":1}]} 2024-09-19T08:58:13.804 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 33 2024-09-19T08:58:13.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.806+0000 7ff11189a640 1 -- 172.21.15.164:0/3878591960 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff0f40a33b0 msgr2=0x7ff0f40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:13.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.806+0000 7ff11189a640 1 --2- 172.21.15.164:0/3878591960 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff0f40a33b0 0x7ff0f40a5870 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc007350 tx=0x7ff0fc0072e0 comp rx=0 tx=0).stop 2024-09-19T08:58:13.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.806+0000 7ff11189a640 1 -- 172.21.15.164:0/3878591960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff10c100e20 msgr2=0x7ff10c19a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:13.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.806+0000 7ff11189a640 1 --2- 172.21.15.164:0/3878591960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff10c100e20 0x7ff10c19a5f0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7ff10000d770 tx=0x7ff10000dc30 comp rx=0 tx=0).stop 2024-09-19T08:58:13.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.806+0000 7ff11189a640 1 -- 172.21.15.164:0/3878591960 shutdown_connections 2024-09-19T08:58:13.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.806+0000 7ff11189a640 1 --2- 172.21.15.164:0/3878591960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff10c102ef0 0x7ff10c19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:13.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.806+0000 7ff11189a640 1 --2- 172.21.15.164:0/3878591960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff10c102570 0x7ff10c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:13.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.806+0000 7ff11189a640 1 --2- 172.21.15.164:0/3878591960 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff0f40a33b0 0x7ff0f40a5870 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:13.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.806+0000 7ff11189a640 1 --2- 172.21.15.164:0/3878591960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff10c100e20 0x7ff10c19a5f0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:13.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.806+0000 7ff11189a640 1 -- 172.21.15.164:0/3878591960 >> 172.21.15.164:0/3878591960 conn(0x7ff10c0fc7d0 msgr2=0x7ff10c106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:58:13.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.806+0000 7ff11189a640 1 -- 172.21.15.164:0/3878591960 shutdown_connections 2024-09-19T08:58:13.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:13.806+0000 7ff11189a640 1 -- 172.21.15.164:0/3878591960 wait complete. 2024-09-19T08:58:14.446 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:14 smithi164 ceph-mon[30928]: pgmap v388: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T08:58:14.446 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:14 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3878591960' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:58:14.473 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T08:58:14.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:14 smithi170 ceph-mon[35072]: pgmap v388: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T08:58:14.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:14 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3878591960' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:58:14.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:14 smithi173 ceph-mon[35306]: pgmap v388: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T08:58:14.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:14 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3878591960' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:58:14.721 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:58:15.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.212+0000 7f31caa15640 1 -- 172.21.15.164:0/677705501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31c4101de0 msgr2=0x7f31c41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:15.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.212+0000 7f31caa15640 1 --2- 172.21.15.164:0/677705501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31c4101de0 0x7f31c41021c0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f31b4009900 tx=0x7f31b402a410 comp rx=0 tx=0).stop 2024-09-19T08:58:15.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.213+0000 7f31caa15640 1 -- 172.21.15.164:0/677705501 shutdown_connections 2024-09-19T08:58:15.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.213+0000 7f31caa15640 1 --2- 172.21.15.164:0/677705501 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31c4106f10 0x7f31c410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:15.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.213+0000 7f31caa15640 1 --2- 172.21.15.164:0/677705501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31c4102790 0x7f31c41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:15.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.213+0000 7f31caa15640 1 --2- 172.21.15.164:0/677705501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31c4101de0 0x7f31c41021c0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:15.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.213+0000 7f31caa15640 1 -- 172.21.15.164:0/677705501 >> 172.21.15.164:0/677705501 conn(0x7f31c40fd8c0 msgr2=0x7f31c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:58:15.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.213+0000 7f31caa15640 1 -- 172.21.15.164:0/677705501 shutdown_connections 2024-09-19T08:58:15.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.213+0000 7f31caa15640 1 -- 172.21.15.164:0/677705501 wait complete. 2024-09-19T08:58:15.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.213+0000 7f31caa15640 1 Processor -- start 2024-09-19T08:58:15.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.213+0000 7f31caa15640 1 -- start start 2024-09-19T08:58:15.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.214+0000 7f31caa15640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31c4101de0 0x7f31c41985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:15.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.214+0000 7f31caa15640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31c4102790 0x7f31c4198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:15.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.214+0000 7f31caa15640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31c4106f10 0x7f31c419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:15.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.214+0000 7f31caa15640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f31c4112060 con 0x7f31c4106f10 2024-09-19T08:58:15.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.214+0000 7f31caa15640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f31c4111ee0 con 0x7f31c4102790 2024-09-19T08:58:15.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.214+0000 7f31caa15640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f31c41121e0 con 0x7f31c4101de0 2024-09-19T08:58:15.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.214+0000 7f31c8fb2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31c4106f10 0x7f31c419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:15.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.214+0000 7f31c8fb2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31c4106f10 0x7f31c419ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39506/0 (socket says 172.21.15.164:39506) 2024-09-19T08:58:15.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.214+0000 7f31c8fb2640 1 -- 172.21.15.164:0/4221454565 learned_addr learned my addr 172.21.15.164:0/4221454565 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:58:15.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.214+0000 7f31c37fe640 1 --2- 172.21.15.164:0/4221454565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31c4102790 0x7f31c4198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:15.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.214+0000 7f31c3fff640 1 --2- 172.21.15.164:0/4221454565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31c4101de0 0x7f31c41985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:15.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.214+0000 7f31c8fb2640 1 -- 172.21.15.164:0/4221454565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31c4101de0 msgr2=0x7f31c41985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:15.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.214+0000 7f31c8fb2640 1 --2- 172.21.15.164:0/4221454565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31c4101de0 0x7f31c41985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:15.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.214+0000 7f31c8fb2640 1 -- 172.21.15.164:0/4221454565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31c4102790 msgr2=0x7f31c4198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:15.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.214+0000 7f31c8fb2640 1 --2- 172.21.15.164:0/4221454565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31c4102790 0x7f31c4198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:15.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.214+0000 7f31c8fb2640 1 -- 172.21.15.164:0/4221454565 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c419d570 con 0x7f31c4106f10 2024-09-19T08:58:15.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.214+0000 7f31c8fb2640 1 --2- 172.21.15.164:0/4221454565 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31c4106f10 0x7f31c419ce90 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f31bc00b970 tx=0x7f31bc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:58:15.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.214+0000 7f31c17fa640 1 -- 172.21.15.164:0/4221454565 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31bc019070 con 0x7f31c4106f10 2024-09-19T08:58:15.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.214+0000 7f31caa15640 1 -- 172.21.15.164:0/4221454565 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31c419d860 con 0x7f31c4106f10 2024-09-19T08:58:15.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.214+0000 7f31c17fa640 1 -- 172.21.15.164:0/4221454565 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31bc014440 con 0x7f31c4106f10 2024-09-19T08:58:15.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.214+0000 7f31c17fa640 1 -- 172.21.15.164:0/4221454565 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31bc0075d0 con 0x7f31c4106f10 2024-09-19T08:58:15.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.214+0000 7f31caa15640 1 -- 172.21.15.164:0/4221454565 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31c41a5140 con 0x7f31c4106f10 2024-09-19T08:58:15.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.215+0000 7f31caa15640 1 -- 172.21.15.164:0/4221454565 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31c4102240 con 0x7f31c4106f10 2024-09-19T08:58:15.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.216+0000 7f31c17fa640 1 -- 172.21.15.164:0/4221454565 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f31bc01c070 con 0x7f31c4106f10 2024-09-19T08:58:15.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.217+0000 7f31c17fa640 1 --2- 172.21.15.164:0/4221454565 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f31b00a33b0 0x7f31b00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:15.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.217+0000 7f31c3fff640 1 --2- 172.21.15.164:0/4221454565 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f31b00a33b0 0x7f31b00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:15.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.217+0000 7f31c17fa640 1 -- 172.21.15.164:0/4221454565 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8499+0+0 (secure 0 0 0) 0x7f31bc0cf1d0 con 0x7f31c4106f10 2024-09-19T08:58:15.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.217+0000 7f31c3fff640 1 --2- 172.21.15.164:0/4221454565 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f31b00a33b0 0x7f31b00a5870 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f31b4000c00 tx=0x7f31b4005a20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:58:15.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.221+0000 7f31c17fa640 1 -- 172.21.15.164:0/4221454565 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f31bc09bcd0 con 0x7f31c4106f10 2024-09-19T08:58:15.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.423+0000 7f31caa15640 1 -- 172.21.15.164:0/4221454565 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31c410a150 con 0x7f31c4106f10 2024-09-19T08:58:15.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.424+0000 7f31c17fa640 1 -- 172.21.15.164:0/4221454565 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11985 (secure 0 0 0) 0x7f31bc09b5d0 con 0x7f31c4106f10 2024-09-19T08:58:15.424 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:58:15.424 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":33,"btime":"2024-09-19T08:57:15:914589+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3695235648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3695235648},{"type":"v1","addr":"172.21.15.164:6835","nonce":3695235648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/199930574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":199930574},{"type":"v1","addr":"172.21.15.164:6837","nonce":199930574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T08:43:53.386044+0000","modified":"2024-09-19T08:57:15.844000+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24449,"mds_2":24442,"mds_3":24424,"mds_4":14595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6839/538276454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":538276454},{"type":"v1","addr":"172.21.15.164:6839","nonce":538276454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.164:6841/2221659234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6840","nonce":2221659234},{"type":"v1","addr":"172.21.15.164:6841","nonce":2221659234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2726071671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2726071671},{"type":"v1","addr":"172.21.15.170:6833","nonce":2726071671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/486946900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":486946900},{"type":"v1","addr":"172.21.15.173:6833","nonce":486946900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/947317705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":947317705},{"type":"v1","addr":"172.21.15.173:6835","nonce":947317705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/321254273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":321254273},{"type":"v1","addr":"172.21.15.170:6835","nonce":321254273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.173:6837/2431090203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2431090203},{"type":"v1","addr":"172.21.15.173:6837","nonce":2431090203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.173:6839/1089658493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":1089658493},{"type":"v1","addr":"172.21.15.173:6839","nonce":1089658493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6837/2350302392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2350302392},{"type":"v1","addr":"172.21.15.170:6837","nonce":2350302392}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6839/1854887138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1854887138},{"type":"v1","addr":"172.21.15.170:6839","nonce":1854887138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24442,24424,24449,14595]},"id":1}]} 2024-09-19T08:58:15.424 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 33 2024-09-19T08:58:15.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.426+0000 7f31caa15640 1 -- 172.21.15.164:0/4221454565 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f31b00a33b0 msgr2=0x7f31b00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:15.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.426+0000 7f31caa15640 1 --2- 172.21.15.164:0/4221454565 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f31b00a33b0 0x7f31b00a5870 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f31b4000c00 tx=0x7f31b4005a20 comp rx=0 tx=0).stop 2024-09-19T08:58:15.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.426+0000 7f31caa15640 1 -- 172.21.15.164:0/4221454565 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31c4106f10 msgr2=0x7f31c419ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:15.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.426+0000 7f31caa15640 1 --2- 172.21.15.164:0/4221454565 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31c4106f10 0x7f31c419ce90 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f31bc00b970 tx=0x7f31bc00bd70 comp rx=0 tx=0).stop 2024-09-19T08:58:15.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.426+0000 7f31caa15640 1 -- 172.21.15.164:0/4221454565 shutdown_connections 2024-09-19T08:58:15.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.427+0000 7f31caa15640 1 --2- 172.21.15.164:0/4221454565 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f31b00a33b0 0x7f31b00a5870 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:15.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.427+0000 7f31caa15640 1 --2- 172.21.15.164:0/4221454565 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31c4106f10 0x7f31c419ce90 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:15.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.427+0000 7f31caa15640 1 --2- 172.21.15.164:0/4221454565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31c4102790 0x7f31c4198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:15.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.427+0000 7f31caa15640 1 --2- 172.21.15.164:0/4221454565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31c4101de0 0x7f31c41985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:15.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.427+0000 7f31caa15640 1 -- 172.21.15.164:0/4221454565 >> 172.21.15.164:0/4221454565 conn(0x7f31c40fd8c0 msgr2=0x7f31c40fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:58:15.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.427+0000 7f31caa15640 1 -- 172.21.15.164:0/4221454565 shutdown_connections 2024-09-19T08:58:15.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:15.427+0000 7f31caa15640 1 -- 172.21.15.164:0/4221454565 wait complete. 2024-09-19T08:58:16.480 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T08:58:16.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:16 smithi164 ceph-mon[30928]: pgmap v389: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 28 op/s 2024-09-19T08:58:16.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:16 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/4221454565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:58:16.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:16 smithi170 ceph-mon[35072]: pgmap v389: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 28 op/s 2024-09-19T08:58:16.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:16 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/4221454565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:58:16.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:16 smithi173 ceph-mon[35306]: pgmap v389: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 28 op/s 2024-09-19T08:58:16.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:16 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/4221454565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T08:58:16.731 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:58:17.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.289+0000 7fa883a8d640 1 -- 172.21.15.164:0/996073486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa87c100a80 msgr2=0x7fa87c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:17.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.289+0000 7fa883a8d640 1 --2- 172.21.15.164:0/996073486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa87c100a80 0x7fa87c100ee0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009cd0 tx=0x7fa86c02a8e0 comp rx=0 tx=0).stop 2024-09-19T08:58:17.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.289+0000 7fa883a8d640 1 -- 172.21.15.164:0/996073486 shutdown_connections 2024-09-19T08:58:17.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.289+0000 7fa883a8d640 1 --2- 172.21.15.164:0/996073486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa87c101420 0x7fa87c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:17.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.289+0000 7fa883a8d640 1 --2- 172.21.15.164:0/996073486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa87c100a80 0x7fa87c100ee0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:17.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.289+0000 7fa883a8d640 1 --2- 172.21.15.164:0/996073486 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa87c106a80 0x7fa87c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:17.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.289+0000 7fa883a8d640 1 -- 172.21.15.164:0/996073486 >> 172.21.15.164:0/996073486 conn(0x7fa87c0fc7a0 msgr2=0x7fa87c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:58:17.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.289+0000 7fa883a8d640 1 -- 172.21.15.164:0/996073486 shutdown_connections 2024-09-19T08:58:17.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.289+0000 7fa883a8d640 1 -- 172.21.15.164:0/996073486 wait complete. 2024-09-19T08:58:17.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.289+0000 7fa883a8d640 1 Processor -- start 2024-09-19T08:58:17.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.289+0000 7fa883a8d640 1 -- start start 2024-09-19T08:58:17.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.290+0000 7fa883a8d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa87c100a80 0x7fa87c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:17.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.290+0000 7fa883a8d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa87c101420 0x7fa87c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:17.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.290+0000 7fa883a8d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa87c106a80 0x7fa87c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:17.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.290+0000 7fa883a8d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c110e80 con 0x7fa87c100a80 2024-09-19T08:58:17.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.290+0000 7fa883a8d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c110d00 con 0x7fa87c101420 2024-09-19T08:58:17.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.290+0000 7fa883a8d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c111000 con 0x7fa87c106a80 2024-09-19T08:58:17.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.290+0000 7fa881829640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa87c100a80 0x7fa87c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:17.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.290+0000 7fa881829640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa87c100a80 0x7fa87c198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39534/0 (socket says 172.21.15.164:39534) 2024-09-19T08:58:17.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.290+0000 7fa881829640 1 -- 172.21.15.164:0/3882112011 learned_addr learned my addr 172.21.15.164:0/3882112011 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:58:17.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.290+0000 7fa88202a640 1 --2- 172.21.15.164:0/3882112011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa87c106a80 0x7fa87c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:17.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.290+0000 7fa881028640 1 --2- 172.21.15.164:0/3882112011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa87c101420 0x7fa87c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:17.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.290+0000 7fa881829640 1 -- 172.21.15.164:0/3882112011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa87c106a80 msgr2=0x7fa87c19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:17.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.290+0000 7fa881829640 1 --2- 172.21.15.164:0/3882112011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa87c106a80 0x7fa87c19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:17.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.290+0000 7fa881829640 1 -- 172.21.15.164:0/3882112011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa87c101420 msgr2=0x7fa87c1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:17.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.290+0000 7fa881829640 1 --2- 172.21.15.164:0/3882112011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa87c101420 0x7fa87c1989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:17.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.290+0000 7fa881829640 1 -- 172.21.15.164:0/3882112011 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87c19d440 con 0x7fa87c100a80 2024-09-19T08:58:17.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.290+0000 7fa88202a640 1 --2- 172.21.15.164:0/3882112011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa87c106a80 0x7fa87c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:58:17.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.290+0000 7fa881829640 1 --2- 172.21.15.164:0/3882112011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa87c100a80 0x7fa87c198490 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fa87000c890 tx=0x7fa87000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:58:17.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.290+0000 7fa87affd640 1 -- 172.21.15.164:0/3882112011 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa870004220 con 0x7fa87c100a80 2024-09-19T08:58:17.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.290+0000 7fa883a8d640 1 -- 172.21.15.164:0/3882112011 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa87c19d730 con 0x7fa87c100a80 2024-09-19T08:58:17.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.290+0000 7fa87affd640 1 -- 172.21.15.164:0/3882112011 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8700043c0 con 0x7fa87c100a80 2024-09-19T08:58:17.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.290+0000 7fa87affd640 1 -- 172.21.15.164:0/3882112011 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8700049e0 con 0x7fa87c100a80 2024-09-19T08:58:17.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.290+0000 7fa883a8d640 1 -- 172.21.15.164:0/3882112011 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa87c1a5010 con 0x7fa87c100a80 2024-09-19T08:58:17.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.291+0000 7fa883a8d640 1 -- 172.21.15.164:0/3882112011 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa87c100ee0 con 0x7fa87c100a80 2024-09-19T08:58:17.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.293+0000 7fa87affd640 1 -- 172.21.15.164:0/3882112011 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fa8700079b0 con 0x7fa87c100a80 2024-09-19T08:58:17.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.293+0000 7fa87affd640 1 --2- 172.21.15.164:0/3882112011 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa8640a33b0 0x7fa8640a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:17.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.293+0000 7fa881028640 1 --2- 172.21.15.164:0/3882112011 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa8640a33b0 0x7fa8640a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:17.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.293+0000 7fa87affd640 1 -- 172.21.15.164:0/3882112011 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8499+0+0 (secure 0 0 0) 0x7fa8700cffa0 con 0x7fa87c100a80 2024-09-19T08:58:17.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.294+0000 7fa881028640 1 --2- 172.21.15.164:0/3882112011 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa8640a33b0 0x7fa8640a5870 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa86c006010 tx=0x7fa86c000f40 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:58:17.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.296+0000 7fa87affd640 1 -- 172.21.15.164:0/3882112011 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa870014030 con 0x7fa87c100a80 2024-09-19T08:58:17.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.465+0000 7fa883a8d640 1 -- 172.21.15.164:0/3882112011 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa87c1997f0 con 0x7fa87c100a80 2024-09-19T08:58:17.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.467+0000 7fa87affd640 1 -- 172.21.15.164:0/3882112011 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+23243 (secure 0 0 0) 0x7fa87009c990 con 0x7fa87c100a80 2024-09-19T08:58:17.468 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-19T08:58:17.468 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":96,"fsid":"1e4b0d74-7662-11ef-bceb-c7b262605968","created":"2024-09-19T08:36:09.564509+0000","modified":"2024-09-19T08:58:08.011081+0000","last_up_change":"2024-09-19T08:43:01.260852+0000","last_in_change":"2024-09-19T08:42:37.492464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:39:13.961749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:43:49.613471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T08:43:51.570675+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":21,"snap_epoch":96,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33b11f37-6ac5-41fc-8cfd-1b08c7eef688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6803","nonce":663038243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6805","nonce":663038243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6809","nonce":663038243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":663038243},{"type":"v1","addr":"172.21.15.164:6807","nonce":663038243}]},"public_addr":"172.21.15.164:6803/663038243","cluster_addr":"172.21.15.164:6805/663038243","heartbeat_back_addr":"172.21.15.164:6809/663038243","heartbeat_front_addr":"172.21.15.164:6807/663038243","state":["exists","up"]},{"osd":1,"uuid":"6de64b0a-46e2-4a98-8005-a326a64bc87d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6801","nonce":2167066371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6803","nonce":2167066371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6807","nonce":2167066371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2167066371},{"type":"v1","addr":"172.21.15.170:6805","nonce":2167066371}]},"public_addr":"172.21.15.170:6801/2167066371","cluster_addr":"172.21.15.170:6803/2167066371","heartbeat_back_addr":"172.21.15.170:6807/2167066371","heartbeat_front_addr":"172.21.15.170:6805/2167066371","state":["exists","up"]},{"osd":2,"uuid":"d4158cc7-9822-42a7-9601-d65089907584","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6801","nonce":1646561048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6803","nonce":1646561048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6807","nonce":1646561048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1646561048},{"type":"v1","addr":"172.21.15.173:6805","nonce":1646561048}]},"public_addr":"172.21.15.173:6801/1646561048","cluster_addr":"172.21.15.173:6803/1646561048","heartbeat_back_addr":"172.21.15.173:6807/1646561048","heartbeat_front_addr":"172.21.15.173:6805/1646561048","state":["exists","up"]},{"osd":3,"uuid":"0cf3f7f1-3a8c-42ce-88a9-9bea52e02221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6811","nonce":406795560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6813","nonce":406795560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6817","nonce":406795560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":406795560},{"type":"v1","addr":"172.21.15.164:6815","nonce":406795560}]},"public_addr":"172.21.15.164:6811/406795560","cluster_addr":"172.21.15.164:6813/406795560","heartbeat_back_addr":"172.21.15.164:6817/406795560","heartbeat_front_addr":"172.21.15.164:6815/406795560","state":["exists","up"]},{"osd":4,"uuid":"6dca048a-2860-4a02-8d8e-450b30bc1a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6809","nonce":4120832165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6811","nonce":4120832165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6815","nonce":4120832165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":4120832165},{"type":"v1","addr":"172.21.15.170:6813","nonce":4120832165}]},"public_addr":"172.21.15.170:6809/4120832165","cluster_addr":"172.21.15.170:6811/4120832165","heartbeat_back_addr":"172.21.15.170:6815/4120832165","heartbeat_front_addr":"172.21.15.170:6813/4120832165","state":["exists","up"]},{"osd":5,"uuid":"3822521f-69d6-4141-a83f-5fc9cecc7b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6809","nonce":2914653552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6811","nonce":2914653552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6815","nonce":2914653552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2914653552},{"type":"v1","addr":"172.21.15.173:6813","nonce":2914653552}]},"public_addr":"172.21.15.173:6809/2914653552","cluster_addr":"172.21.15.173:6811/2914653552","heartbeat_back_addr":"172.21.15.173:6815/2914653552","heartbeat_front_addr":"172.21.15.173:6813/2914653552","state":["exists","up"]},{"osd":6,"uuid":"0f5a965e-2eb2-4c24-b089-577e0639c160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6819","nonce":1138341393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6821","nonce":1138341393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6825","nonce":1138341393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1138341393},{"type":"v1","addr":"172.21.15.164:6823","nonce":1138341393}]},"public_addr":"172.21.15.164:6819/1138341393","cluster_addr":"172.21.15.164:6821/1138341393","heartbeat_back_addr":"172.21.15.164:6825/1138341393","heartbeat_front_addr":"172.21.15.164:6823/1138341393","state":["exists","up"]},{"osd":7,"uuid":"f16cb2c3-639b-46ae-a740-1bd28ca1863c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6817","nonce":2124160360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6819","nonce":2124160360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6823","nonce":2124160360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":2124160360},{"type":"v1","addr":"172.21.15.170:6821","nonce":2124160360}]},"public_addr":"172.21.15.170:6817/2124160360","cluster_addr":"172.21.15.170:6819/2124160360","heartbeat_back_addr":"172.21.15.170:6823/2124160360","heartbeat_front_addr":"172.21.15.170:6821/2124160360","state":["exists","up"]},{"osd":8,"uuid":"161cff01-46ad-48e3-b999-a5473a2ad618","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6817","nonce":2285175061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6819","nonce":2285175061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6823","nonce":2285175061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2285175061},{"type":"v1","addr":"172.21.15.173:6821","nonce":2285175061}]},"public_addr":"172.21.15.173:6817/2285175061","cluster_addr":"172.21.15.173:6819/2285175061","heartbeat_back_addr":"172.21.15.173:6823/2285175061","heartbeat_front_addr":"172.21.15.173:6821/2285175061","state":["exists","up"]},{"osd":9,"uuid":"994666e8-8fbd-4231-ac67-81a79aaf7c2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6827","nonce":3918847952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6829","nonce":3918847952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6833","nonce":3918847952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3918847952},{"type":"v1","addr":"172.21.15.164:6831","nonce":3918847952}]},"public_addr":"172.21.15.164:6827/3918847952","cluster_addr":"172.21.15.164:6829/3918847952","heartbeat_back_addr":"172.21.15.164:6833/3918847952","heartbeat_front_addr":"172.21.15.164:6831/3918847952","state":["exists","up"]},{"osd":10,"uuid":"ff5e9099-cdb4-4905-b888-5b67320ef76b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6825","nonce":2707848227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6827","nonce":2707848227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6831","nonce":2707848227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2707848227},{"type":"v1","addr":"172.21.15.170:6829","nonce":2707848227}]},"public_addr":"172.21.15.170:6825/2707848227","cluster_addr":"172.21.15.170:6827/2707848227","heartbeat_back_addr":"172.21.15.170:6831/2707848227","heartbeat_front_addr":"172.21.15.170:6829/2707848227","state":["exists","up"]},{"osd":11,"uuid":"4d93088f-e791-4938-a207-12f15b56abd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6825","nonce":2831924738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6827","nonce":2831924738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6831","nonce":2831924738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2831924738},{"type":"v1","addr":"172.21.15.173:6829","nonce":2831924738}]},"public_addr":"172.21.15.173:6825/2831924738","cluster_addr":"172.21.15.173:6827/2831924738","heartbeat_back_addr":"172.21.15.173:6831/2831924738","heartbeat_front_addr":"172.21.15.173:6829/2831924738","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:21.787352+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:46.012323+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:39:11.353040+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:39:37.015124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:02.063379+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:27.892865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:40:53.372041+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:41:17.855284+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:41:43.389882+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:08.956895+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:34.069779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:42:59.303897+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.164:0/3424843982":"2024-09-20T08:45:47.960147+0000","172.21.15.164:0/1369265436":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1807120560":"2024-09-20T08:45:47.960147+0000","172.21.15.164:6801/1094864383":"2024-09-20T08:45:47.960147+0000","172.21.15.164:0/1905394497":"2024-09-20T08:36:33.856642+0000","172.21.15.164:6800/2805828009":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/2770797546":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/1837164919":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6800/1173822158":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6800/1094864383":"2024-09-20T08:45:47.960147+0000","172.21.15.164:6801/1173822158":"2024-09-20T08:37:01.663065+0000","172.21.15.164:0/339032808":"2024-09-20T08:37:01.663065+0000","172.21.15.164:6801/2805828009":"2024-09-20T08:36:33.856642+0000","172.21.15.164:0/2087067799":"2024-09-20T08:45:47.960147+0000","172.21.15.164:0/4018459143":"2024-09-20T08:37:01.663065+0000","172.21.15.164:0/2727626049":"2024-09-20T08:45:47.960147+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":8,"length":1},{"begin":21,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":8,"length":1},{"begin":21,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:58:17.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.470+0000 7fa883a8d640 1 -- 172.21.15.164:0/3882112011 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa8640a33b0 msgr2=0x7fa8640a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:17.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.470+0000 7fa883a8d640 1 --2- 172.21.15.164:0/3882112011 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa8640a33b0 0x7fa8640a5870 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa86c006010 tx=0x7fa86c000f40 comp rx=0 tx=0).stop 2024-09-19T08:58:17.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.470+0000 7fa883a8d640 1 -- 172.21.15.164:0/3882112011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa87c100a80 msgr2=0x7fa87c198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:17.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.470+0000 7fa883a8d640 1 --2- 172.21.15.164:0/3882112011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa87c100a80 0x7fa87c198490 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fa87000c890 tx=0x7fa87000cd50 comp rx=0 tx=0).stop 2024-09-19T08:58:17.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.470+0000 7fa883a8d640 1 -- 172.21.15.164:0/3882112011 shutdown_connections 2024-09-19T08:58:17.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.470+0000 7fa883a8d640 1 --2- 172.21.15.164:0/3882112011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa87c106a80 0x7fa87c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:17.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.470+0000 7fa883a8d640 1 --2- 172.21.15.164:0/3882112011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa87c101420 0x7fa87c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:17.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.470+0000 7fa883a8d640 1 --2- 172.21.15.164:0/3882112011 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fa8640a33b0 0x7fa8640a5870 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:17.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.470+0000 7fa883a8d640 1 --2- 172.21.15.164:0/3882112011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa87c100a80 0x7fa87c198490 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:17.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.470+0000 7fa883a8d640 1 -- 172.21.15.164:0/3882112011 >> 172.21.15.164:0/3882112011 conn(0x7fa87c0fc7a0 msgr2=0x7fa87c06aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:58:17.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.470+0000 7fa883a8d640 1 -- 172.21.15.164:0/3882112011 shutdown_connections 2024-09-19T08:58:17.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:17.470+0000 7fa883a8d640 1 -- 172.21.15.164:0/3882112011 wait complete. 2024-09-19T08:58:18.383 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs fail cephfs 2024-09-19T08:58:18.779 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:18 smithi173 ceph-mon[35306]: pgmap v390: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-19T08:58:18.779 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:18 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3882112011' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:58:18.779 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:18 smithi170 ceph-mon[35072]: pgmap v390: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-19T08:58:18.779 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:18 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3882112011' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:58:18.779 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:18 smithi164 ceph-mon[30928]: pgmap v390: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-19T08:58:18.780 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:18 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3882112011' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T08:58:19.040 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:58:19.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.466+0000 7ff7c3926640 1 -- 172.21.15.164:0/3989013435 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7bc074590 msgr2=0x7ff7bc074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:19.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.466+0000 7ff7c3926640 1 --2- 172.21.15.164:0/3989013435 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7bc074590 0x7ff7bc074970 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8009900 tx=0x7ff7b802a480 comp rx=0 tx=0).stop 2024-09-19T08:58:19.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.466+0000 7ff7c3926640 1 -- 172.21.15.164:0/3989013435 shutdown_connections 2024-09-19T08:58:19.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.466+0000 7ff7c3926640 1 --2- 172.21.15.164:0/3989013435 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7bc112b90 0x7ff7bc114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:19.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.466+0000 7ff7c3926640 1 --2- 172.21.15.164:0/3989013435 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff7bc074eb0 0x7ff7bc112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:19.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.466+0000 7ff7c3926640 1 --2- 172.21.15.164:0/3989013435 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7bc074590 0x7ff7bc074970 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:19.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.466+0000 7ff7c3926640 1 -- 172.21.15.164:0/3989013435 >> 172.21.15.164:0/3989013435 conn(0x7ff7bc0fd8c0 msgr2=0x7ff7bc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:58:19.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.466+0000 7ff7c3926640 1 -- 172.21.15.164:0/3989013435 shutdown_connections 2024-09-19T08:58:19.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.466+0000 7ff7c3926640 1 -- 172.21.15.164:0/3989013435 wait complete. 2024-09-19T08:58:19.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.467+0000 7ff7c3926640 1 Processor -- start 2024-09-19T08:58:19.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.467+0000 7ff7c3926640 1 -- start start 2024-09-19T08:58:19.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.467+0000 7ff7c3926640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7bc074590 0x7ff7bc1a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:19.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.467+0000 7ff7c3926640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7bc074eb0 0x7ff7bc1a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:19.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.468+0000 7ff7c3926640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff7bc112b90 0x7ff7bc19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:19.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.468+0000 7ff7c3926640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc118600 con 0x7ff7bc074eb0 2024-09-19T08:58:19.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.468+0000 7ff7c3926640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc118480 con 0x7ff7bc074590 2024-09-19T08:58:19.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.468+0000 7ff7c3926640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc118780 con 0x7ff7bc112b90 2024-09-19T08:58:19.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.468+0000 7ff7c0ec1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7bc074eb0 0x7ff7bc1a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:19.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.468+0000 7ff7c0ec1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7bc074eb0 0x7ff7bc1a4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39560/0 (socket says 172.21.15.164:39560) 2024-09-19T08:58:19.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.468+0000 7ff7c0ec1640 1 -- 172.21.15.164:0/3658895701 learned_addr learned my addr 172.21.15.164:0/3658895701 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:58:19.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.468+0000 7ff7c1ec3640 1 --2- 172.21.15.164:0/3658895701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff7bc112b90 0x7ff7bc19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:19.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.468+0000 7ff7c16c2640 1 --2- 172.21.15.164:0/3658895701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7bc074590 0x7ff7bc1a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:19.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.468+0000 7ff7c0ec1640 1 -- 172.21.15.164:0/3658895701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff7bc112b90 msgr2=0x7ff7bc19e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:19.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.468+0000 7ff7c0ec1640 1 --2- 172.21.15.164:0/3658895701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff7bc112b90 0x7ff7bc19e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:19.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.468+0000 7ff7c0ec1640 1 -- 172.21.15.164:0/3658895701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7bc074590 msgr2=0x7ff7bc1a4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:19.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.468+0000 7ff7c0ec1640 1 --2- 172.21.15.164:0/3658895701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7bc074590 0x7ff7bc1a4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:19.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.468+0000 7ff7c0ec1640 1 -- 172.21.15.164:0/3658895701 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7bc19f060 con 0x7ff7bc074eb0 2024-09-19T08:58:19.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.468+0000 7ff7c16c2640 1 --2- 172.21.15.164:0/3658895701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7bc074590 0x7ff7bc1a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:58:19.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.468+0000 7ff7c0ec1640 1 --2- 172.21.15.164:0/3658895701 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7bc074eb0 0x7ff7bc1a4cb0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac00d770 tx=0x7ff7ac00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:58:19.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.468+0000 7ff7b27fc640 1 -- 172.21.15.164:0/3658895701 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac017070 con 0x7ff7bc074eb0 2024-09-19T08:58:19.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.468+0000 7ff7b27fc640 1 -- 172.21.15.164:0/3658895701 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7ac005500 con 0x7ff7bc074eb0 2024-09-19T08:58:19.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.468+0000 7ff7c3926640 1 -- 172.21.15.164:0/3658895701 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7bc19f350 con 0x7ff7bc074eb0 2024-09-19T08:58:19.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.468+0000 7ff7b27fc640 1 -- 172.21.15.164:0/3658895701 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac004e00 con 0x7ff7bc074eb0 2024-09-19T08:58:19.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.468+0000 7ff7c3926640 1 -- 172.21.15.164:0/3658895701 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7bc1ab640 con 0x7ff7bc074eb0 2024-09-19T08:58:19.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.469+0000 7ff7c3926640 1 -- 172.21.15.164:0/3658895701 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7bc074970 con 0x7ff7bc074eb0 2024-09-19T08:58:19.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.471+0000 7ff7b27fc640 1 -- 172.21.15.164:0/3658895701 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7ff7ac004fa0 con 0x7ff7bc074eb0 2024-09-19T08:58:19.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.471+0000 7ff7b27fc640 1 --2- 172.21.15.164:0/3658895701 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff7a80a3430 0x7ff7a80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:19.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.471+0000 7ff7c16c2640 1 --2- 172.21.15.164:0/3658895701 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff7a80a3430 0x7ff7a80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:19.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.471+0000 7ff7b27fc640 1 -- 172.21.15.164:0/3658895701 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8499+0+0 (secure 0 0 0) 0x7ff7ac0cf740 con 0x7ff7bc074eb0 2024-09-19T08:58:19.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.472+0000 7ff7c16c2640 1 --2- 172.21.15.164:0/3658895701 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff7a80a3430 0x7ff7a80a58f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8000c00 tx=0x7ff7b8005a20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:58:19.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.475+0000 7ff7b27fc640 1 -- 172.21.15.164:0/3658895701 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff7ac09c0d0 con 0x7ff7bc074eb0 2024-09-19T08:58:19.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.675+0000 7ff7c3926640 1 -- 172.21.15.164:0/3658895701 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff7bc02d7c0 con 0x7ff7bc074eb0 2024-09-19T08:58:19.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.677+0000 7ff7b27fc640 1 -- 172.21.15.164:0/3658895701 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v34) ==== 161+0+0 (secure 0 0 0) 0x7ff7ac09b9d0 con 0x7ff7bc074eb0 2024-09-19T08:58:19.676 INFO:teuthology.orchestra.run.smithi164.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T08:58:19.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.680+0000 7ff7c3926640 1 -- 172.21.15.164:0/3658895701 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff7a80a3430 msgr2=0x7ff7a80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:19.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.680+0000 7ff7c3926640 1 --2- 172.21.15.164:0/3658895701 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff7a80a3430 0x7ff7a80a58f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8000c00 tx=0x7ff7b8005a20 comp rx=0 tx=0).stop 2024-09-19T08:58:19.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.680+0000 7ff7c3926640 1 -- 172.21.15.164:0/3658895701 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7bc074eb0 msgr2=0x7ff7bc1a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:19.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.680+0000 7ff7c3926640 1 --2- 172.21.15.164:0/3658895701 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7bc074eb0 0x7ff7bc1a4cb0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac00d770 tx=0x7ff7ac00dc30 comp rx=0 tx=0).stop 2024-09-19T08:58:19.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.680+0000 7ff7c3926640 1 -- 172.21.15.164:0/3658895701 shutdown_connections 2024-09-19T08:58:19.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.680+0000 7ff7c3926640 1 --2- 172.21.15.164:0/3658895701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff7bc112b90 0x7ff7bc19e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:19.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.680+0000 7ff7c3926640 1 --2- 172.21.15.164:0/3658895701 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7ff7a80a3430 0x7ff7a80a58f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:19.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.680+0000 7ff7c3926640 1 --2- 172.21.15.164:0/3658895701 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7bc074eb0 0x7ff7bc1a4cb0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:19.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.680+0000 7ff7c3926640 1 --2- 172.21.15.164:0/3658895701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7bc074590 0x7ff7bc1a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:19.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.680+0000 7ff7c3926640 1 -- 172.21.15.164:0/3658895701 >> 172.21.15.164:0/3658895701 conn(0x7ff7bc0fd8c0 msgr2=0x7ff7bc0ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:58:19.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.681+0000 7ff7c3926640 1 -- 172.21.15.164:0/3658895701 shutdown_connections 2024-09-19T08:58:19.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:19.681+0000 7ff7c3926640 1 -- 172.21.15.164:0/3658895701 wait complete. 2024-09-19T08:58:19.717 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:19 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mon-a[30924]: 2024-09-19T08:58:19.675+0000 7fe1d8dcc640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T08:58:20.032 INFO:journalctl@ceph.mds.a.smithi164.stdout:Sep 19 08:58:19 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-a[71584]: ignoring --setuser ceph since I am not root 2024-09-19T08:58:20.032 INFO:journalctl@ceph.mds.a.smithi164.stdout:Sep 19 08:58:19 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-a[71584]: ignoring --setgroup ceph since I am not root 2024-09-19T08:58:20.032 INFO:journalctl@ceph.mds.a.smithi164.stdout:Sep 19 08:58:19 smithi164 ceph-mds[71588]: -- 172.21.15.164:0/1023663877 <== mon.0 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562136655860 con 0x562136633000 2024-09-19T08:58:20.032 INFO:journalctl@ceph.mds.a.smithi164.stdout:Sep 19 08:58:19 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-a[71584]: starting mds.a at 2024-09-19T08:58:20.033 INFO:journalctl@ceph.mds.g.smithi164.stdout:Sep 19 08:58:19 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-g[71206]: ignoring --setuser ceph since I am not root 2024-09-19T08:58:20.033 INFO:journalctl@ceph.mds.g.smithi164.stdout:Sep 19 08:58:19 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-g[71206]: ignoring --setgroup ceph since I am not root 2024-09-19T08:58:20.033 INFO:journalctl@ceph.mds.g.smithi164.stdout:Sep 19 08:58:19 smithi164 ceph-mds[71218]: -- 172.21.15.164:0/2089050078 <== mon.0 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56123b751860 con 0x56123b72e800 2024-09-19T08:58:20.033 INFO:journalctl@ceph.mds.g.smithi164.stdout:Sep 19 08:58:19 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-g[71206]: starting mds.g at 2024-09-19T08:58:20.125 INFO:journalctl@ceph.mds.b.smithi170.stdout:Sep 19 08:58:19 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-b[65018]: ignoring --setuser ceph since I am not root 2024-09-19T08:58:20.125 INFO:journalctl@ceph.mds.b.smithi170.stdout:Sep 19 08:58:19 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-b[65018]: ignoring --setgroup ceph since I am not root 2024-09-19T08:58:20.125 INFO:journalctl@ceph.mds.b.smithi170.stdout:Sep 19 08:58:19 smithi170 ceph-mds[65022]: -- 172.21.15.170:0/140397770 <== mon.2 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f688f27860 con 0x55f688f04800 2024-09-19T08:58:20.125 INFO:journalctl@ceph.mds.b.smithi170.stdout:Sep 19 08:58:19 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-b[65018]: starting mds.b at 2024-09-19T08:58:20.126 INFO:journalctl@ceph.mds.e.smithi170.stdout:Sep 19 08:58:19 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-e[64416]: ignoring --setuser ceph since I am not root 2024-09-19T08:58:20.126 INFO:journalctl@ceph.mds.e.smithi170.stdout:Sep 19 08:58:19 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-e[64416]: ignoring --setgroup ceph since I am not root 2024-09-19T08:58:20.126 INFO:journalctl@ceph.mds.e.smithi170.stdout:Sep 19 08:58:19 smithi170 ceph-mds[64420]: -- 172.21.15.170:0/1081043018 <== mon.0 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5648d2019860 con 0x5648d1ff6800 2024-09-19T08:58:20.126 INFO:journalctl@ceph.mds.e.smithi170.stdout:Sep 19 08:58:19 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-e[64416]: starting mds.e at 2024-09-19T08:58:20.127 INFO:journalctl@ceph.mds.h.smithi170.stdout:Sep 19 08:58:19 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-h[64715]: ignoring --setuser ceph since I am not root 2024-09-19T08:58:20.127 INFO:journalctl@ceph.mds.h.smithi170.stdout:Sep 19 08:58:19 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-h[64715]: ignoring --setgroup ceph since I am not root 2024-09-19T08:58:20.127 INFO:journalctl@ceph.mds.h.smithi170.stdout:Sep 19 08:58:19 smithi170 ceph-mds[64719]: -- 172.21.15.170:0/1124586771 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56003c97d860 con 0x56003c95b400 2024-09-19T08:58:20.127 INFO:journalctl@ceph.mds.h.smithi170.stdout:Sep 19 08:58:19 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-h[64715]: starting mds.h at 2024-09-19T08:58:20.127 INFO:journalctl@ceph.mds.k.smithi170.stdout:Sep 19 08:58:19 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-k[64104]: ignoring --setuser ceph since I am not root 2024-09-19T08:58:20.127 INFO:journalctl@ceph.mds.k.smithi170.stdout:Sep 19 08:58:19 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-k[64104]: ignoring --setgroup ceph since I am not root 2024-09-19T08:58:20.127 INFO:journalctl@ceph.mds.k.smithi170.stdout:Sep 19 08:58:19 smithi170 ceph-mds[64108]: -- 172.21.15.170:0/2905888215 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56508ec37860 con 0x56508ec14800 2024-09-19T08:58:20.128 INFO:journalctl@ceph.mds.k.smithi170.stdout:Sep 19 08:58:19 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-k[64104]: starting mds.k at 2024-09-19T08:58:20.161 INFO:journalctl@ceph.mds.c.smithi173.stdout:Sep 19 08:58:19 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-c[65367]: ignoring --setuser ceph since I am not root 2024-09-19T08:58:20.161 INFO:journalctl@ceph.mds.c.smithi173.stdout:Sep 19 08:58:19 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-c[65367]: ignoring --setgroup ceph since I am not root 2024-09-19T08:58:20.161 INFO:journalctl@ceph.mds.c.smithi173.stdout:Sep 19 08:58:19 smithi173 ceph-mds[65371]: -- 172.21.15.173:0/4225383092 <== mon.0 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c0a7f2b860 con 0x55c0a7f09000 2024-09-19T08:58:20.161 INFO:journalctl@ceph.mds.c.smithi173.stdout:Sep 19 08:58:19 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-c[65367]: starting mds.c at 2024-09-19T08:58:20.161 INFO:journalctl@ceph.mds.f.smithi173.stdout:Sep 19 08:58:19 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-f[65065]: ignoring --setuser ceph since I am not root 2024-09-19T08:58:20.161 INFO:journalctl@ceph.mds.f.smithi173.stdout:Sep 19 08:58:19 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-f[65065]: ignoring --setgroup ceph since I am not root 2024-09-19T08:58:20.162 INFO:journalctl@ceph.mds.f.smithi173.stdout:Sep 19 08:58:19 smithi173 ceph-mds[65069]: -- 172.21.15.173:0/283428702 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557625e6f860 con 0x557625e4d000 2024-09-19T08:58:20.162 INFO:journalctl@ceph.mds.f.smithi173.stdout:Sep 19 08:58:19 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-f[65065]: starting mds.f at 2024-09-19T08:58:20.162 INFO:journalctl@ceph.mds.i.smithi173.stdout:Sep 19 08:58:19 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-i[65667]: ignoring --setuser ceph since I am not root 2024-09-19T08:58:20.162 INFO:journalctl@ceph.mds.i.smithi173.stdout:Sep 19 08:58:19 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-i[65667]: ignoring --setgroup ceph since I am not root 2024-09-19T08:58:20.162 INFO:journalctl@ceph.mds.i.smithi173.stdout:Sep 19 08:58:19 smithi173 ceph-mds[65671]: -- 172.21.15.173:0/2989019732 <== mon.2 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56309f76b860 con 0x56309f749000 2024-09-19T08:58:20.162 INFO:journalctl@ceph.mds.i.smithi173.stdout:Sep 19 08:58:19 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-i[65667]: starting mds.i at 2024-09-19T08:58:20.162 INFO:journalctl@ceph.mds.l.smithi173.stdout:Sep 19 08:58:19 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-l[64762]: ignoring --setuser ceph since I am not root 2024-09-19T08:58:20.162 INFO:journalctl@ceph.mds.l.smithi173.stdout:Sep 19 08:58:19 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-l[64762]: ignoring --setgroup ceph since I am not root 2024-09-19T08:58:20.162 INFO:journalctl@ceph.mds.l.smithi173.stdout:Sep 19 08:58:19 smithi173 ceph-mds[64766]: -- 172.21.15.173:0/2894241537 <== mon.2 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55af3cfad860 con 0x55af3cf8b400 2024-09-19T08:58:20.163 INFO:journalctl@ceph.mds.l.smithi173.stdout:Sep 19 08:58:19 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-l[64762]: starting mds.l at 2024-09-19T08:58:20.432 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:20 smithi164 ceph-mon[30928]: pgmap v391: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 984 B/s wr, 11 op/s 2024-09-19T08:58:20.432 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:20 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3658895701' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T08:58:20.432 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:20 smithi164 ceph-mon[30928]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T08:58:20.432 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:20 smithi164 ceph-mon[30928]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T08:58:20.433 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:20 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3658895701' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T08:58:20.433 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:20 smithi164 ceph-mon[30928]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T08:58:20.433 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:20 smithi164 ceph-mon[30928]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T08:58:20.433 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:20 smithi164 ceph-mon[30928]: osdmap e98: 12 total, 12 up, 12 in 2024-09-19T08:58:20.459 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T08:58:20.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:20 smithi170 ceph-mon[35072]: pgmap v391: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 984 B/s wr, 11 op/s 2024-09-19T08:58:20.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:20 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3658895701' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T08:58:20.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:20 smithi170 ceph-mon[35072]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T08:58:20.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:20 smithi170 ceph-mon[35072]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T08:58:20.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:20 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3658895701' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T08:58:20.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:20 smithi170 ceph-mon[35072]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T08:58:20.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:20 smithi170 ceph-mon[35072]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T08:58:20.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:20 smithi170 ceph-mon[35072]: osdmap e98: 12 total, 12 up, 12 in 2024-09-19T08:58:20.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:20 smithi173 ceph-mon[35306]: pgmap v391: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 984 B/s wr, 11 op/s 2024-09-19T08:58:20.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:20 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3658895701' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T08:58:20.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:20 smithi173 ceph-mon[35306]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T08:58:20.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:20 smithi173 ceph-mon[35306]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T08:58:20.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:20 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3658895701' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T08:58:20.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:20 smithi173 ceph-mon[35306]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T08:58:20.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:20 smithi173 ceph-mon[35306]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T08:58:20.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:20 smithi173 ceph-mon[35306]: osdmap e98: 12 total, 12 up, 12 in 2024-09-19T08:58:20.708 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:58:21.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.225+0000 7f2df4adc640 1 -- 172.21.15.164:0/1498426918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2df0106a80 msgr2=0x7f2df0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:21.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.225+0000 7f2df4adc640 1 --2- 172.21.15.164:0/1498426918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2df0106a80 0x7f2df0106e60 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009930 tx=0x7f2de002a420 comp rx=0 tx=0).stop 2024-09-19T08:58:21.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.225+0000 7f2df4adc640 1 -- 172.21.15.164:0/1498426918 shutdown_connections 2024-09-19T08:58:21.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.225+0000 7f2df4adc640 1 --2- 172.21.15.164:0/1498426918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2df0101420 0x7f2df010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:21.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.225+0000 7f2df4adc640 1 --2- 172.21.15.164:0/1498426918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2df0100a80 0x7f2df0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:21.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.225+0000 7f2df4adc640 1 --2- 172.21.15.164:0/1498426918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2df0106a80 0x7f2df0106e60 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:21.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.225+0000 7f2df4adc640 1 -- 172.21.15.164:0/1498426918 >> 172.21.15.164:0/1498426918 conn(0x7f2df00fc7a0 msgr2=0x7f2df00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:58:21.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.225+0000 7f2df4adc640 1 -- 172.21.15.164:0/1498426918 shutdown_connections 2024-09-19T08:58:21.325 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.225+0000 7f2df4adc640 1 -- 172.21.15.164:0/1498426918 wait complete. 2024-09-19T08:58:21.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.226+0000 7f2df4adc640 1 Processor -- start 2024-09-19T08:58:21.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.226+0000 7f2df4adc640 1 -- start start 2024-09-19T08:58:21.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.226+0000 7f2df4adc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2df0100a80 0x7f2df01973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:21.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.226+0000 7f2df4adc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2df0101420 0x7f2df01978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:21.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.226+0000 7f2df4adc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2df0106a80 0x7f2df019bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:21.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.226+0000 7f2df4adc640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2df0110e80 con 0x7f2df0101420 2024-09-19T08:58:21.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.226+0000 7f2df4adc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2df0110d00 con 0x7f2df0100a80 2024-09-19T08:58:21.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.226+0000 7f2df4adc640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2df0111000 con 0x7f2df0106a80 2024-09-19T08:58:21.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.226+0000 7f2dedd9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2df0101420 0x7f2df01978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:21.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.226+0000 7f2dedd9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2df0101420 0x7f2df01978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39606/0 (socket says 172.21.15.164:39606) 2024-09-19T08:58:21.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.226+0000 7f2dedd9b640 1 -- 172.21.15.164:0/3971646651 learned_addr learned my addr 172.21.15.164:0/3971646651 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:58:21.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.226+0000 7f2dee59c640 1 --2- 172.21.15.164:0/3971646651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2df0100a80 0x7f2df01973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:21.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.226+0000 7f2deed9d640 1 --2- 172.21.15.164:0/3971646651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2df0106a80 0x7f2df019bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:21.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.227+0000 7f2dedd9b640 1 -- 172.21.15.164:0/3971646651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2df0106a80 msgr2=0x7f2df019bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:21.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.227+0000 7f2dedd9b640 1 --2- 172.21.15.164:0/3971646651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2df0106a80 0x7f2df019bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:21.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.227+0000 7f2dedd9b640 1 -- 172.21.15.164:0/3971646651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2df0100a80 msgr2=0x7f2df01973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:21.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.227+0000 7f2dedd9b640 1 --2- 172.21.15.164:0/3971646651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2df0100a80 0x7f2df01973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:21.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.227+0000 7f2dedd9b640 1 -- 172.21.15.164:0/3971646651 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df019c360 con 0x7f2df0101420 2024-09-19T08:58:21.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.227+0000 7f2dedd9b640 1 --2- 172.21.15.164:0/3971646651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2df0101420 0x7f2df01978f0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f2de400d770 tx=0x7f2de400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:58:21.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.227+0000 7f2ddf7fe640 1 -- 172.21.15.164:0/3971646651 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de4017070 con 0x7f2df0101420 2024-09-19T08:58:21.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.227+0000 7f2ddf7fe640 1 -- 172.21.15.164:0/3971646651 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de4005500 con 0x7f2df0101420 2024-09-19T08:58:21.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.227+0000 7f2df4adc640 1 -- 172.21.15.164:0/3971646651 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df019c650 con 0x7f2df0101420 2024-09-19T08:58:21.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.227+0000 7f2ddf7fe640 1 -- 172.21.15.164:0/3971646651 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de4004e00 con 0x7f2df0101420 2024-09-19T08:58:21.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.227+0000 7f2df4adc640 1 -- 172.21.15.164:0/3971646651 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df01a3f30 con 0x7f2df0101420 2024-09-19T08:58:21.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.228+0000 7f2df4adc640 1 -- 172.21.15.164:0/3971646651 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2df0100ee0 con 0x7f2df0101420 2024-09-19T08:58:21.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.229+0000 7f2ddf7fe640 1 -- 172.21.15.164:0/3971646651 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f2de4004fa0 con 0x7f2df0101420 2024-09-19T08:58:21.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.230+0000 7f2ddf7fe640 1 --2- 172.21.15.164:0/3971646651 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f2dd80a33b0 0x7f2dd80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:21.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.230+0000 7f2dee59c640 1 --2- 172.21.15.164:0/3971646651 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f2dd80a33b0 0x7f2dd80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:21.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.230+0000 7f2ddf7fe640 1 -- 172.21.15.164:0/3971646651 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8885+0+0 (secure 0 0 0) 0x7f2de40cfc90 con 0x7f2df0101420 2024-09-19T08:58:21.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.230+0000 7f2dee59c640 1 --2- 172.21.15.164:0/3971646651 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f2dd80a33b0 0x7f2dd80a5870 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009900 tx=0x7f2de00058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:58:21.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.233+0000 7f2ddf7fe640 1 -- 172.21.15.164:0/3971646651 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2de409beb0 con 0x7f2df0101420 2024-09-19T08:58:21.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:21.435+0000 7f2df4adc640 1 -- 172.21.15.164:0/3971646651 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f2df0108ff0 con 0x7f2df0101420 2024-09-19T08:58:21.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:21 smithi164 ceph-mon[30928]: pgmap v394: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-19T08:58:21.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:21 smithi164 ceph-mon[30928]: mds.? [v2:172.21.15.164:6839/2290011998,v1:172.21.15.164:6841/2290011998] up:boot 2024-09-19T08:58:21.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:21 smithi164 ceph-mon[30928]: mds.? [v2:172.21.15.164:6838/1626801865,v1:172.21.15.164:6840/1626801865] up:boot 2024-09-19T08:58:21.782 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:21 smithi164 ceph-mon[30928]: mds.? [v2:172.21.15.170:6834/1368546562,v1:172.21.15.170:6836/1368546562] up:boot 2024-09-19T08:58:21.783 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:21 smithi164 ceph-mon[30928]: mds.? [v2:172.21.15.170:6833/1483930576,v1:172.21.15.170:6837/1483930576] up:boot 2024-09-19T08:58:21.783 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:21 smithi164 ceph-mon[30928]: mds.? [v2:172.21.15.170:6832/2886446936,v1:172.21.15.170:6835/2886446936] up:boot 2024-09-19T08:58:21.783 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:21 smithi164 ceph-mon[30928]: mds.? [v2:172.21.15.170:6838/3083499350,v1:172.21.15.170:6839/3083499350] up:boot 2024-09-19T08:58:21.783 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:21 smithi164 ceph-mon[30928]: mds.? [v2:172.21.15.173:6832/4066436731,v1:172.21.15.173:6833/4066436731] up:boot 2024-09-19T08:58:21.783 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:21 smithi164 ceph-mon[30928]: mds.? [v2:172.21.15.173:6834/1156249381,v1:172.21.15.173:6837/1156249381] up:boot 2024-09-19T08:58:21.783 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:21 smithi164 ceph-mon[30928]: mds.? [v2:172.21.15.173:6835/3214050023,v1:172.21.15.173:6838/3214050023] up:boot 2024-09-19T08:58:21.783 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:21 smithi164 ceph-mon[30928]: mds.? [v2:172.21.15.173:6836/764363776,v1:172.21.15.173:6839/764363776] up:boot 2024-09-19T08:58:21.783 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:21 smithi164 ceph-mon[30928]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T08:58:21.783 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:21 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T08:58:21.783 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:21 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T08:58:21.783 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:21 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T08:58:21.783 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:21 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T08:58:21.784 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:21 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T08:58:21.784 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:21 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T08:58:21.784 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:21 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T08:58:21.784 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:21 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T08:58:21.784 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:21 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T08:58:21.784 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:21 smithi164 ceph-mon[30928]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T08:58:21.784 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:21 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3971646651' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T08:58:21.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:21 smithi173 ceph-mon[35306]: pgmap v394: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-19T08:58:21.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:21 smithi173 ceph-mon[35306]: mds.? [v2:172.21.15.164:6839/2290011998,v1:172.21.15.164:6841/2290011998] up:boot 2024-09-19T08:58:21.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:21 smithi173 ceph-mon[35306]: mds.? [v2:172.21.15.164:6838/1626801865,v1:172.21.15.164:6840/1626801865] up:boot 2024-09-19T08:58:21.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:21 smithi173 ceph-mon[35306]: mds.? [v2:172.21.15.170:6834/1368546562,v1:172.21.15.170:6836/1368546562] up:boot 2024-09-19T08:58:21.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:21 smithi173 ceph-mon[35306]: mds.? [v2:172.21.15.170:6833/1483930576,v1:172.21.15.170:6837/1483930576] up:boot 2024-09-19T08:58:21.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:21 smithi173 ceph-mon[35306]: mds.? [v2:172.21.15.170:6832/2886446936,v1:172.21.15.170:6835/2886446936] up:boot 2024-09-19T08:58:21.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:21 smithi173 ceph-mon[35306]: mds.? [v2:172.21.15.170:6838/3083499350,v1:172.21.15.170:6839/3083499350] up:boot 2024-09-19T08:58:21.911 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:21 smithi173 ceph-mon[35306]: mds.? [v2:172.21.15.173:6832/4066436731,v1:172.21.15.173:6833/4066436731] up:boot 2024-09-19T08:58:21.912 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:21 smithi173 ceph-mon[35306]: mds.? [v2:172.21.15.173:6834/1156249381,v1:172.21.15.173:6837/1156249381] up:boot 2024-09-19T08:58:21.912 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:21 smithi173 ceph-mon[35306]: mds.? [v2:172.21.15.173:6835/3214050023,v1:172.21.15.173:6838/3214050023] up:boot 2024-09-19T08:58:21.912 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:21 smithi173 ceph-mon[35306]: mds.? [v2:172.21.15.173:6836/764363776,v1:172.21.15.173:6839/764363776] up:boot 2024-09-19T08:58:21.912 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:21 smithi173 ceph-mon[35306]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T08:58:21.912 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:21 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T08:58:21.912 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:21 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T08:58:21.912 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:21 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T08:58:21.912 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:21 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T08:58:21.912 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:21 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T08:58:21.912 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:21 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T08:58:21.912 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:21 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T08:58:21.912 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:21 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T08:58:21.912 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:21 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T08:58:21.913 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:21 smithi173 ceph-mon[35306]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T08:58:21.913 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:21 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3971646651' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T08:58:22.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:21 smithi170 ceph-mon[35072]: pgmap v394: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-19T08:58:22.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:21 smithi170 ceph-mon[35072]: mds.? [v2:172.21.15.164:6839/2290011998,v1:172.21.15.164:6841/2290011998] up:boot 2024-09-19T08:58:22.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:21 smithi170 ceph-mon[35072]: mds.? [v2:172.21.15.164:6838/1626801865,v1:172.21.15.164:6840/1626801865] up:boot 2024-09-19T08:58:22.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:21 smithi170 ceph-mon[35072]: mds.? [v2:172.21.15.170:6834/1368546562,v1:172.21.15.170:6836/1368546562] up:boot 2024-09-19T08:58:22.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:21 smithi170 ceph-mon[35072]: mds.? [v2:172.21.15.170:6833/1483930576,v1:172.21.15.170:6837/1483930576] up:boot 2024-09-19T08:58:22.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:21 smithi170 ceph-mon[35072]: mds.? [v2:172.21.15.170:6832/2886446936,v1:172.21.15.170:6835/2886446936] up:boot 2024-09-19T08:58:22.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:21 smithi170 ceph-mon[35072]: mds.? [v2:172.21.15.170:6838/3083499350,v1:172.21.15.170:6839/3083499350] up:boot 2024-09-19T08:58:22.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:21 smithi170 ceph-mon[35072]: mds.? [v2:172.21.15.173:6832/4066436731,v1:172.21.15.173:6833/4066436731] up:boot 2024-09-19T08:58:22.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:21 smithi170 ceph-mon[35072]: mds.? [v2:172.21.15.173:6834/1156249381,v1:172.21.15.173:6837/1156249381] up:boot 2024-09-19T08:58:22.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:21 smithi170 ceph-mon[35072]: mds.? [v2:172.21.15.173:6835/3214050023,v1:172.21.15.173:6838/3214050023] up:boot 2024-09-19T08:58:22.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:21 smithi170 ceph-mon[35072]: mds.? [v2:172.21.15.173:6836/764363776,v1:172.21.15.173:6839/764363776] up:boot 2024-09-19T08:58:22.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:21 smithi170 ceph-mon[35072]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T08:58:22.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:21 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T08:58:22.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:21 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T08:58:22.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:21 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T08:58:22.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:21 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T08:58:22.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:21 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T08:58:22.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:21 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T08:58:22.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:21 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T08:58:22.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:21 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T08:58:22.127 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:21 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T08:58:22.127 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:21 smithi170 ceph-mon[35072]: from='mgr.15201 172.21.15.164:0/2082442866' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T08:58:22.127 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:21 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3971646651' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T08:58:22.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:22.280+0000 7f2ddf7fe640 1 -- 172.21.15.164:0/3971646651 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v36) ==== 104+0+0 (secure 0 0 0) 0x7f2de40a01b0 con 0x7f2df0101420 2024-09-19T08:58:22.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:22.284+0000 7f2df4adc640 1 -- 172.21.15.164:0/3971646651 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f2dd80a33b0 msgr2=0x7f2dd80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:22.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:22.284+0000 7f2df4adc640 1 --2- 172.21.15.164:0/3971646651 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f2dd80a33b0 0x7f2dd80a5870 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009900 tx=0x7f2de00058d0 comp rx=0 tx=0).stop 2024-09-19T08:58:22.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:22.284+0000 7f2df4adc640 1 -- 172.21.15.164:0/3971646651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2df0101420 msgr2=0x7f2df01978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:22.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:22.284+0000 7f2df4adc640 1 --2- 172.21.15.164:0/3971646651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2df0101420 0x7f2df01978f0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f2de400d770 tx=0x7f2de400dc30 comp rx=0 tx=0).stop 2024-09-19T08:58:22.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:22.284+0000 7f2df4adc640 1 -- 172.21.15.164:0/3971646651 shutdown_connections 2024-09-19T08:58:22.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:22.284+0000 7f2df4adc640 1 --2- 172.21.15.164:0/3971646651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2df0106a80 0x7f2df019bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:22.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:22.284+0000 7f2df4adc640 1 --2- 172.21.15.164:0/3971646651 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f2dd80a33b0 0x7f2dd80a5870 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:22.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:22.284+0000 7f2df4adc640 1 --2- 172.21.15.164:0/3971646651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2df0101420 0x7f2df01978f0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:22.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:22.284+0000 7f2df4adc640 1 --2- 172.21.15.164:0/3971646651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2df0100a80 0x7f2df01973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:22.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:22.284+0000 7f2df4adc640 1 -- 172.21.15.164:0/3971646651 >> 172.21.15.164:0/3971646651 conn(0x7f2df00fc7a0 msgr2=0x7f2df00fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:58:22.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:22.284+0000 7f2df4adc640 1 -- 172.21.15.164:0/3971646651 shutdown_connections 2024-09-19T08:58:22.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:22.284+0000 7f2df4adc640 1 -- 172.21.15.164:0/3971646651 wait complete. 2024-09-19T08:58:23.032 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:22 smithi164 ceph-mon[30928]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T08:58:23.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:22 smithi164 ceph-mon[30928]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T08:58:23.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:22 smithi164 ceph-mon[30928]: Cluster is now healthy 2024-09-19T08:58:23.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:22 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/3971646651' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T08:58:23.033 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:22 smithi164 ceph-mon[30928]: fsmap 12 up:standby 2024-09-19T08:58:23.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:22 smithi170 ceph-mon[35072]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T08:58:23.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:22 smithi170 ceph-mon[35072]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T08:58:23.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:22 smithi170 ceph-mon[35072]: Cluster is now healthy 2024-09-19T08:58:23.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:22 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/3971646651' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T08:58:23.125 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:22 smithi170 ceph-mon[35072]: fsmap 12 up:standby 2024-09-19T08:58:23.159 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T08:58:23.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:22 smithi173 ceph-mon[35306]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T08:58:23.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:22 smithi173 ceph-mon[35306]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T08:58:23.160 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:22 smithi173 ceph-mon[35306]: Cluster is now healthy 2024-09-19T08:58:23.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:22 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/3971646651' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T08:58:23.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:22 smithi173 ceph-mon[35306]: fsmap 12 up:standby 2024-09-19T08:58:23.412 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:58:23.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.873+0000 7fd1471d7640 1 -- 172.21.15.164:0/607036281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd140100a80 msgr2=0x7fd140100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:23.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.873+0000 7fd1471d7640 1 --2- 172.21.15.164:0/607036281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd140100a80 0x7fd140100ee0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fd12c009930 tx=0x7fd12c02a420 comp rx=0 tx=0).stop 2024-09-19T08:58:23.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.873+0000 7fd1471d7640 1 -- 172.21.15.164:0/607036281 shutdown_connections 2024-09-19T08:58:23.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.873+0000 7fd1471d7640 1 --2- 172.21.15.164:0/607036281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd140101420 0x7fd14010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:23.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.873+0000 7fd1471d7640 1 --2- 172.21.15.164:0/607036281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd140100a80 0x7fd140100ee0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:23.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.873+0000 7fd1471d7640 1 --2- 172.21.15.164:0/607036281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd140106a80 0x7fd140106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:23.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.873+0000 7fd1471d7640 1 -- 172.21.15.164:0/607036281 >> 172.21.15.164:0/607036281 conn(0x7fd1400fc7a0 msgr2=0x7fd1400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:58:23.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.873+0000 7fd1471d7640 1 -- 172.21.15.164:0/607036281 shutdown_connections 2024-09-19T08:58:23.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.873+0000 7fd1471d7640 1 -- 172.21.15.164:0/607036281 wait complete. 2024-09-19T08:58:23.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.874+0000 7fd1471d7640 1 Processor -- start 2024-09-19T08:58:23.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.874+0000 7fd1471d7640 1 -- start start 2024-09-19T08:58:23.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.874+0000 7fd1471d7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd140100a80 0x7fd1400fcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:23.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.874+0000 7fd1471d7640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd140101420 0x7fd1400fd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:23.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.874+0000 7fd1471d7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd140106a80 0x7fd1400fd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:23.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.874+0000 7fd1471d7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd140110ea0 con 0x7fd140106a80 2024-09-19T08:58:23.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.874+0000 7fd1471d7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd140110d20 con 0x7fd140100a80 2024-09-19T08:58:23.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.874+0000 7fd1471d7640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd140111020 con 0x7fd140101420 2024-09-19T08:58:23.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.874+0000 7fd145774640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd140106a80 0x7fd1400fd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:23.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.874+0000 7fd145774640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd140106a80 0x7fd1400fd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:34956/0 (socket says 172.21.15.164:34956) 2024-09-19T08:58:23.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.874+0000 7fd145774640 1 -- 172.21.15.164:0/4152104682 learned_addr learned my addr 172.21.15.164:0/4152104682 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:58:23.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.874+0000 7fd144f73640 1 --2- 172.21.15.164:0/4152104682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd140100a80 0x7fd1400fcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:23.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.874+0000 7fd13ffff640 1 --2- 172.21.15.164:0/4152104682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd140101420 0x7fd1400fd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:23.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.875+0000 7fd145774640 1 -- 172.21.15.164:0/4152104682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd140101420 msgr2=0x7fd1400fd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:23.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.875+0000 7fd145774640 1 --2- 172.21.15.164:0/4152104682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd140101420 0x7fd1400fd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:23.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.875+0000 7fd145774640 1 -- 172.21.15.164:0/4152104682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd140100a80 msgr2=0x7fd1400fcdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:23.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.875+0000 7fd145774640 1 --2- 172.21.15.164:0/4152104682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd140100a80 0x7fd1400fcdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:23.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.875+0000 7fd145774640 1 -- 172.21.15.164:0/4152104682 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1401a6990 con 0x7fd140106a80 2024-09-19T08:58:23.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.875+0000 7fd144f73640 1 --2- 172.21.15.164:0/4152104682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd140100a80 0x7fd1400fcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:58:23.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.875+0000 7fd145774640 1 --2- 172.21.15.164:0/4152104682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd140106a80 0x7fd1400fd830 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fd13800cad0 tx=0x7fd13800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:58:23.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.875+0000 7fd13dffb640 1 -- 172.21.15.164:0/4152104682 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13800ae20 con 0x7fd140106a80 2024-09-19T08:58:23.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.875+0000 7fd1471d7640 1 -- 172.21.15.164:0/4152104682 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1401a6c80 con 0x7fd140106a80 2024-09-19T08:58:23.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.875+0000 7fd13dffb640 1 -- 172.21.15.164:0/4152104682 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd138005020 con 0x7fd140106a80 2024-09-19T08:58:23.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.875+0000 7fd13dffb640 1 -- 172.21.15.164:0/4152104682 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13800f6f0 con 0x7fd140106a80 2024-09-19T08:58:23.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.875+0000 7fd1471d7640 1 -- 172.21.15.164:0/4152104682 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1401a7140 con 0x7fd140106a80 2024-09-19T08:58:23.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.876+0000 7fd1471d7640 1 -- 172.21.15.164:0/4152104682 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd140100ee0 con 0x7fd140106a80 2024-09-19T08:58:23.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.877+0000 7fd13dffb640 1 -- 172.21.15.164:0/4152104682 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fd1380051d0 con 0x7fd140106a80 2024-09-19T08:58:23.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.878+0000 7fd13dffb640 1 --2- 172.21.15.164:0/4152104682 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd1300a33b0 0x7fd1300a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:23.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.878+0000 7fd144f73640 1 --2- 172.21.15.164:0/4152104682 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd1300a33b0 0x7fd1300a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:23.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.878+0000 7fd13dffb640 1 -- 172.21.15.164:0/4152104682 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8885+0+0 (secure 0 0 0) 0x7fd1380d0000 con 0x7fd140106a80 2024-09-19T08:58:23.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.878+0000 7fd144f73640 1 --2- 172.21.15.164:0/4152104682 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd1300a33b0 0x7fd1300a5870 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd134009930 tx=0x7fd134008040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:58:23.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:23.882+0000 7fd13dffb640 1 -- 172.21.15.164:0/4152104682 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd138014030 con 0x7fd140106a80 2024-09-19T08:58:24.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:24.050+0000 7fd1471d7640 1 -- 172.21.15.164:0/4152104682 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd14002d7c0 con 0x7fd140106a80 2024-09-19T08:58:24.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:24.283+0000 7fd13dffb640 1 -- 172.21.15.164:0/4152104682 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v99) ==== 181+0+0 (secure 0 0 0) 0x7fd13809c870 con 0x7fd140106a80 2024-09-19T08:58:24.283 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_metadata' removed 2024-09-19T08:58:24.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:24.286+0000 7fd1471d7640 1 -- 172.21.15.164:0/4152104682 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd1300a33b0 msgr2=0x7fd1300a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:24.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:24.286+0000 7fd1471d7640 1 --2- 172.21.15.164:0/4152104682 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd1300a33b0 0x7fd1300a5870 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd134009930 tx=0x7fd134008040 comp rx=0 tx=0).stop 2024-09-19T08:58:24.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:24.286+0000 7fd1471d7640 1 -- 172.21.15.164:0/4152104682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd140106a80 msgr2=0x7fd1400fd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:24.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:24.286+0000 7fd1471d7640 1 --2- 172.21.15.164:0/4152104682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd140106a80 0x7fd1400fd830 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fd13800cad0 tx=0x7fd13800cf90 comp rx=0 tx=0).stop 2024-09-19T08:58:24.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:24.287+0000 7fd1471d7640 1 -- 172.21.15.164:0/4152104682 shutdown_connections 2024-09-19T08:58:24.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:24.287+0000 7fd1471d7640 1 --2- 172.21.15.164:0/4152104682 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7fd1300a33b0 0x7fd1300a5870 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:24.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:24.287+0000 7fd1471d7640 1 --2- 172.21.15.164:0/4152104682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd140106a80 0x7fd1400fd830 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:24.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:24.287+0000 7fd1471d7640 1 --2- 172.21.15.164:0/4152104682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd140101420 0x7fd1400fd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:24.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:24.287+0000 7fd1471d7640 1 --2- 172.21.15.164:0/4152104682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd140100a80 0x7fd1400fcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:24.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:24.287+0000 7fd1471d7640 1 -- 172.21.15.164:0/4152104682 >> 172.21.15.164:0/4152104682 conn(0x7fd1400fc7a0 msgr2=0x7fd14006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:58:24.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:24.287+0000 7fd1471d7640 1 -- 172.21.15.164:0/4152104682 shutdown_connections 2024-09-19T08:58:24.287 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:24.287+0000 7fd1471d7640 1 -- 172.21.15.164:0/4152104682 wait complete. 2024-09-19T08:58:24.299 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:24 smithi164 ceph-mon[30928]: pgmap v395: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T08:58:24.300 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:24 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/4152104682' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T08:58:24.624 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:24 smithi170 ceph-mon[35072]: pgmap v395: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T08:58:24.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:24 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/4152104682' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T08:58:24.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:24 smithi173 ceph-mon[35306]: pgmap v395: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T08:58:24.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:24 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/4152104682' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T08:58:25.059 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T08:58:25.309 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T08:58:25.584 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:25 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/4152104682' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T08:58:25.585 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:25 smithi164 ceph-mon[30928]: osdmap e99: 12 total, 12 up, 12 in 2024-09-19T08:58:25.624 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:25 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/4152104682' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T08:58:25.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:25 smithi170 ceph-mon[35072]: osdmap e99: 12 total, 12 up, 12 in 2024-09-19T08:58:25.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:25 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/4152104682' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T08:58:25.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:25 smithi173 ceph-mon[35306]: osdmap e99: 12 total, 12 up, 12 in 2024-09-19T08:58:25.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.838+0000 7f6a7ac5e640 1 -- 172.21.15.164:0/472581344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a74106f20 msgr2=0x7f6a7410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:25.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.838+0000 7f6a7ac5e640 1 --2- 172.21.15.164:0/472581344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a74106f20 0x7f6a7410eaa0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c009ca0 tx=0x7f6a6c02a8c0 comp rx=0 tx=0).stop 2024-09-19T08:58:25.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.839+0000 7f6a7ac5e640 1 -- 172.21.15.164:0/472581344 shutdown_connections 2024-09-19T08:58:25.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.839+0000 7f6a7ac5e640 1 --2- 172.21.15.164:0/472581344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a74106f20 0x7f6a7410eaa0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:25.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.839+0000 7f6a7ac5e640 1 --2- 172.21.15.164:0/472581344 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a741027a0 0x7f6a741067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:25.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.839+0000 7f6a7ac5e640 1 --2- 172.21.15.164:0/472581344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a74101df0 0x7f6a741021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:25.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.839+0000 7f6a7ac5e640 1 -- 172.21.15.164:0/472581344 >> 172.21.15.164:0/472581344 conn(0x7f6a740fd8b0 msgr2=0x7f6a740ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:58:25.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.839+0000 7f6a7ac5e640 1 -- 172.21.15.164:0/472581344 shutdown_connections 2024-09-19T08:58:25.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.839+0000 7f6a7ac5e640 1 -- 172.21.15.164:0/472581344 wait complete. 2024-09-19T08:58:25.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.839+0000 7f6a7ac5e640 1 Processor -- start 2024-09-19T08:58:25.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.839+0000 7f6a7ac5e640 1 -- start start 2024-09-19T08:58:25.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.839+0000 7f6a7ac5e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a74101df0 0x7f6a74198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:25.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.839+0000 7f6a7ac5e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a741027a0 0x7f6a74198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:25.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.839+0000 7f6a7ac5e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a74106f20 0x7f6a7419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:25.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.840+0000 7f6a7ac5e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6a74111fa0 con 0x7f6a741027a0 2024-09-19T08:58:25.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.840+0000 7f6a7ac5e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6a74111e20 con 0x7f6a74101df0 2024-09-19T08:58:25.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.840+0000 7f6a7ac5e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6a74112120 con 0x7f6a74106f20 2024-09-19T08:58:25.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.840+0000 7f6a73fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a741027a0 0x7f6a74198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:25.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.840+0000 7f6a73fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a741027a0 0x7f6a74198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:34974/0 (socket says 172.21.15.164:34974) 2024-09-19T08:58:25.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.840+0000 7f6a73fff640 1 -- 172.21.15.164:0/2400902451 learned_addr learned my addr 172.21.15.164:0/2400902451 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-19T08:58:25.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.840+0000 7f6a789fa640 1 --2- 172.21.15.164:0/2400902451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a74101df0 0x7f6a74198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:25.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.840+0000 7f6a791fb640 1 --2- 172.21.15.164:0/2400902451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a74106f20 0x7f6a7419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:25.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.840+0000 7f6a73fff640 1 -- 172.21.15.164:0/2400902451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a74106f20 msgr2=0x7f6a7419ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:25.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.840+0000 7f6a73fff640 1 --2- 172.21.15.164:0/2400902451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a74106f20 0x7f6a7419ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:25.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.840+0000 7f6a73fff640 1 -- 172.21.15.164:0/2400902451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a74101df0 msgr2=0x7f6a74198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:25.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.840+0000 7f6a73fff640 1 --2- 172.21.15.164:0/2400902451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a74101df0 0x7f6a74198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:25.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.840+0000 7f6a73fff640 1 -- 172.21.15.164:0/2400902451 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a7419d3d0 con 0x7f6a741027a0 2024-09-19T08:58:25.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.840+0000 7f6a789fa640 1 --2- 172.21.15.164:0/2400902451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a74101df0 0x7f6a74198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:58:25.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.840+0000 7f6a791fb640 1 --2- 172.21.15.164:0/2400902451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a74106f20 0x7f6a7419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:58:25.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.840+0000 7f6a73fff640 1 --2- 172.21.15.164:0/2400902451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a741027a0 0x7f6a74198960 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000b4a0 tx=0x7f6a6000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:58:25.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.840+0000 7f6a71ffb640 1 -- 172.21.15.164:0/2400902451 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a60014070 con 0x7f6a741027a0 2024-09-19T08:58:25.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.840+0000 7f6a7ac5e640 1 -- 172.21.15.164:0/2400902451 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a7419d6c0 con 0x7f6a741027a0 2024-09-19T08:58:25.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.840+0000 7f6a71ffb640 1 -- 172.21.15.164:0/2400902451 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a60010af0 con 0x7f6a741027a0 2024-09-19T08:58:25.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.840+0000 7f6a71ffb640 1 -- 172.21.15.164:0/2400902451 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a60017d10 con 0x7f6a741027a0 2024-09-19T08:58:25.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.841+0000 7f6a7ac5e640 1 -- 172.21.15.164:0/2400902451 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a741a4fa0 con 0x7f6a741027a0 2024-09-19T08:58:25.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.842+0000 7f6a7ac5e640 1 -- 172.21.15.164:0/2400902451 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a741021d0 con 0x7f6a741027a0 2024-09-19T08:58:25.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.842+0000 7f6a71ffb640 1 -- 172.21.15.164:0/2400902451 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f6a60017070 con 0x7f6a741027a0 2024-09-19T08:58:25.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.843+0000 7f6a71ffb640 1 --2- 172.21.15.164:0/2400902451 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f6a640a33b0 0x7f6a640a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:58:25.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.843+0000 7f6a789fa640 1 --2- 172.21.15.164:0/2400902451 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f6a640a33b0 0x7f6a640a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:58:25.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.843+0000 7f6a71ffb640 1 -- 172.21.15.164:0/2400902451 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8345+0+0 (secure 0 0 0) 0x7f6a600cf050 con 0x7f6a741027a0 2024-09-19T08:58:25.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.844+0000 7f6a789fa640 1 --2- 172.21.15.164:0/2400902451 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f6a640a33b0 0x7f6a640a5870 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6a680058c0 tx=0x7f6a68005ec0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:58:25.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:25.847+0000 7f6a71ffb640 1 -- 172.21.15.164:0/2400902451 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6a6009bbf0 con 0x7f6a741027a0 2024-09-19T08:58:26.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:26.016+0000 7f6a7ac5e640 1 -- 172.21.15.164:0/2400902451 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6a7402d7c0 con 0x7f6a741027a0 2024-09-19T08:58:26.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:26.296+0000 7f6a71ffb640 1 -- 172.21.15.164:0/2400902451 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v100) ==== 169+0+0 (secure 0 0 0) 0x7f6a6009b4f0 con 0x7f6a741027a0 2024-09-19T08:58:26.296 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_data' removed 2024-09-19T08:58:26.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:26.299+0000 7f6a7ac5e640 1 -- 172.21.15.164:0/2400902451 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f6a640a33b0 msgr2=0x7f6a640a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:26.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:26.299+0000 7f6a7ac5e640 1 --2- 172.21.15.164:0/2400902451 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f6a640a33b0 0x7f6a640a5870 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6a680058c0 tx=0x7f6a68005ec0 comp rx=0 tx=0).stop 2024-09-19T08:58:26.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:26.299+0000 7f6a7ac5e640 1 -- 172.21.15.164:0/2400902451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a741027a0 msgr2=0x7f6a74198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:58:26.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:26.299+0000 7f6a7ac5e640 1 --2- 172.21.15.164:0/2400902451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a741027a0 0x7f6a74198960 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000b4a0 tx=0x7f6a6000b960 comp rx=0 tx=0).stop 2024-09-19T08:58:26.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:26.299+0000 7f6a7ac5e640 1 -- 172.21.15.164:0/2400902451 shutdown_connections 2024-09-19T08:58:26.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:26.299+0000 7f6a7ac5e640 1 --2- 172.21.15.164:0/2400902451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a74106f20 0x7f6a7419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:26.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:26.299+0000 7f6a7ac5e640 1 --2- 172.21.15.164:0/2400902451 >> [v2:172.21.15.164:6800/173486514,v1:172.21.15.164:6801/173486514] conn(0x7f6a640a33b0 0x7f6a640a5870 secure :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6a680058c0 tx=0x7f6a68005ec0 comp rx=0 tx=0).stop 2024-09-19T08:58:26.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:26.299+0000 7f6a7ac5e640 1 --2- 172.21.15.164:0/2400902451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a741027a0 0x7f6a74198960 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:26.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:26.299+0000 7f6a7ac5e640 1 --2- 172.21.15.164:0/2400902451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a74101df0 0x7f6a74198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:58:26.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:26.299+0000 7f6a7ac5e640 1 -- 172.21.15.164:0/2400902451 >> 172.21.15.164:0/2400902451 conn(0x7f6a740fd8b0 msgr2=0x7f6a740feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:58:26.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:26.300+0000 7f6a7ac5e640 1 -- 172.21.15.164:0/2400902451 shutdown_connections 2024-09-19T08:58:26.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-19T08:58:26.300+0000 7f6a7ac5e640 1 -- 172.21.15.164:0/2400902451 wait complete. 2024-09-19T08:58:26.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:26 smithi170 ceph-mon[35072]: pgmap v397: 65 pgs: 65 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:58:26.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:26 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2400902451' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T08:58:26.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:26 smithi173 ceph-mon[35306]: pgmap v397: 65 pgs: 65 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:58:26.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:26 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2400902451' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T08:58:26.775 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:26 smithi164 ceph-mon[30928]: pgmap v397: 65 pgs: 65 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T08:58:26.775 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:26 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2400902451' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T08:58:27.104 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T08:58:27.131 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T08:58:27.161 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T08:58:27.189 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-19T08:58:27.189 DEBUG:teuthology.orchestra.run.smithi164:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T08:58:27.205 DEBUG:teuthology.orchestra.run.smithi170:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T08:58:27.221 DEBUG:teuthology.orchestra.run.smithi173:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T08:58:27.247 INFO:tasks.cephadm:Stopping all daemons... 2024-09-19T08:58:27.247 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-19T08:58:27.247 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mon.a 2024-09-19T08:58:27.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:27 smithi164 systemd[1]: Stopping Ceph mon.a for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T08:58:27.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:27 smithi164 ceph-mon[30928]: from='client.? 172.21.15.164:0/2400902451' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T08:58:27.532 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:27 smithi164 ceph-mon[30928]: osdmap e100: 12 total, 12 up, 12 in 2024-09-19T08:58:27.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:27 smithi170 ceph-mon[35072]: from='client.? 172.21.15.164:0/2400902451' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T08:58:27.625 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:27 smithi170 ceph-mon[35072]: osdmap e100: 12 total, 12 up, 12 in 2024-09-19T08:58:27.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:27 smithi173 ceph-mon[35306]: from='client.? 172.21.15.164:0/2400902451' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T08:58:27.660 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:27 smithi173 ceph-mon[35306]: osdmap e100: 12 total, 12 up, 12 in 2024-09-19T08:58:27.998 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:27 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mon-a[30924]: 2024-09-19T08:58:27.581+0000 7fe1de5d7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:58:27.998 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:27 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mon-a[30924]: 2024-09-19T08:58:27.581+0000 7fe1de5d7640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-19T08:58:27.998 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:27 smithi164 podman[113721]: 2024-09-19 08:58:27.706508235 +0000 UTC m=+0.256233757 container died 3264bf1a15c86784721d1d56060d27fad20b2b359fc13c3f3db50fb14a7cbd12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-19T08:58:28.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:28 smithi164 podman[113721]: 2024-09-19 08:58:28.089615573 +0000 UTC m=+0.639341073 container cleanup 3264bf1a15c86784721d1d56060d27fad20b2b359fc13c3f3db50fb14a7cbd12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-19T08:58:28.282 INFO:journalctl@ceph.mon.a.smithi164.stdout:Sep 19 08:58:28 smithi164 bash[113721]: ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mon-a 2024-09-19T08:58:28.553 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mon.a.service' 2024-09-19T08:58:28.591 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:58:28.591 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-19T08:58:28.591 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-19T08:58:28.591 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mon.b 2024-09-19T08:58:28.874 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:28 smithi170 systemd[1]: Stopping Ceph mon.b for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T08:58:29.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:29 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mon-b[35068]: 2024-09-19T08:58:29.519+0000 7f2b830d5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:58:29.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:29 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mon-b[35068]: 2024-09-19T08:58:29.519+0000 7f2b830d5640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-19T08:58:29.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:29 smithi170 podman[70882]: 2024-09-19 08:58:29.571393204 +0000 UTC m=+0.721421727 container died 746ea7b73d5526c280f8a0547c32224a48b6efd5c8e31bb40fd8754102af04df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mon-b, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T08:58:30.237 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:29 smithi170 podman[70882]: 2024-09-19 08:58:29.978350208 +0000 UTC m=+1.128378721 container cleanup 746ea7b73d5526c280f8a0547c32224a48b6efd5c8e31bb40fd8754102af04df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-19T08:58:30.238 INFO:journalctl@ceph.mon.b.smithi170.stdout:Sep 19 08:58:29 smithi170 bash[70882]: ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mon-b 2024-09-19T08:58:30.411 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mon.b.service' 2024-09-19T08:58:30.457 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:58:30.458 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-19T08:58:30.458 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-19T08:58:30.458 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mon.c 2024-09-19T08:58:30.769 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:30 smithi173 systemd[1]: Stopping Ceph mon.c for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T08:58:31.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:30 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mon-c[35302]: 2024-09-19T08:58:30.767+0000 7ff6b59e7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:58:31.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:30 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mon-c[35302]: 2024-09-19T08:58:30.767+0000 7ff6b59e7640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-19T08:58:31.161 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:30 smithi173 podman[71278]: 2024-09-19 08:58:30.891116215 +0000 UTC m=+0.233055888 container died e5ca074777f9e05eca868f38677adcb8fe5d452cfc5a7171b45fc38a08928a2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mon-c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-19T08:58:31.545 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:31 smithi173 podman[71278]: 2024-09-19 08:58:31.294020536 +0000 UTC m=+0.635960208 container cleanup e5ca074777f9e05eca868f38677adcb8fe5d452cfc5a7171b45fc38a08928a2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mon-c, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-19T08:58:31.545 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 19 08:58:31 smithi173 bash[71278]: ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mon-c 2024-09-19T08:58:31.721 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mon.c.service' 2024-09-19T08:58:31.769 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:58:31.769 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-19T08:58:31.769 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-19T08:58:31.769 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mgr.x 2024-09-19T08:58:32.094 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:58:31 smithi164 systemd[1]: Stopping Ceph mgr.x for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T08:58:32.411 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:58:32 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x[31154]: 2024-09-19T08:58:32.094+0000 7fb09cee1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:58:32.412 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:58:32 smithi164 podman[113837]: 2024-09-19 08:58:32.125225211 +0000 UTC m=+0.162630044 container died 3287a3ff4ddf6c06dadb892475c84695b5f03768a0b8b72ef2e1d4ebbee6fb44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T08:58:32.745 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:58:32 smithi164 podman[113837]: 2024-09-19 08:58:32.510988316 +0000 UTC m=+0.548393126 container cleanup 3287a3ff4ddf6c06dadb892475c84695b5f03768a0b8b72ef2e1d4ebbee6fb44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-19T08:58:32.745 INFO:journalctl@ceph.mgr.x.smithi164.stdout:Sep 19 08:58:32 smithi164 bash[113837]: ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-x 2024-09-19T08:58:32.871 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-19T08:58:32.907 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:58:32.907 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-19T08:58:32.907 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-19T08:58:32.908 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mgr.y 2024-09-19T08:58:33.261 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:58:32 smithi170 systemd[1]: Stopping Ceph mgr.y for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T08:58:33.558 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:58:33 smithi170 podman[71005]: 2024-09-19 08:58:33.262042378 +0000 UTC m=+0.148586288 container died 5b8dfecde34d1590f2aa36d0bdf618296d34299caea028440196db3201f5d674 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-19T08:58:33.875 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:58:33 smithi170 podman[71005]: 2024-09-19 08:58:33.65867912 +0000 UTC m=+0.545223060 container cleanup 5b8dfecde34d1590f2aa36d0bdf618296d34299caea028440196db3201f5d674 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux ) 2024-09-19T08:58:33.875 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Sep 19 08:58:33 smithi170 bash[71005]: ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-y 2024-09-19T08:58:34.430 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-19T08:58:34.464 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:58:34.465 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-19T08:58:34.465 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-19T08:58:34.465 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mgr.z 2024-09-19T08:58:34.828 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:58:34 smithi173 systemd[1]: Stopping Ceph mgr.z for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T08:58:35.117 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:58:34 smithi173 podman[71402]: 2024-09-19 08:58:34.828622686 +0000 UTC m=+0.169596375 container died a9e0ddc90b8e785ffaed7cb2c29960990abe6aace148f2b497292ba5996d239e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T08:58:35.410 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:58:35 smithi173 podman[71402]: 2024-09-19 08:58:35.21705348 +0000 UTC m=+0.558027158 container cleanup a9e0ddc90b8e785ffaed7cb2c29960990abe6aace148f2b497292ba5996d239e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-19T08:58:35.411 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Sep 19 08:58:35 smithi173 bash[71402]: ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mgr-z 2024-09-19T08:58:35.652 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-19T08:58:35.688 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:58:35.688 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-19T08:58:35.689 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-19T08:58:35.689 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.0 2024-09-19T08:58:35.989 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 19 08:58:35 smithi164 systemd[1]: Stopping Ceph osd.0 for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T08:58:36.282 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 19 08:58:35 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-0[44856]: 2024-09-19T08:58:35.990+0000 7f4018f75640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:58:36.282 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 19 08:58:35 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-0[44856]: 2024-09-19T08:58:35.990+0000 7f4018f75640 -1 osd.0 100 *** Got signal Terminated *** 2024-09-19T08:58:36.282 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 19 08:58:35 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-0[44856]: 2024-09-19T08:58:35.990+0000 7f4018f75640 -1 osd.0 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T08:58:42.032 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 19 08:58:41 smithi164 podman[113960]: 2024-09-19 08:58:41.550922261 +0000 UTC m=+5.670220592 container died 8bea16b48f18593cd7a7c4cbfd4322231fbb423d0fd341a808e3e6e354e5222d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-0, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4) 2024-09-19T08:58:42.407 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 19 08:58:42 smithi164 podman[113960]: 2024-09-19 08:58:42.12420563 +0000 UTC m=+6.243503964 container cleanup 8bea16b48f18593cd7a7c4cbfd4322231fbb423d0fd341a808e3e6e354e5222d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-0, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4) 2024-09-19T08:58:42.408 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 19 08:58:42 smithi164 bash[113960]: ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-0 2024-09-19T08:58:42.747 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 19 08:58:42 smithi164 podman[113976]: 2024-09-19 08:58:42.408789699 +0000 UTC m=+0.859793128 container remove 8bea16b48f18593cd7a7c4cbfd4322231fbb423d0fd341a808e3e6e354e5222d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-2f599a4) 2024-09-19T08:58:43.032 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 19 08:58:42 smithi164 podman[114070]: 2024-09-19 08:58:42.651201999 +0000 UTC m=+0.016023519 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:58:43.032 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 19 08:58:42 smithi164 podman[114070]: 2024-09-19 08:58:42.76814434 +0000 UTC m=+0.132965860 container create 04c2cb65564eb9c84ca1a49fe8240969f7b4171085ace5758bf94161f18ae663 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-19T08:58:43.532 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 19 08:58:43 smithi164 podman[114070]: 2024-09-19 08:58:43.037298441 +0000 UTC m=+0.402119959 container init 04c2cb65564eb9c84ca1a49fe8240969f7b4171085ace5758bf94161f18ae663 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-19T08:58:43.533 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 19 08:58:43 smithi164 podman[114070]: 2024-09-19 08:58:43.040640964 +0000 UTC m=+0.405462486 container start 04c2cb65564eb9c84ca1a49fe8240969f7b4171085ace5758bf94161f18ae663 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-19T08:58:43.533 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 19 08:58:43 smithi164 podman[114070]: 2024-09-19 08:58:43.129100734 +0000 UTC m=+0.493922270 container attach 04c2cb65564eb9c84ca1a49fe8240969f7b4171085ace5758bf94161f18ae663 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-19T08:58:43.533 INFO:journalctl@ceph.osd.0.smithi164.stdout:Sep 19 08:58:43 smithi164 podman[114070]: 2024-09-19 08:58:43.245377509 +0000 UTC m=+0.610199028 container died 04c2cb65564eb9c84ca1a49fe8240969f7b4171085ace5758bf94161f18ae663 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-19T08:58:44.162 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.0.service' 2024-09-19T08:58:44.209 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:58:44.210 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-19T08:58:44.210 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-19T08:58:44.210 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.1 2024-09-19T08:58:44.538 INFO:journalctl@ceph.osd.1.smithi170.stdout:Sep 19 08:58:44 smithi170 systemd[1]: Stopping Ceph osd.1 for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T08:58:44.875 INFO:journalctl@ceph.osd.1.smithi170.stdout:Sep 19 08:58:44 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-1[39460]: 2024-09-19T08:58:44.538+0000 7f4b7e420640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:58:44.875 INFO:journalctl@ceph.osd.1.smithi170.stdout:Sep 19 08:58:44 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-1[39460]: 2024-09-19T08:58:44.538+0000 7f4b7e420640 -1 osd.1 100 *** Got signal Terminated *** 2024-09-19T08:58:44.875 INFO:journalctl@ceph.osd.1.smithi170.stdout:Sep 19 08:58:44 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-1[39460]: 2024-09-19T08:58:44.538+0000 7f4b7e420640 -1 osd.1 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T08:58:50.455 INFO:journalctl@ceph.osd.1.smithi170.stdout:Sep 19 08:58:50 smithi170 podman[71127]: 2024-09-19 08:58:50.172134903 +0000 UTC m=+5.764214731 container died 333615f943974f39a2bda74ff0a79dc5b8673e1d1ba2d03c5d4bb184f227a287 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-1, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4) 2024-09-19T08:58:50.734 INFO:journalctl@ceph.osd.1.smithi170.stdout:Sep 19 08:58:50 smithi170 podman[71127]: 2024-09-19 08:58:50.580207457 +0000 UTC m=+6.172287282 container cleanup 333615f943974f39a2bda74ff0a79dc5b8673e1d1ba2d03c5d4bb184f227a287 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-1, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4) 2024-09-19T08:58:50.734 INFO:journalctl@ceph.osd.1.smithi170.stdout:Sep 19 08:58:50 smithi170 bash[71127]: ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-1 2024-09-19T08:58:51.124 INFO:journalctl@ceph.osd.1.smithi170.stdout:Sep 19 08:58:50 smithi170 podman[71145]: 2024-09-19 08:58:50.832021423 +0000 UTC m=+0.656527328 container remove 333615f943974f39a2bda74ff0a79dc5b8673e1d1ba2d03c5d4bb184f227a287 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-1, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-19T08:58:51.476 INFO:journalctl@ceph.osd.1.smithi170.stdout:Sep 19 08:58:51 smithi170 podman[71242]: 2024-09-19 08:58:51.072430025 +0000 UTC m=+0.016391768 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:58:51.476 INFO:journalctl@ceph.osd.1.smithi170.stdout:Sep 19 08:58:51 smithi170 podman[71242]: 2024-09-19 08:58:51.191481792 +0000 UTC m=+0.135443526 container create eae4cdc595e21e109da52cf5ae3495518649b6f5de7fe6aee03a6322522e79f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-19T08:58:51.476 INFO:journalctl@ceph.osd.1.smithi170.stdout:Sep 19 08:58:51 smithi170 podman[71242]: 2024-09-19 08:58:51.477426722 +0000 UTC m=+0.421388455 container init eae4cdc595e21e109da52cf5ae3495518649b6f5de7fe6aee03a6322522e79f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-19T08:58:51.751 INFO:journalctl@ceph.osd.1.smithi170.stdout:Sep 19 08:58:51 smithi170 podman[71242]: 2024-09-19 08:58:51.481504702 +0000 UTC m=+0.425466435 container start eae4cdc595e21e109da52cf5ae3495518649b6f5de7fe6aee03a6322522e79f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-19T08:58:51.751 INFO:journalctl@ceph.osd.1.smithi170.stdout:Sep 19 08:58:51 smithi170 podman[71242]: 2024-09-19 08:58:51.544093756 +0000 UTC m=+0.488055508 container attach eae4cdc595e21e109da52cf5ae3495518649b6f5de7fe6aee03a6322522e79f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-1-deactivate, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-19T08:58:51.751 INFO:journalctl@ceph.osd.1.smithi170.stdout:Sep 19 08:58:51 smithi170 podman[71242]: 2024-09-19 08:58:51.698604783 +0000 UTC m=+0.642566525 container died eae4cdc595e21e109da52cf5ae3495518649b6f5de7fe6aee03a6322522e79f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4) 2024-09-19T08:58:52.292 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.1.service' 2024-09-19T08:58:52.333 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:58:52.333 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-19T08:58:52.333 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-19T08:58:52.333 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.2 2024-09-19T08:58:52.660 INFO:journalctl@ceph.osd.2.smithi173.stdout:Sep 19 08:58:52 smithi173 systemd[1]: Stopping Ceph osd.2 for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T08:58:53.042 INFO:journalctl@ceph.osd.2.smithi173.stdout:Sep 19 08:58:52 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-2[40262]: 2024-09-19T08:58:52.747+0000 7f43be4ad640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:58:53.042 INFO:journalctl@ceph.osd.2.smithi173.stdout:Sep 19 08:58:52 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-2[40262]: 2024-09-19T08:58:52.747+0000 7f43be4ad640 -1 osd.2 100 *** Got signal Terminated *** 2024-09-19T08:58:53.042 INFO:journalctl@ceph.osd.2.smithi173.stdout:Sep 19 08:58:52 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-2[40262]: 2024-09-19T08:58:52.747+0000 7f43be4ad640 -1 osd.2 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T08:58:58.623 INFO:journalctl@ceph.osd.2.smithi173.stdout:Sep 19 08:58:58 smithi173 podman[71528]: 2024-09-19 08:58:58.342660833 +0000 UTC m=+5.811028823 container died be62044fbceb295a48d6fcdd6d8f36bf8e68245779904d2b354db892178053c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T08:58:58.910 INFO:journalctl@ceph.osd.2.smithi173.stdout:Sep 19 08:58:58 smithi173 podman[71528]: 2024-09-19 08:58:58.722975497 +0000 UTC m=+6.191343488 container cleanup be62044fbceb295a48d6fcdd6d8f36bf8e68245779904d2b354db892178053c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-19T08:58:58.911 INFO:journalctl@ceph.osd.2.smithi173.stdout:Sep 19 08:58:58 smithi173 bash[71528]: ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-2 2024-09-19T08:58:59.388 INFO:journalctl@ceph.osd.2.smithi173.stdout:Sep 19 08:58:59 smithi173 podman[71555]: 2024-09-19 08:58:59.041630416 +0000 UTC m=+0.695353247 container remove be62044fbceb295a48d6fcdd6d8f36bf8e68245779904d2b354db892178053c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T08:58:59.661 INFO:journalctl@ceph.osd.2.smithi173.stdout:Sep 19 08:58:59 smithi173 podman[71642]: 2024-09-19 08:58:59.292601015 +0000 UTC m=+0.016096189 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:58:59.661 INFO:journalctl@ceph.osd.2.smithi173.stdout:Sep 19 08:58:59 smithi173 podman[71642]: 2024-09-19 08:58:59.392677406 +0000 UTC m=+0.116172617 container create 1b008ea5165b01b24799a4fdef36c228f663684f40f7e828bbb21ba6a41707fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4) 2024-09-19T08:58:59.661 INFO:journalctl@ceph.osd.2.smithi173.stdout:Sep 19 08:58:59 smithi173 podman[71642]: 2024-09-19 08:58:59.620409094 +0000 UTC m=+0.343904259 container init 1b008ea5165b01b24799a4fdef36c228f663684f40f7e828bbb21ba6a41707fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-19T08:58:59.661 INFO:journalctl@ceph.osd.2.smithi173.stdout:Sep 19 08:58:59 smithi173 podman[71642]: 2024-09-19 08:58:59.623731939 +0000 UTC m=+0.347227116 container start 1b008ea5165b01b24799a4fdef36c228f663684f40f7e828bbb21ba6a41707fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-19T08:59:00.160 INFO:journalctl@ceph.osd.2.smithi173.stdout:Sep 19 08:58:59 smithi173 podman[71642]: 2024-09-19 08:58:59.72091092 +0000 UTC m=+0.444406098 container attach 1b008ea5165b01b24799a4fdef36c228f663684f40f7e828bbb21ba6a41707fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T08:59:00.161 INFO:journalctl@ceph.osd.2.smithi173.stdout:Sep 19 08:58:59 smithi173 podman[71642]: 2024-09-19 08:58:59.811916344 +0000 UTC m=+0.535411519 container died 1b008ea5165b01b24799a4fdef36c228f663684f40f7e828bbb21ba6a41707fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T08:59:00.510 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.2.service' 2024-09-19T08:59:00.549 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:59:00.549 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-19T08:59:00.550 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-19T08:59:00.550 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.3 2024-09-19T08:59:00.868 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 19 08:59:00 smithi164 systemd[1]: Stopping Ceph osd.3 for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T08:59:01.281 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 19 08:59:00 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-3[51564]: 2024-09-19T08:59:00.867+0000 7f076c7c7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:59:01.282 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 19 08:59:00 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-3[51564]: 2024-09-19T08:59:00.867+0000 7f076c7c7640 -1 osd.3 100 *** Got signal Terminated *** 2024-09-19T08:59:01.282 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 19 08:59:00 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-3[51564]: 2024-09-19T08:59:00.867+0000 7f076c7c7640 -1 osd.3 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T08:59:04.375 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:04 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:04.103+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:05.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:04 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:04.760+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:05.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:05 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:05.151+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:06.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:05 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:05.710+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:06.160 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:05 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:05.758+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:06.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:06 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:06.143+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:06.660 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:06 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:59:06.279+0000 7fe9f5e62640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.609349+0000 front 2024-09-19T08:58:40.609376+0000 (oldest deadline 2024-09-19T08:59:05.909533+0000) 2024-09-19T08:59:06.782 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:06 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:06.700+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:06.782 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 19 08:59:06 smithi164 podman[114171]: 2024-09-19 08:59:06.506087845 +0000 UTC m=+5.754917786 container died 1dae88379a9c2bbb2dfe78958af5b4064a6b71805d09eb211d3e0184d80abe31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T08:59:07.160 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:06 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:06.791+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:07.269 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 19 08:59:07 smithi164 podman[114171]: 2024-09-19 08:59:07.002046244 +0000 UTC m=+6.250876227 container cleanup 1dae88379a9c2bbb2dfe78958af5b4064a6b71805d09eb211d3e0184d80abe31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-19T08:59:07.270 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 19 08:59:07 smithi164 bash[114171]: ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-3 2024-09-19T08:59:07.270 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:06 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:06.858+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.950897+0000 front 2024-09-19T08:58:40.951096+0000 (oldest deadline 2024-09-19T08:59:06.251090+0000) 2024-09-19T08:59:07.532 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 19 08:59:07 smithi164 podman[114187]: 2024-09-19 08:59:07.27023259 +0000 UTC m=+0.765534682 container remove 1dae88379a9c2bbb2dfe78958af5b4064a6b71805d09eb211d3e0184d80abe31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-3, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-19T08:59:07.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:07 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:07.171+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:07.660 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:07 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:59:07.253+0000 7fe9f5e62640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.609349+0000 front 2024-09-19T08:58:40.609376+0000 (oldest deadline 2024-09-19T08:59:05.909533+0000) 2024-09-19T08:59:07.881 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:07 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:07.702+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:07.881 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 19 08:59:07 smithi164 podman[114276]: 2024-09-19 08:59:07.508378191 +0000 UTC m=+0.016031765 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:59:07.881 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 19 08:59:07 smithi164 podman[114276]: 2024-09-19 08:59:07.62953477 +0000 UTC m=+0.137188329 container create b5de745f371db098ff466462891f85e30856a1937d2e89b43f13d44e42000d4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-19T08:59:07.881 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 19 08:59:07 smithi164 podman[114276]: 2024-09-19 08:59:07.882082486 +0000 UTC m=+0.389736050 container init b5de745f371db098ff466462891f85e30856a1937d2e89b43f13d44e42000d4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True) 2024-09-19T08:59:08.160 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:07 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:07.769+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:08.282 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:07 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:07.905+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.950897+0000 front 2024-09-19T08:58:40.951096+0000 (oldest deadline 2024-09-19T08:59:06.251090+0000) 2024-09-19T08:59:08.282 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 19 08:59:07 smithi164 podman[114276]: 2024-09-19 08:59:07.885507527 +0000 UTC m=+0.393161085 container start b5de745f371db098ff466462891f85e30856a1937d2e89b43f13d44e42000d4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-19T08:59:08.283 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 19 08:59:07 smithi164 podman[114276]: 2024-09-19 08:59:07.990381201 +0000 UTC m=+0.498034762 container attach b5de745f371db098ff466462891f85e30856a1937d2e89b43f13d44e42000d4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-19T08:59:08.283 INFO:journalctl@ceph.osd.3.smithi164.stdout:Sep 19 08:59:08 smithi164 podman[114276]: 2024-09-19 08:59:08.068990894 +0000 UTC m=+0.576644466 container died b5de745f371db098ff466462891f85e30856a1937d2e89b43f13d44e42000d4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-19T08:59:08.624 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:08 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:08.196+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:08.660 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:08 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:59:08.283+0000 7fe9f5e62640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.609349+0000 front 2024-09-19T08:58:40.609376+0000 (oldest deadline 2024-09-19T08:59:05.909533+0000) 2024-09-19T08:59:08.716 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.3.service' 2024-09-19T08:59:08.755 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:59:08.755 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-19T08:59:08.755 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-19T08:59:08.755 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.4 2024-09-19T08:59:09.033 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:08 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:08.705+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:09.034 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:08 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:08.876+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.950897+0000 front 2024-09-19T08:58:40.951096+0000 (oldest deadline 2024-09-19T08:59:06.251090+0000) 2024-09-19T08:59:09.067 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 19 08:59:08 smithi170 systemd[1]: Stopping Ceph osd.4 for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T08:59:09.160 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:08 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:08.778+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:09.375 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:09 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:09.163+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:09.375 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 19 08:59:09 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-4[46203]: 2024-09-19T08:59:09.067+0000 7f12e2a76640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:59:09.375 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 19 08:59:09 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-4[46203]: 2024-09-19T08:59:09.067+0000 7f12e2a76640 -1 osd.4 100 *** Got signal Terminated *** 2024-09-19T08:59:09.375 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 19 08:59:09 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-4[46203]: 2024-09-19T08:59:09.067+0000 7f12e2a76640 -1 osd.4 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T08:59:09.660 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:09 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:59:09.288+0000 7fe9f5e62640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.609349+0000 front 2024-09-19T08:58:40.609376+0000 (oldest deadline 2024-09-19T08:59:05.909533+0000) 2024-09-19T08:59:10.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:09 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:09.834+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.950897+0000 front 2024-09-19T08:58:40.951096+0000 (oldest deadline 2024-09-19T08:59:06.251090+0000) 2024-09-19T08:59:10.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:09 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:09.707+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:10.160 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:09 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:09.824+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:10.558 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:10 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:59:10.239+0000 7fe9f5e62640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.609349+0000 front 2024-09-19T08:58:40.609376+0000 (oldest deadline 2024-09-19T08:59:05.909533+0000) 2024-09-19T08:59:10.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:10 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:10.194+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:10.910 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:10 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:10.777+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:11.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:10 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:10.659+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:11.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:10 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:10.850+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.950897+0000 front 2024-09-19T08:58:40.951096+0000 (oldest deadline 2024-09-19T08:59:06.251090+0000) 2024-09-19T08:59:11.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:10 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:10.850+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.251773+0000 front 2024-09-19T08:58:46.251733+0000 (oldest deadline 2024-09-19T08:59:10.351932+0000) 2024-09-19T08:59:11.191 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 19 08:59:10 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-4[46203]: 2024-09-19T08:59:10.899+0000 7f12df08c640 -1 osd.4 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:44.890416+0000 front 2024-09-19T08:58:44.890451+0000 (oldest deadline 2024-09-19T08:59:10.790642+0000) 2024-09-19T08:59:11.624 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:11 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:11.191+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:11.660 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:11 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:59:11.237+0000 7fe9f5e62640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.609349+0000 front 2024-09-19T08:58:40.609376+0000 (oldest deadline 2024-09-19T08:59:05.909533+0000) 2024-09-19T08:59:11.660 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:11 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:59:11.237+0000 7fe9f5e62640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.810738+0000 front 2024-09-19T08:58:48.810684+0000 (oldest deadline 2024-09-19T08:59:11.110573+0000) 2024-09-19T08:59:12.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:11 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:11.685+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:12.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:11 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:11.844+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.950897+0000 front 2024-09-19T08:58:40.951096+0000 (oldest deadline 2024-09-19T08:59:06.251090+0000) 2024-09-19T08:59:12.033 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:11 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:11.844+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.251773+0000 front 2024-09-19T08:58:46.251733+0000 (oldest deadline 2024-09-19T08:59:10.351932+0000) 2024-09-19T08:59:12.125 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 19 08:59:11 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-4[46203]: 2024-09-19T08:59:11.926+0000 7f12df08c640 -1 osd.4 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:44.890416+0000 front 2024-09-19T08:58:44.890451+0000 (oldest deadline 2024-09-19T08:59:10.790642+0000) 2024-09-19T08:59:12.125 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:11 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:11.855+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.460189+0000 front 2024-09-19T08:58:49.460291+0000 (oldest deadline 2024-09-19T08:59:11.660880+0000) 2024-09-19T08:59:12.160 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:11 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:11.789+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:12.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:12 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:12.198+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:12.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:12 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:12.198+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:12.660 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:12 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:59:12.232+0000 7fe9f5e62640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.609349+0000 front 2024-09-19T08:58:40.609376+0000 (oldest deadline 2024-09-19T08:59:05.909533+0000) 2024-09-19T08:59:12.661 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:12 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:59:12.232+0000 7fe9f5e62640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.810738+0000 front 2024-09-19T08:58:48.810684+0000 (oldest deadline 2024-09-19T08:59:11.110573+0000) 2024-09-19T08:59:13.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:12 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:12.878+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.950897+0000 front 2024-09-19T08:58:40.951096+0000 (oldest deadline 2024-09-19T08:59:06.251090+0000) 2024-09-19T08:59:13.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:12 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:12.878+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.251773+0000 front 2024-09-19T08:58:46.251733+0000 (oldest deadline 2024-09-19T08:59:10.351932+0000) 2024-09-19T08:59:13.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:12 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:12.675+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:13.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:12 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:12.675+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:13.160 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:12 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:12.782+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:13.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:12 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:12.782+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:13.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:12 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:12.751+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:13.243 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:12 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:12.883+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.460189+0000 front 2024-09-19T08:58:49.460291+0000 (oldest deadline 2024-09-19T08:59:11.660880+0000) 2024-09-19T08:59:13.243 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 19 08:59:12 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-4[46203]: 2024-09-19T08:59:12.926+0000 7f12df08c640 -1 osd.4 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:44.890416+0000 front 2024-09-19T08:58:44.890451+0000 (oldest deadline 2024-09-19T08:59:10.790642+0000) 2024-09-19T08:59:13.624 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:13 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:13.243+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:13.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:13 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:13.243+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:13.660 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:13 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:59:13.197+0000 7fe9f5e62640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.609349+0000 front 2024-09-19T08:58:40.609376+0000 (oldest deadline 2024-09-19T08:59:05.909533+0000) 2024-09-19T08:59:13.661 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:13 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:59:13.197+0000 7fe9f5e62640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.810738+0000 front 2024-09-19T08:58:48.810684+0000 (oldest deadline 2024-09-19T08:59:11.110573+0000) 2024-09-19T08:59:14.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:13 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:13.884+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.950897+0000 front 2024-09-19T08:58:40.951096+0000 (oldest deadline 2024-09-19T08:59:06.251090+0000) 2024-09-19T08:59:14.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:13 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:13.885+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.251773+0000 front 2024-09-19T08:58:46.251733+0000 (oldest deadline 2024-09-19T08:59:10.351932+0000) 2024-09-19T08:59:14.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:13 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:13.655+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:14.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:13 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:13.655+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:14.125 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:13 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:13.847+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.460189+0000 front 2024-09-19T08:58:49.460291+0000 (oldest deadline 2024-09-19T08:59:11.660880+0000) 2024-09-19T08:59:14.125 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 19 08:59:13 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-4[46203]: 2024-09-19T08:59:13.925+0000 7f12df08c640 -1 osd.4 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:44.890416+0000 front 2024-09-19T08:58:44.890451+0000 (oldest deadline 2024-09-19T08:59:10.790642+0000) 2024-09-19T08:59:14.157 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:13 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:13.721+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:14.157 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:13 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:13.802+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:14.157 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:13 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:13.802+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:14.410 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:14 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:59:14.157+0000 7fe9f5e62640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.609349+0000 front 2024-09-19T08:58:40.609376+0000 (oldest deadline 2024-09-19T08:59:05.909533+0000) 2024-09-19T08:59:14.411 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:14 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:59:14.157+0000 7fe9f5e62640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.810738+0000 front 2024-09-19T08:58:48.810684+0000 (oldest deadline 2024-09-19T08:59:11.110573+0000) 2024-09-19T08:59:14.625 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 19 08:59:14 smithi170 podman[71347]: 2024-09-19 08:59:14.468463266 +0000 UTC m=+5.518839577 container died 4879674e2cb200c9392c1bd8dfc315cf75cceb6b6ce200ab460b6b604bad98a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-4, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-19T08:59:14.626 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:14 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:14.232+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:14.626 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:14 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:14.232+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:15.031 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:14 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:14.864+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.950897+0000 front 2024-09-19T08:58:40.951096+0000 (oldest deadline 2024-09-19T08:59:06.251090+0000) 2024-09-19T08:59:15.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:14 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:14.864+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.251773+0000 front 2024-09-19T08:58:46.251733+0000 (oldest deadline 2024-09-19T08:59:10.351932+0000) 2024-09-19T08:59:15.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:14 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:14.687+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:15.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:14 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:14.687+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:15.125 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:14 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:14.846+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.460189+0000 front 2024-09-19T08:58:49.460291+0000 (oldest deadline 2024-09-19T08:59:11.660880+0000) 2024-09-19T08:59:15.126 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 19 08:59:14 smithi170 podman[71347]: 2024-09-19 08:59:14.867127603 +0000 UTC m=+5.917503903 container cleanup 4879674e2cb200c9392c1bd8dfc315cf75cceb6b6ce200ab460b6b604bad98a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-19T08:59:15.126 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 19 08:59:14 smithi170 bash[71347]: ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-4 2024-09-19T08:59:15.160 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:14 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:14.747+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:15.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:14 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:14.834+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:15.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:14 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:14.834+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:15.489 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 19 08:59:15 smithi170 podman[71364]: 2024-09-19 08:59:15.13973436 +0000 UTC m=+0.666787596 container remove 4879674e2cb200c9392c1bd8dfc315cf75cceb6b6ce200ab460b6b604bad98a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-4, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-19T08:59:15.489 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:15 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:15.228+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:15.489 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:15 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:15.228+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:15.660 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:15 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:59:15.194+0000 7fe9f5e62640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.609349+0000 front 2024-09-19T08:58:40.609376+0000 (oldest deadline 2024-09-19T08:59:05.909533+0000) 2024-09-19T08:59:15.661 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:15 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:59:15.194+0000 7fe9f5e62640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.810738+0000 front 2024-09-19T08:58:48.810684+0000 (oldest deadline 2024-09-19T08:59:11.110573+0000) 2024-09-19T08:59:15.746 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 19 08:59:15 smithi170 podman[71449]: 2024-09-19 08:59:15.392390484 +0000 UTC m=+0.015889670 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:59:15.746 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 19 08:59:15 smithi170 podman[71449]: 2024-09-19 08:59:15.494649225 +0000 UTC m=+0.118148446 container create fe4694f04011196f4abf969db0baef47287d42826a07c6665760c92e3ecd330f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-19T08:59:16.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:15 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:15.821+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.950897+0000 front 2024-09-19T08:58:40.951096+0000 (oldest deadline 2024-09-19T08:59:06.251090+0000) 2024-09-19T08:59:16.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:15 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:15.821+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.251773+0000 front 2024-09-19T08:58:46.251733+0000 (oldest deadline 2024-09-19T08:59:10.351932+0000) 2024-09-19T08:59:16.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:15 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:15.648+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:16.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:15 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:15.648+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:16.125 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:15 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:15.851+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.460189+0000 front 2024-09-19T08:58:49.460291+0000 (oldest deadline 2024-09-19T08:59:11.660880+0000) 2024-09-19T08:59:16.125 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 19 08:59:15 smithi170 podman[71449]: 2024-09-19 08:59:15.747554654 +0000 UTC m=+0.371053838 container init fe4694f04011196f4abf969db0baef47287d42826a07c6665760c92e3ecd330f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-4-deactivate, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-19T08:59:16.125 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 19 08:59:15 smithi170 podman[71449]: 2024-09-19 08:59:15.750881523 +0000 UTC m=+0.374380702 container start fe4694f04011196f4abf969db0baef47287d42826a07c6665760c92e3ecd330f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-19T08:59:16.125 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 19 08:59:15 smithi170 podman[71449]: 2024-09-19 08:59:15.830706929 +0000 UTC m=+0.454206122 container attach fe4694f04011196f4abf969db0baef47287d42826a07c6665760c92e3ecd330f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T08:59:16.125 INFO:journalctl@ceph.osd.4.smithi170.stdout:Sep 19 08:59:15 smithi170 podman[71449]: 2024-09-19 08:59:15.938571919 +0000 UTC m=+0.562071100 container died fe4694f04011196f4abf969db0baef47287d42826a07c6665760c92e3ecd330f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-2f599a4, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T08:59:16.160 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:15 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:15.785+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:16.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:15 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:15.836+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:16.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:15 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:15.836+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:16.592 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:16 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:16.218+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:16.593 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:16 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:16.218+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:16.619 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.4.service' 2024-09-19T08:59:16.659 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:59:16.660 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-19T08:59:16.660 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-19T08:59:16.660 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.5 2024-09-19T08:59:16.661 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:16 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:59:16.200+0000 7fe9f5e62640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.609349+0000 front 2024-09-19T08:58:40.609376+0000 (oldest deadline 2024-09-19T08:59:05.909533+0000) 2024-09-19T08:59:16.661 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:16 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:59:16.200+0000 7fe9f5e62640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.810738+0000 front 2024-09-19T08:58:48.810684+0000 (oldest deadline 2024-09-19T08:59:11.110573+0000) 2024-09-19T08:59:16.970 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:16 smithi173 systemd[1]: Stopping Ceph osd.5 for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T08:59:16.970 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:16 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:16.825+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:16.970 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:16 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:16.806+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:16.971 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:16 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:16.806+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:17.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:16 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:16.822+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.950897+0000 front 2024-09-19T08:58:40.951096+0000 (oldest deadline 2024-09-19T08:59:06.251090+0000) 2024-09-19T08:59:17.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:16 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:16.822+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.251773+0000 front 2024-09-19T08:58:46.251733+0000 (oldest deadline 2024-09-19T08:59:10.351932+0000) 2024-09-19T08:59:17.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:16 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:16.681+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:17.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:16 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:16.681+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:17.125 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:16 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:16.850+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.460189+0000 front 2024-09-19T08:58:49.460291+0000 (oldest deadline 2024-09-19T08:59:11.660880+0000) 2024-09-19T08:59:17.410 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:16 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:59:16.970+0000 7fe9fd854640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:59:17.411 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:16 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:59:16.970+0000 7fe9fd854640 -1 osd.5 100 *** Got signal Terminated *** 2024-09-19T08:59:17.411 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:16 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:59:16.970+0000 7fe9fd854640 -1 osd.5 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T08:59:17.411 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:17 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:59:17.168+0000 7fe9f5e62640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.609349+0000 front 2024-09-19T08:58:40.609376+0000 (oldest deadline 2024-09-19T08:59:05.909533+0000) 2024-09-19T08:59:17.411 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:17 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:59:17.168+0000 7fe9f5e62640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.810738+0000 front 2024-09-19T08:58:48.810684+0000 (oldest deadline 2024-09-19T08:59:11.110573+0000) 2024-09-19T08:59:17.624 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:17 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:17.209+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:17.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:17 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:17.209+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:18.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:17 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:17.810+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.950897+0000 front 2024-09-19T08:58:40.951096+0000 (oldest deadline 2024-09-19T08:59:06.251090+0000) 2024-09-19T08:59:18.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:17 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:17.811+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.251773+0000 front 2024-09-19T08:58:46.251733+0000 (oldest deadline 2024-09-19T08:59:10.351932+0000) 2024-09-19T08:59:18.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:17 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:17.636+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:18.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:17 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:17.636+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:18.124 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:17 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:17.855+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.460189+0000 front 2024-09-19T08:58:49.460291+0000 (oldest deadline 2024-09-19T08:59:11.660880+0000) 2024-09-19T08:59:18.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:17 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:17.782+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:18.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:17 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:17.828+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:18.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:17 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:17.828+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:18.624 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:18 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:18.226+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:18.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:18 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:18.226+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:18.661 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:18 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:59:18.193+0000 7fe9f5e62640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.609349+0000 front 2024-09-19T08:58:40.609376+0000 (oldest deadline 2024-09-19T08:59:05.909533+0000) 2024-09-19T08:59:18.661 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:18 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:59:18.193+0000 7fe9f5e62640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.810738+0000 front 2024-09-19T08:58:48.810684+0000 (oldest deadline 2024-09-19T08:59:11.110573+0000) 2024-09-19T08:59:19.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:18 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:18.617+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:19.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:18 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:18.617+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:19.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:18 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:18.778+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.950897+0000 front 2024-09-19T08:58:40.951096+0000 (oldest deadline 2024-09-19T08:59:06.251090+0000) 2024-09-19T08:59:19.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:18 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:18.778+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.251773+0000 front 2024-09-19T08:58:46.251733+0000 (oldest deadline 2024-09-19T08:59:10.351932+0000) 2024-09-19T08:59:19.033 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:18 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:18.778+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.454006+0000 front 2024-09-19T08:58:55.453886+0000 (oldest deadline 2024-09-19T08:59:18.353927+0000) 2024-09-19T08:59:19.160 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:18 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:18.738+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:19.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:18 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:18.803+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:19.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:18 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:18.803+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:19.246 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:18 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:18.892+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.460189+0000 front 2024-09-19T08:58:49.460291+0000 (oldest deadline 2024-09-19T08:59:11.660880+0000) 2024-09-19T08:59:19.246 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:18 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:18.892+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:52.161680+0000 front 2024-09-19T08:58:52.161732+0000 (oldest deadline 2024-09-19T08:59:18.562288+0000) 2024-09-19T08:59:19.624 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:19 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:19.246+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:19.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:19 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:19.246+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:19.660 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:19 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:59:19.212+0000 7fe9f5e62640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.609349+0000 front 2024-09-19T08:58:40.609376+0000 (oldest deadline 2024-09-19T08:59:05.909533+0000) 2024-09-19T08:59:19.661 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:19 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:59:19.212+0000 7fe9f5e62640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.810738+0000 front 2024-09-19T08:58:48.810684+0000 (oldest deadline 2024-09-19T08:59:11.110573+0000) 2024-09-19T08:59:19.661 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:19 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:59:19.212+0000 7fe9f5e62640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.211384+0000 front 2024-09-19T08:58:55.211404+0000 (oldest deadline 2024-09-19T08:59:18.711562+0000) 2024-09-19T08:59:20.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:19 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:19.622+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:20.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:19 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:19.622+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:20.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:19 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:19.798+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.950897+0000 front 2024-09-19T08:58:40.951096+0000 (oldest deadline 2024-09-19T08:59:06.251090+0000) 2024-09-19T08:59:20.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:19 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:19.798+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.251773+0000 front 2024-09-19T08:58:46.251733+0000 (oldest deadline 2024-09-19T08:59:10.351932+0000) 2024-09-19T08:59:20.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:19 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:19.798+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.454006+0000 front 2024-09-19T08:58:55.453886+0000 (oldest deadline 2024-09-19T08:59:18.353927+0000) 2024-09-19T08:59:20.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:19 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:19.739+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:20.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:19 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:19.739+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.169174+0000 front 2024-09-19T08:58:57.169202+0000 (oldest deadline 2024-09-19T08:59:19.469292+0000) 2024-09-19T08:59:20.162 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:19 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:19.776+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:20.162 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:19 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:19.776+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:20.162 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:19 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:19.776+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:20.199 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:19 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:19.934+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.460189+0000 front 2024-09-19T08:58:49.460291+0000 (oldest deadline 2024-09-19T08:59:11.660880+0000) 2024-09-19T08:59:20.199 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:19 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:19.934+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:52.161680+0000 front 2024-09-19T08:58:52.161732+0000 (oldest deadline 2024-09-19T08:59:18.562288+0000) 2024-09-19T08:59:20.468 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:20 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:20.200+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:20.468 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:20 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:20.200+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:20.468 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:20 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:20.200+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:20.660 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:20 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:59:20.239+0000 7fe9f5e62640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.609349+0000 front 2024-09-19T08:58:40.609376+0000 (oldest deadline 2024-09-19T08:59:05.909533+0000) 2024-09-19T08:59:20.660 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:20 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:59:20.239+0000 7fe9f5e62640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.810738+0000 front 2024-09-19T08:58:48.810684+0000 (oldest deadline 2024-09-19T08:59:11.110573+0000) 2024-09-19T08:59:20.661 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:20 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:59:20.239+0000 7fe9f5e62640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.211384+0000 front 2024-09-19T08:58:55.211404+0000 (oldest deadline 2024-09-19T08:59:18.711562+0000) 2024-09-19T08:59:21.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:20 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:20.593+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:21.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:20 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:20.593+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:21.033 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:20 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:20.795+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.950897+0000 front 2024-09-19T08:58:40.951096+0000 (oldest deadline 2024-09-19T08:59:06.251090+0000) 2024-09-19T08:59:21.033 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:20 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:20.795+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.251773+0000 front 2024-09-19T08:58:46.251733+0000 (oldest deadline 2024-09-19T08:59:10.351932+0000) 2024-09-19T08:59:21.033 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:20 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:20.795+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.454006+0000 front 2024-09-19T08:58:55.453886+0000 (oldest deadline 2024-09-19T08:59:18.353927+0000) 2024-09-19T08:59:21.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:20 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:20.778+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:21.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:20 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:20.778+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.169174+0000 front 2024-09-19T08:58:57.169202+0000 (oldest deadline 2024-09-19T08:59:19.469292+0000) 2024-09-19T08:59:21.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:20 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:20.754+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:21.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:20 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:20.754+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:21.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:20 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:20.754+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:21.247 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:20 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:20.951+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.460189+0000 front 2024-09-19T08:58:49.460291+0000 (oldest deadline 2024-09-19T08:59:11.660880+0000) 2024-09-19T08:59:21.247 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:20 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:20.951+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:52.161680+0000 front 2024-09-19T08:58:52.161732+0000 (oldest deadline 2024-09-19T08:59:18.562288+0000) 2024-09-19T08:59:21.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:21 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:21.247+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:21.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:21 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:21.247+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:21.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:21 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:21.247+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:21.660 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:21 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:59:21.256+0000 7fe9f5e62640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.609349+0000 front 2024-09-19T08:58:40.609376+0000 (oldest deadline 2024-09-19T08:59:05.909533+0000) 2024-09-19T08:59:21.661 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:21 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:59:21.256+0000 7fe9f5e62640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.810738+0000 front 2024-09-19T08:58:48.810684+0000 (oldest deadline 2024-09-19T08:59:11.110573+0000) 2024-09-19T08:59:21.661 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:21 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5[46712]: 2024-09-19T08:59:21.256+0000 7fe9f5e62640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.211384+0000 front 2024-09-19T08:58:55.211404+0000 (oldest deadline 2024-09-19T08:59:18.711562+0000) 2024-09-19T08:59:22.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:21 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:21.588+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:22.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:21 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:21.588+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:22.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:21 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:21.771+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.950897+0000 front 2024-09-19T08:58:40.951096+0000 (oldest deadline 2024-09-19T08:59:06.251090+0000) 2024-09-19T08:59:22.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:21 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:21.771+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.251773+0000 front 2024-09-19T08:58:46.251733+0000 (oldest deadline 2024-09-19T08:59:10.351932+0000) 2024-09-19T08:59:22.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:21 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:21.771+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.454006+0000 front 2024-09-19T08:58:55.453886+0000 (oldest deadline 2024-09-19T08:59:18.353927+0000) 2024-09-19T08:59:22.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:21 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:21.731+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:22.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:21 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:21.731+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.169174+0000 front 2024-09-19T08:58:57.169202+0000 (oldest deadline 2024-09-19T08:59:19.469292+0000) 2024-09-19T08:59:22.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:21 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:21.726+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:22.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:21 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:21.726+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:22.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:21 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:21.726+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:22.285 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:21 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:21.919+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.460189+0000 front 2024-09-19T08:58:49.460291+0000 (oldest deadline 2024-09-19T08:59:11.660880+0000) 2024-09-19T08:59:22.285 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:21 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:21.919+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:52.161680+0000 front 2024-09-19T08:58:52.161732+0000 (oldest deadline 2024-09-19T08:59:18.562288+0000) 2024-09-19T08:59:22.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:22 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:22.286+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:22.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:22 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:22.286+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:22.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:22 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:22.286+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:22.889 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:22 smithi173 podman[71746]: 2024-09-19 08:59:22.583175258 +0000 UTC m=+5.727641352 container died 7d2b7b7911e81118291a82bfe1a2ea1f9435720e6ae929d94d852776d65931aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-19T08:59:22.890 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:22 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:22.751+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:22.890 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:22 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:22.751+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.169174+0000 front 2024-09-19T08:58:57.169202+0000 (oldest deadline 2024-09-19T08:59:19.469292+0000) 2024-09-19T08:59:22.890 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:22 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:22.716+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:22.891 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:22 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:22.716+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:22.891 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:22 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:22.716+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:23.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:22 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:22.785+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.950897+0000 front 2024-09-19T08:58:40.951096+0000 (oldest deadline 2024-09-19T08:59:06.251090+0000) 2024-09-19T08:59:23.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:22 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:22.785+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.251773+0000 front 2024-09-19T08:58:46.251733+0000 (oldest deadline 2024-09-19T08:59:10.351932+0000) 2024-09-19T08:59:23.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:22 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:22.785+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.454006+0000 front 2024-09-19T08:58:55.453886+0000 (oldest deadline 2024-09-19T08:59:18.353927+0000) 2024-09-19T08:59:23.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:22 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:22.548+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:23.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:22 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:22.548+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:23.033 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:22 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:22.548+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.205419+0000 front 2024-09-19T08:58:55.205429+0000 (oldest deadline 2024-09-19T08:59:22.205873+0000) 2024-09-19T08:59:23.160 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:22 smithi173 podman[71746]: 2024-09-19 08:59:22.979291394 +0000 UTC m=+6.123757495 container cleanup 7d2b7b7911e81118291a82bfe1a2ea1f9435720e6ae929d94d852776d65931aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0) 2024-09-19T08:59:23.160 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:22 smithi173 bash[71746]: ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5 2024-09-19T08:59:23.303 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:22 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:22.905+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.460189+0000 front 2024-09-19T08:58:49.460291+0000 (oldest deadline 2024-09-19T08:59:11.660880+0000) 2024-09-19T08:59:23.304 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:22 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:22.905+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:52.161680+0000 front 2024-09-19T08:58:52.161732+0000 (oldest deadline 2024-09-19T08:59:18.562288+0000) 2024-09-19T08:59:23.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:23 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:23.304+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:23.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:23 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:23.304+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:23.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:23 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:23.304+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:23.628 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:23 smithi173 podman[71763]: 2024-09-19 08:59:23.241928529 +0000 UTC m=+0.657677097 container remove 7d2b7b7911e81118291a82bfe1a2ea1f9435720e6ae929d94d852776d65931aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-2f599a4, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-19T08:59:23.910 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:23 smithi173 podman[71860]: 2024-09-19 08:59:23.533198178 +0000 UTC m=+0.017702140 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:59:23.911 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:23 smithi173 podman[71860]: 2024-09-19 08:59:23.648835973 +0000 UTC m=+0.133339912 container create 169f4a3747e9d0bfa616a1cdcfd89f0a8ccfabf26c341466287b4f1c387a44fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True) 2024-09-19T08:59:23.911 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:23 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:23.761+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:23.911 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:23 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:23.761+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.169174+0000 front 2024-09-19T08:58:57.169202+0000 (oldest deadline 2024-09-19T08:59:19.469292+0000) 2024-09-19T08:59:23.911 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:23 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:23.677+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:23.911 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:23 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:23.677+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:23.911 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:23 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:23.677+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:24.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:23 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:23.798+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.950897+0000 front 2024-09-19T08:58:40.951096+0000 (oldest deadline 2024-09-19T08:59:06.251090+0000) 2024-09-19T08:59:24.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:23 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:23.798+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.251773+0000 front 2024-09-19T08:58:46.251733+0000 (oldest deadline 2024-09-19T08:59:10.351932+0000) 2024-09-19T08:59:24.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:23 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:23.798+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.454006+0000 front 2024-09-19T08:58:55.453886+0000 (oldest deadline 2024-09-19T08:59:18.353927+0000) 2024-09-19T08:59:24.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:23 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:23.596+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:24.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:23 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:23.596+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:24.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:23 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:23.596+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.205419+0000 front 2024-09-19T08:58:55.205429+0000 (oldest deadline 2024-09-19T08:59:22.205873+0000) 2024-09-19T08:59:24.307 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:23 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:23.952+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.460189+0000 front 2024-09-19T08:58:49.460291+0000 (oldest deadline 2024-09-19T08:59:11.660880+0000) 2024-09-19T08:59:24.307 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:23 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:23.952+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:52.161680+0000 front 2024-09-19T08:58:52.161732+0000 (oldest deadline 2024-09-19T08:59:18.562288+0000) 2024-09-19T08:59:24.405 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:23 smithi173 podman[71860]: 2024-09-19 08:59:23.91837331 +0000 UTC m=+0.402877253 container init 169f4a3747e9d0bfa616a1cdcfd89f0a8ccfabf26c341466287b4f1c387a44fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T08:59:24.415 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:23 smithi173 podman[71860]: 2024-09-19 08:59:23.921653401 +0000 UTC m=+0.406157348 container start 169f4a3747e9d0bfa616a1cdcfd89f0a8ccfabf26c341466287b4f1c387a44fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5-deactivate, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD) 2024-09-19T08:59:24.416 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:24 smithi173 podman[71860]: 2024-09-19 08:59:24.010248501 +0000 UTC m=+0.494752453 container attach 169f4a3747e9d0bfa616a1cdcfd89f0a8ccfabf26c341466287b4f1c387a44fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T08:59:24.416 INFO:journalctl@ceph.osd.5.smithi173.stdout:Sep 19 08:59:24 smithi173 podman[71860]: 2024-09-19 08:59:24.114976423 +0000 UTC m=+0.599480370 container died 169f4a3747e9d0bfa616a1cdcfd89f0a8ccfabf26c341466287b4f1c387a44fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T08:59:24.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:24 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:24.307+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:24.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:24 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:24.307+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:24.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:24 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:24.307+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:24.665 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:24 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:24.644+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:24.665 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:24 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:24.644+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:24.666 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:24 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:24.644+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:24.748 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.5.service' 2024-09-19T08:59:24.798 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:59:24.799 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-19T08:59:24.799 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-19T08:59:24.799 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.6 2024-09-19T08:59:25.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:24 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:24.635+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:25.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:24 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:24.635+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:25.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:24 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:24.635+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.205419+0000 front 2024-09-19T08:58:55.205429+0000 (oldest deadline 2024-09-19T08:59:22.205873+0000) 2024-09-19T08:59:25.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:24 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:24.807+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.950897+0000 front 2024-09-19T08:58:40.951096+0000 (oldest deadline 2024-09-19T08:59:06.251090+0000) 2024-09-19T08:59:25.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:24 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:24.807+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.251773+0000 front 2024-09-19T08:58:46.251733+0000 (oldest deadline 2024-09-19T08:59:10.351932+0000) 2024-09-19T08:59:25.033 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:24 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:24.807+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.454006+0000 front 2024-09-19T08:58:55.453886+0000 (oldest deadline 2024-09-19T08:59:18.353927+0000) 2024-09-19T08:59:25.033 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:24 smithi164 systemd[1]: Stopping Ceph osd.6 for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T08:59:25.160 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:24 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:24.742+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:25.160 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:24 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:24.742+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.169174+0000 front 2024-09-19T08:58:57.169202+0000 (oldest deadline 2024-09-19T08:59:19.469292+0000) 2024-09-19T08:59:25.336 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:24 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:24.930+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.460189+0000 front 2024-09-19T08:58:49.460291+0000 (oldest deadline 2024-09-19T08:59:11.660880+0000) 2024-09-19T08:59:25.336 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:24 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:24.930+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:52.161680+0000 front 2024-09-19T08:58:52.161732+0000 (oldest deadline 2024-09-19T08:59:18.562288+0000) 2024-09-19T08:59:25.532 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:25 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:25.113+0000 7fbd440a4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:59:25.532 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:25 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:25.113+0000 7fbd440a4640 -1 osd.6 100 *** Got signal Terminated *** 2024-09-19T08:59:25.532 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:25 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:25.113+0000 7fbd440a4640 -1 osd.6 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T08:59:25.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:25 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:25.336+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:25.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:25 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:25.336+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:25.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:25 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:25.336+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:26.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:25 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:25.791+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.950897+0000 front 2024-09-19T08:58:40.951096+0000 (oldest deadline 2024-09-19T08:59:06.251090+0000) 2024-09-19T08:59:26.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:25 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:25.791+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.251773+0000 front 2024-09-19T08:58:46.251733+0000 (oldest deadline 2024-09-19T08:59:10.351932+0000) 2024-09-19T08:59:26.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:25 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:25.791+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.454006+0000 front 2024-09-19T08:58:55.453886+0000 (oldest deadline 2024-09-19T08:59:18.353927+0000) 2024-09-19T08:59:26.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:25 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:25.636+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:26.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:25 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:25.636+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:26.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:25 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:25.636+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.205419+0000 front 2024-09-19T08:58:55.205429+0000 (oldest deadline 2024-09-19T08:59:22.205873+0000) 2024-09-19T08:59:26.160 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:25 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:25.714+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:26.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:25 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:25.714+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.169174+0000 front 2024-09-19T08:58:57.169202+0000 (oldest deadline 2024-09-19T08:59:19.469292+0000) 2024-09-19T08:59:26.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:25 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:25.670+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:26.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:25 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:25.670+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:26.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:25 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:25.670+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:26.362 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:25 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:25.955+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.460189+0000 front 2024-09-19T08:58:49.460291+0000 (oldest deadline 2024-09-19T08:59:11.660880+0000) 2024-09-19T08:59:26.362 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:25 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:25.955+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:52.161680+0000 front 2024-09-19T08:58:52.161732+0000 (oldest deadline 2024-09-19T08:59:18.562288+0000) 2024-09-19T08:59:26.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:26 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:26.362+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:26.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:26 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:26.362+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:26.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:26 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:26.362+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:27.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:26 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:26.808+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.950897+0000 front 2024-09-19T08:58:40.951096+0000 (oldest deadline 2024-09-19T08:59:06.251090+0000) 2024-09-19T08:59:27.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:26 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:26.808+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.251773+0000 front 2024-09-19T08:58:46.251733+0000 (oldest deadline 2024-09-19T08:59:10.351932+0000) 2024-09-19T08:59:27.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:26 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:26.808+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.454006+0000 front 2024-09-19T08:58:55.453886+0000 (oldest deadline 2024-09-19T08:59:18.353927+0000) 2024-09-19T08:59:27.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:26 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:26.605+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:27.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:26 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:26.605+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:27.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:26 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:26.605+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.205419+0000 front 2024-09-19T08:58:55.205429+0000 (oldest deadline 2024-09-19T08:59:22.205873+0000) 2024-09-19T08:59:27.160 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:26 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:26.736+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:27.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:26 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:26.736+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.169174+0000 front 2024-09-19T08:58:57.169202+0000 (oldest deadline 2024-09-19T08:59:19.469292+0000) 2024-09-19T08:59:27.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:26 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:26.699+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:27.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:26 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:26.699+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:27.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:26 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:26.699+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:27.375 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:26 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:26.971+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.460189+0000 front 2024-09-19T08:58:49.460291+0000 (oldest deadline 2024-09-19T08:59:11.660880+0000) 2024-09-19T08:59:27.375 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:26 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:26.971+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:52.161680+0000 front 2024-09-19T08:58:52.161732+0000 (oldest deadline 2024-09-19T08:59:18.562288+0000) 2024-09-19T08:59:27.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:27 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:27.381+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:27.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:27 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:27.381+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:27.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:27 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:27.381+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:28.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:27 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:27.832+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.950897+0000 front 2024-09-19T08:58:40.951096+0000 (oldest deadline 2024-09-19T08:59:06.251090+0000) 2024-09-19T08:59:28.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:27 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:27.832+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.251773+0000 front 2024-09-19T08:58:46.251733+0000 (oldest deadline 2024-09-19T08:59:10.351932+0000) 2024-09-19T08:59:28.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:27 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:27.832+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.454006+0000 front 2024-09-19T08:58:55.453886+0000 (oldest deadline 2024-09-19T08:59:18.353927+0000) 2024-09-19T08:59:28.032 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:27 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:27.832+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:03.054655+0000 front 2024-09-19T08:59:03.054614+0000 (oldest deadline 2024-09-19T08:59:27.154727+0000) 2024-09-19T08:59:28.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:27 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:27.637+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:28.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:27 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:27.637+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:28.033 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:27 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:27.637+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.205419+0000 front 2024-09-19T08:58:55.205429+0000 (oldest deadline 2024-09-19T08:59:22.205873+0000) 2024-09-19T08:59:28.033 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:27 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:27.637+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:02.206362+0000 front 2024-09-19T08:59:02.206384+0000 (oldest deadline 2024-09-19T08:59:26.906456+0000) 2024-09-19T08:59:28.160 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:27 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:27.723+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:28.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:27 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:27.723+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.169174+0000 front 2024-09-19T08:58:57.169202+0000 (oldest deadline 2024-09-19T08:59:19.469292+0000) 2024-09-19T08:59:28.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:27 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:27.723+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:04.770508+0000 front 2024-09-19T08:59:04.770362+0000 (oldest deadline 2024-09-19T08:59:27.070397+0000) 2024-09-19T08:59:28.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:27 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:27.714+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:28.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:27 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:27.714+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:28.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:27 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:27.714+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:28.358 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:28 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:28.019+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.460189+0000 front 2024-09-19T08:58:49.460291+0000 (oldest deadline 2024-09-19T08:59:11.660880+0000) 2024-09-19T08:59:28.359 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:28 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:28.019+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:52.161680+0000 front 2024-09-19T08:58:52.161732+0000 (oldest deadline 2024-09-19T08:59:18.562288+0000) 2024-09-19T08:59:28.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:28 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:28.359+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:28.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:28 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:28.359+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:28.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:28 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:28.359+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:28.877 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:28 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:28.604+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:28.878 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:28 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:28.604+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:28.878 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:28 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:28.604+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.205419+0000 front 2024-09-19T08:58:55.205429+0000 (oldest deadline 2024-09-19T08:59:22.205873+0000) 2024-09-19T08:59:28.878 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:28 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:28.604+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:02.206362+0000 front 2024-09-19T08:59:02.206384+0000 (oldest deadline 2024-09-19T08:59:26.906456+0000) 2024-09-19T08:59:29.160 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:28 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:28.770+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:29.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:28 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:28.770+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.169174+0000 front 2024-09-19T08:58:57.169202+0000 (oldest deadline 2024-09-19T08:59:19.469292+0000) 2024-09-19T08:59:29.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:28 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:28.770+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:04.770508+0000 front 2024-09-19T08:59:04.770362+0000 (oldest deadline 2024-09-19T08:59:27.070397+0000) 2024-09-19T08:59:29.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:28 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:28.755+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:29.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:28 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:28.755+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:29.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:28 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:28.755+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:29.282 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:28 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:28.878+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.950897+0000 front 2024-09-19T08:58:40.951096+0000 (oldest deadline 2024-09-19T08:59:06.251090+0000) 2024-09-19T08:59:29.282 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:28 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:28.878+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.251773+0000 front 2024-09-19T08:58:46.251733+0000 (oldest deadline 2024-09-19T08:59:10.351932+0000) 2024-09-19T08:59:29.282 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:28 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:28.878+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.454006+0000 front 2024-09-19T08:58:55.453886+0000 (oldest deadline 2024-09-19T08:59:18.353927+0000) 2024-09-19T08:59:29.282 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:28 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:28.878+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:03.054655+0000 front 2024-09-19T08:59:03.054614+0000 (oldest deadline 2024-09-19T08:59:27.154727+0000) 2024-09-19T08:59:29.374 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:29 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:29.001+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.460189+0000 front 2024-09-19T08:58:49.460291+0000 (oldest deadline 2024-09-19T08:59:11.660880+0000) 2024-09-19T08:59:29.375 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:29 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:29.001+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:52.161680+0000 front 2024-09-19T08:58:52.161732+0000 (oldest deadline 2024-09-19T08:59:18.562288+0000) 2024-09-19T08:59:29.874 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:29 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:29.403+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:29.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:29 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:29.403+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:29.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:29 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:29.403+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:29.924 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:29 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:29.620+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:29.924 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:29 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:29.620+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:29.924 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:29 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:29.620+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.205419+0000 front 2024-09-19T08:58:55.205429+0000 (oldest deadline 2024-09-19T08:59:22.205873+0000) 2024-09-19T08:59:29.925 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:29 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:29.620+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:02.206362+0000 front 2024-09-19T08:59:02.206384+0000 (oldest deadline 2024-09-19T08:59:26.906456+0000) 2024-09-19T08:59:30.160 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:29 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:29.749+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:30.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:29 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:29.749+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.169174+0000 front 2024-09-19T08:58:57.169202+0000 (oldest deadline 2024-09-19T08:59:19.469292+0000) 2024-09-19T08:59:30.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:29 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:29.749+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:04.770508+0000 front 2024-09-19T08:59:04.770362+0000 (oldest deadline 2024-09-19T08:59:27.070397+0000) 2024-09-19T08:59:30.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:29 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:29.720+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:30.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:29 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:29.720+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:30.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:29 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:29.720+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:30.282 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:29 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:29.925+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.950897+0000 front 2024-09-19T08:58:40.951096+0000 (oldest deadline 2024-09-19T08:59:06.251090+0000) 2024-09-19T08:59:30.282 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:29 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:29.925+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.251773+0000 front 2024-09-19T08:58:46.251733+0000 (oldest deadline 2024-09-19T08:59:10.351932+0000) 2024-09-19T08:59:30.282 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:29 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:29.925+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.454006+0000 front 2024-09-19T08:58:55.453886+0000 (oldest deadline 2024-09-19T08:59:18.353927+0000) 2024-09-19T08:59:30.282 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:29 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6[58331]: 2024-09-19T08:59:29.925+0000 7fbd3c6b2640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:03.054655+0000 front 2024-09-19T08:59:03.054614+0000 (oldest deadline 2024-09-19T08:59:27.154727+0000) 2024-09-19T08:59:30.374 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:29 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:29.961+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.460189+0000 front 2024-09-19T08:58:49.460291+0000 (oldest deadline 2024-09-19T08:59:11.660880+0000) 2024-09-19T08:59:30.375 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:29 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:29.961+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:52.161680+0000 front 2024-09-19T08:58:52.161732+0000 (oldest deadline 2024-09-19T08:59:18.562288+0000) 2024-09-19T08:59:30.375 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:29 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:29.961+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:04.363927+0000 front 2024-09-19T08:59:04.364021+0000 (oldest deadline 2024-09-19T08:59:29.663818+0000) 2024-09-19T08:59:30.863 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:30 smithi164 podman[114377]: 2024-09-19 08:59:30.551888536 +0000 UTC m=+5.544595131 container died 39dc4d6a1c021095f91d6fb510d08d0ee685e6b6de1bbe7372f8d4448cd0d3bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-19T08:59:30.863 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:30 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:30.666+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:30.863 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:30 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:30.666+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:30.863 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:30 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:30.666+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.205419+0000 front 2024-09-19T08:58:55.205429+0000 (oldest deadline 2024-09-19T08:59:22.205873+0000) 2024-09-19T08:59:30.864 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:30 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:30.666+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:02.206362+0000 front 2024-09-19T08:59:02.206384+0000 (oldest deadline 2024-09-19T08:59:26.906456+0000) 2024-09-19T08:59:30.874 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:30 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:30.426+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:30.874 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:30 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:30.426+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:30.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:30 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:30.426+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:31.160 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:30 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:30.785+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:31.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:30 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:30.785+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.169174+0000 front 2024-09-19T08:58:57.169202+0000 (oldest deadline 2024-09-19T08:59:19.469292+0000) 2024-09-19T08:59:31.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:30 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:30.785+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:04.770508+0000 front 2024-09-19T08:59:04.770362+0000 (oldest deadline 2024-09-19T08:59:27.070397+0000) 2024-09-19T08:59:31.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:30 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:30.739+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:31.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:30 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:30.739+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:31.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:30 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:30.739+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:31.272 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:31 smithi164 podman[114377]: 2024-09-19 08:59:31.046567692 +0000 UTC m=+6.039274281 container cleanup 39dc4d6a1c021095f91d6fb510d08d0ee685e6b6de1bbe7372f8d4448cd0d3bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T08:59:31.272 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:31 smithi164 bash[114377]: ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6 2024-09-19T08:59:31.375 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:30 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:30.922+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.460189+0000 front 2024-09-19T08:58:49.460291+0000 (oldest deadline 2024-09-19T08:59:11.660880+0000) 2024-09-19T08:59:31.375 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:30 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:30.922+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:52.161680+0000 front 2024-09-19T08:58:52.161732+0000 (oldest deadline 2024-09-19T08:59:18.562288+0000) 2024-09-19T08:59:31.375 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:30 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:30.922+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:04.363927+0000 front 2024-09-19T08:59:04.364021+0000 (oldest deadline 2024-09-19T08:59:29.663818+0000) 2024-09-19T08:59:31.532 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:31 smithi164 podman[114392]: 2024-09-19 08:59:31.273105568 +0000 UTC m=+0.721598173 container remove 39dc4d6a1c021095f91d6fb510d08d0ee685e6b6de1bbe7372f8d4448cd0d3bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-19T08:59:31.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:31 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:31.418+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:31.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:31 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:31.418+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:31.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:31 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:31.418+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:31.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:31 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:31.418+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:05.222908+0000 front 2024-09-19T08:59:05.222955+0000 (oldest deadline 2024-09-19T08:59:31.122786+0000) 2024-09-19T08:59:31.884 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:31 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:31.643+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:31.884 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:31 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:31.643+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:31.884 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:31 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:31.643+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.205419+0000 front 2024-09-19T08:58:55.205429+0000 (oldest deadline 2024-09-19T08:59:22.205873+0000) 2024-09-19T08:59:31.884 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:31 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:31.643+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:02.206362+0000 front 2024-09-19T08:59:02.206384+0000 (oldest deadline 2024-09-19T08:59:26.906456+0000) 2024-09-19T08:59:31.884 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:31 smithi164 podman[114482]: 2024-09-19 08:59:31.483445507 +0000 UTC m=+0.016117240 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:59:31.884 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:31 smithi164 podman[114482]: 2024-09-19 08:59:31.615849621 +0000 UTC m=+0.148521334 container create 26b4d8b74bbf581308d735dd97a8936e543527125738633b7a9f83c87b0d7497 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-19T08:59:31.885 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:31 smithi164 podman[114482]: 2024-09-19 08:59:31.885029269 +0000 UTC m=+0.417700998 container init 26b4d8b74bbf581308d735dd97a8936e543527125738633b7a9f83c87b0d7497 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-19T08:59:32.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:31 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:31.753+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:32.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:31 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:31.754+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.169174+0000 front 2024-09-19T08:58:57.169202+0000 (oldest deadline 2024-09-19T08:59:19.469292+0000) 2024-09-19T08:59:32.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:31 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:31.754+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:04.770508+0000 front 2024-09-19T08:59:04.770362+0000 (oldest deadline 2024-09-19T08:59:27.070397+0000) 2024-09-19T08:59:32.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:31 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:31.746+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:32.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:31 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:31.746+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:32.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:31 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:31.746+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:32.282 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:31 smithi164 podman[114482]: 2024-09-19 08:59:31.888383598 +0000 UTC m=+0.421055311 container start 26b4d8b74bbf581308d735dd97a8936e543527125738633b7a9f83c87b0d7497 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6-deactivate, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-19T08:59:32.282 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:31 smithi164 podman[114482]: 2024-09-19 08:59:31.985161176 +0000 UTC m=+0.517832911 container attach 26b4d8b74bbf581308d735dd97a8936e543527125738633b7a9f83c87b0d7497 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-19T08:59:32.282 INFO:journalctl@ceph.osd.6.smithi164.stdout:Sep 19 08:59:32 smithi164 podman[114482]: 2024-09-19 08:59:32.078176806 +0000 UTC m=+0.610848531 container died 26b4d8b74bbf581308d735dd97a8936e543527125738633b7a9f83c87b0d7497 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-6-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T08:59:32.375 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:31 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:31.966+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.460189+0000 front 2024-09-19T08:58:49.460291+0000 (oldest deadline 2024-09-19T08:59:11.660880+0000) 2024-09-19T08:59:32.375 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:31 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:31.966+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:52.161680+0000 front 2024-09-19T08:58:52.161732+0000 (oldest deadline 2024-09-19T08:59:18.562288+0000) 2024-09-19T08:59:32.375 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:31 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:31.966+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:04.363927+0000 front 2024-09-19T08:59:04.364021+0000 (oldest deadline 2024-09-19T08:59:29.663818+0000) 2024-09-19T08:59:32.755 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.6.service' 2024-09-19T08:59:32.793 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:59:32.793 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-19T08:59:32.794 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-19T08:59:32.794 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.7 2024-09-19T08:59:32.823 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:32 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:32.399+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:32.823 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:32 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:32.399+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:32.823 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:32 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:32.399+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:32.823 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:32 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:32.399+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:05.222908+0000 front 2024-09-19T08:59:05.222955+0000 (oldest deadline 2024-09-19T08:59:31.122786+0000) 2024-09-19T08:59:33.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:32 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:32.681+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:33.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:32 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:32.681+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:33.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:32 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:32.681+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.205419+0000 front 2024-09-19T08:58:55.205429+0000 (oldest deadline 2024-09-19T08:59:22.205873+0000) 2024-09-19T08:59:33.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:32 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:32.681+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:02.206362+0000 front 2024-09-19T08:59:02.206384+0000 (oldest deadline 2024-09-19T08:59:26.906456+0000) 2024-09-19T08:59:33.125 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:32 smithi170 systemd[1]: Stopping Ceph osd.7 for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T08:59:33.125 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:33 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:33.010+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.460189+0000 front 2024-09-19T08:58:49.460291+0000 (oldest deadline 2024-09-19T08:59:11.660880+0000) 2024-09-19T08:59:33.126 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:33 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:33.010+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:52.161680+0000 front 2024-09-19T08:58:52.161732+0000 (oldest deadline 2024-09-19T08:59:18.562288+0000) 2024-09-19T08:59:33.126 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:33 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:33.010+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:04.363927+0000 front 2024-09-19T08:59:04.364021+0000 (oldest deadline 2024-09-19T08:59:29.663818+0000) 2024-09-19T08:59:33.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:32 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:32.730+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:33.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:32 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:32.730+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.169174+0000 front 2024-09-19T08:58:57.169202+0000 (oldest deadline 2024-09-19T08:59:19.469292+0000) 2024-09-19T08:59:33.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:32 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:32.730+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:04.770508+0000 front 2024-09-19T08:59:04.770362+0000 (oldest deadline 2024-09-19T08:59:27.070397+0000) 2024-09-19T08:59:33.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:32 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:32.790+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:33.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:32 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:32.790+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:33.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:32 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:32.790+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:33.408 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:33 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:33.127+0000 7f20978f6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:59:33.408 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:33 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:33.127+0000 7f20978f6640 -1 osd.7 100 *** Got signal Terminated *** 2024-09-19T08:59:33.408 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:33 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:33.127+0000 7f20978f6640 -1 osd.7 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T08:59:33.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:33 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:33.407+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:33.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:33 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:33.407+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:33.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:33 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:33.407+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:33.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:33 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:33.407+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:05.222908+0000 front 2024-09-19T08:59:05.222955+0000 (oldest deadline 2024-09-19T08:59:31.122786+0000) 2024-09-19T08:59:33.910 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:33 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:33.737+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:33.911 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:33 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:33.737+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.169174+0000 front 2024-09-19T08:58:57.169202+0000 (oldest deadline 2024-09-19T08:59:19.469292+0000) 2024-09-19T08:59:33.911 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:33 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:33.737+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:04.770508+0000 front 2024-09-19T08:59:04.770362+0000 (oldest deadline 2024-09-19T08:59:27.070397+0000) 2024-09-19T08:59:33.911 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:33 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:33.838+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:33.911 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:33 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:33.838+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:33.911 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:33 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:33.838+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:34.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:33 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:33.678+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:34.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:33 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:33.678+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:34.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:33 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:33.678+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.205419+0000 front 2024-09-19T08:58:55.205429+0000 (oldest deadline 2024-09-19T08:59:22.205873+0000) 2024-09-19T08:59:34.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:33 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:33.678+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:02.206362+0000 front 2024-09-19T08:59:02.206384+0000 (oldest deadline 2024-09-19T08:59:26.906456+0000) 2024-09-19T08:59:34.374 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:34 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:34.050+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.460189+0000 front 2024-09-19T08:58:49.460291+0000 (oldest deadline 2024-09-19T08:59:11.660880+0000) 2024-09-19T08:59:34.375 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:34 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:34.050+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:52.161680+0000 front 2024-09-19T08:58:52.161732+0000 (oldest deadline 2024-09-19T08:59:18.562288+0000) 2024-09-19T08:59:34.375 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:34 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:34.050+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:04.363927+0000 front 2024-09-19T08:59:04.364021+0000 (oldest deadline 2024-09-19T08:59:29.663818+0000) 2024-09-19T08:59:34.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:34 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:34.415+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:34.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:34 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:34.415+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:34.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:34 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:34.415+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:34.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:34 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:34.415+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:05.222908+0000 front 2024-09-19T08:59:05.222955+0000 (oldest deadline 2024-09-19T08:59:31.122786+0000) 2024-09-19T08:59:35.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:34 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:34.710+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:35.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:34 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:34.710+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:35.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:34 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:34.710+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.205419+0000 front 2024-09-19T08:58:55.205429+0000 (oldest deadline 2024-09-19T08:59:22.205873+0000) 2024-09-19T08:59:35.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:34 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:34.710+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:02.206362+0000 front 2024-09-19T08:59:02.206384+0000 (oldest deadline 2024-09-19T08:59:26.906456+0000) 2024-09-19T08:59:35.160 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:34 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:34.763+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:35.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:34 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:34.763+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.169174+0000 front 2024-09-19T08:58:57.169202+0000 (oldest deadline 2024-09-19T08:59:19.469292+0000) 2024-09-19T08:59:35.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:34 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:34.763+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:04.770508+0000 front 2024-09-19T08:59:04.770362+0000 (oldest deadline 2024-09-19T08:59:27.070397+0000) 2024-09-19T08:59:35.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:34 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:34.829+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:35.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:34 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:34.829+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:35.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:34 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:34.829+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:35.367 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:35 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:35.046+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.460189+0000 front 2024-09-19T08:58:49.460291+0000 (oldest deadline 2024-09-19T08:59:11.660880+0000) 2024-09-19T08:59:35.367 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:35 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:35.046+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:52.161680+0000 front 2024-09-19T08:58:52.161732+0000 (oldest deadline 2024-09-19T08:59:18.562288+0000) 2024-09-19T08:59:35.368 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:35 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:35.046+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:04.363927+0000 front 2024-09-19T08:59:04.364021+0000 (oldest deadline 2024-09-19T08:59:29.663818+0000) 2024-09-19T08:59:35.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:35 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:35.367+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:35.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:35 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:35.367+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:35.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:35 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:35.367+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:35.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:35 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:35.367+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:05.222908+0000 front 2024-09-19T08:59:05.222955+0000 (oldest deadline 2024-09-19T08:59:31.122786+0000) 2024-09-19T08:59:35.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:35 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:35.367+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:11.123265+0000 front 2024-09-19T08:59:11.123211+0000 (oldest deadline 2024-09-19T08:59:35.223367+0000) 2024-09-19T08:59:36.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:35 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:35.688+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:36.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:35 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:35.688+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:36.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:35 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:35.688+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.205419+0000 front 2024-09-19T08:58:55.205429+0000 (oldest deadline 2024-09-19T08:59:22.205873+0000) 2024-09-19T08:59:36.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:35 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:35.688+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:02.206362+0000 front 2024-09-19T08:59:02.206384+0000 (oldest deadline 2024-09-19T08:59:26.906456+0000) 2024-09-19T08:59:36.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:35 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:35.780+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:36.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:35 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:35.780+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:36.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:35 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:35.780+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:36.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:35 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:35.719+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:36.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:35 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:35.719+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.169174+0000 front 2024-09-19T08:58:57.169202+0000 (oldest deadline 2024-09-19T08:59:19.469292+0000) 2024-09-19T08:59:36.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:35 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:35.719+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:04.770508+0000 front 2024-09-19T08:59:04.770362+0000 (oldest deadline 2024-09-19T08:59:27.070397+0000) 2024-09-19T08:59:36.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:35 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:35.719+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:09.371581+0000 front 2024-09-19T08:59:09.371541+0000 (oldest deadline 2024-09-19T08:59:35.271596+0000) 2024-09-19T08:59:36.375 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:36 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:36.086+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.460189+0000 front 2024-09-19T08:58:49.460291+0000 (oldest deadline 2024-09-19T08:59:11.660880+0000) 2024-09-19T08:59:36.375 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:36 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:36.086+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:52.161680+0000 front 2024-09-19T08:58:52.161732+0000 (oldest deadline 2024-09-19T08:59:18.562288+0000) 2024-09-19T08:59:36.375 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:36 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:36.086+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:04.363927+0000 front 2024-09-19T08:59:04.364021+0000 (oldest deadline 2024-09-19T08:59:29.663818+0000) 2024-09-19T08:59:36.375 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:36 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:36.086+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:09.664245+0000 front 2024-09-19T08:59:09.664266+0000 (oldest deadline 2024-09-19T08:59:35.564285+0000) 2024-09-19T08:59:36.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:36 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:36.387+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:36.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:36 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:36.387+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:36.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:36 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:36.387+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:36.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:36 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:36.387+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:05.222908+0000 front 2024-09-19T08:59:05.222955+0000 (oldest deadline 2024-09-19T08:59:31.122786+0000) 2024-09-19T08:59:36.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:36 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:36.387+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:11.123265+0000 front 2024-09-19T08:59:11.123211+0000 (oldest deadline 2024-09-19T08:59:35.223367+0000) 2024-09-19T08:59:37.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:36 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:36.701+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:37.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:36 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:36.701+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:37.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:36 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:36.701+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.205419+0000 front 2024-09-19T08:58:55.205429+0000 (oldest deadline 2024-09-19T08:59:22.205873+0000) 2024-09-19T08:59:37.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:36 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:36.701+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:02.206362+0000 front 2024-09-19T08:59:02.206384+0000 (oldest deadline 2024-09-19T08:59:26.906456+0000) 2024-09-19T08:59:37.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:36 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:36.754+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:37.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:36 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:36.754+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.169174+0000 front 2024-09-19T08:58:57.169202+0000 (oldest deadline 2024-09-19T08:59:19.469292+0000) 2024-09-19T08:59:37.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:36 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:36.754+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:04.770508+0000 front 2024-09-19T08:59:04.770362+0000 (oldest deadline 2024-09-19T08:59:27.070397+0000) 2024-09-19T08:59:37.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:36 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:36.754+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:09.371581+0000 front 2024-09-19T08:59:09.371541+0000 (oldest deadline 2024-09-19T08:59:35.271596+0000) 2024-09-19T08:59:37.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:36 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:36.824+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:37.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:36 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:36.824+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:37.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:36 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:36.824+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:37.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:36 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:36.824+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.637103+0000 front 2024-09-19T08:59:13.637213+0000 (oldest deadline 2024-09-19T08:59:35.937098+0000) 2024-09-19T08:59:37.375 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:37 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:37.051+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.460189+0000 front 2024-09-19T08:58:49.460291+0000 (oldest deadline 2024-09-19T08:59:11.660880+0000) 2024-09-19T08:59:37.375 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:37 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:37.051+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:52.161680+0000 front 2024-09-19T08:58:52.161732+0000 (oldest deadline 2024-09-19T08:59:18.562288+0000) 2024-09-19T08:59:37.375 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:37 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:37.051+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:04.363927+0000 front 2024-09-19T08:59:04.364021+0000 (oldest deadline 2024-09-19T08:59:29.663818+0000) 2024-09-19T08:59:37.375 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:37 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:37.051+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:09.664245+0000 front 2024-09-19T08:59:09.664266+0000 (oldest deadline 2024-09-19T08:59:35.564285+0000) 2024-09-19T08:59:37.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:37 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:37.388+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:37.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:37 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:37.388+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:37.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:37 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:37.388+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:37.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:37 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:37.388+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:05.222908+0000 front 2024-09-19T08:59:05.222955+0000 (oldest deadline 2024-09-19T08:59:31.122786+0000) 2024-09-19T08:59:37.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:37 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:37.388+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:11.123265+0000 front 2024-09-19T08:59:11.123211+0000 (oldest deadline 2024-09-19T08:59:35.223367+0000) 2024-09-19T08:59:37.947 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:37 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:37.687+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:37.948 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:37 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:37.687+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:37.948 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:37 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:37.687+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.205419+0000 front 2024-09-19T08:58:55.205429+0000 (oldest deadline 2024-09-19T08:59:22.205873+0000) 2024-09-19T08:59:37.948 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:37 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:37.687+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:02.206362+0000 front 2024-09-19T08:59:02.206384+0000 (oldest deadline 2024-09-19T08:59:26.906456+0000) 2024-09-19T08:59:37.948 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:37 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:37.687+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.208595+0000 front 2024-09-19T08:59:13.208581+0000 (oldest deadline 2024-09-19T08:59:37.308391+0000) 2024-09-19T08:59:38.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:37 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:37.718+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:38.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:37 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:37.718+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.169174+0000 front 2024-09-19T08:58:57.169202+0000 (oldest deadline 2024-09-19T08:59:19.469292+0000) 2024-09-19T08:59:38.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:37 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:37.718+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:04.770508+0000 front 2024-09-19T08:59:04.770362+0000 (oldest deadline 2024-09-19T08:59:27.070397+0000) 2024-09-19T08:59:38.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:37 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:37.718+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:09.371581+0000 front 2024-09-19T08:59:09.371541+0000 (oldest deadline 2024-09-19T08:59:35.271596+0000) 2024-09-19T08:59:38.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:37 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:37.779+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:38.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:37 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:37.779+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:38.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:37 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:37.779+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:38.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:37 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:37.779+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.637103+0000 front 2024-09-19T08:59:13.637213+0000 (oldest deadline 2024-09-19T08:59:35.937098+0000) 2024-09-19T08:59:38.375 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:38 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:38.053+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.460189+0000 front 2024-09-19T08:58:49.460291+0000 (oldest deadline 2024-09-19T08:59:11.660880+0000) 2024-09-19T08:59:38.375 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:38 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:38.053+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:52.161680+0000 front 2024-09-19T08:58:52.161732+0000 (oldest deadline 2024-09-19T08:59:18.562288+0000) 2024-09-19T08:59:38.375 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:38 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:38.053+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:04.363927+0000 front 2024-09-19T08:59:04.364021+0000 (oldest deadline 2024-09-19T08:59:29.663818+0000) 2024-09-19T08:59:38.375 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:38 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7[52985]: 2024-09-19T08:59:38.053+0000 7f208ff04640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:09.664245+0000 front 2024-09-19T08:59:09.664266+0000 (oldest deadline 2024-09-19T08:59:35.564285+0000) 2024-09-19T08:59:38.749 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:38 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:38.433+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:38.749 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:38 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:38.433+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:38.749 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:38 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:38.433+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:38.749 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:38 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:38.433+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:05.222908+0000 front 2024-09-19T08:59:05.222955+0000 (oldest deadline 2024-09-19T08:59:31.122786+0000) 2024-09-19T08:59:38.749 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:38 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:38.433+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:11.123265+0000 front 2024-09-19T08:59:11.123211+0000 (oldest deadline 2024-09-19T08:59:35.223367+0000) 2024-09-19T08:59:39.031 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:38 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:38.720+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:39.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:38 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:38.720+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:39.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:38 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:38.720+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.205419+0000 front 2024-09-19T08:58:55.205429+0000 (oldest deadline 2024-09-19T08:59:22.205873+0000) 2024-09-19T08:59:39.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:38 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:38.720+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:02.206362+0000 front 2024-09-19T08:59:02.206384+0000 (oldest deadline 2024-09-19T08:59:26.906456+0000) 2024-09-19T08:59:39.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:38 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:38.720+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.208595+0000 front 2024-09-19T08:59:13.208581+0000 (oldest deadline 2024-09-19T08:59:37.308391+0000) 2024-09-19T08:59:39.037 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:38 smithi170 podman[71564]: 2024-09-19 08:59:38.750178073 +0000 UTC m=+5.748955637 container died cb4500d4e572c12a6226753bdbf733a91311fd41e4b43045e27cef3121dceeb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-19T08:59:39.160 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:38 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:38.698+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:39.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:38 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:38.698+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.169174+0000 front 2024-09-19T08:58:57.169202+0000 (oldest deadline 2024-09-19T08:59:19.469292+0000) 2024-09-19T08:59:39.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:38 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:38.698+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:04.770508+0000 front 2024-09-19T08:59:04.770362+0000 (oldest deadline 2024-09-19T08:59:27.070397+0000) 2024-09-19T08:59:39.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:38 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:38.698+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:09.371581+0000 front 2024-09-19T08:59:09.371541+0000 (oldest deadline 2024-09-19T08:59:35.271596+0000) 2024-09-19T08:59:39.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:38 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:38.740+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:39.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:38 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:38.740+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:39.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:38 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:38.740+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:39.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:38 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:38.740+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.637103+0000 front 2024-09-19T08:59:13.637213+0000 (oldest deadline 2024-09-19T08:59:35.937098+0000) 2024-09-19T08:59:39.375 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:39 smithi170 podman[71564]: 2024-09-19 08:59:39.128389178 +0000 UTC m=+6.127166741 container cleanup cb4500d4e572c12a6226753bdbf733a91311fd41e4b43045e27cef3121dceeb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-19T08:59:39.375 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:39 smithi170 bash[71564]: ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7 2024-09-19T08:59:39.724 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:39 smithi170 podman[71581]: 2024-09-19 08:59:39.413100672 +0000 UTC m=+0.662767610 container remove cb4500d4e572c12a6226753bdbf733a91311fd41e4b43045e27cef3121dceeb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-19T08:59:39.725 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:39 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:39.449+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:39.725 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:39 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:39.449+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:39.725 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:39 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:39.449+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:39.725 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:39 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:39.449+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:05.222908+0000 front 2024-09-19T08:59:05.222955+0000 (oldest deadline 2024-09-19T08:59:31.122786+0000) 2024-09-19T08:59:39.725 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:39 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:39.449+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:11.123265+0000 front 2024-09-19T08:59:11.123211+0000 (oldest deadline 2024-09-19T08:59:35.223367+0000) 2024-09-19T08:59:40.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:39 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:39.703+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:40.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:39 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:39.703+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:40.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:39 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:39.703+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.205419+0000 front 2024-09-19T08:58:55.205429+0000 (oldest deadline 2024-09-19T08:59:22.205873+0000) 2024-09-19T08:59:40.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:39 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:39.703+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:02.206362+0000 front 2024-09-19T08:59:02.206384+0000 (oldest deadline 2024-09-19T08:59:26.906456+0000) 2024-09-19T08:59:40.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:39 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:39.703+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.208595+0000 front 2024-09-19T08:59:13.208581+0000 (oldest deadline 2024-09-19T08:59:37.308391+0000) 2024-09-19T08:59:40.066 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:39 smithi170 podman[71664]: 2024-09-19 08:59:39.628004035 +0000 UTC m=+0.016106795 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:59:40.066 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:39 smithi170 podman[71664]: 2024-09-19 08:59:39.763947883 +0000 UTC m=+0.152050633 container create eec84ad25b28771916daeae21e9082ffbcaaef1776ae4c0cdb91ff7cb2480ba4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7-deactivate, RELEASE=squid-2f599a4, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-19T08:59:40.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:39 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:39.730+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:40.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:39 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:39.730+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.169174+0000 front 2024-09-19T08:58:57.169202+0000 (oldest deadline 2024-09-19T08:59:19.469292+0000) 2024-09-19T08:59:40.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:39 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:39.730+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:04.770508+0000 front 2024-09-19T08:59:04.770362+0000 (oldest deadline 2024-09-19T08:59:27.070397+0000) 2024-09-19T08:59:40.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:39 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:39.730+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:09.371581+0000 front 2024-09-19T08:59:09.371541+0000 (oldest deadline 2024-09-19T08:59:35.271596+0000) 2024-09-19T08:59:40.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:39 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:39.764+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:40.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:39 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:39.764+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:40.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:39 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:39.764+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:40.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:39 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:39.764+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.637103+0000 front 2024-09-19T08:59:13.637213+0000 (oldest deadline 2024-09-19T08:59:35.937098+0000) 2024-09-19T08:59:40.344 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:40 smithi170 podman[71664]: 2024-09-19 08:59:40.067235634 +0000 UTC m=+0.455338390 container init eec84ad25b28771916daeae21e9082ffbcaaef1776ae4c0cdb91ff7cb2480ba4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-19T08:59:40.344 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:40 smithi170 podman[71664]: 2024-09-19 08:59:40.070679073 +0000 UTC m=+0.458781831 container start eec84ad25b28771916daeae21e9082ffbcaaef1776ae4c0cdb91ff7cb2480ba4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD) 2024-09-19T08:59:40.344 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:40 smithi170 podman[71664]: 2024-09-19 08:59:40.133662663 +0000 UTC m=+0.521765427 container attach eec84ad25b28771916daeae21e9082ffbcaaef1776ae4c0cdb91ff7cb2480ba4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4) 2024-09-19T08:59:40.344 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:40 smithi170 podman[71664]: 2024-09-19 08:59:40.267020468 +0000 UTC m=+0.655123221 container died eec84ad25b28771916daeae21e9082ffbcaaef1776ae4c0cdb91ff7cb2480ba4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T08:59:40.594 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:40 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:40.448+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:40.595 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:40 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:40.448+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:40.595 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:40 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:40.448+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:40.595 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:40 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:40.448+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:05.222908+0000 front 2024-09-19T08:59:05.222955+0000 (oldest deadline 2024-09-19T08:59:31.122786+0000) 2024-09-19T08:59:40.595 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:40 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:40.448+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:11.123265+0000 front 2024-09-19T08:59:11.123211+0000 (oldest deadline 2024-09-19T08:59:35.223367+0000) 2024-09-19T08:59:40.844 INFO:journalctl@ceph.osd.7.smithi170.stdout:Sep 19 08:59:40 smithi170 podman[71688]: 2024-09-19 08:59:40.828863973 +0000 UTC m=+0.555505257 container remove eec84ad25b28771916daeae21e9082ffbcaaef1776ae4c0cdb91ff7cb2480ba4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-7-deactivate, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-19T08:59:40.849 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.7.service' 2024-09-19T08:59:40.928 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:59:40.928 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-19T08:59:40.928 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-19T08:59:40.928 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.8 2024-09-19T08:59:40.955 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:40 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:40.684+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:40.955 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:40 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:40.684+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.169174+0000 front 2024-09-19T08:58:57.169202+0000 (oldest deadline 2024-09-19T08:59:19.469292+0000) 2024-09-19T08:59:40.955 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:40 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:40.684+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:04.770508+0000 front 2024-09-19T08:59:04.770362+0000 (oldest deadline 2024-09-19T08:59:27.070397+0000) 2024-09-19T08:59:40.955 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:40 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:40.684+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:09.371581+0000 front 2024-09-19T08:59:09.371541+0000 (oldest deadline 2024-09-19T08:59:35.271596+0000) 2024-09-19T08:59:40.955 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:40 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:40.813+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:40.955 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:40 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:40.813+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:40.955 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:40 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:40.813+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:40.955 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:40 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:40.813+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.637103+0000 front 2024-09-19T08:59:13.637213+0000 (oldest deadline 2024-09-19T08:59:35.937098+0000) 2024-09-19T08:59:41.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:40 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:40.746+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:41.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:40 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:40.746+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:41.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:40 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:40.746+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.205419+0000 front 2024-09-19T08:58:55.205429+0000 (oldest deadline 2024-09-19T08:59:22.205873+0000) 2024-09-19T08:59:41.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:40 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:40.746+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:02.206362+0000 front 2024-09-19T08:59:02.206384+0000 (oldest deadline 2024-09-19T08:59:26.906456+0000) 2024-09-19T08:59:41.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:40 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:40.746+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.208595+0000 front 2024-09-19T08:59:13.208581+0000 (oldest deadline 2024-09-19T08:59:37.308391+0000) 2024-09-19T08:59:41.250 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:40 smithi173 systemd[1]: Stopping Ceph osd.8 for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T08:59:41.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:41 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:41.408+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:41.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:41 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:41.408+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:41.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:41 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:41.408+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:41.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:41 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:41.408+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:05.222908+0000 front 2024-09-19T08:59:05.222955+0000 (oldest deadline 2024-09-19T08:59:31.122786+0000) 2024-09-19T08:59:41.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:41 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:41.408+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:11.123265+0000 front 2024-09-19T08:59:11.123211+0000 (oldest deadline 2024-09-19T08:59:35.223367+0000) 2024-09-19T08:59:41.641 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:41 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:41.250+0000 7f67ecec6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:59:41.641 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:41 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:41.250+0000 7f67ecec6640 -1 osd.8 100 *** Got signal Terminated *** 2024-09-19T08:59:41.641 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:41 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:41.250+0000 7f67ecec6640 -1 osd.8 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T08:59:41.641 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:41 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:41.641+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:41.641 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:41 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:41.641+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.169174+0000 front 2024-09-19T08:58:57.169202+0000 (oldest deadline 2024-09-19T08:59:19.469292+0000) 2024-09-19T08:59:41.641 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:41 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:41.641+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:04.770508+0000 front 2024-09-19T08:59:04.770362+0000 (oldest deadline 2024-09-19T08:59:27.070397+0000) 2024-09-19T08:59:41.641 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:41 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:41.641+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:09.371581+0000 front 2024-09-19T08:59:09.371541+0000 (oldest deadline 2024-09-19T08:59:35.271596+0000) 2024-09-19T08:59:41.911 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:41 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:41.822+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:41.911 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:41 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:41.822+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:41.911 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:41 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:41.822+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:41.911 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:41 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:41.822+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.637103+0000 front 2024-09-19T08:59:13.637213+0000 (oldest deadline 2024-09-19T08:59:35.937098+0000) 2024-09-19T08:59:42.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:41 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:41.785+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:42.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:41 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:41.785+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:42.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:41 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:41.785+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.205419+0000 front 2024-09-19T08:58:55.205429+0000 (oldest deadline 2024-09-19T08:59:22.205873+0000) 2024-09-19T08:59:42.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:41 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:41.785+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:02.206362+0000 front 2024-09-19T08:59:02.206384+0000 (oldest deadline 2024-09-19T08:59:26.906456+0000) 2024-09-19T08:59:42.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:41 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:41.785+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.208595+0000 front 2024-09-19T08:59:13.208581+0000 (oldest deadline 2024-09-19T08:59:37.308391+0000) 2024-09-19T08:59:42.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:42 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:42.365+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:42.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:42 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:42.365+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:42.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:42 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:42.366+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:42.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:42 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:42.366+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:05.222908+0000 front 2024-09-19T08:59:05.222955+0000 (oldest deadline 2024-09-19T08:59:31.122786+0000) 2024-09-19T08:59:42.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:42 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:42.366+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:11.123265+0000 front 2024-09-19T08:59:11.123211+0000 (oldest deadline 2024-09-19T08:59:35.223367+0000) 2024-09-19T08:59:43.160 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:42 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:42.873+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:43.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:42 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:42.873+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:43.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:42 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:42.873+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:43.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:42 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:42.873+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.637103+0000 front 2024-09-19T08:59:13.637213+0000 (oldest deadline 2024-09-19T08:59:35.937098+0000) 2024-09-19T08:59:43.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:42 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:42.684+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:43.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:42 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:42.684+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.169174+0000 front 2024-09-19T08:58:57.169202+0000 (oldest deadline 2024-09-19T08:59:19.469292+0000) 2024-09-19T08:59:43.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:42 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:42.684+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:04.770508+0000 front 2024-09-19T08:59:04.770362+0000 (oldest deadline 2024-09-19T08:59:27.070397+0000) 2024-09-19T08:59:43.161 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:42 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:42.684+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:09.371581+0000 front 2024-09-19T08:59:09.371541+0000 (oldest deadline 2024-09-19T08:59:35.271596+0000) 2024-09-19T08:59:43.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:42 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:42.804+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:43.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:42 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:42.804+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:43.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:42 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:42.804+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.205419+0000 front 2024-09-19T08:58:55.205429+0000 (oldest deadline 2024-09-19T08:59:22.205873+0000) 2024-09-19T08:59:43.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:42 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:42.804+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:02.206362+0000 front 2024-09-19T08:59:02.206384+0000 (oldest deadline 2024-09-19T08:59:26.906456+0000) 2024-09-19T08:59:43.283 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:42 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:42.804+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.208595+0000 front 2024-09-19T08:59:13.208581+0000 (oldest deadline 2024-09-19T08:59:37.308391+0000) 2024-09-19T08:59:43.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:43 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:43.373+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:43.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:43 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:43.373+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:43.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:43 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:43.373+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:43.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:43 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:43.373+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:05.222908+0000 front 2024-09-19T08:59:05.222955+0000 (oldest deadline 2024-09-19T08:59:31.122786+0000) 2024-09-19T08:59:43.626 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:43 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:43.373+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:11.123265+0000 front 2024-09-19T08:59:11.123211+0000 (oldest deadline 2024-09-19T08:59:35.223367+0000) 2024-09-19T08:59:43.910 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:43 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:43.647+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:43.911 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:43 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:43.647+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.169174+0000 front 2024-09-19T08:58:57.169202+0000 (oldest deadline 2024-09-19T08:59:19.469292+0000) 2024-09-19T08:59:43.911 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:43 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:43.647+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:04.770508+0000 front 2024-09-19T08:59:04.770362+0000 (oldest deadline 2024-09-19T08:59:27.070397+0000) 2024-09-19T08:59:43.911 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:43 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:43.647+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:09.371581+0000 front 2024-09-19T08:59:09.371541+0000 (oldest deadline 2024-09-19T08:59:35.271596+0000) 2024-09-19T08:59:43.911 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:43 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:43.844+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:43.911 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:43 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:43.844+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:43.911 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:43 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:43.844+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:43.911 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:43 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:43.844+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.637103+0000 front 2024-09-19T08:59:13.637213+0000 (oldest deadline 2024-09-19T08:59:35.937098+0000) 2024-09-19T08:59:43.911 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:43 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:43.844+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:21.138496+0000 front 2024-09-19T08:59:21.138554+0000 (oldest deadline 2024-09-19T08:59:43.438472+0000) 2024-09-19T08:59:44.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:43 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:43.792+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:44.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:43 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:43.792+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:44.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:43 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:43.792+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.205419+0000 front 2024-09-19T08:58:55.205429+0000 (oldest deadline 2024-09-19T08:59:22.205873+0000) 2024-09-19T08:59:44.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:43 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:43.792+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:02.206362+0000 front 2024-09-19T08:59:02.206384+0000 (oldest deadline 2024-09-19T08:59:26.906456+0000) 2024-09-19T08:59:44.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:43 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:43.792+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.208595+0000 front 2024-09-19T08:59:13.208581+0000 (oldest deadline 2024-09-19T08:59:37.308391+0000) 2024-09-19T08:59:44.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:43 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:43.792+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:17.809227+0000 front 2024-09-19T08:59:17.809265+0000 (oldest deadline 2024-09-19T08:59:43.709200+0000) 2024-09-19T08:59:44.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:44 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:44.396+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:44.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:44 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:44.396+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:44.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:44 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:44.396+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:44.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:44 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:44.396+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:05.222908+0000 front 2024-09-19T08:59:05.222955+0000 (oldest deadline 2024-09-19T08:59:31.122786+0000) 2024-09-19T08:59:44.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:44 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:44.396+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:11.123265+0000 front 2024-09-19T08:59:11.123211+0000 (oldest deadline 2024-09-19T08:59:35.223367+0000) 2024-09-19T08:59:44.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:44 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:44.396+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:18.625037+0000 front 2024-09-19T08:59:18.624943+0000 (oldest deadline 2024-09-19T08:59:43.925164+0000) 2024-09-19T08:59:45.160 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:44 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:44.679+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:45.162 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:44 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:44.679+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.169174+0000 front 2024-09-19T08:58:57.169202+0000 (oldest deadline 2024-09-19T08:59:19.469292+0000) 2024-09-19T08:59:45.162 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:44 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:44.679+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:04.770508+0000 front 2024-09-19T08:59:04.770362+0000 (oldest deadline 2024-09-19T08:59:27.070397+0000) 2024-09-19T08:59:45.162 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:44 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:44.679+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:09.371581+0000 front 2024-09-19T08:59:09.371541+0000 (oldest deadline 2024-09-19T08:59:35.271596+0000) 2024-09-19T08:59:45.162 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:44 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:44.845+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:45.162 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:44 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:44.845+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:45.162 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:44 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:44.845+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:45.162 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:44 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:44.845+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.637103+0000 front 2024-09-19T08:59:13.637213+0000 (oldest deadline 2024-09-19T08:59:35.937098+0000) 2024-09-19T08:59:45.163 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:44 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:44.845+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:21.138496+0000 front 2024-09-19T08:59:21.138554+0000 (oldest deadline 2024-09-19T08:59:43.438472+0000) 2024-09-19T08:59:45.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:44 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:44.830+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:45.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:44 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:44.830+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:45.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:44 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:44.830+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.205419+0000 front 2024-09-19T08:58:55.205429+0000 (oldest deadline 2024-09-19T08:59:22.205873+0000) 2024-09-19T08:59:45.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:44 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:44.830+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:02.206362+0000 front 2024-09-19T08:59:02.206384+0000 (oldest deadline 2024-09-19T08:59:26.906456+0000) 2024-09-19T08:59:45.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:44 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:44.830+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.208595+0000 front 2024-09-19T08:59:13.208581+0000 (oldest deadline 2024-09-19T08:59:37.308391+0000) 2024-09-19T08:59:45.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:44 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:44.830+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:17.809227+0000 front 2024-09-19T08:59:17.809265+0000 (oldest deadline 2024-09-19T08:59:43.709200+0000) 2024-09-19T08:59:45.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:45 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:45.384+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:45.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:45 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:45.384+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:45.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:45 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:45.384+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:45.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:45 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:45.384+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:05.222908+0000 front 2024-09-19T08:59:05.222955+0000 (oldest deadline 2024-09-19T08:59:31.122786+0000) 2024-09-19T08:59:45.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:45 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:45.384+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:11.123265+0000 front 2024-09-19T08:59:11.123211+0000 (oldest deadline 2024-09-19T08:59:35.223367+0000) 2024-09-19T08:59:45.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:45 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:45.384+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:18.625037+0000 front 2024-09-19T08:59:18.624943+0000 (oldest deadline 2024-09-19T08:59:43.925164+0000) 2024-09-19T08:59:45.910 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:45 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:45.635+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:47.768468+0000 front 2024-09-19T08:58:47.768377+0000 (oldest deadline 2024-09-19T08:59:11.868206+0000) 2024-09-19T08:59:45.911 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:45 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:45.635+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.169174+0000 front 2024-09-19T08:58:57.169202+0000 (oldest deadline 2024-09-19T08:59:19.469292+0000) 2024-09-19T08:59:45.911 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:45 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:45.635+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:04.770508+0000 front 2024-09-19T08:59:04.770362+0000 (oldest deadline 2024-09-19T08:59:27.070397+0000) 2024-09-19T08:59:45.911 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:45 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8[53648]: 2024-09-19T08:59:45.635+0000 7f67e54d4640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:09.371581+0000 front 2024-09-19T08:59:09.371541+0000 (oldest deadline 2024-09-19T08:59:35.271596+0000) 2024-09-19T08:59:45.911 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:45 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:45.870+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:45.911 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:45 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:45.870+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:45.911 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:45 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:45.870+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:45.911 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:45 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:45.870+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.637103+0000 front 2024-09-19T08:59:13.637213+0000 (oldest deadline 2024-09-19T08:59:35.937098+0000) 2024-09-19T08:59:45.911 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:45 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:45.870+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:21.138496+0000 front 2024-09-19T08:59:21.138554+0000 (oldest deadline 2024-09-19T08:59:43.438472+0000) 2024-09-19T08:59:46.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:45 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:45.790+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:46.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:45 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:45.790+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:46.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:45 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:45.790+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.205419+0000 front 2024-09-19T08:58:55.205429+0000 (oldest deadline 2024-09-19T08:59:22.205873+0000) 2024-09-19T08:59:46.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:45 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:45.790+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:02.206362+0000 front 2024-09-19T08:59:02.206384+0000 (oldest deadline 2024-09-19T08:59:26.906456+0000) 2024-09-19T08:59:46.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:45 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:45.790+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.208595+0000 front 2024-09-19T08:59:13.208581+0000 (oldest deadline 2024-09-19T08:59:37.308391+0000) 2024-09-19T08:59:46.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:45 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:45.790+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:17.809227+0000 front 2024-09-19T08:59:17.809265+0000 (oldest deadline 2024-09-19T08:59:43.709200+0000) 2024-09-19T08:59:46.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:46 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:46.388+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:46.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:46 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:46.388+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:46.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:46 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:46.388+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:46.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:46 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:46.388+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:05.222908+0000 front 2024-09-19T08:59:05.222955+0000 (oldest deadline 2024-09-19T08:59:31.122786+0000) 2024-09-19T08:59:46.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:46 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:46.388+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:11.123265+0000 front 2024-09-19T08:59:11.123211+0000 (oldest deadline 2024-09-19T08:59:35.223367+0000) 2024-09-19T08:59:46.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:46 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:46.388+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:18.625037+0000 front 2024-09-19T08:59:18.624943+0000 (oldest deadline 2024-09-19T08:59:43.925164+0000) 2024-09-19T08:59:47.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:46 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:46.749+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:47.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:46 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:46.749+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:47.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:46 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:46.749+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.205419+0000 front 2024-09-19T08:58:55.205429+0000 (oldest deadline 2024-09-19T08:59:22.205873+0000) 2024-09-19T08:59:47.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:46 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:46.749+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:02.206362+0000 front 2024-09-19T08:59:02.206384+0000 (oldest deadline 2024-09-19T08:59:26.906456+0000) 2024-09-19T08:59:47.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:46 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:46.749+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.208595+0000 front 2024-09-19T08:59:13.208581+0000 (oldest deadline 2024-09-19T08:59:37.308391+0000) 2024-09-19T08:59:47.032 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:46 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:46.749+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:17.809227+0000 front 2024-09-19T08:59:17.809265+0000 (oldest deadline 2024-09-19T08:59:43.709200+0000) 2024-09-19T08:59:47.109 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:46 smithi173 podman[71968]: 2024-09-19 08:59:46.796856161 +0000 UTC m=+5.666665052 container died adb4883fa29ddbbfc013432c84650e34f74896b61112031050057a42a49345f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS) 2024-09-19T08:59:47.109 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:46 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:46.835+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:47.110 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:46 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:46.835+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:47.110 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:46 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:46.835+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:47.110 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:46 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:46.835+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.637103+0000 front 2024-09-19T08:59:13.637213+0000 (oldest deadline 2024-09-19T08:59:35.937098+0000) 2024-09-19T08:59:47.110 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:46 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:46.835+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:21.138496+0000 front 2024-09-19T08:59:21.138554+0000 (oldest deadline 2024-09-19T08:59:43.438472+0000) 2024-09-19T08:59:47.410 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:47 smithi173 podman[71968]: 2024-09-19 08:59:47.201273459 +0000 UTC m=+6.071082337 container cleanup adb4883fa29ddbbfc013432c84650e34f74896b61112031050057a42a49345f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-19T08:59:47.410 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:47 smithi173 bash[71968]: ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8 2024-09-19T08:59:47.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:47 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:47.360+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:47.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:47 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:47.360+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:47.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:47 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:47.360+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:47.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:47 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:47.360+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:05.222908+0000 front 2024-09-19T08:59:05.222955+0000 (oldest deadline 2024-09-19T08:59:31.122786+0000) 2024-09-19T08:59:47.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:47 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:47.360+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:11.123265+0000 front 2024-09-19T08:59:11.123211+0000 (oldest deadline 2024-09-19T08:59:35.223367+0000) 2024-09-19T08:59:47.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:47 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:47.360+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:18.625037+0000 front 2024-09-19T08:59:18.624943+0000 (oldest deadline 2024-09-19T08:59:43.925164+0000) 2024-09-19T08:59:47.761 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:47 smithi173 podman[71985]: 2024-09-19 08:59:47.428036937 +0000 UTC m=+0.631651197 container remove adb4883fa29ddbbfc013432c84650e34f74896b61112031050057a42a49345f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731) 2024-09-19T08:59:48.031 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:47 smithi173 podman[72073]: 2024-09-19 08:59:47.665481492 +0000 UTC m=+0.015696147 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:59:48.031 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:47 smithi173 podman[72073]: 2024-09-19 08:59:47.787552976 +0000 UTC m=+0.137767658 container create eaec35961f270dbf4a278c16c0a875b57f36ae5825de4760417742a6c2380152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8-deactivate, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-19T08:59:48.031 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:47 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:47.819+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:48.031 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:47 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:47.819+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:48.031 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:47 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:47.819+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:48.032 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:47 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:47.819+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.637103+0000 front 2024-09-19T08:59:13.637213+0000 (oldest deadline 2024-09-19T08:59:35.937098+0000) 2024-09-19T08:59:48.032 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:47 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:47.819+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:21.138496+0000 front 2024-09-19T08:59:21.138554+0000 (oldest deadline 2024-09-19T08:59:43.438472+0000) 2024-09-19T08:59:48.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:47 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:47.784+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:48.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:47 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:47.784+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:48.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:47 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:47.784+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.205419+0000 front 2024-09-19T08:58:55.205429+0000 (oldest deadline 2024-09-19T08:59:22.205873+0000) 2024-09-19T08:59:48.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:47 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:47.784+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:02.206362+0000 front 2024-09-19T08:59:02.206384+0000 (oldest deadline 2024-09-19T08:59:26.906456+0000) 2024-09-19T08:59:48.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:47 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:47.784+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.208595+0000 front 2024-09-19T08:59:13.208581+0000 (oldest deadline 2024-09-19T08:59:37.308391+0000) 2024-09-19T08:59:48.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:47 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:47.784+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:17.809227+0000 front 2024-09-19T08:59:17.809265+0000 (oldest deadline 2024-09-19T08:59:43.709200+0000) 2024-09-19T08:59:48.411 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:48 smithi173 podman[72073]: 2024-09-19 08:59:48.031985718 +0000 UTC m=+0.382200358 container init eaec35961f270dbf4a278c16c0a875b57f36ae5825de4760417742a6c2380152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-19T08:59:48.411 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:48 smithi173 podman[72073]: 2024-09-19 08:59:48.035833628 +0000 UTC m=+0.386048273 container start eaec35961f270dbf4a278c16c0a875b57f36ae5825de4760417742a6c2380152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-19T08:59:48.411 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:48 smithi173 podman[72073]: 2024-09-19 08:59:48.157262164 +0000 UTC m=+0.507476807 container attach eaec35961f270dbf4a278c16c0a875b57f36ae5825de4760417742a6c2380152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T08:59:48.411 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 19 08:59:48 smithi173 podman[72073]: 2024-09-19 08:59:48.225298099 +0000 UTC m=+0.575512750 container died eaec35961f270dbf4a278c16c0a875b57f36ae5825de4760417742a6c2380152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-19T08:59:48.803 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.8.service' 2024-09-19T08:59:48.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:48 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:48.394+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:48.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:48 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:48.394+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:48.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:48 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:48.394+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:48.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:48 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:48.394+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:05.222908+0000 front 2024-09-19T08:59:05.222955+0000 (oldest deadline 2024-09-19T08:59:31.122786+0000) 2024-09-19T08:59:48.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:48 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:48.394+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:11.123265+0000 front 2024-09-19T08:59:11.123211+0000 (oldest deadline 2024-09-19T08:59:35.223367+0000) 2024-09-19T08:59:48.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:48 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:48.394+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:18.625037+0000 front 2024-09-19T08:59:18.624943+0000 (oldest deadline 2024-09-19T08:59:43.925164+0000) 2024-09-19T08:59:48.884 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:59:48.884 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-19T08:59:48.884 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-19T08:59:48.885 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.9 2024-09-19T08:59:48.943 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:48 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:48.829+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:48.943 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:48 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:48.829+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:48.943 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:48 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:48.829+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.205419+0000 front 2024-09-19T08:58:55.205429+0000 (oldest deadline 2024-09-19T08:59:22.205873+0000) 2024-09-19T08:59:48.943 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:48 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:48.829+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:02.206362+0000 front 2024-09-19T08:59:02.206384+0000 (oldest deadline 2024-09-19T08:59:26.906456+0000) 2024-09-19T08:59:48.943 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:48 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:48.829+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.208595+0000 front 2024-09-19T08:59:13.208581+0000 (oldest deadline 2024-09-19T08:59:37.308391+0000) 2024-09-19T08:59:48.943 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:48 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:48.829+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:17.809227+0000 front 2024-09-19T08:59:17.809265+0000 (oldest deadline 2024-09-19T08:59:43.709200+0000) 2024-09-19T08:59:48.943 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:48 smithi164 systemd[1]: Stopping Ceph osd.9 for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T08:59:49.160 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:48 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:48.783+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:49.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:48 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:48.783+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:49.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:48 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:48.783+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:49.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:48 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:48.783+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.637103+0000 front 2024-09-19T08:59:13.637213+0000 (oldest deadline 2024-09-19T08:59:35.937098+0000) 2024-09-19T08:59:49.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:48 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:48.783+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:21.138496+0000 front 2024-09-19T08:59:21.138554+0000 (oldest deadline 2024-09-19T08:59:43.438472+0000) 2024-09-19T08:59:49.532 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:49 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:49.215+0000 7f42873b8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:59:49.532 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:49 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:49.215+0000 7f42873b8640 -1 osd.9 100 *** Got signal Terminated *** 2024-09-19T08:59:49.532 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:49 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:49.215+0000 7f42873b8640 -1 osd.9 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T08:59:49.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:49 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:49.363+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:49.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:49 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:49.363+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:49.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:49 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:49.363+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:49.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:49 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:49.363+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:05.222908+0000 front 2024-09-19T08:59:05.222955+0000 (oldest deadline 2024-09-19T08:59:31.122786+0000) 2024-09-19T08:59:49.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:49 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:49.363+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:11.123265+0000 front 2024-09-19T08:59:11.123211+0000 (oldest deadline 2024-09-19T08:59:35.223367+0000) 2024-09-19T08:59:49.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:49 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:49.363+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:18.625037+0000 front 2024-09-19T08:59:18.624943+0000 (oldest deadline 2024-09-19T08:59:43.925164+0000) 2024-09-19T08:59:50.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:49 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:49.753+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:50.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:49 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:49.753+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:50.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:49 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:49.753+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:50.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:49 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:49.753+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.637103+0000 front 2024-09-19T08:59:13.637213+0000 (oldest deadline 2024-09-19T08:59:35.937098+0000) 2024-09-19T08:59:50.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:49 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:49.753+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:21.138496+0000 front 2024-09-19T08:59:21.138554+0000 (oldest deadline 2024-09-19T08:59:43.438472+0000) 2024-09-19T08:59:50.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:49 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:49.846+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:50.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:49 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:49.846+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:50.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:49 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:49.846+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.205419+0000 front 2024-09-19T08:58:55.205429+0000 (oldest deadline 2024-09-19T08:59:22.205873+0000) 2024-09-19T08:59:50.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:49 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:49.846+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:02.206362+0000 front 2024-09-19T08:59:02.206384+0000 (oldest deadline 2024-09-19T08:59:26.906456+0000) 2024-09-19T08:59:50.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:49 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:49.846+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.208595+0000 front 2024-09-19T08:59:13.208581+0000 (oldest deadline 2024-09-19T08:59:37.308391+0000) 2024-09-19T08:59:50.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:49 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:49.846+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:17.809227+0000 front 2024-09-19T08:59:17.809265+0000 (oldest deadline 2024-09-19T08:59:43.709200+0000) 2024-09-19T08:59:50.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:50 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:50.341+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:50.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:50 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:50.341+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:50.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:50 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:50.341+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:50.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:50 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:50.341+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:05.222908+0000 front 2024-09-19T08:59:05.222955+0000 (oldest deadline 2024-09-19T08:59:31.122786+0000) 2024-09-19T08:59:50.626 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:50 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:50.341+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:11.123265+0000 front 2024-09-19T08:59:11.123211+0000 (oldest deadline 2024-09-19T08:59:35.223367+0000) 2024-09-19T08:59:50.626 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:50 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:50.341+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:18.625037+0000 front 2024-09-19T08:59:18.624943+0000 (oldest deadline 2024-09-19T08:59:43.925164+0000) 2024-09-19T08:59:51.158 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:50 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:50.765+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:51.158 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:50 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:50.765+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:51.158 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:50 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:50.765+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:51.159 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:50 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:50.765+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.637103+0000 front 2024-09-19T08:59:13.637213+0000 (oldest deadline 2024-09-19T08:59:35.937098+0000) 2024-09-19T08:59:51.159 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:50 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:50.765+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:21.138496+0000 front 2024-09-19T08:59:21.138554+0000 (oldest deadline 2024-09-19T08:59:43.438472+0000) 2024-09-19T08:59:51.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:50 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:50.860+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:51.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:50 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:50.860+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:51.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:50 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:50.860+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.205419+0000 front 2024-09-19T08:58:55.205429+0000 (oldest deadline 2024-09-19T08:59:22.205873+0000) 2024-09-19T08:59:51.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:50 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:50.860+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:02.206362+0000 front 2024-09-19T08:59:02.206384+0000 (oldest deadline 2024-09-19T08:59:26.906456+0000) 2024-09-19T08:59:51.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:50 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:50.860+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.208595+0000 front 2024-09-19T08:59:13.208581+0000 (oldest deadline 2024-09-19T08:59:37.308391+0000) 2024-09-19T08:59:51.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:50 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:50.860+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:17.809227+0000 front 2024-09-19T08:59:17.809265+0000 (oldest deadline 2024-09-19T08:59:43.709200+0000) 2024-09-19T08:59:51.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:51 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:51.375+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:51.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:51 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:51.375+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:51.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:51 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:51.375+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:51.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:51 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:51.375+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:05.222908+0000 front 2024-09-19T08:59:05.222955+0000 (oldest deadline 2024-09-19T08:59:31.122786+0000) 2024-09-19T08:59:51.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:51 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:51.375+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:11.123265+0000 front 2024-09-19T08:59:11.123211+0000 (oldest deadline 2024-09-19T08:59:35.223367+0000) 2024-09-19T08:59:51.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:51 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:51.375+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:18.625037+0000 front 2024-09-19T08:59:18.624943+0000 (oldest deadline 2024-09-19T08:59:43.925164+0000) 2024-09-19T08:59:51.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:51 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:51.375+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6822 osd.6 since back 2024-09-19T08:59:27.426197+0000 front 2024-09-19T08:59:27.426168+0000 (oldest deadline 2024-09-19T08:59:50.826300+0000) 2024-09-19T08:59:52.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:51 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:51.755+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:52.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:51 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:51.755+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:52.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:51 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:51.755+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:52.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:51 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:51.755+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.637103+0000 front 2024-09-19T08:59:13.637213+0000 (oldest deadline 2024-09-19T08:59:35.937098+0000) 2024-09-19T08:59:52.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:51 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:51.755+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:21.138496+0000 front 2024-09-19T08:59:21.138554+0000 (oldest deadline 2024-09-19T08:59:43.438472+0000) 2024-09-19T08:59:52.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:51 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:51.755+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6822 osd.6 since back 2024-09-19T08:59:29.839897+0000 front 2024-09-19T08:59:29.839868+0000 (oldest deadline 2024-09-19T08:59:51.539926+0000) 2024-09-19T08:59:52.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:51 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:51.881+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:52.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:51 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:51.881+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:52.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:51 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:51.881+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.205419+0000 front 2024-09-19T08:58:55.205429+0000 (oldest deadline 2024-09-19T08:59:22.205873+0000) 2024-09-19T08:59:52.283 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:51 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:51.881+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:02.206362+0000 front 2024-09-19T08:59:02.206384+0000 (oldest deadline 2024-09-19T08:59:26.906456+0000) 2024-09-19T08:59:52.283 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:51 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:51.881+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.208595+0000 front 2024-09-19T08:59:13.208581+0000 (oldest deadline 2024-09-19T08:59:37.308391+0000) 2024-09-19T08:59:52.283 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:51 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:51.881+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:17.809227+0000 front 2024-09-19T08:59:17.809265+0000 (oldest deadline 2024-09-19T08:59:43.709200+0000) 2024-09-19T08:59:52.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:52 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:52.365+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:52.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:52 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:52.365+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:52.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:52 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:52.365+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:52.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:52 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:52.365+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:05.222908+0000 front 2024-09-19T08:59:05.222955+0000 (oldest deadline 2024-09-19T08:59:31.122786+0000) 2024-09-19T08:59:52.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:52 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:52.365+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:11.123265+0000 front 2024-09-19T08:59:11.123211+0000 (oldest deadline 2024-09-19T08:59:35.223367+0000) 2024-09-19T08:59:52.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:52 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:52.365+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:18.625037+0000 front 2024-09-19T08:59:18.624943+0000 (oldest deadline 2024-09-19T08:59:43.925164+0000) 2024-09-19T08:59:52.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:52 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:52.365+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6822 osd.6 since back 2024-09-19T08:59:27.426197+0000 front 2024-09-19T08:59:27.426168+0000 (oldest deadline 2024-09-19T08:59:50.826300+0000) 2024-09-19T08:59:53.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:52 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:52.791+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:53.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:52 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:52.791+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:53.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:52 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:52.791+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:53.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:52 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:52.791+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.637103+0000 front 2024-09-19T08:59:13.637213+0000 (oldest deadline 2024-09-19T08:59:35.937098+0000) 2024-09-19T08:59:53.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:52 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:52.791+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:21.138496+0000 front 2024-09-19T08:59:21.138554+0000 (oldest deadline 2024-09-19T08:59:43.438472+0000) 2024-09-19T08:59:53.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:52 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:52.791+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6822 osd.6 since back 2024-09-19T08:59:29.839897+0000 front 2024-09-19T08:59:29.839868+0000 (oldest deadline 2024-09-19T08:59:51.539926+0000) 2024-09-19T08:59:53.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:52 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:52.908+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:53.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:52 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:52.908+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:53.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:52 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:52.908+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.205419+0000 front 2024-09-19T08:58:55.205429+0000 (oldest deadline 2024-09-19T08:59:22.205873+0000) 2024-09-19T08:59:53.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:52 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:52.908+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:02.206362+0000 front 2024-09-19T08:59:02.206384+0000 (oldest deadline 2024-09-19T08:59:26.906456+0000) 2024-09-19T08:59:53.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:52 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:52.908+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.208595+0000 front 2024-09-19T08:59:13.208581+0000 (oldest deadline 2024-09-19T08:59:37.308391+0000) 2024-09-19T08:59:53.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:52 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:52.908+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:17.809227+0000 front 2024-09-19T08:59:17.809265+0000 (oldest deadline 2024-09-19T08:59:43.709200+0000) 2024-09-19T08:59:53.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:53 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:53.337+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:53.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:53 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:53.337+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:53.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:53 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:53.337+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:53.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:53 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:53.337+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:05.222908+0000 front 2024-09-19T08:59:05.222955+0000 (oldest deadline 2024-09-19T08:59:31.122786+0000) 2024-09-19T08:59:53.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:53 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:53.337+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:11.123265+0000 front 2024-09-19T08:59:11.123211+0000 (oldest deadline 2024-09-19T08:59:35.223367+0000) 2024-09-19T08:59:53.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:53 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:53.337+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:18.625037+0000 front 2024-09-19T08:59:18.624943+0000 (oldest deadline 2024-09-19T08:59:43.925164+0000) 2024-09-19T08:59:53.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:53 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:53.337+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6822 osd.6 since back 2024-09-19T08:59:27.426197+0000 front 2024-09-19T08:59:27.426168+0000 (oldest deadline 2024-09-19T08:59:50.826300+0000) 2024-09-19T08:59:54.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:53 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:53.823+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:54.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:53 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:53.823+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:54.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:53 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:53.823+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:54.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:53 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:53.823+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.637103+0000 front 2024-09-19T08:59:13.637213+0000 (oldest deadline 2024-09-19T08:59:35.937098+0000) 2024-09-19T08:59:54.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:53 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:53.823+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:21.138496+0000 front 2024-09-19T08:59:21.138554+0000 (oldest deadline 2024-09-19T08:59:43.438472+0000) 2024-09-19T08:59:54.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:53 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:53.823+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6822 osd.6 since back 2024-09-19T08:59:29.839897+0000 front 2024-09-19T08:59:29.839868+0000 (oldest deadline 2024-09-19T08:59:51.539926+0000) 2024-09-19T08:59:54.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:53 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:53.920+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.602896+0000 front 2024-09-19T08:58:40.602869+0000 (oldest deadline 2024-09-19T08:59:05.303080+0000) 2024-09-19T08:59:54.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:53 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:53.920+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:48.204394+0000 front 2024-09-19T08:58:48.204414+0000 (oldest deadline 2024-09-19T08:59:12.304436+0000) 2024-09-19T08:59:54.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:53 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:53.920+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:55.205419+0000 front 2024-09-19T08:58:55.205429+0000 (oldest deadline 2024-09-19T08:59:22.205873+0000) 2024-09-19T08:59:54.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:53 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:53.920+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:02.206362+0000 front 2024-09-19T08:59:02.206384+0000 (oldest deadline 2024-09-19T08:59:26.906456+0000) 2024-09-19T08:59:54.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:53 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:53.920+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.208595+0000 front 2024-09-19T08:59:13.208581+0000 (oldest deadline 2024-09-19T08:59:37.308391+0000) 2024-09-19T08:59:54.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:53 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:53.920+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:17.809227+0000 front 2024-09-19T08:59:17.809265+0000 (oldest deadline 2024-09-19T08:59:43.709200+0000) 2024-09-19T08:59:54.282 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:53 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9[64848]: 2024-09-19T08:59:53.920+0000 7f42839ce640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.164:6822 osd.6 since back 2024-09-19T08:59:29.410770+0000 front 2024-09-19T08:59:29.410783+0000 (oldest deadline 2024-09-19T08:59:53.510931+0000) 2024-09-19T08:59:54.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:54 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:54.366+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:54.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:54 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:54.366+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:54.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:54 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:54.366+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:54.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:54 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:54.366+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:05.222908+0000 front 2024-09-19T08:59:05.222955+0000 (oldest deadline 2024-09-19T08:59:31.122786+0000) 2024-09-19T08:59:54.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:54 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:54.366+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:11.123265+0000 front 2024-09-19T08:59:11.123211+0000 (oldest deadline 2024-09-19T08:59:35.223367+0000) 2024-09-19T08:59:54.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:54 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:54.366+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:18.625037+0000 front 2024-09-19T08:59:18.624943+0000 (oldest deadline 2024-09-19T08:59:43.925164+0000) 2024-09-19T08:59:54.625 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:54 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:54.366+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6822 osd.6 since back 2024-09-19T08:59:27.426197+0000 front 2024-09-19T08:59:27.426168+0000 (oldest deadline 2024-09-19T08:59:50.826300+0000) 2024-09-19T08:59:54.998 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:54 smithi164 podman[114597]: 2024-09-19 08:59:54.627866357 +0000 UTC m=+5.538319706 container died 787e37fd935b2b4a07c98f7f28ebf6b4a125d5e608fde2f9056c87ea8933e731 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-19T08:59:54.998 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:54 smithi164 podman[114597]: 2024-09-19 08:59:54.998859636 +0000 UTC m=+5.909312974 container cleanup 787e37fd935b2b4a07c98f7f28ebf6b4a125d5e608fde2f9056c87ea8933e731 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T08:59:55.160 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:54 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:54.786+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:55.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:54 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:54.786+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:55.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:54 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:54.786+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:55.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:54 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:54.786+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.637103+0000 front 2024-09-19T08:59:13.637213+0000 (oldest deadline 2024-09-19T08:59:35.937098+0000) 2024-09-19T08:59:55.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:54 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:54.786+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:21.138496+0000 front 2024-09-19T08:59:21.138554+0000 (oldest deadline 2024-09-19T08:59:43.438472+0000) 2024-09-19T08:59:55.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:54 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:54.786+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6822 osd.6 since back 2024-09-19T08:59:29.839897+0000 front 2024-09-19T08:59:29.839868+0000 (oldest deadline 2024-09-19T08:59:51.539926+0000) 2024-09-19T08:59:55.258 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:54 smithi164 bash[114597]: ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9 2024-09-19T08:59:55.532 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:55 smithi164 podman[114613]: 2024-09-19 08:59:55.258750913 +0000 UTC m=+0.631115017 container remove 787e37fd935b2b4a07c98f7f28ebf6b4a125d5e608fde2f9056c87ea8933e731 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-19T08:59:55.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:55 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:55.387+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:55.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:55 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:55.387+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:55.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:55 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:55.387+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:55.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:55 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:55.387+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:05.222908+0000 front 2024-09-19T08:59:05.222955+0000 (oldest deadline 2024-09-19T08:59:31.122786+0000) 2024-09-19T08:59:55.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:55 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:55.387+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:11.123265+0000 front 2024-09-19T08:59:11.123211+0000 (oldest deadline 2024-09-19T08:59:35.223367+0000) 2024-09-19T08:59:55.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:55 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:55.387+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:18.625037+0000 front 2024-09-19T08:59:18.624943+0000 (oldest deadline 2024-09-19T08:59:43.925164+0000) 2024-09-19T08:59:55.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:55 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:55.387+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6822 osd.6 since back 2024-09-19T08:59:27.426197+0000 front 2024-09-19T08:59:27.426168+0000 (oldest deadline 2024-09-19T08:59:50.826300+0000) 2024-09-19T08:59:55.903 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:55 smithi164 podman[114702]: 2024-09-19 08:59:55.513015298 +0000 UTC m=+0.015841519 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T08:59:55.904 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:55 smithi164 podman[114702]: 2024-09-19 08:59:55.626509915 +0000 UTC m=+0.129336181 container create 11b7a4c797ee8155733fb6ee36bc6d49634216c6a9a74f1362b8232d91d86972 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T08:59:55.904 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:55 smithi164 podman[114702]: 2024-09-19 08:59:55.837237228 +0000 UTC m=+0.340063449 container init 11b7a4c797ee8155733fb6ee36bc6d49634216c6a9a74f1362b8232d91d86972 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4) 2024-09-19T08:59:55.904 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:55 smithi164 podman[114702]: 2024-09-19 08:59:55.840606347 +0000 UTC m=+0.343432571 container start 11b7a4c797ee8155733fb6ee36bc6d49634216c6a9a74f1362b8232d91d86972 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-19T08:59:55.904 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:55 smithi164 podman[114702]: 2024-09-19 08:59:55.903991443 +0000 UTC m=+0.406817667 container attach 11b7a4c797ee8155733fb6ee36bc6d49634216c6a9a74f1362b8232d91d86972 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2) 2024-09-19T08:59:56.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:55 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:55.821+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:56.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:55 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:55.821+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:56.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:55 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:55.821+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:56.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:55 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:55.821+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.637103+0000 front 2024-09-19T08:59:13.637213+0000 (oldest deadline 2024-09-19T08:59:35.937098+0000) 2024-09-19T08:59:56.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:55 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:55.821+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:21.138496+0000 front 2024-09-19T08:59:21.138554+0000 (oldest deadline 2024-09-19T08:59:43.438472+0000) 2024-09-19T08:59:56.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:55 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:55.821+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6822 osd.6 since back 2024-09-19T08:59:29.839897+0000 front 2024-09-19T08:59:29.839868+0000 (oldest deadline 2024-09-19T08:59:51.539926+0000) 2024-09-19T08:59:56.281 INFO:journalctl@ceph.osd.9.smithi164.stdout:Sep 19 08:59:56 smithi164 podman[114702]: 2024-09-19 08:59:56.030575002 +0000 UTC m=+0.533401223 container died 11b7a4c797ee8155733fb6ee36bc6d49634216c6a9a74f1362b8232d91d86972 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-19T08:59:56.637 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.9.service' 2024-09-19T08:59:56.678 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T08:59:56.679 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-19T08:59:56.679 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-19T08:59:56.679 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.10 2024-09-19T08:59:56.706 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:56 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:56.433+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:56.707 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:56 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:56.433+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:56.707 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:56 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:56.433+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:56.707 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:56 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:56.433+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:05.222908+0000 front 2024-09-19T08:59:05.222955+0000 (oldest deadline 2024-09-19T08:59:31.122786+0000) 2024-09-19T08:59:56.707 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:56 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:56.433+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:11.123265+0000 front 2024-09-19T08:59:11.123211+0000 (oldest deadline 2024-09-19T08:59:35.223367+0000) 2024-09-19T08:59:56.707 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:56 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:56.433+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:18.625037+0000 front 2024-09-19T08:59:18.624943+0000 (oldest deadline 2024-09-19T08:59:43.925164+0000) 2024-09-19T08:59:56.707 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:56 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:56.433+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6822 osd.6 since back 2024-09-19T08:59:27.426197+0000 front 2024-09-19T08:59:27.426168+0000 (oldest deadline 2024-09-19T08:59:50.826300+0000) 2024-09-19T08:59:56.997 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:56 smithi170 systemd[1]: Stopping Ceph osd.10 for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T08:59:57.160 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:56 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:56.816+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:57.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:56 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:56.816+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:57.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:56 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:56.816+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:57.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:56 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:56.816+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.637103+0000 front 2024-09-19T08:59:13.637213+0000 (oldest deadline 2024-09-19T08:59:35.937098+0000) 2024-09-19T08:59:57.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:56 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:56.816+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:21.138496+0000 front 2024-09-19T08:59:21.138554+0000 (oldest deadline 2024-09-19T08:59:43.438472+0000) 2024-09-19T08:59:57.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:56 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:56.816+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6822 osd.6 since back 2024-09-19T08:59:29.839897+0000 front 2024-09-19T08:59:29.839868+0000 (oldest deadline 2024-09-19T08:59:51.539926+0000) 2024-09-19T08:59:57.374 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:56 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:56.997+0000 7f174fa22640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T08:59:57.375 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:56 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:56.997+0000 7f174fa22640 -1 osd.10 100 *** Got signal Terminated *** 2024-09-19T08:59:57.375 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:56 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:56.997+0000 7f174fa22640 -1 osd.10 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T08:59:57.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:57 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:57.386+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:57.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:57 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:57.386+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:57.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:57 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:57.386+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:57.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:57 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:57.386+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:05.222908+0000 front 2024-09-19T08:59:05.222955+0000 (oldest deadline 2024-09-19T08:59:31.122786+0000) 2024-09-19T08:59:57.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:57 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:57.386+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:11.123265+0000 front 2024-09-19T08:59:11.123211+0000 (oldest deadline 2024-09-19T08:59:35.223367+0000) 2024-09-19T08:59:57.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:57 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:57.386+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:18.625037+0000 front 2024-09-19T08:59:18.624943+0000 (oldest deadline 2024-09-19T08:59:43.925164+0000) 2024-09-19T08:59:57.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:57 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:57.386+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6822 osd.6 since back 2024-09-19T08:59:27.426197+0000 front 2024-09-19T08:59:27.426168+0000 (oldest deadline 2024-09-19T08:59:50.826300+0000) 2024-09-19T08:59:58.160 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:57 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:57.798+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:58.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:57 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:57.798+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:58.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:57 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:57.798+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:58.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:57 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:57.798+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.637103+0000 front 2024-09-19T08:59:13.637213+0000 (oldest deadline 2024-09-19T08:59:35.937098+0000) 2024-09-19T08:59:58.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:57 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:57.798+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:21.138496+0000 front 2024-09-19T08:59:21.138554+0000 (oldest deadline 2024-09-19T08:59:43.438472+0000) 2024-09-19T08:59:58.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:57 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:57.798+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6822 osd.6 since back 2024-09-19T08:59:29.839897+0000 front 2024-09-19T08:59:29.839868+0000 (oldest deadline 2024-09-19T08:59:51.539926+0000) 2024-09-19T08:59:58.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:58 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:58.393+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:58.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:58 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:58.393+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:58.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:58 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:58.393+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:58.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:58 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:58.393+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:05.222908+0000 front 2024-09-19T08:59:05.222955+0000 (oldest deadline 2024-09-19T08:59:31.122786+0000) 2024-09-19T08:59:58.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:58 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:58.393+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:11.123265+0000 front 2024-09-19T08:59:11.123211+0000 (oldest deadline 2024-09-19T08:59:35.223367+0000) 2024-09-19T08:59:58.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:58 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:58.393+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:18.625037+0000 front 2024-09-19T08:59:18.624943+0000 (oldest deadline 2024-09-19T08:59:43.925164+0000) 2024-09-19T08:59:58.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:58 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:58.393+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6822 osd.6 since back 2024-09-19T08:59:27.426197+0000 front 2024-09-19T08:59:27.426168+0000 (oldest deadline 2024-09-19T08:59:50.826300+0000) 2024-09-19T08:59:59.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:58 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:58.752+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T08:59:59.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:58 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:58.752+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T08:59:59.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:58 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:58.752+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T08:59:59.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:58 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:58.752+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.637103+0000 front 2024-09-19T08:59:13.637213+0000 (oldest deadline 2024-09-19T08:59:35.937098+0000) 2024-09-19T08:59:59.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:58 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:58.752+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:21.138496+0000 front 2024-09-19T08:59:21.138554+0000 (oldest deadline 2024-09-19T08:59:43.438472+0000) 2024-09-19T08:59:59.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:58 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:58.752+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6822 osd.6 since back 2024-09-19T08:59:29.839897+0000 front 2024-09-19T08:59:29.839868+0000 (oldest deadline 2024-09-19T08:59:51.539926+0000) 2024-09-19T08:59:59.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:59 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:59.397+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T08:59:59.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:59 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:59.397+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T08:59:59.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:59 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:59.397+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T08:59:59.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:59 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:59.397+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:05.222908+0000 front 2024-09-19T08:59:05.222955+0000 (oldest deadline 2024-09-19T08:59:31.122786+0000) 2024-09-19T08:59:59.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:59 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:59.397+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:11.123265+0000 front 2024-09-19T08:59:11.123211+0000 (oldest deadline 2024-09-19T08:59:35.223367+0000) 2024-09-19T08:59:59.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:59 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:59.397+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:18.625037+0000 front 2024-09-19T08:59:18.624943+0000 (oldest deadline 2024-09-19T08:59:43.925164+0000) 2024-09-19T08:59:59.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 08:59:59 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T08:59:59.397+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6822 osd.6 since back 2024-09-19T08:59:27.426197+0000 front 2024-09-19T08:59:27.426168+0000 (oldest deadline 2024-09-19T08:59:50.826300+0000) 2024-09-19T09:00:00.160 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:59 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:59.736+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T09:00:00.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:59 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:59.736+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T09:00:00.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:59 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:59.736+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T09:00:00.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:59 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:59.736+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.637103+0000 front 2024-09-19T08:59:13.637213+0000 (oldest deadline 2024-09-19T08:59:35.937098+0000) 2024-09-19T09:00:00.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:59 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:59.736+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:21.138496+0000 front 2024-09-19T08:59:21.138554+0000 (oldest deadline 2024-09-19T08:59:43.438472+0000) 2024-09-19T09:00:00.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 08:59:59 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T08:59:59.736+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6822 osd.6 since back 2024-09-19T08:59:29.839897+0000 front 2024-09-19T08:59:29.839868+0000 (oldest deadline 2024-09-19T08:59:51.539926+0000) 2024-09-19T09:00:00.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 09:00:00 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T09:00:00.396+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T09:00:00.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 09:00:00 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T09:00:00.396+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T09:00:00.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 09:00:00 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T09:00:00.396+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T09:00:00.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 09:00:00 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T09:00:00.396+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:05.222908+0000 front 2024-09-19T08:59:05.222955+0000 (oldest deadline 2024-09-19T08:59:31.122786+0000) 2024-09-19T09:00:00.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 09:00:00 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T09:00:00.396+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:11.123265+0000 front 2024-09-19T08:59:11.123211+0000 (oldest deadline 2024-09-19T08:59:35.223367+0000) 2024-09-19T09:00:00.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 09:00:00 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T09:00:00.396+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:18.625037+0000 front 2024-09-19T08:59:18.624943+0000 (oldest deadline 2024-09-19T08:59:43.925164+0000) 2024-09-19T09:00:00.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 09:00:00 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T09:00:00.396+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6822 osd.6 since back 2024-09-19T08:59:27.426197+0000 front 2024-09-19T08:59:27.426168+0000 (oldest deadline 2024-09-19T08:59:50.826300+0000) 2024-09-19T09:00:01.160 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:00 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:00.775+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T09:00:01.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:00 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:00.775+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T09:00:01.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:00 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:00.775+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T09:00:01.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:00 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:00.775+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.637103+0000 front 2024-09-19T08:59:13.637213+0000 (oldest deadline 2024-09-19T08:59:35.937098+0000) 2024-09-19T09:00:01.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:00 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:00.775+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:21.138496+0000 front 2024-09-19T08:59:21.138554+0000 (oldest deadline 2024-09-19T08:59:43.438472+0000) 2024-09-19T09:00:01.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:00 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:00.775+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6822 osd.6 since back 2024-09-19T08:59:29.839897+0000 front 2024-09-19T08:59:29.839868+0000 (oldest deadline 2024-09-19T08:59:51.539926+0000) 2024-09-19T09:00:01.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 09:00:01 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T09:00:01.443+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.917972+0000 front 2024-09-19T08:58:40.918188+0000 (oldest deadline 2024-09-19T08:59:03.217983+0000) 2024-09-19T09:00:01.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 09:00:01 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T09:00:01.443+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:49.019722+0000 front 2024-09-19T08:58:49.019748+0000 (oldest deadline 2024-09-19T08:59:11.919963+0000) 2024-09-19T09:00:01.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 09:00:01 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T09:00:01.443+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.121550+0000 front 2024-09-19T08:58:57.121764+0000 (oldest deadline 2024-09-19T08:59:19.421392+0000) 2024-09-19T09:00:01.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 09:00:01 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T09:00:01.443+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6814 osd.3 since back 2024-09-19T08:59:05.222908+0000 front 2024-09-19T08:59:05.222955+0000 (oldest deadline 2024-09-19T08:59:31.122786+0000) 2024-09-19T09:00:01.875 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 09:00:01 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T09:00:01.443+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:11.123265+0000 front 2024-09-19T08:59:11.123211+0000 (oldest deadline 2024-09-19T08:59:35.223367+0000) 2024-09-19T09:00:01.876 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 09:00:01 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T09:00:01.443+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:18.625037+0000 front 2024-09-19T08:59:18.624943+0000 (oldest deadline 2024-09-19T08:59:43.925164+0000) 2024-09-19T09:00:01.876 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 09:00:01 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10[59700]: 2024-09-19T09:00:01.443+0000 7f1748030640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.164:6822 osd.6 since back 2024-09-19T08:59:27.426197+0000 front 2024-09-19T08:59:27.426168+0000 (oldest deadline 2024-09-19T08:59:50.826300+0000) 2024-09-19T09:00:02.160 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:01 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:01.800+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T09:00:02.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:01 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:01.800+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T09:00:02.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:01 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:01.800+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T09:00:02.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:01 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:01.800+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.637103+0000 front 2024-09-19T08:59:13.637213+0000 (oldest deadline 2024-09-19T08:59:35.937098+0000) 2024-09-19T09:00:02.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:01 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:01.800+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:21.138496+0000 front 2024-09-19T08:59:21.138554+0000 (oldest deadline 2024-09-19T08:59:43.438472+0000) 2024-09-19T09:00:02.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:01 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:01.800+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6822 osd.6 since back 2024-09-19T08:59:29.839897+0000 front 2024-09-19T08:59:29.839868+0000 (oldest deadline 2024-09-19T08:59:51.539926+0000) 2024-09-19T09:00:02.739 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 09:00:02 smithi170 podman[71778]: 2024-09-19 09:00:02.461606172 +0000 UTC m=+5.584526773 container died b960d27b5343d32f71125f8b7a49be6b96b6e75b9261a67c1f582ac187da710f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-09-19T09:00:03.080 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 09:00:02 smithi170 podman[71778]: 2024-09-19 09:00:02.847156935 +0000 UTC m=+5.970077537 container cleanup b960d27b5343d32f71125f8b7a49be6b96b6e75b9261a67c1f582ac187da710f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:00:03.081 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 09:00:02 smithi170 bash[71778]: ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10 2024-09-19T09:00:03.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:02 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:02.757+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T09:00:03.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:02 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:02.757+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T09:00:03.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:02 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:02.757+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T09:00:03.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:02 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:02.757+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.637103+0000 front 2024-09-19T08:59:13.637213+0000 (oldest deadline 2024-09-19T08:59:35.937098+0000) 2024-09-19T09:00:03.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:02 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:02.757+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:21.138496+0000 front 2024-09-19T08:59:21.138554+0000 (oldest deadline 2024-09-19T08:59:43.438472+0000) 2024-09-19T09:00:03.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:02 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:02.757+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6822 osd.6 since back 2024-09-19T08:59:29.839897+0000 front 2024-09-19T08:59:29.839868+0000 (oldest deadline 2024-09-19T08:59:51.539926+0000) 2024-09-19T09:00:03.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:02 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:02.757+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6820 osd.7 since back 2024-09-19T08:59:34.941505+0000 front 2024-09-19T08:59:34.941583+0000 (oldest deadline 2024-09-19T09:00:01.941853+0000) 2024-09-19T09:00:03.374 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 09:00:03 smithi170 podman[71795]: 2024-09-19 09:00:03.08170631 +0000 UTC m=+0.614192215 container remove b960d27b5343d32f71125f8b7a49be6b96b6e75b9261a67c1f582ac187da710f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:00:03.700 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 09:00:03 smithi170 podman[71886]: 2024-09-19 09:00:03.399008977 +0000 UTC m=+0.102305690 container create 754d4f46c7fc8e655f32994a23196a40bae0034400d29190d2a67335aa7a43b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-19T09:00:03.700 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 09:00:03 smithi170 podman[71886]: 2024-09-19 09:00:03.312354235 +0000 UTC m=+0.015650940 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:00:03.700 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 09:00:03 smithi170 podman[71886]: 2024-09-19 09:00:03.626385949 +0000 UTC m=+0.329682654 container init 754d4f46c7fc8e655f32994a23196a40bae0034400d29190d2a67335aa7a43b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=) 2024-09-19T09:00:03.701 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 09:00:03 smithi170 podman[71886]: 2024-09-19 09:00:03.629714596 +0000 UTC m=+0.333011302 container start 754d4f46c7fc8e655f32994a23196a40bae0034400d29190d2a67335aa7a43b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:00:03.701 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 09:00:03 smithi170 podman[71886]: 2024-09-19 09:00:03.701109268 +0000 UTC m=+0.404405982 container attach 754d4f46c7fc8e655f32994a23196a40bae0034400d29190d2a67335aa7a43b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-19T09:00:04.112 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 09:00:03 smithi170 podman[71886]: 2024-09-19 09:00:03.819649637 +0000 UTC m=+0.522946345 container died 754d4f46c7fc8e655f32994a23196a40bae0034400d29190d2a67335aa7a43b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-19T09:00:04.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:03 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:03.763+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T09:00:04.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:03 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:03.763+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T09:00:04.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:03 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:03.763+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T09:00:04.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:03 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:03.763+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.637103+0000 front 2024-09-19T08:59:13.637213+0000 (oldest deadline 2024-09-19T08:59:35.937098+0000) 2024-09-19T09:00:04.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:03 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:03.763+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:21.138496+0000 front 2024-09-19T08:59:21.138554+0000 (oldest deadline 2024-09-19T08:59:43.438472+0000) 2024-09-19T09:00:04.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:03 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:03.763+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6822 osd.6 since back 2024-09-19T08:59:29.839897+0000 front 2024-09-19T08:59:29.839868+0000 (oldest deadline 2024-09-19T08:59:51.539926+0000) 2024-09-19T09:00:04.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:03 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:03.763+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6820 osd.7 since back 2024-09-19T08:59:34.941505+0000 front 2024-09-19T08:59:34.941583+0000 (oldest deadline 2024-09-19T09:00:01.941853+0000) 2024-09-19T09:00:04.375 INFO:journalctl@ceph.osd.10.smithi170.stdout:Sep 19 09:00:04 smithi170 podman[71909]: 2024-09-19 09:00:04.354473731 +0000 UTC m=+0.524639835 container remove 754d4f46c7fc8e655f32994a23196a40bae0034400d29190d2a67335aa7a43b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-19T09:00:04.383 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.10.service' 2024-09-19T09:00:04.463 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:00:04.463 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-19T09:00:04.463 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-19T09:00:04.463 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.11 2024-09-19T09:00:04.779 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:04 smithi173 systemd[1]: Stopping Ceph osd.11 for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T09:00:05.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:04 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:04.778+0000 7fcce7ac1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:00:05.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:04 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:04.778+0000 7fcce7ac1640 -1 osd.11 100 *** Got signal Terminated *** 2024-09-19T09:00:05.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:04 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:04.778+0000 7fcce7ac1640 -1 osd.11 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:00:05.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:04 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:04.781+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T09:00:05.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:04 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:04.781+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T09:00:05.162 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:04 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:04.781+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T09:00:05.162 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:04 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:04.781+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.637103+0000 front 2024-09-19T08:59:13.637213+0000 (oldest deadline 2024-09-19T08:59:35.937098+0000) 2024-09-19T09:00:05.162 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:04 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:04.781+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:21.138496+0000 front 2024-09-19T08:59:21.138554+0000 (oldest deadline 2024-09-19T08:59:43.438472+0000) 2024-09-19T09:00:05.162 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:04 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:04.781+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6822 osd.6 since back 2024-09-19T08:59:29.839897+0000 front 2024-09-19T08:59:29.839868+0000 (oldest deadline 2024-09-19T08:59:51.539926+0000) 2024-09-19T09:00:05.162 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:04 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:04.781+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6820 osd.7 since back 2024-09-19T08:59:34.941505+0000 front 2024-09-19T08:59:34.941583+0000 (oldest deadline 2024-09-19T09:00:01.941853+0000) 2024-09-19T09:00:06.160 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:05 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:05.816+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T09:00:06.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:05 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:05.816+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T09:00:06.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:05 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:05.816+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T09:00:06.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:05 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:05.816+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.637103+0000 front 2024-09-19T08:59:13.637213+0000 (oldest deadline 2024-09-19T08:59:35.937098+0000) 2024-09-19T09:00:06.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:05 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:05.816+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:21.138496+0000 front 2024-09-19T08:59:21.138554+0000 (oldest deadline 2024-09-19T08:59:43.438472+0000) 2024-09-19T09:00:06.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:05 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:05.816+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6822 osd.6 since back 2024-09-19T08:59:29.839897+0000 front 2024-09-19T08:59:29.839868+0000 (oldest deadline 2024-09-19T08:59:51.539926+0000) 2024-09-19T09:00:06.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:05 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:05.816+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6820 osd.7 since back 2024-09-19T08:59:34.941505+0000 front 2024-09-19T08:59:34.941583+0000 (oldest deadline 2024-09-19T09:00:01.941853+0000) 2024-09-19T09:00:07.160 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:06 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:06.838+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T09:00:07.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:06 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:06.838+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T09:00:07.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:06 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:06.838+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T09:00:07.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:06 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:06.838+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.637103+0000 front 2024-09-19T08:59:13.637213+0000 (oldest deadline 2024-09-19T08:59:35.937098+0000) 2024-09-19T09:00:07.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:06 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:06.838+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:21.138496+0000 front 2024-09-19T08:59:21.138554+0000 (oldest deadline 2024-09-19T08:59:43.438472+0000) 2024-09-19T09:00:07.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:06 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:06.838+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6822 osd.6 since back 2024-09-19T08:59:29.839897+0000 front 2024-09-19T08:59:29.839868+0000 (oldest deadline 2024-09-19T08:59:51.539926+0000) 2024-09-19T09:00:07.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:06 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:06.838+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6820 osd.7 since back 2024-09-19T08:59:34.941505+0000 front 2024-09-19T08:59:34.941583+0000 (oldest deadline 2024-09-19T09:00:01.941853+0000) 2024-09-19T09:00:08.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:07 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:07.807+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T09:00:08.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:07 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:07.807+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T09:00:08.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:07 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:07.807+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T09:00:08.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:07 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:07.807+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.637103+0000 front 2024-09-19T08:59:13.637213+0000 (oldest deadline 2024-09-19T08:59:35.937098+0000) 2024-09-19T09:00:08.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:07 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:07.807+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:21.138496+0000 front 2024-09-19T08:59:21.138554+0000 (oldest deadline 2024-09-19T08:59:43.438472+0000) 2024-09-19T09:00:08.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:07 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:07.807+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6822 osd.6 since back 2024-09-19T08:59:29.839897+0000 front 2024-09-19T08:59:29.839868+0000 (oldest deadline 2024-09-19T08:59:51.539926+0000) 2024-09-19T09:00:08.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:07 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:07.807+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6820 osd.7 since back 2024-09-19T08:59:34.941505+0000 front 2024-09-19T08:59:34.941583+0000 (oldest deadline 2024-09-19T09:00:01.941853+0000) 2024-09-19T09:00:09.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:08 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:08.849+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6806 osd.0 since back 2024-09-19T08:58:40.531873+0000 front 2024-09-19T08:58:40.531886+0000 (oldest deadline 2024-09-19T08:59:04.631697+0000) 2024-09-19T09:00:09.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:08 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:08.849+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-09-19T08:58:46.734043+0000 front 2024-09-19T08:58:46.734261+0000 (oldest deadline 2024-09-19T08:59:12.534489+0000) 2024-09-19T09:00:09.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:08 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:08.849+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-09-19T08:58:57.235162+0000 front 2024-09-19T08:58:57.235201+0000 (oldest deadline 2024-09-19T08:59:19.535238+0000) 2024-09-19T09:00:09.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:08 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:08.849+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-09-19T08:59:13.637103+0000 front 2024-09-19T08:59:13.637213+0000 (oldest deadline 2024-09-19T08:59:35.937098+0000) 2024-09-19T09:00:09.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:08 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:08.849+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-09-19T08:59:21.138496+0000 front 2024-09-19T08:59:21.138554+0000 (oldest deadline 2024-09-19T08:59:43.438472+0000) 2024-09-19T09:00:09.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:08 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:08.849+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.164:6822 osd.6 since back 2024-09-19T08:59:29.839897+0000 front 2024-09-19T08:59:29.839868+0000 (oldest deadline 2024-09-19T08:59:51.539926+0000) 2024-09-19T09:00:09.161 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:08 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11[60232]: 2024-09-19T09:00:08.849+0000 7fcce00cf640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6820 osd.7 since back 2024-09-19T08:59:34.941505+0000 front 2024-09-19T08:59:34.941583+0000 (oldest deadline 2024-09-19T09:00:01.941853+0000) 2024-09-19T09:00:10.463 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:10 smithi173 podman[72187]: 2024-09-19 09:00:10.192222196 +0000 UTC m=+5.525021244 container died 7b0299d2d0bdcfdd9d065b0abeae9cd6f9844f0d233fce96e6c30377a4be240e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD) 2024-09-19T09:00:10.815 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:10 smithi173 podman[72187]: 2024-09-19 09:00:10.56508084 +0000 UTC m=+5.897879880 container cleanup 7b0299d2d0bdcfdd9d065b0abeae9cd6f9844f0d233fce96e6c30377a4be240e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:00:10.815 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:10 smithi173 bash[72187]: ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11 2024-09-19T09:00:11.160 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:10 smithi173 podman[72203]: 2024-09-19 09:00:10.816308112 +0000 UTC m=+0.619164336 container remove 7b0299d2d0bdcfdd9d065b0abeae9cd6f9844f0d233fce96e6c30377a4be240e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:00:11.468 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:11 smithi173 podman[72289]: 2024-09-19 09:00:11.167036046 +0000 UTC m=+0.108962935 container create 7722e9c531f006c2e99c1e99aec2c00b2de21a871d065229e037bdb666212dd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:00:11.468 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:11 smithi173 podman[72289]: 2024-09-19 09:00:11.073696065 +0000 UTC m=+0.015622923 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:00:11.468 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:11 smithi173 podman[72289]: 2024-09-19 09:00:11.386174249 +0000 UTC m=+0.328101114 container init 7722e9c531f006c2e99c1e99aec2c00b2de21a871d065229e037bdb666212dd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD) 2024-09-19T09:00:11.469 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:11 smithi173 podman[72289]: 2024-09-19 09:00:11.38955957 +0000 UTC m=+0.331486420 container start 7722e9c531f006c2e99c1e99aec2c00b2de21a871d065229e037bdb666212dd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11-deactivate, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-19T09:00:11.469 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:11 smithi173 podman[72289]: 2024-09-19 09:00:11.469159249 +0000 UTC m=+0.411086098 container attach 7722e9c531f006c2e99c1e99aec2c00b2de21a871d065229e037bdb666212dd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=squid-2f599a4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-19T09:00:11.863 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:11 smithi173 podman[72289]: 2024-09-19 09:00:11.580398005 +0000 UTC m=+0.522324873 container died 7722e9c531f006c2e99c1e99aec2c00b2de21a871d065229e037bdb666212dd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, io.buildah.version=1.37.2) 2024-09-19T09:00:12.122 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 19 09:00:12 smithi173 podman[72313]: 2024-09-19 09:00:12.105763274 +0000 UTC m=+0.518718947 container remove 7722e9c531f006c2e99c1e99aec2c00b2de21a871d065229e037bdb666212dd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=) 2024-09-19T09:00:12.128 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@osd.11.service' 2024-09-19T09:00:12.167 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:00:12.167 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-19T09:00:12.167 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-19T09:00:12.168 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.a 2024-09-19T09:00:12.451 INFO:journalctl@ceph.mds.a.smithi164.stdout:Sep 19 09:00:12 smithi164 systemd[1]: Stopping Ceph mds.a for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T09:00:12.782 INFO:journalctl@ceph.mds.a.smithi164.stdout:Sep 19 09:00:12 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-a[71584]: 2024-09-19T09:00:12.450+0000 7f9ef91aa640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:00:12.782 INFO:journalctl@ceph.mds.a.smithi164.stdout:Sep 19 09:00:12 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-a[71584]: 2024-09-19T09:00:12.450+0000 7f9ef91aa640 -1 mds.a *** got signal Terminated *** 2024-09-19T09:00:16.009 INFO:journalctl@ceph.mds.a.smithi164.stdout:Sep 19 09:00:15 smithi164 podman[114813]: 2024-09-19 09:00:15.737878529 +0000 UTC m=+3.379321458 container died 29a373f4a0e40c7e1175ddeaf4ca590d6b0a554a44ecc0c1bfdff9ee2e73d1ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:00:16.282 INFO:journalctl@ceph.mds.a.smithi164.stdout:Sep 19 09:00:16 smithi164 podman[114813]: 2024-09-19 09:00:16.100717627 +0000 UTC m=+3.742160552 container cleanup 29a373f4a0e40c7e1175ddeaf4ca590d6b0a554a44ecc0c1bfdff9ee2e73d1ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-a, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:00:16.282 INFO:journalctl@ceph.mds.a.smithi164.stdout:Sep 19 09:00:16 smithi164 bash[114813]: ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-a 2024-09-19T09:00:16.532 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.a.service' 2024-09-19T09:00:16.566 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:00:16.567 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-19T09:00:16.567 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-19T09:00:16.567 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.d 2024-09-19T09:00:16.879 INFO:journalctl@ceph.mds.d.smithi164.stdout:Sep 19 09:00:16 smithi164 systemd[1]: Stopping Ceph mds.d for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T09:00:17.281 INFO:journalctl@ceph.mds.d.smithi164.stdout:Sep 19 09:00:16 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-d[70399]: 2024-09-19T09:00:16.879+0000 7faef5fcc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:00:17.282 INFO:journalctl@ceph.mds.d.smithi164.stdout:Sep 19 09:00:16 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-d[70399]: 2024-09-19T09:00:16.879+0000 7faef5fcc640 -1 mds.d *** got signal Terminated *** 2024-09-19T09:00:20.912 INFO:journalctl@ceph.mds.d.smithi164.stdout:Sep 19 09:00:20 smithi164 podman[114933]: 2024-09-19 09:00:20.659726415 +0000 UTC m=+3.877902362 container died fcf4f129a2b0e002b4b5d2f7937927378f062eab7770de20e71273f1944e86dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-d, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4) 2024-09-19T09:00:21.282 INFO:journalctl@ceph.mds.d.smithi164.stdout:Sep 19 09:00:21 smithi164 podman[114933]: 2024-09-19 09:00:21.062630728 +0000 UTC m=+4.280806718 container cleanup fcf4f129a2b0e002b4b5d2f7937927378f062eab7770de20e71273f1944e86dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-d, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:00:21.282 INFO:journalctl@ceph.mds.d.smithi164.stdout:Sep 19 09:00:21 smithi164 bash[114933]: ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-d 2024-09-19T09:00:21.542 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.d.service' 2024-09-19T09:00:21.576 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:00:21.576 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-19T09:00:21.576 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-19T09:00:21.576 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.g 2024-09-19T09:00:21.899 INFO:journalctl@ceph.mds.g.smithi164.stdout:Sep 19 09:00:21 smithi164 systemd[1]: Stopping Ceph mds.g for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T09:00:22.282 INFO:journalctl@ceph.mds.g.smithi164.stdout:Sep 19 09:00:21 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-g[71206]: 2024-09-19T09:00:21.900+0000 7f8c41181640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:00:22.282 INFO:journalctl@ceph.mds.g.smithi164.stdout:Sep 19 09:00:21 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-g[71206]: 2024-09-19T09:00:21.900+0000 7f8c41181640 -1 mds.g *** got signal Terminated *** 2024-09-19T09:00:24.032 INFO:journalctl@ceph.mds.g.smithi164.stdout:Sep 19 09:00:23 smithi164 podman[115056]: 2024-09-19 09:00:23.743990498 +0000 UTC m=+1.943622731 container died 085df45388ee07c82fa8ec731cc4304013419c0403a75d9dfdf6150ecb6b3c0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-g, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0) 2024-09-19T09:00:24.442 INFO:journalctl@ceph.mds.g.smithi164.stdout:Sep 19 09:00:24 smithi164 podman[115056]: 2024-09-19 09:00:24.225305721 +0000 UTC m=+2.424938000 container cleanup 085df45388ee07c82fa8ec731cc4304013419c0403a75d9dfdf6150ecb6b3c0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-g, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:00:24.442 INFO:journalctl@ceph.mds.g.smithi164.stdout:Sep 19 09:00:24 smithi164 bash[115056]: ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-g 2024-09-19T09:00:24.574 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.g.service' 2024-09-19T09:00:24.606 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:00:24.607 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-19T09:00:24.607 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-19T09:00:24.607 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.j 2024-09-19T09:00:25.032 INFO:journalctl@ceph.mds.j.smithi164.stdout:Sep 19 09:00:24 smithi164 systemd[1]: Stopping Ceph mds.j for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T09:00:25.032 INFO:journalctl@ceph.mds.j.smithi164.stdout:Sep 19 09:00:24 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-j[70001]: 2024-09-19T09:00:24.937+0000 7f2c6d90b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:00:25.032 INFO:journalctl@ceph.mds.j.smithi164.stdout:Sep 19 09:00:24 smithi164 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-j[70001]: 2024-09-19T09:00:24.937+0000 7f2c6d90b640 -1 mds.j *** got signal Terminated *** 2024-09-19T09:00:29.237 INFO:journalctl@ceph.mds.j.smithi164.stdout:Sep 19 09:00:28 smithi164 podman[115179]: 2024-09-19 09:00:28.951330843 +0000 UTC m=+4.118281359 container died d5aaa7f97ba043430262cc7e49e68603850fcf0da05b7fdf2240ab198d5a8e72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-j, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-19T09:00:29.532 INFO:journalctl@ceph.mds.j.smithi164.stdout:Sep 19 09:00:29 smithi164 podman[115179]: 2024-09-19 09:00:29.412379079 +0000 UTC m=+4.579329595 container cleanup d5aaa7f97ba043430262cc7e49e68603850fcf0da05b7fdf2240ab198d5a8e72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-j, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-19T09:00:29.532 INFO:journalctl@ceph.mds.j.smithi164.stdout:Sep 19 09:00:29 smithi164 bash[115179]: ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-j 2024-09-19T09:00:29.834 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.j.service' 2024-09-19T09:00:29.865 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:00:29.865 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-19T09:00:29.865 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-19T09:00:29.865 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.b 2024-09-19T09:00:30.188 INFO:journalctl@ceph.mds.b.smithi170.stdout:Sep 19 09:00:29 smithi170 systemd[1]: Stopping Ceph mds.b for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T09:00:30.625 INFO:journalctl@ceph.mds.b.smithi170.stdout:Sep 19 09:00:30 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-b[65018]: 2024-09-19T09:00:30.187+0000 7fd12876a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:00:30.625 INFO:journalctl@ceph.mds.b.smithi170.stdout:Sep 19 09:00:30 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-b[65018]: 2024-09-19T09:00:30.187+0000 7fd12876a640 -1 mds.b *** got signal Terminated *** 2024-09-19T09:00:32.021 INFO:journalctl@ceph.mds.b.smithi170.stdout:Sep 19 09:00:31 smithi170 podman[71988]: 2024-09-19 09:00:31.74512401 +0000 UTC m=+1.682331997 container died c4e1796a1740e1592f87b3b525c52b515d8d102befb2965f6c2244687a954fdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:00:32.354 INFO:journalctl@ceph.mds.b.smithi170.stdout:Sep 19 09:00:32 smithi170 podman[71988]: 2024-09-19 09:00:32.120644319 +0000 UTC m=+2.057852299 container cleanup c4e1796a1740e1592f87b3b525c52b515d8d102befb2965f6c2244687a954fdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-b, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-19T09:00:32.354 INFO:journalctl@ceph.mds.b.smithi170.stdout:Sep 19 09:00:32 smithi170 bash[71988]: ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-b 2024-09-19T09:00:32.495 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.b.service' 2024-09-19T09:00:32.530 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:00:32.530 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-19T09:00:32.530 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-19T09:00:32.530 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.e 2024-09-19T09:00:32.866 INFO:journalctl@ceph.mds.e.smithi170.stdout:Sep 19 09:00:32 smithi170 systemd[1]: Stopping Ceph mds.e for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T09:00:33.125 INFO:journalctl@ceph.mds.e.smithi170.stdout:Sep 19 09:00:32 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-e[64416]: 2024-09-19T09:00:32.865+0000 7f7eccf93640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:00:33.125 INFO:journalctl@ceph.mds.e.smithi170.stdout:Sep 19 09:00:32 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-e[64416]: 2024-09-19T09:00:32.865+0000 7f7eccf93640 -1 mds.e *** got signal Terminated *** 2024-09-19T09:00:36.007 INFO:journalctl@ceph.mds.e.smithi170.stdout:Sep 19 09:00:35 smithi170 podman[72110]: 2024-09-19 09:00:35.745096047 +0000 UTC m=+2.999539016 container died 68372a737eac2538bbf161b2e5fdb8de896082ec6b1cbf7512c545fb4ac6559a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-e, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-19T09:00:36.374 INFO:journalctl@ceph.mds.e.smithi170.stdout:Sep 19 09:00:36 smithi170 podman[72110]: 2024-09-19 09:00:36.090656049 +0000 UTC m=+3.345099017 container cleanup 68372a737eac2538bbf161b2e5fdb8de896082ec6b1cbf7512c545fb4ac6559a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-e, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-2f599a4, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-19T09:00:36.375 INFO:journalctl@ceph.mds.e.smithi170.stdout:Sep 19 09:00:36 smithi170 bash[72110]: ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-e 2024-09-19T09:00:36.525 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.e.service' 2024-09-19T09:00:36.558 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:00:36.559 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-19T09:00:36.559 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-19T09:00:36.559 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.h 2024-09-19T09:00:37.125 INFO:journalctl@ceph.mds.h.smithi170.stdout:Sep 19 09:00:36 smithi170 systemd[1]: Stopping Ceph mds.h for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T09:00:37.125 INFO:journalctl@ceph.mds.h.smithi170.stdout:Sep 19 09:00:36 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-h[64715]: 2024-09-19T09:00:36.877+0000 7fe5aab48640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:00:37.125 INFO:journalctl@ceph.mds.h.smithi170.stdout:Sep 19 09:00:36 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-h[64715]: 2024-09-19T09:00:36.878+0000 7fe5aab48640 -1 mds.h *** got signal Terminated *** 2024-09-19T09:00:40.125 INFO:journalctl@ceph.mds.h.smithi170.stdout:Sep 19 09:00:39 smithi170 podman[72228]: 2024-09-19 09:00:39.747381016 +0000 UTC m=+2.964374308 container died e9a4945d27ce27383cb5e0d53608dabcf59f2244a210fe744851953d9a72ab80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-h, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:00:40.625 INFO:journalctl@ceph.mds.h.smithi170.stdout:Sep 19 09:00:40 smithi170 podman[72228]: 2024-09-19 09:00:40.411196688 +0000 UTC m=+3.628189970 container cleanup e9a4945d27ce27383cb5e0d53608dabcf59f2244a210fe744851953d9a72ab80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:00:40.625 INFO:journalctl@ceph.mds.h.smithi170.stdout:Sep 19 09:00:40 smithi170 bash[72228]: ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-h 2024-09-19T09:00:40.847 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.h.service' 2024-09-19T09:00:40.879 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:00:40.879 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-19T09:00:40.879 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-19T09:00:40.879 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.k 2024-09-19T09:00:41.375 INFO:journalctl@ceph.mds.k.smithi170.stdout:Sep 19 09:00:40 smithi170 systemd[1]: Stopping Ceph mds.k for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T09:00:41.375 INFO:journalctl@ceph.mds.k.smithi170.stdout:Sep 19 09:00:41 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-k[64104]: 2024-09-19T09:00:41.181+0000 7f4b1b918640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:00:41.375 INFO:journalctl@ceph.mds.k.smithi170.stdout:Sep 19 09:00:41 smithi170 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-k[64104]: 2024-09-19T09:00:41.181+0000 7f4b1b918640 -1 mds.k *** got signal Terminated *** 2024-09-19T09:00:44.056 INFO:journalctl@ceph.mds.k.smithi170.stdout:Sep 19 09:00:43 smithi170 podman[72351]: 2024-09-19 09:00:43.742173378 +0000 UTC m=+2.650397801 container died 23dd6748303302218cada725d87c8efa82bf66e7c1a4a5203fef6e4ea66856e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-k, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-19T09:00:44.056 INFO:journalctl@ceph.mds.k.smithi170.stdout:Sep 19 09:00:44 smithi170 podman[72351]: 2024-09-19 09:00:44.056443169 +0000 UTC m=+2.964667665 container cleanup 23dd6748303302218cada725d87c8efa82bf66e7c1a4a5203fef6e4ea66856e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-k, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-09-19T09:00:44.375 INFO:journalctl@ceph.mds.k.smithi170.stdout:Sep 19 09:00:44 smithi170 bash[72351]: ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-k 2024-09-19T09:00:44.375 INFO:journalctl@ceph.mds.k.smithi170.stdout:Sep 19 09:00:44 smithi170 podman[72362]: 2024-09-19 09:00:44.282679675 +0000 UTC m=+0.533857154 container remove 23dd6748303302218cada725d87c8efa82bf66e7c1a4a5203fef6e4ea66856e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-k, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-19T09:00:44.420 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.k.service' 2024-09-19T09:00:44.451 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:00:44.452 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-19T09:00:44.452 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-19T09:00:44.452 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.c 2024-09-19T09:00:44.764 INFO:journalctl@ceph.mds.c.smithi173.stdout:Sep 19 09:00:44 smithi173 systemd[1]: Stopping Ceph mds.c for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T09:00:45.027 INFO:journalctl@ceph.mds.c.smithi173.stdout:Sep 19 09:00:44 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-c[65367]: 2024-09-19T09:00:44.762+0000 7ff7fd637640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:00:45.027 INFO:journalctl@ceph.mds.c.smithi173.stdout:Sep 19 09:00:44 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-c[65367]: 2024-09-19T09:00:44.762+0000 7ff7fd637640 -1 mds.c *** got signal Terminated *** 2024-09-19T09:00:48.039 INFO:journalctl@ceph.mds.c.smithi173.stdout:Sep 19 09:00:47 smithi173 podman[72393]: 2024-09-19 09:00:47.765483252 +0000 UTC m=+3.109234730 container died 8658563522b083f049fc01a5e198fb37d73d3cc933ecf6b79e3aee6184410a76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-c, GIT_CLEAN=True, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-19T09:00:48.388 INFO:journalctl@ceph.mds.c.smithi173.stdout:Sep 19 09:00:48 smithi173 podman[72393]: 2024-09-19 09:00:48.163369504 +0000 UTC m=+3.507120971 container cleanup 8658563522b083f049fc01a5e198fb37d73d3cc933ecf6b79e3aee6184410a76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-c, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-19T09:00:48.389 INFO:journalctl@ceph.mds.c.smithi173.stdout:Sep 19 09:00:48 smithi173 bash[72393]: ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-c 2024-09-19T09:00:48.525 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.c.service' 2024-09-19T09:00:48.562 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:00:48.562 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-19T09:00:48.562 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-19T09:00:48.562 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.f 2024-09-19T09:00:48.910 INFO:journalctl@ceph.mds.f.smithi173.stdout:Sep 19 09:00:48 smithi173 systemd[1]: Stopping Ceph mds.f for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T09:00:48.911 INFO:journalctl@ceph.mds.f.smithi173.stdout:Sep 19 09:00:48 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-f[65065]: 2024-09-19T09:00:48.866+0000 7f5ebf46d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:00:48.911 INFO:journalctl@ceph.mds.f.smithi173.stdout:Sep 19 09:00:48 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-f[65065]: 2024-09-19T09:00:48.866+0000 7f5ebf46d640 -1 mds.f *** got signal Terminated *** 2024-09-19T09:00:52.034 INFO:journalctl@ceph.mds.f.smithi173.stdout:Sep 19 09:00:51 smithi173 podman[72526]: 2024-09-19 09:00:51.767037722 +0000 UTC m=+2.987255656 container died d799c13ea99f151a589f00b2b87100d061c188e6a3c2276b9fc264d4d0d3b959 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-f, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:00:52.384 INFO:journalctl@ceph.mds.f.smithi173.stdout:Sep 19 09:00:52 smithi173 podman[72526]: 2024-09-19 09:00:52.158420769 +0000 UTC m=+3.378638691 container cleanup d799c13ea99f151a589f00b2b87100d061c188e6a3c2276b9fc264d4d0d3b959 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-f, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-09-19T09:00:52.384 INFO:journalctl@ceph.mds.f.smithi173.stdout:Sep 19 09:00:52 smithi173 bash[72526]: ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-f 2024-09-19T09:00:52.541 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.f.service' 2024-09-19T09:00:52.574 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:00:52.574 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-19T09:00:52.574 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-19T09:00:52.575 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.i 2024-09-19T09:00:52.889 INFO:journalctl@ceph.mds.i.smithi173.stdout:Sep 19 09:00:52 smithi173 systemd[1]: Stopping Ceph mds.i for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T09:00:53.160 INFO:journalctl@ceph.mds.i.smithi173.stdout:Sep 19 09:00:52 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-i[65667]: 2024-09-19T09:00:52.886+0000 7f21c8cd4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:00:53.161 INFO:journalctl@ceph.mds.i.smithi173.stdout:Sep 19 09:00:52 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-i[65667]: 2024-09-19T09:00:52.886+0000 7f21c8cd4640 -1 mds.i *** got signal Terminated *** 2024-09-19T09:00:56.053 INFO:journalctl@ceph.mds.i.smithi173.stdout:Sep 19 09:00:55 smithi173 podman[72648]: 2024-09-19 09:00:55.764631658 +0000 UTC m=+2.980982738 container died 06eaf68a19e0dc7df99e4cbae06eb7766ee33b21ae8fab0d665672c64f62bf96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-i, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-19T09:00:56.371 INFO:journalctl@ceph.mds.i.smithi173.stdout:Sep 19 09:00:56 smithi173 podman[72648]: 2024-09-19 09:00:56.179207878 +0000 UTC m=+3.395558950 container cleanup 06eaf68a19e0dc7df99e4cbae06eb7766ee33b21ae8fab0d665672c64f62bf96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-19T09:00:56.371 INFO:journalctl@ceph.mds.i.smithi173.stdout:Sep 19 09:00:56 smithi173 bash[72648]: ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-i 2024-09-19T09:00:56.500 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.i.service' 2024-09-19T09:00:56.532 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:00:56.532 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-19T09:00:56.532 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-19T09:00:56.532 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.l 2024-09-19T09:00:56.660 INFO:journalctl@ceph.mds.l.smithi173.stdout:Sep 19 09:00:56 smithi173 systemd[1]: Stopping Ceph mds.l for 1e4b0d74-7662-11ef-bceb-c7b262605968... 2024-09-19T09:00:57.160 INFO:journalctl@ceph.mds.l.smithi173.stdout:Sep 19 09:00:56 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-l[64762]: 2024-09-19T09:00:56.874+0000 7f34112c1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:00:57.161 INFO:journalctl@ceph.mds.l.smithi173.stdout:Sep 19 09:00:56 smithi173 ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-l[64762]: 2024-09-19T09:00:56.874+0000 7f34112c1640 -1 mds.l *** got signal Terminated *** 2024-09-19T09:01:00.049 INFO:journalctl@ceph.mds.l.smithi173.stdout:Sep 19 09:00:59 smithi173 podman[72766]: 2024-09-19 09:00:59.76231895 +0000 UTC m=+3.004617524 container died d7e3e84b85558175d090c64b097bc54d943bd8d7024cb496ba8ce8d34537b244 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-l, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-19T09:01:00.349 INFO:journalctl@ceph.mds.l.smithi173.stdout:Sep 19 09:01:00 smithi173 podman[72766]: 2024-09-19 09:01:00.124120695 +0000 UTC m=+3.366419269 container cleanup d7e3e84b85558175d090c64b097bc54d943bd8d7024cb496ba8ce8d34537b244 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-l, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-19T09:01:00.349 INFO:journalctl@ceph.mds.l.smithi173.stdout:Sep 19 09:01:00 smithi173 bash[72766]: ceph-1e4b0d74-7662-11ef-bceb-c7b262605968-mds-l 2024-09-19T09:01:00.486 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e4b0d74-7662-11ef-bceb-c7b262605968@mds.l.service' 2024-09-19T09:01:00.518 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:01:00.518 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-19T09:01:00.519 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T09:01:00.722 INFO:teuthology.orchestra.run.smithi164.stdout:Deleting cluster with fsid: 1e4b0d74-7662-11ef-bceb-c7b262605968 2024-09-19T09:01:03.240 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T09:01:03.431 INFO:teuthology.orchestra.run.smithi170.stdout:Deleting cluster with fsid: 1e4b0d74-7662-11ef-bceb-c7b262605968 2024-09-19T09:01:05.873 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T09:01:06.076 INFO:teuthology.orchestra.run.smithi173.stdout:Deleting cluster with fsid: 1e4b0d74-7662-11ef-bceb-c7b262605968 2024-09-19T09:01:08.429 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T09:01:08.463 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T09:01:08.496 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T09:01:08.522 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-19T09:01:08.525 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911859/remote/smithi164/crash 2024-09-19T09:01:08.526 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/crash -- . 2024-09-19T09:01:08.555 INFO:teuthology.orchestra.run.smithi164.stderr:tar: /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T09:01:08.555 INFO:teuthology.orchestra.run.smithi164.stderr:tar: Error is not recoverable: exiting now 2024-09-19T09:01:08.557 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911859/remote/smithi170/crash 2024-09-19T09:01:08.558 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/crash -- . 2024-09-19T09:01:08.594 INFO:teuthology.orchestra.run.smithi170.stderr:tar: /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T09:01:08.594 INFO:teuthology.orchestra.run.smithi170.stderr:tar: Error is not recoverable: exiting now 2024-09-19T09:01:08.596 DEBUG:teuthology.misc:Transferring archived files from smithi173:/var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911859/remote/smithi173/crash 2024-09-19T09:01:08.597 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/crash -- . 2024-09-19T09:01:08.635 INFO:teuthology.orchestra.run.smithi173.stderr:tar: /var/lib/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T09:01:08.635 INFO:teuthology.orchestra.run.smithi173.stderr:tar: Error is not recoverable: exiting now 2024-09-19T09:01:08.636 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-19T09:01:08.636 DEBUG:teuthology.orchestra.run.smithi164:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-19T09:01:08.696 INFO:tasks.cephadm:Compressing logs... 2024-09-19T09:01:08.696 DEBUG:teuthology.orchestra.run.smithi164:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T09:01:08.699 DEBUG:teuthology.orchestra.run.smithi170:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T09:01:08.701 DEBUG:teuthology.orchestra.run.smithi173:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T09:01:08.723 INFO:teuthology.orchestra.run.smithi164.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-19T09:01:08.723 INFO:teuthology.orchestra.run.smithi164.stderr:: No such file or directory 2024-09-19T09:01:08.723 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.9.log 2024-09-19T09:01:08.724 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose --/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.0.log: /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T09:01:08.724 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T09:01:08.724 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.a.log 2024-09-19T09:01:08.724 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T09:01:08.730 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-19T09:01:08.730 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.g.log 2024-09-19T09:01:08.730 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-19T09:01:08.730 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph.log 2024-09-19T09:01:08.730 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.d.log 2024-09-19T09:01:08.730 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-19T09:01:08.730 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.j.log 2024-09-19T09:01:08.730 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-19T09:01:08.730 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T09:01:08.731 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 86.9% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T09:01:08.732 INFO:teuthology.orchestra.run.smithi173.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T09:01:08.732 INFO:teuthology.orchestra.run.smithi173.stderr:: No such file or directory 2024-09-19T09:01:08.732 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T09:01:08.733 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.i.log 2024-09-19T09:01:08.733 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-volume.log: /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-19T09:01:08.733 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T09:01:08.733 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.11.log 2024-09-19T09:01:08.733 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mon.c.log: /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-19T09:01:08.733 INFO:teuthology.orchestra.run.smithi173.stderr: 82.3% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T09:01:08.733 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-19T09:01:08.733 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.2.log: --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph.log 2024-09-19T09:01:08.733 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.l.log 2024-09-19T09:01:08.733 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph.log: 90.5% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T09:01:08.733 INFO:teuthology.orchestra.run.smithi173.stderr: 86.9% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T09:01:08.733 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.8.log 2024-09-19T09:01:08.733 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mgr.z.log 2024-09-19T09:01:08.734 INFO:teuthology.orchestra.run.smithi164.stderr: 94.5% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.d.log.gz 2024-09-19T09:01:08.734 INFO:teuthology.orchestra.run.smithi164.stderr: 94.6% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.j.log.gz 2024-09-19T09:01:08.739 INFO:teuthology.orchestra.run.smithi170.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.10.log 2024-09-19T09:01:08.739 INFO:teuthology.orchestra.run.smithi170.stderr:: No such file or directory 2024-09-19T09:01:08.739 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-19T09:01:08.739 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-19T09:01:08.739 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T09:01:08.739 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.b.log 2024-09-19T09:01:08.739 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.k.log 2024-09-19T09:01:08.739 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-19T09:01:08.739 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T09:01:08.739 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T09:01:08.740 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-19T09:01:08.740 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph.log 2024-09-19T09:01:08.740 INFO:teuthology.orchestra.run.smithi170.stderr: 82.4% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T09:01:08.740 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.h.log 2024-09-19T09:01:08.740 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-19T09:01:08.740 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.e.log 2024-09-19T09:01:08.740 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T09:01:08.740 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.e.log: 90.5% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T09:01:08.740 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/cephadm.log: 86.9% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T09:01:08.741 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.8.log: /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.l.log: 90.9% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T09:01:08.741 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.c.log 2024-09-19T09:01:08.750 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mgr.z.log: 92.1% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mgr.z.log.gz 2024-09-19T09:01:08.751 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.f.log 2024-09-19T09:01:08.764 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T09:01:08.772 INFO:teuthology.orchestra.run.smithi164.stderr: 94.5% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.g.log.gz 2024-09-19T09:01:08.787 INFO:teuthology.orchestra.run.smithi170.stderr: 92.0% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-19T09:01:08.787 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T09:01:08.800 INFO:teuthology.orchestra.run.smithi164.stderr: 92.6% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.a.log.gz 2024-09-19T09:01:08.803 INFO:teuthology.orchestra.run.smithi164.stderr: 84.8% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T09:01:08.814 INFO:teuthology.orchestra.run.smithi164.stderr: 90.4% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T09:01:08.829 INFO:teuthology.orchestra.run.smithi173.stderr: 92.7% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.i.log.gz 2024-09-19T09:01:08.832 INFO:teuthology.orchestra.run.smithi173.stderr: 94.5% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.l.log.gz 2024-09-19T09:01:08.839 INFO:teuthology.orchestra.run.smithi170.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T09:01:08.856 INFO:teuthology.orchestra.run.smithi173.stderr: 92.5% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.c.log.gz 2024-09-19T09:01:08.866 INFO:teuthology.orchestra.run.smithi173.stderr: 94.3% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.f.log.gz 2024-09-19T09:01:08.870 INFO:teuthology.orchestra.run.smithi164.stderr: 94.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T09:01:08.880 INFO:teuthology.orchestra.run.smithi170.stderr: 90.9% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T09:01:08.882 INFO:teuthology.orchestra.run.smithi170.stderr: 94.5% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.k.log.gz 2024-09-19T09:01:08.942 INFO:teuthology.orchestra.run.smithi170.stderr: 92.7% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.h.log.gz 2024-09-19T09:01:09.045 INFO:teuthology.orchestra.run.smithi164.stderr: 90.9% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T09:01:09.157 INFO:teuthology.orchestra.run.smithi173.stderr: 91.7% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-19T09:01:09.178 INFO:teuthology.orchestra.run.smithi170.stderr: 91.7% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-19T09:01:09.330 INFO:teuthology.orchestra.run.smithi164.stderr: 90.3% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-19T09:01:10.665 INFO:teuthology.orchestra.run.smithi164.stderr: 90.3% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-19T09:01:15.840 INFO:teuthology.orchestra.run.smithi170.stderr: 97.5% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.e.log.gz 2024-09-19T09:01:36.473 INFO:teuthology.orchestra.run.smithi164.stderr: 93.1% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.9.log.gz 2024-09-19T09:01:39.301 INFO:teuthology.orchestra.run.smithi173.stderr: 93.0% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.8.log.gz 2024-09-19T09:01:43.590 INFO:teuthology.orchestra.run.smithi173.stderr: 93.0% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-19T09:01:43.865 INFO:teuthology.orchestra.run.smithi164.stderr: 93.0% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-19T09:01:44.953 INFO:teuthology.orchestra.run.smithi170.stderr: 93.0% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.10.log.gz 2024-09-19T09:01:45.767 INFO:teuthology.orchestra.run.smithi170.stderr: 92.9% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-19T09:01:47.407 INFO:teuthology.orchestra.run.smithi173.stderr: 93.0% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.11.log.gz 2024-09-19T09:01:48.691 INFO:teuthology.orchestra.run.smithi173.stderr: 93.0% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-19T09:01:48.693 INFO:teuthology.orchestra.run.smithi173.stderr: 2024-09-19T09:01:48.693 INFO:teuthology.orchestra.run.smithi173.stderr:real 0m39.977s 2024-09-19T09:01:48.693 INFO:teuthology.orchestra.run.smithi173.stderr:user 2m19.745s 2024-09-19T09:01:48.693 INFO:teuthology.orchestra.run.smithi173.stderr:sys 0m4.883s 2024-09-19T09:01:51.821 INFO:teuthology.orchestra.run.smithi164.stderr: 93.0% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-19T09:01:52.262 INFO:teuthology.orchestra.run.smithi164.stderr: 92.9% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-19T09:01:52.263 INFO:teuthology.orchestra.run.smithi164.stderr: 2024-09-19T09:01:52.264 INFO:teuthology.orchestra.run.smithi164.stderr:real 0m43.552s 2024-09-19T09:01:52.264 INFO:teuthology.orchestra.run.smithi164.stderr:user 2m11.795s 2024-09-19T09:01:52.264 INFO:teuthology.orchestra.run.smithi164.stderr:sys 0m4.675s 2024-09-19T09:01:58.180 INFO:teuthology.orchestra.run.smithi170.stderr: 93.1% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-19T09:01:59.507 INFO:teuthology.orchestra.run.smithi170.stderr: 93.0% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-19T09:02:04.704 INFO:teuthology.orchestra.run.smithi170.stderr: 94.5% -- replaced with /var/log/ceph/1e4b0d74-7662-11ef-bceb-c7b262605968/ceph-mds.b.log.gz 2024-09-19T09:02:04.705 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-09-19T09:02:04.705 INFO:teuthology.orchestra.run.smithi170.stderr:real 0m55.990s 2024-09-19T09:02:04.706 INFO:teuthology.orchestra.run.smithi170.stderr:user 3m48.973s 2024-09-19T09:02:04.706 INFO:teuthology.orchestra.run.smithi170.stderr:sys 0m7.471s 2024-09-19T09:02:04.707 INFO:tasks.cephadm:Archiving logs... 2024-09-19T09:02:04.707 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911859/remote/smithi164/log 2024-09-19T09:02:04.708 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T09:02:25.967 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911859/remote/smithi170/log 2024-09-19T09:02:25.968 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T09:02:57.170 DEBUG:teuthology.misc:Transferring archived files from smithi173:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911859/remote/smithi173/log 2024-09-19T09:02:57.172 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T09:03:19.722 INFO:tasks.cephadm:Removing cluster... 2024-09-19T09:03:19.722 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 --force 2024-09-19T09:03:19.926 INFO:teuthology.orchestra.run.smithi164.stdout:Deleting cluster with fsid: 1e4b0d74-7662-11ef-bceb-c7b262605968 2024-09-19T09:03:20.361 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 --force 2024-09-19T09:03:20.562 INFO:teuthology.orchestra.run.smithi170.stdout:Deleting cluster with fsid: 1e4b0d74-7662-11ef-bceb-c7b262605968 2024-09-19T09:03:21.076 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1e4b0d74-7662-11ef-bceb-c7b262605968 --force 2024-09-19T09:03:21.277 INFO:teuthology.orchestra.run.smithi173.stdout:Deleting cluster with fsid: 1e4b0d74-7662-11ef-bceb-c7b262605968 2024-09-19T09:03:21.799 INFO:tasks.cephadm:Removing cephadm ... 2024-09-19T09:03:21.799 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T09:03:21.818 DEBUG:teuthology.orchestra.run.smithi170:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T09:03:21.838 DEBUG:teuthology.orchestra.run.smithi173:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T09:03:21.855 INFO:tasks.cephadm:Teardown complete 2024-09-19T09:03:21.855 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-19T09:03:21.866 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-19T09:03:21.866 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T09:03:21.869 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T09:03:21.881 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T09:03:21.983 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-19T09:03:21.993 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-19T09:03:21.994 DEBUG:teuthology.orchestra.run.smithi164:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T09:03:21.996 DEBUG:teuthology.orchestra.run.smithi170:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T09:03:21.999 DEBUG:teuthology.orchestra.run.smithi173:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T09:03:22.011 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-09-19T09:03:22.013 INFO:teuthology.orchestra.run.smithi170.stderr:bash: line 1: ntpq: command not found 2024-09-19T09:03:22.014 INFO:teuthology.orchestra.run.smithi173.stderr:bash: line 1: ntpq: command not found 2024-09-19T09:03:22.015 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T09:03:22.016 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-09-19T09:03:22.016 INFO:teuthology.orchestra.run.smithi164.stdout:^* hv01.front.sepia.ceph.com 3 6 377 30 +1226us[+1230us] +/- 43ms 2024-09-19T09:03:22.016 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 29 -3657us[-3657us] +/- 68ms 2024-09-19T09:03:22.016 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 29 +1621us[+1621us] +/- 66ms 2024-09-19T09:03:22.016 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:03:22.017 INFO:teuthology.orchestra.run.smithi173.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T09:03:22.018 INFO:teuthology.orchestra.run.smithi173.stdout:=============================================================================== 2024-09-19T09:03:22.018 INFO:teuthology.orchestra.run.smithi173.stdout:^* hv01.front.sepia.ceph.com 3 6 377 28 +1188us[+1208us] +/- 43ms 2024-09-19T09:03:22.018 INFO:teuthology.orchestra.run.smithi173.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 29 -3687us[-3666us] +/- 68ms 2024-09-19T09:03:22.018 INFO:teuthology.orchestra.run.smithi173.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 30 +1598us[+1619us] +/- 66ms 2024-09-19T09:03:22.018 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:03:22.019 INFO:teuthology.orchestra.run.smithi170.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T09:03:22.020 INFO:teuthology.orchestra.run.smithi170.stdout:=============================================================================== 2024-09-19T09:03:22.020 INFO:teuthology.orchestra.run.smithi170.stdout:^* hv01.front.sepia.ceph.com 3 6 377 30 +1208us[+1162us] +/- 43ms 2024-09-19T09:03:22.020 INFO:teuthology.orchestra.run.smithi170.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 30 -3663us[-3663us] +/- 68ms 2024-09-19T09:03:22.020 INFO:teuthology.orchestra.run.smithi170.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 30 +1615us[+1615us] +/- 66ms 2024-09-19T09:03:22.020 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:03:22.020 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-19T09:03:22.031 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-19T09:03:22.031 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-19T09:03:22.210 DEBUG:teuthology.orchestra.run.smithi164:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T09:03:22.370 DEBUG:teuthology.orchestra.run.smithi170:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T09:03:22.474 DEBUG:teuthology.orchestra.run.smithi173:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T09:03:22.544 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T09:03:22.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:03:22.578 DEBUG:teuthology.orchestra.run.smithi170:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T09:03:22.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:03:22.608 DEBUG:teuthology.orchestra.run.smithi173:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T09:03:22.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:03:22.636 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-19T09:03:22.648 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-19T09:03:22.658 INFO:teuthology.task.internal:Duration was 2027.203912 seconds 2024-09-19T09:03:22.659 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-19T09:03:22.669 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-19T09:03:22.673 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T09:03:22.676 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T09:03:22.678 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T09:03:22.717 INFO:teuthology.orchestra.run.smithi173.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T09:03:22.722 INFO:teuthology.orchestra.run.smithi170.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T09:03:22.747 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T09:03:23.163 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-19T09:03:23.163 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi164.front.sepia.ceph.com 2024-09-19T09:03:23.164 DEBUG:teuthology.orchestra.run.smithi164:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T09:03:23.188 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi170.front.sepia.ceph.com 2024-09-19T09:03:23.188 DEBUG:teuthology.orchestra.run.smithi170:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T09:03:23.218 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi173.front.sepia.ceph.com 2024-09-19T09:03:23.218 DEBUG:teuthology.orchestra.run.smithi173:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T09:03:23.260 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-19T09:03:23.260 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T09:03:23.262 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T09:03:23.265 DEBUG:teuthology.orchestra.run.smithi173:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T09:03:23.327 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-19T09:03:23.327 DEBUG:teuthology.orchestra.run.smithi164:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T09:03:23.333 DEBUG:teuthology.orchestra.run.smithi170:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T09:03:23.335 DEBUG:teuthology.orchestra.run.smithi173:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T09:03:24.691 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-19T09:03:24.703 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-19T09:03:24.704 DEBUG:teuthology.orchestra.run.smithi164:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T09:03:24.729 DEBUG:teuthology.orchestra.run.smithi170:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T09:03:24.760 DEBUG:teuthology.orchestra.run.smithi173:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T09:03:24.789 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-19T09:03:24.801 DEBUG:teuthology.orchestra.run.smithi164:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T09:03:24.803 DEBUG:teuthology.orchestra.run.smithi170:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T09:03:24.806 DEBUG:teuthology.orchestra.run.smithi173:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T09:03:24.832 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = core 2024-09-19T09:03:24.836 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = core 2024-09-19T09:03:24.856 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern = core 2024-09-19T09:03:24.998 DEBUG:teuthology.orchestra.run.smithi164:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T09:03:25.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:03:25.013 DEBUG:teuthology.orchestra.run.smithi170:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T09:03:25.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:03:25.031 DEBUG:teuthology.orchestra.run.smithi173:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T09:03:25.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:03:25.054 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-19T09:03:25.065 INFO:teuthology.task.internal:Transferring archived files... 2024-09-19T09:03:25.067 DEBUG:teuthology.misc:Transferring archived files from smithi164:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911859/remote/smithi164 2024-09-19T09:03:25.067 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T09:03:25.140 DEBUG:teuthology.misc:Transferring archived files from smithi170:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911859/remote/smithi170 2024-09-19T09:03:25.141 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T09:03:25.182 DEBUG:teuthology.misc:Transferring archived files from smithi173:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911859/remote/smithi173 2024-09-19T09:03:25.183 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T09:03:25.221 INFO:teuthology.task.internal:Removing archive directory... 2024-09-19T09:03:25.221 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T09:03:25.224 DEBUG:teuthology.orchestra.run.smithi170:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T09:03:25.226 DEBUG:teuthology.orchestra.run.smithi173:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T09:03:25.272 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-19T09:03:25.285 INFO:teuthology.task.internal:Not uploading archives. 2024-09-19T09:03:25.285 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-19T09:03:25.297 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-19T09:03:25.297 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T09:03:25.300 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T09:03:25.302 DEBUG:teuthology.orchestra.run.smithi173:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T09:03:25.312 INFO:teuthology.orchestra.run.smithi164.stdout: 265786 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 09:03 /home/ubuntu/cephtest 2024-09-19T09:03:25.317 INFO:teuthology.orchestra.run.smithi170.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 09:03 /home/ubuntu/cephtest 2024-09-19T09:03:25.327 INFO:teuthology.orchestra.run.smithi173.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 09:03 /home/ubuntu/cephtest 2024-09-19T09:03:25.333 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-19T09:03:25.345 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-19T09:03:25.371 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/iogen}} duration: 2027.2039124965668 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-19T09:03:25.372 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T09:03:25.450 INFO:teuthology.run:pass